notescraft/docs/ARCHITECTURE.md

NotesCraft architecture

Last updated: 2026-10-13 (after waves 1-4) · branch develop · see SPEC.md for the contract and ROADMAP.md for status.

Shape

Rust workspace of one crate per concern (crates/<name>, package notescraft-<name>), layered so a crate may depend only on strictly lower layers (cargo xtask layers). Pure data and format crates sit at the bottom; the engine (command registry and Session) sits in the middle; egui lives only in L5 crates. Every user-visible action will be an engine command shared by the UI, notescraft-cli, the control channel, MCP and plugins (SPEC 10).

L0  vault  markdown
L1  index  canvas  diagrams  whiteboard
L2  backlinks  graph  templates
L3  plugins  themes
L4  engine
L5  editor  mcp  i18n
L6  ui-egui                             (egui only in editor and ui-egui, and in apps)
    testkit (test helpers)   apps/notescraft  apps/notescraft-cli   xtask

Crate status and public API

Status: done (implemented, tested, clippy-clean, merged), partial, stub (placeholder with one TODO(phase N)).

Crate	Layer	Status	Src / test LOC	Tests
vault	L0	done	1016 / 587	42
markdown	L0	done	2051 / 222	51
index	L1	done (links, tags, BM25 search)	2887	108
backlinks	L2	done	686	14
canvas	L1	done	1193 / 379	31
diagrams	L1	done: flowchart, sequence, class, state, ER, gantt, pie	5226	102
whiteboard	L1	done	1789 / 551	60
graph	L2	done	915 / 302	33
templates	L2	done	1102 / 287	40
plugins	L3	done	1558 / (integration tests)	58
themes	L3	done	1539 / 319	15
engine	L4	partial: registry, Session, buffers, 28 commands, search, daily notes, themes, settings (plugin wiring pending)	4106	104
i18n	L5	done	~900	47
ui-egui	L6	partial: shell and read-only views (no editor yet)	~4500	~45
editor	L5	partial: highlighter, key behaviours, completion (wired into the app by wave 8, in progress)	~2000	55
mcp	L5	done	2093	74

notescraft-vault (L0)

Vault root, lock, path-traversal guard, atomic IO, watcher. Vault::open(root) / open_read_only, resolve(rel) (rejects .., absolute, drive/UNC, NUL, escaping symlinks, with Error::OutsideVault), read, write (via craft-atomic, returns WriteOutcome::{Written, Unchanged}), rename, delete_to_trash (into .notescraft/trash/), create_dir, list (sorted, hidden dirs skipped, depth 64), watch() (notify, 250 ms debounce, self-write suppression, VaultEvent), write_private for .notescraft/. .obsidian/ and .notescraft/ are Error::Reserved for normal writes. Lock .notescraft/lock with stale-pid takeover (/proc, kill -0, tasklist). Read cap MAX_READ_BYTES = 64 MiB.

notescraft-markdown (L0)

Lossless Obsidian-flavoured parsing. parse(&str) -> Document { source, frontmatter, items }; Document::write() returns the source unchanged; edits go through replace_range(Range, &str) -> Result<String> (never clamps; Error::InvalidRange / NotCharBoundary) and text(range). Items: Heading, WikiLink, Embed, Link, Tag, BlockId, Highlight, Comment, Callout, Math, TaskItem, with iterators wikilinks(), headings(), tags(), embeds(), callouts(), block_ids(), all_tags(). CalloutKind covers the 13 types plus aliases and Unknown. Frontmatter is a minimal YAML subset with get/keys, edit_key / remove_key that change only that key's lines. Helpers: parse_link_target, anchor, anchors_match, decode_percent, is_internal_dest. Nesting depth capped at 64.

notescraft-index (L1)

In-memory note, link and tag index fed (path, content) pairs (no vault dependency). Index::update_note / add_file / remove_note / rename_note (re-keys only, never rewrites other notes), queries notes, note, headings, outgoing, backlinks (source, byte range, context line), unresolved, orphans, tags() -> TagTree (nested counts), notes_with_tag, all_link_edges and node_ids (stable numbering by sorted path, for the graph crate). Link resolution follows SPEC 4.3: case-insensitive basename, nearest-folder ambiguity, path suffix and vault-absolute, alias below basename, #Heading / #^block subpath validation (Resolution::Resolved { path, ambiguous, subpath_ok }), markdown links and embeds. Links are stored unresolved and resolved at query time, so incremental updates need no invalidation. Measured in release: 20 000 notes indexed in 123 ms, worst update_note 16.6 us.

Search (index::search): SearchIndex::{update, remove, rename, search, len} is a separate in-memory inverted index with weighted fields (title x4, headings x2, tags x2, body x1), BM25 (k1 1.2, b 0.75), positions for phrases, deterministic ties. SearchQuery::parse supports terms, "phrase", -term, OR, path:, file:, tag:#a/b (matches children) and line:(a b); /regex/ returns Error::Unsupported until a regex crate is approved. Hit { path, score, title_match, snippets }, at most 3 snippets with highlight ranges. Release numbers: 10 000 notes indexed in 208 ms, average search 3.15 ms. SearchIndex is not yet fed by Session (the engine owns that wiring).

notescraft-canvas (L1)

JSON Canvas 1.0: Canvas { nodes, edges, extra }, Node (Text, File, Link, Group, Unknown), Edge, Color, Side, End; tolerant from_str, canonical to_string_pretty (stable key order, unknown fields preserved), validate() -> Vec<Diagnostic>, add_node / remove_node / add_edge, bounding_box, file_nodes, new_id(seed) / IdGen (16 hex). Caps: 32 MiB, 100 000 nodes. Modules: error, model, read, write, validate, id.

notescraft-whiteboard (L1)

Excalidraw v2 JSON: Whiteboard { elements, app_state, files, extra }, Element + ElementKind (rectangle, diamond, ellipse, arrow, line, freedraw, text, image, frame, unknown), unknown fields preserved, canonical to_string_pretty, builders and IdGen. rough::outline(&Element, Style::{HandDrawn, Clean}) returns deterministic polylines (seeded xorshift; same seed gives identical output). Caps MAX_ELEMENTS, MAX_INPUT_BYTES.

notescraft-diagrams (L1, partial)

Mermaid subset. parse(&str) -> Result<Diagram> with Diagram::{Flowchart, Unsupported{kind, source}}; flowchart supports flowchart/graph + direction, shapes [] () {} (()) ([]), edges --> --- -.-> ==> with labels (both syntaxes), chains, &, nested subgraph (depth 8); malformed lines become Diagnostics; source() is always the input. layout(&Diagram) -> Result<Layout> is Sugiyama-lite (iterative cycle breaking, longest-path ranks, 4 barycenter sweeps, deterministic); to_svg / to_svg_with(Palette) produce escaped SVG. Caps: 1 MiB, 5000 nodes, 20 000 edges. Sequence diagrams are parsed and laid out too (participants/actors with aliases, all arrow kinds, notes, activation, autonumber, loop/alt/opt/par/critical/break blocks, caps 200 participants and 5000 events; 500 messages lay out in 78 us). Class diagrams (UML: members with visibility, stereotypes, generics, relations <|-- *-- o-- --> -- ..> ..|> with cardinalities and labels, caps 500 classes/5000 members; 300 classes lay out in 1.8 ms) and state diagrams ([*], transitions, composites to depth 8, choice/fork; converted to the flowchart model) are parsed and laid out as well. ER, gantt and pie are recognised but not yet parsed.

notescraft-backlinks (L2)

Sits one layer above index (it depends on it). ContentSource trait (no file IO; implemented for BTreeMap<String, String>), linked_mentions (path, byte range, line, snippet, matched text), unlinked_mentions (title or alias, case-insensitive, word boundaries; skips frontmatter, code, existing links, math, comments; caps 10 000 results and skips notes over 2 MiB, reported in UnlinkedReport), link_mention (replaces a mention with [[Title]] or [[Title|text]], fails with Error::Stale if the text changed), counts. 20 000-note scan: 12 ms.

notescraft-graph (L2)

Graph::new(nodes, edges); deterministic Fruchterman-Reingold layout(&Graph, &LayoutConfig) -> Vec<[f32;2]> with a spatial grid, Layout for incremental step, pinning, local_graph(depth), degrees, components, orphans. 5000 nodes / 15 000 edges / 300 iterations lays out in about 525 ms in release.

notescraft-templates (L2)

No chrono: own Date/DateTime/Weekday (civil-from-days), format() (Moment token subset), expand(template, &Context) for {{title}}, {{date[:FMT]}}, {{time[:FMT]}}, daily_note_path, parse_daily_name, month_grid for the calendar widget.

notescraft-plugins (L3)

Sandboxed WebAssembly host on wasmi (SPEC 7). Manifest::from_json (id charset, permissions VaultRead/VaultWrite/Editor, command declarations), trait HostApi (log, vault_read/write/list, search, links, backlinks, active_note, insert_text, notice, now; implemented by the app later), Plugin::load(manifest, wasm, Arc<dyn HostApi>, Limits) validates the module, requires the ABI v1 exports and memory, and links only the 11 ns imports (any other import is an error). Permissions are checked before the host trait is touched. Limits defaults: fuel 50 M, memory 64 MiB, 2 s, 4 MiB results, 8 host calls/ms; breaches are Error::PluginLimit / PluginTrap, and three consecutive failures disable the plugin. PluginSet lists commands as plugin.<id>.<command> and dispatches by that id. Guest pointers are bounds-checked with get. 15 wat fixtures cover hostile modules (loops, memory growth, bad pointers, oversize results, missing exports, wrong ABI, foreign imports, traps).

notescraft-engine (L4, partial: registry + session)

Generic command registry Registry<C> (the context C becomes Session later). CommandSpec<C> has id, label, menu path, shortcut, typed ParamSpecs, enabled and run as plain function pointers. register validates ids ([a-z0-9._-], 1..=128, no empty segments, no duplicates, cap 10 000); register_plugin_command forces the plugin.<id>. prefix and unregister_prefix removes a plugin's commands. list() returns serialisable CommandInfo sorted by id (for CLI commands and MCP list_commands). run checks, in order: unknown id, size cap (4 MiB, iterative walk), object-or-null, enabled, required/typed/unknown params, then calls the command inside catch_unwind so a panic becomes Error::Panicked(id) and the registry stays usable. panic_hunt::adversarial_params() plus a test runs 12 sample commands against 22 hostile inputs. Session::open(root) opens the vault and indexes every note (unreadable files become diagnostics(), never an abort); open_read_only. Buffers: Buffer::edit with the markdown range rules, undo/redo (1000 steps, 50 MiB), dirty tracking, open_buffer/close_buffer(force)/save_buffer (through Vault::write, then re-index). Note operations: read_note, write_note, create_note, delete_note (to trash), rename_note(from, to, update_links) which rewrites inbound links (keeps alias and #heading/#^block, keeps bare vs path form) all-or-nothing (Error::RenameIncomplete). apply_event(VaultEvent) re-indexes external changes and records a conflict instead when the buffer is dirty. register_core registers 16 commands: vault.info, note.{list,read,write,create,delete,rename}, links.{outgoing,backlinks,unresolved}, tags.list, buffer.{open,edit,save,undo,redo}; write commands are disabled on read-only sessions. Wiring slice (wave 5): SessionConfig (serde, validated paths: daily folder/format/template, templates folder, attachment folder, update-links-on-rename, theme, week start) with Session::open_with; the SearchIndex is kept in sync with every write, rename, delete, save and external event (search command, default limit 20, max 200); daily notes from a template (daily.open, calendar.month, template.list, template.create_note, template.insert through the buffer so undo works); themes (theme.list, theme.get, theme.set, unknown id falls back to dark with a diagnostic). 25 command ids in total. Settings (wave 8): AppSettings implements craft_settings::Settings (app id notescraft, schema 1): the SessionConfig, an opaque UI blob (cap 256 KiB), recent vaults (max 20), assistant settings (agent, command, review/write/delete flags, per-plugin tool toggles), enabled plugins and hotkeys, with validate() repairs; load/save go through the craft-settings store (atomic, backup, recovery; NOTESCRAFT_CONFIG_DIR override for tests); commands settings.get, settings.set, settings.reset. Index::remove_file exists now. Gaps: plugins and the control channel are not wired into Session yet.

notescraft-ui-egui (L5, partial) and apps/notescraft

Pure egui (no eframe in the library). Shell: style::apply maps theme tokens to egui visuals (no strokes, 8-step type scale), layout paints the canvas colour and three floating cards (sidebar 260, centre fluid, right 300; right panel hidden below 900 pt, sidebar below 640 pt; 16 pt radius, 8 pt gaps), tree (folders first, collapse state, 32 pt rows), tabs (craft-ui fitting, cap 32), statusbar (words, chars, ~tokens), state (serde, corrupt JSON gives defaults), app::NotesApp over a Session. Views: note_view (read-only markdown rendering: headings, lists, tasks, code, highlights, tags, wikilinks coloured resolved/unresolved, frontmatter card, the 13 callouts, 2 MiB cap), palette (fuzzy command palette over the engine registry), backlinks_panel, outline. apps/notescraft opens notescraft <vault> (or NOTESCRAFT_VAULT) with eframe on the glow renderer (PhotoCraft uses wgpu; switch now that disk allows). Visual pass (PaperCraft design): Inter and JetBrains Mono fonts (assets/fonts, OFL), Lucide icons (assets/icons, ISC) through craft-ui's SVG catalogue (fonts, icons), a sidebar with vault header, search field, tree with folder/file icons, calendar card and tags (sidebar), pill tabs with file icons, breadcrumb with chevrons and the Visual/Markdown pill toggle, a properties card with icon keys and tag chips, callout cards with kind icons and a 4 pt accent bar, mention cards, pill tabs in the right panel. cargo run -p notescraft-ui-egui --example snapshot -- --vault V --out ui.png --size 1440x900 --open "note.md" renders the UI offscreen through egui_kittest (wgpu) so the result can be looked at without a window. Motion and sound: motion (easing curves, progress/value/appear over egui's animation manager, one switch UiState::animations makes everything jump) drives the tree (rows fade in when a folder opens, chevron rotates, hover fades), the active-tab and right-tab pills (they glide between tabs), the note body (fades in on switch) and the palette (pop with overshoot). sounds synthesises four short cues (Pop, Tick, Close, Open: pitch-glide sine bursts, no sample files, click-free, under 0.2 s) and defines the SoundSink trait plus a per-cue Throttle; the library never opens an audio device. apps/notescraft implements the sink with cpal (audio.rs) and a tested voice mixer.rs (12 voices, soft clip); no device or an unsupported format means silence. More motion: a new tab grows in and a closed tab shrinks and fades before it is removed (NotesApp::begin_close, Tabs::closing, TabsResponse::finished; animations off closes at once); the opened note rises 16 pt as it fades in; hover fades with a pointing-hand cursor on the search field, settings icon, calendar days, the Visual/Markdown segments, the right-panel tabs, tab and tab-close targets and mention cards (motion::hover); the selected tree row fades in. Bounce: motion::spring (a damped spring, about 8% overshoot, settles in ~0.5 s) drives the sliding tab and right-panel pills; motion::wobble (a decaying sine, 0.5 s) makes a clicked tree row swell and its content swing sideways; ease_out_back pops new tabs, the folder chevron and the rows that drop in when a folder opens; ease_in_back makes a closing tab swell a little before it shrinks away. Toggles: UiState::sound, volume, animations, commands ui.toggle_sound and ui.toggle_animations. cpal is a new dependency here (SoundCraft, FilmCraft and EffectCraft already use 0.16). Not yet: the editor, settings panel, graph/canvas/whiteboard/diagram renderers, assistant panel, Windows/Linux custom title bar, craft-quit and craft-settings integration, wiring the palette/backlinks/outline/note view into NotesApp.

notescraft-mcp (L5)

Hand-rolled JSON-RPC 2.0 MCP server over Registry<Session> (no SDK). Server::handle_line maps one stdio line to at most one reply; serve is the read-answer-flush loop. initialize (protocol version negotiated from a supported list), ping, tools/list, tools/call, resources/list (paged, note://<path>), resources/read, batches up to 32, lines capped at 8 MiB, errors -32700/-32600/-32601/-32602, never a panic. Every engine command is a tool (id with . replaced by _, schema built from the ParamSpecs) plus the SPEC 10 aliases (read_note, write_note, search, get_backlinks, daily_note, run_command, ...). Tool failures are isError: true results. Safety: write commands need a read-write session (--write), delete needs --allow-delete; paths go through the engine's vault guard.

notescraft-cli (app)

notescraft-cli [--vault P] [--json] <command>: info, search, read, write --yes, append, links, backlinks, tags, daily, template list|new|insert, commands, run <id> [json], doctor, mcp (stub until wave 9), help. Text by default, --json gives {ok, result, error}; exit codes 0 ok / 1 command error / 2 usage / 3 locked vault or unsupported; read commands open the session read-only so a running app is not disturbed. Hand-written argument parser (no clap). 57 tests, including the real binary.

notescraft-i18n (L5)

Catalog (flat dotted keys, TOML with nested tables flattened, caps 2 MiB / 20 000 keys), I18n with the embedded English catalog (locales/en.toml, 60+ keys), t, t_args (single-pass {name} replacement, no re-expansion), plural (zero/one/other), missing-key recording, coverage reports (missing, extra, mismatched placeholders) and locale id validation. The UI crates still hold their strings in constants blocks and move to this crate later.

notescraft-themes (L3)

Theme { colors, callouts, graph, radii, fonts }, Rgba + WCAG contrast_ratio, Theme::from_toml with per-key diagnostics and fallback to the built-in of the same variant, to_toml, eight built-ins (dark, light, catppuccin-{mocha,latte,frappe,macchiato}, solarized-{dark,light}) verified for contrast in tests. Some Latte colours are darkened from stock Catppuccin to pass 4.5:1 (documented in builtin.rs).

Not yet implemented

engine parts: plugin wiring, editor, ui-egui, mcp, i18n, apps. Their stubs carry TODO(phase N).

Key decisions

1. Native egui editor on ropey (no CodeMirror/JS). 2. notescraft-* crate names; craft-* reserved for craft-libs. 3. WASM plugins via wasmi. 4. Hand-rolled MCP over serde_json. 5. Hand-rolled in-memory BM25 index, not Tantivy. 6. Hand-written force layout. 7. Own minimal YAML-subset frontmatter parser; unknown YAML preserved. 8. No wasm/web build in v1. 9. Everything is an engine command. 10. Spec in root SPEC.md. Details and reasons: SPEC section 1.

Lossless-by-construction rule: parsers keep the original source and byte-range items; writers edit only the targeted bytes, so files stay readable and byte-stable for Obsidian.

Quality gates

cargo xtask ci = fmt check + clippy -D warnings (pedantic) + tests + layer check. Perf numbers recorded in tests marked #[ignore] (run with --release -- --ignored).

Build and workflow notes

Use rustup stable (1.99 here); the Homebrew cargo is too old for rust-version = 1.95. Parallel pi agents: see AGENTS.md (at most 2 MiniMax agents at once because of its rate limit; the Ollama Cloud provider glm-5.3-flash ran 3 at once without errors). Each agent's target/agent-* directory is about 600 MB: delete it when the agent is merged, the volume fills up quickly.