151 lines
46 KiB
Markdown
151 lines
46 KiB
Markdown
# NotesCraft architecture
|
|
|
|
Last updated: 2026-10-13 (after waves 1-4) · branch `develop` · see `SPEC.md` for the contract and `ROADMAP.md` for status.
|
|
|
|
## Shape
|
|
|
|
Rust workspace of one crate per concern (`crates/<name>`, package `notescraft-<name>`), layered so a crate may depend only on strictly lower layers (`cargo xtask layers`). Pure data and format crates sit at the bottom; the engine (command registry and `Session`) sits in the middle; egui lives only in L5 crates. Every user-visible action will be an engine command shared by the UI, `notescraft-cli`, the control channel, MCP and plugins (SPEC 10).
|
|
|
|
```
|
|
L0 vault markdown
|
|
L1 index canvas diagrams whiteboard
|
|
L2 backlinks graph templates
|
|
L3 plugins themes
|
|
L4 engine
|
|
L5 editor mcp i18n
|
|
L6 ui-egui (egui only in editor and ui-egui, and in apps)
|
|
testkit (test helpers) apps/notescraft apps/notescraft-cli xtask
|
|
```
|
|
|
|
## Crate status and public API
|
|
|
|
Status: **done** (implemented, tested, clippy-clean, merged), **partial**, **stub** (placeholder with one `TODO(phase N)`).
|
|
|
|
| Crate | Layer | Status | Src / test LOC | Tests |
|
|
|---|---|---|---|---|
|
|
| `vault` | L0 | done | 1016 / 587 | 42 |
|
|
| `markdown` | L0 | done | 2051 / 222 | 51 |
|
|
| `math` | L0 | done | 2157 / 699 | 54 |
|
|
| `index` | L1 | done (links, tags, BM25 search) | 2887 | 108 |
|
|
| `backlinks` | L2 | done | 686 | 14 |
|
|
| `canvas` | L1 | done | 1193 / 379 | 31 |
|
|
| `diagrams` | L1 | done: flowchart, sequence, class, state, ER, gantt, pie | 5226 | 102 |
|
|
| `whiteboard` | L1 | done | 1789 / 551 | 60 |
|
|
| `graph` | L2 | done | 915 / 302 | 33 |
|
|
| `templates` | L2 | done | 1102 / 287 | 40 |
|
|
| `plugins` | L3 | done | 1558 / (integration tests) | 58 |
|
|
| `themes` | L3 | done | 1539 / 319 | 15 |
|
|
| `engine` | L4 | done for v1 scope: registry, `Session`, buffers, search, daily notes, themes, settings, plugin host bridge, attachments, slash menu | ~6900 | 250+ |
|
|
| `i18n` | L5 | done | ~900 | 47 |
|
|
| `ui-egui` | L6 | partial: shell, editing, settings, canvas and whiteboard widgets (graph and diagram views pending) | ~32000 | 590 |
|
|
| `editor` | L5 | partial: highlighter, key behaviours, completion (wired into the app) | ~2000 | 55 |
|
|
| `mcp` | L5 | done | 2093 | 74 |
|
|
| `acp` | L0 | done | 1954 / 480 | 42 |
|
|
|
|
### `notescraft-vault` (L0)
|
|
Vault root, lock, path-traversal guard, atomic IO, watcher. `Vault::open(root)` / `open_read_only`, `resolve(rel)` (rejects `..`, absolute, drive/UNC, NUL, escaping symlinks, with `Error::OutsideVault`), `read`, `write` (via `craft-atomic`, returns `WriteOutcome::{Written, Unchanged}`), `rename`, `delete_to_trash` (into `.notescraft/trash/`), `create_dir`, `list` (sorted, hidden dirs skipped, depth 64), `watch()` (notify, 250 ms debounce, self-write suppression, `VaultEvent`), `write_private` for `.notescraft/`. `.obsidian/` and `.notescraft/` are `Error::Reserved` for normal writes. Lock `.notescraft/lock` with stale-pid takeover (`/proc`, `kill -0`, `tasklist`). Read cap `MAX_READ_BYTES` = 64 MiB.
|
|
|
|
### `notescraft-markdown` (L0)
|
|
Lossless Obsidian-flavoured parsing. `parse(&str) -> Document { source, frontmatter, items }`; `Document::write()` returns the source unchanged; edits go through `replace_range(Range, &str) -> Result<String>` (never clamps; `Error::InvalidRange` / `NotCharBoundary`) and `text(range)`. Items: `Heading`, `WikiLink`, `Embed`, `Link`, `Tag`, `BlockId`, `Highlight`, `Comment`, `Callout`, `Math`, `TaskItem`, with iterators `wikilinks()`, `headings()`, `tags()`, `embeds()`, `callouts()`, `block_ids()`, `all_tags()`. `CalloutKind` covers the 13 types plus aliases and `Unknown`. Frontmatter is a minimal YAML subset with `get`/`keys`, `edit_key` / `remove_key` that change only that key's lines, and the typed properties layer `frontmatter_edit` (SPEC 6 item 20): `set_key(src, key, &YamlValueText)`, `remove_key`, `rename_key` and `value_raw(src, key)` rewrite exactly one entry (append missing keys, create the block at byte 0 when absent, keep comments/quoting/CRLF byte-exact, edit and remove duplicate keys at the first occurrence, cap single-line entries at 1 MiB), and `PropValue` (`Text`/`Number`/`Bool`/`Date`/`DateTime`/`List`) with `render`/`infer`/`infer_list`, `PropType` with `infer_type` and `parse_types_json` for `.obsidian/types.json`. Helpers: `parse_link_target`, `anchor`, `anchors_match`, `decode_percent`, `is_internal_dest`. Nesting depth capped at 64.
|
|
|
|
### `notescraft-vim` (L0)
|
|
Pure modal-editing state machine (SPEC item 29), no egui and no IO. `Vim::handle(Key, &mut Doc) -> Outcome` runs Normal, Insert, Visual, VisualLine, Replace and CommandLine modes over a concrete `Doc { text, cursor, anchor }`; counts and pending keys cap at 100,000 / 4,096 keys, documents at 8 MiB, registers at 32 MiB, and every cursor update is UTF-8-bounded (`get`/`prev`/`next`, never indexing). Motions (`h j k l w b e W B E 0 ^ $ gg G { } % f F t T ; ,` plus viewport-relative `H M L` `Ctrl-d/u`, arrow keys, Home/End/PageUp/PageDown), operators (`d c y > < g~ gu gU` with doubled forms and text objects `iw aw iW aW ip ap i( i) i[ i{ i" i'` plus backtick forms, including `a` variants), single keys (`x X s S C D Y r R J ~ o O A I a i p P u Ctrl-r .` dot-repeat), registers (`"a-z`, unnamed, `"0`, `"-`, `"_`, `"+`/`"*` surfaced as `Outcome.clipboard_set`), marks, `n N *` smart-case literal search with `:noh` highlights and ex line (`:w :q :q! :wq :x :e <path> :bn :bp :s/// :%s/// :noh :set :<number>` and `word.word` ids as `ExCommand::Command`). Markdown helpers `gx gf za` emit `ExCommand::{FollowLink, OpenWikilink, Fold}`. Every edit is a coarse document `Edit` in the outcome, grouped by `UndoGroup`; invalid input returns a message and restores the document. 341 table-driven integration cases plus a 20,000-sequence xorshift fuzz test over a multi-byte document.
|
|
|
|
### `notescraft-math` (L0)
|
|
Pure TeX-math subset parser and type-setter (SPEC 30), no egui and no IO. `parse(src: &str) -> Result<Node>` covers fractions (`\frac \dfrac \tfrac`), roots (optional order), scripts (`^ _ '`, double-script errors), big operators with display limits (`\sum`-family takes them, `\int`/`\oint` keep scripts beside, matching TeX), relations/arrows/Greek/big-ops as Unicode code points (`symbols.rs`, ~140 commands), accents, font commands (`\mathrm \mathbf \mathbb \mathcal \mathscr \mathit`), `\text{}`, `\left..\right` (incl. `.` and `\{`), spacing (`\, \: \; \! \quad \qquad`) and environments (`matrix pmatrix bmatrix vmatrix cases aligned`) with `&`/`\\` cells. Caps: 20 000 chars in, nesting depth 64; errors (`Unknown`, `Unbalanced`, `MissingRight`, `DoubleScript`, `EnvMismatch`, `DepthLimit`, `TooLong`) all carry byte positions and positions are recomputed from the opening tokens, never from the scanner index. `layout(node, &dyn Metrics, Style { size_pt, display }) -> MathBox` turns nodes into `Item::Glyphs`/`Item::Rule`/`Item::Path` (surd hook, stretched parens/bracks/braces/angles as sampled polylines) in points relative to the box baseline, driven by a `Metrics` trait (glyph width, ascent, descent, x-height, rule thickness) so layout is testable without a window; TeX-like rules (Appendix G simplified): 0.7/0.5 script ladder, fraction rule on the math axis with 2-rule clearances, inter-atom spacing table (7 classes, thin/med/thick), italic correction on script bases, environments centred on the axis with 0.3 em gaps. `serde::Serialize` on `MathBox` makes the draw list ready for the `math.render` MCP surface. 54 tests: 29 unit modules, 12 layout-geometry integration cases over a 107-formula corpus, 11 error-position cases and a deterministic 20 000-input fuzz (xorshift over a TeX-ish token alphabet plus mutations of valid formulas) asserting every laid-out box is finite and never panics.
|
|
|
|
### `notescraft-index` (L1)
|
|
In-memory note, link and tag index fed `(path, content)` pairs (no vault dependency). `Index::update_note` / `add_file` / `remove_note` / `rename_note` (re-keys only, never rewrites other notes), queries `notes`, `note`, `headings`, `outgoing`, `backlinks` (source, byte range, context line), `unresolved`, `orphans`, `tags() -> TagTree` (nested counts), `notes_with_tag`, `all_link_edges` and `node_ids` (stable numbering by sorted path, for the graph crate). Link resolution follows SPEC 4.3: case-insensitive basename, nearest-folder ambiguity, path suffix and vault-absolute, alias below basename, `#Heading` / `#^block` subpath validation (`Resolution::Resolved { path, ambiguous, subpath_ok }`), markdown links and embeds. Links are stored unresolved and resolved at query time, so incremental updates need no invalidation. Measured in release: 20 000 notes indexed in 123 ms, worst `update_note` 16.6 us.
|
|
|
|
**Search** (`index::search`): `SearchIndex::{update, remove, rename, search, len}` is a separate in-memory inverted index with weighted fields (title x4, headings x2, tags x2, body x1), BM25 (k1 1.2, b 0.75), positions for phrases, deterministic ties. `SearchQuery::parse` supports terms, `"phrase"`, `-term`, `OR`, `path:`, `file:`, `tag:#a/b` (matches children) and `line:(a b)`; `/regex/` returns `Error::Unsupported` until a regex crate is approved. `Hit { path, score, title_match, snippets }`, at most 3 snippets with highlight ranges. Release numbers: 10 000 notes indexed in 208 ms, average search 3.15 ms. `SearchIndex` is not yet fed by `Session` (the engine owns that wiring).
|
|
|
|
### `notescraft-pdf` (L1)
|
|
PDF 1.7 export, pure: markdown source in, bytes out, no IO. `export(doc_source, &ExportOptions, &dyn AssetSource) -> Result<Vec<u8>>` parses with `notescraft-markdown`, segments the note into blocks (`scan.rs`), lays them out into flat slots with real font metrics (`layout/`), paginates with widow/orphan control, table rows kept whole and headings kept with the text that follows, then paints each page into a content stream. Rendering: H1-H6 at 1.8/1.5/1.3/1.15/1.05/1.0 x base, bold/italic/strikethrough/inline-code/`==highlight==` (yellow rect), links as blue underlined text plus real `/Link` annotations for `http(s)` (wikilinks print as text), bullet/numbered/task lists with nesting (checkboxes are vector squares, never font glyphs), block quotes (left bar), the 13 callout types (tinted box + bar + type name, fixed colour table), tables (bold header, content-driven column widths with a floor, cell wrapping, rows never split mid-row), fenced and indented code (monospace, grey background, no syntax colours, hard wrap), rules, images (PNG decoded and re-encoded as `FlateDecode` with an `SMask` for alpha, JPEG passed through as `DCTDecode`; anything else is a grey placeholder box with the file name), math as italic monospace TeX source (documented limitation, `notescraft-math` is not visible at L1), mermaid/whiteboard/canvas embeds as placeholders, footnotes as a numbered list at the end, and a footer with the title and `page x of y`. Theme-independent: black on white, always. Fonts are caller-supplied (`ExportOptions::fonts: FontSet{regular, bold, italic?, mono}`, `Arc<[u8]>`); a hand-written TrueType parser (`fonts/`) reads `cmap` format 4/12, `hmtx`, `head`, `hhea`, `maxp` and `OS/2` for advances and glyph ids, and programs embed whole as `CIDFontType2` + `Identity-H` with a `ToUnicode` `CMap`, so exported text stays selectable and searchable; no subsetting means a document carries the three whole programs (flated, about 1.1 MB), and a file over `MAX_FONT_BYTES` (2 MiB) is refused. A face without a real italic file is sheared 12 degrees in the text matrix; a missing glyph is the notdef box, never a panic. Caps, all `Error::TooLarge` and never a silent truncation: 2 MiB source, 2000 pages, 128 MiB of image bytes, 200 images, 3 levels of note embeds with cycle detection. Output is byte-for-byte deterministic (`CreationDate` comes from `ExportOptions::created`); `page_count(pdf)` reads the page count back out of the bytes. The writer (`writer/`) builds the objects, xref and trailer by hand and compresses every stream with flate2 level 6. Tests: 79 (unit plus `tests/render.rs` round-tripping ten fixtures through an independent content-stream extractor that reads the xref, inflates the `ToUnicode` `CMap`s and rebuilds the text, and `tests/hostile.rs` with thirty hostile documents); the CLI and engine add their own.
|
|
|
|
### `notescraft-canvas` (L1)
|
|
JSON Canvas 1.0: `Canvas { nodes, edges, extra }`, `Node` (`Text`, `File`, `Link`, `Group`, `Unknown`), `Edge`, `Color`, `Side`, `End`; tolerant `from_str`, canonical `to_string_pretty` (stable key order, unknown fields preserved), `validate() -> Vec<Diagnostic>`, `add_node` / `remove_node` / `add_edge`, `bounding_box`, `file_nodes`, `new_id(seed)` / `IdGen` (16 hex). Caps: 32 MiB, 100 000 nodes. Modules: `error`, `model`, `read`, `write`, `validate`, `id`.
|
|
|
|
### `notescraft-whiteboard` (L1)
|
|
Excalidraw v2 JSON: `Whiteboard { elements, app_state, files, extra }`, `Element` + `ElementKind` (rectangle, diamond, ellipse, arrow, line, freedraw, text, image, frame, unknown), unknown fields preserved, canonical `to_string_pretty`, builders and `IdGen`. `rough::outline(&Element, Style::{HandDrawn, Clean})` returns deterministic polylines (seeded xorshift; same seed gives identical output). Caps `MAX_ELEMENTS`, `MAX_INPUT_BYTES`. Files carry the `.draw` extension in the vault; `.excalidraw` is read and written back under its own name (see `apps/notescraft-cli/src/commands/notes.rs`).
|
|
|
|
### `notescraft-diagrams` (L1, partial)
|
|
Mermaid subset. `parse(&str) -> Result<Diagram>` with `Diagram::{Flowchart, Unsupported{kind, source}}`; flowchart supports `flowchart`/`graph` + direction, shapes `[] () {} (()) ([])`, edges `--> --- -.-> ==>` with labels (both syntaxes), chains, `&`, nested `subgraph` (depth 8); malformed lines become `Diagnostic`s; `source()` is always the input. `layout(&Diagram) -> Result<Layout>` is Sugiyama-lite (iterative cycle breaking, longest-path ranks, 4 barycenter sweeps, deterministic); `to_svg` / `to_svg_with(Palette)` produce escaped SVG. Caps: 1 MiB, 5000 nodes, 20 000 edges. Sequence diagrams are parsed and laid out too (participants/actors with aliases, all arrow kinds, notes, activation, autonumber, `loop`/`alt`/`opt`/`par`/`critical`/`break` blocks, caps 200 participants and 5000 events; 500 messages lay out in 78 us). Class diagrams (UML: members with visibility, stereotypes, generics, relations `<|-- *-- o-- --> -- ..> ..|>` with cardinalities and labels, caps 500 classes/5000 members; 300 classes lay out in 1.8 ms) and state diagrams (`[*]`, transitions, composites to depth 8, choice/fork; converted to the flowchart model) are parsed and laid out as well. ER, gantt and pie are recognised but not yet parsed.
|
|
|
|
### `notescraft-backlinks` (L2)
|
|
Sits one layer above `index` (it depends on it). `ContentSource` trait (no file IO; implemented for `BTreeMap<String, String>`), `linked_mentions` (path, byte range, line, snippet, matched text), `unlinked_mentions` (title or alias, case-insensitive, word boundaries; skips frontmatter, code, existing links, math, comments; caps 10 000 results and skips notes over 2 MiB, reported in `UnlinkedReport`), `link_mention` (replaces a mention with `[[Title]]` or `[[Title|text]]`, fails with `Error::Stale` if the text changed), `counts`. 20 000-note scan: 12 ms.
|
|
|
|
### `notescraft-graph` (L2)
|
|
`Graph::new(nodes, edges)`; deterministic Fruchterman-Reingold `layout(&Graph, &LayoutConfig) -> Vec<[f32;2]>` with a spatial grid, `Layout` for incremental `step`, pinning, `local_graph(depth)`, `degrees`, `components`, `orphans`. 5000 nodes / 15 000 edges / 300 iterations lays out in about 525 ms in release.
|
|
|
|
### `notescraft-templates` (L2)
|
|
No chrono: own `Date`/`DateTime`/`Weekday` (civil-from-days), `format()` (Moment token subset), `expand(template, &Context)` for `{{title}}`, `{{date[:FMT]}}`, `{{time[:FMT]}}`, `daily_note_path`, `parse_daily_name`, `month_grid` for the calendar widget.
|
|
|
|
### `notescraft-plugins` (L3)
|
|
Sandboxed WebAssembly host on `wasmi` (SPEC 7). `Manifest::from_json` (id charset, permissions `VaultRead`/`VaultWrite`/`Editor`, command declarations), `trait HostApi` (log, vault_read/write/list, search, links, backlinks, active_note, insert_text, notice, now; implemented by the app later), `Plugin::load(manifest, wasm, Arc<dyn HostApi>, Limits)` validates the module, requires the ABI v1 exports and memory, and links only the 11 `ns` imports (any other import is an error). Permissions are checked before the host trait is touched. `Limits` defaults: fuel 50 M, memory 64 MiB, 2 s, 4 MiB results, 8 host calls/ms; breaches are `Error::PluginLimit` / `PluginTrap`, and three consecutive failures disable the plugin. `PluginSet` lists commands as `plugin.<id>.<command>` and dispatches by that id. Guest pointers are bounds-checked with `get`. 15 `wat` fixtures cover hostile modules (loops, memory growth, bad pointers, oversize results, missing exports, wrong ABI, foreign imports, traps).
|
|
|
|
### `notescraft-engine` (L4, partial: registry + session)
|
|
Generic command registry `Registry<C>` (the context `C` becomes `Session` later). `CommandSpec<C>` has id, label, menu path, shortcut, typed `ParamSpec`s, `enabled` and `run` as plain function pointers. `register` validates ids (`[a-z0-9._-]`, 1..=128, no empty segments, no duplicates, cap 10 000); `register_plugin_command` forces the `plugin.<id>.` prefix and `unregister_prefix` removes a plugin's commands. `list()` returns serialisable `CommandInfo` sorted by id (for CLI `commands` and MCP `list_commands`). `run` checks, in order: unknown id, size cap (4 MiB, iterative walk), object-or-null, enabled, required/typed/unknown params, then calls the command inside `catch_unwind` so a panic becomes `Error::Panicked(id)` and the registry stays usable. `panic_hunt::adversarial_params()` plus a test runs 12 sample commands against 22 hostile inputs. `Session::open(root)` opens the vault and indexes every note (unreadable files become `diagnostics()`, never an abort); `open_read_only`. Buffers: `Buffer::edit` with the markdown range rules, undo/redo (1000 steps, 50 MiB), dirty tracking, `open_buffer`/`close_buffer(force)`/`save_buffer` (through `Vault::write`, then re-index). Note operations: `read_note`, `write_note`, `create_note`, `delete_note` (to trash), `rename_note(from, to, update_links)` which rewrites inbound links (keeps alias and `#heading`/`#^block`, keeps bare vs path form) all-or-nothing (`Error::RenameIncomplete`). `apply_event(VaultEvent)` re-indexes external changes and records a conflict instead when the buffer is dirty. `register_core` registers 16 commands: `vault.info`, `note.{list,read,write,create,delete,rename}`, `links.{outgoing,backlinks,unresolved}`, `tags.list`, `buffer.{open,edit,save,undo,redo}`; write commands are disabled on read-only sessions. Wiring slice: `SessionConfig` (serde, validated paths: daily folder/format/template, templates folder, attachment folder, update-links-on-rename, theme, week start) with `Session::open_with`; the `SearchIndex` is kept in sync with every write, rename, delete, save and external event (`search` command, default limit 20, max 200); daily notes from a template (`daily.open`, `calendar.month`, `template.list`, `template.create_note`, `template.insert` through the buffer so undo works); themes (`theme.list`, `theme.get`, `theme.set`, unknown id falls back to `dark` with a diagnostic). 25 command ids in total. Settings: `AppSettings` implements `craft_settings::Settings` (app id `notescraft`, schema 1): the `SessionConfig`, an opaque UI blob (cap 256 KiB), recent vaults (max 20), assistant settings (agent, command, review/write/delete flags, per-plugin tool toggles), enabled plugins and hotkeys, with `validate()` repairs; `load`/`save` go through the craft-settings store (atomic, backup, recovery; `NOTESCRAFT_CONFIG_DIR` override for tests); commands `settings.get`, `settings.set`, `settings.reset`. `Index::remove_file` exists now. Properties (SPEC 6 item 20): `note.properties` lists `{key, type, value, raw}` rows in file order plus `has_frontmatter`, importing Obsidian's `.obsidian/types.json` (read-only, junk ignored); `note.property_set` validates the value against the type (explicit, from types.json, or inferred), then `note.property_remove` and `note.property_rename` rewrite exactly one frontmatter entry through `notescraft-markdown`; a write goes through the open buffer as one undoable dirty edit (undo keeps the editor's own edits, disk updates on save) or through `write_note` with re-indexing, and invalid values or a read-only session refuse before anything is written. Gaps: plugins and the control channel are not wired into `Session` yet.
|
|
|
|
**Attachments (SPEC 9):** `attachments.rs` saves pasted or dropped bytes with `Session::save_attachment(hint, ext, bytes)` (`save_attachment_at` takes an injected clock, `save_attachment_from_path` copies an outside file, size-checked by metadata first). The type must be one of 15 allowed extensions (`unsupported_attachment_type` otherwise); a magic-byte sniff (png, jpeg, gif, webp, pdf) fixes a mismatched declared extension, an `.svg` must look like an SVG document, and anything over the `files.max_attachment_mib` cap (default 64 MiB, validated and repaired like every session setting) is refused with `attachment_too_large` before any byte is written. The hint is sanitised (last path component, no control characters, leading dots dropped, 120-byte stem, extension forced), an unnamed paste becomes `Pasted image YYYYMMDDHHMMSS.<ext>`, and collisions get ` 1`, ` 2`, … suffixes, never an overwrite. The folder is created through the vault guard (reserved dirs refused), the file is written through `Vault::write` (craft-atomic) and registered with `Index::add_file`, and the returned `SavedAttachment { path, link }` carries the `![[name]]` embed (bare name when unique in the vault index, vault-relative path otherwise). Commands `attachment.save` (a base64 payload through the strict hand-written decoder in `base64.rs`, 96 MiB decoded cap, correct padding required, ASCII whitespace skipped) and `attachment.save_path`; both writable-gated, and `attachment.save` is the MCP alias `save_attachment`.
|
|
|
|
**Slash menu (SPEC 21):** `slash.rs` holds the menu data: `SlashEntry { id, label, keywords, icon, insert }`, `filter_entries` (case-insensitive prefix beats word-start beats in-order subsequence, ties in menu order) and `trigger_ok(line, col)` (the `/` opens the menu at line start or after a space or tab, never inside inline code, math, an open `[[..]]` or `(..)` link target, or a URL; 41-case table). `slash_menu.rs` is the fixed entry table; `slash_commands.rs` registers the commands, each an ordinary command on menu `["Insert", <label>]` with the flag and keywords, so the command palette lists it too. The insert commands are pure and return `{ text, cursor, selection_end? }`: headings, bullet/numbered/task, quote, code and divider, the link/embed triggers, the 13 callouts (`slash.callout` with a `kind` param plus one `slash.callout.<kind>` per type, caret on the selected `Title` placeholder), `slash.table` (bounded rows and columns), `slash.date` (today in the configured daily format, clock injected), `slash.template` (expanded through the templates crate) and `slash.canvas` (creates `Untitled*.canvas`, writable-gated). `slash.list` serves the filtered entries (empty inside a code fence) and `slash.trigger_ok` the rule; plugin manifest commands cannot be flagged yet because the manifest has no field for it.
|
|
Generic command registry `Registry<C>` (the context `C` becomes `Session` later). `CommandSpec<C>` has id, label, menu path, shortcut, typed `ParamSpec`s, `enabled` and `run` as plain function pointers. `register` validates ids (`[a-z0-9._-]`, 1..=128, no empty segments, no duplicates, cap 10 000); `register_plugin_command` forces the `plugin.<id>.` prefix and `unregister_prefix` removes a plugin's commands. `list()` returns serialisable `CommandInfo` sorted by id (for CLI `commands` and MCP `list_commands`). `run` checks, in order: unknown id, size cap (4 MiB, iterative walk), object-or-null, enabled, required/typed/unknown params, then calls the command inside `catch_unwind` so a panic becomes `Error::Panicked(id)` and the registry stays usable. `panic_hunt::adversarial_params()` plus a test runs 12 sample commands against 22 hostile inputs. `Session::open(root)` opens the vault and indexes every note (unreadable files become `diagnostics()`, never an abort); `open_read_only`. Buffers: `Buffer::edit` with the markdown range rules, undo/redo (1000 steps, 50 MiB), dirty tracking, `open_buffer`/`close_buffer(force)`/`save_buffer` (through `Vault::write`, then re-index). Note operations: `read_note`, `write_note`, `create_note`, `delete_note` (to trash), `rename_note(from, to, update_links)` which rewrites inbound links (keeps alias and `#heading`/`#^block`, keeps bare vs path form) all-or-nothing (`Error::RenameIncomplete`). `apply_event(VaultEvent)` re-indexes external changes and records a conflict instead when the buffer is dirty. `register_core` registers 16 commands: `vault.info`, `note.{list,read,write,create,delete,rename}`, `links.{outgoing,backlinks,unresolved}`, `tags.list`, `buffer.{open,edit,save,undo,redo}`; write commands are disabled on read-only sessions. Wiring slice: `SessionConfig` (serde, validated paths: daily folder/format/template, templates folder, attachment folder, update-links-on-rename, theme, week start) with `Session::open_with`; the `SearchIndex` is kept in sync with every write, rename, delete, save and external event (`search` command, default limit 20, max 200); daily notes from a template (`daily.open`, `calendar.month`, `template.list`, `template.create_note`, `template.insert` through the buffer so undo works); themes (`theme.list`, `theme.get`, `theme.set`, unknown id falls back to `dark` with a diagnostic). 25 command ids in total. Settings: `AppSettings` implements `craft_settings::Settings` (app id `notescraft`, schema 1): the `SessionConfig`, an opaque UI blob (cap 256 KiB), recent vaults (max 20), assistant settings (agent, command, review/write/delete flags, per-plugin tool toggles), enabled plugins and hotkeys, with `validate()` repairs; `load`/`save` go through the craft-settings store (atomic, backup, recovery; `NOTESCRAFT_CONFIG_DIR` override for tests); commands `settings.get`, `settings.set`, `settings.reset`. `Index::remove_file` exists now. Gaps: plugin tools are wired; the control channel is described below.
|
|
**Control channel (SPEC 10, D9):** `notescraft-engine::control` turns a live app session into a small local JSON server, std-only (no async, no new dependency). `ControlServer::start(vault)` binds `127.0.0.1:0` (loopback only), writes `.notescraft/control.json` `{port, token, pid, started}` atomically through the vault's `write_private`, and spawns one listener thread plus one thread per connection (max 8, 1 MiB lines, 30 s idle by default via `ControlOptions`). The wire is newline-delimited JSON: one request line `{"id":N,"token":"…","method":"run"|"commands"|"info"|"ping","params":{…}}`, one reply `{"id":N,"ok":…}`; wrong tokens get a constant-time compare, a 1 s delay and the connection closed after three failures; the server thread never touches the session, it queues `(request, reply)` for `ControlHandle::pump(&mut Session, &Registry)`, which the UI calls once per frame (`NotesApp::control_pump`) and which dispatches through the same `Registry::run` path as UI and MCP under a budget of 16 requests / 4 ms, so a flood (1000 pipelined requests is a test) only costs sliced frames and repaints. Dropping the handle shuts the server down and removes the descriptor; a stale one (dead pid) is overwritten by `start` and refused by `ControlClient::connect(vault)`, the CLI side (`notescraft-cli control ping`, and write subcommands defer to the app first, falling back to local execution when it is offline). Token bytes come from `/dev/urandom` with a documented `RandomState`-mix fallback; the file is owner-only.
|
|
|
|
### `notescraft-ui-egui` (L5, partial) and `apps/notescraft`
|
|
Pure egui (no eframe in the library). Shell: `style::apply` maps theme tokens to egui visuals (no strokes, 8-step type scale), `layout` paints the `canvas` colour and three floating cards (sidebar 260, centre fluid, right 300; right panel hidden below 900 pt, sidebar below 640 pt; 16 pt radius, 8 pt gaps), `tree` (folders first, collapse state, 32 pt rows), `tabs` (craft-ui fitting, cap 32), `statusbar` (words, chars, ~tokens), `state` (serde, corrupt JSON gives defaults), `app::NotesApp` over a `Session`. Views: `note_view` (read-only markdown rendering: headings, lists, tasks, code, highlights, tags, wikilinks coloured resolved/unresolved, frontmatter card, the 13 callouts, 2 MiB cap), `palette` (fuzzy command palette over the engine registry), `backlinks_panel`, `outline`. `apps/notescraft` opens `notescraft <vault>` (or `NOTESCRAFT_VAULT`) with eframe on the **glow** renderer (PhotoCraft uses wgpu; switch now that disk allows). Visual pass (PaperCraft design): Inter and JetBrains Mono fonts (`assets/fonts`, OFL), Lucide icons (`assets/icons`, ISC) through craft-ui's SVG catalogue (`fonts`, `icons`), a sidebar with vault header, search field, tree with folder/file icons, calendar card and tags (`sidebar`), pill tabs with file icons, breadcrumb with chevrons and the Visual/Markdown pill toggle, a properties card with icon keys and tag chips, callout cards with kind icons and a 4 pt accent bar, mention cards, pill tabs in the right panel. `cargo run -p notescraft-ui-egui --example snapshot -- --vault V --out ui.png --size 1440x900 --open "note.md"` renders the UI offscreen through egui_kittest (wgpu) so the result can be looked at without a window. Motion and sound: `motion` (easing curves, `progress`/`value`/`appear` over egui's animation manager, one switch `UiState::animations` makes everything jump) drives the tree (rows fade in when a folder opens, chevron rotates, hover fades), the active-tab and right-tab pills (they glide between tabs), the note body (fades in on switch) and the palette (pop with overshoot). `sounds` synthesises four short cues (`Pop`, `Tick`, `Close`, `Open`: pitch-glide sine bursts, no sample files, click-free, under 0.2 s) and defines the `SoundSink` trait plus a per-cue `Throttle`; the library never opens an audio device. `apps/notescraft` implements the sink with `cpal` (`audio.rs`) and a tested voice `mixer.rs` (12 voices, soft clip); no device or an unsupported format means silence. More motion: a new tab grows in and a closed tab shrinks and fades before it is removed (`NotesApp::begin_close`, `Tabs::closing`, `TabsResponse::finished`; animations off closes at once); the opened note rises 16 pt as it fades in; hover fades with a pointing-hand cursor on the search field, settings icon, calendar days, the Visual/Markdown segments, the right-panel tabs, tab and tab-close targets and mention cards (`motion::hover`); the selected tree row fades in. Bounce: `motion::spring` (a damped spring, about 8% overshoot, settles in ~0.5 s) drives the sliding tab and right-panel pills; `motion::wobble` (a decaying sine, 0.5 s) makes a clicked tree row swell and its content swing sideways; `ease_out_back` pops new tabs, the folder chevron and the rows that drop in when a folder opens; `ease_in_back` makes a closing tab swell a little before it shrinks away. Toggles: `UiState::sound`, `volume`, `animations`, commands `ui.toggle_sound` and `ui.toggle_animations`. `cpal` is a new dependency here (SoundCraft, FilmCraft and EffectCraft already use 0.16). Whiteboard view (`whiteboard_view`, L6): `show_whiteboard(ui, &Tokens, &mut WhiteboardView, &mut Whiteboard) -> WhiteboardResponse { changed, selected_ids, export_requested, cues }`. Modules by concern: `state` (view flags, toolbar/drag state, undo snapshots, cue queue with one cue per kind per frame), `geometry` (zoom, pan, document/screen mapping), `edit` (drafts, commit, delete, eraser, keyboard tools, undo/redo, `Cmd+Shift+L` opens the layers panel), `render` (dot grid, seeded hand-drawn strokes, per-element appear scale 0.85 to 1.0 about the centre, fading selection ring), `panels` + `panels_rows` (floating properties card: stroke width, opacity, fill, line, roundness, roughness, font size; rows share one segmented-button widget that paints from the row rect only, since a second allocation makes the panel `Area` ratchet over the canvas), `palette_panel` + `palette_apply` (Stroke/Fill/Canvas popover: quick picks, 13x5 hue grid, recents, custom colour), `zoom_bar` (zoom in/out/fit, grid and layers toggles), `layers_panel` (`show_layers_panel(ctx, &Tokens, &mut WhiteboardView, &mut Whiteboard, Rect) -> bool`, reorder, hide/show, lock, returns true when the panel asked to close). Panels and popovers animate in once per open (`motion::appear`, 8 pt rise plus opacity); every interactive target paints a 2 pt accent focus ring when focused and sets `WidgetInfo` for AccessKit. Sound is queued, never played: the widget returns `Cue`s that `CenterResponse::whiteboard_cues` forwards to the app sink. Tests: 5 new suites (`whiteboard_cues`, `whiteboard_motion`, `whiteboard_zoom_bar`, `whiteboard_layers`, `whiteboard_a11y`). Export goes through the engine (`file.export_whiteboard_svg`); the shell asks before overwriting and toasts the result. Not yet: the editor, settings panel, graph/canvas/diagram renderers, assistant panel, Windows/Linux custom title bar, craft-quit and craft-settings integration, wiring the palette/backlinks/outline/note view into `NotesApp`.
|
|
|
|
Whiteboard selection/property fixes: IDs allocated after reopening are checked against existing element and group IDs, including deleted elements; bulk duplication builds the occupied-ID set once. Picking uses visible shape geometry with a six-screen-pixel tolerance, exclusive topmost clicks, Shift/Cmd toggling, a three-pixel click/drag threshold and intersection marquees. `whiteboard_view::controls::relevant_controls(tool: WhiteboardTool, selected_kinds: &[&ElementKind]) -> ControlSet` is the pure applicability contract shared by contextual panels and typed property mutations; mixed selections expose the union of supported controls and edits affect only supporting, selected, unlocked elements. Fill popovers anchor to the clicked control, flip/shift within the viewport, consume dismissal gestures and provide transparent, palette and explicit custom-hex picks; each pick is one undo step. New-element defaults now reach all supported fields, and fill paths follow their shape transforms. Fifty regression tests cover pointer selection, reopened IDs, property field isolation/defaults/undo, tool/kind applicability, popup placement/dismissal and contextual visibility.
|
|
|
|
### `notescraft-acp` (L0)
|
|
Client side of the Agent Client Protocol for the assistant panel (SPEC 10.3): std + `thiserror`/`serde`/`serde_json`, no async runtime and no new dependency. `AgentProcess::spawn(AgentCommand { command_line, cwd, env })` splits the command line with the crate's own POSIX shell-words parser (`split_command_line`: quotes, backslash escapes, empty/unterminated quote is an `Error::CommandLine`, never a shell), starts the agent with the vault root as working directory and piped stdio, and starts three threads: a stdout reader that parses newline-delimited JSON, a stderr collector keeping the last 200 lines (`stderr_tail()`), and a pump feeding `events() -> &Receiver<Event>`; `Drop` kills and reaps the child, `close()` is the graceful path (stdin closed, 2 s grace, then kill, then join). Messages: `initialize` (protocolVersion 1, clientInfo `NotesCraft`, no fs and no terminal capabilities), `session/new` (cwd plus the `McpServerConfig` list handed to the agent), `session/prompt` (`PromptBlock::Text` / `ResourceLink`), the `session/cancel` notification and `answer_permission(&RequestId, Option<&str>)` (`None` = the user cancelled); `initialize_blocking(timeout)` / `new_session_blocking(cwd, servers, timeout)` wait for the answer (`DEFAULT_HANDSHAKE_TIMEOUT` = 20 s) through a request tracker that matches response ids to the pending request kind. Incoming lines become `Event::{Update(SessionUpdate), PermissionRequest, PromptDone, Error, Exited}`; unknown update shapes are `SessionUpdate::Other(Value)`, never errors, and agent requests we do not implement (`fs/*`, `terminal/*`) get a `-32601 method not found` reply. Limits: 4 MiB per stdout line (longer lines are dropped with a diagnostic, the stream continues), 10 000 buffered events (backpressure drops the oldest `Update` first and never a permission request), completed-outcome table capped at 128. Malformed lines, unknown response ids and broken pipes become `Event::Error` or `Err`, never a panic. 42 tests: 29 unit (30-case shell-words table plus 10 hostile command lines, reader/queue/tracker/parser units and three deterministic fuzz tests) plus 13 integration tests against the `examples/stub_agent.rs` stub (exact event order, permission allow and cancel round trip, `session/cancel`, garbage lines, a 5 MiB line, `-32601` for `fs/read_text_file`, handshake timeout, mid-prompt crash, stderr tail, kill on drop).
|
|
**Editor tools (`ui-egui`, SPEC 6 items 9, 20, 21):** `properties_view::show_properties` emits typed set/remove/rename and raw frontmatter actions with bounded syntax checks; per-note drafts, imported Obsidian types, calendar controls, chips and a capped expansion map feed the shared command/buffer dispatch after painting. Raw drafts apply on blur and retain invalid input; validation covers the frontmatter subset rather than full YAML grammar. `slash_popup` validates caret triggers through engine commands, filters and navigates raised command rows, and applies returned text/cursor/selection as one undoable buffer edit, cancelling pending choices when the caret leaves the token. `attachment_drop` collects local image-path paste and native/virtual file drops, dispatches attachment commands, inserts one batch of embed links, and preloads bounded image bytes for Visual embeds/lightboxes without disk IO during painting. Disabled image pastes are consumed with a hint, and image width hints preserve aspect ratio. Binary clipboard images need a direct workspace `arboard` dependency; virtual payloads remain subject to the engine registry's parameter cap. Snapshot flags `--editable`, `--properties-raw` and `--slash` expose these states for visual checks.
|
|
|
|
### `notescraft-mcp` (L5)
|
|
Hand-rolled JSON-RPC 2.0 MCP server over `Registry<Session>` (no SDK). `Server::handle_line` maps one stdio line to at most one reply; `serve` is the read-answer-flush loop. `initialize` (protocol version negotiated from a supported list), `ping`, `tools/list`, `tools/call`, `resources/list` (paged, `note://<path>`), `resources/read`, batches up to 32, lines capped at 8 MiB, errors -32700/-32600/-32601/-32602, never a panic. Every engine command is a tool (id with `.` replaced by `_`, schema built from the `ParamSpec`s) plus the SPEC 10 aliases (`read_note`, `write_note`, `search`, `get_backlinks`, `daily_note`, `run_command`, ...). Tool failures are `isError: true` results. Safety: write commands need a read-write session (`--write`), delete needs `--allow-delete`; paths go through the engine's vault guard.
|
|
|
|
### `notescraft-cli` (app)
|
|
`notescraft-cli [--vault P] [--json] <command>`: `info`, `search`, `read`, `write --yes`, `append`, `links`, `backlinks`, `tags`, `daily`, `template list|new|insert`, `export pdf <path> [--out file] [--page a4|letter] [--margin-mm N] [--print]`, `commands`, `run <id> [json]`, `doctor`, `mcp`, `help`. Text by default, `--json` gives `{ok, result, error}`; exit codes 0 ok / 1 command error / 2 usage / 3 locked vault or unsupported; read commands open the session read-only so a running app is not disturbed. Hand-written argument parser (no clap). 57 tests, including the real binary.
|
|
|
|
### Additions: views, settings, plugins bridge
|
|
**Editing (`ui-egui`):** the centre card edits in Source mode (`Cmd/Ctrl+E`, the pill reads SOURCE) with the `editor` crate: every change reaches the session buffer as a minimal diff (`editing::diff_edit`) so undo records, dirty tabs show a dot, `Cmd/Ctrl+S` and a 2 s autosave write through the buffer, `.notescraft/recovery/<hash>.md` mirrors unsaved text and is applied on reopen, autocomplete (titles, aliases, headings, block ids, tags, callouts) pops under the caret, the backlinks `Link` button rewrites the other note, and closing with dirty buffers asks Save all / Discard / Cancel through `craft-quit`. **Settings view:** category nav, search, animated controls, debounced change list applied through `settings.set`. **Canvas view:** pan/zoom, select, move, resize, edges, undo, 2 000-node draw cap, JSON preserved. **Plugins in the engine:** `plugins.rs` loads `.notescraft/plugins/<id>/`, honours enable and permission approvals from settings, and runs plugin commands on a worker thread with a `SessionBridge` that serves host calls on the calling thread (no unsafe); commands `plugins.list/enable/disable` and `plugin.<id>.<command>`. **CLI:** `notescraft-cli mcp [--write] [--allow-delete]` serves MCP over stdio.
|
|
|
|
### Git and sidebar tabs
|
|
**`notescraft-git` (L0):** the `git` command line only (no shell, `--` separators, validated paths and revisions, `GIT_TERMINAL_PROMPT=0`, 30 s timeout, output caps): `status` (porcelain v2), `diff` (hunks with line numbers), `stage`, `unstage`, `restore`, `commit` (message on stdin), `log`, `show`, `restore_version`, `init` (writes a `.gitignore` for `.notescraft/`). **Engine:** `Session::git()`, ten `git.*` commands, `Session::refresh_paths` re-reads files rewritten behind the session (a git restore); dirty buffers keep their text and record a conflict. **UI:** the sidebar has five icon tabs: Explorer, Search (ranked results with snippets), Tags (tag tree to Search filter), Buffers (open notes, dirty dots, Save all) and Source (git: Staged and Changes lists with stage/unstage/discard, commit box, history with restore, coloured diff viewer in the centre card, optional auto-snapshot via `git.auto_commit_minutes`); git runs on one worker thread, never on the UI thread; the UI refreshes sessions after a restore through `take_restored` + `refresh_paths`.
|
|
|
|
### `notescraft-i18n` (L5)
|
|
`Catalog` (flat dotted keys, TOML with nested tables flattened, caps 2 MiB / 20 000 keys), `I18n` with the embedded English catalog (`locales/en.toml`, 60+ keys), `t`, `t_args` (single-pass `{name}` replacement, no re-expansion), `plural` (`zero`/`one`/`other`), missing-key recording, `coverage` reports (missing, extra, mismatched placeholders) and locale id validation. The UI crates still hold their strings in constants blocks and move to this crate later.
|
|
|
|
### `notescraft-themes` (L3)
|
|
`Theme { colors, callouts, graph, radii, fonts }`, `Rgba` + WCAG `contrast_ratio`, `Theme::from_toml` with per-key diagnostics and fallback to the built-in of the same variant, `to_toml`, eight built-ins (`dark`, `light`, `catppuccin-{mocha,latte,frappe,macchiato}`, `solarized-{dark,light}`) verified for contrast in tests. Some Latte colours are darkened from stock Catppuccin to pass 4.5:1 (documented in `builtin.rs`).
|
|
|
|
### Not yet implemented
|
|
`engine` parts: plugin wiring, `editor`, `ui-egui`, `mcp`, `i18n`, apps. Their stubs carry `TODO(phase N)`.
|
|
|
|
## Key decisions
|
|
|
|
1. Native egui editor on `ropey` (no CodeMirror/JS). 2. `notescraft-*` crate names; `craft-*` reserved for craft-libs. 3. WASM plugins via `wasmi`. 4. Hand-rolled MCP over `serde_json`. 5. Hand-rolled in-memory BM25 index, not Tantivy. 6. Hand-written force layout. 7. Own minimal YAML-subset frontmatter parser; unknown YAML preserved. 8. No wasm/web build in v1. 9. Everything is an engine command. 10. Spec in root `SPEC.md`. Details and reasons: SPEC section 1.
|
|
|
|
Lossless-by-construction rule: parsers keep the original source and byte-range items; writers edit only the targeted bytes, so files stay readable and byte-stable for Obsidian.
|
|
|
|
## Quality gates
|
|
|
|
`cargo xtask ci` = fmt check + clippy `-D warnings` (pedantic) + tests + layer check. Perf numbers recorded in tests marked `#[ignore]` (run with `--release -- --ignored`).
|
|
|
|
In the Visual view (`ui-egui/src/views/note_view/edit.rs`) a click on a rendered row or block opens its markdown in place as a text field (Enter continues lists, Up/Down move between blocks, Backspace joins, `Esc` leaves, Cmd+B/I/E and Cmd+Shift+H/X format); edits are reported as source replacements (`NoteEdit`) and applied through the same buffer path as the Source view. Edits made in a block have the text field's own undo only.
|
|
|
|
`cargo xtask i18n` is the SPEC §12 coverage check for user strings: it scans `crates/ui-egui/src/**/*.rs` and `apps/*/src/**/*.rs` (test files, `#[cfg(test)]` modules and example directories are skipped), lexes every file with a small hand-written lexer (comments, raw strings, char literals and escapes handled), and reports string literals passed to egui text APIs (`.label(`, `RichText::new(`, `.button(`, `Button::new(`, `.on_hover_text(`, `.heading(`, `Label::new(`, `selectable_label`, `hint_text(`, painter `.text(`) that are not already covered by a `// i18n-ok` marker line or the ignore rules (empty/symbol-only, format placeholders, identifiers/ids/path-like, `Id::new`). Literals inside `mod strings` blocks are counted separately as "pending migration"; none remain, UI copy lives in `crates/ui-egui/src/text/*.rs` (functions that look up `ui.<panel>.<name>` in `crates/i18n/locales/en.toml` via `notescraft_i18n::english()`). The command is non-failing by default (per-file counts, first 5 offenders, totals; `--json` for tooling) and only `cargo xtask i18n --strict` fails when any non-table literal is found; it is part of `ci`.
|
|
|
|
## Build and workflow notes
|
|
|
|
Use rustup stable (1.99 here); the Homebrew cargo is too old for `rust-version = 1.95`.
|
|
|
|
## Source layout
|
|
|
|
Rule (AGENTS.md, `cargo xtask tree`): no source directory holds more than 12 loose `.rs` files; groups are sub-folders, nested when they grow, and public paths stay stable through `pub use` in `lib.rs`.
|
|
|
|
- `ui-egui/src`: `shell/` (app, layout, panel resize, title bar, status bar, state, Vim navigation; `shell/app/` splits into `editor/`, `vault/`, `window/`), `views/` (note, canvas, diagram, graph, folder, math, diff, properties, settings, whiteboard with `scene/ shape/ chrome/ layers/ palette/`), `panels/` (search, source control, backlinks, bookmarks, buffers, tags, outline, tree, sidebar, right card; `assistant_panel/` with `input/ backend/ chat/ state/`), `widgets/` (dialog, context menu, palette, slash popup, tabs, export dialog, autocomplete), `design/` (icons, Lottie icons, fonts, theme, style, motion, sounds), `editing/`, `text/` (copy tables by area: `settings/ views/ panels/`).
|
|
- `engine/src`: `vault/` (session, notes, buffers, folders, bookmarks, attachments, search, properties, daily), `builtin/` (the registered commands), `ai/` (assistant gate, queue, chat history, connections), `prefs/` (settings, themes, config), `plugin_host/`, `slash_kit/`, `control/`.
|
|
- `acp/src`: `client/` (process, protocol, reader, queue), `openai/` (curl backend and SSE stream).
|
|
- `diagrams/src`: `parsers/`, `layout/`. `vim/src`: `motions/`, `edit/`, `input/`. `mcp/src`: `protocol/`, `calls/`. `whiteboard/src`: `doc/`, `render/`. `lottie/src`: `decode/`, `eval/`. `math/src`: `syntax/`, `layout/`. `index/src`: `reading/`, `lookup/`, `search/`. `markdown/src`: `scanning/`, `frontmatter/`. `backlinks/src`: `mentions/`. `plugins/src`: `sandbox/`.
|