notescraft/specs/vault-knowledge/plan.md

Plan: Vault knowledge

Approach

Pure crate first, then engine commands, then MCP and assistant wiring, then the egui screens that the PaperCraft artboards define, then responsive rules and docs. Retrieval is hybrid lexical (spec Decision): passage BM25 + pseudo-relevance vocabulary expansion + link and tag boosts. Rejected: embeddings (model download, new crate, privacy cost) until the evaluation set proves a gap.

Delegation: each task has a fixed file set so parallel agents never collide. Judgment-heavy work (scoring, vocabulary, incremental updates) uses the main model tier; mechanical work (i18n keys, config plumbing, docs, tests) uses small models. Every UI task must compare its screenshot against the named artboard (papercraft-cli call get_screenshot on the artboard id vs cargo run -p notescraft-ui-egui --example snapshot) and fix differences before finishing. Artboard ids on page Knowledge (b246f3e9e517a782): Related b246f3:1, Assistant sources b246f3:211, Settings b246f3:404, Mobile b246f3:548. Before implementing a screen run papercraft-cli call get_design_spec '{"pageId":"b246f3e9e517a782","nodeId":"<id>","target":"egui"}' (exact tree, tokens, text strings) and after it verify_code with the changed Rust source (target egui) until it passes or every remaining violation is explained.

Architecture

• Data flow: note change -> Session -> knowledge::Index::{update,remove,rename} (passages) -> knowledge.search/context/related -> UI panels, MCP vault_context, assistant prompt builder.
• New: crates/knowledge (L2): chunk (headings then paragraphs, caps), tokens (tokenizer, stop words, light stemmer), index (passage postings, BM25), vocab (co-occurrence table, expansion), graph (link/tag boosts via a Signals trait implemented by the engine over notescraft-index), rank (fusion, signals list), eval helpers in tests.
• New: crates/engine/src/knowledge/{mod,commands,sync}.rs: commands and incremental wiring; crates/engine/src/prefs/knowledge_config.rs.
• New UI: crates/ui-egui/src/panels/related_panel.rs, panels/assistant_panel/chat/sources.rs, views/settings_view/categories/knowledge.rs, widgets/bottom_sheet.rs, style/compact.rs (width class).
• Modified: right card tabs (panels/right_view.rs), assistant send path (context injection), mcp/src/calls/{alias,tools,gate}.rs (alias vault_context), ai/assistant_gate.rs (core plugin knowledge), i18n/locales/en.toml, xtask/src/layers.rs.

Data

• Config SessionConfig::knowledge: { auto_context: true, show_sources: true, passages: 6, token_budget: 1200, follow_links: true, use_tags: true, learn_vocab: true, prefer_recent: false, excluded: [] }, reachable as session.knowledge.*.
• Command shapes: knowledge.search {query, limit?, folder?} -> {passages:[{path, heading, start, end, text, score, signals:[{kind, detail}]}], truncated}; knowledge.context {query, budget?} -> {passages, tokens}; knowledge.related {path, limit?} -> {notes:[{path, score, snippet, signals}]}; knowledge.status; knowledge.rebuild.

Dependencies

None.

Risks

• Recall too low without vectors -> the evaluation set gates release; the scorer trait leaves room for a dense scorer.
• Index memory on big vaults -> passage text is not stored, only byte ranges and postings; text is read from the vault on demand.
• UI drift from the design (the Calendar rework drifted) -> every UI task compares screenshots against the named artboard and records the comparison.

Out of Scope (Defer)

Embeddings, reranking models, attachments, other languages, answer generation.

Test Strategy

Unit per module; eval set for recall; panic hunt on hostile inputs; command parity test (engine = CLI = MCP); snapshot tests per screen at 1440 and 390 widths.