notescraft/specs/plugins-parity/attachment-ocr/spec.md

Attachment OCR (agy gap 8, roadmap 37)

Research (2026-10-21; agy claims checked)

Joplin OCR confirms offline attachment search and desktop processing; this validates workflow, not NotesCraft's proposed .ocr.db storage or handwriting accuracy. Apple Vision VNRecognizeTextRequest is native text-recognition API; its dynamic docs did not yield details through this research tool, so do not infer language/accuracy guarantees. Windows OcrEngine documents supported-language discovery, image-size limit and word boxes. ocrs is real Rust OCR project with model requirements; agy's “pure Rust on Linux” is not justification for shipping new inference/model dependencies automatically. Bear/OneNote parity claims not needed and dropped.

Goal and decision

Offline image/scanned-PDF text extraction, attachment search results with page/box navigation, and explicit Copy detected text. Prefer OS Vision on macOS and Windows.Media.Ocr on Windows over bundled engine in v1: avoids model distribution and duplicating OS resources. Linux returns structured unsupported capability, no cloud or opportunistic executable download. This is selected architecture subject to owner approval of safe binding dependencies and supported packaging. No native adapter implementation until approval; contract/model/index tests can proceed with deterministic fake backend. No new crate: pure result types in existing index crate, orchestration/platform adapters in engine. Direct Objective-C/COM FFI often requires unsafe, forbidden here: do not weaken lint, add C/Swift helper or put unsafe in sibling crate. Approve vetted safe wrapper or stop adapter task and present concrete alternative. Rust-only constraint remains.

D1 Result and bounded worker contract

OcrDocument {version:1,path,source_sha256,backend,backend_version,languages,pages}; page {number,width,height,lines:[{text,rect,words:[{text,rect}]}]}. Rectangles normalized top-left after image orientation/PDF rotation, finite positive area within unit square; reading order returned by backend, source stays byte-identical. Never fabricate confidence when API has none; optional backend confidence only. Caps: 64 MiB source image, 40 million decoded pixels, existing PDF 200 MiB cap, <= 500 pages/job, <= 2 MiB text/page and 16 MiB/job, <= 100,000 words/page. Reject oversized input before recognition, no silent truncation/downscale. User may explicitly select PDF page range <=500; job reports that range so partial extraction cannot masquerade as complete. One worker per vault, bounded queue 32, cancellation on vault switch/removal, 30 seconds/page and 5 minutes/job. Each completion checks current source hash/job generation before publishing; stale result discarded explicitly. Job states queued/running/complete/cancelled/failed/unsupported, progress per page. No UI thread OCR. Platform image cap may be smaller: report supported bound, reject with actionable error. No network for recognition, no auto language/model download.

D2 Cache and indexing

.notescraft/cache/ocr/<source-sha256>/<identity-sha256>.json, schema 1, content hash + backend revision + requested language set in cache identity; atomic write_private. identity-sha256 hashes canonical backend ID/version plus sorted requested language set, so differing recognition configurations cannot collide. Cache derived and local-only, not sync default. Source hash is approved existing sha2 via viewer hashing pattern. Validate cache exactly like hostile backend data; corrupt cache never indexed, diagnostic and explicit regeneration action. No sqlite dependency/new DB. Cache budget 256 MiB, bounded in-memory result snapshot; eviction deletes only recognized OCR cache entries via explicit cache management API, never source files. Existing crates/index/src/search/doc.rs builds Markdown documents; do not pretend attachment text has Markdown line positions. Add separate typed attachment insertion/removal API and existing Hit (crates/index/src/search/rank.rs) attachment location (path, page, normalized rectangles, snippet) preserving note-hit JSON compatibility. Reuse tokenizer/ranker, never index OCR into Markdown/frontmatter or overwrite native text. Source removal/hash change invalidates OCR result and postings. Search image opens image view; PDF hit opens existing PDF tab at page and overlays result box. No live re-recognition per query. Native PDF text extraction, when available from PDF annotation backend, is preferred for textual pages; scanned pages OCR raster from existing pdf_preview path, resolution/capability explicit. If native extraction unavailable, OCR pages with provenance ocr, never describe as original text.

D3 Engine/UI

ocr.capabilities {} -> supported/backend/languages/limits; ocr.start {path,pages?,languages?} -> job ID; ocr.status {job} -> progress/error; ocr.text {path,page?} -> cached text/geometry; ocr.cancel {job}. Start/cancel write-gated because cache/job state changes; capability/status/text read-gated, locked encryption domains refuse all extraction and cached reads. Existing read-only vault can read valid cache, cannot start persistent job. Commands registered in engine, assistant gate and CORE_IDS. Persist top-level AppSettings.ocr (not SessionConfig) with enabled and background false by default; user enables background image indexing or runs explicit attachment action. Installed-language absence has recoverable error; empty result succeeds with no-text state. Strings through i18n, keyboard-accessible Copy detected text, search badge makes provenance clear. Attachment import schedules only after successful save/index, never on paint; no handwritten-text accuracy promise.

Acceptance criteria

1. Fake backend makes image text searchable once; hit points to attachment/page/rect, note results unchanged; source bytes unchanged.
2. Hash replacement/removal/cancel/locked domain removes stale postings and prevents late worker publication; read-only and invalid paths reject before writes.
3. Hostile dimensions, NaN/out-of-bounds box, invalid UTF-8 transport, corrupt cache, unknown schema and all caps fail safely; no silent truncation.
4. Native fixture suite runs offline on approved macOS/Windows targets with installed languages; unsupported OS/language/packaging reports capability, never cloud fallback. Actual supported builds must pass before marked complete.
5. PDF page selection, rotated image, no-text page, cancellation and cache reopen tested; OCR never modifies PDF or Markdown.
6. UI screenshots/kittest cover image/PDF hit navigation, Copy text and progress/error states; background work never restarts every frame.

Open owner questions

• Q1: Approve OS-first macOS/Windows v1 with Linux explicitly unsupported? Bundled ocrs alternative requires separate dependency/model/license/size approval.
• Q2: Approve concrete safe Rust bindings for Vision/WinRT after adapter spike, minimum OS and Windows packaging verification? If no safe interface works under forbidden unsafe/Rust-only rules, adapter blocked; do not relax either rule silently.
• Q3: Accept opt-in background indexing and local-only 256 MiB derived cache? Should OCR text eventually sync (privacy and backend-version policy needed)?
• Q4: Which languages and fixture accuracy targets are required for launch? Proposed printed English + Portuguese; no universal/handwriting guarantee.

Feature ideas (researched, ranked)

1. Accessible PDF export with hidden selectable text, separate feature inspired by Joplin OCR accessible documents.
2. Linux offline engine after explicit model/dependency approval, informed by ocrs.
3. Installed-language selector and word overlay based on Windows OcrEngine capabilities.