Marcos Paulo
9.0 KiB
Zotero/BibTeX citations (agy gap 12, roadmap 39)
Part of specs/plugins-parity/. Existing engine, Markdown/editor and native UI; no new crate or third-party dependency in the baseline.
Research (2026-10-21; agy claims checked)
- Pandoc documents
[@key], grouped citations, bibliographies and CSL processing. Syntax and bibliography formatting are separate concerns: recognizing a key does not implement CSL. https://pandoc.org/MANUAL.html#citations and https://www.pandoc.org/demo/example33/8.20-citation-syntax.html - Zotero documents BibTeX and CSL JSON exchange. Better BibTeX is a Zotero add-on; its “Keep updated” export writes a refreshed file. It is not NotesCraft-to-Zotero bidirectional sync. https://www.zotero.org/support/dev/data_formats and https://retorque.re/zotero-better-bibtex/exporting/auto/
- Zotero has a supported API, but reading its live SQLite database is not this contract. https://www.zotero.org/support/dev/web_api/v3/
- agy's Logseq-native and “Obsidian Citation Suite” comparison was not verified and is dropped. “1-click metadata sync” is narrowed to explicit refresh of a local export; cloud sync and arbitrary CSL formatting are not implied by a citation picker.
Problem / goal
Academic notes need stable citation keys, searchable bibliographic metadata, insertion without copying titles, and an export path that can produce real citations. Preserve normal Markdown [@key] source and users' bibliography files.
Non-goals
A Zotero database editor, cloud credentials, attachment imports, downloading PDFs/DOIs, native CSL implementation, full BibLaTeX/TeX execution, arbitrary embedded HTML, changing citation keys automatically.
Design decisions
D1 Sources and normalized model
Explicitly select one vault-relative .bib or CSL JSON source per note via frontmatter bibliography: references.bib (path relative to the note). Optional csl: styles/apa.csl is also vault-relative; no URLs, absolute paths, traversal or reserved paths. V1 supports one source; a list returns unsupported_bibliography, never silently chooses its first element. Configuration stays in existing note frontmatter; no private database required. Zotero users export manually or configure Better BibTeX auto-export to that file, then choose Refresh bibliography.
Normalized Entry { key, kind, title, authors, issued, doi, url }; creator names support literal institutional names and family/given components, dates retain supplied precision. Retain unknown source fields for round trips only if a future writer exists: v1 never rewrites source files. CSL JSON id is the citation key; duplicate keys reject the whole refresh. The pure model lives in crates/engine/src/citations/{mod,model}.rs; loading and command plumbing remain separate. No new crate, hence no Cargo/layer registration.
Input caps: 8 MiB source, 20,000 entries, 256 bytes/key, 64 KiB/field, nesting 32, 100 authors/entry. Keys start ASCII letter/digit/underscore and continue ASCII letters/digits/underscore/hyphen/colon/period; this intentionally documented subset is narrower than full Pandoc syntax. Reject invalid key, invalid UTF-8, duplicate, malformed record or over-limit input with a structured error. No truncation or guessed repairs.
BibTeX subset: @article, @book, @inproceedings, @misc records with balanced braced/quoted fields, numeric years, nested title braces, and-separated authors, % comments outside values. Unknown record kinds and fields may retain strings but must not be presented as supported semantic mappings. @string, @preamble, concatenation, crossref inheritance and unsupported TeX commands return unsupported_bibtex with location. A fixture containing an unsupported construct fails the refresh; offer CSL JSON/Pandoc conversion, never silently produce corrupted authors. Full BibTeX coverage is owner Q2, not an excuse to grow an ad hoc TeX interpreter.
D2 Commands and buffers
| id | params | result | writes |
|---|---|---|---|
citation.refresh |
{path} note path |
{source, count, generation} |
memory cache only |
citation.search |
{path, query, limit?} max 100, default 20 |
{items:[{key,title,authors,issued}], total, generation} |
no |
citation.resolve |
{path, keys} max 100 |
{items, missing} |
no |
citation.insert |
{path, at, key, expected_source} |
{path, inserted} |
buffer edit |
Refresh validates all records before swapping the previous snapshot; failure preserves snapshot, marks it stale and shows the diagnostic. Cache keyed by canonical guarded source path plus source bytes/version, bounded to 8 sources/64 MiB; explicit refresh invalidates it. Search stable order: exact key, key prefix, title/author substring, key lexical tie-break. No engine IO while painting or per citation per frame. Insert uses current open Buffer::edit, refuses stale expected_source, invalid UTF-8 offset/read-only/unknown key, inserts exactly [@key], and participates in undo. It must not save unrelated edits. Reads work in read-only sessions. Classify refresh/search/resolve as READ, insert as WRITE in assistant gate, and update CORE_IDS.
D3 UI and compatibility
Picker searches key/title/author/year and inserts via citation.insert; shortcut is assigned only after hotkey conflict check. Read preview recognizes standalone [@key] and grouped [@one; @two] outside code, frontmatter, math, links and comments; unsupported locator syntax remains literal source in v1 (Pandoc still handles it on export). Resolved citations show key plus metadata tooltip, missing keys show a localized unresolved indicator and keep their literal text. Never relabel unformatted keys as APA/CSL output. Reuse note-view inline scanner and text tables; no HTML renderer. Refresh button exposes stale diagnostics, including malformed auto-export files; source is never rewritten.
D4 CSL export through existing Pandoc adapter
crates/engine/src/pandoc/{commands,run}.rs already runs an optional installed executable, with argv, timeout, bounded stderr and vault output write. Extend its typed request with explicit citeproc, guarded bibliography and optional local CSL paths; use citation-capable Pandoc Markdown reader for this mode (existing gfm+wikilinks_title_after_pipe is not a citation guarantee). Restrict this to explicit export.pandoc opt-in citations:true; ordinary export remains unchanged. Send validated source snapshot and style bytes from controlled temporary inputs rather than allowing note metadata to introduce remote sources or arbitrary paths. Disable raw HTML in the native preview. Missing Pandoc/invalid style/unsupported source produces actionable failure and no final output. Do not install/bundle Pandoc or add a CSL crate. Library formatting is explicitly unavailable without this opt-in backend.
Acceptance criteria
- CSL JSON and supported
.bibfixtures resolve same keys/title/authors/year, including Unicode and institutional names; duplicates and unsupported BibTeX fail without replacing last good snapshot. - Picker insertion changes only intended buffer range, preserves surrounding UTF-8, undo restores bytes, read-only/stale source refuses insertion.
- Code, comments and literal emails remain unchanged; known and unknown citations preserve source on load/save and grouped keys resolve predictably.
- Refresh reads user export, never writes it; malformed mid-export file yields stale diagnostic and successful later refresh recovers.
- Stub Pandoc verifies argv contains explicit citation mode, guarded files and no shell invocation; timeout/missing tool/path traversal leaves no final export. Optional real-tool fixture produces bibliography from two cited works.
- Warm search over 20,000 entries stays under 50 ms in an ignored release perf fixture; no reads or calls per inline citation during paint.
- Localized picker, keyboard operation, missing-key tooltip and Mocha/Latte screenshots pass; ordinary notes/export regressions remain green.
Open owner questions
- Q1: approve optional installed Pandoc as CSL backend for citation exports? Proposal: reuse existing optional executable, no new bundled code. Native CSL preview remains deferred.
- Q2: is documented BibTeX subset plus full CSL JSON acceptable for v1, or approve a maintained Rust BibTeX/BibLaTeX parser (name/version/license and reason required before adding dependency)? No dependency currently authorized.
- Q3: should Zotero local-API integration follow export-file refresh later? Proposal: export files first; no credentials or network in v1.
Feature ideas (researched, ranked)
- Better BibTeX auto-export guidance and change detection after reliable refresh ships: https://retorque.re/zotero-better-bibtex/exporting/auto/
- Complete locator/prefix/suffix groups and author-in-text citations using Pandoc grammar: https://www.pandoc.org/demo/example33/8.20-citation-syntax.html
- Native CSL preview after approving a maintained processor and conformance fixtures: https://www.zotero.org/support/styles
- Read-only Zotero API source with explicit connection and stable key mapping: https://www.zotero.org/support/dev/web_api/v3/