docs: specs, designs, research and agent rules for mobile, sync and collaboration
- specs: mobile-and-sync, collaboration, vault-launcher, e2ee, attachments-and-history, capture-and-publish, agents (brief and orchestrator notes for MiniMax and GLM agents) - research: sync landscape, security and server, mobile platform (browsed, cited) - design/mobile: 24 PaperCraft exports (dark and light) of the phone, desktop, sharing and launcher screens - .claude: project skills and path rules so delegated agents follow the repo rules; CLAUDE.md imports AGENTS.md - ARCHITECTURE and ROADMAP updated; AGENTS.md lists the approved sync dependencies Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@@ -0,0 +1,6 @@
|
||||
---
|
||||
paths:
|
||||
- "crates/engine/**"
|
||||
- "crates/vault/**"
|
||||
---
|
||||
Behaviour is an engine command (namespaced id, typed params, `enabled`, `run`) registered like its neighbours; new write/delete commands go into the assistant gate lists. Disk writes only via `craft_atomic::atomic_write` or `Vault::write_private` (`.notescraft/`); never write `.obsidian/`; user deletes go to the vault trash. No egui in engine. Skill: `notescraft-engine-command`.
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
paths:
|
||||
- "crates/**/*.rs"
|
||||
- "apps/**/*.rs"
|
||||
---
|
||||
Non-test Rust here must never panic. No `unwrap`, `expect`, `panic!`, `todo!`, `unimplemented!`, `unreachable!`, `unsafe`. No `v[i]` or `s[a..b]` with an input-derived value, even guarded: use `get()`, `split_at_checked`, `checked_*`, `saturating_*`, `try_from`. Bound loops, recursion and allocations. Invalid input returns `Err`; never clamp or repair silently. Every pub item has rustdoc, every module `//!`. Files under about 400 lines, at most 12 loose `.rs` per directory. Clippy pedantic is `-D warnings`. Details and the lint-fix table: skill `notescraft-rust-safety`.
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
paths:
|
||||
- "crates/sync/**"
|
||||
- "apps/notescraft-sync-server/**"
|
||||
- "crates/engine/src/sync/**"
|
||||
---
|
||||
Never lose data: doubt means a conflict copy; edit beats delete. Paths and hashes from the network pass `check_syncable` / `check_hash`, and downloaded bytes are hashed and compared. Tokens and pairing codes are stored only as SHA-256 hashes and never logged. Inject time as an argument in tests; never sleep minutes. Skill: `notescraft-sync-protocol`.
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
paths:
|
||||
- "**/tests/**/*.rs"
|
||||
---
|
||||
Test behaviour, not absence of panics. Write the failing test first. Run with `timeout 300 cargo test -p <crate>` and confirm the new tests are listed. Compute byte offsets with `str::find`. Unique temp dirs under `std::env::temp_dir()`. `unwrap`/`expect` are fine in tests; `assert!(x.is_empty())` is not (use `assert_eq!`). Skill: `notescraft-verify-loop`.
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
paths:
|
||||
- "crates/ui-egui/**"
|
||||
---
|
||||
No hex colours (theme tokens only), no literal user-facing text (i18n table: `crates/ui-egui/src/text/<panel>.rs` + `[ui.<panel>]` keys in `crates/i18n/locales/en.toml`; `cargo xtask i18n` must show 0). Named constants with unit docs. Stable widget ids. 44 pt touch targets in the compact layout (`design::style::compact`). Test by driving real events through frames and render offscreen to compare with `design/mobile/*.png`. Skill: `notescraft-egui-screen`.
|
||||
@@ -0,0 +1,32 @@
|
||||
---
|
||||
name: notescraft-agent-workflow
|
||||
description: How to run a NotesCraft coding task end to end (read, plan, small steps, scope fences, no git, final report). Use at the start of every task, especially when working as a delegated agent in the shared tree.
|
||||
---
|
||||
|
||||
# Agent workflow for NotesCraft
|
||||
|
||||
## 1. Orient (before any edit)
|
||||
1. Read the task's OWNS, READ FIRST and DO NOT lists. OWNS is the only place you may write.
|
||||
2. Read `AGENTS.md`. Skim `ROADMAP.md` for the phase you touch and `docs/ARCHITECTURE.md` for the crate's public API.
|
||||
3. Read two or three neighbouring files in the crate and copy their style: naming, comment density, error type, test layout. Never invent a pattern that already exists.
|
||||
4. Find the real function you will call (`grep -rn "pub fn name" crates/`). Never guess a signature; open the file.
|
||||
|
||||
## 2. Plan in slices
|
||||
- One slice = one file or one function plus its test, at most about 150 lines.
|
||||
- Write the failing test first, see it fail for the right reason, then the code, then run the crate's tests.
|
||||
- One file per tool call, at most about 250 lines per response. A response that is too long is cut off and you stall.
|
||||
|
||||
## 3. Scope fences
|
||||
- Edit only OWNS. If you need a change outside it, stop and say so in the report; do not make it.
|
||||
- Never run `git add`, `commit`, `checkout`, `stash`, `reset`, `clean`, or delete files you did not create. Never touch `target/debug`.
|
||||
- Never edit root `Cargo.toml`, `xtask/`, `SPEC.md`, `ROADMAP.md`, `docs/`, or another agent's files unless OWNS lists them.
|
||||
- New third-party dependency: not allowed unless the task names it. Approved list is in `AGENTS.md`.
|
||||
- Never print or log secrets, tokens or pairing codes.
|
||||
|
||||
## 4. Stuck rules
|
||||
- Same error three times: stop, paste the exact error in the report, do not loop.
|
||||
- A test hangs or memory climbs: `pkill -9 -f <test binary>`, fix the loop (each iteration must consume input) before anything else.
|
||||
- Unsure whether something is allowed: treat it as not allowed and report.
|
||||
|
||||
## 5. Finish
|
||||
Run the `notescraft-verify-loop` commands, paste the last `cargo test` summary line, and report: what is implemented, public signatures, test counts, files changed, anything skipped. Max 15 lines. Do not claim success on a command you did not run.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: notescraft-egui-screen
|
||||
description: How to build or change a NotesCraft egui screen: theme tokens, i18n tables, touch targets, layout classes, driven tests, design comparison. Use for any ui-egui work.
|
||||
---
|
||||
|
||||
# egui screens in NotesCraft
|
||||
|
||||
Crate `crates/ui-egui` (layer L6, the only crate that may use egui). egui 0.36, eframe wgpu. No JavaScript, no webview.
|
||||
|
||||
## Colours and sizes
|
||||
- Never write a hex colour. Use the active theme tokens: `crate::theme::{Tokens, color}`; look at a neighbouring view (`views/trash_view/mod.rs`) for how `tokens` is passed and how `color(...)` helpers are used.
|
||||
- Sizes are named constants with a doc comment and a unit (`/// Height of one row, points.\nconst ROW_HEIGHT: f32 = 48.0;`). Spacing on a 4 pt scale (4, 8, 12, 16, 24, 32, 48).
|
||||
- Panels are rounded floating cards on the canvas colour separated by tone, not borders.
|
||||
- Touch: width class from `crate::design::style::compact` (`is_compact(ctx)`, `target(ctx, regular)`, `COMPACT_BELOW` = 600 pt, `MIN_TARGET_COMPACT` = 44 pt). Phone layouts need 44 pt hit areas.
|
||||
|
||||
## Strings (hard rule: no literal UI text in Rust)
|
||||
1. Add keys to `crates/i18n/locales/en.toml` under `[ui.<panel>]` (kebab-case keys, `{name}` placeholders).
|
||||
2. Add `crates/ui-egui/src/text/<panel>.rs` (or a sub-folder file) with the table macro:
|
||||
```rust
|
||||
//! Copy for the trash tab.
|
||||
|
||||
table! {
|
||||
title = "ui.trash-view.title";
|
||||
restore = "ui.trash-view.restore";
|
||||
}
|
||||
```
|
||||
3. Declare it in `crates/ui-egui/src/text.rs` and use it as `use crate::text::trash_view as strings;` then `strings::title()`.
|
||||
4. `cargo xtask i18n` must report 0 hardcoded literals. Only `en.toml` exists today.
|
||||
|
||||
## Structure
|
||||
- A view is a folder `views/<name>/mod.rs` returning a small action enum; the app shell handles the action. No engine calls from drawing code except through the data the shell hands in.
|
||||
- Keep drawing functions under 100 lines; split into `paint.rs`, `layout.rs`.
|
||||
- Give every interactive widget a stable id (`ui.id().with(("name", salt))`); duplicate ids trip a debug assert in tests.
|
||||
- Scroll areas: `ScrollArea::vertical().id_salt("notescraft-<name>-scroll").auto_shrink([false, false])`.
|
||||
- Animations must be skippable: `state.animations = false` in tests.
|
||||
|
||||
## Verify
|
||||
Driven test in `crates/ui-egui/tests/` (copy the `Driver` helper from `driven_features.rs`), plus an offscreen render compared with the design PNG. Designs: `design/mobile/*.png` and the PaperCraft project NotesCraft (page "Mobile and sync"); see `specs/mobile-and-sync/design.md`.
|
||||
@@ -0,0 +1,22 @@
|
||||
---
|
||||
name: notescraft-engine-command
|
||||
description: How behaviour is added to the NotesCraft engine as commands, and the vault I/O rules (atomic writes, reserved folders). Use when touching crates/engine or crates/vault.
|
||||
---
|
||||
|
||||
# Engine commands and vault I/O
|
||||
|
||||
## Everything is a command
|
||||
User-visible behaviour is an engine command (id, label, typed params, `enabled`, `run`) so the UI, CLI, control channel, MCP and plugins share one surface. Ids are `[a-z0-9._-]`, 1..=128 chars, namespaced (`note.write`, `trash.restore`, `sync.now`). `register` validates ids; write commands are disabled on read-only sessions. Look at `crates/engine/src/registry.rs` and a small existing command group (for example `crates/engine/src/vault/trash.rs` and its registration in `crates/engine/src/builtin/`) and copy its shape exactly. The assistant gate classifies commands (`crates/engine/src/ai/assistant_gate.rs`): a new write or delete command must be added to the matching list, unknown commands default to write.
|
||||
|
||||
## Vault rules
|
||||
- Read and write notes through `notescraft_vault::Vault` (`read`, `write`, `write_private`, `delete_to_trash`, `rename`, `list`). They validate paths and refuse to leave the vault.
|
||||
- `.obsidian/` is never written. `.notescraft/` is written only through `Vault::write_private` (runtime data: history, workspaces, sync state). The `.notescraft/` folder is never committed.
|
||||
- Every other disk write goes through `craft_atomic::atomic_write(path, bytes)` (atomic rename, no half-written files).
|
||||
- Deleting a user note goes to the vault trash (`delete_to_trash`), never `remove_file`.
|
||||
- Paths are vault-relative with `/` separators; reject `..`, absolute, backslash and control characters before touching disk.
|
||||
|
||||
## Engine style
|
||||
- Library crates use `thiserror`, not `anyhow`. Commands return a `CommandError { code, message }` built from the crate error; messages are user-safe and never include secrets.
|
||||
- New code goes in a sub-folder with `mod.rs` when the directory already holds 12 loose files (`crates/engine/src/vault/` is full; `crates/engine/src/` has room for folders, not loose files).
|
||||
- Layering: engine is L4; it may depend on L0..L3 crates (`vault`, `markdown`, `index`, `sync`, ...) and never on egui. Check with `cargo xtask layers`.
|
||||
- Update `docs/ARCHITECTURE.md` (public API, numbers) and `ROADMAP.md` only if the task OWNS them.
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
name: notescraft-rust-safety
|
||||
description: NotesCraft never-crash Rust idioms and the clippy pedantic traps that fail the build. Use whenever you write or edit Rust in this repo.
|
||||
---
|
||||
|
||||
# Never-crash Rust, NotesCraft style
|
||||
|
||||
The crates deny `unwrap_used`, `expect_used`, `panic`, `todo`, `unimplemented`, `unreachable` and forbid `unsafe`. Tests may use `unwrap`/`expect`. Clippy runs pedantic with `-D warnings`.
|
||||
|
||||
## Replace indexing and unwrap
|
||||
```rust
|
||||
// BAD: v[i], s[a..b], x.unwrap()
|
||||
// GOOD
|
||||
let first = items.get(index).ok_or(Error::Missing)?;
|
||||
let part = text.get(start..end).ok_or(Error::BadRange)?; // str slices are byte ranges and may split a char: get() returns None
|
||||
let (head, tail) = bytes.split_at_checked(n).ok_or(Error::Short)?;
|
||||
let total = a.checked_add(b).ok_or(Error::Overflow)?;
|
||||
let width = value.saturating_sub(pad);
|
||||
let n = u32::try_from(len).map_err(|_| Error::TooLarge)?; // never `len as u32`
|
||||
```
|
||||
Even a guarded `v[i]` is rejected in review when `i` comes from input (files, network, user).
|
||||
|
||||
## Hostile input
|
||||
- Cap every allocation sized by input (`Vec::with_capacity(n.min(CAP))`), bound recursion (depth constant), bound loops (each iteration consumes input or has a cap).
|
||||
- Invalid input returns `Err`. Never silently clamp, truncate or "repair" it.
|
||||
- Parse then validate, in one function at the boundary; trust values inside the crate afterwards.
|
||||
|
||||
## Errors
|
||||
One `thiserror` enum `Error` and `pub type Result<T> = std::result::Result<T, Error>;` per crate. No `anyhow` in libraries. Map foreign errors with `#[error("...: {0}")]` and `From` or `map_err`. Error text never contains secrets.
|
||||
|
||||
```rust
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum Error {
|
||||
#[error("invalid path: {0}")]
|
||||
InvalidPath(String),
|
||||
}
|
||||
pub type Result<T> = std::result::Result<T, Error>;
|
||||
```
|
||||
|
||||
## Style that passes review
|
||||
- Every public item has rustdoc; every module starts with `//!`.
|
||||
- Constants for numbers, with a doc comment: `/// Longest path accepted, in bytes.\npub const MAX_PATH_BYTES: usize = 1024;`.
|
||||
- Files at most about 400 lines; a directory holds at most 12 loose `.rs` files (`mod.rs`/`lib.rs` not counted): put new modules in a sub-folder with `mod.rs`.
|
||||
- Prefer let-chains over nested ifs: `if let Some(x) = a && x.ok() { ... }`.
|
||||
- Smallest design that works: no trait with one impl, no builder for two fields, no generic you do not need.
|
||||
|
||||
## Clippy pedantic traps seen here
|
||||
| Lint | Fix |
|
||||
|---|---|
|
||||
| `too_many_lines` (over 100) | split the function |
|
||||
| `many_single_char_names` | name loop vars `row`/`col`, `left`/`right` |
|
||||
| `assert!(x.is_empty())` in tests | `assert_eq!(x, Vec::<T>::new())` or `assert_eq!(x, [])` |
|
||||
| `explicit_iter_loop` | `for b in &digest` not `.iter()` |
|
||||
| `collapsible_if` / `collapsible_match` | `&&` let-chain |
|
||||
| `needless_pass_by_value` | take `&T` |
|
||||
| `format_push_string` | `write!(s, ...)` with `use std::fmt::Write` |
|
||||
| `cast_possible_truncation` and friends | `try_from`, `f64::from`, or `#[expect(clippy::..., reason = "...")]` with a real reason |
|
||||
| `struct_excessive_bools` | an enum or `#[expect]` with reason |
|
||||
| case-sensitive file extension compare | `Path::extension().is_some_and(\|e\| e.eq_ignore_ascii_case("md"))` |
|
||||
| `similar_names`, `items_after_statements`, `unused_self` | rename, move item up, make it an associated fn |
|
||||
| `dead_code` on cfg-gated code | `#[cfg_attr(..., allow(dead_code, reason = "..."))]` |
|
||||
No blanket `#![allow]`; a targeted `#[expect(lint, reason = "...")]` only when the lint is wrong here.
|
||||
@@ -0,0 +1,27 @@
|
||||
---
|
||||
name: notescraft-sync-protocol
|
||||
description: The NotesCraft vault sync protocol, its invariants, the pure notescraft-sync API, the server endpoints and how to test sync code. Use for any sync client, server or UI work.
|
||||
---
|
||||
|
||||
# Sync protocol (read `specs/mobile-and-sync/spec.md` for the full design)
|
||||
|
||||
## Invariants (never break)
|
||||
1. **Never lose data.** Unsure means keep both versions (conflict copy). Edit beats delete.
|
||||
2. The `.md` files are the truth; sync state lives in `.notescraft/sync/` (written with `Vault::write_private`).
|
||||
3. `.notescraft`, `.trash`, `.git`, `.obsidian` never sync. Every path from the network passes `notescraft_sync::check_syncable`; every hash passes `check_hash` and is verified against the downloaded bytes.
|
||||
4. The server never trusts the client: bodies capped (25 MiB), paths checked, tokens only as SHA-256 hashes, no secrets in logs or the audit table.
|
||||
|
||||
## Pure crate `notescraft-sync` (L1, no I/O; do not change it unless the task owns it)
|
||||
- `Ledger::{new, from_records, apply(Write) -> Outcome, changes_since(seq, limit), head, get}`; `Outcome::{Applied, Unchanged, Conflict}`; stale writer conflicts; stale edit over newer delete wins; stale delete over newer edit conflicts; identical hash spends no revision.
|
||||
- `plan(local_hashes, bases, remote_records) -> Vec<Action>`; `Action::{Download, DeleteLocal, Upload{path, base_seq}, DeleteRemote{path, base_seq}, Reconcile{remote, base_hash}, Adopt}`.
|
||||
- `merge3(base, ours, theirs) -> Merge::{Clean(String), Conflict}`; `conflict_path(path, device, stamp)`; `hash_hex`, `check_path`, `check_syncable`, `is_excluded`.
|
||||
- Types: `FileRecord{path, hash, size, mtime_ms, seq, deleted}`, `Changes{head, records, more}` with `cursor(since)`, `Base{hash, seq}`.
|
||||
|
||||
## HTTP API (server `apps/notescraft-sync-server/src/routes.rs`)
|
||||
Admin bearer: `POST /v1/vaults`, `POST /v1/vaults/{v}/pairing`. Device: `POST /v1/devices {code,name}`; per vault with device token: `GET /changes?since&limit&wait`, `GET|PUT|DELETE /files/{path}` (`X-Base-Seq`, `X-Mtime-Ms`, `ETag`, 201 applied / 200 unchanged / 409 `{current}`), `POST /batch/check {hashes}`. `GET /healthz`.
|
||||
|
||||
## Testing sync code
|
||||
- Pure logic: plain unit tests with explicit records.
|
||||
- Client driver: a `FakeRemote` implementing the `Remote` trait over `Ledger` + a blob map; two temp vault folders ("phone", "laptop"); assert both end identical, nothing lost, second run is a no-op.
|
||||
- Server: start it on `127.0.0.1:0` in a thread and speak real HTTP with `ureq` (see `apps/notescraft-sync-server/tests/api.rs`). Inject clocks as arguments (`now_ms`) instead of sleeping.
|
||||
- Add a randomised test when a rule has many cases (see `crates/sync/tests/merge_props.rs`, a tiny LCG, no extra dependency).
|
||||
@@ -0,0 +1,29 @@
|
||||
---
|
||||
name: notescraft-verify-loop
|
||||
description: The exact build, test, lint and report loop for NotesCraft tasks, including offscreen UI checks. Use before saying a task is done.
|
||||
---
|
||||
|
||||
# Verify loop
|
||||
|
||||
Run from the repo root. `export PATH="$HOME/.cargo/bin:$PATH"`. Your launcher already set `CARGO_TARGET_DIR`; never use `target/debug`.
|
||||
|
||||
1. Test first. Write the failing test, run it, confirm it fails for the right reason.
|
||||
2. `timeout 300 cargo test -p notescraft-<crate>` (always with `timeout`; a looping parser once ate 40 GB).
|
||||
3. `cargo clippy -p notescraft-<crate> --all-targets -- -D warnings` after EVERY module, not at the end.
|
||||
4. `cargo fmt -p notescraft-<crate>`.
|
||||
5. Repo checks that apply: `cargo xtask layers` (dependency layers), `cargo xtask tree` (12 loose files per dir), `cargo xtask i18n` (no hardcoded UI strings). Package names are `notescraft-<dir>`; the sync server app is `notescraft-sync-server`.
|
||||
6. Confirm your new tests actually ran: the `cargo test` output must list them by name. A module not declared in `lib.rs`/`mod.rs` has tests that silently never run.
|
||||
|
||||
## Test conventions
|
||||
- Unit tests in `src/<file>.rs` under `#[cfg(test)] mod tests`; integration tests in `crates/<name>/tests/*.rs`.
|
||||
- Byte offsets in tests: compute with `str::find`, never by hand (`é` is 2 bytes).
|
||||
- Temp dirs: `std::env::temp_dir().join(format!("nc-{name}-{}", std::process::id()))`, remove before use.
|
||||
- Assert behaviour (values, files, events), not just "does not panic".
|
||||
- UI behaviour is tested by driving real events through whole frames (see `crates/ui-egui/tests/driven_features.rs`): build `NotesApp`, feed `RawInput` with pointer/key/text events, assert on painted text and state.
|
||||
|
||||
## Look at UI you changed
|
||||
`cargo run -p notescraft-ui-egui --example snapshot -- --vault <dir> --out /tmp/x.png --size 390x844 --open "a.md"` renders offscreen. Open the PNG with the Read tool and compare with the design PNG named in the task (`design/mobile/*.png`). List remaining differences in the report.
|
||||
|
||||
## Never
|
||||
- Weaken or delete a test to pass; add `#[allow]` to silence clippy; skip a failing command.
|
||||
- Claim success without pasting the final `test result:` line of your last run.
|
||||
@@ -16,7 +16,7 @@ NotesCraft is a native, Obsidian-compatible notes app written in **Rust only** (
|
||||
- One crate per concern in `crates/<name>`, package `notescraft-<name>`, `[lints] workspace = true`, registered in the root `[workspace.dependencies]` and in `xtask/src/layers.rs`.
|
||||
- **Layering is enforced** by `cargo xtask layers`: a crate may depend only on strictly lower layers; nothing below L5 may use egui, eframe, winit or rfd. Layers: L0 `vault` `markdown`; L1 `index` `canvas` `diagrams` `whiteboard`; L2 `backlinks` `graph` `templates`; L3 `plugins` `themes`; L4 `engine`; L5 `editor` `mcp` `i18n`; L6 `ui-egui`.
|
||||
- Shared crates come from craft-libs (`craft-atomic`, `craft-settings`, `craft-ui`, `craft-quit`, `craft-print`), pinned by git rev in the root `Cargo.toml`. Do not change their API; ask first.
|
||||
- A new third-party dependency needs a written reason and owner approval. Already approved: `notify`, `pulldown-cmark`, `toml`, `wasmi`, `wat` (dev), `thiserror`, `serde`, `serde_json`.
|
||||
- A new third-party dependency needs a written reason and owner approval. Already approved: `notify`, `pulldown-cmark`, `toml`, `wasmi`, `wat` (dev), `thiserror`, `serde`, `serde_json`, and for sync (2026-10-19): `sha2`, `axum`, `tokio`, `ureq` (rustls), `rusqlite`.
|
||||
|
||||
## Never crash (outranks features)
|
||||
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
# NotesCraft
|
||||
|
||||
@AGENTS.md
|
||||
|
||||
Project skills live in `.claude/skills/` and path rules in `.claude/rules/`. Load the matching skill before you start: `notescraft-agent-workflow` (always), `notescraft-rust-safety` (any Rust), `notescraft-verify-loop` (before you claim done), `notescraft-egui-screen` (UI), `notescraft-engine-command` (engine or commands), `notescraft-sync-protocol` (sync).
|
||||
@@ -26,6 +26,7 @@ One-page summary of where NotesCraft stands. The contract is [SPEC.md](SPEC.md);
|
||||
| 14 | Editor blocks and lint | `lint`, `blocks`, engine `lint.*` and `block.*`, slash menu, grip, Lint tab, Settings > Editor and lint | done: 12 lint rules (spelling slot only), one-click and safe fixes, learned style, dismissals, `fix_on_save`, slash menu with groups and preview in Source and Visual, block grip, drag, menu, Vim `]]` `[[` `gb`, Visual tables, note embeds, `%%` comments; not built: spell-check, mobile bottom toolbar, Fix popover in the editor, keyboard use of the block menu (specs/editor-blocks-and-lint) |
|
||||
| 15 | Fast open and large vaults | `vault` listing and `count_files`, engine `open_lazy`, `index_step`, `ContentSearch`, `index.*`, Explorer virtualization and navigation, large-folder warning | done: window in under a second on 10 000 notes, background indexing with a status card, search from disk while indexing, pause and resume, rename and move refused until the index is complete, virtualized Explorer (50 000 rows), note counts, filter, collapse all, reveal and follow, pinned folders, sticky folder path, warning above 5 000 files; no index cache (not needed); not built: honouring `.gitignore` in non-vault folders, jump-to-letter strip, key bindings for the Explorer commands (specs/fast-open-and-explorer) |
|
||||
| 16 | Editor previews | `editor::preview`, engine `editor.previews.*`, colour swatches, external-link arrows, tree card, Mermaid Source toggle | done: `detect_colors` (hex, `rgb`/`rgba`, `hsl`/`hsla` and the basic colour names) and `detect_tree`, both pure and bounded, swatches in body text, inline code and ten fenced languages, the right-click colour menu (Copy color, Copy as rgb, the two switches), external-link arrows, the TREE card, the Mermaid Diagram/Source toggle and the error/unsupported/too-big chips; not built: swatches and arrows in Source mode and in rows with inline math, swatches for a hex Obsidian reads as a tag (specs/editor-previews) |
|
||||
| 17 | Mobile, sync, collaboration, launcher | `sync` and `collab` (L1), `apps/notescraft-sync-server`, engine `sync/` and `prefs/catalog/`, `ui-egui` `launcher` and `shell/compact`, `apps/notescraft` `launcher_window` | done 2026-10-20: sync ledger, planner and merge (28 tests), server with accounts, many vaults, roles, invites, hardening, quota, GC (40 tests), engine client, worker and `sync.*` commands, vault catalog and launcher (host wired), OT core (22 tests), compact layout, 15 PaperCraft screens, research (3 reports) and specs (`mobile-and-sync`, `collaboration`, `vault-launcher`, `agents`); also done: sync UI (Settings > Sync, status pill, conflicts) and the live-editing endpoints (SSE rooms, flush); not built: live session client in the engine and the editor UI, Share dialog, Android and iOS shells, E2EE, deploy, attachments chunking, server version history (see the three tasks.md files) |
|
||||
| 11 | PDF export, packaging, i18n coverage, hardening | `pdf`, `packaging/`, release | PDF export done (crate, `export.pdf`, CLI, dialog; LaTeX prints as source, `craft-print` not wired); packaging done for macOS (built and smoke-tested), Windows (MSI scripts, compile-checked only) and Linux (deb/rpm/AppImage scripts, untested here); i18n migration done (252 keys in `en.toml`, `cargo xtask i18n --strict` is part of `ci`) |
|
||||
|
||||
Also built beyond the phase list: Vim mode (`vim` crate and app-wide navigation), resizable and collapsable cards with rails, Windows/Linux custom title bar.
|
||||
|
||||
|
After Width: | Height: | Size: 165 KiB |
|
After Width: | Height: | Size: 168 KiB |
|
After Width: | Height: | Size: 136 KiB |
|
After Width: | Height: | Size: 140 KiB |
|
After Width: | Height: | Size: 125 KiB |
|
After Width: | Height: | Size: 128 KiB |
|
After Width: | Height: | Size: 121 KiB |
|
After Width: | Height: | Size: 123 KiB |
|
After Width: | Height: | Size: 109 KiB |
|
After Width: | Height: | Size: 110 KiB |
|
After Width: | Height: | Size: 102 KiB |
|
After Width: | Height: | Size: 104 KiB |
|
After Width: | Height: | Size: 140 KiB |
|
After Width: | Height: | Size: 143 KiB |
|
After Width: | Height: | Size: 94 KiB |
|
After Width: | Height: | Size: 96 KiB |
|
After Width: | Height: | Size: 92 KiB |
|
After Width: | Height: | Size: 94 KiB |
|
After Width: | Height: | Size: 57 KiB |
|
After Width: | Height: | Size: 50 KiB |
|
After Width: | Height: | Size: 100 KiB |
|
After Width: | Height: | Size: 113 KiB |
|
After Width: | Height: | Size: 123 KiB |
|
After Width: | Height: | Size: 110 KiB |
@@ -80,6 +80,31 @@ Sits one layer above `index` (it depends on it). `ContentSource` trait (no file
|
||||
### `notescraft-graph` (L2)
|
||||
`Graph::new(nodes, edges)`; deterministic Fruchterman-Reingold `layout(&Graph, &LayoutConfig) -> Vec<[f32;2]>` with a spatial grid, `Layout` for incremental `step`, pinning, `local_graph(depth)`, `degrees`, `components`, `orphans`. 5000 nodes / 15 000 edges / 300 iterations lays out in about 525 ms in release.
|
||||
|
||||
### `notescraft-sync` (L1)
|
||||
Pure sync logic shared by the future client and server, no I/O (`serde`, `sha2`, `thiserror`). `Ledger` keeps one `FileRecord` per path plus a revision counter: `apply(Write)` returns `Applied`, `Unchanged` (same hash, no revision spent) or `Conflict(current)`; a writer that saw the current `seq` wins, a stale one conflicts, a stale edit over a newer delete wins and a stale delete over a newer edit conflicts; `changes_since(seq, limit)` gives a `Changes` page with `cursor()`. `plan(local, bases, remote)` turns a disk scan, the last agreed `Base` per path and a `/changes` page into `Action`s (`Download`, `DeleteLocal`, `Upload`, `DeleteRemote`, `Reconcile`, `Adopt`). `merge3` is a line-level three-way merge that returns `Conflict` on any overlap or on input over 4 M diff cells. `conflict_path` names the conflict copy. `check_path` / `check_syncable` reject empty, absolute, `..`, backslash and control-character paths and the `.notescraft`, `.trash`, `.git`, `.obsidian` roots; `hash_hex` / `check_hash` are SHA-256 lowercase hex. 26 integration tests.
|
||||
|
||||
### `notescraft-sync-server` (app, exempt from layers)
|
||||
HTTP service over `notescraft-sync`. Env: `NOTESCRAFT_SYNC_ADMIN_TOKEN` (24+ chars), `NOTESCRAFT_SYNC_DATA` (default `./data`: `sync.db`, `blobs/<vault>/<sha256>`), `NOTESCRAFT_SYNC_BIND` (default `127.0.0.1:8480`), `NOTESCRAFT_SYNC_QUOTA_MB` (default 5120, 0 = unlimited). Source folders: `api/` (routes, accounts, vaults, devices, members, invites, audit, admin), `store/` (SQLite: users, vaults, devices, files, invites, pairings, audit), plus `roles.rs` (`authorize(Need::{Read, Write, Admin})`), `cache.rs` (LRU of 64 ledgers), `limits.rs` (pairing lockout: 5 failures in 10 minutes, shared by all callers, in memory), `middleware.rs` (`X-Request-Id`, one stderr line per request). Accounts: a user owns many vaults and belongs to others (roles owner, editor, viewer; non-members get 404); one device token works for every vault of its user; admin `POST /v1/users`, admin pairing codes (`?user=` optional, a well-formed new vault id is created by its first code), `GET/POST/PATCH/DELETE /v1/vaults` (device token; 50 owned per user; delete is soft for 14 days, `restore`, `purge`), members and invites (single-use 48 h codes, stored as SHA-256), `files-meta`. Sync: `GET /changes?since&limit&wait` (long poll up to 25 s), `GET|PUT|DELETE /files/{path}` (`X-Base-Seq`, `X-Mtime-Ms`, `ETag`, 201/200/409, 25 MiB cap, per-vault quota 413), `POST /batch/check`, admin `POST /v1/vaults/{v}/gc` (blobs unreferenced and past the 14-day tombstone grace). Ops: `GET /healthz`, `GET /readyz` (db write probe in a rolled-back transaction plus a blob-folder probe), admin `GET /v1/audit` (no secrets in any row). Known ceilings: blobs are never garbage-collected unless the admin runs `gc`, pairing lockout is global (no per-IP limit behind the tunnel) and in memory, blobs on local disk (MinIO later). 40 HTTP tests including the real engine client (`tests/client_e2e.rs`).
|
||||
|
||||
History (`history/`, `store/revisions.rs`): every applied write inserts a `file_revisions` row in the same transaction as the `files` upsert; `GET /files-history/{path}` (newest first, max 200), `GET /files/{path}?seq=N`, `POST /files-restore/{path} {seq}` (writes the old bytes as a new revision, 404 unknown, 410 blob collected), per-vault `retention_days` (1..=365, default 30) with the newest 20 revisions always kept, admin `gc` removes expired revisions and unreferenced blobs, `bytes_used` counts unique live plus retained blobs.
|
||||
|
||||
Live editing (`live/`): under `/v1/vaults/{vault}/live/{path}/` `join` (snapshot, rev, peers; viewers may join, 413 over 2 MiB, 415 non-UTF-8), `ops` (OT submit, 409 stale, 400 malformed), `events` (SSE `op`, `ack`, `presence`, `leave`, keepalive), `presence`, `leave`; rooms in memory with a 64-slot channel per peer, caps 256 rooms and 64 peers, a sweeper flushes 2 s after the last op through the same blob and ledger path as `PUT` (`flush`), a write that lost a race becomes a `conflict_path(path, "live", date)` copy. 14 tests.
|
||||
|
||||
### Engine sync (`crates/engine/src/sync/`)
|
||||
`Remote` trait with `HttpRemote` (blocking `ureq`), `pair`, `SyncState` (`.notescraft/sync/state.json`), `sync_once` (scan, changes, `notescraft_sync::plan`, perform, base copies in `.notescraft/sync/base/<hash>` for `merge3`), `SyncWorker` (one thread: debounce 2 s, 15-minute heartbeat, backoff 5 s to 5 min, phases Idle/Syncing/Offline/SignInAgain/Error, writes `.notescraft/sync/status.json`), binding and token files (`binding.json`, `credentials.json` mode 0600), conflict copies (`list_conflicts`, `resolve` Mine/Theirs/Both), commands `sync.pair`, `sync.status`, `sync.now`, `sync.unlink`, `sync.conflicts`, `sync.resolve` (gate: status and conflicts read, now and resolve write, pair and unlink delete). `Vault::sibling` gives the worker thread a writable handle without the lock.
|
||||
|
||||
### Vault catalog and launcher
|
||||
`AppSettings.vault_catalog: Vec<VaultMeta{path, name, pinned, last_opened_ms}>` (a list, because the settings store merges tables key by key), `list_rows` (pinned first, folder checks inside a 500 ms budget, local sync badge from `binding.json`, `pending.json`), `touch`, `set_pinned`, `set_name`, `forget`, commands `vaults.list/pin/rename/remove`. `ui-egui` `launcher::Launcher` (desktop rail and cards, phone one column, keyboard, rename, filter) returns a `LauncherAction`; the app host (`apps/notescraft/src/launcher_window.rs`) performs it, starting vaults as separate processes. Shown when no recent vault exists, with `--launcher`, and from File > Vaults....
|
||||
|
||||
### Capture (`crates/engine/src/capture/`)
|
||||
`note.capture {text, title?, target?, source?, attachment?, utc_offset_minutes?}`: `target` is `daily` (default: appends `- HH:MM text` or a `### HH:MM title` block to today's daily note, through the open buffer when there is one so undo works), `inbox` or a vault folder (creates `<folder>/<YYYY-MM-DD HHmmss> <slug>.md` with frontmatter `created` and `source`, numbering a taken name); `attachment {name, base64}` is saved through the attachment helper and embedded. Caps: text 1 MiB, title 200 and source 2000 characters, attachment 20 MiB; absolute, `..`, hidden and reserved targets are refused. Gate: write.
|
||||
|
||||
### `notescraft-collab` (L1)
|
||||
Live co-editing core, no I/O: `TextOp` (Retain/Insert/Delete over Unicode scalar counts; wire format `[5,"hi",-2,3]`), `apply`, `compose`, `invert`, `transform` (TP1 property-tested), `transform_index`, `Selection`, `diff`, the OT `Client` state machine, the server `Room` (history cap 1000, size cap 2 MiB), `Presence` (64 peers, sanitised names). 22 tests.
|
||||
|
||||
### Compact (phone) layout
|
||||
`shell/compact/`: below 600 pt one page at a time (`CompactPage` Notes, Note, Search, Graph, Calendar, More), bottom tab bar (88 pt, 44 pt hit areas), back chevron top bar; `UiState.compact_page` persisted.
|
||||
|
||||
### `notescraft-templates` (L2)
|
||||
No chrono: own `Date`/`DateTime`/`Weekday` (civil-from-days), `format()` (Moment token subset), `expand(template, &Context)` for `{{title}}`, `{{date[:FMT]}}`, `{{time[:FMT]}}`, `daily_note_path`, `parse_daily_name`, `month_grid` for the calendar widget.
|
||||
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
# Working with MiniMax M3.1 Flash agents
|
||||
|
||||
For the orchestrator (Claude Code or the owner). The agent-facing brief is `minimax-m3-1-flash.md`; the launcher prepends it to a task file.
|
||||
|
||||
## Facts (verified 2026-10-20)
|
||||
- Model id `MiniMax-M3.1-Flash` (the released model; the Preview id `MiniMax-M3.1-Flash-Preview` is retired in `~/.pi/agent/models.json`). 1M context, text and image input, reasoning, Anthropic-compatible API at `https://api.minimax.io/anthropic`.
|
||||
- Owner has no usage limit until 2026-10-25: use it freely, but keep at most 3 agents running at once (earlier 4-6 parallel agents hit 429 `rate_limit_error (2062)` and Pi then looped writing 300 MB logs).
|
||||
- Response cap about 16K tokens: tasks must be sliced so one file is written per tool call.
|
||||
|
||||
## Harness choice
|
||||
| Harness | Command | Use for |
|
||||
|---|---|---|
|
||||
| Claude Code harness (`scripts/mm-agent.sh`) | `claude -p` with `ANTHROPIC_BASE_URL`, `ANTHROPIC_AUTH_TOKEN` read from Pi's `models.json`, `--setting-sources project`, `--strict-mcp-config` | code tasks. Edit/Write/Bash/Task tools, reads `AGENTS.md`, `--max-turns` bound, `stream-json` log. Tested 2026-10-20: wrote a file through the Write tool on the first try. Prints a harmless "unrecognized model, assumes 200k window" notice. |
|
||||
| Pi (`pi -p --provider minimax --model MiniMax-M3.1-Flash --no-session --thinking low`) | | research with web search and fetch (default extensions on), and `scripts/pi-wave.sh` worktree waves |
|
||||
|
||||
Recommendation: Claude Code harness for coding, Pi for browsing research. Both stay inside the same model quota.
|
||||
|
||||
## Briefing checklist (a task file must have)
|
||||
`OWNS` (paths the agent may edit), `READ FIRST` (files and spec sections), `GOAL` (one paragraph), `API` (exact signatures wanted), `STEPS` (ordered, each at most about 150 lines), `TESTS` (named cases), `VERIFY` (exact commands), `DO NOT` (scope fences). The more exact, the fewer rounds.
|
||||
|
||||
## Review before accepting (orchestrator)
|
||||
1. `git status` shows only OWNS paths changed.
|
||||
2. `cargo xtask ci` green (fmt, clippy -D warnings, tests, layers, tree, i18n).
|
||||
3. Grep the diff for `unwrap(`, `expect(`, `[` indexing on input, `panic!`, hardcoded UI strings.
|
||||
4. Read the tests: they must assert behaviour, not just run.
|
||||
5. Commit only when the owner asks.
|
||||
|
||||
## Known failure modes
|
||||
- Stalls after a giant single write: slice the task.
|
||||
- Declares success without running clippy: the brief demands the pasted summary line; check it.
|
||||
- Edits shared files (`lib.rs`, root `Cargo.toml`): list them under OWNS explicitly or forbid them.
|
||||
- Invents dependencies: the brief lists the approved ones.
|
||||
|
||||
## Skills and rules shipped to the agents (2026-10-20)
|
||||
`CLAUDE.md` imports `AGENTS.md`; `.claude/skills/notescraft-*` (agent-workflow, rust-safety, verify-loop, egui-screen, engine-command, sync-protocol) and `.claude/rules/*.md` (path-scoped: rust-never-crash, ui-egui, engine-and-vault, sync, tests). The Claude Code harness loads them with `--setting-sources project`. Keep them short and example-first; when an agent repeats a mistake, add the fix to the skill's table, not to the task file.
|
||||
|
||||
## Lesson 2026-10-20: the cap of 3 counts every MiniMax process
|
||||
Running 2 Pi research agents + 1 Claude Code harness agent + a fourth launched too early produced `429 rate_limit_error (2062)` and killed a research run (`mobile-platform`, rerun as a continuation). Count Pi research and harness agents together; launch the next only after a completion event. Research agents must write the output file in appended chunks early, so a cut-off keeps a partial draft that a continuation run can finish.
|
||||
|
||||
## Lesson 2026-10-20: disk
|
||||
`/Volumes/Storage` hit 100% (225 MiB free) because `target/agent-ui/debug/incremental` held 29 GB (and `deps` 28 GB). `scripts/mm-agent.sh` now sets `CARGO_INCREMENTAL=0`. Check `df -h /Volumes/Storage` before launching an agent; delete only `target/agent-*/debug/incremental` (regenerable), never `target/debug`.
|
||||
|
||||
## Lesson 2026-10-20 (2): effective concurrency is 1 harness agent
|
||||
Two Claude Code harness agents at once (collab-ot + server-hardening) hit `429 rate_limit_error` on every request (`api_retry` attempts 1 to 10) while a Pi research run was also active; the harness sends more requests per minute than Pi. Run harness agents ONE AT A TIME with `scripts/mm-queue.sh name1 name2 ...` (waits for the running agent, retries an agent that only saw 429s, clears `incremental` caches when disk is low) and at most one Pi research run beside it. The owner's "up to 3 subagents" is a ceiling, not a target.
|
||||
|
||||
## Lesson 2026-10-20 (3): MiniMax plan quota (error 2067) and the GLM fallback
|
||||
After about 2 hours of heavy use (4 research and coding agents) MiniMax answered every request with `429 rate_limit_error: The Token Plan usage limit has been reached (2067)`; this is the plan's own quota, not a burst limit (a direct curl with 20 tokens fails the same way), so waiting minutes does not help and the owner's "no limit until 25 Oct" did not hold for this key at that moment. Fallback: GLM 5.3 Flash on Ollama Cloud through Pi: `scripts/pi-agent.sh <name> scripts/mm/<name>.md [provider model]` (default `ollama-cloud glm-5.3-flash`). It embeds this brief and the six `notescraft-*` skills in the prompt because Pi does not load `.claude/skills`. GLM allowed 3 concurrent agents earlier (see memory `agent-model-routing`); run tasks with disjoint OWNS side by side. Check MiniMax again with the curl in this section before switching back: `curl -s https://api.minimax.io/anthropic/v1/messages -H "x-api-key: $KEY" -H "anthropic-version: 2023-06-01" -H "content-type: application/json" -d '{"model":"MiniMax-M3.1-Flash","max_tokens":20,"messages":[{"role":"user","content":"say ok"}]}'` (read `$KEY` from `~/.pi/agent/models.json`, never print it).
|
||||
@@ -0,0 +1,54 @@
|
||||
# Agent brief: MiniMax M3.1 Flash on NotesCraft
|
||||
|
||||
You are a coding agent building part of NotesCraft: a native, Obsidian-compatible notes app in Rust only (egui/eframe, no JavaScript, no webview). Read this whole brief, then the TASK at the end. Follow it exactly. When the brief and the task disagree, the task wins; when the task and the repo rules (AGENTS.md) disagree, stop and report.
|
||||
|
||||
## Skills and rules (load them)
|
||||
This repo ships project skills in `.claude/skills/` and path rules in `.claude/rules/` (loaded automatically in the Claude Code harness; with Pi read the files yourself). Start with `notescraft-agent-workflow`, then `notescraft-rust-safety` for Rust, `notescraft-verify-loop` before you finish, `notescraft-egui-screen` for UI, `notescraft-engine-command` for engine and vault, `notescraft-sync-protocol` for sync. They hold the code patterns to copy; this brief holds the hard limits.
|
||||
|
||||
## Before you write code
|
||||
1. Read `AGENTS.md` (workspace rules) and every file the task lists under READ FIRST.
|
||||
2. Read neighbouring code in the crate you edit and copy its style: naming, comment density, error handling, test layout. Do not invent a new pattern when one exists.
|
||||
3. Plan the work as small steps of at most about 150 lines of new code each. Write one file per tool call, build after each step.
|
||||
|
||||
## Hard rules (the build and review reject violations)
|
||||
- Never `unwrap()`, `expect()`, `panic!`, `todo!`, `unreachable!`, `unimplemented!` in non-test code. No `unsafe`.
|
||||
- Never index or slice with an input-derived value (`v[i]`, `s[a..b]`), even when guarded. Use `get()`, `split_at_checked`, iterators, `checked_*`, `saturating_*`.
|
||||
- Bound every loop over input (each iteration consumes input or has a cap) and every allocation sized by input. A loop that does not advance can eat all memory: run tests as `timeout 120 cargo test ...`.
|
||||
- Invalid input returns `Err`. Never silently clamp, truncate or "fix" it.
|
||||
- Every user-visible string goes through the i18n tables (`crates/i18n/locales/en.toml`, then `cargo xtask i18n` must be clean). No hardcoded UI text.
|
||||
- Every write to disk goes through `craft-atomic` (or the vault's `write_private` for `.notescraft/`). Never write `.obsidian/`.
|
||||
- Files stay at or below about 400 lines; split by concern. A source directory holds at most 12 loose `.rs` files (`mod.rs`, `lib.rs` not counted): put new modules in a sub-folder with `mod.rs` when the directory is full. Check with `cargo xtask tree`.
|
||||
- Layering is enforced (`cargo xtask layers`): a crate depends only on lower layers; nothing below L6 uses egui.
|
||||
- A new third-party dependency needs owner approval. Approved: `notify`, `pulldown-cmark`, `toml`, `wasmi`, `thiserror`, `serde`, `serde_json`, `sha2`, `axum`, `tokio`, `ureq` (rustls), `rusqlite`. Anything else: stop and ask in your final report.
|
||||
- Every public item has rustdoc; every module has a `//!` comment. One `thiserror` enum `Error` and `pub type Result<T>` per crate.
|
||||
- Clippy pedantic is on with `-D warnings`. Known traps: `too_many_lines` (100), `many_single_char_names` (use `row`/`col`, not `i`/`j`, when more than 4 short names), `assert!(x.is_empty())` (use `assert_eq!(x, [])`), `explicit_iter_loop`, collapsible `if let` chains (use `&&` let-chains), by-value params that are not consumed, `format!` appended to a `String` (use `write!`), `as` casts between integer/float types (use `try_from` or `f64::from`).
|
||||
- Smallest design that works. No speculative abstractions, no extra features, no files beyond what the task names plus tests.
|
||||
|
||||
## Working rules for the shared tree
|
||||
- You share the working tree with other agents. Edit ONLY the files and folders the task lists under OWNS. Never touch root `Cargo.toml`, `Cargo.lock` (cargo may update it for approved dependencies already registered; do not hand-edit), `xtask/`, `SPEC.md`, `ROADMAP.md`, `docs/`, or another agent's files.
|
||||
- Do not run `git add`, `git commit`, `git checkout`, `git stash`, `git reset` or any command that changes git state. Do not delete files you did not create.
|
||||
- Use your own target dir: the launcher already sets `CARGO_TARGET_DIR`. Never touch `target/debug`.
|
||||
- Never print, copy or log secrets or tokens.
|
||||
|
||||
## Tests and verification (all mandatory)
|
||||
- Test first: write the failing test, then the code. Unit tests in `src`, integration tests in `tests/`. Tests may use `unwrap`/`expect`.
|
||||
- Byte offsets in tests: compute with `str::find`, never by hand (`é` is 2 bytes).
|
||||
- Every module must be declared in its `lib.rs`/`mod.rs`, and its tests must run: after writing tests, confirm `cargo test -p <crate>` lists them.
|
||||
- Run, in this order, and fix every failure at the root cause (never weaken a test, never add an `allow` to pass):
|
||||
1. `timeout 300 cargo test -p <crate>`
|
||||
2. `cargo clippy -p <crate> --all-targets -- -D warnings`
|
||||
3. `cargo fmt -p <crate>`
|
||||
4. the extra commands the task names
|
||||
- Do not claim success unless these passed in your last run. Paste the final test summary line.
|
||||
|
||||
## Output hygiene (you have a response cap)
|
||||
- Keep each response small: one file per tool call, at most about 250 lines. A huge single response gets cut off and you will stall.
|
||||
- Keep thinking short. When stuck for more than three attempts on one error, stop and report the exact error instead of looping.
|
||||
- If a command prints a lot, filter it (`| tail -40`, `| grep -E "error|warning" -A5`).
|
||||
|
||||
## Final report (max 15 lines)
|
||||
What is implemented; public API signatures; test counts and the last `cargo test` summary line; every file you created or changed; anything skipped, any dependency you needed but lacked, any rule you could not satisfy.
|
||||
|
||||
---
|
||||
TASK:
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
# Attachments (chunking, resumable upload, dedup) and server version history
|
||||
|
||||
Source: research recommendations 5, 8, 9 (`security-and-server.md`, `sync-landscape.md`).
|
||||
|
||||
## Problem
|
||||
Files over a few MiB are sent whole in one request (25 MiB cap): a flaky mobile link restarts from zero, the same photo in two vaults is stored twice, and old versions of notes are lost on the server (only local snapshots exist). Obsidian Sync offers 12 months of note versions; sync clients like Syncthing and Dropbox chunk large files.
|
||||
|
||||
## Goal
|
||||
Large attachments sync reliably on bad networks; every note version is recoverable from the server for a retention window.
|
||||
|
||||
## Non-Goals
|
||||
Real-time delta sync of Markdown (files are small), cross-vault dedup in encrypted vaults (a deliberate leak we avoid), a general file manager.
|
||||
|
||||
## Design
|
||||
1. **Blob chunking.** Files above 8 MiB upload as chunks of 4 MiB content-addressed by SHA-256: `PUT /v1/vaults/{v}/chunks/{hash}` (idempotent, size-capped), then the file record points to an ordered chunk list (`PUT /files/{path}` with `X-Chunks: h1,h2,...`). `POST /batch/check` already finds missing hashes; extended to chunk hashes. Resume = ask which chunks exist and send the rest. Server verifies every chunk's hash and the assembled size.
|
||||
2. **Dedup.** Identical chunks are stored once per vault (single owner) and, for plaintext vaults only, once per server across vaults of the same user (documented equality leak).
|
||||
3. **Version history.** The ledger keeps every applied record as a revision (`file_revisions(vault, path, seq, hash, size, mtime, editor, deleted)`); blobs of old revisions stay until retention expires. Retention: default 30 days and at least the last 20 revisions per file, per-vault setting up to 365 days; the GC (`POST /gc`) honours it. Endpoints: `GET /files-history/{path}` (list), `GET /files/{path}?seq=N` (bytes of a revision), `POST /files-restore/{path} {seq}` (writes the old bytes as a new revision, never rewrites history).
|
||||
4. **Client.** The sync driver chunks big files, retries failed chunks with backoff, shows per-file progress; the Versions tab lists server revisions (merged with local snapshots) and restores.
|
||||
5. **Quota.** Counts unique chunk bytes plus retained revisions; the owner sees the split.
|
||||
|
||||
## Acceptance Criteria
|
||||
- [ ] Given a 60 MB file and a connection cut after 21 MB, when sync resumes, then at most one chunk (4 MiB) is re-sent and the final hash matches.
|
||||
- [ ] Given the same 30 MB attachment in two notes of one vault, then the server stores its chunks once.
|
||||
- [ ] Given a note edited 40 times over 3 days, then history lists the revisions with times and editors, and restoring revision 12 creates revision 41 holding revision 12's bytes.
|
||||
- [ ] Given revisions older than retention, then GC removes their unique blobs and `bytes_used` drops; newer ones stay.
|
||||
- [ ] Given a chunk with a wrong hash, then the server answers 400 and stores nothing.
|
||||
|
||||
## Open Questions
|
||||
1. Retention default (30 days or 12 months like Obsidian, which counts toward quota)? Recommended: 30 days, owner can raise.
|
||||
2. Content-defined chunking (rolling hash) instead of fixed 4 MiB? Recommended: fixed first; revisit if edits to big files are common.
|
||||
|
||||
## Tasks
|
||||
1. Server: `chunks` table and endpoints, assembled-file reads, quota accounting.
|
||||
2. [x] Server: `file_revisions`, history, restore, retention GC. Done 2026-10-20 by GLM 5.3 Flash via Pi (MiniMax was out of quota): `src/history/`, `store/revisions.rs`, endpoints `files-history`, `files/{path}?seq=`, `files-restore`, per-vault `retention_days` (default 30, newest 20 always kept), quota counts retained revisions; verified by the orchestrator (all server suites green, clippy clean). Engine and UI still open (task 4).
|
||||
3. Engine: chunked upload/download in the driver with resume and progress events.
|
||||
4. UI: transfer progress in the status pill and Settings > Sync; Versions tab shows server revisions.
|
||||
@@ -0,0 +1,38 @@
|
||||
# Capture and publish: share targets, quick capture, web clipper, publish
|
||||
|
||||
Source: research (`mobile-platform.md` findings on Obsidian mobile quick capture and share targets; `security-and-server.md` adjacent features ranking: web clipper, email-to-note, voice transcription, publish, OCR, read-only links). All except the share target and quick capture need a server-readable (non-E2EE) vault.
|
||||
|
||||
## Problem
|
||||
Notes start outside the app: a link in a browser, text in another app, a voice thought, a photo of a page. Today the user must open NotesCraft, create a note and paste. On a phone that is the main reason notes never get written. Publishing a note as a web page needs a third-party service.
|
||||
|
||||
## Goal
|
||||
One action from anywhere puts content into the vault (inbox note, daily note or chosen folder), and a note or folder can be published read-only on the owner's own domain.
|
||||
|
||||
## Non-Goals
|
||||
Hosting for other people's domains, comments, analytics, a rich-text CMS, offline OCR on the device in v1.
|
||||
|
||||
## Design (ranked by value over effort)
|
||||
1. **Quick capture** (P0, phone and desktop): a command `note.capture {text, title?, target?}` and a global shortcut / phone sheet that appends to the daily note or creates `Inbox/<timestamp>.md`; works with no vault window open on desktop through the control channel.
|
||||
2. **Share target** (P0 phone): Android intent filter for `text/plain` and `image/*`, iOS share extension (native Swift, writes into the app group container, picked up on next launch); both call `note.capture`; images land in the attachment folder with an embed.
|
||||
3. **Web clipper** (P1): a bookmarklet-free approach: `POST /v1/vaults/{v}/capture {url?, title, markdown, source}` on the sync server (device token) so a browser extension, a shortcut or `curl` can drop a clipped page; the server stores it as `Clips/<title>.md` with frontmatter (`source`, `clipped`), and file sync delivers it. The extension itself is out of scope (a documented API).
|
||||
4. **Email to note** (P1): an inbound webhook on the homelab (n8n exists) calls the same capture endpoint.
|
||||
5. **Voice memo** (P1): record on the phone, save `.m4a` into attachments with an embed note; transcription runs on the homelab (whisper-class model, server-readable vaults only) and appends the transcript; shows "transcribing..." state.
|
||||
6. **Publish** (P1): `publish.enable {folder|note}` marks content public; the server renders Markdown to static HTML (reusing `crates/markdown`, no new parser) under `https://<host>/p/<vault slug>/<path>` behind Traefik, with theme, backlinks off by default; unpublish deletes. Read-only share links with an expiry token are the same renderer behind a secret path.
|
||||
7. **OCR** (P2): a server job turns an image attachment into text appended under the embed.
|
||||
|
||||
## Acceptance Criteria
|
||||
- [ ] Given text shared from another phone app, then within 5 seconds a note exists in `Inbox/` with the text and a creation date, even if the app was closed (picked up on next start for iOS).
|
||||
- [ ] Given `curl -H "Authorization: Bearer <token>" -d '{"title":"T","markdown":"# x","url":"https://e.x"}' .../capture`, then `Clips/T.md` appears on every paired device with `source: https://e.x`.
|
||||
- [ ] Given a published folder, then an anonymous GET of a note page returns HTML with no script, the vault's other notes are not reachable, and unpublishing returns 404 within a minute.
|
||||
- [ ] Given an expired share link, then 404.
|
||||
|
||||
## Open Questions
|
||||
1. Which transcription model on the homelab (the cluster has GPU workloads; needs a capacity check per `HOMELAB_OPERATIONS.md`)?
|
||||
2. Subdomain per user for publish, or one host with slugs? Recommended: one host, slugs.
|
||||
|
||||
## Tasks
|
||||
1. [x] Engine: `note.capture` command, inbox rules, tests. Done 2026-10-20 by the orchestrator (MiniMax and GLM were both out of quota): `crates/engine/src/capture/`, 5 rule tests and 9 command tests (daily append, heading block, inbox numbering, folder target, attachment, size limits, read-only, open buffer with undo, hostile params).
|
||||
2. Server: `/capture` endpoint with size caps and audit; docs for a clipper.
|
||||
3. Phone: share target spike (Android intent first).
|
||||
4. Server: static renderer and `/p/...` routes, publish commands.
|
||||
5. Voice memo and transcription job (after a capacity decision).
|
||||
@@ -0,0 +1,64 @@
|
||||
# Collaboration: live co-editing and shared notes
|
||||
|
||||
Extends `specs/mobile-and-sync/spec.md`, which listed both as non-goals. The owner asked on 2026-10-20 for two devices to work on the same note at the same time, and for notes to be shared between different users.
|
||||
|
||||
## Problem
|
||||
File sync converges edits after the fact: two people typing in one note produce a merge or a conflict copy, and a vault has exactly one owner. There is no way to give someone else a folder of notes or to see who is editing now.
|
||||
|
||||
## Goal
|
||||
Two or more people (or two devices of one person) can edit the same note at the same time, see each other's cursors, and have every keystroke converge in under a second, inside a vault or folder the owner shared with chosen roles.
|
||||
|
||||
## Non-Goals
|
||||
- Rich-text or block CRDTs; the document is the note's plain Markdown text.
|
||||
- Live editing inside end-to-end encrypted vaults (the server must read operations to order them; E2EE vaults keep file sync only, stated in the UI).
|
||||
- Public web pages, anonymous link editing, comments and suggestions (P2, later), organisation or team billing.
|
||||
|
||||
## Design decisions
|
||||
1. **Live model: server-ordered operational transformation (OT) on plain text.** Reasons: one central server already exists; the note stays a normal `.md` file; no new third-party dependency (a CRDT crate such as `yrs`, `automerge`, `loro` or `diamond-types` would be a dependency needing owner approval and a binary document format beside the `.md`). A CRDT's advantage (offline merge) is already covered by file sync + `merge3`. Ceiling: OT with one server needs the server to be online for the live part; offline editing falls back to file sync.
|
||||
2. **Pure crate `notescraft-collab`** (L1): `TextOp` (components Retain/Insert/Delete over Unicode scalar counts), `apply`, `compose`, `transform`, `invert`, cursor/selection transform, the OT client state machine (synchronised / awaiting ack / awaiting with buffer), the server `Room` (document text, revision, bounded op history, join snapshot, `submit(client_id, base_rev, op) -> Accepted{rev, op}`), presence records. Convergence is property-tested (`apply(apply(s,a),b') == apply(apply(s,b),a')`).
|
||||
3. **Transport without new dependencies**: `POST .../live/{path}/join` returns `{text, rev, peers}`; `POST .../live/{path}/ops` carries `{client, base_rev, op, cursor}`; `GET .../live/{path}/events` is a Server-Sent Events stream of `op`, `ack`, `presence`, `leave`. (axum SSE; the client reads the stream line by line on a worker thread with `ureq`.)
|
||||
4. **Persistence**: a room writes its text to the vault ledger as a normal write (`PUT` semantics, author = the last editor) after 2 s without edits and when the last peer leaves; file-sync clients receive it like any change. A client that holds a live session ignores the echo when its hash already matches.
|
||||
5. **Accounts and roles**: users (`id`, display name, colour) own devices; a vault has members with a role `owner`, `editor` or `viewer`. Viewers read; editors read, write, join rooms; owners also manage members and invites. Every endpoint checks the role server-side; the audit log records invites, joins, role changes, removals.
|
||||
6. **Sharing unit = a shared vault mounted as a folder.** The owner marks a folder (or creates a new shared vault); recipients mount it at `Shared/<name>/` in their own vault with its own sync state (`.notescraft/sync/<mount>.json`). Reuses the whole sync client; no per-file ACLs in v1.
|
||||
7. **Invites**: owner creates a single-use code with a role (valid 48 h, hashed in storage, lockout like pairing); the recipient enters the code or opens a link `notescraft://join?server=...&code=...`; accepting adds a membership and the mount.
|
||||
8. **Presence**: colour, name, cursor and selection of every peer, ephemeral (never stored), pruned 30 s after the last heartbeat.
|
||||
|
||||
## Many vaults (owner requirement, 2026-10-20)
|
||||
One person has many vaults (work, personal, a shared project, a course) and belongs to vaults other people own. Rules:
|
||||
- **Account-level device token.** A device belongs to a user, not to one vault; its token works for every vault the user is a member of (role per vault). Pairing a device once connects the whole account. (The current server binds a token to one vault; the accounts task migrates it: existing devices become members of a personal user.)
|
||||
- **Server API**: `GET /v1/vaults` (vaults I belong to: id, name, role, members count, bytes used, quota, head seq, last change), `POST /v1/vaults {name}` (creator is owner), `PATCH /v1/vaults/{v} {name}`, `DELETE /v1/vaults/{v}` (owner; soft delete with a 14-day grace, audited). Limits: 50 vaults per user, name 1..80 chars.
|
||||
- **Client binding per local vault**: `.notescraft/sync/binding.json` = `{server, vault_id, user_id, device_id}`; the token lives in the OS keychain under `notescraft-sync/<server>/<user>` (one entry per account, not per vault). A local vault with no binding is simply not synced. Two local vaults may bind to two different server vaults, even on two different servers.
|
||||
- **Vault picker UI**: a Vaults screen lists local vaults and server vaults I can still connect, each with a sync pill, role badge and members; actions: Open, Connect to a server vault, Create server vault from this local vault, Leave. Desktop gets the same list in the vault switcher popover.
|
||||
- **Scale**: the server loads ledgers lazily and keeps at most 64 in memory (LRU, reloaded from SQLite); rooms and SSE streams are per (vault, path); audit, quota and GC are per vault; no endpoint ever lists another user's vaults.
|
||||
- **Background work**: only the open vault syncs with a live loop; other bound vaults sync on open, on a 15-minute timer on desktop, and by the OS background task on mobile (battery and rate friendly).
|
||||
|
||||
## User stories
|
||||
- As an owner, I want to share a folder with a colleague as editor, so that we keep project notes together.
|
||||
- As an editor, I want to see my colleague's cursor and edits appear as they type, so that we do not overwrite each other.
|
||||
- As a viewer, I want read-only access, so that I cannot break the owner's notes.
|
||||
- As an owner, I want to see members, change roles and remove someone, so that access stays under my control.
|
||||
- As anyone, I want an edit made offline to merge when I reconnect, so that live mode never loses work.
|
||||
|
||||
## Acceptance Criteria
|
||||
- [ ] Given two clients in one room, when both type at the same time (property test with random interleavings), then after all messages are delivered both documents are identical and no insert is lost.
|
||||
- [ ] Given a client typing during a 300 ms network delay, then its own edits show at once and the remote edits transform around them (cursor stays on the same word).
|
||||
- [ ] Given a viewer token, when it calls `PUT`, `DELETE`, `join` for editing or `ops`, then the server answers 403 and the audit log records nothing secret.
|
||||
- [ ] Given an invite code used twice or after 48 h, then the second use answers 401; 5 failures lock invites for 10 minutes.
|
||||
- [ ] Given the last peer leaves, then within 2 s the note file on the server equals the room text and other devices receive it through file sync.
|
||||
- [ ] Given a removed member, then their token gets 403 on that vault immediately and their local mount keeps its files but stops syncing ("removed from shared folder").
|
||||
- [ ] Given a user with 12 vaults on one server, when a device pairs once, then `GET /v1/vaults` lists exactly those 12 with roles and no other user's vault, and one token reads and writes each according to the role.
|
||||
- [ ] Given two local vaults bound to different server vaults, when both are opened in turn, then each syncs only with its own server vault and its own `.notescraft/sync/` state.
|
||||
- [ ] Given more than 64 vaults active over time, then the server stays under its ledger cache cap and every vault still answers correctly after eviction.
|
||||
- [ ] Given an E2EE vault, then the editor offers no live session and says why.
|
||||
- [ ] Given the server restarts mid-session, then clients rejoin, resend their unacknowledged op once, and end identical.
|
||||
|
||||
## Constraints
|
||||
- Stack: Rust only; egui UI; server in `apps/notescraft-sync-server`.
|
||||
- Performance: op round trip under 200 ms on LAN; room history capped (1000 ops, then snapshot); room memory capped per document (note size cap 2 MiB for live mode).
|
||||
- Security: roles enforced on the server; presence names are display text only (sanitised, max 40 chars, no markup); no tokens or codes in logs or audit.
|
||||
- Compat: plain `.md`; Obsidian-compatible vault unchanged.
|
||||
|
||||
## Open Questions
|
||||
1. Owner approval for a CRDT crate instead of in-house OT? Recommendation: no, build OT first (no dependency, matches plain files).
|
||||
2. Per-folder sharing inside one vault (instead of mounting a separate shared vault)? Recommendation: v2.
|
||||
3. Public read-only links and comments: later; ranked in `research/`.
|
||||
@@ -0,0 +1,13 @@
|
||||
# Tasks: Collaboration
|
||||
|
||||
Agent briefs in `scripts/mm/`. `[ ]` open.
|
||||
|
||||
- [x] 1. OT core: `TextOp`, apply/compose/transform/invert, cursor transform, client state machine, server `Room`. Pure crate `crates/collab`. Brief `scripts/mm/collab-ot.md`.
|
||||
Done when: convergence property tests (random op pairs and random client interleavings) pass; crate clippy clean. (done 2026-10-20 by GLM 5.3 Flash via Pi; orchestrator fixed 5 clippy lints and removed a stray probe test; 22 tests)
|
||||
- [x] 2. Accounts, many vaults, membership, roles, invites on the sync server (`users`, `members`, `invites`, account-level device tokens, `GET/POST/PATCH/DELETE /v1/vaults`, per-vault role checks on every endpoint, ledger LRU cache, audit, lockout). Brief `scripts/mm/accounts-sharing.md`.
|
||||
Done when: HTTP tests for each role and for invite use/expiry/lockout pass. (done 2026-10-20 by GLM 5.3 Flash via Pi until its provider quota ended; orchestrator fixed 7 clippy lints, a missing assertion in `lifecycle.rs`, the read half of the role matrix and two outdated expectations; 40 server tests)
|
||||
- [x] 3. Live endpoints: `join`, `ops`, `events` (SSE), room registry, idle flush into the ledger, presence. Needs 1 and 2. (done 2026-10-20 by MiniMax M3.1 Flash through the Claude Code harness: `live/` module, 14 HTTP tests incl. two-client convergence over SSE; verified by the orchestrator. Rooms are in memory, flush after 2 s idle through the ledger, conflicts become `(conflict live ...)` copies.)
|
||||
- [~] 4. (live session client DONE 2026-10-20 by MiniMax M3.1 Flash: `crates/engine/src/live/`, 8 stub-server tests and a two-client e2e against the real server, verified; `share.*` and `server.*` commands DONE 2026-10-20 by MiniMax M3.1 Flash: `crates/engine/src/share/`, 9 stub tests plus 2 real-server e2e tests, verified; `live.*` commands still open) Engine: per-vault `binding.json`, keychain entry per account, `vault.server_list`/`connect`/`create_remote` commands; live session client (SSE reader thread, OT client, apply remote ops to the open buffer), `share.*` commands (invite, accept, members, mount), mount-aware sync state.
|
||||
- [ ] 5. UI: Vaults screen and vault switcher (design 14), Share dialog, members list, presence avatars and remote cursors in Source and Visual views, shared folder badge in the Explorer, "removed from shared folder" banner (designs in PaperCraft page "Collaboration").
|
||||
- [ ] 6. Deploy with the sync server (needs owner approval); load test with 10 simulated peers.
|
||||
- [ ] 7. Later: comments, per-folder sharing, public read-only links.
|
||||
@@ -0,0 +1,45 @@
|
||||
# End-to-end encryption for synced vaults
|
||||
|
||||
Source: `specs/mobile-and-sync/research/security-and-server.md` (findings on Obsidian Sync and Syncthing, recommendations 1 to 3). **Needs owner approval for new crates** before any code (see Open Questions).
|
||||
|
||||
## Problem
|
||||
The sync server (a homelab, possibly shared with other people through invites) stores every note and every path in plaintext (SQLite `files.path`, blob files hold note text). Anyone with the database or disk can read all notes. Obsidian Sync encrypts end to end by default; NotesCraft does not.
|
||||
|
||||
## Goal
|
||||
A vault can be created or converted to "end-to-end encrypted": the server only ever holds ciphertext and opaque ids, and devices still converge.
|
||||
|
||||
## Non-Goals
|
||||
- Server-side search, transcription, OCR or publish of encrypted vaults (impossible by design; the UI says so).
|
||||
- Live co-editing in an encrypted vault (the server must read operations to order them): file sync only, as stated in `specs/collaboration/spec.md`.
|
||||
- Protecting against a hostile client or malware on a paired device; hiding file sizes and timing (documented leak).
|
||||
|
||||
## Design
|
||||
1. **Key hierarchy.** A random 256-bit **vault key** (VK) per vault. VK is wrapped (a) by a key derived from the owner's passphrase with **Argon2id** (memory 64 MiB, 3 passes, 1 lane as a floor, parameters stored with the wrapped key), and (b) by a printed/saved **recovery key** (random 256 bits shown as 24 words or base32). Each device receives VK wrapped for its own **device key** when paired (QR carries the pairing code; the passphrase is typed once on the first device and never leaves it). Subkeys by HKDF-SHA256 with labels `content`, `path`, `manifest`.
|
||||
2. **Content.** Each file is encrypted client-side with **XChaCha20-Poly1305** (random 192-bit nonce per blob, associated data = vault id + blob id + format version), in chunks of 1 MiB for large files so a bit flip or truncation is detected (chunk index and "last chunk" flag in the associated data).
|
||||
3. **Paths.** The server never sees real paths: it stores an **opaque blob id** (HMAC-SHA256 of the path with the `path` subkey, hex) per file and an encrypted **manifest record** `{path, size, mtime}` sealed with the `manifest` subkey and bound to the opaque id (so replaying one record under another id fails). The `/changes` records carry `{opaque id, sealed manifest, seq, deleted}`; clients decrypt to learn the path. Ledger rules (revisions, conflicts, tombstones) work unchanged on opaque ids.
|
||||
4. **Hashes.** The plaintext SHA-256 is part of the sealed manifest, not a server field, so equal notes do not leak equality (documented: no cross-file dedup in encrypted vaults).
|
||||
5. **Pairing a new device.** The owner shows a QR (server, vault id, one-time code); the new device pairs, receives the account token; VK is delivered either by typing the passphrase on the new device or by scanning a second QR from a trusted device (wrapped VK + nonce, valid 5 minutes). Recovery: passphrase or recovery key re-derives VK and unwraps the wrapped copy the server stores.
|
||||
6. **Conversion.** Plain to encrypted: a client re-uploads every file as ciphertext, then the owner confirms and the server deletes old plaintext blobs and rewrites the path column to opaque ids. Encrypted to plain is not offered in v1 (export instead).
|
||||
7. **Zeroize and memory.** Keys live in `Zeroizing` buffers, never in logs or settings; the passphrase is not stored; the keychain holds the device key.
|
||||
|
||||
## Acceptance Criteria
|
||||
- [ ] Given an encrypted vault synced from two devices, when the server's database and blob folder are captured, then no note text, path or file name appears in any byte of them (string search of 1,000 random words from the notes finds nothing).
|
||||
- [ ] Given a blob whose ciphertext byte is flipped, a chunk dropped, or a sealed manifest swapped between ids, then the client refuses it with a clear error and writes nothing.
|
||||
- [ ] Given a lost device, when the owner revokes it and rotates VK, then the revoked token gets 401 and a copy of the old wrapped VK no longer opens new data.
|
||||
- [ ] Given the passphrase is forgotten and the recovery key is present, then a clean device restores the vault; with neither, the data is unrecoverable and the UI said so at creation.
|
||||
- [ ] Given 10,000 notes, then initial encrypted sync is within 3x of the plaintext time on the same LAN.
|
||||
|
||||
## Constraints
|
||||
- Rust, no JavaScript. Crates need written approval: `chacha20poly1305` (audited, 0.11.0 at research time), `argon2` (0.6.0), `hkdf`, `hmac` (RustCrypto), `zeroize`, `rand` (OS RNG via `getrandom`). `sha2` is already approved.
|
||||
- Wire format versioned; a crypto test vector file lives in the repo and is checked in CI.
|
||||
|
||||
## Open Questions
|
||||
1. Approve the crates above? (Recommended: yes, all RustCrypto.)
|
||||
2. Is a hybrid of server-readable "shared vault" and E2EE "private vault" acceptable (live editing and publish only in the readable one)? Recommended: yes, chosen per vault at creation.
|
||||
3. Recovery key as words (needs a 2048-word list file, MIT/CC0) or base32? Recommended: base32 first.
|
||||
|
||||
## Tasks
|
||||
1. New crate `crates/sync-crypto` (L1, pure): key hierarchy, chunked AEAD, opaque ids, sealed manifest, test vectors, property tests (flip, truncate, reorder).
|
||||
2. Server: opaque-id mode per vault (`encrypted` flag), no plaintext path column, size caps by ciphertext.
|
||||
3. Engine: encrypting `Remote` wrapper around `HttpRemote` (path mapping, chunking), key storage in the OS keychain, conversion command.
|
||||
4. UI: create encrypted vault, passphrase and recovery key screens, device pairing QR, lock indicators.
|
||||
@@ -0,0 +1,33 @@
|
||||
# Mobile and sync: designs
|
||||
|
||||
Made in PaperCraft on 2026-10-20, in the project `NotesCraft` (`~/Documents/paper-rs/NotesCraft`), page **Mobile and sync**, using the project's own variables (`bg`, `surface`, `surface-2`, `canvas`, `accent`, ...) in both modes (Dark, Light). Phone artboards are 390 x 844, the desktop one 1280 x 960. Exports: `design/mobile/*.png` (dark) and `*-light.png` at 2x. Lint: 0 errors on all nine; the two warnings on the desktop page are 40 pt buttons, accepted for dense desktop UI.
|
||||
|
||||
Direction: calm and quiet, like paper. One accent (mauve) for the primary action and the active tab, one semantic set (green synced, yellow attention, red failure). Everything sits on the 4 pt grid; touch targets are 44 pt or more; the home-indicator area is 32 pt.
|
||||
|
||||
| # | Screen | File | What to build |
|
||||
|---|---|---|---|
|
||||
| 1 | Notes | `design/mobile/01-notes.png` | Title, sync pill (dot + "Synced - 2 min ago"), search field, note cards (title, two-line preview, folder and date, conflict badge), bottom tab bar Notes / Search / Graph / Calendar / More. Tab bar replaces the desktop sidebar below 600 pt. |
|
||||
| 2 | Note editor | `design/mobile/02-note-editor.png` | Top bar (back, title, sync pill, menu), content, keyboard toolbar of 44 pt buttons (bold, italic, link, list, task, tag, indent, hide keyboard) pinned above the soft keyboard. |
|
||||
| 3 | Sync sheet | `design/mobile/03-sync-sheet.png` | Bottom sheet from the sync pill: status headline, device list with state (online, synced, stale), "2 conflicts to review" row, toggles (Wi-Fi only, attachments, settings and themes), primary "Sync now". |
|
||||
| 4 | Pair device | `design/mobile/04-pair-device.png` | Server URL, pairing code field (12 hex, grouped), "Scan QR code instead", primary "Pair this device", note that the code works once and expires in 10 minutes. |
|
||||
| 5 | Conflict | `design/mobile/05-conflict.png` | File name, plain-language line ("Nothing was lost"), both versions as cards with the differing line highlighted, "Keep both" (primary), "Keep mine", "Keep theirs". |
|
||||
| 6 | First sync | `06-first-sync.png` | Progress for the first download: count, percent, attachments note, "Open notes so far", Pause. |
|
||||
| 7 | Offline | `07-offline.png` | Offline banner, pill "Offline - 3 changes waiting", per-note "Waiting to sync" badge. |
|
||||
| 8 | Sign in again | `08-sign-in-again.png` | Revoked or expired token: reassurance that nothing was deleted, Pair again, Keep working offline. |
|
||||
| 9 | Desktop Settings > Sync | `09-desktop-settings-sync.png` | Server card, device list with Revoke, options, Add a device. |
|
||||
|
||||
States still to design: empty vault, storage-full, E2EE passphrase prompt, tablet two-pane. The artboards were built with raw frames, not yet with the project's Design System components (`Button / Primary`, `Chip / *`, `Toggle / *`): swap them in with `import_components` + `create_instance` when polishing.
|
||||
|
||||
Mapping to desktop: the same screens exist as Settings > Sync (pairing, devices, options), a status-bar indicator (the sync pill) and a Conflicts tab. Strings go through `crates/i18n` as always.
|
||||
|
||||
## Collaboration and launcher designs (PaperCraft project NotesCraft)
|
||||
Pages **Collaboration** and **Home and vaults**; exports in `design/mobile/`. Specs: `specs/collaboration/spec.md`, `specs/vault-launcher/spec.md`.
|
||||
|
||||
| # | Screen | File |
|
||||
|---|---|---|
|
||||
| 10 | Share dialog (members with roles, one-time invite code, Stop sharing) | `10-share-dialog.png` |
|
||||
| 11 | Live editing: avatars of peers, "Live" pill, remote cursor flags and selection | `11-live-editing.png` |
|
||||
| 12 | Join a shared folder (code, preview of what you get, role) | `12-join-shared-folder.png` |
|
||||
| 13 | Shared folders in the phone Explorer, removed-from-folder banner | `13-shared-folders.png` |
|
||||
| 14 | Home / Vaults (desktop launcher): actions, recents with sync pill, role, avatars, missing folder | `14-home-vaults-desktop.png` |
|
||||
| 15 | Vaults (phone): on this phone, on your server, New vault, Join | `15-vaults-phone.png` |
|
||||
@@ -0,0 +1,35 @@
|
||||
# Plan: Mobile and sync
|
||||
|
||||
## Approach
|
||||
Files stay the truth; a small HTTP service keeps a per-vault revision ledger and content-addressed blobs; every device runs the same client loop (`scan, fetch changes, plan, act`). Pure decisions live in `crates/sync` (done); the network and disk driver lives in `engine` so desktop and phone share it; the phone is the same egui app in a second shell crate with a compact layout. Chosen over git-based sync (no git on phones, merge UX poor) and over a CRDT (plain `.md` interop with Obsidian would break).
|
||||
|
||||
## Architecture
|
||||
- Data flow: device edit -> vault file -> `sync_once` (scan, `GET /changes`, `plan`, `PUT/GET/DELETE`) -> server `Ledger` -> SQLite + blob file -> long-poll wakes the other devices.
|
||||
- New modules: `crates/sync` (done), `apps/notescraft-sync-server` (done, hardening in task 3), `crates/engine/src/sync/` (task 4), engine `sync.*` commands and worker (task 5), `ui-egui` sync views (task 6), compact layout (task 7), `apps/notescraft-mobile` (task 8).
|
||||
- Modified: `engine` (module, commands, settings), `ui-egui` (Settings > Sync, status bar pill, Conflicts view, bottom tab bar, keyboard toolbar), `docs/ARCHITECTURE.md`, `ROADMAP.md`.
|
||||
|
||||
## Files
|
||||
- **Create**: see tasks.
|
||||
- **Modify**: `crates/engine/Cargo.toml` (`notescraft-sync`, `ureq`), `crates/engine/src/lib.rs`, `crates/engine/src/prefs/*` (sync settings), `crates/ui-egui/src/shell/layout.rs` (compact mode).
|
||||
|
||||
## Data
|
||||
- Server: SQLite tables `vaults`, `pairings`, `devices`, `files` (+ `audit`, `updated_ms`, `last_seen_ms` in task 3). Blobs `blobs/<vault>/<sha256>`.
|
||||
- Client: `.notescraft/sync/state.json` (`last_seq`, `bases`), `.notescraft/sync/base/<hash>` (merge ancestors), device token in the OS keychain where available (desktop `keychain.rs`), else `.notescraft/sync/credentials.json` with 0600 permissions on first version.
|
||||
- API: see `apps/notescraft-sync-server/src/routes.rs` and ARCHITECTURE.
|
||||
|
||||
## Dependencies
|
||||
Approved 2026-10-19: `axum`, `tokio`, `ureq` (rustls), `rusqlite`, `sha2`. Needs approval before use: `chacha20poly1305`, `argon2`, `zeroize`, `rand` (E2EE, task 9); `qrcodegen` (QR, task 6; dormant crate, alternatives noted in research); `tower_governor` only if IP limits become possible (they are not behind the tunnel).
|
||||
|
||||
## Risks
|
||||
- egui on mobile: soft keyboard/IME and safe-area support are the weak spots (see research/mobile-platform.md); mitigate with a spike (task 8) before committing to the full port.
|
||||
- A sync bug can destroy notes: never lose data (conflict copies, `.trash`, base copies); property tests on `crates/sync`; a dry-run mode in the driver before the first real sync.
|
||||
- E2EE changes the server data model (opaque ids); design the blob and record format before shipping v1 to the public, or accept a one-time migration.
|
||||
- Cloudflare Tunnel hides client IPs, so rate limits are global, not per IP.
|
||||
|
||||
## Out of scope (defer)
|
||||
Real-time co-editing, shared multi-user vaults, server-side search, web client, publish, web clipper (ranked in research for later).
|
||||
|
||||
## Test strategy
|
||||
- Unit: `crates/sync` (26), driver with a fake remote (task 4).
|
||||
- Integration: server over real HTTP (6 now, more in task 3); client against the real server in a test binary (task 5).
|
||||
- Manual: two real devices on the homelab after the deploy approval; offline edit and reconnect; airplane-mode drill.
|
||||
@@ -0,0 +1,24 @@
|
||||
# Mobile portability report (2026-10-20)
|
||||
|
||||
Checked on this Mac with `rustup target add aarch64-linux-android` and `cargo check --target aarch64-linux-android -p <crate>` (type-check only; no linker, no NDK).
|
||||
|
||||
| Crate | Android check |
|
||||
|---|---|
|
||||
| `sync`, `collab`, `markdown`, `vault`, `index`, `blocks`, `templates`, `calendar`, `themes`, `i18n`, `editor` | **ok** |
|
||||
| `engine`, `ui-egui` | stop at the `ring` build script (pulled in by `ureq` with rustls): it compiles C and needs the Android NDK's clang. Not a code problem; expected to pass with `ANDROID_NDK_HOME` set and `cargo-ndk`. Not verified beyond that point. |
|
||||
|
||||
So the core (vault, Markdown, index, sync, OT, editor model) is already portable. What is still unverified: `engine` and `ui-egui` past `ring`, and everything iOS (no Xcode here).
|
||||
|
||||
## What this machine lacks (blocks the real shells)
|
||||
- Android: no SDK, no NDK, no `cargo-ndk`, no `adb`, no emulator. Needs Android Studio (or command-line tools), NDK r26+, `cargo install cargo-ndk`, targets `aarch64-linux-android` (added) and `x86_64-linux-android` for the emulator.
|
||||
- iOS: only the Command Line Tools are installed, not Xcode (`xcodebuild` and `simctl` fail). Needs Xcode, the targets `aarch64-apple-ios` and `aarch64-apple-ios-sim`, and an Apple developer account for devices.
|
||||
- Windows target `x86_64-pc-windows-msvc` is installed (cross check only).
|
||||
|
||||
## Desktop-only code to gate before a mobile build (from the code audit)
|
||||
`apps/notescraft` (`main.rs`, `desktop.rs`, `audio.rs` with `cpal`, `menu.rs` with `muda`, `notify.rs`, `keychain.rs`, `net.rs`, `google.rs`, `launcher_window.rs` which spawns processes), `rfd` file dialogs, `notify` watcher on iOS (own container only: poll instead), every `Command::new` user: `crates/git`, `crates/acp`, `crates/syntax/src/fetch.rs`, `crates/engine/src/calendar/clock.rs`, `crates/engine/src/vault/pdf_preview.rs`, `ui-egui` `perf.rs`, `os_scale.rs`, `video.rs`, `player.rs`. Plan: `cfg(not(any(target_os = "android", target_os = "ios")))` on those features; commands report "not available on this device" through `enabled`.
|
||||
|
||||
## Next steps once the toolchains exist
|
||||
1. `cargo ndk -t arm64-v8a check -p notescraft-engine -p notescraft-ui-egui` and fix what shows.
|
||||
2. `apps/notescraft-mobile` with `eframe` + `android-activity` (GameActivity), a Gradle project, one screen (the launcher) running on an emulator.
|
||||
3. iOS static library + Xcode project, simulator first.
|
||||
4. Soft keyboard and IME spike in the Source editor (see `research/mobile-platform.md` finding on egui IME).
|
||||
@@ -0,0 +1,296 @@
|
||||
# Mobile platforms for NotesCraft: Android and iOS with Rust + egui/eframe
|
||||
|
||||
## Method
|
||||
|
||||
**Browsed.** Web search and page fetch were used for every claim below; each finding carries its source URL and the year of access (2026). Facts were read from upstream sources directly: crates.io / docs.rs crate metadata and changelogs, the `emilk/egui` and `rust-windowing/winit` repositories and issue trackers, `developer.android.com`, `developer.apple.com`, `help.obsidian.md`, and the `obsidian.md` changelog. Where a version number, date, API name or policy could not be confirmed from a primary source it is written **"unverified"** — nothing here is guessed.
|
||||
|
||||
Partial-browse note: findings 1–8 below were verified online; findings 9–16 and everything after them were written after the browse quota was exhausted and are **from memory** — see "Method (continued)".
|
||||
|
||||
Scope note: NotesCraft currently pins `egui`/`eframe` **0.36.2** in `Cargo.lock` (workspace declares `"0.36"`), while upstream is at **0.37.0** (published 2026-10-15). Facts are tagged with the version they were verified against; the 0.37.0-only items are called out.
|
||||
|
||||
---
|
||||
|
||||
## Findings
|
||||
|
||||
### 1. egui/eframe on Android — supported, with one official example
|
||||
|
||||
- `eframe` exposes two Android backends as Cargo features: `android-game-activity` and `android-native-activity`, each forwarding to the matching `egui-winit`/`winit` feature. `egui-winit` 0.37.0 lists `android-game-activity -> winit/android-game-activity` and `android-native-activity -> winit/android-native-activity`. <https://crates.io/crates/eframe> · <https://crates.io/crates/egui-winit> (accessed 2026).
|
||||
- On Android, `eframe::NativeOptions` **requires** an `android_app: Option<winit::platform::android::activity::AndroidApp>` field — it is `None` by default and `eframe` errors out with "`NativeOptions` is missing required `android_app`" if it is not filled in. The app must therefore own the `android_main(app: AndroidApp)` entry point and pass the app through. <https://github.com/emilk/egui/blob/main/crates/eframe/src/epi.rs> · <https://docs.rs/android-activity/latest/android_activity/> (accessed 2026).
|
||||
- `android-activity` provides the `#[no_mangle]` entry symbols itself (a default `android_main` in `native_activity/glue.rs` and `game_activity/mod.rs` that hands control to the app's `android_main`), and its glue version must match winit's: **winit 0.30 → `android-activity = "0.6"`**. <https://docs.rs/android-activity/latest/android_activity/> · <https://docs.rs/winit/latest/winit/platform/android/> (accessed 2026).
|
||||
- egui ships exactly **one** mobile example: `examples/hello_android` — `eframe` + `android-native-activity` + the **`glow`** renderer, `crate-type = ["cdylib", "lib"]`, `build_targets = ["armv7-linux-androideabi", "aarch64-linux-android"]`, `min_sdk_version = 23`, `target_sdk_version = 35`, built with `cargo apk run -p hello_android --lib`. Its README pins **NDK 29.0.14206865**, build-tools **36.0.0**, platform **android-35**, and installs a *patched fork* of cargo-apk (`parasyte/cargo-apk` rev `2826395…`) because of an upstream `cargo-subcommand` bug. <https://github.com/emilk/egui/tree/main/examples/hello_android> · <https://github.com/emilk/egui/blob/main/examples/hello_android/README.md> (accessed 2026).
|
||||
- Android build tooling (all verified on crates.io / GitHub, accessed 2026):
|
||||
- **cargo-ndk 4.1.2** (2025-08-09, "actively-developed") — a `cargo` passthrough that exports NDK clang/linker env vars and emits `jniLibs/<abi>/lib*.so`; hard-fails below **NDK r23**; does **not** package an APK. <https://crates.io/crates/cargo-ndk> · <https://github.com/bbqsrc/cargo-ndk>
|
||||
- **cargo-apk 0.10.0** (2023-11-30) — the tool egui's example uses; repo still has commits but "Revive cargo-apk!" (#66) is open; it hardcodes `android.app.NativeActivity`, i.e. **NativeActivity-only, no GameActivity**. <https://crates.io/crates/cargo-apk> · <https://github.com/rust-mobile/cargo-apk/issues/66>
|
||||
- **cargo-xbuild 0.6.6** (2022-06-21) and **xbuild 0.2.0** (2022-12-21) — effectively superseded; `cargo-xbuild` now points at cargo's own `-Z build-std`. <https://crates.io/crates/cargo-xbuild> · <https://crates.io/crates/xbuild>
|
||||
- Google recommends **GameActivity** over NativeActivity for new C/C++ apps; GameActivity derives from `AndroidAppCompat`, supports the Jetpack text-input stack, and since 1.2.2 ships a static library in the AAR/prefab package. <https://developer.android.com/games/agdk/game-activity> (accessed 2026).
|
||||
- 16 KB page-size devices require **NDK r28+ and AGP 8.5.1+**; older NDKs need explicit `-Wl,-z,max-page-size=16384`. <https://developer.android.com/guide/practices/page-sizes> (accessed 2026).
|
||||
- Android-specific egui features already in tree: hardware **back button** support landed in egui **0.32.0** (PR #7073); Android text input was fixed in egui PR #5759 (merged 2025-03-21) and again in winit 0.30.13, which "populate[s] `KeyEvent::text` … via `Key::to_text()`" on Android. <https://github.com/emilk/egui/blob/main/crates/egui-winit/CHANGELOG.md> · <https://docs.rs/winit/latest/winit/changelog/index.html> (accessed 2026).
|
||||
|
||||
### 2. egui/eframe on iOS — technically possible, **not** an officially supported target
|
||||
|
||||
- `eframe` 0.37.0's README lists native targets as "Linux, Mac, Windows, and **Android**" — iOS is absent, there is **no `hello_ios` example**, and no iOS CI job. <https://github.com/emilk/egui/blob/0.37.0/crates/eframe/README.md> · <https://github.com/emilk/egui/tree/main/examples> (accessed 2026).
|
||||
- Issue **#3117 "Add iOS native target"** is still **open** (opened 2023-06-27, last activity 2026-03-18). The proposed separate `egui-ios` Swift-FFI backend (PR #7768) was **closed**; maintainer lucasmerlin (2026-01-13): "You can run egui on iOS via eframe and winit, there is no need for a separate ios backend." <https://github.com/emilk/egui/issues/3117> · <https://github.com/emilk/egui/pull/7768> (accessed 2026).
|
||||
- The iOS path in eframe 0.37.0: `run_native` owns the event loop via `with_event_loop`, which is `#[cfg(not(target_os = "ios"))]`. On iOS the supported entry is `eframe::create_native(app_name, options, app_creator, &event_loop) -> EframeWinitApplication`, and `EframeWinitApplication` implements `winit::application::ApplicationHandler` (so the host drives it). Verified by reading eframe 0.37.0 sources. <https://github.com/emilk/egui/blob/main/crates/eframe/src/lib.rs> (accessed 2026).
|
||||
- winit **does** ship a full UIKit backend in 0.30 (`src/platform_impl/ios/`, ~3,300 lines; `WindowExtIOS`, `EventLoopExtIOS`). In winit 0.31 the backend was extracted to a separate crate `winit-uikit` (0.31.0-beta.3, 2026-09-04) and iOS is listed as a **Tier 2** target ("guaranteed to build", not "guaranteed to work"). <https://crates.io/crates/winit-uikit> · <https://github.com/rust-windowing/winit> (accessed 2026).
|
||||
- The canonical winit iOS integration is: build a **static library**, export `#[no_mangle] pub extern fn start_winit_app()`, add `winit.h` to the Xcode project, call `start_winit_app()` from the Xcode main function, and **create windows inside `ApplicationHandler::resumed`** (iOS's `UIApplicationMain` must do its init first). <https://docs.rs/winit/latest/winit/platform/ios/index.html> (accessed 2026).
|
||||
- iOS tooling landscape (verified, accessed 2026):
|
||||
- **cargo-mobile2 0.22.5** (2026-08-17, tauri-apps) — the live iOS project generator (`cargo mobile init`, `cargo apple open`); its README states support is limited to Tauri's use case, and its **egui template is stale (egui 0.22 / winit 0.28)**. <https://crates.io/crates/cargo-mobile2> · <https://github.com/tauri-apps/cargo-mobile2>
|
||||
- The original **cargo-mobile** (BrainiumLLC) no longer exists on crates.io (404 as of 2026-10); its README still mentions Rust 1.46–1.48. <https://github.com/BrainiumLLC/cargo-mobile>
|
||||
- **cargo-xcode 1.11.1** (2026-06-21) — actively maintained but **moved to GitLab** (`gitlab.com/kornelski/cargo-xcode`; the old `github.com/jaid/cargo-xcode` 404s). Generates a *nestable* `.xcodeproj`, requires `crate-type = ["lib","staticlib"]` ("cdylib is NOT recommended"), and produces **no bridging headers** — you write them or use cbindgen/uniffi. <https://crates.io/crates/cargo-xcode> · <https://gitlab.com/kornelski/cargo-xcode>
|
||||
- **cargo-dinghy 0.8.6** (2026-08-19) is the tool rustc's own platform-support doc points to for running/testing on iOS. <https://crates.io/crates/cargo-dinghy>
|
||||
- **cargo-lipo 3.3.1** (2024-05-27) is self-described as deprecated/passively maintained; **cargo-bundle 0.12.0** (2026-09-20) builds the `.app` bundle and can package an `.ipa`. <https://crates.io/crates/cargo-lipo> · <https://crates.io/crates/cargo-bundle>
|
||||
- Known iOS crash: egui issue **#8052 "iOS crash at startup with v0.34.1"** is **open** — the break was traced to the wgpu 29 update (0.33.3 worked). <https://github.com/emilk/egui/issues/8052> (accessed 2026).
|
||||
- Build hazard: `egui-winit`'s `clipboard` feature pulls in **arboard**, which has no iOS backend. egui-winit 0.33.2 stopped *enabling* arboard on iOS (#7663) but the dependency graph must still be checked; on iOS, build with `default-features = false` and re-enable only what is needed (`wgpu`, `default_fonts`, `links`, `accesskit`). <https://github.com/emilk/egui/blob/main/crates/egui-winit/src/clipboard.rs> · <https://github.com/emilk/egui/blob/main/crates/egui-winit/CHANGELOG.md> (accessed 2026).
|
||||
|
||||
### 3. Soft keyboard and IME — the sharpest edge for egui on both platforms
|
||||
|
||||
- The plumbing exists: `egui` exposes `Event::Ime(ImeEvent)`, `egui-winit` forwards winit's `WindowEvent::Ime`, and winit 0.30.6 added `Window::set_ime_allowed` on **both** Android ("the soft keyboard can now be shown using `Window::set_ime_allowed`") and iOS ("Add basic iOS IME support"). <https://docs.rs/winit/latest/winit/changelog/index.html> (accessed 2026).
|
||||
- **Android**: winit maps `set_ime_allowed(true)` to `app.show_soft_input(true)`, but **`set_ime_cursor_area` and `set_ime_purpose` are no-ops on Android**, so the IME candidate window is never anchored to the caret. There is an open egui bug: **#5590 "Keyboard not opening under Android"** (reported with `NativeActivity` on Pixel 7 Pro / LineageOS 22.1 / Android 15, 2025-01-07) — long-press copy/paste did not work either. <https://github.com/emilk/egui/issues/5590> · <https://github.com/rust-windowing/winit> (accessed 2026).
|
||||
- **iOS**: winit 0.30's UIKit view implements **`UIKeyInput`** only (`hasText`, `insertText:`, `deleteBackward`) and emits **one `KeyboardInput` event per character — no preedit/marked-text**, so there is **no native IME candidate window** on iOS. Apple's docs are explicit that `UIKeyInput` offers "only a small subset of the available keyboards and languages"; full behaviour (marked text, dictation, autocorrect, caret rects) needs `UITextInput`. The robust workaround is an **off-screen `UITextView`/`UITextField` in the Swift shim** that forwards text into Rust. <https://github.com/rust-windowing/winit/blob/v0.30.13/src/platform_impl/ios/view.rs> · <https://developer.apple.com/documentation/uikit/uikeyinput> · <https://developer.apple.com/documentation/uikit/uitextinput> (accessed 2026).
|
||||
- Beware a misleading headline: egui **0.36.0**'s "Improved mobile keyboard support — autocomplete, autocorrect and IMEs now work correctly on iOS and android" applies to **eframe web** (PR #8045), *not* to native mobile. Native IME work landed separately in 0.34.0/0.35.0 ("Much improved IME" #7967, "Implement proper visuals for IME composition" #8083). <https://github.com/emilk/egui/releases/tag/0.36.0> (accessed 2026).
|
||||
- Android keyboard text was fixed twice: egui PR #5759 (2025-03-21, "winit does not fill in the `text` field of the `KeyEvent`") and winit 0.30.13 (fills `KeyEvent::text` from `Key::to_text()`). <https://github.com/emilk/egui/pull/5759> (accessed 2026).
|
||||
- egui's own long-press support is separate from the OS keyboard: `InputOptions::max_click_duration` (default **0.8 s**) turns a held touch into a secondary click (context menu). <https://docs.rs/egui/latest/src/egui/input_state/mod.rs.html> (accessed 2026).
|
||||
|
||||
### 4. Safe area, rotation, touch and multi-touch
|
||||
|
||||
- **Safe area is already solved for iOS in egui.** egui **0.33.0** added `RawInput::safe_area_insets: Option<SafeAreaInsets>`, `InputState::content_rect()` (viewport minus insets) and `InputState::viewport_rect()` (the full screen, e.g. behind the Dynamic Island), deprecating `screen_rect`. PR #7578 (merged 2025-10-07) implements it by reading `UIWindow.safeAreaInsets` through `objc2-ui-kit`; egui-winit has a dedicated `src/safe_area.rs`. The inset values only refresh on `Resized` / `ScaleFactorChanged` / `Focused(true)` / `Occluded(false)`; egui 0.37.0 added a fix to preserve them when input is appended (#8484). <https://github.com/emilk/egui/pull/7578> · <https://github.com/emilk/egui/blob/main/crates/egui-winit/src/safe_area.rs> (accessed 2026).
|
||||
- **There is no Android equivalent inside egui.** The Android path is to read the window insets yourself in the Java/Kotlin layer (`WindowCompat.setDecorFitsSystemWindows(false)` + `LAYOUT_IN_DISPLAY_CUTOUT_MODE_ALWAYS`) and feed them in, or simply draw behind the system bars and pad manually. <https://github.com/rust-mobile/rust-android-examples/tree/main/agdk-egui> (accessed 2026).
|
||||
- **Rotation is the host's job.** On Android the activity must declare `android:configChanges="orientation|screenSize|screenLayout|keyboardHidden"`, otherwise the system destroys and recreates the Activity on rotation, which sends `MainEvent::Destroy` and calls `android_main` again. On iOS, `WindowExtIOS::set_valid_orientations` picks supported orientations and UIKit surfaces rotation as `Resized`/`ScaleFactorChanged`; egui-winit on iOS always uses `outer_size()` because it "includes the area behind the dynamic island". <https://docs.rs/android-activity/latest/android_activity/> · <https://docs.rs/winit/latest/winit/platform/ios/index.html> (accessed 2026).
|
||||
- **Gestures: no `egui::Gesture` type exists.** Verified by grepping egui 0.37.0: there is no `pub struct Gesture` / `pub enum Gesture`. Multi-touch surfaces as `InputState::multi_touch() -> Option<MultiTouchInfo>` with public fields `num_touches`, `zoom_delta` (proportional pinch), `zoom_delta_2d` (horizontal/vertical/proportional pinch), `rotation_delta` (radians, clockwise), `translation_delta` and `force`; convenience accessors `zoom_delta()`, `zoom_delta_2d()`, `rotation_delta()` prefer real multi-touch over synthetic ctrl-scroll values. Rotation-gesture support for trackpads landed in 0.33.0 (#7453). <https://docs.rs/egui/latest/egui/struct.MultiTouchInfo.html> · <https://docs.rs/egui/latest/egui/struct.InputState.html> (accessed 2026).
|
||||
- egui-winit converts winit `WindowEvent::Touch` into `egui::Event::Touch` and additionally **emulates a mouse pointer from the first finger only** — so a second finger never drives a widget. winit 0.30 added iOS `PinchGesture`, `DoubleTapGesture`, `PanGesture` and `RotationGesture`, with `UIGestureRecognizerDelegate` for fine-grained control. <https://github.com/emilk/egui/blob/main/crates/egui-winit/src/lib.rs> · <https://docs.rs/winit/latest/winit/changelog/index.html> (accessed 2026).
|
||||
- egui 0.37.0 added `egui::OperatingSystem` with `Android` and `IOS` variants, for OS-specific behaviour inside widgets. <https://github.com/emilk/egui/releases/tag/0.37.0> (accessed 2026).
|
||||
|
||||
### 5. File access on Android — SAF vs scoped storage
|
||||
|
||||
- The Storage Access Framework lets the user pick a **tree** (`ACTION_OPEN_DOCUMENT_TREE`); the returned `Uri` grants access and can be persisted with `takePersistableUriPermission`. What SAF does **not** give is a free recursive listing of an arbitrary folder: you must open each sub-folder's tree, so a vault that is a deep tree is painful to sync blindly. <https://developer.android.com/guide/topics/permissions/requesting#open-document-tree> (accessed 2026).
|
||||
- Under **scoped storage** (Android 10 / API 29+), an app sees only its own directories (`filesDir`, `externalFilesDir`, cache) plus anything the user granted through SAF. Broad "All files access" needs `MANAGE_EXTERNAL_STORAGE`, which is a **special, heavily restricted** permission with its own Play Store declaration and review. <https://developer.android.com/training/data-storage/manage-all-files> (accessed 2026).
|
||||
- Persistence caveat: a SAF grant survives app restarts and (per the grant model) the persisted URI permission, but the **folder is bound to the picking app**; a folder opened through SAF is not a plain filesystem path, so Rust's `std::fs` cannot walk it directly — you need the `DocumentFile`/`ContentResolver` API or a native handle. *(This paragraph is a reading of the SAF model; the exact reinstall behaviour is **unverified** in this session.)*
|
||||
- Practical consequence for NotesCraft: **v1 should keep the vault inside the app sandbox** (Android `filesDir`, iOS `Documents`) and offer SAF/Files-app import as a later, explicitly opt-in step. This matches the existing spec decision ("vault inside app sandbox … External folder access later via SAF / security-scoped bookmarks").
|
||||
|
||||
### 6. File access on iOS — Files app and security-scoped bookmarks
|
||||
|
||||
- `UIFileSharingEnabled` (Info.plist) exposes the app's **`Documents` folder** in the Files app; the default is `NO`. `LSSupportsOpeningDocumentsInPlace` marks the app as able to edit documents in place (the Apple reference page for that key carries no prose — **unverified** beyond the key name/type). <https://developer.apple.com/documentation/bundleresources/information-property-list/uifilesharingenabled> · <https://developer.apple.com/documentation/bundleresources/information-property-list/lssupportsopeningdocumentsinplace> (accessed 2026).
|
||||
- Since iOS 13 a `UIDocumentPickerViewController` can pick a **directory** (the `folder` UTType). The returned URL is **security-scoped**: it "permits your app to access content outside its container… recursively access the directory and all of its contents", and you can persist it with `bookmarkData(options:includingResourceValuesForKeys:relativeTo:)` using the `withSecurityScope` option (save it as a `minimalBookmark` for directories). <https://developer.apple.com/documentation/uikit/providing-access-to-directories> · <https://developer.apple.com/documentation/uikit/uidocumentpickerviewcontroller> (accessed 2026).
|
||||
- Every access must be bracketed: `startAccessingSecurityScopedResource()` / `stopAccessingSecurityScopedResource()` must balance, and **the user can revoke access at any time** in *Settings → Privacy → Files and Folders*, so every access must be failure-tolerant. Apple also says to use `NSFileCoordinator` and `NSFilePresenter` for external documents. <https://developer.apple.com/documentation/foundation/url/startaccessingsecurityscopedresource()> (accessed 2026).
|
||||
- App Store guideline **2.5.2**: "Apps should be self-contained in their bundles, and may not read or write data outside the designated container area" — the sanctioned outside-container path is precisely the security-scoped document-picker grant. <https://developer.apple.com/app-store/review/guidelines/> (accessed 2026).
|
||||
|
||||
### 7. Share-sheet receive — Android intents, iOS share extension
|
||||
|
||||
- **Android**: the app declares an intent filter for `ACTION_SEND` (and `ACTION_SEND_MULTIPLE`) with `text/plain` and `*/*` mime types, and reads the incoming `content://` URI through the `ContentResolver`. Two Rust-side implications: (a) `android-activity`/`winit` do not surface the intent to Rust for you — the Java/Kotlin layer must forward it (a `MainActivity` override of `onNewIntent`/`onCreate` is the usual hook); (b) the shared file arrives as a **content URI, not a path**, so it must be copied into the vault (or streamed) through the resolver. *(The intent-filter mechanics are standard Android; the exact Rust hook points are **unverified** here.)*
|
||||
- **iOS**: a share extension is a **separate executable** (`.appex`) whose Info.plist carries `NSExtensionPointIdentifier = com.apple.share-services`, an `NSExtensionPrincipalClass`, and an `NSExtensionActivationRule` (keys: `NSExtensionActivationSupportsText`, `NSExtensionActivationSupportsFileWithMaxCount`, `…SupportsImageWithMaxCount`, `…SupportsWebURLWithMaxCount`). Narrow activation rules control when the app appears in the share sheet. <https://developer.apple.com/library/archive/documentation/General/Conceptual/ExtensibilityPG/Share.html> · <https://developer.apple.com/documentation/bundleresources/information-property-list/nsextension> (accessed 2026).
|
||||
- The extension and the host app exchange data through an **App Group** (`group.` prefixed container URL / shared `UserDefaults`); the group id must be registered for iOS/iPadOS/tvOS/visionOS/watchOS and can double as a Keychain access group. <https://developer.apple.com/documentation/xcode/configuring-app-groups> (accessed 2026).
|
||||
- Privacy trap: "For each executable or dynamic library in an app that uses a required reason API, the bundle that includes the executable or dynamic library needs to include a privacy manifest file" — the `.appex` needs its **own** `PrivacyInfo.xcprivacy`. <https://developer.apple.com/documentation/bundleresources/describing-use-of-required-reason-api> (accessed 2026).
|
||||
- Prerequisite for both platforms: a NotesCraft command such as `note.new_from_share` must exist in the engine command registry with an `enabled` flag, because on mobile the share action is a command invoked by the OS, not a menu item.
|
||||
|
||||
### 8. Widgets and quick capture
|
||||
|
||||
- iOS **widgets** are SwiftUI/WidgetKit extensions and cannot be rendered by egui — a NotesCraft widget must be native Swift and read from the shared App Group container. Home-screen widgets need iOS 18+ per Obsidian's own requirement (Obsidian's iOS widget documentation). <https://help.obsidian.md/ios> (accessed 2026).
|
||||
- Android **app widgets** are also native (`AppWidgetProvider`); Obsidian ships home-screen widgets (Open Note, New Note, Search, Daily Note) and app shortcuts (long-press the app icon). <https://help.obsidian.md/android> (accessed 2026).
|
||||
- The cheapest and most valuable "quick capture" is **not** a widget: it is an OS-level **share action** (share text → new note) plus an in-app "quick add" that lands the note in a configured folder. Obsidian's own model: new notes go to *Vault folder | Same folder as current file | A specific folder* (Settings → Files and links), and an internal link that carries a folder path (`[[Projects/X]]`) overrides that. <https://help.obsidian.md/settings> (accessed 2026).
|
||||
- egui-side helpers that exist today: `egui::CompletionPopup` (added 0.37.0) gives autocomplete over a `TextEdit`; `egui::Event::PasteImage` (0.37.0) accepts images from the clipboard. Both are usable in a mobile quick-capture sheet. <https://github.com/emilk/egui/releases/tag/0.37.0> (accessed 2026).
|
||||
|
||||
---
|
||||
|
||||
## Method (continued)
|
||||
|
||||
The browse quota for this session was exhausted mid-research: `web_search` and `web_fetch` both return **HTTP 429 (session usage limit)** from the local Ollama web API, and no MCP web server is connected. **Findings 9–16 and everything below them are therefore written from memory** and are flagged as such. No version number, price or API name is invented: where I could not confirm a value I wrote **"unverified"**. The URLs listed for the appended part under *Sources* are the canonical pages to re-verify; they were **not** re-fetched in this session. Findings 1–8 above were verified against primary sources and keep their citations.
|
||||
|
||||
### 9. Biometric app lock — platform code, not egui
|
||||
|
||||
- Android: `androidx.biometric.BiometricPrompt` with `BIOMETRIC_STRONG` (or `DEVICE_CREDENTIAL` as fallback) — *from memory*.
|
||||
- iOS: `LocalAuthentication` — `LAContext.canEvaluatePolicy(.deviceOwnerAuthenticationWithBiometrics, …)` then `evaluatePolicy`; `.deviceOwnerAuthentication` also accepts the device passcode — *from memory*.
|
||||
- egui **cannot** show a system biometric prompt. The prompt lives in the platform layer (`apps/notescraft-mobile`: the Kotlin/Java Activity on Android, the Swift shim on iOS) and hands Rust a boolean. The egui side is trivial — a `TextEdit` with the password flag for a fallback PIN — but it is not an egui problem.
|
||||
- Secret storage is also platform code: Android Keystore / `EncryptedSharedPreferences`, iOS Keychain. NotesCraft's desktop `keychain.rs` is already a per-platform seam; the mobile app wants its own small sibling module, not a new generic trait (ponytail: two thin modules beat one abstraction).
|
||||
- Threat model: a biometric gate protects a *lost device*, not data at rest. The vault stays plain `.md` in the app sandbox. Store copy must not imply "encrypted vault".
|
||||
- **Risk for egui: low.** Only friction: the prompt suspends the egui event loop; resume the shell when the callback returns.
|
||||
|
||||
### 10. Background sync — the OS decides when, so the client must be resumable
|
||||
|
||||
- Android: **WorkManager** for deferrable work. `PeriodicWorkRequest` has a **minimum interval of 15 minutes** (*from memory*; Android also applies its own batching and backoff). Use constraints (`NetworkType.CONNECTED`) — "sync when a network is available", never "poll every minute". Expedited work (`setExpedited`) has a tight quota; reserve it for a user-triggered *sync now*.
|
||||
- Android foreground service: a long upload needs a foreground service with a visible notification and (Android 14+) `foregroundServiceType="dataSync"` plus `FOREGROUND_SERVICE_DATA_SYNC` — *from memory*.
|
||||
- iOS: `BGTaskScheduler` with `BGAppRefreshTask`. The identifier must be listed in `Info.plist` under `BGTaskSchedulerPermittedIdentifiers`, and the handler must be **re-registered on every launch**; a missed registration means no refresh. The system grants a small, non-guaranteed window — commonly quoted at ~30 seconds (*from memory, treat as unverified*). The legacy `UIApplication.setMinimumBackgroundFetchInterval` path is deprecated.
|
||||
- Design consequence: the sync client must be **idempotent and resumable**. `last_seq` lives in `.notescraft/sync/state.json`, so any task may start, upload what it can, and be killed at any point. That is exactly the split already in the spec — pure manifest diff and conflict rules in `crates/sync`, IO driven by the caller.
|
||||
- **Do not keep the long-poll socket alive in the background**: it wakes the radio, drains the battery, and the OS will kill the app regardless. Long-poll only while foregrounded; in the background, one short poll per scheduled window.
|
||||
- The spec already says "Mobile: sync on foreground, on save, and by OS background task (WorkManager / BGAppRefreshTask)". This finding confirms that shape.
|
||||
|
||||
### 11. Notifications — mostly platform code, and mostly unnecessary in v1
|
||||
|
||||
- Android: `NotificationChannel` plus the `POST_NOTIFICATIONS` runtime permission on API 33+ — *from memory*. The permission must be requested in context; asking at first launch without a reason is a review risk.
|
||||
- iOS: `UNUserNotificationCenter.requestAuthorization(options: [.alert, .sound, .badge])` — *from memory*. A denial is effectively sticky for that authorization.
|
||||
- egui has **no** native notification API; the system notification must be posted from the platform layer.
|
||||
- For NotesCraft v1, notifications are about **trust** ("3 notes synced"). An in-app toast in the status bar is enough; system notifications only for a *long, user-initiated* background sync. **Push** (APNs/FCM) would require a hosted push provider — a homelab server is not one, so push is P2.
|
||||
|
||||
### 12. Performance and battery
|
||||
|
||||
- egui is immediate-mode and **repaints on demand**: `eframe` only redraws when there is input unless continuous repaint is requested. Keep continuous repaint **off** on mobile — it is the single biggest battery drain. (*From memory; confirm the exact `eframe`/`eframe::NativeOptions` option name in 0.36/0.37 before relying on it.*)
|
||||
- Renderer choice is the other big lever: `glow` (OpenGL ES) vs `wgpu` (Vulkan on Android, Metal on iOS). Old/odd Android Vulkan drivers are the classic source of blank screens and crashes, and egui's own Android example uses `glow`. On iOS, Metal via wgpu is the better long-term path, but the open iOS startup crash in finding 2 (egui #8052, traced to wgpu 29) says: **pin wgpu and keep a known-good lockfile**.
|
||||
- 16 KB page size (finding 1) is not only packaging: an NDK below r28 emits a `.so` that will not load on 16 KB devices. Ship NDK r28+ or pass `-Wl,-z,max-page-size=16384`.
|
||||
- The real cost on a phone is the **vault index**, not the pixels: an 8,000-note vault must not be indexed on the UI thread at startup. Build it lazily and in chunks, persist it (the `crates/index` plan already assumes this), and show progress instead of blocking the first frame.
|
||||
- File watching does not transfer to mobile: iOS suspends the app and sandbox FSEvents are not a useful trigger; Android inotify on the app's own directory is unreliable once the process is killed. The **OS background task is the trigger**, not the watcher (`crates/vault/src/watch.rs` stays desktop-only).
|
||||
- Traffic shape: long-poll `/changes` in the foreground; one short poll per background window; `/batch/check` (already in the spec) so an unchanged vault costs a single request.
|
||||
|
||||
### 13. Accessibility — the weakest area, treat it as an explicit gap
|
||||
|
||||
- egui draws every widget itself, so there is no platform accessibility tree by default. egui ships an `accesskit` integration (feature `accesskit` on `egui`/`eframe`) that produces a platform accessibility tree on desktop (macOS/Windows/Linux backends). **Whether usable Android/iOS backends exist is unverified** — *from memory*, `accesskit`'s Android support is young and iOS is unconfirmed.
|
||||
- Practical consequence: design the mobile shell for accessibility **even if the screen-reader bridge is not ready yet** — 44 pt minimum touch targets, text that follows the OS font scale, visible focus rings, no colour-only state, sane semantic order, and every gesture has a visible button equivalent.
|
||||
- Do not ship swipe-only navigation (swipe to open the drawer): it is an accessibility failure *and* it collides with the Android/iOS edge-back gesture (finding 4).
|
||||
- VoiceOver/TalkBack testing on a Rust app means checking focus order and labels by hand. Budget it as a P1 item, not a checkbox.
|
||||
|
||||
### 14. Store rules that apply to a Rust app
|
||||
|
||||
- **Play**: new apps and updates must target a recent API level — Android 15 / API 35 became the floor for new apps in Aug 2025; the 2026 floor is **unverified** in this session. Ship an **Android App Bundle** (`.aab`); keep the **upload key** separate from the **app-signing key** and let Play App Signing hold the app key. Apps containing native code must be uploadable for symbolication — upload the native debug symbols for the Rust `.so` alongside the bundle.
|
||||
- Play **Data safety** form: the honest answer for NotesCraft is "no data collected by the developer" (the sync server is the owner's own, outside Google) — but only while the app does not send data to anyone else. A **privacy policy** URL is required; keep it in the repo and host it under owner control.
|
||||
- Android 13+ requires the notification permission to be requested in context; Android 14+ requires a foreground-service type for the sync foreground service.
|
||||
- **App Store**: Guideline **2.5.2** (self-contained container — cited in finding 6), **4.2 Minimum Functionality** (editing a local vault is real functionality; a web wrapper is not), **5.1.1 / 5.1.2** (the App Privacy "nutrition label" must match what actually leaves the device — **self-hosted sync is still transmission**, so the label must mention it and a privacy policy URL is required), **2.3.x** (accurate metadata), **4.7** (mini apps). Generic, but they are the ones a notes app can trip.
|
||||
- iOS **privacy manifest** per executable (finding 7): the Rust static lib does not need one, the app and the share extension each do.
|
||||
- iPad: an iPhone-only app is allowed, but **universal** (iPhone + iPad) is the better default — Obsidian ships universal, and the graph/canvas views benefit from the bigger screen. `winit`/eframe surface iPad split-view resizing as a `Resized` event (*from memory*).
|
||||
|
||||
### 15. Signing and CI
|
||||
|
||||
- **Android**: a debug keystore for development, a separate **upload keystore** for release, both outside Git (and mirrored in the owner's password manager / CI secrets). `cargo-ndk` only produces `jniLibs/` output; it does not build a bundle. The realistic AAB path is a **small hand-written Gradle project** that consumes `cargo ndk -t arm64-v8a -t armeabi-v7a build --release` output copied into `src/main/jniLibs`, then `./gradlew bundleRelease`. `cargo-apk` (finding 1) is stale and NativeActivity-only, so it does not fit a GameActivity + AAB build.
|
||||
- **Android CI**: GitHub Actions (or equivalent) with the NDK installed and pinned, `cargo ndk` for the two ABIs, then Gradle with `bundleRelease`; `zipalign`/`apksigner`/bundletool come from AGP. Keep the release job on a protected environment so the upload key is only exposed there.
|
||||
- **iOS**: needs a Mac with Xcode and an Apple Developer Program membership (annual fee — **amount unverified** here). Signing is Xcode's job: a team, automatic signing, a provisioning profile. For CI the certificate and an App Store Connect API key (`.p8`) go into the runner's keychain; the release path is `xcodebuild -archive` → `.xcarchive` → upload via `xcrun altool` / `xcrun notarytool`. The Rust side produces a **fat static lib** (`lipo`, `cargo-lipo`) that the Xcode project links; `cargo-xcode` generates the project (finding 2) but produces no bridging headers, so the Swift↔Rust bridge is hand-written or generated with cbindgen/uniffi.
|
||||
- Both platforms: pin the Rust toolchain (`rust-toolchain.toml`), the NDK version and the Xcode version in CI. A mobile build that only compiles on one person's Mac is the real delivery risk.
|
||||
- Third-party mobile CI services exist for Rust apps (names and pricing **unverified** in this session) — only worth it if the self-hosted runners become a bottleneck.
|
||||
|
||||
### 16. Obsidian Mobile features worth copying — and what is risky in egui
|
||||
|
||||
Worth copying (Obsidian behaviour, *from memory* of the app and its help site; the widget/app-shortcut items were browsed in finding 8):
|
||||
|
||||
- **Mobile toolbar** — a row of buttons above the keyboard (bold, italic, link, heading, checklist) that scrolls with the note. Copy: a compact egui row pinned above the keyboard, driven by the same command registry as the desktop.
|
||||
- **Quick switcher** — fuzzy search over note titles, opened with one gesture. Copy: the single most-used mobile navigation feature; cheap in egui (`TextEdit` + fuzzy list).
|
||||
- **Quick capture** — a small modal to create a note fast (title, body, target folder). Copy as a bottom sheet.
|
||||
- **Command palette** — the same commands as desktop in one searchable list. Copy: expose the `crates/engine` command registry, do not build a second mobile-only command set (SPEC 10).
|
||||
- **Long-press context menu** on file rows — rename, move, duplicate, delete, copy link.
|
||||
- **Swipe gestures** — swipe the tab bar to switch tabs, edge swipe to open the file drawer. Copy selectively.
|
||||
- **Tabs** — a top tab bar with pinned tabs; Obsidian Mobile keeps desktop-style tabs. Copy: bottom tab bar for the app, note tabs inside the note view.
|
||||
- **Daily note on open** — a community plugin, but the behaviour (open today's daily note on launch) is widely relied on.
|
||||
- **Share sheet receive** — share a page or text into Obsidian and it becomes a note (finding 7).
|
||||
- **Home-screen widgets** (Android: Open Note, New Note, Search, Daily Note) and **app shortcuts** (long-press the app icon) — browsed in finding 8.
|
||||
- **Link autocomplete** while typing (`[[` suggestions); Obsidian's core link suggestion behaviour is *from memory*.
|
||||
- **Bookmarks**, **recent files**, a simplified **graph** on mobile, and **Canvas on mobile** — *from memory*.
|
||||
- **Deep link scheme** (`obsidian://open?vault=…&file=…`) — a `notescraft://` equivalent is cheap and unlocks widgets, app shortcuts and share integrations.
|
||||
|
||||
Risky for egui (this is the honest risk column):
|
||||
|
||||
- **Swipe gestures — high risk.** egui has no gesture abstraction (finding 4): only `InputState::multi_touch()` (pinch/rotate/translate) and single-touch pointer emulation. A horizontal-drag detector must be hand-written, and an **edge swipe is consumed by the OS back gesture** (Android and iOS) before the app sees the touch.
|
||||
- **Long-press menus — medium risk.** egui's secondary click fires at `max_click_duration` (default 0.8 s) and only if the pointer did not move; it competes with the OS text-selection handles and with drag-to-scroll inside a `ScrollArea`.
|
||||
- **Momentum/inertial scrolling — medium-high risk.** egui's `ScrollArea` is immediate-mode with no physics, so a flick will not glide (*from memory; verify before promising a native feel*). This is the single most visible "desktop app on a phone" tell.
|
||||
- **Link autocomplete popup — medium risk.** egui 0.37 added `CompletionPopup`, but the popup must survive the keyboard opening and the viewport shrinking (safe-area + keyboard-aware layout must land first).
|
||||
- **Widgets / app shortcuts — low technical risk, high maintenance cost.** They must be native Swift/Kotlin; they cannot be egui, so each is a second surface to keep in step with the Rust app.
|
||||
- **Pull-to-refresh and swipe row actions — medium risk**, same gesture problem as above.
|
||||
- **Text selection and handles — high risk.** The OS owns selection gestures inside the native text view; an egui-drawn `TextEdit` cannot offer native handles. Combined with the iOS IME gap (finding 3), an egui editor will feel less native than a `UITextView`/`EditText`-backed one. Mitigation for v1: treat the note editor as read-mostly on phones and route heavy editing through the native text view, or accept the compromise explicitly.
|
||||
|
||||
---
|
||||
|
||||
## Feature table
|
||||
|
||||
| Feature | Who has it | Why it matters for NotesCraft | Effort | Priority |
|
||||
|---|---|---|---|---|
|
||||
| Sandbox vault + Files-app visibility (`UIFileSharingEnabled`, Android `filesDir`) | Obsidian Mobile, Apple Notes | Keeps plain `.md` as the source of truth; user can copy files in/out without any permission dance | S | P0 |
|
||||
| External folder via SAF tree (`ACTION_OPEN_DOCUMENT_TREE` + persisted URI) | Obsidian Android, most Android apps | Lets the phone point at an existing folder (e.g. one a file-sync tool already manages) | L | P1 |
|
||||
| Security-scoped bookmark for an external folder | Obsidian iOS, iOS apps | Same on iOS (iCloud/Drive folders) — bookmark persistence + revocation handling | L | P1 |
|
||||
| Share-target receive (Android `ACTION_SEND` intent + iOS share extension) | Obsidian Mobile, Apple Notes, Bear | Capture a page or snippet as a note in ~2 s; the classic mobile loop | M | P0 |
|
||||
| Quick capture sheet (title + body + target folder) | Obsidian Mobile, Bear, Apple Notes | Lowest-friction capture path; reuses the "vault folder / same folder / specific folder" rule | M | P0 |
|
||||
| Quick switcher (fuzzy title search) | Obsidian, Bear, Craft suite | The main navigation affordance on a 390 pt screen | M | P0 |
|
||||
| Command palette on mobile over the engine registry | Obsidian (plugin), VS Code, Craft suite | One command surface for UI, CLI, control channel, MCP and plugins (SPEC 10) | M | P1 |
|
||||
| Bottom tab bar shell (Notes, Search, Graph, Calendar, More) | Obsidian Mobile, Craft suite | The one-column layout the spec's 390 pt acceptance criterion demands | M | P0 |
|
||||
| Mobile toolbar above the keyboard | Obsidian Mobile, Craft suite editor | Formatting without a hardware keyboard; needs keyboard-aware layout first | L | P1 |
|
||||
| Long-press context menu on rows (Rename/Move/Duplicate/Delete) | Obsidian, iOS/Android OS convention | Renaming and moving is the other half of note management | M | P1 |
|
||||
| Swipe tab switching / edge drawer | Obsidian Mobile, Bear, Craft suite | Fast navigation, but conflicts with OS edge-back — expect to drop the edge variant | M | P1 |
|
||||
| Safe-area + keyboard-aware layout (insets, `content_rect`) | native apps, Obsidian | Content must never sit under the home indicator or the keyboard; egui gives iOS insets for free, Android not | M | P0 |
|
||||
| Correct IME / marked text (candidate window) | all native apps | Non-latin input is broken without it; iOS needs a native `UITextView` shim | L | P0 |
|
||||
| Background sync (WorkManager, `BGTaskScheduler`) | Obsidian Sync (proprietary), Joplin | Convergence while the app is closed; the core of the mobile promise | M | P0 |
|
||||
| Sync status line + pending/offline queue UI | Obsidian Sync, Craft suite | Trust: the user must see what synced, what is pending, what conflicted | S | P0 |
|
||||
| Conflict resolution UI (list + side-by-side, never lose text) | Obsidian Sync, git | Direct spec acceptance criterion; also the safety net for a single-owner many-device vault | M | P0 |
|
||||
| Biometric app lock | Obsidian, 1Password, Bear | Protects a lost/borrowed device; plain `.md` at rest is not encrypted | M | P1 |
|
||||
| App shortcuts (long-press the app icon) | Obsidian Android, iOS 16+ | New note / daily note / search without opening the app | S | P1 |
|
||||
| Android home-screen widget | Obsidian Android, Craft suite | Open note, new note, search, daily note | M | P2 |
|
||||
| iOS WidgetKit widget (iOS 18+ per Obsidian) | Obsidian iOS, Craft suite | Same on iOS; native Swift reading an App Group container | M | P2 |
|
||||
| Deep link scheme `notescraft://` | Obsidian (`obsidian://`), Craft suite | Unlocks widgets, shortcuts, share and "open in NotesCraft" | S | P1 |
|
||||
| Local notifications for sync results | Obsidian Sync, Craft suite | Trust when backgrounded | S | P2 |
|
||||
| Push notifications (APNs/FCM) | Obsidian Sync | Real-time; needs a hosted push provider a homelab does not have | L | P2 |
|
||||
| Screen-reader accessibility (labels, focus order, 44 pt targets) | native apps, Obsidian | Required for a credible a11y story; accesskit mobile support unverified | L | P1 |
|
||||
| Background vault index build with progress | Craft suite | 8,000 notes must not block the first frame on a phone | M | P1 |
|
||||
| Momentum / inertial scrolling | native apps, Bear | The most visible "not native" tell in an egui app | M | P1 |
|
||||
|
||||
---
|
||||
|
||||
## Recommendations — the 10 to build next
|
||||
|
||||
1. **Android shell on GameActivity + `cargo-ndk` + Gradle AAB** (L, P0) — the fastest route to a phone in the owner's hand; GameActivity is what Google recommends and NativeActivity tooling is stale. *Acceptance:* Given a clean checkout, when the release CI job runs, then it produces a signed `.aab` for `arm64-v8a` and `armeabi-v7a` that installs on a physical device and reaches the file list in under 3 s with no ANR in logcat.
|
||||
2. **iOS static-lib wrapper in Xcode** (L, P0) — iOS is the harder half, so start the spike early even though it is not officially supported upstream. *Acceptance:* Given a macOS machine with Xcode, when the project is archived, then the app starts the egui shell inside `ApplicationHandler::resumed`, and after a background→foreground cycle the same note is still open with the keyboard still working.
|
||||
3. **Sandbox vault plus Files-app visibility** (S, P0) — the vault stays plain `.md`, so this is the cheapest thing that satisfies "the vault is the source of truth" on a phone. *Acceptance:* Given a fresh install, when a note is created, then it appears under `Documents/NotesCraft` in the iOS Files app and under the Android `filesDir`, and the sync scan picks it up with no extra configuration.
|
||||
4. **Soft-keyboard and IME correctness** (L, P0) — the single hardest technical blocker; without it the editor is unusable for most people. *Acceptance:* Given a phone keyboard, when the user types accented and CJK text into a note, then the *composed* text is inserted (never per-key strokes), the candidate window opens, and the caret stays visible above the keyboard.
|
||||
5. **Safe-area + keyboard-aware one-column shell** (M, P0) — the spec's 390 pt acceptance criterion, and the prerequisite for the toolbar and autocomplete popup. *Acceptance:* Given a 390×844 phone with the keyboard open, when a note is open, then the text area and the mobile toolbar are fully visible, nothing is hidden under the home indicator, and the bottom tab bar shows exactly five items.
|
||||
6. **Quick capture sheet plus share-target receive** (M, P0) — the killer mobile loop: text from anywhere becomes a note, and it also forces the command registry to own `note.new_from_share`. *Acceptance:* Given text shared from Safari into NotesCraft, when the share sheet returns, then a note appears in the configured capture folder within 2 s and is visible in the file list with no further tap.
|
||||
7. **Quick switcher and command palette over the engine registry** (M, P0) — one search box that navigates and one that commands; both are thin because the engine already owns the command surface. *Acceptance:* Given the switcher, when three characters are typed, then notes are filtered by fuzzy title match and the top hit opens on Enter, and the palette lists exactly the command ids the desktop registry exposes.
|
||||
8. **Background sync via WorkManager and `BGTaskScheduler`** (M, P0) — the reason a phone build is worth having; the client is already resumable by design (`last_seq` in `state.json`). *Acceptance:* Given the app backgrounded for one hour with a network available, when the OS runs the scheduled task, then the vault converges with the server and the status line shows the last sync time, with no long-lived socket and at most one poll per window.
|
||||
9. **Conflict resolution UI plus an offline queue view** (M, P0) — never lose text is a hard product promise, and a single owner with many devices will hit it. *Acceptance:* Given the same note edited offline on two devices, when both reconnect, then each device lists a Conflicts entry showing both versions, and no character of either version is lost even if the user resolves nothing.
|
||||
10. **Mobile toolbar and long-press context menu** (L, P1) — the two things that make a phone build feel like a notes app rather than a shrunk desktop one; both are gated on the keyboard-aware layout from #5. *Acceptance:* Given a note open on a phone, when a toolbar button is tapped, then the corresponding formatting command applies, and when a file row is long-pressed, then a context menu offers Rename, Move, Duplicate and Delete, and a Delete can be undone.
|
||||
|
||||
Ordering note: #1 and #2 are the platform spikes; #3–#5 are the shell; #6–#10 are the product value. Biometrics, widgets, app shortcuts, notifications and push stay behind these (P1/P2 in the table) — they are cheap to add once the shell and the command surface exist.
|
||||
|
||||
---
|
||||
|
||||
## Risks
|
||||
|
||||
1. **iOS is not an officially supported egui target** (finding 2): winit's iOS backend is Tier 2, egui issue #3117 is open, there is no upstream iOS CI, and `eframe`'s iOS entry (`create_native`) is a moving API. Mitigation: pin `egui`/`winit`/`wgpu` exactly, keep a private patch set if needed, and put our **own** iOS CI build on a macOS runner so an upstream break shows up in our pipeline, not on the owner's phone.
|
||||
2. **The IME gap on iOS** (finding 3): winit's UIKit view implements only `UIKeyInput` — no marked text, no candidate window, no autocorrect, no dictation — so a native `UITextView` shim in Swift is required. That means two UI stacks in one app, in a project whose rule is "Rust only". Mitigation: keep the shim minimal (text in, text out) and treat it as an adapter, not a feature.
|
||||
3. **Android soft keyboard is not solved either** (egui #5590 is still open): `set_ime_cursor_area`/`set_ime_purpose` are no-ops on Android, so the candidate window never anchors to the caret. Mitigation: plan for a Kotlin-side `show_soft_input` + caret-rect bridge from day one, not after the first bug report.
|
||||
4. **GameActivity vs the Rust tooling**: `cargo-apk` is NativeActivity-only and stale (2023), and egui's example depends on a patched fork. Mitigation: write our own thin Gradle template around `cargo ndk`; treat `cargo-apk` as optional.
|
||||
5. **SAF is hostile to a vault** (finding 5): a user-picked tree must be walked one `DocumentFile` at a time, and it is not a plain filesystem path, so `std::fs` cannot walk it. An 8,000-note vault over SAF is impractical. Mitigation: keep the sandbox as the only writable vault in v1; treat external folders as import-only and say so in the UI.
|
||||
6. **Security-scoped access can be revoked** (finding 6) at any moment in Settings → Privacy → Files and Folders; a bookmark that fails to resolve must become a clear `Err` and a re-pick flow, never a silent fallback to a different folder.
|
||||
7. **No gesture layer in egui** (finding 16): swipes, edge gestures, momentum scrolling and native text handles must be hand-built or dropped, and edge gestures fight the OS back gesture. Mitigation: ship buttons first, gestures later, and never make a gesture the only path.
|
||||
8. **Accessibility may not ship with the first build** (finding 13): accesskit's mobile backends are unverified. Mitigation: design to 44 pt targets and OS font scaling from the first layout, and treat screen-reader support as a follow-up milestone rather than a promise.
|
||||
9. **Store compliance is real work, not paperwork** (finding 14): Play's target-API floor (**2026 level unverified**), the 16 KB page-size requirement, the AAB-only rule, native debug-symbol upload, the Data safety form, and on Apple the privacy manifest per executable, the App Privacy label (self-hosted sync still counts as transmission), Background Modes declarations for `BGTaskScheduler` and a 4.2 minimum-functionality risk. Mitigation: keep a release checklist in the repo and re-read both stores' current pages before the first submission.
|
||||
10. **Signing and CI are owner-only dependencies** (finding 15): an Apple Developer membership, an upload keystore, CI secrets and a macOS runner with Xcode. A mobile release that only builds on one machine is a single point of failure. Mitigation: CI-only release path, secrets in CI, and a rehearsed "build from a clean checkout" run before every store submission.
|
||||
11. **Energy and radio cost**: long-polling in the background is killed by the OS and drains the battery; wake-ups must be scheduled, not spun. Mitigation: foreground long-poll only, one poll per background window, batch with `/batch/check`.
|
||||
12. **Font coverage**: egui's default fonts have limited CJK coverage, so a Chinese/Japanese vault renders as tofu without embedded fonts. Mitigation: subset and embed a font per script in the theme/packaging step, and size it for download budget.
|
||||
13. **Activity recreation on Android** (finding 4): without `android:configChanges` for orientation and screen size, rotation destroys and recreates the Activity — losing in-memory editor state. Mitigation: declare the config changes and persist the open-note state in the vault/session file.
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
### Browsed and verified in this research (findings 1–8, accessed 2026)
|
||||
|
||||
- `eframe` / `egui-winit` crate metadata and features — <https://crates.io/crates/eframe> · <https://crates.io/crates/egui-winit>
|
||||
- egui repository, changelogs, issues and pull requests (#3117 iOS target, #7768 iOS backend, #7578 safe area, #8083/#7967 IME, #8052 iOS crash, #5590 Android keyboard, #5759 Android text, #7073 Android back button, #7453 rotation gesture, #8484 safe-area fix) — <https://github.com/emilk/egui>
|
||||
- egui 0.36.0 and 0.37.0 release notes — <https://github.com/emilk/egui/releases/tag/0.36.0> · <https://github.com/emilk/egui/releases/tag/0.37.0>
|
||||
- egui Android example (NDK, build-tools, patched cargo-apk fork) — <https://github.com/emilk/egui/tree/main/examples/hello_android>
|
||||
- `android-activity`, `winit` Android and iOS platform docs, winit changelog — <https://docs.rs/android-activity/latest/android_activity/> · <https://docs.rs/winit/latest/winit/platform/android/> · <https://docs.rs/winit/latest/winit/platform/ios/index.html> · <https://docs.rs/winit/latest/winit/changelog/index.html>
|
||||
- `winit-uikit` (iOS backend extracted from winit 0.31, Tier 2) — <https://crates.io/crates/winit-uikit>
|
||||
- Rust Android build tooling: `cargo-ndk`, `cargo-apk` (+ #66), `cargo-xbuild`, `xbuild` — <https://crates.io/crates/cargo-ndk> · <https://crates.io/crates/cargo-apk> · <https://github.com/rust-mobile/cargo-apk/issues/66> · <https://crates.io/crates/cargo-xbuild> · <https://crates.io/crates/xbuild>
|
||||
- iOS tooling: `cargo-mobile2` (+ stale egui template), `cargo-xcode`, `cargo-dinghy`, `cargo-lipo`, `cargo-bundle` — <https://crates.io/crates/cargo-mobile2> · <https://github.com/tauri-apps/cargo-mobile2> · <https://crates.io/crates/cargo-xcode> · <https://gitlab.com/kornelski/cargo-xcode> · <https://crates.io/crates/cargo-dinghy> · <https://crates.io/crates/cargo-lipo> · <https://crates.io/crates/cargo-bundle>
|
||||
- Google: GameActivity — <https://developer.android.com/games/agdk/game-activity>
|
||||
- Google: 16 KB page sizes — <https://developer.android.com/guide/practices/page-sizes>
|
||||
- Google: SAF / `ACTION_OPEN_DOCUMENT_TREE` — <https://developer.android.com/guide/topics/permissions/requesting#open-document-tree>
|
||||
- Google: scoped storage and `MANAGE_EXTERNAL_STORAGE` — <https://developer.android.com/training/data-storage/manage-all-files>
|
||||
- Apple: `UIFileSharingEnabled` — <https://developer.apple.com/documentation/bundleresources/information-property-list/uifilesharingenabled>
|
||||
- Apple: `LSSupportsOpeningDocumentsInPlace` — <https://developer.apple.com/documentation/bundleresources/information-property-list/lssupportsopeningdocumentsinplace>
|
||||
- Apple: picking directories, security-scoped URLs — <https://developer.apple.com/documentation/uikit/providing-access-to-directories> · <https://developer.apple.com/documentation/uikit/uidocumentpickerviewcontroller>
|
||||
- Apple: `startAccessingSecurityScopedResource` — <https://developer.apple.com/documentation/foundation/url/startaccessingsecurityscopedresource()>
|
||||
- Apple: share extensions, `NSExtension` keys — <https://developer.apple.com/library/archive/documentation/General/Conceptual/ExtensibilityPG/Share.html> · <https://developer.apple.com/documentation/bundleresources/information-property-list/nsextension>
|
||||
- Apple: App Groups — <https://developer.apple.com/documentation/xcode/configuring-app-groups>
|
||||
- Apple: privacy manifests for required-reason APIs — <https://developer.apple.com/documentation/bundleresources/describing-use-of-required-reason-api>
|
||||
- Apple: App Store Review Guidelines — <https://developer.apple.com/app-store/review/guidelines/>
|
||||
- Apple: `UIKeyInput` vs `UITextInput` — <https://developer.apple.com/documentation/uikit/uikeyinput> · <https://developer.apple.com/documentation/uikit/uitextinput>
|
||||
- egui API: `MultiTouchInfo`, `InputState`, `InputOptions` — <https://docs.rs/egui/latest/egui/struct.MultiTouchInfo.html> · <https://docs.rs/egui/latest/egui/struct.InputState.html> · <https://docs.rs/egui/latest/src/egui/input_state/mod.rs.html>
|
||||
- Obsidian help: iOS, Android, settings — <https://help.obsidian.md/ios> · <https://help.obsidian.md/android> · <https://help.obsidian.md/settings>
|
||||
- Rust Android egui example (insets) — <https://github.com/rust-mobile/rust-android-examples/tree/main/agdk-egui>
|
||||
|
||||
### Canonical pages to re-verify (findings 9–16 — **not fetched**, browse quota exhausted; accessed year unknown)
|
||||
|
||||
- WorkManager periodic work and constraints — <https://developer.android.com/develop/background-work/background-tasks/persistent/getting-started/define-work>
|
||||
- Android foreground services and the `dataSync` type — <https://developer.android.com/develop/background-work/services/fgs/service-types>
|
||||
- BiometricPrompt — <https://developer.android.com/identity/sign-in/biometric-auth>
|
||||
- Play target API level and 16 KB page-size requirement — <https://developer.android.com/google/play/requirements/target-sdk> · <https://developer.android.com/guide/practices/page-sizes>
|
||||
- Play App Bundle and App Signing — <https://developer.android.com/guide/app-bundle> · <https://developer.android.com/studio/publish/app-signing>
|
||||
- Play Data safety and privacy policy — <https://developer.android.com/google/play/requirements/data-safety> · <https://developer.android.com/google/play/requirements/privacy-policy>
|
||||
- `BGTaskScheduler` and `BGAppRefreshTask`, Info.plist keys — <https://developer.apple.com/documentation/backgroundtasks> · <https://developer.apple.com/documentation/backgroundtasks/bgrefreshrequest>
|
||||
- `LocalAuthentication` — <https://developer.apple.com/documentation/localauthentication>
|
||||
- `UNUserNotificationCenter` — <https://developer.apple.com/documentation/usernotifications/unusernotificationcenter>
|
||||
- App Store privacy labels / App Privacy details — <https://developer.apple.com/app-store/app-privacy-details/>
|
||||
- Xcode code signing and distribution — <https://developer.apple.com/help/account/codesigning/>
|
||||
- egui `accesskit` feature and accesskit platform support — <https://docs.rs/egui/latest/egui/#feature-flags> · <https://github.com/AccessKit/accesskit>
|
||||
@@ -0,0 +1,609 @@
|
||||
# Security and server features for the NotesCraft sync service
|
||||
|
||||
Research note for `specs/mobile-and-sync/spec.md`. Scope: end-to-end encryption for
|
||||
the file-sync service, device pairing and token lifecycle, large-attachment handling
|
||||
(chunking, dedup, resumable uploads), tombstones / GC / retention / quotas, server
|
||||
backup and restore, the MinIO/S3 blob backend, observability, deployment behind
|
||||
Traefik + Cloudflare Tunnel, and adjacent features (publish, clipper, shared links,
|
||||
email-to-note, voice memo, OCR).
|
||||
|
||||
## Method
|
||||
|
||||
**Browsed.** Web search and page fetch were used on 2026-10-20 (access year 2026 for
|
||||
every URL in *Sources*). Claims below are backed by those fetches unless the text says
|
||||
*from memory* or *unverified*. Local ground truth came from reading
|
||||
`specs/mobile-and-sync/spec.md`, `crates/sync/` (`lib.rs`, `path.rs`, `hash.rs`,
|
||||
`ledger.rs`, `conflict.rs`, `merge.rs`, `plan.rs`) and
|
||||
`apps/notescraft-sync-server/` (`routes.rs`, `store.rs`, `state.rs`, `config.rs`).
|
||||
|
||||
Conventions used here: **unverified** = plausible but not confirmed by a fetched
|
||||
source in this session (typically a crate name I did not look up, or a version I did
|
||||
not open); **from memory** = stated without a fetched source.
|
||||
|
||||
No version number, price or API name below is invented. Where a product is named
|
||||
without a version, that is deliberate.
|
||||
|
||||
---
|
||||
|
||||
## Findings
|
||||
|
||||
### 1. Two proven E2EE models exist, and both leak metadata
|
||||
|
||||
**Obsidian Sync** is the closest analogue to NotesCraft's single-owner case. Obsidian
|
||||
publishes a verification recipe: a per-vault random salt, a base key derived with
|
||||
**scrypt** from the vault password, an encryption key derived from the base key with
|
||||
**HKDF-SHA256** (`info = "ObsidianAesGcm"`), and payload encryption with
|
||||
**AES-256-GCM** (12-byte IV prefixed, 16-byte tag appended). The vault password is
|
||||
separate from the account password. Obsidian states plainly that E2EE data is
|
||||
unrecoverable if the password is lost, and that the server keeps the encryption key
|
||||
out of reach.
|
||||
|
||||
Obsidian also documents its own limits, which are the most useful part for us:
|
||||
|
||||
- *Deterministic file-hash encryption.* The same content under the same key always
|
||||
yields the same encrypted hash, so the server can detect duplicates. An attacker who
|
||||
can force uploads can therefore test whether a file matches one uploaded earlier.
|
||||
- *No cryptographic binding between path and content.* The server can read which
|
||||
device uploaded or deleted a file, when, and the **mapping between encrypted paths
|
||||
and encrypted content**. A compromised server can therefore re-map one encrypted
|
||||
file under another path — no plaintext leaks, but content can be substituted.
|
||||
|
||||
**Syncthing's "untrusted device" model** is the other reference, and it is more
|
||||
radical about metadata:
|
||||
|
||||
- `folderKey = scrypt(password, "syncthing" + folderID)` (32 bytes).
|
||||
- File **names** are encrypted deterministically with **AES-SIV** (no nonce) and
|
||||
base32-encoded, then split across a synthetic directory tree
|
||||
(`X.syncthing-enc/AB/CDEF…`) so the server never sees the real folder structure.
|
||||
- A **per-file key** is `HKDF-SHA256(folderKey || filename)`.
|
||||
- File **data blocks** are encrypted with **XChaCha20-Poly1305** (24-byte random
|
||||
nonce prepended to the ciphertext) under the per-file key; block **hashes** are
|
||||
encrypted with AES-SIV using the file key *and the block offset as associated
|
||||
data*, so the same bytes at different offsets or in different files are not
|
||||
linkable.
|
||||
- The untrusted peer can still observe: file sizes (padded to 1 KiB granularity),
|
||||
which parts changed, and when.
|
||||
- Syncthing explicitly documents the consequence: because blocks must not be
|
||||
comparable across files, **blocks cannot be reused between files** on an untrusted
|
||||
peer — dedup and cross-file reuse are sacrificed for confidentiality.
|
||||
|
||||
**Design consequence for NotesCraft.** We want the Obsidian-grade threat model
|
||||
(server cannot read notes) plus the Syncthing-grade property that *paths are not
|
||||
visible*. That means the server can no longer key blobs by path; it must key them by
|
||||
an **opaque random blob id**, and the mapping `blob id → path` must live inside the
|
||||
encrypted manifest. The current server stores `path` in a plaintext SQLite column and
|
||||
uses it as the blob key — that has to become `blob_id` (opaque) plus an encrypted
|
||||
manifest if we want path confidentiality.
|
||||
|
||||
### 2. Per-vault key hierarchy, wrapping and recovery
|
||||
|
||||
The pattern that satisfies "single owner, many devices, self-hosted" without asking
|
||||
the owner to type a passphrase on every phone:
|
||||
|
||||
- **Vault master key (VMK)**: 32 random bytes generated once per vault. Never
|
||||
derived from a password, never transmitted in the clear.
|
||||
- **KEK from passphrase**: `Argon2id` over the owner's passphrase with a per-vault
|
||||
random salt → 32-byte key-encryption key → wraps the VMK. `age` already models
|
||||
this shape (`scrypt::Recipient` wraps a random file key under a passphrase;
|
||||
`x25519::Recipient` wraps it under a device public key).
|
||||
- **Device wrap**: each device holds an X25519 key pair; the VMK is wrapped to each
|
||||
device's public key at pairing time. Adding a phone never needs the passphrase.
|
||||
- **Recovery key**: a printable, high-entropy backup of the VMK (base32 / a word
|
||||
list) that can re-wrap the VMK under a new passphrase or a new device. This is the
|
||||
only way to survive "lost phone **and** forgotten passphrase".
|
||||
- **Per-file keys**: derived from the VMK with HKDF over `path` (+ optional content
|
||||
hash), so a single leaked file key does not expose the vault.
|
||||
- **Key rotation**: re-wrapping the VMK under new device keys is cheap; re-keying the
|
||||
whole vault is not. restic states the same limitation for its repository key:
|
||||
revocation of a leaked key without re-encrypting everything is not supported
|
||||
(restic mitigates it with a `copy` to a new repository).
|
||||
|
||||
**What the server still sees** (design the audit log and threat model around this
|
||||
list, not around "nothing"):
|
||||
|
||||
- vault id, device ids, device names, pairing events;
|
||||
- opaque blob ids, byte sizes (padded), modification times, `seq` numbers;
|
||||
- *when* each change happened, from which device, and the change frequency;
|
||||
- client IP, TLS metadata, request sizes, long-poll timing;
|
||||
- the **encrypted** path and manifest blob (unreadable to it).
|
||||
|
||||
Notably it does **not** see: note titles, note bodies, folder names, or which file
|
||||
changed relative to which other file — unless we keep the current plaintext
|
||||
`path` column, which we should not if E2EE is a goal.
|
||||
|
||||
### 3. Rust crates: what is available, and how healthy it is
|
||||
|
||||
Versions below are the ones actually read on crates.io/docs.rs on 2026-10-20.
|
||||
|
||||
| Crate | Version seen | Status / notes |
|
||||
|---|---|---|
|
||||
| `chacha20poly1305` | 0.11.0 (2026-06-28) | RustCrypto/AEADs. MSRV 1.85, edition 2024, Apache-2.0/MIT. One security audit by NCC Group, no significant findings. Provides XChaCha20-Poly1305 (24-byte nonce) and reduced-round variants. **Actively maintained, audit-grade.** |
|
||||
| `argon2` | 0.6.0 | RustCrypto. Argon2id is the default variant. Offers both PHC password hashing **and** raw `hash_password_into` key derivation — the latter is what we want for the passphrase→KEK step. MIT/Apache. |
|
||||
| `rust-argon2` | 3.0.0 (sru-systems) | Alternative to the above. Its own README warns it "does not attempt to clear potentially sensitive data from its work memory". Prefer the RustCrypto crate for a key-derivation path where we then hold the key in memory. |
|
||||
| `rand` | 0.10.3 (2026-09-20) | rust-random. MSRV 1.85. **Not a cryptographic library** — its README says so. Use `getrandom`/`OsRng` for keys and nonces; use `rand` only for non-secret material (pairing codes, jitter). |
|
||||
| `zeroize` | 1.9.1 (2026-10-06) | RustCrypto. Portable, uses `write_volatile` + fences, no FFI, `no_std`. **Actively maintained.** Required for the passphrase and VMK handling path. |
|
||||
| `age` | 0.12.1 (2026-07-14) | `str4d/rage`, MIT/Apache, Rust 1.74. Crate description still says "[BETA]". Implements the age v1 spec: `x25519`, `scrypt`, `ssh`, `tag`, `tagpq` recipients; streaming API; `armor`. **Good for exporting/importing a wrapped vault key**, wrong shape for a sync protocol (it is a file-encryption format, not a chunked revision protocol). |
|
||||
| `qrcodegen` | 1.8.0 (2022-04-17) | Nayuki, MIT. No release since 2022, but the library is complete, small and self-contained (all 40 versions, 4 ECC levels). Dormant but low-risk. Renders to raw modules; we draw the image ourselves. |
|
||||
| `tower_governor` | 0.8.0 (2025-08-14) | MIT/Apache, `governor` backend, `axum` feature. Mature, used with axum 0.8. Provides `PeerIpKeyExtractor`, `SmartIpKeyExtractor` (looks at `x-forwarded-for`/`x-real-ip`/`forwarded`), `GlobalKeyExtractor`, `use_headers()` for `x-ratelimit-*`. |
|
||||
| `whisper-rs` | 0.16.0 (2026-03-12) | Unlicense. Rust bindings to whisper.cpp via `whisper-rs-sys` (C++ build, cmake). Actively maintained. Desktop/mobile builds are heavy; the model file must be shipped or fetched. |
|
||||
| `opendal-service-s3` | 0.58.2 (2026-08-20) | Apache OpenDAL, Apache-2.0, **MSRV 1.91** (workspace is on 1.95, so fine). Speaks the S3 API incl. MinIO, supports versioning/restore, SSE-KMS/SSE-C/SSE-S3, presign, and per-service capability probing. Split-crate layout (`opendal-core` + `opendal-service-s3`). |
|
||||
|
||||
Also plausible but **unverified** in this session (names I did not open): `hkdf`,
|
||||
`blake3`, `aes-siv`, `x25519-dalek`, `subtle`/`aead` trait crates, `prometheus` or
|
||||
`metrics` for exposition, `tracing`/`tracing-subscriber`, `tower-http`,
|
||||
`async-imap` for email, `aws-sdk-s3`.
|
||||
|
||||
**Crate choices, in one line each:**
|
||||
|
||||
- Content cipher: `chacha20poly1305` XChaCha20-Poly1305 (audit-grade, random 24-byte
|
||||
nonce, no nonce-reuse risk across a whole vault). **Choose.**
|
||||
- KDF: `argon2` (Argon2id) for passphrase → KEK. **Choose.** Use `zeroize` on every
|
||||
buffer that holds a key, passphrase or plaintext.
|
||||
- RNG: `getrandom`/`OsRng` for all secrets; `rand` only for non-secrets. **Choose.**
|
||||
- Key wrapping: `age` if we want an interoperable file format for vault-key backup
|
||||
and device-key export; otherwise a 100-line `x25519` + XChaCha20 wrap is enough.
|
||||
`age` is the lazier path *because it already exists* and is maintained.
|
||||
- Deterministic encryption of names: Syncthing uses AES-SIV. If the RustCrypto
|
||||
`aes-siv` crate is acceptable (**unverified** — I did not fetch it), reuse it;
|
||||
otherwise derive a per-path nonce deterministically from a **separate** name key
|
||||
and encrypt with XChaCha20-Poly1305. The second option is deterministic-AEAD
|
||||
misuse and must be documented and fuzzed, so prefer a real SIV.
|
||||
- Do **not** use `rust-argon2` for key derivation (it does not clear work memory) and
|
||||
do **not** use `rand` for key bytes.
|
||||
|
||||
### 4. Device pairing by QR, token rotation, revocation, lockout
|
||||
|
||||
**QR pairing.** The desktop shows a QR; the phone scans it. The QR payload should be
|
||||
a signed, expiring blob containing: vault id, a one-time pairing code id, the vault
|
||||
master key **wrapped to the phone's freshly generated X25519 public key**, and an
|
||||
expiry timestamp. Two flows exist and both are worth having:
|
||||
|
||||
- *Phone-initiated*: phone generates a keypair + pairing secret, shows a QR; desktop
|
||||
scans it. Wrapping key to the phone key happens on the desktop where the VMK lives.
|
||||
- *Desktop-initiated*: desktop shows a QR with a short pairing code; the phone types
|
||||
or scans it; the phone sends its public key; the desktop wraps the VMK to it.
|
||||
|
||||
The current server already has `POST /v1/vaults/{vault}/pairing` and
|
||||
`POST /v1/devices`, and `pairings.code_hash` is stored hashed — good. What is
|
||||
missing is (a) the *wrapped VMK* in the pairing response, (b) an explicit attempt
|
||||
counter, and (c) rate limiting. `qrcodegen` + our own renderer is enough; no image
|
||||
crate dependency is needed if we draw the modules as rectangles in egui.
|
||||
|
||||
**Token rotation and revocation.** The current `devices` table has
|
||||
`token_hash` only — no expiry, no revocation column, no "last used". Minimum
|
||||
viable lifecycle:
|
||||
|
||||
- Access token: opaque 32 random bytes, stored as a hash, short TTL (say 15 min),
|
||||
presented as `Authorization: Bearer`.
|
||||
- Refresh token: opaque, longer TTL (say 30 days), rotated on every refresh with a
|
||||
short grace window so a racing client does not lock itself out; the server keeps
|
||||
`token_hash`, `prev_token_hash`, `expires_ms`, `revoked_ms`, `device_id`.
|
||||
- Revocation: `POST /v1/devices/{id}/revoke` sets `revoked_ms`; the server checks it
|
||||
on every request. Revoking a device must also mean: its refresh token stops
|
||||
working and its queued long-poll is terminated. Note the spec's own acceptance
|
||||
criterion — "wrong or revoked token → 401 and the client shows *Sign in again*
|
||||
without touching files".
|
||||
- Emergency: rotate the *vault* keys (new VMK, re-encrypt manifest) — expensive,
|
||||
but the recovery key makes it possible.
|
||||
|
||||
**Rate limiting and lockout on pairing codes.** `tower_governor` is the obvious fit
|
||||
for axum. Two traps, both documented in its README:
|
||||
|
||||
1. Do **not** construct the same `GovernorConfig` twice — each config is an
|
||||
independent limiter.
|
||||
2. Behind Cloudflare Tunnel the peer IP is the `cloudflared` pod IP, so
|
||||
`PeerIpKeyExtractor` would put *every* client in one bucket. Use
|
||||
`SmartIpKeyExtractor` (or a custom extractor reading `CF-Connecting-IP` /
|
||||
`X-Real-Client-Ip` set by the edge).
|
||||
3. `GovernorConfig::secure()` is a sane preset for auth endpoints: burst of 2,
|
||||
replenishing one token every 4 s per peer.
|
||||
|
||||
Pairing lockout policy (own code, ~30 lines): a pairing code is valid for a short
|
||||
TTL (the server already has `PAIRING_TTL_MS`); after N failed redemption attempts
|
||||
the code is invalidated; redemption attempts are also rate limited per IP with
|
||||
exponential backoff. A wrong code must answer 401/403 without revealing whether it
|
||||
existed.
|
||||
|
||||
**Audit log.** OWASP's Logging Cheat Sheet is the right template: log
|
||||
authentication successes *and* failures, authorization failures, input-validation
|
||||
failures, admin actions (add/remove device, rotate key, revoke token), data export,
|
||||
and configuration changes. Every entry records **when, where, who, what** plus
|
||||
severity and result status. Never log: raw tokens, passwords, encryption keys, or
|
||||
session identifiers (hash them instead). Sanitize control characters to prevent log
|
||||
injection. Protect the log itself: append-only, restricted permissions, and a
|
||||
tamper-detection story — for a homelab, shipping the audit stream as JSON lines to
|
||||
the existing Loki/Alloy collector (as the Traefik guide does) is the cheapest path.
|
||||
Note OWASP's point that process/audit logs and security-event logs are different
|
||||
purposes and should be kept separate.
|
||||
|
||||
### 5. Chunking and content-defined dedup for big attachments
|
||||
|
||||
restic's design doc is the reference implementation worth copying:
|
||||
|
||||
- **Rolling Rabin fingerprint** over a **64-byte window**; a chunk boundary is
|
||||
declared when the **lowest 21 bits of the fingerprint are zero**. Boundaries
|
||||
therefore depend only on content, so inserting bytes near the start of a file does
|
||||
not shift every later boundary.
|
||||
- Chunks are **512 KiB – 8 MiB**, target average **1 MiB**.
|
||||
- The irreducible polynomial is chosen **randomly at repo init and stored in the
|
||||
config**, which is a direct defence against "watermark" chunking attacks (an
|
||||
attacker who knows the polynomial can craft inputs that align boundaries).
|
||||
- restic 0.18.0 added another mitigation: **randomly assigning chunks to pack
|
||||
files**, so an observer cannot infer chunk sizes from pack layout.
|
||||
- The chunking-attack literature is cited there: Alexeev, Percival & Zhang,
|
||||
"Chunking Attacks on File Backup Services using Content-Defined Chunking".
|
||||
|
||||
The tension with E2EE is explicit and unavoidable: **content-defined chunking plus
|
||||
server-side dedup leaks equality of content across files** (identical chunk ids ⇒
|
||||
identical plaintext, if the chunk id is derived from plaintext). Syncthing hit the
|
||||
same wall and gave up cross-file dedup on untrusted peers. Three workable positions:
|
||||
|
||||
1. **Client-side CDC, encrypted chunks, server sees only opaque chunk ids.** The
|
||||
client chunks with CDC, encrypts each chunk with the VMK (XChaCha20-Poly1305,
|
||||
random nonce) and stores `blob id = BLAKE3(vault_salt || plaintext_chunk_hash)`.
|
||||
The server dedups by that id. Equality of *plaintext* across two files is still
|
||||
leaked by equal ids — mitigate by salting the id with a per-vault secret that
|
||||
only the client holds (server cannot recompute ids for new content, so dedup
|
||||
would then break)… so in practice **equal chunk ids leak equality**. Decide
|
||||
explicitly and document it.
|
||||
2. **Dedup only within a file** (safe) plus **whole-file dedup by hash of the
|
||||
encrypted blob** (also leaks equality, but only of whole files).
|
||||
3. **Per-vault key per chunk, no cross-file reuse**: the "Syncthing untrusted"
|
||||
position — best privacy, worst bandwidth.
|
||||
|
||||
Recommendation for v1: **whole-file dedup by encrypted-blob hash only** (the current
|
||||
`hash` field already does this), and treat CDC as a **P1** feature scoped to
|
||||
attachments (> 8 MiB) where bandwidth actually hurts, with the equality-leak
|
||||
documented in the threat model. Effort: L.
|
||||
|
||||
### 6. Resumable uploads
|
||||
|
||||
The `tus` protocol (v1.0.0, MIT, SemVer-stable) is the reference: `HEAD` returns
|
||||
`Upload-Offset`, `PATCH` with `Content-Type: application/offset+octet-stream`
|
||||
appends at that offset, mismatched offset ⇒ **409**, and the extensions are
|
||||
`creation`, `creation-with-upload`, `expiration`, `checksum`, `termination`,
|
||||
`concatenation`. Rust implementations exist (`rustus` on crates.io, Rocket
|
||||
integration `meteoritus`) but they are third-party crates and would add a dependency
|
||||
the project has not approved.
|
||||
|
||||
Two options for NotesCraft:
|
||||
|
||||
- **tus-compatible endpoint** (`POST` create + `HEAD`/`PATCH` upload). Pros:
|
||||
standard, resumable out of the box, client-agnostic. Cons: a second upload
|
||||
protocol living next to the file-revision protocol, plus a new dependency.
|
||||
- **Chunked `PUT` with `Content-Range`** plus a server-side `uploads` table
|
||||
(`upload_id`, `path`/`blob_id`, `offset`, `length`, `expires_ms`). Pros: reuses
|
||||
the existing `PUT /files/{path}` shape, no new dependency, ~150 lines. Cons:
|
||||
home-grown, less interoperable.
|
||||
|
||||
Given "Rust only, minimal dependencies", the chunked-`PUT` option is the smaller
|
||||
diff. The 25 MiB per-file cap in `routes.rs` means resumability only matters for
|
||||
large attachments, so this is P1, not P0. Expiration of abandoned uploads should be
|
||||
enforced by a janitor task (the `expiration` idea from tus).
|
||||
|
||||
### 7. Tombstones, blob GC, retention, quotas
|
||||
|
||||
The current schema already has the right shape: `files(vault, path, hash, size,
|
||||
mtime_ms, seq, deleted)` with a `deleted` flag, so a tombstone is a row, not a
|
||||
deletion. What is missing:
|
||||
|
||||
- **GC**: a janitor that removes blob objects whose `blob_id` is referenced by no
|
||||
live row and no row inside the retention window. Must run after the tombstone
|
||||
grace period, and must be crash-safe (mark-then-delete, never delete-then-mark).
|
||||
- **Retention of old versions**: the current `files` table keeps only the *latest*
|
||||
row per path. Keeping history means a second table (`file_versions(vault, path,
|
||||
hash, size, seq, created_ms)`) so the server can serve "restore this revision".
|
||||
Obsidian Sync and restic both keep version history; users expect undo-across-
|
||||
devices.
|
||||
- **Bucket lifecycle as the safety net**: MinIO lifecycle rules can expire old
|
||||
object versions (`--noncurrent-expire-days`), expire delete markers
|
||||
(`--expire-delete-marker`), or nuke everything after N days
|
||||
(`--expire-all-object-versions`). Because the app's own metadata is the source of
|
||||
truth for "what is live", MinIO expiry must be configured **looser** than the app's
|
||||
retention, never tighter.
|
||||
- **Quotas**: MinIO has bucket quotas and QoS caps, but the docs are explicit that
|
||||
the quota scanner is **periodic**, so a bucket can exceed its quota between
|
||||
cycles. Enforce the per-vault byte quota **in the app** (sum of live blob sizes)
|
||||
and answer `429` with a `Retry-After`; use the MinIO bucket quota as a backstop.
|
||||
|
||||
### 8. Backup and restore of the server
|
||||
|
||||
Three distinct things need backup, and they are not the same operation:
|
||||
|
||||
1. **Blobs (MinIO bucket)**: enable bucket versioning, and use **bucket or site
|
||||
replication** for off-cluster copies. MinIO's own docs are explicit that
|
||||
replication **requires versioning on both ends**, and that lifecycle expiry does
|
||||
not replicate delete operations. Object lock (GOVERNANCE/COMPLIANCE) gives
|
||||
write-once-read-many if we want a tamper-evident copy.
|
||||
2. **Metadata (SQLite)**: take a consistent snapshot with `VACUUM INTO` (safe while
|
||||
the server runs) rather than copying the file; ship it to the backup bucket.
|
||||
3. **Restore drill**: the only backup that counts is one you have restored. Restore
|
||||
into a scratch bucket, replay the file records, and verify the SHA-256 of every
|
||||
blob against the record. This is a script, and it should run in CI against a
|
||||
seeded vault.
|
||||
|
||||
restic's own repository format is worth stealing two ideas from: content-addressed
|
||||
storage ids (so corruption is detectable by comparing the file name to its hash)
|
||||
and a strict **write order** (pack files → index → snapshot) that makes crash
|
||||
recovery deterministic.
|
||||
|
||||
### 9. MinIO / S3 blob backend
|
||||
|
||||
`opendal-service-s3` 0.58.2 (MSRV 1.91) is the strongest candidate: one dependency
|
||||
gives read/write/stat/list/copy/delete/restore/presign, explicit MinIO support
|
||||
(`endpoint` + `region` + credentials, path style by default), versioning-aware
|
||||
`restore`/`restore_with`, and SSE options. It also exposes `Operator::info()` so we
|
||||
can probe capabilities instead of guessing. The split-crate layout
|
||||
(`opendal-core` + `opendal-service-s3`) keeps the dependency graph shallow.
|
||||
|
||||
Alternatives, **unverified**: `aws-sdk-s3` (official, heavier), `rust-s3`
|
||||
(community), or plain `reqwest` + hand-rolled SigV4 (do not).
|
||||
|
||||
Decisions to make explicitly: path style vs virtual host style (MinIO is usually
|
||||
path style); whether the server encrypts at rest with SSE-S3/SSE-KMS in addition to
|
||||
client-side E2EE (belt and braces — but note that SSE keys live with the server, so
|
||||
it does not help against a malicious server, only against a stolen disk).
|
||||
|
||||
### 10. Observability and health
|
||||
|
||||
- `/healthz` already exists in `routes.rs`. Add `/readyz` that actually checks
|
||||
MinIO reachability and SQLite writability — a liveness probe that returns "ok"
|
||||
while the bucket is unreachable moves the failure from Traefik to the client.
|
||||
- Expose Prometheus metrics: request rate/latency by route, active long-polls,
|
||||
blob bytes stored per vault, sync sequence lag, pairing failures, upload sessions
|
||||
in flight. Crate names **unverified** (`prometheus` vs `metrics`).
|
||||
- Structured logs with `tracing` (**unverified** crate version) plus a request-id
|
||||
middleware, so a client can quote a request id in a bug report.
|
||||
- The long-poll `GET /changes?wait=25` is the interesting metric: it holds
|
||||
connections open, so connection count and long-poll duration are the two numbers
|
||||
that predict trouble.
|
||||
- Alert on: pairing failure spikes, token revocation events, quota rejections, GC
|
||||
failures (a failing GC means unbounded blob growth), and restore-verification
|
||||
failures.
|
||||
|
||||
### 11. Behind Traefik with TLS and Cloudflare Tunnel
|
||||
|
||||
The reference pattern (Cloudflare → `cloudflared` → Traefik → service) is
|
||||
documented in practice; the parts that matter for NotesCraft:
|
||||
|
||||
- `cloudflared` runs **inside** the cluster, so there are **no inbound ports and no
|
||||
public IPs**; DNS CNAMEs point at the tunnel's `*.cfargotunnel.com` address.
|
||||
- Each hostname maps to the Traefik service with `origin_server_name` set so
|
||||
cloudflared presents the right **SNI**, plus `http2_origin = true`.
|
||||
- Traefik must trust Cloudflare's IP ranges in `forwardedHeaders.trustedIPs`,
|
||||
otherwise `X-Forwarded-For` is stripped and every request appears to come from
|
||||
the `cloudflared` pod. This directly breaks IP-based rate limiting (§4).
|
||||
- TLS: ACME DNS challenge through Cloudflare, or let Cloudflare terminate and
|
||||
Traefik serve plain HTTP internally. The guide's TLSOption uses TLS 1.2/1.3,
|
||||
AEAD-only cipher suites, `sniStrict: true`.
|
||||
- Rate limiting per route with `sourceCriterion.requestHeaderName` pointing at the
|
||||
header the edge sets — otherwise every client shares one bucket, as noted above.
|
||||
- **Timeouts vs long-poll**: the guide sets `readTimeout=60s` and
|
||||
`writeTimeout=0s` (disabled) because long-lived connections get killed
|
||||
otherwise. NotesCraft's `wait=25` long-poll needs a read timeout comfortably
|
||||
above 25 s plus the change-detection jitter; mobile clients must also tolerate
|
||||
the proxy closing idle connections, so the client should reconnect on EOF rather
|
||||
than treat it as an error.
|
||||
- Client IP for the audit log: use the header the edge sets (`CF-Connecting-IP` or
|
||||
`X-Real-Client-Ip`), not `RemoteAddr`.
|
||||
|
||||
### 12. Adjacent features: who has them, and how they map onto NotesCraft
|
||||
|
||||
- **Publish a note as a web page.** Obsidian Publish is the product to copy:
|
||||
hover previews, graph view, stacked pages, backlinks, custom themes via CSS/JS,
|
||||
custom domain, **password protection with multiple passwords**, SEO metadata per
|
||||
page, optional analytics, and mobile-friendly output. NotesCraft's version can be
|
||||
much simpler: the server already has the Markdown; render to static HTML on the
|
||||
homelab, push to the bucket, serve via the same Traefik host. No wasm build
|
||||
needed. NotesCraft's advantage: publish from the mobile app directly, and no
|
||||
third-party hosting.
|
||||
- **Web clipper.** Obsidian Web Clipper is a browser extension with templates,
|
||||
highlights, meta-tag/Schema.org/selector extraction and template variables; it
|
||||
emphasises "your clipped content is stored locally in your vault" and
|
||||
"no lock-in — exports to JSON". A browser extension is JavaScript, which clashes
|
||||
with the Rust-only rule, so the NotesCraft-native equivalent is a
|
||||
**share-to-endpoint**: any app POSTs a URL + extracted text to
|
||||
`POST /v1/clip` on the sync server and the vault gets a note. That keeps the
|
||||
client Rust (mobile share sheet, desktop CLI) and lets a browser extension be an
|
||||
optional, third-party convenience later.
|
||||
- **Shared read-only vault links.** Obsidian Publish's password protection is the
|
||||
model. For NotesCraft this needs per-link capability keys: generate a random
|
||||
read key, encrypt a subset manifest under it, and serve a read-only HTML view.
|
||||
Do not reuse the sync VMK for shares — a leaked share must not unlock the vault.
|
||||
- **Email-to-note.** The "Email to Vault" Obsidian plugin does exactly this: a
|
||||
disposable Mail.tm mailbox or your own IMAP server, periodic polling, each message
|
||||
becomes a note with From/To/CC/Subject/Body/attachments, templated path and
|
||||
frontmatter, and it marks messages read only optionally. NotesCraft can do the same
|
||||
with a Rust IMAP client (**crate name unverified**) or with an inbound SMTP
|
||||
relay in front of the sync server.
|
||||
- **Voice memo transcription.** `whisper-rs` 0.16.0 (Unlicense) is the only
|
||||
credible Rust path; whisper.cpp is a C++ dependency with cmake build, and models
|
||||
must be shipped or downloaded. Running it **server-side on the homelab** is much
|
||||
cheaper than shipping a model to every phone: the phone uploads the audio (already
|
||||
part of the attachment pipeline), the server transcribes, the transcript is written
|
||||
back as a note.
|
||||
- **OCR of scanned pages.** Two Rust paths: `leptess` (bindings to Tesseract +
|
||||
Leptonica, MIT, needs C/C++ dev libs and language data; last push 2024-01) and a
|
||||
newer pure-Rust `ocr` crate (Apache-2.0, no C deps, Markdown/hOCR/JSON output) —
|
||||
but that one is a young project (1 star) whose own README says the CRNN model
|
||||
training is **blocked** and that production use should stick to the deterministic
|
||||
pattern-matching engine. Both point the same way: **run OCR server-side**, not on
|
||||
the phone.
|
||||
|
||||
---
|
||||
|
||||
## Feature table
|
||||
|
||||
Effort: S ≈ ≤ 2 days, M ≈ 1–2 weeks, L ≈ > 2 weeks (per developer, including tests).
|
||||
Priority: **P0** must, **P1** should, **P2** later.
|
||||
|
||||
| Feature | Who has it | Why it matters for NotesCraft | Effort | Priority |
|
||||
|---|---|---|---|---|
|
||||
| Vault master key (VMK) + per-file key derivation | Obsidian Sync (scrypt→HKDF→AES-GCM); Syncthing (folder key → per-file key) | The base primitive for any E2EE story; without it the server can read every note | L | P0 |
|
||||
| Client-side XChaCha20-Poly1305 encryption of note content | Obsidian Sync (AES-256-GCM), Syncthing (XChaCha20-Poly1305) | The server must be unable to read the vault; XChaCha's 24-byte nonce removes nonce-reuse risk | M | P0 |
|
||||
| Argon2id passphrase → KEK, with `zeroize` on secret buffers | age (scrypt), Obsidian Sync (scrypt) | Turns "my vault password" into a wrapped VMK instead of a raw secret; Argon2id is the modern choice | S | P0 |
|
||||
| Recovery key (printable VMK backup) | age key files, restic password files, Obsidian (password is the recovery key) | The only path back after losing both the passphrase and a device | S | P0 |
|
||||
| Key wrapping to device public keys (X25519) | age (`x25519` recipient), Syncthing folder keys | Pairing a phone must not require re-typing the passphrase | M | P0 |
|
||||
| Path/filename encryption with opaque blob ids | Syncthing (AES-SIV names + synthetic tree) | Today the server stores plaintext `path` in SQLite; path privacy is the difference between "server is trusted" and "server is blind" | L | P0 |
|
||||
| Encrypted manifest (path↔blob mapping) | Obsidian Sync (path/content mapping stays server-readable — documented as a limitation) | Closes the substitution attack Obsidian documents: a compromised server can currently re-map content to another path | L | P1 |
|
||||
| QR device pairing | Obsidian Sync / most messengers; `qrcodegen` crate | The spec already says "shown on desktop as QR"; `qrcodegen` is a 2022-vintage but complete MIT library | S | P0 |
|
||||
| Pairing code rate limit + attempt lockout | `tower_governor` (governor backend, axum feature) | Brute-forcing a short pairing code is the cheapest attack on the whole system | S | P0 |
|
||||
| Access + refresh token lifecycle, rotation, revocation | OAuth-style token rotation; `tower_governor` `secure()` preset | The spec's acceptance criterion demands 401 on a revoked token; the current `devices` table has no expiry/revocation columns | M | P0 |
|
||||
| Audit log (auth, admin, key rotation, export) | OWASP Logging Cheat Sheet; the Traefik/Alloy/Loki pattern in the homelab | Required to answer "which device did what" after a compromise; the homelab already has a log pipeline to reuse | M | P0 |
|
||||
| Device list + "revoke this device" UI | Obsidian Sync device management | Single-owner, many-devices means a lost phone is a normal event, not an exception | S | P0 |
|
||||
| Server-side path guard + reserved-path rejection | Already in `crates/sync` (`check_syncable`) | Already built; keep it as the single source of truth for both client and server | S | P0 (done) |
|
||||
| Health/readiness endpoints | k3s probes; Traefik `/ping` | `/healthz` exists but does not check MinIO/SQLite; a false "ok" hides the outage | S | P0 |
|
||||
| Prometheus metrics + structured logs | Traefik → Alloy → Loki → Prometheus in the homelab | Reuse the existing collector instead of inventing a new sink | M | P1 |
|
||||
| Chunked/CDC dedup for large attachments | restic (Rabin fingerprint, 64-byte window, 21-bit mask, 512 KiB–8 MiB chunks) | Only matters for big attachments; the CDC implementation is the hard part | L | P1 |
|
||||
| Resumable uploads (chunked `PUT` + offset) | tus 1.0.0 (`HEAD`/`PATCH`/`Upload-Offset`) | Mobile uploads die on network changes; without resume, a 20 MB PDF restarts from zero | M | P1 |
|
||||
| Blob GC (janitor over unreferenced blobs) | restic `prune`; MinIO lifecycle rules | Unbounded blob growth is the classic sync-server leak | M | P1 |
|
||||
| Version history + restore-a-revision | Obsidian Sync, restic snapshots, Syncthing versioning | "Restore the note I had yesterday" is the feature users miss most once sync exists | L | P1 |
|
||||
| Retention policy (tombstone grace, old-version expiry) | MinIO lifecycle (`--noncurrent-expire-days`) | Bounds disk growth and defines what the server keeps after a delete | M | P1 |
|
||||
| Per-vault quota + 429 with `Retry-After` | MinIO bucket quotas (periodic scanner) | MinIO's own docs say the scanner is periodic, so enforce the quota in the app | M | P1 |
|
||||
| MinIO/S3 backend via OpenDAL | OpenDAL `services-s3` (MinIO, versioning, SSE) | Decouples the server from local disk and makes replication/backup standard S3 operations | M | P1 |
|
||||
| Server backup: bucket versioning + replication + `VACUUM INTO` snapshot | MinIO site/bucket replication; restic repository format | Replication requires versioning on both ends; the metadata DB is the piece people forget | M | P1 |
|
||||
| Automated restore drill (restore → verify every blob hash) | restic `check`; the CI idiom | An untested backup is a hypothesis | M | P1 |
|
||||
| Cloudflare Tunnel + Traefik deployment with TLS | Cloudflare Tunnel → cloudflared → Traefik pattern | No inbound ports on the homelab; the pattern is documented end-to-end | M | P1 |
|
||||
| Long-poll-friendly proxy timeouts + client reconnect on EOF | Traefik `readTimeout` guidance in the tunnel guide | The `wait=25` long-poll breaks behind proxies with short read timeouts | S | P1 |
|
||||
| IP-aware rate limiting behind the tunnel (`SmartIpKeyExtractor` / edge header) | `tower_governor` key extractors | Without this, every client shares one token bucket and legit users get 429s | S | P1 |
|
||||
| Publish a note as a web page | Obsidian Publish (hover previews, graph view, backlinks, themes, password protection) | The cheapest "share my thinking" feature and it reuses the server + Markdown already built | M | P1 |
|
||||
| Share-to-endpoint web clipper (URL + selection → note) | Obsidian Web Clipper (browser extension, templates, highlights) | Keeps the client Rust-only: the phone's share sheet and a desktop CLI both hit the same endpoint | M | P1 |
|
||||
| Email-to-note | "Email to Vault" Obsidian plugin (Mail.tm or own IMAP, templated notes) | Receipts and confirmations arrive by mail; capturing them is a real daily-use win | M | P1 |
|
||||
| Voice memo transcription | `whisper-rs` 0.16.0 (whisper.cpp) | Dictation is the fastest capture mode on a phone; run it server-side so no model ships to the phone | M | P1 |
|
||||
| OCR of scanned pages | `leptess` (Tesseract binding) and the young pure-Rust `ocr` crate | Phone-camera notes become searchable text; server-side keeps the binary small | M | P2 |
|
||||
| Shared read-only vault links (capability URLs) | Obsidian Publish password protection | Needs per-link keys separate from the VMK, so a leaked link cannot unlock the vault | L | P2 |
|
||||
| Publish + analytics / SEO / custom domain | Obsidian Publish | Nice-to-have once publishing exists; can be delegated to the edge (Traefik/Cloudflare) | S | P2 |
|
||||
| SSO (OIDC/Logto) instead of pairing codes | The spec's Open Question 4 | Pairing codes are right for one owner; SSO only matters if other humans ever share a vault | M | P2 |
|
||||
| Multi-user / shared writable vaults | Obsidian Sync teams, Nextcloud | Explicitly a non-goal in `specs/mobile-and-sync/spec.md` | L | P2 |
|
||||
|
||||
---
|
||||
|
||||
## Recommendations — the 10 to build next
|
||||
|
||||
1. **Per-vault key hierarchy with client-side encryption.** *Acceptance: with E2EE enabled, a client-side capture of the server's `/changes` response and blob bodies contains no readable note text, path or filename, and the same note syncs unchanged across two devices.*
|
||||
2. **Argon2id passphrase wrapping, recovery key and device key wrapping.** *Acceptance: a new device can be paired by scanning a QR without the owner typing the passphrase, the owner can restore a vault on a clean machine from the printed recovery key alone, and the passphrase never leaves the device.*
|
||||
3. **Encrypted path/blob mapping (opaque blob ids).** *Acceptance: the server's SQLite contains no plaintext note path, yet `GET /files/{path}` still resolves for a paired client and a replayed request cannot substitute one note's content for another's path.*
|
||||
4. **QR pairing with rate-limited, attempt-locked one-time codes.** *Acceptance: a pairing code expires after its TTL, is invalidated after N failed redemptions, and the server answers 401 (not 403) for both expired and non-existent codes so it leaks nothing.*
|
||||
5. **Token rotation and device revocation.** *Acceptance: after revoking a device, its refresh token stops working and the next request with its access token returns 401 with the client showing "Sign in again" and not touching any file.*
|
||||
6. **Audit log of security-relevant events.** *Acceptance: the server writes a structured, append-only record for every pairing, token issue/rotate/revoke, quota rejection and admin action — containing device id, request id and result, and never containing a token, key or passphrase.*
|
||||
7. **Health/readiness endpoints and request-id structured logs.** *Acceptance: `/readyz` fails when MinIO is unreachable or SQLite is read-only, and every response carries a request id that also appears in the server log line for that request.*
|
||||
8. **Blob GC + retention + version history, with a per-vault quota.** *Acceptance: after deleting a file and waiting out the grace period, a GC run removes the blob and the quota total drops; an upload that would exceed the quota is rejected with 429 and a `Retry-After`, and the vault can restore a previous revision of a note.*
|
||||
9. **Resumable uploads and CDC dedup for attachments over 8 MiB.** *Acceptance: an interrupted 30 MB upload resumes from the last acknowledged offset without re-sending the prefix, and two vaults sharing an identical 30 MB attachment transfer it once, with the equality leak documented in the threat model.*
|
||||
10. **Deployment hardening: Cloudflare Tunnel → Traefik with TLS, plus server backup and a restore drill.** *Acceptance: the sync endpoint is reachable only through the tunnel with a valid certificate, IP-based rate limiting still distinguishes clients behind the tunnel, and a scheduled job restores the bucket and metadata into a scratch location and fails the build when any blob hash does not match its record.*
|
||||
|
||||
Adjacent features worth pulling forward if the team has slack, in this order:
|
||||
web clipper via a share endpoint (P1) → email-to-note (P1) → server-side voice
|
||||
transcription (P1) → publish a note as a web page (P1) → OCR (P2) → shared
|
||||
read-only links (P2).
|
||||
|
||||
---
|
||||
|
||||
## Risks
|
||||
|
||||
1. **Scope creep into a secure-multiparty product.** The spec's non-goals exclude
|
||||
E2EE multi-user sharing. A full key hierarchy with per-file keys, per-device
|
||||
wraps and rotation is the single largest item in this document; building it
|
||||
halfway is worse than not starting, because a partially encrypted vault that
|
||||
still leaks paths is harder to reason about than a plaintext one.
|
||||
2. **Metadata leakage survives the content encryption.** Even with E2EE, the server
|
||||
learns sizes, change times, device set and edit rhythm. If the notes are
|
||||
sensitive enough that this matters, the fix is not more cipher but less metadata
|
||||
(padding, batching, or a design like Syncthing's) — a different product.
|
||||
3. **Deterministic vs random trade-off is easy to get wrong.** Randomised per-upload
|
||||
keys break dedup; deterministic keys break the AEAD contract if implemented by
|
||||
hand. This must be a reviewed, documented design decision with test vectors, not
|
||||
an implementation detail. Prefer a real SIV over a hand-rolled deterministic AEAD.
|
||||
4. **Key loss is unrecoverable by construction.** Obsidian states it plainly: lose
|
||||
the E2EE password and the remote data is gone forever. The recovery key must be
|
||||
created, displayed and acknowledged at setup time, and there must be a
|
||||
"prove my recovery key works" path — before the first vault is synced.
|
||||
5. **Locks and rate limits fight the long-poll.** `GET /changes?wait=25` holds a
|
||||
connection; aggressive per-IP limits plus a proxy with a short read timeout will
|
||||
make sync flaky on mobile. Rate-limit the *write* and *pairing* paths hard and the
|
||||
long-poll path barely at all.
|
||||
6. **Chunking attacks.** If CDC is implemented, the chunker polynomial must be
|
||||
random per vault and the chunk→pack mapping randomised; otherwise an attacker who
|
||||
can upload can align boundaries and infer file structure (the restic design doc
|
||||
cites the attack literature).
|
||||
7. **Unapproved dependencies.** `chacha20poly1305`, `argon2`, `zeroize`, `rand`,
|
||||
`age`, `tower_governor`, `qrcodegen`, `whisper-rs` and `opendal` are all new
|
||||
third-party crates and need written owner approval per `AGENTS.md`. `whisper-rs`
|
||||
additionally pulls a C++ dependency (cmake, whisper.cpp) and `leptess` pulls
|
||||
Tesseract + Leptonica, which conflicts with the "pure Rust" spirit of the
|
||||
project — hence the recommendation to run both **server-side** rather than
|
||||
shipping them in the mobile app.
|
||||
8. **Server-side processing of user content raises the privacy bar.** Running
|
||||
transcription/OCR on the homelab means the server processes plaintext audio and
|
||||
images. With E2EE content encryption, the server cannot read the payloads — so
|
||||
either transcription/OCR must happen **on the device** (contradicting the
|
||||
server-side plan), or those files must be excluded from E2EE. This is an
|
||||
architectural contradiction to resolve *before* shipping either feature, not
|
||||
after.
|
||||
9. **Homegrown protocol drift.** A chunked-PUT upload protocol, a custom key-wrap
|
||||
format and a custom capability-token format are three chances to invent a
|
||||
subtly broken scheme. Prefer `age` for the key-wrap format and document the
|
||||
others as first-party, versioned protocols with published test vectors.
|
||||
10. **Backup that is never restored.** Blobs, the SQLite metadata and the recovery
|
||||
key are three different backup jobs; the recovery key usually lives on paper.
|
||||
The restore drill (recommendation 10) is the only control that catches a
|
||||
silently empty bucket or a metadata snapshot taken mid-transaction.
|
||||
11. **Cloudflare Tunnel hides the real client IP.** Any feature that depends on
|
||||
client IP (rate limiting, lockout, audit) silently degrades to per-tunnel
|
||||
granularity unless the edge header is trusted explicitly at both Traefik and
|
||||
the app.
|
||||
12. **Mature-crate rot.** `qrcodegen` has had no release since 2022 and the pure-Rust
|
||||
`ocr` crate is a 1-star project with untrained models. Both are acceptable for
|
||||
their narrow roles, but both need a pinned-version note and an escape plan.
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
All URLs accessed 2026 (this research session, 2026-10-20).
|
||||
|
||||
**E2EE design**
|
||||
- Syncthing, *Untrusted Device Encryption* (spec) — https://docs.syncthing.net/specs/untrusted.html
|
||||
- Syncthing, *Untrusted (Encrypted) Devices* (user guide) — https://docs.syncthing.net/users/untrusted
|
||||
- Syncthing, `lib/protocol/encryption.go` — https://github.com/syncthing/syncthing/blob/main/lib/protocol/encryption.go
|
||||
- T-shaped, *Understanding and verifying Syncthing's untrusted peer encryption* (2024) — https://t-shaped.nl/untrustedpeerencryption
|
||||
- Obsidian, *Security and privacy* (Sync encryption, deterministic hash, path/content binding limits) — https://obsidian.md/help/sync/security
|
||||
- Obsidian, *How to verify Obsidian Sync's end-to-end encryption* (scrypt + HKDF + AES-256-GCM) — https://obsidian.md/blog/verify-obsidian-sync-encryption/
|
||||
|
||||
**Rust crates**
|
||||
- `chacha20poly1305` 0.11.0 — https://crates.io/crates/chacha20poly1305 (and https://docs.rs/crate/chacha20poly1305/latest)
|
||||
- RustCrypto/AEADs, `chacha20poly1305 v0.11.0` release notes (MSRV 1.85, edition 2024) — https://github.com/RustCrypto/AEADs/commit/e37a978ccf0992d9053fbc039470d6527108e393
|
||||
- `argon2` 0.6.0 (RustCrypto) — https://docs.rs/crate/argon2/latest and https://docs.rs/argon2/latest/argon2/
|
||||
- `rust-argon2` 3.0.0 (sru-systems) — https://crates.io/crates/rust-argon2
|
||||
- `rand` 0.10.3 — https://crates.io/crates/rand
|
||||
- `zeroize` 1.9.1 — https://crates.io/crates/zeroize
|
||||
- `age` 0.12.1 — https://crates.io/api/v1/crates/age (crate metadata; docs at https://docs.rs/age/latest/age/)
|
||||
- `qrcodegen` 1.8.0 — https://crates.io/crates/qrcodegen (and https://docs.rs/qrcodegen/latest/qrcodegen/)
|
||||
- `tower_governor` 0.8.0 — https://crates.io/crates/tower_governor (and https://docs.rs/tower_governor/latest/tower_governor/)
|
||||
- `whisper-rs` 0.16.0 — https://crates.io/crates/whisper-rs
|
||||
- `opendal-service-s3` 0.58.2 — https://crates.io/crates/opendal-service-s3 (and https://opendal.apache.org/docs/rust/opendal/services/struct.S3.html)
|
||||
- `leptess` (Tesseract/Leptonica bindings) — https://github.com/houqp/leptess
|
||||
- Pure-Rust `ocr` crate (early project) — https://github.com/yingkitw/ocr
|
||||
|
||||
**Storage, dedup, uploads**
|
||||
- restic, *Foundation — Introducing Content Defined Chunking (CDC)* (Rabin fingerprint, 64-byte window, 21-bit mask, 512 KiB–8 MiB chunks, random polynomial) — https://restic.net/blog/2015-09-12/restic-foundation1-cdc/
|
||||
- restic, *Repository design* (storage ids, write order, pack layout, threat model) — https://github.com/restic/restic/blob/master/doc/design.rst
|
||||
- tus resumable upload protocol 1.0.0 — https://tus.io/protocols/resumable-upload
|
||||
- tus implementations (incl. `rustus`) — https://tus.io/implementations
|
||||
- MinIO, *Object Lifecycle Management* (expiry, noncurrent versions, delete markers, tiering) — https://minio.community/community/minio-object-store/administration/object-management/object-lifecycle-management.html
|
||||
- MinIO, *What You Configure on a Bucket* (versioning, object lock, replication, quotas, QoS) — https://docs.min.io/aistor/administration/bucket-configuration/
|
||||
|
||||
**Security practice**
|
||||
- OWASP, *Logging Cheat Sheet* (security events, "when/where/who/what", data to exclude, tamper protection) — https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html
|
||||
|
||||
**Deployment**
|
||||
- Erfi's Lexicanum, *Traefik on k3s: custom deployment, plugins, middlewares, and Cloudflare Tunnel* (trusted CF ranges, `X-Real-Client-Ip` rate-limit criterion, timeouts) — https://erfi.dev/guides/k3s-traefik/
|
||||
- TheDougie, *Expose K3s Services Safely with Cloudflare Tunnels and Traefik* — https://www.thedougie.com/cloudflared-traefik-k3s-setup/
|
||||
|
||||
**Adjacent features**
|
||||
- Obsidian Publish (product page: features, password protection, custom domain, analytics) — https://obsidian.md/publish?pubDate=20250311
|
||||
- Obsidian Web Clipper (templates, highlights, local storage, export) — https://obsidian.md/clipper
|
||||
- Obsidian community plugin, *Email to Vault* (Mail.tm / IMAP, templated notes, attachments) — https://community.obsidian.md/plugins/email-to-vault
|
||||
- vault-bridges, *Obsidian Email Plugin* (SMTP → API → note) — https://github.com/vault-bridges/obsidian-email-plugin
|
||||
|
||||
**Local ground truth (not web sources)**
|
||||
- `specs/mobile-and-sync/spec.md`, `crates/sync/src/*`, `apps/notescraft-sync-server/src/*` in this repository.
|
||||
@@ -0,0 +1,671 @@
|
||||
# Sync product landscape and feature catalog — NotesCraft (mobile + sync)
|
||||
|
||||
Research date: 2026-10-20. Scope: what existing notes-app and file-sync products do well or badly
|
||||
across the sync dimensions, and what NotesCraft (plain `.md` vault as source of truth, single owner,
|
||||
many devices, self-hosted homelab sync server) should adopt.
|
||||
|
||||
## Method
|
||||
|
||||
**Browsed.** Facts were gathered with web search and page fetch against *official* vendor
|
||||
documentation, official pricing pages, official GitHub repositories/READMEs and official release
|
||||
pages. Every URL below was accessed in **2026** (research date 2026-10-20). Claims that could not be
|
||||
confirmed on an official page are written `[unverified]` — nothing is filled in from memory, and no
|
||||
version number, price or API name is stated unless an official page stated it. Absence claims
|
||||
("product has no version history") are labelled as absences and are `[unverified]` where no official
|
||||
page positively states the absence. Third-party sources are used only for user complaints and are
|
||||
labelled as such. Local grounding (SPEC.md, `specs/mobile-and-sync/spec.md`, `crates/sync`, the sync
|
||||
server routes) was read from this repo to decide the "matters for NotesCraft" column.
|
||||
|
||||
## Findings
|
||||
|
||||
### Obsidian family
|
||||
|
||||
1. **Obsidian Sync — selective sync is per-vault, per-device and never includes dot-files except
|
||||
`.obsidian`.** File-type toggles are on by default for Images, Audio, Video and PDF plus a "sync all
|
||||
other types" switch; a per-vault *Excluded folders* list is managed in Settings; dot-files and
|
||||
dot-folders are always excluded except `.obsidian`. Two traps are documented: adding a file to the
|
||||
exclude list does not remove it from the remote vault, and **sync settings themselves are
|
||||
device-specific and do not sync**, so selective sync must be configured on every device.
|
||||
<https://obsidian.md/help/sync/settings> (accessed 2026)
|
||||
|
||||
2. **Obsidian Sync — version history retention is plan-shaped and asymmetric.** Notes keep **1 month on
|
||||
Standard, 12 months on Plus**; attachments keep only **two weeks**. Restore is reachable from the
|
||||
file explorer ("Open version history"), from *Settings → Sync → Deleted files* (deleted **and
|
||||
renamed** files, restored to their original location) and from a separate *Settings version
|
||||
history* for config files. The local File-recovery plugin's snapshots are **not** synced.
|
||||
<https://obsidian.md/help/sync/version-history> (accessed 2026)
|
||||
|
||||
3. **Obsidian Sync — conflict handling merges Markdown, wins on everything else.** Markdown conflicts
|
||||
are merged with Google's **diff-match-patch**; every other file type (canvases included) is
|
||||
**last-modified-wins**; settings JSON is merged by applying local keys over remote keys. Since
|
||||
Obsidian 1.9.7 the behaviour is selectable: *Automatically merge* (default, may duplicate text) or
|
||||
**Create conflict file**, which writes `original-note-name (Conflicted copy device-name YYYYMMDDHHMM).md`,
|
||||
keeps the **remote** version in the original file and puts the local edits in the conflict file.
|
||||
The choice is device-specific. <https://obsidian.md/help/sync/troubleshoot> (accessed 2026)
|
||||
|
||||
4. **Obsidian Sync — E2EE is real but its limits are published, and that honesty is the lesson.** Two
|
||||
modes per remote vault: end-to-end encryption (default, vault password that cannot be recovered) or
|
||||
standard encryption with an Obsidian-managed key. Crypto is **AES-256-GCM with scrypt**. Obsidian
|
||||
itself discloses two trade-offs: **deterministic file-hash encryption** (same content ⇒ same
|
||||
encrypted hash, so an attacker who can force uploads can test guesses) and **no cryptographic
|
||||
binding between path and content** (the server sees which device wrote what, when, and the path↔
|
||||
content mapping, so a compromised server could swap one file's content under another's path).
|
||||
<https://obsidian.md/help/sync/security> (accessed 2026)
|
||||
|
||||
5. **Obsidian Sync — limits, status UX and pricing.** Max file size **5 MB (Standard) / 200 MB
|
||||
(Plus)**; storage is account-wide across vaults and **includes version history and attachments**;
|
||||
hitting the limit **stops syncing**; no chunking protocol is published `[unverified]`. Status icon
|
||||
has four states — *Synced / Syncing / Paused / Disconnected* — and the sync log is filterable by
|
||||
All / Errors / Skipped / Merge Conflicts and categorised General / Error / Skipped / Account, but
|
||||
**does not persist after the app is closed**. Files sync only while Obsidian runs. Pricing
|
||||
<https://obsidian.md/sync>: **Sync Standard $4/user/month billed annually** (1 vault, 1 GB, 5 MB max
|
||||
file, 1-month history) and **Sync Plus $8/user/month billed annually** (10 vaults, 10 GB upgradeable
|
||||
to 100 GB, 200 MB max file, 12-month history); devices unlimited. Official weaknesses: running a
|
||||
third-party sync service alongside Sync "can lead to conflicts, resulting in duplicate or corrupted
|
||||
files"; **"Sync is not a backup"**; remote vaults are deleted after 30 days of inactive subscription;
|
||||
storage figures lag up to 30 minutes; Windows Defender can quarantine notes containing code blocks;
|
||||
filenames illegal on the receiving OS are silently skipped; iCloud Drive on Windows "may lead to
|
||||
file duplication or corruption". <https://obsidian.md/help/sync/plans>,
|
||||
<https://obsidian.md/help/sync/messages>, <https://obsidian.md/help/sync/faq> (accessed 2026)
|
||||
|
||||
6. **Obsidian Publish is a one-way publishing pipeline, not a sync service — and is priced separately.**
|
||||
Notes are selected (NEW / UNCHANGED / CHANGED dialog), filtered by included/excluded folders and by
|
||||
`publish: true|false` frontmatter (**`publish: true` overrides an excluded folder**), and rendered at
|
||||
`publish.obsidian.md/<site-id>`. Deletions must be ticked manually. There is **no per-page version
|
||||
history** `[unverified]`, **no conflict handling** (publishing before syncing can overwrite a
|
||||
collaborator's changes), and **no E2EE** — the site is public or site-password protected, hosting is
|
||||
contracted to Cloudflare. Files up to **50 MB**, site capped at **4 GB**; no chunking published
|
||||
`[unverified]`. Collaborators may publish and unpublish; only the owner configures the site and
|
||||
manages permissions, and only the owner needs a subscription. Custom domains work **only via
|
||||
Cloudflare** (CNAME → `publish-main.obsidian.md`, proxy on, SSL mode Full). Pricing
|
||||
<https://obsidian.md/publish>: **$8 per site per month billed annually, $10 billed monthly**.
|
||||
<https://obsidian.md/help/publish/publish>, <https://obsidian.md/help/publish/security>,
|
||||
<https://obsidian.md/help/publish/limitations> (accessed 2026)
|
||||
|
||||
7. **Obsidian Git is "whatever you commit", it is not a sync service, and its own maintainer warns
|
||||
against mobile.** No selective-sync UI — `.gitignore` is the policy; the plugin's own recommended
|
||||
block excludes `.obsidian/` (or just `.obsidian/plugins`, or just `workspace*.json`), plus `.trash/`
|
||||
and `.DS_Store`; changing `.gitignore` does **not** untrack already-committed files. History is
|
||||
git's (Source Control View, History View, Diff View, `git blame`). Conflicts are native git with
|
||||
in-editor *keep ours / theirs / both* buttons and a diff3 base comparison. **No E2EE** (optional GPG
|
||||
commit signing only). Large files need **Git LFS**; mobile uses `isomorphic-git` whose repo size is
|
||||
**limited by memory**. Settings/plugins/themes are **not** synced by default. Permissions come from
|
||||
the Git host; there are no per-file roles. No share links or publish (only "Open file on GitHub").
|
||||
The Source Control View *is* the status surface; there is no bandwidth saver, only cadence controls
|
||||
(auto commit-and-sync interval / after stopping edits / after latest commit). Headless sync is plain
|
||||
`git push`/`git pull` — Obsidian's docs state plainly that "syncing isn't automatic. You must manually
|
||||
push and pull changes." The plugin is **free and open source**; the cost is the Git host. The
|
||||
maintainer's own words: "The Git implementation on mobile is **very unstable**! I would not recommend
|
||||
using this plugin on mobile" — no SSH auth on mobile, no rebase merge strategy, no submodules on
|
||||
mobile, possible crash-on-clone and "buffer overflow errors"; Snap unsupported, Flatpak not
|
||||
recommended. <https://github.com/Vinzent03/obsidian-git>,
|
||||
<https://publish.obsidian.md/git-doc/Tips-and-Tricks>, <https://obsidian.md/help/sync-notes>
|
||||
(accessed 2026)
|
||||
|
||||
8. **Obsidian ships a headless client — the newest and most copyable competitive move.** Obsidian
|
||||
Headless (open beta) exposes `ob login`, `ob sync-setup`, `ob sync` / `ob sync --continuous`,
|
||||
`ob sync-config --mode bidirectional|pull-only|mirror-remote --conflict-strategy merge|conflict
|
||||
--file-types --configs --excluded-folders`, `ob sync-status` and `ob sync-unlink`, with **the same
|
||||
encryption as the desktop app**; it requires an active Sync subscription and Obsidian explicitly
|
||||
warns **not to run desktop Sync and Headless on the same device**. The same client drives Publish
|
||||
(`ob publish`, `ob publish-list-sites`, `ob publish-config`). <https://obsidian.md/help/sync/headless>,
|
||||
<https://obsidian.md/help/publish/headless> (accessed 2026)
|
||||
|
||||
### Joplin and Logseq
|
||||
|
||||
9. **Joplin's delta sync is the best-documented change-feed protocol in the market.** `GET
|
||||
/api/files/delta?cursor=CURSOR` returns create/update/delete events plus a new cursor; events are
|
||||
**compressed** (create+update → create, create+delete → NOOP, update+delete → delete); `limit` is
|
||||
advisory; an expired or invalid cursor returns an explicit **`resyncRequired`** error telling the
|
||||
client to discard the cursor and resync from the beginning — clients idle for more than **180 days**
|
||||
hit it; a delete event for a file the client does not have must be treated as a no-op, not an error.
|
||||
<https://joplinapp.org/help/dev/spec/server_delta_sync> (accessed 2026)
|
||||
|
||||
10. **Joplin's conflict model is "conflict notebook" plus, in the newer spec, a bounded three-way
|
||||
merge.** Classic behaviour: Joplin creates a **Conflict notebook**, copies the local note into it and
|
||||
lets the remote note replace the local one. The auto-merge spec keeps a **base per sync target**
|
||||
(`base_body`/`base_title`, recorded after every upload/download/conflict) and merges **line by
|
||||
line**; it falls back to a conflict note when there is no base, the item is locked/read-only, a
|
||||
side cannot be decrypted, both sides edited identical lines, or the diff exceeds
|
||||
**maxEditLength 5000 / timeout 1000 ms**. The spec records that **word-level merging was tried and
|
||||
dropped because it could silently duplicate text**. <https://joplinapp.org/help/dev/spec/conflict_auto_merge>,
|
||||
<https://joplinapp.org/help/apps/conflict> (accessed 2026)
|
||||
|
||||
11. **Joplin's history and E2EE show two patterns worth stealing and one to avoid.** Note history is
|
||||
saved every **10 minutes** (plus a 7-day catch-up rule), stored as **diffs** (text diff for
|
||||
title/body, object diff for other properties), **synced across devices**, retained **90 days** by
|
||||
default — and because revisions sync, the **minimum interval set on any device wins**, which Joplin
|
||||
documents as a wart. Viewing history is desktop-only; restoring **copies** the old version into a
|
||||
"Restored Notes" folder instead of overwriting. E2EE is **manual and one-device-at-a-time** (the
|
||||
first device creates the Master Key; enabling it in parallel on several devices can create several
|
||||
master keys), the password **cannot be recovered**, items are encrypted **only during sync**, and
|
||||
the data format is a "JED" header plus length-framed chunks so a block can be moved between files.
|
||||
<https://joplinapp.org/help/dev/spec/history>, <https://joplinapp.org/help/apps/note_history>,
|
||||
<https://joplinapp.org/help/dev/spec/e2ee> (accessed 2026)
|
||||
|
||||
12. **Joplin's limits, pricing and sync triggers.** Targets are Joplin Cloud, Nextcloud, **S3 (Beta)**,
|
||||
WebDAV, Dropbox, OneDrive and the local filesystem, each a thin driver over a filesystem-like
|
||||
interface. Cloud caps per plan: **10 MB** max note/attachment on Basic, **200 MB** on Pro/Teams,
|
||||
unlimited on Joplin Server Business; the hard mobile limit is documented bluntly — "Resources larger
|
||||
than 10 MB are not currently supported on mobile. They will crash the application when
|
||||
synchronising". Attachment download behaviour is **Always / Auto / Manual** explicitly to "save disk
|
||||
space and network bandwidth, especially on mobile". App settings, plugins and themes are **not**
|
||||
synced (only the per-target `info.json` travels). Publishing a note produces a shareable link that
|
||||
updates when the note changes; linked notes are not published and each share creates a new link.
|
||||
Uploads happen within seconds; **downloads are polled every few minutes**, and that poll model is
|
||||
the documented source of conflicts. Headless sync is `joplin sync` from the terminal (cron-friendly).
|
||||
Pricing (EUR, <https://joplinapp.org/plans>): Basic **2.99/month** monthly or **2.40/month**
|
||||
annual (2 GB), Pro **5.99 / 4.79** (30 GB), Pro 100 GB **9.99 / 7.99**, Teams **7.99 per user
|
||||
(min 2)**; Joplin Server Business **3.33 / 2.92 / 2.50 EUR per user per month** by size band
|
||||
billed annually; the OSS server wants Docker and **PostgreSQL 16.8+** for production (SQLite is
|
||||
dev-only). <https://joplinapp.org/help/apps/attachments>, <https://joplinapp.org/help/apps/sync>,
|
||||
<https://joplinapp.org/help/apps/terminal> (accessed 2026)
|
||||
|
||||
13. **Logseq's own docs forbid running two sync engines on one graph — the single most repeated
|
||||
warning in the whole landscape.** The file version's docs say "any file sync service will enable you
|
||||
to synchronize your Logseq graph" (iCloud, OneDrive, Dropbox, Syncthing all work), but Logseq Sync
|
||||
users are told to **move the graph out of any iCloud/Dropbox/OneDrive-synced folder** and that
|
||||
"only Logseq Sync should be syncing your graph"; Self-hosted LiveSync says the same ("Do not enable
|
||||
this plug-in alongside another synchronisation solution (including iCloud and Obsidian Sync)"), and
|
||||
Obsidian warns that a third-party sync service alongside Sync causes duplicated or corrupted files.
|
||||
Logseq's own issue tracker records **data loss because Logseq creates zero-byte files that iCloud
|
||||
rejects** (#5802) and journals never appearing in iCloud (#5901).
|
||||
<https://github.com/logseq/docs/blob/master/pages/Logseq%20Sync.md>,
|
||||
<https://github.com/vrtmrz/obsidian-livesync#readme>, <https://github.com/logseq/logseq/issues/5802>
|
||||
(accessed 2026)
|
||||
|
||||
14. **Logseq splits into a file version and a DB version, and each syncs differently.** Logseq OG is
|
||||
file-based (plain Markdown, so any file sync works, plus built-in git integration and a desktop
|
||||
**Git Auto-Commit** with 1–600 s intervals that does *not* push). The DB version stores a local
|
||||
SQLite database plus `assets/` and syncs through **Logseq Sync** (RTC — "Real Time Collaboration"),
|
||||
a **paid, invite-only** feature that can also be self-hosted. The file version's Logseq Sync
|
||||
encrypted the graph with **age (ChaCha20-Poly1305)**, including file names and paths, under a graph
|
||||
password that cannot be changed afterwards — and the docs admit the filename scheme is "weaker, bug
|
||||
good enough for filenames". Conflict handling moved from whole-page sync to **Smart Merge** (0.9.14):
|
||||
block-level, built on Myer's diff plus diff-match-patch, **enabled per device**, requiring a
|
||||
re-index, explicitly "still an experimental feature" and designed for *individual users* on multiple
|
||||
devices, **not** collaboration. File-version Sync kept **page history for up to a year**; the DB
|
||||
version recycles deleted pages for **30 days** and keeps **hourly automated backups, last 12**. Beta
|
||||
limits: **10 remote graphs, 10 GB per graph, 100 MB per asset**. Status UX is a cloud icon with a
|
||||
**green = synced / orange = pending** dot and a "Sync now" button; there is no filterable activity
|
||||
log `[unverified]`. Sync cadence is fixed: **mobile every 2 seconds after you stop typing, desktop
|
||||
20 seconds**. File-version Logseq Sync explicitly did **not** support collaboration; the DB version
|
||||
documents RTC and shared remote graphs, with per-node granularity `[unverified]`. Logseq Publish
|
||||
(DB) publishes to `logseq.io` behind Cloudflare with an **optional password per page**; the file
|
||||
version's "Export public pages" produced a static single-page app. Pricing: the file-version beta
|
||||
was **$5/month (Backer) or $15/month (Sponsor)** on Open Collective; the DB version is paid and
|
||||
invite-only and `logseq.com/pricing` renders no prices to a non-JavaScript client, so the current
|
||||
price is `[unverified]`. <https://github.com/logseq/docs/blob/master/db-version.md>,
|
||||
<https://blog.logseq.com/logseq-0-9-14-better-sidebars-and-smart-merge-for-sync/> (accessed 2026)
|
||||
|
||||
### Self-hosted and open-source sync engines
|
||||
|
||||
15. **Syncthing is the reference for block-level transfer, but it has no server, no merge and no iOS
|
||||
client.** Files are split into blocks of **128 KiB up to 16 MiB**, each SHA-256 hashed into a block
|
||||
list, so only differing blocks transfer; there is no per-file size cap. Conflicts are never merged:
|
||||
the loser is renamed `<file>.sync-conflict-<date>-<time>-<modifiedBy>.<ext>`, the **older mtime
|
||||
loses**, ties break on the **larger first 63 bits of the device ID**, and the conflict copy then
|
||||
propagates to every device as an ordinary file. **File versioning defaults to off**, is per
|
||||
folder/device, and only archives changes **received from other devices** — it "will not and can not
|
||||
archive the old version" of a local edit; versions land in `.stversions` and there is no GUI restore
|
||||
dialog `[unverified]`. Ignore rules live in a per-folder `.stignore` (globs, `!` negation, `(?i)`,
|
||||
`(?d)`, `#include`; first match wins). There is no E2EE against a server because there is no server:
|
||||
TLS 1.2/1.3 device-to-device with certificate fingerprints as device IDs. Folder modes are **Send &
|
||||
Receive / Send Only / Receive Only**, and "each user should run their own Syncthing instance" — no
|
||||
per-user roles. Trigger model: watcher plus **hourly full rescan** randomised to 3/4–5/4 of the
|
||||
interval, watcher events accumulated for 10 s, deletions delayed a further minute, **256 MiB**
|
||||
concurrent incoming cap. First-class headless CLI (`syncthing serve`, `syncthing cli …`,
|
||||
`decrypt`, `paths`, `generate`, `upgrade`) plus a JSON `--audit` log, "Recent Changes" and
|
||||
Prometheus metrics. **Free, open source, no device or storage limit, no hosted tier.** Official
|
||||
caveat: no official iOS client ("iOS has significant restrictions on background processing"), P2P
|
||||
requires both sides online simultaneously, and on case-insensitive filesystems the winner of a
|
||||
case conflict is "currently not predictable". <https://docs.syncthing.net/users/syncing.html>,
|
||||
<https://docs.syncthing.net/users/versioning.html>, <https://docs.syncthing.net/users/ignoring.html>,
|
||||
<https://docs.syncthing.net/users/faq.html> (accessed 2026)
|
||||
|
||||
16. **Remotely Save proves the demand for "sync to my own MinIO" and also how a sync product fails.**
|
||||
It is an Obsidian plugin that syncs to **S3/S3-compatible (Cloudflare R2, Backblaze B2, MinIO,
|
||||
Storj…)**, **WebDAV (Nextcloud, ownCloud, InfiniCloud, Synology…)**, Dropbox or OneDrive. Selective
|
||||
sync is two regex lists, `ignorePaths` and `onlyAllowPaths`; when the allow-list is non-empty
|
||||
everything — including config dirs — must be listed explicitly. There is **no user-facing version
|
||||
history or restore UI** `[unverified]`. Free tier conflict resolution is **keep-newer or keep-larger**
|
||||
(the user picks); PRO adds `smart_conflict` = "merge small markdown files, or duplicate large
|
||||
markdown files or any-size non-markdown files". A safety valve worth copying: `protectModifyPercentage`
|
||||
(default 50) **aborts** the sync if more than n% of files would be modified or deleted. E2EE is
|
||||
optional and password-based (rclone-crypt style with per-chunk XSalsa20-Poly1305 and per-path
|
||||
filename encryption, or AES-256-CBC + PBKDF2), but it is **not fully server-blind** — the vault name
|
||||
is not encrypted and directories are stored as zero-byte objects. Config is not synced by default
|
||||
(`syncConfigDir` is experimental because Obsidian rewrites config mtimes on every launch). It is
|
||||
**foreground-only** — "It's technically impossible to auto sync while Obsidian is in background" —
|
||||
and both auto modes **fail silently**. Last release **0.5.25, published 2024-10-20**; PRO is
|
||||
"FREE while beta before Jan 1, 2027" and the PRO code is source-available (PolyForm Strict).
|
||||
<https://github.com/remotely-save/remotely-save#readme>,
|
||||
<https://github.com/remotely-save/remotely-save/blob/master/docs/sync_algorithm/sync_ignoring_large_files.md>
|
||||
(accessed 2026)
|
||||
|
||||
17. **Self-hosted LiveSync is the only product found that does a real, conservative three-way merge.**
|
||||
Automatic merge uses the **nearest shared ancestor revision available in both histories** and
|
||||
explicitly refuses to guess: missing or compacted history *stops* the merge; one side deleting what
|
||||
the other modifies asks the user; **binary contents differ → "semantic merge is unavailable"**; two
|
||||
devices independently creating the same path leave two leaves with no shared base, resolved by hand.
|
||||
Its structural constraint is the one to learn from: **conflicts can only be resolved on a device
|
||||
that still holds the ancestor** — which is exactly why the ancestor revision must be retained by
|
||||
the server, not just by the client. It is also the richest observability product: status bar with
|
||||
Activity / Status / Statistical (uploaded vs downloaded chunks) / Progress indicators, "Write logs
|
||||
into the file", a redacted "Generate full report for opening the issue with debug info", and a
|
||||
conflict inspector listing the winner and every conflict leaf with revision ids, chunk availability
|
||||
and delta size. Content-defined **chunking** exists mainly to stay under CouchDB document-size
|
||||
limits, and "fetch chunks on demand" is available **on CouchDB only** — its S3-compatible mode has
|
||||
**no live push and no direct chunk fetch**. Sync modes are **LIVESYNC (continuous, CouchDB/P2P
|
||||
only) / PERIODIC / ONEVENTS**, with the battery warning "Keep replication active in the background
|
||||
— Desktop only; uses more battery and network". E2EE v2 is **AES-256-GCM + HKDF** with an
|
||||
*independent* key for document IDs and an exportable recovery code (`sls-id-v1:`), path obfuscation
|
||||
optional and **off by default for performance**, and metadata ids, revision counts and ciphertext
|
||||
lengths still visible. Config sync is split into two opt-in features — **Customisation Sync**
|
||||
(snippets/config/plugin files, needs a unique `deviceAndVaultName`) and **Hidden File Sync**
|
||||
(`.obsidian` content with a Merge/Fetch/Overwrite initialisation direction) — with a warning not
|
||||
to use both on the same files. It ships an official headless CLI (`livesync-cli sync|mirror|
|
||||
rebuild|fetch|push|pull|lock-remote` + Dockerfile), is **MIT/free**, and says the server can be a
|
||||
Raspberry Pi. Weaknesses: chunk-hash/size/splitter mismatches can pause sync, encryption/path/
|
||||
case decisions always need a human, mobile needs OS-trusted HTTPS (self-signed not supported),
|
||||
Cloudflare `524` breaks long operations, and **Garbage Collection V3 can make historical revisions
|
||||
unreadable**. <https://github.com/vrtmrz/obsidian-livesync/blob/main/docs/specs_conflict_resolution.md>,
|
||||
<https://github.com/vrtmrz/obsidian-livesync/blob/main/docs/settings.md>,
|
||||
<https://github.com/vrtmrz/obsidian-livesync/blob/main/src/apps/cli/README.md> (accessed 2026)
|
||||
|
||||
18. **Standard Notes is the reference for honest E2EE and for cursor-based item sync — and it has no
|
||||
merge at all.** Encryption is **XChaCha20-Poly1305** with **Argon2** key derivation, changes are
|
||||
encrypted *and signed*, and the self-hosted server is **zero-knowledge** — "the entirety of the
|
||||
encryption and decryption process happens completely offline". The protocol splits a root key into a
|
||||
**master key (never sent to the server)** and a server password, with per-item keys. Sync is
|
||||
client-driven request/response carrying `sync_token`, `cursor_token`, `limit` and `item_hashes`;
|
||||
the server **never pushes** and a long-poll behaviour is `[unverified]`. There is **no merge**:
|
||||
the server returns a `conflicts` array and `sync_conflict` items are surfaced to the client, with
|
||||
the official warning that simultaneous edits "may result in the duplication of notes". History is
|
||||
server-side per item with a History UI (Session History for all users, Remote History **paid
|
||||
only**, revisions ≥5 minutes apart, restore-overwrite or restore-as-copy); plan ladder: free =
|
||||
"Current session only", Productivity = **1 Year**, Professional = **Unlimited**; the client-side
|
||||
session history is ephemeral and does not survive a restart. Files are a Professional-only feature,
|
||||
no size limit (tested to 2 GB), **100 GB** encrypted storage, and no chunking is documented. There
|
||||
are **no shared vaults or per-note ACLs** — Professional allows sharing *subscription benefits* with
|
||||
up to 5 accounts (sharing one storage quota), and real-time collaboration is refused on the grounds
|
||||
that "server-side collaboration would require the server to parse content". Read-only share links
|
||||
existed only through the separate **Listed** extension, and "The Listed platform will permanently
|
||||
shut down December 31, 2026". Pricing: **Standard $0/year, Productivity $90/year, Professional
|
||||
$120/year**. Self-hosting: Docker Compose + MySQL; v2 is "4 total containers (compared to 13 for
|
||||
legacy)" with "570MB in V2 vs. 1.7GB in legacy"; minimum host "Ubuntu 22.04, with 2GB RAM and 1
|
||||
CPU"; there is **no headless client** `[unverified]`. <https://standardnotes.com/plans>,
|
||||
<https://standardnotes.com/help/3/how-does-standard-notes-secure-my-notes>,
|
||||
<https://standardnotes.com/help/47/can-i-self-host-standard-notes>,
|
||||
<https://github.com/standardnotes/snjs/blob/main/packages/snjs/specification.md> (accessed 2026)
|
||||
|
||||
### Platform and consumer baselines
|
||||
|
||||
19. **Anytype's differentiator is a CRDT-style merge, which is exactly what NotesCraft declared out of
|
||||
scope.** Its model is a local encrypted object graph (spaces, not folders), so Obsidian-style
|
||||
per-folder selective sync does not exist — detailed per-object semantics are `[unverified]`.
|
||||
Concurrent edits to the same block are merged automatically rather than duplicated; E2EE is an
|
||||
always-on product claim with a per-space recovery phrase (key-derivation parameters `[unverified]`).
|
||||
Object history with restore exists, retention window `[unverified]`. Spaces can be shared with
|
||||
member roles (exact taxonomy `[unverified]`); there is no verified public web publishing pipeline
|
||||
`[unverified]`; no filterable sync log `[unverified]`; no verified bandwidth saver `[unverified]`;
|
||||
no headless client `[unverified]`; and the price of the paid tier is `[unverified]` (read
|
||||
<https://anytype.io/pricing/> directly before quoting). Third-party reports (user complaints, not
|
||||
vendor-confirmed): heavy local storage/index cost, sync stalls on large graphs, and confusion
|
||||
between the free and paid tiers' sync limits. <https://anytype.io/>,
|
||||
<https://doc.anytype.io/> (accessed 2026)
|
||||
|
||||
20. **Apple Notes / iCloud is the baseline NotesCraft must beat on mobile reliability.** There is no
|
||||
per-note version history (only *Recently Deleted*, commonly 30 days on iOS; iCloud-side retention
|
||||
is `[unverified]`) and **no published conflict semantics** `[unverified]` — the observable behaviour
|
||||
is a last-writer-wins outcome with no merge UI and no conflict copy, and users report edits
|
||||
appearing reverted across devices (third-party). Advanced Data Protection is the E2EE outlier
|
||||
(XChaCha-style server-independent keys, recovery key or recovery contact), but it is **off by
|
||||
default**, must be enabled on every device, and any non-Apple device in the account breaks it for
|
||||
that data. Notes is not a general file store: attachments have no published byte caps and no
|
||||
chunking protocol; large files belong to iCloud Drive. Sync status is only an OS-level indicator
|
||||
plus "Manage Account Storage"; there is **no per-note activity log**. The single most cited
|
||||
complaint is that **Low Data Mode and Low Power Mode defer iCloud syncing on iPhone**, producing
|
||||
multi-hour sync lag. There is no headless path. Storage: **5 GB free**, iCloud+ at 50 GB, 200 GB,
|
||||
2 TB, 6 TB, 12 TB, 18 TB (per-region prices `[unverified]`). Apple's own position — the Notes
|
||||
store is closed and Apple does not support third-party note apps on iCloud Drive — is the
|
||||
regulatory opening NotesCraft can use. <https://support.apple.com/guide/notes>,
|
||||
<https://support.apple.com/guide/security/advanced-data-protection-adr558a4a1ca/web>,
|
||||
<https://support.apple.com/guide/iphone/manage-settings-for-apple-services-iph3c2c1d1e4/ios>
|
||||
(accessed 2026)
|
||||
|
||||
21. **Notion's offline mode is a black box cache with a published warning that it is not a backup.**
|
||||
Offline is all-or-nothing per workspace with a page-size limit (number `[unverified]`); there is no
|
||||
per-folder toggle. Version history exists with a Restore action; free plans are documented at
|
||||
7 days, other tiers `[unverified]`. No merge algorithm is published — behaviour is last-write-wins
|
||||
per page, possibly preserving disjoint block edits `[unverified]` — and there is no conflict-copy
|
||||
naming. **Notion does not offer E2EE**; "enhanced encryption" is admin-managed server-side key
|
||||
control. Upload caps and a monthly upload quota exist but the numbers are `[unverified]` unless
|
||||
read on the plan-comparison page; no chunking protocol is published. Workspace settings live
|
||||
server-side; appearance is a local device preference; there is no plugin/theme system. Sharing has
|
||||
a real role taxonomy (**Full access / Can edit / Can comment / Can view** plus guests), and
|
||||
"Publish to web" publishes a page or a whole site. Audit logs exist on paid plans; the consumer sync
|
||||
indicator is minimal and there is no per-file sync log. Offline mode reduces network use by syncing
|
||||
only pages you open. The REST/SDK API is **not** a sync client and cannot sync a local Markdown
|
||||
vault. Pricing <https://www.notion.com/pricing>: **Free $0; Plus $10 per user/month billed
|
||||
annually ($12 monthly); Business $20 ($24 monthly); Enterprise quoted**. User complaints
|
||||
(third-party): offline edits sometimes overwrite newer server content, and mobile offline mode is
|
||||
restricted. <https://www.notion.com/help/use-pages-offline>,
|
||||
<https://www.notion.com/help/duplicate-delete-and-restore-content>,
|
||||
<https://www.notion.com/help/sharing-and-permissions> (accessed 2026)
|
||||
|
||||
22. **Dropbox-style folder sync is the de-facto standard for conflict naming, version retention and
|
||||
delta feeds.** It is file-level with a cursor-based delta feed (`files/list_folder` + `cursor`;
|
||||
Google's equivalent is `files.changes`), watch + poll on the client, and **no merge**: the loser is
|
||||
named `filename (John's conflicted copy 2024-01-01).txt` and the copies then sync onward — the
|
||||
"conflicted copy sprawl" users complain about. Selective sync chooses **top-level folders**;
|
||||
ignore rules are **extension-based, not glob-based**; Google Drive offers a "Mirror vs Stream
|
||||
files" model instead, and iCloud Drive has **no selective sync at all** (only "Optimize Storage" /
|
||||
"Keep Downloaded"). Version history: **30 days free, 180 days paid**, custom up to 365 days on
|
||||
Business, plus a 30-day account-wide "Rewind"; iCloud Drive has **no per-file history**, only
|
||||
30-day Recently Deleted. Shared folders carry member roles; shared links carry view/edit,
|
||||
optional password and expiry; "Publish to web" exists on Google Drive. The client shows explicit
|
||||
sync states (up to date / syncing / paused / offline / error), a per-file Activity panel and
|
||||
account-level activity logs on paid plans. Bandwidth levers are a daily **bandwidth limit** and
|
||||
**online-only files**; Google Drive's equivalent is "Stream files"; iCloud defers upload under Low
|
||||
Power / Low Data Mode. There is a real CLI (`dropbox start|status|filesync|exclude`) where
|
||||
`dropbox filesync` is a cron-friendly one-way mirror; iCloud Drive has no CLI. No E2EE for user
|
||||
files (TLS + server-side at rest; Dropbox says so explicitly because it indexes content); iCloud
|
||||
Drive E2EE via ADP is the exception with the same all-Apple-device caveat. Pricing:
|
||||
<https://www.dropbox.com/individual> — Basic 2 GB **$0**, Standard 2 TB **$11.99/user/month**,
|
||||
Professional 3 TB **$19.99/user/month**, Team **$20/user/month**, Enterprise quoted (USD, billed
|
||||
monthly); Google Drive free 15 GB, Premium 2 TB **$9.99/month**, 30 GB **$1.99/month**;
|
||||
Workspace Business Starter **$7** / Standard **$14** / Plus **$22** per user/month billed annually.
|
||||
<https://help.dropbox.com/sync/resolve-file-conflicts>,
|
||||
<https://help.dropbox.com/sync/version-history>, <https://help.dropbox.com/sync/selective-sync>,
|
||||
<https://www.dropbox.com/cli> (accessed 2026)
|
||||
|
||||
23. **Cross-cutting: every serious product separates "note sync" from "config sync", and none syncs
|
||||
config by default.** Obsidian syncs `.obsidian` but makes **community plugins opt-in**; LiveSync
|
||||
splits it into Customisation Sync and Hidden File Sync; Remotely Save excludes config by default
|
||||
and calls `syncConfigDir` experimental; Joplin and Logseq DB sync preferences but not plugins or
|
||||
themes; Standard Notes syncs preferences as encrypted items but not extensions. For NotesCraft the
|
||||
sane default is: **notes and attachments always; settings, plugins and themes an explicit
|
||||
per-folder opt-in with its own conflict semantics**; device-local workspace state never synced.
|
||||
|
||||
24. **Cross-cutting: the durable server-side activity log is the strongest differentiator available and
|
||||
almost nobody has it.** Obsidian's log is rich but **does not persist after the app is closed**;
|
||||
LiveSync's is the richest but is client-local and only as durable as the app; Standard Notes has
|
||||
per-note history but no browsable sync-activity viewer; Logseq has a green/orange dot. With SQLite
|
||||
metadata and a monotonic sequence, a durable per-device log (upload / download / delete / conflict /
|
||||
skip, with device name, path, bytes, timestamp) is cheap and is the answer a homelab operator
|
||||
needs at 2 a.m.
|
||||
|
||||
25. **Cross-cutting: background sync on mobile is the hardest problem in the landscape, and no product
|
||||
solves it cleanly.** Obsidian syncs only while the app runs; Remotely Save is foreground-only and
|
||||
fails silently; Syncthing has no iOS client at all; Joplin polls on a timer; LiveSync's continuous
|
||||
mode is CouchDB/P2P only and warns about battery; Standard Notes never gets pushed to. NotesCraft's
|
||||
WorkManager / BGAppRefreshTask plan is *ahead* of the market, but it must be designed as three
|
||||
explicit modes (foreground / on-save / OS-scheduled) rather than one, and must degrade to
|
||||
foreground-only without losing edits.
|
||||
|
||||
## Feature table
|
||||
|
||||
| Feature | Who has it | Why it matters for NotesCraft | Effort | Priority |
|
||||
|---|---|---|---|---|
|
||||
| Cursor change feed with explicit `resync_required` signal + event compression | Joplin (`resyncRequired`, >180-day cursor expiry), Standard Notes (`sync_token`), Dropbox / Google Drive (cursor delta) | Our `/changes?since=seq` is already the right shape; without a resync signal a stale client silently diverges. Compression cuts long-poll payloads. | S | P0 |
|
||||
| Three-way merge against a stored ancestor, with "ask the user" fallback | Joplin (`base_body`/`base_title`, 5000-char / 1000 ms bounds), LiveSync (nearest shared ancestor) | The decided ancestor-hash design is rare and is our differentiator; the LiveSync rule — missing ancestor ⇒ never guess — must be in the spec. | M | P0 |
|
||||
| Conflict-copy naming + a single discoverable Conflicts view | Obsidian `(Conflicted copy device YYYYMMDDHHMM)`, Syncthing `.sync-conflict-…`, Joplin Conflict notebook, Dropbox `conflicted copy` | Our `name (conflict <device> <date>).md` matches ecosystem expectation; one view stops copies multiplying unnoticed. | S | P0 |
|
||||
| Durable server-side activity log (per device, per file) | Obsidian (client log, lost on quit), LiveSync (richest), Standard Notes (weakest) | Best trust signal a self-hosted service can offer; answers "what happened to my file while I was away". | M | P0 |
|
||||
| Sync status with explicit states + last-sync time | Obsidian (Synced/Syncing/Paused/Disconnected), Logseq (green/orange dot), Remotely Save (last successful sync) | Status bar on desktop and a badge on mobile; without it users cannot tell "not synced" from "nothing to sync". | S | P0 |
|
||||
| Three explicit sync modes (foreground / on-save / OS-scheduled) | LiveSync (LIVESYNC / PERIODIC / ONEVENTS), Joplin (upload-now, poll-download) | Mobile background limits vary per OS and per device; a user-chosen mode is honest and prevents surprise battery use. | M | P0 |
|
||||
| Guard against a second sync engine on the same vault | Logseq, LiveSync, Obsidian (all warn explicitly) | iCloud-on-Windows duplication/corruption and zero-byte-file data loss are documented; NotesCraft should detect and warn. | S | P0 |
|
||||
| Rename / delete detection by content hash | Joplin, LiveSync (logical deletion), Syncthing (does not) | Already designed; a rename must not cost a full re-upload and a delete must not resurrect. | S | P0 |
|
||||
| Version history with a restore UI and an explicit retention policy | Obsidian (1 month Standard / 12 months Plus; attachments 2 weeks), Logseq (1 year page history), Standard Notes (per plan) | Our revision ledger exists but has no restore surface and no stated retention; retention is also the main storage driver. | M | P0 |
|
||||
| Conflict inspector showing winner / losing revisions and their hashes | LiveSync (Hatch) | Lets the user resolve without leaving the app; our hashes make this cheap. | M | P1 |
|
||||
| Selective sync: include/exclude regex relative to vault root, evaluated server-side | Syncthing `.stignore`, Remotely Save (two regex lists), LiveSync (regex pair + max size) | Server-side filtering at the change-feed level is impossible for the browser-hosted plugins — a real edge for NotesCraft. | S/M | P1 |
|
||||
| Attachment tier: on-demand download, separate blob feed, per-device size cap | Joplin (Always/Auto/Manual; 10 MB mobile hard limit), Dropbox online-only, Drive "Stream files", LiveSync fetch-chunks-on-demand | Phone data plan and battery; the 25 MB server cap needs a lower mobile cap and a separate download policy. | M | P1 |
|
||||
| Resumable / chunked transfer for large blobs | Syncthing (128 KiB–16 MiB blocks), Dropbox upload sessions, Drive resumable upload, LiveSync content-defined chunks | Mobile uploads fail on flaky links; a resumable PUT is the difference between "lost 20 MB" and "resumed". | M | P1 |
|
||||
| Headless client sharing the sync engine with the GUI | Obsidian Headless, LiveSync `livesync-cli`, Syncthing CLI, Joplin Terminal | CI, seeding, server-side archival, scripted backup — and it answers "can it sync without the app". | M | P1 |
|
||||
| Config / plugin / theme sync as an explicit opt-in with separate semantics | Obsidian (opt-in community plugins), LiveSync (Customisation Sync + Hidden File Sync), Remotely Save (experimental) | Needed for a real multi-device experience, but it must not drag device-local workspace state into conflicts. | M | P1 |
|
||||
| Explicit per-device settings that never sync (device name, conflict strategy, selective rules) | Obsidian (all three are device-specific) | Prevents the classic surprise where one phone's rule silently changes another device's behaviour. | S | P1 |
|
||||
| Safety valve: abort sync when more than X% of files would change | Remotely Save (`protectModifyPercentage`, default 50) | Cheap insurance against a corrupted client or a mis-scanned vault wiping the server. | M | P1 |
|
||||
| Explicit server-blind-E2EE claims with a stated metadata leak surface | Obsidian (deterministic hashes, no path↔content binding), Standard Notes (Argon2 + XChaCha20, zero-knowledge), LiveSync (AES-GCM + HKDF, ID key, visible sizes) | A homelab owner will ask "can the server read my notes?"; the honest answer must name exactly what metadata leaks. | L | P2 |
|
||||
| Revocable read-only share links derived from a retained revision | Obsidian Publish ($8/site/month), Joplin publish-note, Logseq Publish (per-page password), Standard Notes Listed (shutting down 2026-12-31) | Genuinely underserved and a natural fit for a self-hosted service that already stores history. | L | P2 |
|
||||
| Shared vaults with per-user roles | Obsidian (max 20 collaborators, no fine-grained roles), Notion (full role taxonomy), Dropbox (shared folders) | Kept as a v1 non-goal, but the data model should not preclude it later. | L | P2 |
|
||||
| Plan-shaped retention and storage accounting with prune / largest-files view | Obsidian (history counts against the quota; sync stops at the cap) | Even self-hosted needs a storage story once MinIO fills up, and a "prune" tool is the escape hatch. | S | P2 |
|
||||
| Backup as a separate, explicitly-labelled feature from sync | Obsidian ("Sync is not a backup"), Logseq (hourly auto-backups, last 12) | Users conflate the two; a named backup makes the promise explicit and prevents the "sync ate my file" story. | S | P1 |
|
||||
| Single-writer / lock semantics for upgrades and migrations | Joplin (SYNC vs EXCLUSIVE locks with expiry) | Our monotonic `seq` plus a device lease gives this cheaply and prevents two clients upgrading a vault at once. | S | P1 |
|
||||
| Audit trail for sharing and destructive actions | Notion (audit log, paid), Dropbox (admin activity log) | Not needed for single-owner v1, but the event model should be able to record it later. | S | P2 |
|
||||
|
||||
## Recommendations — the 10 features to build next
|
||||
|
||||
1. **Land the conflict-copy rule exactly as designed, plus a single Conflicts view.**
|
||||
*Acceptance:* after a concurrent edit on two devices, the server version stays at the original path,
|
||||
the local version is written as `name (conflict <device> <date>).md`, and the Conflicts view lists
|
||||
exactly one entry offering side-by-side view and per-revision restore.
|
||||
2. **Three-way merge against the stored ancestor, with an explicit "never guess" fallback.**
|
||||
*Acceptance:* when both devices edited disjoint line ranges of a note and the common ancestor is
|
||||
retained, the merged file contains both edits and no conflict copy is created; when the ancestor is
|
||||
missing, a conflict copy is created instead of a guess, and no test ever produces a silently
|
||||
duplicated paragraph.
|
||||
3. **Add `resync_required` plus event compression to `/changes`.**
|
||||
*Acceptance:* a client whose `since` is older than the server's retained window receives a
|
||||
`resync_required` error, performs a full re-scan, and ends with byte-identical files on both sides
|
||||
and no error state left behind.
|
||||
4. **Persist a durable, per-device activity log on the server.**
|
||||
*Acceptance:* after one device uploads three files, the vault activity log shows three upload rows
|
||||
with device name, path, byte count and timestamp, and those rows are still readable after the
|
||||
client quits and restarts.
|
||||
5. **Ship version history with a restore UI and a stated retention policy.**
|
||||
*Acceptance:* a note edited five times shows five revisions with timestamps in the History panel,
|
||||
restoring revision three replaces the file with those bytes while the newest revision is retained,
|
||||
and revisions older than the configured retention are pruned from the server.
|
||||
6. **Make sync status explicit in the UI (desktop status bar and mobile badge).**
|
||||
*Acceptance:* the indicator shows exactly one of *Synced / Syncing / Paused / Offline / Error* plus
|
||||
the last successful sync time, and a revoked token flips it to *Error — sign in again* within one
|
||||
sync cycle without touching any file.
|
||||
7. **Add server-side selective-sync rules (include/exclude regex relative to the vault root).**
|
||||
*Acceptance:* a rule `^archive/` prevents any file under `archive/` from being uploaded or
|
||||
downloaded, and a direct upload of an excluded path is rejected with an error rather than stored.
|
||||
8. **Design the mobile client around three sync modes plus an on-demand attachment tier.**
|
||||
*Acceptance:* with the Android app backgrounded, a note saved five minutes earlier is uploaded by
|
||||
the OS-scheduled job within 15 minutes without the app being foregrounded, and an attachment larger
|
||||
than the mobile cap is not fetched until the user opens it.
|
||||
9. **Ship a headless `notescraft sync` client built on the same core library.**
|
||||
*Acceptance:* `notescraft sync --once --vault <path>` converges a vault with the server in a CI job
|
||||
with no GUI, no egui dependency and no display available, and exits non-zero on a conflict that
|
||||
needs a human.
|
||||
10. **Detect and refuse to compete with another sync engine.**
|
||||
*Acceptance:* opening a vault located inside a detected iCloud Drive, Google Drive, OneDrive or
|
||||
Dropbox folder shows a blocking warning naming the other engine and offering to relocate the
|
||||
vault, and starting NotesCraft sync on a vault with another engine writing to it logs an explicit
|
||||
"competing sync engine" warning.
|
||||
|
||||
## Risks
|
||||
|
||||
- **Documentation drift is the norm, not the exception.** Obsidian's behaviour changed at 1.9.7,
|
||||
Logseq split into two products with different sync models, Remotely Save's last release was
|
||||
2024-10-20, Standard Notes renamed its server twice (`syncing-server` → `syncing-server-js` →
|
||||
`standardnotes/server`), and LiveSync's manifest and docs disagree on versions. Any behaviour copied
|
||||
from this document must be re-verified against the live product before it is written into NotesCraft's
|
||||
spec.
|
||||
- **E2EE claims are easy to overstate.** Obsidian itself discloses that its encrypted hash is
|
||||
deterministic and that path and content are not cryptographically bound; LiveSync leaves document ids,
|
||||
revision counts and ciphertext lengths visible; Standard Notes is the only product found with a
|
||||
genuinely zero-knowledge design, and it pays for it with a database-first model NotesCraft rejects.
|
||||
NotesCraft's honest position is transport + at-rest encryption with a server-held key, plus an
|
||||
optional per-vault passphrase, plus an explicit list of what the server can observe
|
||||
(path, size, mtime, device id, sequence numbers).
|
||||
- **Mobile background sync cannot be promised as an SLA.** iOS schedules BGAppRefreshTask
|
||||
opportunistically, Android WorkManager is best-effort, Remotely Save proved that "background sync"
|
||||
inside an app sandbox is effectively impossible for plugin-style clients, and Obsidian syncs only
|
||||
while the app runs. NotesCraft should promise "eventually, while the app is open or the OS grants a
|
||||
background slot" and offer a foreground "sync now".
|
||||
- **Two engines on one vault corrupt data.** Documented everywhere: iCloud Drive on Windows duplicates
|
||||
or corrupts files, Logseq creates zero-byte files that iCloud rejects (data loss), and Obsidian
|
||||
warns that a third-party sync service alongside Sync produces duplicated or corrupted files. A
|
||||
NotesCraft vault inside a cloud-synced folder is a data-loss risk that must be detected, not
|
||||
documented.
|
||||
- **Conflict-copy sprawl.** Every product that duplicates the loser lets those duplicates sync onward
|
||||
and multiply; the fix is one visible Conflicts view plus a cleanup path, not more copying.
|
||||
- **Case-insensitive filesystems.** Syncthing states the winner of a case conflict is "currently not
|
||||
predictable"; macOS, Windows and Android are all case-insensitive by default, so NotesCraft's path
|
||||
rules and conflict naming must be case-aware and normalised.
|
||||
- **History storage growth and loss.** Obsidian's retention is asymmetric (attachments 2 weeks vs notes
|
||||
12 months), Joplin's synced history means the **minimum interval on any device wins**, and LiveSync's
|
||||
GC can make historical revisions unreadable. The ancestor revision must be retained at least as long
|
||||
as it can serve as a merge base — that is a hard constraint, not a nice-to-have.
|
||||
- **Storage caps that silently stop syncing.** Obsidian stops syncing at the account limit; a
|
||||
self-hosted service will hit a MinIO disk limit the same way. The server must fail loudly and
|
||||
visibly, never silently stop accepting writes.
|
||||
- **The homelab is a single point of failure.** k3s, MinIO and the sync server are all on one cluster.
|
||||
The local vault must remain authoritative and fully editable with the server down; sync may never
|
||||
block editing, and a client that cannot reach the server must degrade to local-only with a visible
|
||||
"offline" state.
|
||||
- **Thin or undocumented dimensions.** Anytype's selective-sync semantics, retention window and
|
||||
pricing; Notion's offline page limit, per-tier history retention and upload quotas; Apple's
|
||||
iCloud-side retention and non-USD pricing; LiveSync's default chunk size; Standard Notes' free-plan
|
||||
file allowance and default sync interval are all `[unverified]`. Do not quote them in product
|
||||
decisions.
|
||||
- **Competitive pressure from Obsidian's headless client.** Obsidian now ships a headless, E2EE-equal
|
||||
sync client for CI and agents. NotesCraft's headless client should not be a later convenience; it is
|
||||
the surface an AI agent or a server-side job will use first.
|
||||
|
||||
## Sources
|
||||
|
||||
All URLs accessed **2026** (research date 2026-10-20). Official vendor documentation, official pricing
|
||||
pages and official repositories unless labelled otherwise.
|
||||
|
||||
**Obsidian (Sync, Publish, Git, CLI)**
|
||||
- <https://obsidian.md/help/sync/settings>
|
||||
- <https://obsidian.md/help/sync/version-history>
|
||||
- <https://obsidian.md/help/sync/troubleshoot>
|
||||
- <https://obsidian.md/help/sync/messages>
|
||||
- <https://obsidian.md/help/sync/security>
|
||||
- <https://obsidian.md/help/sync/plans>
|
||||
- <https://obsidian.md/help/sync/vault-types>
|
||||
- <https://obsidian.md/help/sync/faq>
|
||||
- <https://obsidian.md/help/sync/collaborate>
|
||||
- <https://obsidian.md/help/sync/headless>
|
||||
- <https://obsidian.md/sync>
|
||||
- <https://obsidian.md/pricing>
|
||||
- <https://obsidian.md/help/sync-notes>
|
||||
- <https://obsidian.md/help/cli>
|
||||
- <https://obsidian.md/help/publish/publish>
|
||||
- <https://obsidian.md/help/publish/setup>
|
||||
- <https://obsidian.md/help/publish/security>
|
||||
- <https://obsidian.md/help/publish/limitations>
|
||||
- <https://obsidian.md/help/publish/media>
|
||||
- <https://obsidian.md/help/publish/sites>
|
||||
- <https://obsidian.md/help/publish/collaborate>
|
||||
- <https://obsidian.md/help/publish/domains>
|
||||
- <https://obsidian.md/help/publish/customize>
|
||||
- <https://obsidian.md/help/publish/headless>
|
||||
- <https://obsidian.md/publish>
|
||||
- <https://github.com/Vinzent03/obsidian-git>
|
||||
- <https://publish.obsidian.md/git-doc/Tips-and-Tricks>
|
||||
- <https://publish.obsidian.md/git-doc/Features>
|
||||
- <https://publish.obsidian.md/git-doc/Common+issues>
|
||||
- <https://publish.obsidian.md/git-doc/Getting+Started>
|
||||
- <https://publish.obsidian.md/git-doc/Integration+with+other+tools>
|
||||
- <https://publish.obsidian.md/git-doc/Authentication>
|
||||
- <https://community.obsidian.md/plugins/obsidian-git>
|
||||
|
||||
**Joplin**
|
||||
- <https://joplinapp.org/help/dev/spec/server_delta_sync>
|
||||
- <https://joplinapp.org/help/dev/spec/conflict_auto_merge>
|
||||
- <https://joplinapp.org/help/dev/spec/sync>
|
||||
- <https://joplinapp.org/help/dev/spec/history>
|
||||
- <https://joplinapp.org/help/dev/spec/e2ee>
|
||||
- <https://joplinapp.org/help/dev/spec/sync_lock>
|
||||
- <https://joplinapp.org/help/dev/spec/server_sharing>
|
||||
- <https://joplinapp.org/help/apps/conflict>
|
||||
- <https://joplinapp.org/help/apps/note_history>
|
||||
- <https://joplinapp.org/help/apps/attachments>
|
||||
- <https://joplinapp.org/help/apps/sync/>
|
||||
- <https://joplinapp.org/help/apps/sync/e2ee>
|
||||
- <https://joplinapp.org/help/apps/sync/nextcloud>
|
||||
- <https://joplinapp.org/help/apps/sync/s3>
|
||||
- <https://joplinapp.org/help/apps/sync/webdav>
|
||||
- <https://joplinapp.org/help/apps/terminal>
|
||||
- <https://joplinapp.org/help/apps/publish_note>
|
||||
- <https://joplinapp.org/help/apps/share_notebook>
|
||||
- <https://joplinapp.org/plans/>
|
||||
- <https://github.com/laurent22/joplin/blob/dev/packages/server/README.md>
|
||||
|
||||
**Logseq**
|
||||
- <https://github.com/logseq/docs/blob/master/db-version.md>
|
||||
- <https://github.com/logseq/docs/blob/master/db-version-changes.md>
|
||||
- <https://github.com/logseq/docs/blob/master/pages/Logseq%20Sync.md>
|
||||
- <https://github.com/logseq/docs/blob/master/pages/Logseq%20Sync%20Encryption.md>
|
||||
- <https://github.com/logseq/docs/blob/master/pages/How%20to%20sync%20your%20Logseq%20graph%20across%20devices.md>
|
||||
- <https://github.com/logseq/docs/blob/master/pages/Git%20Auto-Commit.md>
|
||||
- <https://github.com/logseq/docs/blob/master/pages/Publishing.md>
|
||||
- <https://blog.logseq.com/how-to-setup-and-use-logseq-sync/>
|
||||
- <https://blog.logseq.com/logseq-0-9-14-better-sidebars-and-smart-merge-for-sync/>
|
||||
- <https://github.com/logseq/logseq/issues/4886>, <https://github.com/logseq/logseq/issues/5802>, <https://github.com/logseq/logseq/issues/5901>
|
||||
- <https://github.com/logseq/og>, <https://github.com/logseq/logseq/releases>, <https://logseq.com/pricing>
|
||||
|
||||
**Syncthing**
|
||||
- <https://docs.syncthing.net/users/syncing.html>
|
||||
- <https://docs.syncthing.net/users/versioning.html>
|
||||
- <https://docs.syncthing.net/users/ignoring.html>
|
||||
- <https://docs.syncthing.net/users/security.html>
|
||||
- <https://docs.syncthing.net/users/foldertypes.html>
|
||||
- <https://docs.syncthing.net/users/introducer.html>
|
||||
- <https://docs.syncthing.net/users/untrusted.html>
|
||||
- <https://docs.syncthing.net/users/syncthing.html>
|
||||
- <https://docs.syncthing.net/users/faq.html>
|
||||
- <https://docs.syncthing.net/users/tuning.html>
|
||||
- <https://docs.syncthing.net/dev/rest.html>
|
||||
- <https://github.com/syncthing/syncthing/releases>, <https://syncthing.net/>
|
||||
|
||||
**Remotely Save**
|
||||
- <https://github.com/remotely-save/remotely-save#readme>
|
||||
- <https://github.com/remotely-save/remotely-save/blob/master/docs/encryption/rclone.md>
|
||||
- <https://github.com/remotely-save/remotely-save/blob/master/docs/encryption/openssl.md>
|
||||
- <https://github.com/remotely-save/remotely-save/blob/master/docs/encryption/comparation.md>
|
||||
- <https://github.com/remotely-save/remotely-save/blob/master/docs/sync_algorithm/v3/design.md>
|
||||
- <https://github.com/remotely-save/remotely-save/blob/master/docs/sync_algorithm/sync_ignoring_large_files.md>
|
||||
- <https://github.com/remotely-save/remotely-save/blob/master/docs/browser_env_no_background_after_closing.md>
|
||||
- <https://github.com/remotely-save/remotely-save/releases>, <https://remotelysave.com/>
|
||||
|
||||
**Self-hosted LiveSync**
|
||||
- <https://github.com/vrtmrz/obsidian-livesync#readme>
|
||||
- <https://github.com/vrtmrz/obsidian-livesync/blob/main/docs/settings.md>
|
||||
- <https://github.com/vrtmrz/obsidian-livesync/blob/main/docs/datastructure.md>
|
||||
- <https://github.com/vrtmrz/obsidian-livesync/blob/main/docs/specs_conflict_resolution.md>
|
||||
- <https://github.com/vrtmrz/obsidian-livesync/blob/main/docs/specs_garbage_collection.md>
|
||||
- <https://github.com/vrtmrz/obsidian-livesync/blob/main/docs/troubleshooting.md>
|
||||
- <https://github.com/vrtmrz/obsidian-livesync/blob/main/docs/tips/hidden-file-sync.md>
|
||||
- <https://github.com/vrtmrz/obsidian-livesync/blob/main/src/apps/cli/README.md>
|
||||
- <https://github.com/vrtmrz/obsidian-livesync/blob/main/utils/readme.md>
|
||||
|
||||
**Standard Notes**
|
||||
- <https://standardnotes.com/plans>, <https://standardnotes.com/features>
|
||||
- <https://standardnotes.com/help/1/who-can-read-my-private-notes>
|
||||
- <https://standardnotes.com/help/3/how-does-standard-notes-secure-my-notes>
|
||||
- <https://standardnotes.com/help/17/how-do-i-share-a-private-note>
|
||||
- <https://standardnotes.com/help/26/how-do-i-enable-note-version-history>
|
||||
- <https://standardnotes.com/help/36/how-do-i-attach-encrypted-files-to-my-notes>
|
||||
- <https://standardnotes.com/help/47/can-i-self-host-standard-notes>
|
||||
- <https://standardnotes.com/help/50/can-i-collaborate-with-others-on-a-note>
|
||||
- <https://standardnotes.com/help/55/how-can-i-optimize-the-performance-of-standard-notes>
|
||||
- <https://standardnotes.com/help/86/share-subscription-with-another-account>
|
||||
- <https://standardnotes.com/help/self-hosting/docker>
|
||||
- <https://github.com/standardnotes/server>, <https://github.com/standardnotes/snjs/blob/main/packages/snjs/specification.md>
|
||||
|
||||
**Anytype, Apple, Notion, Dropbox / Google Drive / iCloud Drive**
|
||||
- <https://anytype.io/> , <https://doc.anytype.io/> , <https://anytype.io/pricing/>
|
||||
- <https://support.apple.com/guide/notes> , <https://support.apple.com/guide/icloud>
|
||||
- <https://support.apple.com/guide/security/advanced-data-protection-adr558a4a1ca/web>
|
||||
- <https://support.apple.com/en-us/102651> , <https://support.apple.com/en-us/118247>
|
||||
- <https://support.apple.com/guide/iphone/manage-settings-for-apple-services-iph3c2c1d1e4/ios>
|
||||
- <https://www.apple.com/icloud/>
|
||||
- <https://www.notion.com/help/use-pages-offline> , <https://www.notion.com/help/duplicate-delete-and-restore-content>
|
||||
- <https://www.notion.com/help/sharing-and-permissions> , <https://www.notion.com/help/publish-and-share-your-content>
|
||||
- <https://www.notion.com/help/security-and-privacy> , <https://www.notion.com/help/audit-log> , <https://www.notion.com/pricing>
|
||||
- <https://help.dropbox.com/sync/resolve-file-conflicts> , <https://help.dropbox.com/sync/version-history>
|
||||
- <https://help.dropbox.com/sync/selective-sync> , <https://help.dropbox.com/sync/ignore-files>
|
||||
- <https://help.dropbox.com/sync/sync-status> , <https://help.dropbox.com/account-preferences/bandwidth>
|
||||
- <https://help.dropbox.com/share/shared-folders> , <https://help.dropbox.com/share/create-or-manage-shared-links>
|
||||
- <https://www.dropbox.com/individual> , <https://www.dropbox.com/business> , <https://www.dropbox.com/cli>
|
||||
- <https://www.dropbox.com/developers/documentation/http/documentation>
|
||||
- <https://support.google.com/drive/answer/13401920> , <https://support.google.com/drive/answer/2375091>
|
||||
- <https://support.google.com/drive/answer/2424824> , <https://one.google.com/about/plans>
|
||||
@@ -0,0 +1,97 @@
|
||||
# Mobile (Android, iOS) and sync
|
||||
|
||||
## Problem
|
||||
NotesCraft runs on macOS, Windows and Linux only. A vault lives on one machine, so notes cannot follow the owner to a phone, and ROADMAP lists "no mobile build" and "no bottom tab bar". Sync, mobile apps and git remotes are out of scope in SPEC 6.1.
|
||||
|
||||
## Goal
|
||||
Open the same vault on desktop, Android and iOS, edit offline anywhere, and see edits converge on every device within seconds of being online.
|
||||
|
||||
## Non-Goals
|
||||
- End-to-end encrypted multi-user sharing (single owner, many devices, v1).
|
||||
- Real-time co-editing of one note (character-level CRDT). Conflicts are file-level.
|
||||
- Obsidian Sync wire compatibility. Plain `.md` files stay the source of truth, so Obsidian and git keep working on the same folder.
|
||||
- A web (wasm) build.
|
||||
|
||||
## Findings (what the code says today)
|
||||
Desktop-only seams, all small:
|
||||
- `apps/notescraft`: `main.rs`, `desktop.rs`, `audio.rs` (cpal), `menu.rs` (muda), `notify.rs`, `keychain.rs`, `net.rs`, `google.rs`; window via `eframe::NativeOptions` with `wayland`/`x11` features.
|
||||
- `rfd` (file dialogs): no Android/iOS backend for folder pick.
|
||||
- `notify` file watcher (`crates/vault/src/watch.rs`): inotify/FSEvents; no use on iOS sandbox beyond own container, polling fallback needed.
|
||||
- `Command::new` users: `git` (`crates/git`), ACP agents (`crates/acp`), syntax fetch, `ps`/PowerShell (`perf.rs`), PDF preview, video/audio players. None can spawn processes on iOS; Android restricted. These features are desktop-only behind `cfg`.
|
||||
- Pure Rust, portable as is: `vault` `markdown` `index` `graph` `canvas` `backlinks` `blocks` `lint` `templates` `themes` `i18n` `editor` `math` `diagrams` `whiteboard` `engine` (minus control server, mcp, plugins process paths).
|
||||
- `ui-egui`: panels are floating desktop cards; width logic exists (`shell/layout.rs`) but there is no phone layout.
|
||||
|
||||
## Mobile approach
|
||||
1. Shell: keep egui. `eframe` supports Android (`android-activity`, `android-native-activity` feature) and iOS via a static lib wrapped in a thin Xcode project. New app crate `apps/notescraft-mobile` (cdylib/staticlib) calls the same `ui-egui` root. No webview, no JS.
|
||||
2. Layout: below 600 pt width switch to one column: bottom tab bar (Notes, Search, Graph, Calendar, More), full-screen note, bottom sheet for Outline/Backlinks/Related, safe-area insets from the OS, 44 pt targets, soft-keyboard toolbar above the keyboard (the "mobile bottom toolbar" already on the ROADMAP).
|
||||
3. Input: touch scroll, long-press for context menu, two-finger pinch for zoom. Vim mode off by default.
|
||||
4. Storage: vault inside app sandbox (`filesDir` on Android, Documents on iOS, Files-app visible). External folder access later via SAF / security-scoped bookmarks. Desktop-only commands report "not available on this device" through the command registry (`enabled`).
|
||||
5. Cfg split: `cfg(not(any(target_os="android", target_os="ios")))` around process-spawning features; the engine already has `enabled` on commands.
|
||||
|
||||
## Sync design
|
||||
Model: device-local vault is truth while offline. A server stores the latest revision of every file and a change log. Files only, no database inside the vault; sync state lives in `.notescraft/sync/state.json` (written via `write_private`).
|
||||
|
||||
Entities
|
||||
- `Vault` id (uuid). `Device` id. `Revision` = monotonically increasing server `seq` (u64).
|
||||
- File record: `path`, `hash` (sha-256 hex), `size`, `mtime_ms`, `seq`, `deleted`.
|
||||
|
||||
API (HTTPS, JSON, bearer device token). Base `/v1/vaults/{vault}`.
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| POST | `/v1/devices` | register a device with a one-time pairing code, returns device token |
|
||||
| GET | `/changes?since={seq}` | file records with `seq > since`, plus `head` seq; long-poll with `wait=25` |
|
||||
| GET | `/files/{path}` | download blob (`If-None-Match: hash`) |
|
||||
| PUT | `/files/{path}` | upload blob; header `X-Base-Seq` is the seq the client last saw; 201 ok, 409 `{current}` on conflict |
|
||||
| DELETE | `/files/{path}` | tombstone; same `X-Base-Seq` rule |
|
||||
| POST | `/batch/check` | body = list of `{path,hash}`; returns the hashes the server lacks (skip uploads) |
|
||||
|
||||
Client loop (desktop and mobile share one crate)
|
||||
1. Scan vault, hash changed files (size+mtime cache), build local changes.
|
||||
2. `GET /changes?since=last_seq`, apply remote changes whose local file is unchanged since last sync.
|
||||
3. Upload local changes with `X-Base-Seq`. On 409 run conflict rule.
|
||||
4. Store new `last_seq`; long-poll again. Debounce 2 s after local edits. Mobile: sync on foreground, on save, and by OS background task (WorkManager / BGAppRefreshTask).
|
||||
|
||||
Conflict rule (never lose data)
|
||||
- Both sides changed a note: keep the server version at the path, write the local version as `name (conflict <device> <date>).md`, show a Conflicts list. For `.md`, if a three-way merge against the common ancestor hash (kept in local snapshots, `.notescraft/history`) is clean, merge silently.
|
||||
- Edit vs delete: edit wins (restore the file).
|
||||
- Rename = delete + create; the content hash lets the server and client detect it and move instead of re-upload.
|
||||
- Sync never touches `.obsidian/` plugins, `.notescraft/history`, `.trash`; settings sync is opt-in per folder.
|
||||
|
||||
Server (homelab, k3s)
|
||||
- Namespace `notescraft-sync`, one Deployment behind Traefik at `sync.marcospaulo.dev.br`, TLS by cert-manager.
|
||||
- Blobs: private bucket `notescraft-sync` in the existing MinIO (HOMELAB_OPERATIONS: object semantics fit). Metadata (file records, seq, devices): SQLite on a small `local-path` PVC, or a JSON-lines log in the bucket if the owner prefers zero PVC.
|
||||
- Auth: per-device bearer token minted by pairing code (shown on desktop as QR); optional Logto SSO later. Tokens stored hashed. Rate limit, 25 MB per file cap, path guard identical to `Vault::resolve`.
|
||||
- Backup: bucket versioning on; metadata snapshot with the existing backup flow.
|
||||
- Implementation language: Rust (`apps/notescraft-sync-server`) to share protocol types in `crates/sync`.
|
||||
|
||||
Crates
|
||||
- `crates/sync` (L1): protocol types, manifest diff, conflict rules, hashing. Pure, no I/O, fully unit-tested.
|
||||
- `crates/engine`: sync client driver, commands `sync.now`, `sync.status`, `sync.pair`, `sync.conflicts`.
|
||||
- `apps/notescraft-sync-server`: the HTTP service.
|
||||
- `ui-egui`: Settings > Sync page, status-bar indicator, Conflicts view.
|
||||
|
||||
## Many vaults
|
||||
A person has many vaults. The device token is per user and device (not per vault), the client keeps one binding per local vault in `.notescraft/sync/binding.json`, and a Vaults screen connects or creates server vaults. Details and acceptance criteria: `specs/collaboration/spec.md`, section "Many vaults".
|
||||
|
||||
## Acceptance Criteria
|
||||
- [ ] Given two devices paired to one vault, when device A saves a note online, then device B has the same bytes within 10 s.
|
||||
- [ ] Given device A offline edits note N and device B edits N, when both reconnect, then no text is lost and a conflict copy exists unless a clean three-way merge applies.
|
||||
- [ ] Given a file deleted on A and edited on B, when synced, then the edited file survives on both.
|
||||
- [ ] Given an 8,000-note vault, when nothing changed, then a sync cycle sends one `/changes` request and no uploads.
|
||||
- [ ] Given a wrong or revoked token, then the server answers 401 and the client shows "Sign in again" without touching files.
|
||||
- [ ] Given path `../x` or `.notescraft/..`, then the server answers 400.
|
||||
- [ ] Given a phone at 390 pt wide, then the shell shows one column with a bottom tab bar and no horizontal scroll.
|
||||
|
||||
## Constraints
|
||||
- Stack: Rust only, egui, no JS. New third-party dependencies need owner approval (see Open Questions).
|
||||
- Never crash: no `unwrap`, bounded sizes, `checked_*` on all sequence and size maths.
|
||||
- Every disk write through `craft-atomic` / `write_private`.
|
||||
- Layering: `sync` at L1, client driver in `engine` (L4), UI in L6.
|
||||
- Server follows `HOMELAB_OPERATIONS.md`: live discovery first, MinIO for blobs, no secrets in Git, `accessctl` for ingress, explicit CPU/memory limits.
|
||||
|
||||
## Open Questions
|
||||
1. Dependencies: HTTP server (`axum` + `tokio`, or `tiny_http`), HTTP client (`ureq` with rustls), `rusqlite`, `sha2` outside engine. Approve which?
|
||||
2. Deploy to the homelab k3s now, or build and test the server locally first? Deploy mutates the cluster.
|
||||
3. Metadata store: SQLite on a PVC, or MinIO-only?
|
||||
4. Auth: pairing-code device tokens (recommended) or Logto SSO from day one?
|
||||
5. iOS needs a Mac with Xcode and an Apple developer account for device install; Android needs the NDK + `cargo-ndk`. Confirm both toolchains exist here.
|
||||
@@ -0,0 +1,63 @@
|
||||
# Tasks: Mobile and sync
|
||||
|
||||
Rules: each task is independently testable; order is execution order. `[x]` done, `[~]` running with a MiniMax agent, `[ ]` open. Task files for agents: `scripts/mm/<name>.md`; launcher `scripts/mm-agent.sh`; briefing rules `specs/agents/`.
|
||||
|
||||
- [x] 1. Pure sync logic: ledger, planner, three-way merge, conflict names, path and hash gates.
|
||||
Files: `crates/sync/`
|
||||
Done when: `cargo test -p notescraft-sync` passes (26 tests) and `cargo xtask ci` is green. (done 2026-10-19)
|
||||
- [x] 2. Sync server: pairing, device tokens, change log with long poll, file blobs, ETag, batch check.
|
||||
Files: `apps/notescraft-sync-server/`
|
||||
Done when: 6 HTTP tests pass; `cargo xtask ci` green. (done 2026-10-19)
|
||||
- [x] 3. Server hardening: pairing lockout, device list/rotate/revoke, audit log, request ids, `/readyz`, quota, blob GC.
|
||||
Files: `apps/notescraft-sync-server/` (agent brief `scripts/mm/server-hardening.md`)
|
||||
Done when: each item has an HTTP test; crate clippy clean. (done 2026-10-20 by GLM 5.3 Flash via Pi: 22 server tests; verified by the orchestrator)
|
||||
- [x] 4. Sync client driver in the engine: `Remote` trait, `HttpRemote`, `SyncState`, `sync_once`, base copies for merging.
|
||||
Files: `crates/engine/src/sync/` (brief `scripts/mm/sync-client.md`)
|
||||
Done when: 10 driver tests pass with a fake remote; engine tests and clippy green. (done 2026-10-20 by MiniMax M3.1 Flash: 12 driver tests; verified by `apps/notescraft-sync-server/tests/client_e2e.rs`, the real client against the real server)
|
||||
- [x] 5. Engine commands and worker: `sync.pair`, `sync.now`, `sync.status`, `sync.conflicts`, `sync.resolve`, `sync.devices`; background loop (debounce 2 s after edits, long poll, backoff); settings (server URL, Wi-Fi only, attachments); an end-to-end test of the driver against the real server binary.
|
||||
Files: `crates/engine/src/sync/commands.rs`, `crates/engine/src/prefs/`, `crates/engine/tests/sync_server.rs`
|
||||
Done when: two sessions converge through a real server in a test; commands listed in `commands` output. (done 2026-10-20 by GLM 5.3 Flash via Pi: `SyncWorker`, binding and token files (0600), `Vault::sibling`, `sync.pair/status/now/unlink/conflicts/resolve`, conflict resolution; 9 command tests; verified by the orchestrator. The test against the real server binary lives in `apps/notescraft-sync-server/tests/client_e2e.rs`.)
|
||||
- [x] 6. Sync UI (desktop first): Settings > Sync page, status-bar sync pill, sync sheet, Conflicts view, pairing dialog (code entry; QR later).
|
||||
Files: `crates/ui-egui/src/views/settings_view/sync/`, `panels/`, i18n tables
|
||||
Done when: driven tests push pointer events through pairing, sync now and conflict resolution; matches `design/mobile/03-09`. (done 2026-10-20 by MiniMax M3.1 Flash through the Claude Code harness: Settings > Sync, status-bar pill, conflicts with Keep mine/theirs/both, 6 driven tests, full ui-egui suite green. Still open: the phone sync sheet (design 03), QR pairing, devices list from the server.)
|
||||
- [x] 7. Compact (phone) layout: below 600 pt one column, bottom tab bar, sheets instead of side panels, 44 pt targets, keyboard toolbar, safe-area insets.
|
||||
Files: `crates/ui-egui/src/shell/layout.rs` and a new `shell/compact/` folder
|
||||
Done when: offscreen render at 390x844 matches `design/mobile/01-02`; driven tests at that size. (done 2026-10-20 by GLM 5.3 Flash via Pi: tab bar, back chevron, per-page cards, 12 tests; verified by render. Still open: sync pill and overflow menu in the top bar, keyboard toolbar, the note-card home list of design 01)
|
||||
- [ ] 8. Mobile shells spike: `apps/notescraft-mobile` (Android via `android-activity`, iOS static lib); open a vault in the app sandbox, edit, sync. Cfg-gate process-spawning features.
|
||||
Files: `apps/notescraft-mobile/`, cfg attributes in `apps/notescraft`
|
||||
Done when: an APK runs on an emulator and shows the notes list; iOS simulator build opens. Needs NDK and Xcode (owner machine). Status 2026-10-20: BLOCKED on toolchains (no Android SDK/NDK, no Xcode on the build machine); the pure crates already type-check for Android, see `portability.md`.
|
||||
- [ ] 9. End-to-end encryption: key hierarchy, Argon2id wrapping, recovery key, opaque blob ids (see `research/security-and-server.md`). Needs owner approval for crates.
|
||||
Files: `crates/sync-crypto/` (new), engine and server changes
|
||||
Done when: a capture of server traffic holds no readable text or path; two devices still converge.
|
||||
- [ ] 10. Deploy to the homelab k3s per `HOMELAB_OPERATIONS.md` (namespace, Deployment, PVC or MinIO, Traefik ingress, tunnel, backup and restore drill). Needs owner approval.
|
||||
Files: `deploy/sync-server/` manifests, runbook
|
||||
Done when: `GET /readyz` is green through `sync.marcospaulo.dev.br` and a restore drill passes.
|
||||
- [ ] 11. Attachments: chunking, resumable upload, dedup for files over 8 MiB.
|
||||
- [ ] 12. Server-side version history and restore UI (builds on the Versions tab).
|
||||
|
||||
## From the research (`research/sync-landscape.md`, `research/security-and-server.md`), ranked
|
||||
- [ ] 13. `/changes` returns `resync_required` when `since` is older than the retained window; event compression (one record per path already; add tombstone expiry). Client does a full re-scan and ends byte-identical.
|
||||
- [ ] 14. Server activity log per device (upload, download, delete, conflict; path, bytes, time) with an admin and owner endpoint, surfaced in a Sync activity panel.
|
||||
- [ ] 15. Server-side selective sync rules (include/exclude patterns per vault; excluded paths rejected, not stored) and the matching client setting.
|
||||
- [ ] 16. Headless client: `notescraft-cli sync --once --vault <path>` (no egui), non-zero exit when a conflict needs a human; used in CI as the end-to-end test against the real server binary.
|
||||
- [ ] 17. Refuse to compete with another sync engine: warn when a vault sits inside iCloud Drive, Google Drive, OneDrive or Dropbox folders, or contains a Syncthing/`.obsidian/plugins/*sync*` marker.
|
||||
- [ ] 18. Explicit sync status in the UI: Synced / Syncing / Paused / Offline / Error (sign in again) with last sync time (design 03, 07, 08).
|
||||
- [ ] 19. Mobile sync modes: foreground live, OS background job (about every 15 min), manual; attachment tier fetched on open above a size cap.
|
||||
|
||||
## From the mobile research (`research/mobile-platform.md`), ranked
|
||||
- [ ] 20. Android shell on GameActivity + `cargo-ndk` + Gradle AAB (spike first; task 8 is the umbrella).
|
||||
- [ ] 21. iOS static-lib wrapper in Xcode (spike early: not officially supported upstream).
|
||||
- [ ] 22. Sandbox vault with Files-app visibility on both platforms.
|
||||
- [ ] 23. Soft-keyboard and IME correctness (accents, CJK, composition) in the Source and Visual editors: the single hardest blocker.
|
||||
- [ ] 24. Safe-area and keyboard-aware one-column shell (builds on task 7).
|
||||
- [ ] 25. Quick capture sheet and share-target receive (`note.new_from_share` command).
|
||||
- [ ] 26. Quick switcher and command palette on phone, over the engine registry.
|
||||
- [ ] 27. Background sync via WorkManager and `BGTaskScheduler` (the worker from task 5 runs headless).
|
||||
- [ ] 28. Conflict resolution UI plus an offline queue view on phone (builds on task 6).
|
||||
- [ ] 29. Mobile toolbar above the keyboard and long-press context menus (design 02).
|
||||
|
||||
## Related spec sets
|
||||
- `specs/collaboration/` (live co-editing, sharing, many vaults, accounts): tasks in its `tasks.md`.
|
||||
- `specs/vault-launcher/` (main menu, recents catalog).
|
||||
- `specs/e2ee/` (end-to-end encryption, needs crate approval), `specs/attachments-and-history/` (chunking, resumable upload, server revisions), `specs/capture-and-publish/` (quick capture, share target, clipper, publish).
|
||||
- Agent briefs: `scripts/mm/{sync-client,server-hardening,compact-layout,collab-ot,accounts-sharing,vault-catalog}.md`; run order for the server app: server-hardening, then accounts-sharing (same files).
|
||||
@@ -0,0 +1,34 @@
|
||||
# Vault launcher (main menu)
|
||||
|
||||
## Problem
|
||||
Today the app opens the newest recent vault or throws up a folder dialog; recents live only in the File menu (5 entries, no metadata). With many local vaults, shared folders and server vaults there is no place to see them all, pick one, or manage the list. A vault is bound to one `Session` for the life of the process, so switching relaunches the app.
|
||||
|
||||
## Goal
|
||||
A launcher screen: on start (when no vault is forced), from File > Vaults, and when a vault is closed, it lists every vault the user can open, with status, and offers Open folder, Create, Connect to server and Join with a code.
|
||||
|
||||
## Non-Goals
|
||||
- Opening two vaults in one window (the session stays one vault per window; "open in new window" is allowed).
|
||||
- Cloud vault browsing without a connected server; templates gallery (rail entry is a placeholder in v1).
|
||||
|
||||
## Behaviour
|
||||
1. **Launcher shows when**: no recent vault exists or none is found on disk; the user passes `--launcher`; the user chooses File > Vaults... or closes the vault; the user holds Shift at start (desktop). A plain launch with a valid recent still reopens it directly (setting "Reopen last vault", default on).
|
||||
2. **Recent list** (design `design/mobile/14-home-vaults-desktop.png`): per vault name, path, last opened, sync pill (Synced, N changes waiting, Not synced, Offline, Error), role badge and avatars when shared, pinned first, then newest first; filter field; missing folders shown with "folder not found" and Remove from list; row menu: Open, Open in new window, Reveal in file manager, Rename, Pin/Unpin, Copy path, Remove from list (never deletes files).
|
||||
3. **Actions**: Open folder as vault (native dialog; Obsidian vaults open as they are), Create new vault (name + folder, optional "also create on server"), Connect to server (lists `GET /v1/vaults` entries not yet bound, then a folder to sync into), Join with a code (shared folder invite, see collaboration spec).
|
||||
4. **Left rail**: Vaults, Shared with me (vaults where I am not owner), Servers (accounts and servers I paired, with sign out), Templates (placeholder), Settings, Help.
|
||||
5. **Phone** (design `15-vaults-phone.png`): one column "On this phone" and "On your server" (tap Connect), buttons New vault and Join.
|
||||
6. **Data**: `AppSettings.recent_vaults` stays (compat); new `vault_catalog: BTreeMap<path, VaultMeta{name, pinned, last_opened_ms}>` with the existing repair rules (cap 200 entries, hostile data repaired not trusted). Sync status is read cheaply from `<vault>/.notescraft/sync/binding.json` and `state.json` (no network at launch); live status arrives after a vault opens or a background refresh runs.
|
||||
7. **Cap and safety**: the launcher never opens a folder at list time (no indexing); reading metadata of 200 vaults takes under 100 ms; a vault on a missing volume must not block the UI (stat on a worker thread with a 500 ms budget).
|
||||
|
||||
## Acceptance Criteria
|
||||
- [ ] Given 12 recent vaults (2 pinned, 1 missing), the launcher lists pinned first, then newest first, marks the missing one, and Remove from list drops it without touching disk.
|
||||
- [ ] Given a vault bound to a server, its row shows the sync pill derived from local files only, with no network call at launch.
|
||||
- [ ] Given Open in new window, a second process opens that vault while the launcher stays open.
|
||||
- [ ] Given Create new vault with a name that exists as a folder, the app refuses with a message and creates nothing.
|
||||
- [ ] Given keyboard only, Up/Down moves the selection, Enter opens, Cmd/Ctrl+F focuses the filter, Delete asks before removing from the list.
|
||||
- [ ] Given a 390 pt window, the launcher uses the phone layout with 44 pt targets.
|
||||
|
||||
## Tasks
|
||||
1. [x] Engine catalog (`vault_catalog`, repair, commands `vaults.list/pin/rename/remove`), brief `scripts/mm/vault-catalog.md`. Done 2026-10-20 by GLM 5.3 Flash through Pi: 19 tests, clippy clean, verified by the orchestrator.
|
||||
2. [x] `Launcher` view in `ui-egui` (desktop and phone layout), 11 driven tests, `examples/launcher.rs`. Started by GLM 5.3 Flash via Pi until its quota ended; finished by the orchestrator (fixed `format_ago` mixing days and milliseconds, Cmd/Ctrl+F focus, optimistic rename, 7 clippy lints). Polish left: pin star spacing, avatar initials, filter field box, wrapped card descriptions.
|
||||
3. [x] Host wiring in `apps/notescraft` (`launcher_window.rs`): the launcher shows when no recent vault exists or with `--launcher`; Open and Open-in-new-window start the vault as its own process; Pin, Rename, Remove go through the new pure catalog functions `set_pinned`, `set_name`, `forget` (shared with the `vaults.*` commands); every opened vault is recorded with `touch`. Not checked in a live window (this session cannot paint one). The File > Vaults... menu entry (command `ui.open_launcher`, request `VaultRequest::Launcher`) is done. Still open: Connect to server and Join with a code actions (need the sync UI).
|
||||
4. Server vaults in the launcher (needs accounts task of `specs/collaboration`).
|
||||