32 Commits

Author SHA1 Message Date
Marcos Paulo 18d02d3884 feat: assistant panel AI Elements (subagent trace, message footers, chat scroll, reasoning, plan/queue, confirmation, work summary, context meter, suggestions, model selector), checkpoint engine, object type / table formula / quick capture models; sync_sheet test isolation
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 21:01:23 -03:00
Marcos Paulo aaaa58ff80 docs: build hygiene covers test temp dirs and baseline copies
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 20:51:13 -03:00
Marcos Paulo 0437c4217a docs: exclude .worktrees from agent scratch copies
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 19:52:45 -03:00
Marcos Paulo 6fcbe3ed94 docs: build hygiene rules (disk is shared and finite)
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 19:49:50 -03:00
Marcos Paulo 12c82f8b56 docs: assistant panel spec from the shadcn AI Elements audit
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 18:59:48 -03:00
Marcos Paulo cac6b0ed39 feat: registry consumers (grammar, theme, pack), seed reader, wasm plugin host with approval, wanted scan, STIX and Lottie store lookups, plugin auto-disable persistence
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 18:35:47 -03:00
Marcos Paulo c2b48bcba4 docs: opt-level s perf check and installed size numbers
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 17:35:12 -03:00
Marcos Paulo c78fd488cc feat: registry core (model, guard, verify, store, net, install, commands), registry browser UI, signing tool, size profile and font dedupe, lazy Hebrew/Arabic fonts, htmlmd/srs/textdiff crates
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 17:28:10 -03:00
Marcos Paulo e8c8c06d1e feat: smart folder model, media time fragments, link card URL guard (MiniMax); specs for 10 more gaps (Codex) and the registry with bloat measurements
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 14:41:26 -03:00
Marcos Paulo 1cbc26429a docs: MiniMax M3.1 Flash brief template and launch procedure
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 13:39:34 -03:00
Marcos Paulo eb1245bd20 fix: tab switch freeze (unlinked scan off the UI thread); assistant: vault MCP wiring, preamble, wrapping choices, presets; 23 grammars incl. Twig; Tasks query, Templater include, icon picker
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 13:36:33 -03:00
Marcos Paulo b32650767c docs: specs and briefs for smart folders, version diff, media timestamps, clean paste, link cards, spaced repetition
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 12:18:00 -03:00
Marcos Paulo 6ed829e5bd feat: note scan into tasks, Templater inputs, note banner and title icon (MiniMax M3.1 Flash); agy feature-gap research
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 12:07:28 -03:00
Marcos Paulo 16a0786a2d feat: recurring task toggle, Templater render, icons in Explorer rows and tabs (MiniMax M3.1 Flash)
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 11:16:23 -03:00
Marcos Paulo 179a5cdc47 feat: task recurrence rules, tp.date and durations, Lucide icon catalogue and decor snapshot (MiniMax M3.1 Flash)
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 10:23:34 -03:00
Marcos Paulo 3e3a9cf7eb docs: PaperCraft designs for Bases, Maps, Tasks and Kanban (PNG exports, dark and light)
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 09:51:10 -03:00
Marcos Paulo 86abc0c219 feat: task status and toggle, Templater expression parser, decor.list/decor.set (MiniMax M3.1 Flash)
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 08:12:19 -03:00
Marcos Paulo 22b43c1254 feat: task-line parser, Templater segmenter, icon/banner property resolver (MiniMax M3.1 Flash)
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 07:03:19 -03:00
Marcos Paulo ca855c3ee4 docs: MiniMax M3.1 Flash lessons from the 2026-10-21 runs
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 06:41:51 -03:00
Marcos Paulo ca0d68f9ab feat: export.pandoc, properties.rename_all, first wasm plugin (dice) (MiniMax M3.1 Flash)
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 06:39:51 -03:00
Marcos Paulo de3ec375ae docs: specs and briefs for Tasks, Templater, Kanban and icons/banners
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 06:06:54 -03:00
Marcos Paulo 4ea2e9fd5a feat: outliner list moves, notescraft:// URI resolve, periodic notes (MiniMax M3.1 Flash)
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 05:54:18 -03:00
Marcos Paulo 6d3ba3096c feat: tags.rename and tags.merge commands, paste URL over selection (MiniMax M3.1 Flash)
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 05:11:38 -03:00
Marcos Paulo 77d538dd97 feat: Bases Cards, List, embeds, Map view, Maps settings and inline cell editing
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 04:45:32 -03:00
Marcos Paulo b9d21de5b2 feat: Bases tab with the Table view, popovers and the Maps tile service
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
.base files open as a tab: view switcher (add, duplicate, rename, delete, reorder), toolbar, status line, a raw-YAML fallback card with the parse-error line; virtualised table (only visible rows painted, pages fetched on demand) with sticky header, sort arrows, resizable columns persisted to columnSize, grouping with counts and folding, summary row, typed cells, links that open notes; Properties popover (search, reorder, formulas), Sort and Group menus and a filter builder that round-trips the docs' expression forms (Bases tasks 14-16, Claude Sonnet). Maps tile service: https-only fetcher with the project User-Agent, 5 s timeout, 256 KiB cap, image/png check, burst 8 then 1 request per second and 2 in flight, disk cache with a 7-day floor and size cap, maps.status / maps.clear_cache / maps.tile commands, maps.* settings off by default, nothing runs while disabled (task 12, MiniMax M3.1 Flash, OpenCode Zen free route). Verified with the full cargo xtask ci on a copy of the working tree (4572 tests, clippy -D warnings, layers, tree, i18n). Gaps: no Summarize menu, no cards/list/map views yet, view edits bypass base.view.*.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 03:54:54 -03:00
Marcos Paulo 224ae2718a feat: index keeps note properties, size and times; base.* engine commands
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Index: frontmatter properties (200 per note, 4 KiB per value, 500 list items, 3 levels), size and times; persistent cache format version 2 (old caches are replaced). Engine: base.list, base.read, base.properties, base.query over the index (limit 200 default, over 2000 is an error), base.write (validated; never replaces an unreadable file), base.create, base.view.add|remove|rename|duplicate|reorder, base.row.set (types.json, one undo step in an open buffer, file.name renames with links; other file.* and formula.* are read-only). Bases tasks 9, 10, 11 (Claude Sonnet). Verified with the full cargo xtask ci on a copy of the tree (4526 tests, clippy -D warnings, layers, tree, i18n). Gaps: no result cache keyed by index generation, no 15k-note perf test, base.row.set writes numbers as 5.0.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 03:23:00 -03:00
Marcos Paulo 3694e73de8 feat: Bases query pipeline: filter, sort, group, limit, summaries with lazy cells
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
run(Base, view, NoteSource, Env) with base and view filters (and/or/not), stable multi-key sort evaluating only the sort keys, groupBy and groupOrder (null group, hidden groups), limit across groups, built-in and custom summaries (Average, Min, Max, Sum, Range, Median, Stddev, Earliest, Latest, Checked, Unchecked, Empty, Filled, Unique, expressions over values), lazily memoised cells, dependency tracking so unused properties are never read; list methods sum, mean, median, min, max. 15000 rows with one filter and one sort run in well under the 500 ms bound. Bases task 7 (Claude Sonnet). Known gap: an empty groupOrder shows all groups (needs Option<Vec> in the model).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 03:03:30 -03:00
Marcos Paulo aeac86ddd2 feat: Bases model, evaluator, function library and map data model
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
.base model (filters, formulas, properties, views; unknown keys kept in order; limits and invalid-value errors), evaluator with the D5 type rules, lazy if and boolean operators, memoised formulas with cycle errors, depth cap 200, string/number/object/list/date/link/file/regexp functions with lambdas (regex through notescraft-regex), and the map model (Web Mercator, tile ids, viewport tiles, grid clustering, coordinate and colour parsing, OSM attribution constants; Paris z10 is tile 518/352). Bases tasks 2, 5, 6 (Claude Sonnet) and 8 (MiniMax M3.1 Flash, OpenCode Zen free route). Verified: notescraft-bases suites all pass, clippy -D warnings, xtask layers and tree clean. Known gaps: toFixed rounds ties to even, list.unique is quadratic.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 02:58:07 -03:00
Marcos Paulo 00726d5498 feat: Bases YAML subset parser and the expression lexer, parser and AST
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
yaml: lossless block and flow YAML subset with | and > scalars, comments, tab-indent error, limits (1 MiB, depth 12, 10000 nodes), emitter that round-trips; 9 fixtures from the Obsidian Bases docs (Bases tasks 1; written by Claude Sonnet after two MiniMax attempts produced no code). expr: tokens, AST and precedence-climbing parser with regex literals, arrays, objects, member/index/call, dependency collection, depth and length limits (task 3; MiniMax M3.1 Flash, OpenCode Zen free route). Verified: notescraft-bases 43+ tests pass, clippy -D warnings clean, xtask layers OK. The previous bases commit message claimed a YAML parser that did not exist yet.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 02:45:26 -03:00
Marcos Paulo 5e9b5d3cfb feat: audio player, code files with highlighting, ico, draw tools fixed, activity panel, calendar-title link, lightbox and shell fixes
ci / ci (macos-latest) (push) Has been cancelled
ci / ci (ubuntu-latest) (push) Has been cancelled
ci / ci (windows-latest) (push) Has been cancelled
Audio tab and note embeds with waveform, live spectrum (own FFT), seek, speed, loop and volume through ffplay/ffmpeg; code and text files open in an editable source editor with syntax highlighting, a Language chip and a banner to install, pick or ignore a grammar; ico decoding through the largest entry; info card as a clean grid; draw tools: arrows and lines clamped and clipped to the picture with a head, inline text tool, hotkeys; Activity tab with unread count and hotkey; calendar title shows a link icon and a pointing hand; explorer trash button works on press; right panel tab pills shrink so they never cover the collapse button; math: unknown command keeps a parser tooltip; test files grouped into tests/ folders (12-file rule); flaky search-order test waits for the rows. Built by Claude Sonnet and MiniMax M3.1 Flash; the whole tree verified with cargo xtask ci (fmt, clippy -D warnings, 4422 tests, layers, tree, i18n) on a copy of the working tree.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 02:31:52 -03:00
Marcos Paulo bc0b258e04 feat: engine and core crates: comment sidecars hidden and removed with the note, e2e comments test, extension tables, annotate text and arrow items
Sidecars (*.comments.json) stay out of the index and listings and follow rename and delete of their note; end-to-end comments test over a stub server (Ana comments and mentions Bob, Bob resolves, second sync changes nothing); viewer extension tables gain ico, ogv, 3gp, opus, aac; annotate crate: Edit::Replace for undoable text moves and edits, line items carry an arrow flag; syntax test fixed (TeX is a bundled grammar). Built by MiniMax M3.1 Flash and Claude Sonnet; verified with the full cargo xtask ci on the combined tree (4422 tests, clippy -D warnings, layers, tree, i18n).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 02:31:52 -03:00
Marcos Paulo 9993a66fee feat: Bases crate: YAML subset parser, value, date and duration types
notescraft-bases (L2, no I/O): hand-written lossless YAML subset parser and emitter with limits (1 MiB, depth 12, 10000 nodes), value and date types with durations, month clamp and a total order across types (Bases tasks 1 and 4). Built by MiniMax M3.1 Flash (OpenCode Zen free route). Verified as part of the full cargo xtask ci run on the combined tree.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 02:31:52 -03:00
984 changed files with 70202 additions and 1495 deletions
+8
View File
@@ -0,0 +1,8 @@
Disk is a shared, finite resource: a full system volume (`/private/tmp`) stops every agent and the harness itself (2026-10-21 incident, Bash returned ENOSPC). Rules for every agent and every build:
- Build only in `target/agent-<name>` (absolute: `CARGO_TARGET_DIR=$PWD/target/agent-<name>`), never a relative path (it creates `crates/*/target`), never in `/tmp`, never in the user's `target/debug`.
- Scratch copies: `rsync -a --delete --exclude 'target*/' --exclude '.git' --exclude '.notescraft' --exclude '.mm-agents' --exclude '/dist' --exclude '/.worktrees'` into ONE directory per agent under the scratchpad, never a second copy of a copy. Do not use the unanchored pattern `target*` (it drops `assets/icons/lucide/target.svg`): use `target*/` for directories.
- When you finish (success or blocked), delete what you made: your scratch copy and your `target/agent-<name>`, then run `scripts/clean-agent-builds.sh --check` and put the free-space line in your report. Keep a build dir only when the next task of the same agent reuses it, and say so.
- The lead deletes all `target/agent-*` and scratch copies after each commit with `scripts/clean-agent-builds.sh`. Never delete a dir that a running agent uses.
- Before a long build run `df -h /System/Volumes/Data /Volumes/Storage`; under 20 GiB free on either: stop and clean first. Never write build output, copies or logs to the Data volume when `/Volumes/Storage` has room.
- `scripts/` is gitignored (local agent tooling): the helper `scripts/clean-agent-builds.sh` lives there and in `~/.agents/scripts/`; if it is missing, do the same by hand (delete your `target/agent-<name>` and scratch copy).
- Tests must delete the temp dirs they create (guard with `Drop`/`tempfile`-style cleanup; `std::env::temp_dir()` leaks otherwise: 21k `nc-*`/`notescraft-*` dirs ate 5 GB of the system volume). Never make a full copy of the repo or a baseline checkout in `$TMPDIR`/`/private/tmp`; copy to `/Volumes/Storage` (a baseline copy in `$TMPDIR/opencode/nc-fonts-before` took 4.7 GB). `scripts/clean-agent-builds.sh` purges `nc-*`/`notescraft-*` in `$TMPDIR`.
+6 -1
View File
@@ -16,7 +16,7 @@ NotesCraft is a native, Obsidian-compatible notes app written in **Rust only** (
- One crate per concern in `crates/<name>`, package `notescraft-<name>`, `[lints] workspace = true`, registered in the root `[workspace.dependencies]` and in `xtask/src/layers.rs`.
- **Layering is enforced** by `cargo xtask layers`: a crate may depend only on strictly lower layers; nothing below L5 may use egui, eframe, winit or rfd. Layers: L0 `vault` `markdown`; L1 `index` `canvas` `diagrams` `whiteboard`; L2 `backlinks` `graph` `templates`; L3 `plugins` `themes`; L4 `engine`; L5 `editor` `mcp` `i18n`; L6 `ui-egui`.
- Shared crates come from craft-libs (`craft-atomic`, `craft-settings`, `craft-ui`, `craft-quit`, `craft-print`), pinned by git rev in the root `Cargo.toml`. Do not change their API; ask first.
- A new third-party dependency needs a written reason and owner approval. Already approved: `notify`, `pulldown-cmark`, `toml`, `wasmi`, `wat` (dev), `thiserror`, `serde`, `serde_json`, and for sync (2026-10-19): `sha2`, `axum`, `tokio`, `ureq` (rustls), `rusqlite`, and for paste (2026-10-20): `arboard` (image-data), `image` (png feature only).
- A new third-party dependency needs a written reason and owner approval. Already approved: `notify`, `pulldown-cmark`, `toml`, `wasmi`, `wat` (dev), `thiserror`, `serde`, `serde_json`, and for sync (2026-10-19): `sha2`, `axum`, `tokio`, `ureq` (rustls), `rusqlite`, and for paste (2026-10-20): `arboard` (image-data), `image` (png feature only), and for the registry (2026-10-21): `ring` (ed25519 verify, already linked through rustls).
## Never crash (outranks features)
@@ -50,6 +50,11 @@ run `cargo xtask i18n` and do not add hardcoded user strings.
Check that every module is declared in `lib.rs` and that its tests actually run. After finishing, update `ROADMAP.md` and `docs/ARCHITECTURE.md` (status, public API, numbers). Commit only when the owner asks.
## Build hygiene (disk is shared and finite)
- Build only in an absolute `target/agent-<name>` dir, keep one scratch copy per agent with anchored excludes (`target*/` for directories), and delete both when you finish; the full rules are in `.claude/rules/build-hygiene.md`. `scripts/clean-agent-builds.sh --check` reports free space; without `--check` it deletes `target/agent-*`, stray `crates/*/target` and scratch copies (`--keep name` to spare a running agent).
- Under 20 GiB free on the system volume or `/Volumes/Storage`: stop and clean before building. A full system volume stops every agent and the harness.
## Everything is a command
User-visible behaviour is an engine command (id, label, params, `enabled`, `run`) so the UI, CLI, control channel, MCP and plugins share one surface (SPEC 10). The `engine` crate does not exist yet; until it does, keep logic in pure library crates that a command can call.
Generated
+46
View File
@@ -2829,6 +2829,7 @@ dependencies = [
"grep-searcher",
"ignore",
"notescraft-annotate",
"notescraft-bases",
"notescraft-blocks",
"notescraft-calendar",
"notescraft-collab",
@@ -2846,6 +2847,7 @@ dependencies = [
"notescraft-templates",
"notescraft-themes",
"notescraft-vault",
"ring",
"serde",
"serde_json",
"thiserror 2.0.21",
@@ -2868,6 +2870,13 @@ dependencies = [
"thiserror 2.0.21",
]
[[package]]
name = "notescraft-htmlmd"
version = "0.1.0"
dependencies = [
"thiserror 2.0.21",
]
[[package]]
name = "notescraft-i18n"
version = "0.1.0"
@@ -2967,6 +2976,10 @@ dependencies = [
"thiserror 2.0.21",
]
[[package]]
name = "notescraft-srs"
version = "0.1.0"
[[package]]
name = "notescraft-sync"
version = "0.1.0"
@@ -3007,6 +3020,25 @@ dependencies = [
"thiserror 2.0.21",
]
[[package]]
name = "notescraft-tasks"
version = "0.1.0"
dependencies = [
"notescraft-markdown",
"notescraft-regex",
"notescraft-templates",
"thiserror 2.0.21",
]
[[package]]
name = "notescraft-templater"
version = "0.1.0"
dependencies = [
"notescraft-bases",
"notescraft-templates",
"thiserror 2.0.21",
]
[[package]]
name = "notescraft-templates"
version = "0.1.0"
@@ -3018,6 +3050,13 @@ dependencies = [
name = "notescraft-testkit"
version = "0.1.0"
[[package]]
name = "notescraft-textdiff"
version = "0.1.0"
dependencies = [
"thiserror 2.0.21",
]
[[package]]
name = "notescraft-themes"
version = "0.1.0"
@@ -3041,6 +3080,7 @@ dependencies = [
"notescraft-acp",
"notescraft-annotate",
"notescraft-backlinks",
"notescraft-bases",
"notescraft-calendar",
"notescraft-canvas",
"notescraft-collab",
@@ -6104,7 +6144,13 @@ checksum = "ec7a2a501ed189703dba8b08142f057e887dfc4b2cc4db2d343ac6376ba3e0b9"
name = "xtask"
version = "0.1.0"
dependencies = [
"craft-atomic",
"notescraft-engine",
"ring",
"serde",
"serde_json",
"sha2",
"toml",
]
[[package]]
+11 -1
View File
@@ -41,6 +41,11 @@ notescraft-sync = { path = "crates/sync" }
notescraft-collab = { path = "crates/collab" }
notescraft-annotate = { path = "crates/annotate" }
notescraft-calendar = { path = "crates/calendar" }
notescraft-tasks = { path = "crates/tasks" }
notescraft-textdiff = { path = "crates/textdiff" }
notescraft-htmlmd = { path = "crates/htmlmd" }
notescraft-srs = { path = "crates/srs" }
notescraft-templater = { path = "crates/templater" }
notescraft-plugins = { path = "crates/plugins" }
notescraft-themes = { path = "crates/themes" }
notescraft-engine = { path = "crates/engine" }
@@ -85,6 +90,8 @@ tokio = { version = "1", default-features = false, features = ["rt-multi-thread"
rusqlite = { version = "0.37", features = ["bundled"] }
# Blocking HTTP client with rustls (sync server tests now, the engine sync client next).
ureq = { version = "3", default-features = false, features = ["rustls", "json"] }
# ed25519 verification of registry entries (owner-approved 2026-10-21, already linked through rustls).
ring = { version = "0.17", default-features = false }
# Pure-Rust WebAssembly interpreter, same configuration as photocraft-plugins.
wasmi = { version = "2.0.0", default-features = false, features = ["std", "validate", "stable", "portable-dispatch"] }
wat = "1.228"
@@ -111,4 +118,7 @@ debug = "line-tables-only"
[profile.release]
lto = "thin"
codegen-units = 1
codegen-units = 1
# Size: owner-approved 2026-10-21 (specs/registry/measurements.md): -8 MB on macOS; crash backtraces lose symbol names.
opt-level = "s"
strip = "symbols"
+1 -1
View File
@@ -60,7 +60,7 @@ Done (crossed off):
Pending, by area:
- **Code blocks, grammars and formatting (done, 15 of 15 tasks, specs/code-blocks-and-formatting):** `toml` and `ts` fences now highlight through two original grammars embedded in `notescraft-syntax` (`crates/syntax/grammars/*.sublime-syntax`), and the batch review has a third check, "Add missing language labels": `blocks::detect_language` names the language of an unlabeled fence conservatively (shebang, PHP, HTML, XML, JSON, YAML, diff, shell prompts, Rust, Python, SQL, TOML, on the first 4 KiB and 40 non-blank lines) and each `ChangeKind::Label` is a zero-length insert right after the fence run that always needs review (`batch.start` takes `labels`, on by default). Not built — the language picker of the design (a mistyped label is fixed by the suggestion chip, and nothing offers to add a label while typing in source mode), the table-alignment checks of the design, and the "more info" card of the grammar dialog. A note with 50 code blocks draws its first frame in 15 ms median (release), 87 ms in a cold process.
- **Editor previews (done, 11 of 11 tasks, specs/editor-previews):** not built or limited — no swatches or arrows in Source mode or in rows carrying inline math, and a hex that Obsidian reads as a tag (`#cba6f7` in prose) stays a tag, so only inline code and non-tag text get swatches; `Unsupported` is reachable only if the diagram parser returns an unrenderable kind (today it answers `NoHeader` for one it does not know, so those show the Error chip). Scanning 100 rows of 10 colours (1,000 hits) takes 0.05 ms median, 0.15 ms worst, in release.
- **Obsidian basics (specs/obsidian-basics, audit in audit.md): all 18 tasks built.** Find and replace (with regex), back/forward, auto-pair, footnotes, explorer sort and drag-and-drop move, heading and list folding, search operators and `/regex/` search, vault-wide replace, quick switcher headings and blocks, random and unique notes, Open in default app, Copy Obsidian URL, pinned and reorderable tabs, link hover preview, outgoing links, split view (two panes with their own tabs; a click in a pane makes it the editing pane), spelling lint rule, settings (default view, font size, readable width, line numbers, new-note folder, link format and path, deleted files), PDF embeds drawn as an image of the page (`#page=N`; macOS PDFKit, elsewhere `pdftoppm` when installed, else a file card), audio embeds with Play/Stop through the system player, video embeds with a still, and in-note playback (ffmpeg decodes frames, ffplay the sound) when ffmpeg is installed, else Open in the system player, and a graph view with force sliders and colour groups that scales to 6.5k notes. Strict line breaks and show-hidden-files are settings too. Limits kept on purpose: video playback needs `ffmpeg` on the machine (no decoder is linked), no bundled spelling dictionary (the vault is the vocabulary). `crates/ui-egui/tests/driven_features.rs` drives tab drag, pinning, the link hover card, list folding, split view, hidden files, the regex find bar and Explorer drag-and-drop through real input events.
- **Obsidian basics (specs/obsidian-basics, audit in audit.md): all 18 tasks built.** Find and replace (with regex), back/forward, auto-pair, footnotes, explorer sort and drag-and-drop move, heading and list folding, search operators and `/regex/` search, vault-wide replace, quick switcher headings and blocks, random and unique notes, Open in default app, Copy Obsidian URL, pinned and reorderable tabs, link hover preview, outgoing links, split view (two panes with their own tabs; a click in a pane makes it the editing pane), spelling lint rule, settings (default view, font size, readable width, line numbers, new-note folder, link format and path, deleted files), PDF embeds drawn as an image of the page (`#page=N`; macOS PDFKit, elsewhere `pdftoppm` when installed, else a file card), an audio tab and audio embeds that play in the app (ffplay sound, a waveform that seeks, a live spectrum from an own FFT), video embeds with a still, and in-note playback (ffmpeg decodes frames, ffplay the sound) when ffmpeg is installed, else Open in the system player, and a graph view with force sliders and colour groups that scales to 6.5k notes. Strict line breaks and show-hidden-files are settings too. Limits kept on purpose: video playback needs `ffmpeg` on the machine (no decoder is linked), no bundled spelling dictionary (the vault is the vocabulary). `crates/ui-egui/tests/driven_features.rs` drives tab drag, pinning, the link hover card, list folding, split view, hidden files, the regex find bar and Explorer drag-and-drop through real input events.
- **Obsidian basics round 2 (specs/obsidian-basics/tasks.md 19-24): query embeds (19) and stacked tabs (21) built; editor follow-ups (23) built; pop-out windows (22) built but not seen in a native window; RTL (20) built for the reading view with `unicode-bidi` (owner approved; Arabic unjoined, Source mode logical order); round 3 (tasks 25-30, from the audit rows still missing or partial) started: note composer (`note.extract`, `note.merge`, the toolbar button and the Explorer "Merge into…") and trash `trash.list/restore/empty` and the Trash tab built, folder notes (27), reading mode and the indent settings (29), table helpers, user themes, the graph timelapse and Source-mode heading folding (30), workspaces (32), heading/block links (33) and note versions (31, local snapshots plus the git history) and Cmd/Ctrl+Enter on tasks in Source and Visual (36, 37) built; unbuilt: hand check of the packaged app and a Windows playback run.
- **Linux and Windows feedback (2026-10-19):** built: the Explorer calendar folds (chevron, remembered, folded on a first run in a window under 720 pt), the Explorer toolbar is laid out by hand (filter box plus two pill buttons, fixed gaps, flush right), the empty Bookmarks section is gone, the menu bar words are flat text with a soft hover/open background, the Related tab scrolls, a new Outline pane (count, filter, folding, active heading, scroll), a performance overlay (`view.performance`: fps, frame ms, memory, CPU; `/proc`, `ps` or PowerShell, no new dependency), and a first-run UI scale taken from the desktop on Linux when the window was not told its scale (`QT_SCALE_FACTOR`, `GDK_SCALE`, `Xft.dpi`, GNOME text scaling). Not confirmed on those systems: the title bar blink (the bar is now drawn whole on every frame and sits above the cards) and Explorer icons drawn out of bounds on a fast scroll (not reproduced offscreen; rows are now clipped to the viewport). Windows paths (PowerShell sampler, menu bar) were not run.
- **Editor (built, see phase 14):** spell-check needs an owner decision (new dependency or in-house checker); open: mobile bottom toolbar, inline fix popover, block menu by keyboard, embed with other text on one line, checkboxes inside embeds, embed refresh after editing another tab.
@@ -102,6 +102,30 @@ fn a_published_folder_serves_only_its_markdown_notes() {
assert!(!index.contains("Hidden note"), "{index}");
}
#[test]
fn a_comment_sidecar_is_never_published() {
let server = Server::start_temp("publish-sidecar");
let (vault, token, _) = server.owned_vault("sidecar");
note(&server, &vault, &token, "docs/a.md", "# Note A\n");
note(&server, &vault, &token, "docs/a.md.comments.json", "{\"version\":1,\"note\":\"docs/a.md\",\"comments\":[]}\n");
let (status, reply) =
server.json("POST", &format!("/v1/vaults/{vault}/publications"), Some(&token), &[], Some(&json!({"target": "docs", "kind": "folder"})));
assert_eq!(status, 200, "{reply}");
let slug = reply["slug"].as_str().expect("slug").to_owned();
// The index of the folder lists the note, never the sidecar.
let (status, _, body) = server.send_full("GET", &format!("/p/{slug}/docs"), None, &[], None);
assert_eq!(status, 200);
let index = text(&body);
assert!(index.contains("Note A"), "{index}");
assert!(!index.contains("comments.json"), "{index}");
// And the sidecar itself answers 404 with an empty body.
let (status, _, body) = server.send_full("GET", &format!("/p/{slug}/docs/a.md.comments.json"), None, &[], None);
assert_eq!(status, 404);
assert!(body.is_empty(), "the sidecar must answer with an empty body");
}
#[test]
fn unpublishing_answers_404_at_once() {
let server = Server::start_temp("publish-unpublish");
+1 -1
View File
@@ -23,7 +23,7 @@ notescraft-calendar = { workspace = true }
notescraft-templates = { workspace = true }
serde_json = { workspace = true }
egui = { workspace = true }
eframe = { workspace = true, features = ["wgpu", "default_fonts", "accesskit", "wayland", "x11"] }
eframe = { workspace = true, features = ["wgpu", "accesskit", "wayland", "x11"] }
log = { workspace = true }
cpal = { workspace = true }
craft-quit = { workspace = true, features = ["egui"] }
Binary file not shown.
+92
View File
@@ -0,0 +1,92 @@
This Font Software is licensed under the SIL Open Font License,
Version 1.1.
This license is copied below, and is also available with a FAQ at:
http://scripts.sil.org/OFL
-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font
creation efforts of academic and linguistic communities, and to
provide a free and open framework in which fonts may be shared and
improved in partnership with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply to
any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software
components as distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to,
deleting, or substituting -- in part or in whole -- any of the
components of the Original Version, by changing formats or by porting
the Font Software to a new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed,
modify, redistribute, and sell modified and unmodified copies of the
Font Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components, in
Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the
corresponding Copyright Holder. This restriction only applies to the
primary font name as presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created using
the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" stroke-linecap="round" stroke-linejoin="round"><path d="M16 7h.01"/><path d="M3.4 18H12a8 8 0 0 0 8-8V7a4 4 0 0 0-7.28-2.3L2 20"/><path d="m20 7 2 .5-2 .5"/><path d="M10 18v3"/><path d="M14 17.75V21"/></svg>

After

Width:  |  Height:  |  Size: 345 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="m21 16-4 4-4-4"/><path d="M17 20V4"/><path d="m3 8 4-4 4 4"/><path d="M7 4v16"/></svg>

After

Width:  |  Height:  |  Size: 277 B

+14
View File
@@ -0,0 +1,14 @@
<svg
xmlns="http://www.w3.org/2000/svg"
width="24"
height="24"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="2"
stroke-linecap="round"
stroke-linejoin="round"
>
<path d="M10.268 21a2 2 0 0 0 3.464 0" />
<path d="M3.262 15.326A1 1 0 0 0 4 17h16a1 1 0 0 0 .74-1.673C19.41 13.956 18 12.499 18 8A6 6 0 0 0 6 8c0 4.499-1.411 5.956-2.738 7.326" />
</svg>

After

Width:  |  Height:  |  Size: 393 B

+15
View File
@@ -0,0 +1,15 @@
<svg
xmlns="http://www.w3.org/2000/svg"
width="24"
height="24"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="2"
stroke-linecap="round"
stroke-linejoin="round"
>
<circle cx="12" cy="12" r="10" />
<path d="m15 9-6 6" />
<path d="m9 9 6 6" />
</svg>

After

Width:  |  Height:  |  Size: 293 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="m18 16 4-4-4-4"/><path d="m6 8-4 4 4 4"/><path d="m14.5 4-5 16"/></svg>

After

Width:  |  Height:  |  Size: 262 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="9" cy="12" r="1"/><circle cx="9" cy="5" r="1"/><circle cx="9" cy="19" r="1"/><circle cx="15" cy="12" r="1"/><circle cx="15" cy="5" r="1"/><circle cx="15" cy="19" r="1"/></svg>

After

Width:  |  Height:  |  Size: 369 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="3" x2="21" y1="22" y2="22"/><line x1="6" x2="6" y1="18" y2="11"/><line x1="10" x2="10" y1="18" y2="11"/><line x1="14" x2="14" y1="18" y2="11"/><line x1="18" x2="18" y1="18" y2="11"/><polygon points="12 2 20 7 4 7"/></svg>

After

Width:  |  Height:  |  Size: 412 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect width="7" height="7" x="3" y="3" rx="1"/><rect width="7" height="7" x="14" y="3" rx="1"/><rect width="7" height="7" x="14" y="14" rx="1"/><rect width="7" height="7" x="3" y="14" rx="1"/></svg>

After

Width:  |  Height:  |  Size: 380 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M3 6h18"/><path d="M7 12h10"/><path d="M10 18h4"/></svg>

After

Width:  |  Height:  |  Size: 247 B

+13
View File
@@ -0,0 +1,13 @@
<svg
xmlns="http://www.w3.org/2000/svg"
width="24"
height="24"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="2"
stroke-linecap="round"
stroke-linejoin="round"
>
<path d="M21 12a9 9 0 1 1-6.219-8.56" />
</svg>

After

Width:  |  Height:  |  Size: 251 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M20 10c0 4.993-5.539 10.193-7.399 11.799a1 1 0 0 1-1.202 0C9.539 20.193 4 14.993 4 10a8 8 0 0 1 16 0"/><circle cx="12" cy="10" r="3"/></svg>

After

Width:  |  Height:  |  Size: 330 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14.106 5.553a2 2 0 0 0 1.788 0l3.659-1.83A1 1 0 0 1 21 4.619v12.764a1 1 0 0 1-.553.894l-4.553 2.277a2 2 0 0 1-1.788 0l-4.212-2.106a2 2 0 0 0-1.788 0l-3.659 1.83A1 1 0 0 1 3 19.381V6.618a1 1 0 0 1 .553-.894l4.553-2.277a2 2 0 0 1 1.788 0z"/><path d="M15 5.764v15"/><path d="M9 3.236v15"/></svg>

After

Width:  |  Height:  |  Size: 484 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="6" x="2" y="4" rx="2"/><rect width="20" height="6" x="2" y="14" rx="2"/></svg>

After

Width:  |  Height:  |  Size: 285 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M9 3H5a2 2 0 0 0-2 2v4m6-6h10a2 2 0 0 1 2 2v4M9 3v18m0 0h10a2 2 0 0 0 2-2V9M9 21H5a2 2 0 0 1-2-2V9m0 0h18"/></svg>

After

Width:  |  Height:  |  Size: 305 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M3 2v7c0 1.1.9 2 2 2h4a2 2 0 0 0 2-2V2"/><path d="M7 2v20"/><path d="M21 15V2a5 5 0 0 0-5 5v6c0 1.1.9 2 2 2h3Zm0 0v7"/></svg>

After

Width:  |  Height:  |  Size: 315 B

+20
View File
@@ -23,6 +23,7 @@ fn diagnostic(line: &str, reason: &str) -> String {
/// What a parsed stdout line asks the process to do.
#[derive(Debug)]
#[expect(clippy::large_enum_variant, reason = "short-lived value moved straight into a channel; boxing every event costs more")]
pub(crate) enum Action {
/// Emit an event for the application.
Emit(Event),
@@ -188,6 +189,25 @@ mod tests {
assert_eq!(content_summary.as_deref(), Some("Read 1 file"));
}
#[test]
fn tool_call_meta_gives_parent_and_tool_name() {
let nested = emit(
r#"{"method":"session/update","params":{"update":{"sessionUpdate":"tool_call","toolCallId":"n1","title":"Grep","_meta":{"claudeCode":{"toolName":"Grep","parentToolUseId":"task1"}}}}}"#,
);
let Some(Event::Update(SessionUpdate::ToolCall(info))) = nested.first() else {
panic!("expected tool call");
};
assert_eq!(info.parent_id.as_deref(), Some("task1"));
assert_eq!(info.tool_name.as_deref(), Some("Grep"));
let plain = emit(
r#"{"method":"session/update","params":{"update":{"sessionUpdate":"tool_call","toolCallId":"n2","_meta":{"claudeCode":{"parentToolUseId":""}}}}}"#,
);
let Some(Event::Update(SessionUpdate::ToolCall(info))) = plain.first() else {
panic!("expected tool call");
};
assert_eq!(info.parent_id, None);
}
#[test]
fn agent_requests_get_events_or_32601() {
let permission = emit(
+17
View File
@@ -119,6 +119,11 @@ pub struct ToolCallInfo {
pub locations: Vec<String>,
/// Raw tool input as sent by the agent, when present.
pub raw_input: Option<Value>,
/// Tool call this one runs inside (a sub-agent task), from `_meta.claudeCode.parentToolUseId`
/// or `_meta.parentToolCallId`; `None` when the agent gives no link.
pub parent_id: Option<String>,
/// Agent tool name from `_meta.claudeCode.toolName` (for example `Task`), when present.
pub tool_name: Option<String>,
}
/// One option offered in a permission request.
@@ -193,6 +198,8 @@ impl ToolCallInfo {
status: str_field(value, "status").unwrap_or_default(),
locations,
raw_input: value.get("rawInput").filter(|v| !v.is_null()).cloned(),
parent_id: meta_str(value, "parentToolUseId").or_else(|| str_field_in(value.get("_meta"), "parentToolCallId")),
tool_name: meta_str(value, "toolName"),
}
}
}
@@ -211,6 +218,16 @@ fn location_texts(slots: &[Value]) -> Vec<String> {
.collect()
}
/// Reads a string from `_meta.claudeCode.<key>`.
fn meta_str(value: &Value, key: &str) -> Option<String> {
str_field_in(value.get("_meta").and_then(|meta| meta.get("claudeCode")), key)
}
/// Reads a non-empty string field from an optional object value.
fn str_field_in(value: Option<&Value>, key: &str) -> Option<String> {
value?.get(key)?.as_str().filter(|text| !text.is_empty()).map(str::to_string)
}
/// Reads a string field from an object value.
fn str_field(value: &Value, key: &str) -> Option<String> {
value.get(key).and_then(Value::as_str).map(str::to_string)
+11
View File
@@ -18,6 +18,8 @@ pub enum Edit {
Remove(usize, Item),
/// The whole list was cleared; the removed items are kept so undo can restore them.
Clear(Vec<Item>),
/// The item at the index was replaced (moved or re-typed): the old and the new item.
Replace(usize, Item, Item),
}
/// An undo and redo history of whole-list edits, capped at [`MAX_HISTORY`] steps in
@@ -84,6 +86,7 @@ fn forward(edit: Edit, annotations: &mut Annotations) -> bool {
}
!items.is_empty()
}
Edit::Replace(index, _old, new) => swap(annotations, index, new),
}
}
@@ -115,5 +118,13 @@ fn revert(edit: Edit, annotations: &mut Annotations) -> bool {
annotations.items.extend(items);
true
}
Edit::Replace(index, old, _new) => swap(annotations, index, old),
}
}
/// Puts `item` at `index`; returns whether the index exists.
fn swap(annotations: &mut Annotations, index: usize, item: Item) -> bool {
let Some(slot) = annotations.items.get_mut(index) else { return false };
*slot = item;
true
}
+14
View File
@@ -95,3 +95,17 @@ fn clear_is_undoable() {
assert!(!history.undo(&mut notes));
assert_eq!(labels(&notes), Vec::<String>::new());
}
#[test]
fn replace_undoes_and_redoes_in_place() {
let mut notes = Annotations::new(400, 300);
notes.items.extend([text("a"), text("b")]);
let mut history = History::new();
notes.items[0] = text("moved");
history.push(Edit::Replace(0, text("a"), text("moved")));
assert_eq!(labels(&notes), ["moved", "b"]);
assert!(history.undo(&mut notes));
assert_eq!(labels(&notes), ["a", "b"]);
assert!(history.redo(&mut notes));
assert_eq!(labels(&notes), ["moved", "b"]);
}
+13
View File
@@ -66,3 +66,16 @@ fn sidecar_path_and_reverse() {
assert_eq!(image_of_sidecar(&sidecar_path(path)), Some(path.to_string()));
}
}
#[test]
fn an_old_sidecar_with_lines_and_text_still_loads() {
let old = "{\"version\":1,\"image\":{\"hash\":null,\"width\":100,\"height\":80},\"items\":[\
{\"type\":\"line\",\"color\":{\"theme\":0},\"width\":2.0,\"from\":[1,2],\"to\":[30,40],\"arrow\":false},\
{\"type\":\"line\",\"color\":{\"theme\":0},\"width\":2.0,\"from\":[1,2],\"to\":[30,40],\"arrow\":true},\
{\"type\":\"text\",\"color\":{\"theme\":1},\"size\":20.0,\"at\":[5,6],\"text\":\"hi\"}]}";
let parsed = Annotations::from_json(old, 100, 80).unwrap();
assert_eq!(parsed.items.len(), 3);
assert!(matches!(parsed.items[1], Item::Line { arrow: true, .. }));
let hostile = old.replace("\"arrow\":true", "\"arrow\":true,\"future\":1");
assert!(Annotations::from_json(&hostile, 100, 80).is_err(), "unknown item fields stay rejected");
}
+1 -1
View File
@@ -38,7 +38,7 @@ pub(crate) mod text;
pub use error::{Error, Result};
pub use mentions::link::{link_mention, link_mention_as};
pub use mentions::linked::linked_mentions;
pub use mentions::unlinked::{counts, unlinked_mentions};
pub use mentions::unlinked::{UnlinkedTargets, counts, scan_unlinked, unlinked_mentions, unlinked_targets};
pub use model::{Counts, LinkedMention, MAX_MENTIONS, MAX_NOTE_BYTES, MAX_SNIPPET_CHARS, UnlinkedMention, UnlinkedReport, group_by_note};
pub use source::ContentSource;
pub use text::{line_at, snippet_at};
+28 -15
View File
@@ -18,20 +18,30 @@ use crate::text::{line_at, snippet_at, word_matches};
/// [`MAX_MENTIONS`] and [`UnlinkedReport::truncated`] says whether more
/// existed. Mentions are sorted by note path then byte offset.
pub fn unlinked_mentions(index: &Index, source: &dyn ContentSource, target: &str) -> UnlinkedReport {
let Some(info) = index.note(target) else {
return UnlinkedReport::default();
};
let names = target_names(info);
let (names, canonical) = match names {
Some(names) => (names, info.path.clone()),
None => return UnlinkedReport::default(),
};
unlinked_targets(index, target).map_or_else(UnlinkedReport::default, |targets| scan_unlinked(&targets, source))
}
/// What an unlinked-mention scan needs from the index: the names to look for and the notes to read.
/// Owned, so the scan itself can run on another thread while the index stays with the caller.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct UnlinkedTargets {
names: Vec<String>,
paths: Vec<String>,
}
/// The names of `target` and every other indexed note, or `None` when `target` is unknown or nameless.
pub fn unlinked_targets(index: &Index, target: &str) -> Option<UnlinkedTargets> {
let info = index.note(target)?;
let names = target_names(info)?;
let paths = index.notes().into_iter().filter(|path| *path != info.path).map(str::to_owned).collect();
Some(UnlinkedTargets { names, paths })
}
/// The scan of [`unlinked_mentions`], over what [`unlinked_targets`] collected.
pub fn scan_unlinked(targets: &UnlinkedTargets, source: &dyn ContentSource) -> UnlinkedReport {
let mut report = UnlinkedReport::default();
for path in index.notes() {
if path == canonical {
continue;
}
scan_note(path, &content_of(source, path), &names, &mut report);
for path in &targets.paths {
scan_note(path, &content_of(source, path), &targets.names, &mut report);
}
report.mentions.sort_by(|a, b| a.from.cmp(&b.from).then_with(|| a.range.start.cmp(&b.range.start)));
report
@@ -52,12 +62,15 @@ fn scan_note(path: &str, content: &str, names: &[String], report: &mut UnlinkedR
report.truncated = true;
return;
}
let doc = notescraft_markdown::parse(content);
let masks = mask_ranges(&doc);
let (found, more) = word_matches(content, names, MAX_MENTIONS);
if more {
report.truncated = true;
}
if found.is_empty() {
return;
}
let doc = notescraft_markdown::parse(content);
let masks = mask_ranges(&doc);
for (range, name) in found {
if report.mentions.len() >= MAX_MENTIONS {
report.truncated = true;
+280
View File
@@ -0,0 +1,280 @@
//! Reading and writing `.base` files over the YAML subset.
use crate::model::{Base, Direction, Filter, GroupBy, MAX_EXPR_BYTES, MAX_FORMULAS, MAX_VIEWS, PropertyCfg, Sort, View, ViewKind};
use crate::yaml::{Node, emit, parse};
use crate::{Error, Result};
type Entries = [(String, Node)];
fn invalid<T>(message: String) -> Result<T> {
Err(Error::Invalid(message))
}
fn entries<'a>(node: &'a Node, what: &str) -> Result<&'a Entries> {
match node {
Node::Map(map) => Ok(map),
Node::Null => Ok(&[]),
_ => invalid(format!("{what} must be a map")),
}
}
fn items<'a>(node: &'a Node, what: &str) -> Result<&'a [Node]> {
match node {
Node::Seq(seq) => Ok(seq),
Node::Null => Ok(&[]),
_ => invalid(format!("{what} must be a list")),
}
}
fn text(node: &Node, what: &str) -> Result<String> {
match node {
Node::Str(s) => Ok(s.clone()),
_ => invalid(format!("{what} must be a string")),
}
}
fn expr(node: &Node, what: &str) -> Result<String> {
let s = text(node, what)?;
if s.len() > MAX_EXPR_BYTES {
return Err(Error::Limit(format!("{what} longer than {MAX_EXPR_BYTES} bytes")));
}
Ok(s)
}
fn expr_pairs(node: &Node, what: &str, max: usize) -> Result<Vec<(String, String)>> {
let map = entries(node, what)?;
if map.len() > max {
return Err(Error::Limit(format!("more than {max} {what}")));
}
map.iter().map(|(k, v)| Ok((k.clone(), expr(v, &format!("{what} `{k}`"))?))).collect()
}
fn direction(node: Option<&Node>) -> Result<Direction> {
match node {
None => Ok(Direction::Asc),
Some(Node::Str(s)) if s == "ASC" => Ok(Direction::Asc),
Some(Node::Str(s)) if s == "DESC" => Ok(Direction::Desc),
Some(other) => invalid(format!("direction must be ASC or DESC, got {other:?}")),
}
}
fn dir_text(d: Direction) -> &'static str {
if d == Direction::Asc { "ASC" } else { "DESC" }
}
fn get<'a>(map: &'a Entries, key: &str) -> Option<&'a Node> {
map.iter().find(|(k, _)| k == key).map(|(_, v)| v)
}
fn parse_filter(node: &Node) -> Result<Filter> {
let Node::Map(map) = node else { return Ok(Filter::Expr(expr(node, "filter")?)) };
let [(key, value)] = map.as_slice() else { return invalid("a filter group has exactly one of and, or, not".into()) };
let members = items(value, &format!("filter `{key}`"))?.iter().map(parse_filter).collect::<Result<Vec<_>>>()?;
match key.as_str() {
"and" => Ok(Filter::And(members)),
"or" => Ok(Filter::Or(members)),
"not" => Ok(Filter::Not(members)),
other => invalid(format!("unknown filter group `{other}`")),
}
}
fn filter_node(filter: &Filter) -> Node {
let group = |key: &str, members: &[Filter]| Node::Map(vec![(key.to_string(), Node::Seq(members.iter().map(filter_node).collect()))]);
match filter {
Filter::Expr(s) => Node::Str(s.clone()),
Filter::And(m) => group("and", m),
Filter::Or(m) => group("or", m),
Filter::Not(m) => group("not", m),
}
}
fn pairs_node(pairs: &[(String, String)]) -> Node {
Node::Map(pairs.iter().map(|(k, v)| (k.clone(), Node::Str(v.clone()))).collect())
}
/// Orders `known` and `extra` entries as the original file had them, then the new ones.
fn arrange(layout: &[String], mut known: Vec<(String, Node)>, extra: &[(String, Node)]) -> Node {
let mut out = Vec::new();
for key in layout {
if let Some(i) = known.iter().position(|(k, _)| k == key) {
out.push(known.remove(i));
} else if let Some(entry) = extra.iter().find(|(k, _)| k == key) {
out.push(entry.clone());
}
}
out.extend(known);
out.extend(extra.iter().filter(|(k, _)| !layout.contains(k)).cloned());
Node::Map(out)
}
fn parse_view(node: &Node) -> Result<View> {
let map = entries(node, "a view")?;
let kind = match get(map, "type") {
Some(t) => match text(t, "view type")?.as_str() {
"table" => ViewKind::Table,
"cards" => ViewKind::Cards,
"list" => ViewKind::List,
"map" => ViewKind::Map,
other => ViewKind::Other(other.to_string()),
},
None => return invalid("a view needs a type".into()),
};
let mut view = View::new(kind, None);
for (key, value) in map {
view.layout.push(key.clone());
match key.as_str() {
"type" => {}
"name" => view.name = Some(text(value, "view name")?),
"limit" => view.limit = Some(parse_limit(value)?),
"filters" => view.filters = Some(parse_filter(value)?),
"groupBy" => {
let g = entries(value, "groupBy")?;
let property = text(get(g, "property").unwrap_or(&Node::Null), "groupBy property")?;
view.group_by = Some(GroupBy { property, direction: direction(get(g, "direction"))? });
}
"groupOrder" => view.group_order = items(value, "groupOrder")?.to_vec(),
"order" => view.order = items(value, "order")?.iter().map(|n| text(n, "order entry")).collect::<Result<_>>()?,
"sort" => view.sort = items(value, "sort")?.iter().map(parse_sort).collect::<Result<_>>()?,
"summaries" => {
view.summaries = entries(value, "summaries")?.iter().map(|(k, v)| Ok((k.clone(), text(v, "summary name")?))).collect::<Result<_>>()?;
}
_ => view.extra.push((key.clone(), value.clone())),
}
}
Ok(view)
}
fn parse_limit(node: &Node) -> Result<u32> {
match node {
Node::Num(n) => n.to_string().parse().or_else(|_| invalid(format!("limit must be a non-negative whole number, got {n}"))),
other => invalid(format!("limit must be a non-negative whole number, got {other:?}")),
}
}
fn parse_sort(node: &Node) -> Result<Sort> {
let map = entries(node, "a sort entry")?;
let property = text(get(map, "property").unwrap_or(&Node::Null), "sort property")?;
Ok(Sort { property, direction: direction(get(map, "direction"))? })
}
fn view_node(view: &View) -> Node {
let mut known = vec![("type".to_string(), Node::Str(view.kind.as_str().to_string()))];
if let Some(name) = &view.name {
known.push(("name".into(), Node::Str(name.clone())));
}
if let Some(limit) = view.limit {
known.push(("limit".into(), Node::Num(f64::from(limit))));
}
if let Some(filters) = &view.filters {
known.push(("filters".into(), filter_node(filters)));
}
if let Some(g) = &view.group_by {
let map = vec![("property".into(), Node::Str(g.property.clone())), ("direction".into(), Node::Str(dir_text(g.direction).into()))];
known.push(("groupBy".into(), Node::Map(map)));
}
if !view.group_order.is_empty() {
known.push(("groupOrder".into(), Node::Seq(view.group_order.clone())));
}
if !view.order.is_empty() {
known.push(("order".into(), Node::Seq(view.order.iter().map(|s| Node::Str(s.clone())).collect())));
}
if !view.sort.is_empty() {
let sorts = view
.sort
.iter()
.map(|s| Node::Map(vec![("property".into(), Node::Str(s.property.clone())), ("direction".into(), Node::Str(dir_text(s.direction).into()))]));
known.push(("sort".into(), Node::Seq(sorts.collect())));
}
if !view.summaries.is_empty() {
known.push(("summaries".into(), pairs_node(&view.summaries)));
}
arrange(&view.layout, known, &view.extra)
}
fn parse_property(node: &Node) -> Result<PropertyCfg> {
let mut cfg = PropertyCfg::default();
for (key, value) in entries(node, "a property")? {
if key == "displayName" {
cfg.display_name = Some(text(value, "displayName")?);
} else {
cfg.extra.push((key.clone(), value.clone()));
}
}
Ok(cfg)
}
impl Base {
/// Parses the text of a `.base` file. Unknown keys are kept; a missing `views` gives one
/// default table view that `to_yaml` does not write until the views are edited.
///
/// # Errors
/// [`Error::Syntax`] for bad YAML, [`Error::Limit`] past 100 views, 200 formulas or a 4 KiB
/// expression, [`Error::Invalid`] for a bad value (`limit: -1`, `direction: SIDEWAYS`).
pub fn parse(source: &str) -> Result<Self> {
let root = parse(source)?;
let mut base = Base {
filters: None,
formulas: Vec::new(),
properties: Vec::new(),
summaries: Vec::new(),
extra: Vec::new(),
views: Vec::new(),
views_implicit: false,
layout: Vec::new(),
};
for (key, value) in entries(&root, "a base")? {
base.layout.push(key.clone());
match key.as_str() {
"filters" => base.filters = Some(parse_filter(value)?),
"formulas" => base.formulas = expr_pairs(value, "formulas", MAX_FORMULAS)?,
"summaries" => base.summaries = expr_pairs(value, "summaries", MAX_FORMULAS)?,
"properties" => {
base.properties = entries(value, "properties")?.iter().map(|(k, v)| Ok((k.clone(), parse_property(v)?))).collect::<Result<_>>()?;
}
"views" => {
let seq = items(value, "views")?;
if seq.len() > MAX_VIEWS {
return Err(Error::Limit(format!("more than {MAX_VIEWS} views")));
}
base.views = seq.iter().map(parse_view).collect::<Result<_>>()?;
}
_ => base.extra.push((key.clone(), value.clone())),
}
}
if base.views.is_empty() && !base.layout.iter().any(|k| k == "views") {
base.views.push(View::new(ViewKind::Table, Some("Table".to_string())));
base.views_implicit = true;
}
Ok(base)
}
/// Writes the base as YAML, keys in their original order. Comments are not kept.
#[must_use]
pub fn to_yaml(&self) -> String {
let mut known = Vec::new();
if let Some(filters) = &self.filters {
known.push(("filters".to_string(), filter_node(filters)));
}
if !self.formulas.is_empty() {
known.push(("formulas".into(), pairs_node(&self.formulas)));
}
if !self.properties.is_empty() {
let props = self.properties.iter().map(|(id, cfg)| {
let mut map = Vec::new();
if let Some(name) = &cfg.display_name {
map.push(("displayName".to_string(), Node::Str(name.clone())));
}
map.extend(cfg.extra.iter().cloned());
(id.clone(), Node::Map(map))
});
known.push(("properties".into(), Node::Map(props.collect())));
}
if !self.summaries.is_empty() {
known.push(("summaries".into(), pairs_node(&self.summaries)));
}
if !self.views_implicit {
known.push(("views".into(), Node::Seq(self.views.iter().map(view_node).collect())));
}
emit(&arrange(&self.layout, known, &self.extra))
}
}
+179
View File
@@ -0,0 +1,179 @@
//! Ordering, equality and truthiness of values.
//!
//! The order is total, so a sort never sees an incomparable pair: the types are
//! ranked `null < boolean < number < string < date < duration < list < object`
//! and the rest follow in that order. Inside a type the order is natural, except
//! for numbers, where `NaN` sorts last, and strings, which compare without case
//! first and by code point then.
use std::cmp::Ordering;
use crate::value::Value;
/// The rank of a type in the total order.
fn rank(value: &Value) -> u8 {
match value {
Value::Null => 0,
Value::Bool(_) => 1,
Value::Number(_) => 2,
Value::Str(_) => 3,
Value::Date(_) => 4,
Value::Duration(_) => 5,
Value::List(_) => 6,
Value::Object(_) => 7,
Value::Link { .. } => 8,
Value::File { .. } => 9,
Value::Regex { .. } => 10,
Value::Html(_) => 11,
Value::Image(_) => 12,
Value::Icon(_) => 13,
Value::Error(_) => 14,
}
}
/// The total order of two values.
///
/// Values of different types are ordered by their type rank, so no pair is
/// incomparable and a sort is stable. `NaN` sorts after every other number.
#[must_use]
pub fn compare(a: &Value, b: &Value) -> Ordering {
let (rank_a, rank_b) = (rank(a), rank(b));
if rank_a != rank_b {
return rank_a.cmp(&rank_b);
}
match (a, b) {
(Value::Bool(left), Value::Bool(right)) => left.cmp(right),
(Value::Number(left), Value::Number(right)) => compare_numbers(*left, *right),
(Value::Str(left), Value::Str(right)) => compare_strings(left, right),
(Value::Date(left), Value::Date(right)) => left.ms().cmp(&right.ms()).then_with(|| left.has_time().cmp(&right.has_time())),
(Value::Duration(left), Value::Duration(right)) => left.months().cmp(&right.months()).then_with(|| left.ms().cmp(&right.ms())),
(Value::List(left), Value::List(right)) => compare_lists(left, right),
(Value::Object(left), Value::Object(right)) => compare_objects(left, right),
(Value::Link { target: left_target, display: left_display }, Value::Link { target: right_target, display: right_display }) => {
left_target.cmp(right_target).then_with(|| left_display.cmp(right_display))
}
(Value::Regex { pattern: left_pattern, flags: left_flags }, Value::Regex { pattern: right_pattern, flags: right_flags }) => {
left_pattern.cmp(right_pattern).then_with(|| left_flags.cmp(right_flags))
}
(Value::File { path: left }, Value::File { path: right })
| (Value::Html(left), Value::Html(right))
| (Value::Image(left), Value::Image(right))
| (Value::Icon(left), Value::Icon(right))
| (Value::Error(left), Value::Error(right)) => left.cmp(right),
_ => Ordering::Equal,
}
}
/// Whether two values are equal, as the `==` operator reports it.
///
/// A link and a file are equal when the link target is the path of the file, two
/// links are equal when they point to the same file or show the same text, and
/// values of different types are never equal (the comparison does not fail).
#[must_use]
pub fn loose_eq(a: &Value, b: &Value) -> bool {
match (a, b) {
(Value::Bool(left), Value::Bool(right)) => left == right,
(Value::Number(left), Value::Number(right)) => left == right,
(Value::Date(left), Value::Date(right)) => left.ms() == right.ms(),
(Value::Duration(left), Value::Duration(right)) => left == right,
(Value::List(left), Value::List(right)) => left.len() == right.len() && left.iter().zip(right.iter()).all(|(one, other)| loose_eq(one, other)),
(Value::Object(left), Value::Object(right)) => {
left.len() == right.len()
&& left
.iter()
.zip(right.iter())
.all(|((left_key, left_value), (right_key, right_value))| left_key == right_key && loose_eq(left_value, right_value))
}
(Value::Link { target, .. }, Value::File { path }) | (Value::File { path }, Value::Link { target, .. }) => target == path,
(Value::Link { target: left_target, display: left_display }, Value::Link { target: right_target, display: right_display }) => {
left_target == right_target || left_display == right_display
}
(Value::Regex { pattern: left_pattern, flags: left_flags }, Value::Regex { pattern: right_pattern, flags: right_flags }) => {
left_pattern == right_pattern && left_flags == right_flags
}
(Value::Str(left), Value::Str(right))
| (Value::File { path: left }, Value::File { path: right })
| (Value::Html(left), Value::Html(right))
| (Value::Image(left), Value::Image(right))
| (Value::Icon(left), Value::Icon(right))
| (Value::Error(left), Value::Error(right)) => left == right,
_ => false,
}
}
/// Whether a value counts as true, as `isTruthy` reports it.
///
/// Null, `false`, `0`, `NaN`, the empty string, the empty list and an error are
/// falsy; every other value is truthy.
#[must_use]
pub fn truthy(value: &Value) -> bool {
match value {
Value::Bool(flag) => *flag,
Value::Number(number) => *number != 0.0 && !number.is_nan(),
Value::Str(text) => !text.is_empty(),
Value::List(items) => !items.is_empty(),
Value::Null | Value::Error(_) => false,
_ => true,
}
}
/// Orders two numbers, with `NaN` after every other number.
fn compare_numbers(left: f64, right: f64) -> Ordering {
match (left.is_nan(), right.is_nan()) {
(true, true) => Ordering::Equal,
(true, false) => Ordering::Greater,
(false, true) => Ordering::Less,
(false, false) => left.partial_cmp(&right).unwrap_or(Ordering::Equal),
}
}
/// Orders two strings without case first, then by code point.
fn compare_strings(left: &str, right: &str) -> Ordering {
let folded = compare_ignore_case(left, right);
if folded == Ordering::Equal { left.cmp(right) } else { folded }
}
/// Orders two strings without case, without allocating.
fn compare_ignore_case(left: &str, right: &str) -> Ordering {
let mut left_chars = left.chars().flat_map(char::to_lowercase);
let mut right_chars = right.chars().flat_map(char::to_lowercase);
loop {
match (left_chars.next(), right_chars.next()) {
(None, None) => return Ordering::Equal,
(None, Some(_)) => return Ordering::Less,
(Some(_), None) => return Ordering::Greater,
(Some(left_char), Some(right_char)) => {
let order = left_char.cmp(&right_char);
if order != Ordering::Equal {
return order;
}
}
}
}
}
/// Orders two lists element by element, the shorter one first on a tie.
fn compare_lists(left: &[Value], right: &[Value]) -> Ordering {
for (left_item, right_item) in left.iter().zip(right.iter()) {
let order = compare(left_item, right_item);
if order != Ordering::Equal {
return order;
}
}
left.len().cmp(&right.len())
}
/// Orders two objects key by key, then value by value.
fn compare_objects(left: &[(String, Value)], right: &[(String, Value)]) -> Ordering {
for ((left_key, left_value), (right_key, right_value)) in left.iter().zip(right.iter()) {
let order = left_key.cmp(right_key);
if order != Ordering::Equal {
return order;
}
let order = compare(left_value, right_value);
if order != Ordering::Equal {
return order;
}
}
left.len().cmp(&right.len())
}
+249
View File
@@ -0,0 +1,249 @@
//! Dates, durations and the civil-date arithmetic behind them.
//!
//! A [`Date`] is milliseconds since the Unix epoch plus a flag telling whether the
//! value carries a time part (a date-only value renders as `2024-12-01`).
//! A [`Duration`] is whole calendar months plus an exact millisecond part, so a
//! month can clamp to the last day of the target month.
//!
//! The calendar itself lives in the `civil` module, the duration strings in the
//! `duration` module.
mod civil;
mod duration;
use std::fmt::Write;
use thiserror::Error as ThisError;
pub use civil::days_in_month;
pub use duration::{Duration, parse_duration};
use civil::{civil_from_days, days_from_civil, parse_time, parse_ymd, split_date_time};
/// Milliseconds in one second.
pub(crate) const MS_PER_SECOND: i64 = 1_000;
/// Milliseconds in one minute.
pub(crate) const MS_PER_MINUTE: i64 = 60 * MS_PER_SECOND;
/// Milliseconds in one hour.
pub(crate) const MS_PER_HOUR: i64 = 60 * MS_PER_MINUTE;
/// Milliseconds in one day.
pub(crate) const MS_PER_DAY: i64 = 24 * MS_PER_HOUR;
/// Days in one week.
pub(crate) const DAYS_PER_WEEK: i64 = 7;
/// Months in one year.
pub(crate) const MONTHS_PER_YEAR: i64 = 12;
/// Lower bound of the supported years.
const MIN_YEAR: i32 = -9999;
/// Upper bound of the supported years.
pub(crate) const MAX_YEAR: i32 = 9999;
/// Everything that can go wrong while reading or shifting a date.
#[derive(Debug, Clone, PartialEq, Eq, ThisError)]
pub enum DateError {
/// The text is not a duration the format accepts.
#[error("invalid duration: {0}")]
InvalidDuration(String),
/// The text is not a date the format accepts.
#[error("invalid date: {0}")]
InvalidDate(String),
/// The arithmetic left the supported range.
#[error("date out of range: {0}")]
OutOfRange(String),
}
/// The calendar parts of a date.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct DateParts {
/// Year, proleptic Gregorian.
pub year: i32,
/// Month, `1..=12`.
pub month: u8,
/// Day of the month, `1..=31`.
pub day: u8,
/// Hour, `0..=23`.
pub hour: u8,
/// Minute, `0..=59`.
pub minute: u8,
/// Second, `0..=59`.
pub second: u8,
/// Millisecond, `0..=999`.
pub millisecond: u16,
}
/// A moment in time: milliseconds since the Unix epoch, with a flag telling
/// whether the value carries a time part.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub struct Date {
ms: i64,
has_time: bool,
}
impl Date {
/// Builds a date from Unix milliseconds.
#[must_use]
pub const fn new(ms: i64, has_time: bool) -> Self {
Self { ms, has_time }
}
/// Unix milliseconds.
#[must_use]
pub const fn ms(self) -> i64 {
self.ms
}
/// Whether the value carries a time part.
#[must_use]
pub const fn has_time(self) -> bool {
self.has_time
}
/// The current moment, from the clock of the caller.
#[must_use]
pub const fn now(now_ms: i64) -> Self {
Self::new(now_ms, true)
}
/// The day of `now_ms` at midnight, from the clock of the caller.
#[must_use]
pub fn today(now_ms: i64) -> Self {
let day = now_ms.div_euclid(MS_PER_DAY);
Self::new(day.saturating_mul(MS_PER_DAY), false)
}
/// Builds a date from its calendar parts, with a time part.
///
/// # Errors
///
/// [`DateError::InvalidDate`] when a part is out of range and
/// [`DateError::OutOfRange`] when the year leaves the supported range.
pub fn from_parts(parts: DateParts) -> Result<Self, DateError> {
if !(MIN_YEAR..=MAX_YEAR).contains(&parts.year) {
return Err(DateError::OutOfRange(format!("year {}", parts.year)));
}
let last = days_in_month(parts.year, parts.month).ok_or_else(|| DateError::InvalidDate(format!("month {}", parts.month)))?;
if parts.day == 0 || parts.day > last {
return Err(DateError::InvalidDate(format!("day {}", parts.day)));
}
if parts.hour > 23 || parts.minute > 59 || parts.second > 59 || parts.millisecond > 999 {
return Err(DateError::InvalidDate(format!("time {parts:?}")));
}
let days = days_from_civil(parts.year, parts.month, parts.day);
let time = i64::from(parts.hour) * MS_PER_HOUR
+ i64::from(parts.minute) * MS_PER_MINUTE
+ i64::from(parts.second) * MS_PER_SECOND
+ i64::from(parts.millisecond);
Ok(Self::new(days * MS_PER_DAY + time, true))
}
/// Builds a date without a time part.
///
/// # Errors
///
/// The same errors as [`Date::from_parts`].
pub fn from_ymd(year: i32, month: u8, day: u8) -> Result<Self, DateError> {
if !(MIN_YEAR..=MAX_YEAR).contains(&year) {
return Err(DateError::OutOfRange(format!("year {year}")));
}
let last = days_in_month(year, month).ok_or_else(|| DateError::InvalidDate(format!("month {month}")))?;
if day == 0 || day > last {
return Err(DateError::InvalidDate(format!("day {day}")));
}
let days = days_from_civil(year, month, day);
Ok(Self::new(days * MS_PER_DAY, false))
}
/// The calendar parts of the date, in UTC.
#[must_use]
pub fn parts(self) -> DateParts {
let days = self.ms.div_euclid(MS_PER_DAY);
let time = self.ms.rem_euclid(MS_PER_DAY);
let (year, month, day) = civil_from_days(days);
DateParts {
year,
month,
day,
hour: u8::try_from(time / MS_PER_HOUR).unwrap_or(0),
minute: u8::try_from((time / MS_PER_MINUTE) % 60).unwrap_or(0),
second: u8::try_from((time / MS_PER_SECOND) % 60).unwrap_or(0),
millisecond: u16::try_from(time % MS_PER_SECOND).unwrap_or(0),
}
}
/// Reads a date: `2024-12-01`, `2024-12-01 04:03`, `2024-12-01T04:03:00Z`,
/// `2024-12-01 04:03:00.250` or `2024-12-01 04:03:00+02:00`.
///
/// # Errors
///
/// [`DateError::InvalidDate`] when the text is not one of those shapes.
pub fn parse(text: &str) -> Result<Self, DateError> {
let trimmed = text.trim();
if trimmed.is_empty() {
return Err(DateError::InvalidDate(text.to_string()));
}
let (date_text, time_text) = split_date_time(trimmed);
let (year, month, day) = parse_ymd(date_text)?;
let days = days_from_civil(year, month, day);
let mut ms = days * MS_PER_DAY;
let mut has_time = false;
if let Some(rest) = time_text {
ms += parse_time(rest)?;
has_time = true;
}
Ok(Self::new(ms, has_time))
}
/// The date as text: `2024-12-01`, or `2024-12-01 04:03:00` with a time part.
#[must_use]
pub fn format_iso(self) -> String {
let parts = self.parts();
let mut text = format!("{:04}-{:02}-{:02}", parts.year, parts.month, parts.day);
if self.has_time {
let _ = write!(text, " {:02}:{:02}:{:02}", parts.hour, parts.minute, parts.second);
if parts.millisecond != 0 {
let _ = write!(text, ".{:03}", parts.millisecond);
}
}
text
}
}
/// Adds a duration to a date: the calendar months first (the day clamps to the
/// last day of the target month), then the exact millisecond part.
///
/// # Errors
///
/// [`DateError::OutOfRange`] when the result leaves the supported range.
pub fn add_duration(date: Date, duration: Duration) -> Result<Date, DateError> {
let mut ms = date.ms();
let mut has_time = date.has_time();
if duration.months() != 0 {
let parts = date.parts();
let total = i64::from(parts.year) * MONTHS_PER_YEAR + i64::from(parts.month) - 1 + duration.months();
let year = total.div_euclid(MONTHS_PER_YEAR);
if !(i64::from(MIN_YEAR)..=i64::from(MAX_YEAR)).contains(&year) {
return Err(DateError::OutOfRange(format!("year {year}")));
}
let year = i32::try_from(year).map_err(|_| DateError::OutOfRange("year".into()))?;
let month = u8::try_from(total.rem_euclid(MONTHS_PER_YEAR) + 1).map_err(|_| DateError::OutOfRange("month".into()))?;
let last = days_in_month(year, month).ok_or_else(|| DateError::InvalidDate(format!("month {month}")))?;
let day = parts.day.min(last);
let days = days_from_civil(year, month, day);
ms = days * MS_PER_DAY + date.ms().rem_euclid(MS_PER_DAY);
}
if duration.ms() != 0 {
ms = ms.checked_add(duration.ms()).ok_or_else(|| DateError::OutOfRange("milliseconds".into()))?;
if duration.ms() % MS_PER_DAY != 0 {
has_time = true;
}
}
Ok(Date::new(ms, has_time))
}
/// Milliseconds between two dates (`a - b`).
///
/// # Errors
///
/// [`DateError::OutOfRange`] when the difference does not fit.
pub fn sub_dates(a: Date, b: Date) -> Result<i64, DateError> {
a.ms().checked_sub(b.ms()).ok_or_else(|| DateError::OutOfRange("date difference".into()))
}
+156
View File
@@ -0,0 +1,156 @@
//! Proleptic Gregorian calendar: day counts, month lengths and the text of a date.
//!
//! The day-count math is Howard Hinnant's `days_from_civil` / `civil_from_days`
//! algorithm, copied from `notescraft-templates`: both crates sit on layer L2 and
//! `notescraft-bases` may not depend on `notescraft-templates`.
use super::{DateError, MAX_YEAR, MS_PER_HOUR, MS_PER_MINUTE, MS_PER_SECOND};
/// Days since 1970-01-01 for a proleptic Gregorian date.
pub(crate) fn days_from_civil(year: i32, month: u8, day: u8) -> i64 {
let y = i64::from(year) - i64::from(month <= 2);
let era = y.div_euclid(400);
let yoe = u64::try_from(y - era * 400).unwrap_or(0);
let mp = u64::from(if month > 2 { month - 3 } else { month + 9 });
let doy = (153 * mp + 2) / 5 + u64::from(day) - 1;
let doe = yoe * 365 + yoe / 4 - yoe / 100 + doy;
era * 146_097 + i64::try_from(doe).unwrap_or(0) - 719_468
}
/// Proleptic Gregorian date for a day count since 1970-01-01.
pub(crate) fn civil_from_days(days: i64) -> (i32, u8, u8) {
let z = days + 719_468;
let era = z.div_euclid(146_097);
let doe = u64::try_from(z - era * 146_097).unwrap_or(0);
let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let day = u8::try_from(doy - (153 * mp + 2) / 5 + 1).unwrap_or(1);
let month = u8::try_from(if mp < 10 { mp + 3 } else { mp - 9 }).unwrap_or(1);
let year = i32::try_from(yoe).unwrap_or(0) + i32::try_from(era * 400).unwrap_or(0);
let year = year + i32::from(month <= 2);
(year, month, day)
}
/// Length of `month` (1-12) in `year`, `None` for an invalid month.
#[must_use]
pub fn days_in_month(year: i32, month: u8) -> Option<u8> {
let days: u8 = match month {
1 | 3 | 5 | 7 | 8 | 10 | 12 => 31,
4 | 6 | 9 | 11 => 30,
2 if is_leap_year(year) => 29,
2 => 28,
_ => return None,
};
Some(days)
}
/// Whether `year` is a Gregorian leap year (divisible by 4, not by 100 unless also by 400).
#[must_use]
pub const fn is_leap_year(year: i32) -> bool {
year.rem_euclid(4) == 0 && (year.rem_euclid(100) != 0 || year.rem_euclid(400) == 0)
}
/// Splits `2024-12-01 04:03:00` into the date text and the time text.
pub(crate) fn split_date_time(text: &str) -> (&str, Option<&str>) {
let separator = text.find(['T', 't', ' ']);
match separator {
Some(index) => {
let head = text.get(..index).unwrap_or("");
let tail = text.get(index + 1..).unwrap_or("");
let tail = tail.trim();
if tail.is_empty() { (head, None) } else { (head, Some(tail)) }
}
None => (text, None),
}
}
/// Reads `2024-12-01` or `2024-12`.
pub(crate) fn parse_ymd(text: &str) -> Result<(i32, u8, u8), DateError> {
let invalid = || DateError::InvalidDate(text.to_string());
let parts: Vec<&str> = text.split('-').collect();
if parts.len() < 2 || parts.len() > 3 {
return Err(invalid());
}
let year_text = parts.first().copied().unwrap_or("");
let month_text = parts.get(1).copied().unwrap_or("");
let day_text = parts.get(2).copied().unwrap_or("1");
let year: i32 = year_text.parse().map_err(|_| invalid())?;
let month: u8 = month_text.parse().map_err(|_| invalid())?;
let day: u8 = day_text.parse().map_err(|_| invalid())?;
if !(1..=MAX_YEAR).contains(&year) {
return Err(invalid());
}
if !(1..=12).contains(&month) {
return Err(invalid());
}
let last = days_in_month(year, month).ok_or_else(invalid)?;
if day == 0 || day > last {
return Err(invalid());
}
Ok((year, month, day))
}
/// Reads `04:03`, `04:03:00`, `04:03:00.250`, with an optional `Z` or `+02:00`.
pub(crate) fn parse_time(text: &str) -> Result<i64, DateError> {
let invalid = || DateError::InvalidDate(text.to_string());
let mut body = text.trim();
let mut offset_ms: i64 = 0;
if let Some(rest) = body.strip_suffix('Z').or_else(|| body.strip_suffix('z')) {
body = rest.trim();
} else if let Some(index) = body.rfind(['+', '-']) {
let head = body.get(..index).ok_or_else(invalid)?;
let tail = body.get(index..).ok_or_else(invalid)?;
let sign = if tail.starts_with('-') { -1 } else { 1 };
let digits = tail.strip_prefix('+').or_else(|| tail.strip_prefix('-')).ok_or_else(invalid)?;
let (hours, minutes) = parse_hm(digits).ok_or_else(invalid)?;
offset_ms = sign * (i64::from(hours) * MS_PER_HOUR + i64::from(minutes) * MS_PER_MINUTE);
body = head.trim();
}
let parts: Vec<&str> = body.split(':').collect();
if parts.len() < 2 || parts.len() > 3 {
return Err(invalid());
}
let hour: u8 = parts.first().copied().unwrap_or("").parse().map_err(|_| invalid())?;
let minute: u8 = parts.get(1).copied().unwrap_or("").parse().map_err(|_| invalid())?;
let second_text = parts.get(2).copied().unwrap_or("0");
let (second, millisecond) = parse_seconds(second_text).ok_or_else(invalid)?;
if hour > 23 || minute > 59 || second > 59 {
return Err(invalid());
}
let ms = i64::from(hour) * MS_PER_HOUR + i64::from(minute) * MS_PER_MINUTE + i64::from(second) * MS_PER_SECOND + i64::from(millisecond);
Ok(ms - offset_ms)
}
/// Reads `02:00` into hours and minutes.
fn parse_hm(text: &str) -> Option<(u8, u8)> {
let mut parts = text.split(':');
let hours: u8 = parts.next()?.parse().ok()?;
let minutes: u8 = parts.next()?.parse().ok()?;
if parts.next().is_some() || hours > 23 || minutes > 59 {
return None;
}
Some((hours, minutes))
}
/// Reads `07` or `07.250` into seconds and milliseconds.
fn parse_seconds(text: &str) -> Option<(u8, u16)> {
let (whole, fraction) = match text.split_once('.') {
Some((whole, fraction)) => (whole, fraction),
None => (text, ""),
};
let second: u8 = whole.parse().ok()?;
let millisecond: u16 = if fraction.is_empty() {
0
} else {
if fraction.len() > 3 {
return None;
}
let padded = format!("{fraction:0<3}");
padded.get(..3)?.parse().ok()?
};
if millisecond > 999 {
return None;
}
Some((second, millisecond))
}
+216
View File
@@ -0,0 +1,216 @@
//! The duration type and the duration strings of the documentation.
//!
//! A duration is whole calendar months plus an exact millisecond part, so a
//! month can clamp to the last day of the target month.
use super::{DAYS_PER_WEEK, DateError, MONTHS_PER_YEAR, MS_PER_DAY, MS_PER_HOUR, MS_PER_MINUTE, MS_PER_SECOND};
/// A length of time: whole calendar months plus an exact millisecond part.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub struct Duration {
months: i64,
ms: i64,
}
impl Duration {
/// Builds a duration from calendar months and milliseconds.
#[must_use]
pub const fn new(months: i64, ms: i64) -> Self {
Self { months, ms }
}
/// The calendar months of the duration.
#[must_use]
pub const fn months(self) -> i64 {
self.months
}
/// The exact millisecond part of the duration.
#[must_use]
pub const fn ms(self) -> i64 {
self.ms
}
/// Whether the duration is zero.
#[must_use]
pub const fn is_zero(self) -> bool {
self.months == 0 && self.ms == 0
}
/// Multiplies the duration by a whole number.
///
/// # Errors
///
/// [`DateError::OutOfRange`] when the result does not fit.
pub fn scaled(self, factor: i64) -> Result<Self, DateError> {
let months = self.months.checked_mul(factor).ok_or_else(|| DateError::OutOfRange("duration months".into()))?;
let ms = self.ms.checked_mul(factor).ok_or_else(|| DateError::OutOfRange("duration ms".into()))?;
Ok(Self::new(months, ms))
}
/// Adds two durations, component by component.
///
/// # Errors
///
/// [`DateError::OutOfRange`] when the result does not fit.
pub fn plus(self, other: Self) -> Result<Self, DateError> {
let months = self.months.checked_add(other.months).ok_or_else(|| DateError::OutOfRange("duration months".into()))?;
let ms = self.ms.checked_add(other.ms).ok_or_else(|| DateError::OutOfRange("duration ms".into()))?;
Ok(Self::new(months, ms))
}
/// The duration as text, for example `1M 4h 3m`.
#[must_use]
pub fn to_display_string(self) -> String {
let negative = self.ms < 0;
let mut rest = self.ms.saturating_abs();
let days = rest / MS_PER_DAY;
rest %= MS_PER_DAY;
let hours = rest / MS_PER_HOUR;
rest %= MS_PER_HOUR;
let minutes = rest / MS_PER_MINUTE;
rest %= MS_PER_MINUTE;
let seconds = rest / MS_PER_SECOND;
let mut clock: Vec<String> = Vec::new();
if days != 0 {
clock.push(format!("{days}d"));
}
if hours != 0 {
clock.push(format!("{hours}h"));
}
if minutes != 0 {
clock.push(format!("{minutes}m"));
}
if seconds != 0 {
clock.push(format!("{seconds}s"));
}
let clock = if clock.is_empty() {
String::new()
} else {
let text = clock.join(" ");
if negative { format!("-{text}") } else { text }
};
let mut parts: Vec<String> = Vec::new();
if self.months != 0 {
parts.push(format!("{}M", self.months));
}
if !clock.is_empty() {
parts.push(clock);
}
if parts.is_empty() {
return "0s".to_string();
}
parts.join(" ")
}
}
/// The units a duration string accepts.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Unit {
Year,
Month,
Week,
Day,
Hour,
Minute,
Second,
}
impl Unit {
/// The unit of a duration string, `None` when the text is not a unit.
fn of(text: &str) -> Option<Self> {
match text {
"y" | "year" | "years" => Some(Self::Year),
"M" | "month" | "months" => Some(Self::Month),
"w" | "week" | "weeks" => Some(Self::Week),
"d" | "day" | "days" => Some(Self::Day),
"h" | "hour" | "hours" => Some(Self::Hour),
"m" | "minute" | "minutes" => Some(Self::Minute),
"s" | "second" | "seconds" => Some(Self::Second),
_ => None,
}
}
}
/// Reads a duration string: `1M 4h`, `2 hours`, `-1d`, `1M4h`.
///
/// The units are `y`/`year`/`years`, `M`/`month`/`months`, `w`/`week`/`weeks`,
/// `d`/`day`/`days`, `h`/`hour`/`hours`, `m`/`minute`/`minutes` and
/// `s`/`second`/`seconds` (`M` is a month, `m` is a minute).
///
/// # Errors
///
/// [`DateError::InvalidDuration`] when the text holds no valid unit.
pub fn parse_duration(text: &str) -> Result<Duration, DateError> {
let invalid = || DateError::InvalidDuration(text.to_string());
let mut months: i64 = 0;
let mut ms: i64 = 0;
let mut chars = text.chars().peekable();
let mut seen = false;
loop {
while chars.peek().is_some_and(|c| c.is_whitespace()) {
chars.next();
}
let Some(first) = chars.peek().copied() else {
break;
};
let mut negative = false;
if first == '-' || first == '+' {
negative = first == '-';
chars.next();
}
let mut digits = String::new();
while let Some(c) = chars.peek().copied() {
if !c.is_ascii_digit() {
break;
}
digits.push(c);
chars.next();
}
if digits.is_empty() {
return Err(invalid());
}
while chars.peek().is_some_and(|c| c.is_whitespace()) {
chars.next();
}
let mut unit = String::new();
while let Some(c) = chars.peek().copied() {
if !c.is_ascii_alphabetic() {
break;
}
unit.push(c);
chars.next();
}
let unit = Unit::of(&unit).ok_or_else(invalid)?;
let amount: i64 = digits.parse().map_err(|_| invalid())?;
let amount = if negative { amount.checked_neg().ok_or_else(invalid)? } else { amount };
match unit {
Unit::Year => {
months = months.checked_add(amount.checked_mul(MONTHS_PER_YEAR).ok_or_else(invalid)?).ok_or_else(invalid)?;
}
Unit::Month => {
months = months.checked_add(amount).ok_or_else(invalid)?;
}
Unit::Week => {
ms = ms.checked_add(amount.checked_mul(DAYS_PER_WEEK * MS_PER_DAY).ok_or_else(invalid)?).ok_or_else(invalid)?;
}
Unit::Day => {
ms = ms.checked_add(amount.checked_mul(MS_PER_DAY).ok_or_else(invalid)?).ok_or_else(invalid)?;
}
Unit::Hour => {
ms = ms.checked_add(amount.checked_mul(MS_PER_HOUR).ok_or_else(invalid)?).ok_or_else(invalid)?;
}
Unit::Minute => {
ms = ms.checked_add(amount.checked_mul(MS_PER_MINUTE).ok_or_else(invalid)?).ok_or_else(invalid)?;
}
Unit::Second => {
ms = ms.checked_add(amount.checked_mul(MS_PER_SECOND).ok_or_else(invalid)?).ok_or_else(invalid)?;
}
}
seen = true;
}
if !seen {
return Err(invalid());
}
Ok(Duration::new(months, ms))
}
+37
View File
@@ -15,9 +15,46 @@ pub enum Error {
/// What was wrong, in plain words.
message: String,
},
/// A value is well formed YAML but not valid for the format (a negative limit, a bad direction).
#[error("invalid value: {0}")]
Invalid(String),
/// A limit of the format was reached (size, depth, node count).
#[error("limit reached: {0}")]
Limit(String),
/// An object type definition or schema is not valid.
#[error("object type {kind}: {message}")]
Schema {
/// Which rule was broken.
kind: SchemaErrorKind,
/// What was wrong, naming the type or field.
message: String,
},
}
/// The rule an object type schema broke.
#[derive(Debug, Clone, Copy, PartialEq, Eq, ThisError)]
pub enum SchemaErrorKind {
/// An id, tag, label or field key is not acceptable.
#[error("bad name")]
BadName,
/// Two types share an id or a tag, or a type repeats a field key.
#[error("duplicate")]
Duplicate,
/// `extends` names a type that does not exist.
#[error("unknown parent")]
UnknownParent,
/// The `extends` chain loops back on itself.
#[error("cycle")]
Cycle,
/// The `extends` chain is deeper than the limit.
#[error("too deep")]
TooDeep,
/// A default does not fit the kind of its field.
#[error("bad default")]
BadDefault,
/// Two schemas give the same field different kinds or defaults.
#[error("conflict")]
Conflict,
}
/// Result alias of the crate.
+76
View File
@@ -0,0 +1,76 @@
//! The formulas of a base, parsed once, with their dependency cycles found up front.
use crate::expr::{Expr, parse_expr};
/// What a formula name holds.
pub(super) enum Entry {
/// A parsed expression.
Ok(Expr),
/// A source that does not parse, with the message.
Bad(String),
/// A formula that depends on itself, with the message.
Cycle(String),
}
/// The named formulas of a base.
pub struct Formulas {
entries: Vec<(String, Entry)>,
}
impl Formulas {
/// Parses every formula. A bad source or a dependency cycle is kept as an error value of that
/// formula, so the rest of the table still evaluates.
#[must_use]
pub fn new(formulas: &[(String, String)]) -> Self {
let mut entries: Vec<(String, Entry)> = Vec::new();
let mut graph: Vec<(String, Vec<String>)> = Vec::new();
for (name, source) in formulas {
match parse_expr(source) {
Ok(expr) => {
let deps = expr.deps().formula.iter().filter_map(|d| d.split('.').next().map(str::to_string)).collect();
graph.push((name.clone(), deps));
entries.push((name.clone(), Entry::Ok(expr)));
}
Err(e) => entries.push((name.clone(), Entry::Bad(e.to_string()))),
}
}
for (name, entry) in &mut entries {
let mut path = vec![name.clone()];
if matches!(entry, Entry::Ok(_)) && find_cycle(&graph, name, name, &mut path, &mut Vec::new()) {
*entry = Entry::Cycle(format!("formula cycle: {}", path.join(" -> ")));
}
}
Self { entries }
}
/// No formulas at all.
#[must_use]
pub fn empty() -> Self {
Self { entries: Vec::new() }
}
pub(super) fn get(&self, name: &str) -> Option<&Entry> {
self.entries.iter().find(|(n, _)| n == name).map(|(_, e)| e)
}
}
/// Depth-first search for a way from `cur` back to `target`; `path` collects the names on the way.
fn find_cycle(graph: &[(String, Vec<String>)], target: &str, cur: &str, path: &mut Vec<String>, seen: &mut Vec<String>) -> bool {
let Some((_, deps)) = graph.iter().find(|(n, _)| n == cur) else { return false };
for dep in deps {
if dep == target {
path.push(dep.clone());
return true;
}
if seen.contains(dep) {
continue;
}
seen.push(dep.clone());
path.push(dep.clone());
if find_cycle(graph, target, dep, path, seen) {
return true;
}
path.pop();
}
false
}
+253
View File
@@ -0,0 +1,253 @@
//! Global, any-type, string, number and object functions.
//!
//! Every function returns `None` when the name is not one it knows for that receiver, so the
//! caller can try the next group. Arguments arrive evaluated and free of errors.
use super::ops::{MAX_STR, err, slice_range, to_i64, to_usize};
use super::{Ctx, int_to_f64};
use crate::compare::truthy;
use crate::datetime::{Date, parse_duration};
use crate::value::Value;
const GLOBALS: [&str; 15] =
["date", "duration", "number", "min", "max", "now", "today", "random", "list", "link", "escapeHTML", "html", "image", "icon", "file"];
/// Largest `toFixed` precision.
const MAX_PRECISION: usize = 100;
/// Largest `round` digit count.
const MAX_DIGITS: i32 = 15;
fn arity(name: &str, count: usize) -> Value {
err(format!("{name}() called with {count} arguments"))
}
fn wrong(name: &str, value: &Value) -> Value {
err(format!("{name}() does not accept {}", value.type_name()))
}
pub(super) fn global(ctx: &Ctx<'_, '_>, name: &str, args: &[Value]) -> Option<Value> {
let value = match (name, args) {
("now", []) => Value::Date(Date::now(ctx.env.now_ms)),
("today", []) => Value::Date(Date::today(ctx.env.now_ms)),
("random", []) => Value::Number(ctx.next_random()),
("date", [v]) => match v {
Value::Null | Value::Date(_) => v.clone(),
Value::Str(s) => Date::parse(s).map_or_else(|e| err(e.to_string()), Value::Date),
other => wrong(name, other),
},
("duration", [v]) => match v {
Value::Null | Value::Duration(_) => v.clone(),
Value::Str(s) => parse_duration(s).map_or_else(|e| err(e.to_string()), Value::Duration),
other => wrong(name, other),
},
("number", [v]) => to_number(v),
("min" | "max", [_, ..]) => extreme(name == "max", args),
("list", [v]) => match v {
Value::List(_) => v.clone(),
Value::Null => Value::List(Vec::new()),
other => Value::List(vec![other.clone()]),
},
("file", [v]) => super::funcs_file::file(ctx, v),
("link", [path]) => link(path, None),
("link", [path, display]) => link(path, Some(display)),
("escapeHTML", [v]) => match v {
Value::Null => Value::Null,
Value::Str(s) => Value::Str(escape_html(s)),
other => wrong(name, other),
},
("html", [v]) => match v {
Value::Null => Value::Null,
Value::Str(s) => Value::Html(s.clone()),
other => wrong(name, other),
},
("image", [v]) => match v {
Value::Null | Value::Image(_) => v.clone(),
Value::Str(s) => Value::Image(s.clone()),
Value::File { path } => Value::Image(path.clone()),
Value::Link { target, .. } => Value::Image(target.clone()),
other => wrong(name, other),
},
("icon", [v]) => match v {
Value::Null | Value::Icon(_) => v.clone(),
Value::Str(s) => Value::Icon(s.clone()),
other => wrong(name, other),
},
_ if GLOBALS.contains(&name) => arity(name, args.len()),
_ => return None,
};
Some(value)
}
fn to_number(value: &Value) -> Value {
match value {
Value::Number(_) | Value::Null => value.clone(),
Value::Bool(b) => Value::Number(if *b { 1.0 } else { 0.0 }),
Value::Date(d) => Value::Number(int_to_f64(d.ms())),
Value::Str(s) => match s.trim().parse::<f64>() {
Ok(n) if n.is_finite() => Value::Number(n),
_ => err(format!("`{s}` is not a number")),
},
other => wrong("number", other),
}
}
fn extreme(is_max: bool, args: &[Value]) -> Value {
let mut best: Option<f64> = None;
for arg in args {
match arg {
Value::Null => return Value::Null,
Value::Number(n) => best = Some(best.map_or(*n, |b| if is_max { b.max(*n) } else { b.min(*n) })),
other => return wrong(if is_max { "max" } else { "min" }, other),
}
}
best.map_or(Value::Null, Value::Number)
}
/// `link("[[Note|Shown]]")`, `link("path", "Shown")` or `link(file)`.
fn link(path: &Value, display: Option<&Value>) -> Value {
let (target, own) = match path {
Value::Null => return Value::Null,
Value::Link { target, display } => (target.clone(), display.clone()),
Value::File { path } => (path.clone(), String::new()),
Value::Str(s) => {
let inner = s.trim().strip_prefix("[[").and_then(|t| t.strip_suffix("]]")).unwrap_or(s);
match inner.split_once('|') {
Some((target, shown)) => (target.to_string(), shown.to_string()),
None => (inner.to_string(), String::new()),
}
}
other => return wrong("link", other),
};
let display = display.map_or(own, Value::to_display_string);
Value::Link { target, display }
}
fn escape_html(text: &str) -> String {
let mut out = String::with_capacity(text.len());
for c in text.chars() {
match c {
'&' => out.push_str("&amp;"),
'<' => out.push_str("&lt;"),
'>' => out.push_str("&gt;"),
'"' => out.push_str("&quot;"),
'\'' => out.push_str("&#39;"),
c => out.push(c),
}
}
out
}
pub(super) fn method(_ctx: &Ctx<'_, '_>, recv: &Value, name: &str, args: &[Value]) -> Option<Value> {
match (name, args) {
("isTruthy", []) => return Some(Value::Bool(truthy(recv))),
("isType", [Value::Str(t)]) => return Some(Value::Bool(recv.type_name() == t)),
("isType", _) => return Some(err("isType() takes a type name")),
("toString", []) => return Some(Value::Str(recv.to_display_string())),
_ => {}
}
match recv {
Value::Null => Some(if name == "isEmpty" { Value::Bool(true) } else { Value::Null }),
Value::Str(s) => string_method(s, name, args),
Value::Number(n) => number_method(*n, name, args),
Value::Object(fields) => object_method(fields, name, args),
_ => None,
}
}
/// The string arguments of a call, or the error to return.
fn strings<'a>(name: &str, args: &'a [Value]) -> Result<Vec<&'a str>, Value> {
args.iter().map(|a| if let Value::Str(s) = a { Ok(s.as_str()) } else { Err(wrong(name, a)) }).collect()
}
fn string_method(s: &str, name: &str, args: &[Value]) -> Option<Value> {
let bool_of = |f: &dyn Fn(&[&str]) -> bool| strings(name, args).map_or_else(|e| e, |v| Value::Bool(f(&v)));
Some(match (name, args) {
("contains", [_]) => bool_of(&|v| s.contains(v[0])),
("startsWith", [_]) => bool_of(&|v| s.starts_with(v[0])),
("endsWith", [_]) => bool_of(&|v| s.ends_with(v[0])),
("containsAll", _) => bool_of(&|v| v.iter().all(|x| s.contains(x))),
("containsAny", _) => bool_of(&|v| v.iter().any(|x| s.contains(x))),
("isEmpty", []) => Value::Bool(s.is_empty()),
("lower", []) => Value::Str(s.to_lowercase()),
("trim", []) => Value::Str(s.trim().to_string()),
("reverse", []) => Value::Str(s.chars().rev().collect()),
("title", []) => Value::Str(title(s)),
("repeat", [n]) => match to_usize(n) {
Some(count) if s.len().saturating_mul(count) <= MAX_STR => Value::Str(s.repeat(count)),
Some(_) => err("string too long"),
None => err("repeat() needs a non-negative whole number"),
},
("slice", [start] | [start, _]) => match slice_range(s.chars().count(), start, args.get(1)) {
Ok((from, to)) => Value::Str(s.chars().skip(from).take(to - from).collect()),
Err(e) => e,
},
("split", [Value::Str(sep)]) => split(s, sep, None),
("split", [Value::Str(sep), n]) => to_usize(n).map_or_else(|| err("split() needs a whole count"), |n| split(s, sep, Some(n))),
("replace", [Value::Str(from), Value::Str(to)]) => replace(s, from, to),
("replace" | "split", [Value::Regex { .. }, ..]) => return None,
("replace" | "split", _) => err(format!("{name}() needs string arguments")),
("contains" | "startsWith" | "endsWith" | "isEmpty" | "lower" | "trim" | "reverse" | "title" | "repeat" | "slice", _) => arity(name, args.len()),
_ => return None,
})
}
fn title(s: &str) -> String {
let mut out = String::with_capacity(s.len());
let mut start = true;
for c in s.chars() {
if start {
out.extend(c.to_uppercase());
} else {
out.push(c);
}
start = c.is_whitespace();
}
out
}
fn split(s: &str, sep: &str, limit: Option<usize>) -> Value {
let parts: Vec<Value> =
if sep.is_empty() { s.chars().map(|c| Value::Str(c.to_string())).collect() } else { s.split(sep).map(|p| Value::Str(p.to_string())).collect() };
Value::List(parts.into_iter().take(limit.unwrap_or(usize::MAX)).collect())
}
fn replace(s: &str, from: &str, to: &str) -> Value {
let pieces = if from.is_empty() { s.chars().count() + 1 } else { s.matches(from).count() };
if s.len().saturating_add(pieces.saturating_mul(to.len())) > MAX_STR {
return err("string too long");
}
Value::Str(if from.is_empty() { s.to_string() } else { s.replace(from, to) })
}
fn number_method(n: f64, name: &str, args: &[Value]) -> Option<Value> {
Some(match (name, args) {
("abs", []) => Value::Number(n.abs()),
("ceil", []) => Value::Number(n.ceil()),
("floor", []) => Value::Number(n.floor()),
("isEmpty", []) => Value::Bool(false),
("round", []) => Value::Number((n + 0.5).floor()),
("round", [digits]) => match to_i64(digits).and_then(|d| i32::try_from(d).ok()).filter(|d| (0..=MAX_DIGITS).contains(d)) {
Some(d) => {
let factor = 10f64.powi(d);
Value::Number((n * factor + 0.5).floor() / factor)
}
None => err(format!("round() digits must be a whole number from 0 to {MAX_DIGITS}")),
},
("toFixed", [p]) => match to_usize(p).filter(|p| *p <= MAX_PRECISION) {
Some(p) if n.is_finite() => Value::Str(format!("{n:.p$}")),
Some(_) => Value::Str(n.to_string()),
None => err(format!("toFixed() precision must be a whole number from 0 to {MAX_PRECISION}")),
},
("abs" | "ceil" | "floor" | "isEmpty" | "round" | "toFixed", _) => arity(name, args.len()),
_ => return None,
})
}
fn object_method(fields: &[(String, Value)], name: &str, args: &[Value]) -> Option<Value> {
Some(match (name, args) {
("keys", []) => Value::List(fields.iter().map(|(k, _)| Value::Str(k.clone())).collect()),
("values", []) => Value::List(fields.iter().map(|(_, v)| v.clone()).collect()),
("isEmpty", []) => Value::Bool(fields.is_empty()),
("keys" | "values" | "isEmpty", _) => arity(name, args.len()),
_ => return None,
})
}
+159
View File
@@ -0,0 +1,159 @@
//! Date fields and methods: `year`..`millisecond`, `date format time relative isEmpty`.
//!
//! `format` understands the Moment tokens `YYYY YY MMMM MMM MM M DD D Do dddd ddd d HH H hh h mm m
//! ss s SSS A a`; text in `[brackets]` is literal and any other character is copied.
use super::Ctx;
use super::ops::err;
use crate::datetime::{Date, MS_PER_DAY, MS_PER_HOUR, MS_PER_MINUTE, MS_PER_SECOND};
use crate::value::Value;
const MONTHS: [&str; 12] = ["January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December"];
const DAYS: [&str; 7] = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"];
/// Longest token first, so `YYYY` wins over `YY` and `Do` over `D`.
const TOKENS: &[&str] =
&["YYYY", "MMMM", "dddd", "SSS", "ddd", "MMM", "Do", "YY", "MM", "DD", "HH", "hh", "mm", "ss", "M", "D", "d", "H", "h", "m", "s", "A", "a"];
/// The weekday of day 0 (1970-01-01, a Thursday), with Sunday as 0.
const EPOCH_WEEKDAY: i64 = 4;
const DAYS_PER_WEEK: i64 = 7;
/// Moment's thresholds for `relative()`, in milliseconds.
const FEW_SECONDS: i64 = 45 * MS_PER_SECOND;
const A_MINUTE: i64 = 90 * MS_PER_SECOND;
const MINUTES: i64 = 45 * MS_PER_MINUTE;
const AN_HOUR: i64 = 90 * MS_PER_MINUTE;
const HOURS: i64 = 22 * MS_PER_HOUR;
const A_DAY: i64 = 36 * MS_PER_HOUR;
const DAYS_LIMIT: i64 = 26 * MS_PER_DAY;
const A_MONTH: i64 = 45 * MS_PER_DAY;
const MONTHS_LIMIT: i64 = 320 * MS_PER_DAY;
const A_YEAR: i64 = 548 * MS_PER_DAY;
const DAYS_PER_MONTH: i64 = 30;
const DAYS_PER_YEAR: i64 = 365;
pub(super) fn field(date: Date, name: &str) -> Value {
let p = date.parts();
Value::Number(match name {
"year" => f64::from(p.year),
"month" => f64::from(p.month),
"day" => f64::from(p.day),
"hour" => f64::from(p.hour),
"minute" => f64::from(p.minute),
"second" => f64::from(p.second),
"millisecond" => f64::from(p.millisecond),
_ => return err(format!("date has no field `{name}`")),
})
}
pub(super) fn method(ctx: &Ctx<'_, '_>, recv: &Value, name: &str, args: &[Value]) -> Option<Value> {
let Value::Date(date) = recv else { return None };
Some(match (name, args) {
("date", []) => Value::Date(Date::new(date.ms().div_euclid(MS_PER_DAY).saturating_mul(MS_PER_DAY), false)),
("time", []) => Value::Str(format(*date, "HH:mm:ss")),
("format", [Value::Str(fmt)]) => Value::Str(format(*date, fmt)),
("format", [other]) => err(format!("format() does not accept {}", other.type_name())),
("relative", []) => Value::Str(relative(ctx.env.now_ms.saturating_sub(date.ms()))),
("isEmpty", []) => Value::Bool(false),
("date" | "time" | "format" | "relative" | "isEmpty", _) => err(format!("{name}() called with {} arguments", args.len())),
_ => return None,
})
}
fn pad(value: impl std::fmt::Display + Copy, width: usize) -> String {
format!("{value:0>width$}")
}
fn hour12(hour: u8) -> u8 {
if hour.is_multiple_of(12) { 12 } else { hour % 12 }
}
fn ordinal(day: u8) -> String {
let suffix = match (day / 10 % 10, day % 10) {
(1, _) => "th",
(_, 1) => "st",
(_, 2) => "nd",
(_, 3) => "rd",
_ => "th",
};
format!("{day}{suffix}")
}
fn token(date: Date, token: &str) -> String {
let p = date.parts();
let weekday = usize::try_from((date.ms().div_euclid(MS_PER_DAY) + EPOCH_WEEKDAY).rem_euclid(DAYS_PER_WEEK)).unwrap_or(0);
let month = MONTHS.get(usize::from(p.month).wrapping_sub(1)).copied().unwrap_or_default();
let day_name = DAYS.get(weekday).copied().unwrap_or_default();
let short = |name: &str| name.chars().take(3).collect::<String>();
match token {
"YYYY" => pad(p.year, 4),
"YY" => pad(p.year.rem_euclid(100), 2),
"MMMM" => month.to_string(),
"MMM" => short(month),
"MM" => pad(p.month, 2),
"M" => p.month.to_string(),
"DD" => pad(p.day, 2),
"D" => p.day.to_string(),
"Do" => ordinal(p.day),
"dddd" => day_name.to_string(),
"ddd" => short(day_name),
"d" => weekday.to_string(),
"HH" => pad(p.hour, 2),
"H" => p.hour.to_string(),
"hh" => pad(hour12(p.hour), 2),
"h" => hour12(p.hour).to_string(),
"mm" => pad(p.minute, 2),
"m" => p.minute.to_string(),
"ss" => pad(p.second, 2),
"s" => p.second.to_string(),
"SSS" => pad(p.millisecond, 3),
"A" => (if p.hour < 12 { "AM" } else { "PM" }).to_string(),
_ => (if p.hour < 12 { "am" } else { "pm" }).to_string(),
}
}
/// Formats a date with a Moment format string.
pub(crate) fn format(date: Date, fmt: &str) -> String {
let mut out = String::new();
let mut rest = fmt;
while let Some(first) = rest.chars().next() {
if let Some(tail) = rest.strip_prefix('[') {
if let Some((literal, after)) = tail.split_once(']') {
out.push_str(literal);
rest = after;
} else {
out.push('[');
rest = tail;
}
} else if let Some((tok, tail)) = TOKENS.iter().find_map(|t| rest.strip_prefix(t).map(|tail| (*t, tail))) {
out.push_str(&token(date, tok));
rest = tail;
} else {
out.push(first);
rest = rest.get(first.len_utf8()..).unwrap_or_default();
}
}
out
}
fn plural(count: i64, unit: &str) -> String {
format!("{count} {unit}{}", if count == 1 { "" } else { "s" })
}
/// "3 days ago" or "in 2 hours", for `elapsed` = now - date in milliseconds.
fn relative(elapsed: i64) -> String {
let span = elapsed.saturating_abs();
let rounded = |unit: i64| (span + unit / 2) / unit;
let phrase = match span {
s if s < FEW_SECONDS => "a few seconds".to_string(),
s if s < A_MINUTE => "a minute".to_string(),
s if s < MINUTES => plural(rounded(MS_PER_MINUTE), "minute"),
s if s < AN_HOUR => "an hour".to_string(),
s if s < HOURS => plural(rounded(MS_PER_HOUR), "hour"),
s if s < A_DAY => "a day".to_string(),
s if s < DAYS_LIMIT => plural(rounded(MS_PER_DAY), "day"),
s if s < A_MONTH => "a month".to_string(),
s if s < MONTHS_LIMIT => plural(rounded(DAYS_PER_MONTH * MS_PER_DAY).max(2), "month"),
s if s < A_YEAR => "a year".to_string(),
_ => plural(rounded(DAYS_PER_YEAR * MS_PER_DAY), "year"),
};
if elapsed < 0 { format!("in {phrase}") } else { format!("{phrase} ago") }
}
+111
View File
@@ -0,0 +1,111 @@
//! File and link functions: `file()`, `asLink hasLink hasProperty hasTag inFolder`, `asFile linksTo`.
//!
//! Other files are looked up through the [`NoteSource`](crate::NoteSource) of the environment; a
//! file the source does not know answers `false` to the predicates.
use super::Ctx;
use super::ops::err;
use crate::row::RowRef;
use crate::value::Value;
/// The part of a link target that names the file: no `#heading`, no `|alias`.
fn file_part(target: &str) -> &str {
target.split(['#', '|']).next().unwrap_or(target)
}
impl Ctx<'_, '_> {
/// The path a link target resolves to, as seen from `from`.
fn resolve(&self, target: &str, from: &str) -> Option<String> {
self.env.source?.resolve_link(file_part(target), from)
}
/// The path a file, link or string argument stands for.
fn path_of(&self, value: &Value, from: &str) -> Option<String> {
match value {
Value::File { path } => Some(path.clone()),
Value::Link { target, .. } | Value::Str(target) => Some(self.resolve(target, from).unwrap_or_else(|| file_part(target).to_string())),
_ => None,
}
}
fn row_links_to(&self, row: &RowRef<'_>, other: &str) -> bool {
row.links.iter().any(|link| match self.resolve(link, row.path) {
Some(path) => path == other,
None => file_part(link) == other || [file_part(link), ".md"].concat() == other,
})
}
}
/// The global `file(path | file | link)`.
pub(super) fn file(ctx: &Ctx<'_, '_>, value: &Value) -> Value {
match value {
Value::Null => Value::Null,
Value::File { .. } => value.clone(),
Value::Link { target, .. } | Value::Str(target) => match (ctx.resolve(target, ctx.row.path), ctx.env.source) {
(Some(path), _) => Value::File { path },
(None, Some(_)) => Value::Null,
(None, None) => Value::File { path: file_part(target).to_string() },
},
other => err(format!("file() does not accept {}", other.type_name())),
}
}
/// Whether `tag` is `want` or one of its nested tags; case and the leading `#` do not matter.
fn tag_matches(tag: &str, want: &str) -> bool {
let (tag, want) = (tag.trim_start_matches('#').to_lowercase(), want.trim_start_matches('#').to_lowercase());
tag == want || tag.strip_prefix(&want).is_some_and(|rest| rest.starts_with('/'))
}
pub(super) fn method(ctx: &Ctx<'_, '_>, recv: &Value, name: &str, args: &[Value]) -> Option<Value> {
match recv {
Value::File { path } => file_method(ctx, path, name, args),
Value::Link { target, display } => link_method(ctx, target, display, name, args),
_ => None,
}
}
fn file_method(ctx: &Ctx<'_, '_>, path: &str, name: &str, args: &[Value]) -> Option<Value> {
if name == "asLink" {
return Some(match args {
[] => Value::Link { target: path.to_string(), display: String::new() },
[shown] => Value::Link { target: path.to_string(), display: shown.to_display_string() },
_ => err("asLink() takes at most one argument"),
});
}
if !["hasLink", "hasProperty", "hasTag", "inFolder"].contains(&name) {
return None;
}
let Some(row) = ctx.lookup(path) else { return Some(Value::Bool(false)) };
Some(match (name, args) {
("hasLink", [other]) => {
ctx.path_of(other, path).map_or_else(|| err("hasLink() needs a file, link or path"), |p| Value::Bool(ctx.row_links_to(&row, &p)))
}
("hasProperty", [Value::Str(key)]) => Value::Bool(row.prop(key).is_some()),
("hasTag", _) => match texts(name, args) {
Ok(wanted) => Value::Bool(wanted.iter().any(|w| row.tags.iter().any(|t| tag_matches(t, w)))),
Err(e) => e,
},
("inFolder", [Value::Str(folder)]) => {
let folder = folder.trim_matches('/');
Value::Bool(folder.is_empty() || row.folder() == folder || row.folder().strip_prefix(folder).is_some_and(|rest| rest.starts_with('/')))
}
_ => err(format!("{name}() got arguments it does not accept")),
})
}
fn texts<'a>(name: &str, args: &'a [Value]) -> Result<Vec<&'a str>, Value> {
args.iter().map(|a| if let Value::Str(s) = a { Ok(s.as_str()) } else { Err(err(format!("{name}() needs strings"))) }).collect()
}
fn link_method(ctx: &Ctx<'_, '_>, target: &str, _display: &str, name: &str, args: &[Value]) -> Option<Value> {
let from = ctx.row.path;
Some(match (name, args) {
("asFile", []) => ctx.resolve(target, from).map_or(Value::Null, |path| Value::File { path }),
("linksTo", [other]) => {
let (Some(path), Some(wanted)) = (ctx.resolve(target, from), ctx.path_of(other, from)) else { return Some(Value::Bool(false)) };
Value::Bool(ctx.lookup(&path).is_some_and(|row| ctx.row_links_to(&row, &wanted)))
}
("asFile" | "linksTo", _) => err(format!("{name}() called with {} arguments", args.len())),
_ => return None,
})
}
+97
View File
@@ -0,0 +1,97 @@
//! List functions. `filter`, `map` and `reduce` take an expression and bind `value`, `index` and
//! (for `reduce`) `acc` while they evaluate it for each item.
use super::ops::{MAX_STR, err, slice_range};
use super::{Ctx, Scope};
use crate::compare::{compare, loose_eq, truthy};
use crate::expr::Expr;
use crate::value::Value;
/// The lambda methods, evaluated before their arguments.
pub(super) fn lazy(ctx: &Ctx<'_, '_>, recv: &Value, name: &str, args: &[Expr], sc: Scope<'_>) -> Option<Value> {
if !["filter", "map", "reduce"].contains(&name) {
return None;
}
let items = match recv {
Value::List(items) => items,
Value::Null => return Some(Value::Null),
other => return Some(err(format!("{} has no method `{name}`", other.type_name()))),
};
let run =
|body: &Expr, item: &Value, index: usize, acc: Option<&Value>| ctx.eval(body, Scope { value: Some(item), index: Some(index), acc: acc.or(sc.acc) });
Some(match (name, args) {
("filter", [body]) => {
let mut kept = Vec::new();
for (i, item) in items.iter().enumerate() {
match run(body, item, i, None) {
e @ Value::Error(_) => return Some(e),
keep if truthy(&keep) => kept.push(item.clone()),
_ => {}
}
}
Value::List(kept)
}
("map", [body]) => {
let mut out = Vec::with_capacity(items.len());
for (i, item) in items.iter().enumerate() {
match run(body, item, i, None) {
e @ Value::Error(_) => return Some(e),
mapped => out.push(mapped),
}
}
Value::List(out)
}
("reduce", [body, init]) => {
let mut acc = ctx.eval(init, sc);
for (i, item) in items.iter().enumerate() {
if let Value::Error(_) = acc {
break;
}
acc = run(body, item, i, Some(&acc));
}
acc
}
_ => err(format!("{name}() called with {} arguments", args.len())),
})
}
pub(super) fn method(recv: &Value, name: &str, args: &[Value]) -> Option<Value> {
let Value::List(items) = recv else { return None };
Some(match (name, args) {
("contains", [x]) => Value::Bool(items.iter().any(|i| loose_eq(i, x))),
("containsAll", _) => Value::Bool(args.iter().all(|x| items.iter().any(|i| loose_eq(i, x)))),
("containsAny", _) => Value::Bool(args.iter().any(|x| items.iter().any(|i| loose_eq(i, x)))),
("isEmpty", []) => Value::Bool(items.is_empty()),
("flat", []) => Value::List(items.iter().flat_map(|i| if let Value::List(inner) = i { inner.clone() } else { vec![i.clone()] }).collect()),
("reverse", []) => Value::List(items.iter().rev().cloned().collect()),
("sort", []) => {
let mut sorted = items.clone();
sorted.sort_by(compare);
Value::List(sorted)
}
("unique", []) => {
// ponytail: quadratic, lists are capped at 500 items by the index; sort+dedup if that grows
let mut seen: Vec<Value> = Vec::new();
for item in items {
if !seen.iter().any(|s| loose_eq(s, item)) {
seen.push(item.clone());
}
}
Value::List(seen)
}
("join", [Value::Str(sep)]) => {
let text = items.iter().map(Value::to_display_string).collect::<Vec<_>>().join(sep);
if text.len() > MAX_STR { err("string too long") } else { Value::Str(text) }
}
("slice", [start] | [start, _]) => match slice_range(items.len(), start, args.get(1)) {
Ok((from, to)) => Value::List(items.iter().skip(from).take(to - from).cloned().collect()),
Err(e) => e,
},
("sum" | "mean" | "median" | "min" | "max", []) => crate::query::number_stat(name, items)?,
("join", [other]) => err(format!("join() does not accept {}", other.type_name())),
("contains" | "isEmpty" | "flat" | "reverse" | "sort" | "unique" | "join" | "slice", _) => {
err(format!("{name}() called with {} arguments", args.len()))
}
_ => return None,
})
}
+296
View File
@@ -0,0 +1,296 @@
//! The evaluator: walks an [`Expr`] over one row and gives a [`Value`].
//!
//! Nothing here panics or aborts: a failed evaluation is a [`Value::Error`] that flows through the
//! operators and functions like any other value. A missing property is `null`, and `null`
//! propagates through arithmetic and method calls. The recursion is bounded by [`MAX_DEPTH`].
mod formulas;
mod funcs_basic;
mod funcs_date;
mod funcs_file;
mod funcs_list;
mod ops;
mod regex;
use std::cell::{Cell, RefCell};
pub use formulas::Formulas;
pub(crate) use ops::{err, int_to_f64, to_usize};
use crate::compare::truthy;
use crate::expr::{BinOp, Expr};
use crate::row::{NoteSource, RowRef};
use crate::value::Value;
use formulas::Entry;
/// Deepest nesting of one evaluation, formula references included.
const MAX_DEPTH: usize = 200;
/// The multiplier of the golden ratio, mixed into the random stream.
const RANDOM_STEP: u64 = 0x9E37_79B9_7F4A_7C15;
const RANDOM_MIX_1: u64 = 0xBF58_476D_1CE4_E5B9;
const RANDOM_MIX_2: u64 = 0x94D0_49BB_1331_11EB;
/// Distinct values of the 32 random bits taken.
const RANDOM_RANGE: f64 = 4_294_967_296.0;
/// What an evaluation may know besides the row.
#[derive(Clone, Copy)]
pub struct Env<'a> {
/// The current moment, Unix milliseconds. The crate never reads the clock.
pub now_ms: i64,
/// Seed of `random()`, so results repeat.
pub seed: u64,
/// The row `this` stands for.
pub this: Option<RowRef<'a>>,
/// Where other files are looked up (`file()`, `asFile()`, `linksTo()`).
pub source: Option<&'a dyn NoteSource>,
}
impl Env<'_> {
/// An environment with no `this` and no source.
#[must_use]
pub fn new(now_ms: i64, seed: u64) -> Self {
Self { now_ms, seed, this: None, source: None }
}
}
/// The lambda variables in scope: `value`, `index` and `acc`.
#[derive(Clone, Copy, Default)]
pub(crate) struct Scope<'s> {
pub value: Option<&'s Value>,
pub index: Option<usize>,
pub acc: Option<&'s Value>,
}
/// One evaluation: the environment, the row and the formula memo.
pub(crate) struct Ctx<'c, 'r> {
pub env: &'c Env<'r>,
pub row: &'c RowRef<'r>,
formulas: &'c Formulas,
depth: Cell<usize>,
memo: RefCell<Vec<(String, Value)>>,
random_n: Cell<u64>,
}
/// Evaluates `expr` for `row`.
#[must_use]
pub fn eval(expr: &Expr, env: &Env<'_>, row: &RowRef<'_>, formulas: &Formulas) -> Value {
let ctx = Ctx { env, row, formulas, depth: Cell::new(0), memo: RefCell::new(Vec::new()), random_n: Cell::new(0) };
ctx.eval(expr, Scope::default())
}
impl<'r> Ctx<'_, 'r> {
pub(crate) fn eval(&self, expr: &Expr, sc: Scope<'_>) -> Value {
let depth = self.depth.get();
if depth >= MAX_DEPTH {
return err("expression too deep");
}
self.depth.set(depth + 1);
let value = self.eval_node(expr, sc);
self.depth.set(depth);
value
}
fn eval_node(&self, expr: &Expr, sc: Scope<'_>) -> Value {
match expr {
Expr::Null => Value::Null,
Expr::Bool(b) => Value::Bool(*b),
Expr::Num(n) => Value::Number(*n),
Expr::Str(s) => Value::Str(s.clone()),
Expr::Regex { pattern, flags } => Value::Regex { pattern: pattern.clone(), flags: flags.clone() },
Expr::Array(items) => self.eval_args(items, sc).map_or_else(|e| e, Value::List),
Expr::Object(pairs) => {
let mut out = Vec::with_capacity(pairs.len());
for (key, value) in pairs {
match self.eval(value, sc) {
e @ Value::Error(_) => return e,
v => out.push((key.clone(), v)),
}
}
Value::Object(out)
}
Expr::Ident(name) => self.ident(name, sc),
Expr::Member(base, name) => self.root_member(base, name).unwrap_or_else(|| self.field_of(&self.eval(base, sc), name)),
Expr::Index(base, index) => self.index(base, index, sc),
Expr::Call { callee, args } => self.call(callee, args, sc),
Expr::Unary(op, inner) => ops::unary(*op, &self.eval(inner, sc)),
Expr::Binary(op @ (BinOp::And | BinOp::Or), left, right) => {
let l = self.eval(left, sc);
if matches!(l, Value::Error(_)) {
return l;
}
if truthy(&l) == (*op == BinOp::Or) {
return Value::Bool(*op == BinOp::Or);
}
ops::binary(*op, &l, &self.eval(right, sc))
}
Expr::Binary(op, left, right) => ops::binary(*op, &self.eval(left, sc), &self.eval(right, sc)),
}
}
/// The values of `exprs`, or the first error among them.
pub(crate) fn eval_args(&self, exprs: &[Expr], sc: Scope<'_>) -> Result<Vec<Value>, Value> {
let mut out = Vec::with_capacity(exprs.len());
for expr in exprs {
match self.eval(expr, sc) {
e @ Value::Error(_) => return Err(e),
v => out.push(v),
}
}
Ok(out)
}
fn ident(&self, name: &str, sc: Scope<'_>) -> Value {
let var = match name {
"value" => sc.value.cloned(),
"index" => sc.index.map(|i| Value::Number(int_to_f64(i))),
"acc" => sc.acc.cloned(),
_ => None,
};
if let Some(v) = var {
return v;
}
match name {
"file" => Value::File { path: self.row.path.to_string() },
"this" => self.env.this.map_or(Value::Null, |t| Value::File { path: t.path.to_string() }),
"note" => Value::Object(self.row.props.to_vec()),
"formula" => err("`formula` needs a name: formula.name"),
_ => self.row.prop(name).cloned().unwrap_or(Value::Null),
}
}
/// `note.x`, `formula.x`, `file.x` and `this.x`, when `base` is one of those names.
fn root_member(&self, base: &Expr, name: &str) -> Option<Value> {
let Expr::Ident(root) = base else { return None };
match root.as_str() {
"note" => Some(self.row.prop(name).cloned().unwrap_or(Value::Null)),
"formula" => Some(self.formula(name)),
"file" => Some(self.row.field(name).unwrap_or_else(|| err(format!("unknown file field `{name}`")))),
"this" => Some(match self.env.this {
None => Value::Null,
Some(this) if name == "file" => Value::File { path: this.path.to_string() },
Some(this) => this.prop(name).cloned().unwrap_or(Value::Null),
}),
_ => None,
}
}
fn index(&self, base: &Expr, index: &Expr, sc: Scope<'_>) -> Value {
let key = self.eval(index, sc);
if let (Value::Str(name), true) = (&key, matches!(base, Expr::Ident(r) if ["note", "formula", "file", "this"].contains(&r.as_str()))) {
return self.root_member(base, name).unwrap_or(Value::Null);
}
let target = self.eval(base, sc);
match (&target, &key) {
(Value::Error(_), _) => target,
(_, Value::Error(_)) => key,
(Value::Null, _) => Value::Null,
(Value::List(items), k) => to_usize(k).and_then(|i| items.get(i)).cloned().unwrap_or(Value::Null),
(Value::Str(s), k) => to_usize(k).and_then(|i| s.chars().nth(i)).map_or(Value::Null, |c| Value::Str(c.to_string())),
(Value::Object(_) | Value::File { .. }, Value::Str(name)) => self.field_of(&target, name),
_ => err(format!("cannot index {} with {}", target.type_name(), key.type_name())),
}
}
/// The field `name` of a value (`list.length`, `file.name`, `object.key`).
pub(crate) fn field_of(&self, value: &Value, name: &str) -> Value {
match value {
Value::Error(_) => value.clone(),
Value::Null => Value::Null,
Value::Object(fields) => fields.iter().find(|(k, _)| k == name).map_or(Value::Null, |(_, v)| v.clone()),
Value::Date(d) => funcs_date::field(*d, name),
Value::Str(s) if name == "length" => Value::Number(int_to_f64(s.chars().count())),
Value::List(items) if name == "length" => Value::Number(int_to_f64(items.len())),
Value::File { path } => match self.lookup(path) {
Some(row) => row.field(name).unwrap_or_else(|| err(format!("unknown file field `{name}`"))),
None => Value::Null,
},
other => err(format!("{} has no field `{name}`", other.type_name())),
}
}
/// The row of a file: the current row, `this`, or a file of the source.
pub(crate) fn lookup(&self, path: &str) -> Option<RowRef<'r>> {
if self.row.path == path {
return Some(*self.row);
}
if let Some(this) = self.env.this.filter(|t| t.path == path) {
return Some(this);
}
self.env.source?.find(path)
}
fn call(&self, callee: &Expr, args: &[Expr], sc: Scope<'_>) -> Value {
match callee {
Expr::Ident(name) if name == "if" => self.call_if(args, sc),
Expr::Ident(name) => match self.eval_args(args, sc) {
Ok(values) => funcs_basic::global(self, name, &values).unwrap_or_else(|| err(format!("unknown function `{name}`"))),
Err(e) => e,
},
Expr::Member(recv, name) => {
let recv = self.eval(recv, sc);
self.method(&recv, name, args, sc)
}
_ => err("this expression is not a function"),
}
}
/// `if(condition, then, else?)`: only the chosen branch is evaluated.
fn call_if(&self, args: &[Expr], sc: Scope<'_>) -> Value {
let (condition, then, otherwise) = match args {
[c, t] => (c, t, None),
[c, t, e] => (c, t, Some(e)),
_ => return err("if() takes two or three arguments"),
};
match self.eval(condition, sc) {
e @ Value::Error(_) => e,
c if truthy(&c) => self.eval(then, sc),
_ => otherwise.map_or(Value::Null, |e| self.eval(e, sc)),
}
}
fn method(&self, recv: &Value, name: &str, args: &[Expr], sc: Scope<'_>) -> Value {
if let Value::Error(_) = recv {
return recv.clone();
}
if let Some(done) = funcs_list::lazy(self, recv, name, args, sc) {
return done;
}
let values = match self.eval_args(args, sc) {
Ok(values) => values,
Err(e) => return e,
};
funcs_basic::method(self, recv, name, &values)
.or_else(|| funcs_list::method(recv, name, &values))
.or_else(|| funcs_date::method(self, recv, name, &values))
.or_else(|| funcs_file::method(self, recv, name, &values))
.or_else(|| regex::method(recv, name, &values))
.unwrap_or_else(|| err(format!("{} has no method `{name}`", recv.type_name())))
}
/// The value of the formula `name`, evaluated once per evaluation.
fn formula(&self, name: &str) -> Value {
let hit = self.memo.borrow().iter().find(|(n, _)| n == name).map(|(_, v)| v.clone());
if let Some(v) = hit {
return v;
}
let value = match self.formulas.get(name) {
None => err(format!("unknown formula `{name}`")),
Some(Entry::Bad(message) | Entry::Cycle(message)) => err(message.clone()),
Some(Entry::Ok(expr)) => self.eval(expr, Scope::default()),
};
self.memo.borrow_mut().push((name.to_string(), value.clone()));
value
}
/// The next number of the seeded stream, in `[0, 1)`.
pub(crate) fn next_random(&self) -> f64 {
let n = self.random_n.get();
self.random_n.set(n.wrapping_add(1));
let salt = self.row.path.bytes().fold(0u64, |h, b| h.wrapping_mul(31).wrapping_add(u64::from(b)));
let mut z = self.env.seed.wrapping_add(salt).wrapping_add(n.wrapping_mul(RANDOM_STEP));
z = (z ^ (z >> 30)).wrapping_mul(RANDOM_MIX_1);
z = (z ^ (z >> 27)).wrapping_mul(RANDOM_MIX_2);
z ^= z >> 31;
f64::from(u32::try_from(z >> 32).unwrap_or(0)) / RANDOM_RANGE
}
}
+181
View File
@@ -0,0 +1,181 @@
//! Operators and numeric helpers of the evaluator, with the type rules of D5.
use std::cmp::Ordering;
use crate::compare::{compare, loose_eq, truthy};
use crate::datetime::{Date, Duration, add_duration, parse_duration, sub_dates};
use crate::expr::{BinOp, UnOp};
use crate::value::Value;
/// Longest string an operation may build, in bytes.
pub(super) const MAX_STR: usize = 1024 * 1024;
pub(crate) fn err(message: impl Into<String>) -> Value {
Value::Error(message.into())
}
/// An integer as a number, without a lossy cast.
pub(crate) fn int_to_f64(n: impl ToString + Copy) -> f64 {
n.to_string().parse().unwrap_or(f64::NAN)
}
/// A whole finite number as `i64`.
pub(crate) fn to_i64(value: &Value) -> Option<i64> {
match value {
Value::Number(n) if n.is_finite() && n.fract() == 0.0 => n.to_string().parse().ok(),
_ => None,
}
}
/// A whole non-negative number as `usize`.
pub(crate) fn to_usize(value: &Value) -> Option<usize> {
to_i64(value).and_then(|n| usize::try_from(n).ok())
}
/// The `[start, end)` range of `slice(start, end?)` over `len` items; negative indexes count
/// from the end and the range is clamped.
pub(crate) fn slice_range(len: usize, start: &Value, end: Option<&Value>) -> Result<(usize, usize), Value> {
let total = i64::try_from(len).unwrap_or(i64::MAX);
let resolve = |v: &Value| -> Result<usize, Value> {
let n = to_i64(v).ok_or_else(|| err("slice() needs whole numbers"))?;
let at = if n < 0 { total.saturating_add(n).max(0) } else { n.min(total) };
Ok(usize::try_from(at).unwrap_or(len))
};
let from = resolve(start)?;
let to = match end {
Some(v) => resolve(v)?,
None => len,
};
Ok((from, to.max(from)))
}
pub(super) fn unary(op: UnOp, value: &Value) -> Value {
match (op, value) {
(_, Value::Error(_)) => value.clone(),
(UnOp::Not, v) => Value::Bool(!truthy(v)),
(UnOp::Neg, Value::Number(n)) => Value::Number(-n),
(UnOp::Neg, Value::Duration(d)) => d.scaled(-1).map_or_else(|e| err(e.to_string()), Value::Duration),
(UnOp::Neg, Value::Null) => Value::Null,
(UnOp::Neg, other) => err(format!("cannot negate {}", other.type_name())),
}
}
pub(super) fn binary(op: BinOp, left: &Value, right: &Value) -> Value {
if let Value::Error(_) = left {
return left.clone();
}
if let Value::Error(_) = right {
return right.clone();
}
match op {
BinOp::Eq => Value::Bool(equals(left, right)),
BinOp::Ne => Value::Bool(!equals(left, right)),
BinOp::Lt | BinOp::Le | BinOp::Gt | BinOp::Ge => Value::Bool(ordered(op, left, right)),
BinOp::And => Value::Bool(truthy(left) && truthy(right)),
BinOp::Or => Value::Bool(truthy(left) || truthy(right)),
BinOp::Add | BinOp::Sub | BinOp::Mul | BinOp::Div | BinOp::Mod => arith(op, left, right),
}
}
/// A date and a date string compare as dates.
fn as_dates(left: &Value, right: &Value) -> Option<(Date, Date)> {
match (left, right) {
(Value::Date(a), Value::Str(s)) => Some((*a, Date::parse(s).ok()?)),
(Value::Str(s), Value::Date(b)) => Some((Date::parse(s).ok()?, *b)),
_ => None,
}
}
fn equals(left: &Value, right: &Value) -> bool {
if let Some((a, b)) = as_dates(left, right) {
return a.ms() == b.ms();
}
matches!((left, right), (Value::Null, Value::Null)) || loose_eq(left, right)
}
fn ordered(op: BinOp, left: &Value, right: &Value) -> bool {
let order = match (left, right, as_dates(left, right)) {
(Value::Null, _, _) | (_, Value::Null, _) => return false,
(Value::Number(a), Value::Number(b), _) if a.is_nan() || b.is_nan() => return false,
(_, _, Some((a, b))) => a.ms().cmp(&b.ms()),
_ => compare(left, right),
};
match op {
BinOp::Lt => order == Ordering::Less,
BinOp::Le => order != Ordering::Greater,
BinOp::Gt => order == Ordering::Greater,
_ => order != Ordering::Less,
}
}
fn symbol(op: BinOp) -> &'static str {
match op {
BinOp::Add => "+",
BinOp::Sub => "-",
BinOp::Mul => "*",
BinOp::Div => "/",
BinOp::Mod => "%",
_ => "?",
}
}
fn date_result(result: Result<Date, crate::DateError>) -> Value {
result.map_or_else(|e| err(e.to_string()), Value::Date)
}
fn duration_result(result: Result<Duration, crate::DateError>) -> Value {
result.map_or_else(|e| err(e.to_string()), Value::Duration)
}
/// The duration of a duration value or a duration string.
fn as_duration(value: &Value) -> Option<Result<Duration, crate::DateError>> {
match value {
Value::Duration(d) => Some(Ok(*d)),
Value::Str(s) => Some(parse_duration(s)),
_ => None,
}
}
fn concat(left: &Value, right: &Value) -> Value {
let text = [left.to_display_string(), right.to_display_string()].concat();
if text.len() > MAX_STR { err("string too long") } else { Value::Str(text) }
}
fn arith(op: BinOp, left: &Value, right: &Value) -> Value {
if matches!(left, Value::Null) || matches!(right, Value::Null) {
return Value::Null;
}
match (op, left, right) {
(BinOp::Add, Value::Number(a), Value::Number(b)) => Value::Number(a + b),
(BinOp::Sub, Value::Number(a), Value::Number(b)) => Value::Number(a - b),
(BinOp::Mul, Value::Number(a), Value::Number(b)) => Value::Number(a * b),
(BinOp::Div | BinOp::Mod, Value::Number(_), Value::Number(b)) if *b == 0.0 => err(if op == BinOp::Div { "division by zero" } else { "modulo by zero" }),
(BinOp::Div, Value::Number(a), Value::Number(b)) => Value::Number(a / b),
(BinOp::Mod, Value::Number(a), Value::Number(b)) => Value::Number(a % b),
(BinOp::Add, Value::Str(_), _) | (BinOp::Add, _, Value::Str(_)) if !matches!((left, right), (Value::Date(_), Value::Str(_))) => concat(left, right),
(BinOp::Add, Value::Date(d), other) | (BinOp::Add, other, Value::Date(d)) => match as_duration(other) {
Some(Ok(dur)) => date_result(add_duration(*d, dur)),
Some(Err(e)) => err(e.to_string()),
None => mismatch(op, left, right),
},
(BinOp::Sub, Value::Date(a), Value::Date(b)) => sub_dates(*a, *b).map_or_else(|e| err(e.to_string()), |ms| Value::Number(int_to_f64(ms))),
(BinOp::Sub, Value::Date(d), other) => match as_duration(other) {
Some(Ok(dur)) => date_result(dur.scaled(-1).and_then(|neg| add_duration(*d, neg))),
Some(Err(_)) | None => match (other, Date::parse(&other.to_display_string())) {
(Value::Str(_), Ok(b)) => sub_dates(*d, b).map_or_else(|e| err(e.to_string()), |ms| Value::Number(int_to_f64(ms))),
_ => mismatch(op, left, right),
},
},
(BinOp::Add, Value::Duration(a), Value::Duration(b)) => duration_result(a.plus(*b)),
(BinOp::Sub, Value::Duration(a), Value::Duration(b)) => duration_result(b.scaled(-1).and_then(|neg| a.plus(neg))),
(BinOp::Mul, Value::Duration(d), factor @ Value::Number(_)) => match to_i64(factor) {
Some(n) => duration_result(d.scaled(n)),
None => err("a duration can only be multiplied by a whole number"),
},
_ => mismatch(op, left, right),
}
}
fn mismatch(op: BinOp, left: &Value, right: &Value) -> Value {
err(format!("cannot apply `{}` to {} and {}", symbol(op), left.type_name(), right.type_name()))
}
+71
View File
@@ -0,0 +1,71 @@
//! Regular expressions: `/re/flags` values with `matches`, and `replace` / `split` on strings.
//!
//! The engine is `notescraft-regex`, a memoised backtracker with a work bound, so a pathological
//! pattern ends in an error value instead of a hang. The flag `i` ignores case and `g` makes
//! `replace` act on every match; other flags are ignored.
use notescraft_regex::{Match, Regex};
use super::ops::{MAX_STR, err, to_usize};
use crate::value::Value;
/// Most matches one call looks at.
const MAX_MATCHES: usize = 10_000;
fn compile(pattern: &str, flags: &str) -> Result<Regex, Value> {
Regex::new(pattern, flags.contains('i')).map_err(|e| err(e.to_string()))
}
fn find(re: &Regex, text: &str, limit: usize) -> Result<Vec<Match>, Value> {
re.find_all(text, limit).map_err(|e| err(e.to_string()))
}
pub(super) fn method(recv: &Value, name: &str, args: &[Value]) -> Option<Value> {
Some(match (recv, name, args) {
(Value::Regex { .. }, "matches", [Value::Null]) => Value::Bool(false),
(Value::Regex { pattern, flags }, "matches", [Value::Str(text)]) => {
match compile(pattern, flags).and_then(|re| re.is_match(text).map_err(|e| err(e.to_string()))) {
Ok(found) => Value::Bool(found),
Err(e) => e,
}
}
(Value::Regex { .. }, "matches", [other]) => err(format!("matches() does not accept {}", other.type_name())),
(Value::Str(text), "replace", [Value::Regex { pattern, flags }, Value::Str(with)]) => replace(text, pattern, flags, with).unwrap_or_else(|e| e),
(Value::Str(text), "split", [Value::Regex { pattern, flags }, rest @ ..]) => split(text, pattern, flags, rest.first()).unwrap_or_else(|e| e),
_ => return None,
})
}
fn replace(text: &str, pattern: &str, flags: &str, with: &str) -> Result<Value, Value> {
let re = compile(pattern, flags)?;
let limit = if flags.contains('g') { MAX_MATCHES } else { 1 };
let mut out = String::new();
let mut last = 0;
for found in find(&re, text, limit)? {
out.push_str(text.get(last..found.range.start).unwrap_or_default());
out.push_str(&re.expand(&found, text, with));
last = found.range.end;
if out.len() > MAX_STR {
return Err(err("string too long"));
}
}
out.push_str(text.get(last..).unwrap_or_default());
Ok(Value::Str(out))
}
fn split(text: &str, pattern: &str, flags: &str, count: Option<&Value>) -> Result<Value, Value> {
let keep = match count {
None => usize::MAX,
Some(n) => to_usize(n).ok_or_else(|| err("split() needs a whole count"))?,
};
let re = compile(pattern, flags)?;
let mut parts = Vec::new();
let mut last = 0;
for found in find(&re, text, MAX_MATCHES)? {
parts.push(Value::Str(text.get(last..found.range.start).unwrap_or_default().to_string()));
last = found.range.end;
}
parts.push(Value::Str(text.get(last..).unwrap_or_default().to_string()));
parts.truncate(keep);
Ok(Value::List(parts))
}
+232
View File
@@ -0,0 +1,232 @@
//! The abstract syntax tree of the Bases expression language.
//!
//! The tree is plain data: the parser produces it, the evaluator walks it.
//! Nothing here evaluates, reads a clock or touches the disk.
/// A parsed expression.
#[derive(Debug, Clone, PartialEq)]
pub enum Expr {
/// The `null` literal.
Null,
/// A boolean literal, `true` or `false`.
Bool(bool),
/// A number literal.
Num(f64),
/// A string literal.
Str(String),
/// A regular expression literal, written `/pattern/flags`.
Regex {
/// The pattern text, without the slashes.
pattern: String,
/// The flags, without the slashes.
flags: String,
},
/// An array literal, `[a, b]`.
Array(Vec<Expr>),
/// An object literal, `{ "k": v }`. The pairs keep their order.
Object(Vec<(String, Expr)>),
/// A bare name, e.g. `price`.
Ident(String),
/// A member access, `base.name`.
Member(Box<Expr>, String),
/// An index access, `base[index]`.
Index(Box<Expr>, Box<Expr>),
/// A call, `f(a, b)`. A method call has a [`Expr::Member`] callee.
Call {
/// The called expression.
callee: Box<Expr>,
/// The arguments, in order.
args: Vec<Expr>,
},
/// A prefix operator with its operand.
Unary(UnOp, Box<Expr>),
/// A binary operator with its two operands.
Binary(BinOp, Box<Expr>, Box<Expr>),
}
/// The prefix operators.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum UnOp {
/// `!`, logical not.
Not,
/// `-`, negation.
Neg,
}
/// The binary operators, from the loosest to the tightest binding.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum BinOp {
/// `+`, plus.
Add,
/// `-`, minus.
Sub,
/// `*`, multiply.
Mul,
/// `/`, divide.
Div,
/// `%`, modulo.
Mod,
/// `==`, equals.
Eq,
/// `!=`, not equal.
Ne,
/// `<`, less than.
Lt,
/// `<=`, less than or equal.
Le,
/// `>`, greater than.
Gt,
/// `>=`, greater than or equal.
Ge,
/// `&&`, logical and.
And,
/// `||`, logical or.
Or,
}
/// The names an expression reads.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct Deps {
/// Note properties, e.g. `title` or `meta.author`.
pub note: Vec<String>,
/// File properties, e.g. `name` or `folder`.
pub file: Vec<String>,
/// Formula names, e.g. `price`.
pub formula: Vec<String>,
/// Whether the expression reads `this`.
pub uses_this: bool,
}
impl Expr {
/// The names this expression reads.
///
/// A name chain rooted at `note`, `file` or `formula` lands in the matching
/// bucket with the member path as the name (`note.meta.author` gives
/// `meta.author`). A chain rooted at `this` sets [`Deps::uses_this`]. Any
/// other name counts as a note property: a bare `x` gives `x`, a chain
/// `x.y` gives `y`. Every bucket is sorted and free of duplicates.
#[must_use]
pub fn deps(&self) -> Deps {
let mut deps = Deps::default();
self.collect(&mut deps);
deps.note.sort();
deps.note.dedup();
deps.file.sort();
deps.file.dedup();
deps.formula.sort();
deps.formula.dedup();
deps
}
/// Gathers the dependencies of this node and of its children.
fn collect(&self, deps: &mut Deps) {
match self {
Expr::Ident(name) => {
let found = Chain { root: name.clone(), path: Vec::new(), holes: Vec::new() };
record(deps, &found);
}
Expr::Member(base, name) => match chain(base) {
Some(mut found) => {
found.path.push(name.clone());
record(deps, &found);
for hole in found.holes {
hole.collect(deps);
}
}
None => base.collect(deps),
},
Expr::Index(base, index) => {
match chain(base) {
Some(found) => {
record(deps, &found);
for hole in found.holes {
hole.collect(deps);
}
}
None => base.collect(deps),
}
index.collect(deps);
}
Expr::Call { callee, args } => {
callee.collect(deps);
for arg in args {
arg.collect(deps);
}
}
Expr::Array(items) => {
for item in items {
item.collect(deps);
}
}
Expr::Object(pairs) => {
for (_, value) in pairs {
value.collect(deps);
}
}
Expr::Unary(_, inner) => inner.collect(deps),
Expr::Binary(_, left, right) => {
left.collect(deps);
right.collect(deps);
}
Expr::Null | Expr::Bool(_) | Expr::Num(_) | Expr::Str(_) | Expr::Regex { .. } => {}
}
}
}
/// A name chain: the root identifier, the member names above it and the index
/// expressions hanging on it.
struct Chain<'a> {
/// The identifier the chain starts with.
root: String,
/// The member names, from the root upwards.
path: Vec<String>,
/// The index expressions met on the way.
holes: Vec<&'a Expr>,
}
/// The chain of an expression, if it is built from a root identifier.
fn chain(expr: &Expr) -> Option<Chain<'_>> {
match expr {
Expr::Ident(name) => Some(Chain { root: name.clone(), path: Vec::new(), holes: Vec::new() }),
Expr::Member(base, name) => {
let mut found = chain(base)?;
found.path.push(name.clone());
Some(found)
}
Expr::Index(base, index) => {
let mut found = chain(base)?;
found.holes.push(index.as_ref());
Some(found)
}
_ => None,
}
}
/// Puts one chain in the bucket of its root.
fn record(deps: &mut Deps, found: &Chain<'_>) {
let path = joined(&found.path);
match found.root.as_str() {
"note" => {
if let Some(name) = path {
deps.note.push(name);
}
}
"file" => {
if let Some(name) = path {
deps.file.push(name);
}
}
"formula" => {
if let Some(name) = path {
deps.formula.push(name);
}
}
"this" => deps.uses_this = true,
_ => deps.note.push(path.unwrap_or_else(|| found.root.clone())),
}
}
/// Joins a member path with dots.
fn joined(path: &[String]) -> Option<String> {
if path.is_empty() { None } else { Some(path.join(".")) }
}
+371
View File
@@ -0,0 +1,371 @@
//! The tokenizer of the Bases expression language.
//!
//! The lexer turns source text into tokens with their position. A regular
//! expression literal (`/re/flags`) is only a literal where an operand is
//! expected: after an operator, an opener or a comma. Everywhere else `/` is
//! the division operator, so `a / b / c` is two divisions.
use super::token::{Spanned, Token};
use crate::error::{Error, Result};
/// The flags a regular expression literal may carry.
const REGEX_FLAGS: [char; 7] = ['i', 'm', 's', 'x', 'U', 'g', 'u'];
/// The scanner state.
struct Lexer {
/// The source as `(byte offset, character)` pairs.
chars: Vec<(usize, char)>,
/// The next index into `chars`.
idx: usize,
/// The current one-based line.
line: usize,
/// The current one-based column.
column: usize,
/// Whether an operand is expected at the cursor.
expects_operand: bool,
}
/// Splits the source into tokens.
pub(crate) fn lex(src: &str) -> Result<Vec<Spanned>> {
let mut lexer = Lexer { chars: src.char_indices().collect(), idx: 0, line: 1, column: 1, expects_operand: true };
lexer.run()
}
impl Lexer {
/// Scans every token, ending with [`Token::Eof`].
fn run(&mut self) -> Result<Vec<Spanned>> {
let mut tokens = Vec::new();
loop {
self.skip_space();
let (line, column) = self.pos();
let Some((_, ch)) = self.peek() else {
tokens.push(Spanned { token: Token::Eof, line, column });
return Ok(tokens);
};
let token = match ch {
'(' => self.single(Token::LParen),
')' => self.single(Token::RParen),
'[' => self.single(Token::LBracket),
']' => self.single(Token::RBracket),
'{' => self.single(Token::LBrace),
'}' => self.single(Token::RBrace),
',' => self.single(Token::Comma),
'.' => self.single(Token::Dot),
':' => self.single(Token::Colon),
'+' => self.single(Token::Plus),
'-' => self.single(Token::Minus),
'*' => self.single(Token::Star),
'%' => self.single(Token::Percent),
'=' => {
self.bump();
if self.peek().is_some_and(|(_, next)| next == '=') {
self.bump();
Token::EqEq
} else {
return Err(self.error("`=` is not an operator, write `==`"));
}
}
'!' => {
self.bump();
if self.peek().is_some_and(|(_, next)| next == '=') {
self.bump();
Token::BangEq
} else {
Token::Bang
}
}
'<' => {
self.bump();
if self.peek().is_some_and(|(_, next)| next == '=') {
self.bump();
Token::LtEq
} else {
Token::Lt
}
}
'>' => {
self.bump();
if self.peek().is_some_and(|(_, next)| next == '=') {
self.bump();
Token::GtEq
} else {
Token::Gt
}
}
'&' => {
self.bump();
if self.peek().is_some_and(|(_, next)| next == '&') {
self.bump();
Token::AndAnd
} else {
return Err(self.error("`&` is not an operator, write `&&`"));
}
}
'|' => {
self.bump();
if self.peek().is_some_and(|(_, next)| next == '|') {
self.bump();
Token::OrOr
} else {
return Err(self.error("`|` is not an operator, write `||`"));
}
}
'/' => {
if self.expects_operand {
self.regex()?
} else {
self.single(Token::Slash)
}
}
'"' | '\'' => self.string(ch)?,
ch if ch.is_ascii_digit() => self.number()?,
ch if is_ident_start(ch) => self.ident(),
_ => return Err(self.error(&format!("unexpected character `{ch}`"))),
};
self.expects_operand = expects_operand_after(&token);
tokens.push(Spanned { token, line, column });
}
}
/// Reads a one character token.
fn single(&mut self, token: Token) -> Token {
self.bump();
token
}
/// Reads a number literal.
fn number(&mut self) -> Result<Token> {
let mut text = String::new();
while let Some((_, ch)) = self.peek() {
if ch.is_ascii_digit() {
text.push(ch);
self.bump();
} else {
break;
}
}
if self.peek().is_some_and(|(_, ch)| ch == '.') && self.next_is_digit() {
text.push('.');
self.bump();
while let Some((_, ch)) = self.peek() {
if ch.is_ascii_digit() {
text.push(ch);
self.bump();
} else {
break;
}
}
}
if self.peek().is_some_and(|(_, ch)| ch == 'e' || ch == 'E') {
text.push('e');
self.bump();
if self.peek().is_some_and(|(_, ch)| ch == '+' || ch == '-') {
if let Some((_, sign)) = self.peek() {
text.push(sign);
}
self.bump();
}
let mut digits = 0;
while let Some((_, ch)) = self.peek() {
if ch.is_ascii_digit() {
text.push(ch);
digits += 1;
self.bump();
} else {
break;
}
}
if digits == 0 {
return Err(self.error("a number needs digits after the exponent"));
}
}
let value = text.parse::<f64>().map_err(|_| self.error(&format!("`{text}` is not a number")))?;
Ok(Token::Num(value))
}
/// Reads a string literal, with the escapes `\\ \" \' \n \t`.
fn string(&mut self, quote: char) -> Result<Token> {
let (line, column) = self.pos();
self.bump();
let mut text = String::new();
loop {
let Some((_, ch)) = self.peek() else {
return Err(Error::Syntax { line, column, message: "the string is not closed".to_string() });
};
match ch {
'\\' => {
self.bump();
let Some((_, escaped)) = self.peek() else {
return Err(Error::Syntax { line, column, message: "the string is not closed".to_string() });
};
let plain = match escaped {
'\\' => '\\',
'"' => '"',
'\'' => '\'',
'n' => '\n',
't' => '\t',
_ => {
return Err(Error::Syntax { line, column, message: format!("unknown escape `\\{escaped}` in a string") });
}
};
text.push(plain);
self.bump();
}
_ if ch == quote => {
self.bump();
return Ok(Token::Str(text));
}
_ => {
text.push(ch);
self.bump();
}
}
}
}
/// Reads a regular expression literal; the cursor is on the opening slash.
fn regex(&mut self) -> Result<Token> {
let (line, column) = self.pos();
self.bump();
let mut pattern = String::new();
loop {
let Some((_, ch)) = self.peek() else {
return Err(Error::Syntax { line, column, message: "the regular expression is not closed".to_string() });
};
match ch {
'\\' => {
self.bump();
let Some((_, escaped)) = self.peek() else {
return Err(Error::Syntax { line, column, message: "the regular expression is not closed".to_string() });
};
if escaped == '/' {
pattern.push('/');
} else {
pattern.push('\\');
pattern.push(escaped);
}
self.bump();
}
'/' => {
self.bump();
break;
}
'\n' => {
return Err(Error::Syntax { line, column, message: "the regular expression is not closed".to_string() });
}
_ => {
pattern.push(ch);
self.bump();
}
}
}
let mut flags = String::new();
while let Some((_, ch)) = self.peek() {
if REGEX_FLAGS.contains(&ch) {
flags.push(ch);
self.bump();
} else {
break;
}
}
Ok(Token::Regex { pattern, flags })
}
/// Reads a name, or the `true`, `false` and `null` literals.
fn ident(&mut self) -> Token {
let mut name = String::new();
while let Some((_, ch)) = self.peek() {
if is_ident_continue(ch) {
name.push(ch);
self.bump();
} else {
break;
}
}
match name.as_str() {
"true" => Token::Bool(true),
"false" => Token::Bool(false),
"null" => Token::Null,
_ => Token::Ident(name),
}
}
/// Skips spaces, tabs and line breaks.
fn skip_space(&mut self) {
while let Some((_, ch)) = self.peek() {
if ch == '\n' {
self.bump();
self.line += 1;
self.column = 1;
} else if ch.is_whitespace() {
self.bump();
} else {
break;
}
}
}
/// The character after the cursor, if any.
fn next_is_digit(&self) -> bool {
self.chars.get(self.idx + 1).is_some_and(|(_, ch)| ch.is_ascii_digit())
}
/// The current position.
fn pos(&self) -> (usize, usize) {
(self.line, self.column)
}
/// The character at the cursor, without consuming it.
fn peek(&self) -> Option<(usize, char)> {
self.chars.get(self.idx).copied()
}
/// Consumes the character at the cursor.
fn bump(&mut self) -> Option<(usize, char)> {
let got = self.peek();
self.idx = self.idx.saturating_add(1);
self.column += 1;
got
}
/// An error at the cursor.
fn error(&self, message: &str) -> Error {
Error::Syntax { line: self.line, column: self.column, message: message.to_string() }
}
}
/// Whether a name may start with this character.
fn is_ident_start(ch: char) -> bool {
ch.is_alphabetic() || ch == '_' || ch == '$'
}
/// Whether a name may continue with this character.
fn is_ident_continue(ch: char) -> bool {
ch.is_alphanumeric() || ch == '_' || ch == '$'
}
/// Whether an operand may follow this token.
fn expects_operand_after(token: &Token) -> bool {
matches!(
token,
Token::LParen
| Token::LBracket
| Token::LBrace
| Token::Comma
| Token::Colon
| Token::Plus
| Token::Minus
| Token::Star
| Token::Slash
| Token::Percent
| Token::EqEq
| Token::BangEq
| Token::Lt
| Token::LtEq
| Token::Gt
| Token::GtEq
| Token::AndAnd
| Token::OrOr
| Token::Bang
)
}
+32
View File
@@ -0,0 +1,32 @@
//! The expression language of Bases: the tokens, the parser and the tree.
//!
//! The language is the one of the Obsidian Bases formulas: literals (numbers,
//! strings, regular expressions, booleans, `null`), lists, objects, member and
//! index access, calls and the operators of the spec. The precedence, from the
//! loosest to the tightest, is `||`, `&&`, `==` `!=`, `<` `<=` `>` `>=`,
//! `+` `-`, `*` `/` `%`, the prefix `!` `-`, then the member, index and call
//! suffixes.
mod ast;
mod lex;
mod parse;
mod token;
pub use ast::{BinOp, Deps, Expr, UnOp};
use crate::error::Result;
/// Parses one expression into its tree.
///
/// The source is limited to 4 KiB and to 64 levels of nesting; a longer or a
/// deeper source is refused with [`crate::Error::Limit`]. A malformed source
/// is refused with [`crate::Error::Syntax`], which carries the line and the
/// column of the offending token.
///
/// # Errors
///
/// Returns [`crate::Error::Limit`] when the source is too long or too deep,
/// and [`crate::Error::Syntax`] when it is not a valid expression.
pub fn parse_expr(src: &str) -> Result<Expr> {
parse::parse(src)
}
+347
View File
@@ -0,0 +1,347 @@
//! The parser of the Bases expression language.
//!
//! The parser is a precedence climber over the token stream. Recursion is
//! bounded by the nesting limit, so a hostile source can never blow the stack:
//! every group, list, object, index and call counts as one level.
use super::ast::{BinOp, Expr, UnOp};
use super::lex::lex;
use super::token::{Spanned, Token};
use crate::error::{Error, Result};
/// The longest expression accepted, in bytes.
pub(crate) const MAX_SOURCE_BYTES: usize = 4096;
/// The deepest nesting accepted.
pub(crate) const MAX_DEPTH: usize = 64;
/// Parses one expression.
pub(crate) fn parse(src: &str) -> Result<Expr> {
if src.len() > MAX_SOURCE_BYTES {
return Err(Error::Limit(format!("the expression is longer than {MAX_SOURCE_BYTES} bytes")));
}
let tokens = lex(src)?;
let mut parser = Parser { tokens, idx: 0, depth: 0 };
let expr = parser.expression()?;
parser.expect_end()?;
Ok(expr)
}
/// The precedence of a binary operator; the higher the number, the tighter it binds.
fn binary_op(token: &Token) -> Option<(BinOp, u8)> {
match token {
Token::OrOr => Some((BinOp::Or, 1)),
Token::AndAnd => Some((BinOp::And, 2)),
Token::EqEq => Some((BinOp::Eq, 3)),
Token::BangEq => Some((BinOp::Ne, 3)),
Token::Lt => Some((BinOp::Lt, 4)),
Token::LtEq => Some((BinOp::Le, 4)),
Token::Gt => Some((BinOp::Gt, 4)),
Token::GtEq => Some((BinOp::Ge, 4)),
Token::Plus => Some((BinOp::Add, 5)),
Token::Minus => Some((BinOp::Sub, 5)),
Token::Star => Some((BinOp::Mul, 6)),
Token::Slash => Some((BinOp::Div, 6)),
Token::Percent => Some((BinOp::Mod, 6)),
_ => None,
}
}
/// The parser state.
struct Parser {
/// The tokens of the source, the last one being the end.
tokens: Vec<Spanned>,
/// The index of the token at the cursor.
idx: usize,
/// The current nesting depth.
depth: usize,
}
impl Parser {
/// Parses a whole expression.
fn expression(&mut self) -> Result<Expr> {
self.binary(1)
}
/// Parses the binary operators that bind at least as tight as `min_precedence`.
fn binary(&mut self, min_precedence: u8) -> Result<Expr> {
let mut left = self.unary()?;
while let Some(token) = self.peek().cloned() {
let Some((op, precedence)) = binary_op(&token) else { break };
if precedence < min_precedence {
break;
}
self.bump();
let right = self.binary(precedence + 1)?;
left = Expr::Binary(op, Box::new(left), Box::new(right));
}
Ok(left)
}
/// Parses the prefix operators and the operand they apply to.
fn unary(&mut self) -> Result<Expr> {
let mut operators: Vec<UnOp> = Vec::new();
while let Some(token) = self.peek().cloned() {
match token {
Token::Bang => {
self.bump();
operators.push(UnOp::Not);
}
Token::Minus => {
self.bump();
operators.push(UnOp::Neg);
}
_ => break,
}
}
let mut expr = self.postfix()?;
while let Some(op) = operators.pop() {
expr = Expr::Unary(op, Box::new(expr));
}
Ok(expr)
}
/// Parses the member, index and call suffixes.
fn postfix(&mut self) -> Result<Expr> {
let mut expr = self.primary()?;
loop {
match self.peek().cloned() {
Some(Token::Dot) => {
self.bump();
let Some(Token::Ident(name)) = self.peek().cloned() else {
return Err(self.error("a property name is expected after `.`"));
};
self.bump();
expr = Expr::Member(Box::new(expr), name);
}
Some(Token::LBracket) => {
self.bump();
self.enter()?;
let index = self.expression()?;
self.expect(&Token::RBracket, "`]` is expected to close the index")?;
self.leave();
expr = Expr::Index(Box::new(expr), Box::new(index));
}
Some(Token::LParen) => {
self.bump();
let args = self.arguments()?;
expr = Expr::Call { callee: Box::new(expr), args };
}
_ => break,
}
}
Ok(expr)
}
/// Parses a literal, a group, a list or an object.
fn primary(&mut self) -> Result<Expr> {
let Some(token) = self.peek().cloned() else {
return Err(self.error("the expression ends too early"));
};
match token {
Token::Num(value) => {
self.bump();
Ok(Expr::Num(value))
}
Token::Str(text) => {
self.bump();
Ok(Expr::Str(text))
}
Token::Regex { pattern, flags } => {
self.bump();
Ok(Expr::Regex { pattern, flags })
}
Token::Bool(value) => {
self.bump();
Ok(Expr::Bool(value))
}
Token::Null => {
self.bump();
Ok(Expr::Null)
}
Token::Ident(name) => {
self.bump();
Ok(Expr::Ident(name))
}
Token::LParen => self.group(),
Token::LBracket => self.list(),
Token::LBrace => self.object(),
other => Err(self.error(&format!("unexpected {}", describe(&other)))),
}
}
/// Parses a parenthesised expression.
fn group(&mut self) -> Result<Expr> {
self.bump();
self.enter()?;
let inner = self.expression()?;
self.expect(&Token::RParen, "`)` is expected to close the group")?;
self.leave();
Ok(inner)
}
/// Parses a list literal.
fn list(&mut self) -> Result<Expr> {
self.bump();
self.enter()?;
let mut items = Vec::new();
if !matches!(self.peek(), Some(Token::RBracket)) {
loop {
items.push(self.expression()?);
if !self.comma() {
break;
}
}
}
self.expect(&Token::RBracket, "`]` is expected to close the list")?;
self.leave();
Ok(Expr::Array(items))
}
/// Parses an object literal.
fn object(&mut self) -> Result<Expr> {
self.bump();
self.enter()?;
let mut pairs = Vec::new();
if !matches!(self.peek(), Some(Token::RBrace)) {
loop {
let key = self.object_key()?;
self.expect(&Token::Colon, "`:` is expected after the key")?;
let value = self.expression()?;
pairs.push((key, value));
if !self.comma() {
break;
}
}
}
self.expect(&Token::RBrace, "`}` is expected to close the object")?;
self.leave();
Ok(Expr::Object(pairs))
}
/// Parses the arguments of a call.
fn arguments(&mut self) -> Result<Vec<Expr>> {
self.enter()?;
let mut args = Vec::new();
if !matches!(self.peek(), Some(Token::RParen)) {
loop {
args.push(self.expression()?);
if !self.comma() {
break;
}
}
}
self.expect(&Token::RParen, "`)` is expected to close the call")?;
self.leave();
Ok(args)
}
/// Parses the key of an object pair.
fn object_key(&mut self) -> Result<String> {
match self.peek().cloned() {
Some(Token::Str(text)) => {
self.bump();
Ok(text)
}
Some(Token::Ident(name)) => {
self.bump();
Ok(name)
}
_ => Err(self.error("a key is expected in the object")),
}
}
/// Eats a comma; answers whether there is another item after it.
fn comma(&mut self) -> bool {
if !matches!(self.peek(), Some(Token::Comma)) {
return false;
}
self.bump();
!matches!(self.peek(), Some(Token::RBracket | Token::RBrace | Token::RParen))
}
/// Checks that the source is over.
fn expect_end(&self) -> Result<()> {
if matches!(self.peek(), Some(Token::Eof)) { Ok(()) } else { Err(self.error("the expression has extra text after it")) }
}
/// Eats the expected token.
fn expect(&mut self, token: &Token, message: &str) -> Result<()> {
if self.peek() == Some(token) {
self.bump();
Ok(())
} else {
Err(self.error(message))
}
}
/// Counts one nesting level.
fn enter(&mut self) -> Result<()> {
self.depth += 1;
if self.depth > MAX_DEPTH {
return Err(Error::Limit(format!("the expression nests deeper than {MAX_DEPTH} levels")));
}
Ok(())
}
/// Leaves one nesting level.
fn leave(&mut self) {
self.depth = self.depth.saturating_sub(1);
}
/// The token at the cursor.
fn peek(&self) -> Option<&Token> {
self.tokens.get(self.idx).map(|spanned| &spanned.token)
}
/// Eats the token at the cursor.
fn bump(&mut self) -> Option<Token> {
let token = self.peek().cloned();
if token.is_some() {
self.idx += 1;
}
token
}
/// An error at the cursor.
fn error(&self, message: &str) -> Error {
let (line, column) = self.tokens.get(self.idx).map_or((1, 1), |spanned| (spanned.line, spanned.column));
Error::Syntax { line, column, message: message.to_string() }
}
}
/// A name for the token, for the error messages.
fn describe(token: &Token) -> String {
match token {
Token::Num(_) => "a number".to_string(),
Token::Str(_) => "a string".to_string(),
Token::Ident(name) => format!("the name `{name}`"),
Token::Regex { .. } => "a regular expression".to_string(),
Token::Bool(_) => "a boolean".to_string(),
Token::Null => "`null`".to_string(),
Token::LParen => "`(`".to_string(),
Token::RParen => "`)`".to_string(),
Token::LBracket => "`[`".to_string(),
Token::RBracket => "`]`".to_string(),
Token::LBrace => "`{`".to_string(),
Token::RBrace => "`}`".to_string(),
Token::Comma => "`,`".to_string(),
Token::Dot => "`.`".to_string(),
Token::Colon => "`:`".to_string(),
Token::Plus => "`+`".to_string(),
Token::Minus => "`-`".to_string(),
Token::Star => "`*`".to_string(),
Token::Slash => "`/`".to_string(),
Token::Percent => "`%`".to_string(),
Token::EqEq => "`==`".to_string(),
Token::BangEq => "`!=`".to_string(),
Token::Lt => "`<`".to_string(),
Token::LtEq => "`<=`".to_string(),
Token::Gt => "`>`".to_string(),
Token::GtEq => "`>=`".to_string(),
Token::AndAnd => "`&&`".to_string(),
Token::OrOr => "`||`".to_string(),
Token::Bang => "`!`".to_string(),
Token::Eof => "the end of the expression".to_string(),
}
}
+85
View File
@@ -0,0 +1,85 @@
//! The tokens of the Bases expression language.
//!
//! The tokens are the hand-off between the scanner and the parser: the scanner
//! produces them with their position, the parser reads them in order.
/// A token of the expression language.
#[derive(Debug, Clone, PartialEq)]
pub(crate) enum Token {
/// A number literal.
Num(f64),
/// A string literal.
Str(String),
/// A name.
Ident(String),
/// A regular expression literal.
Regex {
/// The pattern text, without the slashes.
pattern: String,
/// The flags, without the slashes.
flags: String,
},
/// The `true` or `false` literal.
Bool(bool),
/// The `null` literal.
Null,
/// `(`.
LParen,
/// `)`.
RParen,
/// `[`.
LBracket,
/// `]`.
RBracket,
/// `{`.
LBrace,
/// `}`.
RBrace,
/// `,`.
Comma,
/// `.`.
Dot,
/// `:`.
Colon,
/// `+`.
Plus,
/// `-`.
Minus,
/// `*`.
Star,
/// `/`.
Slash,
/// `%`.
Percent,
/// `==`.
EqEq,
/// `!=`.
BangEq,
/// `<`.
Lt,
/// `<=`.
LtEq,
/// `>`.
Gt,
/// `>=`.
GtEq,
/// `&&`.
AndAnd,
/// `||`.
OrOr,
/// `!`.
Bang,
/// The end of the source.
Eof,
}
/// A token with the position of its first character.
#[derive(Debug, Clone, PartialEq)]
pub(crate) struct Spanned {
/// The token itself.
pub(crate) token: Token,
/// One-based line.
pub(crate) line: usize,
/// One-based column.
pub(crate) column: usize,
}
+21 -1
View File
@@ -7,6 +7,26 @@
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
mod base;
mod compare;
mod datetime;
mod error;
mod eval;
pub mod expr;
pub mod map;
mod model;
pub mod query;
mod row;
pub mod schema;
mod value;
mod yaml;
pub use error::{Error, Result};
pub use compare::{compare, loose_eq, truthy};
pub use datetime::{Date, DateError, DateParts, Duration, add_duration, parse_duration, sub_dates};
pub use error::{Error, Result, SchemaErrorKind};
pub use eval::{Env, Formulas, eval};
pub use model::{Base, Direction, Filter, GroupBy, PropertyCfg, Sort, View, ViewKind};
pub use query::{Column, Group, QueryResult, Reads, RowId, ViewSel, run};
pub use row::{NoteSource, RowRef};
pub use value::Value;
pub use yaml::{Node, emit, parse};
+60
View File
@@ -0,0 +1,60 @@
//! Grid clustering of map markers.
//!
//! A base can hold thousands of notes with a position. Drawing one pin per
//! note hides the map, so the markers that sit close together are gathered
//! into one pin: the markers are projected to world pixels, put in the
//! squares of a grid of `radius_px` and every square that holds a marker
//! becomes a cluster whose centre is the middle of its markers.
use std::collections::HashMap;
use super::project::{LatLng, floor_div, project, unproject};
/// A point to show on the map.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct Marker {
/// The identifier of the row the marker comes from.
pub id: usize,
/// Where the marker sits.
pub at: LatLng,
}
/// A group of markers that sit close enough to share one pin.
#[derive(Debug, Clone, PartialEq)]
pub struct Cluster {
/// Where the pin goes.
pub center: LatLng,
/// The identifiers of the markers of the pin, in growing order.
pub ids: Vec<usize>,
}
/// The clusters of the markers at a zoom level.
///
/// The markers are gathered in the squares of a grid of `radius_px` world
/// pixels. The centre of a cluster is the middle of its markers. The clusters
/// come back in the order of their first identifier, and every marker
/// belongs to exactly one cluster.
#[must_use]
pub fn cluster(markers: &[Marker], zoom: u8, radius_px: f64) -> Vec<Cluster> {
let radius = radius_px.max(1.0);
let mut buckets: HashMap<(i64, i64), (f64, f64, Vec<usize>)> = HashMap::new();
for marker in markers {
let (x, y) = project(marker.at, zoom);
let key = (floor_div(x, radius), floor_div(y, radius));
let bucket = buckets.entry(key).or_insert((0.0, 0.0, Vec::new()));
bucket.0 += x;
bucket.1 += y;
bucket.2.push(marker.id);
}
let mut clusters: Vec<Cluster> = buckets
.into_values()
.map(|(sum_x, sum_y, mut ids)| {
let count = f64::from(u32::try_from(ids.len()).unwrap_or(u32::MAX).max(1));
let center = unproject(sum_x / count, sum_y / count, zoom);
ids.sort_unstable();
Cluster { center, ids }
})
.collect();
clusters.sort_by(|left, right| left.ids.first().cmp(&right.ids.first()));
clusters
}
+101
View File
@@ -0,0 +1,101 @@
//! The colours of the map view.
//!
//! A base writes the colour of a marker as CSS text: a hex triplet, an
//! `rgb()` call, one of the basic colour names or a theme token such as
//! `var(--color-accent)`.
/// A colour of the map.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Color {
/// A red, green and blue colour.
Rgb(u8, u8, u8),
/// The name of a theme token, without the `--` prefix.
Token(String),
}
/// The sixteen basic colour names of CSS, and their value.
const NAMED: [(&str, (u8, u8, u8)); 16] = [
("black", (0, 0, 0)),
("silver", (192, 192, 192)),
("gray", (128, 128, 128)),
("white", (255, 255, 255)),
("maroon", (128, 0, 0)),
("red", (255, 0, 0)),
("purple", (128, 0, 128)),
("fuchsia", (255, 0, 255)),
("green", (0, 128, 0)),
("lime", (0, 255, 0)),
("olive", (128, 128, 0)),
("yellow", (255, 255, 0)),
("navy", (0, 0, 128)),
("blue", (0, 0, 255)),
("teal", (0, 128, 128)),
("aqua", (0, 255, 255)),
];
/// The colour of a CSS colour text.
///
/// `#rgb`, `#rrggbb`, `rgb(r, g, b)`, one of the sixteen basic colour names
/// and `var(--token)` are understood. Anything else gives `None`.
#[must_use]
pub fn parse_color(s: &str) -> Option<Color> {
let text = s.trim();
if let Some(hex) = text.strip_prefix('#') {
return parse_hex(hex);
}
if let Some(inner) = strip_function(text, "rgb") {
return parse_rgb(inner);
}
if let Some(inner) = strip_function(text, "var") {
let name = inner.trim().strip_prefix("--")?.trim();
if name.is_empty() {
return None;
}
return Some(Color::Token(name.to_string()));
}
named(text)
}
/// The colour of a hex triplet, with three or six digits.
fn parse_hex(hex: &str) -> Option<Color> {
let digits: Vec<char> = hex.chars().collect();
match digits.len() {
3 => Some(Color::Rgb(hex_digit(*digits.first()?)? * 17, hex_digit(*digits.get(1)?)? * 17, hex_digit(*digits.get(2)?)? * 17)),
6 => Some(Color::Rgb(hex_byte(digits.get(0..2)?)?, hex_byte(digits.get(2..4)?)?, hex_byte(digits.get(4..6)?)?)),
_ => None,
}
}
/// The value of one hex digit.
fn hex_digit(character: char) -> Option<u8> {
u8::try_from(character.to_digit(16)?).ok()
}
/// The value of two hex digits.
fn hex_byte(digits: &[char]) -> Option<u8> {
let high = hex_digit(*digits.first()?)?;
let low = hex_digit(*digits.get(1)?)?;
Some(high * 16 + low)
}
/// The colour of an `rgb(r, g, b)` call.
fn parse_rgb(inner: &str) -> Option<Color> {
let parts: Vec<&str> = inner.split(',').map(str::trim).collect();
if parts.len() != 3 {
return None;
}
Some(Color::Rgb(parts.first()?.parse().ok()?, parts.get(1)?.parse().ok()?, parts.get(2)?.parse().ok()?))
}
/// The colour of one of the basic colour names.
fn named(text: &str) -> Option<Color> {
let lowered = text.to_ascii_lowercase();
let (_, rgb) = NAMED.iter().find(|(name, _)| *name == lowered)?;
Some(Color::Rgb(rgb.0, rgb.1, rgb.2))
}
/// The text inside a `name(...)` call.
fn strip_function<'a>(text: &'a str, name: &str) -> Option<&'a str> {
let inner = text.strip_prefix(name)?.strip_prefix('(')?;
inner.strip_suffix(')')
}
+73
View File
@@ -0,0 +1,73 @@
//! The data model of the map view of a base.
//!
//! A base can show its rows on a world map. This module holds the parts the
//! view needs before it draws anything: the position of a row, the tiles the
//! viewport touches, the clusters of the markers and the colour of a pin.
//! Everything here is pure arithmetic, without I/O.
mod cluster;
mod color;
mod project;
pub use cluster::{Cluster, Marker, cluster};
pub use color::{Color, parse_color};
pub use project::{
LatLng, MAX_FIT_ZOOM, MAX_LATITUDE, MAX_VIEWPORT_TILES, MAX_ZOOM, SINGLE_POINT_ZOOM, TILE_SIZE, TileId, fit, project, tile_of, tiles_for_viewport,
unproject, valid,
};
use crate::Value;
/// The credit of the map tiles.
pub struct Attribution;
impl Attribution {
/// The credit line the OpenStreetMap tiles need.
pub const OSM_ATTRIBUTION: &str = "© OpenStreetMap contributors";
/// The address of the OpenStreetMap tile server.
pub const OSM_TILE_URL: &str = "https://tile.openstreetmap.org/{z}/{x}/{y}.png";
}
/// The position a value holds, when it holds one.
///
/// The text `"48.85, 2.29"` and a list of two numbers, or of two numeric
/// strings, are read as a latitude and a longitude. Anything else, and any
/// position outside the valid range, gives `None`.
#[must_use]
pub fn parse_coordinates(v: &Value) -> Option<LatLng> {
let position = match v {
Value::Str(text) => parse_pair(text)?,
Value::List(items) => {
if items.len() != 2 {
return None;
}
LatLng { lat: number_of(items.first()?)?, lng: number_of(items.get(1)?)? }
}
_ => return None,
};
if valid(position) { Some(position) } else { None }
}
/// How many points of a list hold no position.
#[must_use]
pub fn skipped(points: &[Option<LatLng>]) -> usize {
points.iter().filter(|point| point.is_none()).count()
}
/// The position of a `"lat, lng"` text.
fn parse_pair(text: &str) -> Option<LatLng> {
let parts: Vec<&str> = text.split(',').map(str::trim).collect();
if parts.len() != 2 {
return None;
}
Some(LatLng { lat: parts.first()?.parse().ok()?, lng: parts.get(1)?.parse().ok()? })
}
/// The number a value holds, when it holds one.
fn number_of(value: &Value) -> Option<f64> {
match value {
Value::Number(number) => Some(*number),
Value::Str(text) => text.trim().parse().ok(),
_ => None,
}
}
+220
View File
@@ -0,0 +1,220 @@
//! Web Mercator projection on a slippy-map tile pyramid.
//!
//! The map view of a base puts notes on a world map: project a position to
//! world pixels, find the tile that covers it, list the tiles a viewport
//! touches and fit a set of points in a viewport. The projection is the Web
//! Mercator of every tile server: the world is a square of `256 * 2^zoom`
//! pixels and its north edge is the parallel of latitude `85.0511287798`.
use std::collections::HashSet;
/// The size of one tile, in pixels.
pub const TILE_SIZE: f64 = 256.0;
/// The highest latitude the projection can show, in degrees.
pub const MAX_LATITUDE: f64 = 85.051_128_779_8;
/// The highest zoom level the projection accepts.
pub const MAX_ZOOM: u8 = 22;
/// The highest zoom level [`fit`] may choose.
pub const MAX_FIT_ZOOM: u8 = 18;
/// The zoom level [`fit`] chooses when the set holds a single point.
pub const SINGLE_POINT_ZOOM: u8 = 15;
/// The most tiles [`tiles_for_viewport`] returns.
pub const MAX_VIEWPORT_TILES: usize = 256;
/// The most steps [`tiles_for_viewport`] walks before it stops.
const MAX_VIEWPORT_STEPS: usize = MAX_VIEWPORT_TILES * 8;
/// A geographic position.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct LatLng {
/// The latitude, in degrees, positive to the north.
pub lat: f64,
/// The longitude, in degrees, positive to the east.
pub lng: f64,
}
/// The identifier of one tile of the pyramid.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct TileId {
/// The zoom level.
pub z: u8,
/// The column, counted from the west.
pub x: u32,
/// The row, counted from the north.
pub y: u32,
}
/// Whether a position can be shown on the map.
///
/// A position is valid when both numbers are finite, the latitude stays
/// inside the Mercator limit and the longitude stays inside `[-180, 180]`.
#[must_use]
pub fn valid(p: LatLng) -> bool {
p.lat.is_finite() && p.lng.is_finite() && p.lat.abs() <= MAX_LATITUDE && p.lng.abs() <= 180.0
}
/// The world pixel of a position at a zoom level.
///
/// The zoom goes from `0` to [`MAX_ZOOM`]. The result is not clamped: a
/// position outside the valid range gives a pixel outside the world, and a
/// latitude beyond the pole gives `NaN`.
#[must_use]
pub fn project(p: LatLng, zoom: u8) -> (f64, f64) {
let world = world_size(zoom);
let lat = p.lat.to_radians();
let x = (p.lng + 180.0) / 360.0 * world;
let y = (1.0 - (lat.tan() + 1.0 / lat.cos()).ln() / std::f64::consts::PI) / 2.0 * world;
(x, y)
}
/// The position of a world pixel at a zoom level.
///
/// This is the inverse of [`project`].
#[must_use]
pub fn unproject(x: f64, y: f64, zoom: u8) -> LatLng {
let world = world_size(zoom);
let lng = x / world * 360.0 - 180.0;
let lat = (std::f64::consts::PI * (1.0 - 2.0 * y / world)).sinh().atan().to_degrees();
LatLng { lat, lng }
}
/// The tile that covers a position at a zoom level.
///
/// The column wraps around the world, the row is clamped to the pyramid.
#[must_use]
pub fn tile_of(p: LatLng, zoom: u8) -> TileId {
let (world_x, world_y) = project(p, zoom);
let grid = grid(zoom);
let x = wrap_index(floor_div(world_x, TILE_SIZE), grid);
let y = clamp_index(floor_div(world_y, TILE_SIZE), grid);
TileId { z: zoom, x: to_u32(x), y: to_u32(y) }
}
/// The tiles a viewport of `width_px` by `height_px` touches.
///
/// The viewport is centred on `center`. The columns wrap around the
/// antimeridian and the rows are clamped to the pyramid. The result holds at
/// most [`MAX_VIEWPORT_TILES`] tiles, in reading order, without a repeat.
#[must_use]
pub fn tiles_for_viewport(center: LatLng, zoom: u8, width_px: f64, height_px: f64) -> Vec<TileId> {
let grid = grid(zoom);
let (center_x, center_y) = project(center, zoom);
let half_width = width_px.max(0.0) / 2.0;
let half_height = height_px.max(0.0) / 2.0;
let first_x = floor_div(center_x - half_width, TILE_SIZE);
let last_x = floor_div(center_x + half_width, TILE_SIZE);
let first_y = clamp_index(floor_div(center_y - half_height, TILE_SIZE), grid);
let last_y = clamp_index(floor_div(center_y + half_height, TILE_SIZE), grid);
let columns = last_x.saturating_sub(first_x).saturating_add(1).min(grid);
let rows = last_y.saturating_sub(first_y).saturating_add(1);
let mut tiles: Vec<TileId> = Vec::new();
let mut seen: HashSet<(i64, i64)> = HashSet::new();
let mut steps = 0usize;
for column in 0..columns {
for row in 0..rows {
steps += 1;
if tiles.len() >= MAX_VIEWPORT_TILES || steps >= MAX_VIEWPORT_STEPS {
return tiles;
}
let x = wrap_index(first_x.saturating_add(column), grid);
let y = first_y.saturating_add(row);
if seen.insert((x, y)) {
tiles.push(TileId { z: zoom, x: to_u32(x), y: to_u32(y) });
}
}
}
tiles
}
/// The centre and the zoom that show every point in a viewport.
///
/// The zoom is the highest one at or below [`MAX_FIT_ZOOM`] where all points
/// fit in the viewport without `padding_px` on each side. A single point
/// gives [`SINGLE_POINT_ZOOM`]. An empty set, or a set without one valid
/// point, gives `None`.
#[must_use]
pub fn fit(points: &[LatLng], width_px: f64, height_px: f64, padding_px: f64) -> Option<(LatLng, u8)> {
let usable: Vec<LatLng> = points.iter().copied().filter(|point| valid(*point)).collect();
let first = usable.first().copied()?;
if usable.len() == 1 {
return Some((first, SINGLE_POINT_ZOOM));
}
let centre = bounds_centre(&usable);
let available_width = (width_px - 2.0 * padding_px).max(1.0);
let available_height = (height_px - 2.0 * padding_px).max(1.0);
for zoom in (0..=MAX_FIT_ZOOM).rev() {
if spans_fit(&usable, zoom, available_width, available_height) {
return Some((centre, zoom));
}
}
Some((centre, 0))
}
/// The size of the world at a zoom level, in pixels.
fn world_size(zoom: u8) -> f64 {
TILE_SIZE * 2.0f64.powi(i32::from(zoom))
}
/// The number of tiles along one axis at a zoom level.
fn grid(zoom: u8) -> i64 {
1i64.checked_shl(u32::from(zoom)).unwrap_or(i64::MAX)
}
/// The tile index of a world coordinate, rounded down.
pub(super) fn floor_div(value: f64, divisor: f64) -> i64 {
let quotient = (value / divisor).floor();
#[expect(clippy::cast_possible_truncation, reason = "the quotient is rounded down and the cast from float to int saturates")]
let index = quotient as i64;
index
}
/// A tile index wrapped around the world.
fn wrap_index(index: i64, grid: i64) -> i64 {
index.rem_euclid(grid.max(1))
}
/// A tile index clamped to the pyramid.
fn clamp_index(index: i64, grid: i64) -> i64 {
index.clamp(0, grid.saturating_sub(1))
}
/// A tile index as the column or the row of a tile.
fn to_u32(index: i64) -> u32 {
u32::try_from(index).unwrap_or(u32::MAX)
}
/// The middle of the bounding box of the points.
fn bounds_centre(points: &[LatLng]) -> LatLng {
let mut min_lat = f64::INFINITY;
let mut max_lat = f64::NEG_INFINITY;
let mut min_lng = f64::INFINITY;
let mut max_lng = f64::NEG_INFINITY;
for point in points {
min_lat = min_lat.min(point.lat);
max_lat = max_lat.max(point.lat);
min_lng = min_lng.min(point.lng);
max_lng = max_lng.max(point.lng);
}
LatLng { lat: f64::midpoint(min_lat, max_lat), lng: f64::midpoint(min_lng, max_lng) }
}
/// Whether the points fit in a box of the given size at a zoom level.
fn spans_fit(points: &[LatLng], zoom: u8, available_width: f64, available_height: f64) -> bool {
let mut min_x = f64::INFINITY;
let mut max_x = f64::NEG_INFINITY;
let mut min_y = f64::INFINITY;
let mut max_y = f64::NEG_INFINITY;
for point in points {
let (x, y) = project(*point, zoom);
min_x = min_x.min(x);
max_x = max_x.max(x);
min_y = min_y.min(y);
max_y = max_y.max(y);
}
max_x - min_x <= available_width && max_y - min_y <= available_height
}
+175
View File
@@ -0,0 +1,175 @@
//! The data model of a `.base` file: filters, formulas, properties, summaries and views.
//!
//! The model is plain data. Parsing and writing live in `base.rs`; nothing here evaluates an
//! expression.
use crate::yaml::Node;
/// Most views a base may hold.
pub const MAX_VIEWS: usize = 100;
/// Most formulas a base may hold.
pub const MAX_FORMULAS: usize = 200;
/// Longest expression, in bytes.
pub const MAX_EXPR_BYTES: usize = 4096;
/// A filter: one expression or a group of filters.
#[derive(Debug, Clone, PartialEq)]
pub enum Filter {
/// An expression that must be truthy.
Expr(String),
/// Every member must hold.
And(Vec<Filter>),
/// At least one member must hold.
Or(Vec<Filter>),
/// No member may hold.
Not(Vec<Filter>),
}
/// A sort direction.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Direction {
/// `ASC`.
Asc,
/// `DESC`.
Desc,
}
/// One sort key.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Sort {
/// The property id.
pub property: String,
/// The direction.
pub direction: Direction,
}
/// The `groupBy` of a view.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct GroupBy {
/// The property id.
pub property: String,
/// The direction of the groups.
pub direction: Direction,
}
/// The layout of a view.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ViewKind {
/// `table`.
Table,
/// `cards`.
Cards,
/// `list`.
List,
/// `map`.
Map,
/// Any other type, kept as written.
Other(String),
}
impl ViewKind {
/// The `type:` text of the kind.
#[must_use]
pub fn as_str(&self) -> &str {
match self {
ViewKind::Table => "table",
ViewKind::Cards => "cards",
ViewKind::List => "list",
ViewKind::Map => "map",
ViewKind::Other(text) => text,
}
}
}
/// One view of a base.
#[derive(Debug, Clone, PartialEq)]
pub struct View {
/// The layout.
pub kind: ViewKind,
/// The tab name.
pub name: Option<String>,
/// Most rows shown.
pub limit: Option<u32>,
/// View filters, AND-ed with the base filters.
pub filters: Option<Filter>,
/// Grouping.
pub group_by: Option<GroupBy>,
/// Manual group order; `Node::Null` is the "no value" group.
pub group_order: Vec<Node>,
/// Visible columns, in order.
pub order: Vec<String>,
/// Sort keys, first is the most significant.
pub sort: Vec<Sort>,
/// Summary name per column id.
pub summaries: Vec<(String, String)>,
/// Keys the model does not know (`columnSize`, `cardSize`, ...), in original order.
pub extra: Vec<(String, Node)>,
pub(crate) layout: Vec<String>,
}
impl View {
/// An empty view of the given kind.
#[must_use]
pub fn new(kind: ViewKind, name: Option<String>) -> Self {
Self {
kind,
name,
limit: None,
filters: None,
group_by: None,
group_order: Vec::new(),
order: Vec::new(),
sort: Vec::new(),
summaries: Vec::new(),
extra: Vec::new(),
layout: Vec::new(),
}
}
/// The value of an unmodelled key.
#[must_use]
pub fn extra_get(&self, key: &str) -> Option<&Node> {
self.extra.iter().find(|(k, _)| k == key).map(|(_, v)| v)
}
}
/// The settings of one property id.
#[derive(Debug, Clone, PartialEq, Default)]
pub struct PropertyCfg {
/// The column title.
pub display_name: Option<String>,
/// Other keys, in original order.
pub extra: Vec<(String, Node)>,
}
/// A parsed `.base` file.
#[derive(Debug, Clone, PartialEq)]
pub struct Base {
/// Filters of every view.
pub filters: Option<Filter>,
/// Formula name and expression.
pub formulas: Vec<(String, String)>,
/// Property id and its settings.
pub properties: Vec<(String, PropertyCfg)>,
/// Custom summary name and expression.
pub summaries: Vec<(String, String)>,
/// Keys the model does not know, in original order.
pub extra: Vec<(String, Node)>,
pub(crate) views: Vec<View>,
pub(crate) views_implicit: bool,
pub(crate) layout: Vec<String>,
}
impl Base {
/// The views. A file without `views` has one default table view.
#[must_use]
pub fn views(&self) -> &[View] {
&self.views
}
/// The views for editing; from now on they are written to the file.
pub fn views_mut(&mut self) -> &mut Vec<View> {
self.views_implicit = false;
&mut self.views
}
}
+76
View File
@@ -0,0 +1,76 @@
//! The columns of a view and the properties a query reads.
use crate::expr::{Expr, parse_expr};
use crate::model::{Base, View};
/// The column shown when a view lists none.
const DEFAULT_COLUMN: &str = "file.name";
/// One column of a result.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Column {
/// The property id: `file.name`, `note.year`, `formula.ppu` or a bare note property.
pub id: String,
/// The title: the `displayName` of the property, else the id without its `note.` / `file.` /
/// `formula.` prefix.
pub name: String,
/// The summary of the view for this column, a default or a custom name.
pub summary: Option<String>,
}
/// An expression that parsed, or the parse message.
pub(super) type Parsed = Result<Expr, String>;
pub(super) fn parse(src: &str) -> Parsed {
parse_expr(src).map_err(|e| e.to_string())
}
/// The columns of `view` with their parsed expressions (a property id is an expression).
pub(super) fn columns(base: &Base, view: &View) -> (Vec<Column>, Vec<Parsed>) {
let ids: Vec<&str> = if view.order.is_empty() { vec![DEFAULT_COLUMN] } else { view.order.iter().map(String::as_str).collect() };
let column = |id: &&str| {
let display = base.properties.iter().find(|(p, _)| p == id).and_then(|(_, cfg)| cfg.display_name.clone());
let short = ["note.", "file.", "formula."].iter().find_map(|p| id.strip_prefix(p)).unwrap_or(id);
let summary = view.summaries.iter().find(|(p, _)| p == id).map(|(_, s)| s.clone());
Column { id: (*id).to_string(), name: display.unwrap_or_else(|| short.to_string()), summary }
};
(ids.iter().map(column).collect(), ids.iter().map(|id| parse(id)).collect())
}
/// What a query reads: the properties of a row it touches. Nothing else is read.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct Reads {
/// Note properties, formulas expanded.
pub note: Vec<String>,
/// `file.*` fields, formulas expanded.
pub file: Vec<String>,
/// Whether `this` is read.
pub uses_this: bool,
}
impl Reads {
/// Adds the dependencies of `expr` and, transitively, of the formulas it names.
pub(super) fn add(&mut self, expr: &Expr, formulas: &[(String, String)], seen: &mut Vec<String>) {
let deps = expr.deps();
self.note.extend(deps.note);
self.file.extend(deps.file);
self.uses_this |= deps.uses_this;
for dep in deps.formula {
let name = dep.split('.').next().unwrap_or(&dep).to_string();
if seen.contains(&name) {
continue;
}
seen.push(name.clone());
if let Some(Ok(inner)) = formulas.iter().find(|(n, _)| *n == name).map(|(_, src)| parse(src)) {
self.add(&inner, formulas, seen);
}
}
}
pub(super) fn finish(&mut self) {
self.note.sort();
self.note.dedup();
self.file.sort();
self.file.dedup();
}
}
+301
View File
@@ -0,0 +1,301 @@
//! The query pipeline of a base view: filter, sort, group, limit, summaries.
//!
//! [`run`] walks the source once for the filters, evaluates only the sort and group keys, and
//! returns [`RowId`]s. A cell is computed on demand by [`QueryResult::cell`] and memoised, so a
//! row nobody asks for costs nothing. Rows and values of the source are never copied.
mod columns;
mod summary;
use std::cell::{Cell, OnceCell};
use std::cmp::Ordering;
use std::collections::HashMap;
pub use columns::{Column, Reads};
pub(crate) use summary::number_stat;
use self::columns::{Parsed, columns, parse};
use self::summary::{Custom, summarize};
use crate::compare::{compare, loose_eq, truthy};
use crate::error::{Error, Result};
use crate::eval::{Env, Formulas, err, eval};
use crate::model::{Base, Direction, Filter, View};
use crate::row::{NoteSource, RowRef};
use crate::value::Value;
use crate::yaml::Node;
/// A row of the result: its index in the [`NoteSource`], stable while the source does not change.
pub type RowId = usize;
static NULL: Value = Value::Null;
/// Which view to run: its index or its name.
#[derive(Debug, Clone, Copy)]
pub enum ViewSel<'a> {
/// Position in `Base::views`.
Index(usize),
/// The `name` of the view.
Name(&'a str),
}
impl From<usize> for ViewSel<'_> {
fn from(i: usize) -> Self {
ViewSel::Index(i)
}
}
impl<'a> From<&'a str> for ViewSel<'a> {
fn from(name: &'a str) -> Self {
ViewSel::Name(name)
}
}
/// One group of rows. Without `groupBy` there is one group with a `Null` key.
#[derive(Debug, Clone, PartialEq)]
pub struct Group {
/// The value of the `groupBy` property shared by the rows (`Null`: no value).
pub key: Value,
/// The rows, in sort order.
pub rows: Vec<RowId>,
/// The summary of each column over this group; `Null` for a column without one.
pub summaries: Vec<Value>,
}
/// What evaluates expressions: the environment, the formulas and the error count.
struct Eval<'a> {
env: Env<'a>,
formulas: Formulas,
errors: Cell<usize>,
}
impl Eval<'_> {
fn value(&self, expr: &Parsed, row: &RowRef<'_>) -> Value {
let value = match expr {
Ok(expr) => eval(expr, &self.env, row, &self.formulas),
Err(message) => err(message.clone()),
};
if matches!(value, Value::Error(_)) {
self.errors.set(self.errors.get().saturating_add(1));
}
value
}
}
/// A filter with its expressions parsed.
enum Compiled {
Expr(Parsed),
And(Vec<Compiled>),
Or(Vec<Compiled>),
Not(Vec<Compiled>),
}
fn compile(filter: &Filter, track: &mut dyn FnMut(&Parsed)) -> Compiled {
let mut all = |members: &[Filter]| members.iter().map(|m| compile(m, track)).collect::<Vec<_>>();
match filter {
Filter::Expr(src) => {
let parsed = parse(src);
track(&parsed);
Compiled::Expr(parsed)
}
Filter::And(m) => Compiled::And(all(m)),
Filter::Or(m) => Compiled::Or(all(m)),
Filter::Not(m) => Compiled::Not(all(m)),
}
}
impl Eval<'_> {
/// Whether the row passes; an empty group imposes nothing, an error counts as false.
fn passes(&self, filter: &Compiled, row: &RowRef<'_>) -> bool {
match filter {
Compiled::Expr(e) => {
let v = self.value(e, row);
!matches!(v, Value::Error(_)) && truthy(&v)
}
Compiled::And(m) => m.iter().all(|f| self.passes(f, row)),
Compiled::Or(m) => m.is_empty() || m.iter().any(|f| self.passes(f, row)),
Compiled::Not(m) => !m.iter().any(|f| self.passes(f, row)),
}
}
}
/// The outcome of [`run`].
pub struct QueryResult<'a> {
/// The columns of the view.
pub columns: Vec<Column>,
/// The groups, with the limit applied.
pub groups: Vec<Group>,
/// Rows that matched the filters, before the limit.
pub total: usize,
/// The summary of each column over every shown row; `Null` for a column without one.
pub summaries: Vec<Value>,
/// The properties the query reads; every other property is never touched.
pub reads: Reads,
source: &'a dyn NoteSource,
eval: Eval<'a>,
exprs: Vec<Parsed>,
slots: HashMap<RowId, usize>,
cells: Vec<OnceCell<Value>>,
computed: Cell<usize>,
}
impl QueryResult<'_> {
/// The value of column `col` for `row`, computed on first use. `Null` for a row that is not in
/// the result or a column that does not exist.
#[must_use]
pub fn cell(&self, row: RowId, col: usize) -> &Value {
let (Some(slot), Some(expr)) = (self.slots.get(&row), self.exprs.get(col)) else { return &NULL };
let Some(cell) = self.cells.get(slot * self.exprs.len() + col) else { return &NULL };
cell.get_or_init(|| {
self.computed.set(self.computed.get().saturating_add(1));
self.source.row(row).map_or(Value::Null, |r| self.eval.value(expr, &r))
})
}
/// How many cells have been computed so far, summaries included.
#[must_use]
pub fn cells_computed(&self) -> usize {
self.computed.get()
}
/// How many evaluations failed so far: filters, keys and cells.
#[must_use]
pub fn errors(&self) -> usize {
self.eval.errors.get()
}
/// The rows of every group in display order.
pub fn rows(&self) -> impl Iterator<Item = RowId> + '_ {
self.groups.iter().flat_map(|g| g.rows.iter().copied())
}
fn summarize(&self, name: &str, rows: &[RowId], col: usize, custom: &[Custom]) -> Value {
let values: Vec<&Value> = rows.iter().map(|r| self.cell(*r, col)).collect();
summarize(name, &values, custom, &self.eval.env, &self.eval.formulas)
}
}
fn find_view<'b>(base: &'b Base, sel: ViewSel<'_>) -> Result<&'b View> {
let found = match sel {
ViewSel::Index(i) => base.views().get(i),
ViewSel::Name(n) => base.views().iter().find(|v| v.name.as_deref() == Some(n)),
};
found.ok_or_else(|| Error::Invalid(format!("no such view: {sel:?}")))
}
fn ordered(a: &Value, b: &Value, dir: Direction) -> Ordering {
let o = compare(a, b);
if dir == Direction::Desc { o.reverse() } else { o }
}
/// Whether a `groupOrder` entry names the group `key`.
fn names_group(entry: &Node, key: &Value) -> bool {
match entry {
Node::Null => matches!(key, Value::Null),
Node::Bool(b) => loose_eq(key, &Value::Bool(*b)),
Node::Num(n) => loose_eq(key, &Value::Number(*n)),
Node::Str(s) => !matches!(key, Value::Null) && (loose_eq(key, &Value::Str(s.clone())) || key.to_display_string() == *s),
Node::Seq(_) | Node::Map(_) => false,
}
}
/// Runs a view of `base` over `source`.
///
/// `env.source` defaults to `source`. Rows are filtered by the base filters AND the view filters,
/// sorted (stable, multi-key), grouped, cut to the view `limit` and summarised.
///
/// # Errors
/// [`Error::Invalid`] when the view does not exist.
pub fn run<'a>(base: &'a Base, view: impl Into<ViewSel<'a>>, source: &'a dyn NoteSource, env: &Env<'a>) -> Result<QueryResult<'a>> {
let view = find_view(base, view.into())?;
let mut env = *env;
env.source = env.source.or(Some(source));
let eval = Eval { env, formulas: Formulas::new(&base.formulas), errors: Cell::new(0) };
let mut reads = Reads::default();
let mut track = |p: &Parsed| {
if let Ok(expr) = p {
reads.add(expr, &base.formulas, &mut Vec::new());
}
};
let (cols, exprs) = columns(base, view);
exprs.iter().for_each(&mut track);
let filters: Vec<Compiled> = [&base.filters, &view.filters].into_iter().flatten().map(|f| compile(f, &mut track)).collect();
let sorts: Vec<(Parsed, Direction)> = view.sort.iter().map(|s| (parse(&s.property), s.direction)).collect();
let group = view.group_by.as_ref().map(|g| (parse(&g.property), g.direction));
sorts.iter().map(|(p, _)| p).chain(group.iter().map(|(p, _)| p)).for_each(&mut track);
reads.finish();
let kept: Vec<(RowId, RowRef<'a>)> =
(0..source.len()).filter_map(|i| source.row(i).map(|r| (i, r))).filter(|(_, row)| filters.iter().all(|f| eval.passes(f, row))).collect();
let total = kept.len();
let keys: Vec<Vec<Value>> =
if sorts.is_empty() { Vec::new() } else { kept.iter().map(|(_, r)| sorts.iter().map(|(p, _)| eval.value(p, r)).collect()).collect() };
let mut order: Vec<usize> = (0..total).collect();
if !sorts.is_empty() {
order.sort_by(|a, b| {
for (k, (_, dir)) in sorts.iter().enumerate() {
let o = match (keys.get(*a).and_then(|r| r.get(k)), keys.get(*b).and_then(|r| r.get(k))) {
(Some(x), Some(y)) => ordered(x, y, *dir),
_ => Ordering::Equal,
};
if o != Ordering::Equal {
return o;
}
}
Ordering::Equal
});
}
let mut groups: Vec<Group> = Vec::new();
if let Some((expr, dir)) = &group {
let gkeys: Vec<Value> = kept.iter().map(|(_, r)| eval.value(expr, r)).collect();
order.sort_by(|a, b| match (gkeys.get(*a), gkeys.get(*b)) {
(Some(x), Some(y)) => ordered(x, y, *dir),
_ => Ordering::Equal,
});
for pos in &order {
let (Some(key), Some((id, _))) = (gkeys.get(*pos), kept.get(*pos)) else { continue };
match groups.last_mut() {
Some(g) if compare(&g.key, key) == Ordering::Equal => g.rows.push(*id),
_ => groups.push(Group { key: key.clone(), rows: vec![*id], summaries: Vec::new() }),
}
}
if !view.group_order.is_empty() {
let mut pool: Vec<Option<Group>> = groups.into_iter().map(Some).collect();
groups =
view.group_order.iter().filter_map(|entry| pool.iter_mut().find(|g| g.as_ref().is_some_and(|g| names_group(entry, &g.key)))?.take()).collect();
}
} else {
let rows: Vec<RowId> = order.iter().filter_map(|p| kept.get(*p).map(|(id, _)| *id)).collect();
if !rows.is_empty() {
groups.push(Group { key: Value::Null, rows, summaries: Vec::new() });
}
}
if let Some(limit) = view.limit {
let mut left = limit as usize;
for g in &mut groups {
g.rows.truncate(left);
left = left.saturating_sub(g.rows.len());
}
groups.retain(|g| !g.rows.is_empty());
}
let shown: Vec<RowId> = groups.iter().flat_map(|g| g.rows.iter().copied()).collect();
let slots = shown.iter().enumerate().map(|(slot, id)| (*id, slot)).collect();
let cells = (0..shown.len().saturating_mul(exprs.len())).map(|_| OnceCell::new()).collect();
let mut result =
QueryResult { columns: cols, groups: Vec::new(), total, summaries: Vec::new(), reads, source, eval, exprs, slots, cells, computed: Cell::new(0) };
let custom: Vec<Custom> = base.summaries.iter().map(|(n, src)| (n.clone(), parse(src))).collect();
let column_summaries = |rows: &[RowId]| -> Vec<Value> {
result.columns.iter().enumerate().map(|(col, c)| c.summary.as_deref().map_or(Value::Null, |name| result.summarize(name, rows, col, &custom))).collect()
};
for g in &mut groups {
g.summaries = column_summaries(&g.rows);
}
let overall = column_summaries(&shown);
result.groups = groups;
result.summaries = overall;
Ok(result)
}
+128
View File
@@ -0,0 +1,128 @@
//! Summaries: the default ones of the docs (implemented here, not as formulas) and the custom
//! ones, which are expressions over `values`.
use std::cmp::Ordering;
use crate::compare::compare;
use crate::datetime::Duration;
use crate::eval::{Env, Formulas, err, eval, int_to_f64};
use crate::expr::Expr;
use crate::row::RowRef;
use crate::value::Value;
/// A custom summary: name and parsed expression (or the parse message).
pub(super) type Custom = (String, Result<Expr, String>);
/// The result of the named summary over `values`, `Error` when the name is unknown.
pub(super) fn summarize(name: &str, values: &[&Value], custom: &[Custom], env: &Env<'_>, formulas: &Formulas) -> Value {
if let Some(done) = builtin(name, values) {
return done;
}
match custom.iter().find(|(n, _)| n == name) {
None => err(format!("unknown summary `{name}`")),
Some((_, Err(message))) => err(message.clone()),
Some((_, Ok(expr))) => {
let props = [("values".to_string(), Value::List(values.iter().map(|v| (*v).clone()).collect()))];
let row = RowRef { props: &props, ..RowRef::new("") };
eval(expr, env, &row, formulas)
}
}
}
/// `sum`, `mean`, `median`, `min`, `max` over the numbers of a list, `Null` when there are none
/// (`sum` is 0). Used by the summaries and by the list methods of the same names.
pub(crate) fn number_stat(name: &str, items: &[Value]) -> Option<Value> {
let nums = numbers(items.iter());
Some(match name {
"sum" => Value::Number(nums.iter().sum()),
"mean" => mean(&nums),
"median" => median(nums),
"min" => fold(&nums, f64::min),
"max" => fold(&nums, f64::max),
_ => return None,
})
}
fn numbers<'a>(values: impl Iterator<Item = &'a Value>) -> Vec<f64> {
values.filter_map(|v| if let Value::Number(n) = v { Some(*n).filter(|n| !n.is_nan()) } else { None }).collect()
}
fn mean(nums: &[f64]) -> Value {
if nums.is_empty() { Value::Null } else { Value::Number(nums.iter().sum::<f64>() / int_to_f64(nums.len())) }
}
fn fold(nums: &[f64], f: fn(f64, f64) -> f64) -> Value {
nums.iter().copied().reduce(f).map_or(Value::Null, Value::Number)
}
fn median(mut nums: Vec<f64>) -> Value {
nums.sort_by(f64::total_cmp);
let mid = nums.len() / 2;
match (nums.get(mid), mid.checked_sub(1).and_then(|i| nums.get(i))) {
(None, _) => Value::Null,
(Some(hi), Some(lo)) if nums.len().is_multiple_of(2) => Value::Number((lo + hi) / 2.0),
(Some(m), _) => Value::Number(*m),
}
}
/// Population standard deviation.
fn stddev(nums: &[f64]) -> Value {
let Value::Number(m) = mean(nums) else { return Value::Null };
let var = nums.iter().map(|x| (x - m).powi(2)).sum::<f64>() / int_to_f64(nums.len());
Value::Number(var.sqrt())
}
fn dates(values: &[&Value]) -> Vec<crate::datetime::Date> {
values.iter().filter_map(|v| if let Value::Date(d) = v { Some(*d) } else { None }).collect()
}
/// Whether a value counts as empty: null, `""`, an empty list or object.
fn is_empty(v: &Value) -> bool {
match v {
Value::Null => true,
Value::Str(s) => s.is_empty(),
Value::List(l) => l.is_empty(),
Value::Object(o) => o.is_empty(),
_ => false,
}
}
fn count(n: usize) -> Value {
Value::Number(int_to_f64(n))
}
fn builtin(name: &str, values: &[&Value]) -> Option<Value> {
let nums = || numbers(values.iter().copied());
Some(match name {
"Average" => mean(&nums()),
"Sum" => Value::Number(nums().iter().sum()),
"Min" => fold(&nums(), f64::min),
"Max" => fold(&nums(), f64::max),
"Median" => median(nums()),
"Stddev" => stddev(&nums()),
"Range" => {
let n = nums();
if let (Value::Number(lo), Value::Number(hi)) = (fold(&n, f64::min), fold(&n, f64::max)) {
return Some(Value::Number(hi - lo));
}
let d = dates(values);
match (d.iter().map(|d| d.ms()).min(), d.iter().map(|d| d.ms()).max()) {
(Some(lo), Some(hi)) => hi.checked_sub(lo).map_or_else(|| err("date range out of range"), |ms| Value::Duration(Duration::new(0, ms))),
_ => Value::Null,
}
}
"Earliest" => dates(values).into_iter().min_by_key(|d| d.ms()).map_or(Value::Null, Value::Date),
"Latest" => dates(values).into_iter().max_by_key(|d| d.ms()).map_or(Value::Null, Value::Date),
"Checked" => count(values.iter().filter(|v| matches!(v, Value::Bool(true))).count()),
"Unchecked" => count(values.iter().filter(|v| matches!(v, Value::Bool(false))).count()),
"Empty" => count(values.iter().filter(|v| is_empty(v)).count()),
"Filled" => count(values.iter().filter(|v| !is_empty(v)).count()),
"Unique" => {
let mut sorted: Vec<&Value> = values.to_vec();
sorted.sort_by(|a, b| compare(a, b));
sorted.dedup_by(|a, b| compare(a, b) == Ordering::Equal);
count(sorted.len())
}
_ => return None,
})
}
+124
View File
@@ -0,0 +1,124 @@
//! The rows a base queries: the [`NoteSource`] trait and the borrowed [`RowRef`].
//!
//! The evaluator never sees index types; the engine implements [`NoteSource`] over its snapshot
//! and tests implement it over a `Vec` of rows. Nothing is copied: a row borrows its data.
use crate::datetime::Date;
use crate::eval::int_to_f64;
use crate::value::Value;
/// A lazy list of candidate files.
pub trait NoteSource {
/// The number of candidate files.
fn len(&self) -> usize;
/// Whether there is no candidate file.
fn is_empty(&self) -> bool {
self.len() == 0
}
/// The row at `i`, `None` past the end.
fn row(&self, i: usize) -> Option<RowRef<'_>>;
/// The row of the file at the vault-relative `path`.
fn find(&self, path: &str) -> Option<RowRef<'_>>;
/// The path a link target resolves to, `None` when no file matches.
///
/// The default tries the target as a path, then with `.md` appended.
fn resolve_link(&self, target: &str, _from: &str) -> Option<String> {
let found = self.find(target).or_else(|| self.find(&[target, ".md"].concat()))?;
Some(found.path.to_string())
}
}
/// One file of the vault, borrowed from its source.
#[derive(Clone, Copy)]
pub struct RowRef<'a> {
/// Vault-relative path.
pub path: &'a str,
/// Size in bytes.
pub size: u64,
/// Creation time, Unix milliseconds.
pub ctime_ms: i64,
/// Modification time, Unix milliseconds.
pub mtime_ms: i64,
/// Tags, with or without the leading `#`.
pub tags: &'a [String],
/// Link targets of the file.
pub links: &'a [String],
/// Embed targets of the file.
pub embeds: &'a [String],
/// Frontmatter properties.
pub props: &'a [(String, Value)],
/// Paths of the files linking here; computed only when asked for.
pub backlinks: Option<&'a dyn Fn() -> Vec<String>>,
}
impl<'a> RowRef<'a> {
/// A row with a path and nothing else.
#[must_use]
pub fn new(path: &'a str) -> Self {
Self { path, size: 0, ctime_ms: 0, mtime_ms: 0, tags: &[], links: &[], embeds: &[], props: &[], backlinks: None }
}
/// The file name with its extension.
#[must_use]
pub fn name(&self) -> &'a str {
self.path.rsplit_once('/').map_or(self.path, |(_, name)| name)
}
/// The file name without its extension.
#[must_use]
pub fn basename(&self) -> &'a str {
let name = self.name();
match name.rsplit_once('.') {
Some((stem, _)) if !stem.is_empty() => stem,
_ => name,
}
}
/// The extension, without the dot.
#[must_use]
pub fn ext(&self) -> &'a str {
match self.name().rsplit_once('.') {
Some((stem, ext)) if !stem.is_empty() => ext,
_ => "",
}
}
/// The parent folder, empty for the vault root.
#[must_use]
pub fn folder(&self) -> &'a str {
self.path.rsplit_once('/').map_or("", |(folder, _)| folder)
}
/// A frontmatter property.
#[must_use]
pub fn prop(&self, key: &str) -> Option<&'a Value> {
self.props.iter().find(|(k, _)| k == key).map(|(_, v)| v)
}
/// A `file.*` field; `None` when the name is not a file field.
#[must_use]
pub fn field(&self, name: &str) -> Option<Value> {
let links = |targets: &[String]| Value::List(targets.iter().map(|t| Value::Link { target: t.clone(), display: String::new() }).collect());
Some(match name {
"name" => Value::Str(self.name().to_string()),
"basename" => Value::Str(self.basename().to_string()),
"path" => Value::Str(self.path.to_string()),
"folder" => Value::Str(self.folder().to_string()),
"ext" => Value::Str(self.ext().to_string()),
"size" => Value::Number(int_to_f64(self.size)),
"ctime" => Value::Date(Date::new(self.ctime_ms, true)),
"mtime" => Value::Date(Date::new(self.mtime_ms, true)),
"tags" => Value::List(self.tags.iter().cloned().map(Value::Str).collect()),
"links" => links(self.links),
"embeds" => links(self.embeds),
"backlinks" => links(&self.backlinks.map(|f| f()).unwrap_or_default()),
"properties" => Value::Object(self.props.to_vec()),
"file" => Value::File { path: self.path.to_string() },
_ => return None,
})
}
}
+11
View File
@@ -0,0 +1,11 @@
//! Object type schemas: definitions, bounded inheritance, deterministic multi-tag merge and
//! advisory validation of note properties. Pure, no I/O; the JSON codec and the registry file
//! live in the engine.
mod model;
mod resolve;
#[cfg(test)]
mod tests;
pub use model::{FieldDef, FieldKind, MAX_DEPTH, MAX_FIELDS, MAX_IDENT_BYTES, MAX_LABEL_BYTES, MAX_TYPES, TypeDef};
pub use resolve::{Diagnostic, DiagnosticCode, ResolvedSchema, TypeRegistry};
+147
View File
@@ -0,0 +1,147 @@
//! The object type model: definitions, field kinds, limits and value checks.
//!
//! Kinds match the property kinds the properties commands can write (text, number, checkbox,
//! date, datetime, text list). Values are the Bases [`Value`]: text, dates and datetimes are
//! `Value::Str` (as frontmatter reads them) or `Value::Date`, a list holds `Value::Str` items.
use notescraft_markdown::{YamlValueText, is_calendar_date, is_datetime, set_key};
use crate::error::{Error, Result, SchemaErrorKind};
use crate::value::Value;
/// Most types in one registry.
pub const MAX_TYPES: usize = 256;
/// Most fields in one resolved schema (inherited and merged included), and in one definition.
pub const MAX_FIELDS: usize = 64;
/// Longest `extends` chain, the type itself included.
pub const MAX_DEPTH: usize = 16;
/// Longest id or tag, in bytes.
pub const MAX_IDENT_BYTES: usize = 64;
/// Longest label, in bytes.
pub const MAX_LABEL_BYTES: usize = 256;
/// The kind of a field.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum FieldKind {
/// A free string.
Text,
/// A finite number.
Number,
/// `true` or `false`.
Checkbox,
/// A `YYYY-MM-DD` calendar date.
Date,
/// A date with a time part.
DateTime,
/// A list of strings.
List,
}
impl FieldKind {
/// The name used by the properties commands (`text`, `number`, `checkbox`, `date`,
/// `datetime`, `list`).
#[must_use]
pub fn as_str(self) -> &'static str {
match self {
Self::Text => "text",
Self::Number => "number",
Self::Checkbox => "checkbox",
Self::Date => "date",
Self::DateTime => "datetime",
Self::List => "list",
}
}
/// Why `value` does not fit this kind, or `None` when it does. `Null` always fits.
#[must_use]
pub fn mismatch(self, value: &Value) -> Option<String> {
let fits = match (self, value) {
(_, Value::Null) | (Self::Text, Value::Str(_)) | (Self::Checkbox, Value::Bool(_)) => true,
(Self::Number, Value::Number(number)) => number.is_finite(),
(Self::Date, Value::Str(text)) => is_calendar_date(text),
(Self::Date, Value::Date(date)) => !date.has_time(),
(Self::DateTime, Value::Str(text)) => is_datetime(text),
(Self::DateTime, Value::Date(date)) => date.has_time(),
(Self::List, Value::List(items)) => items.iter().all(|item| matches!(item, Value::Str(_))),
_ => false,
};
(!fits).then(|| format!("expected {}, found {}", self.as_str(), value.type_name()))
}
}
/// One field of a type.
#[derive(Debug, Clone, PartialEq)]
pub struct FieldDef {
/// The frontmatter key.
pub key: String,
/// The kind of the value.
pub kind: FieldKind,
/// Whether a present, non-empty value is required.
pub required: bool,
/// The value applied when the field is absent.
pub default: Option<Value>,
}
/// One object type definition.
#[derive(Debug, Clone, PartialEq)]
pub struct TypeDef {
/// Unique id.
pub id: String,
/// Name shown to the user.
pub label: String,
/// The frontmatter tag that makes a note this type; unique.
pub tag: String,
/// The id of the parent type, if any.
pub extends: Option<String>,
/// The fields this type declares itself.
pub fields: Vec<FieldDef>,
}
fn schema(kind: SchemaErrorKind, message: String) -> Error {
Error::Schema { kind, message }
}
fn is_ident(text: &str) -> bool {
!text.is_empty() && text.len() <= MAX_IDENT_BYTES && text.bytes().all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'_' | b'-'))
}
/// Checks one definition on its own (names, limits, defaults); relations between types are the
/// registry's job.
///
/// # Errors
///
/// [`Error::Schema`] for a bad name, duplicate field key or bad default, [`Error::Limit`] for
/// more than [`MAX_FIELDS`] fields or an over-long label.
pub fn check_def(def: &TypeDef) -> Result<()> {
if !is_ident(&def.id) {
return Err(schema(SchemaErrorKind::BadName, format!("id must be 1..{MAX_IDENT_BYTES} letters, digits, _ or -")));
}
let id = &def.id;
if !is_ident(&def.tag) {
return Err(schema(SchemaErrorKind::BadName, format!("{id}: tag must be 1..{MAX_IDENT_BYTES} letters, digits, _ or -")));
}
if def.extends.as_deref().is_some_and(|parent| !is_ident(parent)) {
return Err(schema(SchemaErrorKind::BadName, format!("{id}: parent is not a valid id")));
}
if def.label.is_empty() {
return Err(schema(SchemaErrorKind::BadName, format!("{id}: label is empty")));
}
if def.label.len() > MAX_LABEL_BYTES {
return Err(Error::Limit(format!("{id}: label over {MAX_LABEL_BYTES} bytes")));
}
if def.fields.len() > MAX_FIELDS {
return Err(Error::Limit(format!("{id}: more than {MAX_FIELDS} fields")));
}
for (index, field) in def.fields.iter().enumerate() {
if set_key("", &field.key, &YamlValueText::new("x")).is_err() {
return Err(schema(SchemaErrorKind::BadName, format!("{id}: invalid field key {:?}", field.key)));
}
if def.fields.iter().take(index).any(|earlier| earlier.key == field.key) {
return Err(schema(SchemaErrorKind::Duplicate, format!("{id}: field {} declared twice", field.key)));
}
if let Some(reason) = field.default.as_ref().and_then(|default| field.kind.mismatch(default)) {
return Err(schema(SchemaErrorKind::BadDefault, format!("{id}.{}: {reason}", field.key)));
}
}
Ok(())
}
+207
View File
@@ -0,0 +1,207 @@
//! The registry, inheritance, deterministic multi-tag merge and validation.
use std::collections::HashSet;
use super::model::{FieldDef, MAX_DEPTH, MAX_FIELDS, MAX_TYPES, TypeDef, check_def};
use crate::error::{Error, Result, SchemaErrorKind};
use crate::value::Value;
/// What a [`Diagnostic`] reports.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum DiagnosticCode {
/// A required field is absent, null or empty.
Required,
/// A field holds a value of the wrong kind or an invalid date or number.
WrongType,
}
impl DiagnosticCode {
/// The stable machine name (`required`, `wrong_type`).
#[must_use]
pub fn as_str(self) -> &'static str {
match self {
Self::Required => "required",
Self::WrongType => "wrong_type",
}
}
}
/// One advisory finding about a note's properties.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Diagnostic {
/// The field key.
pub key: String,
/// What kind of finding.
pub code: DiagnosticCode,
/// A plain-words explanation.
pub message: String,
}
/// The merged fields of the types that apply to one note.
#[derive(Debug, Clone, PartialEq)]
pub struct ResolvedSchema {
/// Ids of the applied types, sorted; ancestors of another applied type are left out.
pub types: Vec<String>,
/// Inherited fields first (root to leaf), then merged ones in type-id order.
pub fields: Vec<FieldDef>,
}
impl ResolvedSchema {
/// Checks note properties (`key`, value pairs, first key wins). Required means present,
/// non-null and not empty text or list. Keys outside the schema are ignored.
#[must_use]
pub fn validate(&self, values: &[(String, Value)]) -> Vec<Diagnostic> {
let mut out = Vec::new();
for field in &self.fields {
let found = values.iter().find(|(key, _)| *key == field.key).map(|(_, value)| value);
let missing = match found {
None | Some(Value::Null) => true,
Some(Value::Str(text)) => text.is_empty(),
Some(Value::List(items)) => items.is_empty(),
Some(_) => false,
};
if missing {
if field.required {
out.push(Diagnostic { key: field.key.clone(), code: DiagnosticCode::Required, message: format!("{} is required", field.key) });
}
continue;
}
if let Some(reason) = found.and_then(|value| field.kind.mismatch(value)) {
out.push(Diagnostic { key: field.key.clone(), code: DiagnosticCode::WrongType, message: format!("{}: {reason}", field.key) });
}
}
out
}
}
/// A validated set of object types.
#[derive(Debug, Clone, PartialEq)]
pub struct TypeRegistry {
types: Vec<TypeDef>,
}
fn conflict(message: String) -> Error {
Error::Schema { kind: SchemaErrorKind::Conflict, message }
}
/// Merges `field` into `into`. Kinds must agree; required is OR. With `child_wins` a default of
/// `field` replaces the inherited one, without it two different defaults conflict.
fn merge_field(into: &mut Vec<FieldDef>, field: &FieldDef, child_wins: bool, origin: &str) -> Result<()> {
let Some(slot) = into.iter_mut().find(|existing| existing.key == field.key) else {
if into.len() >= MAX_FIELDS {
return Err(Error::Limit(format!("{origin}: more than {MAX_FIELDS} fields")));
}
into.push(field.clone());
return Ok(());
};
if slot.kind != field.kind {
return Err(conflict(format!("{origin}: field {} is {} and {}", field.key, slot.kind.as_str(), field.kind.as_str())));
}
slot.required |= field.required;
match (&slot.default, &field.default) {
(None, Some(_)) => slot.default.clone_from(&field.default),
(Some(_), Some(new)) if child_wins => slot.default = Some(new.clone()),
(Some(old), Some(new)) if old != new => return Err(conflict(format!("{origin}: field {} has two different defaults", field.key))),
(_, None) | (Some(_), Some(_)) => {}
}
Ok(())
}
impl TypeRegistry {
/// Validates `definitions` and builds the registry.
///
/// # Errors
///
/// [`Error::Limit`] over 256 types or 64 fields; [`Error::Schema`] for a bad name or default,
/// a duplicate id or tag, an unknown parent, a cycle, a chain over 16 deep, or an
/// incompatible field override.
pub fn new(definitions: Vec<TypeDef>) -> Result<Self> {
if definitions.len() > MAX_TYPES {
return Err(Error::Limit(format!("more than {MAX_TYPES} object types")));
}
let mut ids = HashSet::new();
let mut tags = HashSet::new();
for def in &definitions {
check_def(def)?;
if !ids.insert(def.id.as_str()) {
return Err(Error::Schema { kind: SchemaErrorKind::Duplicate, message: format!("id {} declared twice", def.id) });
}
if !tags.insert(def.tag.as_str()) {
return Err(Error::Schema { kind: SchemaErrorKind::Duplicate, message: format!("tag {} used twice", def.tag) });
}
}
let registry = Self { types: definitions };
for def in &registry.types {
registry.resolve_type(&def.id)?;
}
Ok(registry)
}
/// The definitions, in the order given.
#[must_use]
pub fn types(&self) -> &[TypeDef] {
&self.types
}
/// The definition with `id`.
#[must_use]
pub fn get(&self, id: &str) -> Option<&TypeDef> {
self.types.iter().find(|def| def.id == id)
}
/// `id` and its ancestors, root first.
fn chain(&self, id: &str) -> Result<Vec<&TypeDef>> {
let mut chain: Vec<&TypeDef> = Vec::new();
let mut next = Some(id);
while let Some(current) = next {
let def = self.get(current).ok_or_else(|| Error::Schema { kind: SchemaErrorKind::UnknownParent, message: format!("unknown type {current}") })?;
if chain.iter().any(|seen| seen.id == def.id) {
return Err(Error::Schema { kind: SchemaErrorKind::Cycle, message: format!("{id} extends itself through {current}") });
}
chain.push(def);
next = def.extends.as_deref();
}
if chain.len() > MAX_DEPTH {
return Err(Error::Schema { kind: SchemaErrorKind::TooDeep, message: format!("{id} extends {} levels, limit {MAX_DEPTH}", chain.len()) });
}
chain.reverse();
Ok(chain)
}
fn resolve_type(&self, id: &str) -> Result<ResolvedSchema> {
let mut fields = Vec::new();
for def in self.chain(id)? {
for field in &def.fields {
merge_field(&mut fields, field, true, id)?;
}
}
Ok(ResolvedSchema { types: vec![id.to_owned()], fields })
}
/// The schema for a note carrying `tags`: every type whose tag is present, merged in id order
/// whatever the tag order. Tags without a type are ignored. Two applicable types where one is
/// an ancestor of the other count as the descendant alone.
///
/// # Errors
///
/// [`Error::Schema`] with [`SchemaErrorKind::Conflict`] when two types give one field
/// different kinds or defaults; [`Error::Limit`] over 64 merged fields.
pub fn resolve(&self, tags: &[impl AsRef<str>]) -> Result<ResolvedSchema> {
let mut picked: Vec<&TypeDef> = self.types.iter().filter(|def| tags.iter().any(|tag| tag.as_ref() == def.tag)).collect();
let mut ancestors: HashSet<&str> = HashSet::new();
for def in &picked {
ancestors.extend(self.chain(&def.id)?.iter().filter(|link| link.id != def.id).map(|link| link.id.as_str()));
}
picked.retain(|def| !ancestors.contains(def.id.as_str()));
picked.sort_by(|left, right| left.id.cmp(&right.id));
let mut merged = ResolvedSchema { types: Vec::new(), fields: Vec::new() };
for def in picked {
let one = self.resolve_type(&def.id)?;
for field in &one.fields {
merge_field(&mut merged.fields, field, false, &def.id)?;
}
merged.types.push(def.id.clone());
}
Ok(merged)
}
}
+256
View File
@@ -0,0 +1,256 @@
//! Behaviour tests of the object type schema.
use super::*;
use crate::error::{Error, SchemaErrorKind};
use crate::value::Value;
fn field(key: &str, kind: FieldKind, required: bool, default: Option<Value>) -> FieldDef {
FieldDef { key: key.into(), kind, required, default }
}
fn def(id: &str, extends: Option<&str>, fields: Vec<FieldDef>) -> TypeDef {
TypeDef { id: id.into(), label: id.to_uppercase(), tag: id.to_lowercase(), extends: extends.map(Into::into), fields }
}
fn kind_of(result: &crate::Result<TypeRegistry>) -> Option<SchemaErrorKind> {
match result {
Err(Error::Schema { kind, .. }) => Some(*kind),
_ => None,
}
}
fn s(text: &str) -> Value {
Value::Str(text.into())
}
fn chain(len: usize) -> Vec<TypeDef> {
(0..len).map(|n| def(&format!("t{n}"), n.checked_sub(1).map(|p| format!("t{p}")).as_deref(), vec![])).collect()
}
#[test]
fn builds_and_resolves_a_simple_type() {
let book = def("book", None, vec![field("author", FieldKind::Text, true, None), field("rating", FieldKind::Number, false, Some(Value::Number(3.0)))]);
let registry = TypeRegistry::new(vec![book]).unwrap();
let schema = registry.resolve(&["book", "plain"]).unwrap();
assert_eq!(schema.types, ["book"]);
assert_eq!(schema.fields.len(), 2);
assert_eq!(registry.resolve(&["plain"]).unwrap().fields, []);
}
#[test]
fn size_limits() {
let many: Vec<TypeDef> = (0..=MAX_TYPES).map(|n| def(&format!("t{n}"), None, vec![])).collect();
assert!(matches!(TypeRegistry::new(many), Err(Error::Limit(_))));
let at_limit: Vec<TypeDef> = (0..MAX_TYPES).map(|n| def(&format!("t{n}"), None, vec![])).collect();
assert!(TypeRegistry::new(at_limit).is_ok());
let fields = |n: usize| (0..n).map(|k| field(&format!("f{k}"), FieldKind::Text, false, None)).collect::<Vec<_>>();
assert!(TypeRegistry::new(vec![def("a", None, fields(MAX_FIELDS))]).is_ok());
assert!(matches!(TypeRegistry::new(vec![def("a", None, fields(MAX_FIELDS + 1))]), Err(Error::Limit(_))));
let long = "x".repeat(MAX_IDENT_BYTES);
assert!(TypeRegistry::new(vec![TypeDef { id: long.clone(), tag: long.clone(), ..def("a", None, vec![]) }]).is_ok());
let too_long = "x".repeat(MAX_IDENT_BYTES + 1);
let bad_id = TypeDef { id: too_long.clone(), ..def("a", None, vec![]) };
assert_eq!(kind_of(&TypeRegistry::new(vec![bad_id])), Some(SchemaErrorKind::BadName));
let bad_tag = TypeDef { tag: too_long, ..def("a", None, vec![]) };
assert_eq!(kind_of(&TypeRegistry::new(vec![bad_tag])), Some(SchemaErrorKind::BadName));
let big_label = TypeDef { label: "x".repeat(MAX_LABEL_BYTES + 1), ..def("a", None, vec![]) };
assert!(matches!(TypeRegistry::new(vec![big_label]), Err(Error::Limit(_))));
}
#[test]
fn resolved_field_limit_counts_inherited_fields() {
let parent_fields = (0..40).map(|k| field(&format!("p{k}"), FieldKind::Text, false, None)).collect();
let child_fields = (0..25).map(|k| field(&format!("c{k}"), FieldKind::Text, false, None)).collect();
let registry = TypeRegistry::new(vec![def("p", None, parent_fields), def("c", Some("p"), child_fields)]);
assert!(matches!(registry, Err(Error::Limit(_))));
}
#[test]
fn hostile_names_rejected() {
for bad in ["", "a b", "a/b", "é", "a\nb", "a.b"] {
let bad_id = TypeDef { id: bad.into(), ..def("a", None, vec![]) };
assert_eq!(kind_of(&TypeRegistry::new(vec![bad_id])), Some(SchemaErrorKind::BadName), "{bad:?}");
}
for bad_key in ["", "a\nb", ": x", "- x", "#c"] {
let registry = TypeRegistry::new(vec![def("a", None, vec![field(bad_key, FieldKind::Text, false, None)])]);
assert_eq!(kind_of(&registry), Some(SchemaErrorKind::BadName), "{bad_key:?}");
}
let empty_label = TypeDef { label: String::new(), ..def("a", None, vec![]) };
assert_eq!(kind_of(&TypeRegistry::new(vec![empty_label])), Some(SchemaErrorKind::BadName));
}
#[test]
fn duplicates_rejected() {
let dup_id = TypeRegistry::new(vec![def("a", None, vec![]), TypeDef { tag: "other".into(), ..def("a", None, vec![]) }]);
assert_eq!(kind_of(&dup_id), Some(SchemaErrorKind::Duplicate));
let dup_tag = TypeRegistry::new(vec![def("a", None, vec![]), TypeDef { tag: "a".into(), ..def("b", None, vec![]) }]);
assert_eq!(kind_of(&dup_tag), Some(SchemaErrorKind::Duplicate));
let dup_field = def("a", None, vec![field("k", FieldKind::Text, false, None), field("k", FieldKind::Text, false, None)]);
assert_eq!(kind_of(&TypeRegistry::new(vec![dup_field])), Some(SchemaErrorKind::Duplicate));
}
#[test]
fn missing_parent_and_cycles() {
assert_eq!(kind_of(&TypeRegistry::new(vec![def("a", Some("nope"), vec![])])), Some(SchemaErrorKind::UnknownParent));
assert_eq!(kind_of(&TypeRegistry::new(vec![def("a", Some("a"), vec![])])), Some(SchemaErrorKind::Cycle));
let two = TypeRegistry::new(vec![def("a", Some("b"), vec![]), def("b", Some("a"), vec![])]);
assert_eq!(kind_of(&two), Some(SchemaErrorKind::Cycle));
let mut long_cycle = chain(MAX_DEPTH + 5);
long_cycle[0].extends = Some(format!("t{}", MAX_DEPTH + 4));
assert_eq!(kind_of(&TypeRegistry::new(long_cycle)), Some(SchemaErrorKind::Cycle));
}
#[test]
fn depth_16_ok_17_rejected() {
assert!(TypeRegistry::new(chain(MAX_DEPTH)).is_ok());
assert_eq!(kind_of(&TypeRegistry::new(chain(MAX_DEPTH + 1))), Some(SchemaErrorKind::TooDeep));
}
#[test]
fn inheritance_required_or_and_default_override() {
let parent = def("p", None, vec![field("a", FieldKind::Number, true, Some(Value::Number(1.0))), field("b", FieldKind::Text, false, None)]);
let child = def(
"c",
Some("p"),
vec![
field("a", FieldKind::Number, false, Some(Value::Number(2.0))),
field("b", FieldKind::Text, true, None),
field("d", FieldKind::Checkbox, false, None),
],
);
let registry = TypeRegistry::new(vec![parent, child]).unwrap();
let schema = registry.resolve(&["c"]).unwrap();
let keys: Vec<_> = schema.fields.iter().map(|f| f.key.as_str()).collect();
assert_eq!(keys, ["a", "b", "d"]);
assert!(schema.fields[0].required, "required stays true when only the parent requires it");
assert_eq!(schema.fields[0].default, Some(Value::Number(2.0)));
assert!(schema.fields[1].required, "child can add required");
assert_eq!(registry.resolve(&["p"]).unwrap().fields[0].default, Some(Value::Number(1.0)));
}
#[test]
fn incompatible_override_rejected() {
let parent = def("p", None, vec![field("a", FieldKind::Number, false, None)]);
let child = def("c", Some("p"), vec![field("a", FieldKind::Text, false, None)]);
assert_eq!(kind_of(&TypeRegistry::new(vec![parent, child])), Some(SchemaErrorKind::Conflict));
}
#[test]
fn multi_tag_merge_is_order_independent() {
let one = def("one", None, vec![field("k", FieldKind::Text, false, Some(s("x"))), field("only1", FieldKind::Text, true, None)]);
let two = def("two", None, vec![field("k", FieldKind::Text, true, Some(s("x"))), field("only2", FieldKind::Date, false, None)]);
let registry = TypeRegistry::new(vec![one, two]).unwrap();
let ab = registry.resolve(&["one", "two"]).unwrap();
let ba = registry.resolve(&["two", "one"]).unwrap();
assert_eq!(ab, ba);
assert_eq!(ab.types, ["one", "two"]);
assert!(ab.fields.iter().find(|f| f.key == "k").unwrap().required);
}
#[test]
fn multi_tag_collisions_are_errors() {
let kinds = TypeRegistry::new(vec![
def("a", None, vec![field("k", FieldKind::Text, false, None)]),
def("b", None, vec![field("k", FieldKind::Number, false, None)]),
])
.unwrap();
assert!(matches!(kinds.resolve(&["a", "b"]), Err(Error::Schema { kind: SchemaErrorKind::Conflict, .. })));
assert!(kinds.resolve(&["a"]).is_ok());
let defaults = TypeRegistry::new(vec![
def("a", None, vec![field("k", FieldKind::Text, false, Some(s("x")))]),
def("b", None, vec![field("k", FieldKind::Text, false, Some(s("y")))]),
])
.unwrap();
assert!(matches!(defaults.resolve(&["b", "a"]), Err(Error::Schema { kind: SchemaErrorKind::Conflict, .. })));
}
#[test]
fn tagging_parent_and_child_is_not_a_conflict() {
let parent = def("p", None, vec![field("k", FieldKind::Text, false, Some(s("x")))]);
let child = def("c", Some("p"), vec![field("k", FieldKind::Text, false, Some(s("y")))]);
let registry = TypeRegistry::new(vec![parent, child]).unwrap();
let schema = registry.resolve(&["p", "c"]).unwrap();
assert_eq!(schema.types, ["c"]);
assert_eq!(schema.fields[0].default, Some(s("y")));
}
#[test]
fn bad_typed_defaults_rejected() {
let cases = [
(FieldKind::Number, Value::Number(f64::NAN)),
(FieldKind::Number, Value::Number(f64::INFINITY)),
(FieldKind::Number, s("3")),
(FieldKind::Checkbox, s("true")),
(FieldKind::Date, s("2026-02-30")),
(FieldKind::Date, s("2026-01-02T03:04:05")),
(FieldKind::DateTime, s("2026-01-02")),
(FieldKind::Text, Value::Bool(true)),
(FieldKind::List, Value::List(vec![Value::Number(1.0)])),
(FieldKind::List, s("a")),
];
for (kind, default) in cases {
let registry = TypeRegistry::new(vec![def("a", None, vec![field("k", kind, false, Some(default.clone()))])]);
assert_eq!(kind_of(&registry), Some(SchemaErrorKind::BadDefault), "{kind:?} {default:?}");
}
let good = [
(FieldKind::Date, s("2026-02-28")),
(FieldKind::DateTime, s("2026-01-02T03:04:05")),
(FieldKind::List, Value::List(vec![s("a"), s("b")])),
(FieldKind::Checkbox, Value::Bool(false)),
];
for (kind, default) in good {
assert!(TypeRegistry::new(vec![def("a", None, vec![field("k", kind, false, Some(default))])]).is_ok());
}
}
fn book_schema() -> ResolvedSchema {
let book = def(
"book",
None,
vec![
field("author", FieldKind::Text, true, None),
field("pages", FieldKind::Number, false, None),
field("read", FieldKind::Date, false, None),
field("tags2", FieldKind::List, true, None),
],
);
TypeRegistry::new(vec![book]).unwrap().resolve(&["book"]).unwrap()
}
fn row(pairs: &[(&str, Value)]) -> Vec<(String, Value)> {
pairs.iter().map(|(key, value)| ((*key).to_owned(), value.clone())).collect()
}
#[test]
fn validate_required_missing_null_and_empty() {
let schema = book_schema();
for author in [None, Some(Value::Null), Some(s(""))] {
let mut values = vec![("tags2", Value::List(vec![s("a")]))];
if let Some(value) = author {
values.push(("author", value));
}
let diags = schema.validate(&row(&values));
assert_eq!(diags.len(), 1, "{diags:?}");
assert_eq!((diags[0].key.as_str(), diags[0].code), ("author", DiagnosticCode::Required));
}
let diags = schema.validate(&row(&[("author", s("Ann")), ("tags2", Value::List(vec![]))]));
assert_eq!(diags.len(), 1);
assert_eq!(diags[0].key, "tags2");
assert_eq!(diags[0].code.as_str(), "required");
}
#[test]
fn validate_wrong_types_and_unknown_properties() {
let schema = book_schema();
let values =
row(&[("author", s("Ann")), ("tags2", Value::List(vec![s("a")])), ("pages", s("many")), ("read", s("2026-13-01")), ("extra", Value::Bool(true))]);
let diags = schema.validate(&values);
let keys: Vec<_> = diags.iter().map(|d| (d.key.as_str(), d.code)).collect();
assert_eq!(keys, [("pages", DiagnosticCode::WrongType), ("read", DiagnosticCode::WrongType)]);
let ok =
row(&[("author", s("Ann")), ("tags2", Value::List(vec![s("a")])), ("pages", Value::Number(300.0)), ("read", s("2026-01-02")), ("extra", Value::Null)]);
assert_eq!(schema.validate(&ok), []);
assert_eq!(schema.validate(&[("pages".to_owned(), Value::Number(f64::NAN))]).iter().filter(|d| d.code == DiagnosticCode::WrongType).count(), 1);
}
+118
View File
@@ -0,0 +1,118 @@
//! The value type of the expression language.
//!
//! A [`Value`] is what a formula evaluates to and what a cell holds. The types
//! are the ones of the Bases documentation: `null`, `boolean`, `number`, `string`,
//! `date`, `duration`, `list`, `object`, `link`, `file`, `regex`, `html`, `image`
//! and `icon`. A value that failed to evaluate is an [`Value::Error`], never a
//! panic.
use crate::datetime::{Date, Duration};
/// A value of the expression language.
#[derive(Debug, Clone, PartialEq)]
pub enum Value {
/// No value at all.
Null,
/// A boolean.
Bool(bool),
/// A number, `NaN` included.
Number(f64),
/// A string.
Str(String),
/// A date, with or without a time part.
Date(Date),
/// A duration.
Duration(Duration),
/// A list of values.
List(Vec<Value>),
/// An object; the keys keep their order.
Object(Vec<(String, Value)>),
/// A link to another note.
Link {
/// The link target, as written in the note.
target: String,
/// The text shown for the link.
display: String,
},
/// A file of the vault.
File {
/// The vault-relative path of the file.
path: String,
},
/// A regular expression literal.
Regex {
/// The pattern of the expression.
pattern: String,
/// The flags of the expression, for example `i`.
flags: String,
},
/// Pre-rendered HTML.
Html(String),
/// An image reference.
Image(String),
/// The name of an icon.
Icon(String),
/// The message of a failed evaluation.
Error(String),
}
impl Value {
/// The name of the type, as `isType` reports it.
#[must_use]
pub fn type_name(&self) -> &'static str {
match self {
Value::Null => "null",
Value::Bool(_) => "boolean",
Value::Number(_) => "number",
Value::Str(_) => "string",
Value::Date(_) => "date",
Value::Duration(_) => "duration",
Value::List(_) => "list",
Value::Object(_) => "object",
Value::Link { .. } => "link",
Value::File { .. } => "file",
Value::Regex { .. } => "regex",
Value::Html(_) => "html",
Value::Image(_) => "image",
Value::Icon(_) => "icon",
Value::Error(_) => "error",
}
}
/// The value as text, for a cell or a message.
///
/// A null value is empty text, a list is its items separated by `, ` and an
/// object is its `key: value` pairs separated by `, `.
#[must_use]
pub fn to_display_string(&self) -> String {
match self {
Value::Null => String::new(),
Value::Bool(flag) => {
if *flag {
"true".to_string()
} else {
"false".to_string()
}
}
Value::Number(number) => format!("{number}"),
Value::Str(text) => text.clone(),
Value::Date(date) => date.format_iso(),
Value::Duration(duration) => duration.to_display_string(),
Value::List(items) => items.iter().map(Value::to_display_string).collect::<Vec<_>>().join(", "),
Value::Object(fields) => fields.iter().map(|(key, value)| format!("{key}: {}", value.to_display_string())).collect::<Vec<_>>().join(", "),
Value::Link { target, display } => {
if display.is_empty() {
target.clone()
} else {
display.clone()
}
}
Value::File { path } => path.clone(),
Value::Regex { pattern, flags } => format!("/{pattern}/{flags}"),
Value::Html(html) => html.clone(),
Value::Image(source) => source.clone(),
Value::Icon(name) => name.clone(),
Value::Error(message) => message.clone(),
}
}
}
+130
View File
@@ -0,0 +1,130 @@
//! Emitter: 2-space indent, quotes only when needed.
use super::Node;
use super::lex::plain;
const INDENT: usize = 2;
const INDICATORS: &str = "-?:,[]{}#&*!|>'\"%@`\\";
/// Writes the node as YAML text; `parse(&emit(n))` gives back an equal node.
#[must_use]
pub fn emit(node: &Node) -> String {
let mut out = String::new();
match node {
Node::Map(m) if !m.is_empty() => map(&mut out, m, 0),
Node::Seq(s) if !s.is_empty() => seq(&mut out, s, 0),
_ => {
value(&mut out, node, 0);
out = out.trim_start().to_string();
}
}
out
}
fn map(out: &mut String, entries: &[(String, Node)], indent: usize) {
for (k, v) in entries {
out.push_str(&" ".repeat(indent));
out.push_str(&scalar_str(k));
out.push(':');
value(out, v, indent);
}
}
fn seq(out: &mut String, items: &[Node], indent: usize) {
for item in items {
out.push_str(&" ".repeat(indent));
let mut tmp = String::new();
match item {
Node::Map(m) if !m.is_empty() => map(&mut tmp, m, indent + INDENT),
Node::Seq(s) if !s.is_empty() => seq(&mut tmp, s, indent + INDENT),
_ => {
out.push('-');
value(out, item, indent);
continue;
}
}
out.push_str("- ");
out.push_str(tmp.get(indent + INDENT..).unwrap_or(""));
}
}
/// Appends what follows `key:` or `-` for a child of an entry at `indent`.
fn value(out: &mut String, node: &Node, indent: usize) {
match node {
Node::Map(m) if m.is_empty() => out.push_str(" {}\n"),
Node::Seq(s) if s.is_empty() => out.push_str(" []\n"),
Node::Map(m) => {
out.push('\n');
map(out, m, indent + INDENT);
}
Node::Seq(s) => {
out.push('\n');
seq(out, s, indent + INDENT);
}
Node::Str(s) if s.contains('\n') && block_ok(s) => {
let body = s.trim_end_matches('\n');
let trailing = s.len() - body.len();
out.push_str(match trailing {
0 => " |-\n",
1 => " |\n",
_ => " |+\n",
});
for line in body.split('\n') {
if !line.is_empty() {
out.push_str(&" ".repeat(indent + INDENT));
out.push_str(line);
}
out.push('\n');
}
for _ in 1..trailing {
out.push('\n');
}
}
Node::Null => out.push_str(" null\n"),
Node::Bool(b) => out.push_str(&[" ", &b.to_string(), "\n"].concat()),
Node::Num(n) => out.push_str(&[" ", &n.to_string(), "\n"].concat()),
Node::Str(s) => {
out.push(' ');
out.push_str(&scalar_str(s));
out.push('\n');
}
}
}
/// A `|` block can carry the text only when re-reading cannot change it.
fn block_ok(s: &str) -> bool {
let body = s.trim_end_matches('\n');
!body.is_empty()
&& !body.starts_with([' ', '\n'])
&& body.split('\n').all(|l| !l.starts_with('\t') && (l.is_empty() || !l.trim().is_empty()) && !l.contains(char::is_control))
}
fn scalar_str(s: &str) -> String {
if !needs_quote(s) {
return s.to_string();
}
let mut out = String::from("\"");
for c in s.chars() {
match c {
'"' => out.push_str("\\\""),
'\\' => out.push_str("\\\\"),
'\n' => out.push_str("\\n"),
'\t' => out.push_str("\\t"),
'\r' => out.push_str("\\r"),
c if c.is_control() => out.extend(["\\u", &format!("{:04x}", u32::from(c))]),
c => out.push(c),
}
}
out.push('"');
out
}
fn needs_quote(s: &str) -> bool {
s.is_empty()
|| !matches!(plain(s), Node::Str(_))
|| s.starts_with(|c: char| INDICATORS.contains(c) || c.is_whitespace())
|| s.ends_with(|c: char| c.is_whitespace() || c == ':')
|| s.contains(": ")
|| s.contains(" #")
|| s.contains(char::is_control)
}
+172
View File
@@ -0,0 +1,172 @@
//! Line splitting, comment stripping, keys and scalar literals of the YAML subset.
use super::Node;
use crate::{Error, Result};
/// One source line: its number, leading space count and the rest.
#[derive(Clone, Copy)]
pub(super) struct Line<'a> {
pub no: usize,
pub indent: usize,
pub text: &'a str,
}
/// Splits the text into lines; a tab in the indentation of a non-blank line is an error.
pub(super) fn split_lines(src: &str) -> Result<Vec<Line<'_>>> {
let mut out = Vec::new();
for (i, raw) in src.lines().enumerate() {
let text = raw.trim_start_matches(' ');
if text.starts_with('\t') && !text.trim().is_empty() {
return Err(Error::Syntax { line: i + 1, column: 1, message: "tab in indentation".into() });
}
out.push(Line { no: i + 1, indent: raw.len() - text.len(), text });
}
Ok(out)
}
/// Drops a trailing `# comment`; a quote only opens a string at a value start.
pub(super) fn strip_comment(s: &str) -> &str {
let mut open: Option<char> = None;
let mut last: Option<char> = None;
let mut esc = false;
let mut prev_ws = true;
let mut it = s.char_indices().peekable();
while let Some((i, c)) = it.next() {
if let Some(q) = open {
if esc {
esc = false;
} else if q == '"' && c == '\\' {
esc = true;
} else if c == q {
if q == '\'' && it.peek().is_some_and(|&(_, n)| n == '\'') {
it.next();
} else {
open = None;
last = Some(c);
prev_ws = false;
}
}
continue;
}
if matches!(c, '"' | '\'') && last.is_none_or(|l| ":-,[{".contains(l)) {
open = Some(c);
} else if c == '#' && prev_ws {
return s.get(..i).unwrap_or(s);
}
prev_ws = c.is_whitespace();
if !prev_ws {
last = Some(c);
}
}
s
}
/// The content of a line without comment and trailing space.
pub(super) fn content(text: &str) -> &str {
strip_comment(text).trim_end()
}
/// `"- x"` gives `x`, `"-"` gives empty, anything else gives `None`.
pub(super) fn item_rest(c: &str) -> Option<&str> {
if c == "-" { Some("") } else { c.strip_prefix("- ") }
}
/// Splits `key: rest` (key may be quoted); `rest` is trimmed and a suffix of `c`.
pub(super) fn split_key(c: &str) -> Option<(String, &str)> {
if c.starts_with(['[', '{']) {
return None;
}
if c.starts_with(['"', '\'']) {
let (k, rest) = if c.starts_with('"') { unquote_dq(c).ok()? } else { unquote_sq(c).ok()? };
let r = rest.trim_start().strip_prefix(':')?;
if !(r.is_empty() || r.starts_with(' ')) {
return None;
}
return Some((k, r.trim_start()));
}
for (i, _) in c.match_indices(':') {
let after = c.get(i + 1..)?;
if after.is_empty() || after.starts_with(' ') {
let k = c.get(..i)?.trim_end();
if k.is_empty() {
return None;
}
return Some((k.to_string(), after.trim_start()));
}
}
None
}
/// Parses `"..."` at the start of `s`; returns the text and what follows the closing quote.
pub(super) fn unquote_dq(s: &str) -> std::result::Result<(String, &str), &'static str> {
let body = s.get(1..).ok_or("bad string")?;
let mut out = String::new();
let mut it = body.char_indices();
while let Some((i, c)) = it.next() {
match c {
'"' => return Ok((out, body.get(i + 1..).unwrap_or(""))),
'\\' => match it.next().map(|(_, e)| e) {
Some('n') => out.push('\n'),
Some('t') => out.push('\t'),
Some('r') => out.push('\r'),
Some('0') => out.push('\0'),
Some(e @ ('"' | '\\' | '/')) => out.push(e),
Some('u') => {
let mut code = 0u32;
for _ in 0..4 {
let d = it.next().and_then(|(_, h)| h.to_digit(16)).ok_or("bad \\u escape")?;
code = code * 16 + d;
}
out.push(char::from_u32(code).ok_or("bad \\u escape")?);
}
_ => return Err("unknown escape"),
},
_ => out.push(c),
}
}
Err("unterminated string")
}
/// Parses `'...'` at the start of `s` (`''` is a quote); same return as [`unquote_dq`].
pub(super) fn unquote_sq(s: &str) -> std::result::Result<(String, &str), &'static str> {
let body = s.get(1..).ok_or("bad string")?;
let mut out = String::new();
let mut it = body.char_indices().peekable();
while let Some((i, c)) = it.next() {
if c != '\'' {
out.push(c);
} else if it.peek().is_some_and(|&(_, n)| n == '\'') {
it.next();
out.push('\'');
} else {
return Ok((out, body.get(i + 1..).unwrap_or("")));
}
}
Err("unterminated string")
}
/// Plain scalar: `~`/`null`/empty, booleans, numbers, otherwise a string.
pub(super) fn plain(s: &str) -> Node {
match s {
"" | "~" | "null" => Node::Null,
"true" => Node::Bool(true),
"false" => Node::Bool(false),
_ if is_num(s) => s.parse::<f64>().ok().filter(|n| n.is_finite()).map_or_else(|| Node::Str(s.to_string()), Node::Num),
_ => Node::Str(s.to_string()),
}
}
fn is_num(s: &str) -> bool {
let t = s.strip_prefix(['+', '-']).unwrap_or(s);
let (m, e) = match t.split_once(['e', 'E']) {
Some((m, e)) => (m, Some(e)),
None => (t, None),
};
let digits = m.chars().filter(char::is_ascii_digit).count();
let mantissa = digits > 0 && m.chars().all(|c| c.is_ascii_digit() || c == '.') && m.matches('.').count() <= 1;
let exp = e.is_none_or(|e| {
let e = e.strip_prefix(['+', '-']).unwrap_or(e);
!e.is_empty() && e.chars().all(|c| c.is_ascii_digit())
});
mantissa && exp
}
+312
View File
@@ -0,0 +1,312 @@
//! YAML subset of `.base` files: block maps and sequences, flow `[..]`/`{..}`, quoted and
//! plain scalars, `|` and `>` block scalars, comments. No anchors, tags or multi-documents.
mod emit;
mod lex;
pub use emit::emit;
use crate::{Error, Result};
use lex::{Line, content, item_rest, plain, split_key, split_lines, unquote_dq, unquote_sq};
const MAX_INPUT: usize = 1024 * 1024;
const MAX_DEPTH: usize = 12;
const MAX_NODES: usize = 10_000;
/// A YAML value; maps keep the order of their keys.
#[derive(Debug, Clone, PartialEq)]
pub enum Node {
/// `null`, `~` or an empty value.
Null,
/// `true` or `false`.
Bool(bool),
/// Any number.
Num(f64),
/// A string.
Str(String),
/// A sequence.
Seq(Vec<Node>),
/// A map, keys in source order.
Map(Vec<(String, Node)>),
}
/// Parses the YAML subset. An empty document is [`Node::Null`].
///
/// # Errors
/// [`Error::Syntax`] with line and column, [`Error::Limit`] past 1 MiB, depth 12 or 10 000 nodes.
pub fn parse(text: &str) -> Result<Node> {
if text.len() > MAX_INPUT {
return Err(Error::Limit(format!("input larger than {MAX_INPUT} bytes")));
}
let mut p = Parser { lines: split_lines(text)?, pos: 0, depth: 0, nodes: 0, fl_line: 0, fl_col: 0, fl_len: 0 };
let Some(first) = p.peek() else { return Ok(Node::Null) };
let c = content(first.text);
let node = if item_rest(c).is_none() && split_key(c).is_none() {
// A lone scalar document.
p.pos += 1;
p.inline_value(c, first.indent, first.no, first.indent + 1)?
} else {
p.block(first.indent)?
};
match p.peek() {
Some(l) => Err(syntax(l.no, l.indent + 1, "unexpected content")),
None => Ok(node),
}
}
fn syntax(line: usize, column: usize, message: &str) -> Error {
Error::Syntax { line, column, message: message.to_string() }
}
struct Parser<'a> {
lines: Vec<Line<'a>>,
pos: usize,
depth: usize,
nodes: usize,
/// Where the flow text being parsed sits, for error columns.
fl_line: usize,
fl_col: usize,
fl_len: usize,
}
impl<'a> Parser<'a> {
/// Next line that is neither blank nor a comment.
fn peek(&mut self) -> Option<Line<'a>> {
while let Some(l) = self.lines.get(self.pos) {
if !l.text.trim().is_empty() && !l.text.starts_with('#') {
return Some(*l);
}
self.pos += 1;
}
None
}
fn count(&mut self) -> Result<()> {
self.nodes += 1;
if self.nodes > MAX_NODES {
return Err(Error::Limit(format!("more than {MAX_NODES} nodes")));
}
Ok(())
}
fn enter(&mut self) -> Result<()> {
self.depth += 1;
if self.depth > MAX_DEPTH {
return Err(Error::Limit(format!("nesting deeper than {MAX_DEPTH} levels")));
}
self.count()
}
fn leaf(&mut self, n: Node) -> Result<Node> {
self.count()?;
Ok(n)
}
fn block(&mut self, indent: usize) -> Result<Node> {
let is_seq = self.peek().is_some_and(|l| item_rest(content(l.text)).is_some());
if is_seq { self.seq(indent) } else { self.map(indent) }
}
/// The value under `key:` / `-` with nothing after it on the line.
fn nested(&mut self, indent: usize, same_indent_seq: bool) -> Result<Node> {
match self.peek() {
Some(n) if n.indent > indent => self.block(n.indent),
Some(n) if same_indent_seq && n.indent == indent && item_rest(content(n.text)).is_some() => self.seq(indent),
_ => self.leaf(Node::Null),
}
}
fn map(&mut self, indent: usize) -> Result<Node> {
self.enter()?;
let mut entries = Vec::new();
while let Some(l) = self.peek() {
if l.indent < indent {
break;
}
if l.indent > indent {
return Err(syntax(l.no, l.indent + 1, "bad indentation"));
}
let c = content(l.text);
if item_rest(c).is_some() {
break;
}
let Some((key, rest)) = split_key(c) else {
return Err(syntax(l.no, l.indent + 1, "expected `key: value`"));
};
self.pos += 1;
let v = if rest.is_empty() { self.nested(indent, true)? } else { self.inline_value(rest, indent, l.no, l.indent + 1 + c.len() - rest.len())? };
entries.push((key, v));
}
self.depth -= 1;
Ok(Node::Map(entries))
}
fn seq(&mut self, indent: usize) -> Result<Node> {
self.enter()?;
let mut items = Vec::new();
while let Some(l) = self.peek() {
if l.indent < indent {
break;
}
if l.indent > indent {
return Err(syntax(l.no, l.indent + 1, "bad indentation"));
}
let c = content(l.text);
let Some(rest) = item_rest(c) else { break };
let rest = rest.trim_start();
let off = c.len() - rest.len();
if rest.is_empty() {
self.pos += 1;
items.push(self.nested(indent, false)?);
} else if item_rest(rest).is_some() || split_key(rest).is_some() {
// `- key: v` and `- - x`: re-read the rest of the line as a line of its own.
if let Some(slot) = self.lines.get_mut(self.pos) {
*slot = Line { no: l.no, indent: indent + off, text: rest };
}
items.push(self.block(indent + off)?);
} else {
self.pos += 1;
items.push(self.inline_value(rest, indent, l.no, l.indent + 1 + off)?);
}
}
self.depth -= 1;
Ok(Node::Seq(items))
}
/// A value written on the line of its key or dash; the line is already consumed.
fn inline_value(&mut self, text: &str, parent: usize, no: usize, col: usize) -> Result<Node> {
if text.starts_with(['|', '>']) {
return self.block_scalar(text, parent, no, col);
}
if !text.starts_with(['[', '{', '"', '\'']) {
return self.leaf(plain(text));
}
(self.fl_line, self.fl_col, self.fl_len) = (no, col, text.len());
let (n, rest) = self.flow(text)?;
if !rest.trim().is_empty() {
return Err(self.ferr(rest, "unexpected text after value"));
}
Ok(n)
}
fn block_scalar(&mut self, header: &str, parent: usize, no: usize, col: usize) -> Result<Node> {
let folded = header.starts_with('>');
let chomp = header.get(1..).unwrap_or("");
if !["", "-", "+"].contains(&chomp) {
return Err(syntax(no, col, "bad block scalar header"));
}
let mut raw = Vec::new();
while let Some(l) = self.lines.get(self.pos) {
if !l.text.trim().is_empty() && l.indent <= parent {
break;
}
raw.push(*l);
self.pos += 1;
}
let base = raw.iter().find(|l| !l.text.trim().is_empty()).map_or(0, |l| l.indent);
let mut lines: Vec<String> = Vec::new();
for l in &raw {
if l.text.trim().is_empty() {
lines.push(String::new());
} else if l.indent < base {
return Err(syntax(l.no, 1, "bad indentation"));
} else {
lines.push(format!("{}{}", " ".repeat(l.indent - base), l.text));
}
}
let mut trailing = 0;
while lines.last().is_some_and(String::is_empty) {
lines.pop();
trailing += 1;
}
let mut body = String::new();
for (i, l) in lines.iter().enumerate() {
if !folded {
if i > 0 {
body.push('\n');
}
} else if l.is_empty() {
body.push('\n');
} else if !body.is_empty() && !body.ends_with('\n') {
body.push(' ');
}
body.push_str(l);
}
if !body.is_empty() && chomp != "-" {
body.push('\n');
if chomp == "+" {
body.push_str(&"\n".repeat(trailing));
}
}
self.leaf(Node::Str(body))
}
fn ferr(&self, rest: &str, msg: &str) -> Error {
syntax(self.fl_line, self.fl_col + self.fl_len.saturating_sub(rest.len()), msg)
}
/// One flow value; returns it and the text after it.
fn flow<'t>(&mut self, s: &'t str) -> Result<(Node, &'t str)> {
let s = s.trim_start();
match s.chars().next() {
Some(c @ ('[' | '{')) => {
self.enter()?;
let r = self.flow_coll(s.get(1..).unwrap_or(""), c == '{');
self.depth -= 1;
r
}
Some(q @ ('"' | '\'')) => {
let (t, r) = if q == '"' { unquote_dq(s) } else { unquote_sq(s) }.map_err(|m| self.ferr(s, m))?;
Ok((self.leaf(Node::Str(t))?, r))
}
Some(_) => {
let (t, r) = s.split_at_checked(s.find([',', ']', '}']).unwrap_or(s.len())).unwrap_or((s, ""));
Ok((self.leaf(plain(t.trim()))?, r))
}
None => Err(self.ferr(s, "unterminated flow collection")),
}
}
fn flow_coll<'t>(&mut self, mut rest: &'t str, is_map: bool) -> Result<(Node, &'t str)> {
let close = if is_map { '}' } else { ']' };
let (mut items, mut entries) = (Vec::new(), Vec::new());
loop {
rest = rest.trim_start();
if let Some(r) = rest.strip_prefix(close) {
let n = if is_map { Node::Map(entries) } else { Node::Seq(items) };
return Ok((n, r));
}
if rest.starts_with(',') {
return Err(self.ferr(rest, "empty element"));
}
if is_map {
let (key, r) = if rest.starts_with(['"', '\'']) {
if rest.starts_with('"') { unquote_dq(rest) } else { unquote_sq(rest) }.map_err(|m| self.ferr(rest, m))?
} else {
let (k, r) = rest.split_at_checked(rest.find([':', ',', '}']).unwrap_or(rest.len())).unwrap_or((rest, ""));
if k.trim().is_empty() {
return Err(self.ferr(rest, "empty key"));
}
(k.trim().to_string(), r)
};
let Some(r) = r.trim_start().strip_prefix(':') else {
return Err(self.ferr(r, "expected `:` after key"));
};
let r = r.trim_start();
let (v, r) = if r.starts_with([',', '}']) { (self.leaf(Node::Null)?, r) } else { self.flow(r)? };
entries.push((key, v));
rest = r;
} else {
let (n, r) = self.flow(rest)?;
items.push(n);
rest = r;
}
rest = rest.trim_start();
if let Some(r) = rest.strip_prefix(',') {
rest = r;
} else if !rest.starts_with(close) {
return Err(self.ferr(rest, "expected `,` or closing bracket"));
}
}
}
}
+107
View File
@@ -0,0 +1,107 @@
//! Shared fixtures of the evaluator tests: a `Vec` of rows behind `NoteSource`.
#![allow(dead_code)] // shared test helpers: each test file uses only some of them
use notescraft_bases::{Env, Formulas, NoteSource, RowRef, Value, eval};
/// 2024-12-01 12:00:00 UTC.
pub const NOW: i64 = 1_733_054_400_000;
pub struct TestRow {
pub path: String,
pub size: u64,
pub ctime: i64,
pub mtime: i64,
pub tags: Vec<String>,
pub links: Vec<String>,
pub props: Vec<(String, Value)>,
}
impl TestRow {
pub fn new(path: &str) -> Self {
Self { path: path.into(), size: 0, ctime: 0, mtime: 0, tags: vec![], links: vec![], props: vec![] }
}
pub fn prop(mut self, key: &str, value: Value) -> Self {
self.props.push((key.into(), value));
self
}
pub fn tags(mut self, tags: &[&str]) -> Self {
self.tags = tags.iter().map(|t| (*t).to_string()).collect();
self
}
pub fn links(mut self, links: &[&str]) -> Self {
self.links = links.iter().map(|t| (*t).to_string()).collect();
self
}
pub fn row(&self) -> RowRef<'_> {
RowRef {
size: self.size,
ctime_ms: self.ctime,
mtime_ms: self.mtime,
tags: &self.tags,
links: &self.links,
props: &self.props,
..RowRef::new(&self.path)
}
}
}
pub struct Rows(pub Vec<TestRow>);
impl NoteSource for Rows {
fn len(&self) -> usize {
self.0.len()
}
fn row(&self, i: usize) -> Option<RowRef<'_>> {
self.0.get(i).map(TestRow::row)
}
fn find(&self, path: &str) -> Option<RowRef<'_>> {
self.0.iter().find(|r| r.path == path).map(TestRow::row)
}
fn resolve_link(&self, target: &str, _from: &str) -> Option<String> {
let with_md = [target, ".md"].concat();
let found = self.0.iter().find(|r| r.path == target || r.path == with_md || r.row().basename() == target)?;
Some(found.path.clone())
}
}
pub fn n(x: f64) -> Value {
Value::Number(x)
}
pub fn s(x: &str) -> Value {
Value::Str(x.into())
}
pub fn list(items: Vec<Value>) -> Value {
Value::List(items)
}
/// Evaluates `src` over `row` with the formulas, `this` and the source of the arguments.
pub fn run(src: &str, row: &TestRow, formulas: &[(&str, &str)], source: Option<&Rows>, this: Option<&TestRow>) -> Value {
let expr = notescraft_bases::expr::parse_expr(src).unwrap_or_else(|e| panic!("`{src}` does not parse: {e}"));
let formulas: Vec<(String, String)> = formulas.iter().map(|(k, v)| ((*k).to_string(), (*v).to_string())).collect();
let mut env = Env::new(NOW, 7);
env.this = this.map(TestRow::row);
env.source = source.map(|r| r as &dyn NoteSource);
eval(&expr, &env, &row.row(), &Formulas::new(&formulas))
}
/// Evaluates `src` over an empty row.
pub fn ev(src: &str) -> Value {
run(src, &TestRow::new("Folder/Note.md"), &[], None, None)
}
/// Asserts every case, naming the source on a failure.
pub fn cases(table: &[(&str, Value)]) {
for (src, want) in table {
assert_eq!(&ev(src), want, "{src}");
}
}
pub fn is_error(value: &Value) -> bool {
matches!(value, Value::Error(_))
}
+312
View File
@@ -0,0 +1,312 @@
//! Evaluator core: operators, laziness, formulas and the global, any, string, number and object
//! functions (normal, null and wrong-type case for each).
mod common;
use common::{TestRow, cases, ev, is_error, list, n, run, s};
use notescraft_bases::Value;
fn errors(sources: &[&str]) {
for src in sources {
assert!(is_error(&ev(src)), "{src} gave {:?}", ev(src));
}
}
#[test]
fn operators() {
cases(&[
("1 + 2 * 3", n(7.0)),
("(1 + 2) * 3", n(9.0)),
("7 % 4", n(3.0)),
("-(2)", n(-2.0)),
("!0", Value::Bool(true)),
("\"a\" + 1", s("a1")),
("1 + \"a\"", s("1a")),
("\"a\" + \"b\"", s("ab")),
("null + 1", Value::Null),
("1 < null", Value::Bool(false)),
("null == null", Value::Bool(true)),
("null != 1", Value::Bool(true)),
("\"a\" == \"a\"", Value::Bool(true)),
("1 == \"1\"", Value::Bool(false)),
("[1, 2] == [1, 2]", Value::Bool(true)),
("\"b\" > \"a\"", Value::Bool(true)),
("2 >= 2 && 1 <= 1", Value::Bool(true)),
("false && 1 / 0", Value::Bool(false)),
("true || 1 / 0", Value::Bool(true)),
("0 || \"x\"", Value::Bool(true)),
("date(\"2024-01-01\") < \"2024-02-01\"", Value::Bool(true)),
("date(\"2024-01-01\") == \"2024-01-01\"", Value::Bool(true)),
("(now() + \"1d\") - now()", n(86_400_000.0)),
("(date(\"2024-12-01\") + \"1M 4h 3m\").toString()", s("2025-01-01 04:03:00")),
("(date(\"2024-01-31\") + \"1M\").toString()", s("2024-02-29")),
("(date(\"2024-03-01\") - \"1d\").toString()", s("2024-02-29")),
("date(\"2024-03-02\") - date(\"2024-03-01\")", n(86_400_000.0)),
("(duration(\"5h\") * 2).toString()", s("10h")),
("(duration(\"1d\") + duration(\"2h\")).toString()", s("1d 2h")),
]);
errors(&["1 / 0", "5 % 0", "2 * duration(\"5h\")", "[1] + 1", "-\"a\"", "1 / 0 + 1", "date(\"2024-01-01\") + 3", "duration(\"1d\") * 1.5"]);
}
#[test]
fn if_is_lazy() {
cases(&[
("if(true, 1, 1 / 0)", n(1.0)),
("if(false, 1 / 0, 2)", n(2.0)),
("if(false, 1)", Value::Null),
("if(\"\", 1, 2)", n(2.0)),
("if([], 1, 2)", n(2.0)),
("if([0], 1, 2)", n(1.0)),
]);
errors(&["if(1 / 0, 1, 2)", "if(true)", "if(1, 2, 3, 4)"]);
}
#[test]
fn global_functions() {
cases(&[
("now().toString()", s("2024-12-01 12:00:00")),
("today().toString()", s("2024-12-01")),
("date(\"2024-12-01 04:03:00\").toString()", s("2024-12-01 04:03:00")),
("date(null)", Value::Null),
("duration(\"1M 4h\").toString()", s("1M 4h")),
("duration(null)", Value::Null),
("number(\"3.4\")", n(3.4)),
("number(true)", n(1.0)),
("number(false)", n(0.0)),
("number(date(\"1970-01-02\"))", n(86_400_000.0)),
("number(null)", Value::Null),
("min(3, 1, 2)", n(1.0)),
("max(3, 1, 2)", n(3.0)),
("max(null, 1)", Value::Null),
("list(\"a\")", list(vec![s("a")])),
("list([1, 2])", list(vec![n(1.0), n(2.0)])),
("list(null)", list(vec![])),
("link(\"[[Note|Shown]]\")", Value::Link { target: "Note".into(), display: "Shown".into() }),
("link(\"Note\", \"X\")", Value::Link { target: "Note".into(), display: "X".into() }),
("link(file)", Value::Link { target: "Folder/Note.md".into(), display: String::new() }),
("link(null)", Value::Null),
("escapeHTML(\"<a href=\\\"x\\\">&'</a>\")", s("&lt;a href=&quot;x&quot;&gt;&amp;&#39;&lt;/a&gt;")),
("escapeHTML(null)", Value::Null),
("html(\"<b>x</b>\")", Value::Html("<b>x</b>".into())),
("image(\"a.png\")", Value::Image("a.png".into())),
("image(file)", Value::Image("Folder/Note.md".into())),
("icon(\"arrow-right\")", Value::Icon("arrow-right".into())),
]);
errors(&[
"date(\"nope\")",
"date(1)",
"duration(\"x\")",
"duration(1)",
"number(\"x\")",
"number([1])",
"min()",
"max(\"a\", 1)",
"link(1)",
"escapeHTML(1)",
"html(1)",
"icon(1)",
"now(1)",
"nosuch(1)",
]);
}
#[test]
fn random_is_seeded_and_in_range() {
let row = TestRow::new("a.md");
let one = run("random()", &row, &[], None, None);
assert_eq!(one, run("random()", &row, &[], None, None));
assert!(matches!(one, Value::Number(x) if (0.0..1.0).contains(&x)));
assert_ne!(run("random()", &TestRow::new("b.md"), &[], None, None), one);
assert_ne!(run("[random(), random()][0]", &row, &[], None, None), run("[random(), random()][1]", &row, &[], None, None));
}
#[test]
fn any_functions() {
cases(&[
("1.isTruthy()", Value::Bool(true)),
("(0).isTruthy()", Value::Bool(false)),
("\"\".isTruthy()", Value::Bool(false)),
("null.isTruthy()", Value::Bool(false)),
("[].isTruthy()", Value::Bool(false)),
("\"example\".isType(\"string\")", Value::Bool(true)),
("true.isType(\"boolean\")", Value::Bool(true)),
("null.isType(\"null\")", Value::Bool(true)),
("[1].isType(\"string\")", Value::Bool(false)),
("123.toString()", s("123")),
("(1.5).toString()", s("1.5")),
("null.toString()", s("")),
("[1, \"a\"].toString()", s("1, a")),
("true.toString()", s("true")),
]);
errors(&["1.isType(2)", "1.isTruthy(1)"]);
}
#[test]
fn string_functions() {
cases(&[
("\"hello\".length", n(5.0)),
("\"hello\".contains(\"ell\")", Value::Bool(true)),
("\"hello\".contains(\"x\")", Value::Bool(false)),
("\"hello\".containsAll(\"h\", \"e\")", Value::Bool(true)),
("\"hello\".containsAll(\"h\", \"x\")", Value::Bool(false)),
("\"hello\".containsAny(\"x\", \"y\", \"e\")", Value::Bool(true)),
("\"hello\".containsAny(\"x\")", Value::Bool(false)),
("\"hello\".endsWith(\"lo\")", Value::Bool(true)),
("\"hello\".startsWith(\"he\")", Value::Bool(true)),
("\"hello\".startsWith(\"lo\")", Value::Bool(false)),
("\"\".isEmpty()", Value::Bool(true)),
("\"Hello world\".isEmpty()", Value::Bool(false)),
("null.isEmpty()", Value::Bool(true)),
("\"HeLLo\".lower()", s("hello")),
("\"a:b:c:d\".replace(\":\", \"-\")", s("a-b-c-d")),
("\"123\".repeat(2)", s("123123")),
("\"123\".repeat(0)", s("")),
("\"hello\".reverse()", s("olleh")),
("\"hello\".slice(1, 4)", s("ell")),
("\"hello\".slice(2)", s("llo")),
("\"hello\".slice(-3)", s("llo")),
("\"hello\".slice(3, 1)", s("")),
("\"a,b,c,d\".split(\",\", 3)", list(vec![s("a"), s("b"), s("c")])),
("\"a,b\".split(\",\")", list(vec![s("a"), s("b")])),
("\"ab\".split(\"\")", list(vec![s("a"), s("b")])),
("\"hello world\".title()", s("Hello World")),
("\" hi \".trim()", s("hi")),
("null.lower()", Value::Null),
("null.trim()", Value::Null),
("\"héllo\".length", n(5.0)),
("\"abc\"[1]", s("b")),
]);
errors(&[
"\"a\".contains(1)",
"\"a\".contains()",
"\"a\".repeat(-1)",
"\"a\".repeat(\"x\")",
"\"a\".repeat(2000000)",
"\"a\".replace(1, 2)",
"\"a\".slice(\"x\")",
"\"a\".split(1)",
"\"a\".nosuch()",
"\"a\".lower(1)",
]);
}
#[test]
fn number_functions() {
cases(&[
("(-5).abs()", n(5.0)),
("(2.1).ceil()", n(3.0)),
("(2.9).floor()", n(2.0)),
("(2.5).round()", n(3.0)),
("(2.3333).round(2)", n(2.33)),
("(3.14159).toFixed(2)", s("3.14")),
("(3).toFixed(0)", s("3")),
("(5).isEmpty()", Value::Bool(false)),
("null.abs()", Value::Null),
("null.toFixed(2)", Value::Null),
]);
errors(&["(1).toFixed(-1)", "(1).toFixed(101)", "(1).toFixed(\"x\")", "(1).round(99)", "(1).abs(1)", "\"a\".abs()"]);
}
#[test]
fn object_functions() {
cases(&[
("({\"a\": 1, \"b\": 2}).keys()", list(vec![s("a"), s("b")])),
("({\"a\": 1, \"b\": 2}).values()", list(vec![n(1.0), n(2.0)])),
("({}).isEmpty()", Value::Bool(true)),
("({\"a\": 1}).isEmpty()", Value::Bool(false)),
("({\"a\": 1}).a", n(1.0)),
("({\"a\": 1}).b", Value::Null),
("({\"a\": 1})[\"a\"]", n(1.0)),
("null.keys()", Value::Null),
]);
errors(&["({}).keys(1)", "({}).nosuch()"]);
}
fn book() -> TestRow {
let mut row = TestRow::new("Books/Dune.md").prop("price", n(10.0)).prop("age", n(4.0)).prop("x y", s("spaced")).tags(&["#book"]);
row.size = 42;
row
}
#[test]
fn properties_and_file_fields() {
let row = book();
let at = |src: &str| run(src, &row, &[], None, None);
assert_eq!(at("price"), n(10.0));
assert_eq!(at("note.price"), n(10.0));
assert_eq!(at("note[\"x y\"]"), s("spaced"));
assert_eq!(at("missing"), Value::Null);
assert_eq!(at("missing + 1"), Value::Null);
assert_eq!(at("file.name"), s("Dune.md"));
assert_eq!(at("file.basename"), s("Dune"));
assert_eq!(at("file.ext"), s("md"));
assert_eq!(at("file.folder"), s("Books"));
assert_eq!(at("file.path"), s("Books/Dune.md"));
assert_eq!(at("file.size"), n(42.0));
assert_eq!(at("file.tags"), list(vec![s("#book")]));
assert!(is_error(&at("file.nosuch")));
assert!(is_error(&at("formula")));
assert_eq!(at("this"), Value::Null);
assert_eq!(at("this.file"), Value::Null);
}
#[test]
fn this_points_at_the_given_row() {
let row = book();
let this = TestRow::new("Embed/Host.md").prop("title", s("Host"));
let at = |src: &str| run(src, &row, &[], None, Some(&this));
assert_eq!(at("this.file.name"), s("Host.md"));
assert_eq!(at("this.title"), s("Host"));
assert_eq!(at("this.nothing"), Value::Null);
assert_eq!(at("this == this"), Value::Bool(true));
assert_eq!(at("file.name"), s("Dune.md"));
}
#[test]
fn doc_formulas_ppu_and_formatted_price() {
let formulas = [("formatted_price", "if(price, price.toFixed(2) + \" dollars\")"), ("ppu", "(price / age).toFixed(2)")];
let row = book();
assert_eq!(run("formula.ppu", &row, &formulas, None, None), s("2.50"));
assert_eq!(run("formula.formatted_price", &row, &formulas, None, None), s("10.00 dollars"));
assert_eq!(run("formula[\"ppu\"]", &row, &formulas, None, None), s("2.50"));
let no_price = TestRow::new("a.md").prop("age", n(4.0));
assert_eq!(run("formula.formatted_price", &no_price, &formulas, None, None), Value::Null);
assert_eq!(run("formula.ppu", &no_price, &formulas, None, None), Value::Null);
let zero_age = TestRow::new("a.md").prop("price", n(1.0)).prop("age", n(0.0));
assert_eq!(run("formula.ppu", &zero_age, &formulas, None, None), Value::Error("division by zero".into()));
}
#[test]
fn formula_cycles_unknowns_and_bad_sources() {
let formulas = [("a", "formula.b + 1"), ("b", "formula.a"), ("c", "formula.a"), ("d", "1 + 1"), ("self", "formula.self"), ("bad", "1 +")];
let row = TestRow::new("a.md");
let at = |src: &str| run(src, &row, &formulas, None, None);
for name in ["formula.a", "formula.b", "formula.self"] {
assert!(matches!(at(name), Value::Error(m) if m.contains("cycle")), "{name}");
}
assert!(matches!(at("formula.a"), Value::Error(m) if m == "formula cycle: a -> b -> a"));
assert!(is_error(&at("formula.c")));
assert_eq!(at("formula.d"), n(2.0));
assert!(matches!(at("formula.bad"), Value::Error(m) if m.contains("syntax")));
assert!(matches!(at("formula.nope"), Value::Error(m) if m.contains("unknown formula")));
}
#[test]
fn formulas_are_memoised_and_depth_is_bounded() {
let doubling: Vec<(String, String)> =
(0..60).map(|i| (format!("f{i}"), if i == 0 { "1".to_string() } else { format!("formula.f{} + formula.f{}", i - 1, i - 1) })).collect();
let borrowed: Vec<(&str, &str)> = doubling.iter().map(|(a, b)| (a.as_str(), b.as_str())).collect();
let row = TestRow::new("a.md");
assert!(matches!(run("formula.f59", &row, &borrowed, None, None), Value::Number(x) if x > 5e17));
let chain: Vec<(String, String)> = (0..200).map(|i| (format!("g{i}"), if i == 0 { "1".to_string() } else { format!("formula.g{} + 1", i - 1) })).collect();
let borrowed: Vec<(&str, &str)> = chain.iter().map(|(a, b)| (a.as_str(), b.as_str())).collect();
assert_eq!(run("formula.g50", &row, &borrowed, None, None), n(51.0));
assert!(is_error(&run("formula.g199", &row, &borrowed, None, None)));
}
#[test]
fn errors_propagate_and_never_panic() {
errors(&["(1 / 0) + 1", "[1 / 0]", "({\"a\": 1 / 0})", "(1 / 0).toString()", "\"a\".contains(1 / 0)", "max(1, 1 / 0)"]);
let deep = format!("{}1{}", "(".repeat(60), ")".repeat(60));
assert_eq!(ev(&deep), n(1.0));
}
+71
View File
@@ -0,0 +1,71 @@
//! Date fields and methods.
mod common;
use common::{NOW, TestRow, cases, ev, is_error, n, run, s};
use notescraft_bases::Value;
#[test]
fn date_fields() {
let d = "date(\"2024-12-31 23:59:58.250\")";
for (field, want) in [("year", 2024.0), ("month", 12.0), ("day", 31.0), ("hour", 23.0), ("minute", 59.0), ("second", 58.0), ("millisecond", 250.0)] {
assert_eq!(ev(&format!("{d}.{field}")), n(want), "{field}");
}
assert!(is_error(&ev(&format!("{d}.nosuch"))));
assert_eq!(ev("null.year"), Value::Null);
}
#[test]
fn date_methods() {
cases(&[
("now().date().format(\"YYYY-MM-DD HH:mm:ss\")", s("2024-12-01 00:00:00")),
("now().time()", s("12:00:00")),
("date(\"2025-05-27\").format(\"YYYY-MM-DD\")", s("2025-05-27")),
("date(\"2025-05-27 14:05:09\").format(\"HH:mm:ss\")", s("14:05:09")),
("date(\"2025-05-27\").time()", s("00:00:00")),
("date(\"2025-05-27\").format(\"dddd, MMMM Do YYYY\")", s("Tuesday, May 27th 2025")),
("date(\"2025-05-01\").format(\"ddd MMM D YY\")", s("Thu May 1 25")),
("date(\"2025-05-02\").format(\"[Year] YYYY\")", s("Year 2025")),
("date(\"1970-01-01\").format(\"d dddd\")", s("4 Thursday")),
("date(\"1969-12-31\").format(\"dddd\")", s("Wednesday")),
("date(\"2025-05-27 00:30:00\").format(\"h:mm A\")", s("12:30 AM")),
("date(\"2025-05-27 15:30:00\").format(\"hh:mm a\")", s("03:30 pm")),
("date(\"2025-05-27 15:30:00.007\").format(\"ss.SSS\")", s("00.007")),
("date(\"2025-05-27\").format(\"[open\")", s("[open")),
("date(\"2025-05-27\").format(\"\")", s("")),
("date(\"2025-05-27\").isEmpty()", Value::Bool(false)),
("null.format(\"YYYY\")", Value::Null),
("date(\"2025-05-27 04:03:00\").date().toString()", s("2025-05-27")),
]);
for src in ["date(\"2025-05-27\").format(1)", "date(\"2025-05-27\").format()", "date(\"2025-05-27\").time(1)", "date(\"2025-05-27\").nosuch()"] {
assert!(is_error(&ev(src)), "{src}");
}
}
#[test]
fn relative_reads_like_moment() {
let ago = |ms: i64| {
run(
"file.mtime.relative()",
&{
let mut r = TestRow::new("a.md");
r.mtime = NOW - ms;
r
},
&[],
None,
None,
)
};
let day = 86_400_000;
assert_eq!(ago(3 * day), s("3 days ago"));
assert_eq!(ago(day), s("a day ago"));
assert_eq!(ago(10_000), s("a few seconds ago"));
assert_eq!(ago(5 * 60_000), s("5 minutes ago"));
assert_eq!(ago(3 * 3_600_000), s("3 hours ago"));
assert_eq!(ago(60 * day), s("2 months ago"));
assert_eq!(ago(30 * day), s("a month ago"));
assert_eq!(ago(800 * day), s("2 years ago"));
assert_eq!(ago(-2 * day), s("in 2 days"));
assert_eq!(ago(-3 * 3_600_000), s("in 3 hours"));
}
+107
View File
@@ -0,0 +1,107 @@
//! File and link functions through a `NoteSource`.
mod common;
use common::{Rows, TestRow, list, n, run, s};
use notescraft_bases::Value;
fn vault() -> Rows {
Rows(vec![
TestRow::new("Books/Dune.md")
.tags(&["#book", "#scifi/space"])
.links(&["Herbert", "Missing"])
.prop("type", list(vec![Value::Link { target: "Novel".into(), display: String::new() }]))
.prop("year", n(1965.0)),
TestRow::new("Books/Deep/Emma.md")
.tags(&["book"])
.links(&["Books/Dune.md"])
.prop("author", Value::Link { target: "Herbert".into(), display: "Frank".into() }),
TestRow::new("People/Herbert.md").tags(&["#author"]).links(&["Dune#Plot"]),
TestRow::new("Types/Novel.md").prop("icon", s("book-open")),
TestRow::new("Root.md"),
])
}
fn at(src: &str, row: &str) -> Value {
let vault = vault();
let found = vault.0.iter().find(|r| r.path == row).expect("row");
run(src, found, &[], Some(&vault), None)
}
#[test]
fn has_tag_matches_nested_tags_and_any_value() {
let dune = "Books/Dune.md";
assert_eq!(at("file.hasTag(\"book\")", dune), Value::Bool(true));
assert_eq!(at("file.hasTag(\"#book\")", dune), Value::Bool(true));
assert_eq!(at("file.hasTag(\"scifi\")", dune), Value::Bool(true));
assert_eq!(at("file.hasTag(\"scifi/space\")", dune), Value::Bool(true));
assert_eq!(at("file.hasTag(\"sci\")", dune), Value::Bool(false));
assert_eq!(at("file.hasTag(\"a\", \"b\", \"book\")", dune), Value::Bool(true));
assert_eq!(at("file.hasTag()", dune), Value::Bool(false));
assert_eq!(at("file.hasTag(\"BOOK\")", "Books/Deep/Emma.md"), Value::Bool(true));
assert!(matches!(at("file.hasTag(1)", dune), Value::Error(_)));
}
#[test]
fn in_folder_matches_descendants() {
assert_eq!(at("file.inFolder(\"Books\")", "Books/Dune.md"), Value::Bool(true));
assert_eq!(at("file.inFolder(\"Books\")", "Books/Deep/Emma.md"), Value::Bool(true));
assert_eq!(at("file.inFolder(\"Books/Deep\")", "Books/Dune.md"), Value::Bool(false));
assert_eq!(at("file.inFolder(\"Book\")", "Books/Dune.md"), Value::Bool(false));
assert_eq!(at("file.inFolder(\"/\")", "Root.md"), Value::Bool(true));
assert_eq!(at("file.inFolder(\"Books/\")", "Books/Dune.md"), Value::Bool(true));
assert!(matches!(at("file.inFolder(1)", "Root.md"), Value::Error(_)));
}
#[test]
fn has_property_and_has_link() {
assert_eq!(at("file.hasProperty(\"year\")", "Books/Dune.md"), Value::Bool(true));
assert_eq!(at("file.hasProperty(\"nope\")", "Books/Dune.md"), Value::Bool(false));
assert_eq!(at("file.hasLink(\"Herbert\")", "Books/Dune.md"), Value::Bool(true));
assert_eq!(at("file.hasLink(link(\"Herbert\"))", "Books/Dune.md"), Value::Bool(true));
assert_eq!(at("file.hasLink(file(\"People/Herbert.md\"))", "Books/Dune.md"), Value::Bool(true));
assert_eq!(at("file.hasLink(\"Types/Novel.md\")", "Books/Dune.md"), Value::Bool(false));
assert_eq!(at("file.hasLink(\"Dune\")", "People/Herbert.md"), Value::Bool(true));
assert_eq!(at("file.hasLink(\"Books/Dune.md\")", "Books/Deep/Emma.md"), Value::Bool(true));
assert_eq!(at("file.hasLink(\"Missing\")", "Books/Dune.md"), Value::Bool(true));
assert!(matches!(at("file.hasLink(1)", "Root.md"), Value::Error(_)));
}
#[test]
fn as_link_and_file_constructor() {
assert_eq!(at("file.asLink()", "Root.md"), Value::Link { target: "Root.md".into(), display: String::new() });
assert_eq!(at("file.asLink(\"Home\")", "Root.md"), Value::Link { target: "Root.md".into(), display: "Home".into() });
assert_eq!(at("file(\"Herbert\")", "Root.md"), Value::File { path: "People/Herbert.md".into() });
assert_eq!(at("file(link(\"[[Novel]]\"))", "Root.md"), Value::File { path: "Types/Novel.md".into() });
assert_eq!(at("file(\"Nope\")", "Root.md"), Value::Null);
assert_eq!(at("file(null)", "Root.md"), Value::Null);
assert_eq!(at("file(file)", "Root.md"), Value::File { path: "Root.md".into() });
assert!(matches!(at("file(1)", "Root.md"), Value::Error(_)));
assert_eq!(at("file.asLink().asFile()", "Root.md"), Value::File { path: "Root.md".into() });
}
#[test]
fn link_functions_and_file_fields_of_other_files() {
assert_eq!(at("list(type)[0].asFile().properties.icon", "Books/Dune.md"), s("book-open"));
assert_eq!(at("type[0].asFile().name", "Books/Dune.md"), s("Novel.md"));
assert_eq!(at("author.asFile().file", "Books/Deep/Emma.md"), Value::File { path: "People/Herbert.md".into() });
assert_eq!(at("link(\"Nope\").asFile()", "Root.md"), Value::Null);
assert_eq!(at("author.linksTo(file(\"Books/Dune.md\"))", "Books/Deep/Emma.md"), Value::Bool(true));
assert_eq!(at("author.linksTo(file(\"Root.md\"))", "Books/Deep/Emma.md"), Value::Bool(false));
assert_eq!(at("link(\"Herbert\").linksTo(file(\"Books/Dune.md\"))", "Root.md"), Value::Bool(true));
assert_eq!(at("link(\"Herbert\").linksTo(\"Missing\")", "Root.md"), Value::Bool(false));
assert_eq!(at("link(\"Nope\").linksTo(file)", "Root.md"), Value::Bool(false));
assert_eq!(at("author == file(\"People/Herbert.md\")", "Books/Deep/Emma.md"), Value::Bool(false));
assert_eq!(at("file.links[0].asFile().basename", "Books/Dune.md"), s("Herbert"));
assert_eq!(at("[file(\"Types/Novel.md\")].map(value.size)", "Root.md"), list(vec![n(0.0)]));
}
#[test]
fn files_without_a_source_answer_false_or_null() {
let row = TestRow::new("a.md").tags(&["x"]);
assert_eq!(run("file.hasTag(\"x\")", &row, &[], None, None), Value::Bool(true));
assert_eq!(run("link(\"B\").asFile()", &row, &[], None, None), Value::Null);
assert_eq!(run("file(\"b.md\")", &row, &[], None, None), Value::File { path: "b.md".into() });
assert_eq!(run("file(\"b.md\").hasTag(\"x\")", &row, &[], None, None), Value::Bool(false));
assert_eq!(run("file(\"b.md\").name", &row, &[], None, None), Value::Null);
}
+122
View File
@@ -0,0 +1,122 @@
//! List functions, lambdas (`value index acc`), and the regexp functions.
mod common;
use common::{TestRow, cases, ev, is_error, list, n, run, s};
use notescraft_bases::Value;
fn nums(items: &[f64]) -> Value {
list(items.iter().map(|x| n(*x)).collect())
}
#[test]
fn list_functions() {
cases(&[
("[1, 2, 3].length", n(3.0)),
("[1, 2, 3].contains(2)", Value::Bool(true)),
("[1, 2, 3].contains(5)", Value::Bool(false)),
("[\"a\"].contains(\"a\")", Value::Bool(true)),
("[1, 2, 3].containsAll(2, 3)", Value::Bool(true)),
("[1, 2, 3].containsAll(2, 4)", Value::Bool(false)),
("[1, 2, 3].containsAny(3, 4)", Value::Bool(true)),
("[1, 2, 3].containsAny(4)", Value::Bool(false)),
("[1, [2, 3]].flat()", nums(&[1.0, 2.0, 3.0])),
("[1, 2, 3].isEmpty()", Value::Bool(false)),
("[].isEmpty()", Value::Bool(true)),
("[1, 2, 3].join(\",\")", s("1,2,3")),
("[\"a\", \"b\"].join(\" - \")", s("a - b")),
("[1, 2, 3].reverse()", nums(&[3.0, 2.0, 1.0])),
("[1, 2, 3, 4].slice(1, 3)", nums(&[2.0, 3.0])),
("[1, 2, 3, 4].slice(2)", nums(&[3.0, 4.0])),
("[1, 2, 3, 4].slice(-2)", nums(&[3.0, 4.0])),
("[3, 1, 2].sort()", nums(&[1.0, 2.0, 3.0])),
("[\"c\", \"a\", \"b\"].sort()", list(vec![s("a"), s("b"), s("c")])),
("[2, \"a\", null, 1].sort()", list(vec![Value::Null, n(1.0), n(2.0), s("a")])),
("[1, 2, 2, 3, 1].unique()", nums(&[1.0, 2.0, 3.0])),
("null.contains(1)", Value::Null),
("null.join(\",\")", Value::Null),
("null.isEmpty()", Value::Bool(true)),
("[1, 2, 3][0]", n(1.0)),
("[1, 2, 3][5]", Value::Null),
("[1, 2, 3][-1]", Value::Null),
]);
for src in ["[1].contains()", "[1].join(1)", "[1].slice(\"a\")", "[1].sort(1)", "[1].nosuch()", "5.flat()", "[1].reverse(1)"] {
assert!(is_error(&ev(src)), "{src}");
}
}
#[test]
fn lambdas_bind_value_index_and_acc() {
cases(&[
("[1, 2, 3, 4].filter(value > 2)", nums(&[3.0, 4.0])),
("[1, 2, 3, 4].filter(index % 2 == 0)", nums(&[1.0, 3.0])),
("[1, 2, 3, 4].map(value + 1)", nums(&[2.0, 3.0, 4.0, 5.0])),
("[10, 20].map(value + index)", nums(&[10.0, 21.0])),
("[1, 2, 3].reduce(acc + value, 0)", n(6.0)),
("[].reduce(acc + value, 7)", n(7.0)),
("[3, \"x\", 9, 4].filter(value.isType(\"number\")).reduce(if(acc == null || value > acc, value, acc), null)", n(9.0)),
("[1, 2].map([value, index])", list(vec![nums(&[1.0, 0.0]), nums(&[2.0, 1.0])])),
("[[1, 2], [3]].map(value.map(value * 2))", list(vec![nums(&[2.0, 4.0]), nums(&[6.0])])),
("[[1, 2], [3]].map(value.reduce(acc + value, 0))", nums(&[3.0, 3.0])),
("null.filter(value)", Value::Null),
("null.map(value)", Value::Null),
("[\"a\", \"\"].filter(value)", list(vec![s("a")])),
]);
for src in ["5.filter(value)", "[1].filter()", "[1].map(1 / 0)", "[1, 2].reduce(acc + value)", "[1].reduce(acc / 0, 1)", "[1].filter(1 / 0)"] {
assert!(is_error(&ev(src)), "{src}");
}
}
#[test]
fn lambda_variables_do_not_leak() {
let row = TestRow::new("a.md").prop("value", n(100.0));
assert_eq!(run("value", &row, &[], None, None), n(100.0));
assert_eq!(run("[1].map(value)", &row, &[], None, None), nums(&[1.0]));
assert_eq!(run("[1].map(note.value)", &row, &[], None, None), nums(&[100.0]));
assert_eq!(run("index", &row, &[], None, None), Value::Null);
}
#[test]
fn lambdas_can_use_formulas_and_properties() {
let row = TestRow::new("a.md").prop("tags", list(vec![s("x"), s("y")])).prop("min", n(2.0));
let formulas = [("big", "value > min")];
assert_eq!(run("[1, 2, 3].filter(value > min)", &row, &formulas, None, None), nums(&[3.0]));
assert_eq!(run("tags.map(value.upper())", &row, &formulas, None, None).type_name(), "error");
assert_eq!(run("tags.map(value + \"!\").join(\" \")", &row, &formulas, None, None), s("x! y!"));
}
#[test]
fn regexp_functions() {
cases(&[
("/abc/.matches(\"abcde\")", Value::Bool(true)),
("/^abc$/.matches(\"abcde\")", Value::Bool(false)),
("/ABC/i.matches(\"abcde\")", Value::Bool(true)),
("/ABC/.matches(\"abcde\")", Value::Bool(false)),
("/abc/.matches(null)", Value::Bool(false)),
("\"a:b:c:d\".replace(/:/, \"-\")", s("a-b:c:d")),
("\"a:b:c:d\".replace(/:/g, \"-\")", s("a-b-c-d")),
("\"John Smith\".replace(/(\\w+) (\\w+)/, \"$2, $1\")", s("Smith, John")),
("\"abc\".replace(/x/g, \"-\")", s("abc")),
("\"a,b,c,d\".split(/,/, 3)", list(vec![s("a"), s("b"), s("c")])),
("\"a1b22c\".split(/\\d+/)", list(vec![s("a"), s("b"), s("c")])),
("null.replace(/a/, \"b\")", Value::Null),
]);
for src in ["/a/.matches(1)", "/(/.matches(\"a\")", "\"a\".replace(/a/, 1)", "/a/.nosuch()", "\"a\".split(/a/, \"x\")"] {
assert!(is_error(&ev(src)), "{src}");
}
}
#[test]
fn pathological_regexes_are_bounded() {
let hostile = "a".repeat(5000);
let row = TestRow::new("a.md").prop("text", s(&format!("{hostile}!")));
let started = std::time::Instant::now();
assert_eq!(run("/(a+)+$/.matches(text)", &row, &[], None, None), Value::Bool(false));
assert_eq!(run("/(a|aa)+$/.matches(text)", &row, &[], None, None), Value::Bool(false));
assert!(started.elapsed().as_secs() < 5);
// 1 MiB of text against a big program passes the work bound of the engine: an error, not a hang
let big = "a".repeat(1024 * 1024);
let row = TestRow::new("a.md").prop("text", s(&big));
let value = run("/(?:a|b|c|d|e|f|g|h|i|j|k|l|m|n|o|p|q|r|s|t|u|v|w|x|y|z){1,50}!/.matches(text)", &row, &[], None, None);
assert!(matches!(value, Value::Bool(false) | Value::Error(_)), "{value:?}");
}
+179
View File
@@ -0,0 +1,179 @@
//! The parser of the Bases expression language.
use notescraft_bases::Error;
use notescraft_bases::expr::{BinOp, Expr, UnOp, parse_expr};
/// A name.
fn ident(name: &str) -> Expr {
Expr::Ident(name.to_string())
}
/// A member access.
fn member(base: Expr, name: &str) -> Expr {
Expr::Member(Box::new(base), name.to_string())
}
/// An index access.
fn index(base: Expr, at: Expr) -> Expr {
Expr::Index(Box::new(base), Box::new(at))
}
/// A call.
fn call(callee: Expr, args: Vec<Expr>) -> Expr {
Expr::Call { callee: Box::new(callee), args }
}
/// A binary operation.
fn binary(op: BinOp, left: Expr, right: Expr) -> Expr {
Expr::Binary(op, Box::new(left), Box::new(right))
}
/// A prefix operation.
fn unary(op: UnOp, operand: Expr) -> Expr {
Expr::Unary(op, Box::new(operand))
}
/// A string literal.
fn text(value: &str) -> Expr {
Expr::Str(value.to_string())
}
/// Parses, or fails the test with the reason.
fn parse(source: &str) -> Expr {
parse_expr(source).unwrap_or_else(|err| panic!("`{source}` must parse: {err}"))
}
#[test]
fn precedence_follows_the_spec() {
let cases: Vec<(&str, Expr)> = vec![
("a || b && c", binary(BinOp::Or, ident("a"), binary(BinOp::And, ident("b"), ident("c")))),
("a && b || c", binary(BinOp::Or, binary(BinOp::And, ident("a"), ident("b")), ident("c"))),
("a == b && c != d", binary(BinOp::And, binary(BinOp::Eq, ident("a"), ident("b")), binary(BinOp::Ne, ident("c"), ident("d")))),
("a < b == c", binary(BinOp::Eq, binary(BinOp::Lt, ident("a"), ident("b")), ident("c"))),
("a + b * c", binary(BinOp::Add, ident("a"), binary(BinOp::Mul, ident("b"), ident("c")))),
("a * b + c", binary(BinOp::Add, binary(BinOp::Mul, ident("a"), ident("b")), ident("c"))),
("-a * b", binary(BinOp::Mul, unary(UnOp::Neg, ident("a")), ident("b"))),
("!a && b", binary(BinOp::And, unary(UnOp::Not, ident("a")), ident("b"))),
("a - b - c", binary(BinOp::Sub, binary(BinOp::Sub, ident("a"), ident("b")), ident("c"))),
("a / b % c", binary(BinOp::Mod, binary(BinOp::Div, ident("a"), ident("b")), ident("c"))),
("a + b == c + d", binary(BinOp::Eq, binary(BinOp::Add, ident("a"), ident("b")), binary(BinOp::Add, ident("c"), ident("d")))),
];
for (source, expected) in cases {
assert_eq!(parse(source), expected, "{source}");
}
}
#[test]
fn member_index_and_call_chain() {
let expected = call(member(index(member(ident("a"), "b"), Expr::Num(0.0)), "c"), vec![text("x")]);
assert_eq!(parse("a.b[0].c(\"x\")"), expected);
}
#[test]
fn lists_and_objects_are_literals() {
assert_eq!(parse("[latitude, longitude]"), Expr::Array(vec![ident("latitude"), ident("longitude")]));
assert_eq!(parse("{\"k\": 1, \"j\": \"v\"}"), Expr::Object(vec![("k".to_string(), Expr::Num(1.0)), ("j".to_string(), text("v"))]));
assert_eq!(parse("[]"), Expr::Array(Vec::new()));
}
#[test]
fn comparison_and_boolean_operators() {
assert_eq!(
parse("price > 2.1 && status != \"done\""),
binary(BinOp::And, binary(BinOp::Gt, ident("price"), Expr::Num(2.1)), binary(BinOp::Ne, ident("status"), text("done")),)
);
assert_eq!(
parse("file.hasTag(\"a\") || !file.inFolder(\"x\")"),
binary(BinOp::Or, call(member(ident("file"), "hasTag"), vec![text("a")]), unary(UnOp::Not, call(member(ident("file"), "inFolder"), vec![text("x")])),)
);
}
#[test]
fn a_regex_is_a_literal_where_an_operand_is_expected() {
let regex = Expr::Regex { pattern: "ab".to_string(), flags: String::new() };
assert_eq!(parse("(/ab/)"), regex);
assert_eq!(parse("x == /ab/"), binary(BinOp::Eq, ident("x"), regex));
assert_eq!(parse("x == /ab/i"), binary(BinOp::Eq, ident("x"), Expr::Regex { pattern: "ab".to_string(), flags: "i".to_string() }));
}
#[test]
fn a_slash_after_an_operand_is_a_division() {
assert_eq!(parse("a / b"), binary(BinOp::Div, ident("a"), ident("b")));
assert_eq!(parse("a / b / c"), binary(BinOp::Div, binary(BinOp::Div, ident("a"), ident("b")), ident("c")));
}
#[test]
fn strings_know_their_escapes() {
assert_eq!(parse("\"a\\nb\""), text("a\nb"));
assert_eq!(parse("'a\\'b'"), text("a'b"));
assert_eq!(parse("\"a\\\\b\""), text("a\\b"));
}
#[test]
fn an_unterminated_string_reports_its_position() {
let err = parse_expr("\"abc").expect_err("the string is not closed");
match err {
Error::Syntax { line, column, message } => {
assert_eq!(line, 1);
assert_eq!(column, 1);
assert!(message.contains("not closed"), "{message}");
}
other @ (Error::Limit(_) | Error::Invalid(_) | Error::Schema { .. }) => panic!("unexpected error: {other:?}"),
}
}
#[test]
fn a_deep_expression_is_refused() {
let source = format!("{}x{}", "(".repeat(65), ")".repeat(65));
let err = parse_expr(&source).expect_err("65 levels are too deep");
assert!(matches!(err, Error::Limit(_)), "unexpected error: {err:?}");
let source = format!("{}x{}", "(".repeat(64), ")".repeat(64));
assert!(parse_expr(&source).is_ok(), "64 levels are allowed");
}
#[test]
fn a_long_expression_is_refused() {
let source = "a".repeat(4097);
let err = parse_expr(&source).expect_err("4097 bytes are too many");
assert!(matches!(err, Error::Limit(_)), "unexpected error: {err:?}");
}
#[test]
fn deps_collect_the_namespaces() {
let expr = parse("note.title + file.size + formula.total + this.file.ctime + price");
let deps = expr.deps();
assert_eq!(deps.note, vec!["price".to_string(), "title".to_string()]);
assert_eq!(deps.file, vec!["size".to_string()]);
assert_eq!(deps.formula, vec!["total".to_string()]);
assert!(deps.uses_this);
}
#[test]
fn deps_keep_the_member_path() {
let deps = parse("note.meta.author").deps();
assert_eq!(deps.note, vec!["meta.author".to_string()]);
assert_eq!(deps.file, Vec::<String>::new());
assert_eq!(deps.formula, Vec::<String>::new());
assert!(!deps.uses_this);
}
/// The next value of a small linear congruential generator.
fn next_u64(state: &mut u64) -> u64 {
*state = state.wrapping_mul(6_364_136_223_846_793_005).wrapping_add(1_442_695_040_888_963_407);
*state >> 33
}
#[test]
fn random_sources_never_panic() {
let mut state = 0x2545_f491_4f6c_dd1d_u64;
for _ in 0..5000 {
let len = usize::try_from(next_u64(&mut state) % 24).unwrap_or(0);
let mut bytes = Vec::new();
for _ in 0..len {
bytes.push(u8::try_from(next_u64(&mut state) % 256).unwrap_or(0));
}
let source = String::from_utf8_lossy(&bytes);
drop(parse_expr(&source));
}
}
+42
View File
@@ -0,0 +1,42 @@
filters:
or:
- file.hasTag("tag")
- and:
- file.hasTag("book")
- file.hasLink("Textbook")
- not:
- file.hasTag("book")
- file.inFolder("Required Reading")
formulas:
formatted_price: 'if(price, price.toFixed(2) + " dollars")'
ppu: "(price / age).toFixed(2)"
properties:
status:
displayName: Status
formula.formatted_price:
displayName: "Price"
file.ext:
displayName: Extension
summaries:
customAverage: 'values.mean().round(3)'
views:
- type: table
name: "My table"
limit: 10
groupBy:
property: note.age
direction: DESC
filters:
and:
- 'status != "done"'
- or:
- "formula.ppu > 5"
- "price > 2.1"
order:
- file.name
- file.ext
- note.age
- formula.ppu
- formula.formatted_price
summaries:
formula.ppu: Average
+15
View File
@@ -0,0 +1,15 @@
# Simple filter:
filters:
and:
- file.hasTag("tag")
# Complex filter:
filters:
or:
- file.hasTag("tag")
- and:
- file.hasTag("book")
- file.hasLink("Textbook")
- not:
- file.hasTag("book")
- file.inFolder("Required Reading")
+3
View File
@@ -0,0 +1,3 @@
formulas:
formatted_price: 'if(price, price.toFixed(2) + " dollars")'
ppu: "(price / age).toFixed(2)"
+7
View File
@@ -0,0 +1,7 @@
properties:
status:
displayName: Status
formula.formatted_price:
displayName: "Price"
file.ext:
displayName: Extension
+2
View File
@@ -0,0 +1,2 @@
summaries:
customAverage: 'values.mean().round(3)'
+21
View File
@@ -0,0 +1,21 @@
views:
- type: table
name: "My table"
limit: 10
groupBy:
property: note.age
direction: DESC
filters:
and:
- 'status != "done"'
- or:
- "formula.ppu > 5"
- "price > 2.1"
order:
- file.name
- file.ext
- note.age
- formula.ppu
- formula.formatted_price
summaries:
formula.ppu: Average
+10
View File
@@ -0,0 +1,10 @@
views:
- type: kanban
name: Projects
groupBy:
property: note.status
direction: ASC
groupOrder:
- Planned
- In progress
- Done
+7
View File
@@ -0,0 +1,7 @@
coordinates:
- "48.85837"
- "2.294481"
icon: "landmark"
color: "red"
tags:
- places
+7
View File
@@ -0,0 +1,7 @@
# Text property
coordinates: "lat, lng"
# List property
coordinates:
- "lat"
- "lng"
+224
View File
@@ -0,0 +1,224 @@
//! The map data model: projection, tiles, clustering, colours and coordinates.
use std::time::Instant;
use notescraft_bases::Value;
use notescraft_bases::map::{
Attribution, Color, LatLng, Marker, cluster, fit, parse_color, parse_coordinates, project, skipped, tile_of, tiles_for_viewport, unproject, valid,
};
/// A small pseudo-random generator, so the tests do not need a dependency.
struct Xorshift(u64);
impl Xorshift {
fn new(seed: u64) -> Self {
Self(seed)
}
fn next_u32(&mut self) -> u32 {
let mut state = self.0;
state ^= state << 13;
state ^= state >> 7;
state ^= state << 17;
self.0 = state;
u32::try_from(state >> 32).unwrap_or(u32::MAX)
}
fn next_f64(&mut self) -> f64 {
f64::from(self.next_u32()) / 4_294_967_296.0
}
}
const PARIS: LatLng = LatLng { lat: 48.858_37, lng: 2.294_481 };
#[test]
fn paris_is_tile_518_352_at_zoom_ten() {
let tile = tile_of(PARIS, 10);
assert_eq!(tile.z, 10);
assert_eq!(tile.x, 518);
assert_eq!(tile.y, 352);
}
#[test]
fn paris_is_on_the_known_tiles_of_the_other_zooms() {
let whole_world = tile_of(PARIS, 0);
assert_eq!((whole_world.x, whole_world.y), (0, 0));
let streets = tile_of(PARIS, 18);
assert_eq!((streets.x, streets.y), (132_742, 90_183));
}
#[test]
fn project_and_unproject_round_trip() {
let mut random = Xorshift::new(0x5eed);
for zoom in [0u8, 1, 5, 10, 15, 18, 22] {
for _ in 0..30 {
let lat = (random.next_f64() * 170.0) - 85.0;
let lng = (random.next_f64() * 360.0) - 180.0;
let point = LatLng { lat, lng };
let (x, y) = project(point, zoom);
let back = unproject(x, y, zoom);
assert!((back.lat - point.lat).abs() < 1e-9, "lat {lat} at zoom {zoom} came back as {}", back.lat);
assert!((back.lng - point.lng).abs() < 1e-9, "lng {lng} at zoom {zoom} came back as {}", back.lng);
}
}
}
#[test]
fn invalid_positions_are_rejected() {
assert!(!valid(LatLng { lat: 86.0, lng: 0.0 }));
assert!(!valid(LatLng { lat: 0.0, lng: 180.5 }));
assert!(!valid(LatLng { lat: f64::NAN, lng: 0.0 }));
assert!(!valid(LatLng { lat: 0.0, lng: f64::INFINITY }));
assert!(valid(LatLng { lat: 85.0, lng: 180.0 }));
}
#[test]
fn the_viewport_wraps_around_the_antimeridian() {
let tiles = tiles_for_viewport(LatLng { lat: 0.0, lng: 179.9 }, 2, 1024.0, 512.0);
assert_ne!(tiles.len(), 0);
assert!(tiles.len() <= 256);
let columns: Vec<u32> = tiles.iter().map(|tile| tile.x).collect();
assert!(columns.contains(&0), "the wrapped column is missing: {columns:?}");
assert!(columns.contains(&3), "the last column is missing: {columns:?}");
for tile in &tiles {
assert_eq!(tile.z, 2);
assert!(tile.x < 4);
assert!(tile.y < 4);
}
}
#[test]
fn the_viewport_holds_the_tile_of_its_centre() {
let tiles = tiles_for_viewport(PARIS, 10, 512.0, 512.0);
assert!(tiles.contains(&tile_of(PARIS, 10)), "the tile of the centre is missing: {tiles:?}");
for tile in &tiles {
assert_eq!(tile.z, 10);
}
let single = tiles_for_viewport(PARIS, 10, 0.0, 0.0);
assert_eq!(single, vec![tile_of(PARIS, 10)]);
}
#[test]
fn clustering_gathers_the_markers_and_keeps_every_one() {
let mut random = Xorshift::new(0x00c0_ffee);
let markers: Vec<Marker> =
(0..360).map(|id| Marker { id, at: LatLng { lat: (random.next_f64() * 160.0) - 80.0, lng: (random.next_f64() * 360.0) - 180.0 } }).collect();
let started = Instant::now();
let clusters = cluster(&markers, 8, 40.0);
let elapsed = started.elapsed();
assert!(elapsed.as_millis() < 50, "the clustering took {elapsed:?}");
let counted: usize = clusters.iter().map(|group| group.ids.len()).sum();
assert_eq!(counted, markers.len());
let mut identifiers: Vec<usize> = clusters.iter().flat_map(|group| group.ids.clone()).collect();
identifiers.sort_unstable();
assert_eq!(identifiers, (0..360).collect::<Vec<usize>>());
for group in &clusters {
assert_ne!(group.ids.len(), 0);
}
}
#[test]
fn clustering_of_one_marker_gives_one_cluster() {
let clusters = cluster(&[Marker { id: 7, at: PARIS }], 10, 40.0);
assert_eq!(clusters.len(), 1);
assert_eq!(clusters[0].ids, vec![7]);
assert!((clusters[0].center.lat - PARIS.lat).abs() < 1e-6);
assert!((clusters[0].center.lng - PARIS.lng).abs() < 1e-6);
}
#[test]
fn fit_shows_two_points_together() {
let points = [LatLng { lat: 48.85, lng: 2.29 }, LatLng { lat: 48.86, lng: 2.30 }];
let width = 800.0;
let height = 600.0;
let padding = 20.0;
let (centre, zoom) = fit(&points, width, height, padding).expect("two points fit");
assert!((centre.lat - 48.855).abs() < 1e-9);
assert!((centre.lng - 2.295).abs() < 1e-9);
assert!(zoom <= 18, "the zoom {zoom} is above the limit");
let (span_x, span_y) = span_at(&points, zoom);
assert!(span_x <= width - 2.0 * padding, "the points do not fit sideways at zoom {zoom}");
assert!(span_y <= height - 2.0 * padding, "the points do not fit upright at zoom {zoom}");
if zoom < 18 {
let (wider_x, wider_y) = span_at(&points, zoom + 1);
assert!(wider_x > width - 2.0 * padding || wider_y > height - 2.0 * padding, "zoom {} fits as well as {zoom}", zoom + 1);
}
}
#[test]
fn fit_of_one_point_uses_zoom_fifteen() {
let (centre, zoom) = fit(&[PARIS], 800.0, 600.0, 20.0).expect("one point fits");
assert_eq!(zoom, 15);
assert!((centre.lat - PARIS.lat).abs() < 1e-12);
assert!((centre.lng - PARIS.lng).abs() < 1e-12);
}
#[test]
fn fit_of_nothing_is_nothing() {
assert!(fit(&[], 800.0, 600.0, 20.0).is_none());
}
/// The sideways and upright span of the points at a zoom level, in pixels.
fn span_at(points: &[LatLng], zoom: u8) -> (f64, f64) {
let mut min_x = f64::INFINITY;
let mut max_x = f64::NEG_INFINITY;
let mut min_y = f64::INFINITY;
let mut max_y = f64::NEG_INFINITY;
for point in points {
let (x, y) = project(*point, zoom);
min_x = min_x.min(x);
max_x = max_x.max(x);
min_y = min_y.min(y);
max_y = max_y.max(y);
}
(max_x - min_x, max_y - min_y)
}
#[test]
fn coordinates_are_read_from_text_and_lists() {
let from_text = parse_coordinates(&Value::Str("48.85, 2.29".to_string())).expect("the text holds a position");
assert!((from_text.lat - 48.85).abs() < 1e-12);
assert!((from_text.lng - 2.29).abs() < 1e-12);
let from_numbers = parse_coordinates(&Value::List(vec![Value::Number(48.85), Value::Number(2.29)])).expect("the list holds a position");
assert!((from_numbers.lat - 48.85).abs() < 1e-12);
let from_strings =
parse_coordinates(&Value::List(vec![Value::Str("48.85".to_string()), Value::Str("2.29".to_string())])).expect("the list holds a position");
assert!((from_strings.lng - 2.29).abs() < 1e-12);
assert!(parse_coordinates(&Value::Str("not a position".to_string())).is_none());
assert!(parse_coordinates(&Value::List(vec![Value::Number(48.85)])).is_none());
assert!(parse_coordinates(&Value::Number(1.0)).is_none());
assert!(parse_coordinates(&Value::Null).is_none());
assert!(parse_coordinates(&Value::Str("86, 2.29".to_string())).is_none());
}
#[test]
fn skipped_counts_the_positions_that_are_missing() {
let points = [Some(PARIS), None, Some(PARIS), None];
assert_eq!(skipped(&points), 2);
assert_eq!(skipped(&[]), 0);
}
#[test]
fn colours_are_read_from_css() {
assert_eq!(parse_color("#f00"), Some(Color::Rgb(255, 0, 0)));
assert_eq!(parse_color("#00FF80"), Some(Color::Rgb(0, 255, 128)));
assert_eq!(parse_color("rgb(1, 2, 3)"), Some(Color::Rgb(1, 2, 3)));
assert_eq!(parse_color("red"), Some(Color::Rgb(255, 0, 0)));
assert_eq!(parse_color("navy"), Some(Color::Rgb(0, 0, 128)));
assert_eq!(parse_color("var(--color-accent)"), Some(Color::Token("color-accent".to_string())));
assert!(parse_color("#ff00").is_none());
assert!(parse_color("#gggggg").is_none());
assert!(parse_color("rgb(1, 2)").is_none());
assert!(parse_color("rgb(300, 0, 0)").is_none());
assert!(parse_color("mauve").is_none());
assert!(parse_color("var(--)").is_none());
}
#[test]
fn the_tiles_need_their_credit() {
assert!(Attribution::OSM_ATTRIBUTION.contains("OpenStreetMap"));
assert!(Attribution::OSM_TILE_URL.contains("{z}/{x}/{y}"));
}
+86
View File
@@ -0,0 +1,86 @@
//! The `.base` model: parse, write, unknown keys, limits, invalid values.
use notescraft_bases::{Base, Direction, Error, Filter, ViewKind};
const DOC: &str = include_str!("fixtures/Bases_syntax_1.yaml");
#[test]
fn docs_example_round_trips_byte_equal_once_normalised() {
let base = Base::parse(DOC).unwrap_or_else(|e| panic!("{e}"));
let once = base.to_yaml();
let again = Base::parse(&once).unwrap_or_else(|e| panic!("{e}"));
assert_eq!(again, base);
assert_eq!(again.to_yaml(), once);
let view = &base.views()[0];
assert_eq!(view.kind, ViewKind::Table);
assert_eq!(view.name.as_deref(), Some("My table"));
assert_eq!(view.limit, Some(10));
assert_eq!(view.group_by.as_ref().map(|g| g.direction), Some(Direction::Desc));
assert_eq!(base.formulas.len(), 2);
assert!(matches!(&base.filters, Some(Filter::Or(m)) if m.len() == 3));
assert_eq!(base.properties[1].1.display_name.as_deref(), Some("Price"));
}
#[test]
fn canonical_text_is_byte_equal() {
let text = "filters:\n and:\n - \"a > 1\"\n - not:\n - \"b\"\nformulas:\n f: \"a + 1\"\nviews:\n - type: table\n name: T\n limit: 5\n sort:\n - property: a\n direction: DESC\n";
let text = text.replace("\"a > 1\"", "a > 1").replace("- \"b\"", "- b").replace("\"a + 1\"", "a + 1");
let base = Base::parse(&text).unwrap_or_else(|e| panic!("{e}"));
assert_eq!(base.to_yaml(), text);
}
#[test]
fn unknown_keys_and_view_types_survive_in_order() {
let src = "zeta: 1\nviews:\n - type: kanban\n columnSize: 3\n name: K\n weird:\n - x\nalpha: [1, 2]\n";
let base = Base::parse(src).unwrap_or_else(|e| panic!("{e}"));
assert_eq!(base.views()[0].kind, ViewKind::Other("kanban".into()));
assert_eq!(base.views()[0].extra.len(), 2);
assert!(base.views()[0].extra_get("columnSize").is_some());
let out = base.to_yaml();
let pos = |k: &str| out.find(k).unwrap_or(usize::MAX);
assert!(pos("zeta") < pos("views") && pos("views") < pos("alpha"));
assert!(pos("columnSize") < pos("name:") && pos("name:") < pos("weird"));
assert_eq!(Base::parse(&out).unwrap_or_else(|e| panic!("{e}")), base);
}
#[test]
fn limits_are_refused() {
let views: String = (0..101).map(|_| " - type: table\n").collect();
assert!(matches!(Base::parse(&format!("views:\n{views}")), Err(Error::Limit(_))));
let formulas: String = (0..201).fold(String::new(), |acc, i| acc + " f" + &i.to_string() + ": x\n");
assert!(matches!(Base::parse(&format!("formulas:\n{formulas}")), Err(Error::Limit(_))));
let long = "x".repeat(4097);
assert!(matches!(Base::parse(&format!("formulas:\n f: {long}\n")), Err(Error::Limit(_))));
assert!(matches!(Base::parse(&format!("filters: {long}\n")), Err(Error::Limit(_))));
}
#[test]
fn invalid_values_are_errors() {
for bad in [
"views:\n - type: table\n limit: -1\n",
"views:\n - type: table\n limit: 2.5\n",
"views:\n - type: table\n limit: many\n",
"views:\n - type: table\n sort:\n - property: a\n direction: SIDEWAYS\n",
"views:\n - type: table\n groupBy:\n property: a\n direction: up\n",
"views:\n - name: no type\n",
"filters:\n xor:\n - a\n",
"views: 3\n",
"[1, 2]\n",
] {
assert!(matches!(Base::parse(bad), Err(Error::Invalid(_))), "{bad}");
}
assert!(matches!(Base::parse("a:\n\tb: 1\n"), Err(Error::Syntax { .. })));
}
#[test]
fn missing_views_gives_one_default_table_not_written_until_edited() {
let mut base = Base::parse("formulas:\n f: a\n").unwrap_or_else(|e| panic!("{e}"));
assert_eq!(base.views().len(), 1);
assert_eq!(base.views()[0].kind, ViewKind::Table);
assert!(!base.to_yaml().contains("views"));
base.views_mut()[0].limit = Some(3);
assert!(base.to_yaml().contains("views:"));
let empty = Base::parse("").unwrap_or_else(|e| panic!("{e}"));
assert_eq!(empty.views().len(), 1);
assert!(!empty.to_yaml().contains("views"));
}
+293
View File
@@ -0,0 +1,293 @@
//! The query pipeline: filter, sort, group, limit, summaries, laziness and speed.
mod common;
use std::cell::Cell;
use std::time::Instant;
use common::{NOW, Rows, TestRow, n, s};
use notescraft_bases::{Base, Env, NoteSource, QueryResult, RowRef, Value, run};
const BOOKS: [(&str, Option<&str>, f64, f64, bool); 12] = [
("Alpha", Some("sf"), 2001.0, 300.0, true),
("Bravo", Some("sf"), 1999.0, 150.0, false),
("Charlie", Some("fantasy"), 2010.0, 400.0, true),
("Delta", Some("fantasy"), 2010.0, 90.0, false),
("Echo", None, 2005.0, 200.0, true),
("Foxtrot", Some("sf"), 2015.0, 250.0, true),
("Golf", Some("skip"), 2000.0, 500.0, true),
("Hotel", Some("fantasy"), 1990.0, 120.0, false),
("India", None, 1980.0, 110.0, false),
("Juliet", Some("sf"), 2001.0, 180.0, true),
("Kilo", Some("fantasy"), 2010.0, 310.0, true),
("Lima", Some("sf"), 1999.0, 101.0, false),
];
fn books() -> Rows {
Rows(
BOOKS
.iter()
.map(|(name, genre, year, pages, read)| {
let row = TestRow::new(&format!("Books/{name}.md")).prop("year", n(*year)).prop("pages", n(*pages)).prop("read", Value::Bool(*read));
match genre {
Some(g) => row.prop("genre", s(g)),
None => row,
}
})
.collect(),
)
}
const BASE: &str = "
filters:
and:
- pages > 100
summaries:
customAverage: 'values.mean().round(3)'
formulas:
bad: nope(1)
half: pages / 2
views:
- type: table
name: Main
filters:
not:
- genre == \"skip\"
groupBy:
property: genre
direction: ASC
sort:
- property: year
direction: DESC
order:
- file.name
- year
- pages
summaries:
pages: Sum
year: customAverage
";
fn query<'a>(base: &'a Base, view: impl Into<notescraft_bases::ViewSel<'a>>, rows: &'a Rows) -> QueryResult<'a> {
run(base, view, rows, &Env::new(NOW, 7)).unwrap_or_else(|e| panic!("{e}"))
}
fn base(text: &str) -> Base {
Base::parse(text).unwrap_or_else(|e| panic!("{e}"))
}
/// `key: name name ...` per group.
fn table(r: &QueryResult<'_>) -> Vec<String> {
r.groups
.iter()
.map(|g| {
let names: Vec<String> = g.rows.iter().map(|id| r.cell(*id, 0).to_display_string().replace("Books/", "").replace(".md", "")).collect();
format!("{}: {}", g.key.to_display_string(), names.join(" "))
})
.collect()
}
#[test]
fn filter_sort_group_summary_golden() {
let (rows, b) = (books(), base(BASE));
let r = query(&b, "Main", &rows);
assert_eq!(r.total, 10);
assert_eq!(table(&r), [": Echo India", "fantasy: Charlie Kilo Hotel", "sf: Foxtrot Alpha Juliet Bravo Lima"]);
assert_eq!(r.columns.iter().map(|c| c.name.as_str()).collect::<Vec<_>>(), ["name", "year", "pages"]);
let sums: Vec<_> = r.groups.iter().map(|g| g.summaries.clone()).collect();
assert_eq!(sums[0], [Value::Null, n(1992.5), n(310.0)]);
assert_eq!(sums[1], [Value::Null, n(2003.333), n(830.0)]);
assert_eq!(sums[2][2], n(981.0));
assert_eq!(r.summaries, [Value::Null, n(2001.0), n(2121.0)]);
assert_eq!(r.errors(), 0);
}
#[test]
fn group_order_hides_and_places_null() {
let (rows, b) = (books(), base(&BASE.replace(" sort:", " groupOrder:\n - sf\n - null\n sort:")));
assert_eq!(table(&query(&b, 0, &rows)), ["sf: Foxtrot Alpha Juliet Bravo Lima", ": Echo India"]);
}
#[test]
fn desc_groups_limit_and_default_column() {
let rows = books();
let b = base(
"views:\n - type: table\n limit: 3\n groupBy:\n property: genre\n direction: DESC\n sort:\n - property: pages\n direction: ASC\n",
);
let r = query(&b, 0, &rows);
assert_eq!(r.total, 12);
assert_eq!(r.columns.len(), 1);
assert_eq!(r.columns[0].id, "file.name");
assert_eq!(table(&r), ["skip: Golf", "sf: Lima Bravo"]);
}
#[test]
fn multi_key_sort_is_stable_and_missing_view_is_an_error() {
let rows = books();
let b = base(
"views:\n - type: table\n filters: genre == \"sf\"\n sort:\n - property: year\n direction: ASC\n - property: pages\n direction: DESC\n",
);
let r = query(&b, 0, &rows);
assert_eq!(table(&r), [": Bravo Lima Alpha Juliet Foxtrot"]);
assert!(run(&b, 5, &rows, &Env::new(NOW, 7)).is_err());
assert!(run(&b, "nope", &rows, &Env::new(NOW, 7)).is_err());
}
#[test]
fn default_summaries() {
let rows = books();
let b = base(
"views:\n - type: table\n order: [pages, read, genre, year]\n summaries:\n pages: Median\n read: Checked\n genre: Unique\n year: Range\n",
);
let r = query(&b, 0, &rows);
assert_eq!(r.summaries[0], n(190.0)); // median of 12 values: (180 + 200) / 2
assert_eq!(r.summaries[1], n(7.0));
assert_eq!(r.summaries[2], n(4.0)); // null, fantasy, sf, skip
assert_eq!(r.summaries[3], n(35.0));
for (name, want) in [("Min", n(90.0)), ("Max", n(500.0)), ("Unchecked", n(5.0)), ("Filled", n(12.0)), ("Empty", n(0.0)), ("Nope", Value::Null)] {
let col = if name == "Unchecked" { "read" } else { "pages" };
let b = base(&format!("views:\n - type: table\n order: [{col}]\n summaries:\n {col}: {name}\n"));
let got = query(&b, 0, &rows).summaries[0].clone();
if name == "Nope" {
assert!(matches!(got, Value::Error(_)));
} else {
assert_eq!(got, want, "{name}");
}
}
let b = base("views:\n - type: table\n order: [genre, pages]\n summaries:\n genre: Empty\n pages: Stddev\n");
let r = query(&b, 0, &rows);
assert_eq!(r.summaries[0], n(2.0));
assert!(matches!(&r.summaries[1], Value::Number(x) if (x - 123.0).abs() < 50.0));
}
#[test]
fn date_summaries() {
let rows = Rows(vec![
TestRow::new("a.md").prop("d", Value::Date(notescraft_bases::Date::new(0, false))),
TestRow::new("b.md").prop("d", Value::Date(notescraft_bases::Date::new(86_400_000, false))),
]);
let b = base("views:\n - type: table\n order: [d]\n summaries:\n d: Range\n");
assert_eq!(query(&b, 0, &rows).summaries[0].to_display_string(), "1d");
let b = base("views:\n - type: table\n order: [d]\n summaries:\n d: Latest\n");
assert_eq!(query(&b, 0, &rows).summaries[0], Value::Date(notescraft_bases::Date::new(86_400_000, false)));
}
#[test]
fn errors_are_counted_and_do_not_stop_the_table() {
let rows = books();
let b = base(&format!("{BASE}\n").replace(" - pages\n summaries", " - formula.bad\n - formula.half\n summaries"));
let r = query(&b, 0, &rows);
let id = r.groups[0].rows[0];
assert!(matches!(r.cell(id, 2), Value::Error(_)));
assert_eq!(r.cell(id, 3), &n(100.0));
assert!(r.errors() >= 1);
let b = base("filters: nope(1)\nviews:\n - type: table\n");
let r = query(&b, 0, &rows);
assert_eq!(r.total, 0);
assert_eq!(r.errors(), 12);
}
#[test]
fn display_name_and_formula_dependencies() {
let rows = books();
let b = base(
"properties:\n year:\n displayName: Year\nformulas:\n half: pages / 2\nviews:\n - type: table\n order: [year, formula.half]\n sort:\n - property: file.name\n direction: ASC\n",
);
let r = query(&b, 0, &rows);
assert_eq!(r.columns[0].name, "Year");
assert_eq!(r.columns[1].name, "half");
assert_eq!(r.reads.note, ["pages", "year"]);
assert_eq!(r.reads.file, ["name"]);
assert!(!r.reads.uses_this);
}
struct Counted {
rows: Rows,
backlinks: Box<dyn Fn() -> Vec<String>>,
}
impl NoteSource for Counted {
fn len(&self) -> usize {
self.rows.len()
}
fn row(&self, i: usize) -> Option<RowRef<'_>> {
self.rows.row(i).map(|r| RowRef { backlinks: Some(&*self.backlinks), ..r })
}
fn find(&self, path: &str) -> Option<RowRef<'_>> {
self.rows.find(path)
}
}
#[test]
fn unreferenced_properties_are_never_read() {
let calls = std::rc::Rc::new(Cell::new(0));
let c = calls.clone();
let src = Counted {
rows: books(),
backlinks: Box::new(move || {
c.set(c.get() + 1);
vec![]
}),
};
let b = base(
"filters: year > 1990\nviews:\n - type: table\n order: [file.name]\n sort:\n - property: year\n direction: ASC\n - type: table\n order: [file.backlinks.length]\n",
);
let env = Env::new(NOW, 7);
let r = run(&b, 0, &src, &env).unwrap_or_else(|e| panic!("{e}"));
for id in r.rows() {
let _ = r.cell(id, 0);
}
assert_eq!(calls.get(), 0);
let r = run(&b, 1, &src, &env).unwrap_or_else(|e| panic!("{e}"));
assert_eq!(r.cells_computed(), 0);
assert_eq!(r.cell(0, 0), &n(0.0));
assert_eq!(calls.get(), 1);
assert_eq!(r.reads.file, ["backlinks.length"]);
}
fn synthetic(count: usize) -> Rows {
Rows(
(0..count)
.map(|i| {
TestRow::new(&format!("Notes/n{i}.md"))
.prop("year", n(f64::from(u16::try_from(i % 60).unwrap_or(0)) + 1970.0))
.prop("rank", n(f64::from(u32::try_from(i.wrapping_mul(7919) % 10_007).unwrap_or(0))))
})
.collect(),
)
}
const PERF_BASE: &str =
"views:\n - type: table\n filters: year >= 1990\n order: [file.name, year]\n sort:\n - property: rank\n direction: DESC\n";
#[test]
fn no_cell_is_computed_for_rows_not_asked_for() {
let rows = synthetic(15_000);
let b = base(PERF_BASE);
let start = Instant::now();
let r = query(&b, 0, &rows);
let took = start.elapsed();
assert!(took.as_millis() < 500, "15 000 rows took {took:?}");
assert_eq!(r.total, 10_000);
assert_eq!(r.cells_computed(), 0);
let top: Vec<usize> = r.rows().take(5).collect();
for id in &top {
let _ = r.cell(*id, 0);
let _ = r.cell(*id, 0);
}
assert_eq!(r.cells_computed(), 5);
assert_eq!(r.cell(top[0], 1), &r.cell(top[0], 1).clone());
assert_eq!(r.cell(999_999, 0), &Value::Null);
}
#[test]
#[ignore = "perf: run with --release -- --ignored"]
fn perf_15000_rows_filter_and_sort_under_60ms() {
let rows = synthetic(15_000);
let b = base(PERF_BASE);
let start = Instant::now();
let r = query(&b, 0, &rows);
let took = start.elapsed();
assert_eq!(r.total, 10_000);
assert!(took.as_millis() < 60, "took {took:?}");
}
+237
View File
@@ -0,0 +1,237 @@
//! Values, types, dates and durations.
use std::cmp::Ordering;
use notescraft_bases::{Date, Duration, Value, add_duration, compare, loose_eq, parse_duration, sub_dates, truthy};
/// A date read from text, panicking on a bad fixture.
fn day(text: &str) -> Date {
Date::parse(text).expect("a date")
}
/// A duration read from text, panicking on a bad fixture.
fn span(text: &str) -> Duration {
parse_duration(text).expect("a duration")
}
#[test]
fn the_documentation_example_of_a_date_plus_durations() {
let start = day("2024-12-01");
let result = add_duration(start, span("1M")).expect("a date");
let result = add_duration(result, span("4h")).expect("a date");
let result = add_duration(result, span("3m")).expect("a date");
assert_eq!(result.format_iso(), "2025-01-01 04:03:00");
}
#[test]
fn the_documentation_example_of_a_day_between_two_dates() {
let now = Date::now(1_700_000_000_000);
let later = add_duration(now, span("1d")).expect("a date");
assert_eq!(sub_dates(later, now).expect("a difference"), 86_400_000);
}
#[test]
fn month_arithmetic_clamps_to_the_last_day() {
let leap = add_duration(day("2024-01-31"), span("1M")).expect("a date");
assert_eq!(leap.format_iso(), "2024-02-29");
let common = add_duration(day("2025-01-31"), span("1M")).expect("a date");
assert_eq!(common.format_iso(), "2025-02-28");
let forward = add_duration(day("2024-12-31"), span("1M")).expect("a date");
assert_eq!(forward.format_iso(), "2025-01-31");
let back = add_duration(day("2024-03-31"), span("-1M")).expect("a date");
assert_eq!(back.format_iso(), "2024-02-29");
}
#[test]
fn leap_years_and_impossible_dates() {
assert!(Date::parse("2024-02-29").is_ok());
assert!(Date::parse("2000-02-29").is_ok());
assert!(Date::parse("2023-02-29").is_err());
assert!(Date::parse("1900-02-29").is_err());
assert!(Date::parse("2024-13-01").is_err());
assert!(Date::parse("2024-12-32").is_err());
assert!(Date::parse("2024-12-00").is_err());
assert!(Date::parse("not a date").is_err());
assert!(Date::parse("").is_err());
}
#[test]
fn durations_read_the_documented_units() {
let mixed = span("1M 4h");
assert_eq!(mixed.months(), 1);
assert_eq!(mixed.ms(), 4 * 60 * 60 * 1000);
assert_eq!(span("2 hours").ms(), 2 * 60 * 60 * 1000);
assert_eq!(span("1M4h").ms(), 4 * 60 * 60 * 1000);
assert_eq!(span("1y").months(), 12);
assert_eq!(span("2w").ms(), 14 * 24 * 60 * 60 * 1000);
assert_eq!(span("90m").ms(), 90 * 60 * 1000);
assert_eq!(span("30s").ms(), 30 * 1000);
}
#[test]
fn negative_durations_move_backwards() {
let negative = span("-1d");
assert_eq!(negative.ms(), -86_400_000);
assert_eq!(negative.months(), 0);
let back = add_duration(day("2024-12-02"), negative).expect("a date");
assert_eq!(back.format_iso(), "2024-12-01");
let mixed = span("-2h");
assert_eq!(mixed.ms(), -2 * 60 * 60 * 1000);
assert_eq!(mixed.to_display_string(), "-2h");
assert_eq!(span("1M -2h").to_display_string(), "1M -2h");
}
#[test]
fn broken_durations_are_refused() {
assert!(parse_duration("1").is_err());
assert!(parse_duration("1x").is_err());
assert!(parse_duration("d").is_err());
assert!(parse_duration("").is_err());
assert!(parse_duration(" ").is_err());
assert!(parse_duration("1 hour banana").is_err());
}
#[test]
fn durations_scale_and_add() {
let doubled = span("5h").scaled(2).expect("a duration");
assert_eq!(doubled.ms(), 10 * 60 * 60 * 1000);
let sum = span("1d").plus(span("2h")).expect("a duration");
assert_eq!(sum.ms(), 26 * 60 * 60 * 1000);
assert!(span("0s").is_zero());
}
#[test]
fn the_order_runs_from_null_to_the_lists() {
let values = [
Value::Null,
Value::Bool(false),
Value::Bool(true),
Value::Number(1.0),
Value::Str("a".to_owned()),
Value::Date(day("2024-01-01")),
Value::List(vec![Value::Number(1.0)]),
Value::Object(vec![("a".to_owned(), Value::Number(1.0))]),
];
for pair in values.windows(2) {
if let [left, right] = pair {
assert_eq!(compare(left, right), Ordering::Less);
assert_eq!(compare(right, left), Ordering::Greater);
assert_eq!(compare(left, left), Ordering::Equal);
}
}
}
#[test]
fn a_not_a_number_sorts_last() {
let nan = Value::Number(f64::NAN);
assert_eq!(compare(&nan, &Value::Number(1.0)), Ordering::Greater);
assert_eq!(compare(&Value::Number(1.0), &nan), Ordering::Less);
assert_eq!(compare(&nan, &nan), Ordering::Equal);
assert_eq!(compare(&nan, &Value::Str("a".to_owned())), Ordering::Less);
}
#[test]
fn strings_compare_without_case_first() {
assert_eq!(compare(&Value::Str("apple".to_owned()), &Value::Str("Banana".to_owned())), Ordering::Less);
assert_eq!(compare(&Value::Str("A".to_owned()), &Value::Str("a".to_owned())), Ordering::Less);
assert_eq!(compare(&Value::Str("a".to_owned()), &Value::Str("A".to_owned())), Ordering::Greater);
}
#[test]
fn a_link_equals_the_file_it_points_at() {
let link = Value::Link { target: "note.md".to_owned(), display: "Note".to_owned() };
let file = Value::File { path: "note.md".to_owned() };
assert!(loose_eq(&link, &file));
assert!(loose_eq(&file, &link));
let other_file = Value::File { path: "other.md".to_owned() };
assert!(!loose_eq(&link, &other_file));
let same_target = Value::Link { target: "note.md".to_owned(), display: "Another name".to_owned() };
assert!(loose_eq(&link, &same_target));
let same_text = Value::Link { target: "elsewhere.md".to_owned(), display: "Note".to_owned() };
assert!(loose_eq(&link, &same_text));
}
#[test]
fn values_of_different_types_are_never_equal() {
assert!(loose_eq(&Value::Number(1.0), &Value::Number(1.0)));
assert!(!loose_eq(&Value::Number(f64::NAN), &Value::Number(f64::NAN)));
assert!(!loose_eq(&Value::Number(1.0), &Value::Str("1".to_owned())));
assert!(!loose_eq(&Value::Null, &Value::Bool(false)));
assert!(loose_eq(&Value::List(vec![Value::Number(1.0)]), &Value::List(vec![Value::Number(1.0)])));
assert!(!loose_eq(&Value::List(vec![Value::Number(1.0)]), &Value::List(vec![Value::Number(2.0)])));
}
#[test]
fn truthiness_follows_the_documentation() {
assert!(!truthy(&Value::Null));
assert!(!truthy(&Value::Bool(false)));
assert!(truthy(&Value::Bool(true)));
assert!(!truthy(&Value::Number(0.0)));
assert!(truthy(&Value::Number(2.0)));
assert!(!truthy(&Value::Str(String::new())));
assert!(truthy(&Value::Str("x".to_owned())));
assert!(!truthy(&Value::List(Vec::new())));
assert!(truthy(&Value::List(vec![Value::Null])));
assert!(!truthy(&Value::Error("boom".to_owned())));
}
#[test]
fn values_print_as_readable_text() {
assert_eq!(Value::Null.to_display_string(), "");
assert_eq!(Value::Bool(true).to_display_string(), "true");
assert_eq!(Value::Number(2.5).to_display_string(), "2.5");
assert_eq!(Value::Number(3.0).to_display_string(), "3");
assert_eq!(Value::Str("hi".to_owned()).to_display_string(), "hi");
assert_eq!(Value::Date(day("2024-12-01")).to_display_string(), "2024-12-01");
assert_eq!(Value::Date(day("2024-12-01 04:03:00")).to_display_string(), "2024-12-01 04:03:00");
assert_eq!(Value::Duration(span("1M 4h")).to_display_string(), "1M 4h");
assert_eq!(Value::Duration(span("2h")).to_display_string(), "2h");
assert_eq!(Value::List(vec![Value::Number(1.0), Value::Number(2.0)]).to_display_string(), "1, 2");
assert_eq!(Value::Object(vec![("a".to_owned(), Value::Number(1.0))]).to_display_string(), "a: 1");
assert_eq!(Value::Link { target: "note.md".to_owned(), display: "Note".to_owned() }.to_display_string(), "Note");
assert_eq!(Value::File { path: "note.md".to_owned() }.to_display_string(), "note.md");
assert_eq!(Value::Error("boom".to_owned()).to_display_string(), "boom");
}
#[test]
fn the_type_names_are_the_documented_ones() {
assert_eq!(Value::Null.type_name(), "null");
assert_eq!(Value::Bool(true).type_name(), "boolean");
assert_eq!(Value::Number(1.0).type_name(), "number");
assert_eq!(Value::Str("a".to_owned()).type_name(), "string");
assert_eq!(Value::Date(day("2024-01-01")).type_name(), "date");
assert_eq!(Value::Duration(span("1h")).type_name(), "duration");
assert_eq!(Value::List(Vec::new()).type_name(), "list");
assert_eq!(Value::Object(Vec::new()).type_name(), "object");
assert_eq!(Value::Link { target: "a".to_owned(), display: "a".to_owned() }.type_name(), "link");
assert_eq!(Value::File { path: "a.md".to_owned() }.type_name(), "file");
}
#[test]
fn dates_read_the_documented_shapes() {
let date = day("2024-12-01");
assert!(!date.has_time());
assert_eq!(date.format_iso(), "2024-12-01");
let time = day("2024-12-01 04:03:00");
assert!(time.has_time());
assert_eq!(time.format_iso(), "2024-12-01 04:03:00");
let iso = day("2024-12-01T04:03:00Z");
assert_eq!(iso.format_iso(), "2024-12-01 04:03:00");
let millis = day("2024-12-01 04:03:00.250");
assert_eq!(millis.format_iso(), "2024-12-01 04:03:00.250");
let offset = day("2024-12-01 04:03:00+02:00");
assert_eq!(offset.format_iso(), "2024-12-01 02:03:00");
assert_eq!(offset.ms(), time.ms() - 2 * 60 * 60 * 1000);
}
#[test]
fn today_and_now_come_from_the_caller() {
let now_ms = 1_700_000_000_000;
let now = Date::now(now_ms);
assert_eq!(now.ms(), now_ms);
assert!(now.has_time());
let today = Date::today(now_ms);
assert!(!today.has_time());
assert_eq!(today.ms() % 86_400_000, 0);
assert_eq!(today.format_iso(), "2023-11-14");
}
+184
View File
@@ -0,0 +1,184 @@
//! YAML subset: fixtures from the Obsidian Bases docs, scalars, limits and robustness.
use notescraft_bases::{Error, Node, emit, parse};
use std::time::{Duration, Instant};
fn s(x: &str) -> Node {
Node::Str(x.to_string())
}
fn get<'a>(n: &'a Node, key: &str) -> &'a Node {
match n {
Node::Map(m) => m.iter().find(|(k, _)| k == key).map_or(&Node::Null, |(_, v)| v),
_ => &Node::Null,
}
}
#[test]
fn fixtures_parse_and_round_trip() {
let dir = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures");
let mut n = 0;
for e in std::fs::read_dir(dir).unwrap() {
let path = e.unwrap().path();
let text = std::fs::read_to_string(&path).unwrap();
let a = parse(&text).unwrap_or_else(|e| panic!("{}: {e}", path.display()));
let out = emit(&a);
let b = parse(&out).unwrap_or_else(|e| panic!("{} re-parse: {e}\n{out}", path.display()));
assert_eq!(a, b, "{}\n{out}", path.display());
n += 1;
}
assert!(n >= 9);
}
#[test]
fn key_order_and_nesting() {
let n = parse(include_str!("fixtures/Bases_syntax_1.yaml")).unwrap();
let Node::Map(m) = &n else { panic!() };
let keys: Vec<_> = m.iter().map(|(k, _)| k.as_str()).collect();
assert_eq!(keys, ["filters", "formulas", "properties", "summaries", "views"]);
assert_eq!(get(get(&n, "formulas"), "ppu"), &s("(price / age).toFixed(2)"));
let Node::Seq(views) = get(&n, "views") else { panic!() };
assert_eq!(get(&views[0], "limit"), &Node::Num(10.0));
assert_eq!(get(get(&views[0], "groupBy"), "direction"), &s("DESC"));
let emitted = emit(&n);
let pos = |k: &str| emitted.find(k).unwrap();
assert!(pos("filters:") < pos("formulas:") && pos("formulas:") < pos("views:"));
}
#[test]
fn block_scalars_and_chomping() {
let n = parse("a: |\n x\n y\n\nb: |-\n x\n y\nc: |+\n x\n\nd: >\n one\n two\n\n three\ne: >-\n p\n q\nf: 1\n").unwrap();
assert_eq!(get(&n, "a"), &s("x\ny\n"));
assert_eq!(get(&n, "b"), &s("x\ny"));
assert_eq!(get(&n, "c"), &s("x\n\n"));
assert_eq!(get(&n, "d"), &s("one two\nthree\n"));
assert_eq!(get(&n, "e"), &s("p q"));
assert_eq!(get(&n, "f"), &Node::Num(1.0));
assert_eq!(parse(&emit(&n)).unwrap(), n);
}
#[test]
fn quoted_formulas() {
let n = parse("f: 'if(a == \"x\", 1, 2)'\ng: \"say \\\"hi\\\"\\n\\ttab \\u00e9 \\\\\"\nh: 'it''s'\n- x\n".trim_end_matches("- x\n")).unwrap();
assert_eq!(get(&n, "f"), &s("if(a == \"x\", 1, 2)"));
assert_eq!(get(&n, "g"), &s("say \"hi\"\n\ttab \u{e9} \\"));
assert_eq!(get(&n, "h"), &s("it's"));
assert_eq!(parse(&emit(&n)).unwrap(), n);
}
#[test]
fn flow_collections() {
let n = parse("l: [a, 1, \"c, d\", [x, y], {k: v}]\nm: {a: 1, b: [true, ~], \"q r\": 'z'}\ne: []\nf: {}\n").unwrap();
assert_eq!(get(&n, "l"), &Node::Seq(vec![s("a"), Node::Num(1.0), s("c, d"), Node::Seq(vec![s("x"), s("y")]), Node::Map(vec![("k".into(), s("v"))])]));
assert_eq!(get(get(&n, "m"), "q r"), &s("z"));
assert_eq!(get(&n, "e"), &Node::Seq(vec![]));
assert_eq!(parse(&emit(&n)).unwrap(), n);
assert!(matches!(parse("a: [1, 2"), Err(Error::Syntax { line: 1, .. })));
}
#[test]
fn comments_nulls_and_same_indent_seq() {
let n = parse("# top\na: 1 # c\nb: ~\nc:\nd: null\ne: \"x # y\"\nf: x#y\nlist:\n- one\n- two # c\n\n# gap\n- three\nafter: 2\n").unwrap();
assert_eq!(get(&n, "a"), &Node::Num(1.0));
for k in ["b", "c", "d"] {
assert_eq!(get(&n, k), &Node::Null);
}
assert_eq!(get(&n, "e"), &s("x # y"));
assert_eq!(get(&n, "f"), &s("x#y"));
assert_eq!(get(&n, "list"), &Node::Seq(vec![s("one"), s("two"), s("three")]));
assert_eq!(get(&n, "after"), &Node::Num(2.0));
assert_eq!(parse(&emit(&n)).unwrap(), n);
}
#[test]
fn seq_of_maps_and_nested_seqs() {
let n = parse("- a: 1\n b:\n - x\n - y\n- - p\n - q\n- \n k: v\n- plain\n").unwrap();
let Node::Seq(items) = &n else { panic!() };
assert_eq!(items.len(), 4);
assert_eq!(get(&items[0], "b"), &Node::Seq(vec![s("x"), s("y")]));
assert_eq!(items[1], Node::Seq(vec![s("p"), s("q")]));
assert_eq!(get(&items[2], "k"), &s("v"));
assert_eq!(parse(&emit(&n)).unwrap(), n);
}
#[test]
fn emit_quotes_only_when_needed() {
let n = Node::Map(vec![
("plain".into(), s("hello world")),
("empty".into(), s("")),
("num".into(), s("12")),
("null".into(), s("null")),
("colon".into(), s("a: b")),
("hash".into(), s("a #b")),
("lead".into(), s("- x")),
("sp".into(), s(" x")),
("quote".into(), s("a \"q\" b")),
("a: b".into(), Node::Num(2.5)),
("int".into(), Node::Num(3.0)),
("multi".into(), s("l1\nl2\n")),
("ctrl".into(), s(" lead\nx")),
]);
let out = emit(&n);
assert!(out.contains("plain: hello world\n"), "{out}");
assert!(out.contains("int: 3\n"), "{out}");
assert!(out.contains("quote: a \"q\" b\n"), "{out}");
assert!(out.contains("multi: |\n l1\n l2\n"), "{out}");
assert_eq!(parse(&out).unwrap(), n, "{out}");
}
#[test]
fn limits() {
let nest = |n: usize| (0..n - 1).map(|i| " ".repeat(i) + "a:\n").collect::<Vec<_>>().concat() + &" ".repeat(n - 1) + "a: 1\n";
assert!(parse(&nest(12)).is_ok());
assert!(matches!(parse(&nest(13)), Err(Error::Limit(_))));
assert!(matches!(parse(&format!("a: {}1{}\n", "[".repeat(13), "]".repeat(13))), Err(Error::Limit(_))));
assert!(matches!(parse(&format!("a: \"{}\"\n", "x".repeat(2 * 1024 * 1024))), Err(Error::Limit(_))));
let many = (0..10_001).map(|i| ["- ", &i.to_string(), "\n"].concat()).collect::<Vec<_>>().concat();
assert!(matches!(parse(&many), Err(Error::Limit(_))));
}
#[test]
fn tab_indent_is_an_error() {
let e = parse("a:\n b: 1\nc:\n\td: 2\n").unwrap_err();
assert_eq!(e, Error::Syntax { line: 4, column: 1, message: "tab in indentation".into() });
assert!(matches!(parse("a: 1\n b: 2\n"), Err(Error::Syntax { line: 2, .. })));
assert!(matches!(parse("a: 1\nstray\n"), Err(Error::Syntax { line: 2, .. })));
}
#[test]
fn random_bytes_never_panic() {
let mut x: u64 = 0x2545_F491_4F6C_DD1D;
let mut next = move || {
x ^= x << 13;
x ^= x >> 7;
x ^= x << 17;
x
};
let alphabet: Vec<char> = "ab :-#\n\t[]{},\"'|>\\~0.1e+\u{e9}".chars().collect();
let start = Instant::now();
for i in 0..5000 {
let len = usize::try_from(next() % 200).unwrap_or(0);
let text: String = if i % 2 == 0 {
(0..len).map(|_| alphabet[usize::try_from(next()).unwrap_or(0) % alphabet.len()]).collect()
} else {
let bytes: Vec<u8> = (0..len).map(|_| (next() & 0xff) as u8).collect();
String::from_utf8_lossy(&bytes).into_owned()
};
if let Ok(n) = parse(&text) {
assert_eq!(parse(&emit(&n)).as_ref(), Ok(&n), "{text:?}");
}
}
assert!(start.elapsed() < Duration::from_secs(5));
}
#[test]
fn big_flow_list_is_fast() {
let items: Vec<String> = (0..6000).map(|i| format!("\"item-{i:08}\"")).collect();
let text = format!("l: [{}]\n", items.join(","));
assert!(text.len() > 80_000);
let start = Instant::now();
let n = parse(&text).unwrap();
assert!(start.elapsed() < Duration::from_secs(1));
let Node::Seq(v) = get(&n, "l") else { panic!() };
assert_eq!(v.len(), 6000);
}
+9
View File
@@ -14,6 +14,7 @@ mod insert;
mod json;
mod ops;
mod parse;
pub mod table_formulas;
use std::ops::Range;
@@ -56,6 +57,14 @@ pub enum Error {
/// The operation does not apply to this block.
#[error("{0}")]
Unsupported(&'static str),
/// A formula table holds more cells than [`table_formulas::MAX_CELLS`].
#[error("table has {cells} cells, the limit is {max}")]
TableTooLarge {
/// Cells in the table.
cells: usize,
/// The limit.
max: usize,
},
/// The text is not valid JSON.
#[error("invalid JSON at line {line}, column {column}: {message}")]
Json {
+330
View File
@@ -0,0 +1,330 @@
//! Dependency graph and evaluation of parsed formulas, iterative and capped.
use std::collections::{HashMap, VecDeque};
use std::num::FpCategory;
use std::ops::Range;
use super::parse::{Arg, Coord, Expr, Func, Op, parse};
use super::{CellError, ErrorCode, Evaluation, FormulaCell, MAX_CELLS, MAX_EDGES, MAX_OPS};
use crate::{Error, Result};
type Outcome = std::result::Result<f64, CellError>;
struct Node {
row: usize,
col: usize,
span: Range<usize>,
ast: Option<Expr>,
}
enum Target {
Cell(Coord),
Area(Coord, Coord),
}
fn fail(code: ErrorCode, span: Range<usize>) -> CellError {
CellError { code, span }
}
/// Evaluate every formula cell of a table body.
///
/// `cells` are raw cell strings, row by row; they are only read. Short rows
/// count as empty cells. Per-cell problems (syntax, cycles, caps on one
/// formula, ...) are [`CellError`]s in the result.
///
/// # Errors
/// [`Error::TableTooLarge`] when the table holds more than [`MAX_CELLS`] cells.
pub fn evaluate(cells: &[Vec<String>]) -> Result<Evaluation> {
let total = cells.iter().fold(0usize, |n, row| n.saturating_add(row.len()));
if total > MAX_CELLS {
return Err(Error::TableTooLarge { cells: total, max: MAX_CELLS });
}
let mut nodes = Vec::new();
let mut results: Vec<Option<Outcome>> = Vec::new();
let mut index = HashMap::new();
for (row, line) in cells.iter().enumerate() {
for (col, text) in line.iter().enumerate() {
let Some((span, parsed)) = parse(text) else { continue };
index.insert((row, col), nodes.len());
let (ast, done) = match parsed {
Ok(expr) => (Some(expr), None),
Err(err) => (None, Some(Err(err))),
};
nodes.push(Node { row, col, span, ast });
results.push(done);
}
}
let width = cells.iter().map(Vec::len).max().unwrap_or(0);
let mut ctx = Ctx { cells, width, index, results, ops: 0, span: 0..0 };
let (deps, dependents) = build_graph(&nodes, &mut ctx);
run(&nodes, &mut ctx, &deps, &dependents);
let cells = nodes
.into_iter()
.zip(ctx.results)
.map(|(node, result)| FormulaCell { row: node.row + 1, column: node.col + 1, result: result.unwrap_or_else(|| Err(fail(ErrorCode::Cycle, node.span))) })
.collect();
Ok(Evaluation { cells })
}
fn targets(expr: &Expr, out: &mut Vec<Target>) {
match expr {
Expr::Num(_) => {}
Expr::Ref(coord, _) => out.push(Target::Cell(*coord)),
Expr::Neg(inner) => targets(inner, out),
Expr::Bin(_, left, right, _) => {
targets(left, out);
targets(right, out);
}
Expr::Call(_, args, _) => {
for arg in args {
match arg {
Arg::Area { from, to, .. } => out.push(Target::Area(*from, *to)),
Arg::Expr(inner) => targets(inner, out),
}
}
}
}
}
/// Dependencies (formula-cell node ids) and reverse edges; marks over-budget cells failed.
fn build_graph(nodes: &[Node], ctx: &mut Ctx) -> (Vec<Vec<usize>>, Vec<Vec<usize>>) {
let mut deps = vec![Vec::new(); nodes.len()];
let mut dependents = vec![Vec::new(); nodes.len()];
let mut visits = 0usize;
for (id, node) in nodes.iter().enumerate() {
let Some(ast) = &node.ast else { continue };
let mut found = Vec::new();
targets(ast, &mut found);
let mut list = Vec::new();
let mut over = false;
for target in found {
let (from, to) = match target {
Target::Cell(c) => (c, c),
Target::Area(a, b) => (a, b),
};
if !ctx.in_grid(to) {
continue; // reported as out_of_bounds when evaluated
}
let rows = to.row.saturating_sub(from.row).saturating_add(1);
let cols = to.col.saturating_sub(from.col).saturating_add(1);
visits = visits.saturating_add(rows.saturating_mul(cols));
if visits > MAX_EDGES {
over = true;
break;
}
for row in from.row..=to.row {
for col in from.col..=to.col {
if let Some(&dep) = ctx.index.get(&(row, col)) {
list.push(dep);
}
}
}
}
if over {
if let Some(slot) = ctx.results.get_mut(id) {
*slot = Some(Err(fail(ErrorCode::TooManyEdges, node.span.clone())));
}
continue;
}
list.sort_unstable();
list.dedup();
for &dep in &list {
if let Some(rev) = dependents.get_mut(dep) {
rev.push(id);
}
}
if let Some(slot) = deps.get_mut(id) {
*slot = list;
}
}
(deps, dependents)
}
/// Kahn's algorithm; nodes never reached sit on or behind a cycle and stay unresolved.
fn run(nodes: &[Node], ctx: &mut Ctx, deps: &[Vec<usize>], dependents: &[Vec<usize>]) {
let mut waiting: Vec<usize> = deps.iter().map(Vec::len).collect();
let mut queue: VecDeque<usize> = waiting.iter().enumerate().filter(|(_, n)| **n == 0).map(|(id, _)| id).collect();
while let Some(id) = queue.pop_front() {
let preset = ctx.results.get(id).is_some_and(Option::is_some);
if !preset
&& let Some(node) = nodes.get(id)
&& let Some(ast) = &node.ast
{
ctx.span = node.span.clone();
let outcome = ctx.eval(ast);
if let Some(slot) = ctx.results.get_mut(id) {
*slot = Some(outcome);
}
}
for &next in dependents.get(id).map_or(&[][..], Vec::as_slice) {
if let Some(count) = waiting.get_mut(next) {
*count = count.saturating_sub(1);
if *count == 0 {
queue.push_back(next);
}
}
}
}
}
/// Plain number: optional sign, digits with optional fraction (decimal dot), optional exponent.
fn parse_number(text: &str) -> std::result::Result<f64, ErrorCode> {
let bytes = text.as_bytes();
let mut at = usize::from(matches!(bytes.first(), Some(b'+' | b'-')));
let digits = |mut i: usize| {
while bytes.get(i).is_some_and(u8::is_ascii_digit) {
i += 1;
}
i
};
let int_end = digits(at);
let mut mantissa = int_end - at;
at = int_end;
if bytes.get(at) == Some(&b'.') {
let frac_end = digits(at + 1);
mantissa += frac_end - at - 1;
at = frac_end;
}
if mantissa == 0 {
return Err(ErrorCode::Text);
}
if matches!(bytes.get(at), Some(b'e' | b'E')) {
let mut exp = at + 1;
if matches!(bytes.get(exp), Some(b'+' | b'-')) {
exp += 1;
}
let end = digits(exp);
if end == exp {
return Err(ErrorCode::Text);
}
at = end;
}
if at != bytes.len() {
return Err(ErrorCode::Text);
}
match text.parse::<f64>() {
Ok(v) if v.is_finite() => Ok(v),
Ok(_) => Err(ErrorCode::Nonfinite),
Err(_) => Err(ErrorCode::Text),
}
}
struct Ctx<'a> {
cells: &'a [Vec<String>],
width: usize,
index: HashMap<(usize, usize), usize>,
results: Vec<Option<Outcome>>,
ops: usize,
/// Span of the formula being evaluated, used for budget errors.
span: Range<usize>,
}
#[derive(Default)]
struct Acc {
count: u32,
sum: f64,
min: Option<f64>,
max: Option<f64>,
}
impl Acc {
fn add(&mut self, value: f64) {
self.count = self.count.saturating_add(1);
self.sum += value;
self.min = Some(self.min.map_or(value, |m| m.min(value)));
self.max = Some(self.max.map_or(value, |m| m.max(value)));
}
fn finish(&self, func: Func, span: &Range<usize>) -> Outcome {
let bad = |code| fail(code, span.clone());
if !self.sum.is_finite() {
return Err(bad(ErrorCode::Nonfinite));
}
let value = match func {
Func::Sum => Some(self.sum),
Func::Count => Some(f64::from(self.count)),
Func::Avg => (self.count > 0).then(|| self.sum / f64::from(self.count)),
Func::Min => self.min,
Func::Max => self.max,
};
value.filter(|v| v.is_finite()).ok_or_else(|| bad(ErrorCode::EmptyRange))
}
}
impl Ctx<'_> {
fn in_grid(&self, c: Coord) -> bool {
c.row < self.cells.len() && c.col < self.width
}
fn spend(&mut self, n: usize) -> std::result::Result<(), CellError> {
self.ops = self.ops.saturating_add(n);
if self.ops > MAX_OPS {
return Err(fail(ErrorCode::TooManyOps, self.span.clone()));
}
Ok(())
}
/// Value of a cell: `None` for an empty literal cell.
fn value(&self, c: Coord, span: &Range<usize>) -> std::result::Result<Option<f64>, CellError> {
let bad = |code| fail(code, span.clone());
if !self.in_grid(c) {
return Err(bad(ErrorCode::OutOfBounds));
}
if let Some(&id) = self.index.get(&(c.row, c.col)) {
return match self.results.get(id) {
Some(Some(Ok(v))) => Ok(Some(*v)),
Some(Some(Err(e))) => Err(bad(e.code)),
_ => Err(bad(ErrorCode::Cycle)),
};
}
let text = self.cells.get(c.row).and_then(|r| r.get(c.col)).map_or("", String::as_str).trim();
if text.is_empty() {
return Ok(None);
}
parse_number(text).map(Some).map_err(bad)
}
fn eval(&mut self, expr: &Expr) -> Outcome {
self.spend(1)?;
match expr {
Expr::Num(v) => Ok(*v),
Expr::Ref(c, span) => Ok(self.value(*c, span)?.unwrap_or(0.0)),
Expr::Neg(inner) => Ok(-self.eval(inner)?),
Expr::Bin(op, left, right, span) => {
let (a, b) = (self.eval(left)?, self.eval(right)?);
let value = match op {
Op::Add => a + b,
Op::Sub => a - b,
Op::Mul => a * b,
Op::Div if matches!(b.classify(), FpCategory::Zero) => return Err(fail(ErrorCode::DivZero, span.clone())),
Op::Div => a / b,
};
if value.is_finite() { Ok(value) } else { Err(fail(ErrorCode::Nonfinite, span.clone())) }
}
Expr::Call(func, args, span) => {
let mut acc = Acc::default();
for arg in args {
match arg {
Arg::Expr(inner) => acc.add(self.eval(inner)?),
Arg::Area { from, to, span: area } => {
if !self.in_grid(*to) {
return Err(fail(ErrorCode::OutOfBounds, area.clone()));
}
let rows = to.row.saturating_sub(from.row).saturating_add(1);
let cols = to.col.saturating_sub(from.col).saturating_add(1);
self.spend(rows.saturating_mul(cols))?;
for row in from.row..=to.row {
for col in from.col..=to.col {
if let Some(v) = self.value(Coord { row, col }, area)? {
acc.add(v);
}
}
}
}
}
}
acc.finish(*func, span)
}
}
}
}
+129
View File
@@ -0,0 +1,129 @@
//! Pure pipe-table formulas (`NotesCraft` syntax, not Advanced Tables).
//!
//! [`evaluate`] takes the BODY cells of a table (header and delimiter rows
//! excluded; `A1` is the first body row, first column), finds every cell whose
//! trimmed text starts with `=` and computes it. Grammar: `+ - * /`, unary
//! sign, parentheses, cell refs and, as aggregate arguments only, rectangular
//! ranges for `SUM`, `AVG`, `MIN`, `MAX` and `COUNT`. No IO, no scripts. Source
//! cells are never modified; results are separate values.
//!
//! Semantics: a referenced empty cell is 0; text is a typed error. Inside an
//! aggregate, empty cells are skipped, so `AVG`, `MIN` and `MAX` over only
//! empty cells fail with `empty_range`, `SUM` and `COUNT` give 0.
mod eval;
mod parse;
use std::ops::Range;
pub use eval::evaluate;
/// Most cells (all rows together) one table may hold.
pub const MAX_CELLS: usize = 10_000;
/// Longest formula, in bytes of the trimmed cell.
pub const MAX_FORMULA_BYTES: usize = 1024;
/// Deepest AST (and parser nesting) accepted.
pub const MAX_DEPTH: usize = 128;
/// Most dependency edges and expanded-range cell visits per table.
pub const MAX_EDGES: usize = 100_000;
/// Most evaluation operations per table.
pub const MAX_OPS: usize = 1_000_000;
/// Stable code of a per-cell formula failure.
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize)]
#[serde(rename_all = "snake_case")]
pub enum ErrorCode {
/// The formula does not match the grammar.
Syntax,
/// Function name is not `SUM`, `AVG`, `MIN`, `MAX` or `COUNT`.
UnknownFunction,
/// A reference or range lies outside the grid (or is not a valid coordinate).
OutOfBounds,
/// Division by zero.
DivZero,
/// A result or number was infinite or not a number.
Nonfinite,
/// A referenced cell is not a plain number.
Text,
/// `AVG`, `MIN` or `MAX` found no numeric cell.
EmptyRange,
/// The cell takes part in (or depends on) a reference cycle.
Cycle,
/// The formula is longer than [`MAX_FORMULA_BYTES`].
TooLong,
/// The formula nests deeper than [`MAX_DEPTH`].
TooDeep,
/// The table passed [`MAX_EDGES`] dependency visits.
TooManyEdges,
/// The table passed [`MAX_OPS`] operations.
TooManyOps,
}
impl ErrorCode {
/// Stable machine name, the same as the serialized form.
#[must_use]
pub const fn as_str(self) -> &'static str {
match self {
Self::Syntax => "syntax",
Self::UnknownFunction => "unknown_function",
Self::OutOfBounds => "out_of_bounds",
Self::DivZero => "div_zero",
Self::Nonfinite => "nonfinite",
Self::Text => "text",
Self::EmptyRange => "empty_range",
Self::Cycle => "cycle",
Self::TooLong => "too_long",
Self::TooDeep => "too_deep",
Self::TooManyEdges => "too_many_edges",
Self::TooManyOps => "too_many_ops",
}
}
}
/// A failed formula cell: what went wrong and where.
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize)]
pub struct CellError {
/// Stable failure code.
pub code: ErrorCode,
/// Byte span inside the raw cell string (the whole formula for cycles and caps).
pub span: Range<usize>,
}
/// Outcome of one formula cell.
#[derive(Debug, Clone, PartialEq, serde::Serialize)]
pub struct FormulaCell {
/// One-based body row.
pub row: usize,
/// One-based column.
pub column: usize,
/// The finite number or the error.
pub result: std::result::Result<f64, CellError>,
}
/// Results of every formula cell of a table, in row-major order.
#[derive(Debug, Clone, PartialEq, Default, serde::Serialize)]
pub struct Evaluation {
/// One entry per formula cell; non-formula cells are absent.
pub cells: Vec<FormulaCell>,
}
impl Evaluation {
/// The formula cell at one-based `row` and `column`, if that cell is a formula.
#[must_use]
pub fn get(&self, row: usize, column: usize) -> Option<&FormulaCell> {
let at = self.cells.binary_search_by_key(&(row, column), |c| (c.row, c.column)).ok()?;
self.cells.get(at)
}
}
/// Display text of a finite number: Rust shortest-roundtrip, `-0` shown as `0`.
#[must_use]
pub fn format_number(value: f64) -> String {
if matches!(value.classify(), std::num::FpCategory::Zero) {
return "0".to_owned();
}
value.to_string()
}
#[cfg(test)]
mod tests;
+340
View File
@@ -0,0 +1,340 @@
//! Tokenizer and bounded recursive-descent parser for the formula grammar.
use std::ops::Range;
use super::{CellError, ErrorCode, MAX_DEPTH, MAX_FORMULA_BYTES};
/// Zero-based body coordinate.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(super) struct Coord {
pub row: usize,
pub col: usize,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(super) enum Op {
Add,
Sub,
Mul,
Div,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(super) enum Func {
Sum,
Avg,
Min,
Max,
Count,
}
#[derive(Debug)]
pub(super) enum Arg {
/// Rectangle with `from <= to` on both axes.
Area {
from: Coord,
to: Coord,
span: Range<usize>,
},
Expr(Expr),
}
#[derive(Debug)]
pub(super) enum Expr {
Num(f64),
Ref(Coord, Range<usize>),
Neg(Box<Expr>),
Bin(Op, Box<Expr>, Box<Expr>, Range<usize>),
Call(Func, Vec<Arg>, Range<usize>),
}
#[derive(Debug, Clone, PartialEq)]
enum Tok {
Num(f64),
Ref(Coord),
Ident,
Plus,
Minus,
Star,
Slash,
LParen,
RParen,
Comma,
Colon,
}
type Spanned = (Tok, Range<usize>);
type Parsed = Result<(Expr, usize), CellError>;
fn fail(code: ErrorCode, span: Range<usize>) -> CellError {
CellError { code, span }
}
/// Parse a cell: `None` when it is not a formula, else its span and AST.
pub(super) fn parse(cell: &str) -> Option<(Range<usize>, Result<Expr, CellError>)> {
let text = cell.trim();
let lead = cell.len().saturating_sub(cell.trim_start().len());
let body = text.strip_prefix('=')?;
let span = lead..lead.saturating_add(text.len());
if text.len() > MAX_FORMULA_BYTES {
return Some((span.clone(), Err(fail(ErrorCode::TooLong, span))));
}
let base = lead.saturating_add(1);
let result = tokenize(body, base).and_then(|toks| {
let mut parser = Parser { toks, pos: 0, end: span.end, src: body, base };
let (expr, _) = parser.chain(1, 1)?;
if parser.pos < parser.toks.len() {
return Err(parser.syntax());
}
Ok(expr)
});
Some((span, result))
}
fn char_len_at(src: &str, at: usize) -> usize {
src.get(at..).and_then(|rest| rest.chars().next()).map_or(1, char::len_utf8)
}
fn tokenize(src: &str, base: usize) -> Result<Vec<Spanned>, CellError> {
let bytes = src.as_bytes();
let mut out = Vec::new();
let mut i = 0;
while let Some(&byte) = bytes.get(i) {
let start = i;
let (tok, len) = match byte {
b' ' | b'\t' => {
i += 1;
continue;
}
b'+' => (Tok::Plus, 1),
b'-' => (Tok::Minus, 1),
b'*' => (Tok::Star, 1),
b'/' => (Tok::Slash, 1),
b'(' => (Tok::LParen, 1),
b')' => (Tok::RParen, 1),
b',' => (Tok::Comma, 1),
b':' => (Tok::Colon, 1),
b'0'..=b'9' | b'.' => {
let end = number_end(bytes, i);
let span = base + start..base + end;
let value = src.get(start..end).and_then(|t| t.parse::<f64>().ok());
match value {
Some(v) if v.is_finite() => (Tok::Num(v), end - start),
Some(_) => return Err(fail(ErrorCode::Nonfinite, span)),
None => return Err(fail(ErrorCode::Syntax, span)),
}
}
b if b.is_ascii_alphabetic() => {
let end = bytes.iter().skip(i).position(|b| !b.is_ascii_alphanumeric()).map_or(bytes.len(), |n| i + n);
let span = base + start..base + end;
let word = src.get(start..end).unwrap_or_default();
(word_token(word, span)?, end - start)
}
_ => {
let len = char_len_at(src, i);
return Err(fail(ErrorCode::Syntax, base + start..base + start + len));
}
};
i = start + len;
out.push((tok, base + start..base + i));
}
Ok(out)
}
/// End of a number starting at `from`: digits, optional fraction, optional valid exponent.
fn number_end(bytes: &[u8], from: usize) -> usize {
let digits = |mut at: usize| {
while bytes.get(at).is_some_and(u8::is_ascii_digit) {
at += 1;
}
at
};
let mut end = digits(from);
if bytes.get(end) == Some(&b'.') {
end = digits(end + 1);
}
if matches!(bytes.get(end), Some(b'e' | b'E')) {
let mut at = end + 1;
if matches!(bytes.get(at), Some(b'+' | b'-')) {
at += 1;
}
let after = digits(at);
if after > at {
end = after;
}
}
end
}
fn word_token(word: &str, span: Range<usize>) -> Result<Tok, CellError> {
let split = word.bytes().position(|b| b.is_ascii_digit()).unwrap_or(word.len());
let letters = word.get(..split).unwrap_or_default();
let digits = word.get(split..).unwrap_or_default();
if digits.is_empty() {
return Ok(Tok::Ident);
}
if !digits.bytes().all(|b| b.is_ascii_digit()) {
return Err(fail(ErrorCode::Syntax, span));
}
let col = letters
.bytes()
.try_fold(0usize, |acc, b| {
let digit = usize::from(b.to_ascii_uppercase().saturating_sub(b'A')).saturating_add(1);
acc.checked_mul(26)?.checked_add(digit)
})
.and_then(|n| n.checked_sub(1));
let row = digits.parse::<usize>().ok().and_then(|n| n.checked_sub(1));
match (row, col) {
(Some(row), Some(col)) => Ok(Tok::Ref(Coord { row, col })),
_ => Err(fail(ErrorCode::OutOfBounds, span)),
}
}
struct Parser<'a> {
toks: Vec<Spanned>,
pos: usize,
/// End of the formula, where "unexpected end" errors point.
end: usize,
src: &'a str,
base: usize,
}
impl Parser<'_> {
fn peek(&self) -> Option<&Tok> {
self.toks.get(self.pos).map(|(tok, _)| tok)
}
fn peek_at(&self, ahead: usize) -> Option<&Tok> {
self.toks.get(self.pos.saturating_add(ahead)).map(|(tok, _)| tok)
}
fn span(&self) -> Range<usize> {
self.toks.get(self.pos).map_or(self.end..self.end, |(_, span)| span.clone())
}
fn syntax(&self) -> CellError {
fail(ErrorCode::Syntax, self.span())
}
fn eat(&mut self, tok: &Tok) -> bool {
let found = self.peek() == Some(tok);
if found {
self.pos += 1;
}
found
}
fn guard(&self, nest: usize) -> Result<(), CellError> {
if nest > MAX_DEPTH {
return Err(fail(ErrorCode::TooDeep, self.span()));
}
Ok(())
}
/// Level 1 is `+ -`, level 0 is `* /`; both left-associative.
fn chain(&mut self, nest: usize, level: u8) -> Parsed {
self.guard(nest)?;
let mut left = if level == 0 { self.unary(nest)? } else { self.chain(nest, 0)? };
loop {
let (tok, op) = match (self.peek(), level) {
(Some(Tok::Star), 0) => (Tok::Star, Op::Mul),
(Some(Tok::Slash), 0) => (Tok::Slash, Op::Div),
(Some(Tok::Plus), 1) => (Tok::Plus, Op::Add),
(Some(Tok::Minus), 1) => (Tok::Minus, Op::Sub),
_ => return Ok(left),
};
let span = self.span();
self.eat(&tok);
let right = if level == 0 { self.unary(nest)? } else { self.chain(nest, 0)? };
let depth = left.1.max(right.1).saturating_add(1);
if depth > MAX_DEPTH {
return Err(fail(ErrorCode::TooDeep, span));
}
left = (Expr::Bin(op, Box::new(left.0), Box::new(right.0), span), depth);
}
}
fn unary(&mut self, nest: usize) -> Parsed {
self.guard(nest)?;
if self.eat(&Tok::Plus) {
return self.unary(nest + 1);
}
if self.eat(&Tok::Minus) {
let (inner, depth) = self.unary(nest + 1)?;
return Ok((Expr::Neg(Box::new(inner)), depth.saturating_add(1)));
}
self.primary(nest)
}
fn primary(&mut self, nest: usize) -> Parsed {
let span = self.span();
match self.peek().cloned() {
Some(Tok::Num(value)) => {
self.pos += 1;
Ok((Expr::Num(value), 1))
}
Some(Tok::Ref(coord)) => {
self.pos += 1;
Ok((Expr::Ref(coord, span), 1))
}
Some(Tok::LParen) => {
self.pos += 1;
let inner = self.chain(nest + 1, 1)?;
if !self.eat(&Tok::RParen) {
return Err(self.syntax());
}
Ok(inner)
}
Some(Tok::Ident) => self.call(nest, span),
_ => Err(self.syntax()),
}
}
fn call(&mut self, nest: usize, name_span: Range<usize>) -> Parsed {
let name = self.src.get(name_span.start.saturating_sub(self.base)..name_span.end.saturating_sub(self.base)).unwrap_or_default();
self.pos += 1;
if self.peek() != Some(&Tok::LParen) {
return Err(self.syntax());
}
let func = [("SUM", Func::Sum), ("AVG", Func::Avg), ("MIN", Func::Min), ("MAX", Func::Max), ("COUNT", Func::Count)]
.into_iter()
.find(|(n, _)| n.eq_ignore_ascii_case(name))
.map(|(_, f)| f)
.ok_or_else(|| fail(ErrorCode::UnknownFunction, name_span.clone()))?;
self.pos += 1;
let mut args = Vec::new();
let mut depth = 1;
loop {
let (arg, arg_depth) = self.arg(nest + 1)?;
depth = depth.max(arg_depth.saturating_add(1));
args.push(arg);
if !self.eat(&Tok::Comma) {
break;
}
}
let close = self.span();
if !self.eat(&Tok::RParen) {
return Err(self.syntax());
}
if depth > MAX_DEPTH {
return Err(fail(ErrorCode::TooDeep, name_span));
}
Ok((Expr::Call(func, args, name_span.start..close.end), depth))
}
fn arg(&mut self, nest: usize) -> Result<(Arg, usize), CellError> {
if let (Some(Tok::Ref(a)), Some(Tok::Colon), Some(Tok::Ref(b))) = (self.peek().cloned(), self.peek_at(1), self.peek_at(2).cloned()) {
let end = self.toks.get(self.pos + 2).map_or(self.end, |(_, s)| s.end);
let span = self.span().start..end;
self.pos += 3;
let from = Coord { row: a.row.min(b.row), col: a.col.min(b.col) };
let to = Coord { row: a.row.max(b.row), col: a.col.max(b.col) };
return Ok((Arg::Area { from, to, span }, 1));
}
let (expr, depth) = self.chain(nest, 1)?;
Ok(match expr {
Expr::Ref(c, span) => (Arg::Area { from: c, to: c, span }, 1),
other => (Arg::Expr(other), depth),
})
}
}
+237
View File
@@ -0,0 +1,237 @@
//! Behaviour tests for formula evaluation, including hostile input.
use super::{ErrorCode, Evaluation, MAX_CELLS, evaluate, format_number};
use crate::Error;
fn grid(rows: &[&[&str]]) -> Vec<Vec<String>> {
rows.iter().map(|r| r.iter().map(|c| (*c).to_owned()).collect()).collect()
}
fn run(rows: &[&[&str]]) -> Evaluation {
evaluate(&grid(rows)).expect("within caps")
}
fn num(e: &Evaluation, row: usize, col: usize) -> f64 {
// NaN makes every numeric assertion below fail for a missing or failed cell.
match e.get(row, col).map(|c| &c.result) {
Some(Ok(v)) => *v,
_ => f64::NAN,
}
}
fn code(e: &Evaluation, row: usize, col: usize) -> &'static str {
match e.get(row, col).map(|c| &c.result) {
Some(Err(err)) => err.code.as_str(),
Some(Ok(_)) => "<number>",
None => "<not a formula>",
}
}
fn one(formula: &str) -> Evaluation {
run(&[&[formula]])
}
fn assert_num(formula: &str, expected: f64) {
let got = num(&one(formula), 1, 1);
assert!((got - expected).abs() < 1e-9, "{formula} = {got}, expected {expected}");
}
#[test]
fn precedence_and_unary() {
assert_num("=1+2*3", 7.0);
assert_num("=(1+2)*3", 9.0);
assert_num("=-2*-3", 6.0);
assert_num("=- -2", 2.0);
assert_num("=+4", 4.0);
assert_num("=2-3-4", -5.0);
assert_num("=8/2/2", 2.0);
assert_num("=1.5e1+.5", 15.5);
assert_num(" = 1 + 1 ", 2.0);
}
#[test]
fn a1_is_first_body_cell_and_source_is_untouched() {
let cells = grid(&[&["2", "3", "=A1*B1"], &["10", "", "=A1+A2+B2"]]);
let before = cells.clone();
let e = evaluate(&cells).expect("ok");
assert_eq!(cells, before);
assert!((num(&e, 1, 3) - 6.0).abs() < 1e-9);
assert!((num(&e, 2, 3) - 12.0).abs() < 1e-9);
assert!(e.get(1, 1).is_none());
assert_eq!(e.cells.len(), 2);
}
#[test]
fn aggregates_over_ranges() {
let e = run(&[
&["2", "=SUM(A1:A3)", "=AVG(A1:A3)", "=MIN(A1:A3)", "=MAX(A1:A3)", "=COUNT(A1:A3)"],
&["", "=sum(a3:a1)", "", "", "", ""],
&["4", "=SUM(A1:A3, 10, A1)", "", "", "", ""],
]);
assert!((num(&e, 1, 2) - 6.0).abs() < 1e-9);
assert!((num(&e, 1, 3) - 3.0).abs() < 1e-9);
assert!((num(&e, 1, 4) - 2.0).abs() < 1e-9);
assert!((num(&e, 1, 5) - 4.0).abs() < 1e-9);
assert!((num(&e, 1, 6) - 2.0).abs() < 1e-9);
assert!((num(&e, 2, 2) - 6.0).abs() < 1e-9);
assert!((num(&e, 3, 2) - 18.0).abs() < 1e-9);
}
#[test]
fn aggregates_with_empty_and_text() {
let e = run(&[&["", "x", "=SUM(A1:A2)", "=COUNT(A1:A2)", "=AVG(A1:A2)", "=MIN(A1)", "=MAX(A1:A2)", "=SUM(A1:B1)", "=COUNT(B1)"], &[""]]);
assert!(num(&e, 1, 3).abs() < 1e-9);
assert!(num(&e, 1, 4).abs() < 1e-9);
assert_eq!(code(&e, 1, 5), "empty_range");
assert_eq!(code(&e, 1, 6), "empty_range");
assert_eq!(code(&e, 1, 7), "empty_range");
assert_eq!(code(&e, 1, 8), "text");
assert_eq!(code(&e, 1, 9), "text");
}
#[test]
fn empty_reference_is_zero_and_text_is_an_error() {
let e = run(&[&["", "abc", "1,5", "`=1`", "é", "=A1+1", "=B1+1", "=C1", "=D1", "=E1"]]);
assert!((num(&e, 1, 6) - 1.0).abs() < 1e-9);
for col in 7..=10 {
assert_eq!(code(&e, 1, col), "text");
}
}
#[test]
fn literal_number_forms() {
let e = run(&[&["+5", ".5", "5.", "1e3", " 7 ", "-0.25E-1", "=SUM(A1:F1)"]]);
assert!((num(&e, 1, 7) - (5.0 + 0.5 + 5.0 + 1000.0 + 7.0 - 0.025)).abs() < 1e-9);
let bad = run(&[&["inf", "nan", ".", "1e", "0x10", "1 2", "1e999", "=A1", "=B1", "=C1", "=D1", "=E1", "=F1", "=G1"]]);
for col in 8..=13 {
assert_eq!(code(&bad, 1, col), "text", "col {col}");
}
assert_eq!(code(&bad, 1, 14), "nonfinite");
}
#[test]
fn syntax_and_unknown_function() {
for f in ["=", "=1+", "=SUM()", "=A1:B2", "=1 2", "=(1", "=1)", "=SUM(A1", "=SUM", "=A", "=A1B", "=1.2.3", "=2x", "=é", "=1;2", "=SUM(,)"] {
assert_eq!(code(&one(f), 1, 1), "syntax", "{f}");
}
assert_eq!(code(&one("=FOO(1)"), 1, 1), "unknown_function");
assert_eq!(code(&one("=NOW()"), 1, 1), "unknown_function");
}
#[test]
fn error_spans_point_into_the_raw_cell() {
let e = one(" =1/0");
let err = e.get(1, 1).and_then(|c| c.result.as_ref().err()).expect("error");
assert_eq!(err.code, ErrorCode::DivZero);
assert_eq!(err.span, 4..5);
let e = one("=é");
let err = e.get(1, 1).and_then(|c| c.result.as_ref().err()).expect("error");
assert_eq!(err.span, 1..3);
}
#[test]
fn cycles_and_dependents_are_marked() {
let e = run(&[&["=B1", "=A1", "=A1+1", "=D1", "=C1+10", "5"]]);
for col in 1..=4 {
assert_eq!(code(&e, 1, col), "cycle", "col {col}");
}
assert_eq!(code(&e, 1, 5), "cycle");
}
#[test]
fn errors_propagate_to_dependents() {
let e = run(&[&["=1/0", "=A1+1", "=SUM(A1:B1)", "=1+1", "=D1*2"]]);
assert_eq!(code(&e, 1, 1), "div_zero");
assert_eq!(code(&e, 1, 2), "div_zero");
assert_eq!(code(&e, 1, 3), "div_zero");
assert!((num(&e, 1, 5) - 4.0).abs() < 1e-9);
}
#[test]
fn chained_formulas_resolve_in_any_layout() {
let e = run(&[&["=B1+1", "=C1+1", "=1"]]);
assert!((num(&e, 1, 1) - 3.0).abs() < 1e-9);
assert!((num(&e, 1, 2) - 2.0).abs() < 1e-9);
}
#[test]
fn nonfinite_divzero_and_out_of_bounds() {
assert_eq!(code(&one("=1e308*10"), 1, 1), "nonfinite");
assert_eq!(code(&one("=1e999"), 1, 1), "nonfinite");
assert_eq!(code(&one("=1e308+1e308"), 1, 1), "nonfinite");
assert_eq!(code(&one("=SUM(1e308, 1e308)"), 1, 1), "nonfinite");
assert_eq!(code(&one("=1/(1-1)"), 1, 1), "div_zero");
assert_eq!(code(&one("=1/-0"), 1, 1), "div_zero");
for f in ["=B1", "=A2", "=A0", "=SUM(A1:A99)", "=SUM(Z1:Z2)", "=ZZZZZZZZZZZZZZZZZZZZZZZZZ1", "=A99999999999999999999999"] {
assert_eq!(code(&one(f), 1, 1), "out_of_bounds", "{f}");
}
}
#[test]
fn ragged_rows_read_as_empty() {
let e = run(&[&["1", "2", "3"], &["4"], &["=B2+C2+A2", "=SUM(A1:C2)"]]);
assert!((num(&e, 3, 1) - 4.0).abs() < 1e-9);
assert!((num(&e, 3, 2) - 10.0).abs() < 1e-9);
assert_eq!(code(&run(&[&["1", "2"], &["=C1"]]), 2, 1), "out_of_bounds");
}
#[test]
fn depth_caps() {
let parens = |n: usize| format!("={}1{}", "(".repeat(n), ")".repeat(n));
assert_num(&parens(100), 1.0);
assert_eq!(code(&one(&parens(129)), 1, 1), "too_deep");
assert_eq!(code(&one(&format!("=1{}", "+1".repeat(300))), 1, 1), "too_deep");
assert_eq!(code(&one(&format!("={}1", "-".repeat(200))), 1, 1), "too_deep");
assert_eq!(code(&one(&format!("={}1{}", "SUM(".repeat(200), ")".repeat(200))), 1, 1), "too_deep");
}
#[test]
fn formula_length_cap() {
let ok = format!("=1{}", "+1".repeat(500));
assert!(ok.len() <= 1024);
assert_eq!(code(&one(&ok), 1, 1), "too_deep");
let long = format!("=1{}+1", " ".repeat(1030));
assert_eq!(code(&one(&long), 1, 1), "too_long");
}
#[test]
fn cell_cap_is_an_error_not_a_truncation() {
let row = vec![String::new(); 100];
assert!(evaluate(&vec![row.clone(); MAX_CELLS / 100]).is_ok());
let err = evaluate(&vec![row; MAX_CELLS / 100 + 1]).expect_err("over cap");
assert!(matches!(err, Error::TableTooLarge { cells: 10_100, max: MAX_CELLS }));
}
#[test]
fn edge_cap_marks_late_cells() {
let mut rows = vec![vec!["1".to_owned(); 100]; 99];
rows.push(vec!["=SUM(A1:CV99)".to_owned(); 100]);
let e = evaluate(&rows).expect("cells within cap");
assert!((num(&e, 100, 1) - 9900.0).abs() < 1e-9);
assert!((num(&e, 100, 10) - 9900.0).abs() < 1e-9);
assert_eq!(code(&e, 100, 11), "too_many_edges");
assert_eq!(code(&e, 100, 100), "too_many_edges");
}
#[test]
fn operation_cap_marks_late_cells() {
let formula = format!("=1{}", "+1".repeat(60));
let rows = vec![vec![formula; 100]; 100];
let e = evaluate(&rows).expect("cells within cap");
assert!((num(&e, 1, 1) - 61.0).abs() < 1e-9);
assert_eq!(code(&e, 100, 100), "too_many_ops");
}
#[test]
fn results_are_deterministic() {
let cells = grid(&[&["=B1", "=A1", "1/0", "=1/0", "=SUM(A1:D1)"], &["3", "=A2*2", "=SUM(A2:B2)", "x", "=D2"]]);
assert_eq!(evaluate(&cells).expect("ok"), evaluate(&cells).expect("ok"));
}
#[test]
fn formats_numbers() {
assert_eq!(format_number(0.1 + 0.2), "0.30000000000000004");
assert_eq!(format_number(3.0), "3");
assert_eq!(format_number(-0.0), "0");
assert_eq!(format_number(-2.5), "-2.5");
}
+63
View File
@@ -9,6 +9,8 @@ use std::fmt::Write as _;
use crate::pairs;
mod paste;
/// Tab produces four spaces.
pub const TAB: &str = " ";
@@ -316,3 +318,64 @@ fn list_prefix(frag: &str) -> &str {
}
""
}
/// Paste a URL over selected text: the selection becomes `[label](url)`.
///
/// Returns `None` when the paste stays a plain paste: an empty selection, a
/// selection over several lines or longer than 500 bytes, a selection inside
/// code, a link or another URL, a selection that is itself a URL, or a paste
/// that is not one `http(s)` URL of at most 2 048 bytes. No network call is
/// made: the pasted text is the link target.
#[must_use]
pub fn on_paste_url(text: &str, sel: std::ops::Range<usize>, pasted: &str) -> Option<Edit> {
let (start, end) = norm_sel(text, &sel);
let selected = text.get(start..end)?;
if start == end || selected.len() > paste::MAX_LINK_SELECTION_BYTES || selected.contains(['\n', '\r']) {
return None;
}
let url = paste::pasted_url(pasted)?;
if paste::pasted_url(selected).is_some() {
return None; // the selection is a URL: let the paste replace it
}
if paste::in_protected(text, start, end) {
return None; // inside code, a link or another URL
}
let insert = paste::link_text(selected, &url);
let cursor = start + insert.len();
Some(Edit { range: start..end, insert, sel: cursor..cursor })
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn paste_url_links_the_selection() {
let text = "hello world";
let edit = on_paste_url(text, 6..11, "https://example.com").unwrap();
assert_eq!(edit.insert, "[world](https://example.com)");
assert_eq!(edit.sel, 34..34);
}
#[test]
fn paste_url_escapes_brackets() {
let text = "a [b] c";
let edit = on_paste_url(text, 0..7, "https://example.com").unwrap();
assert_eq!(edit.insert, "[a \\[b\\] c](https://example.com)");
}
#[test]
fn paste_url_keeps_a_wrapped_selection() {
let text = "[label](old)";
let edit = on_paste_url(text, 0..12, "https://example.com").unwrap();
assert_eq!(edit.insert, "[label](https://example.com)");
}
#[test]
fn paste_url_needs_one_http_url() {
let text = "hello";
assert!(on_paste_url(text, 0..5, "javascript:alert(1)").is_none());
assert!(on_paste_url(text, 0..5, "file:///etc/passwd").is_none());
assert!(on_paste_url(text, 0..5, "https://example.com/a b").is_none());
}
}
+109
View File
@@ -0,0 +1,109 @@
//! Helpers for the "paste a URL over a selection" behaviour: what counts as a
//! URL, how the markdown link is written and where a link must not be made.
/// Longest selection (bytes) that a pasted URL turns into a markdown link.
pub(crate) const MAX_LINK_SELECTION_BYTES: usize = 500;
/// Longest pasted URL (bytes) accepted as a link target.
pub(crate) const MAX_PASTED_URL_BYTES: usize = 2048;
/// The pasted text as an `http(s)` URL, or `None` when it is not one.
///
/// A URL is one `http` or `https` link (case-insensitive) without whitespace,
/// whose host holds a dot or is `localhost`. Angle brackets around it are
/// dropped.
#[must_use]
pub(crate) fn pasted_url(pasted: &str) -> Option<String> {
let trimmed = pasted.trim();
if trimmed.is_empty() || trimmed.len() > MAX_PASTED_URL_BYTES || trimmed.contains(char::is_whitespace) {
return None;
}
let body = trimmed.strip_prefix('<').and_then(|t| t.strip_suffix('>')).unwrap_or(trimmed);
let lower = body.to_ascii_lowercase();
let rest = lower.strip_prefix("https://").or_else(|| lower.strip_prefix("http://"))?;
let host = rest.split(['/', '?', '#']).next().unwrap_or("");
if host.is_empty() || !(host.contains('.') || host == "localhost" || host.starts_with("localhost:")) {
return None;
}
Some(body.to_owned())
}
/// The markdown link for `selected` and `url`: `[label](url)`.
///
/// A selection already wrapped as a link keeps its label; brackets in any
/// other selection are escaped.
#[must_use]
pub(crate) fn link_text(selected: &str, url: &str) -> String {
let mut out = String::with_capacity(selected.len() + url.len() + 4);
if let Some(head) = link_head(selected) {
out.push_str(head);
} else {
out.push('[');
for ch in selected.chars() {
if ch == '[' || ch == ']' {
out.push('\\');
}
out.push(ch);
}
out.push(']');
}
out.push('(');
out.push_str(url);
out.push(')');
out
}
/// The `[label]` of a selection already wrapped as a link, when it is
/// `[label]` or `[label](target)`.
fn link_head(selected: &str) -> Option<&str> {
if !selected.starts_with('[') {
return None;
}
let mut depth = 0usize;
for (idx, ch) in selected.char_indices() {
if ch == '[' {
depth += 1;
} else if ch == ']' {
depth = depth.saturating_sub(1);
if depth == 0 {
let tail = selected.get(idx + 1..).unwrap_or("");
let linked = tail.is_empty() || (tail.starts_with('(') && tail.ends_with(')'));
return linked.then(|| selected.get(..idx + 1).unwrap_or(""));
}
}
}
None
}
/// True when `start..end` sits in code, in a link or in another URL.
#[must_use]
pub(crate) fn in_protected(text: &str, start: usize, end: usize) -> bool {
let fences = crate::inline::code_fences(text);
let code = crate::inline::inline_code(text, &fences);
code.iter().chain(fences.iter()).any(|region| region.start < end && start < region.end) || in_link(text, start, end) || in_url(text, start, end)
}
/// True when `start..end` sits inside a markdown link `[label](target)`.
fn in_link(text: &str, start: usize, end: usize) -> bool {
let before = text.get(..start).unwrap_or("");
let Some(open) = before.rfind('[') else { return false };
let Some(close) = text.get(open..).and_then(|t| t.find(']')).map(|i| open + i) else { return false };
let after = text.get(close + 1..).unwrap_or("");
if !after.starts_with('(') {
return false;
}
let target_end = after.find(')').map_or(text.len(), |i| close + 2 + i);
(open <= start && end <= close) || (close + 2 <= start && end <= target_end)
}
/// True when `start..end` sits inside a bare `http(s)://` URL.
fn in_url(text: &str, start: usize, end: usize) -> bool {
let before = text.get(..start).unwrap_or("");
let Some(at) = before.rfind("http") else { return false };
let rest = text.get(at..).unwrap_or("");
if !(rest.starts_with("http://") || rest.starts_with("https://")) {
return false;
}
let stop = rest.find(|c: char| c.is_whitespace() || c == ')').map_or(text.len(), |i| at + i);
at < end && start < stop
}
+2
View File
@@ -26,8 +26,10 @@ pub mod stats;
mod fence_syntax;
mod inline;
mod outliner;
pub use notescraft_blocks::{Error, Result};
pub use outliner::{Direction, indent_subtree, move_item};
/// Maximum source size the styler parses; larger documents get plain styling.
pub const MAX_HIGHLIGHT_BYTES: usize = 2 * 1024 * 1024;
+396
View File
@@ -0,0 +1,396 @@
//! Outliner: move list items with their children, and shift whole subtrees.
//!
//! Pure functions over `(text, selection)`: they return the [`Edit`] to apply,
//! so the editor, the CLI and the tests share one behaviour. No egui here.
use std::collections::HashMap;
use std::ops::Range;
use crate::keys::{Edit, Indent};
pub use crate::blocks::Direction;
/// One source line with the metadata the outliner needs.
struct Line {
start: usize, // byte offset of the line start
len: usize, // byte length of the line, terminator included
text: String, // the line text without its terminator
ws: usize, // byte length of the leading whitespace
cols: usize, // leading whitespace columns, a tab counts as one unit
marker: Option<Marker>,
}
/// The list marker a line starts with.
enum Marker {
/// `- `, `* ` or `+ `.
Bullet,
/// `1. ` or `1) `: the byte range of the number inside the line.
Ordered(Range<usize>),
}
/// A list item: its line and the item it hangs under.
struct Item {
line: usize, // index of the item line in the line list
parent: Option<usize>, // index of the parent item in the item list
}
/// Moves the list item at the cursor, with its children, above the previous
/// sibling or below the next sibling.
///
/// The item keeps its own indentation and its children move with it; a fenced
/// code block inside the item moves as well. The cursor lands on the marker of
/// the moved item (the `-`, `*`, `+` or the first digit of the number) in the
/// new text. An ordered list is renumbered after the move (`1.`, `2.`, `3.`).
///
/// Returns `None` when the cursor is not on a list item, when the item has no
/// sibling in that direction, and for the first or the last item of a list: an
/// item never jumps to another list.
///
/// A selection that spans several items moves the item at the start of the
/// selection only.
#[must_use]
pub fn move_item(text: &str, sel: Range<usize>, dir: Direction, indent: Indent) -> Option<Edit> {
let (start, _) = sel_bounds(text, sel);
let lines = split_lines(text, indent);
let items = items_of(&lines);
let current = item_at(&items, &lines, start)?;
let parent = items.get(current)?.parent;
let siblings: Vec<usize> = items.iter().enumerate().filter(|(_, item)| item.parent == parent).map(|(index, _)| index).collect();
let position = siblings.iter().position(|index| *index == current)?;
let other = match dir {
Direction::Up => siblings.get(position.checked_sub(1)?)?,
Direction::Down => siblings.get(position.checked_add(1)?)?,
};
let cur_item = items.get(current)?;
let sib_item = items.get(*other)?;
let cur_block = (cur_item.line, block_end(&lines, cur_item));
let sib_block = (sib_item.line, block_end(&lines, sib_item));
let ordered = matches!(lines.get(cur_item.line)?.marker, Some(Marker::Ordered(_)));
let (region_lo, region_hi) = if ordered {
(items.get(*siblings.first()?)?.line, block_end(&lines, items.get(*siblings.last()?)?))
} else {
(usize::min(cur_block.0, sib_block.0), usize::max(cur_block.1, sib_block.1))
};
let (first, second) = if cur_block.0 < sib_block.0 { (cur_block, sib_block) } else { (sib_block, cur_block) };
let mut order: Vec<usize> = Vec::with_capacity(region_hi - region_lo + 1);
order.extend(region_lo..first.0);
order.extend(second.0..=second.1);
order.extend(first.1 + 1..second.0);
order.extend(first.0..=first.1);
order.extend(second.1 + 1..=region_hi);
let numbers = new_numbers(&items, &lines, &siblings, current, *other)?;
let mut insert = String::with_capacity(text.len());
let mut offset = 0usize;
let mut caret = None;
for src in &order {
let line = lines.get(*src)?;
let body = match numbers.get(src) {
Some(number) => renumbered(line, number)?,
None => line.text.clone(),
};
if *src == cur_item.line {
caret = Some(offset + line.ws);
}
insert.push_str(&body);
offset += body.len();
let terminator = text.get(line.start + line.text.len()..line.start + line.len)?;
insert.push_str(terminator);
offset += terminator.len();
}
let region_start = lines.get(region_lo)?.start;
let region_end = lines.get(region_hi)?.start + lines.get(region_hi)?.len;
let caret = region_start + caret?;
Some(Edit { range: region_start..region_end, insert, sel: caret..caret })
}
/// Indents or outdents the list item at the cursor and its whole subtree.
///
/// The item moves one level deeper (or back to the surface) and its children
/// keep their place inside it. The cursor stays on the same character.
///
/// Returns `None` when the cursor is not on a list item and when an outdent
/// would take the item below the surface of the list. A selection that spans
/// several items shifts every item it touches, each with its own children.
#[must_use]
pub fn indent_subtree(text: &str, sel: Range<usize>, outdent: bool, indent: Indent) -> Option<Edit> {
let (start, end) = sel_bounds(text, sel);
let lines = split_lines(text, indent);
let items = items_of(&lines);
let roots = selected_items(&items, &lines, start, end)?;
if roots.is_empty() {
return None;
}
let mut ranges: Vec<(usize, usize)> = Vec::with_capacity(roots.len());
for root in &roots {
let item = items.get(*root)?;
if outdent && removable(&lines.get(item.line)?.text, indent) == 0 {
return None;
}
ranges.push((item.line, block_end(&lines, item)));
}
let span_lo = ranges.iter().map(|(from, _)| *from).min()?;
let span_hi = ranges.iter().map(|(_, to)| *to).max()?;
let unit = indent.unit();
let mut insert = String::new();
let mut added = 0usize;
let mut removed = 0usize;
for index in span_lo..=span_hi {
let line = lines.get(index)?;
let shifted = ranges.iter().any(|(from, to)| index >= *from && index <= *to);
let body = if !shifted {
line.text.clone()
} else if outdent {
let cut = removable(&line.text, indent);
if index == span_lo {
removed = cut;
}
line.text.get(cut..)?.to_owned()
} else {
if index == span_lo {
added = unit.len();
}
format!("{unit}{}", line.text)
};
insert.push_str(&body);
let terminator = text.get(line.start + line.text.len()..line.start + line.len)?;
insert.push_str(terminator);
}
let span_start = lines.get(span_lo)?.start;
let span_end = lines.get(span_hi)?.start + lines.get(span_hi)?.len;
let sel = if start == end {
let within = start.saturating_sub(span_start);
let caret = span_start.checked_add(within)?.checked_add(added)?.checked_sub(removed)?;
caret..caret
} else {
span_start..(span_start + insert.len())
};
Some(Edit { range: span_start..span_end, insert, sel })
}
/// The lines of `text` with their offsets, indent and list marker.
fn split_lines(text: &str, indent: Indent) -> Vec<Line> {
let mut lines = Vec::new();
let mut start = 0;
for raw in text.split_inclusive('\n') {
let body = raw.trim_end_matches(['\n', '\r']);
let ws = body.len() - body.trim_start().len();
lines.push(Line { start, len: raw.len(), text: String::from(body), ws, cols: indent_columns(body, indent), marker: marker_of(body) });
start += raw.len();
}
lines
}
/// Leading whitespace columns of `line`; a tab counts as one indent unit.
fn indent_columns(line: &str, indent: Indent) -> usize {
let unit = usize::from(indent.width.clamp(1, 8));
line.chars().take_while(|ch| matches!(*ch, ' ' | '\t')).map(|ch| if ch == '\t' { unit } else { 1 }).sum()
}
/// The list marker of `line`, when it starts one.
fn marker_of(line: &str) -> Option<Marker> {
let rest = line.trim_start();
let ws = line.len() - rest.len();
if let Some(mark) = rest.chars().next()
&& matches!(mark, '-' | '*' | '+')
{
let after = rest.get(mark.len_utf8()..).unwrap_or("");
return (after.is_empty() || after.starts_with(' ')).then_some(Marker::Bullet);
}
let digits = rest.bytes().take_while(u8::is_ascii_digit).count();
let after = rest.get(digits..).unwrap_or("");
let tail = match after.get(..1) {
Some("." | ")") => after.get(1..).unwrap_or(""),
_ => return None,
};
if digits == 0 || !(tail.is_empty() || tail.starts_with(' ')) {
return None;
}
Some(Marker::Ordered(ws..ws + digits))
}
/// The list items of `lines`: each with the item it hangs under.
fn items_of(lines: &[Line]) -> Vec<Item> {
let mut items: Vec<Item> = Vec::new();
let mut stack: Vec<(usize, usize)> = Vec::new();
for (index, line) in lines.iter().enumerate() {
if line.marker.is_none() {
continue;
}
while stack.last().is_some_and(|(cols, _)| *cols >= line.cols) {
stack.pop();
}
let parent = stack.last().map(|(_, item)| *item);
items.push(Item { line: index, parent });
stack.push((line.cols, items.len() - 1));
}
items
}
/// The item whose line holds `pos`.
fn item_at(items: &[Item], lines: &[Line], pos: usize) -> Option<usize> {
let line = line_of(lines, pos);
items.iter().enumerate().find(|(_, item)| item.line == line).map(|(index, _)| index)
}
/// The last line of the item's block: the item line, its children and their
/// content. A deeper line belongs to the item, a line at its own level ends it.
fn block_end(lines: &[Line], item: &Item) -> usize {
let item_cols = lines.get(item.line).map_or(0, |line| line.cols);
let mut index = item.line + 1;
let mut fence: Option<(char, usize)> = None;
while let Some(line) = lines.get(index) {
if let Some((mark, count)) = fence {
if closes_fence(&line.text, mark, count) {
fence = None;
}
index += 1;
continue;
}
if line.text.trim().is_empty() {
if deeper_follows(lines, index + 1, item_cols) {
index += 1;
continue;
}
break;
}
if line.cols > item_cols {
if let Some((mark, count)) = opens_fence(&line.text) {
fence = Some((mark, count));
}
index += 1;
continue;
}
break;
}
index.saturating_sub(1)
}
/// Whether the next non-blank line after `from` is deeper indented than the item.
fn deeper_follows(lines: &[Line], from: usize, item_cols: usize) -> bool {
lines.iter().skip(from).find(|line| !line.text.trim().is_empty()).is_some_and(|line| line.cols > item_cols)
}
/// The fence a line opens: the marker character and its run length.
fn opens_fence(line: &str) -> Option<(char, usize)> {
let trimmed = line.trim_start();
let mark = trimmed.chars().next()?;
if !matches!(mark, '`' | '~') {
return None;
}
let count = trimmed.chars().take_while(|ch| *ch == mark).count();
(count >= 3).then_some((mark, count))
}
/// Whether `line` closes the fence opened by `mark` and `count`.
fn closes_fence(line: &str, mark: char, count: usize) -> bool {
let trimmed = line.trim_start();
if !trimmed.starts_with(mark) {
return false;
}
let run = trimmed.chars().take_while(|ch| *ch == mark).count();
run >= count && trimmed.get(run..).is_some_and(|rest| rest.trim().is_empty())
}
/// The numbers the ordered items of the list get after the move.
fn new_numbers(items: &[Item], lines: &[Line], siblings: &[usize], current: usize, other: usize) -> Option<HashMap<usize, String>> {
let mut order = siblings.to_vec();
let here = order.iter().position(|index| *index == current)?;
let there = order.iter().position(|index| *index == other)?;
order.swap(here, there);
let mut numbers = HashMap::new();
let mut counter = 0u32;
for index in order {
let item = items.get(index)?;
let line = lines.get(item.line)?;
if let Some(Marker::Ordered(digits)) = &line.marker {
counter += 1;
let number = counter.to_string();
if line.text.get(digits.clone()) != Some(number.as_str()) {
numbers.insert(item.line, number);
}
}
}
Some(numbers)
}
/// The line with the number `number` in place of the old one.
fn renumbered(line: &Line, number: &str) -> Option<String> {
let Some(Marker::Ordered(digits)) = &line.marker else {
return Some(line.text.clone());
};
let mut out = String::with_capacity(line.text.len() + number.len());
out.push_str(line.text.get(..digits.start)?);
out.push_str(number);
out.push_str(line.text.get(digits.end..)?);
Some(out)
}
/// The items the selection touches: the item at the cursor for an empty
/// selection, otherwise the items whose block meets the selection, without the
/// items nested in another selected item.
fn selected_items(items: &[Item], lines: &[Line], start: usize, end: usize) -> Option<Vec<usize>> {
if start == end {
return Some(vec![item_at(items, lines, start)?]);
}
let first = line_of(lines, start);
let last = line_of(lines, end);
let mut selected = Vec::new();
for (index, item) in items.iter().enumerate() {
let block = (item.line, block_end(lines, item));
if block.0 > last {
break;
}
if block.1 >= first && block.0 <= last {
selected.push(index);
}
}
let roots = selected.iter().copied().filter(|index| !selected.iter().any(|other| *other != *index && is_ancestor(items, *other, *index))).collect();
Some(roots)
}
/// Whether `ancestor` hangs above `item`.
fn is_ancestor(items: &[Item], ancestor: usize, item: usize) -> bool {
let mut current = items.get(item).and_then(|item| item.parent);
while let Some(index) = current {
if index == ancestor {
return true;
}
current = items.get(index).and_then(|item| item.parent);
}
false
}
/// The index of the line that holds `pos`.
fn line_of(lines: &[Line], pos: usize) -> usize {
lines.iter().rposition(|line| line.start <= pos).unwrap_or(0)
}
/// The bytes at the start of `line` one outdent removes.
fn removable(line: &str, indent: Indent) -> usize {
if line.starts_with('\t') {
return 1;
}
let spaces = line.len() - line.trim_start_matches(' ').len();
spaces.min(usize::from(indent.width.clamp(1, 8)))
}
/// The selection bounds: floored to char boundaries, clamped to the text and ordered.
fn sel_bounds(text: &str, sel: Range<usize>) -> (usize, usize) {
let Range { start, end } = sel;
let start = floor_boundary(text, start);
let end = floor_boundary(text, end);
(usize::min(start, end), usize::max(start, end))
}
/// The largest offset at or below `pos` that is on a char boundary.
fn floor_boundary(text: &str, pos: usize) -> usize {
let pos = usize::min(pos, text.len());
if text.is_char_boundary(pos) {
return pos;
}
let mut at = pos;
while at > 0 && !text.is_char_boundary(at) {
at -= 1;
}
at
}
+164
View File
@@ -0,0 +1,164 @@
//! Outliner: moving list items with their children, and shifting subtrees.
use std::fmt::Write as _;
use std::time::{Duration, Instant};
use notescraft_editor::keys::{Edit, Indent};
use notescraft_editor::{Direction, indent_subtree, move_item};
/// The text with the edit applied.
fn applied(text: &str, edit: &Edit) -> String {
let mut out = String::from(text);
out.replace_range(edit.range.clone(), &edit.insert);
out
}
#[test]
fn outliner_moves_an_item_down_and_up() {
let text = "- a\n- b\n- c\n";
let edit = move_item(text, 4..4, Direction::Down, Indent::default()).unwrap();
assert_eq!(applied(text, &edit), "- a\n- c\n- b\n");
let edit = move_item(text, 8..8, Direction::Up, Indent::default()).unwrap();
assert_eq!(applied(text, &edit), "- a\n- c\n- b\n");
}
#[test]
fn outliner_moves_the_children_with_their_item() {
let text = "- a\n - a1\n- b\n";
let edit = move_item(text, 0..0, Direction::Down, Indent::default()).unwrap();
assert_eq!(applied(text, &edit), "- b\n- a\n - a1\n");
let text = "- a\n - a1\n - a2\n - a3\n- b\n";
let edit = move_item(text, 0..0, Direction::Down, Indent::default()).unwrap();
assert_eq!(applied(text, &edit), "- b\n- a\n - a1\n - a2\n - a3\n");
}
#[test]
fn outliner_moves_across_a_task_item() {
let text = "- a\n- [ ] task\n- c\n";
let at = text.find("- c").unwrap();
let edit = move_item(text, at..at, Direction::Up, Indent::default()).unwrap();
assert_eq!(applied(text, &edit), "- a\n- c\n- [ ] task\n");
}
#[test]
fn outliner_renumbers_an_ordered_list() {
let text = "1. a\n2. b\n3. c\n";
let at = text.find("3. c").unwrap();
let edit = move_item(text, at..at, Direction::Up, Indent::default()).unwrap();
assert_eq!(applied(text, &edit), "1. a\n2. c\n3. b\n");
}
#[test]
fn outliner_leaves_the_first_and_the_last_item_alone() {
let text = "- a\n- b\n";
assert!(move_item(text, 0..0, Direction::Up, Indent::default()).is_none());
assert!(move_item(text, 4..4, Direction::Down, Indent::default()).is_none());
assert!(move_item(text, 0..0, Direction::Down, Indent::default()).is_some());
assert!(move_item(text, 4..4, Direction::Up, Indent::default()).is_some());
}
#[test]
fn outliner_moves_a_fenced_code_block_with_its_item() {
let text = "- a\n ```\n code\n ```\n- b\n";
let edit = move_item(text, 0..0, Direction::Down, Indent::default()).unwrap();
assert_eq!(applied(text, &edit), "- b\n- a\n ```\n code\n ```\n");
}
#[test]
fn outliner_keeps_crlf_line_endings() {
let text = "- a\r\n- b\r\n";
let edit = move_item(text, 0..0, Direction::Down, Indent::default()).unwrap();
assert_eq!(applied(text, &edit), "- b\r\n- a\r\n");
}
#[test]
fn outliner_moves_items_indented_with_tabs_and_with_spaces() {
let text = "- a\n\t- b\n- c\n";
let edit = move_item(text, 0..0, Direction::Down, Indent::default()).unwrap();
assert_eq!(applied(text, &edit), "- c\n- a\n\t- b\n");
let text = "- a\n - b\n- c\n";
let edit = move_item(text, 0..0, Direction::Down, Indent::default()).unwrap();
assert_eq!(applied(text, &edit), "- c\n- a\n - b\n");
}
#[test]
fn outliner_keeps_the_cursor_on_the_moved_marker() {
let text = "- a\n- b\n- c\n";
let edit = move_item(text, 4..4, Direction::Down, Indent::default()).unwrap();
assert_eq!(edit.sel, 8..8);
assert_eq!(applied(text, &edit).get(8..9), Some("-"));
let text = "1. a\n2. b\n3. c\n";
let at = text.find("3. c").unwrap();
let edit = move_item(text, at..at, Direction::Up, Indent::default()).unwrap();
assert_eq!(edit.sel, 5..5);
assert_eq!(applied(text, &edit).get(5..6), Some("2"));
}
#[test]
fn outliner_moves_the_first_item_of_a_selection() {
let text = "- a\n- b\n- c\n";
let edit = move_item(text, 0..8, Direction::Down, Indent::default()).unwrap();
assert_eq!(applied(text, &edit), "- b\n- a\n- c\n");
assert_eq!(edit.sel, 4..4);
}
#[test]
fn outliner_moves_a_long_list_quickly() {
let mut text = String::new();
for index in 0..2000 {
writeln!(text, "- item {index}").unwrap();
}
let start = Instant::now();
let edit = move_item(&text, 0..0, Direction::Down, Indent::default());
let elapsed = start.elapsed();
assert!(edit.is_some());
assert!(elapsed < Duration::from_millis(5), "moving an item took {elapsed:?}");
}
#[test]
fn outliner_indents_an_item_with_its_children() {
let text = "- a\n - b\n - c\n- d\n";
let edit = indent_subtree(text, 0..0, false, Indent::default()).unwrap();
assert_eq!(applied(text, &edit), " - a\n - b\n - c\n- d\n");
assert_eq!(edit.sel, 4..4, "the cursor stays on its character");
}
#[test]
fn outliner_outdents_an_item_with_its_children() {
let text = " - a\n - b\n- c\n";
let at = text.find("- a").unwrap();
let edit = indent_subtree(text, at..at, true, Indent::default()).unwrap();
assert_eq!(applied(text, &edit), "- a\n - b\n- c\n");
assert_eq!(edit.sel, 0..0);
}
#[test]
fn outliner_refuses_to_outdent_below_the_surface_of_the_list() {
let text = "- a\n- b\n";
assert!(indent_subtree(text, 0..0, true, Indent::default()).is_none());
}
#[test]
fn outliner_indents_with_the_published_indent_unit() {
let text = "- a\n";
let indent = Indent { tabs: true, width: 4 };
let edit = indent_subtree(text, 0..0, false, indent).unwrap();
assert_eq!(applied(text, &edit), "\t- a\n");
}
#[test]
fn outliner_shifts_every_item_a_selection_touches() {
let text = "- a\n- b\n";
let edit = indent_subtree(text, 0..8, false, Indent::default()).unwrap();
assert_eq!(applied(text, &edit), " - a\n - b\n");
}
#[test]
fn outliner_ignores_text_outside_a_list_item() {
let text = "plain words\n";
assert!(indent_subtree(text, 0..0, false, Indent::default()).is_none());
assert!(move_item(text, 0..0, Direction::Down, Indent::default()).is_none());
}
+159
View File
@@ -0,0 +1,159 @@
//! Pasting a URL over selected text makes a markdown link; every other paste
//! stays a plain paste.
use notescraft_editor::keys::{self, Edit};
fn applied(text: &str, edit: &Edit) -> String {
let mut out = String::from(text);
out.replace_range(edit.range.clone(), &edit.insert);
out
}
fn link(text: &str, range: std::ops::Range<usize>, pasted: &str) -> Option<String> {
keys::on_paste_url(text, range, pasted).map(|edit| applied(text, &edit))
}
const URL: &str = "https://example.com";
#[test]
fn a_plain_word_becomes_a_link() {
let text = "hello world";
assert_eq!(link(text, 6..11, URL), Some(String::from("hello [world](https://example.com)")));
}
#[test]
fn a_phrase_becomes_a_link() {
let text = "the quick brown fox";
assert_eq!(link(text, 4..15, URL), Some(String::from("the [quick brown](https://example.com) fox")));
}
#[test]
fn the_cursor_ends_after_the_closing_paren() {
let text = "hello world";
let edit = keys::on_paste_url(text, 6..11, URL).unwrap();
assert_eq!(edit.sel, 34..34);
assert_eq!(edit.range, 6..11);
}
#[test]
fn brackets_in_the_selection_are_escaped() {
let text = "a [b] c";
assert_eq!(link(text, 0..7, URL), Some(String::from("[a \\[b\\] c](https://example.com)")));
}
#[test]
fn a_wrapped_selection_keeps_its_label() {
let text = "[label](old)";
assert_eq!(link(text, 0..12, URL), Some(String::from("[label](https://example.com)")));
let text = "[label]";
assert_eq!(link(text, 0..7, URL), Some(String::from("[label](https://example.com)")));
}
#[test]
fn an_angle_bracketed_url_is_unwrapped() {
let text = "hello world";
assert_eq!(link(text, 6..11, "<https://example.com>"), Some(String::from("hello [world](https://example.com)")));
}
#[test]
fn other_schemes_are_refused() {
let text = "hello";
assert!(link(text, 0..5, "javascript:alert(1)").is_none());
assert!(link(text, 0..5, "file:///etc/passwd").is_none());
assert!(link(text, 0..5, "mailto:me@example.com").is_none());
}
#[test]
fn a_url_with_spaces_is_refused() {
let text = "hello";
assert!(link(text, 0..5, "https://example.com/a b").is_none());
}
#[test]
fn a_host_without_a_dot_is_refused() {
let text = "hello";
assert!(link(text, 0..5, "http://example").is_none());
assert!(link(text, 0..5, "http://localhost:8080/x").is_some());
}
#[test]
fn the_scheme_is_case_insensitive() {
let text = "hello";
assert!(link(text, 0..5, "HTTPS://EXAMPLE.COM").is_some());
}
#[test]
fn a_url_over_2048_bytes_is_refused() {
let text = "hello";
let fits = format!("https://example.com/{}", "x".repeat(2028));
assert_eq!(fits.len(), 2048);
assert!(link(text, 0..5, &fits).is_some());
let too_long = format!("https://example.com/{}", "x".repeat(2029));
assert!(link(text, 0..5, &too_long).is_none());
}
#[test]
fn an_empty_selection_is_refused() {
let text = "hello world";
assert!(link(text, 6..6, URL).is_none());
}
#[test]
fn a_selection_inside_a_fenced_block_is_refused() {
let text = "```rust\nfn main() {}\n```";
let start = text.find("fn main").unwrap();
assert!(link(text, start..start + 9, URL).is_none());
}
#[test]
fn a_selection_inside_inline_code_is_refused() {
let text = "a `code` b";
let start = text.find("code").unwrap();
assert!(link(text, start..start + 4, URL).is_none());
}
#[test]
fn a_selection_inside_a_link_is_refused() {
let text = "[label](https://example.com)";
let start = text.find("label").unwrap();
assert!(link(text, start..start + 5, URL).is_none());
}
#[test]
fn a_multi_line_selection_is_refused() {
let text = "one\ntwo";
assert!(link(text, 0..7, URL).is_none());
}
#[test]
fn a_selection_over_500_bytes_is_refused() {
let text = "x".repeat(501);
assert!(link(&text, 0..501, URL).is_none());
let text = "x".repeat(500);
assert!(link(&text, 0..500, URL).is_some());
}
#[test]
fn a_selection_that_is_a_url_pastes_normally() {
let text = "https://example.com";
assert!(link(text, 0..19, URL).is_none());
}
#[test]
fn unicode_offsets_are_kept() {
let text = "caf\u{e9} na\u{ef}ve";
let start = text.find("na\u{ef}ve").unwrap();
let (head, tail) = text.split_at_checked(start).unwrap();
assert_eq!(head, "caf\u{e9} ");
assert_eq!(tail, "na\u{ef}ve");
assert_eq!(link(text, start..start + tail.len(), URL), Some(String::from("caf\u{e9} [na\u{ef}ve](https://example.com)")));
}
#[test]
fn a_second_paste_changes_nothing() {
let text = "hello world";
let edit = keys::on_paste_url(text, 6..11, URL).unwrap();
let linked = applied(text, &edit);
assert_eq!(keys::on_paste_url(text, 6..11, URL), Some(edit.clone()));
assert!(keys::on_paste_url(&linked, edit.sel.clone(), URL).is_none());
}
+2
View File
@@ -29,8 +29,10 @@ notescraft-plugins = { workspace = true }
notescraft-syntax = { workspace = true }
notescraft-sync = { workspace = true }
notescraft-annotate = { workspace = true }
notescraft-bases = { workspace = true }
notescraft-collab = { workspace = true }
ureq = { workspace = true }
ring = { workspace = true }
notescraft-regex = { workspace = true }
grep-regex = { workspace = true }
grep-searcher = { workspace = true }

Some files were not shown because too many files have changed in this diff Show More