Files
jyviko d063e81eff Format machinery and design container in a shared crate: solidcraft-format (#14997)
* Format machinery in a shared crate: solidcraft-format, designs and drawings registered with it

What every versioned file format needs moves out of crates/doc into the new shared crate
`solidcraft-format` (crates/format, serde_json only): a family's format tag and its legacy tag,
the version registry (the hand-written steps, then one `v<N>/` folder per format with a file per
change, then the pending changes that make the next number), reading a file's format and its
`format_pending`, telling a file a newer build wrote, the upgrade that runs each step once, and,
for a build script, the generated lists of the change files (`build::lists`). Steps are
JSON-level and return the app's own error; the crate's refusals are `FormatError`, which
`DocError` takes as `Invalid` with the same text.

crates/doc keeps everything that is SolidCraft's: its 22 hand-written steps and their history,
every v*/ and pending/ change file unchanged, the DESIGN and DRAWING families (now values of the
shared type, with the app's name for messages), the one alias a format-29 design may carry for
the engineering material cards (now data on the family instead of a rule in the machinery),
`read`, `decode` and every check, and newer_file. `format::Family`, `Newer`, `FORMAT`,
`format_string`, `version_of` and `pending_names` keep their names, so no caller changes; the
number range check of a family's file is doc's own (`CheckNumbers`). build.rs lists the change
files through the shared crate and keeps its FAMILIES for the release script.

Nothing about reading or writing a file changes: the same steps run in the same order, with the
same messages and the same generated lists. The format tests that built a `Formats` and called
its upgrade now build the family around it (`Family { formats, ..DESIGN }`) and call the
family's, with the same expectations. The crate has its own tests of the registry and the
upgrade, and a README; xtask's layer table lists it as shared.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtL5M2MmcDrkcWhQeHS2K8

* Shared format crate: newer-file keeping and the ZIP container, SolidCraft's design on them

Two more pieces of the file machinery move into `solidcraft-format`, each with what is SolidCraft's
left in its app crates:

- `newer`: opening a file a newer build saved without losing what this build can't read.
  `NewerFile` (its notice, warnings and report, worded with the app's name), the comparison of
  the file with what was read of it and the putting back on saving (`compare`, `restore_into`),
  the sections and list elements left out and kept (`keep_section`, `keep_element`,
  `leave_out_unreadable`, `read_around`). crates/doc keeps what is a design's: a feature it can't
  read stays on the timeline as a placeholder, a parameter is left out, the sections a design
  can't open without, the fields a container adds; `newer_file::NewerFile` is the shared type
  and `newer_file::newer_file(&newer)` makes one for SolidCraft.
- `container`: the ZIP package (`Writer`, `Reader`): parts never extracted to paths, the
  directory checked on opening, content-addressed blobs `blobs/<sha256>.<ext>` hash-checked when
  read, every size within the app's `Limits`, room for optional caches, `Compression`, and the
  `Thumbnail` keyed to its file. crates/io keeps its budgets (the preference-driven limits, the
  STEP and mesh rules), the payload encodings (meshes, point clouds), the HDR and texture
  manifests, the model cache, the preview's meshes and its container tag; `DesignCompression` is
  the shared `Compression`, and the shared crate's refusals read as a design container's.

Files are read and written as before, byte for byte: the same parts, names, levels and keys, the
same checks in the same order, the same messages. The io and doc tests are unchanged. The shared
crate has its own tests of the container (parts, blobs, budgets, the thumbnail's key) and of
newer-file keeping.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtL5M2MmcDrkcWhQeHS2K8

* io: the model cache key covers the shared format crate

The saved model is read through solidcraft-format now, so a change there must invalidate
cached models as a change in doc does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtL5M2MmcDrkcWhQeHS2K8

* Cargo.lock: solidcraft-format at the workspace's 0.40.0

The merge with main kept the new crate's lock entry at 0.40.0-rc.4 while
main's release moved the workspace to 0.40.0, so every --locked cargo call
refused to update the lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DtL5M2MmcDrkcWhQeHS2K8

---------

Co-authored-by: jyviko <2501648+jyviko@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 00:39:47 +00:00

4.8 KiB

solidcraft-format

Versioned file formats: what an app needs to read every file it ever wrote, to open a file a newer build of it wrote without losing what it can't read, and to package a file with its binary payloads. It holds no app types; each app registers its own families of files and their versions, and keeps its own payload encodings.

A file starts with "format": "<prefix>N". A family's versions are, in order: the steps written out by hand for its first formats, then one folder per format (v<N>/, one file per change, each with a JSON step when older files are rewritten as they open), then the changes in flight (pending/), which make the next number until a release folds them into v<N>/. App code only knows the current format: all translation lives in the steps.

API

Item What it does
Family { app, noun, prefix, legacy_prefix, untagged, aliases, formats } One kind of file: its tag, its messages, its versions
Formats { hand, folded, pending } A family's versions: hand-written steps, folded formats and their changes, pending changes
Part { name, step } One change: its file's name, and its JSON step if it has one
Step<E> fn(&mut Value) -> Result<(), E>: a JSON-level step, returning the app's error
Alias { format, name, known_as } Another spelling of a change name that files of one format may carry
Family::current() The format this build writes
Family::format_string(n), Family::version_of(tag) A format's tag, a tag's format (the legacy prefix read too)
Family::pending_names() The changes in flight, written into a file's format_pending
Family::take_version(&mut json) A file's format and its format_pending, taken out before anything else is read; a file not of the family is refused
Family::newer(version, applied) Some(Newer) when a newer build wrote the file: a number past this build's, or a change it doesn't know
Family::upgrade(&mut json, version, applied) Run the steps that bring the file to this build's format, each once
Newer { format, reads_up_to, unknown_changes } How a file shows a newer build wrote it
FormatError This crate's refusals; the app's error takes it with From
build::lists(manifest, families) For a build script: the generated FOLDED and PENDING lists and a module per change file
newer::NewerFile What a file a newer build saved holds that this build doesn't read: new(app, &newer), notice, warnings, report, keep_section, keep_element, compare (the file against what was read of it), restore_into (put it all back on saving), mentions, needs_container, and the container's unknown parts and container
newer::leave_out_unreadable, newer::read_around Leave out, and keep, the top-level sections a file can't be read with
newer::short, newer::looks_default A reader's error's first line; a value a writer leaves out
container::Writer A ZIP container being written: part, blob (content-addressed, blobs/<sha256>.<ext>), room/space for optional caches, finish
container::Reader One being read over a seekable source, its directory checked on opening: document, limited_part, blob, limited_blob, has_part, part_names, hydrated
container::Limits The app's budgets: entries, expanded size, per-part limits, Deflate level, file size
container::Compression, container::Thumbnail, container::digest, container::is_container Fast or compact payloads; a PNG picture keyed to its file; the SHA-256 name of bytes; the ZIP signature

Registering a family

The app's build script lists the folders:

use solidcraft_format::build::Family;
// format 1 is the first record, so its folded formats start at 2 (`base` is the last format before them)
const FAMILIES: [Family; 1] = [Family { folder: "src/format", base: 1, list_prefix: "" }];
let lists = solidcraft_format::build::lists(&manifest, &FAMILIES)?;
// write lists.code to OUT_DIR, rerun on each of lists.watch

and its format module includes them, with its error type:

type Part = solidcraft_format::Part<AppError>;
include!(concat!(env!("OUT_DIR"), "/format_parts.rs"));
pub const DESIGN: solidcraft_format::Family<'static, AppError> = solidcraft_format::Family {
    app: "App", noun: "design", prefix: "app/", legacy_prefix: None, untagged: None, aliases: &[],
    formats: solidcraft_format::Formats { hand: &[], folded: FOLDED, pending: PENDING },
};

A change file is v<N>/<name>.rs or pending/<name>.rs, named with lowercase letters, digits and dashes, starting with a //! note (what is saved, what an older file gets, what the last release does with a file that has it), and with pub fn step(v: &mut Value) -> Result<(), AppError> when older files are rewritten as they open.