mirror of
https://github.com/storytold/craft-ai-integrations.git
synced 2026-11-02 18:06:49 +00:00
main
110 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jeremy
|
950a8e17a2 |
fix(ai-client): reject plans belonging to another app (#118)
* fix(ai-client): reject plans belonging to another app * test(ai-client): keep app-scope mutation additions independent |
||
|
Jeremy
|
14a7fb8696 | fix(ai-client): bind sample approval to reviewed disclosure (#117) | ||
|
imperator
|
e8f320321a |
feat(ai-engine-burn): InSPyReNet for Remove background, built in only (#112)
* feat(ai-engine-burn): InSPyReNet for image.matte (Remove background) A second model on the Burn engine: `inspyrenet` answers `image.matte` from the picture alone, a grayscale PNG at the picture's size (white is the subject, soft along hair and fur). `remove-background` has had no local model; this is one. - Network: InSPyReNet with the Swin-B backbone, written by hand in Burn after the MIT reference code. Module names follow the PyTorch keys, so the published .safetensors loads by name through Burn's PyTorch adapter. A load that leaves a weight unset, fails one, or finds a foreign tensor is refused in plain words. - Any picture size: full resolution up to 1280 px on the long side; above that a reflect-padded square reduced with antialiased Lanczos-3, the matte enlarged back and refined at full resolution with a colour guided filter on the tiles near the edge (the Gyre approach), then a 0.2 fringe shrink. - Weights: 1038lab/inspyrenet inspyrenet.safetensors at a pinned revision, checked tensor for tensor against the author's ckpt_base.pth. Checked on one Windows machine (release): every stage against transparent-background's PyTorch code (worst relative difference 3.2e-5 on the processor, 2.4e-4 on the card, no pixel changing side); the shared conformance suite; on a real PNG the mask is byte-identical to the same port in PhotoCraft. Four new deliberate-bug cases in `be`, all caught. Not done: a catalogue recipe (a recipe names a ComfyUI workflow today), clean timings, macOS and Linux runs, a run through an app. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(ai-features): a recipe that is built in only, InSPyReNet for Remove background `LocalRecipe::comfyui` (default true) marks a recipe ComfyUI runs. With false it is only its built-in routes: never a ComfyUI setup option, plan, install offer or connection, and engine advice does not list ComfyUI for a tool without a ComfyUI recipe. `Feature::comfyui_recipes()` gives those recipes; `engine_recipe`, `plan_native`, `setup_offer_native` and `enable_native` are unchanged. `remove-background` gets one such recipe: InSPyReNet on Burn, the pinned `inspyrenet.safetensors` (1038lab/inspyrenet @ 0f594663, 367,120,444 bytes) in the folder `inspyrenet`, so Set up AI offers one checked download and no ComfyUI. Tests: the catalogue, setup options, plans and offers (`ai-features/tests/built_in_only.rs`), registration and advice (`ai-client/tests/built_in_only.rs`), and through `AiClient` on Burn (`ai-client/tests/burn_matte.rs`, the real run ignored). Mutation group `bo`: 8 of 8 caught. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(tools): retarget two `lc` mutation cases at the ComfyUI recipe lookups `unrunnable-recipe-accepted` and `second-recipe-takes-plain-id` searched for the `feature.locals` lookups in `ai-client/src/local.rs`, which now go through `Feature::comfyui_recipes()`. Same mutations, new targets; group `lc`: 21 caught, one known equivalent. Every case in `GROUPS` again has exactly one target (640 checked). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(ai-engine-burn): InSPyReNet runner checks around a stand-in, readout without panic CI has no graphics card, only a software renderer (lavapipe, WARP), and the engine takes it. There the tiny network was too slow for the conformance suite's poll limit, and on lavapipe the readout failed. Unoptimised, Burn's processor backend takes minutes for the same network. - The loaded model runs the pipeline around a `Predict`: the network, or in tests a stand-in, as the SAM 2.1 tests here use a fake backend. The conformance suite and the mask test use the stand-in (debug: the crate's unit tests in about 18 s). - The tiny network still runs through load and run on the processor in an ignored release test (about 4 s). The runner's device choice is a field, so tests can name the processor. - The prediction is read with `try_into_data`, so a device that fails while running is an error, not a guarded panic. - docs/BURN-ENGINE.md: the test split and that a software renderer counts as a card. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(ai-engine-burn): InSPyReNet moves to the processor when the card fails A card can answer the engine's probe and still fail on the network: on CI a software renderer (lavapipe) failed while reading the answer back. The first failure on a card that is not a cancel now builds the network again on the processor from the same file, runs that picture again there with a notice, and stays on the processor (`inspyrenet/fallback.rs`). A network loaded on the processor in the first place has nothing to move to. Tests with stand-ins: a working card is kept; a failing one moves once and stays; a cancel is not a failure; a rebuild that fails is the answer. Mutation group `be`: three new cases, all seven InSPyReNet cases caught. A real photo on the card still runs on the card. docs/BURN-ENGINE.md: the move, and the client run's picture is 2848 x 4272, not 3101 x 1613. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(ai-engine-burn): the move to the processor stays only after a rebuild that worked The rebuild is tried once. If it fails, its error is the answer and later pictures go to the card again, as `a_rebuild_that_fails_is_the_answer` already pins. The module docs and docs/BURN-ENGINE.md said the model stays on the processor without that condition. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(ai-engine-burn): retry a failed InSPyReNet load on the card on the processor The move to the processor covered failures while predicting, not a load on the card that fails. `load_or_processor` now retries such a load on the processor, unless the file itself is not the checkpoint (`Unsupported`), which is refused at once rather than read a second time. A network loaded on the processor in the first place is loaded once. Tests: a card that loads keeps the network; one that fails loads on the processor; a bad file is not read twice; on the processor there is one load. Mutation group `be`: two new cases, all nine InSPyReNet cases caught; every case in the checker has one target (645). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * style(ai-engine-burn): method references in the guided filter, one name for the fallback path Three SonarCloud findings on #112: two closures in `guided.rs` become method references, and the mutation cases name `inspyrenet/fallback.rs` once. No change in behaviour. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: imperator <imperator.maximus@gmail.com> |
||
|
imperator
|
bcec558fb6 |
fix(ai-engine-burn): run the processor fallback on every core with SIMD (#113)
The crate turns off Burn's default features, which also dropped `rayon` and `simd`: burn-ndarray was built with `std` only, so the processor fallback ran on one core with scalar maths. Turning both on gives burn-ndarray `multi-threads` and `simd`. Measured on the InSPyReNet network (640 x 333 picture, processor, i5-12600KF, release): 22.8 s -> 6.2 s, masks identical to one pixel by one level. SAM 2.1's processor timing was not re-measured; the doc says so. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Dylan M. Taylor
|
95b1e6a41d | ci: add SonarCloud scan via shared reusable workflow (#104) | ||
|
Jeremy
|
60b84aed84 | fix(ai-client): keep linked model folders out of downloads (#109) | ||
|
skylar taylor-barrick
|
6f7078666a | test(ai-engine): cover queued cancellation and clarify widget qualification (#106) | ||
|
skylar taylor-barrick
|
0e55340771 |
Migrate shared AI widgets to egui 0.37 (#105)
* refactor(ai-egui): migrate shared widgets to egui 0.37 * egui037: qualify shared AI widgets and downstream dependency pins * docs(ai-egui): record the committed migration qualification * fix(ci): provide Vulkan snapshots and allow pinned fork sources * test: make cancellation-before-load coverage deterministic |
||
|
Jeremy
|
e2c23c1c2b | path for developer AI suggestions- #103 | ||
|
dependabot[bot]
|
1759a52660 |
ci(deps): bump Swatinem/rust-cache
Bumps the actions group with 1 update in the / directory: [Swatinem/rust-cache](https://github.com/swatinem/rust-cache). Updates `Swatinem/rust-cache` from 2.9.2 to 2.9.3 - [Release notes](https://github.com/swatinem/rust-cache/releases) - [Changelog](https://github.com/Swatinem/rust-cache/blob/master/CHANGELOG.md) - [Commits](https://github.com/swatinem/rust-cache/compare/6323deb102c322ba6fcbdcafc7e3dddab59af2b6...d1527c8f67ef9af357290475d4af4b0fab8f12e0) --- updated-dependencies: - dependency-name: Swatinem/rust-cache dependency-version: 2.9.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com> |
||
|
Saadi
|
40df73f42e |
feat(ai-comfy-env): read a model-paths file given by path
`ComfyInstall::at` only reads `<base>/extra_model_paths.yaml`, so a ComfyUI started with `--extra-model-paths-config <file>` keeps its extra folders invisible and `models_dir_for` falls back to `<base>/models/<folder>`. `with_extra_paths_file` adds the entries from the given file through the same `read_small` + `parse_extra_paths` + `add_extra` path as the base file, so an unreadable, oversized or malformed file adds nothing and the folder cap counts both files together. The caller that attaches to such a ComfyUI passes the same file it started with. Two deliberate-bug cases cover the size bound and the cap. |
||
|
Jeremy
|
7be5af833f |
feat(client): status, picker, route notes and engine advice agree with what the default route can do (#94) (#100)
* feat(client): one start button for setting up a built-in model, with no tick box unless a licence needs one (#94) Part of #94, slice 1 of 3. The shared pieces for a one-click setup existed but nothing joined them. - NativeRoute/NativeSetupPlan::needs_agreement: the licence tick shows, and a download waits for it, only when a file's licence needs agreement. SAM 2.1 needs none. - SetupSession::start/next_task/blocker: one button that checks complete files on disk or downloads, and says why it does nothing (SetupBlock). - FolderDesk::ensure_default(_at): takes the suggested folder only when none is chosen. The widget offers it (UseSuggestedFolder) instead of dead-ending at "Choose a model folder". - ToolDesk::adopt(&CheckedNative): only checked files, binds only if no choice exists, and returns the choice it kept. - Docs: the flow in use-in-your-app, LAYOUT lists the Burn crates, STATUS row. - Mutation group fr (6 cases, 6 caught); two older cases re-aimed at the changed code. Checked: fmt, strict clippy and tests for craft-ai-features, -client, -egui and -comfy-env (473 passed, 2 ignored), wasm check for craft-ai-features, every mutation group --dry clean, group fr run for real, documentation 14/14. Scripted tests only: no real download, no model run through it, no app has mounted it, no macOS or Linux run. * feat(client): try a built-in selection model with a click, and fail a selection that chose nothing (#94) Part of #94, slice 2 of 3 (stacked on slice 1). The launcher's try-it sample for Select subject sent only a description. The default built-in route (SAM 2.1 on rten) refuses descriptions and selects by clicking, so the sample failed on the default route. - sample_request_for(feature, built_in): for image.segment with a built-in route, one click on the middle of the test disc and no description. Everything else is the plain sample. - ToolDesk::start_sample: picks the sample from the tool's current choice (a built-in engine the tool's recipe lists gets the click; ComfyUI and hosted routes keep the description). - ToolDesk::finish: a selection that ran but reports matched: false is a failed test. - Docs: step 5 of the set-up flow, STATUS row. - Mutation group fr gains 2 cases (8 in all, 8 caught). Checked: fmt, strict clippy and tests for craft-ai-features, -client, -egui and -comfy-env (478 passed, 2 ignored), wasm check for craft-ai-features, group fr run for real, documentation 34/34. Scripted tests and a fake provider only: no real model, no app, no macOS or Linux run. * feat(client): status, picker, route notes and engine advice agree with what the default route can do (#94) Part of #94, slice 3 of 3 (stacked on slices 1 and 2). - AiClient::status: a built-in model whose weights file has gone is not Ready. - The picker lists such a model once, as one to set up again, not also as a ready one. - NativeRoute::note: the setup option, the picker row and its description say the built-in SAM 2.1 route selects by clicking on the picture, instead of offering a prompt it refuses. - NativeRoute::fast_on_processor (rten, one Windows machine, about 35 ms a click, not a benchmark): engine advice gives it Pace::Processor, not "slow", and ranks it level with a card engine, so the recipe's first engine is the suggestion with or without a graphics card. - Mutation group fr gains 6 cases (14 in all, 14 caught); the Burn route case is re-aimed. Checked: fmt, strict clippy and tests for craft-ai-features, -client, -egui and -comfy-env (483 passed, 2 ignored), wasm check for craft-ai-features, every mutation group --dry clean, group fr run for real. Documentation 66/67; the one flagged is an untouched neighbour of a doc line. Scripted tests only: no real download, no model run through it, no app, no macOS or Linux run. * fix(client): one real start button, a failed check falls through to download, folder order (#98 review) Fixes the CodeRabbit findings on #98 that held up. - The setup widget showed "Check existing files" and "Download model files" side by side when complete files were on disk, so a person could start a download instead of the check. It now draws one start button from SetupSession::next_task and blocker, and its click calls SetupSession::start. The licence tick shows only when the next task is a download. - After a failed check, start kept choosing the check again, so files that did not match were a dead end. The session now remembers the folder that failed and downloads next; a different folder, or a different target, is checked again. - FolderDesk::ensure_default needed the computer's suggested folder even when the person had already chosen one. It now looks at chosen folders first. - The adopting guide told a host to call ensure_default before drawing the first screen, which makes the folder before the person accepts it. It now says to call it only on their yes. Checked: fmt; strict clippy; client tests and egui tests pass except the existing panel::a_dead_server_is_reported flake, which passed 3 of 3 alone; docs bar 38/38; mutation group fr 11/11 caught. Group nw had a case aimed at the old two-button code; it is re-aimed and caught. * fix(client): the sample copies only the small result outputs, not whole pictures (#99 review) finish cloned every output to read the selection result, copying large inline pictures and videos. It now copies only inline JSON, the form of a selection result. Behaviour is unchanged, so no test can see the copy; the existing sample tests still pass. Checked: fmt, strict clippy, client tests (309 passed), docs bar 64/64, mutation group fr 13/13. * fix(client,engine-candle,features): a selection sample needs its result, Candle sends it, small card pace (#100 review) Fixes the CodeRabbit findings on #100. - A segmentation sample passed when the route returned a mask and no Found result. The contract gives every route both, so the sample now fails with "did not say whether it selected anything". Other tools are unchanged. - Candle SAM 3 returned only a mask, which would now fail that test. It sends the Found result too (matched with its score, or nothing found when the model gave no prediction). - A processor-quick engine whose matching card is too small was called slow. It is now judged by the processor, as the card is not needed. - The adopting guide said every built-in selection model rejects words. It now says the sample uses a click, which every built-in route reads, and names the new rule. Docs bar: undocumented neighbours of the changed hunks are documented, the chain walked up to the first documented function (tests/engine.rs helpers and tests, three Candle runner methods). Checked: fmt; strict clippy (features, client, engine-candle); tests (features, client, candle lib: 440 passed); wasm check of craft-ai-features; docs bar 124/124; mutation group fr 22/22 caught, with three new cases. Not run: the live Candle test (needs the SAM 3 weights), so Candle sending Found is covered by a unit test of the result, not by a real run. * fix(client): only the connection this suite saves gets the click sample (#99 review) start_sample treated any connection whose provider is a built-in engine's name as built in. A custom connection that reuses the name `rten` may serve a model that needs a description, and the click-only request is refused by the planner before it is sent. The click sample now needs the engine's own saved connection id as well. Others keep the description sample. Checked: fmt, strict clippy, client tests (310 passed), docs bar 65/65, mutation group fr 14/14 caught (one new case; 'sample-ignores-the-route' re-aimed at the changed line). * fix(egui): show suggested folder before creating it The suggestion action creates the folder. Display its actual path first, and hide the action if no suggestion can be made. Keep the native path helper available in the egui library, not only its tests. Checked: fmt, strict clippy, 492 focused tests, wasm, 24/24 first-run mutations, docs 134/134. * fix(egui): keep the folder suggestion idle-only and test no suggestion The suggestion now waits for an active check to finish. Tests guard systems without a suggested path and exercise a busy session with discovered read-only files. Checked: 493 affected tests, fmt, strict clippy, wasm, 25/25 first-run mutations, docs 135/135. * fix(client): recover from bad files in a new approved folder A failed check now blocks reuse of the rejected directory and the previously shown destination. The person chooses a fresh folder, and the worker creates its model directory only if absent. The old files remain untouched, even if the newly approved directory appears before the worker begins. Scripted transfer and retry regressions: 388 affected tests, fmt, strict clippy, wasm, 15/15 first-run mutations, docs 49/49. Four unrelated native-feature mutations survive on upstream main too. * fix(client): explain why a fresh model folder could not be made Preserve the atomic create and distinguish an occupied destination, a missing approved parent and permission denial. Red regressions cover missing and occupied paths, and a mutation checks the message. Checked: 390 affected tests, fmt, strict clippy, wasm, 16/16 first-run mutations, docs 66/66. * fix(client): try another model under the built-in connection with words The sample picked a click whenever the connection was the one this suite saves for the engine, without checking the model. Connection management can put another model under that id, one that wants a description, and the click sample then failed for the wrong reason. Match the model the engine's recipe names as well. --------- Co-authored-by: scratch <s@s> |
||
|
Jeremy
|
2e846daff0 |
fix(client): keep an intervening choice when the offered model disappeared
Check for an existing suite choice before validating the offer, within the same store update. A red regression covers an offer whose connection disappeared after someone else made a choice. Checked: fmt, strict clippy, 299 client tests, docs 34/34; removal of the guard is caught by the regression. |
||
|
Jeremy
|
58329bffaf |
fix(client): accepting an offer checks for a newer choice inside the write (#101 review)
accept_recommended looked at the tool's choice, then wrote its own in a second step. A choice made by another app between the two would be overwritten, although the doc promised it is kept. The check now happens inside the same store lock as the write: ToolDesk::choose_if_unchosen writes only when the tool has no suite-wide choice and says whether it did. choose is unchanged. Docs bar: the existing tests/tools.rs had no docs, so adding one flagged its neighbour up the file; all its helpers and tests now have a one-line doc. Checked: fmt, strict clippy, client tests (298 passed). By hand: with the check removed, the new test fails. Not run: a real two-app race. |
||
|
Jeremy
|
d654639d82 |
fix(client): offering a model for an unchosen tool no longer chooses it (#94)
ToolDesk::auto_bind chose a model for every tool with none, local first and otherwise the first hosted one, with no yes from the person. It only wrote the shared choice (nothing was downloaded or loaded), but a hosted service could be chosen that way, and the suite's rule is that a model is chosen for the person only on their yes. Only tests called it. It is replaced by ToolDesk::auto_bind_offer, which returns what could be chosen and writes nothing. The person's yes is ToolDesk::accept_recommended, which applies exactly what was shown and skips a tool that got a choice in the meantime. The preference (this computer first) and the rule that a choice the person made is never covered are unchanged and still tested. Checked: fmt, strict clippy, package tests (297 passed). By hand: with the offer also making the choice, the tools test fails. Not added to tools/mutation (a new group would clash with the registration line in #98). |
||
|
Jeremy
|
d914216822 |
test(ai-comfyui): scripted mask replies close the connection, like the JSON replies do (#94)
The scripted servers answered /view with Content-Length but no Connection: close, then dropped the socket. A client that pools connections can reuse a socket the server has already closed, which shows up as "could not reach ComfyUI". The JSON replies already send Connection: close; the raw mask replies did not. main failed once on ubuntu in sam3_candidates this way, we think. Not proven: the failure did not reproduce on Windows in 30 + 80 runs of the test binary or 400 repeated candidates() calls on one session, so this is hardening, not a verified fix. Fixture change only, 11 reply headers in 6 test files. Checked: fmt, strict clippy, package tests (183 passed, 14 ignored), documentation 11/11. |
||
|
Jeremy
|
31475e9a2d |
fix(deps): update ctutils to 0.4.5 for RUSTSEC-2026-0385 (#94) (#96)
* fix(deps): update ctutils to 0.4.5 for RUSTSEC-2026-0385 (#94) cargo deny failed on main: ctutils 0.4.3 has non-constant-time mask generation, fixed in 0.4.4. It is reached through digest, so it sits in the hashing path only. Cargo.lock change only; cargo check -p craft-ai-comfy-env --locked passes. * fix(deps): update clipboard-win to 5.4.3, 5.4.1 was yanked (#94) cargo deny denies yanked crates (deny.toml: yanked = deny). clipboard-win 5.4.1 arrives through arboard, egui-winit and eframe (a dev dependency of craft-ai-egui, Windows only). Cargo.lock only; cargo check -p craft-ai-egui --all-targets --locked passes. A scan of all 922 registry crates in Cargo.lock against the crates.io index found no other yanked version. |
||
|
Jeremy
|
fe5748da8d |
docs(adopting): guides for using, declaring and serving AI tools (#93)
* docs(adopting): guides for using, declaring and serving AI tools, with every code block run as a test * docs(adopting): address review: keyed register example, cancel that may be unsupported, indented fences, guide tests in the README * docs(adopting): cancel may be unsupported in the provider table and app guide; a job may end straight from Queued --------- Co-authored-by: scratch <s@s> |
||
|
Jeremy
|
1a02afd6be |
feat(client): offer SAM 2.1 on Burn as an optional engine after rten (#91)
* refactor(engine-rten): let another engine reuse the SAM 2.1 request handling Loaded, Backend, Prediction, read_prompt, prompt_points and best_proposal become public. Visibility only: the click and box mapping, the picture kept between requests, the empty-mask answer and the size checks then live in one place for every SAM 2.1 engine. * feat(engine-burn): crate shell and the script that generates the SAM 2.1 architecture tools/burn/generate_sam21.py checks the two published ONNX files against the pinned size and SHA-256, runs the pinned burn-onnx 0.22.0 (an example of craft-ai-burn-setup), applies the five asserted fixes, drops the generated loaders that panic on a bad file, splits by submodule so every file is under 700 lines, formats and writes. --check proves the tree is what the script produces. REVISION must equal the saved-weights cache revision. Cargo.lock additions are Burn with wgpu and ndarray (generated). * feat(burn-setup): prepare SAM 2.1 weights for Burn once, on this computer Brings in craft-ai-burn-setup (feature `conversion`): checks the two published ONNX files by size and SHA-256, converts them to Burn weights with the pinned burn-onnx 0.22.0, and keeps one checked, locked, shared copy. Nothing is hosted or downloaded. Mutation group bp (10 cases). Cargo.lock additions are the optional burn-onnx tree (generated). * feat(engine-burn): GENERATED SAM 2.1 tiny architecture (30 files) Output of tools/burn/generate_sam21.py, revision sam21-tiny-burn-0.22.1-onnx-0.22.0-v1, from the published encoder.onnx and decoder.onnx with burn-onnx 0.22.0 and its five asserted fixes. Do not review line by line; rerun the script with --check instead. Longest file 699 lines. Marked linguist-generated. * feat(engine-burn): SAM 2.1 runner for image.segment on Burn Sam21Runner (engine burn, model sam21) over the generated architecture. The request handling, picture kept between requests, empty-mask answer and click/box mapping are the shared Loaded from craft-ai-engine-rten, so only the model run lives here. Weights load through BurnpackStore and come back as errors; every call into the generated code is guarded against panics. Graphics card when one answers, else the processor. An .onnx file is answered with 'prepare it first', not a guess. 16 unit tests on a fake backend: click and box mapping, empty mask, short mask, a failing model, shared conformance run with Cancel, damaged and truncated files. * test(engine-burn): live comparison with rten, and the shared revision check The ignored live test runs the real SAM 2.1 files through rten (published ONNX) and Burn (weights prepared on this computer) on the astronaut sample: one click and a box, best-mask IoU, timings, and a truncated prepared file refused. The revision test fails when the generator, the compiled architecture and the saved-weights cache key disagree. * test+docs(engine-burn): mutation group be, weight-loading tests and the Burn engine guide Group be: 15 cases, one target each, all caught (saved-weights marker, load completeness, panic guard, answer length, best proposal, file names, declared card makers, the revision shared by script, architecture and cache key, the catalogue route and the .bpk binding rule). fully_applied is split out so the load checks can be tested without the store's own refusals. Group nf's onnx case follows the rewritten extension check. docs/BURN-ENGINE.md: how to regenerate, the five fixes, the revision bump rule, limits. * feat(engine-burn): CRAFT_AI_BURN_DEVICE=cpu forces the processor, and the processor path is measured * feat(engine-burn): prepare the saved copy when loading from the downloaded ONNX files The host step: given the downloaded encoder.onnx with decoder.onnx beside it, the runner calls craft-ai-burn-setup (exact source bytes checked, reused when already saved) and loads from the burn-prepared folder beside the model folder. Behind the default 'prepare' feature; mutation group be gains two cases (17/17); docs say what is measured. * feat(client): optional burn feature and a catalogue route for SAM 2.1 on Burn craft-ai-client gains the cargo feature burn (ProviderFactory::standard() then knows the engine). SAM 2.1's recipe lists a burn NativeRoute after rten, with the same two published ONNX files (one download for both, nothing converted is hosted). rten stays first and the default. The binding accepts the prepared .bpk file. Engines now list rten, burn, candle; the tests that assumed the Burn slot had no files, or that no built-in engine exists whatever cargo features are on, say what they mean. burn_engine.rs runs two apps through the client with the real prepared files (ignored). * fix(burn-setup): sweep the scratch folders a killed run left, and keep craft-ai-features a dev dependency CodeRabbit on #89: tempdir_in(cache) folders hold the two source copies (about 130 MB) and were removed only by Drop, so a killed app leaked them for good. They now carry the cache key and a role in their names, and after the key's lock is taken the same key's leftovers are removed (other keys' folders, the lock file and links are left alone). Red first: leftovers_of_a_killed_run_are_swept_for_this_key_only. Mutation cases burn-killed-run-leftovers-kept and burn-sweep-takes-other-keys, both caught. craft-ai-features is only used by a unit test, so it moves to dev-dependencies. * fix(engine-burn): review findings on #90 CodeRabbit: the generator now refuses an unknown option (a mistyped --chek used to select the run that rewrites src/model; red first: tools/burn/test_generate_sam21.py failed to import parse_args); the preparation live test needs the prepare feature and is ignored by default; the refusal test no longer leaves burn-prepared/<key>.lock in the shared system temp folder (the model folder is nested one level inside the test's own folder; confirmed the old layout leaked it on this PC); docs, STATUS and the mutation README no longer describe the client feature and catalogue route, which arrive in the next PR, as available. * docs(client): say the Burn route and client feature exist, and what the status line really covers CodeRabbit on #91: the STATUS row said the host step was not built. Given the downloaded encoder.onnx the runner prepares the saved copy itself on first load; the row now says that, names the limits (no progress or Cancel during that load, no app run through the shared binding, no live run on macOS or Linux) and LOCAL-BURN-PREPARATION.md no longer calls engine registration separate work. Also restores the client feature and catalogue route in the docs and rustdoc that the #90 review fix had marked as planned, and the three mutation-README claims for them. * fix(burn-setup): sweep only the scratch folders, not a renamed copy of the key's folder CodeRabbit on #89 (second round): sweep_stale removed every folder starting with <key>., so a damaged copy a person renamed to <key>.backup would be deleted on the next preparation. It now matches only <key>.work., <key>.conv. and <key>.out., the names the scratch folders are made with (one constant for both). Red first: the sweep test gained a <key>.backup folder that must survive. Mutation group bp 13/13 caught, including the new case burn-sweep-takes-a-renamed-copy. * test(burn): the generator's usage errors must exit with code 1 CodeRabbit on #90 (second round): the parse_args tests asserted only SystemExit, which a SystemExit(0) would also satisfy. They now assert the code. Checked against a mutant that exits 0: 6 failures. * fix(engine-burn): a damaged saved copy says it is damaged and where it lives CodeRabbit on #90 (third round) pointed out that a saved copy with a bad manifest or weights fails the same way on every retry, behind the message 'could not be read or written. Try again.' It proposed deleting the copy. The rule that a derived copy is refused, never repaired or deleted for the person, stays (#89 and its tests pin it); the dead end was the wording. craft-ai-burn-setup now answers every unusable saved copy with DAMAGED_COPY (remove it, prepare again, no 'try again'), and the Burn engine adds where it lives: the burn-prepared folder beside the model folder. Tests written first (they did not compile without DAMAGED_COPY and explain): a damaged copy of three kinds gets that answer and is left untouched; the engine explains it and passes other errors on. Mutation: bp 2 new cases and be 1 new case, all caught. * fix(burn-setup): a missing or unreadable saved weight is a damaged copy too CodeRabbit on #90 (fourth round): hash_file(...)? on a saved encoder.bpk or decoder.bpk that is gone returned the generic 'try again' error before the damaged-copy branch could run. It now maps to DAMAGED_COPY. Red first: the damaged-copy test gained 'missing encoder' and 'missing decoder' cases and failed with the old message. New mutation case burn-missing-saved-weight-answered-with-try-again; the saved-hash case was re-aimed at the new line. bp 16/16 caught. * fix(burn-setup): a manifest that cannot be opened or read is a damaged copy too CodeRabbit on #91 (round 5): File::open and read_to_end on an existing manifest returned the generic 'try again' error, so the engine gave no deletion guidance. Both now map to DAMAGED_COPY. Red first: a_manifest_that_cannot_be_opened_is_a_damaged_copy (Windows: another handle holds it exclusively; Unix: mode 0, ending quietly where permissions are not enforced) failed with the old message. The Unix variant was not run here. New mutation case burn-unopenable-manifest-answered-with-try-again; bp 17/17 caught (Windows). * test(burn-setup): write the Unix permission as octal (clippy::non_octal_unix_permissions, CI only) --------- Co-authored-by: scratch <s@s> |
||
|
Jeremy
|
fdad29cdd4 |
feat(engine-burn): run SAM 2.1 for image.segment on Burn (#90)
* refactor(engine-rten): let another engine reuse the SAM 2.1 request handling Loaded, Backend, Prediction, read_prompt, prompt_points and best_proposal become public. Visibility only: the click and box mapping, the picture kept between requests, the empty-mask answer and the size checks then live in one place for every SAM 2.1 engine. * feat(engine-burn): crate shell and the script that generates the SAM 2.1 architecture tools/burn/generate_sam21.py checks the two published ONNX files against the pinned size and SHA-256, runs the pinned burn-onnx 0.22.0 (an example of craft-ai-burn-setup), applies the five asserted fixes, drops the generated loaders that panic on a bad file, splits by submodule so every file is under 700 lines, formats and writes. --check proves the tree is what the script produces. REVISION must equal the saved-weights cache revision. Cargo.lock additions are Burn with wgpu and ndarray (generated). * feat(burn-setup): prepare SAM 2.1 weights for Burn once, on this computer Brings in craft-ai-burn-setup (feature `conversion`): checks the two published ONNX files by size and SHA-256, converts them to Burn weights with the pinned burn-onnx 0.22.0, and keeps one checked, locked, shared copy. Nothing is hosted or downloaded. Mutation group bp (10 cases). Cargo.lock additions are the optional burn-onnx tree (generated). * feat(engine-burn): GENERATED SAM 2.1 tiny architecture (30 files) Output of tools/burn/generate_sam21.py, revision sam21-tiny-burn-0.22.1-onnx-0.22.0-v1, from the published encoder.onnx and decoder.onnx with burn-onnx 0.22.0 and its five asserted fixes. Do not review line by line; rerun the script with --check instead. Longest file 699 lines. Marked linguist-generated. * feat(engine-burn): SAM 2.1 runner for image.segment on Burn Sam21Runner (engine burn, model sam21) over the generated architecture. The request handling, picture kept between requests, empty-mask answer and click/box mapping are the shared Loaded from craft-ai-engine-rten, so only the model run lives here. Weights load through BurnpackStore and come back as errors; every call into the generated code is guarded against panics. Graphics card when one answers, else the processor. An .onnx file is answered with 'prepare it first', not a guess. 16 unit tests on a fake backend: click and box mapping, empty mask, short mask, a failing model, shared conformance run with Cancel, damaged and truncated files. * test(engine-burn): live comparison with rten, and the shared revision check The ignored live test runs the real SAM 2.1 files through rten (published ONNX) and Burn (weights prepared on this computer) on the astronaut sample: one click and a box, best-mask IoU, timings, and a truncated prepared file refused. The revision test fails when the generator, the compiled architecture and the saved-weights cache key disagree. * test+docs(engine-burn): mutation group be, weight-loading tests and the Burn engine guide Group be: 15 cases, one target each, all caught (saved-weights marker, load completeness, panic guard, answer length, best proposal, file names, declared card makers, the revision shared by script, architecture and cache key, the catalogue route and the .bpk binding rule). fully_applied is split out so the load checks can be tested without the store's own refusals. Group nf's onnx case follows the rewritten extension check. docs/BURN-ENGINE.md: how to regenerate, the five fixes, the revision bump rule, limits. * feat(engine-burn): CRAFT_AI_BURN_DEVICE=cpu forces the processor, and the processor path is measured * feat(engine-burn): prepare the saved copy when loading from the downloaded ONNX files The host step: given the downloaded encoder.onnx with decoder.onnx beside it, the runner calls craft-ai-burn-setup (exact source bytes checked, reused when already saved) and loads from the burn-prepared folder beside the model folder. Behind the default 'prepare' feature; mutation group be gains two cases (17/17); docs say what is measured. * fix(burn-setup): sweep the scratch folders a killed run left, and keep craft-ai-features a dev dependency CodeRabbit on #89: tempdir_in(cache) folders hold the two source copies (about 130 MB) and were removed only by Drop, so a killed app leaked them for good. They now carry the cache key and a role in their names, and after the key's lock is taken the same key's leftovers are removed (other keys' folders, the lock file and links are left alone). Red first: leftovers_of_a_killed_run_are_swept_for_this_key_only. Mutation cases burn-killed-run-leftovers-kept and burn-sweep-takes-other-keys, both caught. craft-ai-features is only used by a unit test, so it moves to dev-dependencies. * fix(engine-burn): review findings on #90 CodeRabbit: the generator now refuses an unknown option (a mistyped --chek used to select the run that rewrites src/model; red first: tools/burn/test_generate_sam21.py failed to import parse_args); the preparation live test needs the prepare feature and is ignored by default; the refusal test no longer leaves burn-prepared/<key>.lock in the shared system temp folder (the model folder is nested one level inside the test's own folder; confirmed the old layout leaked it on this PC); docs, STATUS and the mutation README no longer describe the client feature and catalogue route, which arrive in the next PR, as available. * fix(burn-setup): sweep only the scratch folders, not a renamed copy of the key's folder CodeRabbit on #89 (second round): sweep_stale removed every folder starting with <key>., so a damaged copy a person renamed to <key>.backup would be deleted on the next preparation. It now matches only <key>.work., <key>.conv. and <key>.out., the names the scratch folders are made with (one constant for both). Red first: the sweep test gained a <key>.backup folder that must survive. Mutation group bp 13/13 caught, including the new case burn-sweep-takes-a-renamed-copy. * test(burn): the generator's usage errors must exit with code 1 CodeRabbit on #90 (second round): the parse_args tests asserted only SystemExit, which a SystemExit(0) would also satisfy. They now assert the code. Checked against a mutant that exits 0: 6 failures. * fix(engine-burn): a damaged saved copy says it is damaged and where it lives CodeRabbit on #90 (third round) pointed out that a saved copy with a bad manifest or weights fails the same way on every retry, behind the message 'could not be read or written. Try again.' It proposed deleting the copy. The rule that a derived copy is refused, never repaired or deleted for the person, stays (#89 and its tests pin it); the dead end was the wording. craft-ai-burn-setup now answers every unusable saved copy with DAMAGED_COPY (remove it, prepare again, no 'try again'), and the Burn engine adds where it lives: the burn-prepared folder beside the model folder. Tests written first (they did not compile without DAMAGED_COPY and explain): a damaged copy of three kinds gets that answer and is left untouched; the engine explains it and passes other errors on. Mutation: bp 2 new cases and be 1 new case, all caught. * fix(burn-setup): a missing or unreadable saved weight is a damaged copy too CodeRabbit on #90 (fourth round): hash_file(...)? on a saved encoder.bpk or decoder.bpk that is gone returned the generic 'try again' error before the damaged-copy branch could run. It now maps to DAMAGED_COPY. Red first: the damaged-copy test gained 'missing encoder' and 'missing decoder' cases and failed with the old message. New mutation case burn-missing-saved-weight-answered-with-try-again; the saved-hash case was re-aimed at the new line. bp 16/16 caught. * fix(burn-setup): a manifest that cannot be opened or read is a damaged copy too CodeRabbit on #91 (round 5): File::open and read_to_end on an existing manifest returned the generic 'try again' error, so the engine gave no deletion guidance. Both now map to DAMAGED_COPY. Red first: a_manifest_that_cannot_be_opened_is_a_damaged_copy (Windows: another handle holds it exclusively; Unix: mode 0, ending quietly where permissions are not enforced) failed with the old message. The Unix variant was not run here. New mutation case burn-unopenable-manifest-answered-with-try-again; bp 17/17 caught (Windows). * test(burn-setup): write the Unix permission as octal (clippy::non_octal_unix_permissions, CI only) |
||
|
Jeremy
|
97dc996251 |
feat(burn-setup): prepare SAM 2.1 weights for Burn once, on this computer (#89)
* refactor(engine-rten): let another engine reuse the SAM 2.1 request handling Loaded, Backend, Prediction, read_prompt, prompt_points and best_proposal become public. Visibility only: the click and box mapping, the picture kept between requests, the empty-mask answer and the size checks then live in one place for every SAM 2.1 engine. * feat(burn-setup): prepare SAM 2.1 weights for Burn once, on this computer Brings in craft-ai-burn-setup (feature `conversion`): checks the two published ONNX files by size and SHA-256, converts them to Burn weights with the pinned burn-onnx 0.22.0, and keeps one checked, locked, shared copy. Nothing is hosted or downloaded. Mutation group bp (10 cases). Cargo.lock additions are the optional burn-onnx tree (generated). * fix(burn-setup): sweep the scratch folders a killed run left, and keep craft-ai-features a dev dependency CodeRabbit on #89: tempdir_in(cache) folders hold the two source copies (about 130 MB) and were removed only by Drop, so a killed app leaked them for good. They now carry the cache key and a role in their names, and after the key's lock is taken the same key's leftovers are removed (other keys' folders, the lock file and links are left alone). Red first: leftovers_of_a_killed_run_are_swept_for_this_key_only. Mutation cases burn-killed-run-leftovers-kept and burn-sweep-takes-other-keys, both caught. craft-ai-features is only used by a unit test, so it moves to dev-dependencies. * fix(burn-setup): sweep only the scratch folders, not a renamed copy of the key's folder CodeRabbit on #89 (second round): sweep_stale removed every folder starting with <key>., so a damaged copy a person renamed to <key>.backup would be deleted on the next preparation. It now matches only <key>.work., <key>.conv. and <key>.out., the names the scratch folders are made with (one constant for both). Red first: the sweep test gained a <key>.backup folder that must survive. Mutation group bp 13/13 caught, including the new case burn-sweep-takes-a-renamed-copy. |
||
|
Jeremy
|
ee83eab4bb |
refactor(engine-rten): let another engine reuse the SAM 2.1 request handling (#88)
Loaded, Backend, Prediction, read_prompt, prompt_points and best_proposal become public. Visibility only: the click and box mapping, the picture kept between requests, the empty-mask answer and the size checks then live in one place for every SAM 2.1 engine. |
||
|
Jeremy
|
870e96048c |
docs: list the rten engine and the selection workflow route, and say what was run (#86)
The README crates table, docs/LAYOUT.md and docs/STATUS.md named only Candle and the SAM 3 clients. They now list craft-ai-engine-rten (SAM 2.1 from the published ONNX files, on the processor), the imported image.segment route on ComfyUI with checked masks, and what was and was not run: Windows only, one picture, no Burn runner, no app through the shared binding yet. |
||
|
Jeremy
|
ed86e4443c |
feat(ai-features): offer SAM 2.1 first and keep an earlier SAM 3 choice (#85)
* feat(ai-core): share the image.segment request and result contract * fix(ai-core): take only an inline PNG as the selection mask * feat(ai-engine-rten): bounded picture decoding, mask encoding and resampling for selection models * feat(ai-engine-rten): run SAM 2.1 through the shared selection contract * fix(ai-engine-rten): bound a thin picture's memory, refuse extreme sides, and make the mask size fallible * feat(ai-features): let each built-in engine follow its own recipe A built-in engine now plans, offers, registers and checks files from the first usable recipe that lists it (`Feature::engine_recipe`), instead of every engine acting on the first usable recipe. A first recipe that lists one engine no longer hides a later recipe that lists another. The plain built-in offer and engine advice still start from the first usable recipe, and `Feature::engines` lists every engine some usable recipe names, once each, in catalogue order. The catalogue is unchanged, so no behaviour changes today; the tests build the two-recipe shape from the real SAM 3 recipe. * feat(ai-features): list SAM 2.1 on the built-in rten engine, reading the published ONNX files SAM 2.1 now lists a `rten` route beside the ComfyUI one: the two published ONNX files (`cstr/sam2.1-hiera-tiny-ONNX` at a pinned revision, size and SHA-256 pinned), read as they are, with no conversion and nothing hosted by us. A built-in route may now name `.onnx` weights as well as `.safetensors`, in the catalogue and when an app is given a weights path. SAM 2.1 stays second in the list, so nothing changes for a person who has already chosen SAM 3. The model is in the Apache-2.0 licence of Meta's SAM 2.1; the publisher of the conversion declares the same. Real-model checks (ignored unless the files are given): two apps get a mask by asking for Select subject, and a hover through the ordinary job path. * feat(ai-features): offer SAM 2.1 first and keep an earlier SAM 3 choice SAM 2.1 (Apache-2.0) is now the first recipe for Select subject, ahead of SAM 3 (SAM License), so the plain built-in offer is SAM 2.1 on rten: two files, no licence to accept. SAM 3 and Candle stay available as their own options. The plain `local-<feature>` id now goes to the recipe that already holds it in the store, or to the first recipe when no connection has it. A SAM 3 choice saved by an earlier build, when SAM 3 came first, keeps pointing at SAM 3, and enabling SAM 3 again updates that connection instead of adding a second one. Tests that assumed SAM 3 first now find a recipe by its workflow name, never by position. * docs(ai-features): the pinned SAM 2.1 export is the tiny size, not the small one A review finding: the doc comment on the ONNX file list said small, but the pinned repository is cstr/sam2.1-hiera-tiny-ONNX and the pinned sizes match the tiny variant. SAM 2.1 also has a separate small variant, so the wrong word could send a maintainer to the wrong file. * fix(ai-engine-rten): judge a match on the mask handed on, not on the raw logits A review finding: matched was set from the model's raw logits before resampling and 8-bit quantization, so one positive logit that a small picture never samples reported a match with an all-black mask. The match is now whether any pixel of the emitted mask is above half, using the same quantization as the PNG. Written first and seen to fail (matched stayed true for a speck at logit 8 in a 10 by 6 picture). The runner's raw-logit test stays as a shortcut that skips the resampling; forcing it open gives the same answer, so its mutation case is retired with the reason in the group header. * docs(mutation): point the usage text at GROUPS instead of a list that was out of date A review finding: the usage line named only the early groups and left out rt, which is new here, and also seg, gs and others. A list in the usage text goes stale each time a group is added, so it now says to take the group from GROUPS and README.md. |
||
|
Jeremy
|
d470ff7b71 |
feat(ai-features): list SAM 2.1 on the built-in rten engine, reading the published ONNX files (#84)
* feat(ai-core): share the image.segment request and result contract * fix(ai-core): take only an inline PNG as the selection mask * feat(ai-engine-rten): bounded picture decoding, mask encoding and resampling for selection models * feat(ai-engine-rten): run SAM 2.1 through the shared selection contract * fix(ai-engine-rten): bound a thin picture's memory, refuse extreme sides, and make the mask size fallible * feat(ai-features): let each built-in engine follow its own recipe A built-in engine now plans, offers, registers and checks files from the first usable recipe that lists it (`Feature::engine_recipe`), instead of every engine acting on the first usable recipe. A first recipe that lists one engine no longer hides a later recipe that lists another. The plain built-in offer and engine advice still start from the first usable recipe, and `Feature::engines` lists every engine some usable recipe names, once each, in catalogue order. The catalogue is unchanged, so no behaviour changes today; the tests build the two-recipe shape from the real SAM 3 recipe. * feat(ai-features): list SAM 2.1 on the built-in rten engine, reading the published ONNX files SAM 2.1 now lists a `rten` route beside the ComfyUI one: the two published ONNX files (`cstr/sam2.1-hiera-tiny-ONNX` at a pinned revision, size and SHA-256 pinned), read as they are, with no conversion and nothing hosted by us. A built-in route may now name `.onnx` weights as well as `.safetensors`, in the catalogue and when an app is given a weights path. SAM 2.1 stays second in the list, so nothing changes for a person who has already chosen SAM 3. The model is in the Apache-2.0 licence of Meta's SAM 2.1; the publisher of the conversion declares the same. Real-model checks (ignored unless the files are given): two apps get a mask by asking for Select subject, and a hover through the ordinary job path. * docs(ai-features): the pinned SAM 2.1 export is the tiny size, not the small one A review finding: the doc comment on the ONNX file list said small, but the pinned repository is cstr/sam2.1-hiera-tiny-ONNX and the pinned sizes match the tiny variant. SAM 2.1 also has a separate small variant, so the wrong word could send a maintainer to the wrong file. * fix(ai-engine-rten): judge a match on the mask handed on, not on the raw logits A review finding: matched was set from the model's raw logits before resampling and 8-bit quantization, so one positive logit that a small picture never samples reported a match with an all-black mask. The match is now whether any pixel of the emitted mask is above half, using the same quantization as the PNG. Written first and seen to fail (matched stayed true for a speck at logit 8 in a 10 by 6 picture). The runner's raw-logit test stays as a shortcut that skips the resampling; forcing it open gives the same answer, so its mutation case is retired with the reason in the group header. * docs(mutation): point the usage text at GROUPS instead of a list that was out of date A review finding: the usage line named only the early groups and left out rt, which is new here, and also seg, gs and others. A list in the usage text goes stale each time a group is added, so it now says to take the group from GROUPS and README.md. |
||
|
Jeremy
|
8d74c5d2b9 |
feat(ai-features): let each built-in engine follow its own recipe (#83)
* feat(ai-core): share the image.segment request and result contract * fix(ai-core): take only an inline PNG as the selection mask * feat(ai-engine-rten): bounded picture decoding, mask encoding and resampling for selection models * feat(ai-engine-rten): run SAM 2.1 through the shared selection contract * fix(ai-engine-rten): bound a thin picture's memory, refuse extreme sides, and make the mask size fallible * feat(ai-features): let each built-in engine follow its own recipe A built-in engine now plans, offers, registers and checks files from the first usable recipe that lists it (`Feature::engine_recipe`), instead of every engine acting on the first usable recipe. A first recipe that lists one engine no longer hides a later recipe that lists another. The plain built-in offer and engine advice still start from the first usable recipe, and `Feature::engines` lists every engine some usable recipe names, once each, in catalogue order. The catalogue is unchanged, so no behaviour changes today; the tests build the two-recipe shape from the real SAM 3 recipe. * fix(ai-engine-rten): judge a match on the mask handed on, not on the raw logits A review finding: matched was set from the model's raw logits before resampling and 8-bit quantization, so one positive logit that a small picture never samples reported a match with an all-black mask. The match is now whether any pixel of the emitted mask is above half, using the same quantization as the PNG. Written first and seen to fail (matched stayed true for a speck at logit 8 in a 10 by 6 picture). The runner's raw-logit test stays as a shortcut that skips the resampling; forcing it open gives the same answer, so its mutation case is retired with the reason in the group header. * docs(mutation): point the usage text at GROUPS instead of a list that was out of date A review finding: the usage line named only the early groups and left out rt, which is new here, and also seg, gs and others. A list in the usage text goes stale each time a group is added, so it now says to take the group from GROUPS and README.md. |
||
|
Jeremy
|
611195b50e |
feat(ai-comfyui): send a selection picture once and resend it only when it is gone (#82)
* feat(ai-comfyui): import a selection workflow and run it with the picture sent first Adds the `image.segment` capability to imported ComfyUI workflows. A selection workflow maps a text prompt and one source picture; the picture is uploaded, the graph is queued with the name ComfyUI gave it, and every queued graph gets a sink name of its own so a repeat is not answered from cache with no output. Clicks or a box are refused in plain words before anything is sent, and a stored workflow file may declare a source binding and no mask. The answer is the graph's image as it comes. Checking the mask is the next change. * feat(ai-comfyui): check a selection workflow's mask and answer in the shared shape A selection workflow's image is now a mask the shared contract can trust. It must be the size of the picture sent (a JPEG may come back turned a quarter, because ComfyUI's loader applies the EXIF turn), is reduced to one 8-bit channel, and is followed by the shared `Found` result. A mask with no pixel above half is an empty mask and `matched: false`, never a failed job. Picture sizes are read from the PNG or JPEG header and bounded. This is not calibrated absence detection: the example Florence graph finds a box for any phrase. * feat(ai-comfyui): send a selection picture once and resend it only when it is gone A repeat selection on the same picture reuses the file already in ComfyUI's scratch area, keyed by a hash of the bytes. Only the last four pictures are remembered. If ComfyUI has cleared its scratch area (a restart), the first refusal of a reused name drops the entry and sends the picture again once; a refusal of a fresh upload is never retried. A failed selection job clears the cache, since the file may be what went wrong. Adds the ignored live test against a real ComfyUI. * test(ai-comfyui): count refused prompts without the deprecated fetch_update Newer Rust renames AtomicUsize::fetch_update to try_update and warns, which the strict Clippy job turns into an error on CI. The scripted server handles one job at a time here, so a load and a fetch_sub do the same job and build on both toolchains. * fix(ai-comfyui): hand on a binary mask, white where selected and black elsewhere A review finding: the first channel was kept as the graph saved it, so pixels from 128 to 254 reached the app as partly selected, while the shared contract says selected pixels are white. Every pixel above half is now 255 and the rest 0. Written first and seen to fail (the answer kept [0, 100, 128, 200, 255]); mutation cases added for a grey kept as selected and a low grey kept. * docs(ai-comfyui): state the selection mask's byte limit, test it, and rename a mutation case Two review findings. The mask is read inline under the provider's output limit (never more than the 32 MiB inline cap) and never to the output folder, because the checks need its bytes in hand; that is now written in the module header and the provider, and a test shows a mask over a lowered limit fails the job and hands on nothing, with a mutation case for the limit being ignored. The case that only removed the media-type guard is renamed so it no longer implies JPEG answers are covered. The limit test passed on first run, as the behaviour already existed: it is a guard, not a red-first fix. * fix(ai-comfyui): forget the cached pictures when any selection job fails A review finding: only an error status from ComfyUI cleared the upload cache, so a job whose mask was fetched but refused (wrong size, not a PNG) left its picture cached and the next selection could reuse a stale scratch name. The cache is now cleared once, after the job's state is settled, whenever a selection job ends failed. Written first and seen to fail (a wrong-size mask left one upload for two selections); the existing mutation case is re-aimed at the single clear. |
||
|
Jeremy
|
4d4ef93a23 |
feat(ai-engine-rten): run SAM 2.1 through the shared selection contract (#79)
* feat(ai-core): share the image.segment request and result contract * fix(ai-core): take only an inline PNG as the selection mask * feat(ai-engine-rten): bounded picture decoding, mask encoding and resampling for selection models * feat(ai-engine-rten): run SAM 2.1 through the shared selection contract * fix(ai-engine-rten): bound a thin picture's memory, refuse extreme sides, and make the mask size fallible * fix(ai-engine-rten): judge a match on the mask handed on, not on the raw logits A review finding: matched was set from the model's raw logits before resampling and 8-bit quantization, so one positive logit that a small picture never samples reported a match with an all-black mask. The match is now whether any pixel of the emitted mask is above half, using the same quantization as the PNG. Written first and seen to fail (matched stayed true for a speck at logit 8 in a 10 by 6 picture). The runner's raw-logit test stays as a shortcut that skips the resampling; forcing it open gives the same answer, so its mutation case is retired with the reason in the group header. * docs(mutation): point the usage text at GROUPS instead of a list that was out of date A review finding: the usage line named only the early groups and left out rt, which is new here, and also seg, gs and others. A list in the usage text goes stale each time a group is added, so it now says to take the group from GROUPS and README.md. |
||
|
Jeremy
|
84d0d90a61 |
feat(ai-engine-rten): bounded picture decoding, mask encoding and resampling for selection models (#78)
* feat(ai-core): share the image.segment request and result contract * fix(ai-core): take only an inline PNG as the selection mask * feat(ai-engine-rten): bounded picture decoding, mask encoding and resampling for selection models * fix(ai-engine-rten): bound a thin picture's memory, refuse extreme sides, and make the mask size fallible * docs(mutation): point the usage text at GROUPS instead of a list that was out of date A review finding: the usage line named only the early groups and left out rt, which is new here, and also seg, gs and others. A list in the usage text goes stale each time a group is added, so it now says to take the group from GROUPS and README.md. |
||
|
Jeremy
|
dae74cc8a7 |
feat(ai-comfyui): check a selection workflow's mask and answer in the shared shape (#81)
* feat(ai-comfyui): import a selection workflow and run it with the picture sent first Adds the `image.segment` capability to imported ComfyUI workflows. A selection workflow maps a text prompt and one source picture; the picture is uploaded, the graph is queued with the name ComfyUI gave it, and every queued graph gets a sink name of its own so a repeat is not answered from cache with no output. Clicks or a box are refused in plain words before anything is sent, and a stored workflow file may declare a source binding and no mask. The answer is the graph's image as it comes. Checking the mask is the next change. * feat(ai-comfyui): check a selection workflow's mask and answer in the shared shape A selection workflow's image is now a mask the shared contract can trust. It must be the size of the picture sent (a JPEG may come back turned a quarter, because ComfyUI's loader applies the EXIF turn), is reduced to one 8-bit channel, and is followed by the shared `Found` result. A mask with no pixel above half is an empty mask and `matched: false`, never a failed job. Picture sizes are read from the PNG or JPEG header and bounded. This is not calibrated absence detection: the example Florence graph finds a box for any phrase. * fix(ai-comfyui): hand on a binary mask, white where selected and black elsewhere A review finding: the first channel was kept as the graph saved it, so pixels from 128 to 254 reached the app as partly selected, while the shared contract says selected pixels are white. Every pixel above half is now 255 and the rest 0. Written first and seen to fail (the answer kept [0, 100, 128, 200, 255]); mutation cases added for a grey kept as selected and a low grey kept. * docs(ai-comfyui): state the selection mask's byte limit, test it, and rename a mutation case Two review findings. The mask is read inline under the provider's output limit (never more than the 32 MiB inline cap) and never to the output folder, because the checks need its bytes in hand; that is now written in the module header and the provider, and a test shows a mask over a lowered limit fails the job and hands on nothing, with a mutation case for the limit being ignored. The case that only removed the media-type guard is renamed so it no longer implies JPEG answers are covered. The limit test passed on first run, as the behaviour already existed: it is a guard, not a red-first fix. |
||
|
Jeremy
|
4effe6a190 |
feat(ai-comfyui): import a selection workflow and run it with the picture sent first (#80)
Adds the `image.segment` capability to imported ComfyUI workflows. A selection workflow maps a text prompt and one source picture; the picture is uploaded, the graph is queued with the name ComfyUI gave it, and every queued graph gets a sink name of its own so a repeat is not answered from cache with no output. Clicks or a box are refused in plain words before anything is sent, and a stored workflow file may declare a source binding and no mask. The answer is the graph's image as it comes. Checking the mask is the next change. |
||
|
Jeremy
|
320951cf9f |
feat(ai-core): share the image.segment request and result contract (#77)
* feat(ai-core): share the image.segment request and result contract * fix(ai-core): take only an inline PNG as the selection mask |
||
|
Dylan M. Taylor
|
0c20e18cf3 |
Use chrono instead of hand-rolled UTC conversion (#74)
* Use chrono for UTC/calendar conversion instead of hand-rolled copies Replaces the hand-rolled civil_from_days/Day::to_days in crates/ai-core/src/history/report.rs and the picker-preset date() in crates/ai-client/src/picker_presets.rs with chrono, added as a workspace dependency (std-only, so ai-core still compiles for wasm). The day/month grouping, timestamps and the "Released <date>" line are byte-identical and pinned by the existing tests. CONTRIBUTING.md now says time goes through chrono. Follow-up to storytold/cadcraft#1785 * Guard chrono date paths: out-of-range fallback and no '+' on 5-digit years Systemic chrono follow-ups to the UTC/calendar PR (this repo has no review threads; its history/report.rs is the same code as craft-libs): - history::civil_from_days used `i32::try_from(..).unwrap_or(0)` before NaiveDate::from_num_days_from_ce_opt, so a CE day count past i32 (e.g. Day::from_days(i64::MAX)) passed a valid day 0 to chrono and read as 0000-12-31 instead of the 0000-01-01 None fallback. Converted with `.ok().and_then(..)` so both failures share the one fallback; regression test covers i64::MAX / i64::MIN. (Mirrors storytold/craft-libs#219.) - picker_presets::date wrote the year with %Y, which prefixes a '+' before years past 9999; the old hand-rolled code wrote a plain year. Restored the plain year via t.year(). Test covers 10000 and the u64::MAX epoch fallback. Doc comments on the chrono-touched to_days/from_days/civil_from_days and date now state the out-of-range behaviour for the docstring gate. * Convert every i32 year, not only chrono's range Same fix as storytold/craft-libs#219 (this crate carries the same report.rs): the day is converted within its 400-year cycle so valid i32 years past chrono's range keep their pre-chrono results in both directions. |
||
|
Mex
|
3441cacd12 |
fix(mutation): redact Unix profile paths from verification logs (#62)
* fix(mutation): scrub Unix profile paths from verification logs Redact /home and /Users account folders while preserving the path suffix and existing Windows/MSYS behavior. * test(mutation): cover Unix and Windows log profile redaction * fix(mutation): redact JSON-escaped Unix profile paths * test(mutation): cover escaped Unix profile paths Add test for JSON escaped Unix profiles to ensure suffix is preserved. * ci: run mutation tooling Python regression tests Add Python unit tests for mutation testing in CI workflow. * test(mutation): document conservative profile redaction Keep the collaborator-reviewed Fedora, WSL and URI redaction policy. Add explicit controls and document the privacy rationale without changing the production regex. All six Python tests and compileall pass. --------- Co-authored-by: Jeremy <jeremy8776@googlemail.com> |
||
|
Saadi
|
954b187ebf |
test(ai-comfyui): catch the is_file check going missing on Unix (#75)
* ci(mutation): let the workflow run one case by name The harness already accepts a case-name substring on the command line, but the Actions workflow only forwarded the group, so checking a single case meant waiting for the whole group: u3 alone is 72 cases and the job allows 120 minutes. Forward an optional filter and name the step after the group it runs. This also makes it practical to get per-OS observations for one case, which is what #6 asks for the Windows-only equivalent entries. * test(ai-comfyui): catch the is_file check going missing on Unix The deliberate-bug case input-folder-read survived on ubuntu-latest and macos-latest: removing the is_file check in read_file left the test green. A folder is opened on Unix and then fails at the read with the same "could not be read" refusal, so the folder case cannot tell the two checks apart. Windows only records it as equivalent because a folder cannot be opened there at all. A character device is the input the check is really for: it opens like a file and reads as no bytes, so without the check an empty body is accepted where a file was expected. /dev/null is that case, Unix only. The equivalence notes now say what was observed on each system rather than leaving Linux and macOS unproven. * fix(mutation): fail a filtered run that names no case A typo in the case filter made the harness skip every case and exit 0, so a workflow run would report success after checking nothing. Choose the cases up front and stop when the filter matches none of them. |
||
|
Jeremy
|
182e248e95 |
feat(ai-egui): a neutral widget for setting up a built-in model's files (#72)
* feat(ai-features): plan and offer a built-in runtime's own files plan_native and setup_offer_native compare the files a runtime reads, which are a different layout from the ComfyUI checkpoint, and need neither ComfyUI nor its node packs. Setup options gain Route::Native (native:<feature>:<provider>) for each runtime that names files. A runtime with no files of its own gets no plan, no offer and no option. The built-in install offer now lists the runtime's three files, not the ComfyUI checkpoint. * feat(ai-client): find, check and download a built-in runtime's own files NativeSetupPlan is an immutable, reviewable plan for one runtime's files: where they are, where a download would go, how many bytes are still needed. FolderDesk finds a runtime's files in the person's folders (only together in one directory), hashes them before reuse, and downloads only into the chosen default folder, with licence approval. The folder scan lists a runtime's tokenizer files on an exact name and size. The picker lists a runtime as needing an adapter when the app has none. * feat(ai-client): a setup session with approval, one running request and worker-safe answers SetupSession keeps the licence approval, one running request with a ticket, and cancel. It refuses late, stale and foreign answers, progress for the wrong request, and a result for a different reviewed directory or runtime. Starting a download consumes the approval. Mutation group nq covers these rules. * feat(ai-egui): a neutral widget for setting up a built-in model's files native_model_setup draws the review (destination, licences, size), keeps Download off until the licences are accepted, reports progress with Cancel, and returns a NativeSetupAction for the host. Checked files are offered for use, not called ready, and a source's error text is never shown. * fix(ai-features): one recipe rule for every built-in runtime path, and a build that compiles alone Planning, the Install offer and the setup options stopped at the first recipe, while registration and engine advice used the first usable one. An unusable recipe in front hid a usable one behind it, and two recipes naming one runtime gave two options with the same id. `Feature::first_usable_recipe` is now the one rule. `plan_native`, the built-in Install offer and `setup_options` use it, and each runtime gets a single option. The picker's route match gets a Native arm that lists nothing yet, so this change builds on main without the next one. * fix(ai-client): skip a registered folder that has gone, use the first usable recipe, test with and without Candle - `native_files` no longer fails every native call when one registered folder is missing (an unplugged drive, a deleted folder). That folder is not a candidate; the others are still read. - `native_route` uses `Feature::first_usable_recipe`, the rule the planner and the setup options now share, so a Set up link that the picker shows can be followed. - The picker tests accept the built-in engine either as NotSetUp (Candle built in) or as NeedsAdapter (not built in), as `picker_native` already did. Merges the fix to #69 (one recipe rule, and the picker's Native arm). * fix(ai-client): progress cannot cancel a download, and a lost model file can be set up again - `SetupSession::review` compares only what the person approved (tool, runtime, destination and availability, `NativeSetupPlan::same_target`). The bytes still needed and the directory where files were found change as a download runs, so a fresh plan for the same target no longer cancels the worker, clears the approval or drops the verified result. - A built-in model whose saved connection has lost its weights file is listed with its Set up link again, and says the files are missing, instead of showing a connection with no way back. Merges the fixes to #69 and #70. * fix(ai-features): list every local model before any built-in runtime A tool with a built-in runtime put its option ahead of a later tool's local models, which broke the documented order. Runtimes are now collected apart and added after all local options and before hosted services. * fix(ai-client): a result belongs to its directory, and a built-in option matches its own model - `SetupSession::review` drops a checked result when a later plan for the same target finds the files in another directory, so the widget cannot offer to use the old one. Progress in the same directory still keeps it. - The picker hides a built-in option only for the connection that option saves (`local-<feature>-<provider>`), not for any connection of the same runtime and role. Merges the ordering fix to #69. * fix(ai-features): the built-in Install offer agrees to words that match what it does The offer reused the ComfyUI terms, which promise that everything stays in the app's own folder and that ComfyUI is installed. The built-in route installs no ComfyUI and saves into the model folder the person chose. `agreement_for(route)` gives it its own wording and version; the ComfyUI route keeps `agreement()` unchanged. The wording is still draft, like the original. * feat(ai-features): a native setup option names its model Route::Native carries the model id its connection will have, so the picker can tell a saved connection for this model from one for another model of the same runtime. * fix(ai-client): match a built-in option to its own model, and drop a late result for a moved-from directory - The picker hides a built-in option only for a saved connection whose model id is the one the option names (`Route::Native::model`), so an older model on the same connection id does not hide the current one. - A worker's answer that arrives after a review moved to another directory is accepted as the running request's answer but not kept as a checked result, and is not an error. * fix(ai-features): the built-in terms name the Download click The built-in flow's control is Download, not Install, so its terms now say Download. * fix(ai-client): the picker matches a built-in option to its own model A saved connection hides a built-in option only if one of its models is the model that option names, so another model of the same runtime does not hide it. * fix(ai-client): keep a download's result when complete files exist elsewhere A checked result is kept for a directory the plan stands for: where it found complete files, or where a download would go. The stricter rule discarded the result of a download into its destination whenever discovery pointed at another complete copy. |
||
|
Jeremy
|
53a949876b |
chore(deps): png 0.18, base64 0.23, jpeg-encoder 0.7 and patch bumps (replaces the buildable part of #68) (#73)
* chore(deps): png 0.18, base64 0.23, jpeg-encoder 0.7, and the patch bumps from #68 Takes the dependency bumps of the cargo group PR (#68) that build cleanly, and leaves egui 0.37 and candle 0.11 for their own decisions. png 0.18 needs a reader that can seek, so decoders take a Cursor, and output_buffer_size is now an Option, which non-test code turns into the existing error. jpeg-encoder 0.7 takes the app segment by value. serde_json 1.0.154, sha2 0.11.1 and sevenz-rust2 0.24.1 are lockfile-only. * chore(deps): keep sevenz-rust2 at 0.24.0, whose patch release needs Rust 1.93 against our 1.90 * chore(deps): take sevenz-rust2 0.24.1 after all The earlier revert assumed 0.24.1 raised the minimum Rust version. 0.24.0 already needs 1.93, so it changes nothing. |
||
|
Jeremy
|
f52feba1f6 |
feat(ai-client): a setup session with approval, one running request and worker-safe answers (#71)
* feat(ai-features): plan and offer a built-in runtime's own files plan_native and setup_offer_native compare the files a runtime reads, which are a different layout from the ComfyUI checkpoint, and need neither ComfyUI nor its node packs. Setup options gain Route::Native (native:<feature>:<provider>) for each runtime that names files. A runtime with no files of its own gets no plan, no offer and no option. The built-in install offer now lists the runtime's three files, not the ComfyUI checkpoint. * feat(ai-client): find, check and download a built-in runtime's own files NativeSetupPlan is an immutable, reviewable plan for one runtime's files: where they are, where a download would go, how many bytes are still needed. FolderDesk finds a runtime's files in the person's folders (only together in one directory), hashes them before reuse, and downloads only into the chosen default folder, with licence approval. The folder scan lists a runtime's tokenizer files on an exact name and size. The picker lists a runtime as needing an adapter when the app has none. * feat(ai-client): a setup session with approval, one running request and worker-safe answers SetupSession keeps the licence approval, one running request with a ticket, and cancel. It refuses late, stale and foreign answers, progress for the wrong request, and a result for a different reviewed directory or runtime. Starting a download consumes the approval. Mutation group nq covers these rules. * fix(ai-features): one recipe rule for every built-in runtime path, and a build that compiles alone Planning, the Install offer and the setup options stopped at the first recipe, while registration and engine advice used the first usable one. An unusable recipe in front hid a usable one behind it, and two recipes naming one runtime gave two options with the same id. `Feature::first_usable_recipe` is now the one rule. `plan_native`, the built-in Install offer and `setup_options` use it, and each runtime gets a single option. The picker's route match gets a Native arm that lists nothing yet, so this change builds on main without the next one. * fix(ai-client): skip a registered folder that has gone, use the first usable recipe, test with and without Candle - `native_files` no longer fails every native call when one registered folder is missing (an unplugged drive, a deleted folder). That folder is not a candidate; the others are still read. - `native_route` uses `Feature::first_usable_recipe`, the rule the planner and the setup options now share, so a Set up link that the picker shows can be followed. - The picker tests accept the built-in engine either as NotSetUp (Candle built in) or as NeedsAdapter (not built in), as `picker_native` already did. Merges the fix to #69 (one recipe rule, and the picker's Native arm). * fix(ai-client): progress cannot cancel a download, and a lost model file can be set up again - `SetupSession::review` compares only what the person approved (tool, runtime, destination and availability, `NativeSetupPlan::same_target`). The bytes still needed and the directory where files were found change as a download runs, so a fresh plan for the same target no longer cancels the worker, clears the approval or drops the verified result. - A built-in model whose saved connection has lost its weights file is listed with its Set up link again, and says the files are missing, instead of showing a connection with no way back. Merges the fixes to #69 and #70. * fix(ai-features): list every local model before any built-in runtime A tool with a built-in runtime put its option ahead of a later tool's local models, which broke the documented order. Runtimes are now collected apart and added after all local options and before hosted services. * fix(ai-client): a result belongs to its directory, and a built-in option matches its own model - `SetupSession::review` drops a checked result when a later plan for the same target finds the files in another directory, so the widget cannot offer to use the old one. Progress in the same directory still keeps it. - The picker hides a built-in option only for the connection that option saves (`local-<feature>-<provider>`), not for any connection of the same runtime and role. Merges the ordering fix to #69. * fix(ai-features): the built-in Install offer agrees to words that match what it does The offer reused the ComfyUI terms, which promise that everything stays in the app's own folder and that ComfyUI is installed. The built-in route installs no ComfyUI and saves into the model folder the person chose. `agreement_for(route)` gives it its own wording and version; the ComfyUI route keeps `agreement()` unchanged. The wording is still draft, like the original. * feat(ai-features): a native setup option names its model Route::Native carries the model id its connection will have, so the picker can tell a saved connection for this model from one for another model of the same runtime. * fix(ai-client): match a built-in option to its own model, and drop a late result for a moved-from directory - The picker hides a built-in option only for a saved connection whose model id is the one the option names (`Route::Native::model`), so an older model on the same connection id does not hide the current one. - A worker's answer that arrives after a review moved to another directory is accepted as the running request's answer but not kept as a checked result, and is not an error. * fix(ai-features): the built-in terms name the Download click The built-in flow's control is Download, not Install, so its terms now say Download. * fix(ai-client): the picker matches a built-in option to its own model A saved connection hides a built-in option only if one of its models is the model that option names, so another model of the same runtime does not hide it. * fix(ai-client): keep a download's result when complete files exist elsewhere A checked result is kept for a directory the plan stands for: where it found complete files, or where a download would go. The stricter rule discarded the result of a download into its destination whenever discovery pointed at another complete copy. |
||
|
Jeremy
|
50b224d875 |
feat(ai-client): find, check and download a built-in runtime's own files (#70)
* feat(ai-features): plan and offer a built-in runtime's own files plan_native and setup_offer_native compare the files a runtime reads, which are a different layout from the ComfyUI checkpoint, and need neither ComfyUI nor its node packs. Setup options gain Route::Native (native:<feature>:<provider>) for each runtime that names files. A runtime with no files of its own gets no plan, no offer and no option. The built-in install offer now lists the runtime's three files, not the ComfyUI checkpoint. * feat(ai-client): find, check and download a built-in runtime's own files NativeSetupPlan is an immutable, reviewable plan for one runtime's files: where they are, where a download would go, how many bytes are still needed. FolderDesk finds a runtime's files in the person's folders (only together in one directory), hashes them before reuse, and downloads only into the chosen default folder, with licence approval. The folder scan lists a runtime's tokenizer files on an exact name and size. The picker lists a runtime as needing an adapter when the app has none. * fix(ai-features): one recipe rule for every built-in runtime path, and a build that compiles alone Planning, the Install offer and the setup options stopped at the first recipe, while registration and engine advice used the first usable one. An unusable recipe in front hid a usable one behind it, and two recipes naming one runtime gave two options with the same id. `Feature::first_usable_recipe` is now the one rule. `plan_native`, the built-in Install offer and `setup_options` use it, and each runtime gets a single option. The picker's route match gets a Native arm that lists nothing yet, so this change builds on main without the next one. * fix(ai-client): skip a registered folder that has gone, use the first usable recipe, test with and without Candle - `native_files` no longer fails every native call when one registered folder is missing (an unplugged drive, a deleted folder). That folder is not a candidate; the others are still read. - `native_route` uses `Feature::first_usable_recipe`, the rule the planner and the setup options now share, so a Set up link that the picker shows can be followed. - The picker tests accept the built-in engine either as NotSetUp (Candle built in) or as NeedsAdapter (not built in), as `picker_native` already did. Merges the fix to #69 (one recipe rule, and the picker's Native arm). * fix(ai-features): list every local model before any built-in runtime A tool with a built-in runtime put its option ahead of a later tool's local models, which broke the documented order. Runtimes are now collected apart and added after all local options and before hosted services. * fix(ai-features): the built-in Install offer agrees to words that match what it does The offer reused the ComfyUI terms, which promise that everything stays in the app's own folder and that ComfyUI is installed. The built-in route installs no ComfyUI and saves into the model folder the person chose. `agreement_for(route)` gives it its own wording and version; the ComfyUI route keeps `agreement()` unchanged. The wording is still draft, like the original. * feat(ai-features): a native setup option names its model Route::Native carries the model id its connection will have, so the picker can tell a saved connection for this model from one for another model of the same runtime. * fix(ai-features): the built-in terms name the Download click The built-in flow's control is Download, not Install, so its terms now say Download. * fix(ai-client): the picker matches a built-in option to its own model A saved connection hides a built-in option only if one of its models is the model that option names, so another model of the same runtime does not hide it. |
||
|
Jeremy
|
ab9e9cb5aa |
feat(ai-features): plan and offer a built-in runtime's own files (#69)
* feat(ai-features): plan and offer a built-in runtime's own files plan_native and setup_offer_native compare the files a runtime reads, which are a different layout from the ComfyUI checkpoint, and need neither ComfyUI nor its node packs. Setup options gain Route::Native (native:<feature>:<provider>) for each runtime that names files. A runtime with no files of its own gets no plan, no offer and no option. The built-in install offer now lists the runtime's three files, not the ComfyUI checkpoint. * fix(ai-features): one recipe rule for every built-in runtime path, and a build that compiles alone Planning, the Install offer and the setup options stopped at the first recipe, while registration and engine advice used the first usable one. An unusable recipe in front hid a usable one behind it, and two recipes naming one runtime gave two options with the same id. `Feature::first_usable_recipe` is now the one rule. `plan_native`, the built-in Install offer and `setup_options` use it, and each runtime gets a single option. The picker's route match gets a Native arm that lists nothing yet, so this change builds on main without the next one. * fix(ai-features): list every local model before any built-in runtime A tool with a built-in runtime put its option ahead of a later tool's local models, which broke the documented order. Runtimes are now collected apart and added after all local options and before hosted services. * fix(ai-features): the built-in Install offer agrees to words that match what it does The offer reused the ComfyUI terms, which promise that everything stays in the app's own folder and that ComfyUI is installed. The built-in route installs no ComfyUI and saves into the model folder the person chose. `agreement_for(route)` gives it its own wording and version; the ComfyUI route keeps `agreement()` unchanged. The wording is still draft, like the original. * feat(ai-features): a native setup option names its model Route::Native carries the model id its connection will have, so the picker can tell a saved connection for this model from one for another model of the same runtime. * fix(ai-features): the built-in terms name the Download click The built-in flow's control is Download, not Install, so its terms now say Download. |
||
|
dependabot[bot]
|
ea9bbcbc65 |
ci(deps): bump actions/checkout (#45)
Bumps the actions group with 1 update in the / directory: [actions/checkout](https://github.com/actions/checkout). Updates `actions/checkout` from 4.4.0 to 7.0.1 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/11d5960a326750d5838078e36cf38b85af677262...3d3c42e5aac5ba805825da76410c181273ba90b1) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 7.0.1 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Jeremy
|
8be4066b1f |
feat(ai-engine-candle): SAM 3 on the Candle engine, and the catalogue names the files it reads (#41)
* feat(ai-engine): a runner contract for models that run on this computer
Candle and Burn were named in the catalogue but no app could share them: each would have needed
its own job handling, cancel, progress and weights loading. This adds the seam so an engine author
writes only the model.
- craft-ai-engine: ModelRunner (engine, model spec, load) and LoadedModel (run), a RunContext for
progress, cancel and notices, an EngineRegistry, and EngineProvider, the one Provider that runs
any registered model (weights loaded once, jobs for one model take turns, a panicking runner
becomes a failed job).
- testing::{FakeRunner, check_runner}: puts a runner behind the real provider and runs the shared
conformance suite.
- craft-ai-client: ProviderFactory::register_engines is the whole of enabling an engine in an app;
register now forgets what was declared about the adapter it replaces, and unregister removes one.
- docs/ADD-AN-ENGINE.md is the crate documentation, so its code is compiled by the doctests.
* feat(ai-engine-candle): SAM 3 on the Candle engine, and the catalogue names the files it reads
The built-in SAM 3 route was documented as reading the same file as ComfyUI. It does not: the
ComfyUI file is Meta's layout (detector.backbone..., 1465 tensors); the Candle port reads the
Hugging Face layout (detector_model..., 1797 tensors) plus vocab.json and merges.txt. A route
pointed at the ComfyUI file could never load.
- ai-features: NativeRoute gains its own `models`, so a runtime can name the files it reads. The
SAM 3 routes name facebook/sam3 at a pinned revision, with sizes and SHA-256 checked against a
real download. NativeRoute::weights and total_bytes read them.
- craft-ai-engine-candle: the model code from LightCraft's lightcraft-segment crate (see NOTICE),
behind the runner contract. image.segment takes a PNG source and either a text prompt or click
points, and answers with a grayscale PNG mask the size of the source. Apache-2.0 only, because
the port is a derivative of Hugging Face Transformers.
- craft-ai-client: the `candle` cargo feature makes ProviderFactory::standard() know the engine, so
an app adopts it by turning the feature on.
* fix(ai-engine,ai-client): keep one provider per connection, and let a cancelled job leave the queue without loading or running
The client asks the factory for a provider on every job, so each job built a fresh EngineProvider and
loaded the weights again; two jobs at once loaded two copies. The factory now keeps one provider per
connection and builds another only when the connection's models or weight files change.
A job waiting behind another for the same model reported Running, and if it was cancelled it still
loaded the weights and ran. It now reports Queued until it holds the model, and a cancel before then
or during a slow load stops it before any work. At most 16 jobs may wait or run at once; a burst is
refused in plain words rather than piling up threads that each hold a copy of the picture.
* fix(ai-client): write the kept-provider key with escapes, not a literal newline and tab
* fix(ai-features,deny): say what the Candle route really runs on, offer no download for Burn, and record the candle advisories
The Candle route said it ran on Nvidia or Apple graphics, but this build declares the Apple card only and
runs on the processor elsewhere; the label now says so. The Burn route named the 3.4 GB Hugging Face files
although no Burn code exists, so it now names none and offers no download.
cargo-deny failed on candle-core's zip (RUSTSEC-2026-0367, 0368, 0369) and paste (RUSTSEC-2024-0436).
Each is ignored with the reason it cannot be reached here: only safetensors are read and no archive is
extracted.
* feat(ai-engine-candle): read JPEG pictures, and turn them upright by their EXIF orientation
A source can now be a PNG or a JPEG, told apart by its first bytes. JPEGs are decoded with zune-jpeg through
the image crate (baseline, progressive, greyscale), refused from the header when over 100 megapixels, and
given an allocation cap. The EXIF orientation is applied, as browsers, photo editors and OpenCV do, so a
phone photo taken sideways is segmented upright and a click normalised to the picture lands where the person
clicked. The mask comes back the size of the upright picture. A PNG is read as stored.
Verified on Meta's checkpoint in release mode: a 128x192 JPEG marked 'turn 90 degrees clockwise' gives a
192x128 mask centred on the disc. Unit tests cover colour and greyscale, every damaged and cut-off case,
a header patched to 30000x30000, and the turn itself (the test fails without it).
jpeg-encoder makes the test JPEGs; it carries the IJG licence, allowed for that one crate in deny.toml.
* test(ai-engine-candle): write the EXIF header with escapes, not raw NUL bytes, so git shows the file as text
* fix(ai-engine-candle): lift the PNG decoder's buffer limit, and bound tokenizer work for long prompts
A one-row picture inside the 100 megapixel limit but with a row over 64 MiB was refused by the png crate's default allowance; the decoder now gets the same 900 MiB cap as the JPEG route. Tokenizer::encode now clips the text to CONTEXT * 32 characters and each word to CONTEXT * 8 before the merge loop, and stops once the ids fill the context. Both found by CodeRabbit on #41.
* docs(ai-engine-candle): doc comments on the functions touched by the review fixes
* fix(ai-engine-candle): collapse whitespace before the tokenizer's character limit
Spaces at the start of a prompt used up the 1,024 character allowance, so a prompt of 1,024 spaces then a word reached the model as nothing. Whitespace is now collapsed first, and the text is copied only as far as the allowance reaches. Found by CodeRabbit on #41.
---------
Co-authored-by: scratch <s@s>
Co-authored-by: t <t@t>
|
||
|
Jeremy
|
7473648830 |
fix(ai-client): keep every recipe's setup link, and refuse a clashing connection id (#67)
Two review findings on #63 arrived after it merged. The picker hid every local setup link as soon as any local connection fit the tool. A connection a recipe registered now counts for that recipe only, so enabling one model leaves the other model's setup link in place. Other local connections count as before. The recipe connection id `local-<feature>-recipe-<workflow>` can equal the id of a custom runtime named `recipe-<workflow>`, and registering one replaced the other while the binding still named the old model. Registering now refuses to replace a connection of another adapter, in either order, and leaves the store as it was. |
||
|
Jeremy
|
11a6cb7f8f |
feat(ai-egui): a consent prompt that asks for how long (#66)
consent_prompt_with_terms shows the shared durations beside Allow and Don't send, starting on This time only, and returns the duration with the choice. The host applies it with ConsentLedger::agree. The duration resets when the disclosure changes and after each answer. The original consent_prompt is unchanged. |
||
|
Jeremy
|
ffb27cd17a |
feat(ai-features): offer SAM 2.1 for Select subject, with its detector and a licence list (#64)
* refactor(ai-features): one catalogue file per model Moves the SAM 3 recipe into catalogue/sam3.rs with no change, so the next recipes do not push one file past the size limit. * feat(ai-features): offer SAM 2.1 for Select subject, with its detector and a licence list SAM 2.1 joins SAM 3 as a second local recipe: the fp16 checkpoint plus the ten pinned Florence-2-large files. LocalRecipe.components lists every part the workflow runs with the terms each publisher declares; disclosure only, it gates nothing. * fix(ai-features): the SAM 2.1 add-on note no longer claims a hash check The readiness scan compares file names and sizes only, so the disclosure now says that instead of saying the file is checked. A test fails if the claim comes back. |
||
|
Jeremy
|
037a011df6 |
feat(ai-features,ai-client): a tool can offer several local models (#63)
* feat(ai-features,ai-client): a tool can offer several local models Feature.local becomes Feature.locals (breaking). plans() gives one plan per recipe, named in Plan.recipe. Setup options are per recipe (local:<feature>:<recipe>, breaking). ToolDesk::enable_local_recipe registers one named recipe as its own connection; the first keeps local-<feature>. * fix(ai-client): keep recipe connections apart from built-in runtimes, and follow the runnable recipe A second recipe's connection is now local-<feature>-recipe-<workflow>, so a recipe named like a runtime (candle, burn) can never replace that runtime's connection. engine_advice, enable_native and enable_native_with use the first recipe that can run, the same one enable_local registers. * docs: say which recipe a built-in runtime belongs to * fix(deps): update rustls-webpki and webpki roots (RUSTSEC-2026-0381, 0382, 0383) rustls-webpki 0.103.15 -> 0.103.16, webpki-roots and webpki-root-certs 1.0.9 -> 1.0.10. The new versions reject certificate paths through the distrusted emSign intermediates, whose private keys are now public. Lockfile only. |
||
|
Jeremy
|
f6a8ab520f |
fix(ai-comfyui): keep uploads and results in ComfyUI's scratch area (#61)
Every selection uploaded a full-size copy of the user's image to ComfyUI's input folder and saved its mask to the output folder, and nothing ever removed them. - The source image is uploaded with type=temp and read as 'name [temp]'. A reply that says it landed anywhere else is refused. - SAM 3 and SAM 2.1 result nodes become PreviewImage, which writes to the scratch area ComfyUI clears when it starts. Results are fetched from there. - Each result node keeps a unique cache key. A PreviewImage with identical inputs is cached, and a cached result node returns no output on the repeat. - Output type 'temp' is accepted alongside 'output'; any other folder is still refused. Mutation group 'sf': 14 of 15 caught, 1 recorded as equivalent. |
||
|
Jeremy
|
a912eebb6d |
feat(ai-comfyui): SAM 2.1 selection session, and refuse to run without the Florence-2-large detector (#58)
* feat(ai-comfyui): SAM 2.1 selection session, and refuse to run without the Florence-2-large detector Sam21Session finds the words of a prompt with Microsoft's Florence-2-large (MIT) and cuts the subject out with the SAM 2.1 checkpoint. Before any picture is uploaded it checks that the Florence2ModelLoader node lists the detector folder, so a missing detector is a clear refusal instead of a graph error. Cancel removes only the session's own prompt. * fix(ai-comfyui): SAM 2.1 uses the device ComfyUI reports, and names itself in errors Addresses the two review findings. - The graph no longer hard-codes cuda and fp16. The session reads the device from /system_stats: cuda keeps fp16, cpu and mps use fp32. A missing or other device (for example xpu) is refused before the image is uploaded. - Mask-fetch errors said SAM 3 when SAM 2.1 was selected. The SAM 2.1 session now rewrites the name. Five mutation cases added to group s21; all eight caught. cpu and mps are covered by scripted tests only. * docs: document the functions this change adds CodeRabbit's docstring check was below its 80% bar. Test functions get a one-line sentence from their name; helpers and session methods get a real description. * fix(ai-comfyui): SAM 2.1 selects every matching subject, and each selection gets its own sink name Both found in review. - Sam2Segmentation keeps only the last box when individual_objects is false, so a second match was silently dropped. It is now true. Checked live on two copies of the sample photo side by side: false selected one copy, true selects both. A scripted server returns a fixed mask and cannot see this, so the graph is pinned by a test and an ignored live test selects both halves. - Identical selections after one prepare queued identical graphs with no client id. ComfyUI 0.22.3 serves those from its cache with no outputs, so the repeat failed with no selection mask. Each selection now has a new sink name; inference nodes are unchanged. Two mutation cases added to group s21; all ten caught. The live test fails with individual_objects false and passes with true. |
||
|
Jeremy
|
1d1ebfefd4 |
fix(ai-comfy-env): keep a model bundle in one sub-folder, and see its small config files in the scan (#57)
* fix(ai-comfy-env): keep a model bundle in one sub-folder, and see its small config files in the scan A download file name may now be one to three plain names joined by '/', such as Florence-2-large/model.safetensors, so a bundle stays together. Dots first or last, empty parts, backslashes and colons are still refused, which rules out '..' and absolute paths. The install scan also lists json, py and txt files up to 2 MiB; before, a bundle with config files could never be seen as complete. * docs: document the functions this change adds CodeRabbit's docstring check was below its 80% bar. Test functions get a one-line sentence from their name; helpers and session methods get a real description. * docs: document the functions the scan and download change touches * fix(ai-comfy-env): refuse a bundle folder that links outside the destination A bundle file such as Bundle/m.safetensors is joined onto the destination folder. If Bundle already existed as a link to somewhere else, the folder creation, the part file and the final rename all followed it and wrote outside the destination. The parent is now resolved and must still be inside the resolved destination before anything is checked, fetched or renamed. A destination that is itself a link still works. Flat files are unaffected. Two mutation cases added to group u3, both caught. The test skips with a message on a machine that does not allow directory links. * fix(ai-comfy-env): check each bundle folder before the next one is made The first check resolved the parent after create_dir_all had already built every missing folder, so a link at the top of a deeper bundle path still got a new folder created behind it. The folders are now made one at a time and each is resolved and checked before the next. This also puts safe_relative's doc comment back on safe_relative; the first fix had left it on the new function. The remaining gap is a swap after the check, which needs write access to the destination and so adds no reach beyond replacing the model file. It is written in the doc comment. * test(ai-comfy-env): keep the name checks off the internet, and make skipped mutation cases visible - file_names_that_could_escape_the_folder_are_refused pointed at example.com. With the validator broken by a mutation run, the downloader tried to reach it. It now uses the scripted local server and asserts that a refused name is never fetched. - The mutation runner passes --nocapture and treats a pass that printed the directory-link skip as skipped, not as a survivor, so a machine that cannot make links no longer reports a false survivor. |
||
|
Jeremy
|
7428cb8c75 |
fix(ai-core,ai-egui): required inputs and test-result addresses, matching the upstream contract fixes (#59)
* refactor(ai-core): move the connection presets into their own file draft.rs was 519 lines. The built-in presets and their static model lists are data that changes on a different schedule from the save rules, so they move to draft/presets.rs unchanged. Three mutation cases follow the code. * fix(ai-core,ai-egui): match the upstream contract fixes for required inputs and test-result addresses A model now has to accept every input a tool sends and require no input the tool cannot supply; the picker, binding, app status and planning agree, and a refusal happens before a key is read or anyone is asked to consent. A connection test result is recorded with the address the test started with (ConnectionDraft::set_models(address, models)), so a late answer for an old address is refused and loaded, preset and tested models keep their address through repeated edits. The shared connection panel passes the captured address. docs/CORE-SYNC.md says what this local copy still adds beyond craft-libs, and the mutation group 'ca' covers the ten safety rules. Two mutation anchors that matched twice are fixed. * docs: document the functions this change adds CodeRabbit's docstring check was below its 80% bar. Test functions get a one-line sentence from their name; helpers and session methods get a real description. |