mirror of
https://github.com/storytold/craft-ai-integrations.git
synced 2026-11-02 18:06:49 +00:00
main
15 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 imperator
|
e8f320321a |
feat(ai-engine-burn): InSPyReNet for Remove background, built in only (#112)
* feat(ai-engine-burn): InSPyReNet for image.matte (Remove background) A second model on the Burn engine: `inspyrenet` answers `image.matte` from the picture alone, a grayscale PNG at the picture's size (white is the subject, soft along hair and fur). `remove-background` has had no local model; this is one. - Network: InSPyReNet with the Swin-B backbone, written by hand in Burn after the MIT reference code. Module names follow the PyTorch keys, so the published .safetensors loads by name through Burn's PyTorch adapter. A load that leaves a weight unset, fails one, or finds a foreign tensor is refused in plain words. - Any picture size: full resolution up to 1280 px on the long side; above that a reflect-padded square reduced with antialiased Lanczos-3, the matte enlarged back and refined at full resolution with a colour guided filter on the tiles near the edge (the Gyre approach), then a 0.2 fringe shrink. - Weights: 1038lab/inspyrenet inspyrenet.safetensors at a pinned revision, checked tensor for tensor against the author's ckpt_base.pth. Checked on one Windows machine (release): every stage against transparent-background's PyTorch code (worst relative difference 3.2e-5 on the processor, 2.4e-4 on the card, no pixel changing side); the shared conformance suite; on a real PNG the mask is byte-identical to the same port in PhotoCraft. Four new deliberate-bug cases in `be`, all caught. Not done: a catalogue recipe (a recipe names a ComfyUI workflow today), clean timings, macOS and Linux runs, a run through an app. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(ai-features): a recipe that is built in only, InSPyReNet for Remove background `LocalRecipe::comfyui` (default true) marks a recipe ComfyUI runs. With false it is only its built-in routes: never a ComfyUI setup option, plan, install offer or connection, and engine advice does not list ComfyUI for a tool without a ComfyUI recipe. `Feature::comfyui_recipes()` gives those recipes; `engine_recipe`, `plan_native`, `setup_offer_native` and `enable_native` are unchanged. `remove-background` gets one such recipe: InSPyReNet on Burn, the pinned `inspyrenet.safetensors` (1038lab/inspyrenet @ 0f594663, 367,120,444 bytes) in the folder `inspyrenet`, so Set up AI offers one checked download and no ComfyUI. Tests: the catalogue, setup options, plans and offers (`ai-features/tests/built_in_only.rs`), registration and advice (`ai-client/tests/built_in_only.rs`), and through `AiClient` on Burn (`ai-client/tests/burn_matte.rs`, the real run ignored). Mutation group `bo`: 8 of 8 caught. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(tools): retarget two `lc` mutation cases at the ComfyUI recipe lookups `unrunnable-recipe-accepted` and `second-recipe-takes-plain-id` searched for the `feature.locals` lookups in `ai-client/src/local.rs`, which now go through `Feature::comfyui_recipes()`. Same mutations, new targets; group `lc`: 21 caught, one known equivalent. Every case in `GROUPS` again has exactly one target (640 checked). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(ai-engine-burn): InSPyReNet runner checks around a stand-in, readout without panic CI has no graphics card, only a software renderer (lavapipe, WARP), and the engine takes it. There the tiny network was too slow for the conformance suite's poll limit, and on lavapipe the readout failed. Unoptimised, Burn's processor backend takes minutes for the same network. - The loaded model runs the pipeline around a `Predict`: the network, or in tests a stand-in, as the SAM 2.1 tests here use a fake backend. The conformance suite and the mask test use the stand-in (debug: the crate's unit tests in about 18 s). - The tiny network still runs through load and run on the processor in an ignored release test (about 4 s). The runner's device choice is a field, so tests can name the processor. - The prediction is read with `try_into_data`, so a device that fails while running is an error, not a guarded panic. - docs/BURN-ENGINE.md: the test split and that a software renderer counts as a card. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(ai-engine-burn): InSPyReNet moves to the processor when the card fails A card can answer the engine's probe and still fail on the network: on CI a software renderer (lavapipe) failed while reading the answer back. The first failure on a card that is not a cancel now builds the network again on the processor from the same file, runs that picture again there with a notice, and stays on the processor (`inspyrenet/fallback.rs`). A network loaded on the processor in the first place has nothing to move to. Tests with stand-ins: a working card is kept; a failing one moves once and stays; a cancel is not a failure; a rebuild that fails is the answer. Mutation group `be`: three new cases, all seven InSPyReNet cases caught. A real photo on the card still runs on the card. docs/BURN-ENGINE.md: the move, and the client run's picture is 2848 x 4272, not 3101 x 1613. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(ai-engine-burn): the move to the processor stays only after a rebuild that worked The rebuild is tried once. If it fails, its error is the answer and later pictures go to the card again, as `a_rebuild_that_fails_is_the_answer` already pins. The module docs and docs/BURN-ENGINE.md said the model stays on the processor without that condition. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(ai-engine-burn): retry a failed InSPyReNet load on the card on the processor The move to the processor covered failures while predicting, not a load on the card that fails. `load_or_processor` now retries such a load on the processor, unless the file itself is not the checkpoint (`Unsupported`), which is refused at once rather than read a second time. A network loaded on the processor in the first place is loaded once. Tests: a card that loads keeps the network; one that fails loads on the processor; a bad file is not read twice; on the processor there is one load. Mutation group `be`: two new cases, all nine InSPyReNet cases caught; every case in the checker has one target (645). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * style(ai-engine-burn): method references in the guided filter, one name for the fallback path Three SonarCloud findings on #112: two closures in `guided.rs` become method references, and the mutation cases name `inspyrenet/fallback.rs` once. No change in behaviour. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: imperator <imperator.maximus@gmail.com> |
||
|
skylar taylor-barrick
|
6f7078666a | test(ai-engine): cover queued cancellation and clarify widget qualification (#106) | ||
|
skylar taylor-barrick
|
0e55340771 |
Migrate shared AI widgets to egui 0.37 (#105)
* refactor(ai-egui): migrate shared widgets to egui 0.37 * egui037: qualify shared AI widgets and downstream dependency pins * docs(ai-egui): record the committed migration qualification * fix(ci): provide Vulkan snapshots and allow pinned fork sources * test: make cancellation-before-load coverage deterministic |
||
|
Jeremy
|
7be5af833f |
feat(client): status, picker, route notes and engine advice agree with what the default route can do (#94) (#100)
* feat(client): one start button for setting up a built-in model, with no tick box unless a licence needs one (#94) Part of #94, slice 1 of 3. The shared pieces for a one-click setup existed but nothing joined them. - NativeRoute/NativeSetupPlan::needs_agreement: the licence tick shows, and a download waits for it, only when a file's licence needs agreement. SAM 2.1 needs none. - SetupSession::start/next_task/blocker: one button that checks complete files on disk or downloads, and says why it does nothing (SetupBlock). - FolderDesk::ensure_default(_at): takes the suggested folder only when none is chosen. The widget offers it (UseSuggestedFolder) instead of dead-ending at "Choose a model folder". - ToolDesk::adopt(&CheckedNative): only checked files, binds only if no choice exists, and returns the choice it kept. - Docs: the flow in use-in-your-app, LAYOUT lists the Burn crates, STATUS row. - Mutation group fr (6 cases, 6 caught); two older cases re-aimed at the changed code. Checked: fmt, strict clippy and tests for craft-ai-features, -client, -egui and -comfy-env (473 passed, 2 ignored), wasm check for craft-ai-features, every mutation group --dry clean, group fr run for real, documentation 14/14. Scripted tests only: no real download, no model run through it, no app has mounted it, no macOS or Linux run. * feat(client): try a built-in selection model with a click, and fail a selection that chose nothing (#94) Part of #94, slice 2 of 3 (stacked on slice 1). The launcher's try-it sample for Select subject sent only a description. The default built-in route (SAM 2.1 on rten) refuses descriptions and selects by clicking, so the sample failed on the default route. - sample_request_for(feature, built_in): for image.segment with a built-in route, one click on the middle of the test disc and no description. Everything else is the plain sample. - ToolDesk::start_sample: picks the sample from the tool's current choice (a built-in engine the tool's recipe lists gets the click; ComfyUI and hosted routes keep the description). - ToolDesk::finish: a selection that ran but reports matched: false is a failed test. - Docs: step 5 of the set-up flow, STATUS row. - Mutation group fr gains 2 cases (8 in all, 8 caught). Checked: fmt, strict clippy and tests for craft-ai-features, -client, -egui and -comfy-env (478 passed, 2 ignored), wasm check for craft-ai-features, group fr run for real, documentation 34/34. Scripted tests and a fake provider only: no real model, no app, no macOS or Linux run. * feat(client): status, picker, route notes and engine advice agree with what the default route can do (#94) Part of #94, slice 3 of 3 (stacked on slices 1 and 2). - AiClient::status: a built-in model whose weights file has gone is not Ready. - The picker lists such a model once, as one to set up again, not also as a ready one. - NativeRoute::note: the setup option, the picker row and its description say the built-in SAM 2.1 route selects by clicking on the picture, instead of offering a prompt it refuses. - NativeRoute::fast_on_processor (rten, one Windows machine, about 35 ms a click, not a benchmark): engine advice gives it Pace::Processor, not "slow", and ranks it level with a card engine, so the recipe's first engine is the suggestion with or without a graphics card. - Mutation group fr gains 6 cases (14 in all, 14 caught); the Burn route case is re-aimed. Checked: fmt, strict clippy and tests for craft-ai-features, -client, -egui and -comfy-env (483 passed, 2 ignored), wasm check for craft-ai-features, every mutation group --dry clean, group fr run for real. Documentation 66/67; the one flagged is an untouched neighbour of a doc line. Scripted tests only: no real download, no model run through it, no app, no macOS or Linux run. * fix(client): one real start button, a failed check falls through to download, folder order (#98 review) Fixes the CodeRabbit findings on #98 that held up. - The setup widget showed "Check existing files" and "Download model files" side by side when complete files were on disk, so a person could start a download instead of the check. It now draws one start button from SetupSession::next_task and blocker, and its click calls SetupSession::start. The licence tick shows only when the next task is a download. - After a failed check, start kept choosing the check again, so files that did not match were a dead end. The session now remembers the folder that failed and downloads next; a different folder, or a different target, is checked again. - FolderDesk::ensure_default needed the computer's suggested folder even when the person had already chosen one. It now looks at chosen folders first. - The adopting guide told a host to call ensure_default before drawing the first screen, which makes the folder before the person accepts it. It now says to call it only on their yes. Checked: fmt; strict clippy; client tests and egui tests pass except the existing panel::a_dead_server_is_reported flake, which passed 3 of 3 alone; docs bar 38/38; mutation group fr 11/11 caught. Group nw had a case aimed at the old two-button code; it is re-aimed and caught. * fix(client): the sample copies only the small result outputs, not whole pictures (#99 review) finish cloned every output to read the selection result, copying large inline pictures and videos. It now copies only inline JSON, the form of a selection result. Behaviour is unchanged, so no test can see the copy; the existing sample tests still pass. Checked: fmt, strict clippy, client tests (309 passed), docs bar 64/64, mutation group fr 13/13. * fix(client,engine-candle,features): a selection sample needs its result, Candle sends it, small card pace (#100 review) Fixes the CodeRabbit findings on #100. - A segmentation sample passed when the route returned a mask and no Found result. The contract gives every route both, so the sample now fails with "did not say whether it selected anything". Other tools are unchanged. - Candle SAM 3 returned only a mask, which would now fail that test. It sends the Found result too (matched with its score, or nothing found when the model gave no prediction). - A processor-quick engine whose matching card is too small was called slow. It is now judged by the processor, as the card is not needed. - The adopting guide said every built-in selection model rejects words. It now says the sample uses a click, which every built-in route reads, and names the new rule. Docs bar: undocumented neighbours of the changed hunks are documented, the chain walked up to the first documented function (tests/engine.rs helpers and tests, three Candle runner methods). Checked: fmt; strict clippy (features, client, engine-candle); tests (features, client, candle lib: 440 passed); wasm check of craft-ai-features; docs bar 124/124; mutation group fr 22/22 caught, with three new cases. Not run: the live Candle test (needs the SAM 3 weights), so Candle sending Found is covered by a unit test of the result, not by a real run. * fix(client): only the connection this suite saves gets the click sample (#99 review) start_sample treated any connection whose provider is a built-in engine's name as built in. A custom connection that reuses the name `rten` may serve a model that needs a description, and the click-only request is refused by the planner before it is sent. The click sample now needs the engine's own saved connection id as well. Others keep the description sample. Checked: fmt, strict clippy, client tests (310 passed), docs bar 65/65, mutation group fr 14/14 caught (one new case; 'sample-ignores-the-route' re-aimed at the changed line). * fix(egui): show suggested folder before creating it The suggestion action creates the folder. Display its actual path first, and hide the action if no suggestion can be made. Keep the native path helper available in the egui library, not only its tests. Checked: fmt, strict clippy, 492 focused tests, wasm, 24/24 first-run mutations, docs 134/134. * fix(egui): keep the folder suggestion idle-only and test no suggestion The suggestion now waits for an active check to finish. Tests guard systems without a suggested path and exercise a busy session with discovered read-only files. Checked: 493 affected tests, fmt, strict clippy, wasm, 25/25 first-run mutations, docs 135/135. * fix(client): recover from bad files in a new approved folder A failed check now blocks reuse of the rejected directory and the previously shown destination. The person chooses a fresh folder, and the worker creates its model directory only if absent. The old files remain untouched, even if the newly approved directory appears before the worker begins. Scripted transfer and retry regressions: 388 affected tests, fmt, strict clippy, wasm, 15/15 first-run mutations, docs 49/49. Four unrelated native-feature mutations survive on upstream main too. * fix(client): explain why a fresh model folder could not be made Preserve the atomic create and distinguish an occupied destination, a missing approved parent and permission denial. Red regressions cover missing and occupied paths, and a mutation checks the message. Checked: 390 affected tests, fmt, strict clippy, wasm, 16/16 first-run mutations, docs 66/66. * fix(client): try another model under the built-in connection with words The sample picked a click whenever the connection was the one this suite saves for the engine, without checking the model. Connection management can put another model under that id, one that wants a description, and the click sample then failed for the wrong reason. Match the model the engine's recipe names as well. --------- Co-authored-by: scratch <s@s> |
||
|
Jeremy
|
1a02afd6be |
feat(client): offer SAM 2.1 on Burn as an optional engine after rten (#91)
* refactor(engine-rten): let another engine reuse the SAM 2.1 request handling Loaded, Backend, Prediction, read_prompt, prompt_points and best_proposal become public. Visibility only: the click and box mapping, the picture kept between requests, the empty-mask answer and the size checks then live in one place for every SAM 2.1 engine. * feat(engine-burn): crate shell and the script that generates the SAM 2.1 architecture tools/burn/generate_sam21.py checks the two published ONNX files against the pinned size and SHA-256, runs the pinned burn-onnx 0.22.0 (an example of craft-ai-burn-setup), applies the five asserted fixes, drops the generated loaders that panic on a bad file, splits by submodule so every file is under 700 lines, formats and writes. --check proves the tree is what the script produces. REVISION must equal the saved-weights cache revision. Cargo.lock additions are Burn with wgpu and ndarray (generated). * feat(burn-setup): prepare SAM 2.1 weights for Burn once, on this computer Brings in craft-ai-burn-setup (feature `conversion`): checks the two published ONNX files by size and SHA-256, converts them to Burn weights with the pinned burn-onnx 0.22.0, and keeps one checked, locked, shared copy. Nothing is hosted or downloaded. Mutation group bp (10 cases). Cargo.lock additions are the optional burn-onnx tree (generated). * feat(engine-burn): GENERATED SAM 2.1 tiny architecture (30 files) Output of tools/burn/generate_sam21.py, revision sam21-tiny-burn-0.22.1-onnx-0.22.0-v1, from the published encoder.onnx and decoder.onnx with burn-onnx 0.22.0 and its five asserted fixes. Do not review line by line; rerun the script with --check instead. Longest file 699 lines. Marked linguist-generated. * feat(engine-burn): SAM 2.1 runner for image.segment on Burn Sam21Runner (engine burn, model sam21) over the generated architecture. The request handling, picture kept between requests, empty-mask answer and click/box mapping are the shared Loaded from craft-ai-engine-rten, so only the model run lives here. Weights load through BurnpackStore and come back as errors; every call into the generated code is guarded against panics. Graphics card when one answers, else the processor. An .onnx file is answered with 'prepare it first', not a guess. 16 unit tests on a fake backend: click and box mapping, empty mask, short mask, a failing model, shared conformance run with Cancel, damaged and truncated files. * test(engine-burn): live comparison with rten, and the shared revision check The ignored live test runs the real SAM 2.1 files through rten (published ONNX) and Burn (weights prepared on this computer) on the astronaut sample: one click and a box, best-mask IoU, timings, and a truncated prepared file refused. The revision test fails when the generator, the compiled architecture and the saved-weights cache key disagree. * test+docs(engine-burn): mutation group be, weight-loading tests and the Burn engine guide Group be: 15 cases, one target each, all caught (saved-weights marker, load completeness, panic guard, answer length, best proposal, file names, declared card makers, the revision shared by script, architecture and cache key, the catalogue route and the .bpk binding rule). fully_applied is split out so the load checks can be tested without the store's own refusals. Group nf's onnx case follows the rewritten extension check. docs/BURN-ENGINE.md: how to regenerate, the five fixes, the revision bump rule, limits. * feat(engine-burn): CRAFT_AI_BURN_DEVICE=cpu forces the processor, and the processor path is measured * feat(engine-burn): prepare the saved copy when loading from the downloaded ONNX files The host step: given the downloaded encoder.onnx with decoder.onnx beside it, the runner calls craft-ai-burn-setup (exact source bytes checked, reused when already saved) and loads from the burn-prepared folder beside the model folder. Behind the default 'prepare' feature; mutation group be gains two cases (17/17); docs say what is measured. * feat(client): optional burn feature and a catalogue route for SAM 2.1 on Burn craft-ai-client gains the cargo feature burn (ProviderFactory::standard() then knows the engine). SAM 2.1's recipe lists a burn NativeRoute after rten, with the same two published ONNX files (one download for both, nothing converted is hosted). rten stays first and the default. The binding accepts the prepared .bpk file. Engines now list rten, burn, candle; the tests that assumed the Burn slot had no files, or that no built-in engine exists whatever cargo features are on, say what they mean. burn_engine.rs runs two apps through the client with the real prepared files (ignored). * fix(burn-setup): sweep the scratch folders a killed run left, and keep craft-ai-features a dev dependency CodeRabbit on #89: tempdir_in(cache) folders hold the two source copies (about 130 MB) and were removed only by Drop, so a killed app leaked them for good. They now carry the cache key and a role in their names, and after the key's lock is taken the same key's leftovers are removed (other keys' folders, the lock file and links are left alone). Red first: leftovers_of_a_killed_run_are_swept_for_this_key_only. Mutation cases burn-killed-run-leftovers-kept and burn-sweep-takes-other-keys, both caught. craft-ai-features is only used by a unit test, so it moves to dev-dependencies. * fix(engine-burn): review findings on #90 CodeRabbit: the generator now refuses an unknown option (a mistyped --chek used to select the run that rewrites src/model; red first: tools/burn/test_generate_sam21.py failed to import parse_args); the preparation live test needs the prepare feature and is ignored by default; the refusal test no longer leaves burn-prepared/<key>.lock in the shared system temp folder (the model folder is nested one level inside the test's own folder; confirmed the old layout leaked it on this PC); docs, STATUS and the mutation README no longer describe the client feature and catalogue route, which arrive in the next PR, as available. * docs(client): say the Burn route and client feature exist, and what the status line really covers CodeRabbit on #91: the STATUS row said the host step was not built. Given the downloaded encoder.onnx the runner prepares the saved copy itself on first load; the row now says that, names the limits (no progress or Cancel during that load, no app run through the shared binding, no live run on macOS or Linux) and LOCAL-BURN-PREPARATION.md no longer calls engine registration separate work. Also restores the client feature and catalogue route in the docs and rustdoc that the #90 review fix had marked as planned, and the three mutation-README claims for them. * fix(burn-setup): sweep only the scratch folders, not a renamed copy of the key's folder CodeRabbit on #89 (second round): sweep_stale removed every folder starting with <key>., so a damaged copy a person renamed to <key>.backup would be deleted on the next preparation. It now matches only <key>.work., <key>.conv. and <key>.out., the names the scratch folders are made with (one constant for both). Red first: the sweep test gained a <key>.backup folder that must survive. Mutation group bp 13/13 caught, including the new case burn-sweep-takes-a-renamed-copy. * test(burn): the generator's usage errors must exit with code 1 CodeRabbit on #90 (second round): the parse_args tests asserted only SystemExit, which a SystemExit(0) would also satisfy. They now assert the code. Checked against a mutant that exits 0: 6 failures. * fix(engine-burn): a damaged saved copy says it is damaged and where it lives CodeRabbit on #90 (third round) pointed out that a saved copy with a bad manifest or weights fails the same way on every retry, behind the message 'could not be read or written. Try again.' It proposed deleting the copy. The rule that a derived copy is refused, never repaired or deleted for the person, stays (#89 and its tests pin it); the dead end was the wording. craft-ai-burn-setup now answers every unusable saved copy with DAMAGED_COPY (remove it, prepare again, no 'try again'), and the Burn engine adds where it lives: the burn-prepared folder beside the model folder. Tests written first (they did not compile without DAMAGED_COPY and explain): a damaged copy of three kinds gets that answer and is left untouched; the engine explains it and passes other errors on. Mutation: bp 2 new cases and be 1 new case, all caught. * fix(burn-setup): a missing or unreadable saved weight is a damaged copy too CodeRabbit on #90 (fourth round): hash_file(...)? on a saved encoder.bpk or decoder.bpk that is gone returned the generic 'try again' error before the damaged-copy branch could run. It now maps to DAMAGED_COPY. Red first: the damaged-copy test gained 'missing encoder' and 'missing decoder' cases and failed with the old message. New mutation case burn-missing-saved-weight-answered-with-try-again; the saved-hash case was re-aimed at the new line. bp 16/16 caught. * fix(burn-setup): a manifest that cannot be opened or read is a damaged copy too CodeRabbit on #91 (round 5): File::open and read_to_end on an existing manifest returned the generic 'try again' error, so the engine gave no deletion guidance. Both now map to DAMAGED_COPY. Red first: a_manifest_that_cannot_be_opened_is_a_damaged_copy (Windows: another handle holds it exclusively; Unix: mode 0, ending quietly where permissions are not enforced) failed with the old message. The Unix variant was not run here. New mutation case burn-unopenable-manifest-answered-with-try-again; bp 17/17 caught (Windows). * test(burn-setup): write the Unix permission as octal (clippy::non_octal_unix_permissions, CI only) --------- Co-authored-by: scratch <s@s> |
||
|
Jeremy
|
fdad29cdd4 |
feat(engine-burn): run SAM 2.1 for image.segment on Burn (#90)
* refactor(engine-rten): let another engine reuse the SAM 2.1 request handling Loaded, Backend, Prediction, read_prompt, prompt_points and best_proposal become public. Visibility only: the click and box mapping, the picture kept between requests, the empty-mask answer and the size checks then live in one place for every SAM 2.1 engine. * feat(engine-burn): crate shell and the script that generates the SAM 2.1 architecture tools/burn/generate_sam21.py checks the two published ONNX files against the pinned size and SHA-256, runs the pinned burn-onnx 0.22.0 (an example of craft-ai-burn-setup), applies the five asserted fixes, drops the generated loaders that panic on a bad file, splits by submodule so every file is under 700 lines, formats and writes. --check proves the tree is what the script produces. REVISION must equal the saved-weights cache revision. Cargo.lock additions are Burn with wgpu and ndarray (generated). * feat(burn-setup): prepare SAM 2.1 weights for Burn once, on this computer Brings in craft-ai-burn-setup (feature `conversion`): checks the two published ONNX files by size and SHA-256, converts them to Burn weights with the pinned burn-onnx 0.22.0, and keeps one checked, locked, shared copy. Nothing is hosted or downloaded. Mutation group bp (10 cases). Cargo.lock additions are the optional burn-onnx tree (generated). * feat(engine-burn): GENERATED SAM 2.1 tiny architecture (30 files) Output of tools/burn/generate_sam21.py, revision sam21-tiny-burn-0.22.1-onnx-0.22.0-v1, from the published encoder.onnx and decoder.onnx with burn-onnx 0.22.0 and its five asserted fixes. Do not review line by line; rerun the script with --check instead. Longest file 699 lines. Marked linguist-generated. * feat(engine-burn): SAM 2.1 runner for image.segment on Burn Sam21Runner (engine burn, model sam21) over the generated architecture. The request handling, picture kept between requests, empty-mask answer and click/box mapping are the shared Loaded from craft-ai-engine-rten, so only the model run lives here. Weights load through BurnpackStore and come back as errors; every call into the generated code is guarded against panics. Graphics card when one answers, else the processor. An .onnx file is answered with 'prepare it first', not a guess. 16 unit tests on a fake backend: click and box mapping, empty mask, short mask, a failing model, shared conformance run with Cancel, damaged and truncated files. * test(engine-burn): live comparison with rten, and the shared revision check The ignored live test runs the real SAM 2.1 files through rten (published ONNX) and Burn (weights prepared on this computer) on the astronaut sample: one click and a box, best-mask IoU, timings, and a truncated prepared file refused. The revision test fails when the generator, the compiled architecture and the saved-weights cache key disagree. * test+docs(engine-burn): mutation group be, weight-loading tests and the Burn engine guide Group be: 15 cases, one target each, all caught (saved-weights marker, load completeness, panic guard, answer length, best proposal, file names, declared card makers, the revision shared by script, architecture and cache key, the catalogue route and the .bpk binding rule). fully_applied is split out so the load checks can be tested without the store's own refusals. Group nf's onnx case follows the rewritten extension check. docs/BURN-ENGINE.md: how to regenerate, the five fixes, the revision bump rule, limits. * feat(engine-burn): CRAFT_AI_BURN_DEVICE=cpu forces the processor, and the processor path is measured * feat(engine-burn): prepare the saved copy when loading from the downloaded ONNX files The host step: given the downloaded encoder.onnx with decoder.onnx beside it, the runner calls craft-ai-burn-setup (exact source bytes checked, reused when already saved) and loads from the burn-prepared folder beside the model folder. Behind the default 'prepare' feature; mutation group be gains two cases (17/17); docs say what is measured. * fix(burn-setup): sweep the scratch folders a killed run left, and keep craft-ai-features a dev dependency CodeRabbit on #89: tempdir_in(cache) folders hold the two source copies (about 130 MB) and were removed only by Drop, so a killed app leaked them for good. They now carry the cache key and a role in their names, and after the key's lock is taken the same key's leftovers are removed (other keys' folders, the lock file and links are left alone). Red first: leftovers_of_a_killed_run_are_swept_for_this_key_only. Mutation cases burn-killed-run-leftovers-kept and burn-sweep-takes-other-keys, both caught. craft-ai-features is only used by a unit test, so it moves to dev-dependencies. * fix(engine-burn): review findings on #90 CodeRabbit: the generator now refuses an unknown option (a mistyped --chek used to select the run that rewrites src/model; red first: tools/burn/test_generate_sam21.py failed to import parse_args); the preparation live test needs the prepare feature and is ignored by default; the refusal test no longer leaves burn-prepared/<key>.lock in the shared system temp folder (the model folder is nested one level inside the test's own folder; confirmed the old layout leaked it on this PC); docs, STATUS and the mutation README no longer describe the client feature and catalogue route, which arrive in the next PR, as available. * fix(burn-setup): sweep only the scratch folders, not a renamed copy of the key's folder CodeRabbit on #89 (second round): sweep_stale removed every folder starting with <key>., so a damaged copy a person renamed to <key>.backup would be deleted on the next preparation. It now matches only <key>.work., <key>.conv. and <key>.out., the names the scratch folders are made with (one constant for both). Red first: the sweep test gained a <key>.backup folder that must survive. Mutation group bp 13/13 caught, including the new case burn-sweep-takes-a-renamed-copy. * test(burn): the generator's usage errors must exit with code 1 CodeRabbit on #90 (second round): the parse_args tests asserted only SystemExit, which a SystemExit(0) would also satisfy. They now assert the code. Checked against a mutant that exits 0: 6 failures. * fix(engine-burn): a damaged saved copy says it is damaged and where it lives CodeRabbit on #90 (third round) pointed out that a saved copy with a bad manifest or weights fails the same way on every retry, behind the message 'could not be read or written. Try again.' It proposed deleting the copy. The rule that a derived copy is refused, never repaired or deleted for the person, stays (#89 and its tests pin it); the dead end was the wording. craft-ai-burn-setup now answers every unusable saved copy with DAMAGED_COPY (remove it, prepare again, no 'try again'), and the Burn engine adds where it lives: the burn-prepared folder beside the model folder. Tests written first (they did not compile without DAMAGED_COPY and explain): a damaged copy of three kinds gets that answer and is left untouched; the engine explains it and passes other errors on. Mutation: bp 2 new cases and be 1 new case, all caught. * fix(burn-setup): a missing or unreadable saved weight is a damaged copy too CodeRabbit on #90 (fourth round): hash_file(...)? on a saved encoder.bpk or decoder.bpk that is gone returned the generic 'try again' error before the damaged-copy branch could run. It now maps to DAMAGED_COPY. Red first: the damaged-copy test gained 'missing encoder' and 'missing decoder' cases and failed with the old message. New mutation case burn-missing-saved-weight-answered-with-try-again; the saved-hash case was re-aimed at the new line. bp 16/16 caught. * fix(burn-setup): a manifest that cannot be opened or read is a damaged copy too CodeRabbit on #91 (round 5): File::open and read_to_end on an existing manifest returned the generic 'try again' error, so the engine gave no deletion guidance. Both now map to DAMAGED_COPY. Red first: a_manifest_that_cannot_be_opened_is_a_damaged_copy (Windows: another handle holds it exclusively; Unix: mode 0, ending quietly where permissions are not enforced) failed with the old message. The Unix variant was not run here. New mutation case burn-unopenable-manifest-answered-with-try-again; bp 17/17 caught (Windows). * test(burn-setup): write the Unix permission as octal (clippy::non_octal_unix_permissions, CI only) |
||
|
Jeremy
|
870e96048c |
docs: list the rten engine and the selection workflow route, and say what was run (#86)
The README crates table, docs/LAYOUT.md and docs/STATUS.md named only Candle and the SAM 3 clients. They now list craft-ai-engine-rten (SAM 2.1 from the published ONNX files, on the processor), the imported image.segment route on ComfyUI with checked masks, and what was and was not run: Windows only, one picture, no Burn runner, no app through the shared binding yet. |
||
|
Jeremy
|
611195b50e |
feat(ai-comfyui): send a selection picture once and resend it only when it is gone (#82)
* feat(ai-comfyui): import a selection workflow and run it with the picture sent first Adds the `image.segment` capability to imported ComfyUI workflows. A selection workflow maps a text prompt and one source picture; the picture is uploaded, the graph is queued with the name ComfyUI gave it, and every queued graph gets a sink name of its own so a repeat is not answered from cache with no output. Clicks or a box are refused in plain words before anything is sent, and a stored workflow file may declare a source binding and no mask. The answer is the graph's image as it comes. Checking the mask is the next change. * feat(ai-comfyui): check a selection workflow's mask and answer in the shared shape A selection workflow's image is now a mask the shared contract can trust. It must be the size of the picture sent (a JPEG may come back turned a quarter, because ComfyUI's loader applies the EXIF turn), is reduced to one 8-bit channel, and is followed by the shared `Found` result. A mask with no pixel above half is an empty mask and `matched: false`, never a failed job. Picture sizes are read from the PNG or JPEG header and bounded. This is not calibrated absence detection: the example Florence graph finds a box for any phrase. * feat(ai-comfyui): send a selection picture once and resend it only when it is gone A repeat selection on the same picture reuses the file already in ComfyUI's scratch area, keyed by a hash of the bytes. Only the last four pictures are remembered. If ComfyUI has cleared its scratch area (a restart), the first refusal of a reused name drops the entry and sends the picture again once; a refusal of a fresh upload is never retried. A failed selection job clears the cache, since the file may be what went wrong. Adds the ignored live test against a real ComfyUI. * test(ai-comfyui): count refused prompts without the deprecated fetch_update Newer Rust renames AtomicUsize::fetch_update to try_update and warns, which the strict Clippy job turns into an error on CI. The scripted server handles one job at a time here, so a load and a fetch_sub do the same job and build on both toolchains. * fix(ai-comfyui): hand on a binary mask, white where selected and black elsewhere A review finding: the first channel was kept as the graph saved it, so pixels from 128 to 254 reached the app as partly selected, while the shared contract says selected pixels are white. Every pixel above half is now 255 and the rest 0. Written first and seen to fail (the answer kept [0, 100, 128, 200, 255]); mutation cases added for a grey kept as selected and a low grey kept. * docs(ai-comfyui): state the selection mask's byte limit, test it, and rename a mutation case Two review findings. The mask is read inline under the provider's output limit (never more than the 32 MiB inline cap) and never to the output folder, because the checks need its bytes in hand; that is now written in the module header and the provider, and a test shows a mask over a lowered limit fails the job and hands on nothing, with a mutation case for the limit being ignored. The case that only removed the media-type guard is renamed so it no longer implies JPEG answers are covered. The limit test passed on first run, as the behaviour already existed: it is a guard, not a red-first fix. * fix(ai-comfyui): forget the cached pictures when any selection job fails A review finding: only an error status from ComfyUI cleared the upload cache, so a job whose mask was fetched but refused (wrong size, not a PNG) left its picture cached and the next selection could reuse a stale scratch name. The cache is now cleared once, after the job's state is settled, whenever a selection job ends failed. Written first and seen to fail (a wrong-size mask left one upload for two selections); the existing mutation case is re-aimed at the single clear. |
||
|
Jeremy
|
dae74cc8a7 |
feat(ai-comfyui): check a selection workflow's mask and answer in the shared shape (#81)
* feat(ai-comfyui): import a selection workflow and run it with the picture sent first Adds the `image.segment` capability to imported ComfyUI workflows. A selection workflow maps a text prompt and one source picture; the picture is uploaded, the graph is queued with the name ComfyUI gave it, and every queued graph gets a sink name of its own so a repeat is not answered from cache with no output. Clicks or a box are refused in plain words before anything is sent, and a stored workflow file may declare a source binding and no mask. The answer is the graph's image as it comes. Checking the mask is the next change. * feat(ai-comfyui): check a selection workflow's mask and answer in the shared shape A selection workflow's image is now a mask the shared contract can trust. It must be the size of the picture sent (a JPEG may come back turned a quarter, because ComfyUI's loader applies the EXIF turn), is reduced to one 8-bit channel, and is followed by the shared `Found` result. A mask with no pixel above half is an empty mask and `matched: false`, never a failed job. Picture sizes are read from the PNG or JPEG header and bounded. This is not calibrated absence detection: the example Florence graph finds a box for any phrase. * fix(ai-comfyui): hand on a binary mask, white where selected and black elsewhere A review finding: the first channel was kept as the graph saved it, so pixels from 128 to 254 reached the app as partly selected, while the shared contract says selected pixels are white. Every pixel above half is now 255 and the rest 0. Written first and seen to fail (the answer kept [0, 100, 128, 200, 255]); mutation cases added for a grey kept as selected and a low grey kept. * docs(ai-comfyui): state the selection mask's byte limit, test it, and rename a mutation case Two review findings. The mask is read inline under the provider's output limit (never more than the 32 MiB inline cap) and never to the output folder, because the checks need its bytes in hand; that is now written in the module header and the provider, and a test shows a mask over a lowered limit fails the job and hands on nothing, with a mutation case for the limit being ignored. The case that only removed the media-type guard is renamed so it no longer implies JPEG answers are covered. The limit test passed on first run, as the behaviour already existed: it is a guard, not a red-first fix. |
||
|
Jeremy
|
4effe6a190 |
feat(ai-comfyui): import a selection workflow and run it with the picture sent first (#80)
Adds the `image.segment` capability to imported ComfyUI workflows. A selection workflow maps a text prompt and one source picture; the picture is uploaded, the graph is queued with the name ComfyUI gave it, and every queued graph gets a sink name of its own so a repeat is not answered from cache with no output. Clicks or a box are refused in plain words before anything is sent, and a stored workflow file may declare a source binding and no mask. The answer is the graph's image as it comes. Checking the mask is the next change. |
||
|
Jeremy
|
8be4066b1f |
feat(ai-engine-candle): SAM 3 on the Candle engine, and the catalogue names the files it reads (#41)
* feat(ai-engine): a runner contract for models that run on this computer
Candle and Burn were named in the catalogue but no app could share them: each would have needed
its own job handling, cancel, progress and weights loading. This adds the seam so an engine author
writes only the model.
- craft-ai-engine: ModelRunner (engine, model spec, load) and LoadedModel (run), a RunContext for
progress, cancel and notices, an EngineRegistry, and EngineProvider, the one Provider that runs
any registered model (weights loaded once, jobs for one model take turns, a panicking runner
becomes a failed job).
- testing::{FakeRunner, check_runner}: puts a runner behind the real provider and runs the shared
conformance suite.
- craft-ai-client: ProviderFactory::register_engines is the whole of enabling an engine in an app;
register now forgets what was declared about the adapter it replaces, and unregister removes one.
- docs/ADD-AN-ENGINE.md is the crate documentation, so its code is compiled by the doctests.
* feat(ai-engine-candle): SAM 3 on the Candle engine, and the catalogue names the files it reads
The built-in SAM 3 route was documented as reading the same file as ComfyUI. It does not: the
ComfyUI file is Meta's layout (detector.backbone..., 1465 tensors); the Candle port reads the
Hugging Face layout (detector_model..., 1797 tensors) plus vocab.json and merges.txt. A route
pointed at the ComfyUI file could never load.
- ai-features: NativeRoute gains its own `models`, so a runtime can name the files it reads. The
SAM 3 routes name facebook/sam3 at a pinned revision, with sizes and SHA-256 checked against a
real download. NativeRoute::weights and total_bytes read them.
- craft-ai-engine-candle: the model code from LightCraft's lightcraft-segment crate (see NOTICE),
behind the runner contract. image.segment takes a PNG source and either a text prompt or click
points, and answers with a grayscale PNG mask the size of the source. Apache-2.0 only, because
the port is a derivative of Hugging Face Transformers.
- craft-ai-client: the `candle` cargo feature makes ProviderFactory::standard() know the engine, so
an app adopts it by turning the feature on.
* fix(ai-engine,ai-client): keep one provider per connection, and let a cancelled job leave the queue without loading or running
The client asks the factory for a provider on every job, so each job built a fresh EngineProvider and
loaded the weights again; two jobs at once loaded two copies. The factory now keeps one provider per
connection and builds another only when the connection's models or weight files change.
A job waiting behind another for the same model reported Running, and if it was cancelled it still
loaded the weights and ran. It now reports Queued until it holds the model, and a cancel before then
or during a slow load stops it before any work. At most 16 jobs may wait or run at once; a burst is
refused in plain words rather than piling up threads that each hold a copy of the picture.
* fix(ai-client): write the kept-provider key with escapes, not a literal newline and tab
* fix(ai-features,deny): say what the Candle route really runs on, offer no download for Burn, and record the candle advisories
The Candle route said it ran on Nvidia or Apple graphics, but this build declares the Apple card only and
runs on the processor elsewhere; the label now says so. The Burn route named the 3.4 GB Hugging Face files
although no Burn code exists, so it now names none and offers no download.
cargo-deny failed on candle-core's zip (RUSTSEC-2026-0367, 0368, 0369) and paste (RUSTSEC-2024-0436).
Each is ignored with the reason it cannot be reached here: only safetensors are read and no archive is
extracted.
* feat(ai-engine-candle): read JPEG pictures, and turn them upright by their EXIF orientation
A source can now be a PNG or a JPEG, told apart by its first bytes. JPEGs are decoded with zune-jpeg through
the image crate (baseline, progressive, greyscale), refused from the header when over 100 megapixels, and
given an allocation cap. The EXIF orientation is applied, as browsers, photo editors and OpenCV do, so a
phone photo taken sideways is segmented upright and a click normalised to the picture lands where the person
clicked. The mask comes back the size of the upright picture. A PNG is read as stored.
Verified on Meta's checkpoint in release mode: a 128x192 JPEG marked 'turn 90 degrees clockwise' gives a
192x128 mask centred on the disc. Unit tests cover colour and greyscale, every damaged and cut-off case,
a header patched to 30000x30000, and the turn itself (the test fails without it).
jpeg-encoder makes the test JPEGs; it carries the IJG licence, allowed for that one crate in deny.toml.
* test(ai-engine-candle): write the EXIF header with escapes, not raw NUL bytes, so git shows the file as text
* fix(ai-engine-candle): lift the PNG decoder's buffer limit, and bound tokenizer work for long prompts
A one-row picture inside the 100 megapixel limit but with a row over 64 MiB was refused by the png crate's default allowance; the decoder now gets the same 900 MiB cap as the JPEG route. Tokenizer::encode now clips the text to CONTEXT * 32 characters and each word to CONTEXT * 8 before the merge loop, and stops once the ids fill the context. Both found by CodeRabbit on #41.
* docs(ai-engine-candle): doc comments on the functions touched by the review fixes
* fix(ai-engine-candle): collapse whitespace before the tokenizer's character limit
Spaces at the start of a prompt used up the 1,024 character allowance, so a prompt of 1,024 spaces then a word reached the model as nothing. Whitespace is now collapsed first, and the text is copied only as far as the allowance reaches. Found by CodeRabbit on #41.
---------
Co-authored-by: scratch <s@s>
Co-authored-by: t <t@t>
|
||
|
James Chapman
|
ea85917afe |
fix(ai-openai): cap a streamed reply and the events a job holds unpollled (#53)
* fix(ai-vertex): send the express key in a header, not the URL query C4 from the 2026-10-17 security audit. url() percent-encoded the API key and appended "?key=" to every generation request, making the Vertex express adapter the only provider whose credential traveled in the request URL. Request lines are what TLS-intercepting proxies and access logs commonly record - and craft-ai-http deliberately trusts the platform's certificate store, so such middleboxes can be reading - while header sets are rarely logged in full. Every other adapter already kept its key in a header. The key now travels only in the x-goog-api-key header, the same header the Gemini API documents for key auth, and url() no longer touches it. Header injection cannot occur: the key was already validated at construction to contain no control characters or whitespace. The test server now asserts the path carries no key and the header carries the key, and two mutation cases (vx group) pin the rule: putting the key back in the URL, or dropping the header, are both caught. Verified against the scripted server only; per the audit's note, one live express-mode run is still wanted before claiming the official endpoint accepts the header, and nothing here was run against a real provider. Co-Authored-By: Claude Code <noreply@anthropic.com> * test(ai-vertex): pin that submit refuses other models and options before any call A CodeRabbit review round on PR #52 flagged that only the Mask-input refusal was covered. The provider refuses a wrong model, a streamed request, a second output and any param before any network call; each case now asserts ErrorKind::Unsupported and that the scripted server was never contacted (count stays 0). Co-Authored-By: Claude Code <noreply@anthropic.com> * fix(ai-openai): cap a streamed reply and the events a job holds unpollled C5 from the 2026-10-17 security audit, found independently by two auditors. The streaming path capped every hostile input except the total: SseState.text grew by push_str for as long as the stream stayed open, tool-call arguments likewise, and every delta was also pushed as a TextDelta event into the job cell, which grew without end if nobody polled. A hostile OpenAI-shaped server could accumulate text twice for the full 600-second request timeout, into the multi-gigabyte range. The reply is now bounded three ways: - Per line: unchanged (MAX_SSE_LINE, 1 MiB). - Total bytes kept: new MAX_STREAM_TOTAL, the same size as the whole-reply cap http::MAX_BODY (128 MiB), so a stream can never hold more than a whole reply would. Text and tool-call name/argument bytes count against one shared budget; a line that passes it fails the stream with "the streamed reply was too large". - Events held for one poll: new MAX_EVENTS (65,536) in JobCell. A stream that produces more without a poll fails the job with "too many events are waiting; poll the job while it runs" and the backlog is dropped, rather than growing for as long as the request timeout allows. The worker's line loop now stops when the cell is terminal, not only when cancelled: a job failed by the backlog cap must also stop reading, drop the connection and free its worker for the rest of the 600-second window it would otherwise burn. Mid-stream a terminal cell is exactly cancel or backlog failure, since the worker sets the terminal states only after the loop returns. chat.rs was at the 500-line soft limit and this grows the streaming half, so the SSE reader moved to stream.rs (SseState, MAX_SSE_LINE, MAX_STREAM_TOTAL) and chat.rs keeps request planning and whole-reply parsing; both are well under the limit. The total cap is a field defaulted to MAX_STREAM_TOTAL with a test-only with_cap, so a stream can be driven past the limit cheaply, the same trick read_capped's parameter takes. Six u1 mutation cases pin the rules, all caught: removing either cap check, unlimited defaults, a loosened event cap, a backlog kept after failure, and a worker still reading a dead cell's stream. The last is caught by a new provider test that streams 70,000 deltas nobody polls: the job fails at the cap and the server sees the connection drop. Verified against the scripted server only; no real provider contacted. Co-Authored-By: Claude Code <noreply@anthropic.com> * test(ai-openai): leave the firehose far more unread than kernel socket buffers CI failed on all three OSes: the worker failed the job at the event cap and dropped the connection, but the server never reported the drop. The scripted server only notices when a write fails, and with 70_000 chunks about 210 KB remained when the worker stopped - little enough that CI's autotuned kernel buffers swallowed it, the server finished writing, and client_dropped never became true. One million chunks leave about 44 MB unread, which no kernel buffers; the drop is observed within the deadline on every platform. CodeRabbit flagged the same line on PR #53 and asked for 400_000; the larger count widens the margin past even generous Linux autotuning. Each line still holds one byte of text and only 65_537 lines are ever read, so the byte total stays far under the stream cap. The dead-cell-still-read mutation case still catches its mutant. Co-Authored-By: Claude Code <noreply@anthropic.com> * docs(ai-openai,ai-vertex): document every function the PR's diff touches CodeRabbit's docstring coverage pre-merge check on PR #53 reads 72% against an 80% threshold, scoped to functions touched by the diff (25 functions across 8 files). The comment names no functions, so every diff-touched function in the stacked diff that lacked a /// now has one: jobs.rs (push, succeed, run_chat and two unit tests), stream.rs (Default, tool_delta and three unit tests), the three touched provider tests in each crate, and vertex lib.rs's lock and generate. Production docs state the bound or contract; test docs are one line naming the behaviour the test pins. No code changes; all tests pass and every mutation target still resolves uniquely. Co-Authored-By: Claude Code <noreply@anthropic.com> --------- Co-authored-by: Claude Code <noreply@anthropic.com> |
||
|
Jeremy
|
72bf336f96 |
feat(ai-core,ai-comfy-env,ai-client): the person's model folders, kept as a map and never reorganised (#40)
* feat(ai-core,ai-comfy-env,ai-client): the person's model folders, kept as a map and never reorganised People already keep models in folders of their own. This lets the suite point at them instead of asking for a layout. - ai-core::model_folders: a folder, what is known to be in it (kind to sub-folders, with ComfyUI's names), the checks every list passes before it is stored (full paths, no sub-folder that leaves its folder, one default, limits), and download_target, which never picks a folder read from another program. StoreFile gains `model_folders`; older files load unchanged. - ai-comfy-env: scan_model_folder reads a folder and records where each kind lives, from a known file name and size, a folder name, or a safetensors header. Read only, bounded by depth, files, entries and header size, and a link is followed once. render_extra_paths writes the map as a ComfyUI extra-model-paths file, leaving out any folder that would not pass the store's checks. - ai-client: FolderDesk to add, rescan, choose the default, remove and write the extra-paths file. - ai-native: the folder the launcher proposes when a person has none. * fix(ai-client,ai-native): removing the last model folder sticks, case counts only where the file system says so, and the macOS test skips The store carries unknown top-level keys through a rewrite, and model_folders was not on its list of known keys. With the last folder removed the typed list is empty and is left out of the output, so the old list was put back from the file: the folder could not be removed. It is a known key now. A duplicate folder was found by lower-casing the whole path, which made /Models and /models the same folder on Linux. Case is ignored only on Windows and macOS now, and the Windows verbatim prefix is dropped so a folder that exists and one that does not still meet. The non-text path test cannot create its folder on macOS, which refuses such names; it skips there. --------- Co-authored-by: scratch <s@s> |
||
|
Jeremy
|
69ebbc0491 |
feat(ai-engine): a runner contract for models that run on this computer (#39)
* feat(ai-engine): a runner contract for models that run on this computer
Candle and Burn were named in the catalogue but no app could share them: each would have needed
its own job handling, cancel, progress and weights loading. This adds the seam so an engine author
writes only the model.
- craft-ai-engine: ModelRunner (engine, model spec, load) and LoadedModel (run), a RunContext for
progress, cancel and notices, an EngineRegistry, and EngineProvider, the one Provider that runs
any registered model (weights loaded once, jobs for one model take turns, a panicking runner
becomes a failed job).
- testing::{FakeRunner, check_runner}: puts a runner behind the real provider and runs the shared
conformance suite.
- craft-ai-client: ProviderFactory::register_engines is the whole of enabling an engine in an app;
register now forgets what was declared about the adapter it replaces, and unregister removes one.
- docs/ADD-AN-ENGINE.md is the crate documentation, so its code is compiled by the doctests.
* fix(ai-engine,ai-client): keep one provider per connection, and let a cancelled job leave the queue without loading or running
The client asks the factory for a provider on every job, so each job built a fresh EngineProvider and
loaded the weights again; two jobs at once loaded two copies. The factory now keeps one provider per
connection and builds another only when the connection's models or weight files change.
A job waiting behind another for the same model reported Running, and if it was cancelled it still
loaded the weights and ran. It now reports Queued until it holds the model, and a cancel before then
or during a slow load stops it before any work. At most 16 jobs may wait or run at once; a burst is
refused in plain words rather than piling up threads that each hold a copy of the picture.
* fix(ai-client): write the kept-provider key with escapes, not a literal newline and tab
---------
Co-authored-by: scratch <s@s>
|
||
|
Jeremy
|
aea912bb55 |
chore: prepare the repository for transfer to storytold/craft-ai-integrations
- Rename the contract crate ai-model to ai-core (package craft-ai-core), matching the crate proposed for craft-libs. - Port the two fixes already made in the craft-libs copy: consent expiry is per kind of data, and models found by a test at a new address survive a save (ConnectionDraft::set_models). - Take docs/research, the SAM 3 handoff notes and their scripts out of the tree (local paths and evidence files). Move the deliberate-bug harness to tools/mutation and repair ten cases that no longer matched the source. - Rewrite the README, add docs/STATUS.md (tested, run for real, not verified), fix LAYOUT, PLATFORM-CONTRACT and CONTRIBUTING, add the missing ai-comfyui manifest fields, point the repository URL at storytold/craft-ai-integrations, and add a wasm check for craft-ai-features. - Allow clippy::large_enum_variant on TestTarget, which the larger draft now trips. Checked on Windows: fmt, clippy --all-targets -D warnings on all 16 crates, 967 tests passed (15 ignored), wasm check for craft-ai-core and craft-ai-features. Not run: cargo-deny, Linux, macOS, live services, the real keychain test. The contract stays provisional until craft-libs review. |