Commit Graph

21 Commits

Author SHA1 Message Date
Jacob Oursland 204e87e005 scripts: two-tier mutation testing harness
Mutation testing finds behavior no check constrains. For solidcore the useful
question is two-tier: a mutant matters only if neither the package's unit tests
nor the unit's difftest notices.

scripts/mutants.sh runs cargo-mutants (unit tests) on a module, a shard or a PR
diff, then patches each unit-test survivor into the working tree and replays it
against difftest --unit <id>, classifying it as caught-by-difftest or
both-tier-missed. Docker mode reuses one container for the replays; the script
prefers the pinned rustup toolchain and runs under nice with CARGO_BUILD_JOBS=2.

docs/mutation-testing.md documents the method, measured costs (~16 s/mutant,
~1 min/survivor), triage, and a CI proposal (per-PR in-diff tier 1, nightly
sharded tier 2), with the pilot findings: topology.topology's change-tracker
bookkeeping is unconstrained by both tiers.

reports/mutants/ is gitignored.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-23 10:51:51 -07:00
bherbruck 543289917c site: track solids (primitives built/passing), corpus placeholder
The primitives example takes --json <path> and records per shape whether it
built, the blocker (unit + upstream symbol), and each check (manifold, Euler,
counts, volume, area, bbox) with expected/actual/rel. error. The checks run
independently (evaluate); check() keeps its first-failure messages, and the
human-readable output is unchanged.

CI's test job runs it and, on pushes to main, uploads solids-report;
scripts/fetch-solids.sh fetches it with fetch-reports.sh's provenance rule
and the Pages build uses it. New Solids page (headline count, table of
primitives with checks and blockers, Corpus (real models) placeholder) and an
overview tile; both say "no data yet" without the report.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 22:55:47 -04:00
Henrik Lampe 6599154c96 Extend SweepStore with native constructors and retained merge services
Use one borrowed/owned store for sweep and primitive access. Preserve caller ownership, source notifications and merge state across errors and unwinding; verify real point-to-edge construction against the original C++ kernel.

Co-Authored-By: Codex <noreply@openai.com>
2026-10-22 15:41:04 +02:00
bherbruck 6fc2a8f199 Merge pull request #486 from tdamsma/difftest/wasm-digest
difftest: Rust-only digest mode and wasm32-wasip1 build (native vs wasm digests)
2026-10-22 05:21:48 +00:00
bherbruck ead02a9ec1 ci: cut CI wall time (parallel jobs, opt-level 0 difftest glue, time-ordered queue)
- difftest no longer waits for the test job; cpp clippy moves to the clippy job and
  the runner tests to the test job (both 8 vCPU), sharing a composite action that
  clones upstream, enables sccache and installs the C++ reference.
- [profile.difftest.package.difftest] opt-level = 0: the ~300k-line runner crate took
  ~5 of the ~6 build minutes at opt-level 3; solidcore stays at 3. Reports are
  byte-identical.
- difftest --jobs orders its queue by each unit's wall time in the previous run
  (reports/difftest/times.tsv, uploaded with main's reports), so the slowest unit
  starts first and the run ends at the longest unit's time.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158BfaB2oaoX64fskuKP1bm
2026-10-22 00:55:52 -04:00
Thijs Damsma b1f65b3513 difftest: --digest-dir (Rust side only) and a wasm32-wasip1 build for native vs wasm digests
- `difftest --digest-dir DIR [--digest-debug]` runs only `run_rust` and writes the
  DIFFTEST_RUST_DIGEST-style per-case digest (and optionally the Debug output).
- On wasm32, solidcore-ffi links no C++: shims stay `env` imports, served by the stub
  module from scripts/wasm_stub.py (SC_ERR_UNLINKED + call counter -> `needs-cpp`).
- scripts/wasm-digest.sh builds both targets, runs native and under wasmtime, compares.
- solidcore-ffi: MAX_LOOP_INDEX `1 << 40` overflowed on 32-bit targets (E0080).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9QwJFBTKLZ9FFK3nVd1X4
2026-10-22 06:37:37 +02:00
bherbruck 688f1a7ac0 fetch-reports: trust only push runs of this repo's ci.yml on main
Selecting artifacts by head_branch alone accepted fork PR runs whose branch
is named main. Runs are now listed per workflow and filtered on event, path,
repository and head repository before their artifact is downloaded.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158BfaB2oaoX64fskuKP1bm
2026-10-21 19:52:03 -04:00
bherbruck bdc9e08694 fetch-reports: look up the artifact directly; runner doc points at plan/units
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-21 19:44:32 -04:00
bherbruck fe4a9c1425 infra: stop shared-file conflicts (per-unit plan files, reports from CI, union merges)
- plan/modules.toml -> plan/units/<id>.toml (scripts/units-split.py), read everywhere
  through harness/plan/units.py (Python) and crates/difftest/src/plan.rs (difftest).
- reports/difftest/ untracked and gitignored: CI uploads main's set as the
  difftest-reports artifact; scripts/fetch-reports.sh downloads it for the progress
  checklist, the site and the --jobs queue order (fixture-size fallback otherwise).
- .gitattributes: merge=union for plan/findings.md and plan/upstream-issues.md.
- scripts/units-migrate-branch.sh moves an old branch's modules.toml edits into the
  unit files after it merges main.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-21 19:39:59 -04:00
bherbruck 100535f457 Merge pull request #333 from tdamsma/perf/basis-eval-stack
perf: point evaluation without a heap basis array (outputs bit-identical)
2026-10-21 14:38:45 +00:00
Thijs Damsma eb7b5391b2 difftest: document n= in TIME line; analyze.py: close profile file
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9QwJFBTKLZ9FFK3nVd1X4
2026-10-21 03:38:03 +02:00
Thijs Damsma 9d53492183 Merge origin/main into perf/kernel-eval-allocs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9QwJFBTKLZ9FFK3nVd1X4
2026-10-21 03:38:03 +02:00
Thijs Damsma 1e6b0bf20e Merge remote-tracking branch 'origin/main' into perf/basis-eval-stack 2026-10-21 01:29:02 +02:00
Thijs Damsma 2c78dac714 perf: point evaluation without a heap basis array (outputs bit-identical)
- KnotVector::basis_eval_into: N_BasisEval into a caller buffer (same
  values, same errors); basis_eval wraps it
- srf_nurb::Surface::eval (N_SrfEvalPt) and crv_nurb::Curve::eval
  (N_CrvEval) keep N on the stack, as upstream; one malloc/free less per
  basis per point evaluation
- test: basis_eval_into equals basis_eval bit for bit, errors included
- scripts/perf/samply_top.py: self/inclusive/callee/allocator tables and a
  Rust/C++ split from samply profiles (no GUI)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9QwJFBTKLZ9FFK3nVd1X4
2026-10-20 22:27:53 +02:00
bherbruck 49ca9d2475 ci: prebuilt C++ reference download + sccache for shims
scripts/cpp-ref-key.sh names a harness/cpp/build.sh output (upstream commit plus a
hash of CMakeLists.txt, build.sh, stubs/**, patches/**). The new cpp-reference
workflow builds it on Ubuntu 24.04 and publishes it as GitHub Release
cpp-ref-<key>; scripts/fetch-cpp.sh downloads and verifies it (curl, no token) or
falls back to build.sh. CI's difftest job fetches instead of building and runs
Rust and the cc-compiled shims through sccache.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pfe7dodPG5rpuN9B8QcnuX
2026-10-20 15:02:38 -04:00
bherbruck 229a42dad7 ci: run every job on Blacksmith, drop the self-hosted route
Pushes to main used the org's self-hosted runners (two of four are offline) while every PR
already ran on Blacksmith's sponsored runners. Move main onto Blacksmith too, give
difftest 16 vCPU, and remove the self-hosted-only steps and warm-target.sh. Caching is
actions/cache throughout. Fallback when the sponsorship ends: blacksmith-* labels back
to ubuntu-latest.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pfe7dodPG5rpuN9B8QcnuX
2026-10-20 13:01:11 -04:00
Thijs Damsma af94851172 difftest: per-unit TIME line, opt-in Rust output digest
- UnitResult.timing (not in the report): wall time of case building,
  run_cpp, run_rust, compare and the slowest Rust case; main prints
  `TIME <unit> ...` to stderr per unit
- DIFFTEST_RUST_DIGEST=<dir>: writes <dir>/<unit>.digest with one
  `<case>\t<fnv1a64 of format!("{rust_output:?}")>` line per case, so two
  builds can be proved bit-identical; off by default
- README: --green takes ~35 min on 16 threads (not ~2 min), TIME line,
  digest, and scripts/perf (ptrace sampler + analyzer, no sudo needed)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9QwJFBTKLZ9FFK3nVd1X4
2026-10-20 16:45:42 +02:00
bherbruck 433e1b2cbd ci: run trusted jobs on the self-hosted runners (#193)
* ci: run trusted jobs on the self-hosted runners

Pushes to main and pull requests from branches of this repository now run fmt,
clippy, tests, wasm, the publish dry run and difftest on the org's self-hosted
runners, which keep their Cargo build directory and C++ reference build between
jobs. cargo-deny (a container action), fork pull requests and Run workflow stay
on GitHub's runners. The C++ cache on GitHub is no longer shared with the
self-hosted build, whose newer compiler and glibc must not reach GitHub's runners.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci: re-run on the self-hosted runners

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 10:05:15 -04:00
bherbruck 11bc973555 difftest --ssi-corpus: review fixes (exact float parsing, copyright, plane NURB deviation)
- difftest: serde_json float_roundtrip, so inputs parse correctly rounded like strtod
  (1610 corpus numbers were 1 ULP off); regenerated reports/ssi-corpus.json and the five
  difftest reports whose values_compared counts moved (all still pass; --green 0 failed)
- ATTRIBUTION.md: ssi-corpus copyright line from its LICENSE-MIT
- surface.plane_p1/p2: record that natural_uv_domain does not cache the NURB as
  upstream GetNaturalUVDomain does, which changes IntersectWithPlane's parallel branch;
  findings entry moved under surface.plane_p1, follow-up in modules.toml notes
- engine: Rust side rejects what the C++ encoder rejects; torus deferrals name
  surface.surface_p2 (owner of SurfaceKind); shim degree bound NL_MAXDEG
- fetch script: refuse an existing checkout with local changes

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BY8UpXKuMci15rAWGXsbsm
2026-10-14 21:43:06 -04:00
bherbruck 87d1ab2c85 difftest --ssi-corpus: full-SHA corpus pin, upstream issues WR-74..80, findings
- scripts/fetch-ssi-corpus.sh pins b04c8a6093a8b3505d865d1dd874ec3879ccb1f9, checks
  the existing checkout's origin URL and requires HEAD to equal the pin
- plan/upstream-issues.md: the 38 C++ misses on the corpus, grouped (WR-74..WR-80)
- plan/findings.md: scorer agreement and the plane NURB state note

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BY8UpXKuMci15rAWGXsbsm
2026-10-14 20:30:24 -04:00
bherbruck d267b51b83 difftest: --ssi-corpus mode scoring Rust and C++ against jyviko's SSI corpus (WIP)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BY8UpXKuMci15rAWGXsbsm
2026-10-14 20:12:39 -04:00