masi/solvecraft
1
0
Fork 0
mirror of https://github.com/bherbruck/solidcraft.git synced 2026-11-03 10:35:58 +00:00
Code Issues Packages Projects Releases 1 Wiki Activity

ladder: every ladder run measures one case at a time

Browse Source 
Runs are compared with the golden's times, so a run sharing the machine between cases flags
slowdowns that are only contention (seen with --jobs 4: four cases two to ten times slower).
--jobs still overrides it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QWhMwdqJYQKQq8RCLSL1LT
This commit is contained in:
jyviko
2026-10-21 06:41:58 -04:00
parent 35ebc816ba
commit 248fee75b9
3 changed files with 8 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/kernel-cases/README.md
+1 -1
View File
@@ -107,7 +107,7 @@ cargo xtask kernel-cases --ladder --kernel both --cli target/release/solidcraft-
cargo xtask kernel-cases --ladder --kernel both --cli target/release/solidcraft-cli --record "why"
```
`--record` measures one case at a time, rewrites the golden and sorts each tier by the first
Ladder runs measure one case at a time (unless `--jobs`); `--record` rewrites the golden and sorts each tier by the first
kernel's time, fastest first; commit both files. The report, `docs/kernel-ladder.md`, gives per
tier and kernel the counts per status and the **frontier**: the first case in ladder order the
kernel fails.
xtask/src/kernel_ladder.rs
+5 -3
View File
@@ -5,8 +5,8 @@
//! areas). Each case is a test like any rig case, graded against its own `expected.json`. On top,
//! `tests/kernel-cases/ladder/golden.json` records per case and kernel the status, the commands'
//! time and what the kernel made (counts, volume, area): the run compares with it and flags
//! slowdowns and changes. `--record REASON` re-measures the golden (one case at a time unless
//! `--jobs` says otherwise, so the times are not shared with other cases) and re-sorts each tier
//! slowdowns and changes; cases run one at a time unless `--jobs` says otherwise, so the times are
//! not shared with other cases. `--record REASON` re-measures the golden and re-sorts each tier
//! by the first kernel's time, fastest first; both files are committed.
//!
//! The report, `docs/kernel-ladder.md`, gives per tier and kernel the counts per status and the
@@ -264,7 +264,9 @@ pub fn run(rest: &[&str]) -> Result<(), String> {
Some(Some(r)) => Some(r.to_string()),
};
let mut o = options(rest)?;
if reason.is_some() && !rest.contains(&"--jobs") {
// Times are compared with the golden: cases run one at a time unless --jobs says otherwise,
// so they don't share the machine with each other.
if !rest.contains(&"--jobs") {
o.jobs = 1;
}
let manifest = read(&manifest_path());
xtask/src/main.rs
+2 -2
View File
@@ -54,8 +54,8 @@ commands:
every result as JSON too)
--ladder [--record REASON]: the complexity ladder instead
(tests/kernel-cases/ladder/ladder.json, tiers in order), graded and compared
with its timing golden, into docs/kernel-ladder.md; --record re-measures the
golden one case at a time and re-sorts each tier by time
with its timing golden (one case at a time unless --jobs), into
docs/kernel-ladder.md; --record re-measures the golden and re-sorts each tier by time
action-matrix [--bless] [--results FILE] [--from DIR]
try every ordinary action on every face, edge, vertex and body of the design
corpus and compare with tests/action_matrix/baseline.json (--bless rewrites it;