CI: the test jobs grade their junit against tests/KNOWN_FAILING_TESTS

Native Linux tests and the GPU path test shards no longer fail on any failed test: nextest's
"tests failed" exit (100) goes to `cargo xtask known-failing --junit`, which fails on a failure
the list does not name and on a listed test that passes. Any other nextest exit (a build error,
a bad filter) still fails the step at once. The tests skill says to list a failing test with its
cause, in place of ignoring it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
jyviko
2026-10-30 20:00:22 -04:00
committed by jyviko
parent a26d3ee41c
commit 5bb43bdb13
2 changed files with 16 additions and 3 deletions
+15 -2
View File
@@ -527,8 +527,15 @@ jobs:
if [ "$FULL" != true ]; then scope=(); for p in $PACKAGES; do scope+=(-p "$p"); done; fi
# The GPU path tracer's tests run in their own job (GPU path tests): one at a time, they
# took about 36 of this job's 45 minutes.
# A failed test does not fail the step by itself: the tests known to fail today
# (tests/KNOWN_FAILING_TESTS) may fail, any other fails it, and so does a listed test
# that passes. nextest's exit code 100 is "tests failed"; any other is its own error
# (a build error, a bad filter) and fails the step at once.
status=0
cargo nextest run "${scope[@]}" --cargo-profile ci --locked --profile ci --no-tests=pass \
--ignore-default-filter -E "($base) & not (package(=solidcraft-gpu) & test(/^path::/))"
--ignore-default-filter -E "($base) & not (package(=solidcraft-gpu) & test(/^path::/))" || status=$?
if [ "$status" -ne 0 ] && [ "$status" -ne 100 ]; then exit "$status"; fi
cargo xtask known-failing --junit "$junit"
- name: Compiler cache statistics
if: always()
run: sccache --show-stats
@@ -620,8 +627,14 @@ jobs:
# None of these is in the default filter (the heavy tests), so pull requests, the merge
# queue, main and nightly all run every one of them, as before. `count` deals the tests
# out in turn (27 each); `hash` split them 41/40/27 across three shards.
# Graded against tests/KNOWN_FAILING_TESTS, as in Native Linux tests.
junit=${CARGO_TARGET_DIR:-target}/nextest/ci/junit.xml
rm -f "$junit"
status=0
cargo nextest run -p solidcraft-gpu --cargo-profile ci --locked --profile ci --no-tests=pass \
--ignore-default-filter -E 'test(/^path::/)' --partition "count:$SHARD/4"
--ignore-default-filter -E 'test(/^path::/)' --partition "count:$SHARD/4" || status=$?
if [ "$status" -ne 0 ] && [ "$status" -ne 100 ]; then exit "$status"; fi
cargo xtask known-failing --junit "$junit"
- name: Compiler cache statistics
if: always()
run: sccache --show-stats
+1 -1
View File
@@ -19,7 +19,7 @@ The long form is `docs/book/src/dev/testing.md`.
- The rig (`tests/kernel-cases/`) asserts geometry: exact volumes, shapes, seam continuity, geometric refusals, through commands with a hand-derived answer, so it survives a kernel swap (`fillets/190-fillet-disc-rim-past-its-2mm-wall`). A test doing both keeps only its logic.
## A failing test needs its fix
A failing test cannot merge without its fix (it turns `main` red): include the fix, or mark the test `#[ignore = "fails until #123 is fixed"]` and remove that line in the PR that fixes it.
A failing test cannot merge without its fix (it turns `main` red): include it, or list the test and cause in `tests/KNOWN_FAILING_TESTS` (`cargo xtask known-failing`); the fix drops the line.
## Running
- `cargo xtask test` builds with the `ci` profile and runs every test binary at once; arguments after `--` go to each binary (`cargo xtask test -- shell`). It builds several times faster than `--release`; a test failing only in a debug build: say so in the PR.