From 5bb43bdb136277f4d0c268edb79e87ad0600a541 Mon Sep 17 00:00:00 2001
From: jyviko <2501648+jyviko@users.noreply.github.com>
Date: Fri, 30 Oct 2026 20:00:22 -0400
Subject: [PATCH] CI: the test jobs grade their junit against
 tests/KNOWN_FAILING_TESTS

Native Linux tests and the GPU path test shards no longer fail on any failed test: nextest's
"tests failed" exit (100) goes to `cargo xtask known-failing --junit`, which fails on a failure
the list does not name and on a listed test that passes. Any other nextest exit (a build error,
a bad filter) still fails the step at once. The tests skill says to list a failing test with its
cause, in place of ignoring it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---
 .github/workflows/ci.yml         | 17 +++++++++++++++--
 skills/solidcraft-tests/SKILL.md |  2 +-
 2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 0d3531e476..1edd64fea2 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -527,8 +527,15 @@ jobs:
           if [ "$FULL" != true ]; then scope=(); for p in $PACKAGES; do scope+=(-p "$p"); done; fi
           # The GPU path tracer's tests run in their own job (GPU path tests): one at a time, they
           # took about 36 of this job's 45 minutes.
+          # A failed test does not fail the step by itself: the tests known to fail today
+          # (tests/KNOWN_FAILING_TESTS) may fail, any other fails it, and so does a listed test
+          # that passes. nextest's exit code 100 is "tests failed"; any other is its own error
+          # (a build error, a bad filter) and fails the step at once.
+          status=0
           cargo nextest run "${scope[@]}" --cargo-profile ci --locked --profile ci --no-tests=pass \
-            --ignore-default-filter -E "($base) & not (package(=solidcraft-gpu) & test(/^path::/))"
+            --ignore-default-filter -E "($base) & not (package(=solidcraft-gpu) & test(/^path::/))" || status=$?
+          if [ "$status" -ne 0 ] && [ "$status" -ne 100 ]; then exit "$status"; fi
+          cargo xtask known-failing --junit "$junit"
       - name: Compiler cache statistics
         if: always()
         run: sccache --show-stats
@@ -620,8 +627,14 @@ jobs:
           # None of these is in the default filter (the heavy tests), so pull requests, the merge
           # queue, main and nightly all run every one of them, as before. `count` deals the tests
           # out in turn (27 each); `hash` split them 41/40/27 across three shards.
+          # Graded against tests/KNOWN_FAILING_TESTS, as in Native Linux tests.
+          junit=${CARGO_TARGET_DIR:-target}/nextest/ci/junit.xml
+          rm -f "$junit"
+          status=0
           cargo nextest run -p solidcraft-gpu --cargo-profile ci --locked --profile ci --no-tests=pass \
-            --ignore-default-filter -E 'test(/^path::/)' --partition "count:$SHARD/4"
+            --ignore-default-filter -E 'test(/^path::/)' --partition "count:$SHARD/4" || status=$?
+          if [ "$status" -ne 0 ] && [ "$status" -ne 100 ]; then exit "$status"; fi
+          cargo xtask known-failing --junit "$junit"
       - name: Compiler cache statistics
         if: always()
         run: sccache --show-stats
diff --git a/skills/solidcraft-tests/SKILL.md b/skills/solidcraft-tests/SKILL.md
index 424440e1cd..9a5a04e8ee 100644
--- a/skills/solidcraft-tests/SKILL.md
+++ b/skills/solidcraft-tests/SKILL.md
@@ -19,7 +19,7 @@ The long form is `docs/book/src/dev/testing.md`.
 - The rig (`tests/kernel-cases/`) asserts geometry: exact volumes, shapes, seam continuity, geometric refusals, through commands with a hand-derived answer, so it survives a kernel swap (`fillets/190-fillet-disc-rim-past-its-2mm-wall`). A test doing both keeps only its logic.
 
 ## A failing test needs its fix
-A failing test cannot merge without its fix (it turns `main` red): include the fix, or mark the test `#[ignore = "fails until #123 is fixed"]` and remove that line in the PR that fixes it.
+A failing test cannot merge without its fix (it turns `main` red): include it, or list the test and cause in `tests/KNOWN_FAILING_TESTS` (`cargo xtask known-failing`); the fix drops the line.
 
 ## Running
 - `cargo xtask test` builds with the `ci` profile and runs every test binary at once; arguments after `--` go to each binary (`cargo xtask test -- shell`). It builds several times faster than `--release`; a test failing only in a debug build: say so in the PR.