mirror of
https://github.com/bherbruck/solidcraft.git
synced 2026-11-03 02:25:55 +00:00
CI: the test jobs grade their junit against tests/KNOWN_FAILING_TESTS
Native Linux tests and the GPU path test shards no longer fail on any failed test: nextest's "tests failed" exit (100) goes to `cargo xtask known-failing --junit`, which fails on a failure the list does not name and on a listed test that passes. Any other nextest exit (a build error, a bad filter) still fails the step at once. The tests skill says to list a failing test with its cause, in place of ignoring it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -527,8 +527,15 @@ jobs:
|
||||
if [ "$FULL" != true ]; then scope=(); for p in $PACKAGES; do scope+=(-p "$p"); done; fi
|
||||
# The GPU path tracer's tests run in their own job (GPU path tests): one at a time, they
|
||||
# took about 36 of this job's 45 minutes.
|
||||
# A failed test does not fail the step by itself: the tests known to fail today
|
||||
# (tests/KNOWN_FAILING_TESTS) may fail, any other fails it, and so does a listed test
|
||||
# that passes. nextest's exit code 100 is "tests failed"; any other is its own error
|
||||
# (a build error, a bad filter) and fails the step at once.
|
||||
status=0
|
||||
cargo nextest run "${scope[@]}" --cargo-profile ci --locked --profile ci --no-tests=pass \
|
||||
--ignore-default-filter -E "($base) & not (package(=solidcraft-gpu) & test(/^path::/))"
|
||||
--ignore-default-filter -E "($base) & not (package(=solidcraft-gpu) & test(/^path::/))" || status=$?
|
||||
if [ "$status" -ne 0 ] && [ "$status" -ne 100 ]; then exit "$status"; fi
|
||||
cargo xtask known-failing --junit "$junit"
|
||||
- name: Compiler cache statistics
|
||||
if: always()
|
||||
run: sccache --show-stats
|
||||
@@ -620,8 +627,14 @@ jobs:
|
||||
# None of these is in the default filter (the heavy tests), so pull requests, the merge
|
||||
# queue, main and nightly all run every one of them, as before. `count` deals the tests
|
||||
# out in turn (27 each); `hash` split them 41/40/27 across three shards.
|
||||
# Graded against tests/KNOWN_FAILING_TESTS, as in Native Linux tests.
|
||||
junit=${CARGO_TARGET_DIR:-target}/nextest/ci/junit.xml
|
||||
rm -f "$junit"
|
||||
status=0
|
||||
cargo nextest run -p solidcraft-gpu --cargo-profile ci --locked --profile ci --no-tests=pass \
|
||||
--ignore-default-filter -E 'test(/^path::/)' --partition "count:$SHARD/4"
|
||||
--ignore-default-filter -E 'test(/^path::/)' --partition "count:$SHARD/4" || status=$?
|
||||
if [ "$status" -ne 0 ] && [ "$status" -ne 100 ]; then exit "$status"; fi
|
||||
cargo xtask known-failing --junit "$junit"
|
||||
- name: Compiler cache statistics
|
||||
if: always()
|
||||
run: sccache --show-stats
|
||||
|
||||
@@ -19,7 +19,7 @@ The long form is `docs/book/src/dev/testing.md`.
|
||||
- The rig (`tests/kernel-cases/`) asserts geometry: exact volumes, shapes, seam continuity, geometric refusals, through commands with a hand-derived answer, so it survives a kernel swap (`fillets/190-fillet-disc-rim-past-its-2mm-wall`). A test doing both keeps only its logic.
|
||||
|
||||
## A failing test needs its fix
|
||||
A failing test cannot merge without its fix (it turns `main` red): include the fix, or mark the test `#[ignore = "fails until #123 is fixed"]` and remove that line in the PR that fixes it.
|
||||
A failing test cannot merge without its fix (it turns `main` red): include it, or list the test and cause in `tests/KNOWN_FAILING_TESTS` (`cargo xtask known-failing`); the fix drops the line.
|
||||
|
||||
## Running
|
||||
- `cargo xtask test` builds with the `ci` profile and runs every test binary at once; arguments after `--` go to each binary (`cargo xtask test -- shell`). It builds several times faster than `--release`; a test failing only in a debug build: say so in the PR.
|
||||
|
||||
Reference in New Issue
Block a user