Kernel rig: every registry command has a case or an exemption (kernel-cases --coverage)

cargo xtask kernel-cases --coverage lists every built registry command that no recipe.json under
tests/kernel-cases runs and fails unless tests/kernel-cases/EXEMPT lists it. EXEMPT holds 252
commands with no geometry to grade and, under [to cover], 133 geometry commands still waiting for
a case; a listed command a case now uses, or one the registry no longer has, fails the check, so
the to-cover section only shrinks. The gate's build job runs it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
jyviko
2026-10-29 16:55:07 -04:00
committed by Irakli Kourtis
parent b67e93211d
commit d671bbbc12
3 changed files with 583 additions and 2 deletions
+399
View File
@@ -0,0 +1,399 @@
# Registry commands with no kernel rig case (`cargo xtask kernel-cases --coverage`). Every other
# command of `solidcraft-cli commands` (planned ones aside) must be run by some recipe.json under
# tests/kernel-cases/. One line a command: `<id> <why it has no case>`; `#` starts a comment.
# Each section is sorted, with no id twice.
#
# Commands with no geometry to grade: the view, rendering, appearance, materials, selection, the
# browser, files and saving, documents, diagnostics, and component, parameter and timeline
# bookkeeping. Each is covered by the engine's and the UI's own tests.
appearance.assign appearance only: colours and textures
appearance.library appearance only: colours and textures
appearance.list appearance only: colours and textures
assembly.bom a bill of materials report
body.rename a body name
browser.collapse_all the browser tree: order, groups and folding
browser.expand_all the browser tree: order, groups and folding
browser.group the browser tree: order, groups and folding
browser.move the browser tree: order, groups and folding
browser.order the browser tree: order, groups and folding
browser.rename_group the browser tree: order, groups and folding
browser.ungroup the browser tree: order, groups and folding
canvas.calibrate reference images, and sketches traced from them (sketch only)
canvas.decal reference images, and sketches traced from them (sketch only)
canvas.delete reference images, and sketches traced from them (sketch only)
canvas.edit reference images, and sketches traced from them (sketch only)
canvas.insert reference images, and sketches traced from them (sketch only)
canvas.to_sketch reference images, and sketches traced from them (sketch only)
component.break_link component links
component.copies a listing of identical copies
component.cycle_color component colours
component.delete removes a component: no geometry built
component.links component links
component.list a listing
component.metadata component metadata
component.metadata.set component metadata
component.properties component properties
component.relink component links
component.rename a component name
component.replace_with_link component links
component.unlock component locking
component.update_links component links
contact.list a listing of contact sets
diagnostics.crash_report crash reports
diagnostics.crash_reports crash reports
diagnostics.crash_reports.clear crash reports
diagnostics.crash_test crash reports
diagnostics.profile timing: where a command's time goes
doc.open opening and recovering documents
doc.recover opening and recovering documents
doc.recovery_discard opening and recovering documents
doc.recovery_list opening and recovering documents
document.metadata document metadata and revisions
document.metadata.remove document metadata and revisions
document.metadata.set document metadata and revisions
document.revision document metadata and revisions
edit.checkpoint undo, redo and checkpoints: an earlier state restored, no new geometry
edit.redo undo, redo and checkpoints: an earlier state restored, no new geometry
edit.restore_checkpoint undo, redo and checkpoints: an earlier state restored, no new geometry
edit.undo undo, redo and checkpoints: an earlier state restored, no new geometry
engine.commands the command registry itself
engine.describe the command registry itself
explode.create exploded views: display offsets
explode.delete exploded views: display offsets
explode.list exploded views: display offsets
explode.show exploded views: display offsets
expr.evaluate expression evaluation
file.open_in_easyprint hands the file to another program
file.save saving the design
file.save_as saving the design
file.save_mesh saving a display mesh
file.send_to_slicer hands the file to a slicer
file.slicers a listing of slicers
frame.profile.list a listing of the profile library
frame.profile.properties a profile's section properties from the library
inspect.accessibility display analysis: shading
inspect.colored_components display: component colours
inspect.curvature_map display analysis: shading
inspect.environment_map display analysis: shading
inspect.silhouette display analysis: outline
inspect.zebra display analysis: shading
joint.list a listing of joints
library.apply material and appearance libraries
library.export material and appearance libraries
library.inspect material and appearance libraries
material.assign materials and their properties
material.create materials and their properties
material.delete materials and their properties
material.edit materials and their properties
material.inspect materials and their properties
material.list materials and their properties
material.properties.directional materials and their properties
material.properties.validate materials and their properties
model.edges a listing of the model's topology
model.faces a listing of the model's topology
model.threads a listing of the model's topology
motion.delete removes a motion study
motion.export writes a study's results to a file
motion.export_animation writes an animation
motion.list a listing of motion studies
motion.play plays a study in the view
occurrence.properties occurrence properties
parameters.comment parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
parameters.complete parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
parameters.delete parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
parameters.export parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
parameters.favorite parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
parameters.graph parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
parameters.import parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
parameters.list parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
parameters.move parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
parameters.rename parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
parameters.users parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
parts.library a listing of the parts library
plastic.rules a listing of plastic rules
render.aging.describe rendering: images, cameras, stages
render.backends rendering: images, cameras, stages
render.camera.fit rendering: images, cameras, stages
render.camera.get rendering: images, cameras, stages
render.camera.set rendering: images, cameras, stages
render.cancel rendering: images, cameras, stages
render.effects.describe rendering: images, cameras, stages
render.export rendering: images, cameras, stages
render.hdr.inspect rendering: images, cameras, stages
render.library.appearance.apply rendering: images, cameras, stages
render.library.describe rendering: images, cameras, stages
render.library.environment.apply rendering: images, cameras, stages
render.library.list rendering: images, cameras, stages
render.library.physical.prepare rendering: images, cameras, stages
render.library.preview rendering: images, cameras, stages
render.library.studio.apply rendering: images, cameras, stages
render.library.studio.prepare rendering: images, cameras, stages
render.library.surface.bind rendering: images, cameras, stages
render.library.surface.prepare rendering: images, cameras, stages
render.library.surface.unbind rendering: images, cameras, stages
render.library.targets rendering: images, cameras, stages
render.optics.bind rendering: images, cameras, stages
render.optics.unbind rendering: images, cameras, stages
render.preflight rendering: images, cameras, stages
render.preset.apply rendering: images, cameras, stages
render.preset.resolve rendering: images, cameras, stages
render.quality.prepare rendering: images, cameras, stages
render.result.pin rendering: images, cameras, stages
render.result.regrade rendering: images, cameras, stages
render.scene rendering: images, cameras, stages
render.settings.apply rendering: images, cameras, stages
render.settings.capture_hdr rendering: images, cameras, stages
render.settings.get rendering: images, cameras, stages
render.settings.set rendering: images, cameras, stages
render.shot.duplicate rendering: images, cameras, stages
render.shot.remove rendering: images, cameras, stages
render.shot.rename rendering: images, cameras, stages
render.stage.backdrop rendering: images, cameras, stages
render.stage.clear rendering: images, cameras, stages
render.stage.ground rendering: images, cameras, stages
render.start rendering: images, cameras, stages
render.status rendering: images, cameras, stages
select.clear the selection
select.set the selection
selection.delete deletes what is selected through the commands that own each kind
sheet.rules a listing of sheet metal rules
sketch.arc.tangent 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.auto_constrain 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.auto_constrain_from_datum 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.break 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.break_link 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.cancel 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.centerline 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.chain 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.chamfer.distance_angle 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.chamfer.equal_distance 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.chamfer.two_distance 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.circle.three_point 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.circle.three_tangent 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.circle.two_point 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.circle.two_tangent 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.constrainer 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.constraint.coincident 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.constraint.collinear 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.constraint.concentric 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.constraint.curvature 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.constraint.equal 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.constraint.midpoint 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.constraint.parallel 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.constraint.perpendicular 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.constraint.polygon 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.constraint.symmetry 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.construction 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.copy 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.cut 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.delete 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.dimension_text 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.drag 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.drag_end 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.draw.dimensioned 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.drawing_bounds 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.edit_text 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.export_dxf 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.extend 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.fillet 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.finish_auto_constrain 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.glyphs 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.infinite 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.insert_dxf 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.insert_svg 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.inspect 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.line.midpoint 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.loose_ends 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.loose_ends.tie 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.mirror 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.move 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.move3d 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.offset 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.offset.break 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.paste 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.pattern.rectangular 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.point3d 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.polygon.edge 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.rectangle.three_point 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.redefine 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.scale 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.slot.arc_center 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.slot.arc_three_point 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.slot.center_point 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.snap 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.solve 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.toggle_driven 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.transform 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.trim 2D sketch only (the sketch solver and engine tests); no kernel operation
sketch.trim.path 2D sketch only (the sketch solver and engine tests); no kernel operation
timeline.dependents a listing of dependents
timeline.export_script writes the history as a script
timeline.inspect a listing of the timeline
timeline.rename a feature name
timeline.rename_group a group name
view.ambient_occlusion the view: camera, layout, display
view.apply the view: camera, layout, display
view.back the view: camera, layout, display
view.bottom the view: camera, layout, display
view.delete the view: camera, layout, display
view.fit the view: camera, layout, display
view.front the view: camera, layout, display
view.graphics_preset the view: camera, layout, display
view.grid the view: camera, layout, display
view.ground_shadow the view: camera, layout, display
view.history_layout the view: camera, layout, display
view.home the view: camera, layout, display
view.isometric the view: camera, layout, display
view.layout the view: camera, layout, display
view.left the view: camera, layout, display
view.list the view: camera, layout, display
view.navigate the view: camera, layout, display
view.navigation_bar the view: camera, layout, display
view.normal_to the view: camera, layout, display
view.perspective the view: camera, layout, display
view.rename the view: camera, layout, display
view.right the view: camera, layout, display
view.save the view: camera, layout, display
view.spin the view: camera, layout, display
view.tool the view: camera, layout, display
view.top the view: camera, layout, display
view.visual_style the view: camera, layout, display
# Commands that make or change geometry and have no case yet. This section only shrinks:
# the pull request that adds a case for one removes its line (the check fails on a listed command
# that a case uses), and a new geometry command gets a case, not a line here.
[to cover]
body.repair repair: the solid a refused or open part becomes
component.copy_bodies assemblies: bodies copied into a component
component.extract assemblies: bodies extracted to a design
component.from_bodies assemblies: a component made from bodies
component.insert_design assemblies: a design inserted
component.move_bodies assemblies: bodies moved between components
component.move_sketches assemblies: sketches moved between components
component.paste_new assemblies: a pasted new component
component.revert_position assemblies: a placement reverted
config.activate configurations: a configured part rebuilt per row
config.column configurations: a configured part rebuilt per row
config.configure configurations: a configured part rebuilt per row
config.preview configurations: a configured part rebuilt per row
config.row configurations: a configured part rebuilt per row
config.rule configurations: a configured part rebuilt per row
config.rules configurations: a configured part rebuilt per row
config.table configurations: a configured part rebuilt per row
construct.axis.edge construction: the plane, axis or point exactly
construct.axis.normal_to_face construction: the plane, axis or point exactly
construct.axis.through_cylinder construction: the plane, axis or point exactly
construct.axis.two_planes construction: the plane, axis or point exactly
construct.plane.at_angle construction: the plane, axis or point exactly
construct.plane.midplane construction: the plane, axis or point exactly
construct.plane.perpendicular construction: the plane, axis or point exactly
construct.plane.tangent construction: the plane, axis or point exactly
construct.plane.three_points construction: the plane, axis or point exactly
construct.plane.two_edges construction: the plane, axis or point exactly
construct.point.along_path construction: the plane, axis or point exactly
construct.point.center construction: the plane, axis or point exactly
construct.point.edge_plane construction: the plane, axis or point exactly
construct.point.three_planes construction: the plane, axis or point exactly
construct.point.two_edges construction: the plane, axis or point exactly
construct.point.vertex construction: the plane, axis or point exactly
contact.check assemblies: contact sets and what they report
contact.create assemblies: contact sets and what they report
contact.disable_all assemblies: contact sets and what they report
contact.edit assemblies: contact sets and what they report
contact.enable_all assemblies: contact sets and what they report
contact.enable_sets assemblies: contact sets and what they report
file.export_view_dxf drawings: the projection of the model a view DXF holds
frame.member frames: member bodies
frame.profile.insert frames: member bodies
inspect.center_of_mass inspect: the number or curves it reports
inspect.clearance inspect: the number or curves it reports
inspect.compare inspect: the number or curves it reports
inspect.continuity inspect: the number or curves it reports
inspect.curvature_comb inspect: the number or curves it reports
inspect.draft inspect: the number or curves it reports
inspect.interference inspect: the number or curves it reports
inspect.isocurve inspect: the number or curves it reports
inspect.minimum_radius inspect: the number or curves it reports
inspect.parameter_sweep inspect: results rebuilt over a parameter range
inspect.section inspect: the number or curves it reports
joint.as_built assemblies: joint placements and values
joint.cam assemblies: joint placements and values
joint.create assemblies: joint placements and values
joint.delete assemblies: joint placements and values
joint.drive assemblies: joint placements and values
joint.edit assemblies: joint placements and values
joint.gear assemblies: joint placements and values
joint.inspect assemblies: joint placements and values
joint.limits assemblies: joint placements and values
joint.motion_link assemblies: joint placements and values
joint.origin assemblies: joint placements and values
joint.rack_pinion assemblies: joint placements and values
joint.reach assemblies: joint placements and values
joint.rigid_group assemblies: joint placements and values
joint.screw assemblies: joint placements and values
joint.solve assemblies: joint placements and values
joint.sweep assemblies: joint placements and values
motion.interference assemblies: motion results
motion.plot assemblies: motion results
motion.study assemblies: motion results
motion.trace assemblies: motion results
motion.trace.edit assemblies: motion results
motion.trace.keep assemblies: motion results
occurrence.copy assemblies: an occurrence copied
occurrence.drag assemblies: an occurrence dragged to a placement
occurrence.reparent assemblies: an occurrence moved in the tree
parts.fastener standard parts: a fastener body
parts.insert standard parts: a part body
parts.resize standard parts: a part resized
plastic.assign_rule plastic: the feature body
plastic.boss plastic: the feature body
plastic.lip plastic: the feature body
plastic.manage_rules plastic: rules that rebuild the features using them
plastic.rest plastic: the feature body
sheet.export_bend_table sheet metal: the bend table of a flat pattern
sheet.export_dxf sheet metal: the flat pattern as DXF
sheet.hem sheet metal: the feature body
sheet.path_flange sheet metal: the feature body
sheet.perforate sheet metal: the feature body
sketch.arc3d sketch: a curve kind no case profiles yet
sketch.auto_project sketch from the model: curves the kernel computes from bodies
sketch.conic sketch: a curve kind no case profiles yet
sketch.fit_curves_to_section sketch from the model: curves the kernel computes from bodies
sketch.include_3d sketch from the model: curves the kernel computes from bodies
sketch.intersect sketch from the model: curves the kernel computes from bodies
sketch.intersection_curve sketch from the model: curves the kernel computes from bodies
sketch.isoparametric_curve sketch from the model: curves the kernel computes from bodies
sketch.project sketch from the model: curves the kernel computes from bodies
sketch.project_to_surface sketch from the model: curves the kernel computes from bodies
sketch.spline.control_point_5 sketch: a curve kind no case profiles yet
sketch.spline3d sketch: a curve kind no case profiles yet
sketch.spun_profile sketch from the model: curves the kernel computes from bodies
solid.align solid modelling: the body it makes
solid.boundary_fill solid modelling: the body it makes
solid.bounding_solid solid modelling: the body it makes
solid.cone solid modelling: the body it makes
solid.pattern.path solid modelling: the body it makes
solid.replace_face solid modelling: the body it makes
solid.scale solid modelling: the body it makes
solid.web solid modelling: the body it makes
surface.blend surfaces: the surface body it makes
surface.boundary surfaces: the surface body it makes
surface.fair surfaces: the surface body it makes
surface.match surfaces: the surface body it makes
surface.offset surfaces: the surface body it makes
surface.reverse_normal surfaces: the surface body it makes
surface.ruled surfaces: the surface body it makes
surface.unstitch surfaces: the surface body it makes
surface.untrim surfaces: the surface body it makes
timeline.compute_all timeline: the bodies after the rebuild
timeline.delete timeline: the bodies after the rebuild
timeline.edit timeline: the bodies after the rebuild
timeline.group timeline: the bodies after the rebuild
timeline.move_many timeline: the bodies after the rebuild
timeline.move_to_group timeline: the bodies after the rebuild
timeline.reorder timeline: the bodies after the rebuild
timeline.reorder_many timeline: the bodies after the rebuild
timeline.roll_to timeline: the bodies after the rebuild
timeline.suppress timeline: the bodies after the rebuild
timeline.ungroup timeline: the bodies after the rebuild
+180 -1
View File
@@ -335,6 +335,10 @@ pub fn run(rest: &[&str]) -> Result<(), String> {
}
let o = options(rest, "target/kernel-cases")?;
let dir = root().join("tests/kernel-cases");
if rest.contains(&"--coverage") {
println!("{}", run_coverage(&o, &dir)?);
return Ok(());
}
let list = cases(&dir, o.area.as_deref())?;
if list.is_empty() {
return Err(format!("no cases (recipe.json + expected.json) under {}", dir.display()));
@@ -410,9 +414,145 @@ fn gate(text: &str, cases: &[String], statuses: &[&str]) -> Result<String, Strin
}
}
/// The commands of the registry (`solidcraft-cli commands`): each id, with whether it is only
/// planned (registered, not built yet).
fn registry(o: &Options) -> Result<Vec<(String, bool)>, String> {
let mut cmd = match &o.cli {
Some(bin) => Command::new(bin),
None => {
let mut c = cargo();
c.args(["run", "-q", "--release", "-p", "solidcraft-cli", "--"]);
c
}
};
let out = cmd.current_dir(root()).arg("commands").output().map_err(|e| format!("solidcraft-cli: {e}"))?;
if !out.status.success() {
return Err(format!("solidcraft-cli commands failed:\n{}", String::from_utf8_lossy(&out.stderr)));
}
let list: Value = serde_json::from_slice(&out.stdout).map_err(|e| format!("commands JSON: {e}"))?;
let list = list.as_array().ok_or("commands JSON: not a list")?;
Ok(list.iter().filter_map(|c| Some((c["id"].as_str()?.to_string(), c["planned"].as_str().is_some_and(|p| !p.is_empty())))).collect())
}
/// Every command id the command scripts (`recipe.json`) under `dir` run, at any depth.
fn used_commands(dir: &Path) -> Result<Vec<String>, String> {
let mut out = Vec::new();
let mut todo = vec![dir.to_path_buf()];
while let Some(d) = todo.pop() {
for e in std::fs::read_dir(&d).map_err(|e| format!("{}: {e}", d.display()))?.flatten() {
let p = e.path();
if p.is_dir() {
todo.push(p);
} else if p.file_name().is_some_and(|n| n == "recipe.json") {
let text = std::fs::read_to_string(&p).map_err(|e| format!("{}: {e}", p.display()))?;
out.extend(recipe_commands(&text).map_err(|e| format!("{}: {e}", p.display()))?);
}
}
}
out.sort();
out.dedup();
Ok(out)
}
/// The command ids a recipe runs: each `commands[].command` of a command script. An oracle
/// recipe (no `commands`) names features, not commands, and counts none.
fn recipe_commands(text: &str) -> Result<Vec<String>, String> {
let v: Value = serde_json::from_str(text).map_err(|e| e.to_string())?;
let Some(list) = v.get("commands") else { return Ok(Vec::new()) };
let list = list.as_array().ok_or("`commands` is not a list")?;
list.iter()
.enumerate()
.map(|(i, c)| c["command"].as_str().map(str::to_string).ok_or_else(|| format!("commands[{i}] has no `command` id")))
.collect()
}
/// One line of `tests/kernel-cases/EXEMPT`: a command with no case (its reason is for readers).
#[derive(Debug, PartialEq)]
struct Exempt<'a> {
id: &'a str,
/// Listed under `[to cover]`: a geometry command still waiting for its case.
to_cover: bool,
}
/// The section header of the commands that make geometry but have no case yet.
const TO_COVER: &str = "[to cover]";
/// The commands `tests/kernel-cases/EXEMPT` exempts from the coverage check: one per line,
/// `<id> <why it has no case>`; `#` starts a comment line. The lines before `[to cover]` have no
/// geometry to grade; those after it make geometry and wait for a case, so that section only
/// shrinks. Each section is sorted, with no id twice in the file, and every line gives a reason.
fn exempt(text: &str) -> Result<Vec<Exempt<'_>>, String> {
let mut out: Vec<Exempt> = Vec::new();
let mut to_cover = false;
let mut last: Option<&str> = None;
for (n, line) in text.lines().enumerate() {
let line = line.trim();
if line.is_empty() || line.starts_with('#') {
continue;
}
if line == TO_COVER {
if to_cover {
return Err(format!("line {}: a second {TO_COVER} section", n + 1));
}
to_cover = true;
last = None;
continue;
}
let (id, why) = line.split_once(char::is_whitespace).map_or((line, ""), |(c, w)| (c, w.trim()));
if why.is_empty() {
return Err(format!("line {}: {id} gives no reason (`<id> <why it has no case>`)", n + 1));
}
if out.iter().any(|e| e.id == id) {
return Err(format!("line {}: {id} is listed twice", n + 1));
}
if let Some(prev) = last
&& prev > id
{
return Err(format!("line {}: {id} is out of order after {prev}: keep each section sorted", n + 1));
}
last = Some(id);
out.push(Exempt { id, to_cover });
}
Ok(out)
}
/// The coverage check: every built command of `registry` (id, planned) is in `used` (run by a
/// case) or exempted by `exempt_text`; an exemption names a registered command that no case
/// uses, so a case that comes removes its line.
fn coverage(registry: &[(String, bool)], used: &[String], exempt_text: &str) -> Result<String, String> {
let listed = exempt(exempt_text)?;
let mut problems = Vec::new();
for e in &listed {
if !registry.iter().any(|(id, _)| id == e.id) {
problems.push(format!("EXEMPT lists {}, which is not a registry command: remove its line", e.id));
} else if used.iter().any(|u| u == e.id) {
problems.push(format!("EXEMPT lists {}, which a case now uses: remove its line", e.id));
}
}
let mut covered = 0;
for (id, _) in registry.iter().filter(|(_, planned)| !planned) {
if used.iter().any(|u| u == id) {
covered += 1;
} else if !listed.iter().any(|e| e.id == id) {
problems.push(format!("NO CASE: {id} (add a case under tests/kernel-cases/, or a line to EXEMPT)"));
}
}
if !problems.is_empty() {
return Err(format!("kernel-cases coverage:\n {}", problems.join("\n ")));
}
let to_cover = listed.iter().filter(|e| e.to_cover).count();
Ok(format!("kernel-cases coverage: {covered} commands have a case, {} have no geometry, {to_cover} still to cover", listed.len() - to_cover))
}
fn run_coverage(o: &Options, dir: &Path) -> Result<String, String> {
let file = dir.join("EXEMPT");
let text = std::fs::read_to_string(&file).map_err(|e| format!("{}: {e}", file.display()))?;
coverage(&registry(o)?, &used_commands(dir)?, &text).map_err(|e| format!("{}: {e}", file.display()))
}
#[cfg(test)]
mod tests {
use super::gate;
use super::{Exempt, coverage, exempt, gate, recipe_commands};
fn run(text: &str, cases: &[(&str, &str)]) -> Result<String, String> {
let names: Vec<String> = cases.iter().map(|(c, _)| c.to_string()).collect();
@@ -438,4 +578,43 @@ mod tests {
assert!(run("a x\na y\n", &[("a", "fail")]).unwrap_err().contains("a repeat of"));
assert!(run("a\n", &[("a", "fail")]).unwrap_err().contains("gives no cause"));
}
#[test]
fn a_recipe_names_the_commands_of_its_script() {
let script = r#"{"base": "../a", "commands": [{"command": "solid.box", "params": {}}, {"command": "solid.fillet"}]}"#;
assert_eq!(recipe_commands(script).unwrap(), ["solid.box", "solid.fillet"]);
assert!(recipe_commands(r#"{"id": "01-box", "features": []}"#).unwrap().is_empty());
assert!(recipe_commands(r#"{"commands": [{"params": {}}]}"#).unwrap_err().contains("no `command` id"));
assert!(recipe_commands(r#"{"commands": {}}"#).unwrap_err().contains("not a list"));
}
#[test]
fn the_exempt_list_has_two_sorted_sections_with_a_reason_each() {
let text = "# header\nview.fit camera only\nview.top camera only\n\n[to cover]\nsolid.cone surfaces area\n";
assert_eq!(
exempt(text).unwrap(),
[Exempt { id: "view.fit", to_cover: false }, Exempt { id: "view.top", to_cover: false }, Exempt { id: "solid.cone", to_cover: true },]
);
assert!(exempt("view.top x\nview.fit x\n").unwrap_err().contains("out of order"));
assert!(exempt("view.fit x\n[to cover]\nview.fit y\n").unwrap_err().contains("listed twice"));
assert!(exempt("view.fit\n").unwrap_err().contains("gives no reason"));
assert!(exempt("[to cover]\n[to cover]\n").unwrap_err().contains("second"));
}
#[test]
fn coverage_fails_on_a_command_with_no_case_and_on_a_stale_exemption() {
let registry: Vec<(String, bool)> =
[("solid.box", false), ("solid.cone", false), ("view.fit", false), ("surface.fillet", true)].map(|(i, p)| (i.to_string(), p)).to_vec();
let used = ["solid.box".to_string()];
let list = "view.fit camera only\n[to cover]\nsolid.cone surfaces area\n";
let ok = coverage(&registry, &used, list).unwrap();
assert!(ok.contains("1 commands have a case, 1 have no geometry, 1 still to cover"), "{ok}");
let missing = coverage(&registry, &used, "view.fit camera only\n").unwrap_err();
assert!(missing.contains("NO CASE: solid.cone"), "{missing}");
assert!(!missing.contains("surface.fillet"), "a planned command needs no case: {missing}");
let covered = coverage(&registry, &["solid.box".into(), "solid.cone".into()], list).unwrap_err();
assert!(covered.contains("EXEMPT lists solid.cone, which a case now uses"), "{covered}");
let gone = coverage(&registry, &used, "view.fit camera only\nview.gone removed\n[to cover]\nsolid.cone x\n").unwrap_err();
assert!(gone.contains("view.gone, which is not a registry command"), "{gone}");
}
}
+4 -1
View File
@@ -47,7 +47,7 @@ commands:
replay the reference tests (tests/oracle/*/recipe.json), compare with each
case's expected.json and write docs/oracle.md (--cli: a solidcraft-cli
already built, instead of cargo run --release)
kernel-cases [--area AREA] [--cli BIN] [--timeout S] [--jobs N] [--out DIR] [--known-failing FILE]
kernel-cases [--area AREA] [--cli BIN] [--timeout S] [--jobs N] [--out DIR] [--known-failing FILE] [--coverage]
the kernel regression rig: run every case in tests/kernel-cases/<area>/<id>/ on
SMLib (one process each, killed after --timeout, default 180 s), grade it against
the case's own answer and write docs/kernel-cases.md; DIR (default
@@ -62,6 +62,9 @@ commands:
docs/kernel-ladder.md (DIR default target/kernel-ladder); --record re-measures
the golden and re-sorts each tier by time; --gate fails on any case whose status
or counts differ from the golden, instead of the report
--coverage: instead of running cases, fail on a registry command (the CLI's
`commands`, planned ones aside) that no recipe.json under tests/kernel-cases/
uses and tests/kernel-cases/EXEMPT does not list, and on a stale EXEMPT line
action-matrix [--bless] [--results FILE] [--from DIR]
try every ordinary action on every face, edge, vertex and body of the design
corpus and compare with tests/action_matrix/baseline.json (--bless rewrites it;