Files
solvecraft/apps/solidcraft-cli
raykholo dad28abecf solidcraft-cli bench-assistant: the standard Assistant model test (#4851) (#4930)
* claim: #4851

* solidcraft-cli bench-assistant: the standard Assistant model test

Runs the eleven v1 tasks of #4851 against a desktop app that is already
running with a control port. Each task starts a new design and a new chat
with All permissions, runs only the prescribed setup, sends the prompt
once and waits for the turn. The result is checked on the design
(document.inspect with measurements): body count, volume within 1%
(0.3% for the fillet), bounding-box size in any orientation, the lowest
corner, the body name, live feature kinds, and changed or unchanged
content; the volume question is checked on the reply. Failures get the
issue's categories from the conversation's items. Prints a table, and
writes the issue's comment template (--markdown) and the raw results
with each conversation (--json).
2026-10-23 13:44:02 +00:00
..