fix(review): salvage the prose when the findings JSON is unparseable
Found by running the reviewer against this branch. The second review of PR #7 ran a full agent loop — 330s of a 540s budget, no timeout — and ended without a parseable ```json block. The code discarded the entire run and posted "AI review produced no parseable output.", losing minutes of work and millions of tokens for a message that tells the maintainer nothing and gives me nothing to debug. Three changes on that path: - salvage_summary() keeps the agent's prose (fenced blocks stripped, tail kept because the conclusion is written last) and posts it under an explicit banner saying it is unstructured and its line numbers were never validated against the diff. A partial review honestly labelled beats no review. - The raw output's length and last 600 chars go to stderr, so the next occurrence is diagnosable from pod logs instead of invisible. - The AI-USAGE section is still rendered. The label asked for it and the tokens were spent either way; dropping the measurement on the failure path is how the cost model stops getting calibration data exactly when it is most interesting. Not fixed here: why the agent went off-format. The likely cause is the 40-step cap in the agent definition being reached on a larger diff (the successful run used 28), which wants either a higher cap or a step-budget warning in the prompt. Needs the next occurrence's stderr to confirm rather than guess. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
This commit is contained in:
+48
-2
@@ -420,6 +420,39 @@ def parse_findings(text: str) -> list[dict]:
|
||||
return out
|
||||
|
||||
|
||||
SALVAGE_MAX_CHARS = 4000
|
||||
|
||||
|
||||
def salvage_summary(text: str, max_chars: int = SALVAGE_MAX_CHARS) -> str:
|
||||
"""Recover something postable from agent output we could not parse.
|
||||
|
||||
An opencode run costs minutes and millions of tokens. When the findings JSON
|
||||
is missing or malformed, the analysis itself is usually still there in the
|
||||
prose — discarding it to post "no parseable output" throws away the whole
|
||||
run and tells the maintainer nothing. This keeps the tail of the prose (the
|
||||
conclusion, which is what the agent writes last), drops fenced code blocks
|
||||
so a half-written JSON blob doesn't dominate, and labels it plainly as
|
||||
unstructured so nobody mistakes it for a normal review.
|
||||
|
||||
Returns "" when there is genuinely nothing to salvage.
|
||||
"""
|
||||
if not text or not text.strip():
|
||||
return ""
|
||||
# Drop fenced blocks — a truncated ```json block is noise here.
|
||||
prose = re.sub(r"```.*?```", "", text, flags=re.DOTALL)
|
||||
prose = re.sub(r"```.*$", "", prose, flags=re.DOTALL) # unterminated fence
|
||||
prose = prose.strip()
|
||||
if not prose:
|
||||
return ""
|
||||
if len(prose) > max_chars:
|
||||
prose = "…" + prose[-max_chars:]
|
||||
return (
|
||||
"⚠️ _The reviewer did not emit a parseable findings block, so there are "
|
||||
"no inline comments. Its raw notes are below — treat them as unverified: "
|
||||
"line numbers were not validated against the diff._\n\n" + prose
|
||||
)
|
||||
|
||||
|
||||
def parse_review_output(text: str) -> tuple[str, list[dict]]:
|
||||
"""Parse the opengine's stdout into (summary, findings).
|
||||
|
||||
@@ -904,9 +937,22 @@ def review_pr(
|
||||
)
|
||||
review_summary, findings = parse_review_output(stdout)
|
||||
if not findings and not review_summary:
|
||||
# opencode produced nothing parseable — fall back to a note.
|
||||
# The findings JSON was missing or malformed. Don't discard the
|
||||
# run: salvage the prose, keep the usage report (the label asked
|
||||
# for it, and the tokens were spent either way), and log enough
|
||||
# of the raw output to diagnose why the agent went off-format.
|
||||
print(
|
||||
f"pragent: {repo}#{index} sha={sha[:8]} unparseable output "
|
||||
f"({len(stdout)} chars); tail: {stdout[-600:]!r}",
|
||||
file=sys.stderr, flush=True,
|
||||
)
|
||||
salvaged = salvage_summary(stdout)
|
||||
usage_section = ""
|
||||
if report_usage and usage:
|
||||
usage_section = format_usage_section(usage, [], model)
|
||||
post_review(api, repo, index, token, format_review_body(
|
||||
"AI review produced no parseable output.", model, sha))
|
||||
salvaged or "AI review produced no parseable output.",
|
||||
model, sha, usage_section=usage_section))
|
||||
return True
|
||||
else:
|
||||
user_prompt = build_user_prompt(title, body, diff, config, prior)
|
||||
|
||||
Reference in New Issue
Block a user