gitea_admin/pragent
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(review): PR-level collapsible metadata + emoji-tagged inline comments

Browse Source 
PR-level comment layout (per operator's format guide):
  * Summary of Changes — 2-4 bullets, sourced from the agent's new
    `summary_changes` JSON field. Falls back to splitting the prose
    `summary` if the list is missing.
  * Key Risks & Concerns — bullets from the new `risks` JSON field.
  * Findings Overview — Markdown table covering every finding
    (severity emoji / location / one-line problem). Both anchored and
    unanchored findings appear here so the table is the single scan point.
  * Unanchored Notes — bullets with severity + fix + Markdown-linked ref,
    for findings with no post-change line to anchor.
  * AI Usage & Run Details — wrapped in a <details>/<summary> collapsible
    so the body stays scannable. Cost line stays inside it.

Inline comment shape:
  * Severity badge: 🔴 [HIGH] / 🟡 [MEDIUM] / 🔵 [LOW] / ⚪ [INFO].
    Unknown severities fall back to � [INFO].
  * 1-2 short paragraphs of problem; **Fix:** label for the fix line.
  * Standard ```suggestion fence for replacement code (Gitea/Forgejo
    apply-on-click). Language-tagged fences are no longer used for
    single-file diffs.
  * Reference as a Markdown hyperlink, visible label truncated to
    <=60 chars; the underlying URL is preserved verbatim.
  * NO per-comment 🪙 token attribution. All telemetry stays in the
    collapsible block on the PR-level comment.

Agent prompt updated to emit `summary_changes` and `risks` in the JSON
output (backward-compatible — older outputs missing them still parse;
they fall back to splitting the prose `summary`).

Tests: 15 new (severity emoji mapping, reference truncation, findings
table escaping, collapsible usage rendering, summary_changes+risks
layout). Existing tests updated for the new structure.
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Marcos
2026-08-20 16:51:20 +00:00
parent 5302e8dcd7
commit f6be2b3c61
3 changed files with 474 additions and 110 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/pilot/test_ai_review.py
+160 -30
View File
@@ -14,8 +14,10 @@ from ai_review import ( # noqa: E402
_balanced_json_substring,
_extract_first_json_object,
_last_balanced_json,
_render_collapsible_usage,
build_user_prompt,
compute_attribution,
findings_table,
format_review_body,
format_usage_section,
inline_comment_body,
@@ -108,17 +110,25 @@ def test_format_review_body_findings():
assert "pragent pilot" in body
assert "glm-5.2:cloud" in body
assert "`abcdef12`" in body # 8-char sha
assert "- [high] x:1" in body
# New layout: always emits Summary of Changes + Key Risks. Findings table
# only shows when findings_for_table is passed (callers pass the actual
# list of finding dicts; plain-string findings arg renders as bullets).
assert "### Summary of Changes" in body
assert "### Key Risks & Concerns" in body
def test_format_review_body_empty_findings():
body = format_review_body("", "glm-5.2:cloud", "abcdef1234567890")
assert "No issues found." in body
# No summary → "no summary provided" sentinel; Findings table absent
# because no findings were passed.
assert "_No summary provided._" in body
assert "_None identified._" in body
assert "### Findings Overview" not in body
def test_format_review_body_whitespace_findings():
body = format_review_body(" \n ", "glm-5.2:cloud", "abcdef1234567890")
assert "No issues found." in body
assert "_No summary provided._" in body
def test_format_review_body_no_sha():
@@ -274,33 +284,98 @@ def test_split_findings_by_anchor():
def test_inline_comment_body_with_suggestion():
f = {"severity": "high", "path": "a", "line": 1, "problem": "bad", "fix": "swap", "suggestion": "good()"}
body = inline_comment_body(f)
assert "**[HIGH]**" in body
# Severity emoji + bracketed label.
assert "🔴 [HIGH]" in body
assert "bad" in body
# no extension → bare fence (Gitea 1.26.x has no apply-suggestion; we tag
# with the file language for highlighting instead of ```suggestion)
assert "```\ngood()\n```" in body
# Standard ```suggestion fence (Gitea/Forgejo apply-on-click).
assert "```suggestion\ngood()\n```" in body
assert "good()" in body
def test_inline_comment_body_suggestion_lang_tagged():
def test_inline_comment_body_suggestion_not_lang_tagged():
# Per the format spec, the suggestion fence is ALWAYS ```suggestion —
# never a language-tagged fence (those are reserved for cross-file
# pattern illustrations, which we don't emit here).
f = {"severity": "high", "path": "src/Foo.java", "line": 1,
"problem": "bad", "fix": "swap", "suggestion": "good();"}
body = inline_comment_body(f)
assert "```java\ngood();\n```" in body
assert "```suggestion\ngood();\n```" in body
assert "```java" not in body
def test_inline_comment_body_no_suggestion():
f = {"severity": "low", "path": "a", "line": 1, "problem": "p", "fix": "f", "suggestion": ""}
body = inline_comment_body(f)
assert "```" not in body
assert "Fix: f" in body
assert "**Fix:** f" in body
def test_inline_comment_body_severity_emoji_mapping():
cases = [
("critical", "🔴 [CRITICAL]"),
("high", "🔴 [HIGH]"),
("medium", "🟡 [MEDIUM]"),
("low", "🔵 [LOW]"),
("info", "⚪ [INFO]"),
("nit", "⚪ [INFO]"), # "nit" maps to the INFO label
("bogus", "⚪ [INFO]"), # unknown severity falls back to INFO
]
for sev, badge in cases:
f = {"severity": sev, "path": "a", "line": 1, "problem": "p", "fix": "",
"suggestion": "", "reference": ""}
assert badge in inline_comment_body(f), f"{sev} → {badge}"
def test_inline_comment_body_no_token_attribution():
# Per spec: no per-comment 🪙 token attribution line.
f = {"severity": "high", "path": "a", "line": 1, "problem": "p",
"fix": "f", "suggestion": "", "reference": "",
"_tok_attrib": 1234, "_tok_pct": 0.3}
body = inline_comment_body(f)
assert "🪙" not in body
assert "tok" not in body.lower().split("fix")[0] # only in fix is OK
assert "attributed" not in body
def test_summary_bullets_format():
fs = [{"severity": "high", "path": "a.py", "line": 7, "problem": "p", "fix": "f", "suggestion": ""}]
b = summary_bullets(fs)
assert "- **[HIGH]**" in b
assert "🔴 [HIGH]" in b
assert "`a.py:7`" in b
assert "**Fix:** f" in b
def test_summary_bullets_with_reference_link():
fs = [{"severity": "medium", "path": "x", "line": 1, "problem": "p",
"fix": "", "suggestion": "", "reference": "https://owasp.org/x"}]
b = summary_bullets(fs)
assert "🔗 **Reference:** [owasp.org/x](https://owasp.org/x)" in b
assert "https://owasp.org/x" in b # URL preserved
def test_findings_table_renders_table():
fs = [
{"severity": "high", "path": "a.py", "line": 1, "problem": "bug", "fix": "", "suggestion": "", "reference": ""},
{"severity": "low", "path": "b.go", "line": 9, "problem": "nit", "fix": "", "suggestion": "", "reference": ""},
]
t = findings_table(fs)
assert t.startswith("| Severity | Location | Finding |")
assert "|---|---|---|" in t
assert "🔴 [HIGH]" in t
assert "🔵 [LOW]" in t
assert "`a.py:1`" in t
assert "`b.go:9`" in t
def test_findings_table_escapes_pipes():
fs = [{"severity": "high", "path": "a", "line": 1,
"problem": "uses | inside", "fix": "", "suggestion": "", "reference": ""}]
t = findings_table(fs)
assert "uses \\| inside" in t
def test_findings_table_empty():
assert findings_table([]) == ""
# ---------------------------------------------------------------------------
@@ -376,7 +451,7 @@ def test_parse_review_output_summary_and_findings():
"]}",
"\n```",
)
summary, fs = parse_review_output("".join(txt))
summary, fs, *_ = parse_review_output("".join(txt))
assert "eval()" in summary
assert len(fs) == 1
assert fs[0]["severity"] == "critical"
@@ -386,16 +461,16 @@ def test_parse_review_output_summary_and_findings():
def test_parse_review_output_bare_findings_no_summary():
txt = '```json\n{"findings":[{"severity":"low","path":"a","line":1,"problem":"p"}]}\n```'
summary, fs = parse_review_output(txt)
summary, fs, *_ = parse_review_output(txt)
assert summary == ""
assert len(fs) == 1
assert fs[0]["reference"] == "" # default
def test_parse_review_output_empty_and_bogus():
assert parse_review_output("") == ("", [])
assert parse_review_output("no json here") == ("", [])
assert parse_review_output('{"findings":[]}') == ("", [])
assert parse_review_output("") == ("", [], [], [])
assert parse_review_output("no json here") == ("", [], [], [])
assert parse_review_output('{"findings":[]}') == ("", [], [], [])
def test_parse_review_output_uses_last_json_block():
@@ -405,7 +480,7 @@ def test_parse_review_output_uses_last_json_block():
"more prose\n"
"```json\n{\"summary\":\"real\",\"findings\":[{\"path\":\"y\",\"line\":2,\"severity\":\"high\"}]}\n```"
)
summary, fs = parse_review_output(txt)
summary, fs, *_ = parse_review_output(txt)
assert summary == "real"
assert len(fs) == 1
assert fs[0]["path"] == "y"
@@ -463,7 +538,7 @@ def test_parse_review_output_unfenced_at_tail():
'{"severity":"high","path":"VoidProtection.java","line":162,'
'"problem":"drop duplication race","fix":"use ItemMeta","suggestion":"","reference":""}]}'
)
summary, fs = parse_review_output(txt)
summary, fs, *_ = parse_review_output(txt)
assert "void protection" in summary.lower()
assert len(fs) == 1
assert fs[0]["path"] == "VoidProtection.java"
@@ -474,7 +549,7 @@ def test_parse_review_output_bare_array_at_tail():
"All wrapped up.\n"
'[{"severity":"low","path":"a","line":1,"problem":"p","fix":"","suggestion":"","reference":""}]'
)
summary, fs = parse_review_output(txt)
summary, fs, *_ = parse_review_output(txt)
assert summary == ""
assert len(fs) == 1
@@ -522,13 +597,26 @@ def test_inline_comment_body_renders_reference():
f = {"severity": "high", "path": "a", "line": 1, "problem": "p", "fix": "f",
"suggestion": "", "reference": "https://cve.example/X"}
body = inline_comment_body(f)
assert "📎 ref: https://cve.example/X" in body
# Per spec: Markdown hyperlink, not raw URL.
assert "🔗 **Reference:** [cve.example/X](https://cve.example/X)" in body
def test_inline_comment_body_no_reference_no_ref_line():
f = {"severity": "low", "path": "a", "line": 1, "problem": "p", "fix": "",
"suggestion": "", "reference": ""}
assert "📎 ref" not in inline_comment_body(f)
assert "🔗" not in inline_comment_body(f)
assert "Reference:" not in inline_comment_body(f)
def test_inline_comment_body_reference_truncates_long_url():
f = {"severity": "high", "path": "a", "line": 1, "problem": "p", "fix": "",
"suggestion": "",
"reference": "https://very-long-domain.example.com/some/very/long/path/that/exceeds/the/sixty/char/limit/x"}
body = inline_comment_body(f)
# Visible label is truncated to ≤60 chars (ellipsis added).
assert "…" in body
# But the underlying URL is preserved verbatim inside the link target.
assert "very-long-domain.example.com" in body
def test_summary_bullets_renders_reference():
@@ -585,13 +673,15 @@ def test_compute_attribution_noop_on_empty_or_zero_budget():
assert "_tok_attrib" not in fs[0]
def test_inline_comment_body_with_attribution_line():
def test_inline_comment_body_no_attribution_line():
# Per spec: NO per-comment token attribution — that telemetry lives in the
# collapsible block on the PR-level comment.
f = {"severity": "high", "path": "a", "line": 1, "problem": "bad", "fix": "swap",
"suggestion": "", "_tok_attrib": 180, "_tok_pct": 0.29}
body = inline_comment_body(f)
assert "🪙 ~180 tok" in body
assert "29%" in body
assert "attributed output" in body
assert "🪙" not in body
assert "tok" not in body
assert "attributed" not in body
def test_inline_comment_body_no_attribution_no_coin_line():
@@ -643,14 +733,14 @@ def test_format_usage_section_cost_nonzero():
assert "billed by provider" in sec
def test_format_review_body_usage_section_between_summary_and_findings():
def test_format_review_body_usage_section_below_findings():
# New layout: header → Summary of Changes → Key Risks → findings → usage.
usage_sec = "## 🔋 AI usage\n\n- model: `m`"
body = format_review_body("- [high] x:1 — b", "glm-5.2:cloud", "abcdef1234567890",
summary="This PR is risky.", usage_section=usage_sec)
# order: header < summary < usage < findings < marker
assert body.index("risky.") < body.index("AI usage")
assert body.index("AI usage") < body.index("[high]")
assert body.index("[high]") < body.index("<!-- pragent:sha=")
assert body.index("risky.") < body.index("[high]")
assert body.index("[high]") < body.index("AI usage")
assert body.index("AI usage") < body.index("<!-- pragent:sha=")
assert "## 🔋 AI usage" in body
@@ -658,6 +748,46 @@ def test_format_review_body_no_usage_section_omitted():
body = format_review_body("- [high] x:1 — b", "glm-5.2:cloud", "abcdef1234567890")
assert "AI usage" not in body
def test_format_review_body_with_summary_changes_and_risks():
body = format_review_body(
"", "glm-5.2:cloud", "abcdef1234567890",
summary_changes=["Adds void-death item rescue.", "Adds chunk-loader pause/rename."],
risks=["Mob spawn leak in force-loaded chunks.", "Race on /chunkloader tempo -1."],
findings_for_table=[
{"severity": "high", "path": "a.py", "line": 1, "problem": "race", "fix": "",
"suggestion": "", "reference": ""},
],
inline_count=1,
)
assert "### Summary of Changes" in body
assert "Adds void-death item rescue." in body
assert "Adds chunk-loader pause/rename." in body
assert "### Key Risks & Concerns" in body
assert "Mob spawn leak in force-loaded chunks." in body
assert "### Findings Overview" in body
assert "1 inline comment(s); 1 total." in body
assert "🔴 [HIGH]" in body
assert "`a.py:1`" in body
def test_render_collapsible_usage_contains_details():
usage = {
"model": "glm-5.2:cloud", "input": 1000, "output": 200, "reasoning": 0,
"cache_read": 0, "cache_write": 0, "total": 1200, "steps": 5, "duration_s": 12.0,
"cost": 0.0,
}
block = _render_collapsible_usage(usage, "glm-5.2:cloud", config=None)
assert "<details>" in block
assert "<summary>🔋 AI Usage & Run Details</summary>" in block
assert "</details>" in block
assert "glm-5.2:cloud" in block
assert "1000 in / 200 out" in block
def test_render_collapsible_usage_empty_when_no_usage():
assert _render_collapsible_usage(None, "glm-5.2:cloud", config=None) == ""
# ---------------------------------------------------------------------------
# parse_diff_anchors — empty context lines
# ---------------------------------------------------------------------------