feat(input): add diff_compress module + prior-review compaction helpers #9
Merged
gitea_admin
merged 8 commits from 2026-08-20 23:05:32 +00:00
feat/cost-display-compress-config into main
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Marcos
|
2b1cf750b7 |
fix(review): correct diff-compression line numbers, prior-review dedupe, triage skip
Four defects, all found reviewing PR #9 (two of them by pragent-bot's own review of that PR, which the anchoring bug then misplaced): * compress_diff dropped context lines but copied the original `@@` hunk header verbatim, so the header no longer described the lines beneath it. parse_diff_anchors then walked stale headers and produced anchor sets shifted by the number of elided lines, misplacing inline comments or demoting them to bullets. Each surviving run of lines is now re-emitted as its own hunk with a recomputed `@@ -a,b +c,d @@`, so the output stays a valid unified diff whose numbers describe the real post-change file. The pseudo-marker `@@ … N context line(s) omitted … @@` is gone; it parsed as a hunk header and reset the anchor counter to 0. Anchoring additionally runs on the raw diff now, so the prompt window can never shrink the anchorable set. * compress_diff's `_FILE_HEADER` regex matched diff *body* lines: a removed YAML `---` separator or an added `++` line was read as a file header, truncating the hunk and dropping its `@@` header with it. Body detection is now prefix-based, with a full-shape hunk-header regex. * extract_finding_bullets could not match the bullets pragent itself posts: summary_bullets renders an emoji severity badge between the `-` and the `[SEV]` tag, which the regex rejected, so compact_prior_reviews always returned [] and every re-review repeated its previous findings. * triage returning `{"lenses":[]}` — documented in .opencode/agents/triage.md as "no lens has surface, skip the fan-out" — ran every lens instead, since _intersect_with_triage mapped an empty selection to "all" and the call site had a second `or reviewers` fallback. `[]` and None are now distinct outcomes: `[]` skips, None fails open. A roster naming only unknown lens ids now fails open rather than silencing the review. The skip path returns a well-formed empty-findings response instead of "", which had landed in ai_review's unparseable-output branch and posted "AI review produced no parseable output" — a malfunction message for a normal verdict. Also: non-URL references (a CVE id, a doc title) rendered as `[CVE-2024-1234](CVE-2024-1234)`, a broken relative link in Gitea — now plain text. PRAGENT_DIFF_CONTEXT and friends parse through _int_env, so a typo logs and falls back instead of killing a review mid-flight. Removed format_usage_section, dead since the collapsible usage block replaced it and carrying a duplicate copy of the price-target logic. Tests: 290 -> 301. New coverage for hunk-header fidelity before/after compression, header-shaped content lines, the bullet round-trip against the real renderer, and triage's three outcomes (previously untested). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN |
||
|
pragent-bot
|
78bcf6a9a0 |
feat(review): multi-lens orchestration — 5 parallel opencode subprocesses (security/docs/code-quality/tests/perf)
On by default, opt-out via "reviewers": []. 290 tests pass.
- pilot/opencode_review.py: ReviewerSpec dataclass, default_reviewers(),
parse_reviewers_config(), parse_triage_config(), resolve_reviewers(),
_normalize_lens_finding(), posthash() (matches feedback.py scheme),
_agreement_hash() (severity-free for cross-lens promotion), _tone_strip(),
synthesize() 7-stage (severity_floor → tone-strip → length cap → per-lens
max → per-file cap → dedup by _posthash → cross-lens severity promote →
per-PR cap), run_lenses() (ThreadPoolExecutor pool=4), triage(),
_intersect_with_triage(), _filter_by_skip_if(), run_lenses_review().
run() routes to fan-out when config.reviewers[] present or PRAGENT_REVIEWERS=1.
- pilot/ai_review.py: parse_repo_config learns reviewers[] and triage objects
(id regex /^[a-z0-9][a-z0-9-]{0,31}$/, 8-entry cap, agent_file/model/
severity_floor/max_findings/activation/skip_if_all_changed_paths/hotpath_globs).
review_pr branches to opencode_review.run_lenses_review when configured.
_render_collapsible_usage shows lenses: ... line when present.
- .opencode/agents/{docs,code-quality,triage}.md: 3 new lens subagents.
- .opencode/skills/lens-orchestration/SKILL.md: strict-JSON contract every
lens subagent MUST honor.
- .opencode/agents/pragent.md: slim to coordinator; no more hardcoded
@security/@tests/@perf delegation; loads lens-orchestration skill.
- .opencode/README.md: rewrite 'Add a review lens' recipe for multi-lens.
- pilot/README-webhook.md: new 'Multi-lens pipeline' section (diagram +
default roster + config schema + env vars + cross-lens dedup contract).
- tests: 36 new tests (test_ai_review.py +12 reviewers/triage/usage,
test_opencode_review.py +24 orchestration). posthash golden-vector matches
feedback.py exactly across 5 severity × 2 line cases.
|
||
|
Marcos
|
e8ebc54362 |
feat(review): ADDITIONAL_CONTEXT_URL — repo-provided static context cached per review
Long agent loops re-send the brief prefix on every step; cheap reusable knowledge (architecture summary, module map, conventions, glossary) belongs in a versioned file the maintainers control so the agent doesn't re-derive it from the source tree on every PR. Two wiring paths, merged (env first): * env var PRAGENT_ADDITIONAL_CONTEXT_URL — comma-separated, deployment-wide * .pr-review.json:additional_context_urls — list[str], read from the PR's base branch (same trust boundary as the rest of the file) Implementation: * _parse_additional_context_env splits/dedupes/trims. * _resolve_additional_context_urls(config) merges env (first) + config (then, skipping env-dupes); caps at 8. * fetch_additional_context(urls) fetches each URL with urllib (5s timeout, http/https only — file://, javascript:, ftp:// rejected defensively), caches by URL in a module-level dict for the pod lifetime, truncates per-URL to 4k chars + total to 16k chars, best-effort (network errors are logged and skipped — never aborts the review). * Result injected into build_user_prompt under "## Repo-provided context" between repo config and prior reviews. In the opencode engine it lands in .pragent/brief.md under its own section. The brief explicitly labels each block's CONTENT as untrusted (same as PR description) — section heading is trustworthy, body isn't. * parse_repo_config accepts the field, caps at 8 entries, drops non-strings and empty strings. Docs: pilot/README-webhook.md "Repo-provided static context" section — env var + JSON example + Nexus raw-hosted recipe. Tests: 14 new (208 total), covering env merging + dedup, scheme rejection, per-URL cap, total cap, caching by URL, brief injection. All mock urllib with a context-manager stand-in (no real network). Co-Authored-By: Claude <noreply@anthropic.com> |
||
|
Marcos
|
998f793ec2 |
feat(agent): tighten prompt to bound beyond-diff reads + de-generalize cost-model labels
Three changes from operator feedback:
1. Per-comment � attribution restored on inline comments (operator wants
it back — the PR-level collapsible is collapsed by default, so the
attribution is the visible signal of per-finding cost share).
Hidden only when no _tok_attrib was computed (legacy callers / ollama
path without usage metering).
2. Agent prompt now bounds reads beyond the diff — the single biggest
driver of input-token bloat on long agent loops:
* ≤ 5 file reads beyond the diff for the entire review
* ≤ 80 lines per read (use --offset + --limit)
* ≤ 3 grep calls beyond the diff (prefer rtk grep)
* no re-reads of files already seen
* no directory walks (ls -R, find .)
* honor .pr-review.json:exclude_paths
3. De-generalize cost_model calibration labels. The OBSERVED_RUNS list
referred to `gitea_admin/pragent#7` — a real internal repo path that
blocks commercialization. Replaced with `internal/hardening-PR (16
files, 1020 insertions / 91 deletions)`. The numbers (input/output
tokens, steps, duration) are unchanged — only the labels are
generic.
Tests:
* test_inline_comment_body_with_attribution_line — asserts 🪙 line
shows when _tok_attrib is set
* test_inline_comment_body_no_attribution_no_coin_line — still
verifies the line is hidden when no attribution data
* test_observed_report_prices_every_model — asserts no internal
repo name appears in the rendered report
Co-Authored-By: Claude <noreply@anthropic.com>
|
||
|
Marcos
|
f6be2b3c61 |
feat(review): PR-level collapsible metadata + emoji-tagged inline comments
PR-level comment layout (per operator's format guide):
* Summary of Changes — 2-4 bullets, sourced from the agent's new
`summary_changes` JSON field. Falls back to splitting the prose
`summary` if the list is missing.
* Key Risks & Concerns — bullets from the new `risks` JSON field.
* Findings Overview — Markdown table covering every finding
(severity emoji / location / one-line problem). Both anchored and
unanchored findings appear here so the table is the single scan point.
* Unanchored Notes — bullets with severity + fix + Markdown-linked ref,
for findings with no post-change line to anchor.
* AI Usage & Run Details — wrapped in a <details>/<summary> collapsible
so the body stays scannable. Cost line stays inside it.
Inline comment shape:
* Severity badge: 🔴 [HIGH] / 🟡 [MEDIUM] / 🔵 [LOW] / ⚪ [INFO].
Unknown severities fall back to � [INFO].
* 1-2 short paragraphs of problem; **Fix:** label for the fix line.
* Standard ```suggestion fence for replacement code (Gitea/Forgejo
apply-on-click). Language-tagged fences are no longer used for
single-file diffs.
* Reference as a Markdown hyperlink, visible label truncated to
<=60 chars; the underlying URL is preserved verbatim.
* NO per-comment 🪙 token attribution. All telemetry stays in the
collapsible block on the PR-level comment.
Agent prompt updated to emit `summary_changes` and `risks` in the JSON
output (backward-compatible — older outputs missing them still parse;
they fall back to splitting the prose `summary`).
Tests: 15 new (severity emoji mapping, reference truncation, findings
table escaping, collapsible usage rendering, summary_changes+risks
layout). Existing tests updated for the new structure.
Co-Authored-By: Claude <noreply@anthropic.com>
|
||
|
Marcos
|
5302e8dcd7 |
fix(review): salvage findings from nested-object fences + bare arrays + unfenced tail JSON
The canalhandia PR review lost all findings because the agent ran out of
context before emitting the closing json fence. Three failure modes hit
the old regex \{.*?\}:
* nested objects inside the fence truncated at the first }
* bare arrays (no {summary, findings} wrapper) returned []
* unfenced JSON in the prose tail was never reached (first not last)
Replace the regex with a balanced-brace scanner:
* _last_json_block walks the fence contents with a depth counter so
nested objects survive
* _last_balanced_json + _balanced_json_substring handle bare arrays and
prose-tail JSON when no fence is present
* _parse_json_tolerant returns list as well as dict; parse_findings and
parse_review_output accept a bare array as the outer value
Agent prompt tightened: reserve the final step for emitting the JSON
block so the analysis isn't lost when context runs out.
10 new tests in tests/pilot/test_ai_review.py cover the new shapes.
Co-Authored-By: Claude <noreply@anthropic.com>
|
||
|
Marcos
|
ec26ec000a |
feat(review): equivalent provider price + enriched .pr-review.json schema
Three things in this commit, all in the review-rendering path: 1. COST DISPLAY — the `## 🔋 AI usage` section used to show $0.00 because the pilot runs on headroom/glm-5.2:cloud at no per-token charge. Now it shows TWO lines: the equivalent provider cost (default Claude Sonnet 5; configurable via .pr-review.json:cost_target or PRAGENT_PRICE_TARGET env) AND the actual $0.00 line. Maintainers can now budget on what the same measured tokens would cost on a paid model. equivalent_cost() builds a cost_model.Usage from the measured dict and runs cost_model.cost() against the resolved provider. _resolve_price_target walks repo config > env > default, surfaces typos as an inline note on the usage line (not a crash). 2. .pr-review.json SCHEMA — seven new optional fields: style strict|balanced|lenient (default: balanced) severity_threshold low|medium|high|critical (per style) max_findings 1..30 (per style) exclude_tests bool (skip test files) require_tests bool (synthetic finding) patterns {allow: [...], deny: [...]} (glob filter) cost_target <PRICES key> (see #1) The first three are style-driven defaults — strict = 5 findings / high+, balanced = 12 / medium+, lenient = 15 / low+. Override per-field. patterns globs support * and **; built-in fnmatch-style with re.escape. 3. APPLY CONFIG — findings are filtered by the new schema before being split into anchored/unanchored. apply_repo_config() drops by exclude_tests / exclude_paths / patterns.deny / patterns.allow / severity_threshold, then caps at max_findings. require_tests=true appends a synthetic 'low' finding when changed paths include non-test files but no test file changed alongside them. build_user_prompt renders the new fields into the brief so the agent knows about style / threshold / patterns explicitly (not just via instructions). Plus plumbing: * review_pr runs compress_diff(diff, context=PRAGENT_DIFF_CONTEXT) before handing the diff to either engine. Default context=1 (enough to anchor; full files are on disk in the workdir anyway). -1 disables. * compact_prior_reviews(prior) keeps only finding-bullet lines, drops the rest. Prior-review cap lowered 8k -> 4k chars in build_user_prompt. * opencode_review.write_brief accepts compression_note (rendered under the PR description, OUTSIDE the untrusted-data fence). 160 new tests covering equivalent_cost (4), format_usage_section cost lines (5), parse_repo_config extended schema (6), apply_repo_config filters (8), effective_config style defaults (2), compact_prior_reviews (2), and the whole diff_compress suite (14 from the previous commit). 174 pass / 0 fail. |
||
|
Marcos
|
770581bf53 |
feat(input): add diff_compress module + prior-review compaction helpers
Two pure stdlib helpers that shrink what lands in the model prompt:
* compress_diff(diff, *, context=2) — re-renders a unified diff so each
hunk keeps only unchanged lines on either side of its +/- lines.
File headers + hunk headers + +/- lines preserved verbatim. Pure-context
hunks dropped (rare but legal — git emits them on whitespace-only diffs).
Collapsed gaps of >=5 lines emit a single '@@ … N context line(s) omitted
… @@' marker so the reviewer knows code was elided. Smaller gaps stay
silent — the marker would be longer than the elision.
* extract_finding_bullets(review_body) — pulls the lines of a prior review
that look like a pragent finding (- **[SEVERITY]** path:line — …) and
drops everything else. The model already has the diff; repeating the
prose is just token burn.
No I/O, no network. Tolerant of malformed input — never raises. 14 unit
tests cover both helpers, including an anchor-preservation check against
parse_diff_anchors to guarantee compress-then-anchor still works.
Wiring in ai_review/opencode_review lives in the next commit.
|