The canalhandia PR review lost all findings because the agent ran out of
context before emitting the closing json fence. Three failure modes hit
the old regex \{.*?\}:
* nested objects inside the fence truncated at the first }
* bare arrays (no {summary, findings} wrapper) returned []
* unfenced JSON in the prose tail was never reached (first not last)
Replace the regex with a balanced-brace scanner:
* _last_json_block walks the fence contents with a depth counter so
nested objects survive
* _last_balanced_json + _balanced_json_substring handle bare arrays and
prose-tail JSON when no fence is present
* _parse_json_tolerant returns list as well as dict; parse_findings and
parse_review_output accept a bare array as the outer value
Agent prompt tightened: reserve the final step for emitting the JSON
block so the analysis isn't lost when context runs out.
10 new tests in tests/pilot/test_ai_review.py cover the new shapes.
Co-Authored-By: Claude <noreply@anthropic.com>
Three things in this commit, all in the review-rendering path:
1. COST DISPLAY — the `## 🔋 AI usage` section used to show $0.00 because
the pilot runs on headroom/glm-5.2:cloud at no per-token charge. Now it
shows TWO lines: the equivalent provider cost (default Claude Sonnet 5;
configurable via .pr-review.json:cost_target or PRAGENT_PRICE_TARGET env)
AND the actual $0.00 line. Maintainers can now budget on what the same
measured tokens would cost on a paid model.
equivalent_cost() builds a cost_model.Usage from the measured dict and
runs cost_model.cost() against the resolved provider. _resolve_price_target
walks repo config > env > default, surfaces typos as an inline note on
the usage line (not a crash).
2. .pr-review.json SCHEMA — seven new optional fields:
style strict|balanced|lenient (default: balanced)
severity_threshold low|medium|high|critical (per style)
max_findings 1..30 (per style)
exclude_tests bool (skip test files)
require_tests bool (synthetic finding)
patterns {allow: [...], deny: [...]} (glob filter)
cost_target <PRICES key> (see #1)
The first three are style-driven defaults — strict = 5 findings / high+,
balanced = 12 / medium+, lenient = 15 / low+. Override per-field.
patterns globs support * and **; built-in fnmatch-style with re.escape.
3. APPLY CONFIG — findings are filtered by the new schema before being
split into anchored/unanchored. apply_repo_config() drops by exclude_tests
/ exclude_paths / patterns.deny / patterns.allow / severity_threshold, then
caps at max_findings. require_tests=true appends a synthetic 'low' finding
when changed paths include non-test files but no test file changed
alongside them.
build_user_prompt renders the new fields into the brief so the agent knows
about style / threshold / patterns explicitly (not just via instructions).
Plus plumbing:
* review_pr runs compress_diff(diff, context=PRAGENT_DIFF_CONTEXT) before
handing the diff to either engine. Default context=1 (enough to anchor;
full files are on disk in the workdir anyway). -1 disables.
* compact_prior_reviews(prior) keeps only finding-bullet lines, drops the
rest. Prior-review cap lowered 8k -> 4k chars in build_user_prompt.
* opencode_review.write_brief accepts compression_note (rendered under
the PR description, OUTSIDE the untrusted-data fence).
160 new tests covering equivalent_cost (4), format_usage_section cost lines
(5), parse_repo_config extended schema (6), apply_repo_config filters (8),
effective_config style defaults (2), compact_prior_reviews (2), and the
whole diff_compress suite (14 from the previous commit). 174 pass / 0 fail.
Two pure stdlib helpers that shrink what lands in the model prompt:
* compress_diff(diff, *, context=2) — re-renders a unified diff so each
hunk keeps only unchanged lines on either side of its +/- lines.
File headers + hunk headers + +/- lines preserved verbatim. Pure-context
hunks dropped (rare but legal — git emits them on whitespace-only diffs).
Collapsed gaps of >=5 lines emit a single '@@ … N context line(s) omitted
… @@' marker so the reviewer knows code was elided. Smaller gaps stay
silent — the marker would be longer than the elision.
* extract_finding_bullets(review_body) — pulls the lines of a prior review
that look like a pragent finding (- **[SEVERITY]** path:line — …) and
drops everything else. The model already has the diff; repeating the
prose is just token burn.
No I/O, no network. Tolerant of malformed input — never raises. 14 unit
tests cover both helpers, including an anchor-preservation check against
parse_diff_anchors to guarantee compress-then-anchor still works.
Wiring in ai_review/opencode_review lives in the next commit.
Audited the working tree and all 26 commits of history for credentials: none
found. No API keys, no private keys, no tokens — the live bot token, webhook
secret and admin token appear nowhere in the repo or its history.
What was there was infrastructure disclosure, which is recon material rather
than a leak, but has no business in a public repo:
- Tailnet addresses and cluster-internal hostnames in code, docs and the CI
template. The model endpoint is now supplied at runtime via
PRAGENT_MODEL_BASE_URL and patched into opencode.json by install_config();
the committed config carries a placeholder, guarded by a test.
- A host path (/home/marcos) as the default rtk directory — now unset.
- Real usernames in the onboarding docs — now alice/acme.
- A standing list of one-time setup tokens that were never revoked, named
individually. Removed. Note that removing the list does not revoke the
tokens: they should still be revoked in the Gitea admin UI.
The substitution happens in Python rather than via opencode's {env:VAR} config
templating, because the reviewer subprocess runs with an allow-listed
environment — resolving it before the process starts keeps that allow-list from
having to grow.
README rewritten for a reader who has never seen the project: what it does and
what that output looks like, honest status (pilot works, framework designed but
unbuilt), the security model up front given what this thing is, and the measured
cost numbers including the two effects that make naive estimates wrong.
History still contains the old addresses. They are tailnet-only and not
credentials, so no rewrite.
Tests: 131 -> 137.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
31 steps, 190s, 2,213,077 in / 9,058 out, cache 0/0 — within 7% of the first
run's input on the same tier, which is the first evidence the recalibrated tier
defaults hold rather than fitting a single point.
Also weakens the step-cap hypothesis from the previous commit: this run used 31
steps (more than the 28 that succeeded, and more than the run that failed) and
parsed fine, so hitting `steps: 40` is not on its own what breaks the output
format. Leaving the cap alone until the stderr logging catches a real failure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
Found by running the reviewer against this branch. The second review of PR #7
ran a full agent loop — 330s of a 540s budget, no timeout — and ended without a
parseable ```json block. The code discarded the entire run and posted "AI review
produced no parseable output.", losing minutes of work and millions of tokens
for a message that tells the maintainer nothing and gives me nothing to debug.
Three changes on that path:
- salvage_summary() keeps the agent's prose (fenced blocks stripped, tail kept
because the conclusion is written last) and posts it under an explicit banner
saying it is unstructured and its line numbers were never validated against
the diff. A partial review honestly labelled beats no review.
- The raw output's length and last 600 chars go to stderr, so the next
occurrence is diagnosable from pod logs instead of invisible.
- The AI-USAGE section is still rendered. The label asked for it and the tokens
were spent either way; dropping the measurement on the failure path is how the
cost model stops getting calibration data exactly when it is most interesting.
Not fixed here: why the agent went off-format. The likely cause is the 40-step
cap in the agent definition being reached on a larger diff (the successful run
used 28), which wants either a higher cap or a step-budget warning in the
prompt. Needs the next occurrence's stderr to confirm rather than guess.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
PR #7 ran under the AI-USAGE label and reported real numbers: 28 agent steps,
348s, 2,071,025 input / 17,303 output tokens, and zero cache reads or writes.
The model predicted ~$0.73 on Opus 5 for that tier. The measurement prices it
at $10.79 — the model was ~15x low.
Two wrong assumptions:
- Step count and per-step growth. `full` assumed 12 steps and 1,200 tokens per
tool result; the run did 28 steps averaging ~3,300. Cost is roughly quadratic
in steps, so this compounds. Tier defaults are re-derived from the measured
per-step growth rather than from guesses.
- Caching. The model defaulted to prompt caching on. The headroom/glm-5.2 path
reports 0 read / 0 write, so the stable prefix is paid at full input price on
every step. Budget with caching off until that column is nonzero.
Adds OBSERVED_RUNS as an append-only calibration anchor, an observed-runs
section in the report, and a regression test asserting the model stays within
2.5x of the measurement — so the next drift is caught by the suite rather than
by a surprising invoice.
Corrected blended figures at 350 PRs/month: ~$1,740 Opus 5, ~$1,755 GPT-5.6
Sol, ~$696 Sonnet 5, ~$348 Haiku 4.5, ~$70 GPT-5.6 Luna.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
Skills — the primary now loads conditionally (each one is input tokens), per a
load table in pragent.md:
- attention-tiering: classify every PR trivial/lite/full/oversized BEFORE
reading anything, and cap file reads, linter runs and subagent fan-out per
tier. This is the cost governor; the other skills defer to its budget.
- linter-playbook: per-ecosystem detect-and-run commands scoped to changed
files, the never-install rule, and how to turn a diagnostic into a finding
instead of pasting tool output.
- security-lens: the inline security checklist for when @security isn't worth
delegating, built around a source -> sink test each finding must pass.
- malicious-change: hostile-PR detection — injection aimed at the reviewer,
install/CI-time hooks, obfuscated payloads, dependency confusion, logic
backdoors. Complements the runtime containment added in the previous commit:
that stops the agent being hijacked, this makes it report the attempt.
- comment-craft: how to write problem/fix/suggestion so a maintainer can act in
one read, and what to cut.
pilot/cost_model.py — prices a review against published Claude and OpenAI rates
(fetched 2026-08-18). Prompt sizes are measured from the factory files rather
than guessed; per-tier workloads come from the tiering budgets. The model is
explicit about the thing that actually dominates an agent loop: the whole
conversation is resent every step, so caching moves ~2.3x of the bill.
Blended over a 5/35/55/5 mix with caching on: ~$0.61/PR on Opus 5 or GPT-5.6
Sol, ~$0.24 on Sonnet 5 or Terra, ~$0.12 on Haiku 4.5, ~$0.02 on Luna. At 350
PRs/month that's ~$212 / ~$85 / ~$43 / ~$8.50.
Tests: 101 -> 122.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
The reviewer runs an opencode agent with `bash: "*": allow` over a checkout of
the PR author's branch, and the pod holds a Gitea Write credential. Those two
facts had no wall between them.
Security
- _build_env now allow-lists the subprocess environment instead of inheriting
it, so PRAGENT_BOT_TOKEN and WEBHOOK_SECRET never reach the agent. This was
the live hole: a PR body or an AGENTS.md could ask the agent to `curl` the
token out, and it had both the value and the tool.
- sanitize_workdir deletes author-controlled agent-instruction files from the
checkout before opencode starts (AGENTS.md at any depth, CLAUDE.md,
.cursorrules, a repo opencode.json/.opencode, copilot-instructions.md).
opencode loads nested AGENTS.md as instructions, so a PR could otherwise ship
its own system prompt. They are still reviewed, as data.
- The brief fences PR title/body and diff in --- UNTRUSTED --- markers under a
trust-boundary preamble; the pragent agent, the three lens subagents and the
review-methodology skill now treat injection attempts as a critical finding
to report rather than an instruction to obey.
- .pr-review.json is read from the PR's base branch, not the head sha. Its
`instructions` field is spliced into the reviewer's prompt, so head-ref
reading let any author rewrite the reviewer's rules. Fields are length-capped.
- Untar rejects escaping symlinks, parent traversal, and writes through a
planted symlink (tar-slip).
- The image runs as uid 10001 instead of root.
Robustness
- Bounded review concurrency (PRAGENT_MAX_CONCURRENT_REVIEWS, default 2). Each
review forks an opencode process; a thread per delivery was a fork bomb on a
burst of labels or Gitea retries.
- An in-flight (repo, index, sha) claim closes the check-then-act race in the
sha-marker dedupe, where two deliveries a second apart both read "not yet
reviewed" and both posted.
- Request bodies are capped before being read into memory.
Correctness
- parse_diff_anchors counts a whitespace-stripped blank context line. Skipping
it desynced the new-line counter for the rest of the hunk and silently
misplaced every later inline comment in that file.
- post_inline_review's body-only fallback folds the anchored findings into the
body. It previously posted a summary saying "N inline comment(s) below" with
no comments and no findings — losing them all on the one path that matters.
- fetch_pr_diff's files-endpoint fallback emits real a// b/ prefixes (so
changed_files and the anchor parser work on it) and reports both HTTP statuses
in its error instead of the same one twice.
- The CI workflow template pins PRAGENT_ENGINE=ollama; review_pr defaults to
opencode, which does not exist on a Gitea Actions runner.
Tests: 68 -> 101, covering each of the above.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
Add per-review + per-comment token accounting, surfaced only when a PR carries
the new AI-USAGE label (on top of the existing AI-REVIEW trigger).
opencode_review:
- run_opencode now uses `--format json`; parse_opencode_events reconstructs the
assistant text from `text` events and sums tokens/cost/steps from every
`step_finish` event (tolerant of noise / missing fields).
- run() measures duration_s around the opencode call and returns (text, usage).
- changed_files(diff) extracts the `+++ b/` paths; the brief now lists them
under a "Changed files" focus block so the agent grounds findings in the
diff's neighbourhood instead of unbounded whole-repo walks.
ai_review:
- format_usage_section renders a `## AI usage` block: measured totals
(in/out/reasoning/cache/cost/steps/duration), the whole-repo scope note, and
an attributed per-finding table. Per-comment counts are output tokens split by
each finding's body weight — labelled "attributed" since one model pass
produces all findings.
- inline_comment_body appends `🪙 ~N tok (X% · attributed output)` when
attribution is present.
- review_pr gains report_usage; compute_attribution stashes _tok_attrib/_tok_pct.
- format_review_body inserts the usage section between summary and findings.
webhook_server:
- Fire on every pull_request action except `closed` (denylist, was an allowlist)
— the AI-REVIEW gate + sha dedupe keep this safe.
- AI-USAGE label detection + PRAGENT_USAGE_ALWAYS env drive report_usage.
.opencode factory + review-methodology skill: new "Ground findings in context"
step — read callers/imports/sibling functions per changed file (1-3 files per
finding), no unbounded walks.
Tests: parse_opencode_events (text+usage sum, malformed tolerance, none-usage),
changed_files, compute_attribution math, inline 🪙 line, format_usage_section
totals/table/cost, format_review_body ordering. 68 passing.
Co-Authored-By: Claude <noreply@anthropic.com>
Gitea 1.26.x has no GitHub-style 'Apply suggestion' button — a ```suggestion
fence is just an unknown-language code block, so chroma does not highlight it
and there is no apply control. Switch inline_comment_body to wrap the suggested
fix in a fence tagged with the file's language (new _lang_for_path helper,
.java→java, .ts→typescript, .py→python, ...), so Gitea syntax-highlights the
code. No capability lost (there was never an apply button on this Gitea
version). Correct the docstrings/skills/README that wrongly claimed an
apply-button was rendered.
Co-Authored-By: Claude <noreply@anthropic.com>
Gitea 1.26.x's POST /pulls/{i}/reviews does NOT honor the line/side fields
used by newer Gitea — it silently drops them, leaving the comment unpositioned.
Gitea then renders a file-level review comment on EVERY diff line of the file,
so a 5-finding review on a 25-line diff showed ~125 comment blocks in the
Files Changed view (the flood reported on canalhandia PR #2).
The 1.26 schema anchors inline review comments with new_position (line in the
post-change file) + old_position: 0. f["line"] is already a validated
post-change (RIGHT-side) line from split_findings, so it maps directly to
new_position. Verified: new_position=98 -> position=98 + populated diff_hunk
(positioned, renders on line 98 only); the old line/side form -> position=0,
empty diff_hunk (unpositioned).
49 tests pass (no test asserted the POST payload shape).
Co-Authored-By: Claude <noreply@anthropic.com>
The npm prefix in the Dockerfile is /usr, so opencode installs to
/usr/bin/opencode (symlink to /usr/lib/node_modules/...), not
/usr/local/bin/opencode. The cluster E2E failed with ENOENT on
/usr/local/bin/opencode. Fix in three places:
- Dockerfile ENV PRAGENT_OPENCODE_BIN=/usr/bin/opencode
- ~/k8s/pragent-webhook.yaml env value
- _opencode_bin() now defensive: if the configured path is missing,
falls back to shutil.which('opencode') before the linuxbrew last-resort.
- Dockerfile + README deploy notes: containerd import is sudoless via
the group-readable raw socket (the microk8s ctr wrapper sudo-wraps).
Verified: rebuilt + reimported + rolled out; PR #5 (sha 985061c0) review
posted in-pod via the opencode engine (findings=2 inline=2 ok=True),
summary + 2 [CRITICAL] inline comments with suggestions + refs + sha marker.
Co-Authored-By: Claude <noreply@anthropic.com>
Replace the single Python model-call reviewer with an opencode agent
factory. A primary 'pragent' agent reads a brief (title/body/diff/config/
prior reviews), inspects the checked-out repo, runs the repo's own linters
via bash, loads review-methodology + findings-schema skills, and emits a
{summary, findings} JSON with per-finding severity/path/line/problem/fix/
suggestion/reference. Dormant security/tests/perf subagent lenses fan out
only on large/risky diffs (lean by default).
pilot/opencode_review.py: fetches the repo archive at the head sha into a
temp workdir, writes .pragent/brief.md, drops the factory, runs
'opencode run --pure --agent pragent --dir <workdir>' headlessly. Isolates
HOME (shared, warmed), strips ANTHROPIC_* env (leaked host vars caused
ProviderModelNotFoundError), stdin=DEVNULL (opencode blocks on stdin),
maps the bare OLLAMA_MODEL to the provider-prefixed ref. No Gitea I/O —
ai_review.review_pr parses + anchors + posts (reuses all v2 logic/tests).
PRAGENT_ENGINE=opencode (default) selects it; =ollama keeps the legacy
direct-call path. Verified end-to-end: posts a real review with a summary
section, inline [CRITICAL]/[HIGH] comments + apply-able suggestions +
reference links, and the sha dedupe marker. 49 tests pass.
Co-Authored-By: Claude <noreply@anthropic.com>
- pilot/webhook_server.py: stdlib HTTP receiver. HMAC-verifies X-Gitea-Signature,
gates on pull_request action + AI-REVIEW label, runs review_pr in a background
thread (responds 202 immediately so Gitea's delivery timeout never fires).
Accepts both GitHub-style (labeled/synchronize) and Gitea event-type-style
(label_updated/synchronized) action names.
- pilot/ai_review.py: extract review_pr() core so both the CI run() and the
webhook server share one review path. run() is now an env-driven wrapper.
- pilot/README-webhook.md: architecture, onboarding, one-time per-owner
user-webhook setup, the Gitea 1.26.1 system-webhook bug, the SSRF
ALLOWED_HOST_LIST change, K8s deploy + script-update recipe.
- README.md + design doc: note the webhook service as the preferred delivery
path (partially reverses 'central webhook = non-goal', pilot only).
Gitea 1.26.1 system webhooks broken (POST /admin/hooks -> 201 but never
persists); user-level webhooks (one per repo-owner) are the working fallback.
Gitea SSRF allow-list blocks in-cluster webhook targets by default; required a
scoped [webhook] ALLOWED_HOST_LIST addition + gitea restart.
E2E verified 2026-08-17: pragent-bot reviewed gitea_admin/pragent PR #2 and
masi/portfolio PR #3 via the webhook service (glm-5.2:cloud).
Co-Authored-By: Claude <noreply@anthropic.com>
The previous /raw/branch/main/<path> form 404s on private repos; Gitea's raw
endpoint is /raw/<path>?ref=<ref>. Verified the bot token fetches the script
with the corrected form.
Co-Authored-By: Claude <noreply@anthropic.com>
Ships a working pragent pilot ahead of the framework build (design doc
deferred). Single stdlib-only reviewer script fetched at runtime by a per-repo
Gitea Action; reviews fire only on PRs with the AI-REVIEW label; model is
glm-5.2:cloud via the on-network headroom proxy; fail-open.
- pilot/ai_review.py: fetch PR diff, call model, post review as pragent-bot
- pilot/workflow-template.yml: per-repo Gitea Action gated on AI-REVIEW label
- pilot/README.md: onboarding (bot collaborator + secret + workflow + label)
- tests/pilot/test_ai_review.py: 15 unit tests for pure helpers (no network)
- README/design doc: note pilot is the bootstrap; framework build deferred
Co-Authored-By: Claude <noreply@anthropic.com>
Marcos