Commit Graph

56 Commits

Author SHA1 Message Date
claude 979c93bdbb feat(review): render walkthrough + risk_verdict + test_coverage 2026-08-22 00:58:51 +00:00
claude 66628aae8d fix(opencode_review): harden _synthesize_summary_fields (none-safe, dedupe emoji map) 2026-08-22 00:52:35 +00:00
claude e5a6e8923d feat(opencode_review): python fallback for summary fields 2026-08-22 00:46:08 +00:00
claude 2e982846d9 feat(review): walkthrough/risk_verdict/test_coverage schema 2026-08-22 00:36:50 +00:00
claude 72b77a96e0 fix(confidence): pass multi_lens_observed kwarg to merge_confidence 2026-08-22 00:29:28 +00:00
claude f3a125666b feat(ai_review): per-PR merge confidence 1-5 in header 2026-08-22 00:23:24 +00:00
claude 9dae850887 test(config): fix cap-at-12 test to actually exercise the cap 2026-08-22 00:19:25 +00:00
claude 0f7903377a feat(config): parse enabled + compare_against 2026-08-22 00:12:06 +00:00
claude 0b295e2443 fix(severity): extend badge label set + update SYSTEM_PROMPT 2026-08-22 00:06:22 +00:00
claude 23ec1bf74a feat(severity): add trivial + info levels 2026-08-21 23:59:48 +00:00
claude 2cf4bdbfe8 feat(ai_review): render humanized token counts in usage + inline 2026-08-21 23:54:10 +00:00
claude 3bcf825104 feat(ai_review): fmt_tokens() humanizes token counts 2026-08-21 23:46:24 +00:00
claude 67339da8d0 feat(cost-model): add GPT, Gemini, Grok prices 2026-08-21 23:42:07 +00:00
claude 99014a4cd3 chore(gitignore): exclude .worktrees/ 2026-08-21 23:39:44 +00:00
claude f52b8d7803 docs(plans): 2026-08-21 pragent update implementation plan 2026-08-21 23:39:11 +00:00
claude f9f6ab4bf0 docs(plans): 2026-08-21 pragent update design
- Repo opt-in (.pr-review.json:enabled) replaces AI-REVIEW/AI-USAGE labels.
- Token humanization (fmt_tokens), multi-provider equivalent cost (Claude,
  GPT, Gemini, Grok), richer review summary (walkthrough + risk verdict +
  test coverage), trivial/info severity levels, per-PR merge confidence 1-5.
- Defers reachability severity demotion, rules-mining from feedback, and
  heavy infra (sequence diagrams, T-rex, cross-repo) to future work.
2026-08-21 23:32:49 +00:00
gitea_admin cdf116ece9 Merge PR #10: read the AI-USAGE opt-in at render time
Late AI-USAGE labels now reach the review body. Verified live on this PR: AI-REVIEW then AI-USAGE six seconds later, usage block present in review #131.
2026-08-20 23:42:36 +00:00
Marcos c58c00181e fix(review): read the AI-USAGE opt-in at render time, not at review start
The previous two commits put the label re-read in the webhook, at review
start. That is too early to help: the review claims on the AI-REVIEW event
and the re-read runs milliseconds later, while the reviewer's second click
(AI-USAGE) is still a second or two away. It would have kept 404ing quietly
if the path fix hadn't landed, and even fixed it caught nothing.

Move the check to where the decision is actually used — just before the
usage block is rendered, after the model has run. That is a minute or more
after the trigger, by which time the label is there. Attribution is computed
in the same branch, so a late opt-in still gets its per-comment token lines.

`pr_has_label` goes through the existing gitea_get helper, which owns the
/api/v1 prefix, so the path can't drift again. Any failure returns False and
the payload's verdict stands: a review is never lost over a usage section.

Reverts the webhook-side re-read from 4ef62f2 and aedea97.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
2026-08-20 23:35:04 +00:00
Marcos aedea973ab fix(webhook): label re-read must hit /api/v1, not the bare host
GITEA_API is the host with no version prefix — ai_review._get / _post append
/api/v1 per call. The re-read helper didn't, so it 404'd on every review and
silently fell back to the payload's verdict: "could not re-read labels for
gitea_admin/pragent#10: HTTP Error 404" in the pod log, usage block still
missing. Caught by labelling PR #10 AI-REVIEW then AI-USAGE, which is the
exact sequence the fix exists to handle.

Test asserts the composed URL, since a wrong path here fails silently by
design (the helper swallows errors so a review is never lost over a usage
section).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
2026-08-20 23:29:03 +00:00
Marcos 4ef62f28bb fix(webhook): re-read PR labels at review start so a late AI-USAGE counts
Labelling a PR is two webhook events. AI-REVIEW arrives first, the review
claims (repo, index, sha) and starts; the AI-USAGE event that follows a
moment later hits the in-flight dedupe and is dropped. `report_usage` was
snapshotted from the first payload, which had not seen AI-USAGE yet, so the
review posted without its usage block even though the label was on the PR by
the time it finished — observed on gitea_admin/pragent#9 (`usage=False` in
the pod log, no <details> section in the posted review).

Re-read the labels from the API at review start and upgrade the flag. The
read is best-effort: any failure logs and returns [], leaving the payload's
verdict intact, because a usage section is not worth failing a review over.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
2026-08-20 23:24:54 +00:00
gitea_admin 7e4fd1975d Merge PR #9: diff compression + prior-review compaction + multi-lens orchestration
Fixes from review of the branch: hunk-header renumbering in compress_diff, raw-diff anchoring, prior-review bullet extraction, triage empty-lens contract.
2026-08-20 23:05:31 +00:00
Marcos 2b1cf750b7 fix(review): correct diff-compression line numbers, prior-review dedupe, triage skip
Four defects, all found reviewing PR #9 (two of them by pragent-bot's own
review of that PR, which the anchoring bug then misplaced):

* compress_diff dropped context lines but copied the original `@@` hunk
  header verbatim, so the header no longer described the lines beneath it.
  parse_diff_anchors then walked stale headers and produced anchor sets
  shifted by the number of elided lines, misplacing inline comments or
  demoting them to bullets. Each surviving run of lines is now re-emitted as
  its own hunk with a recomputed `@@ -a,b +c,d @@`, so the output stays a
  valid unified diff whose numbers describe the real post-change file. The
  pseudo-marker `@@ … N context line(s) omitted … @@` is gone; it parsed as
  a hunk header and reset the anchor counter to 0. Anchoring additionally
  runs on the raw diff now, so the prompt window can never shrink the
  anchorable set.

* compress_diff's `_FILE_HEADER` regex matched diff *body* lines: a removed
  YAML `---` separator or an added `++` line was read as a file header,
  truncating the hunk and dropping its `@@` header with it. Body detection
  is now prefix-based, with a full-shape hunk-header regex.

* extract_finding_bullets could not match the bullets pragent itself posts:
  summary_bullets renders an emoji severity badge between the `-` and the
  `[SEV]` tag, which the regex rejected, so compact_prior_reviews always
  returned [] and every re-review repeated its previous findings.

* triage returning `{"lenses":[]}` — documented in .opencode/agents/triage.md
  as "no lens has surface, skip the fan-out" — ran every lens instead, since
  _intersect_with_triage mapped an empty selection to "all" and the call site
  had a second `or reviewers` fallback. `[]` and None are now distinct
  outcomes: `[]` skips, None fails open. A roster naming only unknown lens
  ids now fails open rather than silencing the review. The skip path returns
  a well-formed empty-findings response instead of "", which had landed in
  ai_review's unparseable-output branch and posted "AI review produced no
  parseable output" — a malfunction message for a normal verdict.

Also: non-URL references (a CVE id, a doc title) rendered as
`[CVE-2024-1234](CVE-2024-1234)`, a broken relative link in Gitea — now
plain text. PRAGENT_DIFF_CONTEXT and friends parse through _int_env, so a
typo logs and falls back instead of killing a review mid-flight. Removed
format_usage_section, dead since the collapsible usage block replaced it and
carrying a duplicate copy of the price-target logic.

Tests: 290 -> 301. New coverage for hunk-header fidelity before/after
compression, header-shaped content lines, the bullet round-trip against the
real renderer, and triage's three outcomes (previously untested).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
2026-08-20 23:05:03 +00:00
pragent-bot 78bcf6a9a0 feat(review): multi-lens orchestration — 5 parallel opencode subprocesses (security/docs/code-quality/tests/perf)
On by default, opt-out via "reviewers": []. 290 tests pass.

- pilot/opencode_review.py: ReviewerSpec dataclass, default_reviewers(),
  parse_reviewers_config(), parse_triage_config(), resolve_reviewers(),
  _normalize_lens_finding(), posthash() (matches feedback.py scheme),
  _agreement_hash() (severity-free for cross-lens promotion), _tone_strip(),
  synthesize() 7-stage (severity_floor → tone-strip → length cap → per-lens
  max → per-file cap → dedup by _posthash → cross-lens severity promote →
  per-PR cap), run_lenses() (ThreadPoolExecutor pool=4), triage(),
  _intersect_with_triage(), _filter_by_skip_if(), run_lenses_review().
  run() routes to fan-out when config.reviewers[] present or PRAGENT_REVIEWERS=1.
- pilot/ai_review.py: parse_repo_config learns reviewers[] and triage objects
  (id regex /^[a-z0-9][a-z0-9-]{0,31}$/, 8-entry cap, agent_file/model/
  severity_floor/max_findings/activation/skip_if_all_changed_paths/hotpath_globs).
  review_pr branches to opencode_review.run_lenses_review when configured.
  _render_collapsible_usage shows lenses: ... line when present.
- .opencode/agents/{docs,code-quality,triage}.md: 3 new lens subagents.
- .opencode/skills/lens-orchestration/SKILL.md: strict-JSON contract every
  lens subagent MUST honor.
- .opencode/agents/pragent.md: slim to coordinator; no more hardcoded
  @security/@tests/@perf delegation; loads lens-orchestration skill.
- .opencode/README.md: rewrite 'Add a review lens' recipe for multi-lens.
- pilot/README-webhook.md: new 'Multi-lens pipeline' section (diagram +
  default roster + config schema + env vars + cross-lens dedup contract).
- tests: 36 new tests (test_ai_review.py +12 reviewers/triage/usage,
  test_opencode_review.py +24 orchestration). posthash golden-vector matches
  feedback.py exactly across 5 severity × 2 line cases.
2026-08-20 22:16:21 +00:00
Marcos e8ebc54362 feat(review): ADDITIONAL_CONTEXT_URL — repo-provided static context cached per review
Long agent loops re-send the brief prefix on every step; cheap reusable
knowledge (architecture summary, module map, conventions, glossary) belongs
in a versioned file the maintainers control so the agent doesn't re-derive
it from the source tree on every PR. Two wiring paths, merged (env first):

* env var PRAGENT_ADDITIONAL_CONTEXT_URL — comma-separated, deployment-wide
* .pr-review.json:additional_context_urls — list[str], read from the PR's
  base branch (same trust boundary as the rest of the file)

Implementation:
* _parse_additional_context_env splits/dedupes/trims.
* _resolve_additional_context_urls(config) merges env (first) + config
  (then, skipping env-dupes); caps at 8.
* fetch_additional_context(urls) fetches each URL with urllib (5s timeout,
  http/https only — file://, javascript:, ftp:// rejected defensively),
  caches by URL in a module-level dict for the pod lifetime, truncates
  per-URL to 4k chars + total to 16k chars, best-effort (network errors
  are logged and skipped — never aborts the review).
* Result injected into build_user_prompt under "## Repo-provided context"
  between repo config and prior reviews. In the opencode engine it lands
  in .pragent/brief.md under its own section. The brief explicitly labels
  each block's CONTENT as untrusted (same as PR description) — section
  heading is trustworthy, body isn't.
* parse_repo_config accepts the field, caps at 8 entries, drops
  non-strings and empty strings.

Docs: pilot/README-webhook.md "Repo-provided static context" section —
env var + JSON example + Nexus raw-hosted recipe.

Tests: 14 new (208 total), covering env merging + dedup, scheme rejection,
per-URL cap, total cap, caching by URL, brief injection. All mock urllib
with a context-manager stand-in (no real network).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-20 17:45:51 +00:00
Marcos 998f793ec2 feat(agent): tighten prompt to bound beyond-diff reads + de-generalize cost-model labels
Three changes from operator feedback:

1. Per-comment � attribution restored on inline comments (operator wants
   it back — the PR-level collapsible is collapsed by default, so the
   attribution is the visible signal of per-finding cost share).
   Hidden only when no _tok_attrib was computed (legacy callers / ollama
   path without usage metering).

2. Agent prompt now bounds reads beyond the diff — the single biggest
   driver of input-token bloat on long agent loops:
     * ≤ 5 file reads beyond the diff for the entire review
     * ≤ 80 lines per read (use --offset + --limit)
     * ≤ 3 grep calls beyond the diff (prefer rtk grep)
     * no re-reads of files already seen
     * no directory walks (ls -R, find .)
     * honor .pr-review.json:exclude_paths

3. De-generalize cost_model calibration labels. The OBSERVED_RUNS list
   referred to `gitea_admin/pragent#7` — a real internal repo path that
   blocks commercialization. Replaced with `internal/hardening-PR (16
   files, 1020 insertions / 91 deletions)`. The numbers (input/output
   tokens, steps, duration) are unchanged — only the labels are
   generic.

Tests:
  * test_inline_comment_body_with_attribution_line — asserts 🪙 line
    shows when _tok_attrib is set
  * test_inline_comment_body_no_attribution_no_coin_line — still
    verifies the line is hidden when no attribution data
  * test_observed_report_prices_every_model — asserts no internal
    repo name appears in the rendered report
Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-20 17:35:06 +00:00
Marcos f6be2b3c61 feat(review): PR-level collapsible metadata + emoji-tagged inline comments
PR-level comment layout (per operator's format guide):
  * Summary of Changes — 2-4 bullets, sourced from the agent's new
    `summary_changes` JSON field. Falls back to splitting the prose
    `summary` if the list is missing.
  * Key Risks & Concerns — bullets from the new `risks` JSON field.
  * Findings Overview — Markdown table covering every finding
    (severity emoji / location / one-line problem). Both anchored and
    unanchored findings appear here so the table is the single scan point.
  * Unanchored Notes — bullets with severity + fix + Markdown-linked ref,
    for findings with no post-change line to anchor.
  * AI Usage & Run Details — wrapped in a <details>/<summary> collapsible
    so the body stays scannable. Cost line stays inside it.

Inline comment shape:
  * Severity badge: 🔴 [HIGH] / 🟡 [MEDIUM] / 🔵 [LOW] /  [INFO].
    Unknown severities fall back to � [INFO].
  * 1-2 short paragraphs of problem; **Fix:** label for the fix line.
  * Standard ```suggestion fence for replacement code (Gitea/Forgejo
    apply-on-click). Language-tagged fences are no longer used for
    single-file diffs.
  * Reference as a Markdown hyperlink, visible label truncated to
    <=60 chars; the underlying URL is preserved verbatim.
  * NO per-comment 🪙 token attribution. All telemetry stays in the
    collapsible block on the PR-level comment.

Agent prompt updated to emit `summary_changes` and `risks` in the JSON
output (backward-compatible — older outputs missing them still parse;
they fall back to splitting the prose `summary`).

Tests: 15 new (severity emoji mapping, reference truncation, findings
table escaping, collapsible usage rendering, summary_changes+risks
layout). Existing tests updated for the new structure.
Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-20 16:51:20 +00:00
Marcos 5302e8dcd7 fix(review): salvage findings from nested-object fences + bare arrays + unfenced tail JSON
The canalhandia PR review lost all findings because the agent ran out of
context before emitting the closing json fence. Three failure modes hit
the old regex \{.*?\}:
  * nested objects inside the fence truncated at the first }
  * bare arrays (no {summary, findings} wrapper) returned []
  * unfenced JSON in the prose tail was never reached (first not last)

Replace the regex with a balanced-brace scanner:
  * _last_json_block walks the fence contents with a depth counter so
    nested objects survive
  * _last_balanced_json + _balanced_json_substring handle bare arrays and
    prose-tail JSON when no fence is present
  * _parse_json_tolerant returns list as well as dict; parse_findings and
    parse_review_output accept a bare array as the outer value

Agent prompt tightened: reserve the final step for emitting the JSON
block so the analysis isn't lost when context runs out.

10 new tests in tests/pilot/test_ai_review.py cover the new shapes.
Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-20 16:29:44 +00:00
Marcos ec26ec000a feat(review): equivalent provider price + enriched .pr-review.json schema
Three things in this commit, all in the review-rendering path:

1. COST DISPLAY — the `## 🔋 AI usage` section used to show $0.00 because
   the pilot runs on headroom/glm-5.2:cloud at no per-token charge. Now it
   shows TWO lines: the equivalent provider cost (default Claude Sonnet 5;
   configurable via .pr-review.json:cost_target or PRAGENT_PRICE_TARGET env)
   AND the actual $0.00 line. Maintainers can now budget on what the same
   measured tokens would cost on a paid model.

   equivalent_cost() builds a cost_model.Usage from the measured dict and
   runs cost_model.cost() against the resolved provider. _resolve_price_target
   walks repo config > env > default, surfaces typos as an inline note on
   the usage line (not a crash).

2. .pr-review.json SCHEMA — seven new optional fields:
     style                strict|balanced|lenient  (default: balanced)
     severity_threshold   low|medium|high|critical (per style)
     max_findings         1..30                     (per style)
     exclude_tests        bool                      (skip test files)
     require_tests        bool                      (synthetic finding)
     patterns             {allow: [...], deny: [...]} (glob filter)
     cost_target          <PRICES key>              (see #1)

   The first three are style-driven defaults — strict = 5 findings / high+,
   balanced = 12 / medium+, lenient = 15 / low+. Override per-field.
   patterns globs support * and **; built-in fnmatch-style with re.escape.

3. APPLY CONFIG — findings are filtered by the new schema before being
   split into anchored/unanchored. apply_repo_config() drops by exclude_tests
   / exclude_paths / patterns.deny / patterns.allow / severity_threshold, then
   caps at max_findings. require_tests=true appends a synthetic 'low' finding
   when changed paths include non-test files but no test file changed
   alongside them.

   build_user_prompt renders the new fields into the brief so the agent knows
   about style / threshold / patterns explicitly (not just via instructions).

Plus plumbing:
  * review_pr runs compress_diff(diff, context=PRAGENT_DIFF_CONTEXT) before
    handing the diff to either engine. Default context=1 (enough to anchor;
    full files are on disk in the workdir anyway). -1 disables.
  * compact_prior_reviews(prior) keeps only finding-bullet lines, drops the
    rest. Prior-review cap lowered 8k -> 4k chars in build_user_prompt.
  * opencode_review.write_brief accepts compression_note (rendered under
    the PR description, OUTSIDE the untrusted-data fence).

160 new tests covering equivalent_cost (4), format_usage_section cost lines
(5), parse_repo_config extended schema (6), apply_repo_config filters (8),
effective_config style defaults (2), compact_prior_reviews (2), and the
whole diff_compress suite (14 from the previous commit). 174 pass / 0 fail.
2026-08-20 16:12:46 +00:00
Marcos 770581bf53 feat(input): add diff_compress module + prior-review compaction helpers
Two pure stdlib helpers that shrink what lands in the model prompt:

  * compress_diff(diff, *, context=2) — re-renders a unified diff so each
    hunk keeps only  unchanged lines on either side of its +/- lines.
    File headers + hunk headers + +/- lines preserved verbatim. Pure-context
    hunks dropped (rare but legal — git emits them on whitespace-only diffs).
    Collapsed gaps of >=5 lines emit a single '@@ … N context line(s) omitted
    … @@' marker so the reviewer knows code was elided. Smaller gaps stay
    silent — the marker would be longer than the elision.

  * extract_finding_bullets(review_body) — pulls the lines of a prior review
    that look like a pragent finding (- **[SEVERITY]** path:line — …) and
    drops everything else. The model already has the diff; repeating the
    prose is just token burn.

No I/O, no network. Tolerant of malformed input — never raises. 14 unit
tests cover both helpers, including an anchor-preservation check against
parse_diff_anchors to guarantee compress-then-anchor still works.

Wiring in ai_review/opencode_review lives in the next commit.
2026-08-20 16:12:33 +00:00
Marcos f59b906395 chore: scrub private infrastructure for a public repo, rewrite README
Audited the working tree and all 26 commits of history for credentials: none
found. No API keys, no private keys, no tokens — the live bot token, webhook
secret and admin token appear nowhere in the repo or its history.

What was there was infrastructure disclosure, which is recon material rather
than a leak, but has no business in a public repo:

- Tailnet addresses and cluster-internal hostnames in code, docs and the CI
  template. The model endpoint is now supplied at runtime via
  PRAGENT_MODEL_BASE_URL and patched into opencode.json by install_config();
  the committed config carries a placeholder, guarded by a test.
- A host path (/home/marcos) as the default rtk directory — now unset.
- Real usernames in the onboarding docs — now alice/acme.
- A standing list of one-time setup tokens that were never revoked, named
  individually. Removed. Note that removing the list does not revoke the
  tokens: they should still be revoked in the Gitea admin UI.

The substitution happens in Python rather than via opencode's {env:VAR} config
templating, because the reviewer subprocess runs with an allow-listed
environment — resolving it before the process starts keeps that allow-list from
having to grow.

README rewritten for a reader who has never seen the project: what it does and
what that output looks like, honest status (pilot works, framework designed but
unbuilt), the security model up front given what this thing is, and the measured
cost numbers including the two effects that make naive estimates wrong.

History still contains the old addresses. They are tailnet-only and not
credentials, so no rewrite.

Tests: 131 -> 137.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
2026-08-18 12:59:52 +00:00
gitea_admin 46513585ae Merge PR #7: harden the pilot against hostile PR content, add review skills + cost model
Five commits: env allow-listing + workdir sanitizing + untrusted-data framing + base-ref config + tar-slip guards + non-root image; bounded concurrency and in-flight dedupe; diff-anchor, fallback and CI-template correctness fixes; five conditionally-loaded review skills; a per-review cost model calibrated against three measured runs; and prose salvage when the findings JSON is unparseable.

Reviewed four times by pragent-bot on the hardened image: no actionable defects.
2026-08-18 05:31:07 +00:00
Marcos 80501840e2 chore(cost-model): record the second measured review
31 steps, 190s, 2,213,077 in / 9,058 out, cache 0/0 — within 7% of the first
run's input on the same tier, which is the first evidence the recalibrated tier
defaults hold rather than fitting a single point.

Also weakens the step-cap hypothesis from the previous commit: this run used 31
steps (more than the 28 that succeeded, and more than the run that failed) and
parsed fine, so hitting `steps: 40` is not on its own what breaks the output
format. Leaving the cap alone until the stderr logging catches a real failure.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
2026-08-18 05:22:02 +00:00
Marcos 5d28f97080 fix(review): salvage the prose when the findings JSON is unparseable
Found by running the reviewer against this branch. The second review of PR #7
ran a full agent loop — 330s of a 540s budget, no timeout — and ended without a
parseable ```json block. The code discarded the entire run and posted "AI review
produced no parseable output.", losing minutes of work and millions of tokens
for a message that tells the maintainer nothing and gives me nothing to debug.

Three changes on that path:

- salvage_summary() keeps the agent's prose (fenced blocks stripped, tail kept
  because the conclusion is written last) and posts it under an explicit banner
  saying it is unstructured and its line numbers were never validated against
  the diff. A partial review honestly labelled beats no review.
- The raw output's length and last 600 chars go to stderr, so the next
  occurrence is diagnosable from pod logs instead of invisible.
- The AI-USAGE section is still rendered. The label asked for it and the tokens
  were spent either way; dropping the measurement on the failure path is how the
  cost model stops getting calibration data exactly when it is most interesting.

Not fixed here: why the agent went off-format. The likely cause is the 40-step
cap in the agent definition being reached on a larger diff (the successful run
used 28), which wants either a higher cap or a step-budget warning in the
prompt. Needs the next occurrence's stderr to confirm rather than guess.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
2026-08-18 05:15:48 +00:00
Marcos 2613b3e3af fix(cost-model): calibrate against the first measured review
PR #7 ran under the AI-USAGE label and reported real numbers: 28 agent steps,
348s, 2,071,025 input / 17,303 output tokens, and zero cache reads or writes.
The model predicted ~$0.73 on Opus 5 for that tier. The measurement prices it
at $10.79 — the model was ~15x low.

Two wrong assumptions:

- Step count and per-step growth. `full` assumed 12 steps and 1,200 tokens per
  tool result; the run did 28 steps averaging ~3,300. Cost is roughly quadratic
  in steps, so this compounds. Tier defaults are re-derived from the measured
  per-step growth rather than from guesses.
- Caching. The model defaulted to prompt caching on. The headroom/glm-5.2 path
  reports 0 read / 0 write, so the stable prefix is paid at full input price on
  every step. Budget with caching off until that column is nonzero.

Adds OBSERVED_RUNS as an append-only calibration anchor, an observed-runs
section in the report, and a regression test asserting the model stays within
2.5x of the measurement — so the next drift is caught by the suite rather than
by a surprising invoice.

Corrected blended figures at 350 PRs/month: ~$1,740 Opus 5, ~$1,755 GPT-5.6
Sol, ~$696 Sonnet 5, ~$348 Haiku 4.5, ~$70 GPT-5.6 Luna.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
2026-08-18 05:08:22 +00:00
Marcos 30d2a3d7da feat(factory): five review skills + a per-review cost model
Skills — the primary now loads conditionally (each one is input tokens), per a
load table in pragent.md:

- attention-tiering: classify every PR trivial/lite/full/oversized BEFORE
  reading anything, and cap file reads, linter runs and subagent fan-out per
  tier. This is the cost governor; the other skills defer to its budget.
- linter-playbook: per-ecosystem detect-and-run commands scoped to changed
  files, the never-install rule, and how to turn a diagnostic into a finding
  instead of pasting tool output.
- security-lens: the inline security checklist for when @security isn't worth
  delegating, built around a source -> sink test each finding must pass.
- malicious-change: hostile-PR detection — injection aimed at the reviewer,
  install/CI-time hooks, obfuscated payloads, dependency confusion, logic
  backdoors. Complements the runtime containment added in the previous commit:
  that stops the agent being hijacked, this makes it report the attempt.
- comment-craft: how to write problem/fix/suggestion so a maintainer can act in
  one read, and what to cut.

pilot/cost_model.py — prices a review against published Claude and OpenAI rates
(fetched 2026-08-18). Prompt sizes are measured from the factory files rather
than guessed; per-tier workloads come from the tiering budgets. The model is
explicit about the thing that actually dominates an agent loop: the whole
conversation is resent every step, so caching moves ~2.3x of the bill.

Blended over a 5/35/55/5 mix with caching on: ~$0.61/PR on Opus 5 or GPT-5.6
Sol, ~$0.24 on Sonnet 5 or Terra, ~$0.12 on Haiku 4.5, ~$0.02 on Luna. At 350
PRs/month that's ~$212 / ~$85 / ~$43 / ~$8.50.

Tests: 101 -> 122.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
2026-08-18 04:53:49 +00:00
Marcos 8c491a7626 harden(pilot): contain hostile PR content, bound the webhook, fix anchoring
The reviewer runs an opencode agent with `bash: "*": allow` over a checkout of
the PR author's branch, and the pod holds a Gitea Write credential. Those two
facts had no wall between them.

Security
- _build_env now allow-lists the subprocess environment instead of inheriting
  it, so PRAGENT_BOT_TOKEN and WEBHOOK_SECRET never reach the agent. This was
  the live hole: a PR body or an AGENTS.md could ask the agent to `curl` the
  token out, and it had both the value and the tool.
- sanitize_workdir deletes author-controlled agent-instruction files from the
  checkout before opencode starts (AGENTS.md at any depth, CLAUDE.md,
  .cursorrules, a repo opencode.json/.opencode, copilot-instructions.md).
  opencode loads nested AGENTS.md as instructions, so a PR could otherwise ship
  its own system prompt. They are still reviewed, as data.
- The brief fences PR title/body and diff in --- UNTRUSTED --- markers under a
  trust-boundary preamble; the pragent agent, the three lens subagents and the
  review-methodology skill now treat injection attempts as a critical finding
  to report rather than an instruction to obey.
- .pr-review.json is read from the PR's base branch, not the head sha. Its
  `instructions` field is spliced into the reviewer's prompt, so head-ref
  reading let any author rewrite the reviewer's rules. Fields are length-capped.
- Untar rejects escaping symlinks, parent traversal, and writes through a
  planted symlink (tar-slip).
- The image runs as uid 10001 instead of root.

Robustness
- Bounded review concurrency (PRAGENT_MAX_CONCURRENT_REVIEWS, default 2). Each
  review forks an opencode process; a thread per delivery was a fork bomb on a
  burst of labels or Gitea retries.
- An in-flight (repo, index, sha) claim closes the check-then-act race in the
  sha-marker dedupe, where two deliveries a second apart both read "not yet
  reviewed" and both posted.
- Request bodies are capped before being read into memory.

Correctness
- parse_diff_anchors counts a whitespace-stripped blank context line. Skipping
  it desynced the new-line counter for the rest of the hunk and silently
  misplaced every later inline comment in that file.
- post_inline_review's body-only fallback folds the anchored findings into the
  body. It previously posted a summary saying "N inline comment(s) below" with
  no comments and no findings — losing them all on the one path that matters.
- fetch_pr_diff's files-endpoint fallback emits real a// b/ prefixes (so
  changed_files and the anchor parser work on it) and reports both HTTP statuses
  in its error instead of the same one twice.
- The CI workflow template pins PRAGENT_ENGINE=ollama; review_pr defaults to
  opencode, which does not exist on a Gitea Actions runner.

Tests: 68 -> 101, covering each of the above.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
2026-08-18 04:44:44 +00:00
gitea_admin ace47d3899 feat(pilot): token-usage reporting gated by AI-USAGE label (#6)
Per-review + per-comment token accounting, surfaced only when a PR carries the AI-USAGE label. opencode --format json NDJSON parsing, attributed output split by finding body weight, webhook denylist, context grounding. 68 tests pass. E2E verified on PR #6.
2026-08-18 04:32:18 +00:00
Marcos d3dcd9c8e5 chore: trigger regression review (AI-USAGE label removed) 2026-08-18 04:23:01 +00:00
Marcos 513de0e3a8 docs(pilot): document AI-USAGE label, token attribution, webhook denylist
- new "AI-USAGE label" section: what gets reported, the attributed-vs-measured
  distinction, PRAGENT_USAGE_ALWAYS, regression-clean without the label.
- new "Webhook fires on any PR update (except closed)" section: denylist
  semantics + why the AI-REVIEW gate + sha dedupe keep it safe.
- arch diagram gate updated (allowlist → denylist; AI-USAGE side-gate).
- opencode engine step 4: --format json + parse_opencode_events → (text, usage).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-18 04:17:32 +00:00
Marcos 087834565d feat(pilot): token-usage reporting gated by AI-USAGE label
Add per-review + per-comment token accounting, surfaced only when a PR carries
the new AI-USAGE label (on top of the existing AI-REVIEW trigger).

opencode_review:
- run_opencode now uses `--format json`; parse_opencode_events reconstructs the
  assistant text from `text` events and sums tokens/cost/steps from every
  `step_finish` event (tolerant of noise / missing fields).
- run() measures duration_s around the opencode call and returns (text, usage).
- changed_files(diff) extracts the `+++ b/` paths; the brief now lists them
  under a "Changed files" focus block so the agent grounds findings in the
  diff's neighbourhood instead of unbounded whole-repo walks.

ai_review:
- format_usage_section renders a `## AI usage` block: measured totals
  (in/out/reasoning/cache/cost/steps/duration), the whole-repo scope note, and
  an attributed per-finding table. Per-comment counts are output tokens split by
  each finding's body weight — labelled "attributed" since one model pass
  produces all findings.
- inline_comment_body appends `🪙 ~N tok (X% · attributed output)` when
  attribution is present.
- review_pr gains report_usage; compute_attribution stashes _tok_attrib/_tok_pct.
- format_review_body inserts the usage section between summary and findings.

webhook_server:
- Fire on every pull_request action except `closed` (denylist, was an allowlist)
  — the AI-REVIEW gate + sha dedupe keep this safe.
- AI-USAGE label detection + PRAGENT_USAGE_ALWAYS env drive report_usage.

.opencode factory + review-methodology skill: new "Ground findings in context"
step — read callers/imports/sibling functions per changed file (1-3 files per
finding), no unbounded walks.

Tests: parse_opencode_events (text+usage sum, malformed tolerance, none-usage),
changed_files, compute_attribution math, inline 🪙 line, format_usage_section
totals/table/cost, format_review_body ordering. 68 passing.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-18 04:15:11 +00:00
Marcos 76b6752f48 fix(review): language-tagged suggestion fence for Gitea syntax highlighting
Gitea 1.26.x has no GitHub-style 'Apply suggestion' button — a ```suggestion
fence is just an unknown-language code block, so chroma does not highlight it
and there is no apply control. Switch inline_comment_body to wrap the suggested
fix in a fence tagged with the file's language (new _lang_for_path helper,
.java→java, .ts→typescript, .py→python, ...), so Gitea syntax-highlights the
code. No capability lost (there was never an apply button on this Gitea
version). Correct the docstrings/skills/README that wrongly claimed an
apply-button was rendered.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-18 01:26:37 +00:00
Marcos 4129f217fa fix(post): anchor inline review comments via new_position (Gitea 1.26.x)
Gitea 1.26.x's POST /pulls/{i}/reviews does NOT honor the line/side fields
used by newer Gitea — it silently drops them, leaving the comment unpositioned.
Gitea then renders a file-level review comment on EVERY diff line of the file,
so a 5-finding review on a 25-line diff showed ~125 comment blocks in the
Files Changed view (the flood reported on canalhandia PR #2).

The 1.26 schema anchors inline review comments with new_position (line in the
post-change file) + old_position: 0. f["line"] is already a validated
post-change (RIGHT-side) line from split_findings, so it maps directly to
new_position. Verified: new_position=98 -> position=98 + populated diff_hunk
(positioned, renders on line 98 only); the old line/side form -> position=0,
empty diff_hunk (unpositioned).

49 tests pass (no test asserted the POST payload shape).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-18 00:54:13 +00:00
Marcos c7c6c3edd2 fix(opencode): resolve /usr/local/bin -> /usr/bin opencode path in pod
The npm prefix in the Dockerfile is /usr, so opencode installs to
/usr/bin/opencode (symlink to /usr/lib/node_modules/...), not
/usr/local/bin/opencode. The cluster E2E failed with ENOENT on
/usr/local/bin/opencode. Fix in three places:
- Dockerfile ENV PRAGENT_OPENCODE_BIN=/usr/bin/opencode
- ~/k8s/pragent-webhook.yaml env value
- _opencode_bin() now defensive: if the configured path is missing,
  falls back to shutil.which('opencode') before the linuxbrew last-resort.
- Dockerfile + README deploy notes: containerd import is sudoless via
  the group-readable raw socket (the microk8s ctr wrapper sudo-wraps).

Verified: rebuilt + reimported + rolled out; PR #5 (sha 985061c0) review
posted in-pod via the opencode engine (findings=2 inline=2 ok=True),
summary + 2 [CRITICAL] inline comments with suggestions + refs + sha marker.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-17 23:31:53 +00:00
Marcos 8912f6721b deploy: combined opencode image + k8s + docs
- pilot/Dockerfile: python:3.12-slim + node 20 + opencode-ai@1.3.10 +
  ripgrep + pyright/typescript-language-server/eslint (npm) + ruff (pip).
  Factory + pilot scripts baked at /app. imagePullPolicy: Never (local
  import into microk8s containerd).
- .dockerignore: lean build context.
- ~/k8s/pragent-webhook.yaml: image pragent-webhook:opencode, emptyDir work
  at /tmp/pragent-work, env (PRAGENT_ENGINE, OPENCODE_MODEL, PRAGENT_FACTORY_DIR,
  PRAGENT_OPENCODE_BIN, PRAGENT_WORK_ROOT, PRAGENT_RTK_DIR='', timeout 540),
  resources 200m/512Mi -> 1500m/2Gi. Drops the ConfigMap scripts volume.
- pilot/README-webhook.md + .opencode/README.md: opencode engine section,
  build/import/apply steps, factory guide fixes (parse_review_output, steps).

Image builds (1.62GB, smoke-tested: opencode+LSPs+factory+env all present).
Import into microk8s containerd + apply pending (ctr needs sudo).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-17 23:11:58 +00:00
Marcos 6e3a9eb5b0 feat: opencode review engine + .opencode factory
Replace the single Python model-call reviewer with an opencode agent
factory. A primary 'pragent' agent reads a brief (title/body/diff/config/
prior reviews), inspects the checked-out repo, runs the repo's own linters
via bash, loads review-methodology + findings-schema skills, and emits a
{summary, findings} JSON with per-finding severity/path/line/problem/fix/
suggestion/reference. Dormant security/tests/perf subagent lenses fan out
only on large/risky diffs (lean by default).

pilot/opencode_review.py: fetches the repo archive at the head sha into a
temp workdir, writes .pragent/brief.md, drops the factory, runs
'opencode run --pure --agent pragent --dir <workdir>' headlessly. Isolates
HOME (shared, warmed), strips ANTHROPIC_* env (leaked host vars caused
ProviderModelNotFoundError), stdin=DEVNULL (opencode blocks on stdin),
maps the bare OLLAMA_MODEL to the provider-prefixed ref. No Gitea I/O —
ai_review.review_pr parses + anchors + posts (reuses all v2 logic/tests).

PRAGENT_ENGINE=opencode (default) selects it; =ollama keeps the legacy
direct-call path. Verified end-to-end: posts a real review with a summary
section, inline [CRITICAL]/[HIGH] comments + apply-able suggestions +
reference links, and the sha dedupe marker. 49 tests pass.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-17 23:01:59 +00:00
Marcos 8758b22802 docs: techspark user webhook (id=6) + org-webhook note + cleanup token
Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-17 20:38:14 +00:00
Marcos 90cea84f6f pilot: dedupe + repo config + inline comments w/ suggestions
- Dedupe: Gitea-as-state. Scan existing reviews for a hidden
  <!-- pragent:sha=... --> marker matching the head sha; skip if present
  (kills duplicate reviews on label-toggle / re-fire). Prior review bodies
  fed back as 'already said' context (light framework §6.1).
- Repo-local focus: optional .pr-review.json at repo root
  ({focus,exclude_paths,languages,instructions}), fetched at head ref.
- Inline comments + apply-able suggestions: model emits JSON findings
  {severity,path,line,problem,fix,suggestion}; diff hunks parsed into valid
  (path,new_line) RIGHT-side anchors; anchored findings become positional
  review comments with a ```suggestion fence (Gitea apply-button);
  unanchored findings fold into the summary body.
- Tests: parse_diff_anchors, parse_findings (tolerant JSON), split_findings,
  inline_comment_body, summary_bullets, parse_repo_config, reviewed_shas,
  prior_review_bodies, sha-marker. 35 pass.
- Bump OLLAMA_MAX_TOKENS default 6000 -> 8000 (suggestions add length).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-17 19:59:36 +00:00
Marcos 789fb38bae pilot: central webhook service (user-level Gitea webhook + AI-REVIEW gate)
- pilot/webhook_server.py: stdlib HTTP receiver. HMAC-verifies X-Gitea-Signature,
  gates on pull_request action + AI-REVIEW label, runs review_pr in a background
  thread (responds 202 immediately so Gitea's delivery timeout never fires).
  Accepts both GitHub-style (labeled/synchronize) and Gitea event-type-style
  (label_updated/synchronized) action names.
- pilot/ai_review.py: extract review_pr() core so both the CI run() and the
  webhook server share one review path. run() is now an env-driven wrapper.
- pilot/README-webhook.md: architecture, onboarding, one-time per-owner
  user-webhook setup, the Gitea 1.26.1 system-webhook bug, the SSRF
  ALLOWED_HOST_LIST change, K8s deploy + script-update recipe.
- README.md + design doc: note the webhook service as the preferred delivery
  path (partially reverses 'central webhook = non-goal', pilot only).

Gitea 1.26.1 system webhooks broken (POST /admin/hooks -> 201 but never
persists); user-level webhooks (one per repo-owner) are the working fallback.
Gitea SSRF allow-list blocks in-cluster webhook targets by default; required a
scoped [webhook] ALLOWED_HOST_LIST addition + gitea restart.

E2E verified 2026-08-17: pragent-bot reviewed gitea_admin/pragent PR #2 and
masi/portfolio PR #3 via the webhook service (glm-5.2:cloud).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-17 19:38:07 +00:00
gitea_admin 9379b52334 retire self-CI AI review workflow (central webhook service takes over) 2026-08-17 19:31:43 +00:00
Marcos 0baefad4ad ci: enable pragent pilot self-review on this repo
Adds .gitea/workflows/ai-review.yml so pragent dogfoods its own pilot: PRs
with the AI-REVIEW label get reviewed by pragent-bot.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-17 18:20:52 +00:00