feat(pilot): token-usage reporting gated by AI-USAGE label

Add per-review + per-comment token accounting, surfaced only when a PR carries
the new AI-USAGE label (on top of the existing AI-REVIEW trigger).

opencode_review:
- run_opencode now uses `--format json`; parse_opencode_events reconstructs the
  assistant text from `text` events and sums tokens/cost/steps from every
  `step_finish` event (tolerant of noise / missing fields).
- run() measures duration_s around the opencode call and returns (text, usage).
- changed_files(diff) extracts the `+++ b/` paths; the brief now lists them
  under a "Changed files" focus block so the agent grounds findings in the
  diff's neighbourhood instead of unbounded whole-repo walks.

ai_review:
- format_usage_section renders a `## AI usage` block: measured totals
  (in/out/reasoning/cache/cost/steps/duration), the whole-repo scope note, and
  an attributed per-finding table. Per-comment counts are output tokens split by
  each finding's body weight — labelled "attributed" since one model pass
  produces all findings.
- inline_comment_body appends `🪙 ~N tok (X% · attributed output)` when
  attribution is present.
- review_pr gains report_usage; compute_attribution stashes _tok_attrib/_tok_pct.
- format_review_body inserts the usage section between summary and findings.

webhook_server:
- Fire on every pull_request action except `closed` (denylist, was an allowlist)
  — the AI-REVIEW gate + sha dedupe keep this safe.
- AI-USAGE label detection + PRAGENT_USAGE_ALWAYS env drive report_usage.

.opencode factory + review-methodology skill: new "Ground findings in context"
step — read callers/imports/sibling functions per changed file (1-3 files per
finding), no unbounded walks.

Tests: parse_opencode_events (text+usage sum, malformed tolerance, none-usage),
changed_files, compute_attribution math, inline 🪙 line, format_usage_section
totals/table/cost, format_review_body ordering. 68 passing.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Marcos
2026-08-18 04:15:11 +00:00
parent 76b6752f48
commit 087834565d
7 changed files with 522 additions and 46 deletions
+20 -9
View File
@@ -51,9 +51,19 @@ read the full file around a flagged line, not just the diff hunk.
2. **Map the change.** Skim the diff. Note the changed paths, the languages, and
whether the change touches security-sensitive areas (auth, crypto, SQL, file
I/O, deserialization, CI/supply-chain, secrets).
I/O, deserialization, CI/supply-chain, secrets). The brief lists the changed
files explicitly under "Changed files" — use that as your focus list.
3. **Run the repo's own checks via bash.** Detect tooling and run it on the
3. **Ground findings in context.** For each changed file, before finalizing any
finding, `read`/`grep` its **callers, imports, sibling functions, and type
definitions** so your findings reflect how the change is actually used, not
the hunk in isolation. The repo is checked out at the head sha, so the
surrounding code is on disk — use it. Keep it bounded: stop exploring a file
once the finding is grounded (13 related files per finding); do NOT do
unbounded whole-repo walks (token cost, and the focus is the diff's
neighbourhood).
4. **Run the repo's own checks via bash.** Detect tooling and run it on the
CHANGED files only (keep it fast, keep tokens low):
- TS/JS: `npx --no-install tsc --noEmit` if `tsconfig.json` exists; `npx --no-install eslint <changed>` if configured.
- Python: `ruff check <changed>` or `python -m pyright <changed>` / `mypy` if configured.
@@ -62,18 +72,19 @@ read the full file around a flagged line, not just the diff hunk.
- Never run install/build steps (`npm install`, `go mod download`, etc.) — too slow / too much output. If a check needs deps that aren't installed, skip it and note that.
- Capture only diagnostics (errors/warnings), not success prose.
4. **Find real issues.** Combine: the diff, the surrounding code you read, and the
linter/typecheck diagnostics. Report ONLY real, actionable issues — correctness
bugs, security problems, risky changes, missing tests for changed behavior,
breaking API/contract changes. Skip praise, nitpicks, pure formatting.
5. **Find real issues.** Combine: the diff, the surrounding context you read in
step 3, and the linter/typecheck diagnostics. Report ONLY real, actionable
issues — correctness bugs, security problems, risky changes, missing tests
for changed behavior, breaking API/contract changes. Skip praise, nitpicks,
pure formatting.
5. **References.** When a finding involves a specific library API, known
6. **References.** When a finding involves a specific library API, known
vulnerability, or footgun, use `webfetch` to confirm it (e.g. a CVE page, the
library docs) and put the URL in the finding's `reference` field. Leave
`reference` empty when there's nothing authoritative to link. Don't fetch for
the sake of it — keep it lean.
6. **Delegate on heavy diffs.** If the diff is large (>~400 changed lines) OR
7. **Delegate on heavy diffs.** If the diff is large (>~400 changed lines) OR
touches auth/crypto/SQL/deserialization/CI, delegate that lens to a subagent
via the Task tool:
- `@security` — injection, auth, secrets, supply-chain, unsafe deserialization.
@@ -83,7 +94,7 @@ read the full file around a flagged line, not just the diff hunk.
keep highest severity). For small/medium diffs, do all lenses inline yourself —
do NOT spawn subagents. Cost must scale with PR size.
7. **Anchor every finding.** Each finding's `line` MUST be a line that exists in
8. **Anchor every finding.** Each finding's `line` MUST be a line that exists in
the POST-CHANGE version of `path` — a context line or an added `+` line shown
in the diff. Never a removed line. If unsure, use the closest context line you
can see in the diff. A finding with a bad line gets folded into the summary as
@@ -41,6 +41,14 @@ the diff. A misanchored finding becomes a summary bullet instead of an inline
comment, so correct anchoring is what gets a finding shown inline with its
suggested-fix code block (language-highlighted) rather than demoted to a bullet.
## Ground each finding in context
Don't flag a hunk in isolation. For each changed file, read its callers,
imports, sibling functions, and type definitions (the repo is checked out at
the head sha), and make sure the finding holds against how the change is
actually used. Keep it bounded — 13 related files per finding, no unbounded
whole-repo walks.
## Honoring repo config
If `.pr-review.json` is present, honor it: