feat(pilot): token-usage reporting gated by AI-USAGE label

Add per-review + per-comment token accounting, surfaced only when a PR carries
the new AI-USAGE label (on top of the existing AI-REVIEW trigger).

opencode_review:
- run_opencode now uses `--format json`; parse_opencode_events reconstructs the
  assistant text from `text` events and sums tokens/cost/steps from every
  `step_finish` event (tolerant of noise / missing fields).
- run() measures duration_s around the opencode call and returns (text, usage).
- changed_files(diff) extracts the `+++ b/` paths; the brief now lists them
  under a "Changed files" focus block so the agent grounds findings in the
  diff's neighbourhood instead of unbounded whole-repo walks.

ai_review:
- format_usage_section renders a `## AI usage` block: measured totals
  (in/out/reasoning/cache/cost/steps/duration), the whole-repo scope note, and
  an attributed per-finding table. Per-comment counts are output tokens split by
  each finding's body weight — labelled "attributed" since one model pass
  produces all findings.
- inline_comment_body appends `🪙 ~N tok (X% · attributed output)` when
  attribution is present.
- review_pr gains report_usage; compute_attribution stashes _tok_attrib/_tok_pct.
- format_review_body inserts the usage section between summary and findings.

webhook_server:
- Fire on every pull_request action except `closed` (denylist, was an allowlist)
  — the AI-REVIEW gate + sha dedupe keep this safe.
- AI-USAGE label detection + PRAGENT_USAGE_ALWAYS env drive report_usage.

.opencode factory + review-methodology skill: new "Ground findings in context"
step — read callers/imports/sibling functions per changed file (1-3 files per
finding), no unbounded walks.

Tests: parse_opencode_events (text+usage sum, malformed tolerance, none-usage),
changed_files, compute_attribution math, inline 🪙 line, format_usage_section
totals/table/cost, format_review_body ordering. 68 passing.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Marcos
2026-08-18 04:15:11 +00:00
parent 76b6752f48
commit 087834565d
7 changed files with 522 additions and 46 deletions
+20 -9
View File
@@ -51,9 +51,19 @@ read the full file around a flagged line, not just the diff hunk.
2. **Map the change.** Skim the diff. Note the changed paths, the languages, and
whether the change touches security-sensitive areas (auth, crypto, SQL, file
I/O, deserialization, CI/supply-chain, secrets).
I/O, deserialization, CI/supply-chain, secrets). The brief lists the changed
files explicitly under "Changed files" — use that as your focus list.
3. **Run the repo's own checks via bash.** Detect tooling and run it on the
3. **Ground findings in context.** For each changed file, before finalizing any
finding, `read`/`grep` its **callers, imports, sibling functions, and type
definitions** so your findings reflect how the change is actually used, not
the hunk in isolation. The repo is checked out at the head sha, so the
surrounding code is on disk — use it. Keep it bounded: stop exploring a file
once the finding is grounded (13 related files per finding); do NOT do
unbounded whole-repo walks (token cost, and the focus is the diff's
neighbourhood).
4. **Run the repo's own checks via bash.** Detect tooling and run it on the
CHANGED files only (keep it fast, keep tokens low):
- TS/JS: `npx --no-install tsc --noEmit` if `tsconfig.json` exists; `npx --no-install eslint <changed>` if configured.
- Python: `ruff check <changed>` or `python -m pyright <changed>` / `mypy` if configured.
@@ -62,18 +72,19 @@ read the full file around a flagged line, not just the diff hunk.
- Never run install/build steps (`npm install`, `go mod download`, etc.) — too slow / too much output. If a check needs deps that aren't installed, skip it and note that.
- Capture only diagnostics (errors/warnings), not success prose.
4. **Find real issues.** Combine: the diff, the surrounding code you read, and the
linter/typecheck diagnostics. Report ONLY real, actionable issues — correctness
bugs, security problems, risky changes, missing tests for changed behavior,
breaking API/contract changes. Skip praise, nitpicks, pure formatting.
5. **Find real issues.** Combine: the diff, the surrounding context you read in
step 3, and the linter/typecheck diagnostics. Report ONLY real, actionable
issues — correctness bugs, security problems, risky changes, missing tests
for changed behavior, breaking API/contract changes. Skip praise, nitpicks,
pure formatting.
5. **References.** When a finding involves a specific library API, known
6. **References.** When a finding involves a specific library API, known
vulnerability, or footgun, use `webfetch` to confirm it (e.g. a CVE page, the
library docs) and put the URL in the finding's `reference` field. Leave
`reference` empty when there's nothing authoritative to link. Don't fetch for
the sake of it — keep it lean.
6. **Delegate on heavy diffs.** If the diff is large (>~400 changed lines) OR
7. **Delegate on heavy diffs.** If the diff is large (>~400 changed lines) OR
touches auth/crypto/SQL/deserialization/CI, delegate that lens to a subagent
via the Task tool:
- `@security` — injection, auth, secrets, supply-chain, unsafe deserialization.
@@ -83,7 +94,7 @@ read the full file around a flagged line, not just the diff hunk.
keep highest severity). For small/medium diffs, do all lenses inline yourself —
do NOT spawn subagents. Cost must scale with PR size.
7. **Anchor every finding.** Each finding's `line` MUST be a line that exists in
8. **Anchor every finding.** Each finding's `line` MUST be a line that exists in
the POST-CHANGE version of `path` — a context line or an added `+` line shown
in the diff. Never a removed line. If unsure, use the closest context line you
can see in the diff. A finding with a bad line gets folded into the summary as