feat(pilot): token-usage reporting gated by AI-USAGE label
Add per-review + per-comment token accounting, surfaced only when a PR carries the new AI-USAGE label (on top of the existing AI-REVIEW trigger). opencode_review: - run_opencode now uses `--format json`; parse_opencode_events reconstructs the assistant text from `text` events and sums tokens/cost/steps from every `step_finish` event (tolerant of noise / missing fields). - run() measures duration_s around the opencode call and returns (text, usage). - changed_files(diff) extracts the `+++ b/` paths; the brief now lists them under a "Changed files" focus block so the agent grounds findings in the diff's neighbourhood instead of unbounded whole-repo walks. ai_review: - format_usage_section renders a `## AI usage` block: measured totals (in/out/reasoning/cache/cost/steps/duration), the whole-repo scope note, and an attributed per-finding table. Per-comment counts are output tokens split by each finding's body weight — labelled "attributed" since one model pass produces all findings. - inline_comment_body appends `🪙 ~N tok (X% · attributed output)` when attribution is present. - review_pr gains report_usage; compute_attribution stashes _tok_attrib/_tok_pct. - format_review_body inserts the usage section between summary and findings. webhook_server: - Fire on every pull_request action except `closed` (denylist, was an allowlist) — the AI-REVIEW gate + sha dedupe keep this safe. - AI-USAGE label detection + PRAGENT_USAGE_ALWAYS env drive report_usage. .opencode factory + review-methodology skill: new "Ground findings in context" step — read callers/imports/sibling functions per changed file (1-3 files per finding), no unbounded walks. Tests: parse_opencode_events (text+usage sum, malformed tolerance, none-usage), changed_files, compute_attribution math, inline 🪙 line, format_usage_section totals/table/cost, format_review_body ordering. 68 passing. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -51,9 +51,19 @@ read the full file around a flagged line, not just the diff hunk.
|
||||
|
||||
2. **Map the change.** Skim the diff. Note the changed paths, the languages, and
|
||||
whether the change touches security-sensitive areas (auth, crypto, SQL, file
|
||||
I/O, deserialization, CI/supply-chain, secrets).
|
||||
I/O, deserialization, CI/supply-chain, secrets). The brief lists the changed
|
||||
files explicitly under "Changed files" — use that as your focus list.
|
||||
|
||||
3. **Run the repo's own checks via bash.** Detect tooling and run it on the
|
||||
3. **Ground findings in context.** For each changed file, before finalizing any
|
||||
finding, `read`/`grep` its **callers, imports, sibling functions, and type
|
||||
definitions** so your findings reflect how the change is actually used, not
|
||||
the hunk in isolation. The repo is checked out at the head sha, so the
|
||||
surrounding code is on disk — use it. Keep it bounded: stop exploring a file
|
||||
once the finding is grounded (1–3 related files per finding); do NOT do
|
||||
unbounded whole-repo walks (token cost, and the focus is the diff's
|
||||
neighbourhood).
|
||||
|
||||
4. **Run the repo's own checks via bash.** Detect tooling and run it on the
|
||||
CHANGED files only (keep it fast, keep tokens low):
|
||||
- TS/JS: `npx --no-install tsc --noEmit` if `tsconfig.json` exists; `npx --no-install eslint <changed>` if configured.
|
||||
- Python: `ruff check <changed>` or `python -m pyright <changed>` / `mypy` if configured.
|
||||
@@ -62,18 +72,19 @@ read the full file around a flagged line, not just the diff hunk.
|
||||
- Never run install/build steps (`npm install`, `go mod download`, etc.) — too slow / too much output. If a check needs deps that aren't installed, skip it and note that.
|
||||
- Capture only diagnostics (errors/warnings), not success prose.
|
||||
|
||||
4. **Find real issues.** Combine: the diff, the surrounding code you read, and the
|
||||
linter/typecheck diagnostics. Report ONLY real, actionable issues — correctness
|
||||
bugs, security problems, risky changes, missing tests for changed behavior,
|
||||
breaking API/contract changes. Skip praise, nitpicks, pure formatting.
|
||||
5. **Find real issues.** Combine: the diff, the surrounding context you read in
|
||||
step 3, and the linter/typecheck diagnostics. Report ONLY real, actionable
|
||||
issues — correctness bugs, security problems, risky changes, missing tests
|
||||
for changed behavior, breaking API/contract changes. Skip praise, nitpicks,
|
||||
pure formatting.
|
||||
|
||||
5. **References.** When a finding involves a specific library API, known
|
||||
6. **References.** When a finding involves a specific library API, known
|
||||
vulnerability, or footgun, use `webfetch` to confirm it (e.g. a CVE page, the
|
||||
library docs) and put the URL in the finding's `reference` field. Leave
|
||||
`reference` empty when there's nothing authoritative to link. Don't fetch for
|
||||
the sake of it — keep it lean.
|
||||
|
||||
6. **Delegate on heavy diffs.** If the diff is large (>~400 changed lines) OR
|
||||
7. **Delegate on heavy diffs.** If the diff is large (>~400 changed lines) OR
|
||||
touches auth/crypto/SQL/deserialization/CI, delegate that lens to a subagent
|
||||
via the Task tool:
|
||||
- `@security` — injection, auth, secrets, supply-chain, unsafe deserialization.
|
||||
@@ -83,7 +94,7 @@ read the full file around a flagged line, not just the diff hunk.
|
||||
keep highest severity). For small/medium diffs, do all lenses inline yourself —
|
||||
do NOT spawn subagents. Cost must scale with PR size.
|
||||
|
||||
7. **Anchor every finding.** Each finding's `line` MUST be a line that exists in
|
||||
8. **Anchor every finding.** Each finding's `line` MUST be a line that exists in
|
||||
the POST-CHANGE version of `path` — a context line or an added `+` line shown
|
||||
in the diff. Never a removed line. If unsure, use the closest context line you
|
||||
can see in the diff. A finding with a bad line gets folded into the summary as
|
||||
|
||||
@@ -41,6 +41,14 @@ the diff. A misanchored finding becomes a summary bullet instead of an inline
|
||||
comment, so correct anchoring is what gets a finding shown inline with its
|
||||
suggested-fix code block (language-highlighted) rather than demoted to a bullet.
|
||||
|
||||
## Ground each finding in context
|
||||
|
||||
Don't flag a hunk in isolation. For each changed file, read its callers,
|
||||
imports, sibling functions, and type definitions (the repo is checked out at
|
||||
the head sha), and make sure the finding holds against how the change is
|
||||
actually used. Keep it bounded — 1–3 related files per finding, no unbounded
|
||||
whole-repo walks.
|
||||
|
||||
## Honoring repo config
|
||||
|
||||
If `.pr-review.json` is present, honor it:
|
||||
|
||||
Reference in New Issue
Block a user