feat: token-usage reporting (AI-USAGE label) #6

Merged
gitea_admin merged 3 commits from feat/token-usage into main 2026-08-18 04:32:19 +00:00
Showing only changes of commit 513de0e3a8 - Show all commits
+55 -7
View File
@@ -8,13 +8,13 @@ service, which gates on the `AI-REVIEW` label and runs the same review core.
## Architecture
```
PR opened/pushed/labeled "AI-REVIEW" (any repo under a covered owner)
PR opened/pushed/labeled/edited/… (any repo under a covered owner)
│ Gitea user-level webhook (events: pull_request)
Service pragent-webhook.pragent.svc.cluster.local (ClusterIP, ns pragent)
│ HMAC-verify (X-Gitea-Signature) → gate: action in {opened,
│ reopened, synchronize/synchronized, labeled/label_updated}
│ HMAC-verify (X-Gitea-Signature) → gate: action ≠ closed
│ AND pull_request.labels ∋ AI-REVIEW
│ (report_usage ← pull_request.labels ∋ AI-USAGE, optional)
ai_review.review_pr() (same core the CI-step uses)
1. fetch existing reviews → dedupe: skip if a review already carries
@@ -60,6 +60,50 @@ No workflow file, no repo secret, no act-runner needed. (The owner must already
be covered by a user-level webhook — see below. If not, do the one-time
per-owner setup first.)
## AI-USAGE label — token-usage reporting (optional, opt-in)
A review always fires on `AI-REVIEW`. Adding a second label **`AI-USAGE`** on
the same PR opts the review into appending a token-usage report:
- a `## 🔋 AI usage` section on the review summary body with the **measured**
review total — input / output / reasoning / cache read+write / total tokens,
agent step count, wall-clock duration, estimated cost, the model, and a scope
note (the agent reviews a whole-repo checkout at the head sha, so input
tokens include files read beyond the diff);
- a per-finding attribution table (severity · location · ≈out tok · %);
- a `🪙 ~N tok (X% · attributed output)` line at the foot of each inline
comment.
**Attributed, not measured.** One opencode agent pass produces *all* findings,
so there is no native per-finding token metering. The per-comment / per-row
counts are the review's measured **output** tokens split by each finding's
rendered-body weight (`len(problem)+len(fix)+len(suggestion)`) — an honest
attribution, labelled as such. The totals are real measurements summed from
opencode's `step_finish` events.
`PRAGENT_USAGE_ALWAYS=1` on the Deployment forces usage reporting on for every
review (testing / a future default-on) regardless of the label.
Without `AI-USAGE` (regression): no usage section, no 🪙 lines — behaviour
identical to before the feature. The usage section is part of the review body,
so it's covered by the existing sha-marker dedupe.
## Webhook fires on any PR update (except `closed`)
The receiver uses a **denylist**, not an allowlist: it reviews on every
`pull_request` action **except `closed`**`opened`, `reopened`,
`synchronize`/`synchronized`, `labeled`/`label_updated`, `edited` (title/body),
`ready_for_review` (draft→ready), `assigned`, `review_requested`, `milestone`,
… . This is safe because of two downstream gates:
- the **AI-REVIEW label gate** — payload `labels` reflect current state, so an
`unlabeled` that *removed* AI-REVIEW fails the gate (no review); an
`unlabeled` of another label still passes;
- the **sha dedupe** — any same-sha re-fire (title edit, assignee, milestone,
a label toggle of another label…) is skipped, so the only newly-effective
actions are ones that change the head sha (`synchronize`, already covered) or
move a draft to ready (`ready_for_review`) on an un-reviewed sha.
## Repo-local focus: `.pr-review.json` (optional)
Drop a `.pr-review.json` at the repo root (committed on the PR's branch, or on
@@ -160,13 +204,17 @@ cramped model call. `pilot/opencode_review.py` is the glue:
`.pr-review.json`, prior reviews, sha, anchor hint).
3. `drop_factory` — copies `opencode.json` + `.opencode/` (agents/skills/commands)
into the workdir as the project config.
4. `run_opencode` — `opencode run --pure --agent pragent --dir <workdir>
--model headroom/glm-5.2:cloud` headlessly; returns the agent's stdout.
4. `run_opencode` — `opencode run --pure --format json --agent pragent
--dir <workdir> --model headroom/glm-5.2:cloud` headlessly. `--format json`
emits NDJSON events: `parse_opencode_events` reconstructs the assistant text
from `text` events and sums tokens/cost/steps from every `step_finish` event.
Returns `(text, usage)`.
It does **no Gitea I/O and no parsing** — `review_pr` parses the stdout into
`(summary, findings)`, validates findings against diff anchors, and posts. So
all v2 logic (dedupe marker, anchor validation, ```suggestion fencing, posting)
is reused and never depends on the model remembering it.
all v2 logic (dedupe marker, anchor validation, language-tagged suggestion
fencing, posting, optional AI-USAGE attribution) is reused and never depends on
the model remembering it.
The factory lives in the pragent repo root: `opencode.json` (provider/model/
permission) + `.opencode/` (agents, skills, commands). It is **both** the