feat: token-usage reporting (AI-USAGE label) #6
+55
-7
@@ -8,13 +8,13 @@ service, which gates on the `AI-REVIEW` label and runs the same review core.
|
|||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
```
|
```
|
||||||
PR opened/pushed/labeled "AI-REVIEW" (any repo under a covered owner)
|
PR opened/pushed/labeled/edited/… (any repo under a covered owner)
|
||||||
│ Gitea user-level webhook (events: pull_request)
|
│ Gitea user-level webhook (events: pull_request)
|
||||||
▼
|
▼
|
||||||
Service pragent-webhook.pragent.svc.cluster.local (ClusterIP, ns pragent)
|
Service pragent-webhook.pragent.svc.cluster.local (ClusterIP, ns pragent)
|
||||||
│ HMAC-verify (X-Gitea-Signature) → gate: action in {opened,
|
│ HMAC-verify (X-Gitea-Signature) → gate: action ≠ closed
|
||||||
│ reopened, synchronize/synchronized, labeled/label_updated}
|
|
||||||
│ AND pull_request.labels ∋ AI-REVIEW
|
│ AND pull_request.labels ∋ AI-REVIEW
|
||||||
|
│ (report_usage ← pull_request.labels ∋ AI-USAGE, optional)
|
||||||
▼
|
▼
|
||||||
ai_review.review_pr() (same core the CI-step uses)
|
ai_review.review_pr() (same core the CI-step uses)
|
||||||
1. fetch existing reviews → dedupe: skip if a review already carries
|
1. fetch existing reviews → dedupe: skip if a review already carries
|
||||||
@@ -60,6 +60,50 @@ No workflow file, no repo secret, no act-runner needed. (The owner must already
|
|||||||
be covered by a user-level webhook — see below. If not, do the one-time
|
be covered by a user-level webhook — see below. If not, do the one-time
|
||||||
per-owner setup first.)
|
per-owner setup first.)
|
||||||
|
|
||||||
|
## AI-USAGE label — token-usage reporting (optional, opt-in)
|
||||||
|
|
||||||
|
A review always fires on `AI-REVIEW`. Adding a second label **`AI-USAGE`** on
|
||||||
|
the same PR opts the review into appending a token-usage report:
|
||||||
|
|
||||||
|
- a `## 🔋 AI usage` section on the review summary body with the **measured**
|
||||||
|
review total — input / output / reasoning / cache read+write / total tokens,
|
||||||
|
agent step count, wall-clock duration, estimated cost, the model, and a scope
|
||||||
|
note (the agent reviews a whole-repo checkout at the head sha, so input
|
||||||
|
tokens include files read beyond the diff);
|
||||||
|
- a per-finding attribution table (severity · location · ≈out tok · %);
|
||||||
|
- a `🪙 ~N tok (X% · attributed output)` line at the foot of each inline
|
||||||
|
comment.
|
||||||
|
|
||||||
|
**Attributed, not measured.** One opencode agent pass produces *all* findings,
|
||||||
|
so there is no native per-finding token metering. The per-comment / per-row
|
||||||
|
counts are the review's measured **output** tokens split by each finding's
|
||||||
|
rendered-body weight (`len(problem)+len(fix)+len(suggestion)`) — an honest
|
||||||
|
attribution, labelled as such. The totals are real measurements summed from
|
||||||
|
opencode's `step_finish` events.
|
||||||
|
|
||||||
|
`PRAGENT_USAGE_ALWAYS=1` on the Deployment forces usage reporting on for every
|
||||||
|
review (testing / a future default-on) regardless of the label.
|
||||||
|
|
||||||
|
Without `AI-USAGE` (regression): no usage section, no 🪙 lines — behaviour
|
||||||
|
identical to before the feature. The usage section is part of the review body,
|
||||||
|
so it's covered by the existing sha-marker dedupe.
|
||||||
|
|
||||||
|
## Webhook fires on any PR update (except `closed`)
|
||||||
|
|
||||||
|
The receiver uses a **denylist**, not an allowlist: it reviews on every
|
||||||
|
`pull_request` action **except `closed`** — `opened`, `reopened`,
|
||||||
|
`synchronize`/`synchronized`, `labeled`/`label_updated`, `edited` (title/body),
|
||||||
|
`ready_for_review` (draft→ready), `assigned`, `review_requested`, `milestone`,
|
||||||
|
… . This is safe because of two downstream gates:
|
||||||
|
|
||||||
|
- the **AI-REVIEW label gate** — payload `labels` reflect current state, so an
|
||||||
|
`unlabeled` that *removed* AI-REVIEW fails the gate (no review); an
|
||||||
|
`unlabeled` of another label still passes;
|
||||||
|
- the **sha dedupe** — any same-sha re-fire (title edit, assignee, milestone,
|
||||||
|
a label toggle of another label…) is skipped, so the only newly-effective
|
||||||
|
actions are ones that change the head sha (`synchronize`, already covered) or
|
||||||
|
move a draft to ready (`ready_for_review`) on an un-reviewed sha.
|
||||||
|
|
||||||
## Repo-local focus: `.pr-review.json` (optional)
|
## Repo-local focus: `.pr-review.json` (optional)
|
||||||
|
|
||||||
Drop a `.pr-review.json` at the repo root (committed on the PR's branch, or on
|
Drop a `.pr-review.json` at the repo root (committed on the PR's branch, or on
|
||||||
@@ -160,13 +204,17 @@ cramped model call. `pilot/opencode_review.py` is the glue:
|
|||||||
`.pr-review.json`, prior reviews, sha, anchor hint).
|
`.pr-review.json`, prior reviews, sha, anchor hint).
|
||||||
3. `drop_factory` — copies `opencode.json` + `.opencode/` (agents/skills/commands)
|
3. `drop_factory` — copies `opencode.json` + `.opencode/` (agents/skills/commands)
|
||||||
into the workdir as the project config.
|
into the workdir as the project config.
|
||||||
4. `run_opencode` — `opencode run --pure --agent pragent --dir <workdir>
|
4. `run_opencode` — `opencode run --pure --format json --agent pragent
|
||||||
--model headroom/glm-5.2:cloud` headlessly; returns the agent's stdout.
|
--dir <workdir> --model headroom/glm-5.2:cloud` headlessly. `--format json`
|
||||||
|
emits NDJSON events: `parse_opencode_events` reconstructs the assistant text
|
||||||
|
from `text` events and sums tokens/cost/steps from every `step_finish` event.
|
||||||
|
Returns `(text, usage)`.
|
||||||
|
|
||||||
It does **no Gitea I/O and no parsing** — `review_pr` parses the stdout into
|
It does **no Gitea I/O and no parsing** — `review_pr` parses the stdout into
|
||||||
`(summary, findings)`, validates findings against diff anchors, and posts. So
|
`(summary, findings)`, validates findings against diff anchors, and posts. So
|
||||||
all v2 logic (dedupe marker, anchor validation, ```suggestion fencing, posting)
|
all v2 logic (dedupe marker, anchor validation, language-tagged suggestion
|
||||||
is reused and never depends on the model remembering it.
|
fencing, posting, optional AI-USAGE attribution) is reused and never depends on
|
||||||
|
the model remembering it.
|
||||||
|
|
||||||
The factory lives in the pragent repo root: `opencode.json` (provider/model/
|
The factory lives in the pragent repo root: `opencode.json` (provider/model/
|
||||||
permission) + `.opencode/` (agents, skills, commands). It is **both** the
|
permission) + `.opencode/` (agents, skills, commands). It is **both** the
|
||||||
|
|||||||
Reference in New Issue
Block a user