feat(pilot): token-usage reporting gated by AI-USAGE label

Add per-review + per-comment token accounting, surfaced only when a PR carries
the new AI-USAGE label (on top of the existing AI-REVIEW trigger).

opencode_review:
- run_opencode now uses `--format json`; parse_opencode_events reconstructs the
  assistant text from `text` events and sums tokens/cost/steps from every
  `step_finish` event (tolerant of noise / missing fields).
- run() measures duration_s around the opencode call and returns (text, usage).
- changed_files(diff) extracts the `+++ b/` paths; the brief now lists them
  under a "Changed files" focus block so the agent grounds findings in the
  diff's neighbourhood instead of unbounded whole-repo walks.

ai_review:
- format_usage_section renders a `## AI usage` block: measured totals
  (in/out/reasoning/cache/cost/steps/duration), the whole-repo scope note, and
  an attributed per-finding table. Per-comment counts are output tokens split by
  each finding's body weight — labelled "attributed" since one model pass
  produces all findings.
- inline_comment_body appends `🪙 ~N tok (X% · attributed output)` when
  attribution is present.
- review_pr gains report_usage; compute_attribution stashes _tok_attrib/_tok_pct.
- format_review_body inserts the usage section between summary and findings.

webhook_server:
- Fire on every pull_request action except `closed` (denylist, was an allowlist)
  — the AI-REVIEW gate + sha dedupe keep this safe.
- AI-USAGE label detection + PRAGENT_USAGE_ALWAYS env drive report_usage.

.opencode factory + review-methodology skill: new "Ground findings in context"
step — read callers/imports/sibling functions per changed file (1-3 files per
finding), no unbounded walks.

Tests: parse_opencode_events (text+usage sum, malformed tolerance, none-usage),
changed_files, compute_attribution math, inline 🪙 line, format_usage_section
totals/table/cost, format_review_body ordering. 68 passing.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Marcos
2026-08-18 04:15:11 +00:00
parent 76b6752f48
commit 087834565d
7 changed files with 522 additions and 46 deletions
+34 -15
View File
@@ -37,13 +37,17 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from ai_review import review_pr
# Pull-request webhook `action` values worth reviewing on. Gitea emits
# GitHub-style payload `action` names (`labeled`, `synchronize`) even though the
# `X-Gitea-Event-Type` header uses `label_updated` / `synchronized` — accept both
# so the gate is robust to either. The label gate below means a non-AI-REVIEW
# label update is a no-op.
REVIEW_ACTIONS = {"opened", "reopened", "synchronize", "synchronized", "labeled", "label_updated"}
# Pull-request webhook `action` values. We fire on EVERY pull_request action
# except `closed` (no point reviewing a closed/merged PR) — the AI-REVIEW label
# gate + sha dedupe downstream make broadening safe: a same-sha re-fire (title
# edit, assignee, milestone, label toggle of another label…) is skipped by
# `review_pr`'s dedupe, and an `unlabeled` event that removed AI-REVIEW fails
# the label gate (payload `labels` reflect current state). Gitea emits
# GitHub-style `action` names (`labeled`, `synchronize`) even though the
# `X-Gitea-Event-Type` header uses `label_updated` / `synchronized`.
SKIP_ACTIONS = {"closed"}
AI_REVIEW_LABEL = "AI-REVIEW"
AI_USAGE_LABEL = "AI-USAGE"
GITEA_API = os.environ.get("GITEA_API", "http://gitea-http.gitea.svc.cluster.local:3000")
BOT_TOKEN = os.environ.get("PRAGENT_BOT_TOKEN", "")
@@ -55,17 +59,23 @@ WEBHOOK_SECRET = os.environ.get("WEBHOOK_SECRET", "").encode()
PORT = int(os.environ.get("WEBHOOK_PORT", "8080"))
def _labels_have_ai_review(labels) -> bool:
def _labels_have(labels, name: str) -> bool:
"""True if the Gitea PR `labels` list (dicts with `name`, or bare strings)
contains `name`."""
if not isinstance(labels, list):
return False
for lab in labels:
if isinstance(lab, dict) and lab.get("name") == AI_REVIEW_LABEL:
if isinstance(lab, dict) and lab.get("name") == name:
return True
if isinstance(lab, str) and lab == AI_REVIEW_LABEL:
if isinstance(lab, str) and lab == name:
return True
return False
def _labels_have_ai_review(labels) -> bool:
return _labels_have(labels, AI_REVIEW_LABEL)
def _verify_signature(raw_body: bytes, headers) -> bool:
if not WEBHOOK_SECRET:
return False # refuse to run without a configured secret
@@ -87,12 +97,13 @@ def _handle_pull_request(payload: dict) -> tuple[int, str]:
repo_obj = payload.get("repository") or {}
repo = repo_obj.get("full_name") or ""
if action not in REVIEW_ACTIONS:
if action in SKIP_ACTIONS:
return 200, f"ignore action={action}"
if not repo:
return 400, "no repository.full_name"
if not _labels_have_ai_review(pr.get("labels")):
labels = pr.get("labels")
if not _labels_have_ai_review(labels):
return 200, f"ignore (no {AI_REVIEW_LABEL} label) action={action}"
index = pr.get("number")
@@ -106,15 +117,22 @@ def _handle_pull_request(payload: dict) -> tuple[int, str]:
if not BOT_TOKEN:
return 500, "PRAGENT_BOT_TOKEN not set"
# AI-USAGE label (opt-in) → append the token-usage section + per-comment 🪙
# lines to the review. PRAGENT_USAGE_ALWAYS forces it on for testing / a
# future default-on.
report_usage = _labels_have(labels, AI_USAGE_LABEL) or bool(
os.environ.get("PRAGENT_USAGE_ALWAYS")
)
threading.Thread(
target=_run_review,
args=(repo, str(index), title, body, sha),
args=(repo, str(index), title, body, sha, report_usage),
daemon=True,
).start()
return 202, f"reviewing {repo}#{index} action={action} sha={sha[:8]}"
return 202, f"reviewing {repo}#{index} action={action} sha={sha[:8]} usage={report_usage}"
def _run_review(repo: str, index: str, title: str, body: str, sha: str) -> None:
def _run_review(repo: str, index: str, title: str, body: str, sha: str, report_usage: bool) -> None:
try:
ok = review_pr(
api=GITEA_API,
@@ -128,8 +146,9 @@ def _run_review(repo: str, index: str, title: str, body: str, sha: str) -> None:
model=OLLAMA_MODEL,
max_tokens=OLLAMA_MAX_TOKENS,
max_chars=DIFF_MAX_CHARS,
report_usage=report_usage,
)
print(f"pragent-webhook: reviewed {repo}#{index} sha={sha[:8]} ok={ok}", flush=True)
print(f"pragent-webhook: reviewed {repo}#{index} sha={sha[:8]} ok={ok} usage={report_usage}", flush=True)
except Exception as e: # review_pr is fail-open, but guard the thread anyway
print(f"pragent-webhook: thread crashed for {repo}#{index}: {e}", flush=True)