pragent/pilot/README-webhook.md

pragent pilot — central webhook service

The CI-step pilot (pilot/README.md) needs a workflow file + secret + label per repo. The central webhook service removes the workflow file, the secret, and the runner dependency: a Gitea webhook posts PR events to an always-on in-cluster service, which gates on the AI-REVIEW label and runs the same review core.

Architecture

PR opened/pushed/labeled/edited/…  (any repo under a covered owner)
        │  Gitea user-level webhook (events: pull_request)
        ▼
Service pragent-webhook.pragent.svc.cluster.local  (ClusterIP, ns pragent)
        │  HMAC-verify (X-Gitea-Signature) → gate: action ≠ closed
        │  AND pull_request.labels ∋ AI-REVIEW
        │  (report_usage ← pull_request.labels ∋ AI-USAGE, optional)
        ▼
ai_review.review_pr()  (same core the CI-step uses)
   1. fetch existing reviews → dedupe: skip if a review already carries
      <!-- pragent:sha=<this sha> -->  (no duplicate on label-toggle / re-fire)
   2. fetch PR diff     → GET .../pulls/{i}.diff
   3. fetch .pr-review.json @ head ref (optional repo-local focus/config)
   4. prior review bodies → fed as "already said" context (light §6.1)
   5. PRAGENT_ENGINE=opencode (default):
      a. fetch repo archive @ head sha → /tmp/pragent-work/<repo>-<sha>
      b. write .pragent/brief.md (title/body/diff/config/prior/sha/anchor-hint)
      c. drop the factory (opencode.json + .opencode/) into the workdir
      d. opencode run --pure --agent pragent --dir <workdir> --model headroom/glm-5.2:cloud
         → the pragent agent reads the brief, inspects the repo, runs the
         repo's own linters via bash, loads review-methodology + findings-schema
         skills, delegates to security/tests/perf subagents only on big/risky
         diffs, and emits: {"summary":..., "findings":[{severity,path,line,
         problem,fix,suggestion,reference}]}
      (=ollama: legacy single POST to http://100.74.17.70:8789/v1/messages)
   6. parse diff hunks → valid (path, new_line) anchors (RIGHT side)
   7. post review       → POST .../pulls/{i}/reviews  (event: COMMENT) as pragent-bot
      - prose summary → review body intro
      - anchored findings → inline line comments, body wraps `suggestion` in a
        language-tagged fenced code block (Gitea syntax-highlights it; Gitea
        1.26.x has no apply-suggestion button); reference → 📎 ref link
      - unanchored findings → summary-body bullets
      - summary body carries the <!-- pragent:sha=... --> marker for dedupe

Fail-open. No duplicate per commit (dedupe). Inline comments + syntax-highlighted suggested-fix blocks where the line anchors cleanly. Repo-local focus via .pr-review.json. Prior reviews fed as context so re-pushes synthesize instead of repeating (light version of framework §6.1).

What "onboarding a repo" means now

1. Add pragent-bot as collaborator with Write (so it can read the diff and post the review). The bot stays a normal user — it is not a site admin.
2. Create the AI-REVIEW label on the repo (one-time; pragent-bot's write:issue scope can do it once it's a collaborator).
3. Label a PR AI-REVIEW.

No workflow file, no repo secret, no act-runner needed. (The owner must already be covered by a user-level webhook — see below. If not, do the one-time per-owner setup first.)

AI-USAGE label — token-usage reporting (optional, opt-in)

A review always fires on AI-REVIEW. Adding a second label AI-USAGE on the same PR opts the review into appending a token-usage report:

• a ## 🔋 AI usage section on the review summary body with the measured review total — input / output / reasoning / cache read+write / total tokens, agent step count, wall-clock duration, estimated cost, the model, and a scope note (the agent reviews a whole-repo checkout at the head sha, so input tokens include files read beyond the diff);
• a per-finding attribution table (severity · location · ≈out tok · %);
• a 🪙 ~N tok (X% · attributed output) line at the foot of each inline comment.

Attributed, not measured. One opencode agent pass produces all findings, so there is no native per-finding token metering. The per-comment / per-row counts are the review's measured output tokens split by each finding's rendered-body weight (len(problem)+len(fix)+len(suggestion)) — an honest attribution, labelled as such. The totals are real measurements summed from opencode's step_finish events.

PRAGENT_USAGE_ALWAYS=1 on the Deployment forces usage reporting on for every review (testing / a future default-on) regardless of the label.

Without AI-USAGE (regression): no usage section, no 🪙 lines — behaviour identical to before the feature. The usage section is part of the review body, so it's covered by the existing sha-marker dedupe.

Webhook fires on any PR update (except closed)

The receiver uses a denylist, not an allowlist: it reviews on every pull_request action except closed — opened, reopened, synchronize/synchronized, labeled/label_updated, edited (title/body), ready_for_review (draft→ready), assigned, review_requested, milestone, … . This is safe because of two downstream gates:

• the AI-REVIEW label gate — payload labels reflect current state, so an unlabeled that removed AI-REVIEW fails the gate (no review); an unlabeled of another label still passes;
• the sha dedupe — any same-sha re-fire (title edit, assignee, milestone, a label toggle of another label…) is skipped, so the only newly-effective actions are ones that change the head sha (synchronize, already covered) or move a draft to ready (ready_for_review) on an un-reviewed sha.

Repo-local focus: .pr-review.json (optional)

Drop a .pr-review.json at the repo root (committed on the PR's branch, or on the default branch) to steer the review for that repo. All fields optional; absent file = defaults. JSON (stdlib, no YAML dependency).

{
  "focus": ["security", "supply-chain", "sql-injection"],
  "exclude_paths": ["vendor/**", "**/*.generated.ts"],
  "languages": ["typescript", "go"],
  "instructions": "We use Result<T,E> for error handling. Flag any bare throw. Flag eval()/exec() on user input as critical."
}

• focus — weight these review areas higher (does not blind the reviewer to critical issues outside them).
• exclude_paths — tell the model to ignore these paths.
• languages — hint the primary languages.
• instructions — free-form house conventions / compliance language.

Fetched at review time from the PR head ref (GET /repos/{o}/{r}/contents/.pr-review.json?ref=<head sha>). Bad/missing file fails open to defaults. The bot's read:repository scope reads it.

One-time per-owner setup: register a user-level webhook

Gitea system webhooks (one webhook for the whole instance — the ideal) are broken in Gitea 1.26.1: POST /admin/hooks returns 201 but the hook never persists (GET /admin/hooks lists 0, no delivery). So we use user-level webhooks instead — one webhook per repo-owner, which fires for every repo that user owns. For a small instance with few owners this is nearly as good.

To onboard a new owner (e.g. masi):

# 1. generate a one-time token for that user (admin CLI, inside the gitea pod)
K="microk8s kubectl"
GPOD=$($K -n gitea get pod -l app=gitea --field-selector=status.phase=Running \
  -o jsonpath='{.items[?(@.status.containerStatuses[0].ready==true)].metadata.name}')
$K -n gitea exec "$GPOD" -c gitea -- \
  gitea admin user generate-access-token --username masi \
  --scopes write:user,read:user --token-name pragent-userhook-masi

# 2. register the user-level webhook (events: pull_request)
#    WEBHOOK_SECRET = the shared HMAC secret in the pragent-webhook K8s Secret
python3 - "$MASI_TOKEN" <<'PY'
import sys, json, urllib.request
tok = sys.argv[1]
GAPI = "http://100.74.17.70:30000/api/v1"
WS = open("/dev/stdin") and __import__("os").environ["WEBHOOK_SECRET"]  # or paste
req = urllib.request.Request(
  f"{GAPI}/user/hooks",
  data=json.dumps({"type":"gitea",
    "config":{"url":"http://pragent-webhook.pragent.svc.cluster.local/webhook",
              "content_type":"json","secret":WS},
    "events":["pull_request"],"active":True}).encode(),
  method="POST", headers={"Authorization":f"token {tok}","Content-Type":"application/json"})
print(urllib.request.urlopen(req).status, urllib.request.urlopen(req).read()[:80])
PY
# 3. revoke the one-time token (Gitea admin UI → Users → masi → Access Tokens).

Owners already covered: gitea_admin (webhook id=4), masi (webhook id=5), techspark (webhook id=6 — a user account, not an org; covers techspark/suaspark-site, techspark/spark-ui, etc.). True orgs need org-level webhooks (POST /orgs/{org}/hooks, requires a token with write:organization).

Gitea SSRF allow-list (required, one-time)

Gitea refuses to POST webhooks to in-cluster addresses by default:

webhook can only call allowed HTTP servers (check your webhook.ALLOWED_HOST_LIST setting),
deny 'pragent-webhook.pragent.svc.cluster.local(10.152.183.170:80)'

Fix: add a scoped [webhook] section to Gitea's app.ini via the helm chart's inline-config secret (gitea-inline-config, key = section name webhook):

ALLOWED_HOST_LIST = external,loopback,*.svc.cluster.local,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/10

Then restart the gitea pod. Scoped to in-cluster + tailnet ranges only — not a blanket "allow all private". A future helm upgrade may overwrite the inline secret; bake it into gitea.config.webhook.ALLOWED_HOST_LIST in the helm values for permanence.

The opencode review engine

The review "brain" runs on opencode (the AI coding-agent CLI), not a single cramped model call. pilot/opencode_review.py is the glue:

1. fetch_archive — GET .../archive/{sha}.tar.gz, untar into a temp workdir (stripping the top dir) so the agent has the real files, not just the diff.
2. write_brief — renders .pragent/brief.md (title, body, diff, repo .pr-review.json, prior reviews, sha, anchor hint).
3. drop_factory — copies opencode.json + .opencode/ (agents/skills/commands) into the workdir as the project config.
4. run_opencode — opencode run --pure --format json --agent pragent --dir <workdir> --model headroom/glm-5.2:cloud headlessly. --format json emits NDJSON events: parse_opencode_events reconstructs the assistant text from text events and sums tokens/cost/steps from every step_finish event. Returns (text, usage).

It does no Gitea I/O and no parsing — review_pr parses the stdout into (summary, findings), validates findings against diff anchors, and posts. So all v2 logic (dedupe marker, anchor validation, language-tagged suggestion fencing, posting, optional AI-USAGE attribution) is reused and never depends on the model remembering it.

The factory lives in the pragent repo root: opencode.json (provider/model/ permission) + .opencode/ (agents, skills, commands). It is both the in-cluster deploy factory and the local interactive factory (run opencode in the repo, or /review via .opencode/commands/review.md). .opencode/README.md is the factory guide: pipeline diagram, how to add a subagent (drop a .md + one allow-list line), how to add a skill, how the engine flag works, how to switch the model. Lean by default: the pragent primary does summary + findings in one pass and runs the repo's own tsc/ruff/eslint/go vet via bash; security/tests/perf subagents are dormant lenses the primary delegates to only on large/security-sensitive diffs, so small PRs never fan out.

Engine flag + model ref

PRAGENT_ENGINE=opencode (default) selects it; =ollama keeps the legacy direct POST .../v1/messages path as a fallback. opencode wants a provider-prefixed model ref, so review_pr maps the bare OLLAMA_MODEL (glm-5.2:cloud) to headroom/glm-5.2:cloud (override with OPENCODE_MODEL). The headroom provider is defined in opencode.json with options.baseURL=http://100.74.17.70:8789/v1 (the headroom Anthropic proxy).

Local one-shot (no posting)

cd ~/Projects/pragent
python3 /tmp/pragent-e2e.py <owner>/<repo> <pr_number>   # driver script
# or, with opencode installed locally:
opencode run --pure --agent pragent --dir <checkout> --model headroom/glm-5.2:cloud \
  "$(python3 -c 'import sys;sys.path.insert(0,"pilot");import opencode_review as o;print(o._PROMPT)')"

Gotchas baked into opencode_review.py

• stdin=DEVNULL — opencode blocks on stdin (permission prompt) when run headlessly via subprocess; closing stdin is required or it hangs to timeout.
• Strip ANTHROPIC_* — the host shell exports ANTHROPIC_BASE_URL / ANTHROPIC_AUTH_TOKEN / ANTHROPIC_DEFAULT_*_MODEL (for Claude Code / headroom). Leaked into opencode, ANTHROPIC_DEFAULT_SONNET_MODEL=glm-5.2:cloud makes opencode look for provider glm-5.2:cloud → ProviderModelNotFoundError. The headroom provider's config is self-contained, so all ANTHROPIC_* are dropped from the subprocess env.
• Shared warmed HOME — opencode bun-installs its @opencode-ai runtime into $HOME/.config/opencode/node_modules on first run (cold-start, ~30-60s, once per pod lifetime). A shared, marker-warmed HOME makes every review a warm run.

K8s deployment

Manifest: ~/k8s/pragent-webhook.yaml (Namespace pragent, Deployment pinned to kubernets, ClusterIP Service). The container image pragent-webhook:opencode (pilot/Dockerfile: python:3.12-slim + node 20 + opencode-ai@1.3.10 + pyright / typescript-language-server / eslint / ruff) is built locally and imported into microk8s containerd — it is not pulled from a registry (imagePullPolicy: Never). The webhook secret + bot token are a Secret (pragent-webhook). An emptyDir at /tmp/pragent-work holds the per-review checkout + the warmed opencode runtime. Verified: a regular pod on kubernets reaches both 100.74.17.70:8789 (headroom/glm) and gitea-http.gitea.svc.cluster.local:3000.

Build + deploy after editing the pilot scripts or the factory:

K="microk8s kubectl"; cd ~/Projects/pragent
# 1. build the image (docker is in the microk8s group, no sudo)
docker build -t pragent-webhook:opencode -f pilot/Dockerfile .
# 2. import into microk8s containerd — sudoless. the `microk8s ctr` wrapper
#    sudo-wraps even in the microk8s group, so use the raw binary against the
#    group-readable containerd socket directly:
docker save pragent-webhook:opencode | \
  /snap/microk8s/current/bin/ctr --address /var/snap/microk8s/common/run/containerd.sock \
    --namespace k8s.io image import -
# 3. apply + roll
$K apply -f ~/k8s/pragent-webhook.yaml
$K -n pragent rollout restart deploy/pragent-webhook
$K -n pragent logs -f deploy/pragent-webhook

Env on the Deployment: PRAGENT_ENGINE, OPENCODE_MODEL, OPENCODE_EXPERIMENTAL_LSP_TOOL, PRAGENT_FACTORY_DIR, PRAGENT_OPENCODE_BIN, PRAGENT_WORK_ROOT, PRAGENT_REVIEW_TIMEOUT, GITEA_API, OLLAMA_URL, OLLAMA_MODEL, OLLAMA_MAX_TOKENS, DIFF_MAX_CHARS are literals; WEBHOOK_SECRET + PRAGENT_BOT_TOKEN come from the Secret.

Relationship to the CI-step pilot

Both paths coexist. The webhook service is strictly less per-repo setup. For repos still on the CI-step workflow (.gitea/workflows/ai-review.yml), retire that file once the owner has a user-level webhook — otherwise a labeled PR gets reviewed twice. gitea_admin/pragent's own self-CI workflow was retired when the webhook service went live.

Known limitations (pilot)

• One model (glm-5.2:cloud); no tiering, no analyzer fan-out, no shared-prefix caching. Those are framework features.
• No status checks, no fail-close (review never blocks a PR).
• Dedupe is per-commit: a re-push (new SHA) always re-reviews (by design — the diff changed). Prior-review context is fed to the model so it doesn't repeat, but the bot does not delete or resolve its own old reviews.
• Inline comments only anchor to post-change lines present in the diff (context + added). A finding whose line the model places on a removed line or outside the diff is folded into the summary as a bullet instead of misplaced.
• Gitea 1.26.1: system webhooks broken (see above) → user-level webhooks instead; hook delivery-history API (.../hooks/{id}/tasks) returns 404, so delivery is observed via the pragent-webhook pod logs (kubectl -n pragent logs -f deploy/pragent-webhook).

Cleanup of one-time setup tokens (manual)

Token revocation via API/CLI is broken in Gitea 1.26.1 (GET /users/{u}/tokens 401, no CLI delete-access-token). Revoke these one-time setup tokens in the Gitea admin UI (Site Administration → Users → user → Manage access tokens), name prefix pragent-:

• gitea_admin: pragent-syswh-reg-2026, pragent-syswh-list-2026, pragent-syswh-test-2026, pragent-syswh-retry-2026, pragent-userhook-test-2026, pragent-payload-look-2026, pragent-cleanup-2026, pragent-cleanup2-2026, pragent-cleanup3-2026.
• masi: pragent-userhook-masi-2026.
• techspark: pragent-userhook-techspark-2026.

(Keep pragent-bot's pragent-ci token — that's the live reviewer credential.)