docs: onboarding uses .pr-review.json:enabled, not labels

This commit is contained in:
claude
2026-08-22 01:33:42 +00:00
parent 2432228d68
commit 3e03fb80a7
2 changed files with 58 additions and 51 deletions
+23 -18
View File
@@ -3,7 +3,8 @@
An AI pull-request reviewer for Gitea that posts **inline comments with suggested
fixes**, not a wall of prose — and reports what each review cost.
Label a PR `AI-REVIEW`. A webhook wakes a service that checks the repo out at the
A webhook wakes for any PR on a repo whose default branch carries a
`.pr-review.json` with `"enabled": true`. The service checks the repo out at the
PR's head commit, reads the changed files *and the code around them*, runs the
repo's own linters, and posts a review anchored to real lines.
@@ -40,14 +41,16 @@ built — see [`docs/plans/`](docs/plans/).
What works today:
- a central webhook service, so onboarding a repo is *add the bot + add the label*
- a central webhook service, so onboarding a repo is *add the bot + commit
`.pr-review.json:enabled = true`*
- whole-repo context: the reviewer reads callers and types, not just the hunk
- inline comments with language-highlighted suggested fixes, anchored to
post-change lines and validated in Python before posting
- per-commit dedupe, and prior reviews fed back so a re-push synthesises rather
than repeats
- `.pr-review.json` for per-repo focus and house rules
- optional token/cost reporting via an `AI-USAGE` label
- `.pr-review.json` for per-repo focus and house rules (also the opt-in flag)
- token-usage reporting on every review, measured from opencode `step_finish`
events
- containment against hostile PR content (see [Security](#security))
Not yet: status checks, fail-close, attention tiering enforced in code (it is
@@ -56,21 +59,22 @@ currently a skill the agent follows), multi-model routing.
## How a review runs
```
PR labelled AI-REVIEW
PR opened on repo with `.pr-review.json:enabled = true`
│ Gitea webhook (HMAC-verified, body-capped, concurrency-bounded)
review_pr()
1. dedupe already reviewed this exact sha? stop.
2. fetch diff + .pr-review.json from the BASE branch
3. checkout repo archive at head sha → temp workdir
4. sanitize delete author-controlled agent-instruction files
5. brief .pragent/brief.md, untrusted parts explicitly fenced
6. review opencode agent: read code, run linters, emit findings JSON
7. anchor validate every line against the diff's post-change lines
8. post inline comments + summary, as pragent-bot
1. opt-in .pr-review.json:enabled=true on base? if not, skip.
2. dedupe already reviewed this exact sha? stop.
3. fetch diff + .pr-review.json from the BASE branch
4. checkout repo archive at head sha → temp workdir
5. sanitize delete author-controlled agent-instruction files
6. brief .pragent/brief.md, untrusted parts explicitly fenced
7. review opencode agent: read code, run linters, emit findings JSON
8. anchor validate every line against the diff's post-change lines
9. post inline comments + summary, as pragent-bot
```
Steps 1, 2, 7 and 8 are deterministic Python. The model's only job is step 6
Steps 1, 3, 8 and 9 are deterministic Python. The model's only job is step 7
producing correct findings. It never talks to Gitea, and a finding whose line
does not validate becomes a summary bullet rather than a misplaced comment.
@@ -79,8 +83,8 @@ does not validate becomes a summary bullet rather than a misplaced comment.
Onboarding a repo, once the service is running for that owner:
1. add `pragent-bot` as a **Write** collaborator
2. create the `AI-REVIEW` label
3. label a PR
2. commit `.pr-review.json: {"enabled": true}` to the repo's default branch
3. open a PR
Standing up the service itself — the webhook, the image, the Gitea SSRF
allow-list, the per-owner webhook registration — is in
@@ -136,8 +140,9 @@ concurrency. Full threat model and residual risks: `pilot/README-webhook.md`.
The pilot runs against a self-hosted model and bills nothing per token, but the
token *work* is real. `pilot/cost_model.py` prices it against published API
rates, calibrated against runs measured through the `AI-USAGE` label
(`OBSERVED_RUNS` in that file — append to it, don't guess).
rates, calibrated against runs measured through the usage telemetry
(`OBSERVED_RUNS` in that file — append to it, don't guess). Tokens are summed
from opencode `step_finish` events per review.
Two measured reviews of a ~1100-line PR in this repo: 28 and 31 agent steps,
~2.1M input tokens each, **zero cache reads or writes**. The demo repo's PR, same
+35 -33
View File
@@ -1,29 +1,31 @@
# pragent pilot — central webhook service
The CI-step pilot (`pilot/README.md`) needs a workflow file + secret + label per
repo. The **central webhook service** removes the workflow file, the secret, and
the runner dependency: a Gitea webhook posts PR events to an always-on in-cluster
service, which gates on the `AI-REVIEW` label and runs the same review core.
The CI-step pilot (`pilot/README.md`) needs a workflow file + secret per repo.
The **central webhook service** removes the workflow file, the secret, and the
runner dependency: a Gitea webhook posts PR events to an always-on in-cluster
service, which gates on `.pr-review.json:enabled = true` and runs the same review
core.
## Architecture
```
PR opened/pushed/labeled/edited/… (any repo under a covered owner)
PR opened/pushed/edited/… (any repo under a covered owner)
│ Gitea user-level webhook (events: pull_request)
Service pragent-webhook.pragent.svc.cluster.local (ClusterIP, ns pragent)
│ body-size cap → HMAC-verify (X-Gitea-Signature)
│ → gate: action ≠ closed AND pull_request.labels ∋ AI-REVIEW
│ → gate: action ≠ closed AND .pr-review.json:enabled = true on base
│ → claim (repo, index, sha) in-flight (closes the dedupe race)
│ → bounded worker (PRAGENT_MAX_CONCURRENT_REVIEWS, default 2)
ai_review.review_pr() (same core the CI-step uses)
1. fetch existing reviews → dedupe: skip if a review already carries
<!-- pragent:sha=<this sha> --> (no duplicate on label-toggle / re-fire)
2. fetch PR diff → GET .../pulls/{i}.diff
3. fetch .pr-review.json @ head ref (optional repo-local focus/config)
4. prior review bodies → fed as "already said" context (light §6.1)
5. PRAGENT_ENGINE=opencode (default):
1. opt-in .pr-review.json:enabled = true on base? if not, skip.
2. fetch existing reviews → dedupe: skip if a review already carries
<!-- pragent:sha=<this sha> --> (no duplicate on title/body-edit re-fire)
3. fetch PR diff → GET .../pulls/{i}.diff
4. fetch .pr-review.json @ base ref (the opt-in flag + repo-local focus/config)
5. prior review bodies → fed as "already said" context (light §6.1)
6. PRAGENT_ENGINE=opencode (default):
a. fetch repo archive @ head sha → /tmp/pragent-work/<repo>-<sha>
(symlink-escape + traversal rejected on untar)
a2. sanitize the workdir: delete author-controlled agent-instruction
@@ -39,8 +41,8 @@ ai_review.review_pr() (same core the CI-step uses)
diffs, and emits: {"summary":..., "findings":[{severity,path,line,
problem,fix,suggestion,reference}]}
(=ollama: legacy single POST to http://<model-proxy-host>:8789/v1/messages)
6. parse diff hunks → valid (path, new_line) anchors (RIGHT side)
7. post review → POST .../pulls/{i}/reviews (event: COMMENT) as pragent-bot
7. parse diff hunks → valid (path, new_line) anchors (RIGHT side)
8. post review → POST .../pulls/{i}/reviews (event: COMMENT) as pragent-bot
- prose summary → review body intro
- anchored findings → inline line comments, body wraps `suggestion` in a
language-tagged fenced code block (Gitea syntax-highlights it; Gitea
@@ -58,13 +60,13 @@ of repeating (light version of framework §6.1).
1. Add `pragent-bot` as collaborator with **Write** (so it can read the diff and
post the review). The bot stays a normal user — it is **not** a site admin.
2. Create the `AI-REVIEW` label on the repo (one-time; `pragent-bot`'s
`write:issue` scope can do it once it's a collaborator).
3. Label a PR `AI-REVIEW`.
2. Commit `.pr-review.json: {"enabled": true}` to the repo's default branch
(so every PR on the repo is auto-reviewed).
3. Open a PR.
No workflow file, no repo secret, no act-runner needed. (The owner must already
be covered by a user-level webhook — see below. If not, do the one-time
per-owner setup first.)
No workflow file, no repo secret, no act-runner, no label needed. (The owner
must already be covered by a user-level webhook — see below. If not, do the
one-time per-owner setup first.)
## Token-usage reporting (always on)
@@ -151,17 +153,17 @@ curl -u techspark -X PUT \
The receiver uses a **denylist**, not an allowlist: it reviews on every
`pull_request` action **except `closed`**`opened`, `reopened`,
`synchronize`/`synchronized`, `labeled`/`label_updated`, `edited` (title/body),
`ready_for_review` (draft→ready), `assigned`, `review_requested`, `milestone`,
… . This is safe because of two downstream gates:
`synchronize`/`synchronized`, `edited` (title/body), `ready_for_review`
(draft→ready), `assigned`, `review_requested`, `milestone`, … . This is safe
because of two downstream gates:
- the **AI-REVIEW label gate** — payload `labels` reflect current state, so an
`unlabeled` that *removed* AI-REVIEW fails the gate (no review); an
`unlabeled` of another label still passes;
- the **sha dedupe** — any same-sha re-fire (title edit, assignee, milestone,
a label toggle of another label…) is skipped, so the only newly-effective
actions are ones that change the head sha (`synchronize`, already covered) or
move a draft to ready (`ready_for_review`) on an un-reviewed sha.
- the **opt-in gate**`.pr-review.json:enabled = true` is read from the base
branch, so only repos that opted in get reviewed. A repo that deletes the
file between pushes opts out;
- the **sha dedupe** — any same-sha re-fire (title edit, assignee, milestone…)
is skipped, so the only newly-effective actions are ones that change the head
sha (`synchronize`, already covered) or move a draft to ready
(`ready_for_review`) on an un-reviewed sha.
## Threat model
@@ -204,8 +206,8 @@ Additionally: the repo archive is untarred with symlink-escape and
parent-traversal rejection (`_extract_tar_strip_one`), the container runs as
uid 10001, and the webhook caps request bodies (`PRAGENT_MAX_BODY_BYTES`,
default 10 MiB) and concurrent reviews (`PRAGENT_MAX_CONCURRENT_REVIEWS`,
default 2 — each review forks an opencode process, so unbounded threads were a
self-inflicted fork bomb on a label-ten-PRs burst).
default 2 — each review forks an opencode process, so unbounded threads would be
a self-inflicted fork bomb on any burst of concurrent PRs).
**Residual risk, accepted for a pilot:** the agent still *executes* hostile repo
content indirectly (running the repo's own linters on it) inside a container
@@ -222,7 +224,7 @@ so the `/tmp/pragent-work` emptyDir is writable.
## Multi-lens pipeline (5 default lenses, on by default)
Default `AI-REVIEW` runs spawn **one opencode subprocess per lens in parallel**
Default reviews spawn **one opencode subprocess per lens in parallel**
and synthesize the merged findings before posting. Cheaper than 5 sequential
reviews because the headroom proxy caches the byte-identical brief across
lens calls (lenses 2..N hit cache).