diff --git a/README.md b/README.md index 164355a..8ee8de8 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,8 @@ An AI pull-request reviewer for Gitea that posts **inline comments with suggested fixes**, not a wall of prose — and reports what each review cost. -Label a PR `AI-REVIEW`. A webhook wakes a service that checks the repo out at the +A webhook wakes for any PR on a repo whose default branch carries a +`.pr-review.json` with `"enabled": true`. The service checks the repo out at the PR's head commit, reads the changed files *and the code around them*, runs the repo's own linters, and posts a review anchored to real lines. @@ -40,14 +41,16 @@ built — see [`docs/plans/`](docs/plans/). What works today: -- a central webhook service, so onboarding a repo is *add the bot + add the label* +- a central webhook service, so onboarding a repo is *add the bot + commit + `.pr-review.json:enabled = true`* - whole-repo context: the reviewer reads callers and types, not just the hunk - inline comments with language-highlighted suggested fixes, anchored to post-change lines and validated in Python before posting - per-commit dedupe, and prior reviews fed back so a re-push synthesises rather than repeats -- `.pr-review.json` for per-repo focus and house rules -- optional token/cost reporting via an `AI-USAGE` label +- `.pr-review.json` for per-repo focus and house rules (also the opt-in flag) +- token-usage reporting on every review, measured from opencode `step_finish` + events - containment against hostile PR content (see [Security](#security)) Not yet: status checks, fail-close, attention tiering enforced in code (it is @@ -56,21 +59,22 @@ currently a skill the agent follows), multi-model routing. ## How a review runs ``` -PR labelled AI-REVIEW +PR opened on repo with `.pr-review.json:enabled = true` │ Gitea webhook (HMAC-verified, body-capped, concurrency-bounded) ▼ review_pr() - 1. dedupe already reviewed this exact sha? stop. - 2. fetch diff + .pr-review.json from the BASE branch - 3. checkout repo archive at head sha → temp workdir - 4. sanitize delete author-controlled agent-instruction files - 5. brief .pragent/brief.md, untrusted parts explicitly fenced - 6. review opencode agent: read code, run linters, emit findings JSON - 7. anchor validate every line against the diff's post-change lines - 8. post inline comments + summary, as pragent-bot + 1. opt-in .pr-review.json:enabled=true on base? if not, skip. + 2. dedupe already reviewed this exact sha? stop. + 3. fetch diff + .pr-review.json from the BASE branch + 4. checkout repo archive at head sha → temp workdir + 5. sanitize delete author-controlled agent-instruction files + 6. brief .pragent/brief.md, untrusted parts explicitly fenced + 7. review opencode agent: read code, run linters, emit findings JSON + 8. anchor validate every line against the diff's post-change lines + 9. post inline comments + summary, as pragent-bot ``` -Steps 1, 2, 7 and 8 are deterministic Python. The model's only job is step 6 — +Steps 1, 3, 8 and 9 are deterministic Python. The model's only job is step 7 — producing correct findings. It never talks to Gitea, and a finding whose line does not validate becomes a summary bullet rather than a misplaced comment. @@ -79,8 +83,8 @@ does not validate becomes a summary bullet rather than a misplaced comment. Onboarding a repo, once the service is running for that owner: 1. add `pragent-bot` as a **Write** collaborator -2. create the `AI-REVIEW` label -3. label a PR +2. commit `.pr-review.json: {"enabled": true}` to the repo's default branch +3. open a PR Standing up the service itself — the webhook, the image, the Gitea SSRF allow-list, the per-owner webhook registration — is in @@ -136,8 +140,9 @@ concurrency. Full threat model and residual risks: `pilot/README-webhook.md`. The pilot runs against a self-hosted model and bills nothing per token, but the token *work* is real. `pilot/cost_model.py` prices it against published API -rates, calibrated against runs measured through the `AI-USAGE` label -(`OBSERVED_RUNS` in that file — append to it, don't guess). +rates, calibrated against runs measured through the usage telemetry +(`OBSERVED_RUNS` in that file — append to it, don't guess). Tokens are summed +from opencode `step_finish` events per review. Two measured reviews of a ~1100-line PR in this repo: 28 and 31 agent steps, ~2.1M input tokens each, **zero cache reads or writes**. The demo repo's PR, same diff --git a/pilot/README-webhook.md b/pilot/README-webhook.md index 2ccd46f..19bd9a8 100644 --- a/pilot/README-webhook.md +++ b/pilot/README-webhook.md @@ -1,29 +1,31 @@ # pragent pilot — central webhook service -The CI-step pilot (`pilot/README.md`) needs a workflow file + secret + label per -repo. The **central webhook service** removes the workflow file, the secret, and -the runner dependency: a Gitea webhook posts PR events to an always-on in-cluster -service, which gates on the `AI-REVIEW` label and runs the same review core. +The CI-step pilot (`pilot/README.md`) needs a workflow file + secret per repo. +The **central webhook service** removes the workflow file, the secret, and the +runner dependency: a Gitea webhook posts PR events to an always-on in-cluster +service, which gates on `.pr-review.json:enabled = true` and runs the same review +core. ## Architecture ``` -PR opened/pushed/labeled/edited/… (any repo under a covered owner) +PR opened/pushed/edited/… (any repo under a covered owner) │ Gitea user-level webhook (events: pull_request) ▼ Service pragent-webhook.pragent.svc.cluster.local (ClusterIP, ns pragent) │ body-size cap → HMAC-verify (X-Gitea-Signature) - │ → gate: action ≠ closed AND pull_request.labels ∋ AI-REVIEW + │ → gate: action ≠ closed AND .pr-review.json:enabled = true on base │ → claim (repo, index, sha) in-flight (closes the dedupe race) │ → bounded worker (PRAGENT_MAX_CONCURRENT_REVIEWS, default 2) ▼ ai_review.review_pr() (same core the CI-step uses) - 1. fetch existing reviews → dedupe: skip if a review already carries - (no duplicate on label-toggle / re-fire) - 2. fetch PR diff → GET .../pulls/{i}.diff - 3. fetch .pr-review.json @ head ref (optional repo-local focus/config) - 4. prior review bodies → fed as "already said" context (light §6.1) - 5. PRAGENT_ENGINE=opencode (default): + 1. opt-in .pr-review.json:enabled = true on base? if not, skip. + 2. fetch existing reviews → dedupe: skip if a review already carries + (no duplicate on title/body-edit re-fire) + 3. fetch PR diff → GET .../pulls/{i}.diff + 4. fetch .pr-review.json @ base ref (the opt-in flag + repo-local focus/config) + 5. prior review bodies → fed as "already said" context (light §6.1) + 6. PRAGENT_ENGINE=opencode (default): a. fetch repo archive @ head sha → /tmp/pragent-work/- (symlink-escape + traversal rejected on untar) a2. sanitize the workdir: delete author-controlled agent-instruction @@ -39,8 +41,8 @@ ai_review.review_pr() (same core the CI-step uses) diffs, and emits: {"summary":..., "findings":[{severity,path,line, problem,fix,suggestion,reference}]} (=ollama: legacy single POST to http://:8789/v1/messages) - 6. parse diff hunks → valid (path, new_line) anchors (RIGHT side) - 7. post review → POST .../pulls/{i}/reviews (event: COMMENT) as pragent-bot + 7. parse diff hunks → valid (path, new_line) anchors (RIGHT side) + 8. post review → POST .../pulls/{i}/reviews (event: COMMENT) as pragent-bot - prose summary → review body intro - anchored findings → inline line comments, body wraps `suggestion` in a language-tagged fenced code block (Gitea syntax-highlights it; Gitea @@ -58,13 +60,13 @@ of repeating (light version of framework §6.1). 1. Add `pragent-bot` as collaborator with **Write** (so it can read the diff and post the review). The bot stays a normal user — it is **not** a site admin. -2. Create the `AI-REVIEW` label on the repo (one-time; `pragent-bot`'s - `write:issue` scope can do it once it's a collaborator). -3. Label a PR `AI-REVIEW`. +2. Commit `.pr-review.json: {"enabled": true}` to the repo's default branch + (so every PR on the repo is auto-reviewed). +3. Open a PR. -No workflow file, no repo secret, no act-runner needed. (The owner must already -be covered by a user-level webhook — see below. If not, do the one-time -per-owner setup first.) +No workflow file, no repo secret, no act-runner, no label needed. (The owner +must already be covered by a user-level webhook — see below. If not, do the +one-time per-owner setup first.) ## Token-usage reporting (always on) @@ -151,17 +153,17 @@ curl -u techspark -X PUT \ The receiver uses a **denylist**, not an allowlist: it reviews on every `pull_request` action **except `closed`** — `opened`, `reopened`, -`synchronize`/`synchronized`, `labeled`/`label_updated`, `edited` (title/body), -`ready_for_review` (draft→ready), `assigned`, `review_requested`, `milestone`, -… . This is safe because of two downstream gates: +`synchronize`/`synchronized`, `edited` (title/body), `ready_for_review` +(draft→ready), `assigned`, `review_requested`, `milestone`, … . This is safe +because of two downstream gates: -- the **AI-REVIEW label gate** — payload `labels` reflect current state, so an - `unlabeled` that *removed* AI-REVIEW fails the gate (no review); an - `unlabeled` of another label still passes; -- the **sha dedupe** — any same-sha re-fire (title edit, assignee, milestone, - a label toggle of another label…) is skipped, so the only newly-effective - actions are ones that change the head sha (`synchronize`, already covered) or - move a draft to ready (`ready_for_review`) on an un-reviewed sha. +- the **opt-in gate** — `.pr-review.json:enabled = true` is read from the base + branch, so only repos that opted in get reviewed. A repo that deletes the + file between pushes opts out; +- the **sha dedupe** — any same-sha re-fire (title edit, assignee, milestone…) + is skipped, so the only newly-effective actions are ones that change the head + sha (`synchronize`, already covered) or move a draft to ready + (`ready_for_review`) on an un-reviewed sha. ## Threat model @@ -204,8 +206,8 @@ Additionally: the repo archive is untarred with symlink-escape and parent-traversal rejection (`_extract_tar_strip_one`), the container runs as uid 10001, and the webhook caps request bodies (`PRAGENT_MAX_BODY_BYTES`, default 10 MiB) and concurrent reviews (`PRAGENT_MAX_CONCURRENT_REVIEWS`, -default 2 — each review forks an opencode process, so unbounded threads were a -self-inflicted fork bomb on a label-ten-PRs burst). +default 2 — each review forks an opencode process, so unbounded threads would be +a self-inflicted fork bomb on any burst of concurrent PRs). **Residual risk, accepted for a pilot:** the agent still *executes* hostile repo content indirectly (running the repo's own linters on it) inside a container @@ -222,7 +224,7 @@ so the `/tmp/pragent-work` emptyDir is writable. ## Multi-lens pipeline (5 default lenses, on by default) -Default `AI-REVIEW` runs spawn **one opencode subprocess per lens in parallel** +Default reviews spawn **one opencode subprocess per lens in parallel** and synthesize the merged findings before posting. Cheaper than 5 sequential reviews because the headroom proxy caches the byte-identical brief across lens calls (lenses 2..N hit cache).