docs: describe review module seams

Group review, feedback, evaluation, observability, and entrypoint code into packages. Keep thin top-level compatibility shims for existing scripts and imports, and mirror the structure in the tests.

.judge-trigger.txt

@@ -0,0 +1 @@
+# judge trigger 1788203999

README.md

@@ -35,9 +35,9 @@ code never has to leave your network.
 
 ## Status
 
-A **pilot** is live and reviewing real PRs. The full framework (`pragent init`,
-tiering as code, analyzer fan-out, `explain` / `replay`) is designed but not
-built — see [`docs/plans/`](docs/plans/).
+A **pilot** is live and reviewing real PRs. The current runtime architecture is
+documented in [`docs/architecture.md`](docs/architecture.md); older framework
+plans remain in [`docs/plans/`](docs/plans/) as historical design material.
 
 What works today:
 
@@ -51,10 +51,12 @@ What works today:
 - `.pr-review.json` for per-repo focus and house rules (also the opt-in flag)
 - token-usage reporting on every review, measured from opencode `step_finish`
   events
+- Langfuse traces, equivalent-cost reporting, evaluation scores, and feedback
+  harvesting
 - containment against hostile PR content (see [Security](#security))
 
 Not yet: status checks, fail-close, attention tiering enforced in code (it is
-currently a skill the agent follows), multi-model routing.
+currently a skill the agent follows), multi-model routing, and a CLI framework.
 
 ## How a review runs
 
@@ -94,9 +96,8 @@ path is in [`pilot/README.md`](pilot/README.md).
 The model endpoint is supplied at runtime via `PRAGENT_MODEL_BASE_URL`; the
 committed `opencode.json` carries a placeholder.
 
-Per-review token spend, latency and equivalent cost are shipped to a
-self-hosted Langfuse, split into `ollama` and `claude` environments so the two
-spend stories stay separate: [`pilot/README-langfuse.md`](pilot/README-langfuse.md).
+Per-review token spend, latency, equivalent cost, and evaluation scores are
+shipped to a self-hosted Langfuse: [`pilot/README-langfuse.md`](pilot/README-langfuse.md).
 Emission is a silent no-op unless `LANGFUSE_HOST` and the key pair are set.
 
 ## Extending it
@@ -175,7 +176,7 @@ python3 pilot/cost_model.py --help      # other mixes, volumes, models
 ## Development
 
 ```bash
-python3 -m pytest tests -q    # 137 tests, stdlib only, no network
+python3 -m pytest tests -q    # stdlib-only tests, no network
 ```
 
 The pilot is stdlib-only Python by design — it runs from a bare `python:slim`
@@ -185,3 +186,5 @@ review time.
 ## License
 
 Not yet chosen. Until one is added, no reuse rights are granted.
+
+_pilot eval judges test 1788201461_

docs/architecture.md

@@ -0,0 +1,72 @@
+# pragent current architecture
+
+Status: pilot implementation, September 2026.
+
+## System shape
+
+```text
+Gitea pull_request webhook
+          │ signed HTTP
+          ▼
+webhook_server ── trusted base config ──► review_config
+          │ bounded worker
+          ▼
+review_pr facade/orchestrator
+   ├── entrypoints/gitea  fetch diff, reviews, config; publish review
+   ├── diff_compress      reduce prompt context
+   ├── opencode_review     isolated checkout + agent execution
+   │       └── model / repo factory (.opencode)
+   ├── review parsing      normalize findings + validate anchors
+   ├── feedback            persist reactions and derive scores
+   └── langfuse_trace      usage, cost, evaluation telemetry
+```
+
+## Seams and responsibilities
+
+The external seam is `ai_review.review_pr(...)`: one call represents one review
+attempt and returns success/skip status. The top-level module and
+`review/ai_review.py` are compatibility facades; the current implementation is
+implemented by `review/pipeline.py`, with pure transforms and adapters split
+into the neighboring modules.
+
+The internal seams are deliberately narrower:
+
+- `review_config.repo_enabled(get, ...)` owns the security-sensitive opt-in
+  decision. It receives a transport function, so malformed configuration and
+  failure behavior are deterministic in tests.
+- `entrypoints/gitea.request()` and `GiteaClient` own HTTP authentication, JSON
+  request encoding, timeout, and Gitea URL construction.
+- `model_client.complete()` owns the legacy Anthropic-compatible request shape.
+  `opencode_review` is the preferred agent adapter and keeps Gitea I/O out of
+  the autonomous process.
+- `review/analysis`, `review/output`, `review/configuration`, and
+  `review/adapters` keep prompt construction, finding parsing, config filtering,
+  rendering, and publishing in focused modules.
+  pure transformations. Their callers do not need to know how model or Gitea
+  transport works.
+- `langfuse_trace` is an optional sink. It is fail-open and cannot change the
+  review result.
+
+## Trust model
+
+The review config is read from the PR base branch, never the PR head. The agent
+checkout is treated as hostile: instruction files are removed, credentials are
+not inherited, and the agent only returns text to the Python publisher. Python
+validates finding paths and post-change line anchors before sending comments.
+
+## Observability
+
+Langfuse is the operational analytics surface. A trace groups runs by
+`owner/repo#PR`; generations carry usage and cost basis; evaluation scores and
+human-feedback scores are attached later. The former SQLite-backed dashboard
+was removed. SQLite remains only as the feedback/evaluation ingestion store.
+
+## Removed surface
+
+The dashboard server, dashboard data module, dashboard tests, dashboard README,
+and dashboard Kubernetes manifest are intentionally gone. Operators use the
+Langfuse UI for review trends and cost analysis, and Gitea for review details
+and configuration changes.
+
+Historical design/implementation plans under `docs/plans/` describe the
+earlier TypeScript framework proposal and are not the runtime architecture.

k8s/pragent-dashboard.yaml

@@ -1,118 +0,0 @@
-# pragent pilot — central dashboard service.
-#
-# Read-only overview + per-repo / per-PR drilldown over the same SQLite
-# feedback DB the webhook writes. Also mutates `.pr-review.json` on covered
-# repos via the Gitea contents API (Tasks C+D in pilot/dashboard.py). Same
-# image as the webhook (`pragent-webhook:optin`) — all pilot modules are
-# baked in at /app/pilot/.
-#
-# Routes: GET /  (overview), GET /r/<o>/<n>  (repo), GET /r/<o>/<n>/<i>  (PR),
-# GET /r/<o>/<n>/<i>/raw  (PR markdown raw), GET /login, GET /static/style.css,
-# POST /login, POST /r/<o>/<n>/edit.
-#
-# Auth: PRAGENT_DASHBOARD_TOKEN in the pragent-webhook Secret, cookie
-# `pragent_dash=<token>`, single-user. Empty / unset = no auth (tailnet-only).
-#
-# NodePort 30082 — only reachable on the Tailscale / LAN side of kubernets
-# (100.74.17.70 / 192.168.1.80) until/unconfigured. Mirrors pragent-webhook.yaml
-# in every other respect (uid 10001, nodeSelector, /data PVC).
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: pragent-dashboard
-  namespace: pragent
-  labels:
-    app: pragent-dashboard
-spec:
-  replicas: 1
-  selector:
-    matchLabels:
-      app: pragent-dashboard
-  template:
-    metadata:
-      labels:
-        app: pragent-dashboard
-    spec:
-      # Same node as the webhook — holds the headroom proxy + the /data PVC.
-      nodeSelector:
-        kubernetes.io/hostname: kubernets
-      # Dashboard is read-only over /data and only mutates Gitea (not local
-      # files), so unprivileged is fine. fsGroup matches the image's USER
-      # directive (10001) so the RO mount is readable.
-      securityContext:
-        runAsNonRoot: true
-        runAsUser: 10001
-        runAsGroup: 10001
-        fsGroup: 10001
-      containers:
-        - name: dashboard
-          image: pragent-webhook:optin
-          imagePullPolicy: Never
-          workingDir: /app
-          command: ["python3", "-m", "pilot.dashboard"]
-          ports:
-            - name: http
-              containerPort: 8081
-          env:
-            - name: PRAGENT_FEEDBACK_DB
-              value: /data/feedback.db
-            - name: PRAGENT_GITEA_API
-              value: http://gitea-http.gitea.svc.cluster.local:3000
-            # Dashboard reads DASHBOARD_PORT (not PORT) — verified in
-            # pilot/dashboard.py:51. Default 8081 if unset.
-            - name: DASHBOARD_PORT
-              value: "8081"
-            # Used by /r/<o>/<n>/edit to PUT updated JSON to the repo's
-            # contents API. Reuses the same bot token the webhook uses.
-            - name: PRAGENT_BOT_TOKEN
-              valueFrom:
-                secretKeyRef:
-                  name: pragent-webhook
-                  key: PRAGENT_BOT_TOKEN
-            # Auth cookie value. Add to the pragent-webhook Secret with:
-            #   kubectl patch secret pragent-webhook -n pragent --type=json \
-            #     -p='[{"op":"add","path":"/data/PRAGENT_DASHBOARD_TOKEN","value":"<base64>"}]'
-            - name: PRAGENT_DASHBOARD_TOKEN
-              valueFrom:
-                secretKeyRef:
-                  name: pragent-webhook
-                  key: PRAGENT_DASHBOARD_TOKEN
-          # /data is read-only — the dashboard doesn't write the SQLite file;
-          # .pr-review.json mutations go through the Gitea contents API, not
-          # local fs. RO avoids any chance of two pods racing the same RWO PVC.
-          volumeMounts:
-            - name: feedback-data
-              mountPath: /data
-              readOnly: true
-          # No /health route in dashboard.py (returns 404 on unknown paths).
-          # Probes omitted intentionally — see pilot/dashboard.py:687-721.
-          # Resources: dashboard is read-heavy + tiny writes. /data RO + no
-          # subprocess fan-out (no opencode) keeps footprint small.
-          resources:
-            requests:
-              cpu: 100m
-              memory: 256Mi
-            limits:
-              cpu: 500m
-              memory: 512Mi
-      volumes:
-        - name: feedback-data
-          persistentVolumeClaim:
-            claimName: pragent-feedback-data
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: pragent-dashboard
-  namespace: pragent
-spec:
-  selector:
-    app: pragent-dashboard
-  ports:
-    - name: http
-      port: 80
-      targetPort: http
-      nodePort: 31540
-  type: NodePort
-  # 31540 — auto-allocated at first apply (30082 was already taken by
-  # habitsnow/habitsnow-proxy). Tailscale / LAN only until a Caddy route is set.

pilot/README-dashboard.md

@@ -1,269 +0,0 @@
-# pragent pilot — central dashboard service
-
-A read-only overview + per-repo / per-PR drilldown over the same SQLite
-feedback DB the webhook writes, plus a small form to mutate `.pr-review.json`
-on a covered repo via the Gitea contents API. Companion to the
-[webhook service](README-webhook.md); reuses the webhook image
-(`pragent-webhook:dashboard`) — the pilot modules are baked into `/app/pilot/`,
-and the dashboard is just `python3 -m pilot.dashboard`.
-
-## Architecture
-
-```
-Browser
-   │
-   ▼
-Caddy (TLS, wildcard cert via Cloudflare DNS-01)
-   │   https://pragent-dashboard.marcospaulo.dev.br → 100.74.17.70:31541
-   ▼
-Service oauth2-proxy-dashboard.pragent.svc.cluster.local  (NodePort 31541, ns pragent)
-   │
-   │   oauth2-proxy fronts the dashboard, enforces Logto SSO + email allowlist
-   │   sets X-Forwarded-User / X-Forwarded-Email on accepted requests
-   ▼
-Service pragent-dashboard.pragent.svc.cluster.local  (ClusterIP, ns pragent)
-   │
-   ▼
-pragent-dashboard pod  (uid 10001, /data RO, no subprocess fan-out)
-   │
-   ├── read /data/feedback.db           (PVC pragent-feedback-data, RO)
-   ├── GET  .../repos/{o}/{r}/...        (Gitea contents API, bot token)
-   └── PUT  .../repos/{o}/{r}/contents/.pr-review.json
-            (edit form submit; Gitea commits a new sha)
-```
-
-Fail-soft. Nothing is ever written to local disk by the dashboard — the
-SQLite file is read-only and `.pr-review.json` mutations go through Gitea's
-contents API so the commit history records who changed what.
-
-The dashboard `Service` is **ClusterIP** — only oauth2-proxy can reach it.
-Public access is gated by Caddy (TLS termination) → oauth2-proxy (Logto SSO
-+ allowlist) → dashboard.
-
-## What it does
-
-- **Overview** (`GET /`): summary stats across all onboarded repos — total
-  reviews, distinct PRs, finding counts by severity, false-positive /
-  accepted-pattern scores (see "Feedback loop" in README-webhook.md), and a
-  sparkline of review activity.
-- **Repo drilldown** (`GET /r/<owner>/<name>`): per-repo PRs with their
-  last-review status, finding counts, and links to PR-level drilldowns.
-- **PR drilldown** (`GET /r/<owner>/<name>/<index>`): the bot's review(s)
-  on that PR, inline findings, and reaction / resolved status harvested
-  by `feedback_harvest.py`.
-- **Raw review** (`GET /r/<owner>/<name>/<index>/raw`): the markdown body
-  of the most recent review, for copy-paste / diff-with-prose workflows.
-- **Edit form** (`POST /r/<owner>/<name>/edit`): a small HTML page that
-  loads the current `.pr-review.json` from the repo's default branch and
-  lets the operator edit the JSON (validated, then PUT to Gitea contents
-  API). This is how repo-local `focus` / `instructions` /
-  `reviewers` / `severity_threshold` get tuned per-repo after seeing
-  the feedback roll-up.
-
-All routes return HTML (or plain text for `/raw`) with the same stylesheet
-(`/static/style.css`).
-
-## Routes
-
-| method | path                              | auth | description                                  |
-|--------|-----------------------------------|------|----------------------------------------------|
-| GET    | `/`                               | yes  | Overview                                     |
-| GET    | `/static/style.css`               | no   | Stylesheet                                   |
-| GET    | `/r/<owner>/<name>`               | yes  | Repo drilldown                               |
-| GET    | `/r/<owner>/<name>/<index>`       | yes  | PR drilldown                                 |
-| GET    | `/r/<owner>/<name>/<index>/raw`   | yes  | Most recent review body as markdown          |
-| POST   | `/r/<owner>/<name>/edit`          | yes  | Edit `.pr-review.json` on the default branch |
-
-Auth is enforced by oauth2-proxy upstream; the dashboard itself only
-checks the `X-Forwarded-User` header that oauth2-proxy sets after a
-successful Logto login + email allowlist match.
-
-There is no `/health` route — don't add one to the k8s probes without
-updating `pilot/dashboard.py` (the handler returns 404 on unknown paths,
-so a probe would loop forever).
-
-## Mutations flow through Gitea, not local fs
-
-The edit endpoint reads the current `.pr-review.json` from
-`GET /repos/{o}/{r}/contents/.pr-review.json?ref=<default-branch>`, lets
-the operator edit it in a form (validated as JSON, length-capped per
-field, no schema migration), and PUTs the new content back via the
-contents API with a commit message like
-`pragent dashboard: update .pr-review.json`. Every edit is a real Gitea
-commit on the default branch, attributable to `pragent-bot`, and the
-next webhook fire picks up the new config — no Pod restart, no image
-rebuild, no pod-level state.
-
-The `/data` mount is **read-only** (see the `readOnly: true` on the
-volumeMount in `~/k8s/pragent-dashboard.yaml`): the dashboard never
-writes the SQLite file, only the webhook + the daily cronjob do, and
-keeping it RO means a buggy deploy can't corrupt the harvested feedback.
-
-## Auth (Logto SSO via oauth2-proxy)
-
-Authentication is delegated to oauth2-proxy, which fronts the dashboard
-in-cluster. The dashboard never sees a cookie or a token — it only
-inspects `X-Forwarded-User` (set by oauth2-proxy after a successful
-Logto login + email allowlist match). Missing header → 401 with
-`WWW-Authenticate: Basic realm="pragent-dashboard"`, which lets
-oauth2-proxy intercept and bounce the browser to Logto.
-
-Email allowlist lives in the ConfigMap `oauth2-proxy-dashboard-emails`
-in namespace `pragent`:
-
-```yaml
-data:
-  authenticated-emails: |
-    marcos.paulodasilva.mp@gmail.com
-    thiago@marcospaulo.dev.br
-```
-
-Edit the ConfigMap to add/remove users; oauth2-proxy hot-reloads the
-file (it logs `watching ... for updates`), no restart needed. This is
-the same isolation pattern as the minecraft-sso / code-server
-allowlists — see `~/.claude/memory/minecraft-sso.md`.
-
-The Logto app is `pragent-dashboard` (tenant `default`, type
-`Traditional`), created by direct INSERT into Logto Postgres mirroring
-the proven `minecraft-sso` pattern. Credentials live in
-`~/k8s/oauth2-proxy-dashboard-secret.yaml` (mode 600, NOT in git).
-
-Public URL: **https://pragent-dashboard.marcospaulo.dev.br** (Caddy
-TLS termination via wildcard cert → Tailscale → NodePort 31541 →
-oauth2-proxy → dashboard ClusterIP).
-
-### Emergency bypass (cookie)
-
-If Logto goes down and you need to access the dashboard before the
-oauth2-proxy restart dance (see `~/.claude/memory/logto-fix.md`),
-`pilot/dashboard.py` can be patched to accept a fallback cookie by
-re-adding the `PRAGENT_DASHBOARD_TOKEN` env path — the route gate is
-isolated in `_is_authed` and the logic is straightforward. The current
-commit intentionally has no bypass because Logto SSO is the single
-source of truth for "who can touch `.pr-review.json`".
-
-## Deploy
-
-The dashboard shares the webhook image, so there's nothing to rebuild
-beyond what the webhook already does. After editing `pilot/dashboard.py`
-or `pilot/dashboard_data.py`, redo the webhook image rebuild + containerd
-import (see `README-webhook.md` § "K8s deployment") and roll both
-deployments.
-
-```bash
-K="microk8s kubectl"
-
-# 1. (one-time) create the Logto app + cookie secret + oauth2-proxy
-#    See ~/.claude/memory/minecraft-sso.md for the SQL INSERT recipe
-#    and ~/k8s/oauth2-proxy-dashboard*.yaml for the manifests.
-
-# 2. apply all pragent-dashboard manifests (dashboard + oauth2-proxy)
-$K apply -f ~/k8s/oauth2-proxy-dashboard.yaml
-$K apply -f ~/k8s/pragent-dashboard.yaml
-
-# 3. roll on image / code changes
-$K -n pragent rollout restart deploy/pragent-dashboard
-$K -n pragent rollout status  deploy/pragent-dashboard --timeout=120s
-$K -n pragent logs -f deploy/pragent-dashboard
-```
-
-K8s manifests:
-
-- `~/k8s/pragent-dashboard.yaml` — Deployment + ClusterIP Service.
-  - `image: pragent-webhook:dashboard` + `imagePullPolicy: Never` —
-    local containerd only, same image as the webhook.
-  - `nodeSelector: kubernetes.io/hostname: kubernets` — pinned to the
-    node holding the `/data` PVC.
-  - `securityContext: runAsNonRoot: true, runAsUser: 10001, runAsGroup:
-    10001, fsGroup: 10001` — matches the image's USER directive;
-    fsGroup makes the RO hostpath volume readable.
-  - `volumeMounts.feedback-data.readOnly: true` — dashboard is
-    read-only over `/data`; mutations go through Gitea, not local fs.
-  - No `readinessProbe` / `livenessProbe` — the dashboard has no
-    `/health` route. If you add one to `pilot/dashboard.py`, add a
-    probe here too.
-  - `resources.requests: {cpu: 100m, memory: 256Mi}` /
-    `limits: {cpu: 500m, memory: 512Mi}` — read-heavy + tiny writes,
-    no opencode subprocess fan-out, much smaller than the webhook.
-  - `Service.type: ClusterIP` — only oauth2-proxy can reach it.
-
-- `~/k8s/oauth2-proxy-dashboard.yaml` — Deployment + ConfigMap +
-  NodePort Service (`oauth2-proxy-dashboard`, NodePort 31541,
-  namespace `pragent`). Same shape as the code-server /
-  minecraft-sso oauth2-proxy. NodePort 31541 was chosen because
-  31540 was the old dashboard NodePort and the 30096..30969 media
-  range + 30350-30351 (other oauth2-proxy NodePorts) were taken.
-
-- `~/k8s/oauth2-proxy-dashboard-secret.yaml` — client-id /
-  client-secret / cookie-secret (mode 600, NOT in git).
-
-## Smoke test
-
-```bash
-# 1. anonymous request → 302 redirect to Logto
-curl -I https://pragent-dashboard.marcospaulo.dev.br/
-
-# 2. pod logs
-microk8s kubectl logs -n pragent -l app=oauth2-proxy-dashboard --tail=50
-microk8s kubectl logs -n pragent -l app=pragent-dashboard --tail=50
-
-# 3. in-cluster direct probe (should 401 without X-Forwarded-User)
-microk8s kubectl port-forward -n pragent svc/pragent-dashboard 8181:80 &
-sleep 2
-curl -I http://localhost:8181/                       # expect 401 + WWW-Authenticate: Basic
-curl -I -H "X-Forwarded-User: marcos@example.com" http://localhost:8181/  # expect 200
-kill %1
-```
-
-The HTML returned with a valid `X-Forwarded-User` should contain a
-`<title>` (whatever the dashboard renders) and **never** `Traceback` or
-any Python exception output. A 401 on the unauthenticated GET is the
-expected behaviour — oauth2-proxy catches it and redirects to Logto.
-
-## Threat model / security notes
-
-- **Behind Logto SSO.** Anonymous traffic gets 302 → Logto. Allowed
-  emails (marcos, thiago) reach the dashboard after Logto login; all
-  others see oauth2-proxy's "not authorized" page. Adding a user is a
-  one-line ConfigMap edit; oauth2-proxy hot-reloads the allowlist.
-- **`PRAGENT_BOT_TOKEN` is Gitea Write scoped** to onboarded repos, so
-  a successful auth bypass on the dashboard is Gitea repo write access,
-  not just read. oauth2-proxy's email allowlist is the only
-  authentication factor — there is no second factor. If this becomes a
-  concern, swap oauth2-proxy for an IdP that supports TOTP/WebAuthn
-  and the dashboard needs no further changes (it just reads the
-  forwarded headers).
-- **CSRF on the edit form.** Per-process random secret embedded as a
-  hidden input + double-submit via the `X-Forwarded-User` context. An
-  attacker would need to (a) steal the user's Logto session cookie
-  from oauth2-proxy and (b) read the rendered HTML to harvest the
-  CSRF token. Both have to happen in the same browser.
-- **Read-only `/data` mount.** The dashboard can't corrupt the
-  harvested SQLite file even if it's compromised. The webhook and the
-  daily cronjob are the only writers.
-- **ClusterIP dashboard Service.** Even if a malicious actor discovered
-  the dashboard's container port, they cannot reach it from outside the
-  cluster — only oauth2-proxy can. NetworkPolicy is the cluster
-  default deny.
-- **`uid 10001` + `runAsNonRoot: true`.** No host-level escalation if
-  the dashboard is popped — it has no caps, no `/proc` mounts.
-- **No author-controlled input is `eval`-ed.** The edit form parses the
-  JSON, validates types / lengths, and re-serialises before the Gitea
-  PUT. The review-side hostile-input concerns from `README-webhook.md`
-  § "Threat model" do **not** apply to the dashboard — the dashboard
-  is a read-mostly viewer over already-harvested, already-posted data.
-
-## Known limitations (pilot)
-
-- Logto SSO is the only auth factor — no per-user sessions, no CSRF
-  token tied to a per-user identity (the per-process CSRF secret is
-  global). Adequate for a single-operator dashboard; not adequate for
-  multi-tenant.
-- No `/health` route — if the dashboard process wedges on a Gitea hang,
-  k8s won't restart it. Add a `/health` route to `pilot/dashboard.py`
-  + a probe here before relying on this in production.
-- The overview is a single-process render over a SQLite file that the
-  daily cronjob also writes. A long Gitea hang during a page render can
-  stall the dashboard until the client request times out (30 s). The
-  underlying SQLite reader is read-only and concurrent-safe, so no
-  data corruption — just a slow page.

pilot/README-langfuse.md

@@ -21,7 +21,7 @@ and lands on the trace's `environment`:
 | `headroom/MiniMax-M2.7`     | `ollama`    |
 | `vllm-qwen38/qwen3.8-27b`   | `ollama`    |
 
-Langfuse takes an environment selector on every dashboard, filter and cost
+Langfuse takes an environment selector on every view, filter and cost
 breakdown, so the two spend stories stay separate inside one project — one key
 pair to rotate instead of two. Tags carry the finer cut:
 `provider:headroom`, `model:<bare>`, `engine:opencode`, `repo:<owner/name>`,
@@ -56,11 +56,11 @@ A model that costs nothing through the headroom proxy is priced against a
 **comparison target** instead: basis `equivalent:<target>`. That covers the
 models absent from `PRICES` (`MiniMax-M2.7` — which is what the webhook
 actually runs — and `glm-5.2:cloud`) as well as entries priced at all zeros
-(the self-hosted vLLM `qwen3.8-27b`). Without this the dashboard would be a
+(the self-hosted vLLM `qwen3.8-27b`). Without this Langfuse would show a
 flat $0.00 line, since the pilot's own path is free.
 
-The target follows the same precedence as the review body, so the PR and the
-dashboard never disagree:
+The target follows the same precedence as the review body, so the PR and
+Langfuse never disagree:
 
     .pr-review.json:cost_target  >  PRAGENT_PRICE_TARGET  >  claude-sonnet-5
 

pilot/README.md

@@ -1,100 +1,83 @@
-# pragent pilot — AI Review bot
+# pragent pilot
 
-A minimal AI code-review bot for Gitea, running as a CI step on the existing
-`act-runner`. This is the **pilot** — a small, self-contained reviewer that
-predates the full `pragent` framework (whose design lives in
-`docs/plans/2026-08-04-pragent-design.md`). The framework will later absorb
-this; until then, this is what runs.
+The pilot is a central, stdlib-only Gitea webhook service. It reviews opted-in
+pull requests with an on-network model, posts inline findings, and emits review
+telemetry to Langfuse. The service is fail-open: a review failure is reported
+as a PR comment and does not block CI.
 
-## How it works
+## Runtime flow
 
-1. You add `pragent-bot` to a repo and commit `.gitea/workflows/ai-review.yml`.
-2. On a PR, you add the **`AI-REVIEW`** label.
-3. Gitea Actions runs the workflow on the `act-runner`; it fetches the PR diff,
-   asks `glm-5.2:cloud` (on-network via the headroom proxy) to review it, and
-   posts the findings back as a PR review authored by `pragent-bot`.
-4. Remove the label to stop re-reviews on further pushes.
+1. Gitea sends a signed `pull_request` webhook.
+2. `webhook_server.py` validates the request, checks the base branch's
+   `.pr-review.json` for `"enabled": true`, and claims `(repo, PR, SHA)`.
+3. `ai_review.review_pr()` fetches the diff, trusted config, and prior reviews.
+4. `opencode_review.py` checks out the PR head in a sanitized temporary
+   directory and runs the review agent. The legacy Ollama-compatible path is
+   still available through `PRAGENT_ENGINE`.
+5. The review output is parsed and normalized, valid post-change line anchors
+   are separated from summary-only findings, and Gitea receives the result.
+6. `langfuse_trace.py` records usage, cost basis, findings, and evaluation
+   scores when Langfuse credentials are configured.
 
-Fail-open: the job always exits 0 and never blocks CI. Errors become a short
-"review failed" comment.
+## Module map
 
-## Onboard a repo (3 steps)
-
-### 1. Add `pragent-bot` as collaborator
-
-Repo → Settings → Collaborators → Add → `pragent-bot` → permission **Write**.
-(Write is required to post reviews/comments.)
-
-Or via API (with an admin/owner token):
-
-```bash
-curl -X PUT -H "Authorization: token $OWNER_TOKEN" \
-  -H "Content-Type: application/json" \
-  -d '{"permission":"write"}' \
-  "http://<gitea-host>:3000/api/v1/repos/OWNER/REPO/collaborators/pragent-bot"
-```
-
-### 2. Add the `PRAGENT_BOT_TOKEN` secret
-
-Repo → Settings → Actions → Secrets → New secret → name `PRAGENT_BOT_TOKEN`,
-value = the bot's access token (ask the platform admin; stored mode-600 at
-`~/.claude/.pragent-bot-token` on the admin host).
-
-### 3. Commit the workflow
-
-Copy `pilot/workflow-template.yml` into the target repo as
-`.gitea/workflows/ai-review.yml` and commit it. That's it.
-
-## Use it
-
-Open a PR (or push to an open one), add the **`AI-REVIEW`** label. The review
-appears within ~30–90s depending on diff size and model latency.
-
-## What's intentionally NOT in the pilot
-
-Deferred to the full framework (by design, see the design doc):
-
-- Attention tiering (trivial/lite/full/oversized) and per-tier cost control.
-- Multiple analyzer fan-out over a shared cached prompt prefix.
-- Prior-comment synthesis (so each push re-posts; the latest review is tagged
-  with the head SHA so it's easy to spot).
-- Inline line comments and status checks.
-- `pragent explain` / `replay` / analytics JSONL.
-- A second forge (GitLab) and the provider matrix.
-
-## Pieces
-
-| File | Role |
+| Module | Responsibility |
 |---|---|
-| `pilot/ai_review.py` | The reviewer script (stdlib only). Single source of truth — fetched at runtime by each repo's workflow. |
-| `pilot/workflow-template.yml` | The Gitea Action consumers copy into `.gitea/workflows/ai-review.yml`. |
-| `tests/pilot/test_ai_review.py` | Unit tests for the pure helpers (no network). |
+| `entrypoints/webhook.py` | HTTP ingress, signature verification, opt-in gate, concurrency |
+| `review/config.py` | Trusted base-branch opt-in policy; transport injected for tests |
+| `entrypoints/gitea.py` | HTTP transport adapter and repository-scoped client |
+| `review/ai_review.py` | Small public review facade |
+| `review/pipeline.py` | Review orchestration, runtime wiring, and compatibility symbols |
+| `review/analysis.py` | Prompt construction, token attribution, and cost helpers |
+| `review/output.py` | Finding parsing, anchor validation, and Markdown rendering |
+| `review/configuration.py` | Repository config parsing, filtering, and prior-review context |
+| `review/adapters.py` | Gitea/model transport and review publishing |
+| `ai_review.py` | Compatibility shim for existing imports and CI execution |
+| `review/model.py` | Anthropic-compatible model adapter and response text extraction |
+| `review/opencode.py` | Hostile-checkout containment and agent execution |
+| `review/diff.py` | Diff compression and prior-review extraction |
+| `feedback/*.py` | Feedback persistence, harvesting, analysis, and Langfuse scores |
+| `observability/langfuse.py` | Fail-open Langfuse ingestion and cost metadata |
+| `observability/cost.py` | Provider price catalog and equivalent-cost calculations |
+| `evaluation/*.py` | Dataset bootstrap, evaluators, and behavioral scoring |
 
-## Run the tests
+The top-level `.py` files are intentionally thin compatibility shims. They keep
+existing workflow commands and imports stable while the implementations live in
+the focused packages above. New code belongs in those packages, not in a shim.
+
+## Onboard a repository
+
+1. Add `pragent-bot` as a Write collaborator.
+2. Commit this file to the default branch:
+
+   ```json
+   {"enabled": true}
+   ```
+
+3. Open or update a pull request.
+
+No per-repository workflow, secret, or label is required for the central
+webhook path. See [`README-webhook.md`](README-webhook.md) for deployment,
+security, and webhook registration details.
+
+## Configuration
+
+| Variable | Default | Purpose |
+|---|---:|---|
+| `GITEA_API` | in-cluster URL | Gitea API base URL |
+| `PRAGENT_BOT_TOKEN` | — | Bot credential |
+| `OLLAMA_URL` / `OLLAMA_MODEL` | headroom / `glm-5.2:cloud` | Legacy model path |
+| `PRAGENT_ENGINE` | `opencode` | `opencode` or legacy model path |
+| `DIFF_MAX_CHARS` | `150000` | Diff input cap |
+| `PRAGENT_MAX_CONCURRENT_REVIEWS` | `2` | Process concurrency bound |
+| `LANGFUSE_HOST` + keys | unset | Enables telemetry; unset is a no-op |
+
+## Tests
 
 ```bash
-cd ~/Projects/pragent
-PYTHONPATH=pilot python3 -m pytest tests/pilot/    # if pytest available
-# or, without pytest:
-python3 - <<'PY'
-import os, sys, importlib.util
-sys.path.insert(0, os.path.abspath("pilot"))
-import ai_review  # noqa: F401
-spec = importlib.util.spec_from_file_location("t", "tests/pilot/test_ai_review.py")
-m = importlib.util.module_from_spec(spec); spec.loader.exec_module(m)
-fails = 0
-for n in sorted(x for x in dir(m) if x.startswith("test_")):
-    try: getattr(m, n)(); print("PASS", n)
-    except Exception as e: fails += 1; print("FAIL", n, e)
-print("failed:", fails)
-PY
+python3 -m pytest tests -q
 ```
 
-## Configuration knobs (env in the workflow)
-
-| Env | Default | Purpose |
-|---|---|---|
-| `OLLAMA_MODEL` | `glm-5.2:cloud` | Model id passed to the headroom proxy. |
-| `OLLAMA_MAX_TOKENS` | `6000` | Output token cap. |
-| `DIFF_MAX_CHARS` | `150000` | Diff truncation cap (with a noted truncation marker). |
-| `OLLAMA_URL` | `http://<model-proxy-host>:8789` | headroom proxy (tailnet). If the act-runner can't reach the tailnet IP, expose 8789 as an in-cluster Service+Endpoints and set this to the cluster DNS name. |
+Tests are grouped under `tests/pilot/*_tests/`, matching the source domains.
+They use mocked transports and local fixtures and do not require Gitea,
+Langfuse, a model endpoint, or network access.

pilot/cost_model.py

@@ -1,434 +1,7 @@
-#!/usr/bin/env python3
-"""pragent pilot — per-review cost model.
-
-Answers "what would this cost on a paid API?" for the pilot's agent loop. The
-pilot currently runs on `glm-5.2:cloud` through the on-network headroom proxy at
-no per-token charge, so every review's measured usage is *free but real*: it
-tells us exactly what the same work would bill on Claude or GPT.
-
-The model is deliberately explicit rather than a single fudge factor, because
-the dominant cost in an agent loop is not the diff — it is **resending the
-conversation on every step**. A 12-step review re-reads its own prefix 12 times.
-Prompt caching is what makes that affordable, and whether caching is on changes
-the answer by ~3x, so it's a parameter, not an assumption.
-
-Token accounting per review:
-
-    step 1 input  = prefix + brief
-    step k input  = prefix + brief + (tool results accumulated through k-1)
-    total input   = sum over steps
-    cached        = the prefix + brief part of steps 2..n (stable, byte-identical)
-    uncached      = step 1 in full + the growing tool-result tail
-
-`prefix` = system + tool schemas + agent definition + the skills this tier loads.
-Those sizes are MEASURED from the files in this repo (see `measure_factory`),
-not guessed. Diff size, file reads, and step count are per-tier assumptions from
-the `attention-tiering` skill's budgets — override them on the CLI to fit your
-own repos.
-
-Prices are per million tokens, from the providers' published pricing pages
-(fetched 2026-08-18 — re-check before quoting):
-  https://platform.claude.com/docs/en/about-claude/pricing
-  https://developers.openai.com/api/docs/pricing
-
-Usage:
-    python3 pilot/cost_model.py                       # all tiers, all models
-    python3 pilot/cost_model.py --prs-per-month 350
-    python3 pilot/cost_model.py --mix 5,35,55,5       # trivial,lite,full,oversized %
-    python3 pilot/cost_model.py --no-cache            # what caching is worth
-"""
-
-from __future__ import annotations
-
-import argparse
-import os
-from dataclasses import dataclass, field
-
-CHARS_PER_TOKEN = 4  # English prose/code rule of thumb; ±15% is normal
-
-
-# ---------------------------------------------------------------------------
-# Prices — USD per million tokens
-# ---------------------------------------------------------------------------
-
-
-@dataclass(frozen=True)
-class Price:
-    """Per-MTok prices. `cache_write` and `cache_read` are absolute rates, not
-    multipliers, so providers with different cache economics stay comparable.
-
-    `provider` is the opencode provider name (`headroom`, `vllm-qwen38`, ...). It
-    doubles as the dispatch key for `.pr-review.json:model` overrides — when
-    a per-repo override is set, `_resolve_display_model` returns
-    `f"{provider}/{key}"` so the opencode subprocess routes correctly.
-    Default `headroom` preserved for the existing roster."""
-
-    name: str
-    input: float
-    output: float
-    cache_write: float
-    cache_read: float
-    provider: str = "headroom"
-
-    @property
-    def batch_input(self) -> float:
-        return self.input / 2
-
-    @property
-    def batch_output(self) -> float:
-        return self.output / 2
-
-
-# Anthropic: cache write = 1.25x input (5-minute TTL), cache read = 0.1x input.
-# OpenAI: cached input is a published rate (0.1x input); there is no separate
-# cache-write charge — writes are billed as ordinary input.
-PRICES: dict[str, Price] = {
-    "claude-opus-5": Price("Claude Opus 5", 5.00, 25.00, 6.25, 0.50),
-    "claude-sonnet-5": Price("Claude Sonnet 5", 2.00, 10.00, 2.50, 0.20),
-    "claude-haiku-4-5": Price("Claude Haiku 4.5", 1.00, 5.00, 1.25, 0.10),
-    "gpt-5.6-sol": Price("GPT-5.6 Sol", 5.00, 30.00, 5.00, 0.50),
-    "gpt-5.6-terra": Price("GPT-5.6 Terra", 2.00, 12.00, 2.00, 0.20),
-    "gpt-5.6-luna": Price("GPT-5.6 Luna", 0.20, 1.20, 0.20, 0.02),
-    # OpenAI — cached_input 0.1x, no separate cache_write
-    "gpt-5": Price("GPT-5", 1.25, 10.00, 1.25, 0.125),
-    "gpt-5-mini": Price("GPT-5 mini", 0.25, 2.00, 0.25, 0.025),
-    # Google Gemini — cache_write = input
-    "gemini-2.5-pro": Price("Gemini 2.5 Pro", 1.875, 12.50, 1.875, 0.1875),
-    "gemini-2.5-flash": Price("Gemini 2.5 Flash", 0.30, 2.50, 0.30, 0.03),
-    # xAI Grok — cache_write = input
-    "grok-4.5": Price("Grok 4.5", 2.00, 6.00, 2.00, 0.30),
-    "grok-4.3": Price("Grok 4.3", 1.25, 2.50, 1.25, 0.20),
-    # Self-hosted — AI workstation RTX 3090, vLLM + DFlash2 spec-decode, no
-    # per-token charge. provider="vllm-qwen38" so the opencode subprocess
-    # routes via the matching provider block in opencode.json
-    # (baseURL=http://192.168.1.79:18020/v1). Equivalent-cost column reads $0
-    # — the cost-comparison signal is that the same work would bill $X on a
-    # paid model.
-    "qwen3.8-27b": Price("Qwen3.8-27B (vLLM, MTP, 150k ctx)", 0.0, 0.0, 0.0, 0.0, provider="vllm-qwen38"),
-}
-
-
-# ---------------------------------------------------------------------------
-# Factory footprint — measured from this repo
-# ---------------------------------------------------------------------------
-
-_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
-
-# Skills the primary always loads, and the conditional ones per tier. Mirrors
-# the load table in .opencode/agents/pragent.md.
-ALWAYS_SKILLS = ("review-methodology", "findings-schema", "attention-tiering")
-TIER_SKILLS: dict[str, tuple[str, ...]] = {
-    "trivial": (),
-    "lite": ("comment-craft",),
-    "full": ("linter-playbook", "security-lens", "comment-craft"),
-    "oversized": ("linter-playbook", "security-lens", "comment-craft", "malicious-change"),
-}
-
-# opencode's own system prompt + the JSON tool schemas it sends (read, grep,
-# glob, bash, webfetch, skill, task, …). Not in this repo, so this is the one
-# component that is an estimate rather than a measurement.
-HARNESS_TOKENS = 3500
-
-
-def _tok(path: str) -> int:
-    try:
-        with open(path, "rb") as f:
-            return len(f.read()) // CHARS_PER_TOKEN
-    except OSError:
-        return 0
-
-
-def measure_factory(root: str = _ROOT) -> dict[str, int]:
-    """Token size of each prompt component, measured from the files on disk."""
-    out = {"agent": _tok(os.path.join(root, ".opencode", "agents", "pragent.md"))}
-    skills_dir = os.path.join(root, ".opencode", "skills")
-    if os.path.isdir(skills_dir):
-        for name in sorted(os.listdir(skills_dir)):
-            p = os.path.join(skills_dir, name, "SKILL.md")
-            if os.path.isfile(p):
-                out[f"skill:{name}"] = _tok(p)
-    for lens in ("security", "tests", "perf"):
-        out[f"subagent:{lens}"] = _tok(os.path.join(root, ".opencode", "agents", f"{lens}.md"))
-    return out
-
-
-def prefix_tokens(tier: str, factory: dict[str, int]) -> int:
-    """Stable per-step prefix: harness + agent definition + loaded skills."""
-    total = HARNESS_TOKENS + factory.get("agent", 0)
-    for s in ALWAYS_SKILLS + TIER_SKILLS.get(tier, ()):
-        total += factory.get(f"skill:{s}", 0)
-    return total
-
-
-# ---------------------------------------------------------------------------
-# Per-tier workload assumptions
-# ---------------------------------------------------------------------------
-
-
-@dataclass
-class Tier:
-    """One tier's workload. Defaults follow the `attention-tiering` budgets."""
-
-    name: str
-    diff_tokens: int          # the diff as it lands in the brief
-    steps: int                # model turns in the agent loop
-    file_reads: int           # files read from the checkout
-    tokens_per_read: int      # avg tokens returned per read/grep/linter result
-    output_tokens: int        # assistant output across all steps (incl. reasoning)
-    subagents: int = 0        # lens subagents spawned
-    brief_fixed: int = 600    # brief template + PR meta + prior reviews
-    share: float = 0.0        # fraction of PRs at this tier (for the monthly mix)
-    _factory: dict = field(default_factory=dict, repr=False)
-
-
-DEFAULT_TIERS = [
-    # diff_tok  steps  reads  tok/read  output  subs  share
-    Tier("trivial",     400,  2,  0,     0,      600,  0, share=0.05),
-    Tier("lite",       1500,  6,  4,  2000,     2500,  0, share=0.35),
-    Tier("full",       6000, 24, 20,  3300,    12000,  0, share=0.55),
-    Tier("oversized", 25000, 35, 30,  3500,    20000,  2, share=0.05),
-]
-
-
-# ---------------------------------------------------------------------------
-# Observed runs — the calibration anchor
-# ---------------------------------------------------------------------------
-
-# Real usage reported by opencode's step_finish events. Keep this list
-# events. Keep this list append-only: it is the only thing separating this model
-# from a guess, and the first entry corrected the tier assumptions by ~15x.
-OBSERVED_RUNS: list[dict] = [
-    {
-        "label": "internal/hardening-PR (16 files, 1020 insertions / 91 deletions)",
-        "date": "2026-08-18",
-        "tier": "full",
-        "diff_tokens": 17_600,     # 16 files, 1020 insertions / 91 deletions
-        "steps": 28,
-        "duration_s": 348.3,
-        "input": 2_071_025,
-        "output": 17_303,
-        "cache_read": 0,
-        "cache_write": 0,
-        "subagents": 0,
-    },
-    {
-        "label": "internal/hardening-PR (same PR, two commits later)",
-        "date": "2026-08-18",
-        "tier": "full",
-        "diff_tokens": 21_000,     # same PR, two commits later
-        "steps": 31,
-        "duration_s": 189.8,
-        "input": 2_213_077,
-        "output": 9_058,
-        "cache_read": 0,
-        "cache_write": 0,
-        "subagents": 0,
-    },
-    # A third run of the same PR (sha 2613b3e, 31 steps' worth of work in 330s)
-    # ended without a parseable findings block and so reported no usage at all —
-    # the reason `salvage_summary` now keeps the usage section on that path.
-]
-
-
-def observed_usage(run: dict) -> Usage:
-    return Usage(
-        uncached_input=run["input"] - run.get("cache_read", 0),
-        cached_input=run.get("cache_read", 0),
-        cache_writes=run.get("cache_write", 0),
-        output=run["output"],
-    )
-
-
-@dataclass
-class Usage:
-    uncached_input: int = 0
-    cached_input: int = 0
-    cache_writes: int = 0
-    output: int = 0
-
-    @property
-    def total_input(self) -> int:
-        return self.uncached_input + self.cached_input
-
-
-def tier_usage(tier: Tier, factory: dict[str, int], caching: bool = True) -> Usage:
-    """Token usage for one review at this tier.
-
-    The agent loop resends the whole conversation each step. The prefix + brief
-    are byte-identical across steps, so with caching they are written once and
-    read back on every later step; the tool-result tail grows and is charged as
-    ordinary input. Without caching every step pays full input price for
-    everything it has accumulated — which is the quadratic term that makes an
-    uncached agent loop expensive.
-    """
-    prefix = prefix_tokens(tier.name, factory)
-    stable = prefix + tier.brief_fixed + tier.diff_tokens
-
-    # Tool results arrive one per step, after the first.
-    result_steps = max(0, min(tier.file_reads, tier.steps - 1))
-    per_result = tier.tokens_per_read
-
-    u = Usage(output=tier.output_tokens)
-
-    if caching:
-        u.cache_writes = stable
-        u.cached_input = stable * max(0, tier.steps - 1)
-        u.uncached_input = 0
-    else:
-        u.uncached_input = stable * tier.steps
-
-    # The growing tail of tool results: a result produced at step i is resent on
-    # every step after it, so it is counted (steps - i) times.
-    tail = 0
-    for i in range(1, result_steps + 1):
-        tail += per_result * (tier.steps - i)
-    u.uncached_input += tail
-
-    # Each lens subagent is its own loop: its own prefix, the diff, a few reads.
-    for _ in range(tier.subagents):
-        sub_prefix = HARNESS_TOKENS + factory.get("subagent:security", 600)
-        sub_stable = sub_prefix + tier.diff_tokens
-        sub_steps = 6
-        if caching:
-            u.cache_writes += sub_stable
-            u.cached_input += sub_stable * (sub_steps - 1)
-        else:
-            u.uncached_input += sub_stable * sub_steps
-        for i in range(1, 4):
-            u.uncached_input += per_result * (sub_steps - i)
-        u.output += 1500
-
-    return u
-
-
-def cost(u: Usage, price: Price, batch: bool = False) -> float:
-    """USD for one review's usage at these prices."""
-    inp = price.batch_input if batch else price.input
-    out = price.batch_output if batch else price.output
-    cw = price.cache_write / 2 if batch else price.cache_write
-    cr = price.cache_read / 2 if batch else price.cache_read
-    return (
-        u.uncached_input * inp
-        + u.cached_input * cr
-        + u.cache_writes * cw
-        + u.output * out
-    ) / 1_000_000
-
-
-# ---------------------------------------------------------------------------
-# Reporting
-# ---------------------------------------------------------------------------
-
-
-def blended_cost(tiers: list[Tier], factory: dict, price: Price, caching: bool) -> float:
-    """Weighted cost of one average PR across the tier mix."""
-    total_share = sum(t.share for t in tiers) or 1.0
-    return sum(
-        cost(tier_usage(t, factory, caching), price) * (t.share / total_share)
-        for t in tiers
-    )
-
-
-def report(tiers: list[Tier], prs_per_month: int, caching: bool, models: list[str]) -> str:
-    factory = measure_factory()
-    lines: list[str] = []
-
-    lines.append(f"Factory footprint (measured, {CHARS_PER_TOKEN} chars/token):")
-    for k, v in sorted(factory.items()):
-        lines.append(f"  {k:<34} {v:>6,} tok")
-    lines.append(f"  {'harness (opencode + tool schemas, est.)':<34} {HARNESS_TOKENS:>6,} tok")
-    lines.append("")
-
-    lines.append(f"Per-review tokens  (prompt caching: {'on' if caching else 'OFF'})")
-    lines.append(f"  {'tier':<11} {'prefix':>8} {'uncached':>10} {'cached':>10} {'cwrite':>8} {'output':>8}")
-    for t in tiers:
-        u = tier_usage(t, factory, caching)
-        lines.append(
-            f"  {t.name:<11} {prefix_tokens(t.name, factory):>8,} {u.uncached_input:>10,} "
-            f"{u.cached_input:>10,} {u.cache_writes:>8,} {u.output:>8,}"
-        )
-    lines.append("")
-
-    lines.append("Cost per review (USD)")
-    header = f"  {'model':<18}" + "".join(f"{t.name:>12}" for t in tiers) + f"{'blended':>12}"
-    lines.append(header)
-    for key in models:
-        p = PRICES[key]
-        row = f"  {p.name:<18}"
-        for t in tiers:
-            row += f"{cost(tier_usage(t, factory, caching), p):>12.4f}"
-        row += f"{blended_cost(tiers, factory, p, caching):>12.4f}"
-        lines.append(row)
-    lines.append("")
-
-    mix = ", ".join(f"{t.name} {t.share:.0%}" for t in tiers)
-    lines.append(f"Monthly at {prs_per_month} PRs/month  (mix: {mix})")
-    lines.append(f"  {'model':<18} {'per PR':>10} {'per month':>12} {'batch -50%':>12}")
-    for key in models:
-        p = PRICES[key]
-        per_pr = blended_cost(tiers, factory, p, caching)
-        lines.append(
-            f"  {p.name:<18} {per_pr:>10.4f} {per_pr * prs_per_month:>12.2f}"
-            f" {per_pr * prs_per_month / 2:>12.2f}"
-        )
-    lines.append("")
-    lines.append("Batch column applies the 50% async discount; it is shown for scale only —")
-    lines.append("PR review is latency-sensitive and a stateful agent loop is not batchable.")
-    lines.append("")
-    lines.append(observed_report(models))
-    return "\n".join(lines)
-
-
-def observed_report(models: list[str]) -> str:
-    """Price the runs actually measured through the opencode usage telemetry."""
-    if not OBSERVED_RUNS:
-        return "No observed runs recorded yet."
-    lines = ["Observed runs (measured via opencode step_finish events)"]
-    for run in OBSERVED_RUNS:
-        u = observed_usage(run)
-        lines.append(
-            f"  {run['label']} — tier {run['tier']}, {run['steps']} steps, "
-            f"{run['duration_s']:.0f}s, {run['input']:,} in / {run['output']:,} out, "
-            f"cache {run['cache_read']:,} read / {run['cache_write']:,} write"
-        )
-        row = "   "
-        for key in models:
-            p = PRICES[key]
-            row += f" {p.name}: ${cost(u, p):.2f} "
-        lines.append(row)
-    lines.append("")
-    lines.append("  NOTE: the pilot's headroom/glm-5.2 path reports zero cache read and zero")
-    lines.append("  cache write, i.e. prompt caching is NOT in play today. On a provider where")
-    lines.append("  it is, the stable prefix (agent + skills + brief + diff, resent every step)")
-    lines.append("  drops to 0.1x — worth roughly a third of the bill on a run like the one")
-    lines.append("  above. Budget with caching OFF until the measured cache columns are nonzero.")
-    return "\n".join(lines)
-
-
-def main(argv: list[str] | None = None) -> int:
-    ap = argparse.ArgumentParser(description="pragent per-review cost model")
-    ap.add_argument("--prs-per-month", type=int, default=350)
-    ap.add_argument("--mix", default="", help="trivial,lite,full,oversized as percentages")
-    ap.add_argument("--no-cache", action="store_true", help="model without prompt caching")
-    ap.add_argument("--models", default=",".join(PRICES))
-    args = ap.parse_args(argv)
-
-    tiers = DEFAULT_TIERS
-    if args.mix:
-        shares = [float(x) for x in args.mix.split(",")]
-        if len(shares) != len(tiers):
-            ap.error(f"--mix needs {len(tiers)} comma-separated values")
-        for t, s in zip(tiers, shares):
-            t.share = s / 100.0
-
-    models = [m.strip() for m in args.models.split(",") if m.strip()]
-    unknown = [m for m in models if m not in PRICES]
-    if unknown:
-        ap.error(f"unknown model(s): {', '.join(unknown)}")
-
-    print(report(tiers, args.prs_per_month, not args.no_cache, models))
-    return 0
-
-
+"""Compatibility import for the cost catalog."""
+import importlib
+import sys
+_module = importlib.import_module("observability.cost")
+sys.modules[__name__] = _module
 if __name__ == "__main__":
-    raise SystemExit(main())
+    raise SystemExit(_module.main())

pilot/dashboard.py

@@ -1,754 +0,0 @@
-#!/usr/bin/env python3
-"""pragent pilot — read-mostly dashboard.
-
-Stdlib HTTP server (mirrors `webhook_server.py`'s BaseHTTPRequestHandler +
-ThreadingHTTPServer shape) that renders three views off the feedback SQLite:
-
-  GET /                                    overview
-  GET /r/<owner>/<name>                    repo summary + edit form
-  GET /r/<owner>/<name>/<index>            one PR's findings
-  GET /r/<owner>/<name>/<index>/raw        raw Markdown body (via Gitea)
-  GET /static/style.css                    CSS
-  POST /r/<owner>/<name>/edit              mutate .pr-review.json (Tasks C+D)
-
-Auth: oauth2-proxy fronts this service in-cluster. Every route except
-`/static/*` requires the `X-Forwarded-User` header (set by oauth2-proxy
-once the user has logged in via Logto). Missing header → 401 +
-`WWW-Authenticate: Basic realm="pragent-dashboard"` so oauth2-proxy
-intercepts the response.
-
-DB: `PRAGENT_FEEDBACK_DB` points at the SQLite file the webhook server
-also writes. Per-request open (SQLite is cheap, no concurrency hazard,
-no stale-conn surprise after the file rotates).
-
-All HTML is rendered via `string.Template` and every dynamic value is
-escaped with `html.escape(..., quote=True)`. No `.format`, no f-string
-templates — see `_render_*` for the discipline.
-"""
-
-from __future__ import annotations
-
-import base64
-import datetime
-import html
-import json
-import os
-import secrets
-import string
-import urllib.error
-import urllib.parse
-import urllib.request
-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
-
-from pilot import dashboard_data
-
-
-# ---------------------------------------------------------------------------
-# Config
-# ---------------------------------------------------------------------------
-
-FEEDBACK_DB = ""  # legacy; readers should call _feedback_db()
-PORT = int(os.environ.get("DASHBOARD_PORT", "8081"))
-
-GITEA_API = ""  # legacy; readers should call _gitea_api()
-BOT_TOKEN = ""  # legacy; readers should call _bot_token()
-
-# CSRF secret for the edit form. Regenerated per process (each Python
-# interpreter launch). Behind oauth2-proxy this is enough — only an
-# already-authenticated same-tab request can read this and echo it back.
-_CSRF_SECRET: str = secrets.token_urlsafe(24)
-
-
-# ---------------------------------------------------------------------------
-# Lazy config readers — tests set env after import, so each request re-reads.
-# Production: env is fixed for the process lifetime; the per-request lookup is
-# a dict access, not a syscall.
-# ---------------------------------------------------------------------------
-
-
-def _feedback_db() -> str:
-    return os.environ.get("PRAGENT_FEEDBACK_DB", "")
-
-
-def _bot_token() -> str:
-    return os.environ.get("PRAGENT_BOT_TOKEN", "")
-
-
-def _gitea_api() -> str:
-    return os.environ.get("GITEA_API", "http://gitea-http.gitea.svc.cluster.local:3000")
-
-
-# ---------------------------------------------------------------------------
-# Stylesheet — small, dark-mode-friendly, deliberately under 100 lines
-# ---------------------------------------------------------------------------
-
-STYLE_CSS = """
-:root { color-scheme: light dark; }
-* { box-sizing: border-box; }
-body {
-  font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", system-ui, sans-serif;
-  margin: 0; padding: 0;
-  background: #0f1115; color: #e6e6e6;
-  line-height: 1.5;
-}
-header {
-  background: #1a1d23; padding: 12px 20px;
-  border-bottom: 1px solid #2a2f38;
-  display: flex; align-items: center; gap: 18px;
-}
-header h1 { font-size: 18px; margin: 0; }
-header nav a {
-  color: #8ab4f8; text-decoration: none; margin-right: 12px;
-}
-header nav a:hover { text-decoration: underline; }
-main { padding: 20px; max-width: 1100px; margin: 0 auto; }
-h2 { margin-top: 24px; font-size: 16px; color: #c9d1d9; }
-.metric-row { display: flex; gap: 16px; flex-wrap: wrap; margin-bottom: 16px; }
-.metric {
-  background: #1a1d23; padding: 14px 18px; border-radius: 8px;
-  min-width: 140px; border: 1px solid #2a2f38;
-}
-.metric .v { font-size: 28px; font-weight: 600; }
-.metric .l { font-size: 12px; color: #8b949e; text-transform: uppercase; letter-spacing: 0.04em; }
-table { width: 100%; border-collapse: collapse; margin: 8px 0 16px; font-size: 14px; }
-th, td { text-align: left; padding: 6px 10px; border-bottom: 1px solid #2a2f38; }
-th { color: #8b949e; font-weight: 500; text-transform: uppercase; font-size: 11px; letter-spacing: 0.04em; }
-tr:hover td { background: #161922; }
-.sev-critical { color: #ff7b72; font-weight: 600; }
-.sev-high { color: #f0883e; }
-.sev-medium { color: #d29922; }
-.sev-low { color: #8b949e; }
-.muted { color: #8b949e; font-size: 12px; }
-.sparkline { font-family: ui-monospace, "SF Mono", monospace; letter-spacing: 1px; }
-form { background: #1a1d23; padding: 14px 18px; border-radius: 8px; border: 1px solid #2a2f38; margin: 12px 0; }
-form label { display: block; margin: 8px 0 4px; color: #c9d1d9; font-size: 13px; }
-form input[type=text], form textarea, form select {
-  background: #0f1115; color: #e6e6e6; border: 1px solid #2a2f38;
-  border-radius: 4px; padding: 6px 8px; font-family: inherit; font-size: 14px;
-  width: 100%;
-}
-form textarea { min-height: 80px; }
-form .row { display: flex; gap: 8px; align-items: center; margin-top: 12px; }
-form button {
-  background: #2ea043; color: white; border: none; border-radius: 4px;
-  padding: 6px 14px; font-size: 14px; cursor: pointer;
-}
-form button:hover { background: #3fb950; }
-.flash { background: #3d1e1e; color: #ff7b72; padding: 8px 12px; border-radius: 4px; margin-bottom: 12px; }
-code { background: #161922; padding: 1px 4px; border-radius: 3px; font-size: 13px; }
-pre { background: #161922; padding: 12px; border-radius: 6px; overflow-x: auto; }
-"""
-
-
-# ---------------------------------------------------------------------------
-# Templates — string.Template so dynamic values are always escaped explicitly
-# ---------------------------------------------------------------------------
-
-_BASE = string.Template("""<!doctype html>
-<html lang="en">
-<head>
-<meta charset="utf-8">
-<title>${title}</title>
-<link rel="stylesheet" href="/static/style.css">
-</head>
-<body>
-<header>
-  <h1>pragent dashboard</h1>
-  <nav>
-    <a href="/">Home</a>
-    <a href="/r/${repos_first}">repos</a>
-  </nav>
-  <span class="muted" style="margin-left:auto">${db_status}</span>
-</header>
-<main>
-${body}
-</main>
-</body>
-</html>""")
-
-
-_OVERVIEW = string.Template("""<h2>Overview</h2>
-<div class="metric-row">
-  <div class="metric"><div class="v">${total_reviews}</div><div class="l">reviews</div></div>
-  <div class="metric"><div class="v">${total_findings}</div><div class="l">findings</div></div>
-  <div class="metric"><div class="v">${total_repos}</div><div class="l">repos</div></div>
-  <div class="metric"><div class="v">${last_30d_reviews}</div><div class="l">last 30d</div></div>
-</div>
-
-<h2>Last 7 days</h2>
-<div class="sparkline">${sparkline}</div>
-<div class="muted">total cost: $${total_cost_usd} — no per-review cost logged</div>
-
-<h2>Top repos</h2>
-${top_repos_table}
-""")
-
-
-_REPO = string.Template("""<h2>Repo: <code>${repo}</code></h2>
-<div class="metric-row">
-  <div class="metric"><div class="v">${total_runs}</div><div class="l">runs</div></div>
-  <div class="metric"><div class="v">${sev_critical}</div><div class="l sev-critical">critical</div></div>
-  <div class="metric"><div class="v">${sev_high}</div><div class="l sev-high">high</div></div>
-  <div class="metric"><div class="v">${sev_medium}</div><div class="l sev-medium">medium</div></div>
-  <div class="metric"><div class="v">${sev_low}</div><div class="l sev-low">low</div></div>
-</div>
-
-<h2>Edit .pr-review.json</h2>
-${flash}
-<form method="post" action="/r/${repo_url}/edit">
-  <input type="hidden" name="_csrf" value="${csrf}">
-  <label for="static_message">Static banner message (max 400 chars)</label>
-  <textarea id="static_message" name="static_message" maxlength="400">${current_static_message}</textarea>
-  <label for="model">Model (PRICES keys)</label>
-  <select id="model" name="model">${model_options}</select>
-  <div class="row">
-    <button type="submit">Save</button>
-    <span class="muted">posted via the bot identity; one commit on the base branch</span>
-  </div>
-</form>
-
-<h2>Top findings (by occurrence)</h2>
-${top_findings_table}
-
-<h2>Runs by day (last 30d)</h2>
-${runs_by_day_table}
-
-<h2>Reviews</h2>
-${reviews_table}
-""")
-
-
-_PR = string.Template("""<h2>PR <code>${repo}</code> #${pr}</h2>
-<div class="muted">head sha: <code>${head_sha}</code></div>
-<div class="muted">posted_at: ${posted_at_iso}</div>
-<div class="muted">review_id_gitea: ${review_id_gitea} · body_comment_id: ${body_comment_id}</div>
-
-<h2>Findings</h2>
-${findings_table}
-
-<p><a href="/r/${repo_url}/${pr}/raw">raw review body (Markdown)</a></p>
-""")
-
-
-# ---------------------------------------------------------------------------
-# Small helpers
-# ---------------------------------------------------------------------------
-
-
-def _esc(s) -> str:
-    """HTML-escape any value to a string."""
-    return html.escape(str(s), quote=True)
-
-
-def _ts_iso(ts: int) -> str:
-    if not ts:
-        return "—"
-    return datetime.datetime.fromtimestamp(int(ts), tz=datetime.timezone.utc).isoformat()
-
-
-def _sparkline(buckets: list[dict]) -> str:
-    """7-bucket sparkline as unicode bars."""
-    bars = "▁▂▃▄▅▆▇█"
-    if not buckets:
-        return ""
-    mx = max((b.get("count", 0) for b in buckets), default=0) or 1
-    out = []
-    for b in buckets:
-        n = b.get("count", 0)
-        idx = min(len(bars) - 1, int(round(n / mx * (len(bars) - 1))))
-        out.append(bars[idx])
-    return "".join(out)
-
-
-# ---------------------------------------------------------------------------
-# Renderers — one per page
-# ---------------------------------------------------------------------------
-
-
-def _overview_body(data: dict) -> str:
-    top_rows = "".join(
-        f"<tr><td><a href=\"/r/{_esc(r['repo'])}\">{_esc(r['repo'])}</a></td>"
-        f"<td>{int(r['run_count'])}</td>"
-        f"<td class=\"muted\">{_ts_iso(int(r['last_seen']))}</td></tr>"
-        for r in data.get("top_repos", [])
-    ) or "<tr><td class=\"muted\">no reviews yet</td></tr>"
-    top_table = f"<table><thead><tr><th>repo</th><th>runs</th><th>last seen</th></tr></thead><tbody>{top_rows}</tbody></table>"
-    return _OVERVIEW.substitute(
-        total_reviews=_esc(data.get("total_reviews", 0)),
-        total_findings=_esc(data.get("total_findings", 0)),
-        total_repos=_esc(data.get("total_repos", 0)),
-        last_30d_reviews=_esc(data.get("last_30d_reviews", 0)),
-        sparkline=_esc(_sparkline(data.get("daily", []))),
-        total_cost_usd=f"{float(data.get('total_cost_usd', 0.0)):.2f}",
-        top_repos_table=top_table,
-    )
-
-
-def _repo_body(data: dict, *, repo_url: str, csrf: str, current_model: str,
-               current_static_message: str, flash: str = "") -> str:
-    fbs = data.get("findings_by_severity", {})
-    tf = data.get("top_findings", [])
-
-    # Top findings table.
-    if tf:
-        rows = "".join(
-            f"<tr><td><code>{_esc(f['path'])}:{_esc(f['line'])}</code></td>"
-            f"<td class=\"sev-{_esc(f.get('severity', 'low').lower())}\">{_esc(f.get('severity', ''))}</td>"
-            f"<td>{_esc(f.get('problem', ''))}</td>"
-            f"<td>{int(f.get('occurrences', 0))}</td>"
-            f"<td>+{int(f.get('upvotes', 0))} / -{int(f.get('downvotes', 0))}</td>"
-            f"<td>{'resolved' if int(f.get('resolved', 0)) else 'open'}</td>"
-            f"<td>{int(f.get('reply_count', 0))}</td></tr>"
-            for f in tf
-        )
-        top_findings_table = (
-            "<table><thead><tr><th>location</th><th>severity</th>"
-            "<th>problem</th><th>occurrences</th><th>votes</th>"
-            "<th>state</th><th>replies</th></tr></thead><tbody>"
-            f"{rows}</tbody></table>"
-        )
-    else:
-        top_findings_table = "<p class=\"muted\">no findings yet</p>"
-
-    # Runs by day.
-    runs = data.get("runs_by_day", [])
-    if runs:
-        rows = "".join(
-            f"<tr><td>{_esc(r['date'])}</td><td>{int(r.get('count', 0))}</td></tr>"
-            for r in runs
-        )
-        runs_by_day_table = (
-            "<table><thead><tr><th>date</th><th>runs</th></tr></thead>"
-            f"<tbody>{rows}</tbody></table>"
-        )
-    else:
-        runs_by_day_table = "<p class=\"muted\">no runs in the last 30 days</p>"
-
-    # Reviews list — derived from finding timestamps; cheap because we
-    # just enumerate the repo's review rows.
-    reviews_table = _repo_reviews_table(repo_url, data.get("recent_reviews", []))
-
-    # Model select (Task D) — sorted PRICES keys + "keep current".
-    from cost_model import PRICES  # local: pilot-only dep
-    model_options = (
-        f"<option value=\"\">— keep current ({_esc(current_model or 'unset')}) —</option>"
-        + "".join(
-            f"<option value=\"{_esc(k)}\" {'selected' if k == current_model else ''}>{_esc(k)}</option>"
-            for k in sorted(PRICES)
-        )
-    )
-
-    return _REPO.substitute(
-        repo=_esc(data.get("repo", "")),
-        repo_url=_esc(repo_url),
-        total_runs=_esc(data.get("total_runs", 0)),
-        sev_critical=_esc(fbs.get("critical", 0)),
-        sev_high=_esc(fbs.get("high", 0)),
-        sev_medium=_esc(fbs.get("medium", 0)),
-        sev_low=_esc(fbs.get("low", 0)),
-        csrf=_esc(csrf),
-        current_static_message=_esc(current_static_message),
-        model_options=model_options,
-        flash=_esc(flash),
-        top_findings_table=top_findings_table,
-        runs_by_day_table=runs_by_day_table,
-        reviews_table=reviews_table,
-    )
-
-
-def _repo_reviews_table(repo_url: str, rows: list[dict]) -> str:
-    if not rows:
-        return "<p class=\"muted\">no reviews yet</p>"
-    out = "<table><thead><tr><th>PR</th><th>head sha</th><th>posted</th></tr></thead><tbody>"
-    for r in rows:
-        out += (
-            f"<tr><td><a href=\"/r/{_esc(repo_url)}/{int(r['pr'])}\">#{int(r['pr'])}</a></td>"
-            f"<td><code>{_esc(r['head_sha'][:10])}</code></td>"
-            f"<td class=\"muted\">{_ts_iso(int(r.get('posted_at', 0)))}</td></tr>"
-        )
-    out += "</tbody></table>"
-    return out
-
-
-def _pr_body(data: dict, *, repo_url: str) -> str:
-    findings = data.get("findings", [])
-    if findings:
-        rows = "".join(
-            f"<tr><td><code>{_esc(f['path'])}:{_esc(f['line'])}</code></td>"
-            f"<td class=\"sev-{_esc(f.get('severity', 'low').lower())}\">{_esc(f.get('severity', ''))}</td>"
-            f"<td>{_esc(f.get('problem', ''))}</td>"
-            f"<td>{_esc(f.get('fix', ''))}</td>"
-            f"<td>{_esc(f.get('suggestion', ''))}</td>"
-            f"<td>+{int(f.get('upvotes', 0))} / -{int(f.get('downvotes', 0))}</td>"
-            f"<td>{'resolved' if int(f.get('resolved', 0)) else 'open'}</td>"
-            f"<td>{int(f.get('reply_count', 0))}</td></tr>"
-            for f in findings
-        )
-        findings_table = (
-            "<table><thead><tr><th>location</th><th>severity</th>"
-            "<th>problem</th><th>fix</th><th>suggestion</th>"
-            "<th>votes</th><th>state</th><th>replies</th></tr></thead>"
-            f"<tbody>{rows}</tbody></table>"
-        )
-    else:
-        findings_table = "<p class=\"muted\">no findings</p>"
-
-    return _PR.substitute(
-        repo=_esc(data.get("repo", "")),
-        repo_url=_esc(repo_url),
-        pr=_esc(data.get("pr", 0)),
-        head_sha=_esc(data.get("head_sha", "")),
-        posted_at_iso=_ts_iso(int(data.get("posted_at", 0))),
-        review_id_gitea=_esc(data.get("review_id_gitea", "") or "—"),
-        body_comment_id=_esc(data.get("body_comment_id", "") or "—"),
-        findings_table=findings_table,
-    )
-
-
-def _page(title: str, body: str, *, repos_first: str = "") -> str:
-    db_status = _feedback_db() or "(no DB configured)"
-    return _BASE.substitute(
-        title=_esc(title),
-        body=body,
-        repos_first=_esc(repos_first),
-        db_status=_esc(db_status),
-    )
-
-
-# ---------------------------------------------------------------------------
-# Gitea HTTP helper — minimal, used by the raw body fetch and the edit endpoint
-# ---------------------------------------------------------------------------
-
-
-def _http(method: str, url: str, *, token: str = "", body: dict | None = None,
-          raw_body: bytes | None = None) -> tuple[int, bytes]:
-    """Like ai_review._http but local: this module is stdlib-only and doesn't
-    depend on the ai_review import (which pulls in a 1700-line reviewer)."""
-    headers = {"Accept": "application/json"}
-    data: bytes | None = None
-    if raw_body is not None:
-        data = raw_body
-        headers["Content-Type"] = "application/json"
-    elif body is not None:
-        data = json.dumps(body).encode()
-        headers["Content-Type"] = "application/json"
-    if token:
-        headers["Authorization"] = f"token {token}"
-    req = urllib.request.Request(url, data=data, headers=headers, method=method)
-    try:
-        with urllib.request.urlopen(req, timeout=30) as r:
-            return r.status, r.read()
-    except urllib.error.HTTPError as e:
-        return e.code, e.read()
-    except urllib.error.URLError as e:
-        raise RuntimeError(f"network error: {e.reason}") from e
-
-
-# ---------------------------------------------------------------------------
-# Auth
-# ---------------------------------------------------------------------------
-
-
-def _is_authed(headers) -> bool:
-    """True when oauth2-proxy forwarded a verified user.
-
-    oauth2-proxy sets `X-Forwarded-User` (and friends) only after a
-    successful Logto login + email allowlist check. Unauthenticated
-    requests never see the header, so the dashboard never has to know
-    about cookies, secrets, or Logto's token shape.
-    """
-    return bool((headers.get("X-Forwarded-User") or "").strip())
-
-
-# ---------------------------------------------------------------------------
-# Routes
-# ---------------------------------------------------------------------------
-
-
-def _route_overview() -> bytes:
-    data = dashboard_data.overview(_feedback_db())
-    body = _overview_body(data)
-    # nav: first repo if any
-    repos_first = ""
-    if data.get("top_repos"):
-        repos_first = data["top_repos"][0]["repo"]
-    return _page("Overview", body, repos_first=repos_first).encode()
-
-
-def _route_repo(owner: str, name: str) -> bytes:
-    repo_url = f"{owner}/{name}"
-    data = dashboard_data.repo_summary(_feedback_db(), repo_url)
-    # Pull current .pr-review.json (best-effort) so the form fields prefill.
-    current_static_message, current_model, flash = "", "", ""
-    cfg, err = _fetch_pr_review_json(repo_url)
-    if cfg:
-        current_static_message = cfg.get("static_message", "")
-        current_model = cfg.get("model", "")
-    elif err and err != "404":
-        flash = f"could not read .pr-review.json: {err}"
-    body = _repo_body(
-        data,
-        repo_url=repo_url,
-        csrf=_CSRF_SECRET,
-        current_model=current_model,
-        current_static_message=current_static_message,
-        flash=flash,
-    )
-    return _page(f"repo {repo_url}", body, repos_first=repo_url).encode()
-
-
-def _route_pr(owner: str, name: str, index: int) -> bytes:
-    repo_url = f"{owner}/{name}"
-    data = dashboard_data.pr_summary(_feedback_db(), repo_url, int(index))
-    body = _pr_body(data, repo_url=repo_url)
-    return _page(f"PR {repo_url}#{index}", body, repos_first=repo_url).encode()
-
-
-def _route_pr_raw(owner: str, name: str, index: int) -> tuple[int, bytes]:
-    repo_url = f"{owner}/{name}"
-    data = dashboard_data.pr_summary(_feedback_db(), repo_url, int(index))
-    body_comment_id = data.get("body_comment_id")
-    if not body_comment_id:
-        return 404, b"no body_comment_id"
-    status, raw = _http(
-        "GET",
-        f"{_gitea_api()}/api/v1/repos/{repo_url}/issues/{index}/comments/{body_comment_id}",
-        token=_bot_token(),
-    )
-    if status != 200:
-        return 404, f"Gitea returned {status}".encode()
-    try:
-        parsed = json.loads(raw)
-        md = parsed.get("body", "")
-    except (json.JSONDecodeError, ValueError):
-        return 404, b"could not parse Gitea response"
-    return 200, md.encode()
-
-
-def _route_static_css() -> bytes:
-    return STYLE_CSS.encode()
-
-
-def _route_edit(owner: str, name: str, form: dict) -> tuple[int, dict, bytes]:
-    """Mutate .pr-review.json via the Gitea contents API (Tasks C+D)."""
-    repo_url = f"{owner}/{name}"
-    csrf = form.get("_csrf", "")
-    if csrf != _CSRF_SECRET:
-        return 302, {"Location": f"/r/{repo_url}"}, b""
-    static_message = (form.get("static_message") or "").strip()[:400]
-    model = (form.get("model") or "").strip()
-
-    # Validate model against PRICES.
-    from cost_model import PRICES
-    if model and model not in PRICES:
-        flash = urllib.parse.quote(f"unknown model {model!r}; not saved")
-        return 302, {"Location": f"/r/{repo_url}?flash={flash}"}, b""
-
-    cfg, err = _fetch_pr_review_json(repo_url)
-    if err and err != "404":
-        flash = urllib.parse.quote(f"could not read .pr-review.json: {err}")
-        return 302, {"Location": f"/r/{repo_url}?flash={flash}"}, b""
-    if cfg is None:
-        cfg = {}
-
-    if static_message:
-        cfg["static_message"] = static_message
-    elif "static_message" in cfg and not static_message:
-        # Empty submission clears the banner.
-        del cfg["static_message"]
-    if model:
-        cfg["model"] = model
-    elif "model" in cfg and not model:
-        del cfg["model"]
-
-    payload = json.dumps(cfg, indent=2, sort_keys=True).encode()
-    b64 = base64.b64encode(payload).decode()
-    body = {"content": b64, "message": "pragent dashboard: update .pr-review.json"}
-    if err == "404":
-        # File didn't exist — Gitea contents PUT still creates the file when
-        # `sha` is omitted, but only on certain versions; passing sha=None is
-        # safer.
-        pass
-    else:
-        # GET returned a sha — include it so Gitea enforces optimistic lock.
-        # The sha lives in cfg's wrapper: re-fetch once to capture it.
-        _, raw = _http(
-            "GET",
-            f"{_gitea_api()}/api/v1/repos/{repo_url}/contents/.pr-review.json",
-            token=_bot_token(),
-        )
-        try:
-            existing = json.loads(raw)
-            sha = existing.get("sha")
-            if sha:
-                body["sha"] = sha
-        except (json.JSONDecodeError, ValueError):
-            pass
-
-    status, _ = _http(
-        "PUT",
-        f"{_gitea_api()}/api/v1/repos/{repo_url}/contents/.pr-review.json",
-        token=_bot_token(),
-        body=body,
-    )
-    if status not in (200, 201):
-        flash = urllib.parse.quote(f"Gitea PUT failed: status {status}")
-        return 302, {"Location": f"/r/{repo_url}?flash={flash}"}, b""
-    return 302, {"Location": f"/r/{repo_url}"}, b""
-
-
-def _fetch_pr_review_json(repo_url: str) -> tuple[dict | None, str | None]:
-    """Return (cfg, None) on success, (None, None) when the file doesn't exist,
-    (None, 'reason') on error."""
-    if not _bot_token():
-        return None, "PRAGENT_BOT_TOKEN not set"
-    status, raw = _http(
-        "GET",
-        f"{_gitea_api()}/api/v1/repos/{repo_url}/contents/.pr-review.json",
-        token=_bot_token(),
-    )
-    if status == 404:
-        return None, "404"
-    if status != 200:
-        return None, f"status {status}"
-    try:
-        wrapper = json.loads(raw)
-        content_b64 = wrapper.get("content", "").replace("\n", "")
-        decoded = base64.b64decode(content_b64).decode("utf-8", errors="replace")
-        cfg = json.loads(decoded)
-    except (json.JSONDecodeError, ValueError) as e:
-        return None, f"parse error: {e}"
-    if not isinstance(cfg, dict):
-        return None, "not a JSON object"
-    return cfg, None
-
-
-# ---------------------------------------------------------------------------
-# Handler
-# ---------------------------------------------------------------------------
-
-
-class Handler(BaseHTTPRequestHandler):
-    def _send(self, status: int, body: bytes, *, content_type: str = "text/html; charset=utf-8",
-              extra_headers: dict | None = None) -> None:
-        self.send_response(status)
-        self.send_header("Content-Type", content_type)
-        self.send_header("Content-Length", str(len(body)))
-        if extra_headers:
-            for k, v in extra_headers.items():
-                self.send_header(k, v)
-        self.end_headers()
-        self.wfile.write(body)
-
-    def _redirect(self, location: str) -> None:
-        body = b""
-        self.send_response(302)
-        self.send_header("Location", location)
-        self.send_header("Content-Length", "0")
-        self.end_headers()
-        self.wfile.write(body)
-
-    def _unauthorized(self) -> None:
-        """401 + Basic challenge so oauth2-proxy intercepts and redirects to Logto."""
-        body = b"unauthorized\n"
-        self.send_response(401)
-        self.send_header("Content-Type", "text/plain; charset=utf-8")
-        self.send_header("Content-Length", str(len(body)))
-        self.send_header("WWW-Authenticate", 'Basic realm="pragent-dashboard"')
-        self.end_headers()
-        self.wfile.write(body)
-
-    # --- GET -----------------------------------------------------------------
-
-    def do_GET(self):
-        path = self.path
-        # Static is exempt from auth (also unauthenticated browser fingerprinting
-        # noise, but it's the same CSS regardless of viewer).
-        if path == "/static/style.css":
-            self._send(200, _route_static_css(), content_type="text/css; charset=utf-8")
-            return
-        if not _is_authed(self.headers):
-            self._unauthorized()
-            return
-
-        if path == "/" or path == "":
-            self._send(200, _route_overview())
-            return
-
-        # /r/<owner>/<name>            → repo
-        # /r/<owner>/<name>/<index>    → PR
-        # /r/<owner>/<name>/<index>/raw → raw Markdown
-        m = _REPO_PR_RAW_RE.match(path)
-        if m:
-            owner, name, idx, raw = m.group(1), m.group(2), m.group(3), m.group(4)
-            if raw:
-                status, body = _route_pr_raw(owner, name, int(idx))
-                self._send(status, body,
-                           content_type="text/plain; charset=utf-8" if status == 200 else "text/plain")
-                return
-            if idx:
-                self._send(200, _route_pr(owner, name, int(idx)))
-                return
-            self._send(200, _route_repo(owner, name))
-            return
-
-        self._send(404, b"not found", content_type="text/plain")
-
-    # --- POST ----------------------------------------------------------------
-
-    def do_POST(self):
-        path = self.path
-        if not _is_authed(self.headers):
-            self._unauthorized()
-            return
-        # /r/<owner>/<name>/edit
-        m = _EDIT_RE.match(path)
-        if m:
-            owner, name = m.group(1), m.group(2)
-            length = int(self.headers.get("Content-Length", "0") or "0")
-            raw = self.rfile.read(length) if length else b""
-            form = urllib.parse.parse_qs(raw.decode("utf-8", errors="replace"))
-            # Collapse lists to single values.
-            form_single = {k: v[0] for k, v in form.items()}
-            status, extra, body = _route_edit(owner, name, form_single)
-            self._send(status, body, content_type="text/plain", extra_headers=extra)
-            return
-        self._send(404, b"not found", content_type="text/plain")
-
-    def log_message(self, fmt, *args):
-        print(f"pragent-dashboard: {self.address_string()} {fmt % args}", flush=True)
-
-
-# ---------------------------------------------------------------------------
-# Routing regexes (compiled at import time)
-# ---------------------------------------------------------------------------
-
-import re  # noqa: E402
-
-_REPO_PR_RAW_RE = re.compile(
-    r"^/r/([^/]+)/([^/]+)(?:/(\d+)(?:/(raw))?)?/?$"
-)
-_EDIT_RE = re.compile(r"^/r/([^/]+)/([^/]+)/edit/?$")
-
-
-# ---------------------------------------------------------------------------
-# Main
-# ---------------------------------------------------------------------------
-
-
-def main() -> int:
-    if not _feedback_db():
-        print("pragent-dashboard: WARNING: PRAGENT_FEEDBACK_DB not set; dashboard will be empty",
-              flush=True)
-    print("pragent-dashboard: auth via oauth2-proxy (X-Forwarded-User required)", flush=True)
-    server = ThreadingHTTPServer(("0.0.0.0", PORT), Handler)
-    print(f"pragent-dashboard: listening on :{PORT}", flush=True)
-    try:
-        server.serve_forever()
-    except KeyboardInterrupt:
-        pass
-    return 0
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

pilot/dashboard_data.py

@@ -1,302 +0,0 @@
-"""pragent pilot — dashboard read-only query layer.
-
-Three functions: overview / repo_summary / pr_summary. Each opens the SQLite
-feedback DB via `feedback.init`, runs the queries it needs, and returns plain
-dicts/lists. NEVER writes — that's the dashboard_server's job (via the Gitea
-contents API). This module is what the dashboard_server's templates render.
-
-All three functions are tolerant of a missing or empty DB: they return the
-shaped dict with zeros/empty lists rather than crashing. The dashboard is a
-read-only view; the pilot can boot with no feedback DB and the dashboard
-should still load.
-
-Cost note: `total_cost_usd` is hardcoded to 0.0. Per-review `usage:cost` is
-not in the feedback SQLite — only the raw `review` / `inline_finding` rows
-are stored there. The equivalent-cost calc lives in `ai_review._render_collapsible_usage`
-and only knows about the latest review's tokens. Surfacing a rolled-up dollar
-figure without per-row telemetry would be guessing, so we don't.
-"""
-from __future__ import annotations
-
-import datetime
-import os
-import sqlite3
-
-from pilot import feedback
-
-
-# ---------------------------------------------------------------------------
-# Helpers
-# ---------------------------------------------------------------------------
-
-
-def _empty_overview() -> dict:
-    return {
-        "total_reviews": 0,
-        "total_findings": 0,
-        "total_repos": 0,
-        "last_30d_reviews": 0,
-        "daily": [{"date": _iso_date(i), "count": 0} for i in range(7)],
-        "top_repos": [],
-        "total_cost_usd": 0.0,
-    }
-
-
-def _empty_repo_summary(repo: str) -> dict:
-    return {
-        "repo": repo,
-        "total_runs": 0,
-        "last_run_ts": 0,
-        "runs_by_day": [],
-        "findings_by_severity": {"critical": 0, "high": 0, "medium": 0, "low": 0},
-        "top_findings": [],
-        # NOTE: review rows don't carry a `model` column in the schema today,
-        # so we have nothing to aggregate. When that lands, replace this
-        # empty list with a `SELECT model, COUNT(*) …` over `review`.
-        "models_used": [],
-    }
-
-
-def _empty_pr_summary(repo: str, pr: int) -> dict:
-    return {
-        "repo": repo,
-        "pr": pr,
-        "head_sha": "",
-        "posted_at": 0,
-        "review_id_gitea": None,
-        "body_comment_id": None,
-        "findings": [],
-        # usage isn't on the review row today; ai_review.py renders it
-        # in-memory at review time. Leave empty.
-        "usage": {},
-    }
-
-
-def _iso_date(days_ago: int) -> str:
-    """Return YYYY-MM-DD for `days_ago` days before today (UTC)."""
-    d = datetime.datetime.now(datetime.timezone.utc).date() - datetime.timedelta(days=days_ago)
-    return d.isoformat()
-
-
-def _open_or_none(db_path: str) -> sqlite3.Connection | None:
-    """Open the DB if it exists and looks like a feedback DB. Else None.
-
-    Tolerates missing files (fresh container) and a schema-less file (the
-    operator dropped a stray DB at the path). Returns a connection with
-    Row factory set so callers can use `row["col"]`.
-    """
-    if not db_path or not os.path.exists(db_path):
-        return None
-    try:
-        conn = feedback.init(db_path)
-    except sqlite3.DatabaseError:
-        return None
-    return conn
-
-
-# ---------------------------------------------------------------------------
-# Public API
-# ---------------------------------------------------------------------------
-
-
-def overview(db_path: str) -> dict:
-    """Top-of-page summary: totals + 7-bucket daily sparkline + top 5 repos."""
-    conn = _open_or_none(db_path)
-    if conn is None:
-        return _empty_overview()
-    try:
-        cur = conn.execute("SELECT COUNT(*) FROM review")
-        total_reviews = cur.fetchone()[0]
-        cur = conn.execute("SELECT COUNT(*) FROM inline_finding")
-        total_findings = cur.fetchone()[0]
-        cur = conn.execute("SELECT COUNT(DISTINCT repo) FROM review")
-        total_repos = cur.fetchone()[0]
-
-        # Last 30d window — reviews AND findings posted within the window.
-        ts_30d_ago = int(datetime.datetime.now(datetime.timezone.utc).timestamp()) - 30 * 86400
-        cur = conn.execute("SELECT COUNT(*) FROM review WHERE posted_at >= ?", (ts_30d_ago,))
-        last_30d_reviews = cur.fetchone()[0]
-
-        # 7-bucket daily sparkline, oldest first. Bucket key is UTC date.
-        cur = conn.execute(
-            "SELECT posted_at FROM review WHERE posted_at >= ?",
-            (int(datetime.datetime.now(datetime.timezone.utc).timestamp()) - 7 * 86400,),
-        )
-        buckets: dict[str, int] = {_iso_date(i): 0 for i in range(7)}
-        for (ts,) in cur.fetchall():
-            d = datetime.datetime.fromtimestamp(ts, tz=datetime.timezone.utc).date().isoformat()
-            if d in buckets:
-                buckets[d] += 1
-        daily = [{"date": _iso_date(i), "count": buckets[_iso_date(i)]} for i in range(7)]
-
-        # Top 5 repos by run count, descending. last_seen is the most recent
-        # review timestamp on that repo.
-        cur = conn.execute(
-            "SELECT repo, COUNT(*) AS runs, MAX(posted_at) AS last_seen "
-            "FROM review GROUP BY repo ORDER BY runs DESC, last_seen DESC LIMIT 5"
-        )
-        top_repos = [
-            {"repo": row[0], "run_count": row[1], "last_seen": int(row[2])}
-            for row in cur.fetchall()
-        ]
-
-        return {
-            "total_reviews": total_reviews,
-            "total_findings": total_findings,
-            "total_repos": total_repos,
-            "last_30d_reviews": last_30d_reviews,
-            "daily": daily,
-            "top_repos": top_repos,
-            "total_cost_usd": 0.0,
-        }
-    finally:
-        conn.close()
-
-
-def repo_summary(db_path: str, repo: str) -> dict:
-    """Per-repo drill-down: runs by day, severity histogram, top findings."""
-    conn = _open_or_none(db_path)
-    if conn is None:
-        return _empty_repo_summary(repo)
-    try:
-        cur = conn.execute(
-            "SELECT COUNT(*), MAX(posted_at) FROM review WHERE repo = ?", (repo,)
-        )
-        row = cur.fetchone()
-        total_runs = row[0] or 0
-        last_run_ts = int(row[1]) if row[1] else 0
-
-        # runs_by_day for the last 30 days, oldest first; zero-buckets included.
-        cur = conn.execute(
-            "SELECT posted_at FROM review WHERE repo = ? AND posted_at >= ?",
-            (repo, int(datetime.datetime.now(datetime.timezone.utc).timestamp()) - 30 * 86400),
-        )
-        buckets: dict[str, int] = {}
-        for d in range(30):
-            buckets[_iso_date(d)] = 0  # newest-day mapped to 0; we'll iterate
-        # Re-key: build oldest-first, days_ago goes 29..0
-        oldest_first = {}
-        for d in range(30):
-            oldest_first[_iso_date(29 - d)] = 0
-        for (ts,) in cur.fetchall():
-            d = datetime.datetime.fromtimestamp(ts, tz=datetime.timezone.utc).date().isoformat()
-            if d in oldest_first:
-                oldest_first[d] += 1
-        runs_by_day = [{"date": k, "count": v} for k, v in oldest_first.items()]
-
-        # findings_by_severity — case-insensitive match; bucket unknown as 'low'.
-        cur = conn.execute(
-            "SELECT severity, COUNT(*) FROM inline_finding WHERE repo = ? GROUP BY severity",
-            (repo,),
-        )
-        fbs = {"critical": 0, "high": 0, "medium": 0, "low": 0}
-        for sev, n in cur.fetchall():
-            k = (sev or "").strip().lower()
-            if k not in fbs:
-                k = "low"
-            fbs[k] += n
-
-        # top_findings — top 5 posthashes by occurrence count, joined with
-        # vote rollups via feedback.findings_with_votes.
-        cur = conn.execute(
-            "SELECT f.path, f.line, MAX(f.severity) AS severity, MAX(f.problem) AS problem, "
-            "COUNT(*) AS occurrences, "
-            "COALESCE(SUM(CASE WHEN rct.content = '+1' THEN 1 ELSE 0 END), 0) AS upvotes, "
-            "COALESCE(SUM(CASE WHEN rct.content = '-1' THEN 1 ELSE 0 END), 0) AS downvotes, "
-            "MAX(ts.resolved) AS resolved, "
-            "COALESCE((SELECT COUNT(*) FROM reply WHERE finding_id IN "
-            "  (SELECT id FROM inline_finding WHERE posthash = f.posthash AND repo = f.repo AND path = f.path AND line = f.line)), 0) AS reply_count "
-            "FROM inline_finding f "
-            "LEFT JOIN reaction rct ON rct.comment_id = f.comment_id "
-            "LEFT JOIN thread_state ts ON ts.finding_id = f.id "
-            "WHERE f.repo = ? "
-            "GROUP BY f.posthash, f.repo, f.path, f.line "
-            "ORDER BY occurrences DESC, upvotes DESC LIMIT 5",
-            (repo,),
-        )
-        top_findings = [
-            {
-                "path": r[0],
-                "line": r[1],
-                "severity": r[2],
-                "problem": r[3],
-                "occurrences": r[4],
-                "upvotes": int(r[5] or 0),
-                "downvotes": int(r[6] or 0),
-                "resolved": int(r[7] or 0),
-                "reply_count": int(r[8] or 0),
-            }
-            for r in cur.fetchall()
-        ]
-
-        return {
-            "repo": repo,
-            "total_runs": total_runs,
-            "last_run_ts": last_run_ts,
-            "runs_by_day": runs_by_day,
-            "findings_by_severity": fbs,
-            "top_findings": top_findings,
-            "models_used": [],  # see _empty_repo_summary NOTE
-        }
-    finally:
-        conn.close()
-
-
-def pr_summary(db_path: str, repo: str, pr: int) -> dict:
-    """Per-PR view: meta + every finding the bot ever posted on that PR."""
-    conn = _open_or_none(db_path)
-    if conn is None:
-        return _empty_pr_summary(repo, pr)
-    try:
-        cur = conn.execute(
-            "SELECT head_sha, posted_at, review_id_gitea, body_comment_id "
-            "FROM review WHERE repo = ? AND pr = ? ORDER BY posted_at DESC LIMIT 1",
-            (repo, pr),
-        )
-        row = cur.fetchone()
-        if row is None:
-            return _empty_pr_summary(repo, pr)
-        head_sha, posted_at, review_id_gitea, body_comment_id = row
-
-        cur = conn.execute(
-            "SELECT f.path, f.line, f.severity, f.problem, f.fix, f.suggestion, "
-            "COALESCE(SUM(CASE WHEN rct.content = '+1' THEN 1 ELSE 0 END), 0) AS upvotes, "
-            "COALESCE(SUM(CASE WHEN rct.content = '-1' THEN 1 ELSE 0 END), 0) AS downvotes, "
-            "MAX(ts.resolved) AS resolved, "
-            "COALESCE((SELECT COUNT(*) FROM reply WHERE finding_id = f.id), 0) AS reply_count "
-            "FROM inline_finding f "
-            "LEFT JOIN reaction rct ON rct.comment_id = f.comment_id "
-            "LEFT JOIN thread_state ts ON ts.finding_id = f.id "
-            "WHERE f.repo = ? AND f.pr = ? "
-            "GROUP BY f.id "
-            "ORDER BY f.path, f.line",
-            (repo, pr),
-        )
-        findings = [
-            {
-                "path": r[0],
-                "line": r[1],
-                "severity": r[2],
-                "problem": r[3],
-                "fix": r[4],
-                "suggestion": r[5],
-                "upvotes": int(r[6] or 0),
-                "downvotes": int(r[7] or 0),
-                "resolved": int(r[8] or 0),
-                "reply_count": int(r[9] or 0),
-            }
-            for r in cur.fetchall()
-        ]
-
-        return {
-            "repo": repo,
-            "pr": pr,
-            "head_sha": head_sha,
-            "posted_at": int(posted_at),
-            "review_id_gitea": review_id_gitea,
-            "body_comment_id": body_comment_id,
-            "findings": findings,
-            "usage": {},
-        }
-    finally:
-        conn.close()

pilot/diff_compress.py

@@ -1,254 +1,5 @@
-#!/usr/bin/env python3
-r"""pragent pilot — diff compression + prior-review compaction.
-
-Two pure helpers that shrink what lands in the model prompt without losing
-signal:
-
-  * ``compress_diff(diff, *, context=2)`` — re-renders a unified diff so each
-    hunk keeps only ``context`` unchanged lines on either side of its +/- lines.
-    The default 2 matches what most reviewers see on GitHub/Gitea, and is
-    enough to anchor every ``+``/``-`` line and give the reviewer the enclosing
-    statement. Wider context = more reading; narrower = less. Set
-    ``context=0`` for +/- only, ``context=-1`` to disable entirely.
-
-    Elided context is not merely deleted: each surviving run of lines is
-    re-emitted as its *own* ``@@ -a,b +c,d @@`` hunk with recomputed line
-    numbers, so the output stays a valid unified diff whose line numbers
-    still describe the post-change file. ``parse_diff_anchors`` (and the
-    model) therefore read the same line numbers before and after compression.
-
-  * ``extract_finding_bullets(review_body)`` — pulls the lines of a prior
-    review that look like a pragent finding (``- 🔴 [HIGH] `path:line` — …``,
-    or the older ``- **[HIGH]** …`` form) and drops everything else. The model
-    already has the diff — repeating the prose ("this PR adds eval() — risky")
-    is just token burn. Bullet-only priors cut ~75% off prior-review bytes on
-    a typical 4-finding review.
-
-Stdlib only. No I/O. Tolerant of malformed input — never raises.
-"""
-
-from __future__ import annotations
-
-import re
-
-# A real hunk header: `@@ -old[,count] +new[,count] @@[ trailing section]`.
-# Captures both starts, both counts, and the trailing function-context text.
-# Matching the full shape (not just a `@@` prefix) matters: a *removed* line
-# whose content begins with `@@` is body, not a header.
-_HUNK_RE = re.compile(
-    r"^@@\s+-(\d+)(?:,(\d+))?\s+\+(\d+)(?:,(\d+))?\s+@@(.*)$"
-)
-
-# Match a pragent summary-bullet line, in any of the shapes the renderer has
-# emitted: `- 🔴 [HIGH] \`path:line\` — …` (current, `_severity_badge`),
-# `- **[HIGH]** …` (bold, pre-badge), `- [high] …` (plain, oldest).
-# Anything between the bullet marker and `[SEV]` (emoji, bold markers,
-# whitespace) is tolerated — it is decoration, not signal.
-_FINDING_BULLET_RE = re.compile(
-    r"^\s*[-*]\s*[^\w\[]*\[(?P<sev>critical|high|medium|low)\]",
-    re.IGNORECASE,
-)
-
-
-def compress_diff(diff: str, *, context: int = 2) -> tuple[str, int, int]:
-    """Re-render `diff` keeping at most `context` unchanged lines around +/-.
-
-    Args:
-        diff: unified-diff text (what `gitea .../pulls/{n}.diff` returns).
-        context: max unchanged lines to keep on each side of a hunk. Use 0
-            for +/- only, -1 to disable compression (raw passthrough).
-
-    Returns:
-        `(text, original_chars, kept_chars)`. `original_chars` is the character
-        length of `diff` as given; `kept_chars` is the character length of
-        `text`. Every emitted hunk header is recomputed to match the lines
-        under it, so the result is a valid unified diff. Lines that are not
-        part of a hunk (`diff --git`, `index …`, `Binary files differ`, mode
-        changes) pass through verbatim.
-    """
-    if not diff:
-        return diff or "", len(diff or ""), len(diff or "")
-    if context < 0:
-        return diff, len(diff), len(diff)
-
-    orig = len(diff)
-    lines = diff.splitlines()
-    out: list[str] = []
-
-    i = 0
-    n = len(lines)
-    while i < n:
-        m = _HUNK_RE.match(lines[i])
-        if m is None:
-            # File header, index line, binary marker, mode change, prose —
-            # anything outside a hunk body. Copy verbatim.
-            out.append(lines[i])
-            i += 1
-            continue
-
-        i += 1
-        body_start = i
-        while i < n and _is_body_line(lines[i]):
-            i += 1
-        body = lines[body_start:i]
-
-        out.extend(
-            _render_hunk(
-                body,
-                old_start=int(m.group(1)),
-                new_start=int(m.group(3)),
-                section=m.group(5) or "",
-                context=context,
-            )
-        )
-
-    text = "\n".join(out) + ("\n" if diff.endswith("\n") else "")
-    if not text.strip():
-        # Nothing survived (or the input was nothing but newlines); fall back
-        # to the original so the worst case is no improvement, not data loss.
-        return diff, orig, orig
-    if len(text) >= orig:
-        # Re-emitted hunk headers can outweigh the context they replace on a
-        # small, densely-changed diff. Never hand back something longer than
-        # what we were given.
-        return diff, orig, orig
-    return text, orig, len(text)
-
-
-def _is_body_line(line: str) -> bool:
-    r"""True if `line` belongs to the current hunk body.
-
-    Hunk bodies contain only ` `/`+`/`-` prefixed lines and `\ No newline at
-    end of file`. An empty line is a context line whose trailing space was
-    stripped (common in mail-formatted diffs), so it counts as body too.
-
-    The check is prefix-based *and* header-aware: a removed line reading
-    `---` or an added line reading `+++` (YAML document separators, setext
-    underlines, `--` SQL comments) is body, not a file header — the previous
-    implementation misread those and silently dropped the rest of the hunk.
-    A new file section always opens with `diff --git`, which ends the body.
-    """
-    if line == "":
-        return True
-    if line.startswith("diff --git ") or line.startswith("Index: "):
-        return False
-    if _HUNK_RE.match(line):
-        return False
-    return line[0] in " +-\\"
-
-
-def _render_hunk(
-    body: list[str],
-    *,
-    old_start: int,
-    new_start: int,
-    section: str,
-    context: int,
-) -> list[str]:
-    r"""Trim `body` to `context` unchanged lines around its +/- lines.
-
-    Each surviving run of consecutive lines is emitted as a standalone hunk
-    with a recomputed ``@@ -a,b +c,d @@`` header, so post-change line numbers
-    stay truthful. A hunk with no +/- lines at all (pure context) is dropped
-    entirely; ``\ No newline at end of file`` markers are dropped as noise.
-
-    Returns the rendered lines (headers included), or [] if nothing survived.
-    """
-    # Number every body line on both sides before anything is dropped.
-    numbered: list[tuple[str, int, int]] = []  # (line, old_no, new_no)
-    old_no, new_no = old_start, new_start
-    for ln in body:
-        if ln.startswith("\\"):
-            continue  # `\ No newline at end of file` — no signal, no numbering
-        kind = ln[0] if ln else " "
-        if kind == "+":
-            numbered.append((ln, -1, new_no))
-            new_no += 1
-        elif kind == "-":
-            numbered.append((ln, old_no, -1))
-            old_no += 1
-        else:
-            numbered.append((ln, old_no, new_no))
-            old_no += 1
-            new_no += 1
-
-    changed = [j for j, (ln, _, _) in enumerate(numbered) if ln[:1] in ("+", "-")]
-    if not changed:
-        return []
-
-    keep: set[int] = set()
-    for k in changed:
-        for j in range(max(0, k - context), min(len(numbered) - 1, k + context) + 1):
-            keep.add(j)
-
-    out: list[str] = []
-    for run in _consecutive_runs(sorted(keep)):
-        chunk = [numbered[j] for j in run]
-        old_count = sum(1 for ln, _, _ in chunk if ln[:1] != "+")
-        new_count = sum(1 for ln, _, _ in chunk if ln[:1] != "-")
-        # A run's start is the first line that exists on that side. When a
-        # side has no lines at all (pure addition / pure deletion), unified
-        # diff convention is `start = line before, count = 0`.
-        old_first = next((o for ln, o, _ in chunk if o >= 0), None)
-        new_first = next((nw for ln, _, nw in chunk if nw >= 0), None)
-        old_hdr = old_first if old_first is not None else max(chunk[0][1], 0)
-        new_hdr = new_first if new_first is not None else max(chunk[0][2], 0)
-        if old_count == 0:
-            old_hdr = _side_start_before(numbered, run[0], side=1)
-        if new_count == 0:
-            new_hdr = _side_start_before(numbered, run[0], side=2)
-        out.append(
-            f"@@ -{old_hdr},{old_count} +{new_hdr},{new_count} @@{section}"
-        )
-        out.extend(ln for ln, _, _ in chunk)
-    return out
-
-
-def _side_start_before(
-    numbered: list[tuple[str, int, int]], idx: int, *, side: int
-) -> int:
-    """Line number on `side` (1=old, 2=new) just before body index `idx`.
-
-    Used for the zero-count header form (`@@ -7,0 +8,3 @@`), where unified
-    diff names the line the change is inserted *after*.
-    """
-    for j in range(idx - 1, -1, -1):
-        no = numbered[j][side]
-        if no >= 0:
-            return no
-    # Nothing before it: derive from the first numbered line on that side.
-    for _, old_no, new_no in numbered:
-        no = old_no if side == 1 else new_no
-        if no >= 0:
-            return max(no - 1, 0)
-    return 0
-
-
-def _consecutive_runs(indices: list[int]) -> list[list[int]]:
-    """Group a sorted index list into runs of consecutive integers."""
-    runs: list[list[int]] = []
-    for j in indices:
-        if runs and j == runs[-1][-1] + 1:
-            runs[-1].append(j)
-        else:
-            runs.append([j])
-    return runs
-
-
-def extract_finding_bullets(review_body: str) -> list[str]:
-    """Pull the finding-bullet lines out of a prior review body.
-
-    Returns the matching lines stripped of surrounding whitespace, preserving
-    the rendered ``[SEV] `path:line` — problem`` shape (badge emoji and bold
-    markers included, whichever the renderer used). Lines that look like
-    bullets but carry no severity tag are dropped — the reviewer synthesizes
-    from the matched ones. Continuation lines (`  - **Fix:** …`) are not
-    finding lines and are dropped with the rest of the prose.
-    """
-    if not review_body:
-        return []
-    out = []
-    for line in review_body.splitlines():
-        if _FINDING_BULLET_RE.match(line):
-            out.append(line.strip())
-    return out
+"""Compatibility import for diff transforms."""
+import importlib
+import sys
+_module = importlib.import_module("review.diff")
+sys.modules[__name__] = _module

pilot/entrypoints/__init__.py

@@ -0,0 +1 @@
+"""Executable integration entry points."""

pilot/entrypoints/gitea.py

@@ -0,0 +1,46 @@
+"""Gitea transport adapter.
+
+This module owns HTTP mechanics only. Review policy, parsing, and publishing
+decisions stay in the review layer so they can be tested without a network.
+"""
+from __future__ import annotations
+
+import json
+import urllib.error
+import urllib.request
+
+
+def request(
+    method: str,
+    url: str,
+    token: str,
+    body: dict | None = None,
+    accept: str = "application/json",
+) -> tuple[int, bytes]:
+    headers = {"Authorization": f"token {token}", "Accept": accept}
+    data = None
+    if body is not None:
+        data = json.dumps(body).encode()
+        headers["Content-Type"] = "application/json"
+    req = urllib.request.Request(url, data=data, headers=headers, method=method)
+    try:
+        with urllib.request.urlopen(req, timeout=180) as response:
+            return response.status, response.read()
+    except urllib.error.HTTPError as exc:
+        return exc.code, exc.read()
+    except urllib.error.URLError as exc:
+        raise RuntimeError(f"network error: {exc.reason}") from exc
+
+
+class GiteaClient:
+    """Small adapter for repository-scoped Gitea calls."""
+
+    def __init__(self, api: str, token: str):
+        self.api = api.rstrip("/")
+        self.token = token
+
+    def get(self, path: str, accept: str = "application/json") -> tuple[int, bytes]:
+        return request("GET", f"{self.api}/api/v1/repos/{path}", self.token, accept=accept)
+
+    def post(self, path: str, body: dict) -> tuple[int, bytes]:
+        return request("POST", f"{self.api}/api/v1/repos/{path}", self.token, body)

pilot/entrypoints/webhook.py

@@ -0,0 +1,306 @@
+#!/usr/bin/env python3
+"""pragent pilot — central webhook receiver.
+
+A stdlib-only HTTP server that Gitea posts user-webhook events to. It gates on
+the PR's base ref having `.pr-review.json` with `"enabled": true`, then runs
+the same review core (`ai_review.review_pr`) the CI-step pilot uses, posting
+findings back as `pragent-bot`.
+
+Per-owner setup: one Gitea **user-level webhook** per repo-owner fires for every
+repo that owner has; this service filters to opted-in PRs. (Gitea 1.26.1 system
+webhooks are broken — see pilot/README-webhook.md.) Onboarding a repo = add the
+bot as a Write collaborator + commit a `.pr-review.json` with `"enabled": true`
+on the base ref.
+
+Stdlib only — no pip install, runs on python:3-slim with the scripts mounted.
+
+Endpoints:
+  POST /webhook   Gitea webhook delivery (HMAC-verified)
+  GET  /health    liveness probe
+
+Env:
+  WEBHOOK_SECRET       shared secret used to register the Gitea webhook (HMAC)
+  GITEA_API            in-cluster Gitea base URL
+  PRAGENT_BOT_TOKEN    pragent-bot access token (non-admin; must be a Write
+                       collaborator on each reviewed repo)
+  OLLAMA_URL           headroom proxy URL, e.g. http://model-proxy.internal:8789
+  OLLAMA_MODEL         model id, e.g. glm-5.2:cloud
+  OLLAMA_MAX_TOKENS    (optional) output cap, default 6000
+  DIFF_MAX_CHARS       (optional) diff truncation cap, default 150000
+  WEBHOOK_PORT         (optional) listen port, default 8080
+  PRAGENT_MAX_CONCURRENT_REVIEWS
+                       (optional) how many reviews may run at once, default 2.
+                       Each review forks an opencode process that checks out a
+                       repo and runs linters, so this is the real resource knob.
+  PRAGENT_MAX_BODY_BYTES
+                       (optional) request-body cap, default 10 MiB
+"""
+
+import base64
+import hashlib
+import hmac
+import json
+import os
+import threading
+import urllib.parse
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+from ai_review import gitea_get, review_pr
+from review_config import repo_enabled
+
+try:
+    import feedback_harvest  # optional — absent in CI-step pod, present in
+                              # central webhook service. Harvesting is the
+                              # collection side of the feedback loop.
+except ImportError:
+    feedback_harvest = None
+
+# Pull-request webhook `action` values. We fire on EVERY pull_request action
+# except `closed` (no point reviewing a closed/merged PR) — the
+# `.pr-review.json:enabled` gate + sha dedupe downstream make broadening safe:
+# a same-sha re-fire (title edit, assignee, milestone, label toggle…) is
+# skipped by `review_pr`'s dedupe. Gitea emits GitHub-style `action` names
+# (`labeled`, `synchronize`) even though the `X-Gitea-Event-Type` header uses
+# `label_updated` / `synchronized`.
+SKIP_ACTIONS = {"closed"}
+
+GITEA_API = os.environ.get("GITEA_API", "http://gitea-http.gitea.svc.cluster.local:3000")
+BOT_TOKEN = os.environ.get("PRAGENT_BOT_TOKEN", "")
+OLLAMA_URL = os.environ.get("OLLAMA_URL", "http://model-proxy.internal:8789")
+OLLAMA_MODEL = os.environ.get("OLLAMA_MODEL", "glm-5.2:cloud")
+OLLAMA_MAX_TOKENS = int(os.environ.get("OLLAMA_MAX_TOKENS", "8000"))
+DIFF_MAX_CHARS = int(os.environ.get("DIFF_MAX_CHARS", "150000"))
+WEBHOOK_SECRET = os.environ.get("WEBHOOK_SECRET", "").encode()
+PORT = int(os.environ.get("WEBHOOK_PORT", "8080"))
+MAX_CONCURRENT = max(1, int(os.environ.get("PRAGENT_MAX_CONCURRENT_REVIEWS", "2")))
+MAX_BODY_BYTES = int(os.environ.get("PRAGENT_MAX_BODY_BYTES", str(10 * 1024 * 1024)))
+# Feedback DB — SQLite mounted at PRAGENT_FEEDBACK_DB. Empty / unset =
+# feedback collection disabled (CI-step path doesn't have it).
+FEEDBACK_DB = os.environ.get("PRAGENT_FEEDBACK_DB", "")
+
+# Bound on reviews running at once. Every review forks an opencode process that
+# untars a repo, reads files and shells out to linters, so an unbounded thread
+# per delivery is a self-inflicted fork bomb the first time someone labels ten
+# PRs (or Gitea retries a burst). Queued deliveries wait here rather than pile
+# onto the box; the handler has already returned 202, so nothing times out.
+_review_slots = threading.Semaphore(MAX_CONCURRENT)
+
+# Reviews currently accepted or running, keyed (repo, index, sha). The
+# sha-marker dedupe in `review_pr` reads Gitea *before* posting, so two
+# deliveries for the same commit in flight together both see "not yet reviewed"
+# and both post — the classic check-then-act race. Common triggers are Gitea
+# retries after a slow 202 response and bursty re-fires from a rapid title /
+# assign / label toggle. This set closes the window inside one process.
+_inflight: set[tuple[str, str, str]] = set()
+_inflight_lock = threading.Lock()
+
+
+def is_repo_enabled(api: str, repo: str, ref: str, token: str) -> bool:
+    """True iff `.pr-review.json` on `ref` has `"enabled": true`.
+
+    Reads from the given ref (typically the PR's base ref). False on any
+    failure: 404, parse error, missing file, missing `enabled`, wrong type.
+    The bool-coerce of `.get("enabled") is True` rejects the common
+    gotchas (`null`, `1`, `"yes"`, missing field all yield False).
+    """
+    return repo_enabled(gitea_get, api, repo, ref, token)
+
+
+def _verify_signature(raw_body: bytes, headers) -> bool:
+    if not WEBHOOK_SECRET:
+        return False  # refuse to run without a configured secret
+    sig_header = headers.get("X-Gitea-Signature") or headers.get("X-Forgejo-Signature")
+    if not sig_header:
+        return False
+    mac = hmac.new(WEBHOOK_SECRET, raw_body, hashlib.sha256).hexdigest()
+    return hmac.compare_digest(mac, sig_header)
+
+
+def _handle_pull_request(payload: dict) -> tuple[int, str]:
+    """Decide whether to review; if so, kick it off in a background thread.
+
+    Returns (status, message) to Gitea immediately — the review itself runs
+    async so Gitea's delivery timeout never fires and causes a retry.
+    """
+    action = payload.get("action", "")
+    pr = payload.get("pull_request") or {}
+    repo_obj = payload.get("repository") or {}
+    repo = repo_obj.get("full_name") or ""
+
+    if action in SKIP_ACTIONS:
+        return 200, f"ignore action={action}"
+    if not repo:
+        return 400, "no repository.full_name"
+
+    index = pr.get("number")
+    if index is None:
+        return 400, "no pull_request.number"
+    title = pr.get("title", "") or ""
+    body = pr.get("body", "") or ""
+    head = pr.get("head") or {}
+    sha = head.get("sha", "") or ""
+
+    base_ref = (pr.get("base") or {}).get("ref", "") or ""
+
+    if not is_repo_enabled(GITEA_API, repo, base_ref or "", BOT_TOKEN):
+        return 200, f"skip (repo not opted in) action={action}"
+
+    if not BOT_TOKEN:
+        return 500, "PRAGENT_BOT_TOKEN not set"
+
+    key = (repo, str(index), sha)
+    if not _claim(key):
+        return 200, f"ignore (already in flight) {repo}#{index} sha={sha[:8]}"
+
+    threading.Thread(
+        target=_run_review,
+        args=(key, title, body, base_ref),
+        daemon=True,
+    ).start()
+    return 202, f"reviewing {repo}#{index} action={action} sha={sha[:8]}"
+
+
+def _claim(key: tuple[str, str, str]) -> bool:
+    """Reserve (repo, index, sha) for review. False if already claimed."""
+    with _inflight_lock:
+        if key in _inflight:
+            return False
+        _inflight.add(key)
+        return True
+
+
+def _release(key: tuple[str, str, str]) -> None:
+    with _inflight_lock:
+        _inflight.discard(key)
+
+
+def _run_review(
+    key: tuple[str, str, str], title: str, body: str, base_ref: str
+) -> None:
+    repo, index, sha = key
+    # Harvest reactions on PRIOR bot comments on this PR (best-effort —
+    # piggy-backs the webhook path so we don't need a separate cron).
+    # Disabled if feedback_harvest isn't importable (CI-step image) or
+    # FEEDBACK_DB isn't set.
+    if FEEDBACK_DB and feedback_harvest is not None:
+        try:
+            hstats = feedback_harvest.harvest_for_pr(
+                api=GITEA_API, token=BOT_TOKEN,
+                repo=repo, pr_index=int(index), db_path=FEEDBACK_DB,
+            )
+            print(
+                f"pragent-webhook: harvested {repo}#{index} "
+                f"reviews={hstats['reviews_seen']} "
+                f"findings={hstats['findings_seen']} "
+                f"reactions={hstats['reactions_recorded']}",
+                flush=True,
+            )
+        except Exception as e:
+            # Harvest must never abort a review.
+            print(f"pragent-webhook: harvest failed for {repo}#{index}: {e}", flush=True)
+
+    try:
+        with _review_slots:
+            ok = review_pr(
+                api=GITEA_API,
+                repo=repo,
+                index=index,
+                title=title,
+                body=body,
+                sha=sha,
+                token=BOT_TOKEN,
+                ollama_url=OLLAMA_URL,
+                model=OLLAMA_MODEL,
+                max_tokens=OLLAMA_MAX_TOKENS,
+                max_chars=DIFF_MAX_CHARS,
+                base_ref=base_ref,
+            )
+        print(f"pragent-webhook: reviewed {repo}#{index} sha={sha[:8]} ok={ok}", flush=True)
+    except Exception as e:  # review_pr is fail-open, but guard the thread anyway
+        print(f"pragent-webhook: thread crashed for {repo}#{index}: {e}", flush=True)
+    finally:
+        _release(key)
+
+
+class Handler(BaseHTTPRequestHandler):
+    def _send(self, status: int, body: str) -> None:
+        data = body.encode()
+        self.send_response(status)
+        self.send_header("Content-Type", "text/plain")
+        self.send_header("Content-Length", str(len(data)))
+        self.end_headers()
+        self.wfile.write(data)
+
+    def do_GET(self):
+        if self.path == "/health":
+            with _inflight_lock:
+                n = len(_inflight)
+            self._send(200, f"ok inflight={n} max_concurrent={MAX_CONCURRENT}")
+        else:
+            self._send(404, "not found")
+
+    def do_POST(self):
+        if self.path != "/webhook":
+            self._send(404, "not found")
+            return
+        try:
+            length = int(self.headers.get("Content-Length", "0") or "0")
+        except ValueError:
+            self._send(400, "bad content-length")
+            return
+        # Cap before reading: the body is read whole into memory, so an
+        # unbounded Content-Length is a one-request OOM.
+        if length < 0 or length > MAX_BODY_BYTES:
+            self._send(413, "payload too large")
+            return
+        raw = self.rfile.read(length) if length else b""
+        if len(raw) != length:
+            self._send(400, "truncated body")
+            return
+
+        if not _verify_signature(raw, self.headers):
+            self._send(401, "invalid signature")
+            return
+
+        try:
+            payload = json.loads(raw)
+        except json.JSONDecodeError:
+            self._send(400, "invalid json")
+            return
+
+        event = self.headers.get("X-Gitea-Event") or payload.get("action") or ""
+        if event != "pull_request":
+            self._send(200, f"ignore event={event}")
+            return
+
+        repo_full = (payload.get("repository") or {}).get("full_name")
+        print(
+            f"pragent-webhook: pull_request action={payload.get('action')} repo={repo_full}",
+            flush=True,
+        )
+        status, msg = _handle_pull_request(payload)
+        self._send(status, msg)
+
+    def log_message(self, fmt, *args):
+        # Keep k8s logs to our own lines (see _run_review / _send paths).
+        print(f"pragent-webhook: {self.address_string()} {fmt % args}", flush=True)
+
+
+def main() -> int:
+    if not WEBHOOK_SECRET:
+        print("pragent-webhook: FATAL: WEBHOOK_SECRET not set", flush=True)
+        return 1
+    if not BOT_TOKEN:
+        print("pragent-webhook: FATAL: PRAGENT_BOT_TOKEN not set", flush=True)
+        return 1
+    server = ThreadingHTTPServer(("0.0.0.0", PORT), Handler)
+    print(f"pragent-webhook: listening on :{PORT} (model={OLLAMA_MODEL})", flush=True)
+    try:
+        server.serve_forever()
+    except KeyboardInterrupt:
+        pass
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

pilot/eval_bootstrap.py

@@ -1,350 +1,7 @@
-#!/usr/bin/env python3
-"""pragent pilot — one-time Langfuse project setup for evaluation.
-
-Three jobs, each idempotent so it can be re-run after any change:
-
-  1. **Score configs.** Registers the schema for every score pragent emits
-     (`eval_scores.SCORE_CONFIGS` + `feedback_scores.SCORE_CONFIGS`). Without
-     these the scores still ingest, but nothing stops a later scorer writing
-     `severity_max="HIGH"` beside today's `"high"` and quietly splitting one
-     series into two. Configs are immutable in Langfuse — a name that already
-     exists is left alone rather than updated.
-
-  2. **Dataset.** Seeds `pragent-reviews` from `feedback.db`: one item per PR
-     the reviewer has actually run on, carrying the repo/PR/sha as input and
-     the findings it posted as `expectedOutput`.
-
-     Read `expectedOutput` here as "what the reviewer said last time", not "what
-     is correct" — no human has labelled any of it. It is a regression baseline:
-     re-run a candidate model over these PRs and the diff against this column is
-     the behaviour change. Promoting an item to real ground truth means a human
-     editing it after reviewing the PR, which is what the dataset view is for.
-
-  3. **Trace backfill** (`--backfill-traces`). Scores only ride along with new
-     reviews, so without this the charts stay empty until the next PR lands.
-     Every trace `langfuse_trace` has ever written already carries the finding
-     count, the severity histogram and the cost in its metadata, which is
-     everything four of the five scorers need. `dropped_findings` is absent from
-     historical traces and is left unscored rather than backfilled as zero.
-
-  4. **Reports** what it found, so the gap between "reviews recorded" and
-     "reviews with human feedback" is visible rather than assumed.
-
-Usage:
-    LANGFUSE_HOST=... LANGFUSE_PUBLIC_KEY=... LANGFUSE_SECRET_KEY=... \\
-        python3 eval_bootstrap.py --db /data/feedback.db
-"""
-from __future__ import annotations
-
-import argparse
-import base64
-import json
-import os
-import sqlite3
+"""Compatibility import for evaluation bootstrap."""
+import importlib
 import sys
-import urllib.error
-import urllib.request
-from datetime import datetime, timezone
-
-sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
-
-import eval_scores  # noqa: E402
-import feedback_scores  # noqa: E402
-
-DATASET_NAME = "pragent-reviews"
-
-
-def _conf() -> tuple[str, str, str]:
-    host = (os.environ.get("LANGFUSE_HOST") or "").strip().rstrip("/")
-    pk = (os.environ.get("LANGFUSE_PUBLIC_KEY") or "").strip()
-    sk = (os.environ.get("LANGFUSE_SECRET_KEY") or "").strip()
-    if not host or not pk or not sk:
-        raise SystemExit("LANGFUSE_HOST / LANGFUSE_PUBLIC_KEY / LANGFUSE_SECRET_KEY must be set")
-    return host, pk, sk
-
-
-def _call(method: str, path: str, body: dict | None = None, timeout: float = 20.0):
-    host, pk, sk = _conf()
-    auth = base64.b64encode(f"{pk}:{sk}".encode()).decode("ascii")
-    data = json.dumps(body).encode() if body is not None else None
-    req = urllib.request.Request(
-        host + path,
-        data=data,
-        headers={
-            "Content-Type": "application/json",
-            "Authorization": f"Basic {auth}",
-            "User-Agent": "pragent-pilot/1.0",
-        },
-        method=method,
-    )
-    try:
-        with urllib.request.urlopen(req, timeout=timeout) as resp:
-            raw = resp.read()
-            return resp.status, (json.loads(raw) if raw else None)
-    except urllib.error.HTTPError as e:
-        return e.code, e.read()[:400].decode("utf-8", "replace")
-
-
-# ---------------------------------------------------------------------------
-# 1. Score configs
-# ---------------------------------------------------------------------------
-
-def ensure_score_configs() -> dict:
-    status, existing = _call("GET", "/api/public/score-configs?limit=100")
-    have = set()
-    if status == 200 and isinstance(existing, dict):
-        have = {c.get("name") for c in existing.get("data", [])}
-
-    created, skipped, failed = [], [], []
-    for cfg in list(eval_scores.SCORE_CONFIGS) + list(feedback_scores.SCORE_CONFIGS):
-        if cfg["name"] in have:
-            skipped.append(cfg["name"])
-            continue
-        st, resp = _call("POST", "/api/public/score-configs", cfg)
-        if st in (200, 201):
-            created.append(cfg["name"])
-        else:
-            failed.append({"name": cfg["name"], "status": st, "error": resp})
-    return {"created": created, "already_present": skipped, "failed": failed}
-
-
-# ---------------------------------------------------------------------------
-# 2. Dataset from recorded reviews
-# ---------------------------------------------------------------------------
-
-def item_id(repo: str, pr) -> str:
-    """A dataset-item id that survives being put in a URL path.
-
-    The obvious `{repo}#{pr}` is unusable: the UI routes items as
-    `/datasets/{id}/items/{item_id}`, so the `/` in `owner/repo` splits into
-    extra path segments and everything after the `#` is a fragment the browser
-    never sends. The item is created fine and then 404s when opened.
-
-    Session ids elsewhere keep the `{repo}#{pr}` form — those are never path
-    segments, and `feedback_scores` depends on that shape.
-    """
-    return f"{repo.replace('/', '__')}__pr{pr}"
-
-
-def _item_metadata(*, repo, pr, head_sha, reviews_run, last_seen, findings) -> dict:
-    """Filterable facets for one dataset item.
-
-    Kept flat and primitive: the filter bar matches a metadata key against a
-    literal, so a nested object or a list is not reachable from the UI.
-    """
-    owner, _, repo_name = str(repo).partition("/")
-    sevs = [str(f["severity"] or "").lower() for f in findings]
-    ranked = [s for s in sevs if s in eval_scores.SEVERITY_RANK]
-    return {
-        "repo": repo,
-        "owner": owner or repo,
-        "repo_name": repo_name or repo,
-        "pr": int(pr),
-        "head_sha": head_sha,
-        "reviews_run": reviews_run,
-        "last_reviewed_at": last_seen,
-        "last_reviewed_iso": datetime.fromtimestamp(last_seen, timezone.utc).isoformat(),
-        "finding_count": len(findings),
-        "has_findings": bool(findings),
-        # "none" rather than omitting the key: a filter for silent reviews needs
-        # something to match, and an absent key matches nothing.
-        "max_severity": (
-            max(ranked, key=lambda s: eval_scores.SEVERITY_RANK[s]) if ranked else "none"
-        ),
-        # Flags that this row is the reviewer's own past output, not a human
-        # judgement. Filter on it before anyone treats the dataset as truth.
-        "labelled_by_human": False,
-    }
-
-
-def read_review_items(db_path: str) -> list[dict]:
-    """One dataset item per (repo, pr) the reviewer has run on.
-
-    Keyed on the PR rather than on each individual review row: the same PR is
-    re-reviewed on every push, and 113 rows over 26 PRs would make a benchmark
-    that is 4x redundant and weighted towards whichever PR churned most.
-    """
-    conn = sqlite3.connect(db_path)
-    conn.row_factory = sqlite3.Row
-    try:
-        prs = conn.execute(
-            """
-            SELECT repo, pr, MAX(posted_at) AS last_seen, COUNT(*) AS reviews,
-                   MAX(head_sha) AS head_sha
-            FROM review GROUP BY repo, pr ORDER BY repo, pr
-            """
-        ).fetchall()
-        items = []
-        for row in prs:
-            findings = conn.execute(
-                """
-                SELECT path, line, severity, problem, fix
-                FROM inline_finding WHERE repo = ? AND pr = ?
-                ORDER BY path, line
-                """,
-                (row["repo"], row["pr"]),
-            ).fetchall()
-            items.append(
-                {
-                    "id": item_id(row["repo"], row["pr"]),
-                    "input": {
-                        "repo": row["repo"],
-                        "pr": int(row["pr"]),
-                        "head_sha": row["head_sha"],
-                    },
-                    "expectedOutput": {
-                        "findings": [dict(f) for f in findings],
-                        "finding_count": len(findings),
-                    },
-                    # The UI's filter bar reads metadata and nothing else, so
-                    # anything worth slicing on is a top-level key here even
-                    # where it duplicates `input`. `owner` and `repo_name` are
-                    # split out because a filter on the joined `repo` can only
-                    # match one repo at a time, never a whole org.
-                    "metadata": _item_metadata(
-                        repo=row["repo"],
-                        pr=row["pr"],
-                        head_sha=row["head_sha"],
-                        reviews_run=int(row["reviews"]),
-                        last_seen=int(row["last_seen"]),
-                        findings=findings,
-                    ),
-                }
-            )
-        return items
-    finally:
-        conn.close()
-
-
-def ensure_dataset(items: list[dict], name: str = DATASET_NAME) -> dict:
-    st, _ = _call(
-        "POST",
-        "/api/public/datasets",
-        {
-            "name": name,
-            "description": (
-                "PRs the pragent pilot has reviewed, seeded from feedback.db. "
-                "expectedOutput is the reviewer's own prior output — a regression "
-                "baseline, not human-verified ground truth."
-            ),
-            "metadata": {"source": "feedback.db", "seeded_by": "eval_bootstrap.py"},
-        },
-    )
-    # A duplicate name is fine: the dataset already exists from an earlier run.
-    dataset_ok = st in (200, 201, 409)
-
-    created, failed = 0, []
-    for item in items:
-        body = {
-            "datasetName": name,
-            "id": item["id"],  # idempotent: same PR updates rather than duplicates
-            "input": item["input"],
-            "expectedOutput": item["expectedOutput"],
-            "metadata": item["metadata"],
-        }
-        ist, resp = _call("POST", "/api/public/dataset-items", body)
-        if ist in (200, 201):
-            created += 1
-        else:
-            failed.append({"item": item["id"], "status": ist, "error": resp})
-    return {"dataset": name, "dataset_created": dataset_ok, "items_upserted": created, "failed": failed}
-
-
-# ---------------------------------------------------------------------------
-# 3. Backfill scores onto traces that predate the scorers
-# ---------------------------------------------------------------------------
-
-def _synth_findings(severities: dict) -> list[dict]:
-    """Rebuild a findings list from a trace's severity histogram.
-
-    Only severity matters to the scorers, and that is all the histogram kept.
-    Reconstructing placeholders is honest here because every scorer being
-    backfilled reads nothing else off a finding.
-    """
-    out = []
-    for sev, count in (severities or {}).items():
-        out.extend({"severity": sev} for _ in range(int(count)))
-    return out
-
-
-def backfill_traces(limit_pages: int = 20) -> dict:
-    import eval_scores as es
-
-    scored, skipped, events = 0, 0, []
-    page = 1
-    while page <= limit_pages:
-        st, resp = _call("GET", f"/api/public/traces?limit=50&page={page}&name=pr-review")
-        if st != 200 or not isinstance(resp, dict):
-            break
-        rows = resp.get("data") or []
-        if not rows:
-            break
-        for tr in rows:
-            meta = tr.get("metadata") or {}
-            severities = meta.get("severities") or {}
-            count = meta.get("findings")
-            if count is None:
-                skipped += 1
-                continue
-            findings = _synth_findings(severities)
-            # The histogram is authoritative when present; a trace that recorded
-            # a count but no histogram still scores its rate.
-            if not findings and count:
-                findings = [{"severity": "medium"} for _ in range(int(count))]
-            batch = es.build_scores(
-                trace_id=tr["id"],
-                findings=findings,
-                environment=tr.get("environment") or "default",
-                cost_usd=(tr.get("totalCost") or meta.get("provider_cost_usd")),
-                timestamp=tr.get("timestamp"),
-                comment="backfilled from trace metadata",
-            )
-            events.extend(batch)
-            scored += 1
-        page += 1
-
-    posted = False
-    status = None
-    if events:
-        import langfuse_trace
-
-        host, pk, sk = _conf()
-        # Chunked: one 2000-event POST is refused, and a partial backfill that
-        # reports success is worse than a slow one.
-        for i in range(0, len(events), 200):
-            status = langfuse_trace._post(host, pk, sk, events[i:i + 200], 30.0)
-            posted = status in (200, 201, 207)
-            if not posted:
-                break
-    return {"traces_scored": scored, "traces_skipped": skipped, "scores": len(events),
-            "posted": posted, "http_status": status}
-
-
-def main() -> int:
-    ap = argparse.ArgumentParser(description="Bootstrap Langfuse evaluation for the pragent pilot")
-    ap.add_argument("--db", default=os.environ.get("PRAGENT_FEEDBACK_DB", "/data/feedback.db"))
-    ap.add_argument("--skip-dataset", action="store_true")
-    ap.add_argument("--skip-configs", action="store_true")
-    ap.add_argument("--backfill-traces", action="store_true",
-                    help="score traces written before the scorers existed")
-    args = ap.parse_args()
-
-    out: dict = {}
-    if not args.skip_configs:
-        out["score_configs"] = ensure_score_configs()
-    if not args.skip_dataset:
-        items = read_review_items(args.db)
-        out["dataset"] = ensure_dataset(items)
-        out["dataset"]["items_read"] = len(items)
-    if args.backfill_traces:
-        out["trace_backfill"] = backfill_traces()
-    print(json.dumps(out, indent=2))
-
-    failed = (out.get("score_configs", {}).get("failed") or []) + (
-        out.get("dataset", {}).get("failed") or []
-    )
-    return 1 if failed else 0
-
-
+_module = importlib.import_module("evaluation.bootstrap")
+sys.modules[__name__] = _module
 if __name__ == "__main__":
-    raise SystemExit(main())
+    raise SystemExit(_module.main())

pilot/eval_experiment.py

@@ -1,212 +1,7 @@
-#!/usr/bin/env python3
-"""pragent pilot — populate the Experiments tab from reviews already traced.
-
-An "experiment" in Langfuse is a dataset run: a set of (dataset item, trace)
-links under one run name. The Experiments tab then shows one row per item with
-its scores, and lets two runs be diffed side by side.
-
-Nothing here re-runs the reviewer. Every PR in `pragent-reviews` has already
-been reviewed, and each of those reviews left a trace carrying its findings,
-cost and scores. This links what exists, which is what makes the tab useful on
-day one instead of after the next N pushes.
-
-Runs are grouped by **model** by default, because that is the comparison the
-pilot actually needs to make: the same PRs reviewed by MiniMax vs whatever
-replaces it, with `finding_rate` and `cost_per_finding` side by side. Group by
-`none` for a single "all traces" run.
-
-One trace per (run, item) — the most recent. A PR re-reviewed on every push has
-many traces, and a dataset run is defined as one output per input; feeding it
-the other five would make the per-run averages meaningless.
-
-Note on the endpoint: `POST /api/public/dataset-run-items` is deprecated in
-favour of the SDK experiment runner / OTel ingestion, and disappears in
-Langfuse v4. This instance is self-hosted v3, which the deprecation notice
-explicitly exempts from the cutoff date, and the pilot is stdlib-only by
-design. Revisit when this deployment moves to v4.
-
-Usage:
-    LANGFUSE_HOST=... LANGFUSE_PUBLIC_KEY=... LANGFUSE_SECRET_KEY=... \\
-        python3 eval_experiment.py --dry-run
-"""
-from __future__ import annotations
-
-import argparse
-import json
-import os
+"""Compatibility import for evaluation experiments."""
+import importlib
 import sys
-import urllib.parse
-from collections import defaultdict
-
-sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
-
-import eval_bootstrap as eb  # noqa: E402
-
-TRACE_NAME = "pr-review"
-
-
-# ---------------------------------------------------------------------------
-# Reading what already exists
-# ---------------------------------------------------------------------------
-
-def fetch_traces(name: str = TRACE_NAME, limit: int = 100, max_pages: int = 50) -> list[dict]:
-    """Every review trace, newest first."""
-    out: list[dict] = []
-    for page in range(1, max_pages + 1):
-        q = urllib.parse.urlencode({"name": name, "limit": limit, "page": page})
-        st, body = eb._call("GET", f"/api/public/traces?{q}")
-        if st != 200 or not isinstance(body, dict):
-            raise SystemExit(f"listing traces failed: {st} {body}")
-        data = body.get("data") or []
-        out.extend(data)
-        meta = body.get("meta") or {}
-        if page * meta.get("limit", limit) >= meta.get("totalItems", 0):
-            break
-    return out
-
-
-def fetch_item_ids(dataset: str) -> set[str]:
-    """Ids present in the dataset, so runs never reference a missing item."""
-    ids: set[str] = set()
-    for page in range(1, 51):
-        q = urllib.parse.urlencode({"datasetName": dataset, "limit": 100, "page": page})
-        st, body = eb._call("GET", f"/api/public/dataset-items?{q}")
-        if st != 200 or not isinstance(body, dict):
-            raise SystemExit(f"listing dataset items failed: {st} {body}")
-        ids.update(i["id"] for i in body.get("data") or [])
-        meta = body.get("meta") or {}
-        if page * meta.get("limit", 100) >= meta.get("totalItems", 0):
-            break
-    return ids
-
-
-# ---------------------------------------------------------------------------
-# Grouping traces into runs
-# ---------------------------------------------------------------------------
-
-def trace_model(trace: dict) -> str:
-    """The model that produced a review, from its `model:` tag."""
-    for tag in trace.get("tags") or []:
-        if tag.startswith("model:"):
-            return tag[len("model:"):] or "unknown"
-    return "unknown"
-
-
-def trace_item_id(trace: dict) -> str | None:
-    """The dataset item a trace belongs to, or None if it is not a PR review."""
-    md = trace.get("metadata") or {}
-    repo, pr = md.get("repo"), md.get("pr")
-    if not repo or pr in (None, ""):
-        return None
-    return eb.item_id(str(repo), pr)
-
-
-def _sort_key(trace: dict):
-    return (trace.get("timestamp") or "", trace.get("id") or "")
-
-
-def plan_runs(traces: list[dict], known_items: set[str], group_by: str = "model") -> dict:
-    """Map run name -> {item id: trace}, keeping only the newest trace per item.
-
-    Traces whose PR is not in the dataset are dropped: `feedback.db` is the
-    source for both, but a review can be traced without its row landing (the
-    posting step can fail after the model ran), and a run item pointing at a
-    non-existent dataset item is rejected.
-    """
-    runs: dict[str, dict[str, dict]] = defaultdict(dict)
-    skipped_no_item, skipped_unknown = 0, 0
-    for tr in traces:
-        iid = trace_item_id(tr)
-        if iid is None:
-            skipped_unknown += 1
-            continue
-        if iid not in known_items:
-            skipped_no_item += 1
-            continue
-        run = "all-traces" if group_by == "none" else trace_model(tr)
-        prev = runs[run].get(iid)
-        if prev is None or _sort_key(tr) > _sort_key(prev):
-            runs[run][iid] = tr
-    return {
-        "runs": dict(runs),
-        "skipped_not_in_dataset": skipped_no_item,
-        "skipped_not_a_review": skipped_unknown,
-    }
-
-
-def run_name(prefix: str, key: str) -> str:
-    return f"{prefix}-{key}" if prefix else key
-
-
-# ---------------------------------------------------------------------------
-# Writing the runs
-# ---------------------------------------------------------------------------
-
-def create_run(name: str, items: dict[str, dict], description: str = "") -> dict:
-    """Link each (item, trace) pair into the named run. Idempotent per pair."""
-    created, failed = 0, []
-    for iid, tr in sorted(items.items()):
-        md = tr.get("metadata") or {}
-        body = {
-            "runName": name,
-            "runDescription": description,
-            "datasetItemId": iid,
-            "traceId": tr["id"],
-            "metadata": {
-                "model": trace_model(tr),
-                "engine": md.get("engine"),
-                "findings": md.get("findings"),
-                "duration_s": md.get("duration_s"),
-                "cost_basis": md.get("cost_basis"),
-                "linked_by": "eval_experiment.py",
-            },
-        }
-        st, resp = eb._call("POST", "/api/public/dataset-run-items", body)
-        if st in (200, 201):
-            created += 1
-        else:
-            failed.append({"item": iid, "status": st, "error": resp})
-    return {"run": name, "items_linked": created, "failed": failed}
-
-
-def main(argv: list[str] | None = None) -> int:
-    ap = argparse.ArgumentParser(description=__doc__)
-    ap.add_argument("--dataset", default=eb.DATASET_NAME)
-    ap.add_argument("--group-by", choices=("model", "none"), default="model")
-    ap.add_argument("--prefix", default="baseline",
-                    help="run name prefix; '' for the bare group key")
-    ap.add_argument("--dry-run", action="store_true")
-    args = ap.parse_args(argv)
-
-    traces = fetch_traces()
-    items = fetch_item_ids(args.dataset)
-    plan = plan_runs(traces, items, group_by=args.group_by)
-
-    report = {
-        "traces_read": len(traces),
-        "dataset_items": len(items),
-        "skipped_not_in_dataset": plan["skipped_not_in_dataset"],
-        "skipped_not_a_review": plan["skipped_not_a_review"],
-        "runs": {},
-    }
-    for key, mapping in sorted(plan["runs"].items()):
-        name = run_name(args.prefix, key)
-        if args.dry_run:
-            report["runs"][name] = {"items_would_link": len(mapping)}
-            continue
-        report["runs"][name] = create_run(
-            name,
-            mapping,
-            description=(
-                "Reviews already run by the pilot, linked after the fact. "
-                "Scores come from the traces; expectedOutput is the reviewer's "
-                "own prior output, not human-verified ground truth."
-            ),
-        )
-    report["dry_run"] = args.dry_run
-    print(json.dumps(report, indent=2))
-    return 0
-
-
+_module = importlib.import_module("evaluation.experiment")
+sys.modules[__name__] = _module
 if __name__ == "__main__":
-    raise SystemExit(main())
+    raise SystemExit(_module.main())

pilot/eval_judges.py

@@ -1,308 +1,7 @@
-#!/usr/bin/env python3
-"""pragent pilot — LLM-as-a-judge evaluators for the reviewer.
-
-The deterministic scorers in `eval_scores.py` measure *behaviour*: how many
-findings, how severe, how much they cost. None of them can say whether a
-finding was any good. With no human labels in `feedback.db`, a judge is the
-only thing that can — so these two ask the questions that need no ground truth,
-only the review itself:
-
-  `finding_actionability` — is each finding concrete enough to act on? A
-    reviewer that says "consider improving error handling" at file level is
-    indistinguishable from a useful one by finding count alone. This is the
-    failure mode a cheap model degrades into first.
-
-  `review_self_consistency` — does the summary agree with the findings it
-    posted? Claiming "no issues found" above a list of two criticals, or
-    describing a problem in prose that never became a finding, is a defect the
-    reviewer can commit entirely on its own.
-
-Neither judge is asked whether a finding is *correct*. That needs the diff,
-which these traces do not carry, and a judge asked to rule on correctness from
-a summary alone will confabulate. Accuracy stays an open question until humans
-start labelling — which is what `feedback_scores.py` is there to capture.
-
-**The judge is a different model from the reviewer.** The reviewer runs
-MiniMax-M2.7; the judge runs kimi-k2.7-code through the same headroom hub. A
-model grading its own output agrees with itself for reasons that have nothing
-to do with quality.
-
-Evaluators score *observations*, and their variable mapping reads the
-observation's own input/output — which is why `langfuse_trace` now writes the
-review onto the generation and not just onto the trace.
-
-Usage:
-    LANGFUSE_HOST=... LANGFUSE_PUBLIC_KEY=... LANGFUSE_SECRET_KEY=... \\
-        python3 eval_judges.py --dry-run
-"""
-from __future__ import annotations
-
-import argparse
-import json
-import os
+"""Compatibility import for evaluation judges."""
+import importlib
 import sys
-
-sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
-
-import eval_bootstrap as eb  # noqa: E402
-
-# The headroom hub in front of the local Ollama, plus a small pass-through
-# proxy (`judge-proxy` on 8802) that patches every `thinking` content block
-# to carry the `signature` field Langfuse's Anthropic adapter requires. The
-# underlying model is kimi-k2.7-code through the hub on 8790; the proxy fixes
-# the shape so Mastra's Zod parse stops failing.
-JUDGE_PROVIDER = "headroom-ollama"
-JUDGE_BASE_URL = os.environ.get("PRAGENT_JUDGE_BASE_URL", "http://100.74.17.70:8802")
-JUDGE_API_KEY = os.environ.get("PRAGENT_JUDGE_API_KEY", "ollama")
-JUDGE_MODEL = os.environ.get("PRAGENT_JUDGE_MODEL", "kimi-k2.7-code:cloud")
-
-# The trace names this project emits (`pr-review` on the trace, `opencode-review`
-# on the generation). Filter on `traceName` rather than observation `name` — the
-# observation-rule schema only exposes `traceName` as a stringOptions column, and
-# every observation inside these traces is the review itself, so the narrowness
-# is the same.
-REVIEW_TRACE_NAMES = ["pr-review", "opencode-review"]
-
-
-def _model_config() -> dict:
-    return {"provider": JUDGE_PROVIDER, "model": JUDGE_MODEL}
-
-
-JUDGES = [
-    {
-        "name": "finding_actionability",
-        "prompt": (
-            "You are auditing the output of an automated code reviewer.\n\n"
-            "PR under review:\n{{input}}\n\n"
-            "What the reviewer produced:\n{{output}}\n\n"
-            "Rate how ACTIONABLE the findings are, from 0 to 1. A finding is "
-            "actionable when a developer could act on it without asking a "
-            "follow-up question: it points at a specific location, names a "
-            "concrete problem, and proposes a fix that could be applied.\n\n"
-            "Score 1.0 when every finding is specific and fixable. Score around "
-            "0.5 when findings identify a real area but leave the developer to "
-            "work out what to change. Score near 0.0 when findings are generic "
-            "advice that would apply to almost any pull request.\n\n"
-            "Judge only specificity and actionability. You cannot see the diff, "
-            "so do NOT attempt to judge whether a finding is factually correct, "
-            "and do not penalise a finding for being one you cannot verify.\n\n"
-            "If the reviewer reported no findings at all, return 1.0 and say in "
-            "your reasoning that there was nothing to judge — a silent review is "
-            "measured by finding_rate, not here."
-        ),
-        "outputDefinition": {
-            "dataType": "NUMERIC",
-            "minValue": 0,
-            "maxValue": 1,
-            "reasoning": {
-                "description": (
-                    "Name the least actionable finding and say what it would "
-                    "need in order to be acted on."
-                )
-            },
-            "score": {"description": "0 = generic advice, 1 = every finding is specific and fixable."},
-        },
-    },
-    {
-        "name": "review_self_consistency",
-        "prompt": (
-            "You are auditing the output of an automated code reviewer.\n\n"
-            "PR under review:\n{{input}}\n\n"
-            "What the reviewer produced:\n{{output}}\n\n"
-            "The output contains a prose `summary` and a list of `findings`. "
-            "Decide whether the summary is CONSISTENT with the findings.\n\n"
-            "Inconsistent means, for example: the summary says no issues were "
-            "found while findings are listed; the summary describes a problem "
-            "that never became a finding; the summary characterises the severity "
-            "of the findings in a way the findings themselves contradict; or the "
-            "summary refers to files that appear in no finding and in no part of "
-            "the PR description.\n\n"
-            "A summary that adds context beyond the findings is NOT inconsistent "
-            "as long as nothing in it contradicts them. A review that found "
-            "nothing and says so is consistent.\n\n"
-            "You cannot see the diff. Judge the summary against the findings and "
-            "the PR title only — never against what you imagine the code does."
-        ),
-        "outputDefinition": {
-            "dataType": "BOOLEAN",
-            "reasoning": {
-                "description": "Quote the part of the summary that conflicts with the findings, if any."
-            },
-            "score": {"description": "true = summary agrees with the findings, false = it contradicts them."},
-        },
-    },
-]
-
-# Both judges read the observation's own input/output.
-MAPPING = [
-    {"variable": "input", "source": "input"},
-    {"variable": "output", "source": "output"},
-]
-
-
-# ---------------------------------------------------------------------------
-# LLM connection
-# ---------------------------------------------------------------------------
-
-def ensure_llm_connection() -> dict:
-    """Point the project at the judge model. Upserted on `provider`."""
-    body = {
-        "provider": JUDGE_PROVIDER,
-        "adapter": "anthropic",
-        "baseURL": JUDGE_BASE_URL,
-        "secretKey": JUDGE_API_KEY,
-        "customModels": [JUDGE_MODEL],
-        # The hub serves two local models and none of Anthropic's, so the
-        # default catalogue would be a list of models that all fail on use.
-        "withDefaultModels": False,
-    }
-    st, resp = eb._call("PUT", "/api/public/llm-connections", body)
-    return {"status": st, "ok": st in (200, 201), "provider": JUDGE_PROVIDER,
-            "error": None if st in (200, 201) else resp}
-
-
-# ---------------------------------------------------------------------------
-# Evaluators
-# ---------------------------------------------------------------------------
-
-def existing_evaluators() -> dict[str, str]:
-    """name -> id for evaluators already in the project."""
-    out: dict[str, str] = {}
-    st, body = eb._call("GET", "/api/public/unstable/evaluators?limit=100")
-    if st == 200 and isinstance(body, dict):
-        for ev in body.get("data") or []:
-            out[ev.get("name")] = ev.get("id")
-    return out
-
-
-def ensure_evaluators() -> dict:
-    """Create each judge if no version exists for the name yet.
-
-    POST /evaluators with a name that already exists creates a new version, not
-    a no-op — re-running this script would pile up versions until the page
-    listing them is unreadable. Skip when an evaluator of that name is present.
-    """
-    created, skipped, failed = {}, [], []
-    existing = set(existing_evaluators())
-    for judge in JUDGES:
-        if judge["name"] in existing:
-            skipped.append(judge["name"])
-            continue
-        body = {
-            "type": "llm_as_judge",
-            "name": judge["name"],
-            "prompt": judge["prompt"],
-            "outputDefinition": judge["outputDefinition"],
-            "modelConfig": _model_config(),
-        }
-        st, resp = eb._call("POST", "/api/public/unstable/evaluators", body, timeout=60.0)
-        if st in (200, 201) and isinstance(resp, dict):
-            created[judge["name"]] = resp.get("id")
-        else:
-            failed.append({"name": judge["name"], "status": st, "error": resp})
-    return {"created": created, "skipped": skipped, "failed": failed}
-
-
-# ---------------------------------------------------------------------------
-# Rules — what gets judged, and how often
-# ---------------------------------------------------------------------------
-
-def rule_body(name: str, judge_name: str, sampling: float) -> dict:
-    """POST /evaluation-rules shape for an LLM-as-judge observation rule.
-
-    The judge is referenced by `name`+`scope`, not by id — ids name specific
-    versions, names name the evaluator across versions. Mapping is required at
-    both the rule root (the server validates it there) and inside `evaluator`
-    (the API echoes it back). Filter is on `traceName` because that is the only
-    stringOptions column the observation-rule schema exposes.
-    """
-    return {
-        "name": name,
-        "enabled": True,
-        "target": "observation",
-        "sampling": sampling,
-        "filter": [
-            {"column": "traceName", "operator": "any of",
-             "value": REVIEW_TRACE_NAMES, "type": "stringOptions"},
-        ],
-        "evaluator": {
-            "name": judge_name,
-            "scope": "project",
-            "variableMapping": MAPPING,
-        },
-        "mapping": MAPPING,
-    }
-
-
-def ensure_rules(evaluator_ids: dict[str, str], sampling: float) -> dict:
-    """Idempotent: existing rules with the same name are skipped, not duplicated.
-
-    The API has no `name`-keyed upsert; the convention is to POST once and
-    re-run the script to verify the response. A duplicate POST raises 409.
-    """
-    created, failed, skipped = [], [], []
-    existing = existing_rule_names()
-    for name, eid in evaluator_ids.items():
-        if not eid:
-            continue
-        rule_name = f"{name}-on-reviews"
-        if rule_name in existing:
-            skipped.append(name)
-            continue
-        st, resp = eb._call(
-            "POST", "/api/public/unstable/evaluation-rules",
-            rule_body(rule_name, name, sampling), timeout=60.0,
-        )
-        if st in (200, 201):
-            created.append(name)
-        else:
-            failed.append({"rule": name, "status": st, "error": resp})
-    return {"created": created, "failed": failed, "skipped": skipped}
-
-
-def existing_rule_names() -> set[str]:
-    """Names of observation-target rules already in the project."""
-    out: set[str] = set()
-    st, body = eb._call("GET", "/api/public/unstable/evaluation-rules?limit=100")
-    if st == 200 and isinstance(body, dict):
-        for r in body.get("data") or []:
-            if r.get("target") == "observation":
-                out.add(r.get("name"))
-    return out
-
-
-def main(argv: list[str] | None = None) -> int:
-    ap = argparse.ArgumentParser(description=__doc__)
-    ap.add_argument("--sampling", type=float, default=1.0,
-                    help="fraction of matching observations to judge (default: all)")
-    ap.add_argument("--skip-connection", action="store_true")
-    ap.add_argument("--dry-run", action="store_true")
-    args = ap.parse_args(argv)
-
-    if args.dry_run:
-        print(json.dumps({
-            "would_connect": {"provider": JUDGE_PROVIDER, "baseURL": JUDGE_BASE_URL,
-                              "model": JUDGE_MODEL},
-            "would_create": [j["name"] for j in JUDGES],
-            "existing_evaluators": sorted(existing_evaluators()),
-            "sampling": args.sampling,
-        }, indent=2))
-        return 0
-
-    report = {}
-    if not args.skip_connection:
-        report["llm_connection"] = ensure_llm_connection()
-    report["evaluators"] = ensure_evaluators()
-    ids = dict(report["evaluators"]["created"])
-    # Fall back to whatever is already registered, so a re-run still wires rules.
-    for name, eid in existing_evaluators().items():
-        ids.setdefault(name, eid)
-    report["rules"] = ensure_rules(
-        {j["name"]: ids.get(j["name"]) for j in JUDGES}, args.sampling
-    )
-    print(json.dumps(report, indent=2))
-    return 0
-
-
+_module = importlib.import_module("evaluation.judges")
+sys.modules[__name__] = _module
 if __name__ == "__main__":
-    raise SystemExit(main())
+    raise SystemExit(_module.main())

pilot/eval_scores.py

@@ -1,233 +1,7 @@
-#!/usr/bin/env python3
-"""pragent pilot — deterministic review scorers.
-
-Four numbers computed from a review that already happened, shipped to Langfuse
-as scores on the review's trace. All are derived from data the reviewer already
-has in hand: no LLM judge, no ground truth, no extra token spend.
-
-Why these four and not `helpfulness`/`quality`
-----------------------------------------------
-They come from what the recorded reviews actually did, not from a generic eval
-checklist:
-
-  * `severity_info_ratio` — of the findings ever posted to a PR, effectively all
-    landed at `info`. Either the model will not commit to a severity or the
-    per-repo `severity_threshold` is filtering the rest out. Trending the ratio
-    per model says which.
-  * `finding_rate` — most reviews post nothing at all. Silence on clean code is
-    the goal; silence because the run degraded is a failure. Same output, two
-    causes, and only the rate over time separates them.
-  * `dropped_findings` — `ai_review.parse_findings` discards any finding whose
-    `path`/`line` is unusable. That happens silently, so a model that emits ten
-    findings at invalid locations is indistinguishable from one that found
-    nothing. This is the only signal here that measures the *model's* output
-    rather than the review's.
-  * `cost_per_finding` — the equivalent-cost number is already trended per
-    review; per finding is what actually compares two models, since a cheaper
-    model that finds nothing is not cheaper.
-
-None of these say whether a finding was *correct*. That needs labels, and the
-labels come from `feedback_scores.py` once maintainers start reacting to review
-comments. Read these as behavioural drift detectors, not as accuracy.
-
-Fail-open, like every other telemetry path here: a scorer that raises returns no
-score rather than failing the review.
-"""
-from __future__ import annotations
-
-import uuid
-from datetime import datetime, timezone
-
-# Mirrors ai_review.SEVERITY_RANK. Duplicated rather than imported because this
-# module is also run standalone (backfill) where ai_review's import side effects
-# are unwanted.
-SEVERITY_RANK = {"info": -1, "trivial": 0, "low": 1, "medium": 2, "high": 3, "critical": 4}
-
-# Findings at or below this rank are "the model declined to commit". `trivial`
-# and `info` are advisory by the reviewer's own prompt contract.
-_ADVISORY_MAX_RANK = 0
-
-# Score names. Named for what is measured, not for the mechanism producing it —
-# these land on every trace and become the axis of every chart.
-FINDING_RATE = "finding_rate"
-SEVERITY_INFO_RATIO = "severity_info_ratio"
-SEVERITY_MAX = "severity_max"
-DROPPED_FINDINGS = "dropped_findings"
-COST_PER_FINDING = "cost_per_finding"
-
-
-def _sev(f: dict) -> str:
-    return str(f.get("severity") or "medium").strip().lower()
-
-
-def finding_rate(findings: list[dict] | None) -> float:
-    """How many findings this review posted. 0.0 is the restraint case."""
-    return float(len(findings or []))
-
-
-def severity_info_ratio(findings: list[dict] | None) -> float | None:
-    """Share of findings the model rated advisory (`info`/`trivial`).
-
-    `None` for a review with no findings — a ratio over an empty set is not 0,
-    it is undefined, and charting it as 0 would read as "perfectly calibrated".
-    """
-    fs = findings or []
-    if not fs:
-        return None
-    advisory = sum(1 for f in fs if SEVERITY_RANK.get(_sev(f), 2) <= _ADVISORY_MAX_RANK)
-    return round(advisory / len(fs), 4)
-
-
-def severity_max(findings: list[dict] | None) -> str:
-    """Highest severity present, or `none` when the review was silent.
-
-    Categorical on purpose: the useful question is "did this review ever surface
-    something serious", and an average of severity ranks answers nothing.
-    """
-    fs = findings or []
-    if not fs:
-        return "none"
-    top = max(fs, key=lambda f: SEVERITY_RANK.get(_sev(f), 2))
-    sev = _sev(top)
-    return sev if sev in SEVERITY_RANK else "medium"
-
-
-def dropped_findings(raw_count: int | None, kept_count: int | None) -> float | None:
-    """Findings the model emitted that the parser could not use.
-
-    `raw_count` is what came back in the JSON; `kept_count` is what survived
-    `_normalize_finding`. `None` when the caller could not determine the raw
-    count — better no score than a fabricated zero.
-    """
-    if raw_count is None or kept_count is None:
-        return None
-    return float(max(0, int(raw_count) - int(kept_count)))
-
-
-def cost_per_finding(cost_usd: float | None, findings: list[dict] | None) -> float | None:
-    """Equivalent USD spent per finding posted.
-
-    `None` when nothing could be priced. A silent review divides by one, not by
-    zero: the run still cost money, and attributing that whole cost to "found
-    nothing" is the honest reading.
-    """
-    if cost_usd is None:
-        return None
-    try:
-        c = float(cost_usd)
-    except (TypeError, ValueError):
-        return None
-    return round(c / max(1, len(findings or [])), 6)
-
-
-def build_scores(
-    *,
-    trace_id: str,
-    findings: list[dict] | None,
-    environment: str,
-    cost_usd: float | None = None,
-    dropped_count: float | None = None,
-    timestamp: str | None = None,
-    comment: str = "",
-) -> list[dict]:
-    """The `score-create` ingestion events for one review.
-
-    `dropped_count` must be measured at parse time, not here: by the time
-    `findings` reaches this function the per-repo config has already filtered it
-    by severity threshold and `max_findings`, and those drops are the config
-    working as intended, not the model emitting garbage.
-
-    Returns [] rather than raising if something is unscoreable — scores are
-    telemetry and must never cost a review.
-    """
-    # The ingestion envelope requires a timestamp on every event; omitting it
-    # gets the whole batch rejected with an HTTP 207 whose per-event 400s are
-    # easy to mistake for success.
-    ts = timestamp or datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
-    out: list[dict] = []
-
-    def add(name: str, value, data_type: str) -> None:
-        if value is None:
-            return
-        body = {
-            "id": str(uuid.uuid4()),
-            "traceId": trace_id,
-            "name": name,
-            "dataType": data_type,
-            "environment": environment,
-        }
-        if data_type == "CATEGORICAL":
-            body["value"] = str(value)
-        else:
-            body["value"] = float(value)
-        if comment:
-            body["comment"] = comment
-        out.append(
-            {
-                "id": str(uuid.uuid4()),
-                "type": "score-create",
-                "timestamp": ts,
-                "body": body,
-            }
-        )
-
-    try:
-        add(FINDING_RATE, finding_rate(findings), "NUMERIC")
-        add(SEVERITY_INFO_RATIO, severity_info_ratio(findings), "NUMERIC")
-        add(SEVERITY_MAX, severity_max(findings), "CATEGORICAL")
-        add(DROPPED_FINDINGS, dropped_count, "NUMERIC")
-        add(COST_PER_FINDING, cost_per_finding(cost_usd, findings), "NUMERIC")
-    except Exception:  # pragma: no cover - defensive
-        return out
-    return out
-
-
-# ---------------------------------------------------------------------------
-# Score configs — the schema these scores must comply with
-# ---------------------------------------------------------------------------
-
-# Registered once per project via `eval_bootstrap.py`. Without configs the
-# scores still ingest, but nothing constrains a future scorer from writing
-# `severity_max="HIGH"` next to today's `"high"` and silently splitting the
-# series in two.
-SCORE_CONFIGS = [
-    {
-        "name": FINDING_RATE,
-        "dataType": "NUMERIC",
-        "minValue": 0,
-        "description": "Findings posted by one review. 0 = the reviewer stayed silent.",
-    },
-    {
-        "name": SEVERITY_INFO_RATIO,
-        "dataType": "NUMERIC",
-        "minValue": 0,
-        "maxValue": 1,
-        "description": "Share of a review's findings rated info/trivial. High = the model is not committing to a severity.",
-    },
-    {
-        "name": SEVERITY_MAX,
-        "dataType": "CATEGORICAL",
-        "categories": [
-            {"label": "none", "value": 0},
-            {"label": "info", "value": 1},
-            {"label": "trivial", "value": 2},
-            {"label": "low", "value": 3},
-            {"label": "medium", "value": 4},
-            {"label": "high", "value": 5},
-            {"label": "critical", "value": 6},
-        ],
-        "description": "Highest severity surfaced by one review; 'none' when it posted nothing.",
-    },
-    {
-        "name": DROPPED_FINDINGS,
-        "dataType": "NUMERIC",
-        "minValue": 0,
-        "description": "Findings the model emitted that the parser rejected for an unusable path/line.",
-    },
-    {
-        "name": COST_PER_FINDING,
-        "dataType": "NUMERIC",
-        "minValue": 0,
-        "description": "Equivalent USD per finding posted. Silent reviews divide by 1, not 0.",
-    },
-]
+"""Compatibility import for evaluation scores."""
+import importlib
+import sys
+_module = importlib.import_module("evaluation.scores")
+sys.modules[__name__] = _module
+if __name__ == "__main__":
+    raise SystemExit(_module.main())

pilot/evaluation/__init__.py

@@ -0,0 +1 @@
+"""Langfuse evaluation bootstrap, experiments, and scoring."""

pilot/evaluation/bootstrap.py

@@ -0,0 +1,350 @@
+#!/usr/bin/env python3
+"""pragent pilot — one-time Langfuse project setup for evaluation.
+
+Three jobs, each idempotent so it can be re-run after any change:
+
+  1. **Score configs.** Registers the schema for every score pragent emits
+     (`eval_scores.SCORE_CONFIGS` + `feedback_scores.SCORE_CONFIGS`). Without
+     these the scores still ingest, but nothing stops a later scorer writing
+     `severity_max="HIGH"` beside today's `"high"` and quietly splitting one
+     series into two. Configs are immutable in Langfuse — a name that already
+     exists is left alone rather than updated.
+
+  2. **Dataset.** Seeds `pragent-reviews` from `feedback.db`: one item per PR
+     the reviewer has actually run on, carrying the repo/PR/sha as input and
+     the findings it posted as `expectedOutput`.
+
+     Read `expectedOutput` here as "what the reviewer said last time", not "what
+     is correct" — no human has labelled any of it. It is a regression baseline:
+     re-run a candidate model over these PRs and the diff against this column is
+     the behaviour change. Promoting an item to real ground truth means a human
+     editing it after reviewing the PR, which is what the dataset view is for.
+
+  3. **Trace backfill** (`--backfill-traces`). Scores only ride along with new
+     reviews, so without this the charts stay empty until the next PR lands.
+     Every trace `langfuse_trace` has ever written already carries the finding
+     count, the severity histogram and the cost in its metadata, which is
+     everything four of the five scorers need. `dropped_findings` is absent from
+     historical traces and is left unscored rather than backfilled as zero.
+
+  4. **Reports** what it found, so the gap between "reviews recorded" and
+     "reviews with human feedback" is visible rather than assumed.
+
+Usage:
+    LANGFUSE_HOST=... LANGFUSE_PUBLIC_KEY=... LANGFUSE_SECRET_KEY=... \\
+        python3 eval_bootstrap.py --db /data/feedback.db
+"""
+from __future__ import annotations
+
+import argparse
+import base64
+import json
+import os
+import sqlite3
+import sys
+import urllib.error
+import urllib.request
+from datetime import datetime, timezone
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+import eval_scores  # noqa: E402
+import feedback_scores  # noqa: E402
+
+DATASET_NAME = "pragent-reviews"
+
+
+def _conf() -> tuple[str, str, str]:
+    host = (os.environ.get("LANGFUSE_HOST") or "").strip().rstrip("/")
+    pk = (os.environ.get("LANGFUSE_PUBLIC_KEY") or "").strip()
+    sk = (os.environ.get("LANGFUSE_SECRET_KEY") or "").strip()
+    if not host or not pk or not sk:
+        raise SystemExit("LANGFUSE_HOST / LANGFUSE_PUBLIC_KEY / LANGFUSE_SECRET_KEY must be set")
+    return host, pk, sk
+
+
+def _call(method: str, path: str, body: dict | None = None, timeout: float = 20.0):
+    host, pk, sk = _conf()
+    auth = base64.b64encode(f"{pk}:{sk}".encode()).decode("ascii")
+    data = json.dumps(body).encode() if body is not None else None
+    req = urllib.request.Request(
+        host + path,
+        data=data,
+        headers={
+            "Content-Type": "application/json",
+            "Authorization": f"Basic {auth}",
+            "User-Agent": "pragent-pilot/1.0",
+        },
+        method=method,
+    )
+    try:
+        with urllib.request.urlopen(req, timeout=timeout) as resp:
+            raw = resp.read()
+            return resp.status, (json.loads(raw) if raw else None)
+    except urllib.error.HTTPError as e:
+        return e.code, e.read()[:400].decode("utf-8", "replace")
+
+
+# ---------------------------------------------------------------------------
+# 1. Score configs
+# ---------------------------------------------------------------------------
+
+def ensure_score_configs() -> dict:
+    status, existing = _call("GET", "/api/public/score-configs?limit=100")
+    have = set()
+    if status == 200 and isinstance(existing, dict):
+        have = {c.get("name") for c in existing.get("data", [])}
+
+    created, skipped, failed = [], [], []
+    for cfg in list(eval_scores.SCORE_CONFIGS) + list(feedback_scores.SCORE_CONFIGS):
+        if cfg["name"] in have:
+            skipped.append(cfg["name"])
+            continue
+        st, resp = _call("POST", "/api/public/score-configs", cfg)
+        if st in (200, 201):
+            created.append(cfg["name"])
+        else:
+            failed.append({"name": cfg["name"], "status": st, "error": resp})
+    return {"created": created, "already_present": skipped, "failed": failed}
+
+
+# ---------------------------------------------------------------------------
+# 2. Dataset from recorded reviews
+# ---------------------------------------------------------------------------
+
+def item_id(repo: str, pr) -> str:
+    """A dataset-item id that survives being put in a URL path.
+
+    The obvious `{repo}#{pr}` is unusable: the UI routes items as
+    `/datasets/{id}/items/{item_id}`, so the `/` in `owner/repo` splits into
+    extra path segments and everything after the `#` is a fragment the browser
+    never sends. The item is created fine and then 404s when opened.
+
+    Session ids elsewhere keep the `{repo}#{pr}` form — those are never path
+    segments, and `feedback_scores` depends on that shape.
+    """
+    return f"{repo.replace('/', '__')}__pr{pr}"
+
+
+def _item_metadata(*, repo, pr, head_sha, reviews_run, last_seen, findings) -> dict:
+    """Filterable facets for one dataset item.
+
+    Kept flat and primitive: the filter bar matches a metadata key against a
+    literal, so a nested object or a list is not reachable from the UI.
+    """
+    owner, _, repo_name = str(repo).partition("/")
+    sevs = [str(f["severity"] or "").lower() for f in findings]
+    ranked = [s for s in sevs if s in eval_scores.SEVERITY_RANK]
+    return {
+        "repo": repo,
+        "owner": owner or repo,
+        "repo_name": repo_name or repo,
+        "pr": int(pr),
+        "head_sha": head_sha,
+        "reviews_run": reviews_run,
+        "last_reviewed_at": last_seen,
+        "last_reviewed_iso": datetime.fromtimestamp(last_seen, timezone.utc).isoformat(),
+        "finding_count": len(findings),
+        "has_findings": bool(findings),
+        # "none" rather than omitting the key: a filter for silent reviews needs
+        # something to match, and an absent key matches nothing.
+        "max_severity": (
+            max(ranked, key=lambda s: eval_scores.SEVERITY_RANK[s]) if ranked else "none"
+        ),
+        # Flags that this row is the reviewer's own past output, not a human
+        # judgement. Filter on it before anyone treats the dataset as truth.
+        "labelled_by_human": False,
+    }
+
+
+def read_review_items(db_path: str) -> list[dict]:
+    """One dataset item per (repo, pr) the reviewer has run on.
+
+    Keyed on the PR rather than on each individual review row: the same PR is
+    re-reviewed on every push, and 113 rows over 26 PRs would make a benchmark
+    that is 4x redundant and weighted towards whichever PR churned most.
+    """
+    conn = sqlite3.connect(db_path)
+    conn.row_factory = sqlite3.Row
+    try:
+        prs = conn.execute(
+            """
+            SELECT repo, pr, MAX(posted_at) AS last_seen, COUNT(*) AS reviews,
+                   MAX(head_sha) AS head_sha
+            FROM review GROUP BY repo, pr ORDER BY repo, pr
+            """
+        ).fetchall()
+        items = []
+        for row in prs:
+            findings = conn.execute(
+                """
+                SELECT path, line, severity, problem, fix
+                FROM inline_finding WHERE repo = ? AND pr = ?
+                ORDER BY path, line
+                """,
+                (row["repo"], row["pr"]),
+            ).fetchall()
+            items.append(
+                {
+                    "id": item_id(row["repo"], row["pr"]),
+                    "input": {
+                        "repo": row["repo"],
+                        "pr": int(row["pr"]),
+                        "head_sha": row["head_sha"],
+                    },
+                    "expectedOutput": {
+                        "findings": [dict(f) for f in findings],
+                        "finding_count": len(findings),
+                    },
+                    # The UI's filter bar reads metadata and nothing else, so
+                    # anything worth slicing on is a top-level key here even
+                    # where it duplicates `input`. `owner` and `repo_name` are
+                    # split out because a filter on the joined `repo` can only
+                    # match one repo at a time, never a whole org.
+                    "metadata": _item_metadata(
+                        repo=row["repo"],
+                        pr=row["pr"],
+                        head_sha=row["head_sha"],
+                        reviews_run=int(row["reviews"]),
+                        last_seen=int(row["last_seen"]),
+                        findings=findings,
+                    ),
+                }
+            )
+        return items
+    finally:
+        conn.close()
+
+
+def ensure_dataset(items: list[dict], name: str = DATASET_NAME) -> dict:
+    st, _ = _call(
+        "POST",
+        "/api/public/datasets",
+        {
+            "name": name,
+            "description": (
+                "PRs the pragent pilot has reviewed, seeded from feedback.db. "
+                "expectedOutput is the reviewer's own prior output — a regression "
+                "baseline, not human-verified ground truth."
+            ),
+            "metadata": {"source": "feedback.db", "seeded_by": "eval_bootstrap.py"},
+        },
+    )
+    # A duplicate name is fine: the dataset already exists from an earlier run.
+    dataset_ok = st in (200, 201, 409)
+
+    created, failed = 0, []
+    for item in items:
+        body = {
+            "datasetName": name,
+            "id": item["id"],  # idempotent: same PR updates rather than duplicates
+            "input": item["input"],
+            "expectedOutput": item["expectedOutput"],
+            "metadata": item["metadata"],
+        }
+        ist, resp = _call("POST", "/api/public/dataset-items", body)
+        if ist in (200, 201):
+            created += 1
+        else:
+            failed.append({"item": item["id"], "status": ist, "error": resp})
+    return {"dataset": name, "dataset_created": dataset_ok, "items_upserted": created, "failed": failed}
+
+
+# ---------------------------------------------------------------------------
+# 3. Backfill scores onto traces that predate the scorers
+# ---------------------------------------------------------------------------
+
+def _synth_findings(severities: dict) -> list[dict]:
+    """Rebuild a findings list from a trace's severity histogram.
+
+    Only severity matters to the scorers, and that is all the histogram kept.
+    Reconstructing placeholders is honest here because every scorer being
+    backfilled reads nothing else off a finding.
+    """
+    out = []
+    for sev, count in (severities or {}).items():
+        out.extend({"severity": sev} for _ in range(int(count)))
+    return out
+
+
+def backfill_traces(limit_pages: int = 20) -> dict:
+    import eval_scores as es
+
+    scored, skipped, events = 0, 0, []
+    page = 1
+    while page <= limit_pages:
+        st, resp = _call("GET", f"/api/public/traces?limit=50&page={page}&name=pr-review")
+        if st != 200 or not isinstance(resp, dict):
+            break
+        rows = resp.get("data") or []
+        if not rows:
+            break
+        for tr in rows:
+            meta = tr.get("metadata") or {}
+            severities = meta.get("severities") or {}
+            count = meta.get("findings")
+            if count is None:
+                skipped += 1
+                continue
+            findings = _synth_findings(severities)
+            # The histogram is authoritative when present; a trace that recorded
+            # a count but no histogram still scores its rate.
+            if not findings and count:
+                findings = [{"severity": "medium"} for _ in range(int(count))]
+            batch = es.build_scores(
+                trace_id=tr["id"],
+                findings=findings,
+                environment=tr.get("environment") or "default",
+                cost_usd=(tr.get("totalCost") or meta.get("provider_cost_usd")),
+                timestamp=tr.get("timestamp"),
+                comment="backfilled from trace metadata",
+            )
+            events.extend(batch)
+            scored += 1
+        page += 1
+
+    posted = False
+    status = None
+    if events:
+        import langfuse_trace
+
+        host, pk, sk = _conf()
+        # Chunked: one 2000-event POST is refused, and a partial backfill that
+        # reports success is worse than a slow one.
+        for i in range(0, len(events), 200):
+            status = langfuse_trace._post(host, pk, sk, events[i:i + 200], 30.0)
+            posted = status in (200, 201, 207)
+            if not posted:
+                break
+    return {"traces_scored": scored, "traces_skipped": skipped, "scores": len(events),
+            "posted": posted, "http_status": status}
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description="Bootstrap Langfuse evaluation for the pragent pilot")
+    ap.add_argument("--db", default=os.environ.get("PRAGENT_FEEDBACK_DB", "/data/feedback.db"))
+    ap.add_argument("--skip-dataset", action="store_true")
+    ap.add_argument("--skip-configs", action="store_true")
+    ap.add_argument("--backfill-traces", action="store_true",
+                    help="score traces written before the scorers existed")
+    args = ap.parse_args()
+
+    out: dict = {}
+    if not args.skip_configs:
+        out["score_configs"] = ensure_score_configs()
+    if not args.skip_dataset:
+        items = read_review_items(args.db)
+        out["dataset"] = ensure_dataset(items)
+        out["dataset"]["items_read"] = len(items)
+    if args.backfill_traces:
+        out["trace_backfill"] = backfill_traces()
+    print(json.dumps(out, indent=2))
+
+    failed = (out.get("score_configs", {}).get("failed") or []) + (
+        out.get("dataset", {}).get("failed") or []
+    )
+    return 1 if failed else 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

pilot/evaluation/experiment.py

@@ -0,0 +1,212 @@
+#!/usr/bin/env python3
+"""pragent pilot — populate the Experiments tab from reviews already traced.
+
+An "experiment" in Langfuse is a dataset run: a set of (dataset item, trace)
+links under one run name. The Experiments tab then shows one row per item with
+its scores, and lets two runs be diffed side by side.
+
+Nothing here re-runs the reviewer. Every PR in `pragent-reviews` has already
+been reviewed, and each of those reviews left a trace carrying its findings,
+cost and scores. This links what exists, which is what makes the tab useful on
+day one instead of after the next N pushes.
+
+Runs are grouped by **model** by default, because that is the comparison the
+pilot actually needs to make: the same PRs reviewed by MiniMax vs whatever
+replaces it, with `finding_rate` and `cost_per_finding` side by side. Group by
+`none` for a single "all traces" run.
+
+One trace per (run, item) — the most recent. A PR re-reviewed on every push has
+many traces, and a dataset run is defined as one output per input; feeding it
+the other five would make the per-run averages meaningless.
+
+Note on the endpoint: `POST /api/public/dataset-run-items` is deprecated in
+favour of the SDK experiment runner / OTel ingestion, and disappears in
+Langfuse v4. This instance is self-hosted v3, which the deprecation notice
+explicitly exempts from the cutoff date, and the pilot is stdlib-only by
+design. Revisit when this deployment moves to v4.
+
+Usage:
+    LANGFUSE_HOST=... LANGFUSE_PUBLIC_KEY=... LANGFUSE_SECRET_KEY=... \\
+        python3 eval_experiment.py --dry-run
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import sys
+import urllib.parse
+from collections import defaultdict
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+import eval_bootstrap as eb  # noqa: E402
+
+TRACE_NAME = "pr-review"
+
+
+# ---------------------------------------------------------------------------
+# Reading what already exists
+# ---------------------------------------------------------------------------
+
+def fetch_traces(name: str = TRACE_NAME, limit: int = 100, max_pages: int = 50) -> list[dict]:
+    """Every review trace, newest first."""
+    out: list[dict] = []
+    for page in range(1, max_pages + 1):
+        q = urllib.parse.urlencode({"name": name, "limit": limit, "page": page})
+        st, body = eb._call("GET", f"/api/public/traces?{q}")
+        if st != 200 or not isinstance(body, dict):
+            raise SystemExit(f"listing traces failed: {st} {body}")
+        data = body.get("data") or []
+        out.extend(data)
+        meta = body.get("meta") or {}
+        if page * meta.get("limit", limit) >= meta.get("totalItems", 0):
+            break
+    return out
+
+
+def fetch_item_ids(dataset: str) -> set[str]:
+    """Ids present in the dataset, so runs never reference a missing item."""
+    ids: set[str] = set()
+    for page in range(1, 51):
+        q = urllib.parse.urlencode({"datasetName": dataset, "limit": 100, "page": page})
+        st, body = eb._call("GET", f"/api/public/dataset-items?{q}")
+        if st != 200 or not isinstance(body, dict):
+            raise SystemExit(f"listing dataset items failed: {st} {body}")
+        ids.update(i["id"] for i in body.get("data") or [])
+        meta = body.get("meta") or {}
+        if page * meta.get("limit", 100) >= meta.get("totalItems", 0):
+            break
+    return ids
+
+
+# ---------------------------------------------------------------------------
+# Grouping traces into runs
+# ---------------------------------------------------------------------------
+
+def trace_model(trace: dict) -> str:
+    """The model that produced a review, from its `model:` tag."""
+    for tag in trace.get("tags") or []:
+        if tag.startswith("model:"):
+            return tag[len("model:"):] or "unknown"
+    return "unknown"
+
+
+def trace_item_id(trace: dict) -> str | None:
+    """The dataset item a trace belongs to, or None if it is not a PR review."""
+    md = trace.get("metadata") or {}
+    repo, pr = md.get("repo"), md.get("pr")
+    if not repo or pr in (None, ""):
+        return None
+    return eb.item_id(str(repo), pr)
+
+
+def _sort_key(trace: dict):
+    return (trace.get("timestamp") or "", trace.get("id") or "")
+
+
+def plan_runs(traces: list[dict], known_items: set[str], group_by: str = "model") -> dict:
+    """Map run name -> {item id: trace}, keeping only the newest trace per item.
+
+    Traces whose PR is not in the dataset are dropped: `feedback.db` is the
+    source for both, but a review can be traced without its row landing (the
+    posting step can fail after the model ran), and a run item pointing at a
+    non-existent dataset item is rejected.
+    """
+    runs: dict[str, dict[str, dict]] = defaultdict(dict)
+    skipped_no_item, skipped_unknown = 0, 0
+    for tr in traces:
+        iid = trace_item_id(tr)
+        if iid is None:
+            skipped_unknown += 1
+            continue
+        if iid not in known_items:
+            skipped_no_item += 1
+            continue
+        run = "all-traces" if group_by == "none" else trace_model(tr)
+        prev = runs[run].get(iid)
+        if prev is None or _sort_key(tr) > _sort_key(prev):
+            runs[run][iid] = tr
+    return {
+        "runs": dict(runs),
+        "skipped_not_in_dataset": skipped_no_item,
+        "skipped_not_a_review": skipped_unknown,
+    }
+
+
+def run_name(prefix: str, key: str) -> str:
+    return f"{prefix}-{key}" if prefix else key
+
+
+# ---------------------------------------------------------------------------
+# Writing the runs
+# ---------------------------------------------------------------------------
+
+def create_run(name: str, items: dict[str, dict], description: str = "") -> dict:
+    """Link each (item, trace) pair into the named run. Idempotent per pair."""
+    created, failed = 0, []
+    for iid, tr in sorted(items.items()):
+        md = tr.get("metadata") or {}
+        body = {
+            "runName": name,
+            "runDescription": description,
+            "datasetItemId": iid,
+            "traceId": tr["id"],
+            "metadata": {
+                "model": trace_model(tr),
+                "engine": md.get("engine"),
+                "findings": md.get("findings"),
+                "duration_s": md.get("duration_s"),
+                "cost_basis": md.get("cost_basis"),
+                "linked_by": "eval_experiment.py",
+            },
+        }
+        st, resp = eb._call("POST", "/api/public/dataset-run-items", body)
+        if st in (200, 201):
+            created += 1
+        else:
+            failed.append({"item": iid, "status": st, "error": resp})
+    return {"run": name, "items_linked": created, "failed": failed}
+
+
+def main(argv: list[str] | None = None) -> int:
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument("--dataset", default=eb.DATASET_NAME)
+    ap.add_argument("--group-by", choices=("model", "none"), default="model")
+    ap.add_argument("--prefix", default="baseline",
+                    help="run name prefix; '' for the bare group key")
+    ap.add_argument("--dry-run", action="store_true")
+    args = ap.parse_args(argv)
+
+    traces = fetch_traces()
+    items = fetch_item_ids(args.dataset)
+    plan = plan_runs(traces, items, group_by=args.group_by)
+
+    report = {
+        "traces_read": len(traces),
+        "dataset_items": len(items),
+        "skipped_not_in_dataset": plan["skipped_not_in_dataset"],
+        "skipped_not_a_review": plan["skipped_not_a_review"],
+        "runs": {},
+    }
+    for key, mapping in sorted(plan["runs"].items()):
+        name = run_name(args.prefix, key)
+        if args.dry_run:
+            report["runs"][name] = {"items_would_link": len(mapping)}
+            continue
+        report["runs"][name] = create_run(
+            name,
+            mapping,
+            description=(
+                "Reviews already run by the pilot, linked after the fact. "
+                "Scores come from the traces; expectedOutput is the reviewer's "
+                "own prior output, not human-verified ground truth."
+            ),
+        )
+    report["dry_run"] = args.dry_run
+    print(json.dumps(report, indent=2))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

pilot/evaluation/judges.py

@@ -0,0 +1,314 @@
+#!/usr/bin/env python3
+"""pragent pilot — LLM-as-a-judge evaluators for the reviewer.
+
+The deterministic scorers in `eval_scores.py` measure *behaviour*: how many
+findings, how severe, how much they cost. None of them can say whether a
+finding was any good. With no human labels in `feedback.db`, a judge is the
+only thing that can — so these two ask the questions that need no ground truth,
+only the review itself:
+
+  `finding_actionability` — is each finding concrete enough to act on? A
+    reviewer that says "consider improving error handling" at file level is
+    indistinguishable from a useful one by finding count alone. This is the
+    failure mode a cheap model degrades into first.
+
+  `review_self_consistency` — does the summary agree with the findings it
+    posted? Claiming "no issues found" above a list of two criticals, or
+    describing a problem in prose that never became a finding, is a defect the
+    reviewer can commit entirely on its own.
+
+Neither judge is asked whether a finding is *correct*. That needs the diff,
+which these traces do not carry, and a judge asked to rule on correctness from
+a summary alone will confabulate. Accuracy stays an open question until humans
+start labelling — which is what `feedback_scores.py` is there to capture.
+
+**The judge is a different model from the reviewer.** The reviewer runs
+MiniMax-M2.7; the judge runs kimi-k2.7-code through the same headroom hub. A
+model grading its own output agrees with itself for reasons that have nothing
+to do with quality.
+
+Evaluators score *observations*, and their variable mapping reads the
+observation's own input/output — which is why `langfuse_trace` now writes the
+review onto the generation and not just onto the trace.
+
+Usage:
+    LANGFUSE_HOST=... LANGFUSE_PUBLIC_KEY=... LANGFUSE_SECRET_KEY=... \\
+        python3 eval_judges.py --dry-run
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import sys
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+import eval_bootstrap as eb  # noqa: E402
+
+# The headroom hub in front of the local Ollama, plus a small pass-through
+# proxy (`judge-proxy` on 8802) that patches every `thinking` content block
+# to carry the `signature` field Langfuse's Anthropic adapter requires. The
+# underlying model is kimi-k2.7-code through the hub on 8790; the proxy fixes
+# the shape so Mastra's Zod parse stops failing.
+JUDGE_PROVIDER = "headroom-ollama"
+JUDGE_BASE_URL = os.environ.get("PRAGENT_JUDGE_BASE_URL", "http://100.74.17.70:8802")
+JUDGE_API_KEY = os.environ.get("PRAGENT_JUDGE_API_KEY", "ollama")
+JUDGE_MODEL = os.environ.get("PRAGENT_JUDGE_MODEL", "kimi-k2.7-code:cloud")
+
+# The trace names this project emits (`pr-review` on the trace, `opencode-review`
+# on the generation). Filter on `traceName` rather than observation `name` — the
+# observation-rule schema only exposes `traceName` as a stringOptions column, and
+# every observation inside these traces is the review itself, so the narrowness
+# is the same.
+REVIEW_TRACE_NAMES = ["pr-review", "opencode-review"]
+
+
+def _model_config() -> dict:
+    return {"provider": JUDGE_PROVIDER, "model": JUDGE_MODEL}
+
+
+JUDGES = [
+    {
+        "name": "finding_actionability",
+        "prompt": (
+            "You are auditing the output of an automated code reviewer.\n\n"
+            "PR under review:\n{{input}}\n\n"
+            "What the reviewer produced:\n{{output}}\n\n"
+            "Rate how ACTIONABLE the findings are, from 0 to 1. A finding is "
+            "actionable when a developer could act on it without asking a "
+            "follow-up question: it points at a specific location, names a "
+            "concrete problem, and proposes a fix that could be applied.\n\n"
+            "Score 1.0 when every finding is specific and fixable. Score around "
+            "0.5 when findings identify a real area but leave the developer to "
+            "work out what to change. Score near 0.0 when findings are generic "
+            "advice that would apply to almost any pull request.\n\n"
+            "Judge only specificity and actionability. You cannot see the diff, "
+            "so do NOT attempt to judge whether a finding is factually correct, "
+            "and do not penalise a finding for being one you cannot verify.\n\n"
+            "If the reviewer reported no findings at all, return 1.0 and say in "
+            "your reasoning that there was nothing to judge — a silent review is "
+            "measured by finding_rate, not here."
+        ),
+        "outputDefinition": {
+            "dataType": "NUMERIC",
+            "minValue": 0,
+            "maxValue": 1,
+            "reasoning": {
+                "description": (
+                    "Name the least actionable finding and say what it would "
+                    "need in order to be acted on."
+                )
+            },
+            "score": {"description": "0 = generic advice, 1 = every finding is specific and fixable."},
+        },
+    },
+    {
+        "name": "review_self_consistency",
+        "prompt": (
+            "You are auditing the output of an automated code reviewer.\n\n"
+            "PR under review:\n{{input}}\n\n"
+            "What the reviewer produced:\n{{output}}\n\n"
+            "The output contains a prose `summary` and a list of `findings`. "
+            "Decide whether the summary is CONSISTENT with the findings.\n\n"
+            "Inconsistent means, for example: the summary says no issues were "
+            "found while findings are listed; the summary describes a problem "
+            "that never became a finding; the summary characterises the severity "
+            "of the findings in a way the findings themselves contradict; or the "
+            "summary refers to files that appear in no finding and in no part of "
+            "the PR description.\n\n"
+            "A summary that adds context beyond the findings is NOT inconsistent "
+            "as long as nothing in it contradicts them. A review that found "
+            "nothing and says so is consistent.\n\n"
+            "You cannot see the diff. Judge the summary against the findings and "
+            "the PR title only — never against what you imagine the code does."
+        ),
+        "outputDefinition": {
+            "dataType": "BOOLEAN",
+            "reasoning": {
+                "description": "Quote the part of the summary that conflicts with the findings, if any."
+            },
+            "score": {"description": "true = summary agrees with the findings, false = it contradicts them."},
+        },
+    },
+]
+
+# Both judges read the observation's own input/output.
+MAPPING = [
+    {"variable": "input", "source": "input"},
+    {"variable": "output", "source": "output"},
+]
+
+
+# ---------------------------------------------------------------------------
+# LLM connection
+# ---------------------------------------------------------------------------
+
+def ensure_llm_connection() -> dict:
+    """Point the project at the judge model. Upserted on `provider`."""
+    body = {
+        "provider": JUDGE_PROVIDER,
+        "adapter": "anthropic",
+        "baseURL": JUDGE_BASE_URL,
+        "secretKey": JUDGE_API_KEY,
+        "customModels": [JUDGE_MODEL],
+        # The hub serves two local models and none of Anthropic's, so the
+        # default catalogue would be a list of models that all fail on use.
+        "withDefaultModels": False,
+    }
+    st, resp = eb._call("PUT", "/api/public/llm-connections", body)
+    return {"status": st, "ok": st in (200, 201), "provider": JUDGE_PROVIDER,
+            "error": None if st in (200, 201) else resp}
+
+
+# ---------------------------------------------------------------------------
+# Evaluators
+# ---------------------------------------------------------------------------
+
+def existing_evaluators() -> dict[str, str]:
+    """name -> id for evaluators already in the project."""
+    out: dict[str, str] = {}
+    st, body = eb._call("GET", "/api/public/unstable/evaluators?limit=100")
+    if st == 200 and isinstance(body, dict):
+        for ev in body.get("data") or []:
+            out[ev.get("name")] = ev.get("id")
+    return out
+
+
+def ensure_evaluators() -> dict:
+    """Create each judge if no version exists for the name yet.
+
+    POST /evaluators with a name that already exists creates a new version, not
+    a no-op — re-running this script would pile up versions until the page
+    listing them is unreadable. Skip when an evaluator of that name is present.
+    """
+    created, skipped, failed = {}, [], []
+    existing = set(existing_evaluators())
+    for judge in JUDGES:
+        if judge["name"] in existing:
+            skipped.append(judge["name"])
+            continue
+        body = {
+            "type": "llm_as_judge",
+            "name": judge["name"],
+            "prompt": judge["prompt"],
+            "outputDefinition": judge["outputDefinition"],
+            "modelConfig": _model_config(),
+        }
+        st, resp = eb._call("POST", "/api/public/unstable/evaluators", body, timeout=60.0)
+        if st in (200, 201) and isinstance(resp, dict):
+            created[judge["name"]] = resp.get("id")
+        else:
+            failed.append({"name": judge["name"], "status": st, "error": resp})
+    return {"created": created, "skipped": skipped, "failed": failed}
+
+
+# ---------------------------------------------------------------------------
+# Rules — what gets judged, and how often
+# ---------------------------------------------------------------------------
+
+def rule_body(name: str, judge_name: str, sampling: float) -> dict:
+    """POST /evaluation-rules shape for an LLM-as-judge trace rule.
+
+    Target is `trace` rather than `observation` on purpose: the standard
+    `/api/public/ingestion` path that ships review traces here feeds only
+    the trace-upsert queue, and `evalService.createEvalJobs` only creates
+    jobs for `targetObject ∈ {TRACE, DATASET}`. Observation rules are
+    triggered exclusively from the OTel ingestion pipeline, which this
+    pilot does not use. A trace rule reads the trace's own input/output —
+    `langfuse_trace` already writes `_review_input`/`_review_output` onto
+    the trace body for exactly this reason.
+
+    Mapping is required at both the rule root (server validates it there)
+    and inside `evaluator` (the API echoes it back).
+    """
+    return {
+        "name": name,
+        "enabled": True,
+        "target": "trace",
+        "sampling": sampling,
+        "filter": [
+            {"column": "traceName", "operator": "any of",
+             "value": REVIEW_TRACE_NAMES, "type": "stringOptions"},
+        ],
+        "evaluator": {
+            "name": judge_name,
+            "scope": "project",
+            "variableMapping": MAPPING,
+        },
+        "mapping": MAPPING,
+    }
+
+
+def ensure_rules(evaluator_ids: dict[str, str], sampling: float) -> dict:
+    """Idempotent: existing rules with the same name are skipped, not duplicated.
+
+    The API has no `name`-keyed upsert; the convention is to POST once and
+    re-run the script to verify the response. A duplicate POST raises 409.
+    """
+    created, failed, skipped = [], [], []
+    existing = existing_rule_names()
+    for name, eid in evaluator_ids.items():
+        if not eid:
+            continue
+        rule_name = f"{name}-on-reviews"
+        if rule_name in existing:
+            skipped.append(name)
+            continue
+        st, resp = eb._call(
+            "POST", "/api/public/unstable/evaluation-rules",
+            rule_body(rule_name, name, sampling), timeout=60.0,
+        )
+        if st in (200, 201):
+            created.append(name)
+        else:
+            failed.append({"rule": name, "status": st, "error": resp})
+    return {"created": created, "failed": failed, "skipped": skipped}
+
+
+def existing_rule_names() -> set[str]:
+    """Names of observation-target rules already in the project."""
+    out: set[str] = set()
+    st, body = eb._call("GET", "/api/public/unstable/evaluation-rules?limit=100")
+    if st == 200 and isinstance(body, dict):
+        for r in body.get("data") or []:
+            if r.get("target") == "observation":
+                out.add(r.get("name"))
+    return out
+
+
+def main(argv: list[str] | None = None) -> int:
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument("--sampling", type=float, default=1.0,
+                    help="fraction of matching observations to judge (default: all)")
+    ap.add_argument("--skip-connection", action="store_true")
+    ap.add_argument("--dry-run", action="store_true")
+    args = ap.parse_args(argv)
+
+    if args.dry_run:
+        print(json.dumps({
+            "would_connect": {"provider": JUDGE_PROVIDER, "baseURL": JUDGE_BASE_URL,
+                              "model": JUDGE_MODEL},
+            "would_create": [j["name"] for j in JUDGES],
+            "existing_evaluators": sorted(existing_evaluators()),
+            "sampling": args.sampling,
+        }, indent=2))
+        return 0
+
+    report = {}
+    if not args.skip_connection:
+        report["llm_connection"] = ensure_llm_connection()
+    report["evaluators"] = ensure_evaluators()
+    ids = dict(report["evaluators"]["created"])
+    # Fall back to whatever is already registered, so a re-run still wires rules.
+    for name, eid in existing_evaluators().items():
+        ids.setdefault(name, eid)
+    report["rules"] = ensure_rules(
+        {j["name"]: ids.get(j["name"]) for j in JUDGES}, args.sampling
+    )
+    print(json.dumps(report, indent=2))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

pilot/evaluation/scores.py

@@ -0,0 +1,233 @@
+#!/usr/bin/env python3
+"""pragent pilot — deterministic review scorers.
+
+Four numbers computed from a review that already happened, shipped to Langfuse
+as scores on the review's trace. All are derived from data the reviewer already
+has in hand: no LLM judge, no ground truth, no extra token spend.
+
+Why these four and not `helpfulness`/`quality`
+----------------------------------------------
+They come from what the recorded reviews actually did, not from a generic eval
+checklist:
+
+  * `severity_info_ratio` — of the findings ever posted to a PR, effectively all
+    landed at `info`. Either the model will not commit to a severity or the
+    per-repo `severity_threshold` is filtering the rest out. Trending the ratio
+    per model says which.
+  * `finding_rate` — most reviews post nothing at all. Silence on clean code is
+    the goal; silence because the run degraded is a failure. Same output, two
+    causes, and only the rate over time separates them.
+  * `dropped_findings` — `ai_review.parse_findings` discards any finding whose
+    `path`/`line` is unusable. That happens silently, so a model that emits ten
+    findings at invalid locations is indistinguishable from one that found
+    nothing. This is the only signal here that measures the *model's* output
+    rather than the review's.
+  * `cost_per_finding` — the equivalent-cost number is already trended per
+    review; per finding is what actually compares two models, since a cheaper
+    model that finds nothing is not cheaper.
+
+None of these say whether a finding was *correct*. That needs labels, and the
+labels come from `feedback_scores.py` once maintainers start reacting to review
+comments. Read these as behavioural drift detectors, not as accuracy.
+
+Fail-open, like every other telemetry path here: a scorer that raises returns no
+score rather than failing the review.
+"""
+from __future__ import annotations
+
+import uuid
+from datetime import datetime, timezone
+
+# Mirrors ai_review.SEVERITY_RANK. Duplicated rather than imported because this
+# module is also run standalone (backfill) where ai_review's import side effects
+# are unwanted.
+SEVERITY_RANK = {"info": -1, "trivial": 0, "low": 1, "medium": 2, "high": 3, "critical": 4}
+
+# Findings at or below this rank are "the model declined to commit". `trivial`
+# and `info` are advisory by the reviewer's own prompt contract.
+_ADVISORY_MAX_RANK = 0
+
+# Score names. Named for what is measured, not for the mechanism producing it —
+# these land on every trace and become the axis of every chart.
+FINDING_RATE = "finding_rate"
+SEVERITY_INFO_RATIO = "severity_info_ratio"
+SEVERITY_MAX = "severity_max"
+DROPPED_FINDINGS = "dropped_findings"
+COST_PER_FINDING = "cost_per_finding"
+
+
+def _sev(f: dict) -> str:
+    return str(f.get("severity") or "medium").strip().lower()
+
+
+def finding_rate(findings: list[dict] | None) -> float:
+    """How many findings this review posted. 0.0 is the restraint case."""
+    return float(len(findings or []))
+
+
+def severity_info_ratio(findings: list[dict] | None) -> float | None:
+    """Share of findings the model rated advisory (`info`/`trivial`).
+
+    `None` for a review with no findings — a ratio over an empty set is not 0,
+    it is undefined, and charting it as 0 would read as "perfectly calibrated".
+    """
+    fs = findings or []
+    if not fs:
+        return None
+    advisory = sum(1 for f in fs if SEVERITY_RANK.get(_sev(f), 2) <= _ADVISORY_MAX_RANK)
+    return round(advisory / len(fs), 4)
+
+
+def severity_max(findings: list[dict] | None) -> str:
+    """Highest severity present, or `none` when the review was silent.
+
+    Categorical on purpose: the useful question is "did this review ever surface
+    something serious", and an average of severity ranks answers nothing.
+    """
+    fs = findings or []
+    if not fs:
+        return "none"
+    top = max(fs, key=lambda f: SEVERITY_RANK.get(_sev(f), 2))
+    sev = _sev(top)
+    return sev if sev in SEVERITY_RANK else "medium"
+
+
+def dropped_findings(raw_count: int | None, kept_count: int | None) -> float | None:
+    """Findings the model emitted that the parser could not use.
+
+    `raw_count` is what came back in the JSON; `kept_count` is what survived
+    `_normalize_finding`. `None` when the caller could not determine the raw
+    count — better no score than a fabricated zero.
+    """
+    if raw_count is None or kept_count is None:
+        return None
+    return float(max(0, int(raw_count) - int(kept_count)))
+
+
+def cost_per_finding(cost_usd: float | None, findings: list[dict] | None) -> float | None:
+    """Equivalent USD spent per finding posted.
+
+    `None` when nothing could be priced. A silent review divides by one, not by
+    zero: the run still cost money, and attributing that whole cost to "found
+    nothing" is the honest reading.
+    """
+    if cost_usd is None:
+        return None
+    try:
+        c = float(cost_usd)
+    except (TypeError, ValueError):
+        return None
+    return round(c / max(1, len(findings or [])), 6)
+
+
+def build_scores(
+    *,
+    trace_id: str,
+    findings: list[dict] | None,
+    environment: str,
+    cost_usd: float | None = None,
+    dropped_count: float | None = None,
+    timestamp: str | None = None,
+    comment: str = "",
+) -> list[dict]:
+    """The `score-create` ingestion events for one review.
+
+    `dropped_count` must be measured at parse time, not here: by the time
+    `findings` reaches this function the per-repo config has already filtered it
+    by severity threshold and `max_findings`, and those drops are the config
+    working as intended, not the model emitting garbage.
+
+    Returns [] rather than raising if something is unscoreable — scores are
+    telemetry and must never cost a review.
+    """
+    # The ingestion envelope requires a timestamp on every event; omitting it
+    # gets the whole batch rejected with an HTTP 207 whose per-event 400s are
+    # easy to mistake for success.
+    ts = timestamp or datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
+    out: list[dict] = []
+
+    def add(name: str, value, data_type: str) -> None:
+        if value is None:
+            return
+        body = {
+            "id": str(uuid.uuid4()),
+            "traceId": trace_id,
+            "name": name,
+            "dataType": data_type,
+            "environment": environment,
+        }
+        if data_type == "CATEGORICAL":
+            body["value"] = str(value)
+        else:
+            body["value"] = float(value)
+        if comment:
+            body["comment"] = comment
+        out.append(
+            {
+                "id": str(uuid.uuid4()),
+                "type": "score-create",
+                "timestamp": ts,
+                "body": body,
+            }
+        )
+
+    try:
+        add(FINDING_RATE, finding_rate(findings), "NUMERIC")
+        add(SEVERITY_INFO_RATIO, severity_info_ratio(findings), "NUMERIC")
+        add(SEVERITY_MAX, severity_max(findings), "CATEGORICAL")
+        add(DROPPED_FINDINGS, dropped_count, "NUMERIC")
+        add(COST_PER_FINDING, cost_per_finding(cost_usd, findings), "NUMERIC")
+    except Exception:  # pragma: no cover - defensive
+        return out
+    return out
+
+
+# ---------------------------------------------------------------------------
+# Score configs — the schema these scores must comply with
+# ---------------------------------------------------------------------------
+
+# Registered once per project via `eval_bootstrap.py`. Without configs the
+# scores still ingest, but nothing constrains a future scorer from writing
+# `severity_max="HIGH"` next to today's `"high"` and silently splitting the
+# series in two.
+SCORE_CONFIGS = [
+    {
+        "name": FINDING_RATE,
+        "dataType": "NUMERIC",
+        "minValue": 0,
+        "description": "Findings posted by one review. 0 = the reviewer stayed silent.",
+    },
+    {
+        "name": SEVERITY_INFO_RATIO,
+        "dataType": "NUMERIC",
+        "minValue": 0,
+        "maxValue": 1,
+        "description": "Share of a review's findings rated info/trivial. High = the model is not committing to a severity.",
+    },
+    {
+        "name": SEVERITY_MAX,
+        "dataType": "CATEGORICAL",
+        "categories": [
+            {"label": "none", "value": 0},
+            {"label": "info", "value": 1},
+            {"label": "trivial", "value": 2},
+            {"label": "low", "value": 3},
+            {"label": "medium", "value": 4},
+            {"label": "high", "value": 5},
+            {"label": "critical", "value": 6},
+        ],
+        "description": "Highest severity surfaced by one review; 'none' when it posted nothing.",
+    },
+    {
+        "name": DROPPED_FINDINGS,
+        "dataType": "NUMERIC",
+        "minValue": 0,
+        "description": "Findings the model emitted that the parser rejected for an unusable path/line.",
+    },
+    {
+        "name": COST_PER_FINDING,
+        "dataType": "NUMERIC",
+        "minValue": 0,
+        "description": "Equivalent USD per finding posted. Silent reviews divide by 1, not 0.",
+    },
+]

pilot/feedback/__init__.py

@@ -0,0 +1,2 @@
+"""Feedback persistence and human-signal processing."""
+from .store import *

pilot/feedback/analyze.py

@@ -0,0 +1,419 @@
+"""pragent pilot — daily feedback analyzer.
+
+Reads `feedback.db` (written by `feedback_harvest.py`) and produces a
+markdown report that:
+
+  1. Ranks inline findings by **net false-positive score** (downvotes +
+     unresolved + negation-phrase replies − upvotes − resolved). Top of
+     this list = "the bot has been wrong about this repeatedly". These
+     are the candidates that *might* belong in the per-repo
+     `.pr-review.json:instructions` addendum.
+  2. Ranks findings by **net acceptance** — repeated 👍 / resolution =
+     "the bot's framing here is genuinely useful". These can be promoted
+     to the shared `architecture.md` so they don't have to be re-derived
+     every PR.
+  3. Reports a **restraint metric** — for every PR where the bot posted
+     zero findings, count how often a human reviewer also posted zero
+     substantive review comments. When the bot is loud on clean code,
+     that's a false-positive rate we can act on (DoorDash lesson:
+     "excessive noise on clean code is its own failure mode").
+  4. Reports a **case-review queue** — every disagreement case (a
+     downvote, unresolved, or a reply matching `FALSE_POSITIVE_PHRASES`)
+     is listed in full so a human can re-read the original PR and decide
+     if the finding was right or wrong.
+
+Output is plain markdown so it can be posted as a Gitea issue / comment
+without rendering work. Designed to be reviewed by a human, not auto-
+applied — per the DoorDash pattern, every material change to model /
+prompt / context goes through a benchmark gate first; this report IS
+that gate (or, more precisely, the queue feeding the gate).
+
+Never raises. A bad DB / no data → returns a friendly empty-state report.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import logging
+import os
+import sqlite3
+from collections import defaultdict
+from datetime import datetime, timezone
+from typing import Optional
+
+import feedback
+from feedback_harvest import (
+    FALSE_POSITIVE_PHRASES,
+    classify_reaction,
+    _is_negation_reply,  # noqa: F401  (re-exported for the test suite)
+)
+
+log = logging.getLogger("pragent.feedback.analyze")
+
+# How many findings to surface in each top-list. Capped because the
+# reports are read by humans; more than 20 per list and they skim.
+TOP_N = 20
+
+# Restraint threshold — fraction of "clean" PRs (zero findings) where
+# the bot produced ANY findings. Above this we recommend `.pr-review.json:
+# exclude_patterns` or a stricter `severity_threshold`.
+RESTRAINT_NOISE_THRESHOLD = 0.25
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+def _net_score(row) -> tuple[int, int]:
+    """Return (false_positive_score, acceptance_score) for one finding row.
+
+    FP signals: downvotes (+1), unresolved (+1), negation-phrase replies (+2).
+    Acceptance signals: upvotes (+1), resolved (+1).
+    """
+    fp = 0
+    ac = 0
+    fp += int(row["downvotes"] or 0)
+    fp += 1 if row["resolved"] == 0 else 0  # 0/1/NULL; 0 = unresolved
+    ac += 1 if row["resolved"] == 1 else 0
+    ac += int(row["upvotes"] or 0)
+    if row["reply_bodies"] and _is_negation_reply(row["reply_bodies"]):
+        fp += 2
+    return fp, ac
+
+
+def _short_problem(problem: str, n: int = 100) -> str:
+    s = (problem or "").strip().replace("\n", " ")
+    return s if len(s) <= n else s[: n - 1] + "…"
+
+
+def _restraint_stats(conn: sqlite3.Connection) -> dict:
+    """How often does the bot post findings on PRs that received zero
+    bot findings (= presumably clean)? Looks at `review.findings_total`
+    if present, otherwise counts `inline_finding` per PR.
+
+    NOTE: until `post_inline_review` records `findings_total`, this falls
+    back to "PRs with at least one finding row" which is an underestimate
+    (a bot review with zero findings leaves no row).
+    """
+    total_prs_with_review = conn.execute(
+        "SELECT COUNT(DISTINCT repo || '#' || pr) FROM review"
+    ).fetchone()[0]
+    prs_with_findings = conn.execute(
+        "SELECT COUNT(DISTINCT repo || '#' || pr) FROM inline_finding"
+    ).fetchone()[0]
+    if total_prs_with_review == 0:
+        return {"total": 0, "noisy": 0, "ratio": 0.0}
+    # This is currently "PRs where the bot left at least one inline
+    # comment". A precise "findings_total per review" needs
+    # post_inline_review to record it (TODO in the wiring step). Until
+    # then, treat this as a floor: real noise is >= this.
+    return {
+        "total": total_prs_with_review,
+        "noisy": prs_with_findings,
+        "ratio": prs_with_findings / total_prs_with_review,
+    }
+
+
+def _case_review_queue(conn: sqlite3.Connection, limit: int = 30) -> list[dict]:
+    """Findings that humans pushed back on — for manual re-review."""
+    rows = feedback.findings_with_votes(conn)
+    cases = []
+    for r in rows:
+        fp_score, _ = _net_score(r)
+        if fp_score <= 0:
+            continue
+        cases.append({
+            "posthash": r["posthash"],
+            "repo": r["repo"],
+            "pr": r["pr"],
+            "path": r["path"],
+            "line": r["line"],
+            "severity": r["severity"],
+            "problem": _short_problem(r["problem"], 200),
+            "fp_score": fp_score,
+            "upvotes": r["upvotes"] or 0,
+            "downvotes": r["downvotes"] or 0,
+            "resolved": r["resolved"],
+            "reply_count": r["reply_count"] or 0,
+            "reply_excerpt": _short_problem(r["reply_bodies"] or "", 200),
+        })
+    cases.sort(key=lambda c: c["fp_score"], reverse=True)
+    return cases[:limit]
+
+
+def _format_table(headers: list[str], rows: list[list[str]]) -> str:
+    if not rows:
+        return "_none yet_\n"
+    out = ["| " + " | ".join(headers) + " |",
+           "|" + "|".join(["---"] * len(headers)) + "|"]
+    for row in rows:
+        out.append("| " + " | ".join(row) + " |")
+    return "\n".join(out) + "\n"
+
+
+def _md_escape(s: str) -> str:
+    """Escape pipes + newlines so the value stays in one table cell."""
+    return (s or "").replace("|", "\\|").replace("\n", " ").strip()
+
+
+# ---------------------------------------------------------------------------
+# Main report builder
+# ---------------------------------------------------------------------------
+
+def analyze(db_path: str, *, since_ts: Optional[int] = None,
+            as_json: bool = False) -> str:
+    """Build the daily report. Returns a markdown string by default;
+    `as_json=True` returns a structured dict (for tests + automation)."""
+    conn = feedback.init(db_path)
+    try:
+        findings = list(feedback.findings_with_votes(conn, since_ts=since_ts))
+        total_findings = len(findings)
+        repo_set = {f["repo"] for f in findings}
+        case_queue = _case_review_queue(conn)
+        restraint = _restraint_stats(conn)
+
+        # Compute scores
+        scored: list[tuple[int, int, sqlite3.Row]] = []
+        for f in findings:
+            fp, ac = _net_score(f)
+            scored.append((fp, ac, f))
+
+        # Top false-positive patterns (sorted by fp score, deduped by posthash).
+        # `occurrences` comes from the inline_finding row — posthash UNIQUE
+        # means a single row can carry a count > 1 (set by record_inline_finding's
+        # ON CONFLICT DO UPDATE).
+        fp_by_hash: dict[str, dict] = {}
+        for fp, ac, f in scored:
+            if fp <= 0:
+                continue
+            ph = f["posthash"]
+            entry = fp_by_hash.setdefault(ph, {
+                "posthash": ph, "fp_score": 0, "ac_score": 0,
+                "repo": f["repo"], "path": f["path"], "line": f["line"],
+                "severity": f["severity"], "problem": f["problem"],
+                "occurrences": f["occurrences"], "upvs": 0, "downs": 0,
+                "resolved_true": 0, "resolved_false": 0,
+            })
+            entry["fp_score"] += fp
+            entry["ac_score"] += ac
+            entry["upvs"] += f["upvotes"] or 0
+            entry["downs"] += f["downvotes"] or 0
+            if f["resolved"] == 1:
+                entry["resolved_true"] += 1
+            elif f["resolved"] == 0:
+                entry["resolved_false"] += 1
+        fp_sorted = sorted(
+            fp_by_hash.values(), key=lambda e: e["fp_score"], reverse=True,
+        )[:TOP_N]
+
+        # Top accepted patterns
+        ac_by_hash: dict[str, dict] = {}
+        for fp, ac, f in scored:
+            if ac <= 0:
+                continue
+            ph = f["posthash"]
+            entry = ac_by_hash.setdefault(ph, {
+                "posthash": ph, "ac_score": 0, "fp_score": 0,
+                "repo": f["repo"], "path": f["path"], "line": f["line"],
+                "severity": f["severity"], "problem": f["problem"],
+                "occurrences": f["occurrences"], "upvs": 0, "downs": 0,
+                "resolved_true": 0,
+            })
+            entry["ac_score"] += ac
+            entry["fp_score"] += fp
+            entry["upvs"] += f["upvotes"] or 0
+            entry["downs"] += f["downvotes"] or 0
+            if f["resolved"] == 1:
+                entry["resolved_true"] += 1
+        ac_sorted = sorted(
+            ac_by_hash.values(), key=lambda e: e["ac_score"], reverse=True,
+        )[:TOP_N]
+
+        # Restraint recommendation
+        if restraint["ratio"] > RESTRAINT_NOISE_THRESHOLD:
+            restraint_msg = (
+                f"⚠️ Bot posted findings on **{restraint['ratio']:.0%}** of "
+                f"reviewed PRs ({restraint['noisy']} / {restraint['total']}). "
+                f"Above the {RESTRAINT_NOISE_THRESHOLD:.0%} threshold — "
+                "consider raising `.pr-review.json:severity_threshold` to "
+                "`medium` or `high` for noisy repos, or adding "
+                "`patterns.deny` to skip stylistic-only findings."
+            )
+        else:
+            restraint_msg = (
+                f"✅ Bot stayed quiet on **{1 - restraint['ratio']:.0%}** of "
+                f"reviewed PRs ({restraint['total'] - restraint['noisy']} / "
+                f"{restraint['total']}). Restraint OK."
+            )
+
+        if as_json:
+            return json.dumps({
+                "total_findings": total_findings,
+                "repos_seen": sorted(repo_set),
+                "restraint": restraint,
+                "top_false_positive": fp_sorted,
+                "top_accepted": ac_sorted,
+                "case_review_queue": case_queue,
+                "restraint_msg": restraint_msg,
+            }, indent=2)
+
+        # Markdown
+        ts_str = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
+        out = [f"# pragent feedback report — {ts_str}", ""]
+        out.append(f"- **findings analyzed**: {total_findings}")
+        out.append(f"- **repos with feedback**: {len(repo_set)} "
+                   f"({', '.join(sorted(repo_set))})")
+        out.append(f"- **case-review queue**: {len(case_queue)} disagreement(s)")
+        out.append("")
+        out.append("## Restraint")
+        out.append("")
+        out.append(restraint_msg)
+        out.append("")
+        out.append("> DoorDash rule (2026-07-06): *excessive noise on clean "
+                   "code is its own failure mode*. `severity_threshold` + "
+                   "`patterns.deny` are the knobs that dial restraint.")
+        out.append("")
+
+        out.append(f"## Top {len(fp_sorted)} false-positive candidates")
+        out.append("")
+        out.append("Aggregated by `posthash` (path:line:severity:problem). "
+                   "Sort key = downvotes + unresolved + negation-phrase replies "
+                   "− upvotes − resolved.")
+        out.append("")
+        rows = []
+        for e in fp_sorted:
+            rows.append([
+                str(e["fp_score"]),
+                f"`{_md_escape(e['repo'])}`",
+                f"`{_md_escape(e['path'])}:{e['line']}`",
+                e["severity"],
+                _md_escape(_short_problem(e["problem"])),
+                f"👍{e['upvs']} 👎{e['downs']}",
+                f"✅{e['resolved_true']} ❌{e['resolved_false']}",
+                str(e["occurrences"]),
+            ])
+        out.append(_format_table(
+            ["FP", "repo", "path:line", "sev", "problem",
+             "votes", "resolved", "seen"],
+            rows,
+        ))
+        out.append("")
+        out.append("_Review each row before adding it to "
+                   "`.pr-review.json:instructions`. Human reactions are NOT "
+                   "ground truth (DoorDash, 2026-07-06: authors accept/reject "
+                   "for workflow reasons) — re-read the PR before acting._")
+        out.append("")
+
+        out.append(f"## Top {len(ac_sorted)} accepted patterns")
+        out.append("")
+        out.append("Aggregated by posthash. Sort key = upvotes + resolved − "
+                   "downvotes − unresolved − negation-phrase replies.")
+        out.append("")
+        rows = []
+        for e in ac_sorted:
+            rows.append([
+                str(e["ac_score"]),
+                f"`{_md_escape(e['repo'])}`",
+                f"`{_md_escape(e['path'])}:{e['line']}`",
+                e["severity"],
+                _md_escape(_short_problem(e["problem"])),
+                f"👍{e['upvs']} 👎{e['downs']}",
+                f"✅{e['resolved_true']}",
+                str(e["occurrences"]),
+            ])
+        out.append(_format_table(
+            ["AC", "repo", "path:line", "sev", "problem",
+             "votes", "resolved", "seen"],
+            rows,
+        ))
+        out.append("")
+        out.append("_Promote widely-accepted patterns into the shared "
+                   "`architecture.md` on Nexus raw-hosted (or the per-repo "
+                   "`additional_context_urls`). These become part of the "
+                   "prompt-cached prefix → ~0 marginal cost on step 2+._")
+        out.append("")
+
+        out.append(f"## Case-review queue ({len(case_queue)})")
+        out.append("")
+        if not case_queue:
+            out.append("_No disagreements recorded yet. Once humans start "
+                       "reacting 👎 / leaving replies / not resolving bot "
+                       "comments, cases will appear here._")
+        else:
+            out.append("Each row needs a human to re-read the original PR and "
+                       "decide: was the bot right? If not, draft an "
+                       "`instructions` addendum or a `patterns.deny` rule.")
+            out.append("")
+            for c in case_queue:
+                url = (
+                    f"https://gitea.marcospaulo.dev.br/{c['repo']}/pulls/"
+                    f"{c['pr']}/files#r{c['posthash']}"
+                )
+                out.append(f"### FP={c['fp_score']} · {c['repo']}#{c['pr']}")
+                out.append(
+                    f"- file: `{_md_escape(c['path'])}:{c['line']}` · "
+                    f"severity: `{c['severity']}`",
+                )
+                out.append(f"- problem: {_md_escape(c['problem'])}")
+                out.append(
+                    f"- signals: 👍{c['upvotes']} 👎{c['downvotes']} · "
+                    f"resolved={c['resolved']} · replies={c['reply_count']}",
+                )
+                if c["reply_excerpt"]:
+                    out.append(
+                        f"- last reply: {_md_escape(c['reply_excerpt'])}",
+                    )
+                out.append(f"- posthash: `{c['posthash']}`")
+                out.append("")
+
+        out.append("## Where this report goes")
+        out.append("")
+        out.append("- **Per-repo actions** (`.pr-review.json:instructions`, "
+                   "`patterns.deny`, `severity_threshold`): edit the file on "
+                   "`main` via a regular PR. The next PR review picks up the "
+                   "change automatically.")
+        out.append("- **Cross-repo actions** (shared house-rules): update the "
+                   "`PRAGENT_ADDITIONAL_CONTEXT_URL` document on Nexus "
+                   "raw-hosted (`canalhandia/architecture.md` etc).")
+        out.append("- **Benchmark gate** (DoorDash pattern): before changing "
+                   "the model / prompt / context window, replay this report "
+                   "against the labeled `posthash` corpus. If a candidate "
+                   "addendum flips ≥ 1 currently-accepted finding into "
+                   "false-positive, drop it.")
+        out.append("")
+        out.append(f"_Generated from `{db_path}` by `feedback_analyze.py`._")
+        return "\n".join(out)
+    finally:
+        conn.close()
+
+
+# ---------------------------------------------------------------------------
+# CLI
+# ---------------------------------------------------------------------------
+
+def main() -> int:
+    p = argparse.ArgumentParser(description="Build the daily feedback report.")
+    p.add_argument("--db", default=os.environ.get(
+        "PRAGENT_FEEDBACK_DB", "/data/feedback.db",
+    ))
+    p.add_argument("--since", type=int, default=None,
+                   help="Unix timestamp; only include findings posted since")
+    p.add_argument("--json", action="store_true",
+                   help="Emit structured JSON instead of markdown")
+    p.add_argument("--out", default="-",
+                   help="Write to this path instead of stdout ('-' = stdout)")
+    args = p.parse_args()
+
+    out = analyze(args.db, since_ts=args.since, as_json=args.json)
+    if args.out == "-":
+        print(out)
+    else:
+        with open(args.out, "w") as f:
+            f.write(out)
+        print(f"wrote {args.out}", file=sys.stderr)
+    return 0
+
+
+if __name__ == "__main__":
+    import sys
+    raise SystemExit(main())

pilot/feedback/harvest.py

@@ -0,0 +1,394 @@
+"""pragent pilot — feedback harvester.
+
+For each PR the webhook server is about to review, walk back through the
+Gitea-side state of every bot comment from every prior review on that PR
+and record:
+  - reactions on the review body + on each inline comment
+  - thread-resolved state (Gitea's `resolver` field; non-empty = resolved)
+  - replies (issue-comments with `review_comment_id` matching ours)
+  - the bot's own findings_count + inline_count per review (for the
+    restraint metric)
+
+Everything is best-effort. A single 404 or 5xx is logged and skipped — we
+must never abort a review because the feedback DB had a hiccup.
+
+The harvester is intentionally separate from `review_pr` so it can be
+called independently (e.g. by the daily analyzer's "backfill" mode) and
+tested in isolation against a mocked Gitea client.
+"""
+from __future__ import annotations
+
+import json
+import logging
+import re
+import time
+import urllib.parse
+import urllib.request
+from typing import Optional
+
+import ai_review  # used as ai_review.gitea_get(...) so test mocks land on the binding
+
+from feedback import (
+    init,
+    record_inline_finding,
+    record_reaction,
+    record_reply,
+    record_review,
+    record_thread_state,
+    posthash,
+)
+
+log = logging.getLogger("pragent.feedback.harvest")
+
+# Reviewer identity — only collect feedback on comments authored by us.
+# Avoids harvesting reactions on human comments (which we never want to
+# count toward "bot usefulness").
+REVIEWER_LOGIN = "pragent-bot"
+
+# Reactions content tokens Gitea uses. We track +1 / -1 explicitly; the
+# others are stored as-is so the analyzer can mine them (👀 eyes,
+# laugh, hooray, confused, heart, rocket, …) without hardcoding a list
+# that drifts across Gitea versions.
+POSITIVE_REACTIONS = {"+1", "heart", "hooray", "laugh", "rocket"}
+NEGATIVE_REACTIONS = {"-1", "confused"}
+# Note: Gitea's `eyes` reaction (👀) means "I'm watching" — not approval
+# or disapproval. Treated as neutral by the analyzer.
+
+# Phrases that, in a reply, indicate the author thinks the bot's finding
+# was wrong. Casing + punctuation ignored; substring match is good enough
+# (false positives in the analyzer cost a human minute; false negatives
+# hide regressions).
+FALSE_POSITIVE_PHRASES = (
+    "false positive", "not actually", "this is fine", "this is intentional",
+    "not a bug", "intentional", "wrong here", "isn't actually",
+    "is not actually", "don't think this is", "i disagree", "this isn't right",
+    "this is correct", "this is expected", "by design", "this is by design",
+)
+
+# Gitea review-comment payload includes a 'body' field that may carry our
+# sha marker + severity header. We extract severity + path/line from it
+# as a fallback when the finding wasn't already seeded at post-time (old
+# reviews before feedback.py existed).
+SEV_RE = re.compile(r"\*\*\[(CRITICAL|HIGH|MEDIUM|LOW|INFO)\]\*\*", re.IGNORECASE)
+PATH_LINE_RE = re.compile(r"`([^?:\n]+?):(\d+)`")
+SHA_MARKER_RE = re.compile(r"<!--\s*pragent:sha=([0-9a-f]+)\s*-->", re.IGNORECASE)
+
+
+# ---------------------------------------------------------------------------
+# Low-level HTTP — tolerant JSON parse (Gitea sometimes returns `null` where
+# we expect `[]`, e.g. reactions on a fresh comment)
+# ---------------------------------------------------------------------------
+
+def _gitea_get_json(api: str, repo: str, path: str, token: str) -> tuple[int, object]:
+    status, raw = ai_review.gitea_get(api, repo, path, token)
+    if status != 200:
+        return status, None
+    try:
+        return status, json.loads(raw.decode("utf-8", errors="replace"))
+    except (json.JSONDecodeError, ValueError):
+        return status, None
+
+
+# ---------------------------------------------------------------------------
+# Parse helpers
+# ---------------------------------------------------------------------------
+
+def _parse_severity(body: str) -> str:
+    m = SEV_RE.search(body or "")
+    return m.group(1).upper() if m else "INFO"
+
+
+def _parse_path_line(body: str) -> tuple[Optional[str], Optional[int]]:
+    m = PATH_LINE_RE.search(body or "")
+    if not m:
+        return None, None
+    path = m.group(1).strip()
+    try:
+        return path, int(m.group(2))
+    except ValueError:
+        return path, None
+
+
+def _parse_sha(body: str) -> Optional[str]:
+    m = SHA_MARKER_RE.search(body or "")
+    return m.group(1) if m else None
+
+
+def _is_negation_reply(body: str) -> bool:
+    if not body:
+        return False
+    norm = body.lower()
+    return any(p in norm for p in FALSE_POSITIVE_PHRASES)
+
+
+# ---------------------------------------------------------------------------
+# Reaction classification (cheap, used by the analyzer — not the harvester
+# itself)
+# ---------------------------------------------------------------------------
+
+def classify_reaction(content: str) -> str:
+    """Bucket a reaction into 'positive', 'negative', or 'neutral'."""
+    c = (content or "").strip().lower()
+    if c in POSITIVE_REACTIONS:
+        return "positive"
+    if c in NEGATIVE_REACTIONS:
+        return "negative"
+    return "neutral"
+
+
+# ---------------------------------------------------------------------------
+# Main harvest entry
+# ---------------------------------------------------------------------------
+
+def harvest_for_pr(
+    *,
+    api: str,
+    token: str,
+    repo: str,
+    pr_index: int,
+    db_path: str,
+    page_size: int = 50,
+) -> dict:
+    """Walk every bot-authored review on the given PR and record reactions
+    + thread state + replies. Returns a stats dict for logging.
+
+    `db_path` is the SQLite file path (env: `PRAGENT_FEEDBACK_DB`,
+    typically `/data/feedback.db` mounted via the `feedback-data` PVC).
+    """
+    conn = init(db_path)
+    stats = {
+        "reviews_seen": 0, "findings_seen": 0,
+        "reactions_recorded": 0, "thread_states_recorded": 0,
+        "replies_recorded": 0, "errors": 0,
+    }
+
+    try:
+        # 1. List every review on the PR (paginated, but PRs rarely have >page_size)
+        status, payload = _gitea_get_json(
+            api, repo, f"pulls/{pr_index}/reviews?per_page={page_size}", token,
+        )
+        if status != 200 or not isinstance(payload, list):
+            log.info("harvest: reviews list failed status=%d", status)
+            stats["errors"] += 1
+            return stats
+
+        for rev in payload:
+            user = (rev.get("user") or {}).get("login", "")
+            if user != REVIEWER_LOGIN:
+                continue
+            stats["reviews_seen"] += 1
+
+            review_id_gitea = rev.get("id")
+            head_sha = rev.get("commit_id", "")
+            review_body = rev.get("body", "") or ""
+            body_sha = _parse_sha(review_body)
+            # Trust the sha marker inside the body — Gitea's commit_id field is
+            # for the LAST commit, not necessarily the reviewed head. If we
+            # can't find a marker, fall back to commit_id.
+            effective_sha = body_sha or head_sha
+            created_at = _parse_iso_ts(rev.get("created_at", ""))
+
+            db_review_id = record_review(
+                conn, repo=repo, pr=pr_index, head_sha=effective_sha,
+                review_id_gitea=review_id_gitea,
+                posted_at=created_at,
+            )
+
+            # 2. Inline comments for this review
+            if review_id_gitea is None:
+                continue
+            rstatus, rpayload = _gitea_get_json(
+                api, repo, f"pulls/{pr_index}/reviews/{review_id_gitea}/comments",
+                token,
+            )
+            if rstatus != 200 or not isinstance(rpayload, list):
+                stats["errors"] += 1
+                continue
+
+            for ic in rpayload:
+                ic_id = ic.get("id")
+                if ic_id is None:
+                    continue
+                ic_body = ic.get("body", "") or ""
+                ic_path = ic.get("path")
+                ic_line = ic.get("position") or ic.get("line")
+                ic_severity = _parse_severity(ic_body)
+                # Fall back to body parse when Gitea didn't echo path/line
+                if not ic_path or not ic_line:
+                    bp, bl = _parse_path_line(ic_body)
+                    ic_path = ic_path or bp
+                    ic_line = ic_line or bl
+
+                if not ic_path or not ic_line:
+                    log.info(
+                        "harvest: inline %s missing path/line, skipping", ic_id,
+                    )
+                    continue
+
+                finding_id = record_inline_finding(
+                    conn, review_id=db_review_id, repo=repo, pr=pr_index,
+                    path=ic_path, line=ic_line, severity=ic_severity,
+                    problem=_strip_severity_header(ic_body),
+                    fix="", suggestion="",
+                    comment_id=ic_id,
+                    posted_at=created_at,
+                )
+                stats["findings_seen"] += 1
+                if finding_id is None:
+                    continue
+
+                # 3. Reactions on the inline comment
+                react_status, react_payload = _gitea_get_json(
+                    api, repo, f"issues/comments/{ic_id}/reactions", token,
+                )
+                if react_status == 200 and isinstance(react_payload, list):
+                    for r in react_payload:
+                        ruser = (r.get("user") or {}).get("login", "") or "?"
+                        # Gitea has occasionally returned `content` as a
+                        # dict on older versions; coerce to str defensively.
+                        rcontent = str(r.get("content") or "").strip()
+                        if not rcontent:
+                            continue
+                        if record_reaction(
+                            conn, comment_id=ic_id, user=ruser,
+                            content=rcontent,
+                            created_at=_parse_iso_ts(r.get("created_at", "")),
+                        ):
+                            stats["reactions_recorded"] += 1
+
+                # 4. Thread state (Gitea's `resolver` field on the inline
+                #    comment). Some Gitea versions serialize this as a user
+                #    object ({login, ...}) instead of a username string —
+                #    coerce defensively before calling .strip().
+                resolver_raw = ic.get("resolver")
+                if isinstance(resolver_raw, dict):
+                    resolver = (resolver_raw.get("login") or "").strip()
+                else:
+                    resolver = str(resolver_raw or "").strip()
+                if resolver_raw is not None:  # field present, even if ""
+                    record_thread_state(
+                        conn, finding_id=finding_id,
+                        resolved=bool(resolver),
+                    )
+                    stats["thread_states_recorded"] += 1
+
+            # 5. Replies on this review (issue-comments whose
+            #    `review_comment_id` points at one of our inline comments).
+            #    Some Gitea versions don't expose `review_comment_id` on the
+            #    issue-comment endpoint — in that case `replies` stays
+            #    empty; we degrade gracefully.
+            try:
+                _harvest_replies(
+                    api=api, repo=repo, token=token,
+                    pr_index=pr_index, review_id=review_id_gitea,
+                    inline_comments=rpayload, conn=conn,
+                    stats=stats,
+                )
+            except Exception as e:
+                log.info("harvest: replies fetch failed: %s", e)
+                stats["errors"] += 1
+    finally:
+        conn.close()
+
+    return stats
+
+
+def _harvest_replies(
+    *, api: str, repo: str, token: str, pr_index: int,
+    review_id: int, inline_comments: list, conn, stats: dict,
+) -> None:
+    """Fetch issue comments on this PR; record those whose
+    `review_comment_id` matches one of our inline comment IDs.
+    Gitea 1.26 doesn't include that field — we fall back to fetching each
+    inline comment individually via `issues/comments/{id}` (does include
+    the field) only if the bulk fetch is empty.
+    """
+    inline_ids = {c.get("id") for c in inline_comments if c.get("id") is not None}
+    if not inline_ids:
+        return
+
+    status, payload = _gitea_get_json(
+        api, repo, f"issues/{pr_index}/comments?per_page=100", token,
+    )
+    if status != 200 or not isinstance(payload, list):
+        return
+
+    # Build mapping inline_id -> finding_id (one SELECT instead of N)
+    rows = conn.execute(
+        "SELECT comment_id, id FROM inline_finding WHERE comment_id IN ("
+        + ",".join("?" * len(inline_ids)) + ")",
+        list(inline_ids),
+    ).fetchall()
+    inline_to_finding = {r[0]: r[1] for r in rows}
+
+    for c in payload:
+        rcid = c.get("review_comment_id")
+        if not rcid or rcid not in inline_to_finding:
+            continue
+        author = (c.get("user") or {}).get("login", "") or "?"
+        body = c.get("body", "") or ""
+        ts = _parse_iso_ts(c.get("created_at", ""))
+        if record_reply(
+            conn, finding_id=inline_to_finding[rcid],
+            author=author, body=body, created_at=ts,
+        ):
+            stats["replies_recorded"] += 1
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+def _strip_severity_header(body: str) -> str:
+    """Drop the leading `**[SEVERITY]**` so the posthash captures the
+    substance, not the severity label."""
+    return SEV_RE.sub("", body or "", count=1).strip()
+
+
+def _parse_iso_ts(s: str) -> int:
+    if not s:
+        return int(time.time())
+    try:
+        # Python 3.11+ fromisoformat tolerates the trailing 'Z'.
+        return int(__import__("datetime").datetime.fromisoformat(
+            s.replace("Z", "+00:00")
+        ).timestamp())
+    except Exception:
+        return int(time.time())
+
+
+# ---------------------------------------------------------------------------
+# CLI for manual backfill / first-time seed
+# ---------------------------------------------------------------------------
+
+def main() -> int:
+    import argparse, os
+    p = argparse.ArgumentParser(
+        description="Harvest reactions/threads/replies on bot PR comments.",
+    )
+    p.add_argument("--api", default=os.environ.get(
+        "GITEA_API", "http://gitea-http.gitea.svc.cluster.local:3000",
+    ))
+    p.add_argument("--token", default=os.environ.get("PRAGENT_BOT_TOKEN", ""))
+    p.add_argument("--repo", required=True, help="owner/name")
+    p.add_argument("--pr", type=int, required=True, help="PR index")
+    p.add_argument("--db", default=os.environ.get(
+        "PRAGENT_FEEDBACK_DB", "/data/feedback.db",
+    ))
+    args = p.parse_args()
+
+    if not args.token:
+        print("PRAGENT_BOT_TOKEN required", flush=True)
+        return 2
+
+    logging.basicConfig(level=logging.INFO)
+    stats = harvest_for_pr(
+        api=args.api, token=args.token,
+        repo=args.repo, pr_index=args.pr, db_path=args.db,
+    )
+    print(json.dumps(stats), flush=True)
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

pilot/feedback/post.py

@@ -0,0 +1,128 @@
+"""pragent pilot — daily feedback report delivery.
+
+Calls `feedback_analyze.analyze()` and posts the markdown report as a
+comment on a single long-lived "feedback roll-up" issue in
+`gitea_admin/pragent`. Comments are append-only history — one comment per
+run, timestamped in the body. This keeps every report in one place, easy
+to scroll, and avoids the issue-explosion of "one issue per day".
+
+If the issue doesn't exist yet, create it. Subsequent runs just add a
+new comment.
+
+Designed for the daily K8s CronJob (`k8s/pragent-feedback-cronjob.yaml`)
+but runnable from CLI for ad-hoc checks.
+
+Env:
+  GITEA_API            in-cluster Gitea base URL
+  PRAGENT_BOT_TOKEN    bot token (Write collaborator on gitea_admin/pragent)
+  PRAGENT_FEEDBACK_DB  path to SQLite (default /data/feedback.db)
+  PRAGENT_FEEDBACK_ISSUE_REPO   default gitea_admin/pragent
+  PRAGENT_FEEDBACK_ISSUE_TITLE  default "pragent feedback roll-up"
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import logging
+import os
+import sys
+
+import ai_review
+
+from feedback_analyze import analyze
+
+log = logging.getLogger("pragent.feedback.post")
+
+
+REPO_DEFAULT = "gitea_admin/pragent"
+TITLE_DEFAULT = "pragent feedback roll-up"
+
+
+def _find_or_create_issue(api: str, token: str, repo: str, title: str) -> int:
+    """Locate the open issue with this title; create one if missing.
+
+    Gitea's issue search is via `GET /repos/{o}/{r}/issues?state=open&q=...`
+    (q matches title + body). We filter client-side for the exact title
+    to avoid query-text false matches.
+    """
+    status, raw = ai_review.gitea_get(api, repo, "issues?state=open&per_page=50", token)
+    if status == 200:
+        try:
+            for issue in json.loads(raw):
+                if issue.get("title") == title:
+                    # NB: the comment URL needs the per-repo `number`, not the
+                    # global `id`. `id=60 num=8` for an early-N create; we want
+                    # `num=8` for `/repos/o/r/issues/8/comments`.
+                    return int(issue["number"])
+        except (json.JSONDecodeError, ValueError, KeyError):
+            pass
+    # Create
+    status, raw = ai_review.gitea_post(
+        api, repo, "issues", token,
+        {"title": title, "body": "pragent feedback roll-up — auto-created."},
+    )
+    if status not in (200, 201):
+        raise RuntimeError(f"issue create failed: HTTP {status} body={raw[:200]!r}")
+    return int(json.loads(raw)["number"])
+
+
+def _post_comment(api: str, token: str, repo: str, issue_number: int, body: str) -> int:
+    status, raw = ai_review.gitea_post(
+        api, repo, f"issues/{issue_number}/comments", token, {"body": body},
+    )
+    if status not in (200, 201):
+        raise RuntimeError(f"comment post failed: HTTP {status} body={raw[:200]!r}")
+    return json.loads(raw)["id"]
+
+
+def deliver(
+    *, api: str, token: str, db_path: str,
+    repo: str = REPO_DEFAULT, title: str = TITLE_DEFAULT,
+    since_ts: int | None = None,
+) -> dict:
+    """Build the report and post it as a comment. Returns a stats dict."""
+    report = analyze(db_path, since_ts=since_ts)
+    issue_id = _find_or_create_issue(api, token, repo, title)
+    comment_id = _post_comment(api, token, repo, issue_id, report)
+    return {
+        "repo": repo, "issue_id": issue_id, "comment_id": comment_id,
+        "report_bytes": len(report.encode()),
+    }
+
+
+def main() -> int:
+    p = argparse.ArgumentParser(
+        description="Post the daily feedback report to Gitea.",
+    )
+    p.add_argument("--api", default=os.environ.get(
+        "GITEA_API", "http://gitea-http.gitea.svc.cluster.local:3000",
+    ))
+    p.add_argument("--token", default=os.environ.get("PRAGENT_BOT_TOKEN", ""))
+    p.add_argument("--db", default=os.environ.get(
+        "PRAGENT_FEEDBACK_DB", "/data/feedback.db",
+    ))
+    p.add_argument("--repo", default=os.environ.get(
+        "PRAGENT_FEEDBACK_ISSUE_REPO", REPO_DEFAULT,
+    ))
+    p.add_argument("--title", default=os.environ.get(
+        "PRAGENT_FEEDBACK_ISSUE_TITLE", TITLE_DEFAULT,
+    ))
+    p.add_argument("--since", type=int, default=None,
+                   help="Unix timestamp; only include findings posted since")
+    args = p.parse_args()
+
+    if not args.token:
+        print("PRAGENT_BOT_TOKEN required", flush=True)
+        return 2
+
+    logging.basicConfig(level=logging.INFO)
+    stats = deliver(
+        api=args.api, token=args.token, db_path=args.db,
+        repo=args.repo, title=args.title, since_ts=args.since,
+    )
+    print(json.dumps(stats), flush=True)
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

pilot/feedback/scores.py

@@ -0,0 +1,247 @@
+#!/usr/bin/env python3
+"""pragent pilot — feedback DB to Langfuse scores.
+
+`feedback.db` already records every reaction, thread resolution and reply a
+maintainer leaves on a bot comment. That is the only ground truth pragent has
+about whether a finding was any good, and until now it went to a markdown report
+nobody reads and nowhere else. This ships it to Langfuse as session-level
+scores, so "was the reviewer right" sits on the same axis as "what did it cost".
+
+Session, not trace
+------------------
+`langfuse_trace` sets `sessionId` to `"{repo}#{pr}"` and lets the trace id be a
+fresh uuid per review. Feedback arrives days later against a PR, not against one
+particular re-run of the reviewer, and nothing in `feedback.db` records which
+trace produced which comment. Scoring the session is therefore both the
+available join and the honest granularity: this is feedback on the review of
+this PR, not on one invocation.
+
+Two scores, deliberately separated
+----------------------------------
+  * `review_engagement` — the share of a PR's findings that got any human
+    response at all. This is a signal about the *feedback loop*, not the
+    reviewer: at the time of writing it is 0.0 across all 113 recorded reviews,
+    which is exactly the fact that makes an accuracy metric impossible today.
+    It must be watched first, because every other quality number is vapour
+    until it moves.
+  * `review_acceptance` — net verdict over the findings that *did* get a
+    response: (upvotes + resolved) - (downvotes + negation replies), normalised
+    to -1..1. Computed only over engaged findings, so an ignored review scores
+    `None` rather than 0. Zero would read as "humans judged this exactly
+    neutral"; the truth is nobody looked.
+
+Fail-open and idempotent. Score ids are derived from (repo, pr, name) so a
+re-run overwrites rather than duplicates.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import sqlite3
+import sys
+import uuid
+from datetime import datetime, timezone
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+from feedback_harvest import classify_reaction, _is_negation_reply  # noqa: E402
+
+REVIEW_ENGAGEMENT = "review_engagement"
+REVIEW_ACCEPTANCE = "review_acceptance"
+
+# Stable namespace so the same (repo, pr, score) always produces the same score
+# id — Langfuse treats a repeated id as an update, which is what a backfill of a
+# still-accumulating PR should do.
+_NS = uuid.UUID("6f1d9c2e-4a77-4f2a-9c1a-0d3b5e8a7c41")
+
+
+def _score_id(repo: str, pr: int, name: str) -> str:
+    return str(uuid.uuid5(_NS, f"{repo}#{pr}#{name}"))
+
+
+def collect_pr_feedback(conn: sqlite3.Connection, repo: str, pr: int) -> dict:
+    """Tally one PR's findings and the human responses attached to them.
+
+    Returns counts only — the scoring maths lives in `score_pr` so it can be
+    tested without a database.
+    """
+    rows = conn.execute(
+        "SELECT id, comment_id FROM inline_finding WHERE repo = ? AND pr = ?",
+        (repo, pr),
+    ).fetchall()
+    total = len(rows)
+    engaged = 0
+    positive = 0
+    negative = 0
+
+    for row in rows:
+        fid = row["id"] if isinstance(row, sqlite3.Row) else row[0]
+        cid = row["comment_id"] if isinstance(row, sqlite3.Row) else row[1]
+        pos = neg = 0
+
+        if cid is not None:
+            for r in conn.execute(
+                "SELECT content FROM reaction WHERE comment_id = ?", (cid,)
+            ):
+                kind = classify_reaction(r[0])
+                if kind == "positive":
+                    pos += 1
+                elif kind == "negative":
+                    neg += 1
+
+        for r in conn.execute(
+            "SELECT resolved FROM thread_state WHERE finding_id = ?", (fid,)
+        ):
+            # A resolved thread means the maintainer acted on the finding.
+            if r[0]:
+                pos += 1
+
+        # A reply counts as engagement either way; only a negation phrase makes
+        # it a vote against. A neutral reply ("done", "good catch, but…") is
+        # deliberately not a positive vote — it says someone looked, not that
+        # they agreed.
+        replied = 0
+        for r in conn.execute(
+            "SELECT body FROM reply WHERE finding_id = ?", (fid,)
+        ):
+            replied += 1
+            if _is_negation_reply(r[0]):
+                neg += 1
+
+        if pos or neg or replied:
+            engaged += 1
+        positive += pos
+        negative += neg
+
+    return {"total": total, "engaged": engaged, "positive": positive, "negative": negative}
+
+
+def score_pr(tally: dict) -> dict:
+    """Turn one PR's tally into score values.
+
+    `review_acceptance` is `None` when nothing was engaged — see the module
+    docstring on why that is not 0.
+    """
+    total = int(tally.get("total") or 0)
+    engaged = int(tally.get("engaged") or 0)
+    pos = int(tally.get("positive") or 0)
+    neg = int(tally.get("negative") or 0)
+
+    engagement = round(engaged / total, 4) if total else None
+    acceptance = None
+    if pos or neg:
+        acceptance = round((pos - neg) / (pos + neg), 4)
+    return {REVIEW_ENGAGEMENT: engagement, REVIEW_ACCEPTANCE: acceptance}
+
+
+def build_score_events(
+    repo: str, pr: int, values: dict, environment: str = "default",
+    timestamp: str | None = None,
+) -> list[dict]:
+    """`score-create` events for one PR's feedback.
+
+    Every event carries a timestamp: the ingestion endpoint rejects those that
+    do not, and it reports the rejection as a per-event 400 inside an HTTP 207,
+    which reads as success to a caller that only checks the status code.
+    """
+    ts = timestamp or datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
+    events = []
+    for name, value in values.items():
+        if value is None:
+            continue
+        events.append(
+            {
+                "id": str(uuid.uuid4()),
+                "type": "score-create",
+                "timestamp": ts,
+                "body": {
+                    "id": _score_id(repo, pr, name),
+                    "sessionId": f"{repo}#{pr}",
+                    "name": name,
+                    "value": float(value),
+                    "dataType": "NUMERIC",
+                    "environment": environment,
+                    "comment": f"from feedback.db · {repo}#{pr}",
+                },
+            }
+        )
+    return events
+
+
+SCORE_CONFIGS = [
+    {
+        "name": REVIEW_ENGAGEMENT,
+        "dataType": "NUMERIC",
+        "minValue": 0,
+        "maxValue": 1,
+        "description": "Share of a PR's findings that drew any human reaction, resolution or reply. 0 = nobody engaged with the review.",
+    },
+    {
+        "name": REVIEW_ACCEPTANCE,
+        "dataType": "NUMERIC",
+        "minValue": -1,
+        "maxValue": 1,
+        "description": "Net human verdict over engaged findings: +1 all accepted, -1 all rejected. Absent when nothing was engaged.",
+    },
+]
+
+
+def iter_prs(conn: sqlite3.Connection):
+    for row in conn.execute(
+        "SELECT DISTINCT repo, pr FROM inline_finding ORDER BY repo, pr"
+    ):
+        yield row[0], int(row[1])
+
+
+def backfill(db_path: str, *, environment: str = "default", dry_run: bool = False) -> dict:
+    """Score every PR in the feedback DB. Returns a summary dict."""
+    conn = sqlite3.connect(db_path)
+    conn.row_factory = sqlite3.Row
+    events: list[dict] = []
+    scanned = 0
+    engaged_prs = 0
+    try:
+        for repo, pr in iter_prs(conn):
+            scanned += 1
+            tally = collect_pr_feedback(conn, repo, pr)
+            values = score_pr(tally)
+            if (values.get(REVIEW_ENGAGEMENT) or 0) > 0:
+                engaged_prs += 1
+            events.extend(build_score_events(repo, pr, values, environment))
+    finally:
+        conn.close()
+
+    summary = {"prs_scanned": scanned, "prs_with_engagement": engaged_prs, "scores": len(events)}
+    if dry_run or not events:
+        summary["posted"] = False
+        return summary
+
+    import langfuse_trace
+
+    conf = langfuse_trace._enabled()
+    if conf is None:
+        summary["posted"] = False
+        summary["error"] = "Langfuse not configured (LANGFUSE_HOST / keys unset)"
+        return summary
+    host, pk, sk = conf
+    status = langfuse_trace._post(host, pk, sk, events, 15.0)
+    summary["posted"] = status in (200, 201, 207)
+    summary["http_status"] = status
+    return summary
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description="Ship feedback.db verdicts to Langfuse as scores")
+    ap.add_argument("--db", default=os.environ.get("PRAGENT_FEEDBACK_DB", "/data/feedback.db"))
+    ap.add_argument("--environment", default="default")
+    ap.add_argument("--dry-run", action="store_true")
+    args = ap.parse_args()
+
+    summary = backfill(args.db, environment=args.environment, dry_run=args.dry_run)
+    print(json.dumps(summary, indent=2))
+    return 0 if summary.get("posted") or args.dry_run else 1
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

pilot/feedback_analyze.py

@@ -1,419 +1,7 @@
-"""pragent pilot — daily feedback analyzer.
-
-Reads `feedback.db` (written by `feedback_harvest.py`) and produces a
-markdown report that:
-
-  1. Ranks inline findings by **net false-positive score** (downvotes +
-     unresolved + negation-phrase replies − upvotes − resolved). Top of
-     this list = "the bot has been wrong about this repeatedly". These
-     are the candidates that *might* belong in the per-repo
-     `.pr-review.json:instructions` addendum.
-  2. Ranks findings by **net acceptance** — repeated 👍 / resolution =
-     "the bot's framing here is genuinely useful". These can be promoted
-     to the shared `architecture.md` so they don't have to be re-derived
-     every PR.
-  3. Reports a **restraint metric** — for every PR where the bot posted
-     zero findings, count how often a human reviewer also posted zero
-     substantive review comments. When the bot is loud on clean code,
-     that's a false-positive rate we can act on (DoorDash lesson:
-     "excessive noise on clean code is its own failure mode").
-  4. Reports a **case-review queue** — every disagreement case (a
-     downvote, unresolved, or a reply matching `FALSE_POSITIVE_PHRASES`)
-     is listed in full so a human can re-read the original PR and decide
-     if the finding was right or wrong.
-
-Output is plain markdown so it can be posted as a Gitea issue / comment
-without rendering work. Designed to be reviewed by a human, not auto-
-applied — per the DoorDash pattern, every material change to model /
-prompt / context goes through a benchmark gate first; this report IS
-that gate (or, more precisely, the queue feeding the gate).
-
-Never raises. A bad DB / no data → returns a friendly empty-state report.
-"""
-from __future__ import annotations
-
-import argparse
-import json
-import logging
-import os
-import sqlite3
-from collections import defaultdict
-from datetime import datetime, timezone
-from typing import Optional
-
-import feedback
-from feedback_harvest import (
-    FALSE_POSITIVE_PHRASES,
-    classify_reaction,
-    _is_negation_reply,  # noqa: F401  (re-exported for the test suite)
-)
-
-log = logging.getLogger("pragent.feedback.analyze")
-
-# How many findings to surface in each top-list. Capped because the
-# reports are read by humans; more than 20 per list and they skim.
-TOP_N = 20
-
-# Restraint threshold — fraction of "clean" PRs (zero findings) where
-# the bot produced ANY findings. Above this we recommend `.pr-review.json:
-# exclude_patterns` or a stricter `severity_threshold`.
-RESTRAINT_NOISE_THRESHOLD = 0.25
-
-
-# ---------------------------------------------------------------------------
-# Helpers
-# ---------------------------------------------------------------------------
-
-def _net_score(row) -> tuple[int, int]:
-    """Return (false_positive_score, acceptance_score) for one finding row.
-
-    FP signals: downvotes (+1), unresolved (+1), negation-phrase replies (+2).
-    Acceptance signals: upvotes (+1), resolved (+1).
-    """
-    fp = 0
-    ac = 0
-    fp += int(row["downvotes"] or 0)
-    fp += 1 if row["resolved"] == 0 else 0  # 0/1/NULL; 0 = unresolved
-    ac += 1 if row["resolved"] == 1 else 0
-    ac += int(row["upvotes"] or 0)
-    if row["reply_bodies"] and _is_negation_reply(row["reply_bodies"]):
-        fp += 2
-    return fp, ac
-
-
-def _short_problem(problem: str, n: int = 100) -> str:
-    s = (problem or "").strip().replace("\n", " ")
-    return s if len(s) <= n else s[: n - 1] + "…"
-
-
-def _restraint_stats(conn: sqlite3.Connection) -> dict:
-    """How often does the bot post findings on PRs that received zero
-    bot findings (= presumably clean)? Looks at `review.findings_total`
-    if present, otherwise counts `inline_finding` per PR.
-
-    NOTE: until `post_inline_review` records `findings_total`, this falls
-    back to "PRs with at least one finding row" which is an underestimate
-    (a bot review with zero findings leaves no row).
-    """
-    total_prs_with_review = conn.execute(
-        "SELECT COUNT(DISTINCT repo || '#' || pr) FROM review"
-    ).fetchone()[0]
-    prs_with_findings = conn.execute(
-        "SELECT COUNT(DISTINCT repo || '#' || pr) FROM inline_finding"
-    ).fetchone()[0]
-    if total_prs_with_review == 0:
-        return {"total": 0, "noisy": 0, "ratio": 0.0}
-    # This is currently "PRs where the bot left at least one inline
-    # comment". A precise "findings_total per review" needs
-    # post_inline_review to record it (TODO in the wiring step). Until
-    # then, treat this as a floor: real noise is >= this.
-    return {
-        "total": total_prs_with_review,
-        "noisy": prs_with_findings,
-        "ratio": prs_with_findings / total_prs_with_review,
-    }
-
-
-def _case_review_queue(conn: sqlite3.Connection, limit: int = 30) -> list[dict]:
-    """Findings that humans pushed back on — for manual re-review."""
-    rows = feedback.findings_with_votes(conn)
-    cases = []
-    for r in rows:
-        fp_score, _ = _net_score(r)
-        if fp_score <= 0:
-            continue
-        cases.append({
-            "posthash": r["posthash"],
-            "repo": r["repo"],
-            "pr": r["pr"],
-            "path": r["path"],
-            "line": r["line"],
-            "severity": r["severity"],
-            "problem": _short_problem(r["problem"], 200),
-            "fp_score": fp_score,
-            "upvotes": r["upvotes"] or 0,
-            "downvotes": r["downvotes"] or 0,
-            "resolved": r["resolved"],
-            "reply_count": r["reply_count"] or 0,
-            "reply_excerpt": _short_problem(r["reply_bodies"] or "", 200),
-        })
-    cases.sort(key=lambda c: c["fp_score"], reverse=True)
-    return cases[:limit]
-
-
-def _format_table(headers: list[str], rows: list[list[str]]) -> str:
-    if not rows:
-        return "_none yet_\n"
-    out = ["| " + " | ".join(headers) + " |",
-           "|" + "|".join(["---"] * len(headers)) + "|"]
-    for row in rows:
-        out.append("| " + " | ".join(row) + " |")
-    return "\n".join(out) + "\n"
-
-
-def _md_escape(s: str) -> str:
-    """Escape pipes + newlines so the value stays in one table cell."""
-    return (s or "").replace("|", "\\|").replace("\n", " ").strip()
-
-
-# ---------------------------------------------------------------------------
-# Main report builder
-# ---------------------------------------------------------------------------
-
-def analyze(db_path: str, *, since_ts: Optional[int] = None,
-            as_json: bool = False) -> str:
-    """Build the daily report. Returns a markdown string by default;
-    `as_json=True` returns a structured dict (for tests + dashboards)."""
-    conn = feedback.init(db_path)
-    try:
-        findings = list(feedback.findings_with_votes(conn, since_ts=since_ts))
-        total_findings = len(findings)
-        repo_set = {f["repo"] for f in findings}
-        case_queue = _case_review_queue(conn)
-        restraint = _restraint_stats(conn)
-
-        # Compute scores
-        scored: list[tuple[int, int, sqlite3.Row]] = []
-        for f in findings:
-            fp, ac = _net_score(f)
-            scored.append((fp, ac, f))
-
-        # Top false-positive patterns (sorted by fp score, deduped by posthash).
-        # `occurrences` comes from the inline_finding row — posthash UNIQUE
-        # means a single row can carry a count > 1 (set by record_inline_finding's
-        # ON CONFLICT DO UPDATE).
-        fp_by_hash: dict[str, dict] = {}
-        for fp, ac, f in scored:
-            if fp <= 0:
-                continue
-            ph = f["posthash"]
-            entry = fp_by_hash.setdefault(ph, {
-                "posthash": ph, "fp_score": 0, "ac_score": 0,
-                "repo": f["repo"], "path": f["path"], "line": f["line"],
-                "severity": f["severity"], "problem": f["problem"],
-                "occurrences": f["occurrences"], "upvs": 0, "downs": 0,
-                "resolved_true": 0, "resolved_false": 0,
-            })
-            entry["fp_score"] += fp
-            entry["ac_score"] += ac
-            entry["upvs"] += f["upvotes"] or 0
-            entry["downs"] += f["downvotes"] or 0
-            if f["resolved"] == 1:
-                entry["resolved_true"] += 1
-            elif f["resolved"] == 0:
-                entry["resolved_false"] += 1
-        fp_sorted = sorted(
-            fp_by_hash.values(), key=lambda e: e["fp_score"], reverse=True,
-        )[:TOP_N]
-
-        # Top accepted patterns
-        ac_by_hash: dict[str, dict] = {}
-        for fp, ac, f in scored:
-            if ac <= 0:
-                continue
-            ph = f["posthash"]
-            entry = ac_by_hash.setdefault(ph, {
-                "posthash": ph, "ac_score": 0, "fp_score": 0,
-                "repo": f["repo"], "path": f["path"], "line": f["line"],
-                "severity": f["severity"], "problem": f["problem"],
-                "occurrences": f["occurrences"], "upvs": 0, "downs": 0,
-                "resolved_true": 0,
-            })
-            entry["ac_score"] += ac
-            entry["fp_score"] += fp
-            entry["upvs"] += f["upvotes"] or 0
-            entry["downs"] += f["downvotes"] or 0
-            if f["resolved"] == 1:
-                entry["resolved_true"] += 1
-        ac_sorted = sorted(
-            ac_by_hash.values(), key=lambda e: e["ac_score"], reverse=True,
-        )[:TOP_N]
-
-        # Restraint recommendation
-        if restraint["ratio"] > RESTRAINT_NOISE_THRESHOLD:
-            restraint_msg = (
-                f"⚠️ Bot posted findings on **{restraint['ratio']:.0%}** of "
-                f"reviewed PRs ({restraint['noisy']} / {restraint['total']}). "
-                f"Above the {RESTRAINT_NOISE_THRESHOLD:.0%} threshold — "
-                "consider raising `.pr-review.json:severity_threshold` to "
-                "`medium` or `high` for noisy repos, or adding "
-                "`patterns.deny` to skip stylistic-only findings."
-            )
-        else:
-            restraint_msg = (
-                f"✅ Bot stayed quiet on **{1 - restraint['ratio']:.0%}** of "
-                f"reviewed PRs ({restraint['total'] - restraint['noisy']} / "
-                f"{restraint['total']}). Restraint OK."
-            )
-
-        if as_json:
-            return json.dumps({
-                "total_findings": total_findings,
-                "repos_seen": sorted(repo_set),
-                "restraint": restraint,
-                "top_false_positive": fp_sorted,
-                "top_accepted": ac_sorted,
-                "case_review_queue": case_queue,
-                "restraint_msg": restraint_msg,
-            }, indent=2)
-
-        # Markdown
-        ts_str = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
-        out = [f"# pragent feedback report — {ts_str}", ""]
-        out.append(f"- **findings analyzed**: {total_findings}")
-        out.append(f"- **repos with feedback**: {len(repo_set)} "
-                   f"({', '.join(sorted(repo_set))})")
-        out.append(f"- **case-review queue**: {len(case_queue)} disagreement(s)")
-        out.append("")
-        out.append("## Restraint")
-        out.append("")
-        out.append(restraint_msg)
-        out.append("")
-        out.append("> DoorDash rule (2026-07-06): *excessive noise on clean "
-                   "code is its own failure mode*. `severity_threshold` + "
-                   "`patterns.deny` are the knobs that dial restraint.")
-        out.append("")
-
-        out.append(f"## Top {len(fp_sorted)} false-positive candidates")
-        out.append("")
-        out.append("Aggregated by `posthash` (path:line:severity:problem). "
-                   "Sort key = downvotes + unresolved + negation-phrase replies "
-                   "− upvotes − resolved.")
-        out.append("")
-        rows = []
-        for e in fp_sorted:
-            rows.append([
-                str(e["fp_score"]),
-                f"`{_md_escape(e['repo'])}`",
-                f"`{_md_escape(e['path'])}:{e['line']}`",
-                e["severity"],
-                _md_escape(_short_problem(e["problem"])),
-                f"👍{e['upvs']} 👎{e['downs']}",
-                f"✅{e['resolved_true']} ❌{e['resolved_false']}",
-                str(e["occurrences"]),
-            ])
-        out.append(_format_table(
-            ["FP", "repo", "path:line", "sev", "problem",
-             "votes", "resolved", "seen"],
-            rows,
-        ))
-        out.append("")
-        out.append("_Review each row before adding it to "
-                   "`.pr-review.json:instructions`. Human reactions are NOT "
-                   "ground truth (DoorDash, 2026-07-06: authors accept/reject "
-                   "for workflow reasons) — re-read the PR before acting._")
-        out.append("")
-
-        out.append(f"## Top {len(ac_sorted)} accepted patterns")
-        out.append("")
-        out.append("Aggregated by posthash. Sort key = upvotes + resolved − "
-                   "downvotes − unresolved − negation-phrase replies.")
-        out.append("")
-        rows = []
-        for e in ac_sorted:
-            rows.append([
-                str(e["ac_score"]),
-                f"`{_md_escape(e['repo'])}`",
-                f"`{_md_escape(e['path'])}:{e['line']}`",
-                e["severity"],
-                _md_escape(_short_problem(e["problem"])),
-                f"👍{e['upvs']} 👎{e['downs']}",
-                f"✅{e['resolved_true']}",
-                str(e["occurrences"]),
-            ])
-        out.append(_format_table(
-            ["AC", "repo", "path:line", "sev", "problem",
-             "votes", "resolved", "seen"],
-            rows,
-        ))
-        out.append("")
-        out.append("_Promote widely-accepted patterns into the shared "
-                   "`architecture.md` on Nexus raw-hosted (or the per-repo "
-                   "`additional_context_urls`). These become part of the "
-                   "prompt-cached prefix → ~0 marginal cost on step 2+._")
-        out.append("")
-
-        out.append(f"## Case-review queue ({len(case_queue)})")
-        out.append("")
-        if not case_queue:
-            out.append("_No disagreements recorded yet. Once humans start "
-                       "reacting 👎 / leaving replies / not resolving bot "
-                       "comments, cases will appear here._")
-        else:
-            out.append("Each row needs a human to re-read the original PR and "
-                       "decide: was the bot right? If not, draft an "
-                       "`instructions` addendum or a `patterns.deny` rule.")
-            out.append("")
-            for c in case_queue:
-                url = (
-                    f"https://gitea.marcospaulo.dev.br/{c['repo']}/pulls/"
-                    f"{c['pr']}/files#r{c['posthash']}"
-                )
-                out.append(f"### FP={c['fp_score']} · {c['repo']}#{c['pr']}")
-                out.append(
-                    f"- file: `{_md_escape(c['path'])}:{c['line']}` · "
-                    f"severity: `{c['severity']}`",
-                )
-                out.append(f"- problem: {_md_escape(c['problem'])}")
-                out.append(
-                    f"- signals: 👍{c['upvotes']} 👎{c['downvotes']} · "
-                    f"resolved={c['resolved']} · replies={c['reply_count']}",
-                )
-                if c["reply_excerpt"]:
-                    out.append(
-                        f"- last reply: {_md_escape(c['reply_excerpt'])}",
-                    )
-                out.append(f"- posthash: `{c['posthash']}`")
-                out.append("")
-
-        out.append("## Where this report goes")
-        out.append("")
-        out.append("- **Per-repo actions** (`.pr-review.json:instructions`, "
-                   "`patterns.deny`, `severity_threshold`): edit the file on "
-                   "`main` via a regular PR. The next PR review picks up the "
-                   "change automatically.")
-        out.append("- **Cross-repo actions** (shared house-rules): update the "
-                   "`PRAGENT_ADDITIONAL_CONTEXT_URL` document on Nexus "
-                   "raw-hosted (`canalhandia/architecture.md` etc).")
-        out.append("- **Benchmark gate** (DoorDash pattern): before changing "
-                   "the model / prompt / context window, replay this report "
-                   "against the labeled `posthash` corpus. If a candidate "
-                   "addendum flips ≥ 1 currently-accepted finding into "
-                   "false-positive, drop it.")
-        out.append("")
-        out.append(f"_Generated from `{db_path}` by `feedback_analyze.py`._")
-        return "\n".join(out)
-    finally:
-        conn.close()
-
-
-# ---------------------------------------------------------------------------
-# CLI
-# ---------------------------------------------------------------------------
-
-def main() -> int:
-    p = argparse.ArgumentParser(description="Build the daily feedback report.")
-    p.add_argument("--db", default=os.environ.get(
-        "PRAGENT_FEEDBACK_DB", "/data/feedback.db",
-    ))
-    p.add_argument("--since", type=int, default=None,
-                   help="Unix timestamp; only include findings posted since")
-    p.add_argument("--json", action="store_true",
-                   help="Emit structured JSON instead of markdown")
-    p.add_argument("--out", default="-",
-                   help="Write to this path instead of stdout ('-' = stdout)")
-    args = p.parse_args()
-
-    out = analyze(args.db, since_ts=args.since, as_json=args.json)
-    if args.out == "-":
-        print(out)
-    else:
-        with open(args.out, "w") as f:
-            f.write(out)
-        print(f"wrote {args.out}", file=sys.stderr)
-    return 0
-
-
+"""Compatibility import for feedback analysis."""
+import importlib
+import sys
+_module = importlib.import_module("feedback.analyze")
+sys.modules[__name__] = _module
 if __name__ == "__main__":
-    import sys
-    raise SystemExit(main())
+    raise SystemExit(_module.main())

pilot/feedback_harvest.py

@@ -1,394 +1,7 @@
-"""pragent pilot — feedback harvester.
-
-For each PR the webhook server is about to review, walk back through the
-Gitea-side state of every bot comment from every prior review on that PR
-and record:
-  - reactions on the review body + on each inline comment
-  - thread-resolved state (Gitea's `resolver` field; non-empty = resolved)
-  - replies (issue-comments with `review_comment_id` matching ours)
-  - the bot's own findings_count + inline_count per review (for the
-    restraint metric)
-
-Everything is best-effort. A single 404 or 5xx is logged and skipped — we
-must never abort a review because the feedback DB had a hiccup.
-
-The harvester is intentionally separate from `review_pr` so it can be
-called independently (e.g. by the daily analyzer's "backfill" mode) and
-tested in isolation against a mocked Gitea client.
-"""
-from __future__ import annotations
-
-import json
-import logging
-import re
-import time
-import urllib.parse
-import urllib.request
-from typing import Optional
-
-import ai_review  # used as ai_review.gitea_get(...) so test mocks land on the binding
-
-from feedback import (
-    init,
-    record_inline_finding,
-    record_reaction,
-    record_reply,
-    record_review,
-    record_thread_state,
-    posthash,
-)
-
-log = logging.getLogger("pragent.feedback.harvest")
-
-# Reviewer identity — only collect feedback on comments authored by us.
-# Avoids harvesting reactions on human comments (which we never want to
-# count toward "bot usefulness").
-REVIEWER_LOGIN = "pragent-bot"
-
-# Reactions content tokens Gitea uses. We track +1 / -1 explicitly; the
-# others are stored as-is so the analyzer can mine them (👀 eyes,
-# laugh, hooray, confused, heart, rocket, …) without hardcoding a list
-# that drifts across Gitea versions.
-POSITIVE_REACTIONS = {"+1", "heart", "hooray", "laugh", "rocket"}
-NEGATIVE_REACTIONS = {"-1", "confused"}
-# Note: Gitea's `eyes` reaction (👀) means "I'm watching" — not approval
-# or disapproval. Treated as neutral by the analyzer.
-
-# Phrases that, in a reply, indicate the author thinks the bot's finding
-# was wrong. Casing + punctuation ignored; substring match is good enough
-# (false positives in the analyzer cost a human minute; false negatives
-# hide regressions).
-FALSE_POSITIVE_PHRASES = (
-    "false positive", "not actually", "this is fine", "this is intentional",
-    "not a bug", "intentional", "wrong here", "isn't actually",
-    "is not actually", "don't think this is", "i disagree", "this isn't right",
-    "this is correct", "this is expected", "by design", "this is by design",
-)
-
-# Gitea review-comment payload includes a 'body' field that may carry our
-# sha marker + severity header. We extract severity + path/line from it
-# as a fallback when the finding wasn't already seeded at post-time (old
-# reviews before feedback.py existed).
-SEV_RE = re.compile(r"\*\*\[(CRITICAL|HIGH|MEDIUM|LOW|INFO)\]\*\*", re.IGNORECASE)
-PATH_LINE_RE = re.compile(r"`([^?:\n]+?):(\d+)`")
-SHA_MARKER_RE = re.compile(r"<!--\s*pragent:sha=([0-9a-f]+)\s*-->", re.IGNORECASE)
-
-
-# ---------------------------------------------------------------------------
-# Low-level HTTP — tolerant JSON parse (Gitea sometimes returns `null` where
-# we expect `[]`, e.g. reactions on a fresh comment)
-# ---------------------------------------------------------------------------
-
-def _gitea_get_json(api: str, repo: str, path: str, token: str) -> tuple[int, object]:
-    status, raw = ai_review.gitea_get(api, repo, path, token)
-    if status != 200:
-        return status, None
-    try:
-        return status, json.loads(raw.decode("utf-8", errors="replace"))
-    except (json.JSONDecodeError, ValueError):
-        return status, None
-
-
-# ---------------------------------------------------------------------------
-# Parse helpers
-# ---------------------------------------------------------------------------
-
-def _parse_severity(body: str) -> str:
-    m = SEV_RE.search(body or "")
-    return m.group(1).upper() if m else "INFO"
-
-
-def _parse_path_line(body: str) -> tuple[Optional[str], Optional[int]]:
-    m = PATH_LINE_RE.search(body or "")
-    if not m:
-        return None, None
-    path = m.group(1).strip()
-    try:
-        return path, int(m.group(2))
-    except ValueError:
-        return path, None
-
-
-def _parse_sha(body: str) -> Optional[str]:
-    m = SHA_MARKER_RE.search(body or "")
-    return m.group(1) if m else None
-
-
-def _is_negation_reply(body: str) -> bool:
-    if not body:
-        return False
-    norm = body.lower()
-    return any(p in norm for p in FALSE_POSITIVE_PHRASES)
-
-
-# ---------------------------------------------------------------------------
-# Reaction classification (cheap, used by the analyzer — not the harvester
-# itself)
-# ---------------------------------------------------------------------------
-
-def classify_reaction(content: str) -> str:
-    """Bucket a reaction into 'positive', 'negative', or 'neutral'."""
-    c = (content or "").strip().lower()
-    if c in POSITIVE_REACTIONS:
-        return "positive"
-    if c in NEGATIVE_REACTIONS:
-        return "negative"
-    return "neutral"
-
-
-# ---------------------------------------------------------------------------
-# Main harvest entry
-# ---------------------------------------------------------------------------
-
-def harvest_for_pr(
-    *,
-    api: str,
-    token: str,
-    repo: str,
-    pr_index: int,
-    db_path: str,
-    page_size: int = 50,
-) -> dict:
-    """Walk every bot-authored review on the given PR and record reactions
-    + thread state + replies. Returns a stats dict for logging.
-
-    `db_path` is the SQLite file path (env: `PRAGENT_FEEDBACK_DB`,
-    typically `/data/feedback.db` mounted via the `feedback-data` PVC).
-    """
-    conn = init(db_path)
-    stats = {
-        "reviews_seen": 0, "findings_seen": 0,
-        "reactions_recorded": 0, "thread_states_recorded": 0,
-        "replies_recorded": 0, "errors": 0,
-    }
-
-    try:
-        # 1. List every review on the PR (paginated, but PRs rarely have >page_size)
-        status, payload = _gitea_get_json(
-            api, repo, f"pulls/{pr_index}/reviews?per_page={page_size}", token,
-        )
-        if status != 200 or not isinstance(payload, list):
-            log.info("harvest: reviews list failed status=%d", status)
-            stats["errors"] += 1
-            return stats
-
-        for rev in payload:
-            user = (rev.get("user") or {}).get("login", "")
-            if user != REVIEWER_LOGIN:
-                continue
-            stats["reviews_seen"] += 1
-
-            review_id_gitea = rev.get("id")
-            head_sha = rev.get("commit_id", "")
-            review_body = rev.get("body", "") or ""
-            body_sha = _parse_sha(review_body)
-            # Trust the sha marker inside the body — Gitea's commit_id field is
-            # for the LAST commit, not necessarily the reviewed head. If we
-            # can't find a marker, fall back to commit_id.
-            effective_sha = body_sha or head_sha
-            created_at = _parse_iso_ts(rev.get("created_at", ""))
-
-            db_review_id = record_review(
-                conn, repo=repo, pr=pr_index, head_sha=effective_sha,
-                review_id_gitea=review_id_gitea,
-                posted_at=created_at,
-            )
-
-            # 2. Inline comments for this review
-            if review_id_gitea is None:
-                continue
-            rstatus, rpayload = _gitea_get_json(
-                api, repo, f"pulls/{pr_index}/reviews/{review_id_gitea}/comments",
-                token,
-            )
-            if rstatus != 200 or not isinstance(rpayload, list):
-                stats["errors"] += 1
-                continue
-
-            for ic in rpayload:
-                ic_id = ic.get("id")
-                if ic_id is None:
-                    continue
-                ic_body = ic.get("body", "") or ""
-                ic_path = ic.get("path")
-                ic_line = ic.get("position") or ic.get("line")
-                ic_severity = _parse_severity(ic_body)
-                # Fall back to body parse when Gitea didn't echo path/line
-                if not ic_path or not ic_line:
-                    bp, bl = _parse_path_line(ic_body)
-                    ic_path = ic_path or bp
-                    ic_line = ic_line or bl
-
-                if not ic_path or not ic_line:
-                    log.info(
-                        "harvest: inline %s missing path/line, skipping", ic_id,
-                    )
-                    continue
-
-                finding_id = record_inline_finding(
-                    conn, review_id=db_review_id, repo=repo, pr=pr_index,
-                    path=ic_path, line=ic_line, severity=ic_severity,
-                    problem=_strip_severity_header(ic_body),
-                    fix="", suggestion="",
-                    comment_id=ic_id,
-                    posted_at=created_at,
-                )
-                stats["findings_seen"] += 1
-                if finding_id is None:
-                    continue
-
-                # 3. Reactions on the inline comment
-                react_status, react_payload = _gitea_get_json(
-                    api, repo, f"issues/comments/{ic_id}/reactions", token,
-                )
-                if react_status == 200 and isinstance(react_payload, list):
-                    for r in react_payload:
-                        ruser = (r.get("user") or {}).get("login", "") or "?"
-                        # Gitea has occasionally returned `content` as a
-                        # dict on older versions; coerce to str defensively.
-                        rcontent = str(r.get("content") or "").strip()
-                        if not rcontent:
-                            continue
-                        if record_reaction(
-                            conn, comment_id=ic_id, user=ruser,
-                            content=rcontent,
-                            created_at=_parse_iso_ts(r.get("created_at", "")),
-                        ):
-                            stats["reactions_recorded"] += 1
-
-                # 4. Thread state (Gitea's `resolver` field on the inline
-                #    comment). Some Gitea versions serialize this as a user
-                #    object ({login, ...}) instead of a username string —
-                #    coerce defensively before calling .strip().
-                resolver_raw = ic.get("resolver")
-                if isinstance(resolver_raw, dict):
-                    resolver = (resolver_raw.get("login") or "").strip()
-                else:
-                    resolver = str(resolver_raw or "").strip()
-                if resolver_raw is not None:  # field present, even if ""
-                    record_thread_state(
-                        conn, finding_id=finding_id,
-                        resolved=bool(resolver),
-                    )
-                    stats["thread_states_recorded"] += 1
-
-            # 5. Replies on this review (issue-comments whose
-            #    `review_comment_id` points at one of our inline comments).
-            #    Some Gitea versions don't expose `review_comment_id` on the
-            #    issue-comment endpoint — in that case `replies` stays
-            #    empty; we degrade gracefully.
-            try:
-                _harvest_replies(
-                    api=api, repo=repo, token=token,
-                    pr_index=pr_index, review_id=review_id_gitea,
-                    inline_comments=rpayload, conn=conn,
-                    stats=stats,
-                )
-            except Exception as e:
-                log.info("harvest: replies fetch failed: %s", e)
-                stats["errors"] += 1
-    finally:
-        conn.close()
-
-    return stats
-
-
-def _harvest_replies(
-    *, api: str, repo: str, token: str, pr_index: int,
-    review_id: int, inline_comments: list, conn, stats: dict,
-) -> None:
-    """Fetch issue comments on this PR; record those whose
-    `review_comment_id` matches one of our inline comment IDs.
-    Gitea 1.26 doesn't include that field — we fall back to fetching each
-    inline comment individually via `issues/comments/{id}` (does include
-    the field) only if the bulk fetch is empty.
-    """
-    inline_ids = {c.get("id") for c in inline_comments if c.get("id") is not None}
-    if not inline_ids:
-        return
-
-    status, payload = _gitea_get_json(
-        api, repo, f"issues/{pr_index}/comments?per_page=100", token,
-    )
-    if status != 200 or not isinstance(payload, list):
-        return
-
-    # Build mapping inline_id -> finding_id (one SELECT instead of N)
-    rows = conn.execute(
-        "SELECT comment_id, id FROM inline_finding WHERE comment_id IN ("
-        + ",".join("?" * len(inline_ids)) + ")",
-        list(inline_ids),
-    ).fetchall()
-    inline_to_finding = {r[0]: r[1] for r in rows}
-
-    for c in payload:
-        rcid = c.get("review_comment_id")
-        if not rcid or rcid not in inline_to_finding:
-            continue
-        author = (c.get("user") or {}).get("login", "") or "?"
-        body = c.get("body", "") or ""
-        ts = _parse_iso_ts(c.get("created_at", ""))
-        if record_reply(
-            conn, finding_id=inline_to_finding[rcid],
-            author=author, body=body, created_at=ts,
-        ):
-            stats["replies_recorded"] += 1
-
-
-# ---------------------------------------------------------------------------
-# Helpers
-# ---------------------------------------------------------------------------
-
-def _strip_severity_header(body: str) -> str:
-    """Drop the leading `**[SEVERITY]**` so the posthash captures the
-    substance, not the severity label."""
-    return SEV_RE.sub("", body or "", count=1).strip()
-
-
-def _parse_iso_ts(s: str) -> int:
-    if not s:
-        return int(time.time())
-    try:
-        # Python 3.11+ fromisoformat tolerates the trailing 'Z'.
-        return int(__import__("datetime").datetime.fromisoformat(
-            s.replace("Z", "+00:00")
-        ).timestamp())
-    except Exception:
-        return int(time.time())
-
-
-# ---------------------------------------------------------------------------
-# CLI for manual backfill / first-time seed
-# ---------------------------------------------------------------------------
-
-def main() -> int:
-    import argparse, os
-    p = argparse.ArgumentParser(
-        description="Harvest reactions/threads/replies on bot PR comments.",
-    )
-    p.add_argument("--api", default=os.environ.get(
-        "GITEA_API", "http://gitea-http.gitea.svc.cluster.local:3000",
-    ))
-    p.add_argument("--token", default=os.environ.get("PRAGENT_BOT_TOKEN", ""))
-    p.add_argument("--repo", required=True, help="owner/name")
-    p.add_argument("--pr", type=int, required=True, help="PR index")
-    p.add_argument("--db", default=os.environ.get(
-        "PRAGENT_FEEDBACK_DB", "/data/feedback.db",
-    ))
-    args = p.parse_args()
-
-    if not args.token:
-        print("PRAGENT_BOT_TOKEN required", flush=True)
-        return 2
-
-    logging.basicConfig(level=logging.INFO)
-    stats = harvest_for_pr(
-        api=args.api, token=args.token,
-        repo=args.repo, pr_index=args.pr, db_path=args.db,
-    )
-    print(json.dumps(stats), flush=True)
-    return 0
-
-
+"""Compatibility import for feedback harvesting."""
+import importlib
+import sys
+_module = importlib.import_module("feedback.harvest")
+sys.modules[__name__] = _module
 if __name__ == "__main__":
-    raise SystemExit(main())
+    raise SystemExit(_module.main())

pilot/feedback_post.py

@@ -1,128 +1,7 @@
-"""pragent pilot — daily feedback report delivery.
-
-Calls `feedback_analyze.analyze()` and posts the markdown report as a
-comment on a single long-lived "feedback roll-up" issue in
-`gitea_admin/pragent`. Comments are append-only history — one comment per
-run, timestamped in the body. This keeps every report in one place, easy
-to scroll, and avoids the issue-explosion of "one issue per day".
-
-If the issue doesn't exist yet, create it. Subsequent runs just add a
-new comment.
-
-Designed for the daily K8s CronJob (`k8s/pragent-feedback-cronjob.yaml`)
-but runnable from CLI for ad-hoc checks.
-
-Env:
-  GITEA_API            in-cluster Gitea base URL
-  PRAGENT_BOT_TOKEN    bot token (Write collaborator on gitea_admin/pragent)
-  PRAGENT_FEEDBACK_DB  path to SQLite (default /data/feedback.db)
-  PRAGENT_FEEDBACK_ISSUE_REPO   default gitea_admin/pragent
-  PRAGENT_FEEDBACK_ISSUE_TITLE  default "pragent feedback roll-up"
-"""
-from __future__ import annotations
-
-import argparse
-import json
-import logging
-import os
+"""Compatibility import for feedback posting."""
+import importlib
 import sys
-
-import ai_review
-
-from feedback_analyze import analyze
-
-log = logging.getLogger("pragent.feedback.post")
-
-
-REPO_DEFAULT = "gitea_admin/pragent"
-TITLE_DEFAULT = "pragent feedback roll-up"
-
-
-def _find_or_create_issue(api: str, token: str, repo: str, title: str) -> int:
-    """Locate the open issue with this title; create one if missing.
-
-    Gitea's issue search is via `GET /repos/{o}/{r}/issues?state=open&q=...`
-    (q matches title + body). We filter client-side for the exact title
-    to avoid query-text false matches.
-    """
-    status, raw = ai_review.gitea_get(api, repo, "issues?state=open&per_page=50", token)
-    if status == 200:
-        try:
-            for issue in json.loads(raw):
-                if issue.get("title") == title:
-                    # NB: the comment URL needs the per-repo `number`, not the
-                    # global `id`. `id=60 num=8` for an early-N create; we want
-                    # `num=8` for `/repos/o/r/issues/8/comments`.
-                    return int(issue["number"])
-        except (json.JSONDecodeError, ValueError, KeyError):
-            pass
-    # Create
-    status, raw = ai_review.gitea_post(
-        api, repo, "issues", token,
-        {"title": title, "body": "pragent feedback roll-up — auto-created."},
-    )
-    if status not in (200, 201):
-        raise RuntimeError(f"issue create failed: HTTP {status} body={raw[:200]!r}")
-    return int(json.loads(raw)["number"])
-
-
-def _post_comment(api: str, token: str, repo: str, issue_number: int, body: str) -> int:
-    status, raw = ai_review.gitea_post(
-        api, repo, f"issues/{issue_number}/comments", token, {"body": body},
-    )
-    if status not in (200, 201):
-        raise RuntimeError(f"comment post failed: HTTP {status} body={raw[:200]!r}")
-    return json.loads(raw)["id"]
-
-
-def deliver(
-    *, api: str, token: str, db_path: str,
-    repo: str = REPO_DEFAULT, title: str = TITLE_DEFAULT,
-    since_ts: int | None = None,
-) -> dict:
-    """Build the report and post it as a comment. Returns a stats dict."""
-    report = analyze(db_path, since_ts=since_ts)
-    issue_id = _find_or_create_issue(api, token, repo, title)
-    comment_id = _post_comment(api, token, repo, issue_id, report)
-    return {
-        "repo": repo, "issue_id": issue_id, "comment_id": comment_id,
-        "report_bytes": len(report.encode()),
-    }
-
-
-def main() -> int:
-    p = argparse.ArgumentParser(
-        description="Post the daily feedback report to Gitea.",
-    )
-    p.add_argument("--api", default=os.environ.get(
-        "GITEA_API", "http://gitea-http.gitea.svc.cluster.local:3000",
-    ))
-    p.add_argument("--token", default=os.environ.get("PRAGENT_BOT_TOKEN", ""))
-    p.add_argument("--db", default=os.environ.get(
-        "PRAGENT_FEEDBACK_DB", "/data/feedback.db",
-    ))
-    p.add_argument("--repo", default=os.environ.get(
-        "PRAGENT_FEEDBACK_ISSUE_REPO", REPO_DEFAULT,
-    ))
-    p.add_argument("--title", default=os.environ.get(
-        "PRAGENT_FEEDBACK_ISSUE_TITLE", TITLE_DEFAULT,
-    ))
-    p.add_argument("--since", type=int, default=None,
-                   help="Unix timestamp; only include findings posted since")
-    args = p.parse_args()
-
-    if not args.token:
-        print("PRAGENT_BOT_TOKEN required", flush=True)
-        return 2
-
-    logging.basicConfig(level=logging.INFO)
-    stats = deliver(
-        api=args.api, token=args.token, db_path=args.db,
-        repo=args.repo, title=args.title, since_ts=args.since,
-    )
-    print(json.dumps(stats), flush=True)
-    return 0
-
-
+_module = importlib.import_module("feedback.post")
+sys.modules[__name__] = _module
 if __name__ == "__main__":
-    raise SystemExit(main())
+    raise SystemExit(_module.main())

pilot/feedback_scores.py

@@ -1,247 +1,7 @@
-#!/usr/bin/env python3
-"""pragent pilot — feedback DB to Langfuse scores.
-
-`feedback.db` already records every reaction, thread resolution and reply a
-maintainer leaves on a bot comment. That is the only ground truth pragent has
-about whether a finding was any good, and until now it went to a markdown report
-nobody reads and nowhere else. This ships it to Langfuse as session-level
-scores, so "was the reviewer right" sits on the same axis as "what did it cost".
-
-Session, not trace
-------------------
-`langfuse_trace` sets `sessionId` to `"{repo}#{pr}"` and lets the trace id be a
-fresh uuid per review. Feedback arrives days later against a PR, not against one
-particular re-run of the reviewer, and nothing in `feedback.db` records which
-trace produced which comment. Scoring the session is therefore both the
-available join and the honest granularity: this is feedback on the review of
-this PR, not on one invocation.
-
-Two scores, deliberately separated
-----------------------------------
-  * `review_engagement` — the share of a PR's findings that got any human
-    response at all. This is a signal about the *feedback loop*, not the
-    reviewer: at the time of writing it is 0.0 across all 113 recorded reviews,
-    which is exactly the fact that makes an accuracy metric impossible today.
-    It must be watched first, because every other quality number is vapour
-    until it moves.
-  * `review_acceptance` — net verdict over the findings that *did* get a
-    response: (upvotes + resolved) - (downvotes + negation replies), normalised
-    to -1..1. Computed only over engaged findings, so an ignored review scores
-    `None` rather than 0. Zero would read as "humans judged this exactly
-    neutral"; the truth is nobody looked.
-
-Fail-open and idempotent. Score ids are derived from (repo, pr, name) so a
-re-run overwrites rather than duplicates.
-"""
-from __future__ import annotations
-
-import argparse
-import json
-import os
-import sqlite3
+"""Compatibility import for feedback scores."""
+import importlib
 import sys
-import uuid
-from datetime import datetime, timezone
-
-sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
-
-from feedback_harvest import classify_reaction, _is_negation_reply  # noqa: E402
-
-REVIEW_ENGAGEMENT = "review_engagement"
-REVIEW_ACCEPTANCE = "review_acceptance"
-
-# Stable namespace so the same (repo, pr, score) always produces the same score
-# id — Langfuse treats a repeated id as an update, which is what a backfill of a
-# still-accumulating PR should do.
-_NS = uuid.UUID("6f1d9c2e-4a77-4f2a-9c1a-0d3b5e8a7c41")
-
-
-def _score_id(repo: str, pr: int, name: str) -> str:
-    return str(uuid.uuid5(_NS, f"{repo}#{pr}#{name}"))
-
-
-def collect_pr_feedback(conn: sqlite3.Connection, repo: str, pr: int) -> dict:
-    """Tally one PR's findings and the human responses attached to them.
-
-    Returns counts only — the scoring maths lives in `score_pr` so it can be
-    tested without a database.
-    """
-    rows = conn.execute(
-        "SELECT id, comment_id FROM inline_finding WHERE repo = ? AND pr = ?",
-        (repo, pr),
-    ).fetchall()
-    total = len(rows)
-    engaged = 0
-    positive = 0
-    negative = 0
-
-    for row in rows:
-        fid = row["id"] if isinstance(row, sqlite3.Row) else row[0]
-        cid = row["comment_id"] if isinstance(row, sqlite3.Row) else row[1]
-        pos = neg = 0
-
-        if cid is not None:
-            for r in conn.execute(
-                "SELECT content FROM reaction WHERE comment_id = ?", (cid,)
-            ):
-                kind = classify_reaction(r[0])
-                if kind == "positive":
-                    pos += 1
-                elif kind == "negative":
-                    neg += 1
-
-        for r in conn.execute(
-            "SELECT resolved FROM thread_state WHERE finding_id = ?", (fid,)
-        ):
-            # A resolved thread means the maintainer acted on the finding.
-            if r[0]:
-                pos += 1
-
-        # A reply counts as engagement either way; only a negation phrase makes
-        # it a vote against. A neutral reply ("done", "good catch, but…") is
-        # deliberately not a positive vote — it says someone looked, not that
-        # they agreed.
-        replied = 0
-        for r in conn.execute(
-            "SELECT body FROM reply WHERE finding_id = ?", (fid,)
-        ):
-            replied += 1
-            if _is_negation_reply(r[0]):
-                neg += 1
-
-        if pos or neg or replied:
-            engaged += 1
-        positive += pos
-        negative += neg
-
-    return {"total": total, "engaged": engaged, "positive": positive, "negative": negative}
-
-
-def score_pr(tally: dict) -> dict:
-    """Turn one PR's tally into score values.
-
-    `review_acceptance` is `None` when nothing was engaged — see the module
-    docstring on why that is not 0.
-    """
-    total = int(tally.get("total") or 0)
-    engaged = int(tally.get("engaged") or 0)
-    pos = int(tally.get("positive") or 0)
-    neg = int(tally.get("negative") or 0)
-
-    engagement = round(engaged / total, 4) if total else None
-    acceptance = None
-    if pos or neg:
-        acceptance = round((pos - neg) / (pos + neg), 4)
-    return {REVIEW_ENGAGEMENT: engagement, REVIEW_ACCEPTANCE: acceptance}
-
-
-def build_score_events(
-    repo: str, pr: int, values: dict, environment: str = "default",
-    timestamp: str | None = None,
-) -> list[dict]:
-    """`score-create` events for one PR's feedback.
-
-    Every event carries a timestamp: the ingestion endpoint rejects those that
-    do not, and it reports the rejection as a per-event 400 inside an HTTP 207,
-    which reads as success to a caller that only checks the status code.
-    """
-    ts = timestamp or datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
-    events = []
-    for name, value in values.items():
-        if value is None:
-            continue
-        events.append(
-            {
-                "id": str(uuid.uuid4()),
-                "type": "score-create",
-                "timestamp": ts,
-                "body": {
-                    "id": _score_id(repo, pr, name),
-                    "sessionId": f"{repo}#{pr}",
-                    "name": name,
-                    "value": float(value),
-                    "dataType": "NUMERIC",
-                    "environment": environment,
-                    "comment": f"from feedback.db · {repo}#{pr}",
-                },
-            }
-        )
-    return events
-
-
-SCORE_CONFIGS = [
-    {
-        "name": REVIEW_ENGAGEMENT,
-        "dataType": "NUMERIC",
-        "minValue": 0,
-        "maxValue": 1,
-        "description": "Share of a PR's findings that drew any human reaction, resolution or reply. 0 = nobody engaged with the review.",
-    },
-    {
-        "name": REVIEW_ACCEPTANCE,
-        "dataType": "NUMERIC",
-        "minValue": -1,
-        "maxValue": 1,
-        "description": "Net human verdict over engaged findings: +1 all accepted, -1 all rejected. Absent when nothing was engaged.",
-    },
-]
-
-
-def iter_prs(conn: sqlite3.Connection):
-    for row in conn.execute(
-        "SELECT DISTINCT repo, pr FROM inline_finding ORDER BY repo, pr"
-    ):
-        yield row[0], int(row[1])
-
-
-def backfill(db_path: str, *, environment: str = "default", dry_run: bool = False) -> dict:
-    """Score every PR in the feedback DB. Returns a summary dict."""
-    conn = sqlite3.connect(db_path)
-    conn.row_factory = sqlite3.Row
-    events: list[dict] = []
-    scanned = 0
-    engaged_prs = 0
-    try:
-        for repo, pr in iter_prs(conn):
-            scanned += 1
-            tally = collect_pr_feedback(conn, repo, pr)
-            values = score_pr(tally)
-            if (values.get(REVIEW_ENGAGEMENT) or 0) > 0:
-                engaged_prs += 1
-            events.extend(build_score_events(repo, pr, values, environment))
-    finally:
-        conn.close()
-
-    summary = {"prs_scanned": scanned, "prs_with_engagement": engaged_prs, "scores": len(events)}
-    if dry_run or not events:
-        summary["posted"] = False
-        return summary
-
-    import langfuse_trace
-
-    conf = langfuse_trace._enabled()
-    if conf is None:
-        summary["posted"] = False
-        summary["error"] = "Langfuse not configured (LANGFUSE_HOST / keys unset)"
-        return summary
-    host, pk, sk = conf
-    status = langfuse_trace._post(host, pk, sk, events, 15.0)
-    summary["posted"] = status in (200, 201, 207)
-    summary["http_status"] = status
-    return summary
-
-
-def main() -> int:
-    ap = argparse.ArgumentParser(description="Ship feedback.db verdicts to Langfuse as scores")
-    ap.add_argument("--db", default=os.environ.get("PRAGENT_FEEDBACK_DB", "/data/feedback.db"))
-    ap.add_argument("--environment", default="default")
-    ap.add_argument("--dry-run", action="store_true")
-    args = ap.parse_args()
-
-    summary = backfill(args.db, environment=args.environment, dry_run=args.dry_run)
-    print(json.dumps(summary, indent=2))
-    return 0 if summary.get("posted") or args.dry_run else 1
-
-
+_module = importlib.import_module("feedback.scores")
+sys.modules[__name__] = _module
 if __name__ == "__main__":
-    raise SystemExit(main())
+    raise SystemExit(_module.main())

pilot/gitea_client.py

@@ -0,0 +1,5 @@
+"""Compatibility import for the Gitea adapter."""
+import importlib
+import sys
+_module = importlib.import_module("entrypoints.gitea")
+sys.modules[__name__] = _module

pilot/langfuse_trace.py

@@ -1,468 +1,5 @@
-#!/usr/bin/env python3
-"""pragent pilot — Langfuse trace emission.
-
-Ships one trace per PR review to a self-hosted Langfuse (v3) so the reviewer's
-token spend, latency and per-model behaviour are queryable outside the review
-body. The review body already renders a usage table; that table is per-PR and
-disappears into Gitea. This is the same numbers, aggregated.
-
-Why hand-rolled instead of the `langfuse` SDK: the pilot image is stdlib-only
-(see pilot/Dockerfile — no requirements.txt anywhere in the repo), and the
-ingestion API is a single authenticated POST of a JSON batch. Pulling an SDK
-plus its otel dependency tree into a fail-open telemetry side-path is a bad
-trade.
-
-Provider split
---------------
-`environment` on every trace is either `ollama` or `claude`, derived from the
-resolved display model (`resolve_environment`). That is what keeps the two
-spend stories separate in Langfuse: every dashboard, filter and cost breakdown
-takes an environment selector, so "what did the local/self-hosted path cost"
-and "what did the Claude path cost" are two views of one project rather than
-two projects with two key pairs to rotate. Tags carry the finer split
-(`provider:headroom`, `model:...`, `engine:opencode`).
-
-Cost
-----
-The pilot's own path bills $0 (headroom proxy, no per-token charge), so the
-`cost` reported to Langfuse is the *equivalent* cost from `cost_model` — what
-the same tokens would bill on the comparison model. That is the number worth
-trending; a chart of $0.00 is not.
-
-A model is "free" when `cost_model.PRICES` has no entry for it (MiniMax-M2.7,
-glm-5.2:cloud) or when its entry is all zeros (the self-hosted vLLM qwen). In
-both cases the reported cost is priced against the comparison target instead —
-same precedence the review body uses: `.pr-review.json:cost_target` >
-`PRAGENT_PRICE_TARGET` > `claude-sonnet-5`. A paid model is priced as itself.
-
-Because a hypothetical and a real charge must never be read as the same
-number, every trace is tagged `cost:actual` or `cost:equivalent:<target>`, and
-the generation's metadata carries `cost_basis`.
-
-Fail-open: every entry point swallows its own exceptions. Telemetry must never
-cost a review.
-
-Env:
-  LANGFUSE_HOST         e.g. http://langfuse-web.langfuse.svc.cluster.local:3000
-  LANGFUSE_PUBLIC_KEY   pk-lf-...
-  LANGFUSE_SECRET_KEY   sk-lf-...
-  LANGFUSE_TIMEOUT      seconds, default 5
-  LANGFUSE_DEBUG        1 to log ingestion failures to stderr
-Disabled (silently) when host or either key is unset.
-"""
-
-from __future__ import annotations
-
-import base64
-import json
-import os
+"""Compatibility import for Langfuse telemetry."""
+import importlib
 import sys
-import time
-import urllib.error
-import urllib.request
-import uuid
-from datetime import datetime, timezone
-
-INGESTION_PATH = "/api/public/ingestion"
-
-# Model-key prefixes that mean "this review ran against Anthropic-shaped
-# billing". Everything else (glm, MiniMax, qwen, local vLLM) is the ollama /
-# self-hosted side of the split.
-_CLAUDE_PREFIXES = ("claude-", "anthropic/")
-
-
-def _now_iso() -> str:
-    return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
-
-
-def _enabled() -> tuple[str, str, str] | None:
-    host = (os.environ.get("LANGFUSE_HOST") or "").strip().rstrip("/")
-    pk = (os.environ.get("LANGFUSE_PUBLIC_KEY") or "").strip()
-    sk = (os.environ.get("LANGFUSE_SECRET_KEY") or "").strip()
-    if not host or not pk or not sk:
-        return None
-    return host, pk, sk
-
-
-def _debug(msg: str) -> None:
-    if os.environ.get("LANGFUSE_DEBUG"):
-        print(f"pragent/langfuse: {msg}", file=sys.stderr, flush=True)
-
-
-def strip_provider(model: str) -> str:
-    """`headroom/claude-sonnet-5` -> `claude-sonnet-5`. Bare names pass through."""
-    return model.split("/", 1)[1] if "/" in model else model
-
-
-def provider_of(model: str) -> str:
-    """The opencode provider block a display model routes through."""
-    return model.split("/", 1)[0] if "/" in model else "headroom"
-
-
-def resolve_environment(model: str) -> str:
-    """Which spend story this review belongs to: `claude` or `ollama`.
-
-    Keyed off the bare model name, not the provider, because both paths route
-    through the same `headroom` proxy — `headroom/claude-sonnet-5` is Claude
-    spend, `headroom/glm-5.2:cloud` is not.
-    """
-    bare = strip_provider(model).lower()
-    return "claude" if bare.startswith(_CLAUDE_PREFIXES) else "ollama"
-
-
-def _usage_details(usage: dict) -> dict:
-    """opencode's usage dict -> Langfuse `usageDetails`.
-
-    Langfuse sums every key except the ones it knows are derived, so `input`
-    here is the *uncached* portion: reporting both `input` (which opencode
-    reports as the full input, cache included) and `cache_read_input_tokens`
-    would double-count.
-    """
-    inp = int(usage.get("input") or 0)
-    cache_read = int(usage.get("cache_read") or 0)
-    cache_write = int(usage.get("cache_write") or 0)
-    details = {
-        "input": max(0, inp - cache_read),
-        "output": int(usage.get("output") or 0),
-    }
-    if cache_read:
-        details["cache_read_input_tokens"] = cache_read
-    if cache_write:
-        details["cache_write_input_tokens"] = cache_write
-    reasoning = int(usage.get("reasoning") or 0)
-    if reasoning:
-        details["reasoning"] = reasoning
-    return details
-
-
-DEFAULT_PRICE_TARGET = "claude-sonnet-5"
-
-
-def resolve_price_target(price_target: str | None = None) -> str:
-    """The model to price free/unknown runs against.
-
-    Mirrors `ai_review._resolve_price_target`: an explicit target (which the
-    caller reads from `.pr-review.json:cost_target`) wins, then
-    `PRAGENT_PRICE_TARGET`, then Sonnet.
-    """
-    if price_target and price_target.strip():
-        return price_target.strip()
-    env = os.environ.get("PRAGENT_PRICE_TARGET", "").strip()
-    return env or DEFAULT_PRICE_TARGET
-
-
-def _is_free(price) -> bool:
-    """A price entry that charges nothing — self-hosted or proxied at no cost."""
-    return price.input == 0 and price.output == 0
-
-
-def _cost_details(usage: dict, model: str, price_target: str | None = None) -> tuple[dict, str]:
-    """USD for this usage plus the basis it was computed on.
-
-    Returns `({"total": …}, basis)` where basis is `actual` for a model that
-    genuinely bills, or `equivalent:<target>` for one that does not. `({}, "")`
-    when nothing can be priced at all — better no number than a wrong one.
-
-    Local import + broad except: `cost_model` is only present on the opencode
-    path, and an unknown model key must not break telemetry.
-    """
-    try:
-        from cost_model import PRICES, Usage, cost
-
-        bare = strip_provider(model)
-        price = PRICES.get(bare)
-        basis = "actual"
-        if price is None or _is_free(price):
-            # MiniMax / glm / self-hosted qwen: $0 through the proxy, so the
-            # useful number is what these tokens would have billed elsewhere.
-            target = resolve_price_target(price_target)
-            price = PRICES.get(target)
-            if price is None:
-                _debug(f"comparison target {target!r} not in PRICES")
-                return {}, ""
-            basis = f"equivalent:{target}"
-
-        u = Usage(
-            uncached_input=max(0, int(usage.get("input") or 0) - int(usage.get("cache_read") or 0)),
-            cached_input=int(usage.get("cache_read") or 0),
-            cache_writes=int(usage.get("cache_write") or 0),
-            output=int(usage.get("output") or 0),
-        )
-        return {"total": round(cost(u, price), 6)}, basis
-    except Exception as e:  # pragma: no cover - defensive
-        _debug(f"cost lookup failed for {model!r}: {e}")
-        return {}, ""
-
-
-def _severity_counts(findings: list[dict] | None) -> dict:
-    counts: dict[str, int] = {}
-    for f in findings or []:
-        sev = str(f.get("severity") or "unknown").lower()
-        counts[sev] = counts.get(sev, 0) + 1
-    return counts
-
-
-def build_batch(
-    *,
-    repo: str,
-    index: str,
-    sha: str,
-    title: str,
-    model: str,
-    usage: dict | None,
-    findings: list[dict] | None = None,
-    summary: str = "",
-    engine: str = "opencode",
-    tier: str = "",
-    lenses: list[str] | None = None,
-    trace_id: str | None = None,
-    release: str = "",
-    price_target: str | None = None,
-    dropped_count: float | None = None,
-) -> list[dict]:
-    """The ingestion batch for one review: a trace, a generation, and scores.
-
-    Split out from `emit_review_trace` so the shape is testable without a
-    Langfuse to POST to.
-
-    `dropped_count` is how many findings the parser rejected for an unusable
-    `path`/`line`, measured where the model output was parsed. Passing it turns
-    on the `dropped_findings` score; leaving it `None` omits that score rather
-    than reporting a zero the caller never measured.
-    """
-    usage = usage or {}
-    tid = trace_id or str(uuid.uuid4())
-    ts = _now_iso()
-    env = resolve_environment(model)
-    duration = float(usage.get("duration_s") or 0.0)
-    started = datetime.fromtimestamp(
-        time.time() - duration, tz=timezone.utc
-    ).isoformat().replace("+00:00", "Z")
-
-    tags = [
-        f"provider:{provider_of(model)}",
-        f"model:{strip_provider(model)}",
-        f"engine:{engine}",
-        f"repo:{repo}",
-    ]
-    if tier:
-        tags.append(f"tier:{tier}")
-    for lens in lenses or []:
-        tags.append(f"lens:{lens}")
-
-    costs, cost_basis = _cost_details(usage, model, price_target) if usage else ({}, "")
-    if cost_basis:
-        # Filterable in Langfuse, so an equivalent-cost chart can never be
-        # mistaken for money actually spent.
-        tags.append(f"cost:{cost_basis}")
-
-    metadata = {
-        "repo": repo,
-        "pr": index,
-        "sha": sha,
-        "engine": engine,
-        "steps": usage.get("steps"),
-        "duration_s": duration or None,
-        "findings": len(findings or []),
-        "severities": _severity_counts(findings),
-        "provider_cost_usd": usage.get("cost"),
-        "cost_basis": cost_basis or None,
-    }
-    if lenses:
-        metadata["lenses"] = lenses
-    if tier:
-        metadata["tier"] = tier
-    metadata = {k: v for k, v in metadata.items() if v not in (None, {}, [])}
-
-    trace_body = {
-        "id": tid,
-        "name": "pr-review",
-        "timestamp": ts,
-        "environment": env,
-        "sessionId": f"{repo}#{index}",
-        "input": _review_input(repo, index, sha, title),
-        "output": _review_output(summary, findings),
-        "metadata": metadata,
-        "tags": tags,
-    }
-    if release:
-        trace_body["release"] = release
-
-    events = [
-        {
-            "id": str(uuid.uuid4()),
-            "type": "trace-create",
-            "timestamp": ts,
-            "body": trace_body,
-        }
-    ]
-
-    if usage:
-        gen_body = {
-            "id": str(uuid.uuid4()),
-            "traceId": tid,
-            "type": "GENERATION",
-            "name": f"{engine}-review",
-            "environment": env,
-            "startTime": started,
-            "endTime": ts,
-            "model": strip_provider(model),
-            "usageDetails": _usage_details(usage),
-            "metadata": metadata,
-            "level": "DEFAULT",
-            # Repeated from the trace on purpose: an evaluator's variable
-            # mapping reads the *observation's* input/output, so a generation
-            # left blank cannot be judged at all.
-            "input": _review_input(repo, index, sha, title),
-            "output": _review_output(summary, findings),
-        }
-        if costs:
-            gen_body["costDetails"] = costs
-        events.append(
-            {
-                "id": str(uuid.uuid4()),
-                "type": "generation-create",
-                "timestamp": ts,
-                "body": gen_body,
-            }
-        )
-
-    events.extend(
-        _score_events(
-            trace_id=tid,
-            findings=findings,
-            environment=env,
-            cost_usd=costs.get("total"),
-            dropped_count=dropped_count,
-            timestamp=ts,
-            cost_basis=cost_basis,
-        )
-    )
-
-    return events
-
-
-MAX_JUDGED_FINDINGS = 25
-_FIELD_CAP = 600
-
-
-def _review_input(repo: str, index, sha: str, title: str) -> dict:
-    return {"repo": repo, "pr": index, "sha": sha, "title": title}
-
-
-def _review_output(summary: str, findings) -> dict:
-    """What the reviewer actually said, in a shape an evaluator can read.
-
-    The findings themselves are included, not just their count. A judge given
-    only `{"summary": ..., "findings": 3}` can say nothing about whether those
-    three findings are specific, actionable, or consistent with the summary —
-    which is the whole question worth asking of a reviewer that has no ground
-    truth to check against.
-
-    Capped rather than complete: this rides in every ingestion batch, and a
-    review with 80 findings would push the payload past what is reasonable to
-    store per trace. `finding_count` stays exact so nothing reading the count
-    is misled by the cap.
-    """
-    items = list(findings or [])
-    return {
-        "summary": summary[:2000],
-        "finding_count": len(items),
-        "findings_truncated": len(items) > MAX_JUDGED_FINDINGS,
-        "findings": [
-            {
-                "path": f.get("path"),
-                "line": f.get("line"),
-                "severity": f.get("severity"),
-                "problem": str(f.get("problem") or "")[:_FIELD_CAP],
-                "fix": str(f.get("fix") or "")[:_FIELD_CAP],
-            }
-            for f in items[:MAX_JUDGED_FINDINGS]
-        ],
-    }
-
-
-def _score_events(*, cost_basis: str, **kwargs) -> list[dict]:
-    """Deterministic scores for this review, or [] if the scorer is missing.
-
-    Local import + blanket except for the same reason the rest of this module
-    swallows: `eval_scores` is optional, and a scoring bug must not cost the
-    trace it was supposed to annotate.
-    """
-    try:
-        import eval_scores
-
-        # The cost score is only meaningful next to its basis — a $/finding
-        # figure computed from an equivalent price is not money that was spent.
-        comment = f"cost basis: {cost_basis}" if cost_basis else ""
-        return eval_scores.build_scores(comment=comment, **kwargs)
-    except Exception as e:  # pragma: no cover - defensive
-        _debug(f"scoring failed: {e}")
-        return []
-
-
-def _post(host: str, pk: str, sk: str, batch: list[dict], timeout: float) -> int:
-    payload = json.dumps({"batch": batch}).encode("utf-8")
-    auth = base64.b64encode(f"{pk}:{sk}".encode("utf-8")).decode("ascii")
-    req = urllib.request.Request(
-        host + INGESTION_PATH,
-        data=payload,
-        headers={
-            "Content-Type": "application/json",
-            "Authorization": f"Basic {auth}",
-            "User-Agent": "pragent-pilot/1.0",
-        },
-        method="POST",
-    )
-    with urllib.request.urlopen(req, timeout=timeout) as resp:
-        _warn_on_rejected_events(resp.read())
-        return resp.status
-
-
-def _warn_on_rejected_events(raw: bytes) -> None:
-    """Surface per-event rejections hiding inside a 207.
-
-    The ingestion endpoint answers 207 Multi-Status when *some* events failed,
-    so a caller that only checks the status code reads a batch where every
-    single event was rejected as a success. That failure mode is invisible
-    exactly when it matters — the traces simply never appear.
-    """
-    try:
-        body = json.loads(raw or b"{}")
-        errors = body.get("errors") or []
-        if errors:
-            first = errors[0]
-            _debug(
-                f"{len(errors)} event(s) rejected by ingestion; "
-                f"first: status={first.get('status')} {first.get('error')}"
-            )
-    except Exception:  # pragma: no cover - never let logging break emission
-        pass
-
-
-def emit_review_trace(**kwargs) -> bool:
-    """Ship one review's trace. Returns True if Langfuse accepted it.
-
-    No-op (False) when Langfuse is unconfigured. Never raises — a telemetry
-    outage must not turn into a failed review.
-    """
-    conf = _enabled()
-    if conf is None:
-        return False
-    host, pk, sk = conf
-    try:
-        timeout = float(os.environ.get("LANGFUSE_TIMEOUT", "5"))
-    except ValueError:
-        timeout = 5.0
-    try:
-        batch = build_batch(**kwargs)
-        status = _post(host, pk, sk, batch, timeout)
-        if status not in (200, 201, 207):
-            _debug(f"ingestion returned HTTP {status}")
-            return False
-        return True
-    except urllib.error.HTTPError as e:
-        _debug(f"ingestion HTTP {e.code}: {e.read()[:300]!r}")
-    except Exception as e:
-        _debug(f"ingestion failed: {e}")
-    return False
+_module = importlib.import_module("observability.langfuse")
+sys.modules[__name__] = _module

pilot/model_client.py

@@ -0,0 +1,5 @@
+"""Compatibility import for the legacy model adapter."""
+import importlib
+import sys
+_module = importlib.import_module("review.model")
+sys.modules[__name__] = _module

pilot/observability/__init__.py

@@ -0,0 +1 @@
+"""Cost modeling and Langfuse telemetry."""

pilot/observability/cost.py

@@ -0,0 +1,434 @@
+#!/usr/bin/env python3
+"""pragent pilot — per-review cost model.
+
+Answers "what would this cost on a paid API?" for the pilot's agent loop. The
+pilot currently runs on `glm-5.2:cloud` through the on-network headroom proxy at
+no per-token charge, so every review's measured usage is *free but real*: it
+tells us exactly what the same work would bill on Claude or GPT.
+
+The model is deliberately explicit rather than a single fudge factor, because
+the dominant cost in an agent loop is not the diff — it is **resending the
+conversation on every step**. A 12-step review re-reads its own prefix 12 times.
+Prompt caching is what makes that affordable, and whether caching is on changes
+the answer by ~3x, so it's a parameter, not an assumption.
+
+Token accounting per review:
+
+    step 1 input  = prefix + brief
+    step k input  = prefix + brief + (tool results accumulated through k-1)
+    total input   = sum over steps
+    cached        = the prefix + brief part of steps 2..n (stable, byte-identical)
+    uncached      = step 1 in full + the growing tool-result tail
+
+`prefix` = system + tool schemas + agent definition + the skills this tier loads.
+Those sizes are MEASURED from the files in this repo (see `measure_factory`),
+not guessed. Diff size, file reads, and step count are per-tier assumptions from
+the `attention-tiering` skill's budgets — override them on the CLI to fit your
+own repos.
+
+Prices are per million tokens, from the providers' published pricing pages
+(fetched 2026-08-18 — re-check before quoting):
+  https://platform.claude.com/docs/en/about-claude/pricing
+  https://developers.openai.com/api/docs/pricing
+
+Usage:
+    python3 pilot/cost_model.py                       # all tiers, all models
+    python3 pilot/cost_model.py --prs-per-month 350
+    python3 pilot/cost_model.py --mix 5,35,55,5       # trivial,lite,full,oversized %
+    python3 pilot/cost_model.py --no-cache            # what caching is worth
+"""
+
+from __future__ import annotations
+
+import argparse
+import os
+from dataclasses import dataclass, field
+
+CHARS_PER_TOKEN = 4  # English prose/code rule of thumb; ±15% is normal
+
+
+# ---------------------------------------------------------------------------
+# Prices — USD per million tokens
+# ---------------------------------------------------------------------------
+
+
+@dataclass(frozen=True)
+class Price:
+    """Per-MTok prices. `cache_write` and `cache_read` are absolute rates, not
+    multipliers, so providers with different cache economics stay comparable.
+
+    `provider` is the opencode provider name (`headroom`, `vllm-qwen38`, ...). It
+    doubles as the dispatch key for `.pr-review.json:model` overrides — when
+    a per-repo override is set, `_resolve_display_model` returns
+    `f"{provider}/{key}"` so the opencode subprocess routes correctly.
+    Default `headroom` preserved for the existing roster."""
+
+    name: str
+    input: float
+    output: float
+    cache_write: float
+    cache_read: float
+    provider: str = "headroom"
+
+    @property
+    def batch_input(self) -> float:
+        return self.input / 2
+
+    @property
+    def batch_output(self) -> float:
+        return self.output / 2
+
+
+# Anthropic: cache write = 1.25x input (5-minute TTL), cache read = 0.1x input.
+# OpenAI: cached input is a published rate (0.1x input); there is no separate
+# cache-write charge — writes are billed as ordinary input.
+PRICES: dict[str, Price] = {
+    "claude-opus-5": Price("Claude Opus 5", 5.00, 25.00, 6.25, 0.50),
+    "claude-sonnet-5": Price("Claude Sonnet 5", 2.00, 10.00, 2.50, 0.20),
+    "claude-haiku-4-5": Price("Claude Haiku 4.5", 1.00, 5.00, 1.25, 0.10),
+    "gpt-5.6-sol": Price("GPT-5.6 Sol", 5.00, 30.00, 5.00, 0.50),
+    "gpt-5.6-terra": Price("GPT-5.6 Terra", 2.00, 12.00, 2.00, 0.20),
+    "gpt-5.6-luna": Price("GPT-5.6 Luna", 0.20, 1.20, 0.20, 0.02),
+    # OpenAI — cached_input 0.1x, no separate cache_write
+    "gpt-5": Price("GPT-5", 1.25, 10.00, 1.25, 0.125),
+    "gpt-5-mini": Price("GPT-5 mini", 0.25, 2.00, 0.25, 0.025),
+    # Google Gemini — cache_write = input
+    "gemini-2.5-pro": Price("Gemini 2.5 Pro", 1.875, 12.50, 1.875, 0.1875),
+    "gemini-2.5-flash": Price("Gemini 2.5 Flash", 0.30, 2.50, 0.30, 0.03),
+    # xAI Grok — cache_write = input
+    "grok-4.5": Price("Grok 4.5", 2.00, 6.00, 2.00, 0.30),
+    "grok-4.3": Price("Grok 4.3", 1.25, 2.50, 1.25, 0.20),
+    # Self-hosted — AI workstation RTX 3090, vLLM + DFlash2 spec-decode, no
+    # per-token charge. provider="vllm-qwen38" so the opencode subprocess
+    # routes via the matching provider block in opencode.json
+    # (baseURL=http://192.168.1.79:18020/v1). Equivalent-cost column reads $0
+    # — the cost-comparison signal is that the same work would bill $X on a
+    # paid model.
+    "qwen3.8-27b": Price("Qwen3.8-27B (vLLM, MTP, 150k ctx)", 0.0, 0.0, 0.0, 0.0, provider="vllm-qwen38"),
+}
+
+
+# ---------------------------------------------------------------------------
+# Factory footprint — measured from this repo
+# ---------------------------------------------------------------------------
+
+_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+
+# Skills the primary always loads, and the conditional ones per tier. Mirrors
+# the load table in .opencode/agents/pragent.md.
+ALWAYS_SKILLS = ("review-methodology", "findings-schema", "attention-tiering")
+TIER_SKILLS: dict[str, tuple[str, ...]] = {
+    "trivial": (),
+    "lite": ("comment-craft",),
+    "full": ("linter-playbook", "security-lens", "comment-craft"),
+    "oversized": ("linter-playbook", "security-lens", "comment-craft", "malicious-change"),
+}
+
+# opencode's own system prompt + the JSON tool schemas it sends (read, grep,
+# glob, bash, webfetch, skill, task, …). Not in this repo, so this is the one
+# component that is an estimate rather than a measurement.
+HARNESS_TOKENS = 3500
+
+
+def _tok(path: str) -> int:
+    try:
+        with open(path, "rb") as f:
+            return len(f.read()) // CHARS_PER_TOKEN
+    except OSError:
+        return 0
+
+
+def measure_factory(root: str = _ROOT) -> dict[str, int]:
+    """Token size of each prompt component, measured from the files on disk."""
+    out = {"agent": _tok(os.path.join(root, ".opencode", "agents", "pragent.md"))}
+    skills_dir = os.path.join(root, ".opencode", "skills")
+    if os.path.isdir(skills_dir):
+        for name in sorted(os.listdir(skills_dir)):
+            p = os.path.join(skills_dir, name, "SKILL.md")
+            if os.path.isfile(p):
+                out[f"skill:{name}"] = _tok(p)
+    for lens in ("security", "tests", "perf"):
+        out[f"subagent:{lens}"] = _tok(os.path.join(root, ".opencode", "agents", f"{lens}.md"))
+    return out
+
+
+def prefix_tokens(tier: str, factory: dict[str, int]) -> int:
+    """Stable per-step prefix: harness + agent definition + loaded skills."""
+    total = HARNESS_TOKENS + factory.get("agent", 0)
+    for s in ALWAYS_SKILLS + TIER_SKILLS.get(tier, ()):
+        total += factory.get(f"skill:{s}", 0)
+    return total
+
+
+# ---------------------------------------------------------------------------
+# Per-tier workload assumptions
+# ---------------------------------------------------------------------------
+
+
+@dataclass
+class Tier:
+    """One tier's workload. Defaults follow the `attention-tiering` budgets."""
+
+    name: str
+    diff_tokens: int          # the diff as it lands in the brief
+    steps: int                # model turns in the agent loop
+    file_reads: int           # files read from the checkout
+    tokens_per_read: int      # avg tokens returned per read/grep/linter result
+    output_tokens: int        # assistant output across all steps (incl. reasoning)
+    subagents: int = 0        # lens subagents spawned
+    brief_fixed: int = 600    # brief template + PR meta + prior reviews
+    share: float = 0.0        # fraction of PRs at this tier (for the monthly mix)
+    _factory: dict = field(default_factory=dict, repr=False)
+
+
+DEFAULT_TIERS = [
+    # diff_tok  steps  reads  tok/read  output  subs  share
+    Tier("trivial",     400,  2,  0,     0,      600,  0, share=0.05),
+    Tier("lite",       1500,  6,  4,  2000,     2500,  0, share=0.35),
+    Tier("full",       6000, 24, 20,  3300,    12000,  0, share=0.55),
+    Tier("oversized", 25000, 35, 30,  3500,    20000,  2, share=0.05),
+]
+
+
+# ---------------------------------------------------------------------------
+# Observed runs — the calibration anchor
+# ---------------------------------------------------------------------------
+
+# Real usage reported by opencode's step_finish events. Keep this list
+# events. Keep this list append-only: it is the only thing separating this model
+# from a guess, and the first entry corrected the tier assumptions by ~15x.
+OBSERVED_RUNS: list[dict] = [
+    {
+        "label": "internal/hardening-PR (16 files, 1020 insertions / 91 deletions)",
+        "date": "2026-08-18",
+        "tier": "full",
+        "diff_tokens": 17_600,     # 16 files, 1020 insertions / 91 deletions
+        "steps": 28,
+        "duration_s": 348.3,
+        "input": 2_071_025,
+        "output": 17_303,
+        "cache_read": 0,
+        "cache_write": 0,
+        "subagents": 0,
+    },
+    {
+        "label": "internal/hardening-PR (same PR, two commits later)",
+        "date": "2026-08-18",
+        "tier": "full",
+        "diff_tokens": 21_000,     # same PR, two commits later
+        "steps": 31,
+        "duration_s": 189.8,
+        "input": 2_213_077,
+        "output": 9_058,
+        "cache_read": 0,
+        "cache_write": 0,
+        "subagents": 0,
+    },
+    # A third run of the same PR (sha 2613b3e, 31 steps' worth of work in 330s)
+    # ended without a parseable findings block and so reported no usage at all —
+    # the reason `salvage_summary` now keeps the usage section on that path.
+]
+
+
+def observed_usage(run: dict) -> Usage:
+    return Usage(
+        uncached_input=run["input"] - run.get("cache_read", 0),
+        cached_input=run.get("cache_read", 0),
+        cache_writes=run.get("cache_write", 0),
+        output=run["output"],
+    )
+
+
+@dataclass
+class Usage:
+    uncached_input: int = 0
+    cached_input: int = 0
+    cache_writes: int = 0
+    output: int = 0
+
+    @property
+    def total_input(self) -> int:
+        return self.uncached_input + self.cached_input
+
+
+def tier_usage(tier: Tier, factory: dict[str, int], caching: bool = True) -> Usage:
+    """Token usage for one review at this tier.
+
+    The agent loop resends the whole conversation each step. The prefix + brief
+    are byte-identical across steps, so with caching they are written once and
+    read back on every later step; the tool-result tail grows and is charged as
+    ordinary input. Without caching every step pays full input price for
+    everything it has accumulated — which is the quadratic term that makes an
+    uncached agent loop expensive.
+    """
+    prefix = prefix_tokens(tier.name, factory)
+    stable = prefix + tier.brief_fixed + tier.diff_tokens
+
+    # Tool results arrive one per step, after the first.
+    result_steps = max(0, min(tier.file_reads, tier.steps - 1))
+    per_result = tier.tokens_per_read
+
+    u = Usage(output=tier.output_tokens)
+
+    if caching:
+        u.cache_writes = stable
+        u.cached_input = stable * max(0, tier.steps - 1)
+        u.uncached_input = 0
+    else:
+        u.uncached_input = stable * tier.steps
+
+    # The growing tail of tool results: a result produced at step i is resent on
+    # every step after it, so it is counted (steps - i) times.
+    tail = 0
+    for i in range(1, result_steps + 1):
+        tail += per_result * (tier.steps - i)
+    u.uncached_input += tail
+
+    # Each lens subagent is its own loop: its own prefix, the diff, a few reads.
+    for _ in range(tier.subagents):
+        sub_prefix = HARNESS_TOKENS + factory.get("subagent:security", 600)
+        sub_stable = sub_prefix + tier.diff_tokens
+        sub_steps = 6
+        if caching:
+            u.cache_writes += sub_stable
+            u.cached_input += sub_stable * (sub_steps - 1)
+        else:
+            u.uncached_input += sub_stable * sub_steps
+        for i in range(1, 4):
+            u.uncached_input += per_result * (sub_steps - i)
+        u.output += 1500
+
+    return u
+
+
+def cost(u: Usage, price: Price, batch: bool = False) -> float:
+    """USD for one review's usage at these prices."""
+    inp = price.batch_input if batch else price.input
+    out = price.batch_output if batch else price.output
+    cw = price.cache_write / 2 if batch else price.cache_write
+    cr = price.cache_read / 2 if batch else price.cache_read
+    return (
+        u.uncached_input * inp
+        + u.cached_input * cr
+        + u.cache_writes * cw
+        + u.output * out
+    ) / 1_000_000
+
+
+# ---------------------------------------------------------------------------
+# Reporting
+# ---------------------------------------------------------------------------
+
+
+def blended_cost(tiers: list[Tier], factory: dict, price: Price, caching: bool) -> float:
+    """Weighted cost of one average PR across the tier mix."""
+    total_share = sum(t.share for t in tiers) or 1.0
+    return sum(
+        cost(tier_usage(t, factory, caching), price) * (t.share / total_share)
+        for t in tiers
+    )
+
+
+def report(tiers: list[Tier], prs_per_month: int, caching: bool, models: list[str]) -> str:
+    factory = measure_factory()
+    lines: list[str] = []
+
+    lines.append(f"Factory footprint (measured, {CHARS_PER_TOKEN} chars/token):")
+    for k, v in sorted(factory.items()):
+        lines.append(f"  {k:<34} {v:>6,} tok")
+    lines.append(f"  {'harness (opencode + tool schemas, est.)':<34} {HARNESS_TOKENS:>6,} tok")
+    lines.append("")
+
+    lines.append(f"Per-review tokens  (prompt caching: {'on' if caching else 'OFF'})")
+    lines.append(f"  {'tier':<11} {'prefix':>8} {'uncached':>10} {'cached':>10} {'cwrite':>8} {'output':>8}")
+    for t in tiers:
+        u = tier_usage(t, factory, caching)
+        lines.append(
+            f"  {t.name:<11} {prefix_tokens(t.name, factory):>8,} {u.uncached_input:>10,} "
+            f"{u.cached_input:>10,} {u.cache_writes:>8,} {u.output:>8,}"
+        )
+    lines.append("")
+
+    lines.append("Cost per review (USD)")
+    header = f"  {'model':<18}" + "".join(f"{t.name:>12}" for t in tiers) + f"{'blended':>12}"
+    lines.append(header)
+    for key in models:
+        p = PRICES[key]
+        row = f"  {p.name:<18}"
+        for t in tiers:
+            row += f"{cost(tier_usage(t, factory, caching), p):>12.4f}"
+        row += f"{blended_cost(tiers, factory, p, caching):>12.4f}"
+        lines.append(row)
+    lines.append("")
+
+    mix = ", ".join(f"{t.name} {t.share:.0%}" for t in tiers)
+    lines.append(f"Monthly at {prs_per_month} PRs/month  (mix: {mix})")
+    lines.append(f"  {'model':<18} {'per PR':>10} {'per month':>12} {'batch -50%':>12}")
+    for key in models:
+        p = PRICES[key]
+        per_pr = blended_cost(tiers, factory, p, caching)
+        lines.append(
+            f"  {p.name:<18} {per_pr:>10.4f} {per_pr * prs_per_month:>12.2f}"
+            f" {per_pr * prs_per_month / 2:>12.2f}"
+        )
+    lines.append("")
+    lines.append("Batch column applies the 50% async discount; it is shown for scale only —")
+    lines.append("PR review is latency-sensitive and a stateful agent loop is not batchable.")
+    lines.append("")
+    lines.append(observed_report(models))
+    return "\n".join(lines)
+
+
+def observed_report(models: list[str]) -> str:
+    """Price the runs actually measured through the opencode usage telemetry."""
+    if not OBSERVED_RUNS:
+        return "No observed runs recorded yet."
+    lines = ["Observed runs (measured via opencode step_finish events)"]
+    for run in OBSERVED_RUNS:
+        u = observed_usage(run)
+        lines.append(
+            f"  {run['label']} — tier {run['tier']}, {run['steps']} steps, "
+            f"{run['duration_s']:.0f}s, {run['input']:,} in / {run['output']:,} out, "
+            f"cache {run['cache_read']:,} read / {run['cache_write']:,} write"
+        )
+        row = "   "
+        for key in models:
+            p = PRICES[key]
+            row += f" {p.name}: ${cost(u, p):.2f} "
+        lines.append(row)
+    lines.append("")
+    lines.append("  NOTE: the pilot's headroom/glm-5.2 path reports zero cache read and zero")
+    lines.append("  cache write, i.e. prompt caching is NOT in play today. On a provider where")
+    lines.append("  it is, the stable prefix (agent + skills + brief + diff, resent every step)")
+    lines.append("  drops to 0.1x — worth roughly a third of the bill on a run like the one")
+    lines.append("  above. Budget with caching OFF until the measured cache columns are nonzero.")
+    return "\n".join(lines)
+
+
+def main(argv: list[str] | None = None) -> int:
+    ap = argparse.ArgumentParser(description="pragent per-review cost model")
+    ap.add_argument("--prs-per-month", type=int, default=350)
+    ap.add_argument("--mix", default="", help="trivial,lite,full,oversized as percentages")
+    ap.add_argument("--no-cache", action="store_true", help="model without prompt caching")
+    ap.add_argument("--models", default=",".join(PRICES))
+    args = ap.parse_args(argv)
+
+    tiers = DEFAULT_TIERS
+    if args.mix:
+        shares = [float(x) for x in args.mix.split(",")]
+        if len(shares) != len(tiers):
+            ap.error(f"--mix needs {len(tiers)} comma-separated values")
+        for t, s in zip(tiers, shares):
+            t.share = s / 100.0
+
+    models = [m.strip() for m in args.models.split(",") if m.strip()]
+    unknown = [m for m in models if m not in PRICES]
+    if unknown:
+        ap.error(f"unknown model(s): {', '.join(unknown)}")
+
+    print(report(tiers, args.prs_per_month, not args.no_cache, models))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

pilot/observability/langfuse.py

@@ -0,0 +1,468 @@
+#!/usr/bin/env python3
+"""pragent pilot — Langfuse trace emission.
+
+Ships one trace per PR review to a self-hosted Langfuse (v3) so the reviewer's
+token spend, latency and per-model behaviour are queryable outside the review
+body. The review body already renders a usage table; that table is per-PR and
+disappears into Gitea. This is the same numbers, aggregated.
+
+Why hand-rolled instead of the `langfuse` SDK: the pilot image is stdlib-only
+(see pilot/Dockerfile — no requirements.txt anywhere in the repo), and the
+ingestion API is a single authenticated POST of a JSON batch. Pulling an SDK
+plus its otel dependency tree into a fail-open telemetry side-path is a bad
+trade.
+
+Provider split
+--------------
+`environment` on every trace is either `ollama` or `claude`, derived from the
+resolved display model (`resolve_environment`). That is what keeps the two
+spend stories separate in Langfuse: every view, filter and cost breakdown
+takes an environment selector, so "what did the local/self-hosted path cost"
+and "what did the Claude path cost" are two views of one project rather than
+two projects with two key pairs to rotate. Tags carry the finer split
+(`provider:headroom`, `model:...`, `engine:opencode`).
+
+Cost
+----
+The pilot's own path bills $0 (headroom proxy, no per-token charge), so the
+`cost` reported to Langfuse is the *equivalent* cost from `cost_model` — what
+the same tokens would bill on the comparison model. That is the number worth
+trending; a chart of $0.00 is not.
+
+A model is "free" when `cost_model.PRICES` has no entry for it (MiniMax-M2.7,
+glm-5.2:cloud) or when its entry is all zeros (the self-hosted vLLM qwen). In
+both cases the reported cost is priced against the comparison target instead —
+same precedence the review body uses: `.pr-review.json:cost_target` >
+`PRAGENT_PRICE_TARGET` > `claude-sonnet-5`. A paid model is priced as itself.
+
+Because a hypothetical and a real charge must never be read as the same
+number, every trace is tagged `cost:actual` or `cost:equivalent:<target>`, and
+the generation's metadata carries `cost_basis`.
+
+Fail-open: every entry point swallows its own exceptions. Telemetry must never
+cost a review.
+
+Env:
+  LANGFUSE_HOST         e.g. http://langfuse-web.langfuse.svc.cluster.local:3000
+  LANGFUSE_PUBLIC_KEY   pk-lf-...
+  LANGFUSE_SECRET_KEY   sk-lf-...
+  LANGFUSE_TIMEOUT      seconds, default 5
+  LANGFUSE_DEBUG        1 to log ingestion failures to stderr
+Disabled (silently) when host or either key is unset.
+"""
+
+from __future__ import annotations
+
+import base64
+import json
+import os
+import sys
+import time
+import urllib.error
+import urllib.request
+import uuid
+from datetime import datetime, timezone
+
+INGESTION_PATH = "/api/public/ingestion"
+
+# Model-key prefixes that mean "this review ran against Anthropic-shaped
+# billing". Everything else (glm, MiniMax, qwen, local vLLM) is the ollama /
+# self-hosted side of the split.
+_CLAUDE_PREFIXES = ("claude-", "anthropic/")
+
+
+def _now_iso() -> str:
+    return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
+
+
+def _enabled() -> tuple[str, str, str] | None:
+    host = (os.environ.get("LANGFUSE_HOST") or "").strip().rstrip("/")
+    pk = (os.environ.get("LANGFUSE_PUBLIC_KEY") or "").strip()
+    sk = (os.environ.get("LANGFUSE_SECRET_KEY") or "").strip()
+    if not host or not pk or not sk:
+        return None
+    return host, pk, sk
+
+
+def _debug(msg: str) -> None:
+    if os.environ.get("LANGFUSE_DEBUG"):
+        print(f"pragent/langfuse: {msg}", file=sys.stderr, flush=True)
+
+
+def strip_provider(model: str) -> str:
+    """`headroom/claude-sonnet-5` -> `claude-sonnet-5`. Bare names pass through."""
+    return model.split("/", 1)[1] if "/" in model else model
+
+
+def provider_of(model: str) -> str:
+    """The opencode provider block a display model routes through."""
+    return model.split("/", 1)[0] if "/" in model else "headroom"
+
+
+def resolve_environment(model: str) -> str:
+    """Which spend story this review belongs to: `claude` or `ollama`.
+
+    Keyed off the bare model name, not the provider, because both paths route
+    through the same `headroom` proxy — `headroom/claude-sonnet-5` is Claude
+    spend, `headroom/glm-5.2:cloud` is not.
+    """
+    bare = strip_provider(model).lower()
+    return "claude" if bare.startswith(_CLAUDE_PREFIXES) else "ollama"
+
+
+def _usage_details(usage: dict) -> dict:
+    """opencode's usage dict -> Langfuse `usageDetails`.
+
+    Langfuse sums every key except the ones it knows are derived, so `input`
+    here is the *uncached* portion: reporting both `input` (which opencode
+    reports as the full input, cache included) and `cache_read_input_tokens`
+    would double-count.
+    """
+    inp = int(usage.get("input") or 0)
+    cache_read = int(usage.get("cache_read") or 0)
+    cache_write = int(usage.get("cache_write") or 0)
+    details = {
+        "input": max(0, inp - cache_read),
+        "output": int(usage.get("output") or 0),
+    }
+    if cache_read:
+        details["cache_read_input_tokens"] = cache_read
+    if cache_write:
+        details["cache_write_input_tokens"] = cache_write
+    reasoning = int(usage.get("reasoning") or 0)
+    if reasoning:
+        details["reasoning"] = reasoning
+    return details
+
+
+DEFAULT_PRICE_TARGET = "claude-sonnet-5"
+
+
+def resolve_price_target(price_target: str | None = None) -> str:
+    """The model to price free/unknown runs against.
+
+    Mirrors `ai_review._resolve_price_target`: an explicit target (which the
+    caller reads from `.pr-review.json:cost_target`) wins, then
+    `PRAGENT_PRICE_TARGET`, then Sonnet.
+    """
+    if price_target and price_target.strip():
+        return price_target.strip()
+    env = os.environ.get("PRAGENT_PRICE_TARGET", "").strip()
+    return env or DEFAULT_PRICE_TARGET
+
+
+def _is_free(price) -> bool:
+    """A price entry that charges nothing — self-hosted or proxied at no cost."""
+    return price.input == 0 and price.output == 0
+
+
+def _cost_details(usage: dict, model: str, price_target: str | None = None) -> tuple[dict, str]:
+    """USD for this usage plus the basis it was computed on.
+
+    Returns `({"total": …}, basis)` where basis is `actual` for a model that
+    genuinely bills, or `equivalent:<target>` for one that does not. `({}, "")`
+    when nothing can be priced at all — better no number than a wrong one.
+
+    Local import + broad except: `cost_model` is only present on the opencode
+    path, and an unknown model key must not break telemetry.
+    """
+    try:
+        from cost_model import PRICES, Usage, cost
+
+        bare = strip_provider(model)
+        price = PRICES.get(bare)
+        basis = "actual"
+        if price is None or _is_free(price):
+            # MiniMax / glm / self-hosted qwen: $0 through the proxy, so the
+            # useful number is what these tokens would have billed elsewhere.
+            target = resolve_price_target(price_target)
+            price = PRICES.get(target)
+            if price is None:
+                _debug(f"comparison target {target!r} not in PRICES")
+                return {}, ""
+            basis = f"equivalent:{target}"
+
+        u = Usage(
+            uncached_input=max(0, int(usage.get("input") or 0) - int(usage.get("cache_read") or 0)),
+            cached_input=int(usage.get("cache_read") or 0),
+            cache_writes=int(usage.get("cache_write") or 0),
+            output=int(usage.get("output") or 0),
+        )
+        return {"total": round(cost(u, price), 6)}, basis
+    except Exception as e:  # pragma: no cover - defensive
+        _debug(f"cost lookup failed for {model!r}: {e}")
+        return {}, ""
+
+
+def _severity_counts(findings: list[dict] | None) -> dict:
+    counts: dict[str, int] = {}
+    for f in findings or []:
+        sev = str(f.get("severity") or "unknown").lower()
+        counts[sev] = counts.get(sev, 0) + 1
+    return counts
+
+
+def build_batch(
+    *,
+    repo: str,
+    index: str,
+    sha: str,
+    title: str,
+    model: str,
+    usage: dict | None,
+    findings: list[dict] | None = None,
+    summary: str = "",
+    engine: str = "opencode",
+    tier: str = "",
+    lenses: list[str] | None = None,
+    trace_id: str | None = None,
+    release: str = "",
+    price_target: str | None = None,
+    dropped_count: float | None = None,
+) -> list[dict]:
+    """The ingestion batch for one review: a trace, a generation, and scores.
+
+    Split out from `emit_review_trace` so the shape is testable without a
+    Langfuse to POST to.
+
+    `dropped_count` is how many findings the parser rejected for an unusable
+    `path`/`line`, measured where the model output was parsed. Passing it turns
+    on the `dropped_findings` score; leaving it `None` omits that score rather
+    than reporting a zero the caller never measured.
+    """
+    usage = usage or {}
+    tid = trace_id or str(uuid.uuid4())
+    ts = _now_iso()
+    env = resolve_environment(model)
+    duration = float(usage.get("duration_s") or 0.0)
+    started = datetime.fromtimestamp(
+        time.time() - duration, tz=timezone.utc
+    ).isoformat().replace("+00:00", "Z")
+
+    tags = [
+        f"provider:{provider_of(model)}",
+        f"model:{strip_provider(model)}",
+        f"engine:{engine}",
+        f"repo:{repo}",
+    ]
+    if tier:
+        tags.append(f"tier:{tier}")
+    for lens in lenses or []:
+        tags.append(f"lens:{lens}")
+
+    costs, cost_basis = _cost_details(usage, model, price_target) if usage else ({}, "")
+    if cost_basis:
+        # Filterable in Langfuse, so an equivalent-cost chart can never be
+        # mistaken for money actually spent.
+        tags.append(f"cost:{cost_basis}")
+
+    metadata = {
+        "repo": repo,
+        "pr": index,
+        "sha": sha,
+        "engine": engine,
+        "steps": usage.get("steps"),
+        "duration_s": duration or None,
+        "findings": len(findings or []),
+        "severities": _severity_counts(findings),
+        "provider_cost_usd": usage.get("cost"),
+        "cost_basis": cost_basis or None,
+    }
+    if lenses:
+        metadata["lenses"] = lenses
+    if tier:
+        metadata["tier"] = tier
+    metadata = {k: v for k, v in metadata.items() if v not in (None, {}, [])}
+
+    trace_body = {
+        "id": tid,
+        "name": "pr-review",
+        "timestamp": ts,
+        "environment": env,
+        "sessionId": f"{repo}#{index}",
+        "input": _review_input(repo, index, sha, title),
+        "output": _review_output(summary, findings),
+        "metadata": metadata,
+        "tags": tags,
+    }
+    if release:
+        trace_body["release"] = release
+
+    events = [
+        {
+            "id": str(uuid.uuid4()),
+            "type": "trace-create",
+            "timestamp": ts,
+            "body": trace_body,
+        }
+    ]
+
+    if usage:
+        gen_body = {
+            "id": str(uuid.uuid4()),
+            "traceId": tid,
+            "type": "GENERATION",
+            "name": f"{engine}-review",
+            "environment": env,
+            "startTime": started,
+            "endTime": ts,
+            "model": strip_provider(model),
+            "usageDetails": _usage_details(usage),
+            "metadata": metadata,
+            "level": "DEFAULT",
+            # Repeated from the trace on purpose: an evaluator's variable
+            # mapping reads the *observation's* input/output, so a generation
+            # left blank cannot be judged at all.
+            "input": _review_input(repo, index, sha, title),
+            "output": _review_output(summary, findings),
+        }
+        if costs:
+            gen_body["costDetails"] = costs
+        events.append(
+            {
+                "id": str(uuid.uuid4()),
+                "type": "generation-create",
+                "timestamp": ts,
+                "body": gen_body,
+            }
+        )
+
+    events.extend(
+        _score_events(
+            trace_id=tid,
+            findings=findings,
+            environment=env,
+            cost_usd=costs.get("total"),
+            dropped_count=dropped_count,
+            timestamp=ts,
+            cost_basis=cost_basis,
+        )
+    )
+
+    return events
+
+
+MAX_JUDGED_FINDINGS = 25
+_FIELD_CAP = 600
+
+
+def _review_input(repo: str, index, sha: str, title: str) -> dict:
+    return {"repo": repo, "pr": index, "sha": sha, "title": title}
+
+
+def _review_output(summary: str, findings) -> dict:
+    """What the reviewer actually said, in a shape an evaluator can read.
+
+    The findings themselves are included, not just their count. A judge given
+    only `{"summary": ..., "findings": 3}` can say nothing about whether those
+    three findings are specific, actionable, or consistent with the summary —
+    which is the whole question worth asking of a reviewer that has no ground
+    truth to check against.
+
+    Capped rather than complete: this rides in every ingestion batch, and a
+    review with 80 findings would push the payload past what is reasonable to
+    store per trace. `finding_count` stays exact so nothing reading the count
+    is misled by the cap.
+    """
+    items = list(findings or [])
+    return {
+        "summary": summary[:2000],
+        "finding_count": len(items),
+        "findings_truncated": len(items) > MAX_JUDGED_FINDINGS,
+        "findings": [
+            {
+                "path": f.get("path"),
+                "line": f.get("line"),
+                "severity": f.get("severity"),
+                "problem": str(f.get("problem") or "")[:_FIELD_CAP],
+                "fix": str(f.get("fix") or "")[:_FIELD_CAP],
+            }
+            for f in items[:MAX_JUDGED_FINDINGS]
+        ],
+    }
+
+
+def _score_events(*, cost_basis: str, **kwargs) -> list[dict]:
+    """Deterministic scores for this review, or [] if the scorer is missing.
+
+    Local import + blanket except for the same reason the rest of this module
+    swallows: `eval_scores` is optional, and a scoring bug must not cost the
+    trace it was supposed to annotate.
+    """
+    try:
+        import eval_scores
+
+        # The cost score is only meaningful next to its basis — a $/finding
+        # figure computed from an equivalent price is not money that was spent.
+        comment = f"cost basis: {cost_basis}" if cost_basis else ""
+        return eval_scores.build_scores(comment=comment, **kwargs)
+    except Exception as e:  # pragma: no cover - defensive
+        _debug(f"scoring failed: {e}")
+        return []
+
+
+def _post(host: str, pk: str, sk: str, batch: list[dict], timeout: float) -> int:
+    payload = json.dumps({"batch": batch}).encode("utf-8")
+    auth = base64.b64encode(f"{pk}:{sk}".encode("utf-8")).decode("ascii")
+    req = urllib.request.Request(
+        host + INGESTION_PATH,
+        data=payload,
+        headers={
+            "Content-Type": "application/json",
+            "Authorization": f"Basic {auth}",
+            "User-Agent": "pragent-pilot/1.0",
+        },
+        method="POST",
+    )
+    with urllib.request.urlopen(req, timeout=timeout) as resp:
+        _warn_on_rejected_events(resp.read())
+        return resp.status
+
+
+def _warn_on_rejected_events(raw: bytes) -> None:
+    """Surface per-event rejections hiding inside a 207.
+
+    The ingestion endpoint answers 207 Multi-Status when *some* events failed,
+    so a caller that only checks the status code reads a batch where every
+    single event was rejected as a success. That failure mode is invisible
+    exactly when it matters — the traces simply never appear.
+    """
+    try:
+        body = json.loads(raw or b"{}")
+        errors = body.get("errors") or []
+        if errors:
+            first = errors[0]
+            _debug(
+                f"{len(errors)} event(s) rejected by ingestion; "
+                f"first: status={first.get('status')} {first.get('error')}"
+            )
+    except Exception:  # pragma: no cover - never let logging break emission
+        pass
+
+
+def emit_review_trace(**kwargs) -> bool:
+    """Ship one review's trace. Returns True if Langfuse accepted it.
+
+    No-op (False) when Langfuse is unconfigured. Never raises — a telemetry
+    outage must not turn into a failed review.
+    """
+    conf = _enabled()
+    if conf is None:
+        return False
+    host, pk, sk = conf
+    try:
+        timeout = float(os.environ.get("LANGFUSE_TIMEOUT", "5"))
+    except ValueError:
+        timeout = 5.0
+    try:
+        batch = build_batch(**kwargs)
+        status = _post(host, pk, sk, batch, timeout)
+        if status not in (200, 201, 207):
+            _debug(f"ingestion returned HTTP {status}")
+            return False
+        return True
+    except urllib.error.HTTPError as e:
+        _debug(f"ingestion HTTP {e.code}: {e.read()[:300]!r}")
+    except Exception as e:
+        _debug(f"ingestion failed: {e}")
+    return False

pilot/review/__init__.py

@@ -0,0 +1 @@
+"""Review pipeline modules: orchestration, model adapters, parsing, and diff work."""

pilot/review/adapters.py

@@ -0,0 +1,326 @@
+from __future__ import annotations
+
+import base64
+import json
+import os
+import re
+import sys
+import urllib.error
+import urllib.parse
+import urllib.request
+
+from . import pipeline
+from .pipeline import *
+from .analysis import parse_text_blocks, truncate_diff
+from .configuration import parse_repo_config
+from .output import inline_comment_body, summary_bullets
+
+# Network helpers
+# ---------------------------------------------------------------------------
+
+
+def _http(method: str, url: str, token: str, body: dict | None = None, accept: str = "application/json") -> tuple[int, bytes]:
+    from gitea_client import request
+    return request(method, url, token, body, accept)
+
+
+def gitea_get(api: str, repo: str, path: str, token: str, accept: str = "application/json") -> tuple[int, bytes]:
+    return _http("GET", f"{api}/api/v1/repos/{repo}/{path}", token, None, accept)
+
+
+def gitea_post(api: str, repo: str, path: str, token: str, body: dict) -> tuple[int, bytes]:
+    return _http("POST", f"{api}/api/v1/repos/{repo}/{path}", token, body)
+
+
+# ---------------------------------------------------------------------------
+# Additional context URLs — static repo-provided background fetched once
+# per review and injected into the brief. The idea is the cheap reusable
+# knowledge (architecture summary, module map, conventions, glossary, past
+# incident write-ups, …) lives in a versioned file the maintainers control,
+# so the agent doesn't have to re-read the source tree to rediscover it on
+# every PR. Cached by URL for the lifetime of the process.
+# ---------------------------------------------------------------------------
+
+# Hard caps — these guard against a single repo-config entry pulling down a
+# 2 MB doc and blowing the brief budget. Per-URL truncation keeps the worst
+# case bounded; total truncation caps the sum across URLs.
+_ADDITIONAL_CONTEXT_MAX_URLS = 8
+_ADDITIONAL_CONTEXT_MAX_PER_URL_CHARS = 4000
+_ADDITIONAL_CONTEXT_MAX_TOTAL_CHARS = 16_000
+_ADDITIONAL_CONTEXT_TIMEOUT_S = 5
+# Module-level cache, keyed by URL. The webhook server is a single Python
+# process per pod and reviews happen sequentially, so this stays bounded.
+_ADDITIONAL_CONTEXT_CACHE: dict[str, str] = {}
+
+
+def _parse_additional_context_env(value: str) -> list[str]:
+    """Comma-split an env var into a deduped, ordered URL list."""
+    if not value:
+        return []
+    seen: set[str] = set()
+    out: list[str] = []
+    for piece in value.split(","):
+        u = piece.strip()
+        if u and u not in seen:
+            seen.add(u)
+            out.append(u)
+    return out
+
+
+def _resolve_additional_context_urls(config: dict | None) -> list[str]:
+    """Merge the env var `PRAGENT_ADDITIONAL_CONTEXT_URL` with the per-repo
+    config field `additional_context_urls`. Env var wins on ordering — it
+    appears first so a one-off override can shadow a stale config entry."""
+    env = _parse_additional_context_env(os.environ.get("PRAGENT_ADDITIONAL_CONTEXT_URL", ""))
+    cfg_raw = (config or {}).get("additional_context_urls") or []
+    cfg: list[str] = []
+    if isinstance(cfg_raw, list):
+        for x in cfg_raw:
+            if isinstance(x, str):
+                u = x.strip()
+                if u and u not in set(env):
+                    cfg.append(u)
+    merged = env + cfg
+    return merged[:_ADDITIONAL_CONTEXT_MAX_URLS]
+
+
+def _fetch_one_additional_context(url: str) -> str | None:
+    """Fetch a single URL. Returns the body (UTF-8, truncated) or None on
+    any failure — never raises; additional-context is best-effort.
+
+    Reject non-http(s) schemes defensively so a misconfigured `file://` or
+    `javascript:` URL cannot escape the pod. Cap per-URL size before parsing
+    to avoid a 50 MB response landing in memory.
+    """
+    try:
+        parsed = urllib.parse.urlparse(url)
+    except ValueError:
+        return None
+    if parsed.scheme not in ("http", "https"):
+        return None
+    try:
+        req = urllib.request.Request(url, headers={"User-Agent": "pragent/1.0 (+context)"})
+        with urllib.request.urlopen(req, timeout=_ADDITIONAL_CONTEXT_TIMEOUT_S) as r:
+            raw = r.read(_ADDITIONAL_CONTEXT_MAX_PER_URL_CHARS + 1)
+            if len(raw) > _ADDITIONAL_CONTEXT_MAX_PER_URL_CHARS:
+                raw = raw[:_ADDITIONAL_CONTEXT_MAX_PER_URL_CHARS]
+                truncated = True
+            else:
+                truncated = False
+            body = raw.decode("utf-8", errors="replace")
+    except (urllib.error.URLError, urllib.error.HTTPError, TimeoutError, OSError, ValueError):
+        return None
+    if truncated:
+        body += "\n…[truncated]"
+    return body
+
+
+def fetch_additional_context(urls: list[str]) -> str:
+    """Fetch a list of URLs, join into one string for the brief. Cached.
+
+    Empty when no URLs are given. Best-effort: a URL that errors is logged
+    to stderr and skipped — never aborts the review. Each fetched body is
+    truncated to `_ADDITIONAL_CONTEXT_MAX_PER_URL_CHARS` and the joined
+    output to `_ADDITIONAL_CONTEXT_MAX_TOTAL_CHARS`. Already-cached URLs
+    are not refetched.
+    """
+    if not urls:
+        return ""
+    blocks: list[str] = []
+    total = 0
+    for url in urls:
+        if url in _ADDITIONAL_CONTEXT_CACHE:
+            body = _ADDITIONAL_CONTEXT_CACHE[url]
+        else:
+            body = _fetch_one_additional_context(url) or ""
+            _ADDITIONAL_CONTEXT_CACHE[url] = body
+        if not body:
+            continue
+        block = f"### {url}\n\n{body}"
+        if total + len(block) > _ADDITIONAL_CONTEXT_MAX_TOTAL_CHARS:
+            remaining = _ADDITIONAL_CONTEXT_MAX_TOTAL_CHARS - total
+            if remaining <= 80:
+                break
+            block = block[:remaining] + "\n…[truncated]"
+            blocks.append(block)
+            total = _ADDITIONAL_CONTEXT_MAX_TOTAL_CHARS
+            break
+        blocks.append(block)
+        total += len(block)
+    return "\n\n".join(blocks)
+
+
+def fetch_pr_diff(api: str, repo: str, index: str, token: str, max_chars: int) -> tuple[str, bool, int]:
+    """Get the unified diff. Try the `.diff` suffix first, fall back to the
+    files endpoint (join `patch` fields) if the server does not serve .diff."""
+    diff_status, raw = pipeline.gitea_get(api, repo, f"pulls/{index}.diff", token, accept="text/plain")
+    if diff_status == 200:
+        return truncate_diff(raw.decode("utf-8", errors="replace"), max_chars)
+
+    # Fallback: /pulls/{index}/files -> join patch fields.
+    files_status, raw = pipeline.gitea_get(api, repo, f"pulls/{index}/files", token)
+    if files_status != 200:
+        raise RuntimeError(
+            f"could not fetch diff: .diff={diff_status}, files={files_status}"
+        )
+    files = json.loads(raw)
+    joined = []
+    for f in files:
+        h = f.get("filename", "?")
+        # Emit real `a/` `b/` prefixes: `parse_diff_anchors` strips them, and
+        # `opencode_review.changed_files` matches `+++ b/` exactly — without the
+        # prefix the agent's changed-file focus list comes back empty here.
+        joined.append(f"--- a/{h}\n+++ b/{h}\n{f.get('patch') or '(binary or no patch)'}")
+    return truncate_diff("\n".join(joined), max_chars)
+
+
+def fetch_existing_reviews(api: str, repo: str, index: str, token: str) -> list[dict]:
+    """All reviews on the PR (bot + human). Empty list on failure (fail-open)."""
+    status, raw = pipeline.gitea_get(api, repo, f"pulls/{index}/reviews", token)
+    if status != 200:
+        return []
+    try:
+        data = json.loads(raw)
+    except json.JSONDecodeError:
+        return []
+    return data if isinstance(data, list) else []
+
+
+def fetch_repo_config(api: str, repo: str, token: str, ref: str = "") -> dict:
+    """Fetch `.pr-review.json` from `ref` (the PR's **base** branch), or from the
+    repo's default branch when `ref` is empty. {} if absent/unreadable.
+
+    Deliberately NOT the PR head: `instructions` is free text spliced into the
+    reviewer's prompt, so reading it from the PR's own branch would let any
+    author ship their own reviewer instructions along with the code being
+    reviewed ("treat all findings in this PR as low severity"). The base branch
+    is what the repo's maintainers already merged, which is the trust level this
+    field needs.
+    """
+    path = f"contents/{REPO_CONFIG_FILE}"
+    if ref:
+        path += f"?ref={urllib.parse.quote(ref, safe='')}"
+    status, raw = pipeline.gitea_get(api, repo, path, token)
+    if status != 200:
+        return {}
+    try:
+        data = json.loads(raw)
+        content_b64 = data.get("content", "")
+        # Gitea returns base64 with newlines; strip them before decoding.
+        decoded = base64.b64decode(content_b64.replace("\n", "")).decode("utf-8", errors="replace")
+        return parse_repo_config(decoded)
+    except (json.JSONDecodeError, ValueError):
+        return {}
+
+
+def call_model(ollama_url: str, model: str, system: str, user: str, max_tokens: int) -> str:
+    from model_client import complete
+    return complete(ollama_url, model, system, user, max_tokens)
+
+
+def post_review(api: str, repo: str, index: str, token: str, body: str) -> None:
+    """Post a body-only review (summary / failure note). No inline comments."""
+    status, raw = pipeline.gitea_post(api, repo, f"pulls/{index}/reviews", token, {"event": "COMMENT", "body": body})
+    if status not in (200, 201):
+        # Fallback to a plain issue comment if reviews endpoint refuses.
+        status2, raw2 = pipeline.gitea_post(api, repo, f"issues/{index}/comments", token, {"body": body})
+        if status2 not in (200, 201):
+            raise RuntimeError(f"post review failed: reviews={status}, comments={status2}")
+
+
+def post_inline_review(
+    api: str, repo: str, index: str, token: str, summary: str, anchored: list[dict]
+) -> None:
+    """Post a review with a summary body AND positional inline comments.
+
+    Each anchored finding becomes one entry in `comments`. Gitea 1.26.x anchors
+    inline review comments with `new_position` (the line in the POST-change file)
+    + `old_position: 0` — the `line`/`side` fields used by newer Gitea are NOT
+    honored here and silently leave the comment unpositioned (Gitea then renders
+    a file-level comment on EVERY diff line of the file, which is the flood we
+    hit). `f["line"]` is already a validated post-change (RIGHT-side) line from
+    `split_findings`, so it maps directly to `new_position`. The body carries a
+    language-tagged fenced code block when the model produced replacement code.
+    """
+    comments = [
+        {
+            "path": f["path"],
+            "new_position": f["line"],
+            "old_position": 0,
+            "body": inline_comment_body(f),
+        }
+        for f in anchored
+    ]
+    payload = {"event": "COMMENT", "body": summary, "comments": comments}
+    status, raw = pipeline.gitea_post(api, repo, f"pulls/{index}/reviews", token, payload)
+    if status in (200, 201):
+        return
+    # If the inline post failed (e.g. a bad line slipped through), retry as a
+    # body-only review — but fold the anchored findings into the body as bullets
+    # first. Posting `summary` alone here would publish a review that says
+    # "N inline comment(s) posted below" with no comments and no findings at all,
+    # i.e. every finding silently lost on the one path where that matters most.
+    degraded = summary
+    if anchored:
+        degraded += (
+            "\n\n_Inline anchoring failed (Gitea returned "
+            f"{status}); findings listed here instead:_\n\n"
+            + summary_bullets(anchored)
+        )
+    post_review(api, repo, index, token, degraded)
+
+
+# ---------------------------------------------------------------------------
+# Main
+# ---------------------------------------------------------------------------
+
+
+def _need(name: str) -> str:
+    v = os.environ.get(name)
+    if not v:
+        raise RuntimeError(f"missing env {name}")
+    return v
+
+
+def _emit_langfuse(
+    *,
+    repo: str,
+    index: str,
+    sha: str,
+    title: str,
+    model: str,
+    usage: dict | None,
+    findings: list[dict],
+    summary: str,
+    engine: str,
+    config: dict | None = None,
+    dropped_count: float | None = None,
+) -> None:
+    """Ship this review's usage to Langfuse, if one is configured.
+
+    Called on both exit paths that spent tokens — the normal post and the
+    salvage path — because an unparseable run costs the same as a clean one and
+    is exactly the kind of thing worth trending.
+
+    Local import + blanket except: `langfuse_trace` is stdlib-only but optional,
+    and telemetry is never allowed to fail a review (see the fail-open contract
+    in `review_pr`). The trace's `environment` is `claude` or `ollama`, so the
+    two spend stories stay separated in every Langfuse view.
+    """
+    try:
+        import langfuse_trace
+
+        # Same comparison model the review body prices against, so the number
+        # in Langfuse and the number in the PR agree. Free/unknown models
+        # (MiniMax, glm, self-hosted qwen) are priced against it; a paid model
+        # is priced as itself.
+        price_target, _err = _resolve_price_target(config)
+
+        langfuse_trace.emit_review_trace(
+            repo=repo, index=index, sha=sha, title=title, model=model,
+            usage=usage, findings=findings, summary=summary or "",
+            engine=engine, lenses=(usage or {}).get("lenses"),
+            price_target=price_target, dropped_count=dropped_count,
+        )
+    except Exception as e:
+        print(f"pragent: langfuse emit skipped: {e}", file=sys.stderr)

pilot/review/ai_review.py

@@ -0,0 +1,14 @@
+"""Public review interface.
+
+Keep this module deliberately small. Existing callers import ``ai_review``
+directly, so the compatibility facade exposes the implementation module under
+the old name while the implementation is free to be split behind package
+seams without changing callers.
+"""
+from __future__ import annotations
+
+import importlib
+import sys
+
+_implementation = importlib.import_module("review.pipeline")
+sys.modules[__name__] = _implementation

pilot/review/analysis.py

@@ -0,0 +1,469 @@
+from __future__ import annotations
+
+import base64
+import json
+import os
+import re
+import sys
+import urllib.error
+import urllib.parse
+import urllib.request
+
+from . import pipeline
+from .pipeline import *
+from .pipeline import _CONFIDENCE_BADGE
+from .configuration import effective_config
+from .output import _string_list, findings_table
+
+# Pure helpers (unit-tested, no network)
+# ---------------------------------------------------------------------------
+
+
+def truncate_diff(text: str, max_chars: int) -> tuple[str, bool, int]:
+    """Return (text, was_truncated, original_len). Never raises on bad input."""
+    if text is None:
+        return "", False, 0
+    orig_len = len(text)
+    if orig_len <= max_chars:
+        return text, False, orig_len
+    return text[:max_chars] + f"\n\n[diff truncated at {max_chars} characters]\n", True, orig_len
+
+
+def fmt_tokens(n) -> str:
+    """1234567 -> '1,234,567 (1.2M)'; 0 -> '0'; <1000 -> comma-only; None/negative -> '?'.
+
+    Always returns the full comma-separated number; the short suffix is a
+    parenthetical for fast scanning. Caps at B; the cost model never exceeds M.
+    """
+    if n is None:
+        return "?"
+    if not isinstance(n, (int, float)) or n < 0:
+        return "?"
+    n = int(n)
+    if n < 1000:
+        return f"{n:,}"
+    if n < 1_000_000:
+        return f"{n:,} ({n / 1000:.1f}K)"
+    if n < 1_000_000_000:
+        return f"{n:,} ({n / 1_000_000:.1f}M)"
+    return f"{n:,} ({n / 1_000_000_000:.1f}B)"
+
+
+def parse_text_blocks(content: list) -> str:
+    """Join `type:"text"` blocks from an Anthropic /v1/messages response.
+
+    Drops `thinking` blocks (glm-5.2:cloud is a reasoning model and emits them).
+    Tolerates missing/malformed blocks by skipping them.
+    """
+    from model_client import parse_text_blocks as _parse_text_blocks
+    return _parse_text_blocks(content)
+
+
+def _int_env(name: str, default: int) -> int:
+    """Read an int from the environment, falling back on anything unparseable.
+
+    A typo in a tuning knob must not take down a review that is already
+    mid-flight — the operator gets a stderr line and the default instead.
+    """
+    raw = os.environ.get(name, "")
+    if not str(raw).strip():
+        return default
+    try:
+        return int(str(raw).strip())
+    except (TypeError, ValueError):
+        print(
+            f"pragent: ignoring {name}={raw!r} (not an integer); using {default}",
+            file=sys.stderr, flush=True,
+        )
+        return default
+
+
+# 1-5 merge-verdict score (higher = safer). Buckets:
+#   5 = clean (or low/info/trivial only — nothing worth blocking on)
+#   4 = medium present
+#   3 = high present (operator should at least look)
+#   1 = critical present (block the merge by default)
+# Cross-lens agreement on any finding takes one more off, floored at 1.
+_CONFIDENCE_BADGE = {5: "🟢", 4: "🟢", 3: "🟡", 2: "🟠", 1: "🔴"}
+
+
+def merge_confidence(findings: list[dict], *, multi_lens_observed: bool = False) -> int:
+    """1-5 merge verdict: higher = safer.
+
+    Tier drops driven by the most severe finding present:
+      - critical  → 1
+      - high      → 3
+      - medium    → 4
+      - else      → 5 (low / trivial / info / unknown → no drop)
+
+    An extra -1 when cross-lens agreement was observed on any finding
+    (``multi_lens_observed``). The flag is passed in explicitly because the
+    raw ``_multi_lens`` marker is stripped from findings by the time they
+    reach this function — first by ``opencode_review.run_lenses_review``
+    (the ``_``-prefix scrub) and again by ``_normalize_finding`` (the
+    7-key schema rebuild). The caller (``review_pr``) must capture the
+    signal before those strips fire. Final score is clamped to [1, 5] so
+    a critical + multi_lens combo doesn't go negative.
+    """
+    if not findings:
+        return 5
+    max_rank = max(SEVERITY_RANK.get(f.get("severity", "low"), 0) for f in findings)
+    if max_rank >= SEVERITY_RANK["critical"]:
+        score = 1
+    elif max_rank >= SEVERITY_RANK["high"]:
+        score = 3
+    elif max_rank >= SEVERITY_RANK["medium"]:
+        score = 4
+    else:
+        score = 5
+    if multi_lens_observed:
+        score -= 1
+    return max(1, min(5, score))
+
+
+def format_review_body(
+    findings: str,
+    model: str,
+    sha: str,
+    summary: str = "",
+    usage_section: str = "",
+    *,
+    summary_changes: list[str] | None = None,
+    risks: list[str] | None = None,
+    findings_for_table: list[dict] | None = None,
+    inline_count: int = 0,
+    confidence: int = 5,
+    walkthrough: list[str] | None = None,
+    risk_verdict: str = "",
+    test_coverage: str = "",
+    static_message: str = "",
+
+) -> str:
+    """Format the posted review summary body.
+
+    Layout (per the operator's format guide):
+
+      * Header line (``🤖 AI Review …``) including the merge-confidence badge.
+      * Optional static banner (``> {static_message}``) — repo-wide call-out
+        from `.pr-review.json:static_message`, placed under the header so
+        every reviewer sees it on every review without scrolling.
+
+      * **Summary of Changes** — 2–4 bullets of what the PR introduces
+        (`summary_changes`); falls back to the opencode prose `summary` if
+        the agent didn't emit the list.
+      * **Risk Verdict** — one-line "<level> risk: <reason>" verdict
+        (`risk_verdict`); omitted when empty.
+      * **Walkthrough** — up to 6 file- or change-grouped bullets
+        (`walkthrough`); the file part is wrapped in backticks so paths
+        render as code in Gitea. Omitted when empty.
+      * **Test Coverage** — short `test_coverage` string ("Tests added" /
+        etc.); omitted when empty.
+      * **Key Risks & Concerns** — bullets of potential bugs/edge cases
+        found across the diff (`risks`).
+      * **Findings Overview** — a Markdown table (severity / location /
+        one-line problem) covering ALL findings, anchored or not.
+      * Unanchored bullets — findings with no post-change line to anchor
+        (the inline ones are posted separately as Gitea review comments).
+      * AI Usage & Run Details — wrapped in a ``<details>`` collapsible so
+        the body stays scannable; cost lines stay inside it.
+      * Hidden SHA marker — for the dedupe pass.
+
+    `confidence` is a 1-5 merge verdict rendered as `<N>/5 <badge>` in the
+    header. Clamped to [1, 5] so a stray value (e.g. 0 from a missing
+    finding list) doesn't print a broken badge.
+
+    Empty `summary_changes` + empty `risks` + empty `summary` collapse into
+    a single "Summary of Changes: _no summary provided._" line so the body
+    never looks half-rendered.
+    """
+    score = max(1, min(5, confidence))
+    badge = _CONFIDENCE_BADGE.get(score, "🟢")
+    confidence_str = f"{score}/5 {badge}"
+    header = REVIEW_HEADER.format(
+        model=model,
+        sha=sha[:8] if sha else "unknown",
+        confidence=confidence_str,
+    )
+    parts: list[str] = [header]
+
+    # Optional free-text banner. Rendered as a Markdown blockquote immediately
+    # after the header — front-of-mind for any maintainer scanning the review.
+    if static_message and static_message.strip():
+        parts.append(f"> {static_message.strip()}")
+
+    # --- Summary of Changes ---
+    sc = list(summary_changes or [])
+    if not sc and summary:
+        sc = _string_list(summary)
+    if sc:
+        sc = sc[:4]
+        items = "\n".join(f"- {item}" for item in sc)
+        parts.append(f"### Summary of Changes\n\n{items}")
+    else:
+        parts.append("### Summary of Changes\n\n_No summary provided._")
+
+    # --- Risk Verdict ---
+    if risk_verdict:
+        parts.append(f"### Risk Verdict\n\n{risk_verdict}")
+
+    # --- Walkthrough ---
+    wt = list(walkthrough or [])
+    if wt:
+        wt = wt[:6]
+        rendered = []
+        for item in wt:
+            # Items typically look like "a.py — adds X" (em-dash separator).
+            # Wrap the file path in backticks so it renders as code in the
+            # Gitea markdown body; leave the description as plain prose. When
+            # no separator is present, render the whole line as plain prose
+            # (the agent's "plain prose" fallback for change-grouped bullets).
+            if " — " in item:
+                path, _, rest = item.partition(" — ")
+                rendered.append(f"- `{path}` — {rest}")
+            else:
+                rendered.append(f"- {item}")
+        parts.append(f"### Walkthrough\n\n" + "\n".join(rendered))
+
+    # --- Test Coverage ---
+    if test_coverage:
+        parts.append(f"### Test Coverage\n\n{test_coverage}")
+
+    # --- Key Risks & Concerns ---
+    rs = list(risks or [])
+    if rs:
+        items = "\n".join(f"- {item}" for item in rs)
+        parts.append(f"### Key Risks & Concerns\n\n{items}")
+    else:
+        parts.append("### Key Risks & Concerns\n\n_None identified._")
+
+    # --- Findings Overview (table) ---
+    table = findings_table(findings_for_table or [])
+    if table:
+        n_inline = inline_count
+        n_total = len(findings_for_table or [])
+        if n_inline:
+            heading = f"### Findings Overview\n\n_{n_inline} inline comment(s); {n_total} total._"
+        else:
+            heading = f"### Findings Overview\n\n_{n_total} finding(s)._"
+        parts.append(f"{heading}\n\n{table}")
+
+    # --- Unanchored bullets ---
+    fb = (findings or "").strip()
+    if fb:
+        parts.append(fb)
+
+    # --- Collapsible usage ---
+    if usage_section:
+        parts.append(usage_section.strip())
+
+    # --- Hidden marker ---
+    marker = SHA_MARKER.format(sha=sha) if sha else ""
+
+    body = "\n\n".join(parts)
+    if marker:
+        body += f"\n{marker}"
+    return body
+
+
+def _finding_weight(f: dict) -> int:
+    """Body-weight used to attribute output tokens to a finding (char length of
+    its rendered problem + fix + suggestion). One model pass produces all
+    findings, so per-finding tokens can't be measured directly — we split the
+    measured output total by this weight as an honest attribution."""
+    return (
+        len(f.get("problem") or "")
+        + len(f.get("fix") or "")
+        + len(f.get("suggestion") or "")
+    )
+
+
+def compute_attribution(findings: list[dict], output_tokens: int) -> None:
+    """Stash `_tok_attrib` (attributed output tokens) and `_tok_pct` (0..1) on
+    each finding, splitting `output_tokens` by each finding's body weight.
+    Mutates in place. No-op when there are no findings or no output budget."""
+    if not findings or not output_tokens:
+        return
+    weights = [_finding_weight(f) for f in findings]
+    total_w = sum(weights)
+    if total_w <= 0:
+        # All-zero weights (no prose): split evenly.
+        share = output_tokens / len(findings)
+        for f in findings:
+            f["_tok_attrib"] = int(round(share))
+            f["_tok_pct"] = 1.0 / len(findings)
+        return
+    for f, w in zip(findings, weights):
+        f["_tok_attrib"] = int(round(output_tokens * w / total_w))
+        f["_tok_pct"] = w / total_w
+
+
+def _resolve_price_target(config: dict | None) -> tuple[str, str | None]:
+    """Pick which provider to compute the equivalent cost against.
+
+    Order: `.pr-review.json:cost_target` > `PRAGENT_PRICE_TARGET` env >
+    `DEFAULT_PRICE_TARGET` (claude-sonnet-5). Returns `(price_key, error)`.
+
+    If any of the user-set keys is unknown, falls back to the default AND
+    reports the error so the operator sees their typo (a config-level typo
+    silently picking the default would defeat the purpose of letting repos
+    opt into a different comparison model).
+    """
+    from cost_model import PRICES  # local import keeps ollama path dep-free
+    candidates: list[tuple[str, str]] = []
+    if isinstance(config, dict) and config.get("cost_target"):
+        candidates.append(("repo config", str(config["cost_target"]).strip()))
+    env = os.environ.get("PRAGENT_PRICE_TARGET", "").strip()
+    if env:
+        candidates.append(("PRAGENT_PRICE_TARGET env", env))
+    candidates.append(("default", DEFAULT_PRICE_TARGET))
+
+    chosen = DEFAULT_PRICE_TARGET
+    for source, key in candidates:
+        if key in PRICES:
+            chosen = key
+            break
+    else:
+        # No candidate was valid. Use default + report.
+        return chosen, (
+            f"unknown price target (checked {', '.join(f'{s}={k!r}' for s, k in candidates)}); "
+            f"valid: {', '.join(sorted(PRICES))}"
+        )
+
+    # Even when we picked a valid key, if the *user* set one and it was
+    # unknown, surface that. (We only get here if a later candidate resolved,
+    # so the invalid one was upstream.)
+    invalid = [(s, k) for s, k in candidates if k not in PRICES and s != "default"]
+    if invalid:
+        return chosen, (
+            f"unknown price target (set {', '.join(f'{s}={k!r}' for s, k in invalid)}); "
+            f"valid: {', '.join(sorted(PRICES))}; falling back to `{chosen}`"
+        )
+    return chosen, None
+
+
+def _resolve_display_model(base_model: str, config: dict | None) -> str:
+    """Resolve the *display* model for one review.
+
+    Precedence (highest first):
+      1. `OPENCODE_MODEL` env var — operator override, used as-is (already a
+         provider-prefixed opencode ref like `headroom/MiniMax-M2.7`).
+      2. `.pr-review.json:model` — per-repo override. Already validated
+         against `cost_model.PRICES` by `parse_repo_config`, so a bare key
+         like `claude-sonnet-5` or `qwen3.8-27b` is safe. Re-prefixed with
+         the model's `provider` field from `cost_model.Price` (default
+         `headroom`) so the opencode subprocess routes correctly — e.g.
+         `qwen3.8-27b` → `vllm-qwen38/qwen3.8-27b` (vLLM on RTX 3090 at
+         192.168.1.79:18020), `claude-sonnet-5` → `headroom/claude-sonnet-5`
+         (Anthropic pricing proxy).
+      3. Default — `f"headroom/{base_model}"` where `base_model` is the bare
+         `OLLAMA_MODEL` (e.g. `"MiniMax-M2.7" → "headroom/MiniMax-M2.7"`).
+
+    The same value flows to every consumer (opencode subprocess, REVIEW_HEADER,
+    cost-line parenthetical) so reviewers never see a mix of `glm-5.2:cloud`
+    and the routed model in one body.
+    """
+    env = os.environ.get("OPENCODE_MODEL")
+    if env:
+        return env
+    cfg_model = (config or {}).get("model")
+    if isinstance(cfg_model, str) and cfg_model.strip():
+        # Look up the provider from PRICES so the opencode subprocess routes
+        # through the right provider block (vllm-qwen38 vs headroom). Lazy
+        # import — the ollama path doesn't touch cost_model.
+        from cost_model import PRICES
+        provider = PRICES.get(cfg_model.strip())
+        if provider is not None:
+            return f"{provider.provider}/{cfg_model.strip()}"
+        # parse_repo_config already drops unknowns, but stay defensive: fall
+        # back to headroom so the review still runs rather than crash.
+        return f"headroom/{cfg_model.strip()}"
+    return f"headroom/{base_model}"
+
+
+def equivalent_cost(usage: dict, price_key: str) -> float:
+    """USD the measured usage would have billed on `price_key`'s provider.
+
+    `usage` is the dict from `parse_opencode_events` (input/output/reasoning/
+    cache_read/cache_write). Builds a `cost_model.Usage` and runs `cost()`. The
+    pilot's actual provider (headroom/glm-5.2:cloud) reports $0 — this is what
+    the same tokens would cost on a paid model, so maintainers can budget.
+    """
+    from cost_model import Usage, cost, PRICES  # local import: ollama path dep-free
+    if price_key not in PRICES:
+        return 0.0
+    u = Usage(
+        uncached_input=(usage.get("input", 0) - usage.get("cache_read", 0)),
+        cached_input=usage.get("cache_read", 0),
+        cache_writes=usage.get("cache_write", 0),
+        output=usage.get("output", 0),
+    )
+    return cost(u, PRICES[price_key])
+
+
+def build_user_prompt(
+    title: str,
+    body: str,
+    diff: str,
+    config: dict | None = None,
+    prior_reviews: list[str] | None = None,
+    additional_context: str = "",
+) -> str:
+    """Assemble the user prompt: repo config + additional context + prior reviews + PR meta + diff."""
+    parts: list[str] = []
+
+    eff = effective_config(config) if config else {}
+    if eff:
+        cfg_lines = []
+        if eff.get("focus"):
+            cfg_lines.append("Focus areas: " + ", ".join(eff["focus"]))
+        if eff.get("exclude_paths"):
+            cfg_lines.append("Ignore paths: " + ", ".join(eff["exclude_paths"]))
+        if eff.get("languages"):
+            cfg_lines.append("Languages: " + ", ".join(eff["languages"]))
+        if eff.get("style"):
+            cfg_lines.append(f"Review style: {eff['style']} "
+                             f"(max {eff['max_findings']} findings, threshold "
+                             f"{eff['severity_threshold']}+)")
+        if eff.get("patterns", {}).get("allow"):
+            cfg_lines.append("Allow paths (only these are reviewed): "
+                             + ", ".join(eff["patterns"]["allow"]))
+        if eff.get("patterns", {}).get("deny"):
+            cfg_lines.append("Deny paths: " + ", ".join(eff["patterns"]["deny"]))
+        if eff.get("exclude_tests"):
+            cfg_lines.append("Skip test files entirely.")
+        if eff.get("require_tests"):
+            cfg_lines.append("Flag behavioral changes that don't add a test "
+                             "alongside (added as a `low` finding).")
+        if eff.get("instructions"):
+            cfg_lines.append("Instructions:\n" + str(eff["instructions"]).strip())
+        if cfg_lines:
+            parts.append("## Repo review config (.pr-review.json)\n" + "\n".join(cfg_lines))
+
+    if additional_context:
+        # Repo-provided static background (architecture summary, module map,
+        # conventions, glossary, …). Cached for the review; the agent reads
+        # this ONCE per review and the prompt-cached prefix absorbs it on
+        # later steps — much cheaper than re-discovering the same facts from
+        # the source tree on every PR.
+        parts.append(
+            "## Repo-provided context (.pr-review.json:additional_context_urls "
+            "+ PRAGENT_ADDITIONAL_CONTEXT_URL — cached per review)\n" + additional_context
+        )
+
+    if prior_reviews:
+        joined = "\n\n---\n\n".join(prior_reviews)
+        if len(joined) > 4000:
+            joined = joined[:4000] + "\n…[prior reviews truncated]"
+        parts.append("## PREVIOUS REVIEWS (already posted — do NOT repeat these points)\n" + joined)
+
+    parts.append(f"## PR\nTitle: {title or '(none)'}")
+    if body and body.strip():
+        b = body.strip()
+        if len(b) > 4000:
+            b = b[:4000] + "\n…[PR body truncated]"
+        parts.append(f"Description:\n{b}")
+    parts.append(f"## Diff\n```diff\n{diff}\n```")
+    return "\n\n".join(parts)
+
+
+# ---------------------------------------------------------------------------

pilot/review/config.py

@@ -0,0 +1,32 @@
+"""Trusted repository configuration and opt-in policy."""
+from __future__ import annotations
+
+import base64
+import json
+import urllib.parse
+from collections.abc import Callable
+
+
+def repo_enabled(
+    get: Callable[..., tuple[int, bytes]],
+    api: str,
+    repo: str,
+    ref: str,
+    token: str,
+) -> bool:
+    """Read the opt-in flag from the trusted base branch.
+
+    The transport is injected so the policy is testable without a live Gitea.
+    Any missing, malformed, or non-boolean value disables review.
+    """
+    path = "contents/.pr-review.json?ref=" + urllib.parse.quote(ref, safe="")
+    status, raw = get(api, repo, path, token)
+    if status != 200:
+        return False
+    try:
+        envelope = json.loads(raw)
+        encoded = envelope.get("content", "").replace("\n", "")
+        config = json.loads(base64.b64decode(encoded).decode("utf-8", errors="replace"))
+    except (AttributeError, TypeError, ValueError, json.JSONDecodeError):
+        return False
+    return isinstance(config, dict) and config.get("enabled") is True

pilot/review/configuration.py

@@ -0,0 +1,459 @@
+from __future__ import annotations
+
+import base64
+import json
+import os
+import re
+import sys
+import urllib.error
+import urllib.parse
+import urllib.request
+
+from . import pipeline
+from .pipeline import *
+
+# Repo config + existing-review helpers
+# ---------------------------------------------------------------------------
+
+
+# Caps on `.pr-review.json`. The file is committed config, not free-form model
+# input, and every byte of it lands in the prompt — bound it so a bloated (or
+# hostile) config can't crowd out the diff or blow the context window.
+CONFIG_MAX_LIST_ITEMS = 32
+CONFIG_MAX_ITEM_CHARS = 200
+CONFIG_MAX_INSTRUCTIONS_CHARS = 4000
+CONFIG_MAX_PATTERNS_ITEMS = 16  # allow + deny separately, total 32 entries
+CONFIG_MAX_FINDINGS = 30
+CONFIG_MAX_STATIC_MESSAGE_CHARS = 400  # free-text banner, mirror of instructions
+
+STYLES = frozenset(STYLE_DEFAULTS)
+SEVERITY_VALUES = frozenset(SEVERITIES)
+
+
+def parse_repo_config(raw: str) -> dict:
+    """Parse a .pr-review.json blob tolerantly. Returns {} on any failure.
+
+    List fields are capped at CONFIG_MAX_LIST_ITEMS entries of
+    CONFIG_MAX_ITEM_CHARS each; `instructions` at CONFIG_MAX_INSTRUCTIONS_CHARS;
+    `patterns.allow` / `patterns.deny` each capped at CONFIG_MAX_PATTERNS_ITEMS
+    of CONFIG_MAX_ITEM_CHARS.
+
+    Recognised keys (all optional):
+      focus, exclude_paths, languages, instructions    — text steer
+      static_message  ≤ CONFIG_MAX_STATIC_MESSAGE_CHARS — banner under header
+      style          strict|balanced|lenient            — default: balanced
+      severity_threshold low|medium|high|critical       — default: per style
+      max_findings   1..CONFIG_MAX_FINDINGS              — default: per style
+      exclude_tests  bool                                — default: False
+      require_tests  bool                                — default: False
+      patterns       {allow:[…], deny:[…]}              — post-filter globs
+      model          <key of cost_model.PRICES>          — per-repo override
+      cost_target    <key of cost_model.PRICES>          — see equivalent_cost
+      additional_context_urls   list[str] (≤ 8)          — see fetch_additional_context
+    """
+    if not raw:
+        return {}
+    try:
+        data = json.loads(raw)
+    except json.JSONDecodeError:
+        return {}
+    if not isinstance(data, dict):
+        return {}
+
+    def _str_list(v):
+        if isinstance(v, list) and all(isinstance(x, str) for x in v):
+            return [x[:CONFIG_MAX_ITEM_CHARS] for x in v[:CONFIG_MAX_LIST_ITEMS]]
+        return None
+
+    out: dict = {}
+    for k in ("focus", "exclude_paths", "languages"):
+        s = _str_list(data.get(k))
+        if s is not None:
+            out[k] = s
+
+    instr = data.get("instructions")
+    if isinstance(instr, str) and instr.strip():
+        out["instructions"] = instr.strip()[:CONFIG_MAX_INSTRUCTIONS_CHARS]
+
+    sm = data.get("static_message")
+    if isinstance(sm, str) and sm.strip():
+        out["static_message"] = sm.strip()[:CONFIG_MAX_STATIC_MESSAGE_CHARS]
+
+    style = data.get("style")
+    if isinstance(style, str) and style.strip().lower() in STYLES:
+        out["style"] = style.strip().lower()
+
+    thresh = data.get("severity_threshold")
+    if isinstance(thresh, str) and thresh.strip().lower() in SEVERITY_VALUES:
+        out["severity_threshold"] = thresh.strip().lower()
+
+    mf = data.get("max_findings")
+    if isinstance(mf, int) and not isinstance(mf, bool) and 1 <= mf <= CONFIG_MAX_FINDINGS:
+        out["max_findings"] = mf
+    elif isinstance(mf, str) and mf.strip().isdigit():
+        n = int(mf.strip())
+        if 1 <= n <= CONFIG_MAX_FINDINGS:
+            out["max_findings"] = n
+
+    for bk in ("exclude_tests", "require_tests"):
+        if isinstance(data.get(bk), bool):
+            out[bk] = data[bk]
+
+    pat = data.get("patterns")
+    if isinstance(pat, dict):
+        allow = _str_list(pat.get("allow"))
+        deny = _str_list(pat.get("deny"))
+        patterns = {}
+        if allow is not None:
+            patterns["allow"] = allow[:CONFIG_MAX_PATTERNS_ITEMS]
+        if deny is not None:
+            patterns["deny"] = deny[:CONFIG_MAX_PATTERNS_ITEMS]
+        if patterns:
+            out["patterns"] = patterns
+
+    ct = data.get("cost_target")
+    if isinstance(ct, str) and ct.strip():
+        out["cost_target"] = ct.strip()
+
+    # Per-repo model override. Validated against cost_model.PRICES so the value
+    # is usable both as the opencode subprocess ref and as the REVIEW_HEADER
+    # label (see _resolve_display_model precedence). Unknown values are dropped
+    # with a stderr pointer to the valid set — silently ignoring would mask
+    # typos from repo admins.
+    raw_model = data.get("model")
+    if raw_model is not None:
+        if isinstance(raw_model, str) and raw_model.strip():
+            from cost_model import PRICES  # lazy: ollama path dep-free
+            candidate = raw_model.strip()
+            if candidate in PRICES:
+                out["model"] = candidate
+            else:
+                print(
+                    f"pragent: .pr-review.json:model={candidate!r} not in "
+                    f"cost_model.PRICES (valid: {', '.join(sorted(PRICES))}); "
+                    f"dropping",
+                    file=sys.stderr, flush=True,
+                )
+
+    acu = data.get("additional_context_urls")
+    if isinstance(acu, list):
+        urls: list[str] = []
+        for x in acu:
+            if isinstance(x, str):
+                u = x.strip()
+                if u:
+                    urls.append(u)
+        if urls:
+            # Cap is also enforced later by _resolve_additional_context_urls;
+            # this just stops a 10k-entry file from making the config huge.
+            out["additional_context_urls"] = urls[:8]
+
+    # Multi-lens reviewers roster. Absent / empty list = the 5-lens default
+    # in pilot/opencode_review.py (security, docs, code-quality, tests, perf).
+    # This is the cheap trigger: once the config declares `reviewers[]`, the
+    # orchestrator spawns one opencode subprocess per lens in parallel. Set
+    # to `[]` to opt out (single-primary fallback). Capped at 8.
+    rev = _parse_reviewers_array(data.get("reviewers"))
+    if rev is not None:
+        out["reviewers"] = rev
+
+    # Triage (cheap pre-filter that picks a subset of lenses). Off by default
+    # to keep the parse deterministic; the orchestrator's own default is
+    # to enable it when `reviewers[]` is present.
+    tr = _parse_triage_object(data.get("triage"))
+    if tr is not None:
+        out["triage"] = tr
+
+    # Repo-level kill-switch: `enabled: false` lets a maintainer pause the bot
+    # for this repo without removing the file (handy during a flaky provider
+    # outage). Always written so callers can do `cfg.get("enabled") is False`
+    # without a separate default — the file itself is committed, so we treat
+    # absent / wrong-type as an explicit off rather than as "config missing".
+    en = data.get("enabled")
+    out["enabled"] = en if isinstance(en, bool) else False
+
+    # Compare-against roster: list of `cost_model.PRICES` keys the render layer
+    # uses to print equivalent-cost lines (one per key) for maintainer
+    # budgeting. Unknown keys are dropped with a stderr line so a typo is loud.
+    # Lazy import: `cost_model` has no dep on `ai_review`, and the ollama
+    # fallback path never hits this branch — keep import-time cost low there.
+    from cost_model import PRICES as _PRICES
+    ca = data.get("compare_against")
+    if isinstance(ca, list):
+        cleaned: list[str] = []
+        for x in ca:
+            if isinstance(x, str) and x.strip() in _PRICES:
+                cleaned.append(x.strip())
+            elif isinstance(x, str):
+                print(
+                    f"pragent: ignoring compare_against entry {x!r} "
+                    f"(not in cost_model.PRICES); valid: {', '.join(sorted(_PRICES))}",
+                    file=sys.stderr, flush=True,
+                )
+        if cleaned:
+            out["compare_against"] = cleaned[:12]
+
+    return out
+
+
+def _parse_reviewers_array(raw) -> list[dict] | None:
+    """Sanitize `.pr-review.json:reviewers[]` to a list of dicts.
+
+    Hard caps: 8 entries (default-reviewers.xml-bound), 200 chars per string
+    field. Untyped / non-list → None (caller keeps the default). Fields we
+    don't know about are dropped (no schema drift allowed).
+    """
+    if not isinstance(raw, list):
+        return None
+    cap = 8
+    out: list[dict] = []
+    for entry in raw[:cap]:
+        if not isinstance(entry, dict):
+            continue
+        spec: dict = {}
+        rid = entry.get("id")
+        if isinstance(rid, str) and rid.strip():
+            cand = rid.strip()[:CONFIG_MAX_ITEM_CHARS]
+            # Same id shape required by opencode_review.parse_reviewers_config:
+            # kebab-case so it maps 1:1 to .opencode/agents/<id>.md
+            import re as _re
+            if _re.match(r"^[a-z0-9][a-z0-9-]{0,31}$", cand):
+                spec["id"] = cand
+        if not spec.get("id"):
+            continue
+        for sk in ("agent_file", "model"):
+            sv = entry.get(sk)
+            if isinstance(sv, str) and sv.strip():
+                spec[sk] = sv.strip()[:CONFIG_MAX_ITEM_CHARS]
+        sf = entry.get("severity_floor")
+        if isinstance(sf, str) and sf.strip().lower() in SEVERITY_VALUES:
+            spec["severity_floor"] = sf.strip().lower()
+        mf = entry.get("max_findings")
+        if isinstance(mf, int) and not isinstance(mf, bool) and 1 <= mf <= CONFIG_MAX_FINDINGS:
+            spec["max_findings"] = mf
+        act = entry.get("activation")
+        if isinstance(act, str) and act.strip().lower() in ("auto", "always", "off"):
+            spec["activation"] = act.strip().lower()
+        skip = entry.get("skip_if_all_changed_paths")
+        if isinstance(skip, str) and skip.strip():
+            spec["skip_if_all_changed_paths"] = skip.strip()[:CONFIG_MAX_ITEM_CHARS]
+        globs = entry.get("hotpath_globs")
+        if isinstance(globs, list):
+            cleaned = [g for g in globs if isinstance(g, str) and g.strip()]
+            if cleaned:
+                spec["hotpath_globs"] = [
+                    g.strip()[:CONFIG_MAX_ITEM_CHARS]
+                    for g in cleaned[:CONFIG_MAX_LIST_ITEMS]
+                ]
+        out.append(spec)
+    return out
+
+
+def _parse_triage_object(raw) -> dict | None:
+    """Sanitize `.pr-review.json:triage` to a dict.
+
+    Returns `None` when absent. When the value is malformed (not an object),
+    returns `{"enabled": False}` so a typo disables triage rather than
+    silently making the orchestrator error.
+    """
+    if raw is None:
+        return None
+    if not isinstance(raw, dict):
+        return {"enabled": False}
+    out: dict = {}
+    if isinstance(raw.get("enabled"), bool):
+        out["enabled"] = raw["enabled"]
+    if isinstance(raw.get("model"), str) and raw["model"].strip():
+        out["model"] = raw["model"].strip()[:CONFIG_MAX_ITEM_CHARS]
+    ml = raw.get("max_lenses")
+    if isinstance(ml, int) and not isinstance(ml, bool) and 1 <= ml <= 8:
+        out["max_lenses"] = ml
+    return out
+
+
+def effective_config(config: dict | None) -> dict:
+    """Apply STYLE_DEFAULTS for any field the config didn't pin.
+
+    Returns a NEW dict combining the user's `.pr-review.json` (if any) with the
+    derived `max_findings` / `severity_threshold`. Style itself is preserved
+    so downstream code can branch on it.
+    """
+    style = (config or {}).get("style", "balanced")
+    max_findings, severity_threshold = STYLE_DEFAULTS.get(style, STYLE_DEFAULTS["balanced"])
+    out = dict(config or {})
+    out.setdefault("style", style)
+    out.setdefault("max_findings", max_findings)
+    out.setdefault("severity_threshold", severity_threshold)
+    return out
+
+
+_TEST_PATH_RE = re.compile(
+    r"(?:^|/)("
+    r"[^/]*[Tt]est\.[A-Za-z]+"           # FooTest.java / foo_test.py
+    r"|[^/]*\.[Tt]est\.[A-Za-z]+"        # foo.Test.java
+    r"|[^/]*_test\.py"                   # foo_test.py
+    r"|test_[^/]*\.py"                   # test_foo.py
+    r"|__tests__/[^/]+"                  # __tests__/foo.js
+    r"|[^/]*\.spec\.[A-Za-z]+"           # foo.spec.ts
+    r")$"
+)
+
+
+def is_test_path(path: str) -> bool:
+    """Heuristic: is `path` a test file by name/path convention?
+
+    Conservative — false positives cost real findings; false negatives just
+    produce one extra line in the summary. Patterns: `FooTest.java`,
+    `foo_test.py`, `test_foo.py`, `__tests__/foo.js`, `foo.spec.ts`, anything
+    ending in `.Test.java`.
+    """
+    if not path:
+        return False
+    return bool(_TEST_PATH_RE.search(path))
+
+
+def _glob_to_regex(glob: str) -> re.Pattern:
+    """Translate a shell-style glob to a compiled regex.
+
+    Supports `*` (any chars except `/`), `**` (any chars including `/`),
+    `?` (single non-`/` char). Other characters are escaped. Used by
+    `apply_repo_config` to test `patterns.allow` / `patterns.deny` globs.
+    """
+    out = []
+    i = 0
+    while i < len(glob):
+        c = glob[i]
+        if c == "*":
+            if i + 1 < len(glob) and glob[i + 1] == "*":
+                out.append(".*")
+                i += 2
+                # swallow a following `/` so `**/x` and `x/**/y` behave
+                if i < len(glob) and glob[i] == "/":
+                    i += 1
+                continue
+            out.append("[^/]*")
+        elif c == "?":
+            out.append("[^/]")
+        else:
+            out.append(re.escape(c))
+        i += 1
+    return re.compile("^" + "".join(out) + "$")
+
+
+def apply_repo_config(
+    findings: list[dict],
+    config: dict | None,
+    changed_paths: list[str] | None = None,
+) -> tuple[list[dict], list[dict]]:
+    """Filter + cap findings per `.pr-review.json` rules. Returns (kept, dropped).
+
+    Filters applied (in order):
+      1. `exclude_tests` + test-path heuristic → drop test files
+      2. `exclude_paths` glob match            → drop matched paths
+      3. `patterns.deny` glob match            → drop matched paths
+      4. `patterns.allow` (if non-empty)       → keep ONLY matched paths
+      5. `severity_threshold`                 → drop below threshold
+      6. `max_findings`                        → keep first N (highest-severity-first)
+      7. `require_tests`                       → append a low-severity finding
+         if changed paths include non-test files but no test files changed
+         alongside them (caller passes `changed_paths` from the brief).
+    """
+    eff = effective_config(config)
+    keep: list[dict] = []
+    drop: list[dict] = []
+    deny_globs = [_glob_to_regex(g) for g in (eff.get("patterns", {}) or {}).get("deny", [])]
+    allow_globs = [_glob_to_regex(g) for g in (eff.get("patterns", {}) or {}).get("allow", [])]
+    deny_path_globs = [_glob_to_regex(g) for g in eff.get("exclude_paths", [])]
+    threshold_rank = SEVERITY_RANK[eff["severity_threshold"]]
+
+    for f in findings:
+        path = f.get("path", "")
+        if eff.get("exclude_tests") and is_test_path(path):
+            drop.append(f); continue
+        if any(rx.search(path) for rx in deny_path_globs):
+            drop.append(f); continue
+        if any(rx.search(path) for rx in deny_globs):
+            drop.append(f); continue
+        if allow_globs and not any(rx.search(path) for rx in allow_globs):
+            drop.append(f); continue
+        sev_rank = SEVERITY_RANK.get(f.get("severity", "low"), 0)
+        if sev_rank < threshold_rank:
+            drop.append(f); continue
+        keep.append(f)
+
+    cap = eff["max_findings"]
+    if len(keep) > cap:
+        dropped = keep[cap:]
+        keep = keep[:cap]
+        drop.extend(dropped)
+
+    if eff.get("require_tests") and changed_paths is not None:
+        non_test = [p for p in changed_paths if not is_test_path(p)]
+        any_test = any(is_test_path(p) for p in changed_paths)
+        if non_test and not any_test:
+            keep.append({
+                "severity": "low",
+                "path": non_test[0],
+                "line": 1,
+                "problem": "no test file changed alongside this behavioral change (require_tests=true)",
+                "fix": "add a unit test exercising the changed branch",
+                "suggestion": "",
+                "reference": "",
+                "_config_synthetic": True,
+            })
+
+    return keep, drop
+
+
+def reviewed_shas(reviews: list[dict]) -> set[str]:
+    """Pull every `<!-- pragent:sha=... -->` marker out of a PR's reviews."""
+    shas: set[str] = set()
+    for r in reviews or []:
+        body = r.get("body") or ""
+        for m in pipeline._SHA_MARKER_RE.finditer(body):
+            shas.add(m.group(1))
+    return shas
+
+
+def prior_review_bodies(reviews: list[dict], current_sha: str, limit: int = 6) -> list[str]:
+    """Bodies of prior bot reviews (older shas), newest-first, bounded."""
+    out = []
+    for r in reviews or []:
+        body = (r.get("body") or "").strip()
+        if not body:
+            continue
+        shas = pipeline._SHA_MARKER_RE.findall(body)
+        # Skip the current sha (that would be a self-reference) and non-bot
+        # noise; keep reviews that carry our marker.
+        if not shas:
+            continue
+        if current_sha and current_sha in shas:
+            continue
+        out.append(body)
+    return out[:limit]
+
+
+def compact_prior_reviews(prior_bodies: list[str]) -> list[str]:
+    """Squeeze prior review bodies down to just the finding bullets.
+
+    Each prior review's prose ("this PR adds eval() — risky") is noise when the
+    model already has the diff; the only thing it needs to *not repeat* is what
+    was already flagged. We extract lines matching `-\\s*\\*\\*[SEV]\\*\\*`
+    plus their directly-attached location reference (so `[CRITICAL]` stays
+    anchored to `path:line`), drop the rest, and return one bullet-list per
+    prior review. A prior review that had no parseable findings becomes an
+    empty string and is dropped.
+
+    Local import keeps the ollama path dep-free (extract_finding_bullets lives
+    in pilot/diff_compress.py).
+    """
+    from diff_compress import extract_finding_bullets
+    out = []
+    for body in prior_bodies or []:
+        bullets = extract_finding_bullets(body)
+        if bullets:
+            out.append("\n".join(bullets))
+    return out
+
+
+# ---------------------------------------------------------------------------

pilot/review/diff.py

@@ -0,0 +1,254 @@
+#!/usr/bin/env python3
+r"""pragent pilot — diff compression + prior-review compaction.
+
+Two pure helpers that shrink what lands in the model prompt without losing
+signal:
+
+  * ``compress_diff(diff, *, context=2)`` — re-renders a unified diff so each
+    hunk keeps only ``context`` unchanged lines on either side of its +/- lines.
+    The default 2 matches what most reviewers see on GitHub/Gitea, and is
+    enough to anchor every ``+``/``-`` line and give the reviewer the enclosing
+    statement. Wider context = more reading; narrower = less. Set
+    ``context=0`` for +/- only, ``context=-1`` to disable entirely.
+
+    Elided context is not merely deleted: each surviving run of lines is
+    re-emitted as its *own* ``@@ -a,b +c,d @@`` hunk with recomputed line
+    numbers, so the output stays a valid unified diff whose line numbers
+    still describe the post-change file. ``parse_diff_anchors`` (and the
+    model) therefore read the same line numbers before and after compression.
+
+  * ``extract_finding_bullets(review_body)`` — pulls the lines of a prior
+    review that look like a pragent finding (``- 🔴 [HIGH] `path:line` — …``,
+    or the older ``- **[HIGH]** …`` form) and drops everything else. The model
+    already has the diff — repeating the prose ("this PR adds eval() — risky")
+    is just token burn. Bullet-only priors cut ~75% off prior-review bytes on
+    a typical 4-finding review.
+
+Stdlib only. No I/O. Tolerant of malformed input — never raises.
+"""
+
+from __future__ import annotations
+
+import re
+
+# A real hunk header: `@@ -old[,count] +new[,count] @@[ trailing section]`.
+# Captures both starts, both counts, and the trailing function-context text.
+# Matching the full shape (not just a `@@` prefix) matters: a *removed* line
+# whose content begins with `@@` is body, not a header.
+_HUNK_RE = re.compile(
+    r"^@@\s+-(\d+)(?:,(\d+))?\s+\+(\d+)(?:,(\d+))?\s+@@(.*)$"
+)
+
+# Match a pragent summary-bullet line, in any of the shapes the renderer has
+# emitted: `- 🔴 [HIGH] \`path:line\` — …` (current, `_severity_badge`),
+# `- **[HIGH]** …` (bold, pre-badge), `- [high] …` (plain, oldest).
+# Anything between the bullet marker and `[SEV]` (emoji, bold markers,
+# whitespace) is tolerated — it is decoration, not signal.
+_FINDING_BULLET_RE = re.compile(
+    r"^\s*[-*]\s*[^\w\[]*\[(?P<sev>critical|high|medium|low)\]",
+    re.IGNORECASE,
+)
+
+
+def compress_diff(diff: str, *, context: int = 2) -> tuple[str, int, int]:
+    """Re-render `diff` keeping at most `context` unchanged lines around +/-.
+
+    Args:
+        diff: unified-diff text (what `gitea .../pulls/{n}.diff` returns).
+        context: max unchanged lines to keep on each side of a hunk. Use 0
+            for +/- only, -1 to disable compression (raw passthrough).
+
+    Returns:
+        `(text, original_chars, kept_chars)`. `original_chars` is the character
+        length of `diff` as given; `kept_chars` is the character length of
+        `text`. Every emitted hunk header is recomputed to match the lines
+        under it, so the result is a valid unified diff. Lines that are not
+        part of a hunk (`diff --git`, `index …`, `Binary files differ`, mode
+        changes) pass through verbatim.
+    """
+    if not diff:
+        return diff or "", len(diff or ""), len(diff or "")
+    if context < 0:
+        return diff, len(diff), len(diff)
+
+    orig = len(diff)
+    lines = diff.splitlines()
+    out: list[str] = []
+
+    i = 0
+    n = len(lines)
+    while i < n:
+        m = _HUNK_RE.match(lines[i])
+        if m is None:
+            # File header, index line, binary marker, mode change, prose —
+            # anything outside a hunk body. Copy verbatim.
+            out.append(lines[i])
+            i += 1
+            continue
+
+        i += 1
+        body_start = i
+        while i < n and _is_body_line(lines[i]):
+            i += 1
+        body = lines[body_start:i]
+
+        out.extend(
+            _render_hunk(
+                body,
+                old_start=int(m.group(1)),
+                new_start=int(m.group(3)),
+                section=m.group(5) or "",
+                context=context,
+            )
+        )
+
+    text = "\n".join(out) + ("\n" if diff.endswith("\n") else "")
+    if not text.strip():
+        # Nothing survived (or the input was nothing but newlines); fall back
+        # to the original so the worst case is no improvement, not data loss.
+        return diff, orig, orig
+    if len(text) >= orig:
+        # Re-emitted hunk headers can outweigh the context they replace on a
+        # small, densely-changed diff. Never hand back something longer than
+        # what we were given.
+        return diff, orig, orig
+    return text, orig, len(text)
+
+
+def _is_body_line(line: str) -> bool:
+    r"""True if `line` belongs to the current hunk body.
+
+    Hunk bodies contain only ` `/`+`/`-` prefixed lines and `\ No newline at
+    end of file`. An empty line is a context line whose trailing space was
+    stripped (common in mail-formatted diffs), so it counts as body too.
+
+    The check is prefix-based *and* header-aware: a removed line reading
+    `---` or an added line reading `+++` (YAML document separators, setext
+    underlines, `--` SQL comments) is body, not a file header — the previous
+    implementation misread those and silently dropped the rest of the hunk.
+    A new file section always opens with `diff --git`, which ends the body.
+    """
+    if line == "":
+        return True
+    if line.startswith("diff --git ") or line.startswith("Index: "):
+        return False
+    if _HUNK_RE.match(line):
+        return False
+    return line[0] in " +-\\"
+
+
+def _render_hunk(
+    body: list[str],
+    *,
+    old_start: int,
+    new_start: int,
+    section: str,
+    context: int,
+) -> list[str]:
+    r"""Trim `body` to `context` unchanged lines around its +/- lines.
+
+    Each surviving run of consecutive lines is emitted as a standalone hunk
+    with a recomputed ``@@ -a,b +c,d @@`` header, so post-change line numbers
+    stay truthful. A hunk with no +/- lines at all (pure context) is dropped
+    entirely; ``\ No newline at end of file`` markers are dropped as noise.
+
+    Returns the rendered lines (headers included), or [] if nothing survived.
+    """
+    # Number every body line on both sides before anything is dropped.
+    numbered: list[tuple[str, int, int]] = []  # (line, old_no, new_no)
+    old_no, new_no = old_start, new_start
+    for ln in body:
+        if ln.startswith("\\"):
+            continue  # `\ No newline at end of file` — no signal, no numbering
+        kind = ln[0] if ln else " "
+        if kind == "+":
+            numbered.append((ln, -1, new_no))
+            new_no += 1
+        elif kind == "-":
+            numbered.append((ln, old_no, -1))
+            old_no += 1
+        else:
+            numbered.append((ln, old_no, new_no))
+            old_no += 1
+            new_no += 1
+
+    changed = [j for j, (ln, _, _) in enumerate(numbered) if ln[:1] in ("+", "-")]
+    if not changed:
+        return []
+
+    keep: set[int] = set()
+    for k in changed:
+        for j in range(max(0, k - context), min(len(numbered) - 1, k + context) + 1):
+            keep.add(j)
+
+    out: list[str] = []
+    for run in _consecutive_runs(sorted(keep)):
+        chunk = [numbered[j] for j in run]
+        old_count = sum(1 for ln, _, _ in chunk if ln[:1] != "+")
+        new_count = sum(1 for ln, _, _ in chunk if ln[:1] != "-")
+        # A run's start is the first line that exists on that side. When a
+        # side has no lines at all (pure addition / pure deletion), unified
+        # diff convention is `start = line before, count = 0`.
+        old_first = next((o for ln, o, _ in chunk if o >= 0), None)
+        new_first = next((nw for ln, _, nw in chunk if nw >= 0), None)
+        old_hdr = old_first if old_first is not None else max(chunk[0][1], 0)
+        new_hdr = new_first if new_first is not None else max(chunk[0][2], 0)
+        if old_count == 0:
+            old_hdr = _side_start_before(numbered, run[0], side=1)
+        if new_count == 0:
+            new_hdr = _side_start_before(numbered, run[0], side=2)
+        out.append(
+            f"@@ -{old_hdr},{old_count} +{new_hdr},{new_count} @@{section}"
+        )
+        out.extend(ln for ln, _, _ in chunk)
+    return out
+
+
+def _side_start_before(
+    numbered: list[tuple[str, int, int]], idx: int, *, side: int
+) -> int:
+    """Line number on `side` (1=old, 2=new) just before body index `idx`.
+
+    Used for the zero-count header form (`@@ -7,0 +8,3 @@`), where unified
+    diff names the line the change is inserted *after*.
+    """
+    for j in range(idx - 1, -1, -1):
+        no = numbered[j][side]
+        if no >= 0:
+            return no
+    # Nothing before it: derive from the first numbered line on that side.
+    for _, old_no, new_no in numbered:
+        no = old_no if side == 1 else new_no
+        if no >= 0:
+            return max(no - 1, 0)
+    return 0
+
+
+def _consecutive_runs(indices: list[int]) -> list[list[int]]:
+    """Group a sorted index list into runs of consecutive integers."""
+    runs: list[list[int]] = []
+    for j in indices:
+        if runs and j == runs[-1][-1] + 1:
+            runs[-1].append(j)
+        else:
+            runs.append([j])
+    return runs
+
+
+def extract_finding_bullets(review_body: str) -> list[str]:
+    """Pull the finding-bullet lines out of a prior review body.
+
+    Returns the matching lines stripped of surrounding whitespace, preserving
+    the rendered ``[SEV] `path:line` — problem`` shape (badge emoji and bold
+    markers included, whichever the renderer used). Lines that look like
+    bullets but carry no severity tag are dropped — the reviewer synthesizes
+    from the matched ones. Continuation lines (`  - **Fix:** …`) are not
+    finding lines and are dropped with the rest of the prose.
+    """
+    if not review_body:
+        return []
+    out = []
+    for line in review_body.splitlines():
+        if _FINDING_BULLET_RE.match(line):
+            out.append(line.strip())
+    return out

pilot/review/model.py

@@ -0,0 +1,36 @@
+"""Model-provider adapter for the legacy Anthropic-compatible endpoint."""
+from __future__ import annotations
+
+import json
+
+try:  # Works both as `python pilot/ai_review.py` and `import pilot.model_client`.
+    from .gitea_client import request
+except ImportError:  # pragma: no cover - script-style runtime
+    from gitea_client import request
+
+
+def parse_text_blocks(content: object) -> str:
+    """Return only text blocks from an Anthropic-style response."""
+    if not isinstance(content, list):
+        return ""
+    return "\n".join(
+        block["text"]
+        for block in content
+        if isinstance(block, dict)
+        and block.get("type") == "text"
+        and isinstance(block.get("text"), str)
+    ).strip()
+
+
+def complete(base_url: str, model: str, system: str, user: str, max_tokens: int) -> str:
+    payload = {
+        "model": model,
+        "max_tokens": max_tokens,
+        "system": system,
+        "messages": [{"role": "user", "content": user}],
+    }
+    status, raw = request("POST", f"{base_url.rstrip('/')}/v1/messages", "ollama", payload)
+    if status != 200:
+        detail = raw[:500].decode("utf-8", errors="replace")
+        raise RuntimeError(f"model call failed: HTTP {status}: {detail}")
+    return parse_text_blocks(json.loads(raw).get("content", []))

pilot/review/output.py

@@ -0,0 +1,764 @@
+from __future__ import annotations
+
+import base64
+import json
+import os
+import re
+import sys
+import urllib.error
+import urllib.parse
+import urllib.request
+
+from . import pipeline
+from .pipeline import *
+from .pipeline import _CONFIDENCE_BADGE, REVIEW_HEADER, SHA_MARKER
+
+# Diff parsing — find valid post-change (RIGHT-side) line anchors per file
+# ---------------------------------------------------------------------------
+
+
+def parse_diff_anchors(diff: str) -> dict[str, set[int]]:
+    """Parse a unified diff into {path: {new_line, ...}} for lines that exist in
+    the post-change version (context + added lines). Removed lines are NOT
+    anchors (they have no RIGHT-side line). Used to validate inline comments.
+
+    Robust to:
+      - `diff --git a/x b/x` and `+++ b/x` path headers (uses the `b/` side)
+      - hunk headers `@@ -a,b +c,d @@` (new line counter starts at c)
+      - No-newline-at-eof markers, binary files, missing hunks.
+    """
+    anchors: dict[str, set[int]] = {}
+    current_path: str | None = None
+    new_line = 0
+    for raw in (diff or "").splitlines():
+        # File path: prefer the `+++ b/` line (handles renames); fall back to
+        # `diff --git a/x b/x`'s second path.
+        if raw.startswith("+++ "):
+            p = raw[4:].strip()
+            if p == "/dev/null":
+                current_path = None
+            else:
+                current_path = _strip_path_prefix(p)
+                anchors.setdefault(current_path, set())
+            continue
+        if raw.startswith("diff --git "):
+            # `diff --git a/foo b/foo` — take the second path as a fallback in
+            # case the `+++` line is missing (binary). Split on " b/".
+            m = re.search(r" b/(.+)$", raw)
+            if m:
+                current_path = m.group(1).strip()
+                anchors.setdefault(current_path, set())
+            continue
+        if raw.startswith("@@"):
+            m = re.search(r"\+(\d+)(?:,\d+)?\s@@", raw)
+            new_line = int(m.group(1)) if m else 0
+            continue
+        if current_path is None:
+            continue
+        if raw.startswith("\\ No newline"):
+            continue
+        if raw.startswith("-"):
+            # removed line — no RIGHT-side anchor
+            continue
+        if raw.startswith("+"):
+            anchors[current_path].add(new_line)
+            new_line += 1
+            continue
+        # Context line: normally " text", but an empty context line arrives as
+        # "" whenever something along the way stripped trailing whitespace (some
+        # forges, some patch tools, copy/paste). Treating "" as "not a line"
+        # would desync `new_line` for the whole rest of the hunk and silently
+        # misplace every later inline comment in the file, so count it.
+        if raw.startswith(" ") or raw == "":
+            anchors[current_path].add(new_line)
+            new_line += 1
+    return anchors
+
+
+def _strip_path_prefix(p: str) -> str:
+    """`b/foo` or `foo` -> `foo`."""
+    if p.startswith("b/"):
+        return p[2:]
+    return p
+
+
+# ---------------------------------------------------------------------------
+# Model output parsing — tolerant JSON findings extraction
+# ---------------------------------------------------------------------------
+
+
+# How many raw findings the last `parse_review_output` / `parse_findings` call
+# rejected for an unusable path/line. A side channel rather than a return value
+# because both parsers already return fixed-width tuples that several callers
+# and their tests unpack positionally; widening them to carry a telemetry
+# number would be a breaking change for a fail-open signal.
+_LAST_PARSE_DROPPED: dict[str, int] = {"n": 0}
+
+
+def last_parse_dropped() -> int:
+    """Findings the last parse discarded. Read it immediately after parsing."""
+    return int(_LAST_PARSE_DROPPED.get("n") or 0)
+
+
+def _normalize_finding(f: dict) -> dict | None:
+    """Validate + normalize one raw finding dict. Returns None if it's unusable
+    (missing path/line). Normalises severity, keeps `reference` (default "")."""
+    if not isinstance(f, dict):
+        return None
+    path = f.get("path")
+    line = f.get("line")
+    if not isinstance(path, str) or not path.strip():
+        return None
+    if not isinstance(line, int) or line < 1:
+        return None
+    sev = str(f.get("severity", "medium")).strip().lower()
+    if sev not in SEVERITIES:
+        sev = "medium"
+    reference = str(f.get("reference", "") or "").strip()
+    return {
+        "severity": sev,
+        "path": path.strip(),
+        "line": line,
+        "problem": str(f.get("problem", "")).strip(),
+        "fix": str(f.get("fix", "")).strip(),
+        "suggestion": str(f.get("suggestion", "") or "").strip(),
+        "reference": reference,
+    }
+
+
+def _last_json_block(text: str) -> str | None:
+    r"""Return the substring of the last JSON object/array in text, or None.
+
+    The pragent agent emits ```json fences around its final block, but real
+    outputs drift:
+      * the fence contains nested objects (regex ``\{.*?\}`` only matches the
+        first ``}``, truncating the JSON — the parser then sees
+        ``json.JSONDecodeError``);
+      * the fence is missing or unterminated, but a balanced JSON object sits
+        in the prose tail;
+      * the agent emits a bare array (findings only, no summary wrapper).
+
+    Strategy:
+      1. Find each fenced block, take the last. Inside it, walk a balanced
+         ``{...}``/``[...]`` scanner (not a regex) so nested structures survive.
+      2. Fall back to a balanced scanner over the whole text, picking the LAST
+         balanced object/array (the agent writes its conclusion last).
+    """
+    s = text or ""
+    if not s:
+        return None
+    # 1. Fenced blocks: take the last ```json ... ``` or ``` ... ``` region.
+    fences = list(re.finditer(r"```(?:json)?\n", s))
+    for m in reversed(fences):
+        start = m.end()
+        # Find the matching closing fence.
+        end = s.find("```", start)
+        if end < 0:
+            # Unterminated fence — try to salvage the balanced object inside.
+            end = len(s)
+        inner = s[start:end].strip()
+        obj = _balanced_json_substring(inner)
+        if obj is not None:
+            return obj
+    # 2. No (parseable) fence — scan the whole text for the LAST balanced
+    #    object/array. The agent's conclusion is at the tail.
+    return _last_balanced_json(s)
+
+
+def parse_findings(text: str) -> list[dict]:
+    """Parse the model's JSON response into a list of finding dicts.
+
+    Tolerant: strips ```json fences, and if the model wrapped JSON in prose,
+    scans for the first balanced `{...}` and extracts its `findings` array.
+    Drops findings missing path/line or with an unknown severity (normalised).
+    Never raises — returns [] on any parse failure.
+
+    Also accepts a bare JSON array as the outer value: ``[{...}, {...}]`` —
+    some agents skip the ``{"summary":..., "findings":[...]}`` wrapper.
+    """
+    _LAST_PARSE_DROPPED["n"] = 0
+    data = _parse_json_tolerant(text)
+    if isinstance(data, dict):
+        findings = data.get("findings")
+    elif isinstance(data, list):
+        findings = data
+    else:
+        return []
+    if not isinstance(findings, list):
+        return []
+    out = []
+    for f in findings:
+        n = _normalize_finding(f)
+        if n is not None:
+            out.append(n)
+    _LAST_PARSE_DROPPED["n"] = len(findings) - len(out)
+    return out
+
+
+SALVAGE_MAX_CHARS = 4000
+
+
+def salvage_summary(text: str, max_chars: int = SALVAGE_MAX_CHARS) -> str:
+    """Recover something postable from agent output we could not parse.
+
+    An opencode run costs minutes and millions of tokens. When the findings JSON
+    is missing or malformed, the analysis itself is usually still there in the
+    prose — discarding it to post "no parseable output" throws away the whole
+    run and tells the maintainer nothing. This keeps the tail of the prose (the
+    conclusion, which is what the agent writes last), drops fenced code blocks
+    so a half-written JSON blob doesn't dominate, and labels it plainly as
+    unstructured so nobody mistakes it for a normal review.
+
+    Returns "" when there is genuinely nothing to salvage.
+    """
+    if not text or not text.strip():
+        return ""
+    # Drop fenced blocks — a truncated ```json block is noise here.
+    prose = re.sub(r"```.*?```", "", text, flags=re.DOTALL)
+    prose = re.sub(r"```.*$", "", prose, flags=re.DOTALL)  # unterminated fence
+    prose = prose.strip()
+    if not prose:
+        return ""
+    if len(prose) > max_chars:
+        prose = "…" + prose[-max_chars:]
+    return (
+        "⚠️ _The reviewer did not emit a parseable findings block, so there are "
+        "no inline comments. Its raw notes are below — treat them as unverified: "
+        "line numbers were not validated against the diff._\n\n" + prose
+    )
+
+
+def parse_review_output(
+    text: str,
+) -> tuple[str, list[dict], list[str], list[str], list[str], str, str]:
+    """Parse the opengine's stdout into a 7-tuple:
+        (summary, findings, summary_changes, risks,
+         walkthrough, risk_verdict, test_coverage)
+
+    Accepts `{"summary": "...", "summary_changes": [...], "risks": [...],
+    "walkthrough": [...], "risk_verdict": "...", "test_coverage": "...",
+    "findings": [...]}` (the opencode pragent agent), the legacy 4-field
+    shape, or a bare `[...]` of finding dicts. The three new fields
+    (`walkthrough`, `risk_verdict`, `test_coverage`) default to empty
+    list / empty strings when absent — older outputs and the bare-array
+    shape stay backward compatible.
+
+    Uses the LAST fenced block (the pragent agent emits JSON as the final
+    block), with a tolerant fallback that scans for the last balanced
+    object/array in the prose tail. Never raises.
+    """
+    _LAST_PARSE_DROPPED["n"] = 0
+    blob = _last_json_block(text)
+    if blob is None:
+        return "", [], [], [], [], "", ""
+    try:
+        data = json.loads(blob)
+    except json.JSONDecodeError:
+        return "", [], [], [], [], "", ""
+    summary = ""
+    summary_changes: list[str] = []
+    risks: list[str] = []
+    walkthrough: list[str] = []
+    risk_verdict = ""
+    test_coverage = ""
+    findings_raw = None
+    if isinstance(data, dict):
+        summary = str(data.get("summary", "") or "").strip()
+        summary_changes = _string_list(data.get("summary_changes"))
+        risks = _string_list(data.get("risks"))
+        walkthrough = _string_list(data.get("walkthrough"))
+        risk_verdict = str(data.get("risk_verdict", "") or "").strip()
+        test_coverage = str(data.get("test_coverage", "") or "").strip()
+        findings_raw = data.get("findings")
+    elif isinstance(data, list):
+        # Bare array: each item is a finding; no summary/sections.
+        findings_raw = data
+    else:
+        return "", [], [], [], [], "", ""
+    out = []
+    if isinstance(findings_raw, list):
+        for f in findings_raw:
+            n = _normalize_finding(f)
+            if n is not None:
+                out.append(n)
+        # A model that emits findings at unusable locations is indistinguishable
+        # from one that found nothing, because both end up with an empty `out`.
+        # Stash the delta so the caller can score it (see `eval_scores`).
+        _LAST_PARSE_DROPPED["n"] = len(findings_raw) - len(out)
+    else:
+        _LAST_PARSE_DROPPED["n"] = 0
+    return summary, out, summary_changes, risks, walkthrough, risk_verdict, test_coverage
+
+
+def _string_list(value) -> list[str]:
+    """Coerce a JSON value into a list of non-empty strings.
+
+    Accepts a list of strings, a single string (split on lines/bullets), or
+    anything else (returns []). Used for `summary_changes` and `risks`,
+    which some agents emit as one big string instead of a list.
+    """
+    if isinstance(value, list):
+        return [str(v).strip() for v in value if str(v).strip()]
+    if isinstance(value, str):
+        s = value.strip()
+        if not s:
+            return []
+        # Split on newlines OR on lines that start with "- " / "* " (markdown
+        # bullets). Strip the bullet markers.
+        out: list[str] = []
+        for line in s.splitlines():
+            line = line.strip()
+            if not line:
+                continue
+            if line[:2] in ("- ", "* "):
+                line = line[2:].strip()
+            if line:
+                out.append(line)
+        return out
+    return []
+
+
+def _parse_json_tolerant(text: str) -> dict | list | None:
+    """Parse a JSON object/array from text: try the last fenced block, then a
+    direct parse, then the first balanced object. Returns None on any failure.
+    Accepts both ``{...}`` (the pragent schema) and bare ``[...]`` arrays
+    (agents that skip the wrapper)."""
+    if not text:
+        return None
+    blob = _last_json_block(text)
+    if blob is not None:
+        try:
+            d = json.loads(blob)
+            if isinstance(d, (dict, list)):
+                return d
+        except json.JSONDecodeError:
+            pass
+    s = text.strip()
+    if s.startswith("```"):
+        s = re.sub(r"^```[a-zA-Z]*\n?", "", s)
+        s = re.sub(r"\n?```$", "", s).strip()
+    try:
+        d = json.loads(s)
+        if isinstance(d, (dict, list)):
+            return d
+    except json.JSONDecodeError:
+        pass
+    obj = _extract_first_json_object(text)
+    if obj is not None:
+        try:
+            d = json.loads(obj)
+            if isinstance(d, (dict, list)):
+                return d
+        except json.JSONDecodeError:
+            pass
+    # Last resort: the JSON lives at the tail of the prose with no fence.
+    # Walk the whole text for the last balanced object/array.
+    last = _last_balanced_json(text)
+    if last is not None:
+        try:
+            d = json.loads(last)
+            if isinstance(d, (dict, list)):
+                return d
+        except json.JSONDecodeError:
+            pass
+    return None
+
+
+def _extract_first_json_object(s: str) -> str | None:
+    """Return the substring of the first balanced top-level `{ ... }` in s."""
+    start = s.find("{")
+    if start < 0:
+        return None
+    end = _scan_balanced(s, start, "{", "}")
+    if end is None:
+        return None
+    return s[start:end + 1]
+
+
+def _last_balanced_json(s: str) -> str | None:
+    """Return the substring of the LAST balanced ``{...}`` or ``[...]`` in s.
+
+    Used when the agent emits no fence: the JSON lives in the prose tail.
+    Picks whichever closer (object or array) appears latest in the text.
+    """
+    if not s:
+        return None
+    last_obj = _find_last_close(s, "{", "}")
+    last_arr = _find_last_close(s, "[", "]")
+    candidates = []
+    if last_obj is not None:
+        candidates.append(last_obj)
+    if last_arr is not None:
+        candidates.append(last_arr)
+    if not candidates:
+        return None
+    end, opener, start = max(candidates, key=lambda t: t[0])
+    return s[start:end + 1]
+
+
+def _balanced_json_substring(s: str) -> str | None:
+    """Return the first balanced ``{...}`` or ``[...]`` substring in ``s``.
+
+    Skips past leading whitespace/non-JSON and returns the full balanced
+    extent (handles nested objects/arrays and string literals with braces).
+    """
+    if not s:
+        return None
+    # Try object first; the pragent schema is an object on the outer level.
+    for i, c in enumerate(s):
+        if c == "{":
+            end = _scan_balanced(s, i, "{", "}")
+            if end is not None:
+                return s[i:end + 1]
+            break
+        if c == "[":
+            end = _scan_balanced(s, i, "[", "]")
+            if end is not None:
+                return s[i:end + 1]
+            break
+    return None
+
+
+def _scan_balanced(s: str, start: int, opener: str, closer: str) -> int | None:
+    """Return the index of the matching ``closer`` for ``s[start] == opener``.
+
+    Tracks string literals (with ``\\`` escapes) so braces inside strings don't
+    fool the depth counter. Returns None if no balance is reached.
+    """
+    depth = 0
+    in_str = False
+    esc = False
+    for i in range(start, len(s)):
+        c = s[i]
+        if in_str:
+            if esc:
+                esc = False
+            elif c == "\\":
+                esc = True
+            elif c == '"':
+                in_str = False
+            continue
+        if c == '"':
+            in_str = True
+        elif c == opener:
+            depth += 1
+        elif c == closer:
+            depth -= 1
+            if depth == 0:
+                return i
+    return None
+
+
+def _find_last_close(s: str, opener: str, closer: str) -> tuple[int, str, int] | None:
+    """Walk ``s`` backwards from the last ``closer`` to find its matching opener.
+
+    Returns ``(close_idx, opener_char, open_idx)`` for the rightmost balanced
+    structure, or None if no pair exists.
+    """
+    # Find the last `closer` candidate.
+    last = s.rfind(closer)
+    while last >= 0:
+        # Walk left, tracking depth from the perspective of the opener.
+        depth = 1
+        in_str = False
+        esc = False
+        for j in range(last - 1, -1, -1):
+            c = s[j]
+            if in_str:
+                if esc:
+                    esc = False
+                elif c == "\\":
+                    esc = True
+                elif c == '"':
+                    in_str = False
+                continue
+            if c == '"':
+                # Approximation: we don't track quotes perfectly walking
+                # backwards, but strings in agent output are short and rare.
+                in_str = not in_str
+            elif c == closer:
+                depth += 1
+            elif c == opener:
+                depth -= 1
+                if depth == 0:
+                    return (last, opener, j)
+        last = s.rfind(closer, 0, last)
+    return None
+
+
+def split_findings(findings: list[dict], anchors: dict[str, set[int]]) -> tuple[list[dict], list[dict]]:
+    """Split findings into (anchored, unanchored).
+
+    A finding is anchored if its path is known AND its line is a valid post-change
+    line for that path. Lines just outside the diff (model off-by-one) are NOT
+    anchored — safer to keep them as summary bullets than to drop or misplace.
+    """
+    anchored, unanchored = [], []
+    for f in findings:
+        valid = anchors.get(f["path"])
+        if valid and f["line"] in valid:
+            anchored.append(f)
+        else:
+            unanchored.append(f)
+    return anchored, unanchored
+
+
+def _lang_for_path(path: str) -> str:
+    """Map a file extension to a chroma language tag for fenced code blocks.
+
+    Used so the suggested-fix block is syntax-highlighted in Gitea. Gitea 1.26.x
+    has no GitHub-style "Apply suggestion" button (the ```suggestion fence is
+    just an unknown-language code block → plain monospace, no apply), so we tag
+    the block with the file's real language for highlighting instead.
+    """
+    ext = path.rsplit(".", 1)[-1].lower() if "." in path else ""
+    return {
+        "java": "java", "kt": "kotlin", "scala": "scala", "groovy": "groovy",
+        "ts": "typescript", "tsx": "tsx", "js": "javascript", "jsx": "jsx",
+        "mjs": "javascript", "cjs": "javascript",
+        "py": "python", "pyi": "python",
+        "go": "go", "rs": "rust", "rb": "ruby", "php": "php",
+        "c": "c", "h": "c", "cpp": "cpp", "cc": "cpp", "hpp": "cpp",
+        "cs": "csharp", "swift": "swift", "m": "objc",
+        "sh": "bash", "bash": "bash", "zsh": "bash",
+        "yml": "yaml", "yaml": "yaml", "json": "json", "jsonc": "json",
+        "toml": "toml", "ini": "ini", "cfg": "ini",
+        "html": "html", "htm": "html", "css": "css", "scss": "scss",
+        "xml": "xml", "svg": "xml", "sql": "sql",
+        "md": "markdown", "dockerfile": "dockerfile",
+    }.get(ext, "")
+
+
+_SEVERITY_EMOJI = {
+    "critical": "🔴",
+    "high": "🔴",
+    "medium": "🟡",
+    "low": "🔵",
+    "trivial": "⚪",
+    "info": "⚪",
+    "nit": "⚪",
+}
+
+# Severities whose own name is rendered verbatim (uppercased) in the badge.
+# Anything outside this set falls back to "INFO" so the badge label stays
+# a clean short token regardless of what the model emits.
+_BADGED_SEVERITY_LABELS = frozenset({
+    "critical", "high", "medium", "low", "trivial", "info", "nit",
+})
+
+
+def _severity_badge(severity: str) -> str:
+    """Render the severity as emoji + uppercase label (e.g. ``🔴 [HIGH]``)."""
+    sev = (severity or "").lower()
+    emoji = _SEVERITY_EMOJI.get(sev, "⚪")
+    label = sev.upper() if sev in _BADGED_SEVERITY_LABELS else "INFO"
+    return f"{emoji} [{label}]"
+
+
+def _format_reference(ref: str) -> str:
+    """Render a reference URL as a clean Markdown hyperlink.
+
+    ``"https://example.com/x"`` → ``"[example.com/x](https://example.com/x)"``.
+    Accepts the bare URL form so older findings still render readably; drops
+    anything that doesn't look like a URL rather than embedding raw text in
+    parens (the spec says: never print raw URLs).
+    """
+    ref = (ref or "").strip()
+    if not ref:
+        return ""
+    if not (ref.startswith("http://") or ref.startswith("https://")):
+        # Non-URL text (e.g. a CVE id, a doc title). Render as plain text —
+        # `[CVE-2024-1](CVE-2024-1)` would render as a broken *relative* link
+        # in Gitea, which is worse than no link at all.
+        return ref
+    # Strip the scheme + www. for the visible label so the link text is short.
+    visible = ref
+    for prefix in ("https://", "http://"):
+        if visible.startswith(prefix):
+            visible = visible[len(prefix):]
+            break
+    if visible.startswith("www."):
+        visible = visible[4:]
+    # Drop trailing slash + truncate any path noise past 60 chars.
+    visible = visible.rstrip("/")
+    if len(visible) > 60:
+        visible = visible[:57] + "…"
+    return f"[{visible}]({ref})"
+
+
+def inline_comment_body(f: dict) -> str:
+    """Render one finding as a positional review-comment body.
+
+    Shape:
+      * Severity badge with emoji (🔴 HIGH / 🟡 MEDIUM / 🔵 LOW / ⚪ INFO).
+      * 1–2 short paragraphs: ``problem`` + optional ``fix``.
+      * ``suggestion`` block (Gitea/Forgejo apply-on-click) when the model
+        produced replacement code. Language-tagged fences are reserved for
+        cross-file patterns the suggestion block can't carry.
+      * Reference as a Markdown hyperlink (``[label](url)``) — never a raw URL.
+      * Per-comment attributed output tokens (`🪙 ~N tok (P% · attributed)`)
+        when the caller passed `compute_attribution` data. Hidden when the
+        finding has no attributed tokens (e.g. legacy callers / ollama path
+        without usage metering).
+    """
+    badge = _severity_badge(f.get("severity", "medium"))
+    body = f"{badge} {f.get('problem', '').strip()}"
+    fix = (f.get("fix") or "").strip()
+    if fix:
+        body += f"\n\n**Fix:** {fix}"
+    suggestion = (f.get("suggestion") or "").strip()
+    if suggestion:
+        # `suggestion` fence is the standard one-click-apply block in
+        # Gitea/Forgejo/GitHub. The agent's replacement lines must already be
+        # indented as in the target file.
+        body += f"\n\n```suggestion\n{suggestion}\n```"
+    ref_md = _format_reference(f.get("reference", ""))
+    if ref_md:
+        body += f"\n\n🔗 **Reference:** {ref_md}"
+    tok = f.get("_tok_attrib")
+    if tok is not None:
+        pct = (f.get("_tok_pct", 0.0) or 0.0) * 100
+        body += f"\n\n🪙 ~{pipeline.fmt_tokens(tok)} tok ({pct:.0f}% · attributed output)"
+    return body
+
+
+def summary_bullets(findings: list[dict]) -> str:
+    """Render unanchored findings as PR-level bullets.
+
+    Used for findings that couldn't be anchored to a post-change line (no
+    inline comment posted). Each bullet carries severity, location, problem,
+    fix, and a Markdown-linked reference.
+    """
+    lines = []
+    for f in findings:
+        loc = f"{f['path']}:{f['line']}" if f["line"] else f["path"]
+        badge = _severity_badge(f.get("severity", "medium"))
+        problem = f.get("problem", "").strip()
+        body = f"- {badge} `{loc}` — {problem}"
+        fix = (f.get("fix") or "").strip()
+        if fix:
+            body += f"\n  - **Fix:** {fix}"
+        ref_md = _format_reference(f.get("reference", ""))
+        if ref_md:
+            body += f"\n  - 🔗 **Reference:** {ref_md}"
+        lines.append(body)
+    return "\n".join(lines)
+
+
+def findings_table(findings: list[dict]) -> str:
+    """Render ALL findings as a Markdown table for the PR-level comment.
+
+    Columns: severity emoji, location (path:line), and a one-line summary.
+    Findings with empty location collapse to just the severity + summary.
+    """
+    if not findings:
+        return ""
+    header = "| Severity | Location | Finding |\n|---|---|---|"
+    rows = []
+    for f in findings:
+        badge = _severity_badge(f.get("severity", "medium"))
+        path = (f.get("path") or "").strip()
+        line = f.get("line")
+        loc = f"`{path}:{line}`" if line else (f"`{path}`" if path else "_(no location)_")
+        problem = (f.get("problem") or "").strip()
+        # Escape pipes inside the finding text so the table stays valid.
+        problem_esc = problem.replace("|", "\\|").replace("\n", " ")
+        rows.append(f"| {badge} | {loc} | {problem_esc} |")
+    return "\n".join([header, *rows])
+
+
+def _render_collapsible_usage(usage: dict | None, model: str, config: dict | None) -> str:
+    """Render the telemetry as a collapsible ``<details>`` block.
+
+    Empty string when `usage` is None. The equivalent-cost table is the
+    operator's budgeting signal — the pilot runs on a free tier, so the
+    `actual` line is $0.00; the table shows what the same measured tokens
+    would bill on mainstream paid APIs (configurable via `compare_against`,
+    defaulting to ``DEFAULT_COMPARE_AGAINST``). The row matching `cost_target`
+    is bolded so the price target stands out. The whole table is omitted when
+    every row would be $0 (no work done). The `actual` parenthetical clause
+    reflects the *actually-routed* model (`model` arg, resolved by caller from
+    `OPENCODE_MODEL` env or `headroom/{OLLAMA_MODEL}`) — cost == 0 → "free
+    tier", nonzero → "billed".
+
+    """
+    if not usage:
+        return ""
+    dur = usage.get("duration_s")
+    dur_s = f"{dur}s" if dur is not None else "?"
+    actual = usage.get("cost") or 0.0
+    actual_s = f"${actual:.4f}" if actual else "$0.00"
+    actual_note = f" ({model} — {'free tier' if not actual else 'billed'})"
+    cost_target, price_err = pipeline._resolve_price_target(config)
+    if price_err:
+        # Surface config typos loudly but do not pollute the posted summary
+        # body — typos at the table-row level would render as English
+        # mid-table and look like a model error.
+        print(f"pragent: {price_err}", file=sys.stderr, flush=True)
+    # Lazy: cost_model has no dep on ai_review, and the ollama path
+    # never reaches this branch.
+    from cost_model import PRICES as _PRICES
+    cfg = config or {}
+    compare: list[str] = list(cfg.get("compare_against") or DEFAULT_COMPARE_AGAINST)
+    # Always include the resolved cost_target (env + config), even when the
+    # operator pinned a different `compare_against` roster — the price target
+    # row is the one maintainers eyeball against. Skip silently if the key
+    # isn't a known Price (e.g. a typo that slipped past stderr earlier).
+    if cost_target in _PRICES and cost_target not in compare:
+        compare.append(cost_target)
+    eq_rows: list[str] = []
+    for key in compare:
+        if key not in _PRICES:
+            continue
+        c = pipeline.equivalent_cost(usage, key)
+        if c <= 0:
+            continue
+        label = _PRICES[key].name
+        cost_str = f"${c:.4f}" if c < 0.01 else f"${c:.2f}"
+        bold = "**" if key == cost_target else ""
+        eq_rows.append(f"| {bold}{label}{bold} | {cost_str} |")
+
+    in_tok = usage.get("input", 0)
+    out_tok = usage.get("output", 0)
+    reason_tok = usage.get("reasoning", 0)
+    cache_r = usage.get("cache_read", 0)
+    cache_w = usage.get("cache_write", 0)
+    total = usage.get("total", 0)
+    scope = (
+        "Whole-repo checkout at head sha (agent can read any file + run "
+        "linters, not just the diff) — input tokens include files read "
+        "beyond the diff. Per-comment output is *attributed* (one model pass "
+        "produces all findings; output split by each finding's body weight)."
+    )
+    lines = [
+        "<details>",
+        "<summary>🔋 AI Usage & Run Details</summary>",
+        "",
+        f"- **Model / Engine**: `{model}` · opencode · {usage.get('steps', 0)} steps · {dur_s}",
+        f"- **Total Tokens**: {pipeline.fmt_tokens(in_tok)} in / {pipeline.fmt_tokens(out_tok)} out "
+        f"({pipeline.fmt_tokens(reason_tok)} reasoning, cache {pipeline.fmt_tokens(cache_r)} read / "
+        f"{pipeline.fmt_tokens(cache_w)} write, {pipeline.fmt_tokens(total)} total)",
+        f"- **Actual**: {actual_s}{actual_note}",
+        f"- **Scope**: {scope}",
+    ]
+    if eq_rows:
+        lines.append("")
+        lines.append("- **Equivalent cost on paid providers** (this run's tokens):")
+        lines.append("")
+        lines.append("| Provider | Cost |")
+        lines.append("|---|---:|")
+        lines.extend(eq_rows)
+    # Multi-lens fan-out: surface the lens roster + summed steps so the user
+    # can see which lenses contributed (and that triage didn't drop them all).
+    lenses = usage.get("lenses")
+    if lenses:
+        ls = usage.get("lens_steps", usage.get("steps", 0))
+        lines.append(
+            f"- **Lenses**: {', '.join(f'`{x}`' for x in lenses)} "
+            f"({len(lenses)} parallel subprocesses, {ls} summed steps)"
+        )
+    lines += ["", "</details>"]
+    return "\n".join(lines)
+
+
+# ---------------------------------------------------------------------------

pilot/review/pipeline.py

@@ -0,0 +1,437 @@
+#!/usr/bin/env python3
+"""pragent pilot — minimal AI PR reviewer.
+
+Runs as a Gitea Actions step OR is called by the central webhook server
+(`webhook_server.py`). Fetches a PR diff, asks glm-5.2:cloud (via the on-network
+headroom proxy, Anthropic /v1/messages format) to review it, and posts the
+findings back as `pragent-bot` — as a **review summary** plus **inline line
+comments** with a fenced suggested-fix block (tagged with the file's language so
+Gitea syntax-highlights it) where the model could produce one and the line
+anchors cleanly to the post-change file.
+
+Features (pilot v2):
+  - **Dedupe / persistence:** Gitea itself is the source of truth. Before
+    reviewing, fetch the PR's existing reviews and look for a hidden
+    `<!-- pragent:sha=... -->` marker matching this commit. If present, skip
+    (no duplicate review on label-toggle / re-fire). Prior review bodies are
+    fed back to the model as "already said" context so a re-push synthesizes
+    instead of repeating (light version of design §6.1).
+  - **Repo-local focus:** if the repo has a `.pr-review.json` at the PR's head
+    ref, its `focus` / `exclude_paths` / `instructions` / `languages` steer the
+    review. Optional — defaults apply when absent.
+  - **Inline comments + suggestions:** the model emits structured JSON
+    findings with `path`/`line`. We parse the diff hunks to learn which
+    `(path, new_line)` pairs are valid post-change anchors and post each
+    anchored finding as a positional review comment; the `suggestion` field, if
+    non-empty, is wrapped in a fenced code block tagged with the file's language
+    (via `_lang_for_path`) so Gitea syntax-highlights it. Gitea 1.26.x has no
+    GitHub-style "Apply suggestion" button, so a language-tagged block is used
+    for highlighting instead of a ```suggestion fence. Findings that don't
+    anchor (bad line, unchanged file, etc.) are folded into the summary body as
+    plain bullets.
+
+Fail-open by design: any error becomes a short "review failed" review comment,
+and review_pr never raises. Stdlib only — no pip install.
+
+Env (CI run() path):
+  GITEA_API            base URL of the in-cluster Gitea
+  GITEA_REPOSITORY     "owner/repo" of the PR (github.repository)
+  PR_INDEX             PR number (github.event.pull_request.number)
+  PR_TITLE             PR title
+  PR_BODY              PR body (optional)
+  PR_BASE_REF          base branch (.pr-review.json is read from here, not the
+                       PR head); optional, defaults to the repo default branch
+  PRAGENT_BOT_TOKEN    bot access token (repo secret)
+  PRAGENT_SHA          head SHA to tag the review
+  OLLAMA_URL           headroom proxy URL, e.g. http://model-proxy.internal:8789
+  OLLAMA_MODEL         model id, e.g. glm-5.2:cloud
+  OLLAMA_MAX_TOKENS    (optional) output cap, default 8000
+  DIFF_MAX_CHARS       (optional) diff truncation cap, default 150000
+"""
+
+import base64
+import json
+import os
+import re
+import sys
+import urllib.error
+import urllib.parse
+import urllib.request
+
+REVIEW_HEADER = "🤖 **AI Review** · pragent pilot · {model} · `{sha}` · Merge confidence: {confidence}"
+# Hidden marker the dedupe pass scans for. Full sha so a re-push (new sha) is
+# never mistaken for an already-reviewed commit, and a label-toggle (same sha)
+# is correctly skipped.
+SHA_MARKER = "<!-- pragent:sha={sha} -->"
+_SHA_MARKER_RE = re.compile(r"<!-- pragent:sha=([0-9a-f]{7,40}) -->")
+
+SEVERITIES = ("critical", "high", "medium", "low", "trivial", "info")
+# Severity rank — higher = more severe. Used by `apply_repo_config` to drop
+# findings below `severity_threshold`. critical=4, high=3, medium=2, low=1,
+# trivial=0, info=-1.
+SEVERITY_RANK = {"info": -1, "trivial": 0, "low": 1, "medium": 2, "high": 3, "critical": 4}
+REPO_CONFIG_FILE = ".pr-review.json"
+
+# Style → (default max_findings, default severity_threshold). Strict is
+# terse/high-signal; lenient shows everything; balanced is the default for
+# unconfigured repos. Repo `.pr-review.json` overrides per-field.
+STYLE_DEFAULTS: dict[str, tuple[int, str]] = {
+    "strict": (5, "high"),
+    "balanced": (12, "medium"),
+    "lenient": (15, "low"),
+}
+
+# Default provider to compare against in the usage section. The pilot runs on
+# headroom/glm-5.2:cloud at $0/MTok, so the actual line shows $0.00 — but the
+# equivalent provider line lets a maintainer see what they would have paid on
+# Claude/GPT for the same measured tokens. Override with PRAGENT_PRICE_TARGET
+# (env) or `.pr-review.json:cost_target` (per repo).
+DEFAULT_PRICE_TARGET = "claude-sonnet-5"
+
+# Default roster of paid providers shown in the equivalent-cost table when
+# `.pr-review.json` does not pin `compare_against`. The pilot is free-tier only,
+# so this list is the operator's budgeting signal — it answers "what would this
+# have cost on a mainstream paid API?". Override per-repo via
+# `.pr-review.json:compare_against` (capped at 12 entries; unknown keys are
+# dropped with a stderr line at parse time).
+DEFAULT_COMPARE_AGAINST = ("claude-sonnet-5", "gpt-5", "gemini-2.5-pro", "grok-4.5")
+
+SYSTEM_PROMPT = """You are a senior, pragmatic code reviewer. Review the pull request diff below.
+
+Report ONLY real, actionable issues: correctness bugs, security problems, risky
+changes, missing tests for changed behaviour, and breaking API/contract changes.
+Honour any repo-specific focus / instructions given in the prompt; if focus is
+given, weight those areas higher, but do not ignore a critical issue outside them.
+
+Output STRICT JSON only — no prose, no markdown fences. Shape:
+{
+  "findings": [
+    {
+      "severity": "critical|high|medium|low|trivial|info",
+      "path": "file path exactly as it appears in the diff (`+++ b/` side)",
+      "line": <int, the NEW-file line number the issue is on, within the diff>,
+      "problem": "one line: what is wrong",
+      "fix": "one line: how to fix it",
+      "suggestion": "<exact replacement lines for that location, or empty string if you cannot produce safe replacement code>"
+    }
+  ],
+  "walkthrough": ["2-6 short bullets, file- or change-grouped, plain prose"],
+  "risk_verdict": "Low|Medium|High|Critical risk: <one-line concrete reason>",
+  "test_coverage": "Tests added" | "Tests changed" | "No tests for behavioral change" | "No test files in repo"
+}
+
+Rules:
+- `line` MUST be a line number that exists in the post-change version of `path`
+  (i.e. a context line or an added `+` line shown in the diff). Never a removed
+  line. If you are unsure of the exact line, set `line` to the closest context
+  line you CAN see in the diff.
+- `suggestion` is the literal new code that should replace the flagged line(s).
+  Keep it minimal — just the changed lines, indented as they would appear in the
+  file. Leave it empty ("") if a safe textual replacement is not possible (e.g.
+  a missing test, an architectural note).
+- `walkthrough`: 2-6 short bullets, file- or change-grouped, plain prose.
+  Default to `[]` when the diff is trivial. Backward compatible: parsers
+  default to `[]` if absent.
+- `risk_verdict`: exactly one line. Lead with "Low|Medium|High|Critical risk:"
+  followed by a concrete reason. Default to `""` when not applicable.
+  Backward compatible: parsers default to `""` if absent.
+- `test_coverage`: short string. One of "Tests added" / "Tests changed" /
+  "No tests for behavioral change" / "No test files in repo". Default to `""`
+  when not applicable. Backward compatible: parsers default to `""` if absent.
+- Skip nitpicks, pure formatting, and praise. At most ~15 findings, highest
+  severity first.
+- If the diff is clean, output: {"findings": []}
+- Do NOT repeat anything already covered in "PREVIOUS REVIEWS" — only surface
+  new or still-unresolved issues."""
+
+# ---------------------------------------------------------------------------
+# Shared render constant retained here for compatibility with the extracted
+# modules and existing callers.
+_CONFIDENCE_BADGE = {5: "🟢", 4: "🟢", 3: "🟡", 2: "🟠", 1: "🔴"}
+
+# Internal modules provide pure transforms and adapters; this file retains
+# the orchestration entry point and backwards-compatible symbols.
+from . import adapters as _adapters
+from . import analysis as _analysis
+from . import configuration as _configuration
+from . import output as _output
+
+for _module in (_analysis, _output, _configuration, _adapters):
+    globals().update({
+        _name: _value
+        for _name, _value in vars(_module).items()
+        if not _name.startswith("__")
+    })
+
+def review_pr(
+    api: str,
+    repo: str,
+    index: str,
+    title: str,
+    body: str,
+    sha: str,
+    token: str,
+    ollama_url: str,
+    model: str,
+    max_tokens: int = 8000,
+    max_chars: int = 150000,
+    base_ref: str = "",
+) -> bool:
+    """Run one review and post it as `pragent-bot`.
+
+    Dedupe: if a prior review already carries this commit's sha marker, skip
+    (no duplicate). Otherwise: fetch repo config + prior-review context, call
+    the model, parse JSON findings, anchor what we can to diff lines, post a
+    review with inline comments + suggestions (unanchored findings → summary
+    bullets).
+
+    `base_ref`: the PR's base branch. `.pr-review.json` is read from there (not
+    from the PR head) so a PR cannot ship its own reviewer instructions; empty
+    means "the repo's default branch".
+
+    The opencode engine's measured token/cost usage is always rendered as a
+    `## 🔋 AI usage` section on the review body and an attributed `🪙 ~N tok`
+    line on each inline comment when usage data is available (i.e. when the
+    opencode subprocess returned a `usage` dict). No-op on the ollama fallback
+    (no usage available — `usage` is None).
+
+    Returns True on success (including a deliberate skip), False on failure
+    (failure note posted when possible). Never raises — fail-open by design.
+    Both the CI `run()` entry point and the central webhook server call this.
+    """
+    try:
+        # Pre-compute a *fallback* display name for the early-exit paths
+        # (already-reviewed dedupe skip, no-diff-content). We re-resolve
+        # properly after `.pr-review.json` is loaded further down — that
+        # version honours `OPENCODE_MODEL` env > `.pr-review.json:model` >
+        # this fallback.
+        display_model = f"headroom/{model}"
+
+        reviews = fetch_existing_reviews(api, repo, index, token)
+        # Dedupe: already reviewed this exact commit -> nothing to do.
+        if sha and sha in reviewed_shas(reviews):
+            print(f"pragent: {repo}#{index} sha={sha[:8]} already reviewed, skipping", flush=True)
+            return True
+
+        raw_diff, _truncated, _orig = fetch_pr_diff(api, repo, index, token, max_chars)
+        if not raw_diff.strip():
+            post_review(api, repo, index, token, format_review_body("No diff content to review.", display_model, sha))
+            return True
+
+        config = fetch_repo_config(api, repo, token, ref=base_ref)
+        prior = compact_prior_reviews(prior_review_bodies(reviews, sha))
+
+        # Re-resolve display_model now that .pr-review.json is available —
+        # per-repo override (`.pr-review.json:model`) takes precedence over
+        # the bare OLLAMA_MODEL fallback, with OPENCODE_MODEL env still
+        # winning above both (see `_resolve_display_model`).
+        display_model = _resolve_display_model(model, config)
+
+        # Trim the diff to +/- hunks plus a narrow context window. The agent
+        # resends the brief prefix every step, so a 25k-char diff becomes
+        # 25k × 30-step × cached-after-step-1 = hundreds of thousands of input
+        # tokens. Default context=1: enough for the reviewer to see what an
+        # added line is replacing; the full file is on disk in the workdir
+        # anyway, so anything more is reading the diff twice. Tunable via
+        # PRAGENT_DIFF_CONTEXT (0 = +/- only; -1 = disable compression).
+        from diff_compress import compress_diff
+        ctx = _int_env("PRAGENT_DIFF_CONTEXT", 1)
+        if ctx < 0:
+            diff = raw_diff
+            compression_note = ""
+        else:
+            diff, orig_chars, kept_chars = compress_diff(raw_diff, context=ctx)
+            if kept_chars < orig_chars:
+                compression_note = (
+                    f"\n\n> _diff compressed: {orig_chars:,} → {kept_chars:,} chars "
+                    f"(context={ctx}; PRAGENT_DIFF_CONTEXT to tune)_"
+                )
+            else:
+                compression_note = ""
+
+        engine = os.environ.get("PRAGENT_ENGINE", "opencode").strip().lower()
+        review_summary = ""
+        # Static repo-provided context (architecture summary, module map, …)
+        # fetched once from `additional_context_urls` (env + .pr-review.json).
+        # Cheap, cached, capped — see fetch_additional_context.
+        additional_context = fetch_additional_context(_resolve_additional_context_urls(config))
+        if engine == "opencode":
+            # The review "brain" runs on opencode: it gets the checked-out repo,
+            # the brief, and the pragent agent factory; returns stdout with a
+            # summary + findings JSON. We parse + anchor + post here.
+            import opencode_review  # local import keeps the ollama path dep-free
+            # Reuse the display_model resolved above for the subprocess — same
+            # provider-prefixed ref goes to the engine and into the review body.
+            oc_model = display_model
+            # Multi-lens fan-out: when the repo declared `reviewers[]` (or the
+            # operator pinned PRAGENT_REVIEWERS=1), spawn one opencode subprocess
+            # per lens in parallel and synthesize. Falls through to the legacy
+            # single-primary path when neither is set.
+            use_lenses = bool((config or {}).get("reviewers")) or bool(
+                os.environ.get("PRAGENT_REVIEWERS")
+            )
+            if use_lenses and hasattr(opencode_review, "run_lenses_review"):
+                stdout, usage = opencode_review.run_lenses_review(
+                    api=api, repo=repo, index=index, sha=sha, token=token,
+                    title=title, body=body, diff=diff, config=config,
+                    prior_reviews=prior, model=oc_model,
+                    compression_note=compression_note,
+                    additional_context=additional_context,
+                )
+            else:
+                stdout, usage = opencode_review.run(
+                    api=api, repo=repo, index=index, sha=sha, token=token,
+                    title=title, body=body, diff=diff, config=config,
+                    prior_reviews=prior, model=oc_model,
+                    compression_note=compression_note,
+                    additional_context=additional_context,
+                )
+            review_summary, findings, summary_changes, risks, _walkthrough, _risk_verdict, _test_coverage = parse_review_output(stdout)
+            parse_dropped = last_parse_dropped()
+            if not findings and not review_summary:
+                # The findings JSON was missing or malformed. Don't discard the
+                # run: salvage the prose, keep the usage report (the tokens were
+                # spent either way), and log enough of the raw output to
+                # diagnose why the agent went off-format.
+                print(
+                    f"pragent: {repo}#{index} sha={sha[:8]} unparseable output "
+                    f"({len(stdout)} chars); tail: {stdout[-600:]!r}",
+                    file=sys.stderr, flush=True,
+                )
+                salvaged = salvage_summary(stdout)
+                usage_section = _render_collapsible_usage(usage, display_model, config=config) if usage else ""
+                post_review(api, repo, index, token, format_review_body(
+                    salvaged or "AI review produced no parseable output.",
+                    display_model, sha, usage_section=usage_section,
+                    static_message=(config or {}).get("static_message", "")))
+                _emit_langfuse(
+                    repo=repo, index=index, sha=sha, title=title,
+                    model=display_model, usage=usage, findings=[],
+                    summary=salvaged, engine=engine, config=config,
+                    dropped_count=parse_dropped,
+                )
+                return True
+        else:
+            user_prompt = build_user_prompt(title, body + compression_note, diff, config, prior, additional_context)
+            raw_findings = call_model(ollama_url, model, SYSTEM_PROMPT, user_prompt, max_tokens)
+            findings = parse_findings(raw_findings)
+            parse_dropped = last_parse_dropped()
+            usage = None
+
+        # Filter / cap findings per `.pr-review.json` (style, threshold, max,
+        # patterns, exclude_tests). Without this every config knob would be a
+        # no-op — the agent has no view into the config beyond instructions.
+        # The synthetic require_tests finding (if any) is appended here.
+        try:
+            changed_paths = sorted({
+                f.get("path", "")
+                for f in findings
+                if f.get("path")
+            })
+        except Exception:
+            changed_paths = []
+        # Capture cross-lens agreement BEFORE apply_repo_config — by the time
+        # findings land in `review_pr` the `_multi_lens` marker has already
+        # been scrubbed (once by `opencode_review.run_lenses_review`'s
+        # `_`-prefix strip, again by `_normalize_finding`'s 7-key rebuild),
+        # so `merge_confidence` cannot read it off the dict. We scan here as
+        # the convergence point for both engine paths; in practice the kwarg
+        # currently always passes False, but the structural plumbing is
+        # correct for any future code path that preserves the flag.
+        multi_lens = any(f.get("_multi_lens") for f in findings)
+        kept, _dropped = apply_repo_config(findings, config, changed_paths=changed_paths)
+        findings = kept
+        if _dropped:
+            print(
+                f"pragent: {repo}#{index} sha={sha[:8]} filtered "
+                f"{len(_dropped)} finding(s) per .pr-review.json "
+                f"(style={(config or {}).get('style', 'balanced')}, "
+                f"threshold={(config or {}).get('severity_threshold', '?')}, "
+                f"max={len(findings)})",
+                flush=True,
+            )
+
+        # Compute attribution so inline comments + the table can show per-comment
+        # estimates. Only meaningful when we have measured usage.
+        if usage and usage.get("output"):
+            compute_attribution(findings, usage["output"])
+        usage_section = _render_collapsible_usage(usage, display_model, config=config) if usage else ""
+
+        # Anchor against the RAW diff, never the compressed one. Compression
+        # drops context lines, so a finding on a line that survived in the file
+        # but not in the prompt would be demoted to a bullet for no reason.
+        # (compress_diff renumbers its hunks, so both are line-accurate; the
+        # raw diff is simply the complete set.)
+        anchors = parse_diff_anchors(raw_diff)
+        anchored, unanchored = split_findings(findings, anchors)
+
+        # Summary body: unanchored bullets fall through to a "Unanchored notes"
+        # section; the structured Findings Overview table covers both anchored
+        # + unanchored so reviewers see the full set even if inline comments
+        # are collapsed.
+        bullets = summary_bullets(unanchored)
+        summary_parts = []
+        if bullets:
+            summary_parts.append("### Unanchored Notes\n\n" + bullets)
+        # 1-5 merge verdict for the header badge. Computed AFTER filtering +
+        # anchoring so the verdict reflects what the operator sees (a critical
+        # finding that fails to anchor is still a critical finding). The
+        # default 5 keeps any failure path (e.g. empty findings) green.
+        # Cross-lens agreement is passed in via kwarg (see multi_lens scan
+        # above) because the `_multi_lens` flag is stripped before findings
+        # reach this call.
+        confidence = merge_confidence(findings, multi_lens_observed=multi_lens)
+        summary_body = format_review_body(
+            "\n\n".join(summary_parts), display_model, sha,
+            summary=review_summary,
+            usage_section=usage_section,
+            summary_changes=summary_changes,
+            risks=risks,
+            findings_for_table=findings,
+            inline_count=len(anchored),
+            confidence=confidence,
+            static_message=(config or {}).get("static_message", ""),
+        )
+
+        post_inline_review(api, repo, index, token, summary_body, anchored)
+        _emit_langfuse(
+            repo=repo, index=index, sha=sha, title=title,
+            model=display_model, usage=usage, findings=findings,
+            summary=review_summary, engine=engine, config=config,
+            dropped_count=parse_dropped,
+        )
+        print(
+            f"pragent: reviewed {repo}#{index} sha={sha[:8]} "
+            f"engine={engine} findings={len(findings)} inline={len(anchored)}",
+            flush=True,
+        )
+        return True
+    except Exception as e:  # fail-open
+        try:
+            post_review(api, repo, index, token, format_review_body(f"⚠️ AI review failed: {e}", display_model, sha))
+        except Exception as e2:
+            print(f"pragent: could not post failure note: {e2}", file=sys.stderr)
+        print(f"pragent: review failed: {e}", file=sys.stderr)
+        return False
+
+
+def run() -> int:
+    review_pr(
+        api=_need("GITEA_API"),
+        repo=_need("GITEA_REPOSITORY"),
+        index=_need("PR_INDEX"),
+        title=os.environ.get("PR_TITLE", ""),
+        body=os.environ.get("PR_BODY", ""),
+        sha=os.environ.get("PRAGENT_SHA", ""),
+        token=_need("PRAGENT_BOT_TOKEN"),
+        ollama_url=_need("OLLAMA_URL"),
+        model=_need("OLLAMA_MODEL"),
+        max_tokens=_int_env("OLLAMA_MAX_TOKENS", 8000),
+        max_chars=_int_env("DIFF_MAX_CHARS", 150000),
+        base_ref=os.environ.get("PR_BASE_REF", ""),
+    )
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(run())

pilot/review/ports.py

@@ -0,0 +1,17 @@
+"""Stable interfaces shared by the review pipeline and its adapters."""
+from __future__ import annotations
+
+from typing import Protocol
+
+
+class Forge(Protocol):
+    def get(self, path: str, accept: str = "application/json") -> tuple[int, bytes]: ...
+    def post(self, path: str, body: dict) -> tuple[int, bytes]: ...
+
+
+class Reviewer(Protocol):
+    def review(self, system: str, user: str, max_tokens: int) -> str: ...
+
+
+class Telemetry(Protocol):
+    def emit(self, **event: object) -> None: ...

pilot/review_config.py

@@ -0,0 +1,5 @@
+"""Compatibility import for trusted review configuration."""
+import importlib
+import sys
+_module = importlib.import_module("review.config")
+sys.modules[__name__] = _module

pilot/review_ports.py

@@ -0,0 +1,5 @@
+"""Compatibility import for review ports."""
+import importlib
+import sys
+_module = importlib.import_module("review.ports")
+sys.modules[__name__] = _module

pilot/webhook_server.py

@@ -1,319 +1,7 @@
-#!/usr/bin/env python3
-"""pragent pilot — central webhook receiver.
-
-A stdlib-only HTTP server that Gitea posts user-webhook events to. It gates on
-the PR's base ref having `.pr-review.json` with `"enabled": true`, then runs
-the same review core (`ai_review.review_pr`) the CI-step pilot uses, posting
-findings back as `pragent-bot`.
-
-Per-owner setup: one Gitea **user-level webhook** per repo-owner fires for every
-repo that owner has; this service filters to opted-in PRs. (Gitea 1.26.1 system
-webhooks are broken — see pilot/README-webhook.md.) Onboarding a repo = add the
-bot as a Write collaborator + commit a `.pr-review.json` with `"enabled": true`
-on the base ref.
-
-Stdlib only — no pip install, runs on python:3-slim with the scripts mounted.
-
-Endpoints:
-  POST /webhook   Gitea webhook delivery (HMAC-verified)
-  GET  /health    liveness probe
-
-Env:
-  WEBHOOK_SECRET       shared secret used to register the Gitea webhook (HMAC)
-  GITEA_API            in-cluster Gitea base URL
-  PRAGENT_BOT_TOKEN    pragent-bot access token (non-admin; must be a Write
-                       collaborator on each reviewed repo)
-  OLLAMA_URL           headroom proxy URL, e.g. http://model-proxy.internal:8789
-  OLLAMA_MODEL         model id, e.g. glm-5.2:cloud
-  OLLAMA_MAX_TOKENS    (optional) output cap, default 6000
-  DIFF_MAX_CHARS       (optional) diff truncation cap, default 150000
-  WEBHOOK_PORT         (optional) listen port, default 8080
-  PRAGENT_MAX_CONCURRENT_REVIEWS
-                       (optional) how many reviews may run at once, default 2.
-                       Each review forks an opencode process that checks out a
-                       repo and runs linters, so this is the real resource knob.
-  PRAGENT_MAX_BODY_BYTES
-                       (optional) request-body cap, default 10 MiB
-"""
-
-import base64
-import hashlib
-import hmac
-import json
-import os
-import threading
-import urllib.parse
-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
-
-from ai_review import gitea_get, review_pr
-
-try:
-    import feedback_harvest  # optional — absent in CI-step pod, present in
-                              # central webhook service. Harvesting is the
-                              # collection side of the feedback loop.
-except ImportError:
-    feedback_harvest = None
-
-# Pull-request webhook `action` values. We fire on EVERY pull_request action
-# except `closed` (no point reviewing a closed/merged PR) — the
-# `.pr-review.json:enabled` gate + sha dedupe downstream make broadening safe:
-# a same-sha re-fire (title edit, assignee, milestone, label toggle…) is
-# skipped by `review_pr`'s dedupe. Gitea emits GitHub-style `action` names
-# (`labeled`, `synchronize`) even though the `X-Gitea-Event-Type` header uses
-# `label_updated` / `synchronized`.
-SKIP_ACTIONS = {"closed"}
-
-GITEA_API = os.environ.get("GITEA_API", "http://gitea-http.gitea.svc.cluster.local:3000")
-BOT_TOKEN = os.environ.get("PRAGENT_BOT_TOKEN", "")
-OLLAMA_URL = os.environ.get("OLLAMA_URL", "http://model-proxy.internal:8789")
-OLLAMA_MODEL = os.environ.get("OLLAMA_MODEL", "glm-5.2:cloud")
-OLLAMA_MAX_TOKENS = int(os.environ.get("OLLAMA_MAX_TOKENS", "8000"))
-DIFF_MAX_CHARS = int(os.environ.get("DIFF_MAX_CHARS", "150000"))
-WEBHOOK_SECRET = os.environ.get("WEBHOOK_SECRET", "").encode()
-PORT = int(os.environ.get("WEBHOOK_PORT", "8080"))
-MAX_CONCURRENT = max(1, int(os.environ.get("PRAGENT_MAX_CONCURRENT_REVIEWS", "2")))
-MAX_BODY_BYTES = int(os.environ.get("PRAGENT_MAX_BODY_BYTES", str(10 * 1024 * 1024)))
-# Feedback DB — SQLite mounted at PRAGENT_FEEDBACK_DB. Empty / unset =
-# feedback collection disabled (CI-step path doesn't have it).
-FEEDBACK_DB = os.environ.get("PRAGENT_FEEDBACK_DB", "")
-
-# Bound on reviews running at once. Every review forks an opencode process that
-# untars a repo, reads files and shells out to linters, so an unbounded thread
-# per delivery is a self-inflicted fork bomb the first time someone labels ten
-# PRs (or Gitea retries a burst). Queued deliveries wait here rather than pile
-# onto the box; the handler has already returned 202, so nothing times out.
-_review_slots = threading.Semaphore(MAX_CONCURRENT)
-
-# Reviews currently accepted or running, keyed (repo, index, sha). The
-# sha-marker dedupe in `review_pr` reads Gitea *before* posting, so two
-# deliveries for the same commit in flight together both see "not yet reviewed"
-# and both post — the classic check-then-act race. Common triggers are Gitea
-# retries after a slow 202 response and bursty re-fires from a rapid title /
-# assign / label toggle. This set closes the window inside one process.
-_inflight: set[tuple[str, str, str]] = set()
-_inflight_lock = threading.Lock()
-
-
-def is_repo_enabled(api: str, repo: str, ref: str, token: str) -> bool:
-    """True iff `.pr-review.json` on `ref` has `"enabled": true`.
-
-    Reads from the given ref (typically the PR's base ref). False on any
-    failure: 404, parse error, missing file, missing `enabled`, wrong type.
-    The bool-coerce of `.get("enabled") is True` rejects the common
-    gotchas (`null`, `1`, `"yes"`, missing field all yield False).
-    """
-    code, raw = gitea_get(
-        api, repo,
-        "contents/.pr-review.json?ref=" + urllib.parse.quote(ref, safe=""),
-        token,
-    )
-    if code != 200:
-        return False
-    try:
-        data = json.loads(raw)
-        content_b64 = data.get("content", "").replace("\n", "")
-        decoded = base64.b64decode(content_b64).decode("utf-8", errors="replace")
-        cfg = json.loads(decoded)
-    except (json.JSONDecodeError, ValueError):
-        return False
-    return isinstance(cfg, dict) and cfg.get("enabled") is True
-
-
-def _verify_signature(raw_body: bytes, headers) -> bool:
-    if not WEBHOOK_SECRET:
-        return False  # refuse to run without a configured secret
-    sig_header = headers.get("X-Gitea-Signature") or headers.get("X-Forgejo-Signature")
-    if not sig_header:
-        return False
-    mac = hmac.new(WEBHOOK_SECRET, raw_body, hashlib.sha256).hexdigest()
-    return hmac.compare_digest(mac, sig_header)
-
-
-def _handle_pull_request(payload: dict) -> tuple[int, str]:
-    """Decide whether to review; if so, kick it off in a background thread.
-
-    Returns (status, message) to Gitea immediately — the review itself runs
-    async so Gitea's delivery timeout never fires and causes a retry.
-    """
-    action = payload.get("action", "")
-    pr = payload.get("pull_request") or {}
-    repo_obj = payload.get("repository") or {}
-    repo = repo_obj.get("full_name") or ""
-
-    if action in SKIP_ACTIONS:
-        return 200, f"ignore action={action}"
-    if not repo:
-        return 400, "no repository.full_name"
-
-    index = pr.get("number")
-    if index is None:
-        return 400, "no pull_request.number"
-    title = pr.get("title", "") or ""
-    body = pr.get("body", "") or ""
-    head = pr.get("head") or {}
-    sha = head.get("sha", "") or ""
-
-    base_ref = (pr.get("base") or {}).get("ref", "") or ""
-
-    if not is_repo_enabled(GITEA_API, repo, base_ref or "", BOT_TOKEN):
-        return 200, f"skip (repo not opted in) action={action}"
-
-    if not BOT_TOKEN:
-        return 500, "PRAGENT_BOT_TOKEN not set"
-
-    key = (repo, str(index), sha)
-    if not _claim(key):
-        return 200, f"ignore (already in flight) {repo}#{index} sha={sha[:8]}"
-
-    threading.Thread(
-        target=_run_review,
-        args=(key, title, body, base_ref),
-        daemon=True,
-    ).start()
-    return 202, f"reviewing {repo}#{index} action={action} sha={sha[:8]}"
-
-
-def _claim(key: tuple[str, str, str]) -> bool:
-    """Reserve (repo, index, sha) for review. False if already claimed."""
-    with _inflight_lock:
-        if key in _inflight:
-            return False
-        _inflight.add(key)
-        return True
-
-
-def _release(key: tuple[str, str, str]) -> None:
-    with _inflight_lock:
-        _inflight.discard(key)
-
-
-def _run_review(
-    key: tuple[str, str, str], title: str, body: str, base_ref: str
-) -> None:
-    repo, index, sha = key
-    # Harvest reactions on PRIOR bot comments on this PR (best-effort —
-    # piggy-backs the webhook path so we don't need a separate cron).
-    # Disabled if feedback_harvest isn't importable (CI-step image) or
-    # FEEDBACK_DB isn't set.
-    if FEEDBACK_DB and feedback_harvest is not None:
-        try:
-            hstats = feedback_harvest.harvest_for_pr(
-                api=GITEA_API, token=BOT_TOKEN,
-                repo=repo, pr_index=int(index), db_path=FEEDBACK_DB,
-            )
-            print(
-                f"pragent-webhook: harvested {repo}#{index} "
-                f"reviews={hstats['reviews_seen']} "
-                f"findings={hstats['findings_seen']} "
-                f"reactions={hstats['reactions_recorded']}",
-                flush=True,
-            )
-        except Exception as e:
-            # Harvest must never abort a review.
-            print(f"pragent-webhook: harvest failed for {repo}#{index}: {e}", flush=True)
-
-    try:
-        with _review_slots:
-            ok = review_pr(
-                api=GITEA_API,
-                repo=repo,
-                index=index,
-                title=title,
-                body=body,
-                sha=sha,
-                token=BOT_TOKEN,
-                ollama_url=OLLAMA_URL,
-                model=OLLAMA_MODEL,
-                max_tokens=OLLAMA_MAX_TOKENS,
-                max_chars=DIFF_MAX_CHARS,
-                base_ref=base_ref,
-            )
-        print(f"pragent-webhook: reviewed {repo}#{index} sha={sha[:8]} ok={ok}", flush=True)
-    except Exception as e:  # review_pr is fail-open, but guard the thread anyway
-        print(f"pragent-webhook: thread crashed for {repo}#{index}: {e}", flush=True)
-    finally:
-        _release(key)
-
-
-class Handler(BaseHTTPRequestHandler):
-    def _send(self, status: int, body: str) -> None:
-        data = body.encode()
-        self.send_response(status)
-        self.send_header("Content-Type", "text/plain")
-        self.send_header("Content-Length", str(len(data)))
-        self.end_headers()
-        self.wfile.write(data)
-
-    def do_GET(self):
-        if self.path == "/health":
-            with _inflight_lock:
-                n = len(_inflight)
-            self._send(200, f"ok inflight={n} max_concurrent={MAX_CONCURRENT}")
-        else:
-            self._send(404, "not found")
-
-    def do_POST(self):
-        if self.path != "/webhook":
-            self._send(404, "not found")
-            return
-        try:
-            length = int(self.headers.get("Content-Length", "0") or "0")
-        except ValueError:
-            self._send(400, "bad content-length")
-            return
-        # Cap before reading: the body is read whole into memory, so an
-        # unbounded Content-Length is a one-request OOM.
-        if length < 0 or length > MAX_BODY_BYTES:
-            self._send(413, "payload too large")
-            return
-        raw = self.rfile.read(length) if length else b""
-        if len(raw) != length:
-            self._send(400, "truncated body")
-            return
-
-        if not _verify_signature(raw, self.headers):
-            self._send(401, "invalid signature")
-            return
-
-        try:
-            payload = json.loads(raw)
-        except json.JSONDecodeError:
-            self._send(400, "invalid json")
-            return
-
-        event = self.headers.get("X-Gitea-Event") or payload.get("action") or ""
-        if event != "pull_request":
-            self._send(200, f"ignore event={event}")
-            return
-
-        repo_full = (payload.get("repository") or {}).get("full_name")
-        print(
-            f"pragent-webhook: pull_request action={payload.get('action')} repo={repo_full}",
-            flush=True,
-        )
-        status, msg = _handle_pull_request(payload)
-        self._send(status, msg)
-
-    def log_message(self, fmt, *args):
-        # Keep k8s logs to our own lines (see _run_review / _send paths).
-        print(f"pragent-webhook: {self.address_string()} {fmt % args}", flush=True)
-
-
-def main() -> int:
-    if not WEBHOOK_SECRET:
-        print("pragent-webhook: FATAL: WEBHOOK_SECRET not set", flush=True)
-        return 1
-    if not BOT_TOKEN:
-        print("pragent-webhook: FATAL: PRAGENT_BOT_TOKEN not set", flush=True)
-        return 1
-    server = ThreadingHTTPServer(("0.0.0.0", PORT), Handler)
-    print(f"pragent-webhook: listening on :{PORT} (model={OLLAMA_MODEL})", flush=True)
-    try:
-        server.serve_forever()
-    except KeyboardInterrupt:
-        pass
-    return 0
-
-
+"""Compatibility import for the webhook entry point."""
+import importlib
+import sys
+_module = importlib.import_module("entrypoints.webhook")
+sys.modules[__name__] = _module
 if __name__ == "__main__":
-    raise SystemExit(main())
+    raise SystemExit(_module.main())

tests/pilot/conftest.py

@@ -0,0 +1,9 @@
+"""Make the pilot package roots available to every categorized test."""
+from __future__ import annotations
+
+import os
+import sys
+
+PILOT_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", "pilot"))
+if PILOT_ROOT not in sys.path:
+    sys.path.insert(0, PILOT_ROOT)

tests/pilot/entrypoint_tests/__init__.py

@@ -0,0 +1 @@
+"""Entrypoint tests."""

tests/pilot/entrypoint_tests/webhook_test.py

@@ -5,7 +5,7 @@ import sys
 import threading
 
 HERE = os.path.dirname(os.path.abspath(__file__))
-ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
+ROOT = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
 sys.path.insert(0, os.path.join(ROOT, "pilot"))
 
 import webhook_server as ws  # noqa: E402

tests/pilot/evaluation_tests/__init__.py

@@ -0,0 +1 @@
+"""Evaluation tests."""

tests/pilot/evaluation_tests/bootstrap_test.py

@@ -4,7 +4,7 @@ import sqlite3
 import sys
 import urllib.parse
 
-sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "pilot"))
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "..", "pilot"))
 
 import eval_bootstrap as eb  # noqa: E402
 

tests/pilot/evaluation_tests/experiment_test.py

@@ -2,7 +2,7 @@
 import os
 import sys
 
-sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "pilot"))
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "..", "pilot"))
 
 import eval_experiment as ex  # noqa: E402
 

tests/pilot/evaluation_tests/judges_test.py

@@ -11,10 +11,16 @@ import eval_judges as ej  # noqa: E402
 
 # --- rule_body ------------------------------------------------------------
 
-def test_rule_body_targets_observations():
-    """Trace-level rules wouldn't see observation input/output."""
+def test_rule_body_targets_traces():
+    """Trace target matches the path `/api/public/ingestion` triggers.
+
+    Observation rules only fire from the OTel ingestion pipeline; this
+    pilot uses standard ingestion, so its jobs only come from
+    `evalService.createEvalJobs` and that dispatcher handles
+    `targetObject ∈ {TRACE, DATASET}`.
+    """
     body = ej.rule_body("rule-x", "finding_actionability", 1.0)
-    assert body["target"] == "observation"
+    assert body["target"] == "trace"
     assert body["enabled"] is True
 
 

tests/pilot/evaluation_tests/scores_test.py

@@ -4,7 +4,7 @@ import sys
 
 import pytest
 
-sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "pilot"))
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "..", "pilot"))
 
 import eval_scores as es  # noqa: E402
 

tests/pilot/feedback_tests/__init__.py

@@ -0,0 +1 @@
+"""Feedback tests."""

tests/pilot/feedback_tests/analyze_test.py

@@ -16,7 +16,7 @@ import tempfile
 import unittest
 
 HERE = os.path.dirname(os.path.abspath(__file__))
-sys.path.insert(0, os.path.join(HERE, "..", "..", "pilot"))
+sys.path.insert(0, os.path.join(HERE, "..", "..", "..", "pilot"))
 
 import feedback  # noqa: E402
 import feedback_analyze  # noqa: E402

tests/pilot/feedback_tests/harvest_test.py

@@ -17,7 +17,7 @@ import unittest
 from unittest.mock import patch
 
 HERE = os.path.dirname(os.path.abspath(__file__))
-sys.path.insert(0, os.path.join(HERE, "..", "..", "pilot"))
+sys.path.insert(0, os.path.join(HERE, "..", "..", "..", "pilot"))
 
 import ai_review  # noqa: E402
 import feedback  # noqa: E402

tests/pilot/feedback_tests/post_test.py

@@ -11,7 +11,7 @@ import unittest
 from unittest.mock import patch
 
 HERE = os.path.dirname(os.path.abspath(__file__))
-sys.path.insert(0, os.path.join(HERE, "..", "..", "pilot"))
+sys.path.insert(0, os.path.join(HERE, "..", "..", "..", "pilot"))
 
 import feedback  # noqa: E402
 import feedback_analyze  # noqa: E402

tests/pilot/feedback_tests/scores_test.py

@@ -4,7 +4,7 @@ import sys
 
 import pytest
 
-sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "pilot"))
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "..", "pilot"))
 
 import feedback  # noqa: E402
 import feedback_scores as fs  # noqa: E402

tests/pilot/observability_tests/__init__.py

@@ -0,0 +1 @@
+"""Observability tests."""

tests/pilot/observability_tests/cost_test.py

@@ -3,7 +3,7 @@ import os
 import sys
 
 HERE = os.path.dirname(os.path.abspath(__file__))
-ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
+ROOT = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
 sys.path.insert(0, os.path.join(ROOT, "pilot"))
 
 import cost_model as cm  # noqa: E402

tests/pilot/observability_tests/langfuse_test.py

@@ -8,7 +8,7 @@ import os
 import sys
 
 HERE = os.path.dirname(os.path.abspath(__file__))
-ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
+ROOT = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
 sys.path.insert(0, os.path.join(ROOT, "pilot"))
 
 import langfuse_trace as lt  # noqa: E402

tests/pilot/review_tests/__init__.py

@@ -0,0 +1 @@
+"""Review tests."""

tests/pilot/review_tests/ai_review_test.py

@@ -6,7 +6,7 @@ import sys
 
 # Allow running without install: add repo root to path.
 HERE = os.path.dirname(os.path.abspath(__file__))
-ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
+ROOT = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
 sys.path.insert(0, os.path.join(ROOT, "pilot"))
 
 import ai_review  # noqa: E402
@@ -2149,4 +2149,3 @@ def test_format_review_body_confidence_clamps_out_of_range():
     body_lo = format_review_body("- x", "glm-5.2:cloud", "abcdef1234567890", confidence=0)
     assert "Merge confidence: 1/5 🔴" in body_lo
 
-

tests/pilot/review_tests/diff_test.py

@@ -4,7 +4,7 @@ import re
 import sys
 
 HERE = os.path.dirname(os.path.abspath(__file__))
-ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
+ROOT = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
 sys.path.insert(0, os.path.join(ROOT, "pilot"))
 
 import diff_compress  # noqa: E402

tests/pilot/review_tests/opencode_test.py

@@ -6,7 +6,7 @@ import sys
 import tarfile
 
 HERE = os.path.dirname(os.path.abspath(__file__))
-ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
+ROOT = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
 sys.path.insert(0, os.path.join(ROOT, "pilot"))
 
 import opencode_review as oc  # noqa: E402

tests/pilot/review_tests/parse_dropped_test.py

@@ -9,7 +9,7 @@ import os
 import sys
 
 HERE = os.path.dirname(os.path.abspath(__file__))
-sys.path.insert(0, os.path.abspath(os.path.join(HERE, "..", "..", "pilot")))
+sys.path.insert(0, os.path.abspath(os.path.join(HERE, "..", "..", "..", "pilot")))
 
 import ai_review  # noqa: E402
 

tests/pilot/test_dashboard.py

@@ -1,203 +0,0 @@
-"""Tests for pilot/dashboard.py — stdlib HTTP server rendering dashboard HTML.
-
-We spin up the server on an ephemeral port in setUp, drive it with
-http.client, and tear it down in tearDown. Auth is now performed by
-oauth2-proxy: the dashboard trusts `X-Forwarded-User` set by the proxy
-and returns 401 (with a Basic challenge) when the header is missing.
-
-The dashboard reads `PRAGENT_FEEDBACK_DB` and renders views via
-`dashboard_data`. We seed an in-memory SQLite at `tmp_path` for each
-scenario that needs rows.
-"""
-import http.client
-import os
-import socket
-import sys
-import tempfile
-import threading
-import time
-import unittest
-
-HERE = os.path.dirname(os.path.abspath(__file__))
-ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
-sys.path.insert(0, os.path.join(ROOT, "pilot"))
-
-import dashboard as dash  # noqa: E402
-from pilot import feedback  # noqa: E402
-
-
-def _free_port() -> int:
-    s = socket.socket()
-    s.bind(("127.0.0.1", 0))
-    port = s.getsockname()[1]
-    s.close()
-    return port
-
-
-class _ServerThread:
-    def __init__(self, port: int, handler):
-        self.server = handler((host := "127.0.0.1", port), None)
-        self.port = port
-        self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
-        self.thread.start()
-
-    def stop(self):
-        self.server.shutdown()
-        self.server.server_close()
-        self.thread.join(timeout=2)
-
-
-def _get(port: int, path: str, headers: dict | None = None) -> tuple[int, dict, bytes]:
-    conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
-    conn.request("GET", path, headers=headers or {})
-    r = conn.getresponse()
-    body = r.read()
-    h = dict(r.getheaders())
-    conn.close()
-    return r.status, h, body
-
-
-def _post(port: int, path: str, body: bytes, headers: dict | None = None) -> tuple[int, dict, bytes]:
-    conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
-    hdrs = {"Content-Type": "application/x-www-form-urlencoded"}
-    if headers:
-        hdrs.update(headers)
-    conn.request("POST", path, body=body, headers=hdrs)
-    r = conn.getresponse()
-    body_b = r.read()
-    h = dict(r.getheaders())
-    conn.close()
-    return r.status, h, body_b
-
-
-class TestDashboardAuth(unittest.TestCase):
-    """Auth gate: require X-Forwarded-User (set by oauth2-proxy).
-
-    When the header is missing every non-static route returns 401 with a
-    Basic challenge, which lets oauth2-proxy redirect the browser to
-    Logto. Static is exempt so the unauthenticated probe traffic doesn't
-    loop the proxy through the auth flow.
-    """
-
-    def setUp(self):
-        self.tmp = tempfile.TemporaryDirectory()
-        self.db = f"{self.tmp.name}/f.db"
-        os.environ["PRAGENT_FEEDBACK_DB"] = self.db
-        os.environ["DASHBOARD_PORT"] = str(0)  # we override below
-
-        self.port = _free_port()
-        from http.server import ThreadingHTTPServer
-        self.srv = ThreadingHTTPServer(("127.0.0.1", self.port), dash.Handler)
-        self.thread = threading.Thread(target=self.srv.serve_forever, daemon=True)
-        self.thread.start()
-
-    def tearDown(self):
-        self.srv.shutdown()
-        self.srv.server_close()
-        self.thread.join(timeout=2)
-        for k in ("PRAGENT_FEEDBACK_DB", "DASHBOARD_PORT"):
-            os.environ.pop(k, None)
-        self.tmp.cleanup()
-
-    def test_anonymous_overview_returns_401_with_basic_challenge(self):
-        status, h, body = _get(self.port, "/")
-        self.assertEqual(status, 401)
-        self.assertEqual(h.get("WWW-Authenticate"), 'Basic realm="pragent-dashboard"')
-        self.assertEqual(body, b"unauthorized\n")
-
-    def test_anonymous_repo_returns_401(self):
-        status, _h, _body = _get(self.port, "/r/alpha/one")
-        self.assertEqual(status, 401)
-
-    def test_anonymous_post_returns_401(self):
-        status, _h, _body = _post(self.port, "/r/alpha/one/edit", b"x=1")
-        self.assertEqual(status, 401)
-
-    def test_authenticated_overview_succeeds(self):
-        status, h, body = _get(self.port, "/", headers={"X-Forwarded-User": "marcos@example.com"})
-        self.assertEqual(status, 200)
-        self.assertIn(b"Overview", body)
-
-    def test_static_does_not_require_auth(self):
-        status, h, body = _get(self.port, "/static/style.css")
-        self.assertEqual(status, 200)
-        self.assertIn("text/css", h.get("Content-Type", ""))
-        self.assertGreater(len(body), 50)
-
-    def test_empty_x_forwarded_user_treated_as_anonymous(self):
-        status, _h, _body = _get(self.port, "/", headers={"X-Forwarded-User": "   "})
-        self.assertEqual(status, 401)
-
-
-class TestDashboardRender(unittest.TestCase):
-    """Render-only tests — X-Forwarded-User set, real seeded data."""
-
-    def setUp(self):
-        self.tmp = tempfile.TemporaryDirectory()
-        self.db = f"{self.tmp.name}/f.db"
-        os.environ["PRAGENT_FEEDBACK_DB"] = self.db
-        # Seed: 2 repos, a couple of reviews + findings each.
-        conn = feedback.init(self.db)
-        for repo, n_prs in (("alpha/one", 2), ("beta/two", 1)):
-            for n in range(n_prs):
-                rid = feedback.record_review(
-                    conn, repo=repo, pr=n + 1, head_sha=f"sha{repo}-{n}",
-                    review_id_gitea=1000 + n, body_comment_id=2000 + n,
-                    posted_at=int(time.time()) - n * 60,
-                )
-                for k in range(3):
-                    feedback.record_inline_finding(
-                        conn, review_id=rid, repo=repo, pr=n + 1,
-                        path=f"src/file_{k}.py", line=k + 1,
-                        severity=["critical", "high", "medium"][k],
-                        problem=f"problem {k}",
-                        fix=f"fix {k}", suggestion=f"suggestion {k}",
-                        comment_id=3000 + n * 10 + k,
-                    )
-        conn.close()
-
-        self.port = _free_port()
-        from http.server import ThreadingHTTPServer
-        self.srv = ThreadingHTTPServer(("127.0.0.1", self.port), dash.Handler)
-        self.thread = threading.Thread(target=self.srv.serve_forever, daemon=True)
-        self.thread.start()
-        self.auth_hdr = {"X-Forwarded-User": "marcos@example.com"}
-
-    def tearDown(self):
-        self.srv.shutdown()
-        self.srv.server_close()
-        self.thread.join(timeout=2)
-        os.environ.pop("PRAGENT_FEEDBACK_DB", None)
-        self.tmp.cleanup()
-
-    def test_overview_200_contains_repo_names(self):
-        status, _h, body = _get(self.port, "/", headers=self.auth_hdr)
-        self.assertEqual(status, 200)
-        text = body.decode()
-        self.assertIn("Overview", text)
-        self.assertIn("alpha/one", text)
-        self.assertIn("beta/two", text)
-
-    def test_repo_page_200(self):
-        status, _h, body = _get(self.port, "/r/alpha/one", headers=self.auth_hdr)
-        self.assertEqual(status, 200)
-        text = body.decode()
-        self.assertIn("alpha/one", text)
-        # The findings table should appear.
-        self.assertIn("src/file_0.py", text)
-
-    def test_pr_page_200(self):
-        status, _h, body = _get(self.port, "/r/alpha/one/1", headers=self.auth_hdr)
-        self.assertEqual(status, 200)
-        text = body.decode()
-        self.assertIn("alpha/one", text)
-        self.assertIn("#1", text)
-        self.assertIn("src/file_0.py", text)
-
-    def test_unknown_route_404(self):
-        status, _h, _body = _get(self.port, "/no/such/route", headers=self.auth_hdr)
-        self.assertEqual(status, 404)
-
-
-if __name__ == "__main__":
-    unittest.main()

tests/pilot/test_dashboard_data.py

@@ -1,249 +0,0 @@
-"""Tests for pilot/dashboard_data.py — read-only query layer over the feedback SQLite.
-
-Covers: empty-DB fallbacks (no crash on missing/empty DB), overview rollups,
-per-repo drill-down (findings by severity, top findings, runs by day), and
-the per-PR view. The dashboard never writes — only reads.
-"""
-import os
-import sys
-import tempfile
-import time
-import unittest
-
-HERE = os.path.dirname(os.path.abspath(__file__))
-ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
-sys.path.insert(0, os.path.join(HERE, "..", ".."))  # so `from pilot import …` works
-
-from pilot import dashboard_data, feedback
-
-
-def _seed_repo(conn, *, repo: str, prs: int, findings_per_pr: int, day_offset: int = 0):
-    """Seed one repo with `prs` PRs each with `findings_per_pr` findings.
-
-    All timestamps cluster on (now - day_offset days). Returns list of review ids.
-    """
-    base = int(time.time()) - day_offset * 86400
-    rids = []
-    for n in range(prs):
-        rid = feedback.record_review(
-            conn, repo=repo, pr=n + 1, head_sha=f"sha{n}",
-            review_id_gitea=1000 + n, body_comment_id=2000 + n,
-            posted_at=base + n * 60,
-        )
-        rids.append(rid)
-        for k in range(findings_per_pr):
-            feedback.record_inline_finding(
-                conn, review_id=rid, repo=repo, pr=n + 1,
-                path=f"src/file_{k}.py", line=k + 1,
-                severity=["critical", "high", "medium", "low"][k % 4],
-                problem=f"problem {k}",
-                fix=f"fix {k}", suggestion=f"suggestion {k}",
-                comment_id=3000 + n * 10 + k,
-                posted_at=base + n * 60,
-            )
-    return rids
-
-
-class TestEmptyDB(unittest.TestCase):
-    def test_missing_file_returns_zero_dict(self):
-        with tempfile.TemporaryDirectory() as d:
-            missing = f"{d}/nope.db"
-            data = dashboard_data.overview(missing)
-            self.assertEqual(data["total_reviews"], 0)
-            self.assertEqual(data["total_findings"], 0)
-            self.assertEqual(data["total_repos"], 0)
-            self.assertEqual(data["last_30d_reviews"], 0)
-            self.assertEqual(len(data["daily"]), 7)
-            self.assertEqual(data["top_repos"], [])
-            self.assertEqual(data["total_cost_usd"], 0.0)
-
-    def test_missing_file_repo_summary_safe(self):
-        with tempfile.TemporaryDirectory() as d:
-            data = dashboard_data.repo_summary(f"{d}/nope.db", "o/r")
-            self.assertEqual(data["repo"], "o/r")
-            self.assertEqual(data["total_runs"], 0)
-            self.assertEqual(data["runs_by_day"], [])
-            for sev in ("critical", "high", "medium", "low"):
-                self.assertEqual(data["findings_by_severity"][sev], 0)
-            self.assertEqual(data["top_findings"], [])
-            self.assertEqual(data["models_used"], [])
-
-    def test_missing_file_pr_summary_safe(self):
-        with tempfile.TemporaryDirectory() as d:
-            data = dashboard_data.pr_summary(f"{d}/nope.db", "o/r", 1)
-            self.assertEqual(data["repo"], "o/r")
-            self.assertEqual(data["pr"], 1)
-            self.assertEqual(data["findings"], [])
-            self.assertEqual(data["usage"], {})
-
-
-class TestEmptyButExistingDB(unittest.TestCase):
-    """`init` creates the schema — DB exists but has no rows."""
-
-    def setUp(self):
-        self.tmp = tempfile.TemporaryDirectory()
-        self.db = f"{self.tmp.name}/f.db"
-        feedback.init(self.db)
-
-    def tearDown(self):
-        self.tmp.cleanup()
-
-    def test_overview_is_zero(self):
-        data = dashboard_data.overview(self.db)
-        self.assertEqual(data["total_reviews"], 0)
-        self.assertEqual(data["total_findings"], 0)
-        self.assertEqual(data["total_repos"], 0)
-
-    def test_repo_summary_is_zero(self):
-        data = dashboard_data.repo_summary(self.db, "o/r")
-        self.assertEqual(data["total_runs"], 0)
-        self.assertEqual(data["findings_by_severity"], {"critical": 0, "high": 0, "medium": 0, "low": 0})
-
-    def test_pr_summary_is_zero(self):
-        data = dashboard_data.pr_summary(self.db, "o/r", 1)
-        self.assertEqual(data["findings"], [])
-
-
-class TestOverview(unittest.TestCase):
-    def setUp(self):
-        self.tmp = tempfile.TemporaryDirectory()
-        self.db = f"{self.tmp.name}/f.db"
-        self.conn = feedback.init(self.db)
-        _seed_repo(self.conn, repo="alpha/one", prs=3, findings_per_pr=2)
-        _seed_repo(self.conn, repo="beta/two", prs=1, findings_per_pr=4)
-        self.conn.close()
-
-    def tearDown(self):
-        self.tmp.cleanup()
-
-    def test_totals(self):
-        data = dashboard_data.overview(self.db)
-        self.assertEqual(data["total_reviews"], 4)
-        self.assertEqual(data["total_findings"], 6 + 4)  # 3*2 + 1*4 = 10
-        self.assertEqual(data["total_repos"], 2)
-        self.assertEqual(data["total_cost_usd"], 0.0)
-
-    def test_top_repos_sorted_by_run_count(self):
-        data = dashboard_data.overview(self.db)
-        repos = [r["repo"] for r in data["top_repos"]]
-        # alpha/one has 3 runs, beta/two has 1.
-        self.assertEqual(repos[0], "alpha/one")
-        self.assertEqual(data["top_repos"][0]["run_count"], 3)
-        self.assertEqual(data["top_repos"][1]["run_count"], 1)
-        # last_seen is a unix timestamp int.
-        for r in data["top_repos"]:
-            self.assertIsInstance(r["last_seen"], int)
-
-    def test_daily_buckets_are_7(self):
-        data = dashboard_data.overview(self.db)
-        self.assertEqual(len(data["daily"]), 7)
-        for b in data["daily"]:
-            self.assertIn("date", b)
-            self.assertIn("count", b)
-
-    def test_last_30d_reviews(self):
-        data = dashboard_data.overview(self.db)
-        self.assertEqual(data["last_30d_reviews"], 4)
-
-
-class TestRepoSummary(unittest.TestCase):
-    def setUp(self):
-        self.tmp = tempfile.TemporaryDirectory()
-        self.db = f"{self.tmp.name}/f.db"
-        self.conn = feedback.init(self.db)
-        # 4 PRs with 2 findings each → 8 findings, severity cycle [c,h,m,l,c,h,m,l]
-        _seed_repo(self.conn, repo="o/r", prs=4, findings_per_pr=2)
-        # Add some reactions so top_findings has signal.
-        rows = self.conn.execute(
-            "SELECT id, comment_id FROM inline_finding WHERE repo=? ORDER BY id LIMIT 3",
-            ("o/r",),
-        ).fetchall()
-        for r in rows:
-            feedback.record_reaction(self.conn, comment_id=r["comment_id"], user="u", content="+1")
-        self.conn.close()
-
-    def tearDown(self):
-        self.tmp.cleanup()
-
-    def test_basic_shape(self):
-        data = dashboard_data.repo_summary(self.db, "o/r")
-        self.assertEqual(data["repo"], "o/r")
-        self.assertEqual(data["total_runs"], 4)
-        self.assertIsInstance(data["last_run_ts"], int)
-
-    def test_findings_by_severity(self):
-        data = dashboard_data.repo_summary(self.db, "o/r")
-        fbs = data["findings_by_severity"]
-        # 4 PRs × 2 findings; per-PR severities are [critical, high].
-        # (k in range(2) → k=0 critical, k=1 high for every PR.)
-        self.assertEqual(fbs["critical"], 4)
-        self.assertEqual(fbs["high"], 4)
-        self.assertEqual(fbs["medium"], 0)
-        self.assertEqual(fbs["low"], 0)
-
-    def test_runs_by_day_is_list(self):
-        data = dashboard_data.repo_summary(self.db, "o/r")
-        self.assertIsInstance(data["runs_by_day"], list)
-        for r in data["runs_by_day"]:
-            self.assertIn("date", r)
-            self.assertIn("count", r)
-
-    def test_top_findings_structure(self):
-        data = dashboard_data.repo_summary(self.db, "o/r")
-        self.assertGreater(len(data["top_findings"]), 0)
-        first = data["top_findings"][0]
-        for k in ("path", "line", "severity", "problem", "occurrences", "upvotes", "downvotes", "resolved", "reply_count"):
-            self.assertIn(k, first)
-
-    def test_models_used_is_empty_list_with_note(self):
-        # The schema has no `model` column on review — the dashboard can't show
-        # model usage from this DB today. We document that via an empty list.
-        data = dashboard_data.repo_summary(self.db, "o/r")
-        self.assertEqual(data["models_used"], [])
-
-
-class TestPRSummary(unittest.TestCase):
-    def setUp(self):
-        self.tmp = tempfile.TemporaryDirectory()
-        self.db = f"{self.tmp.name}/f.db"
-        self.conn = feedback.init(self.db)
-        rid = feedback.record_review(
-            self.conn, repo="o/r", pr=42, head_sha="abc",
-            review_id_gitea=9001, body_comment_id=8001,
-            posted_at=1700000000,
-        )
-        for k in range(3):
-            feedback.record_inline_finding(
-                self.conn, review_id=rid, repo="o/r", pr=42,
-                path=f"src/x_{k}.py", line=k + 10,
-                severity=["critical", "high", "low"][k],
-                problem=f"p{k}", fix=f"f{k}", suggestion=f"s{k}",
-                comment_id=7000 + k,
-            )
-        self.conn.close()
-
-    def tearDown(self):
-        self.tmp.cleanup()
-
-    def test_meta(self):
-        data = dashboard_data.pr_summary(self.db, "o/r", 42)
-        self.assertEqual(data["repo"], "o/r")
-        self.assertEqual(data["pr"], 42)
-        self.assertEqual(data["head_sha"], "abc")
-        self.assertEqual(data["review_id_gitea"], 9001)
-        self.assertEqual(data["body_comment_id"], 8001)
-        self.assertEqual(data["posted_at"], 1700000000)
-        # usage is empty because the schema has no usage column.
-        self.assertEqual(data["usage"], {})
-
-    def test_findings(self):
-        data = dashboard_data.pr_summary(self.db, "o/r", 42)
-        self.assertEqual(len(data["findings"]), 3)
-        for f in data["findings"]:
-            for k in ("path", "line", "severity", "problem", "fix", "suggestion",
-                      "upvotes", "downvotes", "resolved", "reply_count"):
-                self.assertIn(k, f)
-
-
-if __name__ == "__main__":
-    unittest.main()

tests/pilot/test_dashboard_edit.py

@@ -1,209 +0,0 @@
-"""Tests for the edit endpoint — POST /r/<owner>/<name>/edit (Task C).
-
-We mock the Gitea HTTP layer (urllib.request.urlopen) so the test never
-touches the network. The dashboard handler is responsible for:
-  * auth (X-Forwarded-User set by oauth2-proxy) + CSRF
-  * read .pr-review.json via GET (404 → start from {})
-  * validate model against cost_model.PRICES
-  * PUT the updated file back, with sha + base64 content
-  * redirect to /r/<owner>/<name> on success
-"""
-import base64
-import http.client
-import json
-import os
-import socket
-import sys
-import tempfile
-import threading
-import unittest
-from unittest.mock import patch
-
-HERE = os.path.dirname(os.path.abspath(__file__))
-ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
-sys.path.insert(0, os.path.join(ROOT, "pilot"))
-
-import dashboard as dash  # noqa: E402
-from pilot import feedback  # noqa: E402
-
-
-def _free_port() -> int:
-    s = socket.socket()
-    s.bind(("127.0.0.1", 0))
-    port = s.getsockname()[1]
-    s.close()
-    return port
-
-
-def _post(port: int, path: str, body: bytes, *, headers: dict | None = None) -> tuple[int, dict, bytes]:
-    conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
-    hdrs = {"Content-Type": "application/x-www-form-urlencoded"}
-    if headers:
-        hdrs.update(headers)
-    conn.request("POST", path, body=body, headers=hdrs)
-    r = conn.getresponse()
-    body_b = r.read()
-    h = dict(r.getheaders())
-    conn.close()
-    return r.status, h, body_b
-
-
-class _FakeResp:
-    def __init__(self, status: int, body: bytes):
-        self.status = status
-        self._body = body
-
-    def read(self):
-        return self._body
-
-    def __enter__(self):
-        return self
-
-    def __exit__(self, *a):
-        return False
-
-
-class TestDashboardEdit(unittest.TestCase):
-    def setUp(self):
-        self.tmp = tempfile.TemporaryDirectory()
-        self.db = f"{self.tmp.name}/f.db"
-        os.environ["PRAGENT_FEEDBACK_DB"] = self.db
-        os.environ["PRAGENT_BOT_TOKEN"] = "bot-token"
-        # Seed a row so the repo page is meaningful.
-        conn = feedback.init(self.db)
-        feedback.record_review(
-            conn, repo="o/r", pr=1, head_sha="x",
-        )
-        conn.close()
-
-        self.port = _free_port()
-        from http.server import ThreadingHTTPServer
-        self.srv = ThreadingHTTPServer(("127.0.0.1", self.port), dash.Handler)
-        self.thread = threading.Thread(target=self.srv.serve_forever, daemon=True)
-        self.thread.start()
-        # Pull the per-process CSRF secret from the rendered repo page — the
-        # edit form embeds the same token as a hidden input.
-        self.csrf = dash._CSRF_SECRET
-        self.auth_hdr = {"X-Forwarded-User": "marcos@example.com"}
-
-        # Records of HTTP calls made by the handler.
-        self.calls: list[tuple[str, str, dict | None, bytes | None]] = []
-
-    def tearDown(self):
-        self.srv.shutdown()
-        self.srv.server_close()
-        self.thread.join(timeout=2)
-        for k in ("PRAGENT_FEEDBACK_DB", "PRAGENT_BOT_TOKEN"):
-            os.environ.pop(k, None)
-        self.tmp.cleanup()
-
-    def _urlopen(self, req, timeout=30):
-        """Replacement for urllib.request.urlopen that the handler uses."""
-        url = req.full_url if hasattr(req, "full_url") else req
-        method = getattr(req, "method", None) or "GET"
-        body = getattr(req, "data", None)
-        headers = dict(getattr(req, "headers", {}) or {})
-        self.calls.append((method, url, headers, body))
-        # Route based on URL: GET contents/.../raw vs PUT contents/.pr-review.json
-        if method == "GET" and ".pr-review.json" in url:
-            return _FakeResp(200, json.dumps({
-                "content": base64.b64encode(b'{"focus":["x"],"model":"claude-haiku-4-5"}').decode(),
-                "sha": "deadbeef",
-            }).encode())
-        if method == "PUT" and ".pr-review.json" in url:
-            return _FakeResp(200, b'{}')
-        return _FakeResp(404, b'{"message":"not found"}')
-
-    def test_edit_updates_static_message_and_model(self):
-        form = (
-            f"_csrf={self.csrf}"
-            f"&static_message=Hello%20world"
-            f"&model=claude-sonnet-5"
-        ).encode()
-        with patch.object(dash.urllib.request, "urlopen", side_effect=self._urlopen):
-            status, h, _b = _post(self.port, "/r/o/r/edit", form, headers=self.auth_hdr)
-        self.assertEqual(status, 302)
-        self.assertEqual(h.get("Location"), "/r/o/r")
-
-        # Find the PUT call.
-        put_calls = [c for c in self.calls if c[0] == "PUT"]
-        self.assertEqual(len(put_calls), 1, self.calls)
-        method, url, _hdrs, body = put_calls[0]
-        self.assertIn(".pr-review.json", url)
-        payload = json.loads(body)
-        self.assertIn("content", payload)
-        self.assertEqual(payload["sha"], "deadbeef")
-        decoded = base64.b64decode(payload["content"]).decode()
-        cfg = json.loads(decoded)
-        self.assertEqual(cfg.get("static_message"), "Hello world")
-        self.assertEqual(cfg.get("model"), "claude-sonnet-5")
-
-    def test_edit_strips_static_message_to_400(self):
-        long_msg = "x" * 600
-        form = (
-            f"_csrf={self.csrf}"
-            f"&static_message={long_msg}"
-            f"&model=claude-haiku-4-5"
-        ).encode()
-        with patch.object(dash.urllib.request, "urlopen", side_effect=self._urlopen):
-            _post(self.port, "/r/o/r/edit", form, headers=self.auth_hdr)
-        put = next(c for c in self.calls if c[0] == "PUT")
-        cfg = json.loads(base64.b64decode(json.loads(put[3])["content"]))
-        self.assertEqual(len(cfg["static_message"]), 400)
-
-    def test_edit_rejects_unknown_model_with_flash(self):
-        form = (
-            f"_csrf={self.csrf}"
-            f"&static_message=hi"
-            f"&model=does-not-exist"
-        ).encode()
-        with patch.object(dash.urllib.request, "urlopen", side_effect=self._urlopen):
-            status, h, _b = _post(self.port, "/r/o/r/edit", form, headers=self.auth_hdr)
-        self.assertEqual(status, 302)
-        self.assertIn("flash=", h.get("Location", ""))
-        # No PUT should have been issued.
-        put_calls = [c for c in self.calls if c[0] == "PUT"]
-        self.assertEqual(put_calls, [])
-
-    def test_edit_requires_auth(self):
-        form = f"_csrf={self.csrf}&static_message=x&model=claude-haiku-4-5".encode()
-        with patch.object(dash.urllib.request, "urlopen", side_effect=self._urlopen):
-            status, h, _b = _post(self.port, "/r/o/r/edit", form)
-        self.assertEqual(status, 401)
-        self.assertEqual(h.get("WWW-Authenticate"), 'Basic realm="pragent-dashboard"')
-        # No Gitea calls at all — auth gate fires first.
-        self.assertEqual(self.calls, [])
-
-    def test_edit_csrf_mismatch_redirects_without_save(self):
-        form = f"_csrf=wrong&static_message=x&model=claude-haiku-4-5".encode()
-        with patch.object(dash.urllib.request, "urlopen", side_effect=self._urlopen):
-            status, h, _b = _post(self.port, "/r/o/r/edit", form, headers=self.auth_hdr)
-        self.assertEqual(status, 302)
-        self.assertEqual(h.get("Location"), "/r/o/r")
-        put_calls = [c for c in self.calls if c[0] == "PUT"]
-        self.assertEqual(put_calls, [])
-
-    def test_edit_creates_file_when_missing(self):
-        """When GET returns 404, the PUT must still happen (no sha)."""
-        def _route(req, timeout=30):
-            url = req.full_url
-            method = getattr(req, "method", None) or "GET"
-            body = getattr(req, "data", None)
-            self.calls.append((method, url, {}, body))
-            if method == "GET" and ".pr-review.json" in url:
-                return _FakeResp(404, b'{"message":"not found"}')
-            if method == "PUT" and ".pr-review.json" in url:
-                return _FakeResp(201, b"{}")
-            return _FakeResp(404, b"")
-
-        form = f"_csrf={self.csrf}&static_message=hi&model=claude-haiku-4-5".encode()
-        with patch.object(dash.urllib.request, "urlopen", side_effect=_route):
-            status, h, _b = _post(self.port, "/r/o/r/edit", form, headers=self.auth_hdr)
-        self.assertEqual(status, 302)
-        put = next(c for c in self.calls if c[0] == "PUT")
-        payload = json.loads(put[3])
-        self.assertNotIn("sha", payload, "missing-file PUT should omit sha")
-
-
-if __name__ == "__main__":
-    unittest.main()

tests/pilot/test_dashboard_select.py

@@ -1,74 +0,0 @@
-"""Tests for the model <select> in the repo edit form (Task D)."""
-import http.client
-import os
-import socket
-import sys
-import tempfile
-import threading
-import unittest
-
-HERE = os.path.dirname(os.path.abspath(__file__))
-ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
-sys.path.insert(0, os.path.join(ROOT, "pilot"))
-
-import dashboard as dash  # noqa: E402
-from pilot import cost_model, feedback  # noqa: E402
-
-
-def _free_port() -> int:
-    s = socket.socket()
-    s.bind(("127.0.0.1", 0))
-    port = s.getsockname()[1]
-    s.close()
-    return port
-
-
-def _get(port: int, path: str, headers: dict | None = None) -> tuple[int, dict, bytes]:
-    conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
-    conn.request("GET", path, headers=headers or {})
-    r = conn.getresponse()
-    body = r.read()
-    h = dict(r.getheaders())
-    conn.close()
-    return r.status, h, body
-
-
-class TestRepoEditSelect(unittest.TestCase):
-    def setUp(self):
-        self.tmp = tempfile.TemporaryDirectory()
-        self.db = f"{self.tmp.name}/f.db"
-        os.environ["PRAGENT_FEEDBACK_DB"] = self.db
-        os.environ["PRAGENT_BOT_TOKEN"] = ""
-        conn = feedback.init(self.db)
-        feedback.record_review(conn, repo="o/r", pr=1, head_sha="x")
-        conn.close()
-
-        self.port = _free_port()
-        from http.server import ThreadingHTTPServer
-        self.srv = ThreadingHTTPServer(("127.0.0.1", self.port), dash.Handler)
-        self.thread = threading.Thread(target=self.srv.serve_forever, daemon=True)
-        self.thread.start()
-        self.auth_hdr = {"X-Forwarded-User": "marcos@example.com"}
-
-    def tearDown(self):
-        self.srv.shutdown()
-        self.srv.server_close()
-        self.thread.join(timeout=2)
-        for k in ("PRAGENT_FEEDBACK_DB", "PRAGENT_BOT_TOKEN"):
-            os.environ.pop(k, None)
-        self.tmp.cleanup()
-
-    def test_repo_page_renders_select_with_one_option_per_price(self):
-        status, _h, body = _get(self.port, "/r/o/r", headers=self.auth_hdr)
-        self.assertEqual(status, 200)
-        text = body.decode()
-        self.assertIn('<select id="model" name="model">', text)
-        # Every PRICES key should appear as an <option value="…">.
-        for k in sorted(cost_model.PRICES):
-            self.assertIn(f'<option value="{k}"', text, f"missing {k} in select")
-        # Plus the "keep current" placeholder.
-        self.assertIn("— keep current", text)
-
-
-if __name__ == "__main__":
-    unittest.main()