feat(review): multi-lens orchestration — 5 parallel opencode subprocesses (security/docs/code-quality/tests/perf)

On by default, opt-out via "reviewers": []. 290 tests pass. - pilot/opencode_review.py: ReviewerSpec dataclass, default_reviewers(), parse_reviewers_config(), parse_triage_config(), resolve_reviewers(), _normalize_lens_finding(), posthash() (matches feedback.py scheme), _agreement_hash() (severity-free for cross-lens promotion), _tone_strip(), synthesize() 7-stage (severity_floor → tone-strip → length cap → per-lens max → per-file cap → dedup by _posthash → cross-lens severity promote → per-PR cap), run_lenses() (ThreadPoolExecutor pool=4), triage(), _intersect_with_triage(), _filter_by_skip_if(), run_lenses_review(). run() routes to fan-out when config.reviewers[] present or PRAGENT_REVIEWERS=1. - pilot/ai_review.py: parse_repo_config learns reviewers[] and triage objects (id regex /^[a-z0-9][a-z0-9-]{0,31}$/, 8-entry cap, agent_file/model/ severity_floor/max_findings/activation/skip_if_all_changed_paths/hotpath_globs). review_pr branches to opencode_review.run_lenses_review when configured. _render_collapsible_usage shows lenses: ... line when present. - .opencode/agents/{docs,code-quality,triage}.md: 3 new lens subagents. - .opencode/skills/lens-orchestration/SKILL.md: strict-JSON contract every lens subagent MUST honor. - .opencode/agents/pragent.md: slim to coordinator; no more hardcoded @security/@tests/@perf delegation; loads lens-orchestration skill. - .opencode/README.md: rewrite 'Add a review lens' recipe for multi-lens. - pilot/README-webhook.md: new 'Multi-lens pipeline' section (diagram + default roster + config schema + env vars + cross-lens dedup contract). - tests: 36 new tests (test_ai_review.py +12 reviewers/triage/usage, test_opencode_review.py +24 orchestration). posthash golden-vector matches feedback.py exactly across 5 severity × 2 line cases.
2026-08-20 22:16:21 +00:00
parent e8ebc54362
commit 78bcf6a9a0
11 changed files with 1847 additions and 51 deletions
@@ -225,6 +225,82 @@ so the `/tmp/pragent-work` emptyDir is writable.

 [csa]: https://labs.cloudsecurityalliance.org/research/csa-research-note-comment-control-github-prompt-injection-20/

+## Multi-lens pipeline (5 default lenses, on by default)
+
+Default `AI-REVIEW` runs spawn **one opencode subprocess per lens in parallel**
+and synthesize the merged findings before posting. Cheaper than 5 sequential
+reviews because the headroom proxy caches the byte-identical brief across
+lens calls (lenses 2..N hit cache).
+
+```
+   Gitea webhook
+      │
+      ▼
+   pilot/ai_review.review_pr
+      │  resolve config + sort changed paths
+      ▼
+   pilot/opencode_review.run_lenses_review
+      │  spawn 1..N subprocesses (default 5)
+      ▼
+   ┌── security ──┐  ┌── docs ──┐  ┌── code-quality ──┐  ┌── tests ──┐  ┌── perf ──┐
+   │  opencode     │  │ opencode  │  │   opencode       │  │  opencode  │  │ opencode  │
+   │  subprocess   │  │ subprocess│  │   subprocess     │  │  subprocess│  │ subprocess│
+   └──────┬────────┘  └─────┬────┘  └─────────┬────────┘  └─────┬──────┘  └─────┬─────┘
+          └──────────── synthesise (dedup, severity promote, cap) ─────────────┘
+                                  │
+                                  ▼
+                  post_inline_review (existing path, unchanged)
+```
+
+**Default roster** (5 lenses, all on `headroom/glm-5.2:cloud`):
+
+| id            | severity_floor | max_findings | target |
+|---------------|----------------|--------------|--------|
+| `security`    | low            | 12           | auth, crypto, secrets, SQL, file I/O, supply chain |
+| `docs`        | low            | 8            | README, CHANGELOG, docstrings, code-fence breakage |
+| `code-quality`| low            | 8            | dead code, hidden complexity, suppressed errors |
+| `tests`       | low            | 8            | coverage gaps for changed logic, missing assertions |
+| `perf`        | medium         | 6            | hot-path globs, O(n²) loops, N+1 queries |
+
+Set `.pr-review.json: "reviewers": []` to opt out (single-primary fallback).
+
+**Per-lens config** (drop-in):
+
+```jsonc
+{
+  "reviewers": [
+    { "id": "security", "severity_floor": "high", "max_findings": 10 },
+    { "id": "docs",     "activation": "off" },
+    { "id": "perf",     "skip_if_all_changed_paths": "docs/**" },
+    { "id": "my-lens",  "agent_file": ".opencode/agents/my-lens.md", "model": "headroom/glm-5.2:cloud" }
+  ],
+  "triage": { "enabled": true, "max_lenses": 4 },
+  "max_findings": 7
+}
+```
+
+Triage (off by default, but `enabled: true` recommended) runs a tiny
+primary agent that picks a subset of lenses based on the diff's changed
+files. Fail-open: if triage errors, all lenses run.
+
+**Env vars:**
+
+| var | default | effect |
+|-----|---------|--------|
+| `PRAGENT_MAX_PARALLEL_LENSES` | 4 | cap concurrency |
+| `PRAGENT_LENS_TIMEOUT` | 540 | per-lens subprocess timeout (s) |
+| `PRAGENT_REVIEWERS` | unset | force multi-lens fan-out even without `reviewers[]` |
+
+**Cross-lens dedup:** synthesiser drops duplicates by
+`sha256[:16](path|line|severity|problem[:80])` (matches the feedback DB's
+`posthash`), then promotes multi-lens agreement by one severity step
+(never past critical). A `[multi-lens]` tag is added so the summary
+section can flag it.
+
+**Adding a new lens**: drop `.opencode/agents/<id>.md` (use an existing
+one as a template), then add one entry to `reviewers[]`. That's it — no
+Python change, no image rebuild.
+
 ## Repo-local focus: `.pr-review.json` (optional)

 Drop a `.pr-review.json` at the repo root (committed on the PR's branch, or on
@@ -256,6 +332,74 @@ from what maintainers merged, not from the branch under review. A PR that
 Bad/missing file fails open to defaults. Fields are capped (32 list items ×
 200 chars; `instructions` 4000 chars). The bot's `read:repository` scope reads it.

+## Feedback loop (reactions → daily report)
+
+The bot learns from how humans react to its reviews. The loop has three parts:
+
+1. **Harvest** (every PR webhook). `pilot/feedback_harvest.py` walks back over
+   the PR's bot-authored reviews + inline comments + their reactions + their
+   reply threads + their resolved/unresolved state, and writes everything into
+   `/data/feedback.db` (SQLite, on the `pragent-feedback-data` PVC). It runs
+   inside the webhook pod, before the new review is scheduled — piggy-backs on
+   the webhook so there is no second cron just for harvesting. ~50 ms per PR.
+3. **Analyze** (`pilot/feedback_analyze.py`). Aggregates findings by `posthash`
+   (a sha256 of `path:line:severity:problem`) and computes per-finding scores:
+   - **false-positive score** = `-1` reactions + unresolved status + negation-
+     phrase replies ("false positive", "intentional", "not a bug"…) − upvotes
+     − resolved.
+   - **accepted-pattern score** = upvotes + resolved − downvotes − unresolved
+     − negation replies.
+   - **restraint** = fraction of reviewed PRs the bot left a finding on. The
+     DoorDash rule (2026-07-06, [ZenML recap](https://www.zenml.io/blog/llmops-database)):
+     *excessive noise on clean code is its own failure mode*. Above ~25% the
+     report flags ⚠️.
+   Renders markdown: top-N false-positive candidates, top-N accepted patterns,
+   a case-review queue (every disagreement with full context), and a
+   "where to action this" footer.
+4. **Deliver** (`pilot/feedback_post.py`). Posts the markdown as a comment on
+   a single long-lived issue `pragent feedback roll-up` in `gitea_admin/pragent`.
+   Comments are append-only history — one per run, timestamped.
+
+The daily CronJob (`k8s/pragent-feedback-cronjob.yaml`, schedule `7 3 * * *`)
+runs `feedback_post.py`. The webhook pod has `PRAGENT_FEEDBACK_DB=/data/feedback.db`;
+an empty / unset value disables harvesting (CI-step pod never gets the PVC).
+
+Human reactions are **not ground truth** — authors accept/reject for workflow
+reasons as often as for technical ones (DoorDash lesson). Treat the top-N lists
+as a *case-review queue*, not a directive. Re-read the PR before adding
+anything to `.pr-review.json:instructions` or the cross-repo `architecture.md`.
+
+### Acting on the report
+
+- **Per-repo**: add a `patterns.deny` glob to `.pr-review.json`, raise the
+  `severity_threshold`, or amend `instructions` — all read live at the next
+  review.
+- **Cross-repo**: append accepted patterns to the shared
+  `PRAGENT_ADDITIONAL_CONTEXT_URL` document on Nexus raw-hosted (e.g.
+  `canalhandia/architecture.md`). The next review picks it up via the
+  prompt-cached prefix → ~0 marginal cost on step 2+.
+- **Benchmark gate** (DoorDash pattern): before changing the model / prompt /
+  context window, replay the labeled `posthash` corpus against a candidate
+  change. If a candidate flips ≥ 1 currently-accepted finding into
+  false-positive, drop it.
+
+### Manual ops
+
+```bash
+# ad-hoc report (no post)
+python3 pilot/feedback_analyze.py --db /data/feedback.db --out /tmp/report.md
+
+# ad-hoc report for a window
+python3 pilot/feedback_analyze.py --db /data/feedback.db --since 1755000000
+
+# force-run the cron now
+kubectl -n pragent create job --from cronjob/pragent-feedback pragent-fb-now
+kubectl -n pragent logs -l app=pragent-feedback --tail=30
+
+# pause the cron
+kubectl -n pragent patch cronjob pragent-feedback -p '{"spec":{"suspend":true}}'
+```
+
 ## One-time per-owner setup: register a user-level webhook

 Gitea **system webhooks** (one webhook for the whole instance — the ideal) are
@@ -396,9 +540,17 @@ typescript-language-server / eslint / ruff) is built locally and imported into
 microk8s containerd — it is **not** pulled from a registry (`imagePullPolicy:
 Never`). The webhook secret + bot token are a Secret (`pragent-webhook`). An
 emptyDir at `/tmp/pragent-work` holds the per-review checkout + the warmed
-opencode runtime. Verified: a regular pod on kubernets reaches both
+opencode runtime. The PVC `pragent-feedback-data` (1 Gi, microk8s-hostpath,
+ReadWriteOnce) is mounted at `/data` and holds the SQLite file the feedback
+loop reads + writes — both the webhook pod and the daily CronJob pod share it.
+Verified: a regular pod on kubernets reaches both
 `<model-proxy-host>:8789` (headroom/glm) and `gitea-http.gitea.svc.cluster.local:3000`.

+The feedback CronJob lives in `~/k8s/pragent-feedback-cronjob.yaml` — same
+image, same PVC, schedule `7 3 * * *` (nudge off the round-hour). It runs
+`feedback_post.py`, which posts the daily report to the `pragent feedback
+roll-up` issue in `gitea_admin/pragent`.
+
 Build + deploy after editing the pilot scripts or the factory:

 ```bash
@@ -422,7 +574,9 @@ Env on the Deployment: `PRAGENT_ENGINE`, `OPENCODE_MODEL`,
 `PRAGENT_WORK_ROOT`, `PRAGENT_REVIEW_TIMEOUT`, `GITEA_API`, `OLLAMA_URL`,
 `OLLAMA_MODEL`, `OLLAMA_MAX_TOKENS`, `DIFF_MAX_CHARS`,
 `PRAGENT_ADDITIONAL_CONTEXT_URL` (optional, see "Repo-provided static
-context" above), `PRAGENT_MAX_CONCURRENT_REVIEWS`, `PRAGENT_MAX_BODY_BYTES`
+context" above), `PRAGENT_FEEDBACK_DB` (defaults to `/data/feedback.db` on
+the webhook; empty / unset disables harvesting — the CI-step path doesn't
+get the PVC), `PRAGENT_MAX_CONCURRENT_REVIEWS`, `PRAGENT_MAX_BODY_BYTES`
 are literals;
 `WEBHOOK_SECRET` + `PRAGENT_BOT_TOKEN` come from the Secret. The image now runs
 as uid 10001 — add `securityContext: {runAsNonRoot: true, runAsUser: 10001,