refactor: split pilot architecture
Remove the obsolete dashboard now that Langfuse is the analytics surface.\nIntroduce focused transport, model, and configuration modules while preserving the ai_review facade, and document the current runtime architecture.
This commit is contained in:
Claude
+65
-89
@@ -1,100 +1,76 @@
|
||||
# pragent pilot — AI Review bot
|
||||
# pragent pilot
|
||||
|
||||
A minimal AI code-review bot for Gitea, running as a CI step on the existing
|
||||
`act-runner`. This is the **pilot** — a small, self-contained reviewer that
|
||||
predates the full `pragent` framework (whose design lives in
|
||||
`docs/plans/2026-08-04-pragent-design.md`). The framework will later absorb
|
||||
this; until then, this is what runs.
|
||||
The pilot is a central, stdlib-only Gitea webhook service. It reviews opted-in
|
||||
pull requests with an on-network model, posts inline findings, and emits review
|
||||
telemetry to Langfuse. The service is fail-open: a review failure is reported
|
||||
as a PR comment and does not block CI.
|
||||
|
||||
## How it works
|
||||
## Runtime flow
|
||||
|
||||
1. You add `pragent-bot` to a repo and commit `.gitea/workflows/ai-review.yml`.
|
||||
2. On a PR, you add the **`AI-REVIEW`** label.
|
||||
3. Gitea Actions runs the workflow on the `act-runner`; it fetches the PR diff,
|
||||
asks `glm-5.2:cloud` (on-network via the headroom proxy) to review it, and
|
||||
posts the findings back as a PR review authored by `pragent-bot`.
|
||||
4. Remove the label to stop re-reviews on further pushes.
|
||||
1. Gitea sends a signed `pull_request` webhook.
|
||||
2. `webhook_server.py` validates the request, checks the base branch's
|
||||
`.pr-review.json` for `"enabled": true`, and claims `(repo, PR, SHA)`.
|
||||
3. `ai_review.review_pr()` fetches the diff, trusted config, and prior reviews.
|
||||
4. `opencode_review.py` checks out the PR head in a sanitized temporary
|
||||
directory and runs the review agent. The legacy Ollama-compatible path is
|
||||
still available through `PRAGENT_ENGINE`.
|
||||
5. The review output is parsed and normalized, valid post-change line anchors
|
||||
are separated from summary-only findings, and Gitea receives the result.
|
||||
6. `langfuse_trace.py` records usage, cost basis, findings, and evaluation
|
||||
scores when Langfuse credentials are configured.
|
||||
|
||||
Fail-open: the job always exits 0 and never blocks CI. Errors become a short
|
||||
"review failed" comment.
|
||||
## Module map
|
||||
|
||||
## Onboard a repo (3 steps)
|
||||
|
||||
### 1. Add `pragent-bot` as collaborator
|
||||
|
||||
Repo → Settings → Collaborators → Add → `pragent-bot` → permission **Write**.
|
||||
(Write is required to post reviews/comments.)
|
||||
|
||||
Or via API (with an admin/owner token):
|
||||
|
||||
```bash
|
||||
curl -X PUT -H "Authorization: token $OWNER_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"permission":"write"}' \
|
||||
"http://<gitea-host>:3000/api/v1/repos/OWNER/REPO/collaborators/pragent-bot"
|
||||
```
|
||||
|
||||
### 2. Add the `PRAGENT_BOT_TOKEN` secret
|
||||
|
||||
Repo → Settings → Actions → Secrets → New secret → name `PRAGENT_BOT_TOKEN`,
|
||||
value = the bot's access token (ask the platform admin; stored mode-600 at
|
||||
`~/.claude/.pragent-bot-token` on the admin host).
|
||||
|
||||
### 3. Commit the workflow
|
||||
|
||||
Copy `pilot/workflow-template.yml` into the target repo as
|
||||
`.gitea/workflows/ai-review.yml` and commit it. That's it.
|
||||
|
||||
## Use it
|
||||
|
||||
Open a PR (or push to an open one), add the **`AI-REVIEW`** label. The review
|
||||
appears within ~30–90s depending on diff size and model latency.
|
||||
|
||||
## What's intentionally NOT in the pilot
|
||||
|
||||
Deferred to the full framework (by design, see the design doc):
|
||||
|
||||
- Attention tiering (trivial/lite/full/oversized) and per-tier cost control.
|
||||
- Multiple analyzer fan-out over a shared cached prompt prefix.
|
||||
- Prior-comment synthesis (so each push re-posts; the latest review is tagged
|
||||
with the head SHA so it's easy to spot).
|
||||
- Inline line comments and status checks.
|
||||
- `pragent explain` / `replay` / analytics JSONL.
|
||||
- A second forge (GitLab) and the provider matrix.
|
||||
|
||||
## Pieces
|
||||
|
||||
| File | Role |
|
||||
| Module | Responsibility |
|
||||
|---|---|
|
||||
| `pilot/ai_review.py` | The reviewer script (stdlib only). Single source of truth — fetched at runtime by each repo's workflow. |
|
||||
| `pilot/workflow-template.yml` | The Gitea Action consumers copy into `.gitea/workflows/ai-review.yml`. |
|
||||
| `tests/pilot/test_ai_review.py` | Unit tests for the pure helpers (no network). |
|
||||
| `webhook_server.py` | HTTP ingress, signature verification, opt-in gate, concurrency |
|
||||
| `review_config.py` | Trusted base-branch opt-in policy; transport injected for tests |
|
||||
| `gitea_client.py` | HTTP transport adapter and repository-scoped client |
|
||||
| `ai_review.py` | Compatibility facade and review orchestration |
|
||||
| `model_client.py` | Anthropic-compatible model adapter and response text extraction |
|
||||
| `opencode_review.py` | Hostile-checkout containment and agent execution |
|
||||
| `diff_compress.py` | Diff compression and prior-review extraction |
|
||||
| `feedback*.py` | Feedback persistence, harvesting, analysis, and Langfuse scores |
|
||||
| `langfuse_trace.py` | Fail-open Langfuse ingestion and cost metadata |
|
||||
| `cost_model.py` | Provider price catalog and equivalent-cost calculations |
|
||||
| `eval_*.py` | Dataset bootstrap, evaluators, and behavioral scoring |
|
||||
|
||||
## Run the tests
|
||||
`ai_review.py` remains the stable import surface for existing workflow and
|
||||
webhook deployments. New code should put policy, adapters, and pure transforms
|
||||
in the focused modules above rather than adding unrelated functions there.
|
||||
|
||||
## Onboard a repository
|
||||
|
||||
1. Add `pragent-bot` as a Write collaborator.
|
||||
2. Commit this file to the default branch:
|
||||
|
||||
```json
|
||||
{"enabled": true}
|
||||
```
|
||||
|
||||
3. Open or update a pull request.
|
||||
|
||||
No per-repository workflow, secret, or label is required for the central
|
||||
webhook path. See [`README-webhook.md`](README-webhook.md) for deployment,
|
||||
security, and webhook registration details.
|
||||
|
||||
## Configuration
|
||||
|
||||
| Variable | Default | Purpose |
|
||||
|---|---:|---|
|
||||
| `GITEA_API` | in-cluster URL | Gitea API base URL |
|
||||
| `PRAGENT_BOT_TOKEN` | — | Bot credential |
|
||||
| `OLLAMA_URL` / `OLLAMA_MODEL` | headroom / `glm-5.2:cloud` | Legacy model path |
|
||||
| `PRAGENT_ENGINE` | `opencode` | `opencode` or legacy model path |
|
||||
| `DIFF_MAX_CHARS` | `150000` | Diff input cap |
|
||||
| `PRAGENT_MAX_CONCURRENT_REVIEWS` | `2` | Process concurrency bound |
|
||||
| `LANGFUSE_HOST` + keys | unset | Enables telemetry; unset is a no-op |
|
||||
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
cd ~/Projects/pragent
|
||||
PYTHONPATH=pilot python3 -m pytest tests/pilot/ # if pytest available
|
||||
# or, without pytest:
|
||||
python3 - <<'PY'
|
||||
import os, sys, importlib.util
|
||||
sys.path.insert(0, os.path.abspath("pilot"))
|
||||
import ai_review # noqa: F401
|
||||
spec = importlib.util.spec_from_file_location("t", "tests/pilot/test_ai_review.py")
|
||||
m = importlib.util.module_from_spec(spec); spec.loader.exec_module(m)
|
||||
fails = 0
|
||||
for n in sorted(x for x in dir(m) if x.startswith("test_")):
|
||||
try: getattr(m, n)(); print("PASS", n)
|
||||
except Exception as e: fails += 1; print("FAIL", n, e)
|
||||
print("failed:", fails)
|
||||
PY
|
||||
python3 -m pytest tests -q
|
||||
```
|
||||
|
||||
## Configuration knobs (env in the workflow)
|
||||
|
||||
| Env | Default | Purpose |
|
||||
|---|---|---|
|
||||
| `OLLAMA_MODEL` | `glm-5.2:cloud` | Model id passed to the headroom proxy. |
|
||||
| `OLLAMA_MAX_TOKENS` | `6000` | Output token cap. |
|
||||
| `DIFF_MAX_CHARS` | `150000` | Diff truncation cap (with a noted truncation marker). |
|
||||
| `OLLAMA_URL` | `http://<model-proxy-host>:8789` | headroom proxy (tailnet). If the act-runner can't reach the tailnet IP, expose 8789 as an in-cluster Service+Endpoints and set this to the cluster DNS name. |
|
||||
Tests use mocked transports and local fixtures. They do not require Gitea,
|
||||
Langfuse, a model endpoint, or network access.
|
||||
|
||||
Reference in New Issue
Block a user