feat(pilot): minimal AI review bot for Gitea Actions

Ships a working pragent pilot ahead of the framework build (design doc
deferred). Single stdlib-only reviewer script fetched at runtime by a per-repo
Gitea Action; reviews fire only on PRs with the AI-REVIEW label; model is
glm-5.2:cloud via the on-network headroom proxy; fail-open.

- pilot/ai_review.py: fetch PR diff, call model, post review as pragent-bot
- pilot/workflow-template.yml: per-repo Gitea Action gated on AI-REVIEW label
- pilot/README.md: onboarding (bot collaborator + secret + workflow + label)
- tests/pilot/test_ai_review.py: 15 unit tests for pure helpers (no network)
- README/design doc: note pilot is the bootstrap; framework build deferred

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Marcos
2026-08-17 18:18:06 +00:00
parent 8a4239a9d9
commit 6f012e9b66
7 changed files with 529 additions and 2 deletions
+3
View File
@@ -4,3 +4,6 @@ dist/
.pragent/cache/
*.jsonl
.env
__pycache__/
*.pyc
+6 -2
View File
@@ -3,8 +3,12 @@
An extensible, forge-agnostic PR review framework. Not a product — a toolkit that teams
extend with their own review dimensions.
**Status:** design approved, implementation not started. See
[`docs/plans/2026-08-04-pragent-design.md`](docs/plans/2026-08-04-pragent-design.md).
**Status:** design approved; framework build deferred. A **pilot** is live — a
minimal AI review bot running as a Gitea Actions step on `glm-5.2:cloud`. See
[`pilot/README.md`](pilot/README.md) to onboard a repo. The framework design
remains at [`docs/plans/2026-08-04-pragent-design.md`](docs/plans/2026-08-04-pragent-design.md);
the pilot is its bootstrap and will be superseded by `pragent review` when the
framework build resumes.
## What it is
+21
View File
@@ -297,3 +297,24 @@ and they may be worth contributing upstream rather than shipping standalone.
- Auto-fix commits (findings and suggestions only — writing to branches is a later,
separately-gated decision)
- Web dashboard (event schema first; dashboard is a downstream consumer)
## Bootstrap: the pilot (2026-08-17)
Before the framework build, a minimal **pilot** was shipped to get a working AI
review bot live immediately (user decision 2026-08-17). It is deliberately not
this design:
- Single Python script (`pilot/ai_review.py`, stdlib only) fetched at runtime by
a per-repo Gitea Action (`pilot/workflow-template.yml`).
- One model, `glm-5.2:cloud` via the on-network headroom proxy — no tiering, no
analyzer fan-out, no shared-prefix caching, no provider matrix.
- Trigger = `AI-REVIEW` label on a PR (deterministic skip, the cheapest tier
rule, but expressed as a Gitea Actions `if` gate rather than the tier engine).
- Fail-open, comment-only review; no inline line comments, no status checks,
no analytics JSONL, no `explain`/`replay`.
- Re-posts on every push (no prior-comment synthesis — §6.1 — yet).
The pilot validates the delivery model (CI-step + bot user, not a central
webhook) and the on-network provider path. When the framework build resumes,
`pilot/ai_review.py` is replaced by `pragent review`; the per-repo workflow
stays, just calling the CLI instead of curling the script. See `pilot/README.md`.
+100
View File
@@ -0,0 +1,100 @@
# pragent pilot — AI Review bot
A minimal AI code-review bot for Gitea, running as a CI step on the existing
`act-runner`. This is the **pilot** — a small, self-contained reviewer that
predates the full `pragent` framework (whose design lives in
`docs/plans/2026-08-04-pragent-design.md`). The framework will later absorb
this; until then, this is what runs.
## How it works
1. You add `pragent-bot` to a repo and commit `.gitea/workflows/ai-review.yml`.
2. On a PR, you add the **`AI-REVIEW`** label.
3. Gitea Actions runs the workflow on the `act-runner`; it fetches the PR diff,
asks `glm-5.2:cloud` (on-network via the headroom proxy) to review it, and
posts the findings back as a PR review authored by `pragent-bot`.
4. Remove the label to stop re-reviews on further pushes.
Fail-open: the job always exits 0 and never blocks CI. Errors become a short
"review failed" comment.
## Onboard a repo (3 steps)
### 1. Add `pragent-bot` as collaborator
Repo → Settings → Collaborators → Add → `pragent-bot` → permission **Write**.
(Write is required to post reviews/comments.)
Or via API (with an admin/owner token):
```bash
curl -X PUT -H "Authorization: token $OWNER_TOKEN" \
-H "Content-Type: application/json" \
-d '{"permission":"write"}' \
"http://100.74.17.70:30000/api/v1/repos/OWNER/REPO/collaborators/pragent-bot"
```
### 2. Add the `PRAGENT_BOT_TOKEN` secret
Repo → Settings → Actions → Secrets → New secret → name `PRAGENT_BOT_TOKEN`,
value = the bot's access token (ask the platform admin; stored mode-600 at
`~/.claude/.pragent-bot-token` on the admin host).
### 3. Commit the workflow
Copy `pilot/workflow-template.yml` into the target repo as
`.gitea/workflows/ai-review.yml` and commit it. That's it.
## Use it
Open a PR (or push to an open one), add the **`AI-REVIEW`** label. The review
appears within ~3090s depending on diff size and model latency.
## What's intentionally NOT in the pilot
Deferred to the full framework (by design, see the design doc):
- Attention tiering (trivial/lite/full/oversized) and per-tier cost control.
- Multiple analyzer fan-out over a shared cached prompt prefix.
- Prior-comment synthesis (so each push re-posts; the latest review is tagged
with the head SHA so it's easy to spot).
- Inline line comments and status checks.
- `pragent explain` / `replay` / analytics JSONL.
- A second forge (GitLab) and the provider matrix.
## Pieces
| File | Role |
|---|---|
| `pilot/ai_review.py` | The reviewer script (stdlib only). Single source of truth — fetched at runtime by each repo's workflow. |
| `pilot/workflow-template.yml` | The Gitea Action consumers copy into `.gitea/workflows/ai-review.yml`. |
| `tests/pilot/test_ai_review.py` | Unit tests for the pure helpers (no network). |
## Run the tests
```bash
cd ~/Projects/pragent
PYTHONPATH=pilot python3 -m pytest tests/pilot/ # if pytest available
# or, without pytest:
python3 - <<'PY'
import os, sys, importlib.util
sys.path.insert(0, os.path.abspath("pilot"))
import ai_review # noqa: F401
spec = importlib.util.spec_from_file_location("t", "tests/pilot/test_ai_review.py")
m = importlib.util.module_from_spec(spec); spec.loader.exec_module(m)
fails = 0
for n in sorted(x for x in dir(m) if x.startswith("test_")):
try: getattr(m, n)(); print("PASS", n)
except Exception as e: fails += 1; print("FAIL", n, e)
print("failed:", fails)
PY
```
## Configuration knobs (env in the workflow)
| Env | Default | Purpose |
|---|---|---|
| `OLLAMA_MODEL` | `glm-5.2:cloud` | Model id passed to the headroom proxy. |
| `OLLAMA_MAX_TOKENS` | `6000` | Output token cap. |
| `DIFF_MAX_CHARS` | `150000` | Diff truncation cap (with a noted truncation marker). |
| `OLLAMA_URL` | `http://100.74.17.70:8789` | headroom proxy (tailnet). If the act-runner can't reach the tailnet IP, expose 8789 as an in-cluster Service+Endpoints and set this to the cluster DNS name. |
+221
View File
@@ -0,0 +1,221 @@
#!/usr/bin/env python3
"""pragent pilot — minimal AI PR reviewer.
Runs as a Gitea Actions step. Fetches a PR diff, asks glm-5.2:cloud (via the
on-network headroom proxy, Anthropic /v1/messages format) to review it, and
posts the findings back as a PR review authored by pragent-bot.
Fail-open by design: any error becomes a short "review failed" review comment,
and the process always exits 0 so it can never block CI.
Stdlib only — no pip install, fast cold start in CI.
Env:
GITEA_API base URL of the in-cluster Gitea, e.g. http://gitea-http.gitea.svc.cluster.local:3000
GITEA_REPOSITORY "owner/repo" of the PR (github.repository)
PR_INDEX PR number (github.event.pull_request.number)
PR_TITLE PR title
PR_BODY PR body (optional)
PRAGENT_BOT_TOKEN bot access token (repo secret)
PRAGENT_SHA head SHA to tag the review (github.event.pull_request.head.sha)
OLLAMA_URL headroom proxy URL, e.g. http://100.74.17.70:8789
OLLAMA_MODEL model id, e.g. glm-5.2:cloud
OLLAMA_MAX_TOKENS (optional) output cap, default 6000
DIFF_MAX_CHARS (optional) diff truncation cap, default 150000
"""
import json
import os
import sys
import urllib.error
import urllib.request
REVIEW_HEADER = "🤖 **AI Review** · pragent pilot · {model} · `{sha}`"
SYSTEM_PROMPT = """You are a senior, pragmatic code reviewer. Review the pull request diff below.
Report ONLY real, actionable issues: correctness bugs, security problems, risky
changes, missing tests for changed behaviour, and breaking API/contract changes.
For each issue, output exactly one line in this format:
- [SEVERITY] path:line — concise problem. suggested fix.
where SEVERITY is one of: critical, high, medium, low.
Rules:
- Skip nitpicks, pure formatting, and praise.
- If the diff is clean, output exactly: No issues found.
- Be concise. At most ~15 findings, highest severity first.
- Do not restate the diff. Do not include a summary header. Just the findings lines."""
# ---------------------------------------------------------------------------
# Pure helpers (unit-tested, no network)
# ---------------------------------------------------------------------------
def truncate_diff(text: str, max_chars: int) -> tuple[str, bool, int]:
"""Return (text, was_truncated, original_len). Never raises on bad input."""
if text is None:
return "", False, 0
orig_len = len(text)
if orig_len <= max_chars:
return text, False, orig_len
return text[:max_chars] + f"\n\n[diff truncated at {max_chars} characters]\n", True, orig_len
def parse_text_blocks(content: list) -> str:
"""Join `type:"text"` blocks from an Anthropic /v1/messages response.
Drops `thinking` blocks (glm-5.2:cloud is a reasoning model and emits them).
Tolerates missing/malformed blocks by skipping them.
"""
if not isinstance(content, list):
return ""
out = []
for block in content:
if not isinstance(block, dict):
continue
if block.get("type") == "text" and isinstance(block.get("text"), str):
out.append(block["text"])
return "\n".join(out).strip()
def format_review_body(findings: str, model: str, sha: str) -> str:
"""Format the posted review body. Findings empty -> "No issues found."."""
header = REVIEW_HEADER.format(model=model, sha=sha[:8] if sha else "unknown")
findings = (findings or "").strip()
if not findings:
findings = "No issues found."
return f"{header}\n\n{findings}"
def build_user_prompt(title: str, body: str, diff: str) -> str:
parts = [f"## PR\nTitle: {title or '(none)'}"]
if body and body.strip():
b = body.strip()
if len(b) > 4000:
b = b[:4000] + "\n…[PR body truncated]"
parts.append(f"Description:\n{b}")
parts.append(f"## Diff\n```diff\n{diff}\n```")
return "\n\n".join(parts)
# ---------------------------------------------------------------------------
# Network helpers
# ---------------------------------------------------------------------------
def _http(method: str, url: str, token: str, body: dict | None = None, accept: str = "application/json") -> tuple[int, bytes]:
headers = {"Authorization": f"token {token}", "Accept": accept}
data = None
if body is not None:
data = json.dumps(body).encode()
headers["Content-Type"] = "application/json"
req = urllib.request.Request(url, data=data, headers=headers, method=method)
try:
with urllib.request.urlopen(req, timeout=180) as r:
return r.status, r.read()
except urllib.error.HTTPError as e:
return e.code, e.read()
except urllib.error.URLError as e:
raise RuntimeError(f"network error: {e.reason}") from e
def gitea_get(api: str, repo: str, path: str, token: str, accept: str = "application/json") -> tuple[int, bytes]:
return _http("GET", f"{api}/api/v1/repos/{repo}/{path}", token, None, accept)
def gitea_post(api: str, repo: str, path: str, token: str, body: dict) -> tuple[int, bytes]:
return _http("POST", f"{api}/api/v1/repos/{repo}/{path}", token, body)
def fetch_pr_diff(api: str, repo: str, index: str, token: str, max_chars: int) -> tuple[str, bool, int]:
"""Get the unified diff. Try the `.diff` suffix first, fall back to the
files endpoint (join `patch` fields) if the server does not serve .diff."""
status, raw = gitea_get(api, repo, f"pulls/{index}.diff", token, accept="text/plain")
if status == 200:
return truncate_diff(raw.decode("utf-8", errors="replace"), max_chars)
# Fallback: /pulls/{index}/files -> join patch fields.
status, raw = gitea_get(api, repo, f"pulls/{index}/files", token)
if status != 200:
raise RuntimeError(f"could not fetch diff: .diff={status}, files={status}")
files = json.loads(raw)
joined = []
for f in files:
h = f.get("filename", "?")
joined.append(f"--- {h}\n+++ {h}\n{f.get('patch', '(binary or no patch)')}")
return truncate_diff("\n".join(joined), max_chars)
def call_model(ollama_url: str, model: str, system: str, user: str, max_tokens: int) -> str:
payload = {
"model": model,
"max_tokens": max_tokens,
"system": system,
"messages": [{"role": "user", "content": user}],
}
status, raw = _http(
"POST",
f"{ollama_url.rstrip('/')}/v1/messages",
"ollama", # headroom ollama hub uses x-api-key: ollama
payload,
)
if status != 200:
raise RuntimeError(f"model call failed: HTTP {status}: {raw[:500].decode('utf-8', errors='replace')}")
data = json.loads(raw)
return parse_text_blocks(data.get("content", []))
def post_review(api: str, repo: str, index: str, token: str, body: str) -> None:
status, raw = gitea_post(api, repo, f"pulls/{index}/reviews", token, {"event": "COMMENT", "body": body})
if status not in (200, 201):
# Fallback to a plain issue comment if reviews endpoint refuses.
status2, raw2 = gitea_post(api, repo, f"issues/{index}/comments", token, {"body": body})
if status2 not in (200, 201):
raise RuntimeError(f"post review failed: reviews={status}, comments={status2}")
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
def _need(name: str) -> str:
v = os.environ.get(name)
if not v:
raise RuntimeError(f"missing env {name}")
return v
def run() -> int:
api = _need("GITEA_API")
repo = _need("GITEA_REPOSITORY")
index = _need("PR_INDEX")
token = _need("PRAGENT_BOT_TOKEN")
ollama_url = _need("OLLAMA_URL")
model = _need("OLLAMA_MODEL")
title = os.environ.get("PR_TITLE", "")
body = os.environ.get("PR_BODY", "")
sha = os.environ.get("PRAGENT_SHA", "")
max_tokens = int(os.environ.get("OLLAMA_MAX_TOKENS", "6000"))
max_chars = int(os.environ.get("DIFF_MAX_CHARS", "150000"))
try:
diff, truncated, _orig = fetch_pr_diff(api, repo, index, token, max_chars)
if not diff.strip():
post_review(api, repo, index, token, format_review_body("No diff content to review.", model, sha))
return 0
user_prompt = build_user_prompt(title, body, diff)
findings = call_model(ollama_url, model, SYSTEM_PROMPT, user_prompt, max_tokens)
post_review(api, repo, index, token, format_review_body(findings, model, sha))
except Exception as e: # fail-open
try:
post_review(api, repo, index, token, format_review_body(f"⚠️ AI review failed: {e}", model, sha))
except Exception as e2:
print(f"pragent: could not post failure note: {e2}", file=sys.stderr)
print(f"pragent: review failed: {e}", file=sys.stderr)
return 0
if __name__ == "__main__":
sys.exit(run())
+45
View File
@@ -0,0 +1,45 @@
# pragent pilot — AI Review workflow template.
#
# Copy this file into the repo you want reviewed as:
# .gitea/workflows/ai-review.yml
#
# Prerequisites (see pilot/README.md):
# 1. pragent-bot added as a collaborator with Write access.
# 2. repo secret PRAGENT_BOT_TOKEN set to the bot's access token.
#
# Reviews fire ONLY on PRs carrying the `AI-REVIEW` label. Remove the label to
# stop a re-review on subsequent pushes. The job is fail-open (never blocks CI).
name: AI Review
on:
pull_request:
types: [opened, synchronize, reopened, labeled]
jobs:
review:
# Only run when the PR has the AI-REVIEW label. Acts as a cheap gate: no
# model call, no cost, when the label is absent.
if: contains(github.event.pull_request.labels.*.name, 'AI-REVIEW')
runs-on: ubuntu-latest
steps:
- name: Run pragent pilot review
env:
GITEA_API: http://gitea-http.gitea.svc.cluster.local:3000
GITEA_REPOSITORY: ${{ github.repository }}
PR_INDEX: ${{ github.event.pull_request.number }}
PR_TITLE: ${{ github.event.pull_request.title }}
PR_BODY: ${{ github.event.pull_request.body }}
PRAGENT_BOT_TOKEN: ${{ secrets.PRAGENT_BOT_TOKEN }}
PRAGENT_SHA: ${{ github.event.pull_request.head.sha }}
# On-network model: headroom proxy on kubernets (tailnet IP).
OLLAMA_URL: http://100.74.17.70:8789
OLLAMA_MODEL: glm-5.2:cloud
OLLAMA_MAX_TOKENS: "6000"
DIFF_MAX_CHARS: "150000"
run: |
set -e
# Fetch the reviewer script from the pragent repo (private → bot token).
curl -fsS -H "Authorization: token $PRAGENT_BOT_TOKEN" \
"$GITEA_API/api/v1/repos/gitea_admin/pragent/raw/branch/main/pilot/ai_review.py" -o ai_review.py
python3 ai_review.py
+133
View File
@@ -0,0 +1,133 @@
"""Unit tests for pragent pilot pure helpers. No network."""
import os
import sys
# Allow running without install: add repo root to path.
HERE = os.path.dirname(os.path.abspath(__file__))
ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
sys.path.insert(0, os.path.join(ROOT, "pilot"))
from ai_review import ( # noqa: E402
build_user_prompt,
format_review_body,
parse_text_blocks,
truncate_diff,
)
# ---------------------------------------------------------------------------
# truncate_diff
# ---------------------------------------------------------------------------
def test_truncate_diff_short():
text, truncated, n = truncate_diff("abc", 100)
assert text == "abc"
assert truncated is False
assert n == 3
def test_truncate_diff_exact_boundary():
text, truncated, n = truncate_diff("x" * 100, 100)
assert truncated is False
assert n == 100
assert text == "x" * 100
def test_truncate_diff_over_cap():
text, truncated, n = truncate_diff("x" * 250, 100)
assert truncated is True
assert n == 250
assert text.startswith("x" * 100)
assert "[diff truncated at 100 characters]" in text
def test_truncate_diff_none():
text, truncated, n = truncate_diff(None, 100) # type: ignore[arg-type]
assert text == ""
assert truncated is False
assert n == 0
# ---------------------------------------------------------------------------
# parse_text_blocks
# ---------------------------------------------------------------------------
def test_parse_text_blocks_text_only():
content = [{"type": "text", "text": "hello"}, {"type": "text", "text": "world"}]
assert parse_text_blocks(content) == "hello\nworld"
def test_parse_text_blocks_drops_thinking():
content = [
{"type": "thinking", "thinking": "reasoning here"},
{"type": "text", "text": "- [high] a.go:3 — bug. fix."},
]
assert parse_text_blocks(content) == "- [high] a.go:3 — bug. fix."
def test_parse_text_blocks_empty_and_malformed():
assert parse_text_blocks([]) == ""
assert parse_text_blocks(None) == "" # type: ignore[arg-type]
assert parse_text_blocks([{"type": "text"}, "garbage", 5]) == ""
def test_parse_text_blocks_real_glm_shape():
# Captured from glm-5.2:cloud via headroom 8789.
content = [
{"type": "thinking", "thinking": "Analyze the request..."},
{"type": "text", "text": "- [critical] auth.py:12 — token compared with `==`. Use hmac.compare_digest."},
]
assert "compare_digest" in parse_text_blocks(content)
# ---------------------------------------------------------------------------
# format_review_body
# ---------------------------------------------------------------------------
def test_format_review_body_findings():
body = format_review_body("- [high] x:1 — bug. fix.", "glm-5.2:cloud", "abcdef1234567890")
assert "pragent pilot" in body
assert "glm-5.2:cloud" in body
assert "`abcdef12`" in body # 8-char sha
assert "- [high] x:1" in body
def test_format_review_body_empty_findings():
body = format_review_body("", "glm-5.2:cloud", "abcdef1234567890")
assert "No issues found." in body
def test_format_review_body_whitespace_findings():
body = format_review_body(" \n ", "glm-5.2:cloud", "abcdef1234567890")
assert "No issues found." in body
def test_format_review_body_no_sha():
body = format_review_body("- [low] y:2 — nit", "glm-5.2:cloud", "")
assert "`unknown`" in body
# ---------------------------------------------------------------------------
# build_user_prompt
# ---------------------------------------------------------------------------
def test_build_user_prompt_includes_title_and_diff():
p = build_user_prompt("Fix login", "Closes #1", "diff --git a/x b/x")
assert "Fix login" in p
assert "Closes #1" in p
assert "diff --git a/x b/x" in p
def test_build_user_prompt_truncates_long_body():
long_body = "B" * 6000
p = build_user_prompt("t", long_body, "d")
assert "[PR body truncated]" in p
assert p.count("B") < 6000
def test_build_user_prompt_no_body():
p = build_user_prompt("t", "", "d")
assert "Description:" not in p