diff --git a/.gitignore b/.gitignore index 5941895..b914de1 100644 --- a/.gitignore +++ b/.gitignore @@ -4,3 +4,6 @@ dist/ .pragent/cache/ *.jsonl .env + +__pycache__/ +*.pyc diff --git a/README.md b/README.md index 70347fe..f8922d3 100644 --- a/README.md +++ b/README.md @@ -3,8 +3,12 @@ An extensible, forge-agnostic PR review framework. Not a product — a toolkit that teams extend with their own review dimensions. -**Status:** design approved, implementation not started. See -[`docs/plans/2026-08-04-pragent-design.md`](docs/plans/2026-08-04-pragent-design.md). +**Status:** design approved; framework build deferred. A **pilot** is live — a +minimal AI review bot running as a Gitea Actions step on `glm-5.2:cloud`. See +[`pilot/README.md`](pilot/README.md) to onboard a repo. The framework design +remains at [`docs/plans/2026-08-04-pragent-design.md`](docs/plans/2026-08-04-pragent-design.md); +the pilot is its bootstrap and will be superseded by `pragent review` when the +framework build resumes. ## What it is diff --git a/docs/plans/2026-08-04-pragent-design.md b/docs/plans/2026-08-04-pragent-design.md index ba85656..f56758e 100644 --- a/docs/plans/2026-08-04-pragent-design.md +++ b/docs/plans/2026-08-04-pragent-design.md @@ -297,3 +297,24 @@ and they may be worth contributing upstream rather than shipping standalone. - Auto-fix commits (findings and suggestions only — writing to branches is a later, separately-gated decision) - Web dashboard (event schema first; dashboard is a downstream consumer) + +## Bootstrap: the pilot (2026-08-17) + +Before the framework build, a minimal **pilot** was shipped to get a working AI +review bot live immediately (user decision 2026-08-17). It is deliberately not +this design: + +- Single Python script (`pilot/ai_review.py`, stdlib only) fetched at runtime by + a per-repo Gitea Action (`pilot/workflow-template.yml`). +- One model, `glm-5.2:cloud` via the on-network headroom proxy — no tiering, no + analyzer fan-out, no shared-prefix caching, no provider matrix. +- Trigger = `AI-REVIEW` label on a PR (deterministic skip, the cheapest tier + rule, but expressed as a Gitea Actions `if` gate rather than the tier engine). +- Fail-open, comment-only review; no inline line comments, no status checks, + no analytics JSONL, no `explain`/`replay`. +- Re-posts on every push (no prior-comment synthesis — §6.1 — yet). + +The pilot validates the delivery model (CI-step + bot user, not a central +webhook) and the on-network provider path. When the framework build resumes, +`pilot/ai_review.py` is replaced by `pragent review`; the per-repo workflow +stays, just calling the CLI instead of curling the script. See `pilot/README.md`. diff --git a/pilot/README.md b/pilot/README.md new file mode 100644 index 0000000..ca1ed36 --- /dev/null +++ b/pilot/README.md @@ -0,0 +1,100 @@ +# pragent pilot — AI Review bot + +A minimal AI code-review bot for Gitea, running as a CI step on the existing +`act-runner`. This is the **pilot** — a small, self-contained reviewer that +predates the full `pragent` framework (whose design lives in +`docs/plans/2026-08-04-pragent-design.md`). The framework will later absorb +this; until then, this is what runs. + +## How it works + +1. You add `pragent-bot` to a repo and commit `.gitea/workflows/ai-review.yml`. +2. On a PR, you add the **`AI-REVIEW`** label. +3. Gitea Actions runs the workflow on the `act-runner`; it fetches the PR diff, + asks `glm-5.2:cloud` (on-network via the headroom proxy) to review it, and + posts the findings back as a PR review authored by `pragent-bot`. +4. Remove the label to stop re-reviews on further pushes. + +Fail-open: the job always exits 0 and never blocks CI. Errors become a short +"review failed" comment. + +## Onboard a repo (3 steps) + +### 1. Add `pragent-bot` as collaborator + +Repo → Settings → Collaborators → Add → `pragent-bot` → permission **Write**. +(Write is required to post reviews/comments.) + +Or via API (with an admin/owner token): + +```bash +curl -X PUT -H "Authorization: token $OWNER_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"permission":"write"}' \ + "http://100.74.17.70:30000/api/v1/repos/OWNER/REPO/collaborators/pragent-bot" +``` + +### 2. Add the `PRAGENT_BOT_TOKEN` secret + +Repo → Settings → Actions → Secrets → New secret → name `PRAGENT_BOT_TOKEN`, +value = the bot's access token (ask the platform admin; stored mode-600 at +`~/.claude/.pragent-bot-token` on the admin host). + +### 3. Commit the workflow + +Copy `pilot/workflow-template.yml` into the target repo as +`.gitea/workflows/ai-review.yml` and commit it. That's it. + +## Use it + +Open a PR (or push to an open one), add the **`AI-REVIEW`** label. The review +appears within ~30–90s depending on diff size and model latency. + +## What's intentionally NOT in the pilot + +Deferred to the full framework (by design, see the design doc): + +- Attention tiering (trivial/lite/full/oversized) and per-tier cost control. +- Multiple analyzer fan-out over a shared cached prompt prefix. +- Prior-comment synthesis (so each push re-posts; the latest review is tagged + with the head SHA so it's easy to spot). +- Inline line comments and status checks. +- `pragent explain` / `replay` / analytics JSONL. +- A second forge (GitLab) and the provider matrix. + +## Pieces + +| File | Role | +|---|---| +| `pilot/ai_review.py` | The reviewer script (stdlib only). Single source of truth — fetched at runtime by each repo's workflow. | +| `pilot/workflow-template.yml` | The Gitea Action consumers copy into `.gitea/workflows/ai-review.yml`. | +| `tests/pilot/test_ai_review.py` | Unit tests for the pure helpers (no network). | + +## Run the tests + +```bash +cd ~/Projects/pragent +PYTHONPATH=pilot python3 -m pytest tests/pilot/ # if pytest available +# or, without pytest: +python3 - <<'PY' +import os, sys, importlib.util +sys.path.insert(0, os.path.abspath("pilot")) +import ai_review # noqa: F401 +spec = importlib.util.spec_from_file_location("t", "tests/pilot/test_ai_review.py") +m = importlib.util.module_from_spec(spec); spec.loader.exec_module(m) +fails = 0 +for n in sorted(x for x in dir(m) if x.startswith("test_")): + try: getattr(m, n)(); print("PASS", n) + except Exception as e: fails += 1; print("FAIL", n, e) +print("failed:", fails) +PY +``` + +## Configuration knobs (env in the workflow) + +| Env | Default | Purpose | +|---|---|---| +| `OLLAMA_MODEL` | `glm-5.2:cloud` | Model id passed to the headroom proxy. | +| `OLLAMA_MAX_TOKENS` | `6000` | Output token cap. | +| `DIFF_MAX_CHARS` | `150000` | Diff truncation cap (with a noted truncation marker). | +| `OLLAMA_URL` | `http://100.74.17.70:8789` | headroom proxy (tailnet). If the act-runner can't reach the tailnet IP, expose 8789 as an in-cluster Service+Endpoints and set this to the cluster DNS name. | \ No newline at end of file diff --git a/pilot/ai_review.py b/pilot/ai_review.py new file mode 100644 index 0000000..90fe8fa --- /dev/null +++ b/pilot/ai_review.py @@ -0,0 +1,221 @@ +#!/usr/bin/env python3 +"""pragent pilot — minimal AI PR reviewer. + +Runs as a Gitea Actions step. Fetches a PR diff, asks glm-5.2:cloud (via the +on-network headroom proxy, Anthropic /v1/messages format) to review it, and +posts the findings back as a PR review authored by pragent-bot. + +Fail-open by design: any error becomes a short "review failed" review comment, +and the process always exits 0 so it can never block CI. + +Stdlib only — no pip install, fast cold start in CI. + +Env: + GITEA_API base URL of the in-cluster Gitea, e.g. http://gitea-http.gitea.svc.cluster.local:3000 + GITEA_REPOSITORY "owner/repo" of the PR (github.repository) + PR_INDEX PR number (github.event.pull_request.number) + PR_TITLE PR title + PR_BODY PR body (optional) + PRAGENT_BOT_TOKEN bot access token (repo secret) + PRAGENT_SHA head SHA to tag the review (github.event.pull_request.head.sha) + OLLAMA_URL headroom proxy URL, e.g. http://100.74.17.70:8789 + OLLAMA_MODEL model id, e.g. glm-5.2:cloud + OLLAMA_MAX_TOKENS (optional) output cap, default 6000 + DIFF_MAX_CHARS (optional) diff truncation cap, default 150000 +""" + +import json +import os +import sys +import urllib.error +import urllib.request + +REVIEW_HEADER = "🤖 **AI Review** · pragent pilot · {model} · `{sha}`" + +SYSTEM_PROMPT = """You are a senior, pragmatic code reviewer. Review the pull request diff below. + +Report ONLY real, actionable issues: correctness bugs, security problems, risky +changes, missing tests for changed behaviour, and breaking API/contract changes. + +For each issue, output exactly one line in this format: + - [SEVERITY] path:line — concise problem. suggested fix. +where SEVERITY is one of: critical, high, medium, low. + +Rules: +- Skip nitpicks, pure formatting, and praise. +- If the diff is clean, output exactly: No issues found. +- Be concise. At most ~15 findings, highest severity first. +- Do not restate the diff. Do not include a summary header. Just the findings lines.""" + +# --------------------------------------------------------------------------- +# Pure helpers (unit-tested, no network) +# --------------------------------------------------------------------------- + + +def truncate_diff(text: str, max_chars: int) -> tuple[str, bool, int]: + """Return (text, was_truncated, original_len). Never raises on bad input.""" + if text is None: + return "", False, 0 + orig_len = len(text) + if orig_len <= max_chars: + return text, False, orig_len + return text[:max_chars] + f"\n\n[diff truncated at {max_chars} characters]\n", True, orig_len + + +def parse_text_blocks(content: list) -> str: + """Join `type:"text"` blocks from an Anthropic /v1/messages response. + + Drops `thinking` blocks (glm-5.2:cloud is a reasoning model and emits them). + Tolerates missing/malformed blocks by skipping them. + """ + if not isinstance(content, list): + return "" + out = [] + for block in content: + if not isinstance(block, dict): + continue + if block.get("type") == "text" and isinstance(block.get("text"), str): + out.append(block["text"]) + return "\n".join(out).strip() + + +def format_review_body(findings: str, model: str, sha: str) -> str: + """Format the posted review body. Findings empty -> "No issues found.".""" + header = REVIEW_HEADER.format(model=model, sha=sha[:8] if sha else "unknown") + findings = (findings or "").strip() + if not findings: + findings = "No issues found." + return f"{header}\n\n{findings}" + + +def build_user_prompt(title: str, body: str, diff: str) -> str: + parts = [f"## PR\nTitle: {title or '(none)'}"] + if body and body.strip(): + b = body.strip() + if len(b) > 4000: + b = b[:4000] + "\n…[PR body truncated]" + parts.append(f"Description:\n{b}") + parts.append(f"## Diff\n```diff\n{diff}\n```") + return "\n\n".join(parts) + + +# --------------------------------------------------------------------------- +# Network helpers +# --------------------------------------------------------------------------- + + +def _http(method: str, url: str, token: str, body: dict | None = None, accept: str = "application/json") -> tuple[int, bytes]: + headers = {"Authorization": f"token {token}", "Accept": accept} + data = None + if body is not None: + data = json.dumps(body).encode() + headers["Content-Type"] = "application/json" + req = urllib.request.Request(url, data=data, headers=headers, method=method) + try: + with urllib.request.urlopen(req, timeout=180) as r: + return r.status, r.read() + except urllib.error.HTTPError as e: + return e.code, e.read() + except urllib.error.URLError as e: + raise RuntimeError(f"network error: {e.reason}") from e + + +def gitea_get(api: str, repo: str, path: str, token: str, accept: str = "application/json") -> tuple[int, bytes]: + return _http("GET", f"{api}/api/v1/repos/{repo}/{path}", token, None, accept) + + +def gitea_post(api: str, repo: str, path: str, token: str, body: dict) -> tuple[int, bytes]: + return _http("POST", f"{api}/api/v1/repos/{repo}/{path}", token, body) + + +def fetch_pr_diff(api: str, repo: str, index: str, token: str, max_chars: int) -> tuple[str, bool, int]: + """Get the unified diff. Try the `.diff` suffix first, fall back to the + files endpoint (join `patch` fields) if the server does not serve .diff.""" + status, raw = gitea_get(api, repo, f"pulls/{index}.diff", token, accept="text/plain") + if status == 200: + return truncate_diff(raw.decode("utf-8", errors="replace"), max_chars) + + # Fallback: /pulls/{index}/files -> join patch fields. + status, raw = gitea_get(api, repo, f"pulls/{index}/files", token) + if status != 200: + raise RuntimeError(f"could not fetch diff: .diff={status}, files={status}") + files = json.loads(raw) + joined = [] + for f in files: + h = f.get("filename", "?") + joined.append(f"--- {h}\n+++ {h}\n{f.get('patch', '(binary or no patch)')}") + return truncate_diff("\n".join(joined), max_chars) + + +def call_model(ollama_url: str, model: str, system: str, user: str, max_tokens: int) -> str: + payload = { + "model": model, + "max_tokens": max_tokens, + "system": system, + "messages": [{"role": "user", "content": user}], + } + status, raw = _http( + "POST", + f"{ollama_url.rstrip('/')}/v1/messages", + "ollama", # headroom ollama hub uses x-api-key: ollama + payload, + ) + if status != 200: + raise RuntimeError(f"model call failed: HTTP {status}: {raw[:500].decode('utf-8', errors='replace')}") + data = json.loads(raw) + return parse_text_blocks(data.get("content", [])) + + +def post_review(api: str, repo: str, index: str, token: str, body: str) -> None: + status, raw = gitea_post(api, repo, f"pulls/{index}/reviews", token, {"event": "COMMENT", "body": body}) + if status not in (200, 201): + # Fallback to a plain issue comment if reviews endpoint refuses. + status2, raw2 = gitea_post(api, repo, f"issues/{index}/comments", token, {"body": body}) + if status2 not in (200, 201): + raise RuntimeError(f"post review failed: reviews={status}, comments={status2}") + + +# --------------------------------------------------------------------------- +# Main +# --------------------------------------------------------------------------- + + +def _need(name: str) -> str: + v = os.environ.get(name) + if not v: + raise RuntimeError(f"missing env {name}") + return v + + +def run() -> int: + api = _need("GITEA_API") + repo = _need("GITEA_REPOSITORY") + index = _need("PR_INDEX") + token = _need("PRAGENT_BOT_TOKEN") + ollama_url = _need("OLLAMA_URL") + model = _need("OLLAMA_MODEL") + title = os.environ.get("PR_TITLE", "") + body = os.environ.get("PR_BODY", "") + sha = os.environ.get("PRAGENT_SHA", "") + max_tokens = int(os.environ.get("OLLAMA_MAX_TOKENS", "6000")) + max_chars = int(os.environ.get("DIFF_MAX_CHARS", "150000")) + + try: + diff, truncated, _orig = fetch_pr_diff(api, repo, index, token, max_chars) + if not diff.strip(): + post_review(api, repo, index, token, format_review_body("No diff content to review.", model, sha)) + return 0 + user_prompt = build_user_prompt(title, body, diff) + findings = call_model(ollama_url, model, SYSTEM_PROMPT, user_prompt, max_tokens) + post_review(api, repo, index, token, format_review_body(findings, model, sha)) + except Exception as e: # fail-open + try: + post_review(api, repo, index, token, format_review_body(f"⚠️ AI review failed: {e}", model, sha)) + except Exception as e2: + print(f"pragent: could not post failure note: {e2}", file=sys.stderr) + print(f"pragent: review failed: {e}", file=sys.stderr) + return 0 + + +if __name__ == "__main__": + sys.exit(run()) \ No newline at end of file diff --git a/pilot/workflow-template.yml b/pilot/workflow-template.yml new file mode 100644 index 0000000..57b5183 --- /dev/null +++ b/pilot/workflow-template.yml @@ -0,0 +1,45 @@ +# pragent pilot — AI Review workflow template. +# +# Copy this file into the repo you want reviewed as: +# .gitea/workflows/ai-review.yml +# +# Prerequisites (see pilot/README.md): +# 1. pragent-bot added as a collaborator with Write access. +# 2. repo secret PRAGENT_BOT_TOKEN set to the bot's access token. +# +# Reviews fire ONLY on PRs carrying the `AI-REVIEW` label. Remove the label to +# stop a re-review on subsequent pushes. The job is fail-open (never blocks CI). + +name: AI Review + +on: + pull_request: + types: [opened, synchronize, reopened, labeled] + +jobs: + review: + # Only run when the PR has the AI-REVIEW label. Acts as a cheap gate: no + # model call, no cost, when the label is absent. + if: contains(github.event.pull_request.labels.*.name, 'AI-REVIEW') + runs-on: ubuntu-latest + steps: + - name: Run pragent pilot review + env: + GITEA_API: http://gitea-http.gitea.svc.cluster.local:3000 + GITEA_REPOSITORY: ${{ github.repository }} + PR_INDEX: ${{ github.event.pull_request.number }} + PR_TITLE: ${{ github.event.pull_request.title }} + PR_BODY: ${{ github.event.pull_request.body }} + PRAGENT_BOT_TOKEN: ${{ secrets.PRAGENT_BOT_TOKEN }} + PRAGENT_SHA: ${{ github.event.pull_request.head.sha }} + # On-network model: headroom proxy on kubernets (tailnet IP). + OLLAMA_URL: http://100.74.17.70:8789 + OLLAMA_MODEL: glm-5.2:cloud + OLLAMA_MAX_TOKENS: "6000" + DIFF_MAX_CHARS: "150000" + run: | + set -e + # Fetch the reviewer script from the pragent repo (private → bot token). + curl -fsS -H "Authorization: token $PRAGENT_BOT_TOKEN" \ + "$GITEA_API/api/v1/repos/gitea_admin/pragent/raw/branch/main/pilot/ai_review.py" -o ai_review.py + python3 ai_review.py \ No newline at end of file diff --git a/tests/pilot/test_ai_review.py b/tests/pilot/test_ai_review.py new file mode 100644 index 0000000..a633824 --- /dev/null +++ b/tests/pilot/test_ai_review.py @@ -0,0 +1,133 @@ +"""Unit tests for pragent pilot pure helpers. No network.""" +import os +import sys + +# Allow running without install: add repo root to path. +HERE = os.path.dirname(os.path.abspath(__file__)) +ROOT = os.path.abspath(os.path.join(HERE, "..", "..")) +sys.path.insert(0, os.path.join(ROOT, "pilot")) + +from ai_review import ( # noqa: E402 + build_user_prompt, + format_review_body, + parse_text_blocks, + truncate_diff, +) + +# --------------------------------------------------------------------------- +# truncate_diff +# --------------------------------------------------------------------------- + + +def test_truncate_diff_short(): + text, truncated, n = truncate_diff("abc", 100) + assert text == "abc" + assert truncated is False + assert n == 3 + + +def test_truncate_diff_exact_boundary(): + text, truncated, n = truncate_diff("x" * 100, 100) + assert truncated is False + assert n == 100 + assert text == "x" * 100 + + +def test_truncate_diff_over_cap(): + text, truncated, n = truncate_diff("x" * 250, 100) + assert truncated is True + assert n == 250 + assert text.startswith("x" * 100) + assert "[diff truncated at 100 characters]" in text + + +def test_truncate_diff_none(): + text, truncated, n = truncate_diff(None, 100) # type: ignore[arg-type] + assert text == "" + assert truncated is False + assert n == 0 + + +# --------------------------------------------------------------------------- +# parse_text_blocks +# --------------------------------------------------------------------------- + + +def test_parse_text_blocks_text_only(): + content = [{"type": "text", "text": "hello"}, {"type": "text", "text": "world"}] + assert parse_text_blocks(content) == "hello\nworld" + + +def test_parse_text_blocks_drops_thinking(): + content = [ + {"type": "thinking", "thinking": "reasoning here"}, + {"type": "text", "text": "- [high] a.go:3 — bug. fix."}, + ] + assert parse_text_blocks(content) == "- [high] a.go:3 — bug. fix." + + +def test_parse_text_blocks_empty_and_malformed(): + assert parse_text_blocks([]) == "" + assert parse_text_blocks(None) == "" # type: ignore[arg-type] + assert parse_text_blocks([{"type": "text"}, "garbage", 5]) == "" + + +def test_parse_text_blocks_real_glm_shape(): + # Captured from glm-5.2:cloud via headroom 8789. + content = [ + {"type": "thinking", "thinking": "Analyze the request..."}, + {"type": "text", "text": "- [critical] auth.py:12 — token compared with `==`. Use hmac.compare_digest."}, + ] + assert "compare_digest" in parse_text_blocks(content) + + +# --------------------------------------------------------------------------- +# format_review_body +# --------------------------------------------------------------------------- + + +def test_format_review_body_findings(): + body = format_review_body("- [high] x:1 — bug. fix.", "glm-5.2:cloud", "abcdef1234567890") + assert "pragent pilot" in body + assert "glm-5.2:cloud" in body + assert "`abcdef12`" in body # 8-char sha + assert "- [high] x:1" in body + + +def test_format_review_body_empty_findings(): + body = format_review_body("", "glm-5.2:cloud", "abcdef1234567890") + assert "No issues found." in body + + +def test_format_review_body_whitespace_findings(): + body = format_review_body(" \n ", "glm-5.2:cloud", "abcdef1234567890") + assert "No issues found." in body + + +def test_format_review_body_no_sha(): + body = format_review_body("- [low] y:2 — nit", "glm-5.2:cloud", "") + assert "`unknown`" in body + + +# --------------------------------------------------------------------------- +# build_user_prompt +# --------------------------------------------------------------------------- + + +def test_build_user_prompt_includes_title_and_diff(): + p = build_user_prompt("Fix login", "Closes #1", "diff --git a/x b/x") + assert "Fix login" in p + assert "Closes #1" in p + assert "diff --git a/x b/x" in p + + +def test_build_user_prompt_truncates_long_body(): + long_body = "B" * 6000 + p = build_user_prompt("t", long_body, "d") + assert "[PR body truncated]" in p + assert p.count("B") < 6000 + + +def test_build_user_prompt_no_body(): + p = build_user_prompt("t", "", "d") + assert "Description:" not in p \ No newline at end of file