diff --git a/docs/plans/2026-08-04-pragent-implementation.md b/docs/plans/2026-08-04-pragent-implementation.md new file mode 100644 index 0000000..5e2c2d1 --- /dev/null +++ b/docs/plans/2026-08-04-pragent-implementation.md @@ -0,0 +1,1733 @@ +# pragent Phase 1 (Walking Skeleton) Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** `pragent review` runs against a local git range, classifies the change into a tier, runs one analyzer through a real model, prints ranked findings, writes a JSONL run record, and exits with a meaningful code. + +**Architecture:** A CLI composed of small pure modules behind ports. `ForgeAdapter` supplies the diff and receives findings; `TierEngine` is pure functions over a parsed diff; `ModelClient` is a port with a fake for tests and a `pi`-SDK adapter for real runs; `Emitter` writes JSONL. Everything except the two adapters is tested without network access. + +**Tech Stack:** TypeScript 5 (strict, ESM, `NodeNext`), Node 22+, Vitest, Zod for config/output validation, `@earendil-works/pi-coding-agent` + `@earendil-works/pi-ai` for the agent loop, `commander` for the CLI. + +**Repo:** `~/Projects/pragent` (Gitea: `gitea_admin/pragent`). Design doc: `docs/plans/2026-08-04-pragent-design.md`. + +--- + +## Ground rules for the implementer + +- **TDD, strictly.** Write the failing test, watch it fail, write the minimum code, watch it pass, commit. A step that says "run it and see it fail" is not decoration — a test that passes before the implementation exists is a broken test. +- **No network in unit tests.** Only `tests/integration/` may call a model, and it is skipped unless `ANTHROPIC_API_KEY` is set. +- **ESM everywhere.** `"type": "module"` in `package.json`; relative imports carry the `.js` extension even in `.ts` sources (`NodeNext` resolution). This trips people up — it is not a typo. +- **Commit after every task.** Message style: Conventional Commits, imperative subject under 50 chars. +- **Do not build Phase 2+ features.** No Gitea adapter, no `pragent init`, no prompt caching, no fan-out. They are listed at the end as milestones so you know where this is heading, not as work to do now. + +--- + +## Task 1: Project scaffold + +**Files:** +- Create: `package.json`, `tsconfig.json`, `vitest.config.ts`, `src/index.ts`, `tests/smoke.test.ts` + +**Step 1: Initialize the package** + +```bash +cd ~/Projects/pragent +npm init -y +npm pkg set type=module name=pragent version=0.1.0 \ + description="Extensible, forge-agnostic PR review framework" +npm pkg set bin.pragent=dist/cli.js +npm pkg set scripts.build="tsc -p tsconfig.json" +npm pkg set scripts.test="vitest run" +npm pkg set scripts.typecheck="tsc --noEmit" +npm install -D typescript vitest @types/node +npm install commander zod +``` + +**Step 2: Write `tsconfig.json`** + +```json +{ + "compilerOptions": { + "target": "ES2022", + "module": "NodeNext", + "moduleResolution": "NodeNext", + "strict": true, + "noUncheckedIndexedAccess": true, + "declaration": true, + "outDir": "dist", + "rootDir": "src", + "skipLibCheck": true + }, + "include": ["src/**/*.ts"] +} +``` + +**Step 3: Write `vitest.config.ts`** + +```typescript +import { defineConfig } from "vitest/config"; + +export default defineConfig({ + test: { + include: ["tests/**/*.test.ts"], + testTimeout: 20_000, + }, +}); +``` + +**Step 4: Write the smoke test** + +`tests/smoke.test.ts`: + +```typescript +import { expect, test } from "vitest"; +import { VERSION } from "../src/index.js"; + +test("exports a version string", () => { + expect(VERSION).toMatch(/^\d+\.\d+\.\d+$/); +}); +``` + +**Step 5: Run it and watch it fail** + +Run: `npm test` +Expected: FAIL — `Failed to resolve import "../src/index.js"`. + +**Step 6: Write the minimum implementation** + +`src/index.ts`: + +```typescript +export const VERSION = "0.1.0"; +``` + +**Step 7: Run it and watch it pass** + +Run: `npm test` → 1 passed. Then `npm run typecheck` → no errors. + +**Step 8: Commit** + +```bash +git add -A +git commit -m "chore: scaffold TypeScript project with vitest" +``` + +--- + +## Task 2: Diff parsing + +Everything downstream reads a parsed diff, so this is the foundation. Parse unified diff text into structured changed files. + +**Files:** +- Create: `src/diff/types.ts`, `src/diff/parse.ts`, `tests/diff/parse.test.ts` + +**Step 1: Write the failing test** + +`tests/diff/parse.test.ts`: + +```typescript +import { expect, test } from "vitest"; +import { parseUnifiedDiff } from "../../src/diff/parse.js"; + +const SAMPLE = `diff --git a/src/auth/login.ts b/src/auth/login.ts +index 1111111..2222222 100644 +--- a/src/auth/login.ts ++++ b/src/auth/login.ts +@@ -10,3 +10,4 @@ export function login(u: string) { + const t = mint(u); +- return t; ++ log(t); ++ return t; + } +diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml +index 3333333..4444444 100644 +--- a/pnpm-lock.yaml ++++ b/pnpm-lock.yaml +@@ -1,2 +1,2 @@ +-old ++new +`; + +test("parses paths, status, and line counts", () => { + const files = parseUnifiedDiff(SAMPLE); + expect(files).toHaveLength(2); + expect(files[0]!.path).toBe("src/auth/login.ts"); + expect(files[0]!.status).toBe("modified"); + expect(files[0]!.added).toBe(2); + expect(files[0]!.removed).toBe(1); + expect(files[1]!.path).toBe("pnpm-lock.yaml"); +}); + +test("totals are the sum across files", () => { + const files = parseUnifiedDiff(SAMPLE); + const total = files.reduce((n, f) => n + f.added + f.removed, 0); + expect(total).toBe(5); +}); + +test("returns an empty array for an empty diff", () => { + expect(parseUnifiedDiff("")).toEqual([]); +}); + +test("detects added and deleted files", () => { + const d = `diff --git a/new.ts b/new.ts +new file mode 100644 +--- /dev/null ++++ b/new.ts +@@ -0,0 +1,1 @@ ++hello +diff --git a/gone.ts b/gone.ts +deleted file mode 100644 +--- a/gone.ts ++++ /dev/null +@@ -1,1 +0,0 @@ +-bye +`; + const files = parseUnifiedDiff(d); + expect(files[0]!.status).toBe("added"); + expect(files[1]!.status).toBe("deleted"); +}); +``` + +**Step 2: Run it and watch it fail** + +Run: `npx vitest run tests/diff/parse.test.ts` +Expected: FAIL — cannot resolve `parse.js`. + +**Step 3: Write the types** + +`src/diff/types.ts`: + +```typescript +export type FileStatus = "added" | "modified" | "deleted" | "renamed"; + +export interface ChangedFile { + path: string; + status: FileStatus; + added: number; + removed: number; + /** The raw hunk text for this file, passed verbatim to analyzers. */ + patch: string; +} + +export interface ParsedDiff { + files: ChangedFile[]; + totalAdded: number; + totalRemoved: number; +} +``` + +**Step 4: Write the parser** + +`src/diff/parse.ts`: + +```typescript +import type { ChangedFile, FileStatus } from "./types.js"; + +const FILE_HEADER = /^diff --git a\/(.+?) b\/(.+)$/; + +export function parseUnifiedDiff(text: string): ChangedFile[] { + if (!text.trim()) return []; + + const files: ChangedFile[] = []; + let current: ChangedFile | null = null; + + for (const line of text.split("\n")) { + const header = FILE_HEADER.exec(line); + if (header) { + if (current) files.push(current); + current = { + path: header[2]!, + status: "modified", + added: 0, + removed: 0, + patch: "", + }; + continue; + } + if (!current) continue; + + current.patch += line + "\n"; + + if (line.startsWith("new file mode")) current.status = "added"; + else if (line.startsWith("deleted file mode")) current.status = "deleted"; + else if (line.startsWith("rename to ")) current.status = "renamed"; + else if (line.startsWith("+++") || line.startsWith("---")) continue; + else if (line.startsWith("+")) current.added++; + else if (line.startsWith("-")) current.removed++; + } + + if (current) files.push(current); + return files; +} +``` + +**Step 5: Run it and watch it pass** + +Run: `npx vitest run tests/diff/parse.test.ts` → 4 passed. + +**Step 6: Commit** + +```bash +git add src/diff tests/diff +git commit -m "feat: parse unified diffs into changed files" +``` + +--- + +## Task 3: Forge adapter port and local git adapter + +**Files:** +- Create: `src/forge/types.ts`, `src/forge/local.ts`, `tests/forge/local.test.ts` + +The adapter is the seam that makes Gitea and GitLab cheap later. Phase 1 ships only the local one. + +**Step 1: Write the failing test** + +`tests/forge/local.test.ts`: + +```typescript +import { execFileSync } from "node:child_process"; +import { mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { expect, test } from "vitest"; +import { LocalForgeAdapter } from "../../src/forge/local.js"; + +function fixtureRepo(): string { + const dir = mkdtempSync(join(tmpdir(), "pragent-")); + const git = (...args: string[]) => + execFileSync("git", args, { cwd: dir, stdio: "pipe" }); + + git("init", "-q", "-b", "main"); + git("config", "user.email", "t@example.com"); + git("config", "user.name", "Test"); + writeFileSync(join(dir, "app.ts"), "export const a = 1;\n"); + git("add", "-A"); + git("commit", "-q", "-m", "base"); + writeFileSync(join(dir, "app.ts"), "export const a = 1;\nexport const b = 2;\n"); + git("add", "-A"); + git("commit", "-q", "-m", "change"); + return dir; +} + +test("reads a diff for a git range", async () => { + const dir = fixtureRepo(); + const adapter = new LocalForgeAdapter({ cwd: dir, range: "HEAD~1..HEAD" }); + const ctx = await adapter.getContext(); + + expect(ctx.files).toHaveLength(1); + expect(ctx.files[0]!.path).toBe("app.ts"); + expect(ctx.files[0]!.added).toBe(1); + expect(ctx.headSha).toMatch(/^[0-9a-f]{40}$/); +}); + +test("rejects an invalid range with a clear message", async () => { + const dir = fixtureRepo(); + const adapter = new LocalForgeAdapter({ cwd: dir, range: "nope..alsonope" }); + await expect(adapter.getContext()).rejects.toThrow(/git diff failed/i); +}); +``` + +**Step 2: Run it and watch it fail** + +Run: `npx vitest run tests/forge/local.test.ts` → FAIL, module not found. + +**Step 3: Write the port** + +`src/forge/types.ts`: + +```typescript +import type { ChangedFile } from "../diff/types.js"; +import type { Finding } from "../analyze/types.js"; + +export interface ReviewContext { + repo: string; + /** PR/MR number where the forge has one; null for local runs. */ + pullRequest: number | null; + headSha: string; + baseSha: string; + title: string; + files: ChangedFile[]; +} + +export interface ForgeAdapter { + readonly name: string; + getContext(): Promise; + publish(ctx: ReviewContext, findings: Finding[]): Promise; +} +``` + +**Step 4: Write the local adapter** + +`src/forge/local.ts`: + +```typescript +import { execFile } from "node:child_process"; +import { basename } from "node:path"; +import { promisify } from "node:util"; +import { parseUnifiedDiff } from "../diff/parse.js"; +import type { Finding } from "../analyze/types.js"; +import type { ForgeAdapter, ReviewContext } from "./types.js"; + +const run = promisify(execFile); + +export interface LocalForgeOptions { + cwd: string; + /** A git range such as "main..HEAD" or "HEAD~1..HEAD". */ + range: string; +} + +export class LocalForgeAdapter implements ForgeAdapter { + readonly name = "local"; + + constructor(private readonly opts: LocalForgeOptions) {} + + async getContext(): Promise { + const { cwd, range } = this.opts; + let diff: string; + try { + const res = await run("git", ["diff", "--no-color", range], { + cwd, + maxBuffer: 64 * 1024 * 1024, + }); + diff = res.stdout; + } catch (err) { + throw new Error( + `git diff failed for range "${range}" in ${cwd}: ${(err as Error).message}`, + ); + } + + const rev = async (ref: string) => + (await run("git", ["rev-parse", ref], { cwd })).stdout.trim(); + + const [base, head] = range.split(".."); + return { + repo: basename(cwd), + pullRequest: null, + baseSha: await rev(base || "HEAD~1"), + headSha: await rev(head || "HEAD"), + title: `local diff ${range}`, + files: parseUnifiedDiff(diff), + }; + } + + async publish(_ctx: ReviewContext, findings: Finding[]): Promise { + for (const f of findings) { + console.log( + `${f.file}:${f.line} [${f.severity}/${f.confidence}] ${f.claim}` + + (f.fix ? `\n fix: ${f.fix}` : ""), + ); + } + } +} +``` + +**Step 5: Run it and watch it pass** + +Run: `npx vitest run tests/forge/local.test.ts` → 2 passed. +(`src/analyze/types.ts` does not exist yet — create it as a stub now with `export interface Finding { file: string; line: number; severity: string; confidence: number; claim: string; fix?: string }`. Task 6 replaces it with the real schema.) + +**Step 6: Commit** + +```bash +git add src/forge src/analyze tests/forge +git commit -m "feat: add forge adapter port and local git adapter" +``` + +--- + +## Task 4: Tier engine (rules only) + +The heart of the cost model. Pure functions, no I/O, no model calls — and it must record **why** it chose a tier. + +**Files:** +- Create: `src/tier/rules.ts`, `src/tier/types.ts`, `tests/tier/rules.test.ts` + +**Step 1: Write the failing test** + +`tests/tier/rules.test.ts`: + +```typescript +import { expect, test } from "vitest"; +import { classify, DEFAULT_TIER_POLICY } from "../../src/tier/rules.js"; +import type { ChangedFile } from "../../src/diff/types.js"; + +const file = (path: string, added = 10, removed = 0): ChangedFile => ({ + path, + status: "modified", + added, + removed, + patch: "", +}); + +const P = DEFAULT_TIER_POLICY; + +test("lockfile-only changes are trivial", () => { + const r = classify([file("pnpm-lock.yaml", 400, 300)], P); + expect(r.tier).toBe("trivial"); + expect(r.reason).toMatch(/generated/); +}); + +test("a small ordinary change is lite", () => { + const r = classify([file("src/util/format.ts", 20, 5)], P); + expect(r.tier).toBe("lite"); +}); + +test("a risk path forces full even when tiny", () => { + const r = classify([file("src/auth/session.ts", 3, 1)], P); + expect(r.tier).toBe("full"); + expect(r.reason).toMatch(/risk_path/); +}); + +test("a large change is full", () => { + const r = classify([file("src/big.ts", 900, 100)], P); + expect(r.tier).toBe("full"); + expect(r.reason).toMatch(/size/); +}); + +test("a very large change is oversized", () => { + const files = Array.from({ length: 60 }, (_, i) => file(`src/f${i}.ts`, 60, 10)); + const r = classify(files, P); + expect(r.tier).toBe("oversized"); +}); + +test("risk paths win over the generated-file rule", () => { + const r = classify( + [file("pnpm-lock.yaml", 10), file("infra/main.tf", 4)], + P, + ); + expect(r.tier).toBe("full"); + expect(r.reason).toMatch(/risk_path/); +}); + +test("an empty diff is trivial", () => { + expect(classify([], P).tier).toBe("trivial"); +}); +``` + +**Step 2: Run it and watch it fail** + +Run: `npx vitest run tests/tier/rules.test.ts` → FAIL. + +**Step 3: Write the types** + +`src/tier/types.ts`: + +```typescript +export type Tier = "trivial" | "lite" | "full" | "oversized"; + +export interface TierDecision { + tier: Tier; + /** Machine-readable provenance, e.g. "rule:risk_path(**\/auth\/**)". */ + reason: string; +} + +export interface TierPolicy { + riskPaths: string[]; + generatedPaths: string[]; + liteMaxLines: number; + fullMaxLines: number; + fullMaxFiles: number; +} +``` + +**Step 4: Write the rules** + +`src/tier/rules.ts`: + +```typescript +import type { ChangedFile } from "../diff/types.js"; +import type { TierDecision, TierPolicy } from "./types.js"; + +export const DEFAULT_TIER_POLICY: TierPolicy = { + riskPaths: [ + "**/auth/**", + "**/migrations/**", + "infra/**", + "**/*.tf", + "**/security/**", + ], + generatedPaths: [ + "**/*-lock.yaml", + "**/*.lock", + "package-lock.json", + "**/*.pb.go", + "**/generated/**", + ], + liteMaxLines: 150, + fullMaxLines: 2000, + fullMaxFiles: 50, +}; + +/** Minimal glob matcher: supports `**` and `*`. No brace or char classes. */ +export function matchGlob(pattern: string, path: string): boolean { + const rx = pattern + .split("**") + .map((seg) => seg.split("*").map(escapeRegex).join("[^/]*")) + .join(".*"); + return new RegExp(`^${rx}$`).test(path); +} + +function escapeRegex(s: string): string { + return s.replace(/[.+?^${}()|[\]\\]/g, "\\$&"); +} + +export function classify( + files: ChangedFile[], + policy: TierPolicy, +): TierDecision { + if (files.length === 0) { + return { tier: "trivial", reason: "rule:empty_diff" }; + } + + // Risk paths outrank every other rule — a three-line auth change is never trivial. + for (const f of files) { + for (const p of policy.riskPaths) { + if (matchGlob(p, f.path)) { + return { tier: "full", reason: `rule:risk_path(${p})` }; + } + } + } + + const lines = files.reduce((n, f) => n + f.added + f.removed, 0); + + if (lines > policy.fullMaxLines || files.length > policy.fullMaxFiles) { + return { + tier: "oversized", + reason: `rule:size(lines=${lines},files=${files.length})`, + }; + } + + const allGenerated = files.every((f) => + policy.generatedPaths.some((p) => matchGlob(p, f.path)), + ); + if (allGenerated) { + return { tier: "trivial", reason: "rule:generated_only" }; + } + + if (lines <= policy.liteMaxLines) { + return { tier: "lite", reason: `rule:size(lines=${lines})` }; + } + + return { tier: "full", reason: `rule:size(lines=${lines})` }; +} +``` + +**Step 5: Run it and watch it pass** + +Run: `npx vitest run tests/tier/rules.test.ts` → 7 passed. + +**Step 6: Commit** + +```bash +git add src/tier tests/tier +git commit -m "feat: add rules-based tier classification" +``` + +--- + +## Task 5: Config loading + +**Files:** +- Create: `src/config/schema.ts`, `src/config/load.ts`, `tests/config/load.test.ts` + +**Step 1: Write the failing test** + +`tests/config/load.test.ts`: + +```typescript +import { mkdtempSync, mkdirSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { expect, test } from "vitest"; +import { loadConfig } from "../../src/config/load.js"; + +function repoWith(config: string | null): string { + const dir = mkdtempSync(join(tmpdir(), "pragent-cfg-")); + if (config !== null) { + mkdirSync(join(dir, ".pragent")); + writeFileSync(join(dir, ".pragent", "config.json"), config); + } + return dir; +} + +test("returns defaults when no config file exists", async () => { + const cfg = await loadConfig(repoWith(null)); + expect(cfg.tierPolicy.liteMaxLines).toBe(150); + expect(cfg.maxCostUsd).toBeGreaterThan(0); +}); + +test("repo config overrides defaults", async () => { + const dir = repoWith(JSON.stringify({ tierPolicy: { liteMaxLines: 40 } })); + const cfg = await loadConfig(dir); + expect(cfg.tierPolicy.liteMaxLines).toBe(40); + expect(cfg.tierPolicy.fullMaxLines).toBe(2000); // untouched default survives +}); + +test("rejects an invalid config with a useful message", async () => { + const dir = repoWith(JSON.stringify({ tierPolicy: { liteMaxLines: "big" } })); + await expect(loadConfig(dir)).rejects.toThrow(/liteMaxLines/); +}); +``` + +**Step 2: Run it and watch it fail** + +Run: `npx vitest run tests/config/load.test.ts` → FAIL. + +**Step 3: Write the schema** + +`src/config/schema.ts`: + +```typescript +import { z } from "zod"; +import { DEFAULT_TIER_POLICY } from "../tier/rules.js"; + +export const TierPolicySchema = z.object({ + riskPaths: z.array(z.string()).default(DEFAULT_TIER_POLICY.riskPaths), + generatedPaths: z.array(z.string()).default(DEFAULT_TIER_POLICY.generatedPaths), + liteMaxLines: z.number().int().positive().default(DEFAULT_TIER_POLICY.liteMaxLines), + fullMaxLines: z.number().int().positive().default(DEFAULT_TIER_POLICY.fullMaxLines), + fullMaxFiles: z.number().int().positive().default(DEFAULT_TIER_POLICY.fullMaxFiles), +}); + +export const ConfigSchema = z.object({ + tierPolicy: TierPolicySchema.default({}), + /** Fail open above this spend; a partial review beats a blocked pipeline. */ + maxCostUsd: z.number().positive().default(5), + minConfidence: z.number().min(0).max(1).default(0.5), + maxComments: z.number().int().positive().default(20), + models: z + .object({ + lite: z.string().default("claude-sonnet-5"), + full: z.string().default("claude-opus-5"), + }) + .default({}), + emitPath: z.string().default(".pragent/runs.jsonl"), +}); + +export type Config = z.infer; +``` + +**Step 4: Write the loader** + +`src/config/load.ts`: + +```typescript +import { readFile } from "node:fs/promises"; +import { join } from "node:path"; +import { ConfigSchema, type Config } from "./schema.js"; + +export async function loadConfig(repoRoot: string): Promise { + let raw: unknown = {}; + try { + const text = await readFile(join(repoRoot, ".pragent", "config.json"), "utf8"); + raw = JSON.parse(text); + } catch (err) { + if ((err as NodeJS.ErrnoException).code !== "ENOENT") throw err; + } + + const parsed = ConfigSchema.safeParse(raw); + if (!parsed.success) { + const issues = parsed.error.issues + .map((i) => `${i.path.join(".")}: ${i.message}`) + .join("; "); + throw new Error(`invalid .pragent/config.json — ${issues}`); + } + return parsed.data; +} +``` + +**Step 5: Run it and watch it pass** + +Run: `npx vitest run tests/config/load.test.ts` → 3 passed. + +**Step 6: Commit** + +```bash +git add src/config tests/config +git commit -m "feat: load and validate repo config with defaults" +``` + +--- + +## Task 6: Analyzer runner against a fake model + +The model is a **port**. This task builds the analyzer logic and tests it with a fake — no network, deterministic, fast. Task 7 plugs in the real SDK. + +**Files:** +- Create: `src/analyze/types.ts` (replacing the Task 3 stub), `src/analyze/model.ts`, `src/analyze/runner.ts`, `src/analyze/prompts/code-quality.ts`, `tests/analyze/runner.test.ts` + +**Step 1: Write the failing test** + +`tests/analyze/runner.test.ts`: + +```typescript +import { expect, test } from "vitest"; +import { runAnalyzer } from "../../src/analyze/runner.js"; +import type { ModelClient } from "../../src/analyze/model.js"; +import type { ReviewContext } from "../../src/forge/types.js"; + +const ctx: ReviewContext = { + repo: "demo", + pullRequest: null, + headSha: "a".repeat(40), + baseSha: "b".repeat(40), + title: "test", + files: [ + { path: "src/a.ts", status: "modified", added: 3, removed: 1, patch: "+ bad()" }, + ], +}; + +const fakeModel = (reply: string): ModelClient => ({ + name: "fake", + async complete() { + return { text: reply, usage: { inputTokens: 100, outputTokens: 20, costUsd: 0.01 } }; + }, +}); + +test("parses well-formed findings from the model", async () => { + const model = fakeModel( + JSON.stringify({ + findings: [ + { + file: "src/a.ts", + line: 12, + severity: "high", + confidence: 0.9, + category: "correctness", + claim: "Null deref when input is empty", + fix: "Guard the empty case", + }, + ], + }), + ); + + const res = await runAnalyzer({ id: "code-quality", model, ctx }); + expect(res.findings).toHaveLength(1); + expect(res.findings[0]!.analyzer).toBe("code-quality"); + expect(res.usage.costUsd).toBeCloseTo(0.01); +}); + +test("tolerates prose wrapped around the JSON", async () => { + const model = fakeModel( + 'Here is what I found:\n```json\n{"findings":[]}\n```\nThat is all.', + ); + const res = await runAnalyzer({ id: "code-quality", model, ctx }); + expect(res.findings).toEqual([]); +}); + +test("fails open on unparseable output instead of throwing", async () => { + const model = fakeModel("I could not complete the review."); + const res = await runAnalyzer({ id: "code-quality", model, ctx }); + expect(res.findings).toEqual([]); + expect(res.error).toMatch(/parse/i); +}); + +test("drops findings that reference a file outside the diff", async () => { + const model = fakeModel( + JSON.stringify({ + findings: [ + { file: "src/elsewhere.ts", line: 1, severity: "high", confidence: 0.9, + category: "correctness", claim: "hallucinated" }, + { file: "src/a.ts", line: 2, severity: "low", confidence: 0.8, + category: "style", claim: "real" }, + ], + }), + ); + const res = await runAnalyzer({ id: "code-quality", model, ctx }); + expect(res.findings).toHaveLength(1); + expect(res.findings[0]!.claim).toBe("real"); +}); +``` + +That last test matters more than it looks: a reviewer that invents files loses user trust faster than one that misses bugs. + +**Step 2: Run it and watch it fail** + +Run: `npx vitest run tests/analyze/runner.test.ts` → FAIL. + +**Step 3: Write the types** + +`src/analyze/types.ts` (replaces the stub from Task 3): + +```typescript +import { z } from "zod"; + +export const SeveritySchema = z.enum(["critical", "high", "medium", "low"]); + +export const FindingSchema = z.object({ + file: z.string(), + line: z.number().int().nonnegative(), + severity: SeveritySchema, + confidence: z.number().min(0).max(1), + category: z.string(), + claim: z.string().min(1), + fix: z.string().optional(), +}); + +export type Severity = z.infer; +export type RawFinding = z.infer; + +/** A finding after the runner stamps its provenance. */ +export interface Finding extends RawFinding { + analyzer: string; +} + +export const AnalyzerOutputSchema = z.object({ + findings: z.array(FindingSchema), +}); +``` + +**Step 4: Write the model port** + +`src/analyze/model.ts`: + +```typescript +export interface Usage { + inputTokens: number; + outputTokens: number; + costUsd: number; +} + +export interface CompletionResult { + text: string; + usage: Usage; +} + +/** The seam between analyzers and any agent SDK. Keep it this small. */ +export interface ModelClient { + readonly name: string; + complete(prompt: string): Promise; +} +``` + +**Step 5: Write the prompt** + +`src/analyze/prompts/code-quality.ts`: + +```typescript +import type { ReviewContext } from "../../forge/types.js"; + +export function buildCodeQualityPrompt(ctx: ReviewContext): string { + const diff = ctx.files.map((f) => `--- ${f.path} ---\n${f.patch}`).join("\n"); + + return `You are reviewing a code change for correctness and maintainability. + +Report only defects you can point to in the diff below: logic errors, unhandled +cases, resource leaks, race conditions, and changes that contradict the +surrounding code's conventions. Do not report formatting preferences, and do not +report a problem you cannot tie to a specific line. + +Only reference files that appear in the diff. If you find nothing, return an +empty list — that is a valid and useful answer. + +Respond with JSON only, in this shape: +{"findings":[{"file":"path","line":12,"severity":"high","confidence":0.8, +"category":"correctness","claim":"one sentence","fix":"one sentence"}]} + +Repository: ${ctx.repo} +Change: ${ctx.title} + +${diff}`; +} +``` + +**Step 6: Write the runner** + +`src/analyze/runner.ts`: + +```typescript +import type { ReviewContext } from "../forge/types.js"; +import type { ModelClient, Usage } from "./model.js"; +import { buildCodeQualityPrompt } from "./prompts/code-quality.js"; +import { AnalyzerOutputSchema, type Finding } from "./types.js"; + +export interface AnalyzerRun { + id: string; + model: ModelClient; + ctx: ReviewContext; +} + +export interface AnalyzerResult { + analyzer: string; + findings: Finding[]; + usage: Usage; + error?: string; +} + +/** Pull the first balanced JSON object out of a model reply. */ +export function extractJson(text: string): string | null { + const start = text.indexOf("{"); + if (start === -1) return null; + let depth = 0; + for (let i = start; i < text.length; i++) { + if (text[i] === "{") depth++; + else if (text[i] === "}" && --depth === 0) return text.slice(start, i + 1); + } + return null; +} + +export async function runAnalyzer(run: AnalyzerRun): Promise { + const prompt = buildCodeQualityPrompt(run.ctx); + const { text, usage } = await run.model.complete(prompt); + + const json = extractJson(text); + if (!json) { + return { + analyzer: run.id, + findings: [], + usage, + error: "could not parse JSON from model output", + }; + } + + let parsed; + try { + parsed = AnalyzerOutputSchema.safeParse(JSON.parse(json)); + } catch { + return { analyzer: run.id, findings: [], usage, error: "invalid JSON syntax" }; + } + if (!parsed.success) { + return { analyzer: run.id, findings: [], usage, error: "schema parse failed" }; + } + + const known = new Set(run.ctx.files.map((f) => f.path)); + const findings = parsed.data.findings + .filter((f) => known.has(f.file)) + .map((f) => ({ ...f, analyzer: run.id })); + + return { analyzer: run.id, findings, usage }; +} +``` + +**Step 7: Run it and watch it pass** + +Run: `npx vitest run tests/analyze/runner.test.ts` → 4 passed. Then `npm test` — the Task 3 forge test must still pass against the real `Finding` type. + +**Step 8: Commit** + +```bash +git add src/analyze tests/analyze +git commit -m "feat: add analyzer runner with schema-validated findings" +``` + +--- + +## Task 7: `pi`-backed model client + +The only module that touches the SDK. Its unit test asserts wiring, not model behaviour; the real call lives in a gated integration test. + +**Files:** +- Create: `src/analyze/pi-model.ts`, `tests/integration/pi-model.test.ts` + +**Step 1: Install the SDK** + +```bash +npm install @earendil-works/pi-coding-agent @earendil-works/pi-ai +``` + +**Step 2: Write the gated integration test** + +`tests/integration/pi-model.test.ts`: + +```typescript +import { describe, expect, test } from "vitest"; +import { PiModelClient } from "../../src/analyze/pi-model.js"; + +const live = process.env.ANTHROPIC_API_KEY ? describe : describe.skip; + +live("PiModelClient (live)", () => { + test("returns text and non-zero usage", async () => { + const client = new PiModelClient({ model: "claude-sonnet-5" }); + const res = await client.complete( + 'Reply with exactly this JSON and nothing else: {"findings":[]}', + ); + expect(res.text).toContain("findings"); + expect(res.usage.inputTokens).toBeGreaterThan(0); + }, 60_000); +}); +``` + +**Step 3: Run it and watch it skip** + +Run: `npx vitest run tests/integration/pi-model.test.ts` +Expected: skipped when `ANTHROPIC_API_KEY` is unset. That is the correct outcome for CI. + +**Step 4: Write the client** + +`src/analyze/pi-model.ts`: + +```typescript +import { createAgentSession, ModelRuntime } from "@earendil-works/pi-coding-agent"; +import { getModel } from "@earendil-works/pi-ai"; +import type { CompletionResult, ModelClient } from "./model.js"; + +export interface PiModelOptions { + model: string; + thinkingLevel?: "off" | "low" | "medium" | "high"; + cwd?: string; +} + +export class PiModelClient implements ModelClient { + readonly name: string; + + constructor(private readonly opts: PiModelOptions) { + this.name = opts.model; + } + + async complete(prompt: string): Promise { + const modelRuntime = await ModelRuntime.create(); + const model = getModel("anthropic", this.opts.model); + + const { session } = await createAgentSession({ + model, + thinkingLevel: this.opts.thinkingLevel ?? "medium", + // Phase 1 sends the whole diff in the prompt: no filesystem tools, so the + // analyzer cannot wander and the cost is bounded. Tool budgets arrive in Phase 3. + tools: [], + cwd: this.opts.cwd ?? process.cwd(), + modelRuntime, + }); + + let text = ""; + session.subscribe((event: any) => { + if ( + event.type === "message_update" && + event.assistantMessageEvent?.type === "text_delta" + ) { + text += event.assistantMessageEvent.delta; + } + }); + + await session.prompt(prompt); + + // Usage accounting is refined in Phase 3 alongside prompt caching; for now + // record what the session reports and fall back to zeros. + const usage = (session as any).usage ?? {}; + return { + text, + usage: { + inputTokens: usage.inputTokens ?? 0, + outputTokens: usage.outputTokens ?? 0, + costUsd: usage.costUsd ?? 0, + }, + }; + } +} +``` + +**Step 5: Verify against the live API once, by hand** + +```bash +ANTHROPIC_API_KEY=... npx vitest run tests/integration/pi-model.test.ts +``` + +Expected: PASS. **If the SDK surface differs from the snippet above** (event shape, usage location, `getModel` arguments), fix this file against the real types — it is the one place the plan is guessing, and `docs/plans/` is not a contract with the SDK. Record whatever you learn in a short comment at the top of the file. + +**Step 6: Commit** + +```bash +git add src/analyze/pi-model.ts tests/integration package.json package-lock.json +git commit -m "feat: add pi SDK model client" +``` + +--- + +## Task 8: Aggregator + +**Files:** +- Create: `src/aggregate/aggregate.ts`, `tests/aggregate/aggregate.test.ts` + +**Step 1: Write the failing test** + +`tests/aggregate/aggregate.test.ts`: + +```typescript +import { expect, test } from "vitest"; +import { aggregate } from "../../src/aggregate/aggregate.js"; +import type { Finding } from "../../src/analyze/types.js"; + +const f = (over: Partial): Finding => ({ + file: "a.ts", + line: 10, + severity: "medium", + confidence: 0.8, + category: "correctness", + claim: "something", + analyzer: "code-quality", + ...over, +}); + +test("drops findings below the confidence gate", () => { + const r = aggregate([f({ confidence: 0.2 }), f({ confidence: 0.9 })], + { minConfidence: 0.5, maxComments: 10 }); + expect(r.posted).toHaveLength(1); + expect(r.suppressed).toHaveLength(1); +}); + +test("dedupes near-identical findings within a line window", () => { + const r = aggregate( + [f({ line: 10, analyzer: "code-quality" }), f({ line: 12, analyzer: "security" })], + { minConfidence: 0.5, maxComments: 10 }, + ); + expect(r.posted).toHaveLength(1); +}); + +test("ranks by severity then confidence", () => { + const r = aggregate( + [ + f({ file: "a.ts", severity: "low", confidence: 0.99 }), + f({ file: "b.ts", severity: "critical", confidence: 0.6 }), + ], + { minConfidence: 0.5, maxComments: 10 }, + ); + expect(r.posted[0]!.severity).toBe("critical"); +}); + +test("caps comment count but keeps the overflow as suppressed", () => { + const many = Array.from({ length: 30 }, (_, i) => + f({ file: `f${i}.ts`, claim: `c${i}` }), + ); + const r = aggregate(many, { minConfidence: 0.5, maxComments: 5 }); + expect(r.posted).toHaveLength(5); + expect(r.suppressed).toHaveLength(25); +}); +``` + +**Step 2: Run it and watch it fail** + +Run: `npx vitest run tests/aggregate/aggregate.test.ts` → FAIL. + +**Step 3: Write the aggregator** + +`src/aggregate/aggregate.ts`: + +```typescript +import type { Finding, Severity } from "../analyze/types.js"; + +export interface AggregateOptions { + minConfidence: number; + maxComments: number; + /** Findings within this many lines in the same file+category are duplicates. */ + lineWindow?: number; +} + +export interface AggregateResult { + posted: Finding[]; + suppressed: Finding[]; +} + +const RANK: Record = { + critical: 4, + high: 3, + medium: 2, + low: 1, +}; + +export function aggregate( + findings: Finding[], + opts: AggregateOptions, +): AggregateResult { + const window = opts.lineWindow ?? 5; + const suppressed: Finding[] = []; + + const confident = findings.filter((f) => { + if (f.confidence >= opts.minConfidence) return true; + suppressed.push(f); + return false; + }); + + const ranked = [...confident].sort( + (a, b) => + RANK[b.severity] - RANK[a.severity] || b.confidence - a.confidence, + ); + + const kept: Finding[] = []; + for (const f of ranked) { + const dup = kept.some( + (k) => + k.file === f.file && + k.category === f.category && + Math.abs(k.line - f.line) <= window, + ); + if (dup) suppressed.push(f); + else kept.push(f); + } + + return { + posted: kept.slice(0, opts.maxComments), + suppressed: [...suppressed, ...kept.slice(opts.maxComments)], + }; +} +``` + +**Step 4: Run it and watch it pass** + +Run: `npx vitest run tests/aggregate/aggregate.test.ts` → 4 passed. + +**Step 5: Commit** + +```bash +git add src/aggregate tests/aggregate +git commit -m "feat: dedupe, rank, and cap findings" +``` + +--- + +## Task 9: JSONL emitter + +**Files:** +- Create: `src/emit/record.ts`, `src/emit/jsonl.ts`, `tests/emit/jsonl.test.ts` + +**Step 1: Write the failing test** + +`tests/emit/jsonl.test.ts`: + +```typescript +import { mkdtempSync, readFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { expect, test } from "vitest"; +import { JsonlEmitter } from "../../src/emit/jsonl.js"; +import type { RunRecord } from "../../src/emit/record.js"; + +const record: RunRecord = { + runId: "run_1", + repo: "demo", + pullRequest: null, + commit: "a".repeat(40), + tier: "full", + tierReason: "rule:risk_path(**/auth/**)", + analyzers: ["code-quality"], + tokens: { in: 100, out: 20, cached: 0 }, + costUsd: 0.01, + latencyMs: 1200, + findings: { total: 2, posted: 1, suppressed: 1 }, + verdict: "changes_requested", + models: { "code-quality": "claude-opus-5" }, + pragentVersion: "0.1.0", + startedAt: "2026-08-04T00:00:00.000Z", +}; + +test("appends one JSON object per line", async () => { + const dir = mkdtempSync(join(tmpdir(), "pragent-emit-")); + const path = join(dir, "nested", "runs.jsonl"); + const emitter = new JsonlEmitter(path); + + await emitter.emit(record); + await emitter.emit({ ...record, runId: "run_2" }); + + const lines = readFileSync(path, "utf8").trim().split("\n"); + expect(lines).toHaveLength(2); + expect(JSON.parse(lines[0]!).run_id).toBe("run_1"); + expect(JSON.parse(lines[1]!).tier_reason).toMatch(/risk_path/); +}); +``` + +Note the snake_case assertion: the record is camelCase in TypeScript and snake_case on the wire, so downstream consumers (OTel, dashboards, `jq`) see conventional field names. + +**Step 2: Run it and watch it fail** + +Run: `npx vitest run tests/emit/jsonl.test.ts` → FAIL. + +**Step 3: Write the record type** + +`src/emit/record.ts`: + +```typescript +import type { Tier } from "../tier/types.js"; + +export type Verdict = "approved" | "commented" | "changes_requested" | "error"; + +export interface RunRecord { + runId: string; + repo: string; + pullRequest: number | null; + commit: string; + tier: Tier; + tierReason: string; + analyzers: string[]; + tokens: { in: number; out: number; cached: number }; + costUsd: number; + latencyMs: number; + findings: { total: number; posted: number; suppressed: number }; + verdict: Verdict; + models: Record; + pragentVersion: string; + startedAt: string; +} + +export function toWireFormat(r: RunRecord): Record { + return { + run_id: r.runId, + repo: r.repo, + pr: r.pullRequest, + commit: r.commit, + tier: r.tier, + tier_reason: r.tierReason, + analyzers: r.analyzers, + tokens: r.tokens, + cost_usd: r.costUsd, + latency_ms: r.latencyMs, + findings: r.findings, + verdict: r.verdict, + models: r.models, + pragent_version: r.pragentVersion, + started_at: r.startedAt, + }; +} +``` + +**Step 4: Write the emitter** + +`src/emit/jsonl.ts`: + +```typescript +import { appendFile, mkdir } from "node:fs/promises"; +import { dirname } from "node:path"; +import { toWireFormat, type RunRecord } from "./record.js"; + +export interface Emitter { + emit(record: RunRecord): Promise; +} + +export class JsonlEmitter implements Emitter { + constructor(private readonly path: string) {} + + async emit(record: RunRecord): Promise { + await mkdir(dirname(this.path), { recursive: true }); + await appendFile(this.path, JSON.stringify(toWireFormat(record)) + "\n", "utf8"); + } +} +``` + +**Step 5: Run it and watch it pass** + +Run: `npx vitest run tests/emit/jsonl.test.ts` → 1 passed. + +**Step 6: Commit** + +```bash +git add src/emit tests/emit +git commit -m "feat: emit JSONL run records" +``` + +--- + +## Task 10: Wire the CLI + +**Files:** +- Create: `src/cli.ts`, `src/review.ts`, `tests/review.test.ts` + +**Step 1: Write the failing test** + +`tests/review.test.ts` — the orchestration test, with every port faked: + +```typescript +import { mkdtempSync, readFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { expect, test, vi } from "vitest"; +import { review } from "../src/review.js"; +import type { ForgeAdapter, ReviewContext } from "../src/forge/types.js"; +import type { ModelClient } from "../src/analyze/model.js"; + +const ctx: ReviewContext = { + repo: "demo", + pullRequest: null, + headSha: "a".repeat(40), + baseSha: "b".repeat(40), + title: "add login", + files: [ + { path: "src/auth/login.ts", status: "modified", added: 5, removed: 1, patch: "+x" }, + ], +}; + +const forge = (): ForgeAdapter & { published: unknown[] } => { + const published: unknown[] = []; + return { + name: "fake", + published, + async getContext() { return ctx; }, + async publish(_c, findings) { published.push(...findings); }, + }; +}; + +const model: ModelClient = { + name: "fake", + async complete() { + return { + text: JSON.stringify({ + findings: [{ + file: "src/auth/login.ts", line: 3, severity: "high", confidence: 0.9, + category: "correctness", claim: "token is logged in plaintext", + }], + }), + usage: { inputTokens: 100, outputTokens: 20, costUsd: 0.02 }, + }; + }, +}; + +test("classifies, analyzes, publishes, and records", async () => { + const dir = mkdtempSync(join(tmpdir(), "pragent-review-")); + const f = forge(); + + const result = await review({ + forge: f, + model, + repoRoot: dir, + emitPath: join(dir, "runs.jsonl"), + }); + + expect(result.tier).toBe("full"); // risk path + expect(result.verdict).toBe("changes_requested"); + expect(f.published).toHaveLength(1); + + const line = JSON.parse(readFileSync(join(dir, "runs.jsonl"), "utf8").trim()); + expect(line.tier_reason).toMatch(/risk_path/); + expect(line.cost_usd).toBeCloseTo(0.02); +}); + +test("a trivial change skips the model entirely", async () => { + const dir = mkdtempSync(join(tmpdir(), "pragent-review-")); + const spy = vi.fn(model.complete); + const f = forge(); + f.getContext = async () => ({ + ...ctx, + files: [{ path: "pnpm-lock.yaml", status: "modified", added: 200, removed: 100, patch: "" }], + }); + + const result = await review({ + forge: f, + model: { name: "fake", complete: spy }, + repoRoot: dir, + emitPath: join(dir, "runs.jsonl"), + }); + + expect(result.tier).toBe("trivial"); + expect(spy).not.toHaveBeenCalled(); // this is the whole point of tiering + expect(result.verdict).toBe("approved"); +}); +``` + +**Step 2: Run it and watch it fail** + +Run: `npx vitest run tests/review.test.ts` → FAIL. + +**Step 3: Write the orchestrator** + +`src/review.ts`: + +```typescript +import { randomUUID } from "node:crypto"; +import { runAnalyzer } from "./analyze/runner.js"; +import type { ModelClient } from "./analyze/model.js"; +import { aggregate } from "./aggregate/aggregate.js"; +import { loadConfig } from "./config/load.js"; +import { JsonlEmitter } from "./emit/jsonl.js"; +import type { Verdict } from "./emit/record.js"; +import type { ForgeAdapter } from "./forge/types.js"; +import { classify } from "./tier/rules.js"; +import type { Tier } from "./tier/types.js"; +import { VERSION } from "./index.js"; + +export interface ReviewOptions { + forge: ForgeAdapter; + model: ModelClient; + repoRoot: string; + emitPath?: string; +} + +export interface ReviewResult { + tier: Tier; + verdict: Verdict; + postedCount: number; +} + +export async function review(opts: ReviewOptions): Promise { + const startedAt = new Date().toISOString(); + const t0 = Date.now(); + + const config = await loadConfig(opts.repoRoot); + const ctx = await opts.forge.getContext(); + const decision = classify(ctx.files, config.tierPolicy); + + // Trivial changes never reach a model — the tier is the cost control. + if (decision.tier === "trivial") { + await emit("approved", [], [], { in: 0, out: 0, cached: 0 }, 0, []); + return { tier: decision.tier, verdict: "approved", postedCount: 0 }; + } + + const result = await runAnalyzer({ id: "code-quality", model: opts.model, ctx }); + const { posted, suppressed } = aggregate(result.findings, { + minConfidence: config.minConfidence, + maxComments: config.maxComments, + }); + + await opts.forge.publish(ctx, posted); + + const verdict: Verdict = result.error + ? "error" + : posted.some((f) => f.severity === "critical" || f.severity === "high") + ? "changes_requested" + : posted.length > 0 + ? "commented" + : "approved"; + + await emit( + verdict, + posted, + suppressed, + { in: result.usage.inputTokens, out: result.usage.outputTokens, cached: 0 }, + result.usage.costUsd, + ["code-quality"], + ); + + return { tier: decision.tier, verdict, postedCount: posted.length }; + + async function emit( + verdict: Verdict, + posted: unknown[], + suppressed: unknown[], + tokens: { in: number; out: number; cached: number }, + costUsd: number, + analyzers: string[], + ) { + const emitter = new JsonlEmitter(opts.emitPath ?? config.emitPath); + await emitter.emit({ + runId: `run_${randomUUID()}`, + repo: ctx.repo, + pullRequest: ctx.pullRequest, + commit: ctx.headSha, + tier: decision.tier, + tierReason: decision.reason, + analyzers, + tokens, + costUsd, + latencyMs: Date.now() - t0, + findings: { + total: posted.length + suppressed.length, + posted: posted.length, + suppressed: suppressed.length, + }, + verdict, + models: analyzers.length ? { "code-quality": opts.model.name } : {}, + pragentVersion: VERSION, + startedAt, + }); + } +} +``` + +**Step 4: Run it and watch it pass** + +Run: `npx vitest run tests/review.test.ts` → 2 passed. + +**Step 5: Write the CLI** + +`src/cli.ts`: + +```typescript +#!/usr/bin/env node +import { Command } from "commander"; +import { PiModelClient } from "./analyze/pi-model.js"; +import { loadConfig } from "./config/load.js"; +import { LocalForgeAdapter } from "./forge/local.js"; +import { review } from "./review.js"; +import { VERSION } from "./index.js"; + +const program = new Command(); + +program.name("pragent").version(VERSION); + +program + .command("review") + .description("Review a change and publish findings") + .option("--range ", "git range to review", "HEAD~1..HEAD") + .option("--cwd ", "repository root", process.cwd()) + .option("--model ", "model id override") + .action(async (opts) => { + const config = await loadConfig(opts.cwd); + const result = await review({ + forge: new LocalForgeAdapter({ cwd: opts.cwd, range: opts.range }), + model: new PiModelClient({ model: opts.model ?? config.models.full }), + repoRoot: opts.cwd, + }); + + console.error( + `\ntier=${result.tier} verdict=${result.verdict} findings=${result.postedCount}`, + ); + // 0 clean, 1 findings worth blocking on, 2 internal error. + process.exit( + result.verdict === "changes_requested" ? 1 : result.verdict === "error" ? 2 : 0, + ); + }); + +program.parseAsync(); +``` + +**Step 6: Verify end to end by hand** + +```bash +npm run build +cd ~/Projects/pragent +ANTHROPIC_API_KEY=... node dist/cli.js review --range HEAD~1..HEAD +cat .pragent/runs.jsonl | tail -1 | jq +``` + +Expected: findings (or a clean pass) on stdout, `tier=... verdict=...` on stderr, one JSONL line appended, exit code matching the verdict. + +**Step 7: Commit** + +```bash +git add src/cli.ts src/review.ts tests/review.test.ts +git commit -m "feat: wire pragent review CLI end to end" +``` + +--- + +## Task 11: Close out Phase 1 + +**Step 1: Full check** + +```bash +npm test && npm run typecheck && npm run build +``` + +All green before proceeding. + +**Step 2: Update the README** + +Replace "implementation not started" with a short *Current state* section: what `pragent review` does today, the local-only limitation, and how to run it. + +**Step 3: Commit and tag** + +```bash +git add -A +git commit -m "docs: record phase 1 walking skeleton state" +git tag v0.1.0 +git push origin main --tags +``` + +--- + +## Later phases (milestones, not tasks) + +Do not start these without a plan of their own. Each gets the same treatment as above. + +**Phase 2 — Gitea end to end.** `GiteaForgeAdapter` implementing the same port (diff via API, inline review comments, commit status). Woodpecker pipeline step. Token handling via CI secrets, never in config. *Done when:* a real PR in a Gitea repo gets reviewed automatically on push. + +**Phase 3 — Profile and full tier.** `pragent init` writes `.pragent/profile.yml`; analyzers fan out with a byte-identical shared prefix; a cache-prefix invariant test fails the build if any analyzer perturbs it; real token/cost accounting from usage headers; tool budgets per analyzer. *Done when:* a full-tier PR runs 4 analyzers and the measured cost lands within the design doc's ~$0.80–2.00 band. + +**Phase 4 — Extensibility hardening.** Analyzer plugin loading from `.pragent/analyzers/` and npm; config layering with locked org keys; `pragent explain` and `pragent replay`. *Done when:* someone adds a working analyzer without touching `src/`. + +**Phase 5 — Second forge.** GitLab adapter plus a Jenkins recipe. This is the honest test of the port abstraction; expect it to expose one or two leaks. *Done when:* the same analyzer config works unmodified on Gitea and GitLab. + +**Phase 6 — Analytics maturity.** OTel span export, the finding-outcome feedback loop (resolved/👎 → `finding_outcome` records), per-analyzer eval harness built on `replay`. *Done when:* false-positive rate per analyzer is a number you can read off a query. + +--- + +## Known risks + +- **The `pi` SDK surface in Task 7 is from published docs, not a verified local run.** Expect the event and usage shapes to need correcting on first contact. This is deliberately confined to one file behind the `ModelClient` port so a mismatch costs an hour, not a redesign. +- **The glob matcher in Task 4 is minimal** (`*` and `**` only). It is enough for the default risk paths. If repo configs start needing brace expansion or character classes, swap in `picomatch` — the seam is `matchGlob`, one function. +- **Usage accounting is approximate until Phase 3.** Do not build cost dashboards on Phase 1 numbers.