Ship token spend, latency and equivalent cost for every review to the
self-hosted Langfuse so per-model behaviour is queryable as a trend rather
than one PR comment at a time.
langfuse_trace.py is stdlib-only and emits via the public ingestion API.
Traces split into `ollama` and `claude` environments keyed off the bare model
name, not the provider: both paths go through the same headroom proxy, so the
provider prefix says nothing about which spend story a review belongs to. The
pilot's own path bills $0, so the reported cost is the equivalent price from
cost_model.PRICES.
ai_review.py calls _emit_langfuse on both token-spending exit paths (the
normal post and the salvage path). Import and emission are wrapped in a
blanket except: with no LANGFUSE_HOST or key pair the whole thing is a silent
no-op, and a telemetry failure must never fail a review.
These files were previously deployed only by way of the image build's
`COPY . /app`, so a clean checkout would have silently dropped tracing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- opencode_review.install_config: opencode 1.3.10 rejects factory
opencode.json$'\''s top-level "$schema" key with 'Unrecognized key'
at parse time, causing every fresh review to fail as the misleading
'opencode empty text (rc=0)'. Add a small drop-list of known-bad
top-level keys + a sanitizer applied to both the fast (no-env) and
slow (env-substituted) write paths.
- feedback_harvest: Gitea versions occasionally serialize reaction
content / inline-comment resolver as a dict instead of a string,
crashing harvester with 'dict object has no attribute strip'. Coerce
both via str() with a brief comment documenting the WHY.
Affected reviews: techspark/suaspark-dashboard #11 (fixed),
PRAgent reviews on PR #9 era (recovered).
Verified: opencode 1.3.10 emits real '### Summary of Changes' body.
POST /r/<owner>/<name>/edit reads the current .pr-review.json via the
Gitea contents API (404 → start from {}), updates static_message and
model, then PUTs the file back as the bot identity. Static message is
stripped and capped at 400 chars; model is validated against
cost_model.PRICES (unknown → flash + redirect, no save). CSRF token is
the dashboard auth token (same value, simplest binding).
Helpers (_http) moved to urllib.request at module top so tests can patch
the entry point; the helper returns the Gitea status code so 404 (file
missing) cleanly omits sha on the create-PUT path.
6 tests under tests/pilot/test_dashboard_edit.py covering the happy
path, the 400-char cap, the unknown-model rejection, auth + CSRF gates,
and the file-missing creation path.
Mirrors webhook_server.py's BaseHTTPRequestHandler + ThreadingHTTPServer
shape. Pure stdlib, no pip deps. Routes:
GET / overview (totals + 7-day sparkline + top repos)
GET /r/<owner>/<name> repo summary (severity histogram + top findings)
GET /r/<owner>/<name>/<index> one PR's findings
GET /r/<owner>/<name>/<index>/raw raw Markdown body (via Gitea contents API)
GET /static/style.css dark-mode stylesheet
GET /login login form
POST /login compare token, set HttpOnly+SameSite cookie
POST /r/<owner>/<name>/edit (Tasks C+D)
Auth: when PRAGENT_DASHBOARD_TOKEN is set, every route except /login and
/static/* requires Cookie: pragent_dash=<token>. Unset → tailnet-only.
All HTML rendered via string.Template; every dynamic value is escaped
with html.escape(..., quote=True). No .format, no f-string templates.
12 tests under tests/pilot/test_dashboard.py.
Three pure functions — overview / repo_summary / pr_summary — that open
the feedback SQLite via feedback.init, run their queries, and return
plain dicts/lists. All three tolerate a missing or empty DB by returning
a zero-shaped dict.
Cost is hardcoded 0.0: per-review usage:cost isn't stored, only the raw
review rows are. Surfacing a rolled-up dollar figure without telemetry
would be guessing, so we don't.
17 new tests under tests/pilot/test_dashboard_data.py.
Repos that need to pin the review engine (e.g. 'this project requires
claude-sonnet-5 for the budget line' or 'route everything through
gpt-5.6-luna for now') can declare a top-level 'model' string in
.pr-review.json. The parser validates the value against cost_model.PRICES
(lazy import — ollama path stays dep-free) and silently drops unknown
values with a stderr pointer to the valid key set so a typo in the
config file surfaces in the logs instead of silently falling back.
The orchestrator gains a small _resolve_display_model(base, config)
helper that implements a 3-way precedence:
1. OPENCODE_MODEL env (operator override, used verbatim)
2. config['model'] (per-repo override)
3. f'headroom/{base}' (default)
review_pr resolves once (lazy fallback before config is loaded) and
re-resolves after .pr-review.json is fetched, then threads the result
into the opencode subprocess, REVIEW_HEADER, and the cost-line
parenthetical. Same single value everywhere — no more mix of bare
OLLAMA_MODEL id in the header and a stale free-tier literal in the cost
line.
Tests cover parsing acceptance, parsing rejection (capsys stderr),
type validation, precedence in all 4 (env×config) combinations, and an
end-to-end sanity check that format_review_body shows the override and
not the base id.
Repos can pin a free-text notice (e.g. 'this repo is in maintenance mode',
or 'reviewers: focus on the public API only for this quarter') in
.pr-review.json:static_message. The string is stripped and capped at 400
chars (mirror of the existing instructions cap), then rendered as a
Markdown blockquote (> {msg}) directly under the REVIEW_HEADER so it
surfaces on every review without scrolling.
Plumbing: parse_repo_config exposes 'static_message'; format_review_body
accepts a static_message kwarg and inserts the blockquote before the
'### Summary of Changes' section; review_pr threads
config.get('static_message') to both call sites (salvage path + happy
path). Empty / non-string values are silently dropped, mirroring the
parser's 'ignore blank' handling for every other text field.
The webhook pod now routes through headroom's MiniMax-M2.7 endpoint, but the
cost line in the AI-usage collapsible still read 'headroom glm-5.2:cloud'.
Resolve a single display_model at the top of review_pr (OPENCODE_MODEL env
wins, else headroom/{OLLAMA_MODEL}) and pass it to:
* the opencode subprocess (was already doing this on the same line, now
sharing the value)
* format_review_body so REVIEW_HEADER also reflects the actual run
* _render_collapsible_usage so the parenthetical reads
'({display_model} — free tier)' or '({display_model} — billed)'.
Test additions in tests/pilot/test_ai_review.py cover:
* the parenthetical picks up the passed-in model verbatim
* the full provider prefix survives (headroom/<id>) for the opencode path
* nonzero cost flips the inner clause from 'free tier' to 'billed'
* the existing nonzero-cost assertion flips to assert 'billed' instead
of dropping the parenthetical entirely
opencode's @ai-sdk/anthropic provider sends the config's apiKey as the
x-api-key header. The committed opencode.json carries apiKey="ollama"
(placeholder) so the repo can be public. When the headroom upstream
switches to an auth-gated provider (e.g. MiniMax), the placeholder
returns 'No credentials found'. install_config now also patches
options.apiKey from PRAGENT_MODEL_API_KEY when set, mirroring the
existing baseURL patching.
Co-Authored-By: Claude <noreply@anthropic.com>
The previous two commits put the label re-read in the webhook, at review
start. That is too early to help: the review claims on the AI-REVIEW event
and the re-read runs milliseconds later, while the reviewer's second click
(AI-USAGE) is still a second or two away. It would have kept 404ing quietly
if the path fix hadn't landed, and even fixed it caught nothing.
Move the check to where the decision is actually used — just before the
usage block is rendered, after the model has run. That is a minute or more
after the trigger, by which time the label is there. Attribution is computed
in the same branch, so a late opt-in still gets its per-comment token lines.
`pr_has_label` goes through the existing gitea_get helper, which owns the
/api/v1 prefix, so the path can't drift again. Any failure returns False and
the payload's verdict stands: a review is never lost over a usage section.
Reverts the webhook-side re-read from 4ef62f2 and aedea97.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
GITEA_API is the host with no version prefix — ai_review._get / _post append
/api/v1 per call. The re-read helper didn't, so it 404'd on every review and
silently fell back to the payload's verdict: "could not re-read labels for
gitea_admin/pragent#10: HTTP Error 404" in the pod log, usage block still
missing. Caught by labelling PR #10 AI-REVIEW then AI-USAGE, which is the
exact sequence the fix exists to handle.
Test asserts the composed URL, since a wrong path here fails silently by
design (the helper swallows errors so a review is never lost over a usage
section).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
Labelling a PR is two webhook events. AI-REVIEW arrives first, the review
claims (repo, index, sha) and starts; the AI-USAGE event that follows a
moment later hits the in-flight dedupe and is dropped. `report_usage` was
snapshotted from the first payload, which had not seen AI-USAGE yet, so the
review posted without its usage block even though the label was on the PR by
the time it finished — observed on gitea_admin/pragent#9 (`usage=False` in
the pod log, no <details> section in the posted review).
Re-read the labels from the API at review start and upgrade the flag. The
read is best-effort: any failure logs and returns [], leaving the payload's
verdict intact, because a usage section is not worth failing a review over.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
Four defects, all found reviewing PR #9 (two of them by pragent-bot's own
review of that PR, which the anchoring bug then misplaced):
* compress_diff dropped context lines but copied the original `@@` hunk
header verbatim, so the header no longer described the lines beneath it.
parse_diff_anchors then walked stale headers and produced anchor sets
shifted by the number of elided lines, misplacing inline comments or
demoting them to bullets. Each surviving run of lines is now re-emitted as
its own hunk with a recomputed `@@ -a,b +c,d @@`, so the output stays a
valid unified diff whose numbers describe the real post-change file. The
pseudo-marker `@@ … N context line(s) omitted … @@` is gone; it parsed as
a hunk header and reset the anchor counter to 0. Anchoring additionally
runs on the raw diff now, so the prompt window can never shrink the
anchorable set.
* compress_diff's `_FILE_HEADER` regex matched diff *body* lines: a removed
YAML `---` separator or an added `++` line was read as a file header,
truncating the hunk and dropping its `@@` header with it. Body detection
is now prefix-based, with a full-shape hunk-header regex.
* extract_finding_bullets could not match the bullets pragent itself posts:
summary_bullets renders an emoji severity badge between the `-` and the
`[SEV]` tag, which the regex rejected, so compact_prior_reviews always
returned [] and every re-review repeated its previous findings.
* triage returning `{"lenses":[]}` — documented in .opencode/agents/triage.md
as "no lens has surface, skip the fan-out" — ran every lens instead, since
_intersect_with_triage mapped an empty selection to "all" and the call site
had a second `or reviewers` fallback. `[]` and None are now distinct
outcomes: `[]` skips, None fails open. A roster naming only unknown lens
ids now fails open rather than silencing the review. The skip path returns
a well-formed empty-findings response instead of "", which had landed in
ai_review's unparseable-output branch and posted "AI review produced no
parseable output" — a malfunction message for a normal verdict.
Also: non-URL references (a CVE id, a doc title) rendered as
`[CVE-2024-1234](CVE-2024-1234)`, a broken relative link in Gitea — now
plain text. PRAGENT_DIFF_CONTEXT and friends parse through _int_env, so a
typo logs and falls back instead of killing a review mid-flight. Removed
format_usage_section, dead since the collapsible usage block replaced it and
carrying a duplicate copy of the price-target logic.
Tests: 290 -> 301. New coverage for hunk-header fidelity before/after
compression, header-shaped content lines, the bullet round-trip against the
real renderer, and triage's three outcomes (previously untested).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B11e8TZZxJyzHW7jj7KWUN
Long agent loops re-send the brief prefix on every step; cheap reusable
knowledge (architecture summary, module map, conventions, glossary) belongs
in a versioned file the maintainers control so the agent doesn't re-derive
it from the source tree on every PR. Two wiring paths, merged (env first):
* env var PRAGENT_ADDITIONAL_CONTEXT_URL — comma-separated, deployment-wide
* .pr-review.json:additional_context_urls — list[str], read from the PR's
base branch (same trust boundary as the rest of the file)
Implementation:
* _parse_additional_context_env splits/dedupes/trims.
* _resolve_additional_context_urls(config) merges env (first) + config
(then, skipping env-dupes); caps at 8.
* fetch_additional_context(urls) fetches each URL with urllib (5s timeout,
http/https only — file://, javascript:, ftp:// rejected defensively),
caches by URL in a module-level dict for the pod lifetime, truncates
per-URL to 4k chars + total to 16k chars, best-effort (network errors
are logged and skipped — never aborts the review).
* Result injected into build_user_prompt under "## Repo-provided context"
between repo config and prior reviews. In the opencode engine it lands
in .pragent/brief.md under its own section. The brief explicitly labels
each block's CONTENT as untrusted (same as PR description) — section
heading is trustworthy, body isn't.
* parse_repo_config accepts the field, caps at 8 entries, drops
non-strings and empty strings.
Docs: pilot/README-webhook.md "Repo-provided static context" section —
env var + JSON example + Nexus raw-hosted recipe.
Tests: 14 new (208 total), covering env merging + dedup, scheme rejection,
per-URL cap, total cap, caching by URL, brief injection. All mock urllib
with a context-manager stand-in (no real network).
Co-Authored-By: Claude <noreply@anthropic.com>
Three changes from operator feedback:
1. Per-comment � attribution restored on inline comments (operator wants
it back — the PR-level collapsible is collapsed by default, so the
attribution is the visible signal of per-finding cost share).
Hidden only when no _tok_attrib was computed (legacy callers / ollama
path without usage metering).
2. Agent prompt now bounds reads beyond the diff — the single biggest
driver of input-token bloat on long agent loops:
* ≤ 5 file reads beyond the diff for the entire review
* ≤ 80 lines per read (use --offset + --limit)
* ≤ 3 grep calls beyond the diff (prefer rtk grep)
* no re-reads of files already seen
* no directory walks (ls -R, find .)
* honor .pr-review.json:exclude_paths
3. De-generalize cost_model calibration labels. The OBSERVED_RUNS list
referred to `gitea_admin/pragent#7` — a real internal repo path that
blocks commercialization. Replaced with `internal/hardening-PR (16
files, 1020 insertions / 91 deletions)`. The numbers (input/output
tokens, steps, duration) are unchanged — only the labels are
generic.
Tests:
* test_inline_comment_body_with_attribution_line — asserts 🪙 line
shows when _tok_attrib is set
* test_inline_comment_body_no_attribution_no_coin_line — still
verifies the line is hidden when no attribution data
* test_observed_report_prices_every_model — asserts no internal
repo name appears in the rendered report
Co-Authored-By: Claude <noreply@anthropic.com>
PR-level comment layout (per operator's format guide):
* Summary of Changes — 2-4 bullets, sourced from the agent's new
`summary_changes` JSON field. Falls back to splitting the prose
`summary` if the list is missing.
* Key Risks & Concerns — bullets from the new `risks` JSON field.
* Findings Overview — Markdown table covering every finding
(severity emoji / location / one-line problem). Both anchored and
unanchored findings appear here so the table is the single scan point.
* Unanchored Notes — bullets with severity + fix + Markdown-linked ref,
for findings with no post-change line to anchor.
* AI Usage & Run Details — wrapped in a <details>/<summary> collapsible
so the body stays scannable. Cost line stays inside it.
Inline comment shape:
* Severity badge: 🔴 [HIGH] / 🟡 [MEDIUM] / 🔵 [LOW] / ⚪ [INFO].
Unknown severities fall back to � [INFO].
* 1-2 short paragraphs of problem; **Fix:** label for the fix line.
* Standard ```suggestion fence for replacement code (Gitea/Forgejo
apply-on-click). Language-tagged fences are no longer used for
single-file diffs.
* Reference as a Markdown hyperlink, visible label truncated to
<=60 chars; the underlying URL is preserved verbatim.
* NO per-comment 🪙 token attribution. All telemetry stays in the
collapsible block on the PR-level comment.
Agent prompt updated to emit `summary_changes` and `risks` in the JSON
output (backward-compatible — older outputs missing them still parse;
they fall back to splitting the prose `summary`).
Tests: 15 new (severity emoji mapping, reference truncation, findings
table escaping, collapsible usage rendering, summary_changes+risks
layout). Existing tests updated for the new structure.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude