Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 370adcde6f | |||
| 84325e61c1 |
@@ -1 +0,0 @@
|
||||
# judge trigger 1788203999
|
||||
@@ -35,9 +35,9 @@ code never has to leave your network.
|
||||
|
||||
## Status
|
||||
|
||||
A **pilot** is live and reviewing real PRs. The current runtime architecture is
|
||||
documented in [`docs/architecture.md`](docs/architecture.md); older framework
|
||||
plans remain in [`docs/plans/`](docs/plans/) as historical design material.
|
||||
A **pilot** is live and reviewing real PRs. The full framework (`pragent init`,
|
||||
tiering as code, analyzer fan-out, `explain` / `replay`) is designed but not
|
||||
built — see [`docs/plans/`](docs/plans/).
|
||||
|
||||
What works today:
|
||||
|
||||
@@ -51,12 +51,10 @@ What works today:
|
||||
- `.pr-review.json` for per-repo focus and house rules (also the opt-in flag)
|
||||
- token-usage reporting on every review, measured from opencode `step_finish`
|
||||
events
|
||||
- Langfuse traces, equivalent-cost reporting, evaluation scores, and feedback
|
||||
harvesting
|
||||
- containment against hostile PR content (see [Security](#security))
|
||||
|
||||
Not yet: status checks, fail-close, attention tiering enforced in code (it is
|
||||
currently a skill the agent follows), multi-model routing, and a CLI framework.
|
||||
currently a skill the agent follows), multi-model routing.
|
||||
|
||||
## How a review runs
|
||||
|
||||
@@ -96,8 +94,9 @@ path is in [`pilot/README.md`](pilot/README.md).
|
||||
The model endpoint is supplied at runtime via `PRAGENT_MODEL_BASE_URL`; the
|
||||
committed `opencode.json` carries a placeholder.
|
||||
|
||||
Per-review token spend, latency, equivalent cost, and evaluation scores are
|
||||
shipped to a self-hosted Langfuse: [`pilot/README-langfuse.md`](pilot/README-langfuse.md).
|
||||
Per-review token spend, latency and equivalent cost are shipped to a
|
||||
self-hosted Langfuse, split into `ollama` and `claude` environments so the two
|
||||
spend stories stay separate: [`pilot/README-langfuse.md`](pilot/README-langfuse.md).
|
||||
Emission is a silent no-op unless `LANGFUSE_HOST` and the key pair are set.
|
||||
|
||||
## Extending it
|
||||
@@ -176,7 +175,7 @@ python3 pilot/cost_model.py --help # other mixes, volumes, models
|
||||
## Development
|
||||
|
||||
```bash
|
||||
python3 -m pytest tests -q # stdlib-only tests, no network
|
||||
python3 -m pytest tests -q # 137 tests, stdlib only, no network
|
||||
```
|
||||
|
||||
The pilot is stdlib-only Python by design — it runs from a bare `python:slim`
|
||||
@@ -186,5 +185,3 @@ review time.
|
||||
## License
|
||||
|
||||
Not yet chosen. Until one is added, no reuse rights are granted.
|
||||
|
||||
_pilot eval judges test 1788201461_
|
||||
|
||||
@@ -1,68 +0,0 @@
|
||||
# pragent current architecture
|
||||
|
||||
Status: pilot implementation, September 2026.
|
||||
|
||||
## System shape
|
||||
|
||||
```text
|
||||
Gitea pull_request webhook
|
||||
│ signed HTTP
|
||||
▼
|
||||
webhook_server ── trusted base config ──► review_config
|
||||
│ bounded worker
|
||||
▼
|
||||
review_pr facade/orchestrator
|
||||
├── gitea_client fetch diff, reviews, config; publish review
|
||||
├── diff_compress reduce prompt context
|
||||
├── opencode_review isolated checkout + agent execution
|
||||
│ └── model / repo factory (.opencode)
|
||||
├── review parsing normalize findings + validate anchors
|
||||
├── feedback persist reactions and derive scores
|
||||
└── langfuse_trace usage, cost, evaluation telemetry
|
||||
```
|
||||
|
||||
## Seams and responsibilities
|
||||
|
||||
The external seam is `ai_review.review_pr(...)`: one call represents one review
|
||||
attempt and returns success/skip status. The module is retained as a facade for
|
||||
the CI and webhook callers that already import it.
|
||||
|
||||
The internal seams are deliberately narrower:
|
||||
|
||||
- `review_config.repo_enabled(get, ...)` owns the security-sensitive opt-in
|
||||
decision. It receives a transport function, so malformed configuration and
|
||||
failure behavior are deterministic in tests.
|
||||
- `gitea_client.request()` and `GiteaClient` own HTTP authentication, JSON
|
||||
request encoding, timeout, and Gitea URL construction.
|
||||
- `model_client.complete()` owns the legacy Anthropic-compatible request shape.
|
||||
`opencode_review` is the preferred agent adapter and keeps Gitea I/O out of
|
||||
the autonomous process.
|
||||
- `diff_compress`, finding parsing, config filtering, and rendering remain
|
||||
pure transformations. Their callers do not need to know how model or Gitea
|
||||
transport works.
|
||||
- `langfuse_trace` is an optional sink. It is fail-open and cannot change the
|
||||
review result.
|
||||
|
||||
## Trust model
|
||||
|
||||
The review config is read from the PR base branch, never the PR head. The agent
|
||||
checkout is treated as hostile: instruction files are removed, credentials are
|
||||
not inherited, and the agent only returns text to the Python publisher. Python
|
||||
validates finding paths and post-change line anchors before sending comments.
|
||||
|
||||
## Observability
|
||||
|
||||
Langfuse is the operational analytics surface. A trace groups runs by
|
||||
`owner/repo#PR`; generations carry usage and cost basis; evaluation scores and
|
||||
human-feedback scores are attached later. The former SQLite-backed dashboard
|
||||
was removed. SQLite remains only as the feedback/evaluation ingestion store.
|
||||
|
||||
## Removed surface
|
||||
|
||||
The dashboard server, dashboard data module, dashboard tests, dashboard README,
|
||||
and dashboard Kubernetes manifest are intentionally gone. Operators use the
|
||||
Langfuse UI for review trends and cost analysis, and Gitea for review details
|
||||
and configuration changes.
|
||||
|
||||
Historical design/implementation plans under `docs/plans/` describe the
|
||||
earlier TypeScript framework proposal and are not the runtime architecture.
|
||||
@@ -0,0 +1,118 @@
|
||||
# pragent pilot — central dashboard service.
|
||||
#
|
||||
# Read-only overview + per-repo / per-PR drilldown over the same SQLite
|
||||
# feedback DB the webhook writes. Also mutates `.pr-review.json` on covered
|
||||
# repos via the Gitea contents API (Tasks C+D in pilot/dashboard.py). Same
|
||||
# image as the webhook (`pragent-webhook:optin`) — all pilot modules are
|
||||
# baked in at /app/pilot/.
|
||||
#
|
||||
# Routes: GET / (overview), GET /r/<o>/<n> (repo), GET /r/<o>/<n>/<i> (PR),
|
||||
# GET /r/<o>/<n>/<i>/raw (PR markdown raw), GET /login, GET /static/style.css,
|
||||
# POST /login, POST /r/<o>/<n>/edit.
|
||||
#
|
||||
# Auth: PRAGENT_DASHBOARD_TOKEN in the pragent-webhook Secret, cookie
|
||||
# `pragent_dash=<token>`, single-user. Empty / unset = no auth (tailnet-only).
|
||||
#
|
||||
# NodePort 30082 — only reachable on the Tailscale / LAN side of kubernets
|
||||
# (100.74.17.70 / 192.168.1.80) until/unconfigured. Mirrors pragent-webhook.yaml
|
||||
# in every other respect (uid 10001, nodeSelector, /data PVC).
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: pragent-dashboard
|
||||
namespace: pragent
|
||||
labels:
|
||||
app: pragent-dashboard
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: pragent-dashboard
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: pragent-dashboard
|
||||
spec:
|
||||
# Same node as the webhook — holds the headroom proxy + the /data PVC.
|
||||
nodeSelector:
|
||||
kubernetes.io/hostname: kubernets
|
||||
# Dashboard is read-only over /data and only mutates Gitea (not local
|
||||
# files), so unprivileged is fine. fsGroup matches the image's USER
|
||||
# directive (10001) so the RO mount is readable.
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 10001
|
||||
runAsGroup: 10001
|
||||
fsGroup: 10001
|
||||
containers:
|
||||
- name: dashboard
|
||||
image: pragent-webhook:optin
|
||||
imagePullPolicy: Never
|
||||
workingDir: /app
|
||||
command: ["python3", "-m", "pilot.dashboard"]
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 8081
|
||||
env:
|
||||
- name: PRAGENT_FEEDBACK_DB
|
||||
value: /data/feedback.db
|
||||
- name: PRAGENT_GITEA_API
|
||||
value: http://gitea-http.gitea.svc.cluster.local:3000
|
||||
# Dashboard reads DASHBOARD_PORT (not PORT) — verified in
|
||||
# pilot/dashboard.py:51. Default 8081 if unset.
|
||||
- name: DASHBOARD_PORT
|
||||
value: "8081"
|
||||
# Used by /r/<o>/<n>/edit to PUT updated JSON to the repo's
|
||||
# contents API. Reuses the same bot token the webhook uses.
|
||||
- name: PRAGENT_BOT_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: pragent-webhook
|
||||
key: PRAGENT_BOT_TOKEN
|
||||
# Auth cookie value. Add to the pragent-webhook Secret with:
|
||||
# kubectl patch secret pragent-webhook -n pragent --type=json \
|
||||
# -p='[{"op":"add","path":"/data/PRAGENT_DASHBOARD_TOKEN","value":"<base64>"}]'
|
||||
- name: PRAGENT_DASHBOARD_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: pragent-webhook
|
||||
key: PRAGENT_DASHBOARD_TOKEN
|
||||
# /data is read-only — the dashboard doesn't write the SQLite file;
|
||||
# .pr-review.json mutations go through the Gitea contents API, not
|
||||
# local fs. RO avoids any chance of two pods racing the same RWO PVC.
|
||||
volumeMounts:
|
||||
- name: feedback-data
|
||||
mountPath: /data
|
||||
readOnly: true
|
||||
# No /health route in dashboard.py (returns 404 on unknown paths).
|
||||
# Probes omitted intentionally — see pilot/dashboard.py:687-721.
|
||||
# Resources: dashboard is read-heavy + tiny writes. /data RO + no
|
||||
# subprocess fan-out (no opencode) keeps footprint small.
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 256Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
volumes:
|
||||
- name: feedback-data
|
||||
persistentVolumeClaim:
|
||||
claimName: pragent-feedback-data
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: pragent-dashboard
|
||||
namespace: pragent
|
||||
spec:
|
||||
selector:
|
||||
app: pragent-dashboard
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
targetPort: http
|
||||
nodePort: 31540
|
||||
type: NodePort
|
||||
# 31540 — auto-allocated at first apply (30082 was already taken by
|
||||
# habitsnow/habitsnow-proxy). Tailscale / LAN only until a Caddy route is set.
|
||||
@@ -0,0 +1,269 @@
|
||||
# pragent pilot — central dashboard service
|
||||
|
||||
A read-only overview + per-repo / per-PR drilldown over the same SQLite
|
||||
feedback DB the webhook writes, plus a small form to mutate `.pr-review.json`
|
||||
on a covered repo via the Gitea contents API. Companion to the
|
||||
[webhook service](README-webhook.md); reuses the webhook image
|
||||
(`pragent-webhook:dashboard`) — the pilot modules are baked into `/app/pilot/`,
|
||||
and the dashboard is just `python3 -m pilot.dashboard`.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Browser
|
||||
│
|
||||
▼
|
||||
Caddy (TLS, wildcard cert via Cloudflare DNS-01)
|
||||
│ https://pragent-dashboard.marcospaulo.dev.br → 100.74.17.70:31541
|
||||
▼
|
||||
Service oauth2-proxy-dashboard.pragent.svc.cluster.local (NodePort 31541, ns pragent)
|
||||
│
|
||||
│ oauth2-proxy fronts the dashboard, enforces Logto SSO + email allowlist
|
||||
│ sets X-Forwarded-User / X-Forwarded-Email on accepted requests
|
||||
▼
|
||||
Service pragent-dashboard.pragent.svc.cluster.local (ClusterIP, ns pragent)
|
||||
│
|
||||
▼
|
||||
pragent-dashboard pod (uid 10001, /data RO, no subprocess fan-out)
|
||||
│
|
||||
├── read /data/feedback.db (PVC pragent-feedback-data, RO)
|
||||
├── GET .../repos/{o}/{r}/... (Gitea contents API, bot token)
|
||||
└── PUT .../repos/{o}/{r}/contents/.pr-review.json
|
||||
(edit form submit; Gitea commits a new sha)
|
||||
```
|
||||
|
||||
Fail-soft. Nothing is ever written to local disk by the dashboard — the
|
||||
SQLite file is read-only and `.pr-review.json` mutations go through Gitea's
|
||||
contents API so the commit history records who changed what.
|
||||
|
||||
The dashboard `Service` is **ClusterIP** — only oauth2-proxy can reach it.
|
||||
Public access is gated by Caddy (TLS termination) → oauth2-proxy (Logto SSO
|
||||
+ allowlist) → dashboard.
|
||||
|
||||
## What it does
|
||||
|
||||
- **Overview** (`GET /`): summary stats across all onboarded repos — total
|
||||
reviews, distinct PRs, finding counts by severity, false-positive /
|
||||
accepted-pattern scores (see "Feedback loop" in README-webhook.md), and a
|
||||
sparkline of review activity.
|
||||
- **Repo drilldown** (`GET /r/<owner>/<name>`): per-repo PRs with their
|
||||
last-review status, finding counts, and links to PR-level drilldowns.
|
||||
- **PR drilldown** (`GET /r/<owner>/<name>/<index>`): the bot's review(s)
|
||||
on that PR, inline findings, and reaction / resolved status harvested
|
||||
by `feedback_harvest.py`.
|
||||
- **Raw review** (`GET /r/<owner>/<name>/<index>/raw`): the markdown body
|
||||
of the most recent review, for copy-paste / diff-with-prose workflows.
|
||||
- **Edit form** (`POST /r/<owner>/<name>/edit`): a small HTML page that
|
||||
loads the current `.pr-review.json` from the repo's default branch and
|
||||
lets the operator edit the JSON (validated, then PUT to Gitea contents
|
||||
API). This is how repo-local `focus` / `instructions` /
|
||||
`reviewers` / `severity_threshold` get tuned per-repo after seeing
|
||||
the feedback roll-up.
|
||||
|
||||
All routes return HTML (or plain text for `/raw`) with the same stylesheet
|
||||
(`/static/style.css`).
|
||||
|
||||
## Routes
|
||||
|
||||
| method | path | auth | description |
|
||||
|--------|-----------------------------------|------|----------------------------------------------|
|
||||
| GET | `/` | yes | Overview |
|
||||
| GET | `/static/style.css` | no | Stylesheet |
|
||||
| GET | `/r/<owner>/<name>` | yes | Repo drilldown |
|
||||
| GET | `/r/<owner>/<name>/<index>` | yes | PR drilldown |
|
||||
| GET | `/r/<owner>/<name>/<index>/raw` | yes | Most recent review body as markdown |
|
||||
| POST | `/r/<owner>/<name>/edit` | yes | Edit `.pr-review.json` on the default branch |
|
||||
|
||||
Auth is enforced by oauth2-proxy upstream; the dashboard itself only
|
||||
checks the `X-Forwarded-User` header that oauth2-proxy sets after a
|
||||
successful Logto login + email allowlist match.
|
||||
|
||||
There is no `/health` route — don't add one to the k8s probes without
|
||||
updating `pilot/dashboard.py` (the handler returns 404 on unknown paths,
|
||||
so a probe would loop forever).
|
||||
|
||||
## Mutations flow through Gitea, not local fs
|
||||
|
||||
The edit endpoint reads the current `.pr-review.json` from
|
||||
`GET /repos/{o}/{r}/contents/.pr-review.json?ref=<default-branch>`, lets
|
||||
the operator edit it in a form (validated as JSON, length-capped per
|
||||
field, no schema migration), and PUTs the new content back via the
|
||||
contents API with a commit message like
|
||||
`pragent dashboard: update .pr-review.json`. Every edit is a real Gitea
|
||||
commit on the default branch, attributable to `pragent-bot`, and the
|
||||
next webhook fire picks up the new config — no Pod restart, no image
|
||||
rebuild, no pod-level state.
|
||||
|
||||
The `/data` mount is **read-only** (see the `readOnly: true` on the
|
||||
volumeMount in `~/k8s/pragent-dashboard.yaml`): the dashboard never
|
||||
writes the SQLite file, only the webhook + the daily cronjob do, and
|
||||
keeping it RO means a buggy deploy can't corrupt the harvested feedback.
|
||||
|
||||
## Auth (Logto SSO via oauth2-proxy)
|
||||
|
||||
Authentication is delegated to oauth2-proxy, which fronts the dashboard
|
||||
in-cluster. The dashboard never sees a cookie or a token — it only
|
||||
inspects `X-Forwarded-User` (set by oauth2-proxy after a successful
|
||||
Logto login + email allowlist match). Missing header → 401 with
|
||||
`WWW-Authenticate: Basic realm="pragent-dashboard"`, which lets
|
||||
oauth2-proxy intercept and bounce the browser to Logto.
|
||||
|
||||
Email allowlist lives in the ConfigMap `oauth2-proxy-dashboard-emails`
|
||||
in namespace `pragent`:
|
||||
|
||||
```yaml
|
||||
data:
|
||||
authenticated-emails: |
|
||||
marcos.paulodasilva.mp@gmail.com
|
||||
thiago@marcospaulo.dev.br
|
||||
```
|
||||
|
||||
Edit the ConfigMap to add/remove users; oauth2-proxy hot-reloads the
|
||||
file (it logs `watching ... for updates`), no restart needed. This is
|
||||
the same isolation pattern as the minecraft-sso / code-server
|
||||
allowlists — see `~/.claude/memory/minecraft-sso.md`.
|
||||
|
||||
The Logto app is `pragent-dashboard` (tenant `default`, type
|
||||
`Traditional`), created by direct INSERT into Logto Postgres mirroring
|
||||
the proven `minecraft-sso` pattern. Credentials live in
|
||||
`~/k8s/oauth2-proxy-dashboard-secret.yaml` (mode 600, NOT in git).
|
||||
|
||||
Public URL: **https://pragent-dashboard.marcospaulo.dev.br** (Caddy
|
||||
TLS termination via wildcard cert → Tailscale → NodePort 31541 →
|
||||
oauth2-proxy → dashboard ClusterIP).
|
||||
|
||||
### Emergency bypass (cookie)
|
||||
|
||||
If Logto goes down and you need to access the dashboard before the
|
||||
oauth2-proxy restart dance (see `~/.claude/memory/logto-fix.md`),
|
||||
`pilot/dashboard.py` can be patched to accept a fallback cookie by
|
||||
re-adding the `PRAGENT_DASHBOARD_TOKEN` env path — the route gate is
|
||||
isolated in `_is_authed` and the logic is straightforward. The current
|
||||
commit intentionally has no bypass because Logto SSO is the single
|
||||
source of truth for "who can touch `.pr-review.json`".
|
||||
|
||||
## Deploy
|
||||
|
||||
The dashboard shares the webhook image, so there's nothing to rebuild
|
||||
beyond what the webhook already does. After editing `pilot/dashboard.py`
|
||||
or `pilot/dashboard_data.py`, redo the webhook image rebuild + containerd
|
||||
import (see `README-webhook.md` § "K8s deployment") and roll both
|
||||
deployments.
|
||||
|
||||
```bash
|
||||
K="microk8s kubectl"
|
||||
|
||||
# 1. (one-time) create the Logto app + cookie secret + oauth2-proxy
|
||||
# See ~/.claude/memory/minecraft-sso.md for the SQL INSERT recipe
|
||||
# and ~/k8s/oauth2-proxy-dashboard*.yaml for the manifests.
|
||||
|
||||
# 2. apply all pragent-dashboard manifests (dashboard + oauth2-proxy)
|
||||
$K apply -f ~/k8s/oauth2-proxy-dashboard.yaml
|
||||
$K apply -f ~/k8s/pragent-dashboard.yaml
|
||||
|
||||
# 3. roll on image / code changes
|
||||
$K -n pragent rollout restart deploy/pragent-dashboard
|
||||
$K -n pragent rollout status deploy/pragent-dashboard --timeout=120s
|
||||
$K -n pragent logs -f deploy/pragent-dashboard
|
||||
```
|
||||
|
||||
K8s manifests:
|
||||
|
||||
- `~/k8s/pragent-dashboard.yaml` — Deployment + ClusterIP Service.
|
||||
- `image: pragent-webhook:dashboard` + `imagePullPolicy: Never` —
|
||||
local containerd only, same image as the webhook.
|
||||
- `nodeSelector: kubernetes.io/hostname: kubernets` — pinned to the
|
||||
node holding the `/data` PVC.
|
||||
- `securityContext: runAsNonRoot: true, runAsUser: 10001, runAsGroup:
|
||||
10001, fsGroup: 10001` — matches the image's USER directive;
|
||||
fsGroup makes the RO hostpath volume readable.
|
||||
- `volumeMounts.feedback-data.readOnly: true` — dashboard is
|
||||
read-only over `/data`; mutations go through Gitea, not local fs.
|
||||
- No `readinessProbe` / `livenessProbe` — the dashboard has no
|
||||
`/health` route. If you add one to `pilot/dashboard.py`, add a
|
||||
probe here too.
|
||||
- `resources.requests: {cpu: 100m, memory: 256Mi}` /
|
||||
`limits: {cpu: 500m, memory: 512Mi}` — read-heavy + tiny writes,
|
||||
no opencode subprocess fan-out, much smaller than the webhook.
|
||||
- `Service.type: ClusterIP` — only oauth2-proxy can reach it.
|
||||
|
||||
- `~/k8s/oauth2-proxy-dashboard.yaml` — Deployment + ConfigMap +
|
||||
NodePort Service (`oauth2-proxy-dashboard`, NodePort 31541,
|
||||
namespace `pragent`). Same shape as the code-server /
|
||||
minecraft-sso oauth2-proxy. NodePort 31541 was chosen because
|
||||
31540 was the old dashboard NodePort and the 30096..30969 media
|
||||
range + 30350-30351 (other oauth2-proxy NodePorts) were taken.
|
||||
|
||||
- `~/k8s/oauth2-proxy-dashboard-secret.yaml` — client-id /
|
||||
client-secret / cookie-secret (mode 600, NOT in git).
|
||||
|
||||
## Smoke test
|
||||
|
||||
```bash
|
||||
# 1. anonymous request → 302 redirect to Logto
|
||||
curl -I https://pragent-dashboard.marcospaulo.dev.br/
|
||||
|
||||
# 2. pod logs
|
||||
microk8s kubectl logs -n pragent -l app=oauth2-proxy-dashboard --tail=50
|
||||
microk8s kubectl logs -n pragent -l app=pragent-dashboard --tail=50
|
||||
|
||||
# 3. in-cluster direct probe (should 401 without X-Forwarded-User)
|
||||
microk8s kubectl port-forward -n pragent svc/pragent-dashboard 8181:80 &
|
||||
sleep 2
|
||||
curl -I http://localhost:8181/ # expect 401 + WWW-Authenticate: Basic
|
||||
curl -I -H "X-Forwarded-User: marcos@example.com" http://localhost:8181/ # expect 200
|
||||
kill %1
|
||||
```
|
||||
|
||||
The HTML returned with a valid `X-Forwarded-User` should contain a
|
||||
`<title>` (whatever the dashboard renders) and **never** `Traceback` or
|
||||
any Python exception output. A 401 on the unauthenticated GET is the
|
||||
expected behaviour — oauth2-proxy catches it and redirects to Logto.
|
||||
|
||||
## Threat model / security notes
|
||||
|
||||
- **Behind Logto SSO.** Anonymous traffic gets 302 → Logto. Allowed
|
||||
emails (marcos, thiago) reach the dashboard after Logto login; all
|
||||
others see oauth2-proxy's "not authorized" page. Adding a user is a
|
||||
one-line ConfigMap edit; oauth2-proxy hot-reloads the allowlist.
|
||||
- **`PRAGENT_BOT_TOKEN` is Gitea Write scoped** to onboarded repos, so
|
||||
a successful auth bypass on the dashboard is Gitea repo write access,
|
||||
not just read. oauth2-proxy's email allowlist is the only
|
||||
authentication factor — there is no second factor. If this becomes a
|
||||
concern, swap oauth2-proxy for an IdP that supports TOTP/WebAuthn
|
||||
and the dashboard needs no further changes (it just reads the
|
||||
forwarded headers).
|
||||
- **CSRF on the edit form.** Per-process random secret embedded as a
|
||||
hidden input + double-submit via the `X-Forwarded-User` context. An
|
||||
attacker would need to (a) steal the user's Logto session cookie
|
||||
from oauth2-proxy and (b) read the rendered HTML to harvest the
|
||||
CSRF token. Both have to happen in the same browser.
|
||||
- **Read-only `/data` mount.** The dashboard can't corrupt the
|
||||
harvested SQLite file even if it's compromised. The webhook and the
|
||||
daily cronjob are the only writers.
|
||||
- **ClusterIP dashboard Service.** Even if a malicious actor discovered
|
||||
the dashboard's container port, they cannot reach it from outside the
|
||||
cluster — only oauth2-proxy can. NetworkPolicy is the cluster
|
||||
default deny.
|
||||
- **`uid 10001` + `runAsNonRoot: true`.** No host-level escalation if
|
||||
the dashboard is popped — it has no caps, no `/proc` mounts.
|
||||
- **No author-controlled input is `eval`-ed.** The edit form parses the
|
||||
JSON, validates types / lengths, and re-serialises before the Gitea
|
||||
PUT. The review-side hostile-input concerns from `README-webhook.md`
|
||||
§ "Threat model" do **not** apply to the dashboard — the dashboard
|
||||
is a read-mostly viewer over already-harvested, already-posted data.
|
||||
|
||||
## Known limitations (pilot)
|
||||
|
||||
- Logto SSO is the only auth factor — no per-user sessions, no CSRF
|
||||
token tied to a per-user identity (the per-process CSRF secret is
|
||||
global). Adequate for a single-operator dashboard; not adequate for
|
||||
multi-tenant.
|
||||
- No `/health` route — if the dashboard process wedges on a Gitea hang,
|
||||
k8s won't restart it. Add a `/health` route to `pilot/dashboard.py`
|
||||
+ a probe here before relying on this in production.
|
||||
- The overview is a single-process render over a SQLite file that the
|
||||
daily cronjob also writes. A long Gitea hang during a page render can
|
||||
stall the dashboard until the client request times out (30 s). The
|
||||
underlying SQLite reader is read-only and concurrent-safe, so no
|
||||
data corruption — just a slow page.
|
||||
@@ -21,7 +21,7 @@ and lands on the trace's `environment`:
|
||||
| `headroom/MiniMax-M2.7` | `ollama` |
|
||||
| `vllm-qwen38/qwen3.8-27b` | `ollama` |
|
||||
|
||||
Langfuse takes an environment selector on every view, filter and cost
|
||||
Langfuse takes an environment selector on every dashboard, filter and cost
|
||||
breakdown, so the two spend stories stay separate inside one project — one key
|
||||
pair to rotate instead of two. Tags carry the finer cut:
|
||||
`provider:headroom`, `model:<bare>`, `engine:opencode`, `repo:<owner/name>`,
|
||||
@@ -56,11 +56,11 @@ A model that costs nothing through the headroom proxy is priced against a
|
||||
**comparison target** instead: basis `equivalent:<target>`. That covers the
|
||||
models absent from `PRICES` (`MiniMax-M2.7` — which is what the webhook
|
||||
actually runs — and `glm-5.2:cloud`) as well as entries priced at all zeros
|
||||
(the self-hosted vLLM `qwen3.8-27b`). Without this Langfuse would show a
|
||||
(the self-hosted vLLM `qwen3.8-27b`). Without this the dashboard would be a
|
||||
flat $0.00 line, since the pilot's own path is free.
|
||||
|
||||
The target follows the same precedence as the review body, so the PR and
|
||||
Langfuse never disagree:
|
||||
The target follows the same precedence as the review body, so the PR and the
|
||||
dashboard never disagree:
|
||||
|
||||
.pr-review.json:cost_target > PRAGENT_PRICE_TARGET > claude-sonnet-5
|
||||
|
||||
|
||||
+88
-64
@@ -1,76 +1,100 @@
|
||||
# pragent pilot
|
||||
# pragent pilot — AI Review bot
|
||||
|
||||
The pilot is a central, stdlib-only Gitea webhook service. It reviews opted-in
|
||||
pull requests with an on-network model, posts inline findings, and emits review
|
||||
telemetry to Langfuse. The service is fail-open: a review failure is reported
|
||||
as a PR comment and does not block CI.
|
||||
A minimal AI code-review bot for Gitea, running as a CI step on the existing
|
||||
`act-runner`. This is the **pilot** — a small, self-contained reviewer that
|
||||
predates the full `pragent` framework (whose design lives in
|
||||
`docs/plans/2026-08-04-pragent-design.md`). The framework will later absorb
|
||||
this; until then, this is what runs.
|
||||
|
||||
## Runtime flow
|
||||
## How it works
|
||||
|
||||
1. Gitea sends a signed `pull_request` webhook.
|
||||
2. `webhook_server.py` validates the request, checks the base branch's
|
||||
`.pr-review.json` for `"enabled": true`, and claims `(repo, PR, SHA)`.
|
||||
3. `ai_review.review_pr()` fetches the diff, trusted config, and prior reviews.
|
||||
4. `opencode_review.py` checks out the PR head in a sanitized temporary
|
||||
directory and runs the review agent. The legacy Ollama-compatible path is
|
||||
still available through `PRAGENT_ENGINE`.
|
||||
5. The review output is parsed and normalized, valid post-change line anchors
|
||||
are separated from summary-only findings, and Gitea receives the result.
|
||||
6. `langfuse_trace.py` records usage, cost basis, findings, and evaluation
|
||||
scores when Langfuse credentials are configured.
|
||||
1. You add `pragent-bot` to a repo and commit `.gitea/workflows/ai-review.yml`.
|
||||
2. On a PR, you add the **`AI-REVIEW`** label.
|
||||
3. Gitea Actions runs the workflow on the `act-runner`; it fetches the PR diff,
|
||||
asks `glm-5.2:cloud` (on-network via the headroom proxy) to review it, and
|
||||
posts the findings back as a PR review authored by `pragent-bot`.
|
||||
4. Remove the label to stop re-reviews on further pushes.
|
||||
|
||||
## Module map
|
||||
Fail-open: the job always exits 0 and never blocks CI. Errors become a short
|
||||
"review failed" comment.
|
||||
|
||||
| Module | Responsibility |
|
||||
|---|---|
|
||||
| `webhook_server.py` | HTTP ingress, signature verification, opt-in gate, concurrency |
|
||||
| `review_config.py` | Trusted base-branch opt-in policy; transport injected for tests |
|
||||
| `gitea_client.py` | HTTP transport adapter and repository-scoped client |
|
||||
| `ai_review.py` | Compatibility facade and review orchestration |
|
||||
| `model_client.py` | Anthropic-compatible model adapter and response text extraction |
|
||||
| `opencode_review.py` | Hostile-checkout containment and agent execution |
|
||||
| `diff_compress.py` | Diff compression and prior-review extraction |
|
||||
| `feedback*.py` | Feedback persistence, harvesting, analysis, and Langfuse scores |
|
||||
| `langfuse_trace.py` | Fail-open Langfuse ingestion and cost metadata |
|
||||
| `cost_model.py` | Provider price catalog and equivalent-cost calculations |
|
||||
| `eval_*.py` | Dataset bootstrap, evaluators, and behavioral scoring |
|
||||
## Onboard a repo (3 steps)
|
||||
|
||||
`ai_review.py` remains the stable import surface for existing workflow and
|
||||
webhook deployments. New code should put policy, adapters, and pure transforms
|
||||
in the focused modules above rather than adding unrelated functions there.
|
||||
### 1. Add `pragent-bot` as collaborator
|
||||
|
||||
## Onboard a repository
|
||||
Repo → Settings → Collaborators → Add → `pragent-bot` → permission **Write**.
|
||||
(Write is required to post reviews/comments.)
|
||||
|
||||
1. Add `pragent-bot` as a Write collaborator.
|
||||
2. Commit this file to the default branch:
|
||||
|
||||
```json
|
||||
{"enabled": true}
|
||||
```
|
||||
|
||||
3. Open or update a pull request.
|
||||
|
||||
No per-repository workflow, secret, or label is required for the central
|
||||
webhook path. See [`README-webhook.md`](README-webhook.md) for deployment,
|
||||
security, and webhook registration details.
|
||||
|
||||
## Configuration
|
||||
|
||||
| Variable | Default | Purpose |
|
||||
|---|---:|---|
|
||||
| `GITEA_API` | in-cluster URL | Gitea API base URL |
|
||||
| `PRAGENT_BOT_TOKEN` | — | Bot credential |
|
||||
| `OLLAMA_URL` / `OLLAMA_MODEL` | headroom / `glm-5.2:cloud` | Legacy model path |
|
||||
| `PRAGENT_ENGINE` | `opencode` | `opencode` or legacy model path |
|
||||
| `DIFF_MAX_CHARS` | `150000` | Diff input cap |
|
||||
| `PRAGENT_MAX_CONCURRENT_REVIEWS` | `2` | Process concurrency bound |
|
||||
| `LANGFUSE_HOST` + keys | unset | Enables telemetry; unset is a no-op |
|
||||
|
||||
## Tests
|
||||
Or via API (with an admin/owner token):
|
||||
|
||||
```bash
|
||||
python3 -m pytest tests -q
|
||||
curl -X PUT -H "Authorization: token $OWNER_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"permission":"write"}' \
|
||||
"http://<gitea-host>:3000/api/v1/repos/OWNER/REPO/collaborators/pragent-bot"
|
||||
```
|
||||
|
||||
Tests use mocked transports and local fixtures. They do not require Gitea,
|
||||
Langfuse, a model endpoint, or network access.
|
||||
### 2. Add the `PRAGENT_BOT_TOKEN` secret
|
||||
|
||||
Repo → Settings → Actions → Secrets → New secret → name `PRAGENT_BOT_TOKEN`,
|
||||
value = the bot's access token (ask the platform admin; stored mode-600 at
|
||||
`~/.claude/.pragent-bot-token` on the admin host).
|
||||
|
||||
### 3. Commit the workflow
|
||||
|
||||
Copy `pilot/workflow-template.yml` into the target repo as
|
||||
`.gitea/workflows/ai-review.yml` and commit it. That's it.
|
||||
|
||||
## Use it
|
||||
|
||||
Open a PR (or push to an open one), add the **`AI-REVIEW`** label. The review
|
||||
appears within ~30–90s depending on diff size and model latency.
|
||||
|
||||
## What's intentionally NOT in the pilot
|
||||
|
||||
Deferred to the full framework (by design, see the design doc):
|
||||
|
||||
- Attention tiering (trivial/lite/full/oversized) and per-tier cost control.
|
||||
- Multiple analyzer fan-out over a shared cached prompt prefix.
|
||||
- Prior-comment synthesis (so each push re-posts; the latest review is tagged
|
||||
with the head SHA so it's easy to spot).
|
||||
- Inline line comments and status checks.
|
||||
- `pragent explain` / `replay` / analytics JSONL.
|
||||
- A second forge (GitLab) and the provider matrix.
|
||||
|
||||
## Pieces
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `pilot/ai_review.py` | The reviewer script (stdlib only). Single source of truth — fetched at runtime by each repo's workflow. |
|
||||
| `pilot/workflow-template.yml` | The Gitea Action consumers copy into `.gitea/workflows/ai-review.yml`. |
|
||||
| `tests/pilot/test_ai_review.py` | Unit tests for the pure helpers (no network). |
|
||||
|
||||
## Run the tests
|
||||
|
||||
```bash
|
||||
cd ~/Projects/pragent
|
||||
PYTHONPATH=pilot python3 -m pytest tests/pilot/ # if pytest available
|
||||
# or, without pytest:
|
||||
python3 - <<'PY'
|
||||
import os, sys, importlib.util
|
||||
sys.path.insert(0, os.path.abspath("pilot"))
|
||||
import ai_review # noqa: F401
|
||||
spec = importlib.util.spec_from_file_location("t", "tests/pilot/test_ai_review.py")
|
||||
m = importlib.util.module_from_spec(spec); spec.loader.exec_module(m)
|
||||
fails = 0
|
||||
for n in sorted(x for x in dir(m) if x.startswith("test_")):
|
||||
try: getattr(m, n)(); print("PASS", n)
|
||||
except Exception as e: fails += 1; print("FAIL", n, e)
|
||||
print("failed:", fails)
|
||||
PY
|
||||
```
|
||||
|
||||
## Configuration knobs (env in the workflow)
|
||||
|
||||
| Env | Default | Purpose |
|
||||
|---|---|---|
|
||||
| `OLLAMA_MODEL` | `glm-5.2:cloud` | Model id passed to the headroom proxy. |
|
||||
| `OLLAMA_MAX_TOKENS` | `6000` | Output token cap. |
|
||||
| `DIFF_MAX_CHARS` | `150000` | Diff truncation cap (with a noted truncation marker). |
|
||||
| `OLLAMA_URL` | `http://<model-proxy-host>:8789` | headroom proxy (tailnet). If the act-runner can't reach the tailnet IP, expose 8789 as an in-cluster Service+Endpoints and set this to the cluster DNS name. |
|
||||
+38
-6
@@ -185,8 +185,15 @@ def parse_text_blocks(content: list) -> str:
|
||||
Drops `thinking` blocks (glm-5.2:cloud is a reasoning model and emits them).
|
||||
Tolerates missing/malformed blocks by skipping them.
|
||||
"""
|
||||
from model_client import parse_text_blocks as _parse_text_blocks
|
||||
return _parse_text_blocks(content)
|
||||
if not isinstance(content, list):
|
||||
return ""
|
||||
out = []
|
||||
for block in content:
|
||||
if not isinstance(block, dict):
|
||||
continue
|
||||
if block.get("type") == "text" and isinstance(block.get("text"), str):
|
||||
out.append(block["text"])
|
||||
return "\n".join(out).strip()
|
||||
|
||||
|
||||
def _int_env(name: str, default: int) -> int:
|
||||
@@ -1796,8 +1803,19 @@ def compact_prior_reviews(prior_bodies: list[str]) -> list[str]:
|
||||
|
||||
|
||||
def _http(method: str, url: str, token: str, body: dict | None = None, accept: str = "application/json") -> tuple[int, bytes]:
|
||||
from gitea_client import request
|
||||
return request(method, url, token, body, accept)
|
||||
headers = {"Authorization": f"token {token}", "Accept": accept}
|
||||
data = None
|
||||
if body is not None:
|
||||
data = json.dumps(body).encode()
|
||||
headers["Content-Type"] = "application/json"
|
||||
req = urllib.request.Request(url, data=data, headers=headers, method=method)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=180) as r:
|
||||
return r.status, r.read()
|
||||
except urllib.error.HTTPError as e:
|
||||
return e.code, e.read()
|
||||
except urllib.error.URLError as e:
|
||||
raise RuntimeError(f"network error: {e.reason}") from e
|
||||
|
||||
|
||||
def gitea_get(api: str, repo: str, path: str, token: str, accept: str = "application/json") -> tuple[int, bytes]:
|
||||
@@ -1990,8 +2008,22 @@ def fetch_repo_config(api: str, repo: str, token: str, ref: str = "") -> dict:
|
||||
|
||||
|
||||
def call_model(ollama_url: str, model: str, system: str, user: str, max_tokens: int) -> str:
|
||||
from model_client import complete
|
||||
return complete(ollama_url, model, system, user, max_tokens)
|
||||
payload = {
|
||||
"model": model,
|
||||
"max_tokens": max_tokens,
|
||||
"system": system,
|
||||
"messages": [{"role": "user", "content": user}],
|
||||
}
|
||||
status, raw = _http(
|
||||
"POST",
|
||||
f"{ollama_url.rstrip('/')}/v1/messages",
|
||||
"ollama", # headroom ollama hub uses x-api-key: ollama
|
||||
payload,
|
||||
)
|
||||
if status != 200:
|
||||
raise RuntimeError(f"model call failed: HTTP {status}: {raw[:500].decode('utf-8', errors='replace')}")
|
||||
data = json.loads(raw)
|
||||
return parse_text_blocks(data.get("content", []))
|
||||
|
||||
|
||||
def post_review(api: str, repo: str, index: str, token: str, body: str) -> None:
|
||||
|
||||
@@ -0,0 +1,754 @@
|
||||
#!/usr/bin/env python3
|
||||
"""pragent pilot — read-mostly dashboard.
|
||||
|
||||
Stdlib HTTP server (mirrors `webhook_server.py`'s BaseHTTPRequestHandler +
|
||||
ThreadingHTTPServer shape) that renders three views off the feedback SQLite:
|
||||
|
||||
GET / overview
|
||||
GET /r/<owner>/<name> repo summary + edit form
|
||||
GET /r/<owner>/<name>/<index> one PR's findings
|
||||
GET /r/<owner>/<name>/<index>/raw raw Markdown body (via Gitea)
|
||||
GET /static/style.css CSS
|
||||
POST /r/<owner>/<name>/edit mutate .pr-review.json (Tasks C+D)
|
||||
|
||||
Auth: oauth2-proxy fronts this service in-cluster. Every route except
|
||||
`/static/*` requires the `X-Forwarded-User` header (set by oauth2-proxy
|
||||
once the user has logged in via Logto). Missing header → 401 +
|
||||
`WWW-Authenticate: Basic realm="pragent-dashboard"` so oauth2-proxy
|
||||
intercepts the response.
|
||||
|
||||
DB: `PRAGENT_FEEDBACK_DB` points at the SQLite file the webhook server
|
||||
also writes. Per-request open (SQLite is cheap, no concurrency hazard,
|
||||
no stale-conn surprise after the file rotates).
|
||||
|
||||
All HTML is rendered via `string.Template` and every dynamic value is
|
||||
escaped with `html.escape(..., quote=True)`. No `.format`, no f-string
|
||||
templates — see `_render_*` for the discipline.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import datetime
|
||||
import html
|
||||
import json
|
||||
import os
|
||||
import secrets
|
||||
import string
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
|
||||
from pilot import dashboard_data
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Config
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
FEEDBACK_DB = "" # legacy; readers should call _feedback_db()
|
||||
PORT = int(os.environ.get("DASHBOARD_PORT", "8081"))
|
||||
|
||||
GITEA_API = "" # legacy; readers should call _gitea_api()
|
||||
BOT_TOKEN = "" # legacy; readers should call _bot_token()
|
||||
|
||||
# CSRF secret for the edit form. Regenerated per process (each Python
|
||||
# interpreter launch). Behind oauth2-proxy this is enough — only an
|
||||
# already-authenticated same-tab request can read this and echo it back.
|
||||
_CSRF_SECRET: str = secrets.token_urlsafe(24)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Lazy config readers — tests set env after import, so each request re-reads.
|
||||
# Production: env is fixed for the process lifetime; the per-request lookup is
|
||||
# a dict access, not a syscall.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _feedback_db() -> str:
|
||||
return os.environ.get("PRAGENT_FEEDBACK_DB", "")
|
||||
|
||||
|
||||
def _bot_token() -> str:
|
||||
return os.environ.get("PRAGENT_BOT_TOKEN", "")
|
||||
|
||||
|
||||
def _gitea_api() -> str:
|
||||
return os.environ.get("GITEA_API", "http://gitea-http.gitea.svc.cluster.local:3000")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Stylesheet — small, dark-mode-friendly, deliberately under 100 lines
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
STYLE_CSS = """
|
||||
:root { color-scheme: light dark; }
|
||||
* { box-sizing: border-box; }
|
||||
body {
|
||||
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", system-ui, sans-serif;
|
||||
margin: 0; padding: 0;
|
||||
background: #0f1115; color: #e6e6e6;
|
||||
line-height: 1.5;
|
||||
}
|
||||
header {
|
||||
background: #1a1d23; padding: 12px 20px;
|
||||
border-bottom: 1px solid #2a2f38;
|
||||
display: flex; align-items: center; gap: 18px;
|
||||
}
|
||||
header h1 { font-size: 18px; margin: 0; }
|
||||
header nav a {
|
||||
color: #8ab4f8; text-decoration: none; margin-right: 12px;
|
||||
}
|
||||
header nav a:hover { text-decoration: underline; }
|
||||
main { padding: 20px; max-width: 1100px; margin: 0 auto; }
|
||||
h2 { margin-top: 24px; font-size: 16px; color: #c9d1d9; }
|
||||
.metric-row { display: flex; gap: 16px; flex-wrap: wrap; margin-bottom: 16px; }
|
||||
.metric {
|
||||
background: #1a1d23; padding: 14px 18px; border-radius: 8px;
|
||||
min-width: 140px; border: 1px solid #2a2f38;
|
||||
}
|
||||
.metric .v { font-size: 28px; font-weight: 600; }
|
||||
.metric .l { font-size: 12px; color: #8b949e; text-transform: uppercase; letter-spacing: 0.04em; }
|
||||
table { width: 100%; border-collapse: collapse; margin: 8px 0 16px; font-size: 14px; }
|
||||
th, td { text-align: left; padding: 6px 10px; border-bottom: 1px solid #2a2f38; }
|
||||
th { color: #8b949e; font-weight: 500; text-transform: uppercase; font-size: 11px; letter-spacing: 0.04em; }
|
||||
tr:hover td { background: #161922; }
|
||||
.sev-critical { color: #ff7b72; font-weight: 600; }
|
||||
.sev-high { color: #f0883e; }
|
||||
.sev-medium { color: #d29922; }
|
||||
.sev-low { color: #8b949e; }
|
||||
.muted { color: #8b949e; font-size: 12px; }
|
||||
.sparkline { font-family: ui-monospace, "SF Mono", monospace; letter-spacing: 1px; }
|
||||
form { background: #1a1d23; padding: 14px 18px; border-radius: 8px; border: 1px solid #2a2f38; margin: 12px 0; }
|
||||
form label { display: block; margin: 8px 0 4px; color: #c9d1d9; font-size: 13px; }
|
||||
form input[type=text], form textarea, form select {
|
||||
background: #0f1115; color: #e6e6e6; border: 1px solid #2a2f38;
|
||||
border-radius: 4px; padding: 6px 8px; font-family: inherit; font-size: 14px;
|
||||
width: 100%;
|
||||
}
|
||||
form textarea { min-height: 80px; }
|
||||
form .row { display: flex; gap: 8px; align-items: center; margin-top: 12px; }
|
||||
form button {
|
||||
background: #2ea043; color: white; border: none; border-radius: 4px;
|
||||
padding: 6px 14px; font-size: 14px; cursor: pointer;
|
||||
}
|
||||
form button:hover { background: #3fb950; }
|
||||
.flash { background: #3d1e1e; color: #ff7b72; padding: 8px 12px; border-radius: 4px; margin-bottom: 12px; }
|
||||
code { background: #161922; padding: 1px 4px; border-radius: 3px; font-size: 13px; }
|
||||
pre { background: #161922; padding: 12px; border-radius: 6px; overflow-x: auto; }
|
||||
"""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Templates — string.Template so dynamic values are always escaped explicitly
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_BASE = string.Template("""<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<title>${title}</title>
|
||||
<link rel="stylesheet" href="/static/style.css">
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<h1>pragent dashboard</h1>
|
||||
<nav>
|
||||
<a href="/">Home</a>
|
||||
<a href="/r/${repos_first}">repos</a>
|
||||
</nav>
|
||||
<span class="muted" style="margin-left:auto">${db_status}</span>
|
||||
</header>
|
||||
<main>
|
||||
${body}
|
||||
</main>
|
||||
</body>
|
||||
</html>""")
|
||||
|
||||
|
||||
_OVERVIEW = string.Template("""<h2>Overview</h2>
|
||||
<div class="metric-row">
|
||||
<div class="metric"><div class="v">${total_reviews}</div><div class="l">reviews</div></div>
|
||||
<div class="metric"><div class="v">${total_findings}</div><div class="l">findings</div></div>
|
||||
<div class="metric"><div class="v">${total_repos}</div><div class="l">repos</div></div>
|
||||
<div class="metric"><div class="v">${last_30d_reviews}</div><div class="l">last 30d</div></div>
|
||||
</div>
|
||||
|
||||
<h2>Last 7 days</h2>
|
||||
<div class="sparkline">${sparkline}</div>
|
||||
<div class="muted">total cost: $${total_cost_usd} — no per-review cost logged</div>
|
||||
|
||||
<h2>Top repos</h2>
|
||||
${top_repos_table}
|
||||
""")
|
||||
|
||||
|
||||
_REPO = string.Template("""<h2>Repo: <code>${repo}</code></h2>
|
||||
<div class="metric-row">
|
||||
<div class="metric"><div class="v">${total_runs}</div><div class="l">runs</div></div>
|
||||
<div class="metric"><div class="v">${sev_critical}</div><div class="l sev-critical">critical</div></div>
|
||||
<div class="metric"><div class="v">${sev_high}</div><div class="l sev-high">high</div></div>
|
||||
<div class="metric"><div class="v">${sev_medium}</div><div class="l sev-medium">medium</div></div>
|
||||
<div class="metric"><div class="v">${sev_low}</div><div class="l sev-low">low</div></div>
|
||||
</div>
|
||||
|
||||
<h2>Edit .pr-review.json</h2>
|
||||
${flash}
|
||||
<form method="post" action="/r/${repo_url}/edit">
|
||||
<input type="hidden" name="_csrf" value="${csrf}">
|
||||
<label for="static_message">Static banner message (max 400 chars)</label>
|
||||
<textarea id="static_message" name="static_message" maxlength="400">${current_static_message}</textarea>
|
||||
<label for="model">Model (PRICES keys)</label>
|
||||
<select id="model" name="model">${model_options}</select>
|
||||
<div class="row">
|
||||
<button type="submit">Save</button>
|
||||
<span class="muted">posted via the bot identity; one commit on the base branch</span>
|
||||
</div>
|
||||
</form>
|
||||
|
||||
<h2>Top findings (by occurrence)</h2>
|
||||
${top_findings_table}
|
||||
|
||||
<h2>Runs by day (last 30d)</h2>
|
||||
${runs_by_day_table}
|
||||
|
||||
<h2>Reviews</h2>
|
||||
${reviews_table}
|
||||
""")
|
||||
|
||||
|
||||
_PR = string.Template("""<h2>PR <code>${repo}</code> #${pr}</h2>
|
||||
<div class="muted">head sha: <code>${head_sha}</code></div>
|
||||
<div class="muted">posted_at: ${posted_at_iso}</div>
|
||||
<div class="muted">review_id_gitea: ${review_id_gitea} · body_comment_id: ${body_comment_id}</div>
|
||||
|
||||
<h2>Findings</h2>
|
||||
${findings_table}
|
||||
|
||||
<p><a href="/r/${repo_url}/${pr}/raw">raw review body (Markdown)</a></p>
|
||||
""")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Small helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _esc(s) -> str:
|
||||
"""HTML-escape any value to a string."""
|
||||
return html.escape(str(s), quote=True)
|
||||
|
||||
|
||||
def _ts_iso(ts: int) -> str:
|
||||
if not ts:
|
||||
return "—"
|
||||
return datetime.datetime.fromtimestamp(int(ts), tz=datetime.timezone.utc).isoformat()
|
||||
|
||||
|
||||
def _sparkline(buckets: list[dict]) -> str:
|
||||
"""7-bucket sparkline as unicode bars."""
|
||||
bars = "▁▂▃▄▅▆▇█"
|
||||
if not buckets:
|
||||
return ""
|
||||
mx = max((b.get("count", 0) for b in buckets), default=0) or 1
|
||||
out = []
|
||||
for b in buckets:
|
||||
n = b.get("count", 0)
|
||||
idx = min(len(bars) - 1, int(round(n / mx * (len(bars) - 1))))
|
||||
out.append(bars[idx])
|
||||
return "".join(out)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Renderers — one per page
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _overview_body(data: dict) -> str:
|
||||
top_rows = "".join(
|
||||
f"<tr><td><a href=\"/r/{_esc(r['repo'])}\">{_esc(r['repo'])}</a></td>"
|
||||
f"<td>{int(r['run_count'])}</td>"
|
||||
f"<td class=\"muted\">{_ts_iso(int(r['last_seen']))}</td></tr>"
|
||||
for r in data.get("top_repos", [])
|
||||
) or "<tr><td class=\"muted\">no reviews yet</td></tr>"
|
||||
top_table = f"<table><thead><tr><th>repo</th><th>runs</th><th>last seen</th></tr></thead><tbody>{top_rows}</tbody></table>"
|
||||
return _OVERVIEW.substitute(
|
||||
total_reviews=_esc(data.get("total_reviews", 0)),
|
||||
total_findings=_esc(data.get("total_findings", 0)),
|
||||
total_repos=_esc(data.get("total_repos", 0)),
|
||||
last_30d_reviews=_esc(data.get("last_30d_reviews", 0)),
|
||||
sparkline=_esc(_sparkline(data.get("daily", []))),
|
||||
total_cost_usd=f"{float(data.get('total_cost_usd', 0.0)):.2f}",
|
||||
top_repos_table=top_table,
|
||||
)
|
||||
|
||||
|
||||
def _repo_body(data: dict, *, repo_url: str, csrf: str, current_model: str,
|
||||
current_static_message: str, flash: str = "") -> str:
|
||||
fbs = data.get("findings_by_severity", {})
|
||||
tf = data.get("top_findings", [])
|
||||
|
||||
# Top findings table.
|
||||
if tf:
|
||||
rows = "".join(
|
||||
f"<tr><td><code>{_esc(f['path'])}:{_esc(f['line'])}</code></td>"
|
||||
f"<td class=\"sev-{_esc(f.get('severity', 'low').lower())}\">{_esc(f.get('severity', ''))}</td>"
|
||||
f"<td>{_esc(f.get('problem', ''))}</td>"
|
||||
f"<td>{int(f.get('occurrences', 0))}</td>"
|
||||
f"<td>+{int(f.get('upvotes', 0))} / -{int(f.get('downvotes', 0))}</td>"
|
||||
f"<td>{'resolved' if int(f.get('resolved', 0)) else 'open'}</td>"
|
||||
f"<td>{int(f.get('reply_count', 0))}</td></tr>"
|
||||
for f in tf
|
||||
)
|
||||
top_findings_table = (
|
||||
"<table><thead><tr><th>location</th><th>severity</th>"
|
||||
"<th>problem</th><th>occurrences</th><th>votes</th>"
|
||||
"<th>state</th><th>replies</th></tr></thead><tbody>"
|
||||
f"{rows}</tbody></table>"
|
||||
)
|
||||
else:
|
||||
top_findings_table = "<p class=\"muted\">no findings yet</p>"
|
||||
|
||||
# Runs by day.
|
||||
runs = data.get("runs_by_day", [])
|
||||
if runs:
|
||||
rows = "".join(
|
||||
f"<tr><td>{_esc(r['date'])}</td><td>{int(r.get('count', 0))}</td></tr>"
|
||||
for r in runs
|
||||
)
|
||||
runs_by_day_table = (
|
||||
"<table><thead><tr><th>date</th><th>runs</th></tr></thead>"
|
||||
f"<tbody>{rows}</tbody></table>"
|
||||
)
|
||||
else:
|
||||
runs_by_day_table = "<p class=\"muted\">no runs in the last 30 days</p>"
|
||||
|
||||
# Reviews list — derived from finding timestamps; cheap because we
|
||||
# just enumerate the repo's review rows.
|
||||
reviews_table = _repo_reviews_table(repo_url, data.get("recent_reviews", []))
|
||||
|
||||
# Model select (Task D) — sorted PRICES keys + "keep current".
|
||||
from cost_model import PRICES # local: pilot-only dep
|
||||
model_options = (
|
||||
f"<option value=\"\">— keep current ({_esc(current_model or 'unset')}) —</option>"
|
||||
+ "".join(
|
||||
f"<option value=\"{_esc(k)}\" {'selected' if k == current_model else ''}>{_esc(k)}</option>"
|
||||
for k in sorted(PRICES)
|
||||
)
|
||||
)
|
||||
|
||||
return _REPO.substitute(
|
||||
repo=_esc(data.get("repo", "")),
|
||||
repo_url=_esc(repo_url),
|
||||
total_runs=_esc(data.get("total_runs", 0)),
|
||||
sev_critical=_esc(fbs.get("critical", 0)),
|
||||
sev_high=_esc(fbs.get("high", 0)),
|
||||
sev_medium=_esc(fbs.get("medium", 0)),
|
||||
sev_low=_esc(fbs.get("low", 0)),
|
||||
csrf=_esc(csrf),
|
||||
current_static_message=_esc(current_static_message),
|
||||
model_options=model_options,
|
||||
flash=_esc(flash),
|
||||
top_findings_table=top_findings_table,
|
||||
runs_by_day_table=runs_by_day_table,
|
||||
reviews_table=reviews_table,
|
||||
)
|
||||
|
||||
|
||||
def _repo_reviews_table(repo_url: str, rows: list[dict]) -> str:
|
||||
if not rows:
|
||||
return "<p class=\"muted\">no reviews yet</p>"
|
||||
out = "<table><thead><tr><th>PR</th><th>head sha</th><th>posted</th></tr></thead><tbody>"
|
||||
for r in rows:
|
||||
out += (
|
||||
f"<tr><td><a href=\"/r/{_esc(repo_url)}/{int(r['pr'])}\">#{int(r['pr'])}</a></td>"
|
||||
f"<td><code>{_esc(r['head_sha'][:10])}</code></td>"
|
||||
f"<td class=\"muted\">{_ts_iso(int(r.get('posted_at', 0)))}</td></tr>"
|
||||
)
|
||||
out += "</tbody></table>"
|
||||
return out
|
||||
|
||||
|
||||
def _pr_body(data: dict, *, repo_url: str) -> str:
|
||||
findings = data.get("findings", [])
|
||||
if findings:
|
||||
rows = "".join(
|
||||
f"<tr><td><code>{_esc(f['path'])}:{_esc(f['line'])}</code></td>"
|
||||
f"<td class=\"sev-{_esc(f.get('severity', 'low').lower())}\">{_esc(f.get('severity', ''))}</td>"
|
||||
f"<td>{_esc(f.get('problem', ''))}</td>"
|
||||
f"<td>{_esc(f.get('fix', ''))}</td>"
|
||||
f"<td>{_esc(f.get('suggestion', ''))}</td>"
|
||||
f"<td>+{int(f.get('upvotes', 0))} / -{int(f.get('downvotes', 0))}</td>"
|
||||
f"<td>{'resolved' if int(f.get('resolved', 0)) else 'open'}</td>"
|
||||
f"<td>{int(f.get('reply_count', 0))}</td></tr>"
|
||||
for f in findings
|
||||
)
|
||||
findings_table = (
|
||||
"<table><thead><tr><th>location</th><th>severity</th>"
|
||||
"<th>problem</th><th>fix</th><th>suggestion</th>"
|
||||
"<th>votes</th><th>state</th><th>replies</th></tr></thead>"
|
||||
f"<tbody>{rows}</tbody></table>"
|
||||
)
|
||||
else:
|
||||
findings_table = "<p class=\"muted\">no findings</p>"
|
||||
|
||||
return _PR.substitute(
|
||||
repo=_esc(data.get("repo", "")),
|
||||
repo_url=_esc(repo_url),
|
||||
pr=_esc(data.get("pr", 0)),
|
||||
head_sha=_esc(data.get("head_sha", "")),
|
||||
posted_at_iso=_ts_iso(int(data.get("posted_at", 0))),
|
||||
review_id_gitea=_esc(data.get("review_id_gitea", "") or "—"),
|
||||
body_comment_id=_esc(data.get("body_comment_id", "") or "—"),
|
||||
findings_table=findings_table,
|
||||
)
|
||||
|
||||
|
||||
def _page(title: str, body: str, *, repos_first: str = "") -> str:
|
||||
db_status = _feedback_db() or "(no DB configured)"
|
||||
return _BASE.substitute(
|
||||
title=_esc(title),
|
||||
body=body,
|
||||
repos_first=_esc(repos_first),
|
||||
db_status=_esc(db_status),
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Gitea HTTP helper — minimal, used by the raw body fetch and the edit endpoint
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _http(method: str, url: str, *, token: str = "", body: dict | None = None,
|
||||
raw_body: bytes | None = None) -> tuple[int, bytes]:
|
||||
"""Like ai_review._http but local: this module is stdlib-only and doesn't
|
||||
depend on the ai_review import (which pulls in a 1700-line reviewer)."""
|
||||
headers = {"Accept": "application/json"}
|
||||
data: bytes | None = None
|
||||
if raw_body is not None:
|
||||
data = raw_body
|
||||
headers["Content-Type"] = "application/json"
|
||||
elif body is not None:
|
||||
data = json.dumps(body).encode()
|
||||
headers["Content-Type"] = "application/json"
|
||||
if token:
|
||||
headers["Authorization"] = f"token {token}"
|
||||
req = urllib.request.Request(url, data=data, headers=headers, method=method)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=30) as r:
|
||||
return r.status, r.read()
|
||||
except urllib.error.HTTPError as e:
|
||||
return e.code, e.read()
|
||||
except urllib.error.URLError as e:
|
||||
raise RuntimeError(f"network error: {e.reason}") from e
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Auth
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _is_authed(headers) -> bool:
|
||||
"""True when oauth2-proxy forwarded a verified user.
|
||||
|
||||
oauth2-proxy sets `X-Forwarded-User` (and friends) only after a
|
||||
successful Logto login + email allowlist check. Unauthenticated
|
||||
requests never see the header, so the dashboard never has to know
|
||||
about cookies, secrets, or Logto's token shape.
|
||||
"""
|
||||
return bool((headers.get("X-Forwarded-User") or "").strip())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Routes
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _route_overview() -> bytes:
|
||||
data = dashboard_data.overview(_feedback_db())
|
||||
body = _overview_body(data)
|
||||
# nav: first repo if any
|
||||
repos_first = ""
|
||||
if data.get("top_repos"):
|
||||
repos_first = data["top_repos"][0]["repo"]
|
||||
return _page("Overview", body, repos_first=repos_first).encode()
|
||||
|
||||
|
||||
def _route_repo(owner: str, name: str) -> bytes:
|
||||
repo_url = f"{owner}/{name}"
|
||||
data = dashboard_data.repo_summary(_feedback_db(), repo_url)
|
||||
# Pull current .pr-review.json (best-effort) so the form fields prefill.
|
||||
current_static_message, current_model, flash = "", "", ""
|
||||
cfg, err = _fetch_pr_review_json(repo_url)
|
||||
if cfg:
|
||||
current_static_message = cfg.get("static_message", "")
|
||||
current_model = cfg.get("model", "")
|
||||
elif err and err != "404":
|
||||
flash = f"could not read .pr-review.json: {err}"
|
||||
body = _repo_body(
|
||||
data,
|
||||
repo_url=repo_url,
|
||||
csrf=_CSRF_SECRET,
|
||||
current_model=current_model,
|
||||
current_static_message=current_static_message,
|
||||
flash=flash,
|
||||
)
|
||||
return _page(f"repo {repo_url}", body, repos_first=repo_url).encode()
|
||||
|
||||
|
||||
def _route_pr(owner: str, name: str, index: int) -> bytes:
|
||||
repo_url = f"{owner}/{name}"
|
||||
data = dashboard_data.pr_summary(_feedback_db(), repo_url, int(index))
|
||||
body = _pr_body(data, repo_url=repo_url)
|
||||
return _page(f"PR {repo_url}#{index}", body, repos_first=repo_url).encode()
|
||||
|
||||
|
||||
def _route_pr_raw(owner: str, name: str, index: int) -> tuple[int, bytes]:
|
||||
repo_url = f"{owner}/{name}"
|
||||
data = dashboard_data.pr_summary(_feedback_db(), repo_url, int(index))
|
||||
body_comment_id = data.get("body_comment_id")
|
||||
if not body_comment_id:
|
||||
return 404, b"no body_comment_id"
|
||||
status, raw = _http(
|
||||
"GET",
|
||||
f"{_gitea_api()}/api/v1/repos/{repo_url}/issues/{index}/comments/{body_comment_id}",
|
||||
token=_bot_token(),
|
||||
)
|
||||
if status != 200:
|
||||
return 404, f"Gitea returned {status}".encode()
|
||||
try:
|
||||
parsed = json.loads(raw)
|
||||
md = parsed.get("body", "")
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
return 404, b"could not parse Gitea response"
|
||||
return 200, md.encode()
|
||||
|
||||
|
||||
def _route_static_css() -> bytes:
|
||||
return STYLE_CSS.encode()
|
||||
|
||||
|
||||
def _route_edit(owner: str, name: str, form: dict) -> tuple[int, dict, bytes]:
|
||||
"""Mutate .pr-review.json via the Gitea contents API (Tasks C+D)."""
|
||||
repo_url = f"{owner}/{name}"
|
||||
csrf = form.get("_csrf", "")
|
||||
if csrf != _CSRF_SECRET:
|
||||
return 302, {"Location": f"/r/{repo_url}"}, b""
|
||||
static_message = (form.get("static_message") or "").strip()[:400]
|
||||
model = (form.get("model") or "").strip()
|
||||
|
||||
# Validate model against PRICES.
|
||||
from cost_model import PRICES
|
||||
if model and model not in PRICES:
|
||||
flash = urllib.parse.quote(f"unknown model {model!r}; not saved")
|
||||
return 302, {"Location": f"/r/{repo_url}?flash={flash}"}, b""
|
||||
|
||||
cfg, err = _fetch_pr_review_json(repo_url)
|
||||
if err and err != "404":
|
||||
flash = urllib.parse.quote(f"could not read .pr-review.json: {err}")
|
||||
return 302, {"Location": f"/r/{repo_url}?flash={flash}"}, b""
|
||||
if cfg is None:
|
||||
cfg = {}
|
||||
|
||||
if static_message:
|
||||
cfg["static_message"] = static_message
|
||||
elif "static_message" in cfg and not static_message:
|
||||
# Empty submission clears the banner.
|
||||
del cfg["static_message"]
|
||||
if model:
|
||||
cfg["model"] = model
|
||||
elif "model" in cfg and not model:
|
||||
del cfg["model"]
|
||||
|
||||
payload = json.dumps(cfg, indent=2, sort_keys=True).encode()
|
||||
b64 = base64.b64encode(payload).decode()
|
||||
body = {"content": b64, "message": "pragent dashboard: update .pr-review.json"}
|
||||
if err == "404":
|
||||
# File didn't exist — Gitea contents PUT still creates the file when
|
||||
# `sha` is omitted, but only on certain versions; passing sha=None is
|
||||
# safer.
|
||||
pass
|
||||
else:
|
||||
# GET returned a sha — include it so Gitea enforces optimistic lock.
|
||||
# The sha lives in cfg's wrapper: re-fetch once to capture it.
|
||||
_, raw = _http(
|
||||
"GET",
|
||||
f"{_gitea_api()}/api/v1/repos/{repo_url}/contents/.pr-review.json",
|
||||
token=_bot_token(),
|
||||
)
|
||||
try:
|
||||
existing = json.loads(raw)
|
||||
sha = existing.get("sha")
|
||||
if sha:
|
||||
body["sha"] = sha
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
pass
|
||||
|
||||
status, _ = _http(
|
||||
"PUT",
|
||||
f"{_gitea_api()}/api/v1/repos/{repo_url}/contents/.pr-review.json",
|
||||
token=_bot_token(),
|
||||
body=body,
|
||||
)
|
||||
if status not in (200, 201):
|
||||
flash = urllib.parse.quote(f"Gitea PUT failed: status {status}")
|
||||
return 302, {"Location": f"/r/{repo_url}?flash={flash}"}, b""
|
||||
return 302, {"Location": f"/r/{repo_url}"}, b""
|
||||
|
||||
|
||||
def _fetch_pr_review_json(repo_url: str) -> tuple[dict | None, str | None]:
|
||||
"""Return (cfg, None) on success, (None, None) when the file doesn't exist,
|
||||
(None, 'reason') on error."""
|
||||
if not _bot_token():
|
||||
return None, "PRAGENT_BOT_TOKEN not set"
|
||||
status, raw = _http(
|
||||
"GET",
|
||||
f"{_gitea_api()}/api/v1/repos/{repo_url}/contents/.pr-review.json",
|
||||
token=_bot_token(),
|
||||
)
|
||||
if status == 404:
|
||||
return None, "404"
|
||||
if status != 200:
|
||||
return None, f"status {status}"
|
||||
try:
|
||||
wrapper = json.loads(raw)
|
||||
content_b64 = wrapper.get("content", "").replace("\n", "")
|
||||
decoded = base64.b64decode(content_b64).decode("utf-8", errors="replace")
|
||||
cfg = json.loads(decoded)
|
||||
except (json.JSONDecodeError, ValueError) as e:
|
||||
return None, f"parse error: {e}"
|
||||
if not isinstance(cfg, dict):
|
||||
return None, "not a JSON object"
|
||||
return cfg, None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Handler
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def _send(self, status: int, body: bytes, *, content_type: str = "text/html; charset=utf-8",
|
||||
extra_headers: dict | None = None) -> None:
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", content_type)
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
if extra_headers:
|
||||
for k, v in extra_headers.items():
|
||||
self.send_header(k, v)
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def _redirect(self, location: str) -> None:
|
||||
body = b""
|
||||
self.send_response(302)
|
||||
self.send_header("Location", location)
|
||||
self.send_header("Content-Length", "0")
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def _unauthorized(self) -> None:
|
||||
"""401 + Basic challenge so oauth2-proxy intercepts and redirects to Logto."""
|
||||
body = b"unauthorized\n"
|
||||
self.send_response(401)
|
||||
self.send_header("Content-Type", "text/plain; charset=utf-8")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.send_header("WWW-Authenticate", 'Basic realm="pragent-dashboard"')
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
# --- GET -----------------------------------------------------------------
|
||||
|
||||
def do_GET(self):
|
||||
path = self.path
|
||||
# Static is exempt from auth (also unauthenticated browser fingerprinting
|
||||
# noise, but it's the same CSS regardless of viewer).
|
||||
if path == "/static/style.css":
|
||||
self._send(200, _route_static_css(), content_type="text/css; charset=utf-8")
|
||||
return
|
||||
if not _is_authed(self.headers):
|
||||
self._unauthorized()
|
||||
return
|
||||
|
||||
if path == "/" or path == "":
|
||||
self._send(200, _route_overview())
|
||||
return
|
||||
|
||||
# /r/<owner>/<name> → repo
|
||||
# /r/<owner>/<name>/<index> → PR
|
||||
# /r/<owner>/<name>/<index>/raw → raw Markdown
|
||||
m = _REPO_PR_RAW_RE.match(path)
|
||||
if m:
|
||||
owner, name, idx, raw = m.group(1), m.group(2), m.group(3), m.group(4)
|
||||
if raw:
|
||||
status, body = _route_pr_raw(owner, name, int(idx))
|
||||
self._send(status, body,
|
||||
content_type="text/plain; charset=utf-8" if status == 200 else "text/plain")
|
||||
return
|
||||
if idx:
|
||||
self._send(200, _route_pr(owner, name, int(idx)))
|
||||
return
|
||||
self._send(200, _route_repo(owner, name))
|
||||
return
|
||||
|
||||
self._send(404, b"not found", content_type="text/plain")
|
||||
|
||||
# --- POST ----------------------------------------------------------------
|
||||
|
||||
def do_POST(self):
|
||||
path = self.path
|
||||
if not _is_authed(self.headers):
|
||||
self._unauthorized()
|
||||
return
|
||||
# /r/<owner>/<name>/edit
|
||||
m = _EDIT_RE.match(path)
|
||||
if m:
|
||||
owner, name = m.group(1), m.group(2)
|
||||
length = int(self.headers.get("Content-Length", "0") or "0")
|
||||
raw = self.rfile.read(length) if length else b""
|
||||
form = urllib.parse.parse_qs(raw.decode("utf-8", errors="replace"))
|
||||
# Collapse lists to single values.
|
||||
form_single = {k: v[0] for k, v in form.items()}
|
||||
status, extra, body = _route_edit(owner, name, form_single)
|
||||
self._send(status, body, content_type="text/plain", extra_headers=extra)
|
||||
return
|
||||
self._send(404, b"not found", content_type="text/plain")
|
||||
|
||||
def log_message(self, fmt, *args):
|
||||
print(f"pragent-dashboard: {self.address_string()} {fmt % args}", flush=True)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Routing regexes (compiled at import time)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
import re # noqa: E402
|
||||
|
||||
_REPO_PR_RAW_RE = re.compile(
|
||||
r"^/r/([^/]+)/([^/]+)(?:/(\d+)(?:/(raw))?)?/?$"
|
||||
)
|
||||
_EDIT_RE = re.compile(r"^/r/([^/]+)/([^/]+)/edit/?$")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Main
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if not _feedback_db():
|
||||
print("pragent-dashboard: WARNING: PRAGENT_FEEDBACK_DB not set; dashboard will be empty",
|
||||
flush=True)
|
||||
print("pragent-dashboard: auth via oauth2-proxy (X-Forwarded-User required)", flush=True)
|
||||
server = ThreadingHTTPServer(("0.0.0.0", PORT), Handler)
|
||||
print(f"pragent-dashboard: listening on :{PORT}", flush=True)
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,302 @@
|
||||
"""pragent pilot — dashboard read-only query layer.
|
||||
|
||||
Three functions: overview / repo_summary / pr_summary. Each opens the SQLite
|
||||
feedback DB via `feedback.init`, runs the queries it needs, and returns plain
|
||||
dicts/lists. NEVER writes — that's the dashboard_server's job (via the Gitea
|
||||
contents API). This module is what the dashboard_server's templates render.
|
||||
|
||||
All three functions are tolerant of a missing or empty DB: they return the
|
||||
shaped dict with zeros/empty lists rather than crashing. The dashboard is a
|
||||
read-only view; the pilot can boot with no feedback DB and the dashboard
|
||||
should still load.
|
||||
|
||||
Cost note: `total_cost_usd` is hardcoded to 0.0. Per-review `usage:cost` is
|
||||
not in the feedback SQLite — only the raw `review` / `inline_finding` rows
|
||||
are stored there. The equivalent-cost calc lives in `ai_review._render_collapsible_usage`
|
||||
and only knows about the latest review's tokens. Surfacing a rolled-up dollar
|
||||
figure without per-row telemetry would be guessing, so we don't.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
import os
|
||||
import sqlite3
|
||||
|
||||
from pilot import feedback
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _empty_overview() -> dict:
|
||||
return {
|
||||
"total_reviews": 0,
|
||||
"total_findings": 0,
|
||||
"total_repos": 0,
|
||||
"last_30d_reviews": 0,
|
||||
"daily": [{"date": _iso_date(i), "count": 0} for i in range(7)],
|
||||
"top_repos": [],
|
||||
"total_cost_usd": 0.0,
|
||||
}
|
||||
|
||||
|
||||
def _empty_repo_summary(repo: str) -> dict:
|
||||
return {
|
||||
"repo": repo,
|
||||
"total_runs": 0,
|
||||
"last_run_ts": 0,
|
||||
"runs_by_day": [],
|
||||
"findings_by_severity": {"critical": 0, "high": 0, "medium": 0, "low": 0},
|
||||
"top_findings": [],
|
||||
# NOTE: review rows don't carry a `model` column in the schema today,
|
||||
# so we have nothing to aggregate. When that lands, replace this
|
||||
# empty list with a `SELECT model, COUNT(*) …` over `review`.
|
||||
"models_used": [],
|
||||
}
|
||||
|
||||
|
||||
def _empty_pr_summary(repo: str, pr: int) -> dict:
|
||||
return {
|
||||
"repo": repo,
|
||||
"pr": pr,
|
||||
"head_sha": "",
|
||||
"posted_at": 0,
|
||||
"review_id_gitea": None,
|
||||
"body_comment_id": None,
|
||||
"findings": [],
|
||||
# usage isn't on the review row today; ai_review.py renders it
|
||||
# in-memory at review time. Leave empty.
|
||||
"usage": {},
|
||||
}
|
||||
|
||||
|
||||
def _iso_date(days_ago: int) -> str:
|
||||
"""Return YYYY-MM-DD for `days_ago` days before today (UTC)."""
|
||||
d = datetime.datetime.now(datetime.timezone.utc).date() - datetime.timedelta(days=days_ago)
|
||||
return d.isoformat()
|
||||
|
||||
|
||||
def _open_or_none(db_path: str) -> sqlite3.Connection | None:
|
||||
"""Open the DB if it exists and looks like a feedback DB. Else None.
|
||||
|
||||
Tolerates missing files (fresh container) and a schema-less file (the
|
||||
operator dropped a stray DB at the path). Returns a connection with
|
||||
Row factory set so callers can use `row["col"]`.
|
||||
"""
|
||||
if not db_path or not os.path.exists(db_path):
|
||||
return None
|
||||
try:
|
||||
conn = feedback.init(db_path)
|
||||
except sqlite3.DatabaseError:
|
||||
return None
|
||||
return conn
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public API
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def overview(db_path: str) -> dict:
|
||||
"""Top-of-page summary: totals + 7-bucket daily sparkline + top 5 repos."""
|
||||
conn = _open_or_none(db_path)
|
||||
if conn is None:
|
||||
return _empty_overview()
|
||||
try:
|
||||
cur = conn.execute("SELECT COUNT(*) FROM review")
|
||||
total_reviews = cur.fetchone()[0]
|
||||
cur = conn.execute("SELECT COUNT(*) FROM inline_finding")
|
||||
total_findings = cur.fetchone()[0]
|
||||
cur = conn.execute("SELECT COUNT(DISTINCT repo) FROM review")
|
||||
total_repos = cur.fetchone()[0]
|
||||
|
||||
# Last 30d window — reviews AND findings posted within the window.
|
||||
ts_30d_ago = int(datetime.datetime.now(datetime.timezone.utc).timestamp()) - 30 * 86400
|
||||
cur = conn.execute("SELECT COUNT(*) FROM review WHERE posted_at >= ?", (ts_30d_ago,))
|
||||
last_30d_reviews = cur.fetchone()[0]
|
||||
|
||||
# 7-bucket daily sparkline, oldest first. Bucket key is UTC date.
|
||||
cur = conn.execute(
|
||||
"SELECT posted_at FROM review WHERE posted_at >= ?",
|
||||
(int(datetime.datetime.now(datetime.timezone.utc).timestamp()) - 7 * 86400,),
|
||||
)
|
||||
buckets: dict[str, int] = {_iso_date(i): 0 for i in range(7)}
|
||||
for (ts,) in cur.fetchall():
|
||||
d = datetime.datetime.fromtimestamp(ts, tz=datetime.timezone.utc).date().isoformat()
|
||||
if d in buckets:
|
||||
buckets[d] += 1
|
||||
daily = [{"date": _iso_date(i), "count": buckets[_iso_date(i)]} for i in range(7)]
|
||||
|
||||
# Top 5 repos by run count, descending. last_seen is the most recent
|
||||
# review timestamp on that repo.
|
||||
cur = conn.execute(
|
||||
"SELECT repo, COUNT(*) AS runs, MAX(posted_at) AS last_seen "
|
||||
"FROM review GROUP BY repo ORDER BY runs DESC, last_seen DESC LIMIT 5"
|
||||
)
|
||||
top_repos = [
|
||||
{"repo": row[0], "run_count": row[1], "last_seen": int(row[2])}
|
||||
for row in cur.fetchall()
|
||||
]
|
||||
|
||||
return {
|
||||
"total_reviews": total_reviews,
|
||||
"total_findings": total_findings,
|
||||
"total_repos": total_repos,
|
||||
"last_30d_reviews": last_30d_reviews,
|
||||
"daily": daily,
|
||||
"top_repos": top_repos,
|
||||
"total_cost_usd": 0.0,
|
||||
}
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def repo_summary(db_path: str, repo: str) -> dict:
|
||||
"""Per-repo drill-down: runs by day, severity histogram, top findings."""
|
||||
conn = _open_or_none(db_path)
|
||||
if conn is None:
|
||||
return _empty_repo_summary(repo)
|
||||
try:
|
||||
cur = conn.execute(
|
||||
"SELECT COUNT(*), MAX(posted_at) FROM review WHERE repo = ?", (repo,)
|
||||
)
|
||||
row = cur.fetchone()
|
||||
total_runs = row[0] or 0
|
||||
last_run_ts = int(row[1]) if row[1] else 0
|
||||
|
||||
# runs_by_day for the last 30 days, oldest first; zero-buckets included.
|
||||
cur = conn.execute(
|
||||
"SELECT posted_at FROM review WHERE repo = ? AND posted_at >= ?",
|
||||
(repo, int(datetime.datetime.now(datetime.timezone.utc).timestamp()) - 30 * 86400),
|
||||
)
|
||||
buckets: dict[str, int] = {}
|
||||
for d in range(30):
|
||||
buckets[_iso_date(d)] = 0 # newest-day mapped to 0; we'll iterate
|
||||
# Re-key: build oldest-first, days_ago goes 29..0
|
||||
oldest_first = {}
|
||||
for d in range(30):
|
||||
oldest_first[_iso_date(29 - d)] = 0
|
||||
for (ts,) in cur.fetchall():
|
||||
d = datetime.datetime.fromtimestamp(ts, tz=datetime.timezone.utc).date().isoformat()
|
||||
if d in oldest_first:
|
||||
oldest_first[d] += 1
|
||||
runs_by_day = [{"date": k, "count": v} for k, v in oldest_first.items()]
|
||||
|
||||
# findings_by_severity — case-insensitive match; bucket unknown as 'low'.
|
||||
cur = conn.execute(
|
||||
"SELECT severity, COUNT(*) FROM inline_finding WHERE repo = ? GROUP BY severity",
|
||||
(repo,),
|
||||
)
|
||||
fbs = {"critical": 0, "high": 0, "medium": 0, "low": 0}
|
||||
for sev, n in cur.fetchall():
|
||||
k = (sev or "").strip().lower()
|
||||
if k not in fbs:
|
||||
k = "low"
|
||||
fbs[k] += n
|
||||
|
||||
# top_findings — top 5 posthashes by occurrence count, joined with
|
||||
# vote rollups via feedback.findings_with_votes.
|
||||
cur = conn.execute(
|
||||
"SELECT f.path, f.line, MAX(f.severity) AS severity, MAX(f.problem) AS problem, "
|
||||
"COUNT(*) AS occurrences, "
|
||||
"COALESCE(SUM(CASE WHEN rct.content = '+1' THEN 1 ELSE 0 END), 0) AS upvotes, "
|
||||
"COALESCE(SUM(CASE WHEN rct.content = '-1' THEN 1 ELSE 0 END), 0) AS downvotes, "
|
||||
"MAX(ts.resolved) AS resolved, "
|
||||
"COALESCE((SELECT COUNT(*) FROM reply WHERE finding_id IN "
|
||||
" (SELECT id FROM inline_finding WHERE posthash = f.posthash AND repo = f.repo AND path = f.path AND line = f.line)), 0) AS reply_count "
|
||||
"FROM inline_finding f "
|
||||
"LEFT JOIN reaction rct ON rct.comment_id = f.comment_id "
|
||||
"LEFT JOIN thread_state ts ON ts.finding_id = f.id "
|
||||
"WHERE f.repo = ? "
|
||||
"GROUP BY f.posthash, f.repo, f.path, f.line "
|
||||
"ORDER BY occurrences DESC, upvotes DESC LIMIT 5",
|
||||
(repo,),
|
||||
)
|
||||
top_findings = [
|
||||
{
|
||||
"path": r[0],
|
||||
"line": r[1],
|
||||
"severity": r[2],
|
||||
"problem": r[3],
|
||||
"occurrences": r[4],
|
||||
"upvotes": int(r[5] or 0),
|
||||
"downvotes": int(r[6] or 0),
|
||||
"resolved": int(r[7] or 0),
|
||||
"reply_count": int(r[8] or 0),
|
||||
}
|
||||
for r in cur.fetchall()
|
||||
]
|
||||
|
||||
return {
|
||||
"repo": repo,
|
||||
"total_runs": total_runs,
|
||||
"last_run_ts": last_run_ts,
|
||||
"runs_by_day": runs_by_day,
|
||||
"findings_by_severity": fbs,
|
||||
"top_findings": top_findings,
|
||||
"models_used": [], # see _empty_repo_summary NOTE
|
||||
}
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def pr_summary(db_path: str, repo: str, pr: int) -> dict:
|
||||
"""Per-PR view: meta + every finding the bot ever posted on that PR."""
|
||||
conn = _open_or_none(db_path)
|
||||
if conn is None:
|
||||
return _empty_pr_summary(repo, pr)
|
||||
try:
|
||||
cur = conn.execute(
|
||||
"SELECT head_sha, posted_at, review_id_gitea, body_comment_id "
|
||||
"FROM review WHERE repo = ? AND pr = ? ORDER BY posted_at DESC LIMIT 1",
|
||||
(repo, pr),
|
||||
)
|
||||
row = cur.fetchone()
|
||||
if row is None:
|
||||
return _empty_pr_summary(repo, pr)
|
||||
head_sha, posted_at, review_id_gitea, body_comment_id = row
|
||||
|
||||
cur = conn.execute(
|
||||
"SELECT f.path, f.line, f.severity, f.problem, f.fix, f.suggestion, "
|
||||
"COALESCE(SUM(CASE WHEN rct.content = '+1' THEN 1 ELSE 0 END), 0) AS upvotes, "
|
||||
"COALESCE(SUM(CASE WHEN rct.content = '-1' THEN 1 ELSE 0 END), 0) AS downvotes, "
|
||||
"MAX(ts.resolved) AS resolved, "
|
||||
"COALESCE((SELECT COUNT(*) FROM reply WHERE finding_id = f.id), 0) AS reply_count "
|
||||
"FROM inline_finding f "
|
||||
"LEFT JOIN reaction rct ON rct.comment_id = f.comment_id "
|
||||
"LEFT JOIN thread_state ts ON ts.finding_id = f.id "
|
||||
"WHERE f.repo = ? AND f.pr = ? "
|
||||
"GROUP BY f.id "
|
||||
"ORDER BY f.path, f.line",
|
||||
(repo, pr),
|
||||
)
|
||||
findings = [
|
||||
{
|
||||
"path": r[0],
|
||||
"line": r[1],
|
||||
"severity": r[2],
|
||||
"problem": r[3],
|
||||
"fix": r[4],
|
||||
"suggestion": r[5],
|
||||
"upvotes": int(r[6] or 0),
|
||||
"downvotes": int(r[7] or 0),
|
||||
"resolved": int(r[8] or 0),
|
||||
"reply_count": int(r[9] or 0),
|
||||
}
|
||||
for r in cur.fetchall()
|
||||
]
|
||||
|
||||
return {
|
||||
"repo": repo,
|
||||
"pr": pr,
|
||||
"head_sha": head_sha,
|
||||
"posted_at": int(posted_at),
|
||||
"review_id_gitea": review_id_gitea,
|
||||
"body_comment_id": body_comment_id,
|
||||
"findings": findings,
|
||||
"usage": {},
|
||||
}
|
||||
finally:
|
||||
conn.close()
|
||||
+7
-13
@@ -208,24 +208,18 @@ def ensure_evaluators() -> dict:
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def rule_body(name: str, judge_name: str, sampling: float) -> dict:
|
||||
"""POST /evaluation-rules shape for an LLM-as-judge trace rule.
|
||||
"""POST /evaluation-rules shape for an LLM-as-judge observation rule.
|
||||
|
||||
Target is `trace` rather than `observation` on purpose: the standard
|
||||
`/api/public/ingestion` path that ships review traces here feeds only
|
||||
the trace-upsert queue, and `evalService.createEvalJobs` only creates
|
||||
jobs for `targetObject ∈ {TRACE, DATASET}`. Observation rules are
|
||||
triggered exclusively from the OTel ingestion pipeline, which this
|
||||
pilot does not use. A trace rule reads the trace's own input/output —
|
||||
`langfuse_trace` already writes `_review_input`/`_review_output` onto
|
||||
the trace body for exactly this reason.
|
||||
|
||||
Mapping is required at both the rule root (server validates it there)
|
||||
and inside `evaluator` (the API echoes it back).
|
||||
The judge is referenced by `name`+`scope`, not by id — ids name specific
|
||||
versions, names name the evaluator across versions. Mapping is required at
|
||||
both the rule root (the server validates it there) and inside `evaluator`
|
||||
(the API echoes it back). Filter is on `traceName` because that is the only
|
||||
stringOptions column the observation-rule schema exposes.
|
||||
"""
|
||||
return {
|
||||
"name": name,
|
||||
"enabled": True,
|
||||
"target": "trace",
|
||||
"target": "observation",
|
||||
"sampling": sampling,
|
||||
"filter": [
|
||||
{"column": "traceName", "operator": "any of",
|
||||
|
||||
@@ -163,7 +163,7 @@ def _md_escape(s: str) -> str:
|
||||
def analyze(db_path: str, *, since_ts: Optional[int] = None,
|
||||
as_json: bool = False) -> str:
|
||||
"""Build the daily report. Returns a markdown string by default;
|
||||
`as_json=True` returns a structured dict (for tests + automation)."""
|
||||
`as_json=True` returns a structured dict (for tests + dashboards)."""
|
||||
conn = feedback.init(db_path)
|
||||
try:
|
||||
findings = list(feedback.findings_with_votes(conn, since_ts=since_ts))
|
||||
|
||||
@@ -1,46 +0,0 @@
|
||||
"""Gitea transport adapter.
|
||||
|
||||
This module owns HTTP mechanics only. Review policy, parsing, and publishing
|
||||
decisions stay in the review layer so they can be tested without a network.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
|
||||
def request(
|
||||
method: str,
|
||||
url: str,
|
||||
token: str,
|
||||
body: dict | None = None,
|
||||
accept: str = "application/json",
|
||||
) -> tuple[int, bytes]:
|
||||
headers = {"Authorization": f"token {token}", "Accept": accept}
|
||||
data = None
|
||||
if body is not None:
|
||||
data = json.dumps(body).encode()
|
||||
headers["Content-Type"] = "application/json"
|
||||
req = urllib.request.Request(url, data=data, headers=headers, method=method)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=180) as response:
|
||||
return response.status, response.read()
|
||||
except urllib.error.HTTPError as exc:
|
||||
return exc.code, exc.read()
|
||||
except urllib.error.URLError as exc:
|
||||
raise RuntimeError(f"network error: {exc.reason}") from exc
|
||||
|
||||
|
||||
class GiteaClient:
|
||||
"""Small adapter for repository-scoped Gitea calls."""
|
||||
|
||||
def __init__(self, api: str, token: str):
|
||||
self.api = api.rstrip("/")
|
||||
self.token = token
|
||||
|
||||
def get(self, path: str, accept: str = "application/json") -> tuple[int, bytes]:
|
||||
return request("GET", f"{self.api}/api/v1/repos/{path}", self.token, accept=accept)
|
||||
|
||||
def post(self, path: str, body: dict) -> tuple[int, bytes]:
|
||||
return request("POST", f"{self.api}/api/v1/repos/{path}", self.token, body)
|
||||
@@ -16,7 +16,7 @@ Provider split
|
||||
--------------
|
||||
`environment` on every trace is either `ollama` or `claude`, derived from the
|
||||
resolved display model (`resolve_environment`). That is what keeps the two
|
||||
spend stories separate in Langfuse: every view, filter and cost breakdown
|
||||
spend stories separate in Langfuse: every dashboard, filter and cost breakdown
|
||||
takes an environment selector, so "what did the local/self-hosted path cost"
|
||||
and "what did the Claude path cost" are two views of one project rather than
|
||||
two projects with two key pairs to rotate. Tags carry the finer split
|
||||
|
||||
@@ -1,36 +0,0 @@
|
||||
"""Model-provider adapter for the legacy Anthropic-compatible endpoint."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
try: # Works both as `python pilot/ai_review.py` and `import pilot.model_client`.
|
||||
from .gitea_client import request
|
||||
except ImportError: # pragma: no cover - script-style runtime
|
||||
from gitea_client import request
|
||||
|
||||
|
||||
def parse_text_blocks(content: object) -> str:
|
||||
"""Return only text blocks from an Anthropic-style response."""
|
||||
if not isinstance(content, list):
|
||||
return ""
|
||||
return "\n".join(
|
||||
block["text"]
|
||||
for block in content
|
||||
if isinstance(block, dict)
|
||||
and block.get("type") == "text"
|
||||
and isinstance(block.get("text"), str)
|
||||
).strip()
|
||||
|
||||
|
||||
def complete(base_url: str, model: str, system: str, user: str, max_tokens: int) -> str:
|
||||
payload = {
|
||||
"model": model,
|
||||
"max_tokens": max_tokens,
|
||||
"system": system,
|
||||
"messages": [{"role": "user", "content": user}],
|
||||
}
|
||||
status, raw = request("POST", f"{base_url.rstrip('/')}/v1/messages", "ollama", payload)
|
||||
if status != 200:
|
||||
detail = raw[:500].decode("utf-8", errors="replace")
|
||||
raise RuntimeError(f"model call failed: HTTP {status}: {detail}")
|
||||
return parse_text_blocks(json.loads(raw).get("content", []))
|
||||
@@ -1,32 +0,0 @@
|
||||
"""Trusted repository configuration and opt-in policy."""
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import urllib.parse
|
||||
from collections.abc import Callable
|
||||
|
||||
|
||||
def repo_enabled(
|
||||
get: Callable[..., tuple[int, bytes]],
|
||||
api: str,
|
||||
repo: str,
|
||||
ref: str,
|
||||
token: str,
|
||||
) -> bool:
|
||||
"""Read the opt-in flag from the trusted base branch.
|
||||
|
||||
The transport is injected so the policy is testable without a live Gitea.
|
||||
Any missing, malformed, or non-boolean value disables review.
|
||||
"""
|
||||
path = "contents/.pr-review.json?ref=" + urllib.parse.quote(ref, safe="")
|
||||
status, raw = get(api, repo, path, token)
|
||||
if status != 200:
|
||||
return False
|
||||
try:
|
||||
envelope = json.loads(raw)
|
||||
encoded = envelope.get("content", "").replace("\n", "")
|
||||
config = json.loads(base64.b64decode(encoded).decode("utf-8", errors="replace"))
|
||||
except (AttributeError, TypeError, ValueError, json.JSONDecodeError):
|
||||
return False
|
||||
return isinstance(config, dict) and config.get("enabled") is True
|
||||
@@ -1,17 +0,0 @@
|
||||
"""Stable interfaces shared by the review pipeline and its adapters."""
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Protocol
|
||||
|
||||
|
||||
class Forge(Protocol):
|
||||
def get(self, path: str, accept: str = "application/json") -> tuple[int, bytes]: ...
|
||||
def post(self, path: str, body: dict) -> tuple[int, bytes]: ...
|
||||
|
||||
|
||||
class Reviewer(Protocol):
|
||||
def review(self, system: str, user: str, max_tokens: int) -> str: ...
|
||||
|
||||
|
||||
class Telemetry(Protocol):
|
||||
def emit(self, **event: object) -> None: ...
|
||||
+15
-2
@@ -46,7 +46,6 @@ import urllib.parse
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
|
||||
from ai_review import gitea_get, review_pr
|
||||
from review_config import repo_enabled
|
||||
|
||||
try:
|
||||
import feedback_harvest # optional — absent in CI-step pod, present in
|
||||
@@ -103,7 +102,21 @@ def is_repo_enabled(api: str, repo: str, ref: str, token: str) -> bool:
|
||||
The bool-coerce of `.get("enabled") is True` rejects the common
|
||||
gotchas (`null`, `1`, `"yes"`, missing field all yield False).
|
||||
"""
|
||||
return repo_enabled(gitea_get, api, repo, ref, token)
|
||||
code, raw = gitea_get(
|
||||
api, repo,
|
||||
"contents/.pr-review.json?ref=" + urllib.parse.quote(ref, safe=""),
|
||||
token,
|
||||
)
|
||||
if code != 200:
|
||||
return False
|
||||
try:
|
||||
data = json.loads(raw)
|
||||
content_b64 = data.get("content", "").replace("\n", "")
|
||||
decoded = base64.b64decode(content_b64).decode("utf-8", errors="replace")
|
||||
cfg = json.loads(decoded)
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
return False
|
||||
return isinstance(cfg, dict) and cfg.get("enabled") is True
|
||||
|
||||
|
||||
def _verify_signature(raw_body: bytes, headers) -> bool:
|
||||
|
||||
@@ -0,0 +1,203 @@
|
||||
"""Tests for pilot/dashboard.py — stdlib HTTP server rendering dashboard HTML.
|
||||
|
||||
We spin up the server on an ephemeral port in setUp, drive it with
|
||||
http.client, and tear it down in tearDown. Auth is now performed by
|
||||
oauth2-proxy: the dashboard trusts `X-Forwarded-User` set by the proxy
|
||||
and returns 401 (with a Basic challenge) when the header is missing.
|
||||
|
||||
The dashboard reads `PRAGENT_FEEDBACK_DB` and renders views via
|
||||
`dashboard_data`. We seed an in-memory SQLite at `tmp_path` for each
|
||||
scenario that needs rows.
|
||||
"""
|
||||
import http.client
|
||||
import os
|
||||
import socket
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
|
||||
sys.path.insert(0, os.path.join(ROOT, "pilot"))
|
||||
|
||||
import dashboard as dash # noqa: E402
|
||||
from pilot import feedback # noqa: E402
|
||||
|
||||
|
||||
def _free_port() -> int:
|
||||
s = socket.socket()
|
||||
s.bind(("127.0.0.1", 0))
|
||||
port = s.getsockname()[1]
|
||||
s.close()
|
||||
return port
|
||||
|
||||
|
||||
class _ServerThread:
|
||||
def __init__(self, port: int, handler):
|
||||
self.server = handler((host := "127.0.0.1", port), None)
|
||||
self.port = port
|
||||
self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
|
||||
self.thread.start()
|
||||
|
||||
def stop(self):
|
||||
self.server.shutdown()
|
||||
self.server.server_close()
|
||||
self.thread.join(timeout=2)
|
||||
|
||||
|
||||
def _get(port: int, path: str, headers: dict | None = None) -> tuple[int, dict, bytes]:
|
||||
conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
|
||||
conn.request("GET", path, headers=headers or {})
|
||||
r = conn.getresponse()
|
||||
body = r.read()
|
||||
h = dict(r.getheaders())
|
||||
conn.close()
|
||||
return r.status, h, body
|
||||
|
||||
|
||||
def _post(port: int, path: str, body: bytes, headers: dict | None = None) -> tuple[int, dict, bytes]:
|
||||
conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
|
||||
hdrs = {"Content-Type": "application/x-www-form-urlencoded"}
|
||||
if headers:
|
||||
hdrs.update(headers)
|
||||
conn.request("POST", path, body=body, headers=hdrs)
|
||||
r = conn.getresponse()
|
||||
body_b = r.read()
|
||||
h = dict(r.getheaders())
|
||||
conn.close()
|
||||
return r.status, h, body_b
|
||||
|
||||
|
||||
class TestDashboardAuth(unittest.TestCase):
|
||||
"""Auth gate: require X-Forwarded-User (set by oauth2-proxy).
|
||||
|
||||
When the header is missing every non-static route returns 401 with a
|
||||
Basic challenge, which lets oauth2-proxy redirect the browser to
|
||||
Logto. Static is exempt so the unauthenticated probe traffic doesn't
|
||||
loop the proxy through the auth flow.
|
||||
"""
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.db = f"{self.tmp.name}/f.db"
|
||||
os.environ["PRAGENT_FEEDBACK_DB"] = self.db
|
||||
os.environ["DASHBOARD_PORT"] = str(0) # we override below
|
||||
|
||||
self.port = _free_port()
|
||||
from http.server import ThreadingHTTPServer
|
||||
self.srv = ThreadingHTTPServer(("127.0.0.1", self.port), dash.Handler)
|
||||
self.thread = threading.Thread(target=self.srv.serve_forever, daemon=True)
|
||||
self.thread.start()
|
||||
|
||||
def tearDown(self):
|
||||
self.srv.shutdown()
|
||||
self.srv.server_close()
|
||||
self.thread.join(timeout=2)
|
||||
for k in ("PRAGENT_FEEDBACK_DB", "DASHBOARD_PORT"):
|
||||
os.environ.pop(k, None)
|
||||
self.tmp.cleanup()
|
||||
|
||||
def test_anonymous_overview_returns_401_with_basic_challenge(self):
|
||||
status, h, body = _get(self.port, "/")
|
||||
self.assertEqual(status, 401)
|
||||
self.assertEqual(h.get("WWW-Authenticate"), 'Basic realm="pragent-dashboard"')
|
||||
self.assertEqual(body, b"unauthorized\n")
|
||||
|
||||
def test_anonymous_repo_returns_401(self):
|
||||
status, _h, _body = _get(self.port, "/r/alpha/one")
|
||||
self.assertEqual(status, 401)
|
||||
|
||||
def test_anonymous_post_returns_401(self):
|
||||
status, _h, _body = _post(self.port, "/r/alpha/one/edit", b"x=1")
|
||||
self.assertEqual(status, 401)
|
||||
|
||||
def test_authenticated_overview_succeeds(self):
|
||||
status, h, body = _get(self.port, "/", headers={"X-Forwarded-User": "marcos@example.com"})
|
||||
self.assertEqual(status, 200)
|
||||
self.assertIn(b"Overview", body)
|
||||
|
||||
def test_static_does_not_require_auth(self):
|
||||
status, h, body = _get(self.port, "/static/style.css")
|
||||
self.assertEqual(status, 200)
|
||||
self.assertIn("text/css", h.get("Content-Type", ""))
|
||||
self.assertGreater(len(body), 50)
|
||||
|
||||
def test_empty_x_forwarded_user_treated_as_anonymous(self):
|
||||
status, _h, _body = _get(self.port, "/", headers={"X-Forwarded-User": " "})
|
||||
self.assertEqual(status, 401)
|
||||
|
||||
|
||||
class TestDashboardRender(unittest.TestCase):
|
||||
"""Render-only tests — X-Forwarded-User set, real seeded data."""
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.db = f"{self.tmp.name}/f.db"
|
||||
os.environ["PRAGENT_FEEDBACK_DB"] = self.db
|
||||
# Seed: 2 repos, a couple of reviews + findings each.
|
||||
conn = feedback.init(self.db)
|
||||
for repo, n_prs in (("alpha/one", 2), ("beta/two", 1)):
|
||||
for n in range(n_prs):
|
||||
rid = feedback.record_review(
|
||||
conn, repo=repo, pr=n + 1, head_sha=f"sha{repo}-{n}",
|
||||
review_id_gitea=1000 + n, body_comment_id=2000 + n,
|
||||
posted_at=int(time.time()) - n * 60,
|
||||
)
|
||||
for k in range(3):
|
||||
feedback.record_inline_finding(
|
||||
conn, review_id=rid, repo=repo, pr=n + 1,
|
||||
path=f"src/file_{k}.py", line=k + 1,
|
||||
severity=["critical", "high", "medium"][k],
|
||||
problem=f"problem {k}",
|
||||
fix=f"fix {k}", suggestion=f"suggestion {k}",
|
||||
comment_id=3000 + n * 10 + k,
|
||||
)
|
||||
conn.close()
|
||||
|
||||
self.port = _free_port()
|
||||
from http.server import ThreadingHTTPServer
|
||||
self.srv = ThreadingHTTPServer(("127.0.0.1", self.port), dash.Handler)
|
||||
self.thread = threading.Thread(target=self.srv.serve_forever, daemon=True)
|
||||
self.thread.start()
|
||||
self.auth_hdr = {"X-Forwarded-User": "marcos@example.com"}
|
||||
|
||||
def tearDown(self):
|
||||
self.srv.shutdown()
|
||||
self.srv.server_close()
|
||||
self.thread.join(timeout=2)
|
||||
os.environ.pop("PRAGENT_FEEDBACK_DB", None)
|
||||
self.tmp.cleanup()
|
||||
|
||||
def test_overview_200_contains_repo_names(self):
|
||||
status, _h, body = _get(self.port, "/", headers=self.auth_hdr)
|
||||
self.assertEqual(status, 200)
|
||||
text = body.decode()
|
||||
self.assertIn("Overview", text)
|
||||
self.assertIn("alpha/one", text)
|
||||
self.assertIn("beta/two", text)
|
||||
|
||||
def test_repo_page_200(self):
|
||||
status, _h, body = _get(self.port, "/r/alpha/one", headers=self.auth_hdr)
|
||||
self.assertEqual(status, 200)
|
||||
text = body.decode()
|
||||
self.assertIn("alpha/one", text)
|
||||
# The findings table should appear.
|
||||
self.assertIn("src/file_0.py", text)
|
||||
|
||||
def test_pr_page_200(self):
|
||||
status, _h, body = _get(self.port, "/r/alpha/one/1", headers=self.auth_hdr)
|
||||
self.assertEqual(status, 200)
|
||||
text = body.decode()
|
||||
self.assertIn("alpha/one", text)
|
||||
self.assertIn("#1", text)
|
||||
self.assertIn("src/file_0.py", text)
|
||||
|
||||
def test_unknown_route_404(self):
|
||||
status, _h, _body = _get(self.port, "/no/such/route", headers=self.auth_hdr)
|
||||
self.assertEqual(status, 404)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,249 @@
|
||||
"""Tests for pilot/dashboard_data.py — read-only query layer over the feedback SQLite.
|
||||
|
||||
Covers: empty-DB fallbacks (no crash on missing/empty DB), overview rollups,
|
||||
per-repo drill-down (findings by severity, top findings, runs by day), and
|
||||
the per-PR view. The dashboard never writes — only reads.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import unittest
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
|
||||
sys.path.insert(0, os.path.join(HERE, "..", "..")) # so `from pilot import …` works
|
||||
|
||||
from pilot import dashboard_data, feedback
|
||||
|
||||
|
||||
def _seed_repo(conn, *, repo: str, prs: int, findings_per_pr: int, day_offset: int = 0):
|
||||
"""Seed one repo with `prs` PRs each with `findings_per_pr` findings.
|
||||
|
||||
All timestamps cluster on (now - day_offset days). Returns list of review ids.
|
||||
"""
|
||||
base = int(time.time()) - day_offset * 86400
|
||||
rids = []
|
||||
for n in range(prs):
|
||||
rid = feedback.record_review(
|
||||
conn, repo=repo, pr=n + 1, head_sha=f"sha{n}",
|
||||
review_id_gitea=1000 + n, body_comment_id=2000 + n,
|
||||
posted_at=base + n * 60,
|
||||
)
|
||||
rids.append(rid)
|
||||
for k in range(findings_per_pr):
|
||||
feedback.record_inline_finding(
|
||||
conn, review_id=rid, repo=repo, pr=n + 1,
|
||||
path=f"src/file_{k}.py", line=k + 1,
|
||||
severity=["critical", "high", "medium", "low"][k % 4],
|
||||
problem=f"problem {k}",
|
||||
fix=f"fix {k}", suggestion=f"suggestion {k}",
|
||||
comment_id=3000 + n * 10 + k,
|
||||
posted_at=base + n * 60,
|
||||
)
|
||||
return rids
|
||||
|
||||
|
||||
class TestEmptyDB(unittest.TestCase):
|
||||
def test_missing_file_returns_zero_dict(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
missing = f"{d}/nope.db"
|
||||
data = dashboard_data.overview(missing)
|
||||
self.assertEqual(data["total_reviews"], 0)
|
||||
self.assertEqual(data["total_findings"], 0)
|
||||
self.assertEqual(data["total_repos"], 0)
|
||||
self.assertEqual(data["last_30d_reviews"], 0)
|
||||
self.assertEqual(len(data["daily"]), 7)
|
||||
self.assertEqual(data["top_repos"], [])
|
||||
self.assertEqual(data["total_cost_usd"], 0.0)
|
||||
|
||||
def test_missing_file_repo_summary_safe(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
data = dashboard_data.repo_summary(f"{d}/nope.db", "o/r")
|
||||
self.assertEqual(data["repo"], "o/r")
|
||||
self.assertEqual(data["total_runs"], 0)
|
||||
self.assertEqual(data["runs_by_day"], [])
|
||||
for sev in ("critical", "high", "medium", "low"):
|
||||
self.assertEqual(data["findings_by_severity"][sev], 0)
|
||||
self.assertEqual(data["top_findings"], [])
|
||||
self.assertEqual(data["models_used"], [])
|
||||
|
||||
def test_missing_file_pr_summary_safe(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
data = dashboard_data.pr_summary(f"{d}/nope.db", "o/r", 1)
|
||||
self.assertEqual(data["repo"], "o/r")
|
||||
self.assertEqual(data["pr"], 1)
|
||||
self.assertEqual(data["findings"], [])
|
||||
self.assertEqual(data["usage"], {})
|
||||
|
||||
|
||||
class TestEmptyButExistingDB(unittest.TestCase):
|
||||
"""`init` creates the schema — DB exists but has no rows."""
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.db = f"{self.tmp.name}/f.db"
|
||||
feedback.init(self.db)
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def test_overview_is_zero(self):
|
||||
data = dashboard_data.overview(self.db)
|
||||
self.assertEqual(data["total_reviews"], 0)
|
||||
self.assertEqual(data["total_findings"], 0)
|
||||
self.assertEqual(data["total_repos"], 0)
|
||||
|
||||
def test_repo_summary_is_zero(self):
|
||||
data = dashboard_data.repo_summary(self.db, "o/r")
|
||||
self.assertEqual(data["total_runs"], 0)
|
||||
self.assertEqual(data["findings_by_severity"], {"critical": 0, "high": 0, "medium": 0, "low": 0})
|
||||
|
||||
def test_pr_summary_is_zero(self):
|
||||
data = dashboard_data.pr_summary(self.db, "o/r", 1)
|
||||
self.assertEqual(data["findings"], [])
|
||||
|
||||
|
||||
class TestOverview(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.db = f"{self.tmp.name}/f.db"
|
||||
self.conn = feedback.init(self.db)
|
||||
_seed_repo(self.conn, repo="alpha/one", prs=3, findings_per_pr=2)
|
||||
_seed_repo(self.conn, repo="beta/two", prs=1, findings_per_pr=4)
|
||||
self.conn.close()
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def test_totals(self):
|
||||
data = dashboard_data.overview(self.db)
|
||||
self.assertEqual(data["total_reviews"], 4)
|
||||
self.assertEqual(data["total_findings"], 6 + 4) # 3*2 + 1*4 = 10
|
||||
self.assertEqual(data["total_repos"], 2)
|
||||
self.assertEqual(data["total_cost_usd"], 0.0)
|
||||
|
||||
def test_top_repos_sorted_by_run_count(self):
|
||||
data = dashboard_data.overview(self.db)
|
||||
repos = [r["repo"] for r in data["top_repos"]]
|
||||
# alpha/one has 3 runs, beta/two has 1.
|
||||
self.assertEqual(repos[0], "alpha/one")
|
||||
self.assertEqual(data["top_repos"][0]["run_count"], 3)
|
||||
self.assertEqual(data["top_repos"][1]["run_count"], 1)
|
||||
# last_seen is a unix timestamp int.
|
||||
for r in data["top_repos"]:
|
||||
self.assertIsInstance(r["last_seen"], int)
|
||||
|
||||
def test_daily_buckets_are_7(self):
|
||||
data = dashboard_data.overview(self.db)
|
||||
self.assertEqual(len(data["daily"]), 7)
|
||||
for b in data["daily"]:
|
||||
self.assertIn("date", b)
|
||||
self.assertIn("count", b)
|
||||
|
||||
def test_last_30d_reviews(self):
|
||||
data = dashboard_data.overview(self.db)
|
||||
self.assertEqual(data["last_30d_reviews"], 4)
|
||||
|
||||
|
||||
class TestRepoSummary(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.db = f"{self.tmp.name}/f.db"
|
||||
self.conn = feedback.init(self.db)
|
||||
# 4 PRs with 2 findings each → 8 findings, severity cycle [c,h,m,l,c,h,m,l]
|
||||
_seed_repo(self.conn, repo="o/r", prs=4, findings_per_pr=2)
|
||||
# Add some reactions so top_findings has signal.
|
||||
rows = self.conn.execute(
|
||||
"SELECT id, comment_id FROM inline_finding WHERE repo=? ORDER BY id LIMIT 3",
|
||||
("o/r",),
|
||||
).fetchall()
|
||||
for r in rows:
|
||||
feedback.record_reaction(self.conn, comment_id=r["comment_id"], user="u", content="+1")
|
||||
self.conn.close()
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def test_basic_shape(self):
|
||||
data = dashboard_data.repo_summary(self.db, "o/r")
|
||||
self.assertEqual(data["repo"], "o/r")
|
||||
self.assertEqual(data["total_runs"], 4)
|
||||
self.assertIsInstance(data["last_run_ts"], int)
|
||||
|
||||
def test_findings_by_severity(self):
|
||||
data = dashboard_data.repo_summary(self.db, "o/r")
|
||||
fbs = data["findings_by_severity"]
|
||||
# 4 PRs × 2 findings; per-PR severities are [critical, high].
|
||||
# (k in range(2) → k=0 critical, k=1 high for every PR.)
|
||||
self.assertEqual(fbs["critical"], 4)
|
||||
self.assertEqual(fbs["high"], 4)
|
||||
self.assertEqual(fbs["medium"], 0)
|
||||
self.assertEqual(fbs["low"], 0)
|
||||
|
||||
def test_runs_by_day_is_list(self):
|
||||
data = dashboard_data.repo_summary(self.db, "o/r")
|
||||
self.assertIsInstance(data["runs_by_day"], list)
|
||||
for r in data["runs_by_day"]:
|
||||
self.assertIn("date", r)
|
||||
self.assertIn("count", r)
|
||||
|
||||
def test_top_findings_structure(self):
|
||||
data = dashboard_data.repo_summary(self.db, "o/r")
|
||||
self.assertGreater(len(data["top_findings"]), 0)
|
||||
first = data["top_findings"][0]
|
||||
for k in ("path", "line", "severity", "problem", "occurrences", "upvotes", "downvotes", "resolved", "reply_count"):
|
||||
self.assertIn(k, first)
|
||||
|
||||
def test_models_used_is_empty_list_with_note(self):
|
||||
# The schema has no `model` column on review — the dashboard can't show
|
||||
# model usage from this DB today. We document that via an empty list.
|
||||
data = dashboard_data.repo_summary(self.db, "o/r")
|
||||
self.assertEqual(data["models_used"], [])
|
||||
|
||||
|
||||
class TestPRSummary(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.db = f"{self.tmp.name}/f.db"
|
||||
self.conn = feedback.init(self.db)
|
||||
rid = feedback.record_review(
|
||||
self.conn, repo="o/r", pr=42, head_sha="abc",
|
||||
review_id_gitea=9001, body_comment_id=8001,
|
||||
posted_at=1700000000,
|
||||
)
|
||||
for k in range(3):
|
||||
feedback.record_inline_finding(
|
||||
self.conn, review_id=rid, repo="o/r", pr=42,
|
||||
path=f"src/x_{k}.py", line=k + 10,
|
||||
severity=["critical", "high", "low"][k],
|
||||
problem=f"p{k}", fix=f"f{k}", suggestion=f"s{k}",
|
||||
comment_id=7000 + k,
|
||||
)
|
||||
self.conn.close()
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def test_meta(self):
|
||||
data = dashboard_data.pr_summary(self.db, "o/r", 42)
|
||||
self.assertEqual(data["repo"], "o/r")
|
||||
self.assertEqual(data["pr"], 42)
|
||||
self.assertEqual(data["head_sha"], "abc")
|
||||
self.assertEqual(data["review_id_gitea"], 9001)
|
||||
self.assertEqual(data["body_comment_id"], 8001)
|
||||
self.assertEqual(data["posted_at"], 1700000000)
|
||||
# usage is empty because the schema has no usage column.
|
||||
self.assertEqual(data["usage"], {})
|
||||
|
||||
def test_findings(self):
|
||||
data = dashboard_data.pr_summary(self.db, "o/r", 42)
|
||||
self.assertEqual(len(data["findings"]), 3)
|
||||
for f in data["findings"]:
|
||||
for k in ("path", "line", "severity", "problem", "fix", "suggestion",
|
||||
"upvotes", "downvotes", "resolved", "reply_count"):
|
||||
self.assertIn(k, f)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,209 @@
|
||||
"""Tests for the edit endpoint — POST /r/<owner>/<name>/edit (Task C).
|
||||
|
||||
We mock the Gitea HTTP layer (urllib.request.urlopen) so the test never
|
||||
touches the network. The dashboard handler is responsible for:
|
||||
* auth (X-Forwarded-User set by oauth2-proxy) + CSRF
|
||||
* read .pr-review.json via GET (404 → start from {})
|
||||
* validate model against cost_model.PRICES
|
||||
* PUT the updated file back, with sha + base64 content
|
||||
* redirect to /r/<owner>/<name> on success
|
||||
"""
|
||||
import base64
|
||||
import http.client
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
|
||||
sys.path.insert(0, os.path.join(ROOT, "pilot"))
|
||||
|
||||
import dashboard as dash # noqa: E402
|
||||
from pilot import feedback # noqa: E402
|
||||
|
||||
|
||||
def _free_port() -> int:
|
||||
s = socket.socket()
|
||||
s.bind(("127.0.0.1", 0))
|
||||
port = s.getsockname()[1]
|
||||
s.close()
|
||||
return port
|
||||
|
||||
|
||||
def _post(port: int, path: str, body: bytes, *, headers: dict | None = None) -> tuple[int, dict, bytes]:
|
||||
conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
|
||||
hdrs = {"Content-Type": "application/x-www-form-urlencoded"}
|
||||
if headers:
|
||||
hdrs.update(headers)
|
||||
conn.request("POST", path, body=body, headers=hdrs)
|
||||
r = conn.getresponse()
|
||||
body_b = r.read()
|
||||
h = dict(r.getheaders())
|
||||
conn.close()
|
||||
return r.status, h, body_b
|
||||
|
||||
|
||||
class _FakeResp:
|
||||
def __init__(self, status: int, body: bytes):
|
||||
self.status = status
|
||||
self._body = body
|
||||
|
||||
def read(self):
|
||||
return self._body
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *a):
|
||||
return False
|
||||
|
||||
|
||||
class TestDashboardEdit(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.db = f"{self.tmp.name}/f.db"
|
||||
os.environ["PRAGENT_FEEDBACK_DB"] = self.db
|
||||
os.environ["PRAGENT_BOT_TOKEN"] = "bot-token"
|
||||
# Seed a row so the repo page is meaningful.
|
||||
conn = feedback.init(self.db)
|
||||
feedback.record_review(
|
||||
conn, repo="o/r", pr=1, head_sha="x",
|
||||
)
|
||||
conn.close()
|
||||
|
||||
self.port = _free_port()
|
||||
from http.server import ThreadingHTTPServer
|
||||
self.srv = ThreadingHTTPServer(("127.0.0.1", self.port), dash.Handler)
|
||||
self.thread = threading.Thread(target=self.srv.serve_forever, daemon=True)
|
||||
self.thread.start()
|
||||
# Pull the per-process CSRF secret from the rendered repo page — the
|
||||
# edit form embeds the same token as a hidden input.
|
||||
self.csrf = dash._CSRF_SECRET
|
||||
self.auth_hdr = {"X-Forwarded-User": "marcos@example.com"}
|
||||
|
||||
# Records of HTTP calls made by the handler.
|
||||
self.calls: list[tuple[str, str, dict | None, bytes | None]] = []
|
||||
|
||||
def tearDown(self):
|
||||
self.srv.shutdown()
|
||||
self.srv.server_close()
|
||||
self.thread.join(timeout=2)
|
||||
for k in ("PRAGENT_FEEDBACK_DB", "PRAGENT_BOT_TOKEN"):
|
||||
os.environ.pop(k, None)
|
||||
self.tmp.cleanup()
|
||||
|
||||
def _urlopen(self, req, timeout=30):
|
||||
"""Replacement for urllib.request.urlopen that the handler uses."""
|
||||
url = req.full_url if hasattr(req, "full_url") else req
|
||||
method = getattr(req, "method", None) or "GET"
|
||||
body = getattr(req, "data", None)
|
||||
headers = dict(getattr(req, "headers", {}) or {})
|
||||
self.calls.append((method, url, headers, body))
|
||||
# Route based on URL: GET contents/.../raw vs PUT contents/.pr-review.json
|
||||
if method == "GET" and ".pr-review.json" in url:
|
||||
return _FakeResp(200, json.dumps({
|
||||
"content": base64.b64encode(b'{"focus":["x"],"model":"claude-haiku-4-5"}').decode(),
|
||||
"sha": "deadbeef",
|
||||
}).encode())
|
||||
if method == "PUT" and ".pr-review.json" in url:
|
||||
return _FakeResp(200, b'{}')
|
||||
return _FakeResp(404, b'{"message":"not found"}')
|
||||
|
||||
def test_edit_updates_static_message_and_model(self):
|
||||
form = (
|
||||
f"_csrf={self.csrf}"
|
||||
f"&static_message=Hello%20world"
|
||||
f"&model=claude-sonnet-5"
|
||||
).encode()
|
||||
with patch.object(dash.urllib.request, "urlopen", side_effect=self._urlopen):
|
||||
status, h, _b = _post(self.port, "/r/o/r/edit", form, headers=self.auth_hdr)
|
||||
self.assertEqual(status, 302)
|
||||
self.assertEqual(h.get("Location"), "/r/o/r")
|
||||
|
||||
# Find the PUT call.
|
||||
put_calls = [c for c in self.calls if c[0] == "PUT"]
|
||||
self.assertEqual(len(put_calls), 1, self.calls)
|
||||
method, url, _hdrs, body = put_calls[0]
|
||||
self.assertIn(".pr-review.json", url)
|
||||
payload = json.loads(body)
|
||||
self.assertIn("content", payload)
|
||||
self.assertEqual(payload["sha"], "deadbeef")
|
||||
decoded = base64.b64decode(payload["content"]).decode()
|
||||
cfg = json.loads(decoded)
|
||||
self.assertEqual(cfg.get("static_message"), "Hello world")
|
||||
self.assertEqual(cfg.get("model"), "claude-sonnet-5")
|
||||
|
||||
def test_edit_strips_static_message_to_400(self):
|
||||
long_msg = "x" * 600
|
||||
form = (
|
||||
f"_csrf={self.csrf}"
|
||||
f"&static_message={long_msg}"
|
||||
f"&model=claude-haiku-4-5"
|
||||
).encode()
|
||||
with patch.object(dash.urllib.request, "urlopen", side_effect=self._urlopen):
|
||||
_post(self.port, "/r/o/r/edit", form, headers=self.auth_hdr)
|
||||
put = next(c for c in self.calls if c[0] == "PUT")
|
||||
cfg = json.loads(base64.b64decode(json.loads(put[3])["content"]))
|
||||
self.assertEqual(len(cfg["static_message"]), 400)
|
||||
|
||||
def test_edit_rejects_unknown_model_with_flash(self):
|
||||
form = (
|
||||
f"_csrf={self.csrf}"
|
||||
f"&static_message=hi"
|
||||
f"&model=does-not-exist"
|
||||
).encode()
|
||||
with patch.object(dash.urllib.request, "urlopen", side_effect=self._urlopen):
|
||||
status, h, _b = _post(self.port, "/r/o/r/edit", form, headers=self.auth_hdr)
|
||||
self.assertEqual(status, 302)
|
||||
self.assertIn("flash=", h.get("Location", ""))
|
||||
# No PUT should have been issued.
|
||||
put_calls = [c for c in self.calls if c[0] == "PUT"]
|
||||
self.assertEqual(put_calls, [])
|
||||
|
||||
def test_edit_requires_auth(self):
|
||||
form = f"_csrf={self.csrf}&static_message=x&model=claude-haiku-4-5".encode()
|
||||
with patch.object(dash.urllib.request, "urlopen", side_effect=self._urlopen):
|
||||
status, h, _b = _post(self.port, "/r/o/r/edit", form)
|
||||
self.assertEqual(status, 401)
|
||||
self.assertEqual(h.get("WWW-Authenticate"), 'Basic realm="pragent-dashboard"')
|
||||
# No Gitea calls at all — auth gate fires first.
|
||||
self.assertEqual(self.calls, [])
|
||||
|
||||
def test_edit_csrf_mismatch_redirects_without_save(self):
|
||||
form = f"_csrf=wrong&static_message=x&model=claude-haiku-4-5".encode()
|
||||
with patch.object(dash.urllib.request, "urlopen", side_effect=self._urlopen):
|
||||
status, h, _b = _post(self.port, "/r/o/r/edit", form, headers=self.auth_hdr)
|
||||
self.assertEqual(status, 302)
|
||||
self.assertEqual(h.get("Location"), "/r/o/r")
|
||||
put_calls = [c for c in self.calls if c[0] == "PUT"]
|
||||
self.assertEqual(put_calls, [])
|
||||
|
||||
def test_edit_creates_file_when_missing(self):
|
||||
"""When GET returns 404, the PUT must still happen (no sha)."""
|
||||
def _route(req, timeout=30):
|
||||
url = req.full_url
|
||||
method = getattr(req, "method", None) or "GET"
|
||||
body = getattr(req, "data", None)
|
||||
self.calls.append((method, url, {}, body))
|
||||
if method == "GET" and ".pr-review.json" in url:
|
||||
return _FakeResp(404, b'{"message":"not found"}')
|
||||
if method == "PUT" and ".pr-review.json" in url:
|
||||
return _FakeResp(201, b"{}")
|
||||
return _FakeResp(404, b"")
|
||||
|
||||
form = f"_csrf={self.csrf}&static_message=hi&model=claude-haiku-4-5".encode()
|
||||
with patch.object(dash.urllib.request, "urlopen", side_effect=_route):
|
||||
status, h, _b = _post(self.port, "/r/o/r/edit", form, headers=self.auth_hdr)
|
||||
self.assertEqual(status, 302)
|
||||
put = next(c for c in self.calls if c[0] == "PUT")
|
||||
payload = json.loads(put[3])
|
||||
self.assertNotIn("sha", payload, "missing-file PUT should omit sha")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,74 @@
|
||||
"""Tests for the model <select> in the repo edit form (Task D)."""
|
||||
import http.client
|
||||
import os
|
||||
import socket
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import unittest
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
|
||||
sys.path.insert(0, os.path.join(ROOT, "pilot"))
|
||||
|
||||
import dashboard as dash # noqa: E402
|
||||
from pilot import cost_model, feedback # noqa: E402
|
||||
|
||||
|
||||
def _free_port() -> int:
|
||||
s = socket.socket()
|
||||
s.bind(("127.0.0.1", 0))
|
||||
port = s.getsockname()[1]
|
||||
s.close()
|
||||
return port
|
||||
|
||||
|
||||
def _get(port: int, path: str, headers: dict | None = None) -> tuple[int, dict, bytes]:
|
||||
conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
|
||||
conn.request("GET", path, headers=headers or {})
|
||||
r = conn.getresponse()
|
||||
body = r.read()
|
||||
h = dict(r.getheaders())
|
||||
conn.close()
|
||||
return r.status, h, body
|
||||
|
||||
|
||||
class TestRepoEditSelect(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.db = f"{self.tmp.name}/f.db"
|
||||
os.environ["PRAGENT_FEEDBACK_DB"] = self.db
|
||||
os.environ["PRAGENT_BOT_TOKEN"] = ""
|
||||
conn = feedback.init(self.db)
|
||||
feedback.record_review(conn, repo="o/r", pr=1, head_sha="x")
|
||||
conn.close()
|
||||
|
||||
self.port = _free_port()
|
||||
from http.server import ThreadingHTTPServer
|
||||
self.srv = ThreadingHTTPServer(("127.0.0.1", self.port), dash.Handler)
|
||||
self.thread = threading.Thread(target=self.srv.serve_forever, daemon=True)
|
||||
self.thread.start()
|
||||
self.auth_hdr = {"X-Forwarded-User": "marcos@example.com"}
|
||||
|
||||
def tearDown(self):
|
||||
self.srv.shutdown()
|
||||
self.srv.server_close()
|
||||
self.thread.join(timeout=2)
|
||||
for k in ("PRAGENT_FEEDBACK_DB", "PRAGENT_BOT_TOKEN"):
|
||||
os.environ.pop(k, None)
|
||||
self.tmp.cleanup()
|
||||
|
||||
def test_repo_page_renders_select_with_one_option_per_price(self):
|
||||
status, _h, body = _get(self.port, "/r/o/r", headers=self.auth_hdr)
|
||||
self.assertEqual(status, 200)
|
||||
text = body.decode()
|
||||
self.assertIn('<select id="model" name="model">', text)
|
||||
# Every PRICES key should appear as an <option value="…">.
|
||||
for k in sorted(cost_model.PRICES):
|
||||
self.assertIn(f'<option value="{k}"', text, f"missing {k} in select")
|
||||
# Plus the "keep current" placeholder.
|
||||
self.assertIn("— keep current", text)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -11,16 +11,10 @@ import eval_judges as ej # noqa: E402
|
||||
|
||||
# --- rule_body ------------------------------------------------------------
|
||||
|
||||
def test_rule_body_targets_traces():
|
||||
"""Trace target matches the path `/api/public/ingestion` triggers.
|
||||
|
||||
Observation rules only fire from the OTel ingestion pipeline; this
|
||||
pilot uses standard ingestion, so its jobs only come from
|
||||
`evalService.createEvalJobs` and that dispatcher handles
|
||||
`targetObject ∈ {TRACE, DATASET}`.
|
||||
"""
|
||||
def test_rule_body_targets_observations():
|
||||
"""Trace-level rules wouldn't see observation input/output."""
|
||||
body = ej.rule_body("rule-x", "finding_actionability", 1.0)
|
||||
assert body["target"] == "trace"
|
||||
assert body["target"] == "observation"
|
||||
assert body["enabled"] is True
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user