The filter bar matches on metadata only — not on input and not on the item id —
so a dataset seeded with repo/pr in `input` alone could not be sliced by repo
at all. Every facet worth filtering on is now a flat primitive in `metadata`:
repo, owner, repo_name, pr, head_sha, finding_count, has_findings,
max_severity, reviews_run and the review timestamp both ways. `owner` is split
out because a filter on the joined repo matches one repo, never a whole org,
and `max_severity` is "none" rather than absent because an absent key matches
no filter.
`eval_experiment.py` links reviews that already ran into a dataset run, one run
per model, so the Experiments tab is populated without re-running the reviewer.
One trace per (run, item), the most recent: a PR re-reviewed on every push has
many traces and a run is one output per input.
It posts to the deprecated /api/public/dataset-run-items — the notice exempts
self-hosted v3 from the cutoff date and the pilot is stdlib-only by design.
Revisit at v4.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Items were keyed `{repo}#{pr}`, e.g. `netcracker/interview#29`. Both
characters break the UI's item route: the `/` in `owner/repo` splits into
extra path segments, and everything after the `#` is a fragment the browser
never sends. Items were created successfully and then 404'd when opened.
Ids are now `{owner}__{repo}__pr{n}`, which needs no percent-encoding. The
real repo and pr stay intact in `input`, so nothing downstream reads the id
back apart. Session ids elsewhere keep the `{repo}#{pr}` form — those are
never path segments and feedback_scores depends on that shape.
The 28 existing items were unusable and are regenerable from feedback.db;
they were deleted and recreated under the new ids.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Marcos