fix(eval): dataset item ids that survive a URL path

Items were keyed `{repo}#{pr}`, e.g. `netcracker/interview#29`. Both
characters break the UI's item route: the `/` in `owner/repo` splits into
extra path segments, and everything after the `#` is a fragment the browser
never sends. Items were created successfully and then 404'd when opened.

Ids are now `{owner}__{repo}__pr{n}`, which needs no percent-encoding. The
real repo and pr stay intact in `input`, so nothing downstream reads the id
back apart. Session ids elsewhere keep the `{repo}#{pr}` form — those are
never path segments and feedback_scores depends on that shape.

The 28 existing items were unusable and are regenerable from feedback.db;
they were deleted and recreated under the new ids.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Marcos
2026-08-31 15:38:44 +00:00
parent d9eb4d9822
commit 1534a99630
2 changed files with 110 additions and 1 deletions
+15 -1
View File
@@ -111,6 +111,20 @@ def ensure_score_configs() -> dict:
# 2. Dataset from recorded reviews
# ---------------------------------------------------------------------------
def item_id(repo: str, pr) -> str:
"""A dataset-item id that survives being put in a URL path.
The obvious `{repo}#{pr}` is unusable: the UI routes items as
`/datasets/{id}/items/{item_id}`, so the `/` in `owner/repo` splits into
extra path segments and everything after the `#` is a fragment the browser
never sends. The item is created fine and then 404s when opened.
Session ids elsewhere keep the `{repo}#{pr}` form — those are never path
segments, and `feedback_scores` depends on that shape.
"""
return f"{repo.replace('/', '__')}__pr{pr}"
def read_review_items(db_path: str) -> list[dict]:
"""One dataset item per (repo, pr) the reviewer has run on.
@@ -140,7 +154,7 @@ def read_review_items(db_path: str) -> list[dict]:
).fetchall()
items.append(
{
"id": f'{row["repo"]}#{row["pr"]}',
"id": item_id(row["repo"], row["pr"]),
"input": {
"repo": row["repo"],
"pr": int(row["pr"]),