storytold/indd-utils
2
0
Fork 0
mirror of https://github.com/storytold/indd-utils.git synced 2026-11-06 18:42:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(render_compare): correct line-box offsets per page

Browse Source 
The document-wide median per pair of box heights follows pagination
drift on other pages. Each page uses its own median for a pair with at
least 3 matched lines and falls back to the document's median otherwise.
Add --reuse DIR to measure the tool on the renders of an earlier run.
This commit is contained in:
aiongg
2026-10-10 15:13:03 +00:00
parent 3122adfee5
commit f21267f1a8
2 changed files with 103 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/measurement.md
+22 -1
View File
@@ -410,10 +410,30 @@ all boxes equal) and compared:
| Mean difference | Mean absolute difference of the images after a 2 × 2 mean, from 0 to 1 |
| Missing ink, extra ink | Share of pixels with ink (value below 200) in one image and none within 2 pixels in the other, after the 2 × 2 mean |
| Lines | `pdftotext -bbox-layout` lines; a line's key is its text without white space, casefolded. Lines with the same key are matched, nearest first |
| Moved line | A matched line whose left edge or top differs by more than 1 pt. Line boxes depend on the ascent and descent each PDF declares for its fonts, so between R and A the top is first corrected by the document's median difference for lines of the same pair of box heights |
| Moved line | A matched line whose left edge or top differs by more than 1 pt. Between R and A the top is first corrected by a line-box offset (below) |
| Character overlap | Share of the characters of a page (without white space) that the other page also has, counted as a multiset |
| Font-complete | DesignCraft matches every font family of the document exactly (`font.list`); a page of another document is font-complete when every font `pdffonts` lists for the page in R belongs to such a family |
**Line-box offset (R vs A).** A line's box depends on the ascent and
descent each PDF declares for its fonts, so the same line can have a
different top in R and A without moving. Matched lines are grouped by
their pair of box heights (R, A). Each page corrects a group by the
median vertical difference of that group's lines on the page. A group
with fewer than 3 lines on the page takes the median of the group over
the whole document instead: with 3 lines the median is one of the lines
and one line that moved does not change it, while with 2 lines it is
their mean and a moved line shifts both. The per-page median keeps
pagination drift on other pages out of a page's correction. A shift
that most lines of a group share on one page is not counted as moved.
Evidence, on the same renders of 170 same-save documents (2026-10):
with the document's median alone, 17,066 of 42,932 matched lines count
as moved and 1,003 pages as `lines moved`; with the per-page median,
11,786 lines and 905 pages. On one page whose lines have the same
coordinates in two renderer builds, the document's median changed from
11.6 to 73.6 pt between the builds and 29 of 30 matched lines counted as
moved; with the page's median, none do.
**First differing page.** In A vs B, the first page that is not
pixel-identical is where the converter's output first changes the
rendering; later pages often only reflow. For that page the tool lists
@@ -467,6 +487,7 @@ and the Python standard library only. Options:
| `--link-fonts` | Give DesignCraft the font files a package keeps beside the IDML, in a `Document Fonts` folder; DesignCraft reads only that folder |
| `--bin PATH` | Another converter binary |
| `--gaps PATH` | compare.py's `gaps.tsv` to rank first-difference keys against (default `target/compare/gaps.tsv`) |
| `--reuse DIR` | Take A.pdf, B.pdf and DesignCraft's results from `work/` of an earlier run in DIR (which may be `--out` itself) for each triple whose converted IDML has the same bytes as in that run; render the others, which needs `--designcraft`. For measuring a change to the tool on the same renders: pass the earlier run's `--bin` |
| `--images ID:PAGE…` | Write images of pages of the last run (below) and nothing else |
Run it after converter changes, like `diff_outputs.py`: a page whose A vs
tools/render_compare.py
+81 -22
View File
@@ -18,7 +18,7 @@ variable).
Usage: python3 -I tools/render_compare.py --designcraft PATH [--bin PATH]
[--jobs N] [--limit N] [--file SUBSTR] [--exclude PREFIX]...
[--versions 13-21] [--max-pages N] [--link-fonts] [--out DIR]
[--gaps GAPS_TSV]
[--gaps GAPS_TSV] [--reuse DIR]
python3 -I tools/render_compare.py --images ID:PAGE [ID:PAGE...] [--out DIR]
Run from the repository root after `cargo build --release`. Needs
@@ -34,6 +34,12 @@ triples. --max-pages limits the page-by-page comparison of each document
package keeps beside the IDML, in a `Document Fonts` folder (DesignCraft
reads only that folder).
--reuse DIR takes A.pdf, B.pdf and the renderer's results from the work/
folder of an earlier run in DIR (which may be --out itself) instead of
rendering, for each triple whose converted IDML has the same bytes as in
that run; other triples are rendered, which needs --designcraft. Use it
to measure a change to this tool on the same renders.
Outputs go to --out (default target/render-compare): triples.tsv,
docs.tsv, pages.tsv, lines.tsv, dropped.tsv, issues.tsv, summary.json, and
work/ID/ with the converted IDML, A.pdf, B.pdf and the renderer's results.
@@ -69,6 +75,9 @@ import compare # noqa: E402
ROOT = compare.ROOT
CORPUS = ROOT / "corpus"
DPI = 50
# Fewest matched lines of one pair of box heights on a page for the page's
# own median offset; see box_offsets.
MIN_PAGE_LINES = 3
PX = DPI / 72.0
RENDER_TIMEOUT = 600
CONVERT_TIMEOUT = 120
@@ -457,8 +466,8 @@ def line_stats(ref, oth, offsets=None):
"""Line match of a page. A matched line moved when its left edge or the
top of its box differs by more than 1 pt. Line boxes come from the
fonts' ascent and descent as each PDF declares them, so between R and A
the top of the box is first corrected by `offsets`: the document's
median difference for lines of the same box heights."""
the top of the box is first corrected by `offsets`, the page's offsets
from box_offsets."""
pairs, unref, unoth = match(ref, oth)
moved = []
for i, j in pairs:
@@ -476,13 +485,24 @@ def line_stats(ref, oth, offsets=None):
def box_offsets(pairs_by_page):
"""Median vertical difference of matched lines per (reference box
height, other box height) over a document."""
groups = defaultdict(list)
for ref, oth, pairs in pairs_by_page:
"""{page: {(reference box height, other box height): offset}}: the
median vertical difference of the page's matched lines with those box
heights, or the document's median for them when the page has fewer than
MIN_PAGE_LINES such lines. A document-wide median alone follows
pagination drift on other pages: where most lines of a document moved,
it shifts the lines of pages that did not move."""
doc, pages = defaultdict(list), {}
for pno, ref, oth, pairs in pairs_by_page:
groups = defaultdict(list)
for i, j in pairs:
groups[(ref[i]["h"], oth[j]["h"])].append(oth[j]["ty"] - ref[i]["ty"])
return {k: statistics.median(v) for k, v in groups.items()}
for k, v in groups.items():
doc[k].extend(v)
pages[pno] = groups
doc = {k: statistics.median(v) for k, v in doc.items()}
return {pno: {k: statistics.median(v) if len(v) >= MIN_PAGE_LINES else doc[k]
for k, v in groups.items()}
for pno, groups in pages.items()}
def line_checks(lines):
@@ -877,15 +897,48 @@ def ab_flags(p):
return out
def sha256(path):
return hashlib.sha256(path.read_bytes()).hexdigest()
def reuse(cfg, work, res):
"""With --reuse, take A.pdf, B.pdf and the renderer's results from the
earlier run when its converted IDML has the same bytes; True if taken."""
if not cfg["reuse"]:
return False
src = Path(cfg["reuse"]) / "work" / work.name
try:
prev = json.loads((src / "run.json").read_text())
same = prev.get("ours_sha256")
if same is None and src.resolve() != work.resolve():
same = sha256(src / "ours.idml")
except (OSError, ValueError):
return False
pdfs = [src / "A.pdf", src / "B.pdf"]
if same != res["ours_sha256"] or not all(f.exists() for f in pdfs) \
or "A" not in prev or "B" not in prev:
return False
if src.resolve() != work.resolve():
for f in pdfs:
link = work / f.name
link.unlink(missing_ok=True)
link.symlink_to(f.resolve())
res["A"], res["B"], res["reused"] = prev["A"], prev["B"], str(src)
return True
def measure(cfg, t):
"""Render A and B and compare R, A and B page by page."""
work = Path(cfg["out"]) / "work" / t["id"]
ref_idml, r_pdf = CORPUS / t["idml"], CORPUS / t["pdf"]
res = {"id": t["id"]}
a_doc = package(ref_idml, work / "pkgA", ref_idml, cfg["link_fonts"])
res["A"] = designcraft(cfg, a_doc, work / "A.pdf")
b_doc = package(ref_idml, work / "pkgB", work / "ours.idml", cfg["link_fonts"])
res["B"] = designcraft(cfg, b_doc, work / "B.pdf")
res = {"id": t["id"], "ours_sha256": sha256(work / "ours.idml")}
if not reuse(cfg, work, res):
if not cfg["designcraft"]:
raise RuntimeError("renders not reusable and no --designcraft")
a_doc = package(ref_idml, work / "pkgA", ref_idml, cfg["link_fonts"])
res["A"] = designcraft(cfg, a_doc, work / "A.pdf")
b_doc = package(ref_idml, work / "pkgB", work / "ours.idml", cfg["link_fonts"])
res["B"] = designcraft(cfg, b_doc, work / "B.pdf")
(work / "run.json").write_text(json.dumps(res, indent=1))
pdfs = {"R": r_pdf, "A": work / "A.pdf", "B": work / "B.pdf"}
info = {k: boxes(p) for k, p in pdfs.items() if p.exists()}
@@ -940,7 +993,7 @@ def measure(cfg, t):
for p in rows:
r, a, b = lines("R", p["page"]), lines("A", p["page"]), lines("B", p["page"])
if r is not None and a is not None:
ra_pairs.append((r, a, match(r, a)[0]))
ra_pairs.append((p["page"], r, a, match(r, a)[0]))
if a is not None and b is not None:
p["AB_text"] = line_stats(a, b)
offsets = box_offsets(ra_pairs)
@@ -950,7 +1003,7 @@ def measure(cfg, t):
pno = p["page"]
r, a = lines("R", pno), lines("A", pno)
if r is not None and a is not None:
p["RA_text"] = line_stats(r, a, offsets)
p["RA_text"] = line_stats(r, a, offsets.get(pno, {}))
p["fontok"] = all_ok or (res["A"]["exit"] == 0 and all(
font_ok(families, f) for f in page_fonts(r_pdf, pno)))
p["RA_flags"], p["AB_flags"] = ra_flags(p), ab_flags(p)
@@ -1285,6 +1338,7 @@ def report(args, cfg, judged, measured, secs):
},
"designcraft": {
"renders": 2 * len(measured),
"renders_reused": 2 * sum(1 for m in measured if m.get("reused")),
"render_failures": sum(1 for m in measured for k in ("A", "B") if m[k]["exit"] not in (0, "timeout")),
"render_timeouts": sum(1 for m in measured for k in ("A", "B") if m[k]["exit"] == "timeout"),
"render_seconds_median": statistics.median(times) if times else None,
@@ -1345,6 +1399,8 @@ def main():
ap.add_argument("--out", default=str(ROOT / "target" / "render-compare"))
ap.add_argument("--gaps", default=str(ROOT / "target" / "compare" / "gaps.tsv"),
help="compare.py's gaps.tsv, to rank first-difference keys against")
ap.add_argument("--reuse", metavar="DIR",
help="take the renders of an earlier run in DIR where the IDML is the same")
ap.add_argument("--images", nargs="+", metavar="ID:PAGE",
help="write images of these pages of an earlier run, and nothing else")
args = ap.parse_args()
@@ -1354,21 +1410,24 @@ def main():
if args.images:
images(args)
return
if not args.designcraft:
if not args.designcraft and not args.reuse:
ap.error("--designcraft PATH (or DESIGNCRAFT_CLI) is required")
out.mkdir(parents=True, exist_ok=True)
tmp = out / "tmp"
tmp.mkdir(exist_ok=True)
dc = Path(args.designcraft).resolve()
_, _, version, _ = run([str(dc), "--version"], 60, text=True)
cfg = {"bin": str(Path(args.bin).resolve()), "designcraft": str(dc), "out": str(out.resolve()),
"tmp": str(tmp.resolve()), "link_fonts": args.link_fonts, "max_pages": args.max_pages,
dc = Path(args.designcraft).resolve() if args.designcraft else None
version = run([str(dc), "--version"], 60, text=True)[2].strip() if dc else None
cfg = {"bin": str(Path(args.bin).resolve()), "designcraft": str(dc) if dc else None,
"out": str(out.resolve()), "tmp": str(tmp.resolve()), "link_fonts": args.link_fonts,
"max_pages": args.max_pages,
"reuse": str(Path(args.reuse).resolve()) if args.reuse else None,
"tools": {"converter": args.bin,
"repository": subprocess.run(["git", "-C", str(ROOT), "describe", "--always",
"--dirty"], capture_output=True,
text=True).stdout.strip(),
"designcraft": version.strip(),
"designcraft_sha256": hashlib.sha256(dc.read_bytes()).hexdigest()[:16]}}
"designcraft": version,
"designcraft_sha256": sha256(dc)[:16] if dc else None,
"renders_reused_from": str(Path(args.reuse).resolve()) if args.reuse else None}}
start = time.time()
todo = candidates(args)
results = {}