Files
huppiflupp bff4be14c3 M10: Recognize text finds small print on large pages (tiled detection)
## What this changes

Recognize text now finds small print on large, dense pages (newspapers, small print, blurred photos).

The ocrs detection model reads a fixed 600 × 800 pixel input, so ocrs shrinks every page to that size before detecting words. A full page at 300 dpi (2480 × 3508) is shrunk about four times, and small letters disappear. With this change:

- **When (size):** an axis is cut in two when the page is at least twice the detector's input along it (width ≥ 1200 px, height ≥ 1600 px for the ocrs models). Any full page at 300 dpi becomes 2 × 2 tiles; a landscape page may become 2 × 1. The input size is read from the detection model when it loads. Pages below that size (72 to 150 dpi renders, screenshots) are read exactly as before and cost nothing extra.
- **When (text size):** a page that passes the size rule is tiled only if its text is small once the page is shrunk to the detector's input, decided by a cheap pixel signal, the **line pitch** (`crates/ocr/src/pitch.rs`). The page is box-averaged to a grey copy of at most 1600 px a side and cut into a 6 × 6 grid of blocks. In each block, the ink is summed per pixel row, the slow trend (shading, pictures) is removed with a moving average, and the autocorrelation is taken. The block autocorrelations are averaged, and the first strong peak is the pitch. Lines in columns that don't line up still agree on the period, and blank blocks are skipped. The same runs on pixel columns for sideways pages, and the axis with the stronger peak wins. The pitch is converted to detector pixels (pitch × 800 / page height). The rule:
  - pitch **under 9 detector pixels**: tile;
  - ink but **no clear line spacing** (no autocorrelation peak above 0.05): tile, because the measurement showed these pages (blurred, creased, oblique or moiré photos) gain on average;
  - large print, and blank pages: detect whole, as before.
  It costs 6.6 ms per page on average (median 5.1 ms, max 45 ms) and reads every pixel once.
- **How:** tiles overlap by 4 % of the page on each side of a cut. Words are **detected** tile by tile. A word is kept only by the tile whose core cell (its share of the page without the overlap) holds its centre, so a word in an overlap counts once. The words are moved back to page pixels, and then lines are formed and **recognised on the whole page** as before. Lines that cross a cut stay single lines, and recognition reads the same pixels it read before.
- The edge cells reach to infinity, so every finite word centre belongs to exactly one tile, including boxes that stick out past the page.

The change is small and lives in `crates/ocr/src/tiles.rs` plus a `detect_words` step in `Ocr::recognize`. While I was there I also made the existing RGBA length check use checked arithmetic (`width * height * 4` was a `u32` product).

### Measured

MDPBench de/es/fr (480 pages), official evaluation with CDM off, `en` recogniser. I ran the same harness build with and without the change (the baseline reproduces the earlier one exactly):

| | Digital | Photo | All | Time per page |
|---|---|---|---|---|
| before | 78.9 | 41.5 | 50.8 | 0.39 s (190 s total) |
| tiling every large page (this PR's first commit) | 80.4 | 42.8 | 52.2 | 0.58 s (281 s total) |
| **tiling only small or unclear print (this PR)** | **81.2** | **42.9** | **52.5** | 0.54 s (259 s total) |
| earlier prototype (full OCR per tile, outside the app) | 81.4 | 42.4 | 52.1 | n/a |

The three rows of this PR come from one harness binary with the rule switched by an environment variable. Each was timed twice (190/191, 281/281, 259/259 s). Per language, the line-pitch rule against tiling every large page: DE 62.9 vs 62.7, ES 47.9 vs 47.8, FR 46.8 vs 46.1. It tiles **214 of the 457** pages that pass the size rule.

**How the rule was chosen.** I measured first, in Python on the per-page scores of the official evaluation (`overall` per page, base vs. tiled). Per-page score gain from tiling, by measured pitch:

| pitch (detector px) | pages | mean gain | pages gaining / losing > 0.02 |
|---|---|---|---|
| < 4 | 40 | +0.018 | 10 / 4 |
| 4–6 | 40 | **+0.114** | 24 / 2 |
| 6–10 | 82 | +0.004 | 10 / 6 |
| 10–14 | 80 | **−0.013** | 5 / 26 |
| 14–35 | 144 | −0.003 | 22 / 25 |
| no clear line spacing | 82 | +0.029 | – |

The gain sits at small pitch, and pitches of 10–14 detector pixels (ordinary body text on a full page) **lose** a little when tiled. Picking tiled or base per page by this rule, and running the official evaluation on that mix, predicted exactly what the Rust build then scored (52.5 / 81.2 / 42.9). Thresholds between 8 and 10 give 52.4–52.5. Treating "no clear spacing" as "don't tile" instead gives 52.0 with only 132 pages tiled. The choice is tuned on these 480 pages, so take the last few tenths with that in mind.

**Time.** The saving is smaller than the page count suggests. The pages that stay tiled are the dense ones, where tiling finds many more words, and recognising those words costs more than the three extra detector runs. Tiling every large page cost +91 s. The pitch rule costs +69 s, so it removes about a quarter of the overhead and still scores higher.

Word recall in the benchmark's own per-kind table, before → after: de digital 61.9 → 78.5 %, fr digital 17.8 → 24.5 %, de/es/fr photos 38.1/37.7/10.1 → 42.1/43.2/12.6 %. Sideways photos are unchanged; those need rotation, which is a separate PR.

Detection runs four times instead of once on a tiled page (machine: Ryzen AI Max+ 395, CPU).

### How it composes with the open OCR PRs

- #5157 (reading order) ends `recognize` in `layout::reading_order`. This PR changes only the detection step before `find_text_lines`, so tiles are merged before lines are formed and before reading order runs, which is the order we want. I measured both together: the numbers above come from a harness tree that has #5157's reading order applied. The two diffs touch neighbouring lines in `recognize`, so whichever merges second needs a trivial rebase.
- #5171 (layout regions) and #5058 (line padding) work on lines and regions after detection, so they compose the same way.

## How you verified it

- [x] `cargo fmt --check`
- [x] `cargo clippy -p pdfcraft-ocr --all-targets -- -D warnings`
- [x] `cargo test -p pdfcraft-ocr`: 26 passed, **with the OCR models installed** (CI has none, so CI skips the model-backed tests). With the models, I also ran the OCR tests of `pdfcraft-engine` (`ocr`, `recogni` filters) and `pdfcraft-automation` (`ocr`, `scan`) locally, and all pass.
- [x] `cargo xtask layers`, `cargo xtask assets`, `cargo xtask parity`

**Tests.**
- Unit tests, no models (`crates/ocr/src/tiles.rs`): the tile geometry for small pages and for A4 at 300 dpi; every word centre, on cuts, edges and outside the page, belongs to exactly one tile, and NaN or infinity belongs to none; cropping, including out-of-range and overflowing crops, returns `None` without panicking.
- Unit tests, no models (`crates/ocr/src/pitch.rs`), on synthetic A4 pages at 300 dpi: a blank page is not tiled; large print (lines every 100 px, 22.8 detector px) is not; small print (every 30 px, 6.8 detector px) is, in grey, RGB and RGBA alike; sideways small print (lines running down the page) is; noise (ink without lines) is; the first strong autocorrelation peak is the pitch, not its multiple; empty, 1-pixel, short or long buffers, 0 or 5 channels, overflowing sizes, a zero-sized detector and slivers give an answer without panicking.
- Model-backed test `finds_small_print_on_a_large_page`: an A4 page at 300 dpi with four lines of small print. One sits inside a tile, one crosses the vertical cut, one is half in the overlap, and one crosses the horizontal cut. Each line must come back as exactly two words at the right place in page pixels.
- **Seen failing by reverting the fix:**
  - Detection on the whole page instead of tiles: 5 words instead of 8, unreadable ("FIOWOR", "In3").
  - Keeping every tile's words without the centre-in-core rule: 14 instead of 8, because the lines in the overlaps doubled.
  - Finite edge cells: `every_word_centre_belongs_to_exactly_one_tile` fails.
  - Never cutting: the geometry tests fail.
  - Line-pitch rule, each mutation reverted afterwards: tiling every page (the first commit's behaviour) fails `a_blank_page_is_not_tiled` and `large_print_is_not_tiled`. Never tiling on a measured pitch fails `small_print_is_tiled` and `sideways_small_print_is_tiled`. Rows only, without the column profile, fails `sideways_small_print_is_tiled`. Taking the strongest peak instead of the first strong one fails `the_first_strong_peak_is_the_pitch`. Not tiling unclear pages fails `ink_without_lines_is_tiled`. No detrending fails four tests. With the threshold lowered to 5 detector pixels, the model-backed `finds_small_print_on_a_large_page` finds 5 words instead of 8, so it exercises the rule too (its page measures 5.8).

## Never-crash

- [x] No new `unwrap`/`expect`/`panic!`/`unreachable!`/`todo!` outside tests
- [x] No indexing or slicing with input-derived positions: crops use checked arithmetic and `get`; tile bounds are clamped to the page; float-to-int casts are of values clamped to `0..=len`. The pitch measure checks that the buffer matches width × height × channels with checked arithmetic, reads pixels through iterators and `get`, and works on at most 1600 × 1600 values whatever the page size. It divides only by non-zero counts, so it returns an answer for any input.
- [x] No `unsafe`
- [x] n/a, this fixes no crash

## Provenance

- [x] No asset from an Adobe product
- [x] No font files
- [x] No GPL/AGPL code
- [x] Behaviour comes from our own measurements on MDPBench. The approach (overlapping tiles, keep by centre; line pitch from the autocorrelation of a projection profile) is a generic technique, not read from another implementation.
- [x] No new assets. One new dependency edge, `rten-imageproc` 0.26 (MIT/Apache-2.0), which was already in the tree through `ocrs`.

## Security

- [x] No path handling changed
- [x] No MCP or control channel changes
- [x] No network access

## Parity and reach

- [x] `ocr.recognize-text` notes say when pages are tiled (size and line pitch), and that the model-backed test runs only where the models are installed. The tiling and pitch unit tests are evidence.
- [x] Reachable headlessly: this is inside `Ocr::recognize`, so `ocr_recognize` and `ocr_recognize_files` get it unchanged.
- [x] No documented behaviour changed

## Anything a reviewer should look at first

- **The tiling rule:** an axis is cut only if each half still fills the detector's input, so a tile is never padded. A lower threshold would also tile mid-size pages. I kept it conservative so that small pages cost nothing. The prototype tiled every page; under this rule, 468 of the 481 benchmark images are cut along at least one axis and 337 along both.
- **The line-pitch rule (`pitch.rs`).** Its constants (1600 px working size, 6 × 6 blocks, peak floor 0.05, first peak ≥ 60 % of the strongest, 9 detector px) were chosen on MDPBench as described above. "No clear line spacing → tile" is deliberate. My first idea was the opposite, but the measurement favoured tiling.
- **Detect per tile, recognise on the whole page**, rather than full OCR per tile like the prototype. This is simpler: there are no lines to stitch back together and no rows to re-sort. It scores about the same as the prototype (all 52.2 vs 52.1, photo 42.8 vs 42.4, digital 80.4 vs 81.4).
2026-11-18 16:33:38 +07:00
..
…

pdfcraft-ocr

Layer L4 (may use fonts for the text layers' metrics): Scan & OCR — recognising the words in a page raster and turning them back into page content (execution plan M10).

// The caller renders a page to pixels and hands them over:
let image = OcrImage::new(w, h, rgba)?;                  // validated by construction
let lines = recognizer.recognize(&image, &options)?;     // Vec<Line>: words in reading order
let placed: Vec<PlacedWord> = …;                         // PlacedWord::place maps boxes to user space
// The editable output draws real glyphs: plan the words, register the display faces into the
// plan (`pdfcraft_edit::add_type3_fonts`), then generate the layer. An unregistered plan falls
// back to WinAnsi, and characters no registered face has become '?'.
let plan = TextPlan::for_words(placed.iter().map(|w| w.text.as_str()));
add_type3_fonts(&mut doc, page, &mut plan)?;             // names the plan's fonts on the page
let layer = visible_text_layer(&placed, &plan);          // visible text: the editable output
// The searchable layer needs no display fonts: the engine's own layer (engine::ocr::layer)
// keeps every Unicode character with empty Type3 programs and a ToUnicode map beyond WinAnsi.
  • Engines behind one trait. A [Recognizer] reports its id, whether it is available, the languages it reads, and reads Vec<Line> from an image. Two exist: the pure-Rust OcrsRecognizer (the ocrs crate, MIT/Apache-2.0, with its pre-trained models, CC-BY-SA-4.0, fetched by cargo xtask models; found via $PDFCRAFT_MODELS or models/ beside the executable — see ATTRIBUTION.toml), and — off the web build — tesseract::TesseractCli, which drives an installed tesseract binary as an external process: never linked, bundled or downloaded. Implementations are shared across threads (Send + Sync); nothing spawns on wasm32.
  • The tesseract process is fenced in. The program comes from a user-configured location or the PATH, never from document data (the engine validates a configured path as an absolute path to an existing executable file before it is used). The language code is allow-listed (tesseract::language_code, 12 codes) before anything reaches a process; the page image goes to a private temp file (a fresh 0700 directory holding a 0600 file on Unix) that is deleted afterwards; the run has a 120 s timeout, a 64 MiB output cap and a 50 000-word reading cap (an incomplete reading is rejected rather than applied); every failure comes back through OcrError with its cause named. A configured location that is invalid fails closed (it is never replaced by the PATH search). Availability is probed with --list-langs (its list is read from stdout, or from stderr where older versions print it); a failed probe is remembered for 30 s, so an install that appears later is found without a restart. Known limit: at the timeout or the output cap only the direct child is killed; a wrapper script's own children may outlive it.
  • Confidence drives review. One classifier (band_for) puts a word in the High (≥ 90), Medium (70–89) or Low (< 70) band — the review overlay's colours, the legend and the counts; the batch flag rule (low_confidence, default threshold 60) is deliberately a separate threshold. The ocrs models report no confidence (its words carry None); a reading whose confidence is unknown is treated as unknown, never as good.
  • Two engines can be merged (ROVER). Strategies: PrimaryOnly (the default), and two ensembles — ConfidenceWeighted (the secondary engine joins when the primary's mean confidence is under 70 or unknown; an empty primary keeps the fast path) and RoverVote (both engines always run). The merge lets every primary word adopt the best-overlapping unused secondary word at IoU ≥ 0.5; the higher confidence wins text, box and confidence (a tie keeps the primary; against a primary word of unknown confidence — ocrs — the secondary wins only at 70 or more, UNKNOWN_CONFIDENCE_BAR), the survivor is named "ROVER", and unclaimed secondary words are appended under their engine's name. A merge is capped at 100 M word-pair comparisons (over it, the primary reading stands alone), and a failing partner degrades to the primary with a recorded note — a page is never skipped because a partner could not read it.
  • Preprocessing between the raster and recognition, a pure function in fixed order: DPI normalization to 300 (only when one of the steps below is on, since they are calibrated for it; with all of them off the raster is read exactly as rendered, and a scan at or above 300 is left alone), then the opt-in steps — orientation (90°/180°/270° from projection-profile banding), deskew (± 5°, shear-scored, below 0.1° is a no-op), denoise (3×3 median) and binarization (Sauvola, window 8, k 0.34). Every geometric step composes one exact affine InverseTransform, and callers map every word box back to the original raster before placing it. The heuristics are conservative and panic-free: too small to judge, blank, or no clear winner is a no-op, never a guess.
  • Searchable output preserves Unicode without display fonts. The engine uses empty Type3 character programs and /ToUnicode maps for non-WinAnsi words, leaving the scan unchanged. WinAnsi words use /PCHelv; the /OCR layer is fitted to each word's box.
  • Editable output uses display glyphs. visible_text_layer writes visible text whose size comes from the box height and whose horizontal scaling uses real font advances. Beyond WinAnsi, attributed craft-fonts faces supply outline Type3 glyphs with /ToUnicode. Uncovered characters become ? and are reported in editable output notes; searchable output retains them. The engine refuses nonfinite geometry or oversized input before planning fonts: 4 MiB of text and 65,536 words, across all pages for editable export. Empty geometry is skipped; rotated words carry across/up vectors and retain direction.
  • Hostile input is rejected, not panicked at. OcrImage holds private, validated state: non-zero sides, a 10 000-pixel side cap, a 64-megapixel area cap (checked arithmetic — the counts come from documents, so they are untrusted), and an exact buffer length.

Where it is wired

The engine's ocr module renders pages into OcrImages, runs jobs (OcrJob::run) and applies the words: searchable-image mode draws the invisible layer over the page's own image and leaves that image untouched; editable mode writes a NEW <name>_ocr.pdf of visible text (the scan image is not carried over, the source is never overwritten). Skip probes keep pages — and whole files — that already contain text out of the work unless Force OCR overrides them. A searchable result applies as one undoable step; the Editable text output opens the OCR Verify screen first, and Scan & OCR ▸ Correct recognized text reviews any recognition there before it is applied (confidence-coloured boxes, corrections, a per-language user dictionary — a plain-text file in the preferences folder, so words added from a document persist on disk outside it until removed — Re-OCR of a region or a page, Accept/Reject), a multiple-file run writes searchable copies as it goes, and the automation tools ocr_recognize, ocr_words, ocr_status and ocr_recognize_files expose recognition headlessly. ocr_words with retain_review returns a bounded session-local handle; ocr_review_apply applies text corrections at the stored original boxes through its stale-document guard, or discards the result. Empty corrections remove words. Searchable acceptance is one undo step; editable acceptance writes a separate file. ocr_dictionary reads or atomically updates the UI's per-language dictionary format through an explicitly root-confined path.

Not yet: languages beyond the tested set (the ocrs models read only unaccented Latin; the Tesseract side is bounded by its allow-list and whatever its installed language packs cover), OCR in the browser (the tesseract module is compiled out for wasm32), the deep scan-cleanup options (descreen, background removal, edge-shadow removal), and editable glyph coverage beyond the available attributed faces.