storytold/lightcraft
2
0
Fork 0
mirror of https://github.com/storytold/lightcraft.git synced 2026-11-07 19:03:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

Culling: the loupe uses built previews; photos prepared ahead and in the background (#3077)

Browse Source 
* PERF-CULL-0: ui.inspect reports how the loupe showed each photo it was stepped to

perf.loupeSteps records, per step, the ms from the step to the first frame
with anything of the photo, with a full-size picture (camera JPEG, cached
or rendered) and with LightCraft's own render, measured on the frames the
app draws, so culling benchmarks exclude the control channel's round trip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 95f0343848fc0a238f80f07ceb292eaf6779aeaa)

* PERF-CULL-1: browsing a folder shows its first photos within a second

The Local view read and hashed every file of the folder before showing any
of them (2,933 EOS R5 C CR3s: 30.6 s to the first photo). It now hands the
files the library lacks straight to the import, ordered the way the view
sorts them (file time for the date sorts), which adds them batch by batch:
first photo 0.9 s, twelve thumbnails 1.4 s, the whole folder in the same
~34 s as before, in the background.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit b766558e7ef4f802919a144da53e444f70e0186b)

* PERF-CULL-2: the loupe makes the next photos' camera previews ahead of the step

Culling steps through photos, and each step waited for the next raw's
embedded JPEG to be read and decoded (p50 187 ms, p95 332 ms on EOS R5 C
CR3s) and then for a full raw develop. Now:

- the loupe's stand-ins for the next five photos in the direction of
  travel and the one behind are made ahead (Slot::QuickAhead, priority
  just below the loupe's own render) and taken over at the step, so a
  step shows the camera JPEG at once (p50 0 ms);
- a slot keeps its photo while the photo stays wanted, so photos that
  shift places as the user steps are not made twice;
- decoded camera JPEGs stay in the memory LRU (never on disk), so going
  back shows them again without a decode;
- while an arrow key is held (a photo left within 0.25 s), the raw
  develop waits until the photo has stayed 0.12 s, and a render queued
  for a photo already left is dropped, so the workers make the previews
  instead of renders nobody sees.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 0430b95636d2b6adce8887af7b9464687c95549e)

* PERF-CULL-3: CR3 previews take the smallest embedded JPEG that covers the view; plain sRGB JPEGs decode straight to 8-bit

Upstream's embedded_preview_covering (14e0d128) picks the smallest covering
JPEG of a TIFF-based raw; this extends it to Canon CR3 and adds the 8-bit
decode:
- A CR3 keeps an 8192 px JPEG track, a 1620 px PRVW box and a 160 px THMB.
  embedded_preview_covering now picks among the three (cr3_boxes lists the
  boxes), so a grid thumbnail or a loupe up to 1620 px decodes the PRVW.
  Loader at 1280 px on 20 EOS R5 C CR3s: 168-322 ms -> 31 ms a photo.
- lightcraft_codecs::decode_jpeg_srgb8_fit decodes a plain 8-bit sRGB
  JPEG (DCT-scaled) and box-averages it in linear light straight into
  8-bit sRGB, without the float image the general path builds; anything
  with an ICC profile, an Adobe RGB declaration, CMYK or 12-bit takes the
  general path. 45 MP preview fitted to 2560 px: 182 -> 151 ms (min of
  15, loaded machine).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit a67bb398c2f438cbe16f4db1431ae0270f1c8b04)

* PERF-CULL-4: Camera preview label; thumbnails ahead of the step; 100 % shows the camera JPEG first

- While the loupe shows a raw's embedded JPEG (or its thumbnail), a small
  'Camera preview' label at the top right says so, with what that means
  on hover (widget loupe:cameraPreview; nine languages), so colour and
  tone are not judged on the camera's rendering.
- The look-ahead also asks for the next photos' thumbnails (a small
  embedded JPEG each), so a held arrow key has something to show in full
  screen too, where no filmstrip asks for them (2400x1500 window, 40 held
  steps: camera JPEG on 36, nothing on 2; before 1 and 4). Stepping
  through a whole wedding keeps at most 600 thumbnails, those around the
  open photo.
- Zoomed in on an unedited raw, its camera JPEG at full size (8192 px for
  an EOS R5 C) is drawn under the window render until that arrives:
  100 % in ~0.9 s instead of ~5.6 s; released when not zoomed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 9f212b1e16815c3c0ca33130319a7ef41fb05a26)

* PERF-CULL-5: lightcraft-cli bench-cull and tools/cull-bench.py time culling a real shoot

bench-cull FOLDER (headless) times reading the folder into the Local view,
the first screen of grid thumbnails on the app's render threads, camera
previews at 1280 / 2560 px and full size, a cold 2560 px loupe render,
rating 50 photos in a row and importing N files in place / copying M,
as 'name: X ms wall' lines (p50, p95, n) and JSON. Libraries are made
under --work; nothing is written next to the photos.

tools/cull-bench.py drives the running app over the control channel
(browse, first photo, steps at rest, a held arrow key, 100 % zoom,
rating) and reads the app's own frame timings (perf.loupeSteps).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit bd2eaac591b72f8258acae4389c67aa4d1da3b27)

* PERF-CULL-5: bench-cull --files K samples a slow folder; e2e test; cull-bench usage

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit c32c9556924ccdee6f8c103864df51c3e170df98)

* PERF-CULL: docs for culling speed (parity rows, gap 26, preview reads, opening a photo)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit fb69b5975b51f882d82b2ca684e0f7ddc8373908)

* PERF-CULL: the whole-file preview decodes a plain sRGB camera JPEG the quick path's way

Upstream's quick-path test wants the bounded read and the whole file to give the same picture
byte for byte; with the branch's straight-to-8-bit decode on the quick path only, a 49 px
preview came out differently. Both paths now take it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 0c8acab4c2f3e85d1c51fab5faf62b6e997d7c73)

* PERF-THUMB-FIRST: CR3 thumbnails decode the 1620 px PRVW copy, and grid thumbnails from camera JPEGs are kept in the thumbnail cache

A CR3 grid thumbnail decoded the full-size JPEG track (45 MP, ~150 ms each)
whatever size was asked; it now takes the smallest embedded copy that covers
the cell (PRVW, same shape), and the box search stops at mdat (was a scan of
the whole file). The decoded camera JPEG of an unedited raw is written to
the library's thumbnail cache, so a folder seen before fills from small
files. Measured on 250 R5 C raws (loaded machine): first screen fills in
294 ms (was 553), on a second visit 53 ms (was 610).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 12162f966dd859b7e53ac65b5c4564a8e837b7eb)

* PERF-CULL: cargo fmt

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: docs for culling speed (parity rows LR-IMP-LOCAL, LR-IMP-PREVIEW-READ, LR-VIEW-DETAIL; gap 26)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: review fixes (bench work folder, 8-bit APP1 cap, size sort)

- bench-cull makes its libraries in a new folder of the run's own under
  --work and never deletes what is already there (it removed lib-browse,
  lib-add and lib-copy under a folder the caller named).
- The 8-bit sRGB JPEG path refuses more than 128 APP1 segments before its
  decoders walk them again, as the general decode does.
- Browsing by file size keeps unreadable files last in both directions
  (they were sized 0, so first when ascending).
- --files K in the CLI synopsis and help.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: the loupe uses built previews; photos are prepared ahead and in the background

Built standard previews were never what the loupe showed: it rendered
every photo again (~1.8 s on a raw), the rendered previews in memory were
capped at 128 MB (~84 photos) whatever the budget, the disk cache was a
fixed 2 GB (a wedding's previews pruned each other), the cache's writer
skipped disk writes when it fell behind, and Build Previews with one photo
open built only that photo though the menu said "Visible".

- The loupe's render job takes a cached view render of the same settings
  that is at least as big as asked for, fitted down (no render).
- Memory for rendered previews is a quarter of the cache share; the disk
  cache's automatic size makes room for every photo (about 1.5 MB each);
  Settings offers Automatic, 50 GB and 200 GB.
- A build skips photos whose previews are there, writes each to disk
  before counting it, builds at least the loupe's size, and with one photo
  selected builds the view.
- Prepare photos in the background (Settings > Performance), per mode
  (culling, library, editing), app-wide (kept in ui.json) or per library:
  photos ahead rendered for the loupe (a few at a time, 0 = off) and the
  whole view's previews built in the spare time once the user steps
  through photos: two lanes, waiting while the photo on screen renders,
  nearest first in the direction of travel, one cancellable row, replaced
  by a build that is asked for. Commands prefs.backgroundPrep /
  prefs.setBackgroundPrep; ui.inspect perf.previewCache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: the bench work-folder test removes only a folder it made

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: browsing keeps the whole listing for the folder bookkeeping; cull-bench stops on small folders

The browse worker queues only the files the library lacks, but the folder
bookkeeping (browse times, records of files gone) needs every file listed:
a subfolder of known photos only was not stamped as browsed. The known
files now come along marked as already in the library. tools/cull-bench.py
stops waiting at min(12, photos) thumbnails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: review fixes (smart previews honour id, editing builds by the Library row, lenient prep settings)

- library.smartPreviews with an explicit id works on that photo, not the
  whole view.
- While editing, the whole view's background build follows the Library
  row (its switch and which photos); the Editing row's scope is for the
  photos kept ready ahead.
- The app-wide and per-library preparation settings are read leniently:
  a value this build doesn't know is the defaults (app) or none (library),
  and the other settings are kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: cull-bench counts twelve thumbnails while the folder is still loading

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: a view change stops the background build for the old view

The running background build was only re-aimed at the photo on screen; after
another folder, filter or library it went on building photos no longer
shown. It is now stopped when the view's signature changes (not counted as
stopped by the user), and the view now shown is prepared.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: cull-bench picks the thumbnail threshold from the final photo count

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: the view signature covers every photo; the view-change test holds the build explicitly

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: cull-bench stops waiting for twelve thumbnails 30 s after the folder loaded

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: in the marked scope, marking a photo changes what the view's build prepares

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: cull-bench times only the thumbnails the browse adds

Thumbnails already cached before library.browse no longer count towards
the first-thumbnail and twelve-thumbnail times.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: count every thumbnail made, for the browse benchmark

thumbTextures counts resident thumbnails, and evicting old ones can hide
new ones. The renderer now also counts each grid thumbnail it stores
(perf.thumbsMade, never lowered by eviction), and cull-bench times the
browse from that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: the marked-scope test waits for its builds before cleaning up

Both preview builds finish before the test library is removed, and the
removal is checked, so no worker writes into a folder being deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: stand-ins made ahead keep their order, size and photo

Review fixes for the loupe's stand-ins and the browse benchmark:
- a stand-in still queued ahead moves to its new place in the order
  when the user steps;
- one made ahead still shows when the loupe's own request failed;
- one made ahead for a smaller loupe isn't taken over by a larger one;
- quick and full-size camera-preview keys include the photo, so a
  virtual copy gets its own;
- perf.thumbsMade counts photos given a thumbnail, not a render
  replacing a stand-in.
Each has a test that fails without it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a late stand-in never takes another photo's place in the loupe

A stand-in made ahead takes over after the loupe's own request failed
only while the loupe still wants that photo and has no other request
pending; the test covers stepping on before it arrives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a stand-in made ahead and no longer wanted is dropped when it arrives

Freeing a look-ahead slot whose job is already running now also forgets
its request, so the late result isn't kept as a hidden texture, unless
the loupe is waiting for exactly that stand-in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a running stand-in the loupe waits for keeps its look-ahead slot

When the next look-ahead no longer wants the photo the loupe has just
stepped to, the slot whose job is still making that photo's stand-in stays
reserved instead of going to a new photo, so the result still reaches
the loupe.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: CodeRabbit round 10 (browse order ties, library skip, CR3 boxes after mdat, badge once, look-ahead crop key, cli test binary)

- browse_order: import- and edit-date sorts queue new files by name (they
  tie on those keys); ties go by name in the view's direction, as the
  catalog breaks them.
- Local browsing never lists the library's own folder (browse_files, as
  scan_with skips it).
- CR3 PRVW/THMB search skips only the mdat payload: boxes after it (a moov
  holding THMB) are still found.
- The camera-preview badge and the loupe timing are settled once, after
  the window renders (no double badge with the full-size camera JPEG).
- The loupe look-ahead key includes the crop.
- bench-cull CLI tests use binary() after upstream's test split.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: CodeRabbit round 11 (browse listing errors and library root, rating ties, every CR3 PRVW, checked CR3 box math, look-ahead view key, windows wait while settling)

- browse_files: a folder that can't be listed is an error (the scan ends
  as failed, nothing is forgotten as gone), and the library's folder or
  one inside it lists nothing.
- New photos tie on rating: queued by name in the view's direction.
- Every valid CR3 PRVW/THMB is a candidate (the smallest covering one is
  picked); the box offsets use checked arithmetic.
- The loupe look-ahead is keyed to the visible list's generation (a new
  sort, filter or source changes the neighbours).
- While stepping fast, zoomed-in window renders wait like the frame, and a
  window drawn before the frame is timed as the render.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: CodeRabbit round 12 (loupe timing: the After window, over any stand-in)

In a split view only the After side's window says the raw's render is on
screen; and a window drawn over the camera JPEG counts as the render too
(the source shown is a render only when it is "render" or "cached").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: CodeRabbit round 13 (loupe timing: the Before window when the Before is shown alone)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: CodeRabbit round 14 (loupe timing: a Before window side by side with \ held is not the shown view)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: CodeRabbit round 15 (the split-view timing test asserts its precondition)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: CodeRabbit round 16 (side by side with \ held, each window stays in its pane)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a cancelled browse stops ordering its files; browsing hands back earlier probes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: the web build runs the browse job in place as FnMut (it hands the probes back)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a scan hands the probes it took back however it ends (CodeRabbit, #3077 / #3080)

- The browse worker sends a result on every path: the listing, or `Stopped` (the folder couldn't be listed, or
  another folder was chosen) with the probes `ScanInput::new` took from the session. A listing failure is never
  sent as an empty listing.
- A scan stopped without waiting (its cross, or a browse replaced by another) keeps its channel until the worker
  ends, so the probes still come back; a finished scan's probes join the session's instead of replacing them.
- A cancelled scan whose worker ended without a result is not reported as "Scan failed".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: built previews are the ones on disk; stand-ins ahead follow the preparation settings (CodeRabbit, #3077)

- A build counts a photo as built only when its previews are in the disk cache (when it has one): a loupe
  render kept only in memory, its deferred write skipped, is built again and written (`stored_dims`).
- `DiskCache::put` and `PreviewCache::put_at` say whether the file was written; a view preview the cache
  folder can't take fails the photo instead of counting it as built.
- The loupe's stand-ins ahead honour Settings > Performance for the mode: none for "Only the photo on screen"
  or no photos ahead (the queued ones are dropped), only marked photos in the marked scope. The settings are
  part of what the look-ahead was made for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: a preview for a discarded cache fails the photo; cache file sizes are read from 4 KiB first (CodeRabbit, #3077 / #3080)

- `PreviewCache::put_at` returns false for a job made before the cache was cleared or retired (nothing is
  kept), so a build that overlaps Discard Preview Cache counts that photo as failed, not built. The warning
  says which of the two happened.
- `DiskCache::dims` reads a file's first 4 KiB, and up to 64 KiB only when the size marker isn't there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: a photo another build renders counts once it is built; a write pruned away is not kept (CodeRabbit, #3077 / #3080)

- A build that meets a photo another build is rendering waits for that render, then counts the photo as
  built only when its previews are there, else renders it itself (it was counted as shared at once).
- `DiskCache::put` returns false when the pruning its own write starts removes that file (a picture bigger
  than the room pruning leaves).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: put_memory holds the cache generation through its insertion (CodeRabbit, #3080)

A clear can no longer finish between put_memory's look at the generation and its insertion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: browsing skips the library's folder by any name it has (a symlink, a path through ..) (CodeRabbit, #3077)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: rustfmt the library-folder check

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: cancelled builds stop before their test library is removed (CodeRabbit, #3077)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: the library folder is skipped by its resolved path only; lib/../outside is browsed, an unresolvable folder reports its error (CodeRabbit, #3077)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: the replaced background build is held by TestHold, not a 250 ms deadline (CodeRabbit, #3077)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a browsed folder has one spelling (. and .. taken out by name) for the view, its listing and the catalog's records; an unresolvable root inside the library is still skipped (CodeRabbit, #3055 / #3077 / #3080)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: TestHold counts its holders: one test's release doesn't free a build another test holds (CodeRabbit, #3077 / #3080)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: TestHold's release without the deprecated fetch_update

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: browsing resolves `..` the way the system does, and `.` is the current folder

`lexical_path` took `..` out by name, which is wrong on POSIX when the folder before it is a
symlink (`link/..` is the folder above where the link points). `browse_path` makes the root
absolute first (so `.` and `a/..` name the current folder, not an empty path), resolves a link
just before a `..` on POSIX, and keeps every other spelling as written (a mapped drive stays a
drive letter, matching the catalog's records). Windows takes `..` out by name, as before.
Tests: a folder link followed by `..` lists what the system lists there; `.` is the current folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: browsing resolves the folder on its worker, never on the window's thread

`browse()` called `browse_path` (which may canonicalize a link) and `is_dir()` before starting the
worker, so a link into a stalled network mount could freeze the window. The UI now spells the
folder with `browse_spelling` (`.`/`..` by name, no disk access) and shows it as given; the worker
resolves it with `browse_path`, checks it is a folder (else the scan ends as failed, with the
usual toast), and sends the resolved spelling, which the view takes before the listing arrives.
Tests: a browse whose resolution is held returns at once and waits, then reads the folder; a
missing folder fails on the worker; `browse_spelling` matches `browse_path` without links.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: the browse worker resolves the path as given, not the view's spelling (CodeRabbit, #3055 / #3077 / #3080)

On Unix `photos/link/../select` with `link` pointing at `real/day` is `real/select`; the spelling with `..`
taken out by name read `photos/select`. The view keeps the spelling until the worker answers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: bench-cull measures small folders and never writes its results among the photos (CodeRabbit, #3077)

A folder with no more photos than the grid sample was reported with empty camera-preview and loupe timings: the samples now fall back to the photos it has. A --json path in the source folder is refused before the run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: editing renders the loupe from its source; the loupe holds background work back only while its render is on the way (CodeRabbit, #3077)

In Develop a cached view render left the source undecoded until the first slider draft. foreground() was renewed every loupe frame even with the render shown, so background builds mostly waited.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: loupeSteps.total, a step cursor that survives the 256-step list (CodeRabbit, #3077)

tools/cull-bench.py sliced the list by its length, which stops growing at 256 steps, and reported no steps on repeated runs. Also documents perf.thumbsMade and perf.previewCache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: the camera preview is located once when the fast decode declines (CodeRabbit, #3077)

The general decode takes the stand-in JPEG the fast path already found instead of parsing the container again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: the uncapped preview cache warns once per session when its drive runs low

The automatic preview cache budget grows with the library and has no ceiling. When the library
opens, and at most every five minutes while previews are built, the engine reads the free space of
the drive holding the cache (fs4, MIT OR Apache-2.0, pure Rust over rustix / windows-sys); under
10 % or under 20 GB free it raises one alert per session. The desktop app shows it as a notice with
the drive's free space and the cache's size and a button to Settings > Performance, where a size
limit can be set. A drive that can't say its free space gives no alert (logged). library.info
reports lowDiskSpace; the notice is in ui.inspect's notices like the others.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: the free-space read runs off the UI thread; a queued low-disk notice goes with its library; --work outside the source folder (CodeRabbit, #3077)

- Session::check_disk_space reads the drive on a worker thread and takes the answer on a later
  call (take_disk_space_alert, library.info, the next check): a slow network share no longer
  holds up a library open or a UI frame. Test: a_slow_drive_does_not_hold_up_the_check.
- The UI keeps the low-disk notice with the library generation it was raised for and drops it
  when another library opens. Test: a_queued_notice_goes_when_another_library_opens.
- The real-drive test runs on native targets only (read is None on wasm32).
- bench-cull refuses a --work folder inside the measured folder, also one not made yet, before
  anything is created. Test: the_work_folder_never_goes_in_the_source_folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: one free-space read at a time across library switches, the cache counted on its worker; a --json link into the source folder is refused (CodeRabbit, #3077/#3080)

- A library opened while the drive's free-space read still runs keeps that read (marked stale)
  instead of starting a second worker beside it; its answer is dropped and the current library's
  drive is read once it has come. Switching libraries while a share hangs no longer piles up
  workers. Test: switching_libraries_while_a_drive_hangs_starts_no_second_read.
- When the drive is low, the preview cache is counted on the read's worker and the count travels
  with the answer: taking it on the UI thread no longer scans a big cache on a slow share.
  DiskCache::known_size reads the count without touching the disk.
  Test: the_preview_cache_is_counted_off_the_ui_thread.
- bench-cull: a --json path that exists is resolved itself (a link to a photo in the source folder
  is that photo), a link that points nowhere yet by the file writing through it would create, a
  loop of links is refused. Test: a_json_link_into_the_source_folder_is_refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: bench-cull --json replaces its file instead of writing into it, so a hard link to a photo can't overwrite the photo (CodeRabbit, #3077)

The results go to a new file next to the destination, renamed over it; the
temporary file is removed on error. Test:
a_json_hard_link_to_a_photo_leaves_the_photo_alone (fails with the old write).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: bench-cull refuses a --json entry in the source folder, also a link there to a file outside (CodeRabbit, #3080)

Writing the results replaces the --json entry itself, so a link sitting in the
source folder that points outside it would be replaced, removing that entry.
check_outputs now checks where the entry sits (its folder resolved, the final
link not followed) as well as where it resolves to.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a frame without the loupe ends its step timing, so time away from Detail never counts (CodeRabbit, #3055)

LoupeTiming::left was never called: a photo's record stayed open while the grid, Compare, Survey,
Reference, People or the slideshow was up, so dwell / any / full / sharp ms counted time outside
Detail. observe() marks the frame; finish_ui_pass ends the step when the loupe drew nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: Build Standard-Sized Previews keeps 2560 px before a loupe was drawn (CodeRabbit, #3077)

On Automatic with no loupe size known yet (a fresh session) the command left `edge` out, so the
engine built its 2048 px minimum and a typical fit view rendered again. The loupe-aware engine
size still applies once a loupe was drawn; a chosen limit is used as is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: main's merge keeps one Russian "★ Favorites" and its browse test uses the scan's end channel

Both sides had added the same key; main's new Auto Import browse test sends through the scan
channel, which this branch carries as `ScanEnd`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: stepping fast drops the queued Before render too; the thumbnail count is per photo across eviction (CodeRabbit, #3055)

While an arrow is held the Before view's raw render waits like the loupe's own, and one still
queued for a photo left behind is dropped. `thumbsMade` counts each photo once per library, so a
thumbnail evicted and made again, or a library left and opened again, doesn't count twice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: the free space is read every few minutes, not only while previews are built (CodeRabbit, #3080)

Thumbnails and the loupe's previews fill the cache too, so a drive could run low after the library
opened without the notice: the low-disk check now runs whatever fills the cache (still at most
every five minutes). The engine gains two hidden test hooks (a reader and forgetting the last
read).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: a slow free-space read shows in the activity stack and an idle window wakes for the disk check (CodeRabbit, #3077)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: background preparation advances on every logic pass, hidden ones too (CodeRabbit, #3077)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a photo the loupe opens on is never a fast step; only a step inside the loupe has a predecessor (CodeRabbit, #3077)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: the cull benchmark rates one named photo and puts its rating back (CodeRabbit, #3077)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: the cull benchmark refuses a full step history, says when thumbnails were in memory, fails on a lost key and ends a zoom that needs no region (CodeRabbit, #3055)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: a forced build renders a photo another build was rendering itself (CodeRabbit, #3080)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: the cull benchmark stops on a folder with no photos and times a zoom by the region that ended it (CodeRabbit, #3055/#3080)

- browse: a folder that stays loaded with no supported photo ends the browse loop after 3 s
  instead of waiting 30 minutes, and the run stops with a clear message before the loupe
  section instead of failing on an empty selection;
- zoom to 100 %: the time reported is the moment the loop ended on (the region's arrival, or
  the settled whole frame when no region is asked for), so a whole frame that settled before
  a region was requested no longer gives a too-short zoom time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: the cull benchmark says no photos are visible, keeps an earlier result, and records an unfinished zoom as missing (CodeRabbit, #3055/#3077/#3080)

- an empty selection after browsing means no photo is visible (a filter can hide them), not that
  the folder has none: say so and point at the filters;
- LABEL.json is no longer overwritten: an existing one stops the run before it starts unless
  --overwrite is given (the final write opens with "x" then);
- a zoom that did not finish within 60 s is written as null, not as a 60 s zoom time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a raw's preview pointer is read with checked bounds on its offset and size (FredApps, #3080)

The JPEGInterchangeFormat pointer and the undefined-tag JPEGs were sliced with `as usize` casts of
the file's own u64 numbers: safe on 64-bit (the slice is `get`, its end cut at the file's end), but
an offset past 4 GiB would wrap to a small, wrong one on a 32-bit build (wasm32). `file_span` now
converts with `try_from` and saturates the end.

Test: `a_preview_pointer_past_the_end_of_the_file_finds_no_preview` (pointers past the end, lengths
past it, u32::MAX for both; a real strip preview beside the lying pointer is still found; every
truncation of the file is safe).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: bench-cull refuses a --work or --json inside the photo folder (FredApps, #3055)

Nothing is ever written next to the photos: a --work folder or a --json file inside FOLDER (the
folder itself, a subfolder, or a path that goes there through `..`) is refused before anything is
made. Paths are compared resolved (links followed as far as they exist). The cleanup already
removes only what the run made (its own `bench-cull-run-…` folder's `lib-copy`), never --work.

Test: `work_and_json_inside_the_photo_folder_are_refused`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: a test that building previews writes nothing next to the photos (FredApps, #3077)

`building_previews_writes_nothing_next_to_the_photos`: Build Previews and the background
preparation while culling leave the shoot folder with the same files, byte for byte; previews go
only to the library's own cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: cull-bench.py prints its usage on missing arguments and refuses a dangling LABEL.json link up front (CodeRabbit, #3055/#3077)

- PORT / FOLDER missing (or PORT not a number), or an option with no value or an unreadable one,
  stop with the usage line instead of a traceback;
- the early "LABEL.json exists" check uses `os.path.lexists`, so a dangling link there is refused
  before the run, not at the final exclusive write.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a preview cut short by the end of the file is not taken (CodeRabbit, #3077)

`file_span` keeps a span the file's end cuts short only while it still holds a JPEG end marker
(an overstated length with the whole JPEG inside). A truncated JPEG passed the frame-header check,
won as the largest preview and then failed to decode, so no preview was shown.

Test: `a_preview_cut_short_by_the_end_of_the_file_is_not_taken`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-PREVIEW-PREP: the shoot-folder test waits for the background build to finish (CodeRabbit, #3077)

`building_previews_writes_nothing_next_to_the_photos` used a 5 s capped poll and then went on; it
now asserts `wait_finished` (30 s), so a late write by a slow build can't slip past the check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a cut-short preview must reach its own JPEG end marker; cull-bench refuses bad options up front (CodeRabbit, #3055/#3077/#3080)

- raw: file_span keeps a span the file's end cuts short only when walking the JPEG's marker
  segments reaches its end marker inside it, and cuts it there. An FF D9 inside an APP payload no
  longer counts (it is skipped by the segment length), and bytes that don't start with FF D8 are
  refused without scanning the rest of the file. Tests: the cut-short test gains an APP1 payload
  holding FF D9 (taken before this fix) and a whole JPEG followed by other bytes;
  the_end_of_a_jpeg_is_its_own_end_marker covers stuffed bytes, restart markers, bad segment
  lengths and every truncation.
- tools/cull-bench.py: an unknown option, an option where a value belongs (--label --overwrite),
  --steps/--hold-ms below 1, or a port outside 1-65535 stop with the usage line before the run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: cull-bench refuses a run whose steps wouldn't fit the app's loupe history (CodeRabbit, #3055)

The app keeps its last 256 loupe steps. A --steps that can't fit a run (1 + 11 + STEPS + 1) stops
before the run, and each phase checks the room left before it starts, so a hold whose first steps
were dropped is never reported as the whole hold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: bench-cull refuses ".." in --json and --work (CodeRabbit, #3544)

A ".." after a folder the run hasn't made yet can't be resolved by the source-folder check, and
the run then makes that folder: --json NEW/../FOLDER/r.json would land in the source folder.
Test: a_dot_dot_in_the_outputs_is_refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a cut-short preview needs a scan before its end and the walks share a budget; cull-bench catches dropped steps and odd arguments (CodeRabbit, #3055/#3077/#3080)

- raw: jpeg_end accepts an end marker only after a start of scan (an end with no image data
  before it is refused), and the walks of a file's pointers past its end share scan_budget, as
  scan_for_jpeg's do. Tests: a whole-looking JPEG with no scan is not taken;
  the_walks_to_a_jpegs_end_share_a_budget.
- tools/cull-bench.py: the port must be a decimal number int() takes (isdecimal, guarded); a
  FOLDER that looks like an option stops with usage; --steps goes up to 254 (only the hold's
  STEPS + 1 must fit, with one to spare); each phase starts with room to spare, so a full history
  means other steps came in and the phase stops instead of reporting a shifted window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: the camera JPEG goes only under the window render, and a stand-in decoded across a cache clear isn't kept (CodeRabbit, #3544)

- ui-egui: zoomed in, the full-size camera JPEG is painted only where the window render will go
  (window_cover), so a window capped smaller than the view leaves the developed whole-frame render
  in the strips beside it instead of an unlabelled camera JPEG. Test:
  the_camera_jpeg_covers_only_where_the_window_goes.
- engine/preview: QuickJob's camera-JPEG cache carries the cache generation it was made in and
  writes with put_deferred_at / the new put_memory_at, so a job decoding when Discard Preview
  Cache clears the cache can't put its picture back (as render jobs already do). Test:
  a_camera_jpeg_decoded_across_a_cache_clear_is_not_kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a bad build request leaves the background build alone, and preparation settings that can't be saved aren't kept (CodeRabbit, #3544)

- library.buildPreviews stops a running background build only after its own ids are known to be
  good, and a build whose thread can't start is marked finished instead of left "running".
- prep_tick clears started_for when the background build fails to start, so the view retries.
- set_prep_settings / clear_library_prep put the previous settings back when prefs.json can't be
  written. Tests: a_bad_build_request_leaves_the_background_build_running,
  a_prep_setting_that_cant_be_saved_is_not_kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a background build that can't start waits 30 s before the view tries again

Clearing `started_for` on a failed start made every tick call library.buildPreviews again while
the cause lasted (CodeRabbit on #3080). The view now waits RETRY_AFTER first. Test
`a_build_that_cant_start_waits_before_trying_again`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a build waiting to retry asks for a frame at its time; a build whose worker can't start leaves the background one running

CodeRabbit on #3077/#3080: a background build that couldn't start retried only on
the next input. The session now says how long until the retry (prep_retry_in) and the
app asks for a frame then.

CodeRabbit on #3544: a build asked for whose worker thread couldn't start cancelled the
background preparation and left nothing running. The background build is now stopped
only once the new one has started; otherwise it stays the session's build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: a retry time that has passed asks for no more frames

CodeRabbit on #3077 / #3080: once the retry deadline passed, prep_retry_in returned
zero, so a view that couldn't be tried (grid, preparation off) asked for a frame at
once on every pass and kept an idle window drawing. It returns None then.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: CodeRabbit items from #3577 that live in #3077 and #3055

- disk space: a read whose thread can't start (or whose worker ends without
  answering) is tried again after 10 s, not after the 5-minute check interval;
  a hung read is polled every 5 s after its first 5 s instead of every 250 ms
- the low-disk notice of the library before is not drawn in the frame after a
  library switch
- library.smartPreviews reads its ids before stopping the background build
- a forced build renders the grid thumbnail again instead of reading its cache
- the loupe's look-ahead asks for its stand-ins again after a renderer reset
- browse: a broken link before `..` is an error, not the link's parent folder
- the probes of two stopped scans: the later scan's wins whichever ends first
- camera JPEG fast path: a JPEG with an incomplete ICC sequence goes the
  general way; a CR3 without a JPEG track picks its box preview by size too
- tests: bench-cull compares the source photos' bytes, the disk notice test
  waits for the open's read

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* PERF-CULL: CodeRabbit round 4 on #3077 and the #3583 items in its code

- A background smart-preview request cancels the running build only once its worker started;
  a worker that can't start leaves the earlier build running and the session's (test hook).
- The notice logic looks again at the next disk check (10 s after a read that couldn't start).
- CR3 boxes: largest by pixels; every top-level mdat left out; whole-file fallback takes the
  first box of a kind; a copy without an end marker isn't chosen to cover a size; a test
  decodes the chosen copy.
- The fitted JPEG decode reads a SEF file as the image before its trailer; its doc says a
  header that can't be read is Ok(None).
- Settings: a saved cache size the list doesn't offer shows as its own option.
- bench-cull rejects a second FOLDER; a browse through `..` reads again.
- cull-bench.py: --size checked, baseline waits for renders, --skip-browse claims no folder,
  a short rest phase warns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Format the quick-full draw call

* Loupe stand-in choice moves to its own function

* Render results: the newer-draft and display checks move to their own functions

* Render results: a frame left on the GPU is shown by its own function

* Format the frame's texture entry

* Docs: one row each for the import and performance entries

* Format the quick view's key

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
This commit is contained in:
OneSlip
2026-11-05 17:32:26 +10:00
committed by GitHub
parent eaef01e986
commit 61005f3dc1
84 changed files with 7510 additions and 297 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Cargo.lock
Generated
+11
View File
@@ -2139,6 +2139,16 @@ version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aa9a19cbb55df58761df49b23516a86d432839add4af60fc256da840f66ed35b"
[[package]]
name = "fs4"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7e72ed92b67c146290f88e9c89d60ca163ea417a446f61ffd7b72df3e7f1dfd5"
dependencies = [
"rustix 1.1.5",
"windows-sys 0.61.2",
]
[[package]]
name = "futures-core"
version = "0.3.34"
@@ -3732,6 +3742,7 @@ dependencies = [
"ab_glyph",
"chrono",
"craft-docpath",
"fs4",
"harfrust 0.13.3",
"jxl-oxide",
"libc",
Cargo.toml
+2
View File
@@ -58,6 +58,8 @@ log = "0.4"
# All date/time conversion goes through chrono (no hand-rolled calendar code beyond the 400-year range extension; AGENTS.md).
chrono = { version = "0.4.45", default-features = false, features = ["std"] }
rayon = "1"
# free/total space of a drive (engine's low-disk alert); MIT OR Apache-2.0, pure Rust (rustix / windows-sys)
fs4 = { version = "1.1", default-features = false }
winapi-util = "0.1" # lightcraft-catalog reads a file's volume serial number and file index with it on Windows
proptest = "1"
bytemuck = { version = "1", features = ["derive"] }
apps/lightcraft-cli/src/bench_cull.rs
+585
View File
@@ -0,0 +1,585 @@
//! `lightcraft-cli bench-cull FOLDER`: time what culling a shoot costs, headless, on real files.
//!
//! What a wedding photographer does most is open a folder of a few thousand raws, look at each
//! one, rate it and move on. This times the engine side of each step on the files themselves (the
//! desktop app's own frame timings come from `ui.inspect` → `perf.loupeSteps`, see
//! `tools/cull-bench.py`):
//!
//! - browse: reading the folder into the Local view (`library.browse`, the whole folder);
//! - grid thumbnails: the camera JPEG at 512 px for the first screen, on as many threads as the
//! app renders with;
//! - camera preview at loupe sizes (1280, 2560) and at full size (the 100 % view), one at a time;
//! - LightCraft's render of the loupe (2560 px) from a cold source;
//! - rating: `photo.rate` on many photos in a row, journal included;
//! - import: adding N files in place to a fresh library, and copying M of them into it (`--n 0`:
//! no import).
//!
//! `--files K` reads only the first K files (by name) of the folder, as browsing reads them: a
//! sample of a folder on a network share, where reading thousands of raws takes hours.
//!
//! Each number is printed as `name: X ms wall` (median, with p95 and n), the format
//! `cargo xtask bench` parses, and the lot as one JSON object with `--json FILE`. Nothing is
//! written next to the photos: every library is made in a new folder of the run's own under
//! `--work` (default: a folder in the system temp directory; nothing already there is deleted),
//! XMP auto-write stays off, and the source folder is only read.
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::Instant;
use lightcraft_engine::Session;
use lightcraft_engine::catalog::PhotoId;
use serde_json::{Value, json};
const USAGE: &str = "lightcraft-cli bench-cull FOLDER [--files K] [--n N] [--copy M] [--work DIR] [--json FILE]";
/// Threads the app renders thumbnails with (`Renderer::default`: cores − 1, at most 6).
fn render_threads() -> usize {
std::thread::available_parallelism().map(|n| n.get().saturating_sub(1)).unwrap_or(2).clamp(2, 6)
}
/// Median, p95 and count of `ms` (milliseconds).
fn stats(ms: &[f64]) -> Value {
if ms.is_empty() {
return Value::Null;
}
let mut v = ms.to_vec();
v.sort_by(f64::total_cmp);
let at = |q: f64| v.get(((v.len() - 1) as f64 * q).round() as usize).copied().unwrap_or(0.0);
json!({"p50": round(at(0.5)), "p95": round(at(0.95)), "max": round(at(1.0)), "n": v.len()})
}
fn round(x: f64) -> f64 {
(x * 10.0).round() / 10.0
}
fn ms_since(t: Instant) -> f64 {
t.elapsed().as_secs_f64() * 1000.0
}
/// Print one result line the way `xtask bench` parses it (`name: X ms wall`).
fn line(name: &str, s: &Value) {
if let Some(p50) = s.get("p50").and_then(Value::as_f64) {
println!("{name}: {p50:.1} ms wall (p95 {} ms, n {})", s["p95"], s["n"]);
}
}
/// A new, empty folder `name` in this run's own folder; never deletes anything already there.
fn fresh_dir(root: &Path, name: &str) -> Result<PathBuf, String> {
let d = root.join(name);
std::fs::create_dir(&d).map_err(|e| format!("{}: {e}", d.display()))?;
Ok(d)
}
/// A folder of this run's own under `work`, which must not exist yet: a run never deletes or
/// reuses what is already under `--work`.
fn run_dir(work: &Path) -> Result<PathBuf, String> {
std::fs::create_dir_all(work).map_err(|e| format!("{}: {e}", work.display()))?;
let stamp = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map_or(0, |d| d.as_nanos());
static RUNS: AtomicUsize = AtomicUsize::new(0);
let n = RUNS.fetch_add(1, Ordering::Relaxed);
let d = work.join(format!("bench-cull-run-{}-{stamp}-{n}", std::process::id()));
std::fs::create_dir(&d).map_err(|e| format!("{}: {e}", d.display()))?;
Ok(d)
}
/// `take` items of `v` after the first `skip` (photos the stages before have read, so cold in the
/// file cache), or, when `v` has no more than `skip`, its first `take`: a small folder is measured
/// on photos already read rather than not at all.
fn sample_after<T>(v: &[T], skip: usize, take: usize) -> &[T] {
let from = if v.len() > skip { skip } else { 0 };
let rest = v.get(from..).unwrap_or_default();
rest.get(..take.min(rest.len())).unwrap_or_default()
}
/// Whether `path` names a file in `folder` (the source folder, which a run only reads), checked
/// twice: where its own entry sits (the folder holding it, the final link not followed: writing the
/// results replaces that entry, so a link to a file elsewhere that sits in the source folder, maybe
/// standing for a photo, would be replaced; CodeRabbit on #3080), and where it resolves to: a file
/// that is there by itself (a link to a photo there is the photo), a link that points nowhere yet
/// by where writing through it would create the file, and anything else by its nearest folder that
/// exists (a `--work` folder may not be there yet).
fn inside_source(path: &Path, folder: &Path) -> bool {
let Ok(folder) = std::fs::canonicalize(folder) else { return false };
entry_inside(path, &folder) || resolves_inside(path, &folder, 0)
}
/// Whether the folder holding `path`'s own entry (its nearest folder that exists, resolved) is in
/// `folder` (already resolved). The entry itself is not followed.
fn entry_inside(path: &Path, folder: &Path) -> bool {
let mut p = path.parent().filter(|p| !p.as_os_str().is_empty()).unwrap_or(Path::new("."));
loop {
if let Ok(c) = std::fs::canonicalize(p) {
return c.starts_with(folder);
}
match p.parent() {
Some(up) if !up.as_os_str().is_empty() => p = up,
Some(_) => p = Path::new("."),
None => return false,
}
}
}
/// [`inside_source`] for a `folder` already resolved; `hops`: links followed so far (a loop of
/// links counts as inside: refused).
fn resolves_inside(path: &Path, folder: &Path, hops: u32) -> bool {
if let Ok(c) = std::fs::canonicalize(path) {
return c.starts_with(folder);
}
if let Ok(target) = std::fs::read_link(path) {
if hops >= 40 {
return true;
}
let target = path.parent().map_or_else(|| target.clone(), |dir| dir.join(&target));
return resolves_inside(&target, folder, hops + 1);
}
entry_inside(path, folder)
}
/// Refuse `--json` or `--work` in the folder being measured: nothing is written next to the photos,
/// the results and the run's libraries included. Checked before anything is made.
fn check_outputs(folder: &Path, json: Option<&Path>, work: &Path) -> Result<(), String> {
// a `..` after a folder not made yet can't be resolved now, and the run makes that folder:
// `--json NEW/../FOLDER/r.json` would pass the checks below and land in the source folder
// (CodeRabbit on #3544)
for (name, p) in [("--json", json), ("--work", Some(work))] {
if let Some(p) = p.filter(|p| p.components().any(|c| c == std::path::Component::ParentDir)) {
return Err(format!("{}: {name} must not contain \"..\"; give the folder without it", p.display()));
}
}
if let Some(j) = json.filter(|j| inside_source(j, folder)) {
return Err(format!("{}: --json must not be in the folder being measured", j.display()));
}
if inside_source(&work.join("run"), folder) {
return Err(format!("{}: --work must not be in the folder being measured", work.display()));
}
Ok(())
}
fn library(dir: &Path) -> Result<Session, String> {
let mut s = Session::new().with_fs().with_system_clock();
s.open_library(dir, false).map_err(|e| format!("{}: {e}", dir.display()))?;
// nothing is ever written next to the photos
s.execute("library.xmpPreferences", &json!({"autoWrite": false})).map_err(|e| e.to_string())?;
Ok(s)
}
pub fn run(args: &[String]) -> Result<(), String> {
let mut folder = None;
let mut n = 2000usize;
let mut copy = 200usize;
let mut work = None;
let mut json_out = None;
let mut files = None;
let mut i = 0;
while i < args.len() {
let value = |i: usize| args.get(i + 1).cloned().ok_or_else(|| format!("{} needs a value\n{USAGE}", args[i]));
match args[i].as_str() {
"--n" => {
n = value(i)?.parse().map_err(|_| "--n needs a number".to_string())?;
i += 1;
}
"--copy" => {
copy = value(i)?.parse().map_err(|_| "--copy needs a number".to_string())?;
i += 1;
}
"--files" => {
files = Some(value(i)?.parse::<usize>().map_err(|_| "--files needs a number".to_string())?);
i += 1;
}
"--work" => {
work = Some(PathBuf::from(value(i)?));
i += 1;
}
"--json" => {
json_out = Some(PathBuf::from(value(i)?));
i += 1;
}
"-h" | "--help" => {
println!("{USAGE}");
return Ok(());
}
a if a.starts_with("--") => return Err(format!("unknown option {a}\n{USAGE}")),
// (a second folder would replace the first and be timed in its place: CodeRabbit on #3583)
a if folder.is_some() => return Err(format!("{a}: one FOLDER only\n{USAGE}")),
a => folder = Some(PathBuf::from(a)),
}
i += 1;
}
let folder = folder.ok_or_else(|| USAGE.to_string())?;
if !folder.is_dir() {
return Err(format!("{}: not a folder", folder.display()));
}
let work = work.unwrap_or_else(|| std::env::temp_dir().join(format!("lightcraft-bench-cull-{}", std::process::id())));
check_outputs(&folder, json_out.as_deref(), &work)?;
let work = run_dir(&work)?;
let mut out = json!({"folder": folder.display().to_string(), "threads": render_threads()});
// --- browse: the whole folder into the Local view -------------------------------------------
let mut s = library(&fresh_dir(&work, "lib-browse")?)?;
// `--files K`: only the first K files by name, read as browsing reads them (a sample of a slow
// network folder)
let t = Instant::now();
let (photos, ids): (u64, Vec<PhotoId>) = match files {
None => {
let r = s.execute("library.browse", &json!({"path": folder.display().to_string()})).map_err(|e| e.to_string())?;
(r["photos"].as_u64().unwrap_or(0), s.visible_cloned())
}
Some(k) => {
let mut list: Vec<String> = std::fs::read_dir(&folder)
.map_err(|e| format!("{}: {e}", folder.display()))?
.flatten()
.map(|e| e.path())
.filter(|p| p.is_file() && lightcraft_engine::import::is_supported(p))
.map(|p| p.display().to_string())
.collect();
list.sort();
list.truncate(k);
let r = s.execute("library.import", &json!({"paths": list, "mode": "add", "local": true})).map_err(|e| e.to_string())?;
let ids: Vec<PhotoId> = r["imported"].as_array().into_iter().flatten().filter_map(Value::as_u64).map(PhotoId).collect();
(ids.len() as u64, ids)
}
};
let browse_ms = ms_since(t);
let what = if files.is_some() { "first files" } else { "whole folder" };
println!("browse {photos} photos ({what}): {browse_ms:.1} ms wall");
out["browse"] = json!({"ms": round(browse_ms), "photos": photos, "perPhotoMs": round(browse_ms / photos.max(1) as f64)});
let paths: Vec<String> = ids
.iter()
.filter_map(|id| match &s.catalog.photo(*id)?.source {
lightcraft_engine::catalog::Source::File { path } => Some(path.clone()),
_ => None,
})
.collect();
if paths.is_empty() {
return Err(format!("{}: no photos", folder.display()));
}
let loader = s.media.preview_loader.clone().ok_or("no preview loader")?;
// --- grid thumbnails: the first screen, in parallel ------------------------------------------
let screen: Vec<String> = paths.iter().take(48).cloned().collect();
let next = Arc::new(AtomicUsize::new(0));
let t = Instant::now();
let per: Vec<(f64, f64)> = std::thread::scope(|sc| {
let workers: Vec<_> = (0..render_threads())
.map(|_| {
let (next, loader, screen) = (next.clone(), loader.clone(), &screen);
sc.spawn(move || {
let mut done = Vec::new();
while let Some(p) = screen.get(next.fetch_add(1, Ordering::Relaxed)) {
let a = Instant::now();
let _ = loader(p, 512);
done.push((ms_since(a), ms_since(t)));
}
done
})
})
.collect();
workers.into_iter().flat_map(|w| w.join().unwrap_or_default()).collect()
});
let first = per.iter().map(|p| p.1).fold(f64::INFINITY, f64::min);
let all = per.iter().map(|p| p.1).fold(0.0, f64::max);
let each = stats(&per.iter().map(|p| p.0).collect::<Vec<_>>());
println!("grid: first thumbnail: {first:.1} ms wall");
println!("grid: {} thumbnails: {all:.1} ms wall", per.len());
line("grid: one 512 px thumbnail", &each);
out["thumbs"] = json!({"firstMs": round(first), "allMs": round(all), "count": per.len(), "each": each});
// --- camera previews at loupe sizes, one at a time (what a step costs without look-ahead) -----
let sample = sample_after(&paths, 48, 20);
for edge in [1280usize, 2560, usize::MAX] {
let ms: Vec<f64> = sample
.iter()
.map(|p| {
let a = Instant::now();
let _ = loader(p, edge);
ms_since(a)
})
.collect();
let name = if edge == usize::MAX { "camera preview full size (100 %)".to_string() } else { format!("camera preview {edge} px") };
let st = stats(&ms);
line(&name, &st);
out[if edge == usize::MAX { "cameraFull".to_string() } else { format!("camera{edge}") }] = st;
}
// --- LightCraft's loupe render (2560 px) from a cold source --------------------------------
let ms: Vec<f64> = sample_after(&ids, 80, 5)
.iter()
.filter_map(|id| {
let job = s.loupe_job(*id, 2560, 1708, true)?;
let a = Instant::now();
let r = job.run();
let t = ms_since(a);
r.rendered.is_ok().then_some(t)
})
.collect();
let st = stats(&ms);
line("loupe render 2560 px (cold)", &st);
out["loupeRender"] = st;
// --- rating: a culling pass of keypresses ---------------------------------------------------
let ms: Vec<f64> = ids
.iter()
.take(50)
.enumerate()
.filter_map(|(k, id)| {
// the key acts on the selected photo, as in the loupe
s.execute("library.select", &json!({"ids": [id.0], "active": id.0})).ok()?;
let a = Instant::now();
s.execute("photo.rate", &json!({"rating": (k % 5) + 1})).ok()?;
Some(ms_since(a))
})
.collect();
let st = stats(&ms);
line("rate one photo (journal written)", &st);
out["rate"] = st;
drop(s);
// --- import: add in place, then copy ----------------------------------------------------------
if n == 0 {
return finish(out, json_out);
}
let add: Vec<String> = paths.iter().take(n).cloned().collect();
let mut s = library(&fresh_dir(&work, "lib-add")?)?;
let t = Instant::now();
let r = s.execute("library.import", &json!({"paths": add, "mode": "add"})).map_err(|e| e.to_string())?;
let add_ms = ms_since(t);
let added = r["imported"].as_array().map(Vec::len).unwrap_or(0);
println!("import {added} photos in place: {add_ms:.1} ms wall");
out["importAdd"] = json!({"ms": round(add_ms), "photos": added, "perPhotoMs": round(add_ms / added.max(1) as f64)});
drop(s);
if copy > 0 {
let some: Vec<String> = paths.iter().take(copy).cloned().collect();
let lib = fresh_dir(&work, "lib-copy")?;
let mut s = library(&lib)?;
let t = Instant::now();
let r = s
.execute("library.import", &json!({"paths": some, "mode": "copy", "destination": lib.join("Originals").display().to_string()}))
.map_err(|e| e.to_string())?;
let copy_ms = ms_since(t);
let copied = r["imported"].as_array().map(Vec::len).unwrap_or(0);
println!("import {copied} photos by copying: {copy_ms:.1} ms wall");
out["importCopy"] = json!({"ms": round(copy_ms), "photos": copied, "perPhotoMs": round(copy_ms / copied.max(1) as f64)});
drop(s);
let _ = std::fs::remove_dir_all(&lib);
}
finish(out, json_out)
}
fn finish(out: Value, json_out: Option<PathBuf>) -> Result<(), String> {
if let Some(p) = json_out {
write_replacing(&p, serde_json::to_string_pretty(&out).unwrap_or_default().as_bytes())?;
}
println!("{}", serde_json::to_string(&out).unwrap_or_default());
Ok(())
}
/// Write `bytes` to `path` through a new file next to it, renamed over `path`: an existing `path`
/// is replaced as a name, never written into. A hard link there shares its data with another file
/// (maybe a photo in the source folder, which [`inside_source`] can't tell), and writing into it
/// would overwrite that file too (CodeRabbit on #3077). The temporary file is removed on error.
fn write_replacing(path: &Path, bytes: &[u8]) -> Result<(), String> {
use std::io::Write;
let err = |e: std::io::Error| format!("{}: {e}", path.display());
let dir = path.parent().filter(|d| !d.as_os_str().is_empty()).unwrap_or(Path::new("."));
let name = path.file_name().ok_or_else(|| format!("{}: not a file name", path.display()))?;
static TEMPS: AtomicUsize = AtomicUsize::new(0);
let n = TEMPS.fetch_add(1, Ordering::Relaxed);
let mut tmp_name = std::ffi::OsString::from(".");
tmp_name.push(name);
tmp_name.push(format!(".{}-{n}.tmp", std::process::id()));
let tmp = dir.join(tmp_name);
let written =
std::fs::OpenOptions::new().write(true).create_new(true).open(&tmp).and_then(|mut f| f.write_all(bytes).and_then(|()| f.sync_all()));
if let Err(e) = written.and_then(|()| std::fs::rename(&tmp, path)) {
let _ = std::fs::remove_file(&tmp);
return Err(err(e));
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_second_folder_is_an_error() {
let e = run(&["a".to_string(), "b".to_string()]).unwrap_err();
assert!(e.starts_with("b: one FOLDER only"), "{e}");
}
/// `--work` may hold anything: a run makes a new folder of its own there and never deletes
/// what is already in it (it used to remove `lib-browse`, `lib-add` and `lib-copy` there).
#[test]
fn a_run_never_deletes_what_is_under_work() {
// a folder this test owns (made new), so the cleanup below removes nothing else
let work = run_dir(&std::env::temp_dir()).unwrap();
let keep = work.join("lib-browse").join("keep.txt");
std::fs::create_dir_all(keep.parent().unwrap()).unwrap();
std::fs::write(&keep, b"mine").unwrap();
let a = run_dir(&work).unwrap();
let b = run_dir(&work).unwrap();
assert_ne!(a, b);
fresh_dir(&a, "lib-browse").unwrap();
assert!(fresh_dir(&a, "lib-browse").is_err(), "an existing folder is never reused");
assert_eq!(std::fs::read(&keep).unwrap(), b"mine");
let _ = std::fs::remove_dir_all(&work);
}
/// A folder too small for the disjoint samples is measured on the photos it has (CodeRabbit on #3077).
#[test]
fn a_small_folder_is_sampled_from_its_start() {
let v: Vec<u32> = (0..100).collect();
assert_eq!(sample_after(&v, 48, 20), &v[48..68]);
assert_eq!(sample_after(&v, 90, 20), &v[90..100]);
assert_eq!(sample_after(&v[..3], 48, 20), &v[..3]);
assert_eq!(sample_after(&v[..48], 48, 5), &v[..5]);
assert!(sample_after::<u32>(&[], 48, 20).is_empty());
}
/// A file link at `at` to `target`.
#[cfg(unix)]
fn link(target: &Path, at: &Path) -> std::io::Result<()> {
std::os::unix::fs::symlink(target, at)
}
/// A file link at `at` to `target` (Windows: needs Developer Mode or the right to make links).
#[cfg(windows)]
fn link(target: &Path, at: &Path) -> std::io::Result<()> {
std::os::windows::fs::symlink_file(target, at)
}
#[cfg(not(any(unix, windows)))]
fn link(_: &Path, _: &Path) -> std::io::Result<()> {
Err(std::io::Error::other("no file links here"))
}
/// `--json` naming a link (made elsewhere) to a file in the source folder is refused: writing
/// the results would follow it and overwrite a photo (CodeRabbit on #3080). So is a link that
/// points to a file not made yet in the source folder, and a loop of links.
#[test]
fn a_json_link_into_the_source_folder_is_refused() {
let work = run_dir(&std::env::temp_dir()).unwrap();
let src = fresh_dir(&work, "src").unwrap();
let photo = src.join("a.jpg");
std::fs::write(&photo, b"photo").unwrap();
let to_photo = work.join("report.json");
if let Err(e) = link(&photo, &to_photo) {
let _ = std::fs::remove_dir_all(&work);
lightcraft_testkit::skip!(System, "can't make a file link here ({e}); Windows needs Developer Mode");
return;
}
assert!(inside_source(&to_photo, &src), "a link to a photo in the source folder");
let e = check_outputs(&src, Some(&to_photo), &work.join("elsewhere")).unwrap_err();
assert!(e.contains("--json"), "{e}");
let to_new = work.join("new.json");
link(&src.join("not-yet.json"), &to_new).unwrap();
assert!(inside_source(&to_new, &src), "a link to a file not made yet in the source folder");
let (x, y) = (work.join("x.json"), work.join("y.json"));
link(&y, &x).unwrap();
link(&x, &y).unwrap();
assert!(inside_source(&x, &src), "a loop of links");
let to_elsewhere = work.join("fine.json");
link(&work.join("out.json"), &to_elsewhere).unwrap();
assert!(!inside_source(&to_elsewhere, &src), "a link to a file outside is fine");
assert_eq!(std::fs::read(&photo).unwrap(), b"photo");
let _ = std::fs::remove_dir_all(&work);
}
/// `--json` naming a link that sits in the source folder and points to a file outside it is
/// refused: writing the results would replace the link itself, an entry of the source folder
/// (CodeRabbit on #3080). The link stays as it was.
#[test]
fn a_json_link_in_the_source_folder_to_a_file_outside_is_refused() {
let work = run_dir(&std::env::temp_dir()).unwrap();
let src = fresh_dir(&work, "src").unwrap();
let outside = work.join("out.json");
std::fs::write(&outside, b"outside").unwrap();
let in_src = src.join("a.jpg");
if let Err(e) = link(&outside, &in_src) {
let _ = std::fs::remove_dir_all(&work);
lightcraft_testkit::skip!(System, "can't make a file link here ({e}); Windows needs Developer Mode");
return;
}
assert!(inside_source(&in_src, &src), "a link in the source folder, wherever it points");
let e = check_outputs(&src, Some(&in_src), &work.join("elsewhere")).unwrap_err();
assert!(e.contains("--json"), "{e}");
assert!(std::fs::symlink_metadata(&in_src).unwrap().file_type().is_symlink(), "the link stays");
assert_eq!(std::fs::read_link(&in_src).unwrap(), outside);
assert_eq!(std::fs::read(&outside).unwrap(), b"outside");
// a new file in the source folder (nothing there yet) is refused too
assert!(check_outputs(&src, Some(&src.join("new.json")), &work.join("elsewhere")).is_err());
let _ = std::fs::remove_dir_all(&work);
}
/// `--json` naming a hard link (made elsewhere) to a photo in the source folder passes
/// [`inside_source`] (a hard link has no target to resolve), so the results replace the link
/// instead of writing into it: the photo keeps its bytes (CodeRabbit on #3077).
#[test]
fn a_json_hard_link_to_a_photo_leaves_the_photo_alone() {
let work = run_dir(&std::env::temp_dir()).unwrap();
let src = fresh_dir(&work, "src").unwrap();
let photo = src.join("a.jpg");
std::fs::write(&photo, b"photo").unwrap();
let report = work.join("report.json");
if let Err(e) = std::fs::hard_link(&photo, &report) {
let _ = std::fs::remove_dir_all(&work);
lightcraft_testkit::skip!(System, "can't make a hard link here ({e})");
return;
}
finish(json!({"browse": 1}), Some(report.clone())).unwrap();
assert_eq!(std::fs::read(&photo).unwrap(), b"photo", "the photo is unchanged");
let written: Value = serde_json::from_slice(&std::fs::read(&report).unwrap()).unwrap();
assert_eq!(written["browse"], 1);
let left: Vec<_> = std::fs::read_dir(&work).unwrap().flatten().map(|e| e.file_name()).collect();
assert_eq!(left.len(), 2, "no temporary file is left: {left:?}");
// a folder that isn't there: an error, nothing panics
assert!(write_replacing(&work.join("missing").join("r.json"), b"x").is_err());
let _ = std::fs::remove_dir_all(&work);
}
/// `--json` naming a file in the source folder is refused (CodeRabbit on #3077); elsewhere it is fine.
#[test]
fn the_results_never_go_in_the_source_folder() {
let work = run_dir(&std::env::temp_dir()).unwrap();
let src = fresh_dir(&work, "src").unwrap();
let sub = fresh_dir(&src, "sub").unwrap();
assert!(inside_source(&src.join("a.png"), &src));
assert!(inside_source(&sub.join("x.json"), &src));
assert!(inside_source(&src.join("sub").join("..").join("a.png"), &src));
assert!(!inside_source(&work.join("out.json"), &src));
let _ = std::fs::remove_dir_all(&work);
}
/// `--work` in the source folder is refused before anything is made there, also when it doesn't
/// exist yet (CodeRabbit on #3077); elsewhere it is fine.
#[test]
fn the_work_folder_never_goes_in_the_source_folder() {
let work = run_dir(&std::env::temp_dir()).unwrap();
let src = fresh_dir(&work, "src").unwrap();
let inside = src.join("out").join("deeper");
let e = check_outputs(&src, None, &inside).unwrap_err();
assert!(e.contains("--work"), "{e}");
assert!(check_outputs(&src, None, &src).is_err(), "the source folder itself");
assert!(!src.join("out").exists(), "nothing was made in the source folder");
assert!(check_outputs(&src, None, &work.join("elsewhere")).is_ok());
assert!(check_outputs(&src, Some(&src.join("r.json")), &work.join("elsewhere")).is_err());
let _ = std::fs::remove_dir_all(&work);
}
/// A `..` in `--json` or `--work` is refused: after a folder the run hasn't made yet it can't be
/// resolved now, and would lead into the source folder once the run makes it (CodeRabbit on #3544).
#[test]
fn a_dot_dot_in_the_outputs_is_refused() {
let work = run_dir(&std::env::temp_dir()).unwrap();
let src = fresh_dir(&work, "src").unwrap();
let sneaky = work.join("new").join("..").join("src").join("r.json");
let e = check_outputs(&src, Some(&sneaky), &work.join("elsewhere")).unwrap_err();
assert!(e.contains("--json") && e.contains(".."), "{e}");
let e = check_outputs(&src, None, &work.join("new").join("..").join("src")).unwrap_err();
assert!(e.contains("--work"), "{e}");
assert!(!work.join("new").exists(), "nothing was made");
let _ = std::fs::remove_dir_all(&work);
}
}
apps/lightcraft-cli/src/main.rs
+9
View File
@@ -11,11 +11,13 @@
//! lightcraft-cli commands [--json]
//! lightcraft-cli controls [--json]
//! lightcraft-cli calibrate [--max N] [--out DIR] FOLDERS/FILES…
//! lightcraft-cli bench-cull FOLDER [--files K] [--n N] [--copy M] [--work DIR] [--json FILE]
//! ```
#![forbid(unsafe_code)]
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
mod alloc_release;
mod bench_cull;
use std::io::Write;
use std::path::Path;
@@ -110,6 +112,12 @@ USAGE:
model, written as <model>.json to DIR (default: the profiles folder LightCraft reads,
<config>/camera-profiles, or $LIGHTCRAFT_CAMERA_PROFILES). Raws of a profiled model then
take their colour from the profile and only their tone from their own JPEG.
lightcraft-cli bench-cull FOLDER [--files K] [--n N] [--copy M] [--work DIR] [--json FILE]
Time what culling a shoot costs on its own files: reading the folder (browse), the first
screen of grid thumbnails, camera previews at loupe sizes and full size, a loupe render,
rating, and importing N files in place (default 2000) and copying M (default 200). Libraries
are made in a new folder under DIR (default: the temp folder); nothing is written next to the
photos. --files K reads only the first K files by name (a sample of a slow network folder).
lightcraft-cli --version | --help
";
@@ -176,6 +184,7 @@ fn main() -> ExitCode {
Some("controls") => controls(&args[1..]),
Some("denoise-serve") => denoise_serve(&args[1..]),
Some("calibrate") => calibrate(&args[1..]),
Some("bench-cull") => bench_cull::run(&args[1..]),
Some("--version" | "-V" | "version") => {
println!("lightcraft-cli {}", env!("CARGO_PKG_VERSION"));
Ok(())
apps/lightcraft-cli/tests/it/cli.rs
+69
View File
@@ -101,6 +101,75 @@ fn gradient_png(path: &std::path::Path) {
std::fs::write(path, png).unwrap();
}
/// PERF-CULL-5: `bench-cull` times a folder end to end and writes nothing next to the photos.
#[test]
fn bench_cull_times_a_folder_and_leaves_it_untouched() {
let dir = tmp("bench-cull-src");
std::fs::create_dir_all(&dir).unwrap();
for (seed, name) in ["a.png", "b.png", "c.png"].iter().enumerate() {
// three different pictures (the same bytes would import as one photo)
let img = lightcraft_raster::Rgba8::from_fn(120, 80, |x, y| [(x * 2) as u8, (y * 3) as u8, (seed * 80) as u8, 255]);
let png = lightcraft_codecs::encode_png(&lightcraft_codecs::EncodeImage::rgba8(&img), &Default::default()).unwrap();
std::fs::write(dir.join(name), png).unwrap();
}
// every file's name and bytes: nothing added, and no photo written into (CodeRabbit on #3577)
let listing = |d: &std::path::Path| {
let mut v: Vec<(String, Vec<u8>)> = std::fs::read_dir(d)
.unwrap()
.map(|e| e.unwrap())
.map(|e| (e.file_name().to_string_lossy().to_string(), std::fs::read(e.path()).unwrap()))
.collect();
v.sort();
v
};
let before = listing(&dir);
let work = tmp("bench-cull-work");
let json_out = tmp("bench-cull.json");
let r = Command::new(binary())
.arg("bench-cull")
.arg(&dir)
.args(["--n", "3", "--copy", "1", "--work"])
.arg(&work)
.arg("--json")
.arg(&json_out)
.output()
.unwrap();
let stdout = String::from_utf8_lossy(&r.stdout);
assert!(r.status.success(), "{stdout}{}", String::from_utf8_lossy(&r.stderr));
assert!(stdout.contains("browse 3 photos") && stdout.contains("ms wall"), "{stdout}");
let v: Value = serde_json::from_str(&std::fs::read_to_string(&json_out).unwrap()).unwrap();
assert_eq!(v["browse"]["photos"], 3);
assert_eq!(v["importAdd"]["photos"], 3);
assert_eq!(v["importCopy"]["photos"], 1);
assert_eq!(v["rate"]["n"], 3);
// a folder smaller than the samples is measured on the photos it has, not reported empty
for stage in ["camera1280", "camera2560", "cameraFull", "loupeRender"] {
assert_eq!(v[stage]["n"], 3, "{stage}: {}", v[stage]);
}
assert_eq!(listing(&dir), before, "nothing written next to the photos");
// results asked for in the source folder: refused, the photo untouched
let photo = std::fs::read(dir.join("a.png")).unwrap();
let r = Command::new(binary())
.arg("bench-cull")
.arg(&dir)
.args(["--n", "0", "--work"])
.arg(&work)
.arg("--json")
.arg(dir.join("a.png"))
.output()
.unwrap();
assert!(!r.status.success() && String::from_utf8_lossy(&r.stderr).contains("--json"), "{}", String::from_utf8_lossy(&r.stderr));
assert_eq!(std::fs::read(dir.join("a.png")).unwrap(), photo);
assert_eq!(listing(&dir), before);
// a sample of the folder, no import
let r = Command::new(binary()).arg("bench-cull").arg(&dir).args(["--files", "2", "--n", "0", "--work"]).arg(&work).output().unwrap();
let stdout = String::from_utf8_lossy(&r.stdout);
assert!(r.status.success() && stdout.contains("browse 2 photos (first files)") && !stdout.contains("import"), "{stdout}");
// a bad folder is an error, not a panic
let r = Command::new(binary()).args(["bench-cull", "/no/such/folder"]).output().unwrap();
assert!(!r.status.success() && String::from_utf8_lossy(&r.stderr).contains("not a folder"));
}
#[test]
fn calibrate_reads_raf_inputs_and_reports_bad_files() {
let input = tmp("invalid.RAF");
apps/lightcraft/src/main.rs
+1
View File
@@ -371,6 +371,7 @@ fn main() -> eframe::Result {
app.notices.extend(launch_notices);
app.notices.extend(window_notice);
app.gpu_notice = true;
app.disk_space_notice = true;
// an empty library opens where Import is offered (a grid, with the sidebar); before
// the snapshot below, so this alone isn't written to the settings file
if problem.is_none() && files.is_empty() {
crates/codecs/src/jpeg.rs
+207
View File
@@ -862,3 +862,210 @@ mod tests {
}
}
}
/// [`crate::decode_jpeg_srgb8_fit`]: a plain 8-bit sRGB JPEG decoded (DCT-scaled when that still
/// covers the box) and area-averaged in linear light straight into 8-bit sRGB, without the
/// float image the general path builds (for a 45 MP camera preview fitted to 2560 px that float
/// image and its conversions were half the time). `Ok(None)` for anything else.
pub(crate) fn srgb8_fit(bytes: &[u8], max_w: u32, max_h: u32) -> Result<Option<(lightcraft_raster::Rgba8, u16)>> {
// (a Samsung SEF trailer read as the file it stands for, as the general decode does: CodeRabbit on #3583)
let bytes = &*without_sef_trailer(bytes);
let Some(m) = parse_markers(bytes) else { return Ok(None) };
let plain = matches!(m.components, 1 | 3)
&& m.precision == 8
&& matches!(m.sof, 0xC0..=0xC2)
&& m.icc.is_none()
&& !m.icc_invalid
&& m.adobe_transform.is_none();
let summary = m.exif.as_deref().map(exif::summarize).unwrap_or_default();
if !plain || summary.adobe_rgb_hint || max_w == 0 || max_h == 0 {
return Ok(None);
}
let budget = DecodeOptions::default();
check_size(F, m.width as u64, m.height as u64, &budget)?;
// as in the general decode: the decoders below walk every APP1 again
if m.app1 > MAX_APP1 {
return Err(Error::Malformed(F, "too many APP1 markers".into()));
}
let scale_to = (m.width >= max_w.saturating_mul(2) || m.height >= max_h.saturating_mul(2)).then_some((max_w, max_h));
let non_interleaved = matches!(m.sof, 0xC0 | 0xC1) && m.first_scan_components < m.components;
let raw =
if scale_to.is_some() || non_interleaved { decode_jpeg_decoder(bytes, &m, scale_to, &budget)? } else { decode_zune(bytes, &m, &budget)? };
let Buf::U8(px) = &raw.buf else { return Ok(None) };
let ch = match raw.model {
Model::Rgb => 3,
Model::Gray => 1,
Model::Cmyk => return Ok(None),
};
let (w, h) = (raw.width, raw.height);
if w == 0 || h == 0 || px.len() < w.saturating_mul(h).saturating_mul(ch) {
return Err(Error::Malformed(F, "short output".into()));
}
// the size `lightcraft_raster::resample::fit` gives: within the box, aspect kept, never larger
let s = (max_w as f64 / w as f64).min(max_h as f64 / h as f64).min(1.0);
let tw = ((w as f64 * s).round() as usize).clamp(1, w);
let th = ((h as f64 * s).round() as usize).clamp(1, h);
let orientation = summary.orientation.filter(|o| (1..=8).contains(o)).unwrap_or(1);
Ok(Some((area_average_srgb8(px, w, h, ch, tw, th), orientation)))
}
/// Source pixels and weights each output pixel along one axis covers: a box of the scale's width
/// (`n` source pixels onto `out`), partly covered pixels at its ends weighted by their share.
fn box_taps(n: usize, out: usize) -> Vec<(usize, Vec<f32>)> {
let scale = n as f64 / out as f64;
(0..out)
.map(|o| {
let (a, b) = (o as f64 * scale, ((o + 1) as f64 * scale).min(n as f64));
let first = (a.floor() as usize).min(n - 1);
let last = ((b.ceil() as usize).max(first + 1)).min(n);
let ws: Vec<f32> =
(first..last).map(|j| ((b.min(j as f64 + 1.0) - a.max(j as f64)).max(0.0) / (b - a).max(f64::MIN_POSITIVE)) as f32).collect();
(first, ws)
})
.collect()
}
/// `px` (`w × h`, `ch` = 1 or 3 interleaved 8-bit sRGB samples) averaged over boxes in linear
/// light into `tw × th` opaque 8-bit sRGB pixels. A copy when the size doesn't change.
fn area_average_srgb8(px: &[u8], w: usize, h: usize, ch: usize, tw: usize, th: usize) -> lightcraft_raster::Rgba8 {
use lightcraft_color::transfer::{decode_srgb8, encode_srgb8};
let mut out = lightcraft_raster::Rgba8::new(tw, th);
let sample = |x: usize, y: usize, c: usize| px.get((y * w + x) * ch + if ch == 3 { c } else { 0 }).copied().unwrap_or(0);
if (tw, th) == (w, h) {
lightcraft_raster::par_rows(&mut out.data, tw, |y, row| {
for (x, o) in row.iter_mut().enumerate() {
*o = [sample(x, y, 0), sample(x, y, 1), sample(x, y, 2), 255];
}
});
return out;
}
let lut: [f32; 256] = std::array::from_fn(|i| decode_srgb8(i as u8));
let xs = box_taps(w, tw);
let ys = box_taps(h, th);
// one source row filtered across into linear RGB at the output width
let across = |sy: usize, buf: &mut Vec<[f32; 3]>| {
buf.clear();
let Some(src) = px.get(sy * w * ch..(sy + 1) * w * ch) else {
buf.resize(tw, [0.0; 3]);
return;
};
for (x0, wx) in &xs {
let mut a = [0f32; 3];
if let Some(span) = src.get(x0 * ch..(x0 + wx.len()) * ch) {
for (p, &k) in span.chunks_exact(ch).zip(wx) {
match p {
[r, g, b] => {
a[0] += lut[*r as usize] * k;
a[1] += lut[*g as usize] * k;
a[2] += lut[*b as usize] * k;
}
[v] => {
let l = lut[*v as usize] * k;
a = [a[0] + l, a[1] + l, a[2] + l];
}
_ => {}
}
}
}
buf.push(a);
}
};
lightcraft_raster::par_rows(&mut out.data, tw, |y, row| {
let Some((y0, wy)) = ys.get(y) else { return };
let mut acc = vec![[0f32; 3]; tw];
let mut line = Vec::with_capacity(tw);
for (dy, &k) in wy.iter().enumerate() {
across(y0 + dy, &mut line);
for (a, l) in acc.iter_mut().zip(&line) {
a[0] += l[0] * k;
a[1] += l[1] * k;
a[2] += l[2] * k;
}
}
for (o, a) in row.iter_mut().zip(&acc) {
*o = [encode_srgb8(a[0]), encode_srgb8(a[1]), encode_srgb8(a[2]), 255];
}
});
out
}
#[cfg(test)]
mod srgb8_fit_tests {
use super::*;
fn jpeg(w: usize, h: usize, f: impl Fn(usize, usize) -> [u8; 3]) -> Vec<u8> {
let data: Vec<[u8; 4]> = (0..w * h)
.map(|i| {
let [r, g, b] = f(i % w, i / w);
[r, g, b, 255]
})
.collect();
let img = lightcraft_raster::Rgba8 { width: w, height: h, data };
crate::encode_jpeg(&crate::EncodeImage::rgba8(&img), 95, crate::ChromaSubsampling::S420, &crate::EncodeMeta::default()).unwrap()
}
/// The fast path shows what the general decode shows for a plain sRGB JPEG, at the same size.
#[test]
fn matches_the_general_decode_within_a_few_levels() {
let b = jpeg(1203, 801, |x, y| [((x * 7) ^ (y * 3)) as u8, (x / 5 + y / 3) as u8, ((x * y) % 251) as u8]);
for (mw, mh) in [(300u32, 300u32), (640, 640), (1203, 801), (5000, 5000), (500, 120)] {
let (fast, o) = srgb8_fit(&b, mw, mh).unwrap().unwrap();
let slow = crate::decode(&b, DecodeOptions::fit(mw, mh)).unwrap().to_srgb8();
assert_eq!((fast.width, fast.height), (slow.width, slow.height), "box {mw}×{mh}");
assert_eq!(o, 1);
let diff: Vec<u32> =
fast.data.iter().zip(&slow.data).flat_map(|(a, b)| (0..3).map(move |c| (a[c] as i32 - b[c] as i32).unsigned_abs())).collect();
let mean = diff.iter().sum::<u32>() as f64 / diff.len() as f64;
assert!(mean < 3.0, "box {mw}×{mh}: mean difference {mean}");
}
}
/// Anything but plain 8-bit sRGB goes the general way; junk is an error or `None`, never a panic.
#[test]
fn declines_what_it_does_not_cover_and_never_panics() {
let b = jpeg(64, 48, |x, _| [x as u8, 0, 0]);
assert!(srgb8_fit(&b, 0, 10).unwrap().is_none());
assert!(srgb8_fit(b"not a jpeg", 10, 10).unwrap().is_none());
for cut in [0, 2, 20, b.len() / 2, b.len() - 3] {
let _ = srgb8_fit(&b[..cut], 32, 32);
}
// a 1-pixel-wide image into a box much smaller in one axis
let thin = jpeg(1, 300, |_, y| [y as u8, y as u8, y as u8]);
let (img, _) = srgb8_fit(&thin, 10, 10).unwrap().unwrap();
assert_eq!((img.width, img.height), (1, 10));
}
/// An incomplete ICC_PROFILE sequence (chunk 1 of 2) is "has a profile we can't read", not
/// "plain sRGB": the general decode handles it, as the doc says (CodeRabbit on #3577).
#[test]
fn an_incomplete_icc_sequence_goes_the_general_way() {
let b = jpeg(16, 16, |x, y| [x as u8, y as u8, 0]);
let mut seg = vec![0xFF, 0xE2, 0, 0];
seg.extend_from_slice(b"ICC_PROFILE");
seg.extend_from_slice(&[1, 2]);
seg.extend_from_slice(&[0u8; 64]);
let len = u16::try_from(seg.len() - 2).unwrap();
seg[2..4].copy_from_slice(&len.to_be_bytes());
let mut with = b[..2].to_vec();
with.extend_from_slice(&seg);
with.extend_from_slice(&b[2..]);
assert!(parse_markers(&with).unwrap().icc_invalid);
assert!(srgb8_fit(&b, 8, 8).unwrap().is_some());
assert!(srgb8_fit(&with, 8, 8).unwrap().is_none());
}
/// The 8-bit path refuses an APP1 flood before its decoders walk it again, as the general one does.
#[test]
fn an_app1_flood_is_refused_by_the_8_bit_path_too() {
let b = jpeg(16, 16, |x, y| [x as u8, y as u8, 0]);
let mut flood = b[..2].to_vec();
for _ in 0..200 {
flood.extend_from_slice(&[0xFF, 0xE1, 0x00, 0x02]);
}
flood.extend_from_slice(&b[2..]);
match srgb8_fit(&flood, 8, 8) {
Err(Error::Malformed(_, m)) => assert!(m.contains("APP1"), "{m}"),
other => panic!("unexpected: {other:?}"),
}
}
}
crates/codecs/src/lib.rs
+15
View File
@@ -169,6 +169,21 @@ pub fn decode(bytes: &[u8], opts: DecodeOptions) -> Result<Decoded> {
std::panic::catch_unwind(|| decode_unguarded(bytes, format, &opts)).unwrap_or_else(|_| Err(Error::Malformed(format, "decoder panicked".into())))
}
/// A JPEG for display, fast: fitted within `max_w × max_h` (never enlarged) as 8-bit sRGB with its
/// EXIF orientation (1..=8, not applied), the picture `decode(bytes, DecodeOptions::fit(..))
/// .to_srgb8()` gives within a few levels, in about half the time for a large camera preview.
/// `Ok(None)` when the file is not a plain 8-bit sRGB JPEG (an ICC profile or an Adobe RGB EXIF
/// hint, CMYK, 12-bit, lossless or arithmetic coding) or its headers can't be read: use
/// [`decode`], which reports a malformed header as an error. Bad scan data is an error here, and
/// nothing is ever a panic.
pub fn decode_jpeg_srgb8_fit(bytes: &[u8], max_w: u32, max_h: u32) -> Result<Option<(lightcraft_raster::Rgba8, u16)>> {
if sniff(bytes) != Some(Format::Jpeg) {
return Ok(None);
}
std::panic::catch_unwind(|| jpeg::srgb8_fit(bytes, max_w, max_h))
.unwrap_or_else(|_| Err(Error::Malformed(Format::Jpeg, "decoder panicked".into())))
}
/// What a file's headers say about its image: enough to catalogue it without decoding pixels.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Header {
crates/codecs/tests/it/header.rs
+4
View File
@@ -208,6 +208,10 @@ fn samsung_sef_trailer_without_eoi() {
assert_eq!(full.image.data, reference.image.data, "the same pixels as the file with its EOI");
let small = decode(&pano, DecodeOptions::fit(32, 32)).unwrap();
assert_eq!((small.source_width, small.source_height), (160, 96));
// the fast display decode too, scaled (where the trailer after the scan used to fail it)
let (fit, _) = lightcraft_codecs::decode_jpeg_srgb8_fit(&pano, 32, 32).unwrap().unwrap();
let (fit_ref, _) = lightcraft_codecs::decode_jpeg_srgb8_fit(&jpeg, 32, 32).unwrap().unwrap();
assert_eq!((fit.width, fit.height, &fit.data), (fit_ref.width, fit_ref.height, &fit_ref.data));
// an FF D9 inside the SEF data is not the image's end
let eoi_in_data = sef_file(scan, block, b"SEFH", 0x08E0, &[0xFF, 0xC0, 0, 0x11, 0xFF, 0xD9, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], None);
assert_eq!(read_header(&eoi_in_data).unwrap().width, 160);
crates/engine/Cargo.toml
+2
View File
@@ -49,6 +49,8 @@ lightcraft-segment = { workspace = true, optional = true }
lightcraft-fetch = { workspace = true }
# capture-time parsing for face recognition (native only)
chrono = { workspace = true }
# free space of the drive holding the preview cache (the low-disk alert; pure Rust: rustix / windows-sys)
fs4 = { workspace = true }
[features]
# AI masks with SAM 3 (Object / Describe). Off for the web and CLI builds.
crates/engine/src/cmd/library/specs.rs
+4
View File
@@ -1367,6 +1367,8 @@ pub fn specs() -> Vec<CommandSpec> {
),
// ---- persistence
cmd!(query "library.info", "Library Info", [], None, "{}", always, |s, _| {
// the answer of a free-space read that ran on a worker (lowDiskSpace below)
s.poll_disk_space();
let photos = s.catalog.len();
let albums = s.catalog.albums().count();
let (rendered_n, rendered_bytes) = s.media.rendered.mem_usage();
@@ -1402,6 +1404,8 @@ pub fn specs() -> Vec<CommandSpec> {
"lastError": lib.last_error,
// settings files that were unreadable or damaged at open (kept, defaults used)
"settingsWarnings": lib.settings_warnings,
// the drive holding the preview cache ran low this session (warned once)
"lowDiskSpace": s.disk_space_alert_raised(),
// changes applied in memory whose write failed (retried by every save)
"unsavedOps": s.unsaved().map_or(0, |u| u.0),
"unsavedError": s.unsaved().map(|u| u.1),
crates/engine/src/cmd/previews.rs
+360 -48
View File
@@ -31,6 +31,20 @@ pub struct PreviewBuild {
pub what: &'static str,
/// Why the whole build failed (e.g. the smart previews folder isn't there).
pub error: std::sync::Mutex<Option<String>>,
/// Photos whose previews were there already (from an earlier build, the loupe or the photos
/// prepared ahead): counted as done, not rendered again (PERF-PREVIEW-REUSE).
pub skipped: AtomicUsize,
/// Photos left alone because another job was building them at that moment.
pub shared: AtomicUsize,
/// Started by the background preparation ([`crate::prep`]): it yields to the photo on screen,
/// and a build the user (or an import) asks for replaces it.
pub background: bool,
/// The photos in build order (ids), and where the user is among them: the next photo built
/// is the nearest one not built yet from there, in the direction of travel
/// ([`PreviewBuild::focus_on`]).
pub order: Vec<u64>,
pub focus: AtomicUsize,
pub backward: AtomicBool,
/// Test-only: the gate this build's lanes wait at before a photo's first render (see
/// [`test_gate`]). Captured by `build` on the calling thread; production builds never have one.
#[cfg(test)]
@@ -47,10 +61,38 @@ impl PreviewBuild {
"cancelled": self.cancel.load(Ordering::Relaxed),
"what": if self.what.is_empty() { "previews" } else { self.what },
"repaired": self.repaired.load(Ordering::Relaxed),
"skipped": self.skipped.load(Ordering::Relaxed),
"shared": self.shared.load(Ordering::Relaxed),
"background": self.background,
"error": self.error(),
})
}
pub fn running(&self) -> bool {
!self.finished.load(Ordering::Relaxed)
}
/// The user is at photo `id`, moving `backward` or forward: build from there on.
pub fn focus_on(&self, id: lightcraft_catalog::PhotoId, backward: bool) {
if let Some(i) = self.order.iter().position(|p| *p == id.0) {
self.focus.store(i, Ordering::Relaxed);
self.backward.store(backward, Ordering::Relaxed);
}
}
/// Index of the next photo to build in `queue` (`None`: none left): the first one still there
/// from the focus on in the direction of travel, then the rest the other way.
pub(crate) fn next_index<T>(&self, queue: &[Option<T>]) -> Option<usize> {
let n = queue.len();
let at = self.focus.load(Ordering::Relaxed).min(n.saturating_sub(1));
let there = |i: &usize| queue.get(*i).is_some_and(Option::is_some);
if self.backward.load(Ordering::Relaxed) {
(0..=at).rev().find(there).or_else(|| (at..n).find(there))
} else {
(at..n).find(there).or_else(|| (0..at).rev().find(there))
}
}
pub fn error(&self) -> Option<String> {
self.error.lock().unwrap_or_else(std::sync::PoisonError::into_inner).clone()
}
@@ -69,9 +111,93 @@ pub(crate) fn jobs_for(s: &mut Session, id: lightcraft_catalog::PhotoId, edge: O
if e >= thumb { view.into_iter().chain(grid).collect() } else { grid.into_iter().chain(view).collect() }
}
/// The previews the build of one photo would make are all there already, big enough: its view
/// render at least as large as asked for (`view_cache` of the view job) and its grid thumbnail
/// (`cache` of the thumbnail job), on disk when the cache has one: a render the loupe kept only in
/// memory (its deferred disk write skipped) is built again, and written. Only file headers are read.
pub(crate) fn already_built(jobs: &[RenderJob]) -> bool {
!jobs.is_empty()
&& jobs.iter().all(|j| {
if let Some((cache, key)) = &j.view_cache {
let want = j.request.max_w.max(j.request.max_h) as f64;
cache.stored_dims(*key).is_some_and(|(w, h)| w.max(h) as f64 >= want * 0.99)
} else if let Some((cache, key)) = &j.cache {
cache.stored_dims(*key).is_some()
} else {
false
}
})
}
/// Photos some preview build is rendering right now, by (the cache they go to, photo id): two
/// builds (a background one being replaced, and the one replacing it) never render the same photo
/// at once.
static BUILDING: std::sync::Mutex<Vec<(usize, u64)>> = std::sync::Mutex::new(Vec::new());
/// Holds a photo in [`BUILDING`] while it is rendered.
struct Building((usize, u64));
impl Building {
/// `id` of `jobs` (the cache their results go to tells sessions apart).
fn take(id: u64, jobs: &[RenderJob]) -> Option<Building> {
let cache = jobs.iter().find_map(|j| j.view_cache.as_ref().or(j.cache.as_ref())).map_or(0, |(c, _)| Arc::as_ptr(c) as usize);
let id = (cache, id);
let mut b = BUILDING.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
if b.contains(&id) {
return None;
}
b.push(id);
Some(Building(id))
}
/// [`Self::take`], waiting while another build renders the photo (a build asked for replacing the
/// background one may meet it on the same photo): the photo is not counted until that render is over, as
/// built when it made the previews, else rendered here. With whether it had to wait; `None` when `cancel`
/// was asked for meanwhile. (On the web, with one thread, a photo held is not waited for: `Some((None, true))`.)
fn wait(id: u64, jobs: &[RenderJob], cancel: &AtomicBool) -> Option<(Option<Building>, bool)> {
if let Some(b) = Self::take(id, jobs) {
return Some((Some(b), false));
}
Self::wait_held(id, jobs, cancel)
}
/// [`Self::wait`] for a photo another build holds.
#[cfg(not(target_arch = "wasm32"))]
fn wait_held(id: u64, jobs: &[RenderJob], cancel: &AtomicBool) -> Option<(Option<Building>, bool)> {
loop {
if cancel.load(Ordering::Relaxed) {
return None;
}
std::thread::sleep(std::time::Duration::from_millis(10));
if let Some(b) = Self::take(id, jobs) {
return Some((Some(b), true));
}
}
}
/// [`Self::wait`] for a photo another build holds: on the web, with one thread, it is not waited for.
#[cfg(target_arch = "wasm32")]
fn wait_held(_id: u64, _jobs: &[RenderJob], cancel: &AtomicBool) -> Option<(Option<Building>, bool)> {
(!cancel.load(Ordering::Relaxed)).then_some((None, true))
}
}
/// Tests: hold `id` of `jobs` as a build rendering it would, until the returned value is dropped.
#[cfg(test)]
pub(crate) fn hold_building(id: u64, jobs: &[RenderJob]) -> Option<Box<dyn std::any::Any>> {
Building::take(id, jobs).map(|b| Box::new(b) as Box<dyn std::any::Any>)
}
impl Drop for Building {
fn drop(&mut self) {
BUILDING.lock().unwrap_or_else(std::sync::PoisonError::into_inner).retain(|x| *x != self.0);
}
}
/// The build's row in the activity stack (`what` as in [`PreviewBuild::what`]).
fn task_for(s: &Session, state: &PreviewBuild) -> TaskGuard {
let (kind, label) = match state.what {
"" if state.background => ("previews", "Preparing photos in the background"),
"" => ("previews", "Building previews"),
"smart previews" => ("smartPreviews", "Building smart previews"),
_ => ("smartPreviews", "Discarding smart previews"),
@@ -84,7 +210,8 @@ fn task_for(s: &Session, state: &PreviewBuild) -> TaskGuard {
/// Render one photo's jobs in order. The source the first job decodes serves the ones after it
/// (the grid thumbnail is made from the view's source instead of decoding the file again).
/// `None` when `cancel` was asked for before a job started: that photo is not counted as done.
fn run_photo_cancellable(jobs: Vec<RenderJob>, cancel: Option<&AtomicBool>) -> Option<bool> {
/// `force`: the grid thumbnail is rendered too, not taken from the cache it is written to.
fn run_photo_cancellable(jobs: Vec<RenderJob>, cancel: Option<&AtomicBool>, force: bool) -> Option<bool> {
let mut decoded: Option<crate::media::DecodedSource> = None;
let mut ok = true;
for mut j in jobs {
@@ -97,7 +224,33 @@ fn run_photo_cancellable(jobs: Vec<RenderJob>, cancel: Option<&AtomicBool>) -> O
{
j.source = crate::media::SourceRef::Loaded(Box::new(src.clone()));
}
// The view render is written to the disk cache here, before the photo counts as built,
// not left to the cache's writer (which skips writes when it falls behind): a built
// preview is on disk, for this session and the next. (Taken out of the job: it would only
// queue the write, and would serve a smaller cached view.)
// A preview that isn't kept fails the photo: it is not built (a full or unwritable cache folder, or the
// cache discarded while the photo rendered).
let keep = j.view_cache.take().map(|v| (v, j.cache_generation));
// forced: the thumbnail job's cache is only written to, after the render (a job reads its cache first;
// CodeRabbit on #3577)
let rewrite = if force { j.cache.take().map(|c| (c, j.cache_generation)) } else { None };
let r = j.run();
if let (Some(((cache, key), generation)), Ok(img)) = (rewrite, r.rendered.as_ref())
&& !cache.put_at(generation, key, Arc::new(img.image.clone()))
{
log::warn!("building previews: the grid thumbnail could not be kept");
ok = false;
}
if let (Some(((cache, key), generation)), Ok(img)) = (keep, r.rendered.as_ref())
&& !cache.put_at(generation, key, Arc::new(img.image.clone()))
{
if cache.generation() == generation {
log::warn!("building previews: the view preview could not be written to the cache folder");
} else {
log::warn!("building previews: the preview cache was discarded while the photo rendered");
}
ok = false;
}
ok &= r.rendered.is_ok();
if decoded.is_none() {
decoded = r.loaded;
@@ -109,7 +262,7 @@ fn run_photo_cancellable(jobs: Vec<RenderJob>, cancel: Option<&AtomicBool>) -> O
/// [`run_photo_cancellable`] for callers that render a photo without a cancel flag.
#[cfg(test)]
pub(crate) fn run_photo(jobs: Vec<RenderJob>) -> bool {
run_photo_cancellable(jobs, None) == Some(true)
run_photo_cancellable(jobs, None, false) == Some(true)
}
/// How many photos render side by side: as many as an export batch for standard previews (each
@@ -224,34 +377,83 @@ pub(crate) mod test_gate {
pub(crate) fn take() -> Option<Arc<PreviewGate>> {
NEXT_GATE.with(|g| g.borrow_mut().take())
}
thread_local! {
/// The next build started on this thread fails to start its worker (a thread the system
/// won't give), taken by `build`.
static FAIL_SPAWN: std::cell::Cell<bool> = const { std::cell::Cell::new(false) };
}
/// Have the next build started on this thread fail to start its worker.
pub(crate) fn fail_next_spawn() {
FAIL_SPAWN.with(|f| f.set(true));
}
/// Whether this build's worker is to fail to start (once).
pub(crate) fn take_fail_spawn() -> bool {
FAIL_SPAWN.with(std::cell::Cell::take)
}
}
fn run_all(jobs: Vec<Vec<RenderJob>>, state: &PreviewBuild, task: &TaskGuard, lanes: usize) {
/// One photo's work in a build: its id and its render jobs ([`jobs_for`]).
type PhotoJobs = (u64, Vec<RenderJob>);
/// Build every photo of `jobs` on `lanes` threads. `force`: render even the photos whose previews
/// are there. `task`: the build's own row (`None`: another job's row shows it, e.g. the import's
/// Getting photos ready).
fn run_all(jobs: Vec<PhotoJobs>, state: &PreviewBuild, task: Option<&TaskGuard>, lanes: usize, force: bool) {
let total = jobs.len();
let queue = std::sync::Mutex::new(jobs.into_iter());
let queue = std::sync::Mutex::new(jobs.into_iter().map(Some).collect::<Vec<_>>());
let work = || {
loop {
if state.cancel.load(Ordering::Relaxed) {
break;
}
let next = queue.lock().unwrap_or_else(std::sync::PoisonError::into_inner).next();
let Some(photo_jobs) = next else { break };
// the background preparation waits while the photo on screen is being made
if state.background && crate::prep::wait_for_foreground(&state.cancel) {
break;
}
let next = {
let mut q = queue.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
state.next_index(&q).and_then(|i| q.get_mut(i)).and_then(Option::take)
};
let Some((id, photo_jobs)) = next else { break };
// A cancel accepted while this lane waited for the queue (or for another lane's
// photo) stops before this photo's first render; a cancelled photo is not counted.
if state.cancel.load(Ordering::Relaxed) {
break;
}
#[cfg(test)]
if let Some(gate) = &state.gate {
gate.enter();
}
let outcome = crate::guard::catch("building previews", || run_photo_cancellable(photo_jobs, Some(&state.cancel))).unwrap_or(Some(false));
let Some(ok) = outcome else { break };
if !ok {
state.failed.fetch_add(1, Ordering::Relaxed);
if !force && already_built(&photo_jobs) {
state.skipped.fetch_add(1, Ordering::Relaxed);
} else if let Some((held, shared)) = Building::wait(id, &photo_jobs, &state.cancel) {
// another build was rendering this photo: what it made counts, if it made it (not
// for a forced build, which renders every photo itself)
if !force && shared && (held.is_none() || already_built(&photo_jobs)) {
state.shared.fetch_add(1, Ordering::Relaxed);
let done = state.done.fetch_add(1, Ordering::Relaxed).saturating_add(1);
if let Some(task) = task {
task.progress(done as u64, total as u64);
}
continue;
}
#[cfg(test)]
if let Some(gate) = &state.gate {
gate.enter();
}
let outcome =
crate::guard::catch("building previews", || run_photo_cancellable(photo_jobs, Some(&state.cancel), force)).unwrap_or(Some(false));
let Some(ok) = outcome else { break };
if !ok {
state.failed.fetch_add(1, Ordering::Relaxed);
}
} else {
// cancelled while another build was rendering it: not counted
break;
}
let done = state.done.fetch_add(1, Ordering::Relaxed).saturating_add(1);
task.progress(done as u64, total as u64);
if let Some(task) = task {
task.progress(done as u64, total as u64);
}
}
};
#[cfg(not(target_arch = "wasm32"))]
@@ -272,43 +474,99 @@ fn run_all(jobs: Vec<Vec<RenderJob>>, state: &PreviewBuild, task: &TaskGuard, la
state.finished.store(true, Ordering::Relaxed);
}
/// The long edge a standard preview is built at: [`STANDARD_EDGE`], or what the loupe asks for
/// on this screen when that is more ([`Session::loupe_edge`]), so the loupe finds what was built.
pub fn standard_edge(s: &Session) -> usize {
STANDARD_EDGE.max(s.loupe_edge.min(8192))
}
fn build(s: &mut Session, p: &Value) -> Result<Value> {
const C: &str = "library.buildPreviews";
// tests: this invocation's gate, consumed here so an early error cannot attach it to a later build
#[cfg(test)]
let gate = test_gate::take();
let edge = match str_param(p, "size").unwrap_or("standard") {
"standard" => Some(p.get("edge").and_then(Value::as_u64).map_or(STANDARD_EDGE, |e| e.clamp(256, 8192) as usize)),
"standard" => Some(p.get("edge").and_then(Value::as_u64).map_or_else(|| standard_edge(s), |e| e.clamp(256, 8192) as usize)),
"full" | "1:1" => None,
other => return Err(bad(C, format!("unknown size `{other}` (standard|full)"))),
};
if s.preview_build.as_ref().is_some_and(|b| !b.finished.load(Ordering::Relaxed)) {
return Err(bad(C, "a preview build is already running"));
let background = p.get("background").and_then(Value::as_bool).unwrap_or(false);
let force = p.get("force").and_then(Value::as_bool).unwrap_or(false);
// (`row: false`: the job that asked shows the progress in its own row)
let row = p.get("row").and_then(Value::as_bool).unwrap_or(true);
// the background preparation gives way to a build that was asked for; nothing replaces a build
// that was asked for (it carries on, and the photos it built are skipped by the next). The
// background one is stopped only once this request is known to be good (below): a request
// with bad ids leaves it running (CodeRabbit on #3544)
let mut gives_way = None;
if let Some(b) = s.preview_build.as_ref().filter(|b| b.running()) {
if b.background && !background {
gives_way = Some(b.cancel.clone());
} else {
return Err(bad(C, "a preview build is already running"));
}
}
// explicit ids, else the selection, else everything in view
let ids = if p.get("ids").is_some() || p.get("id").is_some() || !s.selection.ids.is_empty() { s.targets(p)? } else { s.visible_cloned() };
let jobs: Vec<Vec<RenderJob>> = ids.iter().map(|id| jobs_for(s, *id, edge)).filter(|j| !j.is_empty()).collect();
s.fit_disk_cache();
// explicit ids, else the selected photos (more than one: as the menu says), else everything in
// view (a single photo selected is the one open, not a choice of photos: "Visible" in the menu)
let ids = if p.get("ids").is_some() || p.get("id").is_some() || (!background && s.selection.ids.len() > 1) {
s.targets(p)?
} else {
s.visible_cloned()
};
let jobs: Vec<PhotoJobs> = ids.iter().map(|id| (id.0, jobs_for(s, *id, edge))).filter(|j| !j.1.is_empty()).collect();
// the background build stays the session's until this one has started (a worker that can't
// start leaves it running and in place: CodeRabbit on #3544)
let previous = s.preview_build.clone();
let order = jobs.iter().map(|j| j.0).collect();
#[cfg_attr(not(test), allow(unused_mut))]
let mut build_state = PreviewBuild { total: jobs.len(), background, order, ..Default::default() };
#[cfg(test)]
let state = Arc::new(PreviewBuild { total: jobs.len(), gate, ..Default::default() });
#[cfg(not(test))]
let state = Arc::new(PreviewBuild { total: jobs.len(), ..Default::default() });
{
build_state.gate = gate;
}
let state = Arc::new(build_state);
if let Some(at) = p.get("from").and_then(Value::as_u64) {
state.focus_on(lightcraft_catalog::PhotoId(at), p.get("backward").and_then(Value::as_bool).unwrap_or(false));
}
s.preview_build = Some(state.clone());
let task = task_for(s, &state);
let lanes = lanes(jobs.len(), edge.is_none());
let task = row.then(|| task_for(s, &state));
// the background preparation leaves most of the machine to the photo on screen and the user
let lanes = if background { crate::prep::background_lanes() } else { lanes(jobs.len(), edge.is_none()) };
let wait = p.get("wait").and_then(Value::as_bool).unwrap_or(false) || cfg!(target_arch = "wasm32");
if wait {
run_all(jobs, &state, &task, lanes);
if let Some(cancel) = gives_way {
cancel.store(true, Ordering::Relaxed);
}
run_all(jobs, &state, task.as_ref(), lanes, force);
return Ok(state.json());
}
#[cfg(not(target_arch = "wasm32"))]
{
// the row stays while the worker runs (it is dropped with the closure, however that ends)
let st = state.clone();
std::thread::Builder::new()
.name("lc-build-previews".into())
.spawn(move || run_all(jobs, &st, &task, lanes))
.map_err(|e| bad(C, format!("could not start: {e}")))?;
#[cfg(test)]
let started = if test_gate::take_fail_spawn() {
drop(task);
Err(std::io::Error::other("no thread (test)"))
} else {
std::thread::Builder::new().name("lc-build-previews".into()).spawn(move || run_all(jobs, &st, task.as_ref(), lanes, force))
};
#[cfg(not(test))]
let started = std::thread::Builder::new().name("lc-build-previews".into()).spawn(move || run_all(jobs, &st, task.as_ref(), lanes, force));
if let Err(e) = started {
// never started: not left "running", which would hold every later build back, and the
// build it was to replace carries on
state.finished.store(true, Ordering::Relaxed);
s.preview_build = previous;
return Err(bad(C, format!("could not start: {e}")));
}
if let Some(cancel) = gives_way {
cancel.store(true, Ordering::Relaxed);
}
}
#[cfg(target_arch = "wasm32")]
let _ = previous;
Ok(state.json())
}
@@ -476,10 +734,18 @@ fn smart(s: &mut Session, p: &Value) -> Result<Value> {
let dir = s.media.smart_dir.clone().ok_or_else(|| bad(C, "smart previews need a library on disk"))?;
let discard = p.get("discard").and_then(Value::as_bool).unwrap_or(false);
let background = p.get("background").and_then(Value::as_bool).unwrap_or(false);
if background && s.preview_build.as_ref().is_some_and(|b| !b.finished.load(Ordering::Relaxed)) {
return Err(bad(C, "a preview build is already running"));
// (as Build Previews: the photos named, or the selected ones when more than one is, else
// everything in view; read first, so a request with bad ids stops nothing: CodeRabbit on #3577)
let ids = if p.get("ids").is_some() || p.get("id").is_some() || s.selection.ids.len() > 1 { s.targets(p)? } else { s.visible_cloned() };
// (the background preparation gives way, as to Build Previews: once this one's worker has started)
let mut gives_way = None;
if background && let Some(b) = s.preview_build.as_ref().filter(|b| b.running()) {
if b.background {
gives_way = Some(b.cancel.clone());
} else {
return Err(bad(C, "a preview build is already running"));
}
}
let ids = if p.get("ids").is_some() || !s.selection.ids.is_empty() { s.targets(p)? } else { s.visible_cloned() };
let mut jobs: Vec<SmartJob> = Vec::new();
for id in ids {
let Some(ph) = s.catalog.photo(id).cloned() else { continue };
@@ -496,19 +762,35 @@ fn smart(s: &mut Session, p: &Value) -> Result<Value> {
if background {
let what = if discard { "discarding smart previews" } else { "smart previews" };
let state = Arc::new(PreviewBuild { total: jobs.len(), what, ..Default::default() });
s.preview_build = Some(state.clone());
let previous = s.preview_build.replace(state.clone());
let task = task_for(s, &state);
let st = state.clone();
std::thread::Builder::new()
.name("lc-smart-previews".into())
.spawn(move || {
if let Err(e) = smart_run(&dir, custom, discard, jobs, Some((&st, &task))) {
log::warn!("{C}: {e}");
*st.error.lock().unwrap_or_else(std::sync::PoisonError::into_inner) = Some(e);
}
st.finished.store(true, Ordering::Relaxed);
})
.map_err(|e| bad(C, format!("could not start: {e}")))?;
let work = move || {
if let Err(e) = smart_run(&dir, custom, discard, jobs, Some((&st, &task))) {
log::warn!("{C}: {e}");
*st.error.lock().unwrap_or_else(std::sync::PoisonError::into_inner) = Some(e);
}
st.finished.store(true, Ordering::Relaxed);
};
#[cfg(test)]
let started = if test_gate::take_fail_spawn() {
drop(work);
Err(std::io::Error::other("no thread (test)"))
} else {
std::thread::Builder::new().name("lc-smart-previews".into()).spawn(work)
};
#[cfg(not(test))]
let started = std::thread::Builder::new().name("lc-smart-previews".into()).spawn(work);
if let Err(e) = started {
// never started: not left "running", and the background build it was to replace carries on
// (CodeRabbit on #3077)
state.finished.store(true, Ordering::Relaxed);
s.preview_build = previous;
return Err(bad(C, format!("could not start: {e}")));
}
if let Some(cancel) = gives_way {
cancel.store(true, Ordering::Relaxed);
}
return Ok(state.json());
}
let r = smart_run(&dir, custom, discard, jobs, None).map_err(|e| bad(C, e))?;
@@ -647,7 +929,7 @@ pub fn specs() -> Vec<CommandSpec> {
"Build Smart Previews",
[],
None,
"{ids?, discard?: bool, background?: bool} — build (or discard) the smart previews of the selected photos (else all in view): compact proxies in the library that keep photos editable and exportable (at proxy size) while their originals are offline; a damaged proxy (cut short) is rebuilt → {built, repaired, removed, failed}; with background (the app's menu) the files are read and written on a worker thread → {total, done, failed, repaired, running, what} like library.buildPreviews (progress: library.previewProgress, stop: library.cancelPreviews)",
"{ids?, discard?: bool, background?: bool} — build (or discard) the smart previews of the selected photos (when more than one is selected; else all in view): compact proxies in the library that keep photos editable and exportable (at proxy size) while their originals are offline; a damaged proxy (cut short) is rebuilt → {built, repaired, removed, failed}; with background (the app's menu) the files are read and written on a worker thread → {total, done, failed, repaired, running, what} like library.buildPreviews (progress: library.previewProgress, stop: library.cancelPreviews)",
always,
smart
),
@@ -674,13 +956,43 @@ pub fn specs() -> Vec<CommandSpec> {
"Build Previews",
[],
None,
"{size?: standard (2048 px, or `edge`) | full (1:1), ids?, wait?: bool} — render the grid thumbnail and loupe view of the selected photos (else all in view) into the preview cache, in the background unless `wait` → {total, done, failed, running}",
"{size?: standard (2048 px, or the loupe's size when bigger, or `edge`) | full (1:1), ids?, wait?: bool, force?: bool, from?: id, backward?: bool, background?: bool, row?: bool} — render the grid thumbnail and loupe view of the selected photos (when more than one is selected; else all in view) into the preview cache (memory and disk), in the background unless `wait`. Photos whose previews are there already (big enough) are skipped unless `force`; the photos nearest `from` go first. `background`: the background preparation's build (two lanes, waits while the photo on screen renders; a build asked for replaces it). `row: false`: no row of its own in the activity stack (the caller's row shows it) → {total, done, failed, skipped, shared, running}",
always,
build
),
cmd!(query "library.previewProgress", "Preview Build Progress", [], None, "{} → {total, done, failed, repaired, running, cancelled, what, error} | null", always, |s, _| {
cmd!(query "library.previewProgress", "Preview Build Progress", [], None, "{} → {total, done, failed, repaired, skipped, shared, background, running, cancelled, what, error} | null", always, |s, _| {
Ok(s.preview_build.as_ref().map_or(Value::Null, |b| b.json()))
}),
cmd!(query "prefs.backgroundPrep", "Background Preparation Settings", [], None, "{} → {app, library|null, settings (in use), own: bool (this library keeps its own), building} — what is prepared in the background per mode (culling | library: previews of every photo in view, ahead = photos ahead rendered for the loupe, eyeAhead = photos ahead whose eye focus strip is worked out, scope all|marked|onScreen; develop: ahead = photos kept decoded ready to edit, scope)", always, |s, _| {
Ok(s.prep_json())
}),
cmd!(
"prefs.setBackgroundPrep",
"Set Background Preparation",
[],
None,
"{settings?: partial {culling?, library?, develop?}, library?: bool (this library's own settings; else the app-wide ones), reset?: bool (this library goes back to the app-wide settings)} → as prefs.backgroundPrep",
always,
|s, p| {
const C: &str = "prefs.setBackgroundPrep";
let library = p.get("library").and_then(Value::as_bool).unwrap_or(false);
if p.get("reset").and_then(Value::as_bool).unwrap_or(false) {
s.clear_library_prep()?;
return Ok(s.prep_json());
}
if library && s.library.is_none() {
return Err(bad(C, "no library is open"));
}
let base = if library { s.prep.library.unwrap_or(s.prep.app) } else { s.prep.app };
let next = match p.get("settings") {
Some(patch) if patch.is_object() => base.patched(patch).map_err(|e| bad(C, e))?,
Some(_) => return Err(bad(C, "settings must be an object")),
None => base,
};
s.set_prep_settings(next, library)?;
Ok(s.prep_json())
}
),
cmd!("library.cancelPreviews", "Cancel Preview Build", [], None, "{}", always, |s, _| {
if let Some(b) = &s.preview_build {
b.cancel.store(true, Ordering::Relaxed);
crates/engine/src/disk_space.rs
+577
View File
@@ -0,0 +1,577 @@
//! A once-per-session alert when the drive holding the library's preview cache runs low.
//!
//! The automatic preview cache budget grows with the library and has no ceiling
//! ([`crate::media::default_disk_cache_bytes`]): a large library can fill its drive. Rather than cap
//! it, the session looks at the drive's free space when the library opens and, at most every
//! [`CHECK_EVERY`] while the app runs (thumbnails and previews fill the cache); when it is under [`LOW_FREE_PERCENT`] of the drive or
//! under [`LOW_FREE_BYTES`], it hands out one [`LowDiskAlert`] ([`crate::Session::take_disk_space_alert`])
//! that the UI shows with a way to Settings ▸ Performance, where the cache can be given a limit. It is
//! not repeated until the next library open or app run. Reading the free space can fail (a network
//! share that doesn't say, the web build): then there is no alert, only a log line.
//! Natively the read runs on a worker thread and its answer is taken on a later call, so a slow
//! network share never holds up the UI thread that asks.
use std::path::Path;
/// The drive is low when less than this share of it (percent) is free…
pub const LOW_FREE_PERCENT: u64 = 10;
/// …or when less than this is free (20 GB), whichever comes first.
pub const LOW_FREE_BYTES: u64 = 20 << 30;
/// While a library is open, the free space is read at most this often (statvfs on the cache's
/// drive: cheap, but not something for every frame or every write).
pub const CHECK_EVERY: std::time::Duration = std::time::Duration::from_secs(5 * 60);
/// A read when the library opens that hasn't answered after this long shows in the activity stack.
const SHOW_ROW_AFTER: std::time::Duration = std::time::Duration::from_millis(200);
/// A read whose thread couldn't start (the system out of threads for a moment) is tried again this
/// soon, not after [`CHECK_EVERY`].
pub const RETRY_AFTER: std::time::Duration = std::time::Duration::from_secs(10);
/// What a read's worker answers: the drive's space and the cache's size when low (`None`: the drive
/// didn't say), or that the read couldn't start.
type Answer = Result<Option<(DiskSpace, u64)>, NotStarted>;
/// A read's thread couldn't be started.
struct NotStarted;
/// Space on one drive, in bytes.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct DiskSpace {
/// Free for this user (what can still be written).
pub available: u64,
/// The drive's size.
pub total: u64,
}
impl DiskSpace {
/// Under [`LOW_FREE_PERCENT`] of the drive free, or under [`LOW_FREE_BYTES`]. A drive that
/// reports no size isn't judged by its share.
pub fn is_low(&self) -> bool {
let under_share = self.total > 0 && u128::from(self.available) * 100 < u128::from(self.total) * u128::from(LOW_FREE_PERCENT);
under_share || self.available < LOW_FREE_BYTES
}
}
/// What the UI tells the user once: the drive's free space and size, and how big the preview
/// cache is now.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct LowDiskAlert {
pub space: DiskSpace,
/// Bytes the preview cache takes on that drive.
pub cache_bytes: u64,
}
/// The free space of the drive holding `dir` (or its parent when `dir` isn't there yet). `None`
/// (logged) when it can't be read.
pub fn read(dir: &Path) -> Option<DiskSpace> {
#[cfg(not(target_arch = "wasm32"))]
{
let probe = if dir.exists() { dir } else { dir.parent().unwrap_or(dir) };
match fs4::statvfs(probe) {
Ok(s) => Some(DiskSpace { available: s.available_space(), total: s.total_space() }),
Err(e) => {
log::info!("disk space: can't read the free space of {}: {e}", probe.display());
None
}
}
}
#[cfg(target_arch = "wasm32")]
{
let _ = dir;
None
}
}
/// Reads the free space of a drive ([`read`]; a test can put a slow one in its place).
pub type Reader = std::sync::Arc<dyn Fn(&Path) -> Option<DiskSpace> + Send + Sync>;
/// The session's state of the alert (reset when a library opens).
#[derive(Default)]
pub(crate) struct DiskAlertState {
/// The alert was handed out (or is waiting to be) this session: never again until reset.
pub(crate) raised: bool,
/// The alert waiting for the UI.
pub(crate) pending: Option<LowDiskAlert>,
/// When the free space was last read.
pub(crate) checked_at: Option<web_time::Instant>,
/// The read running on a worker (natively): a network share can take its time to answer, and
/// the check runs on the UI thread. It answers with the drive's space and, when that is low,
/// the preview cache's size (counting the cache can mean reading every file's size: that too
/// stays off the UI thread).
probe: Option<std::sync::mpsc::Receiver<Answer>>,
/// The running read was started for the library open before this one: its answer is ignored,
/// and the drive is read again for the current library once it has come. Until then no second
/// read starts, so switching libraries while a drive hangs never piles up workers.
stale: bool,
/// In place of [`read`] (tests).
pub(crate) reader: Option<Reader>,
/// Tests: the read's thread that fails to start (`"disk-space"`, `"disk-space-read"`).
pub(crate) fail_spawn: Option<&'static str>,
}
/// Start a named thread; `fail` stands for a system that can't (tests).
#[cfg(not(target_arch = "wasm32"))]
fn spawn(name: &str, fail: bool, f: impl FnOnce() + Send + 'static) -> std::io::Result<()> {
if fail {
return Err(std::io::Error::other("no thread (test)"));
}
std::thread::Builder::new().name(name.into()).spawn(f).map(drop)
}
impl DiskAlertState {
/// A fresh state for a newly opened library (the reader stays; a read still running is kept,
/// marked stale).
pub(crate) fn reset(&mut self) {
let probe = self.probe.take();
let stale = probe.is_some();
*self = DiskAlertState { reader: self.reader.take(), fail_spawn: self.fail_spawn, probe, stale, ..Default::default() };
}
}
impl crate::Session {
/// Read the free space of the drive holding the preview cache, at most every [`CHECK_EVERY`]
/// (`now`: whatever the last read, as when the library opens), and raise the alert if it is low
/// ([`Session::note_disk_space`](crate::Session::note_disk_space)). Nothing once the alert was
/// raised this session, without an on-disk library, or when the space can't be read.
/// The read itself runs on a worker (natively) and its answer is taken by a later call or
/// [`Session::poll_disk_space`]: a slow drive never holds up the caller.
pub fn check_disk_space(&mut self, now: bool) {
self.poll_disk_space();
// (a read still running for an earlier library open starts this library's read when it answers)
if self.disk_alert.raised || self.disk_alert.probe.is_some() {
return;
}
let Some(dir) = self.library.as_ref().filter(|l| l.on_disk).map(crate::library::Library::thumbs_dir) else { return };
let at = web_time::Instant::now();
if !now && self.disk_alert.checked_at.is_some_and(|t| at.saturating_duration_since(t) < CHECK_EVERY) {
return;
}
self.disk_alert.checked_at = Some(at);
let reader = self.disk_alert.reader.clone();
let rendered = self.media.rendered.clone();
let read_now = move || {
let space = match reader {
Some(r) => r(&dir),
None => read(&dir),
}?;
// the cache is counted only when the alert will tell its size
let cache_bytes = if space.is_low() { rendered.disk().map_or(0, |d| d.size()) } else { 0 };
Some((space, cache_bytes))
};
#[cfg(not(target_arch = "wasm32"))]
{
// a read when the library opens that takes more than a moment (a network share slow to
// answer, or the cache counted when the drive is low) shows in the activity stack; a quick
// one, and the quiet periodic ones every few minutes, don't
let activity = now.then(|| self.activity.clone());
let fail = self.disk_alert.fail_spawn;
let (tx, rx) = std::sync::mpsc::channel();
let started = spawn("disk-space", fail == Some("disk-space"), move || {
let answer = match activity {
None => Ok(read_now()),
Some(activity) => {
let (done, answered) = std::sync::mpsc::channel();
let reading = spawn("disk-space-read", fail == Some("disk-space-read"), move || {
let _ = done.send(read_now());
});
match reading {
Err(e) => {
log::info!("disk space: can't start the free-space read: {e}");
Err(NotStarted)
}
Ok(()) => match answered.recv_timeout(SHOW_ROW_AFTER) {
Ok(answer) => Ok(answer),
Err(_) => {
let _row = activity.start("cache", "Checking free disk space", crate::activity::Cancel::No);
Ok(answered.recv().ok().flatten())
}
},
}
}
};
let _ = tx.send(answer);
});
match started {
Ok(()) => self.disk_alert.probe = Some(rx),
Err(e) => {
log::info!("disk space: can't start the free-space read: {e}");
self.retry_disk_space_soon();
}
}
}
#[cfg(target_arch = "wasm32")]
if let Some((space, cache_bytes)) = read_now() {
self.raise_disk_alert(space, cache_bytes);
}
}
/// Take the answer of the free-space read running on a worker, if it has come, and raise the
/// alert when the drive is low. An answer for an earlier library open is dropped and the drive
/// of the current one is read instead. Called by [`Session::check_disk_space`],
/// [`Session::take_disk_space_alert`] and `library.info`.
pub fn poll_disk_space(&mut self) {
let Some(rx) = self.disk_alert.probe.as_ref() else { return };
let answer = match rx.try_recv() {
Ok(answer) => answer,
Err(std::sync::mpsc::TryRecvError::Empty) => return,
// (a worker that ended without answering: tried again soon too)
Err(std::sync::mpsc::TryRecvError::Disconnected) => Err(NotStarted),
};
self.disk_alert.probe = None;
if std::mem::take(&mut self.disk_alert.stale) {
self.check_disk_space(true);
} else {
match answer {
Ok(Some((space, cache_bytes))) => {
self.raise_disk_alert(space, cache_bytes);
}
Ok(None) => {}
Err(NotStarted) => self.retry_disk_space_soon(),
}
}
}
/// A read that couldn't start counts as made [`RETRY_AFTER`] less than [`CHECK_EVERY`] ago: the next check
/// reads the drive after [`RETRY_AFTER`] (CodeRabbit on #3577: it waited the whole five minutes).
fn retry_disk_space_soon(&mut self) {
let now = web_time::Instant::now();
self.disk_alert.checked_at = Some(now.checked_sub(CHECK_EVERY.saturating_sub(RETRY_AFTER)).unwrap_or(now));
}
/// Raise the low-disk alert for `space` when it is low and the alert wasn't raised this session
/// yet (the numbers come from [`read`], or from a test). Returns whether it was raised now.
/// Counts the preview cache here, on the caller's thread: the session's own reads count it on
/// their worker.
pub fn note_disk_space(&mut self, space: DiskSpace) -> bool {
if self.disk_alert.raised || !space.is_low() {
return false;
}
let cache_bytes = self.media.rendered.disk().map_or(0, |d| d.size());
self.raise_disk_alert(space, cache_bytes)
}
/// [`Session::note_disk_space`] with the cache's size already counted.
fn raise_disk_alert(&mut self, space: DiskSpace, cache_bytes: u64) -> bool {
if self.disk_alert.raised || !space.is_low() {
return false;
}
log::warn!("disk space: {} of {} bytes free; the preview cache takes {cache_bytes} bytes", space.available, space.total);
self.disk_alert.raised = true;
self.disk_alert.pending = Some(LowDiskAlert { space, cache_bytes });
true
}
/// The low-disk alert not shown yet, if any (handed out once per session).
pub fn take_disk_space_alert(&mut self) -> Option<LowDiskAlert> {
self.poll_disk_space();
self.disk_alert.pending.take()
}
/// Read the drive's space with `reader` instead of asking the system (tests of the app).
#[doc(hidden)]
pub fn set_disk_space_reader(&mut self, reader: Option<Reader>) {
self.disk_alert.reader = reader;
}
/// Forget when the space was last read, as if [`CHECK_EVERY`] had passed (tests of the app).
#[doc(hidden)]
pub fn forget_disk_space_check(&mut self) {
self.disk_alert.checked_at = None;
}
/// Whether a free-space read is running on its worker (its answer not taken yet).
pub fn disk_space_reading(&self) -> bool {
self.disk_alert.probe.is_some()
}
/// How long until [`Session::check_disk_space`] reads the drive again (zero when it is due, `None`
/// when it won't: the alert was raised, a read runs, or nothing has been read yet).
pub fn disk_space_next_check_in(&self) -> Option<std::time::Duration> {
let a = &self.disk_alert;
if a.raised || a.probe.is_some() {
return None;
}
Some(CHECK_EVERY.saturating_sub(a.checked_at?.elapsed()))
}
/// How long the free-space read running on its worker has run (`Duration::MAX` for one started for the
/// library before), or `None` when none runs.
pub fn disk_space_read_running_for(&self) -> Option<std::time::Duration> {
let a = &self.disk_alert;
a.probe.as_ref()?;
Some(a.checked_at.filter(|_| !a.stale).map_or(std::time::Duration::MAX, |t| t.elapsed()))
}
/// Whether the low-disk alert was raised since the library opened (`library.info` →
/// `lowDiskSpace`).
pub fn disk_space_alert_raised(&self) -> bool {
self.disk_alert.raised
}
}
#[cfg(test)]
mod tests {
use super::{DiskSpace, LOW_FREE_BYTES};
use crate::Session;
const GB: u64 = 1 << 30;
#[test]
fn plenty_of_room_on_both_counts_raises_no_alert() {
let mut s = Session::new();
assert!(!s.note_disk_space(DiskSpace { available: 200 * GB, total: 1000 * GB }), "20 % and 200 GB free");
assert!(!s.note_disk_space(DiskSpace { available: 25 * GB, total: 100 * GB }), "25 % and 25 GB free");
assert!(s.take_disk_space_alert().is_none());
assert!(!s.disk_space_alert_raised());
}
#[test]
fn under_ten_percent_free_raises_the_alert() {
let mut s = Session::new();
// 300 GB is far above 20 GB, but only 7.5 % of a 4 TB drive
let space = DiskSpace { available: 300 * GB, total: 4000 * GB };
assert!(s.note_disk_space(space));
let alert = s.take_disk_space_alert().expect("the alert is handed out");
assert_eq!(alert.space, space);
assert!(s.disk_space_alert_raised());
}
#[test]
fn under_twenty_gb_free_raises_the_alert() {
let mut s = Session::new();
// 15 % of a 100 GB drive is free, but that is under 20 GB
let space = DiskSpace { available: 15 * GB, total: 100 * GB };
assert!(space.available < LOW_FREE_BYTES);
assert!(s.note_disk_space(space));
assert_eq!(s.take_disk_space_alert().map(|a| a.space), Some(space));
}
#[test]
fn the_alert_is_raised_once_per_session() {
let mut s = Session::new();
let low = DiskSpace { available: GB, total: 500 * GB };
assert!(s.note_disk_space(low));
assert!(s.take_disk_space_alert().is_some());
assert!(s.take_disk_space_alert().is_none(), "handed out once");
assert!(!s.note_disk_space(DiskSpace { available: GB / 2, ..low }), "lower still: not again this session");
assert!(s.take_disk_space_alert().is_none());
s.check_disk_space(true);
assert!(s.take_disk_space_alert().is_none(), "a check after the alert reads nothing");
}
#[test]
fn opening_a_library_starts_a_new_session_for_the_alert() {
let dir = std::env::temp_dir().join(format!("lc-disk-alert-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let mut s = Session::new().with_fs();
s.open_library(&dir, false).unwrap();
// whatever this machine's drive holds, raise it by hand
let _ = s.take_disk_space_alert();
let low = DiskSpace { available: GB, total: 500 * GB };
if !s.disk_space_alert_raised() {
assert!(s.note_disk_space(low));
}
assert!(!s.note_disk_space(low), "once per session");
s.close_library().unwrap();
s.open_library(&dir, false).unwrap();
let _ = s.take_disk_space_alert();
let raised_by_open = s.disk_space_alert_raised();
assert!(raised_by_open || s.note_disk_space(low), "a new library open may warn again");
s.close_library().unwrap();
let _ = std::fs::remove_dir_all(&dir);
}
/// A drive that takes its time to answer (a network share) doesn't hold up the library open
/// or the checks on the UI thread: the read runs on a worker and its answer is taken later.
#[test]
fn a_slow_drive_does_not_hold_up_the_check() {
use std::sync::{Arc, Mutex, mpsc};
let dir = std::env::temp_dir().join(format!("lc-disk-alert-slow-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let (release, wait) = mpsc::channel::<()>();
let wait = Mutex::new(wait);
let mut s = Session::new().with_fs();
s.disk_alert.reader = Some(Arc::new(move |_: &std::path::Path| {
// answers once the test lets it (or after 10 s, so a broken test can't hang)
let _ = wait.lock().unwrap().recv_timeout(std::time::Duration::from_secs(10));
Some(DiskSpace { available: GB, total: 500 * GB })
}));
s.open_library(&dir, false).unwrap();
assert!(s.take_disk_space_alert().is_none(), "the open didn't wait for the drive's answer");
// the read the open started shows in the activity stack once it takes more than a moment
// (CodeRabbit on #3077)
let reading = |s: &Session| s.activity.list().iter().any(|t| t.kind == "cache" && t.label == "Checking free disk space");
let until = web_time::Instant::now() + std::time::Duration::from_secs(10);
while !reading(&s) {
assert!(web_time::Instant::now() < until, "{:?}", s.activity.list());
std::thread::sleep(std::time::Duration::from_millis(5));
}
assert!(s.disk_space_reading());
s.check_disk_space(true);
assert!(s.take_disk_space_alert().is_none(), "nor does a check while the read runs");
release.send(()).unwrap();
let until = std::time::Instant::now() + std::time::Duration::from_secs(10);
let alert = loop {
if let Some(a) = s.take_disk_space_alert() {
break a;
}
assert!(std::time::Instant::now() < until, "the drive's answer was never taken");
std::thread::sleep(std::time::Duration::from_millis(5));
};
assert_eq!(alert.space.available, GB);
assert!(s.disk_space_alert_raised());
assert!(!reading(&s) && !s.disk_space_reading(), "the row goes with the read");
s.close_library().unwrap();
let _ = std::fs::remove_dir_all(&dir);
}
/// A drive that hangs while the user switches libraries: the read still running is kept, no
/// second worker starts beside it, its answer (for the old library) is dropped, and the current
/// library's drive is read once it has come.
#[cfg(not(target_arch = "wasm32"))]
#[test]
fn switching_libraries_while_a_drive_hangs_starts_no_second_read() {
use std::sync::{Arc, Mutex, mpsc};
use std::time::{Duration, Instant};
let root = std::env::temp_dir().join(format!("lc-disk-alert-switch-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&root);
let (a, b) = (root.join("a"), root.join("b"));
std::fs::create_dir_all(&a).unwrap();
std::fs::create_dir_all(&b).unwrap();
let (release, wait) = mpsc::channel::<()>();
let (started_tx, started) = mpsc::channel::<std::path::PathBuf>();
let wait = Mutex::new(wait);
let started_tx = Mutex::new(started_tx);
let mut s = Session::new().with_fs();
s.disk_alert.reader = Some(Arc::new(move |dir: &std::path::Path| {
let _ = started_tx.lock().unwrap().send(dir.to_path_buf());
let _ = wait.lock().unwrap().recv_timeout(Duration::from_secs(10));
Some(DiskSpace { available: GB, total: 500 * GB })
}));
s.open_library(&a, false).unwrap();
let first = started.recv_timeout(Duration::from_secs(10)).expect("the open reads the drive");
assert!(first.starts_with(&a), "{first:?}");
s.close_library().unwrap();
s.open_library(&b, false).unwrap();
s.check_disk_space(true);
assert!(started.recv_timeout(Duration::from_millis(300)).is_err(), "no second read beside the hanging one");
release.send(()).unwrap();
let until = Instant::now() + Duration::from_secs(10);
let second = loop {
assert!(s.take_disk_space_alert().is_none(), "the old library's answer is dropped");
if let Ok(dir) = started.try_recv() {
break dir;
}
assert!(Instant::now() < until, "the current library's drive was never read");
std::thread::sleep(Duration::from_millis(5));
};
assert!(second.starts_with(&b), "{second:?}");
assert!(!s.disk_space_alert_raised());
release.send(()).unwrap();
let until = Instant::now() + Duration::from_secs(10);
while s.take_disk_space_alert().is_none() {
assert!(Instant::now() < until, "the current library's answer was never taken");
std::thread::sleep(Duration::from_millis(5));
}
s.close_library().unwrap();
let _ = std::fs::remove_dir_all(&root);
}
/// When the drive is low, the preview cache is counted on the read's worker too: counting it can
/// mean reading the size of every file (a big cache on a slow share), never on the UI thread
/// that takes the answer.
#[cfg(not(target_arch = "wasm32"))]
#[test]
fn the_preview_cache_is_counted_off_the_ui_thread() {
use std::sync::{Arc, Mutex, mpsc};
use std::time::{Duration, Instant};
let dir = std::env::temp_dir().join(format!("lc-disk-alert-count-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let (release, wait) = mpsc::channel::<()>();
let wait = Mutex::new(wait);
let mut s = Session::new().with_fs();
s.disk_alert.reader = Some(Arc::new(move |_: &std::path::Path| {
let _ = wait.lock().unwrap().recv_timeout(Duration::from_secs(10));
Some(DiskSpace { available: GB, total: 500 * GB })
}));
s.open_library(&dir, false).unwrap();
let disk = || s.media.rendered.disk().map(lightcraft_preview::DiskCache::known_size);
assert_eq!(disk(), Some(None), "an on-disk library has a preview cache, not counted yet");
release.send(()).unwrap();
// no poll here: only the worker can count the cache
let until = Instant::now() + Duration::from_secs(10);
let counted = loop {
if let Some(Some(bytes)) = disk() {
break bytes;
}
assert!(Instant::now() < until, "the worker never counted the cache");
std::thread::sleep(Duration::from_millis(5));
};
let until = Instant::now() + Duration::from_secs(10);
let alert = loop {
if let Some(a) = s.take_disk_space_alert() {
break a;
}
assert!(Instant::now() < until, "the drive's answer was never taken");
std::thread::sleep(Duration::from_millis(5));
};
assert_eq!(alert.cache_bytes, counted);
s.close_library().unwrap();
let _ = std::fs::remove_dir_all(&dir);
}
#[cfg(not(target_arch = "wasm32"))]
#[test]
fn the_free_space_of_a_real_drive_is_read() {
let space = super::read(&std::env::temp_dir().join("lc-disk-alert-not-there")).expect("the temp folder's drive says its size");
assert!(space.total > 0 && space.available <= space.total, "{space:?}");
}
/// A read whose thread can't start is tried again after [`super::RETRY_AFTER`], not after the five minutes
/// of [`super::CHECK_EVERY`] (CodeRabbit on #3577); for either of its two threads.
#[cfg(not(target_arch = "wasm32"))]
#[test]
fn a_read_that_could_not_start_is_tried_again_soon() {
use std::time::{Duration, Instant};
for thread in ["disk-space", "disk-space-read"] {
let dir = std::env::temp_dir().join(format!("lc-disk-alert-nothread-{thread}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let mut s = Session::new().with_fs();
s.disk_alert.reader = Some(std::sync::Arc::new(|_: &std::path::Path| Some(DiskSpace { available: GB, total: 500 * GB })));
s.disk_alert.fail_spawn = Some(thread);
s.open_library(&dir, false).unwrap();
let until = Instant::now() + Duration::from_secs(10);
while s.disk_space_reading() {
assert!(Instant::now() < until, "{thread}: the read never ended");
s.poll_disk_space();
std::thread::sleep(Duration::from_millis(5));
}
let since = s.disk_alert.checked_at.expect("a check was made").elapsed();
assert!(
since + super::RETRY_AFTER + Duration::from_secs(1) >= super::CHECK_EVERY,
"{thread}: next read in {:?}",
super::CHECK_EVERY.saturating_sub(since)
);
// the thread can start again: the retry reads the drive and raises the alert
s.disk_alert.fail_spawn = None;
s.disk_alert.checked_at = s.disk_alert.checked_at.and_then(|t| t.checked_sub(super::RETRY_AFTER));
s.check_disk_space(false);
let until = Instant::now() + Duration::from_secs(10);
while s.take_disk_space_alert().is_none() {
assert!(Instant::now() < until, "{thread}: the retry never read the drive");
std::thread::sleep(Duration::from_millis(5));
}
s.close_library().unwrap();
let _ = std::fs::remove_dir_all(&dir);
}
}
#[test]
fn a_drive_that_reports_no_size_is_judged_by_its_free_bytes() {
assert!(!DiskSpace { available: 50 * GB, total: 0 }.is_low());
assert!(DiskSpace { available: 5 * GB, total: 0 }.is_low());
}
}
crates/engine/src/files/fs.rs
+6
View File
@@ -37,6 +37,12 @@ pub(super) fn raf_preview_extent(prefix: &[u8]) -> Option<u64> {
/// `adobe_rgb`: the container declares the preview Adobe RGB ([`PreviewRead::adobe_rgb`]), applied
/// as [`decode_raw_preview`] applies it. `None` when the bytes are not a decodable image.
pub(super) fn quick_image_srgb(header: &[u8], window: &[u8], adobe_rgb: bool, max_edge: usize) -> Option<lightcraft_raster::Rgba8> {
// a plain sRGB camera JPEG (nearly all of them) straight to 8-bit (PERF-CULL-3: 45 MP preview
// of an EOS R5 C fitted to 2560 px, 194 → ~110 ms); anything else takes the general decode
let edge = u32::try_from(max_edge).unwrap_or(u32::MAX);
if !adobe_rgb && let Ok(Some((image, orientation))) = lightcraft_codecs::decode_jpeg_srgb8_fit(window, edge, edge) {
return Some(image.oriented(preview_orientation(header, orientation)));
}
let decoded = if adobe_rgb && window.starts_with(&[0xff, 0xd8]) {
lightcraft_codecs::decode_jpeg_with_fallback(window, fit_box(max_edge), lightcraft_codecs::NamedSpace::AdobeRgb)
} else {
crates/engine/src/files/preview.rs
+19 -4
View File
@@ -33,7 +33,11 @@ pub(crate) fn decode_raw_preview(
opts: lightcraft_codecs::DecodeOptions,
covering: Option<usize>,
) -> Option<lightcraft_codecs::Decoded> {
let jpeg = stand_in_image(bytes, covering)?;
decode_stand_in(bytes, &stand_in_image(bytes, covering)?, opts)
}
/// [`decode_raw_preview`] of the stand-in `jpeg` already found in the raw file `bytes` (by [`stand_in_image`]).
fn decode_stand_in(bytes: &[u8], jpeg: &[u8], opts: lightcraft_codecs::DecodeOptions) -> Option<lightcraft_codecs::Decoded> {
// the colour space is a property of the JPEG; a TIFF stand-in (the file itself, or a Hasselblad RGB preview
// wrapped as a TIFF) and a JPEG XL preview are read by their own decoders
let is_jpeg = jpeg.starts_with(&[0xff, 0xd8]);
@@ -43,9 +47,9 @@ pub(crate) fn decode_raw_preview(
lightcraft_raw::PreviewColorSpace::Srgb => lightcraft_codecs::NamedSpace::Srgb,
lightcraft_raw::PreviewColorSpace::AdobeRgb => lightcraft_codecs::NamedSpace::AdobeRgb,
};
lightcraft_codecs::decode_jpeg_with_fallback(&jpeg, opts, space).ok()
lightcraft_codecs::decode_jpeg_with_fallback(jpeg, opts, space).ok()
}
None => lightcraft_codecs::decode(&jpeg, opts).ok(),
None => lightcraft_codecs::decode(jpeg, opts).ok(),
}
}
@@ -80,7 +84,18 @@ pub fn load_embedded_preview(bytes: &[u8], max_edge: usize) -> Option<(Rgb32f, S
/// loupe and grid show it until the raw itself has been developed ([`crate::media::QuickJob`]). The smallest
/// embedded JPEG that covers `max_edge` is decoded, not the largest.
pub fn embedded_preview_srgb(bytes: &[u8], max_edge: usize) -> Option<lightcraft_raster::Rgba8> {
let mut d = decode_raw_preview(bytes, fit_box(max_edge), Some(max_edge))?;
// a plain sRGB camera JPEG straight to 8-bit, as the quick path decodes it (PERF-CULL-3), so
// both paths give the same picture; anything else takes the general decode
let edge = u32::try_from(max_edge).unwrap_or(u32::MAX);
// (found once: the general decode below takes the same stand-in when the fast path declines)
let jpeg = stand_in_image(bytes, Some(max_edge))?;
if lightcraft_raw::embedded_preview_color_space(bytes) != Some(lightcraft_raw::PreviewColorSpace::AdobeRgb)
&& matches!(jpeg, std::borrow::Cow::Owned(_))
&& let Ok(Some((image, orientation))) = lightcraft_codecs::decode_jpeg_srgb8_fit(&jpeg, edge, edge)
{
return Some(image.oriented(preview_orientation(bytes, orientation)));
}
let mut d = decode_stand_in(bytes, &jpeg, fit_box(max_edge))?;
if d.image.width.max(d.image.height) > max_edge {
d.image = fit(&d.image, max_edge, max_edge, Filter::Box);
d.alpha = None;
crates/engine/src/import.rs
+3 -1
View File
@@ -53,7 +53,9 @@ pub(crate) use files::{expand_noting, probe_all, probe_paths, probe_with};
use job::PreparedItem;
pub use job::{ImportJob, Prepared, candidate_thumb_key, import, import_with};
pub(crate) use scan::scan_files;
pub use scan::{ScanInput, ScanOutput, ScanProgress, scan, scan_with};
pub use scan::{
ScanInput, ScanOutput, ScanProgress, browse_files, browse_order, browse_path, browse_queue, browse_queue_until, browse_spelling, scan, scan_with,
};
#[cfg(test)]
use scan::{library_files, library_hashes};
pub use types::{
crates/engine/src/import/job.rs
+2
View File
@@ -528,6 +528,8 @@ impl Session {
key,
cached: Vec::new(),
embedded,
embedded_cache: None,
embedded_on_disk: false,
minimum_embedded_edge,
small: Some(Box::new(small)),
})
crates/engine/src/import/scan.rs
+180 -1
View File
@@ -1,4 +1,4 @@
use super::files::{Seen, expand, probe_all, reusable_probes};
use super::files::{Seen, expand, is_supported, probe_all, reusable_probes};
use super::types::ImportCandidate;
use super::{HashMap, Path, PathBuf, PhotoId, ProbeInfo, Session, Source};
@@ -98,6 +98,185 @@ impl ScanInput {
pub(crate) fn library(&self) -> std::sync::Arc<lightcraft_catalog::Catalog> {
std::sync::Arc::clone(&self.library)
}
/// The probes of earlier scans this input took from the session ([`ScanInput::new`]), to hand back with a
/// scan's output that probes nothing itself (browsing), so a later import doesn't read those files again.
pub fn take_probes(&mut self) -> HashMap<String, ProbeInfo> {
std::mem::take(&mut self.cache)
}
}
/// What browsing a folder (Local) has to read: the files of `files` the library has no photo at,
/// in the order the view will show them (see [`browse_order`]). Unlike [`scan_with`] nothing is
/// read, so the import that follows adds the photos batch by batch as it reads them, and the first
/// screen of a 3,000-raw wedding shows within a second instead of after every file was read
/// (measured on 2,933 EOS R5 C CR3s: 30.6 s to the first photo before).
pub fn browse_queue(input: &ScanInput, files: Vec<String>, sort: &lightcraft_catalog::Sort) -> Vec<String> {
browse_queue_until(input, files, sort, &std::sync::atomic::AtomicBool::new(false)).unwrap_or_default()
}
/// [`browse_queue`] that stops reading the files' times or sizes once `cancel` is set (another folder was
/// chosen): `None` then.
pub fn browse_queue_until(
input: &ScanInput,
files: Vec<String>,
sort: &lightcraft_catalog::Sort,
cancel: &std::sync::atomic::AtomicBool,
) -> Option<Vec<String>> {
let known = library_files(&input.library, &files);
let mut files: Vec<String> = files.into_iter().filter(|f| !known.contains_key(Path::new(f))).collect();
browse_order_until(&mut files, sort, cancel).then_some(files)
}
/// The files browsing `root` (Local) lists: the supported, non-hidden files in it, or below it
/// with `subfolders`, never inside the library's own folder (as [`scan_with`] skips it).
///
/// A folder that can't be listed whole (gone, no access, a network share dropping out) is an
/// error, not an empty or partial listing: the browse bookkeeping would take the files missing
/// from it for files gone. (With `subfolders` the walk skips the folders below it that it can't
/// read, as the import's scan does.)
pub fn browse_files(input: &ScanInput, root: &str, subfolders: bool) -> std::io::Result<Vec<String>> {
// one spelling for the folder, and so for its files: `lib/../outside` lists `outside`'s files as
// the catalog stores them, not as new photos under another name
let root_path = browse_path(Path::new(root))?;
let root = root_path.to_string_lossy().to_string();
// the library's folder itself, or a folder inside it: nothing to browse
if input.skip.as_deref().is_some_and(|lib| inside(&root_path, lib)) {
return Ok(Vec::new());
}
if subfolders {
// (the root itself must be there to be walked)
std::fs::read_dir(&root)?;
return Ok(expand(&[root], input.skip.as_deref()));
}
let mut v = Vec::new();
for entry in std::fs::read_dir(&root)? {
let entry = entry?;
if crate::walk::is_hidden_entry(&entry) {
continue;
}
let f = entry.path();
if f.is_file() && is_supported(&f) {
v.push(f.to_string_lossy().to_string());
}
}
v.sort();
Ok(v)
}
/// The folder `path` names, as one spelling for the view, its listing and the catalog's records:
/// absolute (`.` is the current folder), with `.` and `..` taken out the way the system reads them.
/// Windows takes `..` out by name; elsewhere `link/..` is the folder above where the link points,
/// so a link before a `..` is resolved first. Everything else keeps its spelling (a mapped drive
/// stays a drive letter, a link not followed by `..` stays the link), as the records were made.
///
/// It may read the disk (a link on a stalled network mount can block it): call it off the UI thread.
/// [`browse_spelling`] is the UI's spelling meanwhile.
pub fn browse_path(path: &Path) -> std::io::Result<PathBuf> {
normalized(path, true)
}
/// [`browse_path`] without reading the disk: `.` and `..` taken out by name. What the view shows
/// until the worker has resolved the folder; the same folder unless a link comes before a `..`
/// (not on Windows, which takes `..` out by name too).
pub fn browse_spelling(path: &Path) -> std::io::Result<PathBuf> {
normalized(path, false)
}
fn normalized(path: &Path, resolve_links: bool) -> std::io::Result<PathBuf> {
use std::path::Component;
// (no disk access: the current folder is the process's)
let path = std::path::absolute(path)?;
let mut out = PathBuf::new();
for c in path.components() {
match c {
Component::CurDir => {}
Component::ParentDir => {
// a link before `..` is left through where it points; a broken one is an error, not
// its parent folder by name (CodeRabbit on #3577)
#[cfg(not(windows))]
if resolve_links && std::fs::symlink_metadata(&out).is_ok_and(|m| m.file_type().is_symlink()) {
out = std::fs::canonicalize(&out)?;
}
#[cfg(windows)]
let _ = resolve_links;
// (above the root: still the root)
if matches!(out.components().next_back(), Some(Component::Normal(_))) {
out.pop();
}
}
c => out.push(c.as_os_str()),
}
}
Ok(out)
}
/// Whether `root` (a [`browse_path`]) is `lib` or a folder inside it: as written, or once both are
/// resolved (a symlink to the library names the same folder), as the recursive walk tells. A root
/// that can't be resolved (gone, or a network share that won't say) is compared as written when it
/// is there; one that isn't there is not inside, and listing it reports why.
fn inside(root: &Path, lib: &Path) -> bool {
let as_written = || root.starts_with(browse_path(lib).unwrap_or_else(|_| lib.to_path_buf()));
match std::fs::canonicalize(root) {
Ok(resolved) => as_written() || std::fs::canonicalize(lib).is_ok_and(|lib| resolved.starts_with(lib)),
Err(_) => root.exists() && as_written(),
}
}
/// Order `files` the way a view sorted by `sort` will show them, as far as the file system can
/// tell before the files are read: by modification time for the capture date (a camera's file
/// time is its capture time, near enough to fill the first screen first), by name, or by size.
/// New photos share one import time and have no edit time, so the import- and edit-date sorts
/// show them by name. Ties go by name in the view's direction, as the catalog breaks them
/// ([`lightcraft_catalog::Sort::order`]); new photos tie on rating too. Random keeps the given order. Files whose
/// time or size can't be read go last.
pub fn browse_order(files: &mut [String], sort: &lightcraft_catalog::Sort) {
browse_order_until(files, sort, &std::sync::atomic::AtomicBool::new(false));
}
/// [`browse_order`] that stops reading the files' times or sizes once `cancel` is set: `false` then, and
/// `files` is left as it was.
fn browse_order_until(files: &mut [String], sort: &lightcraft_catalog::Sort, cancel: &std::sync::atomic::AtomicBool) -> bool {
use lightcraft_catalog::SortKey;
use lightcraft_catalog::query::cmp_name;
use std::cmp::Ordering;
use std::sync::atomic::Ordering::Relaxed;
let asc = sort.ascending;
let name = |f: &str| Path::new(f).file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
let directed = |o: Ordering| if asc { o } else { o.reverse() };
// `key` per file (`None`: unreadable, last in either direction), then the name, in the view's direction
let by_key = |files: &mut [String], key: &dyn Fn(&str) -> Option<u64>| {
let mut keyed: Vec<(Option<u64>, String, String)> = Vec::with_capacity(files.len());
for f in files.iter() {
if cancel.load(Relaxed) {
return false;
}
keyed.push((key(f), name(f), f.clone()));
}
keyed.sort_by(|a, b| match (a.0, b.0) {
(Some(x), Some(y)) => directed(x.cmp(&y).then_with(|| cmp_name(&a.1, &b.1))),
(Some(_), None) => Ordering::Less,
(None, Some(_)) => Ordering::Greater,
(None, None) => directed(cmp_name(&a.1, &b.1)),
});
for (slot, (_, _, f)) in files.iter_mut().zip(keyed) {
*slot = f;
}
true
};
match sort.key {
SortKey::CaptureDate => by_key(files, &|f| {
let t = std::fs::metadata(f).and_then(|m| m.modified()).ok()?;
// (nanoseconds since the epoch fit a u64 until 2554; earlier times count as the epoch)
Some(t.duration_since(std::time::UNIX_EPOCH).map_or(0, |d| u64::try_from(d.as_nanos()).unwrap_or(u64::MAX)))
}),
SortKey::ImportDate | SortKey::EditDate | SortKey::Rating | SortKey::FileName => {
files.sort_by(|a, b| directed(cmp_name(&name(a), &name(b))));
!cancel.load(Relaxed)
}
SortKey::FileSize => by_key(files, &|f| std::fs::metadata(f).ok().map(|m| m.len())),
SortKey::Random => !cancel.load(Relaxed),
}
}
/// Find and probe what importing `paths` would add, marking duplicates (by path, or by content
crates/engine/src/lib.rs
+13
View File
@@ -28,6 +28,7 @@ pub mod crs_masks;
pub mod demo;
pub mod denoise;
pub mod devices;
pub mod disk_space;
pub mod display;
pub mod doc_path;
mod errors;
@@ -61,6 +62,7 @@ mod model_download;
pub mod open_path;
pub mod originals;
mod pdf;
pub mod prep;
pub mod preset_import;
pub mod preset_luminar;
pub mod presets;
@@ -398,10 +400,18 @@ pub struct Session {
pub slideshow_export: Option<std::sync::Arc<slideshow_export::ExportRun>>,
/// The last (or running) Build Previews.
pub preview_build: Option<std::sync::Arc<cmd::previews::PreviewBuild>>,
/// Preparing photos in the background: settings (app-wide and this library's) and the build
/// it runs ([`prep`]).
pub prep: prep::PrepState,
/// The long edge (px) the loupe renders at on this screen, as the frontend last said
/// (0: not known). Standard previews are built at least this big, so the loupe can use them.
pub loupe_edge: usize,
/// Develop defaults applied on import (persisted in prefs.json).
pub import_defaults: import::ImportDefaults,
/// Disk budget of the library's thumbnail cache in MB (0 = default; persisted in prefs.json).
pub cache_mb: u32,
/// The once-per-session alert for a drive running low under the preview cache ([`disk_space`]).
pub(crate) disk_alert: disk_space::DiskAlertState,
/// Slideshow playback settings (`slideshow.playback`; persisted in prefs.json).
pub slideshow: slideshow::PlaybackSettings,
/// Slideshow slide layout: Layout, Options and Backdrop (`slideshow.layout`; persisted in
@@ -537,9 +547,12 @@ impl Session {
folder_changes: None,
activity: Default::default(),
preview_build: None,
prep: Default::default(),
loupe_edge: 0,
slideshow_export: None,
import_defaults: import::ImportDefaults::default(),
cache_mb: 0,
disk_alert: Default::default(),
slideshow: slideshow::PlaybackSettings::default(),
slide_layout: slide::SlideSettings::default(),
slide_overlays: slide_overlay::OverlaySettings::default(),
crates/engine/src/library.rs
+43 -2
View File
@@ -258,6 +258,13 @@ impl Library {
}
}
/// [`PrefsFile`]'s `prep`, read leniently (see there).
fn lenient_prep<'de, D: serde::Deserializer<'de>>(d: D) -> std::result::Result<Option<crate::prep::PrepSettings>, D::Error> {
use serde::Deserialize;
let value = serde_json::Value::deserialize(d)?;
Ok(serde_json::from_value(value).ok())
}
#[derive(Default, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
struct PrefsFile {
@@ -295,6 +302,11 @@ struct PrefsFile {
import: crate::import::ImportDefaults,
/// Thumbnail disk cache budget (MB, 0 = default).
cache_mb: u32,
/// This library's own background preparation settings (`None`: the app-wide ones). Read
/// leniently: a value a newer build wrote that this one doesn't know is `None`, and the rest of
/// the file is kept.
#[serde(skip_serializing_if = "Option::is_none", deserialize_with = "lenient_prep")]
prep: Option<crate::prep::PrepSettings>,
/// Folder for smart previews (default: `Smart Previews` in the library).
smart_previews_dir: Option<String>,
/// Days after which untouched Local records of unbrowsed folders are forgotten (missing =
@@ -585,6 +597,7 @@ impl Session {
self.recent_keywords = prefs.recent_keywords;
self.import_defaults = prefs.import;
self.cache_mb = prefs.cache_mb;
self.prep.library = prefs.prep.map(crate::prep::PrepSettings::sanitized);
self.slideshow = prefs.slideshow.sanitized();
self.slide_layout = prefs.slide_layout.sanitized();
self.slide_overlays = prefs.slide_overlays.sanitized();
@@ -688,6 +701,9 @@ impl Session {
crate::cmd::previews::refresh_stale_smart_previews(self);
}
self.library_identity = std::sync::Arc::new(());
// a library open is a new session for the low-disk alert: look at the drive now
self.disk_alert.reset();
self.check_disk_space(true);
Ok(())
}
@@ -900,6 +916,7 @@ impl Session {
recent_keywords: self.recent_keywords.clone(),
import: self.import_defaults.clone(),
cache_mb: self.cache_mb,
prep: self.prep.library,
smart_previews_dir: self.smart_previews_dir.as_ref().map(|d| d.to_string_lossy().to_string()),
forget_local_days: Some(self.forget_local_days),
slideshow: self.slideshow.clone(),
@@ -927,9 +944,23 @@ impl Session {
new
}
/// The thumbnail disk cache budget in bytes ([`Session::cache_mb`], else the default).
/// The thumbnail disk cache budget in bytes ([`Session::cache_mb`], else the default, which
/// grows with the library: [`crate::media::default_disk_cache_bytes`]).
pub fn cache_bytes(&self) -> u64 {
if self.cache_mb == 0 { crate::media::DISK_CACHE_BYTES } else { u64::from(self.cache_mb) << 20 }
if self.cache_mb == 0 { crate::media::default_disk_cache_bytes(self.catalog.len()) } else { u64::from(self.cache_mb) << 20 }
}
/// Bring the disk cache's budget up to date with the library (photos were added since it was
/// set): cheap, nothing is read or written. Called before previews are built, so it also
/// looks at the drive's free space when that is due ([`Session::check_disk_space`]).
pub fn fit_disk_cache(&mut self) {
let want = self.cache_bytes();
if let Some(d) = self.media.rendered.disk()
&& d.budget() != want
{
d.set_budget(want);
}
self.check_disk_space(false);
}
/// Change the thumbnail disk cache budget (MB, 0 = default) in place: the cached thumbnails
@@ -991,6 +1022,16 @@ fn unlocked_warning(lock: Option<&LibraryLock>) -> Option<String> {
mod resolve_tests {
use super::*;
/// A library's background preparation value this build doesn't know reads as none (the
/// app-wide settings apply) and keeps the rest of `prefs.json`.
#[test]
fn an_unknown_background_preparation_value_keeps_the_other_prefs() {
let v = serde_json::json!({"cacheMb": 77, "prep": {"culling": {"scope": "someday"}}});
let read: PrefsFile = serde_json::from_value(v).unwrap();
assert!(read.prep.is_none());
assert_eq!(read.cache_mb, 77);
}
/// `--library <dir>/catalog.snap` failed with the OS's "File exists" (creating a folder over
/// the file); it opens the library that file belongs to.
#[test]
crates/engine/src/media.rs
+6 -3
View File
@@ -35,14 +35,17 @@ mod settings_hash;
mod source;
pub use cache::DenoiseIndex;
#[cfg(test)]
pub(crate) use cache::rendered_budget;
pub use quick::{QuickJob, QuickSource, content_key, source_info};
pub use render::{RenderJob, RenderResult, ScreenFrame, develop, develop_reporting, fit_for_display};
pub use render::{RenderJob, RenderResult, ScreenFrame, develop, develop_reporting, fit_cached, fit_for_display};
#[cfg(test)]
use settings_hash::SETTINGS_HASH_ENTRIES;
use settings_hash::SettingsHashes;
pub use source::{
DISK_CACHE_BYTES, DecodedSource, DenoiseSpec, FACE_EDGE_STEP, FileLoader, FileProbe, MakeProduct, PairLoader, PreviewLoader, ProbeInfo,
RENDER_CACHE_VERSION, ReducedLoader, SourceLevel, SourceRef, THUMB_SIZES, Twin, face_edge, thumb_bucket,
DISK_CACHE_BYTES, DISK_CACHE_PER_PHOTO, DecodedSource, DenoiseSpec, FACE_EDGE_STEP, FileLoader, FileProbe, MakeProduct, PairLoader,
PreviewLoader, ProbeInfo, RENDER_CACHE_VERSION, ReducedLoader, SourceLevel, SourceRef, THUMB_SIZES, Twin, default_disk_cache_bytes, face_edge,
thumb_bucket,
};
pub struct MediaCache {
crates/engine/src/media/cache.rs
+5 -3
View File
@@ -75,9 +75,11 @@ impl Default for MediaCache {
}
}
/// Memory for rendered previews out of the cache share.
fn rendered_budget(share: usize) -> usize {
RENDERED_MEM_BYTES.min(share / 4)
/// Memory for rendered previews out of the cache share: a quarter of it, so a big budget keeps
/// a whole shoot's previews (a 20 GB budget holds about 230 standard previews, or 1 500 loupe
/// renders at 1 200 px), and at least [`RENDERED_MEM_BYTES`] where the share allows.
pub(crate) fn rendered_budget(share: usize) -> usize {
(share / 4).max(RENDERED_MEM_BYTES.min(share / 2))
}
impl MediaCache {
crates/engine/src/media/quick.rs
+37 -6
View File
@@ -25,6 +25,15 @@ pub struct QuickJob {
pub cached: Vec<(Arc<PreviewCache>, Hash128)>,
/// Embedded preview of an unedited raw: (path, loader, max edge).
pub embedded: Option<(String, PreviewLoader, usize)>,
/// Where a decoded [`Self::embedded`] preview is kept in memory, so stepping back to a photo
/// shows its camera JPEG again without reading and decoding it (culling goes back and forth):
/// the cache (memory only, never its disk), the key for this photo and edge, and the cache's
/// generation when the job was made (a Discard Preview Cache since then keeps the picture out).
pub embedded_cache: Option<(Arc<PreviewCache>, Hash128, u64)>,
/// Whether [`Self::embedded_cache`] keeps the decoded preview on disk too: a grid thumbnail,
/// so a folder seen before fills in from small files at once instead of every raw being read
/// and its camera JPEG decoded again (the loupe's large stand-ins stay in memory).
pub embedded_on_disk: bool,
/// Import tiles need this many pixels when a render can provide them. `None` accepts any
/// camera JPEG immediately (ordinary transient stand-ins and preview-only imports).
/// An undersized JPEG remains the fallback if the render fails or is unavailable.
@@ -79,13 +88,30 @@ impl QuickJob {
}
}
let mut embedded_fallback = None;
if let Some((path, loader, edge)) = &self.embedded
&& let Some(image) = loader(path, *edge)
{
if self.minimum_embedded_edge.is_none_or(|minimum| image.width.max(image.height) >= minimum) {
return done(rendered(image), Some(QuickSource::Embedded));
if let Some((path, loader, edge)) = &self.embedded {
let kept =
self.embedded_cache.as_ref().and_then(|(cache, k, _)| if self.embedded_on_disk { cache.get(*k) } else { cache.get_memory(*k) });
let image = match kept {
Some(img) => Some(Arc::unwrap_or_clone(img)),
None => {
let image = loader(path, *edge);
// written for the generation the job was made in: a cache cleared meanwhile stays clear
if let (Some(image), Some((cache, k, generation))) = (&image, &self.embedded_cache) {
if self.embedded_on_disk {
cache.put_deferred_at(*generation, *k, Arc::new(image.clone()));
} else {
cache.put_memory_at(*generation, *k, Arc::new(image.clone()));
}
}
image
}
};
if let Some(image) = image {
if self.minimum_embedded_edge.is_none_or(|minimum| image.width.max(image.height) >= minimum) {
return done(rendered(image), Some(QuickSource::Embedded));
}
embedded_fallback = Some(image);
}
embedded_fallback = Some(image);
}
if let Some(job) = self.small {
let r = job.run();
@@ -100,6 +126,11 @@ impl QuickJob {
}
}
/// The thumbnail cache's key for the camera JPEG standing for the thumbnail of cache key `k`.
pub(super) fn embedded_thumb_key(k: Hash128) -> Hash128 {
Hasher128::new().str("embedded-thumb").u64((k.0 >> 64) as u64).u64(k.0 as u64).finish()
}
/// What identifies a photo's pixels for caching: its content hash, else its source.
/// Which decode of a photo a loaded source is: its file's content and, when it has one, its database lens profile
/// ([`lens_key`]; the decode fits the starting colour through it). A job's source is kept only when this still
crates/engine/src/media/render.rs
+106 -1
View File
@@ -1,5 +1,5 @@
use super::quick::QuickSource;
use super::source::{DecodedSource, SourceLevel, SourceRef};
use super::source::{DecodedSource, SourceLevel, SourceRef, VIEW_REUSE_SLACK};
use super::{
Arc, DevelopSettings, Hash128, Histogram, PhotoId, PreviewCache, Quality, RenderRequest, Rendered, Rgb32f, Rgba8, Source, SourceInfo, StageCache,
};
@@ -241,8 +241,58 @@ impl RenderJob {
r
}
/// The view render kept for this job's photo and settings ([`Self::view_cache`]), fitted to the
/// size asked for, when it is at least that big, or unchanged when it covers the job's display
/// fit (else `None`: render). Only for what the cached
/// view is: a full-quality, 8-bit, whole-frame SDR render (the cache holds sRGB; a display
/// profile is applied to it like to any cached picture, [`crate::display::present`]).
pub fn cached_view(&self) -> Option<Rgba8> {
let (cache, key) = self.view_cache.as_ref()?;
if self.request.quality != Quality::Full
|| self.request.depth != lightcraft_pipeline::OutputDepth::U8
|| self.request.window.is_some()
|| self.settings.hdr.enabled
|| self.slide.is_some()
{
return None;
}
let img = cache.get_at(self.cache_generation, *key)?;
fit_cached(&img, self.request.max_w, self.request.max_h).or_else(|| {
// the loupe at rest asks for at least the preview level and shows the render shrunk to
// the drawn size (the display fit): a cached view that covers the drawn size is enough
let (fw, fh) = self.display_fit?;
let (w, h) = (img.width, img.height);
let s = (fw as f64 / w.max(1) as f64).min(fh as f64 / h.max(1) as f64);
(w > 0 && h > 0 && fw > 0 && fh > 0 && s <= 1.0 + VIEW_REUSE_SLACK).then(|| Arc::unwrap_or_clone(img))
})
}
/// The photo's render (without slide composition).
fn run_photo(self) -> RenderResult {
// the loupe: a cached view render of these settings at least this big is the render (built
// previews and the photos prepared ahead are used, not rendered again; PERF-PREVIEW-REUSE)
if let Some(mut image) = self.cached_view() {
// as for a render: the histogram of the full picture, then the display fit
let histogram = Histogram::of_srgb8(&image);
if let Some((fw, fh)) = self.display_fit
&& (image.width > fw.max(1) || image.height > fh.max(1))
{
image = fit_for_display(&image, fw, fh);
}
return RenderResult {
request_id: self.request_id,
source_key: self.source_key,
photo: self.photo,
level: self.level,
key: self.key,
rendered: Ok(Rendered { image, histogram, deep: None }),
gpu_fallback: None,
loaded: None,
quick: Some(QuickSource::Cached),
display: None,
frame: None,
};
}
if let Some((cache, key)) = &self.cache
&& let Some(img) = cache.get_at(self.cache_generation, *key)
{
@@ -395,6 +445,61 @@ impl RenderJob {
}
}
/// `img` fitted into `max_w × max_h` (its aspect kept), when it is big enough to need no
/// enlarging (within [`VIEW_REUSE_SLACK`]): as it is, or shrunk by averaging.
pub fn fit_cached(img: &Rgba8, max_w: usize, max_h: usize) -> Option<Rgba8> {
let (w, h) = (img.width, img.height);
if w == 0 || h == 0 || max_w == 0 || max_h == 0 {
return None;
}
let s = (max_w as f64 / w as f64).min(max_h as f64 / h as f64);
if !s.is_finite() || s > 1.0 + VIEW_REUSE_SLACK {
return None;
}
let (nw, nh) = (((w as f64 * s).round() as usize).clamp(1, w), ((h as f64 * s).round() as usize).clamp(1, h));
if nw == w && nh == h {
return Some(img.clone());
}
Some(shrink_rgba(img, nw, nh))
}
/// `img` shrunk to `nw × nh` (no larger than it), each pixel the average of the ones it covers.
fn shrink_rgba(img: &Rgba8, nw: usize, nh: usize) -> Rgba8 {
let (w, h) = (img.width, img.height);
let (nw, nh) = (nw.clamp(1, w.max(1)), nh.clamp(1, h.max(1)));
// the source columns / rows each output column / row covers: [start, end)
let spans = |n: usize, of: usize| -> Vec<(usize, usize)> {
(0..n)
.map(|i| {
let a = i.saturating_mul(of) / n;
let b = (i.saturating_add(1).saturating_mul(of) / n).max(a.saturating_add(1)).min(of);
(a, b)
})
.collect()
};
let (xs, ys) = (spans(nw, w), spans(nh, h));
let mut data = Vec::with_capacity(nw.saturating_mul(nh));
let mut row = vec![[0u32; 4]; nw];
for &(y0, y1) in &ys {
row.iter_mut().for_each(|r| *r = [0; 4]);
for y in y0..y1 {
let Some(line) = img.data.get(y * w..(y + 1) * w) else { continue };
for (acc, &(x0, x1)) in row.iter_mut().zip(&xs) {
for px in line.get(x0..x1).unwrap_or(&[]) {
for (a, v) in acc.iter_mut().zip(px) {
*a += u32::from(*v);
}
}
}
}
for (acc, &(x0, x1)) in row.iter().zip(&xs) {
let n = (x1.saturating_sub(x0).saturating_mul(y1.saturating_sub(y0))).max(1) as u32;
data.push(acc.map(|a| ((a + n / 2) / n).min(255) as u8));
}
}
Rgba8 { width: nw, height: nh, data }
}
/// Render `src`: on the GPU when `gpu` is set and a GPU is available (`lightcraft_gpu`), else on
/// the CPU, reusing `stages` either way. Both produce the same image within 1–3 LSB (see
/// `docs/gpu-pipeline.md`); `LIGHTCRAFT_GPU=0` or [`lightcraft_gpu::set_enabled`] forces the CPU.
crates/engine/src/media/session.rs
+57 -9
View File
@@ -1,4 +1,4 @@
use super::quick::{QuickJob, content_key, source_info};
use super::quick::{QuickJob, content_key, embedded_thumb_key, source_info};
use super::render::{RenderJob, RenderResult};
use super::settings_hash::with_process;
use super::source::{PreviewLoader, RENDER_CACHE_VERSION, SourceLevel, SourceRef, THUMB_SIZES, face_edge, source_edge_needed, thumb_bucket};
@@ -264,12 +264,19 @@ impl crate::Session {
Some(crate::face_tiles::PreviewCut { path, loader, want, min, rect, aspect, edge })
}
/// Size-independent cache key of a photo's view render (loupe) for its current settings, its lens profile
/// included as in [`Self::variant_key`]. `profile_salt`: [`MediaCache::camera_profile_salt`] of the photo's profile.
pub(crate) fn view_key(p: &Photo, apply_crop: bool, profile_salt: u64) -> Hash128 {
/// Size-independent cache key of a photo's view render (loupe) for its current settings. The
/// denoised picture, camera profile and lens profile in use are part of it (as of the render's own
/// key, [`Self::variant_key`]): a view rendered before the photo's AI Denoise result was there is
/// not shown as the denoised one.
pub(crate) fn view_key(&self, p: &Photo, apply_crop: bool) -> Hash128 {
let salt = self.media.denoise_salt(p.id, &p.develop);
let profile_salt = self.media.camera_profile_salt(&p.develop.profile.id);
let mut k = Hasher128::new();
k.str(&content_key(p)).str("view").u64(p.develop.hash64()).u64(apply_crop as u64).u64(RENDER_CACHE_VERSION);
with_process(&mut k, &p.develop).u64(crate::camera_profiles::cache_key());
if salt != 0 {
k.u64(salt);
}
if profile_salt != 0 {
k.str("dcp").u64(profile_salt);
}
@@ -285,8 +292,7 @@ impl crate::Session {
pub fn loupe_job(&mut self, id: PhotoId, max_w: usize, max_h: usize, apply_crop: bool) -> Option<RenderJob> {
let mut job = self.render_job(id, max_w, max_h, false, apply_crop)?;
let p = self.catalog.photo(id)?;
let profile_salt = self.media.camera_profile_salt(&p.develop.profile.id);
job.view_cache = Some((self.media.rendered.clone(), Self::view_key(p, apply_crop, profile_salt)));
job.view_cache = Some((self.media.rendered.clone(), self.view_key(p, apply_crop)));
Some(job)
}
@@ -396,11 +402,49 @@ impl crate::Session {
let small = self.thumb_job(id, THUMB_SIZES[THUMB_SIZES.len() - 1])?;
// (the thumbnail job itself starts with the cached thumbnail, after the sharper embedded preview)
let profile_salt = self.media.camera_profile_salt(&p.develop.profile.id);
let cached = vec![(self.media.rendered.clone(), Self::view_key(&p, apply_crop, profile_salt))];
let h = Hasher128::new().str(&content_key(&p)).str("quick").u64(p.develop.hash64()).u64(apply_crop as u64).u64(profile_salt).finish();
let cached = vec![(self.media.rendered.clone(), self.view_key(&p, apply_crop))];
// (the photo too: a virtual copy shares the file and, unedited, the settings)
let h =
Hasher128::new().str(&content_key(&p)).str("quick").u64(p.develop.hash64()).u64(apply_crop as u64).u64(id.0).u64(profile_salt).finish();
let key = h.0 as u64;
let embedded = self.embedded_of(&p).map(|(path, l)| (path, l, max_edge.clamp(1, SourceLevel::Preview.max_edge())));
Some(QuickJob { request_id: 0, photo: id, key, cached, embedded, minimum_embedded_edge: None, small: apply_crop.then(|| Box::new(small)) })
let embedded_cache = embedded.as_ref().map(|(_, _, edge)| {
let cache = self.media.rendered.clone();
let generation = cache.generation();
(cache, Hasher128::new().str(&content_key(&p)).str("embedded").u64(*edge as u64).finish(), generation)
});
Some(QuickJob {
request_id: 0,
photo: id,
key,
cached,
embedded,
embedded_cache,
embedded_on_disk: false,
minimum_embedded_edge: None,
small: apply_crop.then(|| Box::new(small)),
})
}
/// The camera's embedded JPEG of an unedited raw at up to `max_edge` pixels (its full size for
/// a camera that stores one), for the loupe zoomed to 100 % while LightCraft renders the window
/// on screen: a focus check needs the pixels at once, and a 45 MP render of the window takes
/// seconds. Not kept in any cache (a full-size picture is 180 MB). `None` for other photos.
pub fn embedded_full_job(&mut self, id: PhotoId, max_edge: usize) -> Option<QuickJob> {
let p = self.catalog.photo(id)?.clone();
let (path, l) = self.embedded_of(&p)?;
let key = Hasher128::new().str(&content_key(&p)).str("embedded-full").u64(id.0).u64(max_edge as u64).finish().0 as u64;
Some(QuickJob {
request_id: 0,
photo: id,
key,
cached: Vec::new(),
embedded: Some((path, l, max_edge.max(1))),
embedded_cache: None,
embedded_on_disk: false,
minimum_embedded_edge: None,
small: None,
})
}
/// A first grid thumbnail for an unedited raw: the cached render for `job` (the real
@@ -419,6 +463,10 @@ impl crate::Session {
key: job.key,
cached: job.cache.clone().into_iter().collect(),
embedded: Some((path, l, edge)),
// the camera JPEG is this thumbnail (an unedited raw): kept with the rendered ones, under
// a key of its own, so the folder's next visit reads it from the thumbnail cache
embedded_cache: job.cache.as_ref().map(|(c, k)| (c.clone(), embedded_thumb_key(*k), c.generation())),
embedded_on_disk: true,
minimum_embedded_edge: None,
small: Some(Box::new(job.clone())),
})
crates/engine/src/media/source.rs
+14 -1
View File
@@ -24,10 +24,23 @@ pub fn thumb_bucket(long_edge: usize) -> usize {
/// Memory budgets.
pub(super) const THUMB_SOURCE_BYTES: usize = 384 << 20;
/// The least memory kept for rendered previews (when the budget allows it).
pub(super) const RENDERED_MEM_BYTES: usize = 128 << 20;
/// Disk budget for the thumbnail cache.
/// The smallest disk budget for the thumbnail and preview cache.
pub const DISK_CACHE_BYTES: u64 = 2 << 30;
/// What one photo's previews take on disk, about: its standard preview (a 2048 px JPEG, ~1 MB)
/// and its grid thumbnails. The default disk budget grows by this per photo of the library, so
/// built previews of a whole wedding are not pruned while they are being made (PERF-PREVIEW-REUSE).
pub const DISK_CACHE_PER_PHOTO: u64 = 3 << 19;
/// The default disk budget for a library of `photos`: [`DISK_CACHE_BYTES`], or room for every
/// photo's previews ([`DISK_CACHE_PER_PHOTO`]) when that is more.
pub fn default_disk_cache_bytes(photos: usize) -> u64 {
DISK_CACHE_BYTES.max((photos as u64).saturating_mul(DISK_CACHE_PER_PHOTO))
}
/// A cached view render this much smaller than asked for still serves the loupe (rounding: a
/// fit of the same frame into a box one pixel different).
pub(super) const VIEW_REUSE_SLACK: f64 = 0.01;
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
crates/engine/src/media/thumbnail_hash_tests.rs
+99 -2
View File
@@ -6,6 +6,60 @@ fn session() -> crate::Session {
s.catalog.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
s
}
/// PERF-CULL: an unedited raw's camera JPEG is read once for the loupe and kept in memory for a
/// step back; the full-size one (zoomed in) is made at the size asked for and kept nowhere; an
/// edited raw or another file has none.
#[test]
fn camera_previews_are_kept_for_a_revisit_and_full_size_ones_are_not() {
let reads = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let mut s = crate::Session::new();
let r = reads.clone();
s.media.preview_loader = Some(Arc::new(move |_: &str, edge: usize| {
r.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
Some(Rgba8 { width: edge.min(64), height: 2, data: vec![[9, 8, 7, 255]; edge.min(64) * 2] })
}));
let mut raw = Photo::new(PhotoId(1), Source::File { path: "a.cr3".into() }, "a.cr3", "CR3", 6000, 4000, "2026-01-01");
raw.kind = MediaKind::Raw;
raw.develop = Arc::new(raw.camera_defaults());
s.catalog.apply(Op::AddPhoto { photo: Box::new(raw) }).unwrap();
let jpeg = Photo::new(PhotoId(2), Source::File { path: "b.jpg".into() }, "b.jpg", "JPEG", 600, 400, "2026-01-01");
s.catalog.apply(Op::AddPhoto { photo: Box::new(jpeg) }).unwrap();
for _ in 0..3 {
let q = s.quick_view_job(PhotoId(1), 48, true).unwrap();
let r = q.run();
assert_eq!(r.quick, Some(QuickSource::Embedded));
}
assert_eq!(reads.load(std::sync::atomic::Ordering::SeqCst), 1, "read once, then from memory");
let full = s.embedded_full_job(PhotoId(1), 8192).unwrap();
assert!(full.embedded_cache.is_none() && full.cached.is_empty() && full.small.is_none());
assert_eq!(full.embedded.as_ref().map(|e| e.2), Some(8192));
assert_ne!(full.key, s.quick_view_job(PhotoId(1), 48, true).unwrap().key);
assert_eq!(full.run().quick, Some(QuickSource::Embedded));
assert!(s.embedded_full_job(PhotoId(2), 8192).is_none(), "not a raw");
let mut edited = (*s.catalog.photo(PhotoId(1)).unwrap().develop).clone();
edited.light.exposure = 1.0;
s.catalog.apply(Op::SetDevelop { id: PhotoId(1), settings: Arc::new(edited), label: "edit".into(), edited: None }).unwrap();
assert!(s.embedded_full_job(PhotoId(1), 8192).is_none(), "an edit is shown rendered");
}
/// A virtual copy shares its master's file and, unedited, its settings; its loupe stand-ins
/// are its own all the same (CodeRabbit on #3080).
#[test]
fn a_virtual_copy_has_its_own_stand_in_keys() {
let mut s = crate::Session::new();
s.media.preview_loader = Some(Arc::new(|_: &str, _: usize| None::<Rgba8>));
for id in [1, 2] {
let mut raw = Photo::new(PhotoId(id), Source::File { path: "a.cr3".into() }, "a.cr3", "CR3", 6000, 4000, "2026-01-01");
raw.kind = MediaKind::Raw;
raw.develop = Arc::new(raw.camera_defaults());
s.catalog.apply(Op::AddPhoto { photo: Box::new(raw) }).unwrap();
}
let quick = |s: &mut crate::Session, id: u64| s.quick_view_job(PhotoId(id), 48, true).unwrap().key;
assert_ne!(quick(&mut s, 1), quick(&mut s, 2));
let full = |s: &mut crate::Session, id: u64| s.embedded_full_job(PhotoId(id), 8192).unwrap().key;
assert_ne!(full(&mut s, 1), full(&mut s, 2));
}
/// Settings saved by a newer LightCraft (a process number this build doesn't know) render with
/// this build's latest process, and a build that resolved the number to an older process
/// cached other pixels under the same settings hash: the persistent keys (thumbnails, the
@@ -37,7 +91,7 @@ fn persistent_keys_name_the_process_the_settings_render_with() {
let resolved = u64::from(lightcraft_develop::Process::LATEST.version().0);
let keys = [
(s.thumb_job(PhotoId(1), 256).unwrap().cache.unwrap().1, None, 256),
(crate::Session::view_key(&p, true, 0), Some("view"), 1),
(s.view_key(&p, true), Some("view"), 1),
(crate::Session::variant_key(&p, &newer, 256, 0), Some("variant"), 256),
];
for (key, kind, size) in keys {
@@ -69,7 +123,7 @@ fn v1_and_v2_photos_keep_the_render_cache_keys_they_had() {
};
let keys = [
s.thumb_job(PhotoId(1), 256).unwrap().cache.unwrap().1 == before_v3(None, 256),
crate::Session::view_key(&p, true, 0) == before_v3(Some("view"), 1),
s.view_key(&p, true) == before_v3(Some("view"), 1),
crate::Session::variant_key(&p, &settings, 256, 0) == before_v3(Some("variant"), 256),
];
let old = process <= Process::V2;
@@ -217,6 +271,8 @@ fn a_render_that_panics_fails_instead_of_never_answering() {
key: 9,
cached: Vec::new(),
embedded: Some(("x.arw".into(), Arc::new(|_: &str, _: usize| -> Option<Rgba8> { panic!("preview reader bug") }), 256)),
embedded_cache: None,
embedded_on_disk: false,
minimum_embedded_edge: None,
small: None,
};
@@ -224,3 +280,44 @@ fn a_render_that_panics_fails_instead_of_never_answering() {
assert_eq!((r.request_id, r.key), (7, 9));
assert_eq!(r.rendered.err().as_deref(), Some("the render failed unexpectedly: preview reader bug"));
}
/// CodeRabbit on #3544: a stand-in job decoding a camera JPEG when Discard Preview Cache clears the
/// cache doesn't put the picture back: it writes for the generation it was made in (memory and
/// disk alike), as render jobs do.
#[test]
fn a_camera_jpeg_decoded_across_a_cache_clear_is_not_kept() {
for on_disk in [false, true] {
let cache = Arc::new(PreviewCache::memory(64 << 20));
let made_in = cache.generation();
let key = Hash128(5);
let quick = QuickJob {
request_id: 1,
photo: PhotoId(1),
key: 2,
cached: Vec::new(),
embedded: Some(("x.arw".into(), Arc::new(|_: &str, _: usize| Some(Rgba8::new(16, 16))), 256)),
embedded_cache: Some((cache.clone(), key, made_in)),
embedded_on_disk: on_disk,
minimum_embedded_edge: None,
small: None,
};
// the user discards the cache while the job is on its way
cache.clear();
assert!(quick.run().rendered.is_ok(), "the stand-in is still shown");
assert!(cache.get_memory(key).is_none(), "on disk {on_disk}: nothing kept in the cleared cache");
// a job made after the clear keeps it as before
let quick = QuickJob {
request_id: 1,
photo: PhotoId(1),
key: 2,
cached: Vec::new(),
embedded: Some(("x.arw".into(), Arc::new(|_: &str, _: usize| Some(Rgba8::new(16, 16))), 256)),
embedded_cache: Some((cache.clone(), key, cache.generation())),
embedded_on_disk: on_disk,
minimum_embedded_edge: None,
small: None,
};
assert!(quick.run().rendered.is_ok());
assert!(cache.get_memory(key).is_some(), "on disk {on_disk}: kept when made after the clear");
}
}
crates/engine/src/prep.rs
+532
View File
@@ -0,0 +1,532 @@
//! Preparing photos in the background (PERF-PREVIEW-PREP, notes 47 and 48): while the user culls,
//! the photos ahead are made ready before they are reached, and with the machine otherwise idle
//! the previews of every photo in view are built, so stepping never waits for a render.
//!
//! What is prepared is set per mode, because each mode needs something else:
//! - **Culling** (a folder browsed before import, the Import tab): previews of every photo in the
//! folder, the next photos' loupe renders and their eye focus strips.
//! - **Library**: the same for the photos in the library's view.
//! - **Develop** (editing): the next photos decoded and kept ready to edit.
//!
//! Each mode has its own switch for the whole-view build, how many photos ahead to prepare (0 =
//! none) and which photos ([`Scope`]). The settings are app-wide (the desktop app keeps them in `ui.json`)
//! and a library can keep its own (its `prefs.json`): a timelapse library prepares nothing while a
//! wedding library prepares everything.
//!
//! The whole-view build is the preview build ([`crate::cmd::previews`]) in its background form: two
//! lanes, waiting whenever the photo on screen is being made ([`foreground`]), nearest photos to the
//! user first and re-aimed as they move ([`crate::cmd::previews::PreviewBuild::focus_on`]), photos
//! with previews skipped, one row in the activity stack that can be cancelled. It starts once the
//! user is culling (a few photos stepped through in the loupe), never just because a library was
//! opened (note 36: opening an old library to look at two photos must not start rebuilding).
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use lightcraft_catalog::PhotoId;
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use crate::Session;
use crate::cmd::previews::PreviewBuild;
/// Which photos a mode prepares.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum Scope {
/// Every photo in view (the folder, the collection, the filter).
#[default]
All,
/// Only picks, and photos with stars or a colour label.
Marked,
/// Only the photo on screen: nothing ahead, nothing in the background.
OnScreen,
}
impl Scope {
pub fn parse(s: &str) -> Option<Scope> {
match s {
"all" => Some(Scope::All),
"marked" => Some(Scope::Marked),
"onScreen" => Some(Scope::OnScreen),
_ => None,
}
}
}
/// The most photos prepared ahead of the one on screen.
pub const MAX_AHEAD: u8 = 20;
/// What one mode prepares.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct ModePrep {
/// Build the previews of every photo in scope in the background.
pub previews: bool,
/// Photos ahead (in the direction of travel) whose loupe render is made before they are reached (0 = none).
pub ahead: u8,
/// Photos ahead whose eye focus strip is worked out before they are reached (0 = only the one on
/// screen), for a frontend that shows one (the desktop app doesn't yet).
pub eye_ahead: u8,
pub scope: Scope,
}
impl Default for ModePrep {
fn default() -> Self {
ModePrep { previews: true, ahead: 5, eye_ahead: 8, scope: Scope::All }
}
}
/// Editing: what is kept ready for the next photos.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct DevelopPrep {
/// Photos ahead decoded and kept ready to edit (0 = none).
pub ahead: u8,
pub scope: Scope,
}
impl Default for DevelopPrep {
fn default() -> Self {
DevelopPrep { ahead: 2, scope: Scope::All }
}
}
/// The background preparation settings.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct PrepSettings {
pub culling: ModePrep,
pub library: ModePrep,
pub develop: DevelopPrep,
}
impl PrepSettings {
/// Values from files or agents kept in range.
pub fn sanitized(mut self) -> Self {
for m in [&mut self.culling, &mut self.library] {
m.ahead = m.ahead.min(MAX_AHEAD);
m.eye_ahead = m.eye_ahead.min(MAX_AHEAD);
}
self.develop.ahead = self.develop.ahead.min(MAX_AHEAD);
self
}
/// The settings of a browsing mode ([`Mode::Culling`] or [`Mode::Library`]; Develop prepares
/// views the way the Library does).
pub fn mode(&self, mode: Mode) -> ModePrep {
match mode {
Mode::Culling => self.culling,
Mode::Library | Mode::Develop => self.library,
}
}
/// Photos ahead to render for the loupe in `mode` (0 when its scope is the photo on screen).
pub fn ahead(&self, mode: Mode) -> usize {
match mode {
Mode::Develop if self.develop.scope == Scope::OnScreen => 0,
Mode::Develop => usize::from(self.develop.ahead),
m => {
let p = self.mode(m);
if p.scope == Scope::OnScreen { 0 } else { usize::from(p.ahead) }
}
}
}
/// Photos ahead whose eye focus strip is worked out in `mode`.
pub fn eye_ahead(&self, mode: Mode) -> usize {
let p = self.mode(mode);
if p.scope == Scope::OnScreen { 0 } else { usize::from(p.eye_ahead) }
}
/// The scope that applies in `mode`.
pub fn scope(&self, mode: Mode) -> Scope {
match mode {
Mode::Develop => self.develop.scope,
m => self.mode(m).scope,
}
}
/// Apply the fields of `patch` (a partial object, as `prefs.setBackgroundPrep` takes it).
pub fn patched(&self, patch: &Value) -> Result<PrepSettings, String> {
let mut v = serde_json::to_value(self).map_err(|e| e.to_string())?;
merge(&mut v, patch, 0);
let p: PrepSettings = serde_json::from_value(v).map_err(|e| format!("background preparation settings: {e}"))?;
Ok(p.sanitized())
}
}
/// `patch` merged into `into`, objects field by field (two levels deep: mode → field).
fn merge(into: &mut Value, patch: &Value, depth: usize) {
match (into, patch) {
(Value::Object(a), Value::Object(b)) if depth < 3 => {
for (k, v) in b {
match a.get_mut(k) {
Some(old) => merge(old, v, depth + 1),
None => {
a.insert(k.clone(), v.clone());
}
}
}
}
(a, b) => *a = b.clone(),
}
}
/// What the user is doing, as far as preparation goes.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum Mode {
Culling,
Library,
Develop,
}
impl Mode {
pub fn parse(s: &str) -> Option<Mode> {
match s {
"culling" => Some(Mode::Culling),
"library" => Some(Mode::Library),
"develop" => Some(Mode::Develop),
_ => None,
}
}
/// The mode of what `s` shows when the frontend doesn't say: a folder browsed before import is
/// culling, anything else the library.
pub fn of(s: &Session) -> Mode {
if s.source == crate::LibrarySource::Folder { Mode::Culling } else { Mode::Library }
}
}
/// Photos the user steps through in the loupe before the whole view is prepared: a library opened
/// to look at one photo starts nothing.
pub const STEPS_BEFORE_BUILD: usize = 3;
/// How long a view whose build couldn't start waits before trying again.
pub const RETRY_AFTER: std::time::Duration = std::time::Duration::from_secs(30);
/// The background preparation's state in a session.
#[derive(Default)]
pub struct PrepState {
/// App-wide settings.
pub app: PrepSettings,
/// This library's own (`None`: it uses the app-wide ones).
pub library: Option<PrepSettings>,
/// The background build this preparation started.
build: Option<Arc<PreviewBuild>>,
/// The view (its signature) whose build was started, or that the user stopped.
started_for: Option<u64>,
declined_for: Option<u64>,
/// A build that couldn't start: the view (its signature), and when it may try again (not on
/// every tick while the cause lasts; CodeRabbit on #3080).
pub retry_at: Option<(u64, web_time::Instant)>,
/// Tests: the next starts to fail, as a build thread that can't be spawned does.
#[cfg(test)]
pub(crate) fail_starts: usize,
/// The view the steps were counted in, the photo last seen, and the steps.
steps_in: Option<u64>,
last_at: Option<PhotoId>,
steps: usize,
}
impl std::fmt::Debug for PrepState {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("PrepState").field("app", &self.app).field("library", &self.library).field("steps", &self.steps).finish()
}
}
impl PrepState {
/// The settings in use.
pub fn settings(&self) -> PrepSettings {
self.library.unwrap_or(self.app)
}
}
/// Until when (ms on [`clock_ms`]) the photo on screen is being made: background work waits.
static FOREGROUND_UNTIL: AtomicU64 = AtomicU64::new(0);
/// How long a foreground signal holds background work back (ms): a little more than a frame
/// apart, so a frontend saying it every frame holds it back the whole time.
const FOREGROUND_HOLD_MS: u64 = 250;
fn clock_ms() -> u64 {
static START: std::sync::OnceLock<web_time::Instant> = std::sync::OnceLock::new();
u64::try_from(START.get_or_init(web_time::Instant::now).elapsed().as_millis()).unwrap_or(u64::MAX)
}
/// The frontend says the photo on screen is being made (a render it waits for is on its way):
/// background preparation waits until it has been quiet for a moment.
pub fn foreground() {
FOREGROUND_UNTIL.store(clock_ms().saturating_add(FOREGROUND_HOLD_MS), Ordering::Relaxed);
}
/// Whether the photo on screen is being made now.
pub fn foreground_busy() -> bool {
#[cfg(test)]
if TEST_HOLD.load(Ordering::Relaxed) > 0 {
return true;
}
clock_ms() < FOREGROUND_UNTIL.load(Ordering::Relaxed)
}
/// Tests: hold every background build at its foreground wait until released (a synchronization
/// point that doesn't depend on the hold's 250 ms). A count, not a switch: tests running side by
/// side each hold until their own release, and one test's release doesn't free another's build.
#[cfg(test)]
static TEST_HOLD: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0);
/// Holds background builds while it lives (also when the test fails), whatever other holds do.
#[cfg(test)]
pub(crate) struct TestHold;
#[cfg(test)]
impl TestHold {
pub(crate) fn new() -> Self {
TEST_HOLD.fetch_add(1, Ordering::Relaxed);
TestHold
}
}
#[cfg(test)]
impl Drop for TestHold {
fn drop(&mut self) {
// (one release per hold made: never below 0)
TEST_HOLD.fetch_sub(1, Ordering::Relaxed);
}
}
/// Wait while the photo on screen is being made. `true`: `cancel` was set meanwhile.
pub fn wait_for_foreground(cancel: &AtomicBool) -> bool {
#[cfg(not(target_arch = "wasm32"))]
while foreground_busy() {
if cancel.load(Ordering::Relaxed) {
return true;
}
std::thread::sleep(std::time::Duration::from_millis(30));
}
cancel.load(Ordering::Relaxed)
}
/// Photos the background build renders side by side: each render spreads over the cores already,
/// so two keep the machine busy while leaving room for the photo on screen.
pub fn background_lanes() -> usize {
let cores = std::thread::available_parallelism().map_or(2, |n| n.get());
if cores >= 8 { 2 } else { 1 }
}
/// A cheap identity of the photos a build prepares: a build is started once per view. `scope`
/// [`Scope::Marked`]: which of them are marked counts too (marking a photo changes the set).
fn signature(ids: &[PhotoId], s: &Session, scope: Scope) -> u64 {
let mut h = lightcraft_preview::Hasher128::new();
h.u64(ids.len() as u64).u64(s.catalog.len() as u64);
// every id: a sort that moves photos in the middle is another view
for id in ids {
h.u64(id.0);
if scope == Scope::Marked {
h.u64(u64::from(marked(s, *id)));
}
}
h.finish().0 as u64
}
/// Whether `id` is a pick, or has stars or a colour label (the [`Scope::Marked`] photos).
pub fn marked(s: &Session, id: PhotoId) -> bool {
s.catalog.photo(id).is_some_and(|p| p.flag == lightcraft_catalog::Flag::Pick || p.rating > 0 || p.label.is_some())
}
impl Session {
/// How long until a view whose build couldn't start tries again ([`RETRY_AFTER`]), while one
/// waits: the app asks for a frame then, so the retry doesn't wait for the next input
/// (CodeRabbit on #3077/#3080). `None` once the time has come.
pub fn prep_retry_in(&self) -> Option<std::time::Duration> {
// (a deadline that has passed asks for nothing: the pass at it tried again, or the view
// can't be tried now and a frame at once on every pass would keep an idle window drawing:
// CodeRabbit on #3077 / #3080)
let wait = self.prep.retry_at.map(|(_, at)| at.saturating_duration_since(web_time::Instant::now()))?;
(!wait.is_zero()).then_some(wait)
}
/// The background preparation settings in use (this library's own, else the app-wide ones).
pub fn prep_settings(&self) -> PrepSettings {
self.prep.settings()
}
/// Advance the background preparation (cheap; call every frame). `mode`: what the user is doing;
/// `at`: the photo in the loupe (`None` in the grid), `backward`: stepping back.
pub fn prep_tick(&mut self, mode: Mode, at: Option<PhotoId>, backward: bool) {
let settings = self.prep.settings();
let want = settings.mode(mode);
// the whole view's build follows its own row (Culling, or Library also while editing): its
// switch and its "which photos"; the Editing row's scope is for the photos kept ready ahead
let wanted = want.previews && want.scope != Scope::OnScreen;
// the running build: re-aimed at the user, stopped when switched off
if let Some(b) = self.prep.build.clone() {
if b.running() {
if !wanted {
b.cancel.store(true, Ordering::Relaxed);
} else if self.prep.started_for.is_some_and(|sig| sig != signature(&self.visible_cloned(), self, want.scope)) {
// another view (a folder, a filter, another library): this build's photos are
// not the ones shown; stopped, and the view now shown is prepared in its place
// (not counted as stopped by the user)
b.cancel.store(true, Ordering::Relaxed);
self.prep.started_for = None;
} else if let Some(id) = at {
b.focus_on(id, backward);
}
return;
}
self.prep.build = None;
let mine = self.preview_build.as_ref().is_some_and(|p| Arc::ptr_eq(p, &b));
if b.cancel.load(Ordering::Relaxed) {
if mine && wanted {
// stopped from its row: not again for this view
self.prep.declined_for = self.prep.started_for;
} else {
// replaced by a build that was asked for, or switched off: may start again later
self.prep.started_for = None;
}
}
}
if !wanted {
return;
}
let Some(at) = at else { return };
let ids = self.visible_cloned();
let sig = signature(&ids, self, want.scope);
if self.prep.steps_in != Some(sig) {
self.prep.steps_in = Some(sig);
self.prep.steps = 0;
self.prep.last_at = None;
}
if self.prep.last_at != Some(at) {
self.prep.last_at = Some(at);
self.prep.steps = self.prep.steps.saturating_add(1);
}
let busy = self.preview_build.as_ref().is_some_and(|b| b.running());
if busy
|| self.prep.steps < STEPS_BEFORE_BUILD
|| self.prep.started_for == Some(sig)
|| self.prep.declined_for == Some(sig)
|| self.prep.retry_at.is_some_and(|(view, at)| view == sig && web_time::Instant::now() < at)
|| self.library.as_ref().is_none_or(|l| !l.on_disk)
{
return;
}
let ids: Vec<u64> = match want.scope {
Scope::Marked => ids.into_iter().filter(|id| marked(self, *id)).map(|id| id.0).collect(),
_ => ids.into_iter().map(|id| id.0).collect(),
};
self.prep.started_for = Some(sig);
if ids.is_empty() {
return;
}
let p = json!({"ids": ids, "background": true, "from": at.0, "backward": backward});
// (tests: a start made to fail, as a build thread that can't be spawned fails)
#[cfg(test)]
let fail = self.prep.fail_starts.checked_sub(1).map(|left| self.prep.fail_starts = left).is_some();
#[cfg(not(test))]
let fail = false;
let started = if fail {
Err(crate::EngineError::Disabled("library.buildPreviews".into(), "test: the build couldn't start".into()))
} else {
self.execute("library.buildPreviews", &p)
};
match started {
Ok(_) => {
self.prep.build = self.preview_build.clone();
self.prep.retry_at = None;
}
Err(e) => {
// not started: this view tries again after a while (not on every tick)
self.prep.started_for = None;
self.prep.retry_at = web_time::Instant::now().checked_add(RETRY_AFTER).map(|at| (sig, at));
log::info!("background preparation: {e}");
}
}
}
/// Change the background preparation settings: the app-wide ones, or (`library`) this library's
/// own. Saved at once.
pub fn set_prep_settings(&mut self, settings: PrepSettings, library: bool) -> crate::Result<()> {
let settings = settings.sanitized();
if library {
let previous = self.prep.library;
self.prep.library = Some(settings);
self.save_library_prep(previous)
} else {
let previous = self.prep.app;
self.prep.app = settings;
let saved = self.save_prefs();
if saved.is_err() {
self.prep.app = previous;
}
saved
}
}
/// Back to the app-wide settings for this library.
pub fn clear_library_prep(&mut self) -> crate::Result<()> {
let previous = self.prep.library;
self.prep.library = None;
self.save_library_prep(previous)
}
/// Save this library's preparation settings; when that fails, `previous` is put back, so the
/// session doesn't go on with settings the command reported as not changed (CodeRabbit on #3544).
fn save_library_prep(&mut self, previous: Option<PrepSettings>) -> crate::Result<()> {
let saved = self.save_prefs();
if saved.is_err() {
self.prep.library = previous;
}
saved
}
pub(crate) fn prep_json(&self) -> Value {
json!({
"app": self.prep.app,
"library": self.prep.library,
"settings": self.prep.settings(),
"own": self.prep.library.is_some(),
"building": self.prep.build.as_ref().filter(|b| b.running()).map(|b| b.json()),
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_patch_changes_only_what_it_names_and_is_kept_in_range() {
let p = PrepSettings::default().patched(&json!({"culling": {"ahead": 99}, "develop": {"scope": "onScreen"}})).unwrap();
assert_eq!(p.culling.ahead, MAX_AHEAD);
assert_eq!(p.culling.eye_ahead, 8, "untouched");
assert!(p.library.previews);
assert_eq!(p.develop.scope, Scope::OnScreen);
assert_eq!(p.ahead(Mode::Develop), 0);
assert!(PrepSettings::default().patched(&json!({"culling": {"scope": "sometimes"}})).is_err());
}
/// Tests side by side: one test's release doesn't free the build another test holds (CodeRabbit on #3077).
#[test]
fn one_tests_release_does_not_free_another_tests_hold() {
let a = TestHold::new();
let b = TestHold::new();
drop(a);
assert!(TEST_HOLD.load(Ordering::Relaxed) > 0 && foreground_busy(), "the other hold still holds");
drop(b);
}
#[test]
fn the_photo_on_screen_holds_background_work_back() {
let cancel = AtomicBool::new(false);
foreground();
assert!(foreground_busy());
let t = web_time::Instant::now();
assert!(!wait_for_foreground(&cancel));
assert!(t.elapsed().as_millis() >= 100, "waited for the foreground to go quiet");
cancel.store(true, Ordering::Relaxed);
foreground();
assert!(wait_for_foreground(&cancel), "a cancel ends the wait");
}
}
crates/engine/src/tests.rs
+1
View File
@@ -44,6 +44,7 @@ mod organize;
mod persist;
mod prefs;
mod presets_scale;
mod preview_prep;
mod print;
mod process;
mod recheck;
crates/engine/src/tests/import.rs
+209
View File
@@ -303,6 +303,149 @@ fn browsing_a_folder_lists_its_photos_without_adding_them() {
let _ = std::fs::remove_dir_all(&dir);
}
/// PERF-CULL-1: the Local view reads only what the library lacks, newest file first for the default
/// (newest capture first) sort, so the first screen is read first; other sorts follow their key.
#[test]
fn browsing_queues_unknown_files_in_the_order_the_view_shows_them() {
use lightcraft_catalog::{Sort, SortKey};
let dir = temp_dir("browse-order");
let set_time = |p: &Path, secs: u64| {
let t = std::time::UNIX_EPOCH + std::time::Duration::from_secs(secs);
std::fs::File::options().write(true).open(p).unwrap().set_modified(t).unwrap();
};
for (name, seed, secs) in [("a.png", 1u8, 300u64), ("b.png", 2, 100), ("c.png", 3, 200)] {
write_png(&dir.join(name), seed);
set_time(&dir.join(name), 1_700_000_000 + secs);
}
let mut s = Session::new().with_fs();
let known = dir.join("b.png").to_string_lossy().to_string();
s.execute("library.import", &json!({"paths": [known.clone()]})).unwrap();
let files: Vec<String> = ["a.png", "b.png", "c.png", "gone.png"].iter().map(|n| dir.join(n).to_string_lossy().to_string()).collect();
let (input, _) = crate::import::ScanInput::new(&mut s, &[]);
let names = |v: Vec<String>| v.iter().map(|f| Path::new(f).file_name().unwrap().to_string_lossy().to_string()).collect::<Vec<_>>();
// newest first, the file already in the library left out, an unreadable one last
let q = crate::import::browse_queue(&input, files.clone(), &Sort::default());
assert_eq!(names(q), ["a.png", "c.png", "gone.png"]);
let q = crate::import::browse_queue(&input, files.clone(), &Sort { ascending: true, ..Sort::default() });
assert_eq!(names(q), ["c.png", "a.png", "gone.png"]);
let q = crate::import::browse_queue(&input, files.clone(), &Sort { key: SortKey::FileName, ascending: false, ..Sort::default() });
assert_eq!(names(q), ["gone.png", "c.png", "a.png"]);
// by size an unreadable file is last in both directions (it was sized 0, first when ascending)
for ascending in [true, false] {
let q = crate::import::browse_queue(&input, files.clone(), &Sort { key: SortKey::FileSize, ascending, ..Sort::default() });
assert_eq!(names(q).last().map(String::as_str), Some("gone.png"), "ascending {ascending}");
}
let q = crate::import::browse_queue(&input, files, &Sort { key: SortKey::Rating, ..Sort::default() });
assert_eq!(names(q), ["gone.png", "c.png", "a.png"], "new photos tie on rating: by name, descending as the view");
let _ = std::fs::remove_dir_all(&dir);
}
/// Choosing another folder while a browse orders its files (`cancel` set) stops the ordering: no queue comes
/// back, and nothing more is read (CodeRabbit on #3055). Browsing hands back the probes of earlier scans it took.
#[test]
fn a_cancelled_browse_stops_ordering_and_keeps_earlier_probes() {
use lightcraft_catalog::{Sort, SortKey};
use std::sync::atomic::AtomicBool;
let dir = temp_dir("browse-cancel");
write_png(&dir.join("a.png"), 1);
write_png(&dir.join("b.png"), 2);
let files: Vec<String> = ["a.png", "b.png"].iter().map(|n| dir.join(n).to_string_lossy().to_string()).collect();
let mut s = Session::new().with_fs();
s.import_probes.insert(files[0].clone(), crate::media::ProbeInfo { file_size: 1, ..Default::default() });
let (mut input, _) = crate::import::ScanInput::new(&mut s, &[]);
for key in [SortKey::CaptureDate, SortKey::FileSize, SortKey::FileName, SortKey::Random] {
let sort = Sort { key, ..Sort::default() };
assert_eq!(crate::import::browse_queue_until(&input, files.clone(), &sort, &AtomicBool::new(true)), None, "{key:?}");
assert_eq!(crate::import::browse_queue_until(&input, files.clone(), &sort, &AtomicBool::new(false)).map(|q| q.len()), Some(2), "{key:?}");
}
let probes = input.take_probes();
assert!(probes.contains_key(&files[0]), "the earlier scan's probe comes back");
let _ = std::fs::remove_dir_all(&dir);
}
/// Browsing a folder with its subfolders lists nothing inside the library's own folder (the
/// import that follows would read it as photos), as the import's scan skips it.
#[test]
fn browsing_with_subfolders_skips_the_librarys_own_folder() {
let dir = temp_dir("browse-skip-lib");
std::fs::create_dir_all(dir.join("lib")).unwrap();
let mut s = Session::new().with_fs();
s.open_library(dir.join("lib"), false).unwrap();
write_png(&dir.join("a.png"), 1);
write_png(&dir.join("lib").join("inside.png"), 2);
let (input, _) = crate::import::ScanInput::new(&mut s, &[]);
let root = dir.to_string_lossy().to_string();
let names = |v: Vec<String>| v.iter().map(|f| Path::new(f).file_name().unwrap().to_string_lossy().to_string()).collect::<Vec<_>>();
assert_eq!(names(crate::import::browse_files(&input, &root, true).unwrap()), ["a.png"]);
assert_eq!(names(crate::import::browse_files(&input, &root, false).unwrap()), ["a.png"]);
// the library's folder chosen itself, with or without its subfolders
let lib = dir.join("lib").to_string_lossy().to_string();
assert!(crate::import::browse_files(&input, &lib, false).unwrap().is_empty());
assert!(crate::import::browse_files(&input, &lib, true).unwrap().is_empty());
// the library's folder by another name: through `..`, or a symlink to it
std::fs::create_dir_all(dir.join("other")).unwrap();
let around = dir.join("other").join("..").join("lib").to_string_lossy().to_string();
assert!(crate::import::browse_files(&input, &around, false).unwrap().is_empty(), "{around}");
assert!(crate::import::browse_files(&input, &around, true).unwrap().is_empty(), "{around}");
#[cfg(unix)]
{
let alias = dir.join("alias");
std::os::unix::fs::symlink(dir.join("lib"), &alias).unwrap();
let alias = alias.to_string_lossy().to_string();
assert!(crate::import::browse_files(&input, &alias, false).unwrap().is_empty(), "{alias}");
assert!(crate::import::browse_files(&input, &alias, true).unwrap().is_empty(), "{alias}");
}
// a folder next to the library named through it is outside it: browsed (CodeRabbit on #3077)
std::fs::create_dir_all(dir.join("outside")).unwrap();
write_png(&dir.join("outside").join("next.png"), 3);
let next = dir.join("lib").join("..").join("outside").to_string_lossy().to_string();
assert_eq!(names(crate::import::browse_files(&input, &next, false).unwrap()), ["next.png"], "{next}");
assert_eq!(names(crate::import::browse_files(&input, &next, true).unwrap()), ["next.png"], "{next}");
// under the folder's one spelling: the paths match the folder's own, so the catalog's records are found
// (CodeRabbit on #3055)
let direct = dir.join("outside").to_string_lossy().to_string();
for sub in [false, true] {
assert_eq!(crate::import::browse_files(&input, &next, sub).unwrap(), crate::import::browse_files(&input, &direct, sub).unwrap());
}
// a folder inside the library that isn't there is an error too, not an empty folder
let gone = dir.join("lib").join("not-there").to_string_lossy().to_string();
assert!(crate::import::browse_files(&input, &gone, false).is_err(), "{gone}");
// a folder that can't be listed is an error, not an empty folder (its photos aren't "gone")
let missing = dir.join("not-there").to_string_lossy().to_string();
assert!(crate::import::browse_files(&input, &missing, false).is_err());
assert!(crate::import::browse_files(&input, &missing, true).is_err());
let _ = std::fs::remove_dir_all(&dir);
}
/// Browsing's queue breaks ties the way the catalog does (by name, in the view's direction), and
/// the import- and edit-date sorts, where new photos tie, go by name rather than file time.
#[test]
fn browsing_breaks_ties_by_name_in_the_views_direction() {
use lightcraft_catalog::{Sort, SortKey};
let dir = temp_dir("browse-ties");
// names and file times run in opposite directions; x and y share a time and a size
for (name, secs) in [("x.png", 100u64), ("y.png", 100), ("z.png", 50)] {
write_png(&dir.join(name), 7);
let t = std::time::UNIX_EPOCH + std::time::Duration::from_secs(1_700_000_000 + secs);
std::fs::File::options().write(true).open(dir.join(name)).unwrap().set_modified(t).unwrap();
}
let files: Vec<String> = ["y.png", "z.png", "x.png"].iter().map(|n| dir.join(n).to_string_lossy().to_string()).collect();
let order = |key: SortKey, ascending: bool| {
let mut f = files.clone();
crate::import::browse_order(&mut f, &Sort { key, ascending, ..Sort::default() });
f.iter().map(|f| Path::new(f).file_name().unwrap().to_string_lossy().to_string()).collect::<Vec<_>>()
};
assert_eq!(order(SortKey::CaptureDate, false), ["y.png", "x.png", "z.png"]);
assert_eq!(order(SortKey::CaptureDate, true), ["z.png", "x.png", "y.png"]);
for key in [SortKey::ImportDate, SortKey::EditDate, SortKey::Rating] {
assert_eq!(order(key, true), ["x.png", "y.png", "z.png"], "{key:?}");
assert_eq!(order(key, false), ["z.png", "y.png", "x.png"], "{key:?}");
}
assert_eq!(order(SortKey::FileSize, true), ["x.png", "y.png", "z.png"]);
assert_eq!(order(SortKey::FileSize, false), ["z.png", "y.png", "x.png"]);
let _ = std::fs::remove_dir_all(&dir);
}
/// Photos a.png, b.png, c.png imported from `old`, then: a renamed into `moved`; b renamed, with
/// an impostor (same name and size, other bytes) under its old name; c gone, with two impostors.
fn renamed_and_ambiguous_scene(tag: &str) -> (std::path::PathBuf, Session) {
@@ -2785,3 +2928,69 @@ fn a_file_arriving_beside_its_raws_stem_reads_its_own_sidecar() {
assert_eq!(read(&s, "IMG_3.TIF"), 0, "the catalogued raw owns IMG_3.xmp");
let _ = std::fs::remove_dir_all(&src);
}
/// A folder's one spelling: `.` is the current folder, `.` and `..` are taken out, never above the
/// root (CodeRabbit on #3055 / #3080).
#[test]
fn a_browsed_folder_has_one_spelling() {
use crate::import::browse_path;
let root = std::env::temp_dir();
assert_eq!(browse_path(&root.join("lib").join("..").join("outside")).unwrap(), root.join("outside"));
assert_eq!(browse_path(&root.join(".").join("a").join(".").join("b")).unwrap(), root.join("a").join("b"));
let top: PathBuf = root.ancestors().last().map(Path::to_path_buf).unwrap_or_default();
assert_eq!(browse_path(&top.join("..").join("x")).unwrap(), top.join("x"), "never above the root");
assert_eq!(browse_path(Path::new(".")).unwrap(), std::env::current_dir().unwrap(), "`.` is the current folder");
assert_eq!(browse_path(Path::new("a/..")).unwrap(), std::env::current_dir().unwrap());
// the UI's spelling, without the disk: the same where no link comes before a `..`
let by_name = crate::import::browse_spelling(&root.join("lib").join("..").join("outside")).unwrap();
assert_eq!(by_name, root.join("outside"));
assert_eq!(crate::import::browse_spelling(Path::new(".")).unwrap(), std::env::current_dir().unwrap());
}
/// `link/..` is the folder the system reads (POSIX: above where the link points; Windows: the
/// folder holding the link): browsing it lists what the system lists there (CodeRabbit on #3055 /
/// #3077 / #3080).
#[test]
fn a_link_before_dotdot_names_the_folder_the_system_reads() {
let dir = temp_dir("browse-link-dotdot");
std::fs::create_dir_all(dir.join("real").join("sub")).unwrap();
write_png(&dir.join("real").join("in-real.png"), 1);
write_png(&dir.join("beside-link.png"), 2);
let link = dir.join("link");
#[cfg(unix)]
let made = std::os::unix::fs::symlink(dir.join("real").join("sub"), &link);
#[cfg(windows)]
let made = std::os::windows::fs::symlink_dir(dir.join("real").join("sub"), &link);
if let Err(e) = made {
lightcraft_testkit::skip!(System, "can't make a folder link here: {e}");
let _ = std::fs::remove_dir_all(&dir);
return;
}
let mut s = Session::new().with_fs();
let (input, _) = crate::import::ScanInput::new(&mut s, &[]);
let through = link.join("..");
let names = |v: Vec<String>| v.iter().map(|f| Path::new(f).file_name().unwrap().to_string_lossy().to_string()).collect::<Vec<_>>();
let listed = names(crate::import::browse_files(&input, &through.to_string_lossy(), false).unwrap());
let mut system: Vec<String> =
std::fs::read_dir(&through).unwrap().flatten().map(|e| e.file_name().to_string_lossy().to_string()).filter(|n| n.ends_with(".png")).collect();
system.sort();
assert_eq!(listed, system, "{}", through.display());
#[cfg(unix)]
assert_eq!(listed, ["in-real.png"]);
let _ = std::fs::remove_dir_all(&dir);
}
/// A broken link before `..` is an error, not the link's own folder by name: the system can't read
/// `broken/..` either (CodeRabbit on #3577).
#[cfg(unix)]
#[test]
fn a_broken_link_before_dotdot_is_an_error() {
let dir = temp_dir("browse-broken-link-dotdot");
std::fs::create_dir_all(&dir).unwrap();
let link = dir.join("broken");
std::os::unix::fs::symlink(dir.join("nowhere"), &link).unwrap();
assert!(crate::import::browse_path(&link.join("..")).is_err());
// without reading the disk the spelling is still the folder by name
assert_eq!(crate::import::browse_spelling(&link.join("..")).unwrap(), dir);
let _ = std::fs::remove_dir_all(&dir);
}
crates/engine/src/tests/lens_refresh.rs
+1 -1
View File
@@ -237,7 +237,7 @@ fn a_new_database_profile_changes_the_render_and_cache_keys() {
let render = s.render_job(id, 800, 600, false, true).unwrap().key;
let thumb = s.thumb_job(id, 256).unwrap().cache.unwrap().1;
let p = s.catalog.photo(id).unwrap().clone();
(render, thumb, Session::variant_key(&p, &p.develop, 128, 0), Session::view_key(&p, true, 0))
(render, thumb, Session::variant_key(&p, &p.develop, 128, 0), s.view_key(&p, true))
};
let before = keys(&mut s);
s.catalog.apply(Op::SetEmbeddedLens { id, lens: Some(Box::new(lens(-0.03))) }).unwrap();
crates/engine/src/tests/preview_prep.rs
+635
View File
@@ -0,0 +1,635 @@
//! Previews are made once and used (notes 47–49): the loupe shows a built or prepared preview
//! instead of rendering the photo again, a build skips photos whose previews are there, built
//! previews are on disk when the build says it is done, the import's queue shows one row, and the
//! background preparation follows the user and gives way to a build that was asked for.
use std::path::{Path, PathBuf};
use std::sync::atomic::Ordering;
use lightcraft_catalog::PhotoId;
use serde_json::json;
use crate::Session;
use crate::media::QuickSource;
fn temp_dir(tag: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("lc-preview-prep-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
}
fn png(p: &Path, seed: u8) {
let img = lightcraft_raster::Rgba8::from_fn(600, 400, |x, y| [(x % 251) as u8, (y % 241) as u8, seed, 255]);
let b =
crate::export::encode_image(&img, &crate::export::ExportOptions { format: crate::export::ExportFormat::Png, ..Default::default() }).unwrap();
std::fs::create_dir_all(p.parent().unwrap()).unwrap();
std::fs::write(p, b).unwrap();
}
/// A library on disk with `n` photos (600 × 400 PNGs) imported, in name order.
fn library(tag: &str, n: usize) -> (Session, PathBuf, Vec<PhotoId>) {
let root = temp_dir(tag);
let paths: Vec<String> = (0..n)
.map(|i| {
let p = root.join("shoot").join(format!("p{i:02}.png"));
png(&p, (i * 20) as u8);
p.to_string_lossy().to_string()
})
.collect();
let mut s = Session::new().with_fs();
s.open_library(root.join("lib"), false).unwrap();
s.execute("library.import", &json!({"paths": paths})).unwrap();
let mut ids: Vec<(String, PhotoId)> = s.catalog.photos().map(|p| (p.file_name.clone(), p.id)).collect();
ids.sort();
(s, root, ids.into_iter().map(|(_, id)| id).collect())
}
fn ids_json(ids: &[PhotoId]) -> Vec<u64> {
ids.iter().map(|i| i.0).collect()
}
/// FredApps on #3077: building previews (asked for, and the background preparation while culling)
/// writes only into the library's own cache, never next to the photos: the shoot folder holds the
/// same files afterwards, byte for byte.
#[test]
fn building_previews_writes_nothing_next_to_the_photos() {
let (mut s, root, ids) = library("prep-beside", 4);
let shoot = root.join("shoot");
let listing = |dir: &std::path::Path| {
let mut v: Vec<(String, Vec<u8>)> = std::fs::read_dir(dir)
.unwrap()
.map(|e| e.unwrap().path())
.map(|p| (p.file_name().unwrap().to_string_lossy().to_string(), std::fs::read(&p).unwrap()))
.collect();
v.sort();
v
};
let before = listing(&shoot);
assert_eq!(before.len(), 4);
let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "wait": true})).unwrap();
assert_eq!(r["done"], 4, "{r}");
for id in &ids {
s.prep_tick(crate::prep::Mode::Library, Some(*id), false);
}
if let Some(b) = s.preview_build.clone() {
assert!(wait_finished(&b), "the background build stops (a late write would slip past the check)");
}
assert_eq!(listing(&shoot), before, "nothing was written next to the photos");
let _ = std::fs::remove_dir_all(&root);
}
/// Note 47: a built preview at least as big as the loupe's size is what the loupe shows (fitted
/// down, no render); a smaller one is not (the photo renders).
#[test]
fn the_loupe_uses_a_built_preview_that_is_big_enough() {
let mut s = Session::with_demo();
let id = s.active().unwrap();
s.execute("library.buildPreviews", &json!({"ids": [id.0], "edge": 512, "wait": true})).unwrap();
let r = s.loupe_job(id, 300, 300, true).unwrap().run();
assert_eq!(r.quick, Some(QuickSource::Cached), "taken from the built preview");
let img = r.rendered.unwrap().image;
assert_eq!(img.width.max(img.height), 300, "fitted to the loupe's size");
// bigger than what was built: rendered
let r = s.loupe_job(id, 900, 900, true).unwrap().run();
assert_eq!(r.quick, None, "a preview too small for the loupe is not used");
assert!(r.rendered.is_ok());
// a draft (a slider held) never comes from the cache
let r = s.loupe_job(id, 300, 300, true).unwrap().draft().run();
assert_eq!(r.quick, None);
}
/// Shrinking a cached view keeps its colour (an average of the pixels it covers) and its aspect,
/// and refuses to enlarge.
#[test]
fn a_cached_view_is_fitted_down_never_up() {
let img = lightcraft_raster::Rgba8::from_fn(400, 200, |x, _| if x % 2 == 0 { [200, 100, 0, 255] } else { [100, 50, 0, 255] });
let f = crate::media::fit_cached(&img, 100, 100).unwrap();
assert_eq!((f.width, f.height), (100, 50));
assert!(f.data.iter().all(|p| p[0] == 150 && p[1] == 75), "averaged");
assert!(crate::media::fit_cached(&img, 800, 800).is_none(), "never enlarged");
assert_eq!(crate::media::fit_cached(&img, 401, 999).map(|f| f.width), Some(400), "a pixel short is the same size");
assert!(crate::media::fit_cached(&img, 0, 10).is_none());
}
/// Note 47: the rendered previews in memory follow the memory budget (they were capped at
/// 128 MB, about 84 loupe renders, whatever the budget).
#[test]
fn the_memory_for_rendered_previews_grows_with_the_budget() {
let share = crate::memory::cache_share(20 << 30);
assert!(crate::media::rendered_budget(share) >= 2 << 30, "a 20 GB budget keeps gigabytes of previews");
assert_eq!(crate::media::rendered_budget(256 << 20), 128 << 20, "a small budget keeps the old floor where it can");
assert!(crate::media::rendered_budget(64 << 20) <= 32 << 20, "and never more than the share allows");
}
/// Note 47: the disk cache's default room grows with the library, so a wedding's previews are not
/// pruned while they are built (the fixed 2 GB held about 2 000 standard previews).
#[test]
fn the_disk_cache_default_grows_with_the_library() {
assert_eq!(crate::media::default_disk_cache_bytes(10), crate::media::DISK_CACHE_BYTES);
assert!(crate::media::default_disk_cache_bytes(4000) >= 4000 * (1 << 20));
let (mut s, root, ids) = library("diskroom", 2);
assert_eq!(s.cache_bytes(), crate::media::DISK_CACHE_BYTES);
s.cache_mb = 700;
assert_eq!(s.cache_bytes(), 700 << 20, "a size the user chose is kept");
assert_eq!(ids.len(), 2);
let _ = std::fs::remove_dir_all(root);
}
/// Notes 47 and 49: a build skips photos whose previews are there (built before, or by the loupe),
/// renders those whose preview is too small, and its previews are on disk when it is done (they
/// were left to a writer that skips writes when it falls behind).
#[test]
fn a_build_skips_what_is_there_and_leaves_it_on_disk() {
let (mut s, root, ids) = library("skip", 4);
let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "wait": true})).unwrap();
assert_eq!((r["done"].as_u64(), r["skipped"].as_u64(), r["failed"].as_u64()), (Some(4), Some(0), Some(0)), "{r}");
// on disk now (no waiting for a writer), at the size built
for id in &ids {
let p = s.catalog.photo(*id).unwrap().clone();
let key = s.view_key(&p, true);
let disk = s.media.rendered.disk().expect("a library has a disk cache");
let (w, h) = disk.dims(key).expect("the view preview is on disk");
assert_eq!(w.max(h), 300);
}
// again: nothing is rendered
let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "wait": true})).unwrap();
assert_eq!((r["done"].as_u64(), r["skipped"].as_u64()), (Some(4), Some(4)), "{r}");
// forced, or bigger: rendered
let r = s.execute("library.buildPreviews", &json!({"ids": [ids[0].0], "edge": 300, "wait": true, "force": true})).unwrap();
assert_eq!(r["skipped"], 0, "{r}");
let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 500, "wait": true})).unwrap();
assert_eq!(r["skipped"], 0, "too small for what is asked: built again {r}");
// a library opened again finds them on disk (memory is empty)
drop(s);
let mut s = Session::new().with_fs();
s.open_library(root.join("lib"), false).unwrap();
let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 500, "wait": true})).unwrap();
assert_eq!(r["skipped"], 4, "{r}");
let r = s.loupe_job(ids[1], 400, 400, true).unwrap().run();
assert_eq!(r.quick, Some(QuickSource::Cached), "the loupe uses the preview built in the last session");
let _ = std::fs::remove_dir_all(root);
}
/// A forced build renders the grid thumbnail again too, not the one its cache holds (CodeRabbit on #3577:
/// the thumbnail job read its cache first, so `force` rebuilt only the view).
#[test]
fn a_forced_build_renders_the_thumbnail_again() {
let (mut s, root, ids) = library("force-thumb", 1);
let build = |s: &mut Session, force: bool| {
s.execute("library.buildPreviews", &json!({"ids": [ids[0].0], "edge": 300, "wait": true, "force": force})).unwrap()
};
build(&mut s, false);
let jobs = crate::cmd::previews::jobs_for(&mut s, ids[0], Some(300));
let thumb = jobs.iter().find(|j| j.view_cache.is_none() && j.cache.is_some()).expect("a thumbnail job");
let ((cache, key), generation) = (thumb.cache.clone().unwrap(), thumb.cache_generation);
let stand_in = lightcraft_raster::Rgba8 { width: 1, height: 1, data: vec![[1, 2, 3, 255]] };
assert!(cache.put_at(generation, key, std::sync::Arc::new(stand_in)));
let r = build(&mut s, false);
assert_eq!(r["skipped"], 1, "{r}");
assert_eq!(cache.get_at(generation, key).map(|i| i.width), Some(1), "unforced: kept");
let r = build(&mut s, true);
assert_eq!(r["failed"], 0, "{r}");
assert!(cache.get_at(generation, key).is_some_and(|i| i.width > 1), "forced: rendered again");
let _ = std::fs::remove_dir_all(root);
}
/// Note 47: standard previews are built at least as big as the loupe on this screen asks for.
#[test]
fn standard_previews_are_as_big_as_the_loupe() {
let mut s = Session::with_demo();
assert_eq!(crate::cmd::previews::standard_edge(&s), crate::cmd::previews::STANDARD_EDGE);
s.loupe_edge = 3000;
assert_eq!(crate::cmd::previews::standard_edge(&s), 3000);
s.loupe_edge = 1200;
assert_eq!(crate::cmd::previews::standard_edge(&s), crate::cmd::previews::STANDARD_EDGE);
}
/// Note 48: a background build goes from where the user is, in the direction they step, and
/// follows them when they jump.
#[test]
fn the_background_build_starts_where_the_user_is() {
use crate::cmd::previews::PreviewBuild;
let b = PreviewBuild { order: (10..20).collect(), ..Default::default() };
let mut queue: Vec<Option<u64>> = (10..20).map(Some).collect();
b.focus_on(PhotoId(14), false);
let mut took = Vec::new();
for _ in 0..3 {
let i = b.next_index(&queue).unwrap();
took.push(queue[i].take().unwrap());
}
assert_eq!(took, vec![14, 15, 16], "forward from the photo on screen");
b.focus_on(PhotoId(12), true);
let i = b.next_index(&queue).unwrap();
assert_eq!(queue[i], Some(12), "jumped back: from there, backwards");
queue[i] = None;
assert_eq!(b.next_index(&queue).map(|i| queue[i]), Some(Some(11)));
// the end reached: the rest the other way
b.focus_on(PhotoId(19), false);
for _ in 0..6 {
let i = b.next_index(&queue).unwrap();
queue[i] = None;
}
assert!(b.next_index(&queue).is_none(), "all taken");
}
/// Note 49: the background preparation gives way to a build that is asked for (no error, no
/// two builds), and is not started again for a view the user stopped it in.
#[test]
fn a_build_asked_for_replaces_the_background_one() {
let (mut s, root, ids) = library("replace", 3);
// held back: the photo on screen is being made all along (held until released below, not for a
// deadline a slow test machine could pass before the build asked for comes)
let hold = crate::prep::TestHold::new();
s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "background": true})).unwrap();
let bg = s.preview_build.clone().unwrap();
assert!(bg.background && bg.running());
assert!(s.activity.list().iter().any(|t| t.label == "Preparing photos in the background"), "{:?}", s.activity.list());
// a second background build is refused; one that is asked for replaces it
assert!(s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "background": true})).is_err());
let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "wait": true})).unwrap();
assert_eq!(r["done"], 3, "{r}");
assert!(bg.cancel.load(Ordering::Relaxed), "the background build was stopped");
drop(hold);
// its workers stop before their library is removed under them
assert!(wait_finished(&bg), "a cancelled build stops");
drop(s);
std::fs::remove_dir_all(&root).unwrap();
}
/// Note 48: the background preparation starts once the user steps through photos in the loupe
/// (not when a library opens), builds the view's previews in one cancellable row, and a stop
/// from that row holds for the view.
#[test]
fn the_background_preparation_starts_when_culling_and_can_be_stopped() {
let (mut s, root, ids) = library("prep", 4);
let mode = crate::prep::Mode::Library;
s.prep_tick(mode, None, false);
s.prep_tick(mode, Some(ids[0]), false);
assert!(s.preview_build.is_none(), "opening and looking at one photo starts nothing");
s.prep_tick(mode, Some(ids[1]), false);
s.prep_tick(mode, Some(ids[2]), false);
let b = s.preview_build.clone().expect("culling: the view is prepared");
assert!(b.background);
assert_eq!(b.total, 4);
for _ in 0..600 {
s.prep_tick(mode, Some(ids[2]), false);
if !b.running() {
break;
}
std::thread::sleep(std::time::Duration::from_millis(20));
}
assert!(!b.running(), "built");
assert_eq!(b.done.load(Ordering::Relaxed), 4);
s.prep_tick(mode, Some(ids[3]), false);
assert!(same_build(&s, &b), "not started again for the same view");
// switched off: nothing starts in a new view either
s.execute("prefs.setBackgroundPrep", &json!({"settings": {"library": {"previews": false}}})).unwrap();
let off = s.execute("prefs.backgroundPrep", &json!({})).unwrap();
assert_eq!(off["settings"]["library"]["previews"], false, "{off}");
assert_eq!(off["settings"]["culling"]["previews"], true, "each mode has its own");
let _ = std::fs::remove_dir_all(root);
}
/// While editing, the whole view's build follows the Library row (its switch and "which photos");
/// the Editing row's scope is only for the photos kept ready ahead.
#[test]
fn editing_builds_the_view_by_the_library_rows_scope() {
let (mut s, root, ids) = library("prep-develop", 4);
s.execute("prefs.setBackgroundPrep", &json!({"settings": {"develop": {"scope": "onScreen"}, "library": {"scope": "marked"}}})).unwrap();
s.execute("photo.rate", &json!({"ids": [ids[1].0], "rating": 3})).unwrap();
let mode = crate::prep::Mode::Develop;
for id in &ids {
s.prep_tick(mode, Some(*id), false);
}
let b = s.preview_build.clone().expect("the Library row asks for the view while editing");
assert_eq!(b.total, 1, "only the marked photos, as the Library row says");
b.cancel.store(true, Ordering::Relaxed);
// its workers stop before their library is removed under them
assert!(wait_finished(&b), "a cancelled build stops");
drop(s);
std::fs::remove_dir_all(&root).unwrap();
}
/// With "which photos" = marked, marking another photo changes what the view's build prepares: a
/// build for the newly marked set follows (it wasn't, as the view itself stayed the same).
#[test]
fn marking_a_photo_prepares_it_in_the_marked_scope() {
let (mut s, root, ids) = library("prep-marked", 4);
s.execute("prefs.setBackgroundPrep", &json!({"settings": {"library": {"scope": "marked"}}})).unwrap();
s.execute("photo.rate", &json!({"ids": [ids[1].0], "rating": 3})).unwrap();
let mode = crate::prep::Mode::Library;
let tick = |s: &mut Session, k: usize| s.prep_tick(mode, Some(ids[k % 3]), false);
for k in 0..3 {
tick(&mut s, k);
}
let first = s.preview_build.clone().expect("the marked photos are prepared");
assert_eq!(first.total, 1);
s.execute("photo.rate", &json!({"ids": [ids[3].0], "rating": 2})).unwrap();
let mut next = None;
for k in 0..600 {
tick(&mut s, k);
if let Some(b) = s.preview_build.clone().filter(|b| !std::sync::Arc::ptr_eq(b, &first)) {
next = Some(b);
break;
}
std::thread::sleep(std::time::Duration::from_millis(20));
}
let next = next.expect("a build for the newly marked set");
assert_eq!(next.total, 2);
next.cancel.store(true, Ordering::Relaxed);
// both builds' workers stop before their library is removed under them
for b in [&first, &next] {
assert!(wait_finished(b), "a cancelled build stops");
}
drop(s);
std::fs::remove_dir_all(&root).unwrap();
}
/// Waits (up to 30 s) for a build's workers to finish.
fn wait_finished(b: &crate::cmd::previews::PreviewBuild) -> bool {
for _ in 0..1500 {
if b.finished.load(Ordering::Relaxed) {
return true;
}
std::thread::sleep(std::time::Duration::from_millis(20));
}
false
}
/// A background build started for one view is stopped when another view is shown (here the
/// library changes under it), not left building photos no longer shown; it doesn't count as
/// stopped by the user, so the view now shown is prepared.
#[test]
fn a_view_change_stops_the_background_build_for_the_old_view() {
let (mut s, root, ids) = library("prep-view", 6);
let mode = crate::prep::Mode::Library;
// the build's lanes wait at their foreground check until released below
let hold = crate::prep::TestHold::new();
for id in &ids[..3] {
s.prep_tick(mode, Some(*id), false);
}
let b = s.preview_build.clone().expect("the view is prepared");
assert!(b.running());
s.catalog.apply(lightcraft_catalog::Op::RemovePhoto { id: ids[5] }).unwrap();
s.prep_tick(mode, Some(ids[2]), false);
let stopped = b.cancel.load(Ordering::Relaxed);
drop(hold);
assert!(stopped, "the old view's build is stopped");
// the user steps on in the view now shown
for k in 0..600 {
s.prep_tick(mode, Some(ids[k % 3]), false);
if s.preview_build.as_ref().is_some_and(|p| !std::sync::Arc::ptr_eq(p, &b)) {
break;
}
std::thread::sleep(std::time::Duration::from_millis(20));
}
let next = s.preview_build.clone().expect("a build for the view now shown");
assert!(!std::sync::Arc::ptr_eq(&next, &b));
assert_eq!(next.total, 5);
next.cancel.store(true, Ordering::Relaxed);
// both builds' workers stop before their library is removed under them
for b in [&b, &next] {
assert!(wait_finished(b), "a cancelled build stops");
}
drop(s);
std::fs::remove_dir_all(&root).unwrap();
}
fn same_build(s: &Session, b: &std::sync::Arc<crate::cmd::previews::PreviewBuild>) -> bool {
s.preview_build.as_ref().is_some_and(|p| std::sync::Arc::ptr_eq(p, b))
}
/// Note 48: a library keeps its own background settings (a timelapse library prepares nothing),
/// saved with it; the app-wide ones apply elsewhere.
#[test]
fn a_library_can_keep_its_own_background_settings() {
let (mut s, root, _ids) = library("own", 1);
let r = s.execute("prefs.setBackgroundPrep", &json!({"library": true, "settings": {"culling": {"previews": false, "ahead": 0}}})).unwrap();
assert_eq!(r["own"], true, "{r}");
assert_eq!(r["settings"]["culling"]["ahead"], 0);
assert_eq!(r["app"]["culling"]["ahead"], 5, "the app-wide ones are unchanged");
drop(s);
let mut s = Session::new().with_fs();
s.open_library(root.join("lib"), false).unwrap();
assert_eq!(s.prep_settings().culling.ahead, 0, "kept with the library");
assert_eq!(s.prep_settings().ahead(crate::prep::Mode::Culling), 0);
s.execute("prefs.setBackgroundPrep", &json!({"reset": true})).unwrap();
assert_eq!(s.prep_settings().culling.ahead, 5, "back to the app-wide settings");
assert!(s.execute("prefs.setBackgroundPrep", &json!({"settings": 3})).is_err());
let _ = std::fs::remove_dir_all(root);
}
/// Note 47: Build Previews with one photo open (selected) builds every photo in view, as its menu
/// item says ("Visible"); it built only the open photo, so culling after it found nothing built.
#[test]
fn build_previews_with_one_photo_open_builds_the_view() {
let mut s = Session::with_demo();
let n = s.visible_cloned().len();
assert!(n > 2);
let one = s.visible_cloned()[0];
s.selection = crate::Selection::single(one);
let r = s.execute("library.buildPreviews", &json!({"edge": 256, "wait": true})).unwrap();
assert_eq!(r["total"], n, "{r}");
let two: Vec<PhotoId> = s.visible_cloned().into_iter().take(2).collect();
s.selection = crate::Selection { ids: two.clone(), active: two.first().copied() };
let r = s.execute("library.buildPreviews", &json!({"edge": 256, "wait": true})).unwrap();
assert_eq!(r["total"], 2, "a choice of photos is built alone {r}");
}
/// A view the loupe kept only in memory (its deferred disk write skipped when the writer fell behind) is not
/// counted as built: the build renders it and writes it to disk (CodeRabbit on #3077).
#[test]
fn a_preview_kept_only_in_memory_is_built_and_written() {
let (mut s, root, ids) = library("memonly", 1);
let jobs = crate::cmd::previews::jobs_for(&mut s, ids[0], Some(300));
assert!(!jobs.is_empty());
for j in &jobs {
let img = std::sync::Arc::new(lightcraft_raster::Rgba8::new(300, 200));
if let Some((cache, key)) = &j.view_cache {
cache.put_memory(*key, img);
} else if let Some((cache, key)) = &j.cache {
cache.put_memory(*key, img);
}
}
assert!(!crate::cmd::previews::already_built(&jobs), "a copy only in memory is not a built preview");
let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "wait": true})).unwrap();
assert_eq!((r["skipped"].as_u64(), r["failed"].as_u64()), (Some(0), Some(0)), "{r}");
let p = s.catalog.photo(ids[0]).unwrap().clone();
let key = s.view_key(&p, true);
assert!(s.media.rendered.disk().and_then(|d| d.dims(key)).is_some(), "the view preview is on disk");
let _ = std::fs::remove_dir_all(root);
}
/// A view preview the cache folder can't take (full, or not writable) fails the photo instead of counting it
/// as built (CodeRabbit on #3077).
#[test]
fn a_preview_that_cannot_be_written_fails_the_build() {
let (mut s, root, ids) = library("unwritable", 2);
let dir = s.media.rendered.disk().expect("a library has a disk cache").dir().to_path_buf();
let _ = std::fs::remove_dir_all(&dir);
// a file where the cache folder goes: nothing can be written under it
std::fs::write(&dir, b"not a folder").unwrap();
let r = s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "wait": true})).unwrap();
assert_eq!(r["failed"].as_u64(), Some(2), "{r}");
let _ = std::fs::remove_file(&dir);
let _ = std::fs::remove_dir_all(root);
}
/// A photo whose preview cache is discarded while it renders is not counted as built: its preview isn't kept
/// (CodeRabbit on #3077).
#[test]
fn a_preview_rendered_for_a_discarded_cache_fails_the_photo() {
let (mut s, root, ids) = library("discarded", 1);
let jobs = crate::cmd::previews::jobs_for(&mut s, ids[0], Some(300));
s.execute("library.clearPreviews", &json!({})).unwrap();
assert!(!crate::cmd::previews::run_photo(jobs), "nothing was kept: the photo is not built");
let jobs = crate::cmd::previews::jobs_for(&mut s, ids[0], Some(300));
assert!(crate::cmd::previews::run_photo(jobs), "built for the cache as it is now");
let _ = std::fs::remove_dir_all(root);
}
/// A forced build renders a photo itself even when another build was rendering it and made its previews
/// (CodeRabbit on #3080): the other build's result is not taken as this one's.
#[test]
fn a_forced_build_renders_a_photo_another_build_was_rendering() {
let (mut s, root, ids) = library("forced-shared", 1);
s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "wait": true})).unwrap();
let jobs = crate::cmd::previews::jobs_for(&mut s, ids[0], Some(300));
let held = crate::cmd::previews::hold_building(ids[0].0, &jobs).expect("nobody renders it now");
s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300, "force": true})).unwrap();
let b = s.preview_build.clone().expect("the build");
std::thread::sleep(std::time::Duration::from_millis(300));
assert_eq!(b.done.load(Ordering::Relaxed), 0, "not counted while the other build holds it");
drop(held);
assert!(wait_finished(&b));
let counts =
(b.done.load(Ordering::Relaxed), b.failed.load(Ordering::Relaxed), b.shared.load(Ordering::Relaxed), b.skipped.load(Ordering::Relaxed));
assert_eq!(counts, (1, 0, 0, 0), "rendered here, not taken from the other build");
let _ = std::fs::remove_dir_all(root);
}
/// A photo another build is rendering is not counted until that render is over: here the other build never
/// made its previews, so this build renders it (CodeRabbit on #3080).
#[test]
fn a_photo_another_build_is_rendering_is_counted_only_once_it_is_built() {
let (mut s, root, ids) = library("shared", 1);
let jobs = crate::cmd::previews::jobs_for(&mut s, ids[0], Some(300));
let held = crate::cmd::previews::hold_building(ids[0].0, &jobs).expect("nobody renders it yet");
s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "edge": 300})).unwrap();
let b = s.preview_build.clone().expect("the build");
std::thread::sleep(std::time::Duration::from_millis(300));
assert_eq!(b.done.load(Ordering::Relaxed), 0, "not counted while the other build holds it");
drop(held);
assert!(wait_finished(&b));
assert_eq!((b.done.load(Ordering::Relaxed), b.failed.load(Ordering::Relaxed), b.shared.load(Ordering::Relaxed)), (1, 0, 0));
let p = s.catalog.photo(ids[0]).unwrap().clone();
let key = s.view_key(&p, true);
assert!(s.media.rendered.disk().and_then(|d| d.dims(key)).is_some(), "rendered here, on disk");
let _ = std::fs::remove_dir_all(root);
}
/// CodeRabbit on #3544: a library preparation setting that can't be saved is not kept, nor is
/// going back to the app-wide settings: the session stays as the command's error says.
#[test]
fn a_prep_setting_that_cant_be_saved_is_not_kept() {
use crate::prep::PrepSettings;
let (mut s, root, _) = library("prep-unsaved", 1);
let mut own = PrepSettings::default();
own.culling.ahead = 3;
s.set_prep_settings(own, true).unwrap();
// prefs.json can't be written: a folder stands in its place
let prefs = root.join("lib").join("prefs.json");
let _ = std::fs::remove_file(&prefs);
std::fs::create_dir_all(prefs.join("in-the-way")).unwrap();
let mut other = own;
other.culling.ahead = 5;
assert!(s.set_prep_settings(other, true).is_err());
assert_eq!(s.prep_json()["library"], json!(own), "the setting that couldn't be saved is not kept");
assert!(s.clear_library_prep().is_err());
assert_eq!(s.prep_json()["library"], json!(own), "nor going back to the app-wide settings");
let _ = std::fs::remove_dir_all(&root);
}
/// CodeRabbit on #3544: a build asked for with malformed ids fails without stopping the
/// background preparation that is running.
#[test]
fn a_bad_build_request_leaves_the_background_build_running() {
let _hold = crate::prep::TestHold::new();
let (mut s, root, ids) = library("prep-bad-ids", 3);
s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "background": true})).unwrap();
let b = s.preview_build.clone().expect("the background build");
assert!(b.running());
assert!(s.execute("library.buildPreviews", &json!({"ids": ["x"]})).is_err(), "ids that are not photo ids");
assert!(!b.cancel.load(std::sync::atomic::Ordering::Relaxed), "the background build was not stopped");
let _ = std::fs::remove_dir_all(&root);
}
/// CodeRabbit on #3577: smart previews asked for in the background with malformed ids fail without
/// stopping the background preparation either.
#[test]
fn a_bad_smart_preview_request_leaves_the_background_build_running() {
let _hold = crate::prep::TestHold::new();
let (mut s, root, ids) = library("prep-bad-smart-ids", 3);
s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "background": true})).unwrap();
let b = s.preview_build.clone().expect("the background build");
assert!(b.running());
assert!(s.execute("library.smartPreviews", &json!({"ids": ["x"], "background": true})).is_err(), "ids that are not photo ids");
assert!(!b.cancel.load(std::sync::atomic::Ordering::Relaxed), "the background build was not stopped");
// a smart-preview worker that can't start: the background build carries on, and stays the session's
crate::cmd::previews::test_gate::fail_next_spawn();
assert!(s.execute("library.smartPreviews", &json!({"ids": ids_json(&ids), "background": true})).is_err(), "the worker didn't start");
assert!(!b.cancel.load(std::sync::atomic::Ordering::Relaxed), "the background build was not stopped");
assert!(s.preview_build.as_ref().is_some_and(|now| std::sync::Arc::ptr_eq(now, &b)), "and is still the session's build");
// (stopped and finished before its folder goes)
b.cancel.store(true, std::sync::atomic::Ordering::Relaxed);
drop(_hold);
let until = std::time::Instant::now() + std::time::Duration::from_secs(20);
while b.running() && std::time::Instant::now() < until {
std::thread::sleep(std::time::Duration::from_millis(5));
}
let _ = std::fs::remove_dir_all(&root);
}
/// CodeRabbit on #3544: a build asked for whose worker can't start fails without stopping or
/// replacing the background preparation that is running.
#[test]
fn a_build_that_cant_start_leaves_the_background_build_running() {
let _hold = crate::prep::TestHold::new();
let (mut s, root, ids) = library("prep-no-thread", 3);
s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids), "background": true})).unwrap();
let b = s.preview_build.clone().expect("the background build");
assert!(b.running());
crate::cmd::previews::test_gate::fail_next_spawn();
assert!(s.execute("library.buildPreviews", &json!({"ids": ids_json(&ids)})).is_err(), "the worker didn't start");
assert!(!b.cancel.load(std::sync::atomic::Ordering::Relaxed), "the background build was not stopped");
assert!(s.preview_build.as_ref().is_some_and(|now| std::sync::Arc::ptr_eq(now, &b)), "and is still the session's build");
let _ = std::fs::remove_dir_all(&root);
}
/// CodeRabbit on #3080: a background build that can't start isn't tried again on every tick; the
/// view tries again once [`crate::prep::RETRY_AFTER`] has passed.
#[test]
fn a_build_that_cant_start_waits_before_trying_again() {
let (mut s, root, ids) = library("prep-retry", 3);
let mode = crate::prep::Mode::Library;
s.prep.fail_starts = 1;
for id in &ids {
s.prep_tick(mode, Some(*id), false);
}
assert!(s.preview_build.is_none(), "the start failed");
assert!(s.prep.retry_at.is_some());
// the app is told when to look again (a frame is asked for then)
assert!(s.prep_retry_in().is_some_and(|d| d <= crate::prep::RETRY_AFTER && d > crate::prep::RETRY_AFTER / 2));
s.prep_tick(mode, Some(ids[2]), false);
assert!(s.preview_build.is_none(), "not tried again at once");
// the wait is over: no more frames asked for it (CodeRabbit on #3077 / #3080)
s.prep.retry_at = s.prep.retry_at.map(|(view, _)| (view, web_time::Instant::now()));
assert_eq!(s.prep_retry_in(), None, "a deadline that has passed asks for no frame");
s.prep_tick(mode, Some(ids[2]), false);
let b = s.preview_build.clone().expect("tried again");
assert!(b.background && s.prep.retry_at.is_none());
b.cancel.store(true, Ordering::Relaxed);
let _ = wait_finished(&b);
let _ = std::fs::remove_dir_all(&root);
}
crates/preview/src/disk.rs
+58 -5
View File
@@ -172,15 +172,43 @@ impl DiskCache {
}
}
pub fn put(&self, key: Hash128, img: &Rgba8) {
let Some(bytes) = encode_jpeg(img) else { return };
/// The size (width, height) of the picture kept under `key`, read from the start of its file
/// only (no decode, no hit counted, nothing written): to tell whether a preview is there and
/// big enough without loading it.
pub fn dims(&self, key: Hash128) -> Option<(usize, usize)> {
use std::io::Read;
let p = self.path(key);
let Some(parent) = p.parent() else { return };
if !p.parent().is_some_and(is_real_dir) {
return None;
}
let mut f = std::fs::File::open(&p).or_else(|_| std::fs::File::open(self.named(key, OLD_EXT))).ok()?;
// the size is normally in the first few KiB: more is read only when that is not enough
let mut head = Vec::new();
for limit in [FIRST_HEADER_BYTES, HEADER_BYTES] {
let more = limit.saturating_sub(head.len() as u64);
(&mut f).take(more).read_to_end(&mut head).ok()?;
if let Some(dims) = jpeg_dims(&head) {
return Some(dims);
}
if (head.len() as u64) < limit {
// the whole file was read
break;
}
}
None
}
/// Write `img` under `key`. Returns whether the file was written (a full or unwritable cache folder
/// gives `false`; the cache is unchanged then).
pub fn put(&self, key: Hash128, img: &Rgba8) -> bool {
let Some(bytes) = encode_jpeg(img) else { return false };
let p = self.path(key);
let Some(parent) = p.parent() else { return false };
// a folder the cache makes for itself says that it is a cache; one that was there
// already may hold somebody's files, and is not declared one
let made = !self.dir.exists();
if std::fs::create_dir_all(parent).is_err() || !is_real_dir(parent) {
return;
return false;
}
if made {
let _ = std::fs::write(self.dir.join(TAG), TAG_TEXT);
@@ -197,7 +225,7 @@ impl DiskCache {
let written = std::fs::OpenOptions::new().write(true).create_new(true).open(&tmp).and_then(|mut f| std::io::Write::write_all(&mut f, &bytes));
if written.is_err() || std::fs::rename(&tmp, &p).is_err() {
let _ = std::fs::remove_file(&tmp);
return;
return false;
}
self.writes.fetch_add(1, Ordering::Relaxed);
*used = used.saturating_sub(old_len).saturating_add(bytes.len() as u64);
@@ -205,7 +233,10 @@ impl DiskCache {
drop(total);
if over {
self.prune();
// (a picture bigger than the room pruning leaves goes too: it was not kept)
return p.is_file();
}
true
}
/// All cache files (only names of the cache's own shape, see [`is_cache_file`]):
@@ -374,6 +405,12 @@ impl DiskCache {
*t.get_or_insert_with(|| self.scan().iter().map(|f| f.1).sum())
}
/// Bytes on disk when already known, without reading the folder (`None` until [`Self::size`],
/// a write or a prune has counted them).
pub fn known_size(&self) -> Option<u64> {
*self.total.lock().unwrap_or_else(|e| e.into_inner())
}
pub fn clear(&self) {
for (p, _, _) in self.scan() {
let _ = std::fs::remove_file(p);
@@ -432,6 +469,22 @@ pub fn encode_jpeg(img: &Rgba8) -> Option<Vec<u8>> {
Some(out)
}
/// How much of a cached file [`DiskCache::dims`] reads: the JPEG headers this cache writes come
/// first and are a few hundred bytes.
const HEADER_BYTES: u64 = 64 << 10;
/// What [`DiskCache::dims`] reads first, which holds the size of the JPEGs this cache writes.
const FIRST_HEADER_BYTES: u64 = 4 << 10;
/// A JPEG's (width, height) from its headers (`None`: not a JPEG, or cut short before its frame header).
pub fn jpeg_dims(bytes: &[u8]) -> Option<(usize, usize)> {
use zune_core::bytestream::ZCursor;
let mut d = zune_jpeg::JpegDecoder::new(ZCursor::new(bytes));
d.decode_headers().ok()?;
let info = d.info()?;
let (w, h) = (info.width as usize, info.height as usize);
(w > 0 && h > 0).then_some((w, h))
}
/// Decode a cached thumbnail (see [`encode_jpeg`]).
pub fn decode_jpeg(bytes: &[u8]) -> Option<Rgba8> {
use zune_core::bytestream::ZCursor;
crates/preview/src/lib.rs
+67 -7
View File
@@ -21,7 +21,7 @@ use std::path::Path;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::{Arc, Mutex, RwLock};
pub use disk::{DiskCache, decode_jpeg, encode_jpeg};
pub use disk::{DiskCache, decode_jpeg, encode_jpeg, jpeg_dims};
pub use hash::{Hash128, Hasher128, hash_bytes};
use lightcraft_raster::Rgba8;
pub use lru::{Lru, next_tick};
@@ -108,20 +108,80 @@ impl PreviewCache {
Some(img)
}
pub fn put(&self, key: Hash128, img: Arc<Rgba8>) {
self.put_at(self.generation(), key, img);
/// The size of the image kept under `key` (memory, else disk: read from its file's header), without
/// loading it or changing what is kept.
pub fn dims(&self, key: Hash128) -> Option<(usize, usize)> {
if self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner).is_none() {
return None;
}
if let Some(v) = self.mem.lock().unwrap_or_else(|e| e.into_inner()).peek(&key) {
return Some((v.width, v.height));
}
self.disk.as_ref()?.dims(key)
}
pub fn put_at(&self, generation: u64, key: Hash128, img: Arc<Rgba8>) {
/// The size of the image kept under `key` where it lasts: on disk when the cache has a disk (a copy only
/// in memory, whose deferred write may have been skipped, doesn't count), else in memory. Nothing is
/// loaded or changed.
pub fn stored_dims(&self, key: Hash128) -> Option<(usize, usize)> {
match &self.disk {
Some(d) if self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner).is_some() => d.dims(key),
Some(_) => None,
None => self.dims(key),
}
}
/// The memory LRU's copy of `key`, without looking on disk.
pub fn get_memory(&self, key: Hash128) -> Option<Arc<Rgba8>> {
if self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner).is_none() {
return None;
}
self.mem.lock().unwrap_or_else(|e| e.into_inner()).get(&key).cloned()
}
/// Keep `img` in memory only (it is never written to disk): for pictures that are cheap
/// enough to make again, such as a raw's decoded camera JPEG, but worth keeping for a revisit.
pub fn put_memory(&self, key: Hash128, img: Arc<Rgba8>) {
// (the guard is held through the insertion: a clear can't come between the look and the insertion)
let current = self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner);
if current.is_none() {
return;
}
let c = cost(&img);
self.mem.lock().unwrap_or_else(|e| e.into_inner()).insert(key, img, c);
drop(current);
}
/// [`Self::put_memory`] for a job that captured `generation`: nothing is kept once the cache
/// was cleared (or retired) after that, so a job started before Discard Preview Cache can't
/// put its picture back.
pub fn put_memory_at(&self, generation: u64, key: Hash128, img: Arc<Rgba8>) {
// (the guard is held through the insertion: a clear can't come between the look and the insertion)
let current = self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner);
if *current != Some(generation) {
return;
}
if let Some(d) = &self.disk {
d.put(key, &img);
}
let c = cost(&img);
self.mem.lock().unwrap_or_else(|e| e.into_inner()).insert(key, img, c);
drop(current);
}
pub fn put(&self, key: Hash128, img: Arc<Rgba8>) {
let _ = self.put_at(self.generation(), key, img);
}
/// [`Self::put`] for a job of `generation`. Returns whether the image is kept where it lasts: `false` when
/// the cache was cleared or retired since the job was made (nothing is kept; see [`Self::generation`] to
/// tell), or when the disk cache could not write the file (the image is kept in memory all the same).
pub fn put_at(&self, generation: u64, key: Hash128, img: Arc<Rgba8>) -> bool {
let current = self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner);
if *current != Some(generation) {
return false;
}
let written = self.disk.as_ref().is_none_or(|d| d.put(key, &img));
let c = cost(&img);
self.mem.lock().unwrap_or_else(|e| e.into_inner()).insert(key, img, c);
written
}
/// Like [`Self::put`], but the disk write (a JPEG encode) happens on a background thread, so
crates/preview/src/tests.rs
+60
View File
@@ -792,3 +792,63 @@ fn deferred_writes_are_bounded_and_drain_to_disk() {
assert!(keys.iter().all(|k| cache.get(*k).is_some()));
let _ = std::fs::remove_dir_all(&dir);
}
/// `put_at` says whether the image is kept where it lasts: not for a job made before the cache was cleared or
/// retired, nor when the disk cache can't write it (CodeRabbit on #3077).
#[test]
fn put_at_says_whether_the_image_was_kept() {
let dir = temp_dir("put-at-kept");
let cache = PreviewCache::with_disk(1 << 20, &dir, 1 << 20);
let (key, img) = (hash_bytes(b"kept"), Arc::new(gradient(8, 8, 3)));
let old = cache.generation();
assert!(cache.put_at(old, key, img.clone()), "kept");
cache.clear();
assert!(!cache.put_at(old, key, img.clone()), "a job from before the clear keeps nothing");
assert!(cache.put_at(cache.generation(), key, img.clone()));
let generation = cache.generation();
cache.retire();
assert!(!cache.put_at(generation, key, img.clone()), "nor one for a retired cache");
// a file where the cache folder goes: the write fails
let blocked = temp_dir("put-at-blocked");
std::fs::write(&blocked, b"not a folder").unwrap();
let cache = PreviewCache::with_disk(1 << 20, &blocked, 1 << 20);
assert!(!cache.put_at(cache.generation(), key, img.clone()), "not written");
let memory = PreviewCache::memory(1 << 20);
assert!(memory.put_at(memory.generation(), key, img), "a memory-only cache keeps it");
let _ = std::fs::remove_file(&blocked);
let _ = std::fs::remove_dir_all(&dir);
}
/// `dims` reads a cached file's first few KiB, and further when its size marker comes later (CodeRabbit on #3080).
#[test]
fn dims_finds_a_size_marker_past_the_first_few_kib() {
let dir = temp_dir("dims-far");
let d = DiskCache::new(&dir, 1 << 20);
let key = hash_bytes(b"far");
assert!(d.put(key, &gradient(40, 30, 5)));
assert_eq!(d.dims(key), Some((40, 30)));
// the same picture with a 10 KiB application segment before its frame header
let file = files_under(&dir).into_iter().find(|f| f.ends_with(disk::EXT)).expect("the cached file");
let path = dir.join(file);
let jpeg = std::fs::read(&path).unwrap();
let mut padded = jpeg[..2].to_vec();
let pad = 10 << 10;
padded.extend_from_slice(&[0xFF, 0xEF, ((pad + 2) >> 8) as u8, ((pad + 2) & 0xFF) as u8]);
padded.extend(std::iter::repeat_n(0u8, pad));
padded.extend_from_slice(&jpeg[2..]);
std::fs::write(&path, padded).unwrap();
assert_eq!(d.dims(key), Some((40, 30)), "found past the first read");
let _ = std::fs::remove_dir_all(&dir);
}
/// A picture bigger than the room pruning leaves is removed by the pruning its own write starts: `put` says it
/// was not kept (CodeRabbit on #3077).
#[test]
fn a_write_the_pruning_removes_is_not_reported_as_kept() {
let dir = temp_dir("pruned-away");
let d = DiskCache::new(&dir, 64);
assert!(!d.put(hash_bytes(b"too big"), &gradient(64, 64, 9)), "pruned away at once");
let roomy = DiskCache::new(&dir, 1 << 20);
assert!(roomy.put(hash_bytes(b"fits"), &gradient(64, 64, 9)));
let _ = std::fs::remove_dir_all(&dir);
}
crates/raw/src/preview.rs
+185 -39
View File
@@ -138,12 +138,68 @@ fn is_jxl(b: &[u8]) -> bool {
b.starts_with(&[0xff, 0x0a]) || b.starts_with(&[0, 0, 0, 0x0c, b'J', b'X', b'L', b' ', 0x0d, 0x0a, 0x87, 0x0a])
}
fn candidates<'a>(data: &'a [u8], ifd: &Ifd, base: u64, out: &mut Vec<&'a [u8]>) {
/// The bytes a file's own (offset, length) pair names, cut at the file's end; `None` when the offset
/// lies past it. Both numbers come from the file: never cast them (a `u64` offset past 4 GiB would
/// wrap to a small, wrong one on a 32-bit build) nor add them unchecked. A span the file's end cuts
/// short is kept only when it is a JPEG whose own end marker is inside it, after its scan (an
/// overstated length with the whole JPEG inside), cut at that marker: a truncated JPEG passes the
/// frame-header check, would win as the largest preview and then fail to decode (CodeRabbit on
/// #3077). An `FF D9` inside an APP payload doesn't count, bytes that don't start a JPEG are refused
/// without a scan, and the walks share `budget` (bytes, as [`scan_with_budget`]'s do), so a file of
/// many such pointers stays linear.
fn file_span<'a>(data: &'a [u8], offset: u64, len: u64, budget: &mut usize) -> Option<&'a [u8]> {
let start = usize::try_from(offset).ok()?;
let wanted = usize::try_from(offset.saturating_add(len)).unwrap_or(usize::MAX);
let span = data.get(start..wanted.min(data.len()))?;
if wanted > data.len() {
return span.get(..jpeg_end(span, budget)?);
}
Some(span)
}
/// Where the JPEG at the start of `s` ends (just past its end marker), walking its marker segments:
/// a segment's payload is skipped by its stated length (so an `FF D9` inside APP data isn't taken
/// for the end), entropy-coded bytes one at a time (stuffed `FF 00` and restart markers included).
/// `None` when `s` isn't a JPEG, ends first, reaches its end marker before a start of scan (no image
/// data), or the walk spends `budget` (one per step, shared by a file's walks). Each step moves
/// forward, so it ends within `s`.
fn jpeg_end(s: &[u8], budget: &mut usize) -> Option<usize> {
if !s.starts_with(&[0xff, 0xd8]) {
return None;
}
let (mut i, mut scanned) = (2usize, false);
loop {
*budget = budget.checked_sub(1)?;
match (s.get(i), s.get(i.checked_add(1)?)) {
(None, _) | (Some(&0xff), None) => return None,
(Some(&0xff), Some(&m)) => match m {
0xd9 => return if scanned { i.checked_add(2) } else { None },
// fill byte
0xff => i += 1,
// stuffed zero, restart marker, TEM: no length follows
0x00 | 0x01 | 0xd0..=0xd7 => i = i.checked_add(2)?,
_ => {
let (&hi, &lo) = (s.get(i.checked_add(2)?)?, s.get(i.checked_add(3)?)?);
let len = usize::from(u16::from_be_bytes([hi, lo]));
if len < 2 {
return None;
}
scanned |= m == 0xda;
i = i.checked_add(2)?.checked_add(len)?;
}
},
// entropy-coded data
(Some(_), _) => i += 1,
}
}
}
fn candidates<'a>(data: &'a [u8], ifd: &Ifd, base: u64, out: &mut Vec<&'a [u8]>, budget: &mut usize) {
// whole JPEG files stored as an undefined-type tag value (e.g. Panasonic `JpgFromRaw` 0x002e)
for e in &ifd.entries {
if matches!(e.value, lightcraft_tiff::Value::Undefined(_))
&& e.count() > 1024
&& let Some(s) = data.get(e.offset as usize..(e.offset.saturating_add(e.count() as u64) as usize).min(data.len()))
&& let Some(s) = file_span(data, e.offset, e.count() as u64, budget)
&& s.starts_with(&[0xff, 0xd8])
{
out.push(s);
@@ -151,7 +207,7 @@ fn candidates<'a>(data: &'a [u8], ifd: &Ifd, base: u64, out: &mut Vec<&'a [u8]>)
}
if let (Some(off), Some(len)) = (ifd.u64(t::JPEG_INTERCHANGE_FORMAT), ifd.u64(t::JPEG_INTERCHANGE_FORMAT_LENGTH)) {
let off = off.saturating_add(base);
if let Some(s) = data.get(off as usize..(off.saturating_add(len) as usize).min(data.len())) {
if let Some(s) = file_span(data, off, len, budget) {
out.push(s);
}
}
@@ -196,8 +252,12 @@ fn smallest_covering<'a>(found: &[&'a [u8]], largest: &[u8], min_edge: u32) -> O
let (a, b) = (u64::from(w) * u64::from(bh), u64::from(h) * u64::from(bw));
w > 0 && h > 0 && c == bc && a.abs_diff(b) * 100 <= a
};
// (one cut short before its end marker would fail to decode, and nothing would try another: CodeRabbit
// on #3583)
let whole = |s: &[u8]| trim_eoi(s).ends_with(&[0xff, 0xd9]);
found
.iter()
.filter(|s| whole(s))
.filter_map(|&s| jpeg_frame(s).filter(|&f| like(f) && f.0.max(f.1) >= min_edge).map(|(w, h, _)| (s, w.max(h))))
.min_by_key(|&(_, long)| long)
.map(|(s, _)| s)
@@ -210,6 +270,20 @@ fn preview_covering(bytes: &[u8], min_edge: Option<u32>) -> Option<Vec<u8>> {
}
let format = crate::probe(bytes);
if format == Some(crate::RawFormat::Cr3) {
// a Canon CR3 keeps its full-size JPEG beside a 1620 px `PRVW` (and a 160 px `THMB`): a
// thumbnail decodes the small copy, not 45 MP of JPEG (about 150 ms a photo, the grid's
// "popcorn" on a folder of R5 / R5 C raws)
// (without a JPEG track, the largest box is the picture's shape: the boxes alone may hold several)
if let Some(min) = min_edge {
let mut found = cr3_preview_box_list(bytes);
let full = cr3_full_jpeg(bytes);
found.extend(full);
if let Some(largest) = full.or_else(|| largest_by_pixels(&found))
&& let Some(j) = smallest_covering(&found, largest, min)
{
return Some(trim_eoi(j).to_vec());
}
}
return cr3_preview(bytes).map(|j| trim_eoi(j).to_vec());
}
if matches!(format, Some(crate::RawFormat::Crw | crate::RawFormat::Mrw | crate::RawFormat::X3f)) {
@@ -217,8 +291,10 @@ fn preview_covering(bytes: &[u8], min_edge: Option<u32>) -> Option<Vec<u8>> {
}
let tiff = Tiff::parse(bytes).ok()?;
let mut found: Vec<&[u8]> = Vec::new();
// the walks of pointers past the file's end share one budget (see file_span)
let mut budget = scan_budget(bytes.len());
for ifd in tiff.all_ifds() {
candidates(bytes, ifd, 0, &mut found);
candidates(bytes, ifd, 0, &mut found, &mut budget);
}
// maker-note preview IFDs (e.g. Nikon PreviewIFD 0x0011 holds JPEGInterchangeFormat relative to the note base)
if let Some(exif) = tiff.exif()
@@ -226,11 +302,11 @@ fn preview_covering(bytes: &[u8], min_edge: Option<u32>) -> Option<Vec<u8>> {
{
let make = tiff.find(t::MAKE).and_then(|e| e.value.as_str()).unwrap_or_default();
if let Some(mn) = makernote::parse_makernote(bytes, e.offset, e.count() as u64, tiff.order, &make) {
candidates(bytes, &mn.ifd, mn.base, &mut found);
candidates(bytes, &mn.ifd, mn.base, &mut found, &mut budget);
if let Some(off) = mn.ifd.u64(0x0011)
&& let Ok((pifd, _)) = lightcraft_tiff::parse_ifd_at(bytes, mn.base + off, mn.order, mn.base, false, &Default::default())
{
candidates(bytes, &pifd, mn.base, &mut found);
candidates(bytes, &pifd, mn.base, &mut found, &mut budget);
}
}
}
@@ -442,53 +518,123 @@ fn scan_with_budget(data: &[u8], budget: usize) -> (Option<Vec<u8>>, u64) {
/// Canon CR3: the full-size JPEG track (see [`lightcraft_meta::cr3`]), else the `PRVW` / `THMB` boxes.
fn cr3_preview(bytes: &[u8]) -> Option<&[u8]> {
let full = lightcraft_meta::cr3::parse_cr3(bytes)
.and_then(|c| c.tracks.iter().find(|t| t.kind == lightcraft_meta::cr3::Cr3TrackKind::Jpeg).and_then(|t| t.data));
if let Some(j) = full.and_then(|(at, len)| bytes.get(at..at.checked_add(len)?)).filter(|j| is_dct_jpeg(j)) {
return Some(j);
}
cr3_preview_boxes(bytes)
cr3_full_jpeg(bytes).or_else(|| cr3_preview_boxes(bytes))
}
/// A CR3's full-size JPEG track, when it holds a DCT JPEG.
fn cr3_full_jpeg(bytes: &[u8]) -> Option<&[u8]> {
let full = lightcraft_meta::cr3::parse_cr3(bytes)
.and_then(|c| c.tracks.iter().find(|t| t.kind == lightcraft_meta::cr3::Cr3TrackKind::Jpeg).and_then(|t| t.data));
full.and_then(|(at, len)| bytes.get(at..at.checked_add(len)?)).filter(|j| is_dct_jpeg(j))
}
/// Canon CR3 (ISO base media file): the `PRVW` box (Laurent Clévy's CR3 notes; layout confirmed on a CC0
/// sample) is `u32 size, "PRVW", u32 0, u16 ?, u16 width, u16 height, u16 ?, u32 jpeg length, JPEG`; the smaller
/// `THMB` box has the same shape. Returns the larger valid one.
/// Marker-walk budget of the CR3 fallback: candidates can overlap (each claiming the rest of the file) and
/// [`is_dct_jpeg`] walks each marker by marker, so walking every candidate would be quadratic in file size.
/// The `THMB`/`PRVW` previews this finds are far below the cap together; once it is spent the scan stops.
const CR3_MAX_WALK: usize = 8 << 20;
/// Canon CR3 (ISO base media file): the `PRVW` box (Laurent Clévy's CR3 notes; layout confirmed on a CC0
/// sample) is `u32 size, "PRVW", u32 0, u16 ?, u16 width, u16 height, u16 ?, u32 jpeg length, JPEG`; the smaller
/// `THMB` box has the same shape. Returns the larger valid one.
fn cr3_preview_boxes(bytes: &[u8]) -> Option<&[u8]> {
let mut best: Option<&[u8]> = None;
largest_by_pixels(&cr3_preview_box_list(bytes))
}
/// Of `found`, the JPEG with the longest edge in pixels (a smaller picture can take more bytes); the
/// first of equals. CodeRabbit on #3077.
fn largest_by_pixels<'a>(found: &[&'a [u8]]) -> Option<&'a [u8]> {
let long = |j: &[u8]| jpeg_frame(j).map_or(0, |(w, h, _)| w.max(h));
found.iter().copied().rev().max_by_key(|j| long(j))
}
/// The valid JPEGs of a CR3's `PRVW` and `THMB` boxes, every one: a file may carry more than one
/// `PRVW`, and the smallest that covers a size is picked from them all.
fn cr3_preview_box_list(bytes: &[u8]) -> Vec<&[u8]> {
let mut out: Vec<&[u8]> = Vec::new();
let mut walked = 0usize;
// the boxes sit outside the media data: only the rest of the file is searched (the media data
// is most of the file, and a thumbnail is asked for every photo in view)
let file = bytes;
// (every top-level `mdat` is left out, not only the first: a PRVW-shaped JPEG in media data is not a
// preview; CodeRabbit on #3583)
let media = cr3_media_data(file);
let whole_file = media.is_empty();
let mut regions: Vec<&[u8]> = Vec::new();
let mut from = 0usize;
for (start, end) in media {
if start >= from {
regions.extend(file.get(from..start));
}
from = from.max(end);
}
regions.extend(file.get(from..));
for tag in [b"PRVW", b"THMB"] {
let mut from = 0;
while let Some(i) = bytes.get(from..).and_then(|s| s.windows(4).position(|w| w == tag)).map(|p| p + from) {
from = i + 4;
let Some(start) = i.checked_sub(4) else { continue };
let be32 = |at: usize| bytes.get(at..at + 4).map(|s| u32::from_be_bytes([s[0], s[1], s[2], s[3]]) as usize);
let (Some(size), Some(len)) = (be32(start), be32(start + 20)) else { continue };
let j = start + 24;
// 32-bit box lengths can wrap `usize` on wasm32: check the sums, then slice with `get`
let (Some(end), Some(lim)) = (j.checked_add(len), start.checked_add(size.max(24))) else { continue };
if len < 4 || end > lim {
continue;
}
let Some(jpeg) = bytes.get(j..end) else { continue };
// One that doesn't fit the budget left is skipped; a smaller one later may still fit.
if walked + jpeg.len() > CR3_MAX_WALK {
continue;
}
walked += jpeg.len();
if is_dct_jpeg(jpeg) {
if best.is_none_or(|b| b.len() < jpeg.len()) {
best = Some(jpeg);
}
// the whole file searched (its media data not found): the first valid box of a kind only, as
// the declared one comes before the media data and a later look-alike must not win
let mut taken = false;
for &bytes in &regions {
if whole_file && taken {
break;
}
let mut from = 0;
while let Some(i) = bytes.get(from..).and_then(|s| s.windows(4).position(|w| w == tag)).map(|p| p + from) {
from = i + 4;
let Some(start) = i.checked_sub(4) else { continue };
// (the sizes are the file's: checked, a 32-bit build must not wrap them)
let be32 = |at: usize| bytes.get(at..at.checked_add(4)?).map(|s| u32::from_be_bytes([s[0], s[1], s[2], s[3]]) as usize);
let (Some(size), Some(len)) = (be32(start), start.checked_add(20).and_then(be32)) else { continue };
let (Some(j), Some(box_end)) = (start.checked_add(24), start.checked_add(size.max(24))) else { continue };
let Some(end) = j.checked_add(len).filter(|e| len >= 4 && *e <= box_end && *e <= bytes.len()) else { continue };
let Some(jpeg) = bytes.get(j..end) else { continue };
// One that doesn't fit the budget left is skipped; a smaller one later may still fit.
if walked.saturating_add(jpeg.len()) > CR3_MAX_WALK {
continue;
}
walked = walked.saturating_add(jpeg.len());
if is_dct_jpeg(jpeg) {
out.push(jpeg);
taken = true;
if whole_file {
break;
}
// (past it: a tag inside its JPEG is not another box)
from = end;
}
}
}
}
best
out
}
/// Where a CR3's top-level `mdat` boxes lie (`start..end` each, `end` clamped to the file, in file
/// order): the media data, most of the file. Empty when there is none, or the boxes stop making sense
/// before the first. The metadata and the small previews are in the other top-level boxes.
fn cr3_media_data(bytes: &[u8]) -> Vec<(usize, usize)> {
let mut out = Vec::new();
let _ = cr3_media_walk(bytes, &mut out);
out
}
/// [`cr3_media_data`]'s walk of the top-level boxes; `None` where they stop making sense.
fn cr3_media_walk(bytes: &[u8], out: &mut Vec<(usize, usize)>) -> Option<()> {
let be32 = |at: usize| bytes.get(at..at.checked_add(4)?).map(|s| u64::from(u32::from_be_bytes([s[0], s[1], s[2], s[3]])));
let be64 = |at: usize| bytes.get(at..at.checked_add(8)?).map(|s| u64::from_be_bytes([s[0], s[1], s[2], s[3], s[4], s[5], s[6], s[7]]));
let mut at = 0usize;
for _ in 0..64 {
let size = be32(at)?;
let len = match size {
// (a box running to the end of the file)
0 => u64::try_from(bytes.len().saturating_sub(at)).unwrap_or(u64::MAX),
1 => be64(at.checked_add(8)?)?,
n => n,
};
let next = usize::try_from(len).ok().filter(|l| *l >= 8).and_then(|l| at.checked_add(l));
if bytes.get(at.checked_add(4)?..at.checked_add(8)?) == Some(b"mdat".as_slice()) {
out.push((at, next.unwrap_or(bytes.len()).min(bytes.len())));
}
at = next.filter(|n| *n < bytes.len())?;
}
None
}
/// Trim trailing garbage after the last EOI when a stored length over-reports.
crates/raw/src/preview/tests.rs
+350
View File
@@ -296,6 +296,98 @@ fn cr3_prefers_the_full_size_jpeg_track() {
assert_eq!(embedded_preview(&file).unwrap(), small);
}
/// A CR3 whose `PRVW` box holds `prvw` and whose JPEG track holds `big`.
fn cr3_with(prvw_jpeg: &[u8], big: &[u8]) -> Vec<u8> {
let bx = |kind: &[u8; 4], body: &[u8]| -> Vec<u8> { [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat() };
let full = |kind: &[u8; 4], body: &[u8]| bx(kind, &[&[0u8; 4][..], body].concat());
let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom");
let mut prvw = b"\0\0\0\0\0\x01\x06T\x048\0\x01".to_vec();
prvw.extend_from_slice(&(prvw_jpeg.len() as u32).to_be_bytes());
prvw.extend_from_slice(prvw_jpeg);
file.extend(bx(b"PRVW", &prvw));
let mut craw = vec![0u8; 82];
craw.extend(bx(b"JPEG", &[0; 4]));
let at = 4096u64;
let stsd = full(b"stsd", &[&1u32.to_be_bytes()[..], &bx(b"CRAW", &craw)].concat());
let stsz = full(b"stsz", &[0u32.to_be_bytes(), 1u32.to_be_bytes(), (big.len() as u32).to_be_bytes()].concat());
let co64 = full(b"co64", &[&1u32.to_be_bytes()[..], &at.to_be_bytes()].concat());
let trak = bx(b"trak", &bx(b"mdia", &bx(b"minf", &bx(b"stbl", &[stsd, stsz, co64].concat()))));
file.extend(bx(b"moov", &trak));
file.resize(at as usize, 0);
file.extend_from_slice(big);
file
}
/// A CR3 thumbnail decodes the 1620 px `PRVW` copy, not the full-size JPEG (PERF-THUMB-FIRST:
/// 150 ms a photo for the full one); a size the copy can't cover, and the largest preview, are
/// still the full JPEG; a `PRVW` of another shape is never taken for the picture.
#[test]
fn cr3_thumbnails_take_the_smallest_copy_that_covers_them() {
let (prvw, big) = (sized_jpeg(1620, 1080, 3, 300), sized_jpeg(6000, 4000, 3, 3000));
let file = cr3_with(&prvw, &big);
assert_eq!(embedded_preview_covering(&file, 512).unwrap(), prvw);
assert_eq!(embedded_preview_covering(&file, 1620).unwrap(), prvw);
assert_eq!(embedded_preview_covering(&file, 2000).unwrap(), big);
assert_eq!(embedded_preview(&file).unwrap(), big);
let letterboxed = cr3_with(&sized_jpeg(1620, 1215, 3, 300), &big);
assert_eq!(embedded_preview_covering(&letterboxed, 512).unwrap(), big, "another shape is not a copy");
for n in (0..file.len()).step_by(13) {
let _ = embedded_preview_covering(&file[..n], 512);
}
}
/// PERF-CULL-4: the smallest preview that covers the size asked for, of the largest one's shape;
/// the largest when none covers it. A CR3's 8192 px track, 1620 px `PRVW` and 160 px `THMB`.
#[test]
fn the_smallest_preview_covering_the_size_is_chosen() {
let bx = |kind: &[u8; 4], body: &[u8]| -> Vec<u8> { [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat() };
let full = |kind: &[u8; 4], body: &[u8]| bx(kind, &[&[0u8; 4][..], body].concat());
let boxed = |j: &[u8]| {
let mut b = b"\0\0\0\0\0\x01\x06\x54\x04\x38\0\x01".to_vec();
b.extend_from_slice(&(j.len() as u32).to_be_bytes());
b.extend_from_slice(j);
b
};
let (thmb, prvw, big) = (sized_jpeg(160, 107, 1, 50), sized_jpeg(1620, 1080, 1, 300), sized_jpeg(8192, 5464, 1, 3000));
let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom");
file.extend(bx(b"THMB", &boxed(&thmb)));
file.extend(bx(b"PRVW", &boxed(&prvw)));
let mut craw = vec![0u8; 82];
craw.extend(bx(b"JPEG", &[0; 4]));
let at = 4096u64;
let stsd = full(b"stsd", &[&1u32.to_be_bytes()[..], &bx(b"CRAW", &craw)].concat());
let stsz = full(b"stsz", &[0u32.to_be_bytes(), 1u32.to_be_bytes(), (big.len() as u32).to_be_bytes()].concat());
let co64 = full(b"co64", &[&1u32.to_be_bytes()[..], &at.to_be_bytes()].concat());
let trak = bx(b"trak", &bx(b"mdia", &bx(b"minf", &bx(b"stbl", &[stsd, stsz, co64].concat()))));
file.extend(bx(b"moov", &trak));
file.resize(at as usize, 0);
file.extend_from_slice(&big);
assert_eq!(embedded_preview(&file).unwrap(), big);
assert_eq!(embedded_preview_covering(&file, 512).unwrap(), prvw, "a grid thumbnail");
assert_eq!(embedded_preview_covering(&file, 1620).unwrap(), prvw);
assert_eq!(embedded_preview_covering(&file, 100).unwrap(), thmb);
assert_eq!(embedded_preview_covering(&file, 2560).unwrap(), big, "a large loupe");
assert_eq!(embedded_preview_covering(&file, u32::MAX).unwrap(), big);
// a smaller preview of another shape (letterboxed 16:9) is never a stand-in for the 3:2 one
let mut wide = file.clone();
let p = wide.windows(4).position(|w| w == b"PRVW").unwrap();
// the JPEG starts 20 bytes after the tag; its frame height 7 bytes into it
let height = p + 20 + 7;
wide[height..height + 2].copy_from_slice(&911u16.to_be_bytes());
assert_eq!(embedded_preview_covering(&wide, 512).unwrap(), big);
// without the JPEG track the boxes are chosen from too: the smallest that covers, else the largest
// (CodeRabbit on #3577: a box-only CR3 always gave its largest box)
let boxes_only = &file[..file.windows(4).position(|w| w == b"moov").unwrap() - 4];
assert_eq!(embedded_preview_covering(boxes_only, 100).unwrap(), thmb);
assert_eq!(embedded_preview_covering(boxes_only, 512).unwrap(), prvw);
assert_eq!(embedded_preview_covering(boxes_only, 2560).unwrap(), prvw, "none covers: the largest");
assert_eq!(embedded_preview(boxes_only).unwrap(), prvw);
// damaged files never panic
for n in (0..file.len()).step_by(7) {
let _ = embedded_preview_covering(&file[..n], 512);
}
}
/// A minimal CR3 whose Canon box holds `CMT2` (EXIF ColorSpace `exif`) and `CMT3` (maker-note ColorSpace `note`).
fn cr3_with_color_space(exif: Option<u16>, note: Option<u16>) -> Vec<u8> {
let bx = |kind: &[u8; 4], body: &[u8]| -> Vec<u8> { [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat() };
@@ -502,3 +594,261 @@ fn scanned_containers_give_the_largest_colour_jpeg() {
f.extend_from_slice(&mosaic);
assert_eq!(embedded_preview(&f), None);
}
/// A CR3's `PRVW` / `THMB` boxes are searched outside the media data only, but before and after
/// it: a `moov` after the `mdat` (its Canon `uuid` holding the `THMB`) still gives its preview, and
/// bytes inside the media data that look like a preview box are never taken for one.
#[test]
fn cr3_preview_boxes_after_the_media_data_are_found() {
let bx = |kind: &[u8; 4], body: &[u8]| -> Vec<u8> { [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat() };
let boxed = |j: &[u8]| {
let mut b = b"\0\0\0\0\0\x01\x06\x54\x04\x38\0\x01".to_vec();
b.extend_from_slice(&(j.len() as u32).to_be_bytes());
b.extend_from_slice(j);
b
};
let (thmb, in_media) = (sized_jpeg(160, 107, 1, 50), sized_jpeg(1620, 1080, 1, 300));
let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom");
file.extend(bx(b"mdat", &bx(b"PRVW", &boxed(&in_media))));
file.extend(bx(b"moov", &bx(b"uuid", &[&[0x85u8; 16][..], &bx(b"THMB", &boxed(&thmb))].concat())));
assert_eq!(embedded_preview(&file).unwrap(), thmb);
for n in (0..file.len()).step_by(5) {
let _ = embedded_preview(&file[..n]);
}
}
/// Every valid `PRVW` of a CR3 is a candidate (CodeRabbit on #3055): with a small one first and
/// one that covers the size after it, a thumbnail takes the second, not the full-size track.
#[test]
fn a_cr3_with_two_prvw_boxes_offers_both() {
let bx = |kind: &[u8; 4], body: &[u8]| -> Vec<u8> { [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat() };
let full = |kind: &[u8; 4], body: &[u8]| bx(kind, &[&[0u8; 4][..], body].concat());
let boxed = |j: &[u8]| {
let mut b = b"\0\0\0\0\0\x01\x06\x54\x04\x38\0\x01".to_vec();
b.extend_from_slice(&(j.len() as u32).to_be_bytes());
b.extend_from_slice(j);
b
};
let (small, covering, big) = (sized_jpeg(300, 200, 1, 50), sized_jpeg(1620, 1080, 1, 300), sized_jpeg(8192, 5464, 1, 3000));
let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom");
file.extend(bx(b"PRVW", &boxed(&small)));
file.extend(bx(b"PRVW", &boxed(&covering)));
let mut craw = vec![0u8; 82];
craw.extend(bx(b"JPEG", &[0; 4]));
let at = 4096u64;
let stsd = full(b"stsd", &[&1u32.to_be_bytes()[..], &bx(b"CRAW", &craw)].concat());
let stsz = full(b"stsz", &[0u32.to_be_bytes(), 1u32.to_be_bytes(), (big.len() as u32).to_be_bytes()].concat());
let co64 = full(b"co64", &[&1u32.to_be_bytes()[..], &at.to_be_bytes()].concat());
let trak = bx(b"trak", &bx(b"mdia", &bx(b"minf", &bx(b"stbl", &[stsd, stsz, co64].concat()))));
file.extend(bx(b"moov", &trak));
// (the track in the media data, as a camera writes it, after a `free` box up to it)
let pad = at as usize - 16 - file.len();
file.extend(bx(b"free", &vec![0u8; pad]));
file.extend(bx(b"mdat", &big));
assert_eq!(embedded_preview_covering(&file, 1024).unwrap(), covering);
assert_eq!(embedded_preview_covering(&file, 200).unwrap(), small);
}
/// A CR3 preview box claiming the largest sizes a file can give is skipped, never wrapped or
/// panicked on (CodeRabbit on #3080: on a 32-bit build the plain sums could overflow).
#[test]
fn a_cr3_preview_box_with_hostile_sizes_is_skipped() {
let mut file = b"\0\0\0\x18ftypcrx \0\0\0\x01crx isom".to_vec();
file.extend_from_slice(&u32::MAX.to_be_bytes());
file.extend_from_slice(b"PRVW\0\0\0\0\0\x01\x06\x54\x04\x38\0\x01");
file.extend_from_slice(&u32::MAX.to_be_bytes());
file.extend_from_slice(&fake_jpeg(64));
assert!(embedded_preview(&file).is_none());
assert!(embedded_preview_covering(&file, 64).is_none());
}
/// A corrupt raw whose preview pointer (JPEGInterchangeFormat and its length) lies past the end
/// of the file, or whose length runs past it, finds no preview and never panics; a real preview
/// beside the lying pointer is still found (FredApps on #3080: the preview must be read with
/// bounds on its offset and size).
#[test]
fn a_preview_pointer_past_the_end_of_the_file_finds_no_preview() {
let real = sized_jpeg(640, 480, 3, 200);
let lying = |off: u32, len: u32, with_real: bool| {
let mut ifd = IfdBuilder::new();
ifd.set(t::JPEG_INTERCHANGE_FORMAT, Value::Long(vec![off]));
ifd.set(t::JPEG_INTERCHANGE_FORMAT_LENGTH, Value::Long(vec![len]));
if with_real {
ifd.set(t::COMPRESSION, Value::Short(vec![6]));
ifd.set(t::IMAGE_WIDTH, Value::Long(vec![1]));
ifd.set(t::IMAGE_LENGTH, Value::Long(vec![1]));
ifd.set_image(ImageData::Strips { rows_per_strip: 1, strips: vec![real.clone()] });
}
TiffWriter::default().write(&[ifd]).unwrap()
};
let short = lying(0, 0, false).len() as u32;
for (off, len) in [(u32::MAX, u32::MAX), (u32::MAX, 16), (short + 10, 4096), (8, u32::MAX), (short - 2, u32::MAX - 1)] {
let file = lying(off, len, false);
assert_eq!(embedded_preview(&file), None, "pointer {off:#x}+{len:#x} in a {} byte file", file.len());
assert_eq!(embedded_preview_covering(&file, 64), None);
assert_eq!(embedded_preview_color_space(&file), None);
let with_real = lying(off, len, true);
assert_eq!(embedded_preview(&with_real).as_deref(), Some(&real[..]), "the strip preview beside pointer {off:#x}+{len:#x}");
// and every truncation of the file (a raw cut short by a failed copy) is safe too
for n in 0..with_real.len() {
let _ = embedded_preview_covering(&with_real[..n], 64);
}
}
}
/// A preview pointer whose length runs past the end of a file that cuts its JPEG short (no end
/// marker) finds no preview, so a smaller whole one wins; an overstated length whose JPEG is whole
/// inside the file still finds it (CodeRabbit on #3077).
#[test]
fn a_preview_cut_short_by_the_end_of_the_file_is_not_taken() {
let big = sized_jpeg(1600, 1200, 3, 4000);
let small = sized_jpeg(160, 120, 3, 100);
// the small JPEG as a strip, the big one pointed to at the end of the file
let with_tail = |tail: &[u8], declared: u32| {
let probe = {
let mut ifd = IfdBuilder::new();
ifd.set(t::JPEG_INTERCHANGE_FORMAT, Value::Long(vec![0]));
ifd.set(t::JPEG_INTERCHANGE_FORMAT_LENGTH, Value::Long(vec![0]));
ifd.set(t::COMPRESSION, Value::Short(vec![6]));
ifd.set(t::IMAGE_WIDTH, Value::Long(vec![1]));
ifd.set(t::IMAGE_LENGTH, Value::Long(vec![1]));
ifd.set_image(ImageData::Strips { rows_per_strip: 1, strips: vec![small.clone()] });
ifd
};
let at = TiffWriter::default().write(&[probe]).unwrap().len() as u32;
let mut ifd = IfdBuilder::new();
ifd.set(t::JPEG_INTERCHANGE_FORMAT, Value::Long(vec![at]));
ifd.set(t::JPEG_INTERCHANGE_FORMAT_LENGTH, Value::Long(vec![declared]));
ifd.set(t::COMPRESSION, Value::Short(vec![6]));
ifd.set(t::IMAGE_WIDTH, Value::Long(vec![1]));
ifd.set(t::IMAGE_LENGTH, Value::Long(vec![1]));
ifd.set_image(ImageData::Strips { rows_per_strip: 1, strips: vec![small.clone()] });
let mut file = TiffWriter::default().write(&[ifd]).unwrap();
assert_eq!(file.len() as u32, at, "the pointer lands right after the TIFF");
file.extend_from_slice(tail);
file
};
// the big one with a start of scan after its frame header (sized_jpeg has none), as a real JPEG
let mut big = big;
big.splice(21..21, [0xff, 0xda, 0x00, 0x02]);
let declared = big.len() as u32 + 64;
// cut short: half of the big JPEG, no end marker
let cut = with_tail(&big[..big.len() / 2], declared);
assert_eq!(embedded_preview(&cut).as_deref(), Some(&small[..]), "the truncated JPEG is not taken");
// whole, its length overstated past the file's end: still the largest
let whole = with_tail(&big, declared);
assert_eq!(embedded_preview(&whole).as_deref(), Some(&big[..]), "a whole JPEG with an overstated length is kept");
// an end marker inside an APP1 payload is not the JPEG's end: cut short, still not taken
let mut with_app = vec![0xff, 0xd8, 0xff, 0xe1, 0x00, 0x08, 0x12, 0xff, 0xd9, 0x34, 0x56, 0x78];
with_app.extend_from_slice(&big[2..]);
let declared = with_app.len() as u32 + 64;
let cut = with_tail(&with_app[..with_app.len() / 2], declared);
assert_eq!(embedded_preview(&cut).as_deref(), Some(&small[..]), "an FF D9 in APP1 doesn't make a cut JPEG whole");
// whole, with other bytes after it up to the file's end: kept, cut at its own end marker
let mut tail = with_app.clone();
tail.extend_from_slice(&[0x11, 0xff, 0xd9, 0x22]);
let whole = with_tail(&tail, declared);
assert_eq!(embedded_preview(&whole).as_deref(), Some(&with_app[..]), "a whole JPEG is cut at its own end");
// an end marker before any start of scan: no image data, not taken (CodeRabbit on #3080)
let unscanned = sized_jpeg(1600, 1200, 3, 4000);
let whole = with_tail(&unscanned, unscanned.len() as u32 + 64);
assert_eq!(embedded_preview(&whole).as_deref(), Some(&small[..]), "a JPEG without a scan is not taken");
}
/// The walk to a JPEG's end refuses what isn't one and never runs past the bytes it has.
#[test]
fn the_end_of_a_jpeg_is_its_own_end_marker() {
let jpeg_end = |s: &[u8]| super::jpeg_end(s, &mut (1usize << 20));
assert_eq!(jpeg_end(&[0x12, 0x34, 0xff, 0xd9]), None, "not a JPEG");
assert_eq!(jpeg_end(&[0xff, 0xd8, 0xff, 0xd9]), None, "an end before any scan: no image data");
// a stuffed zero and a restart marker in the scan data, then the end
assert_eq!(jpeg_end(&[0xff, 0xd8, 0xff, 0xda, 0x00, 0x02, 0x55, 0xff, 0x00, 0xff, 0xd3, 0x66, 0xff, 0xd9]), Some(14));
// a segment length of 0 or 1, or one running past the end
assert_eq!(jpeg_end(&[0xff, 0xd8, 0xff, 0xe1, 0x00, 0x01, 0xff, 0xd9]), None);
assert_eq!(jpeg_end(&[0xff, 0xd8, 0xff, 0xe1, 0xff, 0xff, 0xff, 0xd9]), None);
for n in 0..12 {
assert_eq!(jpeg_end(&[0xff, 0xd8, 0xff, 0xe1, 0x00, 0x04, 0xff, 0xd9, 0x55, 0xff, 0xff, 0xff][..n]), None, "{n} bytes");
}
}
/// CodeRabbit on #3080: the walks to a JPEG's end share one budget, so a file of many pointers past
/// its end can't make each walk run the whole file: a walk that would overspend it gives up.
#[test]
fn the_walks_to_a_jpegs_end_share_a_budget() {
use super::jpeg_end;
let mut j = vec![0xff, 0xd8, 0xff, 0xda, 0x00, 0x02];
j.extend(std::iter::repeat_n(0x55u8, 1000));
j.extend_from_slice(&[0xff, 0xd9]);
let mut budget = 1500;
assert_eq!(jpeg_end(&j, &mut budget), Some(j.len()), "the first walk fits");
assert!(budget < 1500 - 1000, "and is charged for its steps: {budget} left");
assert_eq!(jpeg_end(&j, &mut budget), None, "the second would overspend what is left");
assert_eq!(budget, 0);
}
/// A CR3 without a JPEG track whose smaller `PRVW` (another shape) takes more bytes: the picture is
/// the box with the most pixels, not the most bytes (CodeRabbit on #3077).
#[test]
fn a_box_only_cr3_is_judged_by_pixels_not_bytes() {
let bx = |kind: &[u8; 4], body: &[u8]| -> Vec<u8> { [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat() };
let boxed = |j: &[u8]| {
let mut b = b"\0\0\0\0\0\x01\x06\x54\x04\x38\0\x01".to_vec();
b.extend_from_slice(&(j.len() as u32).to_be_bytes());
b.extend_from_slice(j);
b
};
let (big, square) = (sized_jpeg(1620, 1080, 3, 300), sized_jpeg(800, 800, 3, 5000));
let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom");
file.extend(bx(b"PRVW", &boxed(&square)));
file.extend(bx(b"PRVW", &boxed(&big)));
// no media data found: the whole file is searched, and only its first box counts (a later look-alike could be
// in media data; CodeRabbit on #3583)
assert_eq!(embedded_preview(&file).unwrap(), square);
file.extend(bx(b"mdat", &[0u8; 16]));
assert_eq!(embedded_preview(&file).unwrap(), big);
assert_eq!(embedded_preview_covering(&file, 100).unwrap(), big, "the square is another picture's shape");
}
/// Every top-level `mdat` is media data, not only the first: a PRVW-shaped JPEG in a second one is
/// never the preview; and a `PRVW` cut short before its end marker isn't chosen to cover a size
/// (CodeRabbit on #3583).
#[test]
fn cr3_boxes_in_any_media_data_or_cut_short_are_not_previews() {
let bx = |kind: &[u8; 4], body: &[u8]| -> Vec<u8> { [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat() };
let boxed = |j: &[u8]| {
let mut b = b"\0\0\0\0\0\x01\x06\x54\x04\x38\0\x01".to_vec();
b.extend_from_slice(&(j.len() as u32).to_be_bytes());
b.extend_from_slice(j);
b
};
let (real, fake) = (sized_jpeg(1620, 1080, 3, 300), sized_jpeg(3000, 2000, 3, 300));
let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom");
file.extend(bx(b"PRVW", &boxed(&real)));
file.extend(bx(b"mdat", &[0u8; 64]));
file.extend(bx(b"mdat", &bx(b"PRVW", &boxed(&fake))));
assert_eq!(embedded_preview(&file).unwrap(), real, "the look-alike in the second mdat is media data");
// a PRVW cut short (no end marker): the full-size track covers a thumbnail instead
let mut short = real.clone();
short.truncate(real.len() - 2);
let big = sized_jpeg(6000, 4000, 3, 3000);
assert_eq!(embedded_preview_covering(&cr3_with(&short, &big), 512).unwrap(), big);
assert_eq!(embedded_preview_covering(&cr3_with(&real, &big), 512).unwrap(), real, "a whole one is still taken");
}
/// The copy a CR3 thumbnail is given decodes, whichever is chosen (CodeRabbit on #3583: the other CR3 fixtures have
/// no scan): real encoded JPEGs, the `PRVW` for a small size, the full-size track past it.
#[test]
fn the_cr3_copy_chosen_for_a_thumbnail_decodes() {
let encode = |w: u16, h: u16| {
let px: Vec<u8> = (0..usize::from(w) * usize::from(h)).flat_map(|i| [(i % 251) as u8, 90, 160]).collect();
let mut j = Vec::new();
jpeg_encoder::Encoder::new(&mut j, 85).encode(&px, w, h, jpeg_encoder::ColorType::Rgb).unwrap();
j
};
let file = cr3_with(&encode(96, 64), &encode(384, 256));
for (min, want) in [(64u32, (96, 64)), (200, (384, 256))] {
let j = embedded_preview_covering(&file, min).unwrap();
let (_, w, h, c) = crate::tiffraw::lossy_jpeg(&j, 3, 1 << 20).unwrap();
assert_eq!(((w, h), c), (want, 3), "covering {min} px");
}
}
crates/ui-egui/locales/cs.json
+1
View File
@@ -1245,6 +1245,7 @@
"Specific folder": "Konkrétní složka",
"Same folder as original photo": "Stejná složka jako původní fotografie",
"Tidying the thumbnail cache": "Úklid mezipaměti miniatur",
"Checking free disk space": "Kontrola volného místa na disku",
"No masks yet. Choose a mask type above.": "Zatím žádné masky. Vyberte typ masky nahoře.",
"No matching commands": "Žádné odpovídající příkazy",
"No matching photos": "Žádné odpovídající fotografie",
crates/ui-egui/locales/de.json
+3
View File
@@ -2329,6 +2329,7 @@
"Specific folder": "Bestimmter Ordner",
"Same folder as original photo": "Im selben Ordner wie das Originalfoto",
"Tidying the thumbnail cache": "Miniaturcache wird aufgeräumt",
"Checking free disk space": "Freier Speicherplatz wird geprüft",
"Reset all edits (Cmd+Shift+R)": "Alle Bearbeitungen zurücksetzen (⌘⇧R)",
"Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "Smart-Alben aktualisieren sich selbst: Ändere ihre Regeln oder verwende Foto → Foto löschen",
"LightCraft stopped": "LightCraft wurde beendet",
@@ -3459,6 +3460,8 @@
"The photos are imported once it ends": "Die Fotos werden danach importiert",
"An import is running. Drop photos when it has finished": "Ein Import läuft. Lege die Fotos ab, wenn er fertig ist",
"Finish Lightroom catalog import before adding photos": "Schließe den Import des Lightroom-Katalogs ab, bevor du Fotos hinzufügst",
"Camera preview": "Kameravorschau",
"The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "Das eigene JPEG der Kamera, angezeigt, bis LightCraft die Raw-Datei gerendert hat. Farbe und Tonwerte am gerenderten Foto beurteilen.",
"Profile: from the file (the camera's own correction)": "Profil: aus der Datei (Korrektur der Kamera)",
"Downloading the lens database…": "Objektivdatenbank wird geladen…",
"This file names no lens.": "Diese Datei nennt kein Objektiv.",
crates/ui-egui/locales/es.json
+3
View File
@@ -2333,6 +2333,7 @@
"Specific folder": "Carpeta específica",
"Same folder as original photo": "Misma carpeta que la foto original",
"Tidying the thumbnail cache": "Ordenando la caché de miniaturas",
"Checking free disk space": "Comprobando el espacio libre en disco",
"Reset all edits (Cmd+Shift+R)": "Restablecer todas las ediciones (Cmd+Mayús+R)",
"Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "Los álbumes inteligentes se actualizan solos: cambie sus reglas o use Foto → Eliminar foto",
"LightCraft stopped": "LightCraft se detuvo",
@@ -2378,6 +2379,8 @@
"The photos are imported once it ends": "Las fotos se importarán después",
"An import is running. Drop photos when it has finished": "Hay una importación en curso. Suelta las fotos cuando termine",
"Finish Lightroom catalog import before adding photos": "Termina la importación del catálogo de Lightroom antes de añadir fotos",
"Camera preview": "Vista previa de la cámara",
"The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "El JPEG propio de la cámara, mostrado hasta que LightCraft haya renderizado el RAW. Juzga el color y el tono en la foto renderizada.",
"Profile: from the file (the camera's own correction)": "Perfil: del archivo (corrección de la propia cámara)",
"Downloading the lens database…": "Descargando la base de datos de objetivos…",
"This file names no lens.": "Este archivo no indica ningún objetivo.",
crates/ui-egui/locales/fi.json
+1
View File
@@ -2065,6 +2065,7 @@
"Thursday": "torstai",
"Tick the box to accept the model's terms first": "Hyväksy ensin mallin ehdot valitsemalla ruutu",
"Tidying the thumbnail cache": "Siivotaan pikkukuvien välimuistia",
"Checking free disk space": "Tarkistetaan vapaata levytilaa",
"Time zone": "Aikavyöhyke",
"Tint": "Värisävy",
"Title": "Otsikko",
crates/ui-egui/locales/fr.json
+3
View File
@@ -2328,6 +2328,7 @@
"Specific folder": "Dossier spécifique",
"Same folder as original photo": "Même dossier que la photo d’origine",
"Tidying the thumbnail cache": "Rangement du cache des miniatures",
"Checking free disk space": "Vérification de l'espace disque libre",
"Reset all edits (Cmd+Shift+R)": "Réinitialiser toutes les retouches (Cmd+Maj+R)",
"Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "Les albums intelligents se mettent à jour d'eux-mêmes : modifiez leurs règles ou utilisez Photo → Supprimer la photo",
"LightCraft stopped": "LightCraft s’est arrêté",
@@ -2409,6 +2410,8 @@
"The photos are imported once a library is open": "Les photos seront importées dès qu'une bibliothèque sera ouverte",
"Open a library first, then drop the presets again": "Ouvrez d'abord une bibliothèque, puis déposez à nouveau les préréglages",
"This library couldn't be checked:": "Impossible de vérifier cette bibliothèque :",
"Camera preview": "Aperçu de l'appareil",
"The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "Le JPEG de l'appareil photo, affiché jusqu'à ce que LightCraft ait rendu le fichier RAW. Jugez la couleur et les tons sur la photo rendue.",
"Profile: from the file (the camera's own correction)": "Profil : du fichier (correction de l'appareil)",
"Downloading the lens database…": "Téléchargement de la base de données d'objectifs…",
"This file names no lens.": "Ce fichier n'indique aucun objectif.",
crates/ui-egui/locales/id.json
+1
View File
@@ -2062,6 +2062,7 @@
"Thursday": "Kamis",
"Tick the box to accept the model's terms first": "Centang kotak untuk menyetujui ketentuan model terlebih dahulu",
"Tidying the thumbnail cache": "Merapikan cache thumbnail",
"Checking free disk space": "Memeriksa ruang disk kosong",
"Time zone": "Zona waktu",
"Tint": "Rona",
"Title": "Judul",
crates/ui-egui/locales/it.json
+1
View File
@@ -937,6 +937,7 @@
"Specific folder": "Cartella specifica",
"Same folder as original photo": "Stessa cartella della foto originale",
"Tidying the thumbnail cache": "Riordino della cache delle miniature",
"Checking free disk space": "Controllo dello spazio libero su disco",
"No masks yet. Choose a mask type above.": "Ancora nessuna maschera. Scegli un tipo di maschera qui sopra.",
"No matching commands": "Nessun comando corrispondente",
"No matching photos": "Nessuna foto corrispondente",
crates/ui-egui/locales/ja.json
+3
View File
@@ -2331,6 +2331,7 @@
"Specific folder": "指定フォルダー",
"Same folder as original photo": "元の写真と同じフォルダー",
"Tidying the thumbnail cache": "サムネールキャッシュを整理中",
"Checking free disk space": "ディスクの空き容量を確認中",
"Reset all edits (Cmd+Shift+R)": "すべての編集をリセット（⌘⇧R）",
"Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "スマートアルバムは自動で更新されます。ルールを変更するか、写真 → 写真を削除を使ってください",
"LightCraft stopped": "LightCraftが終了しました",
@@ -2376,6 +2377,8 @@
"The photos are imported once it ends": "終わりしだい写真を読み込みます",
"An import is running. Drop photos when it has finished": "読み込み中です。完了してから写真をドロップしてください",
"Finish Lightroom catalog import before adding photos": "写真を追加する前に Lightroom カタログの読み込みを完了してください",
"Camera preview": "カメラのプレビュー",
"The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "LightCraft が RAW を現像するまで表示される、カメラ自身の JPEG です。色とトーンは現像後の写真で判断してください。",
"Profile: from the file (the camera's own correction)": "プロファイル: ファイル内（カメラ自身の補正）",
"Downloading the lens database…": "レンズデータベースをダウンロード中…",
"This file names no lens.": "このファイルにはレンズ名がありません。",
crates/ui-egui/locales/ka.json
+1
View File
@@ -2945,6 +2945,7 @@
"This row covers more than one disk: drop on a folder inside it": "ეს სტრიქონი ერთზე მეტ დისკს მოიცავს: ჩააგდეთ მის შიგნით მყოფ საქაღალდეზე",
"This screen is off: turn it on under Titles to play it.": "ეს ეკრანი გამორთულია: დასაკრავად ჩართეთ „ტიტრებში“.",
"Tidying the thumbnail cache": "ესკიზების ქეში სუფთავდება",
"Checking free disk space": "დისკზე თავისუფალი ადგილის შემოწმება",
"Title text": "ტიტრის ტექსტი",
"Titles": "ტიტრები",
"Tokens: {file}, {name}, {title}, {caption}, {date}, {exposure}, {iso}, {focal}, {camera}, {lens}, {rating}, {creator}, {copyright}. Other text shows as typed.": "ტეგები: {file}, {name}, {title}, {caption}, {date}, {exposure}, {iso}, {focal}, {camera}, {lens}, {rating}, {creator}, {copyright}. სხვა ტექსტი ისე ჩანს, როგორც აკრიფეთ.",
crates/ui-egui/locales/pt-br.json
+3
View File
@@ -2171,6 +2171,7 @@
"Specific folder": "Pasta específica",
"Same folder as original photo": "Mesma pasta da foto original",
"Tidying the thumbnail cache": "Organizando o cache de miniaturas",
"Checking free disk space": "Verificando o espaço livre em disco",
"Reset all edits (Cmd+Shift+R)": "Redefinir todas as edições (Cmd+Shift+R)",
"Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "Álbuns inteligentes se atualizam sozinhos: altere as regras ou use Foto → Excluir foto",
"LightCraft stopped": "O LightCraft parou",
@@ -2222,6 +2223,8 @@
"The photos are imported once it ends": "As fotos serão importadas depois",
"An import is running. Drop photos when it has finished": "Uma importação está em andamento. Solte as fotos quando ela terminar",
"Finish Lightroom catalog import before adding photos": "Conclua a importação do catálogo do Lightroom antes de adicionar fotos",
"Camera preview": "Prévia da câmera",
"The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "O JPEG da própria câmera, mostrado até o LightCraft renderizar o RAW. Avalie cor e tom na foto renderizada.",
"Profile: from the file (the camera's own correction)": "Perfil: do arquivo (correção da própria câmera)",
"Downloading the lens database…": "Baixando o banco de dados de lentes…",
"This file names no lens.": "Este arquivo não indica nenhuma lente.",
crates/ui-egui/locales/ru.json
+3
View File
@@ -2539,6 +2539,7 @@
"Specific folder": "Указанная папка",
"Same folder as original photo": "Та же папка, что и у оригинала",
"Tidying the thumbnail cache": "Упорядочивание кэша миниатюр",
"Checking free disk space": "Проверка свободного места на диске",
"Reset all edits (Cmd+Shift+R)": "Сбросить все правки (Cmd+Shift+R)",
"Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "Смарт-альбомы обновляются сами: измените их правила или выберите Фото → Удалить фото",
"LightCraft stopped": "LightCraft остановился",
@@ -2584,6 +2585,8 @@
"The photos are imported once it ends": "Фото будут импортированы после этого",
"An import is running. Drop photos when it has finished": "Импорт уже выполняется. Перетащите фото, когда он закончится",
"Finish Lightroom catalog import before adding photos": "Завершите импорт каталога Lightroom, прежде чем добавлять фото",
"Camera preview": "Превью камеры",
"The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "Собственный JPEG камеры, показанный, пока LightCraft не обработает RAW. Оценивайте цвет и тон по обработанному фото.",
"Profile: from the file (the camera's own correction)": "Профиль: из файла (коррекция самой камеры)",
"Downloading the lens database…": "Загружается база объективов…",
"This file names no lens.": "В этом файле не указан объектив.",
crates/ui-egui/locales/sv.json
+1
View File
@@ -2242,6 +2242,7 @@
"Thursday": "Torsdag",
"Tick the box to accept the model's terms first": "Markera rutan för att godkänna modellens villkor först",
"Tidying the thumbnail cache": "Städar miniatyrcachen",
"Checking free disk space": "Kontrollerar ledigt diskutrymme",
"Time zone": "Tidszon",
"Tint": "Färgton",
"Title": "Titel",
crates/ui-egui/locales/uk.json
+3
View File
@@ -2329,6 +2329,7 @@
"Specific folder": "Вказана папка",
"Same folder as original photo": "Та сама папка, що й оригінал",
"Tidying the thumbnail cache": "Упорядкування кешу мініатюр",
"Checking free disk space": "Перевірка вільного місця на диску",
"Reset all edits (Cmd+Shift+R)": "Скинути всі зміни (Cmd+Shift+R)",
"Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "Розумні альбоми оновлюються самі: змініть їхні правила або виберіть Фото → Видалити фото",
"LightCraft stopped": "LightCraft зупинився",
@@ -2374,6 +2375,8 @@
"The photos are imported once it ends": "Фото буде імпортовано після цього",
"An import is running. Drop photos when it has finished": "Імпорт уже виконується. Перетягніть фото, коли він завершиться",
"Finish Lightroom catalog import before adding photos": "Завершіть імпорт каталогу Lightroom, перш ніж додавати фото",
"Camera preview": "Прев'ю камери",
"The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "Власний JPEG камери, показаний, доки LightCraft не обробить RAW. Оцінюйте колір і тон за обробленим фото.",
"Profile: from the file (the camera's own correction)": "Профіль: з файлу (корекція самої камери)",
"Downloading the lens database…": "Завантажується база об'єктивів…",
"This file names no lens.": "У цьому файлі не вказано об'єктив.",
crates/ui-egui/locales/vi.json
+1
View File
@@ -2568,6 +2568,7 @@
"Thumbnail size": "Kích thước ảnh thu nhỏ",
"Import Thumbnail Size": "Kích thước ảnh thu nhỏ khi nhập",
"Tidying the thumbnail cache": "Đang dọn dẹp bộ nhớ đệm ảnh thu nhỏ",
"Checking free disk space": "Đang kiểm tra dung lượng đĩa trống",
"Title bar": "Thanh tiêu đề",
"Title text": "Văn bản tiêu đề",
"Titles": "Tiêu đề",
crates/ui-egui/locales/zh-hans.json
+3
View File
@@ -2164,6 +2164,7 @@
"Specific folder": "指定文件夹",
"Same folder as original photo": "与原始照片相同的文件夹",
"Tidying the thumbnail cache": "正在整理缩略图缓存",
"Checking free disk space": "正在检查可用磁盘空间",
"Reset all edits (Cmd+Shift+R)": "重置所有编辑 (Cmd+Shift+R)",
"Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "智能相册会自动更新：请更改其规则，或使用“照片 → 删除照片”",
"LightCraft stopped": "LightCraft 已停止",
@@ -2221,6 +2222,8 @@
"The photos are imported once it ends": "结束后即导入这些照片",
"An import is running. Drop photos when it has finished": "正在导入。导入完成后再拖放照片",
"Finish Lightroom catalog import before adding photos": "添加照片前，请先完成 Lightroom 目录的导入",
"Camera preview": "相机预览",
"The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "相机自己的 JPEG，在 LightCraft 渲染出原始文件之前显示。请在渲染后的照片上判断颜色和影调。",
"Profile: from the file (the camera's own correction)": "配置文件：来自文件（相机自身的校正）",
"Downloading the lens database…": "正在下载镜头数据库…",
"This file names no lens.": "此文件未注明镜头。",
crates/ui-egui/locales/zh-hant.json
+3
View File
@@ -2132,6 +2132,7 @@
"Specific folder": "指定資料夾",
"Same folder as original photo": "與原始照片相同的資料夾",
"Tidying the thumbnail cache": "正在整理縮圖快取",
"Checking free disk space": "正在檢查可用磁碟空間",
"Reset all edits (Cmd+Shift+R)": "重設所有編輯（Cmd+Shift+R）",
"Smart albums update automatically: change their rules, or use Photo ▸ Delete Photo": "智慧型相簿會自動更新：請變更其規則，或使用「照片 → 刪除照片」",
"LightCraft stopped": "LightCraft 已停止",
@@ -2177,6 +2178,8 @@
"The photos are imported once it ends": "結束後即匯入這些照片",
"An import is running. Drop photos when it has finished": "正在匯入。匯入完成後再拖放照片",
"Finish Lightroom catalog import before adding photos": "新增照片前，請先完成 Lightroom 目錄的匯入",
"Camera preview": "相機預覽",
"The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.": "相機自己的 JPEG，在 LightCraft 算繪出原始檔之前顯示。請在算繪後的照片上判斷色彩與色調。",
"Profile: from the file (the camera's own correction)": "設定檔：來自檔案（相機本身的校正）",
"Downloading the lens database…": "正在下載鏡頭資料庫…",
"This file names no lens.": "此檔案未註明鏡頭。",
crates/ui-egui/src/app.rs
+44
View File
@@ -65,6 +65,8 @@ impl LightcraftApp {
#[cfg(test)]
library_passes: vec![],
#[cfg(test)]
prep_ticks: 0,
#[cfg(test)]
missing_count_test_delay: None,
shadow: None,
synthetic: vec![],
@@ -94,6 +96,9 @@ impl LightcraftApp {
space_hand: None,
brush_live: None,
loupe_shown: None,
loupe_timing: Default::default(),
prep_synced: false,
look_ahead: Default::default(),
region_view: None,
region_before_view: None,
size_hold: Default::default(),
@@ -106,6 +111,7 @@ impl LightcraftApp {
merge: merge::MergeState::default(),
import: None,
scan: None,
scans_stopping: Default::default(),
waiting_import: Default::default(),
sync: None,
sync_owns_changes: false,
@@ -123,6 +129,8 @@ impl LightcraftApp {
window_is_fullscreen: false,
gpu_applied: None,
gpu_notice: false,
disk_space_notice: false,
disk_notice: None,
memory_applied: None,
display_applied: None,
display_error: None,
@@ -488,6 +496,41 @@ impl LightcraftApp {
}
}
/// The app-wide background preparation settings live with the app settings (`ui.json`): taken
/// into the session once, then a change made through `prefs.setBackgroundPrep` is saved with
/// them. Run at the start of every logic pass, before any command of that pass (also before
/// the fonts are ready and the window is drawn).
fn sync_prep_settings(&mut self) {
if !self.prep_synced {
self.session.prep.app = self.ui.settings.background_prep.sanitized();
self.prep_synced = true;
} else if self.session.prep.app != self.ui.settings.background_prep {
self.ui.settings.background_prep = self.session.prep.app;
}
}
/// Preparing photos in the background ([`lightcraft_engine::prep`]): the whole view's previews
/// once the user is culling in the loupe, built from the photo on screen in the direction of travel.
/// Advanced on every logic pass, the hidden ones too (a minimised window keeps preparing).
fn background_prep(&mut self, ctx: &egui::Context) {
#[cfg(test)]
{
self.prep_ticks += 1;
}
let mode = panels::detail::prep_mode(self);
let loupe = self.ui.fullscreen || self.ui.view == state::ViewMode::Detail;
let at = if loupe { self.session.active() } else { None };
let backward = self.look_ahead.backward();
self.session.prep_tick(mode, at, backward);
if self.session.preview_build.as_ref().is_some_and(|b| b.running()) {
ctx.request_repaint_after(std::time::Duration::from_millis(500));
}
// a build that couldn't start tries again at its time, also when nothing else draws a frame
if let Some(wait) = self.session.prep_retry_in() {
ctx.request_repaint_after(wait);
}
}
/// AI masks: apply finished background requests (clicks, descriptions, detail passes) and
/// show their errors; start a zoomed-in detail pass once the clicking stops
/// (`ui.detail_due`); watch the model download. Never waits for the model.
@@ -584,6 +627,7 @@ impl LightcraftApp {
/// hidden, from `Context::run_logic`. It may read the refreshed viewport fields only: never
/// `i.time`, events, keys or dropped files.
pub fn logic(&mut self, ctx: &egui::Context) {
self.sync_prep_settings();
i18n::set_language(self.ui.language);
// The fork's AccessKit root language and bidi runs follow the locale; presentation only.
// Compare against the cached string: `Context::language()` allocates on every call, and
crates/ui-egui/src/app/frame.rs
+4 -1
View File
@@ -44,7 +44,8 @@ impl LightcraftApp {
/// monotonic app clock, so a hidden `run_logic` keeps it advancing and a second layout pass is
/// a no-op rather than a duplicate.
pub(super) fn logic_background(&mut self, ctx: &egui::Context) {
panels::notices::logic(self);
self.background_prep(ctx);
panels::notices::logic(self, ctx);
// closing the window (or Quit) with changes only in memory: retry, else ask first
if ctx.input(|i| i.viewport().close_requested()) && !panels::notices::may_close(self) {
ctx.send_viewport_cmd(egui::ViewportCommand::CancelClose);
@@ -358,6 +359,8 @@ impl LightcraftApp {
}
self.widgets = std::mem::take(&mut self.next_widgets);
self.widget_hits = std::mem::take(&mut self.next_widget_hits);
// a frame without the loupe (grid, Compare, Survey, slideshow…) ends the loupe's step timing
self.loupe_timing.end_frame(ctx.input(|i| i.time));
panels::notices::finish_ui_pass(self, ctx);
panels::library_problem::finish_ui_pass(self, ctx);
self.ui_passes = self.ui_passes.wrapping_add(1);
crates/ui-egui/src/control.rs
+18 -1
View File
@@ -133,7 +133,7 @@ pub fn inspect(app: &LightcraftApp, ctx: &egui::Context) -> Value {
"selection": app.session.selection.ids.iter().map(|p| p.0).collect::<Vec<_>>(),
"activeMask": app.session.active_mask,
"widgetCount": app.widgets.len(),
"perf": {"viewFollowed": app.session.view_followed, "viewWorkedOut": app.session.view_worked_out, "frameMs": app.perf.frame_ms, "logicMs": app.perf.logic_ms, "updateMs": app.perf.update_ms, "maxUpdateMs": app.perf.max_update_ms, "fps": app.perf.fps, "lastRenderMs": app.renderer.last_main_ms, "renderQueue": app.renderer.queued(), "rendersInFlight": app.renderer.in_flight(), "pendingSlots": app.renderer.pending_slots(), "mergeRunning": app.merge.busy(), "lastMerge": app.merge.last_result, "rendersDone": app.renderer.completed, "thumbTextures": app.renderer.thumb_textures(), "variantTextures": app.renderer.variant_textures(), "gpu": (lightcraft_engine::gpu::ready() && lightcraft_engine::gpu::available()).then(lightcraft_engine::gpu::adapter_name).flatten(), "gpuReason": lightcraft_engine::gpu::unavailable_reason(), "gpuFallback": lightcraft_engine::gpu::last_fallback(), "gpuWindowDevice": lightcraft_engine::gpu::on_window_device(), "screenTextures": app.renderer.has_screen_textures(), "mainOnGpu": app.renderer.textures.get(&crate::render::Slot::Main).is_some_and(|t| t.native.is_some())},
"perf": {"viewFollowed": app.session.view_followed, "viewWorkedOut": app.session.view_worked_out, "frameMs": app.perf.frame_ms, "logicMs": app.perf.logic_ms, "updateMs": app.perf.update_ms, "maxUpdateMs": app.perf.max_update_ms, "fps": app.perf.fps, "lastRenderMs": app.renderer.last_main_ms, "renderQueue": app.renderer.queued(), "rendersInFlight": app.renderer.in_flight(), "pendingSlots": app.renderer.pending_slots(), "mergeRunning": app.merge.busy(), "lastMerge": app.merge.last_result, "rendersDone": app.renderer.completed, "thumbTextures": app.renderer.thumb_textures(), "thumbsMade": app.renderer.thumbs_made, "variantTextures": app.renderer.variant_textures(), "gpu": (lightcraft_engine::gpu::ready() && lightcraft_engine::gpu::available()).then(lightcraft_engine::gpu::adapter_name).flatten(), "gpuReason": lightcraft_engine::gpu::unavailable_reason(), "gpuFallback": lightcraft_engine::gpu::last_fallback(), "gpuWindowDevice": lightcraft_engine::gpu::on_window_device(), "screenTextures": app.renderer.has_screen_textures(), "mainOnGpu": app.renderer.textures.get(&crate::render::Slot::Main).is_some_and(|t| t.native.is_some()), "loupeSteps": app.loupe_timing.to_json(), "previewCache": preview_cache_json(app)},
"loupe": app.loupe_shown.map(|(p, src)| json!({
"photo": p.0,
"source": src,
@@ -894,3 +894,20 @@ pub fn save_screenshot(app: &mut LightcraftApp, image: &egui::ColorImage, target
}
}
}
/// The preview cache's numbers for `ui.inspect` → `perf.previewCache`: what memory holds, and the
/// disk cache's hits, misses and writes this session (notes 47 and 48: are built previews used?).
fn preview_cache_json(app: &crate::LightcraftApp) -> serde_json::Value {
use std::sync::atomic::Ordering;
let (entries, bytes) = app.session.media.rendered.mem_usage();
let disk = app.session.media.rendered.disk();
serde_json::json!({
"memEntries": entries,
"memBytes": bytes,
"diskHits": disk.map(|d| d.hits.load(Ordering::Relaxed)),
"diskMisses": disk.map(|d| d.misses.load(Ordering::Relaxed)),
"diskWrites": disk.map(|d| d.writes.load(Ordering::Relaxed)),
"diskBudget": disk.map(|d| d.budget()),
"loupeEdge": app.session.loupe_edge,
})
}
crates/ui-egui/src/dialogs/import.rs
+3 -2
View File
@@ -34,11 +34,12 @@ pub(crate) use body::thumb_edge;
use body::{DEFAULT_BELOW_GRID, badge_rect, dup_counts};
pub use body::{DEFAULT_FOLDER_TEMPLATE, body, example_destination, source_summary};
pub(crate) use body::{DIALOG_SIZE, THUMB_SIZE_DEFAULT, note_dialog_bottom, thumb_size};
pub(crate) use scan::StoppingScans;
#[cfg(not(target_arch = "wasm32"))]
pub(crate) use scan::cards_seen;
pub use scan::{ScanTask, browse, open, poll_scan, running};
#[cfg(test)]
pub(crate) use scan::{stall_scan, stall_scan_of};
pub(crate) use scan::{HELD_RESOLVES, ScanEnd, scan_by_hand, stall_scan, stall_scan_of};
pub use scan::{ScanTask, browse, open, poll_scan, running};
pub(crate) use tags::{tag_field_width, tag_help, tag_toggle, unknown_tags_warning};
pub use task::{ImportTask, start, start_paths, tick};
crates/ui-egui/src/dialogs/import/scan.rs
+210 -39
View File
@@ -5,7 +5,7 @@ use super::{ImportDialog, LightcraftApp, Ordering, ScanInput, ScanOutput, ScanPr
/// window must keep answering meanwhile).
pub struct ScanTask {
progress: std::sync::Arc<ScanProgress>,
rx: std::sync::mpsc::Receiver<ScanOutput>,
rx: std::sync::mpsc::Receiver<ScanEnd>,
/// Open the review with "copy into the library" checked (a camera / card).
pub copy: bool,
/// Open the review with this album chosen (an album's context menu ▸ Import…).
@@ -16,6 +16,91 @@ pub struct ScanTask {
sources: Vec<String>,
/// The scan's row in the activity stack (its ✕ and `activity.cancel` set `progress.cancel`).
guard: lightcraft_engine::activity::TaskGuard,
/// Browsing: the folder as the worker resolved it, once it has. The view shows it as given until then
/// (resolving reads the disk, which a stalled network mount can block: never on this thread).
resolved: Option<std::sync::mpsc::Receiver<String>>,
}
/// What a scan's worker sends back. Every way it ends sends one, so the probes of earlier scans that
/// [`ScanInput::new`] took from the session always go back to it (a later import doesn't read those files again).
pub(crate) enum ScanEnd {
/// The scan finished.
Done(ScanOutput),
/// Browsing stopped without a listing: the folder couldn't be listed (`failed`), or another folder was
/// chosen. Never an empty listing, which the browse bookkeeping would take for every file gone.
Stopped { probes: std::collections::HashMap<String, lightcraft_engine::media::ProbeInfo>, failed: bool },
}
impl ScanEnd {
/// The probes this end hands back to the session.
fn into_probes(self) -> std::collections::HashMap<String, lightcraft_engine::media::ProbeInfo> {
match self {
ScanEnd::Done(out) => out.probes,
ScanEnd::Stopped { probes, .. } => probes,
}
}
}
/// Scans stopped without waiting for their workers ([`stop_scan`]), in the order they were stopped, which is
/// the order they started (one scan runs at a time), and which of their probes went back to the session.
#[derive(Default)]
pub(crate) struct StoppingScans {
/// How many scans were stopped: each one's number.
stopped: u64,
/// Each stopped scan's number and channel, kept until its worker ends.
pending: Vec<(u64, std::sync::mpsc::Receiver<ScanEnd>)>,
/// The probes a stopped scan put back: by whom, and what they said (to tell they are still the ones there).
from: std::collections::HashMap<String, (u64, ProbeMark)>,
}
/// What tells two probes of one file apart: a file that changed but kept its size has another content hash.
type ProbeMark = (u64, Option<String>, Option<String>);
fn mark(p: &lightcraft_engine::media::ProbeInfo) -> ProbeMark {
(p.file_size, p.content_hash.clone(), p.file_modified.clone())
}
/// Stop the running scan (if any) without waiting for its worker: the scan is closed at once, and its
/// channel is kept until the worker ends, so the probes it holds still come back ([`collect_stopped`]).
fn stop_scan(app: &mut LightcraftApp) {
if let Some(t) = app.scan.take() {
t.progress.cancel.store(true, Ordering::Relaxed);
let s = &mut app.scans_stopping;
s.stopped = s.stopped.saturating_add(1);
s.pending.push((s.stopped, t.rx));
}
}
/// The probes of stopped scans whose workers have ended go back to the session (called every frame). A probe
/// the session got since a scan started is newer than that scan's and stays, except one that a scan stopped
/// before it put back: a later scan read that file again, so its probe wins, whichever worker ends first
/// (an older probe of a file changed at the same size would hand the copy check a stale content hash).
fn collect_stopped(app: &mut LightcraftApp, ctx: &egui::Context) {
let probes = &mut app.session.import_probes;
let StoppingScans { pending, from, .. } = &mut app.scans_stopping;
pending.retain(|(seq, rx)| match rx.try_recv() {
Ok(end) => {
for (k, v) in end.into_probes() {
let replace = match probes.get(&k) {
None => true,
Some(have) => from.get(&k).is_some_and(|(by, m)| by < seq && *m == mark(have)),
};
if replace {
from.insert(k.clone(), (*seq, mark(&v)));
probes.insert(k, v);
}
}
false
}
Err(std::sync::mpsc::TryRecvError::Empty) => true,
Err(std::sync::mpsc::TryRecvError::Disconnected) => false,
});
if pending.is_empty() {
// no stopped scan left to compare with
from.clear();
} else {
ctx.request_repaint_after(std::time::Duration::from_millis(100));
}
}
/// The activity row of a scan: the review's ("Scanning folder") or the Local view's ("Reading folder").
@@ -66,14 +151,14 @@ pub fn open(app: &mut LightcraftApp, paths: Vec<String>) -> Result<Value, String
let (tx, rx) = std::sync::mpsc::channel();
let p = progress.clone();
let job = move || {
let _ = tx.send(scan_with(input, &paths, &p));
let _ = tx.send(ScanEnd::Done(scan_with(input, &paths, &p)));
};
#[cfg(not(target_arch = "wasm32"))]
std::thread::spawn(job);
#[cfg(target_arch = "wasm32")]
job();
let guard = scan_guard(app, &progress, false);
app.scan = Some(ScanTask { progress, rx, copy: false, album: None, browse: false, sources, guard });
app.scan = Some(ScanTask { progress, rx, copy: false, album: None, browse: false, sources, guard, resolved: None });
Ok(json!({"scanning": true}))
}
@@ -82,75 +167,139 @@ pub fn open(app: &mut LightcraftApp, paths: Vec<String>) -> Result<Value, String
/// and the photos then join the view in small batches under a progress window. A browse already
/// running is replaced; the import review's scan is not.
pub fn browse(app: &mut LightcraftApp, path: &str, subfolders: Option<bool>) -> Result<Value, String> {
let dir = std::path::absolute(std::path::Path::new(path)).map_err(|e| e.to_string())?;
if !dir.is_dir() {
return Err(format!("{path}: not a folder"));
}
// one spelling for the view, the listing and the catalog's records: `a/lib/../outside` is `a/outside`. Taken out
// by name here, without reading the disk (a link on a stalled network mount would freeze the window): the worker
// resolves the folder, checks it is one, and the view takes its spelling when it says (see `poll_scan`)
let dir = lightcraft_engine::import::browse_spelling(std::path::Path::new(path)).map_err(|e| e.to_string())?;
if app.scan.as_ref().is_some_and(|t| !t.browse) || app.import.as_ref().is_some_and(|t| !t.browse) {
return Err("an import is running".into());
}
let dir_s = dir.to_string_lossy().trim_end_matches(['/', '\\']).to_string();
let subfolders = subfolders.unwrap_or_else(|| app.session.browse.as_ref().is_some_and(|b| b.subfolders));
let running = app.scan.as_ref().is_some_and(|t| t.browse) || app.import.as_ref().is_some_and(|t| t.browse);
if running && app.session.browse.as_ref().is_some_and(|b| b.path == dir_s && b.subfolders == subfolders) {
// (a path through `..` can name another folder than its spelling, past a link: only the worker can tell, so it
// browses again; CodeRabbit on #3583)
let through_dotdot = std::path::Path::new(path).components().any(|c| c == std::path::Component::ParentDir);
if running && !through_dotdot && app.session.browse.as_ref().is_some_and(|b| b.path == dir_s && b.subfolders == subfolders) {
// already reading this folder: clicking it again must not restart the progress
app.session.source = lightcraft_engine::LibrarySource::Folder;
app.session.keep_active_in_view();
return Ok(json!({"path": dir_s, "subfolders": subfolders, "scanning": true}));
}
if let Some(t) = app.scan.take() {
t.progress.cancel.store(true, Ordering::Relaxed);
}
stop_scan(app);
app.import = None;
app.session.browse = Some(lightcraft_engine::Browse { path: dir_s.clone(), subfolders });
app.session.source = lightcraft_engine::LibrarySource::Folder;
// the photos the library already knows from this folder show at once: keep or pick the active one
app.session.keep_active_in_view();
let (input, _) = ScanInput::new(&mut app.session, std::slice::from_ref(&dir_s));
let (mut input, _) = ScanInput::new(&mut app.session, std::slice::from_ref(&dir_s));
let progress = std::sync::Arc::new(ScanProgress::default());
let (tx, rx) = std::sync::mpsc::channel();
let p = progress.clone();
let root = dir_s.clone();
// the worker resolves the path as given: on Unix `link/..` is the folder above where the link points, which the
// spelling (`..` taken out by name) can't tell
let given = path.to_owned();
let sort = app.session.sort;
let (resolved_tx, resolved) = std::sync::mpsc::channel();
let job = move || {
let files: Vec<String> = if subfolders {
lightcraft_engine::import::expand(&[root], None)
} else {
let mut v: Vec<String> = std::fs::read_dir(&root)
.map(|rd| {
rd.flatten()
.filter(|e| !lightcraft_engine::walk::is_hidden_entry(e))
.map(|e| e.path())
.filter(|f| f.is_file() && lightcraft_engine::import::is_supported(f))
.map(|f| f.to_string_lossy().to_string())
.collect()
})
.unwrap_or_default();
v.sort();
v
let root = match resolve_browse_root(&given) {
Ok(root) => root,
Err(e) => {
log::warn!("browse: can't read {given}: {e}");
let _ = tx.send(ScanEnd::Stopped { probes: input.take_probes(), failed: true });
return;
}
};
let _ = tx.send(scan_with(input, &files, &p));
let _ = resolved_tx.send(root.clone());
// (never the library's own folder, which the import would otherwise read as photos)
let files = match lightcraft_engine::import::browse_files(&input, &root, subfolders) {
Ok(files) => files,
Err(e) => {
// no listing is sent: the scan ends as failed, and no record is taken for a file gone
log::warn!("browse: can't list {root}: {e}");
let _ = tx.send(ScanEnd::Stopped { probes: input.take_probes(), failed: true });
return;
}
};
// Nothing is read here: the import that follows reads the files and adds them batch by
// batch, first screen first, so the view fills while the rest is read (PERF-CULL-1).
// Another folder chosen meanwhile (`p.cancel`) stops the ordering, and no listing is sent.
let Some(queue) = lightcraft_engine::import::browse_queue_until(&input, files.clone(), &sort, &p.cancel) else {
let _ = tx.send(ScanEnd::Stopped { probes: input.take_probes(), failed: false });
return;
};
p.total.store(queue.len(), Ordering::Relaxed);
p.done.store(queue.len(), Ordering::Relaxed);
// the files the library knows come too, marked as such: the folder bookkeeping (browse
// times, records of files gone) needs the whole listing; only the queue is imported
let queued: std::collections::HashSet<&String> = queue.iter().collect();
let known: Vec<String> = files.iter().filter(|f| !queued.contains(f)).cloned().collect();
let known =
known.into_iter().map(|path| lightcraft_engine::import::ImportCandidate { path, duplicate: Some("path".into()), ..Default::default() });
let candidates =
queue.iter().cloned().map(|path| lightcraft_engine::import::ImportCandidate { path, ..Default::default() }).chain(known).collect();
// the probes of earlier scans go back to the session with the result, for the import that follows
let _ = tx.send(ScanEnd::Done(lightcraft_engine::import::ScanOutput { candidates, probes: input.take_probes() }));
};
#[cfg(not(target_arch = "wasm32"))]
std::thread::spawn(job);
// the job takes the probes back out of `input`, so it is an FnMut where it runs in place
#[cfg(target_arch = "wasm32")]
job();
{
let mut job = job;
job();
}
let guard = scan_guard(app, &progress, true);
app.scan = Some(ScanTask { progress, rx, copy: false, album: None, browse: true, sources: Vec::new(), guard });
app.scan = Some(ScanTask { progress, rx, copy: false, album: None, browse: true, sources: Vec::new(), guard, resolved: Some(resolved) });
app.renderer.forget_imports();
Ok(json!({"path": dir_s, "subfolders": subfolders, "scanning": true}))
}
/// The browsed folder `given` (the path as `browse` received it, not its [`lightcraft_engine::import::browse_spelling`])
/// as [`lightcraft_engine::import::browse_path`] resolves it, checked to be a folder: on the browse worker, as it reads the disk.
fn resolve_browse_root(given: &str) -> Result<String, String> {
#[cfg(test)]
held_resolve(given);
let dir = lightcraft_engine::import::browse_path(std::path::Path::new(given)).map_err(|e| e.to_string())?;
if !dir.is_dir() {
return Err(format!("{given}: not a folder"));
}
Ok(dir.to_string_lossy().trim_end_matches(['/', '\\']).to_string())
}
/// Tests: browsed folders whose resolution waits (as on a stalled network mount) while they are listed here.
#[cfg(test)]
pub(crate) static HELD_RESOLVES: std::sync::Mutex<Vec<String>> = std::sync::Mutex::new(Vec::new());
#[cfg(test)]
fn held_resolve(given: &str) {
let held = || HELD_RESOLVES.lock().unwrap_or_else(std::sync::PoisonError::into_inner).iter().any(|p| p == given);
let start = std::time::Instant::now();
while held() && start.elapsed() < std::time::Duration::from_secs(30) {
std::thread::sleep(std::time::Duration::from_millis(5));
}
}
/// Collect a finished scan and open the review (called every frame).
pub fn poll_scan(app: &mut LightcraftApp, ctx: &egui::Context) {
show_pending_review(app, ctx);
collect_stopped(app, ctx);
let Some(task) = app.scan.as_ref() else { return };
ctx.request_repaint_after(std::time::Duration::from_millis(100));
let out = match task.rx.try_recv() {
// the worker resolved the browsed folder: the view takes its spelling (before its listing, sent after it)
if let Some(root) = task.resolved.as_ref().and_then(|r| r.try_recv().ok())
&& let Some(b) = app.session.browse.as_mut().filter(|b| b.path != root)
{
b.path = root;
app.session.keep_active_in_view();
}
let Some(task) = app.scan.as_ref() else { return };
let end = match task.rx.try_recv() {
Ok(o) => o,
Err(std::sync::mpsc::TryRecvError::Empty) => {
if task.guard.is_cancelled() {
// ✕: the worker stops at its next file; don't wait for it (a NAS read can take a while)
app.scan = None;
stop_scan(app);
return;
}
let (total, done) = (task.progress.total.load(Ordering::Relaxed), task.progress.done.load(Ordering::Relaxed));
@@ -160,16 +309,32 @@ pub fn poll_scan(app: &mut LightcraftApp, ctx: &egui::Context) {
return;
}
Err(_) => {
// a cancelled scan whose worker ended without a result was not a failure
let cancelled = task.guard.is_cancelled() || task.progress.cancel.load(Ordering::Relaxed);
app.scan = None;
app.toast(ctx, crate::i18n::tr("Scan failed"));
if !cancelled {
app.toast(ctx, crate::i18n::tr("Scan failed"));
}
return;
}
};
let Some(task) = app.scan.take() else { return };
if task.progress.cancel.load(Ordering::Relaxed) {
let cancelled = task.progress.cancel.load(Ordering::Relaxed);
let out = match end {
ScanEnd::Done(out) => out,
ScanEnd::Stopped { probes, failed } => {
app.session.import_probes.extend(probes);
if failed && !cancelled {
app.toast(ctx, crate::i18n::tr("Scan failed"));
}
return;
}
};
// the probes go back to the session however the scan ended (the ones of a scan stopped earlier may be there)
app.session.import_probes.extend(out.probes);
if cancelled {
return;
}
app.session.import_probes = out.probes;
if task.browse {
// the folder and the ones its photos are in count as browsed now, as `library.browse` records them, so
// their untouched Local photos aren't forgotten while the folder is in use
@@ -247,11 +412,17 @@ pub(crate) fn stall_scan(app: &mut LightcraftApp) -> Box<dyn std::any::Any> {
/// Tests: the same, for a Local folder browse scan (`browse`) instead of the review's scan.
#[cfg(test)]
pub(crate) fn stall_scan_of(app: &mut LightcraftApp, browse: bool) -> Box<dyn std::any::Any> {
Box::new(scan_by_hand(app, browse))
}
/// Tests: a scan whose worker is the test: what it sends through the returned channel is the scan's end.
#[cfg(test)]
pub(crate) fn scan_by_hand(app: &mut LightcraftApp, browse: bool) -> std::sync::mpsc::Sender<ScanEnd> {
let progress = std::sync::Arc::new(ScanProgress::default());
let (tx, rx) = std::sync::mpsc::channel::<ScanOutput>();
let (tx, rx) = std::sync::mpsc::channel::<ScanEnd>();
let guard = scan_guard(app, &progress, false);
app.scan = Some(ScanTask { progress, rx, copy: false, album: None, browse, sources: Vec::new(), guard });
Box::new(tx)
app.scan = Some(ScanTask { progress, rx, copy: false, album: None, browse, sources: Vec::new(), guard, resolved: None });
tx
}
/// Whether an import or any scan (the review's, or a Local folder browse) is running, or a finished scan's
crates/ui-egui/src/dialogs/settings.rs
+2 -2
View File
@@ -57,8 +57,8 @@ pub const TABS: &[(&str, &str)] = &[
("denoise", "AI Denoise"),
];
/// Thumbnail cache sizes offered (MB).
const CACHE_SIZES: [u32; 5] = [512, 1024, 2048, 4096, 8192];
/// Preview cache sizes offered (MB; 0 = automatic: room for every photo of the library).
const CACHE_SIZES: [u32; 8] = [512, 1024, 2048, 4096, 8192, 51_200, 204_800, 0];
const LABEL_W: f32 = 150.0;
crates/ui-egui/src/dialogs/settings/performance.rs
+127 -4
View File
@@ -49,15 +49,30 @@ pub(super) fn performance_tab(app: &mut LightcraftApp, ui: &mut egui::Ui, t: &To
let opts = [(0u32, auto.as_str()), (512, "512 MB"), (1024, "1 GB"), (2048, "2 GB"), (4096, "4 GB")];
choices(ui, "settingsMemory", &opts, &mut app.ui.settings.memory_mb);
});
heading(ui, t, crate::i18n::tr("Thumbnail cache"));
let cur = (app.session.cache_bytes() >> 20) as u32;
heading(ui, t, crate::i18n::tr("Preview cache"));
let cur = app.session.cache_mb;
row(ui, t, crate::i18n::tr("Size limit"), |ui| {
let opts = [(512u32, "512 MB"), (1024, "1 GB"), (2048, "2 GB"), (4096, "4 GB"), (8192, "8 GB")];
let mut v = if CACHE_SIZES.contains(&cur) { cur } else { 2048 };
let gb = lightcraft_engine::media::default_disk_cache_bytes(app.session.catalog.len()).div_ceil(1 << 30);
let auto = format!("{} ({gb} GB)", crate::i18n::tr("Automatic"));
let custom = custom_cache_option(cur);
let mut opts =
vec![(512u32, "512 MB"), (1024, "1 GB"), (2048, "2 GB"), (4096, "4 GB"), (8192, "8 GB"), (51_200, "50 GB"), (204_800, "200 GB")];
opts.extend(custom.as_ref().map(|(mb, l)| (*mb, l.as_str())));
opts.push((0, auto.as_str()));
// (a saved size the list doesn't offer shows as its own option, not as Automatic: picking
// Automatic then changes it; CodeRabbit on #3583)
let mut v = if CACHE_SIZES.contains(&cur) || custom.is_some() { cur } else { 0 };
if choices(ui, "settingsCache", &opts, &mut v) {
let _ = app.run("library.preferences", json!({"cacheMb": v}));
}
});
hint(
ui,
t,
crate::i18n::tr(
"Grid thumbnails and the previews the loupe shows, kept on disk so photos open at once next time. Automatic makes room for every photo in this library (about 1.5 MB a photo). 50 GB: normal · 200 GB: better, for several weddings or large shoots.",
),
);
let used = app.session.media.rendered.disk().map(|d| d.size());
row(ui, t, crate::i18n::tr("In use"), |ui| {
ui.label(
@@ -70,6 +85,7 @@ pub(super) fn performance_tab(app: &mut LightcraftApp, ui: &mut egui::Ui, t: &To
let _ = app.run("library.clearPreviews", json!({}));
}
});
background_prep(app, ui, t);
heading(ui, t, crate::i18n::tr("Local folders"));
row(ui, t, crate::i18n::tr("Forget unchanged photos"), |ui| {
let opts = [(0u32, "Never"), (7, "After 7 days"), (30, "After 30 days"), (90, "After 90 days"), (365, "After a year")];
@@ -89,6 +105,92 @@ pub(super) fn performance_tab(app: &mut LightcraftApp, ui: &mut egui::Ui, t: &To
smart_previews(app, ui, t);
}
/// Settings ▸ Performance ▸ Prepare photos in the background (notes 47 and 48): per mode, what
/// is made ready before the user gets to it (`prefs.backgroundPrep`), app-wide or for this library.
fn background_prep(app: &mut LightcraftApp, ui: &mut egui::Ui, t: &Tokens) {
use lightcraft_engine::prep::{Mode, Scope};
heading(ui, t, crate::i18n::tr("Prepare photos in the background"));
hint(
ui,
t,
crate::i18n::tr(
"While you go through photos in the loupe, the next ones are made ready before you reach them, and when the computer is otherwise idle the previews of every photo in view are built. The photo on screen always comes first; the work shows in the activity list, where it can be stopped.",
),
);
let own = app.session.prep.library.is_some();
let lib_open = app.session.library.is_some();
if lib_open {
let mut v = own;
if check(ui, "settings.prepOwn", &mut v, "Use different settings for this library") {
let r = if v {
let cur = app.session.prep_settings();
app.run("prefs.setBackgroundPrep", json!({"library": true, "settings": cur}))
} else {
app.run("prefs.setBackgroundPrep", json!({"reset": true}))
};
if let Err(e) = r {
log::warn!("background preparation: {e}");
}
}
hint(ui, t, crate::i18n::tr("For example, a timelapse library can prepare nothing while a wedding library prepares everything."));
}
let cur = app.session.prep_settings();
let mut patch: Option<Value> = None;
let scopes = [(Scope::All, "Every photo"), (Scope::Marked, "Picks, rated and labelled"), (Scope::OnScreen, "Only the photo on screen")];
for (mode, key, title) in [(Mode::Culling, "culling", "Culling (a folder before import)"), (Mode::Library, "library", "Library")] {
let m = cur.mode(mode);
ui.add_space(4.0);
ui.label(RichText::new(crate::i18n::tr(title)).font(t.semibold(12.0)).color(t.text));
let mut previews = m.previews;
row(ui, t, crate::i18n::tr("Previews"), |ui| {
if check(ui, &format!("settings.prep.{key}.previews"), &mut previews, "Build previews for every photo in view") {
patch = Some(json!({key: {"previews": previews}}));
}
});
hint(ui, t, crate::i18n::tr("Starts once you step through a few photos. Uses about 1.5 MB of disk a photo, and the computer's spare time."));
let mut ahead = m.ahead;
row(ui, t, crate::i18n::tr("Photos ahead"), |ui| {
let opts = [(0u8, "Off"), (3, "3"), (5, "5"), (10, "10"), (20, "20")];
if choices(ui, &format!("settingsPrepAhead-{key}"), &opts, &mut ahead) {
patch = Some(json!({key: {"ahead": ahead}}));
}
});
let mut scope = m.scope;
row(ui, t, crate::i18n::tr("Which photos"), |ui| {
if choices(ui, &format!("settingsPrepScope-{key}"), &scopes, &mut scope) {
patch = Some(json!({key: {"scope": scope}}));
}
});
}
ui.add_space(4.0);
ui.label(RichText::new(crate::i18n::tr("Editing")).font(t.semibold(12.0)).color(t.text));
let mut ahead = cur.develop.ahead;
row(ui, t, crate::i18n::tr("Photos ahead"), |ui| {
let opts = [(0u8, "Off"), (1, "1"), (2, "2"), (3, "3"), (5, "5")];
if choices(ui, "settingsPrepAhead-develop", &opts, &mut ahead) {
patch = Some(json!({"develop": {"ahead": ahead}}));
}
});
hint(
ui,
t,
crate::i18n::tr(
"The next photos are opened and kept ready to edit, so the first slider move on them is quick. Uses memory: a few hundred MB a photo.",
),
);
let mut scope = cur.develop.scope;
row(ui, t, crate::i18n::tr("Which photos"), |ui| {
if choices(ui, "settingsPrepScope-develop", &scopes, &mut scope) {
patch = Some(json!({"develop": {"scope": scope}}));
}
});
if let Some(patch) = patch
&& let Err(e) = app.run("prefs.setBackgroundPrep", json!({"library": own, "settings": patch}))
{
log::warn!("background preparation: {e}");
}
}
/// Where this library keeps its smart previews (the offline-editing proxies, which can be large):
/// the effective folder, what is in it, and choosing another one.
#[cfg(not(target_arch = "wasm32"))]
@@ -193,3 +295,24 @@ fn smart_previews(app: &mut LightcraftApp, ui: &mut egui::Ui, t: &Tokens) {
}
hint(ui, t, crate::i18n::tr("Keep it on a drive with room: smart previews are about 1 MB per photo. The thumbnail cache stays in the library."));
}
/// The cache size limit saved in the settings, as an option of its own when the list doesn't offer
/// it (set through `library.preferences`, or by an older version): its value and label.
fn custom_cache_option(cur: u32) -> Option<(u32, String)> {
if cur == 0 || CACHE_SIZES.contains(&cur) {
return None;
}
let label = if cur.is_multiple_of(1024) { format!("{} GB", cur / 1024) } else { format!("{cur} MB") };
Some((cur, label))
}
#[cfg(test)]
mod tests {
#[test]
fn a_saved_cache_size_the_list_does_not_offer_is_an_option() {
assert_eq!(super::custom_cache_option(0), None, "Automatic");
assert_eq!(super::custom_cache_option(51_200), None, "offered");
assert_eq!(super::custom_cache_option(10_240), Some((10_240, "10 GB".to_string())));
assert_eq!(super::custom_cache_option(3000), Some((3000, "3000 MB".to_string())));
}
}
crates/ui-egui/src/headless/tests.rs
+140
View File
@@ -527,6 +527,10 @@ fn demo_grid_snapshot_has_ui_pixels() {
assert!(bright > 30, "no text in the top bar ({bright} bright px)");
// thumbnails arrived (photo textures were drawn)
assert!(h.app.renderer.thumb_textures() > 0);
// and are counted for benchmarks, a count that eviction doesn't lower
let r = h.request("ui.inspect", json!({}), SETTLE);
let made = r["result"]["perf"]["thumbsMade"].as_u64().unwrap_or(0);
assert!(made >= h.app.renderer.thumb_textures() as u64, "{}", r["result"]["perf"]);
}
#[test]
@@ -3416,6 +3420,15 @@ fn browsing_from_the_ui_records_the_folder() {
assert_eq!(h.app.session.visible_cloned().len(), 1);
assert!(h.app.session.catalog.last_browsed(&root).is_some(), "the browsed folder");
assert!(h.app.session.catalog.last_browsed(&day_s).is_some(), "the folder its photo is in");
// browsed again: a subfolder with only photos the library knows (nothing to import) still
// counts as browsed
h.app.session.catalog.apply(lightcraft_catalog::Op::SetBrowsed { folder: day_s.clone(), at: None }).unwrap();
assert!(h.app.session.catalog.last_browsed(&day_s).is_none());
let r = h.request("engine.execute", json!({"command": "library.browse", "params": {"path": root, "subfolders": true}}), t);
assert_eq!(r["result"]["scanning"], true, "{r}");
h.settle(SETTLE);
assert_eq!(h.app.session.visible_cloned().len(), 1);
assert!(h.app.session.catalog.last_browsed(&day_s).is_some(), "a folder of known photos only");
let _ = std::fs::remove_dir_all(&dir);
}
@@ -5057,6 +5070,133 @@ fn screenshot_request_is_answered_without_a_window() {
assert_eq!(r["result"]["width"], 640, "{r}");
}
/// Note 47: stepping in the loupe to a photo whose preview was built shows that preview as the
/// photo (fitted to the loupe, no render), and standard previews are built at the loupe's size.
#[test]
fn the_loupe_shows_built_previews_without_rendering_again() {
use crate::render::Slot;
use lightcraft_engine::media::QuickSource;
let mut h = demo([1000.0, 700.0]);
let t = Duration::from_secs(10);
// nothing prepared ahead here: what the loupe shows comes from the build alone
h.request(
"engine.execute",
json!({"command": "prefs.setBackgroundPrep", "params": {"settings": {"library": {"ahead": 0, "previews": false}}}}),
t,
);
h.request("ui.set", json!({"view": "detail", "right": "none"}), t);
h.settle(SETTLE);
let edge = h.app.session.loupe_edge;
assert!(edge > 100, "the loupe said its size: {edge}");
let ids = h.app.session.visible_cloned();
let all: Vec<u64> = ids.iter().map(|i| i.0).collect();
let r = h.request("engine.execute", json!({"command": "library.buildPreviews", "params": {"wait": true, "edge": edge, "ids": all}}), t);
assert_eq!(r["result"]["done"], ids.len(), "{r}");
let next = ids.get(1).copied().expect("a second photo");
h.request("engine.execute", json!({"command": "library.select", "params": {"ids": [next.0], "active": next.0}}), t);
assert!(h.step_until(SETTLE, |h| h.app.renderer.textures.get(&Slot::Main).is_some_and(|t| t.photo == next)));
let main = h.app.renderer.textures.get(&Slot::Main).unwrap();
assert_eq!(main.quick, Some(QuickSource::Cached), "the built preview is the loupe's picture");
let r = h.request("ui.inspect", json!({}), t);
assert!(r["result"]["perf"]["previewCache"]["memEntries"].as_u64().unwrap_or(0) > 0, "{}", r["result"]["perf"]["previewCache"]);
}
/// Editing, the loupe renders the photo from its source even when a built preview of it is cached
/// (CodeRabbit on #3077): the cached view would show at once but leave the source undecoded, and
/// the first slider draft would wait for it.
#[test]
fn editing_renders_the_loupe_from_its_source_not_a_built_preview() {
use crate::render::Slot;
use lightcraft_engine::media::QuickSource;
let mut h = demo([1000.0, 700.0]);
let t = Duration::from_secs(10);
h.request(
"engine.execute",
json!({"command": "prefs.setBackgroundPrep", "params": {"settings": {"library": {"ahead": 0, "previews": false}, "develop": {"ahead": 0}}}}),
t,
);
h.request("ui.set", json!({"view": "detail", "right": "none"}), t);
h.settle(SETTLE);
let edge = h.app.session.loupe_edge;
let ids = h.app.session.visible_cloned();
let all: Vec<u64> = ids.iter().map(|i| i.0).collect();
let r = h.request("engine.execute", json!({"command": "library.buildPreviews", "params": {"wait": true, "edge": edge, "ids": all}}), t);
assert_eq!(r["result"]["done"], ids.len(), "{r}");
h.request("ui.set", json!({"right": "edit"}), t);
let next = ids.get(1).copied().expect("a second photo");
h.request("engine.execute", json!({"command": "library.select", "params": {"ids": [next.0], "active": next.0}}), t);
assert!(h.step_until(SETTLE, |h| h.app.renderer.textures.get(&Slot::Main).is_some_and(|t| t.photo == next)));
let main = h.app.renderer.textures.get(&Slot::Main).unwrap();
assert_ne!(main.quick, Some(QuickSource::Cached), "rendered from the source while editing");
}
/// The loupe holds background preparation back only while its render is on the way (CodeRabbit on
/// #3077): once the photo is shown, the frames that follow don't renew the hold.
#[test]
fn a_shown_loupe_render_does_not_hold_background_preparation() {
use crate::render::Slot;
let mut h = demo([1000.0, 700.0]);
let t = Duration::from_secs(10);
h.request("ui.set", json!({"view": "detail", "right": "none"}), t);
let id = h.app.session.active().expect("an active photo");
let signals = || crate::panels::detail::FOREGROUND_SIGNALS.with(std::cell::Cell::get);
let before = signals();
assert!(
h.step_until(SETTLE, |h| h.app.renderer.textures.get(&Slot::Main).is_some_and(|t| t.photo == id) && !h.app.renderer.is_pending(Slot::Main))
);
assert!(signals() > before, "the hold was asked for while the render was on its way");
let shown = signals();
for _ in 0..10 {
h.step();
}
assert_eq!(signals(), shown, "no hold while the render is on screen");
}
/// The app-wide background preparation settings are kept with the app settings (`ui.json`):
/// the saved ones reach the session at launch, and a change made through the command is saved.
#[test]
fn app_wide_preparation_settings_are_saved_with_the_app_settings() {
let mut h = demo([1000.0, 700.0]);
let t = Duration::from_secs(10);
let mut saved = lightcraft_engine::prep::PrepSettings::default();
saved.library.ahead = 3;
// (as if read from ui.json before the first frame)
h.app.ui.settings.background_prep = saved;
h.app.prep_synced = false;
h.step();
assert_eq!(h.app.session.prep.app, saved, "the saved settings reach the session");
h.request("engine.execute", json!({"command": "prefs.setBackgroundPrep", "params": {"settings": {"library": {"ahead": 10}}}}), t);
h.step();
assert_eq!(h.app.ui.settings.background_prep.library.ahead, 10, "a change is kept with the app settings");
assert_eq!(h.app.session.prep.app.library.ahead, 10);
}
/// Note 48: the loupe prepares as many photos ahead as the settings say, a few at a time, and
/// none when that is off.
#[test]
fn the_loupe_prepares_the_photos_ahead_the_settings_ask_for() {
let mut h = demo([1000.0, 700.0]);
let t = Duration::from_secs(10);
let n = h.app.session.visible_cloned().len();
assert!(n >= 4, "the demo has photos to step through");
h.request("engine.execute", json!({"command": "prefs.setBackgroundPrep", "params": {"settings": {"library": {"ahead": 3}}}}), t);
h.request("ui.set", json!({"view": "detail", "right": "none"}), t);
h.settle(SETTLE);
for _ in 0..40 {
h.step();
}
h.settle(SETTLE);
// the first photo: three ahead (no photo behind it)
assert_eq!(h.app.renderer.prefetch_slots_asked(), 3.min(n - 1));
h.app.renderer.forget_all();
h.request("engine.execute", json!({"command": "prefs.setBackgroundPrep", "params": {"settings": {"library": {"ahead": 0}}}}), t);
for _ in 0..40 {
h.step();
}
h.settle(SETTLE);
assert_eq!(h.app.renderer.prefetch_slots_asked(), 0, "off: nothing prepared ahead");
}
/// A size typed in the Export dialog is the size exported: the pixel fields kept only multiples
/// of 16, so 1080 became 1088 and 3000 became 3008 (issue #3300).
#[test]
crates/ui-egui/src/lib.rs
+23
View File
@@ -33,6 +33,7 @@ pub mod icons;
pub mod lens_db;
pub mod lightroom_import;
pub mod links;
pub mod loupe_timing;
pub mod menus;
mod missing_sweep;
mod model_setup;
@@ -223,6 +224,9 @@ pub struct LightcraftApp {
/// effects (which the global shortcut guard also hides).
#[cfg(test)]
pub(crate) library_passes: Vec<panels::library_problem::LibraryPass>,
/// Test-only: how many times background preparation was advanced (once per logic pass).
#[cfg(test)]
pub(crate) prep_ticks: u32,
/// Test-only: a delay for the sidebar's Missing Photos count worker (a slow disk). Kept here,
/// not in `ui`, because `ui.set` rebuilds `ui` and would drop it.
#[cfg(test)]
@@ -289,6 +293,14 @@ pub struct LightcraftApp {
/// What the loupe drew last frame: photo and source ("render", "cached", "embedded", "small",
/// "thumb", "none").
pub loupe_shown: Option<(lightcraft_catalog::PhotoId, &'static str)>,
/// When each photo stepped to in the loupe first showed (`ui.inspect` → `perf.loupeSteps`).
pub loupe_timing: loupe_timing::LoupeTiming,
/// The app-wide background preparation settings were handed from the saved app settings to
/// the session ([`Self::background_prep`]).
prep_synced: bool,
/// The loupe's look-ahead: the photo it was made for, the direction of travel (+1 / -1), the
/// catalog revision and the edge, so it is worked out again only when one of them changes.
pub(crate) look_ahead: panels::detail::LookAhead,
/// The window render the loupe asked for last, by job key (see [`region`]); kept for the few
/// windows whose textures can be on screen, and for the inspector.
pub region_view: Option<region::RegionView>,
@@ -314,6 +326,9 @@ pub struct LightcraftApp {
pub import: Option<import::ImportTask>,
/// A folder scan in progress (feeds the import review).
pub scan: Option<import::ScanTask>,
/// Scans stopped without waiting for their workers: each channel is kept until its worker ends, so
/// the probes it took from the session come back.
pub(crate) scans_stopping: import::StoppingScans,
/// Files that waited for a library and wait for the running import or Synchronize Folder too.
pub waiting_import: panels::library_problem::WaitingImport,
/// A Synchronize Folder scan in progress (feeds its dialog).
@@ -350,6 +365,14 @@ pub struct LightcraftApp {
/// Say once, when the GPU device has been tried, that no graphics processor was found
/// ([`panels::notices::gpu_notice`]). The desktop app sets it; headless sessions and tests don't.
pub gpu_notice: bool,
/// Tell the user once per session when the drive holding the preview cache runs low
/// ([`panels::notices::disk_space_notice`]). The desktop app sets it; headless sessions and tests
/// don't (their drive is whatever the machine has).
pub disk_space_notice: bool,
/// The low-disk notice's text while it is queued in [`LightcraftApp::notices`]: it is shown with a
/// button to Settings ▸ Performance. With the library it was raised for
/// ([`lightcraft_engine::Session::library_generation`]): another library open drops it.
pub(crate) disk_notice: Option<(u64, String)>,
/// The memory budget setting last applied (MB, 0 = automatic).
memory_applied: Option<u32>,
/// The display profile setting last applied (`app.displayProfile`).
crates/ui-egui/src/loupe_timing.rs
+269
View File
@@ -0,0 +1,269 @@
//! How long the loupe takes to show each photo it is stepped to, measured on the frames it draws.
//!
//! Culling is stepping through photos, so the number that matters is the time from the step (the
//! first frame that draws the new photo) to the first frame that shows something of it, to the
//! first frame with a full-size picture (the camera's embedded JPEG or a cached render) and to the
//! first frame with LightCraft's own render. `ui.inspect` → `perf.loupeSteps` reports the latest
//! steps; `perf-scratch`-style drivers turn them into medians and p95 without the control
//! channel's own round trip in the numbers.
use std::collections::VecDeque;
use lightcraft_catalog::PhotoId;
use serde_json::{Value, json};
/// Steps kept for `ui.inspect`.
const KEEP: usize = 256;
/// A photo left sooner than this (s) after it was stepped to was skipped past: the user is
/// holding an arrow key or tapping through.
pub const FAST_STEP_SECS: f64 = 0.25;
/// While stepping fast, how long a photo has to stay before its render is started (s). Until then
/// the loupe shows its camera JPEG or thumbnail, and the workers make the next ones' instead of
/// starting a full raw develop that the next step throws away.
pub const SETTLE_SECS: f64 = 0.12;
/// One photo's way onto the screen (times in ms since the frame it was stepped to).
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct StepTiming {
pub photo: PhotoId,
/// First frame that drew anything of the photo (often the grid thumbnail, magnified).
pub any_ms: Option<f64>,
/// What that was.
pub any_source: &'static str,
/// First frame with a full-size stand-in or better: the camera's JPEG, a cached render, the render.
pub full_ms: Option<f64>,
pub full_source: &'static str,
/// First frame with LightCraft's own render of the photo (`render` or `cached`).
pub sharp_ms: Option<f64>,
/// How long the photo stayed in the loupe (ms): a step taken before `sharp_ms` was skipped past.
pub dwell_ms: f64,
}
impl StepTiming {
fn to_json(self) -> Value {
json!({
"photo": self.photo.0,
"anyMs": self.any_ms,
"anySource": self.any_source,
"fullMs": self.full_ms,
"fullSource": self.full_source,
"sharpMs": self.sharp_ms,
"dwellMs": self.dwell_ms,
})
}
}
/// The loupe's step timings.
#[derive(Debug, Default)]
pub struct LoupeTiming {
current: Option<(f64, StepTiming)>,
done: VecDeque<StepTiming>,
/// Steps finished since the app started (never lowered when old ones leave `done`): a driver's
/// cursor for the steps made after it looked.
total: u64,
/// Whether the loupe drew a photo in the frame being drawn ([`LoupeTiming::end_frame`]).
drawn: bool,
/// How long the photo before the current one stayed (ms), when the loupe stepped to it from
/// that photo; `None` when the loupe opened on it (from the grid, or coming back to the loupe).
stepped_from: Option<f64>,
}
/// Sources that show the photo at full size: the camera's JPEG, a cached or a fresh render.
fn full_size(source: &str) -> bool {
matches!(source, "embedded" | "cached" | "render" | "preview")
}
impl LoupeTiming {
/// The loupe drew `photo` from `source` (what [`crate::panels::detail`] reports in
/// `loupe_shown`) in the frame that started at `now` (s).
pub fn observe(&mut self, photo: PhotoId, source: &'static str, now: f64) {
self.drawn = true;
if self.current.as_ref().is_none_or(|(_, t)| t.photo != photo) {
let stepped = self.current.is_some();
self.finish(now);
self.stepped_from = if stepped { self.done.back().map(|t| t.dwell_ms) } else { None };
let t = StepTiming { photo, any_ms: None, any_source: "none", full_ms: None, full_source: "none", sharp_ms: None, dwell_ms: 0.0 };
self.current = Some((now, t));
}
let Some((since, t)) = self.current.as_mut() else { return };
let ms = ((now - *since) * 1000.0).max(0.0);
if !ms.is_finite() {
return;
}
t.dwell_ms = ms;
if source != "none" && t.any_ms.is_none() {
t.any_ms = Some(ms);
t.any_source = source;
}
if full_size(source) && t.full_ms.is_none() {
t.full_ms = Some(ms);
t.full_source = source;
}
if matches!(source, "render" | "cached") && t.sharp_ms.is_none() {
t.sharp_ms = Some(ms);
}
}
/// Whether the user is stepping fast and `photo` (being drawn in the frame at `now`) has not
/// stayed [`SETTLE_SECS`] yet: the photo before it was left within [`FAST_STEP_SECS`].
pub fn settling(&self, photo: PhotoId, now: f64) -> bool {
let (since, prev_dwell) = match &self.current {
// the first frame of a step: `observe` has not seen this photo yet
Some((since, t)) if t.photo != photo => (now, (now - since) * 1000.0),
// (a photo the loupe opened on, not stepped to, never settles: the step that ended by
// leaving the loupe is not its predecessor)
Some((since, _)) => (*since, self.stepped_from.unwrap_or(f64::INFINITY)),
None => return false,
};
prev_dwell.is_finite() && prev_dwell < FAST_STEP_SECS * 1000.0 && now - since < SETTLE_SECS
}
/// The loupe is not showing a photo any more (the grid is up).
pub fn left(&mut self, now: f64) {
self.finish(now);
}
/// A frame (started at `now`) was drawn: when the loupe drew no photo in it (the grid, Compare, Survey,
/// Reference, People or the slideshow is up), the photo it showed was left then, so time away from the
/// loupe never counts as time on that photo.
pub fn end_frame(&mut self, now: f64) {
if !std::mem::take(&mut self.drawn) {
self.left(now);
}
}
fn finish(&mut self, now: f64) {
if let Some((since, mut t)) = self.current.take() {
let dwell = (now - since) * 1000.0;
if dwell.is_finite() {
t.dwell_ms = dwell.max(t.dwell_ms);
}
if self.done.len() >= KEEP {
self.done.pop_front();
}
self.done.push_back(t);
self.total = self.total.saturating_add(1);
}
}
/// The finished steps, oldest first (the latest [`KEEP`]), how many have finished in all, and the photo on screen now.
pub fn to_json(&self) -> Value {
json!({
"steps": self.done.iter().map(|t| t.to_json()).collect::<Vec<_>>(),
"total": self.total,
"current": self.current.as_ref().map(|(_, t)| t.to_json()),
})
}
#[cfg(test)]
fn steps(&self) -> Vec<StepTiming> {
self.done.iter().copied().collect()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_step_records_when_each_kind_of_picture_first_showed() {
let mut l = LoupeTiming::default();
l.observe(PhotoId(1), "none", 10.0);
l.observe(PhotoId(1), "thumb", 10.016);
l.observe(PhotoId(1), "embedded", 10.040);
l.observe(PhotoId(1), "embedded", 10.056);
l.observe(PhotoId(1), "render", 11.0);
l.observe(PhotoId(2), "thumb", 12.0);
let s = l.steps();
assert_eq!(s.len(), 1);
let t = s[0];
assert_eq!(t.photo, PhotoId(1));
assert_eq!((t.any_source, t.full_source), ("thumb", "embedded"));
assert!((t.any_ms.unwrap() - 16.0).abs() < 0.01);
assert!((t.full_ms.unwrap() - 40.0).abs() < 0.01);
assert!((t.sharp_ms.unwrap() - 1000.0).abs() < 0.01);
assert!((t.dwell_ms - 2000.0).abs() < 0.01);
assert_eq!(l.to_json()["current"]["photo"], 2);
}
#[test]
fn stepping_fast_holds_the_render_back_until_the_photo_stays() {
let mut l = LoupeTiming::default();
// the first photo opened: no step before it
assert!(!l.settling(PhotoId(1), 0.0));
l.observe(PhotoId(1), "embedded", 0.0);
// a deliberate step (the photo before stayed 2 s): render at once
assert!(!l.settling(PhotoId(2), 2.0));
l.observe(PhotoId(2), "embedded", 2.0);
// held arrow: the photo before stayed 33 ms
assert!(l.settling(PhotoId(3), 2.033));
l.observe(PhotoId(3), "thumb", 2.033);
assert!(l.settling(PhotoId(3), 2.1));
// …until this one has stayed long enough
assert!(!l.settling(PhotoId(3), 2.033 + SETTLE_SECS + 0.001));
}
/// Leaving the loupe soon after a step and coming back on another photo is not a fast step
/// (CodeRabbit on #3077): the photo the loupe opens on renders at once, on every frame.
#[test]
fn coming_back_to_the_loupe_on_another_photo_is_not_a_fast_step() {
let mut l = LoupeTiming::default();
l.observe(PhotoId(1), "embedded", 0.0);
l.end_frame(0.0);
// the grid is up after 100 ms on the photo
l.end_frame(0.1);
assert!(!l.settling(PhotoId(2), 0.2), "the first frame back");
l.observe(PhotoId(2), "thumb", 0.2);
l.end_frame(0.2);
assert!(!l.settling(PhotoId(2), 0.25), "nor the frames after it");
// a fast step from there still settles
l.observe(PhotoId(2), "embedded", 0.25);
assert!(l.settling(PhotoId(3), 0.3));
l.observe(PhotoId(3), "thumb", 0.3);
assert!(l.settling(PhotoId(3), 0.35));
}
/// A frame without the loupe finishes the photo it showed: coming back to it later starts a new step, and
/// neither counts the time away (CodeRabbit on #3055).
#[test]
fn leaving_the_loupe_finishes_its_photo() {
let mut l = LoupeTiming::default();
l.observe(PhotoId(1), "embedded", 10.0);
l.end_frame(10.0);
l.observe(PhotoId(1), "embedded", 10.5);
l.end_frame(10.5);
assert!(l.steps().is_empty(), "frames that draw the loupe keep the step open");
// the grid is up from 11 s to 60 s
l.end_frame(11.0);
l.end_frame(60.0);
let s = l.steps();
assert_eq!(s.len(), 1);
assert!((s[0].dwell_ms - 1000.0).abs() < 0.01, "dwell ends when the loupe went: {}", s[0].dwell_ms);
// back in the loupe on the same photo: a new step, timed from its return
l.observe(PhotoId(1), "render", 60.0);
l.observe(PhotoId(1), "render", 60.2);
l.end_frame(60.2);
assert_eq!(l.steps().len(), 1);
assert_eq!(l.to_json()["current"]["sharpMs"], 0.0);
assert!((l.to_json()["current"]["dwellMs"].as_f64().unwrap() - 200.0).abs() < 0.01);
}
#[test]
fn a_photo_stepped_past_has_no_sharp_time_and_the_list_is_bounded() {
let mut l = LoupeTiming::default();
for i in 0..(KEEP as u64 + 10) {
l.observe(PhotoId(i), "thumb", i as f64 * 0.033);
}
l.left(1000.0);
let s = l.steps();
assert_eq!(s.len(), KEEP);
// the count of steps goes on past the kept list: a driver's cursor (CodeRabbit on #3077)
assert_eq!(l.to_json()["total"], KEEP as u64 + 10);
assert!(s.iter().all(|t| t.sharp_ms.is_none() && t.full_ms.is_none()));
// a non-finite clock never poisons the record
l.observe(PhotoId(9999), "render", f64::NAN);
l.observe(PhotoId(9999), "render", f64::INFINITY);
assert!(l.to_json().is_object());
}
}
crates/ui-egui/src/menus/menubar.rs
+17 -3
View File
@@ -465,17 +465,31 @@ fn live_label(app: &LightcraftApp, id: &str, label: &str) -> String {
}
/// The parameterized submenus.
/// The params of Build Standard-Sized Previews: automatic builds as big as the loupe on this screen shows photos
/// (the engine's standard size, once a loupe has been drawn); before that, in a fresh session, the standard
/// 2560 px, not the engine's 2048 px minimum, so a typical fit view needn't render again; a chosen limit is used as is.
pub(crate) fn standard_build_params(settings: &crate::state::AppSettings, loupe_edge: usize) -> Value {
match settings.preview_limit {
0 if loupe_edge == 0 => json!({"size": "standard", "edge": settings.standard_preview_edge()}),
0 => json!({"size": "standard"}),
edge => json!({"size": "standard", "edge": edge}),
}
}
pub(crate) fn expanded(app: &LightcraftApp, name: &str) -> Option<Vec<MenuNode>> {
let active = app.session.active().and_then(|id| app.session.catalog.photo(id).cloned());
let has = active.is_some();
Some(match name {
"Previews" => {
let running = app.session.preview_build.as_ref().is_some_and(|b| !b.finished.load(std::sync::atomic::Ordering::Relaxed));
// (the background preparation's build gives way to one asked for here)
let running = app.session.preview_build.as_ref().is_some_and(|b| b.running() && !b.background);
let stoppable = app.session.preview_build.as_ref().is_some_and(|b| b.running());
let standard = standard_build_params(&app.ui.settings, app.session.loupe_edge);
let scope = if app.session.selection.ids.len() > 1 { "Selected" } else { "Visible" };
vec![
item(
"library.buildPreviews",
json!({"size": "standard", "edge": app.ui.settings.standard_preview_edge()}),
standard,
crate::i18n::tr_format!("Build Standard-Sized Previews ({scope})", scope = crate::i18n::tr(scope)),
None,
!running,
@@ -489,7 +503,7 @@ pub(crate) fn expanded(app: &LightcraftApp, name: &str) -> Option<Vec<MenuNode>>
!running,
None,
),
item("library.cancelPreviews", Value::Null, "Stop Building Previews", None, running, None),
item("library.cancelPreviews", Value::Null, "Stop Building Previews", None, stoppable, None),
MenuNode::Separator,
// (read and written on a worker thread: the originals may be on a slow drive)
item(
crates/ui-egui/src/panels/detail.rs
+281 -24
View File
@@ -18,6 +18,7 @@ mod crop;
mod filmstrip;
mod geometry;
mod info;
mod look_ahead;
mod masks;
mod retouch;
mod window;
@@ -41,11 +42,15 @@ use geometry::{animated_rect, middle_pan};
pub(crate) use geometry::{fit_rect, fit_texture_rect, focus_pan, navigate_gesture, output_px, pan_image, snap_fit_rect, texture_side};
#[cfg(test)]
use info::RENDERING_AFTER_SECS;
use info::camera_preview_badge;
use info::{
FILTER_PILL_HEIGHT, FILTER_PILL_INSET, brief_info, filter_pill, filter_pill_shown, info_overlay, info_overlay_extent, label_badge_rect,
preview_only_pill, quick_name,
};
pub(crate) use info::{draw_focus_point, failed_pill, loupe_pill, rendering_overdue};
use look_ahead::BEHIND;
pub use look_ahead::LookAhead;
pub(crate) use look_ahead::{Prepare, look_ahead};
use masks::{RegionContext, RegionEdit, color_range_pick, mask_overlay, region_overlay};
use retouch::{eye_overlay, remove_overlay};
pub(crate) use window::overlay_supports_window;
@@ -229,11 +234,12 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui) {
// one axis only. A box that matches the source to rounding is rendered at exactly its size.
let short = |e: usize, a: f32| ((e as f32 / a).round() as usize).max(1);
let (rw, rh) = if aspect >= 1.0 { (want, short(want, aspect)) } else { (short(want, 1.0 / aspect), want) };
note_loupe_edge(app, sizes.canvas_long, !interacting && !split && !split_view);
// hover and before renders are stand-ins: the preview size is plenty and keeps them cheap
let se = app.ui.settings.stand_in_edge(want, texture_side);
let (sw, sh) = if aspect >= 1.0 { (se, short(se, aspect)) } else { (short(se, 1.0 / aspect), se) };
let ask = LoupeAsk { id, with_crop: !crop_tool, interacting, fit_px, overlay };
request_main(app, ui, &ask, [rw, rh], plan.window_edge.is_none());
let settling = request_main_or_settle(app, ui, &ask, [rw, rh], plan.window_edge.is_none(), now);
// hovering a preset or profile: the photo with that look, shown instead of the loupe render
// once it is ready (nothing is committed)
let hover_key = request_hover(app, &ask, [sw, sh]);
@@ -242,8 +248,14 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui) {
}
// until the loupe has this photo: show its cached render / embedded preview / a thumbnail
request_stand_in(app, id, want, !crop_tool);
if !interacting {
// (what Settings ▸ Performance lets this mode prepare ahead)
let prepare = prepare_ahead(app);
look_ahead(app, id, want.max(8), !crop_tool, prepare);
}
let show_before = app.ui.before_after == BeforeAfter::Original || ui.input(|i| i.key_down(egui::Key::Backslash));
if split || show_before || split_view {
// (the Before render waits for a settled photo like the loupe's own)
if !settling && (split || show_before || split_view) {
request_before(app, &ask, [sw, sh]);
}
// zoomed past what the whole-frame render holds: also render the part on screen, at no more
@@ -263,13 +275,16 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui) {
look,
holding: navigating || app.size_hold.holding(now),
};
let window_views = if window_ctx.frame_edge.is_some() {
// (stepping fast: no window is asked for until the photo stays, as for the frame above)
let window_views = if window_ctx.frame_edge.is_some() && !settling {
let mode = ViewMode { split, show_before, split_view };
window_views(mode, &Views { img_rect, target_rect, before_img, before_target, areas: &areas, main_area, canvas })
} else {
Vec::new()
};
let windows = request_windows(app, &window_ctx, window_views);
let single = hover_key.is_none() && !split && !show_before && !split_view && !interacting;
let camera_full = request_camera_full(app, &photo, single && !windows.is_empty(), native_long.min(texture_side));
let p = ui.painter_at(canvas);
let draw = |slot: Slot, r: Rect, clip: Rect| draw_slot(app, &p, id, ppp, slot, r, clip);
// soft proofing: a paper-white surround and the proof's name, as Lightroom shows it
@@ -306,16 +321,21 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui) {
none_message(app, &p, &t, canvas, id);
}
}
app.loupe_shown = Some((id, shown));
// a wipe shows the Before whole-frame render on its side of the line…
let line = crate::region::wipe_line(img_rect, canvas);
draw_wipe_before(app, &p, id, img_rect, line);
let camera_full_drawn = camera_full && draw_camera_full(app, &p, id, img_rect, display_aspect, window_cover(&windows));
// …and the window renders go over the whole-frame ones, all of them, where they belong in the
// frame (clipped to their pane, or their side of the line)
for (v, view) in &windows {
// (holding `\` in a wipe shows the Before everywhere: it is the Before view alone then)
let mode = if show_before { BeforeAfter::Original } else { app.ui.before_after };
let clip = crate::region::window_clip(mode, v.before, v.visible, canvas, img_rect);
draw_window(&p.with_clip_rect(clip), app, &window_ctx, v, view);
let drawn = draw_windows(app, &p, &window_ctx, &windows, [canvas, img_rect], (show_before, split));
let shown = shown_after_windows(shown, camera_full_drawn, drawn);
app.loupe_shown = Some((id, shown));
app.loupe_timing.observe(id, shown, now);
// the camera's own JPEG is on screen, not LightCraft's rendering of the raw: say so (once), so
// colour and tone are not judged on it (PERF-CULL-4; Lightroom Classic badges its embedded
// previews). Not over a window render, which is LightCraft's own.
if camera_jpeg_shown(app, &photo, shown, drawn) {
camera_preview_badge(ui, canvas, img_rect.intersect(canvas));
}
if let Some((bg, g)) = before_badge {
paint_badge(&p, bg, g);
@@ -462,21 +482,69 @@ fn prefetch_neighbours(app: &mut LightcraftApp, id: PhotoId, main_area: Rect, pp
return;
}
let ids = app.session.visible_cloned();
if let Some(i) = ids.iter().position(|p| *p == id) {
let next = ids.get(i + 1).copied();
let prev = i.checked_sub(1).and_then(|j| ids.get(j)).copied();
for (n, nid) in [next, prev].into_iter().enumerate() {
let Some(nid) = nid else { continue };
let Some(np) = app.session.catalog.photo(nid).cloned() else { continue };
let nf = Frame::with_lens(np.width.max(1) as usize, np.height.max(1) as usize, &np.develop, with_crop, np.embedded_lens.as_ref());
let na = nf.aspect() as f32;
let nr = fit_rect(main_area, na, app.ui.zoom, output_px(&nf), ppp, app.ui.pan);
let nw = app.ui.settings.prefetch_edge(nr.width().max(nr.height()) * ppp, output_px(&nf).into_iter().max().unwrap_or(1), texture_side);
let (w, h) = if na >= 1.0 { (nw, (nw as f32 / na) as usize) } else { ((nw as f32 * na) as usize, nw) };
if let Some(job) = app.session.loupe_job(nid, w.max(8), h.max(8), with_crop) {
app.renderer.prefetch(Slot::Prefetch(n as u8), job, PREFETCH_PRIORITY);
}
let mode = prep_mode(app);
let prep = app.session.prep_settings();
let ahead = prep.ahead(mode);
// editing: the next photos are decoded and kept, ready for their edits (their view render is not
// what is wanted, and a cached one would leave the photo undecoded)
let develop = mode == lightcraft_engine::prep::Mode::Develop;
let Some(i) = ids.iter().position(|p| *p == id).filter(|_| ahead > 0) else { return };
let mut targets = app.look_ahead.targets_n(&ids, i, ahead, BEHIND);
if prep.scope(mode) == lightcraft_engine::prep::Scope::Marked {
targets.retain(|t| lightcraft_engine::prep::marked(&app.session, *t));
}
// a few at a time, nearest first (the pool keeps workers free for the photo the user goes to)
let mut room = PREFETCH_AT_ONCE.saturating_sub(app.renderer.prefetches_pending());
let todo: Vec<(usize, PhotoId)> = targets.into_iter().filter_map(|nid| ids.iter().position(|p| *p == nid).map(|at| (at, nid))).collect();
for (at, nid) in todo {
if room == 0 {
break;
}
let n = prefetch_slot(at);
let Some(mut job) = neighbour_job(app, nid, main_area, ppp, texture_side, with_crop) else { continue };
if develop {
job.view_cache = None;
}
// (asked for already, in this slot: queued, running or made)
if app.renderer.prefetch_asked(n, job.key) {
continue;
}
room -= 1;
app.renderer.prefetch(n, job, PREFETCH_PRIORITY);
}
}
/// The render a neighbour is prepared with: its loupe render at the size it will be drawn at.
fn neighbour_job(
app: &mut LightcraftApp,
nid: PhotoId,
main_area: Rect,
ppp: f32,
texture_side: usize,
with_crop: bool,
) -> Option<lightcraft_engine::media::RenderJob> {
let np = app.session.catalog.photo(nid)?;
let nf = Frame::with_lens(np.width.max(1) as usize, np.height.max(1) as usize, &np.develop, with_crop, np.embedded_lens.as_ref());
let na = nf.aspect() as f32;
let nr = fit_rect(main_area, na, app.ui.zoom, output_px(&nf), ppp, app.ui.pan);
let nw = app.ui.settings.prefetch_edge(nr.width().max(nr.height()) * ppp, output_px(&nf).into_iter().max().unwrap_or(1), texture_side);
let (w, h) = if na >= 1.0 { (nw, (nw as f32 / na) as usize) } else { ((nw as f32 * na) as usize, nw) };
app.session.loupe_job(nid, w.max(8), h.max(8), with_crop)
}
/// What Settings ▸ Performance lets the mode the user is in prepare ahead.
fn prepare_ahead(app: &LightcraftApp) -> Prepare {
let mode = prep_mode(app);
let prep = app.session.prep_settings();
Prepare::of(prep.ahead(mode), prep.scope(mode))
}
/// The most the loupe shows a whole photo at (the canvas's long side; a photo of another shape than
/// this one may fill it), noted while it shows one view fitted (`single`): standard previews are
/// built at least this big.
fn note_loupe_edge(app: &mut LightcraftApp, canvas_long: f32, single: bool) {
if single && matches!(app.ui.zoom, Zoom::Fit) && canvas_long.is_finite() && canvas_long >= 1.0 {
app.session.loupe_edge = canvas_long.ceil() as usize;
}
}
@@ -733,10 +801,138 @@ struct LoupeAsk {
overlay: lightcraft_pipeline::Overlay,
}
/// Stepping fast (an arrow held down): the render waits until the photo stays, the camera JPEG or
/// thumbnail shows meanwhile, and a render queued for a photo already left is dropped. Otherwise
/// the loupe's own render is requested. Whether the photo is still settling.
fn request_main_or_settle(app: &mut LightcraftApp, ui: &egui::Ui, a: &LoupeAsk, size: [usize; 2], whole_draft: bool, now: f64) -> bool {
let settling = !a.interacting && app.loupe_timing.settling(a.id, now);
if !settling {
request_main(app, ui, a, size, whole_draft);
return false;
}
drop_unsettled(&mut app.renderer, a.id);
ui.ctx().request_repaint_after(std::time::Duration::from_secs_f64(crate::loupe_timing::SETTLE_SECS / 2.0));
true
}
/// While the photo settles: drop the raw renders still queued for a photo the user is stepping
/// past (one already running finishes). Its whole-frame render unless that is on screen already,
/// the zoomed-in windows (a region render decodes the raw as much as the frame) and the Before
/// view, which is a raw render too.
pub(crate) fn drop_unsettled(renderer: &mut crate::render::Renderer, photo: PhotoId) {
if renderer.textures.get(&Slot::Main).is_none_or(|t| t.photo != photo) {
renderer.drop_queued(Slot::Main);
}
for slot in [Slot::Region, Slot::RegionBefore, Slot::Before] {
renderer.drop_queued(slot);
}
}
/// Zoomed in on an unedited raw (`zoomed`: one view, window renders up): its camera JPEG at full
/// size under the window render, so a focus check doesn't wait for the raw (PERF-CULL-5; Capture
/// One's Cull view shows the embedded JPEG at 100 % too). Released when the view is not zoomed any
/// more (180 MB). Whether it is wanted.
fn request_camera_full(app: &mut LightcraftApp, photo: &lightcraft_catalog::Photo, zoomed: bool, edge: usize) -> bool {
let wanted = zoomed && app.session.embedded_thumb_is_final(photo);
if wanted {
if let Some(q) = app.session.embedded_full_job(photo.id, edge) {
app.renderer.request_quick(Slot::QuickFull, q, 98);
}
} else if app.renderer.textures.contains_key(&Slot::QuickFull) || app.renderer.is_pending(Slot::QuickFull) {
app.renderer.forget(Slot::QuickFull);
}
wanted
}
/// The camera JPEG at full size, magnified like the whole-frame render, until the window is
/// there: only where the window will go (`cover`, see [`window_cover`]). A window capped smaller
/// than the view leaves strips around it, and those keep the developed whole-frame render rather
/// than an unlabelled camera JPEG (CodeRabbit on #3544). Whether it was drawn.
fn draw_camera_full(app: &LightcraftApp, p: &egui::Painter, id: PhotoId, img_rect: Rect, display_aspect: f32, cover: Option<Rect>) -> bool {
let Some(cover) = cover else { return false };
let Some(tex) = app.renderer.textures.get(&Slot::QuickFull).filter(|t| t.photo == id) else { return false };
if !crate::region::same_aspect(tex.size[0].max(1) as f32 / tex.size[1].max(1) as f32, display_aspect) {
return false;
}
p.with_clip_rect(p.clip_rect().intersect(cover)).image(tex.id(), img_rect, Rect::from_min_max(pos2(0.0, 0.0), pos2(1.0, 1.0)), Color32::WHITE);
true
}
/// Where on screen the requested window renders go (each its window's share of its view's drawn
/// rect), joined: `None` without any.
fn window_cover(windows: &[(WindowView, crate::region::RegionView)]) -> Option<Rect> {
windows
.iter()
.map(|(v, w)| {
let (fw, fh) = (w.full.0.max(1) as f32, w.full.1.max(1) as f32);
let at = v.img.min + vec2(w.window.x as f32 / fw * v.img.width(), w.window.y as f32 / fh * v.img.height());
Rect::from_min_size(at, vec2(w.window.w as f32 / fw * v.img.width(), w.window.h as f32 / fh * v.img.height()))
})
.reduce(|a, b| a.union(b))
}
/// Which window renders [`draw_windows`] drew.
#[derive(Clone, Copy)]
struct WindowsDrawn {
/// Any of them.
any: bool,
/// The window of the view shown whole: the After side's in a split view, the Before's when
/// the Before is shown alone; what the timing and the source shown are about. Holding `\`
/// side by side or top/bottom keeps both panes, and the After pane is still the one shown.
shown: bool,
}
/// Draw the window renders over the whole-frame ones, each where it belongs in the frame
/// (clipped to its pane, or its side of the line).
fn draw_windows(
app: &LightcraftApp,
p: &egui::Painter,
ctx: &WindowCtx,
windows: &[(WindowView, crate::region::RegionView)],
[canvas, img_rect]: [Rect; 2],
(show_before, split): (bool, bool),
) -> WindowsDrawn {
let mut out = WindowsDrawn { any: false, shown: false };
// (holding `\` in a wipe shows the Before everywhere: it is the Before view alone then; side by
// side or top/bottom, both panes stay, and each window stays in its own)
let before_alone = show_before && !split;
for (v, view) in windows {
let mode = if before_alone { BeforeAfter::Original } else { app.ui.before_after };
let clip = crate::region::window_clip(mode, v.before, v.visible, canvas, img_rect);
let drawn = draw_window(&p.with_clip_rect(clip), app, ctx, v, view);
out.any |= drawn;
out.shown |= drawn && (!v.before || before_alone);
}
out
}
/// What is on screen once the window renders are drawn: the camera JPEG at full size stands in
/// for the whole-frame render until a window covers it, and a window rendered over a stand-in
/// (thumbnail or camera JPEG) before the whole frame is the raw's own render.
fn shown_after_windows(shown: &'static str, camera_full_drawn: bool, drawn: WindowsDrawn) -> &'static str {
if camera_full_drawn && !drawn.any {
"embedded"
} else if drawn.shown && !matches!(shown, "render" | "cached") {
"render"
} else {
shown
}
}
/// Whether the camera's own JPEG is what is on screen (not under a window render).
fn camera_jpeg_shown(app: &LightcraftApp, photo: &lightcraft_catalog::Photo, shown: &str, drawn: WindowsDrawn) -> bool {
!drawn.any && (shown == "embedded" || (shown == "thumb" && app.session.embedded_thumb_is_final(photo)))
}
/// Request the loupe's own render at `[rw, rh]` (drafts during drags).
fn request_main(app: &mut LightcraftApp, ui: &egui::Ui, a: &LoupeAsk, [rw, rh]: [usize; 2], whole_draft: bool) {
let (id, interacting, fit_px, overlay) = (a.id, a.interacting, a.fit_px, a.overlay);
if let Some(job) = app.session.loupe_job(id, rw.max(8), rh.max(8), a.with_crop) {
if let Some(mut job) = app.session.loupe_job(id, rw.max(8), rh.max(8), a.with_crop) {
// editing: the photo is rendered from its source, which stays decoded for the first slider
// draft (a cached view render would show at once but leave the source unread until then)
if prep_mode(app) == lightcraft_engine::prep::Mode::Develop {
job.view_cache = None;
}
let job = if interacting { job.draft() } else { job };
// at rest the render is shrunk to exactly the size it is drawn at with a Lanczos filter on
// the worker, so the window draws it texel for texel instead of shrinking it with 2-tap
@@ -745,6 +941,11 @@ fn request_main(app: &mut LightcraftApp, ui: &egui::Ui, a: &LoupeAsk, [rw, rh]:
let job = job.with_overlay(overlay).with_proof(app.ui.soft_proof.then_some(app.ui.proof));
let job = job.with_hidden_grading(app.ui.grading_hidden_for(id));
app.renderer.request(Slot::Main, job, 100);
// the photo on screen comes first: background preparation waits while it is being made
// (only then: this runs every loupe frame, and a render already shown asks for nothing)
if app.renderer.is_pending(Slot::Main) {
signal_foreground();
}
// a drag's draft that renders in a few ms (GPU) is drawn in this frame, not the next (zoomed
// in, what changes on screen is the window's draft)
#[cfg(not(target_arch = "wasm32"))]
@@ -757,6 +958,19 @@ fn request_main(app: &mut LightcraftApp, ui: &egui::Ui, a: &LoupeAsk, [rw, rh]:
}
}
/// Tell background preparation the loupe is waiting for its render ([`lightcraft_engine::prep::foreground`]).
fn signal_foreground() {
#[cfg(test)]
FOREGROUND_SIGNALS.with(|n| n.set(n.get() + 1));
lightcraft_engine::prep::foreground();
}
#[cfg(test)]
thread_local! {
/// Tests: how often this thread's loupe told background preparation to wait.
pub(crate) static FOREGROUND_SIGNALS: std::cell::Cell<u64> = const { std::cell::Cell::new(0) };
}
/// Hovering a preset or profile: the photo with that look, shown instead of the loupe render once
/// it is ready (nothing is committed). The key of its render and the label to show.
fn request_hover(app: &mut LightcraftApp, a: &LoupeAsk, [sw, sh]: [usize; 2]) -> Option<(u64, String)> {
@@ -887,6 +1101,26 @@ fn request_stand_in(app: &mut LightcraftApp, id: PhotoId, edge: usize, with_crop
/// Neighbour prefetch: below on-screen thumbnails, above background thumbnail refreshes.
const PREFETCH_PRIORITY: u32 = 4;
/// Photos prepared ahead at once (each render spreads over the cores already): the others wait,
/// so the photo the user steps to next finds a worker free.
const PREFETCH_AT_ONCE: usize = 2;
/// Prepared-ahead slots, by the photo's place in the view: a step keeps the photos still ahead in
/// the slots they have (nothing is asked for twice), and the jump to another part of the shoot
/// replaces what was waiting.
const PREFETCH_SLOTS: usize = 32;
fn prefetch_slot(at: usize) -> Slot {
Slot::Prefetch((at % PREFETCH_SLOTS) as u8)
}
/// What the user is doing, for the background preparation: editing (a tool panel open), else
/// culling a folder or browsing the library.
pub fn prep_mode(app: &LightcraftApp) -> lightcraft_engine::prep::Mode {
match app.ui.right {
RightPanel::Edit | RightPanel::Crop | RightPanel::Remove | RightPanel::Masking | RightPanel::RedEye => lightcraft_engine::prep::Mode::Develop,
_ => lightcraft_engine::prep::Mode::of(&app.session),
}
}
/// Targeted adjustment tool: dragging up/down on the photo raises/lowers the tone-curve region or
/// the colour-mixer bands under the press point (`develop.targeted`, one call per whole step, all
@@ -1075,6 +1309,29 @@ pub(crate) fn click_zoom_and_pan(app: &mut LightcraftApp, resp: &egui::Response,
#[cfg(test)]
mod tests {
/// The camera JPEG goes only where the window render will: a window of the left half of the
/// frame covers the left half of the drawn photo, not the strips beside it.
#[test]
fn the_camera_jpeg_covers_only_where_the_window_goes() {
use super::{WindowView, window_cover};
use crate::region::RegionView;
use egui::{Rect, pos2};
use lightcraft_pipeline::PixelWindow;
assert_eq!(window_cover(&[]), None, "no window: no camera JPEG");
let img = Rect::from_min_max(pos2(100.0, 50.0), pos2(700.0, 450.0));
let v = WindowView { slot: crate::render::Slot::Region, before: false, img, target: img, visible: img };
let w = RegionView {
photo: lightcraft_catalog::PhotoId(1),
before: false,
key: 1,
full: (6000, 4000),
window: PixelWindow { x: 0, y: 1000, w: 3000, h: 2000 },
settings: 0,
};
let r = window_cover(&[(v, w)]).unwrap();
assert_eq!(r, Rect::from_min_max(pos2(100.0, 150.0), pos2(400.0, 350.0)), "the window's share of the drawn rect");
}
/// The rotation pointer's arc bulges away from the side of the crop box it is on, and follows
/// a box that is itself rotated.
#[test]
crates/ui-egui/src/panels/detail/info.rs
+18
View File
@@ -201,6 +201,24 @@ pub(super) fn filter_pill(app: &mut LightcraftApp, ui: &mut egui::Ui, canvas: Re
}
}
/// A small "Camera preview" label at the top right of the photo while the loupe shows the camera's
/// embedded JPEG (widget `loupe:cameraPreview`), with what that means on hover.
pub(super) fn camera_preview_badge(ui: &egui::Ui, canvas: Rect, over: Rect) {
let t = Tokens::get(ui.ctx());
let p = ui.painter_at(canvas);
let text = crate::i18n::tr("Camera preview");
let g = p.layout_no_wrap(text.to_string(), t.font(11.5), Color32::from_gray(225));
let size = vec2(g.size().x + 16.0, 18.0);
let over = if over.is_positive() { over } else { canvas };
let r = Rect::from_min_size(pos2(over.right() - size.x - 8.0, over.top() + 8.0), size);
p.rect_filled(r, 9.0, Color32::from_black_alpha(165));
p.galley(pos2(r.left() + 8.0, r.center().y - g.size().y / 2.0), g, Color32::from_gray(225));
register(ui, "loupe:cameraPreview", r);
let tip = crate::i18n::tr("The camera's own JPEG, shown until LightCraft has rendered the raw. Judge colour and tone on the rendered photo.");
let resp = ui.interact(r, egui::Id::unique("camera-preview-badge"), Sense::hover()).on_hover_text(tip);
resp.widget_info(|| egui::WidgetInfo::labeled(egui::Role::Label, true, format!("{text}: {tip}")));
}
/// A raw shown from its embedded JPEG (issue #10): a pill at the canvas' top centre saying so;
/// hovering it explains (registered as `notice:previewOnly:loupe`).
pub(super) fn preview_only_pill(ui: &mut egui::Ui, canvas: Rect, reason: &str) {
crates/ui-egui/src/panels/detail/look_ahead.rs
+119
View File
@@ -0,0 +1,119 @@
//! The loupe's look-ahead: stand-ins and thumbnails for the photos it is likely to be stepped to
//! next (PERF-CULL-2).
use super::{LightcraftApp, PhotoId};
/// How many photos ahead (in the direction of travel) the loupe makes its stand-in for, and how
/// many behind (a step back is common when culling).
const AHEAD: usize = 5;
pub(super) const BEHIND: usize = 1;
/// The look-ahead's priority: below the loupe's own render (100), above everything else.
const AHEAD_PRIORITY: u32 = 96;
/// Thumbnails the loupe keeps while stepping (as many as the grid keeps), and how many photos on
/// either side of the open one stay when it trims them.
const MAX_LOUPE_THUMBS: usize = 600;
const KEEP_AROUND: usize = 64;
/// The thumbnail size the look-ahead asks for (the filmstrip's at 2x).
const AHEAD_THUMB_EDGE: usize = 256;
/// What [`look_ahead`] last worked out (see `LightcraftApp::look_ahead`).
#[derive(Debug, Default)]
pub struct LookAhead {
/// The last photo the loupe showed, for the direction of travel.
last: Option<PhotoId>,
/// +1 forward (the default), -1 backward.
dir: i8,
/// (photo, direction, catalog revision, visible list generation, edge, crop, which photos) the
/// requests were made for: a new source, filter or sort changes the neighbours without an edit,
/// and a change of the preparation settings what may be made.
made_for: Option<(PhotoId, i8, u64, u64, usize, bool, Prepare, u64)>,
}
/// Which photos the look-ahead may prepare, from Settings ▸ Performance for what the user is doing
/// ([`lightcraft_engine::prep::Mode`]): none when its "Photos ahead" is 0 or it prepares only the photo
/// on screen, else the photos of its "which photos".
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum Prepare {
Nothing,
Photos(lightcraft_engine::prep::Scope),
}
impl Prepare {
/// For `ahead` photos ahead (the mode's count, 0 for the photo on screen only) of `scope`.
pub(crate) fn of(ahead: usize, scope: lightcraft_engine::prep::Scope) -> Prepare {
if ahead == 0 || scope == lightcraft_engine::prep::Scope::OnScreen { Prepare::Nothing } else { Prepare::Photos(scope) }
}
}
impl LookAhead {
/// The photos to prepare around `id` at position `i` of `ids`: [`AHEAD`] in the direction of
/// travel, nearest first, then [`BEHIND`] the other way, of those `prepare` allows. (The
/// stand-ins are cheap: as many as [`AHEAD`] whatever the mode's count, which is for the full
/// renders made ahead.)
fn targets(&self, app: &LightcraftApp, ids: &[PhotoId], i: usize, prepare: Prepare) -> Vec<PhotoId> {
let Prepare::Photos(scope) = prepare else { return Vec::new() };
let mut targets = self.targets_n(ids, i, AHEAD, BEHIND);
if scope == lightcraft_engine::prep::Scope::Marked {
targets.retain(|t| lightcraft_engine::prep::marked(&app.session, *t));
}
targets
}
/// `ahead` photos in the direction of travel from position `i` of `ids`, nearest first, then
/// `behind` the other way.
pub fn targets_n(&self, ids: &[PhotoId], i: usize, ahead: usize, behind: usize) -> Vec<PhotoId> {
let dir: isize = if self.dir < 0 { -1 } else { 1 };
let at = |k: isize| i.checked_add_signed(k).and_then(|j| ids.get(j)).copied();
let cap = |n: usize| n.min(ids.len()) as isize;
let ahead = (1..=cap(ahead)).filter_map(|k| at(dir * k));
let behind = (1..=cap(behind)).filter_map(|k| at(-dir * k));
ahead.chain(behind).collect()
}
/// Whether the user is stepping backwards.
pub fn backward(&self) -> bool {
self.dir < 0
}
}
/// Culling: make the loupe's stand-ins (the camera JPEG of an unedited raw, or a cached render) of
/// the next photos in the direction of travel, so that a step shows the next one at once instead
/// of after reading and decoding it (see [`Slot::QuickAhead`]).
/// Only the photos `prepare` allows (Settings ▸ Performance): none at all asks for nothing, and drops the
/// stand-ins still queued.
pub(crate) fn look_ahead(app: &mut LightcraftApp, id: PhotoId, edge: usize, crop: bool, prepare: Prepare) {
let rev = app.session.catalog.revision;
if app.look_ahead.last != Some(id) {
let ids = app.session.visible_cloned();
let pos = |p: PhotoId| ids.iter().position(|q| *q == p);
if let (Some(prev), Some(i)) = (app.look_ahead.last, pos(id))
&& let Some(j) = pos(prev)
{
app.look_ahead.dir = if j > i { -1 } else { 1 };
}
app.look_ahead.last = Some(id);
}
let (visible_gen, ids) = app.session.visible_shared();
// (the renderer's epoch: a reset drops the stand-ins made, which are then asked for again)
let want = (id, app.look_ahead.dir, rev, visible_gen, edge, crop, prepare, app.renderer.epoch());
if app.look_ahead.made_for == Some(want) {
return;
}
app.look_ahead.made_for = Some(want);
let Some(i) = ids.iter().position(|p| *p == id) else { return };
let targets = app.look_ahead.targets(app, &ids, i, prepare);
// stepping through a whole wedding in the loupe would keep every thumbnail it ever drew: keep
// the ones around the photo (what the filmstrip can show) once there are many
if app.renderer.thumb_textures() > MAX_LOUPE_THUMBS {
let keep = ids.iter().skip(i.saturating_sub(KEEP_AROUND)).take(2 * KEEP_AROUND + 1).copied().collect();
app.renderer.evict_thumbs(&keep, MAX_LOUPE_THUMBS);
}
// their thumbnails first (a small embedded JPEG each, a few ms): something to show at once
// even when a step outruns the full-size previews, and in full screen, where no filmstrip
// asks for them
for &nid in &targets {
crate::panels::grid::request_thumb(app, nid, AHEAD_THUMB_EDGE, AHEAD_PRIORITY + 2);
}
let jobs = targets.into_iter().filter_map(|nid| app.session.quick_view_job(nid, edge, crop)).collect();
app.renderer.look_ahead(jobs, AHEAD_PRIORITY);
}
crates/ui-egui/src/panels/detail/window.rs
+4 -3
View File
@@ -110,13 +110,14 @@ pub(super) fn request_window(app: &mut LightcraftApp, c: &WindowCtx, v: &WindowV
/// Draw `v`'s window texture over the whole-frame render, if it is the one asked for (or, in a
/// drag, an earlier draft of it).
pub(super) fn draw_window(p: &egui::Painter, app: &LightcraftApp, c: &WindowCtx, v: &WindowView, wanted: &crate::region::RegionView) {
let Some(tex) = app.renderer.textures.get(&v.slot).filter(|t| t.photo == c.id) else { return };
pub(super) fn draw_window(p: &egui::Painter, app: &LightcraftApp, c: &WindowCtx, v: &WindowView, wanted: &crate::region::RegionView) -> bool {
let Some(tex) = app.renderer.textures.get(&v.slot).filter(|t| t.photo == c.id) else { return false };
let Some(tile) = app.region_tiles.get(&(v.before, tex.key)).filter(|t| t.is_current(c.id, wanted.full, wanted.settings, c.interacting)) else {
return;
return false;
};
let (fw, fh) = (tile.full.0 as f32, tile.full.1 as f32);
let at = v.img.min + vec2(tile.window.x as f32 / fw * v.img.width(), tile.window.y as f32 / fh * v.img.height());
let dst = Rect::from_min_size(at, vec2(tile.window.w as f32 / fw * v.img.width(), tile.window.h as f32 / fh * v.img.height()));
p.image(tex.id(), dst, Rect::from_min_max(pos2(0.0, 0.0), pos2(1.0, 1.0)), Color32::WHITE);
true
}
crates/ui-egui/src/panels/notices.rs
+123 -5
View File
@@ -1,6 +1,7 @@
//! Things the user must not miss (issue #103): settings files that were damaged or unreadable when
//! the library opened (and host warnings such as a damaged `ui.json`), and quitting while changes
//! are only in memory or while tasks that would be cut short are running (issue #345).
//! are only in memory or while tasks that would be cut short are running (issue #345). Also, once
//! per session, that the drive holding the preview cache is running low ([`disk_space_notice`]).
use egui::{Align2, RichText, vec2};
@@ -8,11 +9,35 @@ use crate::theme::Tokens;
use crate::widgets::WidgetHit;
use crate::{LightcraftApp, QuitPrompt};
/// Collect the library's settings-file warnings (each is shown once), and the no-GPU notice once the GPU device has
/// been tried.
pub fn logic(app: &mut LightcraftApp) {
/// Collect the library's settings-file warnings (each is shown once), the no-GPU notice once the GPU device has
/// been tried, and the low-disk notice (once per session).
pub fn logic(app: &mut LightcraftApp, ctx: &egui::Context) {
let new = app.session.take_library_warnings();
app.notices.extend(new);
// a low-disk notice still queued from the library that was open before is not this one's
if let Some((generation, text)) = app.disk_notice.as_ref()
&& *generation != app.session.library_generation()
{
let text = text.clone();
app.notices.retain(|n| *n != text);
app.disk_notice = None;
}
if app.disk_space_notice {
// the cache grows with every thumbnail and preview, not only while previews are built: the
// engine reads the free space at most every few minutes (a clock check otherwise)
app.session.check_disk_space(false);
// an idle window still takes the read's answer and makes the next check: a pass soon while
// the read runs, and one when the next check is due
if !app.session.disk_space_alert_raised() {
let s = &app.session;
ctx.request_repaint_after(disk_check_wait(s.disk_space_read_running_for(), s.disk_space_next_check_in()));
}
if let Some(alert) = app.session.take_disk_space_alert() {
let text = disk_space_notice(&alert);
app.disk_notice = Some((app.session.library_generation(), text.clone()));
app.notices.push(text);
}
}
use lightcraft_engine::gpu;
if app.gpu_notice && app.gpu_applied == Some(true) && gpu::ready() {
app.gpu_notice = false;
@@ -22,6 +47,18 @@ pub fn logic(app: &mut LightcraftApp) {
}
}
/// How soon the notice logic looks again for the free-space read: soon while a read runs, less often once it has
/// run for a while (a drive that hangs must not keep an idle window drawing four times a second for good), and
/// when the next check is due (`next_in`: sooner after a read that couldn't start) when none runs.
fn disk_check_wait(running_for: Option<std::time::Duration>, next_in: Option<std::time::Duration>) -> std::time::Duration {
use std::time::Duration;
match running_for {
Some(d) if d < Duration::from_secs(5) => Duration::from_millis(250),
Some(_) => Duration::from_secs(5),
None => next_in.unwrap_or(lightcraft_engine::disk_space::CHECK_EVERY),
}
}
/// What to tell the user when GPU rendering is on (`enabled`) but the GPU could not be used for `reason`: only when no
/// graphics adapter was found at all ("no GPU adapter among …", e.g. no Vulkan loader on Linux), which makes every
/// render much slower without saying so. Nothing when it was turned off (preference, `LIGHTCRAFT_GPU`), a software
@@ -40,6 +77,34 @@ pub fn gpu_notice(reason: Option<&str>, enabled: bool) -> Option<String> {
Some(text)
}
/// Bytes in plain words: "12.3 GB", "850 MB" (binary units, as the cache size choices).
fn size_words(bytes: u64) -> String {
const GB: f64 = (1u64 << 30) as f64;
const MB: f64 = (1u64 << 20) as f64;
let b = bytes as f64;
if b >= 100.0 * GB {
format!("{:.0} GB", b / GB)
} else if b >= GB {
format!("{:.1} GB", b / GB)
} else {
format!("{:.0} MB", b / MB)
}
}
/// What to tell the user when the drive holding the preview cache runs low: its free space, the
/// cache's size, and where a limit can be set (the notice's button opens Settings ▸ Performance).
pub fn disk_space_notice(alert: &lightcraft_engine::disk_space::LowDiskAlert) -> String {
let free = size_words(alert.space.available);
let total = size_words(alert.space.total);
let cache = size_words(alert.cache_bytes);
crate::i18n::tr_format!(
"The drive with this library's previews is almost full: {free} free of {total}. The preview cache takes {cache} and grows as the library does. To keep it smaller, set a size limit in Settings → Performance.",
free = free,
total = total,
cache = cache
)
}
/// The quit prompt's text while `count` cancellable tasks run (`names` lists them).
fn tasks_message(count: usize, names: &str) -> String {
if count == 1 {
@@ -113,6 +178,7 @@ enum Choice {
QuitAnyway,
Cancel,
Ok,
Settings,
}
/// The prompt (quit or settings notice) that owns input this frame, if any.
@@ -597,11 +663,27 @@ pub fn show(app: &mut LightcraftApp, ctx: &egui::Context) {
app.quit_confirmed = true;
app.ui.quit = true;
}
Some(Choice::Cancel | Choice::Ok) | None => app.quit_prompt = None,
Some(Choice::Cancel | Choice::Ok | Choice::Settings) | None => app.quit_prompt = None,
}
return;
}
if let Some(text) = app.notices.first().cloned() {
let disk = app.disk_notice.as_ref().filter(|(_, t)| *t == text).map(|(generation, _)| *generation);
if let Some(generation) = disk {
// another library opened since `logic` ran this frame: the notice is the previous library's,
// and the next `logic` takes it away
if generation != app.session.library_generation() {
return;
}
let buttons = [("noticeSettings", "Open Performance Settings", Choice::Settings), ("noticeOk", "OK", Choice::Ok)];
let Some(i) = window(ctx, "settings-notice", "Drive almost full", &text, &buttons, &mut app.notice_input) else { return };
app.notices.remove(0);
app.disk_notice = None;
if buttons.get(i).is_some_and(|(_, _, choice)| *choice == Choice::Settings) {
app.ui.dialog = Some(crate::state::Dialog::Settings { tab: "performance".into() });
}
return;
}
let buttons = [("noticeOk", "OK", Choice::Ok)];
if window(ctx, "settings-notice", "Settings file problem", &text, &buttons, &mut app.notice_input).is_some() {
app.notices.remove(0);
@@ -613,6 +695,42 @@ pub fn show(app: &mut LightcraftApp, ctx: &egui::Context) {
mod tests {
use super::{PromptKey, gpu_notice, prepare_ui_pass, prompt_allows_event, prompt_key};
/// A library switch between `logic` and drawing leaves the previous library's low-disk notice queued for
/// that frame: it is not drawn, nor answered (CodeRabbit on #3577).
#[test]
fn a_low_disk_notice_of_the_library_before_is_not_drawn() {
let ctx = egui::Context::default();
let mut app = crate::LightcraftApp::new(lightcraft_engine::Session::with_demo(), crate::Services::default());
let text = "Drive almost full".to_string();
app.notices.push(text.clone());
app.disk_notice = Some((app.session.library_generation().wrapping_add(1), text));
// the prompt's first button pressed, as Enter would
app.notice_input.activate = Some(0);
let mut out = ctx.run_ui(egui::RawInput::default(), |ui| super::show(&mut app, ui.ctx()));
out.textures_delta.clear();
assert_eq!(app.notices.len(), 1, "not answered");
assert!(app.ui.dialog.is_none(), "Settings didn't open");
// the current library's: drawn and answered
app.disk_notice = app.disk_notice.take().map(|(_, t)| (app.session.library_generation(), t));
let mut out = ctx.run_ui(egui::RawInput::default(), |ui| super::show(&mut app, ui.ctx()));
out.textures_delta.clear();
assert!(app.notices.is_empty(), "answered");
}
/// The window wakes four times a second while a free-space read runs, then every few seconds once it hangs
/// (CodeRabbit on #3577: it asked for 250 ms passes for as long as the drive hung).
#[test]
fn a_hanging_free_space_read_is_looked_at_less_often() {
use lightcraft_engine::disk_space::{CHECK_EVERY, RETRY_AFTER};
use std::time::Duration;
assert_eq!(super::disk_check_wait(Some(Duration::ZERO), None), Duration::from_millis(250));
assert_eq!(super::disk_check_wait(Some(Duration::from_secs(60)), None), Duration::from_secs(5));
assert_eq!(super::disk_check_wait(Some(Duration::MAX), None), Duration::from_secs(5));
assert_eq!(super::disk_check_wait(None, None), CHECK_EVERY);
// after a read that couldn't start: at the retry, not five minutes on (CodeRabbit on #3077)
assert_eq!(super::disk_check_wait(None, Some(RETRY_AFTER)), RETRY_AFTER);
}
/// The prompt barrier keeps its own keys and pointer input, drops text/wheel input, and drops
/// the start/move half of a new touch gesture while keeping End/Cancel to clear egui's private
/// touch state. No prompt means everything survives.
crates/ui-egui/src/render.rs
+272 -65
View File
@@ -45,6 +45,13 @@ pub enum Slot {
/// Background preparation of a neighbouring photo (no texture: its decoded source and view
/// render are cached by the engine). 0 = next, 1 = previous.
Prefetch(u8),
/// The loupe's stand-in for a photo the user is likely to step to next (the camera JPEG of an
/// unedited raw, or a cached render), made ahead in the direction of travel so a step shows it
/// at once: [`Renderer::request_quick`] takes the texture over into [`Slot::Preview`].
QuickAhead(u8),
/// The loupe zoomed in on an unedited raw: its camera JPEG at full size, drawn under the window
/// render until that arrives (a focus check at 100 % without waiting for the raw).
QuickFull,
/// A variant thumbnail (profile / preset browsers), by its job key ([`Session::variant_job`]).
/// Kept in a bounded LRU ([`VARIANT_TEXTURES`]).
Variant(u64),
@@ -242,6 +249,11 @@ pub const MAX_VARIANT_TEXTURES: usize = 800;
/// Priority of variant thumbnail jobs: below on-screen grid thumbnails and the loupe.
const VARIANT_PRIORITY: u32 = 6;
/// The camera-preview edge a loupe stand-in is made at (0: one that doesn't depend on the size).
fn quick_edge(job: &QuickJob) -> usize {
job.embedded.as_ref().map_or(0, |e| e.2)
}
pub struct Renderer {
#[cfg(test)]
pub(crate) thumb_jobs_built: usize,
@@ -283,6 +295,14 @@ pub struct Renderer {
pub completed: u64,
/// Finished jobs thrown away because a newer request for their slot had replaced them.
pub discarded: u64,
/// Photos given a grid thumbnail since start (a render replacing a stand-in isn't counted again):
/// unlike [`Renderer::thumb_textures`] it never goes down when old thumbnails are evicted (for
/// benchmarks).
pub thumbs_made: u64,
/// The photos [`Self::thumbs_made`] counted, by library (a hash of its folder) and id: a
/// thumbnail made again after eviction, or after going to another library and back, is the
/// same photo (photo ids are a library's own). Never cleared.
thumbs_counted: std::collections::HashSet<(u64, PhotoId)>,
/// Keep a CPU copy of every texture so the UI can be rasterized headlessly (set when the app
/// is driven by the control channel).
pub keep_pixels: bool,
@@ -290,6 +310,10 @@ pub struct Renderer {
stages: HashMap<Slot, Arc<StageCache>>,
/// Slot → key of the last quick job requested for it (each is tried once).
quick_tried: HashMap<Slot, u64>,
/// The camera-preview edge each loupe stand-in ([`Slot::Preview`], [`Slot::QuickAhead`]) was
/// asked for (0: one that doesn't depend on the size): a stand-in made ahead for a smaller loupe
/// isn't taken over by a larger one.
quick_edge: HashMap<Slot, usize>,
/// Jobs that failed (unreadable or missing file…), by slot and job key, with the error and
/// the photo: not requested again until the key changes (an edit, another size) — the grid
/// asks every frame.
@@ -327,6 +351,9 @@ pub struct Renderer {
native_frames: HashMap<Slot, lightcraft_engine::media::ScreenFrame>,
}
/// How many [`Slot::QuickAhead`] stand-ins the loupe keeps ready.
pub const AHEAD_SLOTS: u8 = 6;
/// After this long without renders the GPU renderer's pool of recycled buffers is freed.
#[cfg(not(target_arch = "wasm32"))]
const IDLE_TRIM: std::time::Duration = std::time::Duration::from_secs(3);
@@ -395,9 +422,12 @@ impl Renderer {
last_main_ms: 0.0,
completed: 0,
discarded: 0,
thumbs_made: 0,
thumbs_counted: Default::default(),
keep_pixels: false,
stages: HashMap::new(),
quick_tried: HashMap::new(),
quick_edge: HashMap::new(),
failed: HashMap::new(),
catalog_rev: 0,
prefetched: HashMap::new(),
@@ -731,7 +761,9 @@ impl Renderer {
}
}
/// Request a stand-in for `slot` (once per job key).
/// Request a stand-in for `slot` (once per job key). The loupe's [`Slot::Preview`] takes over
/// a stand-in already made ahead for the same photo ([`Slot::QuickAhead`]) instead of making it
/// again: shown in this frame when it is ready, or as soon as that job finishes.
pub fn request_quick(&mut self, slot: Slot, mut job: QuickJob, priority: u32) {
if self.quick_tried.get(&slot) == Some(&job.key) {
if self.pending.get(&slot).is_some_and(|p| p.0 == job.key && p.1 != priority) {
@@ -758,6 +790,14 @@ impl Renderer {
return;
}
self.quick_tried.insert(slot, job.key);
if matches!(slot, Slot::Preview | Slot::QuickAhead(_)) {
self.quick_edge.insert(slot, quick_edge(&job));
}
if slot == Slot::Preview && self.take_ahead(job.photo, job.key, quick_edge(&job)) {
self.pending.remove(&slot);
self.requests.remove(&slot);
return;
}
let source = job
.small
.as_ref()
@@ -772,7 +812,84 @@ impl Renderer {
);
let key = job.key;
let display = self.display.clone();
self.pool.submit(slot, key, priority, Box::new(move || Self::run_for(|| job.run(), &display)));
let background = matches!(slot, Slot::QuickAhead(_));
self.pool.submit(
slot,
key,
priority,
Box::new(move || {
let run = || if background { lightcraft_engine::memory::in_background(|| job.run()) } else { job.run() };
Self::run_for(run, &display)
}),
);
}
/// Whether the stand-in asked for in `slot` is at least `edge` pixels (see `quick_edge`).
fn ahead_covers(&self, slot: Slot, edge: usize) -> bool {
self.quick_edge.get(&slot).copied().unwrap_or(0) >= edge
}
/// Move a [`Slot::QuickAhead`] texture of `photo` made for `key`, at least `edge` pixels, into
/// [`Slot::Preview`].
fn take_ahead(&mut self, photo: PhotoId, key: u64, edge: usize) -> bool {
let Some(slot) = self
.textures
.iter()
.find(|(s, t)| matches!(s, Slot::QuickAhead(_)) && t.photo == photo && t.key == key && self.ahead_covers(**s, edge))
.map(|(s, _)| *s)
else {
return false;
};
let Some(tex) = self.textures.remove(&slot) else { return false };
self.quick_tried.remove(&slot);
self.textures.insert(Slot::Preview, tex);
true
}
/// Make the loupe's stand-ins for the photos it is likely to be stepped to next (see
/// [`Slot::QuickAhead`]), most likely first: `jobs[0]` gets `priority`, each next one less.
/// A slot keeps a photo for as long as the photo stays wanted (made or being made, it is not
/// made again when the user steps and the photos shift places); slots holding photos not
/// wanted any more are freed for the new ones, and their queued jobs dropped.
pub fn look_ahead(&mut self, jobs: Vec<QuickJob>, priority: u32) {
let slots: Vec<Slot> = (0..AHEAD_SLOTS).map(Slot::QuickAhead).collect();
let held = |r: &Self, s: Slot| r.textures.get(&s).map(|t| t.key).or_else(|| r.quick_tried.get(&s).copied());
// (a stand-in asked for at a smaller loupe size isn't kept for a larger one)
let wanted: HashMap<u64, usize> = jobs.iter().map(|j| (j.key, quick_edge(j))).collect();
let mut free = Vec::new();
for &s in &slots {
let key = held(self, s);
if key.is_none_or(|k| wanted.get(&k).is_none_or(|&edge| !self.ahead_covers(s, edge))) {
self.textures.remove(&s);
// a job already running: its result is thrown away when it arrives (not left as a
// hidden stand-in), unless the loupe is waiting for just that one (a step to the
// photo): then the slot stays reserved for it until it arrives
let preview_edge = self.quick_edge.get(&Slot::Preview).copied().unwrap_or(0);
let loupe_waits =
key.is_some_and(|k| self.pending.get(&Slot::Preview).is_some_and(|p| p.0 == k)) && self.ahead_covers(s, preview_edge);
if !self.drop_queued(s) && loupe_waits {
continue;
}
self.quick_tried.remove(&s);
self.pending.remove(&s);
self.requests.remove(&s);
free.push(s);
}
}
let have: HashMap<u64, Slot> = slots.iter().filter_map(|&s| held(self, s).map(|k| (k, s))).collect();
let mut free = free.into_iter();
for (rank, job) in jobs.into_iter().enumerate() {
let priority = priority.saturating_sub(rank as u32);
if let Some(&s) = have.get(&job.key) {
// kept: a job still queued moves to its new place in the order
if self.quick_tried.get(&s) == Some(&job.key) {
self.request_quick(s, job, priority);
}
continue;
}
let Some(slot) = free.next() else { break };
self.request_quick(slot, job, priority);
}
}
/// Render a slideshow title screen into `slot` (once per screen key, like [`Self::request`]):
@@ -817,6 +934,21 @@ impl Renderer {
);
}
/// Prepared-ahead jobs ([`Slot::Prefetch`]) queued or running.
pub fn prefetches_pending(&self) -> usize {
self.pending.keys().filter(|s| matches!(s, Slot::Prefetch(_))).count()
}
/// Prepared-ahead slots asked for since the renderer last forgot everything.
pub fn prefetch_slots_asked(&self) -> usize {
self.prefetched.keys().filter(|s| matches!(s, Slot::Prefetch(_))).count()
}
/// Whether `slot` was last asked for the job of `key` (queued, running or done).
pub fn prefetch_asked(&self, slot: Slot, key: u64) -> bool {
self.prefetched.get(&slot) == Some(&key)
}
/// Drop the slideshow slides other than `keep`: their textures and their requests that have
/// not started (a slide that is no longer near is not worth rendering).
pub fn release_slides(&mut self, keep: &[lightcraft_engine::slideshow::Item]) {
@@ -869,6 +1001,11 @@ impl Renderer {
self.textures.retain(|_, t| t.native.is_none());
}
/// Changes when [`Self::forget_all`] drops every texture and request: what was asked for before is gone.
pub fn epoch(&self) -> u64 {
self.epoch
}
pub fn is_pending(&self, slot: Slot) -> bool {
self.pending.contains_key(&slot)
}
@@ -1132,6 +1269,89 @@ impl Renderer {
finished
}
/// The slot a finished job goes to. Made ahead for the photo the loupe is now waiting for, it is
/// the loupe's stand-in (the loupe's own request for it, if one was started, is dropped when it
/// arrives); also when the loupe's own request failed and it still wants that photo: the
/// stand-in made ahead still shows, but never over another photo's request.
fn stand_in_slot(&mut self, slot: Slot, r: &RenderResult) -> Slot {
let waiting = self.pending.get(&Slot::Preview).is_some_and(|p| p.0 == r.key);
let failed = !self.pending.contains_key(&Slot::Preview)
&& self.quick_tried.get(&Slot::Preview) == Some(&r.key)
&& self.failed.get(&Slot::Preview).is_some_and(|f| f.0 == r.key && f.3 == r.photo);
if !matches!(slot, Slot::QuickAhead(_))
|| r.rendered.is_err()
|| !(waiting || failed)
|| !self.ahead_covers(slot, self.quick_edge.get(&Slot::Preview).copied().unwrap_or(0))
{
return slot;
}
self.pending.remove(&Slot::Preview);
self.requests.remove(&Slot::Preview);
self.quick_tried.remove(&slot);
Slot::Preview
}
/// A superseded result an interactive view still takes: a draft newer than what it shows.
fn is_newer_draft(&self, slot: Slot, r: &RenderResult) -> bool {
slot.is_view()
&& self.requests.contains_key(&slot)
&& r.request_id >= self.epoch
&& self.shown_ids.get(&slot).is_none_or(|shown| r.request_id > *shown)
&& r.rendered.is_ok()
}
/// Convert `r` to the display profile in use; whether that changed it.
fn convert_for_display(&self, r: &mut RenderResult) -> bool {
let Some(d) = &self.display else { return false };
let before = r.display;
lightcraft_engine::display::present(r, d);
r.display != before
}
/// Show a render that stayed on the GPU (`r.frame`) in `slot`; whether it could be. When it
/// can't, views come back as pixels from then on.
fn show_frame(&mut self, ctx: &egui::Context, slot: Slot, r: &RenderResult, look: Option<(u64, bool)>, ms: f64) -> bool {
let (Some(frame), Ok(rendered)) = (r.frame.as_ref(), r.rendered.as_ref()) else { return false };
self.failed.remove(&slot);
let shown = self.screen.as_mut().and_then(|s| s.show(frame, self.natives.get(&slot).copied()));
let Some(id) = shown else {
// can't be shown: from now on views come back as pixels
log::warn!("render {slot:?}: a GPU-resident render can't be shown; rendering pixels");
self.free_natives();
self.screen = None;
return false;
};
self.natives.insert(slot, id);
if self.keep_pixels {
self.native_frames.insert(slot, frame.clone());
}
let (size, histogram) = ([frame.width, frame.height], rendered.histogram.clone());
match self.textures.get_mut(&slot) {
Some(t) => {
t.show_native(id);
t.look = look;
t.key = r.key;
t.photo = r.photo;
t.size = size;
t.histogram = Some(histogram);
t.ms = ms;
t.quick = None;
}
None => {
// (`tex` is never drawn while `native` is set)
let blank = egui::ColorImage::filled([1, 1], egui::Color32::BLACK);
let tex = ctx.load_texture(format!("{slot:?}"), blank, egui::TextureOptions::LINEAR);
let t =
Tex { look, key: r.key, photo: r.photo, tex, size, histogram: Some(histogram), ms, quick: None, pixels: None, native: Some(id) };
self.textures.insert(slot, t);
}
}
if slot == Slot::Main {
self.last_main_ms = ms;
}
true
}
/// Turn finished jobs into textures. Returns true if anything changed.
fn collect(&mut self, ctx: &egui::Context, session: &mut Session, inline_ms: f64) -> bool {
let finished = self.take_finished(inline_ms);
@@ -1152,12 +1372,7 @@ impl Renderer {
// shows, and the source it decoded: a drag supersedes every request before it
// finishes, and a source thrown away with them is decoded again for every tick.
let (source, look) = if self.requests.get(&slot).map(|request| request.id) != Some(r.request_id) {
let newer_draft = slot.is_view()
&& self.requests.contains_key(&slot)
&& r.request_id >= self.epoch
&& self.shown_ids.get(&slot).is_none_or(|shown| r.request_id > *shown)
&& r.rendered.is_ok();
if !newer_draft {
if !self.is_newer_draft(slot, &r) {
self.discarded += 1;
let replaced = self.requests.contains_key(&slot);
self.decided(|| {
@@ -1180,66 +1395,16 @@ impl Renderer {
self.shown_ids.insert(slot, r.request_id);
}
// (results from an offload, which doesn't know the display profile: converted here)
if let Some(d) = &self.display {
let before = r.display;
lightcraft_engine::display::present(&mut r, d);
if r.display != before {
color = None; // converted after the worker made its texture pixels
}
if self.convert_for_display(&mut r) {
color = None; // converted after the worker made its texture pixels
}
if self.pending.get(&slot).is_some_and(|p| p.0 == r.key) {
self.pending.remove(&slot);
}
slot = self.stand_in_slot(slot, &r);
// a render that stayed on the GPU: drawn from its texture
if let (Some(frame), Ok(rendered)) = (r.frame.as_ref(), r.rendered.as_ref()) {
self.failed.remove(&slot);
let shown = self.screen.as_mut().and_then(|s| s.show(frame, self.natives.get(&slot).copied()));
let Some(id) = shown else {
// can't be shown: from now on views come back as pixels
log::warn!("render {slot:?}: a GPU-resident render can't be shown; rendering pixels");
self.free_natives();
self.screen = None;
continue;
};
self.natives.insert(slot, id);
if self.keep_pixels {
self.native_frames.insert(slot, frame.clone());
}
let (size, histogram) = ([frame.width, frame.height], rendered.histogram.clone());
match self.textures.get_mut(&slot) {
Some(t) => {
t.show_native(id);
t.look = look;
t.key = r.key;
t.photo = r.photo;
t.size = size;
t.histogram = Some(histogram);
t.ms = ms;
t.quick = None;
}
None => {
// (`tex` is never drawn while `native` is set)
let blank = egui::ColorImage::filled([1, 1], egui::Color32::BLACK);
let tex = ctx.load_texture(format!("{slot:?}"), blank, egui::TextureOptions::LINEAR);
let t = Tex {
look,
key: r.key,
photo: r.photo,
tex,
size,
histogram: Some(histogram),
ms,
quick: None,
pixels: None,
native: Some(id),
};
self.textures.insert(slot, t);
}
}
if slot == Slot::Main {
self.last_main_ms = ms;
}
changed = true;
if r.frame.is_some() && r.rendered.is_ok() {
changed |= self.show_frame(ctx, slot, &r, look, ms);
continue;
}
let rendered = match r.rendered {
@@ -1259,7 +1424,7 @@ impl Renderer {
x
}
Err(e) => {
if !matches!(slot, Slot::Prefetch(_)) {
if !matches!(slot, Slot::Prefetch(_) | Slot::QuickAhead(_)) {
let message = render_failure_message(slot, r.photo, source.as_ref(), &e);
#[cfg(test)]
self.failure_messages.push(message.clone());
@@ -1291,6 +1456,11 @@ impl Renderer {
slot = Slot::Thumb(id);
}
}
// (a photo's first thumbnail: not a render replacing its stand-in, nor one made again)
let first_thumb = match slot {
Slot::Thumb(id) | Slot::ThumbQuick(id) => self.thumbs_counted.insert((library_key(session), id)),
_ => false,
};
if let Slot::Thumb(id) = slot {
self.textures.remove(&Slot::ThumbQuick(id));
}
@@ -1332,7 +1502,11 @@ impl Renderer {
);
}
}
if slot == Slot::Main {
if first_thumb {
self.thumbs_made += 1;
}
// (a preview taken from the cache says nothing about how long a render takes)
if slot == Slot::Main && r.quick.is_none() {
self.last_main_ms = ms;
}
changed = true;
@@ -1413,6 +1587,29 @@ impl Renderer {
.collect()
}
/// Forget everything about `slot`: its texture, its queued job, and that its stand-in was asked
/// for (so asking again makes it again).
pub fn forget(&mut self, slot: Slot) {
self.drop_queued(slot);
self.pending.remove(&slot);
self.requests.remove(&slot);
self.quick_tried.remove(&slot);
self.release(slot);
}
/// Drop `slot`'s job if it is still queued (a running one finishes). Returns whether one was.
pub fn drop_queued(&mut self, slot: Slot) -> bool {
let dropped = self.pool.reprioritize(|s, p| if *s == slot { None } else { Some(p) });
let before = self.queue.len();
self.queue.retain(|q| q.slot != slot);
let any = !dropped.is_empty() || self.queue.len() != before;
if any {
self.pending.remove(&slot);
self.requests.remove(&slot);
}
any
}
/// Drop the import review thumbnails (a new review, or the dialog closed).
pub fn forget_imports(&mut self) {
self.retain_imports(&Default::default());
@@ -1773,3 +1970,13 @@ mod color_image_tests {
assert!(color_image(&img).pixels.is_empty());
}
}
/// Which library `session` has open, for [`Renderer::thumbs_made`]: a hash of its folder (the same
/// library opened again is the same), 0 with none.
fn library_key(session: &Session) -> u64 {
use std::hash::{Hash, Hasher};
let Some(lib) = session.library.as_ref() else { return 0 };
let mut h = std::collections::hash_map::DefaultHasher::new();
lib.dir.hash(&mut h);
h.finish()
}
crates/ui-egui/src/render/thumbnail_tests.rs
+404
View File
@@ -1541,3 +1541,407 @@ fn undo_to_embedded_thumb_rejects_late_full_render() {
assert_eq!(quick.pixels.as_ref().unwrap().pixels, pixels_a);
assert!(!app.renderer.is_pending(full_slot));
}
/// PERF-CULL-2: the loupe's stand-ins made ahead are taken over at the step (no job, the texture
/// at once), a photo still wanted keeps its slot as the photos shift, and a step to a photo whose
/// stand-in is still being made shows it when it arrives.
#[test]
fn a_step_takes_over_the_camera_preview_made_ahead() {
let gate = Arc::new(std::sync::Mutex::new(()));
let mut s = Session::new();
let g = gate.clone();
s.media.preview_loader = Some(Arc::new(move |_: &str, _| {
drop(g.lock().unwrap_or_else(std::sync::PoisonError::into_inner));
Some(lightcraft_raster::Rgba8 { width: 4, height: 4, data: vec![[200, 100, 50, 255]; 16] })
}));
let raw = |id: u64| {
let mut p = Photo::new(PhotoId(id), Source::File { path: format!("r{id}.cr3") }, &format!("r{id}.cr3"), "CR3", 6000, 4000, "2026-01-01");
p.kind = lightcraft_catalog::MediaKind::Raw;
p.develop = Arc::new(p.camera_defaults());
p
};
let (mut app, _, ctx) = fixture_with(s, raw(1));
for id in 2..=4 {
app.session.catalog.apply(Op::AddPhoto { photo: Box::new(raw(id)) }).unwrap();
}
let job = |app: &mut crate::LightcraftApp, id: u64| app.session.quick_view_job(PhotoId(id), 64, true).unwrap();
let wait = |app: &mut crate::LightcraftApp, done: &dyn Fn(&Renderer) -> bool| {
let t0 = std::time::Instant::now();
while !done(&app.renderer) {
assert!(t0.elapsed() < std::time::Duration::from_secs(20), "the stand-ins never arrived");
std::thread::sleep(std::time::Duration::from_millis(5));
app.renderer.poll(&ctx, &mut app.session);
}
};
let ahead_of = |r: &Renderer, id: u64| r.textures.iter().any(|(s, t)| matches!(s, Slot::QuickAhead(_)) && t.photo == PhotoId(id));
let jobs = vec![job(&mut app, 2), job(&mut app, 3)];
app.renderer.look_ahead(jobs, 96);
wait(&mut app, &|r| ahead_of(r, 2) && ahead_of(r, 3));
// the step to photo 2: its stand-in is the loupe's at once, nothing is made again
let q = job(&mut app, 2);
app.renderer.request_quick(Slot::Preview, q, 110);
assert!(!app.renderer.is_pending(Slot::Preview));
assert_eq!(app.renderer.textures.get(&Slot::Preview).map(|t| t.photo), Some(PhotoId(2)));
assert!(!ahead_of(&app.renderer, 2));
// the next look-ahead (3 and 4): 3 keeps its slot and is not made again
let held = gate.lock().unwrap();
let jobs = vec![job(&mut app, 3), job(&mut app, 4)];
app.renderer.look_ahead(jobs, 96);
assert!(ahead_of(&app.renderer, 3), "a photo still wanted keeps what was made for it");
let pending_ahead: Vec<Slot> = app.renderer.pending.keys().copied().filter(|s| matches!(s, Slot::QuickAhead(_))).collect();
assert_eq!(pending_ahead.len(), 1, "only photo 4 is being made: {pending_ahead:?}");
// the step to photo 4 while its stand-in is still being made: shown when it arrives
let q = job(&mut app, 4);
app.renderer.request_quick(Slot::Preview, q, 110);
drop(held);
wait(&mut app, &|r| r.textures.get(&Slot::Preview).is_some_and(|t| t.photo == PhotoId(4)) && !r.is_pending(Slot::Preview));
wait(&mut app, &|r| !r.pending.keys().any(|s| matches!(s, Slot::QuickAhead(_) | Slot::Preview)));
assert_eq!(app.renderer.textures.get(&Slot::Preview).map(|t| t.photo), Some(PhotoId(4)));
}
/// A loupe with unedited raws 1..=n whose camera previews are read at once, on a renderer
/// that runs nothing until asked (`run_inline`).
fn raw_loupe(n: u64) -> (crate::LightcraftApp, egui::Context) {
let mut s = Session::new();
s.media.preview_loader = Some(Arc::new(|_: &str, _| Some(lightcraft_raster::Rgba8 { width: 4, height: 4, data: vec![[200, 100, 50, 255]; 16] })));
let raw = |id: u64| {
let mut p = Photo::new(PhotoId(id), Source::File { path: format!("r{id}.cr3") }, &format!("r{id}.cr3"), "CR3", 6000, 4000, "2026-01-01");
p.kind = lightcraft_catalog::MediaKind::Raw;
p.develop = Arc::new(p.camera_defaults());
p
};
let (mut app, _, ctx) = fixture_with(s, raw(1));
for id in 2..=n {
app.session.catalog.apply(Op::AddPhoto { photo: Box::new(raw(id)) }).unwrap();
}
app.renderer.pool = JobPool::new(0);
(app, ctx)
}
/// The look-ahead with every photo allowed (the default preparation settings).
const ALL: crate::panels::detail::Prepare = crate::panels::detail::Prepare::Photos(lightcraft_engine::prep::Scope::All);
fn quick(app: &mut crate::LightcraftApp, id: u64, edge: usize) -> QuickJob {
app.session.quick_view_job(PhotoId(id), edge, true).unwrap()
}
/// The loupe's stand-ins ahead follow Settings ▸ Performance: none when the mode prepares only the photo on
/// screen (or none ahead), and the ones still queued are dropped then; only marked photos in the marked scope
/// (CodeRabbit on #3077).
#[test]
fn the_look_ahead_prepares_only_what_the_preparation_settings_allow() {
use crate::panels::detail::{Prepare, look_ahead};
use lightcraft_engine::prep::Scope;
let (mut app, _ctx) = raw_loupe(4);
app.session.sort.key = lightcraft_catalog::SortKey::FileName;
app.session.sort.ascending = true;
// (asked for: queued, running or made)
let ahead = |r: &Renderer| r.quick_tried.keys().filter(|s| matches!(s, Slot::QuickAhead(_))).count();
look_ahead(&mut app, PhotoId(1), 64, true, Prepare::of(5, Scope::All));
assert_eq!(ahead(&app.renderer), 3, "photos 2 to 4 are made ahead");
// the photo on screen only: nothing ahead, and what was queued goes
look_ahead(&mut app, PhotoId(1), 64, true, Prepare::of(0, Scope::OnScreen));
assert_eq!(ahead(&app.renderer), 0, "nothing is made ahead for the photo on screen only");
look_ahead(&mut app, PhotoId(1), 64, true, Prepare::of(0, Scope::All));
assert_eq!(ahead(&app.renderer), 0, "nor with no photos ahead");
// picks, rated and labelled: only those
app.session.execute("library.select", &serde_json::json!({"ids": [3]})).unwrap();
app.session.execute("photo.rate", &serde_json::json!({"rating": 2})).unwrap();
look_ahead(&mut app, PhotoId(1), 64, true, Prepare::of(5, Scope::Marked));
let k3 = quick(&mut app, 3, 64).key;
let queued: Vec<u64> = app.renderer.quick_tried.iter().filter(|(s, _)| matches!(s, Slot::QuickAhead(_))).map(|(_, k)| *k).collect();
assert_eq!(queued, vec![k3], "only the rated photo is made ahead");
}
/// A renderer reset (`forget_all`: a library switch, a colour-management change) drops the stand-ins asked
/// for ahead; the same photo asks for them again (CodeRabbit on #3577: the look-ahead took them for made).
#[test]
fn the_look_ahead_asks_again_after_a_renderer_reset() {
use crate::panels::detail::look_ahead;
let (mut app, _ctx) = raw_loupe(4);
let ahead = |r: &Renderer| r.quick_tried.keys().filter(|s| matches!(s, Slot::QuickAhead(_))).count();
look_ahead(&mut app, PhotoId(1), 64, true, ALL);
let asked = ahead(&app.renderer);
assert!(asked > 0);
app.renderer.forget_all();
assert_eq!(ahead(&app.renderer), 0);
look_ahead(&mut app, PhotoId(1), 64, true, ALL);
assert_eq!(ahead(&app.renderer), asked, "asked for again");
}
/// A stand-in still queued ahead whose photo moves up in the order (the user stepped) is
/// queued at its new place, not left behind photos now further away (CodeRabbit on #3055).
#[test]
fn a_kept_ahead_job_moves_to_its_new_place_in_the_order() {
let (mut app, _ctx) = raw_loupe(3);
let jobs = vec![quick(&mut app, 2, 64), quick(&mut app, 3, 64)];
let (k2, k3) = (jobs[0].key, jobs[1].key);
app.renderer.look_ahead(jobs, 96);
let priority_of = |r: &Renderer, key: u64| {
let slot = r.quick_tried.iter().find(|(s, k)| matches!(s, Slot::QuickAhead(_)) && **k == key).map(|(s, _)| *s).unwrap();
r.pending[&slot].1
};
assert_eq!((priority_of(&app.renderer, k2), priority_of(&app.renderer, k3)), (96, 95));
let jobs = vec![quick(&mut app, 3, 64), quick(&mut app, 2, 64)];
app.renderer.look_ahead(jobs, 96);
assert_eq!((priority_of(&app.renderer, k3), priority_of(&app.renderer, k2)), (96, 95));
assert_eq!(app.renderer.pool.queued(), 2, "nothing is queued twice");
}
/// A stand-in made ahead that is no longer wanted while it is being made is thrown away when it
/// arrives, not kept as a hidden texture (CodeRabbit on #3077); one the loupe waits for is kept.
#[test]
fn an_ahead_stand_in_no_longer_wanted_is_dropped_when_it_arrives() {
let gate = Arc::new(std::sync::Mutex::new(()));
let mut s = Session::new();
let g = gate.clone();
s.media.preview_loader = Some(Arc::new(move |_: &str, _| {
drop(g.lock().unwrap_or_else(std::sync::PoisonError::into_inner));
Some(lightcraft_raster::Rgba8 { width: 4, height: 4, data: vec![[200, 100, 50, 255]; 16] })
}));
let raw = |id: u64| {
let mut p = Photo::new(PhotoId(id), Source::File { path: format!("r{id}.cr3") }, &format!("r{id}.cr3"), "CR3", 6000, 4000, "2026-01-01");
p.kind = lightcraft_catalog::MediaKind::Raw;
p.develop = Arc::new(p.camera_defaults());
p
};
let (mut app, _, ctx) = fixture_with(s, raw(1));
app.session.catalog.apply(Op::AddPhoto { photo: Box::new(raw(2)) }).unwrap();
let wait = |app: &mut crate::LightcraftApp, done: &dyn Fn(&Renderer) -> bool, what: &str| {
let t0 = std::time::Instant::now();
while !done(&app.renderer) {
assert!(t0.elapsed() < std::time::Duration::from_secs(20), "{what}");
std::thread::sleep(std::time::Duration::from_millis(5));
app.renderer.poll(&ctx, &mut app.session);
}
};
let any_ahead = |r: &Renderer| r.textures.keys().any(|s| matches!(s, Slot::QuickAhead(_)));
let held = gate.lock().unwrap();
let job = quick(&mut app, 2, 64);
app.renderer.look_ahead(vec![job], 96);
wait(&mut app, &|r| r.pool.queued() == 0, "the stand-in never started");
// not wanted any more (the user went elsewhere) while it is being made
app.renderer.look_ahead(Vec::new(), 96);
assert!(!app.renderer.pending.keys().any(|s| matches!(s, Slot::QuickAhead(_))));
let done = app.renderer.completed;
drop(held);
wait(&mut app, &|r| r.completed > done, "the stand-in never finished");
assert!(!any_ahead(&app.renderer), "a stand-in nobody wants is left as a hidden texture");
// the step to photo 2 while its stand-in is being made: the loupe takes it when it arrives
let held = gate.lock().unwrap();
let job = quick(&mut app, 2, 32);
app.renderer.look_ahead(vec![job], 96);
wait(&mut app, &|r| r.pool.queued() == 0, "the stand-in never started");
let job = quick(&mut app, 2, 32);
app.renderer.request_quick(Slot::Preview, job, 110);
app.renderer.look_ahead(Vec::new(), 96);
drop(held);
wait(&mut app, &|r| r.textures.get(&Slot::Preview).is_some_and(|t| t.photo == PhotoId(2)), "the loupe never got its stand-in");
}
/// The loupe waits for a stand-in made ahead that is still being made when the next look-ahead
/// brings new photos: its slot stays reserved, so the result still reaches the loupe (CodeRabbit
/// on #3055 and #3080).
#[test]
fn a_running_stand_in_the_loupe_waits_for_keeps_its_slot() {
// the first camera preview read (photo 2's, made ahead) waits for `first`, every later one
// for `later`
let first = Arc::new(std::sync::Mutex::new(()));
let later = Arc::new(std::sync::Mutex::new(()));
let calls = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let mut s = Session::new();
let (f, l, c) = (first.clone(), later.clone(), calls.clone());
s.media.preview_loader = Some(Arc::new(move |_: &str, _| {
let gate = if c.fetch_add(1, std::sync::atomic::Ordering::SeqCst) == 0 { &f } else { &l };
drop(gate.lock().unwrap_or_else(std::sync::PoisonError::into_inner));
Some(lightcraft_raster::Rgba8 { width: 4, height: 4, data: vec![[200, 100, 50, 255]; 16] })
}));
let raw = |id: u64| {
let mut p = Photo::new(PhotoId(id), Source::File { path: format!("r{id}.cr3") }, &format!("r{id}.cr3"), "CR3", 6000, 4000, "2026-01-01");
p.kind = lightcraft_catalog::MediaKind::Raw;
p.develop = Arc::new(p.camera_defaults());
p
};
let (mut app, _, ctx) = fixture_with(s, raw(1));
for id in 2..=3 {
app.session.catalog.apply(Op::AddPhoto { photo: Box::new(raw(id)) }).unwrap();
}
app.renderer.pool = JobPool::new(4);
let wait = |app: &mut crate::LightcraftApp, done: &dyn Fn(&Renderer) -> bool, what: &str| {
let t0 = std::time::Instant::now();
while !done(&app.renderer) {
assert!(t0.elapsed() < std::time::Duration::from_secs(20), "{what}");
std::thread::sleep(std::time::Duration::from_millis(5));
app.renderer.poll(&ctx, &mut app.session);
}
};
let held_first = first.lock().unwrap();
let held_later = later.lock().unwrap();
let job = quick(&mut app, 2, 64);
app.renderer.look_ahead(vec![job], 96);
wait(&mut app, &|r| r.pool.queued() == 0, "photo 2's stand-in never started");
wait(&mut app, &|_| calls.load(std::sync::atomic::Ordering::SeqCst) == 1, "photo 2's preview was never read");
// the step to photo 2, then the next look-ahead, for photos 3 and 1
let job = quick(&mut app, 2, 64);
app.renderer.request_quick(Slot::Preview, job, 110);
assert!(app.renderer.is_pending(Slot::Preview));
let jobs = vec![quick(&mut app, 3, 64), quick(&mut app, 1, 64)];
app.renderer.look_ahead(jobs, 96);
let done = app.renderer.completed;
drop(held_first);
wait(&mut app, &|r| r.completed > done, "photo 2's stand-in never finished");
assert_eq!(app.renderer.textures.get(&Slot::Preview).map(|t| t.photo), Some(PhotoId(2)), "the loupe lost the stand-in it waited for");
drop(held_later);
wait(&mut app, &|r| r.pool.queued() == 0 && !r.pending.keys().any(|s| matches!(s, Slot::QuickAhead(_))), "the look-ahead never finished");
}
/// The stand-in made ahead still shows when the loupe's own request for it failed first
/// (CodeRabbit on #3055).
#[test]
fn a_stand_in_made_ahead_shows_after_the_loupes_own_request_failed() {
let (mut app, ctx) = raw_loupe(2);
let job = quick(&mut app, 2, 64);
let key = job.key;
app.renderer.look_ahead(vec![job], 96);
let job = quick(&mut app, 2, 64);
app.renderer.request_quick(Slot::Preview, job, 110);
assert!(app.renderer.is_pending(Slot::Preview));
// the loupe's request fails (its job never runs)
assert!(app.renderer.drop_queued(Slot::Preview));
app.renderer.failed.insert(Slot::Preview, (key, "unreadable".into(), 0, PhotoId(2)));
assert_eq!(app.renderer.pool.run_inline(1), 1);
app.renderer.poll(&ctx, &mut app.session);
assert_eq!(app.renderer.textures.get(&Slot::Preview).map(|t| t.photo), Some(PhotoId(2)));
assert!(!app.renderer.failed.contains_key(&Slot::Preview));
// the user has stepped on to photo 3 when photo 2's stand-in arrives: 3's request stays
let (mut app, ctx) = raw_loupe(3);
let job = quick(&mut app, 2, 64);
let key = job.key;
app.renderer.look_ahead(vec![job], 96);
let job = quick(&mut app, 2, 64);
app.renderer.request_quick(Slot::Preview, job, 110);
assert!(app.renderer.drop_queued(Slot::Preview));
app.renderer.failed.insert(Slot::Preview, (key, "unreadable".into(), 0, PhotoId(2)));
let job = quick(&mut app, 3, 64);
let key3 = job.key;
// (queued behind the stand-in made ahead, so that one arrives first)
app.renderer.request_quick(Slot::Preview, job, 50);
let ahead = app.renderer.pool.run_inline(1);
assert_eq!(ahead, 1);
app.renderer.poll(&ctx, &mut app.session);
assert!(app.renderer.textures.get(&Slot::Preview).is_none_or(|t| t.photo != PhotoId(2)), "photo 2 took photo 3's place");
assert_eq!(app.renderer.pending.get(&Slot::Preview).map(|p| p.0), Some(key3), "photo 3's request is kept");
}
/// A stand-in made ahead for a smaller loupe isn't taken over when the loupe has grown: the
/// larger one is made (CodeRabbit on #3077).
#[test]
fn a_stand_in_made_ahead_for_a_smaller_loupe_is_not_taken_over() {
let (mut app, ctx) = raw_loupe(2);
let small = quick(&mut app, 2, 32);
let large = quick(&mut app, 2, 64);
assert_eq!(small.key, large.key, "the same stand-in, at another size");
app.renderer.look_ahead(vec![small], 96);
assert_eq!(app.renderer.pool.run_inline(1), 1);
app.renderer.poll(&ctx, &mut app.session);
assert!(app.renderer.textures.keys().any(|s| matches!(s, Slot::QuickAhead(_))));
app.renderer.request_quick(Slot::Preview, large, 110);
assert!(app.renderer.is_pending(Slot::Preview), "the larger stand-in is made");
assert!(!app.renderer.textures.contains_key(&Slot::Preview));
// the same size is taken over at once
let (mut app, ctx) = raw_loupe(2);
let job = quick(&mut app, 2, 64);
app.renderer.look_ahead(vec![job], 96);
assert_eq!(app.renderer.pool.run_inline(1), 1);
app.renderer.poll(&ctx, &mut app.session);
let job = quick(&mut app, 2, 64);
app.renderer.request_quick(Slot::Preview, job, 110);
assert!(!app.renderer.is_pending(Slot::Preview));
assert_eq!(app.renderer.textures.get(&Slot::Preview).map(|t| t.photo), Some(PhotoId(2)));
}
/// The benchmark's thumbnail count is per photo: a render replacing a photo's stand-in isn't
/// counted again (CodeRabbit on #3055).
#[test]
fn a_thumbnail_replacing_its_stand_in_is_not_counted_again() {
let mut s = Session::new();
s.media.preview_loader = Some(Arc::new(|_, _| Some(lightcraft_raster::Rgba8 { width: 4, height: 4, data: vec![[200, 100, 50, 255]; 16] })));
let mut raw = (*photo()).clone();
raw.kind = lightcraft_catalog::MediaKind::Raw;
raw.develop = Arc::new(raw.camera_defaults());
let (mut app, work, ctx) = fixture_with(s, raw);
app.renderer.pool = JobPool::new(0);
let id = PhotoId(1);
crate::panels::grid::request_thumb(&mut app, id, 384, 10);
assert_eq!(app.renderer.pool.run_inline(1), 1);
app.renderer.poll(&ctx, &mut app.session);
assert!(app.renderer.textures.contains_key(&Slot::ThumbQuick(id)));
assert_eq!(app.renderer.thumbs_made, 1);
let mut edited = (*app.session.catalog.photo(id).unwrap().develop).clone();
edited.light.exposure += 0.5;
app.session.commit("Exposure", Op::SetDevelop { id, settings: Arc::new(edited), label: "Exposure".into(), edited: None }).unwrap();
crate::panels::grid::request_thumb(&mut app, id, 384, 10);
let (slot, job) = take_job(&work);
assert_eq!(slot, Slot::Thumb(id));
finish(&mut app, &work, &ctx, slot, job.run());
assert!(app.renderer.textures.contains_key(&Slot::Thumb(id)), "the render replaced the stand-in");
assert_eq!(app.renderer.thumbs_made, 1, "still one photo");
// evicted and made again (scrolled away and back): the same photo (CodeRabbit on #3055)
app.renderer.evict_thumbs(&Default::default(), 0);
assert!(!app.renderer.textures.contains_key(&Slot::Thumb(id)), "evicted");
crate::panels::grid::request_thumb(&mut app, id, 384, 10);
let (slot, job) = take_job(&work);
finish(&mut app, &work, &ctx, slot, job.run());
assert!(app.renderer.textures.contains_key(&slot), "made again");
assert_eq!(app.renderer.thumbs_made, 1, "an evicted photo made again is not counted again");
}
/// The loupe's look-ahead asks again when only the crop changes (its stand-ins' keys include the
/// crop), instead of keeping stand-ins the loupe can no longer use (CodeRabbit on #3077).
#[test]
fn the_look_ahead_asks_again_when_only_the_crop_changes() {
let (mut app, _ctx) = raw_loupe(3);
let ahead_keys = |r: &Renderer| r.quick_tried.iter().filter(|(s, _)| matches!(s, Slot::QuickAhead(_))).map(|(_, k)| *k).collect::<Vec<u64>>();
crate::panels::detail::look_ahead(&mut app, PhotoId(1), 64, true, ALL);
let cropped = app.session.quick_view_job(PhotoId(2), 64, true).unwrap().key;
assert!(ahead_keys(&app.renderer).contains(&cropped));
crate::panels::detail::look_ahead(&mut app, PhotoId(1), 64, false, ALL);
let uncropped = app.session.quick_view_job(PhotoId(2), 64, false).unwrap().key;
assert_ne!(cropped, uncropped);
assert!(ahead_keys(&app.renderer).contains(&uncropped), "the stand-ins are made again without the crop");
}
/// The loupe's look-ahead asks again when the view's order changes around the same photo (a new
/// sort, filter or source changes its neighbours without an edit; CodeRabbit on #3055).
#[test]
fn the_look_ahead_follows_a_new_order_around_the_same_photo() {
let (mut app, _ctx) = raw_loupe(5);
let ahead_keys = |r: &Renderer| r.quick_tried.iter().filter(|(s, _)| matches!(s, Slot::QuickAhead(_))).map(|(_, k)| *k).collect::<Vec<u64>>();
app.session.sort.key = lightcraft_catalog::SortKey::FileName;
app.session.sort.ascending = true;
crate::panels::detail::look_ahead(&mut app, PhotoId(3), 64, true, ALL);
let one = quick(&mut app, 1, 64).key;
assert!(!ahead_keys(&app.renderer).contains(&one), "r1 is two behind: not prepared");
// reversed: r1 is two ahead now
app.session.sort.ascending = false;
crate::panels::detail::look_ahead(&mut app, PhotoId(3), 64, true, ALL);
assert!(ahead_keys(&app.renderer).contains(&one), "the new neighbours are prepared");
}
/// Stepping fast (an arrow held down), the raw renders still queued for the photo left behind are
/// dropped, the Before view's too (CodeRabbit on #3055); the whole-frame render on screen stays.
#[test]
fn stepping_fast_drops_the_queued_before_render() {
let (mut app, _work, ctx) = fixture();
app.renderer.set_offload(Box::new(Blocked));
for slot in [Slot::Main, Slot::Before, Slot::Region, Slot::RegionBefore] {
let job = app.session.render_job(PhotoId(1), 64, 64, false, true).unwrap();
app.renderer.request(slot, job, 100);
}
app.renderer.poll(&ctx, &mut app.session);
assert!(app.renderer.is_pending(Slot::Before), "queued behind the blocked worker");
crate::panels::detail::drop_unsettled(&mut app.renderer, PhotoId(2));
for slot in [Slot::Main, Slot::Before, Slot::Region, Slot::RegionBefore] {
assert!(!app.renderer.is_pending(slot), "{slot:?} still queued");
}
}
crates/ui-egui/src/state/settings.rs
+16
View File
@@ -166,6 +166,11 @@ pub struct AppSettings {
/// View ▸ View Options ▸ Show message when loading or rendering photos: the loupe says
/// "Rendering…" while the photo it shows is not the finished render.
pub loupe_messages: bool,
/// Settings ▸ Performance ▸ Prepare photos in the background, app-wide (a library may keep its
/// own in its `prefs.json`): handed to the session at launch and kept in step with it. Read
/// leniently: a value this build doesn't know is the defaults, never the other settings lost.
#[serde(deserialize_with = "lenient")]
pub background_prep: lightcraft_engine::prep::PrepSettings,
/// Shortcuts the user changed (Help ▸ Keyboard Shortcuts): command id → shortcut, `""` = none.
pub keymap: crate::shortcuts::Keymap,
/// Auto (follow the system), Dark or Light.
@@ -238,6 +243,7 @@ impl Default for AppSettings {
film_badges: true,
grid_badges: GridBadges::Auto,
loupe_messages: true,
background_prep: Default::default(),
keymap: Default::default(),
appearance_mode: AppearanceMode::Dark,
dark_theme: DarkTheme::Charcoal,
@@ -430,6 +436,16 @@ fn deserialize_favorite_themes<'de, D: serde::Deserializer<'de>>(d: D) -> Result
mod loupe_info_tests {
use super::*;
/// A background preparation value this build doesn't know (a newer build's scope) reads as the
/// defaults and keeps the rest of the settings.
#[test]
fn an_unknown_background_preparation_value_costs_nothing_else() {
let v = serde_json::json!({"filmNames": false, "backgroundPrep": {"culling": {"scope": "someday"}}});
let read: AppSettings = serde_json::from_value(v).unwrap();
assert_eq!(read.background_prep, lightcraft_engine::prep::PrepSettings::default());
assert!(!read.film_names);
}
/// The lines chosen in View Options are saved with the app's settings; a settings file from
/// before they existed, or with a line this build doesn't know, gets the default lines and
/// keeps the rest of its settings.
crates/ui-egui/src/tests.rs
+2
View File
@@ -24,6 +24,7 @@ mod dialog_settings;
mod dialog_settings_text;
mod dialog_tab;
mod dialogs;
mod disk_space;
mod docking_acceptance;
mod drop_import;
mod edit_layout;
@@ -80,6 +81,7 @@ mod recheck;
mod rename_person;
mod rules_editor;
mod sam_setup;
mod scan_probes;
mod scroll;
mod slider_reset;
mod slideshow;
crates/ui-egui/src/tests/disk_space.rs
+151
View File
@@ -0,0 +1,151 @@
//! The low-disk notice: once per session, when the drive holding the preview cache runs low, with a
//! button to Settings ▸ Performance (where the cache's size limit is).
use std::time::Duration;
use lightcraft_engine::disk_space::DiskSpace;
use serde_json::json;
use crate::headless::Headless;
use crate::state::Dialog;
use crate::{LightcraftApp, Services};
const T: Duration = Duration::from_secs(20);
const GB: u64 = 1 << 30;
fn has(h: &Headless, id: &str) -> bool {
h.app.widgets.iter().any(|(w, _)| w == id)
}
#[test]
fn a_low_drive_is_told_once_with_a_way_to_the_performance_settings() {
let mut app = LightcraftApp::new(lightcraft_engine::Session::with_demo(), Services::default());
app.disk_space_notice = true;
let mut h = Headless::new(app, [1300.0, 900.0], 1.0);
h.step();
assert!(h.app.notices.is_empty(), "no notice before the drive is low");
// 12 GB free of a 500 GB drive: under 20 GB and under 10 %
assert!(h.app.session.note_disk_space(DiskSpace { available: 12 * GB, total: 500 * GB }));
h.step();
h.step();
assert_eq!(h.app.notices.len(), 1, "{:?}", h.app.notices);
let text = h.app.notices[0].clone();
assert!(text.contains("12.0 GB free of 500 GB") && text.contains("Settings → Performance"), "{text}");
let inspect = h.request("ui.inspect", json!({}), T);
assert_eq!(inspect["result"]["notices"][0], text.as_str(), "{inspect}");
assert!(has(&h, "button:noticeSettings") && has(&h, "button:noticeOk"), "the notice offers the settings");
let r = h.request("ui.clickWidget", json!({"id": "button:noticeSettings"}), T);
assert_eq!(r["ok"], true, "{r}");
assert!(h.app.notices.is_empty(), "the button dismisses the notice");
assert_eq!(h.app.ui.dialog, Some(Dialog::Settings { tab: "performance".into() }), "and opens Settings ▸ Performance");
// lower still, later in the same session: not again
assert!(!h.app.session.note_disk_space(DiskSpace { available: 2 * GB, total: 500 * GB }));
h.step();
h.step();
assert!(h.app.notices.is_empty(), "once per session");
}
#[test]
fn without_the_desktop_flag_no_notice_is_shown() {
let mut h = Headless::new(LightcraftApp::new(lightcraft_engine::Session::with_demo(), Services::default()), [1300.0, 900.0], 1.0);
assert!(h.app.session.note_disk_space(DiskSpace { available: GB, total: 500 * GB }));
h.step();
assert!(h.app.notices.is_empty(), "headless sessions and tests don't show it");
}
/// A notice still queued when another library opens belongs to the library before: it goes, and
/// the new library's drive is judged on its own.
#[test]
fn a_queued_notice_goes_when_another_library_opens() {
let dir = std::env::temp_dir().join(format!("lc-disk-notice-switch-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let mut app = LightcraftApp::new(lightcraft_engine::Session::with_demo().with_fs(), Services::default());
app.disk_space_notice = true;
let mut h = Headless::new(app, [1300.0, 900.0], 1.0);
h.step();
assert!(h.app.session.note_disk_space(DiskSpace { available: GB, total: 500 * GB }));
h.step();
assert_eq!(h.app.notices.len(), 1, "{:?}", h.app.notices);
// another library opens before the notice was answered
h.app.session.open_library(&dir, false).unwrap();
h.step();
h.step();
assert!(h.app.notices.iter().all(|n| !n.contains("1.0 GB free of 500 GB")), "the previous library's notice is gone: {:?}", h.app.notices);
let _ = h.app.session.close_library();
let _ = std::fs::remove_dir_all(&dir);
}
/// The drive is read again every few minutes whatever is filling the cache, not only while
/// previews are built (CodeRabbit on #3080): thumbnails and the loupe's previews alone can take a
/// drive under the line after the library opened.
#[test]
fn a_drive_that_runs_low_without_a_preview_build_is_told() {
use std::sync::atomic::{AtomicBool, Ordering};
let dir = std::env::temp_dir().join(format!("lc-disk-notice-later-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let low = std::sync::Arc::new(AtomicBool::new(false));
let mut session = lightcraft_engine::Session::with_demo().with_fs();
let l = low.clone();
session.set_disk_space_reader(Some(std::sync::Arc::new(move |_: &std::path::Path| {
Some(DiskSpace { available: if l.load(Ordering::Relaxed) { GB } else { 400 * GB }, total: 500 * GB })
})));
let mut app = LightcraftApp::new(session, Services::default());
app.disk_space_notice = true;
let mut h = Headless::new(app, [1300.0, 900.0], 1.0);
h.app.session.open_library(&dir, false).unwrap();
// (the open's read answered: a pass before that would see no alert yet whatever the drive holds)
assert!(h.step_until(T, |h| !h.app.session.disk_space_reading() && h.app.session.preview_build.is_none()));
assert!(!h.app.session.disk_space_alert_raised());
for _ in 0..5 {
h.step();
}
assert!(h.app.notices.is_empty(), "plenty of space at open: {:?}", h.app.notices);
// later, the drive fills up while nothing builds previews
low.store(true, Ordering::Relaxed);
h.app.session.forget_disk_space_check();
assert!(h.app.session.preview_build.as_ref().is_none_or(|b| !b.running()));
assert!(h.step_until(T, |h| !h.app.notices.is_empty()), "the low drive is told");
assert!(h.app.notices[0].contains("1.0 GB free of 500 GB"), "{:?}", h.app.notices);
let _ = h.app.session.close_library();
let _ = std::fs::remove_dir_all(&dir);
}
/// An idle window still takes the drive's answer and makes the next check (CodeRabbit on #3077):
/// while a read runs the notice logic asks for a pass soon, and otherwise for one when the next
/// check is due; once the alert was raised it asks for nothing.
#[test]
fn an_idle_window_wakes_for_the_disk_check() {
use std::sync::{Arc, Mutex, mpsc};
let dir = std::env::temp_dir().join(format!("lc-disk-notice-idle-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let (release, wait) = mpsc::channel::<()>();
let wait = Mutex::new(wait);
let mut session = lightcraft_engine::Session::with_demo().with_fs();
session.set_disk_space_reader(Some(Arc::new(move |_: &std::path::Path| {
let _ = wait.lock().unwrap().recv_timeout(Duration::from_secs(10));
Some(DiskSpace { available: GB, total: 500 * GB })
})));
let mut app = LightcraftApp::new(session, Services::default());
app.disk_space_notice = true;
app.session.open_library(&dir, false).unwrap();
assert!(app.session.disk_space_reading());
let ctx = egui::Context::default();
let pass = |app: &mut LightcraftApp| {
let mut out = ctx.run_ui(egui::RawInput::default(), |_| crate::panels::notices::logic(app, &ctx));
out.textures_delta.clear();
out.viewport_output.values().map(|v| v.repaint_delay).min().unwrap_or(Duration::MAX)
};
assert!(pass(&mut app) <= Duration::from_millis(250), "a pass soon while the read runs");
release.send(()).unwrap();
let until = std::time::Instant::now() + T;
while app.notices.is_empty() {
assert!(std::time::Instant::now() < until, "the answer was never taken");
pass(&mut app);
}
assert!(pass(&mut app) > Duration::from_secs(60), "raised: nothing more to wake for");
let _ = app.session.close_library();
let _ = std::fs::remove_dir_all(&dir);
}
crates/ui-egui/src/tests/drop_import.rs
+2 -3
View File
@@ -1193,13 +1193,12 @@ fn auto_import_waits_for_a_browse_that_starts_during_its_listing() {
let dir = Scratch::new("auto-browse");
let mut h = app(&dir);
second_auto_import_listing(&mut h, &dir);
let scan = crate::import::stall_scan_of(&mut h.app, true);
let tx = crate::import::scan_by_hand(&mut h.app, true);
apply_auto_import_listing(&mut h);
assert!(h.app.import.is_none(), "no Auto Import for the browse to replace");
// the browse finds one photo of its own and imports it
let Ok(tx) = scan.downcast::<std::sync::mpsc::Sender<ScanOutput>>() else { panic!("the stalled scan's sender") };
let found = ImportCandidate { path: dir.path("trip/a.png"), ..Default::default() };
tx.send(ScanOutput { candidates: vec![found], probes: Default::default() }).unwrap();
tx.send(crate::import::ScanEnd::Done(ScanOutput { candidates: vec![found], probes: Default::default() })).unwrap();
drop(tx);
assert!(h.step_until(T, |h| h.app.import.is_some()), "the browse starts its import");
assert!(h.step_until(T, |h| h.app.import.is_none()), "and it ends");
crates/ui-egui/src/tests/lifecycle_037.rs
+40
View File
@@ -2332,3 +2332,43 @@ fn synthetic_modifier_ownership_follows_host_changes_across_frames() {
assert_eq!(app.synthetic_mods, egui::Modifiers::NONE);
assert!(!app.synthetic_mods_release);
}
/// Background preparation is advanced on every logic pass, the hidden ones too (CodeRabbit on
/// #3077): a minimised window keeps preparing the view's previews, and a shown frame ticks it once.
#[test]
fn background_preparation_advances_while_the_window_is_hidden() {
let mut app = app();
let ctx = egui::Context::default();
let hidden = viewport_raw(egui::vec2(800.0, 600.0), false);
let _ = ctx.run_logic(&hidden, |ctx| app.logic(ctx));
let _ = ctx.run_logic(&hidden, |ctx| app.logic(ctx));
assert_eq!(app.prep_ticks, 2, "each hidden pass advances it");
let visible = viewport_raw(egui::vec2(800.0, 600.0), true);
let _ = ctx.run_logic(&visible, |ctx| app.logic(ctx));
assert_eq!(app.prep_ticks, 3, "a shown pass advances it once");
}
/// CodeRabbit on #3077/#3080: a background build that couldn't start asks for a frame at its retry
/// time, so the retry happens with no input and no other timed frame.
#[test]
fn a_build_waiting_to_retry_asks_for_a_frame_at_its_time() {
let mut app = app();
let ctx = egui::Context::default();
let delays = std::sync::Arc::new(std::sync::Mutex::new(Vec::new()));
let seen = delays.clone();
ctx.set_request_repaint_callback(move |info| seen.lock().unwrap().push(info.delay));
let wait = std::time::Duration::from_secs(7);
let hidden = viewport_raw(egui::vec2(800.0, 600.0), false);
// (the first passes ask for frames at once, which would hide a later request)
for _ in 0..3 {
let _ = ctx.run_logic(&hidden, |ctx| app.logic(ctx));
}
delays.lock().unwrap().clear();
app.session.prep.retry_at = Some((0, std::time::Instant::now() + wait));
let _ = ctx.run_logic(&hidden, |ctx| app.logic(ctx));
let got = delays.lock().unwrap().clone();
assert!(got.iter().any(|d| *d > std::time::Duration::from_secs(6) && *d <= wait), "a frame at the retry time: {got:?}");
app.session.prep.retry_at = None;
delays.lock().unwrap().clear();
let _ = ctx.run_logic(&hidden, |ctx| app.logic(ctx));
assert!(!delays.lock().unwrap().iter().any(|d| *d > std::time::Duration::from_secs(6) && *d <= wait), "none without a retry waiting");
}
crates/ui-egui/src/tests/preview_limit.rs
+72
View File
@@ -1290,6 +1290,38 @@ mod in_the_loupe {
assert!(window[0].1.right() >= canvas.right() - 0.5, "the Before window stops at the wipe line: {:?}", window[0].1);
}
// Given Before and After side by side at 1:1 with `\` held, the After pane is still the view
// shown: a Before window drawn while the After pane shows a stand-in does not make the loupe
// say the render is on screen
#[test]
fn holding_backslash_side_by_side_the_before_window_is_not_the_shown_render() {
let (_fx, mut h, _) = detail_in([1800.0, 900.0]);
h.app.ui.before_after = crate::state::BeforeAfter::SideBySide;
h.request("engine.execute", json!({"command": "view.zoom100"}), T);
h.settle(SETTLE);
h.events.push(egui::Event::Key {
key: egui::Key::Backslash,
physical_key: Some(egui::Key::Backslash),
pressed: true,
repeat: false,
modifiers: egui::Modifiers::NONE,
});
h.settle(SETTLE);
// (the key's own shortcut switches to the Before view; side by side is chosen again with the key down)
h.app.ui.before_after = crate::state::BeforeAfter::SideBySide;
h.settle(SETTLE);
assert!(h.app.renderer.textures.contains_key(&Slot::RegionBefore), "the Before window is rendered");
// the After pane back to a stand-in: no whole-frame render and no window of its own
h.app.renderer.textures.remove(&Slot::Main);
h.app.renderer.textures.remove(&Slot::Region);
h.step();
assert!(!drawn(&h, Slot::RegionBefore).is_empty(), "the Before window is painted");
// (the step must not bring the After pane's render back, or the check below proves nothing)
assert!(!h.app.renderer.textures.contains_key(&Slot::Main), "the whole-frame render came back during the step");
assert!(!h.app.renderer.textures.contains_key(&Slot::Region), "the After window came back during the step");
assert_ne!(h.app.loupe_shown.map(|l| l.1), Some("render"), "the After pane shows a stand-in, not the render");
}
// Given Before and After side by side at 1:1, each window is painted inside its own pane: its
// margin (a window is wider than what its pane shows) does not spill into the neighbour
#[test]
@@ -1306,6 +1338,33 @@ mod in_the_loupe {
assert!(after.iter().all(|(_, clip, _)| clip.left() >= canvas.center().x), "the After window spills into the Before pane: {after:?}");
}
// Given Before and After side by side at 1:1 with `\` held, both panes stay, so each window
// stays clipped to its own pane: the Before window's margin does not cover the After pane
#[test]
fn holding_backslash_side_by_side_keeps_each_window_in_its_pane() {
let (_fx, mut h, _) = detail_in([1800.0, 900.0]);
h.app.ui.before_after = crate::state::BeforeAfter::SideBySide;
h.request("engine.execute", json!({"command": "view.zoom100"}), T);
h.settle(SETTLE);
h.events.push(egui::Event::Key {
key: egui::Key::Backslash,
physical_key: Some(egui::Key::Backslash),
pressed: true,
repeat: false,
modifiers: egui::Modifiers::NONE,
});
h.settle(SETTLE);
// (the key's own shortcut switches to the Before view; side by side is chosen again with the key down)
h.app.ui.before_after = crate::state::BeforeAfter::SideBySide;
h.settle(SETTLE);
h.step();
let canvas = h.app.canvas_rect.unwrap();
let (before, after) = (drawn(&h, Slot::RegionBefore), drawn(&h, Slot::Region));
assert!(!before.is_empty() && !after.is_empty());
assert!(before.iter().all(|(_, clip, _)| clip.right() <= canvas.center().x), "the Before window spills into the After pane: {before:?}");
assert!(after.iter().all(|(_, clip, _)| clip.left() >= canvas.center().x), "the After window spills into the Before pane: {after:?}");
}
// Given a Before and an After with the same look (an unedited photo), both windows are kept
// apart: the job key is the same, and one tile must not replace the other
#[test]
@@ -1583,3 +1642,16 @@ mod in_the_loupe {
assert_ne!(after.settings, before.settings);
}
}
// Build Standard-Sized Previews on Automatic: the loupe's size once one was drawn (the engine's standard), the
// standard 2560 px in a fresh session, not the engine's 2048 px minimum a fit view would render again over
// (CodeRabbit on #3077); a chosen limit as is
#[test]
fn standard_build_keeps_the_standard_size_before_a_loupe_was_drawn() {
use crate::menubar::standard_build_params;
use serde_json::json;
assert_eq!(standard_build_params(&with_limit(0), 0), json!({"size": "standard", "edge": STANDARD_PREVIEW_EDGE}));
assert_eq!(standard_build_params(&with_limit(0), 2400), json!({"size": "standard"}));
assert_eq!(standard_build_params(&with_limit(3840), 0), json!({"size": "standard", "edge": 3840}));
assert_eq!(standard_build_params(&with_limit(3840), 2400), json!({"size": "standard", "edge": 3840}));
}
crates/ui-egui/src/tests/scan_probes.rs
+204
View File
@@ -0,0 +1,204 @@
//! A scan takes the probes of earlier scans from the session (`ScanInput::new`) and hands them back
//! however it ends: finished, stopped by another folder or by ✕, or unable to list its folder, so a later
//! import doesn't read those files again.
use std::time::Duration;
use crate::LightcraftApp;
use crate::headless::Headless;
use crate::import::ScanEnd;
const T: Duration = Duration::from_secs(10);
const KEPT: &str = "/nowhere/kept.cr3";
fn demo() -> Headless {
let services = crate::Services { png: None, ..Default::default() };
Headless::new(LightcraftApp::new(lightcraft_engine::Session::with_demo(), services), [1000.0, 700.0], 1.0)
}
fn probes() -> std::collections::HashMap<String, lightcraft_engine::media::ProbeInfo> {
[(KEPT.to_string(), lightcraft_engine::media::ProbeInfo { file_size: 7, ..Default::default() })].into_iter().collect()
}
fn kept(h: &Headless) -> bool {
h.app.session.import_probes.get(KEPT).is_some_and(|p| p.file_size == 7)
}
fn failed_toast(h: &Headless) -> bool {
h.app.ui.toast.as_ref().is_some_and(|t| t.0.contains("Scan failed"))
}
fn cancel_scan(h: &mut Headless) {
let id = h.app.session.activity.list().iter().find(|t| t.kind == "scan").map(|t| t.id).expect("the scan's row");
h.app.session.activity.cancel(id).expect("the scan can be cancelled");
}
#[test]
fn a_browse_replaced_by_another_folder_hands_back_its_probes() {
let mut h = demo();
let base = std::env::temp_dir().join(format!("lc-ui-scan-probes-{}", std::process::id()));
let (a, b) = (base.join("a"), base.join("b"));
std::fs::create_dir_all(&a).unwrap();
std::fs::create_dir_all(&b).unwrap();
h.app.session.import_probes = probes();
crate::import::browse(&mut h.app, &a.to_string_lossy(), None).unwrap();
assert!(h.app.session.import_probes.is_empty(), "the first browse took the probes");
// another folder at once: the first browse is stopped (or its result left unread)
crate::import::browse(&mut h.app, &b.to_string_lossy(), None).unwrap();
assert!(h.step_until(T, |h| h.app.scan.is_none() && kept(h)), "the first browse's probes came back");
let _ = std::fs::remove_dir_all(&base);
}
#[test]
fn a_browse_that_cannot_list_its_folder_hands_back_its_probes() {
let mut h = demo();
let tx = crate::import::scan_by_hand(&mut h.app, true);
tx.send(ScanEnd::Stopped { probes: probes(), failed: true }).unwrap();
h.step();
assert!(h.app.scan.is_none());
assert!(kept(&h), "the probes came back");
assert!(failed_toast(&h), "the failure is reported: {:?}", h.app.ui.toast);
}
#[test]
fn a_scan_closed_with_its_cross_hands_back_its_probes_when_the_worker_ends() {
let mut h = demo();
let tx = crate::import::scan_by_hand(&mut h.app, false);
cancel_scan(&mut h);
h.step();
assert!(h.app.scan.is_none(), "✕ closes the scan at once");
// the worker ends later, with the probes it took
tx.send(ScanEnd::Done(lightcraft_engine::import::ScanOutput { candidates: Vec::new(), probes: probes() })).unwrap();
assert!(h.step_until(T, kept), "the probes came back");
assert!(h.app.ui.dialog.is_none(), "no review opened");
assert!(!failed_toast(&h));
}
#[test]
fn a_cancelled_scan_whose_worker_ends_without_a_result_is_not_a_failure() {
let mut h = demo();
let tx = crate::import::scan_by_hand(&mut h.app, true);
cancel_scan(&mut h);
drop(tx);
h.step();
assert!(h.app.scan.is_none());
assert!(!failed_toast(&h), "a cancelled scan says nothing: {:?}", h.app.ui.toast);
}
/// Two stopped scans hand back a probe of one file: the later scan read it again, so its probe stays,
/// whichever worker ends first (CodeRabbit on #3577: the older one won when its worker ended first,
/// and its content hash is stale for a file changed at the same size).
#[test]
fn the_later_stopped_scans_probe_wins_whichever_worker_ends_first() {
let probe = |hash: &str| {
let p = lightcraft_engine::media::ProbeInfo { file_size: 7, content_hash: Some(hash.into()), ..Default::default() };
[(KEPT.to_string(), p)].into_iter().collect()
};
for later_ends_first in [false, true] {
let mut h = demo();
let older = crate::import::scan_by_hand(&mut h.app, true);
cancel_scan(&mut h);
h.step();
let newer = crate::import::scan_by_hand(&mut h.app, true);
cancel_scan(&mut h);
h.step();
assert!(h.app.scan.is_none(), "both stopped");
let a = ScanEnd::Stopped { probes: probe("older"), failed: false };
let b = ScanEnd::Stopped { probes: probe("newer"), failed: false };
if later_ends_first {
newer.send(b).unwrap();
h.step();
older.send(a).unwrap();
} else {
older.send(a).unwrap();
h.step();
newer.send(b).unwrap();
}
h.step();
let hash = h.app.session.import_probes.get(KEPT).and_then(|p| p.content_hash.clone());
assert_eq!(hash.as_deref(), Some("newer"), "later worker ends first: {later_ends_first}");
}
}
/// Browsing reads the disk only on its worker (CodeRabbit on #3077): a folder whose resolution is held (a link on
/// a stalled network mount) still lets `browse` return at once, the view shows the folder as given, and the scan
/// waits; once resolved, the folder is read.
#[test]
fn a_browse_whose_folder_resolves_slowly_returns_at_once_and_waits() {
let mut h = demo();
let dir = std::env::temp_dir().join(format!("lc-ui-browse-held-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
// (spelled with `sub/..`: the worker is handed the path as given, not the view's spelling of it)
let given = dir.join("sub").join("..").to_string_lossy().to_string();
let spelled = lightcraft_engine::import::browse_spelling(&dir).unwrap().to_string_lossy().trim_end_matches(['/', '\\']).to_string();
crate::import::HELD_RESOLVES.lock().unwrap().push(given.clone());
let start = std::time::Instant::now();
let r = crate::import::browse(&mut h.app, &given, None).unwrap();
assert!(start.elapsed() < Duration::from_secs(2), "returned after {:?}", start.elapsed());
assert_eq!(r["scanning"], true, "{r}");
assert_eq!(h.app.session.browse.as_ref().map(|b| b.path.clone()), Some(spelled), "shown as spelled meanwhile");
for _ in 0..5 {
h.step();
}
assert!(h.app.scan.is_some(), "still waiting for the folder");
crate::import::HELD_RESOLVES.lock().unwrap().retain(|p| *p != given);
assert!(h.step_until(T, |h| h.app.scan.is_none()), "read once resolved");
assert!(!failed_toast(&h), "{:?}", h.app.ui.toast);
let _ = std::fs::remove_dir_all(&dir);
}
/// A path that isn't a folder is found out by the worker, not by the window's thread: `browse` returns, and the scan
/// then ends as failed.
#[test]
fn a_browse_of_a_missing_folder_fails_on_its_worker() {
let mut h = demo();
let gone = std::env::temp_dir().join(format!("lc-ui-browse-gone-{}", std::process::id())).join("not-there");
assert!(crate::import::browse(&mut h.app, &gone.to_string_lossy(), None).is_ok());
assert!(h.step_until(T, |h| h.app.scan.is_none()));
assert!(failed_toast(&h), "the failure is reported: {:?}", h.app.ui.toast);
}
/// The worker resolves the path as given (CodeRabbit on #3055 / #3077 / #3080): on Unix `photos/link/../select`, with
/// `link` pointing at `real/day`, is `real/select`, not the `photos/select` that taking `..` out by name would read.
#[cfg(unix)]
#[test]
fn a_browse_through_a_link_and_dot_dot_reads_the_folder_the_system_names() {
let mut h = demo();
let base = std::env::temp_dir().join(format!("lc-ui-browse-link-{}", std::process::id()));
let (day, real_select, photos) = (base.join("real/day"), base.join("real/select"), base.join("photos"));
for d in [&day, &real_select, &photos.join("select")] {
std::fs::create_dir_all(d).unwrap();
}
std::os::unix::fs::symlink(&day, photos.join("link")).unwrap();
let given = photos.join("link").join("..").join("select");
crate::import::browse(&mut h.app, &given.to_string_lossy(), None).unwrap();
assert!(h.step_until(T, |h| h.app.scan.is_none()), "the browse ended");
assert!(!failed_toast(&h), "{:?}", h.app.ui.toast);
let want = std::fs::canonicalize(&real_select).unwrap();
let shown = h.app.session.browse.as_ref().map(|b| std::fs::canonicalize(&b.path).unwrap());
assert_eq!(shown, Some(want), "the folder the system names, not photos/select");
let _ = std::fs::remove_dir_all(&base);
}
/// Browsing a folder that is being read again keeps its progress, but not through `..` (CodeRabbit on #3583): past
/// a link, `x/link/../y` can be another folder than the `x/y` it is spelled as, so the worker reads it again.
#[test]
fn a_browse_through_dot_dot_reads_again_while_its_spelling_is_being_read() {
let mut h = demo();
let dir = std::env::temp_dir().join(format!("lc-ui-browse-dotdot-{}", std::process::id()));
std::fs::create_dir_all(dir.join("sub")).unwrap();
let plain = dir.to_string_lossy().to_string();
crate::import::HELD_RESOLVES.lock().unwrap().push(plain.clone());
let row = |h: &Headless| h.app.session.activity.list().iter().find(|t| t.kind == "scan").map(|t| t.id);
crate::import::browse(&mut h.app, &plain, None).unwrap();
let first = row(&h);
assert!(first.is_some(), "the browse's row");
crate::import::browse(&mut h.app, &plain, None).unwrap();
assert_eq!(row(&h), first, "the same folder again: the same read");
let given = dir.join("sub").join("..").to_string_lossy().to_string();
crate::import::browse(&mut h.app, &given, None).unwrap();
assert_ne!(row(&h), first, "through `..`: read again");
crate::import::HELD_RESOLVES.lock().unwrap().retain(|p| *p != plain);
assert!(h.step_until(T, |h| h.app.scan.is_none()), "the browse ended");
let _ = std::fs::remove_dir_all(&dir);
}
docs/control-protocol.md
+1 -1
View File
@@ -89,7 +89,7 @@ filmstrip is visible.
| `engine.commands` | — | Engine + UI commands: id, label, menu, shortcut, params doc, enabled |
| `ui.menu.list` | — | Menu entries (flat: id, label, menu path, shortcut, enabled) |
| `ui.menu.tree` | — | The menu bar as shown (File … Help): items `{id, params?, label, shortcut?, enabled, checked?}`, separators, submenus — the model behind the native macOS menu bar and the in-window menus |
| `ui.inspect` | — | UI state, window, `loupe` (`source`, `pending`, `failed` = why its render failed, else `null`, and `region: {full, window, pending}` when a zoomed view also renders the window on screen at no more than 100 % — `full` is the frame it is cut from, `window` its `[x, y, w, h]` — and `regionBefore` for the Before side of a Before/After view), canvas/image rects, `scroll: {grid, filmstrip}` (scroll offsets in points, `null` until drawn), active photo, selection, perf (`frameMs` = layout, `logicMs` = per-frame logic before it, `updateMs` = both, `maxUpdateMs`, `fps`, render queue …, `gpu` = adapter in use, `gpuReason` = why renders don't use the GPU, `gpuFallback` = latest render redone on the CPU and why, `gpuWindowDevice` / `screenTextures` / `mainOnGpu` = renders run on the window's device / can be drawn from the GPU / the loupe shows one now — see `docs/gpu-pipeline.md`), status, memory (bytes per cache, see `library.memory`; plus stage caches — `budgetBytes`, `trimmed`, `sharedSourceBytes` (the one device copy of a big original) — and textures), `export: {running: {total, done, current} \| null, last}`, `notices` (warnings waiting to be shown, e.g. a damaged settings file; OK = `button:noticeOk`), `quitPrompt` (why quitting was stopped: tasks that would be cut short are running — `button:quitAnyway` cancels them and quits, `button:quitCancel` keeps working — or unsaved changes: `button:quitRetry` / `button:quitAnyway` / `button:quitCancel`), `import: {done, total, imported, cancelled} \| null` (an import runs on a worker thread), `tasks` (other background work: `Find Missing Photos`, `Auto Import`), `activity` (every long-running task in flight as the activity stack in the top-left corner shows it, same shape as `activity.list`; a row appears once its task is 0.5 s old, ids `activity:row:<id>`, ✕ = `activity:cancel:<id>`, `activity:more` expands past 3 rows), `fileDialogs` (commands waiting on a native file dialog, which runs off the UI thread; they run again with the answer when it closes), `copied` (the text the UI last put on the clipboard, `null` until something is copied) |
| `ui.inspect` | — | UI state, window, `loupe` (`source`, `pending`, `failed` = why its render failed, else `null`, and `region: {full, window, pending}` when a zoomed view also renders the window on screen at no more than 100 % — `full` is the frame it is cut from, `window` its `[x, y, w, h]` — and `regionBefore` for the Before side of a Before/After view), canvas/image rects, `scroll: {grid, filmstrip}` (scroll offsets in points, `null` until drawn), active photo, selection, perf (`frameMs` = layout, `logicMs` = per-frame logic before it, `updateMs` = both, `maxUpdateMs`, `fps`, render queue …, `gpu` = adapter in use, `gpuReason` = why renders don't use the GPU, `gpuFallback` = latest render redone on the CPU and why, `gpuWindowDevice` / `screenTextures` / `mainOnGpu` = renders run on the window's device / can be drawn from the GPU / the loupe shows one now — see `docs/gpu-pipeline.md`, `loupeSteps` = how the loupe showed each photo it was stepped to, measured on the frames it drew: `{steps: [{photo, anyMs, anySource, fullMs, fullSource, sharpMs, dwellMs}], total, current}` — ms from the step to the first frame with anything of the photo, with a full-size picture (`embedded` camera JPEG, `cached` or `render`), and with LightCraft's own render; `dwellMs` is how long it stayed, the latest 256 steps; `total` counts every step finished since launch, a cursor that keeps rising when old steps leave the list; `thumbsMade` = photos given a grid thumbnail since launch (never lowered when old thumbnails are evicted, unlike `thumbTextures`); `previewCache` = the rendered-preview cache: `memEntries` / `memBytes` in memory, the disk cache's `diskHits`, `diskMisses`, `diskWrites` and `diskBudget` (bytes; all `null` without a library on disk), and `loupeEdge` (px, the loupe's longest edge, the size standard previews are built at)), status, memory (bytes per cache, see `library.memory`; plus stage caches — `budgetBytes`, `trimmed`, `sharedSourceBytes` (the one device copy of a big original) — and textures), `export: {running: {total, done, current} \| null, last}`, `notices` (warnings waiting to be shown, e.g. a damaged settings file, or once per session the drive holding the preview cache running low; OK = `button:noticeOk`, and the low-drive notice's `button:noticeSettings` opens Settings ▸ Performance), `quitPrompt` (why quitting was stopped: tasks that would be cut short are running — `button:quitAnyway` cancels them and quits, `button:quitCancel` keeps working — or unsaved changes: `button:quitRetry` / `button:quitAnyway` / `button:quitCancel`), `import: {done, total, imported, cancelled} \| null` (an import runs on a worker thread), `tasks` (other background work: `Find Missing Photos`, `Auto Import`), `activity` (every long-running task in flight as the activity stack in the top-left corner shows it, same shape as `activity.list`; a row appears once its task is 0.5 s old, ids `activity:row:<id>`, ✕ = `activity:cancel:<id>`, `activity:more` expands past 3 rows), `fileDialogs` (commands waiting on a native file dialog, which runs off the UI thread; they run again with the answer when it closes), `copied` (the text the UI last put on the clipboard, `null` until something is copied) |
| `ui.widgets` | `{filter?}` | On-screen widgets `{id, rect: [x, y, w, h]}` (screen points) |
| `ui.clickWidget` / `ui.dragWidget` / `ui.hoverWidget` | `{id, count?, button?, fx?, fy?}` (`button: "right"` right-clicks; `count` is capped at 16) / `{id, toX?, toY?, dx?, dy?, steps?}` (steps are capped at 1000) / `{id, fx?, fy?}` | Real egui input on a widget (hover: the pointer rests on it, e.g. for preset/profile previews). `fx`/`fy` are fractions of the registered rect and may lie outside `[0,1]`; the resulting point must still be inside egui's clipped interaction rect, and a non-finite fraction or resolved point is rejected before any input is queued. Geometry comes from the latest completed layout; if the widget moves before the first event or before a press, or its latest clipped rect no longer contains the point, the request fails with an error instead of clicking another widget or retargeting. A target below an ordinary modal dialog may succeed as a no-op or a backdrop click, because egui routes that input to the dialog; a target covered by a non-modal overlay fails. The blocking library-problem/launch-list window is explicit: its dim takes the modal layer, but a background target fails instead of reporting a no-op success, and only that window's own controls (`button:library…`) pass. Its creation frame is shielded and redrawn with empty input like a prompt's, so background text/pointer input cannot act while the new modal areas settle. A quit prompt or settings notice is a real egui modal scope with a foreground backdrop. Only its own buttons (`button:quit…`, `button:noticeOk`) accept a `ui.clickWidget`; background targets fail, and background keystrokes, text/clipboard input, wheel/zoom input and AccessKit actions are dropped before panels see them. The prompt itself takes Tab/Shift+Tab, Enter/Space on the focused button, and Escape/Enter for the safe default Cancel; AccessKit Focus/Click works when targeted at a prompt button. Its first creation frame (including a Tasks→Unsaved or notice→next-notice transition) is shielded and redrawn with empty input, so a press/release in that frame, or a held press from before it, cannot act underneath it; a synthetic drag already in progress is canceled. Once settled, raw `ui.click`/`ui.drag`/`ui.pointer` and backdrop clicks are still delivered to the prompt backdrop as no-ops, and the prompt's own button clicks work normally. The area/layer check cannot see a different widget covering the point in the same layer. A `ui.clickWidget` whose widget disappears before its release also fails, after canceling the pointer press without a click or drag (the same rules apply when a click spans several host frames) |
| `ui.move` / `ui.click` / `ui.drag` | `{x, y, count?, button?}` / `{x, y, toX, toY, steps?}` | Raw pointer input, screen points (the same click and step caps apply). In every pointer and scroll gesture a number other than `count` and `steps` must lie between −1,000,000 and 1,000,000, or the request is refused and queues nothing |
docs/gaps.md
+4 -2
View File
File diff suppressed because one or more lines are too long
docs/gpu-pipeline.md
+5
View File
@@ -428,6 +428,11 @@ view shows a GPU-resident draft reads that texture back for the picture (`Screen
- The next and previous photos in filmstrip order are prepared in the background once the current
one is rendered (decoded source kept, view render cached): stepping through photos shows the
developed image in ~50 ms.
- Culling (PERF-CULL): the stand-ins of the next five photos in the direction of travel and the one
behind are made ahead (`render::Slot::QuickAhead`) and taken over at the step, so a step shows
the camera JPEG at once; while an arrow key is held (a photo left within 0.25 s) the raw render
of a photo waits until it has stayed 0.12 s, and a render queued for a photo already left is
dropped. `ui.inspect` → `perf.loupeSteps` times each step on the frames drawn.
## Memory (M5.6)
A view's stages keep an uploaded source only up to 96 MB (a Preview-level source); a bigger one (the original a zoom window is cut from) is kept once, outside the views' stages, for all views of the photo (the Before and After windows and every pan share it; the Before window of a Before/After waits at first open until the After's job has handed the original over, or each would decode and upload its own copy) and given back when the app idles with no zoom window on screen (a zoomed view keeps it for its next pan: a 64 MP original is 800 MB to upload again; issue #323: kept per view it counted against the stage budget on every tick of a drag, and not kept every new window uploaded 288 MB again).
docs/parity-checklist.md
+13 -9
View File
File diff suppressed because one or more lines are too long
docs/raw-preview-reads.md
+18
View File
@@ -127,6 +127,24 @@ thumbnails of another shape and the grey masks an iPhone ProRAW DNG keeps at the
Sources for editing (an unsupported raw's JPEG, `load_embedded_preview`) and the camera-look fit still
use the largest JPEG, so their geometry and colour do not depend on the size level.
A Canon CR3 picks the same way among its three JPEGs: an EOS R5 C file keeps an 8192 × 5464 JPEG
track, a 1620 × 1080 `PRVW` box and a 160 × 120 `THMB`. Decoding the 8192 px one costs about 100 ms
of entropy decoding however small it is asked for (DCT scaling only saves the inverse transform),
against a few milliseconds for the 1620 px one. So a grid thumbnail and a loupe up to 1620 px read
the `PRVW`, and a bigger loupe still reads the track (`cr3_preview_box_list` lists the boxes for
`embedded_preview_covering`).
A plain 8-bit sRGB JPEG (no ICC profile, no Adobe RGB declaration from the container) is then decoded
straight to 8-bit (`lightcraft_codecs::decode_jpeg_srgb8_fit`: DCT-scaled, box-averaged in linear
light) without the float image the general decode builds; anything else keeps the general path.
`lightcraft-cli bench-cull FOLDER` times these steps (and the rest of a culling pass) on a real
folder; `tools/cull-bench.py` times them in the running app.
Measured on 20 EOS R5 C CR3s from a wedding (local NVMe SSD, machine shared with other work): the
loader at 1280 px went from 168–322 ms to **31 ms** a photo; at 2560 px the decode of the 8192 px
JPEG went from 182 to 151 ms (minimum of 15 runs; most of what remains is entropy decoding).
## Tests
- `crates/raw/src/preview.rs`: the header-only range matches `embedded_preview` byte for byte; a
tools/cull-bench.py
+336
View File
@@ -0,0 +1,336 @@
#!/usr/bin/env python3
"""Drive a running LightCraft over its control channel and time the cull-before-import flow.
usage: tools/cull-bench.py PORT FOLDER [--steps N] [--hold-ms 33] [--label NAME] [--size WxH] [--skip-browse] [--overwrite]
Start the app with its own library and settings first, e.g.
LIGHTCRAFT_CONFIG_DIR=/tmp/lc-cfg lightcraft --control 18520 --library /tmp/lc-lib --no-demo
It browses FOLDER into the Local view (with XMP auto-write off, the default, nothing is written
next to the photos), opens the first photo, steps, holds the arrow key, zooms to 100 % and rates,
and writes LABEL.json (an existing LABEL.json is kept unless --overwrite is given). The headless counterpart is `lightcraft-cli bench-cull FOLDER`.
Browse times are wall clock from this client (polling; each control call costs ~85 ms on this
machine, so they are good to ~0.1 s). Loupe times come from the app itself (`perf.loupeSteps`,
measured on the frames it draws), so they exclude the control channel's round trip.
"""
import json, os, socket, statistics, sys, threading, time
USAGE = __doc__.strip().splitlines()[2]
# (isdecimal, not isdigit: "²" is a digit int() refuses; a decimal string too long for int() too)
try:
PORT = int(sys.argv[1]) if len(sys.argv) >= 3 and sys.argv[1].isdecimal() else 0
except ValueError:
PORT = 0
if not 0 < PORT < 65536 or sys.argv[2].startswith("--"):
sys.exit(USAGE)
FOLDER = sys.argv[2]
args = sys.argv[3:]
# a run takes minutes: a misspelt option, or an option where a value belongs, stops it now
VALUED = {"--steps", "--hold-ms", "--label", "--size"}
FLAGS = {"--skip-browse", "--overwrite"}
_i = 0
while _i < len(args):
a = args[_i]
if a in VALUED:
if _i + 1 >= len(args) or args[_i + 1].startswith("--"):
sys.exit(f"{a} needs a value; {USAGE}")
_i += 2
elif a in FLAGS:
_i += 1
else:
sys.exit(f"unknown option {a!r}; {USAGE}")
def opt(name, default):
"""The value after option `name` (as `default`'s type), `default` when it isn't given; a
missing or unreadable value stops with the usage line."""
if name not in args:
return default
i = args.index(name) + 1
if i >= len(args):
sys.exit(f"{name} needs a value; {USAGE}")
try:
return type(default)(args[i])
except ValueError:
sys.exit(f"{name} {args[i]!r}: not a {type(default).__name__}; {USAGE}")
STEPS = opt("--steps", 40)
HOLD_MS = opt("--hold-ms", 33)
if STEPS < 1 or HOLD_MS < 1:
sys.exit(f"--steps and --hold-ms must be 1 or more; {USAGE}")
# the app keeps its last 256 loupe steps; each phase reads only its own, and the hold needs STEPS + 1
# of them with one to spare (a full history is how a phase tells that steps were dropped)
KEPT_STEPS = 256
if STEPS + 1 >= KEPT_STEPS:
sys.exit(f"--steps {STEPS}: the app keeps {KEPT_STEPS} loupe steps, so at most {KEPT_STEPS - 2}; {USAGE}")
LABEL = opt("--label", "run")
OVERWRITE = "--overwrite" in args
# a run takes minutes: refuse an existing result (or a link there, even a dangling one) before it
# starts, not after
if not OVERWRITE and os.path.lexists(LABEL + ".json"):
sys.exit(f"{LABEL}.json exists: pass --label NAME for a new one, or --overwrite to replace it")
class Conn:
def __init__(self):
self.s = socket.create_connection(("127.0.0.1", PORT), timeout=600)
self.s.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
self.f = self.s.makefile("rb")
def call(self, method, params=None):
self.s.sendall((json.dumps({"id": 1, "method": method, "params": params or {}}) + "\n").encode())
r = json.loads(self.f.readline())
if not r.get("ok", True):
raise RuntimeError(f"{method}: {r.get('error')}")
return r.get("result", r)
C = Conn()
def ex(command, params=None):
return C.call("engine.execute", {"command": command, "params": params or {}})
def inspect():
return C.call("ui.inspect")
def loupe(st):
return st.get("loupe") or {}
def pct(xs, p):
xs = sorted(xs)
return xs[min(len(xs) - 1, max(0, round(p / 100 * (len(xs) - 1))))]
def stats(xs):
xs = [x for x in xs if x is not None]
if not xs:
return None
return {"n": len(xs), "p50": round(statistics.median(xs), 1), "p95": round(pct(xs, 95), 1), "max": round(max(xs), 1)}
def wait(pred, timeout=120, every=0.0):
t0 = time.perf_counter()
while True:
st = inspect()
if pred(st):
return time.perf_counter() - t0, st
if time.perf_counter() - t0 > timeout:
return None, st
if every:
time.sleep(every)
def steps_total(s=None):
"""A cursor for the steps finished so far: `total` keeps counting after the app drops its oldest
steps (it keeps the latest 256); the length of the list is the fallback for builds without it."""
s = s or inspect()["perf"]["loupeSteps"]
return s.get("total", len(s["steps"]))
def steps_since(n0):
s = inspect()["perf"]["loupeSteps"]
new = steps_total(s) - n0
if new > len(s["steps"]):
# the app dropped some of this phase's steps: its numbers would be for part of it
sys.exit(f"{new} loupe steps since the phase began but the app kept {len(s['steps'])}: use fewer --steps")
return (s["steps"][-new:] if new > 0 else []), s
SKIP_BROWSE = "--skip-browse" in args
# (--skip-browse times whatever view the app shows: no folder is claimed for it)
out = {"label": LABEL, "folder": None if SKIP_BROWSE else FOLDER, "skipBrowse": SKIP_BROWSE, "holdMs": HOLD_MS}
SIZE = opt("--size", "")
if SIZE:
try:
w, h = (int(v) for v in SIZE.split("x"))
except ValueError:
sys.exit(f"--size {SIZE!r}: not WIDTHxHEIGHT; {USAGE}")
if w < 1 or h < 1:
sys.exit(f"--size {SIZE!r}: both sides must be 1 or more; {USAGE}")
C.call("ui.resize", {"width": w, "height": h})
time.sleep(1.0)
out["window"] = C.call("ui.inspect").get("window")
rtt = []
for _ in range(10):
t = time.perf_counter(); inspect(); rtt.append((time.perf_counter() - t) * 1000)
out["control_rtt_ms"] = stats(rtt)
# --- browse -------------------------------------------------------------------------------------
if not SKIP_BROWSE:
# thumbnails already in the cache aren't this browse's: count only the ones it adds. thumbsMade
# counts photos given a thumbnail (eviction doesn't lower it); thumbTextures is the fallback for
# builds without it
def thumbs_made(st):
p = st.get("perf") or {}
return p.get("thumbsMade", p.get("thumbTextures", 0))
# (thumbnails of the view before, still being made, would count as this browse's: let them finish)
def idle_renders(st):
p = st.get("perf") or {}
return not p.get("renderQueue") and not p.get("rendersInFlight")
wait(idle_renders, 30, 0.05)
pre_thumbs = thumbs_made(inspect())
t0 = time.perf_counter()
ex("library.browse", {"path": FOLDER})
first = thumbs12 = thumb1 = allp = None
thumb_times = {} # thumbnails drawn → when that many first were
empty_since = None # since when the folder has stood loaded with no photo in it
shown = 0
while time.perf_counter() - t0 < 1800:
st = inspect()
el = time.perf_counter() - t0
shown = ex("library.state").get("visibleCount", 0)
tt = max(0, thumbs_made(st) - pre_thumbs)
for k in range(1, min(12, tt) + 1):
thumb_times.setdefault(k, el)
if shown and first is None:
first = el
if tt >= 1 and thumb1 is None:
thumb1 = el
idle = st.get("import") is None and st.get("scan") is None
if first is not None and idle and allp is None:
allp = el
# a folder with no supported photo: nothing will show, so stop a few seconds after loading
if not shown and idle:
empty_since = el if empty_since is None else empty_since
if el - empty_since > 3:
break
else:
empty_since = None
# twelve thumbnails, or every photo's in a smaller folder: known once the folder has loaded
if allp is not None and thumbs12 is None:
thumbs12 = thumb_times.get(min(12, shown))
if allp is not None and (thumbs12 is not None or not shown):
break
# every thumbnail was in memory already (a folder browsed before in this session): no new
# ones will come, so this run can't time them; said in the output instead of waiting
if allp is not None and tt == 0 and el - allp > 1:
break
# a layout that never holds twelve thumbnails: give up on that number a while after loading
if allp is not None and el - allp > 30:
break
out["browse_first_photo_s"] = first
out["browse_first_thumb_s"] = thumb1
out["browse_12_thumbs_s"] = thumbs12
out["browse_all_s"] = allp
# the thumbnail times are a cold browse's: none when every one was already in memory
out["browse_thumbs_were_in_memory"] = thumb1 is None and allp is not None
out["photos"] = shown
# --- loupe on the first photo ----------------------------------------------------------------------
ex("library.selectAll")
ids = inspect().get("selection") or []
if not ids:
sys.exit(f"{FOLDER}: no photos visible to open in the loupe; check the filters")
ex("library.select", {"ids": [ids[0]]})
n0 = steps_total()
ex("view.detail")
wait(lambda s: loupe(s).get("source") in ("render", "cached") and not loupe(s).get("pending"), 120)
first_cur = inspect()["perf"]["loupeSteps"]["current"]
out["loupe_first"] = first_cur
# --- deliberate stepping: a step, then rest until it is sharp ---------------------------------------
n0 = steps_total()
for i in range(10):
ex("library.next")
wait(lambda s: loupe(s).get("source") in ("render", "cached") and not loupe(s).get("pending"), 60)
time.sleep(0.4)
ex("library.next")
time.sleep(0.3)
done, _ = steps_since(n0)
if len(done) < 10:
# (a folder of fewer than eleven photos: the last steps went nowhere)
print(f"step_rest: only {len(done)} of 10 steps were made", file=sys.stderr)
done = done[:10]
out["step_rest"] = {
"steps": len(done),
"any": stats([d["anyMs"] for d in done]),
"full": stats([d["fullMs"] for d in done]),
"sharp": stats([d["sharpMs"] for d in done]),
"fullSources": sorted({d["fullSource"] for d in done}),
}
# --- held arrow: a key every HOLD_MS ---------------------------------------------------------------
wait(lambda s: loupe(s).get("source") in ("render", "cached") and not loupe(s).get("pending"), 60)
n0 = steps_total()
conns = [Conn() for _ in range(4)]
t0 = time.perf_counter() + 0.05
sender_errors = []
def sender(k):
try:
for i in range(k, STEPS, len(conns)):
delay = t0 + i * HOLD_MS / 1000 - time.perf_counter()
if delay > 0:
time.sleep(delay)
conns[k].call("ui.key", {"key": "ArrowRight"})
except Exception as e: # a failed key: the hold's numbers would be for fewer steps than sent
sender_errors.append(f"sender {k}: {e}")
threads = [threading.Thread(target=sender, args=(k,)) for k in range(len(conns))]
[t.start() for t in threads]
[t.join() for t in threads]
if sender_errors:
sys.exit("the held arrow failed: " + "; ".join(sender_errors))
held_for = time.perf_counter() - t0
rel_t, _ = wait(lambda s: loupe(s).get("source") in ("render", "cached") and not loupe(s).get("pending"), 60)
time.sleep(0.2)
ex("library.next")
time.sleep(0.2)
done, _ = steps_since(n0)
held = done[:-1] # the last one is the landing photo (rested on)
landing = done[-1] if done else None
out["hold"] = {
"steps_sent": STEPS,
"photos_drawn": len(held),
"seconds": round(held_for, 2),
"any": stats([d["anyMs"] for d in held]),
"shownFull": sum(1 for d in held if d["fullMs"] is not None),
"full": stats([d["fullMs"] for d in held]),
"anySources": {k: sum(1 for d in held if d["anySource"] == k) for k in {d["anySource"] for d in held}},
"dwell": stats([d["dwellMs"] for d in held]),
"landing": landing,
"release_to_sharp_s_client": rel_t,
}
# --- zoom to 100 % ------------------------------------------------------------------------------
wait(lambda s: loupe(s).get("source") in ("render", "cached") and not loupe(s).get("pending"), 60)
t0 = time.perf_counter()
ex("view.zoom100")
cam = None
settled_whole = None # since when the whole-frame render has stood settled with no region asked for
zoom_end = None # when the zoom was done: the region's arrival, or the settled whole frame
while time.perf_counter() - t0 < 60:
st = inspect()
now = time.perf_counter() - t0
if cam is None and loupe(st).get("source") == "embedded":
cam = now
region = loupe(st).get("region")
settled = not loupe(st).get("pending") and loupe(st).get("source") in ("render", "cached")
if settled and region is not None and region.get("pending") is False:
zoom_end = now
break
# the whole-frame render already holds 100 %: no region is asked for (the app asks within a
# frame of the zoom, so half a second without one means none is coming)
if settled and region is None:
settled_whole = now if settled_whole is None else settled_whole
if now - settled_whole >= 0.5:
zoom_end = settled_whole
break
else:
settled_whole = None
# (with no region, the zoom was done when the whole-frame render stood settled, not half a second on;
# a region that came after a settled whole frame ends it when the region is ready)
t = zoom_end # None: the zoom did not finish within 60 s (missing data, not a 60 s zoom)
out["zoom100_camera_s_client"] = cam
out["zoom100_s_client"] = t
out["zoom100_region"] = loupe(st).get("region")
ex("view.zoomFit")
# --- rating / flagging latency (control round trip; the app's own time is in persistence) --------
# one photo, named on every call: the folder's filter may hide a new rating, and an id-less rating
# would then move the selection and rate the next photo; its own rating is put back afterwards
target = ex("photo.inspect")
target_id, target_rating = target["id"], target.get("rating") or 0
rate = []
try:
for i in range(20):
t0 = time.perf_counter()
ex("photo.rate", {"rating": (i % 5) + 1, "ids": [target_id]})
rate.append((time.perf_counter() - t0) * 1000)
finally:
ex("photo.rate", {"rating": target_rating, "ids": [target_id]})
out["rate_cmd_ms_client"] = stats(rate)
info = ex("library.info")
out["persistence"] = {k: v for k, v in (info.get("persistence") or {}).items() if k in ("appends", "lastAppendMs", "maxAppendMs")}
p = inspect()["perf"]
out["perf"] = {k: p.get(k) for k in ("fps", "maxUpdateMs", "gpu", "rendersDone", "thumbTextures")}
print(json.dumps(out, indent=1, default=str))
open(LABEL + ".json", "w" if OVERWRITE else "x", encoding="utf-8").write(json.dumps(out, indent=1, default=str))