storytold/filmcraft
2
0
Fork 0
mirror of https://github.com/storytold/filmcraft.git synced 2026-11-02 18:09:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

platform: VideoToolbox export takes the RGBA pictures and converts them itself (#3030)

Browse Source 
Export > Hardware encoding on macOS was CPU-bound by its own colour conversion: the
encode thread converted every RGBA frame to NV12 (rgba_to_yuv420_8) while the rayon
pool rendered the next batch, so the serial conversion took 20-27 ms a 1080p frame
and the media engine waited (perf.stats: convertMs 29.6 s of a 33.7 s export). The
VideoToolbox session now takes the export's RGBA pictures as 32BGRA buffers tagged
BT.709 and converts them itself, as NVENC has since #486; the encode thread copies
8 MB a frame (0.3-0.4 ms). A VideoToolbox that refuses RGB input on the first picture
gets the planar session and the CPU conversion back, so nothing depends on it.

Apple M2 Max, A/B interleaved, median of 3: hardware-encode export of a 60 s 1080p
timeline 35.8 s -> 29.4 s (now render-bound, waitMs 18-25 s), with GPU rendering
22.2 s -> 18.7 s at 277 % -> 219 % CPU; the software-encoder export is byte-identical.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Jason Luker
2026-10-23 19:12:08 -07:00
parent 0f87f0018e
commit f98c6e20bb
7 changed files with 599 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture.md
+6
View File
@@ -298,6 +298,12 @@ file ──► codecs (MP4/MOV, MKV, audio) demux + decode, GOP-aware see
`filmcraft_export::register_format_probe`, so the H.265 format exists where an HEVC encoder does,
and choosing it is the opt-in (the Hardware encoding toggle governs H.264 only). With no software
H.265 encoder, what NVENC declines is counted and ends the export with an error that says why.
On macOS, `platform::hardware_encode` (VideoToolbox H.264, opt-in the same way, and H.265) is
registered likewise. It too takes the RGBA pictures as they are (`32BGRA` buffers tagged BT.709)
and VideoToolbox converts them to 4:2:0, so the encode thread only copies: converting on the CPU
there competed with the render of the next frames for the thread pool and cost more than the
hardware encode (20 ms a 1080p frame). A VideoToolbox that refuses RGB input on the first picture
gets the CPU's conversion into NV12 instead, and the export goes on.
- **Compositor.** `render` is the reference for monitors, thumbnails and export. `render::plan`
turns a frame into GPU layers, each with its opacity and blend mode. All 27 blend modes run on the
GPU (`filmcraft-gpu`): Normal and Dissolve with fixed-function "over" blending, the others by