Audio: mix playback 200 ms ahead so a busy Mac doesn't drop sound

Problem: playback decoded and mixed every ~10 ms device buffer inside the
cpal callback (AAC decode, effects, opening a clip's file at a cut, rayon,
locks). On a loaded Mac any stall missed the device deadline: static and
dropouts, different on every play.

Solution: play_ahead mixes on its own thread into a lock-free ring kept
200 ms ahead; the callback only copies. An underrun plays silence, is
counted, and the mixer skips the same span so sound stays locked to the
picture. Any device width plays the channels it shares with the mix.
perf.stats reports playback.audio (callbacks, underruns, missingFrames,
minLeadMs). The program mix moved to playback_mix so bench_audio can play a
real project through a simulated 48 kHz device and count dropouts.

Context: on a real 22-sequence project, load average 38-58, 15 s per cut:
mixing in the callback overran 8-170 times, mixing ahead 0 (ring never
below 149 ms). The web build keeps mixing in its callback (no threads).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

apps/filmcraft/src/audio.rs

@@ -12,6 +12,7 @@ pub struct CpalOut {
     stream: Option<cpal::Stream>,
     played: Arc<AtomicU64>,
     rate: u32,
+    channels: u16,
     hw: AudioHardwarePrefs,
     /// Sequence sample rate for "Attempt to force hardware to document sample rate".
     document_rate: Option<u32>,
@@ -44,7 +45,14 @@ impl CpalOut {
         let host = cpal::default_host();
         let dev = host.default_output_device()?;
         let cfg = dev.default_output_config().ok()?;
-        Some(Self { stream: None, played: Arc::new(AtomicU64::new(0)), rate: cfg.sample_rate().0, hw: AudioHardwarePrefs::default(), document_rate: None })
+        Some(Self {
+            stream: None,
+            played: Arc::new(AtomicU64::new(0)),
+            rate: cfg.sample_rate().0,
+            channels: cfg.channels(),
+            hw: AudioHardwarePrefs::default(),
+            document_rate: None,
+        })
     }
 
     /// The stream configuration the settings ask for, falling back to the device default.
@@ -77,6 +85,7 @@ impl AudioOut for CpalOut {
             config.buffer_size = cpal::BufferSize::Fixed(self.hw.buffer_size);
         }
         self.rate = cfg.sample_rate().0;
+        self.channels = cfg.channels();
         self.played.store(0, Ordering::SeqCst);
         let played = self.played.clone();
         let err = |e| eprintln!("filmcraft: audio stream error: {e}");
@@ -103,6 +112,9 @@ impl AudioOut for CpalOut {
     fn sample_rate(&self) -> u32 {
         self.rate
     }
+    fn channels(&self) -> usize {
+        self.channels as usize
+    }
     fn played_frames(&self) -> Option<u64> {
         self.stream.as_ref().map(|_| self.played.load(Ordering::SeqCst))
     }
@@ -122,6 +134,7 @@ impl AudioOut for CpalOut {
             && let Ok(cfg) = self.config(&dev)
         {
             self.rate = cfg.sample_rate().0;
+            self.channels = cfg.channels();
         }
     }
 }

crates/ui-egui/examples/bench_audio.rs

@@ -0,0 +1,205 @@
+//! Headless audio playback bench: plays a project's sequences through a simulated 48 kHz stereo
+//! device with a 512-frame buffer (the app's defaults) and counts dropouts, the way the device
+//! would hear them. Nothing is played out loud.
+//!
+//! ```sh
+//! FILMCRAFT_DATA_DIR=/tmp/fc-bench cargo run --release -p filmcraft-ui-egui --example bench_audio -- \
+//!     "Project.fcproj" --seconds 20 [--sequence "One hero" …] [--mode direct|ahead|both] [--json out.json] [--wav dir]
+//! ```
+//!
+//! `direct` mixes inside the device callback (how playback worked before sound was mixed ahead);
+//! `ahead` plays through [`play_ahead`] as the app does now. A callback that returns after its
+//! buffer was due is a dropout, and the device cycles it overran are lost. For `ahead` the ring's
+//! own underruns (silence played because the mixer fell behind) count too. `--wav` writes what a
+//! device would have played for each run, to listen to.
+//!
+//! The simulated device thread runs at normal priority (CoreAudio's own I/O thread runs higher),
+//! so absolute counts are pessimistic for both modes; compare the modes on the same machine load.
+
+use std::sync::Arc;
+use std::sync::atomic::Ordering;
+use std::time::{Duration, Instant};
+
+use filmcraft_engine::Session;
+use filmcraft_project::ItemId;
+use filmcraft_ui_egui::play_ahead::{self, AudioStats};
+use serde_json::{Value, json};
+
+const SR: u32 = 48_000;
+const PERIOD: usize = 512;
+const CH: usize = 2;
+
+#[derive(Default)]
+struct Run {
+    callbacks: u64,
+    /// Callbacks that returned after their buffer was due.
+    late: u64,
+    /// Callbacks whose own work took longer than one buffer: a dropout on any device.
+    overruns: u64,
+    /// Device cycles lost to late callbacks.
+    lost_cycles: u64,
+    max_callback_ms: f64,
+    /// `ahead` only: callbacks the ring could not fill.
+    ring_underruns: u64,
+    min_lead_ms: Option<f64>,
+    /// What a device whose I/O thread always wakes on time plays (interleaved stereo): a callback
+    /// that overran loses its buffer and the slots it overran to silence.
+    heard: Vec<f32>,
+}
+
+impl Run {
+    /// What a device whose I/O thread always wakes on time would hear: overrunning callbacks plus
+    /// buffers the ring could not fill. (`late` also counts this bench's normal-priority device
+    /// thread waking late under load, which CoreAudio's real-time thread does not.)
+    fn dropouts(&self) -> u64 {
+        self.overruns + self.ring_underruns
+    }
+}
+
+/// Calls `fill` once per device period for `seconds` of wall time, as a device does.
+fn device(fill: &mut dyn FnMut(&mut [f32], usize), seconds: f64) -> Run {
+    let period = PERIOD as f64 / SR as f64;
+    let total = (seconds / period) as u64;
+    let mut buf = vec![0.0f32; PERIOD * CH];
+    let mut r = Run::default();
+    let t0 = Instant::now();
+    let mut k = 0u64;
+    while k < total {
+        let due = Duration::from_secs_f64(period * k as f64);
+        if let Some(wait) = due.checked_sub(t0.elapsed()) {
+            std::thread::sleep(wait);
+        }
+        let c0 = Instant::now();
+        fill(&mut buf, CH);
+        r.callbacks += 1;
+        let took = c0.elapsed().as_secs_f64();
+        r.max_callback_ms = r.max_callback_ms.max(took * 1e3);
+        if took > period {
+            r.overruns += 1;
+            let lost = (took / period).ceil() as usize;
+            r.heard.resize(r.heard.len() + lost * PERIOD * CH, 0.0);
+        } else {
+            r.heard.extend_from_slice(&buf);
+        }
+        let done = t0.elapsed().as_secs_f64();
+        let deadline = period * (k + 1) as f64;
+        if done > deadline {
+            r.late += 1;
+            let next = (done / period).ceil() as u64;
+            r.lost_cycles += next.saturating_sub(k + 1);
+            k = next;
+        } else {
+            k += 1;
+        }
+    }
+    r
+}
+
+/// 32-bit float stereo WAV at the device rate.
+fn write_wav(path: &std::path::Path, samples: &[f32]) -> std::io::Result<()> {
+    let data = (samples.len() * 4) as u32;
+    let mut b = Vec::with_capacity(44 + data as usize);
+    b.extend_from_slice(b"RIFF");
+    b.extend_from_slice(&(36 + data).to_le_bytes());
+    b.extend_from_slice(b"WAVEfmt ");
+    b.extend_from_slice(&16u32.to_le_bytes());
+    b.extend_from_slice(&3u16.to_le_bytes()); // IEEE float
+    b.extend_from_slice(&(CH as u16).to_le_bytes());
+    b.extend_from_slice(&SR.to_le_bytes());
+    b.extend_from_slice(&(SR * CH as u32 * 4).to_le_bytes());
+    b.extend_from_slice(&(CH as u16 * 4).to_le_bytes());
+    b.extend_from_slice(&32u16.to_le_bytes());
+    b.extend_from_slice(b"data");
+    b.extend_from_slice(&data.to_le_bytes());
+    for x in samples {
+        b.extend_from_slice(&x.to_le_bytes());
+    }
+    std::fs::write(path, b)
+}
+
+fn play(s: &Session, seq: ItemId, mode: &str, seconds: f64) -> Run {
+    let mix = filmcraft_ui_egui::playback_mix(s, seq, SR);
+    match mode {
+        "direct" => {
+            let mut cursor = 0i64;
+            let mut fill = move |buf: &mut [f32], ch: usize| {
+                mix(cursor, buf, ch);
+                cursor += (buf.len() / ch) as i64;
+            };
+            device(&mut fill, seconds)
+        }
+        _ => {
+            let stats = Arc::new(AudioStats::default());
+            let mut fill = play_ahead::spawn(mix, 0, SR, CH, stats.clone());
+            let mut r = device(&mut *fill, seconds);
+            r.ring_underruns = stats.underruns.load(Ordering::Relaxed);
+            let min = stats.min_lead_frames.load(Ordering::Relaxed);
+            r.min_lead_ms = (min != u64::MAX).then(|| min as f64 * 1e3 / SR as f64);
+            r
+        }
+    }
+}
+
+fn main() {
+    filmcraft_platform::register();
+    let mut args = std::env::args().skip(1);
+    let (mut project, mut seconds, mut only, mut mode, mut json_out, mut wav_dir) = (None, 20.0, Vec::new(), "both".to_string(), None, None);
+    while let Some(a) = args.next() {
+        match a.as_str() {
+            "--seconds" => seconds = args.next().and_then(|v| v.parse().ok()).unwrap_or(seconds),
+            "--sequence" => only.extend(args.next()),
+            "--mode" => mode = args.next().unwrap_or(mode),
+            "--json" => json_out = args.next(),
+            "--wav" => wav_dir = args.next().map(std::path::PathBuf::from),
+            p => project = Some(p.to_string()),
+        }
+    }
+    let Some(project) = project else {
+        eprintln!("usage: bench_audio <project.fcproj> [--seconds N] [--sequence NAME] [--mode direct|ahead|both] [--json PATH] [--wav DIR]");
+        std::process::exit(2);
+    };
+    let mut s = Session::default();
+    if let Err(e) = s.execute("file.open", json!({ "path": project })) {
+        eprintln!("{project}: {e}");
+        std::process::exit(1);
+    }
+    let mut seqs: Vec<(ItemId, String)> = s.project.sequences().map(|i| (i.id, i.name.clone())).filter(|(_, n)| only.is_empty() || only.contains(n)).collect();
+    seqs.sort_by(|a, b| a.1.cmp(&b.1));
+    let modes: Vec<&str> = if mode == "both" { vec!["direct", "ahead"] } else { vec![mode.as_str()] };
+    println!(
+        "{:<48} {:<7} {:>9} {:>8} {:>9} {:>9} {:>8} {:>11} {:>9}",
+        "sequence", "mode", "callbacks", "dropouts", "overruns", "underruns", "late", "max cb (ms)", "min lead"
+    );
+    let mut out = Vec::<Value>::new();
+    for (id, name) in &seqs {
+        for m in &modes {
+            let r = play(&s, *id, m, seconds);
+            if let Some(d) = &wav_dir
+                && let Err(e) = write_wav(&d.join(format!("{name} - {m}.wav")), &r.heard)
+            {
+                eprintln!("{}: {e}", d.display());
+            }
+            let lost_ms = r.lost_cycles as f64 * PERIOD as f64 * 1e3 / SR as f64;
+            let lead = r.min_lead_ms.map_or("-".to_string(), |v| format!("{v:.0} ms"));
+            println!(
+                "{:<48} {:<7} {:>9} {:>8} {:>9} {:>9} {:>8} {:>11.1} {:>9}",
+                name,
+                m,
+                r.callbacks,
+                r.dropouts(),
+                r.overruns,
+                r.ring_underruns,
+                r.late,
+                r.max_callback_ms,
+                lead
+            );
+            out.push(json!({"sequence": name, "mode": m, "seconds": seconds, "callbacks": r.callbacks, "dropouts": r.dropouts(), "overruns": r.overruns, "late": r.late,
+                "lostMs": lost_ms, "ringUnderruns": r.ring_underruns, "maxCallbackMs": r.max_callback_ms, "minLeadMs": r.min_lead_ms}));
+        }
+    }
+    if let Some(p) = json_out
+        && let Err(e) = std::fs::write(&p, serde_json::to_string_pretty(&out).unwrap_or_default())
+    {
+        eprintln!("{p}: {e}");
+    }
+}

crates/ui-egui/src/lib.rs

@@ -19,6 +19,8 @@ pub mod links;
 pub mod menus;
 pub mod panels;
 pub mod perf;
+#[cfg(not(target_arch = "wasm32"))]
+pub mod play_ahead;
 pub mod state;
 pub mod theme;
 pub mod widgets;
@@ -45,6 +47,10 @@ pub trait AudioOut {
     fn stop(&mut self);
     /// Device sample rate.
     fn sample_rate(&self) -> u32;
+    /// Device output channels (what `fill` will be called with).
+    fn channels(&self) -> usize {
+        2
+    }
     /// Frames played since `start` (the playback master clock), if the device reports it.
     fn played_frames(&self) -> Option<u64>;
     /// Hosts and devices that can be chosen in Settings ▸ Audio Hardware.
@@ -118,6 +124,9 @@ pub struct Playback {
     anchor_tick: Tick,
     /// Audio frames played at anchor (when the audio clock drives).
     pub audio_clock: bool,
+    /// Audio underruns for the current (or last) play (desktop: sound is mixed ahead).
+    #[cfg(not(target_arch = "wasm32"))]
+    pub audio_stats: std::sync::Arc<play_ahead::AudioStats>,
     /// Shown / dropped frame accounting for the current (or last) play.
     pub meter: frames::PlaybackMeter,
     /// Waiting for the first frames before starting the clock: when the wait began (egui time,
@@ -555,57 +564,24 @@ impl FilmcraftApp {
             return;
         }
         let Some(seq_id) = self.session.state.active_sequence else { return };
-        let project = self.session.project.clone();
-        let provider = self.session.media.provider(project.clone(), self.session.services.clone());
-        let previews = self.session.previews.clone();
         let start_tick = self.session.playhead();
-        // Settings ▸ Audio Hardware ▸ Output Mapping
-        let map = [self.session.prefs.audio_hardware.map_left, self.session.prefs.audio_hardware.map_right];
-        // Preferences ▸ Audio ▸ 5.1 Mixdown Type: how a 5.1 Mix plays on a stereo device
-        let mixdown = filmcraft_audio_dsp::channels::Mixdown::from_id(&self.session.prefs.audio.mixdown_type).unwrap_or_default();
         let Some(a) = self.audio.as_mut() else { return };
         let sr = a.sample_rate();
-        let mut cursor = start_tick.to_units_floor(sr as i64);
-        let cues = panels::voiceover::cues(&self.session, sr);
-        previews.live.publish_project(project.clone());
-        let fill = Box::new(move |buf: &mut [f32], ch: usize| {
-            // the newest project snapshot: mixer moves and other edits are heard while playing
-            let project = previews.live.project().filter(|p| p.sequence(seq_id).is_some()).unwrap_or_else(|| project.clone());
-            let Some(seq) = project.sequence(seq_id) else { return };
-            let n = buf.len() / ch.max(1);
-            // Mix at the sequence rate; convert when the device rate differs (nearest sample).
-            let seq_sr = seq.settings.sample_rate;
-            // a 5.1 Mix plays as six channels (L, R, C, LFE, Ls, Rs) on a device with at least six
-            use filmcraft_audio_dsp::channels::Layout;
-            let layout = if ch >= 6 && seq.settings.audio_master == filmcraft_project::AudioChannels::Surround51 { Layout::Surround51 } else { Layout::Stereo };
-            let mix = if seq_sr == sr {
-                previews.mix_layout(&project, seq_id, cursor, n, &provider, layout, mixdown)
-            } else {
-                let s0 = (cursor as i128 * seq_sr as i128 / sr as i128) as i64;
-                let m = n * seq_sr as usize / sr as usize + 2;
-                let b = previews.mix_layout(&project, seq_id, s0, m, &provider, layout, mixdown);
-                let mut out = filmcraft_frame::AudioBuffer::silence(sr, b.channels.len(), n);
-                for (o, c) in out.channels.iter_mut().zip(&b.channels) {
-                    for (i, x) in o.iter_mut().enumerate() {
-                        let j = (i * seq_sr as usize / sr as usize).min(m - 1);
-                        *x = c[j];
-                    }
-                }
-                out
-            };
-            if layout == Layout::Surround51 {
-                buf.fill(0.0);
-                for (i, frame) in buf.chunks_mut(ch).enumerate() {
-                    for (c, x) in frame.iter_mut().take(6).enumerate() {
-                        *x = mix.channels[c][i];
-                    }
-                }
-            } else {
-                filmcraft_engine::settings::map_output(&mix.channels[0], &mix.channels[1.min(mix.channels.len() - 1)], buf, ch, map);
-            }
-            panels::voiceover::mix_cues(buf, ch, cursor, &cues);
-            cursor += n as i64;
-        });
+        let cursor = start_tick.to_units_floor(sr as i64);
+        let mix = playback_mix(&self.session, seq_id, sr);
+        // the old stream stops before the new mixer resets the underrun counters
+        a.stop();
+        // Desktop: mix ahead on a thread so the device callback never waits on decoding.
+        #[cfg(not(target_arch = "wasm32"))]
+        let fill = play_ahead::spawn(mix, cursor, sr, a.channels(), self.playback.audio_stats.clone());
+        #[cfg(target_arch = "wasm32")]
+        let fill = {
+            let mut cursor = cursor;
+            Box::new(move |buf: &mut [f32], ch: usize| {
+                mix(cursor, buf, ch);
+                cursor += (buf.len() / ch.max(1)) as i64;
+            })
+        };
         match a.start(fill) {
             Ok(_) => self.playback.audio_clock = true,
             Err(e) => {
@@ -1313,6 +1289,61 @@ impl eframe::App for FilmcraftApp {
     }
 }
 
+/// The program mix that playback plays: `mix(device_frame, interleaved, channels)` renders the
+/// active sequence at device rate `sr` from `device_frame`, with the live project snapshot (edits
+/// made while playing are heard), Output Mapping, the 5.1 mixdown and voice-over cues.
+pub fn playback_mix(session: &Session, seq_id: filmcraft_project::ItemId, sr: u32) -> impl Fn(i64, &mut [f32], usize) + Send + 'static {
+    let project = session.project.clone();
+    let provider = session.media.provider(project.clone(), session.services.clone());
+    let previews = session.previews.clone();
+    // Settings ▸ Audio Hardware ▸ Output Mapping
+    let map = [session.prefs.audio_hardware.map_left, session.prefs.audio_hardware.map_right];
+    // Preferences ▸ Audio ▸ 5.1 Mixdown Type: how a 5.1 Mix plays on a stereo device
+    let mixdown = filmcraft_audio_dsp::channels::Mixdown::from_id(&session.prefs.audio.mixdown_type).unwrap_or_default();
+    let cues = panels::voiceover::cues(session, sr);
+    previews.live.publish_project(project.clone());
+    move |cursor: i64, buf: &mut [f32], ch: usize| {
+        // the newest project snapshot: mixer moves and other edits are heard while playing
+        let project = previews.live.project().filter(|p| p.sequence(seq_id).is_some()).unwrap_or_else(|| project.clone());
+        let Some(seq) = project.sequence(seq_id) else {
+            buf.fill(0.0);
+            return;
+        };
+        let n = buf.len() / ch.max(1);
+        // Mix at the sequence rate; convert when the device rate differs (nearest sample).
+        let seq_sr = seq.settings.sample_rate;
+        // a 5.1 Mix plays as six channels (L, R, C, LFE, Ls, Rs) on a device with at least six
+        use filmcraft_audio_dsp::channels::Layout;
+        let layout = if ch >= 6 && seq.settings.audio_master == filmcraft_project::AudioChannels::Surround51 { Layout::Surround51 } else { Layout::Stereo };
+        let mix = if seq_sr == sr {
+            previews.mix_layout(&project, seq_id, cursor, n, &provider, layout, mixdown)
+        } else {
+            let s0 = (cursor as i128 * seq_sr as i128 / sr as i128) as i64;
+            let m = n * seq_sr as usize / sr as usize + 2;
+            let b = previews.mix_layout(&project, seq_id, s0, m, &provider, layout, mixdown);
+            let mut out = filmcraft_frame::AudioBuffer::silence(sr, b.channels.len(), n);
+            for (o, c) in out.channels.iter_mut().zip(&b.channels) {
+                for (i, x) in o.iter_mut().enumerate() {
+                    let j = (i * seq_sr as usize / sr as usize).min(m - 1);
+                    *x = c[j];
+                }
+            }
+            out
+        };
+        if layout == Layout::Surround51 {
+            buf.fill(0.0);
+            for (i, frame) in buf.chunks_mut(ch).enumerate() {
+                for (c, x) in frame.iter_mut().take(6).enumerate() {
+                    *x = mix.channels[c][i];
+                }
+            }
+        } else {
+            filmcraft_engine::settings::map_output(&mix.channels[0], &mix.channels[1.min(mix.channels.len() - 1)], buf, ch, map);
+        }
+        panels::voiceover::mix_cues(buf, ch, cursor, &cues);
+    }
+}
+
 #[cfg(test)]
 mod gpu_fallback_tests {
     use filmcraft_render::plan::{FramePlan, PlanLayer};

crates/ui-egui/src/perf.rs

@@ -20,6 +20,11 @@ pub fn stats(app: &FilmcraftApp) -> Value {
         "resolution": app.ui.program.res.label(),
         "draftDecode": app.session.prefs.playback.draft_decode,
     });
+    #[cfg(not(target_arch = "wasm32"))]
+    {
+        let sr = app.audio.as_ref().map_or(0, |a| a.sample_rate());
+        v["playback"]["audio"] = app.playback.audio_stats.to_json(sr);
+    }
     let mut frames = app.frames.stats().to_json();
     frames["workers"] = json!(app.frames.workers());
     frames["queued"] = json!(app.frames.queue_len());

crates/ui-egui/src/play_ahead.rs

@@ -0,0 +1,193 @@
+//! Audio played ahead: playback mixes on its own thread into a ring buffer that stays
+//! [`LEAD_S`] ahead of the device, and the device callback only copies samples out of it.
+//!
+//! Mixing (decoding AAC, effects, opening a clip's file at a cut) used to run inside the device
+//! callback, which has one I/O buffer (~10 ms) to return. On a busy Mac any stall there missed the
+//! device's deadline, heard as static and dropouts that were different on every play. Now a stall
+//! only drains part of the lead; the callback never decodes, allocates or takes a lock.
+//!
+//! When the ring does run dry (an underrun), the callback plays silence for the missing frames and
+//! counts them, and the mixer skips the same span, so sound stays locked to the picture.
+//!
+//! Desktop only: the web build has no threads and keeps mixing in its callback.
+
+use std::sync::Arc;
+use std::sync::atomic::{AtomicBool, AtomicU32, AtomicU64, Ordering};
+use std::time::Duration;
+
+/// How far ahead of the device the mixer keeps the ring (s). Edits made while playing are heard
+/// this much later.
+pub const LEAD_S: f64 = 0.2;
+/// Frames mixed per block on the mixing thread.
+pub const BLOCK_FRAMES: usize = 1024;
+/// Ceiling on mixed channels (large interfaces and aggregate devices stay well under it).
+const MAX_CHANNELS: usize = 64;
+
+/// Underrun counters for the current (or last) play, readable from any thread (`perf.stats`).
+#[derive(Debug, Default)]
+pub struct AudioStats {
+    /// Device callbacks served.
+    pub callbacks: AtomicU64,
+    /// Callbacks the ring could not fill completely.
+    pub underruns: AtomicU64,
+    /// Frames played as silence because the ring was empty.
+    pub missing_frames: AtomicU64,
+    /// Fewest frames left in the ring after a callback, once it first filled to the lead
+    /// (`u64::MAX` until then).
+    pub min_lead_frames: AtomicU64,
+}
+
+impl AudioStats {
+    pub fn reset(&self) {
+        self.callbacks.store(0, Ordering::Relaxed);
+        self.underruns.store(0, Ordering::Relaxed);
+        self.missing_frames.store(0, Ordering::Relaxed);
+        self.min_lead_frames.store(u64::MAX, Ordering::Relaxed);
+    }
+
+    pub fn to_json(&self, sample_rate: u32) -> serde_json::Value {
+        let min = self.min_lead_frames.load(Ordering::Relaxed);
+        serde_json::json!({
+            "callbacks": self.callbacks.load(Ordering::Relaxed),
+            "underruns": self.underruns.load(Ordering::Relaxed),
+            "missingFrames": self.missing_frames.load(Ordering::Relaxed),
+            "minLeadMs": if min == u64::MAX || sample_rate == 0 { serde_json::Value::Null } else { serde_json::json!(min as f64 * 1e3 / sample_rate as f64) },
+        })
+    }
+}
+
+/// Single-producer, single-consumer ring of interleaved f32 samples (stored as bits, so it needs
+/// no `unsafe`). Indices count samples written / read since the start and only grow.
+struct Ring {
+    slots: Vec<AtomicU32>,
+    write: AtomicU64,
+    read: AtomicU64,
+}
+
+impl Ring {
+    fn new(samples: usize) -> Self {
+        Self { slots: (0..samples.max(1)).map(|_| AtomicU32::new(0)).collect(), write: AtomicU64::new(0), read: AtomicU64::new(0) }
+    }
+
+    fn len(&self) -> usize {
+        self.write.load(Ordering::Acquire).saturating_sub(self.read.load(Ordering::Acquire)) as usize
+    }
+
+    /// Producer: append as many samples as fit; returns how many were written.
+    fn push(&self, src: &[f32]) -> usize {
+        let cap = self.slots.len();
+        let w = self.write.load(Ordering::Relaxed);
+        let free = cap.saturating_sub(w.saturating_sub(self.read.load(Ordering::Acquire)) as usize);
+        let n = src.len().min(free);
+        for (i, x) in src.iter().take(n).enumerate() {
+            if let Some(s) = self.slots.get(((w + i as u64) % cap as u64) as usize) {
+                s.store(x.to_bits(), Ordering::Relaxed);
+            }
+        }
+        self.write.store(w + n as u64, Ordering::Release);
+        n
+    }
+
+    /// Consumer: fill `dst` (`dev_ch` interleaved channels) with whole frames of `ch` channels from
+    /// the ring: the first `min(ch, dev_ch)` channels copied, any further device channels zeroed.
+    /// Returns the frames copied; the rest of `dst` is left for the caller.
+    fn pop(&self, dst: &mut [f32], dev_ch: usize, ch: usize) -> usize {
+        let cap = self.slots.len();
+        let r = self.read.load(Ordering::Relaxed);
+        let avail = self.write.load(Ordering::Acquire).saturating_sub(r) as usize / ch.max(1);
+        let n = (dst.len() / dev_ch.max(1)).min(avail);
+        for (f, frame) in dst.chunks_mut(dev_ch.max(1)).take(n).enumerate() {
+            for (c, x) in frame.iter_mut().enumerate() {
+                *x = if c < ch {
+                    self.slots.get(((r + (f * ch + c) as u64) % cap as u64) as usize).map_or(0.0, |s| f32::from_bits(s.load(Ordering::Relaxed)))
+                } else {
+                    0.0
+                };
+            }
+        }
+        self.read.store(r + (n * ch) as u64, Ordering::Release);
+        n
+    }
+}
+
+/// Stops the mixing thread when the device callback that owns it is dropped (stream stopped).
+struct StopOnDrop(Arc<AtomicBool>);
+
+impl Drop for StopOnDrop {
+    fn drop(&mut self) {
+        self.0.store(true, Ordering::Release);
+    }
+}
+
+/// Mixes `mix(cursor, buf, channels)` (interleaved frames from device frame `cursor`) on its own
+/// thread, from `start`, and returns the device callback that plays it (`fill(buf, channels)`).
+/// The first block is mixed before this returns, so playback starts with sound. The thread runs
+/// until the returned callback is dropped.
+pub fn spawn<M>(mut mix: M, start: i64, sample_rate: u32, channels: usize, stats: Arc<AudioStats>) -> Box<dyn FnMut(&mut [f32], usize) + Send>
+where
+    M: FnMut(i64, &mut [f32], usize) + Send + 'static,
+{
+    stats.reset();
+    let ch = channels.clamp(1, MAX_CHANNELS);
+    let lead = ((sample_rate as f64 * LEAD_S) as usize).max(BLOCK_FRAMES * 2);
+    let ring = Arc::new(Ring::new(lead * ch));
+    let mut block = vec![0.0; BLOCK_FRAMES * ch];
+    mix(start, &mut block, ch);
+    ring.push(&block);
+    // Frames the callback played as silence that the mixer has not skipped yet.
+    let skip = Arc::new(AtomicU64::new(0));
+    let stop = Arc::new(AtomicBool::new(false));
+    {
+        let (ring, skip, stop) = (ring.clone(), skip.clone(), stop.clone());
+        let body = move || {
+            let mut cursor = start + BLOCK_FRAMES as i64;
+            while !stop.load(Ordering::Acquire) {
+                if ring.len() + block.len() > lead * ch {
+                    std::thread::sleep(Duration::from_millis(2));
+                    continue;
+                }
+                cursor += skip.swap(0, Ordering::AcqRel) as i64;
+                block.fill(0.0);
+                mix(cursor, &mut block, ch);
+                cursor += BLOCK_FRAMES as i64;
+                // silence played while this block was mixing: its first frames are already late
+                let late = skip.swap(0, Ordering::AcqRel) as usize;
+                let drop = late.min(BLOCK_FRAMES);
+                cursor += (late - drop) as i64;
+                ring.push(block.get(drop * ch..).unwrap_or_default());
+            }
+        };
+        let spawned = std::thread::Builder::new().name("filmcraft-audio-mix".into()).spawn(move || {
+            if std::panic::catch_unwind(std::panic::AssertUnwindSafe(body)).is_err() {
+                log::error!("audio mixing stopped after a panic; playback continues silent");
+            }
+        });
+        if let Err(e) = spawned {
+            log::error!("audio mixing thread: {e}");
+        }
+    }
+    let guard = StopOnDrop(stop);
+    let mut primed = false;
+    Box::new(move |buf: &mut [f32], dev_ch: usize| {
+        let _keep = &guard;
+        stats.callbacks.fetch_add(1, Ordering::Relaxed);
+        // a device wider or narrower than the mix (it changed since `channels` was read) still
+        // plays the channels both have
+        let dev_ch = dev_ch.max(1);
+        let have = ring.pop(buf, dev_ch, ch);
+        if let Some(rest) = buf.get_mut(have * dev_ch..) {
+            rest.fill(0.0);
+        }
+        let n = buf.len() / dev_ch;
+        if have < n {
+            stats.underruns.fetch_add(1, Ordering::Relaxed);
+            stats.missing_frames.fetch_add((n - have) as u64, Ordering::Relaxed);
+            skip.fetch_add((n - have) as u64, Ordering::AcqRel);
+        }
+        let left = ring.len();
+        primed |= left + 2 * BLOCK_FRAMES * ch > lead * ch;
+        if primed {
+            stats.min_lead_frames.fetch_min((left / ch) as u64, Ordering::Relaxed);
+        }
+    })
+}

crates/ui-egui/tests/play_ahead.rs

@@ -0,0 +1,126 @@
+//! Audio mixed ahead (`play_ahead`): driven by a wall-clock device. Its own test binary, because one
+//! test panics a mixer on purpose and the crash hook's tests read the process-wide last panic.
+
+use std::sync::Arc;
+use std::sync::atomic::{AtomicU64, Ordering};
+use std::time::{Duration, Instant};
+
+use filmcraft_ui_egui::play_ahead::{AudioStats, BLOCK_FRAMES, LEAD_S, spawn};
+
+const SR: u32 = 48_000;
+const PERIOD: usize = 512;
+
+/// A mixer whose every sample is its own device frame number, so the output shows which frame
+/// played where.
+fn frame_numbers(delay: Duration, calls: Arc<AtomicU64>) -> impl FnMut(i64, &mut [f32], usize) + Send + 'static {
+    move |cursor, buf, ch| {
+        calls.fetch_add(1, Ordering::Relaxed);
+        std::thread::sleep(delay);
+        for (i, f) in buf.chunks_mut(ch).enumerate() {
+            f.fill((cursor + i as i64) as f32);
+        }
+    }
+}
+
+/// Plays `callbacks` device periods, one every period on the wall clock as a device does, and
+/// returns the interleaved output.
+fn play(fill: &mut Box<dyn FnMut(&mut [f32], usize) + Send>, ch: usize, callbacks: usize) -> Vec<f32> {
+    let period = Duration::from_secs_f64(PERIOD as f64 / SR as f64);
+    let t0 = Instant::now();
+    let mut out = Vec::new();
+    for k in 0..callbacks {
+        let mut buf = vec![f32::NAN; PERIOD * ch];
+        fill(&mut buf, ch);
+        out.extend_from_slice(&buf);
+        if let Some(wait) = (period * (k as u32 + 1)).checked_sub(t0.elapsed()) {
+            std::thread::sleep(wait);
+        }
+    }
+    out
+}
+
+#[test]
+fn plays_every_frame_in_order_from_the_start() {
+    let stats = Arc::new(AudioStats::default());
+    let mut fill = spawn(frame_numbers(Duration::ZERO, Arc::default()), 1000, SR, 2, stats.clone());
+    std::thread::sleep(Duration::from_millis(30));
+    let out = play(&mut fill, 2, 60);
+    for (p, f) in out.chunks(2).enumerate() {
+        assert_eq!(f, [(1000 + p) as f32; 2], "frame {p}");
+    }
+    assert_eq!(stats.underruns.load(Ordering::Relaxed), 0);
+    assert_eq!(stats.callbacks.load(Ordering::Relaxed), 60);
+}
+
+#[test]
+fn a_mixer_slower_than_the_device_plays_silence_and_stays_in_time() {
+    // each 1024-frame block takes 40 ms to mix, about twice its 21 ms of sound
+    let stats = Arc::new(AudioStats::default());
+    let mut fill = spawn(frame_numbers(Duration::from_millis(40), Arc::default()), 1, SR, 2, stats.clone());
+    let out = play(&mut fill, 2, 120);
+    assert!(stats.underruns.load(Ordering::Relaxed) > 0);
+    assert!(stats.missing_frames.load(Ordering::Relaxed) > 0);
+    let mut heard = 0;
+    for (p, f) in out.chunks(2).enumerate() {
+        assert!(f.iter().all(|x| x.is_finite()), "frame {p} left unwritten");
+        if f[0] != 0.0 {
+            // every frame that plays is the one due at that moment: no drift after an underrun
+            assert_eq!(f[0], (1 + p) as f32, "frame {p}");
+            heard += 1;
+        }
+    }
+    assert!(heard > PERIOD, "some sound still plays");
+}
+
+#[test]
+fn a_mixer_that_panics_leaves_silence_not_a_crash() {
+    let stats = Arc::new(AudioStats::default());
+    let mut first = true;
+    let mix = move |cursor: i64, buf: &mut [f32], ch: usize| {
+        if !first {
+            panic!("mixer bug");
+        }
+        first = false;
+        frame_numbers(Duration::ZERO, Arc::default())(cursor, buf, ch);
+    };
+    let mut fill = spawn(mix, 1, SR, 2, stats.clone());
+    let out = play(&mut fill, 2, 20);
+    assert!(out.iter().all(|x| x.is_finite()));
+    assert_eq!(out.get(..2), Some(&[1.0, 1.0][..]), "the block mixed before the panic plays");
+    assert!(stats.underruns.load(Ordering::Relaxed) > 0);
+}
+
+#[test]
+fn a_device_with_other_channels_plays_the_channels_both_have() {
+    for dev_ch in [1, 6, 10, 16] {
+        let stats = Arc::new(AudioStats::default());
+        let mut fill = spawn(frame_numbers(Duration::ZERO, Arc::default()), 1, SR, 2, stats.clone());
+        let out = play(&mut fill, dev_ch, 4);
+        for (p, f) in out.chunks(dev_ch).enumerate() {
+            let want: Vec<f32> = (0..dev_ch).map(|c| if c < 2 { (1 + p) as f32 } else { 0.0 }).collect();
+            assert_eq!(f, &want[..], "{dev_ch} channels, frame {p}");
+        }
+        assert_eq!(stats.underruns.load(Ordering::Relaxed), 0, "{dev_ch} channels");
+    }
+}
+
+#[test]
+fn stopping_the_stream_stops_the_mixer() {
+    let calls = Arc::new(AtomicU64::new(0));
+    let fill = spawn(frame_numbers(Duration::ZERO, calls.clone()), 1, SR, 2, Arc::default());
+    std::thread::sleep(Duration::from_millis(50));
+    drop(fill);
+    std::thread::sleep(Duration::from_millis(20));
+    let after_stop = calls.load(Ordering::Relaxed);
+    std::thread::sleep(Duration::from_millis(50));
+    assert_eq!(calls.load(Ordering::Relaxed), after_stop);
+}
+
+#[test]
+fn the_ring_keeps_no_more_than_the_lead() {
+    let calls = Arc::new(AtomicU64::new(0));
+    let _fill = spawn(frame_numbers(Duration::ZERO, calls.clone()), 1, SR, 2, Arc::default());
+    std::thread::sleep(Duration::from_millis(100));
+    let lead_blocks = (SR as f64 * LEAD_S / BLOCK_FRAMES as f64).ceil() as u64;
+    assert!(calls.load(Ordering::Relaxed) <= lead_blocks + 1, "mixed {} blocks", calls.load(Ordering::Relaxed));
+}

docs/control-protocol.md

@@ -29,7 +29,7 @@ Methods (handlers in `crates/ui-egui/src/control.rs`):
 | `ui.playback` | `{action: play|stop|toggle, speed?}` | |
 | `ui.screenshot` | `{path?, panel?}` | PNG of the window or one panel; fails after 10 s when no frame is presented (window hidden, display asleep) |
 | `ui.resize`, `ui.focus`, `app.quit` | | `ui.focus` is the only request that activates the app and takes keyboard focus |
-| `perf.stats` | – | performance counters (also the command `perf.stats`, so `engine.execute` and MCP `command_run` reach it; headless sessions return the engine part). `decode`: GOP-cache requests / hits / `cacheHitRate`, decoder seeks, samples decoded and skipped while catching up, `draftFrames` (decoded in draft mode), `h264Threads` (frame-threading workers per H.264 decoder), `liveDecoders` (sources holding a decoder now), `cacheMB` / `cacheBudgetMB` (decoded frames all sources hold, and the shared budget), `planePoolMB` / `planesReused` (idle plane buffers of evicted frames, and planes decoded into a recycled one), decoder ms (total and per sample), `framesDecoded`, `hardware` (Settings ▸ Playback ▸ Hardware decoding: `enabled`, pictures from hardware decoders `frames` vs `softwareFrames`, `sessions`, `declined` streams, mid-stream `fallbacks`); `playback`: playing, shown / dropped frames and `dropRate` of the current or last play, resolution, `draftDecode` (Settings ▸ Playback ▸ Draft decoding); `frames`: frame-worker jobs, cancelled, `requestHitRate`, `decodeMs` / `renderMs` (mean, p50, p95 of the last 256 jobs), queue, render-cost estimate, cache use; `ui`: fps and frame ms; `process`: CPU seconds; `media`, `jobs`. Counters are cumulative: diff two readings to measure an interval ([performance.md](performance.md)) |
+| `perf.stats` | – | performance counters (also the command `perf.stats`, so `engine.execute` and MCP `command_run` reach it; headless sessions return the engine part). `decode`: GOP-cache requests / hits / `cacheHitRate`, decoder seeks, samples decoded and skipped while catching up, `draftFrames` (decoded in draft mode), `h264Threads` (frame-threading workers per H.264 decoder), `liveDecoders` (sources holding a decoder now), `cacheMB` / `cacheBudgetMB` (decoded frames all sources hold, and the shared budget), `planePoolMB` / `planesReused` (idle plane buffers of evicted frames, and planes decoded into a recycled one), decoder ms (total and per sample), `framesDecoded`, `hardware` (Settings ▸ Playback ▸ Hardware decoding: `enabled`, pictures from hardware decoders `frames` vs `softwareFrames`, `sessions`, `declined` streams, mid-stream `fallbacks`); `playback`: playing, shown / dropped frames and `dropRate` of the current or last play, resolution, `draftDecode` (Settings ▸ Playback ▸ Draft decoding), `audio` (desktop: sound is mixed 200 ms ahead of the device; device `callbacks`, `underruns` and `missingFrames` played as silence because the mixer fell behind, `minLeadMs` the least sound left buffered); `frames`: frame-worker jobs, cancelled, `requestHitRate`, `decodeMs` / `renderMs` (mean, p50, p95 of the last 256 jobs), queue, render-cost estimate, cache use; `ui`: fps and frame ms; `process`: CPU seconds; `media`, `jobs`. Counters are cumulative: diff two readings to measure an interval ([performance.md](performance.md)) |
 
 **Selection or explicit targets.** A command that acts on the selection is "not available right now" when
 nothing is selected. When `params` name the targets under a key the command documents (`clips` /

docs/testing.md

@@ -203,7 +203,11 @@ brute-force barycentric reference, `.cube`/`.3dl` round trips), `filmcraft-gpu`
 
 Not covered headless: `ui.screenshot` (needs a real viewport), the wgpu monitor path (the harness
 runs the CPU texture path), audio output, and wall-clock playback advance (kittest frames do not
-advance real time). Use `Driver` for new UI regressions: `d.exec(command, params)`,
+advance real time). Playing sound is covered without a device: `play_ahead` tests drive the
+mixed-ahead ring with a wall-clock device (order, underruns stay in time, mixer panic, stop), and
+`cargo run --release -p filmcraft-ui-egui --example bench_audio -- <project.fcproj>` plays a real
+project's sequences through a simulated 48 kHz device and counts dropouts, mixing in the callback
+(`direct`, as before) against mixing ahead (`ahead`). Use `Driver` for new UI regressions: `d.exec(command, params)`,
 `d.ok(method, params)`, `d.frames(n)`.
 
 ### Interactive checks