storytold/filmcraft
2
0
Fork 0
mirror of https://github.com/storytold/filmcraft.git synced 2026-11-02 18:09:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #3746 from aiongg/backlog-r1

Browse Source 
Integration: backlog round r1 (244 PRs)
This commit is contained in:
aiongg
2026-10-31 01:56:51 -04:00
committed by GitHub
parent 6c2825bc95 562f550221
commit 142fddc191
416 changed files with 47015 additions and 2957 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ATTRIBUTION.md
+6 -1
View File
@@ -9,12 +9,17 @@ iconography, images or other Adobe assets; open licences only; every asset attri
| Asset | Author | Source | Licence |
|---|---|---|---|
| `crates/ui-egui/src/i18n/sv.tsv` | FilmCraft contributors | Original work: clean-room Swedish interface translations | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/it.tsv` | FilmCraft contributors | Original work: clean-room Italian interface translations | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/tr.tsv` | FilmCraft contributors | Original work: clean-room Turkish interface translations | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/id.tsv` | FilmCraft contributors | Original work: clean-room Indonesian interface translations | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/ko.tsv` | FilmCraft contributors | Original work: clean-room Korean interface translations | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/cs.tsv` | FilmCraft contributors | Original work: AI-assisted Czech interface translations; not yet proofread by a native speaker | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/fr.tsv` | FilmCraft contributors | Original work: clean-room French interface translations | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/pl.tsv` | FilmCraft contributors | Original work: clean-room Polish interface translations | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/pt-br.tsv` | FilmCraft contributors | Original work: clean-room Brazilian Portuguese interface translations | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/ru.tsv` | FilmCraft contributors | Original work: clean-room Russian interface translations | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/bs.tsv` | FilmCraft contributors | Original work: AI-assisted clean-room Bosnian interface translations, not yet native-reviewed | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/ru.tsv` | FilmCraft contributors | Original work: clean-room Russian interface translation (full catalog) | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/uk.tsv` | FilmCraft contributors | Original work: clean-room Ukrainian interface translations | MIT OR Apache-2.0 |
| `assets/fonts/Inter-Regular.ttf` | The Inter Project Authors | https://github.com/rsms/inter | OFL-1.1 (`assets/fonts/OFL-Inter.txt`) |
| `assets/fonts/Inter-Medium.ttf` | The Inter Project Authors | https://github.com/rsms/inter | OFL-1.1 (`assets/fonts/OFL-Inter.txt`) |
ROADMAP.md
+9 -3
View File
@@ -5,7 +5,7 @@
**Stage: alpha** · next: beta, ~19 points of ready-for-real-work and ~400–700 h away, plus opening Premiere's `.prproj` ([why](docs/target-app-parity.md#stage); passes the [core-workflow gate](docs/roadmap.md#alpha-gate))
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-22 · **Change:** minor (ProRes alpha final-run conformance and native macOS validation) · **Target:** Adobe Premiere Pro 2026 (26.5.2)
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-22 · **Change:** minor (ProRes alpha final-run conformance and native macOS validation; language table: Czech, #1156) · **Target:** Adobe Premiere Pro 2026 (26.5.2)
Progress toward parity with Adobe Premiere Pro. This page is the summary; the detail is in
[docs/target-app-parity.md](docs/target-app-parity.md) (assessment, methodology, calibration),
@@ -72,7 +72,7 @@ Evidence for every row: [docs/target-app-parity.md](docs/target-app-parity.md#by
## Languages
Premiere ships 10 interface languages. Measured from our catalogs (3,498 strings):
Premiere ships 10 interface languages. Measured from our catalogs (3,593 strings, last measured 2026-10-19):
| Language | Status | Strings |
|---|---|---|
@@ -85,12 +85,15 @@ Premiere ships 10 interface languages. Measured from our catalogs (3,498 strings
| Portuguese (Brazil) | partial (whole interface; native review welcome) | ~100% |
| Indonesian | partial (whole interface; native review welcome) | ~100% |
| Italian | partial (whole interface; native review welcome) | ~100% |
| Bosnian | partial (whole interface; AI-assisted, native review welcome) | ~100% |
| Korean | partial (whole interface; needs an installed Hangul font; native review welcome) | ~100% |
| Czech | partial (whole interface; native review needed) | 100% |
| Russian | partial (whole interface; native review welcome) | ~100% |
| Japanese | partial (whole interface) | 99.7% |
| German | partial (whole interface; menus native-reviewed, the rest welcome) | ~100% |
| Vietnamese | none | 0% |
Also shipped: Ukrainian (menus only, 10%). Detail: [docs/localization-parity.md](docs/localization-parity.md).
Also shipped: Swedish (whole interface; native review welcome, ~100%), Ukrainian (menus only, 10%) and Polish (whole interface; AI-drafted, native review welcome). Detail: [docs/localization-parity.md](docs/localization-parity.md).
## Upcoming
@@ -109,6 +112,7 @@ Ranked; detail and "done when" in [docs/roadmap.md](docs/roadmap.md) and [docs/g
- **2026-10-22:** ProRes 4444 / XQ alpha slices include the final singleton run code required by RDD 36:2022. Six HD fixtures rejected by native macOS decoding now pass; syntax regression tests cover final runs and slice widths ([details](crates/prores/README.md#alpha-bitstream-conformance), EffectCraft #293).
- **2026-10-20:** the on/off items of the Sequence and Markers menus (Snap in Timeline, Linked Selection, Selection Follows Playhead, Show Through Edits, Ripple Sequence Markers, Copy Paste Includes Sequence Markers) show their checkmark, so choosing one visibly turns it on or off (#2623)
- **2026-10-12:** Transform's motion-blur shutter on adjustment layers uses the sequence frame rate instead of 24 fps, so a 360° shutter in a 60 fps sequence spans one frame, not 2.5 (#1088)
- **2026-10-10 (changed media):** Settings ▸ Media ▸ Automatically refresh growing files works: media files that change on disk while the project is open (a recording still being written, a render written over the file a clip uses) are read again every *Refresh growing files every* seconds and as soon as the window comes back to the front: the new length, picture and identity, without an undo step (`media.refreshChanged`, `media_watch`).
- **2026-10-10 (re-measure):** full parity re-measure against Premiere Pro 26.5.2: menu label diff 338 / 366 (92%), installed bundle listings (10 UI languages, exporter and codec frameworks), 188 open issues. Breadth ~86% (file formats now counted), ready ~55%, stage **alpha**; progress docs reorganized ([target-app-parity](docs/target-app-parity.md), [gaps](docs/gaps.md), [roadmap](docs/roadmap.md), [codec](docs/codec-parity.md), [file-format](docs/file-format-parity.md), [hardware](docs/hardware-parity.md), [UI](docs/ui-parity.md), [localization](docs/localization-parity.md)).
@@ -169,6 +173,8 @@ Ranked; detail and "done when" in [docs/roadmap.md](docs/roadmap.md) and [docs/g
| Date | Change | Summary |
|---|---|---|
| 2026-10-22 | minor | ProRes alpha: write the final singleton run code; 4444/XQ HD clips that failed native macOS decoding now decode, with syntax regression tests (EffectCraft #293) |
| 2026-10-19 | minor | Language table: Czech (#1156, 100%) |
| 2026-10-20 | trivial | Progress log: checkmarks on the Sequence and Markers menu toggles (#2623) |
| 2026-10-13 | minor | Language table: German covers the whole interface (~100%) |
| 2026-10-12 | trivial | Progress log: Transform shutter on adjustment layers (#1088) |
| 2026-10-11 | minor | Language table: German ships the menus (#630, 10.6%) |
apps/filmcraft/src/audio.rs
+38 -12
View File
@@ -61,6 +61,32 @@ impl Note {
}
}
/// Whether the settings name a sample rate the device does not already run at, so its supported
/// configurations have to be listed. `0` is the settings' "device default".
fn needs_rate_search(default_rate: u32, want: u32) -> bool {
want != 0 && want != default_rate
}
/// The stream configuration the settings ask for, falling back to the device default.
///
/// Listing a device's configurations means probing every format and rate in the driver:
/// 150-215 ms on WASAPI (measured), while opening the stream itself costs ~20 ms warm (0.5-1 s
/// from a cold audio engine). So the list is only asked for when the settings name a rate the
/// device does not already run at (#2633).
fn stream_config(hw: &AudioHardwarePrefs, document_rate: Option<u32>, dev: &cpal::Device) -> Result<cpal::SupportedStreamConfig, String> {
let default = dev.default_output_config().map_err(|e| e.to_string())?;
let want = if hw.force_document_rate { document_rate.unwrap_or(hw.sample_rate) } else { hw.sample_rate };
if !needs_rate_search(default.sample_rate().0, want) {
return Ok(default);
}
let exact = dev.supported_output_configs().ok().and_then(|it| {
it.filter(|c| c.channels() > 0 && (c.min_sample_rate().0..=c.max_sample_rate().0).contains(&want))
.max_by_key(|c| (c.channels() == default.channels(), c.sample_format() == cpal::SampleFormat::F32, c.sample_format() == default.sample_format()))
.and_then(|c| c.try_with_sample_rate(cpal::SampleRate(want)))
});
Ok(exact.unwrap_or(default))
}
/// The host named in the settings (empty or unknown = the default host).
fn host(name: &str) -> cpal::Host {
if !name.is_empty()
@@ -158,18 +184,9 @@ impl CpalOut {
}
}
/// The stream configuration the settings ask for, falling back to the device default.
/// The stream configuration the settings ask for (see [`stream_config`]).
fn config(&self, dev: &cpal::Device) -> Result<cpal::SupportedStreamConfig, String> {
let default = dev.default_output_config().map_err(|e| e.to_string())?;
let want = if self.hw.force_document_rate { self.document_rate.unwrap_or(self.hw.sample_rate) } else { self.hw.sample_rate };
let exact = dev.supported_output_configs().ok().and_then(|it| {
it.filter(|c| c.channels() > 0 && (c.min_sample_rate().0..=c.max_sample_rate().0).contains(&want))
.max_by_key(|c| {
(c.channels() == default.channels(), c.sample_format() == cpal::SampleFormat::F32, c.sample_format() == default.sample_format())
})
.and_then(|c| c.try_with_sample_rate(cpal::SampleRate(want)))
});
Ok(exact.unwrap_or(default))
stream_config(&self.hw, self.document_rate, dev)
}
fn converted_stream<T>(&self, dev: &cpal::Device, config: &cpal::StreamConfig, fill: Arc<Mutex<Fill>>) -> Result<cpal::Stream, cpal::BuildStreamError>
@@ -328,7 +345,7 @@ impl Watchdog {
// An error that arrives with the stream still consuming samples is only a hiccup.
self.pending_error |= errored;
let stalled = now.saturating_sub(self.seen.1) >= STALL_RESTART;
if !lost && !(self.pending_error && stalled) {
if !(lost || (self.pending_error && stalled)) {
return Verdict::Healthy;
}
self.restarts.retain(|t| now.saturating_sub(*t) < RESTART_WINDOW);
@@ -472,6 +489,15 @@ mod tests {
assert!(!Note::NoDevice.text().is_empty());
}
/// #2633: the rate search costs ~190 ms on WASAPI, so the device default is taken as is.
#[test]
fn the_device_default_rate_is_used_without_listing_every_configuration() {
assert!(!needs_rate_search(48_000, 48_000), "the settings ask for the rate the device runs at");
assert!(!needs_rate_search(44_100, 0), "0 is the device default");
assert!(needs_rate_search(48_000, 44_100), "another rate has to be searched for");
assert!(needs_rate_search(48_000, 96_000));
}
#[test]
fn integer_conversion_clips_and_preserves_interleaved_channels() {
let mut signed = [0_i16; 8];
apps/filmcraft/src/audio_in.rs
+39 -1
View File
@@ -16,14 +16,32 @@ pub struct CpalIn {
stop: Option<mpsc::Sender<()>>,
worker: Option<std::thread::JoinHandle<()>>,
error: Arc<Mutex<Option<String>>>,
/// Frames of `buf` already measured by [`AudioInput::take_peaks`].
metered: usize,
}
impl CpalIn {
pub fn new(host: &str) -> Self {
Self { host: host.to_string(), buf: Arc::default(), stop: None, worker: None, error: Arc::default() }
Self { host: host.to_string(), buf: Arc::default(), stop: None, worker: None, error: Arc::default(), metered: 0 }
}
}
/// At most this many trailing frames are measured per meter poll (2 s at 48 kHz), so a meter that
/// was not polled for a while doesn't scan the whole take.
const PEAK_WINDOW: usize = 96_000;
/// Per-channel peak of the frames from `from` on (only the last [`PEAK_WINDOW`]), and the new
/// measured position; no peaks when nothing new arrived.
fn peaks(buf: &[Vec<f32>], from: usize) -> (Vec<f32>, usize) {
let len = buf.first().map_or(0, Vec::len);
let start = from.max(len.saturating_sub(PEAK_WINDOW));
if start >= len {
return (Vec::new(), len);
}
let peaks = buf.iter().map(|c| c.get(start..).unwrap_or_default().iter().fold(0f32, |m, s| m.max(s.abs()))).collect();
(peaks, len)
}
fn host(name: &str) -> cpal::Host {
if !name.is_empty()
&& let Some(id) = cpal::available_hosts().into_iter().find(|h| h.name() == name)
@@ -114,6 +132,7 @@ impl AudioInput for CpalIn {
fn start(&mut self, name: &str, sample_rate: u32) -> Result<InputFormat, String> {
self.stop();
lock(&self.buf).clear();
self.metered = 0;
*lock(&self.error) = None;
let (host_name, name) = (self.host.clone(), name.to_string());
let buf = self.buf.clone();
@@ -188,6 +207,7 @@ impl AudioInput for CpalIn {
}
fn read(&mut self, frames: usize) -> Vec<Vec<f32>> {
let mut b = lock(&self.buf);
self.metered = self.metered.saturating_sub(b.first().map_or(0, |c| frames.min(c.len())));
b.iter_mut()
.map(|c| {
let n = frames.min(c.len());
@@ -195,8 +215,14 @@ impl AudioInput for CpalIn {
})
.collect()
}
fn take_peaks(&mut self) -> Vec<f32> {
let (p, metered) = peaks(&lock(&self.buf), self.metered);
self.metered = metered;
p
}
fn discard(&mut self) {
lock(&self.buf).iter_mut().for_each(Vec::clear);
self.metered = 0;
}
fn error(&self) -> Option<String> {
lock(&self.error).clone()
@@ -241,6 +267,18 @@ mod tests {
assert!(lock(&error).is_none());
}
#[test]
fn peaks_measure_only_new_frames_and_stop_when_nothing_arrives() {
let buf = vec![vec![0.1, -0.8, 0.2], vec![0.0, 0.3, -0.4]];
let (p, at) = peaks(&buf, 0);
assert_eq!((p, at), (vec![0.8, 0.4], 3));
assert_eq!(peaks(&buf, 3), (Vec::new(), 3), "nothing new: no peaks");
assert_eq!(peaks(&buf, 2).0, [0.2, 0.4], "only frames from the measured position");
assert_eq!(peaks(&[], 5), (Vec::new(), 0));
let long = vec![[vec![1.0], vec![0.25; PEAK_WINDOW]].concat()];
assert_eq!(peaks(&long, 0).0, [0.25], "only the trailing window is scanned");
}
#[test]
fn invalid_input_is_reported_without_panicking_or_appending() {
let buf = Mutex::new(Vec::new());
apps/filmcraft/src/file_filters.rs
+26 -1
View File
@@ -6,7 +6,9 @@
//! as `shot.MP4` or `shot.Mp4` would be hidden. The helper appends one bracket pattern per
//! extension (`mp4` -> `[mM][pP]4`) after the literal extensions, which stay first so save
//! dialogs keep their normal default suffix. macOS and Windows dialogs already ignore case
//! and receive the plain list.
//! and receive the plain list. The Open Project dialog also offers [`all_files`] beside the
//! project filter, so a project stays openable where the system MIME database does not
//! know `.fcproj` and the project filter matches nothing (#1644).
pub fn extensions(exts: &[&str]) -> Vec<String> {
let extensions: Vec<_> = exts.iter().map(|ext| (*ext).to_owned()).collect();
@@ -37,6 +39,15 @@ pub fn extensions(exts: &[&str]) -> Vec<String> {
extensions
}
/// Extensions for an "All files" filter: a single `*` and nothing else. rfd's XDG portal
/// backend special-cases `*` into the glob `*` (its GTK and Zenity backends build `*.*`),
/// so every file shows — including a saved project on a system whose shared-mime-info
/// package never registered `.fcproj`, where the "FilmCraft Project" filter hides them
/// all and the dialog offers no other way to reach the file (#1644).
pub fn all_files() -> Vec<String> {
extensions(&["*"])
}
#[cfg(test)]
mod tests {
use super::extensions;
@@ -55,6 +66,20 @@ mod tests {
}
}
#[test]
fn all_files_filter_admits_every_file() {
// One entry only: the `*` the backends turn into a match-everything glob (the XDG
// portal backend sends `*` itself; GTK/Zenity send `*.*`). Build the globs the way
// the portal backend does and check the files the project filter hid in #1644 —
// and everything else — are admitted.
assert_eq!(super::all_files(), ["*"]);
let filters: Vec<_> =
super::all_files().iter().map(|ext| if ext == "*" { ext.clone() } else { format!("*.{ext}") }).map(|g| glob::Pattern::new(&g).unwrap()).collect();
for name in ["project.fcproj", "Project.FCPROJ", "my film.filmcraft", "clip.mp4", "notes.txt", "noextension"] {
assert!(filters.iter().any(|f| f.matches(name)), "All files hides {name}");
}
}
#[test]
fn save_filters_keep_the_literal_default_extension() {
assert_eq!(extensions(&["mp4", "mov"]).first().map(String::as_str), Some("mp4"));
apps/filmcraft/src/graphics.rs
+3 -4
View File
@@ -444,9 +444,8 @@ fn apply_for(options: &mut NativeOptions, mode: Mode, backend_override: Option<B
// includes GL. Creating that backend can crash inside AMD's atio6axx.dll before adapter
// selection or any Rust error handling runs, so the window never appears. Choosing an
// adapter afterwards is too late: only leaving GL out of the instance avoids it. Keep
// Vulkan and DX12 both: forcing DX12 alone sends shaders through FXC (which rejects
// fx.wgsl, silently falling back to the CPU) and leaves machines without DX12 with no
// backend at all.
// Vulkan and DX12 both: forcing DX12 alone leaves machines without DX12 with no backend
// at all.
Mode::Default if windows => {
create.instance_descriptor.backends = backend_override.unwrap_or(Backends::PRIMARY);
// wgpu's own choice of adapter, except for a GPU that Windows lists through two
@@ -458,7 +457,7 @@ fn apply_for(options: &mut NativeOptions, mode: Mode, backend_override: Option<B
Some(Arc::new(move |adapters: &[wgpu::Adapter], surface: Option<&wgpu::Surface<'_>>| filmcraft_gpu::adapter::pick(adapters, surface, power)));
}
Mode::Default => {}
// Without Vulkan, whose driver is what crashes; GPU effects may fall back to the CPU.
// Without Vulkan, whose driver is what crashes.
Mode::Dx12 => create.instance_descriptor.backends = Backends::DX12,
Mode::Software => {
create.instance_descriptor.backends = software_backends(windows);
apps/filmcraft/src/main.rs
+19 -4
View File
@@ -352,6 +352,9 @@ fn app_creator(launch: Launch, graphics: graphics::GraphicsStart, created: Arc<A
app.hooks.pick_open_project = Some(Box::new(|| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("FilmCraft Project"), &file_filters::extensions(&["fcproj"]))
// #1644: where the system MIME database doesn't know `.fcproj`, the project
// filter hides every saved project; "All files" keeps them reachable.
.add_filter(filmcraft_ui_egui::i18n::t("All files"), &file_filters::all_files())
.pick_file()
.map(|p| p.to_string_lossy().to_string())
}));
@@ -488,6 +491,8 @@ fn run_dialog_off_thread(dialog: filmcraft_ui_egui::native_dialogs::FileDialog)
let t = |s: &str| filmcraft_ui_egui::i18n::t(s).to_string();
let media = t("Media");
let project = t("FilmCraft Project");
let all_files = t("All files");
let all_files_exts = file_filters::all_files();
let project_exts = project_patterns();
let dialog = prepare_dialog(dialog);
let (tx, rx) = std::sync::mpsc::channel();
@@ -500,11 +505,19 @@ fn run_dialog_off_thread(dialog: filmcraft_ui_egui::native_dialogs::FileDialog)
}
FileDialog::OpenFile { filter, exts } => rfd::FileDialog::new().add_filter(&filter, &exts).pick_file().map(path).into_iter().collect(),
FileDialog::Relink { exts, .. } => rfd::FileDialog::new().add_filter(&media, &exts).pick_file().map(path).into_iter().collect(),
FileDialog::OpenProject => rfd::FileDialog::new().add_filter(&project, &project_exts).pick_file().map(path).into_iter().collect(),
FileDialog::OpenProject => rfd::FileDialog::new()
.add_filter(&project, &project_exts)
// #1644: same "All files" fallback as the synchronous picker, so a
// project hidden by an unregistered `.fcproj` MIME type stays openable.
.add_filter(&all_files, &all_files_exts)
.pick_file()
.map(path)
.into_iter()
.collect(),
FileDialog::SaveProject { name } => {
rfd::FileDialog::new().add_filter(&project, &project_exts).set_file_name(name).save_file().map(path).into_iter().collect()
}
FileDialog::SaveAs { filter, exts, name } => {
FileDialog::SaveAs { filter, exts, name, .. } => {
rfd::FileDialog::new().add_filter(&filter, &exts).set_file_name(name).save_file().map(path).into_iter().collect()
}
FileDialog::Folder { at } => {
@@ -541,7 +554,7 @@ fn prepare_dialog(dialog: filmcraft_ui_egui::native_dialogs::FileDialog) -> film
match dialog {
FileDialog::OpenFiles { exts } => FileDialog::OpenFiles { exts: patterns(exts) },
FileDialog::OpenFile { filter, exts } => FileDialog::OpenFile { filter: t(&filter), exts: patterns(exts) },
FileDialog::SaveAs { filter, exts, name } => FileDialog::SaveAs { filter: t(&filter), exts: patterns(exts), name },
FileDialog::SaveAs { filter, exts, name, adds_ext } => FileDialog::SaveAs { filter: t(&filter), exts: patterns(exts), name, adds_ext },
FileDialog::Relink { exts, hint } => FileDialog::Relink { exts: patterns(exts), hint },
other => other,
}
@@ -579,7 +592,7 @@ mod tests {
for dialog in [
FileDialog::OpenFiles { exts: mp4() },
FileDialog::OpenFile { filter: "Media".into(), exts: mp4() },
FileDialog::SaveAs { filter: "Media".into(), exts: mp4(), name: "clip.mp4".into() },
FileDialog::SaveAs { filter: "Media".into(), exts: mp4(), name: "clip.mp4".into(), adds_ext: true },
FileDialog::Relink { exts: mp4(), hint },
] {
let exts = match super::prepare_dialog(dialog) {
@@ -592,5 +605,7 @@ mod tests {
let project = super::project_patterns();
assert_eq!(project.first().map(String::as_str), Some("fcproj"));
assert!(project.iter().any(|e| e == "[fF][cC][pP][rR][oO][jJ]"), "{project:?}");
// #1644: the Open Project dialog's "All files" fallback is a single `*`.
assert_eq!(super::file_filters::all_files(), ["*"]);
}
}
apps/filmcraft/src/native_menu.rs
+10
View File
@@ -145,8 +145,18 @@ pub fn install(app: &FilmcraftApp, ctx: egui::Context) -> (Receiver<String>, Men
filmcraft_ui_egui::i18n::Language::Id
} else if checked("app.language.italian") {
filmcraft_ui_egui::i18n::Language::It
} else if checked("app.language.bosnian") {
filmcraft_ui_egui::i18n::Language::Bs
} else if checked("app.language.korean") {
filmcraft_ui_egui::i18n::Language::Ko
} else if checked("app.language.czech") {
filmcraft_ui_egui::i18n::Language::Cs
} else if checked("app.language.polish") {
filmcraft_ui_egui::i18n::Language::Pl
} else if checked("app.language.swedish") {
filmcraft_ui_egui::i18n::Language::Sv
} else if checked("app.language.turkish") {
filmcraft_ui_egui::i18n::Language::Tr
} else {
filmcraft_ui_egui::i18n::Language::En
};
crates/aac/src/decoder.rs
+39 -15
View File
@@ -5,8 +5,8 @@ use filmcraft_bitstream::BitReader;
use crate::config::{AudioSpecificConfig, ElementType, ProgramConfig};
use crate::huffman::{self, INTENSITY_HCB, INTENSITY_HCB2, NOISE_HCB, ZERO_HCB};
use crate::ics::{IcsInfo, WindowSequence};
use crate::mdct::{WindowShape, mdct_long, mdct_short, window};
use crate::tables::{pow43, sf_gain};
use crate::mdct::{MdctScratch, WindowShape, mdct_long, mdct_short, window};
use crate::tables::{pow43, pow43_table, sf_gain};
use crate::tns::{self, TnsData};
use crate::{Error, Result};
@@ -41,6 +41,23 @@ struct ChannelState {
prev_shape: WindowShape,
}
/// Buffers reused by every frame so the steady state does not allocate per channel.
struct Scratch {
/// Quantised values of the channel being parsed.
q: Vec<i32>,
/// Windowed output of the current block (2048) and the raw inverse transform (2048 / 256).
buf: Vec<f32>,
y: Vec<f32>,
long: MdctScratch,
short: MdctScratch,
}
impl Scratch {
fn new() -> Scratch {
Scratch { q: vec![0; 1024], buf: vec![0.0; 2048], y: vec![0.0; 2048], long: mdct_long().scratch(), short: mdct_short().scratch() }
}
}
/// AAC-LC decoder.
pub struct Decoder {
asc: AudioSpecificConfig,
@@ -49,6 +66,7 @@ pub struct Decoder {
offsets: Vec<usize>,
channels: Vec<ChannelState>,
noise_state: u32,
scratch: Scratch,
/// A fill element carried SBR data (implicitly signalled HE-AAC).
sbr_seen: bool,
}
@@ -73,7 +91,8 @@ impl Decoder {
if layout.is_empty() && asc.channel_config != 0 {
return Err(Error::InvalidConfig("unsupported channel configuration"));
}
let mut d = Decoder { asc, layout: Vec::new(), offsets: Vec::new(), channels: Vec::new(), noise_state: 0x2545_F491, sbr_seen: false };
let mut d =
Decoder { asc, layout: Vec::new(), offsets: Vec::new(), channels: Vec::new(), noise_state: 0x2545_F491, scratch: Scratch::new(), sbr_seen: false };
d.set_layout(layout);
Ok(d)
}
@@ -389,7 +408,9 @@ impl Decoder {
}
// spectral_data
let mut q = vec![0i32; 1024];
let mut q = std::mem::take(&mut self.scratch.q);
q.clear();
q.resize(1024, 0);
let mut vals = [0i32; 4];
for g in 0..info.num_groups {
let w0 = info.group_start(g);
@@ -421,6 +442,7 @@ impl Decoder {
}
// dequantise
let t43 = pow43_table();
for g in 0..info.num_groups {
let w0 = info.group_start(g);
for sfb in 0..info.max_sfb {
@@ -430,14 +452,16 @@ impl Decoder {
}
let gain = sf_gain(ics.sf[g][sfb]);
for w in w0..w0 + info.group_len[g] as usize {
for k in swb[sfb] as usize..swb[sfb + 1] as usize {
let v = q[w * 128 + k];
let m = pow43(v.unsigned_abs()) * gain;
ics.spec[w * 128 + k] = if v < 0 { -m } else { m };
let r = w * 128 + swb[sfb] as usize..w * 128 + swb[sfb + 1] as usize;
for (o, &v) in ics.spec[r.clone()].iter_mut().zip(&q[r]) {
let a = v.unsigned_abs();
let m = t43.get(a as usize).copied().unwrap_or_else(|| pow43(a)) * gain;
*o = if v < 0 { -m } else { m };
}
}
}
}
self.scratch.q = q;
Ok(ics)
}
@@ -487,23 +511,24 @@ impl Decoder {
if let Some(t) = &ics.tns {
tns::apply_decoder(&mut ics.spec, &ics.info, self.sf_index(), t);
}
synth(&mut self.channels[ch], &ics.info, &ics.spec)
synth(&mut self.channels[ch], &mut self.scratch, &ics.info, &ics.spec)
}
}
/// Inverse MDCT, windowing and overlap-add (§4.6.18), output scaled to ±1.0 full scale.
fn synth(state: &mut ChannelState, info: &IcsInfo, spec: &[f32]) -> Vec<f32> {
let mut buf = vec![0f32; 2048];
fn synth(state: &mut ChannelState, sc: &mut Scratch, info: &IcsInfo, spec: &[f32]) -> Vec<f32> {
let Scratch { buf, y, long, short, .. } = sc;
buf.fill(0.0);
let prev = state.prev_shape;
let cur = info.window_shape;
let norm = 1.0 / 32768.0;
match info.window_sequence {
WindowSequence::EightShort => {
let t = mdct_short();
let mut y = vec![0f32; 256];
let y = &mut y[..256];
let wc = window(cur, false);
for w in 0..8 {
t.inverse(&spec[w * 128..(w + 1) * 128], &mut y);
t.inverse_with(&spec[w * 128..(w + 1) * 128], y, short);
let wl = window(if w == 0 { prev } else { cur }, false);
let off = 448 + 128 * w;
let s = norm / 128.0;
@@ -514,8 +539,7 @@ fn synth(state: &mut ChannelState, info: &IcsInfo, spec: &[f32]) -> Vec<f32> {
}
}
seq => {
let mut y = vec![0f32; 2048];
mdct_long().inverse(&spec[..1024], &mut y);
mdct_long().inverse_with(&spec[..1024], y, long);
let s = norm / 1024.0;
match seq {
WindowSequence::LongStop => {
crates/aac/src/encoder.rs
+18 -12
View File
@@ -25,7 +25,7 @@ use crate::tables::{sample_rate_index, swb_offsets_long, swb_offsets_short, tns_
use crate::tns::{self, TnsData, TnsFilter};
use crate::{Error, FRAME_LEN, Result};
use psy::Psy;
use quant::{ChannelInput, QuantChannel, quantise_channel, write_ics};
use quant::{ChannelInput, QuantCache, QuantChannel, quantise_channel, write_ics};
/// Rate control mode.
#[derive(Debug, Clone, Copy, PartialEq)]
@@ -110,6 +110,8 @@ struct ElementState {
struct ElementFrame {
info: IcsInfo,
chans: Vec<ChannelInput>,
/// Rate-loop memo per channel, filled while the frame is quantised.
caches: Vec<QuantCache>,
ms_mask: Option<(u32, Vec<bool>)>,
}
@@ -575,14 +577,15 @@ impl Encoder {
};
ms_mask = Some((present, used));
}
let chans = specs.iter().zip(thrs.iter()).zip(tnss).map(|((s, th), tn)| ChannelInput::new(info, swb, s, th, tn)).collect();
let chans: Vec<ChannelInput> = specs.iter().zip(thrs.iter()).zip(tnss).map(|((s, th), tn)| ChannelInput::new(info, swb, s, th, tn)).collect();
let caches = chans.iter().map(QuantCache::new).collect();
let e = &mut self.elements[ei];
e.prev_seq = seq;
e.short_next = Some(short_next);
if short {
self.stats.short_blocks += 1;
}
(ElementFrame { info, chans, ms_mask }, pe)
(ElementFrame { info, chans, caches, ms_mask }, pe)
}
fn element_overhead(&self, ei: usize, f: &ElementFrame) -> u32 {
@@ -599,15 +602,18 @@ impl Encoder {
}
}
fn quantise_all(frames: &[ElementFrame], lambda: f32) -> Vec<Vec<QuantChannel>> {
fn quantise_all(frames: &mut [ElementFrame], lambda: f32) -> Vec<Vec<QuantChannel>> {
#[cfg(feature = "rayon")]
{
use rayon::prelude::*;
frames.par_iter().map(|f| f.chans.par_iter().map(|c| quantise_channel(c, lambda)).collect()).collect()
frames
.par_iter_mut()
.map(|f| f.chans.par_iter().zip(f.caches.par_iter_mut()).map(|(c, cache)| quantise_channel(c, cache, lambda)).collect())
.collect()
}
#[cfg(not(feature = "rayon"))]
{
frames.iter().map(|f| f.chans.iter().map(|c| quantise_channel(c, lambda)).collect()).collect()
frames.iter_mut().map(|f| f.chans.iter().zip(f.caches.iter_mut()).map(|(c, cache)| quantise_channel(c, cache, lambda)).collect()).collect()
}
}
@@ -660,16 +666,16 @@ impl Encoder {
BitrateMode::Vbr(q) => {
let d = q.clamp(0.5, 6.0) - 1.0;
let mut lambda = 10f32.powf(-0.2 - 1.2 * d + 0.09 * d * d);
let mut res = Self::quantise_all(&frames, lambda);
let mut res = Self::quantise_all(&mut frames, lambda);
while total_bits(&res) > target && lambda < 1e9 {
lambda *= 2.0;
res = Self::quantise_all(&frames, lambda);
res = Self::quantise_all(&mut frames, lambda);
}
res
}
BitrateMode::Cbr(_) => {
let mut lam = self.lambda.clamp(1e-6, 1e9);
let mut cur = Self::quantise_all(&frames, lam);
let mut cur = Self::quantise_all(&mut frames, lam);
let (mut lo, mut hi, mut best);
if total_bits(&cur) <= target {
hi = lam;
@@ -680,7 +686,7 @@ impl Encoder {
if lo < 1e-6 {
break;
}
cur = Self::quantise_all(&frames, lo);
cur = Self::quantise_all(&mut frames, lo);
if total_bits(&cur) > target {
break;
}
@@ -691,7 +697,7 @@ impl Encoder {
lo = lam;
loop {
lam *= 4.0;
cur = Self::quantise_all(&frames, lam);
cur = Self::quantise_all(&mut frames, lam);
if total_bits(&cur) <= target || lam > 1e9 {
break;
}
@@ -705,7 +711,7 @@ impl Encoder {
break;
}
let mid = (lo * hi).sqrt();
let r = Self::quantise_all(&frames, mid);
let r = Self::quantise_all(&mut frames, mid);
if total_bits(&r) <= target {
hi = mid;
best = r;
crates/aac/src/encoder/quant.rs
+134 -49
View File
@@ -8,7 +8,7 @@ use filmcraft_bitstream::BitWriter;
use crate::huffman::{self, ZERO_HCB};
use crate::ics::IcsInfo;
use crate::tables::pow43;
use crate::tables::pow43_table;
use crate::tns::TnsData;
/// Magic rounding offset for AAC quantisation (`0.5 - 0.0946`), minimising the error for the
@@ -67,6 +67,42 @@ impl ChannelInput {
}
}
/// Scalefactor-keyed results of one band, valid until the band is quantised at another scalefactor.
#[derive(Clone, Copy)]
struct BandMemo {
/// Scalefactor the cached values belong to (`NO_SF` when empty).
sf: i32,
maxq: i32,
/// Spectral bits per codebook plus one for an all-zero band (see `band_costs`), once computed.
cost: Option<[u32; 12]>,
}
const NO_SF: i32 = i32::MIN;
/// Work that depends only on (band, scalefactor), not on λ, kept across the rate-loop iterations of
/// one frame. One cache per channel; callers hold it mutably, so parallel channels share nothing.
pub struct QuantCache {
/// Distortion per (band, scalefactor), NaN while unknown.
dist: Vec<f32>,
bands: Vec<BandMemo>,
/// Quantised values of every band at its memoised scalefactor.
q: Vec<i32>,
/// Cost row of a band that quantises to zeros.
zero_cost: Vec<Option<[u32; 12]>>,
}
impl QuantCache {
pub fn new(ch: &ChannelInput) -> QuantCache {
let nb = ch.bands.len();
QuantCache {
dist: vec![f32::NAN; nb * 256],
bands: vec![BandMemo { sf: NO_SF, maxq: 0, cost: None }; nb],
q: vec![0; ch.x.len()],
zero_cost: vec![None; nb],
}
}
}
/// A quantised channel ready to be written.
#[derive(Clone, Default)]
pub struct QuantChannel {
@@ -90,46 +126,54 @@ fn gain(sf: i32) -> f32 {
(0.25 * (sf as f32 - 100.0)).exp2()
}
/// Quantise a slice at `sf`, returning the distortion; writes the (signed) values to `out` if given.
fn quantise(x: &[f32], xabs: &[f32], x34: &[f32], sf: i32, out: Option<&mut [i32]>) -> f32 {
/// Distortion of a slice quantised at `sf`. Kept out of line so the accumulator stays in a register.
#[inline(never)]
fn distortion(xabs: &[f32], x34: &[f32], sf: i32) -> f32 {
let g34 = gain34(sf);
let g = gain(sf);
let t = pow43_table();
let mut d = 0f32;
match out {
Some(out) => {
for i in 0..x.len() {
let q = ((x34[i] * g34 + ROUND) as i32).min(MAX_Q);
let e = xabs[i] - pow43(q as u32) * g;
d += e * e;
out[i] = if x[i] < 0.0 { -q } else { q };
}
}
None => {
for i in 0..x.len() {
let q = ((x34[i] * g34 + ROUND) as i32).min(MAX_Q);
let e = xabs[i] - pow43(q as u32) * g;
d += e * e;
}
}
for (&a, &b) in xabs.iter().zip(x34) {
let q = ((b * g34 + ROUND) as i32).min(MAX_Q);
let e = a - t[(q as usize) & 8191] * g;
d += e * e;
}
d
}
/// Quantise a slice at `sf`, writing the (signed) values to `out`.
fn quantise(x: &[f32], sf: i32, x34: &[f32], out: &mut [i32]) {
let g34 = gain34(sf);
for ((o, &xv), &b) in out.iter_mut().zip(x).zip(x34) {
let q = ((b * g34 + ROUND) as i32).min(MAX_Q);
*o = if xv < 0.0 { -q } else { q };
}
}
/// Largest scalefactor whose distortion stays within `allowed`.
fn search_sf(ch: &ChannelInput, b: &Band, allowed: f32) -> i32 {
fn search_sf(ch: &ChannelInput, bi: usize, dist: &mut [f32], allowed: f32) -> i32 {
let b = &ch.bands[bi];
let r = b.start..b.end;
let (x, xa, x34) = (&ch.x[r.clone()], &ch.xabs[r.clone()], &ch.x34[r]);
let (xa, x34) = (&ch.xabs[r.clone()], &ch.x34[r]);
let dist = &mut dist[bi * 256..bi * 256 + 256];
let mut dist_at = |sf: i32| {
let d = &mut dist[sf as usize];
if d.is_nan() {
*d = distortion(xa, x34, sf);
}
*d
};
let l2 = b.max34.log2();
let lo = ((100.0 - ((MAX_Q as f32 + 0.5).log2() - l2) / 0.1875).ceil() as i32).clamp(0, 255);
let zero = ((100.0 - ((1.0 - ROUND).log2() - l2) / 0.1875).floor() as i32 + 1).clamp(0, 256);
let mut a = lo;
let mut hi = (zero - 1).clamp(lo, 255);
if quantise(x, xa, x34, a, None) > allowed {
if dist_at(a) > allowed {
return a;
}
while a < hi {
let mid = (a + hi + 1) / 2;
if quantise(x, xa, x34, mid, None) <= allowed {
if dist_at(mid) <= allowed {
a = mid;
} else {
hi = mid - 1;
@@ -176,8 +220,9 @@ fn limit_sf_steps(sf: &mut [i32]) {
}
/// Quantise every band at its scalefactor into `q`, recording each band's largest magnitude in
/// `maxq`. A band that quantises to all zeros leaves the chain (`sf = -1`); returns whether any did.
fn quantise_bands(ch: &ChannelInput, sf: &mut [i32], q: &mut [i32], maxq: &mut [i32]) -> bool {
/// `maxq`; a band already quantised at that scalefactor is taken from `cache`. A band that quantises
/// to all zeros leaves the chain (`sf = -1`); returns whether any did.
fn quantise_bands(ch: &ChannelInput, cache: &mut QuantCache, sf: &mut [i32], q: &mut [i32], maxq: &mut [i32]) -> bool {
let mut removed = false;
for (i, b) in ch.bands.iter().enumerate() {
let r = b.start..b.end;
@@ -186,8 +231,13 @@ fn quantise_bands(ch: &ChannelInput, sf: &mut [i32], q: &mut [i32], maxq: &mut [
maxq[i] = 0;
continue;
}
quantise(&ch.x[r.clone()], &ch.xabs[r.clone()], &ch.x34[r.clone()], sf[i], Some(&mut q[r.clone()]));
maxq[i] = q[r].iter().map(|v| v.abs()).max().unwrap_or(0);
let memo = &mut cache.bands[i];
if memo.sf != sf[i] {
quantise(&ch.x[r.clone()], sf[i], &ch.x34[r.clone()], &mut cache.q[r.clone()]);
*memo = BandMemo { sf: sf[i], maxq: cache.q[r.clone()].iter().map(|v| v.abs()).max().unwrap_or(0), cost: None };
}
q[r.clone()].copy_from_slice(&cache.q[r]);
maxq[i] = memo.maxq;
if maxq[i] == 0 {
sf[i] = -1;
removed = true;
@@ -199,21 +249,39 @@ fn quantise_bands(ch: &ChannelInput, sf: &mut [i32], q: &mut [i32], maxq: &mut [
/// Cost sentinel for a codebook that cannot code a band.
const INF: u32 = u32::MAX / 4;
/// Bits to code each band of a group with each codebook (`INF` when the codebook cannot).
fn codebook_costs(ch: &ChannelInput, q: &[i32], maxq: &[i32], base: usize, n: usize) -> Vec<[u32; 12]> {
/// Bits per candidate codebook for a band holding `vals` (largest magnitude `maxq`); `INF`
/// marks codebooks that cannot code it. An all-zero band inside a coded section also needs a (zero)
/// scalefactor difference, hence the extra bit.
fn band_costs(vals: &[i32], maxq: i32) -> [u32; 12] {
let mut cost = [INF; 12];
let extra = (maxq == 0) as u32;
for &c in candidates(maxq) {
match c {
0 => cost[0] = 0,
// odd codebook below 11 and its successor share everything but the length table
c if c % 2 == 1 && c < huffman::ESC_HCB => {
let (a, b) = huffman::count_bits_pair(c, vals);
cost[c as usize] = a + extra;
cost[c as usize + 1] = b + extra;
}
c if c % 2 == 0 => {} // filled in by the pair above
c => cost[c as usize] = huffman::count_bits(c, vals) + extra,
}
}
cost
}
/// Bits to code each band of a group with each codebook (`INF` when the codebook cannot), from the
/// per-band memo.
fn codebook_costs(ch: &ChannelInput, cache: &mut QuantCache, sf: &[i32], q: &[i32], maxq: &[i32], base: usize, n: usize) -> Vec<[u32; 12]> {
let mut cost = vec![[INF; 12]; n];
for i in 0..n {
let bi = base + i;
let b = &ch.bands[bi];
let vals = &q[b.start..b.end];
for &c in candidates(maxq[bi]) {
cost[i][c as usize] = if c == 0 {
0
} else {
// all-zero band inside a coded section also needs a (zero) scalefactor difference
huffman::count_bits(c, vals) + if maxq[bi] == 0 { 1 } else { 0 }
};
}
// A band that ends up uncoded (`sf < 0`) holds zeros, whatever its memo says.
let row = if sf[bi] >= 0 { &mut cache.bands[bi].cost } else { &mut cache.zero_cost[bi] };
cost[i] = *row.get_or_insert_with(|| band_costs(vals, maxq[bi]));
}
cost
}
@@ -264,8 +332,8 @@ fn best_codebooks(cost: &[[u32; 12]], new_sec: u32) -> Vec<u8> {
}
/// Run-length encode a group's codebooks into `(codebook, length)` sections, and sum the spectral
/// bits of its non-zero sections. `base` is the index of the group's first band.
fn group_sections(ch: &ChannelInput, q: &[i32], cbs: &[u8], base: usize) -> (Vec<(u8, usize)>, u32) {
/// bits of its non-zero sections from the cost rows. `base` is the index of the group's first band.
fn group_sections(cost: &[[u32; 12]], maxq: &[i32], cbs: &[u8], base: usize) -> (Vec<(u8, usize)>, u32) {
let mut secs: Vec<(u8, usize)> = Vec::new();
let mut spectral_bits = 0u32;
for (i, &c) in cbs.iter().enumerate() {
@@ -274,8 +342,8 @@ fn group_sections(ch: &ChannelInput, q: &[i32], cbs: &[u8], base: usize) -> (Vec
_ => secs.push((c, 1)),
}
if c != ZERO_HCB {
let b = &ch.bands[base + i];
spectral_bits += huffman::count_bits(c, &q[b.start..b.end]);
// the cost row carries one extra bit for an all-zero band (its scalefactor difference)
spectral_bits += cost[i][c as usize] - (maxq[base + i] == 0) as u32;
}
}
(secs, spectral_bits)
@@ -283,7 +351,7 @@ fn group_sections(ch: &ChannelInput, q: &[i32], cbs: &[u8], base: usize) -> (Vec
/// Section selection: a codebook per band, the sections of each group and the bits they cost
/// `(codebooks, sections, spectral_bits, section_bits)`.
fn select_sections(ch: &ChannelInput, q: &[i32], maxq: &[i32]) -> (Vec<u8>, Vec<Vec<(u8, usize)>>, u32, u32) {
fn select_sections(ch: &ChannelInput, cache: &mut QuantCache, sf: &[i32], q: &[i32], maxq: &[i32]) -> (Vec<u8>, Vec<Vec<(u8, usize)>>, u32, u32) {
let info = &ch.info;
let (sect_bits, esc) = if info.is_short() { (3u32, 7usize) } else { (5u32, 31usize) };
let mut cb = vec![ZERO_HCB; ch.bands.len()];
@@ -297,9 +365,9 @@ fn select_sections(ch: &ChannelInput, q: &[i32], maxq: &[i32]) -> (Vec<u8>, Vec<
sections.push(Vec::new());
continue;
}
let cost = codebook_costs(ch, q, maxq, base, n);
let cost = codebook_costs(ch, cache, sf, q, maxq, base, n);
cb[base..base + n].copy_from_slice(&best_codebooks(&cost, 4 + sect_bits));
let (secs, bits) = group_sections(ch, q, &cb[base..base + n], base);
let (secs, bits) = group_sections(&cost, maxq, &cb[base..base + n], base);
spectral_bits += bits;
for &(_, len) in &secs {
section_bits += 4 + sect_bits * (len / esc + 1) as u32;
@@ -310,7 +378,7 @@ fn select_sections(ch: &ChannelInput, q: &[i32], maxq: &[i32]) -> (Vec<u8>, Vec<
}
/// Quantise one channel at rate-distortion multiplier `lambda` and select sections.
pub fn quantise_channel(ch: &ChannelInput, lambda: f32) -> QuantChannel {
pub fn quantise_channel(ch: &ChannelInput, cache: &mut QuantCache, lambda: f32) -> QuantChannel {
let nb = ch.bands.len();
let mut sf = vec![-1i32; nb];
for (i, b) in ch.bands.iter().enumerate() {
@@ -318,7 +386,7 @@ pub fn quantise_channel(ch: &ChannelInput, lambda: f32) -> QuantChannel {
if b.max34 <= 0.0 || b.en <= allowed {
continue;
}
sf[i] = search_sf(ch, b, allowed);
sf[i] = search_sf(ch, i, &mut cache.dist, allowed);
}
// Scalefactor differences between coded bands must stay within ±60: lower (refine) offenders,
// then quantise; repeat if a band unexpectedly quantised to all zeros (it leaves the chain).
@@ -326,12 +394,12 @@ pub fn quantise_channel(ch: &ChannelInput, lambda: f32) -> QuantChannel {
let mut maxq = vec![0i32; nb];
loop {
limit_sf_steps(&mut sf);
if !quantise_bands(ch, &mut sf, &mut q, &mut maxq) {
if !quantise_bands(ch, cache, &mut sf, &mut q, &mut maxq) {
break;
}
}
let (cb, sections, spectral_bits, section_bits) = select_sections(ch, &q, &maxq);
let (cb, sections, spectral_bits, section_bits) = select_sections(ch, cache, &sf, &q, &maxq);
// Scalefactors: global gain is the first coded band's value; bands coded with a non-zero
// codebook but all-zero values reuse the previous value (difference 0).
@@ -412,10 +480,27 @@ mod tests {
let thr = vec![vec![1000.0f32; 64]];
let ch = ChannelInput::new(info, swb, &spec, &thr, None);
for lambda in [0.01f32, 1.0, 100.0, 1e5] {
let qc = quantise_channel(&ch, lambda);
let mut cache = QuantCache::new(&ch);
let qc = quantise_channel(&ch, &mut cache, lambda);
let mut bw = BitWriter::new();
write_ics(&mut bw, &ch, &qc, false);
assert_eq!(bw.bit_len() as u32, qc.bits, "lambda {lambda}");
}
}
#[test]
fn reused_cache_gives_the_same_result_as_a_fresh_one() {
let swb = swb_offsets_long(4);
let info = IcsInfo { window_sequence: WindowSequence::OnlyLong, window_shape: WindowShape::Sine, max_sfb: 40, ..Default::default() };
let spec: Vec<f32> = (0..1024).map(|i| ((i * 7919 % 1000) as f32 - 500.0) * (1000.0 / (i as f32 + 10.0))).collect();
let thr = vec![vec![1000.0f32; 64]];
let ch = ChannelInput::new(info, swb, &spec, &thr, None);
let mut shared = QuantCache::new(&ch);
// up and down, like the rate loop, with repeats
for lambda in [1.0f32, 4.0, 16.0, 0.25, 1.0, 100.0, 3.0, 3.0, 1e5, 0.01, 1.0] {
let a = quantise_channel(&ch, &mut shared, lambda);
let b = quantise_channel(&ch, &mut QuantCache::new(&ch), lambda);
assert_eq!((&a.q, &a.sf, &a.cb, &a.sections, a.global_gain, a.bits), (&b.q, &b.sf, &b.cb, &b.sections, b.global_gain, b.bits), "lambda {lambda}");
}
}
}
crates/aac/src/huffman.rs
+156 -37
View File
@@ -53,60 +53,92 @@ pub fn book(cb: u8) -> &'static Book {
&BOOKS[(cb as usize).clamp(1, 11) - 1]
}
const LEAF: u32 = 1 << 31;
/// Bits resolved by the first table level.
const L1: u32 = 9;
/// Flag of a first-level entry that points to a sub-table.
const SUB: u32 = 1 << 31;
/// Binary decoding tree for one codebook.
struct Tree {
nodes: Vec<[u32; 2]>,
/// Two-level decoding table for one prefix code. A leaf entry is `len << 24 | symbol`; a first-level
/// entry with `SUB` set is `SUB | offset << 5 | width` and selects a sub-table of `1 << width`
/// leaves indexed by the next `width` bits. An entry of 0 is a bit pattern that no codeword starts
/// with (the codebooks are complete, so this does not occur).
struct Lut {
t: Vec<u32>,
}
impl Tree {
fn build(codes: &[(u32, u8)]) -> Tree {
let mut nodes: Vec<[u32; 2]> = vec![[0, 0]];
for (sym, &(code, len)) in codes.iter().enumerate() {
let mut node = 0usize;
for i in (0..len).rev() {
let bit = ((code >> i) & 1) as usize;
if i == 0 {
nodes[node][bit] = LEAF | sym as u32;
} else {
if nodes[node][bit] == 0 {
nodes.push([0, 0]);
nodes[node][bit] = (nodes.len() - 1) as u32;
}
node = nodes[node][bit] as usize;
}
impl Lut {
fn build(codes: &[(u32, u8)]) -> Lut {
let mut t = vec![0u32; 1 << L1];
// sub-table width per first-level prefix
let mut width = vec![0u32; 1 << L1];
for &(code, len) in codes {
let len = len as u32;
if len > L1 {
let p = (code >> (len - L1)) as usize;
width[p] = width[p].max(len - L1);
}
}
Tree { nodes }
for (p, &w) in width.iter().enumerate() {
if w > 0 {
t[p] = SUB | (t.len() as u32) << 5 | w;
t.resize(t.len() + (1 << w), 0);
}
}
for (sym, &(code, len)) in codes.iter().enumerate() {
let len = len as u32;
let leaf = len << 24 | sym as u32;
if len <= L1 {
let first = (code << (L1 - len)) as usize;
t[first..first + (1 << (L1 - len))].fill(leaf);
} else {
let e = t[(code >> (len - L1)) as usize];
let (off, w) = (((e & !SUB) >> 5) as usize, e & 31);
let rest = code & ((1 << (len - L1)) - 1);
let first = off + (rest << (w - (len - L1))) as usize;
t[first..first + (1 << (w - (len - L1)))].fill(leaf);
}
}
Lut { t }
}
/// The symbol of the codeword at the reader position; `Eof` if the codeword is cut off.
#[inline]
fn decode(&self, br: &mut BitReader) -> Result<u32> {
let mut node = 0usize;
loop {
let bit = br.read_bits(1)? as usize;
let next = self.nodes[node][bit];
if next & LEAF != 0 {
return Ok(next & !LEAF);
}
if next == 0 {
return Err(Error::Bitstream("invalid Huffman codeword"));
}
node = next as usize;
let bits = peek32(br);
let mut e = self.t[(bits >> (32 - L1)) as usize];
if e & SUB != 0 {
let w = e & 31;
e = self.t[((e & !SUB) >> 5) as usize + ((bits << L1) >> (32 - w)) as usize];
}
let len = e >> 24;
if len == 0 {
return Err(Error::Bitstream("invalid Huffman codeword"));
}
br.skip(len as usize).map_err(|_| Error::Eof)?;
Ok(e & 0xFF_FFFF)
}
}
fn trees() -> &'static [Tree; 12] {
static TREES: OnceLock<[Tree; 12]> = OnceLock::new();
TREES.get_or_init(|| std::array::from_fn(|i| if i == 0 { Tree::build(&SCALEFACTOR) } else { Tree::build(BOOKS[i - 1].codes) }))
/// The next 32 bits (zero padded past the end), from one 8-byte load when possible.
#[inline]
fn peek32(br: &BitReader) -> u32 {
let pos = br.position();
let byte = pos >> 3;
match br.data().get(byte..byte.saturating_add(8)).and_then(|b| <[u8; 8]>::try_from(b).ok()) {
Some(b) => ((u64::from_be_bytes(b) << (pos & 7)) >> 32) as u32,
None => br.peek(32),
}
}
fn luts() -> &'static [Lut; 12] {
static LUTS: OnceLock<[Lut; 12]> = OnceLock::new();
LUTS.get_or_init(|| std::array::from_fn(|i| if i == 0 { Lut::build(&SCALEFACTOR) } else { Lut::build(BOOKS[i - 1].codes) }))
}
/// Decode one scalefactor difference (`-60..=60`).
#[inline]
pub fn decode_sf(br: &mut BitReader) -> Result<i32> {
Ok(trees()[0].decode(br)? as i32 - 60)
Ok(luts()[0].decode(br)? as i32 - 60)
}
/// Write one scalefactor difference (`-60..=60`).
@@ -125,7 +157,7 @@ pub fn sf_bits(diff: i32) -> u32 {
/// Decode one codeword of codebook `cb` (1..=11) into `out[..dim]`, including sign bits and escapes.
pub fn decode_spectral(br: &mut BitReader, cb: u8, out: &mut [i32]) -> Result<()> {
let b = book(cb);
let mut idx = trees()[cb as usize].decode(br)?;
let mut idx = luts()[cb as usize].decode(br)?;
for i in (0..b.dim).rev() {
out[i] = (idx % b.modulo) as i32;
idx /= b.modulo;
@@ -198,6 +230,27 @@ pub fn count_bits(cb: u8, vals: &[i32]) -> u32 {
bits
}
/// Bits for codebooks `cb` and `cb + 1` in one pass. `cb` must be odd and below 11: each such pair
/// shares dimension, modulus and signedness, so the index is computed once and only the length table
/// differs.
pub fn count_bits_pair(cb: u8, vals: &[i32]) -> (u32, u32) {
debug_assert!(cb % 2 == 1 && cb < ESC_HCB);
let (b, b2) = (book(cb), book(cb + 1));
let (mut x, mut y) = (0u32, 0u32);
for chunk in vals.chunks_exact(b.dim) {
let i = index_of(b, chunk);
x += b.codes[i].1 as u32;
y += b2.codes[i].1 as u32;
if !b.signed {
// sign bits (no escapes below codebook 11)
let nz = chunk.iter().filter(|&&v| v != 0).count() as u32;
x += nz;
y += nz;
}
}
(x, y)
}
/// Write `vals` with codebook `cb` (1..=11).
pub fn write_spectral(bw: &mut BitWriter, cb: u8, vals: &[i32]) {
let b = book(cb);
@@ -259,6 +312,53 @@ mod tests {
}
}
/// Every codeword, followed by arbitrary bits, decodes to its own symbol and length.
#[test]
fn lut_decodes_every_codeword() {
let mut seed = 7u32;
for (i, lut) in luts().iter().enumerate() {
let codes: &[(u32, u8)] = if i == 0 { &SCALEFACTOR } else { BOOKS[i - 1].codes };
for (sym, &(code, len)) in codes.iter().enumerate() {
for _ in 0..3 {
seed = seed.wrapping_mul(1664525).wrapping_add(1013904223);
let mut bw = BitWriter::new();
bw.write_bits(code, len as u32);
bw.write_bits(seed >> 8, 24);
let data = bw.finish();
let mut br = BitReader::new(&data);
assert_eq!(lut.decode(&mut br).unwrap(), sym as u32, "book {i}");
assert_eq!(br.position(), len as usize);
}
}
}
}
/// A codeword cut off by the end of the data is `Eof`, also at an unaligned position.
#[test]
fn truncated_codeword_is_eof() {
for (i, lut) in luts().iter().enumerate() {
let codes: &[(u32, u8)] = if i == 0 { &SCALEFACTOR } else { BOOKS[i - 1].codes };
let &(code, len) = codes.iter().max_by_key(|c| c.1).unwrap();
for lead in 0..8u32 {
let mut bw = BitWriter::new();
bw.write_bits(0, lead);
bw.write_bits(code, len as u32);
let total = lead as usize + len as usize;
let data = bw.finish();
// drop the last byte if that cuts the codeword, else the codeword is whole
let short = &data[..data.len() - 1];
let mut br = BitReader::new(short);
br.skip(lead as usize).unwrap();
let r = lut.decode(&mut br);
if short.len() * 8 < total {
assert!(matches!(r, Err(Error::Eof)), "book {i} lead {lead}");
} else {
assert!(r.is_ok());
}
}
}
}
#[test]
fn spectral_roundtrip_every_book() {
for cb in 1..=11u8 {
@@ -291,6 +391,25 @@ mod tests {
}
}
#[test]
fn pair_count_matches_single_count() {
let mut seed = 99u32;
for cb in (1..11u8).step_by(2) {
let b = book(cb);
for round in 0..40 {
let mut vals = Vec::new();
for _ in 0..(8 * b.dim) {
seed = seed.wrapping_mul(1664525).wrapping_add(1013904223);
let m = ((seed >> 8) % (b.lav + 1)) as i32;
let m = if round % 4 == 0 { 0 } else { m };
let neg = if b.signed { seed & 0x8000_0000 != 0 } else { seed & 1 == 1 };
vals.push(if neg { -m } else { m });
}
assert_eq!(count_bits_pair(cb, &vals), (count_bits(cb, &vals), count_bits(cb + 1, &vals)), "codebook {cb}");
}
}
}
#[test]
fn scalefactor_roundtrip() {
let mut bw = BitWriter::new();
crates/aac/src/mdct.rs
+115 -28
View File
@@ -7,7 +7,11 @@ use std::sync::OnceLock;
/// Radix-2 complex FFT (forward, `e^{-2πi nk/P}`).
struct Fft {
n: usize,
twiddle: Vec<[f32; 2]>,
/// Twiddle of the length-2 stage, `e^{-2πi 0/n}`.
w0: [f32; 2],
/// Twiddles of the stages of length 4, 8, ..., n, one contiguous run of `len / 2` per stage.
stage_re: Vec<f32>,
stage_im: Vec<f32>,
rev: Vec<u32>,
}
@@ -16,13 +20,23 @@ impl Fft {
assert!(n.is_power_of_two());
let bits = n.trailing_zeros();
let rev = (0..n as u32).map(|i| if bits == 0 { 0 } else { i.reverse_bits() >> (32 - bits) }).collect();
let twiddle = (0..n / 2)
let twiddle: Vec<[f32; 2]> = (0..n / 2)
.map(|k| {
let a = -2.0 * PI * k as f64 / n as f64;
[a.cos() as f32, a.sin() as f32]
})
.collect();
Fft { n, twiddle, rev }
let (mut stage_re, mut stage_im) = (Vec::new(), Vec::new());
let mut len = 4;
while len <= n {
let step = n / len;
for k in 0..len / 2 {
stage_re.push(twiddle[k * step][0]);
stage_im.push(twiddle[k * step][1]);
}
len <<= 1;
}
Fft { n, w0: twiddle.first().copied().unwrap_or([1.0, 0.0]), stage_re, stage_im, rev }
}
fn run(&self, re: &mut [f32], im: &mut [f32]) {
@@ -34,24 +48,80 @@ impl Fft {
im.swap(i, j);
}
}
let mut len = 2;
while len <= n {
let half = len / 2;
let step = n / len;
for start in (0..n).step_by(len) {
for k in 0..half {
let [wr, wi] = self.twiddle[k * step];
let a = start + k;
let b = a + half;
let xr = re[b] * wr - im[b] * wi;
let xi = re[b] * wi + im[b] * wr;
re[b] = re[a] - xr;
im[b] = im[a] - xi;
re[a] += xr;
im[a] += xi;
if n >= 2 {
let [wr, wi] = self.w0;
let ((re2, _), (im2, _)) = (re.as_chunks_mut::<2>(), im.as_chunks_mut::<2>());
for (r, i) in re2.iter_mut().zip(im2) {
let xr = r[1] * wr - i[1] * wi;
let xi = r[1] * wi + i[1] * wr;
r[1] = r[0] - xr;
i[1] = i[0] - xi;
r[0] += xr;
i[0] += xi;
}
}
let mut len = 4;
let mut off = 0;
// Two radix-2 stages (len, 2 * len) per pass over the data: the same butterflies in the same
// order per element, so the result is bit-identical to running the stages one by one.
while len * 2 <= n {
let h = len / 2;
let (t1r, t1i) = (&self.stage_re[off..off + h], &self.stage_im[off..off + h]);
let (t2r, t2i) = (&self.stage_re[off + h..off + 3 * h], &self.stage_im[off + h..off + 3 * h]);
off += 3 * h;
for (rc, ic) in re.chunks_exact_mut(4 * h).zip(im.chunks_exact_mut(4 * h)) {
let (ra, rest) = rc.split_at_mut(h);
let (rb, rest) = rest.split_at_mut(h);
let (rc2, rd) = rest.split_at_mut(h);
let (ia, rest) = ic.split_at_mut(h);
let (ib, rest) = rest.split_at_mut(h);
let (ic2, id) = rest.split_at_mut(h);
for k in 0..h {
let (w1r, w1i) = (t1r[k], t1i[k]);
// stage `len`: (a, b) and (c, d)
let xr = rb[k] * w1r - ib[k] * w1i;
let xi = rb[k] * w1i + ib[k] * w1r;
let (br, bi) = (ra[k] - xr, ia[k] - xi);
let (ar, ai) = (ra[k] + xr, ia[k] + xi);
let xr = rd[k] * w1r - id[k] * w1i;
let xi = rd[k] * w1i + id[k] * w1r;
let (dr, di) = (rc2[k] - xr, ic2[k] - xi);
let (cr, ci) = (rc2[k] + xr, ic2[k] + xi);
// stage `2 * len`: (a, c) with w2[k], (b, d) with w2[k + h]
let (w2r, w2i) = (t2r[k], t2i[k]);
let xr = cr * w2r - ci * w2i;
let xi = cr * w2i + ci * w2r;
rc2[k] = ar - xr;
ic2[k] = ai - xi;
ra[k] = ar + xr;
ia[k] = ai + xi;
let (w2r, w2i) = (t2r[k + h], t2i[k + h]);
let xr = dr * w2r - di * w2i;
let xi = dr * w2i + di * w2r;
rd[k] = br - xr;
id[k] = bi - xi;
rb[k] = br + xr;
ib[k] = bi + xi;
}
}
len <<= 2;
}
if len <= n {
let half = len / 2;
let (twr, twi) = (&self.stage_re[off..off + half], &self.stage_im[off..off + half]);
for (rc, ic) in re.chunks_exact_mut(len).zip(im.chunks_exact_mut(len)) {
let (ra, rb) = rc.split_at_mut(half);
let (ia, ib) = ic.split_at_mut(half);
for k in 0..half {
let (wr, wi) = (twr[k], twi[k]);
let xr = rb[k] * wr - ib[k] * wi;
let xi = rb[k] * wi + ib[k] * wr;
rb[k] = ra[k] - xr;
ib[k] = ia[k] - xi;
ra[k] += xr;
ia[k] += xi;
}
}
len <<= 1;
}
}
}
@@ -83,19 +153,22 @@ impl Mdct {
Mdct { m, fft: Fft::new(q), pre, post }
}
/// DCT-IV: `out[k] = Σ u[n] cos(π/M (n+½)(k+½))`.
fn dct4(&self, u: &[f32], out: &mut [f32]) {
/// Working buffers for this transform size.
pub fn scratch(&self) -> MdctScratch {
MdctScratch { re: vec![0.0; self.m / 2], im: vec![0.0; self.m / 2], u: vec![0.0; self.m] }
}
/// DCT-IV: `out[k] = Σ u[n] cos(π/M (n+½)(k+½))`; `re` and `im` (length M/2) are overwritten.
fn dct4(&self, u: &[f32], out: &mut [f32], re: &mut [f32], im: &mut [f32]) {
let m = self.m;
let q = m / 2;
let mut re = vec![0f32; q];
let mut im = vec![0f32; q];
for n in 0..q {
let (a, b) = (u[2 * n], u[m - 1 - 2 * n]);
let [c, s] = self.pre[n];
re[n] = a * c - b * s;
im[n] = a * s + b * c;
}
self.fft.run(&mut re, &mut im);
self.fft.run(re, im);
for k in 0..q {
let [c, s] = self.post[k];
let wr = re[k] * c - im[k] * s;
@@ -109,21 +182,28 @@ impl Mdct {
pub fn forward(&self, x: &[f32], out: &mut [f32]) {
let m = self.m;
let h = m / 2;
let mut u = vec![0f32; m];
let MdctScratch { re, im, mut u } = self.scratch();
for n in 0..h {
// quarters a, b, c, d of length M/2
u[n] = -x[m + h - 1 - n] - x[m + h + n];
u[h + n] = x[n] - x[m - 1 - n];
}
self.dct4(&u, out);
let (mut re, mut im) = (re, im);
self.dct4(&u, out, &mut re, &mut im);
}
/// Inverse MDCT (unscaled): `y[n] = Σ X[k] cos(π/M (n + ½ + M/2)(k + ½))`, `y` has length 2M.
#[cfg(test)]
pub fn inverse(&self, spec: &[f32], y: &mut [f32]) {
self.inverse_with(spec, y, &mut self.scratch());
}
/// [`Self::inverse`] with caller-provided working buffers from [`Self::scratch`].
pub fn inverse_with(&self, spec: &[f32], y: &mut [f32], sc: &mut MdctScratch) {
let m = self.m;
let h = m / 2;
let mut u = vec![0f32; m];
self.dct4(spec, &mut u);
let MdctScratch { re, im, u } = sc;
self.dct4(spec, u, re, im);
for n in 0..h {
y[n] = u[h + n];
y[m - 1 - n] = -u[h + n];
@@ -133,6 +213,13 @@ impl Mdct {
}
}
/// Working buffers of one [`Mdct`] size, reusable across transforms.
pub struct MdctScratch {
re: Vec<f32>,
im: Vec<f32>,
u: Vec<f32>,
}
/// Cached transforms for 1024 and 128 lines.
pub fn mdct_long() -> &'static Mdct {
static M: OnceLock<Mdct> = OnceLock::new();
crates/aac/src/tables.rs
+25 -5
View File
@@ -99,12 +99,22 @@ pub fn tns_max_order(short: bool) -> usize {
if short { 7 } else { 12 }
}
/// `x^(4/3)` for the integer quantised magnitudes 0..=8191.
pub fn pow43(q: u32) -> f32 {
/// The `x^(4/3)` table for the integer quantised magnitudes 0..=8191.
pub fn pow43_table() -> &'static [f32; 8192] {
use std::sync::OnceLock;
static TABLE: OnceLock<Vec<f32>> = OnceLock::new();
let t = TABLE.get_or_init(|| (0..8192u32).map(|i| (i as f64).powf(4.0 / 3.0) as f32).collect());
match t.get(q as usize) {
static TABLE: OnceLock<Box<[f32; 8192]>> = OnceLock::new();
TABLE.get_or_init(|| {
let mut t = Box::new([0f32; 8192]);
for (i, v) in t.iter_mut().enumerate() {
*v = (i as f64).powf(4.0 / 3.0) as f32;
}
t
})
}
/// `x^(4/3)` for an integer quantised magnitude (table lookup up to 8191).
pub fn pow43(q: u32) -> f32 {
match pow43_table().get(q as usize) {
Some(v) => *v,
None => (q as f64).powf(4.0 / 3.0) as f32,
}
@@ -113,6 +123,16 @@ pub fn pow43(q: u32) -> f32 {
/// `2^(0.25 * (sf - 100))`, the dequantisation gain of scalefactor `sf`.
#[inline]
pub fn sf_gain(sf: i32) -> f32 {
use std::sync::OnceLock;
static TABLE: OnceLock<[f32; 256]> = OnceLock::new();
let t = TABLE.get_or_init(|| std::array::from_fn(|i| sf_gain_exact(i as i32)));
match usize::try_from(sf).ok().and_then(|i| t.get(i)) {
Some(g) => *g,
None => sf_gain_exact(sf),
}
}
fn sf_gain_exact(sf: i32) -> f32 {
(0.25 * (sf as f64 - 100.0)).exp2() as f32
}
crates/audio-dsp/src/biquad.rs
+95
View File
@@ -228,4 +228,99 @@ mod tests {
}
assert_eq!(last, 0.0);
}
// ----------------------------------------------------------------- non-finite input
fn flush_reference(x: f64) -> f64 {
if x.abs() < 1e-30 { 0.0 } else { x }
}
fn flush32_reference(x: f32) -> f32 {
if x.abs() < 1e-25 { 0.0 } else { x }
}
#[test]
fn flush_is_unchanged_for_finite_values() {
let mut rng = Rng::new(21);
let mut xs = vec![0.0, -0.0, 1e-30, -1e-30, 1e-300, f64::MIN_POSITIVE, 5e-324, f64::MAX, f64::MIN, 1.0, -1.0];
xs.extend([1e-30f64.next_down(), 1e-30f64.next_up(), (-1e-30f64).next_down(), (-1e-30f64).next_up()]);
for _ in 0..100_000 {
let m = rng.uniform() as f64;
let e = (rng.uniform() * 300.0) as i32;
xs.push(m * 10f64.powi(e));
}
for &x in &xs {
assert_eq!(crate::flush(x).to_bits(), flush_reference(x).to_bits(), "{x:e}");
let y = x as f32;
if y.is_finite() {
assert_eq!(crate::flush32(y).to_bits(), flush32_reference(y).to_bits(), "{y:e}");
}
}
for y in [1e-25f32, 1e-25f32.next_down(), -1e-25f32, f32::MAX, f32::MIN, 1e-45] {
assert_eq!(crate::flush32(y).to_bits(), flush32_reference(y).to_bits(), "{y:e}");
}
for x in [f64::NAN, f64::INFINITY, f64::NEG_INFINITY] {
assert_eq!(crate::flush(x), 0.0);
assert_eq!(crate::flush32(x as f32), 0.0);
}
}
/// Run `tick` over noise with one non-finite sample at index 100 and return the output.
fn run_with_bad_sample(bad: f32, mut tick: impl FnMut(f32) -> f32) -> Vec<f32> {
let mut rng = Rng::new(4);
(0..4800)
.map(|i| {
let x = if i == 100 { bad } else { 0.5 * rng.uniform() };
tick(x)
})
.collect()
}
#[test]
fn one_non_finite_sample_does_not_poison_filter_state() {
use crate::effects::util::{AllPass1, DcBlock, OnePole};
for bad in [f32::NAN, f32::INFINITY, f32::NEG_INFINITY] {
let c = Coeffs::design(FilterType::Peaking, SR, 1000.0, 1.0, 6.0);
let mut bq = Biquad::new();
let mut lp = OnePole::default();
let mut hp = OnePole::default();
let a = OnePole::alpha(500.0, SR as f32);
let mut ap = AllPass1::default();
let apc = AllPass1::coef(1000.0, SR as f32);
let mut dc = DcBlock::default();
let r = DcBlock::r(SR as f32);
let outputs = [
("Biquad", run_with_bad_sample(bad, |x| bq.tick(&c, x as f64) as f32)),
("OnePole lp", run_with_bad_sample(bad, |x| lp.lp(a, x))),
("OnePole hp", run_with_bad_sample(bad, |x| hp.hp(a, x))),
("AllPass1", run_with_bad_sample(bad, |x| ap.tick(apc, x as f64) as f32)),
("DcBlock", run_with_bad_sample(bad, |x| dc.tick(r, x))),
];
for (name, y) in outputs {
assert!(y[..100].iter().all(|v| v.is_finite()), "{name} {bad}");
assert!(y[102..].iter().all(|v| v.is_finite()), "{name} stays non-finite after {bad}");
}
}
}
#[test]
fn effects_recover_after_one_nan_sample() {
for id in ["parametric_eq", "simple_eq", "graphic_eq_10", "notch_filter"] {
let mut fx = crate::create_effect(id, SR as f32, 2).unwrap_or_else(|| panic!("{id}"));
for p in fx.params() {
if p.unit == crate::Unit::Decibels && p.id != "output" {
fx.set_param(p.id, 6.0);
}
}
let mut rng = Rng::new(8);
let mut l: Vec<f32> = (0..4800).map(|_| 0.5 * rng.uniform()).collect();
let mut r: Vec<f32> = (0..4800).map(|_| 0.5 * rng.uniform()).collect();
l[100] = f32::NAN;
fx.process(&mut [&mut l, &mut r]);
let mut l: Vec<f32> = (0..4800).map(|_| 0.5 * rng.uniform()).collect();
let mut r: Vec<f32> = (0..4800).map(|_| 0.5 * rng.uniform()).collect();
fx.process(&mut [&mut l, &mut r]);
assert!(l.iter().chain(&r).all(|v| v.is_finite()), "{id} stays NaN after one NaN input sample");
}
}
}
crates/audio-dsp/src/effects/dynamics.rs
+14 -19
View File
@@ -3,20 +3,11 @@
//! Detection is stereo-linked (the loudest channel drives one common gain) so the image does
//! not shift. Gain computers work in dB; attack/release are one-pole smoothers on the gain.
use super::util::{coef, lin_db};
use crate::oversample::{Interpolator, PolyphaseBank, true_peak_factor};
use crate::{AudioEffect, ParamSpec, ParamValues, Smoothed, Unit, block_len, db_to_gain, param_plumbing};
use std::collections::VecDeque;
/// One-pole coefficient for a time constant in ms.
fn coef(ms: f32, sr: f32) -> f32 {
if ms <= 0.0 { 0.0 } else { (-1.0 / (ms * 0.001 * sr)).exp() }
}
#[inline]
fn lin_to_db(x: f32) -> f32 {
20.0 * (x.max(1e-10)).log10()
}
/// Static compressor curve with a quadratic soft knee (output level in dB for input level `x`).
pub fn compressor_curve(x: f32, threshold: f32, ratio: f32, knee: f32) -> f32 {
let d = x - threshold;
@@ -138,7 +129,7 @@ impl AudioEffect for Compressor {
}
10.0 * self.ms.max(1e-20).log10()
} else {
lin_to_db(peak)
lin_db(peak)
};
let target = self.curve(level_db) - level_db;
let c = if target < self.env { self.att } else { self.rel };
@@ -225,7 +216,7 @@ impl AudioEffect for Gate {
let n = block_len(channels);
for i in 0..n {
let peak = channels.iter().fold(0.0f32, |a, c| a.max(c[i].abs()));
let level = lin_to_db(peak);
let level = lin_db(peak);
let target = self.curve(level) - level;
if target >= self.env {
// Opening (or staying open): attack.
@@ -360,6 +351,7 @@ impl AudioEffect for Limiter {
let nch = channels.len().min(self.delay.len());
let dlen = self.delay[0].len();
let window = (self.look + 1) as u64;
let mut last_g = None;
for i in 0..n {
let g_in = self.in_gain.tick();
let det_pos = (self.wpos + dlen - self.det_delay) % dlen;
@@ -407,7 +399,10 @@ impl AudioEffect for Limiter {
if self.t.is_multiple_of(65536) {
self.box_sum = self.box_ring.iter().map(|&v| v as f64).sum();
}
self.gr_meter = lin_to_db(g);
last_g = Some(g);
}
if let Some(g) = last_g {
self.gr_meter = lin_db(g);
}
}
}
@@ -457,7 +452,7 @@ mod tests {
let mut l = vec![v; 48000];
let mut r = vec![-v; 48000];
c.process(&mut [&mut l, &mut r]);
let out_db = lin_to_db(l[47999]);
let out_db = lin_db(l[47999]);
let expect = compressor_curve(level_db, -20.0, 4.0, 0.0) + 3.0;
assert!((out_db - expect).abs() < 0.05, "det {detector} in {level_db}: {out_db} vs {expect}");
assert_eq!(l[47999], -r[47999]);
@@ -476,7 +471,7 @@ mod tests {
c.process(&mut [&mut x]);
// After one time constant ~63% of the final reduction (in dB) is applied.
let final_db = compressor_curve(0.0, -20.0, 20.0, 0.0);
let at_tau = lin_to_db(x[480]);
let at_tau = lin_db(x[480]);
assert!((at_tau / final_db - 0.632).abs() < 0.02, "{at_tau} / {final_db}");
}
@@ -492,11 +487,11 @@ mod tests {
let quiet = db_to_gain(-50.0);
let mut x: Vec<f32> = (0..48000).map(|i| if i < 24000 { loud } else { quiet }).collect();
g.process(&mut [&mut x]);
assert!((lin_to_db(x[23999]) + 10.0).abs() < 0.01);
assert!((lin_db(x[23999]) + 10.0).abs() < 0.01);
// During hold (100 ms) the gate stays open.
assert!((lin_to_db(x[24000 + 2400]) + 50.0).abs() < 0.01);
assert!((lin_db(x[24000 + 2400]) + 50.0).abs() < 0.01);
// Well after hold + release: attenuated by the full range.
assert!((lin_to_db(x[47999]) + 90.0).abs() < 0.1, "{}", lin_to_db(x[47999]));
assert!((lin_db(x[47999]) + 90.0).abs() < 0.1, "{}", lin_db(x[47999]));
assert!((expander_curve(-40.0, -30.0, 2.0, 60.0) + 50.0).abs() < 1e-6);
}
@@ -553,6 +548,6 @@ mod tests {
lim.process(&mut [&mut x]);
let tail = &x[24000..];
let peak = tail.iter().fold(0.0f32, |a, v| a.max(v.abs()));
assert!((lin_to_db(peak) + 6.0).abs() < 0.3, "{}", lin_to_db(peak));
assert!((lin_db(peak) + 6.0).abs() < 0.3, "{}", lin_db(peak));
}
}
crates/audio-dsp/src/effects/eq.rs
+73 -6
View File
@@ -136,8 +136,11 @@ impl BandBank {
// Keep the exact per-sample arithmetic and recursive state; only the settled
// parameter/smoother bookkeeping is hoisted out of the hot loop.
let g = out_gain.value() as f64;
let bands = &self.bands;
// Channels are filtered in pairs so two independent recurrences share one loop.
let mut held: Option<(&mut [f32], &mut [Biquad])> = None;
for (channel, state) in channels.iter_mut().take(nch).zip(&mut self.state) {
for (b, st) in self.bands.iter().zip(state.iter_mut()) {
for (b, st) in bands.iter().zip(state.iter_mut()) {
if b.mix.value() <= 0.0 {
st.reset();
}
@@ -145,7 +148,7 @@ impl BandBank {
// A settled unit transfer with zero state is exactly transparent for finite
// f32 input. Do not discard a tail or bypass a non-finite sample. Bounded
// coefficients also ensure intermediate f64 products cannot overflow.
let transparent = self.bands.iter().zip(state.iter()).all(|(b, st)| {
let transparent = bands.iter().zip(state.iter()).all(|(b, st)| {
b.mix.value() <= 0.0
|| (st.is_clear()
&& b.coeffs.b0 == 1.0
@@ -155,7 +158,7 @@ impl BandBank {
&& b.coeffs.b2.abs() <= 4.0)
});
if transparent && channel.iter().take(n).all(|x| x.is_finite()) {
let active = self.bands.iter().any(|b| b.mix.value() > 0.0);
let active = bands.iter().any(|b| b.mix.value() > 0.0);
for sample in channel.iter_mut().take(n) {
let x = *sample as f64;
// An active identity biquad adds +0, which normalizes negative zero.
@@ -163,9 +166,16 @@ impl BandBank {
}
continue;
}
for sample in channel.iter_mut().take(n) {
let (channel, state) = (&mut channel[..n], state.as_mut_slice());
match held.take() {
Some((first, first_state)) => filter_pair(bands, first, first_state, channel, state, g),
None => held = Some((channel, state)),
}
}
if let Some((channel, state)) = held {
for sample in channel.iter_mut() {
let mut x = *sample as f64;
for (b, st) in self.bands.iter().zip(state.iter_mut()) {
for (b, st) in bands.iter().zip(state.iter_mut()) {
let m = b.mix.value() as f64;
if m > 0.0 {
let y = st.tick(&b.coeffs, x);
@@ -215,6 +225,24 @@ impl BandBank {
}
}
/// The settled band cascade for two channels, with the same per-channel arithmetic as one.
fn filter_pair(bands: &[Band], l: &mut [f32], l_state: &mut [Biquad], r: &mut [f32], r_state: &mut [Biquad], g: f64) {
for (a, b) in l.iter_mut().zip(r.iter_mut()) {
let (mut x, mut y) = (*a as f64, *b as f64);
for ((band, sl), sr) in bands.iter().zip(l_state.iter_mut()).zip(r_state.iter_mut()) {
let m = band.mix.value() as f64;
if m > 0.0 {
let yl = sl.tick(&band.coeffs, x);
x += m * (yl - x);
let yr = sr.tick(&band.coeffs, y);
y += m * (yr - y);
}
}
*a = (x * g) as f32;
*b = (y * g) as f32;
}
}
macro_rules! eq_params {
($(($n:literal, $on:expr, $ty:expr, $f:expr, $q:expr)),* $(,)?) => {
&[
@@ -549,7 +577,8 @@ mod throughput_tests {
}
let n = [0, 1, 17, 64, 513][block % 5];
let input: Vec<f32> = (0..n).map(|i| if block > 400 { 0.0 } else { ((i + block * 13) as f32 * 0.17).sin() * 0.3 }).collect();
let mut a = [input.clone(), input.clone()];
let right: Vec<f32> = input.iter().enumerate().map(|(i, x)| if block > 450 { 0.0 } else { 0.5 * x + (i as f32 * 0.031).cos() * 0.2 }).collect();
let mut a = [input, right];
let mut b = a.clone();
let [al, ar] = &mut a;
let [bl, br] = &mut b;
@@ -561,4 +590,42 @@ mod throughput_tests {
assert_eq!(fast.counter, reference.counter);
}
}
#[test]
fn settled_eq_pairs_channels_like_original_loop() {
let kinds = [FilterType::HighPass, FilterType::LowShelf, FilterType::Peaking, FilterType::Notch, FilterType::HighShelf, FilterType::LowPass];
for (flat, nch) in [(false, 1), (false, 2), (false, 3), (false, 5), (true, 3)] {
let mut fast = BandBank::new(48000.0, nch, kinds.len());
for (i, kind) in kinds.iter().enumerate() {
let gain = if flat { 0.0 } else { 3.0 * i as f32 - 7.0 };
let kind = if flat { FilterType::Peaking } else { *kind };
fast.set_band(i, BandSettings::new(i != 2, kind, 40.0 * 3.1f32.powi(i as i32), gain, 0.5 + 0.4 * i as f32, true));
}
fast.reset();
let mut reference = fast.clone();
let mut gain = Smoothed::new(0.9, 0);
let mut ref_gain = gain;
for block in 0..40 {
let n = [512, 1, 17, 0, 64][block % 5];
let mut a: Vec<Vec<f32>> = (0..nch)
.map(|ch| {
(0..n)
.map(|i| match (ch, block) {
(1, 10..=12) if flat => f32::NAN,
(2, 20..) => 0.0,
_ => ((i + block * 31) as f32 * (0.05 + 0.07 * ch as f32)).sin() * 0.4,
})
.collect()
})
.collect();
let mut b = a.clone();
let mut ar: Vec<&mut [f32]> = a.iter_mut().map(|c| c.as_mut_slice()).collect();
let mut br: Vec<&mut [f32]> = b.iter_mut().map(|c| c.as_mut_slice()).collect();
fast.process(&mut ar, &mut gain);
reference.process_reference(&mut br, &mut ref_gain);
for (x, y) in a.iter().flatten().zip(b.iter().flatten()) {
assert_eq!(x.to_bits(), y.to_bits(), "nch {nch} flat {flat} block {block}");
}
}
}
}
}
crates/audio-dsp/src/effects/modulation.rs
+2 -2
View File
@@ -2,7 +2,7 @@
//! Delay. All modulated delays read with linear interpolation; feedback loops are bounded
//! (|g| < 1, or saturating for the Analog Delay).
use super::util::{AllPass1, DelayLine, Envelope, Lfo, OnePole, coef, uni_sine, uni_tri};
use super::util::{AllPass1, DelayLine, Envelope, Lfo, OnePole, coef, uni_sine, uni_tri, wrap_phase};
use crate::{AudioEffect, ParamSpec, ParamValues, Smoothed, Unit, block_len, db_to_gain, param_plumbing};
const SM: f32 = 20.0;
@@ -212,7 +212,7 @@ impl AudioEffect for Flanger {
let line = &mut self.lines[ch];
let y = line.read(d0 + (d1 - d0) * shape(p));
let wet = if special {
let y2 = line.read(d0 + (d1 - d0) * shape((p + 0.5).rem_euclid(1.0)));
let y2 = line.read(d0 + (d1 - d0) * shape(wrap_phase(p + 0.5)));
0.5 * (y - y2) + 0.5 * y
} else {
y
crates/audio-dsp/src/effects/pitch.rs
+91 -17
View File
@@ -16,7 +16,9 @@ const OVERLAP: usize = 8;
#[derive(Clone, Debug)]
struct PvState {
last_phase: Vec<f32>,
/// Previous frame's analysis spectrum (bins 0..=n/2); its phase is only needed at peaks.
last_re: Vec<f32>,
last_im: Vec<f32>,
/// Accumulated phase rotation per output bin (from the previous frame).
rot: Vec<f32>,
new_rot: Vec<f32>,
@@ -26,7 +28,6 @@ struct PvState {
#[derive(Clone, Debug)]
struct Scratch {
mag: Vec<f32>,
phase: Vec<f32>,
out_re: Vec<f32>,
out_im: Vec<f32>,
peaks: Vec<usize>,
@@ -65,14 +66,8 @@ impl PitchShifter {
hop: n / OVERLAP,
spec: Spectral::new(n, false),
stft: vec![StftChannel::new(n); ch],
state: vec![PvState { last_phase: vec![0.0; bins], rot: vec![0.0; bins], new_rot: vec![0.0; bins] }; ch],
scratch: Scratch {
mag: vec![0.0; bins],
phase: vec![0.0; bins],
out_re: vec![0.0; bins],
out_im: vec![0.0; bins],
peaks: Vec::with_capacity(bins),
},
state: vec![PvState { last_re: vec![0.0; bins], last_im: vec![0.0; bins], rot: vec![0.0; bins], new_rot: vec![0.0; bins] }; ch],
scratch: Scratch { mag: vec![0.0; bins], out_re: vec![0.0; bins], out_im: vec![0.0; bins], peaks: Vec::with_capacity(bins) },
target_ratio: 1.0,
ratio: 1.0,
};
@@ -95,7 +90,6 @@ fn shift_frame(spec: &mut Spectral, st: &mut PvState, sc: &mut Scratch, ratio: f
for k in 0..bins {
let (re, im) = (spec.re[k], spec.im[k]);
sc.mag[k] = (re * re + im * im).sqrt();
sc.phase[k] = im.atan2(re);
max_mag = max_mag.max(sc.mag[k]);
sc.out_re[k] = 0.0;
sc.out_im[k] = 0.0;
@@ -133,27 +127,32 @@ fn shift_frame(spec: &mut Spectral, st: &mut PvState, sc: &mut Scratch, ratio: f
bins
};
// Instantaneous frequency of the peak (bins).
let d = wrap(sc.phase[p] - st.last_phase[p] - p as f32 * expect);
let phase = spec.im[p].atan2(spec.re[p]);
let last_phase = st.last_im[p].atan2(st.last_re[p]);
let d = wrap(phase - last_phase - p as f32 * expect);
let f = p as f32 + d / expect;
let target = (p as f32 * ratio).round() as isize;
let delta = target - p as isize;
let prev_rot = if (0..bins as isize).contains(&target) { st.rot[target as usize] } else { 0.0 };
let rot = wrap(prev_rot + (ratio - 1.0) * f * expect);
// mag·e^{i(φ+rot)} = X·e^{i·rot}: one sin_cos per region instead of one per bin.
let (s, c) = rot.sin_cos();
for k in start..end {
let t = k as isize + delta;
if t < 0 || t >= bins as isize {
continue;
}
let t = t as usize;
let (s, c) = (sc.phase[k] + rot).sin_cos();
sc.out_re[t] += sc.mag[k] * c;
sc.out_im[t] += sc.mag[k] * s;
let (re, im) = (spec.re[k], spec.im[k]);
sc.out_re[t] += re * c - im * s;
sc.out_im[t] += re * s + im * c;
st.new_rot[t] = rot;
}
start = end;
}
for k in 0..bins {
st.last_phase[k] = sc.phase[k];
st.last_re[k] = spec.re[k];
st.last_im[k] = spec.im[k];
spec.re[k] = sc.out_re[k];
spec.im[k] = sc.out_im[k];
}
@@ -168,7 +167,8 @@ impl AudioEffect for PitchShifter {
fn reset(&mut self) {
self.stft.iter_mut().for_each(StftChannel::reset);
for s in &mut self.state {
s.last_phase.iter_mut().for_each(|v| *v = 0.0);
s.last_re.iter_mut().for_each(|v| *v = 0.0);
s.last_im.iter_mut().for_each(|v| *v = 0.0);
s.rot.iter_mut().for_each(|v| *v = 0.0);
}
self.ratio = self.target_ratio;
@@ -241,6 +241,80 @@ mod tests {
}
}
/// The previous per-bin version of `shift_frame` (atan2 and sin_cos for every bin).
fn shift_frame_reference(re: &mut [f32], im: &mut [f32], last_phase: &mut [f32], rot: &mut [f32], ratio: f32, expect: f32) {
let bins = last_phase.len();
let mag: Vec<f32> = (0..bins).map(|k| (re[k] * re[k] + im[k] * im[k]).sqrt()).collect();
let phase: Vec<f32> = (0..bins).map(|k| im[k].atan2(re[k])).collect();
let max_mag = mag.iter().fold(0.0f32, |a, &m| a.max(m));
let (mut out_re, mut out_im, mut new_rot) = (vec![0.0f32; bins], vec![0.0f32; bins], vec![0.0f32; bins]);
let floor = max_mag * 1e-5;
let peaks: Vec<usize> = (0..bins)
.filter(|&k| {
let m = mag[k];
m > floor && (k.saturating_sub(2)..k).all(|j| mag[j] < m) && (k + 1..=(k + 2).min(bins - 1)).all(|j| mag[j] <= m)
})
.collect();
let mut start = 0usize;
for (i, &p) in peaks.iter().enumerate() {
let end = match peaks.get(i + 1) {
Some(&q) => (p + 1..q).fold(p + 1, |mi, j| if mag[j] < mag[mi] { j } else { mi }),
None => bins,
};
let d = wrap(phase[p] - last_phase[p] - p as f32 * expect);
let f = p as f32 + d / expect;
let target = (p as f32 * ratio).round() as isize;
let delta = target - p as isize;
let prev_rot = if (0..bins as isize).contains(&target) { rot[target as usize] } else { 0.0 };
let r = wrap(prev_rot + (ratio - 1.0) * f * expect);
for k in start..end {
let t = k as isize + delta;
if t < 0 || t >= bins as isize {
continue;
}
let t = t as usize;
let (s, c) = (phase[k] + r).sin_cos();
out_re[t] += mag[k] * c;
out_im[t] += mag[k] * s;
new_rot[t] = r;
}
start = end;
}
last_phase.copy_from_slice(&phase);
re[..bins].copy_from_slice(&out_re);
im[..bins].copy_from_slice(&out_im);
rot.copy_from_slice(&new_rot);
}
#[test]
fn matches_per_bin_reference_to_rounding() {
let n = fft_size(SR);
let (bins, hop) = (n / 2 + 1, n / OVERLAP);
let expect = 2.0 * PI * hop as f32 / n as f32;
let mut rng = Rng::new(3);
let tones = [sine(220.0, 0.3, SR as f64, 40 * hop + n, 0.0), sine(1234.5, 0.2, SR as f64, 40 * hop + n, 1.0)];
let x: Vec<f32> = (0..40 * hop + n).map(|i| tones[0][i] + tones[1][i] + 0.05 * rng.uniform()).collect();
let mut spec = Spectral::new(n, false);
let mut st = PvState { last_re: vec![0.0; bins], last_im: vec![0.0; bins], rot: vec![0.0; bins], new_rot: vec![0.0; bins] };
let mut sc = Scratch { mag: vec![0.0; bins], out_re: vec![0.0; bins], out_im: vec![0.0; bins], peaks: Vec::new() };
let (mut last_phase, mut rot) = (vec![0.0f32; bins], vec![0.0f32; bins]);
let (mut ratio, mut worst) = (1.0f32, 0.0f32);
for f in 0..40 {
let target = if f < 20 { 2f32.powf(7.0 / 12.0) } else { 2f32.powf(-5.0 / 12.0) };
ratio += (target - ratio) * 0.5;
spec.analyse(&x[f * hop..f * hop + n]);
let (mut re, mut im) = (spec.re.clone(), spec.im.clone());
shift_frame(&mut spec, &mut st, &mut sc, ratio, expect);
shift_frame_reference(&mut re, &mut im, &mut last_phase, &mut rot, ratio, expect);
assert!(st.rot.iter().zip(&rot).all(|(a, b)| a.to_bits() == b.to_bits()), "frame {f}: rotations differ");
let peak = (0..bins).map(|k| re[k].hypot(im[k])).fold(0.0f32, f32::max);
for k in 0..bins {
worst = worst.max((spec.re[k] - re[k]).hypot(spec.im[k] - im[k]) / peak);
}
}
assert!(worst < 1e-5, "relative deviation {worst}");
}
#[test]
fn zero_shift_is_delayed_identity() {
let (y, lat) = shifted(0.0, 440.0);
crates/audio-dsp/src/effects/reverbs.rs
+61 -1
View File
@@ -96,6 +96,8 @@ pub struct ConvolutionReverb {
pre: DelayLine,
dry_delay: Vec<DelayLine>,
key: (usize, f32, f32, f32),
/// Impulse, size or damping changed since the last rebuild; rebuilt at the next `process`.
dirty: bool,
mix: Smoothed,
gain: Smoothed,
width: Smoothed,
@@ -138,6 +140,7 @@ impl ConvolutionReverb {
pre: DelayLine::new((0.5 * sample_rate) as usize + 2),
dry_delay: vec![DelayLine::new(PART + 2); channels.max(1)],
key: (usize::MAX, 0.0, 0.0, 0.0),
dirty: false,
mix: Smoothed::with_ms(0.0, sample_rate, 20.0),
gain: Smoothed::with_ms(1.0, sample_rate, 20.0),
width: Smoothed::with_ms(1.0, sample_rate, 20.0),
@@ -205,7 +208,11 @@ impl ConvolutionReverb {
}
fn apply_params(&mut self, snap: bool) {
self.rebuild();
if snap {
self.rebuild();
} else {
self.dirty = true;
}
self.mix.set(self.pv.v("mix") / 100.0);
self.gain.set(db_to_gain(self.pv.v("gain")));
self.width.set(self.pv.v("width") / 100.0);
@@ -296,6 +303,10 @@ impl AudioEffect for ConvolutionReverb {
if nch == 0 {
return;
}
if self.dirty {
self.dirty = false;
self.rebuild();
}
for i in 0..n {
let (mix, gain, width, pd) = (self.mix.tick(), self.gain.tick(), self.width.tick(), self.predelay.tick());
let mono = channels[..nch].iter().map(|c| c[i]).sum::<f32>() / nch as f32;
@@ -472,3 +483,52 @@ impl AudioEffect for SurroundReverb {
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::testutil::Rng;
#[test]
fn set_param_defers_the_impulse_rebuild_to_process() {
let mut fx = ConvolutionReverb::new(48_000.0, 2);
let key = fx.key;
fx.set_param("damping_hf", 8000.0);
fx.set_param("room_size", 60.0);
assert_eq!(fx.key, key, "set_param rebuilt the impulse");
fx.process(&mut [&mut [0.0; 16][..], &mut [0.0; 16][..]]);
assert_eq!(fx.key, (key.0, 60.0, key.2, 8000.0));
assert!(!fx.dirty);
}
#[test]
fn deferred_rebuild_output_is_identical() {
let changes: &[&[(&str, f32)]] = &[
&[("room_size", 70.0)],
&[],
&[("damping_lf", 200.0), ("damping_hf", 6000.0)],
&[("impulse", 2.0), ("mix", 80.0)],
&[("room_size", 30.0), ("room_size", 90.0)],
&[("impulse", 4.0), ("damping_hf", 20_000.0), ("damping_lf", 10.0)],
];
let mut eager = ConvolutionReverb::new(48_000.0, 2);
let mut deferred = ConvolutionReverb::new(48_000.0, 2);
let mut rng = Rng::new(5);
for (block, set) in changes.iter().cycle().take(24).enumerate() {
for &(id, v) in *set {
eager.set_param(id, v);
eager.rebuild();
deferred.set_param(id, v);
}
let n = 300 + 37 * block;
let l: Vec<f32> = (0..n).map(|_| rng.uniform()).collect();
let r: Vec<f32> = (0..n).map(|_| rng.uniform()).collect();
let (mut l1, mut r1, mut l2, mut r2) = (l.clone(), r.clone(), l, r);
eager.process(&mut [&mut l1, &mut r1]);
deferred.process(&mut [&mut l2, &mut r2]);
for (a, b) in l1.iter().chain(&r1).zip(l2.iter().chain(&r2)) {
assert_eq!(a.to_bits(), b.to_bits(), "block {block}");
}
}
}
}
crates/audio-dsp/src/effects/util.rs
+54 -3
View File
@@ -43,8 +43,10 @@ impl DelayLine {
let d = d.clamp(0.0, self.max_delay());
let di = d.floor();
let frac = d - di;
let i0 = (self.w + len - 1 - di as usize) % len;
let i1 = (i0 + len - 1) % len;
// `w < len` and `di <= len - 3`, so one conditional subtract wraps the index.
let i0 = self.w + len - 1 - di as usize;
let i0 = if i0 >= len { i0 - len } else { i0 };
let i1 = if i0 == 0 { len - 1 } else { i0 - 1 };
self.buf[i0] + (self.buf[i1] - self.buf[i0]) * frac
}
/// Integer read (`d` samples ago, 0 = newest).
@@ -76,7 +78,7 @@ impl Lfo {
/// Current phase plus `offset` (cycles), wrapped to 0..1.
#[inline]
pub(crate) fn phase(&self, offset: f64) -> f64 {
(self.phase + offset).rem_euclid(1.0)
wrap_phase(self.phase + offset)
}
#[inline]
pub(crate) fn advance(&mut self, hz: f32, sr: f32) {
@@ -87,6 +89,18 @@ impl Lfo {
}
}
/// `p.rem_euclid(1.0)`, without the `fmod` for phase sums in 0..2 (`p - 1.0` is exact there).
#[inline]
pub(crate) fn wrap_phase(p: f64) -> f64 {
if (0.0..1.0).contains(&p) {
p
} else if (1.0..2.0).contains(&p) {
p - 1.0
} else {
p.rem_euclid(1.0)
}
}
/// Sine of a phase in cycles, mapped to 0..1.
#[inline]
pub(crate) fn uni_sine(phase: f64) -> f32 {
@@ -228,6 +242,43 @@ mod tests {
assert!((d.read(1.5) - 2.5).abs() < 1e-6);
}
#[test]
fn delay_read_matches_modulo_indexing() {
let reference = |line: &DelayLine, d: f32| {
let len = line.buf.len();
let d = d.clamp(0.0, line.max_delay());
let di = d.floor();
let frac = d - di;
let i0 = (line.w + len - 1 - di as usize) % len;
let i1 = (i0 + len - 1) % len;
line.buf[i0] + (line.buf[i1] - line.buf[i0]) * frac
};
for max in [0, 1, 2, 7] {
let mut line = DelayLine::new(max);
for k in 0..3 * (max + 3) {
line.push(k as f32 * 0.37 - 1.0);
for d in (-8..=4 * max as i32 + 12).map(|i| i as f32 * 0.25).chain([f32::NAN, f32::INFINITY, -f32::INFINITY, 1e30]) {
assert_eq!(line.read(d).to_bits(), reference(&line, d).to_bits(), "max {max} w {} d {d}", line.w);
}
}
}
}
#[test]
fn wrap_phase_matches_rem_euclid() {
let edges = [0.0, -0.0, 1.0, 2.0, 3.0, -1.0, 0.5, 1.5, 1e300, f64::MIN_POSITIVE, f64::INFINITY, f64::NEG_INFINITY, f64::NAN];
let near = edges.iter().flat_map(|&p: &f64| [p, f64::from_bits(p.to_bits().wrapping_add(1)), f64::from_bits(p.to_bits().wrapping_sub(1))]);
let mut r = Prng::new(7);
let sums = (0..100_000).map(|_| {
let mut lfo = Lfo::default();
lfo.advance((r.uniform() + 1.0) * 24000.0, 48000.0);
lfo.phase + (r.uniform() as f64 + 1.0) * 1.5
});
for p in near.chain(sums) {
assert_eq!(wrap_phase(p).to_bits(), p.rem_euclid(1.0).to_bits(), "{p:e}");
}
}
#[test]
fn allpass_is_unit_magnitude() {
let a = AllPass1::coef(1000.0, 48000.0);
crates/audio-dsp/src/fft.rs
+124 -27
View File
@@ -9,9 +9,12 @@ use std::f64::consts::PI;
#[derive(Clone, Debug)]
pub struct Fft {
n: usize,
/// e^{-2πik/n} for k in 0..n/2.
/// Per-stage twiddles, contiguous: the stage with half-length `h` reads e^{-2πik/(2h)} for
/// k in 0..h at offset `h - 1` (values taken from the n-point table, so they match it exactly).
tw_re: Vec<f32>,
tw_im: Vec<f32>,
/// `tw_im` negated, for the inverse transform.
tw_im_inv: Vec<f32>,
rev: Vec<u32>,
}
@@ -21,13 +24,19 @@ impl Fft {
assert!(n >= 2 && n.is_power_of_two(), "FFT size must be a power of two");
let bits = n.trailing_zeros();
let rev = (0..n as u32).map(|i| i.reverse_bits() >> (32 - bits)).collect();
let (tw_re, tw_im) = (0..n / 2)
.map(|k| {
let a = -2.0 * PI * k as f64 / n as f64;
(a.cos() as f32, a.sin() as f32)
})
.unzip();
Fft { n, tw_re, tw_im, rev }
let (mut tw_re, mut tw_im) = (Vec::with_capacity(n - 1), Vec::with_capacity(n - 1));
let mut half = 1;
while half < n {
let stride = n / (2 * half);
for k in 0..half {
let a = -2.0 * PI * (k * stride) as f64 / n as f64;
tw_re.push(a.cos() as f32);
tw_im.push(a.sin() as f32);
}
half <<= 1;
}
let tw_im_inv = tw_im.iter().map(|w| -w).collect();
Fft { n, tw_re, tw_im, tw_im_inv, rev }
}
pub fn len(&self) -> usize {
@@ -51,6 +60,7 @@ impl Fft {
fn run(&self, re: &mut [f32], im: &mut [f32], inverse: bool) {
let n = self.n;
assert!(re.len() >= n && im.len() >= n);
let (re, im) = (&mut re[..n], &mut im[..n]);
for i in 0..n {
let j = self.rev[i] as usize;
if j > i {
@@ -58,28 +68,35 @@ impl Fft {
im.swap(i, j);
}
}
let sign = if inverse { -1.0 } else { 1.0 };
let mut len = 2;
while len <= n {
let half = len / 2;
let stride = n / len;
let mut start = 0;
while start < n {
let tw_im = if inverse { &self.tw_im_inv } else { &self.tw_im };
// First stage: the twiddle is 1 + i·(∓0). The multiplications by 1 are exact and dropped;
// the ones by ∓0 stay so signed zeros come out exactly as in the general butterfly.
let wi = tw_im[0];
for (r, i) in re.as_chunks_mut::<2>().0.iter_mut().zip(im.as_chunks_mut::<2>().0) {
let xr = r[1] - i[1] * wi;
let xi = r[1] * wi + i[1];
r[1] = r[0] - xr;
i[1] = i[0] - xi;
r[0] += xr;
i[0] += xi;
}
let mut half = 2;
while half < n {
let wr = &self.tw_re[half - 1..2 * half - 1];
let wi = &tw_im[half - 1..2 * half - 1];
for (r, i) in re.chunks_exact_mut(2 * half).zip(im.chunks_exact_mut(2 * half)) {
let (ra, rb) = r.split_at_mut(half);
let (ia, ib) = i.split_at_mut(half);
for k in 0..half {
let wr = self.tw_re[k * stride];
let wi = sign * self.tw_im[k * stride];
let a = start + k;
let b = a + half;
let xr = re[b] * wr - im[b] * wi;
let xi = re[b] * wi + im[b] * wr;
re[b] = re[a] - xr;
im[b] = im[a] - xi;
re[a] += xr;
im[a] += xi;
let xr = rb[k] * wr[k] - ib[k] * wi[k];
let xi = rb[k] * wi[k] + ib[k] * wr[k];
rb[k] = ra[k] - xr;
ib[k] = ia[k] - xi;
ra[k] += xr;
ia[k] += xi;
}
start += len;
}
len <<= 1;
half <<= 1;
}
}
}
@@ -112,4 +129,84 @@ mod tests {
assert!((im[t] / n as f32).abs() < 1e-5);
}
}
/// The previous kernel (strided reads from one n/2-entry twiddle table), kept as reference.
fn run_reference(n: usize, re: &mut [f32], im: &mut [f32], inverse: bool) {
let rev: Vec<u32> = (0..n as u32).map(|i| i.reverse_bits() >> (32 - n.trailing_zeros())).collect();
let (tw_re, tw_im): (Vec<f32>, Vec<f32>) = (0..n / 2)
.map(|k| {
let a = -2.0 * PI * k as f64 / n as f64;
(a.cos() as f32, a.sin() as f32)
})
.unzip();
for i in 0..n {
let j = rev[i] as usize;
if j > i {
re.swap(i, j);
im.swap(i, j);
}
}
let sign = if inverse { -1.0 } else { 1.0 };
let mut len = 2;
while len <= n {
let half = len / 2;
let stride = n / len;
let mut start = 0;
while start < n {
for k in 0..half {
let wr = tw_re[k * stride];
let wi = sign * tw_im[k * stride];
let a = start + k;
let b = a + half;
let xr = re[b] * wr - im[b] * wi;
let xi = re[b] * wi + im[b] * wr;
re[b] = re[a] - xr;
im[b] = im[a] - xi;
re[a] += xr;
im[a] += xi;
}
start += len;
}
len <<= 1;
}
}
#[test]
fn bit_identical_to_reference_kernel() {
let mut rng = crate::testutil::Rng::new(17);
for bits in 1..=13 {
let n = 1usize << bits;
let fft = Fft::new(n);
for case in 0..4 {
let mut re: Vec<f32> = (0..n).map(|_| rng.uniform()).collect();
let mut im: Vec<f32> = match case {
0 => vec![0.0; n],
1 => vec![-0.0; n],
_ => (0..n).map(|_| rng.uniform() * 1e3).collect(),
};
if case == 3 {
// Exact and signed zeros, subnormals and large values.
for (i, (r, m)) in re.iter_mut().zip(im.iter_mut()).enumerate() {
match i % 5 {
0 => (*r, *m) = (0.0, -0.0),
1 => (*r, *m) = (-0.0, 0.0),
2 => *r = 1e-40,
3 => *m = -3e30,
_ => {}
}
}
}
for inverse in [false, true] {
let (mut r1, mut i1) = (re.clone(), im.clone());
let (mut r2, mut i2) = (re.clone(), im.clone());
fft.run(&mut r1, &mut i1, inverse);
run_reference(n, &mut r2, &mut i2, inverse);
for k in 0..n {
assert_eq!(r1[k].to_bits(), r2[k].to_bits(), "n {n} case {case} inverse {inverse} re {k}");
assert_eq!(i1[k].to_bits(), i2[k].to_bits(), "n {n} case {case} inverse {inverse} im {k}");
}
}
}
}
}
}
crates/audio-dsp/src/lib.rs
+4 -3
View File
@@ -224,16 +224,17 @@ pub fn gain_to_db(g: f32) -> f32 {
20.0 * g.abs().log10()
}
/// Flush tiny values to zero (denormal protection for recursive state).
/// Flush tiny values to zero (denormal protection for recursive state). Non-finite values are
/// zeroed too, so one NaN or infinite input sample cannot poison the state until `reset`.
#[inline(always)]
pub(crate) fn flush(x: f64) -> f64 {
if x.abs() < 1e-30 { 0.0 } else { x }
if (1e-30..=f64::MAX).contains(&x.abs()) { x } else { 0.0 }
}
/// f32 variant of [`flush`] for f32 delay lines.
#[inline(always)]
pub(crate) fn flush32(x: f32) -> f32 {
if x.abs() < 1e-25 { 0.0 } else { x }
if (1e-25..=f32::MAX).contains(&x.abs()) { x } else { 0.0 }
}
/// Common block length of planar channels.
crates/audio-dsp/src/oversample.rs
+60 -4
View File
@@ -109,15 +109,26 @@ impl Interpolator {
self.hist[self.pos + TAPS_PER_PHASE] = x;
let win = &self.hist[self.pos..self.pos + TAPS_PER_PHASE];
let mut peak = 0.0f32;
for p in 0..bank.factor {
let c = &bank.coeffs[p * TAPS_PER_PHASE..(p + 1) * TAPS_PER_PHASE];
let y: f32 = c.iter().zip(win).map(|(a, b)| a * b).sum();
peak = peak.max(y.abs());
for c in bank.coeffs.as_chunks::<TAPS_PER_PHASE>().0 {
peak = peak.max(dot(c, win).abs());
}
peak
}
}
/// Dot product of one phase with the window. Eight partial sums (instead of one sequential sum)
/// let the compiler vectorise it; the result differs from the sequential sum only by rounding.
#[inline(always)]
fn dot(c: &[f32; TAPS_PER_PHASE], win: &[f32]) -> f32 {
let mut acc = [0.0f32; 8];
for (cc, ww) in c.as_chunks::<8>().0.iter().zip(win.as_chunks::<8>().0) {
for ((a, &ci), &wi) in acc.iter_mut().zip(cc).zip(ww) {
*a += ci * wi;
}
}
(acc[0] + acc[4]) + (acc[1] + acc[5]) + (acc[2] + acc[6]) + (acc[3] + acc[7])
}
#[cfg(test)]
mod tests {
use super::*;
@@ -141,4 +152,49 @@ mod tests {
}
assert!((peak - 1.0).abs() < 0.02, "peak {peak}");
}
/// The previous sequential-sum interpolator.
fn push_peak_reference(ip: &mut Interpolator, bank: &PolyphaseBank, x: f32) -> f32 {
ip.pos = if ip.pos == 0 { TAPS_PER_PHASE - 1 } else { ip.pos - 1 };
ip.hist[ip.pos] = x;
ip.hist[ip.pos + TAPS_PER_PHASE] = x;
let win = &ip.hist[ip.pos..ip.pos + TAPS_PER_PHASE];
let mut peak = 0.0f32;
for p in 0..bank.factor {
let c = &bank.coeffs[p * TAPS_PER_PHASE..(p + 1) * TAPS_PER_PHASE];
let y: f32 = c.iter().zip(win).map(|(a, b)| a * b).sum();
peak = peak.max(y.abs());
}
peak
}
#[test]
fn partial_sums_match_sequential_sum_to_rounding() {
let mut rng = crate::testutil::Rng::new(9);
let mut signals: Vec<Vec<f32>> = vec![(0..48_000).map(|_| rng.uniform()).collect(), (0..48_000).map(|_| 1e-4 * rng.uniform()).collect()];
for f in [997.0, 11_025.0, 19_000.0, 23_900.0] {
signals.push(crate::testutil::sine(f, 0.99, 48_000.0, 48_000, 0.3));
}
let (mut max_abs, mut max_rel, mut max_db) = (0.0f32, 0.0f32, 0.0f64);
for factor in [2, 4] {
let bank = PolyphaseBank::new(factor);
for x in &signals {
let (mut a, mut b) = (Interpolator::default(), Interpolator::default());
let (mut pa, mut pb) = (0.0f32, 0.0f32);
for &v in x {
let (ya, yb) = (a.push_peak(&bank, v), push_peak_reference(&mut b, &bank, v));
max_abs = max_abs.max((ya - yb).abs());
if yb > 0.0 {
max_rel = max_rel.max((ya - yb).abs() / yb);
}
pa = pa.max(ya);
pb = pb.max(yb);
}
max_db = max_db.max((20.0 * (pa as f64 / pb as f64).log10()).abs());
}
}
println!("true-peak interpolator vs sequential sum: max abs {max_abs:e}, max rel {max_rel:e}, max peak {max_db:e} dB");
assert!(max_abs < 1e-6, "abs {max_abs}");
assert!(max_db < 1e-5, "dB {max_db}");
}
}
crates/audio-dsp/src/remix.rs
+139 -28
View File
@@ -309,6 +309,48 @@ fn mel_to_hz(m: f64) -> f64 {
700.0 * (10f64.powf(m / 2595.0) - 1.0)
}
/// Mel bands of the cepstra.
const BANDS: usize = 26;
/// Triangular mel filterbank (26 bands, 30 Hz … min(8 kHz, Nyquist)) for an `n`-point STFT: per
/// band, the bins it covers with their weights, in increasing bin order. Every other bin has
/// weight 0 in that band.
fn mel_weights(sr: u32, n: usize) -> Vec<Vec<(usize, f64)>> {
let hz = |k: usize| k as f64 * sr as f64 / n as f64;
let fmax = (sr as f64 / 2.0).min(8000.0);
let (m0, m1) = (hz_to_mel(30.0), hz_to_mel(fmax));
let edges: Vec<f64> = (0..BANDS + 2).map(|i| mel_to_hz(m0 + (m1 - m0) * i as f64 / (BANDS + 1) as f64)).collect();
(0..BANDS)
.map(|b| {
let (l, cn, r) = (edges[b], edges[b + 1], edges[b + 2]);
(0..n / 2 + 1)
.filter_map(|k| {
let f = hz(k);
if f > l && f < cn {
Some((k, (f - l) / (cn - l)))
} else if f >= cn && f < r {
Some((k, (r - f) / (r - cn)))
} else {
None
}
})
.collect()
})
.collect()
}
/// Mel band energies of one magnitude spectrum.
fn mel_bands(mel: &[Vec<(usize, f64)>], mag: &[f32]) -> [f64; BANDS] {
let mut band = [0f64; BANDS];
for (e, weights) in band.iter_mut().zip(mel) {
for &(k, w) in weights {
let v = mag[k];
*e += w * (v as f64) * (v as f64);
}
}
band
}
/// Per-beat chroma (12) and cepstra (12) from a 4× larger STFT.
fn beat_features(x: &[f32], sr: u32, beats: &[i64]) -> (Vec<[f32; 12]>, Vec<[f32; 12]>) {
let n = (onset_frame(sr) * 2).max(1024);
@@ -323,11 +365,7 @@ fn beat_features(x: &[f32], sr: u32, beats: &[i64]) -> (Vec<[f32; 12]>, Vec<[f32
(55.0..=5000.0).contains(&f).then(|| (((12.0 * (f / 440.0).log2()).round() as i64 + 9).rem_euclid(12)) as usize)
})
.collect();
// mel filterbank (26 bands, 30 Hz … min(8 kHz, Nyquist))
const BANDS: usize = 26;
let fmax = (sr as f64 / 2.0).min(8000.0);
let (m0, m1) = (hz_to_mel(30.0), hz_to_mel(fmax));
let edges: Vec<f64> = (0..BANDS + 2).map(|i| mel_to_hz(m0 + (m1 - m0) * i as f64 / (BANDS + 1) as f64)).collect();
let mel = mel_weights(sr, n);
let mut frame_chroma = Vec::with_capacity(mags.len());
let mut frame_cep = Vec::with_capacity(mags.len());
for m in &mags {
@@ -338,21 +376,7 @@ fn beat_features(x: &[f32], sr: u32, beats: &[i64]) -> (Vec<[f32; 12]>, Vec<[f32
}
}
frame_chroma.push(c);
let mut band = [0f64; BANDS];
for (b, e) in band.iter_mut().enumerate() {
let (l, cn, r) = (edges[b], edges[b + 1], edges[b + 2]);
for (k, v) in m.iter().enumerate() {
let f = hz(k);
let w = if f > l && f < cn {
(f - l) / (cn - l)
} else if f >= cn && f < r {
(r - f) / (r - cn)
} else {
0.0
};
*e += w * (*v as f64) * (*v as f64);
}
}
let band = mel_bands(&mel, m);
let lb: Vec<f64> = band.iter().map(|e| (e + 1e-10).ln()).collect();
let mut cep = [0f32; 12];
for (q, c) in cep.iter_mut().enumerate() {
@@ -404,13 +428,30 @@ fn similarity_cells(beats: usize) -> Result<usize, RemixError> {
beats.checked_mul(beats).filter(|&n| n <= MAX_SIMILARITY_CELLS).ok_or(RemixError::TooComplex)
}
fn cos(a: &[f32; 12], b: &[f32; 12]) -> f32 {
fn norm(a: &[f32; 12]) -> f32 {
a.iter().map(|v| v * v).sum::<f32>().sqrt()
}
/// Cosine similarity of `a` and `b`, given their norms.
fn cos(a: &[f32; 12], na: f32, b: &[f32; 12], nb: f32) -> f32 {
let d: f32 = a.iter().zip(b).map(|(x, y)| x * y).sum();
let na = a.iter().map(|v| v * v).sum::<f32>().sqrt();
let nb = b.iter().map(|v| v * v).sum::<f32>().sqrt();
if na > 0.0 && nb > 0.0 { d / (na * nb) } else { 0.0 }
}
/// Fills the beat self-similarity matrix `sim` (`m²`, row-major, symmetric).
fn fill_similarity(sim: &mut [f32], chroma: &[[f32; 12]], cep: &[[f32; 12]]) {
let m = chroma.len();
let chroma_norm: Vec<f32> = chroma.iter().map(norm).collect();
let cep_norm: Vec<f32> = cep.iter().map(norm).collect();
for i in 0..m {
for j in i..m {
let s = 0.6 * cos(&chroma[i], chroma_norm[i], &chroma[j], chroma_norm[j]) + 0.4 * (1.0 + cos(&cep[i], cep_norm[i], &cep[j], cep_norm[j])) * 0.5;
sim[i * m + j] = s;
sim[j * m + i] = s;
}
}
}
/// How far [`Analysis::beats`] sit before the attack of each beat (half a crossfade), samples.
pub fn beat_lead(sample_rate: u32) -> i64 {
(XFADE_S * 0.5 * sample_rate as f64).round() as i64
@@ -465,11 +506,7 @@ pub fn analyze(channels: &[&[f32]], sample_rate: u32) -> Result<Analysis, RemixE
let (chroma, cep) = beat_features(&mono, sample_rate, &beats);
let m = beats.len();
let mut sim = vec![0f32; similarity_cells(m)?];
for i in 0..m {
for j in 0..m {
sim[i * m + j] = 0.6 * cos(&chroma[i], &chroma[j]) + 0.4 * (1.0 + cos(&cep[i], &cep[j])) * 0.5;
}
}
fill_similarity(&mut sim, &chroma, &cep);
let period_s = period * hop as f64;
Ok(Analysis { sample_rate, len: len as i64, beats, period: period_s, bpm: 60.0 * sample_rate as f64 / period_s, sim })
}
@@ -896,6 +933,80 @@ mod tests {
}
}
/// The mel bands and similarity matrix as computed before the sparse filterbank and the
/// precomputed norms.
fn reference_bands(x: &[f32], sr: u32) -> Vec<[f64; BANDS]> {
let n = (onset_frame(sr) * 2).max(1024);
let hz = |k: usize| k as f64 * sr as f64 / n as f64;
let fmax = (sr as f64 / 2.0).min(8000.0);
let (m0, m1) = (hz_to_mel(30.0), hz_to_mel(fmax));
let edges: Vec<f64> = (0..BANDS + 2).map(|i| mel_to_hz(m0 + (m1 - m0) * i as f64 / (BANDS + 1) as f64)).collect();
stft_mag(x, n, n / 2)
.iter()
.map(|m| {
let mut band = [0f64; BANDS];
for (b, e) in band.iter_mut().enumerate() {
let (l, cn, r) = (edges[b], edges[b + 1], edges[b + 2]);
for (k, v) in m.iter().enumerate() {
let f = hz(k);
let w = if f > l && f < cn {
(f - l) / (cn - l)
} else if f >= cn && f < r {
(r - f) / (r - cn)
} else {
0.0
};
*e += w * (*v as f64) * (*v as f64);
}
}
band
})
.collect()
}
fn reference_similarity(chroma: &[[f32; 12]], cep: &[[f32; 12]]) -> Vec<f32> {
fn cos(a: &[f32; 12], b: &[f32; 12]) -> f32 {
let d: f32 = a.iter().zip(b).map(|(x, y)| x * y).sum();
let na = a.iter().map(|v| v * v).sum::<f32>().sqrt();
let nb = b.iter().map(|v| v * v).sum::<f32>().sqrt();
if na > 0.0 && nb > 0.0 { d / (na * nb) } else { 0.0 }
}
let m = chroma.len();
let mut sim = vec![0f32; m * m];
for i in 0..m {
for j in 0..m {
sim[i * m + j] = 0.6 * cos(&chroma[i], &chroma[j]) + 0.4 * (1.0 + cos(&cep[i], &cep[j])) * 0.5;
}
}
sim
}
#[test]
fn sparse_mel_bands_and_symmetric_similarity_are_bit_identical() {
for sr in [8_000, 11_025, 22_050, 44_100, 48_000, 96_000] {
let (m, truth) = test_music(sr, 120.0, 8, 3);
let x = &m[0];
let n = (onset_frame(sr) * 2).max(1024);
let mel = mel_weights(sr, n);
let mags = stft_mag(x, n, n / 2);
let want = reference_bands(x, sr);
assert_eq!(mags.len(), want.len());
for (frame, want) in mags.iter().zip(want) {
assert_eq!(mel_bands(&mel, frame).map(f64::to_bits), want.map(f64::to_bits), "{sr} Hz");
}
let (chroma, cep) = beat_features(x, sr, &truth);
let mut sim = vec![0f32; chroma.len() * chroma.len()];
fill_similarity(&mut sim, &chroma, &cep);
let want = reference_similarity(&chroma, &cep);
assert_eq!(sim.iter().map(|v| v.to_bits()).collect::<Vec<_>>(), want.iter().map(|v| v.to_bits()).collect::<Vec<_>>(), "{sr} Hz");
}
// zero vectors (silent beats) keep their 0 cosine
let z = [[0f32; 12]; 3];
let mut sim = vec![0f32; 9];
fill_similarity(&mut sim, &z, &z);
assert_eq!(sim, reference_similarity(&z, &z));
}
#[test]
fn text_roundtrip() {
let p = Plan { pieces: vec![Piece { src: 0, len: 10 }, Piece { src: 50, len: 7 }], xfade: 3 };
crates/audio-dsp/src/resample.rs
+90 -6
View File
@@ -33,11 +33,12 @@ impl StreamResampler {
(self.from, self.to)
}
/// The source frame whose sample lands at (or just before) output frame `out`.
fn source_index(&self, out: i64) -> (i64, f32) {
/// The source frame whose sample lands at (or just before) output frame `out`, and the
/// remainder of the exact position in units of `1 / to` source frames.
fn source_index(&self, out: i64) -> (i64, i64) {
let p = out as i128 * self.from as i128;
let to = self.to as i128;
(p.div_euclid(to) as i64, (p.rem_euclid(to) as f64 / to as f64) as f32)
(p.div_euclid(to) as i64, p.rem_euclid(to) as i64)
}
/// Output frames `[out, out + n)`, planar. `pull(start, frames)` returns source frames
@@ -47,7 +48,7 @@ impl StreamResampler {
if n == 0 {
return vec![Vec::new(); self.held.len().max(1)];
}
let (first, _) = self.source_index(out);
let (first, first_rem) = self.source_index(out);
let (last, _) = self.source_index(out.saturating_add(n as i64 - 1));
// interpolation reads source frames first - 1 ..= last
if self.next_out != Some(out) || self.held.is_empty() {
@@ -67,8 +68,11 @@ impl StreamResampler {
}
}
let mut dst = vec![vec![0.0f32; n]; self.held.len()];
for (i, frame) in (out..).take(n).enumerate() {
let (s, frac) = self.source_index(frame);
let to = self.to as i64;
let (step, rem_step) = ((self.from / self.to) as i64, (self.from % self.to) as i64);
let (mut s, mut rem) = (first, first_rem);
for i in 0..n {
let frac = (rem as f64 / to as f64) as f32;
// held starts at first - 1, so the earlier frame is always held
let k = usize::try_from(s - self.held_start).unwrap_or(0);
for (d, h) in dst.iter_mut().zip(&self.held) {
@@ -78,6 +82,12 @@ impl StreamResampler {
*x = a + (b - a) * frac;
}
}
s = s.wrapping_add(step);
rem += rem_step;
if rem >= to {
rem -= to;
s = s.wrapping_add(1);
}
}
// keep what the next block can still read: from its first frame - 1 on
let (next_first, _) = self.source_index(out.saturating_add(n as i64));
@@ -133,6 +143,80 @@ mod tests {
}
}
/// `process` as it was with one i128 division per output sample.
fn reference_process(r: &mut StreamResampler, out: i64, n: usize, mut pull: impl FnMut(i64, usize) -> Vec<Vec<f32>>) -> Vec<Vec<f32>> {
let (from, to) = (r.from as i128, r.to as i128);
let index = |out: i64| {
let p = out as i128 * from;
(p.div_euclid(to) as i64, (p.rem_euclid(to) as f64 / to as f64) as f32)
};
if n == 0 {
return vec![Vec::new(); r.held.len().max(1)];
}
let (first, _) = index(out);
let (last, _) = index(out.saturating_add(n as i64 - 1));
if r.next_out != Some(out) || r.held.is_empty() {
r.held_start = first - 1;
r.held.clear();
}
let held_end = r.held_start + r.held.first().map_or(0, Vec::len) as i64;
if last + 1 > held_end {
let want = ((last + 1 - held_end) as usize).min(MAX_RESAMPLE_SOURCE_SPAN);
let got = pull(held_end, want);
if r.held.is_empty() {
r.held = vec![Vec::new(); got.len().max(1)];
}
for (c, h) in r.held.iter_mut().enumerate() {
let src = got.get(c).map(Vec::as_slice).unwrap_or_default();
h.extend((0..want).map(|i| src.get(i).copied().unwrap_or(0.0)));
}
}
let mut dst = vec![vec![0.0f32; n]; r.held.len()];
for (i, frame) in (out..).take(n).enumerate() {
let (s, frac) = index(frame);
let k = usize::try_from(s - r.held_start).unwrap_or(0);
for (d, h) in dst.iter_mut().zip(&r.held) {
let a = k.checked_sub(1).and_then(|j| h.get(j)).copied().unwrap_or(0.0);
let b = h.get(k).copied().unwrap_or(0.0);
if let Some(x) = d.get_mut(i) {
*x = a + (b - a) * frac;
}
}
}
let (next_first, _) = index(out.saturating_add(n as i64));
let drop = (next_first - 1 - r.held_start).clamp(0, r.held.first().map_or(0, Vec::len) as i64) as usize;
for h in &mut r.held {
h.drain(..drop);
}
r.held_start += drop as i64;
r.next_out = Some(out.saturating_add(n as i64));
dst
}
#[test]
fn stepped_source_positions_are_bit_identical_to_per_sample_division() {
let bits = |b: &[Vec<f32>]| b.iter().map(|c| c.iter().map(|v| v.to_bits()).collect::<Vec<_>>()).collect::<Vec<_>>();
for (from, to) in
[(48_000, 44_100), (44_100, 48_000), (96_000, 48_000), (48_000, 48_000), (48_000, 96_000), (44_100, 22_050), (7, 384_000), (176_400, 1_001)]
{
let (mut new, mut old) = (StreamResampler::new(from, to), StreamResampler::new(from, to));
let mut k = 0;
for start in [0i64, -5_000, 1_000_000_007, -3, i64::MAX / 2_000_000] {
let mut pos = start;
for _ in 0..12 {
let n = [1, 7, 333, 512, 64][k % 5];
k += 1;
let (mut pa, mut pb) = (Vec::new(), Vec::new());
let a = new.process(pos, n, sine(440.0, from, &mut pa));
let b = reference_process(&mut old, pos, n, sine(440.0, from, &mut pb));
assert_eq!(bits(&a), bits(&b), "{from}→{to} at {pos} + {n}");
assert_eq!(pa, pb);
pos += n as i64;
}
}
}
}
#[test]
fn a_tone_converts_to_the_same_tone() {
// 1 kHz at 48 kHz played on a 44.1 kHz device: within linear interpolation's error of the
crates/automation/src/server/tests.rs
+28 -10
View File
@@ -277,23 +277,41 @@ async fn export_progress_and_cancel() {
}
}
c.send(r#"{"jsonrpc":"2.0","method":"notifications/cancelled","params":{"requestId":4}}"#).await;
let t0 = std::time::Instant::now();
while dir.join("b.mp4").exists() || t0.elapsed() < std::time::Duration::from_millis(500) {
// The export stops at its next frame, which a slow machine (software rendering in a CI
// container) can be well into: wait until the job has finished, not for a fixed time.
let (t0, mut id) = (std::time::Instant::now(), 100u64);
let last = loop {
id += 1;
c.send(&json!({"jsonrpc":"2.0","id":id,"method":"tools/call","params":{"name":"command_run","arguments":{"id":"jobs.list"}}}).to_string()).await;
let m = loop {
let m = c.next().await;
assert_ne!(m["id"], 4, "no response for a cancelled request: {m}");
if m["id"] == id {
break m;
}
};
let jobs: Value = serde_json::from_str(m["result"]["content"][0]["text"].as_str().unwrap()).unwrap();
let last = jobs.as_array().and_then(|a| a.last()).cloned().unwrap_or_default();
if last["finished"] == true {
break last;
}
assert!(t0.elapsed().as_secs() < 30, "the cancelled export is still running: {jobs}");
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
};
assert_eq!(last["result"]["error"], "cancelled", "{last}");
while dir.join("b.mp4").exists() {
assert!(t0.elapsed().as_secs() < 30, "partial file still there");
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
}
c.send(r#"{"jsonrpc":"2.0","id":5,"method":"tools/call","params":{"name":"command_run","arguments":{"id":"jobs.list"}}}"#).await;
let m = loop {
// and still no response for the cancelled request
c.send(r#"{"jsonrpc":"2.0","id":5,"method":"ping"}"#).await;
loop {
let m = c.next().await;
assert_ne!(m["id"], 4, "no response for a cancelled request: {m}");
if m["id"] == 5 {
break m;
break;
}
};
let jobs: Value = serde_json::from_str(m["result"]["content"][0]["text"].as_str().unwrap()).unwrap();
let last = jobs.as_array().and_then(|a| a.last()).cloned().unwrap_or_default();
assert_eq!(last["result"]["error"], "cancelled", "{jobs}");
assert!(!dir.join("b.mp4").exists());
}
let _ = std::fs::remove_dir_all(&dir);
}
crates/av1/README.md
+7 -1
View File
@@ -25,7 +25,7 @@ reference) is used only as an external test oracle.
| Tiles and tile groups (uniform and explicit spacing, tile size bytes) | 5.11 | done (tiles decoded in parallel) |
| Intra frame mode info: partitions, skip, segment id, CDEF index, delta q / lf, y / uv modes, angle deltas, CfL alphas, palette (colors, cache, color index map), filter intra, tx size (incl. var-tx syntax) | 5.11 | done, bit-exact |
| Coefficients (all_zero, eob, base / br levels, Golomb, dc sign, tx type sets, scans) | 5.11.39 | done, bit-exact |
| Dequantisation incl. quantiser matrices | 7.12 | done (qmatrix not yet oracle-tested) |
| Dequantisation incl. quantiser matrices | 7.12 | done |
| Inverse transforms: DCT 4–64, ADST 4/8/16, flip ADST, identity 4–32, WHT (lossless), rectangular | 7.13 | done, bit-exact |
| Intra prediction: DC, V/H + directional with edge filter and upsampling, smooth (3), Paeth, recursive filter intra, CfL, palette | 7.11.2, 7.11.4, 7.11.5 | done, bit-exact |
| Inter mode info: segment id prediction, skip mode, reference frames (single / compound, uni / bi), modes, DRL, MV coding, inter-intra, motion mode, compound type, interpolation filters | 5.11.18–5.11.32 | done, bit-exact |
@@ -43,6 +43,12 @@ reference) is used only as an external test oracle.
## Accuracy
OBU and tile length regressions (`decoder::tests::hostile_lengths_*`, #2801) use synthetic
inputs, including truncations and bit flips, without ffmpeg. Oversized LEB128 OBU lengths and
four-byte tile lengths return `Error::Truncated` instead of wrapping or truncating on 32-bit
targets. Valid partial tile groups retain their payloads. See
[32-bit parser tests](../../docs/testing.md#32-bit-parser-regressions).
`tests/it/oracle_intra.rs` encodes all-intra streams with ffmpeg's libsvtav1 (with and without the
in-loop filters; 64×64 and 128×128 superblocks; intra edge filter on and off) and compares every
decoded frame with libdav1d: **bit-exact** on all 13 fixtures (8- and 10-bit 4:2:0; 128×128 to
crates/av1/src/bits.rs
+28 -1
View File
@@ -120,5 +120,32 @@ pub(crate) fn leb128(data: &[u8]) -> Result<(u64, usize)> {
return Ok((v, i + 1));
}
}
Ok((v, 8))
Err(Error::Truncated)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn hostile_lengths_leb128_requires_termination() {
for byte in [0x80, 0xff] {
let mut data = [byte; 9];
data[8] = 0; // A ninth byte cannot terminate an eight-byte AV1 LEB128.
for len in 0..=data.len() {
let result = std::panic::catch_unwind(|| leb128(&data[..len]));
assert!(matches!(result.expect("LEB128 parsing must not panic"), Err(Error::Truncated)));
}
}
}
#[test]
fn hostile_lengths_leb128_preserves_terminated_values() {
assert_eq!(leb128(&[0]).unwrap(), (0, 1));
for len in 1..=8 {
let mut data = [0xff; 9];
data[len - 1] = 0x7f;
assert_eq!(leb128(&data).unwrap(), ((1u64 << (7 * len)) - 1, len));
}
}
}
crates/av1/src/decoder.rs
+208 -33
View File
@@ -93,6 +93,17 @@ impl<T: Clone> Slot<T> {
g = self.cv.wait(g).unwrap_or_else(|e| e.into_inner());
}
}
/// [`Slot::wait`] for the slot's only consumer: moves the value out instead of cloning it.
fn take(&self) -> Result<T> {
let mut g = self.value.lock().unwrap_or_else(|e| e.into_inner());
loop {
if let Some(v) = g.take() {
return v;
}
g = self.cv.wait(g).unwrap_or_else(|e| e.into_inner());
}
}
}
type FrameSlot = Slot<Arc<RefFrame>>;
@@ -110,16 +121,18 @@ struct FrameJob {
tile_covered: Vec<bool>,
slot: Arc<FrameSlot>,
/// The output picture, when the frame is shown.
picture: Option<Arc<Slot<Arc<Picture>>>>,
picture: Option<Arc<Slot<Picture>>>,
pts: i64,
apply_film_grain: bool,
/// Draft mode: the frame refreshes no reference slot, so its in-loop filters are skipped.
draft: bool,
#[cfg(test)]
force_panic: bool,
}
/// A shown picture in output order.
enum Output {
Decoded(Arc<Slot<Arc<Picture>>>),
Decoded(Arc<Slot<Picture>>),
/// show_existing_frame: built from the reference slot when it is finished.
Existing(Box<ExistingOutput>),
}
@@ -145,7 +158,7 @@ impl Output {
fn take(self) -> Result<Picture> {
match self {
Output::Decoded(s) => s.wait().map(|p| Arc::try_unwrap(p).unwrap_or_else(|p| (*p).clone())),
Output::Decoded(s) => s.take(),
Output::Existing(e) => {
let ExistingOutput { slot, seq, width, height, grain, apply_film_grain, pts } = *e;
let rf = slot.wait()?;
@@ -229,6 +242,9 @@ pub struct Decoder {
out: VecDeque<Output>,
/// pts of the temporal unit being decoded.
pts: i64,
/// Make every frame job panic (tests of the worker panic handling).
#[cfg(test)]
force_panic: bool,
}
fn default_threads() -> usize {
@@ -289,6 +305,8 @@ impl Decoder {
max_in_flight: fthreads + 2,
out: VecDeque::new(),
pts: 0,
#[cfg(test)]
force_panic: false,
}
}
@@ -418,7 +436,7 @@ impl Decoder {
let size = if has_size {
let (v, n) = leb128(data.get(p..).ok_or(Error::Truncated)?)?;
p += n;
v as usize
usize::try_from(v).map_err(|_| Error::Truncated)?
} else {
data.len().checked_sub(p).ok_or(Error::Truncated)?
};
@@ -552,6 +570,8 @@ impl Decoder {
pts: self.pts,
apply_film_grain: self.apply_film_grain,
draft,
#[cfg(test)]
force_panic: self.force_panic,
}));
Ok(None)
}
@@ -581,12 +601,14 @@ impl Decoder {
let size = if tile_num == tg_end {
data.len().checked_sub(pos).ok_or(Error::Truncated)?
} else {
let prefix_end = pos.checked_add(tsb as usize).ok_or(Error::Truncated)?;
let prefix = data.get(pos..prefix_end).ok_or(Error::Truncated)?;
let mut v = 0usize;
for i in 0..tsb as usize {
v |= (*data.get(pos + i).ok_or(Error::Truncated)? as usize) << (8 * i);
for (i, &byte) in prefix.iter().enumerate() {
v |= usize::from(byte) << (8 * i);
}
pos += tsb as usize;
v + 1
pos = prefix_end;
v.checked_add(1).ok_or(Error::Truncated)?
};
let end = pos.checked_add(size).filter(|&e| e <= data.len()).ok_or(Error::Truncated)?;
if job.tile_covered.get(tile_num).copied() != Some(false) {
@@ -658,13 +680,16 @@ impl Decoder {
}
}
/// Decode a frame job and publish its results.
/// Decode a frame job and publish its results. A panic while decoding (a decoder bug hit by
/// hostile input) becomes an error in both slots: otherwise a frame worker would die with its
/// slots unset and every later `wait` on them would block forever.
fn run_job(job: FrameJob, pool: &Pool, stats: &Mutex<DecodeStats>) {
let mut st = DecodeStats::default();
match decode_frame(&job, pool, &mut st) {
let res = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| decode_frame(&job, pool, &mut st)));
match res.unwrap_or(Err(Error::Invalid("decoder panicked"))) {
Ok((rf, pic)) => {
if let Some(ps) = &job.picture {
ps.set(pic.map(Arc::new).ok_or(Error::Invalid("shown frame without picture")));
ps.set(pic.ok_or(Error::Invalid("shown frame without picture")));
}
job.slot.set(Ok(rf));
}
@@ -683,6 +708,8 @@ fn run_job(job: FrameJob, pool: &Pool, stats: &Mutex<DecodeStats>) {
/// The whole decoding process of one frame: setup, tiles, post-filters, motion vector storage
/// and the output picture.
fn decode_frame(job: &FrameJob, pool: &Pool, stats: &mut DecodeStats) -> Result<(Arc<RefFrame>, Option<Picture>)> {
#[cfg(test)]
assert!(!job.force_panic, "forced frame job panic");
let fh = &job.fh;
let seq = &job.seq;
// Wait for the reference frames this frame reads.
@@ -708,13 +735,14 @@ fn decode_frame(job: &FrameJob, pool: &Pool, stats: &mut DecodeStats) -> Result<
fs.ref_info = job.ref_info.clone();
if fh.primary_ref_frame != PRIMARY_REF_NONE {
// load_previous_segment_ids( )
if let Some(rf) = &fs.refs[fh.ref_frame_idx[fh.primary_ref_frame]]
let (mi_cols, mi_rows) = (fs.mi.cols, fs.mi.rows);
let sh = &mut fs.sh;
if let Some(rf) = &sh.refs[fh.ref_frame_idx[fh.primary_ref_frame]]
&& fh.seg.enabled
&& rf.mi_cols == fs.mi.cols
&& rf.mi_rows == fs.mi.rows
&& rf.mi_cols == mi_cols
&& rf.mi_rows == mi_rows
{
let ids = rf.segment_ids.clone();
fs.prev_segment_ids.copy_from_slice(&ids);
sh.prev_segment_ids.copy_from_slice(&rf.segment_ids);
}
}
if fh.use_ref_frame_mvs {
@@ -892,23 +920,21 @@ fn motion_field_estimation(fs: &mut FrameState) {
for r in LAST_FRAME..=ALTREF_FRAME {
fs.motion_field[r] = vec![[crate::mvpred::INVALID_MV, crate::mvpred::INVALID_MV]; w8 * h8];
}
let fh = fs.fh.clone();
let seq = fs.seq.clone();
let rd = |a: u32, b: u32| crate::header::relative_dist(&seq, a, b);
let last_idx = fh.ref_frame_idx[0];
let cur_gold = fh.order_hints[GOLDEN_FRAME];
let last_alt = fs.ref_info[last_idx].saved_order_hints[ALTREF_FRAME];
if last_alt != cur_gold {
let (fh, seq) = (&fs.fh, &fs.seq);
let after_cur = |r: usize| crate::header::relative_dist(seq, fh.order_hints[r], fh.order_hint) > 0;
let (bwd, alt2, alt) = (after_cur(BWDREF_FRAME), after_cur(ALTREF2_FRAME), after_cur(ALTREF_FRAME));
let last_alt = fs.ref_info[fh.ref_frame_idx[0]].saved_order_hints[ALTREF_FRAME];
if last_alt != fh.order_hints[GOLDEN_FRAME] {
project(fs, LAST_FRAME, -1);
}
let mut ref_stamp = MFMV_STACK_SIZE as i32 - 2;
if rd(fh.order_hints[BWDREF_FRAME], fh.order_hint) > 0 && project(fs, BWDREF_FRAME, 1) {
if bwd && project(fs, BWDREF_FRAME, 1) {
ref_stamp -= 1;
}
if rd(fh.order_hints[ALTREF2_FRAME], fh.order_hint) > 0 && project(fs, ALTREF2_FRAME, 1) {
if alt2 && project(fs, ALTREF2_FRAME, 1) {
ref_stamp -= 1;
}
if rd(fh.order_hints[ALTREF_FRAME], fh.order_hint) > 0 && ref_stamp >= 0 && project(fs, ALTREF_FRAME, 1) {
if alt && ref_stamp >= 0 && project(fs, ALTREF_FRAME, 1) {
ref_stamp -= 1;
}
if ref_stamp >= 0 {
@@ -936,17 +962,16 @@ fn project_pos(v8: i32, delta: i32, dst_sign: i32, max8: i32, max_off8: i32) ->
/// Projection process (7.9.2).
fn project(fs: &mut FrameState, src: usize, dst_sign: i32) -> bool {
let fh = fs.fh.clone();
let seq = fs.seq.clone();
let FrameShared { seq, fh, refs, ref_info, motion_field, .. } = &mut fs.sh;
let src_idx = fh.ref_frame_idx[src - LAST_FRAME];
let w8 = fh.mi_cols as i32 >> 1;
let h8 = fh.mi_rows as i32 >> 1;
let ri = fs.ref_info[src_idx].clone();
let ri = &ref_info[src_idx];
if ri.mi_rows != fh.mi_rows || ri.mi_cols != fh.mi_cols || ri.frame_type == INTRA_ONLY_FRAME as u8 || ri.frame_type == KEY_FRAME as u8 {
return false;
}
let Some(rf) = fs.refs[src_idx].clone() else { return false };
let rd = |a: u32, b: u32| crate::header::relative_dist(&seq, a, b);
let Some(rf) = &refs[src_idx] else { return false };
let rd = |a: u32, b: u32| crate::header::relative_dist(seq, a, b);
let mi_cols = fh.mi_cols as usize;
for y8 in 0..h8 {
for x8 in 0..w8 {
@@ -966,7 +991,7 @@ fn project(fs: &mut FrameState, src: usize, dst_sign: i32) -> bool {
for dst in LAST_FRAME..=ALTREF_FRAME {
let ref_to_dst = rd(fh.order_hint, fh.order_hints[dst]);
let pm = get_mv_projection(mv, ref_to_dst, ref_offset);
fs.motion_field[dst][(py * w8 + px) as usize] = pm;
motion_field[dst][(py * w8 + px) as usize] = pm;
}
}
}
@@ -978,7 +1003,157 @@ fn project(fs: &mut FrameState, src: usize, dst_sign: i32) -> bool {
#[cfg(test)]
mod tests {
use super::frame_size_allowed;
use super::*;
use crate::header::TileInfo;
fn tile_group_decoder(tile_size_bytes: u32) -> Decoder {
let mut decoder = Decoder::with_threads(1);
decoder.pending = Some(Box::new(FrameJob {
seq: SequenceHeader::default(),
fh: FrameHeader { tile_info: TileInfo { cols: 3, rows: 1, cols_log2: 2, tile_size_bytes, ..Default::default() }, ..Default::default() },
refs: Default::default(),
ref_info: Default::default(),
tiles: Vec::new(),
tile_covered: vec![false; 3],
slot: Slot::new(),
picture: None,
pts: 0,
apply_film_grain: false,
draft: false,
#[cfg(test)]
force_panic: false,
}));
decoder
}
#[test]
fn hostile_lengths_tile_size_does_not_wrap() {
// #2801: tile_size_minus_1 + 1 overflowed usize on wasm32. A partial group
// (tiles 0..1 of three) isolates parsing without submitting synthetic tile data.
let result = std::panic::catch_unwind(|| tile_group_decoder(4).tile_group(&[0x88, 0xff, 0xff, 0xff, 0xff]));
assert!(matches!(result.expect("tile size parsing must not panic"), Err(Error::Truncated)));
}
#[test]
fn hostile_lengths_obu_size_does_not_truncate() {
// #2801: the LEB128 size 2^32 became zero on wasm32, accepting a truncated OBU.
let result = std::panic::catch_unwind(|| Decoder::with_threads(1).decode(&[0x7a, 0x80, 0x80, 0x80, 0x80, 0x10]));
assert!(matches!(result.expect("OBU size parsing must not panic"), Err(Error::Truncated)));
}
#[test]
fn hostile_lengths_obu_size_requires_termination() {
let mut data = [0x80; 9];
data[0] = 0x7a; // Padding OBU with eight unterminated size bytes.
let result = std::panic::catch_unwind(|| Decoder::with_threads(1).decode(&data));
assert!(matches!(result.expect("OBU size parsing must not panic"), Err(Error::Truncated)));
data[8] = 0; // A terminated eight-byte encoding of zero remains valid.
assert!(Decoder::with_threads(1).decode(&data).unwrap().is_empty());
}
#[test]
fn hostile_lengths_tile_mutations_and_truncations() {
for width in 1..=4 {
let mut data = vec![0x88]; // tiles 0..1 of three
data.extend_from_slice(&0u32.to_le_bytes()[..width]); // first tile has one byte
data.extend([0xab, 0xcd]);
let mut decoder = tile_group_decoder(width as u32);
assert!(decoder.tile_group(&data).unwrap().is_none());
assert_eq!(decoder.pending.as_ref().unwrap().tiles, vec![(0, vec![0xab]), (1, vec![0xcd])]);
for end in 0..=data.len() {
std::panic::catch_unwind(|| tile_group_decoder(width as u32).tile_group(&data[..end])).expect("truncated tile group must not panic").ok();
}
// Keep the group header unchanged so mutations cannot submit a decoding job.
for index in 1..=width {
for bit in 0..8 {
let mut mutated = data.clone();
mutated[index] ^= 1 << bit;
std::panic::catch_unwind(|| tile_group_decoder(width as u32).tile_group(&mutated)).expect("mutated tile length must not panic").ok();
}
}
}
}
#[test]
fn hostile_lengths_obu_mutations_and_truncations() {
// Padding OBUs are ignored, but their declared size must still fit the input.
let data = [0x7a, 3, 0, 0, 0, 0x12, 0]; // padding, then a temporal delimiter
assert!(Decoder::with_threads(1).decode(&data).unwrap().is_empty());
for end in 0..=data.len() {
std::panic::catch_unwind(|| Decoder::with_threads(1).decode(&data[..end])).expect("truncated OBU must not panic").ok();
}
for size in [u32::MAX as u64, 1u64 << 32, (1u64 << 32) + 1, (1u64 << 56) - 1] {
let mut data = vec![0x7a];
let mut remaining = size;
loop {
let byte = (remaining & 0x7f) as u8;
remaining >>= 7;
data.push(byte | if remaining == 0 { 0 } else { 0x80 });
if remaining == 0 {
break;
}
}
data.push(0); // the low 32 bits of 2^32 + 1 would fit this payload
let result = std::panic::catch_unwind(|| Decoder::with_threads(1).decode(&data));
assert!(matches!(result.expect("oversized OBU must not panic"), Err(Error::Truncated)));
for index in 1..data.len() - 1 {
for bit in 0..8 {
let mut mutated = data.clone();
mutated[index] ^= 1 << bit;
std::panic::catch_unwind(|| Decoder::with_threads(1).decode(&mutated)).expect("mutated OBU length must not panic").ok();
}
}
}
}
/// A 16x16 libaom key frame: temporal delimiter, sequence header, frame OBU.
const KEY_FRAME_16X16: &[u8] = &[
0x12, 0x00, 0x0a, 0x0a, 0x00, 0x00, 0x00, 0x01, 0x9f, 0xf9, 0xf5, 0xf3, 0x00, 0x80, 0x32, 0x0b, 0x10, 0x00, 0xc0, 0x00, 0x00, 0x02, 0x80, 0x00, 0x00,
0x00, 0x9c,
];
fn decode_key_frame(threads: usize, force_panic: bool) -> crate::Result<usize> {
let mut dec = Decoder::with_threads(threads);
dec.force_panic = force_panic;
let mut n = dec.decode(KEY_FRAME_16X16)?.len();
n += dec.flush_result()?.len();
Ok(n)
}
/// `f()` on another thread; a hang fails the test instead of blocking it.
fn within_a_minute<T: Send + 'static>(f: impl FnOnce() -> T + Send + 'static) -> T {
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let _ = tx.send(f());
});
rx.recv_timeout(std::time::Duration::from_secs(60)).expect("decoder hung after a frame job panicked")
}
#[test]
fn a_panicking_frame_job_is_an_error_not_a_hang() {
for threads in [1, 3] {
assert_eq!(decode_key_frame(threads, false), Ok(1), "threads {threads}");
assert_eq!(within_a_minute(move || decode_key_frame(threads, true)), Err(Error::Invalid("decoder panicked")), "threads {threads}");
}
}
#[test]
fn the_decoder_recovers_at_the_next_key_frame_after_a_panic() {
let n = within_a_minute(|| {
let mut dec = Decoder::with_threads(3);
dec.force_panic = true;
// The synchronous build reports the error from decode(); the threaded
// build may report it from decode() or from flush_result().
let first = dec.decode(KEY_FRAME_16X16).map(|_| ());
let second = dec.flush_result().map(|_| ());
assert!(first.is_err() || second.is_err(), "the panicking job reported no error");
dec.force_panic = false;
let mut pics = dec.decode(KEY_FRAME_16X16).expect("decode");
pics.extend(dec.flush_result().expect("flush"));
pics.len()
});
assert_eq!(n, 1);
}
#[test]
fn frames_up_to_the_level_6_3_limits_are_allowed() {
crates/av1/src/frame.rs
+12 -6
View File
@@ -12,8 +12,8 @@ const POOL_MIN_LEN: usize = 1 << 16;
/// Samples kept at most (96 M samples = 192 MB, a few 4K frames).
const POOL_MAX_SAMPLES: usize = 96 << 20;
/// A zeroed buffer of `len` samples (recycled memory for large ones).
fn zeroed(len: usize) -> Vec<u16> {
/// An empty buffer with room for `len` samples (recycled memory for large ones).
fn recycled(len: usize) -> Vec<u16> {
if len >= POOL_MIN_LEN {
let found = {
let mut pool = PLANE_POOL.lock().unwrap_or_else(|e| e.into_inner());
@@ -21,11 +21,17 @@ fn zeroed(len: usize) -> Vec<u16> {
};
if let Some(mut v) = found {
v.clear();
v.resize(len, 0);
return v;
}
}
vec![0; len]
Vec::with_capacity(len)
}
/// A zeroed buffer of `len` samples (recycled memory for large ones).
fn zeroed(len: usize) -> Vec<u16> {
let mut v = recycled(len);
v.resize(len, 0);
v
}
/// One plane of samples (u16 at every bit depth), or a rectangular region of one.
@@ -42,8 +48,8 @@ pub struct Plane {
impl Clone for Plane {
fn clone(&self) -> Plane {
let mut data = zeroed(self.data.len());
data.copy_from_slice(&self.data);
let mut data = recycled(self.data.len());
data.extend_from_slice(&self.data);
Plane { data, stride: self.stride, rows: self.rows, ox: self.ox, oy: self.oy }
}
}
crates/av1/src/header.rs
+5 -12
View File
@@ -424,16 +424,9 @@ pub struct FrameHeader {
pub gm_params: [[i32; 6]; 8],
pub prev_gm_params: [[i32; 6]; 8],
pub film_grain: FilmGrainParams,
/// Temporal / spatial id of the OBU carrying the header.
pub temporal_id: u32,
pub spatial_id: u32,
}
impl FrameHeader {
pub fn get_relative_dist(&self, seq: &SequenceHeader, a: u32, b: u32) -> i32 {
relative_dist(seq, a, b)
}
/// get_qindex( ignoreDeltaQ, segmentId ) with CurrentQIndex.
pub fn get_qindex(&self, ignore_delta_q: bool, segment_id: usize, current_q_index: i32) -> i32 {
let base = self.quant.base_q_idx as i32;
@@ -487,7 +480,7 @@ impl FrameHeader {
/// uncompressed_header( ). `st` holds the cross-frame state.
pub(crate) fn parse(r: &mut BitReader, st: &mut HeaderState, temporal_id: u32, spatial_id: u32) -> Result<FrameHeader> {
let seq = st.seq;
let mut h = FrameHeader { temporal_id, spatial_id, ..Default::default() };
let mut h = FrameHeader::default();
let id_len = if seq.frame_id_numbers_present { seq.additional_frame_id_length_minus_1 + seq.delta_frame_id_length_minus_2 + 3 } else { 0 };
if seq.reduced_still_picture_header {
h.frame_type = KEY_FRAME as u8;
@@ -514,7 +507,7 @@ impl FrameHeader {
};
h.order_hint = r.f(seq.order_hint_bits)?;
h.primary_ref_frame = if h.frame_is_intra || h.error_resilient_mode { PRIMARY_REF_NONE } else { r.f(3)? as usize };
h.skip_buffer_removal_times(r, seq)?;
h.skip_buffer_removal_times(r, seq, temporal_id, spatial_id)?;
h.refresh_frame_flags = if h.frame_type == SWITCH_FRAME as u8 || (h.frame_type == KEY_FRAME as u8 && h.show_frame) { ALL_FRAMES } else { r.f(8)? };
h.read_ref_order_hints(r, st)?;
if h.frame_is_intra {
@@ -641,13 +634,13 @@ impl FrameHeader {
}
/// Reads and discards the buffer_removal_time of every operating point that applies to this layer.
fn skip_buffer_removal_times(&self, r: &mut BitReader, seq: &SequenceHeader) -> Result<()> {
fn skip_buffer_removal_times(&self, r: &mut BitReader, seq: &SequenceHeader, temporal_id: u32, spatial_id: u32) -> Result<()> {
if seq.decoder_model_info_present && r.flag()? {
for op in 0..=seq.operating_points_cnt_minus_1 {
if seq.decoder_model_present_for_this_op[op] {
let idc = seq.operating_point_idc[op];
let in_t = (idc >> self.temporal_id) & 1;
let in_s = (idc >> (self.spatial_id + 8)) & 1;
let in_t = (idc >> temporal_id) & 1;
let in_s = (idc >> (spatial_id + 8)) & 1;
if idc == 0 || (in_t == 1 && in_s == 1) {
r.f(seq.buffer_removal_time_length_minus_1 + 1)?;
}
crates/av1/src/lib.rs
+2 -2
View File
@@ -13,7 +13,7 @@ mod cdf;
mod decoder;
mod frame;
mod grain;
pub mod header;
mod header;
mod inter;
mod inter_info;
mod intra;
@@ -29,7 +29,7 @@ mod tile;
mod transform;
pub use decoder::Decoder;
pub use header::{ColorConfig, FrameHeader, SequenceHeader};
pub use header::{ColorConfig, SequenceHeader};
pub use stats::{DecodeStats, Stage};
/// Decoder errors.
crates/av1/src/tile.rs
+2 -1
View File
@@ -113,7 +113,8 @@ pub(crate) struct TileDecoder<'a, 'f> {
pub b: Block,
pub mvs: crate::mvpred::MvStack,
pub nb: crate::inter_info::Neighbours,
/// Quant[] for the current transform block (raster order, stride per Adjusted size).
/// Quant[] for the current transform block (raster order, stride per Adjusted size); all
/// zero between transform blocks (`reconstruct` clears what it reads).
quant: Vec<i32>,
dequant: Vec<i32>,
residual: Vec<i32>,
crates/av1/src/tile/coeffs.rs
+9 -10
View File
@@ -174,8 +174,6 @@ impl<'a, 'f> TileDecoder<'a, 'f> {
pub(super) fn coeffs(&mut self, plane: usize, start_x: usize, start_y: usize, tx_sz: usize) -> usize {
let pos = TxbPos { x4: start_x >> 2, y4: start_y >> 2, w4: TX_WIDTH[tx_sz] as usize >> 2, h4: TX_HEIGHT[tx_sz] as usize >> 2 };
let tx_sz_ctx = (TX_SIZE_SQR[tx_sz] as usize + TX_SIZE_SQR_UP[tx_sz] as usize + 1) >> 1;
let seg_eob = if tx_sz == TX_16X64 || tx_sz == TX_64X16 { 512 } else { 1024.min(TX_WIDTH[tx_sz] as usize * TX_HEIGHT[tx_sz] as usize) };
self.quant[..seg_eob].iter_mut().for_each(|q| *q = 0);
let mut eob = 0usize;
let mut cul_level = 0u32;
let mut dc_category = 0u8;
@@ -457,11 +455,12 @@ impl<'a, 'f> TileDecoder<'a, 'f> {
}
/// Reconstruct process (7.12.3).
pub(super) fn reconstruct(&mut self, plane: usize, x: usize, y: usize, tx_sz: usize) {
let dq_denom: i64 = match tx_sz {
TX_32X32 | TX_16X32 | TX_32X16 | TX_16X64 | TX_64X16 => 2,
TX_64X64 | TX_32X64 | TX_64X32 => 4,
_ => 1,
pub(super) fn reconstruct(&mut self, plane: usize, x: usize, y: usize, tx_sz: usize, dc_only: bool) {
// dqDenom (7.12.3) is 1, 2 or 4 and the dividend is non-negative.
let dq_shift = match tx_sz {
TX_32X32 | TX_16X32 | TX_32X16 | TX_16X64 | TX_64X16 => 1,
TX_64X64 | TX_32X64 | TX_64X32 => 2,
_ => 0,
};
let log2w = TX_WIDTH_LOG2[tx_sz] as u32;
let log2h = TX_HEIGHT_LOG2[tx_sz] as u32;
@@ -492,7 +491,7 @@ impl<'a, 'f> TileDecoder<'a, 'f> {
}
for i in 0..th {
for j in 0..tw {
let qv = self.quant[i * tw + j];
let qv = std::mem::take(&mut self.quant[i * tw + j]);
if qv == 0 {
continue;
}
@@ -504,13 +503,13 @@ impl<'a, 'f> TileDecoder<'a, 'f> {
q
};
let dq = qv as i64 * q2;
let mag = (dq.abs() & 0xFFFFFF) / dq_denom;
let mag = (dq.abs() & 0xFFFFFF) >> dq_shift;
let dq2 = if dq < 0 { -mag } else { mag };
self.dequant[i * 64 + j] = dq2.clamp(-lim, lim - 1) as i32;
}
}
let lossless = self.b.lossless;
inverse_transform_2d(&self.dequant, &mut self.residual, tx_sz, t, lossless, bd);
inverse_transform_2d(&self.dequant, &mut self.residual, tx_sz, t, lossless, bd, dc_only);
let pl = &mut self.t.cur.planes[plane];
let max = (1i32 << bd) - 1;
for i in 0..h {
crates/av1/src/tile/predict.rs
+1 -1
View File
@@ -262,7 +262,7 @@ impl<'a, 'f> TileDecoder<'a, 'f> {
if !self.b.skip {
let eob = self.coeffs(plane, start_x, start_y, tx_sz);
if eob > 0 {
self.reconstruct(plane, start_x, start_y, tx_sz);
self.reconstruct(plane, start_x, start_y, tx_sz, eob == 1);
}
}
for i in 0..step_y {
crates/av1/src/transform.rs
+141 -53
View File
@@ -15,15 +15,25 @@ pub(crate) fn round2(x: i32, n: u32) -> i32 {
if n == 0 { x } else { (x + (1 << (n - 1))) >> n }
}
/// cos128( angle ) for every `angle & 255`.
static COS128: [i32; 256] = {
let mut t = [0; 256];
let mut a = 0;
while a < 256 {
t[a] = match a {
0..=64 => COS128_LOOKUP[a] as i32,
65..=128 => -(COS128_LOOKUP[128 - a] as i32),
129..=192 => -(COS128_LOOKUP[a - 128] as i32),
_ => COS128_LOOKUP[256 - a] as i32,
};
a += 1;
}
t
};
#[inline(always)]
fn cos128(angle: i32) -> i64 {
let a = (angle & 255) as usize;
(match a {
0..=64 => COS128_LOOKUP[a] as i32,
65..=128 => -(COS128_LOOKUP[128 - a] as i32),
129..=192 => -(COS128_LOOKUP[a - 128] as i32),
_ => COS128_LOOKUP[256 - a] as i32,
}) as i64
COS128[(angle & 255) as usize] as i64
}
#[inline(always)]
@@ -125,15 +135,32 @@ impl<const L: usize> Lane for [i32; L] {
}
}
fn brev(num_bits: u32, x: usize) -> usize {
const fn brev(num_bits: u32, x: usize) -> usize {
let mut t = 0;
for i in 0..num_bits {
let mut i = 0;
while i < num_bits {
let bit = (x >> i) & 1;
t += bit << (num_bits - 1 - i);
i += 1;
}
t
}
/// BREV[n][x] = brev( n, x ) for n <= 6.
const BREV: [[u8; 64]; 7] = {
let mut t = [[0; 64]; 7];
let mut n = 0;
while n < 7 {
let mut x = 0;
while x < 1 << n {
t[n][x] = brev(n as u32, x) as u8;
x += 1;
}
n += 1;
}
t
};
/// Butterfly rotation B( a, b, angle, flip ).
#[inline(always)]
fn bf<T: Lane>(t: &mut [T], a: usize, b: usize, angle: i32, flip: bool) {
@@ -158,57 +185,57 @@ fn hd<T: Lane>(t: &mut [T], a: usize, b: usize, flip: bool, r: u32) {
/// Rotations and Hadamards that build the odd half of the larger transforms (first part of 7.13.2.3).
#[inline(always)]
fn dct_stage_outer_rotations<T: Lane>(t: &mut [T], n: u32, r: u32) {
if n == 6 {
fn dct_stage_outer_rotations<T: Lane, const N: u32>(t: &mut [T], r: u32) {
if N == 6 {
for i in 0..16 {
bf(t, 32 + i, 63 - i, 63 - 4 * brev(4, i) as i32, false);
bf(t, 32 + i, 63 - i, 63 - 4 * BREV[4][i] as i32, false);
}
}
if n >= 5 {
if N >= 5 {
for i in 0..8 {
bf(t, 16 + i, 31 - i, 6 + ((brev(3, 7 - i) as i32) << 3), false);
bf(t, 16 + i, 31 - i, 6 + ((BREV[3][7 - i] as i32) << 3), false);
}
}
if n == 6 {
if N == 6 {
for i in 0..16 {
hd(t, 32 + i * 2, 33 + i * 2, i & 1 == 1, r);
}
}
if n >= 4 {
if N >= 4 {
for i in 0..4 {
bf(t, 8 + i, 15 - i, 12 + ((brev(2, 3 - i) as i32) << 4), false);
bf(t, 8 + i, 15 - i, 12 + ((BREV[2][3 - i] as i32) << 4), false);
}
}
if n >= 5 {
if N >= 5 {
for i in 0..8 {
hd(t, 16 + 2 * i, 17 + 2 * i, i & 1 == 1, r);
}
}
if n == 6 {
if N == 6 {
for i in 0..4 {
for j in 0..2 {
bf(t, 62 - i * 4 - j, 33 + i * 4 + j, 60 - 16 * brev(2, i) as i32 + 64 * j as i32, true);
bf(t, 62 - i * 4 - j, 33 + i * 4 + j, 60 - 16 * BREV[2][i] as i32 + 64 * j as i32, true);
}
}
}
if n >= 3 {
if N >= 3 {
for i in 0..2 {
bf(t, 4 + i, 7 - i, 56 - 32 * i as i32, false);
}
}
if n >= 4 {
if N >= 4 {
for i in 0..4 {
hd(t, 8 + 2 * i, 9 + 2 * i, i & 1 == 1, r);
}
}
if n >= 5 {
if N >= 5 {
for i in 0..2 {
for j in 0..2 {
bf(t, 30 - 4 * i - j, 17 + 4 * i + j, 24 + ((j as i32) << 6) + (((1 - i) as i32) << 5), true);
}
}
}
if n == 6 {
if N == 6 {
for i in 0..8 {
for j in 0..2 {
hd(t, 32 + i * 4 + j, 35 + i * 4 - j, i & 1 == 1, r);
@@ -219,28 +246,28 @@ fn dct_stage_outer_rotations<T: Lane>(t: &mut [T], n: u32, r: u32) {
/// Butterflies of the smallest sub-transforms (second part of 7.13.2.3).
#[inline(always)]
fn dct_stage_even_half<T: Lane>(t: &mut [T], n: u32, r: u32) {
fn dct_stage_even_half<T: Lane, const N: u32>(t: &mut [T], r: u32) {
for i in 0..2 {
bf(t, 2 * i, 2 * i + 1, 32 + 16 * i as i32, i == 0);
}
if n >= 3 {
if N >= 3 {
for i in 0..2 {
hd(t, 4 + 2 * i, 5 + 2 * i, i == 1, r);
}
}
if n >= 4 {
if N >= 4 {
for i in 0..2 {
bf(t, 14 - i, 9 + i, 48 + 64 * i as i32, true);
}
}
if n >= 5 {
if N >= 5 {
for i in 0..4 {
for j in 0..2 {
hd(t, 16 + 4 * i + j, 19 + 4 * i - j, i & 1 == 1, r);
}
}
}
if n == 6 {
if N == 6 {
for i in 0..2 {
for j in 0..4 {
bf(t, 61 - i * 8 - j, 34 + i * 8 + j, 56 - i as i32 * 32 + (j as i32 >> 1) * 64, true);
@@ -251,50 +278,50 @@ fn dct_stage_even_half<T: Lane>(t: &mut [T], n: u32, r: u32) {
/// Merges the quarter-size sub-transforms (third part of 7.13.2.3).
#[inline(always)]
fn dct_stage_merge_quarters<T: Lane>(t: &mut [T], n: u32, r: u32) {
fn dct_stage_merge_quarters<T: Lane, const N: u32>(t: &mut [T], r: u32) {
for i in 0..2 {
hd(t, i, 3 - i, false, r);
}
if n >= 3 {
if N >= 3 {
bf(t, 6, 5, 32, true);
}
if n >= 4 {
if N >= 4 {
for i in 0..2 {
for j in 0..2 {
hd(t, 8 + 4 * i + j, 11 + 4 * i - j, i == 1, r);
}
}
}
if n >= 5 {
if N >= 5 {
for i in 0..4 {
bf(t, 29 - i, 18 + i, 48 + (i as i32 >> 1) * 64, true);
}
}
if n == 6 {
if N == 6 {
for i in 0..4 {
for j in 0..4 {
hd(t, 32 + 8 * i + j, 39 + 8 * i - j, i & 1 == 1, r);
}
}
}
if n >= 3 {
if N >= 3 {
for i in 0..4 {
hd(t, i, 7 - i, false, r);
}
}
if n >= 4 {
if N >= 4 {
for i in 0..2 {
bf(t, 13 - i, 10 + i, 32, true);
}
}
if n >= 5 {
if N >= 5 {
for i in 0..2 {
for j in 0..4 {
hd(t, 16 + i * 8 + j, 23 + i * 8 - j, i == 1, r);
}
}
}
if n == 6 {
if N == 6 {
for i in 0..8 {
bf(t, 59 - i, 36 + i, if i < 4 { 48 } else { 112 }, true);
}
@@ -303,34 +330,34 @@ fn dct_stage_merge_quarters<T: Lane>(t: &mut [T], n: u32, r: u32) {
/// Final merges of the half-size sub-transforms (last part of 7.13.2.3).
#[inline(always)]
fn dct_stage_merge_halves<T: Lane>(t: &mut [T], n: u32, r: u32) {
if n >= 4 {
fn dct_stage_merge_halves<T: Lane, const N: u32>(t: &mut [T], r: u32) {
if N >= 4 {
for i in 0..8 {
hd(t, i, 15 - i, false, r);
}
}
if n >= 5 {
if N >= 5 {
for i in 0..4 {
bf(t, 27 - i, 20 + i, 32, true);
}
}
if n == 6 {
if N == 6 {
for i in 0..8 {
hd(t, 32 + i, 47 - i, false, r);
hd(t, 48 + i, 63 - i, true, r);
}
}
if n >= 5 {
if N >= 5 {
for i in 0..16 {
hd(t, i, 31 - i, false, r);
}
}
if n == 6 {
if N == 6 {
for i in 0..8 {
bf(t, 55 - i, 40 + i, 32, true);
}
}
if n == 6 {
if N == 6 {
for i in 0..32 {
hd(t, i, 63 - i, false, r);
}
@@ -339,17 +366,27 @@ fn dct_stage_merge_halves<T: Lane>(t: &mut [T], n: u32, r: u32) {
/// Inverse DCT of length 2^n (7.13.2.3), in place.
pub(crate) fn inverse_dct<T: Lane>(t: &mut [T], n: u32, r: u32) {
match n {
2 => inverse_dct_n::<T, 2>(t, r),
3 => inverse_dct_n::<T, 3>(t, r),
4 => inverse_dct_n::<T, 4>(t, r),
5 => inverse_dct_n::<T, 5>(t, r),
_ => inverse_dct_n::<T, 6>(t, r),
}
}
fn inverse_dct_n<T: Lane, const N: u32>(t: &mut [T], r: u32) {
// 7.13.2.2 permutation
let len = 1usize << n;
let len = 1usize << N;
let mut copy = [T::ZERO; 64];
copy[..len].copy_from_slice(&t[..len]);
for i in 0..len {
t[i] = copy[brev(n, i)];
t[i] = copy[BREV[N as usize][i] as usize];
}
dct_stage_outer_rotations(t, n, r);
dct_stage_even_half(t, n, r);
dct_stage_merge_quarters(t, n, r);
dct_stage_merge_halves(t, n, r);
dct_stage_outer_rotations::<T, N>(t, r);
dct_stage_even_half::<T, N>(t, r);
dct_stage_merge_quarters::<T, N>(t, r);
dct_stage_merge_halves::<T, N>(t, r);
}
const SINPI_1_9: i64 = 1321;
@@ -543,8 +580,9 @@ fn col_kind(tx_type: usize) -> Kind1d {
}
/// 2D inverse transform (7.13.3). `dequant` is 64x64 (row stride 64, only the top-left 32x32
/// may be non-zero); the residual is written to `residual` with row stride `w`.
pub(crate) fn inverse_transform_2d(dequant: &[i32], residual: &mut [i32], tx_sz: usize, tx_type: usize, lossless: bool, bit_depth: u32) {
/// may be non-zero; with `dc_only` only the first value); the residual is written to
/// `residual` with row stride `w`.
pub(crate) fn inverse_transform_2d(dequant: &[i32], residual: &mut [i32], tx_sz: usize, tx_type: usize, lossless: bool, bit_depth: u32, dc_only: bool) {
let log2w = TX_WIDTH_LOG2[tx_sz] as u32;
let log2h = TX_HEIGHT_LOG2[tx_sz] as u32;
let w = 1usize << log2w;
@@ -559,6 +597,9 @@ pub(crate) fn inverse_transform_2d(dequant: &[i32], residual: &mut [i32], tx_sz:
let rect2 = log2w.abs_diff(log2h) == 1;
if !lossless && bit_depth <= 10 {
let p = Tx2d { w, h, log2w, log2h, row_shift, col_shift, row_clamp, col_clamp, rk, ck, rect2 };
if dc_only && rk == Kind1d::Dct && ck == Kind1d::Dct && transform_dc_only(dequant[0], residual, &p) {
return;
}
if w >= 8 && h >= 8 {
transform_lanes::<8>(dequant, residual, &p);
} else {
@@ -638,6 +679,32 @@ fn transform_1d<T: Lane>(t: &mut [T], kind: Kind1d, n: u32, r: u32) {
}
}
/// [`transform_lanes`] for a DCT_DCT block whose only coefficient is DC: the other rows are
/// zero and every column gets the same input, so one row and one column are transformed and
/// the column is replicated. Returns false (nothing written) if the row output is not flat.
fn transform_dc_only(dc: i32, residual: &mut [i32], p: &Tx2d) -> bool {
let (w, h) = (p.w, p.h);
let lo = -(1i32 << (p.col_clamp - 1));
let hi = (1i32 << (p.col_clamp - 1)) - 1;
let mut t = [[0i32; 1]; 64];
t[0] = [dc];
if p.rect2 {
t[0] = t[0].mul_round12(2896);
}
inverse_dct(&mut t[..], p.log2w, p.row_clamp);
let v = round2(t[0][0], p.row_shift).clamp(lo, hi);
if t[1..w].iter().any(|x| round2(x[0], p.row_shift).clamp(lo, hi) != v) {
return false;
}
let mut c = [[0i32; 1]; 64];
c[0] = [v];
inverse_dct(&mut c[..], p.log2h, p.col_clamp);
for i in 0..h {
residual[i * w..i * w + w].fill(round2(c[i][0], p.col_shift));
}
true
}
/// The 2D inverse transform with `L` rows (row pass) or columns (column pass) at a time;
/// bit-identical to the one-at-a-time process for bit depths up to 10.
fn transform_lanes<const L: usize>(dequant: &[i32], residual: &mut [i32], p: &Tx2d) {
@@ -692,6 +759,7 @@ fn transform_lanes<const L: usize>(dequant: &[i32], residual: &mut [i32], p: &Tx
#[cfg(test)]
mod tests {
use super::*;
use crate::spec_tables::{TX_HEIGHT, TX_SIZES_ALL, TX_WIDTH};
/// The DCT step list must implement an orthogonal transform: a DC-only input gives a flat
/// output equal to DC * cos(pi/4) (in the 2^12 fixed-point scale).
@@ -708,6 +776,26 @@ mod tests {
}
}
/// The DC-only shortcut gives the same residual as the full 2D transform.
#[test]
fn dc_only_matches_full_transform() {
for tx_sz in 0..TX_SIZES_ALL {
let (w, h) = (TX_WIDTH[tx_sz] as usize, TX_HEIGHT[tx_sz] as usize);
for bd in [8, 10] {
let lim = 1i32 << (7 + bd);
for dc in [1, -1, 37, -1000, 4096, lim - 1, -lim] {
let mut dequant = vec![0; 64 * 64];
dequant[0] = dc;
let mut full = vec![0; 64 * 64];
let mut fast = vec![1; 64 * 64];
inverse_transform_2d(&dequant, &mut full, tx_sz, DCT_DCT, false, bd, false);
inverse_transform_2d(&dequant, &mut fast, tx_sz, DCT_DCT, false, bd, true);
assert_eq!(full[..w * h], fast[..w * h], "tx_sz {tx_sz} bd {bd} dc {dc}");
}
}
}
}
/// Compare each inverse DCT basis vector with the real-valued DCT-III.
#[test]
fn dct_matches_float() {
crates/av1/tests/it/common/mod.rs
+1 -1
View File
@@ -76,7 +76,7 @@ pub fn make(ff: &Path, spec: &Spec) -> PathBuf {
args.extend(spec.enc.iter().map(String::as_str));
args.extend(["-f", "ivf", t.as_str()]);
run(ff, &args);
std::fs::rename(&tmp, &path).unwrap();
filmcraft_testkit::fixtures::publish(&tmp, &path).unwrap();
}
path
}
crates/av1/tests/it/oracle_inter.rs
+3 -1
View File
@@ -15,12 +15,14 @@ fn specs() -> Vec<Spec> {
]
}
/// Super-resolution (denominator 12 on the inter frames) and film grain synthesis.
/// Super-resolution (denominator 12 on the inter frames), film grain synthesis and quantiser
/// matrices.
fn specs_tools() -> Vec<Spec> {
let p = |params: &'static str| -> Vec<&'static str> { vec!["-preset", "6", "-crf", "35", "-svtav1-params", params] };
vec![
Spec::new("inter_superres", "testsrc2=s=416x240:r=25", 8, "yuv420p", &p("superres-mode=1:superres-denom=12:superres-kf-denom=13")),
Spec::new("inter_film_grain", "testsrc2=s=352x288:r=25,noise=alls=20:allf=t", 8, "yuv420p", &p("film-grain=10")),
Spec::new("inter_qmatrix_10bit", "mandelbrot=s=352x288:r=25", 8, "yuv420p10le", &p("enable-qm=1:qm-min=0:qm-max=8")),
]
}
crates/av1/tests/it/oracle_intra.rs
+1
View File
@@ -34,6 +34,7 @@ fn specs_filters() -> Vec<Spec> {
Spec::new("intra_lr_mandel", "mandelbrot=s=352x288:r=25", 2, "yuv420p", &p("4", "40", "enable-cdef=0:keyint=1")),
Spec::new("intra_all_testsrc", "testsrc2=s=640x360:r=25", 2, "yuv420p", &p("4", "45", "keyint=1")),
Spec::new("intra_all_10bit", "mandelbrot=s=320x240:r=25", 2, "yuv420p10le", &p("4", "40", "keyint=1")),
Spec::new("intra_qmatrix_noise", "testsrc2=s=352x288:r=25,noise=alls=10:allf=t", 2, "yuv420p", &p("6", "30", "enable-qm=1:qm-min=0:qm-max=8:keyint=1")),
Spec::new("intra_all_noise_odd", "testsrc2=s=250x150:r=25,noise=alls=20:allf=t", 2, "yuv420p", &p("3", "35", "keyint=1")),
// 128x128 superblocks, intra edge filter off, Wiener LR on all planes
Spec::new("intra_all_grad_noise_10bit", "gradients=s=640x360:r=25,noise=alls=25:allf=t", 1, "yuv420p10le", &p("1", "50", "keyint=1")),
crates/avi/src/lib.rs
+31 -9
View File
@@ -297,17 +297,38 @@ pub fn open_with<S: ByteSource + ?Sized>(src: &S, opts: OpenOptions) -> Result<A
} else if let (false, Some((at, size)), Some(first)) = (opts.ignore_idx1, idx1, movis.first()) {
clear(&mut streams);
read_idx1(src, &mut streams, at, size, first.0);
// idx1 only covers the first RIFF: the AVIX extensions are scanned
for &(_, from, to) in movis.iter().skip(1) {
scan(src, &mut streams, from, to, 0);
if streams.iter().all(|s| s.chunks.is_empty()) {
scan_all(src, &mut streams, &movis)
} else {
scan_missing(src, &mut streams, first);
// idx1 only covers the first RIFF: the AVIX extensions are scanned
for &(_, from, to) in movis.iter().skip(1) {
scan(src, &mut streams, &|_| true, from, to, 0);
}
IndexSource::Idx1
}
if streams.iter().all(|s| s.chunks.is_empty()) { scan_all(src, &mut streams, &movis) } else { IndexSource::Idx1 }
} else {
scan_all(src, &mut streams, &movis)
};
Ok(AviFile { header, streams, index })
}
/// A stream that declares data (`length`) but that `idx1` left without a single chunk while
/// others have theirs (its entries damaged or missing) gets its chunks from a scan of the first
/// `movi` list (the one `idx1` covers), as a file without an index does, before the AVIX
/// extensions add theirs; its key frames are then unknown. The indexed streams keep their
/// tables, and a stream that declares no data is not looked for.
fn scan_missing<S: ByteSource + ?Sized>(src: &S, streams: &mut [Stream], &(_, from, to): &(u64, u64, u64)) {
let missing: Vec<bool> = streams.iter().map(|s| s.chunks.is_empty() && s.length > 0).collect();
if !missing.contains(&true) {
return;
}
scan(src, streams, &|n| missing.get(n) == Some(&true), from, to, 0);
for (s, _) in streams.iter_mut().zip(&missing).filter(|(_, m)| **m) {
s.keyframes_known = false;
}
}
fn clear(streams: &mut [Stream]) {
for s in streams.iter_mut() {
s.chunks.clear();
@@ -318,7 +339,7 @@ fn clear(streams: &mut [Stream]) {
fn scan_all<S: ByteSource + ?Sized>(src: &S, streams: &mut [Stream], movis: &[(u64, u64, u64)]) -> IndexSource {
clear(streams);
for &(_, from, to) in movis {
scan(src, streams, from, to, 0);
scan(src, streams, &|_| true, from, to, 0);
}
for s in streams.iter_mut() {
s.keyframes_known = false;
@@ -535,22 +556,23 @@ fn read_idx1<S: ByteSource + ?Sized>(src: &S, streams: &mut [Stream], at: u64, s
}
}
/// Append the data chunks of a `movi` list (`[from, to)`) to the streams' tables, in file order.
fn scan<S: ByteSource + ?Sized>(src: &S, streams: &mut [Stream], from: u64, to: u64, depth: u32) {
/// Append the data chunks of a `movi` list (`[from, to)`) to the tables of the streams `want`
/// picks, in file order.
fn scan<S: ByteSource + ?Sized>(src: &S, streams: &mut [Stream], want: &dyn Fn(usize) -> bool, from: u64, to: u64, depth: u32) {
if depth >= MAX_DEPTH {
return;
}
for (id, size, data) in children(src, from, to) {
if id == *b"LIST" {
if size >= 4 {
scan(src, streams, data + 4, data + size, depth + 1);
scan(src, streams, want, data + 4, data + size, depth + 1);
}
continue;
}
if &id[..2] == b"ix" || &id[2..] == b"pc" {
continue;
}
if let Some(s) = Stream::number(&id).and_then(|n| streams.get_mut(n)) {
if let Some(s) = Stream::number(&id).filter(|&n| want(n)).and_then(|n| streams.get_mut(n)) {
s.chunks.push(Chunk { offset: data, size: u32::try_from(size).unwrap_or(u32::MAX), key: true });
}
}
crates/avi/src/tests.rs
+70
View File
@@ -23,6 +23,8 @@ fn strl(video: bool) -> Vec<u8> {
let (scale, rate, ss) = if video { (1u32, 25u32, 0u32) } else { (4, 192_000, 4) };
strh[20..24].copy_from_slice(&scale.to_le_bytes());
strh[24..28].copy_from_slice(&rate.to_le_bytes());
// declared length: 3 frames, 6 audio blocks
strh[32..36].copy_from_slice(&(if video { 3u32 } else { 6 }).to_le_bytes());
strh[44..48].copy_from_slice(&ss.to_le_bytes());
let strf = if video {
let mut b = vec![0u8; 40];
@@ -150,6 +152,74 @@ fn not_avi_and_truncated_recordings() {
assert!(f.read_chunk(&cut, 0, 0).is_ok());
}
/// The idx1 entries of one stream damaged (offsets past the file) while the other's are fine: the
/// video still comes from a scan of `movi` instead of vanishing, its key frames unknown, and the
/// audio keeps its index. A stream with no data at all stays empty.
#[test]
fn a_stream_whose_index_entries_are_damaged_is_scanned() {
let mut file = avi(true, false, false);
let ix = file.windows(4).rposition(|w| w == b"idx1").unwrap() + 8;
for e in file[ix..].as_chunks_mut::<16>().0.iter_mut().filter(|e| &e[..4] == b"00dc") {
e[8..12].copy_from_slice(&0xFFFF_FF00u32.to_le_bytes());
}
let f = open(&file).unwrap();
assert_eq!(f.index, IndexSource::Idx1);
assert_eq!(table(&f, 0), vec![(10, true), (6, true), (7, true)]);
assert!(!f.streams[0].keyframes_known);
assert_eq!(f.read_chunk(&file, 0, 1).unwrap(), vec![3; 6]);
assert_eq!(table(&f, 1), vec![(16, true), (8, true)]);
assert!(f.streams[1].keyframes_known);
// with an AVIX extension after it: the first RIFF's frames come first, then the extension's
let mut extended = file.clone();
extended.extend(chunk(b"RIFF", &[b"AVIX".to_vec(), list(b"movi", &chunk(b"00dc", &[6; 9]))].concat()));
let f = open(&extended).unwrap();
assert_eq!(table(&f, 0), vec![(10, true), (6, true), (7, true), (9, true)]);
assert_eq!(table(&f, 1), vec![(16, true), (8, true)]);
// an audio-only recording of a file that declares video: nothing to find
let mut silent = avi(true, false, false);
while let Some(i) = silent.windows(4).position(|w| w == b"00dc") {
silent[i..i + 4].copy_from_slice(b"JUNK");
}
let f = open(&silent).unwrap();
assert!(f.streams[0].chunks.is_empty());
assert_eq!(table(&f, 1), vec![(16, true), (8, true)]);
}
/// A healthy file whose index leaves a stream empty because it declares no data (an audio track
/// that never recorded a sample) opens from the index alone: the stream is not looked for in
/// `movi`, which a large file would pay for with a read per chunk.
#[test]
fn an_empty_stream_that_declares_no_data_is_not_scanned() {
struct Counted<'a>(&'a [u8], std::cell::Cell<usize>);
impl ByteSource for Counted<'_> {
fn len(&self) -> u64 {
self.0.len() as u64
}
fn read_at(&self, offset: u64, buf: &mut [u8]) -> std::io::Result<()> {
self.1.set(self.1.get() + 1);
self.0.read_at(offset, buf)
}
}
let open_counted = |file: &[u8]| {
let src = Counted(file, Default::default());
(open(&src).unwrap(), src.1.get())
};
let intact = avi(true, false, false);
// the audio's index entries gone and its length zero; its chunks stay where a scan would find them
let mut file = intact.clone();
let ix = file.windows(4).rposition(|w| w == b"idx1").unwrap() + 8;
for e in file[ix..].as_chunks_mut::<16>().0.iter_mut().filter(|e| &e[..4] == b"01wb") {
e[..4].copy_from_slice(b"JUNK");
}
let strh = file.windows(4).enumerate().filter(|(_, w)| w == b"strh").nth(1).unwrap().0 + 8;
file[strh + 32..strh + 36].fill(0);
let (f, reads) = open_counted(&file);
assert_eq!(f.index, IndexSource::Idx1);
assert_eq!(table(&f, 0), vec![(10, true), (6, false), (7, true)]);
assert!(f.streams[1].chunks.is_empty() && f.streams[1].keyframes_known);
assert_eq!(reads, open_counted(&intact).1);
}
/// Every single-byte change and every truncation of a valid file opens or fails, never panics, and
/// whatever opens reads its chunks without panicking.
#[test]
crates/bitstream/src/lib.rs
+45
View File
@@ -3,3 +3,48 @@
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable))]
pub use craft_media::bitstream::*;
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn hostile_lengths_nal_size_does_not_wrap() {
// #2801: adding the four-byte prefix to u32::MAX overflowed usize on wasm32.
for data in [&[0xff; 4][..], &[0, 0, 0, 1, 0x67, 0xff, 0xff, 0xff, 0xff][..]] {
let result = std::panic::catch_unwind(|| length_prefixed_nals(data, 4));
assert_eq!(result.expect("NAL splitting must not panic"), Err(BitError::Eof));
}
}
#[test]
fn hostile_lengths_nal_prefix_width_is_bounded() {
for width in [0, 5, 8, usize::MAX] {
for data in [&[][..], &[1, 0x67][..]] {
let result = std::panic::catch_unwind(|| length_prefixed_nals(data, width));
assert_eq!(result.expect("invalid prefix widths must not panic or loop"), Err(BitError::InvalidNalLengthSize));
}
}
}
#[test]
fn hostile_lengths_nal_mutations_and_truncations() {
for width in 1..=4 {
let mut data = vec![0; width]; // retain an empty NAL
data.extend_from_slice(&1u32.to_be_bytes()[4 - width..]);
data.push(0x67);
assert_eq!(length_prefixed_nals(&data, width).unwrap(), vec![&[][..], &[0x67][..]]);
assert_eq!(length_prefixed_nals(&[], width).unwrap(), Vec::<&[u8]>::new());
for end in 0..=data.len() {
std::panic::catch_unwind(|| length_prefixed_nals(&data[..end], width)).expect("truncated NAL sample must not panic").ok();
}
for index in 0..data.len() {
for bit in 0..8 {
let mut mutated = data.clone();
mutated[index] ^= 1 << bit;
std::panic::catch_unwind(|| length_prefixed_nals(&mutated, width)).expect("mutated NAL sample must not panic").ok();
}
}
}
}
}
crates/captions/src/burn.rs
+166 -5
View File
@@ -1,7 +1,7 @@
//! Caption layout and burn-in.
//!
//! [`overlay`] lays out the caption showing on a track at a time with the track's
//! [`CaptionStyle`] (size relative to a 1080-line frame, colour, background box, outline,
//! [`CaptionStyle`] (font style, size relative to a 1080-line frame, colour, background box, outline,
//! alignment, anchor and margin; WebVTT `line:N%` and `align:` cue settings override the style)
//! and rasterises it into a small premultiplied linear-light RGBA [`Overlay`] positioned in the
//! frame. Renderers composite overlays over the finished picture.
@@ -55,10 +55,12 @@ impl Overlay {
}
}
/// The caption face: the track's font family (Inter when empty or not installed) in SemiBold at `px`.
fn caption_style(font: &str, px: f32) -> TextStyle {
/// The caption face: the track's font family (Inter when empty or not installed) in the track's
/// font style (SemiBold when empty) at `px`.
fn caption_style(font: &str, font_style: &str, px: f32) -> TextStyle {
let family = if font.trim().is_empty() { filmcraft_text::fonts::DEFAULT_FAMILY } else { font.trim() };
TextStyle { family: family.into(), style: "SemiBold".into(), size: px, ..Default::default() }
let style = if font_style.trim().is_empty() { "SemiBold" } else { font_style.trim() };
TextStyle { family: family.into(), style: style.into(), size: px, ..Default::default() }
}
fn lin(c: [u8; 4]) -> [f32; 4] {
@@ -118,7 +120,7 @@ pub fn render_caption(c: &Caption, style: &CaptionStyle, w: usize, h: usize) ->
let scale = h as f32 / 1080.0;
let px = (style.size.clamp(4.0, MAX_CAPTION_SIZE_1080) * scale).clamp(4.0, h as f32);
let max_w = w as f32 * 0.9;
let ts = caption_style(&style.font, px);
let ts = caption_style(&style.font, &style.font_style, px);
let mut lines: Vec<String> = Vec::new();
// (speaker names are metadata; like Premiere, they are not burned in)
for l in c.plain_lines() {
@@ -377,6 +379,165 @@ mod tests {
}
}
/// `render_caption` as it was before the caption box settings (#3354-#3358), kept verbatim
/// (with the SemiBold face inlined) so every new style field can prove its default leaves
/// existing projects pixel-identical.
#[allow(clippy::needless_range_loop)]
fn reference_render_caption(c: &Caption, style: &CaptionStyle, w: usize, h: usize) -> Option<Overlay> {
if w == 0 || h == 0 {
return None;
}
let scale = h as f32 / 1080.0;
let px = (style.size.clamp(4.0, MAX_CAPTION_SIZE_1080) * scale).clamp(4.0, h as f32);
let max_w = w as f32 * 0.9;
let family = if style.font.trim().is_empty() { filmcraft_text::fonts::DEFAULT_FAMILY } else { style.font.trim() };
let ts = TextStyle { family: family.into(), style: "SemiBold".into(), size: px, ..Default::default() };
let mut lines: Vec<String> = Vec::new();
// (speaker names are metadata; like Premiere, they are not burned in)
for l in c.plain_lines() {
lines.extend(wrap(l.trim(), &ts, max_w));
}
lines.retain(|l| !l.is_empty());
if lines.is_empty() {
return None;
}
let vm = filmcraft_text::fonts::face(filmcraft_text::resolve(&ts.family, &ts.style).face).metrics(px);
let (asc, desc) = (vm.ascent, vm.descent);
let lh = px * style.line_spacing.clamp(0.8, 4.0);
let pad_x = (px * 0.3).round();
let block_h = lh * lines.len() as f32;
let (line_override, align_override) = vtt_overrides(&c.settings);
let align = align_override.unwrap_or(style.align);
let margin = style.margin.clamp(0.0, 0.45) * h as f32;
let top = match (line_override, style.anchor) {
(Some(f), _) => (f * h as f32).min(h as f32 - block_h),
(None, CaptionAnchor::Top) => margin,
(None, CaptionAnchor::Middle) => (h as f32 - block_h) / 2.0,
(None, CaptionAnchor::Bottom) => h as f32 - margin - block_h,
}
.max(0.0)
.round();
let side = w as f32 * 0.05;
let widths: Vec<f32> = lines.iter().map(|l| filmcraft_text::measure(l, &ts)).collect();
let xs: Vec<f32> = widths
.iter()
.map(|&lw| match align {
CaptionAlign::Left => side + pad_x,
CaptionAlign::Right => w as f32 - side - pad_x - lw,
CaptionAlign::Center => (w as f32 - lw) / 2.0,
})
.collect();
let outline = (style.outline * scale).clamp(0.0, MAX_OUTLINE_RADIUS);
let grow = outline.ceil() + 1.0;
let bx0 = xs.iter().zip(&widths).map(|(x, _)| x - pad_x - grow).fold(f32::MAX, f32::min).floor();
let bx1 = xs.iter().zip(&widths).map(|(x, lw)| x + lw + pad_x + grow).fold(f32::MIN, f32::max).ceil();
let by0 = (top - grow).floor();
let by1 = (top + block_h + grow).ceil();
// Text can be wider than the frame (an unbroken imported WebVTT word, for example). Raster
// only the visible intersection; `draw` receives the original coordinates and clips into it.
let (left, top) = (bx0.clamp(0.0, w as f32), by0.clamp(0.0, h as f32));
let (right, bottom) = (bx1.clamp(0.0, w as f32), by1.clamp(0.0, h as f32));
if right <= left || bottom <= top {
return None;
}
let (ox, oy) = (left as i32, top as i32);
let (ow, oh) = ((right - left) as usize, (bottom - top) as usize);
if ow.checked_mul(oh).is_none_or(|pixels| pixels > MAX_OVERLAY_PIXELS) {
return None;
}
let mut cover = vec![0.0f32; ow * oh];
let mut bg = vec![0.0f32; ow * oh];
for (i, line) in lines.iter().enumerate() {
let lt = top + lh * i as f32;
if style.background && style.background_color[3] > 0 {
let (x0, x1) = (xs[i] - pad_x, xs[i] + widths[i] + pad_x);
let (y0, y1) = (lt, lt + lh);
for y in (y0.floor() as i32).max(oy)..(y1.ceil() as i32).min(oy + oh as i32) {
let fy = ((y as f32 + 1.0).min(y1) - (y as f32).max(y0)).clamp(0.0, 1.0);
for x in (x0.floor() as i32).max(ox)..(x1.ceil() as i32).min(ox + ow as i32) {
let fx = ((x as f32 + 1.0).min(x1) - (x as f32).max(x0)).clamp(0.0, 1.0);
let j = (y - oy) as usize * ow + (x - ox) as usize;
bg[j] = (bg[j] + fx * fy).min(1.0);
}
}
}
let baseline = lt + (lh - (asc + desc)) / 2.0 + asc;
let l = layout(line, &ts, &ParagraphStyle::default());
let mut m = filmcraft_text::Mask { w: ow, h: oh, a: std::mem::take(&mut cover) };
render::draw(&l, &render::at(xs[i], baseline.round()), &mut m, (ox, oy));
cover = m.a;
}
let stroke = if outline > 0.0 { dilate(&cover, ow, oh, outline) } else { Vec::new() };
let (tc, bc, oc) = (lin(style.color), lin(style.background_color), lin(style.outline_color));
let mut out = vec![0.0f32; ow * oh * 4];
for j in 0..ow * oh {
let mut p = [0.0f32; 4];
let over = |p: &mut [f32; 4], c: [f32; 4], a: f32| {
if a > 0.0 {
for k in 0..4 {
p[k] = c[k] * a + p[k] * (1.0 - c[3] * a);
}
}
};
over(&mut p, bc, bg[j]);
if !stroke.is_empty() {
over(&mut p, oc, stroke[j]);
}
over(&mut p, tc, cover[j]);
out[j * 4..j * 4 + 4].copy_from_slice(&p);
}
Some(Overlay { x: ox, y: oy, w: ow, h: oh, px: out })
}
/// Styles and captions existing projects use, at the frame sizes they are drawn at.
fn reference_cases() -> Vec<(Caption, CaptionStyle, usize, usize)> {
let (long, unbroken) = ("word ".repeat(40), "x".repeat(300));
let texts = ["Hello world", "Two\nlines here", long.as_str(), unbroken.as_str(), "<i>Styled</i> &amp; done"];
let styles = [
CaptionStyle::default(),
CaptionStyle { anchor: CaptionAnchor::Top, align: CaptionAlign::Left, size: 40.0, ..Default::default() },
CaptionStyle { anchor: CaptionAnchor::Middle, align: CaptionAlign::Right, line_spacing: 1.6, ..Default::default() },
CaptionStyle { outline: 3.0, background_color: [20, 40, 200, 128], margin: 0.31, ..Default::default() },
CaptionStyle { background: false, font: "Noto Serif".into(), ..Default::default() },
CaptionStyle { size: 200.0, margin: 0.0, anchor: CaptionAnchor::Top, ..Default::default() },
];
let mut out = Vec::new();
for (w, h) in [(1920, 1080), (1280, 720), (1080, 1920), (640, 360)] {
for st in &styles {
for t in texts {
let mut c = cap(t);
out.push((c.clone(), st.clone(), w, h));
c.settings = "line:30% align:end".into();
out.push((c, st.clone(), w, h));
}
}
}
out
}
#[test]
fn default_style_renders_like_the_reference() {
for (c, st, w, h) in reference_cases() {
assert_eq!(render_caption(&c, &st, w, h), reference_render_caption(&c, &st, w, h), "{:?} {w}x{h} {st:?}", c.text);
}
}
#[test]
fn font_style_is_used() {
// two styles of one family differ; an empty style is the default SemiBold
let semi = render_caption(&cap("Hello world"), &CaptionStyle::default(), 1920, 1080).unwrap();
for style in ["Regular", "Bold"] {
let st = CaptionStyle { font_style: style.into(), ..Default::default() };
let o = render_caption(&cap("Hello world"), &st, 1920, 1080).unwrap();
assert_ne!((o.w, &o.px), (semi.w, &semi.px), "{style} renders like SemiBold");
}
let empty = CaptionStyle { font_style: " ".into(), ..Default::default() };
assert_eq!(render_caption(&cap("Hello world"), &empty, 1920, 1080), Some(semi.clone()));
// a style the family lacks falls back to the nearest weight, like graphic text
let missing = CaptionStyle { font_style: "No Such Style".into(), ..Default::default() };
assert!(render_caption(&cap("Hello world"), &missing, 1920, 1080).is_some());
}
#[test]
fn hidden_track_draws_nothing() {
let mut t = CaptionTrack::new(TrackId(1), "S".into(), CaptionFormat::Subtitle);
crates/captions/src/mcc.rs
+17 -3
View File
@@ -626,11 +626,11 @@ pub fn parse(text: &str) -> Result<Document> {
continue;
};
let tc = tc.split('.').next().unwrap_or(tc);
let parts: Vec<i64> = tc.split([':', ';']).filter_map(|p| p.parse().ok()).collect();
let [h, m, s, f] = parts[..] else {
let parts: Option<Vec<i64>> = tc.split([':', ';']).map(|p| p.parse().ok()).collect();
let Some([h, m, s, f]) = parts.as_deref() else {
return Err(Error::Syntax { line: n + 1, msg: format!("bad timecode \"{tc}\"") });
};
let frame = fields_to_frames(h, m, s, f, rate.0, rate.1 || tc.contains(';'));
let frame = fields_to_frames(*h, *m, *s, *f, rate.0, rate.1 || tc.contains(';'));
let Some(bytes) = expand(data) else {
doc.warnings.push(format!("line {}: bad caption data", n + 1));
continue;
@@ -731,6 +731,20 @@ mod tests {
assert!(parse("garbage").is_err());
}
/// Reject malformed fields instead of silently dropping them from the timecode.
#[test]
fn rejects_timecodes_with_unparseable_extra_fields() {
let doc = Document { cues: vec![cue(60, 120, "Caption")], ..Default::default() };
let text = write(&doc, false);
assert_eq!(parse(&text).unwrap().cues, doc.cues);
let (line_index, line) = text.lines().enumerate().find(|(_, line)| line.contains('\t')).unwrap();
let (tc, data) = line.split_once('\t').unwrap();
for prefix in ["bad:", ":", "9223372036854775808:", "00:"] {
let malformed = text.replacen(line, &format!("{prefix}{tc}\t{data}"), 1);
assert!(matches!(parse(&malformed), Err(Error::Syntax { line, .. }) if line == line_index + 1), "accepted malformed timecode: {prefix}{tc}");
}
}
/// Keep separated short captions compact while preserving cues and valid CDP packets.
/// Exercise each stream combination with both drop-frame and non-drop timecode labels.
#[test]
crates/captions/src/stl.rs
+57 -14
View File
@@ -266,8 +266,16 @@ fn ascii_field(b: &[u8]) -> String {
String::from_utf8_lossy(b).trim().to_string()
}
fn tc_frames(b: &[u8], rate: FrameRate) -> i64 {
fields_to_frames(b[0] as i64, b[1] as i64, b[2] as i64, b[3] as i64, rate, false)
fn tc_frames(b: &[u8], rate: FrameRate) -> Option<i64> {
let [hours, minutes, seconds, frames] = b else { return None };
if rate.num <= 0 || rate.den <= 0 {
return None;
}
let max_frames = rate.num / rate.den + i64::from(rate.num % rate.den != 0);
if *minutes >= 60 || *seconds >= 60 || i64::from(*frames) >= max_frames {
return None;
}
Some(fields_to_frames(*hours as i64, *minutes as i64, *seconds as i64, *frames as i64, rate, false))
}
pub fn parse(bytes: &[u8]) -> Result<Document> {
@@ -287,9 +295,9 @@ pub fn parse(bytes: &[u8]) -> Result<Document> {
0
};
let mut doc = Document::default();
let mut cur: Option<(u16, i64, i64, String, bool, bool)> = None;
let finish = |c: Option<(u16, i64, i64, String, bool, bool)>, doc: &mut Document| {
if let Some((_, tin, tout, mut text, italic, underline)) = c {
let mut cur: Option<(u16, Option<(i64, i64)>, String, bool, bool)> = None;
let finish = |c: Option<(u16, Option<(i64, i64)>, String, bool, bool)>, doc: &mut Document| {
if let Some((_, times, mut text, italic, underline)) = c {
if italic {
text.push_str("</i>");
}
@@ -297,8 +305,12 @@ pub fn parse(bytes: &[u8]) -> Result<Document> {
text.push_str("</u>");
}
let text = text.lines().map(str::trim).filter(|l| !l.is_empty()).collect::<Vec<_>>().join("\n");
if tout > tin && !text.is_empty() {
doc.cues.push(Cue { start: rate.tick_of((tin - start).max(0)), end: rate.tick_of((tout - start).max(0)), text, ..Default::default() });
if let Some((tin, tout)) = times {
if tout > tin && !text.is_empty() {
doc.cues.push(Cue { start: rate.tick_of((tin - start).max(0)), end: rate.tick_of((tout - start).max(0)), text, ..Default::default() });
}
} else {
doc.warnings.push("skipped EBU STL cue with invalid timecode".into());
}
}
};
@@ -309,18 +321,17 @@ pub fn parse(bytes: &[u8]) -> Result<Document> {
if ebn == 0xfe || cf != 0 {
continue; // user data / comment
}
let tin = tc_frames(&blk[5..9], rate);
let tout = tc_frames(&blk[9..13], rate);
let same = cur.as_ref().is_some_and(|c| c.0 == sn);
if !same {
finish(cur.take(), &mut doc);
cur = Some((sn, tin, tout, String::new(), false, false));
let times = tc_frames(&blk[5..9], rate).zip(tc_frames(&blk[9..13], rate));
cur = Some((sn, times, String::new(), false, false));
}
if let Some(c) = cur.as_mut() {
let (mut it, mut ul) = (c.4, c.5);
decode_tf(&blk[16..16 + TF], &mut c.3, &mut it, &mut ul);
c.4 = it;
c.5 = ul;
let (mut it, mut ul) = (c.3, c.4);
decode_tf(&blk[16..16 + TF], &mut c.2, &mut it, &mut ul);
c.3 = it;
c.4 = ul;
}
if ebn == 0xff {
finish(cur.take(), &mut doc);
@@ -474,6 +485,38 @@ mod tests {
assert!(parse(&b[..500]).is_err());
}
#[test]
fn rejects_out_of_range_stl_timecodes() {
let doc = Document {
cues: vec![Cue { start: FrameRate::FPS_25.tick_of(25), end: FrameRate::FPS_25.tick_of(75), text: "A".into(), ..Default::default() }],
..Default::default()
};
let mut bytes = write(&doc, None);
bytes[GSI + 6] = 60;
let parsed = parse(&bytes).expect("invalid cue timecodes do not fail the file");
assert!(parsed.cues.is_empty());
assert_eq!(parsed.warnings, ["skipped EBU STL cue with invalid timecode"]);
assert_eq!(tc_frames(&[0, 0, 0, 24], FrameRate::FPS_24), None);
let mut continuation = write(
&Document {
cues: vec![Cue { start: FrameRate::FPS_25.tick_of(25), end: FrameRate::FPS_25.tick_of(75), text: "x".repeat(150), ..Default::default() }],
..Default::default()
},
None,
);
continuation[GSI + TTI + 5..GSI + TTI + 13].fill(60);
let parsed = parse(&continuation).expect("continuation timecodes are ignored");
assert_eq!(parsed.cues.len(), 1);
assert!(parsed.warnings.is_empty());
let mut long_hours = write(&doc, None);
long_hours[GSI + 5] = 25;
long_hours[GSI + 9] = 26;
let parsed = parse(&long_hours).expect("hours beyond 23 are valid");
assert_eq!(parsed.cues.len(), 1);
}
#[test]
fn literal_angle_brackets_are_not_discarded_as_markup() {
let literal = "2 < 3 > 1, x < y > z, and 5 <7> 6";
crates/captions/src/vtt.rs
+65 -6
View File
@@ -46,6 +46,15 @@ pub fn split_voice(text: &str) -> (Option<String>, String) {
(None, text.to_string())
}
/// Undo [`write`]'s cue-text escape, so `A --> B` comes back as `A --> B` and not `A --&gt; B`.
///
/// Only the one substitution the writer makes is reversed: a literal `-->` in cue text would read
/// as a timing line to anything scanning the block, so it is written `--&gt;`. Other entities
/// (`&amp;`, `&lt;`, …) are cue markup and stay verbatim, as `parse` has always kept them.
fn unescape_text(text: &str) -> String {
text.replace("--&gt;", "-->")
}
pub fn parse(text: &str) -> Result<Document> {
let mut doc = Document::default();
let mut lines = text.lines().peekable();
@@ -79,19 +88,21 @@ pub fn parse(text: &str) -> Result<Document> {
let mut seen_cue = false;
for b in blocks {
let first = b[0].trim();
if !b.iter().any(|l| l.contains("-->")) && (first == "NOTE" || first.starts_with("NOTE ") || first.starts_with("NOTE\t")) {
let timing_idx = b.iter().position(|line| parse_timing_line(line).is_some());
if timing_idx.is_none() && (first == "NOTE" || first.starts_with("NOTE ") || first.starts_with("NOTE\t")) {
doc.blocks.push(b.join("\n"));
continue;
}
if !seen_cue && !b.iter().any(|l| l.contains("-->")) && (first == "STYLE" || first == "REGION") {
if !seen_cue && timing_idx.is_none() && (first == "STYLE" || first == "REGION") {
doc.blocks.push(b.join("\n"));
continue;
}
let (id, timing_idx) = if b[0].contains("-->") { (None, 0) } else { (Some(b[0].trim().to_string()), 1) };
let Some(timing) = b.get(timing_idx) else {
let Some(timing_idx) = timing_idx else {
doc.warnings.push(format!("ignored block \"{first}\""));
continue;
};
let id = (timing_idx > 0).then(|| b[..timing_idx].join("\n").trim().to_string());
let timing = &b[timing_idx];
let Some((start, end, settings)) = parse_timing_line(timing) else {
doc.warnings.push(format!("bad timing line \"{}\"", timing.trim()));
continue;
@@ -102,7 +113,7 @@ pub fn parse(text: &str) -> Result<Document> {
continue;
}
let raw = b[timing_idx + 1..].iter().map(|l| l.trim_end()).collect::<Vec<_>>().join("\n");
let (speaker, text) = split_voice(&raw);
let (speaker, text) = split_voice(&unescape_text(&raw));
doc.cues.push(Cue { start, end, text, speaker, id, settings: settings.to_string() });
}
if !has_sig && doc.cues.is_empty() && !text.trim().is_empty() {
@@ -122,7 +133,7 @@ pub fn write(doc: &Document) -> String {
}
for c in &doc.cues {
if let Some(id) = c.id.as_deref().map(str::trim).filter(|i| !i.is_empty() && !i.contains("-->")) {
out.push_str(id);
out.push_str(&id.replace(['\r', '\n'], " "));
out.push('\n');
}
out.push_str(&format!("{} --> {}", format_clock(c.start, '.'), format_clock(c.end, '.')));
@@ -149,6 +160,7 @@ pub fn write(doc: &Document) -> String {
mod tests {
use super::*;
use crate::TICKS_PER_MS;
use filmcraft_time::Tick;
const SAMPLE: &str = "WEBVTT - with header\nKind: captions\n\nSTYLE\n::cue { color: yellow }\n\nNOTE a comment\nspanning lines\n\nintro\n00:01.000 --> 00:00:03.500 line:90% align:start\n<v.loud Ann Smith>Hello there\nsecond line\n\n00:00:04.000 --> 00:00:05.000\n<i>plain</i> &amp; tags\n";
@@ -179,6 +191,24 @@ mod tests {
assert_eq!(parse(&w).unwrap().cues, d.cues);
}
#[test]
fn newline_in_cue_id_round_trips() {
let mut doc = Document::default();
doc.cues.push(Cue {
start: crate::Tick(0),
end: crate::Tick(254_016_000),
text: "x".into(),
speaker: None,
id: Some("first\nsecond".into()),
settings: String::new(),
});
let output = write(&doc);
assert!(output.contains("first second\n00:00:00.000 --> 00:00:00.001\n"), "{output:?}");
assert!(!output.contains("first\nsecond"));
assert_eq!(parse(&output).unwrap().cues[0].id.as_deref(), Some("first second"));
}
#[test]
fn voices() {
assert_eq!(split_voice("<v Bob>hi</v>"), (Some("Bob".into()), "hi".into()));
@@ -213,4 +243,33 @@ mod tests {
assert_eq!(d.cues.len(), 1);
assert!(parse("just text").is_err());
}
/// #2066: the writer escapes a cue's `-->` as `--&gt;` (it would read as a timing line
/// otherwise), but the parser never decoded it, so `A --> B` came back as `A --&gt; B` on
/// every VTT round trip.
#[test]
fn an_arrow_in_cue_text_survives_a_round_trip() {
let mut d = Document::default();
// A second of ticks: `format_clock` writes milliseconds, so a sub-millisecond end would
// re-parse as `end == start` and the cue would be dropped as backwards.
let ms = TICKS_PER_MS;
d.cues.push(Cue { start: Tick(0), end: Tick(1000 * ms), text: "A --> B".into(), ..Cue::default() });
let w = write(&d);
assert!(w.contains("A --&gt; B"), "the writer still escapes it: {w}");
let back = parse(&w).unwrap();
assert_eq!(back.cues.len(), 1);
assert_eq!(back.cues[0].text, "A --> B", "the parser decodes the writer's escape");
assert_eq!(write(&back), w, "stable from there on");
}
/// Only the writer's own `-->` escape is reversed: cue markup keeps its entities verbatim,
/// as `parse` has always done.
#[test]
fn other_entities_stay_verbatim_in_cue_text() {
let d = parse(SAMPLE).unwrap();
assert_eq!(d.cues[1].text, "<i>plain</i> &amp; tags");
let w = write(&d);
assert!(w.contains("&amp; tags"), "not re-escaped on the way out: {w}");
assert_eq!(parse(&w).unwrap().cues[1].text, "<i>plain</i> &amp; tags");
}
}
crates/cfb/src/lib.rs
+6 -5
View File
@@ -102,15 +102,16 @@ fn upper(u: u16) -> u16 {
/// The sibling order of directory entries (§2.6.4): shorter names first, then a code-unit
/// comparison of the upper-cased names.
pub fn compare_names(a: &str, b: &str) -> Ordering {
let ua: Vec<u16> = a.encode_utf16().collect();
let ub: Vec<u16> = b.encode_utf16().collect();
ua.len().cmp(&ub.len()).then_with(|| ua.iter().map(|&u| upper(u)).cmp(ub.iter().map(|&u| upper(u))))
// No allocation: the iterators are walked in place (`child` calls this once per sibling).
let (la, lb) = (a.encode_utf16().count(), b.encode_utf16().count());
la.cmp(&lb).then_with(|| a.encode_utf16().map(upper).cmp(b.encode_utf16().map(upper)))
}
/// Check a directory entry name (§2.6.1): 1–31 UTF-16 code units, none of `/ \ : !`.
/// Check a directory entry name (§2.6.1): 1–31 UTF-16 code units, none of `/ \ : !`, and no NUL
/// (the stored name is NUL-terminated, so a NUL would cut the name short when read back).
pub fn validate_name(name: &str) -> Result<()> {
let n = name.encode_utf16().count();
if n == 0 || n > MAX_NAME_LEN || name.contains(['/', '\\', ':', '!']) {
if n == 0 || n > MAX_NAME_LEN || name.contains(['/', '\\', ':', '!', '\0']) {
return Err(Error::BadName(name.to_string()));
}
Ok(())
crates/cfb/src/read.rs
+26 -18
View File
@@ -1,6 +1,6 @@
//! Reading compound files ([MS-CFB] §2.2–2.6).
use crate::{ENDOFCHAIN, Error, FREESECT, MAXREGSECT, MINI_SECTOR_SIZE, NOSTREAM, Result, SIGNATURE, compare_names};
use crate::{ENDOFCHAIN, Error, MAXREGSECT, MINI_SECTOR_SIZE, NOSTREAM, Result, SIGNATURE, compare_names};
/// Directory entry object type (§2.6.1).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
@@ -26,7 +26,9 @@ pub struct Entry {
/// First sector (regular or mini) of a stream; the mini stream's for the root.
pub start: u32,
pub size: u64,
/// Child entries of a storage, in sibling-tree order (sorted by [`compare_names`]).
/// Child entries of a storage, in the in-order position of its sibling tree. For a well-formed
/// file that is sorted by [`compare_names`]; the reader does not validate the tree's order, so
/// a hostile file may give any order (lookups by name do not depend on it).
pub children: Vec<usize>,
left: u32,
right: u32,
@@ -59,6 +61,13 @@ fn u64_at(b: &[u8], at: usize) -> u64 {
u64::from_le_bytes(b[at..at + 8].try_into().unwrap_or([0; 8]))
}
/// Byte range `[start, end)` of unit `index + skip` of `unit` bytes each, or `None` when the
/// arithmetic overflows `usize` (reachable on 32-bit targets such as wasm32).
pub(crate) fn span(index: usize, skip: usize, unit: usize) -> Option<(usize, usize)> {
let start = index.checked_add(skip)?.checked_mul(unit)?;
Some((start, start.checked_add(unit)?))
}
fn invalid(s: impl Into<String>) -> Error {
Error::Invalid(s.into())
}
@@ -157,23 +166,25 @@ impl<'a> CompoundFile<'a> {
/// Sector `n` (a whole sector; a truncated last sector is an error).
fn sector(&self, n: u32) -> Result<&'a [u8]> {
let start = (n as usize + 1).checked_mul(self.sector_size).ok_or_else(|| invalid("sector number overflow"))?;
let end = start.checked_add(self.sector_size).ok_or_else(|| invalid("sector number overflow"))?;
if n > MAXREGSECT || end > self.data.len() {
if n > MAXREGSECT {
return Err(invalid(format!("sector {n} is outside the file")));
}
Ok(&self.data[start..end])
// sector 0 follows the header sector
let (start, end) = usize::try_from(n).ok().and_then(|i| span(i, 1, self.sector_size)).ok_or_else(|| invalid("sector number overflow"))?;
self.data.get(start..end).ok_or_else(|| invalid(format!("sector {n} is outside the file")))
}
/// Concatenate a regular sector chain; `limit` stops once that many bytes are read.
fn read_chain(&self, start: u32, limit: Option<usize>) -> Result<Vec<u8>> {
let mut out = Vec::new();
// `limit` is at most the file length (callers check), and an unlimited chain holds at most
// `max_steps` sectors, so this capacity is bounded by the file's own size.
let max_steps = self.data.len() / self.sector_size + 1;
let mut out = Vec::with_capacity(limit.unwrap_or(0).min(self.data.len()));
let mut s = start;
let mut steps = 0usize;
// A chain cannot visit more sectors than the file holds. Not `self.fat.len()`: the FAT can
// be far longer (one FAT sector named by many header / DIFAT slots), and a looping chain
// would then append a whole sector per FAT entry (hundreds of MB, or an abort).
let max_steps = self.data.len() / self.sector_size + 1;
while s != ENDOFCHAIN {
if s > MAXREGSECT {
return Err(invalid(format!("bad sector {s:#x} in a chain")));
@@ -187,10 +198,8 @@ impl<'a> CompoundFile<'a> {
if limit.is_some_and(|l| out.len() >= l) {
break;
}
// a free entry (`FREESECT`) is above `MAXREGSECT` and is rejected at the top of the loop
s = *self.fat.get(s as usize).ok_or_else(|| invalid(format!("sector {s} has no FAT entry")))?;
if s == FREESECT {
return Err(invalid("chain runs into a free sector"));
}
}
if let Some(l) = limit {
if out.len() < l {
@@ -213,8 +222,11 @@ impl<'a> CompoundFile<'a> {
if steps > self.minifat.len().max(1) {
return Err(invalid("mini sector chain loops"));
}
let at = s as usize * MINI_SECTOR_SIZE;
let b = self.mini_stream.get(at..at + MINI_SECTOR_SIZE).ok_or_else(|| invalid("mini sector outside the mini stream"))?;
let b = usize::try_from(s)
.ok()
.and_then(|i| span(i, 0, MINI_SECTOR_SIZE))
.and_then(|(start, end)| self.mini_stream.get(start..end))
.ok_or_else(|| invalid("mini sector outside the mini stream"))?;
out.extend_from_slice(b);
s = *self.minifat.get(s as usize).ok_or_else(|| invalid("mini sector has no mini FAT entry"))?;
}
@@ -234,7 +246,7 @@ impl<'a> CompoundFile<'a> {
// in-order walk of the sibling tree, iterative, cycle-safe
let mut stack: Vec<u32> = Vec::new();
let mut cur = self.entries[parent].child;
let mut steps = 0;
// terminates without a step counter: every push claims an unclaimed entry in `owner`
while cur != NOSTREAM || !stack.is_empty() {
while cur != NOSTREAM {
let i = cur as usize;
@@ -244,10 +256,6 @@ impl<'a> CompoundFile<'a> {
owner[i] = true;
stack.push(cur);
cur = self.entries[i].left;
steps += 1;
if steps > n {
return Err(invalid("directory tree loops"));
}
}
if let Some(top) = stack.pop() {
kids.push(top as usize);
crates/cfb/src/tests.rs
+89
View File
@@ -291,3 +291,92 @@ fn chains_filling_the_file_still_read() {
assert_eq!(CompoundFile::open(&b).unwrap().read_path("big").unwrap(), data);
}
}
#[test]
fn nul_in_a_name_is_rejected() {
assert!(validate_name("a\0b").is_err());
assert!(validate_name("\0").is_err());
let mut w = Writer::default();
assert!(matches!(w.stream(Writer::ROOT, "a\0b", vec![1]), Err(Error::BadName(_))));
assert!(matches!(w.storage(Writer::ROOT, "x\0", [0; 16]), Err(Error::BadName(_))));
}
#[test]
fn span_is_checked() {
use crate::read::span;
assert_eq!(span(0, 1, 512), Some((512, 1024)));
assert_eq!(span(3, 0, 64), Some((192, 256)));
// On a 32-bit `usize` these are the overflows of `sector` and `read_mini_chain`; the checked
// arithmetic is the same code on 64-bit, so usize::MAX exercises it here. (A real 32-bit
// target is not run.)
assert_eq!(span(usize::MAX, 1, 512), None);
assert_eq!(span(usize::MAX, 0, 64), None);
assert_eq!(span(usize::MAX / 64, 0, 64), None, "start fits, end does not");
assert_eq!(span(usize::MAX - 1, 1, 1), None);
}
/// The allocating comparison compare_names used before the iterator rewrite.
fn compare_names_old(a: &str, b: &str) -> std::cmp::Ordering {
let ua: Vec<u16> = a.encode_utf16().collect();
let ub: Vec<u16> = b.encode_utf16().collect();
ua.len().cmp(&ub.len()).then_with(|| ua.iter().map(|&u| upper(u)).cmp(ub.iter().map(|&u| upper(u))))
}
#[test]
fn compare_names_matches_the_allocating_version() {
let names = [
"",
"a",
"A",
"b",
"B",
"ab",
"AB",
"aB",
"abc",
"ABD",
"Abc",
"aBC",
"z",
"Z",
"{",
"_",
"ß",
"SS",
"ss",
"ü",
"Ü",
"é",
"É",
"dž",
"Dž",
"DŽ",
"ストリーム",
"ｓｔ",
"Ｓｔ",
"😀",
"😁",
"a😀",
"A😀",
"\u{ffff}",
"\u{10000}",
"Header-2",
"HEADER-2",
"header-2",
"Mobs-1901{a}",
"mobs-1901{A}",
];
for a in names {
for b in names {
assert_eq!(compare_names(a, b), compare_names_old(a, b), "{a:?} vs {b:?}");
}
}
// every BMP scalar against a few fixed names
for c in (0u32..=0x10FFFF).filter_map(char::from_u32) {
let s = c.to_string();
for other in ["a", "Z", "ü", "dž"] {
assert_eq!(compare_names(&s, other), compare_names_old(&s, other));
assert_eq!(compare_names(other, &s), compare_names_old(other, &s));
}
}
}
crates/codecs/src/audio.rs
+21
View File
@@ -788,6 +788,12 @@ pub fn decode_dvd_lpcm(data: &[u8], channels: usize, bits: u32) -> Vec<Vec<f32>>
/// Decode interleaved PCM bytes into planar f32.
pub fn decode_pcm(data: &[u8], cfg: &filmcraft_isobmff::PcmConfig) -> Vec<Vec<f32>> {
let ch = cfg.channels.max(1) as usize;
// A hostile `ipcm`/`fpcm` sample entry can declare `bits == 0` (`a.sample_size` is
// applied unchecked, unlike the `sowt`/`twos` arms of `qt_pcm` which default it): `bps`
// would be 0 and the length division below would panic. There are no samples to decode.
if cfg.bits == 0 {
return vec![Vec::new(); ch];
}
let bps = (cfg.bits as usize).div_ceil(8);
let n = data.len() / (bps * ch);
let mut out = vec![Vec::with_capacity(n); ch];
@@ -1417,3 +1423,18 @@ mod gapless_tests {
assert_eq!(src.samples[0].len(), AUDIO_FRAMES * 1152 - (576 + 529));
}
}
#[cfg(test)]
mod decode_pcm_tests {
use super::*;
#[test]
fn zero_bits_decode_to_empty_channels_without_panicking() {
// A hostile `ipcm`/`fpcm` sample entry with `sample_size == 0` used to divide by zero
// (`bps * ch == 0`) and panic the audio worker on import/playback.
let cfg = filmcraft_isobmff::PcmConfig { bits: 0, float: false, big_endian: true, signed: true, channels: 2, sample_rate: 48_000.0 };
let out = decode_pcm(&[0u8; 64], &cfg);
assert_eq!(out.len(), 2);
assert!(out.iter().all(Vec::is_empty));
}
}
crates/codecs/src/avi.rs
+13 -4
View File
@@ -194,10 +194,11 @@ fn split_frames(
let Some(here) = win.get(pos - base..) else { break };
match filmcraft_mpegts::frame_bytes(ts, here) {
Some(n) if pos + n <= total => {
let fi = filmcraft_mpegts::frame_info(ts, here);
if info.is_none() {
info = filmcraft_mpegts::frame_info(ts, here);
info = fi;
}
let k = filmcraft_mpegts::frame_info(ts, here).map_or(0, |f| f.samples);
let k = fi.map_or(0, |f| f.samples);
first.push(pieces.len());
let mut p = pos;
while p < pos + n {
@@ -234,7 +235,12 @@ impl AviSource {
pub fn open_reader(name: &str, reader: filmcraft_media::SharedReader) -> crate::Result<Self> {
let bytes = crate::Src(reader);
let file = filmcraft_avi::open(&bytes).map_err(|e| CodecError::Container(e.to_string()))?;
let vstream = file.streams.iter().position(|s| s.kind == StreamKind::Video && s.video.is_some() && !s.chunks.is_empty());
// A video stream that declares frames but whose chunks a damaged file lost is still the
// file's video when no other has pictures: its frames say so, rather than the file passing
// for audio only.
let video = |has: fn(&filmcraft_avi::Stream) -> bool| file.streams.iter().position(|s| s.kind == StreamKind::Video && s.video.is_some() && has(s));
let vstream = video(|s| !s.chunks.is_empty()).or_else(|| video(|s| s.length > 0));
let pictures = vstream.and_then(|i| file.streams.get(i)).is_some_and(|s| !s.chunks.is_empty());
let astreams: Vec<usize> = file
.streams
.iter()
@@ -243,7 +249,7 @@ impl AviSource {
.map(|(i, _)| i)
.take(filmcraft_media::MAX_AUDIO_STREAMS)
.collect();
if vstream.is_none() && astreams.is_empty() {
if !pictures && astreams.is_empty() {
return Err(CodecError::Unsupported("no playable streams".into()));
}
// video: frame slots, pictures and key frames
@@ -522,6 +528,9 @@ impl MediaSource for AviSource {
fn video_frame(&self, req: FrameRequest) -> Result<Arc<VideoFrame>, MediaError> {
let s = self.vstream.ok_or(MediaError::NoStream("video"))?;
if self.vframes.is_empty() {
return Err(MediaError::Decode("the video stream's frames are missing (a damaged file)".into()));
}
let t = req.time.max(Tick::ZERO);
let slot = |t: Tick| self.rate.frame_at(t);
let late = filmcraft_media::cancel::catch_up().map(|m| slot(t - m));
crates/codecs/src/avi_tests.rs
+55
View File
@@ -230,3 +230,58 @@ fn copied_h264_keeps_its_timing_with_a_fine_time_base() {
let mid = src.video_frame(FrameRequest::full(filmcraft_time::Tick::from_rational(1, 1, 50))).unwrap();
assert!(yuv_bytes(&mid) == reference[0..frame]);
}
/// The `idx1` entries of the video damaged, the audio's intact: the pictures come from the movie
/// data, exactly as from the undamaged file, and the sound still plays. With the video chunks gone
/// altogether the file is still a movie whose frames say it is damaged, not an audio file.
#[test]
fn damaged_video_index_entries_lose_no_pictures() {
let Some((path, good)) = avi("avi_mjpeg_pcm.avi", &["-c:v", "mjpeg", "-q:v", "3", "-pix_fmt", "yuvj420p"], &["-c:a", "pcm_s16le"]) else { return };
let bytes = std::fs::read(&path).unwrap();
let ix = bytes.windows(4).rposition(|w| w == b"idx1").unwrap() + 8;
let mut damaged = bytes.clone();
let mut hit = 0;
for e in damaged[ix..].as_chunks_mut::<16>().0.iter_mut().filter(|e| &e[..4] == b"00dc") {
e[8..12].copy_from_slice(&0xFFFF_FF00u32.to_le_bytes());
hit += 1;
}
assert_eq!(hit, 50, "every video entry");
let src = crate::open_bytes("damaged.avi", damaged.into()).unwrap();
check_info("damaged index", &src, "Motion JPEG", "PCM 16-bit");
let rate = src.info().frame_rate();
for k in [0, 31, 49, 7] {
let at = || FrameRequest::full(rate.tick_of(k));
assert!(src.video_frame(at()).unwrap().to_rgba8() == good.video_frame(at()).unwrap().to_rgba8(), "frame {k}");
}
assert_eq!(ours(&src, 4_800), ours(&good, 4_800));
let mut lost = bytes;
while let Some(i) = lost.windows(4).position(|w| w == b"00dc") {
lost[i..i + 4].copy_from_slice(b"JUNK");
}
let src = crate::open_bytes("lost.avi", lost.into()).unwrap();
assert_eq!(src.info().kind, filmcraft_media::MediaKind::Movie);
let e = src.video_frame(FrameRequest::full(filmcraft_time::Tick::ZERO)).unwrap_err().to_string();
assert!(e.contains("damaged"), "{e}");
assert_eq!(ours(&src, 4_800), ours(&good, 4_800));
}
/// Two video streams, the first declaring frames whose chunks are all gone: the second, which has
/// its pictures, is the file's video.
#[test]
fn a_video_stream_with_pictures_comes_before_an_empty_one() {
let mut args = PIC.to_vec();
args.extend(["-map", "0:v", "-map", "0:v", "-c:v", "mjpeg", "-q:v", "3", "-pix_fmt", "yuvj420p", "-f", "avi"]);
let Some(path) = fixture_path("avi_two_mjpeg.avi", &args) else { return };
let bytes = std::fs::read(&path).unwrap();
let mut lost = bytes.clone();
while let Some(i) = lost.windows(4).position(|w| w == b"00dc") {
lost[i..i + 4].copy_from_slice(b"JUNK");
}
let (good, src) = (crate::open_bytes("two.avi", bytes.into()).unwrap(), crate::open_bytes("lost.avi", lost.into()).unwrap());
let rate = src.info().frame_rate();
for k in [0, 31] {
let at = || FrameRequest::full(rate.tick_of(k));
assert!(src.video_frame(at()).unwrap().to_rgba8() == good.video_frame(at()).unwrap().to_rgba8(), "frame {k}");
}
}
crates/codecs/src/gop/tests.rs
+37 -6
View File
@@ -207,21 +207,52 @@ fn rayon_frame_requests_while_decoding_on_rayon_never_deadlock() {
let _ = tx.send(());
});
// A deadlock stops every request of a round for good. On a loaded machine the 3,840
// requests can take minutes, so the limit applies to the time between two answered ones.
let mut answered = 0;
// requests can take minutes, so the limit applies to the time since the last answered one.
if let Err(frames) = wait_while_answered(&rx, &done, Duration::from_secs(120)) {
panic!("frame requests deadlocked after {frames} frames");
}
}
/// Waits for `rx` while `done` keeps growing. Returns `Err` with the count in `done` once
/// `limit` has passed since the count last grew (or since the call, before it grows).
fn wait_while_answered(rx: &std::sync::mpsc::Receiver<()>, done: &AtomicUsize, limit: Duration) -> Result<(), usize> {
let mut answered = done.load(Ordering::Relaxed);
let mut last_answer = Instant::now();
loop {
match rx.recv_timeout(Duration::from_secs(120)) {
Ok(()) => break,
match rx.recv_timeout(limit / 120) {
Ok(()) => return Ok(()),
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {
let now = done.load(Ordering::Relaxed);
assert!(now > answered, "frame requests deadlocked after {now} frames");
answered = now;
if now > answered {
answered = now;
last_answer = Instant::now();
} else if last_answer.elapsed() >= limit {
return Err(now);
}
}
Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => panic!("a frame request failed"),
}
}
}
#[test]
fn a_stall_right_after_an_answer_is_reported_once_the_limit_has_passed() {
let (_tx, rx) = std::sync::mpsc::channel();
let done = Arc::new(AtomicUsize::new(0));
let answering = done.clone();
let (answer_tx, answer_rx) = std::sync::mpsc::channel();
let limit = Duration::from_secs(2);
std::thread::spawn(move || {
std::thread::sleep(Duration::from_millis(50));
let answer = Instant::now();
answering.fetch_add(1, Ordering::Relaxed);
let _ = answer_tx.send(answer);
});
assert_eq!(wait_while_answered(&rx, &done, limit), Err(1));
let waited = answer_rx.recv().expect("answer time").elapsed();
assert!(waited >= limit && waited < limit * 3 / 2, "the stall was reported {waited:?} after the answer");
}
#[test]
fn running_decoder_continues_through_a_nearby_sync_sample() {
let s = samples(200, 30, 0, false);
crates/codecs/src/mkv.rs
+18 -2
View File
@@ -156,7 +156,9 @@ fn audio_info(t: &filmcraft_matroska::Track) -> Option<AudioStreamInfo> {
channels = h.channels as u32;
}
let (rate, channels) = filmcraft_media::bounded_audio_dimensions(rate, channels).ok()?;
Some(AudioStreamInfo { sample_rate: rate, channels, codec: codec_label(&t.codec), bits_per_sample: a.bit_depth.map(|b| b as u32) })
// an audio codec without a decoder plays silent: say so in the media info, as MPEG / MXF do (#1422)
let codec = if audio_decodable(&t.codec) { codec_label(&t.codec) } else { format!("{} (unsupported)", codec_label(&t.codec)) };
Some(AudioStreamInfo { sample_rate: rate, channels, codec, bits_per_sample: a.bit_depth.map(|b| b as u32) })
}
/// The track timestamp that media time zero stands for (#715): the earliest start among the
@@ -338,10 +340,24 @@ fn codec_label(c: &Codec) -> String {
Codec::Mp2 | Codec::Mp1 => "MPEG Audio".into(),
Codec::Other(id) if id == "V_MPEG2" => "MPEG-2 Video".into(),
Codec::Other(id) if id == "V_MPEG1" => "MPEG-1 Video".into(),
Codec::Other(id) if id.starts_with("A_DTS") => "DTS".into(),
Codec::Other(id) if id == "A_TRUEHD" => "Dolby TrueHD".into(),
other => other.name().to_string(),
}
}
/// Whether the audio of codec `c` decodes: PCM directly, the rest through
/// [`MkvSource::audio_decoder`] (keep the two in step).
fn audio_decodable(c: &Codec) -> bool {
match c {
Codec::Pcm { .. } | Codec::Aac { .. } | Codec::Opus { .. } | Codec::Mp3 | Codec::Mp2 | Codec::Mp1 | Codec::Ac3 | Codec::Eac3 | Codec::Flac { .. } => {
true
}
Codec::Vorbis { headers } => headers.len() == 3,
_ => false,
}
}
impl MkvSource {
pub fn open(name: &str, bytes: Arc<[u8]>) -> crate::Result<Self> {
Self::open_reader(name, Arc::new(filmcraft_media::reader::MemReader(bytes)))
@@ -507,7 +523,7 @@ impl MkvSource {
extra.extend_from_slice(&headers[2]);
PacketDecoder::new(CODEC_TYPE_VORBIS, rate, Some(extra))
}
other => Err(CodecError::Unsupported(format!("{} audio", codec_label(other)))),
other => Err(CodecError::Unsupported(format!("{} audio (FilmCraft has no decoder for it)", codec_label(other)))),
}
}
crates/codecs/src/mp4.rs
+101 -20
View File
@@ -9,7 +9,7 @@ use std::sync::{Arc, Mutex};
use filmcraft_color::ColorInfo;
use filmcraft_frame::{AudioBuffer, Region, VideoFrame};
use filmcraft_isobmff::{ByteSource, CleanAperture, CodecConfig, Mp4File, TrackKind};
use filmcraft_isobmff::{ByteSource, CleanAperture, CodecConfig, Mp4File, Sample, TrackKind};
use filmcraft_media::{AudioStreamInfo, FrameRequest, MediaError, MediaInfo, MediaKind, MediaSource, SharedSource, VideoStreamInfo};
use filmcraft_time::{FrameRate, Tick};
@@ -26,6 +26,30 @@ struct DecodedAudioPacket {
failed: bool,
}
/// How far a track's nominal rate may sit from a standard rate and still be read as it. Timestamps
/// rounded to a coarse unit (whole milliseconds in a 90 kHz track, as phones and messaging apps
/// write them) move the average by less than this at every standard rate (#1700).
const STANDARD_RATE_TOLERANCE: f64 = 0.005;
/// A video track's nominal frame rate: the mean of the sample durations within a quarter frame of
/// the median duration, read as the closest standard rate when close enough.
///
/// A duration near the median is the same frame period as rounding has quantised it; one much
/// longer belongs to a variable-rate frame and stays out of the mean. The mean sees through a
/// millisecond rounder the median cannot: 30 fps over 90 kHz is 2970 and 3060 units in turn, whose
/// median reads 30.303 fps but whose mean is 30 fps (#1700).
fn nominal_rate(timescale: u32, samples: &[Sample]) -> Option<FrameRate> {
let mut durs: Vec<u32> = samples.iter().map(|s| s.duration).filter(|&d| d > 0).collect();
durs.sort_unstable();
let &median = durs.get(durs.len() / 2)?;
let near: Vec<f64> = durs.iter().filter(|&&d| d.abs_diff(median) * 4 <= median).map(|&d| f64::from(d)).collect();
let mean = near.iter().sum::<f64>() / near.len().max(1) as f64;
if mean <= 0.0 {
return None;
}
Some(FrameRate::from_f64_within(f64::from(timescale) / mean, STANDARD_RATE_TOLERANCE).sane())
}
struct AudioState {
decoder: Option<PacketDecoder>,
/// Decoded packets by sample index.
@@ -266,13 +290,8 @@ impl Mp4Source {
};
aperture = vp.clean_aperture.and_then(|c| clean_aperture(c, (w, h)));
let (w, h) = aperture.map_or((w, h), |r| (r.w as u32, r.h as u32));
// frame rate from the median sample duration
let mut durs: Vec<u32> = t.samples.iter().take(240).map(|s| s.duration).collect();
durs.sort_unstable();
let d = durs.get(durs.len() / 2).copied().unwrap_or(1).max(1);
let rate = FrameRate::from_f64(t.timescale as f64 / d as f64);
// A rate that rounds to zero (one sample per huge duration) can't be divided by.
let rate = if rate.num > 0 && rate.den > 0 { rate } else { FrameRate::default() };
let rate = nominal_rate(t.timescale, &t.samples).unwrap_or_default();
color = color_from(entry, w, h);
if entry
.video
@@ -560,6 +579,66 @@ impl MediaSource for Mp4Source {
}
}
/// Channel layout and sample rate metadata for AC-3 / E-AC-3 in ISO-BMFF. The sample-entry channel
/// count is required to be two, but the real layout is in `dac3` / `dec3` (ETSI TS 102 366 Annex F).
/// Without a usable descriptor, a primary syncframe header is a safe fallback.
fn ac3_stream_info(t: &filmcraft_isobmff::Track, bytes: &crate::Src) -> Option<(u32, u32)> {
let entry = t.entries.first()?;
if let Some((rate, channels)) = match &entry.codec {
CodecConfig::Ac3 { dac3 } => dac3_layout(dac3),
CodecConfig::Eac3 { dec3 } => dec3_layout(dec3),
_ => None,
} {
return Some((rate, channels));
}
let data = t.samples.first().and_then(|s| filmcraft_media::reader::read_range(&*bytes.0, s.offset, s.size as usize).ok());
let h = filmcraft_ac3::parse_header(data.as_deref()?).ok()?;
// A dependent E-AC-3 substream doesn't describe the full program; report only if the first
// packet is the primary independent substream.
let is_primary = if let CodecConfig::Eac3 { .. } = &entry.codec { h.is_primary() } else { true };
if !is_primary {
return None;
}
let rate = h.sample_rate;
let channels = u32::try_from(h.channels()).ok()?;
(rate > 0 && channels > 0).then_some((rate, channels))
}
/// E-AC-3 `ec-3` entry: the program-layout fields are the 24 bits starting at byte 2 of the
/// descriptor, then `bsmod(5) | acmod(3) | lfeon(1)`. Matches FFmpeg's `dec3` demuxer.
fn dec3_layout(payload: &[u8]) -> Option<(u32, u32)> {
if payload.len() < 5 {
return None;
}
let bits = u32::from(payload[2]) << 16 | u32::from(payload[3]) << 8 | u32::from(payload[4]);
let acmod = ((bits >> 9) & 0x7) as u8;
let lfeon = bits & 0x100 != 0;
let channels = dac3_channels(acmod, lfeon)?;
Some((0, channels))
}
/// AC-3 `ac-3` entry: `dac3` begins with the syncframe bitstream information: sample rate code,
/// bit stream id, bit stream mode, channel mode and LFE flag. The descriptor has no sample rate,
/// so the caller keeps the sample-entry rate.
fn dac3_layout(payload: &[u8]) -> Option<(u32, u32)> {
if payload.len() < 3 {
return None;
}
// The descriptor begins with the bitstream information: fscod(2), bsid(5), bsmod(3),
// acmod(3) and lfeon(1), continuing in the second payload byte.
let bits = u32::from_be_bytes([0, 0, payload[1], payload[2]]);
let acmod = ((bits >> 11) & 0x7) as u8;
let lfeon = bits & 0x400 != 0;
let channels = dac3_channels(acmod, lfeon)?;
Some((0, channels))
}
/// Full-bandwidth channels per AC-3 channel mode (Table 5.8) plus the optional LFE channel.
fn dac3_channels(acmod: u8, lfeon: bool) -> Option<u32> {
let n = [2u32, 1, 2, 3, 3, 4, 4, 5].get(usize::from(acmod & 7))?.checked_add(u32::from(lfeon))?;
(n > 0).then_some(n)
}
/// What the file says about one audio track (`None`: the track has no sample description).
fn audio_stream_info(t: &filmcraft_isobmff::Track, bytes: &crate::Src) -> Option<AudioStreamInfo> {
let entry = t.entries.first()?;
@@ -587,6 +666,17 @@ fn audio_stream_info(t: &filmcraft_isobmff::Track, bytes: &crate::Src) -> Option
),
CodecConfig::Pcm(p) => (p.sample_rate as u32, p.channels, Some(p.bits as u32)),
CodecConfig::Opus(o) => (crate::audio::OPUS_RATE, (o.output_channels as u32).max(1), None),
CodecConfig::Ac3 { .. } | CodecConfig::Eac3 { .. } => match ac3_stream_info(t, bytes) {
// A syncframe fallback names the sample rate; a descriptor alone does not, so keep
// the sample-entry rate, which the muxer must supply.
Some((rate, ch)) if rate > 0 => (rate, ch, None),
// A partial descriptor can still say the exact number of channels even without
// enough bytes for the sample-rate field.
Some((_, ch)) => (if ap.sample_rate > 0.0 { ap.sample_rate as u32 } else { t.timescale }, ch, None),
// Without a descriptor, a normal primary-frame fallback gives both fields.
None if ap.sample_rate > 0.0 => (ap.sample_rate as u32, ap.channels.max(1), None),
None => (t.timescale, ap.channels.max(1), None),
},
_ => (if ap.sample_rate > 0.0 { ap.sample_rate as u32 } else { t.timescale }, ap.channels.max(1), None),
};
let codec = match &entry.codec {
@@ -744,19 +834,10 @@ pub fn opener(name: &str, bytes: Arc<[u8]>) -> Option<Result<SharedSource, Media
fn snap_to_frame_grid(file: &mut Mp4File, vtrack: Option<usize>) {
let Some(t) = vtrack.and_then(|i| file.tracks.get_mut(i)) else { return };
let Some(first) = t.samples.iter().map(|s| s.pts).min() else { return };
// The nominal duration: the median sample duration, averaged with the durations one unit from
// it. Those are the same duration rounded to the timescale (1501 and 1502 at 59.94 fps over
// 90 kHz): the median alone drifts half a unit a frame, enough to snap a short clip onto the
// wrong points.
let mut durs: Vec<u32> = t.samples.iter().map(|s| s.duration).collect();
durs.sort_unstable();
let Some(&median) = durs.get(durs.len() / 2) else { return };
let near: Vec<f64> = durs.iter().filter(|&&d| d.abs_diff(median) <= 1).map(|&d| f64::from(d)).collect();
let mean = near.iter().sum::<f64>() / near.len().max(1) as f64;
if mean <= 0.0 {
return;
}
let rate = FrameRate::from_f64(f64::from(t.timescale) / mean);
// The nominal duration: the mean of the samples near the median, with the rate snapped to its
// standard form when rounding moved it. Whole-millisecond timestamps turn 30 fps into 33/34 ms
// durations; the median alone picks either side, while the recovered mean keeps the whole grid.
let Some(rate) = nominal_rate(t.timescale, &t.samples) else { return };
// one frame lasts `step_num / step_den` track units
let (step_num, step_den) = (i128::from(t.timescale) * i128::from(rate.den), i128::from(rate.num));
if step_num <= 0 || step_den <= 0 {
crates/codecs/src/rounded_pts_tests.rs
+31
View File
@@ -187,6 +187,37 @@ fn mp4_durations(ts: u32, durations: &[u32]) -> Arc<[u8]> {
w.finish().unwrap().into_inner().into()
}
/// A clip whose frame times were rounded to whole milliseconds inside a 90 kHz track (#1700). A
/// 30 fps clip's durations are 2970 and 3060 units in turn, and a 60 fps clip's 1440 and 1530; the
/// old median reads the first as 30.303 fps and the second as 58.824 fps. Both mean to 3000/1500.
fn mp4_ms_rounded(rate: FrameRate) -> Arc<[u8]> {
let mut w = Mp4Writer::new(Cursor::new(Vec::new()), WriterOptions::new(Brand::Mov)).unwrap();
let t = w.add_track(TrackConfig::new(SampleEntry::jpeg((BITS * BLOCK) as u16, BLOCK as u16), 90_000)).unwrap();
let n = frame_count(rate);
let nominal_scale = i128::from(rate.num) / i128::from(rate.den);
let grid = 90_000 / nominal_scale as i64;
let stamp = |k: usize| ((2 * k as i64 * grid + 90) / 180) * 90;
for k in 0..n {
w.write_sample(t, WriteSample { data: &picture(k), duration: (stamp(k + 1) - stamp(k)) as u32, composition_offset: 0, is_sync: true }).unwrap();
}
w.finish().unwrap().into_inner().into()
}
/// A phone-style 90 kHz track with whole-millisecond frame times keeps its rate and every frame.
#[test]
fn mp4_ms_rounded_frame_times_get_back_to_a_standard_rate() {
let mut failures = Vec::new();
for rate in [FrameRate::FPS_30, FrameRate::FPS_60] {
let src = crate::open_bytes("synthetic.mov", mp4_ms_rounded(rate)).unwrap();
let got = src.info().video.as_ref().map(|v| v.frame_rate);
if got != Some(rate) {
failures.push(format!("{rate}: reads as {got:?}"));
}
failures.extend(problems(src.as_ref(), rate).into_iter().map(|p| format!("{rate}: {p}")));
}
assert!(failures.is_empty(), "mov:\n{}", failures.join("\n"));
}
/// Files that are not a constant rate gone slightly off its grid keep their own times: whatever is
/// asked (each frame's start, the unit before it, every 30 fps grid point), the frame shown is the
/// last one starting at or before it.
crates/codecs/src/tests.rs
+13
View File
@@ -263,6 +263,19 @@ fn quicktime_uncompressed_video() {
}
}
/// DTS has no decoder: the stream says so in the media info instead of looking like a normal,
/// silent track (#1422); decodable codecs keep their plain label.
#[test]
fn matroska_undecodable_audio_is_labelled_unsupported() {
let tone = ["-f", "lavfi", "-i", "sine=frequency=1000:sample_rate=48000:d=1", "-ac", "2"];
for (name, codec, want) in [("tone_dts.mkv", "dca", "DTS (unsupported)"), ("tone_aac.mkv", "aac", "AAC")] {
let args: Vec<&str> = tone.iter().copied().chain(["-c:a", codec, "-strict", "-2"]).collect();
let Some(b) = fixture(name, &args) else { return };
let src = crate::open_bytes(name, b).expect("open");
assert_eq!(src.info().audio_streams.first().map(|a| a.codec.as_str()), Some(want), "{name}");
}
}
#[test]
fn matroska_prores_pcm() {
check_mkv("blue_prores_pcm.mkv", &["-c:v", "prores_ks", "-profile:v", "2"], &["-c:a", "pcm_s16le"]);
crates/codecs/src/video.rs
+13 -5
View File
@@ -342,10 +342,10 @@ impl VideoDecoder for H264Decoder {
self.dec.flush().into_iter().map(Self::convert).collect()
}
fn reset(&mut self) {
if let Ok(d) = Self::make_inner(&self.avcc, self.threads) {
self.dec = d;
self.dec.reset();
if !self.avcc.is_empty() {
let _ = self.dec.configure_avcc(&self.avcc);
}
self.dec.set_draft(self.draft);
}
fn name(&self) -> &str {
"FilmCraft H.264"
@@ -364,8 +364,16 @@ impl VideoDecoder for H264Decoder {
h264_disposable(sample, self.length_size)
}
fn is_random_access(&self, sample: &[u8]) -> Option<bool> {
// Annex B samples carry their parameter sets: an IDR access unit is a starting point.
(self.length_size == 0).then(|| filmcraft_bitstream::annexb_nals(sample).iter().any(|n| n.first().is_some_and(|h| h & 0x1f == 5)))
// Annex B samples carry their parameter sets: an IDR access unit is a starting point, and
// so is a non-IDR one with a recovery point SEI of recovery_frame_cnt 0 (open GOP), the
// same rule as the transport stream splitter's `key`.
(self.length_size == 0).then(|| {
filmcraft_bitstream::annexb_nals(sample).iter().any(|n| match n.first().map(|h| h & 0x1f) {
Some(5) => true,
Some(6) => filmcraft_mpegts::sei_has_recovery_point(n),
_ => false,
})
})
}
}
crates/codecs/tests/it/ac3_mp4_layout.rs
+95
View File
@@ -0,0 +1,95 @@
//! MP4 AC-3 / E-AC-3 channel layout against ffmpeg (#1701). ISO-BMFF mandates a stereo channel
//! count in the sample entry for both codecs; the real layout is in the `dac3` / `dec3` descriptor.
//! Each channel carries a distinct tone, so a dropped centre/surround sample fails the oracle.
use crate::common;
use std::fs;
use std::io::Cursor;
use common::*;
use filmcraft_isobmff::{Brand, CodecConfig, Mp4Writer, TrackConfig, WriteSample, WriterOptions};
const RATE: u32 = 48_000;
fn test_file(ff: &std::path::Path, codec: &str) -> std::path::PathBuf {
fixture(
ff,
&format!("layout_51_{codec}.mp4"),
&[
"-f",
"lavfi",
"-i",
"aevalsrc=0.25*sin(2*PI*300*t)|0.25*sin(2*PI*400*t)|0.25*sin(2*PI*500*t)|0.25*sin(2*PI*60*t)|0.25*sin(2*PI*700*t)|0.25*sin(2*PI*800*t):s=48000:d=0.5:c=5.1",
"-c:a",
codec,
"-b:a",
"384k",
],
)
.unwrap_or_else(|| panic!("ffmpeg can't encode {codec}"))
}
fn rewrap(path: &std::path::Path, codec: &str) -> Vec<u8> {
let data = fs::read(path).unwrap();
let file = filmcraft_isobmff::open(data.as_slice()).unwrap();
let track = file.tracks.iter().find(|t| t.kind == filmcraft_isobmff::TrackKind::Audio).unwrap();
let mut entry = track.entries[0].clone();
entry.audio.as_mut().unwrap().channels = 2;
if codec == "fallback"
&& let CodecConfig::Ac3 { dac3 } = &mut entry.codec
{
dac3.clear();
}
let mut mux = Mp4Writer::new(Cursor::new(Vec::new()), WriterOptions::new(Brand::Mp4)).unwrap();
let ti = mux.add_track(TrackConfig::new(entry, track.timescale)).unwrap();
for s in &track.samples {
let au = data[s.offset as usize..s.offset as usize + s.size as usize].to_vec();
mux.write_sample(ti, WriteSample { data: &au, duration: s.duration, composition_offset: 0, is_sync: true }).unwrap();
}
mux.finish().unwrap().into_inner()
}
#[test]
fn ac3_and_eac3_mp4_report_and_decode_all_six_channels() {
let ff = filmcraft_testkit::require_ffmpeg!();
for codec in ["ac3", "eac3"] {
let path = test_file(&ff, codec);
let data = bytes(&path);
let source = filmcraft_codecs::open_bytes(&format!("{codec}.mp4"), data.clone()).unwrap();
let info = source.info().audio().unwrap();
assert_eq!(info.sample_rate, RATE, "{codec} sample rate");
assert_eq!(info.channels, 6, "{codec} channels");
let want = ffmpeg_audio_f32(&ff, &path, &[]);
let ours = source.audio(0, want.len() / 6, RATE).unwrap().channels;
assert_eq!(ours.len(), 6, "{codec} decoded channels");
// Skip decoder warm-up, then require a strong oracle match in every channel (including
// centre/dialogue, LFE and both surrounds).
for (c, channel) in ours.iter().enumerate() {
let (mut signal, mut error) = (0.0_f64, 0.0_f64);
for i in 1024..channel.len().saturating_sub(1024) {
let reference = f64::from(want[i * 6 + c]);
signal += reference * reference;
error += (f64::from(channel[i]) - reference).powi(2);
}
let snr = 10.0 * (signal / error.max(1e-30)).log10();
assert!(snr > 20.0, "{codec} channel {c}: {snr:.1} dB");
}
}
}
#[test]
fn descriptor_layout_survives_the_mandated_stereo_entry() {
let ff = filmcraft_testkit::require_ffmpeg!();
let out = rewrap(&test_file(&ff, "ac3"), "ac3");
let src = filmcraft_codecs::open_bytes("rewrapped-ac3.mp4", out.into()).unwrap();
assert_eq!(src.info().audio().unwrap().channels, 6);
}
#[test]
fn missing_descriptor_falls_back_to_a_primary_frame() {
let ff = filmcraft_testkit::require_ffmpeg!();
let out = rewrap(&test_file(&ff, "ac3"), "fallback");
let src = filmcraft_codecs::open_bytes("fallback-ac3.mp4", out.into()).unwrap();
assert_eq!(src.info().audio().unwrap().channels, 6);
}
crates/codecs/tests/it/main.rs
+1
View File
@@ -1,4 +1,5 @@
mod aac_channels_oracle;
mod ac3_mp4_layout;
mod adts_file;
mod common;
mod draft_decoders;
crates/codecs/tests/it/mpeg_oracle.rs
+87
View File
@@ -7,6 +7,7 @@
use crate::common;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use common::*;
use filmcraft_codecs::MpegSource;
@@ -33,6 +34,29 @@ fn spec(name: &str) -> Vec<String> {
NOISE,
&["-t", "1.2", "-c:v", "libx264", "-bf", "2", "-g", "12", "-pix_fmt", "yuv420p", "-c:a", "aac", "-aac_pns", "0", "-b:a", "128k", "-f", "mpegts"],
]),
// open GOPs: after the first picture every I picture is a non-IDR one with a recovery point SEI
"ts_h264_opengop.ts" => cat(&[
&["-f", "lavfi", "-i", "testsrc2=size=320x240:rate=25"],
&[
"-t",
"2.4",
"-c:v",
"libx264",
"-bf",
"2",
"-pix_fmt",
"yuv420p",
"-x264-params",
"open-gop=1:keyint=12:min-keyint=12:scenecut=0",
"-f",
"mpegts",
],
]),
// gradual refresh: no I pictures, a recovery point SEI starts each refresh wave
"ts_h264_intra_refresh.ts" => cat(&[
&["-f", "lavfi", "-i", "testsrc2=size=320x240:rate=25"],
&["-t", "2.4", "-c:v", "libx264", "-bf", "0", "-pix_fmt", "yuv420p", "-x264-params", "intra-refresh=1:keyint=12:min-keyint=12", "-f", "mpegts"],
]),
// AVCHD-style BDAV (192-byte packets): H.264 + Blu-ray LPCM
"m2ts_h264_lpcm.m2ts" => cat(&[
&["-f", "lavfi", "-i", "testsrc2=size=640x360:rate=30000/1001"],
@@ -161,6 +185,8 @@ fn spec(name: &str) -> Vec<String> {
pub const ALL: &[&str] = &[
"ts_mpeg2_mp2.ts",
"ts_h264_aac.ts",
"ts_h264_opengop.ts",
"ts_h264_intra_refresh.ts",
"m2ts_h264_lpcm.m2ts",
"m2ts_h264_ac3.mts",
"ts_hevc_latm.ts",
@@ -392,6 +418,67 @@ fn transport_stream_h264_and_aac() {
println!("ts_h264_aac: AAC max error {e:.2e}");
}
/// Open-GOP H.264 (non-IDR I pictures with recovery point SEI): every GOP start is a random-access
/// point, and a stream cut to start at one (no IDR at all) opens and gives the frames of the full
/// stream's decode from there.
#[test]
fn transport_stream_h264_recovery_points() {
let ff = filmcraft_testkit::require_ffmpeg!();
let f = make(&ff, "ts_h264_opengop.ts");
check_video(&ff, &f, 0, 12);
let data = bytes(&f);
let idx = filmcraft_mpegts::open(&data[..]).unwrap();
let vi = idx.streams.iter().position(|s| s.codec == filmcraft_mpegts::Codec::H264).unwrap();
let keys: Vec<usize> = (0..idx.streams[vi].units.len()).filter(|&i| idx.streams[vi].units[i].key).collect();
assert!(keys.len() >= 4, "{} key units in {} (IDR only: 1)", keys.len(), idx.streams[vi].units.len());
// cut at the third random-access point (the packet that starts its PES), keeping the PAT /
// PMT / SDT packets in front
let head = idx.read_unit(&data[..], vi, keys[2]).unwrap();
let hit = data.windows(100).position(|w| w == &head[..100]).unwrap();
let at = hit / 188 * 188;
assert!(data[at] == 0x47 && data[at + 1] & 0x40 != 0 && at > 3 * 188, "packet at {at}");
let cut: Arc<[u8]> = [&data[..3 * 188], &data[at..]].concat().into();
let full = open(&f);
let cut = MpegSource::open("cut.ts", cut).unwrap();
let rate = full.info().frame_rate();
let (n, m) = (rate.frame_at(full.info().duration), rate.frame_at(cut.info().duration));
assert!(m > 0 && m < n, "{m} of {n} frames");
let frame_of = |s: &MpegSource, i: i64| planes(&s.video_frame(FrameRequest::full(rate.tick_of(i))).unwrap());
// leading pictures of the cut I picture are not shown; the rest equals the full decode
for j in 0..m {
assert!(frame_of(&cut, j) == frame_of(&full, n - m + j), "cut frame {j}");
}
// seeking inside the cut stream
let mut rng = Rng(0xC07);
let fresh = MpegSource::open("cut.ts", [&data[..3 * 188], &data[at..]].concat().into()).unwrap();
for _ in 0..8 {
let j = rng.below(m as u64) as i64;
assert!(frame_of(&fresh, j) == frame_of(&full, n - m + j), "seek to cut frame {j}");
}
}
/// Gradual decoder refresh (recovery_frame_cnt > 0): its recovery points are not key units, so a
/// stream cut mid-refresh is refused cleanly.
#[test]
fn transport_stream_h264_intra_refresh_cut_is_refused() {
let ff = filmcraft_testkit::require_ffmpeg!();
let f = make(&ff, "ts_h264_intra_refresh.ts");
let data = bytes(&f);
let idx = filmcraft_mpegts::open(&data[..]).unwrap();
let vi = idx.streams.iter().position(|s| s.codec == filmcraft_mpegts::Codec::H264).unwrap();
// recovery points with recovery_frame_cnt > 0 are not random access points: only the IDR is key
let n = idx.streams[vi].units.len();
let keys: Vec<usize> = (0..n).filter(|&i| idx.streams[vi].units[i].key).collect();
assert_eq!(keys, [0], "key units in {n}");
let head = idx.read_unit(&data[..], vi, n / 2).unwrap();
let hit = data.windows(100).position(|w| w == &head[..100]).unwrap();
let at = hit / 188 * 188;
assert!(data[at] == 0x47 && data[at + 1] & 0x40 != 0 && at > 3 * 188, "packet at {at}");
// a cut inside the refresh wave has no safe start picture: refused, never a panic
let cut = MpegSource::open("cut.ts", [&data[..3 * 188], &data[at..]].concat().into());
assert!(cut.is_err(), "a stream cut mid-refresh has no random-access picture");
}
#[test]
fn bdav_h264_with_lpcm_sample_exact() {
let ff = filmcraft_testkit::require_ffmpeg!();
crates/codecs/tests/mpeg_opengop_seek.rs
+47
View File
@@ -0,0 +1,47 @@
//! Seeking in an open-GOP H.264 transport stream restarts the decoder at the recovery point I
//! picture of the wanted GOP, not at the IDR at the start of the file. This file has its own
//! process so that the global GOP counters are not shared with other tests.
#[path = "it/common/mod.rs"]
mod common;
use common::*;
use filmcraft_codecs::MpegSource;
use filmcraft_media::{FrameRequest, MediaSource};
#[test]
fn seek_restarts_at_recovery_point_not_at_idr() {
let ff = filmcraft_testkit::require_ffmpeg!();
// same fixture as `mpeg_oracle`: after the first picture every I picture is a non-IDR one
// with a recovery point SEI, one per 12 pictures
let args = [
"-f",
"lavfi",
"-i",
"testsrc2=size=320x240:rate=25",
"-t",
"2.4",
"-c:v",
"libx264",
"-bf",
"2",
"-pix_fmt",
"yuv420p",
"-x264-params",
"open-gop=1:keyint=12:min-keyint=12:scenecut=0",
"-f",
"mpegts",
];
let f = fixture(&ff, "ts_h264_opengop.ts", &args).expect("fixture");
let src = MpegSource::open("ts_h264_opengop.ts", bytes(&f)).unwrap();
let rate = src.info().frame_rate();
let n = rate.frame_at(src.info().duration);
assert!(n >= 48, "{n} frames");
// a frame in the last GOP: from the IDR it is 48+ samples away, from its recovery key at most 12
let g0 = filmcraft_codecs::gop_stats();
src.video_frame(FrameRequest::full(rate.tick_of(n - 3))).unwrap();
let g1 = filmcraft_codecs::gop_stats();
let (seeks, decoded) = (g1.seeks - g0.seeks, g1.decoded - g0.decoded);
assert_eq!(seeks, 1, "one decoder restart");
assert!(decoded <= 30, "{decoded} samples decoded for frame {} of {n}: the seek went back to the IDR", n - 3);
}
crates/dnx/tests/it/common/mod.rs
+2 -2
View File
@@ -49,7 +49,7 @@ pub fn make(ff: &Path, spec: &Spec) -> PathBuf {
let t = tmp.to_str().unwrap().to_string();
args.push(&t);
run(ff, &args);
std::fs::rename(&tmp, &path).unwrap();
filmcraft_testkit::fixtures::publish(&tmp, &path).unwrap();
}
path
}
@@ -278,7 +278,7 @@ pub fn source_frames(ff: &Path, spec: SourceFrameSpec<'_>) -> Vec<Frame> {
let tmp = filmcraft_testkit::temp_path(&path);
let f = frames.to_string();
run(ff, &["-f", "lavfi", "-i", lavfi, "-frames:v", &f, "-f", "rawvideo", "-pix_fmt", pix, tmp.to_str().unwrap()]);
std::fs::rename(&tmp, &path).unwrap();
filmcraft_testkit::fixtures::publish(&tmp, &path).unwrap();
}
let bytes = std::fs::read(&path).unwrap();
let s: Vec<u16> = if depth == 8 {
crates/edit/src/lib.rs
+27 -6
View File
@@ -176,13 +176,16 @@ pub fn track_range_empty(track: &Track, range: TimeRange) -> bool {
!track.items.iter().any(|i| i.range().overlaps(&range))
}
/// Drop transitions whose clips no longer exist or are no longer adjacent to them.
/// Drop transitions whose clips no longer exist or are no longer adjacent to them (a transition
/// between two clips must sit on their cut: the outgoing clip's end = the incoming clip's start).
/// A one-sided fade needs only its clip. `Session::edit` runs this on every sequence an edit
/// changed, so commands need not call it themselves.
pub fn remove_orphan_transitions(track: &mut Track) {
let ids: HashMap<ClipId, (Tick, Tick)> = track.items.iter().map(|i| (i.id, (i.start, i.end()))).collect();
track.transitions.retain(|tr| {
let from_ok = tr.from.is_none_or(|f| ids.contains_key(&f));
let to_ok = tr.to.is_none_or(|t| ids.contains_key(&t));
from_ok && to_ok && (tr.from.is_some() || tr.to.is_some())
track.transitions.retain(|tr| match (tr.from, tr.to) {
(Some(f), Some(t)) => matches!((ids.get(&f), ids.get(&t)), (Some(&(_, end)), Some(&(start, _))) if end == start),
(Some(c), None) | (None, Some(c)) => ids.contains_key(&c),
(None, None) => false,
});
}
@@ -516,6 +519,9 @@ pub fn ripple_delete_items(seq: &mut Sequence, items: &[ClipId]) -> Result<Vec<T
}
shift_track_from(tr, span.end(), -span.duration);
}
for ct in work.caption_tracks.iter_mut().filter(|c| !c.locked && c.sync_lock) {
captions::extract_range(ct, *span);
}
}
*seq = work;
Ok(spans)
@@ -690,6 +696,13 @@ fn media_len(ctx: &EditCtx, item: &TrackItem) -> Option<Tick> {
/// Media available before a clip's source In (its head handle, media ticks).
fn head(ctx: &EditCtx, item: &TrackItem) -> Tick {
// media with no length (an image, graphic, or adjustment layer, ..) has no set time,
// therefore head limit does not apply, like its tail.
// its source_in may go below 0, which keeps its keyframes true to timeline position (#3367).
// checked before frame hold, so a held one with a negative source_in still has no limit.
if (ctx.media_duration)(item.item).is_none() {
return Tick::MAX;
}
if item.frame_hold.is_some() {
return item.source_in;
}
@@ -1004,7 +1017,15 @@ pub fn slip(seq: &mut Sequence, clip: ClipId, delta_media: Tick, ctx: &mut EditC
let (_, it) = seq.find_item(clip).ok_or(EditError::NoItem(clip))?;
let used = src_of(it.duration, it.speed);
let max_in = media_len(ctx, it).map(|m| m - used).unwrap_or(Tick::MAX);
let lo = if it.frame_hold.is_some() { Tick::ZERO } else { (ctx.media_start)(it.item) };
let lo = if (ctx.media_duration)(it.item).is_none() {
// media with no length has no set time, therefore no floor either.
// its source_in can already be below 0 from a left extension (#3367).
Tick::MIN
} else if it.frame_hold.is_some() {
Tick::ZERO
} else {
(ctx.media_start)(it.item)
};
let new_in = (it.source_in + delta_media).clamp(lo, max_in.max(lo));
let d = new_in - it.source_in;
let (_, it) = seq.find_item_mut(clip).ok_or(EditError::NoItem(clip))?;
crates/edit/src/tests.rs
+285 -1
View File
@@ -1,5 +1,5 @@
use super::*;
use filmcraft_project::{Label, Project, SequenceSettings, TrackKind};
use filmcraft_project::{Caption, CaptionFormat, CaptionTrack, Label, Project, SequenceSettings, TrackKind};
use filmcraft_time::FrameRate;
use proptest::prelude::*;
@@ -75,6 +75,22 @@ impl Fx {
fn spans(&self, track: TrackId) -> Vec<(i64, i64)> {
self.seq.track(track).unwrap().items.iter().map(|i| (R.frame_at(i.start), R.frame_at(i.duration))).collect()
}
fn add_caption_track(&mut self, name: &str, format: CaptionFormat) -> TrackId {
let tid = TrackId(self.next);
self.next += 1;
let ct = CaptionTrack::new(tid, name.into(), format);
self.seq.caption_tracks.push(ct);
tid
}
fn add_caption(&mut self, track: TrackId, start: i64, dur: i64, text: &str) -> ClipId {
let mut n = self.next;
let id = captions::add_caption(&mut self.seq, track, f(start), f(dur), text, &mut Self::ctx(&mut n)).unwrap();
self.next = n;
id
}
fn caption_spans(&self, track: TrackId) -> Vec<(i64, i64, String)> {
self.seq.caption_track(track).unwrap().captions.iter().map(|c| (R.frame_at(c.start), R.frame_at(c.duration), c.text.clone())).collect()
}
}
#[test]
@@ -239,6 +255,107 @@ fn ripple_delete_closes_the_time_free_on_every_track() {
assert_eq!(fx.seq, before);
}
#[test]
fn ripple_delete_sync_locked_caption_tracks() {
// 1. Basic ripple delete with before, inside, overlapping, and after captions.
let mut fx = Fx::new();
let v1 = fx.v(0);
let a1 = fx.a(0);
let c1 = fx.add_caption_track("C1", CaptionFormat::Subtitle);
let _v0 = fx.put(v1, 0, 10, 0);
let v1_clip = fx.put(v1, 10, 10, 0);
let _v2 = fx.put(v1, 20, 10, 0);
let _a0 = fx.put(a1, 0, 10, 0);
let a1_clip = fx.put(a1, 10, 10, 0);
let _a2 = fx.put(a1, 20, 10, 0);
fx.add_caption(c1, 2, 6, "before");
fx.add_caption(c1, 12, 4, "inside");
fx.add_caption(c1, 18, 6, "overlap");
fx.add_caption(c1, 26, 4, "after");
ripple_delete_items(&mut fx.seq, &[v1_clip, a1_clip]).unwrap();
assert_eq!(
fx.caption_spans(c1),
vec![(2, 6, "before".into()), (10, 4, "overlap".into()), (16, 4, "after".into()),],
"captions follow ripple delete on sync-locked track"
);
// 2. Locked caption track and sync_lock=false caption track remain untouched.
let mut fx = Fx::new();
let v1 = fx.v(0);
let c_locked = fx.add_caption_track("C_Locked", CaptionFormat::Subtitle);
let c_unsynced = fx.add_caption_track("C_Unsynced", CaptionFormat::Subtitle);
fx.seq.caption_track_mut(c_locked).unwrap().locked = true;
fx.seq.caption_track_mut(c_unsynced).unwrap().sync_lock = false;
fx.seq.caption_track_mut(c_locked).unwrap().captions.push(Caption {
id: ClipId(50001),
start: f(15),
duration: f(10),
text: "locked".into(),
speaker: None,
cue_id: None,
settings: String::new(),
});
fx.seq.caption_track_mut(c_unsynced).unwrap().captions.push(Caption {
id: ClipId(50002),
start: f(15),
duration: f(10),
text: "unsynced".into(),
speaker: None,
cue_id: None,
settings: String::new(),
});
let _v0 = fx.put(v1, 0, 10, 0);
let v1_clip = fx.put(v1, 10, 10, 0);
let _v2 = fx.put(v1, 20, 10, 0);
ripple_delete_items(&mut fx.seq, &[v1_clip]).unwrap();
assert_eq!(fx.caption_spans(c_locked), vec![(15, 10, "locked".into())]);
assert_eq!(fx.caption_spans(c_unsynced), vec![(15, 10, "unsynced".into())]);
// 3. Multiple separated spans: right-to-left ripple updates captions correctly for each span.
let mut fx = Fx::new();
let v1 = fx.v(0);
let c1 = fx.add_caption_track("C1", CaptionFormat::Subtitle);
let _v0 = fx.put(v1, 0, 10, 0);
let v1_clip = fx.put(v1, 10, 10, 0);
let _v2 = fx.put(v1, 20, 10, 0);
let v3_clip = fx.put(v1, 30, 10, 0);
let _v4 = fx.put(v1, 40, 10, 0);
fx.add_caption(c1, 5, 3, "c0");
fx.add_caption(c1, 12, 4, "c1_inside");
fx.add_caption(c1, 22, 6, "c2_between");
fx.add_caption(c1, 32, 4, "c3_inside");
fx.add_caption(c1, 42, 6, "c4_after");
ripple_delete_items(&mut fx.seq, &[v1_clip, v3_clip]).unwrap();
assert_eq!(fx.caption_spans(c1), vec![(5, 3, "c0".into()), (12, 6, "c2_between".into()), (22, 6, "c4_after".into()),]);
// 4. A later span can already ripple captions before an earlier media conflict aborts.
let mut fx = Fx::new();
let (v1, a1) = (fx.v(0), fx.a(0));
let c1 = fx.add_caption_track("C1", CaptionFormat::Subtitle);
let _v0 = fx.put(v1, 0, 10, 0);
let v1_clip = fx.put(v1, 10, 10, 0);
let _v2 = fx.put(v1, 20, 10, 0);
let v3_clip = fx.put(v1, 30, 10, 0);
let _v4 = fx.put(v1, 40, 10, 0);
fx.put(a1, 12, 5, 0);
fx.add_caption(c1, 2, 4, "cap_before");
fx.add_caption(c1, 12, 4, "cap_inside");
fx.add_caption(c1, 42, 4, "cap_after");
let before = fx.seq.clone();
assert_eq!(ripple_delete_items(&mut fx.seq, &[v1_clip, v3_clip]), Err(EditError::SyncLockConflict("A1".into())));
assert_eq!(fx.seq, before, "conflict restores the complete sequence after a later span was processed");
}
#[test]
fn close_gap_works() {
let mut fx = Fx::new();
@@ -289,6 +406,143 @@ fn regular_trim_respects_neighbours_and_handles() {
fx.seq.check().unwrap();
}
#[test]
fn a_graphic_extends_from_its_left_edge() {
let mut fx = Fx::new();
let v1 = fx.v(0);
let g = fx.put(v1, 20, 10, 0);
fx.put(v1, 0, 8, 0);
fn ctx(n: &mut u64) -> EditCtx<'_> {
EditCtx { next_id: n, media_duration: &|_| None, media_start: &|_| Tick::ZERO, min_duration: f(1) }
}
let mut n = fx.next;
// dragging the left edge 30 frames earlier stops at the previous clip, 12 frames away
let d = trim(&mut fx.seq, g, Edge::In, TrimMode::Regular, -f(30), &mut ctx(&mut n)).unwrap();
assert_eq!(d, -f(12));
let it = fx.seq.find_item(g).unwrap().1;
assert_eq!((it.start, it.duration, it.source_in), (f(8), f(22), -f(12)));
// a keyframe at media time 4 is still at timeline frame 24
assert_eq!(it.effect_time_at(f(24)), f(4));
// shrinking it back returns exactly to where it started
trim(&mut fx.seq, g, Edge::In, TrimMode::Regular, f(12), &mut ctx(&mut n)).unwrap();
let it = fx.seq.find_item(g).unwrap().1;
assert_eq!((it.start, it.duration, it.source_in), (f(20), f(10), Tick::ZERO));
// a clip with real media and no footage before it is still blocked
let a = fx.put(fx.v(1), 20, 10, 0);
let d = trim(&mut fx.seq, a, Edge::In, TrimMode::Regular, -f(3), &mut Fx::ctx(&mut n)).unwrap();
assert_eq!(d, Tick::ZERO);
fx.seq.check().unwrap();
}
#[test]
fn a_graphic_ripple_extends_from_its_left_edge() {
let mut fx = Fx::new();
let v1 = fx.v(0);
let g = fx.put(v1, 20, 10, 0);
let mut n = fx.next;
let mut ctx = EditCtx { next_id: &mut n, media_duration: &|_| None, media_start: &|_| Tick::ZERO, min_duration: f(1) };
let d = ripple_trim_group(&mut fx.seq, &[g], Edge::In, -f(5), &mut ctx).unwrap();
assert_eq!(d, -f(5));
let it = fx.seq.find_item(g).unwrap().1;
assert_eq!((it.duration, it.source_in), (f(15), -f(5)));
// the ripple keeps the start, so what showed at media time 4 now shows 5 frames later
assert_eq!(it.effect_time_at(it.start + f(9)), f(4));
fx.seq.check().unwrap();
}
/// Item 2 is a graphic (no media length); everything else is long media.
fn graphic_media(i: ItemId) -> Option<Tick> {
if i == ItemId(2) { None } else { Some(f(1000)) }
}
fn graphic_ctx(n: &mut u64) -> EditCtx<'_> {
EditCtx { next_id: n, media_duration: &graphic_media, media_start: &|_| Tick::ZERO, min_duration: f(1) }
}
#[test]
fn a_roll_extends_a_graphic_from_its_left_edge() {
let mut fx = Fx::new();
let v1 = fx.v(0);
let a = fx.put(v1, 0, 10, 10);
let g = fx.put(v1, 10, 10, 0);
fx.seq.find_item_mut(g).unwrap().1.item = ItemId(2);
let mut n = fx.next;
let d = roll(&mut fx.seq, a, g, -f(5), &mut graphic_ctx(&mut n)).unwrap();
assert_eq!(d, -f(5));
assert_eq!(fx.spans(v1), vec![(0, 5), (5, 15)]);
let it = fx.seq.find_item(g).unwrap().1;
assert_eq!(it.source_in, -f(5));
// what showed at frame 12 still does
assert_eq!(it.effect_time_at(f(12)), f(2));
fx.seq.check().unwrap();
}
#[test]
fn a_slide_extends_a_graphic_from_its_left_edge() {
let mut fx = Fx::new();
let v1 = fx.v(0);
fx.put(v1, 0, 10, 10);
let b = fx.put(v1, 10, 10, 10);
let g = fx.put(v1, 20, 10, 0);
fx.seq.find_item_mut(g).unwrap().1.item = ItemId(2);
let mut n = fx.next;
let d = slide(&mut fx.seq, b, -f(5), &mut graphic_ctx(&mut n)).unwrap();
assert_eq!(d, -f(5));
assert_eq!(fx.spans(v1), vec![(0, 5), (5, 10), (15, 15)]);
let it = fx.seq.find_item(g).unwrap().1;
assert_eq!(it.source_in, -f(5));
assert_eq!(it.effect_time_at(f(22)), f(2));
fx.seq.check().unwrap();
}
#[test]
fn a_slip_moves_a_stretched_graphic_by_what_was_asked() {
let mut fx = Fx::new();
let v1 = fx.v(0);
let g = fx.put(v1, 20, 10, 0);
fx.seq.find_item_mut(g).unwrap().1.item = ItemId(2);
let mut n = fx.next;
trim(&mut fx.seq, g, Edge::In, TrimMode::Regular, -f(12), &mut graphic_ctx(&mut n)).unwrap();
assert_eq!(fx.seq.find_item(g).unwrap().1.source_in, -f(12));
let d = slip(&mut fx.seq, g, f(1), &mut graphic_ctx(&mut n)).unwrap();
assert_eq!(d, f(1));
assert_eq!(fx.seq.find_item(g).unwrap().1.source_in, -f(11));
}
#[test]
fn a_slip_moves_a_frame_held_stretched_graphic_by_what_was_asked() {
let mut fx = Fx::new();
let v1 = fx.v(0);
let g = fx.put(v1, 20, 10, 0);
{
let it = fx.seq.find_item_mut(g).unwrap().1;
it.item = ItemId(2);
it.source_in = -f(12);
it.frame_hold = Some(Tick::ZERO);
}
let mut n = fx.next;
let d = slip(&mut fx.seq, g, f(1), &mut graphic_ctx(&mut n)).unwrap();
assert_eq!(d, f(1));
assert_eq!(fx.seq.find_item(g).unwrap().1.source_in, -f(11));
}
#[test]
fn a_roll_extends_a_frame_held_graphic() {
let mut fx = Fx::new();
let v1 = fx.v(0);
let a = fx.put(v1, 0, 10, 10);
let g = fx.put(v1, 10, 10, 0);
{
let it = fx.seq.find_item_mut(g).unwrap().1;
it.item = ItemId(2);
it.source_in = -f(5);
it.frame_hold = Some(Tick::ZERO);
}
let mut n = fx.next;
let d = roll(&mut fx.seq, a, g, -f(3), &mut graphic_ctx(&mut n)).unwrap();
assert_eq!(d, -f(3));
}
#[test]
fn trim_on_a_locked_track_is_refused() {
let mut fx = Fx::new();
@@ -605,6 +859,36 @@ fn a_regular_trim_shortens_a_transition_longer_than_its_clips() {
fx.seq.check().unwrap();
}
/// A crossfade needs both clips and their cut; a fade needs only its clip.
#[test]
fn orphan_transitions_are_ones_whose_clips_are_gone_or_apart() {
let mut fx = Fx::new();
let v1 = fx.v(0);
let a = fx.put(v1, 0, 10, 0);
let b = fx.put(v1, 10, 10, 0);
let c = fx.put(v1, 25, 10, 0);
let tr = |id, from, to| Transition {
id: TransitionId(id),
effect: filmcraft_project::find_effect("cross_dissolve").unwrap().instance(),
start: f(8),
duration: f(4),
from,
to,
align: Default::default(),
reverse: false,
};
let t = fx.seq.track_mut(v1).unwrap();
t.transitions = vec![
tr(1, Some(a), Some(b)), // on the a|b cut: stays
tr(2, Some(b), Some(c)), // b and c are 5 frames apart: goes
tr(3, Some(c), None), // fade out of c: stays
tr(4, Some(ClipId(999)), None), // its clip is gone: goes
tr(5, None, None), // joins nothing: goes
];
remove_orphan_transitions(t);
assert_eq!(t.transitions.iter().map(|x| x.id.0).collect::<Vec<_>>(), vec![1, 3]);
}
#[test]
fn transitions_travel_with_moved_clips() {
// V1: a [0,10) fading in, b [10,20), c [20,30); b→c crossfade, c fading out
crates/engine/src/aaf_omf.rs
+3 -2
View File
@@ -13,7 +13,7 @@
//! `file.exportOmf {path, sequence?, audio?: "embedded"|"separate", audioFormat?, sampleRate?,
//! bitDepth?, trimAudio? (default on), handles?, renderAudioEffects?, breakoutToMono?}`
//!
//! `file.importAaf {path}` (also `file.import` of `.aaf` / `.omf` files).
//! `file.importAaf {path, bin?}` (also `file.import` of `.aaf` / `.omf` files).
use std::sync::{Arc, Mutex};
@@ -456,5 +456,6 @@ pub fn import_document(s: &mut Session, p: &Value, cmd: &str) -> Result<Value> {
let bytes = s.services.read_file(&path).map_err(|e| EngineError::Other(format!("{path}: {e}")))?;
let format = crate::interchange::detect(&path, &bytes).filter(|f| matches!(f, filmcraft_interchange::Format::Aaf | filmcraft_interchange::Format::Omf));
let format = format.ok_or_else(|| bad(cmd, format!("{path} is not an AAF or OMF file")))?;
crate::interchange::import(s, &path, &bytes, format)
let bin = u64_p(p, "bin").map(filmcraft_project::BinId);
crate::interchange::import(s, &path, &bytes, format, bin)
}
crates/engine/src/autosave.rs
+264 -24
View File
@@ -20,6 +20,7 @@ use std::fs::{self, File};
use std::io;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{Receiver, RecvTimeoutError, Sender, channel};
use std::thread::JoinHandle;
use std::time::Duration;
@@ -648,7 +649,7 @@ enum Msg {
Clean,
Prefs(AutoSavePrefs),
AutoSaveNow(Sender<Result<PathBuf, String>>),
Flush(Sender<()>),
Flush(Sender<Result<Option<u64>, String>>),
Stop,
}
@@ -678,6 +679,25 @@ pub enum WorkerEvent {
Error(String),
}
const WORKER_STOPPED: &str =
"Auto-save worker stopped; automatic saving and recovery journaling are unavailable. Use File > Save or Save As to save your changes.";
/// Shared by the thread boundary and callers: a stopped worker is one failure, even when
/// several queued requests or subsequent edits discover the disconnected channel.
struct WorkerFailure {
reported: AtomicBool,
events: Sender<WorkerEvent>,
}
impl WorkerFailure {
fn report(&self) {
if !self.reported.swap(true, Ordering::AcqRel) {
log::error!("{WORKER_STOPPED}");
let _ = self.events.send(WorkerEvent::Error(WORKER_STOPPED.into()));
}
}
}
/// Last activity, for `file.autoSaveStatus` and the UI.
#[derive(Clone, Debug, Default, Serialize)]
#[serde(rename_all = "camelCase")]
@@ -707,6 +727,7 @@ pub struct Persistence {
tx: Sender<Msg>,
events: Receiver<WorkerEvent>,
thread: Option<JoinHandle<()>>,
failure: Arc<WorkerFailure>,
}
impl Persistence {
@@ -728,6 +749,7 @@ impl Persistence {
let candidates = scan_candidates(&cfg.data_dir, Some(&session_dir));
let (tx, rx) = channel();
let (etx, events) = channel();
let failure = Arc::new(WorkerFailure { reported: AtomicBool::new(false), events: etx.clone() });
let mut w = Worker {
data_dir: cfg.data_dir.clone(),
session_dir: session_dir.clone(),
@@ -744,7 +766,12 @@ impl Persistence {
meta: JournalMeta { pid, started_unix: started, generator: filmcraft_format::generator(), ..Default::default() },
};
w.next_autosave = Instant::now() + w.autosave_interval();
let thread = std::thread::Builder::new().name("filmcraft-autosave".into()).spawn(move || w.run(rx))?;
let worker_failure = failure.clone();
let thread = std::thread::Builder::new().name("filmcraft-autosave".into()).spawn(move || {
if std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| w.run(rx))).is_err() {
worker_failure.report();
}
})?;
Ok(Self {
data_dir: cfg.data_dir.clone(),
session_dir,
@@ -756,38 +783,61 @@ impl Persistence {
tx,
events,
thread: Some(thread),
failure,
})
}
pub(crate) fn send_dirty(&self, project: Arc<Project>, revision: u64, path: Option<String>) {
let _ = self.tx.send(Msg::Dirty(Snapshot { project, revision, path }));
self.send(Msg::Dirty(Snapshot { project, revision, path }));
}
pub(crate) fn send_clean(&self) {
let _ = self.tx.send(Msg::Clean);
self.send(Msg::Clean);
}
pub fn set_prefs(&self, p: AutoSavePrefs) {
let _ = self.tx.send(Msg::Prefs(p));
self.send(Msg::Prefs(p));
}
/// Write any pending journal now and wait for it.
pub fn flush(&self) {
let (tx, rx) = channel();
if self.tx.send(Msg::Flush(tx)).is_ok() {
let _ = rx.recv_timeout(Duration::from_secs(30));
fn send(&self, msg: Msg) {
if self.tx.send(msg).is_err() {
self.failure.report();
}
}
fn stopped_error(&self) -> String {
self.failure.report();
WORKER_STOPPED.into()
}
pub(crate) fn is_running(&self) -> bool {
!self.failure.reported.load(Ordering::Acquire) && self.thread.as_ref().is_some_and(|t| !t.is_finished())
}
/// Write any pending journal now and return its confirmed revision (`None` when clean).
/// Disabled journaling, failed writes and a stopped/unresponsive worker are errors.
pub fn flush(&self) -> Result<Option<u64>, String> {
let (tx, rx) = channel();
self.tx.send(Msg::Flush(tx)).map_err(|_| self.stopped_error())?;
rx.recv_timeout(Duration::from_secs(30)).map_err(|e| match e {
RecvTimeoutError::Timeout => "recovery journal flush timed out".to_string(),
RecvTimeoutError::Disconnected => {
self.failure.report();
"auto-save worker stopped before confirming the recovery journal".to_string()
}
})?
}
/// Write an auto-save now (if there are unsaved changes) and wait for the result.
pub fn auto_save_now(&self) -> Result<Option<PathBuf>, String> {
let (tx, rx) = channel();
self.tx.send(Msg::AutoSaveNow(tx)).map_err(|_| "auto-save worker stopped".to_string())?;
self.tx.send(Msg::AutoSaveNow(tx)).map_err(|_| self.stopped_error())?;
match rx.recv_timeout(Duration::from_secs(60)) {
Ok(Ok(p)) => Ok(Some(p)),
Ok(Err(e)) if e.is_empty() => Ok(None),
Ok(Err(e)) => Err(e),
Err(_) => Err("auto-save timed out".into()),
Err(RecvTimeoutError::Timeout) => Err("auto-save timed out".into()),
Err(RecvTimeoutError::Disconnected) => Err(self.stopped_error()),
}
}
@@ -818,7 +868,9 @@ impl Persistence {
/// directory — marked as a clean exit — so the changes are offered next time; otherwise the
/// directory is removed.
pub fn close(mut self) {
self.flush();
if let Err(e) = self.flush() {
log::warn!("auto-save at shutdown: {e}");
}
self.stop_worker();
let snap = self.session_dir.join(SNAPSHOT);
self.lock.take();
@@ -835,15 +887,17 @@ impl Persistence {
/// Stop the worker and release the lock *without* cleaning up — what a crash leaves behind.
#[doc(hidden)]
pub fn simulate_crash(mut self) {
self.flush();
let _ = self.flush();
self.stop_worker();
self.lock.take();
}
fn stop_worker(&mut self) {
let _ = self.tx.send(Msg::Stop);
if let Some(t) = self.thread.take() {
let _ = t.join();
if let Some(t) = self.thread.take()
&& t.join().is_err()
{
self.failure.report();
}
}
@@ -946,17 +1000,21 @@ impl Worker {
let _ = reply.send(r);
}
Some(Msg::Flush(ack)) => {
if self.journal_due().is_some() {
self.write_journal();
}
let _ = ack.send(());
let result = match self.latest.as_ref() {
None => Ok(None),
Some(_) if !self.prefs.recovery_journal => Err("recovery journal is disabled".to_string()),
Some(s) if s.revision == self.journaled => Ok(Some(self.journaled)),
Some(_) => self.write_journal().map(Some),
};
let _ = ack.send(result);
}
Some(Msg::Stop) => return,
None => {}
}
let now = Instant::now();
if self.journal_due().is_some_and(|d| d <= now) {
self.write_journal();
// The worker reports write failures through its event channel.
let _ = self.write_journal();
}
if self.prefs.enabled && self.next_autosave <= now {
if self.latest.as_ref().is_some_and(|s| s.revision != self.autosaved) {
@@ -972,8 +1030,8 @@ impl Worker {
let _ = self.events.send(WorkerEvent::Error(e));
}
fn write_journal(&mut self) {
let Some(s) = &self.latest else { return };
fn write_journal(&mut self) -> Result<u64, String> {
let Some(s) = &self.latest else { return Err("no unsaved recovery state to write".into()) };
let t0 = Instant::now();
let bytes = filmcraft_format::encode(&s.project, false);
let serialize_ms = t0.elapsed().as_secs_f64() * 1000.0;
@@ -994,12 +1052,15 @@ impl Worker {
self.journaled = s.revision;
self.first_pending = None;
let _ = self.events.send(WorkerEvent::Journaled { revision: s.revision, unix, serialize_ms, write_ms, bytes: bytes.len() });
Ok(s.revision)
}
Err(e) => {
// Try again at the next interval rather than spinning.
self.first_pending = Some(Instant::now());
self.last_change = Instant::now();
self.error(format!("recovery journal: {e}"));
let message = format!("recovery journal: {e}");
self.error(message.clone());
Err(message)
}
}
}
@@ -1056,3 +1117,182 @@ impl Worker {
}
}
}
#[cfg(test)]
mod durability_tests {
use super::*;
#[test]
fn recovery_keeps_original_when_worker_has_stopped() {
let d = std::env::temp_dir().join(format!("fc-recover-worker-stopped-{}", std::process::id()));
let _ = fs::remove_dir_all(&d);
let old_dir = recovery_root(&d).join("1-1");
fs::create_dir_all(&old_dir).unwrap();
let bytes = filmcraft_format::encode(&Project::default(), false);
let original = old_dir.join(SNAPSHOT);
fs::write(&original, &bytes).unwrap();
let mut session = crate::Session::default();
session.start_autosave(AutosaveConfig::new(&d)).unwrap();
session.persistence.as_mut().unwrap().stop_worker();
let result = session.execute("file.recover", json!({"id": "1-1"}));
assert!(original.is_file(), "a stopped worker must not consume the original snapshot");
assert_eq!(fs::read(&original).unwrap(), bytes);
assert_eq!(session.recovery_candidates().len(), 1);
assert!(result.unwrap_err().to_string().contains("worker stopped"));
session.shutdown();
fs::remove_dir_all(&d).unwrap();
}
}
#[cfg(test)]
mod worker_failure_tests {
use super::*;
use crate::{Event, Session};
fn session(tag: &str, offset: fn(i64) -> i32) -> (Session, PathBuf) {
let dir = std::env::temp_dir().join(format!("fc-worker-{tag}-{}", std::process::id()));
fs::create_dir_all(&dir).unwrap();
let mut prefs = Preferences::default();
prefs.auto_save.recovery_journal = false;
prefs.save(&dir.join("preferences.json")).unwrap();
let mut s = Session::default();
s.execute("file.openDemoProject", json!({})).unwrap();
let mut cfg = AutosaveConfig::new(&dir);
cfg.local_offset = offset;
s.start_autosave(cfg).unwrap();
s.execute("sequence.addEdit", json!({"seconds": 0.5})).unwrap();
(s, dir)
}
fn failure_toasts(s: &mut Session) -> usize {
s.drain_events().into_iter().filter(|e| matches!(e, Event::Toast { error: true, .. })).count()
}
fn verify_report_and_foreground_save(s: &mut Session, dir: &Path) {
s.poll_persistence();
assert_eq!(s.execute("file.autoSaveStatus", json!({})).unwrap()["running"], false);
let message = s.persistence.as_ref().unwrap().status.last_error.as_deref().unwrap();
assert!(message.contains("stopped") && message.contains("Save"), "{message}");
assert_eq!(failure_toasts(s), 1, "worker failure must be visible exactly once");
let log: Vec<_> = s.log.entries.iter().filter(|e| e.source == "autosave").collect();
assert_eq!(log.len(), 1);
assert_eq!(log[0].count, 1);
let before = s.revision;
s.execute("sequence.addEdit", json!({"seconds": 1.0})).unwrap();
assert!(s.revision > before && s.is_dirty(), "the edit was applied despite autosave failure");
s.execute("prefs.set", json!({"key": "autoSave.maxVersions", "value": 7})).unwrap();
s.poll_persistence();
s.poll_persistence();
assert_eq!(failure_toasts(s), 0, "later handoffs must not repeat the failure");
let path = dir.join("manual.fcproj");
s.execute("file.saveAs", json!({"path": path.to_string_lossy()})).unwrap();
assert!(!s.is_dirty());
let loaded = filmcraft_format::decode(&fs::read(&path).unwrap()).unwrap();
assert_eq!(loaded.project, *s.project);
s.poll_persistence();
assert_eq!(failure_toasts(s), 0);
}
#[test]
fn worker_panic_is_contained_and_reported() {
fn broken_clock(_: i64) -> i32 {
panic!("synthetic worker clock failure")
}
let (mut s, dir) = session("panic", broken_clock);
let error = s.execute("file.autoSaveNow", json!({})).unwrap_err().to_string();
let start = Instant::now();
while !s.persistence.as_ref().unwrap().thread.as_ref().unwrap().is_finished() {
assert!(start.elapsed() < Duration::from_secs(10), "worker did not exit");
std::thread::sleep(Duration::from_millis(10));
}
let joined = s.persistence.as_mut().unwrap().thread.take().unwrap().join();
assert!(joined.is_ok(), "panic escaped the autosave worker boundary");
assert!(error.contains("stopped") && !error.contains("timed out"), "{error}");
verify_report_and_foreground_save(&mut s, &dir);
s.shutdown();
fs::remove_dir_all(dir).unwrap();
}
fn disconnected_handoff(tag: &str) {
let (mut s, dir) = session(tag, |_| 0);
s.persistence.as_mut().unwrap().stop_worker();
match tag {
"dirty" => {
s.execute("sequence.addEdit", json!({"seconds": 2.0})).unwrap();
}
"clean" => {
s.execute("file.saveAs", json!({"path": dir.join("initial.fcproj").to_string_lossy()})).unwrap();
}
"prefs" => {
s.execute("prefs.set", json!({"key": "autoSave.maxVersions", "value": 3})).unwrap();
}
"flush" => {
assert!(s.persistence.as_ref().unwrap().flush().unwrap_err().contains("worker stopped"));
}
"request" => {
assert!(s.execute("file.autoSaveNow", json!({})).unwrap_err().to_string().contains("stopped"));
}
_ => panic!("unknown test handoff"),
}
verify_report_and_foreground_save(&mut s, &dir);
s.shutdown();
fs::remove_dir_all(dir).unwrap();
}
#[test]
fn worker_disconnect_dirty_reports_once() {
disconnected_handoff("dirty");
}
#[test]
fn worker_disconnect_clean_reports_once() {
disconnected_handoff("clean");
}
#[test]
fn worker_disconnect_preferences_reports_once() {
disconnected_handoff("prefs");
}
#[test]
fn worker_disconnect_flush_reports_once() {
disconnected_handoff("flush");
}
#[test]
fn worker_disconnect_flush_reply_reports_once() {
let (mut s, dir) = session("flush-reply", |_| 0);
s.persistence.as_mut().unwrap().stop_worker();
let (tx, rx) = channel();
s.persistence.as_mut().unwrap().tx = tx;
let worker = std::thread::spawn(move || match rx.recv().unwrap() {
Msg::Flush(reply) => drop(reply),
_ => panic!("expected a flush request"),
});
let error = s.persistence.as_ref().unwrap().flush().unwrap_err();
worker.join().unwrap();
assert!(error.contains("worker stopped"), "{error}");
verify_report_and_foreground_save(&mut s, &dir);
s.shutdown();
fs::remove_dir_all(dir).unwrap();
}
#[test]
fn worker_disconnect_request_reports_once() {
disconnected_handoff("request");
}
#[test]
fn worker_remains_available_after_an_ordinary_write_error() {
let (mut s, dir) = session("write-error", |_| 0);
let blocked = dir.join(names::AUTO_SAVE_DIR);
fs::write(&blocked, b"occupied directory path").unwrap();
assert!(s.execute("file.autoSaveNow", json!({})).is_err());
s.poll_persistence();
assert_eq!(s.execute("file.autoSaveStatus", json!({})).unwrap()["running"], true);
assert!(s.persistence.as_ref().unwrap().status.last_error.as_deref().unwrap().contains("auto-save to"));
assert_eq!(failure_toasts(&mut s), 1);
fs::remove_file(&blocked).unwrap();
let saved = s.execute("file.autoSaveNow", json!({})).unwrap();
assert!(Path::new(saved["path"].as_str().unwrap()).is_file());
assert!(s.is_dirty(), "auto-save should not mark the foreground project saved");
s.shutdown();
fs::remove_dir_all(dir).unwrap();
}
}
crates/engine/src/autosave_tests.rs
+104
View File
@@ -231,6 +231,110 @@ fn quitting_with_unsaved_changes_keeps_them_for_next_launch() {
let _ = std::fs::remove_dir_all(&d);
}
/// A disabled or unwritable replacement journal must never consume the original recovery copy.
fn recovery_preserves_original(tag: &str, block: Option<&str>, journal_enabled: bool) {
let d = temp_dir(tag);
let old_dir = autosave::recovery_root(&d).join("1-1");
std::fs::create_dir_all(&old_dir).unwrap();
let project = (*demo().project).clone();
let bytes = filmcraft_format::encode(&project, false);
let original = old_dir.join("snapshot.fcproj");
std::fs::write(&original, &bytes).unwrap();
let mut s = Session::default();
let mut cfg = autosave::AutosaveConfig::new(&d);
cfg.journal_debounce = std::time::Duration::from_secs(60);
s.start_autosave(cfg).unwrap();
s.execute("prefs.set", json!({"key": "autoSave.recoveryJournal", "value": journal_enabled})).unwrap();
if let Some(name) = block {
// A directory where the atomic write needs a file reliably fails on Windows and Unix.
std::fs::create_dir(s.persistence.as_ref().unwrap().session_dir.join(name)).unwrap();
}
let result = s.execute("file.recover", json!({"id": "1-1"}));
assert!(original.is_file(), "failed replacement must preserve the original snapshot");
assert_eq!(std::fs::read(&original).unwrap(), bytes);
assert_eq!(s.recovery_candidates().len(), 1, "the original must remain discoverable");
assert!(s.persistence.as_ref().unwrap().is_running(), "ordinary journal errors must not mark the worker stopped");
assert!(result.is_err(), "lack of a confirmed replacement must be reported");
assert_eq!(*s.project, project, "the recovered project remains open for Save As");
assert!(s.is_dirty());
s.shutdown();
let found = autosave::scan_candidates(&d, None);
assert!(found.iter().any(|c| c.id == "1-1"), "another launch can still recover the original");
std::fs::remove_dir_all(&d).unwrap();
}
#[test]
fn recovery_keeps_original_when_snapshot_write_fails() {
recovery_preserves_original("recover-snapshot-fails", Some("snapshot.fcproj"), true);
}
#[test]
fn recovery_keeps_original_when_metadata_write_fails() {
recovery_preserves_original("recover-meta-fails", Some("session.json"), true);
}
#[test]
fn recovery_keeps_original_when_journal_is_disabled() {
recovery_preserves_original("recover-journal-disabled", None, false);
}
#[test]
fn recovery_can_retry_after_a_failed_journal_write() {
let d = temp_dir("recover-retry");
let old_dir = autosave::recovery_root(&d).join("1-1");
std::fs::create_dir_all(&old_dir).unwrap();
let project = (*demo().project).clone();
std::fs::write(old_dir.join("snapshot.fcproj"), filmcraft_format::encode(&project, false)).unwrap();
let mut s = Session::default();
let mut cfg = autosave::AutosaveConfig::new(&d);
cfg.journal_debounce = std::time::Duration::from_secs(60);
s.start_autosave(cfg).unwrap();
let obstruction = s.persistence.as_ref().unwrap().session_dir.join("snapshot.fcproj");
std::fs::create_dir(&obstruction).unwrap();
assert!(s.execute("file.recover", json!({"id": "1-1"})).is_err());
assert!(old_dir.join("snapshot.fcproj").is_file());
std::fs::remove_dir(&obstruction).unwrap();
s.execute("file.recover", json!({"id": "1-1"})).unwrap();
assert!(!old_dir.exists());
assert!(s.recovery_candidates().is_empty());
let replacement = std::fs::read(&obstruction).unwrap();
assert_eq!(filmcraft_format::decode(&replacement).unwrap().project, project);
assert_eq!(journaled(&mut s), s.revision);
s.shutdown();
std::fs::remove_dir_all(&d).unwrap();
}
#[cfg(windows)]
#[test]
fn recovery_cleanup_failure_reports_open_project_and_safe_retry() {
use std::os::windows::fs::OpenOptionsExt;
let d = temp_dir("recover-cleanup-failure");
let old_dir = autosave::recovery_root(&d).join("1-1");
std::fs::create_dir_all(&old_dir).unwrap();
let project = filmcraft_project::Project::default();
let original = old_dir.join("snapshot.fcproj");
std::fs::write(&original, filmcraft_format::encode(&project, false)).unwrap();
// Permit the loader's read while denying deletion of this old snapshot.
let held = std::fs::OpenOptions::new().read(true).share_mode(3).open(&original).unwrap();
let mut s = Session::default();
start(&mut s, &d);
let error = s.execute("file.recover", json!({"id": "1-1"})).unwrap_err().to_string();
assert_eq!(*s.project, project, "recovery already installed the project");
assert!(s.is_dirty());
assert_eq!(recovery_ids(&mut s), ["1-1"], "failed cleanup must keep the candidate listed");
let replacement = &s.persistence.as_ref().unwrap().session_dir;
assert!(replacement.join("snapshot.fcproj").is_file());
assert!(replacement.join("session.json").is_file());
assert!(error.contains("Recovered project is open") && error.contains("new recovery copy is confirmed"), "{error}");
assert!(error.contains("candidate remains listed") && error.contains("Save As"), "{error}");
assert!(error.contains("if the original recovery copy remains intact") && error.contains("unsaved edits"), "{error}");
drop(held);
s.shutdown();
std::fs::remove_dir_all(d).unwrap();
}
#[test]
fn auto_save_ring_rotates_and_can_save_the_project() {
let d = temp_dir("ring");
crates/engine/src/captions.rs
+6 -1
View File
@@ -175,6 +175,7 @@ pub fn copy(s: &mut Session) -> usize {
s.state.caption_clipboard = picked.into_iter().map(|(ti, c)| (ti, filmcraft_project::Caption { start: c.start - min, ..c })).collect();
s.state.clipboard.clear();
s.state.clipboard_markers.clear();
s.state.clipboard_transitions.clear();
s.state.caption_clipboard.len()
}
@@ -248,6 +249,7 @@ fn track_json(s: &Session, t: &CaptionTrack, index: usize) -> Value {
"background": t.style.background, "backgroundColor": hex(t.style.background_color),
"align": format!("{:?}", t.style.align).to_lowercase(), "anchor": format!("{:?}", t.style.anchor).to_lowercase(),
"margin": t.style.margin, "lineSpacing": t.style.line_spacing, "outline": t.style.outline, "outlineColor": hex(t.style.outline_color),
"fontStyle": t.style.font_style,
},
"captions": t.captions.iter().map(|c| caption_json(s, c)).collect::<Vec<_>>(),
})
@@ -352,7 +354,7 @@ pub fn commands() -> Vec<CommandSpec> {
"Caption Track Style",
&[],
None,
r##"{"track":id|"C1","font":str?,"size":f32?,"color":"#rrggbb[aa]"?,"background":bool?,"backgroundColor":"#rrggbbaa"?,"align":"left|center|right"?,"anchor":"top|middle|bottom"?,"margin":0..0.45 (fraction of frame height)?,"lineSpacing":f32?,"outline":f32?,"outlineColor":str?,"reset":bool?}"##,
r##"{"track":id|"C1","font":str?,"fontStyle":str="SemiBold" (Regular, Medium, Bold…)?,"size":f32?,"color":"#rrggbb[aa]"?,"background":bool?,"backgroundColor":"#rrggbbaa"?,"align":"left|center|right"?,"anchor":"top|middle|bottom"?,"margin":0..0.45 (fraction of frame height)?,"lineSpacing":f32?,"outline":f32?,"outlineColor":str?,"reset":bool?}"##,
has_caption_track,
|s, p| {
let tid = track_param(s, p).ok_or_else(|| {
@@ -375,6 +377,9 @@ pub fn commands() -> Vec<CommandSpec> {
if let Some(v) = str_p(&p, "font") {
y.font = v.to_string();
}
if let Some(v) = str_p(&p, "fontStyle") {
y.font_style = v.trim().to_string();
}
if let Some(v) = f64_p(&p, "size") {
y.size = (v as f32).clamp(4.0, 400.0);
}
crates/engine/src/captions/tests.rs
+161
View File
@@ -164,6 +164,155 @@ fn caption_spans(s: &Session) -> Vec<(f64, f64, String)> {
s.active_sequence().unwrap().caption_tracks[0].captions.iter().map(|c| (secs(c.start), secs(c.end()), c.text.clone())).collect()
}
/// (clip id, start) of the clips on V1, and every caption's start.
fn v1_and_caption_starts(s: &Session) -> (Vec<(u64, Tick)>, Vec<Tick>) {
let q = s.active_sequence().unwrap();
(q.video_tracks[0].items.iter().map(|i| (i.id.0, i.start)).collect(), q.caption_tracks[0].captions.iter().map(|c| c.start).collect())
}
#[test]
fn timeline_move_takes_the_listed_captions_along_in_one_undo_step() {
let mut s = demo();
for (sec, text) in [(1.0, "early"), (9.0, "late one"), (12.0, "late two")] {
s.execute("playhead.set", json!({"seconds": sec})).unwrap();
s.execute("captions.add", json!({"text": text, "durationSeconds": 1.0})).unwrap();
}
let q = s.active_sequence().unwrap();
let ids: Vec<u64> = q.caption_tracks[0].captions.iter().map(|c| c.id.0).collect();
let (clip, start) = (q.video_tracks[0].items[2].id, q.video_tracks[0].items[2].start);
let (clips_before, caps_before) = v1_and_caption_starts(&s);
let undo_before = s.history.undo.len();
let shift = 2 * TICKS_PER_SECOND;
// the captions named move by the clip's own offset; `early` is not named and stays
let r = s
.execute(
"timeline.move",
json!({"moves": [{"clip": clip.0, "track": "V1", "time": (start + Tick(shift)).0}], "linked": false, "captions": [ids[1], ids[2], ids[1]]}),
)
.unwrap();
assert_eq!(r["captions"], json!([ids[1], ids[2]]), "each caption once");
let (clips_after, caps_after) = v1_and_caption_starts(&s);
assert_eq!(clips_after.iter().find(|c| c.0 == clip.0).unwrap().1, start + Tick(shift));
assert_eq!(caps_after, vec![caps_before[0], caps_before[1] + Tick(shift), caps_before[2] + Tick(shift)]);
assert_eq!(s.history.undo.len(), undo_before + 1, "clips and captions are one undo step");
s.active_sequence().unwrap().check().unwrap();
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(v1_and_caption_starts(&s), (clips_before, caps_before), "undo restores both");
}
#[test]
fn timeline_move_with_captions_that_cannot_follow_changes_nothing() {
let mut s = demo();
for (sec, text) in [(2.0, "stays"), (5.0, "moves")] {
s.execute("playhead.set", json!({"seconds": sec})).unwrap();
s.execute("captions.add", json!({"text": text, "durationSeconds": 1.0})).unwrap();
}
let q = s.active_sequence().unwrap();
let moving = q.caption_tracks[0].captions[1].id.0;
let (clip, start) = (q.video_tracks[0].items[1].id, q.video_tracks[0].items[1].start);
let before = v1_and_caption_starts(&s);
let undo_before = s.history.undo.len();
// 4 s earlier would put `moves` on top of `stays`: the clip does not move either
let back = Tick(start.0 - 4 * TICKS_PER_SECOND).max(Tick::ZERO);
let r = s.execute("timeline.move", json!({"moves": [{"clip": clip.0, "track": "V1", "time": back.0}], "linked": false, "captions": [moving]}));
assert!(r.is_err(), "the captions cannot follow: {r:?}");
assert_eq!(v1_and_caption_starts(&s), before);
assert_eq!(s.history.undo.len(), undo_before, "a refused move leaves no undo step");
// a copy leaves the originals where they are, so there is nothing for captions to follow
let r = s.execute("timeline.move", json!({"moves": [{"clip": clip.0, "track": "V2", "time": start.0}], "copy": true, "captions": [moving]}));
assert!(r.is_err());
assert_eq!(v1_and_caption_starts(&s), before);
// a caption that does not exist, or sits on a locked track, refuses the move as well
let missing = s.execute(
"timeline.move",
json!({"moves": [{"clip": clip.0, "track": "V1", "time": (start + Tick(TICKS_PER_SECOND)).0}], "captions": [987_654_321u64]}),
);
assert!(missing.is_err());
assert_eq!(v1_and_caption_starts(&s), before, "the clip did not move either");
s.execute("captions.setTrack", json!({"track": "C1", "locked": true})).unwrap();
let locked =
s.execute("timeline.move", json!({"moves": [{"clip": clip.0, "track": "V1", "time": (start + Tick(TICKS_PER_SECOND)).0}], "captions": [moving]}));
assert!(locked.is_err());
assert_eq!(v1_and_caption_starts(&s), before);
assert_eq!(s.history.undo.len(), undo_before + 1, "only the lock itself was recorded");
s.active_sequence().unwrap().check().unwrap();
}
/// One caption at 5 s, and the second clip of V1 (its id and start).
fn caption_and_second_clip(s: &mut Session) -> (u64, filmcraft_project::ClipId, Tick) {
s.execute("playhead.set", json!({"seconds": 5.0})).unwrap();
s.execute("captions.add", json!({"text": "one", "durationSeconds": 1.0})).unwrap();
let q = s.active_sequence().unwrap();
(q.caption_tracks[0].captions[0].id.0, q.video_tracks[0].items[1].id, q.video_tracks[0].items[1].start)
}
#[test]
fn timeline_move_refuses_a_malformed_captions_list() {
let mut s = demo();
let (cap, clip, start) = caption_and_second_clip(&mut s);
let before = v1_and_caption_starts(&s);
let undo_before = s.history.undo.len();
let later = (start + Tick(TICKS_PER_SECOND)).0;
// an absent `captions` is an empty list; anything but an array of caption ids is refused
for captions in [json!("x"), json!(5), json!(null), json!({"id": cap}), json!([cap, "oops"]), json!([cap, null]), json!([cap, -1]), json!([cap, 1.5])] {
let r = s.execute("timeline.move", json!({"moves": [{"clip": clip.0, "track": "V1", "time": later}], "linked": false, "captions": captions}));
assert!(r.is_err(), "{captions} should be refused");
assert_eq!(v1_and_caption_starts(&s), before, "{captions}: nothing moved, the clip included");
}
assert_eq!(s.history.undo.len(), undo_before, "refused moves leave no undo step");
s.execute("timeline.move", json!({"moves": [{"clip": clip.0, "track": "V1", "time": later}], "linked": false})).unwrap();
}
#[test]
fn timeline_move_checks_the_captions_even_when_the_clips_keep_their_time() {
let mut s = demo();
let (cap, clip, start) = caption_and_second_clip(&mut s);
let before = v1_and_caption_starts(&s);
let same_time =
|track: &str, captions: serde_json::Value| json!({"moves": [{"clip": clip.0, "track": track, "time": start.0}], "linked": false, "captions": captions});
// a clip that only changes track gives the captions an offset of zero, but an unknown id is still refused
assert!(s.execute("timeline.move", same_time("V2", json!([987_654_321u64]))).is_err());
assert_eq!(v1_and_caption_starts(&s), before);
// a real caption is accepted, and is not reported as moved: it did not
let r = s.execute("timeline.move", same_time("V2", json!([cap]))).unwrap();
assert!(r.get("captions").is_none(), "no caption moved: {r}");
assert_eq!(s.active_sequence().unwrap().caption_tracks[0].captions[0].start, before.1[0]);
// a locked caption track is refused at an offset of zero too
s.execute("captions.setTrack", json!({"track": "C1", "locked": true})).unwrap();
assert!(s.execute("timeline.move", same_time("V1", json!([cap]))).is_err());
assert_eq!(s.active_sequence().unwrap().video_tracks[1].items.iter().filter(|i| i.id == clip).count(), 1, "the clip stayed on V2");
// when they do move, the reply lists them
s.execute("captions.setTrack", json!({"track": "C1", "locked": false})).unwrap();
let later = (start + Tick(TICKS_PER_SECOND)).0;
let r = s.execute("timeline.move", json!({"moves": [{"clip": clip.0, "track": "V2", "time": later}], "linked": false, "captions": [cap]})).unwrap();
assert_eq!(r["captions"], json!([cap]), "{r}");
}
/// An insert already carries the unlocked, sync-locked caption tracks (once), so naming captions as
/// well would shift them a second time: the combination is refused.
#[test]
fn timeline_move_with_insert_leaves_sync_locked_captions_to_the_insert() {
let mut s = demo();
let (cap, clip, _) = caption_and_second_clip(&mut s);
let before = v1_and_caption_starts(&s);
let undo_before = s.history.undo.len();
// inserted at the start, the clip lands before the caption, which the insert pushes later
let to = 0;
let r = s.execute("timeline.move", json!({"moves": [{"clip": clip.0, "track": "V1", "time": to}], "linked": false, "insert": true, "captions": [cap]}));
assert!(r.is_err(), "insert and captions together are refused");
assert_eq!(v1_and_caption_starts(&s), before);
assert_eq!(s.history.undo.len(), undo_before);
// the insert alone is unchanged, and still moves the sync-locked caption itself
s.execute("timeline.move", json!({"moves": [{"clip": clip.0, "track": "V1", "time": to}], "linked": false, "insert": true})).unwrap();
assert_ne!(s.active_sequence().unwrap().caption_tracks[0].captions[0].start, before.1[0], "the insert shifted the caption on its own");
}
#[test]
fn captions_duplicate_copies_and_overwrites() {
let mut s = demo();
@@ -224,3 +373,15 @@ fn captions_copy_cut_and_paste() {
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(caption_spans(&s).len(), 1);
}
#[test]
fn set_style_font_style() {
let mut s = demo();
s.execute("captions.add", json!({"text": "Styled", "seconds": 1.0})).unwrap();
assert_eq!(s.active_sequence().unwrap().caption_tracks[0].style.font_style, "SemiBold");
s.execute("captions.setStyle", json!({"fontStyle": " Medium "})).unwrap();
assert_eq!(s.active_sequence().unwrap().caption_tracks[0].style.font_style, "Medium");
assert_eq!(s.execute("captions.list", json!({})).unwrap()["tracks"][0]["style"]["fontStyle"], "Medium");
s.execute("captions.setStyle", json!({"reset": true})).unwrap();
assert_eq!(s.active_sequence().unwrap().caption_tracks[0].style.font_style, "SemiBold");
}
crates/engine/src/clip_mismatch_tests.rs
+146
View File
@@ -0,0 +1,146 @@
//! Settings ▸ Timeline ▸ Show Clip Mismatch Warning dialog: [`crate::commands::clip_mismatch`] and
//! `matchSequence` on `timeline.place`, `source.insert` and `source.overwrite`.
use filmcraft_media::MediaKind;
use filmcraft_project::{ItemId, ItemKind};
use filmcraft_time::FrameRate;
use serde_json::json;
use crate::Session;
use crate::commands::clip_mismatch;
/// A movie of the demo project (23.976 fps, 1920 x 1080).
fn demo_movie(s: &Session, not: Option<ItemId>) -> ItemId {
s.project
.items
.values()
.find(|i| Some(i.id) != not && i.has_video() && i.has_audio() && i.as_media().is_some_and(|m| m.info.kind == MediaKind::Movie))
.map(|i| i.id)
.unwrap()
}
/// The demo project with a new, empty sequence open (1920 x 1080, 23.976 fps), and one of the
/// demo's movies made to play at 120 fps, like a phone or game capture.
fn empty_sequence_and_120fps_movie() -> (Session, ItemId) {
let mut s = Session::default();
s.execute("file.openDemoProject", json!({})).unwrap();
let movie = demo_movie(&s, None);
s.edit("120 fps", |pr, _| {
if let Some(ItemKind::Media(m)) = pr.item_mut(movie).map(|i| &mut i.kind)
&& let Some(v) = m.info.video.as_mut()
{
v.frame_rate = FrameRate::FPS_120;
}
Ok(())
})
.unwrap();
s.execute("file.newSequence", json!({"name": "Empty"})).unwrap();
(s, movie)
}
fn settings(s: &Session) -> (u32, u32, FrameRate) {
let q = s.active_sequence().unwrap();
(q.settings.width, q.settings.height, q.settings.frame_rate)
}
fn clips(s: &Session) -> usize {
s.active_sequence().unwrap().all_tracks().map(|t| t.items.len()).sum()
}
fn last_undo(s: &Session) -> Option<&str> {
s.history.undo.last().map(|u| u.0.as_str())
}
#[test]
fn a_first_clip_that_differs_from_an_empty_sequence_mismatches() {
let (mut s, movie) = empty_sequence_and_120fps_movie();
assert_eq!(settings(&s), (1920, 1080, FrameRate::FPS_23_976));
let st = clip_mismatch(&s, movie).expect("a 120 fps movie does not match a 23.976 fps sequence");
assert_eq!((st.width, st.height, st.frame_rate), (1920, 1080, FrameRate::FPS_120));
assert_eq!(st.preset, "1920x1080 120");
assert_eq!(st.sample_rate, s.active_sequence().unwrap().settings.sample_rate, "the sequence's other settings stay");
// once the sequence holds a clip, the next one is not asked about
s.execute("timeline.place", json!({"item": movie.0, "time": 0})).unwrap();
assert!(clip_mismatch(&s, movie).is_none());
}
#[test]
fn a_matching_movie_sound_a_still_and_a_sequence_do_not_mismatch() {
let (mut s, movie) = empty_sequence_and_120fps_movie();
assert!(clip_mismatch(&s, demo_movie(&s, Some(movie))).is_none(), "the demo's other movies match the sequence");
let sound = s.project.items.values().find(|i| !i.has_video() && i.has_audio()).map(|i| i.id).unwrap();
assert!(clip_mismatch(&s, sound).is_none());
let sequence = s.project.items.values().find(|i| i.as_sequence().is_some()).map(|i| i.id).unwrap();
assert!(clip_mismatch(&s, sequence).is_none());
// a still of another rate: stills have no frame rate of their own to match
s.edit("still", |pr, _| {
if let Some(ItemKind::Media(m)) = pr.item_mut(movie).map(|i| &mut i.kind) {
m.info.kind = MediaKind::Still;
}
Ok(())
})
.unwrap();
assert!(clip_mismatch(&s, movie).is_none());
}
#[test]
fn match_sequence_takes_the_clips_settings_in_the_same_undo_step() {
let (mut s, movie) = empty_sequence_and_120fps_movie();
let steps = s.history.undo.len();
let r = s.execute("timeline.place", json!({"item": movie.0, "time": 0, "matchSequence": true})).unwrap();
assert_eq!(r["clips"].as_array().map(Vec::len), Some(2), "picture and sound: {r}");
assert_eq!(settings(&s), (1920, 1080, FrameRate::FPS_120));
assert_eq!(s.active_sequence().unwrap().settings.preset, "1920x1080 120");
assert_eq!(s.history.undo.len(), steps + 1, "one undo step");
assert_eq!(last_undo(&s), Some("Place Clip"));
s.undo();
assert_eq!(clips(&s), 0);
assert_eq!(settings(&s), (1920, 1080, FrameRate::FPS_23_976), "the one undo puts the settings back too");
}
#[test]
fn without_match_sequence_the_settings_stay() {
for params in [json!({"time": 0}), json!({"time": 0, "matchSequence": false})] {
let (mut s, movie) = empty_sequence_and_120fps_movie();
let mut p = params.clone();
p["item"] = json!(movie.0);
s.execute("timeline.place", p).unwrap();
assert_eq!(clips(&s), 2, "{params}");
assert_eq!(settings(&s), (1920, 1080, FrameRate::FPS_23_976), "{params}");
}
}
#[test]
fn match_sequence_leaves_a_sequence_that_holds_clips_alone() {
let (mut s, movie) = empty_sequence_and_120fps_movie();
s.execute("timeline.place", json!({"item": demo_movie(&s, Some(movie)).0, "time": 0})).unwrap();
s.execute("timeline.place", json!({"item": movie.0, "seconds": 60.0, "matchSequence": true})).unwrap();
assert_eq!(settings(&s), (1920, 1080, FrameRate::FPS_23_976));
assert_eq!(clips(&s), 4);
}
#[test]
fn match_sequence_on_items_insert_and_overwrite() {
// `items`: the first item decides
let (mut s, movie) = empty_sequence_and_120fps_movie();
s.execute("timeline.place", json!({"items": [movie.0], "time": 0, "matchSequence": true})).unwrap();
assert_eq!(settings(&s).2, FrameRate::FPS_120);
assert_eq!(last_undo(&s), Some("Place Clips"));
// Insert and Overwrite from the Source monitor
for (command, label) in [("source.insert", "Insert"), ("source.overwrite", "Overwrite")] {
let (mut s, movie) = empty_sequence_and_120fps_movie();
s.execute("source.open", json!({"item": movie.0})).unwrap();
s.execute(command, json!({"matchSequence": true})).unwrap();
assert_eq!(settings(&s).2, FrameRate::FPS_120, "{command}");
assert_eq!(clips(&s), 2, "{command}");
assert_eq!(last_undo(&s), Some(label), "{command}");
s.undo();
assert_eq!((settings(&s).2, clips(&s)), (FrameRate::FPS_23_976, 0), "{command}");
}
}
#[test]
fn the_setting_is_wired() {
// the UI's Clip Mismatch Warning reads it, so Settings no longer lists it as kept only for parity
assert!(crate::settings::field("timeline.clipMismatchWarning").is_some_and(|f| f.wired));
}
crates/engine/src/clip_ops.rs
+2 -2
View File
@@ -185,7 +185,7 @@ pub(crate) fn commands() -> Vec<CommandSpec> {
"Frame Hold Options…",
&["Clip", "Video Options"],
None,
r#"{"clips":[id]?,"enabled":bool=true,"holdOn":"sourceTimecode|sequenceTime|in|out|playhead","time":ticks?,"timecode":str?,"holdFilters":bool=false}"#,
r#"{"clips":[id]?,"enabled":bool=true,"holdOn":"sourceTimecode|sequenceTime|in|out|playhead"?,"time":ticks?,"timecode":str?,"holdFilters":bool=false}"#,
has_selection,
frame_hold_options,
),
@@ -203,7 +203,7 @@ pub(crate) fn commands() -> Vec<CommandSpec> {
"Field Options…",
&["Clip", "Video Options"],
None,
r#"{"clips":[id]?,"reverseFieldDominance":bool=false,"processing":"none|alwaysDeinterlace|flickerRemoval"}"#,
r#"{"clips":[id]?,"reverseFieldDominance":bool=false,"processing":"none|alwaysDeinterlace|flickerRemoval"?}"#,
has_selection,
field_options,
),
crates/engine/src/clip_ops_tests.rs
+142 -1
View File
@@ -1,7 +1,7 @@
//! Tests of the M3.10 Edit / Clip / File menu commands (`clip_ops`): each one executes, checks the
//! result, undoes / redoes where it edits, and checks its disabled state.
use filmcraft_project::{AudioChannels, FieldProcessing, ItemId, ItemKind, Label, TimeInterpolation, TrackItem};
use filmcraft_project::{AudioChannels, ClipId, FieldProcessing, ItemId, ItemKind, Label, TimeInterpolation, TrackItem};
use filmcraft_time::{TICKS_PER_SECOND, Tick};
use serde_json::json;
@@ -993,6 +993,77 @@ fn speed_duration_moves_fade_out_to_the_new_clip_end() {
assert_eq!(fade.to, None);
}
/// An edit that moves a crossfade's clips apart drops the crossfade, whatever the command: a
/// regular trim opening a gap, a razor through the outgoing clip, an insert on the cut. Ripple
/// trims keep the cut, so the crossfade stays on it. `Session::edit` does this for every command.
#[test]
fn a_regular_trim_that_opens_a_gap_drops_the_crossfade() {
let mut s = Session::default();
s.execute("file.newSequence", json!({"name": "Trim crossfade", "video": 1, "audio": 0})).unwrap();
let rate = s.sequence_rate();
let matte = s.execute("file.newColorMatte", json!({"color": "#00aa00", "seconds": 5.0})).unwrap()["item"].as_u64().unwrap();
let dur = rate.tick_of(60);
let a = s.execute("timeline.place", json!({"item": matte, "track": "V1", "time": Tick::ZERO.0, "insert": true, "duration": dur.0})).unwrap()["clips"][0]
.as_u64()
.unwrap();
let b = s.execute("timeline.place", json!({"item": matte, "track": "V1", "time": dur.0, "insert": true, "duration": dur.0})).unwrap()["clips"][0]
.as_u64()
.unwrap();
s.execute("sequence.applyVideoTransition", json!({"clip": a, "edge": "out", "frames": 24})).unwrap();
let transitions = |s: &Session| s.active_sequence().unwrap().video_tracks[0].transitions.len();
assert_eq!(transitions(&s), 1);
// a's end trimmed back: gap before b, the crossfade goes away
s.execute("timeline.trim", json!({"clip": a, "edge": "out", "mode": "regular", "deltaFrames": -10})).unwrap();
assert_eq!(transitions(&s), 0, "a gap before b must drop the crossfade");
// b's start trimmed later is just as fatal
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(transitions(&s), 1);
s.execute("timeline.trim", json!({"clip": b, "edge": "in", "mode": "regular", "deltaFrames": 10})).unwrap();
assert_eq!(transitions(&s), 0, "a gap after a must drop the crossfade");
// ripples keep the cut, so the crossfade stays centred on it
s.execute("edit.undo", json!({})).unwrap();
s.execute("timeline.trim", json!({"clip": a, "edge": "out", "mode": "ripple", "deltaFrames": -10})).unwrap();
let q = s.active_sequence().unwrap();
assert_eq!(q.video_tracks[0].transitions.len(), 1, "a ripple trim keeps the crossfade");
let tr = &q.video_tracks[0].transitions[0];
let cut = q.video_tracks[0].items.iter().find(|i| i.id.0 == b).unwrap().start;
assert_eq!(tr.start + rate.tick_of(12), cut, "still centred on the surviving cut");
// a razor through the crossfade's outgoing half splits a away from b
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(transitions(&s), 1);
s.execute("timeline.razor", json!({"frame": 54})).unwrap();
assert_eq!(transitions(&s), 0, "a cut through the outgoing clip drops the crossfade");
// a clip inserted on the cut pushes b away from a
s.execute("edit.undo", json!({})).unwrap();
s.execute("timeline.place", json!({"item": matte, "track": "V1", "time": dur.0, "insert": true, "duration": rate.tick_of(5).0})).unwrap();
assert_eq!(transitions(&s), 0, "an insert on the cut drops the crossfade");
}
/// The sweep runs on the sequences an edit changes only: a project saved with a stale transition
/// still opens and keeps it, in a sequence no command touched.
#[test]
fn edits_leave_untouched_sequences_alone() {
let mut s = Session::default();
let old = s.execute("file.newSequence", json!({"name": "Old", "video": 1, "audio": 0})).unwrap()["sequence"].as_u64().unwrap();
{
let q = std::sync::Arc::make_mut(&mut s.project).sequence_mut(ItemId(old)).unwrap();
q.video_tracks[0].transitions.push(filmcraft_project::Transition {
id: filmcraft_project::TransitionId(9999),
effect: filmcraft_project::find_effect("cross_dissolve").unwrap().instance(),
start: Tick::ZERO,
duration: Tick(1),
from: Some(ClipId(1)),
to: Some(ClipId(2)),
align: Default::default(),
reverse: false,
});
}
s.execute("file.newSequence", json!({"name": "New", "video": 1, "audio": 0})).unwrap();
let matte = s.execute("file.newColorMatte", json!({"color": "#00aa00", "seconds": 5.0})).unwrap()["item"].as_u64().unwrap();
s.execute("timeline.place", json!({"item": matte, "track": "V1", "time": 0})).unwrap();
assert_eq!(s.project.sequence(ItemId(old)).unwrap().video_tracks[0].transitions.len(), 1);
}
/// A damaged project whose subclip is its own parent (#66): `media_duration` followed the chain
/// without a bound and overflowed the stack. It gives up after a few hops now.
#[test]
@@ -1053,6 +1124,31 @@ fn linked_slip_keeps_picture_and_sound_aligned_at_unequal_media_limits() {
assert_eq!((clip(&s, v).source_in, clip(&s, a).source_in), (Tick(fr.0 * 10), Tick(fr.0 * 10)));
}
#[test]
fn slipping_by_the_most_negative_delta_clamps_instead_of_overflowing() {
// `slip_together` took `abs()` of the requested delta: i64::MIN (from the control channel, MCP
// or a script, as ticks or as frames, which saturate to it) overflowed, a panic in debug builds
let mut s = Session::default();
s.execute("file.newProject", json!({"name": "Slip"})).unwrap();
s.execute("file.newSequence", json!({"name": "Sequence", "fps": 25, "video": 1, "audio": 1, "width": 16, "height": 16})).unwrap();
let r = s.execute("file.newOfflineFile", json!({"name": "Media", "seconds": 4, "fps": 25, "video": true, "audio": true})).unwrap();
let item = r["item"].as_u64().unwrap();
let fr = s.sequence_rate().tick_of(1);
let r =
s.execute("timeline.place", json!({"item": item, "track": "V1", "audioTrack": "A1", "time": 0, "sourceIn": fr.0 * 20, "duration": fr.0 * 60})).unwrap();
let (v, a) = (r["clips"][0].as_u64().unwrap(), r["clips"][1].as_u64().unwrap());
for params in [json!({"clip": v, "delta": i64::MIN}), json!({"clip": a, "deltaFrames": i64::MIN})] {
s.execute("timeline.slip", params).unwrap();
// both stop at the start of the media, together
assert_eq!((clip(&s, v).source_in, clip(&s, a).source_in), (Tick::ZERO, Tick::ZERO));
// back off the start, so a case that slipped nothing could not pass on the previous one's result
s.execute("timeline.slip", json!({"clip": v, "deltaFrames": 20})).unwrap();
assert_eq!((clip(&s, v).source_in, clip(&s, a).source_in), (Tick(fr.0 * 20), Tick(fr.0 * 20)));
}
s.execute("timeline.slip", json!({"clip": v, "delta": i64::MAX})).unwrap();
assert_eq!((clip(&s, v).source_in, clip(&s, a).source_in), (Tick(fr.0 * 40), Tick(fr.0 * 40)), "and at its end the other way");
}
#[test]
fn reset_keeps_keyframes_and_resets_one_parameter() {
// #284: Reset Effect used to replace the effect with a fresh one, deleting every keyframe
@@ -1136,6 +1232,26 @@ fn linked_rate_stretch_keeps_one_speed_when_a_neighbour_blocks_the_sound() {
assert!((r["speed"].as_f64().unwrap() - 60.0 / 70.0).abs() < 1e-9);
}
/// `document_epoch` changes whenever another project replaces the session's (so work that
/// finishes later, like a Save dialog's answer, can tell), and never with an ordinary edit.
#[test]
fn document_epoch_changes_with_the_project_not_with_edits() {
let mut s = crate::Session::default();
let g = |s: &crate::Session| s.document_epoch;
let mut last = g(&s);
s.execute("file.newBin", serde_json::json!({"name": "B"})).unwrap();
assert_eq!(g(&s), last, "an edit");
for (cmd, p) in [
("file.openDemoProject", serde_json::json!({})),
("file.newProject", serde_json::json!({"name": "N"})),
("file.closeProject", serde_json::json!({"force": true})),
] {
s.execute(cmd, p).unwrap();
assert!(g(&s) > last, "{cmd}");
last = g(&s);
}
}
/// #2358: Toggle Animation on a parameter an older effect instance lacks (it was added to the
/// effect later) creates it from its default and animates it, instead of "no such param".
#[test]
@@ -1216,3 +1332,28 @@ fn video_effect_on_only_an_audio_clip_is_refused_without_an_undo_step() {
assert!(r.is_err(), "{r:?}");
assert_eq!(s.history.undo.len(), undo0, "no undo step");
}
/// `clip.rename` with `clips` renames every one of them (a linked picture and sound) in one step.
#[test]
fn rename_takes_several_clips() {
let mut s = demo();
let ids: Vec<u64> = v1(&s).iter().take(2).map(|c| c.id.0).collect();
let undo = s.history.undo.len();
s.execute("clip.rename", json!({"clips": ids, "name": "Both"})).unwrap();
for id in &ids {
assert_eq!(clip(&s, *id).name, "Both");
}
assert_eq!(s.history.undo.len(), undo + 1);
assert!(s.execute("clip.rename", json!({"clips": [ids[0], 999_999], "name": "x"})).is_err(), "an unknown clip renames nothing");
assert_eq!(clip(&s, ids[0]).name, "Both");
// nothing to rename and a blank name are refused without an undo step
let undo = s.history.undo.len();
assert!(s.execute("clip.rename", json!({"clips": [], "name": "x"})).is_err());
assert!(s.execute("clip.rename", json!({"clips": ids, "name": " "})).is_err());
// a list with one entry that is not an id renames nothing
assert!(s.execute("clip.rename", json!({"clips": [ids[0], "x"], "name": "y"})).is_err());
assert_eq!(clip(&s, ids[0]).name, "Both");
// the same name again changes nothing and records nothing
s.execute("clip.rename", json!({"clips": ids, "name": "Both"})).unwrap();
assert_eq!(s.history.undo.len(), undo);
}
crates/engine/src/color.rs
+2 -3
View File
@@ -392,7 +392,7 @@ fn list_presets(_: &mut Session, p: &Value) -> Result<Value> {
}
/// Lumetri Presets ▸ apply: a new Lumetri Color configured as the preset on each video clip
/// (standard effects go before the intrinsic ones, like `effects.apply`).
/// (added at the end of the clip's effects, like `effects.apply`).
fn apply_preset(s: &mut Session, p: &Value) -> Result<Value> {
let cmd = "lumetri.applyPreset";
let name = str_p(p, "name").ok_or_else(|| bad(cmd, "need `name`"))?;
@@ -406,8 +406,7 @@ fn apply_preset(s: &mut Session, p: &Value) -> Result<Value> {
let mut n = 0;
for t in q.video_tracks.iter_mut() {
for it in t.items.iter_mut().filter(|i| clips.contains(&i.id)) {
let pos = it.effects.iter().position(|e| e.def().is_some_and(|d| d.intrinsic)).unwrap_or(it.effects.len());
it.effects.insert(pos, inst.clone());
it.effects.push(inst.clone());
n += 1;
}
}
crates/engine/src/color_tests.rs
+2 -2
View File
@@ -203,14 +203,14 @@ fn lumetri_presets_list_apply_and_thumbnails() {
assert_eq!(r["folders"], json!(["Cinematic", "Film Emulation", "Monochrome", "Technical"]));
let mono = s.execute("lumetri.presets", json!({"folder": "monochrome"})).unwrap();
assert!(mono["presets"].as_array().unwrap().iter().all(|p| p["folder"] == "Monochrome"));
// apply: one undo step adding a configured Lumetri Color before the intrinsic effects
// apply: one undo step adding a configured Lumetri Color at the end of the clip's effects
let before = s.active_sequence().unwrap().find_item(clip).unwrap().1.effects.len();
let r = s.execute("lumetri.applyPreset", json!({"name": "Neutral Mono"})).unwrap();
assert_eq!(r["clips"], 1);
let it = s.active_sequence().unwrap().find_item(clip).unwrap().1.clone();
assert_eq!(it.effects.len(), before + 1);
let k = it.effects.iter().position(|e| e.effect == "lumetri").unwrap();
assert!(it.effects[k + 1..].iter().all(|e| e.def().is_some_and(|d| d.intrinsic)) || k + 1 == it.effects.len());
assert_eq!(k + 1, it.effects.len());
assert_eq!(it.effects[k].param("saturation").unwrap().value, ParamValue::Float(0.0));
let img = s.render_program(0.25).unwrap();
let grey = img.px.as_chunks::<4>().0.iter().filter(|p| p[3] > 0.5).all(|p| (p[0] - p[1]).abs() < 0.01 && (p[1] - p[2]).abs() < 0.01);
crates/engine/src/commands.rs
+354 -64
View File
@@ -9,7 +9,7 @@ use filmcraft_media::Generator;
use filmcraft_media::generators::GeneratorSource;
use filmcraft_project::{
ClipId, ItemId, ItemKind, Label, Marker, MarkerId, MarkerKind, MediaClip, MediaRef, ParamValue, Project, SequenceSettings, Track, TrackId, TrackKind,
Transition, TransitionId, resolve_auto_points,
Transition, TransitionAlign, TransitionId, resolve_auto_points,
};
use filmcraft_time::{FrameRate, TICKS_PER_SECOND, Tick, TimeRange, parse_timecode};
use serde_json::{Value, json};
@@ -190,9 +190,52 @@ pub(crate) fn named_items(s: &Session, spec: &CommandSpec, p: &Value) -> Option<
// ---------- param helpers ----------
/// The longest marker `markers.edit` takes, in frames (over 46 hours at 60 fps).
const MAX_MARKER_FRAMES: i64 = 10_000_000;
/// `markers.edit`'s `durationFrames`: absent or null is `None`; anything but a whole number from 0
/// to [`MAX_MARKER_FRAMES`] is an error.
fn marker_frames_p(p: &Value) -> Result<Option<i64>> {
match p.get("durationFrames").filter(|v| !v.is_null()) {
None => Ok(None),
Some(v) => match v.as_i64() {
Some(f @ 0..=MAX_MARKER_FRAMES) => Ok(Some(f)),
_ => Err(bad("markers.edit", "`durationFrames` must be a whole number from 0 to 10000000")),
},
}
}
/// `markers.edit`'s `kind`: absent or null is `None`; anything but one of the kind names is an
/// error.
pub(crate) fn marker_kind_p(p: &Value) -> Result<Option<MarkerKind>> {
match p.get("kind").filter(|v| !v.is_null()) {
None => Ok(None),
Some(v) => {
v.as_str().and_then(marker_kind).map(Some).ok_or_else(|| bad("markers.edit", "`kind` must be comment, chapter, segmentation, webLink or flashCue"))
}
}
}
/// A marker type by its `markers.edit` name.
fn marker_kind(name: &str) -> Option<MarkerKind> {
Some(match name {
"comment" => MarkerKind::Comment,
"chapter" => MarkerKind::Chapter,
"segmentation" => MarkerKind::Segmentation,
"webLink" => MarkerKind::WebLink,
"flashCue" => MarkerKind::FlashCue,
_ => return None,
})
}
pub(crate) fn bad(cmd: &str, msg: impl Into<String>) -> EngineError {
EngineError::BadParams { cmd: cmd.into(), msg: msg.into() }
}
/// A call that only reports, so it is not journaled although its command edits: `sequence.settings`
/// with nothing to change (`null` or `{}`) answers the settings, as `sequence.inspect` does (#1540).
pub(crate) fn reports_only(id: &str, p: &Value) -> bool {
id == "sequence.settings" && (p.is_null() || p.as_object().is_some_and(|m| m.is_empty()))
}
/// A marker's `color` param: absent (or null) is `None`; anything but a label name is an error
/// that lists the accepted names, instead of silently becoming Green (#857).
pub(crate) fn marker_color_p(p: &Value, cmd: &str) -> Result<Option<Label>> {
@@ -394,7 +437,10 @@ pub(crate) fn item_p(p: &Value, k: &str) -> Option<ItemId> {
/// before and after (ticks) and the clips what is left of it now lives on as (none = removed; a
/// clip a moved clip lands inside is split in two). With `copy: true` (Option-drag) the clips stay
/// where they are and copies land at the destinations instead; the copies are selected and listed
/// as `copied`.
/// as `copied`. `captions` (caption ids) move with them by the same offset in the same undo step,
/// and are listed as `captions` when they moved; if they cannot go that far (they would overlap
/// captions that stay, or sit on a locked track) or an entry is not a caption, nothing moves. Not
/// with `copy`, and not with `insert`, which already carries the sync-locked caption tracks.
fn move_clips(s: &mut Session, p: &Value) -> Result<Value> {
const CMD: &str = "timeline.move";
let mut listed: Vec<(ClipId, TrackId, Tick, bool)> = Vec::new();
@@ -421,6 +467,13 @@ fn move_clips(s: &mut Session, p: &Value) -> Result<Value> {
if listed.is_empty() {
return Err(bad(CMD, "need `moves`"));
}
// One destination per clip: a clip listed twice would end up on both tracks under one id
// (#1903). Copies may repeat a source, each copy getting its own id.
if !bool_p(p, "copy").unwrap_or(false)
&& let Some(dup) = listed.iter().enumerate().find_map(|(i, m)| listed[..i].iter().any(|o| o.0 == m.0).then_some(m.0))
{
return Err(bad(CMD, format!("clip {} is listed more than once; a move takes one destination per clip (copies may repeat a source)", dup.0)));
}
let follow = bool_p(p, "linked").unwrap_or(s.state.linked_selection);
let q = s.active_sequence().ok_or(EngineError::NoSequence)?;
// (clip, destination track, new start, where a partner starts now, new track requested).
@@ -440,6 +493,23 @@ fn move_clips(s: &mut Session, p: &Value) -> Result<Value> {
}
}
let late = plan.iter().map(|m| -m.2).max().unwrap_or(0).max(0);
// `captions` ride along by the first listed clip's offset (shift above included), so subtitles
// stay on the dialogue they belong to
let caption_ids = caption_ids_p(p)?;
if !caption_ids.is_empty() && bool_p(p, "copy").unwrap_or(false) {
return Err(bad(CMD, "`captions` cannot be combined with `copy`"));
}
// an insert already carries the unlocked, sync-locked caption tracks; moving captions as well would shift them twice
if !caption_ids.is_empty() && bool_p(p, "insert").unwrap_or(false) {
return Err(bad(CMD, "`captions` cannot be combined with `insert`: an insert already moves sync-locked captions"));
}
let caption_delta = match plan.first() {
Some(&(clip, _, start, ..)) if !caption_ids.is_empty() => {
let from = q.find_item(clip).map(|(_, it)| i128::from(it.start.0)).ok_or(edit::EditError::NoItem(clip))?;
Tick(i64::try_from(start + late - from).map_err(|_| bad(CMD, "`time` is out of range"))?)
}
_ => Tick::ZERO,
};
let mut moves: Vec<(ClipId, TrackId, Tick, bool)> = Vec::with_capacity(plan.len());
for (clip, track, start, was, new_track) in plan {
// a partner that ends up where it is does not move (its listed clip only changed track):
@@ -499,7 +569,18 @@ fn move_clips(s: &mut Session, p: &Value) -> Result<Value> {
} else {
s.edit_sequence(if ins { "Move (Insert)" } else { "Move" }, |q, ctx, _| {
let destinations = prepare(q, ctx)?;
Ok(edit::move_items(q, &destinations, ins, ctx)?)
edit::move_items(q, &destinations, ins, ctx)?;
// all or nothing: captions that cannot go as far as the clips would drift off their dialogue.
// The ids are checked (they exist, are not locked) even when the clips do not change time.
if !caption_ids.is_empty() {
match edit::captions::move_captions(q, &caption_ids, caption_delta) {
Ok(applied) if applied == caption_delta => {}
Ok(_) => return Err(bad(CMD, "the captions cannot move as far as the clips (they would overlap other captions)")),
Err(edit::EditError::Nothing) if caption_delta == Tick::ZERO => {}
Err(e) => return Err(e.into()),
}
}
Ok(())
})?;
Vec::new()
};
@@ -523,7 +604,27 @@ fn move_clips(s: &mut Session, p: &Value) -> Result<Value> {
if copy {
return Ok(json!({"copied": copied.iter().map(|c| c.0).collect::<Vec<_>>(), "overwritten": overwritten}));
}
Ok(json!({"moved": moves.iter().map(|m| m.0.0).collect::<Vec<_>>(), "overwritten": overwritten}))
let mut result = json!({"moved": moves.iter().map(|m| m.0.0).collect::<Vec<_>>(), "overwritten": overwritten});
// only the captions that moved
if caption_delta != Tick::ZERO {
result["captions"] = json!(caption_ids.iter().map(|c| c.0).collect::<Vec<_>>());
}
Ok(result)
}
/// `captions` of `timeline.move`: absent means none; otherwise an array of caption ids, each kept once.
fn caption_ids_p(p: &Value) -> Result<Vec<ClipId>> {
let Some(v) = p.get("captions") else { return Ok(Vec::new()) };
let list = v.as_array().ok_or_else(|| bad("timeline.move", "`captions` must be an array of caption ids"))?;
let mut seen = std::collections::HashSet::new();
let mut ids = Vec::with_capacity(list.len().min(1024));
for entry in list {
let id = entry.as_u64().ok_or_else(|| bad("timeline.move", "`captions` entries must be caption ids"))?;
if seen.insert(id) {
ids.push(ClipId(id));
}
}
Ok(ids)
}
/// Slip clips together: the media delta of each clip is scaled down by the same factor so that the
@@ -540,7 +641,8 @@ pub(crate) fn slip_together(q: &mut filmcraft_project::Sequence, ctx: &mut edit:
Err(e) => return Err(e.into()),
Ok(x) => {
any = true;
scale = scale.min(x.0.abs() as f64 / d.0.abs() as f64);
// unsigned: `abs` of i64::MIN (a hostile `delta`) overflows
scale = scale.min(x.0.unsigned_abs() as f64 / d.0.unsigned_abs() as f64);
}
}
}
@@ -789,6 +891,44 @@ pub(crate) fn default_seq_settings_for(media: &MediaClip) -> SequenceSettings {
st
}
/// Settings ▸ Timeline ▸ Show Clip Mismatch Warning dialog: the settings the active sequence takes
/// to match `item` when that is the first clip put in it, its frame size, frame rate and pixel
/// aspect ratio (Interpret Footage's, else the file's), the sequence's other settings kept. `None`
/// when the sequence already holds clips, the item has no moving picture (sound alone, a still, a
/// sequence) or nothing differs. `matchSequence` on `timeline.place`, `source.insert` and
/// `source.overwrite` applies it.
pub fn clip_mismatch(s: &Session, item: ItemId) -> Option<SequenceSettings> {
let q = s.active_sequence()?;
if q.all_tracks().any(|t| !t.items.is_empty()) {
return None;
}
let (_, m, _) = s.project.resolve_media(item)?;
let v = m.info.video.as_ref().filter(|_| m.info.kind != filmcraft_media::MediaKind::Still)?;
let mut st = q.settings.clone();
(st.width, st.height, st.frame_rate, st.par) = (v.width, v.height, m.frame_rate(), m.pixel_aspect());
if (st.width, st.height, st.frame_rate, st.par) == (q.settings.width, q.settings.height, q.settings.frame_rate, q.settings.par) {
return None;
}
st.drop_frame &= st.frame_rate.supports_drop_frame();
st.preset = format!("{}x{} {}", st.width, st.height, st.frame_rate.label());
// a damaged file's size or rate must not reach the sequence
st.validate().ok().map(|()| st)
}
/// `matchSequence`: before the edit, the active (still empty) sequence takes the clip's settings
/// ([`clip_mismatch`]), in the same undo step as the edit.
fn edit_matching_sequence(s: &mut Session, p: &Value, item: Option<ItemId>, label: &str, f: impl FnOnce(&mut Session) -> Result<Value>) -> Result<Value> {
let Some(st) = item.filter(|_| bool_p(p, "matchSequence") == Some(true)).and_then(|i| clip_mismatch(s, i)) else { return f(s) };
let id = s.state.active_sequence.ok_or(EngineError::NoSequence)?;
s.edit_group(label, |s| {
s.edit("Sequence Settings", |pr, _| {
pr.sequence_mut(id).ok_or(EngineError::NoSequence)?.settings = st;
Ok(())
})?;
f(s)
})
}
/// Import bytes as a media item (engine-level; the UI reads files through services).
pub fn import_bytes(s: &mut Session, path: &str, bytes: std::sync::Arc<[u8]>, bin: Option<filmcraft_project::BinId>) -> Result<ItemId> {
let name = std::path::Path::new(path).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_else(|| path.to_string());
@@ -1137,6 +1277,34 @@ fn place_items(s: &mut Session, p: &Value, items: &Value) -> Result<Value> {
})
}
impl Session {
/// How many video and audio tracks placing `item` (its `range`) fills, counted from the
/// destination tracks: video lanes go up from the video destination, audio lanes down from the
/// audio destination. It follows [`place_item`]: one lane per stream the item carries, one
/// audio lane per Modify ▸ Audio Channels clip and further container stream, and with the nest
/// toggle off one lane per source-sequence track that has clips in `range`. `video` / `audio`
/// off leave that kind out. Used by the Timeline's drop preview (#1887).
pub fn placement_lanes(&self, item: ItemId, range: TimeRange, video: bool, audio: bool) -> (usize, usize) {
let Some(pi) = self.project.item(item) else { return (0, 0) };
if self.state.sequences_as_clips
&& let Some(q) = pi.as_sequence().filter(|q| q.multicam.is_none())
{
let lanes = |tracks: &[Track]| tracks.iter().filter(|t| t.items.iter().any(|it| edit::through::clip_item_to(it, range).is_some())).count();
return (if video { lanes(&q.video_tracks) } else { 0 }, if audio { lanes(&q.audio_tracks) } else { 0 });
}
let (has_video, has_audio) = self.project.source_streams(item);
let audio_lanes = if audio && has_audio {
let media = self.project.resolve_media(item);
let clips = media.and_then(|(_, m, _)| m.interpret.audio_channels.as_ref()).map(|m| m.clips.clone()).unwrap_or_default();
let streams = media.map_or(1, |(_, m, _)| m.info.audio_streams.len());
audio_placement_specs(&clips, streams).len()
} else {
0
};
(usize::from(video && has_video), audio_lanes)
}
}
/// Place a project item on the timeline (drag from Project, or Insert/Overwrite from source).
pub(crate) struct ItemPlacement<'a> {
pub(crate) item: ItemId,
@@ -1483,7 +1651,7 @@ fn build() -> Vec<CommandSpec> {
Label::Lavender,
p
)),
cmd!("file.newColorMatte", "Color Matte…", ["File", "New"], None, r##"{"color":"#rrggbb","seconds":f64}"##, always, |s, p| {
cmd!("file.newColorMatte", "Color Matte…", ["File", "New"], None, r##"{"color":"#rrggbb"?,"seconds":f64=5}"##, always, |s, p| {
let c = str_p(p, "color").and_then(filmcraft_color::parse_hex).unwrap_or([0.1, 0.1, 0.1, 1.0]);
new_generator(s, Generator::ColorMatte { color: c }, "Color Matte", Label::Lavender, p)
}),
@@ -1572,7 +1740,7 @@ fn build() -> Vec<CommandSpec> {
Err(e) => errors.push(format!("{path}: {e}")),
}
} else if let Some(fmt) = crate::interchange::detect(&path, &b) {
match crate::interchange::import(s, &path, &b, fmt) {
match crate::interchange::import(s, &path, &b, fmt, bin) {
Ok(r) => {
sequences.extend(r["sequences"].as_array().cloned().unwrap_or_default());
reports.push(r);
@@ -1684,7 +1852,9 @@ fn build() -> Vec<CommandSpec> {
has_seq,
|s, p| crate::aaf_omf::export_omf(s, p)
),
cmd!("file.importAaf", "Import AAF…", [], None, r#"{"path":str}"#, always, |s, p| crate::aaf_omf::import_document(s, p, "file.importAaf")),
cmd!("file.importAaf", "Import AAF…", [], None, r#"{"path":str,"bin":binId?}"#, always, |s, p| {
crate::aaf_omf::import_document(s, p, "file.importAaf")
}),
cmd!("file.save", "Save", ["File"], Some("Cmd+S"), r#"{"path":str?}"#, always, |s, p| {
let path = str_p(p, "path").map(str::to_string).or_else(|| s.path.clone()).ok_or_else(|| bad("file.save", "no path (use Save As)"))?;
write_project(s, &path, true)
@@ -1754,7 +1924,7 @@ fn build() -> Vec<CommandSpec> {
query!("file.autoSaveStatus", "Auto Save Status", "{}", |s, _| {
let Some(per) = s.persistence.as_ref() else { return Ok(json!({"running": false, "prefs": s.prefs.auto_save})) };
Ok(json!({
"running": true,
"running": per.is_running(),
"prefs": s.prefs.auto_save,
"dataDir": per.data_dir,
"sessionDir": per.session_dir,
@@ -2022,16 +2192,37 @@ fn build() -> Vec<CommandSpec> {
Ok(Value::Null)
}),
// ================= Clip =================
cmd!("clip.rename", "Rename…", ["Clip"], None, r#"{"clip":id?,"item":id?,"name":str}"#, always, |s, p| {
let name = str_p(p, "name").ok_or_else(|| bad("clip.rename", "need `name`"))?.to_string();
cmd!("clip.rename", "Rename…", ["Clip"], None, r#"{"clip":id?,"clips":[id]?,"item":id?,"name":str?}"#, always, |s, p| {
// a blank name is refused, as the Project panel's inline rename keeps the old one
let name = str_p(p, "name").map(str::trim).filter(|n| !n.is_empty()).ok_or_else(|| bad("clip.rename", "need a non-empty `name`"))?.to_string();
let item = item_p(p, "item");
let clip = clip_p(p, "clip").or_else(|| s.state.selection.first().copied());
// `clips` renames them all (a linked picture and sound, as Clip ▸ Rename… does)
let clips: Vec<ClipId> = match p.get("clips").and_then(Value::as_array) {
// an explicit list is renamed as a whole: one entry that is not a clip id refuses it
Some(a) => a.iter().map(|v| v.as_u64().map(ClipId)).collect::<Option<_>>().ok_or_else(|| bad("clip.rename", "`clips` must be clip ids"))?,
None => clip_p(p, "clip").or_else(|| s.state.selection.first().copied()).into_iter().collect(),
};
// nothing to rename is refused before the edit, so no empty undo step is recorded
if item.is_none() && clips.is_empty() {
return Err(bad("clip.rename", "need a clip or an item"));
}
// OK on an unchanged name records no undo step and leaves the project unmodified
let unchanged = match item {
Some(i) => s.project.item(i).is_some_and(|it| it.name == name),
None => s.active_sequence().is_some_and(|q| clips.iter().all(|c| q.find_item(*c).is_some_and(|(_, it)| it.name == name))),
};
if unchanged {
return Ok(Value::Null);
}
let seq = s.state.active_sequence;
s.edit("Rename", |pr, _| {
if let Some(i) = item {
pr.item_mut(i).ok_or_else(|| bad("clip.rename", "no such item"))?.name = name;
} else if let (Some(c), Some(q)) = (clip, seq) {
pr.sequence_mut(q).and_then(|q| q.find_item_mut(c)).ok_or_else(|| bad("clip.rename", "no such clip"))?.1.name = name;
} else {
let q = seq.and_then(|q| pr.sequence_mut(q)).ok_or_else(|| bad("clip.rename", "no active sequence"))?;
for c in &clips {
q.find_item_mut(*c).ok_or_else(|| bad("clip.rename", "no such clip"))?.1.name = name.clone();
}
}
Ok(())
})?;
@@ -2243,7 +2434,7 @@ fn build() -> Vec<CommandSpec> {
cmd!("clip.frameHold", "Add Frame Hold", ["Clip", "Video Options"], None, r#"{"clips":[id]?,"time":ticks?}"#, has_seq, |s, p| {
crate::clip_ops::add_frame_hold(s, p)
}),
cmd!("clip.nest", "Nest…", ["Clip"], None, r#"{"name":str}"#, has_selection, |s, p| nest(s, p)),
cmd!("clip.nest", "Nest…", ["Clip"], None, r#"{"name":str?}"#, has_selection, |s, p| nest(s, p)),
// Reveal in Project (clip context menu) and Reveal Sequence in Project (Timeline tab menu):
// select the item in the Project panel and show it there
cmd!("clip.revealInProject", "Reveal in Project", [], None, r#"{"clip":id?}"#, has_seq, |s, p| {
@@ -2366,7 +2557,7 @@ fn build() -> Vec<CommandSpec> {
"Apply Video Transition",
["Sequence"],
Some("Cmd+D"),
r#"{"clip":id?,"effect":"cross_dissolve|Cross Dissolve|…"?,"frames":i64?,"edge":"in"|"out"?,"params":{param:value}?,"reverse":bool?}"#,
r#"{"clip":id?,"effect":"cross_dissolve|Cross Dissolve|…"?,"frames":i64?,"edge":"in"|"out"?,"align":"center"|"start"|"end"?,"params":{param:value}?,"reverse":bool?}"#,
has_seq,
|s, p| apply_transition(s, p, TrackKind::Video)
),
@@ -2375,7 +2566,7 @@ fn build() -> Vec<CommandSpec> {
"Apply Audio Transition",
["Sequence"],
Some("Cmd+Shift+D"),
r#"{"clip":id?,"effect":"constant_power|Constant Power|…"?,"frames":i64?,"edge":"in"|"out"?,"params":{param:value}?,"reverse":bool?}"#,
r#"{"clip":id?,"effect":"constant_power|Constant Power|…"?,"frames":i64?,"edge":"in"|"out"?,"align":"center"|"start"|"end"?,"params":{param:value}?,"reverse":bool?}"#,
has_seq,
|s, p| apply_transition(s, p, TrackKind::Audio)
),
@@ -2454,6 +2645,15 @@ fn build() -> Vec<CommandSpec> {
|s, p| {
const CMD: &str = "sequence.settings";
let id = s.state.active_sequence.ok_or(EngineError::NoSequence)?;
// nothing to change: report the settings as `sequence.inspect` does, without an empty
// undo step (#1540)
if reports_only(CMD, p) {
let q = s.project.sequence(id).ok_or(EngineError::NoSequence)?;
return Ok(json!({"settings": q.settings}));
}
if !p.is_object() {
return Err(bad(CMD, "`params` must be an object"));
}
let fps = fps_p(p, CMD)?;
let mix = match str_p(p, "mix") {
Some(m) => Some(crate::mixer::channels_from(m).ok_or_else(|| bad(CMD, format!("unknown mix `{m}` (Stereo, Mono, 5.1, Adaptive)")))?),
@@ -2779,9 +2979,12 @@ fn build() -> Vec<CommandSpec> {
"Edit Marker…",
[],
None,
r#"{"target":"program|source","marker":id,"name":str?,"comment":str?,"color":label?,"durationFrames":i64?}"#,
r#"{"target":"program|source","marker":id,"name":str?,"comment":str?,"color":label?,"durationFrames":i64?,"kind":"comment|chapter|segmentation|webLink|flashCue"?}"#,
always,
|s, p| {
// checked before routing, so Source markers are held to the same limits
let frames = marker_frames_p(p)?;
let kind = marker_kind_p(p)?;
if let Some(v) = crate::source_monitor::route(s, "markers.edit", p)? {
return Ok(v);
}
@@ -2800,9 +3003,12 @@ fn build() -> Vec<CommandSpec> {
if let Some(c) = color {
m.color = c;
}
if let Some(f) = p.get("durationFrames").and_then(Value::as_i64) {
if let Some(f) = frames {
m.duration = rate.tick_of(f);
}
if let Some(k) = kind {
m.kind = k;
}
Ok(())
})?;
Ok(Value::Null)
@@ -2886,19 +3092,29 @@ fn build() -> Vec<CommandSpec> {
let v = crate::clip_ops::source_view(s, item).ok_or_else(|| bad("source.inspect", "the source item is gone"))?;
Ok(v.to_json(s.state.source_playhead))
}),
cmd!("source.insert", "Insert", ["Clip"], Some(","), "{}", has_source, |s, _| edit_at_source(s, true)),
cmd!("source.overwrite", "Overwrite", ["Clip"], Some("."), "{}", has_source, |s, _| edit_at_source(s, false)),
cmd!("source.insert", "Insert", ["Clip"], Some(","), r#"{"matchSequence":bool=false}"#, has_source, |s, p| {
let item = s.state.source_item;
edit_matching_sequence(s, p, item, "Insert", |s| edit_at_source(s, true))
}),
cmd!("source.overwrite", "Overwrite", ["Clip"], Some("."), r#"{"matchSequence":bool=false}"#, has_source, |s, p| {
let item = s.state.source_item;
edit_matching_sequence(s, p, item, "Overwrite", |s| edit_at_source(s, false))
}),
// ================= Timeline gestures =================
cmd!(
"timeline.place",
"Place Clip",
[],
None,
r#"{"item":id|"items":[id] (back to back, one undo step),"track":"V1"|id|"A1" (sound only)|"new" (a video track after the last)?,"audioTrack":"A1"|id|"new"?,"time":ticks|"frame":i64|"seconds":f64,"insert":bool,"sourceIn":ticks|{"frame"|"seconds"|"timecode"}?,"duration":ticks|{"frame"|"seconds"|"timecode"}?,"video":bool=true,"audio":bool=true}"#,
r#"{"item":id|"items":[id] (back to back, one undo step),"track":"V1"|id|"A1" (sound only)|"new" (a video track after the last)?,"audioTrack":"A1"|id|"new"?,"time":ticks|"frame":i64|"seconds":f64,"insert":bool,"sourceIn":ticks|{"frame"|"seconds"|"timecode"}?,"duration":ticks|{"frame"|"seconds"|"timecode"}?,"video":bool=true,"audio":bool=true,"matchSequence":bool=false (an empty sequence first takes the clip's frame size and rate)}"#,
has_seq,
|s, p| match p.get("items") {
Some(items) => place_items(s, p, items),
None => place_one(s, p),
|s, p| {
let first = item_p(p, "item").or_else(|| p.get("items").and_then(Value::as_array).and_then(|a| a.first()).and_then(Value::as_u64).map(ItemId));
let label = if p.get("items").is_some() { "Place Clips" } else { "Place Clip" };
edit_matching_sequence(s, p, first, label, |s| match p.get("items") {
Some(items) => place_items(s, p, items),
None => place_one(s, p),
})
}
),
cmd!(
@@ -2960,7 +3176,7 @@ fn build() -> Vec<CommandSpec> {
"Move Clips",
[],
None,
r#"{"moves":[{"clip":id,"track":id|"V2"|"new" (after the last of its kind),"time":ticks}],"insert":bool,"linked":bool?,"copy":bool?}"#,
r#"{"moves":[{"clip":id,"track":id|"V2"|"new" (after the last of its kind),"time":ticks}],"insert":bool,"linked":bool?,"copy":bool?,"captions":[id]?}"#,
has_seq,
move_clips
),
@@ -3302,10 +3518,11 @@ fn build() -> Vec<CommandSpec> {
// intrinsic effects (Motion) are laid out in the sequence frame, as `reset_params` does
let frame = if def.intrinsic { (fw, fh) } else { size };
resolve_auto_points(&mut inst, frame, size);
// standard effects go before the intrinsic ones (render order)
let pos = i.effects.iter().position(|e| e.def().is_some_and(|d| d.intrinsic)).unwrap_or(i.effects.len());
let pos = if is_audio { i.effects.len() } else { pos };
i.effects.insert(pos, inst);
// at the end of the stack, as in Premiere, so the indices already
// handed out (graphics.list effectIndex…) stay valid (#1879); the
// renderer applies the intrinsic ones after the standard ones anyway
i.effects.push(inst);
let pos = i.effects.len() - 1;
applied.push(json!({"clip": i.id.0, "index": pos, "effect": def.id}));
}
}
@@ -3384,7 +3601,7 @@ fn build() -> Vec<CommandSpec> {
}
let (keyframes, value) = s.edit_sequence_as("Change Effect Parameter", merge.as_deref(), |q, _, _| {
let (_, it) = q.find_item_mut(c).ok_or(filmcraft_edit::EditError::NoItem(c))?;
let mt = it.source_time_at(at.unwrap_or_else(|| ph.clamp(it.start, (it.end() - Tick(1)).max(it.start))));
let mt = it.effect_time_at(at.unwrap_or_else(|| ph.clamp(it.start, (it.end() - Tick(1)).max(it.start))));
let e = match &eff {
Value::Number(n) => it.effects.get_mut(n.as_u64().unwrap_or(0) as usize),
Value::String(sid) => it.effects.iter_mut().find(|e| &e.effect == sid),
@@ -3465,7 +3682,7 @@ fn build() -> Vec<CommandSpec> {
let ph = s.playhead();
s.edit_sequence("Toggle Animation", |q, _, _| {
let (_, it) = q.find_item_mut(c).ok_or(filmcraft_edit::EditError::NoItem(c))?;
let mt = it.source_time_at(ph.clamp(it.start, (it.end() - Tick(1)).max(it.start)));
let mt = it.effect_time_at(ph.clamp(it.start, (it.end() - Tick(1)).max(it.start)));
let e = match &eff {
Value::Number(n) => it.effects.get_mut(n.as_u64().unwrap_or(0) as usize),
Value::String(sid) => it.effects.iter_mut().find(|e| &e.effect == sid),
@@ -3505,7 +3722,7 @@ fn build() -> Vec<CommandSpec> {
let ph = s.playhead();
s.edit_sequence("Reset Effect", |q, _, _| {
let (_, it) = q.find_item_mut(c).ok_or(filmcraft_edit::EditError::NoItem(c))?;
let mt = it.source_time_at(ph.clamp(it.start, (it.end() - Tick(1)).max(it.start)));
let mt = it.effect_time_at(ph.clamp(it.start, (it.end() - Tick(1)).max(it.start)));
if let Some(e) = it.effects.get_mut(idx) {
reset_params(e, None, frame, source, mt);
}
@@ -3522,7 +3739,7 @@ fn build() -> Vec<CommandSpec> {
let ph = s.playhead();
s.edit_sequence("Reset Parameter", |q, _, _| {
let (_, it) = q.find_item_mut(c).ok_or(filmcraft_edit::EditError::NoItem(c))?;
let mt = it.source_time_at(ph.clamp(it.start, (it.end() - Tick(1)).max(it.start)));
let mt = it.effect_time_at(ph.clamp(it.start, (it.end() - Tick(1)).max(it.start)));
let e = match &eff {
Value::Number(n) => it.effects.get_mut(n.as_u64().unwrap_or(0) as usize),
Value::String(sid) => it.effects.iter_mut().find(|e| &e.effect == sid),
@@ -3571,9 +3788,9 @@ fn build() -> Vec<CommandSpec> {
"Keyframe Interpolation",
[],
None,
r#"{"clip":id,"effect":str|index,"param":str,"mediaTime":ticks,"interpolation":"linear|bezier|autoBezier|continuousBezier|hold|easeIn|easeOut"}"#,
r#"{"clip":id,"effect":str|index,"param":str,"mediaTime":ticks,"interpolation":"linear|bezier|autoBezier|continuousBezier|hold|easeIn|easeOut"}|{"keyframes":[{"clip":id,"effect":index,"param":str,"mask":n?,"time":mediaTicks}],"interpolation":str}"#,
has_seq,
|s, p| keyframe_op(s, p, "interp")
|s, p| if p.get("keyframes").is_some() { crate::keyframe_move::interpolate_many(s, p) } else { keyframe_op(s, p, "interp") }
),
cmd!(
"effects.setKeyframe",
@@ -3676,7 +3893,6 @@ fn build() -> Vec<CommandSpec> {
if let Some(q) = pr.sequence_mut(sid) {
for t in q.all_tracks_mut() {
t.items.retain(|x| !gone.contains(&x.item));
filmcraft_edit::remove_orphan_transitions(t);
}
}
}
@@ -3830,6 +4046,19 @@ fn reset_params(e: &mut filmcraft_project::EffectInstance, only: Option<&str>, f
any
}
/// The `interpolation` parameter; a missing or unknown name is linear.
pub(crate) fn interpolation_p(p: &Value) -> filmcraft_project::Interpolation {
match str_p(p, "interpolation").unwrap_or("linear").to_ascii_lowercase().as_str() {
"bezier" => filmcraft_project::Interpolation::Bezier,
"autobezier" => filmcraft_project::Interpolation::AutoBezier,
"continuousbezier" => filmcraft_project::Interpolation::ContinuousBezier,
"hold" => filmcraft_project::Interpolation::Hold,
"easein" => filmcraft_project::Interpolation::EaseIn,
"easeout" => filmcraft_project::Interpolation::EaseOut,
_ => filmcraft_project::Interpolation::Linear,
}
}
/// `effects.addKeyframe` / `deleteKeyframe` / `moveKeyframe` / `setKeyframe` / `setKeyframeInterpolation`.
/// An explicit `time` keeps its place, outside the clip too (#1055, as in `effects.setParam`); only
/// the playhead fallback is pinned to the clip's last tick. `add` replies `{added, mediaTime}`:
@@ -3843,7 +4072,7 @@ fn keyframe_op(s: &mut Session, p: &Value, op: &str) -> Result<Value> {
let playhead = s.playhead();
let mtime = p.get("mediaTime").and_then(Value::as_i64).map(Tick);
let to = p.get("to").and_then(Value::as_i64).map(Tick);
let interp = str_p(p, "interpolation").map(str::to_string);
let interp = interpolation_p(p);
let label = match op {
"add" => "Add Keyframe",
"delete" => "Delete Keyframe",
@@ -3859,7 +4088,7 @@ fn keyframe_op(s: &mut Session, p: &Value, op: &str) -> Result<Value> {
s.edit_sequence_as(label, merge.as_deref(), |q, _, _| {
let mut reply = Value::Null;
let (_, it) = q.find_item_mut(c).ok_or(filmcraft_edit::EditError::NoItem(c))?;
let mt_now = it.source_time_at(at.unwrap_or_else(|| playhead.clamp(it.start, (it.end() - Tick(1)).max(it.start))));
let mt_now = it.effect_time_at(at.unwrap_or_else(|| playhead.clamp(it.start, (it.end() - Tick(1)).max(it.start))));
let e = match &eff {
Value::Number(n) => it.effects.get_mut(n.as_u64().unwrap_or(0) as usize),
Value::String(sid) => it.effects.iter_mut().find(|e| &e.effect == sid),
@@ -3909,17 +4138,8 @@ fn keyframe_op(s: &mut Session, p: &Value, op: &str) -> Result<Value> {
}
_ => {
let at = mtime.unwrap_or(mt_now);
let kind = match interp.as_deref().unwrap_or("linear").to_ascii_lowercase().as_str() {
"bezier" => filmcraft_project::Interpolation::Bezier,
"autobezier" => filmcraft_project::Interpolation::AutoBezier,
"continuousbezier" => filmcraft_project::Interpolation::ContinuousBezier,
"hold" => filmcraft_project::Interpolation::Hold,
"easein" => filmcraft_project::Interpolation::EaseIn,
"easeout" => filmcraft_project::Interpolation::EaseOut,
_ => filmcraft_project::Interpolation::Linear,
};
let k = prm.keyframes.iter_mut().find(|k| k.time == at).ok_or_else(|| bad("keyframe", "no keyframe at that time"))?;
k.interp = kind;
k.interp = interp;
}
}
Ok(reply)
@@ -4054,6 +4274,9 @@ fn copy_selection(s: &mut Session) {
Vec::new()
};
let mut clip = Vec::new();
// transitions go along with their clips: those whose every clip is copied (#3450)
let mut transitions = Vec::new();
let copied = |c: Option<ClipId>| c.is_none_or(|c| sel.contains(&c));
for (kind, tracks) in [(TrackKind::Video, &q.video_tracks), (TrackKind::Audio, &q.audio_tracks)] {
for (ti, t) in tracks.iter().enumerate() {
for i in t.items.iter().filter(|i| sel.contains(&i.id)) {
@@ -4061,9 +4284,13 @@ fn copy_selection(s: &mut Session) {
c.start -= min;
clip.push((kind, ti, c));
}
for x in t.transitions.iter().filter(|x| (x.from.is_some() || x.to.is_some()) && copied(x.from) && copied(x.to)) {
transitions.push((kind, ti, Transition { start: x.start - min, ..x.clone() }));
}
}
}
set_clipboard(&mut s.state, (clip, markers));
s.state.clipboard_transitions = transitions;
}
/// Timeline clips (track kind, track index, item) and the sequence markers that go with them.
@@ -4072,6 +4299,7 @@ type Clipboard = (Vec<(TrackKind, usize, filmcraft_project::TrackItem)>, Vec<Mar
fn set_clipboard(st: &mut crate::EditorState, (clips, markers): Clipboard) {
st.clipboard = clips;
st.clipboard_markers = markers;
st.clipboard_transitions.clear();
st.caption_clipboard.clear();
}
@@ -4122,17 +4350,20 @@ fn paste(s: &mut Session, insert: bool) -> Result<Value> {
let at = s.playhead();
let clip = s.state.clipboard.clone();
let markers = if s.state.copy_paste_sequence_markers { s.state.clipboard_markers.clone() } else { Vec::new() };
let transitions = s.state.clipboard_transitions.clone();
let end = s.edit_sequence(if insert { "Paste Insert" } else { "Paste" }, |q, ctx, st| {
let mut placements = Vec::new();
let mut links = std::collections::HashMap::new();
// pasted clips get groups of their own, as their links: sharing the originals' group
// would select (and so drag or delete) the originals with them
let mut groups = std::collections::HashMap::new();
let mut new_ids = std::collections::HashMap::new();
for (kind, ti, item) in clip {
let tracks = q.tracks(kind);
let Some(t) = tracks.get(ti).or(tracks.last()) else { continue };
let mut it = item.clone();
it.id = ClipId(ctx.alloc());
new_ids.insert(item.id, it.id);
it.start += at;
if let Some(l) = it.link {
it.link = Some(*links.entry(l).or_insert_with(|| ctx.alloc()));
@@ -4144,6 +4375,19 @@ fn paste(s: &mut Session, insert: bool) -> Result<Value> {
}
let end = placements.iter().map(|p| p.1.end()).max().unwrap_or(at);
let ids = if insert { edit::insert(q, placements, ctx)? } else { edit::overwrite(q, placements, ctx)? };
// the copied transitions go onto the pasted clips, on the track those clips landed on
for (_, _, x) in &transitions {
let from = x.from.and_then(|c| new_ids.get(&c).copied());
let to = x.to.and_then(|c| new_ids.get(&c).copied());
let tracks: Vec<TrackId> = [from, to].into_iter().flatten().filter_map(|c| q.find_item(c).map(|(t, _)| t)).collect();
let Some(&track) = tracks.first() else { continue };
if tracks.len() != [x.from, x.to].into_iter().flatten().count() || tracks.iter().any(|t| *t != track) {
continue;
}
let Some(t) = q.track_mut(track) else { continue };
t.transitions.push(Transition { id: TransitionId(ctx.alloc()), start: x.start + at, from, to, ..x.clone() });
t.sort();
}
if insert && st.ripple_sequence_markers {
crate::sequence_tools::ripple_markers(&mut q.markers, at, end - at);
}
@@ -4262,7 +4506,6 @@ fn nest(s: &mut Session, p: &Value) -> Result<Value> {
}
for t in nq.all_tracks_mut() {
t.sort();
edit::remove_orphan_transitions(t);
}
nq.check().map_err(EngineError::Other)?;
}
@@ -4668,8 +4911,8 @@ fn apply_transition(s: &mut Session, p: &Value, kind: TrackKind) -> Result<Value
}
/// Plan an applied transition without editing the project or allocating an id.
/// Uses the same `effect`, `clip`, `edge`, `frames` and settings as transition application,
/// so a drag preview shows the exact range that release will commit.
/// Uses the same `effect`, `clip`, `edge`, `align`, `frames` and settings as transition
/// application, so a drag preview shows the exact range that release will commit.
pub fn preview_transition(s: &Session, p: &Value, kind: TrackKind) -> Result<(TrackId, Transition)> {
let eff_id = str_p(p, "effect")
.map(str::to_string)
@@ -4724,14 +4967,51 @@ pub fn preview_transition(s: &Session, p: &Value, kind: TrackKind) -> Result<(Tr
if q.track(track).is_some_and(|tr| tr.locked) {
return Err(edit::EditError::Locked.into());
}
let start = if from.is_some() && to.is_some() {
cut - dur.mul_ratio(1, 2)
} else if to.is_some() {
cut
} else {
cut - dur
// A transition has to fit the clips around its cut: `edit::transitions::set_span` keeps that
// invariant when one is resized, but nothing limited the duration when a transition was
// created, so a default longer than a short clip can hold went in unchecked (#3455). Clamp
// the planned duration to the same limit: a centered crossfade spends half on each side of
// the cut, a fade to/from nothing spends it all on its one side.
let (lo, hi) = {
let track_ref = q.track(track).ok_or_else(|| EngineError::Other("no track at the edit point".into()))?;
let f = from.and_then(|c| track_ref.item(c));
let n = to.and_then(|c| track_ref.item(c));
match (f, n) {
(Some(f), Some(n)) => (f.start, n.end()),
(None, Some(n)) => (n.start, n.end()),
(Some(f), None) => (f.start, f.end()),
(None, None) => return Err(EngineError::Other("the edit point has no clip to transition on".into())),
}
};
let tr = Transition { id: TransitionId(0), effect: instance, start: rate.snap(start), duration: dur, from, to, align: Default::default(), reverse };
let room = if from.is_some() && to.is_some() {
(cut - lo).min(hi - cut).mul_ratio(2, 1)
} else if to.is_some() {
hi - cut
} else {
cut - lo
};
let dur = dur.min(room);
if dur < rate.tick_of(1) {
return Err(bad("transition", "no room for a transition at this cut: the clip(s) around it are shorter than a frame"));
}
// Where the transition sits on the cut: Center at Cut (default), Start at Cut (its start on
// the cut) or End at Cut (its end on the cut). A one-sided fade keeps its clip edge.
let align = match str_p(p, "align") {
Some(a) => edit::transitions::Alignment::parse(a).ok_or_else(|| bad("transition", format!("`align` is center, start or end, not `{a}`")))?,
None => edit::transitions::Alignment::Center,
};
let (start, align) = if from.is_some() && to.is_some() {
match align {
edit::transitions::Alignment::Start => (cut, TransitionAlign::StartAtCut),
edit::transitions::Alignment::End => (cut - dur, TransitionAlign::EndAtCut),
_ => (cut - dur.mul_ratio(1, 2), TransitionAlign::CenterAtCut),
}
} else if to.is_some() {
(cut, TransitionAlign::StartAtCut)
} else {
(cut - dur, TransitionAlign::EndAtCut)
};
let tr = Transition { id: TransitionId(0), effect: instance, start: rate.snap(start), duration: dur, from, to, align, reverse };
Ok((track, tr))
}
@@ -4916,7 +5196,7 @@ fn recover(s: &mut Session, id: Option<&str>) -> Result<Value> {
Some(id) => per.candidates.iter().position(|c| c.id == id).ok_or_else(|| bad("file.recover", format!("no recovery session `{id}`")))?,
None => 0,
};
let c = per.candidates[idx].clone();
let c = per.candidates.get(idx).cloned().ok_or_else(|| bad("file.recover", "no recovery session"))?;
let saved_at = per.local_time(c.meta.saved_unix);
let loaded = crate::autosave::load_candidate(&c).map_err(EngineError::Other)?;
install_project(s, loaded.project, c.meta.project_path.clone(), false);
@@ -4926,11 +5206,21 @@ fn recover(s: &mut Session, id: Option<&str>) -> Result<Value> {
crate::relink::on_open(s);
// Our own journal must hold the recovered state before the old one is deleted.
s.sync_persistence();
if let Some(per) = s.persistence.as_mut() {
per.flush();
per.candidates.remove(idx);
let per = s.persistence.as_mut().ok_or_else(|| EngineError::Other("recovery is not running".into()))?;
let preserve = |reason: String| {
EngineError::Other(format!(
"Recovered project is open, but its new recovery copy was not confirmed: {reason}. The original recovery copy is retained; save the project or use Save As."
))
};
if per.flush().map_err(preserve)? != Some(s.revision) {
return Err(preserve("journal revision does not match the recovered project".into()));
}
crate::autosave::discard_candidate(&c).map_err(|e| EngineError::Other(e.to_string()))?;
crate::autosave::discard_candidate(&c).map_err(|e| {
EngineError::Other(format!(
"Recovered project is open and its new recovery copy is confirmed, but the old recovery folder could not be fully removed: {e}. The recovery candidate remains listed; some old files may already be gone. Save the project or use Save As before retrying; if the original recovery copy remains intact, running recovery again can overwrite unsaved edits."
))
})?;
per.candidates.remove(idx);
s.toast(format!("Recovered unsaved changes to {} from {saved_at}", c.meta.project_name));
Ok(json!({"id": c.id, "name": c.meta.project_name, "path": c.meta.project_path, "savedAt": saved_at, "revision": c.meta.revision}))
}
crates/engine/src/demo.rs
+2 -2
View File
@@ -48,14 +48,14 @@ pub fn add_generator(p: &mut Project, pool: &MediaPool, src: GeneratorSource, na
pub fn demo_project(pool: &MediaPool) -> (Project, ItemId) {
build_demo(pool).unwrap_or_else(|| {
// Only reachable if the built-in items could not be placed: an empty demo, not a crash.
let mut p = Project::new("FilmCraft Demo");
let mut p = Project::new("FilmCraft – Demo Project");
let seq = p.new_sequence("Main Edit", SequenceSettings::default(), 3, 3, None);
(p, seq)
})
}
fn build_demo(pool: &MediaPool) -> Option<(Project, ItemId)> {
let mut p = Project::new("FilmCraft Demo");
let mut p = Project::new("FilmCraft – Demo Project");
let footage = p.add_bin("Footage", None);
let audio_bin = p.add_bin("Audio", None);
let gfx = p.add_bin("Graphics", None);
crates/engine/src/export_tests.rs
+44
View File
@@ -65,6 +65,23 @@ fn export_without_wait_says_where_to_poll() {
assert!(r.get("poll").is_none() && r["result"].is_object(), "{r}");
}
/// An export reads its own copy of each file, not the source the monitors read: the two would
/// otherwise take turns on one decoder, seeking it between the export's frame and the playhead's
/// (93 % of a 4K60 playback's frames dropped while an export of the same file ran).
#[test]
fn an_export_does_not_read_the_monitors_source() {
let movie = crate::media_test_util::tmp_dir("export-own-source").join("a.mp4");
crate::media_test_util::make_movie(&movie, filmcraft_media::DemoScene::Dunes, 64, 36, 12);
let (mut s, items, seq) = crate::media_test_util::session_with(&[&movie]);
s.media.clear();
let dir = Scratch::new("own-source");
let out = dir.path("a.mp4");
let r = s.execute("file.exportMedia", json!({"sequence": seq.0, "path": out, "format": "h264", "wait": true})).unwrap();
assert!(r["result"].is_object(), "{r}");
assert!(std::fs::metadata(&out).is_ok_and(|m| m.len() > 0));
assert!(s.media.cached(items[0]).is_none(), "the export opened the monitors' source");
}
/// A scratch directory under the workspace `target/`, removed when dropped.
struct Scratch(PathBuf);
@@ -744,6 +761,33 @@ fn hardware_encoding_is_off_unless_asked_for() {
assert_eq!(old.hardware_encoding, Off);
}
/// A `bitrateKbps` in `settings` is the bitrate, as the flat `bitrateKbps` is, also over a preset
/// whose bitrate adapts to the frame size (every Match Source preset).
#[test]
fn settings_bitrate_replaces_a_presets_adaptive_bitrate() {
use filmcraft_export::ExportSettings;
use filmcraft_export::presets::DEFAULT_PRESET;
use filmcraft_time::FrameRate;
let s = demo();
let settings = |p: &Value| crate::export_tools::settings_from_params(&s, p, "file.exportMedia").map(|(_, st)| st).unwrap();
let target = |st: &ExportSettings| st.resolve(720, 480, FrameRate::FPS_29_97, 48_000).target_kbps;
// the preset alone: 720 x 480 x 29.97 fps x 0.2 bits per pixel
assert_eq!(target(&settings(&json!({"path": "x.mp4", "preset": DEFAULT_PRESET}))), 2072);
for p in [
json!({"path": "x.mp4", "preset": DEFAULT_PRESET, "settings": {"bitrateKbps": 5000}}),
json!({"path": "x.mp4", "preset": DEFAULT_PRESET, "bitrateKbps": 5000}),
json!({"path": "x.mp4", "settings": {"bitrateKbps": 5000}}),
json!({"path": "x.mp4", "bitrateKbps": 5000}),
] {
let st = settings(&p);
assert_eq!(st.adaptive_bitrate, None, "{p}");
assert_eq!(target(&st), 5000, "{p}");
}
// an adaptive bitrate given with it still wins
let p = json!({"path": "x.mp4", "preset": DEFAULT_PRESET, "settings": {"bitrateKbps": 5000, "adaptiveBitrate": 0.1}});
assert_eq!(target(&settings(&p)), 1036);
}
#[test]
fn crf_param_picks_constant_quality() {
use filmcraft_export::BitrateMode::{Cbr, Crf, Vbr1Pass};
crates/engine/src/export_tools.rs
+8 -2
View File
@@ -218,6 +218,11 @@ fn apply_nested_settings(settings: &mut ExportSettings, params: &Value, cmd: &st
let mut value = serde_json::to_value(&*settings).map_err(|error| EngineError::Other(error.to_string()))?;
merge(&mut value, patch);
*settings = serde_json::from_value(value).map_err(|error| bad(cmd, format!("settings: {error}")))?;
// A bitrate given in `settings` is the bitrate, as the flat `bitrateKbps` is: an adaptive
// bitrate from the preset would otherwise replace it (`ExportSettings::resolve`).
if patch.get("bitrateKbps").is_some() && patch.get("adaptiveBitrate").is_none() {
settings.adaptive_bitrate = None;
}
// serde ignores unknown keys; detect misspelled or misplaced settings before returning.
let known = serde_json::to_value(&*settings).map_err(|error| EngineError::Other(error.to_string()))?;
if let Some(path) = unknown_key(&known, patch, "") {
@@ -507,8 +512,9 @@ pub fn spawn_export(s: &mut Session, project: Arc<Project>, seq: ItemId, mut set
write_sidecar(s, &project, seq, &settings)?;
let id = s.jobs.iter().map(|j| j.id).max().unwrap_or(0) + 1;
let job = crate::Job { id, label, progress: Default::default(), result: Default::default() };
// Export always renders full-resolution media, whatever the proxy toggle says.
let provider = s.media.full_res_provider(project.clone(), s.services.clone()).recording();
// Export always renders full-resolution media, whatever the proxy toggle says, from its own
// copies of the files so the monitors keep their decoders.
let provider = s.media.full_res_provider(project.clone(), s.services.clone()).own_sources().recording();
let (prog, res) = (job.progress.clone(), job.result.clone());
let offline = s.export_offline.entry(id).or_default().clone();
if cfg!(target_arch = "wasm32") && !wait && filmcraft_export::stepped(format) {
crates/engine/src/graphic_templates.rs
+9 -3
View File
@@ -910,7 +910,8 @@ fn upgrade_caption(s: &mut Session, p: &Value) -> Result<Value> {
};
let col = |c: [u8; 4]| [c[0] as f32 / 255.0, c[1] as f32 / 255.0, c[2] as f32 / 255.0, 1.0];
set(&mut layer, "font", ParamValue::Text(st.font.clone()));
set(&mut layer, "font_style", ParamValue::Text("SemiBold".into()));
let font_style = if st.font_style.trim().is_empty() { "SemiBold" } else { st.font_style.trim() };
set(&mut layer, "font_style", ParamValue::Text(font_style.into()));
set(&mut layer, "align", ParamValue::Choice(st.align as u32));
set(&mut layer, "fill_color", ParamValue::Color(col(st.color)));
set(&mut layer, "leading", ParamValue::Float(line_h - size * 1.21));
@@ -1067,7 +1068,8 @@ pub fn fonts_used(s: &Session) -> Vec<(String, String, usize)> {
for q in s.project.sequences() {
if let ItemKind::Sequence(sq) = &q.kind {
for t in &sq.caption_tracks {
add(t.style.font.clone(), "SemiBold".into());
let st = if t.style.font_style.trim().is_empty() { "SemiBold" } else { t.style.font_style.trim() };
add(t.style.font.clone(), st.into());
}
}
}
@@ -1123,8 +1125,12 @@ fn replace_fonts(s: &mut Session, p: &Value) -> Result<Value> {
for id in ids {
if let Some(sq) = pr.sequence_mut(id) {
for t in &mut sq.caption_tracks {
if t.style.font.eq_ignore_ascii_case(&from) && from_style.is_none() {
let st = if t.style.font_style.trim().is_empty() { "SemiBold" } else { t.style.font_style.trim() };
if matches(&t.style.font, st) {
t.style.font = to.clone();
if let Some(ts) = &to_style {
t.style.font_style = ts.clone();
}
n += 1;
}
}
crates/engine/src/graphic_templates/tests.rs
+53
View File
@@ -40,6 +40,25 @@ fn clip_of(v: &Value) -> ClipId {
ClipId(v["clip"].as_u64().unwrap())
}
#[test]
fn template_exports_keep_distinct_non_ascii_names() {
let mut s = demo("gt-unicode-export");
let c = clip_of(&s.execute("graphics.newText", json!({"text": "Headline"})).unwrap());
let first = s.execute("graphics.template.export", json!({"clip": c.0, "name": "片头"})).unwrap();
let first_path = first["path"].as_str().unwrap();
let first_bytes = std::fs::read(first_path).unwrap();
let second = s.execute("graphics.template.export", json!({"clip": c.0, "name": "片尾"})).unwrap();
assert_ne!(first["path"], second["path"], "distinct names must not overwrite the first export");
assert_ne!(first["id"], second["id"], "distinct templates need distinct library identities");
assert_eq!(std::fs::read(first_path).unwrap(), first_bytes);
let second_file: Value = serde_json::from_slice(&std::fs::read(second["path"].as_str().unwrap()).unwrap()).unwrap();
assert_eq!(second_file["name"], "片尾");
let installed = s.execute("graphics.template.list", json!({"source": "user"})).unwrap();
for exported in [&first, &second] {
assert!(installed.as_array().unwrap().iter().any(|entry| entry["id"] == exported["id"] && entry["name"] == exported["name"]));
}
}
#[test]
fn template_export_install_apply_round_trip() {
let mut s = demo("gt-export");
@@ -331,6 +350,27 @@ fn upgrade_caption_to_graphic() {
assert_eq!(s.execute("captions.list", json!({})).unwrap()["tracks"][0]["captions"].as_array().unwrap().len(), before);
}
#[test]
fn caption_font_style_is_upgraded_listed_and_replaced() {
let mut s = demo("gt-caption-font");
s.execute("captions.add", json!({"text": "Styled", "seconds": 600, "durationSeconds": 2})).unwrap();
s.execute("captions.setStyle", json!({"font": "Inter", "fontStyle": "Medium"})).unwrap();
let used = s.execute("graphics.fonts.used", json!({})).unwrap().to_string();
assert!(used.contains("Medium"), "{used}");
// a style-specific replacement skips other styles and moves this one
s.execute("file.replaceFonts", json!({"from": {"family": "Inter", "style": "Bold"}, "to": "Noto Serif"})).unwrap();
assert_eq!(s.active_sequence().unwrap().caption_tracks[0].style.font, "Inter");
// upgrading carries the style onto the graphic's text
let r = s.execute("graphics.upgradeCaption", json!({})).unwrap();
let it = item(&s, ClipId(r["clips"][0].as_u64().unwrap()));
let e = it.effects.iter().find(|e| e.effect == "graphic_text").unwrap();
assert_eq!(e.param("font_style").unwrap().value, ParamValue::Text("Medium".into()));
s.execute("edit.undo", json!({})).unwrap();
s.execute("file.replaceFonts", json!({"from": {"family": "Inter", "style": "Medium"}, "to": {"family": "Noto Serif", "style": "Bold"}})).unwrap();
let st = &s.active_sequence().unwrap().caption_tracks[0].style;
assert_eq!((st.font.as_str(), st.font_style.as_str()), ("Noto Serif", "Bold"));
}
#[test]
fn source_graphics_share_edits() {
let mut s = demo("gt-source");
@@ -459,3 +499,16 @@ fn character_style_merge_preserves_default_calls_and_separate_clips() {
}
assert_eq!(s.history.undo.len(), n + 2, "distinct layers do not merge");
}
/// Issue #3094: a gradient stop set as `#rrggbbaa` keeps its opacity.
#[test]
fn gradient_stop_keeps_its_alpha() {
let mut s = demo("gt-gradient-alpha");
let c = clip_of(&s.execute("graphics.newText", json!({"text": "G"})).unwrap());
s.execute("graphics.set", json!({"clip": c.0, "props": {"gradient_start": "#ff000080", "gradient_end": "#0000ff"}})).unwrap();
let it = item(&s, c);
let col = |id: &str| it.effects.iter().find_map(|e| e.params.get(id)).and_then(|p| p.value.as_color());
let start = col("gradient_start").unwrap();
assert!((start[3] - 128.0 / 255.0).abs() < 1e-3, "{start:?}");
assert_eq!(col("gradient_end").unwrap()[3], 1.0);
}
crates/engine/src/graphics.rs
+88 -11
View File
@@ -249,24 +249,37 @@ fn param_entry<'a>(e: &'a mut EffectInstance, id: &str) -> Option<&'a mut Param>
/// Set properties `props` on layer `e` (keyframe-aware at media time `mt`). Changing the text
/// keeps per-character styles on their characters.
fn apply_props(e: &mut EffectInstance, props: &serde_json::Map<String, Value>, mt: Tick) -> Result<()> {
///
/// With `keyframe`, each property gets a keyframe at `mt` (it becomes animated, the same as
/// `effects.setParam` with `keyframe: true`) instead of changing its static value when it is not
/// animated yet; the result lists each such property's keyframe count (#1877).
fn apply_props(e: &mut EffectInstance, props: &serde_json::Map<String, Value>, mt: Tick, keyframe: bool) -> Result<Vec<(String, usize)>> {
let mut counts = Vec::new();
for (k, v) in props {
if k == "enabled" {
e.enabled = v.as_bool().unwrap_or(true);
continue;
}
let id = param_id(k);
if keyframe && filmcraft_project::find_effect(&e.effect).and_then(|d| d.param(id)).is_some_and(|d| !d.animatable) {
return Err(bad("graphics.set", format!("`{k}` cannot be animated")));
}
let prm = param_entry(e, id).ok_or_else(|| bad("graphics.set", format!("no property `{k}`")))?;
let pv = to_param(&prm.value, id, v).ok_or_else(|| bad("graphics.set", format!("`{k}`: value has the wrong type")))?;
let old = prm.value_at(mt);
prm.set_at(mt, pv.clone());
if keyframe {
prm.put_keyframe(mt, pv.clone());
counts.push((k.clone(), prm.keyframes.len()));
} else {
prm.set_at(mt, pv.clone());
}
if let (ParamValue::Text(a), ParamValue::Text(b), "text") = (&old, &pv, id)
&& let Some(x) = e.layer.as_mut()
{
x.runs = filmcraft_project::graphic_design::adjust_runs(a, b, &x.runs);
}
}
Ok(())
Ok(counts)
}
/// Set properties `props` on a layer (keyframe-aware at time `tl`). Changing the text keeps
@@ -281,14 +294,32 @@ pub(crate) fn set_props(
label: &str,
merge: Option<&str>,
) -> Result<()> {
set_props_keyed(s, clip, eidx, props, tl, label, merge, false).map(|_| ())
}
/// [`set_props`] that can put a keyframe on each property (`keyframe`); returns the keyframe count
/// of each property it did that for.
#[allow(clippy::too_many_arguments)]
pub(crate) fn set_props_keyed(
s: &mut Session,
clip: ClipId,
eidx: usize,
props: &serde_json::Map<String, Value>,
tl: Tick,
label: &str,
merge: Option<&str>,
keyframe: bool,
) -> Result<Vec<(String, usize)>> {
let props = props.clone();
let mut counts = Vec::new();
s.edit_sequence_as(label, merge, |q, _, _| {
let (_, it) = q.find_item_mut(clip).ok_or(filmcraft_edit::EditError::NoItem(clip))?;
let mt = it.source_time_at(tl.clamp(it.start, it.end() - Tick(1)));
let e = it.effects.get_mut(eidx).ok_or_else(|| bad("graphics.set", "no such layer"))?;
apply_props(e, &props, mt)?;
counts = apply_props(e, &props, mt, keyframe)?;
Ok(())
})
})?;
Ok(counts)
}
/// Turn a text layer into point text (no box; the lines the box wrapped become real lines) or
@@ -359,7 +390,7 @@ fn set_text_type(s: &mut Session, p: &Value) -> Result<Value> {
s.edit_sequence("Text Layer Type", |q, _, _| {
let (_, it) = q.find_item_mut(clip).ok_or(filmcraft_edit::EditError::NoItem(clip))?;
let e = it.effects.get_mut(ei).ok_or_else(|| bad(ID, "no such layer"))?;
apply_props(e, &props, mt)?;
apply_props(e, &props, mt, false)?;
if let Some(a) = param_entry(e, "anchor") {
a.map_values(|v| match v.as_vec2() {
Some(a) => ParamValue::Vec2(Vec2::new(a.x + shift.0 as f64, a.y + shift.1 as f64)),
@@ -478,14 +509,31 @@ pub(crate) fn move_layers(s: &mut Session, clip: ClipId, moves: Vec<(usize, Vec2
s.edit_sequence(label, |q, _, _| {
let (_, it) = q.find_item_mut(clip).ok_or(filmcraft_edit::EditError::NoItem(clip))?;
let mt = it.source_time_at(tl.clamp(it.start, it.end() - Tick(1)));
// how far each moving layer goes, by uid, before anything changes
let deltas: Vec<(u64, f64, f64)> = moves
.iter()
.filter_map(|(ei, pos)| {
let e = it.effects.get(*ei)?;
let uid = e.layer.as_ref().map(|x| x.uid).filter(|&u| u != 0)?;
let old = e.params.get("position")?.value_at(mt).as_vec2()?;
Some((uid, pos.x - old.x, pos.y - old.y))
})
.collect();
for (ei, pos) in &moves {
let Some(e) = it.effects.get_mut(*ei) else { continue };
let Some(prm) = e.params.get_mut("position") else { continue };
let old = prm.value_at(mt).as_vec2().unwrap_or(*pos);
prm.set_at(mt, ParamValue::Vec2(*pos));
// a pinned layer keeps following its target from its new place
// a pinned layer keeps following its target from its new place: its offsets change
// by how much further it moved than its target (when both move, by
// the same delta (a drag) nothing changes; by different ones (Align, Distribute),
// the difference)
if let Some(pin) = e.layer.as_mut().and_then(|x| x.pin.as_mut()) {
let (dx, dy) = (pos.x - old.x, pos.y - old.y);
let (tx, ty) = match pin.to {
filmcraft_project::graphic_design::PinTarget::Layer(u) => deltas.iter().find(|d| d.0 == u).map_or((0.0, 0.0), |d| (d.1, d.2)),
filmcraft_project::graphic_design::PinTarget::Frame => (0.0, 0.0),
};
let (dx, dy) = (pos.x - old.x - tx, pos.y - old.y - ty);
pin.offsets[0] += dx;
pin.offsets[2] += dx;
pin.offsets[1] += dy;
@@ -1064,7 +1112,7 @@ pub fn commands() -> Vec<CommandSpec> {
"Set Graphic Properties",
&[],
None,
r##"{"clip":id?,"layer":n|name?,"props":{"font":"Inter","font_style":"Bold","size":120,"align":"center","tracking":50,"leading":0,"fill_color":"#ffcc00","stroke":true,"stroke_width":6,"background":true,"shadow":true,"position":[x,y],"scale":100,"rotation":0,"opacity":100,…},"time":ticks?,"merge":bool?,"begin":bool?}"##,
r##"{"clip":id?,"layer":n|name?,"props":{"font":"Inter","font_style":"Bold","size":120,"align":"center","tracking":50,"leading":0,"fill_color":"#ffcc00","stroke":true,"stroke_width":6,"background":true,"shadow":true,"position":[x,y],"scale":100,"rotation":0,"opacity":100,…},"time":ticks? (timeline time, default the playhead: a property that is already animated gets or changes a keyframe there, any other changes its fixed value),"keyframe":bool? (true: every property in `props` gets a keyframe at `time` and becomes animated; the result lists each one's keyframe count),"merge":bool?,"begin":bool?}"##,
has_graphic,
|s, p| {
let clip = target_clip(s, p).ok_or_else(|| bad("graphics.set", "no graphic clip"))?;
@@ -1076,8 +1124,14 @@ pub fn commands() -> Vec<CommandSpec> {
if bool_p(p, "begin").unwrap_or(false) {
s.history.merge_key = None;
}
set_props(s, clip, ei, &props, tl, "Change Graphic Property", merge.as_deref())?;
Ok(Value::Null)
let keyframe = bool_p(p, "keyframe").unwrap_or(false);
let counts = set_props_keyed(s, clip, ei, &props, tl, "Change Graphic Property", merge.as_deref(), keyframe)?;
// unchanged for a call without `keyframe`: nothing to report
if keyframe {
Ok(json!({"keyframes": counts.into_iter().map(|(k, n)| (k, json!(n))).collect::<serde_json::Map<_, _>>()}))
} else {
Ok(Value::Null)
}
},
),
spec(
@@ -1089,6 +1143,29 @@ pub fn commands() -> Vec<CommandSpec> {
has_graphic,
set_text_type,
),
spec(
"graphics.moveLayers",
"Move Graphic Layers",
&[],
None,
r#"{"clip":id?,"layers":[n]? (default: the selected layers),"delta":[dx,dy] (graphic pixels)}"#,
has_graphic,
|s, p| {
// #2186: several selected layers dragged together on the Program monitor move as one
// undo step, each keeping its place relative to the others (pinned layers keep
// following their targets from the new place, as with Align)
let clip = target_clip(s, p).ok_or_else(|| bad("graphics.moveLayers", "no graphic clip"))?;
let mut layers = layers_p(s, clip, p)?;
layers.sort_unstable();
layers.dedup();
let d =
vec2_p(p, "delta").filter(|d| d.x.is_finite() && d.y.is_finite()).ok_or_else(|| bad("graphics.moveLayers", "need `delta`: [dx, dy]"))?;
let (boxes, _) = layer_boxes(s, clip, &layers)?;
let moves = boxes.into_iter().map(|(ei, own, _)| (ei, Vec2::new(own.x + d.x, own.y + d.y))).collect();
move_layers(s, clip, moves, "Move Graphic Layers")?;
Ok(json!({"clip": clip.0, "layers": layers}))
},
),
spec("graphics.selectLayer", "Select Graphic Layer", &[], None, r#"{"clip":id?,"layers":[n]}"#, has_graphic, |s, p| {
let clip = target_clip(s, p).ok_or_else(|| bad("graphics.selectLayer", "no graphic clip"))?;
let layers: Vec<usize> =
crates/engine/src/graphics/tests.rs
+232
View File
@@ -714,3 +714,235 @@ fn align_to_frame_moves_selected_video_clips_by_position_only() {
s.execute("edit.undo", json!({})).unwrap();
assert!((pos(&s).0.y - 200.0).abs() < 1e-6, "one undo step per alignment");
}
/// #1877: `graphics.set` with `time` only changes a keyframe of a property that is already
/// animated; `keyframe: true` puts keyframes on the properties it lists, as `effects.setParam` does.
mod set_with_keyframes {
use super::*;
use filmcraft_project::ParamValue;
const AT_1_5_S: i64 = 381_024_000_000;
fn ellipse() -> (Session, ClipId) {
let mut s = Session::default();
s.execute("file.newSequence", json!({"name": "R", "width": 1920, "height": 1080, "fps": 30, "video": 4, "audio": 1})).unwrap();
let r = s.execute("graphics.newEllipse", json!({"position": [300, 540], "size": [200, 200], "track": 1, "time": 0, "seconds": 2})).unwrap();
let clip = ClipId(r["clip"].as_u64().unwrap());
(s, clip)
}
/// The layer's `position`: its keyframes (time, x) and its fixed value's x.
fn position(s: &Session, clip: ClipId) -> (Vec<(i64, f32)>, f32) {
let q = s.active_sequence().unwrap();
let (_, it) = q.find_item(clip).unwrap();
let e = &it.effects[layer_indices(&it.effects)[0]];
let p = &e.params["position"];
let x = |v: &ParamValue| v.as_vec2().unwrap().x as f32;
(p.keyframes.iter().map(|k| (k.time.0, x(&k.value))).collect(), x(&p.value))
}
#[test]
fn without_keyframe_the_last_call_wins_as_before() {
let (mut s, clip) = ellipse();
s.execute("graphics.set", json!({"clip": clip.0, "layer": 0, "time": 0, "props": {"position": [300, 540]}})).unwrap();
let r = s.execute("graphics.set", json!({"clip": clip.0, "layer": 0, "time": AT_1_5_S, "props": {"position": [1600, 540]}})).unwrap();
assert!(r.is_null(), "the result of a call without `keyframe` is unchanged: {r}");
let (keys, fixed) = position(&s, clip);
assert!(keys.is_empty(), "{keys:?}");
assert_eq!(fixed, 1600.0);
}
#[test]
fn keyframe_true_animates_the_property() {
let (mut s, clip) = ellipse();
let r = s.execute("graphics.set", json!({"clip": clip.0, "layer": 0, "time": 0, "keyframe": true, "props": {"position": [300, 540]}})).unwrap();
assert_eq!(r, json!({"keyframes": {"position": 1}}));
let r = s.execute("graphics.set", json!({"clip": clip.0, "layer": 0, "time": AT_1_5_S, "keyframe": true, "props": {"position": [1600, 540]}})).unwrap();
assert_eq!(r, json!({"keyframes": {"position": 2}}));
let (keys, _) = position(&s, clip);
assert_eq!(keys, vec![(0, 300.0), (AT_1_5_S, 1600.0)]);
// the layer really moves: between the keyframes it is between the two positions
let q = s.active_sequence().unwrap();
let (_, it) = q.find_item(clip).unwrap();
let e = &it.effects[layer_indices(&it.effects)[0]];
let mid = e.params["position"].value_at(it.source_time_at(Tick(AT_1_5_S / 2))).as_vec2().unwrap().x as f32;
assert!(mid > 300.0 && mid < 1600.0, "{mid}");
}
#[test]
fn a_property_that_is_already_animated_keeps_following_time() {
let (mut s, clip) = ellipse();
s.execute("graphics.set", json!({"clip": clip.0, "layer": 0, "time": 0, "keyframe": true, "props": {"position": [300, 540]}})).unwrap();
// no `keyframe`: the property is animated now, so a call with `time` adds a keyframe (as before)
s.execute("graphics.set", json!({"clip": clip.0, "layer": 0, "time": AT_1_5_S, "props": {"position": [1600, 540]}})).unwrap();
assert_eq!(position(&s, clip).0.len(), 2);
}
#[test]
fn one_undo_step_per_call_and_properties_that_cannot_be_animated_are_refused() {
let (mut s, clip) = ellipse();
s.execute("graphics.set", json!({"clip": clip.0, "layer": 0, "time": 0, "keyframe": true, "props": {"position": [300, 540]}})).unwrap();
s.execute("graphics.set", json!({"clip": clip.0, "layer": 0, "time": AT_1_5_S, "keyframe": true, "props": {"position": [1600, 540]}})).unwrap();
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(position(&s, clip).0.len(), 1, "undo takes back the second keyframe only");
let e = s.execute("graphics.set", json!({"clip": clip.0, "layer": 0, "keyframe": true, "props": {"font": "Inter"}}));
assert!(e.is_err(), "a font cannot be keyframed");
let e = s.execute("graphics.set", json!({"clip": clip.0, "layer": 0, "keyframe": true, "props": {"nope": 1}}));
assert!(e.is_err());
}
}
/// #2186: several layers move together by one delta, as one undo step, keeping their spacing.
#[test]
fn selected_layers_move_together_in_one_undo_step() {
let mut s = demo();
let clip = three_shapes(&mut s);
let before = layers(&s, ClipId(clip));
s.execute("graphics.selectLayer", json!({"clip": clip, "layers": [0, 2]})).unwrap();
let undo = s.history.undo.len();
let r = s.execute("graphics.moveLayers", json!({"clip": clip, "delta": [40, -25]})).unwrap();
assert_eq!(r["layers"], json!([0, 2]), "the selected layers by default");
let after = layers(&s, ClipId(clip));
for i in [0, 2] {
assert!((after[i].transform.position.x - before[i].transform.position.x - 40.0).abs() < 1e-9);
assert!((after[i].transform.position.y - before[i].transform.position.y + 25.0).abs() < 1e-9);
}
assert_eq!(after[1].transform.position, before[1].transform.position, "an unselected layer stays");
assert_eq!(s.history.undo.len(), undo + 1, "one undo step");
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(
layers(&s, ClipId(clip)).iter().map(|l| l.transform.position).collect::<Vec<_>>(),
before.iter().map(|l| l.transform.position).collect::<Vec<_>>()
);
// explicit layers, duplicates ignored; bad deltas and layers are refused
s.execute("graphics.moveLayers", json!({"clip": clip, "layers": [1, 1], "delta": [10, 0]})).unwrap();
assert!((layers(&s, ClipId(clip))[1].transform.position.x - before[1].transform.position.x - 10.0).abs() < 1e-9);
assert!(s.execute("graphics.moveLayers", json!({"clip": clip, "layers": [0], "delta": [f64::MAX, "x"]})).is_err());
assert!(s.execute("graphics.moveLayers", json!({"clip": clip, "layers": [9], "delta": [1, 1]})).is_err());
assert!(s.execute("graphics.moveLayers", json!({"clip": clip, "layers": [0]})).is_err());
}
/// a pinned layer moved together with the layer it is pinned to moves once, with
/// it, not twice (once with its target, once by its own offsets); undo puts both back.
#[test]
fn a_pinned_layer_moved_with_its_target_moves_once() {
let mut s = demo();
let clip = three_shapes(&mut s);
s.execute("graphics.pin", json!({"clip": clip, "layer": 0, "to": 1, "edges": ["left", "top"]})).unwrap();
// where each layer is shown, pins resolved (not the position stored in it)
let shown = |s: &Session, l: usize| {
let q = s.active_sequence().unwrap();
let (_, it) = q.find_item(ClipId(clip)).unwrap();
let ei = layer_indices(&it.effects)[l];
let specs = filmcraft_render::graphic_clip::item_layer_specs(it, Tick::ZERO, (q.settings.width, q.settings.height));
specs.iter().find(|(x, _)| *x == ei).unwrap().1.transform.position
};
let (p0, p1) = (shown(&s, 0), shown(&s, 1));
s.execute("graphics.moveLayers", json!({"clip": clip, "layers": [0, 1], "delta": [40, -25]})).unwrap();
let (a0, a1) = (shown(&s, 0), shown(&s, 1));
assert!((a1.x - p1.x - 40.0).abs() < 1e-6 && (a1.y - p1.y + 25.0).abs() < 1e-6, "the target moved: {p1:?} → {a1:?}");
assert!((a0.x - p0.x - 40.0).abs() < 1e-6 && (a0.y - p0.y + 25.0).abs() < 1e-6, "the pinned layer moved once: {p0:?} → {a0:?}");
// moved on its own it still moves (its offsets to the target change)
s.execute("graphics.moveLayers", json!({"clip": clip, "layers": [0], "delta": [10, 0]})).unwrap();
assert!((shown(&s, 0).x - a0.x - 10.0).abs() < 1e-6);
s.execute("edit.undo", json!({})).unwrap();
s.execute("edit.undo", json!({})).unwrap();
assert_eq!((shown(&s, 0), shown(&s, 1)), (p0, p1), "undo puts both back");
}
/// Align moves a pinned layer and its target by different amounts; the pinned
/// layer ends where Align put it, not where its target's move would drag it.
#[test]
fn align_puts_a_pinned_layer_where_it_asks() {
let mut s = demo();
let clip = three_shapes(&mut s);
s.execute("graphics.pin", json!({"clip": clip, "layer": 0, "to": 1, "edges": ["left", "top"]})).unwrap();
let shown = |s: &Session, l: usize| {
let q = s.active_sequence().unwrap();
let (_, it) = q.find_item(ClipId(clip)).unwrap();
let ei = layer_indices(&it.effects)[l];
let specs = filmcraft_render::graphic_clip::item_layer_specs(it, Tick::ZERO, (q.settings.width, q.settings.height));
specs.iter().find(|(x, _)| *x == ei).unwrap().1.transform.position
};
let (rect, ellipse) = (shown(&s, 0), shown(&s, 1));
// left edges to the selection's: the rectangle is already leftmost, so only the ellipse moves
s.execute("graphics.align", json!({"clip": clip, "layers": [0, 1], "align": "left", "to": "selection"})).unwrap();
let (r1, e1) = (shown(&s, 0), shown(&s, 1));
assert!(e1.x < ellipse.x - 100.0, "the ellipse moved left: {ellipse:?} → {e1:?}");
assert!((r1.x - rect.x).abs() < 1e-6 && (r1.y - rect.y).abs() < 1e-6, "the pinned rectangle stays where Align put it: {rect:?} → {r1:?}");
}
#[test]
fn applied_effects_stack_after_the_clip_s_own_and_keep_their_indices() {
// #1879
let mut s = Session::default();
s.execute("file.newSequence", json!({"name": "R", "width": 1920, "height": 1080, "fps": 30, "video": 2, "audio": 1})).unwrap();
let r = s.execute("graphics.newEllipse", json!({"position": [960, 540], "size": [400, 400], "track": 1, "time": 0, "seconds": 2})).unwrap();
let clip = r["clip"].as_u64().unwrap();
let ids = |s: &Session| s.active_sequence().unwrap().find_item(ClipId(clip)).unwrap().1.effects.iter().map(|e| e.effect.clone()).collect::<Vec<_>>();
let before = ids(&s);
let layer = s.execute("graphics.list", json!({"clip": clip})).unwrap()["layers"][0]["effectIndex"].clone();
s.execute("effects.apply", json!({"clips": [clip], "effect": "gaussian_blur"})).unwrap();
s.execute("effects.apply", json!({"clips": [clip], "effect": "tint"})).unwrap();
let after = ids(&s);
assert_eq!(after[..before.len()], before[..], "{after:?}");
assert_eq!(after[before.len()..], ["gaussian_blur", "tint"], "{after:?}");
assert_eq!(s.execute("graphics.list", json!({"clip": clip})).unwrap()["layers"][0]["effectIndex"], layer);
s.execute("effects.setParam", json!({"clip": clip, "effect": layer, "param": "position", "value": [100, 100]})).unwrap();
}
/// #1879 moved applied effects from before the clip's intrinsic effects to the end of its stack.
/// The frame must not change: the same clips render the same pixels with the effects laid out the
/// old way (built by hand) and the new way.
#[test]
fn applied_effects_render_the_same_before_or_after_the_intrinsic_ones() {
fn frame(s: &Session) -> Vec<f32> {
s.render_program(0.25).unwrap().px
}
// the layout `effects.apply` gave before #1879: the applied effects (the last `n`) before the
// clip's first intrinsic effect, in the order they were applied
fn before_intrinsics(s: &mut Session, clip: ClipId, n: usize) {
let seq = s.state.active_sequence.unwrap();
let q = std::sync::Arc::make_mut(&mut s.project).sequence_mut(seq).unwrap();
let it = q.video_tracks.iter_mut().flat_map(|t| t.items.iter_mut()).find(|i| i.id == clip).unwrap();
let applied = it.effects.split_off(it.effects.len() - n);
let pos = it.effects.iter().position(|e| e.def().is_some_and(|d| d.intrinsic)).unwrap();
it.effects.splice(pos..pos, applied);
}
let apply = |s: &mut Session, clip: u64| {
s.execute("effects.apply", json!({"clips": [clip], "effect": "gaussian_blur"})).unwrap();
s.execute("effects.setParam", json!({"clip": clip, "effect": "gaussian_blur", "param": "blurriness", "value": 12.0})).unwrap();
s.execute("effects.apply", json!({"clips": [clip], "effect": "posterize"})).unwrap();
s.execute("effects.setParam", json!({"clip": clip, "effect": "posterize", "param": "levels", "value": 3.0})).unwrap();
};
let check = |mut s: Session, clip: u64, case: &str| {
let plain = frame(&s);
apply(&mut s, clip);
let ids: Vec<_> = s.active_sequence().unwrap().find_item(ClipId(clip)).unwrap().1.effects.iter().map(|e| e.effect.clone()).collect();
assert_eq!(ids[ids.len() - 2..], ["gaussian_blur", "posterize"], "{case}: {ids:?}");
let after = frame(&s);
assert!(after != plain, "{case}: the applied effects show in the frame");
before_intrinsics(&mut s, ClipId(clip), 2);
let ids: Vec<_> = s.active_sequence().unwrap().find_item(ClipId(clip)).unwrap().1.effects.iter().map(|e| e.effect.clone()).collect();
assert!(ids.iter().position(|e| e == "posterize") < ids.iter().position(|e| e == "motion"), "{case}: {ids:?}");
assert!(after == frame(&s), "{case}: same pixels with the applied effects before the intrinsic ones");
};
// (a) a graphic clip with its layer
let mut s = Session::default();
s.execute("file.newSequence", json!({"name": "R", "width": 640, "height": 360, "fps": 30, "video": 2, "audio": 1})).unwrap();
let r = s.execute("graphics.newEllipse", json!({"position": [320, 180], "size": [200, 120], "track": 1, "time": 0, "seconds": 2})).unwrap();
let clip = r["clip"].as_u64().unwrap();
check(s, clip, "graphic clip");
// (b) a video clip with Motion and Opacity off their defaults
let mut s = demo();
let clip = s.active_sequence().unwrap().video_tracks[0].items[0].id.0;
// past the dip to black that opens the demo
s.execute("playhead.set", json!({"seconds": 2.5})).unwrap();
for (effect, param, value) in
[("motion", "position", json!([700, 400])), ("motion", "scale", json!(70.0)), ("motion", "rotation", json!(15.0)), ("opacity", "opacity", json!(80.0))]
{
s.execute("effects.setParam", json!({"clip": clip, "effect": effect, "param": param, "value": value})).unwrap();
}
check(s, clip, "video clip");
}
crates/engine/src/interchange.rs
+11 -5
View File
@@ -3,14 +3,18 @@
use serde_json::{Value, json};
use filmcraft_interchange::{ExportOptions, Format, ImportOptions};
use filmcraft_project::{ItemId, ItemKind, MediaRef};
use filmcraft_project::{BinId, ItemId, ItemKind, MediaRef};
use crate::{EngineError, Result, Session};
/// File extensions of the interchange documents `file.import` reads (lower case). The Media
/// Browser lists the same set.
pub const EXTENSIONS: [&str; 7] = ["edl", "xml", "fcpxml", "otio", "aaf", "omf", "omfi"];
/// The interchange format of a file, if it is one (by content, with the extension as a hint).
pub fn detect(path: &str, bytes: &[u8]) -> Option<Format> {
let ext = std::path::Path::new(path).extension().and_then(|e| e.to_str()).map(str::to_ascii_lowercase);
if !matches!(ext.as_deref(), Some("edl" | "xml" | "fcpxml" | "otio" | "aaf" | "omf" | "omfi")) {
if !ext.as_deref().is_some_and(|e| EXTENSIONS.contains(&e)) {
return None;
}
filmcraft_interchange::detect(bytes, ext.as_deref())
@@ -47,8 +51,10 @@ fn write_embedded_audio(s: &Session, path: &str, wav: &[u8]) -> Result<String> {
}
/// Import a document: merge its bins, media and sequences into the project (one undo step), then
/// link each media file that exists on disk. Returns the new sequences.
pub fn import(s: &mut Session, path: &str, bytes: &[u8], format: Format) -> Result<Value> {
/// link each media file that exists on disk. Returns the new sequences. The document's items go
/// into `bin` (the Project panel's current bin, as for media files); the root when it is `None`
/// or no longer exists.
pub fn import(s: &mut Session, path: &str, bytes: &[u8], format: Format, bin: Option<BinId>) -> Result<Value> {
let p = std::path::Path::new(path);
let opts = ImportOptions {
base_dir: p.parent().map(|d| d.to_string_lossy().to_string()),
@@ -83,7 +89,7 @@ pub fn import(s: &mut Session, path: &str, bytes: &[u8], format: Format) -> Resu
let before: std::collections::HashSet<ItemId> = s.project.items.keys().copied().collect();
let name = opts.name.clone().unwrap_or_else(|| format.name().to_string());
let seqs = s.edit(&format!("Import {name}"), |proj, _| {
let seqs = filmcraft_interchange::merge_into(proj, fragment, None);
let seqs = filmcraft_interchange::merge_into(proj, fragment, bin);
// A document that sets only some Motion parameters leaves the others at their "auto"
// (NaN) defaults. Resolve them as placing a clip does, for every format. A media file's
// picture size is only what the document says so far (an EDL says nothing: the importer
crates/engine/src/interchange_auto_points_tests.rs
+67
View File
@@ -256,3 +256,70 @@ fn an_interchange_import_links_media_without_reading_the_files_whole() {
assert_eq!(s.project.source_size(clip.item), Some((320, 180)), "the file's real size, from its index");
let _ = std::fs::remove_dir_all(&d);
}
/// A Final Cut slideshow remains editable offline, survives saving, and renders after relinking.
#[test]
fn fcpxml_video_still_can_be_relinked_after_save_and_reopen() {
let d = tmp_dir("fcpxml-video-still");
let doc = d.join("slides.fcpxml");
let xml = r#"<fcpxml version="1.11"><resources>
<format id="f" frameDuration="1/30s" width="16" height="16"/>
<asset id="a" name="Slide" start="0s" duration="0s" hasVideo="1" format="f">
<media-rep kind="original-media" src="slide.png"/>
</asset>
</resources><project name="Slides"><sequence format="f" tcStart="0s"><spine>
<video ref="a" name="Slide" offset="0s" start="3600s" duration="10s"/>
</spine></sequence></project></fcpxml>"#;
std::fs::write(&doc, xml).unwrap();
let mut s = Session::default();
let before = filmcraft_format::encode(&s.project, false);
let (report, seq) = import(&mut s, &doc);
assert_eq!(report["linkedMedia"], 0);
assert_eq!(report["offlineMedia"].as_array().unwrap().len(), 1, "{report}");
let clip = first_clip(&s, seq);
assert_eq!(clip.duration, Tick::from_seconds_f64(10.0));
let missing = s.execute("media.linkMedia", json!({})).unwrap();
assert_eq!(missing["missing"].as_array().unwrap().len(), 1, "the slideshow must expose a relinkable media item: {missing}");
assert_eq!(missing["missing"][0]["status"], "missing");
assert_eq!(missing["missing"][0]["item"], clip.item.0);
let imported = filmcraft_format::encode(&s.project, false);
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(filmcraft_format::encode(&s.project, false), before);
s.execute("edit.redo", json!({})).unwrap();
assert_eq!(filmcraft_format::encode(&s.project, false), imported);
let saved = d.join("slides.fcproj");
s.execute("file.saveAs", json!({"path": saved.to_string_lossy()})).unwrap();
let mut reopened = Session::default();
reopened.execute("file.open", json!({"path": saved.to_string_lossy()})).unwrap();
assert_eq!(first_clip(&reopened, seq).item, clip.item);
let waiting = filmcraft_format::encode(&reopened.project, false);
let image = d.join("found.png");
image::RgbaImage::from_pixel(16, 16, image::Rgba([255, 0, 0, 255])).save(&image).unwrap();
let linked = reopened.execute("media.relink", json!({"item": clip.item.0, "path": image.to_string_lossy(), "force": true})).unwrap();
assert_eq!(linked["relinked"].as_array().unwrap().len(), 1, "{linked}");
let media = reopened.project.item(clip.item).unwrap().as_media().unwrap();
assert!(!media.offline);
assert_eq!(media.info.kind, filmcraft_media::MediaKind::Still);
assert_eq!(first_clip(&reopened, seq).source_in, clip.source_in);
let after_link = filmcraft_format::encode(&reopened.project, false);
reopened.execute("edit.undo", json!({})).unwrap();
assert_eq!(filmcraft_format::encode(&reopened.project, false), waiting);
reopened.execute("edit.redo", json!({})).unwrap();
assert_eq!(filmcraft_format::encode(&reopened.project, false), after_link);
for frame in [0, 150, 299] {
let (w, h, rgba) = crate::media_test_util::frame_rgba(&mut reopened, frame, 1.0);
assert_eq!((w, h), (16, 16));
assert!(rgba.as_chunks::<4>().0.iter().all(|p| *p == [255, 0, 0, 255]), "the relinked still must render for the whole clip");
}
// When the original path exists, normal import links it without a separate relink step.
std::fs::copy(&image, d.join("slide.png")).unwrap();
let mut online = Session::default();
let (report, online_seq) = import(&mut online, &doc);
assert_eq!(report["linkedMedia"], 1, "{report}");
assert!(report["offlineMedia"].as_array().unwrap().is_empty());
assert_eq!(first_clip(&online, online_seq).duration, clip.duration);
assert_eq!(crate::media_test_util::frame_rgba(&mut online, 150, 1.0), crate::media_test_util::frame_rgba(&mut reopened, 150, 1.0));
let _ = std::fs::remove_dir_all(d);
}
crates/engine/src/interchange_bin_tests.rs
+149
View File
@@ -0,0 +1,149 @@
//! Interchange documents (EDL, FCP7 XML, AAF, …) imported while a bin is open in the Project
//! panel land in that bin, as media files do, not at the project root (#2036).
use filmcraft_project::{BinId, ItemId};
use serde_json::{Value, json};
use crate::Session;
use crate::media_test_util::{make_movie, session_with, tmp_dir};
const RATE: &str = "<rate><timebase>24</timebase><ntsc>FALSE</ntsc></rate>";
/// A one-clip FCP7 XML sequence of `shot.mov`.
fn fcp7_xml() -> String {
format!(
r#"<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE xmeml>
<xmeml version="4">
<sequence id="sequence-1">
<name>Cut</name>
<duration>24</duration>
{RATE}
<media>
<video>
<format><samplecharacteristics>{RATE}<width>1920</width><height>1080</height></samplecharacteristics></format>
<track>
<clipitem id="clipitem-1">
<name>shot</name>
{RATE}
<start>0</start><end>24</end><in>0</in><out>24</out>
<file id="file-1">
<name>shot.mov</name>
<pathurl>shot.mov</pathurl>
{RATE}
<duration>24</duration>
</file>
</clipitem>
</track>
</video>
</media>
</sequence>
</xmeml>
"#
)
}
const EDL: &str = "TITLE: Cut\nFCM: NON-DROP FRAME\n\n001 AX V C 00:00:00:00 00:00:01:00 01:00:00:00 01:00:01:00\n* FROM CLIP NAME: shot.mov\n* SOURCE FILE: shot.mov\n";
/// Bins "Avis" ▸ "Kordex", as in the report.
fn nested_bins(s: &mut Session) -> (BinId, BinId) {
let outer = s.execute("file.newBin", json!({"name": "Avis"})).unwrap()["bin"].as_u64().unwrap();
let inner = s.execute("file.newBin", json!({"name": "Kordex", "parent": outer})).unwrap()["bin"].as_u64().unwrap();
(BinId(outer), BinId(inner))
}
/// The items an import added (every one of them: the sequences and the media they use).
fn added(s: &Session, before: &[ItemId]) -> Vec<ItemId> {
s.project.items.keys().copied().filter(|i| !before.contains(i)).collect()
}
/// Whether `id` is in `bin` or one of its sub-bins (a document's own bins come along inside it).
fn within(s: &Session, bin: BinId, id: ItemId) -> bool {
let mut all = Vec::new();
if let Some(b) = s.project.root.find_bin(bin) {
b.all_items(&mut all);
}
all.contains(&id)
}
fn sequences_of(r: &Value) -> Vec<ItemId> {
let docs = r["documents"].as_array().cloned().unwrap_or_default();
let mut seqs: Vec<ItemId> = r["sequences"].as_array().cloned().unwrap_or_default().iter().filter_map(Value::as_u64).map(ItemId).collect();
if seqs.is_empty() {
seqs = docs.iter().flat_map(|d| d["sequences"].as_array().cloned().unwrap_or_default()).filter_map(|v| v.as_u64()).map(ItemId).collect();
}
seqs
}
#[test]
fn an_interchange_document_lands_in_the_bin_it_was_imported_into() {
let d = tmp_dir("ix-bin");
make_movie(&d.join("shot.mov"), filmcraft_media::DemoScene::OceanSunset, 64, 36, 24);
std::fs::write(d.join("cut.xml"), fcp7_xml()).unwrap();
std::fs::write(d.join("cut.edl"), EDL).unwrap();
for doc in ["cut.xml", "cut.edl"] {
let mut s = Session::default();
let (_, kordex) = nested_bins(&mut s);
let before_items: Vec<ItemId> = s.project.items.keys().copied().collect();
let before = s.project.clone();
let path = d.join(doc).to_string_lossy().into_owned();
let r = s.execute("file.import", json!({"paths": [path], "bin": kordex.0})).unwrap();
let seqs = sequences_of(&r);
assert_eq!(seqs.len(), 1, "{doc}: {r}");
let new = added(&s, &before_items);
assert!(new.len() >= 2, "{doc}: a sequence and its media: {new:?}");
for id in &new {
assert!(within(&s, kordex, *id), "{doc}: item {id:?} is in the bin the import was aimed at");
}
// the import (and Link Media) undo back to the project as it was
while *s.project != *before {
s.execute("edit.undo", json!({})).unwrap();
}
assert!(s.project.root.find_bin(kordex).is_some_and(|b| b.children.is_empty()), "{doc}");
}
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn an_interchange_document_aimed_at_an_unknown_bin_lands_at_the_root() {
let d = tmp_dir("ix-bin-unknown");
std::fs::write(d.join("cut.xml"), fcp7_xml()).unwrap();
let mut s = Session::default();
let (avis, _) = nested_bins(&mut s);
let before: Vec<ItemId> = s.project.items.keys().copied().collect();
let path = d.join("cut.xml").to_string_lossy().into_owned();
s.execute("file.import", json!({"paths": [path], "bin": 987_654_321u64})).unwrap();
let new = added(&s, &before);
assert!(!new.is_empty());
for id in new {
assert!(!within(&s, avis, id), "item {id:?} is not in a bin it wasn't aimed at");
assert!(within(&s, s.project.root.id, id), "item {id:?} is in the project");
}
let _ = std::fs::remove_dir_all(&d);
}
/// AAF goes through the same import, from `file.import` and from `file.importAaf`.
#[test]
fn an_aaf_lands_in_the_bin_from_file_import_and_file_import_aaf() {
let d = tmp_dir("ix-bin-aaf");
let a = d.join("a.mov");
make_movie(&a, filmcraft_media::DemoScene::Aurora, 64, 36, 24);
let (mut s, _, seq) = session_with(&[&a]);
let aaf = d.join("cut.aaf").to_string_lossy().into_owned();
s.execute("file.exportAaf", json!({"path": aaf, "audio": "embedded", "sequence": seq.0})).unwrap();
for (cmd, params) in [("file.import", json!({"paths": [aaf]})), ("file.importAaf", json!({"path": aaf}))] {
let mut t = Session::default();
let (_, kordex) = nested_bins(&mut t);
let before: Vec<ItemId> = t.project.items.keys().copied().collect();
let mut params = params;
params["bin"] = json!(kordex.0);
let r = t.execute(cmd, params).unwrap();
assert_eq!(sequences_of(&r).len(), 1, "{cmd}: {r}");
let new = added(&t, &before);
assert!(new.len() >= 2, "{cmd}: {new:?}");
for id in new {
assert!(within(&t, kordex, id), "{cmd}: item {id:?}");
}
}
let _ = std::fs::remove_dir_all(&d);
}
crates/engine/src/keyboard.rs
+21 -8
View File
@@ -100,15 +100,14 @@ fn go_to_selected(s: &mut Session, end: bool) -> Result<Value> {
Ok(json!({"time": s.playhead().0}))
}
/// Reveal Nested Sequence: open the nested sequence of the selected clip (or the topmost nested
/// clip under the playhead on a targeted track) with its playhead on the matching frame.
fn reveal_nested(s: &mut Session) -> Result<Value> {
/// The clip Reveal Nested Sequence opens: the first selected nested sequence clip, else the
/// topmost one under the playhead on a targeted track.
fn nested_to_reveal(s: &Session) -> Option<filmcraft_project::TrackItem> {
let ph = s.playhead();
let tg = s.targeting().targeted;
let seq = s.active_sequence().ok_or(EngineError::NoSequence)?;
let seq = s.active_sequence()?;
let is_nest = |it: &filmcraft_project::TrackItem| s.project.sequence(it.item).is_some();
let pick = s
.state
s.state
.selection
.iter()
.filter_map(|c| seq.find_item(*c).map(|(_, i)| i))
@@ -116,7 +115,21 @@ fn reveal_nested(s: &mut Session) -> Result<Value> {
.or_else(|| {
seq.video_tracks.iter().rev().chain(seq.audio_tracks.iter()).filter(|t| tg.contains(&t.id)).filter_map(|t| t.item_at(ph)).find(|i| is_nest(i))
})
.ok_or_else(|| EngineError::Other("select a nested sequence clip".into()))?;
.cloned()
}
/// Reveal Nested Sequence is enabled when there is a nested clip to reveal, so the Timeline's
/// clip menu and the Sequence menu grey it out on plain clips instead of failing when chosen.
fn has_nested_to_reveal(s: &Session) -> std::result::Result<(), String> {
has_seq(s)?;
nested_to_reveal(s).map(|_| ()).ok_or_else(|| "select a nested sequence clip".into())
}
/// Reveal Nested Sequence: open the nested sequence of the selected clip (or the topmost nested
/// clip under the playhead on a targeted track) with its playhead on the matching frame.
fn reveal_nested(s: &mut Session) -> Result<Value> {
let ph = s.playhead();
let pick = nested_to_reveal(s).ok_or_else(|| EngineError::Other("select a nested sequence clip".into()))?;
let (nested, t) = (pick.item, pick.source_time_at(ph.clamp(pick.start, pick.end() - Tick(1))));
s.execute("sequence.open", json!({"item": nested.0}))?;
s.set_playhead(t);
@@ -723,7 +736,7 @@ pub fn commands() -> Vec<CommandSpec> {
spec("playhead.prevEditAnyTrack", "Go to Previous Edit Point on Any Track", Some("Shift+Up"), "{}", has_seq, |s, _| go_to_edit(s, false, true)),
spec("playhead.selectedClipStart", "Go to Selected Clip Start", Some("Shift+Home"), "{}", has_selection, |s, _| go_to_selected(s, false)),
spec("playhead.selectedClipEnd", "Go to Selected Clip End", Some("Shift+End"), "{}", has_selection, |s, _| go_to_selected(s, true)),
spec("sequence.revealNested", "Reveal Nested Sequence", Some("Cmd+Alt+F"), "{}", has_seq, |s, _| reveal_nested(s)),
spec("sequence.revealNested", "Reveal Nested Sequence", Some("Cmd+Alt+F"), "{}", has_nested_to_reveal, |s, _| reveal_nested(s)),
// ---- selection
spec("timeline.selectClipAtPlayhead", "Select Clip at Playhead", Some("D"), "{}", has_seq, |s, _| select_clip_at_playhead(s)),
spec("timeline.selectNextClip", "Select Next Clip", Some("Cmd+Down"), "{}", has_seq, |s, _| select_step(s, true)),
crates/engine/src/keyframe_move.rs
+61 -7
View File
@@ -1,5 +1,6 @@
//! Atomic movement of keys across effect/mask parameters; the common timeline delta keeps
//! relative timing intact, including at the clip boundaries. Selection itself is a UI concern.
//! Atomic edits of keys across effect/mask parameters: movement, where the common timeline delta
//! keeps relative timing intact, including at the clip boundaries, and interpolation. Selection
//! itself is a UI concern.
use std::collections::{BTreeMap, HashSet};
use filmcraft_project::{ClipId, Keyframe, Param, Sequence, TrackItem};
@@ -25,14 +26,14 @@ struct Key {
type Lane = (u64, usize, Option<usize>, String);
fn param_mut<'a>(sequence: &'a mut Sequence, lane: &Lane) -> Result<&'a mut Param> {
let (_, it) = sequence.find_item_mut(ClipId(lane.0)).ok_or_else(|| bad(COMMAND, "clip does not exist"))?;
let effect = it.effects.get_mut(lane.1).ok_or_else(|| bad(COMMAND, "effect does not exist"))?;
fn param_mut<'a>(sequence: &'a mut Sequence, lane: &Lane, command: &str) -> Result<&'a mut Param> {
let (_, it) = sequence.find_item_mut(ClipId(lane.0)).ok_or_else(|| bad(command, "clip does not exist"))?;
let effect = it.effects.get_mut(lane.1).ok_or_else(|| bad(command, "effect does not exist"))?;
match lane.2 {
Some(mask) => effect.masks.get_mut(mask).and_then(|m| m.param_mut(&lane.3)),
None => effect.params.get_mut(&lane.3),
}
.ok_or_else(|| bad(COMMAND, "parameter does not exist"))
.ok_or_else(|| bad(command, "parameter does not exist"))
}
fn direction(it: &TrackItem) -> f64 {
@@ -94,7 +95,7 @@ pub(crate) fn move_many(s: &mut Session, p: &Value) -> Result<Value> {
}
s.edit_sequence("Move Keyframes", |sequence, _, _| {
for (lane, moved) in lanes {
let param = param_mut(sequence, &lane)?;
let param = param_mut(sequence, &lane, COMMAND)?;
let from: HashSet<_> = moved.iter().map(|(t, _)| *t).collect();
let to: HashSet<_> = moved.iter().map(|(_, k)| k.time).collect();
// Remove all originals before inserting any destination: selected keys can move into
@@ -108,9 +109,32 @@ pub(crate) fn move_many(s: &mut Session, p: &Value) -> Result<Value> {
Ok(json!({"delta":delta,"keyframes":keys}))
}
/// `effects.setInterpolation` given a `keyframes` list: every listed key changes in one undo step,
/// so a menu applies to the whole selection (#1916).
pub(crate) fn interpolate_many(s: &mut Session, p: &Value) -> Result<Value> {
const SET: &str = "effects.setInterpolation";
let raw = p.get("keyframes").and_then(Value::as_array).ok_or_else(|| bad(SET, "need keyframes"))?;
if raw.is_empty() || raw.len() > MAX_KEYS {
return Err(bad(SET, "keyframe count must be between 1 and 4096"));
}
let keys: Vec<Key> = serde_json::from_value(Value::Array(raw.clone())).map_err(|_| bad(SET, "invalid keyframe reference"))?;
let kind = crate::commands::interpolation_p(p);
s.edit_sequence("Keyframe Interpolation", |sequence, _, _| {
for key in &keys {
let lane = (key.clip, key.effect, key.mask, key.param.clone());
let param = param_mut(sequence, &lane, SET)?;
let k = param.keyframes.iter_mut().find(|k| k.time == key.time).ok_or_else(|| bad(SET, "keyframe does not exist"))?;
k.interp = kind;
}
Ok(())
})?;
Ok(Value::Null)
}
#[cfg(test)]
mod tests {
use super::*;
use filmcraft_project::Interpolation;
use std::sync::Arc;
fn fixture() -> (Session, Vec<Value>) {
@@ -196,6 +220,36 @@ mod tests {
}
}
#[test]
fn interpolation_of_listed_keys_across_lanes_is_one_undo_step() {
const SET: &str = "effects.setInterpolation";
let (mut s, keys) = fixture();
let project = s.project.clone();
let undo = s.history.undo.len();
s.execute(SET, json!({"keyframes":[keys[0],keys[2]],"interpolation":"hold"})).unwrap();
let interps = |s: &Session, key: &Value| values(s, key).iter().map(|k| k.interp).collect::<Vec<_>>();
assert_eq!(interps(&s, &keys[0]), [Interpolation::Hold, Interpolation::Linear], "the unlisted Scale key stays linear");
assert_eq!(interps(&s, &keys[2]), [Interpolation::Hold]);
assert_eq!(s.history.undo.len(), undo + 1);
s.undo();
assert_eq!(serde_json::to_value(&s.project).unwrap(), serde_json::to_value(&project).unwrap());
let before = s.project.clone();
let mut invalid = keys[1].clone();
invalid["time"] = json!(invalid["time"].as_i64().unwrap() + 1);
for request in [
json!({"keyframes":[keys[0],invalid],"interpolation":"hold"}),
json!({"keyframes":[],"interpolation":"hold"}),
json!({"keyframes":vec![keys[0].clone();MAX_KEYS+1],"interpolation":"hold"}),
json!({"keyframes":[{"clip":keys[0]["clip"],"effect":u64::MAX,"param":"scale","time":0}],"interpolation":"hold"}),
json!({"keyframes":[{"clip":keys[0]["clip"],"effect":0,"param":"scale","mask":3,"time":0}],"interpolation":"hold"}),
json!({"keyframes":"all","interpolation":"hold"}),
] {
assert!(s.execute(SET, request).is_err());
assert!(Arc::ptr_eq(&before, &s.project));
}
}
#[test]
fn media_times_follow_clip_speed_and_reverse() {
for reverse in [false, true] {
crates/engine/src/lib.rs
+54 -3
View File
@@ -29,6 +29,8 @@ mod marker_export;
pub mod masks;
pub mod media_browser;
pub mod media_pool;
#[cfg(not(target_arch = "wasm32"))]
mod media_queue;
pub mod media_watch;
pub mod mixer;
pub mod multicam;
@@ -401,6 +403,11 @@ pub struct EditorState {
/// Sequence markers copied with the clipboard clips (start relative to the copied range).
#[serde(skip)]
pub clipboard_markers: Vec<filmcraft_project::Marker>,
/// Transitions copied with the clipboard clips (#3450): those whose every clip is copied, with
/// their track kind and relative track index (start relative to the copied range, `from`/`to`
/// still the copied clips' original ids).
#[serde(skip)]
pub clipboard_transitions: Vec<(TrackKind, usize, filmcraft_project::Transition)>,
/// Captions copied from the Timeline, with their caption track's index (starts relative to the
/// copied range). Copying clips empties it and copying captions empties `clipboard`, so Paste
/// places whichever was copied last.
@@ -632,6 +639,8 @@ impl Session {
pub fn start_autosave(&mut self, cfg: autosave::AutosaveConfig) -> std::io::Result<()> {
let prefs_path = cfg.data_dir.join("preferences.json");
self.prefs = autosave::Preferences::load(&prefs_path);
// a restart restores the persisted Selection Follows Playhead toggle (#3089)
self.state.selection_follows_playhead = self.prefs.timeline.selection_follows_playhead;
self.prefs.apply_hardware_decoding();
self.media.set_use_proxies(self.prefs.media.enable_proxies);
self.shortcuts.set_dir(&cfg.data_dir);
@@ -704,6 +713,8 @@ impl Session {
pub fn set_prefs(&mut self, p: autosave::Preferences) -> std::io::Result<()> {
let cache_changed = self.prefs.media_cache != p.media_cache;
self.prefs = p;
// the session mirrors the persisted Selection Follows Playhead toggle (#3089)
self.state.selection_follows_playhead = self.prefs.timeline.selection_follows_playhead;
self.prefs.apply_hardware_decoding();
if cache_changed {
self.apply_media_cache();
@@ -747,7 +758,8 @@ impl Session {
/// Run a command by id.
pub fn execute(&mut self, id: &str, params: Value) -> Result<Value> {
let spec = commands::find(id).ok_or_else(|| EngineError::UnknownCommand(id.to_string()))?;
if self.exec_depth == 0 && spec.journal && self.trim_play.active() {
let journal = spec.journal && !commands::reports_only(id, &params);
if self.exec_depth == 0 && journal && self.trim_play.active() {
self.settle_trim_playback(id);
}
if let Err(why) = self.enabled_for(spec, &params) {
@@ -757,6 +769,14 @@ impl Session {
}
return Err(e);
}
// Only an editing command can introduce clips for the sequence-only scope. Loading a
// document or restoring an undo snapshot must not run recognition or change that snapshot.
let transcription_before = (self.exec_depth == 0
&& spec.journal
&& self.prefs.media_analysis.auto_transcribe
&& self.prefs.media_analysis.auto_transcribe_scope == "sequenceClips"
&& !matches!(id, "edit.undo" | "edit.redo" | "file.newProject" | "file.openDemoProject" | "file.open" | "file.revert" | "file.recover"))
.then(|| self.project.clone());
self.exec_depth += 1;
// Last-resort guard: a panic in a command (a bug that escaped the never-crash rules)
// becomes an error for the caller (UI, CLI, control channel, MCP) instead of taking the
@@ -768,9 +788,18 @@ impl Session {
source_monitor::track_history(self);
}
// source graphics: an edited instance updates the shared layers and the other instances
if r.is_ok() && spec.journal && id.starts_with("graphics.") && !self.project.source_graphics.is_empty() {
if r.is_ok() && journal && id.starts_with("graphics.") && !self.project.source_graphics.is_empty() {
graphic_templates::sync_source_graphics(self);
}
if r.is_ok()
&& let Some(before) = transcription_before
&& !Arc::ptr_eq(&before, &self.project)
&& let Err(e) = transcript::auto_transcribe_sequence_clips(self, &before)
{
let message = format!("Automatic transcription: {e}");
self.log.push(panels::Level::Warning, id, message.clone());
self.events.push(Event::Toast { message, error: true });
}
if self.exec_depth == 0
&& let Err(e) = &r
{
@@ -779,7 +808,7 @@ impl Session {
self.sync_persistence();
// playback reads the newest snapshot (mixer moves, mutes… are heard while playing)
self.previews.live.publish_project(self.project.clone());
if r.is_ok() && spec.journal {
if r.is_ok() && journal {
self.journal.push((id.to_string(), params));
if self.journal.len() > 10_000 {
self.journal.drain(..1000);
@@ -887,12 +916,29 @@ impl Session {
}
}
/// Every edit passes through here too: a transition whose clip the edit removed, or whose two
/// clips it moved apart, goes away (`filmcraft_edit::remove_orphan_transitions`), whichever
/// command made the edit. Sequences the edit did not touch are left as they are (the edit
/// cloned only what it changed, so this costs nothing for them).
fn drop_orphan_transitions(&self, edited: &mut Project) {
for (id, it) in &mut edited.items {
let filmcraft_project::ItemKind::Sequence(q) = &mut it.kind else { continue };
if matches!(self.project.items.get(id).map(|o| &o.kind), Some(filmcraft_project::ItemKind::Sequence(o)) if Arc::ptr_eq(o, q)) {
continue;
}
for t in Arc::make_mut(q).all_tracks_mut() {
filmcraft_edit::remove_orphan_transitions(t);
}
}
}
/// Apply an undoable project edit. The closure gets a mutable copy; on error nothing changes.
pub fn edit<R>(&mut self, label: &str, f: impl FnOnce(&mut Project, &mut EditorState) -> Result<R>) -> Result<R> {
let mut p = (*self.project).clone();
let mut st = self.state.clone();
let r = f(&mut p, &mut st)?;
self.refuse_self_nesting(&p)?;
self.drop_orphan_transitions(&mut p);
let old = std::mem::replace(&mut self.project, Arc::new(p));
self.history.undo.push((label.to_string(), old));
if self.history.undo.len() > self.history.limit {
@@ -919,6 +965,7 @@ impl Session {
let mut st = self.state.clone();
let r = f(&mut p, &mut st)?;
self.refuse_self_nesting(&p)?;
self.drop_orphan_transitions(&mut p);
self.project = Arc::new(p);
self.state = st;
self.drop_stale_gap();
@@ -1276,6 +1323,8 @@ mod autosave_tests;
#[cfg(test)]
mod avi_import_tests;
#[cfg(test)]
mod clip_mismatch_tests;
#[cfg(test)]
mod clip_ops_tests;
#[cfg(test)]
mod color_tests;
@@ -1298,6 +1347,8 @@ mod individual_selection_tests;
#[cfg(test)]
mod interchange_auto_points_tests;
#[cfg(test)]
mod interchange_bin_tests;
#[cfg(test)]
mod keyboard_tests;
#[cfg(test)]
mod linked_speed_tests;
crates/engine/src/locked_clip_edits_tests.rs
+46
View File
@@ -82,3 +82,49 @@ fn locked_track_refuses_roll_slip_slide_speed() {
assert_eq!(*s.project, *snap, "{} must leave the sequence unchanged", cmd["id"]);
}
}
/// #1903: a move that lists one clip twice is refused before anything changes; it used to leave
/// the same clip id on both destination tracks. A copy may repeat a source.
#[test]
fn move_refuses_a_clip_listed_twice() {
let mut s = Session::default();
s.execute("file.newProject", json!({"name": "Twice"})).unwrap();
s.execute("file.newSequence", json!({"name": "Sequence", "fps": 25, "video": 2, "audio": 2, "width": 16, "height": 16})).unwrap();
s.execute("file.newOfflineFile", json!({"name": "Media", "seconds": 4, "fps": 25, "video": true, "audio": true, "width": 16, "height": 16})).unwrap();
let item = s.project.items.values().find(|i| i.as_media().is_some()).map(|i| i.id.0).unwrap();
let placed = s.execute("timeline.place", json!({"item": item, "track": "V1", "audioTrack": "A1", "time": 0})).unwrap();
let clip = placed["clips"][0].as_u64().unwrap();
let before = s.project.to_json();
let undo_len = s.history.undo.len();
let t = 2_032_128_000_000i64;
let err = s
.execute("timeline.move", json!({"moves": [{"clip": clip, "track": "V1", "time": t}, {"clip": clip, "track": "V2", "time": t}], "linked": false}))
.unwrap_err();
assert!(err.to_string().contains("more than once"), "{err}");
assert_eq!(s.project.to_json(), before, "nothing moved");
assert_eq!(s.history.undo.len(), undo_len, "no undo step");
let ids: Vec<u64> = s.active_sequence().unwrap().all_tracks().flat_map(|t| t.items.iter().map(|i| i.id.0)).collect();
let mut sorted = ids.clone();
sorted.sort_unstable();
sorted.dedup();
assert_eq!(sorted.len(), ids.len(), "every clip id is on one track: {ids:?}");
// the same with insert
let err = s
.execute(
"timeline.move",
json!({"moves": [{"clip": clip, "track": "V1", "time": t}, {"clip": clip, "track": "V2", "time": t}], "linked": false, "insert": true}),
)
.unwrap_err();
assert!(err.to_string().contains("more than once"), "{err}");
// copies may repeat a source: each copy is a new clip
let r = s
.execute(
"timeline.move",
json!({"moves": [{"clip": clip, "track": "V2", "time": 0}, {"clip": clip, "track": "V2", "time": t}], "linked": false, "copy": true}),
)
.unwrap();
let copied: Vec<u64> = r["copied"].as_array().unwrap().iter().filter_map(|v| v.as_u64()).collect();
assert_eq!(copied.len(), 2, "{r}");
assert_ne!(copied[0], copied[1]);
assert!(!copied.contains(&clip));
}
crates/engine/src/masks.rs
+2 -2
View File
@@ -184,7 +184,7 @@ fn edit_mask<R>(s: &mut Session, p: &Value, label: &str, f: impl FnOnce(&mut Mas
let body = move |pr: &mut filmcraft_project::Project, _: &mut crate::EditorState| -> Result<R> {
let q = pr.sequence_mut(seq_id).ok_or(EngineError::NoSequence)?;
let (_, it) = q.find_item_mut(clip).ok_or(filmcraft_edit::EditError::NoItem(clip))?;
let mt = it.source_time_at(ph.clamp(it.start, it.end() - Tick(1)));
let mt = it.effect_time_at(ph.clamp(it.start, it.end() - Tick(1)));
let ei = effect_index(&it.effects, eff.as_ref(), sel, clip).ok_or_else(|| bad(label, "no such effect on clip"))?;
let e = &mut it.effects[ei];
let mi = mask_i.ok_or_else(|| bad(label, "need `mask`"))?;
@@ -557,7 +557,7 @@ fn list(s: &mut Session, p: &Value) -> Result<Value> {
if only.is_some_and(|c| c != it.id) {
continue;
}
let mt = it.source_time_at(ph.clamp(it.start, it.end() - Tick(1)));
let mt = it.effect_time_at(ph.clamp(it.start, it.end() - Tick(1)));
for (ei, e) in it.effects.iter().enumerate() {
for (mi, m) in e.masks.iter().enumerate() {
out.push(json!({
crates/engine/src/media_browser.rs
+58 -6
View File
@@ -11,7 +11,7 @@
//! | `mediaBrowser.select` | the selected files (File ▸ Import from Media Browser uses them) |
//! | `mediaBrowser.favorite` | Add to / Remove from Favorites |
//! | `mediaBrowser.clearRecent` | Clear Recent Directories |
//! | `mediaBrowser.settings` | file types, view (list / thumbnails), columns (Edit Columns…), Import as Image Sequence, Hover Scrub, thumbnail size |
//! | `mediaBrowser.settings` | file types, view (list / thumbnails), columns (Edit Columns…) and their widths, Import as Image Sequence, Hover Scrub, thumbnail size |
//! | `mediaBrowser.import` | Import (the selection or `paths`; a folder becomes a bin, its sub-folders bins inside it; Project ▸ Ingest settings apply) |
//! | `mediaBrowser.openInSource` | Open In Source Monitor (Shift+O) |
//! | `mediaBrowser.probe` | a file's media properties (frame rate, duration, video / audio info), cached |
@@ -19,7 +19,8 @@
//! Navigation history and the selection are session state ([`BrowserState`]); Favorites, recent
//! directories and the view settings are preferences ([`MediaBrowserPrefs`]).
use std::collections::BTreeMap;
use std::collections::{BTreeMap, BTreeSet};
use std::sync::{Arc, Mutex, PoisonError};
use filmcraft_media::MediaInfo;
use filmcraft_project::ItemId;
@@ -73,9 +74,11 @@ pub const ALL_COLUMNS: [&str; 10] =
["Name", "Media Type", "Size", "Date Modified", "Frame Rate", "Media Duration", "Video Info", "Audio Info", "Video Codec", "Audio Codec"];
pub const DEFAULT_COLUMNS: [&str; 5] = ["Name", "Frame Rate", "Media Duration", "Video Info", "Size"];
/// Project and interchange files the browser can import. Avid Log Exchange (`.ale`) is left out:
/// FilmCraft writes it (File ▸ Export) but, like Premiere, does not read it (#1292).
const PROJECT_EXTENSIONS: [&str; 5] = ["fcproj", "edl", "xml", "fcpxml", "otio"];
/// Project and interchange files the browser can import: FilmCraft projects and every
/// interchange document `file.import` reads ([`crate::interchange::EXTENSIONS`], AAF and OMF
/// included). Avid Log Exchange (`.ale`) is left out: FilmCraft writes it (File ▸ Export) but,
/// like Premiere, does not read it (#1292).
const PROJECT_EXTENSIONS: [&str; 8] = ["fcproj", "edl", "xml", "fcpxml", "otio", "aaf", "omf", "omfi"];
const CAPTION_EXTENSIONS: [&str; 3] = ["srt", "vtt", "scc"];
const RECENT_MAX: usize = 10;
@@ -93,6 +96,8 @@ pub struct MediaBrowserPrefs {
/// `list` | `thumbnails`.
pub view: String,
pub columns: Vec<String>,
/// List view column widths (points) the user dragged; the others keep their default width.
pub column_widths: BTreeMap<String, f32>,
pub import_as_image_sequence: bool,
pub hover_scrub: bool,
pub thumbnail_size: f32,
@@ -107,6 +112,7 @@ impl Default for MediaBrowserPrefs {
file_types: "all".into(),
view: "list".into(),
columns: DEFAULT_COLUMNS.iter().map(|c| c.to_string()).collect(),
column_widths: BTreeMap::new(),
import_as_image_sequence: false,
hover_scrub: true,
thumbnail_size: 120.0,
@@ -126,6 +132,11 @@ pub struct BrowserState {
/// Probed media properties by path (None = not a readable media file).
#[serde(skip)]
pub probes: BTreeMap<String, Option<MediaInfo>>,
/// Paths [`probe_background`] is probing, and the results its threads hand back.
#[serde(skip)]
pub probing: BTreeSet<String>,
#[serde(skip)]
pub probe_results: Arc<Mutex<Vec<(String, Option<MediaInfo>)>>>,
}
/// A listed entry with what the browser shows about it.
@@ -403,6 +414,13 @@ fn settings(s: &mut Session, p: &Value) -> Result<Value> {
}
mb.columns = cols;
}
if let Some(o) = p.get("columnWidths").and_then(Value::as_object) {
for (c, w) in o {
let c = ALL_COLUMNS.iter().find(|x| x.eq_ignore_ascii_case(c)).ok_or_else(|| bad("mediaBrowser.settings", format!("unknown column `{c}`")))?;
let w = w.as_f64().filter(|w| w.is_finite()).ok_or_else(|| bad("mediaBrowser.settings", format!("width of `{c}` is not a number")))?;
mb.column_widths.insert(c.to_string(), (w as f32).clamp(30.0, 800.0));
}
}
if let Some(v) = bool_p(p, "importAsImageSequence") {
mb.import_as_image_sequence = v;
}
@@ -598,6 +616,40 @@ pub fn probe(s: &mut Session, path: &str) -> Option<MediaInfo> {
info
}
/// At most this many [`probe_background`] threads at once.
const PROBE_THREADS: usize = 4;
/// [`probe`] without blocking: an unknown file is probed on a background thread and `None` is
/// returned until its result is in (big files take seconds to open).
pub fn probe_background(s: &mut Session, path: &str) -> Option<Option<MediaInfo>> {
let done = std::mem::take(&mut *s.browser.probe_results.lock().unwrap_or_else(PoisonError::into_inner));
for (p, info) in done {
s.browser.probing.remove(&p);
s.browser.probes.insert(p, info);
}
if let Some(v) = s.browser.probes.get(path) {
return Some(v.clone());
}
if cfg!(target_arch = "wasm32") || !kind_of(path).is_some_and(|k| matches!(k, "video" | "audio" | "image")) {
return Some(probe(s, path));
}
if s.browser.probing.contains(path) || s.browser.probing.len() >= PROBE_THREADS {
return None;
}
let (media, services, results, key) = (s.media.clone(), s.services.clone(), s.browser.probe_results.clone(), path.to_string());
let run = move || {
let info =
std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| media.probe_file(&key, &*services).ok().map(|src| src.info().clone()))).ok().flatten();
results.lock().unwrap_or_else(PoisonError::into_inner).push((key, info));
};
if std::thread::Builder::new().name("filmcraft-browser-probe".into()).spawn(run).is_ok() {
s.browser.probing.insert(path.to_string());
None
} else {
Some(probe(s, path))
}
}
/// The text of a list column for an entry (`probe`: its media properties, when known).
pub fn column_text(e: &Entry, column: &str, probe: Option<&MediaInfo>) -> String {
match column {
@@ -689,7 +741,7 @@ pub(crate) fn commands() -> Vec<CommandSpec> {
c(
"mediaBrowser.settings",
"Media Browser Settings",
r#"{"fileTypes":"all|video|audio|image|project|caption|<ext>"?,"view":"list|thumbnails"?,"columns":[str]?,"importAsImageSequence":bool?,"hoverScrub":bool?,"thumbnailSize":f32?}"#,
r#"{"fileTypes":"all|video|audio|image|project|caption|<ext>"?,"view":"list|thumbnails"?,"columns":[str]?,"columnWidths":{str:f32}?,"importAsImageSequence":bool?,"hoverScrub":bool?,"thumbnailSize":f32?}"#,
settings,
),
c("mediaBrowser.import", "Import", r#"{"paths":[str]?,"bin":binId?,"imageSequence":bool?}"#, import),
crates/engine/src/media_browser_tests.rs
+59
View File
@@ -130,6 +130,18 @@ fn file_type_filter() {
assert_eq!(s.prefs.media_browser.columns, ["Name", "Size", "Frame Rate"]);
}
#[test]
fn column_widths_persist_clamped() {
let mut s = session();
s.execute("mediaBrowser.settings", json!({"columnWidths": {"name": 300, "Size": 5}})).unwrap();
s.execute("mediaBrowser.settings", json!({"columnWidths": {"Frame Rate": 90}})).unwrap();
let w = &s.prefs.media_browser.column_widths;
assert_eq!((w.get("Name"), w.get("Size"), w.get("Frame Rate")), (Some(&300.0), Some(&30.0), Some(&90.0)));
assert!(s.execute("mediaBrowser.settings", json!({"columnWidths": {"Colour": 100}})).is_err());
assert!(s.execute("mediaBrowser.settings", json!({"columnWidths": {"Name": "wide"}})).is_err());
assert_eq!(s.prefs.media_browser.column_widths.len(), 3);
}
#[test]
fn navigation_back_forward_up_and_recent() {
let mut s = session();
@@ -327,3 +339,50 @@ fn ale_logs_are_not_offered_for_import() {
assert!(!media_browser::extensions().contains(&"ale"));
assert_eq!(media_browser::kind_of("cut.edl"), Some("project"));
}
/// The background probe returns `None` until its thread has the result, then the same properties
/// as the blocking probe; a path that isn't media is answered at once.
#[cfg(any(unix, windows))]
#[test]
fn probe_background_returns_the_blocking_probe_result() {
let dir = crate::media_test_util::tmp_dir("browser-probe-background");
let wav = dir.join("voice.wav");
std::fs::write(&wav, filmcraft_media::wav::write_wav16(&[0.1; 9_600], 2, 48_000)).unwrap();
let path = wav.to_string_lossy().to_string();
let mut s = Session::default();
let mut got = None;
for _ in 0..500 {
got = media_browser::probe_background(&mut s, &path);
if got.is_some() {
break;
}
std::thread::sleep(std::time::Duration::from_millis(10));
}
let background = got.expect("background probe finished").expect("wav has properties");
let mut fresh = Session::default();
let blocking = media_browser::probe(&mut fresh, &path).unwrap();
assert_eq!(background.duration, blocking.duration);
assert_eq!(background.audio().map(|a| a.sample_rate), blocking.audio().map(|a| a.sample_rate));
let notes = dir.join("notes.txt");
assert_eq!(media_browser::probe_background(&mut s, &notes.to_string_lossy()), Some(None));
let _ = std::fs::remove_dir_all(&dir);
}
/// AAF and OMF documents import through `file.import` like EDL, FCP7 XML, FCPXML and OTIO, so the
/// browser lists them as projects too, including the `.omfi` spelling the importer accepts.
#[test]
fn every_interchange_document_the_importer_reads_is_listed() {
for ext in crate::interchange::EXTENSIONS {
let name = format!("cut.{ext}");
assert_eq!(media_browser::kind_of(&name), Some("project"), "{name}");
assert_eq!(media_browser::kind_of(&name.to_ascii_uppercase()), Some("project"), "{name}");
assert!(media_browser::extensions().contains(&ext), "{ext}");
}
let mut s = session();
s.services.write_file("/home/me/Movies/cut.aaf", &[0; 16]).unwrap();
s.services.write_file("/home/me/Movies/mix.omf", &[0; 16]).unwrap();
let l = s.execute("mediaBrowser.list", json!({"path": "/home/me/Movies", "fileTypes": "project"})).unwrap();
assert_eq!(names(&l), ["Selects", "cut.aaf", "edit.edl", "mix.omf"]);
let l = s.execute("mediaBrowser.list", json!({"path": "/home/me/Movies", "fileTypes": "aaf"})).unwrap();
assert_eq!(names(&l), ["Selects", "cut.aaf"]);
}
crates/engine/src/media_pool.rs
+134 -2
View File
@@ -11,6 +11,9 @@
//! from it. Proxy frames are smaller; the source still reports the full-resolution size, and the
//! compositor derives its pixel scale from the frame it gets, so effects and Motion render the
//! same picture at lower resolution. Export asks for [`MediaPool::full_res_provider`].
//! - Background jobs (exports, Render Previews, proxies) read their own copy of each file
//! ([`MediaPool::own_source`], [`PoolProvider::own_sources`]): with the monitors' source they
//! would take turns on one decoder, seeking it back and forth between their frames.
//! - Paths that came from a document (the project file, a template, an interchange file) are
//! opened through [`crate::doc_path`] ([`MediaPool::follow`]); a refused one renders as missing.
@@ -361,6 +364,25 @@ impl MediaPool {
Some(src)
}
/// A copy of `item`'s file for a background job: the file opened again through the host's
/// reader, with decoders and decoded frames of its own, closed when the job drops it. None
/// when the item shares the pool's source instead: not a file (a generator), made offline, an
/// image sequence, a host without a reader (the browser), a format that would be read whole,
/// or a file that does not open or that the pool last saw offline (the pool's source renders
/// the slate and says why, so the job shows what the monitors show).
pub fn own_source(&self, p: &Project, item: ItemId, services: &dyn Services) -> Option<SharedSource> {
let (item, m, _) = p.resolve_media(item)?;
let MediaRef::File { path } = &m.media else { return None };
if m.offline || m.info.kind == filmcraft_media::MediaKind::ImageSequence || self.offline_status(item).is_some() {
return None;
}
let file = self.follow_media(path).ok()?;
let reader = services.reader(&file)?.ok()?;
let openers = self.openers.read().unwrap_or_else(PoisonError::into_inner).clone();
let src = filmcraft_media::reader::open_reader_within(&file_name(&file), reader, &filmcraft_codecs::reader_openers(), &openers, 0).ok()?;
Some(self.unsupported_as_slate(item, path, src))
}
fn set_offline(&self, item: ItemId, reason: OfflineReason, path: &str, error: &str) {
self.offline.write().unwrap_or_else(PoisonError::into_inner).insert(item, OfflineStatus { reason, path: path.to_string(), error: error.to_string() });
}
@@ -400,12 +422,12 @@ impl MediaPool {
/// A render-side provider bound to a project snapshot (proxies when enabled).
pub fn provider(self: &Arc<Self>, project: Arc<Project>, services: Arc<dyn Services>) -> PoolProvider {
PoolProvider { pool: self.clone(), project, services, full_res: false, used: None }
PoolProvider { pool: self.clone(), project, services, full_res: false, used: None, own: None }
}
/// A provider that always reads full-resolution media (export).
pub fn full_res_provider(self: &Arc<Self>, project: Arc<Project>, services: Arc<dyn Services>) -> PoolProvider {
PoolProvider { pool: self.clone(), project, services, full_res: true, used: None }
PoolProvider { pool: self.clone(), project, services, full_res: true, used: None, own: None }
}
}
@@ -418,9 +440,19 @@ pub struct PoolProvider {
pub full_res: bool,
/// Items asked for so far, when recording ([`PoolProvider::recording`]).
used: Option<Mutex<BTreeSet<ItemId>>>,
/// Per media item, its own copy or None to share the pool's source ([`PoolProvider::own_sources`]).
own: Option<Mutex<HashMap<ItemId, Option<SharedSource>>>>,
}
impl PoolProvider {
/// This full-resolution provider, reading its own copy of each file ([`MediaPool::own_source`])
/// instead of the source the monitors read: a background job then never makes them wait for
/// its frames, or seek their decoder away from the playhead. The copies close with the provider.
pub fn own_sources(mut self) -> Self {
self.own = self.full_res.then(Mutex::default);
self
}
/// This provider, remembering the items it is asked for so [`PoolProvider::offline_used`] can
/// tell which of them rendered offline.
pub fn recording(mut self) -> Self {
@@ -455,6 +487,26 @@ impl filmcraft_render::SourceProvider for PoolProvider {
if let Some(used) = &self.used {
used.lock().unwrap_or_else(PoisonError::into_inner).insert(item);
}
if let Some(own) = &self.own
&& let Some((media, _, _)) = self.project.resolve_media(item)
{
// opened under the lock: the job's threads open each file once
let mut own = own.lock().unwrap_or_else(PoisonError::into_inner);
let src = match own.get(&media) {
Some(src) => src.clone(),
None => {
let src = self.pool.own_source(&self.project, media, &*self.services);
// the host is still fetching the file's bytes (the browser): try again next time
if src.is_some() || !filmcraft_media::pending::is_set() {
own.insert(media, src.clone());
}
src
}
};
if let Some(src) = src {
return Some(src);
}
}
if self.full_res { self.pool.full_res_source(&self.project, item, &*self.services) } else { self.pool.source_for(&self.project, item, &*self.services) }
}
}
@@ -690,6 +742,86 @@ mod tests {
}
}
/// A host that reads files only whole (the browser without a reader).
struct WholeFiles;
impl Services for WholeFiles {
fn read_file(&self, path: &str) -> std::io::Result<Vec<u8>> {
std::fs::read(path)
}
fn write_file(&self, _path: &str, _data: &[u8]) -> std::io::Result<()> {
Ok(())
}
}
/// A host whose first reader is still fetching the file (the browser's `BlobReader`): it marks
/// the read pending, then reads the file.
struct FetchingFirst(Arc<[u8]>, std::sync::atomic::AtomicBool);
impl Services for FetchingFirst {
fn read_file(&self, path: &str) -> std::io::Result<Vec<u8>> {
Err(std::io::Error::other(format!("{path}: read whole")))
}
fn write_file(&self, _path: &str, _data: &[u8]) -> std::io::Result<()> {
Ok(())
}
fn reader(&self, _path: &str) -> Option<std::io::Result<filmcraft_media::SharedReader>> {
if !self.1.swap(true, Ordering::Relaxed) {
filmcraft_media::pending::mark();
return Some(Err(filmcraft_media::pending::would_block()));
}
Some(Ok(Arc::new(filmcraft_media::reader::MemReader(self.0.clone()))))
}
}
/// A file the host is still fetching gets its own copy once the bytes are there, instead of
/// the job reading the monitors' source to its end.
#[test]
fn own_sources_retry_a_file_still_being_fetched() {
use filmcraft_render::SourceProvider;
let movie = crate::media_test_util::tmp_dir("own-sources-pending").join("a.mp4");
crate::media_test_util::make_movie(&movie, filmcraft_media::DemoScene::Aurora, 64, 36, 12);
let (s, items, _) = crate::media_test_util::session_with(&[&movie]);
let monitor = s.media.full_res_source(&s.project, items[0], &*s.services).unwrap();
let host = Arc::new(FetchingFirst(std::fs::read(&movie).unwrap().into(), Default::default()));
let job = s.media.full_res_provider(s.project.clone(), host).own_sources();
assert!(job.source(items[0]).is_some_and(|x| Arc::ptr_eq(&x, &monitor)), "shared while the bytes are fetched");
assert!(filmcraft_media::pending::take());
assert!(job.source(items[0]).is_some_and(|x| !Arc::ptr_eq(&x, &monitor)), "its own copy once they are there");
}
/// A background job reads its own copy of a file, so the monitors' source keeps its decoder
/// and frames wherever the job reads. What cannot be copied is shared, as before.
#[test]
fn own_sources_read_their_own_copy_of_each_file() {
use filmcraft_render::SourceProvider;
let movie = crate::media_test_util::tmp_dir("own-sources").join("a.mp4");
crate::media_test_util::make_movie(&movie, filmcraft_media::DemoScene::Aurora, 64, 36, 12);
let (mut s, items, _) = crate::media_test_util::session_with(&[&movie]);
let item = items[0];
let monitor = s.media.source_for(&s.project, item, &*s.services).unwrap();
let job = s.media.full_res_provider(s.project.clone(), s.services.clone()).own_sources();
let own = job.source(item).unwrap();
assert!(!Arc::ptr_eq(&own, &monitor), "the job reads the monitors' source");
assert!(Arc::ptr_eq(&own, &job.source(item).unwrap()), "one copy per job");
let req = FrameRequest { time: filmcraft_time::Tick::ZERO, scale: 1.0 };
assert_eq!(own.video_frame(req).unwrap().to_rgba8(), monitor.video_frame(req).unwrap().to_rgba8());
assert!(s.media.full_res_provider(s.project.clone(), s.services.clone()).own_sources().source(item).is_some_and(|o| !Arc::ptr_eq(&o, &own)));
// shared: without own sources, a provider that may read proxies, a host without a reader
let shared = |p: PoolProvider| p.source(item).is_some_and(|x| Arc::ptr_eq(&x, &monitor));
assert!(shared(s.media.full_res_provider(s.project.clone(), s.services.clone())));
assert!(shared(s.media.provider(s.project.clone(), s.services.clone()).own_sources()));
assert!(shared(s.media.full_res_provider(s.project.clone(), Arc::new(WholeFiles)).own_sources()));
// a file the pool last saw offline renders as the monitors show it
s.media.set_offline(item, OfflineReason::Unreadable, &movie.to_string_lossy(), "damaged");
assert!(shared(s.media.full_res_provider(s.project.clone(), s.services.clone()).own_sources()));
// and media that is not a file
let r = s.execute("file.importDemoFootage", serde_json::json!({"scene": "Aurora"})).unwrap();
let generator = ItemId(r["items"][0].as_u64().unwrap());
let pooled = s.media.full_res_source(&s.project, generator, &*s.services).unwrap();
let job = s.media.full_res_provider(s.project.clone(), s.services.clone()).own_sources();
assert!(job.source(generator).is_some_and(|x| Arc::ptr_eq(&x, &pooled)));
}
#[test]
fn open_file_prefers_the_hosts_reader() {
let wav = filmcraft_media::wav::write_wav16(&[0.0, 0.5, -0.5, 0.25], 2, 48_000);
crates/engine/src/media_queue.rs
+176
View File
@@ -0,0 +1,176 @@
//! A first-in first-out line for background media jobs (proxies, transcodes, ingest).
//!
//! Each of those jobs is an export that already uses every core and holds decoded frames, so
//! starting a second one while the first runs only makes both slower and doubles the memory in
//! use (#1831). A job takes a [`Ticket`] when it is started, in click order, and waits for its turn
//! on its own thread; a cancelled job gives its place up.
use std::collections::BTreeSet;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Condvar, Mutex, MutexGuard, PoisonError};
use std::time::Duration;
struct State {
/// Tickets handed out so far.
issued: u64,
/// The ticket whose job may run now.
serving: u64,
/// Tickets given up before their turn came.
skipped: BTreeSet<u64>,
}
/// One line of jobs.
pub struct Queue {
state: Mutex<State>,
turn: Condvar,
}
/// How often a waiting job looks at its cancel flag.
const POLL: Duration = Duration::from_millis(100);
impl Queue {
pub const fn new() -> Self {
Self { state: Mutex::new(State { issued: 0, serving: 0, skipped: BTreeSet::new() }), turn: Condvar::new() }
}
fn lock(&self) -> MutexGuard<'_, State> {
self.state.lock().unwrap_or_else(PoisonError::into_inner)
}
/// A place at the back of the line. The ticket frees its place when dropped.
pub fn ticket(&'static self) -> Ticket {
let mut st = self.lock();
let n = st.issued;
st.issued += 1;
Ticket { queue: self, n }
}
/// How many jobs a new ticket would stand behind (running or waiting).
pub fn ahead(&self) -> u64 {
let st = self.lock();
st.issued - st.serving - st.skipped.len() as u64
}
fn finish(&self, n: u64) {
let mut st = self.lock();
if n == st.serving {
st.serving += 1;
loop {
let next = st.serving;
if !st.skipped.remove(&next) {
break;
}
st.serving += 1;
}
} else {
st.skipped.insert(n);
}
drop(st);
self.turn.notify_all();
}
}
/// A place in a [`Queue`].
pub struct Ticket {
queue: &'static Queue,
n: u64,
}
impl Ticket {
/// Wait until every earlier job is done. `false` when `cancel` was raised first; the place is
/// then given up when the ticket is dropped.
pub fn wait(&self, cancel: &AtomicBool) -> bool {
let mut st = self.queue.lock();
loop {
if st.serving == self.n {
return true;
}
if cancel.load(Ordering::Relaxed) {
return false;
}
st = self.queue.turn.wait_timeout(st, POLL).unwrap_or_else(PoisonError::into_inner).0;
}
}
}
impl Drop for Ticket {
fn drop(&mut self) {
self.queue.finish(self.n);
}
}
/// The line every media job in this process stands in.
pub static MEDIA_JOBS: Queue = Queue::new();
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Arc;
use std::sync::atomic::AtomicUsize;
fn leaked() -> &'static Queue {
Box::leak(Box::new(Queue::new()))
}
#[test]
fn jobs_run_one_at_a_time_in_the_order_they_were_started() {
let q = leaked();
let running = Arc::new(AtomicUsize::new(0));
let most = Arc::new(AtomicUsize::new(0));
let order = Arc::new(Mutex::new(Vec::new()));
// every ticket is taken here, in order, before any job thread runs
let tickets: Vec<Ticket> = (0..6).map(|_| q.ticket()).collect();
assert_eq!(q.ahead(), 6);
let handles: Vec<_> = tickets
.into_iter()
.enumerate()
.rev() // start the threads in the wrong order on purpose
.map(|(i, t)| {
let (running, most, order) = (running.clone(), most.clone(), order.clone());
std::thread::spawn(move || {
assert!(t.wait(&AtomicBool::new(false)));
most.fetch_max(running.fetch_add(1, Ordering::SeqCst) + 1, Ordering::SeqCst);
order.lock().unwrap().push(i);
std::thread::sleep(Duration::from_millis(10));
running.fetch_sub(1, Ordering::SeqCst);
})
})
.collect();
for h in handles {
h.join().unwrap();
}
assert_eq!(most.load(Ordering::SeqCst), 1, "never two at once");
assert_eq!(*order.lock().unwrap(), [0, 1, 2, 3, 4, 5]);
assert_eq!(q.ahead(), 0);
}
#[test]
fn a_cancelled_job_gives_its_place_up_without_stalling_the_line() {
let q = leaked();
let (first, second, third) = (q.ticket(), q.ticket(), q.ticket());
let cancel = Arc::new(AtomicBool::new(false));
let waiting = {
let cancel = cancel.clone();
std::thread::spawn(move || second.wait(&cancel))
};
// the second job is cancelled while the first still runs
std::thread::sleep(Duration::from_millis(30));
cancel.store(true, Ordering::Relaxed);
assert!(!waiting.join().unwrap(), "a cancelled wait does not get a turn");
assert_eq!(q.ahead(), 2, "its ticket was dropped with the thread, the first and third remain");
drop(first);
assert!(third.wait(&AtomicBool::new(false)), "the third job is next, not stuck behind the cancelled one");
drop(third);
assert_eq!(q.ahead(), 0);
}
#[test]
fn a_ticket_dropped_out_of_order_is_skipped_when_its_turn_comes() {
let q = leaked();
let (a, b, c) = (q.ticket(), q.ticket(), q.ticket());
drop(b);
assert_eq!(q.ahead(), 2);
drop(a);
assert!(c.wait(&AtomicBool::new(false)));
}
}
crates/engine/src/media_watch.rs
+13 -8
View File
@@ -6,8 +6,10 @@
//! Each file media item's file is compared by size and modification time with what it was when
//! last seen. The desktop app takes the stamps every *Refresh growing files every* seconds and as
//! soon as its window comes back to the front, on a worker thread so a file on a slow network share
//! never holds up the interface ([`Session::start_media_scan`], [`Session::poll_media_scan`]);
//! `media.refreshChanged` takes them on the spot (CLI, agents).
//! never holds up the interface ([`Session::start_media_scan`], [`Session::poll_media_scan`]); the
//! worker also lists the files, which in a project of thousands of clips is a few milliseconds the
//! interface would otherwise spend on it. `media.refreshChanged` takes them on the spot (CLI,
//! agents).
//!
//! Refreshing opens the file again: the item gets the file's new [`MediaInfo`] (a longer
//! duration, another frame size) and identity (so relinking compares against the new file), the
@@ -59,6 +61,12 @@ pub fn media_files(p: &Project, pool: &MediaPool) -> BTreeMap<String, String> {
.collect()
}
/// Whether `p` has a file media item to watch (made-offline ones aside): the interface's check on
/// every frame, which stops at the first one ([`media_files`] lists them all).
pub fn has_media_files(p: &Project) -> bool {
p.items.values().any(|i| matches!(&i.kind, ItemKind::Media(m) if !m.offline && matches!(m.media, MediaRef::File { .. })))
}
/// `media.refreshChanged`: stamp the project's media files now and read the changed ones again.
pub(crate) fn refresh_changed(s: &mut Session, _: &Value) -> Result<Value> {
let now = media_files(&s.project, &s.media).into_iter().map(|(p, file)| (p, stamp(&file))).collect();
@@ -69,17 +77,14 @@ impl Session {
/// Start stamping the project's media files on a worker thread, unless a look is still
/// running (or the build has no files to look at: the web app).
pub fn start_media_scan(&mut self) {
if self.media_scan.is_some() || cfg!(target_arch = "wasm32") {
return;
}
let files = media_files(&self.project, &self.media);
if files.is_empty() {
if self.media_scan.is_some() || cfg!(target_arch = "wasm32") || !has_media_files(&self.project) {
return;
}
let (project, pool) = (self.project.clone(), self.media.clone());
let slot: MediaScan = Arc::default();
let out = slot.clone();
let spawned = std::thread::Builder::new().name("media-watch".into()).spawn(move || {
let stamps = files.into_iter().map(|(p, file)| (p, stamp(&file))).collect();
let stamps = media_files(&project, &pool).into_iter().map(|(p, file)| (p, stamp(&file))).collect();
*out.lock().unwrap_or_else(PoisonError::into_inner) = Some(stamps);
});
if spawned.is_ok() {
crates/engine/src/media_watch_tests.rs
+15
View File
@@ -98,3 +98,18 @@ fn a_background_look_reads_changed_media_again() {
assert_eq!(look(&mut s), json!({"refreshed": [items[0].0]}));
assert!(s.media_scan.is_none());
}
/// The interface asks on every frame whether there is anything to watch: file media that is not
/// made offline. Generators are not files.
#[test]
fn only_file_media_is_watched() {
let mut s = Session::default();
s.execute("file.importDemoFootage", json!({"scene": "Aurora"})).unwrap();
assert!(!crate::media_watch::has_media_files(&s.project));
let a = tmp_dir("watch-has-files").join("a.mov");
make_movie(&a, DemoScene::Aurora, W, H, 12);
let item = s.execute("file.import", json!({"paths": [a.to_string_lossy()]})).unwrap()["items"][0].as_u64().unwrap();
assert!(crate::media_watch::has_media_files(&s.project));
s.execute("media.makeOffline", json!({"items": [item]})).unwrap();
assert!(!crate::media_watch::has_media_files(&s.project));
}
crates/engine/src/presets.rs
+3 -5
View File
@@ -9,7 +9,7 @@
//! - **Geometry.** Point parameters and mask paths are in clip pixels; they are scaled by the
//! target / source frame-size ratio so a preset made on 4K media lands in the same place on HD.
//! - **Intrinsic effects** (Motion, Opacity, Time Remapping) replace the target clip's own instance;
//! standard effects are added before the intrinsic ones (render order), like `effects.apply`.
//! standard effects are added at the end of the clip's effects, like `effects.apply`.
//! - **Library.** Built-in presets are authored in code ([`builtin_presets`]); user presets persist
//! in `<data dir>/effect-presets.json`. Preset files (export/import) are JSON:
//! `{"format": "filmcraft.effect-presets", "version": 1, "presets": [EffectPreset…]}`.
@@ -377,7 +377,7 @@ fn merge_keyframes(new: &mut EffectInstance, old: &EffectInstance) {
}
/// Put preset effects on a clip: intrinsic ones replace the clip's instance (keeping its keyframes
/// outside the span the preset animates), others are inserted before the intrinsic effects.
/// outside the span the preset animates), others are added at the end, like `effects.apply`.
pub fn put_effects(it: &mut TrackItem, effects: Vec<EffectInstance>) {
for mut e in effects {
let intrinsic = e.def().is_some_and(|d| d.intrinsic);
@@ -386,9 +386,7 @@ pub fn put_effects(it: &mut TrackItem, effects: Vec<EffectInstance>) {
*slot = e;
continue;
}
let audio = e.def().is_some_and(|d| d.kind == filmcraft_project::EffectKind::Audio);
let pos = if audio { it.effects.len() } else { it.effects.iter().position(|x| x.def().is_some_and(|d| d.intrinsic)).unwrap_or(it.effects.len()) };
it.effects.insert(pos, e);
it.effects.push(e);
}
}
crates/engine/src/presets_tests.rs
+2 -2
View File
@@ -57,8 +57,8 @@ fn keyframe_modes_retime_onto_the_target() {
let it = item(&s, b);
assert_eq!(it.effects.len(), before + 1);
let e = it.effects.iter().rfind(|e| e.effect == "gaussian_blur").unwrap();
// standard effects go before the intrinsic ones
assert!(it.effects.iter().position(|x| x.effect == "gaussian_blur").unwrap() < it.effects.iter().position(|x| x.effect == "motion").unwrap());
// standard effects go after the intrinsic ones, like effects.apply (#1879)
assert!(it.effects.iter().position(|x| x.effect == "gaussian_blur").unwrap() > it.effects.iter().position(|x| x.effect == "motion").unwrap());
let p = e.param("blurriness").unwrap();
assert_eq!(
p.keyframes.len(),

Some files were not shown because too many files have changed in this diff Show More