storytold/cloud-worker
2
0
Fork 0
mirror of https://github.com/storytold/cloud-worker.git synced 2026-10-09 00:09:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

reliability: failure-stats + output-validation tooling; persistent output dir

Browse Source 
- bench/failures.py: per-GPU success rates and error-class breakdown by config
- bench/validate_outputs.py: ffprobe every recorded output via /view
- finding: 24GB cards fail on multi-ref (TE vision OOM), not duration/size;
  100% of successful runs produce valid videos
- outputs now written to /workspace (survives pod resets)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Hanashi
2026-08-06 14:41:00 -04:00
parent fbc68bb749
commit a293d0a2c4
13 changed files with 308 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
bench/results.jsonl
+13
View File
@@ -81,3 +81,16 @@
{"label": "b16_refheavy_4ref_max", "task": "ref2v", "model_file": "minimax_h3_ref2va_bf16.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 3, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "max", "ref_count": 4, "warm": true, "filename_prefix": "bench/b16_refheavy_4ref_max", "ok": true, "status": "success", "wall_s": 91.7, "exec_s": 90.8, "peak_vram_gb": 155.5, "outputs": ["b16_refheavy_4ref_max_00001_.mp4"], "error": null, "ts": "2026-08-06T09:14:11Z"}
{"label": "b16_refheavy_8ref_max", "task": "ref2v", "model_file": "minimax_h3_ref2va_bf16.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 3, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "max", "ref_count": 8, "warm": true, "filename_prefix": "bench/b16_refheavy_8ref_max", "ok": true, "status": "success", "wall_s": 785.5, "exec_s": 783.9, "peak_vram_gb": 163.9, "outputs": ["b16_refheavy_8ref_max_00001_.mp4"], "error": null, "ts": "2026-08-06T09:27:16Z"}
{"label": "b16_refheavy_8ref_match_1344", "task": "ref2v", "model_file": "minimax_h3_ref2va_bf16.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 1344, "height": 768, "seconds": 5, "length": 124, "steps": 20, "seed": 3, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 8, "warm": true, "filename_prefix": "bench/b16_refheavy_8ref_match_1344", "ok": true, "status": "success", "wall_s": 282.9, "exec_s": 282.4, "peak_vram_gb": 157.8, "outputs": ["b16_refheavy_8ref_match_1344_00001_.mp4"], "error": null, "ts": "2026-08-06T09:31:59Z"}
{"label": "pi8v2_key_ref2v_1ref_r0", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "Use <Picture 1> as the setting. A slow cinematic pan across the sunset city skyline, clouds drifting, lights turning on in the towers, ambient city sounds.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 10, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/pi8v2_key_ref2v_1ref_r0", "ok": true, "status": "success", "wall_s": 139.6, "exec_s": 137.8, "peak_vram_gb": 175.1, "outputs": ["pi8v2_key_ref2v_1ref_r0_00001_.mp4"], "error": null, "ts": "2026-08-06T17:52:25Z"}
{"label": "pi8v2_key_ref2v_1ref_r1", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "Use <Picture 1> as the setting. A slow cinematic pan across the sunset city skyline, clouds drifting, lights turning on in the towers, ambient city sounds.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 11, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/pi8v2_key_ref2v_1ref_r1", "ok": true, "status": "success", "wall_s": 66.8, "exec_s": 66.4, "peak_vram_gb": 174.9, "outputs": ["pi8v2_key_ref2v_1ref_r1_00001_.mp4"], "error": null, "ts": "2026-08-06T17:53:32Z"}
{"label": "pi8v2_key_ref2v_1ref_r2", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "Use <Picture 1> as the setting. A slow cinematic pan across the sunset city skyline, clouds drifting, lights turning on in the towers, ambient city sounds.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 12, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/pi8v2_key_ref2v_1ref_r2", "ok": true, "status": "success", "wall_s": 66.6, "exec_s": 65.6, "peak_vram_gb": 175.4, "outputs": ["pi8v2_key_ref2v_1ref_r2_00001_.mp4"], "error": null, "ts": "2026-08-06T17:54:39Z"}
{"label": "pi8v2_key_ref2v_8ref_r0", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 10, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 8, "warm": true, "filename_prefix": "bench/pi8v2_key_ref2v_8ref_r0", "ok": true, "status": "success", "wall_s": 95.7, "exec_s": 95.2, "peak_vram_gb": 173.6, "outputs": ["pi8v2_key_ref2v_8ref_r0_00001_.mp4"], "error": null, "ts": "2026-08-06T17:56:15Z"}
{"label": "pi8v2_key_ref2v_8ref_r1", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 11, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 8, "warm": true, "filename_prefix": "bench/pi8v2_key_ref2v_8ref_r1", "ok": true, "status": "success", "wall_s": 95.8, "exec_s": 95.0, "peak_vram_gb": 175.7, "outputs": ["pi8v2_key_ref2v_8ref_r1_00001_.mp4"], "error": null, "ts": "2026-08-06T17:57:51Z"}
{"label": "pi8v2_key_ref2v_8ref_r2", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 12, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 8, "warm": true, "filename_prefix": "bench/pi8v2_key_ref2v_8ref_r2", "ok": true, "status": "success", "wall_s": 95.9, "exec_s": 94.0, "peak_vram_gb": 176.7, "outputs": ["pi8v2_key_ref2v_8ref_r2_00001_.mp4"], "error": null, "ts": "2026-08-06T17:59:27Z"}
{"label": "pi8v2_key_ref2v_8ref_max", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 20, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "max", "ref_count": 8, "warm": true, "filename_prefix": "bench/pi8v2_key_ref2v_8ref_max", "ok": true, "status": "success", "wall_s": 841.8, "exec_s": 840.9, "peak_vram_gb": 172.0, "outputs": ["pi8v2_key_ref2v_8ref_max_00001_.mp4"], "error": null, "ts": "2026-08-06T18:13:28Z"}
{"label": "conc_warm_A", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 30, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": false, "filename_prefix": "bench/conc_warm_A", "ok": true, "status": "success", "wall_s": 237.7, "exec_s": 236.4, "peak_vram_gb": 43.2, "outputs": ["conc_warm_A_00001_.mp4"], "error": null, "ts": "2026-08-06T18:19:11Z"}
{"label": "conc_warm_B", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 30, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": false, "filename_prefix": "bench/conc_warm_B", "ok": true, "status": "success", "wall_s": 179.2, "exec_s": 177.9, "peak_vram_gb": 84.6, "outputs": ["conc_warm_B_00001_.mp4"], "error": null, "ts": "2026-08-06T18:22:10Z"}
{"label": "conc_seq_1", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 31, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/conc_seq_1", "ok": true, "status": "success", "wall_s": 64.7, "exec_s": 62.6, "peak_vram_gb": 88.1, "outputs": ["conc_seq_1_00001_.mp4"], "error": null, "ts": "2026-08-06T18:23:15Z"}
{"label": "conc_seq_2", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 32, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/conc_seq_2", "ok": true, "status": "success", "wall_s": 62.9, "exec_s": 62.5, "peak_vram_gb": 88.2, "outputs": ["conc_seq_2_00001_.mp4"], "error": null, "ts": "2026-08-06T18:24:18Z"}
{"label": "conc_par_A", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 33, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/conc_par_A", "ok": true, "status": "success", "wall_s": 142.7, "exec_s": 142.4, "peak_vram_gb": 93.6, "outputs": ["conc_par_A_00001_.mp4"], "error": null, "ts": "2026-08-06T18:26:41Z"}
{"label": "conc_par_B", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 34, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/conc_par_B", "ok": true, "status": "success", "wall_s": 144.7, "exec_s": 142.9, "peak_vram_gb": 93.3, "outputs": ["conc_par_B_00001_.mp4"], "error": null, "ts": "2026-08-06T18:26:43Z"}