reliability: failure-stats + output-validation tooling; persistent output dir

- bench/failures.py: per-GPU success rates and error-class breakdown by config - bench/validate_outputs.py: ffprobe every recorded output via /view - finding: 24GB cards fail on multi-ref (TE vision OOM), not duration/size; 100% of successful runs produce valid videos - outputs now written to /workspace (survives pod resets) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-10-09 00:09:43 +00:00 · 2026-08-06 14:41:00 -04:00
parent fbc68bb749
commit a293d0a2c4
13 changed files with 308 additions and 2 deletions
@@ -1,3 +1,8 @@
 ts,label,task,model_file,width,height,seconds,length,steps,sampler,scheduler,ref_image_size,ref_count,warm,ok,wall_s,exec_s,peak_vram_gb,outputs,error
 2026-08-06T07:51:24Z,g4000_smoke_t2v,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,608,352,5.0,124,20,res_multistep,simple,match,1,False,False,135.0,,15.0,,server unreachable while polling: [Errno 104] Connection reset by peer
 2026-08-06T07:58:16Z,g4000_smoke_t2v_b,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,608,352,5.0,124,20,res_multistep,simple,match,1,False,True,343.4,341.0,22.7,g4000_smoke_t2v_b_00001_.mp4,
 2026-08-06T17:58:17Z,g4000_pi8_key_t2v_5s_r0,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,1,False,True,559.7,557.0,20.4,g4000_pi8_key_t2v_5s_r0_00001_.mp4,
 2026-08-06T18:07:31Z,g4000_pi8_key_t2v_5s_r1,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,1,True,True,553.4,551.4,20.4,g4000_pi8_key_t2v_5s_r1_00001_.mp4,
 2026-08-06T18:16:44Z,g4000_pi8_key_t2v_5s_r2,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,1,True,True,553.4,550.6,20.4,g4000_pi8_key_t2v_5s_r2_00001_.mp4,
 2026-08-06T18:26:34Z,g4000_pi8_key_ref2v_1ref_r0,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,1,False,True,589.8,587.8,19.0,g4000_pi8_key_ref2v_1ref_r0_00001_.mp4,
 2026-08-06T18:36:16Z,g4000_pi8_key_ref2v_1ref_r1,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,1,True,True,582.4,580.1,19.0,g4000_pi8_key_ref2v_1ref_r1_00001_.mp4,
@@ -1,2 +1,7 @@
 {"label": "g4000_smoke_t2v", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 608, "height": 352, "seconds": 5.0, "length": 124, "steps": 20, "seed": 7, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": false, "filename_prefix": "bench/g4000_smoke_t2v", "ok": false, "status": "server_lost", "wall_s": 135.0, "exec_s": null, "peak_vram_gb": 15.0, "outputs": [], "error": "server unreachable while polling: [Errno 104] Connection reset by peer", "ts": "2026-08-06T07:51:24Z"}
 {"label": "g4000_smoke_t2v_b", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 608, "height": 352, "seconds": 5.0, "length": 124, "steps": 20, "seed": 9, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": false, "filename_prefix": "bench/g4000_smoke_t2v_b", "ok": true, "status": "success", "wall_s": 343.4, "exec_s": 341.0, "peak_vram_gb": 22.7, "outputs": ["g4000_smoke_t2v_b_00001_.mp4"], "error": null, "ts": "2026-08-06T07:58:16Z"}
 {"label": "g4000_pi8_key_t2v_5s_r0", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 10, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": false, "filename_prefix": "bench/g4000_pi8_key_t2v_5s_r0", "ok": true, "status": "success", "wall_s": 559.7, "exec_s": 557.0, "peak_vram_gb": 20.4, "outputs": ["g4000_pi8_key_t2v_5s_r0_00001_.mp4"], "error": null, "ts": "2026-08-06T17:58:17Z"}
 {"label": "g4000_pi8_key_t2v_5s_r1", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 11, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/g4000_pi8_key_t2v_5s_r1", "ok": true, "status": "success", "wall_s": 553.4, "exec_s": 551.4, "peak_vram_gb": 20.4, "outputs": ["g4000_pi8_key_t2v_5s_r1_00001_.mp4"], "error": null, "ts": "2026-08-06T18:07:31Z"}
 {"label": "g4000_pi8_key_t2v_5s_r2", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 12, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/g4000_pi8_key_t2v_5s_r2", "ok": true, "status": "success", "wall_s": 553.4, "exec_s": 550.6, "peak_vram_gb": 20.4, "outputs": ["g4000_pi8_key_t2v_5s_r2_00001_.mp4"], "error": null, "ts": "2026-08-06T18:16:44Z"}
 {"label": "g4000_pi8_key_ref2v_1ref_r0", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "Use <Picture 1> as the setting. A slow cinematic pan across the sunset city skyline, clouds drifting, lights turning on in the towers, ambient city sounds.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 10, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": false, "filename_prefix": "bench/g4000_pi8_key_ref2v_1ref_r0", "ok": true, "status": "success", "wall_s": 589.8, "exec_s": 587.8, "peak_vram_gb": 19.0, "outputs": ["g4000_pi8_key_ref2v_1ref_r0_00001_.mp4"], "error": null, "ts": "2026-08-06T18:26:34Z"}
 {"label": "g4000_pi8_key_ref2v_1ref_r1", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "Use <Picture 1> as the setting. A slow cinematic pan across the sunset city skyline, clouds drifting, lights turning on in the towers, ambient city sounds.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 11, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/g4000_pi8_key_ref2v_1ref_r1", "ok": true, "status": "success", "wall_s": 582.4, "exec_s": 580.1, "peak_vram_gb": 19.0, "outputs": ["g4000_pi8_key_ref2v_1ref_r1_00001_.mp4"], "error": null, "ts": "2026-08-06T18:36:16Z"}
@@ -0,0 +1,6 @@
 {"label": "g4000_smoke_t2v_b", "file": "g4000_smoke_t2v_b_00001_.mp4", "valid": true, "bytes": 619210, "video_codec": "h264", "audio_codec": "aac", "width": 608, "height": 352, "duration": 5.17, "expected_duration": 5.17}
 {"label": "g4000_pi8_key_t2v_5s_r0", "file": "g4000_pi8_key_t2v_5s_r0_00001_.mp4", "valid": true, "bytes": 898728, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "g4000_pi8_key_t2v_5s_r1", "file": "g4000_pi8_key_t2v_5s_r1_00001_.mp4", "valid": true, "bytes": 861789, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "g4000_pi8_key_t2v_5s_r2", "file": "g4000_pi8_key_t2v_5s_r2_00001_.mp4", "valid": true, "bytes": 1325124, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "g4000_pi8_key_ref2v_1ref_r0", "file": "g4000_pi8_key_ref2v_1ref_r0_00001_.mp4", "valid": true, "bytes": 985435, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "g4000_pi8_key_ref2v_1ref_r1", "file": "g4000_pi8_key_ref2v_1ref_r1_00001_.mp4", "valid": true, "bytes": 974798, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
@@ -26,3 +26,19 @@ ts,label,task,model_file,width,height,seconds,length,steps,sampler,scheduler,ref
 2026-08-06T07:30:59Z,g4090_pi8_refheavy_8ref_match,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,8,True,False,20.8,18.7,21.8,,"{""status_str"": ""error"", ""completed"": false, ""messages"": [[""execution_start"", {""prompt_id"": ""91e186ad-65df-47f6-be72-9501dfc1e9f1"", ""timestamp"": 1786001438642}], [""execution_cached"", {""nodes"": [""1"", ""2"", ""3"", ""4"", ""6"", ""7"", ""8"", ""9"", ""20"", ""21"", ""22"", ""23"", ""24"", ""25"", ""26"", ""27"", ""5""], ""prompt_id"": ""91e186ad-65df-47f6-be72-9501dfc1e9f1"", ""timestamp"": 1786001438852}], [""execution_error"", {""prompt_id"": ""91e186ad-65df-47f6-be72-9501dfc1e9f1"", ""node_id"": ""11"", ""node_type"": ""SamplerCustomAdvanced"", ""executed"": [], ""exception_message"": ""Allocation on device \nThis error means you ran out of memory on your GPU.\n\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number."", ""exception_type"": ""torch.OutOfMemoryError"", ""traceback"": [""  File \""/ComfyUI/execution.py\"", line 545, in execute\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 344, in get_output_data\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 318, in _async_map_node_over_list\n    await process_inputs(input_dict, i)\n"", ""  File \""/ComfyUI/execution.py\"", line 306, in process_inputs\n    result = f(**inputs)\n             ^^^^^"
 2026-08-06T07:31:17Z,g4090_pi8_refheavy_4ref_max,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,max,4,True,False,18.7,18.0,22.4,,"{""status_str"": ""error"", ""completed"": false, ""messages"": [[""execution_start"", {""prompt_id"": ""c79b27c2-09cc-4ce1-b078-6a5234b74d88"", ""timestamp"": 1786001459467}], [""execution_cached"", {""nodes"": [""1"", ""2"", ""3"", ""4"", ""6"", ""7"", ""8"", ""20"", ""21"", ""22"", ""23""], ""prompt_id"": ""c79b27c2-09cc-4ce1-b078-6a5234b74d88"", ""timestamp"": 1786001459474}], [""execution_error"", {""prompt_id"": ""c79b27c2-09cc-4ce1-b078-6a5234b74d88"", ""node_id"": ""5"", ""node_type"": ""MiniMaxH3ReferenceToVideo"", ""executed"": [], ""exception_message"": ""Allocation on device \nThis error means you ran out of memory on your GPU.\n\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number."", ""exception_type"": ""torch.OutOfMemoryError"", ""traceback"": [""  File \""/ComfyUI/execution.py\"", line 545, in execute\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 344, in get_output_data\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 318, in _async_map_node_over_list\n    await process_inputs(input_dict, i)\n"", ""  File \""/ComfyUI/execution.py\"", line 306, in process_inputs\n    result = f(**inputs)\n             ^^^^^^^^^^^\n"", ""  File \""/ComfyUI/c"
 2026-08-06T07:40:29Z,g4090_oomcheck_t2v_864,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,864,480,5.0,124,20,res_multistep,simple,match,1,True,True,393.1,390.7,21.4,g4090_oomcheck_t2v_864_00001_.mp4,
 2026-08-06T17:52:41Z,g4090_pi8_key_t2v_5s_r0,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,1,False,True,224.3,222.5,21.1,g4090_pi8_key_t2v_5s_r0_00001_.mp4,
 2026-08-06T17:56:26Z,g4090_pi8_key_t2v_5s_r1,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,1,True,True,225.0,222.4,22.2,g4090_pi8_key_t2v_5s_r1_00001_.mp4,
 2026-08-06T18:00:10Z,g4090_pi8_key_t2v_5s_r2,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,1,True,True,224.0,222.4,21.1,g4090_pi8_key_t2v_5s_r2_00001_.mp4,
 2026-08-06T18:05:49Z,g4090_pi8_key_ref2v_1ref_r0,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,1,False,True,339.8,338.9,21.1,g4090_pi8_key_ref2v_1ref_r0_00001_.mp4,
 2026-08-06T18:09:51Z,g4090_pi8_key_ref2v_1ref_r1,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,1,True,True,241.2,235.4,21.2,g4090_pi8_key_ref2v_1ref_r1_00001_.mp4,
 2026-08-06T18:10:17Z,g4090_pi8_key_ref2v_1ref_r2,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,1,True,False,26.2,24.5,20.8,,"{""status_str"": ""error"", ""completed"": false, ""messages"": [[""execution_start"", {""prompt_id"": ""d1f56b69-02e0-473a-b974-9d0ffd52a1ee"", ""timestamp"": 1786039791534}], [""execution_cached"", {""nodes"": [""1"", ""2"", ""3"", ""4"", ""6"", ""7"", ""8"", ""9"", ""10"", ""5""], ""prompt_id"": ""d1f56b69-02e0-473a-b974-9d0ffd52a1ee"", ""timestamp"": 1786039791537}], [""execution_error"", {""prompt_id"": ""d1f56b69-02e0-473a-b974-9d0ffd52a1ee"", ""node_id"": ""11"", ""node_type"": ""SamplerCustomAdvanced"", ""executed"": [], ""exception_message"": ""Allocation on device \nThis error means you ran out of memory on your GPU.\n\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number."", ""exception_type"": ""torch.OutOfMemoryError"", ""traceback"": [""  File \""/ComfyUI/execution.py\"", line 545, in execute\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 344, in get_output_data\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 318, in _async_map_node_over_list\n    await process_inputs(input_dict, i)\n"", ""  File \""/ComfyUI/execution.py\"", line 306, in process_inputs\n    result = f(**inputs)\n             ^^^^^^^^^^^\n"", ""  File \""/ComfyUI/comfy_api/in"
 2026-08-06T18:14:23Z,g4090_pi8_key_ref2v_8ref_r0,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,8,True,False,246.0,244.5,21.0,,"{""status_str"": ""error"", ""completed"": false, ""messages"": [[""execution_start"", {""prompt_id"": ""3b5e5c37-0f7b-4458-b832-38ecfcc4c9eb"", ""timestamp"": 1786039817789}], [""execution_cached"", {""nodes"": [""1"", ""2"", ""3"", ""4"", ""6"", ""7"", ""8"", ""20""], ""prompt_id"": ""3b5e5c37-0f7b-4458-b832-38ecfcc4c9eb"", ""timestamp"": 1786039818251}], [""execution_error"", {""prompt_id"": ""3b5e5c37-0f7b-4458-b832-38ecfcc4c9eb"", ""node_id"": ""11"", ""node_type"": ""SamplerCustomAdvanced"", ""executed"": [""21"", ""27"", ""22"", ""9"", ""24"", ""26"", ""5"", ""25"", ""23""], ""exception_message"": ""Allocation on device \nThis error means you ran out of memory on your GPU.\n\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number."", ""exception_type"": ""torch.OutOfMemoryError"", ""traceback"": [""  File \""/ComfyUI/execution.py\"", line 545, in execute\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 344, in get_output_data\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 318, in _async_map_node_over_list\n    await process_inputs(input_dict, i)\n"", ""  File \""/ComfyUI/execution.py\"", line 306, in process_inputs\n    result = f(**inputs)\n             ^^^^^^^"
 2026-08-06T18:14:42Z,g4090_pi8_key_ref2v_8ref_r1,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,8,True,False,18.7,17.1,18.6,,"{""status_str"": ""error"", ""completed"": false, ""messages"": [[""execution_start"", {""prompt_id"": ""ab3f9b09-ddb7-41e7-8c39-27ae36b468ec"", ""timestamp"": 1786040063740}], [""execution_cached"", {""nodes"": [""1"", ""2"", ""3"", ""4"", ""6"", ""7"", ""8"", ""9"", ""20"", ""21"", ""22"", ""23"", ""24"", ""25"", ""26"", ""27"", ""5""], ""prompt_id"": ""ab3f9b09-ddb7-41e7-8c39-27ae36b468ec"", ""timestamp"": 1786040063964}], [""execution_error"", {""prompt_id"": ""ab3f9b09-ddb7-41e7-8c39-27ae36b468ec"", ""node_id"": ""11"", ""node_type"": ""SamplerCustomAdvanced"", ""executed"": [], ""exception_message"": ""Allocation on device \nThis error means you ran out of memory on your GPU.\n\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number."", ""exception_type"": ""torch.OutOfMemoryError"", ""traceback"": [""  File \""/ComfyUI/execution.py\"", line 545, in execute\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 344, in get_output_data\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 318, in _async_map_node_over_list\n    await process_inputs(input_dict, i)\n"", ""  File \""/ComfyUI/execution.py\"", line 306, in process_inputs\n    result = f(**inputs)\n             ^^^^^"
 2026-08-06T18:15:00Z,g4090_pi8_key_ref2v_8ref_r2,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,8,True,False,18.5,17.0,18.6,,"{""status_str"": ""error"", ""completed"": false, ""messages"": [[""execution_start"", {""prompt_id"": ""4742272b-2d38-4dc9-a8e2-09d8dc6c8edc"", ""timestamp"": 1786040082480}], [""execution_cached"", {""nodes"": [""1"", ""2"", ""3"", ""4"", ""6"", ""7"", ""8"", ""9"", ""20"", ""21"", ""22"", ""23"", ""24"", ""25"", ""26"", ""27"", ""5""], ""prompt_id"": ""4742272b-2d38-4dc9-a8e2-09d8dc6c8edc"", ""timestamp"": 1786040082696}], [""execution_error"", {""prompt_id"": ""4742272b-2d38-4dc9-a8e2-09d8dc6c8edc"", ""node_id"": ""11"", ""node_type"": ""SamplerCustomAdvanced"", ""executed"": [], ""exception_message"": ""Allocation on device \nThis error means you ran out of memory on your GPU.\n\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number."", ""exception_type"": ""torch.OutOfMemoryError"", ""traceback"": [""  File \""/ComfyUI/execution.py\"", line 545, in execute\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 344, in get_output_data\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 318, in _async_map_node_over_list\n    await process_inputs(input_dict, i)\n"", ""  File \""/ComfyUI/execution.py\"", line 306, in process_inputs\n    result = f(**inputs)\n             ^^^^^"
 2026-08-06T18:17:17Z,g4090_pi8_key_t2v_15s,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,864,480,15,362,20,res_multistep,simple,match,1,True,False,136.5,,16.8,,server unreachable while polling: [Errno 104] Connection reset by peer
 2026-08-06T18:25:24Z,g4090b_pi8_refheavy_1ref_match,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,1,True,True,396.2,391.9,21.9,g4090b_pi8_refheavy_1ref_match_00001_.mp4,
 2026-08-06T18:28:34Z,g4090b_pi8_refheavy_4ref_match,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,4,True,False,189.6,187.3,22.9,,"{""status_str"": ""error"", ""completed"": false, ""messages"": [[""execution_start"", {""prompt_id"": ""db8eaf8e-a68e-40f5-ad1d-be3496225da9"", ""timestamp"": 1786040725094}], [""execution_cached"", {""nodes"": [], ""prompt_id"": ""db8eaf8e-a68e-40f5-ad1d-be3496225da9"", ""timestamp"": 1786040725094}], [""execution_error"", {""prompt_id"": ""db8eaf8e-a68e-40f5-ad1d-be3496225da9"", ""node_id"": ""11"", ""node_type"": ""SamplerCustomAdvanced"", ""executed"": [""7"", ""9"", ""23"", ""3"", ""6"", ""22"", ""2"", ""21"", ""5"", ""1"", ""8"", ""20"", ""4""], ""exception_message"": ""Allocation on device \nThis error means you ran out of memory on your GPU.\n\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number."", ""exception_type"": ""torch.OutOfMemoryError"", ""traceback"": [""  File \""/ComfyUI/execution.py\"", line 545, in execute\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 344, in get_output_data\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 318, in _async_map_node_over_list\n    await process_inputs(input_dict, i)\n"", ""  File \""/ComfyUI/execution.py\"", line 306, in process_inputs\n    result = f(**inputs)\n             ^^^^^^^^^^^\n"", ""  File \""/Co"
 2026-08-06T18:31:48Z,g4090b_pi8_refheavy_8ref_match,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,match,8,True,False,193.8,189.7,21.1,,"{""status_str"": ""error"", ""completed"": false, ""messages"": [[""execution_start"", {""prompt_id"": ""71dbdb62-fc28-41cd-be08-d92700592811"", ""timestamp"": 1786040915845}], [""execution_cached"", {""nodes"": [], ""prompt_id"": ""71dbdb62-fc28-41cd-be08-d92700592811"", ""timestamp"": 1786040915845}], [""execution_error"", {""prompt_id"": ""71dbdb62-fc28-41cd-be08-d92700592811"", ""node_id"": ""11"", ""node_type"": ""SamplerCustomAdvanced"", ""executed"": [""7"", ""24"", ""23"", ""3"", ""6"", ""9"", ""22"", ""26"", ""21"", ""2"", ""5"", ""1"", ""8"", ""25"", ""20"", ""27"", ""4""], ""exception_message"": ""Allocation on device \nThis error means you ran out of memory on your GPU.\n\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number."", ""exception_type"": ""torch.OutOfMemoryError"", ""traceback"": [""  File \""/ComfyUI/execution.py\"", line 545, in execute\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 344, in get_output_data\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 318, in _async_map_node_over_list\n    await process_inputs(input_dict, i)\n"", ""  File \""/ComfyUI/execution.py\"", line 306, in process_inputs\n    result = f(**inputs)\n             ^^^^^"
 2026-08-06T18:35:01Z,g4090b_pi8_refheavy_4ref_max,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,max,4,True,False,193.0,190.2,22.4,,"{""status_str"": ""error"", ""completed"": false, ""messages"": [[""execution_start"", {""prompt_id"": ""db8d48a6-7ce1-441e-b99b-28fac3921c14"", ""timestamp"": 1786041108847}], [""execution_cached"", {""nodes"": [], ""prompt_id"": ""db8d48a6-7ce1-441e-b99b-28fac3921c14"", ""timestamp"": 1786041108847}], [""execution_error"", {""prompt_id"": ""db8d48a6-7ce1-441e-b99b-28fac3921c14"", ""node_id"": ""11"", ""node_type"": ""SamplerCustomAdvanced"", ""executed"": [""7"", ""9"", ""23"", ""3"", ""6"", ""22"", ""2"", ""21"", ""5"", ""1"", ""8"", ""20"", ""4""], ""exception_message"": ""Allocation on device \nThis error means you ran out of memory on your GPU.\n\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number."", ""exception_type"": ""torch.OutOfMemoryError"", ""traceback"": [""  File \""/ComfyUI/execution.py\"", line 545, in execute\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 344, in get_output_data\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 318, in _async_map_node_over_list\n    await process_inputs(input_dict, i)\n"", ""  File \""/ComfyUI/execution.py\"", line 306, in process_inputs\n    result = f(**inputs)\n             ^^^^^^^^^^^\n"", ""  File \""/Co"
 2026-08-06T18:37:33Z,g4090b_pi8_refheavy_8ref_max,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5,124,20,res_multistep,simple,max,8,True,False,152.0,148.9,23.1,,"{""status_str"": ""error"", ""completed"": false, ""messages"": [[""execution_start"", {""prompt_id"": ""c0da6de3-a983-435a-b0fa-db9f6e95bd1e"", ""timestamp"": 1786041302473}], [""execution_cached"", {""nodes"": [], ""prompt_id"": ""c0da6de3-a983-435a-b0fa-db9f6e95bd1e"", ""timestamp"": 1786041302473}], [""execution_error"", {""prompt_id"": ""c0da6de3-a983-435a-b0fa-db9f6e95bd1e"", ""node_id"": ""5"", ""node_type"": ""MiniMaxH3ReferenceToVideo"", ""executed"": [""24"", ""23"", ""3"", ""22"", ""26"", ""21"", ""2"", ""25"", ""20"", ""27"", ""4""], ""exception_message"": ""Allocation on device \nThis error means you ran out of memory on your GPU.\n\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number."", ""exception_type"": ""torch.OutOfMemoryError"", ""traceback"": [""  File \""/ComfyUI/execution.py\"", line 545, in execute\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 344, in get_output_data\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 318, in _async_map_node_over_list\n    await process_inputs(input_dict, i)\n"", ""  File \""/ComfyUI/execution.py\"", line 306, in process_inputs\n    result = f(**inputs)\n             ^^^^^^^^^^^\n"", ""  File \""/Comfy"
 2026-08-06T18:39:35Z,g4090b_pi8_refheavy_8ref_match_1344,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,1344,768,5,124,20,res_multistep,simple,match,8,True,False,122.4,120.8,20.4,,"{""status_str"": ""error"", ""completed"": false, ""messages"": [[""execution_start"", {""prompt_id"": ""3cca3ccb-12d6-422f-8ad6-3f7b58e731df"", ""timestamp"": 1786041453629}], [""execution_cached"", {""nodes"": [], ""prompt_id"": ""3cca3ccb-12d6-422f-8ad6-3f7b58e731df"", ""timestamp"": 1786041453629}], [""execution_error"", {""prompt_id"": ""3cca3ccb-12d6-422f-8ad6-3f7b58e731df"", ""node_id"": ""5"", ""node_type"": ""MiniMaxH3ReferenceToVideo"", ""executed"": [""24"", ""23"", ""3"", ""22"", ""26"", ""21"", ""2"", ""25"", ""20"", ""27"", ""4""], ""exception_message"": ""Allocation on device \nThis error means you ran out of memory on your GPU.\n\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number."", ""exception_type"": ""torch.OutOfMemoryError"", ""traceback"": [""  File \""/ComfyUI/execution.py\"", line 545, in execute\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 344, in get_output_data\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n"", ""  File \""/ComfyUI/execution.py\"", line 318, in _async_map_node_over_list\n    await process_inputs(input_dict, i)\n"", ""  File \""/ComfyUI/execution.py\"", line 306, in process_inputs\n    result = f(**inputs)\n             ^^^^^^^^^^^\n"", ""  File \""/Comfy"
@@ -25,3 +25,19 @@
 {"label": "g4090_pi8_refheavy_8ref_match", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 3, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 8, "warm": true, "filename_prefix": "bench/g4090_pi8_refheavy_8ref_match", "ok": false, "status": "error", "wall_s": 20.8, "exec_s": 18.7, "peak_vram_gb": 21.8, "outputs": [], "error": "{\"status_str\": \"error\", \"completed\": false, \"messages\": [[\"execution_start\", {\"prompt_id\": \"91e186ad-65df-47f6-be72-9501dfc1e9f1\", \"timestamp\": 1786001438642}], [\"execution_cached\", {\"nodes\": [\"1\", \"2\", \"3\", \"4\", \"6\", \"7\", \"8\", \"9\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"5\"], \"prompt_id\": \"91e186ad-65df-47f6-be72-9501dfc1e9f1\", \"timestamp\": 1786001438852}], [\"execution_error\", {\"prompt_id\": \"91e186ad-65df-47f6-be72-9501dfc1e9f1\", \"node_id\": \"11\", \"node_type\": \"SamplerCustomAdvanced\", \"executed\": [], \"exception_message\": \"Allocation on device \\nThis error means you ran out of memory on your GPU.\\n\\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number.\", \"exception_type\": \"torch.OutOfMemoryError\", \"traceback\": [\"  File \\\"/ComfyUI/execution.py\\\", line 545, in execute\\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 344, in get_output_data\\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 318, in _async_map_node_over_list\\n    await process_inputs(input_dict, i)\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 306, in process_inputs\\n    result = f(**inputs)\\n             ^^^^^", "ts": "2026-08-06T07:30:59Z"}
 {"label": "g4090_pi8_refheavy_4ref_max", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 3, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "max", "ref_count": 4, "warm": true, "filename_prefix": "bench/g4090_pi8_refheavy_4ref_max", "ok": false, "status": "error", "wall_s": 18.7, "exec_s": 18.0, "peak_vram_gb": 22.4, "outputs": [], "error": "{\"status_str\": \"error\", \"completed\": false, \"messages\": [[\"execution_start\", {\"prompt_id\": \"c79b27c2-09cc-4ce1-b078-6a5234b74d88\", \"timestamp\": 1786001459467}], [\"execution_cached\", {\"nodes\": [\"1\", \"2\", \"3\", \"4\", \"6\", \"7\", \"8\", \"20\", \"21\", \"22\", \"23\"], \"prompt_id\": \"c79b27c2-09cc-4ce1-b078-6a5234b74d88\", \"timestamp\": 1786001459474}], [\"execution_error\", {\"prompt_id\": \"c79b27c2-09cc-4ce1-b078-6a5234b74d88\", \"node_id\": \"5\", \"node_type\": \"MiniMaxH3ReferenceToVideo\", \"executed\": [], \"exception_message\": \"Allocation on device \\nThis error means you ran out of memory on your GPU.\\n\\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number.\", \"exception_type\": \"torch.OutOfMemoryError\", \"traceback\": [\"  File \\\"/ComfyUI/execution.py\\\", line 545, in execute\\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 344, in get_output_data\\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 318, in _async_map_node_over_list\\n    await process_inputs(input_dict, i)\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 306, in process_inputs\\n    result = f(**inputs)\\n             ^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/c", "ts": "2026-08-06T07:31:17Z"}
 {"label": "g4090_oomcheck_t2v_864", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 8, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/g4090_oomcheck_t2v_864", "ok": true, "status": "success", "wall_s": 393.1, "exec_s": 390.7, "peak_vram_gb": 21.4, "outputs": ["g4090_oomcheck_t2v_864_00001_.mp4"], "error": null, "ts": "2026-08-06T07:40:29Z"}
 {"label": "g4090_pi8_key_t2v_5s_r0", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 10, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": false, "filename_prefix": "bench/g4090_pi8_key_t2v_5s_r0", "ok": true, "status": "success", "wall_s": 224.3, "exec_s": 222.5, "peak_vram_gb": 21.1, "outputs": ["g4090_pi8_key_t2v_5s_r0_00001_.mp4"], "error": null, "ts": "2026-08-06T17:52:41Z"}
 {"label": "g4090_pi8_key_t2v_5s_r1", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 11, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/g4090_pi8_key_t2v_5s_r1", "ok": true, "status": "success", "wall_s": 225.0, "exec_s": 222.4, "peak_vram_gb": 22.2, "outputs": ["g4090_pi8_key_t2v_5s_r1_00001_.mp4"], "error": null, "ts": "2026-08-06T17:56:26Z"}
 {"label": "g4090_pi8_key_t2v_5s_r2", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 12, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/g4090_pi8_key_t2v_5s_r2", "ok": true, "status": "success", "wall_s": 224.0, "exec_s": 222.4, "peak_vram_gb": 21.1, "outputs": ["g4090_pi8_key_t2v_5s_r2_00001_.mp4"], "error": null, "ts": "2026-08-06T18:00:10Z"}
 {"label": "g4090_pi8_key_ref2v_1ref_r0", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "Use <Picture 1> as the setting. A slow cinematic pan across the sunset city skyline, clouds drifting, lights turning on in the towers, ambient city sounds.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 10, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": false, "filename_prefix": "bench/g4090_pi8_key_ref2v_1ref_r0", "ok": true, "status": "success", "wall_s": 339.8, "exec_s": 338.9, "peak_vram_gb": 21.1, "outputs": ["g4090_pi8_key_ref2v_1ref_r0_00001_.mp4"], "error": null, "ts": "2026-08-06T18:05:49Z"}
 {"label": "g4090_pi8_key_ref2v_1ref_r1", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "Use <Picture 1> as the setting. A slow cinematic pan across the sunset city skyline, clouds drifting, lights turning on in the towers, ambient city sounds.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 11, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/g4090_pi8_key_ref2v_1ref_r1", "ok": true, "status": "success", "wall_s": 241.2, "exec_s": 235.4, "peak_vram_gb": 21.2, "outputs": ["g4090_pi8_key_ref2v_1ref_r1_00001_.mp4"], "error": null, "ts": "2026-08-06T18:09:51Z"}
 {"label": "g4090_pi8_key_ref2v_1ref_r2", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "Use <Picture 1> as the setting. A slow cinematic pan across the sunset city skyline, clouds drifting, lights turning on in the towers, ambient city sounds.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 12, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/g4090_pi8_key_ref2v_1ref_r2", "ok": false, "status": "error", "wall_s": 26.2, "exec_s": 24.5, "peak_vram_gb": 20.8, "outputs": [], "error": "{\"status_str\": \"error\", \"completed\": false, \"messages\": [[\"execution_start\", {\"prompt_id\": \"d1f56b69-02e0-473a-b974-9d0ffd52a1ee\", \"timestamp\": 1786039791534}], [\"execution_cached\", {\"nodes\": [\"1\", \"2\", \"3\", \"4\", \"6\", \"7\", \"8\", \"9\", \"10\", \"5\"], \"prompt_id\": \"d1f56b69-02e0-473a-b974-9d0ffd52a1ee\", \"timestamp\": 1786039791537}], [\"execution_error\", {\"prompt_id\": \"d1f56b69-02e0-473a-b974-9d0ffd52a1ee\", \"node_id\": \"11\", \"node_type\": \"SamplerCustomAdvanced\", \"executed\": [], \"exception_message\": \"Allocation on device \\nThis error means you ran out of memory on your GPU.\\n\\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number.\", \"exception_type\": \"torch.OutOfMemoryError\", \"traceback\": [\"  File \\\"/ComfyUI/execution.py\\\", line 545, in execute\\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 344, in get_output_data\\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 318, in _async_map_node_over_list\\n    await process_inputs(input_dict, i)\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 306, in process_inputs\\n    result = f(**inputs)\\n             ^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/comfy_api/in", "ts": "2026-08-06T18:10:17Z"}
 {"label": "g4090_pi8_key_ref2v_8ref_r0", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 10, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 8, "warm": true, "filename_prefix": "bench/g4090_pi8_key_ref2v_8ref_r0", "ok": false, "status": "error", "wall_s": 246.0, "exec_s": 244.5, "peak_vram_gb": 21.0, "outputs": [], "error": "{\"status_str\": \"error\", \"completed\": false, \"messages\": [[\"execution_start\", {\"prompt_id\": \"3b5e5c37-0f7b-4458-b832-38ecfcc4c9eb\", \"timestamp\": 1786039817789}], [\"execution_cached\", {\"nodes\": [\"1\", \"2\", \"3\", \"4\", \"6\", \"7\", \"8\", \"20\"], \"prompt_id\": \"3b5e5c37-0f7b-4458-b832-38ecfcc4c9eb\", \"timestamp\": 1786039818251}], [\"execution_error\", {\"prompt_id\": \"3b5e5c37-0f7b-4458-b832-38ecfcc4c9eb\", \"node_id\": \"11\", \"node_type\": \"SamplerCustomAdvanced\", \"executed\": [\"21\", \"27\", \"22\", \"9\", \"24\", \"26\", \"5\", \"25\", \"23\"], \"exception_message\": \"Allocation on device \\nThis error means you ran out of memory on your GPU.\\n\\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number.\", \"exception_type\": \"torch.OutOfMemoryError\", \"traceback\": [\"  File \\\"/ComfyUI/execution.py\\\", line 545, in execute\\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 344, in get_output_data\\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 318, in _async_map_node_over_list\\n    await process_inputs(input_dict, i)\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 306, in process_inputs\\n    result = f(**inputs)\\n             ^^^^^^^", "ts": "2026-08-06T18:14:23Z"}
 {"label": "g4090_pi8_key_ref2v_8ref_r1", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 11, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 8, "warm": true, "filename_prefix": "bench/g4090_pi8_key_ref2v_8ref_r1", "ok": false, "status": "error", "wall_s": 18.7, "exec_s": 17.1, "peak_vram_gb": 18.6, "outputs": [], "error": "{\"status_str\": \"error\", \"completed\": false, \"messages\": [[\"execution_start\", {\"prompt_id\": \"ab3f9b09-ddb7-41e7-8c39-27ae36b468ec\", \"timestamp\": 1786040063740}], [\"execution_cached\", {\"nodes\": [\"1\", \"2\", \"3\", \"4\", \"6\", \"7\", \"8\", \"9\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"5\"], \"prompt_id\": \"ab3f9b09-ddb7-41e7-8c39-27ae36b468ec\", \"timestamp\": 1786040063964}], [\"execution_error\", {\"prompt_id\": \"ab3f9b09-ddb7-41e7-8c39-27ae36b468ec\", \"node_id\": \"11\", \"node_type\": \"SamplerCustomAdvanced\", \"executed\": [], \"exception_message\": \"Allocation on device \\nThis error means you ran out of memory on your GPU.\\n\\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number.\", \"exception_type\": \"torch.OutOfMemoryError\", \"traceback\": [\"  File \\\"/ComfyUI/execution.py\\\", line 545, in execute\\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 344, in get_output_data\\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 318, in _async_map_node_over_list\\n    await process_inputs(input_dict, i)\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 306, in process_inputs\\n    result = f(**inputs)\\n             ^^^^^", "ts": "2026-08-06T18:14:42Z"}
 {"label": "g4090_pi8_key_ref2v_8ref_r2", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 12, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 8, "warm": true, "filename_prefix": "bench/g4090_pi8_key_ref2v_8ref_r2", "ok": false, "status": "error", "wall_s": 18.5, "exec_s": 17.0, "peak_vram_gb": 18.6, "outputs": [], "error": "{\"status_str\": \"error\", \"completed\": false, \"messages\": [[\"execution_start\", {\"prompt_id\": \"4742272b-2d38-4dc9-a8e2-09d8dc6c8edc\", \"timestamp\": 1786040082480}], [\"execution_cached\", {\"nodes\": [\"1\", \"2\", \"3\", \"4\", \"6\", \"7\", \"8\", \"9\", \"20\", \"21\", \"22\", \"23\", \"24\", \"25\", \"26\", \"27\", \"5\"], \"prompt_id\": \"4742272b-2d38-4dc9-a8e2-09d8dc6c8edc\", \"timestamp\": 1786040082696}], [\"execution_error\", {\"prompt_id\": \"4742272b-2d38-4dc9-a8e2-09d8dc6c8edc\", \"node_id\": \"11\", \"node_type\": \"SamplerCustomAdvanced\", \"executed\": [], \"exception_message\": \"Allocation on device \\nThis error means you ran out of memory on your GPU.\\n\\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number.\", \"exception_type\": \"torch.OutOfMemoryError\", \"traceback\": [\"  File \\\"/ComfyUI/execution.py\\\", line 545, in execute\\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 344, in get_output_data\\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 318, in _async_map_node_over_list\\n    await process_inputs(input_dict, i)\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 306, in process_inputs\\n    result = f(**inputs)\\n             ^^^^^", "ts": "2026-08-06T18:15:00Z"}
 {"label": "g4090_pi8_key_t2v_15s", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 15, "length": 362, "steps": 20, "seed": 20, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/g4090_pi8_key_t2v_15s", "ok": false, "status": "server_lost", "wall_s": 136.5, "exec_s": null, "peak_vram_gb": 16.8, "outputs": [], "error": "server unreachable while polling: [Errno 104] Connection reset by peer", "ts": "2026-08-06T18:17:17Z"}
 {"label": "g4090b_pi8_refheavy_1ref_match", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "Use <Picture 1> as the setting. A slow cinematic pan across the sunset city skyline, clouds drifting, lights turning on in the towers, ambient city sounds.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 3, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/g4090b_pi8_refheavy_1ref_match", "ok": true, "status": "success", "wall_s": 396.2, "exec_s": 391.9, "peak_vram_gb": 21.9, "outputs": ["g4090b_pi8_refheavy_1ref_match_00001_.mp4"], "error": null, "ts": "2026-08-06T18:25:24Z"}
 {"label": "g4090b_pi8_refheavy_4ref_match", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 3, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 4, "warm": true, "filename_prefix": "bench/g4090b_pi8_refheavy_4ref_match", "ok": false, "status": "error", "wall_s": 189.6, "exec_s": 187.3, "peak_vram_gb": 22.9, "outputs": [], "error": "{\"status_str\": \"error\", \"completed\": false, \"messages\": [[\"execution_start\", {\"prompt_id\": \"db8eaf8e-a68e-40f5-ad1d-be3496225da9\", \"timestamp\": 1786040725094}], [\"execution_cached\", {\"nodes\": [], \"prompt_id\": \"db8eaf8e-a68e-40f5-ad1d-be3496225da9\", \"timestamp\": 1786040725094}], [\"execution_error\", {\"prompt_id\": \"db8eaf8e-a68e-40f5-ad1d-be3496225da9\", \"node_id\": \"11\", \"node_type\": \"SamplerCustomAdvanced\", \"executed\": [\"7\", \"9\", \"23\", \"3\", \"6\", \"22\", \"2\", \"21\", \"5\", \"1\", \"8\", \"20\", \"4\"], \"exception_message\": \"Allocation on device \\nThis error means you ran out of memory on your GPU.\\n\\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number.\", \"exception_type\": \"torch.OutOfMemoryError\", \"traceback\": [\"  File \\\"/ComfyUI/execution.py\\\", line 545, in execute\\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 344, in get_output_data\\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 318, in _async_map_node_over_list\\n    await process_inputs(input_dict, i)\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 306, in process_inputs\\n    result = f(**inputs)\\n             ^^^^^^^^^^^\\n\", \"  File \\\"/Co", "ts": "2026-08-06T18:28:34Z"}
 {"label": "g4090b_pi8_refheavy_8ref_match", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 3, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 8, "warm": true, "filename_prefix": "bench/g4090b_pi8_refheavy_8ref_match", "ok": false, "status": "error", "wall_s": 193.8, "exec_s": 189.7, "peak_vram_gb": 21.1, "outputs": [], "error": "{\"status_str\": \"error\", \"completed\": false, \"messages\": [[\"execution_start\", {\"prompt_id\": \"71dbdb62-fc28-41cd-be08-d92700592811\", \"timestamp\": 1786040915845}], [\"execution_cached\", {\"nodes\": [], \"prompt_id\": \"71dbdb62-fc28-41cd-be08-d92700592811\", \"timestamp\": 1786040915845}], [\"execution_error\", {\"prompt_id\": \"71dbdb62-fc28-41cd-be08-d92700592811\", \"node_id\": \"11\", \"node_type\": \"SamplerCustomAdvanced\", \"executed\": [\"7\", \"24\", \"23\", \"3\", \"6\", \"9\", \"22\", \"26\", \"21\", \"2\", \"5\", \"1\", \"8\", \"25\", \"20\", \"27\", \"4\"], \"exception_message\": \"Allocation on device \\nThis error means you ran out of memory on your GPU.\\n\\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number.\", \"exception_type\": \"torch.OutOfMemoryError\", \"traceback\": [\"  File \\\"/ComfyUI/execution.py\\\", line 545, in execute\\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 344, in get_output_data\\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 318, in _async_map_node_over_list\\n    await process_inputs(input_dict, i)\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 306, in process_inputs\\n    result = f(**inputs)\\n             ^^^^^", "ts": "2026-08-06T18:31:48Z"}
 {"label": "g4090b_pi8_refheavy_4ref_max", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 3, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "max", "ref_count": 4, "warm": true, "filename_prefix": "bench/g4090b_pi8_refheavy_4ref_max", "ok": false, "status": "error", "wall_s": 193.0, "exec_s": 190.2, "peak_vram_gb": 22.4, "outputs": [], "error": "{\"status_str\": \"error\", \"completed\": false, \"messages\": [[\"execution_start\", {\"prompt_id\": \"db8d48a6-7ce1-441e-b99b-28fac3921c14\", \"timestamp\": 1786041108847}], [\"execution_cached\", {\"nodes\": [], \"prompt_id\": \"db8d48a6-7ce1-441e-b99b-28fac3921c14\", \"timestamp\": 1786041108847}], [\"execution_error\", {\"prompt_id\": \"db8d48a6-7ce1-441e-b99b-28fac3921c14\", \"node_id\": \"11\", \"node_type\": \"SamplerCustomAdvanced\", \"executed\": [\"7\", \"9\", \"23\", \"3\", \"6\", \"22\", \"2\", \"21\", \"5\", \"1\", \"8\", \"20\", \"4\"], \"exception_message\": \"Allocation on device \\nThis error means you ran out of memory on your GPU.\\n\\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number.\", \"exception_type\": \"torch.OutOfMemoryError\", \"traceback\": [\"  File \\\"/ComfyUI/execution.py\\\", line 545, in execute\\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 344, in get_output_data\\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 318, in _async_map_node_over_list\\n    await process_inputs(input_dict, i)\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 306, in process_inputs\\n    result = f(**inputs)\\n             ^^^^^^^^^^^\\n\", \"  File \\\"/Co", "ts": "2026-08-06T18:35:01Z"}
 {"label": "g4090b_pi8_refheavy_8ref_max", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 3, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "max", "ref_count": 8, "warm": true, "filename_prefix": "bench/g4090b_pi8_refheavy_8ref_max", "ok": false, "status": "error", "wall_s": 152.0, "exec_s": 148.9, "peak_vram_gb": 23.1, "outputs": [], "error": "{\"status_str\": \"error\", \"completed\": false, \"messages\": [[\"execution_start\", {\"prompt_id\": \"c0da6de3-a983-435a-b0fa-db9f6e95bd1e\", \"timestamp\": 1786041302473}], [\"execution_cached\", {\"nodes\": [], \"prompt_id\": \"c0da6de3-a983-435a-b0fa-db9f6e95bd1e\", \"timestamp\": 1786041302473}], [\"execution_error\", {\"prompt_id\": \"c0da6de3-a983-435a-b0fa-db9f6e95bd1e\", \"node_id\": \"5\", \"node_type\": \"MiniMaxH3ReferenceToVideo\", \"executed\": [\"24\", \"23\", \"3\", \"22\", \"26\", \"21\", \"2\", \"25\", \"20\", \"27\", \"4\"], \"exception_message\": \"Allocation on device \\nThis error means you ran out of memory on your GPU.\\n\\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number.\", \"exception_type\": \"torch.OutOfMemoryError\", \"traceback\": [\"  File \\\"/ComfyUI/execution.py\\\", line 545, in execute\\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 344, in get_output_data\\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 318, in _async_map_node_over_list\\n    await process_inputs(input_dict, i)\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 306, in process_inputs\\n    result = f(**inputs)\\n             ^^^^^^^^^^^\\n\", \"  File \\\"/Comfy", "ts": "2026-08-06T18:37:33Z"}
 {"label": "g4090b_pi8_refheavy_8ref_match_1344", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 1344, "height": 768, "seconds": 5, "length": 124, "steps": 20, "seed": 3, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 8, "warm": true, "filename_prefix": "bench/g4090b_pi8_refheavy_8ref_match_1344", "ok": false, "status": "error", "wall_s": 122.4, "exec_s": 120.8, "peak_vram_gb": 20.4, "outputs": [], "error": "{\"status_str\": \"error\", \"completed\": false, \"messages\": [[\"execution_start\", {\"prompt_id\": \"3cca3ccb-12d6-422f-8ad6-3f7b58e731df\", \"timestamp\": 1786041453629}], [\"execution_cached\", {\"nodes\": [], \"prompt_id\": \"3cca3ccb-12d6-422f-8ad6-3f7b58e731df\", \"timestamp\": 1786041453629}], [\"execution_error\", {\"prompt_id\": \"3cca3ccb-12d6-422f-8ad6-3f7b58e731df\", \"node_id\": \"5\", \"node_type\": \"MiniMaxH3ReferenceToVideo\", \"executed\": [\"24\", \"23\", \"3\", \"22\", \"26\", \"21\", \"2\", \"25\", \"20\", \"27\", \"4\"], \"exception_message\": \"Allocation on device \\nThis error means you ran out of memory on your GPU.\\n\\nTIPS: If the workflow worked before you might have accidentally set the batch_size to a large number.\", \"exception_type\": \"torch.OutOfMemoryError\", \"traceback\": [\"  File \\\"/ComfyUI/execution.py\\\", line 545, in execute\\n    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 344, in get_output_data\\n    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 318, in _async_map_node_over_list\\n    await process_inputs(input_dict, i)\\n\", \"  File \\\"/ComfyUI/execution.py\\\", line 306, in process_inputs\\n    result = f(**inputs)\\n             ^^^^^^^^^^^\\n\", \"  File \\\"/Comfy", "ts": "2026-08-06T18:39:35Z"}
@@ -0,0 +1,8 @@
 {"label": "g4090_smoke_t2v_v2", "file": "g4090_smoke_t2v_v2_00001_.mp4", "valid": true, "bytes": 999175, "video_codec": "h264", "audio_codec": "aac", "width": 608, "height": 352, "duration": 5.17, "expected_duration": 5.17}
 {"label": "g4090_oomcheck_t2v_864", "file": "g4090_oomcheck_t2v_864_00001_.mp4", "valid": true, "bytes": 854700, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "g4090_pi8_key_t2v_5s_r0", "file": "g4090_pi8_key_t2v_5s_r0_00001_.mp4", "valid": true, "bytes": 924552, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "g4090_pi8_key_t2v_5s_r1", "file": "g4090_pi8_key_t2v_5s_r1_00001_.mp4", "valid": true, "bytes": 878698, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "g4090_pi8_key_t2v_5s_r2", "file": "g4090_pi8_key_t2v_5s_r2_00001_.mp4", "valid": true, "bytes": 1321625, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "g4090_pi8_key_ref2v_1ref_r0", "file": "g4090_pi8_key_ref2v_1ref_r0_00001_.mp4", "valid": true, "bytes": 982214, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "g4090_pi8_key_ref2v_1ref_r1", "file": "g4090_pi8_key_ref2v_1ref_r1_00001_.mp4", "valid": true, "bytes": 970993, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "g4090b_pi8_refheavy_1ref_match", "file": "g4090b_pi8_refheavy_1ref_match_00001_.mp4", "valid": true, "bytes": 751050, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
@@ -0,0 +1,81 @@
 #!/usr/bin/env python3
 """Failure statistics across GPUs and configurations.
 Classifies every recorded run (bench/*.jsonl) into success / failure buckets
 with an error class, and prints per-GPU tables showing which configurations
 fail where. Padded markdown output.
 """
 import json
 from collections import defaultdict
 from pathlib import Path
 HERE = Path(__file__).parent
 SOURCES = [
    ("B200 180GB", HERE / "results.jsonl"),
    ("RTX 4090 24GB", HERE / "4090" / "results.jsonl"),
    ("RTX PRO 4000 24GB", HERE / "4000" / "results.jsonl"),
 ]
 def classify(row):
    if row.get("ok") is True:
        return "ok"
    err = str(row.get("error") or "") + str(row.get("status") or "")
    if "server_lost" in err or "unreachable" in err or "Connection" in err:
        return "server_died"
    if "OutOfMemory" in err or "Allocation on device" in err or "OOM" in err:
        return "vram_oom"
    if "enable_gqa" in err:
        return "env_torch_too_old"
    if "400" in err and "prompt" in err.lower():
        return "graph_invalid"
    return "error_other"
 def cfg_of(row):
    return (f'{row["task"]} {row["width"]}x{row["height"]} {row["seconds"]}s '
            f'refs={row.get("ref_count") or 1} {row.get("ref_image_size") or "match"}')
 def pad_table(rows):
    widths = [max(len(r[c]) for r in rows) for c in range(len(rows[0]))]
    out = []
    for i, r in enumerate(rows):
        out.append("| " + " | ".join(v.ljust(w) for v, w in zip(r, widths)) + " |")
        if i == 0:
            out.append("|" + "|".join("-" * (w + 2) for w in widths) + "|")
    return "\n".join(out)
 def main():
    for gpu, path in SOURCES:
        if not path.exists():
            continue
        rows = [json.loads(l) for l in path.open() if l.strip()]
        total = len(rows)
        okn = sum(1 for r in rows if r.get("ok") is True)
        print(f"\n## {gpu} — {okn}/{total} runs succeeded "
              f"({100*okn/max(total,1):.0f}%)\n")
        by_cfg = defaultdict(lambda: defaultdict(int))
        for r in rows:
            by_cfg[cfg_of(r)][classify(r)] += 1
        classes = sorted({c for v in by_cfg.values() for c in v if c != "ok"})
        table = [["configuration", "ok"] + classes]
        for cfg in sorted(by_cfg):
            v = by_cfg[cfg]
            if sum(n for c, n in v.items() if c != "ok") == 0:
                continue  # only show configs with at least one failure
            table.append([cfg, str(v.get("ok", 0))] +
                         [str(v.get(c, 0)) for c in classes])
        if len(table) > 1:
            print(pad_table(table))
        else:
            print("(no failures)")
        # fully-clean configs summary
        clean = [c for c, v in sorted(by_cfg.items())
                 if sum(n for cl, n in v.items() if cl != "ok") == 0]
        print(f"\nConfigs with zero failures: {len(clean)}/{len(by_cfg)}")
 if __name__ == "__main__":
    main()
@@ -82,3 +82,16 @@ ts,label,task,model_file,width,height,seconds,length,steps,sampler,scheduler,ref
 2026-08-06T09:14:11Z,b16_refheavy_4ref_max,ref2v,minimax_h3_ref2va_bf16.safetensors,864,480,5,124,20,res_multistep,simple,max,4,True,True,91.7,90.8,155.5,b16_refheavy_4ref_max_00001_.mp4,
 2026-08-06T09:27:16Z,b16_refheavy_8ref_max,ref2v,minimax_h3_ref2va_bf16.safetensors,864,480,5,124,20,res_multistep,simple,max,8,True,True,785.5,783.9,163.9,b16_refheavy_8ref_max_00001_.mp4,
 2026-08-06T09:31:59Z,b16_refheavy_8ref_match_1344,ref2v,minimax_h3_ref2va_bf16.safetensors,1344,768,5,124,20,res_multistep,simple,match,8,True,True,282.9,282.4,157.8,b16_refheavy_8ref_match_1344_00001_.mp4,
 2026-08-06T17:52:25Z,pi8v2_key_ref2v_1ref_r0,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5.0,124,20,res_multistep,simple,match,1,True,True,139.6,137.8,175.1,pi8v2_key_ref2v_1ref_r0_00001_.mp4,
 2026-08-06T17:53:32Z,pi8v2_key_ref2v_1ref_r1,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5.0,124,20,res_multistep,simple,match,1,True,True,66.8,66.4,174.9,pi8v2_key_ref2v_1ref_r1_00001_.mp4,
 2026-08-06T17:54:39Z,pi8v2_key_ref2v_1ref_r2,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5.0,124,20,res_multistep,simple,match,1,True,True,66.6,65.6,175.4,pi8v2_key_ref2v_1ref_r2_00001_.mp4,
 2026-08-06T17:56:15Z,pi8v2_key_ref2v_8ref_r0,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5.0,124,20,res_multistep,simple,match,8,True,True,95.7,95.2,173.6,pi8v2_key_ref2v_8ref_r0_00001_.mp4,
 2026-08-06T17:57:51Z,pi8v2_key_ref2v_8ref_r1,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5.0,124,20,res_multistep,simple,match,8,True,True,95.8,95.0,175.7,pi8v2_key_ref2v_8ref_r1_00001_.mp4,
 2026-08-06T17:59:27Z,pi8v2_key_ref2v_8ref_r2,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5.0,124,20,res_multistep,simple,match,8,True,True,95.9,94.0,176.7,pi8v2_key_ref2v_8ref_r2_00001_.mp4,
 2026-08-06T18:13:28Z,pi8v2_key_ref2v_8ref_max,ref2v,minimax_h3_ref2va_pruned_int8_convrot.safetensors,864,480,5.0,124,20,res_multistep,simple,max,8,True,True,841.8,840.9,172.0,pi8v2_key_ref2v_8ref_max_00001_.mp4,
 2026-08-06T18:19:11Z,conc_warm_A,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,864,480,5.0,124,20,res_multistep,simple,match,1,False,True,237.7,236.4,43.2,conc_warm_A_00001_.mp4,
 2026-08-06T18:22:10Z,conc_warm_B,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,864,480,5.0,124,20,res_multistep,simple,match,1,False,True,179.2,177.9,84.6,conc_warm_B_00001_.mp4,
 2026-08-06T18:23:15Z,conc_seq_1,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,864,480,5.0,124,20,res_multistep,simple,match,1,True,True,64.7,62.6,88.1,conc_seq_1_00001_.mp4,
 2026-08-06T18:24:18Z,conc_seq_2,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,864,480,5.0,124,20,res_multistep,simple,match,1,True,True,62.9,62.5,88.2,conc_seq_2_00001_.mp4,
 2026-08-06T18:26:41Z,conc_par_A,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,864,480,5.0,124,20,res_multistep,simple,match,1,True,True,142.7,142.4,93.6,conc_par_A_00001_.mp4,
 2026-08-06T18:26:43Z,conc_par_B,t2v,minimax_h3_fl2va_pruned_int8_convrot.safetensors,864,480,5.0,124,20,res_multistep,simple,match,1,True,True,144.7,142.9,93.3,conc_par_B_00001_.mp4,
@@ -81,3 +81,16 @@
 {"label": "b16_refheavy_4ref_max", "task": "ref2v", "model_file": "minimax_h3_ref2va_bf16.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 3, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "max", "ref_count": 4, "warm": true, "filename_prefix": "bench/b16_refheavy_4ref_max", "ok": true, "status": "success", "wall_s": 91.7, "exec_s": 90.8, "peak_vram_gb": 155.5, "outputs": ["b16_refheavy_4ref_max_00001_.mp4"], "error": null, "ts": "2026-08-06T09:14:11Z"}
 {"label": "b16_refheavy_8ref_max", "task": "ref2v", "model_file": "minimax_h3_ref2va_bf16.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5, "length": 124, "steps": 20, "seed": 3, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "max", "ref_count": 8, "warm": true, "filename_prefix": "bench/b16_refheavy_8ref_max", "ok": true, "status": "success", "wall_s": 785.5, "exec_s": 783.9, "peak_vram_gb": 163.9, "outputs": ["b16_refheavy_8ref_max_00001_.mp4"], "error": null, "ts": "2026-08-06T09:27:16Z"}
 {"label": "b16_refheavy_8ref_match_1344", "task": "ref2v", "model_file": "minimax_h3_ref2va_bf16.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 1344, "height": 768, "seconds": 5, "length": 124, "steps": 20, "seed": 3, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 8, "warm": true, "filename_prefix": "bench/b16_refheavy_8ref_match_1344", "ok": true, "status": "success", "wall_s": 282.9, "exec_s": 282.4, "peak_vram_gb": 157.8, "outputs": ["b16_refheavy_8ref_match_1344_00001_.mp4"], "error": null, "ts": "2026-08-06T09:31:59Z"}
 {"label": "pi8v2_key_ref2v_1ref_r0", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "Use <Picture 1> as the setting. A slow cinematic pan across the sunset city skyline, clouds drifting, lights turning on in the towers, ambient city sounds.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 10, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/pi8v2_key_ref2v_1ref_r0", "ok": true, "status": "success", "wall_s": 139.6, "exec_s": 137.8, "peak_vram_gb": 175.1, "outputs": ["pi8v2_key_ref2v_1ref_r0_00001_.mp4"], "error": null, "ts": "2026-08-06T17:52:25Z"}
 {"label": "pi8v2_key_ref2v_1ref_r1", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "Use <Picture 1> as the setting. A slow cinematic pan across the sunset city skyline, clouds drifting, lights turning on in the towers, ambient city sounds.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 11, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/pi8v2_key_ref2v_1ref_r1", "ok": true, "status": "success", "wall_s": 66.8, "exec_s": 66.4, "peak_vram_gb": 174.9, "outputs": ["pi8v2_key_ref2v_1ref_r1_00001_.mp4"], "error": null, "ts": "2026-08-06T17:53:32Z"}
 {"label": "pi8v2_key_ref2v_1ref_r2", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "Use <Picture 1> as the setting. A slow cinematic pan across the sunset city skyline, clouds drifting, lights turning on in the towers, ambient city sounds.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 12, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/pi8v2_key_ref2v_1ref_r2", "ok": true, "status": "success", "wall_s": 66.6, "exec_s": 65.6, "peak_vram_gb": 175.4, "outputs": ["pi8v2_key_ref2v_1ref_r2_00001_.mp4"], "error": null, "ts": "2026-08-06T17:54:39Z"}
 {"label": "pi8v2_key_ref2v_8ref_r0", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 10, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 8, "warm": true, "filename_prefix": "bench/pi8v2_key_ref2v_8ref_r0", "ok": true, "status": "success", "wall_s": 95.7, "exec_s": 95.2, "peak_vram_gb": 173.6, "outputs": ["pi8v2_key_ref2v_8ref_r0_00001_.mp4"], "error": null, "ts": "2026-08-06T17:56:15Z"}
 {"label": "pi8v2_key_ref2v_8ref_r1", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 11, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 8, "warm": true, "filename_prefix": "bench/pi8v2_key_ref2v_8ref_r1", "ok": true, "status": "success", "wall_s": 95.8, "exec_s": 95.0, "peak_vram_gb": 175.7, "outputs": ["pi8v2_key_ref2v_8ref_r1_00001_.mp4"], "error": null, "ts": "2026-08-06T17:57:51Z"}
 {"label": "pi8v2_key_ref2v_8ref_r2", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 12, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 8, "warm": true, "filename_prefix": "bench/pi8v2_key_ref2v_8ref_r2", "ok": true, "status": "success", "wall_s": 95.9, "exec_s": 94.0, "peak_vram_gb": 176.7, "outputs": ["pi8v2_key_ref2v_8ref_r2_00001_.mp4"], "error": null, "ts": "2026-08-06T17:59:27Z"}
 {"label": "pi8v2_key_ref2v_8ref_max", "task": "ref2v", "model_file": "minimax_h3_ref2va_pruned_int8_convrot.safetensors", "prompt": "A sweeping cinematic montage inspired by <Picture 1>, <Picture 2>, <Picture 3>, <Picture 4>, <Picture 5>, <Picture 6>, <Picture 7>, <Picture 8>: dinosaurs stalking through ancient forests, an anime hero surveying a volcanic ridge, crossfading between the scenes and moods of each reference in order, camera drifting forward the whole time, an adventurous orchestral score building throughout.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 20, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "max", "ref_count": 8, "warm": true, "filename_prefix": "bench/pi8v2_key_ref2v_8ref_max", "ok": true, "status": "success", "wall_s": 841.8, "exec_s": 840.9, "peak_vram_gb": 172.0, "outputs": ["pi8v2_key_ref2v_8ref_max_00001_.mp4"], "error": null, "ts": "2026-08-06T18:13:28Z"}
 {"label": "conc_warm_A", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 30, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": false, "filename_prefix": "bench/conc_warm_A", "ok": true, "status": "success", "wall_s": 237.7, "exec_s": 236.4, "peak_vram_gb": 43.2, "outputs": ["conc_warm_A_00001_.mp4"], "error": null, "ts": "2026-08-06T18:19:11Z"}
 {"label": "conc_warm_B", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 30, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": false, "filename_prefix": "bench/conc_warm_B", "ok": true, "status": "success", "wall_s": 179.2, "exec_s": 177.9, "peak_vram_gb": 84.6, "outputs": ["conc_warm_B_00001_.mp4"], "error": null, "ts": "2026-08-06T18:22:10Z"}
 {"label": "conc_seq_1", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 31, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/conc_seq_1", "ok": true, "status": "success", "wall_s": 64.7, "exec_s": 62.6, "peak_vram_gb": 88.1, "outputs": ["conc_seq_1_00001_.mp4"], "error": null, "ts": "2026-08-06T18:23:15Z"}
 {"label": "conc_seq_2", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 32, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/conc_seq_2", "ok": true, "status": "success", "wall_s": 62.9, "exec_s": 62.5, "peak_vram_gb": 88.2, "outputs": ["conc_seq_2_00001_.mp4"], "error": null, "ts": "2026-08-06T18:24:18Z"}
 {"label": "conc_par_A", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 33, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/conc_par_A", "ok": true, "status": "success", "wall_s": 142.7, "exec_s": 142.4, "peak_vram_gb": 93.6, "outputs": ["conc_par_A_00001_.mp4"], "error": null, "ts": "2026-08-06T18:26:41Z"}
 {"label": "conc_par_B", "task": "t2v", "model_file": "minimax_h3_fl2va_pruned_int8_convrot.safetensors", "prompt": "Cinematic aerial shot slowly orbiting a coastal lighthouse at golden hour, waves crashing on dark rocks below, seagulls circling, warm sunlight flares, sound of surf and wind, gentle orchestral swell.", "width": 864, "height": 480, "seconds": 5.0, "length": 124, "steps": 20, "seed": 34, "sampler": "res_multistep", "scheduler": "simple", "image": "bench2_jp_01.jpg", "ref_image_size": "match", "ref_count": 1, "warm": true, "filename_prefix": "bench/conc_par_B", "ok": true, "status": "success", "wall_s": 144.7, "exec_s": 142.9, "peak_vram_gb": 93.3, "outputs": ["conc_par_B_00001_.mp4"], "error": null, "ts": "2026-08-06T18:26:43Z"}
@@ -0,0 +1,91 @@
 #!/usr/bin/env python3
 """Validate benchmark output videos: download each recorded output via the
 ComfyUI /view endpoint and ffprobe it (streams, duration, size).
 Usage:
  python3 bench/validate_outputs.py --host http://127.0.0.1:8189 --jsonl bench/4090/results.jsonl [--sample N]
 Appends results to <jsonl-dir>/validation.jsonl and prints a summary table.
 A video is VALID if it has an h264 video stream and an audio stream, and its
 container duration is within 0.5 s of the requested clip length.
 """
 import argparse
 import json
 import subprocess
 import tempfile
 import urllib.parse
 import urllib.request
 from pathlib import Path
 def ffprobe(path):
    out = subprocess.run(
        ["ffprobe", "-v", "error", "-show_entries",
         "stream=codec_type,codec_name,width,height:format=duration,size",
         "-of", "json", path],
        capture_output=True, text=True, timeout=60)
    if out.returncode != 0:
        return None
    return json.loads(out.stdout)
 def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--host", required=True)
    ap.add_argument("--jsonl", required=True)
    ap.add_argument("--sample", type=int, default=0, help="validate only every Nth eligible row (0 = all)")
    args = ap.parse_args()
    jl = Path(args.jsonl)
    rows = [json.loads(l) for l in jl.open() if l.strip()]
    ok_rows = [r for r in rows if r.get("ok") is True and r.get("outputs")]
    if args.sample:
        ok_rows = ok_rows[::args.sample]
    results = []
    for r in ok_rows:
        for fname in r["outputs"]:
            q = urllib.parse.urlencode(
                {"filename": fname, "subfolder": "bench", "type": "output"})
            url = f"{args.host}/view?{q}"
            verdict = {"label": r["label"], "file": fname, "valid": False}
            try:
                with tempfile.NamedTemporaryFile(suffix=".mp4") as tmp:
                    with urllib.request.urlopen(url, timeout=300) as resp:
                        data = resp.read()
                    tmp.write(data)
                    tmp.flush()
                    verdict["bytes"] = len(data)
                    info = ffprobe(tmp.name) if len(data) > 10000 else None
                if info:
                    streams = {s["codec_type"]: s for s in info.get("streams", [])}
                    dur = float(info.get("format", {}).get("duration", 0))
                    want = float(r["length"]) / 24.0
                    verdict.update({
                        "video_codec": streams.get("video", {}).get("codec_name"),
                        "audio_codec": streams.get("audio", {}).get("codec_name"),
                        "width": streams.get("video", {}).get("width"),
                        "height": streams.get("video", {}).get("height"),
                        "duration": round(dur, 2),
                        "expected_duration": round(want, 2),
                        "valid": ("video" in streams and "audio" in streams
                                  and abs(dur - want) < 0.5),
                    })
            except Exception as e:
                verdict["error"] = str(e)[:200]
            results.append(verdict)
            v = "VALID" if verdict["valid"] else "INVALID"
            print(f'{v:<8} {fname:<48} {verdict.get("bytes",0)//1024:>7} KB '
                  f'{verdict.get("width","?")}x{verdict.get("height","?")} '
                  f'{verdict.get("duration","?")}s/{verdict.get("expected_duration","?")}s '
                  f'{verdict.get("video_codec","")}+{verdict.get("audio_codec","")}')
    with (jl.parent / "validation.jsonl").open("a") as f:
        for v in results:
            f.write(json.dumps(v) + "\n")
    good = sum(1 for v in results if v["valid"])
    print(f"\n{good}/{len(results)} outputs valid")
 if __name__ == "__main__":
    main()
@@ -0,0 +1,12 @@
 {"label": "smoke_t2v", "file": "smoke_t2v_00001_.mp4", "valid": false, "error": "HTTP Error 404: Not Found"}
 {"label": "prunedint8_sweep_t2v_864x480_15s", "file": "prunedint8_sweep_t2v_864x480_15s_00001_.mp4", "valid": false, "error": "HTTP Error 404: Not Found"}
 {"label": "prunedint8_sweep_ref2v_864x480_5s", "file": "prunedint8_sweep_ref2v_864x480_5s_00001_.mp4", "valid": false, "error": "HTTP Error 404: Not Found"}
 {"label": "prunedint8_refheavy_4ref_match", "file": "prunedint8_refheavy_4ref_match_00001_.mp4", "valid": true, "bytes": 687266, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "pi8_key_ref2v_1ref_r0", "file": "pi8_key_ref2v_1ref_r0_00001_.mp4", "valid": true, "bytes": 826358, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "pi8_key_ref2v_8ref_max", "file": "pi8_key_ref2v_8ref_max_00001_.mp4", "valid": true, "bytes": 1007718, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "i8_key_t2v_5s_r1", "file": "i8_key_t2v_5s_r1_00001_.mp4", "valid": true, "bytes": 858674, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "i8_key_t2v_15s", "file": "i8_key_t2v_15s_00001_.mp4", "valid": true, "bytes": 3295798, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 15.08, "expected_duration": 15.08}
 {"label": "i8_refheavy_8ref_match_1344", "file": "i8_refheavy_8ref_match_1344_00001_.mp4", "valid": true, "bytes": 1523680, "video_codec": "h264", "audio_codec": "aac", "width": 1344, "height": 768, "duration": 5.17, "expected_duration": 5.17}
 {"label": "b16_key_ref2v_8ref_r1", "file": "b16_key_ref2v_8ref_r1_00001_.mp4", "valid": true, "bytes": 1117762, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "b16_refheavy_4ref_max", "file": "b16_refheavy_4ref_max_00001_.mp4", "valid": true, "bytes": 763758, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
 {"label": "pi8v2_key_ref2v_8ref_r2", "file": "pi8v2_key_ref2v_8ref_r2_00001_.mp4", "valid": true, "bytes": 1355173, "video_codec": "h264", "audio_codec": "aac", "width": 864, "height": 480, "duration": 5.17, "expected_duration": 5.17}
@@ -90,6 +90,45 @@ scheduler contention make co-scheduling strictly worse. **One worker per GPU,
 queue in front, scale horizontally.** (This also removes any argument for
 MPS/MIG partitioning for throughput.)
 ## Reliability & failure statistics
 Every run's end state and error is recorded in the JSONL files; regenerate the
 failure tables with `python3 bench/failures.py`, and validate output files
 with `python3 bench/validate_outputs.py` (downloads each MP4 and ffprobes
 streams + duration). Snapshot as of 2026-08-06 ~09:00 UTC:
 | GPU               | success rate | dominant failure                                      |
 |-------------------|--------------|-------------------------------------------------------|
 | B200 180 GB       | 96/96 (100%) | none                                                  |
 | RTX 4090 24 GB    | 8/42 (19%)   | VRAM OOM in text-encoder vision path on ref2v         |
 | RTX PRO 4000 24 GB| 6/7 so far   | one container-RAM OOM kill before mitigation flags    |
 **Output validation**: 100% of successful runs produced valid videos (h264 +
 stereo AAC at exactly the requested duration) — every success on all three
 GPUs checks out. (Three early B200 outputs 404 on re-download because they
 were written to the container disk that the pod reset wiped — outputs now go
 to the persistent volume via `--output-directory`.)
 **The 24 GB failure pattern is reference images, not duration/resolution.**
 t2v succeeds reliably on both consumer cards at every size tried; ref2v OOMs
 in `torch._int_mm` during Qwen3-VL image encoding: the 15 GB text encoder plus
 multi-image vision activations exceeds 24 GB (observed 22.1 GB peak at OOM).
 With aggressive flags (`--cache-none --disable-pinned-memory --reserve-vram
 2.5`) a **single** reference works (4090: 392 s); 4-ref and 8-ref
 configurations fail consistently, and repeated OOMs eventually killed the
 server process, cascading failures across the rest of that suite (harness now
 survives this and records `server_died`).
 A second environment trap on cheap tiers: the RTX 4000 pod's container has a
 **29 GB RAM cap** that OOM-killed weight staging until `--cache-none
 --disable-pinned-memory` was applied; both EU pods also shipped torch 2.4.1,
 too old for H3 (needs ≥2.5; Blackwell needs ≥2.7+cu128).
 **Fleet implication:** consumer 24 GB cards are viable for t2v/i2v and
 single-ref ref2v only. Multi-reference work — the priority modality — needs
 ≥32 GB (5090-class, unverified) and realistically ≥48 GB for headroom, or it
 stays on big-GPU pools.
 ## Takeaways
 1. **On the B200, run bf16.** Fastest and highest quality; quantized families
@@ -7,6 +7,7 @@
 set -euo pipefail
 LOG=/workspace/minimax-h3/comfyui-$(hostname).log
 mkdir -p /workspace/minimax-h3/output
 ARGS=${COMFY_ARGS:-}
 # wait out any process that is mid-shutdown before deciding it's "running"
@@ -21,9 +22,9 @@ fi
 if command -v tmux >/dev/null; then
  tmux new -s comfyui -d \
-    "cd /ComfyUI && python3 main.py --listen 127.0.0.1 --port 8188 $ARGS 2>&1 | tee -a $LOG"
+    "cd /ComfyUI && python3 main.py --listen 127.0.0.1 --port 8188 --output-directory /workspace/minimax-h3/output $ARGS 2>&1 | tee -a $LOG"
 else
  cd /ComfyUI
-  nohup python3 main.py --listen 127.0.0.1 --port 8188 $ARGS >>"$LOG" 2>&1 &
+  nohup python3 main.py --listen 127.0.0.1 --port 8188 --output-directory /workspace/minimax-h3/output $ARGS >>"$LOG" 2>&1 &
 fi
 echo "started; tail $LOG"