Expand MiniMax R2V evaluation matrix

This commit is contained in:
Morpheus
2026-09-25 21:43:20 +00:00
parent c0c1876f38
commit ff2e6d7fd4
2 changed files with 77 additions and 33 deletions
+70 -33
View File
@@ -1,50 +1,87 @@
# MiniMax H3 — optimized reference-to-video # MiniMax H3 — optimized reference-to-video
This is the first cleaned MiniMax H3 R2V evaluation workflow. It is derived This is the MiniMax H3 R2V evaluation workflow. It is derived from the
from the compact official MiniMax H3 int8 R2V graph and is kept separate from imported MiniMax H3 int8 graph and kept separate from the source workflows
the imported source workflows under `models/minimax/`. under `models/minimax/`.
## Fixed variables ## Fixed comparison inputs
- Reference media: `reference.png`, supplied to the workflow as Picture 1 - Reference: `reference.png`, supplied as Picture 1
- Prompt: `prompt.txt`, fixed for this comparison batch - Prompt: `prompt.txt`
- Seed: keep fixed for each comparison batch - Seed: fixed within each comparison batch
- Resolution: 864×480 (480p landscape) - Resolution: 864×480 (480p landscape)
- Duration: 3 seconds for the initial efficiency matrix - Initial duration: 3 seconds
- CFG: **1.0** - CFG: **1.0**
- Model, text encoder, VAE, batch size, and denoise: fixed - Model, text encoder, VAE, batch size, and denoise: fixed
- Audio: none for the initial R2V matrix
## Base workflow Only the variables in the matrix may change. Before each row is run, confirm
that the named node or sampler exists on the live server. Unsupported rows
must be marked `Unsupported` with the exact validation error; do not silently
substitute another configuration.
- `workflow.json` — derived ComfyUI web workflow ## Files
- `reference.png` — fixed character reference for the initial batch
- `workflow.json` — ComfyUI web workflow derived from the source graph
- `reference.png` — fixed character reference
- `prompt.txt` — fixed haunted-room investigation prompt - `prompt.txt` — fixed haunted-room investigation prompt
- Source control graph: `models/minimax/minimax-h3-int8-r2v/workflow.json` - `variables.json` — fixed comparison variables
- The imported source is not modified. - `results/` — labelled outputs and matching JSON metadata
## Efficiency test matrix ## Test matrix
The first two tests compare a normal control against the first compatible All rows are planned; only the scheduled MM-R2V-01 and MM-R2V-02 control/
memory-saving patch. All rows use the fixed variables above. The initial test memory tests should be run first. Later phases are gated on valid 3-second
does not vary steps, sampler, or scheduler so that the efficiency difference results and should not be queued automatically with Phase 1.
is isolated.
| ID | Memory path | Steps | Sampler | Scheduler | Duration | CFG | Runtime (s) | VRAM | Tested? | Valid? | Current Best | | ID | Phase | Efficiency / node combination | Steps | Sampler | Scheduler | Duration (s) | CFG | Tested? | Valid? | Current Best | Runtime (s) |
|---|---|---:|---|---|---:|---:|---:|---:|---|---|---| |---|---|---|---:|---|---|---:|---:|---|---|---|---:|
| MM-R2V-01 | Normal/control | 8 | workflow default | workflow default | 3 | 1.0 | — | — | No | — | No | | MM-R2V-01 | A control | Base workflow | 8 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-02 | MiniMaxH3MemoryEfficientSageAttentionPatch | 8 | workflow default | workflow default | 3 | 1.0 | — | — | No | — | No | | MM-R2V-02 | A memory | MemoryEfficientSageAttentionPatch | 8 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-03 | A memory | Model cache/offload path only | 8 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-04 | A attention | Sol-Attn only | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-05 | A attention | Spectrum only | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-06 | A LoRA | Turbo LoRA only | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-07 | A combined | Memory + Sol-Attn | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-08 | A combined | Memory + Spectrum | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-09 | A combined | Memory + Turbo LoRA | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-10 | A combined | Sol-Attn + Turbo LoRA | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-11 | A combined | Spectrum + Turbo LoRA | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-12 | A combined | Memory + Sol-Attn + Turbo LoRA | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-13 | B steps | Best valid Phase A path | 4 | best valid | best valid | 3 | 1.0 | No | — | No | — |
| MM-R2V-14 | B steps | Best valid Phase A path | 6 | best valid | best valid | 3 | 1.0 | No | — | No | — |
| MM-R2V-15 | B steps | Best valid Phase A path | 8 | best valid | best valid | 3 | 1.0 | No | — | No | — |
| MM-R2V-16 | B steps | Best valid Phase A path | 12 | best valid | best valid | 3 | 1.0 | No | — | No | — |
| MM-R2V-17 | B sampler | Best valid Phase A path | 6 | euler | simple | 3 | 1.0 | No | — | No | — |
| MM-R2V-18 | B sampler | Best valid Phase A path | 6 | euler_cfg_pp | simple | 3 | 1.0 | No | — | No | — |
| MM-R2V-19 | B sampler | Best valid Phase A path | 6 | res_multistep | simple | 3 | 1.0 | No | — | No | — |
| MM-R2V-20 | B sampler | Best valid Phase A path | 6 | dpmpp_2m | simple | 3 | 1.0 | No | — | No | — |
| MM-R2V-21 | B scheduler | Best valid Phase A path | 6 | best valid | beta | 3 | 1.0 | No | — | No | — |
| MM-R2V-22 | B scheduler | Best valid Phase A path | 6 | best valid | sgm_uniform | 3 | 1.0 | No | — | No | — |
| MM-R2V-23 | B scheduler | Best valid Phase A path | 6 | best valid | karras | 3 | 1.0 | No | — | No | — |
| MM-R2V-24 | B balance | Best valid Phase A path | 8 | best valid | best valid | 3 | 1.0 | No | — | No | — |
| MM-R2V-25 | C duration | Best 3-second path | best | best | best | 6 | 1.0 | No | — | No | — |
| MM-R2V-26 | C duration | Best 3-second path | best | best | best | 10 | 1.0 | No | — | No | — |
| MM-R2V-27 | C duration | Best 3-second path | best | best | best | 20 | 1.0 | No | — | No | — |
| MM-R2V-28 | D upscale | Best 3-second path + MiniMax latent upscale | best | best | best | 3 | 1.0 | No | — | No | — |
| MM-R2V-29 | D upscale | Best 3-second path + RTX Super Resolution | best | best | best | 3 | 1.0 | No | — | No | — |
| MM-R2V-30 | E extend | Best valid path, concatenate two clips | best | best | best | 6 | 1.0 | No | — | No | — |
| MM-R2V-31 | E extend | Best valid path, concatenate three clips | best | best | best | 9 | 1.0 | No | — | No | — |
## Planned expansion ## Result requirements
After these two tests: Each run must save `results/<test-id>.mp4` and matching
`results/<test-id>.json`. Metadata must include the exact reference checksum,
prompt hash, seed, resolution, duration, CFG, steps, sampler, scheduler,
active nodes, runtime, VRAM, output checksum, validity, and visual/motion
verdict. A result becomes Current Best only when it is valid and objectively
better than the existing best; keep exactly one Current Best.
1. Test other compatible memory nodes individually. ## Execution order
2. Test safe combinations of compatible nodes.
3. Sweep steps, sampler, and scheduler with the best efficiency path. 1. Run MM-R2V-01 and MM-R2V-02 as the initial paired efficiency test.
4. Run only the strongest 3-second configurations at 20 seconds. 2. Run Phase A one-node and safe-combination tests on 3 seconds.
5. Test reference-driven continuation/concatenation. 3. Optimize steps, sampler, scheduler, and quality at 3 seconds.
6. Compare latent upscale and RTX Video Super Resolution toward 1080p. 4. Only then test 6/10/20-second clips.
5. Test continuation/concatenation, then latent upscale and RTX Super Resolution.
Every result must be saved as `results/<test-id>.*` with matching JSON
metadata containing the exact reference, prompt, seed, resolution, duration,
CFG, node settings, runtime, VRAM, validity, and visual verdict.
+7
View File
@@ -5,6 +5,13 @@
"resolution": [864, 480], "resolution": [864, 480],
"duration_seconds": 3, "duration_seconds": 3,
"cfg": 1, "cfg": 1,
"test_phases": [
"A: efficiency nodes and safe combinations",
"B: steps, samplers, and schedulers",
"C: duration extension",
"D: latent upscale and RTX Super Resolution",
"E: clip continuation and concatenation"
],
"model": "fixed by workflow", "model": "fixed by workflow",
"text_encoder": "fixed by workflow", "text_encoder": "fixed by workflow",
"vae": "fixed by workflow", "vae": "fixed by workflow",