Expand MiniMax R2V evaluation matrix

This commit is contained in:
Morpheus
2026-09-25 21:43:20 +00:00
parent c0c1876f38
commit ff2e6d7fd4
2 changed files with 77 additions and 33 deletions
+70 -33
View File
@@ -1,50 +1,87 @@
# MiniMax H3 — optimized reference-to-video
This is the first cleaned MiniMax H3 R2V evaluation workflow. It is derived
from the compact official MiniMax H3 int8 R2V graph and is kept separate from
the imported source workflows under `models/minimax/`.
This is the MiniMax H3 R2V evaluation workflow. It is derived from the
imported MiniMax H3 int8 graph and kept separate from the source workflows
under `models/minimax/`.
## Fixed variables
## Fixed comparison inputs
- Reference media: `reference.png`, supplied to the workflow as Picture 1
- Prompt: `prompt.txt`, fixed for this comparison batch
- Seed: keep fixed for each comparison batch
- Reference: `reference.png`, supplied as Picture 1
- Prompt: `prompt.txt`
- Seed: fixed within each comparison batch
- Resolution: 864×480 (480p landscape)
- Duration: 3 seconds for the initial efficiency matrix
- Initial duration: 3 seconds
- CFG: **1.0**
- Model, text encoder, VAE, batch size, and denoise: fixed
- Audio: none for the initial R2V matrix
## Base workflow
Only the variables in the matrix may change. Before each row is run, confirm
that the named node or sampler exists on the live server. Unsupported rows
must be marked `Unsupported` with the exact validation error; do not silently
substitute another configuration.
- `workflow.json` — derived ComfyUI web workflow
- `reference.png` — fixed character reference for the initial batch
## Files
- `workflow.json` — ComfyUI web workflow derived from the source graph
- `reference.png` — fixed character reference
- `prompt.txt` — fixed haunted-room investigation prompt
- Source control graph: `models/minimax/minimax-h3-int8-r2v/workflow.json`
- The imported source is not modified.
- `variables.json` — fixed comparison variables
- `results/` — labelled outputs and matching JSON metadata
## Efficiency test matrix
## Test matrix
The first two tests compare a normal control against the first compatible
memory-saving patch. All rows use the fixed variables above. The initial test
does not vary steps, sampler, or scheduler so that the efficiency difference
is isolated.
All rows are planned; only the scheduled MM-R2V-01 and MM-R2V-02 control/
memory tests should be run first. Later phases are gated on valid 3-second
results and should not be queued automatically with Phase 1.
| ID | Memory path | Steps | Sampler | Scheduler | Duration | CFG | Runtime (s) | VRAM | Tested? | Valid? | Current Best |
|---|---|---:|---|---|---:|---:|---:|---:|---|---|---|
| MM-R2V-01 | Normal/control | 8 | workflow default | workflow default | 3 | 1.0 | — | — | No | — | No |
| MM-R2V-02 | MiniMaxH3MemoryEfficientSageAttentionPatch | 8 | workflow default | workflow default | 3 | 1.0 | — | — | No | — | No |
| ID | Phase | Efficiency / node combination | Steps | Sampler | Scheduler | Duration (s) | CFG | Tested? | Valid? | Current Best | Runtime (s) |
|---|---|---|---:|---|---|---:|---:|---|---|---|---:|
| MM-R2V-01 | A control | Base workflow | 8 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-02 | A memory | MemoryEfficientSageAttentionPatch | 8 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-03 | A memory | Model cache/offload path only | 8 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-04 | A attention | Sol-Attn only | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-05 | A attention | Spectrum only | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-06 | A LoRA | Turbo LoRA only | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-07 | A combined | Memory + Sol-Attn | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-08 | A combined | Memory + Spectrum | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-09 | A combined | Memory + Turbo LoRA | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-10 | A combined | Sol-Attn + Turbo LoRA | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-11 | A combined | Spectrum + Turbo LoRA | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-12 | A combined | Memory + Sol-Attn + Turbo LoRA | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
| MM-R2V-13 | B steps | Best valid Phase A path | 4 | best valid | best valid | 3 | 1.0 | No | — | No | — |
| MM-R2V-14 | B steps | Best valid Phase A path | 6 | best valid | best valid | 3 | 1.0 | No | — | No | — |
| MM-R2V-15 | B steps | Best valid Phase A path | 8 | best valid | best valid | 3 | 1.0 | No | — | No | — |
| MM-R2V-16 | B steps | Best valid Phase A path | 12 | best valid | best valid | 3 | 1.0 | No | — | No | — |
| MM-R2V-17 | B sampler | Best valid Phase A path | 6 | euler | simple | 3 | 1.0 | No | — | No | — |
| MM-R2V-18 | B sampler | Best valid Phase A path | 6 | euler_cfg_pp | simple | 3 | 1.0 | No | — | No | — |
| MM-R2V-19 | B sampler | Best valid Phase A path | 6 | res_multistep | simple | 3 | 1.0 | No | — | No | — |
| MM-R2V-20 | B sampler | Best valid Phase A path | 6 | dpmpp_2m | simple | 3 | 1.0 | No | — | No | — |
| MM-R2V-21 | B scheduler | Best valid Phase A path | 6 | best valid | beta | 3 | 1.0 | No | — | No | — |
| MM-R2V-22 | B scheduler | Best valid Phase A path | 6 | best valid | sgm_uniform | 3 | 1.0 | No | — | No | — |
| MM-R2V-23 | B scheduler | Best valid Phase A path | 6 | best valid | karras | 3 | 1.0 | No | — | No | — |
| MM-R2V-24 | B balance | Best valid Phase A path | 8 | best valid | best valid | 3 | 1.0 | No | — | No | — |
| MM-R2V-25 | C duration | Best 3-second path | best | best | best | 6 | 1.0 | No | — | No | — |
| MM-R2V-26 | C duration | Best 3-second path | best | best | best | 10 | 1.0 | No | — | No | — |
| MM-R2V-27 | C duration | Best 3-second path | best | best | best | 20 | 1.0 | No | — | No | — |
| MM-R2V-28 | D upscale | Best 3-second path + MiniMax latent upscale | best | best | best | 3 | 1.0 | No | — | No | — |
| MM-R2V-29 | D upscale | Best 3-second path + RTX Super Resolution | best | best | best | 3 | 1.0 | No | — | No | — |
| MM-R2V-30 | E extend | Best valid path, concatenate two clips | best | best | best | 6 | 1.0 | No | — | No | — |
| MM-R2V-31 | E extend | Best valid path, concatenate three clips | best | best | best | 9 | 1.0 | No | — | No | — |
## Planned expansion
## Result requirements
After these two tests:
Each run must save `results/<test-id>.mp4` and matching
`results/<test-id>.json`. Metadata must include the exact reference checksum,
prompt hash, seed, resolution, duration, CFG, steps, sampler, scheduler,
active nodes, runtime, VRAM, output checksum, validity, and visual/motion
verdict. A result becomes Current Best only when it is valid and objectively
better than the existing best; keep exactly one Current Best.
1. Test other compatible memory nodes individually.
2. Test safe combinations of compatible nodes.
3. Sweep steps, sampler, and scheduler with the best efficiency path.
4. Run only the strongest 3-second configurations at 20 seconds.
5. Test reference-driven continuation/concatenation.
6. Compare latent upscale and RTX Video Super Resolution toward 1080p.
## Execution order
1. Run MM-R2V-01 and MM-R2V-02 as the initial paired efficiency test.
2. Run Phase A one-node and safe-combination tests on 3 seconds.
3. Optimize steps, sampler, scheduler, and quality at 3 seconds.
4. Only then test 6/10/20-second clips.
5. Test continuation/concatenation, then latent upscale and RTX Super Resolution.
Every result must be saved as `results/<test-id>.*` with matching JSON
metadata containing the exact reference, prompt, seed, resolution, duration,
CFG, node settings, runtime, VRAM, validity, and visual verdict.
+7
View File
@@ -5,6 +5,13 @@
"resolution": [864, 480],
"duration_seconds": 3,
"cfg": 1,
"test_phases": [
"A: efficiency nodes and safe combinations",
"B: steps, samplers, and schedulers",
"C: duration extension",
"D: latent upscale and RTX Super Resolution",
"E: clip continuation and concatenation"
],
"model": "fixed by workflow",
"text_encoder": "fixed by workflow",
"vae": "fixed by workflow",