Expand MiniMax R2V evaluation matrix
This commit is contained in:
@@ -1,50 +1,87 @@
|
||||
# MiniMax H3 — optimized reference-to-video
|
||||
|
||||
This is the first cleaned MiniMax H3 R2V evaluation workflow. It is derived
|
||||
from the compact official MiniMax H3 int8 R2V graph and is kept separate from
|
||||
the imported source workflows under `models/minimax/`.
|
||||
This is the MiniMax H3 R2V evaluation workflow. It is derived from the
|
||||
imported MiniMax H3 int8 graph and kept separate from the source workflows
|
||||
under `models/minimax/`.
|
||||
|
||||
## Fixed variables
|
||||
## Fixed comparison inputs
|
||||
|
||||
- Reference media: `reference.png`, supplied to the workflow as Picture 1
|
||||
- Prompt: `prompt.txt`, fixed for this comparison batch
|
||||
- Seed: keep fixed for each comparison batch
|
||||
- Reference: `reference.png`, supplied as Picture 1
|
||||
- Prompt: `prompt.txt`
|
||||
- Seed: fixed within each comparison batch
|
||||
- Resolution: 864×480 (480p landscape)
|
||||
- Duration: 3 seconds for the initial efficiency matrix
|
||||
- Initial duration: 3 seconds
|
||||
- CFG: **1.0**
|
||||
- Model, text encoder, VAE, batch size, and denoise: fixed
|
||||
- Audio: none for the initial R2V matrix
|
||||
|
||||
## Base workflow
|
||||
Only the variables in the matrix may change. Before each row is run, confirm
|
||||
that the named node or sampler exists on the live server. Unsupported rows
|
||||
must be marked `Unsupported` with the exact validation error; do not silently
|
||||
substitute another configuration.
|
||||
|
||||
- `workflow.json` — derived ComfyUI web workflow
|
||||
- `reference.png` — fixed character reference for the initial batch
|
||||
## Files
|
||||
|
||||
- `workflow.json` — ComfyUI web workflow derived from the source graph
|
||||
- `reference.png` — fixed character reference
|
||||
- `prompt.txt` — fixed haunted-room investigation prompt
|
||||
- Source control graph: `models/minimax/minimax-h3-int8-r2v/workflow.json`
|
||||
- The imported source is not modified.
|
||||
- `variables.json` — fixed comparison variables
|
||||
- `results/` — labelled outputs and matching JSON metadata
|
||||
|
||||
## Efficiency test matrix
|
||||
## Test matrix
|
||||
|
||||
The first two tests compare a normal control against the first compatible
|
||||
memory-saving patch. All rows use the fixed variables above. The initial test
|
||||
does not vary steps, sampler, or scheduler so that the efficiency difference
|
||||
is isolated.
|
||||
All rows are planned; only the scheduled MM-R2V-01 and MM-R2V-02 control/
|
||||
memory tests should be run first. Later phases are gated on valid 3-second
|
||||
results and should not be queued automatically with Phase 1.
|
||||
|
||||
| ID | Memory path | Steps | Sampler | Scheduler | Duration | CFG | Runtime (s) | VRAM | Tested? | Valid? | Current Best |
|
||||
|---|---|---:|---|---|---:|---:|---:|---:|---|---|---|
|
||||
| MM-R2V-01 | Normal/control | 8 | workflow default | workflow default | 3 | 1.0 | — | — | No | — | No |
|
||||
| MM-R2V-02 | MiniMaxH3MemoryEfficientSageAttentionPatch | 8 | workflow default | workflow default | 3 | 1.0 | — | — | No | — | No |
|
||||
| ID | Phase | Efficiency / node combination | Steps | Sampler | Scheduler | Duration (s) | CFG | Tested? | Valid? | Current Best | Runtime (s) |
|
||||
|---|---|---|---:|---|---|---:|---:|---|---|---|---:|
|
||||
| MM-R2V-01 | A control | Base workflow | 8 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-02 | A memory | MemoryEfficientSageAttentionPatch | 8 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-03 | A memory | Model cache/offload path only | 8 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-04 | A attention | Sol-Attn only | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-05 | A attention | Spectrum only | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-06 | A LoRA | Turbo LoRA only | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-07 | A combined | Memory + Sol-Attn | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-08 | A combined | Memory + Spectrum | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-09 | A combined | Memory + Turbo LoRA | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-10 | A combined | Sol-Attn + Turbo LoRA | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-11 | A combined | Spectrum + Turbo LoRA | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-12 | A combined | Memory + Sol-Attn + Turbo LoRA | 4 | workflow default | workflow default | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-13 | B steps | Best valid Phase A path | 4 | best valid | best valid | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-14 | B steps | Best valid Phase A path | 6 | best valid | best valid | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-15 | B steps | Best valid Phase A path | 8 | best valid | best valid | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-16 | B steps | Best valid Phase A path | 12 | best valid | best valid | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-17 | B sampler | Best valid Phase A path | 6 | euler | simple | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-18 | B sampler | Best valid Phase A path | 6 | euler_cfg_pp | simple | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-19 | B sampler | Best valid Phase A path | 6 | res_multistep | simple | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-20 | B sampler | Best valid Phase A path | 6 | dpmpp_2m | simple | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-21 | B scheduler | Best valid Phase A path | 6 | best valid | beta | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-22 | B scheduler | Best valid Phase A path | 6 | best valid | sgm_uniform | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-23 | B scheduler | Best valid Phase A path | 6 | best valid | karras | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-24 | B balance | Best valid Phase A path | 8 | best valid | best valid | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-25 | C duration | Best 3-second path | best | best | best | 6 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-26 | C duration | Best 3-second path | best | best | best | 10 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-27 | C duration | Best 3-second path | best | best | best | 20 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-28 | D upscale | Best 3-second path + MiniMax latent upscale | best | best | best | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-29 | D upscale | Best 3-second path + RTX Super Resolution | best | best | best | 3 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-30 | E extend | Best valid path, concatenate two clips | best | best | best | 6 | 1.0 | No | — | No | — |
|
||||
| MM-R2V-31 | E extend | Best valid path, concatenate three clips | best | best | best | 9 | 1.0 | No | — | No | — |
|
||||
|
||||
## Planned expansion
|
||||
## Result requirements
|
||||
|
||||
After these two tests:
|
||||
Each run must save `results/<test-id>.mp4` and matching
|
||||
`results/<test-id>.json`. Metadata must include the exact reference checksum,
|
||||
prompt hash, seed, resolution, duration, CFG, steps, sampler, scheduler,
|
||||
active nodes, runtime, VRAM, output checksum, validity, and visual/motion
|
||||
verdict. A result becomes Current Best only when it is valid and objectively
|
||||
better than the existing best; keep exactly one Current Best.
|
||||
|
||||
1. Test other compatible memory nodes individually.
|
||||
2. Test safe combinations of compatible nodes.
|
||||
3. Sweep steps, sampler, and scheduler with the best efficiency path.
|
||||
4. Run only the strongest 3-second configurations at 20 seconds.
|
||||
5. Test reference-driven continuation/concatenation.
|
||||
6. Compare latent upscale and RTX Video Super Resolution toward 1080p.
|
||||
## Execution order
|
||||
|
||||
1. Run MM-R2V-01 and MM-R2V-02 as the initial paired efficiency test.
|
||||
2. Run Phase A one-node and safe-combination tests on 3 seconds.
|
||||
3. Optimize steps, sampler, scheduler, and quality at 3 seconds.
|
||||
4. Only then test 6/10/20-second clips.
|
||||
5. Test continuation/concatenation, then latent upscale and RTX Super Resolution.
|
||||
|
||||
Every result must be saved as `results/<test-id>.*` with matching JSON
|
||||
metadata containing the exact reference, prompt, seed, resolution, duration,
|
||||
CFG, node settings, runtime, VRAM, validity, and visual verdict.
|
||||
|
||||
@@ -5,6 +5,13 @@
|
||||
"resolution": [864, 480],
|
||||
"duration_seconds": 3,
|
||||
"cfg": 1,
|
||||
"test_phases": [
|
||||
"A: efficiency nodes and safe combinations",
|
||||
"B: steps, samplers, and schedulers",
|
||||
"C: duration extension",
|
||||
"D: latent upscale and RTX Super Resolution",
|
||||
"E: clip continuation and concatenation"
|
||||
],
|
||||
"model": "fixed by workflow",
|
||||
"text_encoder": "fixed by workflow",
|
||||
"vae": "fixed by workflow",
|
||||
|
||||
Reference in New Issue
Block a user