119 lines
7.5 KiB
Markdown
119 lines
7.5 KiB
Markdown
# MiniMax H3 sparse-attention latent-upscale study
|
||
|
||
This workflow variant is based on the supplied CSGlide H3 workflow. It uses:
|
||
|
||
- `fastvideo_fasth3_8step_v2_pruned_int8_convrot.safetensors`
|
||
- Block Sparse Attention with VSA selection
|
||
- Comfy Kitchen attention, MiniMax low-VRAM attention and chunked feed-forward
|
||
- 8-step Euler + beta initial generation
|
||
- MiniMax latent upscaling with tiled spatial splitting and latent refinement
|
||
- The supplied VAE, audio VAE, references, seed and one approximately 10-second clip
|
||
|
||
The workflow is preserved in `workflow-source.json`; `workflow.json` is the tracked test copy with the enhanced common prompt and corrected reference filenames.
|
||
|
||
For the visual ComfyUI editor, load [`workflow-ui.json`](workflow-ui.json). It is the canvas-format version of the same validated baseline, with node positions and links. The API-format `workflow.json` remains the file used by the automated submission scripts.
|
||
|
||
After `CSGlideVideoCS` saves each clip, the workflow now passes its filepath
|
||
through `LayerUtility: PurgeVRAM V2` with both cache and model purging enabled.
|
||
This is a cleanup boundary between sequential clips: it does not alter the
|
||
filepath, frames or audio, but asks ComfyUI to release cached models and CUDA
|
||
memory before the next clip starts. It is intended to reduce retained VRAM
|
||
pressure; it cannot prevent an individual clip from exceeding the card's
|
||
capacity.
|
||
|
||
For sequential projects, node `253` also saves the newly generated clip by
|
||
itself as `dragon_clip_standalone_*.mp4`. Use this standalone output as the
|
||
continuation source for the next clip. Node `149` remains the cumulative Glide
|
||
Join output for final assembly, but must not be fed back into the next render;
|
||
doing so makes every later job re-encode an increasingly long film.
|
||
|
||
## Fixed test input
|
||
|
||
- **References:** `references/Scene022.png` as `@image1`, `references/Char034_TurnaroundFull_001.png` as `@image2`
|
||
- **Clip:** 243 frames at 24 fps, approximately 10.125 seconds
|
||
- **Seed:** `317182692766673` for the initial sampler; keep it unchanged in every test
|
||
- **Initial sampler:** Euler, beta scheduler, 8 steps, denoise 1.0
|
||
- **Latent refinement:** SA solver, simple scheduler, 3 steps, denoise 0.2, CFG 1
|
||
- **Prompt:** `prompt.txt`; do not change it between tests
|
||
|
||
## Test matrix
|
||
|
||
Every test changes only the listed variables. Record runtime, peak VRAM, success/failure, output dimensions, visible seams, detail retention, anatomy, motion stability and audio output in the matching result JSON.
|
||
|
||
### Phase A — Upscale ceiling
|
||
|
||
| ID | Initial latent | Target latent | Tile | Generation time (s) | Purpose |
|
||
|---|---:|---:|---|---:|---|
|
||
| SLU-01 | 672×384 | 1344×768 | supplied baseline tile | — | Reproduce the supplied good result |
|
||
| SLU-02 | 672×384 | 1920×1080 | supplied baseline tile | — | Test the desired 1080p endpoint |
|
||
| SLU-03 | 672×384 | 1680×960 | supplied baseline tile | — | Intermediate fallback if 1080p fails or is too slow |
|
||
| SLU-04 | 672×384 | 1536×864 | supplied baseline tile | — | Practical quality/runtime fallback |
|
||
|
||
### Phase B — Tile size
|
||
|
||
Use the best successful target from Phase A and compare tile sizes while keeping overlap and all other settings fixed:
|
||
|
||
- SLU-05: 512×512 tiles
|
||
- SLU-06: 640×640 tiles
|
||
- SLU-07: 768×768 tiles
|
||
- SLU-08: 864×480-style wide tiles from the supplied workflow
|
||
- SLU-09: full-frame/no split if VRAM permits
|
||
|
||
For each tile, preserve 128-pixel spatial overlap where possible, then check seam visibility, runtime and peak VRAM.
|
||
|
||
### Phase C — Initial generation dimensions
|
||
|
||
Use the best target and tile combination:
|
||
|
||
- SLU-10: 672×384 supplied baseline
|
||
- SLU-11: 768×432
|
||
- SLU-12: 832×480 supplied alternate resolution node
|
||
- SLU-13: 960×544
|
||
- SLU-14: 1024×576 if the 8GB card remains stable
|
||
|
||
The initial aspect ratio must remain 16:9 and the same final target must be used for fair comparison.
|
||
|
||
### Phase D — Combined candidates
|
||
|
||
- SLU-15: best upscale ceiling + best tile + best initial dimensions
|
||
- SLU-16: quality-first combination, accepting a longer runtime if the visual improvement is clear
|
||
- SLU-17: practical production combination targeting a predictable sub-hour render
|
||
- SLU-18: rectangular 1024×640 tile shape with four effective tile passes
|
||
- SLU-19: 96px spatial overlap on the accepted 1024×640 rectangular tile layout
|
||
|
||
## First two runs
|
||
|
||
Run **SLU-01** and **SLU-02** first. SLU-01 validates the supplied configuration. SLU-02 directly tests whether 1080p is feasible on the RTX 3070. If SLU-02 exhausts VRAM, stalls or becomes impractically slow, stop that run and continue with SLU-03/04 rather than changing the prompt or seed.
|
||
|
||
## Evidence and verdict
|
||
|
||
The test is **VALIDATED** only when a configuration completes with a usable video and recorded measurements. A completed render with unacceptable seams, detail loss or motion instability is **PARTIAL**, not a success. Keep all result JSON and MP4 files under `results/`.
|
||
|
||
## Recorded valid runs
|
||
|
||
These are the valid runs recorded so far. Runtime is wall-clock generation time; the normalized figure is included because the output sizes differ.
|
||
|
||
| ID | Initial → output | Tile | Runtime | Runtime per output megapixel | Output | Review |
|
||
|---|---|---:|---:|---:|---:|---|
|
||
| SLU-01 | 672×384 → 1344×768 | 864×480 | 20m 09s | 19.51 min/Mpx | 1344×768 | Pending |
|
||
| SLU-02 | 672×384 → 1920×1088 | 864×480 | 32m 06s | 15.37 min/Mpx | 1920×1088 | Baseline |
|
||
| SLU-05 | 672×384 → 1920×1088 | 512×512 | 44m 48s | 21.45 min/Mpx | 1920×1088 | Valid; slower |
|
||
| SLU-06 | 672×384 → 1920×1088 | 640×640 | 26m 52s | 12.88 min/Mpx | 1920×1088 | Reviewed; good |
|
||
| SLU-07 | 672×384 → 1920×1088 | 768×768 | 26m 01s | **12.45 min/Mpx** | 1920×1088 | **Current best** |
|
||
| SLU-08 | 672×384 → 1920×1088 | 864×480 | 31m 13s | 14.93 min/Mpx | 1920×1088 | Valid; visual comparison pending |
|
||
| SLU-11 | 768×432 → 1920×1088 | 768×768 | 31m 07s | 14.91 min/Mpx | 1920×1088 | Valid; review pending |
|
||
| SLU-18 | 672×384 → 1920×1088 | 1024×640 | **25m 14s** | **12.08 min/Mpx** | 1920×1088 | **Reviewed; new current best** |
|
||
| SLU-19 | 672×384 → 1920×1088 | 1024×640, 96px overlap | **18m 11s** | **8.71 min/Mpx** | 1920×1088 | **Reviewed; accepted current best** |
|
||
|
||
### Current best
|
||
|
||
**SLU-19 remains the fastest validated single-clip baseline**: 672×384 initial generation, 1024×640 tiles in a 2×2 layout, 96-pixel overlap and a 1920×1088 output. It was reviewed and accepted with good quality and no visible seams. At 18m 11s it is below the 20-minute target and has the best normalized throughput so far at 8.71 min/Mpx.
|
||
|
||
The attention `keep_percent 5` test was slower at 31m 05s, so the baseline value of 10 remains in use.
|
||
|
||
**Pinned continuation setting:** use 768×768 tiles in a 2×3 layout with 96px overlap for sequential continuation clips. The 1024×640 layout is not stable with continuation context on the RTX 3070; 768×768 is the accepted stable setting for multi-clip work. Keep the 1024×640 result as a single-clip performance record only.
|
||
|
||
SLU-09 full-frame/no-split failed with GPU out-of-memory on the 8 GB card and is not a viable baseline. The tracked `workflow.json` uses the pinned 768×768 continuation settings. Static prompt, seed, references, model, samplers, schedulers, steps, denoise and final target remain unchanged.
|
||
|
||
The earlier tile submissions that produced 1024×1024 output are excluded from this table because their target-resolution override was not applied. The current baseline uses 96px overlap; further reductions should only be attempted if a visible-seam comparison is worthwhile.
|