Files
comfyui-workflows/optimized/R2V/minimax/sparse-latent-upscale/README.md
T

119 lines
7.5 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# MiniMax H3 sparse-attention latent-upscale study
This workflow variant is based on the supplied CSGlide H3 workflow. It uses:
- `fastvideo_fasth3_8step_v2_pruned_int8_convrot.safetensors`
- Block Sparse Attention with VSA selection
- Comfy Kitchen attention, MiniMax low-VRAM attention and chunked feed-forward
- 8-step Euler + beta initial generation
- MiniMax latent upscaling with tiled spatial splitting and latent refinement
- The supplied VAE, audio VAE, references, seed and one approximately 10-second clip
The workflow is preserved in `workflow-source.json`; `workflow.json` is the tracked test copy with the enhanced common prompt and corrected reference filenames.
For the visual ComfyUI editor, load [`workflow-ui.json`](workflow-ui.json). It is the canvas-format version of the same validated baseline, with node positions and links. The API-format `workflow.json` remains the file used by the automated submission scripts.
After `CSGlideVideoCS` saves each clip, the workflow now passes its filepath
through `LayerUtility: PurgeVRAM V2` with both cache and model purging enabled.
This is a cleanup boundary between sequential clips: it does not alter the
filepath, frames or audio, but asks ComfyUI to release cached models and CUDA
memory before the next clip starts. It is intended to reduce retained VRAM
pressure; it cannot prevent an individual clip from exceeding the card's
capacity.
For sequential projects, node `253` also saves the newly generated clip by
itself as `dragon_clip_standalone_*.mp4`. Use this standalone output as the
continuation source for the next clip. Node `149` remains the cumulative Glide
Join output for final assembly, but must not be fed back into the next render;
doing so makes every later job re-encode an increasingly long film.
## Fixed test input
- **References:** `references/Scene022.png` as `@image1`, `references/Char034_TurnaroundFull_001.png` as `@image2`
- **Clip:** 243 frames at 24 fps, approximately 10.125 seconds
- **Seed:** `317182692766673` for the initial sampler; keep it unchanged in every test
- **Initial sampler:** Euler, beta scheduler, 8 steps, denoise 1.0
- **Latent refinement:** SA solver, simple scheduler, 3 steps, denoise 0.2, CFG 1
- **Prompt:** `prompt.txt`; do not change it between tests
## Test matrix
Every test changes only the listed variables. Record runtime, peak VRAM, success/failure, output dimensions, visible seams, detail retention, anatomy, motion stability and audio output in the matching result JSON.
### Phase A — Upscale ceiling
| ID | Initial latent | Target latent | Tile | Generation time (s) | Purpose |
|---|---:|---:|---|---:|---|
| SLU-01 | 672×384 | 1344×768 | supplied baseline tile | — | Reproduce the supplied good result |
| SLU-02 | 672×384 | 1920×1080 | supplied baseline tile | — | Test the desired 1080p endpoint |
| SLU-03 | 672×384 | 1680×960 | supplied baseline tile | — | Intermediate fallback if 1080p fails or is too slow |
| SLU-04 | 672×384 | 1536×864 | supplied baseline tile | — | Practical quality/runtime fallback |
### Phase B — Tile size
Use the best successful target from Phase A and compare tile sizes while keeping overlap and all other settings fixed:
- SLU-05: 512×512 tiles
- SLU-06: 640×640 tiles
- SLU-07: 768×768 tiles
- SLU-08: 864×480-style wide tiles from the supplied workflow
- SLU-09: full-frame/no split if VRAM permits
For each tile, preserve 128-pixel spatial overlap where possible, then check seam visibility, runtime and peak VRAM.
### Phase C — Initial generation dimensions
Use the best target and tile combination:
- SLU-10: 672×384 supplied baseline
- SLU-11: 768×432
- SLU-12: 832×480 supplied alternate resolution node
- SLU-13: 960×544
- SLU-14: 1024×576 if the 8GB card remains stable
The initial aspect ratio must remain 16:9 and the same final target must be used for fair comparison.
### Phase D — Combined candidates
- SLU-15: best upscale ceiling + best tile + best initial dimensions
- SLU-16: quality-first combination, accepting a longer runtime if the visual improvement is clear
- SLU-17: practical production combination targeting a predictable sub-hour render
- SLU-18: rectangular 1024×640 tile shape with four effective tile passes
- SLU-19: 96px spatial overlap on the accepted 1024×640 rectangular tile layout
## First two runs
Run **SLU-01** and **SLU-02** first. SLU-01 validates the supplied configuration. SLU-02 directly tests whether 1080p is feasible on the RTX 3070. If SLU-02 exhausts VRAM, stalls or becomes impractically slow, stop that run and continue with SLU-03/04 rather than changing the prompt or seed.
## Evidence and verdict
The test is **VALIDATED** only when a configuration completes with a usable video and recorded measurements. A completed render with unacceptable seams, detail loss or motion instability is **PARTIAL**, not a success. Keep all result JSON and MP4 files under `results/`.
## Recorded valid runs
These are the valid runs recorded so far. Runtime is wall-clock generation time; the normalized figure is included because the output sizes differ.
| ID | Initial → output | Tile | Runtime | Runtime per output megapixel | Output | Review |
|---|---|---:|---:|---:|---:|---|
| SLU-01 | 672×384 → 1344×768 | 864×480 | 20m 09s | 19.51 min/Mpx | 1344×768 | Pending |
| SLU-02 | 672×384 → 1920×1088 | 864×480 | 32m 06s | 15.37 min/Mpx | 1920×1088 | Baseline |
| SLU-05 | 672×384 → 1920×1088 | 512×512 | 44m 48s | 21.45 min/Mpx | 1920×1088 | Valid; slower |
| SLU-06 | 672×384 → 1920×1088 | 640×640 | 26m 52s | 12.88 min/Mpx | 1920×1088 | Reviewed; good |
| SLU-07 | 672×384 → 1920×1088 | 768×768 | 26m 01s | **12.45 min/Mpx** | 1920×1088 | **Current best** |
| SLU-08 | 672×384 → 1920×1088 | 864×480 | 31m 13s | 14.93 min/Mpx | 1920×1088 | Valid; visual comparison pending |
| SLU-11 | 768×432 → 1920×1088 | 768×768 | 31m 07s | 14.91 min/Mpx | 1920×1088 | Valid; review pending |
| SLU-18 | 672×384 → 1920×1088 | 1024×640 | **25m 14s** | **12.08 min/Mpx** | 1920×1088 | **Reviewed; new current best** |
| SLU-19 | 672×384 → 1920×1088 | 1024×640, 96px overlap | **18m 11s** | **8.71 min/Mpx** | 1920×1088 | **Reviewed; accepted current best** |
### Current best
**SLU-19 remains the fastest validated single-clip baseline**: 672×384 initial generation, 1024×640 tiles in a 2×2 layout, 96-pixel overlap and a 1920×1088 output. It was reviewed and accepted with good quality and no visible seams. At 18m 11s it is below the 20-minute target and has the best normalized throughput so far at 8.71 min/Mpx.
The attention `keep_percent 5` test was slower at 31m 05s, so the baseline value of 10 remains in use.
**Pinned continuation setting:** use 768×768 tiles in a 2×3 layout with 96px overlap for sequential continuation clips. The 1024×640 layout is not stable with continuation context on the RTX 3070; 768×768 is the accepted stable setting for multi-clip work. Keep the 1024×640 result as a single-clip performance record only.
SLU-09 full-frame/no-split failed with GPU out-of-memory on the 8 GB card and is not a viable baseline. The tracked `workflow.json` uses the pinned 768×768 continuation settings. Static prompt, seed, references, model, samplers, schedulers, steps, denoise and final target remain unchanged.
The earlier tile submissions that produced 1024×1024 output are excluded from this table because their target-resolution override was not applied. The current baseline uses 96px overlap; further reductions should only be attempted if a visible-seam comparison is worthwhile.