Files
comfyui-workflows/optimized/R2V/minimax/sparse-latent-upscale/README.md
T

6.3 KiB
Raw Blame History

MiniMax H3 sparse-attention latent-upscale study

This workflow variant is based on the supplied CSGlide H3 workflow. It uses:

  • fastvideo_fasth3_8step_v2_pruned_int8_convrot.safetensors
  • Block Sparse Attention with VSA selection
  • Comfy Kitchen attention, MiniMax low-VRAM attention and chunked feed-forward
  • 8-step Euler + beta initial generation
  • MiniMax latent upscaling with tiled spatial splitting and latent refinement
  • The supplied VAE, audio VAE, references, seed and one approximately 10-second clip

The workflow is preserved in workflow-source.json; workflow.json is the tracked test copy with the enhanced common prompt and corrected reference filenames.

For the visual ComfyUI editor, load workflow-ui.json. It is the canvas-format version of the same validated baseline, with node positions and links. The API-format workflow.json remains the file used by the automated submission scripts.

Fixed test input

  • References: references/Scene022.png as @image1, references/Char034_TurnaroundFull_001.png as @image2
  • Clip: 243 frames at 24 fps, approximately 10.125 seconds
  • Seed: 317182692766673 for the initial sampler; keep it unchanged in every test
  • Initial sampler: Euler, beta scheduler, 8 steps, denoise 1.0
  • Latent refinement: SA solver, simple scheduler, 3 steps, denoise 0.2, CFG 1
  • Prompt: prompt.txt; do not change it between tests

Test matrix

Every test changes only the listed variables. Record runtime, peak VRAM, success/failure, output dimensions, visible seams, detail retention, anatomy, motion stability and audio output in the matching result JSON.

Phase A — Upscale ceiling

ID Initial latent Target latent Tile Purpose
SLU-01 672×384 1344×768 supplied baseline tile Reproduce the supplied good result
SLU-02 672×384 1920×1080 supplied baseline tile Test the desired 1080p endpoint
SLU-03 672×384 1680×960 supplied baseline tile Intermediate fallback if 1080p fails or is too slow
SLU-04 672×384 1536×864 supplied baseline tile Practical quality/runtime fallback

Phase B — Tile size

Use the best successful target from Phase A and compare tile sizes while keeping overlap and all other settings fixed:

  • SLU-05: 512×512 tiles
  • SLU-06: 640×640 tiles
  • SLU-07: 768×768 tiles
  • SLU-08: 864×480-style wide tiles from the supplied workflow
  • SLU-09: full-frame/no split if VRAM permits

For each tile, preserve 128-pixel spatial overlap where possible, then check seam visibility, runtime and peak VRAM.

Phase C — Initial generation dimensions

Use the best target and tile combination:

  • SLU-10: 672×384 supplied baseline
  • SLU-11: 768×432
  • SLU-12: 832×480 supplied alternate resolution node
  • SLU-13: 960×544
  • SLU-14: 1024×576 if the 8GB card remains stable

The initial aspect ratio must remain 16:9 and the same final target must be used for fair comparison.

Phase D — Combined candidates

  • SLU-15: best upscale ceiling + best tile + best initial dimensions
  • SLU-16: quality-first combination, accepting a longer runtime if the visual improvement is clear
  • SLU-17: practical production combination targeting a predictable sub-hour render
  • SLU-18: rectangular 1024×640 tile shape with four effective tile passes
  • SLU-19: 96px spatial overlap on the accepted 1024×640 rectangular tile layout

First two runs

Run SLU-01 and SLU-02 first. SLU-01 validates the supplied configuration. SLU-02 directly tests whether 1080p is feasible on the RTX 3070. If SLU-02 exhausts VRAM, stalls or becomes impractically slow, stop that run and continue with SLU-03/04 rather than changing the prompt or seed.

Evidence and verdict

The test is VALIDATED only when a configuration completes with a usable video and recorded measurements. A completed render with unacceptable seams, detail loss or motion instability is PARTIAL, not a success. Keep all result JSON and MP4 files under results/.

Recorded valid runs

These are the valid runs recorded so far. Runtime is wall-clock generation time; the normalized figure is included because the output sizes differ.

ID Initial → output Tile Runtime Runtime per output megapixel Output Review
SLU-01 672×384 → 1344×768 864×480 20m 09s 19.51 min/Mpx 1344×768 Pending
SLU-02 672×384 → 1920×1088 864×480 32m 06s 15.37 min/Mpx 1920×1088 Baseline
SLU-05 672×384 → 1920×1088 512×512 44m 48s 21.45 min/Mpx 1920×1088 Valid; slower
SLU-06 672×384 → 1920×1088 640×640 26m 52s 12.88 min/Mpx 1920×1088 Reviewed; good
SLU-07 672×384 → 1920×1088 768×768 26m 01s 12.45 min/Mpx 1920×1088 Current best
SLU-08 672×384 → 1920×1088 864×480 31m 13s 14.93 min/Mpx 1920×1088 Valid; visual comparison pending
SLU-11 768×432 → 1920×1088 768×768 31m 07s 14.91 min/Mpx 1920×1088 Valid; review pending
SLU-18 672×384 → 1920×1088 1024×640 25m 14s 12.08 min/Mpx 1920×1088 Reviewed; new current best
SLU-19 672×384 → 1920×1088 1024×640, 96px overlap 18m 11s 8.71 min/Mpx 1920×1088 Reviewed; accepted current best

Current best

SLU-19 is now the current best baseline: 672×384 initial generation, 1024×640 tiles in a 2×2 layout, 96-pixel overlap and a 1920×1088 output. It was reviewed and accepted with good quality and no visible seams. At 18m 11s it is below the 20-minute target and has the best normalized throughput so far at 8.71 min/Mpx. The attention keep_percent 5 test was slower at 31m 05s, so the baseline value of 10 remains in use. SLU-18 remains the same tile shape with 128px overlap at 25m 14s.

SLU-09 full-frame/no-split failed with GPU out-of-memory on the 8 GB card and is not a viable baseline. The tracked workflow.json now uses the SLU-18 settings. Static prompt, seed, references, model, samplers, schedulers, steps, denoise and final target remain unchanged.

The earlier tile submissions that produced 1024×1024 output are excluded from this table because their target-resolution override was not applied. The current baseline uses 96px overlap; further reductions should only be attempted if a visible-seam comparison is worthwhile.