Compare commits

...
30 Commits
Author SHA1 Message Date
chris.dumas e8599055a1 Force temporal chunks for CUDA latent upscale 2026-09-04 10:22:41 +00:00
chris.dumas 618a48e4d9 Harden latent upscale GPU cleanup 2026-09-04 10:05:24 +00:00
chris.dumas cbabcf8208 Offload H3 before latent upscale 2026-09-04 09:49:34 +00:00
chris.dumas fbfbe4ef5a Extend H3 spatial fallback ladder 2026-09-04 09:22:35 +00:00
chris.dumas 5b52793fad Clamp H3 spatial overlaps to tile size 2026-09-04 09:05:29 +00:00
chris.dumas 7e45c81589 Reset overlap in H3 spatial retries 2026-09-04 08:48:20 +00:00
chris.dumas 027865b9ca Tighten H3 spatial fallback retries 2026-09-04 08:27:54 +00:00
chris.dumas 9f0c546681 Remove latent upscale CPU fallback 2026-09-04 08:13:27 +00:00
chris.dumas 30ae81b86b Jump latent upscale fallback to spatial split 2026-09-04 08:00:06 +00:00
chris.dumas 179fa0778e Add CPU fallback for latent upscale OOM 2026-09-04 07:33:10 +00:00
chris.dumas a34c9eb4ff Reduce latent upscale VRAM pressure 2026-09-04 07:21:38 +00:00
chris.dumas 782a7d658b Fix temporal chunking to use H3 token grid 2026-09-04 07:06:08 +00:00
chris.dumas 04a61af874 Add temporal OOM backoff for latent upscale 2026-09-04 06:54:35 +00:00
chris.dumas a0d80bcc47 Add temporal chunking to latent upscale 2026-09-04 06:41:03 +00:00
chris.dumas fbb5f799cd Unload latent upscale model on OOM 2026-09-03 20:30:58 +00:00
chris.dumas 9dc3c405a6 Clamp latent upscale fallback tile minimum 2026-09-03 20:15:58 +00:00
chris.dumas 21ae4d62e0 Keep latent upscale shrink steps 32-aligned 2026-09-03 19:59:50 +00:00
chris.dumas 836a6eba33 Back off latent upscale tile size on OOM 2026-09-03 19:43:15 +00:00
chris.dumas 3b63d33ec7 Tile latent upscale model inference 2026-09-03 19:01:13 +00:00
chris.dumas c62921c3e8 Stop retrying latent upscale OOMs as sampling 2026-09-03 18:46:56 +00:00
chris.dumas 3ab6342ce5 Handle legacy latent upscale payloads 2026-09-03 18:33:01 +00:00
chris.dumas 51e03a39b7 Reduce latent upscale branch memory 2026-09-03 18:08:44 +00:00
chris.dumas f7b94ccaef Free first-pass latent before refinement 2026-09-03 16:34:12 +00:00
chris.dumas f6120a8500 Adopt full MMH3 spatial split controls 2026-09-03 16:15:43 +00:00
chris.dumas 0e119646ab Add remaining spatial split settings 2026-09-03 15:47:59 +00:00
chris.dumas c89570eae9 Expand latent upscale spatial stitch controls 2026-09-03 15:35:39 +00:00
chris.dumas 36d9f4369c Add spatial batching to latent upscale 2026-09-03 15:12:42 +00:00
chris.dumas 2cb3c694f0 Fix latent upscale summary scope leak 2026-09-03 14:37:24 +00:00
chris.dumas 5c12cd18a3 Expose latent upscale sampler controls 2026-09-03 13:05:32 +00:00
chris.dumas c1d937e0e2 Add H3 latent upscale refinement stage 2026-09-03 12:53:57 +00:00
10 changed files with 2021 additions and 324 deletions
+27 -66
View File
@@ -1149,81 +1149,42 @@ These two belong together.
If you change `shift_video`, you usually need to change `shift_audio` in proportion. If you change `shift_video`, you usually need to change `shift_audio` in proportion.
## Group 8: Detail Pass ## Group 8: Latent Upscale
This is the optional second pass. This is the optional latent refinement stage, used before decode.
### `detail_pass` The long-video node now expects a separate `Dumas H3 Latent Upscale Params` node for this stage.
Wire that node into the `latent_upscale_param` input when you want the shot to be upscaled and lightly
Enables the refinement pass. re-sampled before decode.
What this really means: What this really means:
- the node renders the beat once - the node renders the beat once
- then runs a second sampler pass over that result - the sampled latent is upscaled in latent space to the target size
- the goal is to polish, not to invent a whole different shot - the conditioning is rebuilt at that target size
- the node then runs a short refinement pass over the upscaled latent with the sampler, scheduler, step count, denoise, and megapixel target you picked on the latent-upscale params node
### `detail_sampler_name` - if the target is larger than the spatial tile size, that refinement pass is processed in spatial batches using the same tile defaults as the upstream latent-split node
- the spatial stitch mode follows the upstream overlap controls, including `linear`, `smoothstep`, `overwrite`, and `midpoint`
Sampler for the refinement pass. - the node also carries the upstream split compatibility knobs (`chunk_length`, `temporal_overlap`, `resize_conditioning`, and `anchor_strength`) so the control surface stays in one place
### `detail_scheduler`
Scheduler for the refinement pass.
### `detail_steps`
Extra steps for the refinement pass.
What this really means:
- more steps gives the second pass more opportunity to change the image
- that can help detail
- but after a point it stops being "cleanup" and starts becoming "rewrite"
### `detail_denoise`
How strongly the refinement pass is allowed to rewrite the beat.
What this really means:
- low denoise = polish what is already there
- high denoise = let the second pass substantially alter what is already there
### How The Detail-Pass Settings Work Together
The detail pass starts from the first-pass result and tries to polish it.
Gentle settings:
- low to medium `detail_steps`
- low `detail_denoise`
Aggressive settings:
- high `detail_steps`
- high `detail_denoise`
Aggressive settings can improve texture, but they can also:
- change faces
- pull away from references
- break continuity
That is why this group should be read as one combined strength control:
- `detail_pass` decides whether the second pass exists
- `detail_steps` decides how long it keeps working
- `detail_denoise` decides how free it is to change things
- `detail_sampler_name` and `detail_scheduler` shape how that rewrite behaves
Good starting point: Good starting point:
- `detail_pass = on` - use the `model` mode when you want the strongest latent detail recovery
- `detail_sampler_name = euler` - use the interpolation mode when you want a cheaper resize-only path
- `detail_scheduler = beta` - start with `euler_ancestral`, `simple`, `2` steps, and `0.2` denoise
- `detail_steps = 4` to `8` - leave width and height at `0` unless you want an exact override; otherwise `megapixels` drives the target size
- `detail_denoise = 0.20` to `0.35` - for long shots on smaller cards, start with `chunk_length = 85` and `temporal_overlap = 17` so the latent upscaler works in shorter temporal passes
- leave the spatial tile inputs at their defaults first: `512x512` tiles, `64` overlap, `0` fade width, `earlier` overlap mode
- keep `linear` blend first unless you want to reproduce a specific upstream stitch style
- leave the split compatibility knobs alone unless you specifically need to mirror the upstream node behavior
The important part is that this stage is still a latent pass, not a pixel-space resize:
- it happens before decode
- it can change structure more than a normal image upscale
- it is the place to recover detail without adding another full detail-pass toggle
If you do not wire the helper node, the long-video node skips latent upscale entirely and renders as before.
## Group 9: Performance, Decode, And Upscale ## Group 9: Performance, Decode, And Upscale
+6
View File
@@ -43,8 +43,14 @@
- Prompt `<Picture N>` tags now map to the actual ref socket numbers you wire, even with gaps such as only `ref_2` and `ref_7` connected. - Prompt `<Picture N>` tags now map to the actual ref socket numbers you wire, even with gaps such as only `ref_2` and `ref_7` connected.
- Character refs now contribute appearance and wardrobe context from the same structured object, while location refs contribute environment context from theirs. - Character refs now contribute appearance and wardrobe context from the same structured object, while location refs contribute environment context from theirs.
- The default ref2v bias is now stronger: `ref_mode` defaults to `auto ref2v` so untagged prompts condition every shot instead of only shot 1, and `ref_noise_aug` defaults to `0.95` rather than the upstream-literal `0.999`. - The default ref2v bias is now stronger: `ref_mode` defaults to `auto ref2v` so untagged prompts condition every shot instead of only shot 1, and `ref_noise_aug` defaults to `0.95` rather than the upstream-literal `0.999`.
- `Dumas H3 Latent Upscale Params` provides the optional pre-decode latent refinement stage for the long-video node.
- Per-shot directives now support `continuity:`, `ref_mode:`, `ref_noise_aug:`, `anchor_add:`, `soundscape:`, and `music:` in addition to the existing timing and wardrobe directives. - Per-shot directives now support `continuity:`, `ref_mode:`, `ref_noise_aug:`, `anchor_add:`, `soundscape:`, and `music:` in addition to the existing timing and wardrobe directives.
- `Dumas H3 Latent Upscale Params`
- Inputs: `mode`, `model_name`, `method`, `width`, `height`, `device`, `precision`, `sampler_name`, `scheduler`, `steps`, `denoise`, `megapixels`, `tile_width`, `tile_height`, `overlap`, `fade_width`, `fade_height`, `overlap_mode`, `overlap_blend`, `tile_size_mode`, `grid_rows`, `grid_cols`, `spatial_w_overlap`, `spatial_h_overlap`, `min_tile_size`, `masked_area_noise`, `brightness_match`, `dynamic_fade`, `dynamic_fade_min`, `chunk_length`, `temporal_overlap`, `resize_conditioning`, `anchor_strength`
- Output: `latent_upscale_param`
- Bundles the optional latent-space upscaler settings used by `Dumas H3 Long Videos` before decode, so the main node can rebuild conditioning at the target size and run a short refinement pass with your chosen sampler, scheduler, step count, denoise, and the full upstream spatial split controls.
- `Dumas H3 Beat Prompt` - `Dumas H3 Beat Prompt`
- Inputs: authored through the custom front-end beat editor - Inputs: authored through the custom front-end beat editor
- Output: `prompt` - Output: `prompt`
+6
View File
@@ -14,6 +14,10 @@ from .dumas_h3_longvideos import (
NODE_CLASS_MAPPINGS as H3_LONGVIDEO_NODE_CLASS_MAPPINGS, NODE_CLASS_MAPPINGS as H3_LONGVIDEO_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as H3_LONGVIDEO_NODE_DISPLAY_NAME_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS as H3_LONGVIDEO_NODE_DISPLAY_NAME_MAPPINGS,
) )
from .dumas_h3_latent_upscale import (
NODE_CLASS_MAPPINGS as H3_LATENT_UPSCALE_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as H3_LATENT_UPSCALE_NODE_DISPLAY_NAME_MAPPINGS,
)
from .dumas_h3_shot_length import ( from .dumas_h3_shot_length import (
NODE_CLASS_MAPPINGS as H3_SHOT_LENGTH_NODE_CLASS_MAPPINGS, NODE_CLASS_MAPPINGS as H3_SHOT_LENGTH_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as H3_SHOT_LENGTH_NODE_DISPLAY_NAME_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS as H3_SHOT_LENGTH_NODE_DISPLAY_NAME_MAPPINGS,
@@ -31,6 +35,7 @@ NODE_CLASS_MAPPINGS = {}
NODE_CLASS_MAPPINGS.update(JSON_NODE_CLASS_MAPPINGS) NODE_CLASS_MAPPINGS.update(JSON_NODE_CLASS_MAPPINGS)
NODE_CLASS_MAPPINGS.update(IMAGE_NODE_CLASS_MAPPINGS) NODE_CLASS_MAPPINGS.update(IMAGE_NODE_CLASS_MAPPINGS)
NODE_CLASS_MAPPINGS.update(H3_LONGVIDEO_NODE_CLASS_MAPPINGS) NODE_CLASS_MAPPINGS.update(H3_LONGVIDEO_NODE_CLASS_MAPPINGS)
NODE_CLASS_MAPPINGS.update(H3_LATENT_UPSCALE_NODE_CLASS_MAPPINGS)
NODE_CLASS_MAPPINGS.update(H3_SHOT_LENGTH_NODE_CLASS_MAPPINGS) NODE_CLASS_MAPPINGS.update(H3_SHOT_LENGTH_NODE_CLASS_MAPPINGS)
NODE_CLASS_MAPPINGS.update(H3_INSPECTOR_NODE_CLASS_MAPPINGS) NODE_CLASS_MAPPINGS.update(H3_INSPECTOR_NODE_CLASS_MAPPINGS)
NODE_CLASS_MAPPINGS.update(H3_BEAT_PROMPT_NODE_CLASS_MAPPINGS) NODE_CLASS_MAPPINGS.update(H3_BEAT_PROMPT_NODE_CLASS_MAPPINGS)
@@ -39,6 +44,7 @@ NODE_DISPLAY_NAME_MAPPINGS = {}
NODE_DISPLAY_NAME_MAPPINGS.update(JSON_NODE_DISPLAY_NAME_MAPPINGS) NODE_DISPLAY_NAME_MAPPINGS.update(JSON_NODE_DISPLAY_NAME_MAPPINGS)
NODE_DISPLAY_NAME_MAPPINGS.update(IMAGE_NODE_DISPLAY_NAME_MAPPINGS) NODE_DISPLAY_NAME_MAPPINGS.update(IMAGE_NODE_DISPLAY_NAME_MAPPINGS)
NODE_DISPLAY_NAME_MAPPINGS.update(H3_LONGVIDEO_NODE_DISPLAY_NAME_MAPPINGS) NODE_DISPLAY_NAME_MAPPINGS.update(H3_LONGVIDEO_NODE_DISPLAY_NAME_MAPPINGS)
NODE_DISPLAY_NAME_MAPPINGS.update(H3_LATENT_UPSCALE_NODE_DISPLAY_NAME_MAPPINGS)
NODE_DISPLAY_NAME_MAPPINGS.update(H3_SHOT_LENGTH_NODE_DISPLAY_NAME_MAPPINGS) NODE_DISPLAY_NAME_MAPPINGS.update(H3_SHOT_LENGTH_NODE_DISPLAY_NAME_MAPPINGS)
NODE_DISPLAY_NAME_MAPPINGS.update(H3_INSPECTOR_NODE_DISPLAY_NAME_MAPPINGS) NODE_DISPLAY_NAME_MAPPINGS.update(H3_INSPECTOR_NODE_DISPLAY_NAME_MAPPINGS)
NODE_DISPLAY_NAME_MAPPINGS.update(H3_BEAT_PROMPT_NODE_DISPLAY_NAME_MAPPINGS) NODE_DISPLAY_NAME_MAPPINGS.update(H3_BEAT_PROMPT_NODE_DISPLAY_NAME_MAPPINGS)
+10 -1
View File
@@ -150,7 +150,16 @@ class H3ModelInspector:
@classmethod @classmethod
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return {"required": {"model": ("MODEL",)}} return {
"required": {
"model": (
"MODEL",
{
"tooltip": "MiniMax / H3 model to inspect for quantization and tensor format."
},
)
}
}
def inspect(self, model): def inspect(self, model):
label, _counts, report = _detect(model) label, _counts, report = _detect(model)
File diff suppressed because it is too large Load Diff
+517 -68
View File
@@ -32,9 +32,10 @@ and your VRAM, chains them, and returns the finished video + audio.
Requirements: H3 is CFG-free (cfg 1) and needs no negative prompt -- the node Requirements: H3 is CFG-free (cfg 1) and needs no negative prompt -- the node
makes an empty one internally. The main pass keeps denoise fixed at 1.0: a makes an empty one internally. The main pass keeps denoise fixed at 1.0: a
partial denoise desyncs the joint audio/video schedule. An optional refinement partial denoise desyncs the joint audio/video schedule. An optional latent
pass can use its own denoise later, before any upscale, while keeping the output upscale stage can rebuild the conditioning at a target size, run a short
video-only. refinement pass with its own sampler controls, and keep the output video-only
before the final pixel-space upscale options.
Verified against ComfyUI core (comfy_extras/nodes_minimax_h3.py, model_base.py, Verified against ComfyUI core (comfy_extras/nodes_minimax_h3.py, model_base.py,
ldm/minimax/model.py, text_encoders/minimax.py, sd.py). ldm/minimax/model.py, text_encoders/minimax.py, sd.py).
@@ -59,6 +60,7 @@ import node_helpers
try: try:
from . import dumas_h3_overlay as _overlay from . import dumas_h3_overlay as _overlay
from .dumas_h3_latent_upscale import upscale_latent_video as _upscale_latent_video
from . import dumas_image_nodes as _image_nodes from . import dumas_image_nodes as _image_nodes
except ImportError: # loaded as a bare file (test_prompt_logic.py), not as a package except ImportError: # loaded as a bare file (test_prompt_logic.py), not as a package
import importlib.util as _ilu import importlib.util as _ilu
@@ -70,6 +72,13 @@ except ImportError: # loaded as a bare file (test_prompt_logic.py), not as
) )
_overlay = _ilu.module_from_spec(_spec) _overlay = _ilu.module_from_spec(_spec)
_spec.loader.exec_module(_overlay) _spec.loader.exec_module(_overlay)
_latent_spec = _ilu.spec_from_file_location(
"dumas_h3_latent_upscale",
_os.path.join(_os.path.dirname(_os.path.abspath(__file__)), "dumas_h3_latent_upscale.py"),
)
_latent_upscale = _ilu.module_from_spec(_latent_spec)
_latent_spec.loader.exec_module(_latent_upscale)
_upscale_latent_video = _latent_upscale.upscale_latent_video
_image_nodes = _sys.modules.get("dumas_image_nodes") _image_nodes = _sys.modules.get("dumas_image_nodes")
if _image_nodes is None: if _image_nodes is None:
_img_spec = _ilu.spec_from_file_location( _img_spec = _ilu.spec_from_file_location(
@@ -279,8 +288,6 @@ ADDED_WIDGETS = (
"exposed_terms", "anatomy_guard", "lock_restraints", "solidity_guard", "exposed_terms", "anatomy_guard", "lock_restraints", "solidity_guard",
"motion_guard", "contact_guard", "motion_guard", "contact_guard",
"auto_soundscape", "allow_nonspeech_vocals", "auto_soundscape", "allow_nonspeech_vocals",
"detail_pass", "detail_sampler_name", "detail_scheduler",
"detail_steps", "detail_denoise",
) )
NL = "\n" NL = "\n"
@@ -4066,6 +4073,237 @@ def _video_only_refined_latent(base_latent, refined_latent):
return refined_latent return refined_latent
def _latent_upscale_target_size(base_w, base_h, param):
width = int(param.get("width", 0) or 0)
height = int(param.get("height", 0) or 0)
if width > 0 and height > 0:
return width, height
megapixels = float(param.get("megapixels", 0.0) or 0.0)
if megapixels > 0:
return scale_to_megapixels(base_w, base_h, megapixels)
return int(base_w), int(base_h)
def _latent_upscale_mode(param):
if not isinstance(param, dict):
return "off"
mode = str(param.get("mode") or "").strip()
if mode:
return mode
if "model_name" in param and str(param.get("model_name") or "").strip() not in ("", "none"):
return "model"
if "method" in param:
return "interp"
return "off"
def _latent_spatial_grid(h, w, th, tw, ol_h, ol_w):
if th <= 0 or tw <= 0:
raise ValueError("tile dimensions must be positive")
if ol_h >= th or ol_w >= tw:
raise ValueError("overlap must be smaller than the tile size")
sh = th - ol_h
sw = tw - ol_w
nrows = 1 if h <= th else math.ceil((h - ol_h) / sh)
if (nrows - 1) * sh + th < h:
nrows += 1
ncols = 1 if w <= tw else math.ceil((w - ol_w) / sw)
if (ncols - 1) * sw + tw < w:
ncols += 1
rows = [i * sh for i in range(nrows)]
cols = [j * sw for j in range(ncols)]
trows = [min(th, h - r) for r in rows]
tcols = [min(tw, w - c) for c in cols]
return rows, cols, trows, tcols
def _latent_spatial_blend_weights(t, overlap_mode, overlap_blend="linear"):
if overlap_blend == "overwrite":
return torch.ones_like(t) if overlap_mode == "later" else torch.zeros_like(t)
if overlap_blend == "midpoint":
base = (t >= 0.5).to(t.dtype)
elif overlap_blend == "smoothstep":
base = t * t * (3.0 - 2.0 * t)
else:
base = t
if overlap_mode == "later":
return base
return 1.0 - base
def _param_value(mapping, key, default):
value = mapping.get(key, default)
return default if value is None else value
def _tag_oom_stage(exc, stage):
if _is_oom(exc):
exc._h3_stage = stage
return exc
def _grid_1d(size, tile, ol, min_tile):
if size <= tile:
return [0], [size], [0]
sh = tile - ol
n = math.ceil((size - ol) / sh)
if (n - 1) * sh + tile < size:
n += 1
rows = [i * sh for i in range(n)]
trows = [min(tile, size - r) for r in rows]
if min_tile > 0 and n >= 2:
edge = size - rows[-1]
if edge < min_tile:
new_last = size - min_tile
if rows[-2] < new_last < rows[-2] + trows[-2]:
rows[-1] = new_last
trows[-1] = size - new_last
ovl = [0] * n
for i in range(1, n):
ovl[i] = max(0, rows[i - 1] + trows[i - 1] - rows[i])
return rows, trows, ovl
def compute_spatial_grid(h, w, th, tw, ol_h, ol_w, min_th=0, min_tw=0):
if th <= 0 or tw <= 0:
raise ValueError("tile dimensions must be positive")
if ol_h >= th or ol_w >= tw:
raise ValueError("overlap must be smaller than the tile size")
if min_th < 0 or min_tw < 0:
raise ValueError("minimum tile size must be non-negative")
if min_th > th or min_tw > tw:
raise ValueError("minimum tile size must not exceed the tile size")
rows, trows, row_ovl = _grid_1d(h, th, ol_h, min_th)
cols, tcols, col_ovl = _grid_1d(w, tw, ol_w, min_tw)
return rows, cols, trows, tcols, row_ovl, col_ovl
def spatial_fade_mask(tile_h, tile_w, ol_h, ol_w, done_top, done_left, fade_h=0, fade_w=0):
mask = torch.ones(tile_h, tile_w, dtype=torch.float32)
if done_left and ol_w > 0:
if fade_w == 0:
mask[:, :ol_w] = 0.0
else:
f = min(fade_w, ol_w)
frozen_w = ol_w - f
w = torch.linspace(0.0, 1.0, f)
mask[:, :frozen_w] = 0.0
mask[:, frozen_w:ol_w] = torch.minimum(mask[:, frozen_w:ol_w], w[None, :])
if done_top and ol_h > 0:
if fade_h == 0:
mask[:ol_h, :] = 0.0
else:
f = min(fade_h, ol_h)
frozen_h = ol_h - f
w = torch.linspace(0.0, 1.0, f)
mask[:frozen_h, :] = 0.0
mask[frozen_h:ol_h, :] = torch.minimum(mask[frozen_h:ol_h, :], w[:, None])
return mask
def _fade_band(band, fade, axis):
n = band.shape[axis]
f = min(int(fade), n)
if f == 0:
band[:] = 0.0
return
w = torch.linspace(0.0, 1.0, f, dtype=band.dtype, device=band.device)
frozen = n - f
if axis == 1:
w = w[None, :]
band[:, :frozen] = torch.minimum(
band[:, :frozen], torch.zeros(frozen, dtype=band.dtype, device=band.device)
)
band[:, frozen:] = torch.minimum(band[:, frozen:], w)
else:
w = w[:, None]
band[:frozen, :] = torch.minimum(
band[:frozen, :], torch.zeros((frozen, 1), dtype=band.dtype, device=band.device)
)
band[frozen:, :] = torch.minimum(band[frozen:, :], w)
def make_fade_mask(tile_h, tile_w, ol_h, ol_w, done_top, done_left, fade_h=0, fade_w=0):
mask = torch.ones(tile_h, tile_w, dtype=torch.float32)
if done_left and ol_w > 0:
_fade_band(mask[:, :ol_w], fade_w, 1)
if done_top and ol_h > 0:
_fade_band(mask[:ol_h, :], fade_h, 0)
return mask
def bright_match_tile(tile, ref, clamp=0.05):
d = (tile - ref).float().reshape(tile.shape[0], tile.shape[1], tile.shape[2], -1)
dc = d.median(dim=-1).values.clamp(-clamp, clamp)
return tile - dc.to(tile.dtype).view(tile.shape[0], tile.shape[1], tile.shape[2], 1, 1)
def _dynamic_fade_closure(sp, fw, fh, tr, tc, tr_s, tc_s, ovh, ovw, done_top, done_left, video_flat, mn=0.0):
schedule = sp.get("dynamic_fade", "off")
if schedule == "off":
return None
fmin_w = int(sp.get("dynamic_fade_min", 0)) // 16
fmin_h = int(sp.get("dynamic_fade_min", 0)) // 16
if fw <= fmin_w and fh <= fmin_h:
return None
fw_start, fh_start = max(fw, 0), max(fh, 0)
fmin_w, fmin_h = min(fmin_w, fw_start), min(fmin_h, fh_start)
done_top = done_top and ovh > 0
done_left = done_left and ovw > 0
s_tok = tr_s * tc_s
n_frames = video_flat // s_tok
def fade_at(p):
if schedule == "widening":
return fmin_w + (fw_start - fmin_w) * p, fmin_h + (fh_start - fmin_h) * p
return fw_start - (fw_start - fmin_w) * p, fh_start - (fh_start - fmin_h) * p
cache = {}
def step_fn(sigma, denoise_mask, **kwargs):
sigmas = kwargs.get("extra_options", {}).get("sigmas")
n_sigmas = int(sigmas.numel()) if sigmas is not None else 0
masks = cache.get(n_sigmas)
if masks is None:
step_count = max(n_sigmas - 1, 1)
masks = []
for i in range(n_sigmas - 1):
p = i / (step_count - 1) if step_count > 1 else 0.0
cw, ch = fade_at(p)
m = make_fade_mask(tr_s, tc_s, ovh, ovw, done_top, done_left,
fade_h=round(ch), fade_w=round(cw))
m[tr:tr_s, :] = 0.0
m[:, tc:tc_s] = 0.0
if mn > 0:
m = m + mn * (1.0 - m)
masks.append(m)
cache[n_sigmas] = masks
idx = 0
if sigmas is not None:
idx = int((sigmas > sigma + 1e-6).sum())
m = masks[min(idx, len(masks) - 1)]
flat = denoise_mask.clone()
flat.reshape(denoise_mask.shape[0], -1)[:, :video_flat] = \
m.reshape(1, -1).repeat(denoise_mask.shape[0], n_frames)
return flat
return step_fn
def _nested_tensor_parts(samples):
if samples is None:
return ()
parts = getattr(samples, "tensors", None)
if parts is not None:
return tuple(parts)
if hasattr(samples, "unbind"):
try:
return tuple(samples.unbind())
except Exception:
return ()
return ()
def _coerce_bool_flag(value): def _coerce_bool_flag(value):
if isinstance(value, str): if isinstance(value, str):
text = value.strip().lower() text = value.strip().lower()
@@ -4097,7 +4335,7 @@ def _format_timing_note(shot_timings):
"total": 0.0, "total": 0.0,
"retry_elapsed": 0.0, "retry_elapsed": 0.0,
"sample": 0.0, "sample": 0.0,
"detail_sample": 0.0, "latent_upscale_sample": 0.0,
"decode_video": 0.0, "decode_video": 0.0,
"decode_audio": 0.0, "decode_audio": 0.0,
"cleanup": 0.0, "cleanup": 0.0,
@@ -4108,7 +4346,7 @@ def _format_timing_note(shot_timings):
totals["total"] += float(shot.get("total", 0.0) or 0.0) totals["total"] += float(shot.get("total", 0.0) or 0.0)
totals["retry_elapsed"] += float(shot.get("retry_elapsed", 0.0) or 0.0) totals["retry_elapsed"] += float(shot.get("retry_elapsed", 0.0) or 0.0)
totals["sample"] += float(shot.get("sample", 0.0) or 0.0) totals["sample"] += float(shot.get("sample", 0.0) or 0.0)
totals["detail_sample"] += float(shot.get("detail_sample", 0.0) or 0.0) totals["latent_upscale_sample"] += float(shot.get("latent_upscale_sample", 0.0) or 0.0)
totals["decode_video"] += float(shot.get("decode_video", 0.0) or 0.0) totals["decode_video"] += float(shot.get("decode_video", 0.0) or 0.0)
totals["decode_audio"] += float(shot.get("decode_audio", 0.0) or 0.0) totals["decode_audio"] += float(shot.get("decode_audio", 0.0) or 0.0)
totals["cleanup"] += float(shot.get("cleanup", 0.0) or 0.0) totals["cleanup"] += float(shot.get("cleanup", 0.0) or 0.0)
@@ -4124,8 +4362,8 @@ def _format_timing_note(shot_timings):
] ]
if totals["retry_elapsed"]: if totals["retry_elapsed"]:
pieces.append(f"retry elapsed {_format_elapsed_seconds(totals['retry_elapsed'])}") pieces.append(f"retry elapsed {_format_elapsed_seconds(totals['retry_elapsed'])}")
if totals["detail_sample"]: if totals["latent_upscale_sample"]:
pieces.append(f"detail {_format_elapsed_seconds(totals['detail_sample'])}") pieces.append(f"latent upscale {_format_elapsed_seconds(totals['latent_upscale_sample'])}")
if totals["retries"]: if totals["retries"]:
pieces.append(f"retries {totals['retries']}") pieces.append(f"retries {totals['retries']}")
if slowest is not None: if slowest is not None:
@@ -5847,6 +6085,29 @@ def _evict_all_but(keep_model):
pass pass
def _evict_for_latent_upscale(model):
"""Clear the sampler model before loading the auxiliary latent upscaler."""
try:
unload_clones = getattr(mm, "unload_model_and_clones", None)
if callable(unload_clones):
try:
unload_clones(model, unload_additional_models=False)
mm.soft_empty_cache()
return
except Exception:
pass
mm.unload_all_models()
except Exception:
pass
try:
mm.soft_empty_cache(True)
except Exception:
try:
mm.soft_empty_cache()
except Exception:
pass
@@ -6084,6 +6345,12 @@ class H3LongVideos:
"decode_tile_size": ("INT", {"default": 0, "min": 0, "max": 1024, "step": 32, "decode_tile_size": ("INT", {"default": 0, "min": 0, "max": 1024, "step": 32,
"tooltip": "Spatial tile size for the VAE decode (tile_x/tile_y). 0 = ComfyUI default. " "tooltip": "Spatial tile size for the VAE decode (tile_x/tile_y). 0 = ComfyUI default. "
"Try 256 on a tight card at 1344x768."}), "Try 256 on a tight card at 1344x768."}),
"latent_upscale_param": ("DUMAS_H3_LATENT_UPSCALE_PARAM", {
"tooltip": "Output of 'Dumas H3 Latent Upscale Params'. When connected, the first-pass "
"latent is upscaled and run through a short refinement pass before decode, "
"using the sampler, scheduler, steps, denoise, megapixel target, and optional "
"spatial batching from that node. "
"Leave unconnected to skip latent upscaling entirely."}),
"upscale": (["off", "rtx", "model", "lanczos"], {"default": "off", "upscale": (["off", "rtx", "model", "lanczos"], {"default": "off",
"tooltip": "Optional post-pass on the finished frames. 'rtx' = NVIDIA RTX Video Super " "tooltip": "Optional post-pass on the finished frames. 'rtx' = NVIDIA RTX Video Super "
"Resolution (Tensor Cores -- fastest and best for video; needs the " "Resolution (Tensor Cores -- fastest and best for video; needs the "
@@ -6363,26 +6630,6 @@ class H3LongVideos:
"the jacket, 'wardrobe: += sunglasses' adds one. TWO+ PEOPLE: name them -- " "the jacket, 'wardrobe: += sunglasses' adds one. TWO+ PEOPLE: name them -- "
"'Maya = grey shorts, red jacket; Jon = navy overalls', then edit one at a " "'Maya = grey shorts, red jacket; Jon = navy overalls', then edit one at a "
"time: 'wardrobe: Maya -= jacket' leaves Jon untouched."}), "time: 'wardrobe: Maya -= jacket' leaves Jon untouched."}),
"detail_pass": ("BOOLEAN", {"default": False,
"tooltip": "Run a second refinement sampler on each beat BEFORE any upscale. "
"It reuses the same conditioning and keeps the output video-only by "
"preserving the first pass's audio latent. Use it for detail cleanup, not "
"for huge rewrites: too many steps or too much denoise can pull identity or "
"continuity away from the main pass."}),
"detail_sampler_name": (comfy.samplers.KSampler.SAMPLERS, {"default": "euler",
"tooltip": "Sampler for the optional refinement pass. Euler is the maintained default "
"direction for this lane."}),
"detail_scheduler": (comfy.samplers.KSampler.SCHEDULERS, {"default": "beta",
"tooltip": "Scheduler for the optional refinement pass. Beta is the maintained default "
"direction for the H3 enhancement lane."}),
"detail_steps": ("INT", {"default": 8, "min": 1, "max": 200,
"tooltip": "Extra steps for the refinement pass only. Start around 4-8. More is not "
"automatically better; once the pass starts rewriting instead of polishing, "
"identity and continuity can drift."}),
"detail_denoise": ("FLOAT", {"default": 0.4, "min": 0.0, "max": 1.0, "step": 0.01,
"tooltip": "How hard the refinement pass is allowed to rewrite the beat latent. Start "
"around 0.20-0.35 for gentle cleanup; 0.4+ is stronger and can noticeably "
"change faces, motion or composition."}),
}, },
# Read-only graph access, for SLA-LoRA detection: a LoRA's filename is # Read-only graph access, for SLA-LoRA detection: a LoRA's filename is
# the only thing that identifies an SLA build, and the graph is the only # the only thing that identifies an SLA build, and the graph is the only
@@ -6407,9 +6654,8 @@ class H3LongVideos:
def _render(self, model, clip, vae, audio_vae, negative, prompt, w, h, ln, fps, tiled, sa, def _render(self, model, clip, vae, audio_vae, negative, prompt, w, h, ln, fps, tiled, sa,
handoff, decode_tile_frames=0, decode_tile_size=0, handoff, decode_tile_frames=0, decode_tile_size=0,
refs=None, ref_image_size="match", ref_noise_aug=None, silent=False, refs=None, ref_image_size="match", ref_noise_aug=None, silent=False,
detail_pass=False, detail_sampler_name="euler", detail_scheduler="beta", latent_upscale_param=None, timing_sink=None):
detail_steps=8, detail_denoise=0.4, timing_sink=None): timing = {"sample": 0.0, "latent_upscale_sample": 0.0, "decode_video": 0.0, "decode_audio": 0.0, "cleanup": 0.0}
timing = {"sample": 0.0, "detail_sample": 0.0, "decode_video": 0.0, "decode_audio": 0.0, "cleanup": 0.0}
positive, latent = _build_shot_conditioning(clip, vae, prompt, w, h, ln, fps, handoff, positive, latent = _build_shot_conditioning(clip, vae, prompt, w, h, ln, fps, handoff,
ref_images=refs, ref_image_size=ref_image_size, ref_images=refs, ref_image_size=ref_image_size,
ref_noise_aug=ref_noise_aug, ref_noise_aug=ref_noise_aug,
@@ -6428,43 +6674,206 @@ class H3LongVideos:
# OOM retry cannot help an OOM raised here -- it just re-runs the whole # OOM retry cannot help an OOM raised here -- it just re-runs the whole
# sampling pass and fails the same way, which on a 362-frame shot is four # sampling pass and fails the same way, which on a 362-frame shot is four
# more minutes for nothing. # more minutes for nothing.
if _is_oom(e): raise _tag_oom_stage(e, "sampling")
e._h3_stage = "sampling"
raise
refined_out = out refined_out = out
detail_pass = _coerce_bool_flag(detail_pass) latent_upscale_param = latent_upscale_param or None
if detail_pass: latent_upscale_mode = _latent_upscale_mode(latent_upscale_param)
detail_latent = _latent_with_replaced_samples(latent, out) if latent_upscale_mode != "off":
try: try:
detail_start = time.perf_counter() latent_start = time.perf_counter()
out_samples = out["samples"]
parts = _nested_tensor_parts(out_samples)
if not getattr(out_samples, "is_nested", False) or len(parts) < 2:
raise RuntimeError("latent upscale expects a nested AV latent")
if latent_upscale_mode == "model" and str(latent_upscale_param.get("device", "cuda")) == "cuda":
_evict_for_latent_upscale(model)
upscaled_video, up_h, up_w = _upscale_latent_video(parts[0], latent_upscale_param)
full_audio = parts[1]
# Drop the first-pass sampling state before we start the refinement
# pass; otherwise the 12-step base latent and the upscale latent sit
# in memory together and can trigger a retry loop.
out["samples"] = comfy.nested_tensor.NestedTensor((upscaled_video, full_audio))
del out_samples, positive, latent, parts
mm.soft_empty_cache()
target_w = int(up_w) * 16
target_h = int(up_h) * 16
if target_w <= 0 or target_h <= 0:
raise RuntimeError("latent upscale target size must be positive")
upscale_cond, upscale_latent = _build_shot_conditioning(
clip, vae, prompt, target_w, target_h, ln, fps, handoff,
ref_images=refs, ref_image_size=ref_image_size,
ref_noise_aug=ref_noise_aug, audio_vae=audio_vae, silent=silent)
upscale_latent["samples"] = comfy.nested_tensor.NestedTensor(
(upscaled_video, full_audio))
refine_steps = int(latent_upscale_param.get("steps", 2) or 2)
refine_sampler = latent_upscale_param.get("sampler_name", sn)
refine_scheduler = latent_upscale_param.get("scheduler", sch)
refine_denoise_value = latent_upscale_param.get("denoise", latent_upscale_param.get("refine_denoise", 0.2))
refine_denoise = 0.2 if refine_denoise_value is None else float(refine_denoise_value)
tile_size_mode = str(latent_upscale_param.get("tile_size_mode", "specific_size"))
tile_w_px = int(latent_upscale_param.get("tile_width", 512) or 512)
tile_h_px = int(latent_upscale_param.get("tile_height", 512) or 512)
overlap_px = max(0, int(_param_value(latent_upscale_param, "overlap", 64)))
fade_w_px = max(0, int(_param_value(latent_upscale_param, "fade_width", 32)))
fade_h_px = max(0, int(_param_value(latent_upscale_param, "fade_height", 32)))
overlap_mode = str(latent_upscale_param.get("overlap_mode", "earlier"))
overlap_blend = str(latent_upscale_param.get("overlap_blend", "linear"))
grid_rows = max(1, int(_param_value(latent_upscale_param, "grid_rows", 2)))
grid_cols = max(1, int(_param_value(latent_upscale_param, "grid_cols", 2)))
spatial_w_overlap_px = max(0, int(_param_value(latent_upscale_param, "spatial_w_overlap", overlap_px)))
spatial_h_overlap_px = max(0, int(_param_value(latent_upscale_param, "spatial_h_overlap", overlap_px)))
min_tile_size_px = max(0, int(_param_value(latent_upscale_param, "min_tile_size", 256)))
masked_area_noise = float(_param_value(latent_upscale_param, "masked_area_noise", 0.0))
brightness_match = bool(latent_upscale_param.get("brightness_match", False))
dynamic_fade = str(latent_upscale_param.get("dynamic_fade", "off"))
dynamic_fade_min_px = max(0, int(_param_value(latent_upscale_param, "dynamic_fade_min", 32)))
if tile_size_mode == "rows_cols":
tile_w_px, spatial_w_overlap_px = _solve_equal_tiles(target_w, grid_cols, spatial_w_overlap_px, 16)
tile_h_px, spatial_h_overlap_px = _solve_equal_tiles(target_h, grid_rows, spatial_h_overlap_px, 16)
if tile_w_px < min_tile_size_px or tile_h_px < min_tile_size_px:
raise ValueError(
f"rows_cols mode: solved tile size is {tile_h_px}x{tile_w_px}px "
f"(grid {grid_rows}x{grid_cols} over {target_h}x{target_w}px), "
f"which is smaller than min_tile_size ({min_tile_size_px}px). "
f"Reduce grid_rows/grid_cols, or lower min_tile_size to at most "
f"{min(tile_w_px, tile_h_px)}px."
)
fade_w_px = min(fade_w_px, spatial_w_overlap_px)
fade_h_px = min(fade_h_px, spatial_h_overlap_px)
else:
for name, value in (
("tile_width", tile_w_px),
("tile_height", tile_h_px),
("overlap", overlap_px),
("fade_width", fade_w_px),
("fade_height", fade_h_px),
("min_tile_size", min_tile_size_px),
):
if value % 32 != 0:
raise ValueError(f"'{name}' must be a multiple of 32 pixels; got {value}.")
if overlap_px >= tile_w_px:
raise ValueError("overlap must be smaller than tile_width")
if overlap_px >= tile_h_px:
raise ValueError("overlap must be smaller than tile_height")
if fade_w_px > spatial_w_overlap_px:
raise ValueError("fade_width must not exceed spatial_w_overlap")
if fade_h_px > spatial_h_overlap_px:
raise ValueError("fade_height must not exceed spatial_h_overlap")
if min_tile_size_px > tile_w_px or min_tile_size_px > tile_h_px:
raise ValueError("min_tile_size must not exceed the tile size")
tile_tw = max(1, tile_w_px // 16)
tile_th = max(1, tile_h_px // 16)
ol_tw = max(0, min(tile_tw - 1, spatial_w_overlap_px // 16))
ol_th = max(0, min(tile_th - 1, spatial_h_overlap_px // 16))
fw_tw = max(0, min(ol_tw, fade_w_px // 16))
fw_th = max(0, min(ol_th, fade_h_px // 16))
min_tile_tw = max(0, min_tile_size_px // 16)
rows, cols, trows, tcols, row_ovl, col_ovl = compute_spatial_grid(
int(up_h), int(up_w), tile_th, tile_tw, ol_th, ol_tw, min_tile_tw, min_tile_tw
)
if len(rows) == 1 and len(cols) == 1:
(refined_out,) = nodes.common_ksampler( (refined_out,) = nodes.common_ksampler(
model, seed, int(detail_steps), cfg, detail_sampler_name, detail_scheduler, model, seed, refine_steps, cfg, refine_sampler, refine_scheduler, upscale_cond, negative, upscale_latent,
positive, negative, detail_latent, denoise=float(detail_denoise)) denoise=refine_denoise)
timing["detail_sample"] += time.perf_counter() - detail_start else:
refined_video = upscaled_video.clone()
for row_index, r0 in enumerate(rows):
tr = trows[row_index]
ovh = row_ovl[row_index]
for col_index, c0 in enumerate(cols):
tc = tcols[col_index]
ovw = col_ovl[col_index]
tile_target_w = int(tc) * 16
tile_target_h = int(tr) * 16
tile_cond, tile_latent = _build_shot_conditioning(
clip, vae, prompt, tile_target_w, tile_target_h, ln, fps, handoff,
ref_images=refs, ref_image_size=ref_image_size,
ref_noise_aug=ref_noise_aug, audio_vae=audio_vae, silent=silent)
tile_video = upscaled_video[:, :, :, r0:r0 + tr, c0:c0 + tc].contiguous()
tr_s = tr + (tr % 2)
tc_s = tc + (tc % 2)
tile = torch.zeros((1, tile_video.shape[1], tile_video.shape[2], tr_s, tc_s),
device=tile_video.device, dtype=tile_video.dtype)
tile[:, :, :, :tr, :tc] = tile_video
if col_index > 0 and ovw > 0:
tile[:, :, :, :tr, :ovw] = refined_video[:, :, :, r0:r0 + tr, c0:c0 + ovw]
if row_index > 0 and ovh > 0:
tile[:, :, :, :ovh, :tc] = refined_video[:, :, :, r0:r0 + ovh, c0:c0 + tc]
mask = make_fade_mask(tr_s, tc_s, ovh, ovw, row_index > 0, col_index > 0,
fade_h=fw_th, fade_w=fw_tw)
mask[tr:tr_s, :] = 0.0
mask[:, tc:tc_s] = 0.0
mv = (mask + masked_area_noise * (1.0 - mask))[None, None, None].to(tile.dtype)
ma = torch.zeros((1, 32, 2, full_audio.shape[-1]), device=full_audio.device, dtype=full_audio.dtype)
tile_latent["samples"] = comfy.nested_tensor.NestedTensor((tile, full_audio))
tile_latent["noise_mask"] = comfy.nested_tensor.NestedTensor((mv, ma))
dynamic = _dynamic_fade_closure(
latent_upscale_param, fw_tw, fw_th, tr, tc, tr_s, tc_s, ovh, ovw,
row_index > 0, col_index > 0, math.prod(tile.shape[1:]), mn=masked_area_noise
)
if dynamic is not None:
model.set_model_denoise_mask_function(dynamic)
try:
tile_out, = nodes.common_ksampler(
model, seed, refine_steps, cfg, refine_sampler, refine_scheduler, tile_cond, negative, tile_latent,
denoise=refine_denoise)
finally:
if dynamic is not None:
model.model_options.pop("denoise_mask_function", None)
tile_out = _video_only_refined_latent(
{"samples": comfy.nested_tensor.NestedTensor((tile_video, full_audio))},
tile_out)
tile_video_out = tile_out["samples"].tensors[0]
if brightness_match:
tile_video_out = bright_match_tile(
tile_video_out,
upscaled_video[:, :, :, r0:r0 + tr, c0:c0 + tc]
)
region = refined_video[:, :, :, r0:r0 + tr, c0:c0 + tc]
base_region = region.clone()
region.copy_(tile_video_out)
if col_index > 0 and ol_tw > 0:
t = torch.linspace(0.0, 1.0, ol_tw, device=region.device, dtype=region.dtype)
w = _latent_spatial_blend_weights(t, overlap_mode, overlap_blend)
if fw_tw > 0:
w = w.clone()
w[:fw_tw] = 0.0
region[:, :, :, :, :ol_tw] = (
base_region[:, :, :, :, :ol_tw] * (1.0 - w[None, None, None, None, :]) +
tile_video_out[:, :, :, :, :ol_tw] * w[None, None, None, None, :]
)
if row_index > 0 and ol_th > 0:
t = torch.linspace(0.0, 1.0, ol_th, device=region.device, dtype=region.dtype)
w = _latent_spatial_blend_weights(t, overlap_mode, overlap_blend)
if fw_th > 0:
w = w.clone()
w[:fw_th] = 0.0
region[:, :, :, :ol_th, :] = (
base_region[:, :, :, :ol_th, :] * (1.0 - w[None, None, None, :, None]) +
tile_video_out[:, :, :, :ol_th, :] * w[None, None, None, :, None]
)
refined_out = {"samples": comfy.nested_tensor.NestedTensor((refined_video, full_audio))}
timing["latent_upscale_sample"] += time.perf_counter() - latent_start
refined_out = _video_only_refined_latent(upscale_latent, refined_out)
del upscale_latent, upscaled_video, full_audio
mm.soft_empty_cache()
except Exception as e: except Exception as e:
if _is_oom(e): raise _tag_oom_stage(e, "latent_upscale")
e._h3_stage = "sampling"
raise
refined_out = _video_only_refined_latent(out, refined_out)
del detail_latent
# Keep a CPU copy of the sampled latent BEFORE decoding, for the `latent` # Keep a CPU copy of the sampled latent BEFORE decoding, for the `latent`
# output. Latents are ~1000x smaller than the frames they decode to (a # output. Latents are ~1000x smaller than the frames they decode to (a
# 1344x768 124f shot is ~1.5MB against ~1.5GB), so carrying one per shot for # 1344x768 124f shot is ~1.5MB against ~1.5GB), so carrying one per shot for
# the whole chain is free. Detached and moved off the card immediately, for # the whole chain is free. Detached and moved off the card immediately, for
# the same reason the decoded frames are. # the same reason the decoded frames are.
decode_video_start = time.perf_counter()
shot_latent = _copy_sample_latent(refined_out)
decode_audio_start = time.perf_counter() decode_audio_start = time.perf_counter()
audio = _decode_audio(audio_vae, out) audio = _decode_audio(audio_vae, out)
timing["decode_audio"] += time.perf_counter() - decode_audio_start timing["decode_audio"] += time.perf_counter() - decode_audio_start
# Audio is much smaller than the video decode. Drop the first-pass decode_video_start = time.perf_counter()
# conditioning before the VAE work so the optional detail pass does not shot_latent = _copy_sample_latent(refined_out)
# keep both sampled latents resident across the heaviest allocation.
del out, positive, latent
video = _decode_video(vae, refined_out, tiled, free_first=model, video = _decode_video(vae, refined_out, tiled, free_first=model,
tile_t=decode_tile_frames, tile_xy=decode_tile_size) tile_t=decode_tile_frames, tile_xy=decode_tile_size)
timing["decode_video"] += time.perf_counter() - decode_video_start timing["decode_video"] += time.perf_counter() - decode_video_start
cleanup_start = time.perf_counter() cleanup_start = time.perf_counter()
del out
del refined_out del refined_out
_deep_cleanup() _deep_cleanup()
timing["cleanup"] += time.perf_counter() - cleanup_start timing["cleanup"] += time.perf_counter() - cleanup_start
@@ -6500,8 +6909,7 @@ class H3LongVideos:
ref_5=None, ref_6=None, ref_7=None, ref_8=None, ref_5=None, ref_6=None, ref_7=None, ref_8=None,
ref_9=None, ref_9=None,
ref_mode="auto ref2v", ref_image_size="match", ref_noise_aug=0.95, ref_mode="auto ref2v", ref_image_size="match", ref_noise_aug=0.95,
detail_pass=False, detail_sampler_name="euler", detail_scheduler="beta", latent_upscale_param=None,
detail_steps=8, detail_denoise=0.4,
graph=None, node_id=None): graph=None, node_id=None):
# FIRST: detect a checkpoint swap since the previous execution and hard-flush. # FIRST: detect a checkpoint swap since the previous execution and hard-flush.
@@ -6565,12 +6973,42 @@ class H3LongVideos:
ms_note = "" ms_note = ""
if apply_model_sampling: if apply_model_sampling:
model, ms_note = apply_h3_model_sampling(model, shift_video, shift_audio) model, ms_note = apply_h3_model_sampling(model, shift_video, shift_audio)
detail_note = "" latent_upscale_note = ""
if detail_pass: latent_upscale_mode = _latent_upscale_mode(latent_upscale_param)
detail_note = (f" detail pass: {int(detail_steps)} step(s) via " if latent_upscale_mode != "off":
f"{detail_sampler_name}/{detail_scheduler} at denoise " target_w, target_h = _latent_upscale_target_size(w, h, latent_upscale_param)
f"{float(detail_denoise):.2f}; video-only refinement keeps " mode = latent_upscale_mode
f"audio from the first pass") detail = f" via {mode}"
if mode == "model":
detail += f"/{latent_upscale_param.get('model_name', 'none')}"
else:
detail += f"/{latent_upscale_param.get('method', 'bilinear')}"
denoise_value = latent_upscale_param.get("denoise", latent_upscale_param.get("refine_denoise", 0.2))
denoise = 0.2 if denoise_value is None else float(denoise_value)
refine_sampler = latent_upscale_param.get("sampler_name", "euler_ancestral")
refine_scheduler = latent_upscale_param.get("scheduler", "simple")
batch_note = ""
tile_size_mode = str(latent_upscale_param.get("tile_size_mode", "specific_size"))
tile_w_px = int(latent_upscale_param.get("tile_width", 512) or 512)
tile_h_px = int(latent_upscale_param.get("tile_height", 512) or 512)
overlap_px = max(0, int(_param_value(latent_upscale_param, "overlap", 64)))
overlap_blend = str(latent_upscale_param.get("overlap_blend", "linear"))
grid_rows = max(1, int(_param_value(latent_upscale_param, "grid_rows", 2)))
grid_cols = max(1, int(_param_value(latent_upscale_param, "grid_cols", 2)))
if tile_size_mode == "rows_cols":
batch_note = f"; spatial batches {grid_rows}x{grid_cols} rows_cols over {target_w}x{target_h}px"
elif tile_w_px > 0 and tile_h_px > 0 and (tile_w_px < target_w or tile_h_px < target_h):
spatial_w_overlap_px = max(0, int(_param_value(latent_upscale_param, "spatial_w_overlap", overlap_px)))
spatial_h_overlap_px = max(0, int(_param_value(latent_upscale_param, "spatial_h_overlap", overlap_px)))
batch_note = (
f"; spatial batches {tile_w_px}x{tile_h_px}px "
f"overlap {spatial_w_overlap_px}x{spatial_h_overlap_px}px {overlap_blend}"
)
latent_upscale_note = (
f" latent upscale: target {target_w}x{target_h}px{detail}; "
f"{int(latent_upscale_param.get('steps', 2) or 2)}-step refinement "
f"{refine_sampler}/{refine_scheduler} denoise {denoise:.2f}{batch_note}"
)
paras = split_paragraphs(prompt, "##") paras = split_paragraphs(prompt, "##")
if anchor_override.strip(): if anchor_override.strip():
@@ -6881,6 +7319,7 @@ class H3LongVideos:
+ (" ANCHOR: " + "; ".join(anchor_hazards) + "." + (" ANCHOR: " + "; ".join(anchor_hazards) + "."
if anchor_hazards else "") if anchor_hazards else "")
+ (f"{anatomy_note}." if anatomy_note else "") + (f"{anatomy_note}." if anatomy_note else "")
+ (f"{latent_upscale_note}." if latent_upscale_note else "")
+ (f"{plan_audio}." if plan_audio else "") + (f"{plan_audio}." if plan_audio else "")
+ (" EXPOSURE -- " + "; ".join(wardrobe_notes) + "." + (" EXPOSURE -- " + "; ".join(wardrobe_notes) + "."
if wardrobe_notes else "") if wardrobe_notes else "")
@@ -7029,13 +7468,18 @@ class H3LongVideos:
model, clip, vae, audio_vae, negative, gen_prompt, w, h, ln_i, fps, model, clip, vae, audio_vae, negative, gen_prompt, w, h, ln_i, fps,
tiled, sa, shot_handoff, decode_tile_frames, decode_tile_size, tiled, sa, shot_handoff, decode_tile_frames, decode_tile_size,
shot_refs, ref_image_size, shot_aug, shot_silent, shot_refs, ref_image_size, shot_aug, shot_silent,
detail_pass, detail_sampler_name, detail_scheduler, latent_upscale_param=latent_upscale_param, timing_sink=shot_timing)
detail_steps, detail_denoise, timing_sink=shot_timing)
break break
except (torch.cuda.OutOfMemoryError, RuntimeError) as e: except (torch.cuda.OutOfMemoryError, RuntimeError) as e:
shot_retry_elapsed += time.perf_counter() - attempt_start shot_retry_elapsed += time.perf_counter() - attempt_start
if not _is_oom(e): if not _is_oom(e):
raise raise
if getattr(e, "_h3_stage", "") == "latent_upscale":
raise RuntimeError(
f"H3 Long Videos: shot {i + 1} of {len(gens)} ran out of VRAM "
f"during latent upscale. The latent-upscale stage is too large "
f"for this card at {w}x{h} with the current spatial settings."
) from e
mm.soft_empty_cache(True) mm.soft_empty_cache(True)
if not tiled: if not tiled:
tiled = True; backoff.append("tiled decode") tiled = True; backoff.append("tiled decode")
@@ -7052,10 +7496,16 @@ class H3LongVideos:
model, clip, vae, audio_vae, negative, gen_prompt, w, h, ln_i, fps, model, clip, vae, audio_vae, negative, gen_prompt, w, h, ln_i, fps,
tiled, sa, shot_handoff, decode_tile_frames, decode_tile_size, tiled, sa, shot_handoff, decode_tile_frames, decode_tile_size,
shot_refs, ref_image_size, shot_aug, shot_silent, shot_refs, ref_image_size, shot_aug, shot_silent,
detail_pass, detail_sampler_name, detail_scheduler, latent_upscale_param=latent_upscale_param, timing_sink=shot_timing)
detail_steps, detail_denoise, timing_sink=shot_timing)
except (torch.cuda.OutOfMemoryError, RuntimeError) as e: except (torch.cuda.OutOfMemoryError, RuntimeError) as e:
shot_retry_elapsed += time.perf_counter() - attempt_start shot_retry_elapsed += time.perf_counter() - attempt_start
stage = getattr(e, "_h3_stage", "")
if _is_oom(e) and stage == "latent_upscale":
raise RuntimeError(
f"H3 Long Videos: shot {i + 1} of {len(gens)} ran out of VRAM "
f"during latent upscale. The latent-upscale stage is too large "
f"for this card at {w}x{h} with the current spatial settings."
) from e
if _is_oom(e) and getattr(e, "_h3_stage", "") == "sampling": if _is_oom(e) and getattr(e, "_h3_stage", "") == "sampling":
# Retrying with tiles would re-run the whole sampling pass and # Retrying with tiles would re-run the whole sampling pass and
# fail identically. Fail now, and say what actually shrinks it. # fail identically. Fail now, and say what actually shrinks it.
@@ -7072,8 +7522,7 @@ class H3LongVideos:
model, clip, vae, audio_vae, negative, gen_prompt, w, h, ln_i, fps, model, clip, vae, audio_vae, negative, gen_prompt, w, h, ln_i, fps,
tiled, sa, shot_handoff, decode_tile_frames, decode_tile_size, tiled, sa, shot_handoff, decode_tile_frames, decode_tile_size,
shot_refs, ref_image_size, shot_aug, shot_silent, shot_refs, ref_image_size, shot_aug, shot_silent,
detail_pass, detail_sampler_name, detail_scheduler, latent_upscale_param=latent_upscale_param, timing_sink=shot_timing)
detail_steps, detail_denoise, timing_sink=shot_timing)
shot_retry_elapsed += time.perf_counter() - attempt_start shot_retry_elapsed += time.perf_counter() - attempt_start
shot_total = time.perf_counter() - shot_total_start shot_total = time.perf_counter() - shot_total_start
@@ -7343,7 +7792,7 @@ class H3LongVideos:
f"shot before them ended on dialogue." if mouth_settled else "") f"shot before them ended on dialogue." if mouth_settled else "")
+ (f"{anatomy_note}." if anatomy_note else "") + (f"{anatomy_note}." if anatomy_note else "")
+ (f"{latent_note}." if latent_note else "") + (f"{latent_note}." if latent_note else "")
+ (f"{detail_note}." if detail_note else "") + (f"{latent_upscale_note}." if latent_upscale_note else "")
+ (f" SLA LoRA '{os.path.basename(str(sla_name))}' paired with sparse attention." + (f" SLA LoRA '{os.path.basename(str(sla_name))}' paired with sparse attention."
if sla_name and sparse_on else "") if sla_name and sparse_on else "")
+ (f" {beats_note}." if beats_note else "") + (f" {beats_note}." if beats_note else "")
+4 -3
View File
@@ -39,10 +39,11 @@ class H3ShotLength:
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
"required": { "required": {
"shot_seconds": ("FLOAT", {"default": 5.0, "min": 0.2, "max": 15.1, "step": 0.5, "shot_seconds": ("FLOAT", {"default": 3.0, "min": 0.2, "max": 15.1, "step": 0.5,
"tooltip": "Length of each shot. Feeds the sampler's shot_seconds AND (as frames) " "tooltip": "Length of each shot. Feeds the sampler's shot_seconds AND (as frames) "
"the preview override. Max ~15s (362 frames)."}), "the preview override. Default 3s matches the common one-beat H3 test shot. Max ~15s (362 frames)."}),
"fps": ("INT", {"default": 24, "min": 1, "max": 60}), "fps": ("INT", {"default": 24, "min": 1, "max": 60,
"tooltip": "Frame rate used for the seconds->frames conversion. H3 itself renders at 24fps, so 24 is the realistic default."}),
}, },
"optional": { "optional": {
"cap_to_h3_max": ("BOOLEAN", {"default": True, "cap_to_h3_max": ("BOOLEAN", {"default": True,
+50 -29
View File
@@ -277,7 +277,14 @@ class DumasJSONStringToObjectNode:
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
"required": { "required": {
"json_string": ("STRING", {"multiline": True}), "json_string": (
"STRING",
{
"multiline": True,
"default": '{\n "shots": [\n {\n "prompt": "Francine stands by the window."\n }\n ]\n}',
"tooltip": "Raw JSON text to parse into a structured JSON object."
},
),
} }
} }
@@ -299,7 +306,14 @@ class DumasStripIterationSuffixNode:
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
"required": { "required": {
"filename": ("STRING", {"default": "", "multiline": False}), "filename": (
"STRING",
{
"default": "francine_pose_final.png",
"multiline": False,
"tooltip": "Filename to normalize by removing everything after the first underscore in the stem."
},
),
} }
} }
@@ -316,7 +330,14 @@ class DumasSlugifyStringNode:
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
"required": { "required": {
"text": ("STRING", {"default": "", "multiline": False}), "text": (
"STRING",
{
"default": "Francine Coffee Shop",
"multiline": False,
"tooltip": "Text to slugify into lowercase ASCII words joined with hyphens."
},
),
} }
} }
@@ -334,8 +355,8 @@ class DumasJSONObjectToStringNode:
return { return {
"required": { "required": {
"json_object": ("JSON",), "json_object": ("JSON",),
"pretty": ("BOOLEAN", {"default": True}), "pretty": ("BOOLEAN", {"default": True, "tooltip": "Pretty-print the JSON with indentation."}),
"sort_keys": ("BOOLEAN", {"default": False}), "sort_keys": ("BOOLEAN", {"default": False, "tooltip": "Sort object keys alphabetically before serializing."}),
} }
} }
@@ -355,7 +376,7 @@ class DumasJSONGetValueNode:
return { return {
"required": { "required": {
"json_object": ("JSON",), "json_object": ("JSON",),
"path": ("STRING", {"default": "", "multiline": False}), "path": ("STRING", {"default": "shots.0.prompt", "multiline": False, "tooltip": "Dot-path to read, such as 'shots.0.prompt'."}),
} }
} }
@@ -373,8 +394,8 @@ class DumasJSONSetValueNode:
return { return {
"required": { "required": {
"json_object": ("JSON",), "json_object": ("JSON",),
"path": ("STRING", {"default": "", "multiline": False}), "path": ("STRING", {"default": "shots.0.prompt", "multiline": False, "tooltip": "Dot-path to write, such as 'shots.0.prompt' or 'shots.1.duration'."}),
"value_json": ("STRING", {"multiline": True, "default": "null"}), "value_json": ("STRING", {"multiline": True, "default": '"Francine stands by the window."', "tooltip": "JSON value to store at the path. Must be valid JSON, so strings need quotes."}),
} }
} }
@@ -398,7 +419,7 @@ class DumasJSONHasKeyNode:
return { return {
"required": { "required": {
"json_object": ("JSON",), "json_object": ("JSON",),
"path": ("STRING", {"default": "", "multiline": False}), "path": ("STRING", {"default": "shots.0.prompt", "multiline": False, "tooltip": "Dot-path to test for existence."}),
} }
} }
@@ -416,7 +437,7 @@ class DumasJSONRemoveKeyNode:
return { return {
"required": { "required": {
"json_object": ("JSON",), "json_object": ("JSON",),
"path": ("STRING", {"default": "", "multiline": False}), "path": ("STRING", {"default": "shots.0.prompt", "multiline": False, "tooltip": "Dot-path to remove from the object."}),
} }
} }
@@ -434,7 +455,7 @@ class DumasJSONPickFieldsNode:
return { return {
"required": { "required": {
"json_object": ("JSON",), "json_object": ("JSON",),
"paths": ("STRING", {"multiline": True, "default": ""}), "paths": ("STRING", {"multiline": True, "default": "shots.0.prompt\nshots.0.duration", "tooltip": "One dot-path per line. Only those fields are copied into the output object."}),
} }
} }
@@ -463,8 +484,8 @@ class DumasJSONMergeObjectsNode:
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
"required": { "required": {
"base_object": ("JSON",), "base_object": ("JSON", {"tooltip": "Base JSON object to start from."}),
"overlay_object": ("JSON",), "overlay_object": ("JSON", {"tooltip": "Overlay JSON object whose keys replace or merge into the base object."}),
} }
} }
@@ -482,7 +503,7 @@ class DumasJSONKeysNode:
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
"required": { "required": {
"json_object": ("JSON",), "json_object": ("JSON", {"tooltip": "JSON object whose top-level keys should be listed."}),
} }
} }
@@ -502,7 +523,7 @@ class DumasJSONArrayLengthNode:
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
"required": { "required": {
"json_array": ("JSON",), "json_array": ("JSON", {"tooltip": "JSON array whose length should be measured."}),
} }
} }
@@ -521,8 +542,8 @@ class DumasJSONArrayAppendNode:
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
"required": { "required": {
"json_array": ("JSON",), "json_array": ("JSON", {"tooltip": "JSON array to append to."}),
"value_json": ("STRING", {"multiline": True, "default": "null"}), "value_json": ("STRING", {"multiline": True, "default": '{"prompt":"Francine looks toward the door."}', "tooltip": "JSON value to append. Must be valid JSON."}),
} }
} }
@@ -548,10 +569,10 @@ class DumasJSONArraySliceNode:
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
"required": { "required": {
"json_array": ("JSON",), "json_array": ("JSON", {"tooltip": "JSON array to slice."}),
"start": ("INT", {"default": 0, "step": 1}), "start": ("INT", {"default": 0, "step": 1, "tooltip": "Zero-based start index."}),
"end": ("INT", {"default": 0, "step": 1}), "end": ("INT", {"default": 0, "step": 1, "tooltip": "Zero-based end index. Use 0 to mean 'to the end'."}),
"step": ("INT", {"default": 1, "step": 1, "min": 1}), "step": ("INT", {"default": 1, "step": 1, "min": 1, "tooltip": "Slice step size."}),
} }
} }
@@ -572,9 +593,9 @@ class DumasJSONArrayIteratorNode:
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
"required": { "required": {
"json_input": ("JSON",), "json_input": ("JSON", {"tooltip": "JSON array to iterate over."}),
"index": ("INT", {"default": 0, "min": 0, "step": 1}), "index": ("INT", {"default": 0, "min": 0, "step": 1, "tooltip": "Current zero-based index."}),
"mode": (["fixed", "incr", "decr"], {"default": "fixed"}), "mode": (["fixed", "incr", "decr"], {"default": "fixed", "tooltip": "Keep the index fixed, increment it, or decrement it before reading."}),
} }
} }
@@ -600,9 +621,9 @@ class DumasJSONObjectIteratorNode:
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
"required": { "required": {
"json_input": ("JSON",), "json_input": ("JSON", {"tooltip": "JSON object whose key/value pairs should be iterated in insertion order."}),
"index": ("INT", {"default": 0, "min": 0, "step": 1}), "index": ("INT", {"default": 0, "min": 0, "step": 1, "tooltip": "Current zero-based index into the object's items."}),
"mode": (["fixed", "incr", "decr"], {"default": "fixed"}), "mode": (["fixed", "incr", "decr"], {"default": "fixed", "tooltip": "Keep the index fixed, increment it, or decrement it before reading."}),
} }
} }
@@ -628,7 +649,7 @@ class DumasJSONFlattenNode:
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
"required": { "required": {
"json_input": ("JSON",), "json_input": ("JSON", {"tooltip": "Nested JSON value to flatten into dot-path keys."}),
} }
} }
@@ -645,7 +666,7 @@ class DumasJSONUnflattenNode:
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
"required": { "required": {
"flat_json_object": ("JSON",), "flat_json_object": ("JSON", {"tooltip": "Flat JSON object whose keys are dot-paths to rebuild into nested JSON."}),
} }
} }
+1 -2
View File
@@ -53,11 +53,10 @@ const GROUPS = [
}, },
{ {
id: "finish", id: "finish",
label: "Upscale/Detail", label: "Upscale",
defaultCollapsed: true, defaultCollapsed: true,
widgets: [ widgets: [
"upscale", "upscale_model", "upscale_target_short_edge", "upscale_batch", "upscale", "upscale_model", "upscale_target_short_edge", "upscale_batch",
"detail_pass", "detail_sampler_name", "detail_scheduler", "detail_steps", "detail_denoise",
], ],
}, },
{ {
+254 -22
View File
@@ -24,6 +24,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
"PIL.Image", "PIL.Image",
"folder_paths", "folder_paths",
"dumas_image_nodes", "dumas_image_nodes",
"dumas_h3_latent_upscale",
"dumas_h3_longvideos", "dumas_h3_longvideos",
) )
} }
@@ -207,7 +208,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
"retry_elapsed": 1.2, "retry_elapsed": 1.2,
"attempts": 2, "attempts": 2,
"sample": 8.0, "sample": 8.0,
"detail_sample": 0.5, "latent_upscale_sample": 0.5,
"decode_video": 2.1, "decode_video": 2.1,
"decode_audio": 0.4, "decode_audio": 0.4,
"cleanup": 0.2, "cleanup": 0.2,
@@ -230,11 +231,11 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.assertIn("decode audio 0.7s", note) self.assertIn("decode audio 0.7s", note)
self.assertIn("cleanup 0.3s", note) self.assertIn("cleanup 0.3s", note)
self.assertIn("retry elapsed 1.2s", note) self.assertIn("retry elapsed 1.2s", note)
self.assertIn("detail 0.5s", note) self.assertIn("latent upscale 0.5s", note)
self.assertIn("retries 1", note) self.assertIn("retries 1", note)
self.assertIn("slowest shot 1 12.4s", note) self.assertIn("slowest shot 1 12.4s", note)
def test_detail_pass_refines_video_but_preserves_audio(self): def test_latent_upscale_refines_video_but_preserves_audio(self):
class FakeTensor: class FakeTensor:
def __init__(self, name): def __init__(self, name):
self.name = name self.name = name
@@ -263,6 +264,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
original_decode_video = self.module._decode_video original_decode_video = self.module._decode_video
original_decode_audio = self.module._decode_audio original_decode_audio = self.module._decode_audio
original_cleanup = self.module._deep_cleanup original_cleanup = self.module._deep_cleanup
original_upscale = self.module._upscale_latent_video
original_copy_sample = self.module._copy_sample_latent
original_nested = getattr(self.module.comfy.nested_tensor, "NestedTensor", None) original_nested = getattr(self.module.comfy.nested_tensor, "NestedTensor", None)
try: try:
self.module.comfy.nested_tensor.NestedTensor = FakeNestedTensor self.module.comfy.nested_tensor.NestedTensor = FakeNestedTensor
@@ -277,6 +280,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
{"samples": FakeNestedTensor((FakeTensor("basev"), FakeTensor("basea")))}, {"samples": FakeNestedTensor((FakeTensor("basev"), FakeTensor("basea")))},
) )
self.module._evict_all_but = lambda *_args, **_kwargs: None self.module._evict_all_but = lambda *_args, **_kwargs: None
self.module._upscale_latent_video = lambda video, param: (FakeTensor("upv"), 8, 16)
self.module._copy_sample_latent = lambda sampled: sampled["samples"].unbind()
self.module._decode_video = lambda _vae, out_latent, *_args, **_kwargs: out_latent self.module._decode_video = lambda _vae, out_latent, *_args, **_kwargs: out_latent
self.module._decode_audio = lambda _vae, out_latent: out_latent self.module._decode_audio = lambda _vae, out_latent: out_latent
self.module._deep_cleanup = lambda: None self.module._deep_cleanup = lambda: None
@@ -298,18 +303,25 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
tiled=False, tiled=False,
sa=(123, 20, 1.0, "res_multistep", "simple", 1.0), sa=(123, 20, 1.0, "res_multistep", "simple", 1.0),
handoff=None, handoff=None,
detail_pass=True, latent_upscale_param={
detail_sampler_name="euler", "mode": "model",
detail_scheduler="beta", "model_name": "upscale.safetensors",
detail_steps=5, "device": "cpu",
detail_denoise=0.4, "precision": "fp16",
"sampler_name": "euler_ancestral",
"scheduler": "simple",
"steps": 2,
"denoise": 0.4,
"megapixels": 1.5,
},
) )
self.assertEqual(len(calls), 2) self.assertEqual(len(calls), 2)
self.assertIsNot(calls[1][0][8], first_out) self.assertIsNot(calls[1][0][8], first_out)
self.assertIs(calls[1][0][8]["samples"], first_out["samples"]) self.assertEqual(calls[1][0][8]["samples"].unbind()[0].name, "upv")
self.assertEqual(calls[1][0][4], "euler") self.assertEqual(calls[1][0][2], 2)
self.assertEqual(calls[1][0][5], "beta") self.assertEqual(calls[1][0][4], "euler_ancestral")
self.assertEqual(calls[1][0][5], "simple")
self.assertAlmostEqual(calls[1][1]["denoise"], 0.4) self.assertAlmostEqual(calls[1][1]["denoise"], 0.4)
self.assertEqual(result[1], first_out) self.assertEqual(result[1], first_out)
self.assertEqual(result[2][0].name, "v2") self.assertEqual(result[2][0].name, "v2")
@@ -323,12 +335,14 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.module._decode_video = original_decode_video self.module._decode_video = original_decode_video
self.module._decode_audio = original_decode_audio self.module._decode_audio = original_decode_audio
self.module._deep_cleanup = original_cleanup self.module._deep_cleanup = original_cleanup
self.module._upscale_latent_video = original_upscale
self.module._copy_sample_latent = original_copy_sample
if original_nested is None: if original_nested is None:
delattr(self.module.comfy.nested_tensor, "NestedTensor") delattr(self.module.comfy.nested_tensor, "NestedTensor")
else: else:
self.module.comfy.nested_tensor.NestedTensor = original_nested self.module.comfy.nested_tensor.NestedTensor = original_nested
def test_detail_pass_decodes_audio_before_video_and_cleans_up(self): def test_latent_upscale_decodes_audio_before_video_and_cleans_up(self):
class FakeTensor: class FakeTensor:
def __init__(self, name): def __init__(self, name):
self.name = name self.name = name
@@ -357,12 +371,16 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
original_decode_video = self.module._decode_video original_decode_video = self.module._decode_video
original_decode_audio = self.module._decode_audio original_decode_audio = self.module._decode_audio
original_cleanup = self.module._deep_cleanup original_cleanup = self.module._deep_cleanup
original_upscale = self.module._upscale_latent_video
original_copy_sample = self.module._copy_sample_latent
original_unload = getattr(self.module.mm, "unload_model_and_clones", None)
original_unload_all = self.module.mm.unload_all_models
original_nested = getattr(self.module.comfy.nested_tensor, "NestedTensor", None) original_nested = getattr(self.module.comfy.nested_tensor, "NestedTensor", None)
try: try:
self.module.comfy.nested_tensor.NestedTensor = FakeNestedTensor self.module.comfy.nested_tensor.NestedTensor = FakeNestedTensor
def common_ksampler(*args, **kwargs): def common_ksampler(*args, **kwargs):
order.append("detail_sample" if len(order) else "sample") order.append("latent_upscale_sample" if len(order) else "sample")
return (first_out if len([x for x in order if x.endswith("sample")]) == 1 else second_out,) return (first_out if len([x for x in order if x.endswith("sample")]) == 1 else second_out,)
def decode_audio(_vae, out_latent): def decode_audio(_vae, out_latent):
@@ -379,12 +397,27 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
def cleanup(): def cleanup():
order.append("cleanup") order.append("cleanup")
def unload_model_and_clones(*_args, **_kwargs):
order.append("unload_h3_failed")
raise RuntimeError("model wrapper does not expose clone metadata")
def unload_all_models(*_args, **_kwargs):
order.append("unload_all")
def upscale_latent_video(video, param):
order.append("upscale")
return FakeTensor("upv"), 8, 16
self.module.nodes.common_ksampler = common_ksampler self.module.nodes.common_ksampler = common_ksampler
self.module._build_shot_conditioning = lambda *_args, **_kwargs: ( self.module._build_shot_conditioning = lambda *_args, **_kwargs: (
"cond", "cond",
{"samples": FakeNestedTensor((FakeTensor("basev"), FakeTensor("basea")))}, {"samples": FakeNestedTensor((FakeTensor("basev"), FakeTensor("basea")))},
) )
self.module._evict_all_but = lambda *_args, **_kwargs: None self.module._evict_all_but = lambda *_args, **_kwargs: None
self.module.mm.unload_model_and_clones = unload_model_and_clones
self.module.mm.unload_all_models = unload_all_models
self.module._upscale_latent_video = upscale_latent_video
self.module._copy_sample_latent = lambda sampled: sampled["samples"].unbind()
self.module._decode_video = decode_video self.module._decode_video = decode_video
self.module._decode_audio = decode_audio self.module._decode_audio = decode_audio
self.module._deep_cleanup = cleanup self.module._deep_cleanup = cleanup
@@ -406,15 +439,22 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
tiled=False, tiled=False,
sa=(123, 20, 1.0, "res_multistep", "simple", 1.0), sa=(123, 20, 1.0, "res_multistep", "simple", 1.0),
handoff=None, handoff=None,
detail_pass=True, latent_upscale_param={
detail_sampler_name="euler", "mode": "model",
detail_scheduler="beta", "model_name": "upscale.safetensors",
detail_steps=5, "device": "cuda",
detail_denoise=0.4, "precision": "fp16",
"sampler_name": "euler_ancestral",
"scheduler": "simple",
"steps": 2,
"denoise": 0.4,
"megapixels": 1.5,
},
) )
self.assertEqual(order[0], "sample") self.assertEqual(order[0], "sample")
self.assertEqual(order[1], "detail_sample") self.assertLess(order.index("unload_all"), order.index("upscale"))
self.assertLess(order.index("upscale"), order.index("latent_upscale_sample"))
self.assertLess(order.index("audio"), order.index("video")) self.assertLess(order.index("audio"), order.index("video"))
self.assertEqual(order[-1], "cleanup") self.assertEqual(order[-1], "cleanup")
finally: finally:
@@ -424,12 +464,19 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.module._decode_video = original_decode_video self.module._decode_video = original_decode_video
self.module._decode_audio = original_decode_audio self.module._decode_audio = original_decode_audio
self.module._deep_cleanup = original_cleanup self.module._deep_cleanup = original_cleanup
self.module._upscale_latent_video = original_upscale
self.module._copy_sample_latent = original_copy_sample
if original_unload is None:
delattr(self.module.mm, "unload_model_and_clones")
else:
self.module.mm.unload_model_and_clones = original_unload
self.module.mm.unload_all_models = original_unload_all
if original_nested is None: if original_nested is None:
delattr(self.module.comfy.nested_tensor, "NestedTensor") delattr(self.module.comfy.nested_tensor, "NestedTensor")
else: else:
self.module.comfy.nested_tensor.NestedTensor = original_nested self.module.comfy.nested_tensor.NestedTensor = original_nested
def test_detail_pass_treats_falsey_strings_as_disabled(self): def test_latent_upscale_off_skips_second_pass(self):
calls = [] calls = []
original_common_ksampler = self.module.nodes.common_ksampler original_common_ksampler = self.module.nodes.common_ksampler
original_build = self.module._build_shot_conditioning original_build = self.module._build_shot_conditioning
@@ -437,6 +484,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
original_decode_video = self.module._decode_video original_decode_video = self.module._decode_video
original_decode_audio = self.module._decode_audio original_decode_audio = self.module._decode_audio
original_cleanup = self.module._deep_cleanup original_cleanup = self.module._deep_cleanup
original_upscale = self.module._upscale_latent_video
original_copy_sample = self.module._copy_sample_latent
try: try:
self.module.nodes.common_ksampler = lambda *args, **kwargs: (calls.append((args, kwargs)) or {"samples": "latent"},) self.module.nodes.common_ksampler = lambda *args, **kwargs: (calls.append((args, kwargs)) or {"samples": "latent"},)
self.module._build_shot_conditioning = lambda *_args, **_kwargs: ("cond", {"samples": "base"}) self.module._build_shot_conditioning = lambda *_args, **_kwargs: ("cond", {"samples": "base"})
@@ -444,6 +493,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.module._decode_video = lambda _vae, out_latent, *_args, **_kwargs: out_latent self.module._decode_video = lambda _vae, out_latent, *_args, **_kwargs: out_latent
self.module._decode_audio = lambda _vae, out_latent: out_latent self.module._decode_audio = lambda _vae, out_latent: out_latent
self.module._deep_cleanup = lambda: None self.module._deep_cleanup = lambda: None
self.module._upscale_latent_video = lambda *_args, **_kwargs: (_ for _ in ()).throw(RuntimeError("should not run"))
self.module._copy_sample_latent = lambda sampled: sampled
self.module.H3LongVideos()._render( self.module.H3LongVideos()._render(
model=object(), model=object(),
@@ -459,7 +510,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
tiled=False, tiled=False,
sa=(123, 20, 1.0, "res_multistep", "simple", 1.0), sa=(123, 20, 1.0, "res_multistep", "simple", 1.0),
handoff=None, handoff=None,
detail_pass="false", latent_upscale_param={"mode": "off"},
) )
self.assertEqual(len(calls), 1) self.assertEqual(len(calls), 1)
@@ -470,6 +521,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.module._decode_video = original_decode_video self.module._decode_video = original_decode_video
self.module._decode_audio = original_decode_audio self.module._decode_audio = original_decode_audio
self.module._deep_cleanup = original_cleanup self.module._deep_cleanup = original_cleanup
self.module._upscale_latent_video = original_upscale
self.module._copy_sample_latent = original_copy_sample
def test_distribute_generations_canonicalizes_per_shot_audio_and_anchor_directives(self): def test_distribute_generations_canonicalizes_per_shot_audio_and_anchor_directives(self):
generations = self.module.distribute_generations( generations = self.module.distribute_generations(
@@ -736,6 +789,12 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.assertNotIn(f"ref_image_{index}", optional) self.assertNotIn(f"ref_image_{index}", optional)
self.assertNotIn("per_beat_length", optional) self.assertNotIn("per_beat_length", optional)
self.assertNotIn("cleanup_between_shots", optional) self.assertNotIn("cleanup_between_shots", optional)
self.assertNotIn("detail_pass", optional)
self.assertNotIn("detail_sampler_name", optional)
self.assertNotIn("detail_scheduler", optional)
self.assertNotIn("detail_steps", optional)
self.assertNotIn("detail_denoise", optional)
self.assertIn("latent_upscale_param", optional)
def test_shot_seconds_tooltip_describes_ceiling_behavior(self): def test_shot_seconds_tooltip_describes_ceiling_behavior(self):
optional = self.module.H3LongVideos.INPUT_TYPES()["optional"] optional = self.module.H3LongVideos.INPUT_TYPES()["optional"]
@@ -743,7 +802,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.assertIn("GLOBAL per-shot maximum", tooltip) self.assertIn("GLOBAL per-shot maximum", tooltip)
self.assertIn("A beat's own `seconds:` directive can still ask for less", tooltip) self.assertIn("A beat's own `seconds:` directive can still ask for less", tooltip)
self.assertIn("honoring it; may spill to system RAM (slow) or OOM", tooltip) self.assertIn("let the render fail instead of shrinking it", tooltip)
def test_resolve_shot_frames_honors_forced_request_over_budget(self): def test_resolve_shot_frames_honors_forced_request_over_budget(self):
original_estimate_shot_frames = self.module.estimate_shot_frames original_estimate_shot_frames = self.module.estimate_shot_frames
@@ -928,12 +987,23 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
plan_only=True, plan_only=True,
ref_1={"image": "live-1"}, ref_1={"image": "live-1"},
ref_3={"image": "live-3"}, ref_3={"image": "live-3"},
latent_upscale_param={
"mode": "interp",
"method": "bilinear",
"sampler_name": "euler_ancestral",
"scheduler": "simple",
"steps": 2,
"denoise": 0.2,
"megapixels": 1.5,
},
) )
self.assertEqual(calls["refs"][0]["image"], "live-1") self.assertEqual(calls["refs"][0]["image"], "live-1")
self.assertIsNone(calls["refs"][1]) self.assertIsNone(calls["refs"][1])
self.assertEqual(calls["refs"][2]["image"], "live-3") self.assertEqual(calls["refs"][2]["image"], "live-3")
self.assertEqual(result[2].count("ref2va: 2 reference image(s)"), 1) self.assertEqual(result[2].count("ref2va: 2 reference image(s)"), 1)
self.assertIn("latent upscale:", result[2])
self.assertIn("euler_ancestral/simple", result[2])
finally: finally:
self.module.parse_resolution = original_parse_resolution self.module.parse_resolution = original_parse_resolution
self.module._connected_refs = original_connected_refs self.module._connected_refs = original_connected_refs
@@ -1061,6 +1131,168 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
{"DumasH3LongVideos": "Dumas H3 Long Videos (FL2VA + REF2VA)"}, {"DumasH3LongVideos": "Dumas H3 Long Videos (FL2VA + REF2VA)"},
) )
def test_latent_upscale_params_node_is_exposed(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
required = latent.H3LatentUpscaleParams.INPUT_TYPES()["required"]
self.assertEqual(
latent.NODE_CLASS_MAPPINGS,
{"DumasH3LatentUpscaleParams": latent.H3LatentUpscaleParams},
)
self.assertEqual(
latent.NODE_DISPLAY_NAME_MAPPINGS,
{"DumasH3LatentUpscaleParams": "Dumas H3 Latent Upscale Params"},
)
self.assertEqual(required["sampler_name"][1]["default"], "euler_ancestral")
self.assertEqual(required["scheduler"][1]["default"], "simple")
self.assertEqual(required["steps"][1]["default"], 2)
self.assertEqual(required["denoise"][1]["default"], 0.2)
self.assertEqual(required["megapixels"][1]["default"], 1.0)
self.assertEqual(required["tile_width"][1]["default"], 512)
self.assertEqual(required["tile_height"][1]["default"], 512)
self.assertEqual(required["overlap"][1]["default"], 64)
self.assertEqual(required["fade_width"][1]["default"], 32)
self.assertEqual(required["fade_height"][1]["default"], 32)
self.assertEqual(required["overlap_mode"][1]["default"], "earlier")
self.assertEqual(required["overlap_blend"][1]["default"], "linear")
self.assertEqual(required["tile_size_mode"][1]["default"], "specific_size")
self.assertEqual(required["grid_rows"][1]["default"], 2)
self.assertEqual(required["grid_cols"][1]["default"], 2)
self.assertEqual(required["spatial_w_overlap"][1]["default"], 128)
self.assertEqual(required["spatial_h_overlap"][1]["default"], 128)
self.assertEqual(required["min_tile_size"][1]["default"], 256)
self.assertEqual(required["masked_area_noise"][1]["default"], 0.0)
self.assertFalse(required["brightness_match"][1]["default"])
self.assertEqual(required["dynamic_fade"][1]["default"], "off")
self.assertEqual(required["dynamic_fade_min"][1]["default"], 32)
self.assertEqual(required["chunk_length"][1]["default"], 17)
self.assertEqual(required["temporal_overlap"][1]["default"], 0)
self.assertFalse(required["resize_conditioning"][1]["default"])
self.assertEqual(required["anchor_strength"][1]["default"], 0.999)
def test_latent_upscale_mode_infers_legacy_model_payloads(self):
self.assertEqual(self.module._latent_upscale_mode({"model_name": "foo.safetensors"}), "model")
self.assertEqual(self.module._latent_upscale_mode({"method": "bilinear"}), "interp")
self.assertEqual(self.module._latent_upscale_mode({"mode": "model"}), "model")
self.assertEqual(self.module._latent_upscale_mode({}), "off")
def test_tag_oom_stage_marks_oom_exceptions(self):
exc = RuntimeError("CUDA out of memory")
tagged = self.module._tag_oom_stage(exc, "latent_upscale")
self.assertIs(tagged, exc)
self.assertEqual(getattr(tagged, "_h3_stage", ""), "latent_upscale")
def test_shrink_model_tile_param_reduces_tile_size(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
smaller = latent._shrink_model_tile_param({
"tile_size_mode": "specific_size",
"tile_width": 512,
"tile_height": 512,
"overlap": 64,
"fade_width": 32,
"fade_height": 32,
})
self.assertIsNotNone(smaller)
self.assertEqual(smaller["tile_size_mode"], "rows_cols")
self.assertEqual(smaller["grid_rows"], 4)
self.assertEqual(smaller["grid_cols"], 4)
self.assertEqual(smaller["spatial_w_overlap"], 0)
self.assertEqual(smaller["spatial_h_overlap"], 0)
self.assertEqual(smaller["fade_width"], 0)
self.assertEqual(smaller["fade_height"], 0)
self.assertEqual(smaller["min_tile_size"], 32)
def test_shrink_model_tile_param_rows_cols_resets_overlap(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
smaller = latent._shrink_model_tile_param({
"tile_size_mode": "rows_cols",
"grid_rows": 4,
"grid_cols": 4,
"spatial_w_overlap": 128,
"spatial_h_overlap": 128,
"fade_width": 64,
"fade_height": 64,
"min_tile_size": 256,
})
self.assertIsNotNone(smaller)
self.assertEqual(smaller["grid_rows"], 8)
self.assertEqual(smaller["grid_cols"], 8)
self.assertEqual(smaller["spatial_w_overlap"], 0)
self.assertEqual(smaller["spatial_h_overlap"], 0)
self.assertEqual(smaller["fade_width"], 0)
self.assertEqual(smaller["fade_height"], 0)
self.assertEqual(smaller["min_tile_size"], 32)
def test_shrink_model_tile_param_rows_cols_can_reach_thirty_two(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
smaller = latent._shrink_model_tile_param({
"tile_size_mode": "rows_cols",
"grid_rows": 16,
"grid_cols": 16,
"spatial_w_overlap": 0,
"spatial_h_overlap": 0,
"fade_width": 0,
"fade_height": 0,
"min_tile_size": 32,
})
self.assertIsNotNone(smaller)
self.assertEqual(smaller["grid_rows"], 32)
self.assertEqual(smaller["grid_cols"], 32)
def test_temporal_segments_split_long_sequences(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
bounds = latent._temporal_segments(36, 85, 17)
self.assertGreater(len(bounds), 1)
self.assertEqual(bounds[0][0], 0)
self.assertEqual(bounds[-1][2], 36)
def test_shrink_temporal_param_reduces_chunk_length(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
smaller = latent._shrink_temporal_param({
"chunk_length": 85,
"temporal_overlap": 17,
})
self.assertIsNotNone(smaller)
self.assertEqual(smaller["chunk_length"], 17)
self.assertEqual(smaller["temporal_overlap"], 0)
def test_cuda_model_temporal_params_cap_saved_workflows(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
chunk_length, temporal_overlap = latent._effective_temporal_params({
"mode": "model",
"device": "cuda",
"chunk_length": 85,
"temporal_overlap": 17,
})
self.assertEqual(chunk_length, 17)
self.assertEqual(temporal_overlap, 0)
def test_interp_temporal_params_preserve_upstream_defaults(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
chunk_length, temporal_overlap = latent._effective_temporal_params({
"mode": "interp",
"device": "cuda",
"chunk_length": 85,
"temporal_overlap": 17,
})
self.assertEqual(chunk_length, 85)
self.assertEqual(temporal_overlap, 17)
def test_upscale_video_model_raises_when_gpu_cannot_shrink(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
original_tiled = latent._upscale_video_model_tiled
original_shrink = latent._shrink_model_tile_param
try:
latent._shrink_model_tile_param = lambda _param: None
latent._upscale_video_model_tiled = lambda *_args, **_kwargs: (_ for _ in ()).throw(RuntimeError("out of memory"))
with self.assertRaisesRegex(RuntimeError, "H3 latent upscale exhausted its GPU spatial fallbacks"):
latent.upscale_video_model("video", {"device": "cuda", "precision": "fp16"})
finally:
latent._upscale_video_model_tiled = original_tiled
latent._shrink_model_tile_param = original_shrink
def test_compose_persistent_does_not_expand_ambiguous_plural_to_full_cast(self): def test_compose_persistent_does_not_expand_ambiguous_plural_to_full_cast(self):
active = self.module.parse_wardrobe( active = self.module.parse_wardrobe(
"Maya = she, red jacket\n" "Maya = she, red jacket\n"