Compare commits

...
17 Commits
Author SHA1 Message Date
chris.dumas e973a0d785 Expose H3 prompt curator components 2026-09-09 15:34:06 +00:00
chris.dumas 388de1d837 Sync helper preset text boxes 2026-09-07 13:26:29 +00:00
chris.dumas 338150648b Add BGM helper for H3 prompt curator 2026-09-07 08:13:45 +00:00
chris.dumas 7de62226f2 Strip legacy anchor style note 2026-09-07 08:01:22 +00:00
chris.dumas a51141cb26 Add location helper reference outputs 2026-09-07 07:46:24 +00:00
chris.dumas dd8ef84379 Add character helper reference outputs 2026-09-05 18:00:17 +00:00
chris.dumas d20b257134 Add H3 prompt subject count guard 2026-09-05 17:54:43 +00:00
chris.dumas 173205ca51 Add H3 prompt curator 2026-09-05 16:43:05 +00:00
chris.dumas 98833ba99d Restore general character and location helpers 2026-09-04 15:52:40 +00:00
chris.dumas a7f7225b3a Bypass learned latent upscaler on low VRAM 2026-09-04 15:24:30 +00:00
chris.dumas 5dc5c2ce8b Prioritize temporal fallback for latent upscale OOM 2026-09-04 15:10:38 +00:00
chris.dumas 83645811f4 Reuse conditioning for refinement tiles 2026-09-04 14:52:43 +00:00
chris.dumas 216bc05762 Retry temporal split after latent upscale OOM 2026-09-04 14:43:25 +00:00
chris.dumas 49e099ee7f Log H3 latent upscale tiling plan 2026-09-04 14:14:24 +00:00
chris.dumas 32a16645b5 Use adaptive CUDA latent upscale chunking 2026-09-04 12:09:59 +00:00
chris.dumas 7bb0b0abea Reuse conditioning for latent upscale refine 2026-09-04 11:49:21 +00:00
chris.dumas 0b189dcf7a Keep latent upscaler loaded during pass 2026-09-04 11:04:26 +00:00
7 changed files with 1916 additions and 80 deletions
+32 -2
View File
@@ -56,6 +56,14 @@
- Output: `prompt` - Output: `prompt`
- Builds one H3 prompt block per beat, with quick controls for per-shot timing, continuity, ref behavior, anchor additions, soundscape, and music while staying compatible with direct text editing. - Builds one H3 prompt block per beat, with quick controls for per-shot timing, continuity, ref behavior, anchor additions, soundscape, and music while staying compatible with direct text editing.
- `Dumas H3 Prompt Curator`
- Inputs: `action_prompt`, `anatomy_guard`, `subject_count_guard`, optional `anchor`, optional `soundscape`, optional `bgm`, optional `ref_1` through `ref_9`
- Outputs: `prompt`, `ref_image_1` through `ref_image_9`, `reference_count`, `debug`, `anchor`, `sounds`, `bgm`, `original_ref_1` through `original_ref_9`, `reference_description`
- Builds one standalone MiniMax H3 prompt from your final action text plus structured character/location references.
- The action text can mention references by character/location name, alias, `<Picture N>`, or `<refN>`. Only mentioned references are emitted, and the output images are compacted/renumbered so skipped inputs do not leave gaps.
- Extra component outputs expose the cleaned anchor, sounds, BGM, selected structured references in compacted order, and the compiled reference-description block used inside the prompt.
- Adds curated reference context, anatomy guard text, optional subject-count guard text, anchor/style text, `overall_soundscape:` text, and `background_music:` text while respecting MiniMax H3's reference-generation shape: one prompt plus up to nine reference images.
- `Dumas H3 Shot Length` - `Dumas H3 Shot Length`
- Inputs: `shot_seconds`, `fps`, optional `cap_to_h3_max` - Inputs: `shot_seconds`, `fps`, optional `cap_to_h3_max`
- Outputs: `seconds`, `frames`, `info` - Outputs: `seconds`, `frames`, `info`
@@ -76,12 +84,34 @@
- Output: `reference` - Output: `reference`
- Builds one structured `REFERENCE` object for a location/environment so H3 can use the same socket type for both character and scenic refs. - Builds one structured `REFERENCE` object for a location/environment so H3 can use the same socket type for both character and scenic refs.
- `Dumas Character Helper`
- Inputs: `image1`, `image2`, picture IDs, character identity fields, `general`, `wardrobe`
- Outputs: `image1`, `image2`, `reference_prompt`, `wardrobe`, `reference1`, `reference2`
- Restores the original general-purpose helper shape while also emitting two structured `REFERENCE` objects for the prompt curator.
- The structured references carry the same character name, alias, age, height, gender, nationality, occupation, accent, wardrobe, and notes, so mentioning the character name in `Dumas H3 Prompt Curator` can include both helper images and the character facts automatically.
- `Dumas Location Helper`
- Inputs: `image1`, `image2`, picture IDs, `location_id`, `name`, `alias`, `description`, `general`
- Outputs: `image1`, `image2`, `reference_prompt`, `reference1`, `reference2`
- Matching general-purpose helper for environments/locations: pass two images through unchanged, emit location reference prompt text, and provide two structured `REFERENCE` objects for the prompt curator.
- The structured references carry the same location name, alias, description, and notes, so mentioning the location name in `Dumas H3 Prompt Curator` can include both helper images and the location context automatically.
- `Dumas Anchor Style` - `Dumas Anchor Style`
- Inputs: `anchor_style`, `style_description` - Inputs: `anchor_style`, `style_description`
- Output: `anchor` - Output: `anchor`
- Offers a large preset dropdown of anchor-style titles such as cinematic action movie, comedy, found footage, 90s sitcom, mobile/cell phone captured, news broadcast, mockumentary, heist thriller, cyberpunk neon, nature documentary, courtroom drama, and more. - Offers a large preset dropdown of anchor-style titles such as cinematic action movie, comedy, found footage, 90s sitcom, mobile/cell phone captured, news broadcast, mockumentary, heist thriller, cyberpunk neon, nature documentary, courtroom drama, and more.
- The preset wording is tuned for H3-safe persistent anchors: camera language, lighting, texture, production treatment, and tone, without naming characters or describing one-off actions. - The preset wording is tuned for H3-safe persistent anchors: camera language, lighting, texture, production treatment, and tone, without naming characters or describing one-off actions.
- Selecting a preset fills the editable description field, and the edited multiline description is the `STRING` value passed downstream into H3 anchor sockets such as `anchor_override`. - Selecting a preset fills the editable description field, and the edited multiline description is the `STRING` value passed downstream.
- `Dumas Soundscape Helper`
- Inputs: `soundscape`, `soundscape_description`
- Output: `soundscape`
- Matching soundscape helper for standalone H3 prompts. Pick a preset such as quiet interior, rainy street, cafe, city night, forest, industrial, or silent; the preset fills the editable textbox, and the edited text flows into `Dumas H3 Prompt Curator`.
- `Dumas Background Music Helper`
- Inputs: `bgm`, `bgm_description`
- Output: `bgm`
- Matching BGM helper for standalone H3 prompts. Pick a preset such as subtle tension, cinematic suspense, emotional piano, dark ambient, hopeful orchestral, retro synth, action pulse, lo-fi, or no vocals; the preset fills the editable textbox, and the edited text flows into `Dumas H3 Prompt Curator`.
- `Dumas JSON String to Object` - `Dumas JSON String to Object`
- Input: `json_string` - Input: `json_string`
@@ -239,7 +269,7 @@ decr -> use index - 1
`Dumas H3 Plan Attach Scene Images` and `Dumas H3 Plan Extract Scene Images` are a companion pair for `ComfyUI-MiniMaxH3-Contex-Loop` and the local `ref2v` lane. The upstream H3 plan node cannot dynamically grow nine new image sockets for every JSON-defined scene, so Dumas stores scene image bindings beside the plan using a lightweight token and an in-memory registry. That keeps `plan.json` archiving intact while still letting you wire up nine IMAGE sockets per scene through chained helper nodes. `Dumas H3 Plan Attach Scene Images` and `Dumas H3 Plan Extract Scene Images` are a companion pair for `ComfyUI-MiniMaxH3-Contex-Loop` and the local `ref2v` lane. The upstream H3 plan node cannot dynamically grow nine new image sockets for every JSON-defined scene, so Dumas stores scene image bindings beside the plan using a lightweight token and an in-memory registry. That keeps `plan.json` archiving intact while still letting you wire up nine IMAGE sockets per scene through chained helper nodes.
`Dumas Character Reference` and `Dumas Location Reference` live in `Dumas/MiniMax`. Both output a structured `REFERENCE` object that carries the image plus its semantic payload. `Dumas H3 Long Videos` accepts those `REFERENCE` sockets directly on `ref_1`..`ref_9`, resolves `<Picture N>` against the wired slot positions, and can also pull character wardrobe context from the structured ref data when `character_memory` is left blank. `Dumas Character Helper` is the restored two-image/text helper for general H3 workflows, and `Dumas Location Helper` mirrors it for scene/environment references. Both helpers also emit structured `REFERENCE` sockets for the curator. The structured `Dumas Character Reference` and `Dumas Location Reference` nodes remain available separately for workflows that want a single `REFERENCE` socket. `Dumas H3 Prompt Curator` consumes those structured references plus optional anchor, soundscape, and BGM strings, assigns the final `<Picture N>` numbering, and outputs only the compacted images the prompt actually mentions.
`Dumas Strip Iteration Suffix` keeps the part before the first underscore and drops the rest. Names like `char123_pose_final.png` become `char123.png`, while names with no underscore such as `char123.png` are left untouched. `Dumas Strip Iteration Suffix` keeps the part before the first underscore and drops the rest. Names like `char123_pose_final.png` become `char123.png`, while names with no underscore such as `char123.png` are left untouched.
+164 -29
View File
@@ -1,6 +1,8 @@
from contextlib import contextmanager
from functools import lru_cache from functools import lru_cache
import gc import gc
import glob import glob
import logging
import math import math
import os import os
import re import re
@@ -40,9 +42,12 @@ LATENTS_STD = [
] ]
_LATENT_UPSCALE_FOLDER = "latent_upscale_models" _LATENT_UPSCALE_FOLDER = "latent_upscale_models"
LOGGER = logging.getLogger(__name__)
MP_UNIT = 1024 * 1024 MP_UNIT = 1024 * 1024
RES_MULTIPLE = 32 RES_MULTIPLE = 32
CUDA_MODEL_CHUNK_LENGTH = 17 CUDA_MODEL_CHUNK_LENGTH = 17
CUDA_MODEL_MIN_TOTAL_VRAM_BYTES = 10 * 1024 * 1024 * 1024
_MODEL_HOLD_DEPTH = 0
def _uses_cuda_model_upscale(param): def _uses_cuda_model_upscale(param):
@@ -53,15 +58,83 @@ def _uses_cuda_model_upscale(param):
return device == "cuda" and (mode == "model" or has_model_name) return device == "cuda" and (mode == "model" or has_model_name)
def _effective_temporal_params(param): def _effective_temporal_params(param, frame_count=None):
chunk_length = int(param.get("chunk_length", 0) or 0) chunk_length = int(param.get("chunk_length", 0) or 0)
temporal_overlap = int(param.get("temporal_overlap", 0) or 0) temporal_overlap = int(param.get("temporal_overlap", 0) or 0)
if _uses_cuda_model_upscale(param): if _uses_cuda_model_upscale(param):
chunk_length = CUDA_MODEL_CHUNK_LENGTH if chunk_length <= 0 else min(chunk_length, CUDA_MODEL_CHUNK_LENGTH) if chunk_length <= 0:
temporal_overlap = min(max(0, temporal_overlap), max(0, chunk_length - 17)) chunk_length = 85
temporal_overlap = 17
return chunk_length, temporal_overlap return chunk_length, temporal_overlap
def _is_oom_error(exc):
text = str(exc).lower()
return "out of memory" in text or "exhausted its gpu spatial fallbacks" in text
def _should_retry_temporal_before_spatial(param):
return _uses_cuda_model_upscale(param) and int(param.get("chunk_length", 0) or 0) > CUDA_MODEL_CHUNK_LENGTH
def _cuda_total_memory_bytes():
try:
if not torch.cuda.is_available():
return 0
if hasattr(torch.cuda, "mem_get_info"):
_free, total = torch.cuda.mem_get_info()
return int(total)
current_device = torch.cuda.current_device() if hasattr(torch.cuda, "current_device") else 0
props = torch.cuda.get_device_properties(current_device)
return int(getattr(props, "total_memory", 0) or 0)
except Exception:
return 0
def _should_skip_cuda_model_upscale(param):
total = _cuda_total_memory_bytes()
return _uses_cuda_model_upscale(param) and 0 < total < CUDA_MODEL_MIN_TOTAL_VRAM_BYTES
def _fallback_to_interp(video, param, reason):
method = param.get("method", "bilinear")
LOGGER.warning("H3 latent upscale model %s; using %s interpolation instead", reason, method)
try:
gc.collect()
except Exception:
pass
if torch.cuda.is_available():
try:
torch.cuda.empty_cache()
except Exception:
pass
interp_param = dict(param)
interp_param["mode"] = "interp"
return upscale_video_interp(video, interp_param)
def _retry_with_smaller_temporal(video, param, upscaler, exc):
if not _is_oom_error(exc):
raise exc
smaller = _shrink_temporal_param(param)
if smaller is None:
raise exc
LOGGER.info(
"H3 latent upscale temporal OOM: retrying with chunk_length=%s overlap=%s",
smaller.get("chunk_length"), smaller.get("temporal_overlap"),
)
try:
gc.collect()
except Exception:
pass
if torch.cuda.is_available():
try:
torch.cuda.empty_cache()
except Exception:
pass
return _upscale_video_temporal_chunks(video, smaller, upscaler)
def _models_dir(): def _models_dir():
try: try:
if _LATENT_UPSCALE_FOLDER not in folder_paths.folder_names_and_paths: if _LATENT_UPSCALE_FOLDER not in folder_paths.folder_names_and_paths:
@@ -349,7 +422,7 @@ def load_upscale_model(name, device, precision):
return model return model
def unload_upscale_model(name, device, precision): def _unload_upscale_model_now(name, device, precision):
cache_key = f"{name}::{device}::{precision}" cache_key = f"{name}::{device}::{precision}"
model = _MODEL_CACHE.get(cache_key) model = _MODEL_CACHE.get(cache_key)
if model is not None and str(next(model.parameters()).device) != "cpu": if model is not None and str(next(model.parameters()).device) != "cpu":
@@ -361,6 +434,22 @@ def unload_upscale_model(name, device, precision):
pass pass
def unload_upscale_model(name, device, precision):
if _MODEL_HOLD_DEPTH > 0:
return
_unload_upscale_model_now(name, device, precision)
@contextmanager
def _hold_upscale_model_loaded():
global _MODEL_HOLD_DEPTH
_MODEL_HOLD_DEPTH += 1
try:
yield
finally:
_MODEL_HOLD_DEPTH -= 1
def _compute_upscale_target(width, height, h_in, w_in): def _compute_upscale_target(width, height, h_in, w_in):
ds = 16 ds = 16
w_px = float(width) w_px = float(width)
@@ -617,12 +706,25 @@ def upscale_video_model(video, param):
except RuntimeError as exc: except RuntimeError as exc:
if "out of memory" not in str(exc).lower(): if "out of memory" not in str(exc).lower():
raise raise
if _should_retry_temporal_before_spatial(param):
raise RuntimeError(
"out of memory: retry H3 latent upscale with a smaller temporal chunk "
"before spatial fallback"
) from exc
smaller = _shrink_model_tile_param(param) smaller = _shrink_model_tile_param(param)
if smaller is None: if smaller is None:
raise RuntimeError( raise RuntimeError(
"H3 latent upscale exhausted its GPU spatial fallbacks. " "H3 latent upscale exhausted its GPU spatial fallbacks. "
"Reduce the target size, tile size, or split the shot earlier." "Reduce the target size, tile size, or split the shot earlier."
) from exc ) from exc
LOGGER.info(
"H3 latent upscale model OOM: retrying with tile_size_mode=%s rows=%s cols=%s tile=%sx%s",
smaller.get("tile_size_mode"),
smaller.get("grid_rows"),
smaller.get("grid_cols"),
smaller.get("tile_width"),
smaller.get("tile_height"),
)
try: try:
gc.collect() gc.collect()
except Exception: except Exception:
@@ -683,12 +785,22 @@ def _upscale_video_model_tiled(video, param):
# If the requested tile is not smaller than the target on either axis, # If the requested tile is not smaller than the target on either axis,
# the tiled path would just duplicate work. # the tiled path would just duplicate work.
if len(rows) == 1 and len(cols) == 1: if len(rows) == 1 and len(cols) == 1:
LOGGER.info(
"H3 latent upscale model: single core pass, tokens=%s target=%sx%s",
int(video.shape[2]), w_out, h_out,
)
return _upscale_video_model_core(video, param) return _upscale_video_model_core(video, param)
scale_h = h_out / float(h_in) scale_h = h_out / float(h_in)
scale_w = w_out / float(w_in) scale_w = w_out / float(w_in)
orig_dtype = video.dtype orig_dtype = video.dtype
out = torch.zeros((video.shape[0], video.shape[1], video.shape[2], h_out, w_out), device="cpu", dtype=orig_dtype) out = torch.zeros((video.shape[0], video.shape[1], video.shape[2], h_out, w_out), device="cpu", dtype=orig_dtype)
LOGGER.info(
"H3 latent upscale model: %s spatial tiles, tokens=%s target=%sx%s tile_mode=%s tile=%sx%s overlap=%sx%s",
len(rows) * len(cols), int(video.shape[2]), w_out, h_out, mode, tile_w, tile_h,
spatial_w_overlap if mode == "rows_cols" else overlap,
spatial_h_overlap if mode == "rows_cols" else overlap,
)
for i, r0 in enumerate(rows): for i, r0 in enumerate(rows):
tr = trows[i] tr = trows[i]
@@ -753,43 +865,51 @@ def upscale_video_interp(video, param):
def _upscale_video_temporal_chunks(video, param, upscaler): def _upscale_video_temporal_chunks(video, param, upscaler):
if video.device.type != "cpu": if video.device.type != "cpu":
video = video.to(device="cpu", copy=True) video = video.to(device="cpu", copy=True)
chunk_length, temporal_overlap = _effective_temporal_params(param) t = int(video.shape[2])
frame_count = _frames_for_tokens(t)
chunk_length, temporal_overlap = _effective_temporal_params(param, frame_count)
chunk_param = dict(param) chunk_param = dict(param)
chunk_param["chunk_length"] = chunk_length chunk_param["chunk_length"] = chunk_length
chunk_param["temporal_overlap"] = temporal_overlap chunk_param["temporal_overlap"] = temporal_overlap
anchor_strength = float(param.get("anchor_strength", 0.999) or 0.999) anchor_strength = float(param.get("anchor_strength", 0.999) or 0.999)
t = int(video.shape[2])
frame_count = _frames_for_tokens(t)
if chunk_length <= 0 or frame_count <= chunk_length: if chunk_length <= 0 or frame_count <= chunk_length:
return upscaler(video, chunk_param) LOGGER.info(
"H3 latent upscale: single temporal chunk, tokens=%s frames=%s chunk_length=%s overlap=%s",
t, frame_count, chunk_length, temporal_overlap,
)
try:
return upscaler(video, chunk_param)
except RuntimeError as exc:
return _retry_with_smaller_temporal(video, chunk_param, upscaler, exc)
bounds = _temporal_segments(t, chunk_length, temporal_overlap) bounds = _temporal_segments(t, chunk_length, temporal_overlap)
if len(bounds) <= 1: if len(bounds) <= 1:
return upscaler(video, chunk_param) LOGGER.info(
"H3 latent upscale: single temporal segment, tokens=%s frames=%s chunk_length=%s overlap=%s",
t, frame_count, chunk_length, temporal_overlap,
)
try:
return upscaler(video, chunk_param)
except RuntimeError as exc:
return _retry_with_smaller_temporal(video, chunk_param, upscaler, exc)
orig_dtype = video.dtype orig_dtype = video.dtype
out = None out = None
out_h = out_w = None out_h = out_w = None
LOGGER.info(
"H3 latent upscale: %s temporal chunks, tokens=%s frames=%s chunk_length=%s overlap=%s",
len(bounds), t, frame_count, chunk_length, temporal_overlap,
)
for i, (k0, f0, k1, f1) in enumerate(bounds): for i, (k0, f0, k1, f1) in enumerate(bounds):
chunk = video[:, :, k0:k1].contiguous() chunk = video[:, :, k0:k1].contiguous()
LOGGER.info(
"H3 latent upscale: temporal chunk %s/%s tokens %s:%s frames %s:%s",
i + 1, len(bounds), k0, k1, f0, f1,
)
try: try:
chunk_out, chunk_h, chunk_w = upscaler(chunk, chunk_param) chunk_out, chunk_h, chunk_w = upscaler(chunk, chunk_param)
except RuntimeError as exc: except RuntimeError as exc:
if "out of memory" not in str(exc).lower(): return _retry_with_smaller_temporal(video, chunk_param, upscaler, exc)
raise
smaller = _shrink_temporal_param(chunk_param)
if smaller is None:
raise
try:
gc.collect()
except Exception:
pass
if torch.cuda.is_available():
try:
torch.cuda.empty_cache()
except Exception:
pass
return _upscale_video_temporal_chunks(video, smaller, upscaler)
chunk_out = chunk_out.to(device="cpu", dtype=orig_dtype) chunk_out = chunk_out.to(device="cpu", dtype=orig_dtype)
if out is None: if out is None:
out_h, out_w = chunk_h, chunk_w out_h, out_w = chunk_h, chunk_w
@@ -819,7 +939,22 @@ def upscale_latent_video(video, param):
if mode == "off": if mode == "off":
return video, video.shape[-2], video.shape[-1] return video, video.shape[-2], video.shape[-1]
if mode == "model": if mode == "model":
return _upscale_video_temporal_chunks(video, param, upscale_video_model) if _should_skip_cuda_model_upscale(param):
return _fallback_to_interp(video, param, "requires more than this card's VRAM")
model_name = param.get("model_name")
device = param.get("device", "cuda")
precision = param.get("precision", "fp16")
dev = torch.device(device if (device == "cpu" or torch.cuda.is_available()) else "cpu")
try:
with _hold_upscale_model_loaded():
try:
return _upscale_video_temporal_chunks(video, param, upscale_video_model)
finally:
_unload_upscale_model_now(model_name, dev, precision)
except RuntimeError as exc:
if not _is_oom_error(exc):
raise
return _fallback_to_interp(video, param, "exhausted GPU memory")
return _upscale_video_temporal_chunks(video, param, upscale_video_interp) return _upscale_video_temporal_chunks(video, param, upscale_video_interp)
@@ -892,10 +1027,10 @@ class H3LatentUpscaleParams:
"tooltip": "Temporal fade schedule over each tile's sampling. Off keeps the fade fixed; narrowing shrinks it over steps; widening grows it over steps."}), "tooltip": "Temporal fade schedule over each tile's sampling. Off keeps the fade fixed; narrowing shrinks it over steps; widening grows it over steps."}),
"dynamic_fade_min": ("INT", {"default": 32, "min": 0, "max": 4096, "step": 32, "dynamic_fade_min": ("INT", {"default": 32, "min": 0, "max": 4096, "step": 32,
"tooltip": "Minimum fade width used by dynamic_fade when it is enabled."}), "tooltip": "Minimum fade width used by dynamic_fade when it is enabled."}),
"chunk_length": ("INT", {"default": 17, "min": 17, "max": 100000, "step": 17, "chunk_length": ("INT", {"default": 85, "min": 17, "max": 100000, "step": 17,
"tooltip": "Temporal chunk length for latent upscale. CUDA model upscale is capped to 17 internally so short long-video shots do not bypass splitting and OOM."}), "tooltip": "Temporal chunk length for latent upscale. CUDA model upscale keeps this when it splits the shot, but uses 17/0 if this would otherwise process the whole shot as one OOM-prone batch."}),
"temporal_overlap": ("INT", {"default": 0, "min": 0, "max": 100000, "step": 17, "temporal_overlap": ("INT", {"default": 17, "min": 0, "max": 100000, "step": 17,
"tooltip": "Temporal overlap between latent chunks. CUDA model upscale uses 0 when capped to one H3 block to minimize peak VRAM."}), "tooltip": "Temporal overlap between latent chunks. 17 matches the upstream split example; CUDA model upscale drops overlap only for the emergency 17-frame guard path."}),
"resize_conditioning": ("BOOLEAN", {"default": False, "resize_conditioning": ("BOOLEAN", {"default": False,
"tooltip": "Reserved for upstream split compatibility. Leave OFF unless you need the original fallback behavior."}), "tooltip": "Reserved for upstream split compatibility. Leave OFF unless you need the original fallback behavior."}),
"anchor_strength": ("FLOAT", {"default": 0.999, "min": 0.0, "max": 1.0, "step": 0.01, "anchor_strength": ("FLOAT", {"default": 0.999, "min": 0.0, "max": 1.0, "step": 0.01,
+102 -13
View File
@@ -4040,6 +4040,88 @@ def _copy_sample_latent(out_latent):
return None return None
def _retarget_conditioning_spatial(cond, latent_h, latent_w):
"""Resize H3 keyframe latents in existing conditioning to a new latent grid."""
latent_h = int(latent_h)
latent_w = int(latent_w)
if latent_h <= 0 or latent_w <= 0:
raise RuntimeError("conditioning target latent size must be positive")
out = []
for item in cond:
try:
tensor, data = item
except Exception:
out.append(item)
continue
nd = dict(data)
keyframes = nd.get("minimax_keyframes")
if keyframes:
resized_keyframes = []
for keyframe in keyframes:
nkf = dict(keyframe)
latent_value = nkf.get("latent")
if latent_value is not None and len(getattr(latent_value, "shape", ())) >= 5:
if latent_value.shape[3] != latent_h or latent_value.shape[4] != latent_w:
b, c, t, h, w = latent_value.shape
resized = torch.nn.functional.interpolate(
latent_value.to(torch.float32).reshape(b * t, c, h, w),
size=(latent_h, latent_w),
mode="bilinear",
align_corners=False,
).reshape(b, c, t, latent_h, latent_w)
nkf["latent"] = resized.to(device=latent_value.device, dtype=latent_value.dtype)
resized_keyframes.append(nkf)
nd["minimax_keyframes"] = resized_keyframes
out.append([tensor, nd])
return out
def _pad_to_h3_patch_size(tensor):
try:
import comfy.ldm.common_dit as common_dit
return common_dit.pad_to_patch_size(tensor, (1, 2, 2))
except Exception:
return tensor
def _crop_conditioning_to_tile(cond, source_h, source_w, row, col, tile_h, tile_w):
"""Crop H3 keyframe latents in existing conditioning for a spatial tile."""
out = []
for item in cond:
try:
tensor, data = item
except Exception:
out.append(item)
continue
nd = dict(data)
keyframes = nd.get("minimax_keyframes")
if keyframes:
cropped_keyframes = []
for keyframe in keyframes:
nkf = dict(keyframe)
latent_value = nkf.get("latent")
if latent_value is not None and len(getattr(latent_value, "shape", ())) >= 5:
kh, kw = latent_value.shape[3], latent_value.shape[4]
if kh != source_h or kw != source_w:
b, c, t, h, w = latent_value.shape
latent_value = torch.nn.functional.interpolate(
latent_value.to(torch.float32).reshape(b * t, c, h, w),
size=(source_h, source_w),
mode="bilinear",
align_corners=False,
).reshape(b, c, t, source_h, source_w).to(
device=latent_value.device,
dtype=latent_value.dtype,
)
nkf["latent"] = _pad_to_h3_patch_size(
latent_value[:, :, :, row:row + tile_h, col:col + tile_w].contiguous()
)
cropped_keyframes.append(nkf)
nd["minimax_keyframes"] = cropped_keyframes
out.append([tensor, nd])
return out
def _latent_with_replaced_samples(template_latent, sampled_latent): def _latent_with_replaced_samples(template_latent, sampled_latent):
"""Reuse the original latent payload, but swap in freshly sampled tensors.""" """Reuse the original latent payload, but swap in freshly sampled tensors."""
if not isinstance(template_latent, dict): if not isinstance(template_latent, dict):
@@ -6693,18 +6775,22 @@ class H3LongVideos:
# pass; otherwise the 12-step base latent and the upscale latent sit # pass; otherwise the 12-step base latent and the upscale latent sit
# in memory together and can trigger a retry loop. # in memory together and can trigger a retry loop.
out["samples"] = comfy.nested_tensor.NestedTensor((upscaled_video, full_audio)) out["samples"] = comfy.nested_tensor.NestedTensor((upscaled_video, full_audio))
del out_samples, positive, latent, parts del out_samples, parts
mm.soft_empty_cache() mm.soft_empty_cache()
target_w = int(up_w) * 16 target_w = int(up_w) * 16
target_h = int(up_h) * 16 target_h = int(up_h) * 16
if target_w <= 0 or target_h <= 0: if target_w <= 0 or target_h <= 0:
raise RuntimeError("latent upscale target size must be positive") raise RuntimeError("latent upscale target size must be positive")
upscale_cond, upscale_latent = _build_shot_conditioning( try:
clip, vae, prompt, target_w, target_h, ln, fps, handoff, upscale_cond = _retarget_conditioning_spatial(positive, int(up_h), int(up_w))
ref_images=refs, ref_image_size=ref_image_size, upscale_latent = dict(latent) if isinstance(latent, dict) else {}
ref_noise_aug=ref_noise_aug, audio_vae=audio_vae, silent=silent) except Exception:
upscale_latent["samples"] = comfy.nested_tensor.NestedTensor( upscale_cond, upscale_latent = _build_shot_conditioning(
(upscaled_video, full_audio)) clip, vae, prompt, target_w, target_h, ln, fps, handoff,
ref_images=refs, ref_image_size=ref_image_size,
ref_noise_aug=ref_noise_aug, audio_vae=audio_vae, silent=silent)
upscale_latent["samples"] = comfy.nested_tensor.NestedTensor((upscaled_video, full_audio))
del positive, latent
refine_steps = int(latent_upscale_param.get("steps", 2) or 2) refine_steps = int(latent_upscale_param.get("steps", 2) or 2)
refine_sampler = latent_upscale_param.get("sampler_name", sn) refine_sampler = latent_upscale_param.get("sampler_name", sn)
refine_scheduler = latent_upscale_param.get("scheduler", sch) refine_scheduler = latent_upscale_param.get("scheduler", sch)
@@ -6771,6 +6857,11 @@ class H3LongVideos:
rows, cols, trows, tcols, row_ovl, col_ovl = compute_spatial_grid( rows, cols, trows, tcols, row_ovl, col_ovl = compute_spatial_grid(
int(up_h), int(up_w), tile_th, tile_tw, ol_th, ol_tw, min_tile_tw, min_tile_tw int(up_h), int(up_w), tile_th, tile_tw, ol_th, ol_tw, min_tile_tw, min_tile_tw
) )
logging.info(
"H3 latent refine: %s spatial sampler tiles, target=%sx%s tile_mode=%s tile=%sx%s overlap=%sx%s",
len(rows) * len(cols), target_w, target_h, tile_size_mode, tile_w_px, tile_h_px,
spatial_w_overlap_px, spatial_h_overlap_px,
)
if len(rows) == 1 and len(cols) == 1: if len(rows) == 1 and len(cols) == 1:
(refined_out,) = nodes.common_ksampler( (refined_out,) = nodes.common_ksampler(
model, seed, refine_steps, cfg, refine_sampler, refine_scheduler, upscale_cond, negative, upscale_latent, model, seed, refine_steps, cfg, refine_sampler, refine_scheduler, upscale_cond, negative, upscale_latent,
@@ -6783,12 +6874,10 @@ class H3LongVideos:
for col_index, c0 in enumerate(cols): for col_index, c0 in enumerate(cols):
tc = tcols[col_index] tc = tcols[col_index]
ovw = col_ovl[col_index] ovw = col_ovl[col_index]
tile_target_w = int(tc) * 16 tile_cond = _crop_conditioning_to_tile(
tile_target_h = int(tr) * 16 upscale_cond, int(up_h), int(up_w), r0, c0, tr, tc
tile_cond, tile_latent = _build_shot_conditioning( )
clip, vae, prompt, tile_target_w, tile_target_h, ln, fps, handoff, tile_latent = dict(upscale_latent) if isinstance(upscale_latent, dict) else {}
ref_images=refs, ref_image_size=ref_image_size,
ref_noise_aug=ref_noise_aug, audio_vae=audio_vae, silent=silent)
tile_video = upscaled_video[:, :, :, r0:r0 + tr, c0:c0 + tc].contiguous() tile_video = upscaled_video[:, :, :, r0:r0 + tr, c0:c0 + tc].contiguous()
tr_s = tr + (tr % 2) tr_s = tr + (tr % 2)
tc_s = tc + (tc % 2) tc_s = tc + (tc % 2)
+1001 -6
View File
File diff suppressed because it is too large Load Diff
+51 -7
View File
@@ -3,7 +3,7 @@ import { app } from "/scripts/app.js";
const NODE_NAME = "DumasAnchorStyle"; const NODE_NAME = "DumasAnchorStyle";
const STYLE_INPUT = "anchor_style"; const STYLE_INPUT = "anchor_style";
const DESCRIPTION_INPUT = "style_description"; const DESCRIPTION_INPUT = "style_description";
const H3_NOTE = " Keep this anchor focused on persistent camera language, lighting, texture, environment treatment, and tone; do not name characters or describe one-off actions."; const H3_NOTE = "";
const PRESETS = { const PRESETS = {
"cinematic action movie": "Big-screen action cinema with assertive visual storytelling: dynamic camera placement, strong forward momentum, crisp geography, muscular lighting contrast, practical atmosphere, and a sense of physical consequence. Favor heroic framing, controlled handheld energy or motivated tracking moves, dramatic silhouettes, tasteful lens flares, impact-driven pacing, and polished studio spectacle without drifting into comic-book unreality unless the shot explicitly asks for it." + H3_NOTE, "cinematic action movie": "Big-screen action cinema with assertive visual storytelling: dynamic camera placement, strong forward momentum, crisp geography, muscular lighting contrast, practical atmosphere, and a sense of physical consequence. Favor heroic framing, controlled handheld energy or motivated tracking moves, dramatic silhouettes, tasteful lens flares, impact-driven pacing, and polished studio spectacle without drifting into comic-book unreality unless the shot explicitly asks for it." + H3_NOTE,
"comedy": "Play the scene for comedic readability and timing: clear staging, expressive performances, slightly heightened reactions, clean eyelines, and visual beats that leave room for the joke to land. Use bright approachable lighting, grounded but playful production design, readable framing, and a tone that feels observant, awkward, or absurd without becoming broad parody unless the action supports it." + H3_NOTE, "comedy": "Play the scene for comedic readability and timing: clear staging, expressive performances, slightly heightened reactions, clean eyelines, and visual beats that leave room for the joke to land. Use bright approachable lighting, grounded but playful production design, readable framing, and a tone that feels observant, awkward, or absurd without becoming broad parody unless the action supports it." + H3_NOTE,
@@ -51,28 +51,72 @@ const PRESETS = {
"fantasy adventure": "Rousing fantasy-adventure language: scenic scale, adventurous clarity, tactile costume-and-prop detail, and camera movement that feels exploratory rather than oppressive. Favor storybook geography, weathered materials, golden or stormy atmosphere, and a tone of peril, wonder, and forward motion." + H3_NOTE, "fantasy adventure": "Rousing fantasy-adventure language: scenic scale, adventurous clarity, tactile costume-and-prop detail, and camera movement that feels exploratory rather than oppressive. Favor storybook geography, weathered materials, golden or stormy atmosphere, and a tone of peril, wonder, and forward motion." + H3_NOTE,
}; };
const SOUNDSCAPE_PRESETS = {
"quiet interior": "quiet indoor room tone, faint ventilation and distant household ambience",
"rainy street": "steady rain, wet pavement, distant traffic hum",
"cafe": "low room tone, faint glassware, cutlery, and muted conversation",
"city night": "distant traffic hum, occasional horn, night air",
"forest": "wind in leaves, distant birds, soft natural ambience",
"industrial": "large interior reverb, distant metal ticks, low machine hum",
"silent": "no dialogue, no vocals, only the natural ambient bed of the scene",
"custom": "",
};
const BGM_PRESETS = {
"none": "",
"subtle tension": "low, restrained tension bed with sparse pulses and no vocals",
"cinematic suspense": "cinematic suspense score with muted strings, low drones, and controlled rising pressure",
"emotional piano": "soft emotional piano underscoring with gentle space and no vocals",
"dark ambient": "dark ambient music bed with deep drones, distant texture, and slow unease",
"hopeful orchestral": "hopeful orchestral underscore with warm strings, gentle brass, and restrained lift",
"retro synth": "retro synth score with analog pulses, warm pads, and steady momentum",
"action pulse": "driving action pulse with percussion, rhythmic bass, and urgent forward motion",
"lo-fi": "soft lo-fi instrumental bed with mellow rhythm and warm tape texture",
"no vocals": "instrumental background music only, no singing, no lyrics, no vocal hooks",
"custom": "",
};
const NODE_CONFIGS = {
[NODE_NAME]: {
presetInput: STYLE_INPUT,
descriptionInput: DESCRIPTION_INPUT,
presets: PRESETS,
},
DumasSoundscapeHelper: {
presetInput: "soundscape",
descriptionInput: "soundscape_description",
presets: SOUNDSCAPE_PRESETS,
},
DumasBackgroundMusicHelper: {
presetInput: "bgm",
descriptionInput: "bgm_description",
presets: BGM_PRESETS,
},
};
function findWidget(node, name) { function findWidget(node, name) {
return (node.widgets || []).find((widget) => widget?.name === name) || null; return (node.widgets || []).find((widget) => widget?.name === name) || null;
} }
app.registerExtension({ app.registerExtension({
name: "Dumas.AnchorStyle", name: "Dumas.PresetTextHelpers",
async beforeRegisterNodeDef(nodeType, nodeData) { async beforeRegisterNodeDef(nodeType, nodeData) {
if (nodeData?.name !== NODE_NAME) return; const config = NODE_CONFIGS[nodeData?.name];
if (!config) return;
const originalOnNodeCreated = nodeType.prototype.onNodeCreated; const originalOnNodeCreated = nodeType.prototype.onNodeCreated;
nodeType.prototype.onNodeCreated = function onNodeCreated() { nodeType.prototype.onNodeCreated = function onNodeCreated() {
const created = originalOnNodeCreated?.apply(this, arguments); const created = originalOnNodeCreated?.apply(this, arguments);
const styleWidget = findWidget(this, STYLE_INPUT); const styleWidget = findWidget(this, config.presetInput);
const descriptionWidget = findWidget(this, DESCRIPTION_INPUT); const descriptionWidget = findWidget(this, config.descriptionInput);
if (!styleWidget || !descriptionWidget) return created; if (!styleWidget || !descriptionWidget) return created;
const originalCallback = styleWidget.callback; const originalCallback = styleWidget.callback;
styleWidget.callback = (...args) => { styleWidget.callback = (...args) => {
const selected = String(styleWidget.value || ""); const selected = String(styleWidget.value || "");
if (Object.hasOwn(PRESETS, selected)) { if (Object.hasOwn(config.presets, selected)) {
descriptionWidget.value = PRESETS[selected]; descriptionWidget.value = config.presets[selected];
descriptionWidget.inputEl?.dispatchEvent(new Event("input", { bubbles: true })); descriptionWidget.inputEl?.dispatchEvent(new Event("input", { bubbles: true }));
} }
this.setDirtyCanvas?.(true, true); this.setDirtyCanvas?.(true, true);
+243 -10
View File
@@ -362,6 +362,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
return self._parts return self._parts
order = [] order = []
build_calls = []
first_out = {"samples": FakeNestedTensor((FakeTensor("v1"), FakeTensor("a1")))} first_out = {"samples": FakeNestedTensor((FakeTensor("v1"), FakeTensor("a1")))}
second_out = {"samples": FakeNestedTensor((FakeTensor("v2"), FakeTensor("a2")))} second_out = {"samples": FakeNestedTensor((FakeTensor("v2"), FakeTensor("a2")))}
@@ -408,11 +409,15 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
order.append("upscale") order.append("upscale")
return FakeTensor("upv"), 8, 16 return FakeTensor("upv"), 8, 16
def build_conditioning(*_args, **_kwargs):
build_calls.append(True)
return (
[["cond", {}]],
{"samples": FakeNestedTensor((FakeTensor("basev"), FakeTensor("basea")))},
)
self.module.nodes.common_ksampler = common_ksampler self.module.nodes.common_ksampler = common_ksampler
self.module._build_shot_conditioning = lambda *_args, **_kwargs: ( self.module._build_shot_conditioning = build_conditioning
"cond",
{"samples": FakeNestedTensor((FakeTensor("basev"), FakeTensor("basea")))},
)
self.module._evict_all_but = lambda *_args, **_kwargs: None self.module._evict_all_but = lambda *_args, **_kwargs: None
self.module.mm.unload_model_and_clones = unload_model_and_clones self.module.mm.unload_model_and_clones = unload_model_and_clones
self.module.mm.unload_all_models = unload_all_models self.module.mm.unload_all_models = unload_all_models
@@ -457,6 +462,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.assertLess(order.index("upscale"), order.index("latent_upscale_sample")) self.assertLess(order.index("upscale"), order.index("latent_upscale_sample"))
self.assertLess(order.index("audio"), order.index("video")) self.assertLess(order.index("audio"), order.index("video"))
self.assertEqual(order[-1], "cleanup") self.assertEqual(order[-1], "cleanup")
self.assertEqual(len(build_calls), 1)
finally: finally:
self.module.nodes.common_ksampler = original_common_ksampler self.module.nodes.common_ksampler = original_common_ksampler
self.module._build_shot_conditioning = original_build self.module._build_shot_conditioning = original_build
@@ -476,6 +482,12 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
else: else:
self.module.comfy.nested_tensor.NestedTensor = original_nested self.module.comfy.nested_tensor.NestedTensor = original_nested
def test_latent_refine_tiles_do_not_rebuild_conditioning(self):
source = inspect.getsource(self.module.H3LongVideos._render)
tile_branch = source[source.index("for col_index, c0 in enumerate(cols):"):]
self.assertIn("_crop_conditioning_to_tile", tile_branch)
self.assertNotIn("_build_shot_conditioning(", tile_branch)
def test_latent_upscale_off_skips_second_pass(self): def test_latent_upscale_off_skips_second_pass(self):
calls = [] calls = []
original_common_ksampler = self.module.nodes.common_ksampler original_common_ksampler = self.module.nodes.common_ksampler
@@ -1165,8 +1177,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.assertFalse(required["brightness_match"][1]["default"]) self.assertFalse(required["brightness_match"][1]["default"])
self.assertEqual(required["dynamic_fade"][1]["default"], "off") self.assertEqual(required["dynamic_fade"][1]["default"], "off")
self.assertEqual(required["dynamic_fade_min"][1]["default"], 32) self.assertEqual(required["dynamic_fade_min"][1]["default"], 32)
self.assertEqual(required["chunk_length"][1]["default"], 17) self.assertEqual(required["chunk_length"][1]["default"], 85)
self.assertEqual(required["temporal_overlap"][1]["default"], 0) self.assertEqual(required["temporal_overlap"][1]["default"], 17)
self.assertFalse(required["resize_conditioning"][1]["default"]) self.assertFalse(required["resize_conditioning"][1]["default"])
self.assertEqual(required["anchor_strength"][1]["default"], 0.999) self.assertEqual(required["anchor_strength"][1]["default"], 0.999)
@@ -1256,16 +1268,61 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.assertEqual(smaller["chunk_length"], 17) self.assertEqual(smaller["chunk_length"], 17)
self.assertEqual(smaller["temporal_overlap"], 0) self.assertEqual(smaller["temporal_overlap"], 0)
def test_cuda_model_temporal_params_cap_saved_workflows(self): def test_cuda_model_temporal_params_keep_splitting_saved_workflows(self):
latent = importlib.import_module("dumas_h3_latent_upscale") latent = importlib.import_module("dumas_h3_latent_upscale")
chunk_length, temporal_overlap = latent._effective_temporal_params({ chunk_length, temporal_overlap = latent._effective_temporal_params({
"mode": "model", "mode": "model",
"device": "cuda", "device": "cuda",
"chunk_length": 85, "chunk_length": 85,
"temporal_overlap": 17, "temporal_overlap": 17,
}) }, frame_count=124)
self.assertEqual(chunk_length, 17) self.assertEqual(chunk_length, 85)
self.assertEqual(temporal_overlap, 0) self.assertEqual(temporal_overlap, 17)
def test_cuda_model_temporal_params_keep_short_saved_workflows_until_oom(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
chunk_length, temporal_overlap = latent._effective_temporal_params({
"mode": "model",
"device": "cuda",
"chunk_length": 85,
"temporal_overlap": 17,
}, frame_count=85)
self.assertEqual(chunk_length, 85)
self.assertEqual(temporal_overlap, 17)
def test_cuda_model_oom_retries_temporal_before_spatial_fallback(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
calls = []
original_tiled = latent._upscale_video_model_tiled
original_shrink_model = latent._shrink_model_tile_param
try:
def tiled(_video, param):
calls.append(("tiled", param.get("chunk_length"), param.get("tile_size_mode")))
raise RuntimeError("out of memory")
latent._upscale_video_model_tiled = tiled
latent._shrink_model_tile_param = (
lambda param: calls.append(("shrink_spatial", param.get("tile_size_mode"))) or None
)
with self.assertRaisesRegex(RuntimeError, "smaller temporal chunk"):
latent.upscale_video_model(
"video",
{
"mode": "model",
"device": "cuda",
"model_name": "upscale.safetensors",
"chunk_length": 85,
"temporal_overlap": 17,
},
)
self.assertEqual(calls[0], ("tiled", 85, None))
self.assertNotIn(("shrink_spatial", None), calls)
finally:
latent._upscale_video_model_tiled = original_tiled
latent._shrink_model_tile_param = original_shrink_model
def test_interp_temporal_params_preserve_upstream_defaults(self): def test_interp_temporal_params_preserve_upstream_defaults(self):
latent = importlib.import_module("dumas_h3_latent_upscale") latent = importlib.import_module("dumas_h3_latent_upscale")
@@ -1278,6 +1335,182 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.assertEqual(chunk_length, 85) self.assertEqual(chunk_length, 85)
self.assertEqual(temporal_overlap, 17) self.assertEqual(temporal_overlap, 17)
def test_unload_upscale_model_defers_while_held(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
class FakeParam:
device = "cuda"
class FakeModel:
def __init__(self):
self.moves = []
def parameters(self):
return iter((FakeParam(),))
def to(self, device):
self.moves.append(device)
return self
cache_key = "upscale.safetensors::cuda::fp16"
original_cache_value = latent._MODEL_CACHE.get(cache_key)
original_hold_depth = latent._MODEL_HOLD_DEPTH
fake_model = FakeModel()
try:
latent._MODEL_CACHE[cache_key] = fake_model
latent._MODEL_HOLD_DEPTH = 0
with latent._hold_upscale_model_loaded():
latent.unload_upscale_model("upscale.safetensors", "cuda", "fp16")
self.assertEqual(fake_model.moves, [])
latent.unload_upscale_model("upscale.safetensors", "cuda", "fp16")
self.assertEqual(fake_model.moves, ["cpu"])
finally:
latent._MODEL_HOLD_DEPTH = original_hold_depth
if original_cache_value is None:
latent._MODEL_CACHE.pop(cache_key, None)
else:
latent._MODEL_CACHE[cache_key] = original_cache_value
def test_model_upscale_releases_cached_model_after_pass(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
calls = []
original_temporal = latent._upscale_video_temporal_chunks
original_unload_now = latent._unload_upscale_model_now
original_cuda = latent.torch.cuda
original_device = getattr(latent.torch, "device", None)
try:
latent.torch.cuda = types.SimpleNamespace(is_available=lambda: True)
latent.torch.device = lambda value: value
def temporal(video, param, upscaler):
calls.append(("temporal", latent._MODEL_HOLD_DEPTH))
return "video", 8, 16
def unload_now(name, device, precision):
calls.append(("unload", name, device, precision, latent._MODEL_HOLD_DEPTH))
latent._upscale_video_temporal_chunks = temporal
latent._unload_upscale_model_now = unload_now
result = latent.upscale_latent_video("source", {
"mode": "model",
"model_name": "upscale.safetensors",
"device": "cuda",
"precision": "fp16",
})
self.assertEqual(result, ("video", 8, 16))
self.assertEqual(calls[0], ("temporal", 1))
self.assertEqual(calls[1], ("unload", "upscale.safetensors", "cuda", "fp16", 1))
self.assertEqual(latent._MODEL_HOLD_DEPTH, 0)
finally:
latent._upscale_video_temporal_chunks = original_temporal
latent._unload_upscale_model_now = original_unload_now
latent.torch.cuda = original_cuda
if original_device is None:
delattr(latent.torch, "device")
else:
latent.torch.device = original_device
def test_model_upscale_oom_falls_back_to_interp(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
calls = []
original_temporal = latent._upscale_video_temporal_chunks
original_interp = latent.upscale_video_interp
original_unload_now = latent._unload_upscale_model_now
original_cuda = latent.torch.cuda
original_device = getattr(latent.torch, "device", None)
try:
latent.torch.cuda = types.SimpleNamespace(
is_available=lambda: True,
empty_cache=lambda: calls.append(("empty_cache",)),
)
latent.torch.device = lambda value: value
def temporal(_video, _param, _upscaler):
calls.append(("temporal", latent._MODEL_HOLD_DEPTH))
raise RuntimeError("H3 latent upscale exhausted its GPU spatial fallbacks")
def interp(video, param):
calls.append(("interp", video, param.get("mode"), param.get("method")))
return "interp_video", 8, 16
def unload_now(name, device, precision):
calls.append(("unload", name, device, precision, latent._MODEL_HOLD_DEPTH))
latent._upscale_video_temporal_chunks = temporal
latent.upscale_video_interp = interp
latent._unload_upscale_model_now = unload_now
result = latent.upscale_latent_video("source", {
"mode": "model",
"model_name": "upscale.safetensors",
"method": "bilinear",
"device": "cuda",
"precision": "fp16",
})
self.assertEqual(result, ("interp_video", 8, 16))
self.assertEqual(calls[0], ("temporal", 1))
self.assertEqual(calls[1], ("unload", "upscale.safetensors", "cuda", "fp16", 1))
self.assertIn(("empty_cache",), calls)
self.assertEqual(calls[-1], ("interp", "source", "interp", "bilinear"))
self.assertEqual(latent._MODEL_HOLD_DEPTH, 0)
finally:
latent._upscale_video_temporal_chunks = original_temporal
latent.upscale_video_interp = original_interp
latent._unload_upscale_model_now = original_unload_now
latent.torch.cuda = original_cuda
if original_device is None:
delattr(latent.torch, "device")
else:
latent.torch.device = original_device
def test_model_upscale_skips_learned_model_on_8gb_cuda(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
calls = []
original_temporal = latent._upscale_video_temporal_chunks
original_interp = latent.upscale_video_interp
original_cuda = latent.torch.cuda
try:
latent.torch.cuda = types.SimpleNamespace(
is_available=lambda: True,
mem_get_info=lambda: (1 * 1024 * 1024 * 1024, 8 * 1024 * 1024 * 1024),
empty_cache=lambda: calls.append(("empty_cache",)),
)
def temporal(_video, _param, _upscaler):
calls.append(("temporal",))
raise AssertionError("learned model path should be skipped on 8GB CUDA")
def interp(video, param):
calls.append(("interp", video, param.get("mode"), param.get("method")))
return "interp_video", 8, 16
latent._upscale_video_temporal_chunks = temporal
latent.upscale_video_interp = interp
result = latent.upscale_latent_video("source", {
"mode": "model",
"model_name": "upscale.safetensors",
"method": "bilinear",
"device": "cuda",
"precision": "fp16",
})
self.assertEqual(result, ("interp_video", 8, 16))
self.assertNotIn(("temporal",), calls)
self.assertIn(("empty_cache",), calls)
self.assertEqual(calls[-1], ("interp", "source", "interp", "bilinear"))
finally:
latent._upscale_video_temporal_chunks = original_temporal
latent.upscale_video_interp = original_interp
latent.torch.cuda = original_cuda
def test_upscale_video_model_raises_when_gpu_cannot_shrink(self): def test_upscale_video_model_raises_when_gpu_cannot_shrink(self):
latent = importlib.import_module("dumas_h3_latent_upscale") latent = importlib.import_module("dumas_h3_latent_upscale")
+310
View File
@@ -356,6 +356,303 @@ class DumasImageNodeTests(unittest.TestCase):
required = self.image_nodes.DumasLocationReferenceNode.INPUT_TYPES()["required"] required = self.image_nodes.DumasLocationReferenceNode.INPUT_TYPES()["required"]
self.assertNotIn("picture_id", required) self.assertNotIn("picture_id", required)
def test_character_helper_restores_image_and_text_outputs(self):
node = self.image_nodes.DumasCharacterHelperNode()
image1 = FakeTensorBatch()
image2 = FakeTensorBatch()
result = node.build_character_text(
image1=image1,
image2=image2,
image1_picture_id="1",
image2_picture_id="2",
character_id="char_dave",
name="Dave",
alias="The Locksmith",
gender="male",
age="41",
nationality="English",
occupation="a detective",
height_feet="6",
height_inches="2",
accent="English",
general="Moves carefully and notices every exit",
wardrobe="weathered red flight jacket, grey cargo shorts, black boots",
)
self.assertIs(result[0], image1)
self.assertIs(result[1], image2)
self.assertIn("<Picture 1> and <Picture 2> reference the same character", result[2])
self.assertIn("Dave is also known as The Locksmith", result[2])
self.assertIn("is 41 years old", result[2])
self.assertEqual(result[3], "Dave = weathered red flight jacket, grey cargo shorts, black boots")
self.assertIs(result[4]["image"], image1)
self.assertIs(result[5]["image"], image2)
self.assertEqual(result[4]["id"], "char-dave")
self.assertEqual(result[5]["id"], "char-dave")
self.assertEqual(result[4]["name"], "Dave")
self.assertEqual(result[4]["aliases"], ["The Locksmith"])
self.assertEqual(result[4]["facts"]["age"], "41")
self.assertEqual(result[4]["facts"]["height_feet"], "6")
self.assertEqual(result[4]["facts"]["height_inches"], "2")
self.assertEqual(result[4]["wardrobe"], "weathered red flight jacket, grey cargo shorts, black boots")
self.assertEqual(len(result), 6)
def test_location_helper_matches_character_helper_shape_without_wardrobe(self):
node = self.image_nodes.DumasLocationHelperNode()
image1 = FakeTensorBatch()
image2 = FakeTensorBatch()
result = node.build_location_text(
image1=image1,
image2=image2,
image1_picture_id="3",
image2_picture_id="4",
location_id="coffee-shop-01",
name="Coffee Shop",
alias="Cafe Interior",
description="Warm tungsten lighting, narrow counter, rainy front window",
general="Evening ambience, cramped but cozy",
)
self.assertIs(result[0], image1)
self.assertIs(result[1], image2)
self.assertIn("<Picture 3> and <Picture 4> reference the same location", result[2])
self.assertIn("Coffee Shop is also known as Cafe Interior", result[2])
self.assertIn("Warm tungsten lighting, narrow counter, rainy front window.", result[2])
self.assertIn("Evening ambience, cramped but cozy.", result[2])
self.assertIs(result[3]["image"], image1)
self.assertIs(result[4]["image"], image2)
self.assertEqual(result[3]["kind"], "location")
self.assertEqual(result[4]["kind"], "location")
self.assertEqual(result[3]["id"], "coffee-shop-01")
self.assertEqual(result[4]["id"], "coffee-shop-01")
self.assertEqual(result[3]["name"], "Coffee Shop")
self.assertEqual(result[3]["aliases"], ["Cafe Interior"])
self.assertEqual(result[3]["description"], "Warm tungsten lighting, narrow counter, rainy front window")
self.assertEqual(result[3]["general"], "Evening ambience, cramped but cozy")
self.assertEqual(len(result), 5)
def test_soundscape_helper_defaults_to_selected_preset_description(self):
node = self.image_nodes.DumasSoundscapeHelperNode()
result = node.build_soundscape("rainy street", "")
self.assertEqual(result[0], "steady rain, wet pavement, distant traffic hum")
def test_background_music_helper_defaults_to_selected_preset_description(self):
node = self.image_nodes.DumasBackgroundMusicHelperNode()
result = node.build_bgm("subtle tension", "")
self.assertEqual(result[0], "low, restrained tension bed with sparse pulses and no vocals")
def test_h3_prompt_curator_compacts_named_references(self):
node = self.image_nodes.DumasH3PromptCuratorNode()
dave_image = FakeTensorBatch()
cafe_image = FakeTensorBatch()
van_image = FakeTensorBatch()
dave = self.image_nodes.make_reference(
kind="character",
image=dave_image,
name="Dave",
aliases="The Locksmith",
description="tired eyes, cropped brown hair",
wardrobe="red flight jacket",
)
cafe = self.image_nodes.make_reference(
kind="location",
image=cafe_image,
name="Coffee Shop",
description="warm tungsten lighting and rainy windows",
)
van = self.image_nodes.make_reference(
kind="location",
image=van_image,
name="Blue Van",
description="scuffed blue delivery van",
)
result = node.curate_prompt(
action_prompt="Dave runs from the Coffee Shop into the rain.",
anatomy_guard="auto",
subject_count_guard="auto",
anchor="grounded handheld thriller",
soundscape="steady rain",
bgm="low suspense music",
ref_1=dave,
ref_2=van,
ref_3=cafe,
)
prompt = result[0]
self.assertIn("<Picture 1> Dave", prompt)
self.assertIn("<Picture 2> Coffee Shop", prompt)
self.assertIn("Action: Dave runs from the Coffee Shop into the rain.", prompt)
self.assertIn("Anatomy guard:", prompt)
self.assertIn("Subject count guard:", prompt)
self.assertIn("overall_soundscape: steady rain", prompt)
self.assertIn("background_music: low suspense music", prompt)
self.assertIn("exactly one named character: <Picture 1> Dave", prompt)
self.assertIs(result[1], dave_image)
self.assertIs(result[2], cafe_image)
self.assertIsNone(result[3])
self.assertEqual(result[10], 2)
self.assertIn("input 3-><Picture 2> Coffee Shop", result[11])
self.assertEqual(result[12], "grounded handheld thriller")
self.assertEqual(result[13], "steady rain")
self.assertEqual(result[14], "low suspense music")
self.assertEqual(result[15]["name"], "Dave")
self.assertIs(result[15]["image"], dave_image)
self.assertEqual(result[16]["name"], "Coffee Shop")
self.assertIs(result[16]["image"], cafe_image)
self.assertIsNone(result[17])
self.assertIn("<Picture 1> Dave", result[24])
self.assertIn("<Picture 2> Coffee Shop", result[24])
self.assertIn("Location context for <Picture 2> Coffee Shop", result[24])
def test_h3_prompt_curator_renumbers_explicit_reference_tags(self):
node = self.image_nodes.DumasH3PromptCuratorNode()
image1 = FakeTensorBatch()
image3 = FakeTensorBatch()
unused = FakeTensorBatch()
first = self.image_nodes.make_reference(kind="character", image=image1, name="Maya")
second = self.image_nodes.make_reference(kind="location", image=unused, name="Lobby")
third = self.image_nodes.make_reference(kind="location", image=image3, name="Rooftop")
result = node.curate_prompt(
action_prompt="<Picture 1> Maya crosses to <ref3> as the wind rises.",
anatomy_guard="off",
subject_count_guard="off",
ref_1=first,
ref_2=second,
ref_3=third,
)
prompt = result[0]
self.assertIn("<Picture 1> Maya crosses to <Picture 2>", prompt)
self.assertNotIn("<Picture 3>", prompt)
self.assertIs(result[1], image1)
self.assertIs(result[2], image3)
self.assertIsNone(result[3])
self.assertEqual(result[10], 2)
def test_h3_prompt_curator_can_force_subject_count_without_character_refs(self):
node = self.image_nodes.DumasH3PromptCuratorNode()
result = node.curate_prompt(
action_prompt="A locked-off shot of the empty corridor.",
anatomy_guard="off",
subject_count_guard="on",
)
self.assertIn("Subject count guard:", result[0])
self.assertIn("Only include the people explicitly described", result[0])
self.assertEqual(result[10], 0)
def test_h3_prompt_curator_treats_helper_image_pair_as_one_character(self):
helper = self.image_nodes.DumasCharacterHelperNode()
curator = self.image_nodes.DumasH3PromptCuratorNode()
image1 = FakeTensorBatch()
image2 = FakeTensorBatch()
helper_result = helper.build_character_text(
image1=image1,
image2=image2,
image1_picture_id="1",
image2_picture_id="2",
character_id="char_dave",
name="Dave",
alias="The Locksmith",
gender="male",
age="41",
nationality="English",
occupation="detective",
height_feet="6",
height_inches="2",
accent="English",
general="Tired eyes, cropped brown hair",
wardrobe="weathered red flight jacket",
)
result = curator.curate_prompt(
action_prompt="Dave checks the locked door.",
anatomy_guard="on",
subject_count_guard="auto",
ref_1=helper_result[4],
ref_2=helper_result[5],
)
self.assertIs(result[1], image1)
self.assertIs(result[2], image2)
self.assertEqual(result[10], 2)
self.assertIn("Character facts for <Picture 1> Dave", result[0])
self.assertIn("41 years old", result[0])
self.assertIn("6 foot 2 tall", result[0])
self.assertIn("exactly one named character: <Picture 1> Dave", result[0])
self.assertNotIn("exactly 2 named characters", result[0])
def test_h3_prompt_curator_uses_location_helper_references_by_name(self):
helper = self.image_nodes.DumasLocationHelperNode()
curator = self.image_nodes.DumasH3PromptCuratorNode()
image1 = FakeTensorBatch()
image2 = FakeTensorBatch()
helper_result = helper.build_location_text(
image1=image1,
image2=image2,
image1_picture_id="1",
image2_picture_id="2",
location_id="coffee_shop",
name="Coffee Shop",
alias="Cafe Interior",
description="Warm tungsten lighting, narrow counter, rainy front window",
general="Evening ambience, cramped but cozy",
)
result = curator.curate_prompt(
action_prompt="A slow push through the Coffee Shop as rain streaks the windows.",
anatomy_guard="on",
subject_count_guard="auto",
ref_1=helper_result[3],
ref_2=helper_result[4],
)
self.assertIs(result[1], image1)
self.assertIs(result[2], image2)
self.assertEqual(result[10], 2)
self.assertIn("<Picture 1> Coffee Shop", result[0])
self.assertIn("<Picture 2> Coffee Shop", result[0])
self.assertIn("Location context for <Picture 1> Coffee Shop", result[0])
self.assertIn("Warm tungsten lighting", result[0])
self.assertNotIn("Subject count guard:", result[0])
def test_h3_prompt_curator_defaults_anatomy_guard_to_on(self):
required = self.image_nodes.DumasH3PromptCuratorNode.INPUT_TYPES()["required"]
self.assertEqual(required["anatomy_guard"][1]["default"], "on")
def test_helper_node_mappings_use_general_purpose_helpers(self):
mappings = self.image_nodes.NODE_CLASS_MAPPINGS
display = self.image_nodes.NODE_DISPLAY_NAME_MAPPINGS
self.assertIs(mappings["DumasCharacterHelper"], self.image_nodes.DumasCharacterHelperNode)
self.assertIs(mappings["DumasLocationHelper"], self.image_nodes.DumasLocationHelperNode)
self.assertIs(mappings["DumasSoundscapeHelper"], self.image_nodes.DumasSoundscapeHelperNode)
self.assertIs(mappings["DumasBackgroundMusicHelper"], self.image_nodes.DumasBackgroundMusicHelperNode)
self.assertIs(mappings["DumasH3PromptCurator"], self.image_nodes.DumasH3PromptCuratorNode)
self.assertEqual(display["DumasCharacterHelper"], "Dumas Character Helper")
self.assertEqual(display["DumasLocationHelper"], "Dumas Location Helper")
self.assertEqual(display["DumasSoundscapeHelper"], "Dumas Soundscape Helper")
self.assertEqual(display["DumasBackgroundMusicHelper"], "Dumas Background Music Helper")
self.assertEqual(display["DumasH3PromptCurator"], "Dumas H3 Prompt Curator")
def test_h3_prompt_curator_uses_documented_reference_limits(self):
node = self.image_nodes.DumasH3PromptCuratorNode()
self.assertEqual(len(node.RETURN_TYPES), 25)
self.assertEqual(node.RETURN_NAMES[1:10], tuple(f"ref_image_{i}" for i in range(1, 10)))
self.assertEqual(node.RETURN_NAMES[12:15], ("anchor", "sounds", "bgm"))
self.assertEqual(node.RETURN_NAMES[15:24], tuple(f"original_ref_{i}" for i in range(1, 10)))
self.assertEqual(node.RETURN_NAMES[24], "reference_description")
def test_normalize_reference_upgrades_generic_summary_with_socket_picture_id(self): def test_normalize_reference_upgrades_generic_summary_with_socket_picture_id(self):
image = FakeTensorBatch() image = FakeTensorBatch()
@@ -417,6 +714,19 @@ class DumasImageNodeTests(unittest.TestCase):
self.assertIn("real time", result[0]) self.assertIn("real time", result[0])
self.assertNotIn("persistent camera language", result[0]) self.assertNotIn("persistent camera language", result[0])
def test_anchor_style_node_strips_legacy_persistent_anchor_note(self):
node = self.image_nodes.DumasAnchorStyleNode()
legacy = (
"Gritty handheld realism. Keep this anchor focused on persistent camera "
"language, lighting, texture, environment treatment, and tone; do not "
"name characters or describe one-off actions."
)
result = node.build_anchor("cinematic action movie", legacy)
self.assertEqual(result[0], "Gritty handheld realism.")
self.assertNotIn("persistent camera language", result[0])
def test_anchor_style_node_prefers_manual_description_edits(self): def test_anchor_style_node_prefers_manual_description_edits(self):
node = self.image_nodes.DumasAnchorStyleNode() node = self.image_nodes.DumasAnchorStyleNode()
custom = "Lo-fi pirate broadcast with smeared highlights and anxious zoom corrections." custom = "Lo-fi pirate broadcast with smeared highlights and anxious zoom corrections."