Compare commits

...
52 Commits
Author SHA1 Message Date
chris.dumas 8d4232a142 Restore compiled prompt reference descriptions 2026-09-10 08:34:32 +00:00
chris.dumas d68d04254f Prune stale prompt curator outputs 2026-09-10 08:30:42 +00:00
chris.dumas 22398c5213 Simplify H3 prompt curator component outputs 2026-09-10 08:26:39 +00:00
chris.dumas 1a4cfbfd7e Add compiled prompt reference output pairs 2026-09-10 08:15:33 +00:00
chris.dumas eac4e73b47 Expose individual prompt reference descriptions 2026-09-10 07:59:11 +00:00
chris.dumas e973a0d785 Expose H3 prompt curator components 2026-09-09 15:34:06 +00:00
chris.dumas 388de1d837 Sync helper preset text boxes 2026-09-07 13:26:29 +00:00
chris.dumas 338150648b Add BGM helper for H3 prompt curator 2026-09-07 08:13:45 +00:00
chris.dumas 7de62226f2 Strip legacy anchor style note 2026-09-07 08:01:22 +00:00
chris.dumas a51141cb26 Add location helper reference outputs 2026-09-07 07:46:24 +00:00
chris.dumas dd8ef84379 Add character helper reference outputs 2026-09-05 18:00:17 +00:00
chris.dumas d20b257134 Add H3 prompt subject count guard 2026-09-05 17:54:43 +00:00
chris.dumas 173205ca51 Add H3 prompt curator 2026-09-05 16:43:05 +00:00
chris.dumas 98833ba99d Restore general character and location helpers 2026-09-04 15:52:40 +00:00
chris.dumas a7f7225b3a Bypass learned latent upscaler on low VRAM 2026-09-04 15:24:30 +00:00
chris.dumas 5dc5c2ce8b Prioritize temporal fallback for latent upscale OOM 2026-09-04 15:10:38 +00:00
chris.dumas 83645811f4 Reuse conditioning for refinement tiles 2026-09-04 14:52:43 +00:00
chris.dumas 216bc05762 Retry temporal split after latent upscale OOM 2026-09-04 14:43:25 +00:00
chris.dumas 49e099ee7f Log H3 latent upscale tiling plan 2026-09-04 14:14:24 +00:00
chris.dumas 32a16645b5 Use adaptive CUDA latent upscale chunking 2026-09-04 12:09:59 +00:00
chris.dumas 7bb0b0abea Reuse conditioning for latent upscale refine 2026-09-04 11:49:21 +00:00
chris.dumas 0b189dcf7a Keep latent upscaler loaded during pass 2026-09-04 11:04:26 +00:00
chris.dumas e8599055a1 Force temporal chunks for CUDA latent upscale 2026-09-04 10:22:41 +00:00
chris.dumas 618a48e4d9 Harden latent upscale GPU cleanup 2026-09-04 10:05:24 +00:00
chris.dumas cbabcf8208 Offload H3 before latent upscale 2026-09-04 09:49:34 +00:00
chris.dumas fbfbe4ef5a Extend H3 spatial fallback ladder 2026-09-04 09:22:35 +00:00
chris.dumas 5b52793fad Clamp H3 spatial overlaps to tile size 2026-09-04 09:05:29 +00:00
chris.dumas 7e45c81589 Reset overlap in H3 spatial retries 2026-09-04 08:48:20 +00:00
chris.dumas 027865b9ca Tighten H3 spatial fallback retries 2026-09-04 08:27:54 +00:00
chris.dumas 9f0c546681 Remove latent upscale CPU fallback 2026-09-04 08:13:27 +00:00
chris.dumas 30ae81b86b Jump latent upscale fallback to spatial split 2026-09-04 08:00:06 +00:00
chris.dumas 179fa0778e Add CPU fallback for latent upscale OOM 2026-09-04 07:33:10 +00:00
chris.dumas a34c9eb4ff Reduce latent upscale VRAM pressure 2026-09-04 07:21:38 +00:00
chris.dumas 782a7d658b Fix temporal chunking to use H3 token grid 2026-09-04 07:06:08 +00:00
chris.dumas 04a61af874 Add temporal OOM backoff for latent upscale 2026-09-04 06:54:35 +00:00
chris.dumas a0d80bcc47 Add temporal chunking to latent upscale 2026-09-04 06:41:03 +00:00
chris.dumas fbb5f799cd Unload latent upscale model on OOM 2026-09-03 20:30:58 +00:00
chris.dumas 9dc3c405a6 Clamp latent upscale fallback tile minimum 2026-09-03 20:15:58 +00:00
chris.dumas 21ae4d62e0 Keep latent upscale shrink steps 32-aligned 2026-09-03 19:59:50 +00:00
chris.dumas 836a6eba33 Back off latent upscale tile size on OOM 2026-09-03 19:43:15 +00:00
chris.dumas 3b63d33ec7 Tile latent upscale model inference 2026-09-03 19:01:13 +00:00
chris.dumas c62921c3e8 Stop retrying latent upscale OOMs as sampling 2026-09-03 18:46:56 +00:00
chris.dumas 3ab6342ce5 Handle legacy latent upscale payloads 2026-09-03 18:33:01 +00:00
chris.dumas 51e03a39b7 Reduce latent upscale branch memory 2026-09-03 18:08:44 +00:00
chris.dumas f7b94ccaef Free first-pass latent before refinement 2026-09-03 16:34:12 +00:00
chris.dumas f6120a8500 Adopt full MMH3 spatial split controls 2026-09-03 16:15:43 +00:00
chris.dumas 0e119646ab Add remaining spatial split settings 2026-09-03 15:47:59 +00:00
chris.dumas c89570eae9 Expand latent upscale spatial stitch controls 2026-09-03 15:35:39 +00:00
chris.dumas 36d9f4369c Add spatial batching to latent upscale 2026-09-03 15:12:42 +00:00
chris.dumas 2cb3c694f0 Fix latent upscale summary scope leak 2026-09-03 14:37:24 +00:00
chris.dumas 5c12cd18a3 Expose latent upscale sampler controls 2026-09-03 13:05:32 +00:00
chris.dumas c1d937e0e2 Add H3 latent upscale refinement stage 2026-09-03 12:53:57 +00:00
14 changed files with 3988 additions and 356 deletions
+27 -66
View File
@@ -1149,81 +1149,42 @@ These two belong together.
If you change `shift_video`, you usually need to change `shift_audio` in proportion.
## Group 8: Detail Pass
## Group 8: Latent Upscale
This is the optional second pass.
This is the optional latent refinement stage, used before decode.
### `detail_pass`
Enables the refinement pass.
The long-video node now expects a separate `Dumas H3 Latent Upscale Params` node for this stage.
Wire that node into the `latent_upscale_param` input when you want the shot to be upscaled and lightly
re-sampled before decode.
What this really means:
- the node renders the beat once
- then runs a second sampler pass over that result
- the goal is to polish, not to invent a whole different shot
### `detail_sampler_name`
Sampler for the refinement pass.
### `detail_scheduler`
Scheduler for the refinement pass.
### `detail_steps`
Extra steps for the refinement pass.
What this really means:
- more steps gives the second pass more opportunity to change the image
- that can help detail
- but after a point it stops being "cleanup" and starts becoming "rewrite"
### `detail_denoise`
How strongly the refinement pass is allowed to rewrite the beat.
What this really means:
- low denoise = polish what is already there
- high denoise = let the second pass substantially alter what is already there
### How The Detail-Pass Settings Work Together
The detail pass starts from the first-pass result and tries to polish it.
Gentle settings:
- low to medium `detail_steps`
- low `detail_denoise`
Aggressive settings:
- high `detail_steps`
- high `detail_denoise`
Aggressive settings can improve texture, but they can also:
- change faces
- pull away from references
- break continuity
That is why this group should be read as one combined strength control:
- `detail_pass` decides whether the second pass exists
- `detail_steps` decides how long it keeps working
- `detail_denoise` decides how free it is to change things
- `detail_sampler_name` and `detail_scheduler` shape how that rewrite behaves
- the sampled latent is upscaled in latent space to the target size
- the conditioning is rebuilt at that target size
- the node then runs a short refinement pass over the upscaled latent with the sampler, scheduler, step count, denoise, and megapixel target you picked on the latent-upscale params node
- if the target is larger than the spatial tile size, that refinement pass is processed in spatial batches using the same tile defaults as the upstream latent-split node
- the spatial stitch mode follows the upstream overlap controls, including `linear`, `smoothstep`, `overwrite`, and `midpoint`
- the node also carries the upstream split compatibility knobs (`chunk_length`, `temporal_overlap`, `resize_conditioning`, and `anchor_strength`) so the control surface stays in one place
Good starting point:
- `detail_pass = on`
- `detail_sampler_name = euler`
- `detail_scheduler = beta`
- `detail_steps = 4` to `8`
- `detail_denoise = 0.20` to `0.35`
- use the `model` mode when you want the strongest latent detail recovery
- use the interpolation mode when you want a cheaper resize-only path
- start with `euler_ancestral`, `simple`, `2` steps, and `0.2` denoise
- leave width and height at `0` unless you want an exact override; otherwise `megapixels` drives the target size
- for long shots on smaller cards, start with `chunk_length = 85` and `temporal_overlap = 17` so the latent upscaler works in shorter temporal passes
- leave the spatial tile inputs at their defaults first: `512x512` tiles, `64` overlap, `0` fade width, `earlier` overlap mode
- keep `linear` blend first unless you want to reproduce a specific upstream stitch style
- leave the split compatibility knobs alone unless you specifically need to mirror the upstream node behavior
The important part is that this stage is still a latent pass, not a pixel-space resize:
- it happens before decode
- it can change structure more than a normal image upscale
- it is the place to recover detail without adding another full detail-pass toggle
If you do not wire the helper node, the long-video node skips latent upscale entirely and renders as before.
## Group 9: Performance, Decode, And Upscale
+38 -2
View File
@@ -43,13 +43,27 @@
- Prompt `<Picture N>` tags now map to the actual ref socket numbers you wire, even with gaps such as only `ref_2` and `ref_7` connected.
- Character refs now contribute appearance and wardrobe context from the same structured object, while location refs contribute environment context from theirs.
- The default ref2v bias is now stronger: `ref_mode` defaults to `auto ref2v` so untagged prompts condition every shot instead of only shot 1, and `ref_noise_aug` defaults to `0.95` rather than the upstream-literal `0.999`.
- `Dumas H3 Latent Upscale Params` provides the optional pre-decode latent refinement stage for the long-video node.
- Per-shot directives now support `continuity:`, `ref_mode:`, `ref_noise_aug:`, `anchor_add:`, `soundscape:`, and `music:` in addition to the existing timing and wardrobe directives.
- `Dumas H3 Latent Upscale Params`
- Inputs: `mode`, `model_name`, `method`, `width`, `height`, `device`, `precision`, `sampler_name`, `scheduler`, `steps`, `denoise`, `megapixels`, `tile_width`, `tile_height`, `overlap`, `fade_width`, `fade_height`, `overlap_mode`, `overlap_blend`, `tile_size_mode`, `grid_rows`, `grid_cols`, `spatial_w_overlap`, `spatial_h_overlap`, `min_tile_size`, `masked_area_noise`, `brightness_match`, `dynamic_fade`, `dynamic_fade_min`, `chunk_length`, `temporal_overlap`, `resize_conditioning`, `anchor_strength`
- Output: `latent_upscale_param`
- Bundles the optional latent-space upscaler settings used by `Dumas H3 Long Videos` before decode, so the main node can rebuild conditioning at the target size and run a short refinement pass with your chosen sampler, scheduler, step count, denoise, and the full upstream spatial split controls.
- `Dumas H3 Beat Prompt`
- Inputs: authored through the custom front-end beat editor
- Output: `prompt`
- Builds one H3 prompt block per beat, with quick controls for per-shot timing, continuity, ref behavior, anchor additions, soundscape, and music while staying compatible with direct text editing.
- `Dumas H3 Prompt Curator`
- Inputs: `action_prompt`, `anatomy_guard`, `subject_count_guard`, optional `anchor`, optional `soundscape`, optional `bgm`, optional `ref_1` through `ref_9`
- Outputs: `prompt`, `ref_image_1` through `ref_image_9`, `reference_count`, `debug`, `anchor`, `sounds`, `bgm`, `original_ref_1` through `original_ref_9`, `compiled_ref_description_1` through `compiled_ref_description_9`
- Builds one standalone MiniMax H3 prompt from your final action text plus structured character/location references.
- The action text can mention references by character/location name, alias, `<Picture N>`, or `<refN>`. Only mentioned references are emitted, and the output images are compacted/renumbered so skipped inputs do not leave gaps.
- Extra component outputs expose the cleaned anchor, sounds, BGM, and each selected original reference image plus its compiled reference description in compacted order.
- Adds curated reference context, anatomy guard text, optional subject-count guard text, anchor/style text, `overall_soundscape:` text, and `background_music:` text while respecting MiniMax H3's reference-generation shape: one prompt plus up to nine reference images.
- `Dumas H3 Shot Length`
- Inputs: `shot_seconds`, `fps`, optional `cap_to_h3_max`
- Outputs: `seconds`, `frames`, `info`
@@ -70,12 +84,34 @@
- Output: `reference`
- Builds one structured `REFERENCE` object for a location/environment so H3 can use the same socket type for both character and scenic refs.
- `Dumas Character Helper`
- Inputs: `image1`, `image2`, picture IDs, character identity fields, `general`, `wardrobe`
- Outputs: `image1`, `image2`, `reference_prompt`, `wardrobe`, `reference1`, `reference2`
- Restores the original general-purpose helper shape while also emitting two structured `REFERENCE` objects for the prompt curator.
- The structured references carry the same character name, alias, age, height, gender, nationality, occupation, accent, wardrobe, and notes, so mentioning the character name in `Dumas H3 Prompt Curator` can include both helper images and the character facts automatically.
- `Dumas Location Helper`
- Inputs: `image1`, `image2`, picture IDs, `location_id`, `name`, `alias`, `description`, `general`
- Outputs: `image1`, `image2`, `reference_prompt`, `reference1`, `reference2`
- Matching general-purpose helper for environments/locations: pass two images through unchanged, emit location reference prompt text, and provide two structured `REFERENCE` objects for the prompt curator.
- The structured references carry the same location name, alias, description, and notes, so mentioning the location name in `Dumas H3 Prompt Curator` can include both helper images and the location context automatically.
- `Dumas Anchor Style`
- Inputs: `anchor_style`, `style_description`
- Output: `anchor`
- Offers a large preset dropdown of anchor-style titles such as cinematic action movie, comedy, found footage, 90s sitcom, mobile/cell phone captured, news broadcast, mockumentary, heist thriller, cyberpunk neon, nature documentary, courtroom drama, and more.
- The preset wording is tuned for H3-safe persistent anchors: camera language, lighting, texture, production treatment, and tone, without naming characters or describing one-off actions.
- Selecting a preset fills the editable description field, and the edited multiline description is the `STRING` value passed downstream into H3 anchor sockets such as `anchor_override`.
- Selecting a preset fills the editable description field, and the edited multiline description is the `STRING` value passed downstream.
- `Dumas Soundscape Helper`
- Inputs: `soundscape`, `soundscape_description`
- Output: `soundscape`
- Matching soundscape helper for standalone H3 prompts. Pick a preset such as quiet interior, rainy street, cafe, city night, forest, industrial, or silent; the preset fills the editable textbox, and the edited text flows into `Dumas H3 Prompt Curator`.
- `Dumas Background Music Helper`
- Inputs: `bgm`, `bgm_description`
- Output: `bgm`
- Matching BGM helper for standalone H3 prompts. Pick a preset such as subtle tension, cinematic suspense, emotional piano, dark ambient, hopeful orchestral, retro synth, action pulse, lo-fi, or no vocals; the preset fills the editable textbox, and the edited text flows into `Dumas H3 Prompt Curator`.
- `Dumas JSON String to Object`
- Input: `json_string`
@@ -233,7 +269,7 @@ decr -> use index - 1
`Dumas H3 Plan Attach Scene Images` and `Dumas H3 Plan Extract Scene Images` are a companion pair for `ComfyUI-MiniMaxH3-Contex-Loop` and the local `ref2v` lane. The upstream H3 plan node cannot dynamically grow nine new image sockets for every JSON-defined scene, so Dumas stores scene image bindings beside the plan using a lightweight token and an in-memory registry. That keeps `plan.json` archiving intact while still letting you wire up nine IMAGE sockets per scene through chained helper nodes.
`Dumas Character Reference` and `Dumas Location Reference` live in `Dumas/MiniMax`. Both output a structured `REFERENCE` object that carries the image plus its semantic payload. `Dumas H3 Long Videos` accepts those `REFERENCE` sockets directly on `ref_1`..`ref_9`, resolves `<Picture N>` against the wired slot positions, and can also pull character wardrobe context from the structured ref data when `character_memory` is left blank.
`Dumas Character Helper` is the restored two-image/text helper for general H3 workflows, and `Dumas Location Helper` mirrors it for scene/environment references. Both helpers also emit structured `REFERENCE` sockets for the curator. The structured `Dumas Character Reference` and `Dumas Location Reference` nodes remain available separately for workflows that want a single `REFERENCE` socket. `Dumas H3 Prompt Curator` consumes those structured references plus optional anchor, soundscape, and BGM strings, assigns the final `<Picture N>` numbering, and outputs only the compacted images the prompt actually mentions.
`Dumas Strip Iteration Suffix` keeps the part before the first underscore and drops the rest. Names like `char123_pose_final.png` become `char123.png`, while names with no underscore such as `char123.png` are left untouched.
+6
View File
@@ -14,6 +14,10 @@ from .dumas_h3_longvideos import (
NODE_CLASS_MAPPINGS as H3_LONGVIDEO_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as H3_LONGVIDEO_NODE_DISPLAY_NAME_MAPPINGS,
)
from .dumas_h3_latent_upscale import (
NODE_CLASS_MAPPINGS as H3_LATENT_UPSCALE_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as H3_LATENT_UPSCALE_NODE_DISPLAY_NAME_MAPPINGS,
)
from .dumas_h3_shot_length import (
NODE_CLASS_MAPPINGS as H3_SHOT_LENGTH_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as H3_SHOT_LENGTH_NODE_DISPLAY_NAME_MAPPINGS,
@@ -31,6 +35,7 @@ NODE_CLASS_MAPPINGS = {}
NODE_CLASS_MAPPINGS.update(JSON_NODE_CLASS_MAPPINGS)
NODE_CLASS_MAPPINGS.update(IMAGE_NODE_CLASS_MAPPINGS)
NODE_CLASS_MAPPINGS.update(H3_LONGVIDEO_NODE_CLASS_MAPPINGS)
NODE_CLASS_MAPPINGS.update(H3_LATENT_UPSCALE_NODE_CLASS_MAPPINGS)
NODE_CLASS_MAPPINGS.update(H3_SHOT_LENGTH_NODE_CLASS_MAPPINGS)
NODE_CLASS_MAPPINGS.update(H3_INSPECTOR_NODE_CLASS_MAPPINGS)
NODE_CLASS_MAPPINGS.update(H3_BEAT_PROMPT_NODE_CLASS_MAPPINGS)
@@ -39,6 +44,7 @@ NODE_DISPLAY_NAME_MAPPINGS = {}
NODE_DISPLAY_NAME_MAPPINGS.update(JSON_NODE_DISPLAY_NAME_MAPPINGS)
NODE_DISPLAY_NAME_MAPPINGS.update(IMAGE_NODE_DISPLAY_NAME_MAPPINGS)
NODE_DISPLAY_NAME_MAPPINGS.update(H3_LONGVIDEO_NODE_DISPLAY_NAME_MAPPINGS)
NODE_DISPLAY_NAME_MAPPINGS.update(H3_LATENT_UPSCALE_NODE_DISPLAY_NAME_MAPPINGS)
NODE_DISPLAY_NAME_MAPPINGS.update(H3_SHOT_LENGTH_NODE_DISPLAY_NAME_MAPPINGS)
NODE_DISPLAY_NAME_MAPPINGS.update(H3_INSPECTOR_NODE_DISPLAY_NAME_MAPPINGS)
NODE_DISPLAY_NAME_MAPPINGS.update(H3_BEAT_PROMPT_NODE_DISPLAY_NAME_MAPPINGS)
+10 -1
View File
@@ -150,7 +150,16 @@ class H3ModelInspector:
@classmethod
def INPUT_TYPES(cls):
return {"required": {"model": ("MODEL",)}}
return {
"required": {
"model": (
"MODEL",
{
"tooltip": "MiniMax / H3 model to inspect for quantization and tensor format."
},
)
}
}
def inspect(self, model):
label, _counts, report = _detect(model)
File diff suppressed because it is too large Load Diff
+606 -68
View File
@@ -32,9 +32,10 @@ and your VRAM, chains them, and returns the finished video + audio.
Requirements: H3 is CFG-free (cfg 1) and needs no negative prompt -- the node
makes an empty one internally. The main pass keeps denoise fixed at 1.0: a
partial denoise desyncs the joint audio/video schedule. An optional refinement
pass can use its own denoise later, before any upscale, while keeping the output
video-only.
partial denoise desyncs the joint audio/video schedule. An optional latent
upscale stage can rebuild the conditioning at a target size, run a short
refinement pass with its own sampler controls, and keep the output video-only
before the final pixel-space upscale options.
Verified against ComfyUI core (comfy_extras/nodes_minimax_h3.py, model_base.py,
ldm/minimax/model.py, text_encoders/minimax.py, sd.py).
@@ -59,6 +60,7 @@ import node_helpers
try:
from . import dumas_h3_overlay as _overlay
from .dumas_h3_latent_upscale import upscale_latent_video as _upscale_latent_video
from . import dumas_image_nodes as _image_nodes
except ImportError: # loaded as a bare file (test_prompt_logic.py), not as a package
import importlib.util as _ilu
@@ -70,6 +72,13 @@ except ImportError: # loaded as a bare file (test_prompt_logic.py), not as
)
_overlay = _ilu.module_from_spec(_spec)
_spec.loader.exec_module(_overlay)
_latent_spec = _ilu.spec_from_file_location(
"dumas_h3_latent_upscale",
_os.path.join(_os.path.dirname(_os.path.abspath(__file__)), "dumas_h3_latent_upscale.py"),
)
_latent_upscale = _ilu.module_from_spec(_latent_spec)
_latent_spec.loader.exec_module(_latent_upscale)
_upscale_latent_video = _latent_upscale.upscale_latent_video
_image_nodes = _sys.modules.get("dumas_image_nodes")
if _image_nodes is None:
_img_spec = _ilu.spec_from_file_location(
@@ -279,8 +288,6 @@ ADDED_WIDGETS = (
"exposed_terms", "anatomy_guard", "lock_restraints", "solidity_guard",
"motion_guard", "contact_guard",
"auto_soundscape", "allow_nonspeech_vocals",
"detail_pass", "detail_sampler_name", "detail_scheduler",
"detail_steps", "detail_denoise",
)
NL = "\n"
@@ -4033,6 +4040,88 @@ def _copy_sample_latent(out_latent):
return None
def _retarget_conditioning_spatial(cond, latent_h, latent_w):
"""Resize H3 keyframe latents in existing conditioning to a new latent grid."""
latent_h = int(latent_h)
latent_w = int(latent_w)
if latent_h <= 0 or latent_w <= 0:
raise RuntimeError("conditioning target latent size must be positive")
out = []
for item in cond:
try:
tensor, data = item
except Exception:
out.append(item)
continue
nd = dict(data)
keyframes = nd.get("minimax_keyframes")
if keyframes:
resized_keyframes = []
for keyframe in keyframes:
nkf = dict(keyframe)
latent_value = nkf.get("latent")
if latent_value is not None and len(getattr(latent_value, "shape", ())) >= 5:
if latent_value.shape[3] != latent_h or latent_value.shape[4] != latent_w:
b, c, t, h, w = latent_value.shape
resized = torch.nn.functional.interpolate(
latent_value.to(torch.float32).reshape(b * t, c, h, w),
size=(latent_h, latent_w),
mode="bilinear",
align_corners=False,
).reshape(b, c, t, latent_h, latent_w)
nkf["latent"] = resized.to(device=latent_value.device, dtype=latent_value.dtype)
resized_keyframes.append(nkf)
nd["minimax_keyframes"] = resized_keyframes
out.append([tensor, nd])
return out
def _pad_to_h3_patch_size(tensor):
try:
import comfy.ldm.common_dit as common_dit
return common_dit.pad_to_patch_size(tensor, (1, 2, 2))
except Exception:
return tensor
def _crop_conditioning_to_tile(cond, source_h, source_w, row, col, tile_h, tile_w):
"""Crop H3 keyframe latents in existing conditioning for a spatial tile."""
out = []
for item in cond:
try:
tensor, data = item
except Exception:
out.append(item)
continue
nd = dict(data)
keyframes = nd.get("minimax_keyframes")
if keyframes:
cropped_keyframes = []
for keyframe in keyframes:
nkf = dict(keyframe)
latent_value = nkf.get("latent")
if latent_value is not None and len(getattr(latent_value, "shape", ())) >= 5:
kh, kw = latent_value.shape[3], latent_value.shape[4]
if kh != source_h or kw != source_w:
b, c, t, h, w = latent_value.shape
latent_value = torch.nn.functional.interpolate(
latent_value.to(torch.float32).reshape(b * t, c, h, w),
size=(source_h, source_w),
mode="bilinear",
align_corners=False,
).reshape(b, c, t, source_h, source_w).to(
device=latent_value.device,
dtype=latent_value.dtype,
)
nkf["latent"] = _pad_to_h3_patch_size(
latent_value[:, :, :, row:row + tile_h, col:col + tile_w].contiguous()
)
cropped_keyframes.append(nkf)
nd["minimax_keyframes"] = cropped_keyframes
out.append([tensor, nd])
return out
def _latent_with_replaced_samples(template_latent, sampled_latent):
"""Reuse the original latent payload, but swap in freshly sampled tensors."""
if not isinstance(template_latent, dict):
@@ -4066,6 +4155,237 @@ def _video_only_refined_latent(base_latent, refined_latent):
return refined_latent
def _latent_upscale_target_size(base_w, base_h, param):
width = int(param.get("width", 0) or 0)
height = int(param.get("height", 0) or 0)
if width > 0 and height > 0:
return width, height
megapixels = float(param.get("megapixels", 0.0) or 0.0)
if megapixels > 0:
return scale_to_megapixels(base_w, base_h, megapixels)
return int(base_w), int(base_h)
def _latent_upscale_mode(param):
if not isinstance(param, dict):
return "off"
mode = str(param.get("mode") or "").strip()
if mode:
return mode
if "model_name" in param and str(param.get("model_name") or "").strip() not in ("", "none"):
return "model"
if "method" in param:
return "interp"
return "off"
def _latent_spatial_grid(h, w, th, tw, ol_h, ol_w):
if th <= 0 or tw <= 0:
raise ValueError("tile dimensions must be positive")
if ol_h >= th or ol_w >= tw:
raise ValueError("overlap must be smaller than the tile size")
sh = th - ol_h
sw = tw - ol_w
nrows = 1 if h <= th else math.ceil((h - ol_h) / sh)
if (nrows - 1) * sh + th < h:
nrows += 1
ncols = 1 if w <= tw else math.ceil((w - ol_w) / sw)
if (ncols - 1) * sw + tw < w:
ncols += 1
rows = [i * sh for i in range(nrows)]
cols = [j * sw for j in range(ncols)]
trows = [min(th, h - r) for r in rows]
tcols = [min(tw, w - c) for c in cols]
return rows, cols, trows, tcols
def _latent_spatial_blend_weights(t, overlap_mode, overlap_blend="linear"):
if overlap_blend == "overwrite":
return torch.ones_like(t) if overlap_mode == "later" else torch.zeros_like(t)
if overlap_blend == "midpoint":
base = (t >= 0.5).to(t.dtype)
elif overlap_blend == "smoothstep":
base = t * t * (3.0 - 2.0 * t)
else:
base = t
if overlap_mode == "later":
return base
return 1.0 - base
def _param_value(mapping, key, default):
value = mapping.get(key, default)
return default if value is None else value
def _tag_oom_stage(exc, stage):
if _is_oom(exc):
exc._h3_stage = stage
return exc
def _grid_1d(size, tile, ol, min_tile):
if size <= tile:
return [0], [size], [0]
sh = tile - ol
n = math.ceil((size - ol) / sh)
if (n - 1) * sh + tile < size:
n += 1
rows = [i * sh for i in range(n)]
trows = [min(tile, size - r) for r in rows]
if min_tile > 0 and n >= 2:
edge = size - rows[-1]
if edge < min_tile:
new_last = size - min_tile
if rows[-2] < new_last < rows[-2] + trows[-2]:
rows[-1] = new_last
trows[-1] = size - new_last
ovl = [0] * n
for i in range(1, n):
ovl[i] = max(0, rows[i - 1] + trows[i - 1] - rows[i])
return rows, trows, ovl
def compute_spatial_grid(h, w, th, tw, ol_h, ol_w, min_th=0, min_tw=0):
if th <= 0 or tw <= 0:
raise ValueError("tile dimensions must be positive")
if ol_h >= th or ol_w >= tw:
raise ValueError("overlap must be smaller than the tile size")
if min_th < 0 or min_tw < 0:
raise ValueError("minimum tile size must be non-negative")
if min_th > th or min_tw > tw:
raise ValueError("minimum tile size must not exceed the tile size")
rows, trows, row_ovl = _grid_1d(h, th, ol_h, min_th)
cols, tcols, col_ovl = _grid_1d(w, tw, ol_w, min_tw)
return rows, cols, trows, tcols, row_ovl, col_ovl
def spatial_fade_mask(tile_h, tile_w, ol_h, ol_w, done_top, done_left, fade_h=0, fade_w=0):
mask = torch.ones(tile_h, tile_w, dtype=torch.float32)
if done_left and ol_w > 0:
if fade_w == 0:
mask[:, :ol_w] = 0.0
else:
f = min(fade_w, ol_w)
frozen_w = ol_w - f
w = torch.linspace(0.0, 1.0, f)
mask[:, :frozen_w] = 0.0
mask[:, frozen_w:ol_w] = torch.minimum(mask[:, frozen_w:ol_w], w[None, :])
if done_top and ol_h > 0:
if fade_h == 0:
mask[:ol_h, :] = 0.0
else:
f = min(fade_h, ol_h)
frozen_h = ol_h - f
w = torch.linspace(0.0, 1.0, f)
mask[:frozen_h, :] = 0.0
mask[frozen_h:ol_h, :] = torch.minimum(mask[frozen_h:ol_h, :], w[:, None])
return mask
def _fade_band(band, fade, axis):
n = band.shape[axis]
f = min(int(fade), n)
if f == 0:
band[:] = 0.0
return
w = torch.linspace(0.0, 1.0, f, dtype=band.dtype, device=band.device)
frozen = n - f
if axis == 1:
w = w[None, :]
band[:, :frozen] = torch.minimum(
band[:, :frozen], torch.zeros(frozen, dtype=band.dtype, device=band.device)
)
band[:, frozen:] = torch.minimum(band[:, frozen:], w)
else:
w = w[:, None]
band[:frozen, :] = torch.minimum(
band[:frozen, :], torch.zeros((frozen, 1), dtype=band.dtype, device=band.device)
)
band[frozen:, :] = torch.minimum(band[frozen:, :], w)
def make_fade_mask(tile_h, tile_w, ol_h, ol_w, done_top, done_left, fade_h=0, fade_w=0):
mask = torch.ones(tile_h, tile_w, dtype=torch.float32)
if done_left and ol_w > 0:
_fade_band(mask[:, :ol_w], fade_w, 1)
if done_top and ol_h > 0:
_fade_band(mask[:ol_h, :], fade_h, 0)
return mask
def bright_match_tile(tile, ref, clamp=0.05):
d = (tile - ref).float().reshape(tile.shape[0], tile.shape[1], tile.shape[2], -1)
dc = d.median(dim=-1).values.clamp(-clamp, clamp)
return tile - dc.to(tile.dtype).view(tile.shape[0], tile.shape[1], tile.shape[2], 1, 1)
def _dynamic_fade_closure(sp, fw, fh, tr, tc, tr_s, tc_s, ovh, ovw, done_top, done_left, video_flat, mn=0.0):
schedule = sp.get("dynamic_fade", "off")
if schedule == "off":
return None
fmin_w = int(sp.get("dynamic_fade_min", 0)) // 16
fmin_h = int(sp.get("dynamic_fade_min", 0)) // 16
if fw <= fmin_w and fh <= fmin_h:
return None
fw_start, fh_start = max(fw, 0), max(fh, 0)
fmin_w, fmin_h = min(fmin_w, fw_start), min(fmin_h, fh_start)
done_top = done_top and ovh > 0
done_left = done_left and ovw > 0
s_tok = tr_s * tc_s
n_frames = video_flat // s_tok
def fade_at(p):
if schedule == "widening":
return fmin_w + (fw_start - fmin_w) * p, fmin_h + (fh_start - fmin_h) * p
return fw_start - (fw_start - fmin_w) * p, fh_start - (fh_start - fmin_h) * p
cache = {}
def step_fn(sigma, denoise_mask, **kwargs):
sigmas = kwargs.get("extra_options", {}).get("sigmas")
n_sigmas = int(sigmas.numel()) if sigmas is not None else 0
masks = cache.get(n_sigmas)
if masks is None:
step_count = max(n_sigmas - 1, 1)
masks = []
for i in range(n_sigmas - 1):
p = i / (step_count - 1) if step_count > 1 else 0.0
cw, ch = fade_at(p)
m = make_fade_mask(tr_s, tc_s, ovh, ovw, done_top, done_left,
fade_h=round(ch), fade_w=round(cw))
m[tr:tr_s, :] = 0.0
m[:, tc:tc_s] = 0.0
if mn > 0:
m = m + mn * (1.0 - m)
masks.append(m)
cache[n_sigmas] = masks
idx = 0
if sigmas is not None:
idx = int((sigmas > sigma + 1e-6).sum())
m = masks[min(idx, len(masks) - 1)]
flat = denoise_mask.clone()
flat.reshape(denoise_mask.shape[0], -1)[:, :video_flat] = \
m.reshape(1, -1).repeat(denoise_mask.shape[0], n_frames)
return flat
return step_fn
def _nested_tensor_parts(samples):
if samples is None:
return ()
parts = getattr(samples, "tensors", None)
if parts is not None:
return tuple(parts)
if hasattr(samples, "unbind"):
try:
return tuple(samples.unbind())
except Exception:
return ()
return ()
def _coerce_bool_flag(value):
if isinstance(value, str):
text = value.strip().lower()
@@ -4097,7 +4417,7 @@ def _format_timing_note(shot_timings):
"total": 0.0,
"retry_elapsed": 0.0,
"sample": 0.0,
"detail_sample": 0.0,
"latent_upscale_sample": 0.0,
"decode_video": 0.0,
"decode_audio": 0.0,
"cleanup": 0.0,
@@ -4108,7 +4428,7 @@ def _format_timing_note(shot_timings):
totals["total"] += float(shot.get("total", 0.0) or 0.0)
totals["retry_elapsed"] += float(shot.get("retry_elapsed", 0.0) or 0.0)
totals["sample"] += float(shot.get("sample", 0.0) or 0.0)
totals["detail_sample"] += float(shot.get("detail_sample", 0.0) or 0.0)
totals["latent_upscale_sample"] += float(shot.get("latent_upscale_sample", 0.0) or 0.0)
totals["decode_video"] += float(shot.get("decode_video", 0.0) or 0.0)
totals["decode_audio"] += float(shot.get("decode_audio", 0.0) or 0.0)
totals["cleanup"] += float(shot.get("cleanup", 0.0) or 0.0)
@@ -4124,8 +4444,8 @@ def _format_timing_note(shot_timings):
]
if totals["retry_elapsed"]:
pieces.append(f"retry elapsed {_format_elapsed_seconds(totals['retry_elapsed'])}")
if totals["detail_sample"]:
pieces.append(f"detail {_format_elapsed_seconds(totals['detail_sample'])}")
if totals["latent_upscale_sample"]:
pieces.append(f"latent upscale {_format_elapsed_seconds(totals['latent_upscale_sample'])}")
if totals["retries"]:
pieces.append(f"retries {totals['retries']}")
if slowest is not None:
@@ -5847,6 +6167,29 @@ def _evict_all_but(keep_model):
pass
def _evict_for_latent_upscale(model):
"""Clear the sampler model before loading the auxiliary latent upscaler."""
try:
unload_clones = getattr(mm, "unload_model_and_clones", None)
if callable(unload_clones):
try:
unload_clones(model, unload_additional_models=False)
mm.soft_empty_cache()
return
except Exception:
pass
mm.unload_all_models()
except Exception:
pass
try:
mm.soft_empty_cache(True)
except Exception:
try:
mm.soft_empty_cache()
except Exception:
pass
@@ -6084,6 +6427,12 @@ class H3LongVideos:
"decode_tile_size": ("INT", {"default": 0, "min": 0, "max": 1024, "step": 32,
"tooltip": "Spatial tile size for the VAE decode (tile_x/tile_y). 0 = ComfyUI default. "
"Try 256 on a tight card at 1344x768."}),
"latent_upscale_param": ("DUMAS_H3_LATENT_UPSCALE_PARAM", {
"tooltip": "Output of 'Dumas H3 Latent Upscale Params'. When connected, the first-pass "
"latent is upscaled and run through a short refinement pass before decode, "
"using the sampler, scheduler, steps, denoise, megapixel target, and optional "
"spatial batching from that node. "
"Leave unconnected to skip latent upscaling entirely."}),
"upscale": (["off", "rtx", "model", "lanczos"], {"default": "off",
"tooltip": "Optional post-pass on the finished frames. 'rtx' = NVIDIA RTX Video Super "
"Resolution (Tensor Cores -- fastest and best for video; needs the "
@@ -6363,26 +6712,6 @@ class H3LongVideos:
"the jacket, 'wardrobe: += sunglasses' adds one. TWO+ PEOPLE: name them -- "
"'Maya = grey shorts, red jacket; Jon = navy overalls', then edit one at a "
"time: 'wardrobe: Maya -= jacket' leaves Jon untouched."}),
"detail_pass": ("BOOLEAN", {"default": False,
"tooltip": "Run a second refinement sampler on each beat BEFORE any upscale. "
"It reuses the same conditioning and keeps the output video-only by "
"preserving the first pass's audio latent. Use it for detail cleanup, not "
"for huge rewrites: too many steps or too much denoise can pull identity or "
"continuity away from the main pass."}),
"detail_sampler_name": (comfy.samplers.KSampler.SAMPLERS, {"default": "euler",
"tooltip": "Sampler for the optional refinement pass. Euler is the maintained default "
"direction for this lane."}),
"detail_scheduler": (comfy.samplers.KSampler.SCHEDULERS, {"default": "beta",
"tooltip": "Scheduler for the optional refinement pass. Beta is the maintained default "
"direction for the H3 enhancement lane."}),
"detail_steps": ("INT", {"default": 8, "min": 1, "max": 200,
"tooltip": "Extra steps for the refinement pass only. Start around 4-8. More is not "
"automatically better; once the pass starts rewriting instead of polishing, "
"identity and continuity can drift."}),
"detail_denoise": ("FLOAT", {"default": 0.4, "min": 0.0, "max": 1.0, "step": 0.01,
"tooltip": "How hard the refinement pass is allowed to rewrite the beat latent. Start "
"around 0.20-0.35 for gentle cleanup; 0.4+ is stronger and can noticeably "
"change faces, motion or composition."}),
},
# Read-only graph access, for SLA-LoRA detection: a LoRA's filename is
# the only thing that identifies an SLA build, and the graph is the only
@@ -6407,9 +6736,8 @@ class H3LongVideos:
def _render(self, model, clip, vae, audio_vae, negative, prompt, w, h, ln, fps, tiled, sa,
handoff, decode_tile_frames=0, decode_tile_size=0,
refs=None, ref_image_size="match", ref_noise_aug=None, silent=False,
detail_pass=False, detail_sampler_name="euler", detail_scheduler="beta",
detail_steps=8, detail_denoise=0.4, timing_sink=None):
timing = {"sample": 0.0, "detail_sample": 0.0, "decode_video": 0.0, "decode_audio": 0.0, "cleanup": 0.0}
latent_upscale_param=None, timing_sink=None):
timing = {"sample": 0.0, "latent_upscale_sample": 0.0, "decode_video": 0.0, "decode_audio": 0.0, "cleanup": 0.0}
positive, latent = _build_shot_conditioning(clip, vae, prompt, w, h, ln, fps, handoff,
ref_images=refs, ref_image_size=ref_image_size,
ref_noise_aug=ref_noise_aug,
@@ -6428,43 +6756,213 @@ class H3LongVideos:
# OOM retry cannot help an OOM raised here -- it just re-runs the whole
# sampling pass and fails the same way, which on a 362-frame shot is four
# more minutes for nothing.
if _is_oom(e):
e._h3_stage = "sampling"
raise
raise _tag_oom_stage(e, "sampling")
refined_out = out
detail_pass = _coerce_bool_flag(detail_pass)
if detail_pass:
detail_latent = _latent_with_replaced_samples(latent, out)
latent_upscale_param = latent_upscale_param or None
latent_upscale_mode = _latent_upscale_mode(latent_upscale_param)
if latent_upscale_mode != "off":
try:
detail_start = time.perf_counter()
latent_start = time.perf_counter()
out_samples = out["samples"]
parts = _nested_tensor_parts(out_samples)
if not getattr(out_samples, "is_nested", False) or len(parts) < 2:
raise RuntimeError("latent upscale expects a nested AV latent")
if latent_upscale_mode == "model" and str(latent_upscale_param.get("device", "cuda")) == "cuda":
_evict_for_latent_upscale(model)
upscaled_video, up_h, up_w = _upscale_latent_video(parts[0], latent_upscale_param)
full_audio = parts[1]
# Drop the first-pass sampling state before we start the refinement
# pass; otherwise the 12-step base latent and the upscale latent sit
# in memory together and can trigger a retry loop.
out["samples"] = comfy.nested_tensor.NestedTensor((upscaled_video, full_audio))
del out_samples, parts
mm.soft_empty_cache()
target_w = int(up_w) * 16
target_h = int(up_h) * 16
if target_w <= 0 or target_h <= 0:
raise RuntimeError("latent upscale target size must be positive")
try:
upscale_cond = _retarget_conditioning_spatial(positive, int(up_h), int(up_w))
upscale_latent = dict(latent) if isinstance(latent, dict) else {}
except Exception:
upscale_cond, upscale_latent = _build_shot_conditioning(
clip, vae, prompt, target_w, target_h, ln, fps, handoff,
ref_images=refs, ref_image_size=ref_image_size,
ref_noise_aug=ref_noise_aug, audio_vae=audio_vae, silent=silent)
upscale_latent["samples"] = comfy.nested_tensor.NestedTensor((upscaled_video, full_audio))
del positive, latent
refine_steps = int(latent_upscale_param.get("steps", 2) or 2)
refine_sampler = latent_upscale_param.get("sampler_name", sn)
refine_scheduler = latent_upscale_param.get("scheduler", sch)
refine_denoise_value = latent_upscale_param.get("denoise", latent_upscale_param.get("refine_denoise", 0.2))
refine_denoise = 0.2 if refine_denoise_value is None else float(refine_denoise_value)
tile_size_mode = str(latent_upscale_param.get("tile_size_mode", "specific_size"))
tile_w_px = int(latent_upscale_param.get("tile_width", 512) or 512)
tile_h_px = int(latent_upscale_param.get("tile_height", 512) or 512)
overlap_px = max(0, int(_param_value(latent_upscale_param, "overlap", 64)))
fade_w_px = max(0, int(_param_value(latent_upscale_param, "fade_width", 32)))
fade_h_px = max(0, int(_param_value(latent_upscale_param, "fade_height", 32)))
overlap_mode = str(latent_upscale_param.get("overlap_mode", "earlier"))
overlap_blend = str(latent_upscale_param.get("overlap_blend", "linear"))
grid_rows = max(1, int(_param_value(latent_upscale_param, "grid_rows", 2)))
grid_cols = max(1, int(_param_value(latent_upscale_param, "grid_cols", 2)))
spatial_w_overlap_px = max(0, int(_param_value(latent_upscale_param, "spatial_w_overlap", overlap_px)))
spatial_h_overlap_px = max(0, int(_param_value(latent_upscale_param, "spatial_h_overlap", overlap_px)))
min_tile_size_px = max(0, int(_param_value(latent_upscale_param, "min_tile_size", 256)))
masked_area_noise = float(_param_value(latent_upscale_param, "masked_area_noise", 0.0))
brightness_match = bool(latent_upscale_param.get("brightness_match", False))
dynamic_fade = str(latent_upscale_param.get("dynamic_fade", "off"))
dynamic_fade_min_px = max(0, int(_param_value(latent_upscale_param, "dynamic_fade_min", 32)))
if tile_size_mode == "rows_cols":
tile_w_px, spatial_w_overlap_px = _solve_equal_tiles(target_w, grid_cols, spatial_w_overlap_px, 16)
tile_h_px, spatial_h_overlap_px = _solve_equal_tiles(target_h, grid_rows, spatial_h_overlap_px, 16)
if tile_w_px < min_tile_size_px or tile_h_px < min_tile_size_px:
raise ValueError(
f"rows_cols mode: solved tile size is {tile_h_px}x{tile_w_px}px "
f"(grid {grid_rows}x{grid_cols} over {target_h}x{target_w}px), "
f"which is smaller than min_tile_size ({min_tile_size_px}px). "
f"Reduce grid_rows/grid_cols, or lower min_tile_size to at most "
f"{min(tile_w_px, tile_h_px)}px."
)
fade_w_px = min(fade_w_px, spatial_w_overlap_px)
fade_h_px = min(fade_h_px, spatial_h_overlap_px)
else:
for name, value in (
("tile_width", tile_w_px),
("tile_height", tile_h_px),
("overlap", overlap_px),
("fade_width", fade_w_px),
("fade_height", fade_h_px),
("min_tile_size", min_tile_size_px),
):
if value % 32 != 0:
raise ValueError(f"'{name}' must be a multiple of 32 pixels; got {value}.")
if overlap_px >= tile_w_px:
raise ValueError("overlap must be smaller than tile_width")
if overlap_px >= tile_h_px:
raise ValueError("overlap must be smaller than tile_height")
if fade_w_px > spatial_w_overlap_px:
raise ValueError("fade_width must not exceed spatial_w_overlap")
if fade_h_px > spatial_h_overlap_px:
raise ValueError("fade_height must not exceed spatial_h_overlap")
if min_tile_size_px > tile_w_px or min_tile_size_px > tile_h_px:
raise ValueError("min_tile_size must not exceed the tile size")
tile_tw = max(1, tile_w_px // 16)
tile_th = max(1, tile_h_px // 16)
ol_tw = max(0, min(tile_tw - 1, spatial_w_overlap_px // 16))
ol_th = max(0, min(tile_th - 1, spatial_h_overlap_px // 16))
fw_tw = max(0, min(ol_tw, fade_w_px // 16))
fw_th = max(0, min(ol_th, fade_h_px // 16))
min_tile_tw = max(0, min_tile_size_px // 16)
rows, cols, trows, tcols, row_ovl, col_ovl = compute_spatial_grid(
int(up_h), int(up_w), tile_th, tile_tw, ol_th, ol_tw, min_tile_tw, min_tile_tw
)
logging.info(
"H3 latent refine: %s spatial sampler tiles, target=%sx%s tile_mode=%s tile=%sx%s overlap=%sx%s",
len(rows) * len(cols), target_w, target_h, tile_size_mode, tile_w_px, tile_h_px,
spatial_w_overlap_px, spatial_h_overlap_px,
)
if len(rows) == 1 and len(cols) == 1:
(refined_out,) = nodes.common_ksampler(
model, seed, int(detail_steps), cfg, detail_sampler_name, detail_scheduler,
positive, negative, detail_latent, denoise=float(detail_denoise))
timing["detail_sample"] += time.perf_counter() - detail_start
model, seed, refine_steps, cfg, refine_sampler, refine_scheduler, upscale_cond, negative, upscale_latent,
denoise=refine_denoise)
else:
refined_video = upscaled_video.clone()
for row_index, r0 in enumerate(rows):
tr = trows[row_index]
ovh = row_ovl[row_index]
for col_index, c0 in enumerate(cols):
tc = tcols[col_index]
ovw = col_ovl[col_index]
tile_cond = _crop_conditioning_to_tile(
upscale_cond, int(up_h), int(up_w), r0, c0, tr, tc
)
tile_latent = dict(upscale_latent) if isinstance(upscale_latent, dict) else {}
tile_video = upscaled_video[:, :, :, r0:r0 + tr, c0:c0 + tc].contiguous()
tr_s = tr + (tr % 2)
tc_s = tc + (tc % 2)
tile = torch.zeros((1, tile_video.shape[1], tile_video.shape[2], tr_s, tc_s),
device=tile_video.device, dtype=tile_video.dtype)
tile[:, :, :, :tr, :tc] = tile_video
if col_index > 0 and ovw > 0:
tile[:, :, :, :tr, :ovw] = refined_video[:, :, :, r0:r0 + tr, c0:c0 + ovw]
if row_index > 0 and ovh > 0:
tile[:, :, :, :ovh, :tc] = refined_video[:, :, :, r0:r0 + ovh, c0:c0 + tc]
mask = make_fade_mask(tr_s, tc_s, ovh, ovw, row_index > 0, col_index > 0,
fade_h=fw_th, fade_w=fw_tw)
mask[tr:tr_s, :] = 0.0
mask[:, tc:tc_s] = 0.0
mv = (mask + masked_area_noise * (1.0 - mask))[None, None, None].to(tile.dtype)
ma = torch.zeros((1, 32, 2, full_audio.shape[-1]), device=full_audio.device, dtype=full_audio.dtype)
tile_latent["samples"] = comfy.nested_tensor.NestedTensor((tile, full_audio))
tile_latent["noise_mask"] = comfy.nested_tensor.NestedTensor((mv, ma))
dynamic = _dynamic_fade_closure(
latent_upscale_param, fw_tw, fw_th, tr, tc, tr_s, tc_s, ovh, ovw,
row_index > 0, col_index > 0, math.prod(tile.shape[1:]), mn=masked_area_noise
)
if dynamic is not None:
model.set_model_denoise_mask_function(dynamic)
try:
tile_out, = nodes.common_ksampler(
model, seed, refine_steps, cfg, refine_sampler, refine_scheduler, tile_cond, negative, tile_latent,
denoise=refine_denoise)
finally:
if dynamic is not None:
model.model_options.pop("denoise_mask_function", None)
tile_out = _video_only_refined_latent(
{"samples": comfy.nested_tensor.NestedTensor((tile_video, full_audio))},
tile_out)
tile_video_out = tile_out["samples"].tensors[0]
if brightness_match:
tile_video_out = bright_match_tile(
tile_video_out,
upscaled_video[:, :, :, r0:r0 + tr, c0:c0 + tc]
)
region = refined_video[:, :, :, r0:r0 + tr, c0:c0 + tc]
base_region = region.clone()
region.copy_(tile_video_out)
if col_index > 0 and ol_tw > 0:
t = torch.linspace(0.0, 1.0, ol_tw, device=region.device, dtype=region.dtype)
w = _latent_spatial_blend_weights(t, overlap_mode, overlap_blend)
if fw_tw > 0:
w = w.clone()
w[:fw_tw] = 0.0
region[:, :, :, :, :ol_tw] = (
base_region[:, :, :, :, :ol_tw] * (1.0 - w[None, None, None, None, :]) +
tile_video_out[:, :, :, :, :ol_tw] * w[None, None, None, None, :]
)
if row_index > 0 and ol_th > 0:
t = torch.linspace(0.0, 1.0, ol_th, device=region.device, dtype=region.dtype)
w = _latent_spatial_blend_weights(t, overlap_mode, overlap_blend)
if fw_th > 0:
w = w.clone()
w[:fw_th] = 0.0
region[:, :, :, :ol_th, :] = (
base_region[:, :, :, :ol_th, :] * (1.0 - w[None, None, None, :, None]) +
tile_video_out[:, :, :, :ol_th, :] * w[None, None, None, :, None]
)
refined_out = {"samples": comfy.nested_tensor.NestedTensor((refined_video, full_audio))}
timing["latent_upscale_sample"] += time.perf_counter() - latent_start
refined_out = _video_only_refined_latent(upscale_latent, refined_out)
del upscale_latent, upscaled_video, full_audio
mm.soft_empty_cache()
except Exception as e:
if _is_oom(e):
e._h3_stage = "sampling"
raise
refined_out = _video_only_refined_latent(out, refined_out)
del detail_latent
raise _tag_oom_stage(e, "latent_upscale")
# Keep a CPU copy of the sampled latent BEFORE decoding, for the `latent`
# output. Latents are ~1000x smaller than the frames they decode to (a
# 1344x768 124f shot is ~1.5MB against ~1.5GB), so carrying one per shot for
# the whole chain is free. Detached and moved off the card immediately, for
# the same reason the decoded frames are.
decode_video_start = time.perf_counter()
shot_latent = _copy_sample_latent(refined_out)
decode_audio_start = time.perf_counter()
audio = _decode_audio(audio_vae, out)
timing["decode_audio"] += time.perf_counter() - decode_audio_start
# Audio is much smaller than the video decode. Drop the first-pass
# conditioning before the VAE work so the optional detail pass does not
# keep both sampled latents resident across the heaviest allocation.
del out, positive, latent
decode_video_start = time.perf_counter()
shot_latent = _copy_sample_latent(refined_out)
video = _decode_video(vae, refined_out, tiled, free_first=model,
tile_t=decode_tile_frames, tile_xy=decode_tile_size)
timing["decode_video"] += time.perf_counter() - decode_video_start
cleanup_start = time.perf_counter()
del out
del refined_out
_deep_cleanup()
timing["cleanup"] += time.perf_counter() - cleanup_start
@@ -6500,8 +6998,7 @@ class H3LongVideos:
ref_5=None, ref_6=None, ref_7=None, ref_8=None,
ref_9=None,
ref_mode="auto ref2v", ref_image_size="match", ref_noise_aug=0.95,
detail_pass=False, detail_sampler_name="euler", detail_scheduler="beta",
detail_steps=8, detail_denoise=0.4,
latent_upscale_param=None,
graph=None, node_id=None):
# FIRST: detect a checkpoint swap since the previous execution and hard-flush.
@@ -6565,12 +7062,42 @@ class H3LongVideos:
ms_note = ""
if apply_model_sampling:
model, ms_note = apply_h3_model_sampling(model, shift_video, shift_audio)
detail_note = ""
if detail_pass:
detail_note = (f" detail pass: {int(detail_steps)} step(s) via "
f"{detail_sampler_name}/{detail_scheduler} at denoise "
f"{float(detail_denoise):.2f}; video-only refinement keeps "
f"audio from the first pass")
latent_upscale_note = ""
latent_upscale_mode = _latent_upscale_mode(latent_upscale_param)
if latent_upscale_mode != "off":
target_w, target_h = _latent_upscale_target_size(w, h, latent_upscale_param)
mode = latent_upscale_mode
detail = f" via {mode}"
if mode == "model":
detail += f"/{latent_upscale_param.get('model_name', 'none')}"
else:
detail += f"/{latent_upscale_param.get('method', 'bilinear')}"
denoise_value = latent_upscale_param.get("denoise", latent_upscale_param.get("refine_denoise", 0.2))
denoise = 0.2 if denoise_value is None else float(denoise_value)
refine_sampler = latent_upscale_param.get("sampler_name", "euler_ancestral")
refine_scheduler = latent_upscale_param.get("scheduler", "simple")
batch_note = ""
tile_size_mode = str(latent_upscale_param.get("tile_size_mode", "specific_size"))
tile_w_px = int(latent_upscale_param.get("tile_width", 512) or 512)
tile_h_px = int(latent_upscale_param.get("tile_height", 512) or 512)
overlap_px = max(0, int(_param_value(latent_upscale_param, "overlap", 64)))
overlap_blend = str(latent_upscale_param.get("overlap_blend", "linear"))
grid_rows = max(1, int(_param_value(latent_upscale_param, "grid_rows", 2)))
grid_cols = max(1, int(_param_value(latent_upscale_param, "grid_cols", 2)))
if tile_size_mode == "rows_cols":
batch_note = f"; spatial batches {grid_rows}x{grid_cols} rows_cols over {target_w}x{target_h}px"
elif tile_w_px > 0 and tile_h_px > 0 and (tile_w_px < target_w or tile_h_px < target_h):
spatial_w_overlap_px = max(0, int(_param_value(latent_upscale_param, "spatial_w_overlap", overlap_px)))
spatial_h_overlap_px = max(0, int(_param_value(latent_upscale_param, "spatial_h_overlap", overlap_px)))
batch_note = (
f"; spatial batches {tile_w_px}x{tile_h_px}px "
f"overlap {spatial_w_overlap_px}x{spatial_h_overlap_px}px {overlap_blend}"
)
latent_upscale_note = (
f" latent upscale: target {target_w}x{target_h}px{detail}; "
f"{int(latent_upscale_param.get('steps', 2) or 2)}-step refinement "
f"{refine_sampler}/{refine_scheduler} denoise {denoise:.2f}{batch_note}"
)
paras = split_paragraphs(prompt, "##")
if anchor_override.strip():
@@ -6881,6 +7408,7 @@ class H3LongVideos:
+ (" ANCHOR: " + "; ".join(anchor_hazards) + "."
if anchor_hazards else "")
+ (f"{anatomy_note}." if anatomy_note else "")
+ (f"{latent_upscale_note}." if latent_upscale_note else "")
+ (f"{plan_audio}." if plan_audio else "")
+ (" EXPOSURE -- " + "; ".join(wardrobe_notes) + "."
if wardrobe_notes else "")
@@ -7029,13 +7557,18 @@ class H3LongVideos:
model, clip, vae, audio_vae, negative, gen_prompt, w, h, ln_i, fps,
tiled, sa, shot_handoff, decode_tile_frames, decode_tile_size,
shot_refs, ref_image_size, shot_aug, shot_silent,
detail_pass, detail_sampler_name, detail_scheduler,
detail_steps, detail_denoise, timing_sink=shot_timing)
latent_upscale_param=latent_upscale_param, timing_sink=shot_timing)
break
except (torch.cuda.OutOfMemoryError, RuntimeError) as e:
shot_retry_elapsed += time.perf_counter() - attempt_start
if not _is_oom(e):
raise
if getattr(e, "_h3_stage", "") == "latent_upscale":
raise RuntimeError(
f"H3 Long Videos: shot {i + 1} of {len(gens)} ran out of VRAM "
f"during latent upscale. The latent-upscale stage is too large "
f"for this card at {w}x{h} with the current spatial settings."
) from e
mm.soft_empty_cache(True)
if not tiled:
tiled = True; backoff.append("tiled decode")
@@ -7052,10 +7585,16 @@ class H3LongVideos:
model, clip, vae, audio_vae, negative, gen_prompt, w, h, ln_i, fps,
tiled, sa, shot_handoff, decode_tile_frames, decode_tile_size,
shot_refs, ref_image_size, shot_aug, shot_silent,
detail_pass, detail_sampler_name, detail_scheduler,
detail_steps, detail_denoise, timing_sink=shot_timing)
latent_upscale_param=latent_upscale_param, timing_sink=shot_timing)
except (torch.cuda.OutOfMemoryError, RuntimeError) as e:
shot_retry_elapsed += time.perf_counter() - attempt_start
stage = getattr(e, "_h3_stage", "")
if _is_oom(e) and stage == "latent_upscale":
raise RuntimeError(
f"H3 Long Videos: shot {i + 1} of {len(gens)} ran out of VRAM "
f"during latent upscale. The latent-upscale stage is too large "
f"for this card at {w}x{h} with the current spatial settings."
) from e
if _is_oom(e) and getattr(e, "_h3_stage", "") == "sampling":
# Retrying with tiles would re-run the whole sampling pass and
# fail identically. Fail now, and say what actually shrinks it.
@@ -7072,8 +7611,7 @@ class H3LongVideos:
model, clip, vae, audio_vae, negative, gen_prompt, w, h, ln_i, fps,
tiled, sa, shot_handoff, decode_tile_frames, decode_tile_size,
shot_refs, ref_image_size, shot_aug, shot_silent,
detail_pass, detail_sampler_name, detail_scheduler,
detail_steps, detail_denoise, timing_sink=shot_timing)
latent_upscale_param=latent_upscale_param, timing_sink=shot_timing)
shot_retry_elapsed += time.perf_counter() - attempt_start
shot_total = time.perf_counter() - shot_total_start
@@ -7343,7 +7881,7 @@ class H3LongVideos:
f"shot before them ended on dialogue." if mouth_settled else "")
+ (f"{anatomy_note}." if anatomy_note else "")
+ (f"{latent_note}." if latent_note else "")
+ (f"{detail_note}." if detail_note else "")
+ (f"{latent_upscale_note}." if latent_upscale_note else "")
+ (f" SLA LoRA '{os.path.basename(str(sla_name))}' paired with sparse attention."
if sla_name and sparse_on else "")
+ (f" {beats_note}." if beats_note else "")
+4 -3
View File
@@ -39,10 +39,11 @@ class H3ShotLength:
def INPUT_TYPES(cls):
return {
"required": {
"shot_seconds": ("FLOAT", {"default": 5.0, "min": 0.2, "max": 15.1, "step": 0.5,
"shot_seconds": ("FLOAT", {"default": 3.0, "min": 0.2, "max": 15.1, "step": 0.5,
"tooltip": "Length of each shot. Feeds the sampler's shot_seconds AND (as frames) "
"the preview override. Max ~15s (362 frames)."}),
"fps": ("INT", {"default": 24, "min": 1, "max": 60}),
"the preview override. Default 3s matches the common one-beat H3 test shot. Max ~15s (362 frames)."}),
"fps": ("INT", {"default": 24, "min": 1, "max": 60,
"tooltip": "Frame rate used for the seconds->frames conversion. H3 itself renders at 24fps, so 24 is the realistic default."}),
},
"optional": {
"cap_to_h3_max": ("BOOLEAN", {"default": True,
+1012 -6
View File
File diff suppressed because it is too large Load Diff
+50 -29
View File
@@ -277,7 +277,14 @@ class DumasJSONStringToObjectNode:
def INPUT_TYPES(cls):
return {
"required": {
"json_string": ("STRING", {"multiline": True}),
"json_string": (
"STRING",
{
"multiline": True,
"default": '{\n "shots": [\n {\n "prompt": "Francine stands by the window."\n }\n ]\n}',
"tooltip": "Raw JSON text to parse into a structured JSON object."
},
),
}
}
@@ -299,7 +306,14 @@ class DumasStripIterationSuffixNode:
def INPUT_TYPES(cls):
return {
"required": {
"filename": ("STRING", {"default": "", "multiline": False}),
"filename": (
"STRING",
{
"default": "francine_pose_final.png",
"multiline": False,
"tooltip": "Filename to normalize by removing everything after the first underscore in the stem."
},
),
}
}
@@ -316,7 +330,14 @@ class DumasSlugifyStringNode:
def INPUT_TYPES(cls):
return {
"required": {
"text": ("STRING", {"default": "", "multiline": False}),
"text": (
"STRING",
{
"default": "Francine Coffee Shop",
"multiline": False,
"tooltip": "Text to slugify into lowercase ASCII words joined with hyphens."
},
),
}
}
@@ -334,8 +355,8 @@ class DumasJSONObjectToStringNode:
return {
"required": {
"json_object": ("JSON",),
"pretty": ("BOOLEAN", {"default": True}),
"sort_keys": ("BOOLEAN", {"default": False}),
"pretty": ("BOOLEAN", {"default": True, "tooltip": "Pretty-print the JSON with indentation."}),
"sort_keys": ("BOOLEAN", {"default": False, "tooltip": "Sort object keys alphabetically before serializing."}),
}
}
@@ -355,7 +376,7 @@ class DumasJSONGetValueNode:
return {
"required": {
"json_object": ("JSON",),
"path": ("STRING", {"default": "", "multiline": False}),
"path": ("STRING", {"default": "shots.0.prompt", "multiline": False, "tooltip": "Dot-path to read, such as 'shots.0.prompt'."}),
}
}
@@ -373,8 +394,8 @@ class DumasJSONSetValueNode:
return {
"required": {
"json_object": ("JSON",),
"path": ("STRING", {"default": "", "multiline": False}),
"value_json": ("STRING", {"multiline": True, "default": "null"}),
"path": ("STRING", {"default": "shots.0.prompt", "multiline": False, "tooltip": "Dot-path to write, such as 'shots.0.prompt' or 'shots.1.duration'."}),
"value_json": ("STRING", {"multiline": True, "default": '"Francine stands by the window."', "tooltip": "JSON value to store at the path. Must be valid JSON, so strings need quotes."}),
}
}
@@ -398,7 +419,7 @@ class DumasJSONHasKeyNode:
return {
"required": {
"json_object": ("JSON",),
"path": ("STRING", {"default": "", "multiline": False}),
"path": ("STRING", {"default": "shots.0.prompt", "multiline": False, "tooltip": "Dot-path to test for existence."}),
}
}
@@ -416,7 +437,7 @@ class DumasJSONRemoveKeyNode:
return {
"required": {
"json_object": ("JSON",),
"path": ("STRING", {"default": "", "multiline": False}),
"path": ("STRING", {"default": "shots.0.prompt", "multiline": False, "tooltip": "Dot-path to remove from the object."}),
}
}
@@ -434,7 +455,7 @@ class DumasJSONPickFieldsNode:
return {
"required": {
"json_object": ("JSON",),
"paths": ("STRING", {"multiline": True, "default": ""}),
"paths": ("STRING", {"multiline": True, "default": "shots.0.prompt\nshots.0.duration", "tooltip": "One dot-path per line. Only those fields are copied into the output object."}),
}
}
@@ -463,8 +484,8 @@ class DumasJSONMergeObjectsNode:
def INPUT_TYPES(cls):
return {
"required": {
"base_object": ("JSON",),
"overlay_object": ("JSON",),
"base_object": ("JSON", {"tooltip": "Base JSON object to start from."}),
"overlay_object": ("JSON", {"tooltip": "Overlay JSON object whose keys replace or merge into the base object."}),
}
}
@@ -482,7 +503,7 @@ class DumasJSONKeysNode:
def INPUT_TYPES(cls):
return {
"required": {
"json_object": ("JSON",),
"json_object": ("JSON", {"tooltip": "JSON object whose top-level keys should be listed."}),
}
}
@@ -502,7 +523,7 @@ class DumasJSONArrayLengthNode:
def INPUT_TYPES(cls):
return {
"required": {
"json_array": ("JSON",),
"json_array": ("JSON", {"tooltip": "JSON array whose length should be measured."}),
}
}
@@ -521,8 +542,8 @@ class DumasJSONArrayAppendNode:
def INPUT_TYPES(cls):
return {
"required": {
"json_array": ("JSON",),
"value_json": ("STRING", {"multiline": True, "default": "null"}),
"json_array": ("JSON", {"tooltip": "JSON array to append to."}),
"value_json": ("STRING", {"multiline": True, "default": '{"prompt":"Francine looks toward the door."}', "tooltip": "JSON value to append. Must be valid JSON."}),
}
}
@@ -548,10 +569,10 @@ class DumasJSONArraySliceNode:
def INPUT_TYPES(cls):
return {
"required": {
"json_array": ("JSON",),
"start": ("INT", {"default": 0, "step": 1}),
"end": ("INT", {"default": 0, "step": 1}),
"step": ("INT", {"default": 1, "step": 1, "min": 1}),
"json_array": ("JSON", {"tooltip": "JSON array to slice."}),
"start": ("INT", {"default": 0, "step": 1, "tooltip": "Zero-based start index."}),
"end": ("INT", {"default": 0, "step": 1, "tooltip": "Zero-based end index. Use 0 to mean 'to the end'."}),
"step": ("INT", {"default": 1, "step": 1, "min": 1, "tooltip": "Slice step size."}),
}
}
@@ -572,9 +593,9 @@ class DumasJSONArrayIteratorNode:
def INPUT_TYPES(cls):
return {
"required": {
"json_input": ("JSON",),
"index": ("INT", {"default": 0, "min": 0, "step": 1}),
"mode": (["fixed", "incr", "decr"], {"default": "fixed"}),
"json_input": ("JSON", {"tooltip": "JSON array to iterate over."}),
"index": ("INT", {"default": 0, "min": 0, "step": 1, "tooltip": "Current zero-based index."}),
"mode": (["fixed", "incr", "decr"], {"default": "fixed", "tooltip": "Keep the index fixed, increment it, or decrement it before reading."}),
}
}
@@ -600,9 +621,9 @@ class DumasJSONObjectIteratorNode:
def INPUT_TYPES(cls):
return {
"required": {
"json_input": ("JSON",),
"index": ("INT", {"default": 0, "min": 0, "step": 1}),
"mode": (["fixed", "incr", "decr"], {"default": "fixed"}),
"json_input": ("JSON", {"tooltip": "JSON object whose key/value pairs should be iterated in insertion order."}),
"index": ("INT", {"default": 0, "min": 0, "step": 1, "tooltip": "Current zero-based index into the object's items."}),
"mode": (["fixed", "incr", "decr"], {"default": "fixed", "tooltip": "Keep the index fixed, increment it, or decrement it before reading."}),
}
}
@@ -628,7 +649,7 @@ class DumasJSONFlattenNode:
def INPUT_TYPES(cls):
return {
"required": {
"json_input": ("JSON",),
"json_input": ("JSON", {"tooltip": "Nested JSON value to flatten into dot-path keys."}),
}
}
@@ -645,7 +666,7 @@ class DumasJSONUnflattenNode:
def INPUT_TYPES(cls):
return {
"required": {
"flat_json_object": ("JSON",),
"flat_json_object": ("JSON", {"tooltip": "Flat JSON object whose keys are dot-paths to rebuild into nested JSON."}),
}
}
+51 -7
View File
@@ -3,7 +3,7 @@ import { app } from "/scripts/app.js";
const NODE_NAME = "DumasAnchorStyle";
const STYLE_INPUT = "anchor_style";
const DESCRIPTION_INPUT = "style_description";
const H3_NOTE = " Keep this anchor focused on persistent camera language, lighting, texture, environment treatment, and tone; do not name characters or describe one-off actions.";
const H3_NOTE = "";
const PRESETS = {
"cinematic action movie": "Big-screen action cinema with assertive visual storytelling: dynamic camera placement, strong forward momentum, crisp geography, muscular lighting contrast, practical atmosphere, and a sense of physical consequence. Favor heroic framing, controlled handheld energy or motivated tracking moves, dramatic silhouettes, tasteful lens flares, impact-driven pacing, and polished studio spectacle without drifting into comic-book unreality unless the shot explicitly asks for it." + H3_NOTE,
"comedy": "Play the scene for comedic readability and timing: clear staging, expressive performances, slightly heightened reactions, clean eyelines, and visual beats that leave room for the joke to land. Use bright approachable lighting, grounded but playful production design, readable framing, and a tone that feels observant, awkward, or absurd without becoming broad parody unless the action supports it." + H3_NOTE,
@@ -51,28 +51,72 @@ const PRESETS = {
"fantasy adventure": "Rousing fantasy-adventure language: scenic scale, adventurous clarity, tactile costume-and-prop detail, and camera movement that feels exploratory rather than oppressive. Favor storybook geography, weathered materials, golden or stormy atmosphere, and a tone of peril, wonder, and forward motion." + H3_NOTE,
};
const SOUNDSCAPE_PRESETS = {
"quiet interior": "quiet indoor room tone, faint ventilation and distant household ambience",
"rainy street": "steady rain, wet pavement, distant traffic hum",
"cafe": "low room tone, faint glassware, cutlery, and muted conversation",
"city night": "distant traffic hum, occasional horn, night air",
"forest": "wind in leaves, distant birds, soft natural ambience",
"industrial": "large interior reverb, distant metal ticks, low machine hum",
"silent": "no dialogue, no vocals, only the natural ambient bed of the scene",
"custom": "",
};
const BGM_PRESETS = {
"none": "",
"subtle tension": "low, restrained tension bed with sparse pulses and no vocals",
"cinematic suspense": "cinematic suspense score with muted strings, low drones, and controlled rising pressure",
"emotional piano": "soft emotional piano underscoring with gentle space and no vocals",
"dark ambient": "dark ambient music bed with deep drones, distant texture, and slow unease",
"hopeful orchestral": "hopeful orchestral underscore with warm strings, gentle brass, and restrained lift",
"retro synth": "retro synth score with analog pulses, warm pads, and steady momentum",
"action pulse": "driving action pulse with percussion, rhythmic bass, and urgent forward motion",
"lo-fi": "soft lo-fi instrumental bed with mellow rhythm and warm tape texture",
"no vocals": "instrumental background music only, no singing, no lyrics, no vocal hooks",
"custom": "",
};
const NODE_CONFIGS = {
[NODE_NAME]: {
presetInput: STYLE_INPUT,
descriptionInput: DESCRIPTION_INPUT,
presets: PRESETS,
},
DumasSoundscapeHelper: {
presetInput: "soundscape",
descriptionInput: "soundscape_description",
presets: SOUNDSCAPE_PRESETS,
},
DumasBackgroundMusicHelper: {
presetInput: "bgm",
descriptionInput: "bgm_description",
presets: BGM_PRESETS,
},
};
function findWidget(node, name) {
return (node.widgets || []).find((widget) => widget?.name === name) || null;
}
app.registerExtension({
name: "Dumas.AnchorStyle",
name: "Dumas.PresetTextHelpers",
async beforeRegisterNodeDef(nodeType, nodeData) {
if (nodeData?.name !== NODE_NAME) return;
const config = NODE_CONFIGS[nodeData?.name];
if (!config) return;
const originalOnNodeCreated = nodeType.prototype.onNodeCreated;
nodeType.prototype.onNodeCreated = function onNodeCreated() {
const created = originalOnNodeCreated?.apply(this, arguments);
const styleWidget = findWidget(this, STYLE_INPUT);
const descriptionWidget = findWidget(this, DESCRIPTION_INPUT);
const styleWidget = findWidget(this, config.presetInput);
const descriptionWidget = findWidget(this, config.descriptionInput);
if (!styleWidget || !descriptionWidget) return created;
const originalCallback = styleWidget.callback;
styleWidget.callback = (...args) => {
const selected = String(styleWidget.value || "");
if (Object.hasOwn(PRESETS, selected)) {
descriptionWidget.value = PRESETS[selected];
if (Object.hasOwn(config.presets, selected)) {
descriptionWidget.value = config.presets[selected];
descriptionWidget.inputEl?.dispatchEvent(new Event("input", { bubbles: true }));
}
this.setDirtyCanvas?.(true, true);
+1 -2
View File
@@ -53,11 +53,10 @@ const GROUPS = [
},
{
id: "finish",
label: "Upscale/Detail",
label: "Upscale",
defaultCollapsed: true,
widgets: [
"upscale", "upscale_model", "upscale_target_short_edge", "upscale_batch",
"detail_pass", "detail_sampler_name", "detail_scheduler", "detail_steps", "detail_denoise",
],
},
{
+85
View File
@@ -0,0 +1,85 @@
import { app } from "/scripts/app.js";
const NODE_NAME = "DumasH3PromptCurator";
const EXPECTED_OUTPUTS = [
"prompt",
"ref_image_1",
"ref_image_2",
"ref_image_3",
"ref_image_4",
"ref_image_5",
"ref_image_6",
"ref_image_7",
"ref_image_8",
"ref_image_9",
"reference_count",
"debug",
"anchor",
"sounds",
"bgm",
"original_ref_1",
"original_ref_2",
"original_ref_3",
"original_ref_4",
"original_ref_5",
"original_ref_6",
"original_ref_7",
"original_ref_8",
"original_ref_9",
"compiled_ref_description_1",
"compiled_ref_description_2",
"compiled_ref_description_3",
"compiled_ref_description_4",
"compiled_ref_description_5",
"compiled_ref_description_6",
"compiled_ref_description_7",
"compiled_ref_description_8",
"compiled_ref_description_9",
];
const EXPECTED_NAMES = new Set(EXPECTED_OUTPUTS);
function pruneStaleOutputs(node) {
if (!Array.isArray(node.outputs)) return;
const byName = new Map();
for (const output of node.outputs) {
if (!output?.name || !EXPECTED_NAMES.has(output.name) || byName.has(output.name)) continue;
byName.set(output.name, output);
}
const nextOutputs = [];
for (const name of EXPECTED_OUTPUTS) {
const existing = byName.get(name);
if (existing) {
nextOutputs.push(existing);
}
}
if (nextOutputs.length && nextOutputs.length !== node.outputs.length) {
node.outputs = nextOutputs;
node.size = node.computeSize?.() || node.size;
node.setDirtyCanvas?.(true, true);
}
}
app.registerExtension({
name: "Dumas.H3PromptCuratorOutputs",
async beforeRegisterNodeDef(nodeType, nodeData) {
if (nodeData?.name !== NODE_NAME) return;
const originalOnNodeCreated = nodeType.prototype.onNodeCreated;
const originalOnConfigure = nodeType.prototype.onConfigure;
nodeType.prototype.onNodeCreated = function onNodeCreated() {
const created = originalOnNodeCreated?.apply(this, arguments);
pruneStaleOutputs(this);
return created;
};
nodeType.prototype.onConfigure = function onConfigure() {
const configured = originalOnConfigure?.apply(this, arguments);
pruneStaleOutputs(this);
return configured;
};
},
});
+490 -25
View File
@@ -24,6 +24,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
"PIL.Image",
"folder_paths",
"dumas_image_nodes",
"dumas_h3_latent_upscale",
"dumas_h3_longvideos",
)
}
@@ -207,7 +208,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
"retry_elapsed": 1.2,
"attempts": 2,
"sample": 8.0,
"detail_sample": 0.5,
"latent_upscale_sample": 0.5,
"decode_video": 2.1,
"decode_audio": 0.4,
"cleanup": 0.2,
@@ -230,11 +231,11 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.assertIn("decode audio 0.7s", note)
self.assertIn("cleanup 0.3s", note)
self.assertIn("retry elapsed 1.2s", note)
self.assertIn("detail 0.5s", note)
self.assertIn("latent upscale 0.5s", note)
self.assertIn("retries 1", note)
self.assertIn("slowest shot 1 12.4s", note)
def test_detail_pass_refines_video_but_preserves_audio(self):
def test_latent_upscale_refines_video_but_preserves_audio(self):
class FakeTensor:
def __init__(self, name):
self.name = name
@@ -263,6 +264,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
original_decode_video = self.module._decode_video
original_decode_audio = self.module._decode_audio
original_cleanup = self.module._deep_cleanup
original_upscale = self.module._upscale_latent_video
original_copy_sample = self.module._copy_sample_latent
original_nested = getattr(self.module.comfy.nested_tensor, "NestedTensor", None)
try:
self.module.comfy.nested_tensor.NestedTensor = FakeNestedTensor
@@ -277,6 +280,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
{"samples": FakeNestedTensor((FakeTensor("basev"), FakeTensor("basea")))},
)
self.module._evict_all_but = lambda *_args, **_kwargs: None
self.module._upscale_latent_video = lambda video, param: (FakeTensor("upv"), 8, 16)
self.module._copy_sample_latent = lambda sampled: sampled["samples"].unbind()
self.module._decode_video = lambda _vae, out_latent, *_args, **_kwargs: out_latent
self.module._decode_audio = lambda _vae, out_latent: out_latent
self.module._deep_cleanup = lambda: None
@@ -298,18 +303,25 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
tiled=False,
sa=(123, 20, 1.0, "res_multistep", "simple", 1.0),
handoff=None,
detail_pass=True,
detail_sampler_name="euler",
detail_scheduler="beta",
detail_steps=5,
detail_denoise=0.4,
latent_upscale_param={
"mode": "model",
"model_name": "upscale.safetensors",
"device": "cpu",
"precision": "fp16",
"sampler_name": "euler_ancestral",
"scheduler": "simple",
"steps": 2,
"denoise": 0.4,
"megapixels": 1.5,
},
)
self.assertEqual(len(calls), 2)
self.assertIsNot(calls[1][0][8], first_out)
self.assertIs(calls[1][0][8]["samples"], first_out["samples"])
self.assertEqual(calls[1][0][4], "euler")
self.assertEqual(calls[1][0][5], "beta")
self.assertEqual(calls[1][0][8]["samples"].unbind()[0].name, "upv")
self.assertEqual(calls[1][0][2], 2)
self.assertEqual(calls[1][0][4], "euler_ancestral")
self.assertEqual(calls[1][0][5], "simple")
self.assertAlmostEqual(calls[1][1]["denoise"], 0.4)
self.assertEqual(result[1], first_out)
self.assertEqual(result[2][0].name, "v2")
@@ -323,12 +335,14 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.module._decode_video = original_decode_video
self.module._decode_audio = original_decode_audio
self.module._deep_cleanup = original_cleanup
self.module._upscale_latent_video = original_upscale
self.module._copy_sample_latent = original_copy_sample
if original_nested is None:
delattr(self.module.comfy.nested_tensor, "NestedTensor")
else:
self.module.comfy.nested_tensor.NestedTensor = original_nested
def test_detail_pass_decodes_audio_before_video_and_cleans_up(self):
def test_latent_upscale_decodes_audio_before_video_and_cleans_up(self):
class FakeTensor:
def __init__(self, name):
self.name = name
@@ -348,6 +362,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
return self._parts
order = []
build_calls = []
first_out = {"samples": FakeNestedTensor((FakeTensor("v1"), FakeTensor("a1")))}
second_out = {"samples": FakeNestedTensor((FakeTensor("v2"), FakeTensor("a2")))}
@@ -357,12 +372,16 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
original_decode_video = self.module._decode_video
original_decode_audio = self.module._decode_audio
original_cleanup = self.module._deep_cleanup
original_upscale = self.module._upscale_latent_video
original_copy_sample = self.module._copy_sample_latent
original_unload = getattr(self.module.mm, "unload_model_and_clones", None)
original_unload_all = self.module.mm.unload_all_models
original_nested = getattr(self.module.comfy.nested_tensor, "NestedTensor", None)
try:
self.module.comfy.nested_tensor.NestedTensor = FakeNestedTensor
def common_ksampler(*args, **kwargs):
order.append("detail_sample" if len(order) else "sample")
order.append("latent_upscale_sample" if len(order) else "sample")
return (first_out if len([x for x in order if x.endswith("sample")]) == 1 else second_out,)
def decode_audio(_vae, out_latent):
@@ -379,12 +398,31 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
def cleanup():
order.append("cleanup")
self.module.nodes.common_ksampler = common_ksampler
self.module._build_shot_conditioning = lambda *_args, **_kwargs: (
"cond",
def unload_model_and_clones(*_args, **_kwargs):
order.append("unload_h3_failed")
raise RuntimeError("model wrapper does not expose clone metadata")
def unload_all_models(*_args, **_kwargs):
order.append("unload_all")
def upscale_latent_video(video, param):
order.append("upscale")
return FakeTensor("upv"), 8, 16
def build_conditioning(*_args, **_kwargs):
build_calls.append(True)
return (
[["cond", {}]],
{"samples": FakeNestedTensor((FakeTensor("basev"), FakeTensor("basea")))},
)
self.module.nodes.common_ksampler = common_ksampler
self.module._build_shot_conditioning = build_conditioning
self.module._evict_all_but = lambda *_args, **_kwargs: None
self.module.mm.unload_model_and_clones = unload_model_and_clones
self.module.mm.unload_all_models = unload_all_models
self.module._upscale_latent_video = upscale_latent_video
self.module._copy_sample_latent = lambda sampled: sampled["samples"].unbind()
self.module._decode_video = decode_video
self.module._decode_audio = decode_audio
self.module._deep_cleanup = cleanup
@@ -406,17 +444,25 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
tiled=False,
sa=(123, 20, 1.0, "res_multistep", "simple", 1.0),
handoff=None,
detail_pass=True,
detail_sampler_name="euler",
detail_scheduler="beta",
detail_steps=5,
detail_denoise=0.4,
latent_upscale_param={
"mode": "model",
"model_name": "upscale.safetensors",
"device": "cuda",
"precision": "fp16",
"sampler_name": "euler_ancestral",
"scheduler": "simple",
"steps": 2,
"denoise": 0.4,
"megapixels": 1.5,
},
)
self.assertEqual(order[0], "sample")
self.assertEqual(order[1], "detail_sample")
self.assertLess(order.index("unload_all"), order.index("upscale"))
self.assertLess(order.index("upscale"), order.index("latent_upscale_sample"))
self.assertLess(order.index("audio"), order.index("video"))
self.assertEqual(order[-1], "cleanup")
self.assertEqual(len(build_calls), 1)
finally:
self.module.nodes.common_ksampler = original_common_ksampler
self.module._build_shot_conditioning = original_build
@@ -424,12 +470,25 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.module._decode_video = original_decode_video
self.module._decode_audio = original_decode_audio
self.module._deep_cleanup = original_cleanup
self.module._upscale_latent_video = original_upscale
self.module._copy_sample_latent = original_copy_sample
if original_unload is None:
delattr(self.module.mm, "unload_model_and_clones")
else:
self.module.mm.unload_model_and_clones = original_unload
self.module.mm.unload_all_models = original_unload_all
if original_nested is None:
delattr(self.module.comfy.nested_tensor, "NestedTensor")
else:
self.module.comfy.nested_tensor.NestedTensor = original_nested
def test_detail_pass_treats_falsey_strings_as_disabled(self):
def test_latent_refine_tiles_do_not_rebuild_conditioning(self):
source = inspect.getsource(self.module.H3LongVideos._render)
tile_branch = source[source.index("for col_index, c0 in enumerate(cols):"):]
self.assertIn("_crop_conditioning_to_tile", tile_branch)
self.assertNotIn("_build_shot_conditioning(", tile_branch)
def test_latent_upscale_off_skips_second_pass(self):
calls = []
original_common_ksampler = self.module.nodes.common_ksampler
original_build = self.module._build_shot_conditioning
@@ -437,6 +496,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
original_decode_video = self.module._decode_video
original_decode_audio = self.module._decode_audio
original_cleanup = self.module._deep_cleanup
original_upscale = self.module._upscale_latent_video
original_copy_sample = self.module._copy_sample_latent
try:
self.module.nodes.common_ksampler = lambda *args, **kwargs: (calls.append((args, kwargs)) or {"samples": "latent"},)
self.module._build_shot_conditioning = lambda *_args, **_kwargs: ("cond", {"samples": "base"})
@@ -444,6 +505,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.module._decode_video = lambda _vae, out_latent, *_args, **_kwargs: out_latent
self.module._decode_audio = lambda _vae, out_latent: out_latent
self.module._deep_cleanup = lambda: None
self.module._upscale_latent_video = lambda *_args, **_kwargs: (_ for _ in ()).throw(RuntimeError("should not run"))
self.module._copy_sample_latent = lambda sampled: sampled
self.module.H3LongVideos()._render(
model=object(),
@@ -459,7 +522,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
tiled=False,
sa=(123, 20, 1.0, "res_multistep", "simple", 1.0),
handoff=None,
detail_pass="false",
latent_upscale_param={"mode": "off"},
)
self.assertEqual(len(calls), 1)
@@ -470,6 +533,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.module._decode_video = original_decode_video
self.module._decode_audio = original_decode_audio
self.module._deep_cleanup = original_cleanup
self.module._upscale_latent_video = original_upscale
self.module._copy_sample_latent = original_copy_sample
def test_distribute_generations_canonicalizes_per_shot_audio_and_anchor_directives(self):
generations = self.module.distribute_generations(
@@ -736,6 +801,12 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.assertNotIn(f"ref_image_{index}", optional)
self.assertNotIn("per_beat_length", optional)
self.assertNotIn("cleanup_between_shots", optional)
self.assertNotIn("detail_pass", optional)
self.assertNotIn("detail_sampler_name", optional)
self.assertNotIn("detail_scheduler", optional)
self.assertNotIn("detail_steps", optional)
self.assertNotIn("detail_denoise", optional)
self.assertIn("latent_upscale_param", optional)
def test_shot_seconds_tooltip_describes_ceiling_behavior(self):
optional = self.module.H3LongVideos.INPUT_TYPES()["optional"]
@@ -743,7 +814,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
self.assertIn("GLOBAL per-shot maximum", tooltip)
self.assertIn("A beat's own `seconds:` directive can still ask for less", tooltip)
self.assertIn("honoring it; may spill to system RAM (slow) or OOM", tooltip)
self.assertIn("let the render fail instead of shrinking it", tooltip)
def test_resolve_shot_frames_honors_forced_request_over_budget(self):
original_estimate_shot_frames = self.module.estimate_shot_frames
@@ -928,12 +999,23 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
plan_only=True,
ref_1={"image": "live-1"},
ref_3={"image": "live-3"},
latent_upscale_param={
"mode": "interp",
"method": "bilinear",
"sampler_name": "euler_ancestral",
"scheduler": "simple",
"steps": 2,
"denoise": 0.2,
"megapixels": 1.5,
},
)
self.assertEqual(calls["refs"][0]["image"], "live-1")
self.assertIsNone(calls["refs"][1])
self.assertEqual(calls["refs"][2]["image"], "live-3")
self.assertEqual(result[2].count("ref2va: 2 reference image(s)"), 1)
self.assertIn("latent upscale:", result[2])
self.assertIn("euler_ancestral/simple", result[2])
finally:
self.module.parse_resolution = original_parse_resolution
self.module._connected_refs = original_connected_refs
@@ -1061,6 +1143,389 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
{"DumasH3LongVideos": "Dumas H3 Long Videos (FL2VA + REF2VA)"},
)
def test_latent_upscale_params_node_is_exposed(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
required = latent.H3LatentUpscaleParams.INPUT_TYPES()["required"]
self.assertEqual(
latent.NODE_CLASS_MAPPINGS,
{"DumasH3LatentUpscaleParams": latent.H3LatentUpscaleParams},
)
self.assertEqual(
latent.NODE_DISPLAY_NAME_MAPPINGS,
{"DumasH3LatentUpscaleParams": "Dumas H3 Latent Upscale Params"},
)
self.assertEqual(required["sampler_name"][1]["default"], "euler_ancestral")
self.assertEqual(required["scheduler"][1]["default"], "simple")
self.assertEqual(required["steps"][1]["default"], 2)
self.assertEqual(required["denoise"][1]["default"], 0.2)
self.assertEqual(required["megapixels"][1]["default"], 1.0)
self.assertEqual(required["tile_width"][1]["default"], 512)
self.assertEqual(required["tile_height"][1]["default"], 512)
self.assertEqual(required["overlap"][1]["default"], 64)
self.assertEqual(required["fade_width"][1]["default"], 32)
self.assertEqual(required["fade_height"][1]["default"], 32)
self.assertEqual(required["overlap_mode"][1]["default"], "earlier")
self.assertEqual(required["overlap_blend"][1]["default"], "linear")
self.assertEqual(required["tile_size_mode"][1]["default"], "specific_size")
self.assertEqual(required["grid_rows"][1]["default"], 2)
self.assertEqual(required["grid_cols"][1]["default"], 2)
self.assertEqual(required["spatial_w_overlap"][1]["default"], 128)
self.assertEqual(required["spatial_h_overlap"][1]["default"], 128)
self.assertEqual(required["min_tile_size"][1]["default"], 256)
self.assertEqual(required["masked_area_noise"][1]["default"], 0.0)
self.assertFalse(required["brightness_match"][1]["default"])
self.assertEqual(required["dynamic_fade"][1]["default"], "off")
self.assertEqual(required["dynamic_fade_min"][1]["default"], 32)
self.assertEqual(required["chunk_length"][1]["default"], 85)
self.assertEqual(required["temporal_overlap"][1]["default"], 17)
self.assertFalse(required["resize_conditioning"][1]["default"])
self.assertEqual(required["anchor_strength"][1]["default"], 0.999)
def test_latent_upscale_mode_infers_legacy_model_payloads(self):
self.assertEqual(self.module._latent_upscale_mode({"model_name": "foo.safetensors"}), "model")
self.assertEqual(self.module._latent_upscale_mode({"method": "bilinear"}), "interp")
self.assertEqual(self.module._latent_upscale_mode({"mode": "model"}), "model")
self.assertEqual(self.module._latent_upscale_mode({}), "off")
def test_tag_oom_stage_marks_oom_exceptions(self):
exc = RuntimeError("CUDA out of memory")
tagged = self.module._tag_oom_stage(exc, "latent_upscale")
self.assertIs(tagged, exc)
self.assertEqual(getattr(tagged, "_h3_stage", ""), "latent_upscale")
def test_shrink_model_tile_param_reduces_tile_size(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
smaller = latent._shrink_model_tile_param({
"tile_size_mode": "specific_size",
"tile_width": 512,
"tile_height": 512,
"overlap": 64,
"fade_width": 32,
"fade_height": 32,
})
self.assertIsNotNone(smaller)
self.assertEqual(smaller["tile_size_mode"], "rows_cols")
self.assertEqual(smaller["grid_rows"], 4)
self.assertEqual(smaller["grid_cols"], 4)
self.assertEqual(smaller["spatial_w_overlap"], 0)
self.assertEqual(smaller["spatial_h_overlap"], 0)
self.assertEqual(smaller["fade_width"], 0)
self.assertEqual(smaller["fade_height"], 0)
self.assertEqual(smaller["min_tile_size"], 32)
def test_shrink_model_tile_param_rows_cols_resets_overlap(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
smaller = latent._shrink_model_tile_param({
"tile_size_mode": "rows_cols",
"grid_rows": 4,
"grid_cols": 4,
"spatial_w_overlap": 128,
"spatial_h_overlap": 128,
"fade_width": 64,
"fade_height": 64,
"min_tile_size": 256,
})
self.assertIsNotNone(smaller)
self.assertEqual(smaller["grid_rows"], 8)
self.assertEqual(smaller["grid_cols"], 8)
self.assertEqual(smaller["spatial_w_overlap"], 0)
self.assertEqual(smaller["spatial_h_overlap"], 0)
self.assertEqual(smaller["fade_width"], 0)
self.assertEqual(smaller["fade_height"], 0)
self.assertEqual(smaller["min_tile_size"], 32)
def test_shrink_model_tile_param_rows_cols_can_reach_thirty_two(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
smaller = latent._shrink_model_tile_param({
"tile_size_mode": "rows_cols",
"grid_rows": 16,
"grid_cols": 16,
"spatial_w_overlap": 0,
"spatial_h_overlap": 0,
"fade_width": 0,
"fade_height": 0,
"min_tile_size": 32,
})
self.assertIsNotNone(smaller)
self.assertEqual(smaller["grid_rows"], 32)
self.assertEqual(smaller["grid_cols"], 32)
def test_temporal_segments_split_long_sequences(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
bounds = latent._temporal_segments(36, 85, 17)
self.assertGreater(len(bounds), 1)
self.assertEqual(bounds[0][0], 0)
self.assertEqual(bounds[-1][2], 36)
def test_shrink_temporal_param_reduces_chunk_length(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
smaller = latent._shrink_temporal_param({
"chunk_length": 85,
"temporal_overlap": 17,
})
self.assertIsNotNone(smaller)
self.assertEqual(smaller["chunk_length"], 17)
self.assertEqual(smaller["temporal_overlap"], 0)
def test_cuda_model_temporal_params_keep_splitting_saved_workflows(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
chunk_length, temporal_overlap = latent._effective_temporal_params({
"mode": "model",
"device": "cuda",
"chunk_length": 85,
"temporal_overlap": 17,
}, frame_count=124)
self.assertEqual(chunk_length, 85)
self.assertEqual(temporal_overlap, 17)
def test_cuda_model_temporal_params_keep_short_saved_workflows_until_oom(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
chunk_length, temporal_overlap = latent._effective_temporal_params({
"mode": "model",
"device": "cuda",
"chunk_length": 85,
"temporal_overlap": 17,
}, frame_count=85)
self.assertEqual(chunk_length, 85)
self.assertEqual(temporal_overlap, 17)
def test_cuda_model_oom_retries_temporal_before_spatial_fallback(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
calls = []
original_tiled = latent._upscale_video_model_tiled
original_shrink_model = latent._shrink_model_tile_param
try:
def tiled(_video, param):
calls.append(("tiled", param.get("chunk_length"), param.get("tile_size_mode")))
raise RuntimeError("out of memory")
latent._upscale_video_model_tiled = tiled
latent._shrink_model_tile_param = (
lambda param: calls.append(("shrink_spatial", param.get("tile_size_mode"))) or None
)
with self.assertRaisesRegex(RuntimeError, "smaller temporal chunk"):
latent.upscale_video_model(
"video",
{
"mode": "model",
"device": "cuda",
"model_name": "upscale.safetensors",
"chunk_length": 85,
"temporal_overlap": 17,
},
)
self.assertEqual(calls[0], ("tiled", 85, None))
self.assertNotIn(("shrink_spatial", None), calls)
finally:
latent._upscale_video_model_tiled = original_tiled
latent._shrink_model_tile_param = original_shrink_model
def test_interp_temporal_params_preserve_upstream_defaults(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
chunk_length, temporal_overlap = latent._effective_temporal_params({
"mode": "interp",
"device": "cuda",
"chunk_length": 85,
"temporal_overlap": 17,
})
self.assertEqual(chunk_length, 85)
self.assertEqual(temporal_overlap, 17)
def test_unload_upscale_model_defers_while_held(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
class FakeParam:
device = "cuda"
class FakeModel:
def __init__(self):
self.moves = []
def parameters(self):
return iter((FakeParam(),))
def to(self, device):
self.moves.append(device)
return self
cache_key = "upscale.safetensors::cuda::fp16"
original_cache_value = latent._MODEL_CACHE.get(cache_key)
original_hold_depth = latent._MODEL_HOLD_DEPTH
fake_model = FakeModel()
try:
latent._MODEL_CACHE[cache_key] = fake_model
latent._MODEL_HOLD_DEPTH = 0
with latent._hold_upscale_model_loaded():
latent.unload_upscale_model("upscale.safetensors", "cuda", "fp16")
self.assertEqual(fake_model.moves, [])
latent.unload_upscale_model("upscale.safetensors", "cuda", "fp16")
self.assertEqual(fake_model.moves, ["cpu"])
finally:
latent._MODEL_HOLD_DEPTH = original_hold_depth
if original_cache_value is None:
latent._MODEL_CACHE.pop(cache_key, None)
else:
latent._MODEL_CACHE[cache_key] = original_cache_value
def test_model_upscale_releases_cached_model_after_pass(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
calls = []
original_temporal = latent._upscale_video_temporal_chunks
original_unload_now = latent._unload_upscale_model_now
original_cuda = latent.torch.cuda
original_device = getattr(latent.torch, "device", None)
try:
latent.torch.cuda = types.SimpleNamespace(is_available=lambda: True)
latent.torch.device = lambda value: value
def temporal(video, param, upscaler):
calls.append(("temporal", latent._MODEL_HOLD_DEPTH))
return "video", 8, 16
def unload_now(name, device, precision):
calls.append(("unload", name, device, precision, latent._MODEL_HOLD_DEPTH))
latent._upscale_video_temporal_chunks = temporal
latent._unload_upscale_model_now = unload_now
result = latent.upscale_latent_video("source", {
"mode": "model",
"model_name": "upscale.safetensors",
"device": "cuda",
"precision": "fp16",
})
self.assertEqual(result, ("video", 8, 16))
self.assertEqual(calls[0], ("temporal", 1))
self.assertEqual(calls[1], ("unload", "upscale.safetensors", "cuda", "fp16", 1))
self.assertEqual(latent._MODEL_HOLD_DEPTH, 0)
finally:
latent._upscale_video_temporal_chunks = original_temporal
latent._unload_upscale_model_now = original_unload_now
latent.torch.cuda = original_cuda
if original_device is None:
delattr(latent.torch, "device")
else:
latent.torch.device = original_device
def test_model_upscale_oom_falls_back_to_interp(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
calls = []
original_temporal = latent._upscale_video_temporal_chunks
original_interp = latent.upscale_video_interp
original_unload_now = latent._unload_upscale_model_now
original_cuda = latent.torch.cuda
original_device = getattr(latent.torch, "device", None)
try:
latent.torch.cuda = types.SimpleNamespace(
is_available=lambda: True,
empty_cache=lambda: calls.append(("empty_cache",)),
)
latent.torch.device = lambda value: value
def temporal(_video, _param, _upscaler):
calls.append(("temporal", latent._MODEL_HOLD_DEPTH))
raise RuntimeError("H3 latent upscale exhausted its GPU spatial fallbacks")
def interp(video, param):
calls.append(("interp", video, param.get("mode"), param.get("method")))
return "interp_video", 8, 16
def unload_now(name, device, precision):
calls.append(("unload", name, device, precision, latent._MODEL_HOLD_DEPTH))
latent._upscale_video_temporal_chunks = temporal
latent.upscale_video_interp = interp
latent._unload_upscale_model_now = unload_now
result = latent.upscale_latent_video("source", {
"mode": "model",
"model_name": "upscale.safetensors",
"method": "bilinear",
"device": "cuda",
"precision": "fp16",
})
self.assertEqual(result, ("interp_video", 8, 16))
self.assertEqual(calls[0], ("temporal", 1))
self.assertEqual(calls[1], ("unload", "upscale.safetensors", "cuda", "fp16", 1))
self.assertIn(("empty_cache",), calls)
self.assertEqual(calls[-1], ("interp", "source", "interp", "bilinear"))
self.assertEqual(latent._MODEL_HOLD_DEPTH, 0)
finally:
latent._upscale_video_temporal_chunks = original_temporal
latent.upscale_video_interp = original_interp
latent._unload_upscale_model_now = original_unload_now
latent.torch.cuda = original_cuda
if original_device is None:
delattr(latent.torch, "device")
else:
latent.torch.device = original_device
def test_model_upscale_skips_learned_model_on_8gb_cuda(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
calls = []
original_temporal = latent._upscale_video_temporal_chunks
original_interp = latent.upscale_video_interp
original_cuda = latent.torch.cuda
try:
latent.torch.cuda = types.SimpleNamespace(
is_available=lambda: True,
mem_get_info=lambda: (1 * 1024 * 1024 * 1024, 8 * 1024 * 1024 * 1024),
empty_cache=lambda: calls.append(("empty_cache",)),
)
def temporal(_video, _param, _upscaler):
calls.append(("temporal",))
raise AssertionError("learned model path should be skipped on 8GB CUDA")
def interp(video, param):
calls.append(("interp", video, param.get("mode"), param.get("method")))
return "interp_video", 8, 16
latent._upscale_video_temporal_chunks = temporal
latent.upscale_video_interp = interp
result = latent.upscale_latent_video("source", {
"mode": "model",
"model_name": "upscale.safetensors",
"method": "bilinear",
"device": "cuda",
"precision": "fp16",
})
self.assertEqual(result, ("interp_video", 8, 16))
self.assertNotIn(("temporal",), calls)
self.assertIn(("empty_cache",), calls)
self.assertEqual(calls[-1], ("interp", "source", "interp", "bilinear"))
finally:
latent._upscale_video_temporal_chunks = original_temporal
latent.upscale_video_interp = original_interp
latent.torch.cuda = original_cuda
def test_upscale_video_model_raises_when_gpu_cannot_shrink(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
original_tiled = latent._upscale_video_model_tiled
original_shrink = latent._shrink_model_tile_param
try:
latent._shrink_model_tile_param = lambda _param: None
latent._upscale_video_model_tiled = lambda *_args, **_kwargs: (_ for _ in ()).throw(RuntimeError("out of memory"))
with self.assertRaisesRegex(RuntimeError, "H3 latent upscale exhausted its GPU spatial fallbacks"):
latent.upscale_video_model("video", {"device": "cuda", "precision": "fp16"})
finally:
latent._upscale_video_model_tiled = original_tiled
latent._shrink_model_tile_param = original_shrink
def test_compose_persistent_does_not_expand_ambiguous_plural_to_full_cast(self):
active = self.module.parse_wardrobe(
"Maya = she, red jacket\n"
+313
View File
@@ -356,6 +356,306 @@ class DumasImageNodeTests(unittest.TestCase):
required = self.image_nodes.DumasLocationReferenceNode.INPUT_TYPES()["required"]
self.assertNotIn("picture_id", required)
def test_character_helper_restores_image_and_text_outputs(self):
node = self.image_nodes.DumasCharacterHelperNode()
image1 = FakeTensorBatch()
image2 = FakeTensorBatch()
result = node.build_character_text(
image1=image1,
image2=image2,
image1_picture_id="1",
image2_picture_id="2",
character_id="char_dave",
name="Dave",
alias="The Locksmith",
gender="male",
age="41",
nationality="English",
occupation="a detective",
height_feet="6",
height_inches="2",
accent="English",
general="Moves carefully and notices every exit",
wardrobe="weathered red flight jacket, grey cargo shorts, black boots",
)
self.assertIs(result[0], image1)
self.assertIs(result[1], image2)
self.assertIn("<Picture 1> and <Picture 2> reference the same character", result[2])
self.assertIn("Dave is also known as The Locksmith", result[2])
self.assertIn("is 41 years old", result[2])
self.assertEqual(result[3], "Dave = weathered red flight jacket, grey cargo shorts, black boots")
self.assertIs(result[4]["image"], image1)
self.assertIs(result[5]["image"], image2)
self.assertEqual(result[4]["id"], "char-dave")
self.assertEqual(result[5]["id"], "char-dave")
self.assertEqual(result[4]["name"], "Dave")
self.assertEqual(result[4]["aliases"], ["The Locksmith"])
self.assertEqual(result[4]["facts"]["age"], "41")
self.assertEqual(result[4]["facts"]["height_feet"], "6")
self.assertEqual(result[4]["facts"]["height_inches"], "2")
self.assertEqual(result[4]["wardrobe"], "weathered red flight jacket, grey cargo shorts, black boots")
self.assertEqual(len(result), 6)
def test_location_helper_matches_character_helper_shape_without_wardrobe(self):
node = self.image_nodes.DumasLocationHelperNode()
image1 = FakeTensorBatch()
image2 = FakeTensorBatch()
result = node.build_location_text(
image1=image1,
image2=image2,
image1_picture_id="3",
image2_picture_id="4",
location_id="coffee-shop-01",
name="Coffee Shop",
alias="Cafe Interior",
description="Warm tungsten lighting, narrow counter, rainy front window",
general="Evening ambience, cramped but cozy",
)
self.assertIs(result[0], image1)
self.assertIs(result[1], image2)
self.assertIn("<Picture 3> and <Picture 4> reference the same location", result[2])
self.assertIn("Coffee Shop is also known as Cafe Interior", result[2])
self.assertIn("Warm tungsten lighting, narrow counter, rainy front window.", result[2])
self.assertIn("Evening ambience, cramped but cozy.", result[2])
self.assertIs(result[3]["image"], image1)
self.assertIs(result[4]["image"], image2)
self.assertEqual(result[3]["kind"], "location")
self.assertEqual(result[4]["kind"], "location")
self.assertEqual(result[3]["id"], "coffee-shop-01")
self.assertEqual(result[4]["id"], "coffee-shop-01")
self.assertEqual(result[3]["name"], "Coffee Shop")
self.assertEqual(result[3]["aliases"], ["Cafe Interior"])
self.assertEqual(result[3]["description"], "Warm tungsten lighting, narrow counter, rainy front window")
self.assertEqual(result[3]["general"], "Evening ambience, cramped but cozy")
self.assertEqual(len(result), 5)
def test_soundscape_helper_defaults_to_selected_preset_description(self):
node = self.image_nodes.DumasSoundscapeHelperNode()
result = node.build_soundscape("rainy street", "")
self.assertEqual(result[0], "steady rain, wet pavement, distant traffic hum")
def test_background_music_helper_defaults_to_selected_preset_description(self):
node = self.image_nodes.DumasBackgroundMusicHelperNode()
result = node.build_bgm("subtle tension", "")
self.assertEqual(result[0], "low, restrained tension bed with sparse pulses and no vocals")
def test_h3_prompt_curator_compacts_named_references(self):
node = self.image_nodes.DumasH3PromptCuratorNode()
dave_image = FakeTensorBatch()
cafe_image = FakeTensorBatch()
van_image = FakeTensorBatch()
dave = self.image_nodes.make_reference(
kind="character",
image=dave_image,
name="Dave",
aliases="The Locksmith",
description="tired eyes, cropped brown hair",
wardrobe="red flight jacket",
)
cafe = self.image_nodes.make_reference(
kind="location",
image=cafe_image,
name="Coffee Shop",
description="warm tungsten lighting and rainy windows",
)
van = self.image_nodes.make_reference(
kind="location",
image=van_image,
name="Blue Van",
description="scuffed blue delivery van",
)
result = node.curate_prompt(
action_prompt="Dave runs from the Coffee Shop into the rain.",
anatomy_guard="auto",
subject_count_guard="auto",
anchor="grounded handheld thriller",
soundscape="steady rain",
bgm="low suspense music",
ref_1=dave,
ref_2=van,
ref_3=cafe,
)
prompt = result[0]
self.assertIn("<Picture 1> Dave", prompt)
self.assertIn("<Picture 2> Coffee Shop", prompt)
self.assertIn("Action: Dave runs from the Coffee Shop into the rain.", prompt)
self.assertIn("Anatomy guard:", prompt)
self.assertIn("Subject count guard:", prompt)
self.assertIn("overall_soundscape: steady rain", prompt)
self.assertIn("background_music: low suspense music", prompt)
self.assertIn("exactly one named character: <Picture 1> Dave", prompt)
self.assertIs(result[1], dave_image)
self.assertIs(result[2], cafe_image)
self.assertIsNone(result[3])
self.assertEqual(result[10], 2)
self.assertIn("input 3-><Picture 2> Coffee Shop", result[11])
self.assertEqual(result[12], "grounded handheld thriller")
self.assertEqual(result[13], "steady rain")
self.assertEqual(result[14], "low suspense music")
self.assertIs(result[15], dave_image)
self.assertIs(result[16], cafe_image)
self.assertIsNone(result[17])
self.assertIn("<Picture 1> Dave", result[24])
self.assertNotIn("<Picture 2> Coffee Shop", result[24])
self.assertIn("<Picture 2> Coffee Shop", result[25])
self.assertIn("Location context for <Picture 2> Coffee Shop", result[25])
self.assertEqual(result[26], "")
def test_h3_prompt_curator_renumbers_explicit_reference_tags(self):
node = self.image_nodes.DumasH3PromptCuratorNode()
image1 = FakeTensorBatch()
image3 = FakeTensorBatch()
unused = FakeTensorBatch()
first = self.image_nodes.make_reference(kind="character", image=image1, name="Maya")
second = self.image_nodes.make_reference(kind="location", image=unused, name="Lobby")
third = self.image_nodes.make_reference(kind="location", image=image3, name="Rooftop")
result = node.curate_prompt(
action_prompt="<Picture 1> Maya crosses to <ref3> as the wind rises.",
anatomy_guard="off",
subject_count_guard="off",
ref_1=first,
ref_2=second,
ref_3=third,
)
prompt = result[0]
self.assertIn("<Picture 1> Maya crosses to <Picture 2>", prompt)
self.assertNotIn("<Picture 3>", prompt)
self.assertIs(result[1], image1)
self.assertIs(result[2], image3)
self.assertIsNone(result[3])
self.assertEqual(result[10], 2)
def test_h3_prompt_curator_can_force_subject_count_without_character_refs(self):
node = self.image_nodes.DumasH3PromptCuratorNode()
result = node.curate_prompt(
action_prompt="A locked-off shot of the empty corridor.",
anatomy_guard="off",
subject_count_guard="on",
)
self.assertIn("Subject count guard:", result[0])
self.assertIn("Only include the people explicitly described", result[0])
self.assertEqual(result[10], 0)
def test_h3_prompt_curator_treats_helper_image_pair_as_one_character(self):
helper = self.image_nodes.DumasCharacterHelperNode()
curator = self.image_nodes.DumasH3PromptCuratorNode()
image1 = FakeTensorBatch()
image2 = FakeTensorBatch()
helper_result = helper.build_character_text(
image1=image1,
image2=image2,
image1_picture_id="1",
image2_picture_id="2",
character_id="char_dave",
name="Dave",
alias="The Locksmith",
gender="male",
age="41",
nationality="English",
occupation="detective",
height_feet="6",
height_inches="2",
accent="English",
general="Tired eyes, cropped brown hair",
wardrobe="weathered red flight jacket",
)
result = curator.curate_prompt(
action_prompt="Dave checks the locked door.",
anatomy_guard="on",
subject_count_guard="auto",
ref_1=helper_result[4],
ref_2=helper_result[5],
)
self.assertIs(result[1], image1)
self.assertIs(result[2], image2)
self.assertEqual(result[10], 2)
self.assertIn("Character facts for <Picture 1> Dave", result[0])
self.assertIn("41 years old", result[0])
self.assertIn("6 foot 2 tall", result[0])
self.assertIn("exactly one named character: <Picture 1> Dave", result[0])
self.assertNotIn("exactly 2 named characters", result[0])
def test_h3_prompt_curator_uses_location_helper_references_by_name(self):
helper = self.image_nodes.DumasLocationHelperNode()
curator = self.image_nodes.DumasH3PromptCuratorNode()
image1 = FakeTensorBatch()
image2 = FakeTensorBatch()
helper_result = helper.build_location_text(
image1=image1,
image2=image2,
image1_picture_id="1",
image2_picture_id="2",
location_id="coffee_shop",
name="Coffee Shop",
alias="Cafe Interior",
description="Warm tungsten lighting, narrow counter, rainy front window",
general="Evening ambience, cramped but cozy",
)
result = curator.curate_prompt(
action_prompt="A slow push through the Coffee Shop as rain streaks the windows.",
anatomy_guard="on",
subject_count_guard="auto",
ref_1=helper_result[3],
ref_2=helper_result[4],
)
self.assertIs(result[1], image1)
self.assertIs(result[2], image2)
self.assertEqual(result[10], 2)
self.assertIn("<Picture 1> Coffee Shop", result[0])
self.assertIn("<Picture 2> Coffee Shop", result[0])
self.assertIn("Location context for <Picture 1> Coffee Shop", result[0])
self.assertIn("Warm tungsten lighting", result[0])
self.assertNotIn("Subject count guard:", result[0])
def test_h3_prompt_curator_defaults_anatomy_guard_to_on(self):
required = self.image_nodes.DumasH3PromptCuratorNode.INPUT_TYPES()["required"]
self.assertEqual(required["anatomy_guard"][1]["default"], "on")
def test_helper_node_mappings_use_general_purpose_helpers(self):
mappings = self.image_nodes.NODE_CLASS_MAPPINGS
display = self.image_nodes.NODE_DISPLAY_NAME_MAPPINGS
self.assertIs(mappings["DumasCharacterHelper"], self.image_nodes.DumasCharacterHelperNode)
self.assertIs(mappings["DumasLocationHelper"], self.image_nodes.DumasLocationHelperNode)
self.assertIs(mappings["DumasSoundscapeHelper"], self.image_nodes.DumasSoundscapeHelperNode)
self.assertIs(mappings["DumasBackgroundMusicHelper"], self.image_nodes.DumasBackgroundMusicHelperNode)
self.assertIs(mappings["DumasH3PromptCurator"], self.image_nodes.DumasH3PromptCuratorNode)
self.assertEqual(display["DumasCharacterHelper"], "Dumas Character Helper")
self.assertEqual(display["DumasLocationHelper"], "Dumas Location Helper")
self.assertEqual(display["DumasSoundscapeHelper"], "Dumas Soundscape Helper")
self.assertEqual(display["DumasBackgroundMusicHelper"], "Dumas Background Music Helper")
self.assertEqual(display["DumasH3PromptCurator"], "Dumas H3 Prompt Curator")
def test_h3_prompt_curator_uses_documented_reference_limits(self):
node = self.image_nodes.DumasH3PromptCuratorNode()
self.assertEqual(len(node.RETURN_TYPES), 33)
self.assertEqual(node.RETURN_NAMES[1:10], tuple(f"ref_image_{i}" for i in range(1, 10)))
self.assertEqual(node.RETURN_NAMES[12:15], ("anchor", "sounds", "bgm"))
self.assertEqual(node.RETURN_NAMES[15:24], tuple(f"original_ref_{i}" for i in range(1, 10)))
self.assertEqual(
node.RETURN_NAMES[24:33],
tuple(f"compiled_ref_description_{i}" for i in range(1, 10)),
)
def test_normalize_reference_upgrades_generic_summary_with_socket_picture_id(self):
image = FakeTensorBatch()
@@ -417,6 +717,19 @@ class DumasImageNodeTests(unittest.TestCase):
self.assertIn("real time", result[0])
self.assertNotIn("persistent camera language", result[0])
def test_anchor_style_node_strips_legacy_persistent_anchor_note(self):
node = self.image_nodes.DumasAnchorStyleNode()
legacy = (
"Gritty handheld realism. Keep this anchor focused on persistent camera "
"language, lighting, texture, environment treatment, and tone; do not "
"name characters or describe one-off actions."
)
result = node.build_anchor("cinematic action movie", legacy)
self.assertEqual(result[0], "Gritty handheld realism.")
self.assertNotIn("persistent camera language", result[0])
def test_anchor_style_node_prefers_manual_description_edits(self):
node = self.image_nodes.DumasAnchorStyleNode()
custom = "Lo-fi pirate broadcast with smeared highlights and anxious zoom corrections."