Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a7f7225b3a | ||
|
|
5dc5c2ce8b | ||
|
|
83645811f4 | ||
|
|
216bc05762 | ||
|
|
49e099ee7f | ||
|
|
32a16645b5 | ||
|
|
7bb0b0abea | ||
|
|
0b189dcf7a | ||
|
|
e8599055a1 | ||
|
|
618a48e4d9 | ||
|
|
cbabcf8208 | ||
|
|
fbfbe4ef5a | ||
|
|
5b52793fad | ||
|
|
7e45c81589 | ||
|
|
027865b9ca | ||
|
|
9f0c546681 | ||
|
|
30ae81b86b | ||
|
|
179fa0778e | ||
|
|
a34c9eb4ff | ||
|
|
782a7d658b | ||
|
|
04a61af874 | ||
|
|
a0d80bcc47 | ||
|
|
fbb5f799cd | ||
|
|
9dc3c405a6 | ||
|
|
21ae4d62e0 | ||
|
|
836a6eba33 | ||
|
|
3b63d33ec7 | ||
|
|
c62921c3e8 | ||
|
|
3ab6342ce5 | ||
|
|
51e03a39b7 | ||
|
|
f7b94ccaef | ||
|
|
f6120a8500 | ||
|
|
0e119646ab | ||
|
|
c89570eae9 | ||
|
|
36d9f4369c | ||
|
|
2cb3c694f0 | ||
|
|
5c12cd18a3 | ||
|
|
c1d937e0e2 |
+27
-66
@@ -1149,81 +1149,42 @@ These two belong together.
|
|||||||
|
|
||||||
If you change `shift_video`, you usually need to change `shift_audio` in proportion.
|
If you change `shift_video`, you usually need to change `shift_audio` in proportion.
|
||||||
|
|
||||||
## Group 8: Detail Pass
|
## Group 8: Latent Upscale
|
||||||
|
|
||||||
This is the optional second pass.
|
This is the optional latent refinement stage, used before decode.
|
||||||
|
|
||||||
### `detail_pass`
|
The long-video node now expects a separate `Dumas H3 Latent Upscale Params` node for this stage.
|
||||||
|
Wire that node into the `latent_upscale_param` input when you want the shot to be upscaled and lightly
|
||||||
Enables the refinement pass.
|
re-sampled before decode.
|
||||||
|
|
||||||
What this really means:
|
What this really means:
|
||||||
|
|
||||||
- the node renders the beat once
|
- the node renders the beat once
|
||||||
- then runs a second sampler pass over that result
|
- the sampled latent is upscaled in latent space to the target size
|
||||||
- the goal is to polish, not to invent a whole different shot
|
- the conditioning is rebuilt at that target size
|
||||||
|
- the node then runs a short refinement pass over the upscaled latent with the sampler, scheduler, step count, denoise, and megapixel target you picked on the latent-upscale params node
|
||||||
### `detail_sampler_name`
|
- if the target is larger than the spatial tile size, that refinement pass is processed in spatial batches using the same tile defaults as the upstream latent-split node
|
||||||
|
- the spatial stitch mode follows the upstream overlap controls, including `linear`, `smoothstep`, `overwrite`, and `midpoint`
|
||||||
Sampler for the refinement pass.
|
- the node also carries the upstream split compatibility knobs (`chunk_length`, `temporal_overlap`, `resize_conditioning`, and `anchor_strength`) so the control surface stays in one place
|
||||||
|
|
||||||
### `detail_scheduler`
|
|
||||||
|
|
||||||
Scheduler for the refinement pass.
|
|
||||||
|
|
||||||
### `detail_steps`
|
|
||||||
|
|
||||||
Extra steps for the refinement pass.
|
|
||||||
|
|
||||||
What this really means:
|
|
||||||
|
|
||||||
- more steps gives the second pass more opportunity to change the image
|
|
||||||
- that can help detail
|
|
||||||
- but after a point it stops being "cleanup" and starts becoming "rewrite"
|
|
||||||
|
|
||||||
### `detail_denoise`
|
|
||||||
|
|
||||||
How strongly the refinement pass is allowed to rewrite the beat.
|
|
||||||
|
|
||||||
What this really means:
|
|
||||||
|
|
||||||
- low denoise = polish what is already there
|
|
||||||
- high denoise = let the second pass substantially alter what is already there
|
|
||||||
|
|
||||||
### How The Detail-Pass Settings Work Together
|
|
||||||
|
|
||||||
The detail pass starts from the first-pass result and tries to polish it.
|
|
||||||
|
|
||||||
Gentle settings:
|
|
||||||
|
|
||||||
- low to medium `detail_steps`
|
|
||||||
- low `detail_denoise`
|
|
||||||
|
|
||||||
Aggressive settings:
|
|
||||||
|
|
||||||
- high `detail_steps`
|
|
||||||
- high `detail_denoise`
|
|
||||||
|
|
||||||
Aggressive settings can improve texture, but they can also:
|
|
||||||
|
|
||||||
- change faces
|
|
||||||
- pull away from references
|
|
||||||
- break continuity
|
|
||||||
|
|
||||||
That is why this group should be read as one combined strength control:
|
|
||||||
|
|
||||||
- `detail_pass` decides whether the second pass exists
|
|
||||||
- `detail_steps` decides how long it keeps working
|
|
||||||
- `detail_denoise` decides how free it is to change things
|
|
||||||
- `detail_sampler_name` and `detail_scheduler` shape how that rewrite behaves
|
|
||||||
|
|
||||||
Good starting point:
|
Good starting point:
|
||||||
|
|
||||||
- `detail_pass = on`
|
- use the `model` mode when you want the strongest latent detail recovery
|
||||||
- `detail_sampler_name = euler`
|
- use the interpolation mode when you want a cheaper resize-only path
|
||||||
- `detail_scheduler = beta`
|
- start with `euler_ancestral`, `simple`, `2` steps, and `0.2` denoise
|
||||||
- `detail_steps = 4` to `8`
|
- leave width and height at `0` unless you want an exact override; otherwise `megapixels` drives the target size
|
||||||
- `detail_denoise = 0.20` to `0.35`
|
- for long shots on smaller cards, start with `chunk_length = 85` and `temporal_overlap = 17` so the latent upscaler works in shorter temporal passes
|
||||||
|
- leave the spatial tile inputs at their defaults first: `512x512` tiles, `64` overlap, `0` fade width, `earlier` overlap mode
|
||||||
|
- keep `linear` blend first unless you want to reproduce a specific upstream stitch style
|
||||||
|
- leave the split compatibility knobs alone unless you specifically need to mirror the upstream node behavior
|
||||||
|
|
||||||
|
The important part is that this stage is still a latent pass, not a pixel-space resize:
|
||||||
|
|
||||||
|
- it happens before decode
|
||||||
|
- it can change structure more than a normal image upscale
|
||||||
|
- it is the place to recover detail without adding another full detail-pass toggle
|
||||||
|
|
||||||
|
If you do not wire the helper node, the long-video node skips latent upscale entirely and renders as before.
|
||||||
|
|
||||||
## Group 9: Performance, Decode, And Upscale
|
## Group 9: Performance, Decode, And Upscale
|
||||||
|
|
||||||
|
|||||||
@@ -43,8 +43,14 @@
|
|||||||
- Prompt `<Picture N>` tags now map to the actual ref socket numbers you wire, even with gaps such as only `ref_2` and `ref_7` connected.
|
- Prompt `<Picture N>` tags now map to the actual ref socket numbers you wire, even with gaps such as only `ref_2` and `ref_7` connected.
|
||||||
- Character refs now contribute appearance and wardrobe context from the same structured object, while location refs contribute environment context from theirs.
|
- Character refs now contribute appearance and wardrobe context from the same structured object, while location refs contribute environment context from theirs.
|
||||||
- The default ref2v bias is now stronger: `ref_mode` defaults to `auto ref2v` so untagged prompts condition every shot instead of only shot 1, and `ref_noise_aug` defaults to `0.95` rather than the upstream-literal `0.999`.
|
- The default ref2v bias is now stronger: `ref_mode` defaults to `auto ref2v` so untagged prompts condition every shot instead of only shot 1, and `ref_noise_aug` defaults to `0.95` rather than the upstream-literal `0.999`.
|
||||||
|
- `Dumas H3 Latent Upscale Params` provides the optional pre-decode latent refinement stage for the long-video node.
|
||||||
- Per-shot directives now support `continuity:`, `ref_mode:`, `ref_noise_aug:`, `anchor_add:`, `soundscape:`, and `music:` in addition to the existing timing and wardrobe directives.
|
- Per-shot directives now support `continuity:`, `ref_mode:`, `ref_noise_aug:`, `anchor_add:`, `soundscape:`, and `music:` in addition to the existing timing and wardrobe directives.
|
||||||
|
|
||||||
|
- `Dumas H3 Latent Upscale Params`
|
||||||
|
- Inputs: `mode`, `model_name`, `method`, `width`, `height`, `device`, `precision`, `sampler_name`, `scheduler`, `steps`, `denoise`, `megapixels`, `tile_width`, `tile_height`, `overlap`, `fade_width`, `fade_height`, `overlap_mode`, `overlap_blend`, `tile_size_mode`, `grid_rows`, `grid_cols`, `spatial_w_overlap`, `spatial_h_overlap`, `min_tile_size`, `masked_area_noise`, `brightness_match`, `dynamic_fade`, `dynamic_fade_min`, `chunk_length`, `temporal_overlap`, `resize_conditioning`, `anchor_strength`
|
||||||
|
- Output: `latent_upscale_param`
|
||||||
|
- Bundles the optional latent-space upscaler settings used by `Dumas H3 Long Videos` before decode, so the main node can rebuild conditioning at the target size and run a short refinement pass with your chosen sampler, scheduler, step count, denoise, and the full upstream spatial split controls.
|
||||||
|
|
||||||
- `Dumas H3 Beat Prompt`
|
- `Dumas H3 Beat Prompt`
|
||||||
- Inputs: authored through the custom front-end beat editor
|
- Inputs: authored through the custom front-end beat editor
|
||||||
- Output: `prompt`
|
- Output: `prompt`
|
||||||
|
|||||||
@@ -14,6 +14,10 @@ from .dumas_h3_longvideos import (
|
|||||||
NODE_CLASS_MAPPINGS as H3_LONGVIDEO_NODE_CLASS_MAPPINGS,
|
NODE_CLASS_MAPPINGS as H3_LONGVIDEO_NODE_CLASS_MAPPINGS,
|
||||||
NODE_DISPLAY_NAME_MAPPINGS as H3_LONGVIDEO_NODE_DISPLAY_NAME_MAPPINGS,
|
NODE_DISPLAY_NAME_MAPPINGS as H3_LONGVIDEO_NODE_DISPLAY_NAME_MAPPINGS,
|
||||||
)
|
)
|
||||||
|
from .dumas_h3_latent_upscale import (
|
||||||
|
NODE_CLASS_MAPPINGS as H3_LATENT_UPSCALE_NODE_CLASS_MAPPINGS,
|
||||||
|
NODE_DISPLAY_NAME_MAPPINGS as H3_LATENT_UPSCALE_NODE_DISPLAY_NAME_MAPPINGS,
|
||||||
|
)
|
||||||
from .dumas_h3_shot_length import (
|
from .dumas_h3_shot_length import (
|
||||||
NODE_CLASS_MAPPINGS as H3_SHOT_LENGTH_NODE_CLASS_MAPPINGS,
|
NODE_CLASS_MAPPINGS as H3_SHOT_LENGTH_NODE_CLASS_MAPPINGS,
|
||||||
NODE_DISPLAY_NAME_MAPPINGS as H3_SHOT_LENGTH_NODE_DISPLAY_NAME_MAPPINGS,
|
NODE_DISPLAY_NAME_MAPPINGS as H3_SHOT_LENGTH_NODE_DISPLAY_NAME_MAPPINGS,
|
||||||
@@ -31,6 +35,7 @@ NODE_CLASS_MAPPINGS = {}
|
|||||||
NODE_CLASS_MAPPINGS.update(JSON_NODE_CLASS_MAPPINGS)
|
NODE_CLASS_MAPPINGS.update(JSON_NODE_CLASS_MAPPINGS)
|
||||||
NODE_CLASS_MAPPINGS.update(IMAGE_NODE_CLASS_MAPPINGS)
|
NODE_CLASS_MAPPINGS.update(IMAGE_NODE_CLASS_MAPPINGS)
|
||||||
NODE_CLASS_MAPPINGS.update(H3_LONGVIDEO_NODE_CLASS_MAPPINGS)
|
NODE_CLASS_MAPPINGS.update(H3_LONGVIDEO_NODE_CLASS_MAPPINGS)
|
||||||
|
NODE_CLASS_MAPPINGS.update(H3_LATENT_UPSCALE_NODE_CLASS_MAPPINGS)
|
||||||
NODE_CLASS_MAPPINGS.update(H3_SHOT_LENGTH_NODE_CLASS_MAPPINGS)
|
NODE_CLASS_MAPPINGS.update(H3_SHOT_LENGTH_NODE_CLASS_MAPPINGS)
|
||||||
NODE_CLASS_MAPPINGS.update(H3_INSPECTOR_NODE_CLASS_MAPPINGS)
|
NODE_CLASS_MAPPINGS.update(H3_INSPECTOR_NODE_CLASS_MAPPINGS)
|
||||||
NODE_CLASS_MAPPINGS.update(H3_BEAT_PROMPT_NODE_CLASS_MAPPINGS)
|
NODE_CLASS_MAPPINGS.update(H3_BEAT_PROMPT_NODE_CLASS_MAPPINGS)
|
||||||
@@ -39,6 +44,7 @@ NODE_DISPLAY_NAME_MAPPINGS = {}
|
|||||||
NODE_DISPLAY_NAME_MAPPINGS.update(JSON_NODE_DISPLAY_NAME_MAPPINGS)
|
NODE_DISPLAY_NAME_MAPPINGS.update(JSON_NODE_DISPLAY_NAME_MAPPINGS)
|
||||||
NODE_DISPLAY_NAME_MAPPINGS.update(IMAGE_NODE_DISPLAY_NAME_MAPPINGS)
|
NODE_DISPLAY_NAME_MAPPINGS.update(IMAGE_NODE_DISPLAY_NAME_MAPPINGS)
|
||||||
NODE_DISPLAY_NAME_MAPPINGS.update(H3_LONGVIDEO_NODE_DISPLAY_NAME_MAPPINGS)
|
NODE_DISPLAY_NAME_MAPPINGS.update(H3_LONGVIDEO_NODE_DISPLAY_NAME_MAPPINGS)
|
||||||
|
NODE_DISPLAY_NAME_MAPPINGS.update(H3_LATENT_UPSCALE_NODE_DISPLAY_NAME_MAPPINGS)
|
||||||
NODE_DISPLAY_NAME_MAPPINGS.update(H3_SHOT_LENGTH_NODE_DISPLAY_NAME_MAPPINGS)
|
NODE_DISPLAY_NAME_MAPPINGS.update(H3_SHOT_LENGTH_NODE_DISPLAY_NAME_MAPPINGS)
|
||||||
NODE_DISPLAY_NAME_MAPPINGS.update(H3_INSPECTOR_NODE_DISPLAY_NAME_MAPPINGS)
|
NODE_DISPLAY_NAME_MAPPINGS.update(H3_INSPECTOR_NODE_DISPLAY_NAME_MAPPINGS)
|
||||||
NODE_DISPLAY_NAME_MAPPINGS.update(H3_BEAT_PROMPT_NODE_DISPLAY_NAME_MAPPINGS)
|
NODE_DISPLAY_NAME_MAPPINGS.update(H3_BEAT_PROMPT_NODE_DISPLAY_NAME_MAPPINGS)
|
||||||
|
|||||||
+10
-1
@@ -150,7 +150,16 @@ class H3ModelInspector:
|
|||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def INPUT_TYPES(cls):
|
def INPUT_TYPES(cls):
|
||||||
return {"required": {"model": ("MODEL",)}}
|
return {
|
||||||
|
"required": {
|
||||||
|
"model": (
|
||||||
|
"MODEL",
|
||||||
|
{
|
||||||
|
"tooltip": "MiniMax / H3 model to inspect for quantization and tensor format."
|
||||||
|
},
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
def inspect(self, model):
|
def inspect(self, model):
|
||||||
label, _counts, report = _detect(model)
|
label, _counts, report = _detect(model)
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
+606
-68
@@ -32,9 +32,10 @@ and your VRAM, chains them, and returns the finished video + audio.
|
|||||||
|
|
||||||
Requirements: H3 is CFG-free (cfg 1) and needs no negative prompt -- the node
|
Requirements: H3 is CFG-free (cfg 1) and needs no negative prompt -- the node
|
||||||
makes an empty one internally. The main pass keeps denoise fixed at 1.0: a
|
makes an empty one internally. The main pass keeps denoise fixed at 1.0: a
|
||||||
partial denoise desyncs the joint audio/video schedule. An optional refinement
|
partial denoise desyncs the joint audio/video schedule. An optional latent
|
||||||
pass can use its own denoise later, before any upscale, while keeping the output
|
upscale stage can rebuild the conditioning at a target size, run a short
|
||||||
video-only.
|
refinement pass with its own sampler controls, and keep the output video-only
|
||||||
|
before the final pixel-space upscale options.
|
||||||
|
|
||||||
Verified against ComfyUI core (comfy_extras/nodes_minimax_h3.py, model_base.py,
|
Verified against ComfyUI core (comfy_extras/nodes_minimax_h3.py, model_base.py,
|
||||||
ldm/minimax/model.py, text_encoders/minimax.py, sd.py).
|
ldm/minimax/model.py, text_encoders/minimax.py, sd.py).
|
||||||
@@ -59,6 +60,7 @@ import node_helpers
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
from . import dumas_h3_overlay as _overlay
|
from . import dumas_h3_overlay as _overlay
|
||||||
|
from .dumas_h3_latent_upscale import upscale_latent_video as _upscale_latent_video
|
||||||
from . import dumas_image_nodes as _image_nodes
|
from . import dumas_image_nodes as _image_nodes
|
||||||
except ImportError: # loaded as a bare file (test_prompt_logic.py), not as a package
|
except ImportError: # loaded as a bare file (test_prompt_logic.py), not as a package
|
||||||
import importlib.util as _ilu
|
import importlib.util as _ilu
|
||||||
@@ -70,6 +72,13 @@ except ImportError: # loaded as a bare file (test_prompt_logic.py), not as
|
|||||||
)
|
)
|
||||||
_overlay = _ilu.module_from_spec(_spec)
|
_overlay = _ilu.module_from_spec(_spec)
|
||||||
_spec.loader.exec_module(_overlay)
|
_spec.loader.exec_module(_overlay)
|
||||||
|
_latent_spec = _ilu.spec_from_file_location(
|
||||||
|
"dumas_h3_latent_upscale",
|
||||||
|
_os.path.join(_os.path.dirname(_os.path.abspath(__file__)), "dumas_h3_latent_upscale.py"),
|
||||||
|
)
|
||||||
|
_latent_upscale = _ilu.module_from_spec(_latent_spec)
|
||||||
|
_latent_spec.loader.exec_module(_latent_upscale)
|
||||||
|
_upscale_latent_video = _latent_upscale.upscale_latent_video
|
||||||
_image_nodes = _sys.modules.get("dumas_image_nodes")
|
_image_nodes = _sys.modules.get("dumas_image_nodes")
|
||||||
if _image_nodes is None:
|
if _image_nodes is None:
|
||||||
_img_spec = _ilu.spec_from_file_location(
|
_img_spec = _ilu.spec_from_file_location(
|
||||||
@@ -279,8 +288,6 @@ ADDED_WIDGETS = (
|
|||||||
"exposed_terms", "anatomy_guard", "lock_restraints", "solidity_guard",
|
"exposed_terms", "anatomy_guard", "lock_restraints", "solidity_guard",
|
||||||
"motion_guard", "contact_guard",
|
"motion_guard", "contact_guard",
|
||||||
"auto_soundscape", "allow_nonspeech_vocals",
|
"auto_soundscape", "allow_nonspeech_vocals",
|
||||||
"detail_pass", "detail_sampler_name", "detail_scheduler",
|
|
||||||
"detail_steps", "detail_denoise",
|
|
||||||
)
|
)
|
||||||
|
|
||||||
NL = "\n"
|
NL = "\n"
|
||||||
@@ -4033,6 +4040,88 @@ def _copy_sample_latent(out_latent):
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _retarget_conditioning_spatial(cond, latent_h, latent_w):
|
||||||
|
"""Resize H3 keyframe latents in existing conditioning to a new latent grid."""
|
||||||
|
latent_h = int(latent_h)
|
||||||
|
latent_w = int(latent_w)
|
||||||
|
if latent_h <= 0 or latent_w <= 0:
|
||||||
|
raise RuntimeError("conditioning target latent size must be positive")
|
||||||
|
out = []
|
||||||
|
for item in cond:
|
||||||
|
try:
|
||||||
|
tensor, data = item
|
||||||
|
except Exception:
|
||||||
|
out.append(item)
|
||||||
|
continue
|
||||||
|
nd = dict(data)
|
||||||
|
keyframes = nd.get("minimax_keyframes")
|
||||||
|
if keyframes:
|
||||||
|
resized_keyframes = []
|
||||||
|
for keyframe in keyframes:
|
||||||
|
nkf = dict(keyframe)
|
||||||
|
latent_value = nkf.get("latent")
|
||||||
|
if latent_value is not None and len(getattr(latent_value, "shape", ())) >= 5:
|
||||||
|
if latent_value.shape[3] != latent_h or latent_value.shape[4] != latent_w:
|
||||||
|
b, c, t, h, w = latent_value.shape
|
||||||
|
resized = torch.nn.functional.interpolate(
|
||||||
|
latent_value.to(torch.float32).reshape(b * t, c, h, w),
|
||||||
|
size=(latent_h, latent_w),
|
||||||
|
mode="bilinear",
|
||||||
|
align_corners=False,
|
||||||
|
).reshape(b, c, t, latent_h, latent_w)
|
||||||
|
nkf["latent"] = resized.to(device=latent_value.device, dtype=latent_value.dtype)
|
||||||
|
resized_keyframes.append(nkf)
|
||||||
|
nd["minimax_keyframes"] = resized_keyframes
|
||||||
|
out.append([tensor, nd])
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _pad_to_h3_patch_size(tensor):
|
||||||
|
try:
|
||||||
|
import comfy.ldm.common_dit as common_dit
|
||||||
|
return common_dit.pad_to_patch_size(tensor, (1, 2, 2))
|
||||||
|
except Exception:
|
||||||
|
return tensor
|
||||||
|
|
||||||
|
|
||||||
|
def _crop_conditioning_to_tile(cond, source_h, source_w, row, col, tile_h, tile_w):
|
||||||
|
"""Crop H3 keyframe latents in existing conditioning for a spatial tile."""
|
||||||
|
out = []
|
||||||
|
for item in cond:
|
||||||
|
try:
|
||||||
|
tensor, data = item
|
||||||
|
except Exception:
|
||||||
|
out.append(item)
|
||||||
|
continue
|
||||||
|
nd = dict(data)
|
||||||
|
keyframes = nd.get("minimax_keyframes")
|
||||||
|
if keyframes:
|
||||||
|
cropped_keyframes = []
|
||||||
|
for keyframe in keyframes:
|
||||||
|
nkf = dict(keyframe)
|
||||||
|
latent_value = nkf.get("latent")
|
||||||
|
if latent_value is not None and len(getattr(latent_value, "shape", ())) >= 5:
|
||||||
|
kh, kw = latent_value.shape[3], latent_value.shape[4]
|
||||||
|
if kh != source_h or kw != source_w:
|
||||||
|
b, c, t, h, w = latent_value.shape
|
||||||
|
latent_value = torch.nn.functional.interpolate(
|
||||||
|
latent_value.to(torch.float32).reshape(b * t, c, h, w),
|
||||||
|
size=(source_h, source_w),
|
||||||
|
mode="bilinear",
|
||||||
|
align_corners=False,
|
||||||
|
).reshape(b, c, t, source_h, source_w).to(
|
||||||
|
device=latent_value.device,
|
||||||
|
dtype=latent_value.dtype,
|
||||||
|
)
|
||||||
|
nkf["latent"] = _pad_to_h3_patch_size(
|
||||||
|
latent_value[:, :, :, row:row + tile_h, col:col + tile_w].contiguous()
|
||||||
|
)
|
||||||
|
cropped_keyframes.append(nkf)
|
||||||
|
nd["minimax_keyframes"] = cropped_keyframes
|
||||||
|
out.append([tensor, nd])
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
def _latent_with_replaced_samples(template_latent, sampled_latent):
|
def _latent_with_replaced_samples(template_latent, sampled_latent):
|
||||||
"""Reuse the original latent payload, but swap in freshly sampled tensors."""
|
"""Reuse the original latent payload, but swap in freshly sampled tensors."""
|
||||||
if not isinstance(template_latent, dict):
|
if not isinstance(template_latent, dict):
|
||||||
@@ -4066,6 +4155,237 @@ def _video_only_refined_latent(base_latent, refined_latent):
|
|||||||
return refined_latent
|
return refined_latent
|
||||||
|
|
||||||
|
|
||||||
|
def _latent_upscale_target_size(base_w, base_h, param):
|
||||||
|
width = int(param.get("width", 0) or 0)
|
||||||
|
height = int(param.get("height", 0) or 0)
|
||||||
|
if width > 0 and height > 0:
|
||||||
|
return width, height
|
||||||
|
megapixels = float(param.get("megapixels", 0.0) or 0.0)
|
||||||
|
if megapixels > 0:
|
||||||
|
return scale_to_megapixels(base_w, base_h, megapixels)
|
||||||
|
return int(base_w), int(base_h)
|
||||||
|
|
||||||
|
|
||||||
|
def _latent_upscale_mode(param):
|
||||||
|
if not isinstance(param, dict):
|
||||||
|
return "off"
|
||||||
|
mode = str(param.get("mode") or "").strip()
|
||||||
|
if mode:
|
||||||
|
return mode
|
||||||
|
if "model_name" in param and str(param.get("model_name") or "").strip() not in ("", "none"):
|
||||||
|
return "model"
|
||||||
|
if "method" in param:
|
||||||
|
return "interp"
|
||||||
|
return "off"
|
||||||
|
|
||||||
|
|
||||||
|
def _latent_spatial_grid(h, w, th, tw, ol_h, ol_w):
|
||||||
|
if th <= 0 or tw <= 0:
|
||||||
|
raise ValueError("tile dimensions must be positive")
|
||||||
|
if ol_h >= th or ol_w >= tw:
|
||||||
|
raise ValueError("overlap must be smaller than the tile size")
|
||||||
|
sh = th - ol_h
|
||||||
|
sw = tw - ol_w
|
||||||
|
nrows = 1 if h <= th else math.ceil((h - ol_h) / sh)
|
||||||
|
if (nrows - 1) * sh + th < h:
|
||||||
|
nrows += 1
|
||||||
|
ncols = 1 if w <= tw else math.ceil((w - ol_w) / sw)
|
||||||
|
if (ncols - 1) * sw + tw < w:
|
||||||
|
ncols += 1
|
||||||
|
rows = [i * sh for i in range(nrows)]
|
||||||
|
cols = [j * sw for j in range(ncols)]
|
||||||
|
trows = [min(th, h - r) for r in rows]
|
||||||
|
tcols = [min(tw, w - c) for c in cols]
|
||||||
|
return rows, cols, trows, tcols
|
||||||
|
|
||||||
|
|
||||||
|
def _latent_spatial_blend_weights(t, overlap_mode, overlap_blend="linear"):
|
||||||
|
if overlap_blend == "overwrite":
|
||||||
|
return torch.ones_like(t) if overlap_mode == "later" else torch.zeros_like(t)
|
||||||
|
if overlap_blend == "midpoint":
|
||||||
|
base = (t >= 0.5).to(t.dtype)
|
||||||
|
elif overlap_blend == "smoothstep":
|
||||||
|
base = t * t * (3.0 - 2.0 * t)
|
||||||
|
else:
|
||||||
|
base = t
|
||||||
|
if overlap_mode == "later":
|
||||||
|
return base
|
||||||
|
return 1.0 - base
|
||||||
|
|
||||||
|
|
||||||
|
def _param_value(mapping, key, default):
|
||||||
|
value = mapping.get(key, default)
|
||||||
|
return default if value is None else value
|
||||||
|
|
||||||
|
|
||||||
|
def _tag_oom_stage(exc, stage):
|
||||||
|
if _is_oom(exc):
|
||||||
|
exc._h3_stage = stage
|
||||||
|
return exc
|
||||||
|
|
||||||
|
|
||||||
|
def _grid_1d(size, tile, ol, min_tile):
|
||||||
|
if size <= tile:
|
||||||
|
return [0], [size], [0]
|
||||||
|
sh = tile - ol
|
||||||
|
n = math.ceil((size - ol) / sh)
|
||||||
|
if (n - 1) * sh + tile < size:
|
||||||
|
n += 1
|
||||||
|
rows = [i * sh for i in range(n)]
|
||||||
|
trows = [min(tile, size - r) for r in rows]
|
||||||
|
if min_tile > 0 and n >= 2:
|
||||||
|
edge = size - rows[-1]
|
||||||
|
if edge < min_tile:
|
||||||
|
new_last = size - min_tile
|
||||||
|
if rows[-2] < new_last < rows[-2] + trows[-2]:
|
||||||
|
rows[-1] = new_last
|
||||||
|
trows[-1] = size - new_last
|
||||||
|
ovl = [0] * n
|
||||||
|
for i in range(1, n):
|
||||||
|
ovl[i] = max(0, rows[i - 1] + trows[i - 1] - rows[i])
|
||||||
|
return rows, trows, ovl
|
||||||
|
|
||||||
|
|
||||||
|
def compute_spatial_grid(h, w, th, tw, ol_h, ol_w, min_th=0, min_tw=0):
|
||||||
|
if th <= 0 or tw <= 0:
|
||||||
|
raise ValueError("tile dimensions must be positive")
|
||||||
|
if ol_h >= th or ol_w >= tw:
|
||||||
|
raise ValueError("overlap must be smaller than the tile size")
|
||||||
|
if min_th < 0 or min_tw < 0:
|
||||||
|
raise ValueError("minimum tile size must be non-negative")
|
||||||
|
if min_th > th or min_tw > tw:
|
||||||
|
raise ValueError("minimum tile size must not exceed the tile size")
|
||||||
|
rows, trows, row_ovl = _grid_1d(h, th, ol_h, min_th)
|
||||||
|
cols, tcols, col_ovl = _grid_1d(w, tw, ol_w, min_tw)
|
||||||
|
return rows, cols, trows, tcols, row_ovl, col_ovl
|
||||||
|
|
||||||
|
|
||||||
|
def spatial_fade_mask(tile_h, tile_w, ol_h, ol_w, done_top, done_left, fade_h=0, fade_w=0):
|
||||||
|
mask = torch.ones(tile_h, tile_w, dtype=torch.float32)
|
||||||
|
if done_left and ol_w > 0:
|
||||||
|
if fade_w == 0:
|
||||||
|
mask[:, :ol_w] = 0.0
|
||||||
|
else:
|
||||||
|
f = min(fade_w, ol_w)
|
||||||
|
frozen_w = ol_w - f
|
||||||
|
w = torch.linspace(0.0, 1.0, f)
|
||||||
|
mask[:, :frozen_w] = 0.0
|
||||||
|
mask[:, frozen_w:ol_w] = torch.minimum(mask[:, frozen_w:ol_w], w[None, :])
|
||||||
|
if done_top and ol_h > 0:
|
||||||
|
if fade_h == 0:
|
||||||
|
mask[:ol_h, :] = 0.0
|
||||||
|
else:
|
||||||
|
f = min(fade_h, ol_h)
|
||||||
|
frozen_h = ol_h - f
|
||||||
|
w = torch.linspace(0.0, 1.0, f)
|
||||||
|
mask[:frozen_h, :] = 0.0
|
||||||
|
mask[frozen_h:ol_h, :] = torch.minimum(mask[frozen_h:ol_h, :], w[:, None])
|
||||||
|
return mask
|
||||||
|
|
||||||
|
|
||||||
|
def _fade_band(band, fade, axis):
|
||||||
|
n = band.shape[axis]
|
||||||
|
f = min(int(fade), n)
|
||||||
|
if f == 0:
|
||||||
|
band[:] = 0.0
|
||||||
|
return
|
||||||
|
w = torch.linspace(0.0, 1.0, f, dtype=band.dtype, device=band.device)
|
||||||
|
frozen = n - f
|
||||||
|
if axis == 1:
|
||||||
|
w = w[None, :]
|
||||||
|
band[:, :frozen] = torch.minimum(
|
||||||
|
band[:, :frozen], torch.zeros(frozen, dtype=band.dtype, device=band.device)
|
||||||
|
)
|
||||||
|
band[:, frozen:] = torch.minimum(band[:, frozen:], w)
|
||||||
|
else:
|
||||||
|
w = w[:, None]
|
||||||
|
band[:frozen, :] = torch.minimum(
|
||||||
|
band[:frozen, :], torch.zeros((frozen, 1), dtype=band.dtype, device=band.device)
|
||||||
|
)
|
||||||
|
band[frozen:, :] = torch.minimum(band[frozen:, :], w)
|
||||||
|
|
||||||
|
|
||||||
|
def make_fade_mask(tile_h, tile_w, ol_h, ol_w, done_top, done_left, fade_h=0, fade_w=0):
|
||||||
|
mask = torch.ones(tile_h, tile_w, dtype=torch.float32)
|
||||||
|
if done_left and ol_w > 0:
|
||||||
|
_fade_band(mask[:, :ol_w], fade_w, 1)
|
||||||
|
if done_top and ol_h > 0:
|
||||||
|
_fade_band(mask[:ol_h, :], fade_h, 0)
|
||||||
|
return mask
|
||||||
|
|
||||||
|
|
||||||
|
def bright_match_tile(tile, ref, clamp=0.05):
|
||||||
|
d = (tile - ref).float().reshape(tile.shape[0], tile.shape[1], tile.shape[2], -1)
|
||||||
|
dc = d.median(dim=-1).values.clamp(-clamp, clamp)
|
||||||
|
return tile - dc.to(tile.dtype).view(tile.shape[0], tile.shape[1], tile.shape[2], 1, 1)
|
||||||
|
|
||||||
|
|
||||||
|
def _dynamic_fade_closure(sp, fw, fh, tr, tc, tr_s, tc_s, ovh, ovw, done_top, done_left, video_flat, mn=0.0):
|
||||||
|
schedule = sp.get("dynamic_fade", "off")
|
||||||
|
if schedule == "off":
|
||||||
|
return None
|
||||||
|
fmin_w = int(sp.get("dynamic_fade_min", 0)) // 16
|
||||||
|
fmin_h = int(sp.get("dynamic_fade_min", 0)) // 16
|
||||||
|
if fw <= fmin_w and fh <= fmin_h:
|
||||||
|
return None
|
||||||
|
fw_start, fh_start = max(fw, 0), max(fh, 0)
|
||||||
|
fmin_w, fmin_h = min(fmin_w, fw_start), min(fmin_h, fh_start)
|
||||||
|
done_top = done_top and ovh > 0
|
||||||
|
done_left = done_left and ovw > 0
|
||||||
|
s_tok = tr_s * tc_s
|
||||||
|
n_frames = video_flat // s_tok
|
||||||
|
|
||||||
|
def fade_at(p):
|
||||||
|
if schedule == "widening":
|
||||||
|
return fmin_w + (fw_start - fmin_w) * p, fmin_h + (fh_start - fmin_h) * p
|
||||||
|
return fw_start - (fw_start - fmin_w) * p, fh_start - (fh_start - fmin_h) * p
|
||||||
|
|
||||||
|
cache = {}
|
||||||
|
|
||||||
|
def step_fn(sigma, denoise_mask, **kwargs):
|
||||||
|
sigmas = kwargs.get("extra_options", {}).get("sigmas")
|
||||||
|
n_sigmas = int(sigmas.numel()) if sigmas is not None else 0
|
||||||
|
masks = cache.get(n_sigmas)
|
||||||
|
if masks is None:
|
||||||
|
step_count = max(n_sigmas - 1, 1)
|
||||||
|
masks = []
|
||||||
|
for i in range(n_sigmas - 1):
|
||||||
|
p = i / (step_count - 1) if step_count > 1 else 0.0
|
||||||
|
cw, ch = fade_at(p)
|
||||||
|
m = make_fade_mask(tr_s, tc_s, ovh, ovw, done_top, done_left,
|
||||||
|
fade_h=round(ch), fade_w=round(cw))
|
||||||
|
m[tr:tr_s, :] = 0.0
|
||||||
|
m[:, tc:tc_s] = 0.0
|
||||||
|
if mn > 0:
|
||||||
|
m = m + mn * (1.0 - m)
|
||||||
|
masks.append(m)
|
||||||
|
cache[n_sigmas] = masks
|
||||||
|
idx = 0
|
||||||
|
if sigmas is not None:
|
||||||
|
idx = int((sigmas > sigma + 1e-6).sum())
|
||||||
|
m = masks[min(idx, len(masks) - 1)]
|
||||||
|
flat = denoise_mask.clone()
|
||||||
|
flat.reshape(denoise_mask.shape[0], -1)[:, :video_flat] = \
|
||||||
|
m.reshape(1, -1).repeat(denoise_mask.shape[0], n_frames)
|
||||||
|
return flat
|
||||||
|
|
||||||
|
return step_fn
|
||||||
|
|
||||||
|
|
||||||
|
def _nested_tensor_parts(samples):
|
||||||
|
if samples is None:
|
||||||
|
return ()
|
||||||
|
parts = getattr(samples, "tensors", None)
|
||||||
|
if parts is not None:
|
||||||
|
return tuple(parts)
|
||||||
|
if hasattr(samples, "unbind"):
|
||||||
|
try:
|
||||||
|
return tuple(samples.unbind())
|
||||||
|
except Exception:
|
||||||
|
return ()
|
||||||
|
return ()
|
||||||
|
|
||||||
|
|
||||||
def _coerce_bool_flag(value):
|
def _coerce_bool_flag(value):
|
||||||
if isinstance(value, str):
|
if isinstance(value, str):
|
||||||
text = value.strip().lower()
|
text = value.strip().lower()
|
||||||
@@ -4097,7 +4417,7 @@ def _format_timing_note(shot_timings):
|
|||||||
"total": 0.0,
|
"total": 0.0,
|
||||||
"retry_elapsed": 0.0,
|
"retry_elapsed": 0.0,
|
||||||
"sample": 0.0,
|
"sample": 0.0,
|
||||||
"detail_sample": 0.0,
|
"latent_upscale_sample": 0.0,
|
||||||
"decode_video": 0.0,
|
"decode_video": 0.0,
|
||||||
"decode_audio": 0.0,
|
"decode_audio": 0.0,
|
||||||
"cleanup": 0.0,
|
"cleanup": 0.0,
|
||||||
@@ -4108,7 +4428,7 @@ def _format_timing_note(shot_timings):
|
|||||||
totals["total"] += float(shot.get("total", 0.0) or 0.0)
|
totals["total"] += float(shot.get("total", 0.0) or 0.0)
|
||||||
totals["retry_elapsed"] += float(shot.get("retry_elapsed", 0.0) or 0.0)
|
totals["retry_elapsed"] += float(shot.get("retry_elapsed", 0.0) or 0.0)
|
||||||
totals["sample"] += float(shot.get("sample", 0.0) or 0.0)
|
totals["sample"] += float(shot.get("sample", 0.0) or 0.0)
|
||||||
totals["detail_sample"] += float(shot.get("detail_sample", 0.0) or 0.0)
|
totals["latent_upscale_sample"] += float(shot.get("latent_upscale_sample", 0.0) or 0.0)
|
||||||
totals["decode_video"] += float(shot.get("decode_video", 0.0) or 0.0)
|
totals["decode_video"] += float(shot.get("decode_video", 0.0) or 0.0)
|
||||||
totals["decode_audio"] += float(shot.get("decode_audio", 0.0) or 0.0)
|
totals["decode_audio"] += float(shot.get("decode_audio", 0.0) or 0.0)
|
||||||
totals["cleanup"] += float(shot.get("cleanup", 0.0) or 0.0)
|
totals["cleanup"] += float(shot.get("cleanup", 0.0) or 0.0)
|
||||||
@@ -4124,8 +4444,8 @@ def _format_timing_note(shot_timings):
|
|||||||
]
|
]
|
||||||
if totals["retry_elapsed"]:
|
if totals["retry_elapsed"]:
|
||||||
pieces.append(f"retry elapsed {_format_elapsed_seconds(totals['retry_elapsed'])}")
|
pieces.append(f"retry elapsed {_format_elapsed_seconds(totals['retry_elapsed'])}")
|
||||||
if totals["detail_sample"]:
|
if totals["latent_upscale_sample"]:
|
||||||
pieces.append(f"detail {_format_elapsed_seconds(totals['detail_sample'])}")
|
pieces.append(f"latent upscale {_format_elapsed_seconds(totals['latent_upscale_sample'])}")
|
||||||
if totals["retries"]:
|
if totals["retries"]:
|
||||||
pieces.append(f"retries {totals['retries']}")
|
pieces.append(f"retries {totals['retries']}")
|
||||||
if slowest is not None:
|
if slowest is not None:
|
||||||
@@ -5847,6 +6167,29 @@ def _evict_all_but(keep_model):
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _evict_for_latent_upscale(model):
|
||||||
|
"""Clear the sampler model before loading the auxiliary latent upscaler."""
|
||||||
|
try:
|
||||||
|
unload_clones = getattr(mm, "unload_model_and_clones", None)
|
||||||
|
if callable(unload_clones):
|
||||||
|
try:
|
||||||
|
unload_clones(model, unload_additional_models=False)
|
||||||
|
mm.soft_empty_cache()
|
||||||
|
return
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
mm.unload_all_models()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
try:
|
||||||
|
mm.soft_empty_cache(True)
|
||||||
|
except Exception:
|
||||||
|
try:
|
||||||
|
mm.soft_empty_cache()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -6084,6 +6427,12 @@ class H3LongVideos:
|
|||||||
"decode_tile_size": ("INT", {"default": 0, "min": 0, "max": 1024, "step": 32,
|
"decode_tile_size": ("INT", {"default": 0, "min": 0, "max": 1024, "step": 32,
|
||||||
"tooltip": "Spatial tile size for the VAE decode (tile_x/tile_y). 0 = ComfyUI default. "
|
"tooltip": "Spatial tile size for the VAE decode (tile_x/tile_y). 0 = ComfyUI default. "
|
||||||
"Try 256 on a tight card at 1344x768."}),
|
"Try 256 on a tight card at 1344x768."}),
|
||||||
|
"latent_upscale_param": ("DUMAS_H3_LATENT_UPSCALE_PARAM", {
|
||||||
|
"tooltip": "Output of 'Dumas H3 Latent Upscale Params'. When connected, the first-pass "
|
||||||
|
"latent is upscaled and run through a short refinement pass before decode, "
|
||||||
|
"using the sampler, scheduler, steps, denoise, megapixel target, and optional "
|
||||||
|
"spatial batching from that node. "
|
||||||
|
"Leave unconnected to skip latent upscaling entirely."}),
|
||||||
"upscale": (["off", "rtx", "model", "lanczos"], {"default": "off",
|
"upscale": (["off", "rtx", "model", "lanczos"], {"default": "off",
|
||||||
"tooltip": "Optional post-pass on the finished frames. 'rtx' = NVIDIA RTX Video Super "
|
"tooltip": "Optional post-pass on the finished frames. 'rtx' = NVIDIA RTX Video Super "
|
||||||
"Resolution (Tensor Cores -- fastest and best for video; needs the "
|
"Resolution (Tensor Cores -- fastest and best for video; needs the "
|
||||||
@@ -6363,26 +6712,6 @@ class H3LongVideos:
|
|||||||
"the jacket, 'wardrobe: += sunglasses' adds one. TWO+ PEOPLE: name them -- "
|
"the jacket, 'wardrobe: += sunglasses' adds one. TWO+ PEOPLE: name them -- "
|
||||||
"'Maya = grey shorts, red jacket; Jon = navy overalls', then edit one at a "
|
"'Maya = grey shorts, red jacket; Jon = navy overalls', then edit one at a "
|
||||||
"time: 'wardrobe: Maya -= jacket' leaves Jon untouched."}),
|
"time: 'wardrobe: Maya -= jacket' leaves Jon untouched."}),
|
||||||
"detail_pass": ("BOOLEAN", {"default": False,
|
|
||||||
"tooltip": "Run a second refinement sampler on each beat BEFORE any upscale. "
|
|
||||||
"It reuses the same conditioning and keeps the output video-only by "
|
|
||||||
"preserving the first pass's audio latent. Use it for detail cleanup, not "
|
|
||||||
"for huge rewrites: too many steps or too much denoise can pull identity or "
|
|
||||||
"continuity away from the main pass."}),
|
|
||||||
"detail_sampler_name": (comfy.samplers.KSampler.SAMPLERS, {"default": "euler",
|
|
||||||
"tooltip": "Sampler for the optional refinement pass. Euler is the maintained default "
|
|
||||||
"direction for this lane."}),
|
|
||||||
"detail_scheduler": (comfy.samplers.KSampler.SCHEDULERS, {"default": "beta",
|
|
||||||
"tooltip": "Scheduler for the optional refinement pass. Beta is the maintained default "
|
|
||||||
"direction for the H3 enhancement lane."}),
|
|
||||||
"detail_steps": ("INT", {"default": 8, "min": 1, "max": 200,
|
|
||||||
"tooltip": "Extra steps for the refinement pass only. Start around 4-8. More is not "
|
|
||||||
"automatically better; once the pass starts rewriting instead of polishing, "
|
|
||||||
"identity and continuity can drift."}),
|
|
||||||
"detail_denoise": ("FLOAT", {"default": 0.4, "min": 0.0, "max": 1.0, "step": 0.01,
|
|
||||||
"tooltip": "How hard the refinement pass is allowed to rewrite the beat latent. Start "
|
|
||||||
"around 0.20-0.35 for gentle cleanup; 0.4+ is stronger and can noticeably "
|
|
||||||
"change faces, motion or composition."}),
|
|
||||||
},
|
},
|
||||||
# Read-only graph access, for SLA-LoRA detection: a LoRA's filename is
|
# Read-only graph access, for SLA-LoRA detection: a LoRA's filename is
|
||||||
# the only thing that identifies an SLA build, and the graph is the only
|
# the only thing that identifies an SLA build, and the graph is the only
|
||||||
@@ -6407,9 +6736,8 @@ class H3LongVideos:
|
|||||||
def _render(self, model, clip, vae, audio_vae, negative, prompt, w, h, ln, fps, tiled, sa,
|
def _render(self, model, clip, vae, audio_vae, negative, prompt, w, h, ln, fps, tiled, sa,
|
||||||
handoff, decode_tile_frames=0, decode_tile_size=0,
|
handoff, decode_tile_frames=0, decode_tile_size=0,
|
||||||
refs=None, ref_image_size="match", ref_noise_aug=None, silent=False,
|
refs=None, ref_image_size="match", ref_noise_aug=None, silent=False,
|
||||||
detail_pass=False, detail_sampler_name="euler", detail_scheduler="beta",
|
latent_upscale_param=None, timing_sink=None):
|
||||||
detail_steps=8, detail_denoise=0.4, timing_sink=None):
|
timing = {"sample": 0.0, "latent_upscale_sample": 0.0, "decode_video": 0.0, "decode_audio": 0.0, "cleanup": 0.0}
|
||||||
timing = {"sample": 0.0, "detail_sample": 0.0, "decode_video": 0.0, "decode_audio": 0.0, "cleanup": 0.0}
|
|
||||||
positive, latent = _build_shot_conditioning(clip, vae, prompt, w, h, ln, fps, handoff,
|
positive, latent = _build_shot_conditioning(clip, vae, prompt, w, h, ln, fps, handoff,
|
||||||
ref_images=refs, ref_image_size=ref_image_size,
|
ref_images=refs, ref_image_size=ref_image_size,
|
||||||
ref_noise_aug=ref_noise_aug,
|
ref_noise_aug=ref_noise_aug,
|
||||||
@@ -6428,43 +6756,213 @@ class H3LongVideos:
|
|||||||
# OOM retry cannot help an OOM raised here -- it just re-runs the whole
|
# OOM retry cannot help an OOM raised here -- it just re-runs the whole
|
||||||
# sampling pass and fails the same way, which on a 362-frame shot is four
|
# sampling pass and fails the same way, which on a 362-frame shot is four
|
||||||
# more minutes for nothing.
|
# more minutes for nothing.
|
||||||
if _is_oom(e):
|
raise _tag_oom_stage(e, "sampling")
|
||||||
e._h3_stage = "sampling"
|
|
||||||
raise
|
|
||||||
refined_out = out
|
refined_out = out
|
||||||
detail_pass = _coerce_bool_flag(detail_pass)
|
latent_upscale_param = latent_upscale_param or None
|
||||||
if detail_pass:
|
latent_upscale_mode = _latent_upscale_mode(latent_upscale_param)
|
||||||
detail_latent = _latent_with_replaced_samples(latent, out)
|
if latent_upscale_mode != "off":
|
||||||
try:
|
try:
|
||||||
detail_start = time.perf_counter()
|
latent_start = time.perf_counter()
|
||||||
|
out_samples = out["samples"]
|
||||||
|
parts = _nested_tensor_parts(out_samples)
|
||||||
|
if not getattr(out_samples, "is_nested", False) or len(parts) < 2:
|
||||||
|
raise RuntimeError("latent upscale expects a nested AV latent")
|
||||||
|
if latent_upscale_mode == "model" and str(latent_upscale_param.get("device", "cuda")) == "cuda":
|
||||||
|
_evict_for_latent_upscale(model)
|
||||||
|
upscaled_video, up_h, up_w = _upscale_latent_video(parts[0], latent_upscale_param)
|
||||||
|
full_audio = parts[1]
|
||||||
|
# Drop the first-pass sampling state before we start the refinement
|
||||||
|
# pass; otherwise the 12-step base latent and the upscale latent sit
|
||||||
|
# in memory together and can trigger a retry loop.
|
||||||
|
out["samples"] = comfy.nested_tensor.NestedTensor((upscaled_video, full_audio))
|
||||||
|
del out_samples, parts
|
||||||
|
mm.soft_empty_cache()
|
||||||
|
target_w = int(up_w) * 16
|
||||||
|
target_h = int(up_h) * 16
|
||||||
|
if target_w <= 0 or target_h <= 0:
|
||||||
|
raise RuntimeError("latent upscale target size must be positive")
|
||||||
|
try:
|
||||||
|
upscale_cond = _retarget_conditioning_spatial(positive, int(up_h), int(up_w))
|
||||||
|
upscale_latent = dict(latent) if isinstance(latent, dict) else {}
|
||||||
|
except Exception:
|
||||||
|
upscale_cond, upscale_latent = _build_shot_conditioning(
|
||||||
|
clip, vae, prompt, target_w, target_h, ln, fps, handoff,
|
||||||
|
ref_images=refs, ref_image_size=ref_image_size,
|
||||||
|
ref_noise_aug=ref_noise_aug, audio_vae=audio_vae, silent=silent)
|
||||||
|
upscale_latent["samples"] = comfy.nested_tensor.NestedTensor((upscaled_video, full_audio))
|
||||||
|
del positive, latent
|
||||||
|
refine_steps = int(latent_upscale_param.get("steps", 2) or 2)
|
||||||
|
refine_sampler = latent_upscale_param.get("sampler_name", sn)
|
||||||
|
refine_scheduler = latent_upscale_param.get("scheduler", sch)
|
||||||
|
refine_denoise_value = latent_upscale_param.get("denoise", latent_upscale_param.get("refine_denoise", 0.2))
|
||||||
|
refine_denoise = 0.2 if refine_denoise_value is None else float(refine_denoise_value)
|
||||||
|
tile_size_mode = str(latent_upscale_param.get("tile_size_mode", "specific_size"))
|
||||||
|
tile_w_px = int(latent_upscale_param.get("tile_width", 512) or 512)
|
||||||
|
tile_h_px = int(latent_upscale_param.get("tile_height", 512) or 512)
|
||||||
|
overlap_px = max(0, int(_param_value(latent_upscale_param, "overlap", 64)))
|
||||||
|
fade_w_px = max(0, int(_param_value(latent_upscale_param, "fade_width", 32)))
|
||||||
|
fade_h_px = max(0, int(_param_value(latent_upscale_param, "fade_height", 32)))
|
||||||
|
overlap_mode = str(latent_upscale_param.get("overlap_mode", "earlier"))
|
||||||
|
overlap_blend = str(latent_upscale_param.get("overlap_blend", "linear"))
|
||||||
|
grid_rows = max(1, int(_param_value(latent_upscale_param, "grid_rows", 2)))
|
||||||
|
grid_cols = max(1, int(_param_value(latent_upscale_param, "grid_cols", 2)))
|
||||||
|
spatial_w_overlap_px = max(0, int(_param_value(latent_upscale_param, "spatial_w_overlap", overlap_px)))
|
||||||
|
spatial_h_overlap_px = max(0, int(_param_value(latent_upscale_param, "spatial_h_overlap", overlap_px)))
|
||||||
|
min_tile_size_px = max(0, int(_param_value(latent_upscale_param, "min_tile_size", 256)))
|
||||||
|
masked_area_noise = float(_param_value(latent_upscale_param, "masked_area_noise", 0.0))
|
||||||
|
brightness_match = bool(latent_upscale_param.get("brightness_match", False))
|
||||||
|
dynamic_fade = str(latent_upscale_param.get("dynamic_fade", "off"))
|
||||||
|
dynamic_fade_min_px = max(0, int(_param_value(latent_upscale_param, "dynamic_fade_min", 32)))
|
||||||
|
if tile_size_mode == "rows_cols":
|
||||||
|
tile_w_px, spatial_w_overlap_px = _solve_equal_tiles(target_w, grid_cols, spatial_w_overlap_px, 16)
|
||||||
|
tile_h_px, spatial_h_overlap_px = _solve_equal_tiles(target_h, grid_rows, spatial_h_overlap_px, 16)
|
||||||
|
if tile_w_px < min_tile_size_px or tile_h_px < min_tile_size_px:
|
||||||
|
raise ValueError(
|
||||||
|
f"rows_cols mode: solved tile size is {tile_h_px}x{tile_w_px}px "
|
||||||
|
f"(grid {grid_rows}x{grid_cols} over {target_h}x{target_w}px), "
|
||||||
|
f"which is smaller than min_tile_size ({min_tile_size_px}px). "
|
||||||
|
f"Reduce grid_rows/grid_cols, or lower min_tile_size to at most "
|
||||||
|
f"{min(tile_w_px, tile_h_px)}px."
|
||||||
|
)
|
||||||
|
fade_w_px = min(fade_w_px, spatial_w_overlap_px)
|
||||||
|
fade_h_px = min(fade_h_px, spatial_h_overlap_px)
|
||||||
|
else:
|
||||||
|
for name, value in (
|
||||||
|
("tile_width", tile_w_px),
|
||||||
|
("tile_height", tile_h_px),
|
||||||
|
("overlap", overlap_px),
|
||||||
|
("fade_width", fade_w_px),
|
||||||
|
("fade_height", fade_h_px),
|
||||||
|
("min_tile_size", min_tile_size_px),
|
||||||
|
):
|
||||||
|
if value % 32 != 0:
|
||||||
|
raise ValueError(f"'{name}' must be a multiple of 32 pixels; got {value}.")
|
||||||
|
if overlap_px >= tile_w_px:
|
||||||
|
raise ValueError("overlap must be smaller than tile_width")
|
||||||
|
if overlap_px >= tile_h_px:
|
||||||
|
raise ValueError("overlap must be smaller than tile_height")
|
||||||
|
if fade_w_px > spatial_w_overlap_px:
|
||||||
|
raise ValueError("fade_width must not exceed spatial_w_overlap")
|
||||||
|
if fade_h_px > spatial_h_overlap_px:
|
||||||
|
raise ValueError("fade_height must not exceed spatial_h_overlap")
|
||||||
|
if min_tile_size_px > tile_w_px or min_tile_size_px > tile_h_px:
|
||||||
|
raise ValueError("min_tile_size must not exceed the tile size")
|
||||||
|
tile_tw = max(1, tile_w_px // 16)
|
||||||
|
tile_th = max(1, tile_h_px // 16)
|
||||||
|
ol_tw = max(0, min(tile_tw - 1, spatial_w_overlap_px // 16))
|
||||||
|
ol_th = max(0, min(tile_th - 1, spatial_h_overlap_px // 16))
|
||||||
|
fw_tw = max(0, min(ol_tw, fade_w_px // 16))
|
||||||
|
fw_th = max(0, min(ol_th, fade_h_px // 16))
|
||||||
|
min_tile_tw = max(0, min_tile_size_px // 16)
|
||||||
|
rows, cols, trows, tcols, row_ovl, col_ovl = compute_spatial_grid(
|
||||||
|
int(up_h), int(up_w), tile_th, tile_tw, ol_th, ol_tw, min_tile_tw, min_tile_tw
|
||||||
|
)
|
||||||
|
logging.info(
|
||||||
|
"H3 latent refine: %s spatial sampler tiles, target=%sx%s tile_mode=%s tile=%sx%s overlap=%sx%s",
|
||||||
|
len(rows) * len(cols), target_w, target_h, tile_size_mode, tile_w_px, tile_h_px,
|
||||||
|
spatial_w_overlap_px, spatial_h_overlap_px,
|
||||||
|
)
|
||||||
|
if len(rows) == 1 and len(cols) == 1:
|
||||||
(refined_out,) = nodes.common_ksampler(
|
(refined_out,) = nodes.common_ksampler(
|
||||||
model, seed, int(detail_steps), cfg, detail_sampler_name, detail_scheduler,
|
model, seed, refine_steps, cfg, refine_sampler, refine_scheduler, upscale_cond, negative, upscale_latent,
|
||||||
positive, negative, detail_latent, denoise=float(detail_denoise))
|
denoise=refine_denoise)
|
||||||
timing["detail_sample"] += time.perf_counter() - detail_start
|
else:
|
||||||
|
refined_video = upscaled_video.clone()
|
||||||
|
for row_index, r0 in enumerate(rows):
|
||||||
|
tr = trows[row_index]
|
||||||
|
ovh = row_ovl[row_index]
|
||||||
|
for col_index, c0 in enumerate(cols):
|
||||||
|
tc = tcols[col_index]
|
||||||
|
ovw = col_ovl[col_index]
|
||||||
|
tile_cond = _crop_conditioning_to_tile(
|
||||||
|
upscale_cond, int(up_h), int(up_w), r0, c0, tr, tc
|
||||||
|
)
|
||||||
|
tile_latent = dict(upscale_latent) if isinstance(upscale_latent, dict) else {}
|
||||||
|
tile_video = upscaled_video[:, :, :, r0:r0 + tr, c0:c0 + tc].contiguous()
|
||||||
|
tr_s = tr + (tr % 2)
|
||||||
|
tc_s = tc + (tc % 2)
|
||||||
|
tile = torch.zeros((1, tile_video.shape[1], tile_video.shape[2], tr_s, tc_s),
|
||||||
|
device=tile_video.device, dtype=tile_video.dtype)
|
||||||
|
tile[:, :, :, :tr, :tc] = tile_video
|
||||||
|
if col_index > 0 and ovw > 0:
|
||||||
|
tile[:, :, :, :tr, :ovw] = refined_video[:, :, :, r0:r0 + tr, c0:c0 + ovw]
|
||||||
|
if row_index > 0 and ovh > 0:
|
||||||
|
tile[:, :, :, :ovh, :tc] = refined_video[:, :, :, r0:r0 + ovh, c0:c0 + tc]
|
||||||
|
mask = make_fade_mask(tr_s, tc_s, ovh, ovw, row_index > 0, col_index > 0,
|
||||||
|
fade_h=fw_th, fade_w=fw_tw)
|
||||||
|
mask[tr:tr_s, :] = 0.0
|
||||||
|
mask[:, tc:tc_s] = 0.0
|
||||||
|
mv = (mask + masked_area_noise * (1.0 - mask))[None, None, None].to(tile.dtype)
|
||||||
|
ma = torch.zeros((1, 32, 2, full_audio.shape[-1]), device=full_audio.device, dtype=full_audio.dtype)
|
||||||
|
tile_latent["samples"] = comfy.nested_tensor.NestedTensor((tile, full_audio))
|
||||||
|
tile_latent["noise_mask"] = comfy.nested_tensor.NestedTensor((mv, ma))
|
||||||
|
dynamic = _dynamic_fade_closure(
|
||||||
|
latent_upscale_param, fw_tw, fw_th, tr, tc, tr_s, tc_s, ovh, ovw,
|
||||||
|
row_index > 0, col_index > 0, math.prod(tile.shape[1:]), mn=masked_area_noise
|
||||||
|
)
|
||||||
|
if dynamic is not None:
|
||||||
|
model.set_model_denoise_mask_function(dynamic)
|
||||||
|
try:
|
||||||
|
tile_out, = nodes.common_ksampler(
|
||||||
|
model, seed, refine_steps, cfg, refine_sampler, refine_scheduler, tile_cond, negative, tile_latent,
|
||||||
|
denoise=refine_denoise)
|
||||||
|
finally:
|
||||||
|
if dynamic is not None:
|
||||||
|
model.model_options.pop("denoise_mask_function", None)
|
||||||
|
tile_out = _video_only_refined_latent(
|
||||||
|
{"samples": comfy.nested_tensor.NestedTensor((tile_video, full_audio))},
|
||||||
|
tile_out)
|
||||||
|
tile_video_out = tile_out["samples"].tensors[0]
|
||||||
|
if brightness_match:
|
||||||
|
tile_video_out = bright_match_tile(
|
||||||
|
tile_video_out,
|
||||||
|
upscaled_video[:, :, :, r0:r0 + tr, c0:c0 + tc]
|
||||||
|
)
|
||||||
|
region = refined_video[:, :, :, r0:r0 + tr, c0:c0 + tc]
|
||||||
|
base_region = region.clone()
|
||||||
|
region.copy_(tile_video_out)
|
||||||
|
if col_index > 0 and ol_tw > 0:
|
||||||
|
t = torch.linspace(0.0, 1.0, ol_tw, device=region.device, dtype=region.dtype)
|
||||||
|
w = _latent_spatial_blend_weights(t, overlap_mode, overlap_blend)
|
||||||
|
if fw_tw > 0:
|
||||||
|
w = w.clone()
|
||||||
|
w[:fw_tw] = 0.0
|
||||||
|
region[:, :, :, :, :ol_tw] = (
|
||||||
|
base_region[:, :, :, :, :ol_tw] * (1.0 - w[None, None, None, None, :]) +
|
||||||
|
tile_video_out[:, :, :, :, :ol_tw] * w[None, None, None, None, :]
|
||||||
|
)
|
||||||
|
if row_index > 0 and ol_th > 0:
|
||||||
|
t = torch.linspace(0.0, 1.0, ol_th, device=region.device, dtype=region.dtype)
|
||||||
|
w = _latent_spatial_blend_weights(t, overlap_mode, overlap_blend)
|
||||||
|
if fw_th > 0:
|
||||||
|
w = w.clone()
|
||||||
|
w[:fw_th] = 0.0
|
||||||
|
region[:, :, :, :ol_th, :] = (
|
||||||
|
base_region[:, :, :, :ol_th, :] * (1.0 - w[None, None, None, :, None]) +
|
||||||
|
tile_video_out[:, :, :, :ol_th, :] * w[None, None, None, :, None]
|
||||||
|
)
|
||||||
|
refined_out = {"samples": comfy.nested_tensor.NestedTensor((refined_video, full_audio))}
|
||||||
|
timing["latent_upscale_sample"] += time.perf_counter() - latent_start
|
||||||
|
refined_out = _video_only_refined_latent(upscale_latent, refined_out)
|
||||||
|
del upscale_latent, upscaled_video, full_audio
|
||||||
|
mm.soft_empty_cache()
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
if _is_oom(e):
|
raise _tag_oom_stage(e, "latent_upscale")
|
||||||
e._h3_stage = "sampling"
|
|
||||||
raise
|
|
||||||
refined_out = _video_only_refined_latent(out, refined_out)
|
|
||||||
del detail_latent
|
|
||||||
# Keep a CPU copy of the sampled latent BEFORE decoding, for the `latent`
|
# Keep a CPU copy of the sampled latent BEFORE decoding, for the `latent`
|
||||||
# output. Latents are ~1000x smaller than the frames they decode to (a
|
# output. Latents are ~1000x smaller than the frames they decode to (a
|
||||||
# 1344x768 124f shot is ~1.5MB against ~1.5GB), so carrying one per shot for
|
# 1344x768 124f shot is ~1.5MB against ~1.5GB), so carrying one per shot for
|
||||||
# the whole chain is free. Detached and moved off the card immediately, for
|
# the whole chain is free. Detached and moved off the card immediately, for
|
||||||
# the same reason the decoded frames are.
|
# the same reason the decoded frames are.
|
||||||
decode_video_start = time.perf_counter()
|
|
||||||
shot_latent = _copy_sample_latent(refined_out)
|
|
||||||
decode_audio_start = time.perf_counter()
|
decode_audio_start = time.perf_counter()
|
||||||
audio = _decode_audio(audio_vae, out)
|
audio = _decode_audio(audio_vae, out)
|
||||||
timing["decode_audio"] += time.perf_counter() - decode_audio_start
|
timing["decode_audio"] += time.perf_counter() - decode_audio_start
|
||||||
# Audio is much smaller than the video decode. Drop the first-pass
|
decode_video_start = time.perf_counter()
|
||||||
# conditioning before the VAE work so the optional detail pass does not
|
shot_latent = _copy_sample_latent(refined_out)
|
||||||
# keep both sampled latents resident across the heaviest allocation.
|
|
||||||
del out, positive, latent
|
|
||||||
video = _decode_video(vae, refined_out, tiled, free_first=model,
|
video = _decode_video(vae, refined_out, tiled, free_first=model,
|
||||||
tile_t=decode_tile_frames, tile_xy=decode_tile_size)
|
tile_t=decode_tile_frames, tile_xy=decode_tile_size)
|
||||||
timing["decode_video"] += time.perf_counter() - decode_video_start
|
timing["decode_video"] += time.perf_counter() - decode_video_start
|
||||||
cleanup_start = time.perf_counter()
|
cleanup_start = time.perf_counter()
|
||||||
|
del out
|
||||||
del refined_out
|
del refined_out
|
||||||
_deep_cleanup()
|
_deep_cleanup()
|
||||||
timing["cleanup"] += time.perf_counter() - cleanup_start
|
timing["cleanup"] += time.perf_counter() - cleanup_start
|
||||||
@@ -6500,8 +6998,7 @@ class H3LongVideos:
|
|||||||
ref_5=None, ref_6=None, ref_7=None, ref_8=None,
|
ref_5=None, ref_6=None, ref_7=None, ref_8=None,
|
||||||
ref_9=None,
|
ref_9=None,
|
||||||
ref_mode="auto ref2v", ref_image_size="match", ref_noise_aug=0.95,
|
ref_mode="auto ref2v", ref_image_size="match", ref_noise_aug=0.95,
|
||||||
detail_pass=False, detail_sampler_name="euler", detail_scheduler="beta",
|
latent_upscale_param=None,
|
||||||
detail_steps=8, detail_denoise=0.4,
|
|
||||||
graph=None, node_id=None):
|
graph=None, node_id=None):
|
||||||
|
|
||||||
# FIRST: detect a checkpoint swap since the previous execution and hard-flush.
|
# FIRST: detect a checkpoint swap since the previous execution and hard-flush.
|
||||||
@@ -6565,12 +7062,42 @@ class H3LongVideos:
|
|||||||
ms_note = ""
|
ms_note = ""
|
||||||
if apply_model_sampling:
|
if apply_model_sampling:
|
||||||
model, ms_note = apply_h3_model_sampling(model, shift_video, shift_audio)
|
model, ms_note = apply_h3_model_sampling(model, shift_video, shift_audio)
|
||||||
detail_note = ""
|
latent_upscale_note = ""
|
||||||
if detail_pass:
|
latent_upscale_mode = _latent_upscale_mode(latent_upscale_param)
|
||||||
detail_note = (f" detail pass: {int(detail_steps)} step(s) via "
|
if latent_upscale_mode != "off":
|
||||||
f"{detail_sampler_name}/{detail_scheduler} at denoise "
|
target_w, target_h = _latent_upscale_target_size(w, h, latent_upscale_param)
|
||||||
f"{float(detail_denoise):.2f}; video-only refinement keeps "
|
mode = latent_upscale_mode
|
||||||
f"audio from the first pass")
|
detail = f" via {mode}"
|
||||||
|
if mode == "model":
|
||||||
|
detail += f"/{latent_upscale_param.get('model_name', 'none')}"
|
||||||
|
else:
|
||||||
|
detail += f"/{latent_upscale_param.get('method', 'bilinear')}"
|
||||||
|
denoise_value = latent_upscale_param.get("denoise", latent_upscale_param.get("refine_denoise", 0.2))
|
||||||
|
denoise = 0.2 if denoise_value is None else float(denoise_value)
|
||||||
|
refine_sampler = latent_upscale_param.get("sampler_name", "euler_ancestral")
|
||||||
|
refine_scheduler = latent_upscale_param.get("scheduler", "simple")
|
||||||
|
batch_note = ""
|
||||||
|
tile_size_mode = str(latent_upscale_param.get("tile_size_mode", "specific_size"))
|
||||||
|
tile_w_px = int(latent_upscale_param.get("tile_width", 512) or 512)
|
||||||
|
tile_h_px = int(latent_upscale_param.get("tile_height", 512) or 512)
|
||||||
|
overlap_px = max(0, int(_param_value(latent_upscale_param, "overlap", 64)))
|
||||||
|
overlap_blend = str(latent_upscale_param.get("overlap_blend", "linear"))
|
||||||
|
grid_rows = max(1, int(_param_value(latent_upscale_param, "grid_rows", 2)))
|
||||||
|
grid_cols = max(1, int(_param_value(latent_upscale_param, "grid_cols", 2)))
|
||||||
|
if tile_size_mode == "rows_cols":
|
||||||
|
batch_note = f"; spatial batches {grid_rows}x{grid_cols} rows_cols over {target_w}x{target_h}px"
|
||||||
|
elif tile_w_px > 0 and tile_h_px > 0 and (tile_w_px < target_w or tile_h_px < target_h):
|
||||||
|
spatial_w_overlap_px = max(0, int(_param_value(latent_upscale_param, "spatial_w_overlap", overlap_px)))
|
||||||
|
spatial_h_overlap_px = max(0, int(_param_value(latent_upscale_param, "spatial_h_overlap", overlap_px)))
|
||||||
|
batch_note = (
|
||||||
|
f"; spatial batches {tile_w_px}x{tile_h_px}px "
|
||||||
|
f"overlap {spatial_w_overlap_px}x{spatial_h_overlap_px}px {overlap_blend}"
|
||||||
|
)
|
||||||
|
latent_upscale_note = (
|
||||||
|
f" latent upscale: target {target_w}x{target_h}px{detail}; "
|
||||||
|
f"{int(latent_upscale_param.get('steps', 2) or 2)}-step refinement "
|
||||||
|
f"{refine_sampler}/{refine_scheduler} denoise {denoise:.2f}{batch_note}"
|
||||||
|
)
|
||||||
|
|
||||||
paras = split_paragraphs(prompt, "##")
|
paras = split_paragraphs(prompt, "##")
|
||||||
if anchor_override.strip():
|
if anchor_override.strip():
|
||||||
@@ -6881,6 +7408,7 @@ class H3LongVideos:
|
|||||||
+ (" ANCHOR: " + "; ".join(anchor_hazards) + "."
|
+ (" ANCHOR: " + "; ".join(anchor_hazards) + "."
|
||||||
if anchor_hazards else "")
|
if anchor_hazards else "")
|
||||||
+ (f"{anatomy_note}." if anatomy_note else "")
|
+ (f"{anatomy_note}." if anatomy_note else "")
|
||||||
|
+ (f"{latent_upscale_note}." if latent_upscale_note else "")
|
||||||
+ (f"{plan_audio}." if plan_audio else "")
|
+ (f"{plan_audio}." if plan_audio else "")
|
||||||
+ (" EXPOSURE -- " + "; ".join(wardrobe_notes) + "."
|
+ (" EXPOSURE -- " + "; ".join(wardrobe_notes) + "."
|
||||||
if wardrobe_notes else "")
|
if wardrobe_notes else "")
|
||||||
@@ -7029,13 +7557,18 @@ class H3LongVideos:
|
|||||||
model, clip, vae, audio_vae, negative, gen_prompt, w, h, ln_i, fps,
|
model, clip, vae, audio_vae, negative, gen_prompt, w, h, ln_i, fps,
|
||||||
tiled, sa, shot_handoff, decode_tile_frames, decode_tile_size,
|
tiled, sa, shot_handoff, decode_tile_frames, decode_tile_size,
|
||||||
shot_refs, ref_image_size, shot_aug, shot_silent,
|
shot_refs, ref_image_size, shot_aug, shot_silent,
|
||||||
detail_pass, detail_sampler_name, detail_scheduler,
|
latent_upscale_param=latent_upscale_param, timing_sink=shot_timing)
|
||||||
detail_steps, detail_denoise, timing_sink=shot_timing)
|
|
||||||
break
|
break
|
||||||
except (torch.cuda.OutOfMemoryError, RuntimeError) as e:
|
except (torch.cuda.OutOfMemoryError, RuntimeError) as e:
|
||||||
shot_retry_elapsed += time.perf_counter() - attempt_start
|
shot_retry_elapsed += time.perf_counter() - attempt_start
|
||||||
if not _is_oom(e):
|
if not _is_oom(e):
|
||||||
raise
|
raise
|
||||||
|
if getattr(e, "_h3_stage", "") == "latent_upscale":
|
||||||
|
raise RuntimeError(
|
||||||
|
f"H3 Long Videos: shot {i + 1} of {len(gens)} ran out of VRAM "
|
||||||
|
f"during latent upscale. The latent-upscale stage is too large "
|
||||||
|
f"for this card at {w}x{h} with the current spatial settings."
|
||||||
|
) from e
|
||||||
mm.soft_empty_cache(True)
|
mm.soft_empty_cache(True)
|
||||||
if not tiled:
|
if not tiled:
|
||||||
tiled = True; backoff.append("tiled decode")
|
tiled = True; backoff.append("tiled decode")
|
||||||
@@ -7052,10 +7585,16 @@ class H3LongVideos:
|
|||||||
model, clip, vae, audio_vae, negative, gen_prompt, w, h, ln_i, fps,
|
model, clip, vae, audio_vae, negative, gen_prompt, w, h, ln_i, fps,
|
||||||
tiled, sa, shot_handoff, decode_tile_frames, decode_tile_size,
|
tiled, sa, shot_handoff, decode_tile_frames, decode_tile_size,
|
||||||
shot_refs, ref_image_size, shot_aug, shot_silent,
|
shot_refs, ref_image_size, shot_aug, shot_silent,
|
||||||
detail_pass, detail_sampler_name, detail_scheduler,
|
latent_upscale_param=latent_upscale_param, timing_sink=shot_timing)
|
||||||
detail_steps, detail_denoise, timing_sink=shot_timing)
|
|
||||||
except (torch.cuda.OutOfMemoryError, RuntimeError) as e:
|
except (torch.cuda.OutOfMemoryError, RuntimeError) as e:
|
||||||
shot_retry_elapsed += time.perf_counter() - attempt_start
|
shot_retry_elapsed += time.perf_counter() - attempt_start
|
||||||
|
stage = getattr(e, "_h3_stage", "")
|
||||||
|
if _is_oom(e) and stage == "latent_upscale":
|
||||||
|
raise RuntimeError(
|
||||||
|
f"H3 Long Videos: shot {i + 1} of {len(gens)} ran out of VRAM "
|
||||||
|
f"during latent upscale. The latent-upscale stage is too large "
|
||||||
|
f"for this card at {w}x{h} with the current spatial settings."
|
||||||
|
) from e
|
||||||
if _is_oom(e) and getattr(e, "_h3_stage", "") == "sampling":
|
if _is_oom(e) and getattr(e, "_h3_stage", "") == "sampling":
|
||||||
# Retrying with tiles would re-run the whole sampling pass and
|
# Retrying with tiles would re-run the whole sampling pass and
|
||||||
# fail identically. Fail now, and say what actually shrinks it.
|
# fail identically. Fail now, and say what actually shrinks it.
|
||||||
@@ -7072,8 +7611,7 @@ class H3LongVideos:
|
|||||||
model, clip, vae, audio_vae, negative, gen_prompt, w, h, ln_i, fps,
|
model, clip, vae, audio_vae, negative, gen_prompt, w, h, ln_i, fps,
|
||||||
tiled, sa, shot_handoff, decode_tile_frames, decode_tile_size,
|
tiled, sa, shot_handoff, decode_tile_frames, decode_tile_size,
|
||||||
shot_refs, ref_image_size, shot_aug, shot_silent,
|
shot_refs, ref_image_size, shot_aug, shot_silent,
|
||||||
detail_pass, detail_sampler_name, detail_scheduler,
|
latent_upscale_param=latent_upscale_param, timing_sink=shot_timing)
|
||||||
detail_steps, detail_denoise, timing_sink=shot_timing)
|
|
||||||
shot_retry_elapsed += time.perf_counter() - attempt_start
|
shot_retry_elapsed += time.perf_counter() - attempt_start
|
||||||
|
|
||||||
shot_total = time.perf_counter() - shot_total_start
|
shot_total = time.perf_counter() - shot_total_start
|
||||||
@@ -7343,7 +7881,7 @@ class H3LongVideos:
|
|||||||
f"shot before them ended on dialogue." if mouth_settled else "")
|
f"shot before them ended on dialogue." if mouth_settled else "")
|
||||||
+ (f"{anatomy_note}." if anatomy_note else "")
|
+ (f"{anatomy_note}." if anatomy_note else "")
|
||||||
+ (f"{latent_note}." if latent_note else "")
|
+ (f"{latent_note}." if latent_note else "")
|
||||||
+ (f"{detail_note}." if detail_note else "")
|
+ (f"{latent_upscale_note}." if latent_upscale_note else "")
|
||||||
+ (f" SLA LoRA '{os.path.basename(str(sla_name))}' paired with sparse attention."
|
+ (f" SLA LoRA '{os.path.basename(str(sla_name))}' paired with sparse attention."
|
||||||
if sla_name and sparse_on else "")
|
if sla_name and sparse_on else "")
|
||||||
+ (f" {beats_note}." if beats_note else "")
|
+ (f" {beats_note}." if beats_note else "")
|
||||||
|
|||||||
@@ -39,10 +39,11 @@ class H3ShotLength:
|
|||||||
def INPUT_TYPES(cls):
|
def INPUT_TYPES(cls):
|
||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"shot_seconds": ("FLOAT", {"default": 5.0, "min": 0.2, "max": 15.1, "step": 0.5,
|
"shot_seconds": ("FLOAT", {"default": 3.0, "min": 0.2, "max": 15.1, "step": 0.5,
|
||||||
"tooltip": "Length of each shot. Feeds the sampler's shot_seconds AND (as frames) "
|
"tooltip": "Length of each shot. Feeds the sampler's shot_seconds AND (as frames) "
|
||||||
"the preview override. Max ~15s (362 frames)."}),
|
"the preview override. Default 3s matches the common one-beat H3 test shot. Max ~15s (362 frames)."}),
|
||||||
"fps": ("INT", {"default": 24, "min": 1, "max": 60}),
|
"fps": ("INT", {"default": 24, "min": 1, "max": 60,
|
||||||
|
"tooltip": "Frame rate used for the seconds->frames conversion. H3 itself renders at 24fps, so 24 is the realistic default."}),
|
||||||
},
|
},
|
||||||
"optional": {
|
"optional": {
|
||||||
"cap_to_h3_max": ("BOOLEAN", {"default": True,
|
"cap_to_h3_max": ("BOOLEAN", {"default": True,
|
||||||
|
|||||||
+50
-29
@@ -277,7 +277,14 @@ class DumasJSONStringToObjectNode:
|
|||||||
def INPUT_TYPES(cls):
|
def INPUT_TYPES(cls):
|
||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"json_string": ("STRING", {"multiline": True}),
|
"json_string": (
|
||||||
|
"STRING",
|
||||||
|
{
|
||||||
|
"multiline": True,
|
||||||
|
"default": '{\n "shots": [\n {\n "prompt": "Francine stands by the window."\n }\n ]\n}',
|
||||||
|
"tooltip": "Raw JSON text to parse into a structured JSON object."
|
||||||
|
},
|
||||||
|
),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -299,7 +306,14 @@ class DumasStripIterationSuffixNode:
|
|||||||
def INPUT_TYPES(cls):
|
def INPUT_TYPES(cls):
|
||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"filename": ("STRING", {"default": "", "multiline": False}),
|
"filename": (
|
||||||
|
"STRING",
|
||||||
|
{
|
||||||
|
"default": "francine_pose_final.png",
|
||||||
|
"multiline": False,
|
||||||
|
"tooltip": "Filename to normalize by removing everything after the first underscore in the stem."
|
||||||
|
},
|
||||||
|
),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -316,7 +330,14 @@ class DumasSlugifyStringNode:
|
|||||||
def INPUT_TYPES(cls):
|
def INPUT_TYPES(cls):
|
||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"text": ("STRING", {"default": "", "multiline": False}),
|
"text": (
|
||||||
|
"STRING",
|
||||||
|
{
|
||||||
|
"default": "Francine Coffee Shop",
|
||||||
|
"multiline": False,
|
||||||
|
"tooltip": "Text to slugify into lowercase ASCII words joined with hyphens."
|
||||||
|
},
|
||||||
|
),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -334,8 +355,8 @@ class DumasJSONObjectToStringNode:
|
|||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"json_object": ("JSON",),
|
"json_object": ("JSON",),
|
||||||
"pretty": ("BOOLEAN", {"default": True}),
|
"pretty": ("BOOLEAN", {"default": True, "tooltip": "Pretty-print the JSON with indentation."}),
|
||||||
"sort_keys": ("BOOLEAN", {"default": False}),
|
"sort_keys": ("BOOLEAN", {"default": False, "tooltip": "Sort object keys alphabetically before serializing."}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -355,7 +376,7 @@ class DumasJSONGetValueNode:
|
|||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"json_object": ("JSON",),
|
"json_object": ("JSON",),
|
||||||
"path": ("STRING", {"default": "", "multiline": False}),
|
"path": ("STRING", {"default": "shots.0.prompt", "multiline": False, "tooltip": "Dot-path to read, such as 'shots.0.prompt'."}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -373,8 +394,8 @@ class DumasJSONSetValueNode:
|
|||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"json_object": ("JSON",),
|
"json_object": ("JSON",),
|
||||||
"path": ("STRING", {"default": "", "multiline": False}),
|
"path": ("STRING", {"default": "shots.0.prompt", "multiline": False, "tooltip": "Dot-path to write, such as 'shots.0.prompt' or 'shots.1.duration'."}),
|
||||||
"value_json": ("STRING", {"multiline": True, "default": "null"}),
|
"value_json": ("STRING", {"multiline": True, "default": '"Francine stands by the window."', "tooltip": "JSON value to store at the path. Must be valid JSON, so strings need quotes."}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -398,7 +419,7 @@ class DumasJSONHasKeyNode:
|
|||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"json_object": ("JSON",),
|
"json_object": ("JSON",),
|
||||||
"path": ("STRING", {"default": "", "multiline": False}),
|
"path": ("STRING", {"default": "shots.0.prompt", "multiline": False, "tooltip": "Dot-path to test for existence."}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -416,7 +437,7 @@ class DumasJSONRemoveKeyNode:
|
|||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"json_object": ("JSON",),
|
"json_object": ("JSON",),
|
||||||
"path": ("STRING", {"default": "", "multiline": False}),
|
"path": ("STRING", {"default": "shots.0.prompt", "multiline": False, "tooltip": "Dot-path to remove from the object."}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -434,7 +455,7 @@ class DumasJSONPickFieldsNode:
|
|||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"json_object": ("JSON",),
|
"json_object": ("JSON",),
|
||||||
"paths": ("STRING", {"multiline": True, "default": ""}),
|
"paths": ("STRING", {"multiline": True, "default": "shots.0.prompt\nshots.0.duration", "tooltip": "One dot-path per line. Only those fields are copied into the output object."}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -463,8 +484,8 @@ class DumasJSONMergeObjectsNode:
|
|||||||
def INPUT_TYPES(cls):
|
def INPUT_TYPES(cls):
|
||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"base_object": ("JSON",),
|
"base_object": ("JSON", {"tooltip": "Base JSON object to start from."}),
|
||||||
"overlay_object": ("JSON",),
|
"overlay_object": ("JSON", {"tooltip": "Overlay JSON object whose keys replace or merge into the base object."}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -482,7 +503,7 @@ class DumasJSONKeysNode:
|
|||||||
def INPUT_TYPES(cls):
|
def INPUT_TYPES(cls):
|
||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"json_object": ("JSON",),
|
"json_object": ("JSON", {"tooltip": "JSON object whose top-level keys should be listed."}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -502,7 +523,7 @@ class DumasJSONArrayLengthNode:
|
|||||||
def INPUT_TYPES(cls):
|
def INPUT_TYPES(cls):
|
||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"json_array": ("JSON",),
|
"json_array": ("JSON", {"tooltip": "JSON array whose length should be measured."}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -521,8 +542,8 @@ class DumasJSONArrayAppendNode:
|
|||||||
def INPUT_TYPES(cls):
|
def INPUT_TYPES(cls):
|
||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"json_array": ("JSON",),
|
"json_array": ("JSON", {"tooltip": "JSON array to append to."}),
|
||||||
"value_json": ("STRING", {"multiline": True, "default": "null"}),
|
"value_json": ("STRING", {"multiline": True, "default": '{"prompt":"Francine looks toward the door."}', "tooltip": "JSON value to append. Must be valid JSON."}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -548,10 +569,10 @@ class DumasJSONArraySliceNode:
|
|||||||
def INPUT_TYPES(cls):
|
def INPUT_TYPES(cls):
|
||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"json_array": ("JSON",),
|
"json_array": ("JSON", {"tooltip": "JSON array to slice."}),
|
||||||
"start": ("INT", {"default": 0, "step": 1}),
|
"start": ("INT", {"default": 0, "step": 1, "tooltip": "Zero-based start index."}),
|
||||||
"end": ("INT", {"default": 0, "step": 1}),
|
"end": ("INT", {"default": 0, "step": 1, "tooltip": "Zero-based end index. Use 0 to mean 'to the end'."}),
|
||||||
"step": ("INT", {"default": 1, "step": 1, "min": 1}),
|
"step": ("INT", {"default": 1, "step": 1, "min": 1, "tooltip": "Slice step size."}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -572,9 +593,9 @@ class DumasJSONArrayIteratorNode:
|
|||||||
def INPUT_TYPES(cls):
|
def INPUT_TYPES(cls):
|
||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"json_input": ("JSON",),
|
"json_input": ("JSON", {"tooltip": "JSON array to iterate over."}),
|
||||||
"index": ("INT", {"default": 0, "min": 0, "step": 1}),
|
"index": ("INT", {"default": 0, "min": 0, "step": 1, "tooltip": "Current zero-based index."}),
|
||||||
"mode": (["fixed", "incr", "decr"], {"default": "fixed"}),
|
"mode": (["fixed", "incr", "decr"], {"default": "fixed", "tooltip": "Keep the index fixed, increment it, or decrement it before reading."}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -600,9 +621,9 @@ class DumasJSONObjectIteratorNode:
|
|||||||
def INPUT_TYPES(cls):
|
def INPUT_TYPES(cls):
|
||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"json_input": ("JSON",),
|
"json_input": ("JSON", {"tooltip": "JSON object whose key/value pairs should be iterated in insertion order."}),
|
||||||
"index": ("INT", {"default": 0, "min": 0, "step": 1}),
|
"index": ("INT", {"default": 0, "min": 0, "step": 1, "tooltip": "Current zero-based index into the object's items."}),
|
||||||
"mode": (["fixed", "incr", "decr"], {"default": "fixed"}),
|
"mode": (["fixed", "incr", "decr"], {"default": "fixed", "tooltip": "Keep the index fixed, increment it, or decrement it before reading."}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -628,7 +649,7 @@ class DumasJSONFlattenNode:
|
|||||||
def INPUT_TYPES(cls):
|
def INPUT_TYPES(cls):
|
||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"json_input": ("JSON",),
|
"json_input": ("JSON", {"tooltip": "Nested JSON value to flatten into dot-path keys."}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -645,7 +666,7 @@ class DumasJSONUnflattenNode:
|
|||||||
def INPUT_TYPES(cls):
|
def INPUT_TYPES(cls):
|
||||||
return {
|
return {
|
||||||
"required": {
|
"required": {
|
||||||
"flat_json_object": ("JSON",),
|
"flat_json_object": ("JSON", {"tooltip": "Flat JSON object whose keys are dot-paths to rebuild into nested JSON."}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -53,11 +53,10 @@ const GROUPS = [
|
|||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: "finish",
|
id: "finish",
|
||||||
label: "Upscale/Detail",
|
label: "Upscale",
|
||||||
defaultCollapsed: true,
|
defaultCollapsed: true,
|
||||||
widgets: [
|
widgets: [
|
||||||
"upscale", "upscale_model", "upscale_target_short_edge", "upscale_batch",
|
"upscale", "upscale_model", "upscale_target_short_edge", "upscale_batch",
|
||||||
"detail_pass", "detail_sampler_name", "detail_scheduler", "detail_steps", "detail_denoise",
|
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -24,6 +24,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
"PIL.Image",
|
"PIL.Image",
|
||||||
"folder_paths",
|
"folder_paths",
|
||||||
"dumas_image_nodes",
|
"dumas_image_nodes",
|
||||||
|
"dumas_h3_latent_upscale",
|
||||||
"dumas_h3_longvideos",
|
"dumas_h3_longvideos",
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
@@ -207,7 +208,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
"retry_elapsed": 1.2,
|
"retry_elapsed": 1.2,
|
||||||
"attempts": 2,
|
"attempts": 2,
|
||||||
"sample": 8.0,
|
"sample": 8.0,
|
||||||
"detail_sample": 0.5,
|
"latent_upscale_sample": 0.5,
|
||||||
"decode_video": 2.1,
|
"decode_video": 2.1,
|
||||||
"decode_audio": 0.4,
|
"decode_audio": 0.4,
|
||||||
"cleanup": 0.2,
|
"cleanup": 0.2,
|
||||||
@@ -230,11 +231,11 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
self.assertIn("decode audio 0.7s", note)
|
self.assertIn("decode audio 0.7s", note)
|
||||||
self.assertIn("cleanup 0.3s", note)
|
self.assertIn("cleanup 0.3s", note)
|
||||||
self.assertIn("retry elapsed 1.2s", note)
|
self.assertIn("retry elapsed 1.2s", note)
|
||||||
self.assertIn("detail 0.5s", note)
|
self.assertIn("latent upscale 0.5s", note)
|
||||||
self.assertIn("retries 1", note)
|
self.assertIn("retries 1", note)
|
||||||
self.assertIn("slowest shot 1 12.4s", note)
|
self.assertIn("slowest shot 1 12.4s", note)
|
||||||
|
|
||||||
def test_detail_pass_refines_video_but_preserves_audio(self):
|
def test_latent_upscale_refines_video_but_preserves_audio(self):
|
||||||
class FakeTensor:
|
class FakeTensor:
|
||||||
def __init__(self, name):
|
def __init__(self, name):
|
||||||
self.name = name
|
self.name = name
|
||||||
@@ -263,6 +264,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
original_decode_video = self.module._decode_video
|
original_decode_video = self.module._decode_video
|
||||||
original_decode_audio = self.module._decode_audio
|
original_decode_audio = self.module._decode_audio
|
||||||
original_cleanup = self.module._deep_cleanup
|
original_cleanup = self.module._deep_cleanup
|
||||||
|
original_upscale = self.module._upscale_latent_video
|
||||||
|
original_copy_sample = self.module._copy_sample_latent
|
||||||
original_nested = getattr(self.module.comfy.nested_tensor, "NestedTensor", None)
|
original_nested = getattr(self.module.comfy.nested_tensor, "NestedTensor", None)
|
||||||
try:
|
try:
|
||||||
self.module.comfy.nested_tensor.NestedTensor = FakeNestedTensor
|
self.module.comfy.nested_tensor.NestedTensor = FakeNestedTensor
|
||||||
@@ -277,6 +280,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
{"samples": FakeNestedTensor((FakeTensor("basev"), FakeTensor("basea")))},
|
{"samples": FakeNestedTensor((FakeTensor("basev"), FakeTensor("basea")))},
|
||||||
)
|
)
|
||||||
self.module._evict_all_but = lambda *_args, **_kwargs: None
|
self.module._evict_all_but = lambda *_args, **_kwargs: None
|
||||||
|
self.module._upscale_latent_video = lambda video, param: (FakeTensor("upv"), 8, 16)
|
||||||
|
self.module._copy_sample_latent = lambda sampled: sampled["samples"].unbind()
|
||||||
self.module._decode_video = lambda _vae, out_latent, *_args, **_kwargs: out_latent
|
self.module._decode_video = lambda _vae, out_latent, *_args, **_kwargs: out_latent
|
||||||
self.module._decode_audio = lambda _vae, out_latent: out_latent
|
self.module._decode_audio = lambda _vae, out_latent: out_latent
|
||||||
self.module._deep_cleanup = lambda: None
|
self.module._deep_cleanup = lambda: None
|
||||||
@@ -298,18 +303,25 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
tiled=False,
|
tiled=False,
|
||||||
sa=(123, 20, 1.0, "res_multistep", "simple", 1.0),
|
sa=(123, 20, 1.0, "res_multistep", "simple", 1.0),
|
||||||
handoff=None,
|
handoff=None,
|
||||||
detail_pass=True,
|
latent_upscale_param={
|
||||||
detail_sampler_name="euler",
|
"mode": "model",
|
||||||
detail_scheduler="beta",
|
"model_name": "upscale.safetensors",
|
||||||
detail_steps=5,
|
"device": "cpu",
|
||||||
detail_denoise=0.4,
|
"precision": "fp16",
|
||||||
|
"sampler_name": "euler_ancestral",
|
||||||
|
"scheduler": "simple",
|
||||||
|
"steps": 2,
|
||||||
|
"denoise": 0.4,
|
||||||
|
"megapixels": 1.5,
|
||||||
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
self.assertEqual(len(calls), 2)
|
self.assertEqual(len(calls), 2)
|
||||||
self.assertIsNot(calls[1][0][8], first_out)
|
self.assertIsNot(calls[1][0][8], first_out)
|
||||||
self.assertIs(calls[1][0][8]["samples"], first_out["samples"])
|
self.assertEqual(calls[1][0][8]["samples"].unbind()[0].name, "upv")
|
||||||
self.assertEqual(calls[1][0][4], "euler")
|
self.assertEqual(calls[1][0][2], 2)
|
||||||
self.assertEqual(calls[1][0][5], "beta")
|
self.assertEqual(calls[1][0][4], "euler_ancestral")
|
||||||
|
self.assertEqual(calls[1][0][5], "simple")
|
||||||
self.assertAlmostEqual(calls[1][1]["denoise"], 0.4)
|
self.assertAlmostEqual(calls[1][1]["denoise"], 0.4)
|
||||||
self.assertEqual(result[1], first_out)
|
self.assertEqual(result[1], first_out)
|
||||||
self.assertEqual(result[2][0].name, "v2")
|
self.assertEqual(result[2][0].name, "v2")
|
||||||
@@ -323,12 +335,14 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
self.module._decode_video = original_decode_video
|
self.module._decode_video = original_decode_video
|
||||||
self.module._decode_audio = original_decode_audio
|
self.module._decode_audio = original_decode_audio
|
||||||
self.module._deep_cleanup = original_cleanup
|
self.module._deep_cleanup = original_cleanup
|
||||||
|
self.module._upscale_latent_video = original_upscale
|
||||||
|
self.module._copy_sample_latent = original_copy_sample
|
||||||
if original_nested is None:
|
if original_nested is None:
|
||||||
delattr(self.module.comfy.nested_tensor, "NestedTensor")
|
delattr(self.module.comfy.nested_tensor, "NestedTensor")
|
||||||
else:
|
else:
|
||||||
self.module.comfy.nested_tensor.NestedTensor = original_nested
|
self.module.comfy.nested_tensor.NestedTensor = original_nested
|
||||||
|
|
||||||
def test_detail_pass_decodes_audio_before_video_and_cleans_up(self):
|
def test_latent_upscale_decodes_audio_before_video_and_cleans_up(self):
|
||||||
class FakeTensor:
|
class FakeTensor:
|
||||||
def __init__(self, name):
|
def __init__(self, name):
|
||||||
self.name = name
|
self.name = name
|
||||||
@@ -348,6 +362,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
return self._parts
|
return self._parts
|
||||||
|
|
||||||
order = []
|
order = []
|
||||||
|
build_calls = []
|
||||||
first_out = {"samples": FakeNestedTensor((FakeTensor("v1"), FakeTensor("a1")))}
|
first_out = {"samples": FakeNestedTensor((FakeTensor("v1"), FakeTensor("a1")))}
|
||||||
second_out = {"samples": FakeNestedTensor((FakeTensor("v2"), FakeTensor("a2")))}
|
second_out = {"samples": FakeNestedTensor((FakeTensor("v2"), FakeTensor("a2")))}
|
||||||
|
|
||||||
@@ -357,12 +372,16 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
original_decode_video = self.module._decode_video
|
original_decode_video = self.module._decode_video
|
||||||
original_decode_audio = self.module._decode_audio
|
original_decode_audio = self.module._decode_audio
|
||||||
original_cleanup = self.module._deep_cleanup
|
original_cleanup = self.module._deep_cleanup
|
||||||
|
original_upscale = self.module._upscale_latent_video
|
||||||
|
original_copy_sample = self.module._copy_sample_latent
|
||||||
|
original_unload = getattr(self.module.mm, "unload_model_and_clones", None)
|
||||||
|
original_unload_all = self.module.mm.unload_all_models
|
||||||
original_nested = getattr(self.module.comfy.nested_tensor, "NestedTensor", None)
|
original_nested = getattr(self.module.comfy.nested_tensor, "NestedTensor", None)
|
||||||
try:
|
try:
|
||||||
self.module.comfy.nested_tensor.NestedTensor = FakeNestedTensor
|
self.module.comfy.nested_tensor.NestedTensor = FakeNestedTensor
|
||||||
|
|
||||||
def common_ksampler(*args, **kwargs):
|
def common_ksampler(*args, **kwargs):
|
||||||
order.append("detail_sample" if len(order) else "sample")
|
order.append("latent_upscale_sample" if len(order) else "sample")
|
||||||
return (first_out if len([x for x in order if x.endswith("sample")]) == 1 else second_out,)
|
return (first_out if len([x for x in order if x.endswith("sample")]) == 1 else second_out,)
|
||||||
|
|
||||||
def decode_audio(_vae, out_latent):
|
def decode_audio(_vae, out_latent):
|
||||||
@@ -379,12 +398,31 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
def cleanup():
|
def cleanup():
|
||||||
order.append("cleanup")
|
order.append("cleanup")
|
||||||
|
|
||||||
self.module.nodes.common_ksampler = common_ksampler
|
def unload_model_and_clones(*_args, **_kwargs):
|
||||||
self.module._build_shot_conditioning = lambda *_args, **_kwargs: (
|
order.append("unload_h3_failed")
|
||||||
"cond",
|
raise RuntimeError("model wrapper does not expose clone metadata")
|
||||||
|
|
||||||
|
def unload_all_models(*_args, **_kwargs):
|
||||||
|
order.append("unload_all")
|
||||||
|
|
||||||
|
def upscale_latent_video(video, param):
|
||||||
|
order.append("upscale")
|
||||||
|
return FakeTensor("upv"), 8, 16
|
||||||
|
|
||||||
|
def build_conditioning(*_args, **_kwargs):
|
||||||
|
build_calls.append(True)
|
||||||
|
return (
|
||||||
|
[["cond", {}]],
|
||||||
{"samples": FakeNestedTensor((FakeTensor("basev"), FakeTensor("basea")))},
|
{"samples": FakeNestedTensor((FakeTensor("basev"), FakeTensor("basea")))},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
self.module.nodes.common_ksampler = common_ksampler
|
||||||
|
self.module._build_shot_conditioning = build_conditioning
|
||||||
self.module._evict_all_but = lambda *_args, **_kwargs: None
|
self.module._evict_all_but = lambda *_args, **_kwargs: None
|
||||||
|
self.module.mm.unload_model_and_clones = unload_model_and_clones
|
||||||
|
self.module.mm.unload_all_models = unload_all_models
|
||||||
|
self.module._upscale_latent_video = upscale_latent_video
|
||||||
|
self.module._copy_sample_latent = lambda sampled: sampled["samples"].unbind()
|
||||||
self.module._decode_video = decode_video
|
self.module._decode_video = decode_video
|
||||||
self.module._decode_audio = decode_audio
|
self.module._decode_audio = decode_audio
|
||||||
self.module._deep_cleanup = cleanup
|
self.module._deep_cleanup = cleanup
|
||||||
@@ -406,17 +444,25 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
tiled=False,
|
tiled=False,
|
||||||
sa=(123, 20, 1.0, "res_multistep", "simple", 1.0),
|
sa=(123, 20, 1.0, "res_multistep", "simple", 1.0),
|
||||||
handoff=None,
|
handoff=None,
|
||||||
detail_pass=True,
|
latent_upscale_param={
|
||||||
detail_sampler_name="euler",
|
"mode": "model",
|
||||||
detail_scheduler="beta",
|
"model_name": "upscale.safetensors",
|
||||||
detail_steps=5,
|
"device": "cuda",
|
||||||
detail_denoise=0.4,
|
"precision": "fp16",
|
||||||
|
"sampler_name": "euler_ancestral",
|
||||||
|
"scheduler": "simple",
|
||||||
|
"steps": 2,
|
||||||
|
"denoise": 0.4,
|
||||||
|
"megapixels": 1.5,
|
||||||
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
self.assertEqual(order[0], "sample")
|
self.assertEqual(order[0], "sample")
|
||||||
self.assertEqual(order[1], "detail_sample")
|
self.assertLess(order.index("unload_all"), order.index("upscale"))
|
||||||
|
self.assertLess(order.index("upscale"), order.index("latent_upscale_sample"))
|
||||||
self.assertLess(order.index("audio"), order.index("video"))
|
self.assertLess(order.index("audio"), order.index("video"))
|
||||||
self.assertEqual(order[-1], "cleanup")
|
self.assertEqual(order[-1], "cleanup")
|
||||||
|
self.assertEqual(len(build_calls), 1)
|
||||||
finally:
|
finally:
|
||||||
self.module.nodes.common_ksampler = original_common_ksampler
|
self.module.nodes.common_ksampler = original_common_ksampler
|
||||||
self.module._build_shot_conditioning = original_build
|
self.module._build_shot_conditioning = original_build
|
||||||
@@ -424,12 +470,25 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
self.module._decode_video = original_decode_video
|
self.module._decode_video = original_decode_video
|
||||||
self.module._decode_audio = original_decode_audio
|
self.module._decode_audio = original_decode_audio
|
||||||
self.module._deep_cleanup = original_cleanup
|
self.module._deep_cleanup = original_cleanup
|
||||||
|
self.module._upscale_latent_video = original_upscale
|
||||||
|
self.module._copy_sample_latent = original_copy_sample
|
||||||
|
if original_unload is None:
|
||||||
|
delattr(self.module.mm, "unload_model_and_clones")
|
||||||
|
else:
|
||||||
|
self.module.mm.unload_model_and_clones = original_unload
|
||||||
|
self.module.mm.unload_all_models = original_unload_all
|
||||||
if original_nested is None:
|
if original_nested is None:
|
||||||
delattr(self.module.comfy.nested_tensor, "NestedTensor")
|
delattr(self.module.comfy.nested_tensor, "NestedTensor")
|
||||||
else:
|
else:
|
||||||
self.module.comfy.nested_tensor.NestedTensor = original_nested
|
self.module.comfy.nested_tensor.NestedTensor = original_nested
|
||||||
|
|
||||||
def test_detail_pass_treats_falsey_strings_as_disabled(self):
|
def test_latent_refine_tiles_do_not_rebuild_conditioning(self):
|
||||||
|
source = inspect.getsource(self.module.H3LongVideos._render)
|
||||||
|
tile_branch = source[source.index("for col_index, c0 in enumerate(cols):"):]
|
||||||
|
self.assertIn("_crop_conditioning_to_tile", tile_branch)
|
||||||
|
self.assertNotIn("_build_shot_conditioning(", tile_branch)
|
||||||
|
|
||||||
|
def test_latent_upscale_off_skips_second_pass(self):
|
||||||
calls = []
|
calls = []
|
||||||
original_common_ksampler = self.module.nodes.common_ksampler
|
original_common_ksampler = self.module.nodes.common_ksampler
|
||||||
original_build = self.module._build_shot_conditioning
|
original_build = self.module._build_shot_conditioning
|
||||||
@@ -437,6 +496,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
original_decode_video = self.module._decode_video
|
original_decode_video = self.module._decode_video
|
||||||
original_decode_audio = self.module._decode_audio
|
original_decode_audio = self.module._decode_audio
|
||||||
original_cleanup = self.module._deep_cleanup
|
original_cleanup = self.module._deep_cleanup
|
||||||
|
original_upscale = self.module._upscale_latent_video
|
||||||
|
original_copy_sample = self.module._copy_sample_latent
|
||||||
try:
|
try:
|
||||||
self.module.nodes.common_ksampler = lambda *args, **kwargs: (calls.append((args, kwargs)) or {"samples": "latent"},)
|
self.module.nodes.common_ksampler = lambda *args, **kwargs: (calls.append((args, kwargs)) or {"samples": "latent"},)
|
||||||
self.module._build_shot_conditioning = lambda *_args, **_kwargs: ("cond", {"samples": "base"})
|
self.module._build_shot_conditioning = lambda *_args, **_kwargs: ("cond", {"samples": "base"})
|
||||||
@@ -444,6 +505,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
self.module._decode_video = lambda _vae, out_latent, *_args, **_kwargs: out_latent
|
self.module._decode_video = lambda _vae, out_latent, *_args, **_kwargs: out_latent
|
||||||
self.module._decode_audio = lambda _vae, out_latent: out_latent
|
self.module._decode_audio = lambda _vae, out_latent: out_latent
|
||||||
self.module._deep_cleanup = lambda: None
|
self.module._deep_cleanup = lambda: None
|
||||||
|
self.module._upscale_latent_video = lambda *_args, **_kwargs: (_ for _ in ()).throw(RuntimeError("should not run"))
|
||||||
|
self.module._copy_sample_latent = lambda sampled: sampled
|
||||||
|
|
||||||
self.module.H3LongVideos()._render(
|
self.module.H3LongVideos()._render(
|
||||||
model=object(),
|
model=object(),
|
||||||
@@ -459,7 +522,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
tiled=False,
|
tiled=False,
|
||||||
sa=(123, 20, 1.0, "res_multistep", "simple", 1.0),
|
sa=(123, 20, 1.0, "res_multistep", "simple", 1.0),
|
||||||
handoff=None,
|
handoff=None,
|
||||||
detail_pass="false",
|
latent_upscale_param={"mode": "off"},
|
||||||
)
|
)
|
||||||
|
|
||||||
self.assertEqual(len(calls), 1)
|
self.assertEqual(len(calls), 1)
|
||||||
@@ -470,6 +533,8 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
self.module._decode_video = original_decode_video
|
self.module._decode_video = original_decode_video
|
||||||
self.module._decode_audio = original_decode_audio
|
self.module._decode_audio = original_decode_audio
|
||||||
self.module._deep_cleanup = original_cleanup
|
self.module._deep_cleanup = original_cleanup
|
||||||
|
self.module._upscale_latent_video = original_upscale
|
||||||
|
self.module._copy_sample_latent = original_copy_sample
|
||||||
|
|
||||||
def test_distribute_generations_canonicalizes_per_shot_audio_and_anchor_directives(self):
|
def test_distribute_generations_canonicalizes_per_shot_audio_and_anchor_directives(self):
|
||||||
generations = self.module.distribute_generations(
|
generations = self.module.distribute_generations(
|
||||||
@@ -736,6 +801,12 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
self.assertNotIn(f"ref_image_{index}", optional)
|
self.assertNotIn(f"ref_image_{index}", optional)
|
||||||
self.assertNotIn("per_beat_length", optional)
|
self.assertNotIn("per_beat_length", optional)
|
||||||
self.assertNotIn("cleanup_between_shots", optional)
|
self.assertNotIn("cleanup_between_shots", optional)
|
||||||
|
self.assertNotIn("detail_pass", optional)
|
||||||
|
self.assertNotIn("detail_sampler_name", optional)
|
||||||
|
self.assertNotIn("detail_scheduler", optional)
|
||||||
|
self.assertNotIn("detail_steps", optional)
|
||||||
|
self.assertNotIn("detail_denoise", optional)
|
||||||
|
self.assertIn("latent_upscale_param", optional)
|
||||||
|
|
||||||
def test_shot_seconds_tooltip_describes_ceiling_behavior(self):
|
def test_shot_seconds_tooltip_describes_ceiling_behavior(self):
|
||||||
optional = self.module.H3LongVideos.INPUT_TYPES()["optional"]
|
optional = self.module.H3LongVideos.INPUT_TYPES()["optional"]
|
||||||
@@ -743,7 +814,7 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
|
|
||||||
self.assertIn("GLOBAL per-shot maximum", tooltip)
|
self.assertIn("GLOBAL per-shot maximum", tooltip)
|
||||||
self.assertIn("A beat's own `seconds:` directive can still ask for less", tooltip)
|
self.assertIn("A beat's own `seconds:` directive can still ask for less", tooltip)
|
||||||
self.assertIn("honoring it; may spill to system RAM (slow) or OOM", tooltip)
|
self.assertIn("let the render fail instead of shrinking it", tooltip)
|
||||||
|
|
||||||
def test_resolve_shot_frames_honors_forced_request_over_budget(self):
|
def test_resolve_shot_frames_honors_forced_request_over_budget(self):
|
||||||
original_estimate_shot_frames = self.module.estimate_shot_frames
|
original_estimate_shot_frames = self.module.estimate_shot_frames
|
||||||
@@ -928,12 +999,23 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
plan_only=True,
|
plan_only=True,
|
||||||
ref_1={"image": "live-1"},
|
ref_1={"image": "live-1"},
|
||||||
ref_3={"image": "live-3"},
|
ref_3={"image": "live-3"},
|
||||||
|
latent_upscale_param={
|
||||||
|
"mode": "interp",
|
||||||
|
"method": "bilinear",
|
||||||
|
"sampler_name": "euler_ancestral",
|
||||||
|
"scheduler": "simple",
|
||||||
|
"steps": 2,
|
||||||
|
"denoise": 0.2,
|
||||||
|
"megapixels": 1.5,
|
||||||
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
self.assertEqual(calls["refs"][0]["image"], "live-1")
|
self.assertEqual(calls["refs"][0]["image"], "live-1")
|
||||||
self.assertIsNone(calls["refs"][1])
|
self.assertIsNone(calls["refs"][1])
|
||||||
self.assertEqual(calls["refs"][2]["image"], "live-3")
|
self.assertEqual(calls["refs"][2]["image"], "live-3")
|
||||||
self.assertEqual(result[2].count("ref2va: 2 reference image(s)"), 1)
|
self.assertEqual(result[2].count("ref2va: 2 reference image(s)"), 1)
|
||||||
|
self.assertIn("latent upscale:", result[2])
|
||||||
|
self.assertIn("euler_ancestral/simple", result[2])
|
||||||
finally:
|
finally:
|
||||||
self.module.parse_resolution = original_parse_resolution
|
self.module.parse_resolution = original_parse_resolution
|
||||||
self.module._connected_refs = original_connected_refs
|
self.module._connected_refs = original_connected_refs
|
||||||
@@ -1061,6 +1143,389 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
|
|||||||
{"DumasH3LongVideos": "Dumas H3 Long Videos (FL2VA + REF2VA)"},
|
{"DumasH3LongVideos": "Dumas H3 Long Videos (FL2VA + REF2VA)"},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def test_latent_upscale_params_node_is_exposed(self):
|
||||||
|
latent = importlib.import_module("dumas_h3_latent_upscale")
|
||||||
|
required = latent.H3LatentUpscaleParams.INPUT_TYPES()["required"]
|
||||||
|
|
||||||
|
self.assertEqual(
|
||||||
|
latent.NODE_CLASS_MAPPINGS,
|
||||||
|
{"DumasH3LatentUpscaleParams": latent.H3LatentUpscaleParams},
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
latent.NODE_DISPLAY_NAME_MAPPINGS,
|
||||||
|
{"DumasH3LatentUpscaleParams": "Dumas H3 Latent Upscale Params"},
|
||||||
|
)
|
||||||
|
self.assertEqual(required["sampler_name"][1]["default"], "euler_ancestral")
|
||||||
|
self.assertEqual(required["scheduler"][1]["default"], "simple")
|
||||||
|
self.assertEqual(required["steps"][1]["default"], 2)
|
||||||
|
self.assertEqual(required["denoise"][1]["default"], 0.2)
|
||||||
|
self.assertEqual(required["megapixels"][1]["default"], 1.0)
|
||||||
|
self.assertEqual(required["tile_width"][1]["default"], 512)
|
||||||
|
self.assertEqual(required["tile_height"][1]["default"], 512)
|
||||||
|
self.assertEqual(required["overlap"][1]["default"], 64)
|
||||||
|
self.assertEqual(required["fade_width"][1]["default"], 32)
|
||||||
|
self.assertEqual(required["fade_height"][1]["default"], 32)
|
||||||
|
self.assertEqual(required["overlap_mode"][1]["default"], "earlier")
|
||||||
|
self.assertEqual(required["overlap_blend"][1]["default"], "linear")
|
||||||
|
self.assertEqual(required["tile_size_mode"][1]["default"], "specific_size")
|
||||||
|
self.assertEqual(required["grid_rows"][1]["default"], 2)
|
||||||
|
self.assertEqual(required["grid_cols"][1]["default"], 2)
|
||||||
|
self.assertEqual(required["spatial_w_overlap"][1]["default"], 128)
|
||||||
|
self.assertEqual(required["spatial_h_overlap"][1]["default"], 128)
|
||||||
|
self.assertEqual(required["min_tile_size"][1]["default"], 256)
|
||||||
|
self.assertEqual(required["masked_area_noise"][1]["default"], 0.0)
|
||||||
|
self.assertFalse(required["brightness_match"][1]["default"])
|
||||||
|
self.assertEqual(required["dynamic_fade"][1]["default"], "off")
|
||||||
|
self.assertEqual(required["dynamic_fade_min"][1]["default"], 32)
|
||||||
|
self.assertEqual(required["chunk_length"][1]["default"], 85)
|
||||||
|
self.assertEqual(required["temporal_overlap"][1]["default"], 17)
|
||||||
|
self.assertFalse(required["resize_conditioning"][1]["default"])
|
||||||
|
self.assertEqual(required["anchor_strength"][1]["default"], 0.999)
|
||||||
|
|
||||||
|
def test_latent_upscale_mode_infers_legacy_model_payloads(self):
|
||||||
|
self.assertEqual(self.module._latent_upscale_mode({"model_name": "foo.safetensors"}), "model")
|
||||||
|
self.assertEqual(self.module._latent_upscale_mode({"method": "bilinear"}), "interp")
|
||||||
|
self.assertEqual(self.module._latent_upscale_mode({"mode": "model"}), "model")
|
||||||
|
self.assertEqual(self.module._latent_upscale_mode({}), "off")
|
||||||
|
|
||||||
|
def test_tag_oom_stage_marks_oom_exceptions(self):
|
||||||
|
exc = RuntimeError("CUDA out of memory")
|
||||||
|
tagged = self.module._tag_oom_stage(exc, "latent_upscale")
|
||||||
|
self.assertIs(tagged, exc)
|
||||||
|
self.assertEqual(getattr(tagged, "_h3_stage", ""), "latent_upscale")
|
||||||
|
|
||||||
|
def test_shrink_model_tile_param_reduces_tile_size(self):
|
||||||
|
latent = importlib.import_module("dumas_h3_latent_upscale")
|
||||||
|
smaller = latent._shrink_model_tile_param({
|
||||||
|
"tile_size_mode": "specific_size",
|
||||||
|
"tile_width": 512,
|
||||||
|
"tile_height": 512,
|
||||||
|
"overlap": 64,
|
||||||
|
"fade_width": 32,
|
||||||
|
"fade_height": 32,
|
||||||
|
})
|
||||||
|
self.assertIsNotNone(smaller)
|
||||||
|
self.assertEqual(smaller["tile_size_mode"], "rows_cols")
|
||||||
|
self.assertEqual(smaller["grid_rows"], 4)
|
||||||
|
self.assertEqual(smaller["grid_cols"], 4)
|
||||||
|
self.assertEqual(smaller["spatial_w_overlap"], 0)
|
||||||
|
self.assertEqual(smaller["spatial_h_overlap"], 0)
|
||||||
|
self.assertEqual(smaller["fade_width"], 0)
|
||||||
|
self.assertEqual(smaller["fade_height"], 0)
|
||||||
|
self.assertEqual(smaller["min_tile_size"], 32)
|
||||||
|
|
||||||
|
def test_shrink_model_tile_param_rows_cols_resets_overlap(self):
|
||||||
|
latent = importlib.import_module("dumas_h3_latent_upscale")
|
||||||
|
smaller = latent._shrink_model_tile_param({
|
||||||
|
"tile_size_mode": "rows_cols",
|
||||||
|
"grid_rows": 4,
|
||||||
|
"grid_cols": 4,
|
||||||
|
"spatial_w_overlap": 128,
|
||||||
|
"spatial_h_overlap": 128,
|
||||||
|
"fade_width": 64,
|
||||||
|
"fade_height": 64,
|
||||||
|
"min_tile_size": 256,
|
||||||
|
})
|
||||||
|
self.assertIsNotNone(smaller)
|
||||||
|
self.assertEqual(smaller["grid_rows"], 8)
|
||||||
|
self.assertEqual(smaller["grid_cols"], 8)
|
||||||
|
self.assertEqual(smaller["spatial_w_overlap"], 0)
|
||||||
|
self.assertEqual(smaller["spatial_h_overlap"], 0)
|
||||||
|
self.assertEqual(smaller["fade_width"], 0)
|
||||||
|
self.assertEqual(smaller["fade_height"], 0)
|
||||||
|
self.assertEqual(smaller["min_tile_size"], 32)
|
||||||
|
|
||||||
|
def test_shrink_model_tile_param_rows_cols_can_reach_thirty_two(self):
|
||||||
|
latent = importlib.import_module("dumas_h3_latent_upscale")
|
||||||
|
smaller = latent._shrink_model_tile_param({
|
||||||
|
"tile_size_mode": "rows_cols",
|
||||||
|
"grid_rows": 16,
|
||||||
|
"grid_cols": 16,
|
||||||
|
"spatial_w_overlap": 0,
|
||||||
|
"spatial_h_overlap": 0,
|
||||||
|
"fade_width": 0,
|
||||||
|
"fade_height": 0,
|
||||||
|
"min_tile_size": 32,
|
||||||
|
})
|
||||||
|
self.assertIsNotNone(smaller)
|
||||||
|
self.assertEqual(smaller["grid_rows"], 32)
|
||||||
|
self.assertEqual(smaller["grid_cols"], 32)
|
||||||
|
|
||||||
|
def test_temporal_segments_split_long_sequences(self):
|
||||||
|
latent = importlib.import_module("dumas_h3_latent_upscale")
|
||||||
|
bounds = latent._temporal_segments(36, 85, 17)
|
||||||
|
self.assertGreater(len(bounds), 1)
|
||||||
|
self.assertEqual(bounds[0][0], 0)
|
||||||
|
self.assertEqual(bounds[-1][2], 36)
|
||||||
|
|
||||||
|
def test_shrink_temporal_param_reduces_chunk_length(self):
|
||||||
|
latent = importlib.import_module("dumas_h3_latent_upscale")
|
||||||
|
smaller = latent._shrink_temporal_param({
|
||||||
|
"chunk_length": 85,
|
||||||
|
"temporal_overlap": 17,
|
||||||
|
})
|
||||||
|
self.assertIsNotNone(smaller)
|
||||||
|
self.assertEqual(smaller["chunk_length"], 17)
|
||||||
|
self.assertEqual(smaller["temporal_overlap"], 0)
|
||||||
|
|
||||||
|
def test_cuda_model_temporal_params_keep_splitting_saved_workflows(self):
|
||||||
|
latent = importlib.import_module("dumas_h3_latent_upscale")
|
||||||
|
chunk_length, temporal_overlap = latent._effective_temporal_params({
|
||||||
|
"mode": "model",
|
||||||
|
"device": "cuda",
|
||||||
|
"chunk_length": 85,
|
||||||
|
"temporal_overlap": 17,
|
||||||
|
}, frame_count=124)
|
||||||
|
self.assertEqual(chunk_length, 85)
|
||||||
|
self.assertEqual(temporal_overlap, 17)
|
||||||
|
|
||||||
|
def test_cuda_model_temporal_params_keep_short_saved_workflows_until_oom(self):
|
||||||
|
latent = importlib.import_module("dumas_h3_latent_upscale")
|
||||||
|
chunk_length, temporal_overlap = latent._effective_temporal_params({
|
||||||
|
"mode": "model",
|
||||||
|
"device": "cuda",
|
||||||
|
"chunk_length": 85,
|
||||||
|
"temporal_overlap": 17,
|
||||||
|
}, frame_count=85)
|
||||||
|
self.assertEqual(chunk_length, 85)
|
||||||
|
self.assertEqual(temporal_overlap, 17)
|
||||||
|
|
||||||
|
def test_cuda_model_oom_retries_temporal_before_spatial_fallback(self):
|
||||||
|
latent = importlib.import_module("dumas_h3_latent_upscale")
|
||||||
|
|
||||||
|
calls = []
|
||||||
|
original_tiled = latent._upscale_video_model_tiled
|
||||||
|
original_shrink_model = latent._shrink_model_tile_param
|
||||||
|
try:
|
||||||
|
def tiled(_video, param):
|
||||||
|
calls.append(("tiled", param.get("chunk_length"), param.get("tile_size_mode")))
|
||||||
|
raise RuntimeError("out of memory")
|
||||||
|
|
||||||
|
latent._upscale_video_model_tiled = tiled
|
||||||
|
latent._shrink_model_tile_param = (
|
||||||
|
lambda param: calls.append(("shrink_spatial", param.get("tile_size_mode"))) or None
|
||||||
|
)
|
||||||
|
|
||||||
|
with self.assertRaisesRegex(RuntimeError, "smaller temporal chunk"):
|
||||||
|
latent.upscale_video_model(
|
||||||
|
"video",
|
||||||
|
{
|
||||||
|
"mode": "model",
|
||||||
|
"device": "cuda",
|
||||||
|
"model_name": "upscale.safetensors",
|
||||||
|
"chunk_length": 85,
|
||||||
|
"temporal_overlap": 17,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(calls[0], ("tiled", 85, None))
|
||||||
|
self.assertNotIn(("shrink_spatial", None), calls)
|
||||||
|
finally:
|
||||||
|
latent._upscale_video_model_tiled = original_tiled
|
||||||
|
latent._shrink_model_tile_param = original_shrink_model
|
||||||
|
|
||||||
|
def test_interp_temporal_params_preserve_upstream_defaults(self):
|
||||||
|
latent = importlib.import_module("dumas_h3_latent_upscale")
|
||||||
|
chunk_length, temporal_overlap = latent._effective_temporal_params({
|
||||||
|
"mode": "interp",
|
||||||
|
"device": "cuda",
|
||||||
|
"chunk_length": 85,
|
||||||
|
"temporal_overlap": 17,
|
||||||
|
})
|
||||||
|
self.assertEqual(chunk_length, 85)
|
||||||
|
self.assertEqual(temporal_overlap, 17)
|
||||||
|
|
||||||
|
def test_unload_upscale_model_defers_while_held(self):
|
||||||
|
latent = importlib.import_module("dumas_h3_latent_upscale")
|
||||||
|
|
||||||
|
class FakeParam:
|
||||||
|
device = "cuda"
|
||||||
|
|
||||||
|
class FakeModel:
|
||||||
|
def __init__(self):
|
||||||
|
self.moves = []
|
||||||
|
|
||||||
|
def parameters(self):
|
||||||
|
return iter((FakeParam(),))
|
||||||
|
|
||||||
|
def to(self, device):
|
||||||
|
self.moves.append(device)
|
||||||
|
return self
|
||||||
|
|
||||||
|
cache_key = "upscale.safetensors::cuda::fp16"
|
||||||
|
original_cache_value = latent._MODEL_CACHE.get(cache_key)
|
||||||
|
original_hold_depth = latent._MODEL_HOLD_DEPTH
|
||||||
|
fake_model = FakeModel()
|
||||||
|
try:
|
||||||
|
latent._MODEL_CACHE[cache_key] = fake_model
|
||||||
|
latent._MODEL_HOLD_DEPTH = 0
|
||||||
|
with latent._hold_upscale_model_loaded():
|
||||||
|
latent.unload_upscale_model("upscale.safetensors", "cuda", "fp16")
|
||||||
|
self.assertEqual(fake_model.moves, [])
|
||||||
|
|
||||||
|
latent.unload_upscale_model("upscale.safetensors", "cuda", "fp16")
|
||||||
|
self.assertEqual(fake_model.moves, ["cpu"])
|
||||||
|
finally:
|
||||||
|
latent._MODEL_HOLD_DEPTH = original_hold_depth
|
||||||
|
if original_cache_value is None:
|
||||||
|
latent._MODEL_CACHE.pop(cache_key, None)
|
||||||
|
else:
|
||||||
|
latent._MODEL_CACHE[cache_key] = original_cache_value
|
||||||
|
|
||||||
|
def test_model_upscale_releases_cached_model_after_pass(self):
|
||||||
|
latent = importlib.import_module("dumas_h3_latent_upscale")
|
||||||
|
calls = []
|
||||||
|
|
||||||
|
original_temporal = latent._upscale_video_temporal_chunks
|
||||||
|
original_unload_now = latent._unload_upscale_model_now
|
||||||
|
original_cuda = latent.torch.cuda
|
||||||
|
original_device = getattr(latent.torch, "device", None)
|
||||||
|
try:
|
||||||
|
latent.torch.cuda = types.SimpleNamespace(is_available=lambda: True)
|
||||||
|
latent.torch.device = lambda value: value
|
||||||
|
|
||||||
|
def temporal(video, param, upscaler):
|
||||||
|
calls.append(("temporal", latent._MODEL_HOLD_DEPTH))
|
||||||
|
return "video", 8, 16
|
||||||
|
|
||||||
|
def unload_now(name, device, precision):
|
||||||
|
calls.append(("unload", name, device, precision, latent._MODEL_HOLD_DEPTH))
|
||||||
|
|
||||||
|
latent._upscale_video_temporal_chunks = temporal
|
||||||
|
latent._unload_upscale_model_now = unload_now
|
||||||
|
|
||||||
|
result = latent.upscale_latent_video("source", {
|
||||||
|
"mode": "model",
|
||||||
|
"model_name": "upscale.safetensors",
|
||||||
|
"device": "cuda",
|
||||||
|
"precision": "fp16",
|
||||||
|
})
|
||||||
|
|
||||||
|
self.assertEqual(result, ("video", 8, 16))
|
||||||
|
self.assertEqual(calls[0], ("temporal", 1))
|
||||||
|
self.assertEqual(calls[1], ("unload", "upscale.safetensors", "cuda", "fp16", 1))
|
||||||
|
self.assertEqual(latent._MODEL_HOLD_DEPTH, 0)
|
||||||
|
finally:
|
||||||
|
latent._upscale_video_temporal_chunks = original_temporal
|
||||||
|
latent._unload_upscale_model_now = original_unload_now
|
||||||
|
latent.torch.cuda = original_cuda
|
||||||
|
if original_device is None:
|
||||||
|
delattr(latent.torch, "device")
|
||||||
|
else:
|
||||||
|
latent.torch.device = original_device
|
||||||
|
|
||||||
|
def test_model_upscale_oom_falls_back_to_interp(self):
|
||||||
|
latent = importlib.import_module("dumas_h3_latent_upscale")
|
||||||
|
calls = []
|
||||||
|
|
||||||
|
original_temporal = latent._upscale_video_temporal_chunks
|
||||||
|
original_interp = latent.upscale_video_interp
|
||||||
|
original_unload_now = latent._unload_upscale_model_now
|
||||||
|
original_cuda = latent.torch.cuda
|
||||||
|
original_device = getattr(latent.torch, "device", None)
|
||||||
|
try:
|
||||||
|
latent.torch.cuda = types.SimpleNamespace(
|
||||||
|
is_available=lambda: True,
|
||||||
|
empty_cache=lambda: calls.append(("empty_cache",)),
|
||||||
|
)
|
||||||
|
latent.torch.device = lambda value: value
|
||||||
|
|
||||||
|
def temporal(_video, _param, _upscaler):
|
||||||
|
calls.append(("temporal", latent._MODEL_HOLD_DEPTH))
|
||||||
|
raise RuntimeError("H3 latent upscale exhausted its GPU spatial fallbacks")
|
||||||
|
|
||||||
|
def interp(video, param):
|
||||||
|
calls.append(("interp", video, param.get("mode"), param.get("method")))
|
||||||
|
return "interp_video", 8, 16
|
||||||
|
|
||||||
|
def unload_now(name, device, precision):
|
||||||
|
calls.append(("unload", name, device, precision, latent._MODEL_HOLD_DEPTH))
|
||||||
|
|
||||||
|
latent._upscale_video_temporal_chunks = temporal
|
||||||
|
latent.upscale_video_interp = interp
|
||||||
|
latent._unload_upscale_model_now = unload_now
|
||||||
|
|
||||||
|
result = latent.upscale_latent_video("source", {
|
||||||
|
"mode": "model",
|
||||||
|
"model_name": "upscale.safetensors",
|
||||||
|
"method": "bilinear",
|
||||||
|
"device": "cuda",
|
||||||
|
"precision": "fp16",
|
||||||
|
})
|
||||||
|
|
||||||
|
self.assertEqual(result, ("interp_video", 8, 16))
|
||||||
|
self.assertEqual(calls[0], ("temporal", 1))
|
||||||
|
self.assertEqual(calls[1], ("unload", "upscale.safetensors", "cuda", "fp16", 1))
|
||||||
|
self.assertIn(("empty_cache",), calls)
|
||||||
|
self.assertEqual(calls[-1], ("interp", "source", "interp", "bilinear"))
|
||||||
|
self.assertEqual(latent._MODEL_HOLD_DEPTH, 0)
|
||||||
|
finally:
|
||||||
|
latent._upscale_video_temporal_chunks = original_temporal
|
||||||
|
latent.upscale_video_interp = original_interp
|
||||||
|
latent._unload_upscale_model_now = original_unload_now
|
||||||
|
latent.torch.cuda = original_cuda
|
||||||
|
if original_device is None:
|
||||||
|
delattr(latent.torch, "device")
|
||||||
|
else:
|
||||||
|
latent.torch.device = original_device
|
||||||
|
|
||||||
|
def test_model_upscale_skips_learned_model_on_8gb_cuda(self):
|
||||||
|
latent = importlib.import_module("dumas_h3_latent_upscale")
|
||||||
|
calls = []
|
||||||
|
|
||||||
|
original_temporal = latent._upscale_video_temporal_chunks
|
||||||
|
original_interp = latent.upscale_video_interp
|
||||||
|
original_cuda = latent.torch.cuda
|
||||||
|
try:
|
||||||
|
latent.torch.cuda = types.SimpleNamespace(
|
||||||
|
is_available=lambda: True,
|
||||||
|
mem_get_info=lambda: (1 * 1024 * 1024 * 1024, 8 * 1024 * 1024 * 1024),
|
||||||
|
empty_cache=lambda: calls.append(("empty_cache",)),
|
||||||
|
)
|
||||||
|
|
||||||
|
def temporal(_video, _param, _upscaler):
|
||||||
|
calls.append(("temporal",))
|
||||||
|
raise AssertionError("learned model path should be skipped on 8GB CUDA")
|
||||||
|
|
||||||
|
def interp(video, param):
|
||||||
|
calls.append(("interp", video, param.get("mode"), param.get("method")))
|
||||||
|
return "interp_video", 8, 16
|
||||||
|
|
||||||
|
latent._upscale_video_temporal_chunks = temporal
|
||||||
|
latent.upscale_video_interp = interp
|
||||||
|
|
||||||
|
result = latent.upscale_latent_video("source", {
|
||||||
|
"mode": "model",
|
||||||
|
"model_name": "upscale.safetensors",
|
||||||
|
"method": "bilinear",
|
||||||
|
"device": "cuda",
|
||||||
|
"precision": "fp16",
|
||||||
|
})
|
||||||
|
|
||||||
|
self.assertEqual(result, ("interp_video", 8, 16))
|
||||||
|
self.assertNotIn(("temporal",), calls)
|
||||||
|
self.assertIn(("empty_cache",), calls)
|
||||||
|
self.assertEqual(calls[-1], ("interp", "source", "interp", "bilinear"))
|
||||||
|
finally:
|
||||||
|
latent._upscale_video_temporal_chunks = original_temporal
|
||||||
|
latent.upscale_video_interp = original_interp
|
||||||
|
latent.torch.cuda = original_cuda
|
||||||
|
|
||||||
|
def test_upscale_video_model_raises_when_gpu_cannot_shrink(self):
|
||||||
|
latent = importlib.import_module("dumas_h3_latent_upscale")
|
||||||
|
|
||||||
|
original_tiled = latent._upscale_video_model_tiled
|
||||||
|
original_shrink = latent._shrink_model_tile_param
|
||||||
|
try:
|
||||||
|
latent._shrink_model_tile_param = lambda _param: None
|
||||||
|
latent._upscale_video_model_tiled = lambda *_args, **_kwargs: (_ for _ in ()).throw(RuntimeError("out of memory"))
|
||||||
|
|
||||||
|
with self.assertRaisesRegex(RuntimeError, "H3 latent upscale exhausted its GPU spatial fallbacks"):
|
||||||
|
latent.upscale_video_model("video", {"device": "cuda", "precision": "fp16"})
|
||||||
|
finally:
|
||||||
|
latent._upscale_video_model_tiled = original_tiled
|
||||||
|
latent._shrink_model_tile_param = original_shrink
|
||||||
|
|
||||||
def test_compose_persistent_does_not_expand_ambiguous_plural_to_full_cast(self):
|
def test_compose_persistent_does_not_expand_ambiguous_plural_to_full_cast(self):
|
||||||
active = self.module.parse_wardrobe(
|
active = self.module.parse_wardrobe(
|
||||||
"Maya = she, red jacket\n"
|
"Maya = she, red jacket\n"
|
||||||
|
|||||||
Reference in New Issue
Block a user