Compare commits

..
8 Commits
5 changed files with 1435 additions and 12 deletions
+30 -1
View File
@@ -56,6 +56,13 @@
- Output: `prompt`
- Builds one H3 prompt block per beat, with quick controls for per-shot timing, continuity, ref behavior, anchor additions, soundscape, and music while staying compatible with direct text editing.
- `Dumas H3 Prompt Curator`
- Inputs: `action_prompt`, `anatomy_guard`, `subject_count_guard`, optional `anchor`, optional `soundscape`, optional `bgm`, optional `ref_1` through `ref_9`
- Outputs: `prompt`, `ref_image_1` through `ref_image_9`, `reference_count`, `debug`
- Builds one standalone MiniMax H3 prompt from your final action text plus structured character/location references.
- The action text can mention references by character/location name, alias, `<Picture N>`, or `<refN>`. Only mentioned references are emitted, and the output images are compacted/renumbered so skipped inputs do not leave gaps.
- Adds curated reference context, anatomy guard text, optional subject-count guard text, anchor/style text, `overall_soundscape:` text, and `background_music:` text while respecting MiniMax H3's reference-generation shape: one prompt plus up to nine reference images.
- `Dumas H3 Shot Length`
- Inputs: `shot_seconds`, `fps`, optional `cap_to_h3_max`
- Outputs: `seconds`, `frames`, `info`
@@ -76,6 +83,18 @@
- Output: `reference`
- Builds one structured `REFERENCE` object for a location/environment so H3 can use the same socket type for both character and scenic refs.
- `Dumas Character Helper`
- Inputs: `image1`, `image2`, picture IDs, character identity fields, `general`, `wardrobe`
- Outputs: `image1`, `image2`, `reference_prompt`, `wardrobe`, `reference1`, `reference2`
- Restores the original general-purpose helper shape while also emitting two structured `REFERENCE` objects for the prompt curator.
- The structured references carry the same character name, alias, age, height, gender, nationality, occupation, accent, wardrobe, and notes, so mentioning the character name in `Dumas H3 Prompt Curator` can include both helper images and the character facts automatically.
- `Dumas Location Helper`
- Inputs: `image1`, `image2`, picture IDs, `location_id`, `name`, `alias`, `description`, `general`
- Outputs: `image1`, `image2`, `reference_prompt`, `reference1`, `reference2`
- Matching general-purpose helper for environments/locations: pass two images through unchanged, emit location reference prompt text, and provide two structured `REFERENCE` objects for the prompt curator.
- The structured references carry the same location name, alias, description, and notes, so mentioning the location name in `Dumas H3 Prompt Curator` can include both helper images and the location context automatically.
- `Dumas Anchor Style`
- Inputs: `anchor_style`, `style_description`
- Output: `anchor`
@@ -83,6 +102,16 @@
- The preset wording is tuned for H3-safe persistent anchors: camera language, lighting, texture, production treatment, and tone, without naming characters or describing one-off actions.
- Selecting a preset fills the editable description field, and the edited multiline description is the `STRING` value passed downstream into H3 anchor sockets such as `anchor_override`.
- `Dumas Soundscape Helper`
- Inputs: `soundscape`, `soundscape_description`
- Output: `soundscape`
- Matching soundscape helper for standalone H3 prompts. Pick a preset such as quiet interior, rainy street, cafe, city night, forest, industrial, or silent, then edit the text that flows into `Dumas H3 Prompt Curator`.
- `Dumas Background Music Helper`
- Inputs: `bgm`, `bgm_description`
- Output: `bgm`
- Matching BGM helper for standalone H3 prompts. Pick a preset such as subtle tension, cinematic suspense, emotional piano, dark ambient, hopeful orchestral, retro synth, action pulse, lo-fi, or no vocals, then edit the text that flows into `Dumas H3 Prompt Curator`.
- `Dumas JSON String to Object`
- Input: `json_string`
- Output: parsed `JSON`
@@ -239,7 +268,7 @@ decr -> use index - 1
`Dumas H3 Plan Attach Scene Images` and `Dumas H3 Plan Extract Scene Images` are a companion pair for `ComfyUI-MiniMaxH3-Contex-Loop` and the local `ref2v` lane. The upstream H3 plan node cannot dynamically grow nine new image sockets for every JSON-defined scene, so Dumas stores scene image bindings beside the plan using a lightweight token and an in-memory registry. That keeps `plan.json` archiving intact while still letting you wire up nine IMAGE sockets per scene through chained helper nodes.
`Dumas Character Reference` and `Dumas Location Reference` live in `Dumas/MiniMax`. Both output a structured `REFERENCE` object that carries the image plus its semantic payload. `Dumas H3 Long Videos` accepts those `REFERENCE` sockets directly on `ref_1`..`ref_9`, resolves `<Picture N>` against the wired slot positions, and can also pull character wardrobe context from the structured ref data when `character_memory` is left blank.
`Dumas Character Helper` is the restored two-image/text helper for general H3 workflows, and `Dumas Location Helper` mirrors it for scene/environment references. Both helpers also emit structured `REFERENCE` sockets for the curator. The structured `Dumas Character Reference` and `Dumas Location Reference` nodes remain available separately for workflows that want a single `REFERENCE` socket. `Dumas H3 Prompt Curator` consumes those structured references plus optional anchor, soundscape, and BGM strings, assigns the final `<Picture N>` numbering, and outputs only the compacted images the prompt actually mentions.
`Dumas Strip Iteration Suffix` keeps the part before the first underscore and drops the rest. Names like `char123_pose_final.png` become `char123.png`, while names with no underscore such as `char123.png` are left untouched.
+49 -5
View File
@@ -46,6 +46,7 @@ LOGGER = logging.getLogger(__name__)
MP_UNIT = 1024 * 1024
RES_MULTIPLE = 32
CUDA_MODEL_CHUNK_LENGTH = 17
CUDA_MODEL_MIN_TOTAL_VRAM_BYTES = 10 * 1024 * 1024 * 1024
_MODEL_HOLD_DEPTH = 0
@@ -76,6 +77,42 @@ def _should_retry_temporal_before_spatial(param):
return _uses_cuda_model_upscale(param) and int(param.get("chunk_length", 0) or 0) > CUDA_MODEL_CHUNK_LENGTH
def _cuda_total_memory_bytes():
try:
if not torch.cuda.is_available():
return 0
if hasattr(torch.cuda, "mem_get_info"):
_free, total = torch.cuda.mem_get_info()
return int(total)
current_device = torch.cuda.current_device() if hasattr(torch.cuda, "current_device") else 0
props = torch.cuda.get_device_properties(current_device)
return int(getattr(props, "total_memory", 0) or 0)
except Exception:
return 0
def _should_skip_cuda_model_upscale(param):
total = _cuda_total_memory_bytes()
return _uses_cuda_model_upscale(param) and 0 < total < CUDA_MODEL_MIN_TOTAL_VRAM_BYTES
def _fallback_to_interp(video, param, reason):
method = param.get("method", "bilinear")
LOGGER.warning("H3 latent upscale model %s; using %s interpolation instead", reason, method)
try:
gc.collect()
except Exception:
pass
if torch.cuda.is_available():
try:
torch.cuda.empty_cache()
except Exception:
pass
interp_param = dict(param)
interp_param["mode"] = "interp"
return upscale_video_interp(video, interp_param)
def _retry_with_smaller_temporal(video, param, upscaler, exc):
if not _is_oom_error(exc):
raise exc
@@ -902,15 +939,22 @@ def upscale_latent_video(video, param):
if mode == "off":
return video, video.shape[-2], video.shape[-1]
if mode == "model":
if _should_skip_cuda_model_upscale(param):
return _fallback_to_interp(video, param, "requires more than this card's VRAM")
model_name = param.get("model_name")
device = param.get("device", "cuda")
precision = param.get("precision", "fp16")
dev = torch.device(device if (device == "cpu" or torch.cuda.is_available()) else "cpu")
with _hold_upscale_model_loaded():
try:
return _upscale_video_temporal_chunks(video, param, upscale_video_model)
finally:
_unload_upscale_model_now(model_name, dev, precision)
try:
with _hold_upscale_model_loaded():
try:
return _upscale_video_temporal_chunks(video, param, upscale_video_model)
finally:
_unload_upscale_model_now(model_name, dev, precision)
except RuntimeError as exc:
if not _is_oom_error(exc):
raise
return _fallback_to_interp(video, param, "exhausted GPU memory")
return _upscale_video_temporal_chunks(video, param, upscale_video_interp)
+963 -6
View File
File diff suppressed because it is too large Load Diff
+97
View File
@@ -1414,6 +1414,103 @@ class DumasH3LongVideosHelperTests(unittest.TestCase):
else:
latent.torch.device = original_device
def test_model_upscale_oom_falls_back_to_interp(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
calls = []
original_temporal = latent._upscale_video_temporal_chunks
original_interp = latent.upscale_video_interp
original_unload_now = latent._unload_upscale_model_now
original_cuda = latent.torch.cuda
original_device = getattr(latent.torch, "device", None)
try:
latent.torch.cuda = types.SimpleNamespace(
is_available=lambda: True,
empty_cache=lambda: calls.append(("empty_cache",)),
)
latent.torch.device = lambda value: value
def temporal(_video, _param, _upscaler):
calls.append(("temporal", latent._MODEL_HOLD_DEPTH))
raise RuntimeError("H3 latent upscale exhausted its GPU spatial fallbacks")
def interp(video, param):
calls.append(("interp", video, param.get("mode"), param.get("method")))
return "interp_video", 8, 16
def unload_now(name, device, precision):
calls.append(("unload", name, device, precision, latent._MODEL_HOLD_DEPTH))
latent._upscale_video_temporal_chunks = temporal
latent.upscale_video_interp = interp
latent._unload_upscale_model_now = unload_now
result = latent.upscale_latent_video("source", {
"mode": "model",
"model_name": "upscale.safetensors",
"method": "bilinear",
"device": "cuda",
"precision": "fp16",
})
self.assertEqual(result, ("interp_video", 8, 16))
self.assertEqual(calls[0], ("temporal", 1))
self.assertEqual(calls[1], ("unload", "upscale.safetensors", "cuda", "fp16", 1))
self.assertIn(("empty_cache",), calls)
self.assertEqual(calls[-1], ("interp", "source", "interp", "bilinear"))
self.assertEqual(latent._MODEL_HOLD_DEPTH, 0)
finally:
latent._upscale_video_temporal_chunks = original_temporal
latent.upscale_video_interp = original_interp
latent._unload_upscale_model_now = original_unload_now
latent.torch.cuda = original_cuda
if original_device is None:
delattr(latent.torch, "device")
else:
latent.torch.device = original_device
def test_model_upscale_skips_learned_model_on_8gb_cuda(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
calls = []
original_temporal = latent._upscale_video_temporal_chunks
original_interp = latent.upscale_video_interp
original_cuda = latent.torch.cuda
try:
latent.torch.cuda = types.SimpleNamespace(
is_available=lambda: True,
mem_get_info=lambda: (1 * 1024 * 1024 * 1024, 8 * 1024 * 1024 * 1024),
empty_cache=lambda: calls.append(("empty_cache",)),
)
def temporal(_video, _param, _upscaler):
calls.append(("temporal",))
raise AssertionError("learned model path should be skipped on 8GB CUDA")
def interp(video, param):
calls.append(("interp", video, param.get("mode"), param.get("method")))
return "interp_video", 8, 16
latent._upscale_video_temporal_chunks = temporal
latent.upscale_video_interp = interp
result = latent.upscale_latent_video("source", {
"mode": "model",
"model_name": "upscale.safetensors",
"method": "bilinear",
"device": "cuda",
"precision": "fp16",
})
self.assertEqual(result, ("interp_video", 8, 16))
self.assertNotIn(("temporal",), calls)
self.assertIn(("empty_cache",), calls)
self.assertEqual(calls[-1], ("interp", "source", "interp", "bilinear"))
finally:
latent._upscale_video_temporal_chunks = original_temporal
latent.upscale_video_interp = original_interp
latent.torch.cuda = original_cuda
def test_upscale_video_model_raises_when_gpu_cannot_shrink(self):
latent = importlib.import_module("dumas_h3_latent_upscale")
+296
View File
@@ -356,6 +356,289 @@ class DumasImageNodeTests(unittest.TestCase):
required = self.image_nodes.DumasLocationReferenceNode.INPUT_TYPES()["required"]
self.assertNotIn("picture_id", required)
def test_character_helper_restores_image_and_text_outputs(self):
node = self.image_nodes.DumasCharacterHelperNode()
image1 = FakeTensorBatch()
image2 = FakeTensorBatch()
result = node.build_character_text(
image1=image1,
image2=image2,
image1_picture_id="1",
image2_picture_id="2",
character_id="char_dave",
name="Dave",
alias="The Locksmith",
gender="male",
age="41",
nationality="English",
occupation="a detective",
height_feet="6",
height_inches="2",
accent="English",
general="Moves carefully and notices every exit",
wardrobe="weathered red flight jacket, grey cargo shorts, black boots",
)
self.assertIs(result[0], image1)
self.assertIs(result[1], image2)
self.assertIn("<Picture 1> and <Picture 2> reference the same character", result[2])
self.assertIn("Dave is also known as The Locksmith", result[2])
self.assertIn("is 41 years old", result[2])
self.assertEqual(result[3], "Dave = weathered red flight jacket, grey cargo shorts, black boots")
self.assertIs(result[4]["image"], image1)
self.assertIs(result[5]["image"], image2)
self.assertEqual(result[4]["id"], "char-dave")
self.assertEqual(result[5]["id"], "char-dave")
self.assertEqual(result[4]["name"], "Dave")
self.assertEqual(result[4]["aliases"], ["The Locksmith"])
self.assertEqual(result[4]["facts"]["age"], "41")
self.assertEqual(result[4]["facts"]["height_feet"], "6")
self.assertEqual(result[4]["facts"]["height_inches"], "2")
self.assertEqual(result[4]["wardrobe"], "weathered red flight jacket, grey cargo shorts, black boots")
self.assertEqual(len(result), 6)
def test_location_helper_matches_character_helper_shape_without_wardrobe(self):
node = self.image_nodes.DumasLocationHelperNode()
image1 = FakeTensorBatch()
image2 = FakeTensorBatch()
result = node.build_location_text(
image1=image1,
image2=image2,
image1_picture_id="3",
image2_picture_id="4",
location_id="coffee-shop-01",
name="Coffee Shop",
alias="Cafe Interior",
description="Warm tungsten lighting, narrow counter, rainy front window",
general="Evening ambience, cramped but cozy",
)
self.assertIs(result[0], image1)
self.assertIs(result[1], image2)
self.assertIn("<Picture 3> and <Picture 4> reference the same location", result[2])
self.assertIn("Coffee Shop is also known as Cafe Interior", result[2])
self.assertIn("Warm tungsten lighting, narrow counter, rainy front window.", result[2])
self.assertIn("Evening ambience, cramped but cozy.", result[2])
self.assertIs(result[3]["image"], image1)
self.assertIs(result[4]["image"], image2)
self.assertEqual(result[3]["kind"], "location")
self.assertEqual(result[4]["kind"], "location")
self.assertEqual(result[3]["id"], "coffee-shop-01")
self.assertEqual(result[4]["id"], "coffee-shop-01")
self.assertEqual(result[3]["name"], "Coffee Shop")
self.assertEqual(result[3]["aliases"], ["Cafe Interior"])
self.assertEqual(result[3]["description"], "Warm tungsten lighting, narrow counter, rainy front window")
self.assertEqual(result[3]["general"], "Evening ambience, cramped but cozy")
self.assertEqual(len(result), 5)
def test_soundscape_helper_defaults_to_selected_preset_description(self):
node = self.image_nodes.DumasSoundscapeHelperNode()
result = node.build_soundscape("rainy street", "")
self.assertEqual(result[0], "steady rain, wet pavement, distant traffic hum")
def test_background_music_helper_defaults_to_selected_preset_description(self):
node = self.image_nodes.DumasBackgroundMusicHelperNode()
result = node.build_bgm("subtle tension", "")
self.assertEqual(result[0], "low, restrained tension bed with sparse pulses and no vocals")
def test_h3_prompt_curator_compacts_named_references(self):
node = self.image_nodes.DumasH3PromptCuratorNode()
dave_image = FakeTensorBatch()
cafe_image = FakeTensorBatch()
van_image = FakeTensorBatch()
dave = self.image_nodes.make_reference(
kind="character",
image=dave_image,
name="Dave",
aliases="The Locksmith",
description="tired eyes, cropped brown hair",
wardrobe="red flight jacket",
)
cafe = self.image_nodes.make_reference(
kind="location",
image=cafe_image,
name="Coffee Shop",
description="warm tungsten lighting and rainy windows",
)
van = self.image_nodes.make_reference(
kind="location",
image=van_image,
name="Blue Van",
description="scuffed blue delivery van",
)
result = node.curate_prompt(
action_prompt="Dave runs from the Coffee Shop into the rain.",
anatomy_guard="auto",
subject_count_guard="auto",
anchor="grounded handheld thriller",
soundscape="steady rain",
bgm="low suspense music",
ref_1=dave,
ref_2=van,
ref_3=cafe,
)
prompt = result[0]
self.assertIn("<Picture 1> Dave", prompt)
self.assertIn("<Picture 2> Coffee Shop", prompt)
self.assertIn("Action: Dave runs from the Coffee Shop into the rain.", prompt)
self.assertIn("Anatomy guard:", prompt)
self.assertIn("Subject count guard:", prompt)
self.assertIn("overall_soundscape: steady rain", prompt)
self.assertIn("background_music: low suspense music", prompt)
self.assertIn("exactly one named character: <Picture 1> Dave", prompt)
self.assertIs(result[1], dave_image)
self.assertIs(result[2], cafe_image)
self.assertIsNone(result[3])
self.assertEqual(result[10], 2)
self.assertIn("input 3-><Picture 2> Coffee Shop", result[11])
def test_h3_prompt_curator_renumbers_explicit_reference_tags(self):
node = self.image_nodes.DumasH3PromptCuratorNode()
image1 = FakeTensorBatch()
image3 = FakeTensorBatch()
unused = FakeTensorBatch()
first = self.image_nodes.make_reference(kind="character", image=image1, name="Maya")
second = self.image_nodes.make_reference(kind="location", image=unused, name="Lobby")
third = self.image_nodes.make_reference(kind="location", image=image3, name="Rooftop")
result = node.curate_prompt(
action_prompt="<Picture 1> Maya crosses to <ref3> as the wind rises.",
anatomy_guard="off",
subject_count_guard="off",
ref_1=first,
ref_2=second,
ref_3=third,
)
prompt = result[0]
self.assertIn("<Picture 1> Maya crosses to <Picture 2>", prompt)
self.assertNotIn("<Picture 3>", prompt)
self.assertIs(result[1], image1)
self.assertIs(result[2], image3)
self.assertIsNone(result[3])
self.assertEqual(result[10], 2)
def test_h3_prompt_curator_can_force_subject_count_without_character_refs(self):
node = self.image_nodes.DumasH3PromptCuratorNode()
result = node.curate_prompt(
action_prompt="A locked-off shot of the empty corridor.",
anatomy_guard="off",
subject_count_guard="on",
)
self.assertIn("Subject count guard:", result[0])
self.assertIn("Only include the people explicitly described", result[0])
self.assertEqual(result[10], 0)
def test_h3_prompt_curator_treats_helper_image_pair_as_one_character(self):
helper = self.image_nodes.DumasCharacterHelperNode()
curator = self.image_nodes.DumasH3PromptCuratorNode()
image1 = FakeTensorBatch()
image2 = FakeTensorBatch()
helper_result = helper.build_character_text(
image1=image1,
image2=image2,
image1_picture_id="1",
image2_picture_id="2",
character_id="char_dave",
name="Dave",
alias="The Locksmith",
gender="male",
age="41",
nationality="English",
occupation="detective",
height_feet="6",
height_inches="2",
accent="English",
general="Tired eyes, cropped brown hair",
wardrobe="weathered red flight jacket",
)
result = curator.curate_prompt(
action_prompt="Dave checks the locked door.",
anatomy_guard="on",
subject_count_guard="auto",
ref_1=helper_result[4],
ref_2=helper_result[5],
)
self.assertIs(result[1], image1)
self.assertIs(result[2], image2)
self.assertEqual(result[10], 2)
self.assertIn("Character facts for <Picture 1> Dave", result[0])
self.assertIn("41 years old", result[0])
self.assertIn("6 foot 2 tall", result[0])
self.assertIn("exactly one named character: <Picture 1> Dave", result[0])
self.assertNotIn("exactly 2 named characters", result[0])
def test_h3_prompt_curator_uses_location_helper_references_by_name(self):
helper = self.image_nodes.DumasLocationHelperNode()
curator = self.image_nodes.DumasH3PromptCuratorNode()
image1 = FakeTensorBatch()
image2 = FakeTensorBatch()
helper_result = helper.build_location_text(
image1=image1,
image2=image2,
image1_picture_id="1",
image2_picture_id="2",
location_id="coffee_shop",
name="Coffee Shop",
alias="Cafe Interior",
description="Warm tungsten lighting, narrow counter, rainy front window",
general="Evening ambience, cramped but cozy",
)
result = curator.curate_prompt(
action_prompt="A slow push through the Coffee Shop as rain streaks the windows.",
anatomy_guard="on",
subject_count_guard="auto",
ref_1=helper_result[3],
ref_2=helper_result[4],
)
self.assertIs(result[1], image1)
self.assertIs(result[2], image2)
self.assertEqual(result[10], 2)
self.assertIn("<Picture 1> Coffee Shop", result[0])
self.assertIn("<Picture 2> Coffee Shop", result[0])
self.assertIn("Location context for <Picture 1> Coffee Shop", result[0])
self.assertIn("Warm tungsten lighting", result[0])
self.assertNotIn("Subject count guard:", result[0])
def test_h3_prompt_curator_defaults_anatomy_guard_to_on(self):
required = self.image_nodes.DumasH3PromptCuratorNode.INPUT_TYPES()["required"]
self.assertEqual(required["anatomy_guard"][1]["default"], "on")
def test_helper_node_mappings_use_general_purpose_helpers(self):
mappings = self.image_nodes.NODE_CLASS_MAPPINGS
display = self.image_nodes.NODE_DISPLAY_NAME_MAPPINGS
self.assertIs(mappings["DumasCharacterHelper"], self.image_nodes.DumasCharacterHelperNode)
self.assertIs(mappings["DumasLocationHelper"], self.image_nodes.DumasLocationHelperNode)
self.assertIs(mappings["DumasSoundscapeHelper"], self.image_nodes.DumasSoundscapeHelperNode)
self.assertIs(mappings["DumasBackgroundMusicHelper"], self.image_nodes.DumasBackgroundMusicHelperNode)
self.assertIs(mappings["DumasH3PromptCurator"], self.image_nodes.DumasH3PromptCuratorNode)
self.assertEqual(display["DumasCharacterHelper"], "Dumas Character Helper")
self.assertEqual(display["DumasLocationHelper"], "Dumas Location Helper")
self.assertEqual(display["DumasSoundscapeHelper"], "Dumas Soundscape Helper")
self.assertEqual(display["DumasBackgroundMusicHelper"], "Dumas Background Music Helper")
self.assertEqual(display["DumasH3PromptCurator"], "Dumas H3 Prompt Curator")
def test_h3_prompt_curator_uses_documented_reference_limits(self):
node = self.image_nodes.DumasH3PromptCuratorNode()
self.assertEqual(len(node.RETURN_TYPES), 12)
self.assertEqual(node.RETURN_NAMES[1:10], tuple(f"ref_image_{i}" for i in range(1, 10)))
def test_normalize_reference_upgrades_generic_summary_with_socket_picture_id(self):
image = FakeTensorBatch()
@@ -417,6 +700,19 @@ class DumasImageNodeTests(unittest.TestCase):
self.assertIn("real time", result[0])
self.assertNotIn("persistent camera language", result[0])
def test_anchor_style_node_strips_legacy_persistent_anchor_note(self):
node = self.image_nodes.DumasAnchorStyleNode()
legacy = (
"Gritty handheld realism. Keep this anchor focused on persistent camera "
"language, lighting, texture, environment treatment, and tone; do not "
"name characters or describe one-off actions."
)
result = node.build_anchor("cinematic action movie", legacy)
self.assertEqual(result[0], "Gritty handheld realism.")
self.assertNotIn("persistent camera language", result[0])
def test_anchor_style_node_prefers_manual_description_edits(self):
node = self.image_nodes.DumasAnchorStyleNode()
custom = "Lo-fi pirate broadcast with smeared highlights and anxious zoom corrections."