feat: finalize FL2VA terminal merge

This commit is contained in:
ukr8b3g-cmyk
2026-08-22 21:25:34 +09:00
parent 498c467b55
commit d77a467796
23 changed files with 2921 additions and 21 deletions
+47 -1
View File
@@ -51,7 +51,8 @@ Static checks do not replace a real GPU generation. Record workflow, prompt, med
- PASS `MiniMax_H3_Continuum_V31_00032_.mp4`: standard Ref2VA model; visual result accepted. - PASS `MiniMax_H3_Continuum_V31_00032_.mp4`: standard Ref2VA model; visual result accepted.
- FAIL `MiniMax_H3_Continuum_V31_00034_.mp4`: Hybrid FLF+Reference; visible jumps near 5 s and 9 s. - FAIL `MiniMax_H3_Continuum_V31_00034_.mp4`: Hybrid FLF+Reference; visible jumps near 5 s and 9 s.
- FAIL `MiniMax_H3_Continuum_V31_00035_.mp4`: Ref2VA model plus FL2V Turbo 8-step LoRA; two visible flashes/jumps. - FAIL `MiniMax_H3_Continuum_V31_00035_.mp4`: Ref2VA model plus FL2V Turbo 8-step LoRA; two visible flashes/jumps.
- Current FLF baseline is `context_bound_terminal_v1`: final-chunk context-tail and Last Frame anchors only. - Current FLF candidate is `context_bound_terminal_last_only_v2`: the final chunk keeps its 22-frame continuation context and the single Last Frame anchor retained by `prepare_conditioning()`, without adding a duplicate context-tail boundary keyframe.
- GPU verification of this candidate is pending; it is not yet a stable baseline.
- `context_bound_terminal_bridge_v2` is withdrawn because its synthetic intermediate latent correlated with the later terminal flash regression. - `context_bound_terminal_bridge_v2` is withdrawn because its synthetic intermediate latent correlated with the later terminal flash regression.
- Preserve Hybrid/Reference support, Run Storage identity, UI, audio paths, and permissive prompt handling when repairing FLF. - Preserve Hybrid/Reference support, Run Storage identity, UI, audio paths, and permissive prompt handling when repairing FLF.
@@ -78,3 +79,48 @@ Static checks do not replace a real GPU generation. Record workflow, prompt, med
- The initial rollback after 00038 temporarily returned to bridge v2, but the user clarified that the required target is the exact code state used for PASS 00031. - The initial rollback after 00038 temporarily returned to bridge v2, but the user clarified that the required target is the exact code state used for PASS 00031.
- `00031` completed at 2026-08-21 02:32:59; `pre-flf-terminal-bridge-v2-repo-20260821_023553` was captured immediately afterward and before bridge v2 was introduced. - `00031` completed at 2026-08-21 02:32:59; `pre-flf-terminal-bridge-v2-repo-20260821_023553` was captured immediately afterward and before bridge v2 was introduced.
- Current FLF strategy is restored exactly from that snapshot: `context_bound_terminal_v1`. Bridge v2 is withdrawn and is not active. - Current FLF strategy is restored exactly from that snapshot: `context_bound_terminal_v1`. Bridge v2 is withdrawn and is not active.
## M1 Hybrid diagnostics (2026-08-21)
- Canonical implementation source: ComfyUI_W runtime.
- Runtime pre-change snapshot: D:\Codex\_snapshots\ComfyUI-H3-Continuum\pre-m1-comfyui-w-canonical-20260821_172324.
- Added display-only Hybrid conditioning labels; the stored/runtime conditioning mode remains unchanged.
- Hybrid reference warnings now use the actual global Qwen <Picture N> numbering.
- Added README numbering guidance and focused CPU-only regression tests.
- No Sampling, Conditioning payload, Run Storage contract, UI socket, or model-selection behavior changed.
- Tests were added but not executed in this implementation step.
## Terminal Merge Physical Seed v2 (2026-08-22 00:49:57)
- Terminal Merge uses one physical seed per merged pair; an initial 2x5s pair uses base_seed directly when reroll is inactive.
- Later terminal pairs derive once from the zero-based pair-start index; both logical entries store that same physical seed.
- Run Storage distinguishes the scoped terminal_merged_10s_seed_v2 / physical_window_seed_v2 execution semantics without changing the global schema version.
## Terminal Merge diagnostic harness (2026-08-22)
- Production generation semantics remain unchanged; diagnostics live under tests only.
- Diagnostic stages are ordered A Pre-Sampler, B First MODEL Call, C sampled physical AV, D split/recombine, E decode/assembly.
- Tensor identity uses shape, dtype, and SHA-256 of contiguous CPU raw bytes; numerical mismatch also records max/mean absolute difference and allclose.
- Terminal split/recombine requires bit-exact equality.
- Real Core vs Continuum GPU capture remains pending; this unit-test phase does not load checkpoints.
## GPU Core/Continuum diagnostic runner (2026-08-22)
- Source: `tools/gpu_diagnostic_node/`
- Installer: `tools/install_gpu_diagnostic_node.ps1`
- Runtime: `D:\StabilityMatrix\Data\Packages\ComfyUI_W\custom_nodes\H3_Continuum_GPU_Diagnostics`
- Node: `H3 Core vs Continuum GPU Diagnostic`
- Production Continuum files and generation semantics are not modified by this diagnostic plugin.
- Captures A: pre-sampler, B: first MODEL call, C: physical sampled AV latent, D: terminal split/recombine, E1: decode comparison, E2: common 243-to-240 adjustment.
- Artifacts: JSON manifest plus safetensors under the configured ComfyUI output diagnostics directory.
- Pre-change snapshot: `D:\Codex\_snapshots\ComfyUI-H3-Continuum\pre-gpu-core-continuum-diagnostic-runner-20260822_022625` (138/138 files verified).
- GPU comparison has not yet been executed.
## Long Terminal Merge for distinct timeline sections (2026-08-22)
- For FL2VA with 3 or more 5-second logical chunks, the final two chunks may now share one physical Terminal Merge sample even when their prompt sections differ.
- Distinct final prompts are rebased to a local physical timeline (`[0-5s]` and `[5-10s]`); identical prompts retain the accepted 2x5 Core-equivalent prompt path unchanged.
- Expected 3x5 execution: 3 logical chunks, 2 physical sampling passes, and 2 physical VAE decode groups; the terminal group is 260 frames with a 22-frame decoded prefix trim and 238 retained frames.
- Timeline Video remains excluded from Terminal Merge. UI, public sockets, Conditioning payload construction, Physical Seed v2, Assembly, and Seam semantics are otherwise unchanged.
- GPU acceptance passed for `MiniMax_H3_Continuum_V31_00008_.mp4`: 3 logical chunks, 2 physical sampling passes, decoded groups of 124 and 238 retained frames, 362-to-360 exact-duration adjustment, and 15.000-second output.
- Core comparison `MiniMax_H3_00005_.mp4` showed the same short terminal still period. This is accepted as normal FL2VA Last Frame convergence, not a Continuum-specific defect.
- Long Terminal Merge is formally adopted. Do not retune Conditioning, Physical Seed v2, physical decode grouping, or terminal prompt pairing without new contrary regression evidence.
+17
View File
@@ -15,6 +15,7 @@ V3.4 focuses on the two reference workflows that are most useful in normal produ
- **Driving Audio**: use an existing audio source as the preserved final audio while it guides the H3 generation across chunks. - **Driving Audio**: use an existing audio source as the preserved final audio while it guides the H3 generation across chunks.
- **Video Reference**: use an existing video as persistent visual reference for appearance, motion, framing, and timing without treating it as a frame-by-frame copy. - **Video Reference**: use an existing video as persistent visual reference for appearance, motion, framing, and timing without treating it as a frame-by-frame copy.
- **Hybrid FLF + Reference**: use First Frame, Last Frame, and persistent Reference Images together without adding a separate mode or model allowlist. - **Hybrid FLF + Reference**: use First Frame, Last Frame, and persistent Reference Images together without adding a separate mode or model allowlist.
- **FL2VA Terminal Merge**: keep the final two 5-second FL2VA chunks as one physical 10-second sampling and decode unit for Core-equivalent Last Frame handling.
- **Restartable chunks**: reuse completed chunks with Run Storage and regenerate only the selected part when the generation contract remains compatible. - **Restartable chunks**: reuse completed chunks with Run Storage and regenerate only the selected part when the generation contract remains compatible.
- **Core compatibility**: remove Continuum-only rejection of unknown upstream/custom nodes and remove the obsolete `strict_compatibility` control. - **Core compatibility**: remove Continuum-only rejection of unknown upstream/custom nodes and remove the obsolete `strict_compatibility` control.
- **Spectrum interoperability**: use the official H3 Continuum Interop API v1 when Spectrum is installed; Spectrum remains optional. - **Spectrum interoperability**: use the official H3 Continuum Interop API v1 when Spectrum is installed; Spectrum remains optional.
@@ -83,6 +84,18 @@ V3.4 supports First Frame, Last Frame, and persistent Reference Images in the sa
Local checks passed with the B2049 hybrid variant. FL-only models could run, but showed less reliable transitions when FLF and Reference Images were combined; a hybrid-capable model is therefore recommended for this path. Local checks passed with the B2049 hybrid variant. FL-only models could run, but showed less reliable transitions when FLF and Reference Images were combined; a hybrid-capable model is therefore recommended for this path.
### FL2VA Terminal Merge
For 5-second FL2VA workflows, Continuum keeps the final two logical chunks together as one physical sampling and VAE decode unit.
- `2 x 5s`: one 243-frame physical sample, decoded once, then adjusted to the requested 240-frame output.
- `3 x 5s` and longer: earlier chunks use normal Continuation; the final two chunks use one 260-frame physical sample containing the 22-frame continuation context.
- After the terminal physical decode, the 22-frame context prefix is removed. For `3 x 5s`, this gives `124 + 238 = 362` frames before exact-duration adjustment to 360 frames.
- If the final two prompt sections differ, they are rebased internally to a local `[0-5s]` / `[5-10s]` timeline. Identical prompts retain the accepted shared-prompt path.
- The terminal pair uses one physical seed and one physical decode while Run Storage continues to preserve normal logical chunk entries.
Local GPU comparison confirmed that `2 x 5s` matches Core 10-second FL2VA sampling and terminal decode behavior. A `3 x 5s` Timeline run also passed with three logical chunks, two physical sampling passes, two physical decode groups, and a 15.000-second output. The short still period near the supplied Last Frame was also present in the equivalent Core result and is treated as normal FL2VA convergence rather than a Continuum-specific failure.
### Core-first, permissive execution ### Core-first, permissive execution
V3.4 removes or relaxes Continuum-only rejection rules that blocked otherwise runnable Core H3 experiments. V3.4 removes or relaxes Continuum-only rejection rules that blocked otherwise runnable Core H3 experiments.
@@ -295,6 +308,8 @@ V3.4 has been exercised locally with:
- Spectrum enabled and disabled - Spectrum enabled and disabled
- I2VA, Reference, and selected FL2VA configurations - I2VA, Reference, and selected FL2VA configurations
- Hybrid FLF + Reference with the B2049 hybrid variant - Hybrid FLF + Reference with the B2049 hybrid variant
- Core-equivalent `2 x 5s` FL2VA Terminal Merge
- `3 x 5s` FL2VA with the final two logical chunks merged into one 260-frame physical sample and decode group
- Driving Audio with short, exact-length, and longer sources - Driving Audio with short, exact-length, and longer sources
- Video Reference with source audio routed separately - Video Reference with source audio routed separately
- 0.4 MP and 0.6 MP reference sizing - 0.4 MP and 0.6 MP reference sizing
@@ -308,6 +323,8 @@ No OOM was observed in the cited recent local V3.4 checks, including two-chunk 8
- Continuation does not guarantee frame-perfect identity or motion. - Continuation does not guarantee frame-perfect identity or motion.
- Video Reference guides H3; it does not reproduce every source frame. - Video Reference guides H3; it does not reproduce every source frame.
- Driving Audio preserves selected audio, but visual lip synchronization remains model-dependent. - Driving Audio preserves selected audio, but visual lip synchronization remains model-dependent.
- FL2VA may settle on the supplied Last Frame before the requested duration ends; equivalent Core runs can show the same behavior.
- Long, high-resolution sequences may exceed system RAM during final decode and assembly even when chunk sampling succeeds.
- Match Output can be substantially slower than 0.4 MP or 0.6 MP. - Match Output can be substantially slower than 0.4 MP or 0.6 MP.
- Seam correction may keep the native boundary when a proposed correction is not safer. - Seam correction may keep the native boundary when a proposed correction is not safer.
- Optional template nodes must be installed, replaced, or bypassed by the user. - Optional template nodes must be installed, replaced, or bypassed by the user.
+62
View File
@@ -60,3 +60,65 @@ D:\Codex\_snapshots\ComfyUI-H3-Continuum\pre-rollback-after-00038-fail-20260821_
- Restored only `v2/sequence.py` from `pre-flf-terminal-bridge-v2-repo-20260821_023553` to the repository and active ComfyUI_W runtime. - Restored only `v2/sequence.py` from `pre-flf-terminal-bridge-v2-repo-20260821_023553` to the repository and active ComfyUI_W runtime.
- Active FLF strategy is `context_bound_terminal_v1`; bridge v2 is no longer active. - Active FLF strategy is `context_bound_terminal_v1`; bridge v2 is no longer active.
- No UI or unrelated feature files were changed. GPU retest was not run. - No UI or unrelated feature files were changed. GPU retest was not run.
## 2026-08-21 - M1 Hybrid diagnostics
- Used ComfyUI_W as the authoritative source after confirming repository/runtime differences.
- Created and verified a runtime snapshot before edits.
- Implemented display-only Hybrid labels and global Picture-number warnings.
- Documented Hybrid Picture numbering and added focused regression tests.
- Runtime generation behavior and public UI were intentionally left unchanged.
- Tests were not run in this implementation step.
## 2026-08-21 - FLF duplicate boundary anchor removal
- Preserved the active ComfyUI_W M3 display-label and M4 Last Frame session checks while synchronizing `v2/sequence.py` to the repository.
- Changed the FLF contract to `context_bound_terminal_last_only_v2` so Run Storage cannot reuse chunks produced by the previous strategy.
- Removed only the final-chunk call that duplicated the continuation-context tail as a boundary keyframe.
- Kept the 22-frame continuation context and the single supplied Last Frame anchor retained by `prepare_conditioning()`.
- No UI, public sockets, Assembly, audio, Spectrum, or general continuation behavior changed.
- Automated and GPU validation were not run in this implementation step.
- 2026-08-22 00:49:57 Terminal Merge Physical Seed v2 implemented locally after snapshot pre-change-20260822_004423. Changed v2/sequence.py and run_storage.py; added seed and revision regressions. No commit or push.
- 2026-08-22 00:51:45 Validation: 220 passed, 1 pre-existing failure in test_v2_sequence_orchestration because conditioning_display_label is absent from conditioning.py. All Physical Seed v2 and Run Storage revision tests passed. GPU generation not run.
## 2026-08-22 - Terminal Merge staged diagnostics
- Snapshot: D:\Codex\_snapshots\ComfyUI-H3-Continuum\pre-terminal-merge-diagnostics-20260822_020527
- Added tests-only deterministic capture/comparison helpers for stages A-E.
- Added unit coverage for noise, PackedLayout/position_ids, sampled-stage ordering, numerical tolerance, and bit-exact 243f Terminal split/recombine.
- No Production code, UI, Conditioning, Seed v2, Terminal Merge, Assembly, or Seam behavior changed.
- Commit/push not performed.
## 2026-08-22 - Standalone Core vs Continuum GPU diagnostics
- Added a standalone diagnostic node without changing Production runtime semantics.
- Added capture helpers and regression tests.
- Installed to ComfyUI_W as a separate custom node and verified source/runtime SHA-256 equality.
- Registration import: PASS (`H3CoreContinuumGpuDiagnostic`).
- Targeted tests: 6 passed.
- Full suite: 232 passed, 1 pre-existing unrelated failure (`conditioning_display_label` missing for unfinished M3 wiring).
- GPU execution: not run.
- Commit/push: not performed.
## 2026-08-22 - Terminal Merge physical decode groups
- Preserved logical chunks for Session and Run Storage.
- Recombined terminal logical AV latent slices bit-exactly before external Core VAE Decode.
- Added assembly_plan.decode_groups so one terminal physical sample is decoded once.
- Assembly hardening and Video Seam now operate at physical decode-group boundaries.
- Sampling, Physical Seed v2, Conditioning, UI, sockets, and Run Storage schema were unchanged.
- Synced implementation to ComfyUI_W.
- Pending validation: targeted pytest, full pytest, and GPU Core-vs-Continuum decode comparison.
## 2026-08-22 - Long Terminal Merge prompt pairing
- Snapshots: `pre-long-terminal-merge-repo`, `pre-long-terminal-merge-comfyui-w`, and `pre-long-terminal-merge-comfyui-wan` under `D:\Codex\_snapshots\ComfyUI-H3-Continuum`.
- Removed only the final-prompt-hash equality gate from Terminal Merge eligibility.
- Added a scoped physical prompt policy: identical prompts remain byte-identical; distinct final logical prompts are rebased to `[0-5s]` / `[5-10s]` for one terminal physical sample.
- Added scoped Run Storage execution semantics and 3-logical-to-2-physical decode-group regression coverage.
- No UI, public sockets, model selection, Spectrum, Assembly, Seam, or global Run Storage schema changes.
- Focused regression result: 22 passed; GPU generation not yet run for the new 3x5 path.
- Runtime correction: `run_storage.py` was also synchronized to ComfyUI_W and ComfyUI_WAN after the first GPU attempt exposed an old `RunStorageController.prepare()` signature. Run Storage plus Terminal Merge regressions: 47 passed.
- GPU PASS: `MiniMax_H3_Continuum_V31_00008_.mp4` exercised 3x5s FL2VA with `terminal_pair=2-3`, `physical_frames=260`, `sampling_passes=1` for the pair, `paired_timeline_v1`, two physical decode groups, and final 360 frames / 15.000 seconds.
- Core comparison `MiniMax_H3_00005_.mp4` reproduced the same short terminal still period. Long Terminal Merge was formally accepted; terminal stillness is Core-equivalent FL2VA convergence and is not a Continuum repair target.
- Final release validation: full pytest `249 passed`; `tools/verify_runtime.py --comfy-root D:\StabilityMatrix\Data\Packages\ComfyUI_W` passed V3.4 registration, native PackedLayout, and Fixed 3x5s prompt-plan checks. GPU acceptance is recorded above.
+40
View File
@@ -59,6 +59,46 @@ def conditioning_mode_label(mode: str) -> str:
raise ValueError(f"unknown conditioning mode: {mode!r}") from exc raise ValueError(f"unknown conditioning mode: {mode!r}") from exc
def conditioning_display_label(
*,
has_first: bool,
has_last: bool,
has_reference: bool,
) -> str:
"""Return a user-facing label without changing the runtime mode contract."""
if has_reference:
if has_first and has_last:
return "Hybrid FL2VA + Reference + Continuation"
if has_first:
return "Hybrid I2VA + Reference + Continuation"
if has_last:
return "Hybrid L2VA + Reference + Continuation"
return "Reference + Continuation"
mode = conditioning_mode_from_presence(
has_first=has_first,
has_last=has_last,
has_reference=False,
)
return conditioning_mode_label(mode)
def conditioning_mode_display_label(
mode: str,
*,
has_first: bool = False,
has_last: bool = False,
has_reference: bool = False,
) -> str:
"""Legacy display helper retained for existing callers and tests."""
if has_first or has_last or has_reference:
return conditioning_display_label(
has_first=has_first,
has_last=has_last,
has_reference=has_reference,
)
return conditioning_mode_label(mode)
def conditioning_mode_uses_video_vae(mode: str) -> bool: def conditioning_mode_uses_video_vae(mode: str) -> bool:
if mode not in CONDITIONING_MODES: if mode not in CONDITIONING_MODES:
raise ValueError(f"unknown conditioning mode: {mode!r}") raise ValueError(f"unknown conditioning mode: {mode!r}")
+124 -2
View File
@@ -21,6 +21,15 @@ def _strict_int(name: str, value: Any, minimum: int | None = None) -> int:
return value return value
def _assembly_decode_units(plan: Mapping[str, Any]) -> list[Any]:
if "decode_groups" not in plan:
return plan["chunks"]
groups = plan["decode_groups"]
if not isinstance(groups, list) or not groups:
raise ValueError("assembly_plan.decode_groups must be a non-empty list")
return groups
def enrich_assembly_plan(plan: dict[str, Any]) -> dict[str, Any]: def enrich_assembly_plan(plan: dict[str, Any]) -> dict[str, Any]:
"""Add redundant metadata without correcting any existing value.""" """Add redundant metadata without correcting any existing value."""
chunks = plan.get("chunks") chunks = plan.get("chunks")
@@ -184,6 +193,117 @@ def validate_assembly_plan_contract(plan: Mapping[str, Any]) -> None:
"sum(chunk.net_frames) does not match assembly_plan.natural_frames" "sum(chunk.net_frames) does not match assembly_plan.natural_frames"
) )
if "decode_groups" in plan:
decode_groups = _assembly_decode_units(plan)
if "physical_decode_group_count" in plan:
group_count = _strict_int(
"assembly_plan.physical_decode_group_count",
plan["physical_decode_group_count"],
1,
)
if group_count != len(decode_groups):
raise ValueError(
"assembly_plan.physical_decode_group_count does not match decode_groups"
)
expected_logical_indices = [int(chunk["chunk_index"]) for chunk in chunks]
covered_logical_indices: list[int] = []
decode_natural_frames = 0
for position, group in enumerate(decode_groups, start=1):
if not isinstance(group, Mapping):
raise ValueError(
f"assembly_plan decode group {position} must be a mapping"
)
total_frames = _strict_int(
f"assembly_plan decode group {position}.total_frames",
group.get("total_frames"),
5,
)
trim_frames = _strict_int(
f"assembly_plan decode group {position}.trim_frames",
group.get("trim_frames"),
0,
)
net_frames = _strict_int(
f"assembly_plan decode group {position}.net_frames",
group.get("net_frames"),
1,
)
context_frames = _strict_int(
f"assembly_plan decode group {position}.context_frames",
group.get("context_frames"),
0,
)
_strict_int(
f"assembly_plan decode group {position}.expected_video_latent_t",
group.get("expected_video_latent_t"),
1,
)
_strict_int(
f"assembly_plan decode group {position}.expected_audio_latent_t",
group.get("expected_audio_latent_t"),
1,
)
if (total_frames - 5) % 17 != 0:
raise ValueError(
f"assembly_plan decode group {position}.total_frames is not on "
"the H3 17k+5 grid"
)
if total_frames - trim_frames != net_frames:
raise ValueError(
f"assembly_plan decode group {position}: "
"total_frames - trim_frames != net_frames"
)
if trim_frames > 0 and context_frames != trim_frames:
raise ValueError(
f"assembly_plan decode group {position}: "
"continuation context and trim differ"
)
logical_indices = group.get("logical_chunk_indices")
if not isinstance(logical_indices, list) or not logical_indices:
raise ValueError(
f"assembly_plan decode group {position}.logical_chunk_indices "
"must be a non-empty list"
)
covered_logical_indices.extend(
_strict_int(
f"assembly_plan decode group {position}.logical_chunk_indices",
value,
1,
)
for value in logical_indices
)
if "terminal_merged" in group and type(group["terminal_merged"]) is not bool:
raise ValueError(
f"assembly_plan decode group {position}.terminal_merged must be boolean"
)
frame_start = _strict_int(
f"assembly_plan decode group {position}.frame_start",
group.get("frame_start"),
0,
)
frame_stop = _strict_int(
f"assembly_plan decode group {position}.frame_stop",
group.get("frame_stop"),
1,
)
if (
frame_start != decode_natural_frames
or frame_stop != decode_natural_frames + net_frames
):
raise ValueError(
f"assembly_plan decode group {position}: natural frame boundary "
"is discontinuous"
)
decode_natural_frames += net_frames
if covered_logical_indices != expected_logical_indices:
raise ValueError(
"assembly_plan.decode_groups do not cover logical chunks exactly once"
)
if decode_natural_frames != natural_frames:
raise ValueError(
"sum(decode_group.net_frames) does not match logical natural frames"
)
def preflight_decoded_chunks( def preflight_decoded_chunks(
images: Any, images: Any,
@@ -192,7 +312,7 @@ def preflight_decoded_chunks(
) -> None: ) -> None:
"""Validate all decoded chunks before the assembler allocates output buffers.""" """Validate all decoded chunks before the assembler allocates output buffers."""
validate_assembly_plan_contract(plan) validate_assembly_plan_contract(plan)
chunks = plan["chunks"] chunks = _assembly_decode_units(plan)
if not isinstance(images, (list, tuple)): if not isinstance(images, (list, tuple)):
raise ValueError("decoded images must be a chunk list") raise ValueError("decoded images must be a chunk list")
@@ -518,7 +638,9 @@ def assemble_with_hardening(base: Any, args: tuple[Any, ...], kwargs: dict[str,
lines: list[str] = [] lines: list[str] = []
if diagnostics_is_full(diagnostics): if diagnostics_is_full(diagnostics):
lines.append(format_memory_snapshot("assemble preflight", images, audio)) lines.append(format_memory_snapshot("assemble preflight", images, audio))
natural_frames = sum(int(chunk["net_frames"]) for chunk in plan["chunks"]) natural_frames = sum(
int(chunk["net_frames"]) for chunk in _assembly_decode_units(plan)
)
target_frames = int(plan["target_frames"]) target_frames = int(plan["target_frames"])
correction = target_frames - natural_frames correction = target_frames - natural_frames
if abs(correction) > 2: if abs(correction) > 2:
+5
View File
@@ -571,6 +571,7 @@ def build_sampling_contract(
driving_audio_vae: Any = None, driving_audio_vae: Any = None,
reference_video_contract: dict[str, Any] | None = None, reference_video_contract: dict[str, Any] | None = None,
timeline_video_contract: dict[str, Any] | None = None, timeline_video_contract: dict[str, Any] | None = None,
execution_semantics: dict[str, Any] | None = None,
) -> tuple[dict[str, Any], bool, list[str]]: ) -> tuple[dict[str, Any], bool, list[str]]:
model_value, model_safe = _model_signature(model, model_fingerprint_value) model_value, model_safe = _model_signature(model, model_fingerprint_value)
clip_value, clip_safe = _clip_signature(clip) clip_value, clip_safe = _clip_signature(clip)
@@ -660,6 +661,8 @@ def build_sampling_contract(
} }
if graph_authoritative: if graph_authoritative:
global_contract["upstream_graph"] = dict(upstream_graph_contract) global_contract["upstream_graph"] = dict(upstream_graph_contract)
if execution_semantics is not None:
global_contract["execution_semantics"] = dict(execution_semantics)
if uses_video_vae: if uses_video_vae:
global_contract["video_vae"] = video_vae_value global_contract["video_vae"] = video_vae_value
if has_first: if has_first:
@@ -1017,6 +1020,7 @@ class RunStorageController:
driving_audio_vae: Any = None, driving_audio_vae: Any = None,
reference_video_contract: dict[str, Any] | None = None, reference_video_contract: dict[str, Any] | None = None,
timeline_video_contract: dict[str, Any] | None = None, timeline_video_contract: dict[str, Any] | None = None,
execution_semantics: dict[str, Any] | None = None,
) -> dict[str, Any] | None: ) -> dict[str, Any] | None:
if existing_session is not None: if existing_session is not None:
raise RunStorageError("Run Storage cannot be combined with an explicit Session") raise RunStorageError("Run Storage cannot be combined with an explicit Session")
@@ -1054,6 +1058,7 @@ class RunStorageController:
driving_audio_vae=driving_audio_vae, driving_audio_vae=driving_audio_vae,
reference_video_contract=reference_video_contract, reference_video_contract=reference_video_contract,
timeline_video_contract=timeline_video_contract, timeline_video_contract=timeline_video_contract,
execution_semantics=execution_semantics,
) )
effective_nonce, nonce_decision = self._resolve_effective_nonce( effective_nonce, nonce_decision = self._resolve_effective_nonce(
contract, requested_nonce=int(reroll_nonce), resume_safe=safe contract, requested_nonce=int(reroll_nonce), resume_safe=safe
+378
View File
@@ -0,0 +1,378 @@
"""Tests-only capture and comparison helpers for Terminal Merge diagnostics.
This module deliberately has no production imports or runtime hooks. Tests can
monkeypatch Core and Continuum boundaries, feed the captured values here, and
identify the first pipeline stage that differs without changing generation
semantics.
"""
from __future__ import annotations
from dataclasses import dataclass
from enum import Enum
from hashlib import sha256
import json
from pathlib import Path
from typing import Any, Mapping
import torch
STAGE_ORDER = (
"A_pre_sampler",
"B_first_model_call",
"C_sampled_physical_av",
"D_split_recombine",
"E_decode_assembly",
)
@dataclass(frozen=True)
class TensorFingerprint:
shape: tuple[int, ...]
dtype: str
sha256: str
def as_dict(self) -> dict[str, Any]:
return {
"shape": list(self.shape),
"dtype": self.dtype,
"sha256": self.sha256,
}
@dataclass(frozen=True)
class TensorDifference:
same_shape: bool
same_dtype: bool
bit_exact: bool
allclose: bool
max_abs_diff: float | None
mean_abs_diff: float | None
@dataclass(frozen=True)
class StageSnapshot:
name: str
sha256: str
payload: Any
@dataclass(frozen=True)
class StageComparison:
name: str
equal: bool
core_sha256: str | None
continuum_sha256: str | None
missing_from: str | None = None
def _tensor_raw_bytes(tensor: torch.Tensor) -> bytes:
value = tensor.detach().to("cpu").contiguous()
if value.numel() == 0:
return b""
return value.reshape(-1).view(torch.uint8).numpy().tobytes()
def tensor_fingerprint(tensor: torch.Tensor) -> TensorFingerprint:
if not torch.is_tensor(tensor):
raise TypeError("tensor_fingerprint expects a torch.Tensor")
return TensorFingerprint(
shape=tuple(int(size) for size in tensor.shape),
dtype=str(tensor.dtype),
sha256=sha256(_tensor_raw_bytes(tensor)).hexdigest(),
)
def tensor_difference(
core: torch.Tensor,
continuum: torch.Tensor,
*,
rtol: float = 1.0e-5,
atol: float = 1.0e-6,
) -> TensorDifference:
if not torch.is_tensor(core) or not torch.is_tensor(continuum):
raise TypeError("tensor_difference expects two torch.Tensor values")
same_shape = tuple(core.shape) == tuple(continuum.shape)
same_dtype = core.dtype == continuum.dtype
bit_exact = (
same_shape
and same_dtype
and tensor_fingerprint(core).sha256 == tensor_fingerprint(continuum).sha256
)
if not same_shape:
return TensorDifference(
same_shape=False,
same_dtype=same_dtype,
bit_exact=False,
allclose=False,
max_abs_diff=None,
mean_abs_diff=None,
)
core_value = core.detach().to("cpu")
continuum_value = continuum.detach().to("cpu")
if core_value.is_complex() or continuum_value.is_complex():
core_numeric = core_value.to(torch.complex128)
continuum_numeric = continuum_value.to(torch.complex128)
else:
core_numeric = core_value.to(torch.float64)
continuum_numeric = continuum_value.to(torch.float64)
if core_numeric.numel() == 0:
max_abs_diff = 0.0
mean_abs_diff = 0.0
else:
absolute = (core_numeric - continuum_numeric).abs()
max_abs_diff = float(absolute.max().item())
mean_abs_diff = float(absolute.mean().item())
return TensorDifference(
same_shape=True,
same_dtype=same_dtype,
bit_exact=bit_exact,
allclose=bool(
torch.allclose(
core_numeric,
continuum_numeric,
rtol=rtol,
atol=atol,
equal_nan=True,
)
),
max_abs_diff=max_abs_diff,
mean_abs_diff=mean_abs_diff,
)
def packed_layout_snapshot(layout: Any) -> dict[str, Any]:
return {
"signature": stable_snapshot(getattr(layout, "signature", None)),
"segments": stable_snapshot(getattr(layout, "segments", None)),
"position_ids": stable_snapshot(getattr(layout, "position_ids", None)),
}
def stable_snapshot(value: Any) -> Any:
"""Convert diagnostic data to deterministic JSON-compatible state."""
if torch.is_tensor(value):
return {"kind": "tensor", **tensor_fingerprint(value).as_dict()}
if isinstance(value, TensorFingerprint):
return {"kind": "tensor", **value.as_dict()}
if value is None or isinstance(value, (bool, int, str)):
return value
if isinstance(value, float):
return {"kind": "float", "hex": value.hex()}
if isinstance(value, bytes):
return {
"kind": "bytes",
"length": len(value),
"sha256": sha256(value).hexdigest(),
}
if isinstance(value, (Path, torch.device, torch.dtype)):
return str(value)
if isinstance(value, Enum):
return {
"kind": "enum",
"type": f"{type(value).__module__}.{type(value).__qualname__}",
"value": stable_snapshot(value.value),
}
if isinstance(value, slice):
return {
"kind": "slice",
"start": value.start,
"stop": value.stop,
"step": value.step,
}
if isinstance(value, Mapping):
return {
"kind": "mapping",
"items": [
[str(key), stable_snapshot(item)]
for key, item in sorted(value.items(), key=lambda pair: str(pair[0]))
],
}
if isinstance(value, tuple):
return {"kind": "tuple", "items": [stable_snapshot(item) for item in value]}
if isinstance(value, list):
return {"kind": "list", "items": [stable_snapshot(item) for item in value]}
if isinstance(value, (set, frozenset)):
items = [stable_snapshot(item) for item in value]
items.sort(key=lambda item: json.dumps(item, sort_keys=True, separators=(",", ":")))
return {"kind": type(value).__name__, "items": items}
if callable(value):
return {
"kind": "callable",
"module": getattr(value, "__module__", type(value).__module__),
"qualname": getattr(value, "__qualname__", type(value).__qualname__),
}
if all(hasattr(value, name) for name in ("signature", "segments", "position_ids")):
return {"kind": "packed_layout", **packed_layout_snapshot(value)}
if hasattr(value, "__dict__"):
public_state = {
key: item
for key, item in vars(value).items()
if not key.startswith("_") and not callable(item)
}
return {
"kind": "object",
"type": f"{type(value).__module__}.{type(value).__qualname__}",
"state": stable_snapshot(public_state),
}
raise TypeError(f"Unsupported diagnostic value: {type(value)!r}")
def snapshot_sha256(payload: Any) -> str:
normalized = stable_snapshot(payload)
encoded = json.dumps(
normalized,
ensure_ascii=True,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
return sha256(encoded).hexdigest()
class DiagnosticCapture:
def __init__(self) -> None:
self._stages: dict[str, StageSnapshot] = {}
@property
def stages(self) -> Mapping[str, StageSnapshot]:
return self._stages
def capture(self, stage: str, **payload: Any) -> StageSnapshot:
if stage not in STAGE_ORDER:
raise ValueError(f"Unknown diagnostic stage: {stage}")
if stage in self._stages:
raise ValueError(f"Diagnostic stage already captured: {stage}")
normalized = stable_snapshot(payload)
snapshot = StageSnapshot(
name=stage,
sha256=snapshot_sha256(normalized),
payload=normalized,
)
self._stages[stage] = snapshot
return snapshot
def capture_pre_sampler(
self,
*,
physical_frames: int,
seed: int,
conditioning: Any,
latent: Any,
noise: torch.Tensor,
keyframes: Any = None,
qwen_images: Any = None,
minimax_refs: Any = None,
) -> StageSnapshot:
return self.capture(
"A_pre_sampler",
physical_frames=physical_frames,
seed=seed,
conditioning=conditioning,
latent=latent,
noise=noise,
keyframes=keyframes,
qwen_images=qwen_images,
minimax_refs=minimax_refs,
)
def capture_first_model_call(
self,
*,
minimax_payload: Any,
transformer_options: Any,
) -> StageSnapshot:
return self.capture(
"B_first_model_call",
minimax_payload=minimax_payload,
transformer_options=transformer_options,
)
def capture_sampled_physical_av(
self,
*,
video: torch.Tensor,
audio: torch.Tensor,
) -> StageSnapshot:
return self.capture(
"C_sampled_physical_av",
video=video,
audio=audio,
)
def capture_split_recombine(
self,
*,
physical_video: torch.Tensor,
physical_audio: torch.Tensor,
recombined_video: torch.Tensor,
recombined_audio: torch.Tensor,
) -> StageSnapshot:
return self.capture(
"D_split_recombine",
physical_video=physical_video,
physical_audio=physical_audio,
recombined_video=recombined_video,
recombined_audio=recombined_audio,
video_difference=tensor_difference(physical_video, recombined_video).__dict__,
audio_difference=tensor_difference(physical_audio, recombined_audio).__dict__,
)
def capture_decode_assembly(
self,
*,
physical_decoded: torch.Tensor,
assembled_decoded: torch.Tensor,
) -> StageSnapshot:
return self.capture(
"E_decode_assembly",
physical_decoded=physical_decoded,
assembled_decoded=assembled_decoded,
difference=tensor_difference(physical_decoded, assembled_decoded).__dict__,
)
def compare_captures(
core: DiagnosticCapture,
continuum: DiagnosticCapture,
) -> tuple[StageComparison, ...]:
comparisons = []
for stage in STAGE_ORDER:
core_stage = core.stages.get(stage)
continuum_stage = continuum.stages.get(stage)
missing = None
if core_stage is None and continuum_stage is None:
missing = "both"
elif core_stage is None:
missing = "core"
elif continuum_stage is None:
missing = "continuum"
comparisons.append(
StageComparison(
name=stage,
equal=(
core_stage is not None
and continuum_stage is not None
and core_stage.sha256 == continuum_stage.sha256
),
core_sha256=None if core_stage is None else core_stage.sha256,
continuum_sha256=None if continuum_stage is None else continuum_stage.sha256,
missing_from=missing,
)
)
return tuple(comparisons)
def first_divergent_stage(
core: DiagnosticCapture,
continuum: DiagnosticCapture,
) -> str | None:
for comparison in compare_captures(core, continuum):
if not comparison.equal:
return comparison.name
return None
+81
View File
@@ -0,0 +1,81 @@
from __future__ import annotations
import importlib.util
import sys
from pathlib import Path
import torch
MODULE_PATH = Path(__file__).parents[1] / "tools" / "gpu_diagnostic_node" / "capture.py"
SPEC = importlib.util.spec_from_file_location("h3_gpu_diagnostic_capture", MODULE_PATH)
capture = importlib.util.module_from_spec(SPEC)
assert SPEC.loader is not None
sys.modules[SPEC.name] = capture
SPEC.loader.exec_module(capture)
def test_tensor_fingerprint_is_deterministic():
value = torch.arange(12, dtype=torch.float32).reshape(1, 3, 4)
assert capture.tensor_fingerprint(value) == capture.tensor_fingerprint(value.clone())
def test_tensor_comparison_records_numerical_difference():
left = torch.tensor([1.0, 2.0], dtype=torch.float32)
right = torch.tensor([1.0, 2.000001], dtype=torch.float32)
result = capture.compare_tensors(left, right)
assert not result.bit_exact
assert result.allclose
assert result.matches
assert result.max_abs_diff > 0.0
def test_terminal_video_recombine_is_bit_exact():
physical = torch.arange(72, dtype=torch.float32).reshape(1, 1, 72, 1, 1)
first = physical[:, :, 0:37]
second = physical[:, :, 30:72]
rebuilt = capture.recombine_terminal_streams(
first,
second,
first_range=(0, 37),
second_range=(30, 72),
dim=2,
)
assert torch.equal(rebuilt, physical)
def test_terminal_audio_recombine_is_bit_exact():
physical = torch.arange(405, dtype=torch.float32).reshape(1, 1, 405)
first = physical[..., 0:207]
second = physical[..., 170:405]
rebuilt = capture.recombine_terminal_streams(
first,
second,
first_range=(0, 207),
second_range=(170, 405),
dim=-1,
)
assert torch.equal(rebuilt, physical)
def test_stage_comparison_reports_first_field():
left = capture.DiagnosticCapture("core")
right = capture.DiagnosticCapture("continuum")
left.add_value("A_pre_sampler", "seed", 42)
right.add_value("A_pre_sampler", "seed", 43)
left.add_tensor("A_pre_sampler", "sigmas", torch.ones(2))
right.add_tensor("A_pre_sampler", "sigmas", torch.ones(2))
result = capture.compare_stage(left, right, "A_pre_sampler")
assert result["status"] == "DIFFER"
assert result["first_field"] == "seed"
def test_summary_uses_stage_order_and_first_divergence(tmp_path):
comparison = {
"A_pre_sampler": {"status": "MATCH", "first_field": None},
"B_first_model_call": {"status": "DIFFER", "first_field": "position_ids"},
}
text = capture.format_comparison(comparison, tmp_path)
assert "A_pre_sampler: MATCH" in text
assert "B_first_model_call: DIFFER" in text
assert "First divergence: B_first_model_call.position_ids" in text
+59
View File
@@ -0,0 +1,59 @@
from __future__ import annotations
import pytest
from ComfyUI_H3_Continuum_Join.conditioning import (
conditioning_display_label,
conditioning_mode_from_presence,
)
@pytest.mark.parametrize(
("has_first", "has_last", "has_reference", "expected"),
[
(False, False, False, "T2VA + Continuation"),
(True, False, False, "I2VA + Continuation"),
(False, True, False, "Last Frame + Continuation"),
(True, True, False, "FL2VA + Continuation"),
(False, False, True, "Reference + Continuation"),
(True, False, True, "Hybrid I2VA + Reference + Continuation"),
(False, True, True, "Hybrid L2VA + Reference + Continuation"),
(True, True, True, "Hybrid FL2VA + Reference + Continuation"),
],
)
def test_conditioning_display_label_matrix(
has_first: bool,
has_last: bool,
has_reference: bool,
expected: str,
) -> None:
assert (
conditioning_display_label(
has_first=has_first,
has_last=has_last,
has_reference=has_reference,
)
== expected
)
@pytest.mark.parametrize(
("has_first", "has_last"),
[
(True, False),
(False, True),
(True, True),
],
)
def test_hybrid_display_does_not_change_internal_reference_mode(
has_first: bool,
has_last: bool,
) -> None:
assert (
conditioning_mode_from_presence(
has_first=has_first,
has_last=has_last,
has_reference=True,
)
== "reference"
)
+183
View File
@@ -0,0 +1,183 @@
from __future__ import annotations
from types import SimpleNamespace
import torch
from ComfyUI_H3_Continuum_Join.v2.sequence import (
_split_terminal_merged_latents,
_terminal_pair_contract,
)
from ComfyUI_H3_Continuum_Join.tests.terminal_merge_diagnostics import (
DiagnosticCapture,
first_divergent_stage,
tensor_difference,
tensor_fingerprint,
)
def _capture_all_stages(*, sampled_offset: float = 0.0) -> DiagnosticCapture:
capture = DiagnosticCapture()
capture.capture_pre_sampler(
physical_frames=243,
seed=42,
conditioning=[[torch.tensor([1.0]), {"minimax_frame_count": 243}]],
latent={"video": torch.zeros(1, 24, 72, 2, 2)},
noise=torch.arange(8, dtype=torch.float32),
keyframes=(0, 242),
qwen_images=("first", "last"),
minimax_refs=None,
)
layout = SimpleNamespace(
signature=(1, 2, 3),
segments=(("target_video", 0, 72),),
position_ids=torch.arange(12, dtype=torch.float32).reshape(6, 2),
)
capture.capture_first_model_call(
minimax_payload={
"layout": layout,
"cond_video_latents": torch.ones(1, 24, 2, 2, 2),
"target_video_shape": (1, 24, 72, 2, 2),
"target_audio_shape": (1, 32, 2, 405),
},
transformer_options={"patches": {}},
)
video = torch.full((1, 24, 72, 2, 2), sampled_offset)
audio = torch.zeros(1, 32, 2, 405)
capture.capture_sampled_physical_av(video=video, audio=audio)
capture.capture_split_recombine(
physical_video=video,
physical_audio=audio,
recombined_video=video.clone(),
recombined_audio=audio.clone(),
)
decoded = torch.zeros(243, 2, 2, 3)
capture.capture_decode_assembly(
physical_decoded=decoded,
assembled_decoded=decoded.clone(),
)
return capture
def test_tensor_fingerprint_is_deterministic_and_dtype_sensitive():
value = torch.arange(12, dtype=torch.float32).reshape(3, 4)
first = tensor_fingerprint(value)
second = tensor_fingerprint(value.clone())
assert first == second
assert first.shape == (3, 4)
assert first.dtype == "torch.float32"
assert first.sha256 != tensor_fingerprint(value.to(torch.float64)).sha256
def test_tensor_difference_keeps_hash_mismatch_distinct_from_allclose():
core = torch.tensor([1.0], dtype=torch.float32)
continuum = torch.tensor([1.0 + 1.0e-7], dtype=torch.float32)
difference = tensor_difference(core, continuum)
assert difference.bit_exact is False
assert difference.allclose is True
assert difference.max_abs_diff is not None
assert difference.max_abs_diff > 0.0
assert difference.mean_abs_diff == difference.max_abs_diff
def test_pre_sampler_capture_includes_initial_noise():
core = _capture_all_stages()
continuum = _capture_all_stages()
assert first_divergent_stage(core, continuum) is None
changed = DiagnosticCapture()
changed.capture_pre_sampler(
physical_frames=243,
seed=42,
conditioning=[[torch.tensor([1.0]), {"minimax_frame_count": 243}]],
latent={"video": torch.zeros(1, 24, 72, 2, 2)},
noise=torch.arange(8, dtype=torch.float32) + 1.0,
keyframes=(0, 242),
qwen_images=("first", "last"),
minimax_refs=None,
)
assert first_divergent_stage(core, changed) == "A_pre_sampler"
def test_first_model_call_capture_detects_packed_layout_difference():
core = DiagnosticCapture()
continuum = DiagnosticCapture()
shared = dict(
physical_frames=243,
seed=42,
conditioning=[torch.zeros(1)],
latent=torch.zeros(1),
noise=torch.zeros(1),
)
core.capture_pre_sampler(**shared)
continuum.capture_pre_sampler(**shared)
core_layout = SimpleNamespace(
signature=(1, 2),
segments=(("target_video", 0, 72),),
position_ids=torch.zeros(4, 2),
)
continuum_layout = SimpleNamespace(
signature=(1, 2),
segments=(("target_video", 0, 72),),
position_ids=torch.ones(4, 2),
)
core.capture_first_model_call(
minimax_payload={"layout": core_layout},
transformer_options={},
)
continuum.capture_first_model_call(
minimax_payload={"layout": continuum_layout},
transformer_options={},
)
assert first_divergent_stage(core, continuum) == "B_first_model_call"
def test_first_divergent_stage_uses_pipeline_order():
core = _capture_all_stages(sampled_offset=0.0)
continuum = _capture_all_stages(sampled_offset=1.0)
assert first_divergent_stage(core, continuum) == "C_sampled_physical_av"
def test_terminal_split_recombine_is_bit_exact():
contract = _terminal_pair_contract(initial_pair=True, chunk_seconds=5.0)
video = torch.arange(
1 * 24 * contract["video_slices"][-1][1] * 2 * 2,
dtype=torch.float32,
).reshape(1, 24, contract["video_slices"][-1][1], 2, 2)
audio = torch.arange(
1 * 32 * 2 * contract["audio_slices"][-1][1],
dtype=torch.float32,
).reshape(1, 32, 2, contract["audio_slices"][-1][1])
(video_1, audio_1), (video_2, audio_2) = _split_terminal_merged_latents(
video,
audio,
contract,
)
video_overlap = contract["video_slices"][0][1] - contract["video_slices"][1][0]
audio_overlap = contract["audio_slices"][0][1] - contract["audio_slices"][1][0]
recombined_video = torch.cat((video_1, video_2[:, :, video_overlap:]), dim=2)
recombined_audio = torch.cat((audio_1, audio_2[..., audio_overlap:]), dim=-1)
assert contract["physical_frames"] == 243
assert contract["logical_frames"] == (124, 141)
assert torch.equal(recombined_video, video)
assert torch.equal(recombined_audio, audio)
assert tensor_difference(video, recombined_video).bit_exact is True
assert tensor_difference(audio, recombined_audio).bit_exact is True
capture = DiagnosticCapture()
stage = capture.capture_split_recombine(
physical_video=video,
physical_audio=audio,
recombined_video=recombined_video,
recombined_audio=recombined_audio,
)
assert stage.name == "D_split_recombine"
+165
View File
@@ -0,0 +1,165 @@
from __future__ import annotations
import pytest
import torch
from ComfyUI_H3_Continuum_Join.v2.sequence import (
_atomic_terminal_prefix,
_split_terminal_merged_latents,
_terminal_flf_merge_enabled,
_terminal_pair_prompt,
_terminal_physical_seed_plan,
_terminal_pair_contract,
_terminal_sampling_plan,
)
from ComfyUI_H3_Continuum_Join.v2.seeds import derive_chunk_seed
@pytest.mark.parametrize(
("initial_pair", "physical_frames", "logical_frames", "video_slices", "audio_slices"),
[
(True, 243, (124, 141), ((0, 37), (30, 72)), ((0, 207), (170, 405))),
(False, 260, (141, 141), ((0, 42), (35, 77)), ((0, 235), (198, 433))),
],
)
def test_terminal_pair_contract_uses_native_temporal_grid(
initial_pair,
physical_frames,
logical_frames,
video_slices,
audio_slices,
):
contract = _terminal_pair_contract(initial_pair=initial_pair, chunk_seconds=5.0)
assert contract["physical_frames"] == physical_frames
assert contract["logical_frames"] == logical_frames
assert contract["video_slices"] == video_slices
assert contract["audio_slices"] == audio_slices
@pytest.mark.parametrize("initial_pair", [True, False])
def test_terminal_pair_split_preserves_expected_overlap(initial_pair):
contract = _terminal_pair_contract(initial_pair=initial_pair, chunk_seconds=5.0)
video_t = contract["video_slices"][-1][1]
audio_t = contract["audio_slices"][-1][1]
video = torch.arange(video_t, dtype=torch.float32).reshape(1, 1, video_t, 1, 1)
audio = torch.arange(audio_t, dtype=torch.float32).reshape(1, 1, 1, audio_t)
first, second = _split_terminal_merged_latents(video, audio, contract)
assert first[0].shape[2] == contract["video_slices"][0][1]
assert first[1].shape[-1] == contract["audio_slices"][0][1]
assert second[0][0, 0, 0, 0, 0].item() == contract["video_slices"][1][0]
assert second[1][0, 0, 0, 0].item() == contract["audio_slices"][1][0]
def test_two_by_five_uses_one_physical_sample():
normal, terminal = _terminal_sampling_plan(chunks=2, completed=0, merge_enabled=True)
assert normal == ()
assert terminal is True
assert len(normal) + int(terminal) == 1
def test_three_by_five_uses_one_normal_and_one_terminal_sample():
normal, terminal = _terminal_sampling_plan(chunks=3, completed=0, merge_enabled=True)
assert normal == (0,)
assert terminal is True
assert len(normal) + int(terminal) == 2
def test_terminal_merge_accepts_distinct_prompts_but_not_timeline_video():
common = dict(multi_chunk_flf=True, chunks=2, chunk_seconds=5.0)
assert _terminal_flf_merge_enabled(**common, prompt_hashes=["same", "same"], timeline_video_source=None)
assert _terminal_flf_merge_enabled(**common, prompt_hashes=["a", "b"], timeline_video_source=None)
assert not _terminal_flf_merge_enabled(**common, prompt_hashes=["same", "same"], timeline_video_source=object())
def test_terminal_pair_prompt_preserves_shared_prompt_exactly():
prompt, policy = _terminal_pair_prompt(["same", "same"], pair_start=0, chunk_seconds=5.0)
assert prompt == "same"
assert policy == "shared_prompt_v1"
def test_terminal_pair_prompt_rebases_distinct_sections_to_local_timeline():
prompt, policy = _terminal_pair_prompt(["first action", "second action"], pair_start=0, chunk_seconds=5.0)
assert prompt == "[0-5s]\nfirst action\n\n[5-10s]\nsecond action"
assert policy == "paired_timeline_v1"
def test_partial_terminal_pair_is_discarded_for_atomic_resume():
preserved = [{"chunk": 1}, {"chunk": 2}]
result, reset = _atomic_terminal_prefix(preserved, chunks=3, reroll_from_chunk=0)
assert result == preserved[:1]
assert reset is True
def test_rerolling_final_chunk_regenerates_both_terminal_chunks():
preserved = [{"chunk": 1}, {"chunk": 2}, {"chunk": 3}]
result, reset = _atomic_terminal_prefix(preserved, chunks=3, reroll_from_chunk=3)
assert result == preserved[:1]
assert reset is True
def test_complete_terminal_pair_remains_reusable_without_reroll():
preserved = [{"chunk": 1}, {"chunk": 2}, {"chunk": 3}]
result, reset = _atomic_terminal_prefix(preserved, chunks=3, reroll_from_chunk=0)
assert result == preserved
assert reset is False
@pytest.mark.parametrize("base_seed", [42, 123456789])
def test_initial_terminal_pair_uses_base_seed_directly(base_seed):
plan = _terminal_physical_seed_plan(
base_seed=base_seed,
physical_window_start_index=0,
reroll_nonce=0,
)
assert plan["physical_seed"] == base_seed
assert plan["logical_entry_seeds"] == (base_seed, base_seed)
def test_later_terminal_pair_derives_once_from_pair_start():
plan = _terminal_physical_seed_plan(
base_seed=42,
physical_window_start_index=2,
reroll_nonce=0,
)
assert plan["physical_seed"] == derive_chunk_seed(42, 2, 0)
assert plan["physical_seed"] == 1808285636802909398
assert plan["logical_entry_seeds"] == (plan["physical_seed"],) * 2
def test_terminal_pair_reroll_derives_once_and_shares_new_seed():
original = _terminal_physical_seed_plan(
base_seed=42,
physical_window_start_index=2,
reroll_nonce=0,
)
rerolled = _terminal_physical_seed_plan(
base_seed=42,
physical_window_start_index=2,
reroll_nonce=1,
)
assert rerolled["physical_seed"] == derive_chunk_seed(42, 2, 1)
assert rerolled["physical_seed"] == 4306926667627841302
assert rerolled["physical_seed"] != original["physical_seed"]
assert rerolled["logical_entry_seeds"] == (rerolled["physical_seed"],) * 2
def test_non_terminal_seed_derivation_golden_values_are_unchanged():
assert derive_chunk_seed(42, 0, 0) == 15348518480342951850
assert derive_chunk_seed(42, 1, 0) == 13021120982267964226
def test_last_frame_absent_does_not_enable_terminal_merge():
assert not _terminal_flf_merge_enabled(
multi_chunk_flf=False,
chunks=2,
chunk_seconds=5.0,
prompt_hashes=["same", "same"],
timeline_video_source=None,
)
def test_one_chunk_keeps_normal_sampling_and_seed_behavior():
normal, terminal = _terminal_sampling_plan(chunks=1, completed=0, merge_enabled=False)
assert normal == (0,)
assert terminal is False
assert derive_chunk_seed(42, normal[0], 0) == 15348518480342951850
+61
View File
@@ -283,6 +283,7 @@ def _sampling_contract(
last_frame_hash="", reference_contract=None, last_frame_hash="", reference_contract=None,
upstream_graph_contract=None, upstream_graph_safe=None, upstream_graph_contract=None, upstream_graph_safe=None,
upstream_graph_reasons=None, upstream_graph_reasons=None,
execution_semantics=None,
): ):
return build_sampling_contract( return build_sampling_contract(
model=model or _Model(), model=model or _Model(),
@@ -312,9 +313,69 @@ def _sampling_contract(
upstream_graph_contract=upstream_graph_contract, upstream_graph_contract=upstream_graph_contract,
upstream_graph_safe=upstream_graph_safe, upstream_graph_safe=upstream_graph_safe,
upstream_graph_reasons=upstream_graph_reasons, upstream_graph_reasons=upstream_graph_reasons,
execution_semantics=execution_semantics,
) )
def test_terminal_seed_v2_revision_is_scoped_and_distinct():
old_contract, old_safe, old_reasons = _sampling_contract(
conditioning_mode="fl2va",
first_frame_hash="a" * 64,
last_frame_hash="b" * 64,
)
new_contract, new_safe, new_reasons = _sampling_contract(
conditioning_mode="fl2va",
first_frame_hash="a" * 64,
last_frame_hash="b" * 64,
execution_semantics={
"flf_execution": "terminal_merged_10s_seed_v2",
"terminal_seed_policy": "physical_window_seed_v2",
},
)
assert old_safe, old_reasons
assert new_safe, new_reasons
assert "execution_semantics" not in old_contract["global"]
assert new_contract["global"]["execution_semantics"] == {
"flf_execution": "terminal_merged_10s_seed_v2",
"terminal_seed_policy": "physical_window_seed_v2",
}
assert revision_identity(old_contract) != revision_identity(new_contract)
def test_terminal_seed_v2_revision_identity_is_deterministic():
semantics = {
"flf_execution": "terminal_merged_10s_seed_v2",
"terminal_seed_policy": "physical_window_seed_v2",
}
model = _Model()
clip = _Clip()
video_vae = _VideoVAE()
sampler = _Sampler()
first, first_safe, first_reasons = _sampling_contract(
model=model,
clip=clip,
video_vae=video_vae,
sampler=sampler,
conditioning_mode="fl2va",
first_frame_hash="a" * 64,
last_frame_hash="b" * 64,
execution_semantics=semantics,
)
second, second_safe, second_reasons = _sampling_contract(
model=model,
clip=clip,
video_vae=video_vae,
sampler=sampler,
conditioning_mode="fl2va",
first_frame_hash="a" * 64,
last_frame_hash="b" * 64,
execution_semantics=semantics,
)
assert first_safe, first_reasons
assert second_safe, second_reasons
assert revision_identity(first) == revision_identity(second)
def test_v322_golden_reference_contract_preserves_revision_identity(): def test_v322_golden_reference_contract_preserves_revision_identity():
model = _Model() model = _Model()
clip = _Clip() clip = _Clip()
+149
View File
@@ -0,0 +1,149 @@
from __future__ import annotations
import pytest
import torch
from ComfyUI_H3_Continuum_Join.v3 import plan as plan_module
def _logical_plan():
return {
"chunks": [
{
"sequence_index": 0,
"chunk_index": 1,
"total_frames": 124,
"trim_frames": 0,
"net_frames": 124,
"context_frames": 0,
"expected_video_latent_t": 37,
"expected_audio_latent_t": 207,
"frame_start": 0,
"frame_stop": 124,
},
{
"sequence_index": 1,
"chunk_index": 2,
"total_frames": 141,
"trim_frames": 22,
"net_frames": 119,
"context_frames": 22,
"expected_video_latent_t": 42,
"expected_audio_latent_t": 235,
"frame_start": 124,
"frame_stop": 243,
},
],
"target_frames": 240,
}
def _terminal_entries():
physical_video = torch.arange(72, dtype=torch.float32).reshape(1, 1, 72, 1, 1)
physical_audio = torch.arange(405, dtype=torch.float32).reshape(1, 1, 405)
entries = [
{
"video": physical_video[:, :, 0:37].clone(),
"audio": physical_audio[..., 0:207].clone(),
},
{
"video": physical_video[:, :, 30:72].clone(),
"audio": physical_audio[..., 170:405].clone(),
},
]
return entries, physical_video, physical_audio
def test_terminal_logical_entries_recombine_bit_exact(monkeypatch):
entries, physical_video, physical_audio = _terminal_entries()
monkeypatch.setattr(
plan_module,
"make_assembly_plan",
lambda *args, **kwargs: _logical_plan(),
)
monkeypatch.setattr(plan_module, "validate_assembly_plan", lambda plan: plan)
decode_entries, assembly_plan = plan_module.prepare_physical_decode_entries(
entries,
chunk_seconds=5.0,
preserve_final_frame=True,
terminal_merged=True,
)
assert len(decode_entries) == 1
assert torch.equal(decode_entries[0]["video"], physical_video)
assert torch.equal(decode_entries[0]["audio"], physical_audio)
assert assembly_plan["logical_chunk_count"] == 2
assert assembly_plan["physical_decode_group_count"] == 1
assert assembly_plan["decode_groups"][0]["logical_chunk_indices"] == [1, 2]
assert assembly_plan["decode_groups"][0]["total_frames"] == 243
assert assembly_plan["decode_groups"][0]["trim_frames"] == 0
assert assembly_plan["decode_groups"][0]["net_frames"] == 243
def test_terminal_recombine_rejects_changed_overlap(monkeypatch):
entries, _, _ = _terminal_entries()
entries[1]["video"][:, :, 0].add_(1.0)
monkeypatch.setattr(
plan_module,
"make_assembly_plan",
lambda *args, **kwargs: _logical_plan(),
)
with pytest.raises(ValueError, match="overlap differs"):
plan_module.prepare_physical_decode_entries(
entries,
chunk_seconds=5.0,
preserve_final_frame=True,
terminal_merged=True,
)
def test_three_logical_chunks_use_two_physical_decode_groups(monkeypatch):
from ComfyUI_H3_Continuum_Join.v2.sequence import (
_split_terminal_merged_latents,
_terminal_pair_contract,
)
contract = _terminal_pair_contract(initial_pair=False, chunk_seconds=5.0)
video_t = contract["video_slices"][-1][1]
audio_t = contract["audio_slices"][-1][1]
physical_video = torch.arange(video_t, dtype=torch.float32).reshape(1, 1, video_t, 1, 1)
physical_audio = torch.arange(audio_t, dtype=torch.float32).reshape(1, 1, audio_t)
terminal_parts = _split_terminal_merged_latents(physical_video, physical_audio, contract)
first_entry = {
"video": torch.zeros((1, 1, 37, 1, 1), dtype=torch.float32),
"audio": torch.zeros((1, 1, 207), dtype=torch.float32),
}
logical_plan = {
"chunks": [
{"sequence_index": 0, "chunk_index": 1, "total_frames": 124, "trim_frames": 0, "net_frames": 124, "context_frames": 0, "expected_video_latent_t": 37, "expected_audio_latent_t": 207, "frame_start": 0, "frame_stop": 124},
{"sequence_index": 1, "chunk_index": 2, "total_frames": 141, "trim_frames": 22, "net_frames": 119, "context_frames": 22, "expected_video_latent_t": 42, "expected_audio_latent_t": 235, "frame_start": 124, "frame_stop": 243},
{"sequence_index": 2, "chunk_index": 3, "total_frames": 141, "trim_frames": 22, "net_frames": 119, "context_frames": 22, "expected_video_latent_t": 42, "expected_audio_latent_t": 235, "frame_start": 243, "frame_stop": 362},
],
"target_frames": 360,
}
entries = [
first_entry,
{"video": terminal_parts[0][0], "audio": terminal_parts[0][1]},
{"video": terminal_parts[1][0], "audio": terminal_parts[1][1]},
]
monkeypatch.setattr(plan_module, "make_assembly_plan", lambda *args, **kwargs: logical_plan)
monkeypatch.setattr(plan_module, "validate_assembly_plan", lambda plan: plan)
decode_entries, assembly_plan = plan_module.prepare_physical_decode_entries(
entries,
chunk_seconds=5.0,
preserve_final_frame=True,
terminal_merged=True,
)
assert len(decode_entries) == 2
assert torch.equal(decode_entries[1]["video"], physical_video)
assert torch.equal(decode_entries[1]["audio"], physical_audio)
assert assembly_plan["physical_decode_group_count"] == 2
terminal_group = assembly_plan["decode_groups"][1]
assert terminal_group["logical_chunk_indices"] == [2, 3]
assert terminal_group["total_frames"] == 260
assert terminal_group["trim_frames"] == 22
assert terminal_group["net_frames"] == 238
+32
View File
@@ -0,0 +1,32 @@
# H3 Core vs Continuum GPU Diagnostic
This is a standalone diagnostic node. It does not change or register through the
production H3 Continuum package.
## Install for ComfyUI_W
Run from the repository root:
```powershell
.\tools\install_gpu_diagnostic_node.ps1
```
Restart ComfyUI and search for `H3 Core vs Continuum GPU Diagnostic`.
## Connections
- Connect the same direct MODEL used by both tests. Keep Spectrum off.
- Connect the same CLIP, Video VAE, Audio VAE, sampler, and sigmas.
- Connect the exact same First Frame and Last Frame.
- Use 640x640, seed 42, 8 steps, Euler/simple for the baseline comparison.
- Run Storage and Session are not used by this node.
One queue executes Core FL2VA at 243 frames, then Continuum Terminal Merge 2x5s.
Results are written below:
```text
ComfyUI/output/h3_continuum/diagnostics/<timestamp>/
```
The report identifies the first difference across pre-sampler data, first H3
MODEL call, sampled physical latent, split/recombine, and decode/assembly.
+9
View File
@@ -0,0 +1,9 @@
"""Standalone GPU diagnostics for H3 Continuum.
This package is intentionally separate from the production node package. Copy the
directory into ComfyUI/custom_nodes only while diagnostics are needed.
"""
from .nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"]
+302
View File
@@ -0,0 +1,302 @@
"""Deterministic tensor capture and comparison helpers for the GPU runner."""
from __future__ import annotations
from collections.abc import Mapping, Sequence
from dataclasses import asdict, dataclass
from hashlib import sha256
import json
from pathlib import Path
import re
from typing import Any
import torch
STAGE_ORDER = (
"A_pre_sampler",
"B_first_model_call",
"C_physical_sample",
"D_split_recombine",
"E1_decode_243",
"E2_final_240",
)
def _dense_cpu(tensor: torch.Tensor) -> torch.Tensor:
return tensor.detach().to("cpu").contiguous()
def _raw_sha256(tensor: torch.Tensor) -> str:
value = _dense_cpu(tensor)
raw = value.view(torch.uint8).numpy().tobytes()
return sha256(raw).hexdigest()
def tensor_fingerprint(tensor: torch.Tensor) -> dict[str, Any]:
value = _dense_cpu(tensor)
return {
"shape": list(value.shape),
"dtype": str(value.dtype),
"sha256": _raw_sha256(value),
"numel": int(value.numel()),
}
@dataclass(frozen=True)
class TensorDifference:
comparable: bool
bit_exact: bool
allclose: bool
max_abs_diff: float | None
mean_abs_diff: float | None
reason: str | None = None
@property
def matches(self) -> bool:
return self.comparable and (self.bit_exact or self.allclose)
def to_dict(self) -> dict[str, Any]:
value = asdict(self)
value["matches"] = self.matches
return value
def compare_tensors(
left: torch.Tensor,
right: torch.Tensor,
*,
rtol: float = 1.0e-4,
atol: float = 1.0e-5,
) -> TensorDifference:
a = _dense_cpu(left)
b = _dense_cpu(right)
if tuple(a.shape) != tuple(b.shape):
return TensorDifference(False, False, False, None, None, "shape mismatch")
if a.dtype != b.dtype:
return TensorDifference(False, False, False, None, None, "dtype mismatch")
bit_exact = bool(torch.equal(a, b))
if not (a.is_floating_point() or a.is_complex()):
return TensorDifference(True, bit_exact, bit_exact, 0.0 if bit_exact else None, 0.0 if bit_exact else None)
af = a.to(torch.float64)
bf = b.to(torch.float64)
delta = (af - bf).abs()
max_abs = float(delta.max().item()) if delta.numel() else 0.0
mean_abs = float(delta.mean().item()) if delta.numel() else 0.0
close = bool(torch.allclose(af, bf, rtol=rtol, atol=atol, equal_nan=True))
return TensorDifference(True, bit_exact, close, max_abs, mean_abs)
def _safe_tensor_key(value: str) -> str:
return re.sub(r"[^A-Za-z0-9_.-]+", "_", value).strip("_") or "tensor"
def _is_primitive(value: Any) -> bool:
return value is None or isinstance(value, (bool, int, float, str))
def _primitive_sequence(value: Any) -> bool:
return isinstance(value, (list, tuple)) and all(_is_primitive(item) for item in value)
class DiagnosticCapture:
def __init__(self, side: str):
self.side = side
self.manifest: dict[str, Any] = {
"format": "h3-core-continuum-gpu-diagnostic-v1",
"side": side,
"stages": {},
}
self.tensors: dict[str, torch.Tensor] = {}
self.runtime: dict[str, Any] = {}
def _stage(self, stage: str) -> dict[str, Any]:
return self.manifest["stages"].setdefault(stage, {"values": {}, "tensors": {}})
def add_value(self, stage: str, field: str, value: Any) -> None:
if not (_is_primitive(value) or _primitive_sequence(value)):
value = json.loads(json.dumps(value, default=str))
self._stage(stage)["values"][field] = value
def add_tensor(self, stage: str, field: str, tensor: torch.Tensor, *, persist: bool = True) -> None:
if not torch.is_tensor(tensor):
raise TypeError(f"{field} is not a tensor")
if getattr(tensor, "is_nested", False):
parts = list(tensor.unbind())
self.add_value(stage, f"{field}.components", len(parts))
for index, part in enumerate(parts):
self.add_tensor(stage, f"{field}.component_{index}", part, persist=persist)
return
value = _dense_cpu(tensor)
key = _safe_tensor_key(f"{stage}.{field}")
self._stage(stage)["tensors"][field] = {
**tensor_fingerprint(value),
"stored_as": key if persist else None,
}
if persist:
self.tensors[key] = value.clone()
def capture_value(
self,
stage: str,
field: str,
value: Any,
*,
persist_tensors: bool = True,
depth: int = 0,
) -> None:
if depth > 7:
self.add_value(stage, f"{field}.truncated", True)
return
if torch.is_tensor(value):
self.add_tensor(stage, field, value, persist=persist_tensors)
return
if _is_primitive(value):
self.add_value(stage, field, value)
return
if _primitive_sequence(value):
self.add_value(stage, field, list(value))
return
if isinstance(value, Mapping):
keys = sorted(str(key) for key in value.keys())
self.add_value(stage, f"{field}.keys", keys)
for key in keys[:256]:
try:
child = value[key]
except (KeyError, TypeError):
child = next((item for raw, item in value.items() if str(raw) == key), None)
self.capture_value(
stage,
f"{field}.{key}",
child,
persist_tensors=persist_tensors,
depth=depth + 1,
)
return
if isinstance(value, Sequence) and not isinstance(value, (str, bytes, bytearray)):
self.add_value(stage, f"{field}.length", len(value))
for index, child in enumerate(value[:256]):
self.capture_value(
stage,
f"{field}.{index}",
child,
persist_tensors=persist_tensors,
depth=depth + 1,
)
return
if hasattr(value, "unbind") and value.__class__.__name__ == "NestedTensor":
parts = list(value.unbind())
self.add_value(stage, f"{field}.components", len(parts))
for index, child in enumerate(parts):
label = ("video", "audio")[index] if index < 2 else f"component_{index}"
self.capture_value(
stage,
f"{field}.{label}",
child,
persist_tensors=persist_tensors,
depth=depth + 1,
)
return
if all(hasattr(value, name) for name in ("signature", "segments", "position_ids")):
self.add_value(stage, f"{field}.type", value.__class__.__name__)
self.add_value(stage, f"{field}.signature", list(value.signature))
self.add_value(stage, f"{field}.segments", [list(item) for item in value.segments])
for name in ("position_ids", "img_pos", "img_update", "audio_pos", "audio_update"):
child = getattr(value, name, None)
if torch.is_tensor(child):
self.add_tensor(stage, f"{field}.{name}", child, persist=persist_tensors)
return
self.add_value(stage, f"{field}.type", value.__class__.__name__)
def save(self, directory: Path) -> None:
directory.mkdir(parents=True, exist_ok=True)
from safetensors.torch import save_file
save_file(self.tensors, str(directory / "tensors.safetensors"))
(directory / "manifest.json").write_text(
json.dumps(self.manifest, indent=2, ensure_ascii=False),
encoding="utf-8",
)
def compare_stage(
left: DiagnosticCapture,
right: DiagnosticCapture,
stage: str,
) -> dict[str, Any]:
a = left.manifest["stages"].get(stage, {"values": {}, "tensors": {}})
b = right.manifest["stages"].get(stage, {"values": {}, "tensors": {}})
details: dict[str, Any] = {}
first_field: str | None = None
value_fields = list(a["values"].keys()) + [key for key in b["values"] if key not in a["values"]]
for field in value_fields:
av = a["values"].get(field, {"missing": True})
bv = b["values"].get(field, {"missing": True})
matches = av == bv
details[field] = {"kind": "value", "matches": matches, "left": av, "right": bv}
if not matches and first_field is None:
first_field = field
tensor_fields = list(a["tensors"].keys()) + [key for key in b["tensors"] if key not in a["tensors"]]
for field in tensor_fields:
a_meta = a["tensors"].get(field)
b_meta = b["tensors"].get(field)
if a_meta is None or b_meta is None:
result = {"kind": "tensor", "matches": False, "reason": "missing tensor"}
else:
a_key = a_meta.get("stored_as")
b_key = b_meta.get("stored_as")
if a_key in left.tensors and b_key in right.tensors:
result = {"kind": "tensor", **compare_tensors(left.tensors[a_key], right.tensors[b_key]).to_dict()}
else:
matches = a_meta == b_meta
result = {"kind": "fingerprint", "matches": matches, "left": a_meta, "right": b_meta}
details[field] = result
if not result["matches"] and first_field is None:
first_field = field
return {
"status": "MATCH" if first_field is None else "DIFFER",
"first_field": first_field,
"fields": details,
}
def recombine_terminal_streams(
first: torch.Tensor,
second: torch.Tensor,
*,
first_range: tuple[int, int] | list[int],
second_range: tuple[int, int] | list[int],
dim: int,
) -> torch.Tensor:
first_start, first_stop = (int(value) for value in first_range)
second_start, second_stop = (int(value) for value in second_range)
overlap = first_stop - second_start
if first_start != 0 or overlap < 0:
raise ValueError("terminal split ranges do not form an overlapping sequence")
if first.shape[dim] != first_stop - first_start:
raise ValueError("first logical stream length does not match its range")
if second.shape[dim] != second_stop - second_start:
raise ValueError("second logical stream length does not match its range")
if overlap > second.shape[dim]:
raise ValueError("terminal overlap exceeds the second logical stream")
slices = [slice(None)] * second.ndim
slices[dim] = slice(overlap, None)
return torch.cat((first, second[tuple(slices)]), dim=dim).contiguous()
def format_comparison(comparison: Mapping[str, Any], output_directory: Path) -> str:
lines = []
first_divergence = None
for stage in STAGE_ORDER:
result = comparison.get(stage, {"status": "NOT_RUN", "first_field": None})
lines.append(f"{stage}: {result['status']}")
if first_divergence is None and result["status"] == "DIFFER":
field = result.get("first_field")
first_divergence = f"{stage}.{field}" if field else stage
lines.append("")
lines.append(f"First divergence: {first_divergence or 'none'}")
lines.append(f"Output: {output_directory}")
return "\n".join(lines)
+715
View File
@@ -0,0 +1,715 @@
"""Standalone ComfyUI node for Core-vs-Continuum GPU diagnostics."""
from __future__ import annotations
from contextlib import contextmanager
from datetime import datetime
import gc
import importlib
import json
from pathlib import Path, PurePosixPath
import random
import sys
import traceback
from types import SimpleNamespace
from typing import Any
import numpy as np
import torch
from .capture import (
DiagnosticCapture,
STAGE_ORDER,
compare_stage,
compare_tensors,
format_comparison,
recombine_terminal_streams,
)
def _resolve_loaded_module(path: Path):
target = path.resolve()
for module in tuple(sys.modules.values()):
source = getattr(module, "__file__", None)
if source and Path(source).resolve() == target:
return module
return None
def _resolve_continuum_modules() -> SimpleNamespace:
import nodes as comfy_nodes
sampler_class = comfy_nodes.NODE_CLASS_MAPPINGS.get("H3ContinuumSamplerV34")
if sampler_class is None:
raise RuntimeError("H3ContinuumSamplerV34 is not loaded; install and enable H3 Continuum V3.4 first")
owner = sys.modules.get(sampler_class.__module__)
source = getattr(owner, "__file__", None)
if source is None:
raise RuntimeError("cannot resolve the active H3 Continuum package")
root = Path(source).resolve().parent.parent
module_name = sampler_class.__module__
base_name = module_name.split(".v3.", 1)[0] if ".v3." in module_name else module_name.rsplit(".", 2)[0]
def load(relative: str, suffix: str):
target = root / relative
loaded = _resolve_loaded_module(target)
if loaded is not None:
return loaded
return importlib.import_module(f"{base_name}.{suffix}")
return SimpleNamespace(
root=root,
constants=load("constants.py", "constants"),
state=load("state.py", "state"),
sequence=load("v2/sequence.py", "v2.sequence"),
sampling=load("v2/sampling.py", "v2.sampling"),
prompts=load("v2/prompts.py", "v2.prompts"),
decoder=load("v2/decoder.py", "v2.decoder"),
)
def _reset_rng(seed: int) -> None:
random.seed(seed)
np.random.seed(seed & 0xFFFFFFFF)
torch.manual_seed(seed)
if torch.cuda.is_available():
torch.cuda.manual_seed_all(seed)
def _release_cuda() -> None:
gc.collect()
if torch.cuda.is_available():
torch.cuda.empty_cache()
def _safe_output_directory(subdir: str) -> Path:
import folder_paths
clean = str(subdir).replace("\\", "/").strip("/") or "h3_continuum/diagnostics"
parts = PurePosixPath(clean).parts
if any(part in ("", ".", "..") for part in parts):
raise ValueError("output_subdir must stay below the ComfyUI output directory")
stamp = datetime.now().strftime("%Y%m%d_%H%M%S_%f")
return Path(folder_paths.get_output_directory()).joinpath(*parts, stamp)
def _capture_av_value(capture: DiagnosticCapture, stage: str, prefix: str, value: Any) -> None:
candidate = value.get("samples") if isinstance(value, dict) and "samples" in value else value
if hasattr(candidate, "unbind") and candidate.__class__.__name__ == "NestedTensor":
parts = list(candidate.unbind())
capture.add_value(stage, f"{prefix}.components", len(parts))
for index, part in enumerate(parts):
label = ("video", "audio")[index] if index < 2 else f"component_{index}"
capture.add_tensor(stage, f"{prefix}.{label}", part)
return
if torch.is_tensor(candidate) and getattr(candidate, "is_nested", False):
parts = list(candidate.unbind())
capture.add_value(stage, f"{prefix}.components", len(parts))
for index, part in enumerate(parts):
label = ("video", "audio")[index] if index < 2 else f"component_{index}"
capture.add_tensor(stage, f"{prefix}.{label}", part)
return
capture.capture_value(stage, prefix, candidate)
def _capture_pre_sampler(
capture: DiagnosticCapture,
modules: SimpleNamespace,
*,
conditioning: Any,
latent: dict[str, Any],
sigmas: torch.Tensor,
seed: int,
) -> None:
stage = "A_pre_sampler"
capture.add_value(stage, "seed", int(seed))
capture.add_value(stage, "presentation_order", ["First Frame", "Last Frame"])
capture.add_tensor(stage, "sigmas", sigmas)
video, audio = modules.state.extract_av_streams(latent)
capture.add_tensor(stage, "target.video", video)
capture.add_tensor(stage, "target.audio", audio)
capture.add_value(stage, "target.video_shape", list(video.shape))
capture.add_value(stage, "target.audio_shape", list(audio.shape))
if not conditioning:
capture.add_value(stage, "conditioning.empty", True)
return
first = conditioning[0]
cond_tensor = first[0]
metadata = first[1] if len(first) > 1 and isinstance(first[1], dict) else {}
capture.capture_value(stage, "conditioning.positive", cond_tensor)
capture.add_value(stage, "conditioning.metadata_keys", sorted(str(key) for key in metadata))
capture.add_value(stage, "physical_frames", int(metadata.get("minimax_frame_count", 243)))
for key in ("minimax_token_tags", "minimax_keyframes", "minimax_refs", "minimax_frame_count"):
if key in metadata:
capture.capture_value(stage, f"conditioning.{key}", metadata[key])
def _capture_physical_sample(
capture: DiagnosticCapture,
modules: SimpleNamespace,
result: dict[str, Any],
) -> tuple[torch.Tensor, torch.Tensor]:
video, audio = modules.state.extract_av_streams(result)
video_cpu = video.detach().to("cpu").contiguous()
audio_cpu = audio.detach().to("cpu").contiguous()
capture.add_tensor("C_physical_sample", "video", video_cpu)
capture.add_tensor("C_physical_sample", "audio", audio_cpu)
capture.runtime["physical_streams"] = (video_cpu, audio_cpu)
return video_cpu, audio_cpu
@contextmanager
def _runtime_capture(capture: DiagnosticCapture):
import comfy.model_base
import comfy.sample
original_noise = comfy.sample.prepare_noise
original_apply = comfy.model_base.BaseModel._apply_model
noise_seen = False
model_seen = False
def prepare_noise(*args, **kwargs):
nonlocal noise_seen
result = original_noise(*args, **kwargs)
if not noise_seen:
noise_seen = True
_capture_av_value(capture, "A_pre_sampler", "actual_noise", result)
return result
def apply_model(
self,
x,
t,
c_concat=None,
c_crossattn=None,
control=None,
transformer_options={},
**kwargs,
):
nonlocal model_seen
if not model_seen and self.__class__.__name__ == "MiniMaxH3":
model_seen = True
capture.capture_value("B_first_model_call", "input_x", x)
capture.capture_value("B_first_model_call", "timestep", t)
capture.capture_value(
"B_first_model_call",
"transformer_options",
transformer_options,
persist_tensors=False,
)
capture.capture_value("B_first_model_call", "minimax_payload", kwargs.get("minimax_payload"))
capture.capture_value("B_first_model_call", "latent_shapes", kwargs.get("latent_shapes"))
return original_apply(
self,
x,
t,
c_concat,
c_crossattn,
control,
transformer_options,
**kwargs,
)
comfy.sample.prepare_noise = prepare_noise
comfy.model_base.BaseModel._apply_model = apply_model
try:
yield
finally:
comfy.sample.prepare_noise = original_noise
comfy.model_base.BaseModel._apply_model = original_apply
restored = (
comfy.sample.prepare_noise is original_noise
and comfy.model_base.BaseModel._apply_model is original_apply
)
capture.add_value("B_first_model_call", "patches_restored", restored)
capture.add_value("A_pre_sampler", "noise_captured", noise_seen)
capture.add_value("B_first_model_call", "model_call_captured", model_seen)
def _run_core(
capture: DiagnosticCapture,
modules: SimpleNamespace,
*,
model: Any,
clip: Any,
video_vae: Any,
sampler: Any,
sigmas: torch.Tensor,
first_frame: torch.Tensor,
last_frame: torch.Tensor,
prompt: str,
width: int,
height: int,
seed: int,
) -> None:
from comfy_extras.nodes_minimax_h3 import MiniMaxH3ImageToVideo
core_model = model.clone()
output = MiniMaxH3ImageToVideo().execute(
clip,
video_vae,
prompt,
int(width),
int(height),
243,
first_frame=first_frame,
last_frame=last_frame,
)
conditioning, latent = output[0], output[1]
_capture_pre_sampler(
capture,
modules,
conditioning=conditioning,
latent=latent,
sigmas=sigmas,
seed=seed,
)
with _runtime_capture(capture):
result = modules.sampling.sample_chunk(
model=core_model,
conditioning=conditioning,
latent=latent,
sampler=sampler,
sigmas=sigmas,
seed=int(seed),
enable_preview=False,
)
_capture_physical_sample(capture, modules, result)
capture.add_value("C_physical_sample", "sampling_passes", 1)
capture.add_value("C_physical_sample", "physical_seed", int(seed))
del result, latent, conditioning, core_model
def _capture_terminal_split(
capture: DiagnosticCapture,
video: torch.Tensor,
audio: torch.Tensor,
contract: dict[str, Any],
result: Any,
) -> None:
(video_1, audio_1), (video_2, audio_2) = result
video_ranges = contract["video_slices"]
audio_ranges = contract["audio_slices"]
video_recombined = recombine_terminal_streams(
video_1,
video_2,
first_range=video_ranges[0],
second_range=video_ranges[1],
dim=2,
)
audio_recombined = recombine_terminal_streams(
audio_1,
audio_2,
first_range=audio_ranges[0],
second_range=audio_ranges[1],
dim=-1,
)
video_diff = compare_tensors(video.detach().to("cpu"), video_recombined.detach().to("cpu"))
audio_diff = compare_tensors(audio.detach().to("cpu"), audio_recombined.detach().to("cpu"))
stage = "D_split_recombine"
capture.add_value(stage, "contract", contract)
capture.add_tensor(stage, "video.physical", video)
capture.add_tensor(stage, "video.chunk_1", video_1)
capture.add_tensor(stage, "video.chunk_2", video_2)
capture.add_tensor(stage, "video.recombined", video_recombined)
capture.add_tensor(stage, "audio.physical", audio)
capture.add_tensor(stage, "audio.chunk_1", audio_1)
capture.add_tensor(stage, "audio.chunk_2", audio_2)
capture.add_tensor(stage, "audio.recombined", audio_recombined)
capture.add_value(stage, "video_difference", video_diff.to_dict())
capture.add_value(stage, "audio_difference", audio_diff.to_dict())
capture.runtime["logical_streams"] = (
(video_1.detach().to("cpu").contiguous(), audio_1.detach().to("cpu").contiguous()),
(video_2.detach().to("cpu").contiguous(), audio_2.detach().to("cpu").contiguous()),
)
capture.runtime["terminal_contract"] = contract
def _run_continuum(
capture: DiagnosticCapture,
modules: SimpleNamespace,
*,
model: Any,
clip: Any,
video_vae: Any,
audio_vae: Any,
sampler: Any,
sigmas: torch.Tensor,
first_frame: torch.Tensor,
last_frame: torch.Tensor,
prompt: str,
width: int,
height: int,
seed: int,
) -> None:
sequence = modules.sequence
original_sample = sequence.sample_chunk
original_split = sequence._split_terminal_merged_latents
sample_calls = 0
split_calls = 0
def sample_chunk(**kwargs):
nonlocal sample_calls
sample_calls += 1
if sample_calls == 1:
_capture_pre_sampler(
capture,
modules,
conditioning=kwargs["conditioning"],
latent=kwargs["latent"],
sigmas=kwargs["sigmas"],
seed=int(kwargs["seed"]),
)
result = original_sample(**kwargs)
if sample_calls == 1:
_capture_physical_sample(capture, modules, result)
capture.add_value("C_physical_sample", "physical_seed", int(kwargs["seed"]))
return result
def split_terminal(video, audio, contract):
nonlocal split_calls
split_calls += 1
result = original_split(video, audio, contract)
if split_calls == 1:
_capture_terminal_split(capture, video, audio, contract, result)
return result
sequence.sample_chunk = sample_chunk
sequence._split_terminal_merged_latents = split_terminal
try:
prompt_plan = modules.prompts.make_prompt_plan(
mode=modules.constants.PROMPT_MODE_FIXED,
script=prompt,
chunks=2,
chunk_seconds=5.0,
)
continuum_model = model.clone()
with _runtime_capture(capture):
sequence.run_sequence(
model=continuum_model,
clip=clip,
video_vae=video_vae,
audio_vae=audio_vae,
sampler=sampler,
sigmas=sigmas,
first_frame=first_frame,
last_frame=last_frame,
prompt_plan=prompt_plan,
width=int(width),
height=int(height),
continuity=modules.constants.CONTINUITY_OPTIONS[0],
base_seed=int(seed),
audio_continuity=True,
exact_total_duration=True,
diagnostics_mode=modules.constants.DIAGNOSTICS_BASIC,
reroll_from_chunk=0,
reroll_nonce=0,
strict_compatibility=False,
debug=False,
seam_correction=modules.constants.SEAM_CORRECTION_OFF,
enable_preview=False,
session=None,
initial_state=None,
latent_only=True,
reference_assets=None,
reference_audio_source=None,
reference_audio_vae=None,
driving_audio_source=None,
driving_audio_vae=None,
reference_video_source=None,
timeline_video_source=None,
)
capture.add_value("C_physical_sample", "sampling_passes", sample_calls)
capture.add_value("D_split_recombine", "split_calls", split_calls)
del continuum_model
finally:
sequence.sample_chunk = original_sample
sequence._split_terminal_merged_latents = original_split
capture.add_value(
"D_split_recombine",
"sequence_patches_restored",
sequence.sample_chunk is original_sample
and sequence._split_terminal_merged_latents is original_split,
)
def _make_av_latent(video: torch.Tensor, audio: torch.Tensor) -> dict[str, Any]:
from comfy.nested_tensor import NestedTensor
return {"samples": NestedTensor([video, audio])}
def _copy_audio(audio: dict[str, Any]) -> dict[str, Any]:
return {
"waveform": audio["waveform"].clone(),
"sample_rate": int(audio["sample_rate"]),
}
def _run_decode_diagnostics(capture: DiagnosticCapture, modules: SimpleNamespace, video_vae: Any, audio_vae: Any) -> dict[str, Any]:
physical = capture.runtime.get("physical_streams")
logical = capture.runtime.get("logical_streams")
contract = capture.runtime.get("terminal_contract")
if physical is None or logical is None or contract is None:
return {
"E1_decode_243": {"status": "NOT_RUN", "first_field": "missing terminal capture"},
"E2_final_240": {"status": "NOT_RUN", "first_field": "missing terminal capture"},
}
decoder = modules.decoder
physical_video, physical_audio = physical
direct_latent = _make_av_latent(physical_video, physical_audio)
direct_images = decoder.decode_video(video_vae, direct_latent)
direct_audio = decoder.decode_audio(audio_vae, direct_latent)
decoded_images = []
decoded_audio = []
for video, audio in logical:
latent = _make_av_latent(video, audio)
decoded_images.append(decoder.decode_video(video_vae, latent))
decoded_audio.append(decoder.decode_audio(audio_vae, latent))
trim_frames = 22
reconstructed_images = torch.cat(
(decoded_images[0][:124], decoded_images[1][:141][trim_frames:]),
dim=0,
).contiguous()
sample_rate = int(decoded_audio[0]["sample_rate"])
if int(decoded_audio[1]["sample_rate"]) != sample_rate:
raise ValueError("chunk audio sample rates differ during decode diagnostics")
total_samples = int(round(243 / 24.0 * sample_rate))
reconstructed_waveform = torch.empty(
(*decoded_audio[0]["waveform"].shape[:-1], total_samples),
dtype=decoded_audio[0]["waveform"].dtype,
device="cpu",
)
segment_1 = decoder._slice_audio_for_timeline(
decoded_audio[0], trim_frames=0, frame_start=0, frame_stop=124
)
segment_2 = decoder._slice_audio_for_timeline(
decoded_audio[1], trim_frames=trim_frames, frame_start=124, frame_stop=243
)
split_sample = int(round(124 / 24.0 * sample_rate))
reconstructed_waveform[..., :split_sample].copy_(segment_1)
reconstructed_waveform[..., split_sample:].copy_(segment_2)
reconstructed_audio = {"waveform": reconstructed_waveform, "sample_rate": sample_rate}
direct_images, direct_audio, _ = decoder.enforce_total_frames(
direct_images,
direct_audio,
target_frames=243,
preserve_final_frame=False,
)
reconstructed_images, reconstructed_audio, _ = decoder.enforce_total_frames(
reconstructed_images,
reconstructed_audio,
target_frames=243,
preserve_final_frame=False,
)
image_e1 = compare_tensors(direct_images, reconstructed_images)
audio_e1 = compare_tensors(direct_audio["waveform"], reconstructed_audio["waveform"])
stage = "E1_decode_243"
capture.add_tensor(stage, "direct.images", direct_images, persist=False)
capture.add_tensor(stage, "split.images", reconstructed_images, persist=False)
capture.add_tensor(stage, "direct.audio", direct_audio["waveform"], persist=False)
capture.add_tensor(stage, "split.audio", reconstructed_audio["waveform"], persist=False)
capture.add_value(stage, "images_difference", image_e1.to_dict())
capture.add_value(stage, "audio_difference", audio_e1.to_dict())
boundary = slice(112, 129)
capture.add_tensor(stage, "direct.boundary_images", direct_images[boundary])
capture.add_tensor(stage, "split.boundary_images", reconstructed_images[boundary])
boundary_diff = compare_tensors(direct_images[boundary], reconstructed_images[boundary])
capture.add_value(stage, "boundary_images_difference", boundary_diff.to_dict())
direct_240_images, direct_240_audio, _ = decoder.enforce_total_frames(
direct_images,
_copy_audio(direct_audio),
target_frames=240,
preserve_final_frame=True,
)
split_240_images, split_240_audio, _ = decoder.enforce_total_frames(
reconstructed_images,
_copy_audio(reconstructed_audio),
target_frames=240,
preserve_final_frame=True,
)
image_e2 = compare_tensors(direct_240_images, split_240_images)
audio_e2 = compare_tensors(direct_240_audio["waveform"], split_240_audio["waveform"])
stage = "E2_final_240"
capture.add_tensor(stage, "direct.images", direct_240_images, persist=False)
capture.add_tensor(stage, "split.images", split_240_images, persist=False)
capture.add_tensor(stage, "direct.audio", direct_240_audio["waveform"], persist=False)
capture.add_tensor(stage, "split.audio", split_240_audio["waveform"], persist=False)
capture.add_value(stage, "images_difference", image_e2.to_dict())
capture.add_value(stage, "audio_difference", audio_e2.to_dict())
capture.add_tensor(stage, "direct.boundary_images", direct_240_images[boundary])
capture.add_tensor(stage, "split.boundary_images", split_240_images[boundary])
return {
"E1_decode_243": {
"status": "MATCH" if image_e1.matches and audio_e1.matches else "DIFFER",
"first_field": None if image_e1.matches and audio_e1.matches else ("images" if not image_e1.matches else "audio"),
"images": image_e1.to_dict(),
"audio": audio_e1.to_dict(),
"boundary_images": boundary_diff.to_dict(),
},
"E2_final_240": {
"status": "MATCH" if image_e2.matches and audio_e2.matches else "DIFFER",
"first_field": None if image_e2.matches and audio_e2.matches else ("images" if not image_e2.matches else "audio"),
"images": image_e2.to_dict(),
"audio": audio_e2.to_dict(),
},
}
class H3CoreContinuumGpuDiagnostic:
"""Run an isolated Core 243f and Continuum 2x5s comparison in one queue."""
OUTPUT_NODE = True
CATEGORY = "H3-Continuum/Diagnostics"
FUNCTION = "run"
RETURN_TYPES = ("STRING",)
RETURN_NAMES = ("diagnostic_report",)
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"model": ("MODEL",),
"clip": ("CLIP",),
"video_vae": ("VAE",),
"audio_vae": ("VAE",),
"sampler": ("SAMPLER",),
"sigmas": ("SIGMAS",),
"first_frame": ("IMAGE",),
"last_frame": ("IMAGE",),
"prompt": ("STRING", {"multiline": True, "default": "The subject moves continuously from the First Frame to the Last Frame."}),
"width": ("INT", {"default": 640, "min": 64, "max": 4096, "step": 16}),
"height": ("INT", {"default": 640, "min": 64, "max": 4096, "step": 16}),
"seed": ("INT", {"default": 42, "min": 0, "max": 0xFFFFFFFFFFFFFFFF}),
"output_subdir": ("STRING", {"default": "h3_continuum/diagnostics"}),
}
}
def run(
self,
model,
clip,
video_vae,
audio_vae,
sampler,
sigmas,
first_frame,
last_frame,
prompt,
width,
height,
seed,
output_subdir,
):
output_directory = _safe_output_directory(output_subdir)
core = DiagnosticCapture("core")
continuum = DiagnosticCapture("continuum")
comparison: dict[str, Any] = {}
modules = _resolve_continuum_modules()
common = {
"seed": int(seed),
"width": int(width),
"height": int(height),
"physical_frames": 243,
"spectrum": "must be OFF at the model input",
"run_storage": "Off",
"session": None,
"continuum_root": str(modules.root),
}
core.manifest["conditions"] = common
continuum.manifest["conditions"] = common
try:
_reset_rng(int(seed))
_run_core(
core,
modules,
model=model,
clip=clip,
video_vae=video_vae,
sampler=sampler,
sigmas=sigmas,
first_frame=first_frame,
last_frame=last_frame,
prompt=prompt,
width=width,
height=height,
seed=seed,
)
_release_cuda()
_reset_rng(int(seed))
_run_continuum(
continuum,
modules,
model=model,
clip=clip,
video_vae=video_vae,
audio_vae=audio_vae,
sampler=sampler,
sigmas=sigmas,
first_frame=first_frame,
last_frame=last_frame,
prompt=prompt,
width=width,
height=height,
seed=seed,
)
for stage in ("A_pre_sampler", "B_first_model_call", "C_physical_sample"):
comparison[stage] = compare_stage(core, continuum, stage)
d_values = continuum.manifest["stages"].get("D_split_recombine", {}).get("values", {})
d_video = d_values.get("video_difference", {})
d_audio = d_values.get("audio_difference", {})
d_match = bool(d_video.get("matches") and d_audio.get("matches"))
comparison["D_split_recombine"] = {
"status": "MATCH" if d_match else "DIFFER",
"first_field": None if d_match else ("video" if not d_video.get("matches") else "audio"),
"video": d_video,
"audio": d_audio,
}
comparison.update(_run_decode_diagnostics(continuum, modules, video_vae, audio_vae))
except Exception as exc:
comparison["error"] = {
"type": type(exc).__name__,
"message": str(exc),
"traceback": traceback.format_exc(),
}
finally:
output_directory.mkdir(parents=True, exist_ok=True)
core.save(output_directory / "core")
continuum.save(output_directory / "continuum")
for stage in STAGE_ORDER:
comparison.setdefault(stage, {"status": "NOT_RUN", "first_field": None})
summary = format_comparison(comparison, output_directory)
(output_directory / "comparison.json").write_text(
json.dumps(comparison, indent=2, ensure_ascii=False),
encoding="utf-8",
)
(output_directory / "comparison.txt").write_text(summary + "\n", encoding="utf-8")
_release_cuda()
if "error" in comparison:
error = comparison["error"]
raise RuntimeError(
f"GPU diagnostic failed; partial results were saved to {output_directory}: "
f"{error['type']}: {error['message']}"
)
return (summary,)
NODE_CLASS_MAPPINGS = {
"H3CoreContinuumGpuDiagnostic": H3CoreContinuumGpuDiagnostic,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"H3CoreContinuumGpuDiagnostic": "H3 Core vs Continuum GPU Diagnostic",
}
+29
View File
@@ -0,0 +1,29 @@
param(
[string]$ComfyUIRoot = 'D:\StabilityMatrix\Data\Packages\ComfyUI_W'
)
$ErrorActionPreference = 'Stop'
$source = Join-Path $PSScriptRoot 'gpu_diagnostic_node'
$customNodes = Join-Path $ComfyUIRoot 'custom_nodes'
$destination = Join-Path $customNodes 'H3_Continuum_GPU_Diagnostics'
if (-not (Test-Path -LiteralPath $source -PathType Container)) {
throw "Diagnostic source was not found: $source"
}
if (-not (Test-Path -LiteralPath $customNodes -PathType Container)) {
throw "ComfyUI custom_nodes directory was not found: $customNodes"
}
New-Item -ItemType Directory -Path $destination -Force | Out-Null
foreach ($name in @('__init__.py', 'capture.py', 'nodes.py', 'README.md')) {
Copy-Item -LiteralPath (Join-Path $source $name) -Destination (Join-Path $destination $name) -Force
}
$sourceCount = (Get-ChildItem -LiteralPath $source -File).Count
$destinationCount = (Get-ChildItem -LiteralPath $destination -File).Count
if ($destinationCount -lt $sourceCount) {
throw "Diagnostic install verification failed: $destinationCount < $sourceCount"
}
Write-Output "Installed standalone diagnostics: $destination"
Write-Output 'Restart ComfyUI before using the diagnostic node.'
+283 -12
View File
@@ -24,14 +24,21 @@ from ..driving_audio import (
) )
from ..model_patch import clone_model_for_chunk from ..model_patch import clone_model_for_chunk
from ..state import assert_context_unchanged, context_fingerprint, make_plan, select_context, validate_state from ..state import assert_context_unchanged, context_fingerprint, make_plan, select_context, validate_state
from ..temporal import align_frame_count_up, largest_context_capacity, make_extension_shape from ..temporal import (
align_frame_count_up,
audio_latent_t,
context_slots,
largest_context_capacity,
make_extension_shape,
video_latent_t,
)
from ..version import PACKAGE_VERSION from ..version import PACKAGE_VERSION
from .decoder import decode_sequence, decode_sequence_with_seam, enforce_total_frames from .decoder import decode_sequence, decode_sequence_with_seam, enforce_total_frames
from .context_diagnostics import ContextDiagnosticsTracker from .context_diagnostics import ContextDiagnosticsTracker
from .h3_builder import attach_keyframes, empty_h3_latent, encode_identity_latents, encode_prompt_conditioning, prepare_identity_assets from .h3_builder import attach_keyframes, empty_h3_latent, encode_identity_latents, encode_prompt_conditioning, prepare_identity_assets
from .motion import choose_context_frames from .motion import choose_context_frames
from .prompts import prompt_plan_report, validate_prompt_plan from .prompts import prompt_plan_report, validate_prompt_plan
from .sampling import sample_chunk from .sampling import latent_from_cpu, latent_to_cpu, sample_chunk
from .seeds import derive_chunk_seed from .seeds import derive_chunk_seed
from .session import ( from .session import (
SessionValidationError, entry_to_state, make_chunk_entry, make_session, model_fingerprint, SessionValidationError, entry_to_state, make_chunk_entry, make_session, model_fingerprint,
@@ -62,7 +69,7 @@ def _check_decode_memory_budget(*,width,height,chunks,chunk_seconds):
def _clone_entry_for_reuse(entry): def _clone_entry_for_reuse(entry):
entry=validate_chunk_entry(entry); result=dict(entry); result["plan"]=copy.deepcopy(entry["plan"]); result["reused"]=True; return result entry=validate_chunk_entry(entry); result=dict(entry); result["plan"]=copy.deepcopy(entry["plan"]); result["reused"]=True; return result
def _preserved_prefix(*,session,prompt_hashes,chunks,reroll_from_chunk,width,height,chunk_seconds,identity_hash): def _preserved_prefix(*,session,prompt_hashes,chunks,reroll_from_chunk,width,height,chunk_seconds,identity_hash,last_frame_hash):
if session is None: return [],[] if session is None: return [],[]
notes=[] notes=[]
try: session=validate_session(session) try: session=validate_session(session)
@@ -70,6 +77,8 @@ def _preserved_prefix(*,session,prompt_hashes,chunks,reroll_from_chunk,width,hei
if int(session["width"])!=int(width) or int(session["height"])!=int(height): return [],["saved session resolution differs; generated a fresh run"] if int(session["width"])!=int(width) or int(session["height"])!=int(height): return [],["saved session resolution differs; generated a fresh run"]
if abs(float(session["chunk_seconds"])-float(chunk_seconds))>1e-6: return [],["saved session chunk duration differs; generated a fresh run"] if abs(float(session["chunk_seconds"])-float(chunk_seconds))>1e-6: return [],["saved session chunk duration differs; generated a fresh run"]
if str(session.get("identity_hash","none"))!=str(identity_hash): return [],["saved session identity differs; generated a fresh run"] if str(session.get("identity_hash","none"))!=str(identity_hash): return [],["saved session identity differs; generated a fresh run"]
saved_last_frame_hash=(session.get("settings") or {}).get("last_frame_hash")
if str(saved_last_frame_hash or "none")!=str(last_frame_hash or "none"): return [],["saved session Last Frame differs; generated a fresh run"]
if reroll_from_chunk<0 or reroll_from_chunk>chunks: raise SessionValidationError("reroll_from_chunk must be 0 or a valid one-based chunk index") if reroll_from_chunk<0 or reroll_from_chunk>chunks: raise SessionValidationError("reroll_from_chunk must be 0 or a valid one-based chunk index")
limit=min(len(session["chunks"]),chunks) limit=min(len(session["chunks"]),chunks)
if reroll_from_chunk>0: limit=min(limit,reroll_from_chunk-1) if reroll_from_chunk>0: limit=min(limit,reroll_from_chunk-1)
@@ -103,7 +112,12 @@ def _conditioning_cache(*,clip,prompts,assets,final_has_last_frame,reference_ass
return cache return cache
FLF_STRATEGY = "context_bound_terminal_v1" FLF_STRATEGY = "terminal_merged_10s_seed_v2"
TERMINAL_SEED_POLICY = "physical_window_seed_v2"
TERMINAL_MERGE_CHUNK_SECONDS = 5.0
TERMINAL_MERGE_CONTEXT_FRAMES = 22
TERMINAL_PROMPT_POLICY_SHARED = "shared_prompt_v1"
TERMINAL_PROMPT_POLICY_TIMELINE = "paired_timeline_v1"
def _flf_contract_identity(base_hash: str) -> str: def _flf_contract_identity(base_hash: str) -> str:
@@ -111,6 +125,191 @@ def _flf_contract_identity(base_hash: str) -> str:
return hashlib.sha256(payload.encode("utf-8")).hexdigest() return hashlib.sha256(payload.encode("utf-8")).hexdigest()
def _terminal_flf_merge_enabled(
*,
multi_chunk_flf: bool,
chunks: int,
chunk_seconds: float,
prompt_hashes: list[str],
timeline_video_source,
) -> bool:
return bool(
multi_chunk_flf
and int(chunks) >= 2
and abs(float(chunk_seconds) - TERMINAL_MERGE_CHUNK_SECONDS) <= 1e-6
and timeline_video_source is None
and len(prompt_hashes) >= 2
)
def _terminal_pair_prompt(
prompts: list[str],
*,
pair_start: int,
chunk_seconds: float,
) -> tuple[str, str]:
"""Build one physical prompt without changing either logical prompt."""
first = str(prompts[int(pair_start)])
second = str(prompts[int(pair_start) + 1])
if first == second:
return first, TERMINAL_PROMPT_POLICY_SHARED
split = f"{float(chunk_seconds):g}"
stop = f"{float(chunk_seconds) * 2.0:g}"
return (
f"[0-{split}s]\n{first}\n\n[{split}-{stop}s]\n{second}",
TERMINAL_PROMPT_POLICY_TIMELINE,
)
def _atomic_terminal_prefix(
preserved: list[dict[str, Any]],
*,
chunks: int,
reroll_from_chunk: int,
) -> tuple[list[dict[str, Any]], bool]:
"""Never reuse only one logical half of a shared terminal sample."""
pair_start = int(chunks) - 2
reroll_hits_pair = int(reroll_from_chunk) in (int(chunks) - 1, int(chunks))
partial_pair = pair_start < len(preserved) < int(chunks)
if (reroll_hits_pair or partial_pair) and len(preserved) > pair_start:
return preserved[:pair_start], True
return preserved, False
def _terminal_sampling_plan(
*,
chunks: int,
completed: int,
merge_enabled: bool,
) -> tuple[tuple[int, ...], bool]:
if completed >= int(chunks):
return (), False
if not merge_enabled:
return tuple(range(completed, int(chunks))), False
pair_start = int(chunks) - 2
if completed > pair_start:
raise SequenceRuntimeError("terminal merged pair cannot resume from only one logical half")
return tuple(range(completed, pair_start)), True
def _terminal_physical_seed_plan(
*,
base_seed: int,
physical_window_start_index: int,
reroll_nonce: int,
) -> dict[str, Any]:
"""Return the one physical seed and the matching logical-entry metadata."""
start_index = int(physical_window_start_index)
nonce = int(reroll_nonce)
if start_index < 0:
raise ValueError("physical_window_start_index must be non-negative")
if nonce < 0:
raise ValueError("reroll_nonce must be non-negative")
if start_index == 0 and nonce == 0:
physical_seed = int(base_seed)
else:
physical_seed = derive_chunk_seed(base_seed, start_index, nonce)
return {
"physical_seed": physical_seed,
"logical_entry_seeds": (physical_seed, physical_seed),
}
def _terminal_execution_semantics(
*,
merge_enabled: bool,
prompt_policy: str | None = None,
) -> dict[str, str] | None:
if not merge_enabled:
return None
semantics = {
"flf_execution": FLF_STRATEGY,
"terminal_seed_policy": TERMINAL_SEED_POLICY,
}
if prompt_policy:
semantics["terminal_prompt_policy"] = str(prompt_policy)
return semantics
def _terminal_pair_contract(*, initial_pair: bool, chunk_seconds: float) -> dict[str, Any]:
"""Derive physical and logical AV ranges without fixed latent indices."""
if abs(float(chunk_seconds) - TERMINAL_MERGE_CHUNK_SECONDS) > 1e-6:
raise SequenceRuntimeError("terminal merged sampling currently requires 5-second logical chunks")
initial_frames = align_frame_count_up(int(round(float(chunk_seconds) * FPS)))
extension = make_extension_shape(
TERMINAL_MERGE_CONTEXT_FRAMES,
float(chunk_seconds),
)
if initial_pair:
physical_frames = align_frame_count_up(int(round(2.0 * float(chunk_seconds) * FPS)))
logical_frames = (initial_frames, extension.total_frames)
logical_trims = (0, TERMINAL_MERGE_CONTEXT_FRAMES)
first_video_stop = video_latent_t(initial_frames)
first_audio_stop = audio_latent_t(initial_frames)
else:
physical_frames = make_extension_shape(
TERMINAL_MERGE_CONTEXT_FRAMES,
2.0 * float(chunk_seconds),
).total_frames
logical_frames = (extension.total_frames, extension.total_frames)
logical_trims = (
TERMINAL_MERGE_CONTEXT_FRAMES,
TERMINAL_MERGE_CONTEXT_FRAMES,
)
first_video_stop = video_latent_t(extension.total_frames)
first_audio_stop = audio_latent_t(extension.total_frames)
physical_video_stop = video_latent_t(physical_frames)
physical_audio_stop = audio_latent_t(physical_frames)
video_overlap = context_slots(TERMINAL_MERGE_CONTEXT_FRAMES)
audio_overlap = audio_latent_t(TERMINAL_MERGE_CONTEXT_FRAMES)
video_slices = (
(0, first_video_stop),
(first_video_stop - video_overlap, physical_video_stop),
)
audio_slices = (
(0, first_audio_stop),
(first_audio_stop - audio_overlap, physical_audio_stop),
)
for index in range(2):
expected_video = video_latent_t(logical_frames[index])
expected_audio = audio_latent_t(logical_frames[index])
if video_slices[index][1] - video_slices[index][0] != expected_video:
raise SequenceRuntimeError("derived terminal video split does not match its logical frame count")
if audio_slices[index][1] - audio_slices[index][0] != expected_audio:
raise SequenceRuntimeError("derived terminal audio split does not match its logical frame count")
return {
"initial_pair": bool(initial_pair),
"physical_frames": int(physical_frames),
"physical_context_frames": 0 if initial_pair else TERMINAL_MERGE_CONTEXT_FRAMES,
"logical_frames": logical_frames,
"logical_trims": logical_trims,
"video_slices": video_slices,
"audio_slices": audio_slices,
}
def _split_terminal_merged_latents(
video: torch.Tensor,
audio: torch.Tensor,
contract: dict[str, Any],
) -> tuple[tuple[torch.Tensor, torch.Tensor], tuple[torch.Tensor, torch.Tensor]]:
expected_video = video_latent_t(int(contract["physical_frames"]))
expected_audio = audio_latent_t(int(contract["physical_frames"]))
if int(video.shape[2]) != expected_video or int(audio.shape[-1]) != expected_audio:
raise SequenceRuntimeError(
"terminal merged sampler output does not match the derived physical AV grid: "
f"video T={video.shape[2]} (expected {expected_video}), "
f"audio T={audio.shape[-1]} (expected {expected_audio})"
)
parts=[]
for video_range,audio_range in zip(contract["video_slices"],contract["audio_slices"]):
video_part=video[:,:,video_range[0]:video_range[1],...].contiguous()
audio_part=audio[...,audio_range[0]:audio_range[1]].contiguous()
parts.append((video_part,audio_part))
return parts[0],parts[1]
def _attach_terminal_flf_keyframes( def _attach_terminal_flf_keyframes(
conditioning, conditioning,
*, *,
@@ -141,7 +340,7 @@ def _attach_terminal_flf_keyframes(
return conditioning return conditioning
def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,sigmas:torch.Tensor,first_frame:torch.Tensor|None,last_frame:torch.Tensor|None,prompt_plan:dict[str,Any],width:int,height:int,continuity:str,base_seed:int,audio_continuity:bool,exact_total_duration:bool,diagnostics_mode:str,reroll_from_chunk:int,reroll_nonce:int,strict_compatibility:bool,debug:bool,seam_correction:str=SEAM_CORRECTION_OFF,enable_preview:bool=True,session:dict[str,Any]|None=None,initial_state:dict[str,Any]|None=None,latent_only:bool=False,reference_assets=None,reference_audio_source=None,reference_audio_vae=None,driving_audio_source=None,driving_audio_vae=None,reference_video_source=None,timeline_video_source=None): def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,sigmas:torch.Tensor,first_frame:torch.Tensor|None,last_frame:torch.Tensor|None,prompt_plan:dict[str,Any],width:int,height:int,continuity:str,base_seed:int,audio_continuity:bool,exact_total_duration:bool,diagnostics_mode:str,reroll_from_chunk:int,reroll_nonce:int,strict_compatibility:bool,debug:bool,seam_correction:str=SEAM_CORRECTION_OFF,enable_preview:bool=True,session:dict[str,Any]|None=None,initial_state:dict[str,Any]|None=None,latent_only:bool=False,reference_assets=None,reference_audio_source=None,reference_audio_vae=None,driving_audio_source=None,driving_audio_vae=None,reference_video_source=None,timeline_video_source=None):
from ..conditioning import detect_conditioning_mode, conditioning_mode_label from ..conditioning import detect_conditioning_mode, conditioning_display_label
from ..run_storage import get_active_run_storage from ..run_storage import get_active_run_storage
storage_controller=get_active_run_storage() storage_controller=get_active_run_storage()
diagnostics_mode=normalize_diagnostics_mode(diagnostics_mode); plan=validate_prompt_plan(prompt_plan); chunks=int(plan["chunks"]); chunk_seconds=float(plan["chunk_seconds"]); prompts=list(plan["prompts"]); prompt_hashes=list(plan["hashes"]); width,height=int(width),int(height) diagnostics_mode=normalize_diagnostics_mode(diagnostics_mode); plan=validate_prompt_plan(prompt_plan); chunks=int(plan["chunks"]); chunk_seconds=float(plan["chunk_seconds"]); prompts=list(plan["prompts"]); prompt_hashes=list(plan["hashes"]); width,height=int(width),int(height)
@@ -157,6 +356,7 @@ def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,si
if initial_state is not None and int(reroll_from_chunk) not in (0,1): raise SequenceRuntimeError("with initial_state, reroll_from_chunk can only be 0 or 1") if initial_state is not None and int(reroll_from_chunk) not in (0,1): raise SequenceRuntimeError("with initial_state, reroll_from_chunk can only be 0 or 1")
try: conditioning_mode=detect_conditioning_mode(first_frame=first_frame,last_frame=last_frame,reference_assets=reference_assets) try: conditioning_mode=detect_conditioning_mode(first_frame=first_frame,last_frame=last_frame,reference_assets=reference_assets)
except ValueError as exc: raise SequenceRuntimeError(str(exc)) from exc except ValueError as exc: raise SequenceRuntimeError(str(exc)) from exc
conditioning_display=conditioning_display_label(has_first=first_frame is not None,has_last=last_frame is not None,has_reference=reference_assets is not None)
reference_warning="" reference_warning=""
if reference_assets is not None: if reference_assets is not None:
from ..reference import validate_reference_prompts from ..reference import validate_reference_prompts
@@ -181,6 +381,11 @@ def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,si
issues=check_comfy_h3_runtime() issues=check_comfy_h3_runtime()
assets=prepare_identity_assets(video_vae,width=width,height=height,first_frame=first_frame,last_frame=last_frame,encode_latents=False) assets=prepare_identity_assets(video_vae,width=width,height=height,first_frame=first_frame,last_frame=last_frame,encode_latents=False)
multi_chunk_flf=bool(chunks>=2 and first_frame is not None and last_frame is not None) multi_chunk_flf=bool(chunks>=2 and first_frame is not None and last_frame is not None)
terminal_merge_enabled=_terminal_flf_merge_enabled(multi_chunk_flf=multi_chunk_flf,chunks=chunks,chunk_seconds=chunk_seconds,prompt_hashes=prompt_hashes,timeline_video_source=timeline_video_source)
terminal_prompt=None
terminal_prompt_policy=None
if terminal_merge_enabled:
terminal_prompt,terminal_prompt_policy=_terminal_pair_prompt(prompts,pair_start=chunks-2,chunk_seconds=chunk_seconds)
from ..reference import combine_hybrid_visual_identity from ..reference import combine_hybrid_visual_identity
visual_identity_hash=combine_hybrid_visual_identity(keyframe_identity_hash=assets.identity_hash,reference_assets=reference_assets,has_first=first_frame is not None,has_last=last_frame is not None) visual_identity_hash=combine_hybrid_visual_identity(keyframe_identity_hash=assets.identity_hash,reference_assets=reference_assets,has_first=first_frame is not None,has_last=last_frame is not None)
sequence_identity_hash=combine_reference_audio_identity(visual_identity_hash,reference_audio_source) sequence_identity_hash=combine_reference_audio_identity(visual_identity_hash,reference_audio_source)
@@ -191,20 +396,27 @@ def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,si
sequence_identity_hash=_flf_contract_identity(sequence_identity_hash) sequence_identity_hash=_flf_contract_identity(sequence_identity_hash)
current_model_fingerprint=model_fingerprint(model,extra_wrapper_keys=("h3_continuum_join.apply_model.v1",)) current_model_fingerprint=model_fingerprint(model,extra_wrapper_keys=("h3_continuum_join.apply_model.v1",))
if storage_controller is not None: if storage_controller is not None:
stored_session=storage_controller.prepare(model=model,model_fingerprint_value=current_model_fingerprint,clip=clip,video_vae=video_vae,sampler=sampler,sigmas=sigmas,prompt_plan=plan,width=width,height=height,chunk_seconds=chunk_seconds,continuity=continuity,audio_continuity=audio_continuity,base_seed=base_seed,reroll_from_chunk=reroll_from_chunk,reroll_nonce=reroll_nonce,first_frame_hash=assets.first_frame_hash,last_frame_hash=assets.last_frame_hash,identity_hash=sequence_identity_hash,strict_compatibility=strict_compatibility,existing_session=session,reference_contract=reference_assets.contract if reference_assets is not None else None,conditioning_mode=conditioning_mode,reference_audio_contract=reference_audio_source.contract if reference_audio_source is not None else None,reference_audio_vae=reference_audio_vae,driving_audio_contract=driving_audio_source.contract if driving_audio_source is not None else None,driving_audio_vae=driving_audio_vae,reference_video_contract=reference_video_source.contract if reference_video_source is not None else None,timeline_video_contract=timeline_video_source.contract if timeline_video_source is not None else None) stored_session=storage_controller.prepare(model=model,model_fingerprint_value=current_model_fingerprint,clip=clip,video_vae=video_vae,sampler=sampler,sigmas=sigmas,prompt_plan=plan,width=width,height=height,chunk_seconds=chunk_seconds,continuity=continuity,audio_continuity=audio_continuity,base_seed=base_seed,reroll_from_chunk=reroll_from_chunk,reroll_nonce=reroll_nonce,first_frame_hash=assets.first_frame_hash,last_frame_hash=assets.last_frame_hash,identity_hash=sequence_identity_hash,strict_compatibility=strict_compatibility,existing_session=session,reference_contract=reference_assets.contract if reference_assets is not None else None,conditioning_mode=conditioning_mode,reference_audio_contract=reference_audio_source.contract if reference_audio_source is not None else None,reference_audio_vae=reference_audio_vae,driving_audio_contract=driving_audio_source.contract if driving_audio_source is not None else None,driving_audio_vae=driving_audio_vae,reference_video_contract=reference_video_source.contract if reference_video_source is not None else None,timeline_video_contract=timeline_video_source.contract if timeline_video_source is not None else None,execution_semantics=_terminal_execution_semantics(merge_enabled=terminal_merge_enabled,prompt_policy=terminal_prompt_policy))
reroll_nonce=storage_controller.effective_reroll_nonce reroll_nonce=storage_controller.effective_reroll_nonce
if stored_session is not None: session=stored_session if stored_session is not None: session=stored_session
accelerators=accelerator_summary(model) accelerators=accelerator_summary(model)
if "Continuum APPLY_MODEL wrapper installed" not in accelerators: accelerators+="; Continuum APPLY_MODEL wrapper installed" if "Continuum APPLY_MODEL wrapper installed" not in accelerators: accelerators+="; Continuum APPLY_MODEL wrapper installed"
effective_reroll_from_chunk=0 if storage_controller is not None and session is not None and bool((session.get("settings") or {}).get("run_storage_validated_prefix")) else int(reroll_from_chunk) effective_reroll_from_chunk=0 if storage_controller is not None and session is not None and bool((session.get("settings") or {}).get("run_storage_validated_prefix")) else int(reroll_from_chunk)
preserved,reuse_notes=_preserved_prefix(session=session,prompt_hashes=prompt_hashes,chunks=chunks,reroll_from_chunk=effective_reroll_from_chunk,width=width,height=height,chunk_seconds=chunk_seconds,identity_hash=sequence_identity_hash) preserved,reuse_notes=_preserved_prefix(session=session,prompt_hashes=prompt_hashes,chunks=chunks,reroll_from_chunk=effective_reroll_from_chunk,width=width,height=height,chunk_seconds=chunk_seconds,identity_hash=sequence_identity_hash,last_frame_hash=assets.last_frame_hash)
if multi_chunk_flf and session is not None: if multi_chunk_flf and session is not None:
saved_strategy=str((session.get("settings") or {}).get("flf_execution", "")) saved_strategy=str((session.get("settings") or {}).get("flf_execution", ""))
if saved_strategy!=FLF_STRATEGY: if saved_strategy!=FLF_STRATEGY:
preserved=[] preserved=[]
reuse_notes.append("FLF chunk contract changed; regenerated the requested FLF sequence.") reuse_notes.append("FLF chunk contract changed; regenerated the requested FLF sequence.")
if terminal_merge_enabled:
preserved,atomic_reset=_atomic_terminal_prefix(preserved,chunks=chunks,reroll_from_chunk=int(reroll_from_chunk))
if atomic_reset:
reuse_notes.append("terminal merged pair is atomic; regenerated both final logical chunks")
if multi_chunk_flf: if multi_chunk_flf:
reuse_notes.insert(0,"FLF execution: context-bound terminal v1; one sampling pass per chunk; the final chunk is anchored from the continuation-context tail to the supplied Last Frame.") if terminal_merge_enabled:
reuse_notes.insert(0,"FLF execution: terminal merged 10s seed v2; the final two 5-second logical chunks share one physical sample and are split back into normal chunk entries.")
else:
reuse_notes.insert(0,"FLF execution: terminal merge unavailable for this request; retained one sampling pass per logical chunk.")
if reference_assets is not None: if reference_assets is not None:
reuse_notes.insert(0,f"Reference conditioning: {reference_assets.count} image(s), size={reference_assets.size_mode}; persistent across all chunks.") reuse_notes.insert(0,f"Reference conditioning: {reference_assets.count} image(s), size={reference_assets.size_mode}; persistent across all chunks.")
if reference_warning: reuse_notes.append(reference_warning) if reference_warning: reuse_notes.append(reference_warning)
@@ -239,6 +451,10 @@ def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,si
reference_video_assets=encode_reference_video(video_vae,reference_video_source) reference_video_assets=encode_reference_video(video_vae,reference_video_source)
if timeline_video_source is None: if timeline_video_source is None:
cache=_conditioning_cache(clip=clip,prompts=prompts,assets=assets,final_has_last_frame=last_frame is not None,reference_assets=reference_assets,reference_audio_assets=reference_audio_assets,timeline_video_assets=reference_video_assets) cache=_conditioning_cache(clip=clip,prompts=prompts,assets=assets,final_has_last_frame=last_frame is not None,reference_assets=reference_assets,reference_audio_assets=reference_audio_assets,timeline_video_assets=reference_video_assets)
if terminal_merge_enabled and terminal_prompt is not None:
terminal_key=_conditioning_cache_key(terminal_prompt,include_last=True,reference_assets=reference_assets)
if terminal_key not in cache:
cache.update(_conditioning_cache(clip=clip,prompts=[terminal_prompt],assets=assets,final_has_last_frame=True,reference_assets=reference_assets,reference_audio_assets=reference_audio_assets,timeline_video_assets=reference_video_assets))
entries=preserved[:]; previous_state=None entries=preserved[:]; previous_state=None
if entries: if entries:
try: previous_state=entry_to_state(entries[-1]) try: previous_state=entry_to_state(entries[-1])
@@ -261,7 +477,8 @@ def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,si
_record_context_diagnostics(tracker=context_diagnostics,reports=sampling_reports,state=entry_to_state(reused_entry),continuity=continuity,reused=True) _record_context_diagnostics(tracker=context_diagnostics,reports=sampling_reports,state=entry_to_state(reused_entry),continuity=continuity,reused=True)
elif previous_state is not None: elif previous_state is not None:
_record_context_diagnostics(tracker=context_diagnostics,reports=sampling_reports,state=previous_state,continuity=continuity,reused=True) _record_context_diagnostics(tracker=context_diagnostics,reports=sampling_reports,state=previous_state,continuity=continuity,reused=True)
for sequence_index in range(len(entries),chunks): normal_indices,terminal_merge_pending=_terminal_sampling_plan(chunks=chunks,completed=len(entries),merge_enabled=terminal_merge_enabled)
for sequence_index in normal_indices:
prompt=prompts[sequence_index]; prompt_hash_value=prompt_hashes[sequence_index]; is_final=sequence_index==chunks-1; effective_reroll_nonce=int(reroll_nonce) if int(reroll_from_chunk)>0 and sequence_index+1>=int(reroll_from_chunk) else 0; seed=derive_chunk_seed(base_seed,sequence_index,effective_reroll_nonce); motion_score=0.0; video_context=None; audio_context=None; context_before=None prompt=prompts[sequence_index]; prompt_hash_value=prompt_hashes[sequence_index]; is_final=sequence_index==chunks-1; effective_reroll_nonce=int(reroll_nonce) if int(reroll_from_chunk)>0 and sequence_index+1>=int(reroll_from_chunk) else 0; seed=derive_chunk_seed(base_seed,sequence_index,effective_reroll_nonce); motion_score=0.0; video_context=None; audio_context=None; context_before=None
timeline_video_assets=reference_video_assets timeline_video_assets=reference_video_assets
chunk_cache=cache chunk_cache=cache
@@ -283,8 +500,6 @@ def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,si
base_conditioning=attach_keyframes(chunk_cache[conditioning_key],frame_count=shape.total_frames,first_latent=chunk_assets.first_latent,last_latent=chunk_assets.last_latent if include_chunk_last else None) base_conditioning=attach_keyframes(chunk_cache[conditioning_key],frame_count=shape.total_frames,first_latent=chunk_assets.first_latent,last_latent=chunk_assets.last_latent if include_chunk_last else None)
video_context,audio_context,grid_offset=select_context(previous_state,context_frames,include_audio=bool(audio_continuity)); context_before=context_fingerprint(video_context,audio_context) video_context,audio_context,grid_offset=select_context(previous_state,context_frames,include_audio=bool(audio_continuity)); context_before=context_fingerprint(video_context,audio_context)
conditioning=prepare_conditioning(base_conditioning,video_context=video_context,audio_context=audio_context,audio_grid_offset=grid_offset,context_frames=context_frames,new_frame_count=shape.total_frames,first_frame_policy=POLICY_REPLACE,preserve_last_frame=True) conditioning=prepare_conditioning(base_conditioning,video_context=video_context,audio_context=audio_context,audio_grid_offset=grid_offset,context_frames=context_frames,new_frame_count=shape.total_frames,first_frame_policy=POLICY_REPLACE,preserve_last_frame=True)
if multi_chunk_flf and is_final:
conditioning=_attach_terminal_flf_keyframes(conditioning,video_context=video_context,context_frames=context_frames,frame_count=shape.total_frames,last_latent=assets.last_latent)
clip_index=int(previous_state["clip_index"])+1; chunk_plan=make_plan(continuation=True,clip_index=clip_index,total_frames=shape.total_frames,trim_frames=context_frames,width=width,height=height,context_frames=context_frames,state_capacity_frames=largest_context_capacity(shape.net_new_frames),requested_extend_seconds=chunk_seconds,debug=debug) clip_index=int(previous_state["clip_index"])+1; chunk_plan=make_plan(continuation=True,clip_index=clip_index,total_frames=shape.total_frames,trim_frames=context_frames,width=width,height=height,context_frames=context_frames,state_capacity_frames=largest_context_capacity(shape.net_new_frames),requested_extend_seconds=chunk_seconds,debug=debug)
driving_audio_latent=slice_driving_audio_latent(driving_audio_assets,cumulative_retained_before=retained_frames,total_frames=int(chunk_plan["total_frames"]),trim_frames=int(chunk_plan["trim_frames"]),fps=FPS) driving_audio_latent=slice_driving_audio_latent(driving_audio_assets,cumulative_retained_before=retained_frames,total_frames=int(chunk_plan["total_frames"]),trim_frames=int(chunk_plan["trim_frames"]),fps=FPS)
conditioning=attach_driving_audio(conditioning,driving_audio_latent) conditioning=attach_driving_audio(conditioning,driving_audio_latent)
@@ -298,6 +513,62 @@ def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,si
sampling_reports.append(f"chunk {sequence_index+1}/{chunks}: seed={seed}, frames={chunk_plan['total_frames']}, trim={chunk_plan['trim_frames']}, retained_total={retained_frames}, context={context_frames} ({reason}), motion={motion_score:.6f}, "+(f"interop=emitted actual_prefix={CONTINUUM_ACTUAL_PREFIX_STEPS} consumer=not_observable" if context_before is not None else "interop=not_emitted")) sampling_reports.append(f"chunk {sequence_index+1}/{chunks}: seed={seed}, frames={chunk_plan['total_frames']}, trim={chunk_plan['trim_frames']}, retained_total={retained_frames}, context={context_frames} ({reason}), motion={motion_score:.6f}, "+(f"interop=emitted actual_prefix={CONTINUUM_ACTUAL_PREFIX_STEPS} consumer=not_observable" if context_before is not None else "interop=not_emitted"))
del sampled,latent,conditioning,chunk_model,chunk_cache del sampled,latent,conditioning,chunk_model,chunk_cache
if timeline_video_source is not None and timeline_video_assets is not None: del timeline_video_assets if timeline_video_source is not None and timeline_video_assets is not None: del timeline_video_assets
if terminal_merge_pending:
pair_start=chunks-2
if len(entries)!=pair_start:
raise SequenceRuntimeError(f"terminal merged pair expected {pair_start} completed chunks, got {len(entries)}")
prompt=terminal_prompt
seed_nonce=int(reroll_nonce) if int(reroll_from_chunk)>0 else 0
terminal_seed_plan=_terminal_physical_seed_plan(base_seed=base_seed,physical_window_start_index=pair_start,reroll_nonce=seed_nonce)
physical_seed=int(terminal_seed_plan["physical_seed"])
initial_pair=previous_state is None
contract=_terminal_pair_contract(initial_pair=initial_pair,chunk_seconds=chunk_seconds)
physical_frames=int(contract["physical_frames"])
physical_context_frames=int(contract["physical_context_frames"])
latent=empty_h3_latent(width,height,physical_frames)
conditioning_key=_conditioning_cache_key(prompt,include_last=True,reference_assets=reference_assets)
base_conditioning=attach_keyframes(cache[conditioning_key],frame_count=physical_frames,first_latent=assets.first_latent,last_latent=assets.last_latent)
video_context=None; audio_context=None; context_before=None; motion_score=0.0
if initial_pair:
conditioning=base_conditioning
physical_clip_index=1
reason="terminal merged initial 10-second sample"
else:
selected_context,motion_score,selected_reason=choose_context_frames(continuity,previous_state)
video_context,audio_context,grid_offset=select_context(previous_state,TERMINAL_MERGE_CONTEXT_FRAMES,include_audio=bool(audio_continuity))
context_before=context_fingerprint(video_context,audio_context)
conditioning=prepare_conditioning(base_conditioning,video_context=video_context,audio_context=audio_context,audio_grid_offset=grid_offset,context_frames=TERMINAL_MERGE_CONTEXT_FRAMES,new_frame_count=physical_frames,first_frame_policy=POLICY_REPLACE,preserve_last_frame=True)
physical_clip_index=int(previous_state["clip_index"])+1
reason=f"terminal merged 10-second sample with 22-frame context; continuity selection was {selected_context} ({selected_reason})"
driving_audio_latent=slice_driving_audio_latent(driving_audio_assets,cumulative_retained_before=retained_frames,total_frames=physical_frames,trim_frames=physical_context_frames,fps=FPS)
conditioning=attach_driving_audio(conditioning,driving_audio_latent)
chunk_model=clone_model_for_chunk(model,strict=bool(strict_compatibility),debug=bool(debug),chunk_index=physical_clip_index,context_frames=physical_context_frames if not initial_pair else None)
sampled=sample_chunk(model=chunk_model,conditioning=conditioning,latent=latent,sampler=sampler,sigmas=sigmas,seed=physical_seed,enable_preview=bool(enable_preview))
if context_before is not None and video_context is not None:
assert_context_unchanged(video_context,audio_context,context_before)
sampled_video,sampled_audio=latent_to_cpu(sampled)
logical_parts=_split_terminal_merged_latents(sampled_video,sampled_audio,contract)
sampling_reports.append(f"terminal physical sample: terminal_pair={pair_start+1}-{chunks}, physical_seed={physical_seed}, physical_frames={physical_frames}, sampling_passes=1, shared_physical_sample=true, terminal_prompt_policy={terminal_prompt_policy}, trim={physical_context_frames}, reason={reason}")
for offset,(video_part,audio_part) in enumerate(logical_parts):
sequence_index=pair_start+offset
total_frames=int(contract["logical_frames"][offset])
trim_frames=int(contract["logical_trims"][offset])
continuation=bool(trim_frames)
clip_index=physical_clip_index+offset
context_frames=trim_frames if continuation else 5
chunk_plan=make_plan(continuation=continuation,clip_index=clip_index,total_frames=total_frames,trim_frames=trim_frames,width=width,height=height,context_frames=context_frames,state_capacity_frames=largest_context_capacity(total_frames-trim_frames),requested_extend_seconds=chunk_seconds,debug=debug)
logical_latent=latent_from_cpu(video_part,audio_part)
logical_seed=int(terminal_seed_plan["logical_entry_seeds"][offset])
entry=make_chunk_entry(latent=logical_latent,plan=chunk_plan,prompt=prompts[sequence_index],prompt_hash=prompt_hashes[sequence_index],seed=logical_seed,context_frames=trim_frames,motion_score=motion_score,reused=False)
entries.append(entry)
previous_state=entry_to_state(entry)
_record_context_diagnostics(tracker=context_diagnostics,reports=sampling_reports,state=previous_state,continuity=continuity,reused=False)
if storage_controller is not None:
storage_controller.commit_chunk(entry,position=sequence_index)
retained_frames+=int(chunk_plan["net_frames"])
sampling_reports.append(f"chunk {sequence_index+1}/{chunks}: seed={logical_seed}, frames={total_frames}, trim={trim_frames}, retained_total={retained_frames}, context={trim_frames}, shared_physical_sample=terminal_10s_seed_v2")
del logical_latent
del sampled,sampled_video,sampled_audio,logical_parts,latent,conditioning,chunk_model
if len(entries)!=chunks: raise SequenceRuntimeError(f"internal sequence length mismatch: expected {chunks}, got {len(entries)}") if len(entries)!=chunks: raise SequenceRuntimeError(f"internal sequence length mismatch: expected {chunks}, got {len(entries)}")
if latent_only: if latent_only:
last_state=entry_to_state(entries[-1]); parent_id=session.get("session_id") if session is not None else None last_state=entry_to_state(entries[-1]); parent_id=session.get("session_id") if session is not None else None
@@ -308,7 +579,7 @@ def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,si
if reference_video_source is not None: settings["reference_video_contract"]=reference_video_source.contract if reference_video_source is not None: settings["reference_video_contract"]=reference_video_source.contract
if timeline_video_source is not None: settings["timeline_video_contract"]=timeline_video_source.contract if timeline_video_source is not None: settings["timeline_video_contract"]=timeline_video_source.contract
new_session=make_session(chunks=entries,width=width,height=height,chunk_seconds=chunk_seconds,identity_hash=sequence_identity_hash,model_fingerprint_value=current_model_fingerprint,parent_session_id=parent_id,reroll_from_chunk=int(reroll_from_chunk),settings=settings) new_session=make_session(chunks=entries,width=width,height=height,chunk_seconds=chunk_seconds,identity_hash=sequence_identity_hash,model_fingerprint_value=current_model_fingerprint,parent_session_id=parent_id,reroll_from_chunk=int(reroll_from_chunk),settings=settings)
report_lines=[f"H3 Continuum V3 {PACKAGE_VERSION}",f"Conditioning mode: {conditioning_mode_label(conditioning_mode)}.",prompt_plan_report(plan),"Decode: external ComfyUI Core VAE nodes; full raw AV chunks retained.",accelerators,"Execution: conditioning precomputed; single call-local MODEL clone per chunk; no internal VAE decode.",*reuse_notes] report_lines=[f"H3 Continuum V3 {PACKAGE_VERSION}",f"Conditioning mode: {conditioning_display}.",prompt_plan_report(plan),"Decode: external ComfyUI Core VAE nodes; full raw AV chunks retained.",accelerators,"Execution: conditioning precomputed; single call-local MODEL clone per chunk; no internal VAE decode.",*reuse_notes]
if diagnostics_mode!=DIAGNOSTICS_OFF: report_lines.extend(sampling_reports) if diagnostics_mode!=DIAGNOSTICS_OFF: report_lines.extend(sampling_reports)
report_lines.extend([session_summary(new_session),f"Output: {len(entries)} raw AV latent chunk(s); connect Core VAE Decode nodes, then H3 Continuum Assemble V3."]) report_lines.extend([session_summary(new_session),f"Output: {len(entries)} raw AV latent chunk(s); connect Core VAE Decode nodes, then H3 Continuum Assemble V3."])
return entries,last_state,new_session,"\n".join(report_lines) return entries,last_state,new_session,"\n".join(report_lines)
+3 -1
View File
@@ -71,7 +71,9 @@ def assemble_decoded_chunks(
if audio_seam not in AUDIO_SEAM_OPTIONS: if audio_seam not in AUDIO_SEAM_OPTIONS:
raise ValueError(f"unknown Audio Seam mode: {audio_seam!r}") raise ValueError(f"unknown Audio Seam mode: {audio_seam!r}")
chunk_plans = list(plan["chunks"]) chunk_plans = list(
plan["decode_groups"] if "decode_groups" in plan else plan["chunks"]
)
if len(images) != len(chunk_plans) or len(audio) != len(chunk_plans): if len(images) != len(chunk_plans) or len(audio) != len(chunk_plans):
raise ValueError( raise ValueError(
"decoded chunk count does not match the V3 assembly plan: " "decoded chunk count does not match the V3 assembly plan: "
+23 -3
View File
@@ -338,13 +338,33 @@ class H3ContinuumSamplerV3:
**clip_prompt_inputs, **clip_prompt_inputs,
) )
assembly_plan = make_assembly_plan( from ..v2.sequence import _terminal_flf_merge_enabled
from .plan import prepare_physical_decode_entries
terminal_merged = _terminal_flf_merge_enabled(
multi_chunk_flf=(
first_frame is not None
and advanced_values["last_frame"] is not None
and len(entries) > 1
),
chunks=len(entries),
chunk_seconds=float(chunk_seconds),
prompt_hashes=[str(entry["prompt_hash"]) for entry in entries],
timeline_video_source=timeline_video_source,
)
decode_entries, assembly_plan = prepare_physical_decode_entries(
entries, entries,
chunk_seconds=float(chunk_seconds), chunk_seconds=float(chunk_seconds),
preserve_final_frame=advanced_values["last_frame"] is not None, preserve_final_frame=advanced_values["last_frame"] is not None,
terminal_merged=terminal_merged,
)
video_latents = [{"samples": entry["video"]} for entry in decode_entries]
audio_latents = [{"samples": entry["audio"]} for entry in decode_entries]
if terminal_merged:
report = str(report).rstrip() + (
"\nDecode: terminal merged latent retained as one physical external "
"Core VAE decode group; logical Run Storage entries are unchanged."
) )
video_latents = [{"samples": entry["video"]} for entry in entries]
audio_latents = [{"samples": entry["audio"]} for entry in entries]
result = { result = {
"last_state": last_state, "last_state": last_state,
"session": session, "session": session,
+148
View File
@@ -141,3 +141,151 @@ def make_assembly_plan(*args, **kwargs):
_enrich_assembly_plan(plan) _enrich_assembly_plan(plan)
validate_assembly_plan(plan) validate_assembly_plan(plan)
return plan return plan
def _recombine_terminal_tensor(
first,
second,
*,
slices,
time_dim: int,
name: str,
):
"""Invert the terminal logical split without changing any tensor value."""
import torch
if not torch.is_tensor(first) or not torch.is_tensor(second):
raise ValueError(f"terminal {name} entries must be tensors")
if first.ndim != second.ndim:
raise ValueError(f"terminal {name} tensor ranks differ")
dim = int(time_dim)
if dim < 0:
dim += first.ndim
if dim < 0 or dim >= first.ndim:
raise ValueError(f"terminal {name} time dimension is invalid")
for axis, (left, right) in enumerate(zip(first.shape, second.shape)):
if axis != dim and int(left) != int(right):
raise ValueError(f"terminal {name} tensor geometry differs")
(first_start, first_stop), (second_start, second_stop) = slices
first_start = int(first_start)
first_stop = int(first_stop)
second_start = int(second_start)
second_stop = int(second_stop)
if first_start != 0 or second_start > first_stop or second_stop <= first_stop:
raise ValueError(f"terminal {name} split ranges do not form one physical unit")
if int(first.shape[dim]) != first_stop - first_start:
raise ValueError(f"terminal {name} first slice length is inconsistent")
if int(second.shape[dim]) != second_stop - second_start:
raise ValueError(f"terminal {name} second slice length is inconsistent")
overlap_start = max(first_start, second_start)
overlap_stop = min(first_stop, second_stop)
overlap_length = max(0, overlap_stop - overlap_start)
if overlap_length:
first_overlap = first.narrow(dim, overlap_start - first_start, overlap_length)
second_overlap = second.narrow(dim, overlap_start - second_start, overlap_length)
if not torch.equal(first_overlap, second_overlap):
raise ValueError(
f"terminal {name} overlap differs; refusing lossy physical reconstruction"
)
output_shape = list(first.shape)
output_shape[dim] = second_stop
output = first.new_empty(output_shape)
output.narrow(dim, first_start, first_stop - first_start).copy_(first)
output.narrow(dim, second_start, second_stop - second_start).copy_(second)
return output.contiguous()
def prepare_physical_decode_entries(
entries,
*,
chunk_seconds: float,
preserve_final_frame: bool,
terminal_merged: bool,
):
"""Return external-VAE decode units while preserving logical plan metadata."""
logical_entries = list(entries)
plan = make_assembly_plan(
logical_entries,
chunk_seconds=float(chunk_seconds),
preserve_final_frame=bool(preserve_final_frame),
)
if not terminal_merged:
return logical_entries, plan
if len(logical_entries) < 2:
raise ValueError("terminal physical decode requires two logical entries")
from ..v2.sequence import _terminal_pair_contract
initial_pair = len(logical_entries) == 2
contract = _terminal_pair_contract(
initial_pair=initial_pair,
chunk_seconds=float(chunk_seconds),
)
first_entry = logical_entries[-2]
second_entry = logical_entries[-1]
physical_video = _recombine_terminal_tensor(
first_entry["video"],
second_entry["video"],
slices=contract["video_slices"],
time_dim=2,
name="video",
)
physical_audio = _recombine_terminal_tensor(
first_entry["audio"],
second_entry["audio"],
slices=contract["audio_slices"],
time_dim=-1,
name="audio",
)
physical_entry = dict(first_entry)
physical_entry["video"] = physical_video
physical_entry["audio"] = physical_audio
decode_entries = [*logical_entries[:-2], physical_entry]
logical_chunks = list(plan["chunks"])
decode_groups = []
for chunk in logical_chunks[:-2]:
group = dict(chunk)
group["logical_chunk_indices"] = [int(chunk["chunk_index"])]
group["terminal_merged"] = False
decode_groups.append(group)
first_chunk = logical_chunks[-2]
second_chunk = logical_chunks[-1]
frame_start = int(first_chunk.get("frame_start", 0))
net_frames = int(contract["physical_frames"]) - int(
contract["physical_context_frames"]
)
terminal_group = dict(first_chunk)
terminal_group.update(
{
"total_frames": int(contract["physical_frames"]),
"trim_frames": int(contract["physical_context_frames"]),
"net_frames": net_frames,
"context_frames": int(contract["physical_context_frames"]),
"expected_video_latent_t": int(physical_video.shape[2]),
"expected_audio_latent_t": int(physical_audio.shape[-1]),
"frame_start": frame_start,
"frame_stop": frame_start + net_frames,
"logical_chunk_indices": [
int(first_chunk["chunk_index"]),
int(second_chunk["chunk_index"]),
],
"terminal_merged": True,
}
)
decode_groups.append(terminal_group)
plan = dict(plan)
plan["decode_group_version"] = 1
plan["logical_chunk_count"] = len(logical_chunks)
plan["physical_decode_group_count"] = len(decode_groups)
plan["decode_groups"] = decode_groups
validate_assembly_plan(plan)
return decode_entries, plan
+6 -2
View File
@@ -390,7 +390,9 @@ def analyze_decoded_boundaries(
"""Analyze every decoded chunk boundary without changing decoded content.""" """Analyze every decoded chunk boundary without changing decoded content."""
plan = validate_assembly_plan(assembly_plan) plan = validate_assembly_plan(assembly_plan)
chunks = list(plan["chunks"]) chunks = list(
plan["decode_groups"] if "decode_groups" in plan else plan["chunks"]
)
if len(images) != len(chunks): if len(images) != len(chunks):
raise ValueError("decoded image count does not match the assembly plan") raise ValueError("decoded image count does not match the assembly plan")
analyses: list[VideoBoundaryAnalysis] = [] analyses: list[VideoBoundaryAnalysis] = []
@@ -428,7 +430,9 @@ def correct_decoded_boundaries(
"""Normalize qualified transient exposure shifts without changing motion.""" """Normalize qualified transient exposure shifts without changing motion."""
plan = validate_assembly_plan(assembly_plan) plan = validate_assembly_plan(assembly_plan)
chunks = list(plan["chunks"]) chunks = list(
plan["decode_groups"] if "decode_groups" in plan else plan["chunks"]
)
if len(images) != len(chunks): if len(images) != len(chunks):
raise ValueError("decoded image count does not match the assembly plan") raise ValueError("decoded image count does not match the assembly plan")
corrected = list(images) corrected = list(images)