fix: stabilize V3.4 hybrid FLF continuation
This commit is contained in:
@@ -1,7 +1,7 @@
|
|||||||
# AGENTS.md
|
# AGENTS.md
|
||||||
|
|
||||||
## Scope
|
## Scope
|
||||||
Native MiniMax H3 audio/video continuation for ComfyUI. V1 supplies manual Join/Finish nodes; V2 integrates N chunks over the same continuation core.
|
MiniMax H3 Continuum V3.4 for ComfyUI. Treat the V3.4 sampler and assembler as the current public baseline; preserve legacy workflow loading without letting legacy controls change V3.4 behavior.
|
||||||
|
|
||||||
## Non-negotiable rules
|
## Non-negotiable rules
|
||||||
- Follow ComfyUI Core behavior for user-facing validation. Continuum-only prompt restrictions, model allowlists, compatibility gates, and policy-based execution stops are prohibited.
|
- Follow ComfyUI Core behavior for user-facing validation. Continuum-only prompt restrictions, model allowlists, compatibility gates, and policy-based execution stops are prohibited.
|
||||||
@@ -10,7 +10,7 @@ Native MiniMax H3 audio/video continuation for ComfyUI. V1 supplies manual Join/
|
|||||||
- Keep warnings diagnostic-only unless execution cannot continue safely. Hard stops are limited to corrupt internal contracts such as invalid latent/layout topology, incompatible Assembly Plan data, broken stored revision schema/hash, unusable required payloads, or decode formats that cannot be assembled.
|
- Keep warnings diagnostic-only unless execution cannot continue safely. Hard stops are limited to corrupt internal contracts such as invalid latent/layout topology, incompatible Assembly Plan data, broken stored revision schema/hash, unusable required payloads, or decode formats that cannot be assembled.
|
||||||
- Unknown upstream nodes, models, wrappers, or merged models must not be rejected solely because they are unknown.
|
- Unknown upstream nodes, models, wrappers, or merged models must not be rejected solely because they are unknown.
|
||||||
- Keep legacy `strict_compatibility` inputs loadable for old workflows, but ignore their value. They must never enable blocking behavior.
|
- Keep legacy `strict_compatibility` inputs loadable for old workflows, but ignore their value. They must never enable blocking behavior.
|
||||||
- Before code changes, create a snapshot of the active local runtime. Preserve the current compact V3.4 UI unless the user explicitly approves a UI change.
|
- Before every change, run `tools\snapshot.ps1` against the authoritative source. Preserve the compact V3.4 UI unless the user explicitly approves a UI change.
|
||||||
- Runtime tests must record the workflow, prompt, media inputs, model/LoRA/sampler/step settings, dimensions, chunk count/duration, elapsed time, and observed result. Inspect embedded workflow metadata when present.
|
- Runtime tests must record the workflow, prompt, media inputs, model/LoRA/sampler/step settings, dimensions, chunk count/duration, elapsed time, and observed result. Inspect embedded workflow metadata when present.
|
||||||
- Do not globally replace `PackedLayout`, `MiniMaxH3.extra_conds`, or other ComfyUI classes.
|
- Do not globally replace `PackedLayout`, `MiniMaxH3.extra_conds`, or other ComfyUI classes.
|
||||||
- Keep SageAttention, Sol-Attn, and Spectrum external; integrate through public ComfyUI wrappers and H3 payload contracts.
|
- Keep SageAttention, Sol-Attn, and Spectrum external; integrate through public ComfyUI wrappers and H3 payload contracts.
|
||||||
@@ -18,17 +18,23 @@ Native MiniMax H3 audio/video continuation for ComfyUI. V1 supplies manual Join/
|
|||||||
- Never silently resize a continuation State or Session to a different resolution.
|
- Never silently resize a continuation State or Session to a different resolution.
|
||||||
- Unknown H3 layout contracts must fail clearly instead of rendering a subtly incorrect join.
|
- Unknown H3 layout contracts must fail clearly instead of rendering a subtly incorrect join.
|
||||||
- State and Session files remain safetensors + JSON and are written atomically.
|
- State and Session files remain safetensors + JSON and are written atomically.
|
||||||
- UI nodes call reusable non-UI modules. V1 and V2 must use the same temporal, state, continuation, and layout logic.
|
- UI nodes call reusable non-UI modules. Legacy nodes and V3.4 must share temporal, state, continuation, and layout logic where their contracts overlap.
|
||||||
- V2 must not decode Video/Audio VAE between H3 chunks.
|
- V2 must not decode Video/Audio VAE between H3 chunks.
|
||||||
- Accepted full chunk latents remain on CPU; do not accumulate generated chunks in VRAM.
|
- Accepted full chunk latents remain on CPU; do not accumulate generated chunks in VRAM.
|
||||||
- Spectrum history must remain run-scoped through its public sampler wrappers; never import Spectrum private runtime functions.
|
- Spectrum history must remain run-scoped through its public sampler wrappers; never import Spectrum private runtime functions.
|
||||||
- Changes must preserve V1 node identifiers and State schema unless an explicit migration is included.
|
- Changes must preserve V1 node identifiers and State schema unless an explicit migration is included.
|
||||||
|
|
||||||
|
## Project files
|
||||||
|
|
||||||
|
- `AGENTS.md`: durable rules and prohibitions only.
|
||||||
|
- `PROJECT_STATE.md`: current paths, version, active direction, and known gaps.
|
||||||
|
- `WORKLOG.md`: append-only summaries of completed work and evidence.
|
||||||
|
- Do not duplicate long history across these files. Update `PROJECT_STATE.md` when facts change and append one concise entry to `WORKLOG.md` after completed work.
|
||||||
|
|
||||||
## Validation
|
## Validation
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python -m compileall -q .
|
powershell -ExecutionPolicy Bypass -File tools\validate.ps1
|
||||||
pytest -q
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Runtime installer verification must check native PackedLayout behavior and all registered nodes.
|
Runtime installer verification must check native PackedLayout behavior and all registered nodes.
|
||||||
@@ -38,7 +44,8 @@ GPU checks must cover Sage only, Sage+Sol, Sage+Spectrum, and Sage+Sol+Spectrum
|
|||||||
## Reusable local environment
|
## Reusable local environment
|
||||||
|
|
||||||
- Source repository: `D:\Codex\_git_push_work\ComfyUI-H3-Continuum`
|
- Source repository: `D:\Codex\_git_push_work\ComfyUI-H3-Continuum`
|
||||||
- Active ComfyUI runtime: `D:\StabilityMatrix\Data\Packages\ComfyUI_WAN\custom_nodes\ComfyUI-H3-Continuum`
|
- Primary tested runtime: `D:\StabilityMatrix\Data\Packages\ComfyUI_W\custom_nodes\ComfyUI-H3-Continuum`
|
||||||
|
- Secondary runtime: `D:\StabilityMatrix\Data\Packages\ComfyUI_WAN\custom_nodes\ComfyUI-H3-Continuum`
|
||||||
- Reuse the repository's existing pytest configuration and helper scripts instead of rediscovering paths on every task.
|
- Reuse the repository's existing pytest configuration and helper scripts instead of rediscovering paths on every task.
|
||||||
- Read this file before each task. Record durable project decisions, paths, commands, failure causes, and validation methods here so later work reuses them instead of restarting from zero.
|
- Read this file before each task. Record durable project decisions, paths, commands, failure causes, and validation methods here so later work reuses them instead of restarting from zero.
|
||||||
- GitHub authentication for this repository is already configured. Do not request a new login or token unless an actual authentication command fails.
|
- GitHub authentication for this repository is already configured. Do not request a new login or token unless an actual authentication command fails.
|
||||||
|
|||||||
@@ -0,0 +1,80 @@
|
|||||||
|
# Project State
|
||||||
|
|
||||||
|
Updated: 2026-08-20
|
||||||
|
|
||||||
|
## Baseline
|
||||||
|
|
||||||
|
- Current public baseline: V3.4.0
|
||||||
|
- Authoritative working repository: `D:\Codex\_git_push_work\ComfyUI-H3-Continuum`
|
||||||
|
- Primary tested runtime: `D:\StabilityMatrix\Data\Packages\ComfyUI_W\custom_nodes\ComfyUI-H3-Continuum`
|
||||||
|
- Secondary runtime: `D:\StabilityMatrix\Data\Packages\ComfyUI_WAN\custom_nodes\ComfyUI-H3-Continuum`
|
||||||
|
- Old checkout to keep synchronized with published `main`: `D:\Codex\ComfyUI-H3-Continuum`
|
||||||
|
|
||||||
|
## Active direction
|
||||||
|
|
||||||
|
- Driving Audio preserves the supplied audio as final output while conditioning every chunk.
|
||||||
|
- Video Reference is persistent across chunks and supports selectable reference sizing.
|
||||||
|
- Prompt syntax is permissive: invalid or empty input warns and falls back instead of blocking generation.
|
||||||
|
- Unknown models, wrappers, merged models, and upstream nodes are not rejected merely because they are unknown.
|
||||||
|
- Legacy `strict_compatibility` remains loadable only for workflow compatibility and must not enforce a stop.
|
||||||
|
- Keep the compact V3.4 interface and preserve all public sockets and node identifiers unless a UI/API change is explicitly approved.
|
||||||
|
|
||||||
|
## Current working tree
|
||||||
|
|
||||||
|
The authoritative repository may contain active, uncommitted Hybrid reference work. Preserve it. Never reset, overwrite, or publish it without explicit instruction.
|
||||||
|
|
||||||
|
## Standard procedure
|
||||||
|
|
||||||
|
1. Read `AGENTS.md` and this file.
|
||||||
|
2. Inspect `git status -sb` and confirm the authoritative source/runtime.
|
||||||
|
3. Run `tools\snapshot.ps1` before edits.
|
||||||
|
4. Make the smallest relevant change.
|
||||||
|
5. Run `tools\validate.ps1` only when validation is requested or publication is authorized.
|
||||||
|
6. Append the result and remaining gaps to `WORKLOG.md`.
|
||||||
|
7. Commit or push only when explicitly requested.
|
||||||
|
|
||||||
|
## Validation boundary
|
||||||
|
|
||||||
|
Static checks do not replace a real GPU generation. Record workflow, prompt, media, model/LoRA, sampler, steps, dimensions, chunks, elapsed time, and observed result for GPU tests.
|
||||||
|
|
||||||
|
## Active FLF implementation
|
||||||
|
|
||||||
|
- Hybrid FLF Chunk Contract v1 is implemented locally in `v2/sequence.py`.
|
||||||
|
- It replaces the previous single global sample/split strategy with a 0.4 MP full-sequence guide, intermediate boundary anchors, and separately sampled FL2VA chunks.
|
||||||
|
- Reference Images remain persistent in the guide and every output chunk.
|
||||||
|
- Real GPU generation is still required before this path can be considered stable.
|
||||||
|
|
||||||
|
## FLF GPU regression baseline (2026-08-21)
|
||||||
|
|
||||||
|
- PASS `MiniMax_H3_Continuum_V31_00029_.mp4`: pure FLF skateboard sequence; visual continuity accepted.
|
||||||
|
- PASS `MiniMax_H3_Continuum_V31_00031_.mp4`: Hybrid model plus Reference Image; visual result accepted.
|
||||||
|
- PASS `MiniMax_H3_Continuum_V31_00032_.mp4`: standard Ref2VA model; visual result accepted.
|
||||||
|
- FAIL `MiniMax_H3_Continuum_V31_00034_.mp4`: Hybrid FLF+Reference; visible jumps near 5 s and 9 s.
|
||||||
|
- FAIL `MiniMax_H3_Continuum_V31_00035_.mp4`: Ref2VA model plus FL2V Turbo 8-step LoRA; two visible flashes/jumps.
|
||||||
|
- Current FLF baseline is `context_bound_terminal_v1`: final-chunk context-tail and Last Frame anchors only.
|
||||||
|
- `context_bound_terminal_bridge_v2` is withdrawn because its synthetic intermediate latent correlated with the later terminal flash regression.
|
||||||
|
- Preserve Hybrid/Reference support, Run Storage identity, UI, audio paths, and permissive prompt handling when repairing FLF.
|
||||||
|
|
||||||
|
## FLF GPU regression update: 00036 (2026-08-21)
|
||||||
|
|
||||||
|
- FAIL `MiniMax_H3_Continuum_V31_00036_.mp4`.
|
||||||
|
- Workflow: Hybrid B20-49, FL2V Turbo 8-step LoRA, 8 steps, Euler/simple, Spectrum Off, 2 x 5 s, 640x640, Video Reference Efficient 0.4 MP, randomized seed.
|
||||||
|
- The 5 s chunk boundary was visually stable, but a bright ghost/flash and discontinuous disappearance of the reference fox occurred near 8 s.
|
||||||
|
- Project acceptance requires whole-sequence continuity; a clean chunk seam alone is insufficient. This run is not a partial pass.
|
||||||
|
- Restoring `context_bound_terminal_v1` did not resolve the terminal Hybrid FLF+Reference discontinuity and is not accepted as the final repair.
|
||||||
|
|
||||||
|
## FLF GPU regression update: 00037 (2026-08-21)
|
||||||
|
|
||||||
|
- FAIL `MiniMax_H3_Continuum_V31_00037_.mp4`.
|
||||||
|
- Workflow: official FL2VA pruned INT8 ConvRot model, FL2V Turbo 8-step LoRA, 8 steps, Euler/simple, Spectrum Off, 2 x 5 s, 640x640, one reference image, Video Reference Efficient 0.4 MP, randomized seed.
|
||||||
|
- The 5 s chunk boundary remained visually stable, but the reference fox became translucent and disappeared with a bright ghost/flash near 7.5-8 s before the Last Frame composition.
|
||||||
|
- This reproduces the terminal discontinuity seen in 00036 and is a whole-sequence failure.
|
||||||
|
- Do not roll back merely to terminal bridge v2: 00034 and 00035 already failed there. Restore only an exact baseline associated with the accepted 00029/00031 behavior.
|
||||||
|
|
||||||
|
## FLF GPU regression update: 00038 and rollback (2026-08-21)
|
||||||
|
|
||||||
|
- FAIL `MiniMax_H3_Continuum_V31_00038_.mp4` by user visual evaluation.
|
||||||
|
- Embedded workflow: Ref2VA pruned INT8 ConvRot model, FL2V Turbo 8-step LoRA, Euler/simple, 8 steps, Spectrum Off, 2 x 5 s, randomized seed, Video Reference Efficient 0.4 MP.
|
||||||
|
- The initial rollback after 00038 temporarily returned to bridge v2, but the user clarified that the required target is the exact code state used for PASS 00031.
|
||||||
|
- `00031` completed at 2026-08-21 02:32:59; `pre-flf-terminal-bridge-v2-repo-20260821_023553` was captured immediately afterward and before bridge v2 was introduced.
|
||||||
|
- Current FLF strategy is restored exactly from that snapshot: `context_bound_terminal_v1`. Bridge v2 is withdrawn and is not active.
|
||||||
+62
@@ -0,0 +1,62 @@
|
|||||||
|
# Work Log
|
||||||
|
|
||||||
|
Append concise completed-work records here. Do not paste full console logs or duplicate `PROJECT_STATE.md`.
|
||||||
|
|
||||||
|
## 2026-08-20 - Project guidance organization
|
||||||
|
|
||||||
|
- Established the three-file structure: `AGENTS.md`, `PROJECT_STATE.md`, and `WORKLOG.md`.
|
||||||
|
- Added reusable snapshot and validation scripts under `tools`.
|
||||||
|
- Kept the existing uncommitted Hybrid reference work untouched.
|
||||||
|
- No tests were run for this documentation and tooling-only change.
|
||||||
|
|
||||||
|
## 2026-08-21 - Hybrid FLF Chunk Contract v1
|
||||||
|
|
||||||
|
- Replaced the single global FLF sample/split path with a 0.4 MP guide and separately sampled FL2VA chunks.
|
||||||
|
- Kept Reference Images persistent in the guide and every output chunk.
|
||||||
|
- Preserved the existing V3.4 UI and non-FLF execution paths.
|
||||||
|
- GPU behavior remains unverified.
|
||||||
|
|
||||||
|
## 2026-08-21 - Restore successful FLF terminal baseline
|
||||||
|
|
||||||
|
- Recorded GPU visual baselines: PASS 00029/00031/00032; FAIL 00034/00035.
|
||||||
|
- Reverted only the FLF terminal strategy from `context_bound_terminal_bridge_v2` to `context_bound_terminal_v1`.
|
||||||
|
- Removed the synthetic interpolated terminal keyframe; retained the continuation-context boundary anchor and supplied Last Frame anchor.
|
||||||
|
- No UI, Hybrid presentation, Reference, Run Storage, Driving Audio, Video Reference, or prompt-policy changes.
|
||||||
|
- Synced the modified `v2/sequence.py` to the active `ComfyUI_W` runtime.
|
||||||
|
- Runtime GPU retest intentionally pending user execution.
|
||||||
|
|
||||||
|
## 2026-08-21 - GPU result 00036 classified FAIL
|
||||||
|
|
||||||
|
- User confirmed that 00036 is a full failure, not a partial pass.
|
||||||
|
- The 5 s boundary was stable, but the sequence failed near 8 s with a bright ghost/flash and discontinuous loss of the reference fox.
|
||||||
|
- Recorded that whole-sequence continuity is the acceptance criterion.
|
||||||
|
- Recorded that the `context_bound_terminal_v1` restoration remains unresolved for terminal Hybrid FLF+Reference behavior.
|
||||||
|
- No code or runtime file was changed for this classification update.
|
||||||
|
|
||||||
|
## 2026-08-21 - GPU result 00037 classified FAIL
|
||||||
|
|
||||||
|
- Confirmed a second full failure after 00036.
|
||||||
|
- Embedded workflow: official FL2VA pruned INT8 ConvRot model, FL2V Turbo 8-step LoRA, 8 steps, Euler/simple, Spectrum Off, 2 x 5 s, 640x640, one reference image, Video Reference Efficient 0.4 MP.
|
||||||
|
- The fox remained stable across the 5 s seam, then became translucent and disappeared with a bright ghost/flash near 7.5-8 s.
|
||||||
|
- Rollback decision: do not use the immediately previous bridge-v2 implementation because its 00034/00035 results failed; identify and restore the exact accepted 00029/00031 baseline.
|
||||||
|
- No code or runtime file was changed in this classification step.
|
||||||
|
|
||||||
|
## 2026-08-21 - 00038 FAIL; withdrew terminal-v1 restoration
|
||||||
|
|
||||||
|
- User classified 00038 as FAIL and requested rollback.
|
||||||
|
- Confirmed embedded workflow used Ref2VA pruned INT8 ConvRot, FL2V Turbo 8-step LoRA, Euler/simple 8 steps, Spectrum Off, 2 x 5 s, randomized seed, and Video Reference Efficient 0.4 MP.
|
||||||
|
- Verified repository and ComfyUI_W `v2/sequence.py` matched before modification.
|
||||||
|
- Snapshotted repository/runtime sequence files plus project records to:
|
||||||
|
D:\Codex\_snapshots\ComfyUI-H3-Continuum\pre-rollback-after-00038-fail-20260821_032550
|
||||||
|
- Restored only `v2/sequence.py` from the exact pre-restoration snapshot `pre-restore-flf-success-baseline-repo-20260821_030316`.
|
||||||
|
- Synced the restored file to ComfyUI_W. No UI or unrelated feature files were changed.
|
||||||
|
- The restored bridge-v2 implementation remains unapproved; this action only reverses the failed terminal-v1 restoration.
|
||||||
|
|
||||||
|
|
||||||
|
## 2026-08-21 - Corrected rollback target to exact 00031 state
|
||||||
|
|
||||||
|
- User clarified that the rollback target is the code state that generated PASS `MiniMax_H3_Continuum_V31_00031_.mp4`.
|
||||||
|
- Correlated filesystem times: 00031 completed at 02:32:59; the pre-bridge snapshot was captured at 02:35:53.
|
||||||
|
- Restored only `v2/sequence.py` from `pre-flf-terminal-bridge-v2-repo-20260821_023553` to the repository and active ComfyUI_W runtime.
|
||||||
|
- Active FLF strategy is `context_bound_terminal_v1`; bridge v2 is no longer active.
|
||||||
|
- No UI or unrelated feature files were changed. GPU retest was not run.
|
||||||
+62
-2
@@ -18,6 +18,7 @@ import torch
|
|||||||
|
|
||||||
REFERENCE_CONTRACT_VERSION = 1
|
REFERENCE_CONTRACT_VERSION = 1
|
||||||
REFERENCE_PREPROCESS_VERSION = 1
|
REFERENCE_PREPROCESS_VERSION = 1
|
||||||
|
HYBRID_PRESENTATION_VERSION = 1
|
||||||
REFERENCE_SIZE_MATCH_OUTPUT = "Match Output"
|
REFERENCE_SIZE_MATCH_OUTPUT = "Match Output"
|
||||||
REFERENCE_SIZE_MAX_IDENTITY = "Max Identity"
|
REFERENCE_SIZE_MAX_IDENTITY = "Max Identity"
|
||||||
REFERENCE_SIZE_OPTIONS = (
|
REFERENCE_SIZE_OPTIONS = (
|
||||||
@@ -192,7 +193,59 @@ def encode_reference_latents(video_vae: Any, assets: ReferenceAssets) -> Referen
|
|||||||
return replace(assets, latents=latents)
|
return replace(assets, latents=latents)
|
||||||
|
|
||||||
|
|
||||||
def validate_reference_prompts(prompts: list[str], reference_count: int) -> str:
|
def build_hybrid_presentation_items(
|
||||||
|
reference_items: list[dict[str, Any]],
|
||||||
|
*,
|
||||||
|
first_image: torch.Tensor | None = None,
|
||||||
|
last_image: torch.Tensor | None = None,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
"""Build the Core Qwen Picture list without changing DiT payloads."""
|
||||||
|
items: list[dict[str, Any]] = []
|
||||||
|
if first_image is not None:
|
||||||
|
items.append({"type": "image", "data": first_image})
|
||||||
|
if last_image is not None:
|
||||||
|
items.append({"type": "image", "data": last_image})
|
||||||
|
items.extend(dict(item) for item in reference_items)
|
||||||
|
return items
|
||||||
|
|
||||||
|
|
||||||
|
def combine_hybrid_visual_identity(
|
||||||
|
*,
|
||||||
|
keyframe_identity_hash: str,
|
||||||
|
reference_assets: ReferenceAssets | None,
|
||||||
|
has_first: bool,
|
||||||
|
has_last: bool,
|
||||||
|
) -> str:
|
||||||
|
"""Preserve pure-mode identity and version only the hybrid contract."""
|
||||||
|
if reference_assets is None:
|
||||||
|
return str(keyframe_identity_hash)
|
||||||
|
if not has_first and not has_last:
|
||||||
|
return reference_assets.combined_hash
|
||||||
|
return _canonical_hash(
|
||||||
|
{
|
||||||
|
"hybrid_visual_identity_version": 1,
|
||||||
|
"hybrid_presentation_version": HYBRID_PRESENTATION_VERSION,
|
||||||
|
"keyframe_identity_hash": str(keyframe_identity_hash),
|
||||||
|
"reference_hash": reference_assets.combined_hash,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def validate_reference_prompts(
|
||||||
|
prompts: list[str],
|
||||||
|
reference_count: int,
|
||||||
|
picture_offset: int = 0,
|
||||||
|
) -> str:
|
||||||
|
if picture_offset > 0:
|
||||||
|
def _shift_picture_tag(match: re.Match[str]) -> str:
|
||||||
|
number = int(match.group(1))
|
||||||
|
if number <= picture_offset:
|
||||||
|
return ""
|
||||||
|
return f"<Picture {number - picture_offset}>"
|
||||||
|
|
||||||
|
shifted = [_PICTURE_TAG.sub(_shift_picture_tag, str(prompt)) for prompt in prompts]
|
||||||
|
return validate_reference_prompts(shifted, reference_count)
|
||||||
|
|
||||||
found: set[int] = set()
|
found: set[int] = set()
|
||||||
for prompt in prompts:
|
for prompt in prompts:
|
||||||
found.update(int(value) for value in _PICTURE_TAG.findall(str(prompt)))
|
found.update(int(value) for value in _PICTURE_TAG.findall(str(prompt)))
|
||||||
@@ -226,6 +279,8 @@ def encode_reference_prompt(
|
|||||||
clip: Any,
|
clip: Any,
|
||||||
prompt: str,
|
prompt: str,
|
||||||
assets: ReferenceAssets,
|
assets: ReferenceAssets,
|
||||||
|
first_image: torch.Tensor | None = None,
|
||||||
|
last_image: torch.Tensor | None = None,
|
||||||
reference_audio_assets=None,
|
reference_audio_assets=None,
|
||||||
timeline_video_assets=None,
|
timeline_video_assets=None,
|
||||||
) -> list[list[Any]]:
|
) -> list[list[Any]]:
|
||||||
@@ -241,8 +296,13 @@ def encode_reference_prompt(
|
|||||||
from .reference_audio import reference_audio_item
|
from .reference_audio import reference_audio_item
|
||||||
|
|
||||||
reference_items.append(reference_audio_item())
|
reference_items.append(reference_audio_item())
|
||||||
|
presentation_items = build_hybrid_presentation_items(
|
||||||
|
reference_items,
|
||||||
|
first_image=first_image,
|
||||||
|
last_image=last_image,
|
||||||
|
)
|
||||||
try:
|
try:
|
||||||
tokens = clip.tokenize(str(prompt), minimax_ref_items=reference_items)
|
tokens = clip.tokenize(str(prompt), minimax_ref_items=presentation_items)
|
||||||
except TypeError as exc:
|
except TypeError as exc:
|
||||||
raise ReferenceConditioningError(
|
raise ReferenceConditioningError(
|
||||||
"This ComfyUI Core/CLIP does not support MiniMax H3 image references"
|
"This ComfyUI Core/CLIP does not support MiniMax H3 image references"
|
||||||
|
|||||||
+21
-3
@@ -574,9 +574,11 @@ def build_sampling_contract(
|
|||||||
) -> tuple[dict[str, Any], bool, list[str]]:
|
) -> tuple[dict[str, Any], bool, list[str]]:
|
||||||
model_value, model_safe = _model_signature(model, model_fingerprint_value)
|
model_value, model_safe = _model_signature(model, model_fingerprint_value)
|
||||||
clip_value, clip_safe = _clip_signature(clip)
|
clip_value, clip_safe = _clip_signature(clip)
|
||||||
|
has_first = str(first_frame_hash).lower() not in {"", "none", "null"}
|
||||||
|
has_last = str(last_frame_hash).lower() not in {"", "none", "null"}
|
||||||
inferred_mode = conditioning_mode_from_presence(
|
inferred_mode = conditioning_mode_from_presence(
|
||||||
has_first=str(first_frame_hash).lower() not in {"", "none", "null"},
|
has_first=has_first,
|
||||||
has_last=str(last_frame_hash).lower() not in {"", "none", "null"},
|
has_last=has_last,
|
||||||
has_reference=reference_contract is not None,
|
has_reference=reference_contract is not None,
|
||||||
)
|
)
|
||||||
conditioning_mode = inferred_mode if conditioning_mode is None else str(conditioning_mode)
|
conditioning_mode = inferred_mode if conditioning_mode is None else str(conditioning_mode)
|
||||||
@@ -660,10 +662,26 @@ def build_sampling_contract(
|
|||||||
global_contract["upstream_graph"] = dict(upstream_graph_contract)
|
global_contract["upstream_graph"] = dict(upstream_graph_contract)
|
||||||
if uses_video_vae:
|
if uses_video_vae:
|
||||||
global_contract["video_vae"] = video_vae_value
|
global_contract["video_vae"] = video_vae_value
|
||||||
if conditioning_mode in {"i2va", "fl2va"}:
|
if has_first:
|
||||||
global_contract["first_frame_hash"] = str(first_frame_hash)
|
global_contract["first_frame_hash"] = str(first_frame_hash)
|
||||||
if reference_contract is not None:
|
if reference_contract is not None:
|
||||||
global_contract["reference"] = dict(reference_contract)
|
global_contract["reference"] = dict(reference_contract)
|
||||||
|
if has_first or has_last:
|
||||||
|
from .reference import HYBRID_PRESENTATION_VERSION
|
||||||
|
|
||||||
|
picture_order: list[str] = []
|
||||||
|
if has_first:
|
||||||
|
picture_order.append("first_frame")
|
||||||
|
if has_last:
|
||||||
|
picture_order.append("last_frame")
|
||||||
|
picture_order.extend(
|
||||||
|
f"reference_image_{index}"
|
||||||
|
for index in range(1, int(reference_contract.get("count", 0)) + 1)
|
||||||
|
)
|
||||||
|
global_contract["hybrid_qwen_presentation"] = {
|
||||||
|
"version": HYBRID_PRESENTATION_VERSION,
|
||||||
|
"picture_order": picture_order,
|
||||||
|
}
|
||||||
if reference_audio_contract is not None:
|
if reference_audio_contract is not None:
|
||||||
global_contract["reference_audio"] = dict(reference_audio_contract)
|
global_contract["reference_audio"] = dict(reference_audio_contract)
|
||||||
global_contract["reference_audio_vae"] = audio_vae_value
|
global_contract["reference_audio_vae"] = audio_vae_value
|
||||||
|
|||||||
@@ -0,0 +1,121 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
import torch
|
||||||
|
|
||||||
|
from ComfyUI_H3_Continuum_Join.reference import (
|
||||||
|
build_hybrid_presentation_items,
|
||||||
|
combine_hybrid_visual_identity,
|
||||||
|
validate_reference_prompts,
|
||||||
|
)
|
||||||
|
from ComfyUI_H3_Continuum_Join.v2.sequence import _conditioning_cache_key
|
||||||
|
|
||||||
|
|
||||||
|
def _image(value: float) -> torch.Tensor:
|
||||||
|
return torch.full((1, 2, 2, 3), value, dtype=torch.float32)
|
||||||
|
|
||||||
|
|
||||||
|
def test_hybrid_picture_order_first_last_then_references() -> None:
|
||||||
|
first = _image(1.0)
|
||||||
|
last = _image(2.0)
|
||||||
|
ref1 = _image(3.0)
|
||||||
|
ref2 = _image(4.0)
|
||||||
|
items = build_hybrid_presentation_items(
|
||||||
|
[{"type": "image", "data": ref1}, {"type": "image", "data": ref2}],
|
||||||
|
first_image=first,
|
||||||
|
last_image=last,
|
||||||
|
)
|
||||||
|
assert [id(item["data"]) for item in items] == [id(first), id(last), id(ref1), id(ref2)]
|
||||||
|
|
||||||
|
|
||||||
|
def test_hybrid_picture_order_supports_first_only() -> None:
|
||||||
|
first = _image(1.0)
|
||||||
|
ref = _image(2.0)
|
||||||
|
items = build_hybrid_presentation_items(
|
||||||
|
[{"type": "image", "data": ref}],
|
||||||
|
first_image=first,
|
||||||
|
)
|
||||||
|
assert [id(item["data"]) for item in items] == [id(first), id(ref)]
|
||||||
|
|
||||||
|
|
||||||
|
def test_hybrid_picture_order_supports_last_only() -> None:
|
||||||
|
last = _image(1.0)
|
||||||
|
ref = _image(2.0)
|
||||||
|
items = build_hybrid_presentation_items(
|
||||||
|
[{"type": "image", "data": ref}],
|
||||||
|
last_image=last,
|
||||||
|
)
|
||||||
|
assert [id(item["data"]) for item in items] == [id(last), id(ref)]
|
||||||
|
|
||||||
|
|
||||||
|
def test_hybrid_prompt_validation_offsets_reference_numbers() -> None:
|
||||||
|
assert validate_reference_prompts(
|
||||||
|
["<Picture 1> starts the shot and <Picture 3> defines identity."],
|
||||||
|
1,
|
||||||
|
picture_offset=2,
|
||||||
|
) == ""
|
||||||
|
|
||||||
|
|
||||||
|
def test_pure_fl_identity_is_unchanged() -> None:
|
||||||
|
assert combine_hybrid_visual_identity(
|
||||||
|
keyframe_identity_hash="fl-identity",
|
||||||
|
reference_assets=None,
|
||||||
|
has_first=True,
|
||||||
|
has_last=True,
|
||||||
|
) == "fl-identity"
|
||||||
|
|
||||||
|
|
||||||
|
def test_pure_reference_identity_is_unchanged() -> None:
|
||||||
|
references = SimpleNamespace(combined_hash="reference-identity")
|
||||||
|
assert combine_hybrid_visual_identity(
|
||||||
|
keyframe_identity_hash="unused",
|
||||||
|
reference_assets=references,
|
||||||
|
has_first=False,
|
||||||
|
has_last=False,
|
||||||
|
) == "reference-identity"
|
||||||
|
|
||||||
|
|
||||||
|
def test_hybrid_identity_uses_keyframes_and_references() -> None:
|
||||||
|
references = SimpleNamespace(combined_hash="reference-identity")
|
||||||
|
first = combine_hybrid_visual_identity(
|
||||||
|
keyframe_identity_hash="keyframe-a",
|
||||||
|
reference_assets=references,
|
||||||
|
has_first=True,
|
||||||
|
has_last=False,
|
||||||
|
)
|
||||||
|
second = combine_hybrid_visual_identity(
|
||||||
|
keyframe_identity_hash="keyframe-b",
|
||||||
|
reference_assets=references,
|
||||||
|
has_first=True,
|
||||||
|
has_last=False,
|
||||||
|
)
|
||||||
|
assert first != "keyframe-a"
|
||||||
|
assert first != "reference-identity"
|
||||||
|
assert first != second
|
||||||
|
|
||||||
|
|
||||||
|
def test_hybrid_conditioning_cache_key_reuses_encoded_reference_prompt() -> None:
|
||||||
|
references = SimpleNamespace(count=1)
|
||||||
|
assert _conditioning_cache_key(
|
||||||
|
"final prompt",
|
||||||
|
include_last=True,
|
||||||
|
reference_assets=references,
|
||||||
|
) == ("final prompt", False)
|
||||||
|
|
||||||
|
|
||||||
|
def test_pure_fl_conditioning_cache_key_keeps_final_frame_variant() -> None:
|
||||||
|
assert _conditioning_cache_key(
|
||||||
|
"final prompt",
|
||||||
|
include_last=True,
|
||||||
|
reference_assets=None,
|
||||||
|
) == ("final prompt", True)
|
||||||
|
|
||||||
|
|
||||||
|
def test_pure_reference_conditioning_cache_key_stays_without_last_frame() -> None:
|
||||||
|
references = SimpleNamespace(count=1)
|
||||||
|
assert _conditioning_cache_key(
|
||||||
|
"reference prompt",
|
||||||
|
include_last=False,
|
||||||
|
reference_assets=references,
|
||||||
|
) == ("reference prompt", False)
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
param(
|
||||||
|
[string]$Source = (Split-Path -Parent $PSScriptRoot),
|
||||||
|
[string]$SnapshotRoot = 'D:\Codex\_snapshots\ComfyUI-H3-Continuum',
|
||||||
|
[string]$Label = 'pre-change'
|
||||||
|
)
|
||||||
|
|
||||||
|
$ErrorActionPreference = 'Stop'
|
||||||
|
$sourcePath = (Resolve-Path -LiteralPath $Source).Path
|
||||||
|
$stamp = Get-Date -Format 'yyyyMMdd_HHmmss'
|
||||||
|
$safeLabel = $Label -replace '[^A-Za-z0-9._-]', '-'
|
||||||
|
$destination = Join-Path $SnapshotRoot "$safeLabel-$stamp"
|
||||||
|
$rootFull = [IO.Path]::GetFullPath($SnapshotRoot)
|
||||||
|
$destinationFull = [IO.Path]::GetFullPath($destination)
|
||||||
|
|
||||||
|
if (-not $destinationFull.StartsWith($rootFull, [StringComparison]::OrdinalIgnoreCase)) {
|
||||||
|
throw 'Snapshot destination escaped the configured snapshot root.'
|
||||||
|
}
|
||||||
|
|
||||||
|
New-Item -ItemType Directory -Path $destination -Force | Out-Null
|
||||||
|
Get-ChildItem -LiteralPath $sourcePath -Force |
|
||||||
|
Where-Object { $_.Name -ne '.git' } |
|
||||||
|
ForEach-Object { Copy-Item -LiteralPath $_.FullName -Destination $destination -Recurse -Force }
|
||||||
|
|
||||||
|
$sourceCount = (Get-ChildItem -LiteralPath $sourcePath -Recurse -Force -File |
|
||||||
|
Where-Object { $_.FullName -notmatch '[\\/]\.git[\\/]' } |
|
||||||
|
Measure-Object).Count
|
||||||
|
$destinationCount = (Get-ChildItem -LiteralPath $destination -Recurse -Force -File | Measure-Object).Count
|
||||||
|
|
||||||
|
if ($sourceCount -ne $destinationCount) {
|
||||||
|
throw "Snapshot verification failed: source=$sourceCount destination=$destinationCount"
|
||||||
|
}
|
||||||
|
|
||||||
|
[pscustomobject]@{
|
||||||
|
Source = $sourcePath
|
||||||
|
Snapshot = $destination
|
||||||
|
Files = $destinationCount
|
||||||
|
}
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
param(
|
||||||
|
[string]$Python = 'python',
|
||||||
|
[string]$ComfyRoot = '',
|
||||||
|
[switch]$SkipTests,
|
||||||
|
[switch]$SkipRuntime
|
||||||
|
)
|
||||||
|
|
||||||
|
$ErrorActionPreference = 'Stop'
|
||||||
|
$repository = Split-Path -Parent $PSScriptRoot
|
||||||
|
Push-Location $repository
|
||||||
|
try {
|
||||||
|
& $Python -m compileall -q .
|
||||||
|
if ($LASTEXITCODE -ne 0) { throw 'compileall failed' }
|
||||||
|
|
||||||
|
if (-not $SkipTests) {
|
||||||
|
& $Python -m pytest -q
|
||||||
|
if ($LASTEXITCODE -ne 0) { throw 'pytest failed' }
|
||||||
|
}
|
||||||
|
|
||||||
|
if (-not $SkipRuntime) {
|
||||||
|
$arguments = @('tools/verify_runtime.py')
|
||||||
|
if ($ComfyRoot) { $arguments += @('--comfy-root', $ComfyRoot) }
|
||||||
|
& $Python @arguments
|
||||||
|
if ($LASTEXITCODE -ne 0) { throw 'runtime verification failed' }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
finally {
|
||||||
|
Pop-Location
|
||||||
|
}
|
||||||
+70
-7
@@ -6,7 +6,7 @@ runs for every chunk without intermediate VAE decoding, and decoding/assembly
|
|||||||
happens after the sampling phase.
|
happens after the sampling phase.
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
import copy, logging
|
import copy, hashlib, logging
|
||||||
from typing import Any
|
from typing import Any
|
||||||
import torch
|
import torch
|
||||||
from ..compatibility import accelerator_summary, check_comfy_h3_runtime
|
from ..compatibility import accelerator_summary, check_comfy_h3_runtime
|
||||||
@@ -86,18 +86,60 @@ def _preserved_prefix(*,session,prompt_hashes,chunks,reroll_from_chunk,width,hei
|
|||||||
elif reroll_from_chunk>0: notes.append(f"preserved chunks 1-{len(preserved)}; regenerated from chunk {reroll_from_chunk}")
|
elif reroll_from_chunk>0: notes.append(f"preserved chunks 1-{len(preserved)}; regenerated from chunk {reroll_from_chunk}")
|
||||||
return preserved,notes
|
return preserved,notes
|
||||||
|
|
||||||
|
def _conditioning_cache_key(prompt, *, include_last, reference_assets=None):
|
||||||
|
return (prompt, False if reference_assets is not None else bool(include_last))
|
||||||
|
|
||||||
|
|
||||||
def _conditioning_cache(*,clip,prompts,assets,final_has_last_frame,reference_assets=None,reference_audio_assets=None,timeline_video_assets=None):
|
def _conditioning_cache(*,clip,prompts,assets,final_has_last_frame,reference_assets=None,reference_audio_assets=None,timeline_video_assets=None):
|
||||||
cache={}; final_index=len(prompts)-1
|
cache={}; final_index=len(prompts)-1
|
||||||
for index,prompt in enumerate(prompts):
|
for index,prompt in enumerate(prompts):
|
||||||
include_last=bool(final_has_last_frame and index==final_index); key=(prompt,include_last)
|
include_last=bool(final_has_last_frame and index==final_index); key=_conditioning_cache_key(prompt,include_last=include_last,reference_assets=reference_assets)
|
||||||
if key in cache: continue
|
if key in cache: continue
|
||||||
if reference_assets is not None:
|
if reference_assets is not None:
|
||||||
from ..reference import encode_reference_prompt
|
from ..reference import encode_reference_prompt
|
||||||
cache[key]=encode_reference_prompt(clip,prompt,reference_assets,reference_audio_assets=reference_audio_assets,timeline_video_assets=timeline_video_assets)
|
cache[key]=encode_reference_prompt(clip,prompt,reference_assets,first_image=assets.first_image,last_image=assets.last_image,reference_audio_assets=reference_audio_assets,timeline_video_assets=timeline_video_assets)
|
||||||
else:
|
else:
|
||||||
cache[key]=encode_prompt_conditioning(clip,prompt,first_image=assets.first_image,last_image=assets.last_image if include_last else None,reference_audio_assets=reference_audio_assets,timeline_video_assets=timeline_video_assets)
|
cache[key]=encode_prompt_conditioning(clip,prompt,first_image=assets.first_image,last_image=assets.last_image if include_last else None,reference_audio_assets=reference_audio_assets,timeline_video_assets=timeline_video_assets)
|
||||||
return cache
|
return cache
|
||||||
|
|
||||||
|
|
||||||
|
FLF_STRATEGY = "context_bound_terminal_v1"
|
||||||
|
|
||||||
|
|
||||||
|
def _flf_contract_identity(base_hash: str) -> str:
|
||||||
|
payload = f"{base_hash}|flf_strategy={FLF_STRATEGY}"
|
||||||
|
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _attach_terminal_flf_keyframes(
|
||||||
|
conditioning,
|
||||||
|
*,
|
||||||
|
video_context: torch.Tensor,
|
||||||
|
context_frames: int,
|
||||||
|
frame_count: int,
|
||||||
|
last_latent: torch.Tensor,
|
||||||
|
):
|
||||||
|
final_index = int(frame_count) - 1
|
||||||
|
boundary_index = max(0, min(final_index - 1, int(context_frames) - 1))
|
||||||
|
boundary_latent = video_context[:, :, -1:, ...].contiguous()
|
||||||
|
excluded = {boundary_index, final_index}
|
||||||
|
for _, metadata in conditioning:
|
||||||
|
keyframes = [
|
||||||
|
dict(item)
|
||||||
|
for item in (metadata.get("minimax_keyframes") or [])
|
||||||
|
if int(item.get("resolved_frame_index", -1)) not in excluded
|
||||||
|
]
|
||||||
|
keyframes.extend(
|
||||||
|
[
|
||||||
|
{"resolved_frame_index": boundary_index, "latent": boundary_latent},
|
||||||
|
{"resolved_frame_index": final_index, "latent": last_latent},
|
||||||
|
]
|
||||||
|
)
|
||||||
|
keyframes.sort(key=lambda item: int(item.get("resolved_frame_index", 0)))
|
||||||
|
metadata["minimax_keyframes"] = keyframes
|
||||||
|
metadata["minimax_frame_count"] = int(frame_count)
|
||||||
|
return conditioning
|
||||||
|
|
||||||
def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,sigmas:torch.Tensor,first_frame:torch.Tensor|None,last_frame:torch.Tensor|None,prompt_plan:dict[str,Any],width:int,height:int,continuity:str,base_seed:int,audio_continuity:bool,exact_total_duration:bool,diagnostics_mode:str,reroll_from_chunk:int,reroll_nonce:int,strict_compatibility:bool,debug:bool,seam_correction:str=SEAM_CORRECTION_OFF,enable_preview:bool=True,session:dict[str,Any]|None=None,initial_state:dict[str,Any]|None=None,latent_only:bool=False,reference_assets=None,reference_audio_source=None,reference_audio_vae=None,driving_audio_source=None,driving_audio_vae=None,reference_video_source=None,timeline_video_source=None):
|
def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,sigmas:torch.Tensor,first_frame:torch.Tensor|None,last_frame:torch.Tensor|None,prompt_plan:dict[str,Any],width:int,height:int,continuity:str,base_seed:int,audio_continuity:bool,exact_total_duration:bool,diagnostics_mode:str,reroll_from_chunk:int,reroll_nonce:int,strict_compatibility:bool,debug:bool,seam_correction:str=SEAM_CORRECTION_OFF,enable_preview:bool=True,session:dict[str,Any]|None=None,initial_state:dict[str,Any]|None=None,latent_only:bool=False,reference_assets=None,reference_audio_source=None,reference_audio_vae=None,driving_audio_source=None,driving_audio_vae=None,reference_video_source=None,timeline_video_source=None):
|
||||||
from ..conditioning import detect_conditioning_mode, conditioning_mode_label
|
from ..conditioning import detect_conditioning_mode, conditioning_mode_label
|
||||||
from ..run_storage import get_active_run_storage
|
from ..run_storage import get_active_run_storage
|
||||||
@@ -118,7 +160,7 @@ def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,si
|
|||||||
reference_warning=""
|
reference_warning=""
|
||||||
if reference_assets is not None:
|
if reference_assets is not None:
|
||||||
from ..reference import validate_reference_prompts
|
from ..reference import validate_reference_prompts
|
||||||
reference_warning=validate_reference_prompts(prompts,reference_assets.count)
|
reference_warning=validate_reference_prompts(prompts,reference_assets.count,picture_offset=int(first_frame is not None)+int(last_frame is not None))
|
||||||
from ..reference_audio import (
|
from ..reference_audio import (
|
||||||
combine_reference_audio_identity,
|
combine_reference_audio_identity,
|
||||||
validate_reference_audio_prompts,
|
validate_reference_audio_prompts,
|
||||||
@@ -138,11 +180,15 @@ def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,si
|
|||||||
if not latent_only: decode_estimate_gib,available_ram_gib=_check_decode_memory_budget(width=width,height=height,chunks=chunks,chunk_seconds=chunk_seconds)
|
if not latent_only: decode_estimate_gib,available_ram_gib=_check_decode_memory_budget(width=width,height=height,chunks=chunks,chunk_seconds=chunk_seconds)
|
||||||
issues=check_comfy_h3_runtime()
|
issues=check_comfy_h3_runtime()
|
||||||
assets=prepare_identity_assets(video_vae,width=width,height=height,first_frame=first_frame,last_frame=last_frame,encode_latents=False)
|
assets=prepare_identity_assets(video_vae,width=width,height=height,first_frame=first_frame,last_frame=last_frame,encode_latents=False)
|
||||||
visual_identity_hash=reference_assets.combined_hash if reference_assets is not None else assets.identity_hash
|
multi_chunk_flf=bool(chunks>=2 and first_frame is not None and last_frame is not None)
|
||||||
|
from ..reference import combine_hybrid_visual_identity
|
||||||
|
visual_identity_hash=combine_hybrid_visual_identity(keyframe_identity_hash=assets.identity_hash,reference_assets=reference_assets,has_first=first_frame is not None,has_last=last_frame is not None)
|
||||||
sequence_identity_hash=combine_reference_audio_identity(visual_identity_hash,reference_audio_source)
|
sequence_identity_hash=combine_reference_audio_identity(visual_identity_hash,reference_audio_source)
|
||||||
sequence_identity_hash=combine_driving_audio_identity(sequence_identity_hash,driving_audio_source)
|
sequence_identity_hash=combine_driving_audio_identity(sequence_identity_hash,driving_audio_source)
|
||||||
sequence_identity_hash=combine_reference_video_identity(sequence_identity_hash,reference_video_source)
|
sequence_identity_hash=combine_reference_video_identity(sequence_identity_hash,reference_video_source)
|
||||||
sequence_identity_hash=combine_timeline_video_identity(sequence_identity_hash,timeline_video_source)
|
sequence_identity_hash=combine_timeline_video_identity(sequence_identity_hash,timeline_video_source)
|
||||||
|
if multi_chunk_flf:
|
||||||
|
sequence_identity_hash=_flf_contract_identity(sequence_identity_hash)
|
||||||
current_model_fingerprint=model_fingerprint(model,extra_wrapper_keys=("h3_continuum_join.apply_model.v1",))
|
current_model_fingerprint=model_fingerprint(model,extra_wrapper_keys=("h3_continuum_join.apply_model.v1",))
|
||||||
if storage_controller is not None:
|
if storage_controller is not None:
|
||||||
stored_session=storage_controller.prepare(model=model,model_fingerprint_value=current_model_fingerprint,clip=clip,video_vae=video_vae,sampler=sampler,sigmas=sigmas,prompt_plan=plan,width=width,height=height,chunk_seconds=chunk_seconds,continuity=continuity,audio_continuity=audio_continuity,base_seed=base_seed,reroll_from_chunk=reroll_from_chunk,reroll_nonce=reroll_nonce,first_frame_hash=assets.first_frame_hash,last_frame_hash=assets.last_frame_hash,identity_hash=sequence_identity_hash,strict_compatibility=strict_compatibility,existing_session=session,reference_contract=reference_assets.contract if reference_assets is not None else None,conditioning_mode=conditioning_mode,reference_audio_contract=reference_audio_source.contract if reference_audio_source is not None else None,reference_audio_vae=reference_audio_vae,driving_audio_contract=driving_audio_source.contract if driving_audio_source is not None else None,driving_audio_vae=driving_audio_vae,reference_video_contract=reference_video_source.contract if reference_video_source is not None else None,timeline_video_contract=timeline_video_source.contract if timeline_video_source is not None else None)
|
stored_session=storage_controller.prepare(model=model,model_fingerprint_value=current_model_fingerprint,clip=clip,video_vae=video_vae,sampler=sampler,sigmas=sigmas,prompt_plan=plan,width=width,height=height,chunk_seconds=chunk_seconds,continuity=continuity,audio_continuity=audio_continuity,base_seed=base_seed,reroll_from_chunk=reroll_from_chunk,reroll_nonce=reroll_nonce,first_frame_hash=assets.first_frame_hash,last_frame_hash=assets.last_frame_hash,identity_hash=sequence_identity_hash,strict_compatibility=strict_compatibility,existing_session=session,reference_contract=reference_assets.contract if reference_assets is not None else None,conditioning_mode=conditioning_mode,reference_audio_contract=reference_audio_source.contract if reference_audio_source is not None else None,reference_audio_vae=reference_audio_vae,driving_audio_contract=driving_audio_source.contract if driving_audio_source is not None else None,driving_audio_vae=driving_audio_vae,reference_video_contract=reference_video_source.contract if reference_video_source is not None else None,timeline_video_contract=timeline_video_source.contract if timeline_video_source is not None else None)
|
||||||
@@ -152,6 +198,13 @@ def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,si
|
|||||||
if "Continuum APPLY_MODEL wrapper installed" not in accelerators: accelerators+="; Continuum APPLY_MODEL wrapper installed"
|
if "Continuum APPLY_MODEL wrapper installed" not in accelerators: accelerators+="; Continuum APPLY_MODEL wrapper installed"
|
||||||
effective_reroll_from_chunk=0 if storage_controller is not None and session is not None and bool((session.get("settings") or {}).get("run_storage_validated_prefix")) else int(reroll_from_chunk)
|
effective_reroll_from_chunk=0 if storage_controller is not None and session is not None and bool((session.get("settings") or {}).get("run_storage_validated_prefix")) else int(reroll_from_chunk)
|
||||||
preserved,reuse_notes=_preserved_prefix(session=session,prompt_hashes=prompt_hashes,chunks=chunks,reroll_from_chunk=effective_reroll_from_chunk,width=width,height=height,chunk_seconds=chunk_seconds,identity_hash=sequence_identity_hash)
|
preserved,reuse_notes=_preserved_prefix(session=session,prompt_hashes=prompt_hashes,chunks=chunks,reroll_from_chunk=effective_reroll_from_chunk,width=width,height=height,chunk_seconds=chunk_seconds,identity_hash=sequence_identity_hash)
|
||||||
|
if multi_chunk_flf and session is not None:
|
||||||
|
saved_strategy=str((session.get("settings") or {}).get("flf_execution", ""))
|
||||||
|
if saved_strategy!=FLF_STRATEGY:
|
||||||
|
preserved=[]
|
||||||
|
reuse_notes.append("FLF chunk contract changed; regenerated the requested FLF sequence.")
|
||||||
|
if multi_chunk_flf:
|
||||||
|
reuse_notes.insert(0,"FLF execution: context-bound terminal v1; one sampling pass per chunk; the final chunk is anchored from the continuation-context tail to the supplied Last Frame.")
|
||||||
if reference_assets is not None:
|
if reference_assets is not None:
|
||||||
reuse_notes.insert(0,f"Reference conditioning: {reference_assets.count} image(s), size={reference_assets.size_mode}; persistent across all chunks.")
|
reuse_notes.insert(0,f"Reference conditioning: {reference_assets.count} image(s), size={reference_assets.size_mode}; persistent across all chunks.")
|
||||||
if reference_warning: reuse_notes.append(reference_warning)
|
if reference_warning: reuse_notes.append(reference_warning)
|
||||||
@@ -215,15 +268,23 @@ def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,si
|
|||||||
if timeline_video_source is not None:
|
if timeline_video_source is not None:
|
||||||
from ..timeline_video import encode_timeline_video_chunk
|
from ..timeline_video import encode_timeline_video_chunk
|
||||||
timeline_video_assets=encode_timeline_video_chunk(video_vae,timeline_video_source,sequence_index)
|
timeline_video_assets=encode_timeline_video_chunk(video_vae,timeline_video_source,sequence_index)
|
||||||
|
chunk_assets=assets
|
||||||
|
if timeline_video_source is not None:
|
||||||
chunk_cache=_conditioning_cache(clip=clip,prompts=[prompt],assets=assets,final_has_last_frame=bool(last_frame is not None and is_final),reference_assets=reference_assets,reference_audio_assets=reference_audio_assets,timeline_video_assets=timeline_video_assets)
|
chunk_cache=_conditioning_cache(clip=clip,prompts=[prompt],assets=assets,final_has_last_frame=bool(last_frame is not None and is_final),reference_assets=reference_assets,reference_audio_assets=reference_audio_assets,timeline_video_assets=timeline_video_assets)
|
||||||
if previous_state is None:
|
if previous_state is None:
|
||||||
total_frames=initial_frame_count; latent=empty_h3_latent(width,height,total_frames); conditioning=attach_keyframes(chunk_cache[(prompt,bool(last_frame is not None and is_final))],frame_count=total_frames,first_latent=assets.first_latent,last_latent=assets.last_latent if is_final else None); clip_index=1; context_frames=0
|
include_chunk_last=bool(last_frame is not None and is_final)
|
||||||
|
conditioning_key=_conditioning_cache_key(prompt,include_last=include_chunk_last,reference_assets=reference_assets)
|
||||||
|
total_frames=initial_frame_count; latent=empty_h3_latent(width,height,total_frames); conditioning=attach_keyframes(chunk_cache[conditioning_key],frame_count=total_frames,first_latent=chunk_assets.first_latent,last_latent=chunk_assets.last_latent if include_chunk_last else None); clip_index=1; context_frames=0
|
||||||
chunk_plan=make_plan(continuation=False,clip_index=clip_index,total_frames=total_frames,trim_frames=0,width=width,height=height,context_frames=5,state_capacity_frames=largest_context_capacity(total_frames),requested_extend_seconds=chunk_seconds,debug=debug); reason="initial clip"
|
chunk_plan=make_plan(continuation=False,clip_index=clip_index,total_frames=total_frames,trim_frames=0,width=width,height=height,context_frames=5,state_capacity_frames=largest_context_capacity(total_frames),requested_extend_seconds=chunk_seconds,debug=debug); reason="initial clip"
|
||||||
else:
|
else:
|
||||||
context_frames,motion_score,reason=choose_context_frames(continuity,previous_state); desired_cumulative=int(round((sequence_index+1)*chunk_seconds*FPS)); requested_new_frames=max(1,desired_cumulative-retained_frames); shape=make_extension_shape(context_frames,requested_new_frames/FPS); latent=empty_h3_latent(width,height,shape.total_frames)
|
context_frames,motion_score,reason=choose_context_frames(continuity,previous_state); desired_cumulative=int(round((sequence_index+1)*chunk_seconds*FPS)); requested_new_frames=max(1,desired_cumulative-retained_frames); shape=make_extension_shape(context_frames,requested_new_frames/FPS); latent=empty_h3_latent(width,height,shape.total_frames)
|
||||||
base_conditioning=attach_keyframes(chunk_cache[(prompt,bool(last_frame is not None and is_final))],frame_count=shape.total_frames,first_latent=assets.first_latent,last_latent=assets.last_latent if is_final else None)
|
include_chunk_last=bool(last_frame is not None and is_final)
|
||||||
|
conditioning_key=_conditioning_cache_key(prompt,include_last=include_chunk_last,reference_assets=reference_assets)
|
||||||
|
base_conditioning=attach_keyframes(chunk_cache[conditioning_key],frame_count=shape.total_frames,first_latent=chunk_assets.first_latent,last_latent=chunk_assets.last_latent if include_chunk_last else None)
|
||||||
video_context,audio_context,grid_offset=select_context(previous_state,context_frames,include_audio=bool(audio_continuity)); context_before=context_fingerprint(video_context,audio_context)
|
video_context,audio_context,grid_offset=select_context(previous_state,context_frames,include_audio=bool(audio_continuity)); context_before=context_fingerprint(video_context,audio_context)
|
||||||
conditioning=prepare_conditioning(base_conditioning,video_context=video_context,audio_context=audio_context,audio_grid_offset=grid_offset,context_frames=context_frames,new_frame_count=shape.total_frames,first_frame_policy=POLICY_REPLACE,preserve_last_frame=True)
|
conditioning=prepare_conditioning(base_conditioning,video_context=video_context,audio_context=audio_context,audio_grid_offset=grid_offset,context_frames=context_frames,new_frame_count=shape.total_frames,first_frame_policy=POLICY_REPLACE,preserve_last_frame=True)
|
||||||
|
if multi_chunk_flf and is_final:
|
||||||
|
conditioning=_attach_terminal_flf_keyframes(conditioning,video_context=video_context,context_frames=context_frames,frame_count=shape.total_frames,last_latent=assets.last_latent)
|
||||||
clip_index=int(previous_state["clip_index"])+1; chunk_plan=make_plan(continuation=True,clip_index=clip_index,total_frames=shape.total_frames,trim_frames=context_frames,width=width,height=height,context_frames=context_frames,state_capacity_frames=largest_context_capacity(shape.net_new_frames),requested_extend_seconds=chunk_seconds,debug=debug)
|
clip_index=int(previous_state["clip_index"])+1; chunk_plan=make_plan(continuation=True,clip_index=clip_index,total_frames=shape.total_frames,trim_frames=context_frames,width=width,height=height,context_frames=context_frames,state_capacity_frames=largest_context_capacity(shape.net_new_frames),requested_extend_seconds=chunk_seconds,debug=debug)
|
||||||
driving_audio_latent=slice_driving_audio_latent(driving_audio_assets,cumulative_retained_before=retained_frames,total_frames=int(chunk_plan["total_frames"]),trim_frames=int(chunk_plan["trim_frames"]),fps=FPS)
|
driving_audio_latent=slice_driving_audio_latent(driving_audio_assets,cumulative_retained_before=retained_frames,total_frames=int(chunk_plan["total_frames"]),trim_frames=int(chunk_plan["trim_frames"]),fps=FPS)
|
||||||
conditioning=attach_driving_audio(conditioning,driving_audio_latent)
|
conditioning=attach_driving_audio(conditioning,driving_audio_latent)
|
||||||
@@ -241,6 +302,7 @@ def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,si
|
|||||||
if latent_only:
|
if latent_only:
|
||||||
last_state=entry_to_state(entries[-1]); parent_id=session.get("session_id") if session is not None else None
|
last_state=entry_to_state(entries[-1]); parent_id=session.get("session_id") if session is not None else None
|
||||||
settings={"continuity":continuity,"audio_continuity":bool(audio_continuity),"exact_total_duration":False,"prompt_mode":plan["mode"],"conditioning_mode":conditioning_mode,"base_seed":int(base_seed),"reroll_nonce":int(reroll_nonce),"diagnostics_mode":diagnostics_mode,"initial_state_source":initial_state is not None,"latent_first":True,"first_frame_hash":assets.first_frame_hash,"last_frame_hash":assets.last_frame_hash,"reference_contract":reference_assets.contract if reference_assets is not None else None}
|
settings={"continuity":continuity,"audio_continuity":bool(audio_continuity),"exact_total_duration":False,"prompt_mode":plan["mode"],"conditioning_mode":conditioning_mode,"base_seed":int(base_seed),"reroll_nonce":int(reroll_nonce),"diagnostics_mode":diagnostics_mode,"initial_state_source":initial_state is not None,"latent_first":True,"first_frame_hash":assets.first_frame_hash,"last_frame_hash":assets.last_frame_hash,"reference_contract":reference_assets.contract if reference_assets is not None else None}
|
||||||
|
if multi_chunk_flf: settings["flf_execution"]=FLF_STRATEGY
|
||||||
if reference_audio_source is not None: settings["reference_audio_contract"]=reference_audio_source.contract
|
if reference_audio_source is not None: settings["reference_audio_contract"]=reference_audio_source.contract
|
||||||
if driving_audio_source is not None: settings["driving_audio_contract"]=driving_audio_source.contract
|
if driving_audio_source is not None: settings["driving_audio_contract"]=driving_audio_source.contract
|
||||||
if reference_video_source is not None: settings["reference_video_contract"]=reference_video_source.contract
|
if reference_video_source is not None: settings["reference_video_contract"]=reference_video_source.contract
|
||||||
@@ -257,6 +319,7 @@ def run_sequence(*,model:Any,clip:Any,video_vae:Any,audio_vae:Any,sampler:Any,si
|
|||||||
target_frames=int(round(chunks*chunk_seconds*FPS)); images,audio,duration_report=enforce_total_frames(images,audio,target_frames=target_frames,preserve_final_frame=last_frame is not None)
|
target_frames=int(round(chunks*chunk_seconds*FPS)); images,audio,duration_report=enforce_total_frames(images,audio,target_frames=target_frames,preserve_final_frame=last_frame is not None)
|
||||||
last_state=entry_to_state(entries[-1]); parent_id=session.get("session_id") if session is not None else None
|
last_state=entry_to_state(entries[-1]); parent_id=session.get("session_id") if session is not None else None
|
||||||
settings={"continuity":continuity,"audio_continuity":bool(audio_continuity),"exact_total_duration":bool(exact_total_duration),"prompt_mode":plan["mode"],"base_seed":int(base_seed),"reroll_nonce":int(reroll_nonce),"diagnostics_mode":diagnostics_mode,"initial_state_source":initial_state is not None,"first_frame_hash":assets.first_frame_hash,"last_frame_hash":assets.last_frame_hash,"reference_contract":reference_assets.contract if reference_assets is not None else None}
|
settings={"continuity":continuity,"audio_continuity":bool(audio_continuity),"exact_total_duration":bool(exact_total_duration),"prompt_mode":plan["mode"],"base_seed":int(base_seed),"reroll_nonce":int(reroll_nonce),"diagnostics_mode":diagnostics_mode,"initial_state_source":initial_state is not None,"first_frame_hash":assets.first_frame_hash,"last_frame_hash":assets.last_frame_hash,"reference_contract":reference_assets.contract if reference_assets is not None else None}
|
||||||
|
if multi_chunk_flf: settings["flf_execution"]=FLF_STRATEGY
|
||||||
new_session=make_session(chunks=entries,width=width,height=height,chunk_seconds=chunk_seconds,identity_hash=sequence_identity_hash,model_fingerprint_value=current_model_fingerprint,parent_session_id=parent_id,reroll_from_chunk=int(reroll_from_chunk),settings=settings)
|
new_session=make_session(chunks=entries,width=width,height=height,chunk_seconds=chunk_seconds,identity_hash=sequence_identity_hash,model_fingerprint_value=current_model_fingerprint,parent_session_id=parent_id,reroll_from_chunk=int(reroll_from_chunk),settings=settings)
|
||||||
decoded_gib=float(images.shape[0])*float(width)*float(height)*3.0*4.0/(1024.0**3)
|
decoded_gib=float(images.shape[0])*float(width)*float(height)*3.0*4.0/(1024.0**3)
|
||||||
report_lines=[f"H3 Continuum V2 {PACKAGE_VERSION}",prompt_plan_report(plan),f"Seam correction: {seam_correction}.",accelerators,"Execution: conditioning precomputed; single call-local MODEL clone per chunk; decode deferred until sampling completed.",*reuse_notes]
|
report_lines=[f"H3 Continuum V2 {PACKAGE_VERSION}",prompt_plan_report(plan),f"Seam correction: {seam_correction}.",accelerators,"Execution: conditioning precomputed; single call-local MODEL clone per chunk; decode deferred until sampling completed.",*reuse_notes]
|
||||||
|
|||||||
Reference in New Issue
Block a user