Release H3 Continuum V3.4

This commit is contained in:
ukr8b3g-cmyk
2026-08-19 20:00:10 +09:00
parent 9753fdf49d
commit f17af8f4e5
18 changed files with 5528 additions and 480 deletions
+156 -416
View File
@@ -1,488 +1,228 @@
# ComfyUI-H3-Continuum 3.3.0 # ComfyUI-H3-Continuum 3.4.0
<img width="5250" height="2020" alt="workflow (8)" src="https://github.com/user-attachments/assets/58bc27d7-cab4-4f7d-9f58-ed94bdad4a1d" />
Long-form MiniMax H3 video generation for ComfyUI with restartable chunks, persistent references, Driving Audio, Video Reference, seam correction, and optional Spectrum interoperability.
![H3 Continuum V3.4 workflow overview](docs/images/v34-workflow-overview.png)
Native long-form MiniMax H3 video and audio continuation for ComfyUI. > V3.4 is the current release candidate. V3.3 legacy nodes remain available for existing workflows.
**Stable release:** V3.3.0. It adds chunk-local Timeline Video and guarded decoded-video seam correction while preserving the stable sampler, Run Storage contracts, and Spectrum Interop API v1. ## Example workflows
https://github.com/user-attachments/assets/bfa8c683-fc9d-48f1-9cdb-477ca110cdf2 - [V3.4 standard](examples/workflows/MiniMax_H3_Continuum_V34.json)
- [V3.4 Turbo](examples/workflows/MiniMax_H3_Continuum_V34_turbo.json)
**H3 Continuum Sampler V3.3** generates 1 to 16 linked H3 chunks, carries raw video/audio latent context between chunks, supports T2VA, I2VA, FL2VA, multi-image Reference conditioning, and optional Timeline Video, and can resume completed chunks from disk. The templates include optional external nodes such as Spectrum, Video Helper Suite, rgthree, EasyUse, and RTX Video Super Resolution. Install, replace, connect, or bypass them according to your installation. Media and acceleration choices remain under user control.
V3 delegates video and audio decoding to normal **ComfyUI Core VAE Decode nodes**. No model weights or third-party accelerator code are bundled. ## What's new in V3.4
## Sample workflows ### Driving Audio
- [Standard workflow](examples/workflows/MiniMax_H3_Continuum_V33.json) - quality-oriented `res_multistep` profile with Spectrum enabled. Driving Audio is the primary V3.4 audio path.
- [Turbo workflow](examples/workflows/MiniMax_H3_Continuum_V33_turbo.json) - 8-step `euler` profile with Spectrum bypassed.
Download a JSON file and drag it onto the ComfyUI canvas. The examples use optional external custom nodes and local image/audio inputs; replace or bypass unavailable assets for your environment. - Source audio guides each chunk at its absolute sequence position.
- The original effective stream is preserved for final output.
- Generated audio and Audio Seam processing are bypassed while Driving Audio is active.
- Audio is not independently rewritten at every chunk boundary.
- Perfect lip sync is not guaranteed; results still depend on H3, source material, prompts, and sampling.
Turbo weights are available from [LightX2V MiniMax H3 Turbo](https://huggingface.co/lightx2v/Minimax-h3-Turbo/tree/main). ### Video Reference
## Key features Video Reference provides a persistent native H3 video reference across all chunks.
- Raw paired video/audio latent continuation without decode/re-encode between chunks. - It guides motion, framing, timing, and appearance.
- One production sampler UI with normal controls first and infrequent controls marked Advanced. - It is not a direct pixel-copy or deterministic identity-transfer system.
- T2VA, First Frame, Last Frame, First + Last Frame, and Reference conditioning. - Video and audio are independent. Route a loader's IMAGE output to Video Reference and AUDIO output to Driving Audio.
- Up to three active Reference images across every chunk, compacted in Core connection order. - Source decoding and frame-rate conversion remain loader responsibilities. Continuum does not impose an unnecessary forced 24 fps conversion.
- One native Reference Audio item across every chunk.
- Disk-backed Run Storage with automatic resume and compatible Revision reuse. | Video Reference Size | Purpose |
- Fixed, List, and Timeline prompts through one Sequence Prompt input. |---|---|
- ComfyUI Core Video/Audio VAE Decode and a separate Continuum Assemble stage. | Efficient - 0.4 MP | Default; lower token cost and faster iteration. |
- Chunk-local Timeline Video conditioning with an efficient 0.4 MP default. | Balanced - 0.6 MP | More detail at a higher compute cost. |
- Guarded Audio Seam and Video Seam correction after Core decode. | Match Output | Largest reference input; potentially much slower and heavier. |
- Optional Spectrum Actual Prefix 2 interoperability.
- External SageAttention, LoRA, Turbo, and Spectrum MODEL chains remain composable. ### Core-first, permissive execution
- Existing legacy node identifiers remain registered for saved-workflow compatibility.
V3.4 removes or relaxes Continuum-only rejection rules that blocked otherwise runnable Core H3 experiments.
- Prompt problems prefer warnings and fallback behavior instead of stopping.
- Unknown upstream/custom wrapper classes are not blanket-rejected.
- There is no model allowlist and no automatic model replacement.
- The obsolete strict_compatibility control is removed from the V3.4 interface.
- Current Core PackedLayout behavior is supported without requiring the removed legacy frame_count argument.
Hard stops remain only for states that cannot execute safely, including corrupt latent topology, incompatible assembly data, invalid persisted revisions, or unusable mandatory payloads.
### Cleaner interface
The public nodes focus on normal production controls. Developer diagnostics and detailed reports are available through ComfyUI settings.
![V3.4 sampler, Spectrum, and assembler](docs/images/v34-sampler-spectrum-assemble.png)
### Timeline paths
Experimental Timeline Video and earlier reference/timeline audio paths are no longer public V3.4 inputs. They are hidden rather than destructively removed. Existing V3.3 workflows can continue through legacy nodes.
## Installation ## Installation
### ComfyUI Manager Search for H3 Continuum or Continuum in ComfyUI Manager, or install manually:
Open **ComfyUI Manager**, search for **H3 Continuum** or **Continuum**, and select **Install**. ~~~bash
cd ComfyUI/custom_nodes
![H3 Continuum in ComfyUI Manager](docs/images/comfyui-manager-h3-continuum.png)
Restart ComfyUI after installation, then search for **H3 Continuum Sampler V3.3** in the node menu.
### Manual installation
From `ComfyUI/custom_nodes/`:
```bash
git clone https://github.com/ukr8b3g-cmyk/ComfyUI-H3-Continuum.git git clone https://github.com/ukr8b3g-cmyk/ComfyUI-H3-Continuum.git
``` ~~~
Restart ComfyUI and search for **H3 Continuum Sampler V3.3**. Restart ComfyUI after installation or update.
For ZIP installation, extract the repository as `ComfyUI-H3-Continuum` under `ComfyUI/custom_nodes/`, then restart ComfyUI. ## V3.4 nodes
Use an up-to-date ComfyUI. ComfyUI v0.32.0 or later is recommended for the current MiniMax H3 Core VAE fixes and memory improvements. ### H3 Continuum Sampler V3.4
## Current and legacy nodes Main inputs:
V3.3 uses two current workflow nodes: - model, clip, video_vae, sampler, sigmas
- Sequence Prompt
- first_frame, last_frame
- reference_image_1 through reference_image_3
- Video Reference
- driving_audio, audio_vae
- **H3 Continuum Sampler V3.3** - unified sampler for normal generation and optional Timeline Video conditioning. Outputs:
- **H3 Continuum Assemble + Seam** - Core-decoded chunk assembly with Audio Seam and guarded Video Seam correction.
The V3.2.4 nodes remain registered only so existing saved workflows continue to load: - video_latents, audio_latents
- assembly_plan, status
- driving_audio
- **[Legacy] H3 Continuum Sampler V3.2.4** Visible controls:
- **[Legacy] H3 Continuum Assemble V3.2.4**
New workflows should use the two V3.3 nodes. The legacy Node IDs are preserved, but they do not receive the integrated Timeline Video and current seam workflow surface. - Prompt Format, chunks, chunk_seconds
- width, height, continuity
- base_seed, control after generate
- audio_continuity
- Run Storage
- Reference Size
- Video Reference Size
No additional Python package is required beyond a working ComfyUI MiniMax H3 environment. ### H3 Continuum Assemble + Seam V3.4
## Standard connection Inputs: images, audio, assembly_plan, and driving_audio.
```text Controls: Audio Seam and Video Seam.
H3 MODEL Loader
-> optional SageAttention
-> optional Turbo/LoRA
-> optional Spectrum
-> H3 Continuum Sampler V3.3.model
MiniMax H3 CLIP/Text Encoder ------> clip When Driving Audio is connected, preserved source audio is selected for final output and generated audio seam processing is bypassed.
MiniMax H3 Video VAE -------------> video_vae
Sampler --------------------------> sampler
Sigmas ---------------------------> sigmas
Text (Multiline) -----------------> Sequence Prompt
Optional images ------------------> first_frame / last_frame / reference_image_1/2/3
video_latents --> Core VAE Decode -----------+ ## Connection order
audio_latents --> Core VAE Decode Audio -----+--> H3 Continuum Assemble V3
assembly_plan -------------------------------+
H3 Continuum Assemble V3.images/audio --> Create Video ~~~text
``` H3 model / CLIP / Video VAE / sampler / sigmas
|
v
H3 Continuum Sampler V3.4
| | |
video_latents | assembly_plan
|
audio_latents
The square grid icon on `video_latents` and `audio_latents` means a list of chunk latents. Connect each output to one normal Core decode node; ComfyUI maps the decoder across the chunk list. video_latents -> Core VAE Decode ------- images --+
audio_latents -> Core VAE Decode Audio -- audio --+--> H3 Continuum Assemble + Seam V3.4
assembly_plan -------------------------------------+
driving_audio -------------------------------------+
~~~
The `video_vae` input is used for image conditioning when needed. Continuum does not perform final Video VAE decoding internally. For a source video with sound:
## Conditioning modes ~~~text
Video loader IMAGE -> Video Reference
Video loader AUDIO -> driving_audio
~~~
The sampler selects the mode from connected image inputs. The supplied templates use Video Helper Suite for this split. Any compatible IMAGE/AUDIO loader may be substituted.
| Connected images | Mode | ## Prompt formats
| --- | --- |
| None | T2VA + Continuation |
| First Frame | I2VA + Continuation |
| Last Frame | Last-frame-conditioned + Continuation |
| First + Last Frame | FL2VA + Continuation |
| Reference 1, optionally Reference 2 and 3 | Reference + Continuation |
Reference images and First/Last Frame are mutually exclusive. Active Reference inputs are compacted in Core connection order; bypassed sockets are ignored. Timeline:
Reference Image 1, 1+2, or 1+2+3 may be connected. A single active image is treated as Picture 1, and active images are numbered without gaps. **Ref2VA** is the reference-specialized checkpoint and may provide stronger reference fidelity, while **FL2VA + Reference** is also allowed. Checkpoint classification is diagnostic only; the sampler never switches MODELs automatically. **Strict Compatibility** remains reserved for H3 contracts that are actually unsafe or unsupported. ~~~text
[0-5s] First section: action, camera movement, and scene progression.
[5-10s] Continue from the exact final state of the previous section.
[10-15s] Continue naturally without resetting the scene.
~~~
Reference prompts should identify the images explicitly: List prompts use --- separators. Fixed reuses one prompt for every chunk. Auto detects the available structure. If a timeline cannot be parsed, V3.4 reports a warning and uses an applicable fallback rather than rejecting an otherwise runnable workflow.
```text ## Run Storage
<Picture 1> defines the subject's face and identity.
<Picture 2> defines the subject's clothing, body design, and skateboard.
<Picture 3> optionally defines another ordered identity, object, or environment reference.
```
Reference images persist across all Continuum chunks. Run Storage preserves completed chunks and can reuse them after restarting ComfyUI. Regenerate From selects the first chunk to regenerate; earlier compatible chunks remain unchanged.
### Reference Size ![Regenerate From](docs/images/v34-regenerate-from.png)
- **Match Output**: preserve aspect ratio and downscale toward the output pixel area. - Use a fixed seed for reproducible resume.
- **Max Identity**: retain a larger identity reference, downscaling only when necessary. - Changing persistent models, LoRAs, references, prompts, resolution, or sampling settings can create a new revision.
- Unknown upstream node classes no longer cause rejection by themselves; reuse still depends on compatible observable contracts and hashes.
Both modes avoid stretching and use crop-free Reference preprocessing. ## Spectrum interoperability
## Quick start Spectrum is optional. Current Spectrum releases officially support H3 Continuum Interop API v1.
1. Add **H3 Continuum Sampler V3.3**. - Chunk 1 uses the normal initial path.
2. Connect MODEL, CLIP, Video VAE, sampler, sigmas, and one Text (Multiline) node. - Chunk 2 and later request Actual Prefix 2.
3. Set `Prompt Format = Auto`. - Successful continuation logs accepted H3 Continuum API v1, actual prefix=2.
4. Start with `chunks = 3`, `chunk_seconds = 5.0`, and `Balanced - 22 frames`. - Unsupported Spectrum versions fall back without a private patch.
5. Connect the raw latent outputs to Core VAE Decode and Core VAE Decode Audio.
6. Connect decoded outputs and `assembly_plan` to **H3 Continuum Assemble + Seam**.
7. Connect Assemble images/audio to Create Video.
Recommended safe defaults: See [Spectrum v0.2.15 H3 Continuum interoperability](https://github.com/xmarre/ComfyUI-Spectrum-MiniMax-H3#v0215-h3-continuum-interoperability).
| Setting | Starting value | Turbo LoRA and Spectrum are not mutually exclusive. Quality and speed remain workflow-dependent.
| --- | --- |
| Prompt Format | Auto |
| Continuity | Balanced - 22 frames |
| Audio Continuity | On |
| Audio Seam | Auto |
| Regenerate From | Auto |
| Run Storage | Off for short tests; Save + Auto Resume for long runs |
| SageAttention | Auto when backend compatibility is unknown |
## H3 Continuum Sampler UI ## Issue and pull-request response
The V3.3 production node keeps normal generation controls visible and moves developer-only options to ComfyUI Settings. Connection-specific controls appear only when the related input or Run Storage mode is active. ### Issue #3
![H3 Continuum V3.3 streamlined sampler and assembler](docs/images/h3-continuum-v33-streamlined-ui.png) V3.4 removes blanket rejection of unknown upstream/custom class names. Run Storage evaluates the observable generation contract instead of treating an unfamiliar wrapper as automatically incompatible.
### Main inputs and controls See [Issue #3](https://github.com/ukr8b3g-cmyk/ComfyUI-H3-Continuum/issues/3).
| Group | Input or control | Purpose | ### Issue #4
| --- | --- | --- |
| H3 pipeline | `model`, `clip`, `video_vae`, `sampler`, `sigmas` | Connect the normal MiniMax H3 MODEL, text encoder, Video VAE, sampler, and sigma schedule. |
| Prompt | `Sequence Prompt` | Fixed, List, or Timeline text for the complete Continuum run. |
| Keyframes | `first_frame`, `last_frame` | Optional I2VA, last-frame-conditioned, or FL2VA keyframes. |
| Reference images | `reference_image_1`, `reference_image_2`, `reference_image_3` | Up to three ordered Picture references. Bypassed inputs are ignored. |
| Reference audio | `reference_audio_1`, `reference_audio_vae` | Optional native H3 Reference Audio. Connect the Audio VAE only when Reference Audio is used. |
| Duration | `chunks`, `chunk_seconds` | Number of linked chunks and duration of each chunk. Start with `3 x 5.0` seconds. |
| Canvas | `width`, `height` | Output dimensions. Use values compatible with the current H3 Core path, normally multiples of 32. |
| Continuity | `continuity` | Number of previous raw frames carried into the next chunk. `Balanced - 22 frames` is the practical default. |
| Seed | `base_seed` | Base seed used to derive deterministic per-chunk seeds. |
Reference images and First/Last Frame are mutually exclusive. Reference Audio can be combined with either keyframe conditioning or Reference Image conditioning. V3.4 follows the current Core H3 layout contract and no longer requires the legacy frame_count parameter. The old strict-compatibility toggle is not part of the public interface.
### Advanced controls See [Issue #4](https://github.com/ukr8b3g-cmyk/ComfyUI-H3-Continuum/issues/4).
| Control | Default | Use | ### Pull request #1
| --- | --- | --- |
| Prompt Format | Auto | Detect Fixed, List, or Timeline prompts. |
| Audio Continuity | On | Carry raw audio context between chunks. |
| Run Storage | Off | Use `Save + Auto Resume` for long or interruptible runs. |
| Run Name | blank | Appears with `Save + Auto Resume`; optionally overrides the automatic storage identity. |
| Regenerate From | Auto | Appears with `Save + Auto Resume`; reuse normally or regenerate from a selected saved chunk. |
| Variation Nonce | 0 | Appears only for explicit chunk regeneration and creates a deliberate alternate revision. |
| Reference Size | Match Output | Appears when a Reference Image is connected. |
| Timeline Video Size | Efficient - 0.4 MP | Appears when Timeline Video is connected. |
### ComfyUI settings Relevant current-Core compatibility ideas are selectively adapted to V3.4 rather than copying the pull request wholesale. The pull request itself is not described as merged.
The following package-wide options are available in ComfyUI Settings instead of occupying every sampler and assembler node: See [Pull request #1](https://github.com/ukr8b3g-cmyk/ComfyUI-H3-Continuum/pull/1).
![H3 Continuum settings](docs/images/h3-continuum-v33-settings.png) ### Pull request #2
| Setting | Default | Use | The proposed eight-reference expansion and additional high-resolution assembly RAM changes are not included in this update. They remain a separate review item.
| --- | --- | --- |
| H3 Continuum: Sampling Preview | On | Show normal sampling preview/progress information. |
| H3 Continuum: Developer Diagnostics | Off | Enable developer-oriented diagnostics when troubleshooting. |
| H3 Continuum: Detailed Report | Off | Expand sampler and assembly status text. Normal generation does not require it. |
These settings are applied when the workflow is queued. Hidden legacy widgets remain compatible with older saved workflows, while unknown or merged MODEL filenames are not rejected by name. See [Pull request #2](https://github.com/ukr8b3g-cmyk/ComfyUI-H3-Continuum/pull/2).
### Outputs ## Current validation status
| Output | Connect to | V3.4 has been exercised locally with:
| --- | --- |
| `video_latents` | One normal ComfyUI Core Video VAE Decode node. |
| `audio_latents` | One normal ComfyUI Core Audio VAE Decode node. |
| `assembly_plan` | **H3 Continuum Assemble V3**. |
| `status` | Core **Preview as Text** when a readable run report is needed. |
## Sequence Prompt - 1, 2, and 3 chunks
- standard and Turbo paths
- Spectrum enabled and disabled
- I2VA, Reference, and selected FL2VA configurations
- Driving Audio with short, exact-length, and longer sources
- Video Reference with source audio routed separately
- 0.4 MP and 0.6 MP reference sizing
- Run Storage reuse and selected-chunk regeneration
- Core VAE Decode and final assembly
### Fixed No OOM was observed in the cited recent local V3.4 checks, including two-chunk 800 x 800 runs. This is not a universal memory guarantee. Model precision, LoRAs, source resolution, optional nodes, GPU, and RAM affect memory use.
One prompt is reused for all chunks. ## Limits
```text - Continuation does not guarantee frame-perfect identity or motion.
A continuous cinematic shot. Preserve the same subject, camera direction, - Video Reference guides H3; it does not reproduce every source frame.
lighting, movement, voice, soundscape, and music across every continuation. - Driving Audio preserves selected audio, but visual lip synchronization remains model-dependent.
``` - Match Output can be substantially slower than 0.4 MP or 0.6 MP.
- Seam correction may keep the native boundary when a proposed correction is not safer.
### List - Optional template nodes must be installed, replaced, or bypassed by the user.
Separate one prompt per chunk with a line containing three hyphens.
```text
Opening action.
---
Continue seamlessly from the exact previous movement.
---
Complete the action naturally without a cut or pose reset.
```
If fewer sections are provided than `chunks`, the last section is repeated.
### Timeline
```text
[0-5s]
Opening action.
[5-10s]
Continuous middle action.
[10-15s]
Natural ending.
```
`Prompt Format = Auto` detects Fixed, List, or Timeline syntax.
### Local LLM / Qwen prompt generation
MiniMax provides the official [H3 prompt-writing skill](https://github.com/MiniMax-AI/MiniMax-H3/tree/main/skills/h3-prompt-writing) for T2VA, I2VA, FL2VA, L2VA, and Ref2VA. Give a local agent the complete `h3-prompt-writing` folder, not only `SKILL.md`, because the detailed formats and examples are stored under `references/`.
The official skill defines the H3 prompt fields and reference-label rules. H3 Continuum adds one routing requirement: divide the visual progression into one prompt section per Continuum chunk. The currently validated workflow uses 5-second chunks.
For reliable output, use `Prompt Format = Timeline` or `Auto` and require:
- exactly one non-empty section per configured chunk;
- sequential ranges such as `[0-5s]`, `[5-10s]`, and `[10-15s]`;
- shared Ref2VA subject and reference definitions only once, before the first timeline section;
- only labels for active connected inputs, using compact Core numbering;
- a different action, camera, and scene progression for each section;
- no explanatory text outside the final H3 prompt.
If fewer usable sections are produced, Continuum may reuse the previous available section. Matching the requested chunk count is therefore strongly recommended.
Copy-paste system instruction for a local LLM:
```text
You write production prompts for ComfyUI-H3-Continuum V3.3.
Use the complete official MiniMax H3 prompt-writing skill as the authority for
T2VA, I2VA, FL2VA, L2VA, and Ref2VA field order, syntax, and reference labels.
The user will provide:
- generation mode;
- chunk count;
- seconds per chunk;
- active image, video, and audio inputs;
- scene requirements.
Output rules:
1. Output only the final H3 prompt. Do not add explanations or Markdown fences.
2. Produce exactly one non-empty Timeline section per requested chunk.
3. Use sequential headers calculated from the supplied chunk duration. For three
5-second chunks, use exactly [0-5s], [5-10s], and [10-15s].
4. Every section must describe the action, camera movement, scene progression,
and relevant sound for that interval. Continue naturally from the previous
section; do not repeat the first section.
5. Mention only active connected references. Number Picture, Video, and Audio
labels compactly in ComfyUI Core order. Never mention bypassed, missing, or
unconnected inputs.
6. For Ref2VA, write subject_definitions, summary, and retention_analysis once
before the first Timeline header. In each Timeline section, provide that
chunk's detailed_description, overall_soundscape, and non_diegetic_music.
7. For T2VA, I2VA, FL2VA, and L2VA, each Timeline section must contain the
official integrated_multimodal_description, overall_soundscape, and
non_diegetic_music structure for that chunk.
8. Preserve requested dialogue, lyrics, and visible text verbatim and in their
requested language. Follow the official H3 dialogue and reference syntax.
9. Do not claim guaranteed frame-perfect continuity or exact audio copying.
```
## Run Storage and automatic resume
For long generations:
```text
Run Storage = Save + Auto Resume
Regenerate From = Auto
reroll_nonce = 0
```
![H3 Continuum chunk auto-resume settings](docs/images/v31b-auto-resume.png)
Each completed raw AV chunk is atomically saved. If generation stops, Queue the same compatible workflow again:
```text
Chunk 1 saved
Chunk 2 saved
Chunk 3 interrupted
Next Queue:
Chunk 1 reused
Chunk 2 reused
Chunk 3 generated
```
Run Storage creates deterministic Revisions from the sampling contract. When Run Name and Auto Resume ID are blank, a stable storage ID is derived from the sampler node. It tracks observable inputs including:
- H3 MODEL loader and ordered MODEL patch/wrapper chain.
- LoRA/Turbo enabled state, filename, and strength.
- CLIP/Qwen encoder and Video VAE loaders.
- Sampler, exact sigma schedule, resolution, duration, continuity, seed, and prompts.
- Conditioning mode, First/Last Frame hashes, and Reference image hashes/settings.
Changing FL2VA to Ref2VA, changing LoRA strength, or changing another tracked setting creates or selects a different Revision. Returning to an earlier compatible configuration reuses its earlier Revision.
The upstream graph and runtime MODEL/CLIP/VAE weight probes are both retained. Unknown wrappers or incomplete runtime probes fail safe: chunks may still be saved, but automatic reuse is disabled when identity cannot be established reliably.
Saved chunk files are verified by size and SHA-256 before reuse. Storage schema changes preserve old files but do not automatically reuse unverifiable older chunks.
Even when all sampling chunks are reused, Core VAE Decode and final assembly run again. A reused 15-second test can therefore still take roughly one minute.
### Regenerate from a chunk
Select `Chunk N` in `Regenerate From` to reuse chunks before N and regenerate N onward. Keep `Auto` for normal resume. Chunk regeneration requires `Run Storage = Save + Auto Resume`.
`reroll_nonce = 0` uses automatic Revision behavior. A positive nonce is retained for compatibility and explicit branch selection.
## Spectrum-aware continuation
With [ComfyUI Spectrum MiniMax H3](https://github.com/xmarre/ComfyUI-Spectrum-MiniMax-H3), Continuum requests **Actual Prefix 2** for continuation chunks through Interop API v1. First-class H3 Continuum API v1 interoperability is included upstream in Spectrum v0.2.15 and later, so a current standard Spectrum installation can receive this request without a separate fork. Confirm the acceptance log below during runtime validation.
```text
Chunk 1: normal path
Chunk 2+: inherited raw AV Context
-> at least two Actual Transformer steps
-> normal Spectrum forecast schedule
```
This does not add solver steps. It protects the first evaluations after a new continuation Context before forecasting begins.
A compatible receiver logs:
```text
Spectrum H3: accepted H3 Continuum API v1, actual prefix=2
```
Actual Prefix 2 is active only when the Spectrum log reports that the Continuum API request was accepted. Older or incompatible Spectrum versions continue without that receiver behavior.
### Benefits of the upstream Spectrum integration
The Continuum receiver is included in the official Spectrum release, so users no longer need a Continuum-specific Spectrum fork or local patch. Installing or updating Spectrum through its normal distribution path preserves the integration.
- **Automatic handshake:** Continuum emits the request only for continuation chunks, and Spectrum applies Actual Prefix 2 when it recognizes Interop API v1.
- **Safer chunk transitions:** the first two Transformer evaluations after a new raw AV Context are evaluated normally before Spectrum forecasting resumes. This is intended to reduce forecast instability immediately after a chunk boundary; it does not guarantee that every visible seam or flicker will disappear.
- **No additional solver steps:** Actual Prefix 2 changes which existing evaluations are calculated as Actual steps; it does not increase the configured step count.
- **Update-friendly operation:** the receiver becomes part of upstream Spectrum, so a normal Spectrum update does not overwrite a separate local interoperability patch.
- **Optional composition:** Continuum still works without Spectrum. MODEL selection, Turbo/LoRA, SageAttention, and Spectrum remain external workflow choices, and Continuum does not install or switch them automatically.
After updating Spectrum, confirm that Chunk 2 and later print the acceptance log shown above. Its presence is the runtime proof that the upstream receiver is active.
### Standard and Turbo profiles
```text
Quality
standard approximately 20-step H3 sampling
Spectrum ON
Actual Prefix 2 automatic
Fast
Turbo / 8-step profile
Spectrum OFF
Experimental
Turbo + Spectrum
```
Turbo + Spectrum is not the recommended default because local testing showed a higher risk of artifacts. Sol-Attn is also not part of the current recommended continuity profile.
Turbo LoRA files are available from [LightX2V MiniMax H3 Turbo](https://huggingface.co/lightx2v/Minimax-h3-Turbo/tree/main). The ComfyUI-specific FL2VA and Ref2VA variants can be selected in the upstream MODEL/LoRA chain; Continuum does not bundle or switch them automatically.
## Timeline Video
`H3 Continuum Sampler V3.3` includes an optional `timeline_video` input. When connected, the source is sliced per 5-second chunk, resized independently from the output, encoded only for chunks that must be generated, and released after use. When it is not connected, the same node operates as the normal Continuum sampler.
- `Efficient - 0.4 MP` is the default and recommended starting point.
- `Match Output` preserves more source detail but can be substantially slower and heavier.
- Timeline-video audio is not used; connect Reference Audio separately when needed.
- Results still depend on the checkpoint, prompt, source motion, and reference compatibility.
## Assemble + Seam
Connect decoded chunk images, audio, and `assembly_plan` to `H3 Continuum Assemble + Seam`.
- Default: `Audio Seam = Auto`, `Video Seam = Auto`.
- Video modes: `Auto`, `Auto 2`, `Analyze Only`, `Off`.
- `Auto` applies guarded transient and micro-flash correction without frame deletion.
- `Auto 2` additionally enables guarded exposure-ramp correction and remains experimental.
- `Analyze Only` reports the boundary classification without changing images or audio.
## Why Core Decode is external
Continuum owns sampling, continuation planning, chunk metadata, context trimming, audio alignment, exact duration, and final assembly.
ComfyUI Core owns Video/Audio VAE execution, tiled decode behavior, memory strategy, and future H3 VAE improvements.
This allows the normal Core `VAE Decode` to benefit from ComfyUI updates without copying H3 VAE implementation code into Continuum. Use Core `VAE Decode (Tiled)` when the normal decode path is unsuitable for the selected resolution or available memory.
Full raw chunks are decoded before Continuum trims repeated context. Trimming latents before decode would change temporal VAE conditioning.
## Validation summary
V3.3.0 passed the automated suite and Windows runtime checks for the production sampler, Reference Image/Audio, Run Storage, Timeline Video, Spectrum Actual Prefix 2, Audio Seam Auto, and guarded Video Seam Auto.
`Auto 2` is included as an experimental fallback when `Auto` does not sufficiently reduce a boundary exposure ramp. Detailed test conditions, results, and remaining limits are recorded in [docs/VALIDATION_RESULTS.md](docs/VALIDATION_RESULTS.md).
## Diagnostics
`status` is a normal STRING output. Connect it to Core **Preview as Text** when you need the sampling and Run Storage report.
Useful lines include:
```text
Conditioning mode: Reference + Continuation.
interop=emitted actual_prefix=2
3 reused, 0 generated
resume=complete
```
`accelerator markers not detected (informational only)` is not an error. It only means the optional accelerator marker was not observable.
Keep **Detailed Report** and **Developer Diagnostics** disabled normally. Enable them in ComfyUI Settings only when troubleshooting or collecting a diagnostic report.
## Compatibility and scope
- Native MiniMax H3 PackedLayout is preserved.
- Video and audio VAEs are not run between sampling chunks.
- Accepted raw chunk latents are retained on CPU.
- MODEL input is cloned once per chunk; the workflow MODEL is not mutated.
- SageAttention, LoRA/Turbo, Spectrum, and model weights remain external.
- Unknown H3 layout contracts fail clearly.
- Legacy V1/V2/V3 node identifiers remain registered for older workflows but are not the recommended starting point.
Continuum is independently implemented. It does not copy or bundle source code from Spectrum or [H3 Motion Context](https://github.com/NikoDemon80/ComfyUI-H3-Motion-Context).
## Development checks
```bash
python -m compileall -q .
python -m pytest -q
```
The GitHub Actions workflow runs compileall and the full pytest suite with its explicit test dependencies.
## License ## License
MIT. Model files and third-party custom nodes retain their own licenses. MiniMax H3 weights are not redistributed. See [LICENSE](LICENSE).
+4 -6
View File
@@ -82,7 +82,6 @@ def check_comfy_h3_runtime() -> list[str]:
keywords=( keywords=(
"keyframes", "keyframes",
"refs", "refs",
"frame_count",
), ),
) )
for name in missing: for name in missing:
@@ -171,7 +170,7 @@ def run_native_layout_self_test() -> str:
MARK_AUDIO_OVERHANG: -1.0 / 3.0, MARK_AUDIO_OVERHANG: -1.0 / 3.0,
} }
layout = h3_model.PackedLayout( layout = h3_model.PackedLayout(
3, 7, 2, 2, 10, keyframes=None, refs=[ref], frame_count=22 3, 7, 2, 2, 10, keyframes=None, refs=[ref]
) )
position_id = id(layout.position_ids) position_id = id(layout.position_ids)
payload = {"layout": layout, "keyframes": [], "refs": [ref]} payload = {"layout": layout, "keyframes": [], "refs": [ref]}
@@ -196,9 +195,6 @@ def ensure_native_h3_base_model(base_model: Any) -> None:
inner = getattr(base_model, "diffusion_model", None) inner = getattr(base_model, "diffusion_model", None)
if inner is None: if inner is None:
raise CompatibilityError("MODEL does not expose diffusion_model") raise CompatibilityError("MODEL does not expose diffusion_model")
actual = f"{type(inner).__module__}.{type(inner).__name__}"
if actual != "comfy.ldm.minimax.model.MiniMaxH3Model":
raise CompatibilityError(f"native MiniMaxH3Model required; got {actual}")
missing = [ missing = [
name name
for name in ( for name in (
@@ -212,8 +208,10 @@ def ensure_native_h3_base_model(base_model: Any) -> None:
if not hasattr(inner, name) if not hasattr(inner, name)
] ]
if missing: if missing:
actual = f"{type(inner).__module__}.{type(inner).__name__}"
raise CompatibilityError( raise CompatibilityError(
"MiniMaxH3Model contract changed; missing " + ", ".join(missing) f"MiniMax H3 model contract is unavailable on {actual}; missing "
+ ", ".join(missing)
) )
-4
View File
@@ -31,10 +31,6 @@ _MODE_LABELS = {
def conditioning_mode_from_presence( def conditioning_mode_from_presence(
*, has_first: bool, has_last: bool, has_reference: bool *, has_first: bool, has_last: bool, has_reference: bool
) -> str: ) -> str:
if has_reference and (has_first or has_last):
raise ValueError(
"Reference Images cannot be combined with First Frame or Last Frame"
)
if has_reference: if has_reference:
return CONDITIONING_MODE_REFERENCE return CONDITIONING_MODE_REFERENCE
if has_first and has_last: if has_first and has_last:
+6 -3
View File
@@ -66,6 +66,7 @@ def prepare_conditioning(
new_frame_count: int, new_frame_count: int,
first_frame_policy: str, first_frame_policy: str,
preserve_last_frame: bool, preserve_last_frame: bool,
source_frame_count: int | None = None,
): ):
"""Add one native H3 video/video-audio reference as timeline context.""" """Add one native H3 video/video-audio reference as timeline context."""
if first_frame_policy not in FIRST_FRAME_POLICIES: if first_frame_policy not in FIRST_FRAME_POLICIES:
@@ -110,6 +111,8 @@ def prepare_conditioning(
output = clone_conditioning(conditioning) output = clone_conditioning(conditioning)
for _, metadata in output: for _, metadata in output:
old_frame_count = metadata.get("minimax_frame_count") old_frame_count = metadata.get("minimax_frame_count")
if old_frame_count is None:
old_frame_count = source_frame_count
old_keyframes = [dict(item) for item in (metadata.get("minimax_keyframes") or [])] old_keyframes = [dict(item) for item in (metadata.get("minimax_keyframes") or [])]
kept = [] kept = []
for keyframe in old_keyframes: for keyframe in old_keyframes:
@@ -130,9 +133,9 @@ def prepare_conditioning(
elif is_last: elif is_last:
continue continue
else: else:
raise ValueError( # Core H3 accepts arbitrary guide positions. Preserve them and
f"unsupported existing H3 keyframe index {frame_index}; only first/last are safe" # let the native keyframe/layout path decide their semantics.
) kept.append(keyframe)
if kept: if kept:
metadata["minimax_keyframes"] = kept metadata["minimax_keyframes"] = kept
else: else:
Binary file not shown.

After

Width:  |  Height:  |  Size: 61 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 433 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 569 KiB

+178
View File
@@ -0,0 +1,178 @@
"""Core-aligned Driving Audio preparation for H3 Continuum V3.4."""
from __future__ import annotations
from dataclasses import dataclass
import hashlib
import json
from typing import Any
import torch
DRIVING_AUDIO_CONTRACT_VERSION = 1
DRIVING_AUDIO_PREPROCESS_VERSION = 1
H3_AUDIO_LATENT_FPS = 40
def _tensor_sha256(value: torch.Tensor) -> str:
tensor = value.detach().to("cpu").contiguous()
digest = hashlib.sha256()
digest.update(str(tuple(int(v) for v in tensor.shape)).encode("ascii"))
digest.update(str(tensor.dtype).encode("ascii"))
digest.update(tensor.numpy().tobytes(order="C"))
return digest.hexdigest()
def _canonical_sha256(value: Any) -> str:
payload = json.dumps(
value,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=True,
).encode("utf-8")
return hashlib.sha256(payload).hexdigest()
@dataclass(frozen=True)
class DrivingAudioSource:
source_audio: dict[str, Any]
resampled_waveform: torch.Tensor
resolved_vae_sample_rate: int
contract: dict[str, Any]
combined_hash: str
@dataclass(frozen=True)
class DrivingAudioAssets:
source: DrivingAudioSource
audio_latent: torch.Tensor
def prepare_driving_audio_source(
audio: dict[str, Any] | None,
audio_vae: Any,
*,
target_frames: int,
fps: int,
) -> DrivingAudioSource | None:
"""Prepare only the effective target-duration prefix; never pad a short source."""
if audio is None:
return None
if audio_vae is None:
raise ValueError("anchoring guide audio needs the audio_vae input")
waveform = audio["waveform"]
sample_rate = int(audio["sample_rate"])
if not isinstance(waveform, torch.Tensor):
raise TypeError("AUDIO waveform must be a torch.Tensor")
target_samples = max(0, round(int(target_frames) * sample_rate / int(fps)))
effective_waveform = waveform[..., : min(int(waveform.shape[-1]), target_samples)]
source_audio = {
"waveform": effective_waveform,
"sample_rate": sample_rate,
}
resolved_rate = int(getattr(audio_vae, "audio_sample_rate", 32000))
resampled = effective_waveform
if sample_rate != resolved_rate:
import torchaudio
resampled = torchaudio.functional.resample(
effective_waveform,
sample_rate,
resolved_rate,
)
contract = {
"driving_audio_contract_version": DRIVING_AUDIO_CONTRACT_VERSION,
"source_sample_rate": sample_rate,
"source_shape": list(effective_waveform.shape),
"source_dtype": str(effective_waveform.dtype),
"source_sha256": _tensor_sha256(effective_waveform),
"resolved_vae_sample_rate": resolved_rate,
"resampled_shape": list(resampled.shape),
"resampled_dtype": str(resampled.dtype),
"resampled_sha256": _tensor_sha256(resampled),
"preprocess_version": DRIVING_AUDIO_PREPROCESS_VERSION,
}
return DrivingAudioSource(
source_audio=source_audio,
resampled_waveform=resampled,
resolved_vae_sample_rate=resolved_rate,
contract=contract,
combined_hash=_canonical_sha256(contract),
)
def encode_driving_audio(
source: DrivingAudioSource | None,
audio_vae: Any,
) -> DrivingAudioAssets | None:
if source is None:
return None
latent = audio_vae.encode(source.resampled_waveform[:1].movedim(1, -1))
return DrivingAudioAssets(source=source, audio_latent=latent)
def slice_driving_audio_latent(
assets: DrivingAudioAssets | None,
*,
cumulative_retained_before: int,
total_frames: int,
trim_frames: int,
fps: int,
) -> torch.Tensor | None:
if assets is None:
return None
source_start_frame = max(
0,
int(cumulative_retained_before) - int(trim_frames),
)
source_end_frame = source_start_frame + int(total_frames)
latent_start = round(source_start_frame * H3_AUDIO_LATENT_FPS / int(fps))
latent_end = round(source_end_frame * H3_AUDIO_LATENT_FPS / int(fps))
latent_length = int(assets.audio_latent.shape[-1])
latent_start = min(max(0, latent_start), latent_length)
latent_end = min(max(latent_start, latent_end), latent_length)
if latent_end <= latent_start:
return None
return assets.audio_latent[..., latent_start:latent_end].clone()
def combine_driving_audio_identity(
visual_identity_hash: str,
source: DrivingAudioSource | None,
) -> str:
if source is None:
return visual_identity_hash
return _canonical_sha256(
{
"driving_audio_identity_version": 1,
"visual_identity_hash": visual_identity_hash,
"driving_audio_hash": source.combined_hash,
}
)
def attach_driving_audio(
conditioning: list,
audio_latent: torch.Tensor | None,
) -> list:
"""Attach a Core-compatible guide-audio keyframe without tokenizer refs."""
if audio_latent is None:
return conditioning
output = []
for tokens, metadata in conditioning:
copied = dict(metadata)
keyframes = list(copied.get("minimax_keyframes", []))
keyframes.append(
{
"resolved_frame_index": 0,
"audio_latent": audio_latent,
}
)
copied["minimax_keyframes"] = keyframes
output.append([tokens, copied])
return output
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+15 -34
View File
@@ -1,4 +1,4 @@
"""Deterministic, fail-closed fingerprints for ComfyUI upstream loader graphs.""" """Deterministic fingerprints for arbitrary ComfyUI upstream loader graphs."""
from __future__ import annotations from __future__ import annotations
@@ -7,29 +7,7 @@ import json
from typing import Any from typing import Any
GRAPH_CONTRACT_VERSION = 2 GRAPH_CONTRACT_VERSION = 3
# Keep this deliberately explicit. New loader/patch nodes must be audited before
# their saved chunks can be resumed automatically.
_KNOWN_MODEL_NODES = {
"UNETLoader",
"LoraLoader",
"LoraLoaderModelOnly",
"Power Lora Loader (rgthree)",
"PathchSageAttentionKJ",
"PatchSageAttentionKJ",
"SpectrumApplyMiniMaxH3",
"MiniMaxH3MemoryEfficientSageAttention",
"MiniMaxH3MemoryEfficientSageAttentionPatch",
"MiniMaxH3SigmaShift",
"ModelSamplingMiniMaxH3",
}
_KNOWN_CLIP_NODES = {"CLIPLoader", "DualCLIPLoader", "TripleCLIPLoader"}
_KNOWN_VAE_NODES = {"VAELoader"}
_KNOWN_SHARED_NODES = {
"Reroute",
"Any Switch (rgthree)",
}
CHECKPOINT_REF2VA = "ref2va" CHECKPOINT_REF2VA = "ref2va"
CHECKPOINT_FL2VA = "fl2va" CHECKPOINT_FL2VA = "fl2va"
@@ -66,15 +44,15 @@ def _literal(value: Any) -> tuple[Any, bool]:
return {"type": f"{type(value).__module__}.{type(value).__qualname__}"}, False return {"type": f"{type(value).__module__}.{type(value).__qualname__}"}, False
def _node(prompt: dict[str, Any], node_id: str, *, allowed: set[str], stack: set[str]) -> tuple[dict[str, Any], bool, list[str]]: def _node(prompt: dict[str, Any], node_id: str, *, stack: set[str]) -> tuple[dict[str, Any], bool, list[str]]:
if node_id in stack: if node_id in stack:
return {"node_id": node_id, "error": "cycle"}, False, [f"upstream graph contains a cycle at node {node_id}"] return {"node_id": node_id, "error": "cycle"}, False, [f"upstream graph contains a cycle at node {node_id}"]
raw = prompt.get(str(node_id)) raw = prompt.get(str(node_id))
if not isinstance(raw, dict): if not isinstance(raw, dict):
return {"node_id": node_id, "error": "missing"}, False, [f"upstream node {node_id} is missing"] return {"node_id": node_id, "error": "missing"}, False, [f"upstream node {node_id} is missing"]
class_type = str(raw.get("class_type", "")) class_type = str(raw.get("class_type", ""))
safe = class_type in allowed or class_type in _KNOWN_SHARED_NODES safe = bool(class_type)
reasons = [] if safe else [f"unknown upstream node class: {class_type or '<missing>'}"] reasons = [] if safe else [f"upstream node {node_id} has no class_type"]
inputs = raw.get("inputs") or {} inputs = raw.get("inputs") or {}
if not isinstance(inputs, dict): if not isinstance(inputs, dict):
return {"node_id": node_id, "class_type": class_type, "error": "invalid inputs"}, False, reasons + [f"upstream node {node_id} inputs are invalid"] return {"node_id": node_id, "class_type": class_type, "error": "invalid inputs"}, False, reasons + [f"upstream node {node_id} inputs are invalid"]
@@ -91,7 +69,7 @@ def _node(prompt: dict[str, Any], node_id: str, *, allowed: set[str], stack: set
and str(value[0]) in prompt and str(value[0]) in prompt
): ):
child, child_safe, child_reasons = _node( child, child_safe, child_reasons = _node(
prompt, str(value[0]), allowed=allowed, stack=next_stack prompt, str(value[0]), stack=next_stack
) )
rendered_inputs[str(name)] = {"output": int(value[1]), "node": child} rendered_inputs[str(name)] = {"output": int(value[1]), "node": child}
safe = safe and child_safe safe = safe and child_safe
@@ -112,6 +90,7 @@ def _node(prompt: dict[str, Any], node_id: str, *, allowed: set[str], stack: set
def build_upstream_graph_contract( def build_upstream_graph_contract(
prompt: Any, unique_id: Any, *, require_video_vae: bool, prompt: Any, unique_id: Any, *, require_video_vae: bool,
require_reference_audio_vae: bool = False, require_reference_audio_vae: bool = False,
require_audio_vae: bool = False,
) -> tuple[dict[str, Any], bool, list[str]]: ) -> tuple[dict[str, Any], bool, list[str]]:
"""Fingerprint MODEL/CLIP/VAE routes feeding the Continuum sampler.""" """Fingerprint MODEL/CLIP/VAE routes feeding the Continuum sampler."""
if not isinstance(prompt, dict): if not isinstance(prompt, dict):
@@ -125,13 +104,15 @@ def build_upstream_graph_contract(
safe = True safe = True
reasons: list[str] = [] reasons: list[str] = []
specs = [ specs = [
("model", _KNOWN_MODEL_NODES, True), ("model", True),
("clip", _KNOWN_CLIP_NODES, True), ("clip", True),
("video_vae", _KNOWN_VAE_NODES, bool(require_video_vae)), ("video_vae", bool(require_video_vae)),
] ]
if require_reference_audio_vae: if require_reference_audio_vae:
specs.append(("reference_audio_vae", _KNOWN_VAE_NODES, True)) specs.append(("reference_audio_vae", True))
for name, allowed, required in specs: if require_audio_vae:
specs.append(("audio_vae", True))
for name, required in specs:
if not required: if not required:
continue continue
link = inputs.get(name) link = inputs.get(name)
@@ -146,7 +127,7 @@ def build_upstream_graph_contract(
reasons.append(f"{name} upstream connection is unavailable") reasons.append(f"{name} upstream connection is unavailable")
continue continue
route, route_safe, route_reasons = _node( route, route_safe, route_reasons = _node(
prompt, str(link[0]), allowed=allowed, stack={node_id} prompt, str(link[0]), stack={node_id}
) )
routes[name] = {"output": int(link[1]), "node": route} routes[name] = {"output": int(link[1]), "node": route}
safe = safe and route_safe safe = safe and route_safe
+39 -10
View File
@@ -36,8 +36,10 @@ def normalize_condition_latents(payload):
keyframes=list(payload.get("keyframes") or ()); refs=list(payload.get("refs") or ()) keyframes=list(payload.get("keyframes") or ()); refs=list(payload.get("refs") or ())
video_latents=[item["latent"] for item in keyframes if item.get("latent") is not None] video_latents=[item["latent"] for item in keyframes if item.get("latent") is not None]
video_latents.extend(item["latent"] for item in refs if item.get("latent") is not None) video_latents.extend(item["latent"] for item in refs if item.get("latent") is not None)
audio_latents=[item["audio_latent"] for item in keyframes if item.get("audio_latent") is not None]
audio_latents.extend(item["audio_latent"] for item in refs if item.get("audio_latent") is not None)
payload["cond_video_latents"]=video_latents payload["cond_video_latents"]=video_latents
payload["cond_audio_latents"]=[item["audio_latent"] for item in refs if item.get("audio_latent") is not None] payload["cond_audio_latents"]=audio_latents
def _input_device(x): def _input_device(x):
device=getattr(x,"device",None) device=getattr(x,"device",None)
if isinstance(device,torch.device): return device if isinstance(device,torch.device): return device
@@ -104,6 +106,24 @@ def _map_refs_to_segments(layout,refs):
result.append(mapping) result.append(mapping)
if cursor!=len(available): raise LayoutCompatibilityError(f"layout contains {len(available)-cursor} unmapped reference segments") if cursor!=len(available): raise LayoutCompatibilityError(f"layout contains {len(available)-cursor} unmapped reference segments")
return result return result
def _map_keyframes_to_segments(layout,keyframes):
available=[(int(a),int(b),str(kind)) for a,b,kind in layout.segments if kind in ("cond","cond_audio")]
cursor=0; result=[]
def consume(expected):
nonlocal cursor
if cursor>=len(available): raise LayoutCompatibilityError(f"layout ended while mapping keyframe '{expected}'")
a,b,kind=available[cursor]; cursor+=1
if kind!=expected: raise LayoutCompatibilityError(f"keyframe layout mismatch: expected '{expected}', found '{kind}'")
return a,b
for keyframe in keyframes:
mapping={"audio":None,"video":None}
if keyframe.get("latent") is not None: mapping["video"]=consume("cond")
if keyframe.get("audio_latent") is not None: mapping["audio"]=consume("cond_audio")
if mapping["video"] is None and mapping["audio"] is None:
raise LayoutCompatibilityError("H3 keyframe has neither video nor audio latent")
result.append(mapping)
if cursor!=len(available): raise LayoutCompatibilityError(f"layout contains {len(available)-cursor} unmapped keyframe segments")
return result
def _patch_signature(payload,layout): def _patch_signature(payload,layout):
return ( return (
tuple((int(kf.get("resolved_frame_index",-1)),int(kf.get(MARK_VIDEO_SLOT,-1))) for kf in (payload.get("keyframes") or ())), tuple((int(kf.get("resolved_frame_index",-1)),int(kf.get(MARK_VIDEO_SLOT,-1))) for kf in (payload.get("keyframes") or ())),
@@ -185,22 +205,31 @@ def patch_layout_in_place(payload,*,strict=True,debug=False):
original_time=getattr(layout,LAYOUT_ORIGINAL_TIME_ATTR) original_time=getattr(layout,LAYOUT_ORIGINAL_TIME_ATTR)
if not torch.is_tensor(original_time) or original_time.shape!=position_ids[:,0].shape: raise LayoutCompatibilityError("stored layout baseline is incompatible") if not torch.is_tensor(original_time) or original_time.shape!=position_ids[:,0].shape: raise LayoutCompatibilityError("stored layout baseline is incompatible")
position_ids[:,0].copy_(original_time) position_ids[:,0].copy_(original_time)
keyframes=list(payload.get("keyframes") or ()); refs=list(payload.get("refs") or ()); cond_segments=[(int(a),int(b)) for a,b,kind in layout.segments if kind=="cond"] keyframes=list(payload.get("keyframes") or ()); refs=list(payload.get("refs") or ()); keyframe_mappings=_map_keyframes_to_segments(layout,keyframes)
if len(cond_segments)!=len(keyframes): raise LayoutCompatibilityError(f"layout has {len(cond_segments)} cond segments for {len(keyframes)} keyframes")
video_start,video_stop=_single_segment(layout,"video"); _single_segment(layout,"audio") video_start,video_stop=_single_segment(layout,"video"); _single_segment(layout,"audio")
text_segments=[(int(a),int(b)) for a,b,kind in layout.segments if kind=="text"] text_segments=[(int(a),int(b)) for a,b,kind in layout.segments if kind=="text"]
if len(text_segments)!=1 or text_segments[0][0]!=0: raise LayoutCompatibilityError("unexpected H3 text segment") if len(text_segments)!=1 or text_segments[0][0]!=0: raise LayoutCompatibilityError("unexpected H3 text segment")
text_len=text_segments[0][1]; latent_t=int(layout.signature[1]); video_rows=video_stop-video_start text_len=text_segments[0][1]; latent_t=int(layout.signature[1]); video_rows=video_stop-video_start
if latent_t<=0 or video_rows%latent_t: raise LayoutCompatibilityError(f"video rows {video_rows} are incompatible with latent T={latent_t}") if latent_t<=0 or video_rows%latent_t: raise LayoutCompatibilityError(f"video rows {video_rows} are incompatible with latent T={latent_t}")
frame_rows=video_rows//latent_t; target_origin=float(position_ids[video_start,0]); reference_shift=target_origin-float(text_len) frame_rows=video_rows//latent_t; target_origin=float(position_ids[video_start,0])
patched_video_slots=0 patched_video_slots=0; patched_keyframe_audio=0
for (start,stop),keyframe in zip(cond_segments,keyframes): for keyframe,mapping in zip(keyframes,keyframe_mappings):
if stop-start!=frame_rows: raise LayoutCompatibilityError("keyframe rows no longer equal one target slot") frame_index=int(keyframe.get("resolved_frame_index",0))
if frame_index<0: raise LayoutCompatibilityError(f"negative H3 keyframe index {frame_index}")
desired_start=target_origin+FRAME_RESCALE*float(frame_index)
video_segment=mapping.get("video")
if MARK_VIDEO_SLOT in keyframe: if MARK_VIDEO_SLOT in keyframe:
if video_segment is None: raise LayoutCompatibilityError("legacy Continuum video slot has no cond rows")
start,stop=video_segment
if stop-start!=frame_rows: raise LayoutCompatibilityError("keyframe rows no longer equal one target slot")
slot=int(keyframe[MARK_VIDEO_SLOT]) slot=int(keyframe[MARK_VIDEO_SLOT])
if not (0<=slot<latent_t): raise LayoutCompatibilityError(f"context slot {slot} is outside target latent T={latent_t}") if not (0<=slot<latent_t): raise LayoutCompatibilityError(f"context slot {slot} is outside target latent T={latent_t}")
target_row=video_start+slot*frame_rows; position_ids[start:stop].copy_(position_ids[target_row:target_row+frame_rows]); patched_video_slots+=1 target_row=video_start+slot*frame_rows; position_ids[start:stop].copy_(position_ids[target_row:target_row+frame_rows]); patched_video_slots+=1
elif reference_shift: position_ids[start:stop,0].add_(reference_shift) elif video_segment is not None:
start,stop=video_segment; old_start=float(position_ids[start,0]); position_ids[start:stop,0].add_(desired_start-old_start)
audio_segment=mapping.get("audio")
if audio_segment is not None:
start,stop=audio_segment; old_start=float(position_ids[start,0]); position_ids[start:stop,0].add_(desired_start-old_start); patched_keyframe_audio+=1
ref_mappings=_map_refs_to_segments(layout,refs); patched_video_refs=0; patched_audio=0 ref_mappings=_map_refs_to_segments(layout,refs); patched_video_refs=0; patched_audio=0
for ref,mapping in zip(refs,ref_mappings): for ref,mapping in zip(refs,ref_mappings):
if _is_video_context(ref): if _is_video_context(ref):
@@ -222,5 +251,5 @@ def patch_layout_in_place(payload,*,strict=True,debug=False):
if strict: raise LayoutCompatibilityError(message) if strict: raise LayoutCompatibilityError(message)
LOG.warning(message) LOG.warning(message)
setattr(layout,LAYOUT_SIGNATURE_ATTR,signature) setattr(layout,LAYOUT_SIGNATURE_ATTR,signature)
if debug: LOG.info("Continuum layout: video_refs=%d legacy_slots=%d audio_windows=%d origin=%.6f rows=%d pos_id=%s",patched_video_refs,patched_video_slots,patched_audio,target_origin,position_ids.shape[0],id(position_ids)) if debug: LOG.info("Continuum layout: video_refs=%d legacy_slots=%d keyframe_audio=%d audio_windows=%d origin=%.6f rows=%d pos_id=%s",patched_video_refs,patched_video_slots,patched_keyframe_audio,patched_audio,target_origin,position_ids.shape[0],id(position_ids))
return {"status":"patched","video_contexts":patched_video,"video_refs":patched_video_refs,"legacy_video_slots":patched_video_slots,"audio_windows":patched_audio,"target_origin":target_origin,"position_ids_id":id(position_ids)} return {"status":"patched","video_contexts":patched_video,"video_refs":patched_video_refs,"legacy_video_slots":patched_video_slots,"keyframe_audio_windows":patched_keyframe_audio,"audio_windows":patched_audio,"target_origin":target_origin,"position_ids_id":id(position_ids)}
+1 -1
View File
@@ -1,4 +1,4 @@
[project] [project]
name = H3 Continuum name = H3 Continuum
version = 3.3.0 version = 3.4.0
description = Native MiniMax H3 continuation with Core VAE decode, Reference conditioning and crash-safe Run Storage. description = Native MiniMax H3 continuation with Core VAE decode, Reference conditioning and crash-safe Run Storage.
+16 -2
View File
@@ -37,7 +37,14 @@ def _wrapper_factory(*, strict: bool, debug: bool) -> Callable[..., Any]:
if not has_continuum and not has_mixed_keyframe_refs: if not has_continuum and not has_mixed_keyframe_refs:
return executor(*args, **kwargs) return executor(*args, **kwargs)
try: try:
ensure_native_h3_base_model(executor.class_obj) try:
ensure_native_h3_base_model(executor.class_obj)
except CompatibilityError as exc:
logging.getLogger(__name__).warning(
"H3 Continuum could not pre-classify this model; continuing with "
"ComfyUI runtime validation: %s",
exc,
)
patched_payload = dict(payload) patched_payload = dict(payload)
if has_continuum: if has_continuum:
patch_layout_in_place(patched_payload, strict=True, debug=debug) patch_layout_in_place(patched_payload, strict=True, debug=debug)
@@ -77,7 +84,14 @@ def configure_continuum_model(model: Any, *, strict: bool, debug: bool):
base_model = getattr(model, "model", None) base_model = getattr(model, "model", None)
if base_model is None: if base_model is None:
raise RuntimeError("MODEL does not expose the ComfyUI BaseModel") raise RuntimeError("MODEL does not expose the ComfyUI BaseModel")
ensure_native_h3_base_model(base_model) try:
ensure_native_h3_base_model(base_model)
except CompatibilityError as exc:
logging.getLogger(__name__).warning(
"H3 Continuum could not pre-classify this model; installing the wrapper "
"and deferring compatibility to ComfyUI runtime validation: %s",
exc,
)
patched = model patched = model
patched.remove_wrappers_with_key(WrappersMP.APPLY_MODEL, MODEL_WRAPPER_KEY) patched.remove_wrappers_with_key(WrappersMP.APPLY_MODEL, MODEL_WRAPPER_KEY)
patched.add_wrapper_with_key( patched.add_wrapper_with_key(
+11 -2
View File
@@ -181,6 +181,7 @@ class H3ContinuumJoin:
new_frame_count=shape.total_frames, new_frame_count=shape.total_frames,
first_frame_policy=first_frame_policy, first_frame_policy=first_frame_policy,
preserve_last_frame=bool(preserve_last_frame), preserve_last_frame=bool(preserve_last_frame),
source_frame_count=current_frames,
) )
clip_index = int(state.get("clip_index", 0)) + 1 clip_index = int(state.get("clip_index", 0)) + 1
plan = make_plan( plan = make_plan(
@@ -373,12 +374,20 @@ from .v3.nodes import (
NODE_CLASS_MAPPINGS.update(V3_NODE_CLASS_MAPPINGS) NODE_CLASS_MAPPINGS.update(V3_NODE_CLASS_MAPPINGS)
NODE_DISPLAY_NAME_MAPPINGS.update(V3_NODE_DISPLAY_NAME_MAPPINGS) NODE_DISPLAY_NAME_MAPPINGS.update(V3_NODE_DISPLAY_NAME_MAPPINGS)
from .v3.driving_nodes import (
NODE_CLASS_MAPPINGS as V34_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as V34_NODE_DISPLAY_NAME_MAPPINGS,
)
NODE_CLASS_MAPPINGS.update(V34_NODE_CLASS_MAPPINGS)
NODE_DISPLAY_NAME_MAPPINGS.update(V34_NODE_DISPLAY_NAME_MAPPINGS)
# The stable facade and its pack helpers are the public workflow surface. Keep # The stable facade and its pack helpers are the public workflow surface. Keep
# every older identifier registered so saved workflows still load, while # every older identifier registered so saved workflows still load, while
# ComfyUI's native deprecated-node filter hides legacy building blocks. # ComfyUI's native deprecated-node filter hides legacy building blocks.
_primary_node_ids = { _primary_node_ids = {
"H3ContinuumSamplerTimelineVideo", "H3ContinuumSamplerV34",
"H3ContinuumAssembleSeamExperimental", "H3ContinuumAssembleSeamV34",
"H3ContinuumClipOverrides", "H3ContinuumClipOverrides",
"H3ContinuumResult", "H3ContinuumResult",
} }
+1 -1
View File
@@ -1,6 +1,6 @@
[project] [project]
name = "comfyui-h3-continuum" name = "comfyui-h3-continuum"
version = "3.3.0" version = "3.4.0"
description = "Integrated N-chunk continuation for ComfyUI native MiniMax H3 video and audio" description = "Integrated N-chunk continuation for ComfyUI native MiniMax H3 video and audio"
readme = "README.md" readme = "README.md"
requires-python = ">=3.10" requires-python = ">=3.10"
+257
View File
@@ -0,0 +1,257 @@
"""Persistent MiniMax H3 video-reference conditioning for the V3.4 sampler."""
from __future__ import annotations
from dataclasses import dataclass
import hashlib
import json
import math
from typing import Any
import torch
REFERENCE_VIDEO_CONTRACT_VERSION = 1
REFERENCE_VIDEO_PREPROCESS_VERSION = 2
REFERENCE_VIDEO_SIZE_EFFICIENT = "Efficient - 0.4 MP"
REFERENCE_VIDEO_SIZE_BALANCED = "Balanced - 0.6 MP"
REFERENCE_VIDEO_SIZE_MATCH_OUTPUT = "Match Output"
REFERENCE_VIDEO_SIZE_OPTIONS = (
REFERENCE_VIDEO_SIZE_EFFICIENT,
REFERENCE_VIDEO_SIZE_BALANCED,
REFERENCE_VIDEO_SIZE_MATCH_OUTPUT,
)
_CANVAS_MULTIPLE = 32
_EFFICIENT_AREA = 400_000
_BALANCED_AREA = 600_000
_SOURCE_FPS = 24
class ReferenceVideoError(ValueError):
pass
def _canonical_hash(value: Any) -> str:
payload = json.dumps(
value, sort_keys=True, separators=(",", ":"), ensure_ascii=True
).encode("utf-8")
return hashlib.sha256(payload).hexdigest()
def _tensor_hash(value: torch.Tensor) -> str:
tensor = value.detach().to(device="cpu").contiguous()
digest = hashlib.sha256()
digest.update(str(tuple(int(v) for v in tensor.shape)).encode("ascii"))
digest.update(str(tensor.dtype).encode("ascii"))
digest.update(tensor.numpy().tobytes(order="C"))
return digest.hexdigest()
def _round_canvas(value: float) -> int:
return max(
_CANVAS_MULTIPLE,
int(round(float(value) / _CANVAS_MULTIPLE)) * _CANVAS_MULTIPLE,
)
def _resolved_size(
source_width: int,
source_height: int,
*,
output_width: int,
output_height: int,
size_mode: str,
) -> tuple[int, int]:
if size_mode == REFERENCE_VIDEO_SIZE_MATCH_OUTPUT:
target_area = int(output_width) * int(output_height)
elif size_mode == REFERENCE_VIDEO_SIZE_BALANCED:
target_area = _BALANCED_AREA
else:
target_area = _EFFICIENT_AREA
source_area = int(source_width) * int(source_height)
scale = min(1.0, math.sqrt(float(target_area) / float(source_area)))
target_width = _round_canvas(int(source_width) * scale)
target_height = _round_canvas(int(source_height) * scale)
if source_width >= _CANVAS_MULTIPLE:
target_width = min(
target_width,
max(_CANVAS_MULTIPLE, (int(source_width) // _CANVAS_MULTIPLE) * _CANVAS_MULTIPLE),
)
if source_height >= _CANVAS_MULTIPLE:
target_height = min(
target_height,
max(_CANVAS_MULTIPLE, (int(source_height) // _CANVAS_MULTIPLE) * _CANVAS_MULTIPLE),
)
return target_width, target_height
@dataclass(frozen=True)
class ReferenceVideoSource:
frames: torch.Tensor
source_shape: tuple[int, ...]
source_dtype: str
source_sha256: str
size_mode: str
target_width: int
target_height: int
frame_count: int
combined_hash: str
@property
def contract(self) -> dict[str, Any]:
return {
"reference_video_contract_version": REFERENCE_VIDEO_CONTRACT_VERSION,
"source_shape": list(self.source_shape),
"source_dtype": self.source_dtype,
"source_sha256": self.source_sha256,
"source_fps": _SOURCE_FPS,
"size_mode": self.size_mode,
"target_width": self.target_width,
"target_height": self.target_height,
"frame_count": self.frame_count,
"preprocess_version": REFERENCE_VIDEO_PREPROCESS_VERSION,
"combined_hash": self.combined_hash,
}
@dataclass(frozen=True)
class ReferenceVideoAssets:
source: ReferenceVideoSource
item: dict[str, Any]
block: dict[str, Any]
def prepare_reference_video_source(
reference_video_1: Any,
*,
target_frames: int,
output_width: int,
output_height: int,
size_mode: str = REFERENCE_VIDEO_SIZE_EFFICIENT,
) -> ReferenceVideoSource | None:
if reference_video_1 is None:
return None
if not torch.is_tensor(reference_video_1) or reference_video_1.ndim != 4:
raise ReferenceVideoError(
"Video Reference must be a ComfyUI IMAGE batch with shape [T, H, W, C]"
)
if int(reference_video_1.shape[-1]) < 3:
raise ReferenceVideoError("Video Reference must contain RGB channels")
frames = reference_video_1[:, :, :, :3].detach().to(
device="cpu", dtype=torch.float32
).contiguous()
if not bool(torch.isfinite(frames).all().item()):
raise ReferenceVideoError("Video Reference contains NaN or infinity")
source_shape = tuple(int(value) for value in frames.shape)
source_sha256 = _tensor_hash(frames)
frame_count = min(int(frames.shape[0]), int(target_frames))
if frame_count < 5:
raise ReferenceVideoError("Video Reference requires at least 5 frames")
while frame_count > 5 and frame_count % 17 != 5:
frame_count -= 1
normalized_size_mode = (
str(size_mode)
if str(size_mode) in REFERENCE_VIDEO_SIZE_OPTIONS
else REFERENCE_VIDEO_SIZE_EFFICIENT
)
target_width, target_height = _resolved_size(
int(frames.shape[2]),
int(frames.shape[1]),
output_width=int(output_width),
output_height=int(output_height),
size_mode=normalized_size_mode,
)
contract_base = {
"reference_video_contract_version": REFERENCE_VIDEO_CONTRACT_VERSION,
"source_shape": list(source_shape),
"source_dtype": str(frames.dtype),
"source_sha256": source_sha256,
"source_fps": _SOURCE_FPS,
"size_mode": normalized_size_mode,
"target_width": target_width,
"target_height": target_height,
"frame_count": frame_count,
"preprocess_version": REFERENCE_VIDEO_PREPROCESS_VERSION,
}
return ReferenceVideoSource(
frames=frames,
source_shape=source_shape,
source_dtype=str(frames.dtype),
source_sha256=source_sha256,
size_mode=normalized_size_mode,
target_width=target_width,
target_height=target_height,
frame_count=frame_count,
combined_hash=_canonical_hash(contract_base),
)
def encode_reference_video(
video_vae: Any,
source: ReferenceVideoSource,
) -> ReferenceVideoAssets:
frames = source.frames[: source.frame_count]
if (
int(frames.shape[2]) != source.target_width
or int(frames.shape[1]) != source.target_height
):
try:
import comfy.utils
except Exception as exc:
raise ReferenceVideoError(
"ComfyUI Core image resize support is unavailable"
) from exc
frames = comfy.utils.common_upscale(
frames.movedim(-1, 1),
source.target_width,
source.target_height,
"lanczos",
"disabled",
).movedim(1, -1).contiguous()
try:
latent = video_vae.encode(frames).detach().to("cpu").contiguous()
except Exception as exc:
raise ReferenceVideoError("Video Reference VAE Encode failed") from exc
qwen_frames = frames[::_SOURCE_FPS // 2].contiguous()
item = {
"type": "video",
"data": qwen_frames,
"timestamps": [index / 2.0 for index in range(int(qwen_frames.shape[0]))],
}
block = {
"kind": "video",
"latent_t": int(latent.shape[2]),
"latent_h": source.target_height // 16,
"latent_w": source.target_width // 16,
"ref_audio_t": 0,
"latent": latent,
"audio_latent": None,
}
return ReferenceVideoAssets(source=source, item=item, block=block)
def combine_reference_video_identity(
visual_identity_hash: str,
source: ReferenceVideoSource | None,
) -> str:
if source is None:
return str(visual_identity_hash)
return _canonical_hash(
{
"reference_video_identity_version": 1,
"visual_identity_hash": str(visual_identity_hash),
"reference_video_hash": source.combined_hash,
}
)
def validate_reference_video_prompts(
prompts: list[str], source: ReferenceVideoSource | None
) -> str:
if source is None or any("<Video 1>" in str(prompt) for prompt in prompts):
return ""
return (
"H3C-P103 Warning: Video Reference is connected but the prompt contains no "
"<Video 1> tag; video still conditions generation, but an explicit tag is "
"recommended."
)
+1 -1
View File
@@ -1,6 +1,6 @@
"""Package version and public schema identifiers.""" """Package version and public schema identifiers."""
PACKAGE_VERSION = "3.3.0" PACKAGE_VERSION = "3.4.0"
STATE_SCHEMA_VERSION = 1 STATE_SCHEMA_VERSION = 1
PLAN_SCHEMA_VERSION = 1 PLAN_SCHEMA_VERSION = 1
PUBLIC_API_VERSION = 3 PUBLIC_API_VERSION = 3