# Architecture — H3 Continuum Join 2.0 ## 1. Design goals 1. Preserve the V1 native H3 continuation behavior. 2. Integrate N chunks without duplicating V1 temporal/layout logic. 3. Keep SageAttention, Sol-Attn, and Spectrum external and independently updatable. 4. Avoid global ComfyUI class mutation. 5. Fail closed when native H3 layout contracts change. 6. Prevent H3/VAE/CLIP model switching between continuation chunks. 7. Keep accepted chunk latents on CPU and bound transient decode memory. 8. Preserve exact long-chain A/V timing. ## 2. Module layers ```text UI nodes.py V1 compatibility nodes v2/nodes.py integrated V2 nodes V2 orchestration v2/sequence.py phases, session reuse, reroll, reports v2/sampling.py ComfyUI-native sampler path v2/h3_builder.py empty AV latent, prompt and keyframe conditioning v2/decoder.py deferred decode, preallocated assembly v2/prompts.py Fixed/List/Timeline parser v2/session.py in-memory accepted chunk/session model v2/session_io.py atomic session persistence v2/motion.py conservative latent motion analysis v2/seeds.py deterministic per-chunk seed derivation v2/diagnostics.py lightweight seam measurements Shared continuation core continuation.py native H3 video/video_audio reference construction model_patch.py per-MODEL APPLY_MODEL wrapper layout_adapter.py in-place MM-RoPE placement and payload normalization temporal.py H3 17k+5 / 24fps / 40Hz grid state.py V1 State and Plan schemas media.py V1 trim/assembly helpers compatibility.py runtime contract checks ``` V2 calls the shared continuation core. It does not implement a second continuation algorithm. ## 3. Three execution phases ### Phase 1 — conditioning preparation - Resize first/last images once. - VAE-encode their keyframe latents once. - Tokenize and encode each unique Prompt once. - Clone the externally patched MODEL once and add the Continuum APPLY_MODEL wrapper. No H3 diffusion sampling begins until this phase completes. ### Phase 2 — H3-only loop For each chunk: 1. Create native H3 NestedTensor AV latent. 2. Select 5/22/39-frame Context or Auto. 3. Add one native `video` / `video_audio` reference block. 4. Call the standard ComfyUI guider/sampler path. 5. Let Spectrum create and tear down its own run through its public wrappers. 6. Copy the sampled full AV latent to CPU once. 7. Derive the next V1-compatible State from that CPU chunk. Video and Audio VAE decode are deliberately absent from this loop. ### Phase 3 — deferred decode - Decode each CPU chunk after all H3 sampling. - Immediately move decoded tensors to CPU. - Remove the overlap. - Copy images into one preallocated final IMAGE tensor. - Place audio into cumulative 24fps-derived sample ranges. - Release each raw chunk before decoding the next. - Apply exact final duration if enabled. ## 4. Native H3 continuation representation V1 and V2 both represent the previous tail as one reference block: ```text kind: video or video_audio latent: native H3 video latent tail latent_t/h/w: native geometry audio_latent: optional native H3 audio latent tail ref_audio_t: audio latent length _h3cj_* metadata: timeline placement contract ``` The outer APPLY_MODEL wrapper receives the payload built by ComfyUI, moves only marked Continuum reference rows onto the target timeline, and normalizes keyframe/reference latent lists. ## 5. Accelerator composition ### SageAttention Continuum never replaces the attention kernel. Existing attention overrides remain intact. ### Sol-Attn Sol-Attn registers H3 target spans by the identity of `PackedLayout.position_ids`. Continuum modifies this tensor in place and never replaces it. Sol conditioning sink and Morton tracking therefore retain their registration. ### Spectrum Spectrum remains an external OUTER_SAMPLE / PREDICT_NOISE / SAMPLER_SAMPLE / DIFFUSION_MODEL wrapper stack. V2 calls the normal ComfyUI `guider.sample()` once per chunk. Spectrum starts and ends an independent runtime run for each call. Continuum imports no Spectrum private functions. The recommended model chain is: ```text UNET → Sage → Sol → LoRA → Spectrum → V2 Sampler ``` V2 then clones this ModelPatcher once; Spectrum's ON_CLONE callback creates an independent runtime binding on the clone. ## 6. State and Session ### V1 State Contains only the maximum retained continuation tail and temporal metadata. It can start V2 through `initial_state`. ### V2 Session Contains all accepted raw chunk AV latents plus: - Prompt and SHA-256 Prompt hash - per-chunk Seed - Continuum Plan - selected Context frames and motion score - model/accelerator fingerprint - branch parent/session identifiers Tensors remain on CPU. Save Session writes safetensors and a JSON mirror atomically. The authoritative metadata is embedded in safetensors. ## 7. Reroll semantics - `reroll_from_chunk=0`: reuse every matching accepted prefix. - `reroll_from_chunk=N`: preserve chunks before N and regenerate N onward. - A Prompt mismatch automatically stops reuse before the changed chunk. - A first-frame fingerprint mismatch fails explicitly. - A model/accelerator fingerprint mismatch is reported but accepted chunks remain fixed, because their latent output is already approved. - The new output creates a child Session; the input Session is not mutated. ## 8. Audio timeline H3 video is 24fps and audio latent is 40Hz. Decoded PCM is placed using cumulative integer boundaries: ```text start_sample = round(cumulative_frames / 24 * sample_rate) stop_sample = round(next_cumulative_frames / 24 * sample_rate) ``` This avoids the one-sample drift that can occur when every 119-frame segment is rounded independently. ## 9. Memory model Full accepted AV latents are comparatively small and stored on CPU. The final decoded IMAGE is large, so V2: - estimates final float32 IMAGE + one raw chunk + headroom before sampling; - refuses unsafe jobs before expensive H3 work; - allocates the final IMAGE exactly once; - copies each decoded segment into its final range; - avoids progressive whole-sequence concatenation. ## 10. Compatibility policy The package checks: - native `MiniMaxH3Model` class and required attributes; - `PackedLayout` capability, including transparent `*args/**kwargs` wrappers; - H3 temporal grid `(1,4,4,4,4)`; - APPLY_MODEL and DIFFUSION_MODEL wrapper APIs; - NestedTensor support; - real PackedLayout row placement through the installer self-test. Marked Continuum payload failures never silently fall back to stock reference placement, because that could render a plausible but semantically wrong join.