Release V2.1.4 UI simplification

This commit is contained in:
ukr8b3g-cmyk
2026-08-10 16:28:20 +09:00
commit 83590a3935
76 changed files with 10956 additions and 0 deletions
+177
View File
@@ -0,0 +1,177 @@
# Architecture — H3 Continuum Join 2.0
## 1. Design goals
1. Preserve the V1 native H3 continuation behavior.
2. Integrate N chunks without duplicating V1 temporal/layout logic.
3. Keep SageAttention, Sol-Attn, and Spectrum external and independently updatable.
4. Avoid global ComfyUI class mutation.
5. Fail closed when native H3 layout contracts change.
6. Prevent H3/VAE/CLIP model switching between continuation chunks.
7. Keep accepted chunk latents on CPU and bound transient decode memory.
8. Preserve exact long-chain A/V timing.
## 2. Module layers
```text
UI
nodes.py V1 compatibility nodes
v2/nodes.py integrated V2 nodes
V2 orchestration
v2/sequence.py phases, session reuse, reroll, reports
v2/sampling.py ComfyUI-native sampler path
v2/h3_builder.py empty AV latent, prompt and keyframe conditioning
v2/decoder.py deferred decode, preallocated assembly
v2/prompts.py Fixed/List/Timeline parser
v2/session.py in-memory accepted chunk/session model
v2/session_io.py atomic session persistence
v2/motion.py conservative latent motion analysis
v2/seeds.py deterministic per-chunk seed derivation
v2/diagnostics.py lightweight seam measurements
Shared continuation core
continuation.py native H3 video/video_audio reference construction
model_patch.py per-MODEL APPLY_MODEL wrapper
layout_adapter.py in-place MM-RoPE placement and payload normalization
temporal.py H3 17k+5 / 24fps / 40Hz grid
state.py V1 State and Plan schemas
media.py V1 trim/assembly helpers
compatibility.py runtime contract checks
```
V2 calls the shared continuation core. It does not implement a second continuation algorithm.
## 3. Three execution phases
### Phase 1 — conditioning preparation
- Resize first/last images once.
- VAE-encode their keyframe latents once.
- Tokenize and encode each unique Prompt once.
- Clone the externally patched MODEL once and add the Continuum APPLY_MODEL wrapper.
No H3 diffusion sampling begins until this phase completes.
### Phase 2 — H3-only loop
For each chunk:
1. Create native H3 NestedTensor AV latent.
2. Select 5/22/39-frame Context or Auto.
3. Add one native `video` / `video_audio` reference block.
4. Call the standard ComfyUI guider/sampler path.
5. Let Spectrum create and tear down its own run through its public wrappers.
6. Copy the sampled full AV latent to CPU once.
7. Derive the next V1-compatible State from that CPU chunk.
Video and Audio VAE decode are deliberately absent from this loop.
### Phase 3 — deferred decode
- Decode each CPU chunk after all H3 sampling.
- Immediately move decoded tensors to CPU.
- Remove the overlap.
- Copy images into one preallocated final IMAGE tensor.
- Place audio into cumulative 24fps-derived sample ranges.
- Release each raw chunk before decoding the next.
- Apply exact final duration if enabled.
## 4. Native H3 continuation representation
V1 and V2 both represent the previous tail as one reference block:
```text
kind: video or video_audio
latent: native H3 video latent tail
latent_t/h/w: native geometry
audio_latent: optional native H3 audio latent tail
ref_audio_t: audio latent length
_h3cj_* metadata: timeline placement contract
```
The outer APPLY_MODEL wrapper receives the payload built by ComfyUI, moves only marked Continuum reference rows onto the target timeline, and normalizes keyframe/reference latent lists.
## 5. Accelerator composition
### SageAttention
Continuum never replaces the attention kernel. Existing attention overrides remain intact.
### Sol-Attn
Sol-Attn registers H3 target spans by the identity of `PackedLayout.position_ids`. Continuum modifies this tensor in place and never replaces it. Sol conditioning sink and Morton tracking therefore retain their registration.
### Spectrum
Spectrum remains an external OUTER_SAMPLE / PREDICT_NOISE / SAMPLER_SAMPLE / DIFFUSION_MODEL wrapper stack. V2 calls the normal ComfyUI `guider.sample()` once per chunk. Spectrum starts and ends an independent runtime run for each call. Continuum imports no Spectrum private functions.
The recommended model chain is:
```text
UNET → Sage → Sol → LoRA → Spectrum → V2 Sampler
```
V2 then clones this ModelPatcher once; Spectrum's ON_CLONE callback creates an independent runtime binding on the clone.
## 6. State and Session
### V1 State
Contains only the maximum retained continuation tail and temporal metadata. It can start V2 through `initial_state`.
### V2 Session
Contains all accepted raw chunk AV latents plus:
- Prompt and SHA-256 Prompt hash
- per-chunk Seed
- Continuum Plan
- selected Context frames and motion score
- model/accelerator fingerprint
- branch parent/session identifiers
Tensors remain on CPU. Save Session writes safetensors and a JSON mirror atomically. The authoritative metadata is embedded in safetensors.
## 7. Reroll semantics
- `reroll_from_chunk=0`: reuse every matching accepted prefix.
- `reroll_from_chunk=N`: preserve chunks before N and regenerate N onward.
- A Prompt mismatch automatically stops reuse before the changed chunk.
- A first-frame fingerprint mismatch fails explicitly.
- A model/accelerator fingerprint mismatch is reported but accepted chunks remain fixed, because their latent output is already approved.
- The new output creates a child Session; the input Session is not mutated.
## 8. Audio timeline
H3 video is 24fps and audio latent is 40Hz. Decoded PCM is placed using cumulative integer boundaries:
```text
start_sample = round(cumulative_frames / 24 * sample_rate)
stop_sample = round(next_cumulative_frames / 24 * sample_rate)
```
This avoids the one-sample drift that can occur when every 119-frame segment is rounded independently.
## 9. Memory model
Full accepted AV latents are comparatively small and stored on CPU. The final decoded IMAGE is large, so V2:
- estimates final float32 IMAGE + one raw chunk + headroom before sampling;
- refuses unsafe jobs before expensive H3 work;
- allocates the final IMAGE exactly once;
- copies each decoded segment into its final range;
- avoids progressive whole-sequence concatenation.
## 10. Compatibility policy
The package checks:
- native `MiniMaxH3Model` class and required attributes;
- `PackedLayout` capability, including transparent `*args/**kwargs` wrappers;
- H3 temporal grid `(1,4,4,4,4)`;
- APPLY_MODEL and DIFFUSION_MODEL wrapper APIs;
- NestedTensor support;
- real PackedLayout row placement through the installer self-test.
Marked Continuum payload failures never silently fall back to stock reference placement, because that could render a plausible but semantically wrong join.
+16
View File
@@ -0,0 +1,16 @@
# Future roadmap after V2
V2 is the integrated production path. Future changes must preserve V1/V2 schemas and remain optional.
Possible additions:
- streaming decode/save that does not retain the complete final IMAGE tensor;
- interactive per-chunk approval UI with a frontend timeline;
- persisted multiple child branches and branch comparison;
- richer identity diagnostics using optional external encoders;
- adaptive Spectrum seam policy only if Spectrum exposes a stable public hook;
- reference-video / Ref2VA identity-memory mode;
- multi-resolution continuation through explicit regeneration, never silent resize;
- batch-independent server/API orchestration.
These are not promises for a specific release. The current V2 avoids private accelerator APIs so updates remain maintainable.
+35
View File
@@ -0,0 +1,35 @@
# V2.1.4 UI Simplification
The integrated `H3 Continuum Sampler V2` is the primary workflow node.
## Default workflow
```text
Text (Multiline) -> Sequence Prompt
Image -> first_frame
Sampler images/audio -> Create Video
```
The default node surface shows Sequence Prompt, first frame, chunk timing,
continuity, seed, and seam correction. Individual clip overrides and Advanced
settings are collapsed.
## Advanced settings
Advanced settings expose `last_frame`, `initial_state`, `session`,
`prompt_plan`, reroll controls, diagnostics, `last_state`, session output, and
report output. A saved workflow with an existing Advanced connection expands
the section automatically.
The Advanced Settings button remains the section header when expanded. The
button is placed before the expanded controls instead of moving below them.
## Legacy nodes
V1 Join, Finish, Assemble, state utilities, and auxiliary V2 plan/session nodes
remain registered for saved-workflow compatibility. They are marked deprecated
and moved to `MiniMax H3/Continuum/Legacy`, so ComfyUI can omit them from the
normal node picker.
No sampling, State/Session schema, output order, or accelerator behavior changes
in V2.1.4.
+150
View File
@@ -0,0 +1,150 @@
# H3 Continuum Join 2.0 — 実機検証
## 1. インストール検査
ComfyUIを終了後、展開フォルダーで実行します。
```powershell
.\install_windows.bat
```
または:
```powershell
D:\StabilityMatrix\Data\Packages\ComfyUI_W\venv\Scripts\python.exe `
.\tools\verify_runtime.py `
--comfy-root D:\StabilityMatrix\Data\Packages\ComfyUI_W
```
確認項目:
- native H3 API
- Sol-Attn形式の`*args/**kwargs` PackedLayout wrapper互換
- real PackedLayout行構造とin-place座標補正
- V1/V2計10ノード登録
- Fixed 3×5秒Prompt Plan
## 2. 最初のV2基準生成
```text
chunks: 3
chunk_seconds: 5.0
continuity: Balanced — 22 frames
audio_continuity: true
exact_total_duration: true
diagnostics: Basic
reroll_from_chunk: 0
reroll_nonce: 0
strict_compatibility: true
```
Model:
```text
H3 INT8 ConvRot
20 steps / CFG 1
Turbo LoRA OFF
```
同じ入力画像とPromptで、既存V1の3×5秒ワークフローとV2を比較します。
## 3. アクセラレータ比較
同じBase Seed、Prompt、解像度で次を順番に確認します。
1. SageAttentionのみ
2. Sage + Sol-Attn
3. Sage + Spectrum
4. Sage + Sol-Attn + Spectrum
RTX 5060 Ti環境で既に動作したSage backendを基準にします。クラッシュしたbackendへ切り替えません。
確認:
- 全チャンクが順番に完了
- Spectrumのrunが各チャンクで開始・終了
- `Sol-Attn takes first refusal...`ログ
- 黒出力、NaN、CUDA errorなし
- VAE Decodeが全H3 Sampling後に始まる
- 最終出力が指定尺になる
## 4. 接続品質
Basic確認:
- 人物ID、髪、服、背景
- 前チャンクからの姿勢・移動方向・カメラ速度
- 口形、歌唱・発話の継続
- 音声tick、無音、語尾重複
- 最終フレーム数と音声samples
Full DiagnosticsではReportに次を出します。
```text
video overlap MAE
video overlap PSNR
audio overlap correlation
audio boundary jump
```
絶対的な合格値ではなく、V1と各アクセラレータ構成の相対比較に使います。
## 5. Session再開
1. 3チャンク生成
2. `Save Session slot 1`
3. ComfyUI再起動
4. `Load Session slot 1`
5. chunksを4へ変更し、sessionへ接続
6. Chunk 1〜3がreused、Chunk 4だけSamplingされることをReportで確認
## 6. リロール
```text
reroll_from_chunk: 3
reroll_nonce: 1
```
確認:
- Chunk 1〜2はSamplingされない
- Chunk 3以降だけ新Seed
- 入力Sessionファイルは変更されない
- 出力Sessionの`parent_session_id`が入力Sessionを示す
## 7. Promptモード
- Fixed:Qwen encodeが1回
- List:`---`で3 Prompt
- Timeline:`[0-5s] / [5-10s] / [10-15s]`
- Prompt変更時、変更前チャンクだけSession再利用される
## 8. Auto Context
Balanced 22fを品質基準にします。その後Autoと比較します。
- 静かな動き:5fを選ぶ可能性
- 通常:22f
- 強いlatent motionかつ39f State:39f
Autoの結果が悪い内容ではBalancedへ固定します。
## 9. RAM guard
高解像度・多数チャンクで、Sampling前にRAM見積りエラーが出ることを確認します。これは意図した安全停止です。chunks、秒数、解像度を下げます。
## 10. 報告時に必要な情報
```text
ComfyUI version / revision
Python / Torch / CUDA
GPU / VRAM / RAM
H3 checkpoint
Sage backend
Sol settings
Spectrum settings
chunks / seconds / context
steps / scheduler / sampler
Console log
V2 report
```