Files
DumasNodes/BUNDLE_FORMAT.md
T
chris.dumas 70d0672ee3 Squashed 'vendor/dumas_minimax_h3_refmod/' content from commit 517e8c4
git-subtree-dir: vendor/dumas_minimax_h3_refmod
git-subtree-split: 517e8c419a8584fbf69fceb487bc39d83e6a6ec8
2026-10-20 13:43:44 +00:00

83 lines
3.3 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# RefMod single-file bundle format
Version 5 is a container of independent H3 references. It does not concatenate
audio with visual latents or change H3 conditioning semantics. Existing
standalone version-4 files remain supported and keep their `latent` tensor key.
## Detection and layout
Read JSON from the safetensors header's `refmod_meta` key without loading tensors.
A bundle has `_format_version: 5`, `kind: "bundle"`, a display `name`, and an
ordered `members` array containing 1–256 ordinary RefMod metadata objects.
Nested bundles are not supported. A member's `kind` is `image`, `video`, or `audio`.
```json
{
"_format_version": 5,
"kind": "bundle",
"name": "hero",
"members": [
{
"_format_version": 4,
"name": "hero_visual",
"kind": "video",
"latent_t": 3,
"latent_h": 32,
"latent_w": 32
},
{
"_format_version": 4,
"name": "hero_audio",
"kind": "audio",
"latent_t": 200,
"sample_rate": 32000
}
]
}
```
This example omits optional member metadata. Each member keeps the usual mode,
source, tags, description, concept type and optional `refmod_config` JSON string.
Tensor `ref_0` belongs to member 0, `ref_1` to member 1, and so on.
| Kind | Tensor layout | Tokens |
| --- | --- | --- |
| image | `[1,24,1,H,W]` | `(H/2) × (W/2)` |
| video | `[1,24,T,H,W]` | `T × (H/2) × (W/2)` |
| audio | `[1,32,2,T]` | `2 × T` |
Visual H/W must be positive and even; T must be positive. Audio uses the existing
32 kHz H3 codec convention. Each member retains its own latent dimensions.
## Picker and loader integration
1. Detect `kind == "bundle"` and validate the supported format version.
2. Inspect members to display whether the file contains visuals, audio, or both.
3. Filter by **All**, **Visual** (`image` + `video`), or **Audio** before reading
tensor data. Skip members whose modality strength is zero.
4. Load the selected `ref_i` tensors and expand them into ordinary
`(H3RefMod, strength)` entries, preserving relative member order.
5. Multiply the slot strength by the modality strength. Account for each selected
member's token cost and runtime copies.
In this pack, `bundle.load_bundle(path_without_extension, selection,
visual_strength, audio_strength)` performs selection and returns ordinary
entries with modality strengths. The caller applies the slot strength/copies.
Members retain their container `path` and `bundle_index` for Config updates.
Save writes each distinct reference object once; runtime strengths and duplicate
copies are not stored. Repacking a loaded bundle flattens its selected members.
Config updates preserve other members and unknown container metadata/tensors.
## Compatibility and limits
New readers load older standalone files. Older readers expecting only a `latent`
tensor will need bundle support; there is no promise of forward compatibility.
The standalone Save node remains available for exporting members.
The container carries encoded references, not original audio bytes, explicit
cross-modal timestamps, or speaker-to-character assignments. Shared storage does
not guarantee synchronized generation or stop multiple references from mixing.
Per-slot Audio strength affects all audio members in that slot; this initial UI
does not select individual audio tracks within a multi-audio file.