git-subtree-dir: vendor/dumas_minimax_h3_refmod git-subtree-split: 517e8c419a8584fbf69fceb487bc39d83e6a6ec8
14 KiB
Changelog
All notable changes are tracked here. Each version is also published as a GitHub Release, so you can keep using an older version if a new one changes something you rely on.
Unreleased
-
Correct visual reference sampling to H3's 17k+5 video grid in the node and CLI. Preserve all frames of aligned clips and both endpoints when uniformly sampling; references/caps below five frames use the first image.
-
Support MPS-resident RefMod latents without global CPU fallback: pool and blur explicitly on CPU, preserving return device, dtype and autograd. Synchronize MPS in the refinement timing harness.
-
Add a reference_map output to Inspect H3 RefMod for LLM prompt generation before Text Encode, using the same reference numbering (#19).
-
Fix Loader/Axis settings resetting when switching workflow tabs or reopening workflows: defer slot regrouping until widget restoration finishes (#18).
-
Repair Apply's reactive MatchType socket on older frontends whose canvas socket classes use private fields, preventing missing controls while preserving conditioning type matching (#17).
v0.2.6 — 2026-09-12
- Make Loader/Axis additive: start with one visible slot, add/remove slots up to eight, group modality controls with their reference, preserve old workflow values and protect connected slots.
- Add version-5 single-file RefMod bundles containing independent image/video/audio members, with atomic saves and preserved member order.
- Add Save H3 RefMod Bundle as an output node and opt-in Master save_layout=bundle. Keep standalone saving and existing workflow defaults.
- Add All/Visual/Audio selection and separate modality strengths to every Loader/Axis slot. Skip excluded tensors and count only selected references in token budgets.
- Discover combined files in the library; preserve unselected members during Config updates and support export to standalone files.
- Document the container schema for third-party pickers. Bundling does not add voice binding, AV synchronization, or original-waveform passthrough.
v0.2.5 — 2026-09-08
- Rename Extract nodes to Create, with unchanged node IDs. Full Reference and Compressed Reference replace the visible encode/training names; old mode values remain accepted. Rename the identity control to Refinement Steps.
- Add visual budget_policy: truncate preserves existing behavior; error stops before saving when the visual token budget is exceeded. Master inherits the option.
- Allow larger token/frame limits and preserve the requested frame count in the motion preset. Loader budgets default to 0 (disabled).
- Add experimental H3 RefMod Text Encode with numbered reference presentation and a reference map. Support compatible ClipProj encoders and native batched video-VAE output.
- Add optional full-video inspection with correct IMAGE output dimensions; retain the lightweight first-frame default.
- Add playable music/video README examples and exclude local tests from distribution.
Validation: 51 local regression tests passed; creation-label hooks checked with Node.js. Full GPU generation, projected-encoder quality and voice transfer remain unverified. Voice cloning is not claimed to work.
Text Encode already attaches references: do not inject the same bundle again with Apply/Bridge. Integrated Continuum samplers still need a dedicated integration to use this conditioning. Full visual previews and text/vision presentation increase memory use.
v0.2.0 — 2026-09-07
Known limitation: voice transfer
Audio reference support includes an observed music-transfer example, but current speaker-identity tests failed. This release does not provide working voice cloning. The cause has not been isolated between H3/checkpoint behavior and the reference presentation difference in the current RefMod integration.
Follow-up implementation (2026-09-07)
-
Clarify Master saving logs: deferred extraction, Created/Replaced destinations, explicit save=False and a final token/file summary; report saved_paths in details.
-
Add Save H3 RefMods as an output node: save mixed bundles without connecting a Preview or sampler, with prefix/subfolder options and copy deduplication.
-
Save new mods/presets in the first registered refmods root (extra_model_paths); keep the standard models/refmods fallback and existing-file Config locations.
-
Remove sibling-pack imports and startup dependency warning. Legacy av_encoder inputs use native ComfyUI video VAE loading; standard VAE inputs are preferred.
-
Add Extract H3 RefMod Master: optional image/video and audio inputs, sequential extraction through the shared extractors, combined token budget, one bundle output and separate compatible visual/audio files under a shared name.
-
Step Curve now mixes the current payload without retaining tensors and handles keyframe offsets, audio refs, empty conditioning and chained curves.
-
Bridge uses the MODEL outer-sampling hook with finally cleanup; removes global state and third-party monkey patches. The old Disarm node is deprecated.
-
Align image/mask center crops; share resize and causal-frame helpers with CLI.
-
Invalidate loaded files and queue cache on changes; cap cache at 256 MiB; enforce copies/numeric limits; bound video decode buffers with unknown lengths.
-
Add native audio RefMods, legacy AudioMod import, audio Extract, library UI, inspection/optional previews, save subfolders, explicit scramble modes, extraction presets and total-token budgets.
-
Atomic saves, deduplicated config writes and grouped mean-MSE refinement.
-
Historical entries below describe earlier unreleased revisions; the behavior above and the current README supersede their bridge/cache descriptions.
Fixed: linked loader inputs and subfolders
- Both loaders accept unresolved/empty mod names and connected STRING/combo outputs. Fixed missing names still fail validation; connected names resolve at execution with a missing-file error when necessary.
- Discover RefMods recursively as
folder/name, preserving metadata filtering and skipping graph presets. Listing cache tracks nested files, nanosecond timestamps, and legacy JSON sidecars. - Normalize Windows separators and check relative paths at execution. Cache mods by file path so identical metadata names in different folders do not collide when saving their configuration.
- Add regression tests using real safetensors and ComfyUI queue validation.
New: Continuum integration
- Continuum RefMod Bridge node — injects your mods into
ComfyUI-H3-Continuum's
chunked long-video samplers (V2/V3/V3.4), which build their conditioning
internally and have no CONDITIONING socket. While armed, every chunk
carries the bundle's ref blocks as native
minimax_refs— persistent across all chunks, no Continuum markers so its continuity layout never re-times them. Version-checked against one public seam; if Continuum changes internally the bridge reports and degrades to no-injection rather than breaking a run. One active bundle per graph; per-run scramble seeds work normally. Wire it on the sampler's model line (Load Model -> bridge -> samplermodel): ComfyUI never executes fully disconnected nodes, so the earlier "anywhere upstream" placement silently did nothing.
Fixed: why insertions felt weak, random, or "possessed"
Three defects were found and fixed; each is reproducible with the new
gauntlet_harness.py (run it with ComfyUI's Python — no server needed).
- Apply's default curve silently HALVED every multi-frame mod. The old
default (
concept_at_end+ease) runs its envelope across the mod's own ref frames — mean multiplier ≈ 0.50 at any frame count — so video/ stacked mods shipped half-blurred unless you touched widgets. Worse, ref tokens are not bound to output-video time at all: no frame curve can place a concept "at the end of the video"; it only weights which reference content dominates. Defaults are nowconstant+linear+1.0(= full strength, official-ref parity); saved workflows keep their saved values. The curve widgets remain for stack weighting, with honest tooltips. For real output-timing control use H3 RefMod Step Curve (denoise timeline). - Image-kind mods ignored every curve dial silently (
curve_strengthsshort-circuits at 1 latent frame), so single-image identity mods always injected at full strength — the "concept bleeds into everything" case. Nowcurve_valueacts as a plain strength cap on single-frame mods (e.g. value 0.4 = the ref blends 40% toward its blurred self), with a console note. - H3 RefMod Step Curve leaked state across generations. The wrapper cached pristine/blurred latents plus the marked-ref index mapping on the first forward and never reset them — but ComfyUI reuses the patched MODEL and conditioning payload across queued runs, so run #2 mixed run #1's cached tensors whenever the scramble seed/mods/strengths changed (corrupted, "ghost" output). The wrapper now keys its state on the injected refs' identities (which it never mutates), re-keying cleanly on every new run while keeping pristine stable within one; chaining multiple Step Curve nodes composes deterministically, and progress is normalized to each run's actual schedule start (partial denoise included).
Also new: Apply now prints each row's effective strength (row × retention × curve-mean) and flags anything below 0.30 — the usual suspect when an insertion feels weak.
New: fixed configs, shipped with the mod
- Fix H3 RefMod Config node — bakes tuned Apply/Step-Curve settings
(
retention+curve+step_curve) into a mod's safetensors metadata and re-saves the file in place, so a concept ships with the settings that make it work. overridetoggle on Apply H3 RefMod and H3 RefMod Step Curve — reads the first mod in the bundle that carries a fixed config and uses those settings instead of the widgets; falls back to the manual parameters (with a console note) when no mod has one.- The debug curve graph reflects the overridden curve, so what you see is what actually ran.
New: merge mode on Extract H3 RefMod
mergetoggle (training mode): one shared latent refined jointly against every reference's full encode (mean reconstruction error) instead of stacking each ref's own pooled latent. A whole collection becomes ONE tiny consensus mod — pulled toward what's common across all the views (structure/motion/identity) rather than any single shot's framing or background, at one ref block's worth of tokens.identitydials the joint refinement; masks andbackground_retentionstill apply per ref.motion_onlytoggle (training mode, experimental): video refs are converted to per-frame temporal differences before encoding, so the mod carries where/how things move instead of what they look like — static appearance (lineart look, background) never enters the latent. A soft motion guide, not a ControlNet (the ref channel is content-based).
Fixed
- Curve directions now mean what they say.
concept_at_endactually puts the concept at the end of the video (and at the end of the denoise timeline on H3 RefMod Step Curve);concept_at_startputs it at the start. The old envelopes were inverted relative to the names. Legacydecrease/increaseworkflow values and the preset names (fade_in/fade_out/bump/dip) still resolve to their original envelopes, so old workflows keep their behavior. - H3 RefMod Step Curve no longer touches your input video. The wrapper used to re-mix every ref latent, including the original video's identity anchor — that blurred the source subject and let a different person "pop". It now identifies the refs injected by Apply H3 RefMod (via a marker on the blocks) and only re-mixes those; native ref2va refs pass through untouched, and a console note explains when there is nothing to re-mix.
v0.1.0 — 2026-08-16
First tagged release. Extract references once as tiny .safetensors
"mods" and inject them through conditioning — no full video/image loading
every generation, no training.
Nodes
- Extract H3 RefMod — image / video / GIF → one small mod file. Modes:
training(pooled concept/identity thumbnails; thepooldial trades concept ↔ identity) andencode(full-resolution VAE encode). Identity refinement steps, token cap with dedup, data multiplier for short clips, optionalav_encoderinput, and folder bulk-loading. - Load H3 RefMods — LoRA-loader-style rows with a typed strength and a
copiesmultiplier (2-10x — the manual row-duplication trick as a knob). - Load H3 RefMod Axis — signed A/B sliders: negative picks the A mod, positive the B mod, one dial controls both.
- Load H3 RefMod Folder — every image/video in a folder as an ordered ref list.
- Apply H3 RefMod — one node for the pack's
MINIMAX_H3_CONDand the built-inCONDITIONING.retentionmaster strength; a curve split intocurve_direction(constant / concept_at_start / concept_at_middle / concept_at_end / concept_at_ends) +curve_shape(linear / ease / sigmoid / tanh / quadratic / cubic / exponential / stair / elastic / bump / dip) +curve_value;scramble_seed; optional curve-graphdebugIMAGE output; shareable PNG graph presets (graph embedded in the image metadata, legacy.jsonstill loads). - H3 RefMod Step Curve — the same curve widgets, but over the denoise
timeline: re-mixes every ref latent once per step (early steps lock
composition/identity, late steps stay clean or refine detail) via a
ComfyUI
DIFFUSION_MODELwrapper, attached between the model loader and the sampler.
Reference math
- Weakening a ref blends toward a blurred copy of itself instead of noise or zero — stays on the latent manifold, so no grey/static output.
- The per-frame curve mixes each ref latent frame with
retention * curve(x)instead of one flat strength. - Greedy temporal dedup + budget-fit resampling make the token cap cheap.
Misc
extract_mod.pystandalone CLI.- Mods live in
ComfyUI/models/refmods/(created on first run, next to loras/ and unet/); older mods in the pack'smods/folder still load.