Files
DumasNodes/vendor/dumas_minimax_h3_refmod/CHANGELOG.md
T

14 KiB
Raw Blame History

Changelog

All notable changes are tracked here. Each version is also published as a GitHub Release, so you can keep using an older version if a new one changes something you rely on.

Unreleased

  • Correct visual reference sampling to H3's 17k+5 video grid in the node and CLI. Preserve all frames of aligned clips and both endpoints when uniformly sampling; references/caps below five frames use the first image.

  • Support MPS-resident RefMod latents without global CPU fallback: pool and blur explicitly on CPU, preserving return device, dtype and autograd. Synchronize MPS in the refinement timing harness.

  • Add a reference_map output to Inspect H3 RefMod for LLM prompt generation before Text Encode, using the same reference numbering (#19).

  • Fix Loader/Axis settings resetting when switching workflow tabs or reopening workflows: defer slot regrouping until widget restoration finishes (#18).

  • Repair Apply's reactive MatchType socket on older frontends whose canvas socket classes use private fields, preventing missing controls while preserving conditioning type matching (#17).

v0.2.6 — 2026-09-12

  • Make Loader/Axis additive: start with one visible slot, add/remove slots up to eight, group modality controls with their reference, preserve old workflow values and protect connected slots.
  • Add version-5 single-file RefMod bundles containing independent image/video/audio members, with atomic saves and preserved member order.
  • Add Save H3 RefMod Bundle as an output node and opt-in Master save_layout=bundle. Keep standalone saving and existing workflow defaults.
  • Add All/Visual/Audio selection and separate modality strengths to every Loader/Axis slot. Skip excluded tensors and count only selected references in token budgets.
  • Discover combined files in the library; preserve unselected members during Config updates and support export to standalone files.
  • Document the container schema for third-party pickers. Bundling does not add voice binding, AV synchronization, or original-waveform passthrough.

v0.2.5 — 2026-09-08

  • Rename Extract nodes to Create, with unchanged node IDs. Full Reference and Compressed Reference replace the visible encode/training names; old mode values remain accepted. Rename the identity control to Refinement Steps.
  • Add visual budget_policy: truncate preserves existing behavior; error stops before saving when the visual token budget is exceeded. Master inherits the option.
  • Allow larger token/frame limits and preserve the requested frame count in the motion preset. Loader budgets default to 0 (disabled).
  • Add experimental H3 RefMod Text Encode with numbered reference presentation and a reference map. Support compatible ClipProj encoders and native batched video-VAE output.
  • Add optional full-video inspection with correct IMAGE output dimensions; retain the lightweight first-frame default.
  • Add playable music/video README examples and exclude local tests from distribution.

Validation: 51 local regression tests passed; creation-label hooks checked with Node.js. Full GPU generation, projected-encoder quality and voice transfer remain unverified. Voice cloning is not claimed to work.

Text Encode already attaches references: do not inject the same bundle again with Apply/Bridge. Integrated Continuum samplers still need a dedicated integration to use this conditioning. Full visual previews and text/vision presentation increase memory use.

v0.2.0 — 2026-09-07

Known limitation: voice transfer

Audio reference support includes an observed music-transfer example, but current speaker-identity tests failed. This release does not provide working voice cloning. The cause has not been isolated between H3/checkpoint behavior and the reference presentation difference in the current RefMod integration.

Follow-up implementation (2026-09-07)

  • Clarify Master saving logs: deferred extraction, Created/Replaced destinations, explicit save=False and a final token/file summary; report saved_paths in details.

  • Add Save H3 RefMods as an output node: save mixed bundles without connecting a Preview or sampler, with prefix/subfolder options and copy deduplication.

  • Save new mods/presets in the first registered refmods root (extra_model_paths); keep the standard models/refmods fallback and existing-file Config locations.

  • Remove sibling-pack imports and startup dependency warning. Legacy av_encoder inputs use native ComfyUI video VAE loading; standard VAE inputs are preferred.

  • Add Extract H3 RefMod Master: optional image/video and audio inputs, sequential extraction through the shared extractors, combined token budget, one bundle output and separate compatible visual/audio files under a shared name.

  • Step Curve now mixes the current payload without retaining tensors and handles keyframe offsets, audio refs, empty conditioning and chained curves.

  • Bridge uses the MODEL outer-sampling hook with finally cleanup; removes global state and third-party monkey patches. The old Disarm node is deprecated.

  • Align image/mask center crops; share resize and causal-frame helpers with CLI.

  • Invalidate loaded files and queue cache on changes; cap cache at 256 MiB; enforce copies/numeric limits; bound video decode buffers with unknown lengths.

  • Add native audio RefMods, legacy AudioMod import, audio Extract, library UI, inspection/optional previews, save subfolders, explicit scramble modes, extraction presets and total-token budgets.

  • Atomic saves, deduplicated config writes and grouped mean-MSE refinement.

  • Historical entries below describe earlier unreleased revisions; the behavior above and the current README supersede their bridge/cache descriptions.

Fixed: linked loader inputs and subfolders

  • Both loaders accept unresolved/empty mod names and connected STRING/combo outputs. Fixed missing names still fail validation; connected names resolve at execution with a missing-file error when necessary.
  • Discover RefMods recursively as folder/name, preserving metadata filtering and skipping graph presets. Listing cache tracks nested files, nanosecond timestamps, and legacy JSON sidecars.
  • Normalize Windows separators and check relative paths at execution. Cache mods by file path so identical metadata names in different folders do not collide when saving their configuration.
  • Add regression tests using real safetensors and ComfyUI queue validation.

New: Continuum integration

  • Continuum RefMod Bridge node — injects your mods into ComfyUI-H3-Continuum's chunked long-video samplers (V2/V3/V3.4), which build their conditioning internally and have no CONDITIONING socket. While armed, every chunk carries the bundle's ref blocks as native minimax_refs — persistent across all chunks, no Continuum markers so its continuity layout never re-times them. Version-checked against one public seam; if Continuum changes internally the bridge reports and degrades to no-injection rather than breaking a run. One active bundle per graph; per-run scramble seeds work normally. Wire it on the sampler's model line (Load Model -> bridge -> sampler model): ComfyUI never executes fully disconnected nodes, so the earlier "anywhere upstream" placement silently did nothing.

Fixed: why insertions felt weak, random, or "possessed"

Three defects were found and fixed; each is reproducible with the new gauntlet_harness.py (run it with ComfyUI's Python — no server needed).

  • Apply's default curve silently HALVED every multi-frame mod. The old default (concept_at_end + ease) runs its envelope across the mod's own ref frames — mean multiplier ≈ 0.50 at any frame count — so video/ stacked mods shipped half-blurred unless you touched widgets. Worse, ref tokens are not bound to output-video time at all: no frame curve can place a concept "at the end of the video"; it only weights which reference content dominates. Defaults are now constant + linear + 1.0 (= full strength, official-ref parity); saved workflows keep their saved values. The curve widgets remain for stack weighting, with honest tooltips. For real output-timing control use H3 RefMod Step Curve (denoise timeline).
  • Image-kind mods ignored every curve dial silently (curve_strengths short-circuits at 1 latent frame), so single-image identity mods always injected at full strength — the "concept bleeds into everything" case. Now curve_value acts as a plain strength cap on single-frame mods (e.g. value 0.4 = the ref blends 40% toward its blurred self), with a console note.
  • H3 RefMod Step Curve leaked state across generations. The wrapper cached pristine/blurred latents plus the marked-ref index mapping on the first forward and never reset them — but ComfyUI reuses the patched MODEL and conditioning payload across queued runs, so run #2 mixed run #1's cached tensors whenever the scramble seed/mods/strengths changed (corrupted, "ghost" output). The wrapper now keys its state on the injected refs' identities (which it never mutates), re-keying cleanly on every new run while keeping pristine stable within one; chaining multiple Step Curve nodes composes deterministically, and progress is normalized to each run's actual schedule start (partial denoise included).

Also new: Apply now prints each row's effective strength (row × retention × curve-mean) and flags anything below 0.30 — the usual suspect when an insertion feels weak.

New: fixed configs, shipped with the mod

  • Fix H3 RefMod Config node — bakes tuned Apply/Step-Curve settings (retention + curve + step_curve) into a mod's safetensors metadata and re-saves the file in place, so a concept ships with the settings that make it work.
  • override toggle on Apply H3 RefMod and H3 RefMod Step Curve — reads the first mod in the bundle that carries a fixed config and uses those settings instead of the widgets; falls back to the manual parameters (with a console note) when no mod has one.
  • The debug curve graph reflects the overridden curve, so what you see is what actually ran.

New: merge mode on Extract H3 RefMod

  • merge toggle (training mode): one shared latent refined jointly against every reference's full encode (mean reconstruction error) instead of stacking each ref's own pooled latent. A whole collection becomes ONE tiny consensus mod — pulled toward what's common across all the views (structure/motion/identity) rather than any single shot's framing or background, at one ref block's worth of tokens. identity dials the joint refinement; masks and background_retention still apply per ref.
  • motion_only toggle (training mode, experimental): video refs are converted to per-frame temporal differences before encoding, so the mod carries where/how things move instead of what they look like — static appearance (lineart look, background) never enters the latent. A soft motion guide, not a ControlNet (the ref channel is content-based).

Fixed

  • Curve directions now mean what they say. concept_at_end actually puts the concept at the end of the video (and at the end of the denoise timeline on H3 RefMod Step Curve); concept_at_start puts it at the start. The old envelopes were inverted relative to the names. Legacy decrease/increase workflow values and the preset names (fade_in/fade_out/bump/dip) still resolve to their original envelopes, so old workflows keep their behavior.
  • H3 RefMod Step Curve no longer touches your input video. The wrapper used to re-mix every ref latent, including the original video's identity anchor — that blurred the source subject and let a different person "pop". It now identifies the refs injected by Apply H3 RefMod (via a marker on the blocks) and only re-mixes those; native ref2va refs pass through untouched, and a console note explains when there is nothing to re-mix.

v0.1.0 — 2026-08-16

First tagged release. Extract references once as tiny .safetensors "mods" and inject them through conditioning — no full video/image loading every generation, no training.

Nodes

  • Extract H3 RefMod — image / video / GIF → one small mod file. Modes: training (pooled concept/identity thumbnails; the pool dial trades concept ↔ identity) and encode (full-resolution VAE encode). Identity refinement steps, token cap with dedup, data multiplier for short clips, optional av_encoder input, and folder bulk-loading.
  • Load H3 RefMods — LoRA-loader-style rows with a typed strength and a copies multiplier (2-10x — the manual row-duplication trick as a knob).
  • Load H3 RefMod Axis — signed A/B sliders: negative picks the A mod, positive the B mod, one dial controls both.
  • Load H3 RefMod Folder — every image/video in a folder as an ordered ref list.
  • Apply H3 RefMod — one node for the pack's MINIMAX_H3_COND and the built-in CONDITIONING. retention master strength; a curve split into curve_direction (constant / concept_at_start / concept_at_middle / concept_at_end / concept_at_ends) + curve_shape (linear / ease / sigmoid / tanh / quadratic / cubic / exponential / stair / elastic / bump / dip) + curve_value; scramble_seed; optional curve-graph debug IMAGE output; shareable PNG graph presets (graph embedded in the image metadata, legacy .json still loads).
  • H3 RefMod Step Curve — the same curve widgets, but over the denoise timeline: re-mixes every ref latent once per step (early steps lock composition/identity, late steps stay clean or refine detail) via a ComfyUI DIFFUSION_MODEL wrapper, attached between the model loader and the sampler.

Reference math

  • Weakening a ref blends toward a blurred copy of itself instead of noise or zero — stays on the latent manifold, so no grey/static output.
  • The per-frame curve mixes each ref latent frame with retention * curve(x) instead of one flat strength.
  • Greedy temporal dedup + budget-fit resampling make the token cap cheap.

Misc

  • extract_mod.py standalone CLI.
  • Mods live in ComfyUI/models/refmods/ (created on first run, next to loras/ and unet/); older mods in the pack's mods/ folder still load.