Files
DumasNodes/CHANGELOG.md
T
chris.dumas 70d0672ee3 Squashed 'vendor/dumas_minimax_h3_refmod/' content from commit 517e8c4
git-subtree-dir: vendor/dumas_minimax_h3_refmod
git-subtree-split: 517e8c419a8584fbf69fceb487bc39d83e6a6ec8
2026-10-20 13:43:44 +00:00

235 lines
14 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Changelog
All notable changes are tracked here. Each version is also published as a
[GitHub Release](https://github.com/Luisacaotica/ComfyUI-MiniMaxH3Mod/releases),
so you can keep using an older version if a new one changes something you
rely on.
## Unreleased
- Correct visual reference sampling to H3's 17k+5 video grid in the node and CLI. Preserve all frames of aligned clips and both endpoints when uniformly sampling; references/caps below five frames use the first image.
- Support MPS-resident RefMod latents without global CPU fallback: pool and blur explicitly on CPU, preserving return device, dtype and autograd. Synchronize MPS in the refinement timing harness.
- Add a reference_map output to Inspect H3 RefMod for LLM prompt generation before Text Encode, using the same reference numbering (#19).
- Fix Loader/Axis settings resetting when switching workflow tabs or reopening workflows: defer slot regrouping until widget restoration finishes (#18).
- Repair Apply's reactive MatchType socket on older frontends whose canvas socket classes use private fields, preventing missing controls while preserving conditioning type matching (#17).
## v0.2.6 — 2026-09-12
- Make Loader/Axis additive: start with one visible slot, add/remove slots up to eight, group modality controls with their reference, preserve old workflow values and protect connected slots.
- Add version-5 single-file RefMod bundles containing independent image/video/audio members, with atomic saves and preserved member order.
- Add Save H3 RefMod Bundle as an output node and opt-in Master save_layout=bundle. Keep standalone saving and existing workflow defaults.
- Add All/Visual/Audio selection and separate modality strengths to every Loader/Axis slot. Skip excluded tensors and count only selected references in token budgets.
- Discover combined files in the library; preserve unselected members during Config updates and support export to standalone files.
- Document the container schema for third-party pickers. Bundling does not add voice binding, AV synchronization, or original-waveform passthrough.
## v0.2.5 — 2026-09-08
- Rename Extract nodes to Create, with unchanged node IDs. Full Reference and Compressed Reference replace the visible encode/training names; old mode values remain accepted. Rename the identity control to Refinement Steps.
- Add visual budget_policy: truncate preserves existing behavior; error stops before saving when the visual token budget is exceeded. Master inherits the option.
- Allow larger token/frame limits and preserve the requested frame count in the motion preset. Loader budgets default to 0 (disabled).
- Add experimental H3 RefMod Text Encode with numbered reference presentation and a reference map. Support compatible ClipProj encoders and native batched video-VAE output.
- Add optional full-video inspection with correct IMAGE output dimensions; retain the lightweight first-frame default.
- Add playable music/video README examples and exclude local tests from distribution.
Validation: 51 local regression tests passed; creation-label hooks checked with Node.js. Full GPU generation, projected-encoder quality and voice transfer remain unverified. Voice cloning is not claimed to work.
Text Encode already attaches references: do not inject the same bundle again with Apply/Bridge. Integrated Continuum samplers still need a dedicated integration to use this conditioning. Full visual previews and text/vision presentation increase memory use.
## v0.2.0 — 2026-09-07
### Known limitation: voice transfer
Audio reference support includes an observed music-transfer example, but current
speaker-identity tests failed. This release does not provide working voice cloning.
The cause has not been isolated between H3/checkpoint behavior and the reference
presentation difference in the current RefMod integration.
### Follow-up implementation (2026-09-07)
- Clarify Master saving logs: deferred extraction, Created/Replaced destinations,
explicit save=False and a final token/file summary; report saved_paths in details.
- Add Save H3 RefMods as an output node: save mixed bundles without connecting
a Preview or sampler, with prefix/subfolder options and copy deduplication.
- Save new mods/presets in the first registered refmods root (extra_model_paths);
keep the standard models/refmods fallback and existing-file Config locations.
- Remove sibling-pack imports and startup dependency warning. Legacy av_encoder
inputs use native ComfyUI video VAE loading; standard VAE inputs are preferred.
- Add Extract H3 RefMod Master: optional image/video and audio inputs, sequential
extraction through the shared extractors, combined token budget, one bundle
output and separate compatible visual/audio files under a shared name.
- Step Curve now mixes the current payload without retaining tensors and handles
keyframe offsets, audio refs, empty conditioning and chained curves.
- Bridge uses the MODEL outer-sampling hook with finally cleanup; removes global
state and third-party monkey patches. The old Disarm node is deprecated.
- Align image/mask center crops; share resize and causal-frame helpers with CLI.
- Invalidate loaded files and queue cache on changes; cap cache at 256 MiB;
enforce copies/numeric limits; bound video decode buffers with unknown lengths.
- Add native audio RefMods, legacy AudioMod import, audio Extract, library UI,
inspection/optional previews, save subfolders, explicit scramble modes,
extraction presets and total-token budgets.
- Atomic saves, deduplicated config writes and grouped mean-MSE refinement.
- Historical entries below describe earlier unreleased revisions; the behavior
above and the current README supersede their bridge/cache descriptions.
### Fixed: linked loader inputs and subfolders
- Both loaders accept unresolved/empty mod names and connected STRING/combo
outputs. Fixed missing names still fail validation; connected names resolve
at execution with a missing-file error when necessary.
- Discover RefMods recursively as `folder/name`, preserving metadata filtering
and skipping graph presets. Listing cache tracks nested files, nanosecond
timestamps, and legacy JSON sidecars.
- Normalize Windows separators and check relative paths at execution. Cache
mods by file path so identical metadata names in different folders do not
collide when saving their configuration.
- Add regression tests using real safetensors and ComfyUI queue validation.
### New: Continuum integration
- **Continuum RefMod Bridge** node — injects your mods into
[ComfyUI-H3-Continuum](https://github.com/xiaolibai-sys/ComfyUI-H3-Continuum)'s
chunked long-video samplers (V2/V3/V3.4), which build their conditioning
internally and have no CONDITIONING socket. While armed, every chunk
carries the bundle's ref blocks as native `minimax_refs` — persistent
across all chunks, no Continuum markers so its continuity layout never
re-times them. Version-checked against one public seam; if Continuum
changes internally the bridge reports and degrades to no-injection rather
than breaking a run. One active bundle per graph; per-run scramble seeds
work normally. Wire it on the sampler's model line (Load Model -> bridge
-> sampler `model`): ComfyUI never executes fully disconnected nodes, so
the earlier "anywhere upstream" placement silently did nothing.
### Fixed: why insertions felt weak, random, or "possessed"
Three defects were found and fixed; each is reproducible with the new
`gauntlet_harness.py` (run it with ComfyUI's Python — no server needed).
- **Apply's default curve silently HALVED every multi-frame mod.** The old
default (`concept_at_end` + `ease`) runs its envelope across the mod's own
ref frames — mean multiplier ≈ **0.50** at any frame count — so video/
stacked mods shipped half-blurred unless you touched widgets. Worse, ref
tokens are not bound to output-video time at all: no frame curve can place
a concept "at the end of the video"; it only weights which reference
content dominates. Defaults are now `constant` + `linear` + `1.0`
(= full strength, official-ref parity); saved workflows keep their saved
values. The curve widgets remain for stack weighting, with honest tooltips.
For real output-timing control use **H3 RefMod Step Curve** (denoise
timeline).
- **Image-kind mods ignored every curve dial silently** (`curve_strengths`
short-circuits at 1 latent frame), so single-image identity mods always
injected at full strength — the "concept bleeds into everything" case.
Now `curve_value` acts as a plain strength cap on single-frame mods
(e.g. value 0.4 = the ref blends 40% toward its blurred self), with a
console note.
- **H3 RefMod Step Curve leaked state across generations.** The wrapper
cached pristine/blurred latents plus the marked-ref index mapping on the
first forward and never reset them — but ComfyUI reuses the patched MODEL
and conditioning payload across queued runs, so run #2 mixed run #1's
cached tensors whenever the scramble seed/mods/strengths changed
(corrupted, "ghost" output). The wrapper now keys its state on the
injected refs' identities (which it never mutates), re-keying cleanly on
every new run while keeping pristine stable within one; chaining multiple
Step Curve nodes composes deterministically, and progress is normalized to
each run's actual schedule start (partial denoise included).
Also new: Apply now prints each row's *effective* strength (row × retention
× curve-mean) and flags anything below 0.30 — the usual suspect when an
insertion feels weak.
### New: fixed configs, shipped with the mod
- **Fix H3 RefMod Config** node — bakes tuned Apply/Step-Curve settings
(`retention` + `curve` + `step_curve`) into a mod's safetensors metadata
and re-saves the file in place, so a concept ships with the settings that
make it work.
- **`override` toggle** on Apply H3 RefMod and H3 RefMod Step Curve — reads
the first mod in the bundle that carries a fixed config and uses those
settings instead of the widgets; falls back to the manual parameters
(with a console note) when no mod has one.
- The debug curve graph reflects the overridden curve, so what you see is
what actually ran.
### New: merge mode on Extract H3 RefMod
- **`merge` toggle** (training mode): one shared latent refined jointly
against every reference's full encode (mean reconstruction error) instead
of stacking each ref's own pooled latent. A whole collection becomes ONE
tiny consensus mod — pulled toward what's common across all the views
(structure/motion/identity) rather than any single shot's framing or
background, at one ref block's worth of tokens. `identity` dials the
joint refinement; masks and `background_retention` still apply per ref.
- **`motion_only` toggle** (training mode, experimental): video refs are
converted to per-frame temporal differences before encoding, so the mod
carries where/how things move instead of what they look like — static
appearance (lineart look, background) never enters the latent. A soft
motion guide, not a ControlNet (the ref channel is content-based).
### Fixed
- **Curve directions now mean what they say.** `concept_at_end` actually
puts the concept at the **end** of the video (and at the end of the
denoise timeline on H3 RefMod Step Curve); `concept_at_start` puts it at
the start. The old envelopes were inverted relative to the names. Legacy
`decrease`/`increase` workflow values and the preset names
(`fade_in`/`fade_out`/`bump`/`dip`) still resolve to their original
envelopes, so old workflows keep their behavior.
- **H3 RefMod Step Curve no longer touches your input video.** The wrapper
used to re-mix *every* ref latent, including the original video's
identity anchor — that blurred the source subject and let a different
person "pop". It now identifies the refs injected by Apply H3 RefMod (via
a marker on the blocks) and only re-mixes those; native ref2va refs pass
through untouched, and a console note explains when there is nothing to
re-mix.
## v0.1.0 — 2026-08-16
First tagged release. Extract references once as tiny `.safetensors`
"mods" and inject them through conditioning — no full video/image loading
every generation, no training.
### Nodes
- **Extract H3 RefMod** — image / video / GIF → one small mod file. Modes:
`training` (pooled concept/identity thumbnails; the `pool` dial trades
concept ↔ identity) and `encode` (full-resolution VAE encode). Identity
refinement steps, token cap with dedup, data multiplier for short clips,
optional `av_encoder` input, and folder bulk-loading.
- **Load H3 RefMods** — LoRA-loader-style rows with a typed strength and a
`copies` multiplier (2-10x — the manual row-duplication trick as a knob).
- **Load H3 RefMod Axis** — signed A/B sliders: negative picks the A mod,
positive the B mod, one dial controls both.
- **Load H3 RefMod Folder** — every image/video in a folder as an ordered
ref list.
- **Apply H3 RefMod** — one node for the pack's `MINIMAX_H3_COND` and the
built-in `CONDITIONING`. `retention` master strength; a curve split into
`curve_direction` (constant / concept_at_start / concept_at_middle /
concept_at_end / concept_at_ends) + `curve_shape` (linear / ease /
sigmoid / tanh / quadratic / cubic / exponential / stair / elastic /
bump / dip) + `curve_value`; `scramble_seed`; optional curve-graph
`debug` IMAGE output; shareable PNG graph presets (graph embedded in the
image metadata, legacy `.json` still loads).
- **H3 RefMod Step Curve** — the same curve widgets, but over the **denoise
timeline**: re-mixes every ref latent once per step (early steps lock
composition/identity, late steps stay clean or refine detail) via a
ComfyUI `DIFFUSION_MODEL` wrapper, attached between the model loader and
the sampler.
### Reference math
- Weakening a ref blends toward a blurred copy of itself instead of noise
or zero — stays on the latent manifold, so no grey/static output.
- The per-frame curve mixes each ref latent frame with
`retention * curve(x)` instead of one flat strength.
- Greedy temporal dedup + budget-fit resampling make the token cap cheap.
### Misc
- `extract_mod.py` standalone CLI.
- Mods live in `ComfyUI/models/refmods/` (created on first run, next to
loras/ and unet/); older mods in the pack's `mods/` folder still load.