Squashed 'vendor/dumas_minimax_h3_refmod/' content from commit 517e8c4
git-subtree-dir: vendor/dumas_minimax_h3_refmod git-subtree-split: 517e8c419a8584fbf69fceb487bc39d83e6a6ec8
This commit is contained in:
+234
@@ -0,0 +1,234 @@
|
||||
# Changelog
|
||||
|
||||
All notable changes are tracked here. Each version is also published as a
|
||||
[GitHub Release](https://github.com/Luisacaotica/ComfyUI-MiniMaxH3Mod/releases),
|
||||
so you can keep using an older version if a new one changes something you
|
||||
rely on.
|
||||
|
||||
## Unreleased
|
||||
|
||||
- Correct visual reference sampling to H3's 17k+5 video grid in the node and CLI. Preserve all frames of aligned clips and both endpoints when uniformly sampling; references/caps below five frames use the first image.
|
||||
- Support MPS-resident RefMod latents without global CPU fallback: pool and blur explicitly on CPU, preserving return device, dtype and autograd. Synchronize MPS in the refinement timing harness.
|
||||
|
||||
- Add a reference_map output to Inspect H3 RefMod for LLM prompt generation before Text Encode, using the same reference numbering (#19).
|
||||
- Fix Loader/Axis settings resetting when switching workflow tabs or reopening workflows: defer slot regrouping until widget restoration finishes (#18).
|
||||
- Repair Apply's reactive MatchType socket on older frontends whose canvas socket classes use private fields, preventing missing controls while preserving conditioning type matching (#17).
|
||||
|
||||
## v0.2.6 — 2026-09-12
|
||||
|
||||
- Make Loader/Axis additive: start with one visible slot, add/remove slots up to eight, group modality controls with their reference, preserve old workflow values and protect connected slots.
|
||||
- Add version-5 single-file RefMod bundles containing independent image/video/audio members, with atomic saves and preserved member order.
|
||||
- Add Save H3 RefMod Bundle as an output node and opt-in Master save_layout=bundle. Keep standalone saving and existing workflow defaults.
|
||||
- Add All/Visual/Audio selection and separate modality strengths to every Loader/Axis slot. Skip excluded tensors and count only selected references in token budgets.
|
||||
- Discover combined files in the library; preserve unselected members during Config updates and support export to standalone files.
|
||||
- Document the container schema for third-party pickers. Bundling does not add voice binding, AV synchronization, or original-waveform passthrough.
|
||||
|
||||
## v0.2.5 — 2026-09-08
|
||||
|
||||
- Rename Extract nodes to Create, with unchanged node IDs. Full Reference and Compressed Reference replace the visible encode/training names; old mode values remain accepted. Rename the identity control to Refinement Steps.
|
||||
- Add visual budget_policy: truncate preserves existing behavior; error stops before saving when the visual token budget is exceeded. Master inherits the option.
|
||||
- Allow larger token/frame limits and preserve the requested frame count in the motion preset. Loader budgets default to 0 (disabled).
|
||||
- Add experimental H3 RefMod Text Encode with numbered reference presentation and a reference map. Support compatible ClipProj encoders and native batched video-VAE output.
|
||||
- Add optional full-video inspection with correct IMAGE output dimensions; retain the lightweight first-frame default.
|
||||
- Add playable music/video README examples and exclude local tests from distribution.
|
||||
|
||||
Validation: 51 local regression tests passed; creation-label hooks checked with Node.js. Full GPU generation, projected-encoder quality and voice transfer remain unverified. Voice cloning is not claimed to work.
|
||||
|
||||
Text Encode already attaches references: do not inject the same bundle again with Apply/Bridge. Integrated Continuum samplers still need a dedicated integration to use this conditioning. Full visual previews and text/vision presentation increase memory use.
|
||||
|
||||
## v0.2.0 — 2026-09-07
|
||||
|
||||
### Known limitation: voice transfer
|
||||
|
||||
Audio reference support includes an observed music-transfer example, but current
|
||||
speaker-identity tests failed. This release does not provide working voice cloning.
|
||||
The cause has not been isolated between H3/checkpoint behavior and the reference
|
||||
presentation difference in the current RefMod integration.
|
||||
|
||||
### Follow-up implementation (2026-09-07)
|
||||
|
||||
- Clarify Master saving logs: deferred extraction, Created/Replaced destinations,
|
||||
explicit save=False and a final token/file summary; report saved_paths in details.
|
||||
|
||||
- Add Save H3 RefMods as an output node: save mixed bundles without connecting
|
||||
a Preview or sampler, with prefix/subfolder options and copy deduplication.
|
||||
|
||||
- Save new mods/presets in the first registered refmods root (extra_model_paths);
|
||||
keep the standard models/refmods fallback and existing-file Config locations.
|
||||
- Remove sibling-pack imports and startup dependency warning. Legacy av_encoder
|
||||
inputs use native ComfyUI video VAE loading; standard VAE inputs are preferred.
|
||||
|
||||
- Add Extract H3 RefMod Master: optional image/video and audio inputs, sequential
|
||||
extraction through the shared extractors, combined token budget, one bundle
|
||||
output and separate compatible visual/audio files under a shared name.
|
||||
|
||||
- Step Curve now mixes the current payload without retaining tensors and handles
|
||||
keyframe offsets, audio refs, empty conditioning and chained curves.
|
||||
- Bridge uses the MODEL outer-sampling hook with finally cleanup; removes global
|
||||
state and third-party monkey patches. The old Disarm node is deprecated.
|
||||
- Align image/mask center crops; share resize and causal-frame helpers with CLI.
|
||||
- Invalidate loaded files and queue cache on changes; cap cache at 256 MiB;
|
||||
enforce copies/numeric limits; bound video decode buffers with unknown lengths.
|
||||
- Add native audio RefMods, legacy AudioMod import, audio Extract, library UI,
|
||||
inspection/optional previews, save subfolders, explicit scramble modes,
|
||||
extraction presets and total-token budgets.
|
||||
- Atomic saves, deduplicated config writes and grouped mean-MSE refinement.
|
||||
- Historical entries below describe earlier unreleased revisions; the behavior
|
||||
above and the current README supersede their bridge/cache descriptions.
|
||||
|
||||
### Fixed: linked loader inputs and subfolders
|
||||
|
||||
- Both loaders accept unresolved/empty mod names and connected STRING/combo
|
||||
outputs. Fixed missing names still fail validation; connected names resolve
|
||||
at execution with a missing-file error when necessary.
|
||||
- Discover RefMods recursively as `folder/name`, preserving metadata filtering
|
||||
and skipping graph presets. Listing cache tracks nested files, nanosecond
|
||||
timestamps, and legacy JSON sidecars.
|
||||
- Normalize Windows separators and check relative paths at execution. Cache
|
||||
mods by file path so identical metadata names in different folders do not
|
||||
collide when saving their configuration.
|
||||
- Add regression tests using real safetensors and ComfyUI queue validation.
|
||||
|
||||
### New: Continuum integration
|
||||
|
||||
- **Continuum RefMod Bridge** node — injects your mods into
|
||||
[ComfyUI-H3-Continuum](https://github.com/xiaolibai-sys/ComfyUI-H3-Continuum)'s
|
||||
chunked long-video samplers (V2/V3/V3.4), which build their conditioning
|
||||
internally and have no CONDITIONING socket. While armed, every chunk
|
||||
carries the bundle's ref blocks as native `minimax_refs` — persistent
|
||||
across all chunks, no Continuum markers so its continuity layout never
|
||||
re-times them. Version-checked against one public seam; if Continuum
|
||||
changes internally the bridge reports and degrades to no-injection rather
|
||||
than breaking a run. One active bundle per graph; per-run scramble seeds
|
||||
work normally. Wire it on the sampler's model line (Load Model -> bridge
|
||||
-> sampler `model`): ComfyUI never executes fully disconnected nodes, so
|
||||
the earlier "anywhere upstream" placement silently did nothing.
|
||||
|
||||
### Fixed: why insertions felt weak, random, or "possessed"
|
||||
|
||||
Three defects were found and fixed; each is reproducible with the new
|
||||
`gauntlet_harness.py` (run it with ComfyUI's Python — no server needed).
|
||||
|
||||
- **Apply's default curve silently HALVED every multi-frame mod.** The old
|
||||
default (`concept_at_end` + `ease`) runs its envelope across the mod's own
|
||||
ref frames — mean multiplier ≈ **0.50** at any frame count — so video/
|
||||
stacked mods shipped half-blurred unless you touched widgets. Worse, ref
|
||||
tokens are not bound to output-video time at all: no frame curve can place
|
||||
a concept "at the end of the video"; it only weights which reference
|
||||
content dominates. Defaults are now `constant` + `linear` + `1.0`
|
||||
(= full strength, official-ref parity); saved workflows keep their saved
|
||||
values. The curve widgets remain for stack weighting, with honest tooltips.
|
||||
For real output-timing control use **H3 RefMod Step Curve** (denoise
|
||||
timeline).
|
||||
- **Image-kind mods ignored every curve dial silently** (`curve_strengths`
|
||||
short-circuits at 1 latent frame), so single-image identity mods always
|
||||
injected at full strength — the "concept bleeds into everything" case.
|
||||
Now `curve_value` acts as a plain strength cap on single-frame mods
|
||||
(e.g. value 0.4 = the ref blends 40% toward its blurred self), with a
|
||||
console note.
|
||||
- **H3 RefMod Step Curve leaked state across generations.** The wrapper
|
||||
cached pristine/blurred latents plus the marked-ref index mapping on the
|
||||
first forward and never reset them — but ComfyUI reuses the patched MODEL
|
||||
and conditioning payload across queued runs, so run #2 mixed run #1's
|
||||
cached tensors whenever the scramble seed/mods/strengths changed
|
||||
(corrupted, "ghost" output). The wrapper now keys its state on the
|
||||
injected refs' identities (which it never mutates), re-keying cleanly on
|
||||
every new run while keeping pristine stable within one; chaining multiple
|
||||
Step Curve nodes composes deterministically, and progress is normalized to
|
||||
each run's actual schedule start (partial denoise included).
|
||||
|
||||
Also new: Apply now prints each row's *effective* strength (row × retention
|
||||
× curve-mean) and flags anything below 0.30 — the usual suspect when an
|
||||
insertion feels weak.
|
||||
|
||||
### New: fixed configs, shipped with the mod
|
||||
|
||||
- **Fix H3 RefMod Config** node — bakes tuned Apply/Step-Curve settings
|
||||
(`retention` + `curve` + `step_curve`) into a mod's safetensors metadata
|
||||
and re-saves the file in place, so a concept ships with the settings that
|
||||
make it work.
|
||||
- **`override` toggle** on Apply H3 RefMod and H3 RefMod Step Curve — reads
|
||||
the first mod in the bundle that carries a fixed config and uses those
|
||||
settings instead of the widgets; falls back to the manual parameters
|
||||
(with a console note) when no mod has one.
|
||||
- The debug curve graph reflects the overridden curve, so what you see is
|
||||
what actually ran.
|
||||
|
||||
### New: merge mode on Extract H3 RefMod
|
||||
|
||||
- **`merge` toggle** (training mode): one shared latent refined jointly
|
||||
against every reference's full encode (mean reconstruction error) instead
|
||||
of stacking each ref's own pooled latent. A whole collection becomes ONE
|
||||
tiny consensus mod — pulled toward what's common across all the views
|
||||
(structure/motion/identity) rather than any single shot's framing or
|
||||
background, at one ref block's worth of tokens. `identity` dials the
|
||||
joint refinement; masks and `background_retention` still apply per ref.
|
||||
- **`motion_only` toggle** (training mode, experimental): video refs are
|
||||
converted to per-frame temporal differences before encoding, so the mod
|
||||
carries where/how things move instead of what they look like — static
|
||||
appearance (lineart look, background) never enters the latent. A soft
|
||||
motion guide, not a ControlNet (the ref channel is content-based).
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Curve directions now mean what they say.** `concept_at_end` actually
|
||||
puts the concept at the **end** of the video (and at the end of the
|
||||
denoise timeline on H3 RefMod Step Curve); `concept_at_start` puts it at
|
||||
the start. The old envelopes were inverted relative to the names. Legacy
|
||||
`decrease`/`increase` workflow values and the preset names
|
||||
(`fade_in`/`fade_out`/`bump`/`dip`) still resolve to their original
|
||||
envelopes, so old workflows keep their behavior.
|
||||
- **H3 RefMod Step Curve no longer touches your input video.** The wrapper
|
||||
used to re-mix *every* ref latent, including the original video's
|
||||
identity anchor — that blurred the source subject and let a different
|
||||
person "pop". It now identifies the refs injected by Apply H3 RefMod (via
|
||||
a marker on the blocks) and only re-mixes those; native ref2va refs pass
|
||||
through untouched, and a console note explains when there is nothing to
|
||||
re-mix.
|
||||
|
||||
## v0.1.0 — 2026-08-16
|
||||
|
||||
First tagged release. Extract references once as tiny `.safetensors`
|
||||
"mods" and inject them through conditioning — no full video/image loading
|
||||
every generation, no training.
|
||||
|
||||
### Nodes
|
||||
|
||||
- **Extract H3 RefMod** — image / video / GIF → one small mod file. Modes:
|
||||
`training` (pooled concept/identity thumbnails; the `pool` dial trades
|
||||
concept ↔ identity) and `encode` (full-resolution VAE encode). Identity
|
||||
refinement steps, token cap with dedup, data multiplier for short clips,
|
||||
optional `av_encoder` input, and folder bulk-loading.
|
||||
- **Load H3 RefMods** — LoRA-loader-style rows with a typed strength and a
|
||||
`copies` multiplier (2-10x — the manual row-duplication trick as a knob).
|
||||
- **Load H3 RefMod Axis** — signed A/B sliders: negative picks the A mod,
|
||||
positive the B mod, one dial controls both.
|
||||
- **Load H3 RefMod Folder** — every image/video in a folder as an ordered
|
||||
ref list.
|
||||
- **Apply H3 RefMod** — one node for the pack's `MINIMAX_H3_COND` and the
|
||||
built-in `CONDITIONING`. `retention` master strength; a curve split into
|
||||
`curve_direction` (constant / concept_at_start / concept_at_middle /
|
||||
concept_at_end / concept_at_ends) + `curve_shape` (linear / ease /
|
||||
sigmoid / tanh / quadratic / cubic / exponential / stair / elastic /
|
||||
bump / dip) + `curve_value`; `scramble_seed`; optional curve-graph
|
||||
`debug` IMAGE output; shareable PNG graph presets (graph embedded in the
|
||||
image metadata, legacy `.json` still loads).
|
||||
- **H3 RefMod Step Curve** — the same curve widgets, but over the **denoise
|
||||
timeline**: re-mixes every ref latent once per step (early steps lock
|
||||
composition/identity, late steps stay clean or refine detail) via a
|
||||
ComfyUI `DIFFUSION_MODEL` wrapper, attached between the model loader and
|
||||
the sampler.
|
||||
|
||||
### Reference math
|
||||
|
||||
- Weakening a ref blends toward a blurred copy of itself instead of noise
|
||||
or zero — stays on the latent manifold, so no grey/static output.
|
||||
- The per-frame curve mixes each ref latent frame with
|
||||
`retention * curve(x)` instead of one flat strength.
|
||||
- Greedy temporal dedup + budget-fit resampling make the token cap cheap.
|
||||
|
||||
### Misc
|
||||
|
||||
- `extract_mod.py` standalone CLI.
|
||||
- Mods live in `ComfyUI/models/refmods/` (created on first run, next to
|
||||
loras/ and unet/); older mods in the pack's `mods/` folder still load.
|
||||
Reference in New Issue
Block a user