diff --git a/vendor/dumas_minimax_h3_refmod/.comfyignore b/vendor/dumas_minimax_h3_refmod/.comfyignore new file mode 100644 index 0000000..8a0f078 --- /dev/null +++ b/vendor/dumas_minimax_h3_refmod/.comfyignore @@ -0,0 +1,2 @@ +/tests/ +/node.zip diff --git a/vendor/dumas_minimax_h3_refmod/.gitignore b/vendor/dumas_minimax_h3_refmod/.gitignore new file mode 100644 index 0000000..ace1811 --- /dev/null +++ b/vendor/dumas_minimax_h3_refmod/.gitignore @@ -0,0 +1,3 @@ +__pycache__/ +*.pyc +/tests/ diff --git a/vendor/dumas_minimax_h3_refmod/BUNDLE_FORMAT.md b/vendor/dumas_minimax_h3_refmod/BUNDLE_FORMAT.md new file mode 100644 index 0000000..44b1fe3 --- /dev/null +++ b/vendor/dumas_minimax_h3_refmod/BUNDLE_FORMAT.md @@ -0,0 +1,82 @@ +# RefMod single-file bundle format + +Version 5 is a container of independent H3 references. It does not concatenate +audio with visual latents or change H3 conditioning semantics. Existing +standalone version-4 files remain supported and keep their `latent` tensor key. + +## Detection and layout + +Read JSON from the safetensors header's `refmod_meta` key without loading tensors. +A bundle has `_format_version: 5`, `kind: "bundle"`, a display `name`, and an +ordered `members` array containing 1–256 ordinary RefMod metadata objects. +Nested bundles are not supported. A member's `kind` is `image`, `video`, or `audio`. + +```json +{ + "_format_version": 5, + "kind": "bundle", + "name": "hero", + "members": [ + { + "_format_version": 4, + "name": "hero_visual", + "kind": "video", + "latent_t": 3, + "latent_h": 32, + "latent_w": 32 + }, + { + "_format_version": 4, + "name": "hero_audio", + "kind": "audio", + "latent_t": 200, + "sample_rate": 32000 + } + ] +} +``` + +This example omits optional member metadata. Each member keeps the usual mode, +source, tags, description, concept type and optional `refmod_config` JSON string. +Tensor `ref_0` belongs to member 0, `ref_1` to member 1, and so on. + +| Kind | Tensor layout | Tokens | +| --- | --- | --- | +| image | `[1,24,1,H,W]` | `(H/2) × (W/2)` | +| video | `[1,24,T,H,W]` | `T × (H/2) × (W/2)` | +| audio | `[1,32,2,T]` | `2 × T` | + +Visual H/W must be positive and even; T must be positive. Audio uses the existing +32 kHz H3 codec convention. Each member retains its own latent dimensions. + +## Picker and loader integration + +1. Detect `kind == "bundle"` and validate the supported format version. +2. Inspect members to display whether the file contains visuals, audio, or both. +3. Filter by **All**, **Visual** (`image` + `video`), or **Audio** before reading + tensor data. Skip members whose modality strength is zero. +4. Load the selected `ref_i` tensors and expand them into ordinary + `(H3RefMod, strength)` entries, preserving relative member order. +5. Multiply the slot strength by the modality strength. Account for each selected + member's token cost and runtime copies. + +In this pack, `bundle.load_bundle(path_without_extension, selection, +visual_strength, audio_strength)` performs selection and returns ordinary +entries with modality strengths. The caller applies the slot strength/copies. +Members retain their container `path` and `bundle_index` for Config updates. + +Save writes each distinct reference object once; runtime strengths and duplicate +copies are not stored. Repacking a loaded bundle flattens its selected members. +Config updates preserve other members and unknown container metadata/tensors. + +## Compatibility and limits + +New readers load older standalone files. Older readers expecting only a `latent` +tensor will need bundle support; there is no promise of forward compatibility. +The standalone Save node remains available for exporting members. + +The container carries encoded references, not original audio bytes, explicit +cross-modal timestamps, or speaker-to-character assignments. Shared storage does +not guarantee synchronized generation or stop multiple references from mixing. +Per-slot Audio strength affects all audio members in that slot; this initial UI +does not select individual audio tracks within a multi-audio file. diff --git a/vendor/dumas_minimax_h3_refmod/CHANGELOG.md b/vendor/dumas_minimax_h3_refmod/CHANGELOG.md new file mode 100644 index 0000000..d8c2e92 --- /dev/null +++ b/vendor/dumas_minimax_h3_refmod/CHANGELOG.md @@ -0,0 +1,234 @@ +# Changelog + +All notable changes are tracked here. Each version is also published as a +[GitHub Release](https://github.com/Luisacaotica/ComfyUI-MiniMaxH3Mod/releases), +so you can keep using an older version if a new one changes something you +rely on. + +## Unreleased + +- Correct visual reference sampling to H3's 17k+5 video grid in the node and CLI. Preserve all frames of aligned clips and both endpoints when uniformly sampling; references/caps below five frames use the first image. +- Support MPS-resident RefMod latents without global CPU fallback: pool and blur explicitly on CPU, preserving return device, dtype and autograd. Synchronize MPS in the refinement timing harness. + +- Add a reference_map output to Inspect H3 RefMod for LLM prompt generation before Text Encode, using the same reference numbering (#19). +- Fix Loader/Axis settings resetting when switching workflow tabs or reopening workflows: defer slot regrouping until widget restoration finishes (#18). +- Repair Apply's reactive MatchType socket on older frontends whose canvas socket classes use private fields, preventing missing controls while preserving conditioning type matching (#17). + +## v0.2.6 — 2026-09-12 + +- Make Loader/Axis additive: start with one visible slot, add/remove slots up to eight, group modality controls with their reference, preserve old workflow values and protect connected slots. +- Add version-5 single-file RefMod bundles containing independent image/video/audio members, with atomic saves and preserved member order. +- Add Save H3 RefMod Bundle as an output node and opt-in Master save_layout=bundle. Keep standalone saving and existing workflow defaults. +- Add All/Visual/Audio selection and separate modality strengths to every Loader/Axis slot. Skip excluded tensors and count only selected references in token budgets. +- Discover combined files in the library; preserve unselected members during Config updates and support export to standalone files. +- Document the container schema for third-party pickers. Bundling does not add voice binding, AV synchronization, or original-waveform passthrough. + +## v0.2.5 — 2026-09-08 + +- Rename Extract nodes to Create, with unchanged node IDs. Full Reference and Compressed Reference replace the visible encode/training names; old mode values remain accepted. Rename the identity control to Refinement Steps. +- Add visual budget_policy: truncate preserves existing behavior; error stops before saving when the visual token budget is exceeded. Master inherits the option. +- Allow larger token/frame limits and preserve the requested frame count in the motion preset. Loader budgets default to 0 (disabled). +- Add experimental H3 RefMod Text Encode with numbered reference presentation and a reference map. Support compatible ClipProj encoders and native batched video-VAE output. +- Add optional full-video inspection with correct IMAGE output dimensions; retain the lightweight first-frame default. +- Add playable music/video README examples and exclude local tests from distribution. + +Validation: 51 local regression tests passed; creation-label hooks checked with Node.js. Full GPU generation, projected-encoder quality and voice transfer remain unverified. Voice cloning is not claimed to work. + +Text Encode already attaches references: do not inject the same bundle again with Apply/Bridge. Integrated Continuum samplers still need a dedicated integration to use this conditioning. Full visual previews and text/vision presentation increase memory use. + +## v0.2.0 — 2026-09-07 + +### Known limitation: voice transfer + +Audio reference support includes an observed music-transfer example, but current +speaker-identity tests failed. This release does not provide working voice cloning. +The cause has not been isolated between H3/checkpoint behavior and the reference +presentation difference in the current RefMod integration. + +### Follow-up implementation (2026-09-07) + +- Clarify Master saving logs: deferred extraction, Created/Replaced destinations, + explicit save=False and a final token/file summary; report saved_paths in details. + +- Add Save H3 RefMods as an output node: save mixed bundles without connecting + a Preview or sampler, with prefix/subfolder options and copy deduplication. + +- Save new mods/presets in the first registered refmods root (extra_model_paths); + keep the standard models/refmods fallback and existing-file Config locations. +- Remove sibling-pack imports and startup dependency warning. Legacy av_encoder + inputs use native ComfyUI video VAE loading; standard VAE inputs are preferred. + +- Add Extract H3 RefMod Master: optional image/video and audio inputs, sequential + extraction through the shared extractors, combined token budget, one bundle + output and separate compatible visual/audio files under a shared name. + +- Step Curve now mixes the current payload without retaining tensors and handles + keyframe offsets, audio refs, empty conditioning and chained curves. +- Bridge uses the MODEL outer-sampling hook with finally cleanup; removes global + state and third-party monkey patches. The old Disarm node is deprecated. +- Align image/mask center crops; share resize and causal-frame helpers with CLI. +- Invalidate loaded files and queue cache on changes; cap cache at 256 MiB; + enforce copies/numeric limits; bound video decode buffers with unknown lengths. +- Add native audio RefMods, legacy AudioMod import, audio Extract, library UI, + inspection/optional previews, save subfolders, explicit scramble modes, + extraction presets and total-token budgets. +- Atomic saves, deduplicated config writes and grouped mean-MSE refinement. +- Historical entries below describe earlier unreleased revisions; the behavior + above and the current README supersede their bridge/cache descriptions. + +### Fixed: linked loader inputs and subfolders + +- Both loaders accept unresolved/empty mod names and connected STRING/combo + outputs. Fixed missing names still fail validation; connected names resolve + at execution with a missing-file error when necessary. +- Discover RefMods recursively as `folder/name`, preserving metadata filtering + and skipping graph presets. Listing cache tracks nested files, nanosecond + timestamps, and legacy JSON sidecars. +- Normalize Windows separators and check relative paths at execution. Cache + mods by file path so identical metadata names in different folders do not + collide when saving their configuration. +- Add regression tests using real safetensors and ComfyUI queue validation. + +### New: Continuum integration + +- **Continuum RefMod Bridge** node — injects your mods into + [ComfyUI-H3-Continuum](https://github.com/xiaolibai-sys/ComfyUI-H3-Continuum)'s + chunked long-video samplers (V2/V3/V3.4), which build their conditioning + internally and have no CONDITIONING socket. While armed, every chunk + carries the bundle's ref blocks as native `minimax_refs` — persistent + across all chunks, no Continuum markers so its continuity layout never + re-times them. Version-checked against one public seam; if Continuum + changes internally the bridge reports and degrades to no-injection rather + than breaking a run. One active bundle per graph; per-run scramble seeds + work normally. Wire it on the sampler's model line (Load Model -> bridge + -> sampler `model`): ComfyUI never executes fully disconnected nodes, so + the earlier "anywhere upstream" placement silently did nothing. + +### Fixed: why insertions felt weak, random, or "possessed" + +Three defects were found and fixed; each is reproducible with the new +`gauntlet_harness.py` (run it with ComfyUI's Python — no server needed). + +- **Apply's default curve silently HALVED every multi-frame mod.** The old + default (`concept_at_end` + `ease`) runs its envelope across the mod's own + ref frames — mean multiplier ≈ **0.50** at any frame count — so video/ + stacked mods shipped half-blurred unless you touched widgets. Worse, ref + tokens are not bound to output-video time at all: no frame curve can place + a concept "at the end of the video"; it only weights which reference + content dominates. Defaults are now `constant` + `linear` + `1.0` + (= full strength, official-ref parity); saved workflows keep their saved + values. The curve widgets remain for stack weighting, with honest tooltips. + For real output-timing control use **H3 RefMod Step Curve** (denoise + timeline). +- **Image-kind mods ignored every curve dial silently** (`curve_strengths` + short-circuits at 1 latent frame), so single-image identity mods always + injected at full strength — the "concept bleeds into everything" case. + Now `curve_value` acts as a plain strength cap on single-frame mods + (e.g. value 0.4 = the ref blends 40% toward its blurred self), with a + console note. +- **H3 RefMod Step Curve leaked state across generations.** The wrapper + cached pristine/blurred latents plus the marked-ref index mapping on the + first forward and never reset them — but ComfyUI reuses the patched MODEL + and conditioning payload across queued runs, so run #2 mixed run #1's + cached tensors whenever the scramble seed/mods/strengths changed + (corrupted, "ghost" output). The wrapper now keys its state on the + injected refs' identities (which it never mutates), re-keying cleanly on + every new run while keeping pristine stable within one; chaining multiple + Step Curve nodes composes deterministically, and progress is normalized to + each run's actual schedule start (partial denoise included). + +Also new: Apply now prints each row's *effective* strength (row × retention +× curve-mean) and flags anything below 0.30 — the usual suspect when an +insertion feels weak. + +### New: fixed configs, shipped with the mod + +- **Fix H3 RefMod Config** node — bakes tuned Apply/Step-Curve settings + (`retention` + `curve` + `step_curve`) into a mod's safetensors metadata + and re-saves the file in place, so a concept ships with the settings that + make it work. +- **`override` toggle** on Apply H3 RefMod and H3 RefMod Step Curve — reads + the first mod in the bundle that carries a fixed config and uses those + settings instead of the widgets; falls back to the manual parameters + (with a console note) when no mod has one. +- The debug curve graph reflects the overridden curve, so what you see is + what actually ran. + +### New: merge mode on Extract H3 RefMod + +- **`merge` toggle** (training mode): one shared latent refined jointly + against every reference's full encode (mean reconstruction error) instead + of stacking each ref's own pooled latent. A whole collection becomes ONE + tiny consensus mod — pulled toward what's common across all the views + (structure/motion/identity) rather than any single shot's framing or + background, at one ref block's worth of tokens. `identity` dials the + joint refinement; masks and `background_retention` still apply per ref. +- **`motion_only` toggle** (training mode, experimental): video refs are + converted to per-frame temporal differences before encoding, so the mod + carries where/how things move instead of what they look like — static + appearance (lineart look, background) never enters the latent. A soft + motion guide, not a ControlNet (the ref channel is content-based). + +### Fixed + +- **Curve directions now mean what they say.** `concept_at_end` actually + puts the concept at the **end** of the video (and at the end of the + denoise timeline on H3 RefMod Step Curve); `concept_at_start` puts it at + the start. The old envelopes were inverted relative to the names. Legacy + `decrease`/`increase` workflow values and the preset names + (`fade_in`/`fade_out`/`bump`/`dip`) still resolve to their original + envelopes, so old workflows keep their behavior. +- **H3 RefMod Step Curve no longer touches your input video.** The wrapper + used to re-mix *every* ref latent, including the original video's + identity anchor — that blurred the source subject and let a different + person "pop". It now identifies the refs injected by Apply H3 RefMod (via + a marker on the blocks) and only re-mixes those; native ref2va refs pass + through untouched, and a console note explains when there is nothing to + re-mix. + +## v0.1.0 — 2026-08-16 + +First tagged release. Extract references once as tiny `.safetensors` +"mods" and inject them through conditioning — no full video/image loading +every generation, no training. + +### Nodes + +- **Extract H3 RefMod** — image / video / GIF → one small mod file. Modes: + `training` (pooled concept/identity thumbnails; the `pool` dial trades + concept ↔ identity) and `encode` (full-resolution VAE encode). Identity + refinement steps, token cap with dedup, data multiplier for short clips, + optional `av_encoder` input, and folder bulk-loading. +- **Load H3 RefMods** — LoRA-loader-style rows with a typed strength and a + `copies` multiplier (2-10x — the manual row-duplication trick as a knob). +- **Load H3 RefMod Axis** — signed A/B sliders: negative picks the A mod, + positive the B mod, one dial controls both. +- **Load H3 RefMod Folder** — every image/video in a folder as an ordered + ref list. +- **Apply H3 RefMod** — one node for the pack's `MINIMAX_H3_COND` and the + built-in `CONDITIONING`. `retention` master strength; a curve split into + `curve_direction` (constant / concept_at_start / concept_at_middle / + concept_at_end / concept_at_ends) + `curve_shape` (linear / ease / + sigmoid / tanh / quadratic / cubic / exponential / stair / elastic / + bump / dip) + `curve_value`; `scramble_seed`; optional curve-graph + `debug` IMAGE output; shareable PNG graph presets (graph embedded in the + image metadata, legacy `.json` still loads). +- **H3 RefMod Step Curve** — the same curve widgets, but over the **denoise + timeline**: re-mixes every ref latent once per step (early steps lock + composition/identity, late steps stay clean or refine detail) via a + ComfyUI `DIFFUSION_MODEL` wrapper, attached between the model loader and + the sampler. + +### Reference math + +- Weakening a ref blends toward a blurred copy of itself instead of noise + or zero — stays on the latent manifold, so no grey/static output. +- The per-frame curve mixes each ref latent frame with + `retention * curve(x)` instead of one flat strength. +- Greedy temporal dedup + budget-fit resampling make the token cap cheap. + +### Misc + +- `extract_mod.py` standalone CLI. +- Mods live in `ComfyUI/models/refmods/` (created on first run, next to + loras/ and unet/); older mods in the pack's `mods/` folder still load. diff --git a/vendor/dumas_minimax_h3_refmod/LICENSE b/vendor/dumas_minimax_h3_refmod/LICENSE new file mode 100644 index 0000000..f4729f6 --- /dev/null +++ b/vendor/dumas_minimax_h3_refmod/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Luisa (luisacaotica) + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/vendor/dumas_minimax_h3_refmod/PLAN_0.3.0.md b/vendor/dumas_minimax_h3_refmod/PLAN_0.3.0.md new file mode 100644 index 0000000..64cc46b --- /dev/null +++ b/vendor/dumas_minimax_h3_refmod/PLAN_0.3.0.md @@ -0,0 +1,131 @@ +# MiniMaxH3Mod 0.3.0 plan + +Status: proposed scope, 2026-09-13. Direction selected by the maintainer: **Library and workflow QoL, with measured quality improvements.** This document plans implementation; it does not certify features or experimental quality results. + +## Release outcome + +Users should be able to recognize a saved RefMod, find it quickly, understand its reference-token cost, combine it with other assets, and see how to refer to it in a prompt. Existing workflows must preserve their settings. Quality claims must come from comparable generation results. + +The release has four headline additions: a visual library, cost feedback, contextual controls/reference guidance, and stack composition. Most work extends existing nodes; only stack composition needs a new node in the committed scope. + +## What the research changes—and what it does not establish + +Both supplied reports informed this plan. Their embedded instructions and example code are proposals, not implementation requirements. Their opaque citation markers are not independently reusable evidence. + +| Research point | Verified baseline / planning decision | +|---|---| +| Add aspect-aware grids | Already implemented by `core.py:aspect_grid()` and used during extraction. Show the resolved grid and token count instead of implementing another pooling path. | +| Add subject masking | `mask` and `background_retention` already exist. Improve their workflow and preview; automatic segmentation is optional external preprocessing. | +| Add a library and tags | Search/type filtering and token counts already exist in the library; RefMod objects already have tags. Extend metadata exposure and browsing. | +| Add reference labels | Text Encode already produces a `reference_map`. Make the mapping visible and usable rather than introducing a second incompatible numbering scheme. | +| RefMods solve character bleeding / merge removes backgrounds | Not established. Treat these as quality questions, not release claims. | +| Voice failure has a known architectural cause | Not established by the supplied evidence. Audio cleanup is an experiment, not a voice-cloning fix. | +| Hard 16,384-token limit / exact VRAM from tokens | Do not adopt either claim. Report reference tokens and stored latent bytes separately from measured total generation VRAM. | +| Denoise presets imply output-time morphing | Denoise progress and video playback time differ. Name presets by their actual schedule. | +| Tests are unavailable | Tests exist locally, but `/tests/` is ignored and `git ls-files tests` returns no files. Make selected regressions reproducible from a clean clone. | + +Current checkout also has `pyproject.toml` at 0.2.7 while `__init__.py` reports 0.2.6. Reconcile release metadata during packaging. Existing user changes to `pyproject.toml` and `node.zip` are not part of this planning edit. + +## Committed scope + +### 1. Visual RefMod library — P0, medium effort + +Extend the existing dialog with thumbnail cards, list/grid view, tag filtering, and sorting by name, modified date, and tokens. Expand bundle details to show member names, modalities, and per-member costs. Keep the existing slot picker and connected-slot protection. + +Creation can embed an optional small source thumbnail for browsing. Label it **Source preview**; it is not a reconstruction or a prediction of generated quality. Keep Inspect's VAE reconstruction as a separate action. Audio assets get a modality placeholder in this release; playable original audio is outside this thumbnail feature. + +Proposed storage: bounded image bytes encoded into optional metadata, with a 160-pixel longest edge, a 32 KiB encoded limit per preview, and a 256 KiB aggregate preview limit per file. Final encoding must be checked against existing save/load/update paths. Missing or oversized previews fall back to a placeholder. List responses should omit image payloads; fetch previews on demand. Never load a VAE or full latent tensors just to browse. + +Preserve optional preview metadata through standalone and bundle round trips, Config updates, and export. Test v4/v5 readers before deciding whether an additive field is sufficient; do not create v6 merely for thumbnails. Older files remain usable without regeneration, and adding previews to old files is an explicit action. + +**Acceptance:** browse a 1,000-entry metadata fixture with incremental rendering; no VAE/GPU work; search/sort/tag filters compose correctly; bundle inspection and selection preserve member order; broken previews cannot prevent loading valid RefMods; reopening the dialog preserves useful view preferences. + +### 2. Reference-cost feedback — P0, medium effort + +Show each selected slot's active reference count and token contribution, plus the combined total and budget remaining. Account for bundle member selection, copies, zero strengths, and visual/audio strengths using the same rules as execution. + +Use metadata and tensor headers for the preflight. Clearly distinguish **reference tokens**, **stored latent bytes**, and **total generation VRAM**. The first two can be computed for known inputs; the third depends on the model, output size, attention backend, and offloading. Connected or dynamically produced values show “resolved at execution” rather than a false exact total. Apply-level curves, retention, or scrambling may further change the effective total; the loader estimate must name its scope. + +Explain directly that reducing a nonzero strength generally does not save reference tokens, while copies increase them. Offer actionable budget guidance; keep execution-time validation authoritative. + +**Acceptance:** preflight agrees with loader execution for standalone/bundled visual and audio fixtures, modality exclusions, copies, and zero strength; unknown connected values are explicit; changing settings updates the display without queuing generation. + +### 3. Contextual controls and reference guidance — P0, medium effort + +Dim controls that the effective mode or preset ignores, and explain why. Preserve values and sockets so returning to Manual restores the user's choices. Derive behavior from actual backend preset resolution: do not copy the report's blanket rule that every preset disables every refinement control. + +Expose the existing Text Encode reference map clearly, with a copy action and matching asset names/previews. Explain the difference between Apply's latent injection and Text Encode's multimodal presentation. Make the single-injection workflow obvious in the example graph. + +Mapping must reflect actual active member order, modalities, exclusions, and copies. When the graph is dynamic, show the executed mapping rather than promising fixed numbers. Semantic aliases such as `hero` are a later extension, not a new model-recognized identity token. + +Add descriptive Step Curve presets such as Constant, Early emphasis, and Late emphasis only after schedule tests. Preserve manual settings. Avoid “identity lock” or “texture lock” names until output evidence supports them. + +**Acceptance:** Full/Compressed and preset transitions retain manual values; connected controls stay connected; graph reload/tab switching preserves all settings; displayed labels match the Text Encode payload; examples do not inject the same bundle twice. + +### 4. Stack composition — P1, small-to-medium effort + +Add **Join H3 RefMods**, with two `H3_REF_MODS` inputs and one `H3_REF_MODS` output. Repeated joins combine any number of loader outputs without changing the existing eight-slot loader schema or introducing another bundle type. + +Preserve A-then-B order, per-reference strengths, duplicates, and bundle provenance. Do not mutate inputs or concatenate/copy latent tensors. Existing downstream budgets remain authoritative. A larger list does not imply an unlimited model context or free memory. + +**Acceptance:** two loaders can produce more than eight selected references through joins; empty input lists work; order/strengths are exact; source lists remain unchanged; loader, Apply, Text Encode, Inspect, and Save Bundle accept the result. + +### 5. Compatibility and reproducible delivery — P0, medium effort + +Track the relevant tests in Git while continuing to exclude tests and fixtures from the install artifact where appropriate. Add CPU/schema and JavaScript checks to CI, plus real frontend workflow fixtures. Preserve node IDs, old widget positions, mode aliases, v4/v5 assets, and existing defaults. + +Test the reported legacy combination (ComfyUI 0.34.0 / frontend 1.39.2), the reported newer combination (0.35.0 / 1.52.7), and one current supported combination selected at implementation time. Record tested combinations instead of claiming every intermediate version works. Exercise classic canvas and Nodes 2 where available, including tabs, save/reopen, copy/paste, and subgraphs. + +Verify #17/#18 in real browsers; the current JavaScript regressions alone do not establish that matrix. Generate the release ZIP from the selected source revision, verify extraction/import, and synchronize version strings. Publish a small set of runnable workflow JSONs for library selection, composition, prompting, and quality comparison. + +Audit V3 registration/replacement lifecycle early. Migrate only with an API version that supports the required features and with old-workflow coverage. A wholesale V1-to-V3 conversion is not a prerequisite for library QoL. Official guidance recommends `ComfyExtension` / `comfy_entrypoint` and documents `latest` as under development: [V3 migration](https://docs.comfy.org/custom-nodes/v3_migration), [node replacement](https://docs.comfy.org/custom-nodes/backend/node-replacement). + +## Measured quality work + +Establish a baseline before changing extraction math or promoting quality presets. Use three visual cases: one identity, two similar subjects, and a short motion reference. Run at least three fixed seeds across four paths: + +1. Native Ref2VA. +2. Full RefMod + Apply. +3. Full RefMod + Text Encode. +4. Compressed RefMod + Text Encode. + +That is 36 planned visual runs. Match source media, crop/frame selection, checkpoint and encoder, prompt intent, output dimensions/duration, sampler, steps, and seed. Record differences that cannot be equalized. Add a small native-versus-RefMod music smoke comparison; speaker identity remains a separately scored experiment. + +Store workflows, settings, asset/checkpoint hashes, output media, reference tokens, VAE/encoder/sampling timings, and peak VRAM. Score identity, prompt adherence, subject swaps/bleeding, and motion consistency through paired visual review. Optional embedding metrics supplement that review. + +Only promote a quality preset when its results show a repeatable benefit on its stated use case and disclose cost/tradeoffs. If no improvement is demonstrated, ship the QoL release with the baseline and keep experimental paths labeled. Do not change defaults to manufacture a quality headline. + +## Stretch scope and deferred research + +| Proposal | Decision / promotion gate | +|---|---| +| Native text/tag loader | First stretch item after core QoL. Reuse the loader resolver, define escaping and line errors, preserve order, and support the existing runtime type. No `eval`, code execution, or ad hoc path bypasses. | +| Budget auto-fit | Experimental preview-only planner first: display proposed per-reference reductions and protected refs. Applying reductions must be explicit and leave source files intact. Automatic spatial/temporal changes require quality comparisons before promotion. | +| Stable prompt aliases / character profiles | Stretch after reference-map correctness. Aliases resolve to actual runtime labels; they do not guarantee identity separation. Defer richer profiles and a format revision until an actual consumer needs them. | +| SAM/YOLO auto-segmentation | Provide an optional example feeding the existing MASK socket. Avoid a mandatory segmentation dependency or hidden model download inside Master. | +| Background-plate latent subtraction / zeroed backgrounds | Research only. These operations do not establish clean foreground separation; compare with the existing mask path before considering production. | +| Vocal separation / audio normalization | Optional external preprocessing workflow and A/B experiment. Do not prescribe an arbitrary level or advertise a voice-cloning fix. | +| Spatial attention / bounding-box binding | Defer beyond 0.3.0 until the target H3 integration exposes a workable supported mechanism and experiments show less bleeding. Merely attaching metadata is insufficient. | +| Continuum per-chunk scheduling | Defer until chunk identity/timing and resume behavior have an explicit integration contract. Denoise schedules cannot substitute for a chunk timeline. | +| Conditioning cache / LRU changes | Profile first; defer unless current measurements show a user-visible bottleneck. Any diagnostics stay local. | +| Bundle v6 / provenance framework | Defer unless additive metadata demonstrably cannot serve a committed feature. Preserve existing asset readers. | +| Server path-policy changes | Separate opt-in deployment work; do not break desktop absolute-folder workflows as an incidental QoL change. | + +## Implementation order and release gates + +| Milestone | Deliverables | Exit condition | +|---|---|---| +| A — Baseline | Track tests, freeze old workflow fixtures, choose compatibility targets, capture quality inputs | Clean clone reproduces automated checks; benchmark workflows are runnable | +| B — Visual library | Metadata/preview persistence and endpoint, then cards/filtering/bundle details | Old and new assets round-trip; browsing performs no tensor/VAE loading | +| C — Workflow polish | Cost feedback, contextual controls, reference map, Join node, examples | Preflight/runtime agreement; stack and persistence regressions pass | +| D — Quality and RC | Execute comparisons, qualify presets, browser matrix, artifact verification | Results published, no settings-loss or connection regressions, install artifact validated | + +Start baseline generation before UI implementation; it need not wait for the library. Preview storage precedes the gallery, and correct member accounting precedes cost feedback. The text/tag loader is the first item to cut if scope grows. Auto-fit, spatial binding, and voice research never block the committed release. + +No calendar promise is attached yet: GPU benchmark availability and the oldest supported frontend are the main scheduling uncertainties. The next implementation step is milestone A followed by the preview metadata contract—not a broad node rewrite. + +## Source notes + +- Supplied `C:/Users/Pichau/Downloads/deep-research-report.md`: architecture, reproducibility, compatibility, and benchmark proposals. +- Supplied `C:/Users/Pichau/Downloads/research.md`: feature and QoL candidates, treated as hypotheses where their quality claims exceed evidence. +- Locally checked: `core.py`, `nodes.py`, `prompt.py`, `library.py`, `web/library_dialog.js`, `.gitignore`, `CHANGELOG.md`, `__init__.py`, and `pyproject.toml`. diff --git a/vendor/dumas_minimax_h3_refmod/README.md b/vendor/dumas_minimax_h3_refmod/README.md new file mode 100644 index 0000000..b6be0ff --- /dev/null +++ b/vendor/dumas_minimax_h3_refmod/README.md @@ -0,0 +1,885 @@ +*Update: https://github.com/Luisacaotica/ComfyUI-MiniMaxH3Mod/discussions/36 + +# ComfyUI-MiniMaxH3Mod - Changing the F#cking world with cigarretes and coffe. + +> 🚧 **Under construction** — API and node schemas are still evolving. Mods +> stay compatible, but expect node names/inputs to shift between versions. + +[![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/C0C2EV9GW) + +## What's new — v0.2.6 + +- **Standalone creators** — Create H3 RefMod and Master now run without a + downstream output node. Both return a `details` string with token counts and + saved paths. `save=False` still produces an in-memory bundle without files. +- **Different-size masks** — route a MASK list through **Collect H3 RefMod + Masks** into the creator's `mask_list` input. Each mask retains its dimensions + until it is resized/cropped with its corresponding reference. The collector + consumes the list once, avoiding one extraction per list item. Use `mask` or + `mask_list`; one mask broadcasts, otherwise provide one per reference in + image-slot, video-slot, then folder-bundle order. Existing MASK batches work + as before. Pixels cropped by an upstream batch node cannot be recovered. +- **Full Reference video sampling** — choose a causal frame count before + sampling, so frame limits such as 16 no longer drop the end of the clip. + +- **Compact additive loaders** — Loader and Axis start with one visible slot. + Use **+ Add RefMod** to reveal more, up to the existing eight slots. Each + reference's controls stay together; connected slots are protected from removal. +- **Single-file visual + audio bundles** — Master can save both modalities in + one `.safetensors` with `save_layout=bundle`. **Save H3 RefMod Bundle** also + packs existing references without running extraction again. +- **Independent loader controls** — each Loader/Axis slot offers **All / Visual / + Audio**, plus visual and audio strengths. Excluded modalities and modalities + with strength zero are not loaded. Existing standalone files remain supported. +- **Bundle-aware library and Config** — find combined files in the library and + edit selected components without dropping the other contents. + +This is a storage and selection update. It does not fix voice identity or enforce +audiovisual synchronization. See [single-file bundles](#single-file-bundles). + +### v0.2.5 and subsequent fixes + +On `main` after v0.2.5: **Refresh RefMods** updates the current Loader/Axis +dropdowns from the RefMod folders without refreshing all ComfyUI models. +Selections and connected inputs are preserved. The library's Refresh also +updates that loader's dropdowns. + +- **Clearer creation controls** — Create node titles, Full/Compressed Reference modes and Refinement Steps; old workflows remain accepted. +- **Explicit visual budget policy** — choose truncate or error before saving, including Master. +- **Compatibility fixes** — projected ClipProj encoders and batched H3 VAE decoding in Text Encode. +- **Full visual inspection** — optional full-video preview, with a lightweight first-frame default. + +Since v0.2.0: extraction widgets accept token budgets above +65,536 and frame limits above 16, including Master. The `motion_sequence` +preset now respects the requested frame limit. Counts remain integers; +defaults are unchanged. + +**Experimental: H3 RefMod Text Encode** presents saved references to the native +H3 text/vision encoder and reports their ``, `