Compare commits
25 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6c6ead6f24 | ||
|
|
fdc9dac5f1 | ||
|
|
8c10e887c8 | ||
|
|
93ed99b9d7 | ||
|
|
750c8961c9 | ||
|
|
0d6d848a26 | ||
|
|
2a684aaa4c | ||
|
|
b67ad92eef | ||
|
|
722777cc13 | ||
|
|
21dd076689 | ||
|
|
4411e71be4 | ||
|
|
142cc0fdbc | ||
|
|
d105088195 | ||
|
|
6a63adc563 | ||
|
|
fd164b480c | ||
|
|
6542586cdc | ||
|
|
d12e1501b7 | ||
|
|
c30e8963ef | ||
|
|
6a2c0ad51f | ||
|
|
32e8b9f898 | ||
|
|
38835f62d3 | ||
|
|
92e1f32859 | ||
|
|
3fc4219a38 | ||
|
|
656a665705 | ||
|
|
3337bf18ff |
151
README.md
151
README.md
@@ -1,5 +1,14 @@
|
||||
|
||||
A Complete Timeline Editor For LTX 2.3. This is the sucessor of my previous nodes, and has loads of features in it. It was originally based off of [Kijai's Prompt Relay node](https://github.com/kijai/ComfyUI-PromptRelay) and my LTX Sequencer/Multi Image Loader nodes.
|
||||
|
||||
https://github.com/user-attachments/assets/68dc826f-c65f-4f1e-86cd-4f2df17bacd8
|
||||
|
||||
|
||||
|
||||
# Overview
|
||||
|
||||
Just added a couple of things from Director Node from "WhatdreamsCost" or Jonathan that I thought would be cool
|
||||
|
||||
This will be a collection of free resources for ComfyUI.
|
||||
|
||||
Hopefully it will make creating cool stuff easier.
|
||||
@@ -9,8 +18,9 @@ All of my nodes are created with the help of AI, so there may or may not be redu
|
||||
## ▶️ YouTube Tutorial Videos
|
||||
|
||||
<a href="https://youtu.be/j28z5PZXkKk?si=TPeWKm2BW-uA3BAx">
|
||||
<img width="642" height="718" alt="Capture d'écran 2026-06-17 234136" src="https://github.com/user-attachments/assets/89433110-8b78-4971-8098-3a9e9226f460" />
|
||||
|
||||
<p align="center">
|
||||
<img width="620" height="675" alt="Capture d'écran 2026-06-27 143149" src="https://github.com/user-attachments/assets/da3ded29-8c98-4f05-8246-7b9eb72b99f2" />
|
||||
</p>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
@@ -32,14 +42,13 @@ All of my nodes are created with the help of AI, so there may or may not be redu
|
||||
## ❓ How to install nodes
|
||||
|
||||
- Navigate to your `/ComfyUI/custom_nodes/ folder`
|
||||
- Delete your old "WhatDreamsCost-ComfyUI" Folder
|
||||
- Run `git clone -b main_cs https://github.com/CGlide/WhatDreamsCost-ComfyUI.git`
|
||||
- Run `git clone -b main_cs https://github.com/CGlide/WhatDreamsCost-CSGlide.git`
|
||||
|
||||
|
||||
**❗❗IMPORTANT❗❗**
|
||||
|
||||
|
||||
This is a Modded LTX director node with some extra options to help you create videos with references sheets
|
||||
This is a Modded LTX director node from "WhatDreamsCost" with some extra options to help you create videos with references sheets
|
||||
|
||||
This node uses Ollama Locally : you will need to install it (very lightweight) and
|
||||
|
||||
@@ -79,81 +88,12 @@ If anyone finds anymore bugs or has idea for improvements please let me know!
|
||||
* **LTX Director Example Workflow Fix**
|
||||
- Minor fix to the example workflow (i forgot to set the clip loader type to ltxv lol)
|
||||
|
||||
**v1.3.0**
|
||||
* **New nodes: LTX Director and LTX Director Guide**
|
||||
- A complete timeline editor that can do almost everything. It's my most ambitious node so far and the successor to LTX Sequencer/Multi Image Loader.
|
||||
|
||||
**v1.2.9**
|
||||
* **Fixed every known issue with Multi Image Loader and added text output to Speech Length Calculator**
|
||||
|
||||
- Removed the completely useless drag and drop animations (now it's snappy and no longer finicky)
|
||||
- Fixed the node resizing on nodes 2.0
|
||||
- Updated grid logic to fit images better
|
||||
- Added ablity to right click images to copy/open/save images
|
||||
- Fixed the "invisible hitbox" underneath node issue (actually this time).
|
||||
|
||||
Also added a text output to the Speech Length Calculator node (can't believe i didn't do this initially)
|
||||
|
||||
<details>
|
||||
<summary>Click to view older Updates</summary>
|
||||
|
||||
**v1.2.8**
|
||||
* **Updated Load Video UI and Color Conversion**
|
||||
* Added crop mode, a simple interface to crop videos. It also include various aspect ratio presets.
|
||||
* Updated color conversion to ensure colors are as accurate as possible. Will first check metadata for colorspace, and if metadata is missing then it will guess the colorspace based on video dimensions.
|
||||
* Updated display mode toggle UI to be more understandable
|
||||
|
||||
**v1.2.7**
|
||||
* **New Node: Load Video UI**
|
||||
|
||||
Custom Node to Trim, Resize, and Preview Videos in Realtime
|
||||
|
||||
**v1.2.6**
|
||||
* **Updated Speech Length Calculator UI**
|
||||
|
||||
Also added duration output to the Load Audio UI node
|
||||
|
||||
**v1.2.5**
|
||||
* **Updated Load Audio UI Node**
|
||||
* Added Duration Setting
|
||||
* Made the whole selection bar draggable
|
||||
* Fixed Trimmed UI to show centiseconds
|
||||
|
||||
**v1.2.4**
|
||||
* **New Node: Load Audio UI**
|
||||
|
||||
Overhaul of the load audio node. Features a simple interface to easily trim audio. Also allows dragging and dropping files (fixes the original node that doesn't allow dropping in videos). Also compatible with nodes 2.0.
|
||||
|
||||
**v1.2.3**
|
||||
* **Workflow Update + Minor Bug Fix**
|
||||
* Added new workflow that is compatible with the latest ComfyUI version (as of 4/27/26). The new workflow also included an option to include custom audio, and has minor improvements of the previous workflows.
|
||||
* Fixed minor bug with Multi Image Loader that blocked mouse input in a small area under the node 🤷♂️
|
||||
|
||||
**v1.2.0**
|
||||
* **New Node: Speech Length Calculator**
|
||||
|
||||
Automatically output in realtime how long a video should be based on the dialouge.
|
||||
|
||||
**v1.1.0**
|
||||
* Added resize_method to the Multi Image Loader node for more resize options
|
||||
* Added insert_mode which allows you to enter in seconds instead of frames on the LTX Sequencer node
|
||||
* Updated workflows with more notes
|
||||
* Re-added tiny vae to workflows
|
||||
* Fixed various bugs
|
||||
* more things i can't rememeber
|
||||
|
||||
**This update will change the node layouts, so be sure to update your workflows or else they won't work properly.**
|
||||
|
||||
❗❗❗ **New Tutorial on using these nodes available: https://www.youtube.com/watch?v=aXDIr8eNovI** ❗❗❗
|
||||
</details>
|
||||
|
||||
# ⚙️ Custom Nodes
|
||||
|
||||
|
||||
## LTX Director
|
||||
<img width="1481" height="833" alt="Clipboard Image (2)" src="https://github.com/user-attachments/assets/08f3fe53-9393-4f5d-9de5-58b229fbed47" />
|
||||
|
||||
A Complete Timeline Editor For LTX 2.3. This is the sucessor of my previous nodes, and has loads of features in it. It was originally based off of [Kijai's Prompt Relay node](https://github.com/kijai/ComfyUI-PromptRelay) and my LTX Sequencer/Multi Image Loader nodes.
|
||||
|
||||
|
||||
**Main Features:**
|
||||
- **Fully Functional Timeline Editor:** I spent hours studying various video editors and ended up with this design. If anyone has ideas for improvements let me know! I will adding documentation on all the functions soon.
|
||||
@@ -168,55 +108,7 @@ Download workflows here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyU
|
||||
**Tutorial videos and documentation coming soon**
|
||||
|
||||
|
||||
## Multi Image Loader
|
||||
<img width="1280" height="720" alt="Multi_Image_Loader_Wide_Gif" src="https://github.com/user-attachments/assets/99b6afd8-5197-4e6c-81da-a7bd156c42c7" />
|
||||
|
||||
An Image loader that features a built in gallery, allowing your to easily rearrange images and output them seperately or batched together. It also combines the image resize node and LTXVPreprocess node to reduce clutter in LTX workflows.
|
||||
|
||||
## LTX Sequencer
|
||||

|
||||
|
||||
An overhaul of the LTXVAddGuideMulti node. It allows you to quickly create FFLF (First Frame Last Frame) videos, shot sequences, supports any number of middle frames.
|
||||
|
||||
Connect the Multi Image Loader node's multi_output to automatically update the node's widgets.
|
||||
|
||||
It also has a sync feature that syncs all LTX Sequencer nodes together in realtime, removing the need to edit every single node manually every time you want to make a change to something.
|
||||
|
||||
|
||||
## LTX Keyframer
|
||||
<img width="1082" height="608" alt="LTX Keyframer Wide" src="https://github.com/user-attachments/assets/850ba4a2-dbca-4e5a-a580-1c271e9f0c41" />
|
||||
|
||||
An overhaul of the LTXVImgToVideoInplaceKJ node. It allows you to quickly create FFLF (First Frame Last Frame) videos and shot sequences. Also upports any number of middle frames.
|
||||
|
||||
Connect the Multi Image Loader node's multi_output to automatically update the node's widgets.
|
||||
|
||||
It also has a sync feature that syncs all LTX Keyframer nodes together in realtime, removing the need to edit every single node manually every time you want to make a change to something.
|
||||
|
||||
**I would recommend using the LTX Sequencer Node over this node, after further testing it seems superior in at pretty much everything. I'll leave it in just in case more people want to test it**
|
||||
|
||||
## Speech Length Calculator
|
||||
<img width="1280" height="720" alt="Speech Length Calculator v2 Gif" src="https://github.com/user-attachments/assets/04b9a1cf-20e4-4b7b-a9c6-4a5a0825995b" />
|
||||
<br>
|
||||
<br>
|
||||
This node calculates in realtime how long a video should be based on the dialogue. Any words in quotations will be considered as speech. The node updates in realtime without having to run the workflow, and outputs the length depending on how fast the speech is.
|
||||
|
||||
If you connect another string/text node to the text_input, it will still update in the length in realtime.
|
||||
|
||||
I kept having to play the guessing game on my own generations so I made this node to make it easier :man_shrugging:
|
||||
|
||||
## Load Video UI
|
||||
<table width="100%">
|
||||
<tr>
|
||||
<td width="50%" align="center">
|
||||
<p>Simple Controls</p>
|
||||
<img src="https://github.com/user-attachments/assets/fb76ff03-a6ff-4837-bd63-7e429f5f3d37" width="100%" />
|
||||
</td>
|
||||
<td width="50%" align="center">
|
||||
<p>New Crop Mode!</p>
|
||||
<img src="https://github.com/user-attachments/assets/28cfb4ca-e42a-44da-9afb-f20cb01b9722" width="100%" />
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<br>
|
||||
<br>
|
||||
@@ -232,22 +124,9 @@ An upgraded Load Video node. It has the following features:
|
||||
|
||||
Please note that due to ComfyUI limitations (and the fact that this node doesn't use any addtional libraries), this node will not work well for outputting large videos. You can trim any length of video without a problem, but if the output is still large it will end up using a lot of RAM. I have implemented various optimizations though to make it use less memory.
|
||||
|
||||
## Load Audio UI
|
||||
<img width="1280" height="720" alt="Load_Audio_UI_V2" src="https://github.com/user-attachments/assets/e3dc5c8d-d0b9-4336-8196-944204719239" />
|
||||
<br>
|
||||
<br>
|
||||
An upgraded Load Audio node. Features a simple interface to easily trim audio. Also allows dragging and dropping files (fixes the original node that doesn't allow dropping in videos). Also compatible with nodes 2.0.
|
||||
|
||||
# 💡 Workflows
|
||||
<img width="3120" height="990" alt="LTX I2V First Last Frame 3 Stage Workflow v6" src="https://github.com/user-attachments/assets/c993ef2f-ac4b-4091-a7f6-5ff1674c3718" />
|
||||
<br>
|
||||
<br>
|
||||
This is a compact LTX 2.3 workflow for I2V and First Frame, Middle Frame, Last frame video generation.
|
||||
I seperated and organized everything into subraphs to make things as clean as possible, and added toggles to customize the workflow quickly.
|
||||
|
||||
Download workflows here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/tree/main/example_workflows
|
||||
|
||||
Or drag and drop the image into ComfyUI to import workflow.
|
||||
|
||||
# ❗ Known Issues
|
||||
|
||||
|
||||
83
__init__.py
83
__init__.py
@@ -1,50 +1,33 @@
|
||||
from .ltx_keyframer import LTXKeyframer
|
||||
from .multi_image_loader import MultiImageLoader
|
||||
from .ltx_sequencer import LTXSequencer
|
||||
from .speech_length_calculator import SpeechLengthCalculator
|
||||
from .load_audio_ui import LoadAudioUI
|
||||
from .load_video_ui import LoadVideoUI
|
||||
from .ltx_director import LTXDirector
|
||||
from .ltx_director_guide import LTXDirectorGuide
|
||||
from comfy_api.latest import ComfyExtension, io
|
||||
from typing_extensions import override
|
||||
from .latent_slice import CleanLatentSlice
|
||||
|
||||
class PromptRelay(ComfyExtension):
|
||||
@override
|
||||
async def get_node_list(self) -> list[type[io.ComfyNode]]:
|
||||
return [
|
||||
LTXDirector,
|
||||
LTXDirectorGuide
|
||||
]
|
||||
|
||||
async def comfy_entrypoint() -> PromptRelay:
|
||||
return PromptRelay()
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"LTXKeyframer": LTXKeyframer,
|
||||
"MultiImageLoader": MultiImageLoader,
|
||||
"LTXSequencer": LTXSequencer,
|
||||
"SpeechLengthCalculator": SpeechLengthCalculator,
|
||||
"LoadAudioUI": LoadAudioUI,
|
||||
"LoadVideoUI": LoadVideoUI,
|
||||
"LTXDirector": LTXDirector,
|
||||
"LTXDirectorGuide": LTXDirectorGuide,
|
||||
"CleanLatentSlice": CleanLatentSlice,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"LTXKeyframer": "LTX Keyframer",
|
||||
"MultiImageLoader": "Multi Image Loader",
|
||||
"LTXSequencer": "LTX Sequencer",
|
||||
"SpeechLengthCalculator": "Speech Length Calculator",
|
||||
"LoadAudioUI": "Load Audio UI",
|
||||
"LoadVideoUI": "Load Video UI",
|
||||
"LTXDirector": "LTX Director",
|
||||
"LTXDirectorGuide": "LTX Director Guide",
|
||||
"CleanLatentSlice": "Clean Latent Slice",
|
||||
}
|
||||
|
||||
WEB_DIRECTORY = "./js"
|
||||
|
||||
__all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS', 'WEB_DIRECTORY']
|
||||
from .ltx_director import LTXDirector
|
||||
from .ltx_director_guide import LTXDirectorGuide, LTXDirectorCropGuides
|
||||
from comfy_api.latest import ComfyExtension, io
|
||||
from typing_extensions import override
|
||||
from .latent_slice import CleanLatentSlice
|
||||
|
||||
class PromptRelay(ComfyExtension):
|
||||
@override
|
||||
async def get_node_list(self) -> list[type[io.ComfyNode]]:
|
||||
return [
|
||||
LTXDirector,
|
||||
]
|
||||
|
||||
async def comfy_entrypoint() -> PromptRelay:
|
||||
return PromptRelay()
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"LTXDirectorCS": LTXDirector,
|
||||
"LTXDirectorGuideCS": LTXDirectorGuide,
|
||||
"LTXDirectorCropGuidesCS": LTXDirectorCropGuides,
|
||||
"CleanLatentSliceCS": CleanLatentSlice,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"LTXDirectorCS": "LTX Director CS",
|
||||
"LTXDirectorGuideCS": "LTX Director Guide CS",
|
||||
"LTXDirectorCropGuidesCS": "LTX Director Crop Guides CS",
|
||||
"CleanLatentSliceCS": "Clean Latent Slice CS",
|
||||
}
|
||||
|
||||
WEB_DIRECTORY = "./js"
|
||||
|
||||
__all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS', 'WEB_DIRECTORY']
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
379053
debug_prompt_relay.log
Normal file
379053
debug_prompt_relay.log
Normal file
File diff suppressed because it is too large
Load Diff
1
example_workflows/LTX Director CS.json
Normal file
1
example_workflows/LTX Director CS.json
Normal file
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load Diff
Binary file not shown.
|
Before Width: | Height: | Size: 825 KiB |
File diff suppressed because it is too large
Load Diff
Binary file not shown.
|
Before Width: | Height: | Size: 290 KiB |
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
1514
js/load_video_ui.js
1514
js/load_video_ui.js
File diff suppressed because it is too large
Load Diff
11432
js/ltx_director.js
11432
js/ltx_director.js
File diff suppressed because it is too large
Load Diff
@@ -3,15 +3,22 @@ import { app } from "../../scripts/app.js";
|
||||
// LTX Director Guide is a pure pass-through processor node.
|
||||
// All configuration (images, insert frames, strengths) comes from
|
||||
// the guide_data output of Prompt Relay Encode (Timeline).
|
||||
// No dynamic widgets or sync logic needed.
|
||||
app.registerExtension({
|
||||
name: "Comfy.LTXDirectorGuide",
|
||||
name: "Comfy.LTXDirectorGuideCS",
|
||||
async nodeCreated(node) {
|
||||
if (node.comfyClass !== "LTXDirectorGuide") return;
|
||||
// Nothing to initialize — the node has no configurable widgets.
|
||||
},
|
||||
});
|
||||
if (node.comfyClass !== "LTXDirectorGuideCS") return;
|
||||
|
||||
// Nothing to initialize — the node has no configurable widgets.
|
||||
// Hide retake_mode widget on LiteGraph as it is dynamically auto-detected from the timeline data.
|
||||
const w = node.widgets?.find(x => x.name === "retake_mode");
|
||||
if (w) {
|
||||
w.hidden = true;
|
||||
if (!w.options) w.options = {};
|
||||
w.options.hidden = true;
|
||||
if (!window.LiteGraph || !window.LiteGraph.vueNodesMode) {
|
||||
w.computeSize = () => [0, -4];
|
||||
w.draw = () => { };
|
||||
}
|
||||
if (w.element) w.element.style.display = "none";
|
||||
}
|
||||
},
|
||||
});
|
||||
@@ -1,419 +0,0 @@
|
||||
import { app } from "../../scripts/app.js";
|
||||
|
||||
// Global registry to track all LTXKeyframer nodes across all subgraphs
|
||||
window._LTXKeyframerGlobalNodes = window._LTXKeyframerGlobalNodes || new Set();
|
||||
|
||||
// ComfyUI native trick to cleanly hide/show widgets without deleting them
|
||||
function toggleWidget(widget, visible) {
|
||||
if (visible) {
|
||||
if (widget.origType !== undefined) {
|
||||
widget.type = widget.origType;
|
||||
widget.computeSize = widget.origComputeSize;
|
||||
delete widget.origType;
|
||||
delete widget.origComputeSize;
|
||||
}
|
||||
} else {
|
||||
if (widget.type !== "hidden") {
|
||||
widget.origType = widget.type;
|
||||
widget.origComputeSize = widget.computeSize;
|
||||
widget.type = "hidden";
|
||||
widget.computeSize = () => [0, -4];
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --- NEW SYNC HELPER FUNCTION ---
|
||||
// Finds all other LTXKeyframer nodes globally and mirrors the value to them
|
||||
function syncWidgetAcrossNodes(sourceNode, widgetName, value) {
|
||||
if (!window._LTXKeyframerGlobalNodes) return;
|
||||
|
||||
for (const targetNode of window._LTXKeyframerGlobalNodes) {
|
||||
// Target all OTHER LTXKeyframer nodes by direct object reference
|
||||
if (targetNode !== sourceNode) {
|
||||
|
||||
// 1. Always update the hidden properties cache so it remembers the sync
|
||||
// even if the widget isn't currently visible (e.g. fewer images loaded right now)
|
||||
targetNode.properties[widgetName] = value;
|
||||
|
||||
// 2. If the widget is currently visible on the UI, update it visually
|
||||
if (targetNode.widgets) {
|
||||
const targetWidget = targetNode.widgets.find(w => w.name === widgetName);
|
||||
if (targetWidget && targetWidget.value !== value) {
|
||||
targetWidget.value = value;
|
||||
targetNode.setDirtyCanvas(true, false);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "Comfy.LTXKeyframer.DynamicInputs",
|
||||
async nodeCreated(node) {
|
||||
if (node.comfyClass !== "LTXKeyframer") return;
|
||||
|
||||
// Register this node instance globally
|
||||
window._LTXKeyframerGlobalNodes.add(node);
|
||||
|
||||
node._currentImageCount = -1; // Force first update
|
||||
|
||||
// Initialize persistent properties cache
|
||||
node.properties = node.properties || {};
|
||||
|
||||
// Add subtle separator line above images_loaded
|
||||
node.addCustomWidget({
|
||||
name: "num_images_separator",
|
||||
type: "text",
|
||||
draw(ctx, node, widget_width, y, widget_height) {
|
||||
ctx.save();
|
||||
ctx.strokeStyle = "#444";
|
||||
ctx.lineWidth = 1;
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(10, y + 5);
|
||||
ctx.lineTo(widget_width - 10, y + 5);
|
||||
ctx.stroke();
|
||||
ctx.restore();
|
||||
},
|
||||
computeSize(width) {
|
||||
return [width, 10];
|
||||
}
|
||||
});
|
||||
|
||||
// Move separator before num_images
|
||||
const moveSeparator = () => {
|
||||
const idx = node.widgets.findIndex(w => w.name === "num_images");
|
||||
const sepIdx = node.widgets.findIndex(w => w.name === "num_images_separator");
|
||||
if (idx !== -1 && sepIdx !== -1) {
|
||||
const separator = node.widgets.splice(sepIdx, 1)[0];
|
||||
node.widgets.splice(idx, 0, separator);
|
||||
}
|
||||
};
|
||||
setTimeout(moveSeparator, 50); // Small delay to ensure num_images is present
|
||||
|
||||
// Core update: synchronize widget visibility to match imageCount
|
||||
node._applyWidgetCount = function(count) {
|
||||
const isInitialLoad = this._currentImageCount === -1;
|
||||
|
||||
if (this._currentImageCount === count && !isInitialLoad) return;
|
||||
this._currentImageCount = count;
|
||||
|
||||
const initialWidth = this.size[0];
|
||||
const numWidget = this.widgets?.find(w => w.name === "num_images");
|
||||
if (numWidget) {
|
||||
numWidget.label = "images_loaded";
|
||||
numWidget.value = Math.max(0, Math.min(count || 0, 50));
|
||||
}
|
||||
|
||||
// 1. Store current widget values in properties BEFORE removing them
|
||||
// We skip reading from `this.widgets` on the initial load because it might be scrambling.
|
||||
if (!isInitialLoad && this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) {
|
||||
this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// 2. Remove all existing dynamic insert_frame/strength/header widgets
|
||||
if (this.widgets) {
|
||||
this.widgets = this.widgets.filter(w =>
|
||||
!w.name.startsWith("insert_frame_") &&
|
||||
!w.name.startsWith("strength_") &&
|
||||
!w.name.startsWith("header_")
|
||||
);
|
||||
} else {
|
||||
this.widgets = [];
|
||||
}
|
||||
|
||||
// 3. Add back exactly the right amount of widgets using the cached values
|
||||
for (let i = 1; i <= count; i++) {
|
||||
// Add header/separator widget for grouping
|
||||
const headerName = `header_${i}`;
|
||||
this.addCustomWidget({
|
||||
name: headerName,
|
||||
type: "text",
|
||||
value: `Image #${i}`,
|
||||
draw(ctx, node, widget_width, y, widget_height) {
|
||||
ctx.save();
|
||||
const margin = 10;
|
||||
const topPadding = 15;
|
||||
|
||||
// Subtle separator line
|
||||
ctx.strokeStyle = "#333";
|
||||
ctx.lineWidth = 1;
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(margin, y + 5);
|
||||
ctx.lineTo(widget_width - margin, y + 5);
|
||||
ctx.stroke();
|
||||
|
||||
// Text label
|
||||
ctx.fillStyle = "#dddddd"; // Light gray
|
||||
ctx.font = "bold 12px Arial";
|
||||
ctx.textAlign = "left";
|
||||
ctx.fillText(`Image #${i}`, margin, y + topPadding + 10);
|
||||
ctx.restore();
|
||||
},
|
||||
computeSize(width) {
|
||||
return [width, 35]; // Vertical gap + label height
|
||||
}
|
||||
});
|
||||
|
||||
const insertFrameWidgetName = `insert_frame_${i}`;
|
||||
const strengthWidgetName = `strength_${i}`;
|
||||
|
||||
// Add insert_frame widget with Sync Callback
|
||||
const savedInsertFrameValue = this.properties[insertFrameWidgetName];
|
||||
this.addWidget("number", insertFrameWidgetName,
|
||||
savedInsertFrameValue !== undefined ? savedInsertFrameValue : 0,
|
||||
(value) => {
|
||||
const rounded = Math.round(value);
|
||||
this.properties[insertFrameWidgetName] = rounded;
|
||||
syncWidgetAcrossNodes(this, insertFrameWidgetName, rounded); // Sync out
|
||||
}, { min: -9999, max: 9999, step: 10, precision: 0 }
|
||||
);
|
||||
|
||||
// Add strength widget with Sync Callback
|
||||
const savedStrengthValue = this.properties[strengthWidgetName];
|
||||
this.addWidget("number", strengthWidgetName,
|
||||
savedStrengthValue !== undefined ? savedStrengthValue : 1.0,
|
||||
(value) => {
|
||||
this.properties[strengthWidgetName] = value;
|
||||
syncWidgetAcrossNodes(this, strengthWidgetName, value); // Sync out
|
||||
}, { min: 0.0, max: 1.0, step: 0.01 }
|
||||
);
|
||||
}
|
||||
|
||||
this.setDirtyCanvas(true, true);
|
||||
requestAnimationFrame(() => {
|
||||
if (this.computeSize) {
|
||||
this.setSize(this.computeSize());
|
||||
this.size[0] = initialWidth; // keep width fixed when restructuring
|
||||
}
|
||||
});
|
||||
};
|
||||
|
||||
// --- STRICT ARRAY MAPPER: FIXES ALL SHIFTING FOREVER ---
|
||||
// This runs the exact instant the node is loaded, before any UI widgets shift indices.
|
||||
// It locks the perfectly mapped array values directly into our properties dictionary.
|
||||
const origConfigure = node.configure;
|
||||
node.configure = function(info) {
|
||||
if (origConfigure) {
|
||||
origConfigure.apply(this, arguments);
|
||||
}
|
||||
if (this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name === "num_images" || w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) {
|
||||
this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
// Handle deserialization to load properties properly from JSON
|
||||
const originalOnConfigure = node.onConfigure;
|
||||
node.onConfigure = function(info) {
|
||||
if (originalOnConfigure) {
|
||||
originalOnConfigure.apply(this, arguments);
|
||||
}
|
||||
if (info.properties) {
|
||||
this.properties = { ...this.properties, ...info.properties };
|
||||
}
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
// Fallback to properties.num_images if source node disconnected
|
||||
let targetCount = count !== null ? count : (this.properties.num_images || 0);
|
||||
this._applyWidgetCount(targetCount);
|
||||
}, 100);
|
||||
};
|
||||
|
||||
// --- STRICT ARRAY GENERATOR ---
|
||||
// Completely detach from ComfyUI's blind visual array saving.
|
||||
// We construct an exact 101-element strict array that Python expects.
|
||||
// This makes your node 100% immune to UI/Header index shifting.
|
||||
const originalOnSerialize = node.onSerialize;
|
||||
node.onSerialize = function(info) {
|
||||
// Ensure properties are strictly synced with current widget values before building
|
||||
if (this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name === "num_images" || w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) {
|
||||
this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
if (originalOnSerialize) {
|
||||
originalOnSerialize.apply(this, arguments);
|
||||
}
|
||||
|
||||
info.properties = { ...this.properties };
|
||||
|
||||
// Build the exact strict array that maps 1-to-1 to the Python backend
|
||||
const strictArray = [];
|
||||
const numWidgetVal = this.properties["num_images"];
|
||||
strictArray.push(numWidgetVal !== undefined ? numWidgetVal : 1);
|
||||
|
||||
for (let i = 1; i <= 50; i++) {
|
||||
const fVal = this.properties[`insert_frame_${i}`];
|
||||
const sVal = this.properties[`strength_${i}`];
|
||||
strictArray.push(fVal !== undefined ? fVal : 0);
|
||||
strictArray.push(sVal !== undefined ? sVal : 1.0);
|
||||
}
|
||||
|
||||
info.widgets_values = strictArray;
|
||||
};
|
||||
|
||||
// Set up manual num_images widget callback
|
||||
setTimeout(() => {
|
||||
const numWidget = node.widgets?.find(w => w.name === "num_images");
|
||||
if (numWidget) {
|
||||
numWidget.callback = (val) => {
|
||||
node.properties["num_images"] = val;
|
||||
node._applyWidgetCount(val);
|
||||
};
|
||||
}
|
||||
}, 100);
|
||||
|
||||
// Exposed receiver for push-based notifications
|
||||
node._syncImageCount = function(count) {
|
||||
this._applyWidgetCount(count);
|
||||
};
|
||||
|
||||
// Helper: read image count from a connected MultiImageLoader node
|
||||
function readSourceImageCount(self) {
|
||||
const multiInput = self.inputs?.find(inp => inp.name === "multi_input");
|
||||
if (!multiInput || !multiInput.link) return null;
|
||||
|
||||
const nodeGraph = self.graph || app.graph;
|
||||
|
||||
// Helper to safely trace back through Reroutes and ComfyUI Group Nodes/Subgraphs
|
||||
function traceUpstream(graph, linkId, visited = new Set()) {
|
||||
if (!linkId || visited.has(linkId)) return null;
|
||||
visited.add(linkId);
|
||||
|
||||
const link = graph.links[linkId];
|
||||
if (!link) return null;
|
||||
|
||||
const originNode = graph.getNodeById(link.origin_id);
|
||||
if (!originNode) return null;
|
||||
|
||||
if (originNode.comfyClass === "MultiImageLoader") {
|
||||
return originNode;
|
||||
}
|
||||
|
||||
// Traverse Reroute nodes
|
||||
if (originNode.type === "Reroute" || originNode.comfyClass === "Reroute") {
|
||||
if (originNode.inputs && originNode.inputs.length > 0 && originNode.inputs[0].link) {
|
||||
return traceUpstream(graph, originNode.inputs[0].link, visited);
|
||||
}
|
||||
}
|
||||
|
||||
// Traverse standard ComfyUI Group Nodes (subgraphs)
|
||||
if (typeof originNode.getInnerNode === "function") {
|
||||
try {
|
||||
const innerNode = originNode.getInnerNode(link.origin_slot);
|
||||
if (innerNode && innerNode.comfyClass === "MultiImageLoader") {
|
||||
return innerNode;
|
||||
}
|
||||
} catch (e) {
|
||||
console.warn("Could not trace inner node", e);
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
let sourceNode = traceUpstream(nodeGraph, multiInput.link);
|
||||
|
||||
// Helper to extract the count once we find a node
|
||||
function getCountFromNode(n) {
|
||||
if (typeof n._imageCount === "number") return n._imageCount;
|
||||
const pathsWidget = n.widgets?.find(w => w.name === "image_paths");
|
||||
if (pathsWidget) {
|
||||
return (pathsWidget.value || "").split('\n').map(p => p.trim()).filter(p => p.length > 0).length;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
if (sourceNode) {
|
||||
return getCountFromNode(sourceNode);
|
||||
}
|
||||
|
||||
// Fallback Strategy: If it is connected to something, but we couldn't resolve it
|
||||
// directly (e.g. complex nested 3rd party subgraphs), scan the entire UI.
|
||||
// If there is EXACTLY ONE MultiImageLoader in the workspace, safely assume that's the one.
|
||||
let multiImageLoaders = [];
|
||||
function findAllLoaders(nodes) {
|
||||
if (!nodes) return;
|
||||
for (let n of nodes) {
|
||||
if (n.comfyClass === "MultiImageLoader") {
|
||||
multiImageLoaders.push(n);
|
||||
}
|
||||
if (n.subgraph && n.subgraph._nodes) {
|
||||
findAllLoaders(n.subgraph._nodes);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (app.graph && app.graph._nodes) {
|
||||
findAllLoaders(app.graph._nodes);
|
||||
}
|
||||
|
||||
if (multiImageLoaders.length === 1) {
|
||||
return getCountFromNode(multiImageLoaders[0]);
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
// --- Backup polling via setInterval (4 Hz) ---
|
||||
const pollInterval = setInterval(() => {
|
||||
if (!node.graph) {
|
||||
clearInterval(pollInterval);
|
||||
return;
|
||||
}
|
||||
const count = readSourceImageCount(node);
|
||||
if (count !== null) {
|
||||
node._applyWidgetCount(count);
|
||||
}
|
||||
}, 250);
|
||||
|
||||
// Clean up the interval when the node is deleted
|
||||
const origOnRemoved = node.onRemoved;
|
||||
node.onRemoved = function() {
|
||||
// Unregister this node instance
|
||||
window._LTXKeyframerGlobalNodes.delete(node);
|
||||
|
||||
clearInterval(pollInterval);
|
||||
if (origOnRemoved) origOnRemoved.apply(this, arguments);
|
||||
};
|
||||
|
||||
// --- Connection change handler ---
|
||||
const onConnectionsChange = node.onConnectionsChange;
|
||||
node.onConnectionsChange = function(type, index, connected, link_info) {
|
||||
if (onConnectionsChange) onConnectionsChange.apply(this, arguments);
|
||||
|
||||
if (type === 1) { // 1 = Input
|
||||
const input = this.inputs[index];
|
||||
if (input && input.name === "multi_input") {
|
||||
if (connected) {
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
|
||||
}, 100);
|
||||
} else {
|
||||
this._applyWidgetCount(0);
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// --- Initial sync when first placed on canvas ---
|
||||
const origOnAdded = node.onAdded;
|
||||
node.onAdded = function() {
|
||||
if (origOnAdded) origOnAdded.apply(this, arguments);
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
|
||||
}, 100);
|
||||
};
|
||||
}
|
||||
});
|
||||
@@ -1,398 +0,0 @@
|
||||
import { app } from "../../scripts/app.js";
|
||||
|
||||
// Global registry to track all LTXSequencer nodes across all subgraphs
|
||||
window._LTXSequencerGlobalNodes = window._LTXSequencerGlobalNodes || new Set();
|
||||
|
||||
// ComfyUI native trick to cleanly hide/show widgets without deleting them
|
||||
function toggleWidget(widget, visible) {
|
||||
if (visible) {
|
||||
if (widget.origType !== undefined) {
|
||||
widget.type = widget.origType;
|
||||
widget.computeSize = widget.origComputeSize;
|
||||
delete widget.origType;
|
||||
delete widget.origComputeSize;
|
||||
}
|
||||
} else {
|
||||
if (widget.type !== "hidden") {
|
||||
widget.origType = widget.type;
|
||||
widget.origComputeSize = widget.computeSize;
|
||||
widget.type = "hidden";
|
||||
widget.computeSize = () => [0, -4];
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* FULL STATE SYNC:
|
||||
* Instead of syncing one widget, we push the entire properties object
|
||||
* to ensure no values are ever lost during subgraph transitions or deletions.
|
||||
*/
|
||||
function syncFullStateAcrossNodes(sourceNode) {
|
||||
if (!window._LTXSequencerGlobalNodes) return;
|
||||
|
||||
for (const targetNode of window._LTXSequencerGlobalNodes) {
|
||||
if (targetNode === sourceNode) continue;
|
||||
|
||||
// 1. Mirror the properties object completely
|
||||
// We use a shallow copy to ensure we don't accidentally share object references
|
||||
const newState = { ...sourceNode.properties };
|
||||
targetNode.properties = { ...targetNode.properties, ...newState };
|
||||
|
||||
// 2. Check if we need to rebuild the widget list (if num_images changed)
|
||||
const targetImageCount = targetNode.properties["num_images"] || 0;
|
||||
const currentVisibleCount = targetNode._currentImageCount;
|
||||
|
||||
if (targetImageCount !== currentVisibleCount) {
|
||||
targetNode._applyWidgetCount(targetImageCount);
|
||||
}
|
||||
|
||||
// 3. Update all existing widget values visually
|
||||
if (targetNode.widgets) {
|
||||
let modeChanged = false;
|
||||
targetNode.widgets.forEach(w => {
|
||||
const newValue = targetNode.properties[w.name];
|
||||
if (newValue !== undefined && w.value !== newValue) {
|
||||
w.value = newValue;
|
||||
if (w.name === "insert_mode") modeChanged = true;
|
||||
}
|
||||
});
|
||||
|
||||
if (modeChanged && targetNode._updateVisibility) {
|
||||
targetNode._updateVisibility();
|
||||
}
|
||||
}
|
||||
|
||||
targetNode.setDirtyCanvas(true, false);
|
||||
}
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "Comfy.LTXSequencer.DynamicInputs",
|
||||
async nodeCreated(node) {
|
||||
if (node.comfyClass !== "LTXSequencer") return;
|
||||
|
||||
// Register this node instance globally
|
||||
window._LTXSequencerGlobalNodes.add(node);
|
||||
|
||||
node._currentImageCount = -1; // Force first update
|
||||
|
||||
// Initialize persistent properties cache
|
||||
node.properties = node.properties || {};
|
||||
|
||||
// Add subtle separator line above images_loaded
|
||||
node.addCustomWidget({
|
||||
name: "num_images_separator",
|
||||
type: "text",
|
||||
draw(ctx, node, widget_width, y, widget_height) {
|
||||
ctx.save();
|
||||
ctx.strokeStyle = "#444";
|
||||
ctx.lineWidth = 1;
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(10, y + 5);
|
||||
ctx.lineTo(widget_width - 10, y + 5);
|
||||
ctx.stroke();
|
||||
ctx.restore();
|
||||
},
|
||||
computeSize(width) {
|
||||
return [width, 10];
|
||||
}
|
||||
});
|
||||
|
||||
// Move separator before num_images
|
||||
const moveSeparator = () => {
|
||||
const idx = node.widgets.findIndex(w => w.name === "num_images");
|
||||
const sepIdx = node.widgets.findIndex(w => w.name === "num_images_separator");
|
||||
if (idx !== -1 && sepIdx !== -1) {
|
||||
const separator = node.widgets.splice(sepIdx, 1)[0];
|
||||
node.widgets.splice(idx, 0, separator);
|
||||
}
|
||||
};
|
||||
setTimeout(moveSeparator, 50);
|
||||
|
||||
// Binds custom callbacks to python-schema generated widgets
|
||||
node._hookStaticWidgets = function() {
|
||||
if (!this.widgets) return;
|
||||
const staticNames = ["num_images", "insert_mode", "frame_rate"];
|
||||
staticNames.forEach(name => {
|
||||
const w = this.widgets.find(w => w.name === name);
|
||||
if (w && !w._has_custom_callback) {
|
||||
const orig = w.callback;
|
||||
w.callback = (val) => {
|
||||
this.properties[name] = val;
|
||||
|
||||
if (name === "num_images") {
|
||||
this._applyWidgetCount(val);
|
||||
}
|
||||
|
||||
// Push full state to siblings
|
||||
syncFullStateAcrossNodes(this);
|
||||
|
||||
if (name === "insert_mode") {
|
||||
this._updateVisibility();
|
||||
}
|
||||
if (orig) orig.apply(w, [val]);
|
||||
};
|
||||
w._has_custom_callback = true;
|
||||
}
|
||||
});
|
||||
};
|
||||
|
||||
// Handles show/hiding widgets based on insertion method
|
||||
node._updateVisibility = function() {
|
||||
const mode = this.properties["insert_mode"] || "frames";
|
||||
if (!this.widgets) return;
|
||||
|
||||
let changed = false;
|
||||
for (const w of this.widgets) {
|
||||
let shouldBeVisible = true;
|
||||
|
||||
if (w.name.startsWith("insert_frame_")) {
|
||||
shouldBeVisible = (mode === "frames");
|
||||
} else if (w.name.startsWith("insert_second_")) {
|
||||
shouldBeVisible = (mode === "seconds");
|
||||
}
|
||||
|
||||
const isHidden = (w.type === "hidden");
|
||||
if (shouldBeVisible && isHidden) {
|
||||
toggleWidget(w, true);
|
||||
changed = true;
|
||||
} else if (!shouldBeVisible && !isHidden) {
|
||||
toggleWidget(w, false);
|
||||
changed = true;
|
||||
}
|
||||
}
|
||||
|
||||
if (changed) {
|
||||
this.setDirtyCanvas(true, true);
|
||||
requestAnimationFrame(() => {
|
||||
if (this.computeSize) {
|
||||
this.setSize(this.computeSize());
|
||||
}
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
// Core update: synchronize widget visibility to match imageCount
|
||||
node._applyWidgetCount = function(count) {
|
||||
this._hookStaticWidgets();
|
||||
|
||||
const isInitialLoad = this._currentImageCount === -1;
|
||||
if (this._currentImageCount === count && !isInitialLoad) return;
|
||||
this._currentImageCount = count;
|
||||
|
||||
const initialWidth = this.size[0];
|
||||
const numWidget = this.widgets?.find(w => w.name === "num_images");
|
||||
if (numWidget) {
|
||||
numWidget.label = "images_loaded";
|
||||
numWidget.value = Math.max(0, Math.min(count || 0, 50));
|
||||
}
|
||||
|
||||
// 1. Update properties from current widget values before restructuring
|
||||
if (this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) {
|
||||
if (w.type !== "hidden" && w.type !== "button") this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// 2. Clear dynamic widgets
|
||||
if (this.widgets) {
|
||||
this.widgets = this.widgets.filter(w =>
|
||||
!w.name.startsWith("insert_frame_") &&
|
||||
!w.name.startsWith("insert_second_") &&
|
||||
!w.name.startsWith("strength_") &&
|
||||
!w.name.startsWith("header_")
|
||||
);
|
||||
} else {
|
||||
this.widgets = [];
|
||||
}
|
||||
|
||||
// 3. Rebuild precisely
|
||||
for (let i = 1; i <= count; i++) {
|
||||
// Header
|
||||
const headerName = `header_${i}`;
|
||||
this.addCustomWidget({
|
||||
name: headerName,
|
||||
type: "text",
|
||||
value: `Image #${i}`,
|
||||
draw(ctx, node, widget_width, y, widget_height) {
|
||||
ctx.save();
|
||||
const margin = 10;
|
||||
const topPadding = 15;
|
||||
ctx.strokeStyle = "#333";
|
||||
ctx.lineWidth = 1;
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(margin, y + 5);
|
||||
ctx.lineTo(widget_width - margin, y + 5);
|
||||
ctx.stroke();
|
||||
ctx.fillStyle = "#dddddd";
|
||||
ctx.font = "bold 12px Arial";
|
||||
ctx.textAlign = "left";
|
||||
ctx.fillText(`Image #${i}`, margin, y + topPadding + 10);
|
||||
ctx.restore();
|
||||
},
|
||||
computeSize(width) { return [width, 35]; }
|
||||
});
|
||||
|
||||
// Helper to add widget with full sync
|
||||
const addSyncedWidget = (type, name, def, options) => {
|
||||
const saved = this.properties[name];
|
||||
return this.addWidget(type, name, saved !== undefined ? saved : def, (val) => {
|
||||
this.properties[name] = val;
|
||||
syncFullStateAcrossNodes(this);
|
||||
}, options);
|
||||
};
|
||||
|
||||
addSyncedWidget("number", `insert_frame_${i}`, 0, { min: -9999, max: 9999, step: 10, precision: 0 });
|
||||
addSyncedWidget("number", `insert_second_${i}`, 0.0, { min: 0.0, max: 9999.0, step: 0.1, precision: 2 });
|
||||
addSyncedWidget("number", `strength_${i}`, 1.0, { min: 0.0, max: 1.0, step: 0.01 });
|
||||
}
|
||||
|
||||
this._updateVisibility();
|
||||
this.setDirtyCanvas(true, true);
|
||||
requestAnimationFrame(() => {
|
||||
if (this.computeSize) {
|
||||
this.setSize(this.computeSize());
|
||||
this.size[0] = initialWidth;
|
||||
}
|
||||
});
|
||||
};
|
||||
|
||||
const origConfigure = node.configure;
|
||||
node.configure = function(info) {
|
||||
if (origConfigure) origConfigure.apply(this, arguments);
|
||||
if (this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) {
|
||||
if (w.type !== "button") this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
node.onConfigure = function(info) {
|
||||
if (info.properties) {
|
||||
this.properties = { ...this.properties, ...info.properties };
|
||||
}
|
||||
this._hookStaticWidgets();
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
let targetCount = count !== null ? count : (this.properties.num_images || 0);
|
||||
this._applyWidgetCount(targetCount);
|
||||
this._updateVisibility();
|
||||
}, 100);
|
||||
};
|
||||
|
||||
const originalOnSerialize = node.onSerialize;
|
||||
node.onSerialize = function(info) {
|
||||
if (this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) {
|
||||
if (w.type !== "hidden" && w.type !== "button") this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
if (originalOnSerialize) originalOnSerialize.apply(this, arguments);
|
||||
info.properties = { ...this.properties };
|
||||
|
||||
const strictArray = [];
|
||||
strictArray.push(this.properties["num_images"] !== undefined ? this.properties["num_images"] : 1);
|
||||
strictArray.push(this.properties["insert_mode"] !== undefined ? this.properties["insert_mode"] : "frames");
|
||||
strictArray.push(this.properties["frame_rate"] !== undefined ? this.properties["frame_rate"] : 24);
|
||||
|
||||
for (let i = 1; i <= 50; i++) {
|
||||
strictArray.push(this.properties[`insert_frame_${i}`] !== undefined ? this.properties[`insert_frame_${i}`] : 0);
|
||||
strictArray.push(this.properties[`insert_second_${i}`] !== undefined ? this.properties[`insert_second_${i}`] : 0.0);
|
||||
strictArray.push(this.properties[`strength_${i}`] !== undefined ? this.properties[`strength_${i}`] : 1.0);
|
||||
}
|
||||
info.widgets_values = strictArray;
|
||||
};
|
||||
|
||||
function readSourceImageCount(self) {
|
||||
const multiInput = self.inputs?.find(inp => inp.name === "multi_input");
|
||||
if (!multiInput || !multiInput.link) return null;
|
||||
const nodeGraph = self.graph || app.graph;
|
||||
|
||||
function traceUpstream(graph, linkId, visited = new Set()) {
|
||||
if (!linkId || visited.has(linkId)) return null;
|
||||
visited.add(linkId);
|
||||
const link = graph.links[linkId];
|
||||
if (!link) return null;
|
||||
const originNode = graph.getNodeById(link.origin_id);
|
||||
if (!originNode) return null;
|
||||
if (originNode.comfyClass === "MultiImageLoader") return originNode;
|
||||
if (originNode.type === "Reroute" || originNode.comfyClass === "Reroute") {
|
||||
if (originNode.inputs?.[0]?.link) return traceUpstream(graph, originNode.inputs[0].link, visited);
|
||||
}
|
||||
if (typeof originNode.getInnerNode === "function") {
|
||||
try {
|
||||
const innerNode = originNode.getInnerNode(link.origin_slot);
|
||||
if (innerNode?.comfyClass === "MultiImageLoader") return innerNode;
|
||||
} catch (e) {}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
let sourceNode = traceUpstream(nodeGraph, multiInput.link);
|
||||
function getCountFromNode(n) {
|
||||
if (typeof n._imageCount === "number") return n._imageCount;
|
||||
const pathsWidget = n.widgets?.find(w => w.name === "image_paths");
|
||||
return pathsWidget ? (pathsWidget.value || "").split('\n').filter(p => p.trim()).length : null;
|
||||
}
|
||||
|
||||
if (sourceNode) return getCountFromNode(sourceNode);
|
||||
|
||||
let multiImageLoaders = [];
|
||||
function findAllLoaders(nodes) {
|
||||
if (!nodes) return;
|
||||
for (let n of nodes) {
|
||||
if (n.comfyClass === "MultiImageLoader") multiImageLoaders.push(n);
|
||||
if (n.subgraph?._nodes) findAllLoaders(n.subgraph._nodes);
|
||||
}
|
||||
}
|
||||
if (app.graph?._nodes) findAllLoaders(app.graph._nodes);
|
||||
if (multiImageLoaders.length === 1) return getCountFromNode(multiImageLoaders[0]);
|
||||
return null;
|
||||
}
|
||||
|
||||
const pollInterval = setInterval(() => {
|
||||
if (!node.graph) {
|
||||
clearInterval(pollInterval);
|
||||
return;
|
||||
}
|
||||
const count = readSourceImageCount(node);
|
||||
if (count !== null && count !== node._currentImageCount) {
|
||||
node._applyWidgetCount(count);
|
||||
syncFullStateAcrossNodes(node); // Sync the new count to others
|
||||
}
|
||||
}, 500);
|
||||
|
||||
const origOnRemoved = node.onRemoved;
|
||||
node.onRemoved = function() {
|
||||
window._LTXSequencerGlobalNodes.delete(node);
|
||||
clearInterval(pollInterval);
|
||||
if (origOnRemoved) origOnRemoved.apply(this, arguments);
|
||||
};
|
||||
|
||||
node.onConnectionsChange = function(type, index, connected) {
|
||||
if (type === 1 && this.inputs[index]?.name === "multi_input") {
|
||||
if (connected) {
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
|
||||
}, 100);
|
||||
} else {
|
||||
this._applyWidgetCount(0);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
node.onAdded = function() {
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
|
||||
}, 100);
|
||||
};
|
||||
}
|
||||
});
|
||||
@@ -1,688 +0,0 @@
|
||||
import { app } from "../../scripts/app.js";
|
||||
import { api } from "../../scripts/api.js";
|
||||
|
||||
app.registerExtension({
|
||||
name: "Comfy.MultiImageLoader",
|
||||
async nodeCreated(node) {
|
||||
if (node.comfyClass !== "MultiImageLoader") return;
|
||||
|
||||
// Helper to detect if we are in the new Nodes 2.0 / V3 Web Component frontend
|
||||
let v3NodeElement = null;
|
||||
function checkIsV3() {
|
||||
if (v3NodeElement) return true;
|
||||
let el = container.parentElement;
|
||||
while (el) {
|
||||
if ((el.tagName && el.tagName.toLowerCase().includes('comfy-node')) ||
|
||||
(el.classList && el.classList.contains('comfy-node'))) {
|
||||
v3NodeElement = el;
|
||||
return true;
|
||||
}
|
||||
el = el.parentElement || (el.getRootNode ? el.getRootNode().host : null);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
// --- 1. UI Setup: Main Container ---
|
||||
const container = document.createElement("div");
|
||||
container.style.cssText = `
|
||||
width: 100%;
|
||||
min-height: 250px;
|
||||
min-width: 100px; /* Reduced from 400px to allow thin resizing in V3 */
|
||||
background: #222222;
|
||||
border: 1px solid #353545;
|
||||
border-radius: 4px;
|
||||
margin-top: 5px;
|
||||
padding: 10px;
|
||||
box-sizing: border-box;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 10px;
|
||||
pointer-events: auto;
|
||||
overflow: hidden;
|
||||
`;
|
||||
|
||||
// Top Bar for Actions
|
||||
const topBar = document.createElement("div");
|
||||
// Added flex-wrap: wrap so buttons stack if the node gets extremely thin
|
||||
topBar.style.cssText = "display: flex; flex-wrap: wrap; justify-content: flex-start; align-items: center; width: 100%; gap: 8px;";
|
||||
|
||||
const uploadBtn = document.createElement("button");
|
||||
uploadBtn.innerText = "Upload Images";
|
||||
uploadBtn.style.cssText = `
|
||||
background: #3a3f4b; color: white; border: 1px solid #5a5f6b;
|
||||
padding: 3px 8px; border-radius: 3px; cursor: pointer; font-size: 10px;
|
||||
`;
|
||||
|
||||
const removeAllBtn = document.createElement("button");
|
||||
removeAllBtn.innerText = "Remove All";
|
||||
removeAllBtn.style.cssText = `
|
||||
background: #cc2222; color: white; border: 1px solid #aa1111;
|
||||
padding: 3px 8px; border-radius: 3px; cursor: pointer; font-size: 10px;
|
||||
transition: background 0.2s;
|
||||
`;
|
||||
removeAllBtn.onmouseenter = () => { removeAllBtn.style.background = "#ff3333"; };
|
||||
removeAllBtn.onmouseleave = () => { removeAllBtn.style.background = "#cc2222"; };
|
||||
removeAllBtn.onclick = () => {
|
||||
setWidgetValue([], false);
|
||||
};
|
||||
|
||||
topBar.appendChild(uploadBtn);
|
||||
topBar.appendChild(removeAllBtn);
|
||||
container.appendChild(topBar);
|
||||
|
||||
const gridWrapper = document.createElement("div");
|
||||
gridWrapper.style.cssText = `
|
||||
position: relative;
|
||||
flex-grow: 1;
|
||||
width: 100%;
|
||||
min-height: 0;
|
||||
`;
|
||||
|
||||
const grid = document.createElement("div");
|
||||
grid.style.cssText = `
|
||||
position: absolute;
|
||||
top: 0; left: 0; right: 0; bottom: 0;
|
||||
display: grid;
|
||||
gap: 8px;
|
||||
justify-content: center;
|
||||
align-content: center;
|
||||
`;
|
||||
|
||||
gridWrapper.appendChild(grid);
|
||||
container.appendChild(gridWrapper);
|
||||
|
||||
const fileInput = document.createElement("input");
|
||||
fileInput.type = "file";
|
||||
fileInput.multiple = true;
|
||||
fileInput.accept = "image/*";
|
||||
fileInput.style.display = "none";
|
||||
container.appendChild(fileInput);
|
||||
|
||||
// Add the Widget to the Node
|
||||
const galleryWidget = node.addDOMWidget("Gallery", "html_gallery", container, { serialize: false });
|
||||
|
||||
galleryWidget.computeSize = function() {
|
||||
const galleryY = this.last_y || 40;
|
||||
const minOutputsHeight = (node.outputs ? node.outputs.length : 1) * 20;
|
||||
const requiredGalleryHeight = Math.max(250, minOutputsHeight + 40 - galleryY);
|
||||
return [150, requiredGalleryHeight]; // Changed minimum theoretical widget width
|
||||
};
|
||||
|
||||
// --- SAFELY HIDE THE IMAGE_PATHS WIDGET ---
|
||||
const pathsWidget = node.widgets.find(w => w.name === "image_paths");
|
||||
if (pathsWidget) {
|
||||
// Forcefully lock the hidden state against V3's reactive redraws
|
||||
Object.defineProperty(pathsWidget, 'hidden', {
|
||||
get: () => true,
|
||||
set: () => {} // Ignore attempts by V3 to unhide it
|
||||
});
|
||||
Object.defineProperty(pathsWidget, 'type', {
|
||||
get: () => "hidden",
|
||||
set: () => {} // Ignore attempts by V3 to reset the type
|
||||
});
|
||||
|
||||
pathsWidget.computeSize = function() {
|
||||
return [0, 0];
|
||||
};
|
||||
|
||||
// Catch for V3 delayed DOM rendering to ensure no stubborn inputs appear
|
||||
const hideInterval = setInterval(() => {
|
||||
if (pathsWidget.element) {
|
||||
pathsWidget.element.style.display = "none";
|
||||
}
|
||||
}, 50);
|
||||
setTimeout(() => clearInterval(hideInterval), 1000);
|
||||
}
|
||||
|
||||
const oldCallback = pathsWidget?.callback;
|
||||
|
||||
function setWidgetValue(newPathsArray, isRearranging = false) {
|
||||
if (!pathsWidget) return;
|
||||
const val = newPathsArray.join("\n");
|
||||
|
||||
const tempCallback = pathsWidget.callback;
|
||||
pathsWidget.callback = null;
|
||||
|
||||
pathsWidget.value = val;
|
||||
if (oldCallback) oldCallback.apply(pathsWidget, [val]);
|
||||
|
||||
pathsWidget.callback = tempCallback;
|
||||
refreshGallery(isRearranging);
|
||||
}
|
||||
|
||||
// --- 2. Logic: Output Syncing & Dynamic Packing ---
|
||||
function syncOutputs(count) {
|
||||
if (!node.outputs) return;
|
||||
|
||||
let changed = false;
|
||||
const targetTotal = count + 1;
|
||||
|
||||
const wasFresh = node.outputs.length >= 50;
|
||||
|
||||
while (node.outputs.length > targetTotal && node.outputs.length > 1) {
|
||||
node.removeOutput(node.outputs.length - 1);
|
||||
changed = true;
|
||||
}
|
||||
|
||||
for (let i = node.outputs.length; i < targetTotal; i++) {
|
||||
node.addOutput(`image_${i}`, "IMAGE");
|
||||
changed = true;
|
||||
}
|
||||
|
||||
if (changed || wasFresh) {
|
||||
updateLayout(wasFresh);
|
||||
}
|
||||
}
|
||||
|
||||
function notifyConnectedNodes(imageCount) {
|
||||
if (!node.outputs) return;
|
||||
for (const output of node.outputs) {
|
||||
if (!output.links) continue;
|
||||
for (const linkId of output.links) {
|
||||
const link = app.graph.links[linkId];
|
||||
if (!link) continue;
|
||||
const targetNode = app.graph.getNodeById(link.target_id);
|
||||
if (targetNode && typeof targetNode._syncImageCount === "function") {
|
||||
targetNode._syncImageCount(imageCount);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function optimizeGrid(gridW, gridH) {
|
||||
const paths = (pathsWidget?.value || "").split(/\n|,/).map(s => s.trim()).filter(s => s);
|
||||
const N = paths.length;
|
||||
|
||||
if (N === 0) {
|
||||
grid.style.gridTemplateColumns = 'repeat(auto-fit, minmax(75px, 1fr))';
|
||||
grid.style.gridAutoRows = 'max-content';
|
||||
return;
|
||||
}
|
||||
|
||||
if (gridW <= 0 || gridH <= 0) return;
|
||||
|
||||
let bestS = 0;
|
||||
let bestCols = 1;
|
||||
|
||||
for (let c = 1; c <= N; c++) {
|
||||
const r = Math.ceil(N / c);
|
||||
// Math.max guarantees we don't end up with negative max width in ultra-thin layouts
|
||||
const maxW = Math.max(5, (gridW - (c - 1) * 8) / c);
|
||||
const maxH = Math.max(5, (gridH - (r - 1) * 8) / r);
|
||||
const size = Math.min(maxW, maxH);
|
||||
|
||||
// By using >= instead of strict > (with a tiny 0.1 buffer for float precision),
|
||||
// if multiple column counts yield the exact same optimal cell size
|
||||
// (which happens when height is the bottleneck), we aggressively pack
|
||||
// more items horizontally onto the row to fill empty space.
|
||||
if (size >= bestS - 0.1) {
|
||||
bestS = size;
|
||||
bestCols = c;
|
||||
}
|
||||
}
|
||||
|
||||
// Allow grid cells to shrink down to 10px instead of 15 to prevent horizontal overflow in V3
|
||||
bestS = Math.max(10, Math.floor(bestS));
|
||||
|
||||
grid.style.gridTemplateColumns = `repeat(${bestCols}, ${bestS}px)`;
|
||||
grid.style.gridAutoRows = `${bestS}px`;
|
||||
}
|
||||
|
||||
let v3EventsAttached = false;
|
||||
|
||||
function enforceV3CSS() {
|
||||
const isV3 = checkIsV3();
|
||||
if (isV3 && v3NodeElement) {
|
||||
const paddingBottom = 15;
|
||||
const galleryY = galleryWidget.last_y || 40;
|
||||
const minOutputsHeight = (node.outputs ? node.outputs.length : 1) * 20;
|
||||
const absoluteMinHeight = Math.max(galleryY + 250 + paddingBottom, minOutputsHeight + 40);
|
||||
|
||||
// For Nodes 2.0 (V3), we remove the min-width constraint completely.
|
||||
// We leave min-height so outputs don't bleed out vertically.
|
||||
v3NodeElement.style.removeProperty('min-width');
|
||||
v3NodeElement.style.setProperty('min-height', absoluteMinHeight + 'px', 'important');
|
||||
|
||||
// Attach drag & drop to the entire V3 Web Component
|
||||
if (!v3EventsAttached) {
|
||||
v3EventsAttached = true;
|
||||
v3NodeElement.addEventListener("dragover", (e) => {
|
||||
e.preventDefault();
|
||||
});
|
||||
v3NodeElement.addEventListener("drop", (e) => {
|
||||
if (e.dataTransfer && e.dataTransfer.files) {
|
||||
const files = Array.from(e.dataTransfer.files).filter(f => f.type.startsWith('image/'));
|
||||
if (files.length > 0) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
handleFiles(files);
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let isLayouting = false;
|
||||
|
||||
function updateLayout(forceShrink = false) {
|
||||
if (isLayouting) return;
|
||||
isLayouting = true;
|
||||
|
||||
const isV3 = checkIsV3();
|
||||
const minW = isV3 ? 100 : 200; // 100 in V3, 440 in V1
|
||||
const paddingBottom = isV3 ? 15 : 25; // Apply extra 20px pad on V1
|
||||
|
||||
const galleryY = galleryWidget.last_y || 40;
|
||||
const minOutputsHeight = (node.outputs ? node.outputs.length : 1) * 20;
|
||||
const absoluteMinHeight = Math.max(galleryY + 250 + paddingBottom, minOutputsHeight + 40);
|
||||
|
||||
node.min_size = [minW, absoluteMinHeight];
|
||||
enforceV3CSS();
|
||||
|
||||
let targetW = Math.max(node.size[0], minW);
|
||||
let targetH = forceShrink ? absoluteMinHeight : node.size[1];
|
||||
|
||||
targetH = Math.max(targetH, absoluteMinHeight);
|
||||
|
||||
if (node.size[0] !== targetW || node.size[1] !== targetH) {
|
||||
node.setSize([targetW, targetH]);
|
||||
app.graph.setDirtyCanvas(true, true);
|
||||
}
|
||||
|
||||
const availableGalleryHeight = targetH - galleryY - paddingBottom;
|
||||
container.style.height = availableGalleryHeight + "px";
|
||||
|
||||
isLayouting = false;
|
||||
}
|
||||
|
||||
// --- OVERRIDE LOGIC FOR RESIZING ---
|
||||
const origOnResize = node.onResize;
|
||||
node.onResize = function(size) {
|
||||
const isV3 = checkIsV3();
|
||||
const minW = isV3 ? 100 : 220; // Adjust limits based on frontend
|
||||
const paddingBottom = isV3 ? 15 : 25; // Apply extra 20px pad on V1
|
||||
|
||||
const galleryY = galleryWidget.last_y || 40;
|
||||
const minOutputsHeight = (this.outputs ? this.outputs.length : 1) * 20;
|
||||
const absoluteMinHeight = Math.max(galleryY + 250 + paddingBottom, minOutputsHeight + 40);
|
||||
|
||||
size[0] = Math.max(size[0], minW);
|
||||
size[1] = Math.max(size[1], absoluteMinHeight);
|
||||
|
||||
if (origOnResize) origOnResize.call(this, size);
|
||||
if (isLayouting) return;
|
||||
|
||||
node.min_size = [minW, absoluteMinHeight];
|
||||
enforceV3CSS();
|
||||
|
||||
const availableGalleryHeight = size[1] - galleryY - paddingBottom;
|
||||
container.style.height = availableGalleryHeight + "px";
|
||||
};
|
||||
|
||||
const origComputeSize = node.computeSize;
|
||||
node.computeSize = function(out) {
|
||||
const isV3 = checkIsV3();
|
||||
const minW = isV3 ? 100 : 220;
|
||||
const paddingBottom = isV3 ? 15 : 25;
|
||||
|
||||
let res = origComputeSize ? origComputeSize.apply(this, arguments) : [minW, 250];
|
||||
const galleryY = galleryWidget.last_y || 40;
|
||||
const minOutputsHeight = (this.outputs ? this.outputs.length : 1) * 20;
|
||||
const absoluteMinHeight = Math.max(galleryY + 250 + paddingBottom, minOutputsHeight + 40);
|
||||
|
||||
this.min_size = [minW, absoluteMinHeight];
|
||||
res[0] = Math.max(res[0], minW);
|
||||
res[1] = Math.max(res[1], absoluteMinHeight);
|
||||
|
||||
enforceV3CSS();
|
||||
return res;
|
||||
};
|
||||
|
||||
const origSetSize = node.setSize;
|
||||
node.setSize = function(size) {
|
||||
const isV3 = checkIsV3();
|
||||
const minW = isV3 ? 100 : 220;
|
||||
const paddingBottom = isV3 ? 15 : 25;
|
||||
|
||||
const galleryY = galleryWidget.last_y || 40;
|
||||
const minOutputsHeight = (this.outputs ? this.outputs.length : 1) * 20;
|
||||
const absoluteMinHeight = Math.max(galleryY + 250 + paddingBottom, minOutputsHeight + 40);
|
||||
|
||||
size[0] = Math.max(size[0], minW);
|
||||
size[1] = Math.max(size[1], absoluteMinHeight);
|
||||
|
||||
if (origSetSize) {
|
||||
origSetSize.call(this, size);
|
||||
} else {
|
||||
this.size = size;
|
||||
}
|
||||
enforceV3CSS();
|
||||
};
|
||||
|
||||
let lastObservedWidth = 0;
|
||||
let lastObservedHeight = 0;
|
||||
|
||||
const resizeObserver = new ResizeObserver((entries) => {
|
||||
enforceV3CSS();
|
||||
for (const entry of entries) {
|
||||
const w = Math.round(entry.contentRect.width);
|
||||
const h = Math.round(entry.contentRect.height);
|
||||
|
||||
if (Math.abs(w - lastObservedWidth) > 1 || Math.abs(h - lastObservedHeight) > 1) {
|
||||
lastObservedWidth = w;
|
||||
lastObservedHeight = h;
|
||||
if (h > 0) {
|
||||
optimizeGrid(w, h);
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
resizeObserver.observe(gridWrapper);
|
||||
|
||||
// --- 3. Logic: Gallery Rendering ---
|
||||
let draggedNode = null;
|
||||
let lastSwapX = 0;
|
||||
let lastSwapY = 0;
|
||||
let lastSwapTime = 0;
|
||||
|
||||
function refreshGallery(isRearranging = false) {
|
||||
grid.innerHTML = "";
|
||||
const paths = (pathsWidget?.value || "").split(/\n|,/).map(s => s.trim()).filter(s => s);
|
||||
|
||||
if (!isRearranging) {
|
||||
syncOutputs(paths.length);
|
||||
}
|
||||
node._imageCount = paths.length;
|
||||
notifyConnectedNodes(paths.length);
|
||||
|
||||
paths.forEach((path, index) => {
|
||||
const item = document.createElement("div");
|
||||
item.dataset.path = path;
|
||||
item.draggable = true;
|
||||
item.style.cssText = `
|
||||
position: relative;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
aspect-ratio: 1 / 1;
|
||||
background: #000000;
|
||||
border-radius: 4px;
|
||||
border: 1px solid #444;
|
||||
overflow: hidden;
|
||||
cursor: grab;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
`;
|
||||
|
||||
const img = document.createElement("img");
|
||||
img.src = `/api/view?filename=${encodeURIComponent(path)}&type=input`;
|
||||
// Allow pointer-events so context menu interacts directly with the image
|
||||
img.style.cssText = "max-width: 100%; max-height: 100%; object-fit: contain; pointer-events: auto; display: block;";
|
||||
img.draggable = false; // Prevent native browser ghost dragging on the image itself
|
||||
|
||||
const del = document.createElement("div");
|
||||
del.style.cssText = `
|
||||
position: absolute; top: 0; right: 0;
|
||||
background: #cc2222; color: white;
|
||||
width: 18px; height: 18px;
|
||||
display: flex; align-items: center; justify-content: center;
|
||||
font-size: 14px; cursor: pointer; z-index: 10;
|
||||
font-family: Arial, sans-serif; font-weight: bold;
|
||||
line-height: 1; border-bottom-left-radius: 4px;
|
||||
transition: background 0.2s;
|
||||
`;
|
||||
del.innerHTML = `<svg width="10" height="10" viewBox="0 0 10 10" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<path d="M1 1L9 9M9 1L1 9" stroke="white" stroke-width="2" stroke-linecap="round"/>
|
||||
</svg>`;
|
||||
|
||||
del.onmouseenter = () => { del.style.background = "#ff3333"; };
|
||||
del.onmouseleave = () => { del.style.background = "#cc2222"; };
|
||||
|
||||
del.onclick = (e) => {
|
||||
e.stopPropagation();
|
||||
const newPaths = paths.filter((_, i) => i !== index);
|
||||
setWidgetValue(newPaths, false);
|
||||
};
|
||||
|
||||
const numBadge = document.createElement("div");
|
||||
numBadge.style.cssText = `
|
||||
position: absolute; bottom: 0; left: 0;
|
||||
background: rgba(0, 0, 0, 0.75); color: #fff;
|
||||
padding: 2px 6px; font-size: 11px; font-family: sans-serif;
|
||||
font-weight: bold; border-top-right-radius: 4px; pointer-events: none;
|
||||
z-index: 5;
|
||||
`;
|
||||
numBadge.innerText = (index + 1).toString();
|
||||
|
||||
// Prevent LiteGraph context menu and instead show standard browser context menu (Copy, Save, Open)
|
||||
item.addEventListener("contextmenu", (e) => {
|
||||
e.stopPropagation();
|
||||
});
|
||||
|
||||
item.ondragstart = (e) => {
|
||||
draggedNode = item;
|
||||
|
||||
e.dataTransfer.setData('text/plain', path);
|
||||
e.dataTransfer.effectAllowed = "move";
|
||||
|
||||
setTimeout(() => {
|
||||
if (draggedNode === item) {
|
||||
// Style as an empty dashed placeholder
|
||||
item.style.background = "transparent";
|
||||
item.style.border = "2px dashed #666";
|
||||
// Hide the visual children (image, delete button, badge)
|
||||
Array.from(item.children).forEach(c => c.style.opacity = "0");
|
||||
}
|
||||
}, 0);
|
||||
};
|
||||
|
||||
item.ondragend = () => {
|
||||
if (draggedNode) {
|
||||
// Restore original appearance
|
||||
draggedNode.style.background = "#000000";
|
||||
draggedNode.style.border = "1px solid #444";
|
||||
Array.from(draggedNode.children).forEach(c => c.style.opacity = "1");
|
||||
}
|
||||
draggedNode = null;
|
||||
|
||||
const newPaths = Array.from(grid.children).map(n => n.dataset.path);
|
||||
const currentVal = (pathsWidget?.value || "").trim();
|
||||
if (newPaths.join("\n") !== currentVal) {
|
||||
setWidgetValue(newPaths, true);
|
||||
}
|
||||
};
|
||||
|
||||
item.ondragover = (e) => {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
if (!draggedNode || draggedNode === item) return;
|
||||
|
||||
const distMoved = Math.hypot(e.clientX - lastSwapX, e.clientY - lastSwapY);
|
||||
if (Date.now() - lastSwapTime < 50 && distMoved < 5) {
|
||||
return;
|
||||
}
|
||||
|
||||
const itemRect = item.getBoundingClientRect();
|
||||
const bufferX = itemRect.width * 0.25;
|
||||
const bufferY = itemRect.height * 0.25;
|
||||
|
||||
if (e.clientX < itemRect.left + bufferX || e.clientX > itemRect.right - bufferX ||
|
||||
e.clientY < itemRect.top + bufferY || e.clientY > itemRect.bottom - bufferY) {
|
||||
return;
|
||||
}
|
||||
|
||||
const items = Array.from(grid.children);
|
||||
const draggedIdx = items.indexOf(draggedNode);
|
||||
const targetIdx = items.indexOf(item);
|
||||
|
||||
// Instantly snap the placeholder to its new position, moving items aside
|
||||
if (draggedIdx < targetIdx) {
|
||||
grid.insertBefore(draggedNode, item.nextSibling);
|
||||
} else {
|
||||
grid.insertBefore(draggedNode, item);
|
||||
}
|
||||
|
||||
lastSwapX = e.clientX;
|
||||
lastSwapY = e.clientY;
|
||||
lastSwapTime = Date.now();
|
||||
};
|
||||
|
||||
item.ondrop = (e) => {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
};
|
||||
|
||||
item.appendChild(img);
|
||||
item.appendChild(del);
|
||||
item.appendChild(numBadge);
|
||||
grid.appendChild(item);
|
||||
});
|
||||
|
||||
if (!isRearranging) {
|
||||
requestAnimationFrame(() => {
|
||||
updateLayout();
|
||||
if (gridWrapper.offsetWidth > 0) optimizeGrid(gridWrapper.offsetWidth, gridWrapper.offsetHeight);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// --- 4. Logic: File Handling ---
|
||||
async function handleFiles(files) {
|
||||
const uploaded = [];
|
||||
for (const file of files) {
|
||||
const body = new FormData();
|
||||
body.append("image", file);
|
||||
try {
|
||||
const resp = await api.fetchApi("/upload/image", { method: "POST", body });
|
||||
if (resp.status === 200) {
|
||||
const data = await resp.json();
|
||||
let name = data.name;
|
||||
if (data.subfolder) name = data.subfolder + "/" + name;
|
||||
uploaded.push(name);
|
||||
}
|
||||
} catch (e) { console.error("Upload error", e); }
|
||||
}
|
||||
if (uploaded.length > 0) {
|
||||
const current = (pathsWidget?.value || "").trim();
|
||||
const allPaths = current ? current.split('\n').concat(uploaded) : uploaded;
|
||||
setWidgetValue(allPaths, false);
|
||||
}
|
||||
}
|
||||
|
||||
// Apply drag & drop to LiteGraph node container bounds (V1)
|
||||
const origOnDragDrop = node.onDragDrop;
|
||||
node.onDragDrop = function(e) {
|
||||
let handled = false;
|
||||
if (e.dataTransfer && e.dataTransfer.files) {
|
||||
const files = Array.from(e.dataTransfer.files).filter(f => f.type.startsWith('image/'));
|
||||
if (files.length > 0) {
|
||||
e.preventDefault();
|
||||
handleFiles(files);
|
||||
handled = true;
|
||||
}
|
||||
}
|
||||
if (!handled && origOnDragDrop) {
|
||||
return origOnDragDrop.apply(this, arguments);
|
||||
}
|
||||
return handled;
|
||||
};
|
||||
|
||||
const origOnDragOver = node.onDragOver;
|
||||
node.onDragOver = function(e) {
|
||||
if (e.dataTransfer && e.dataTransfer.items) {
|
||||
const hasImage = Array.from(e.dataTransfer.items).some(f => f.kind === 'file' && f.type.startsWith('image/'));
|
||||
if (hasImage) {
|
||||
e.preventDefault();
|
||||
return true;
|
||||
}
|
||||
}
|
||||
if (origOnDragOver) {
|
||||
return origOnDragOver.apply(this, arguments);
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
uploadBtn.onclick = () => fileInput.click();
|
||||
fileInput.onchange = (e) => handleFiles(e.target.files);
|
||||
|
||||
container.ondragover = (e) => {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
container.style.borderColor = "#4CAF50";
|
||||
};
|
||||
container.ondragleave = (e) => {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
container.style.borderColor = "#353545";
|
||||
};
|
||||
container.ondrop = (e) => {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
container.style.borderColor = "#353545";
|
||||
if (e.dataTransfer.files.length > 0) handleFiles(e.dataTransfer.files);
|
||||
};
|
||||
|
||||
// --- 5. Logic: Paste Handling ---
|
||||
const pasteHandler = (e) => {
|
||||
if (app.canvas.selected_nodes && app.canvas.selected_nodes[node.id]) {
|
||||
const items = e.clipboardData?.items;
|
||||
if (!items) return;
|
||||
|
||||
const files = [];
|
||||
for (let i = 0; i < items.length; i++) {
|
||||
if (items[i].kind === 'file' && items[i].type.startsWith('image/')) {
|
||||
files.push(items[i].getAsFile());
|
||||
}
|
||||
}
|
||||
|
||||
if (files.length > 0) {
|
||||
e.preventDefault();
|
||||
e.stopImmediatePropagation();
|
||||
handleFiles(files);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
document.addEventListener("paste", pasteHandler, { capture: true });
|
||||
|
||||
const origOnRemoved = node.onRemoved;
|
||||
node.onRemoved = function() {
|
||||
document.removeEventListener("paste", pasteHandler, { capture: true });
|
||||
resizeObserver.disconnect();
|
||||
if (origOnRemoved) origOnRemoved.apply(this, arguments);
|
||||
};
|
||||
|
||||
if (pathsWidget) {
|
||||
pathsWidget.callback = (v) => {
|
||||
if (oldCallback) oldCallback.apply(pathsWidget, [v]);
|
||||
refreshGallery();
|
||||
};
|
||||
}
|
||||
|
||||
// Run immediately to trim blank outputs before LiteGraph does its first layout calculations
|
||||
refreshGallery();
|
||||
|
||||
// Enforce the tightest possible packing explicitly upon being dropped into the graph (V1 specifically)
|
||||
const origOnAdded = node.onAdded;
|
||||
node.onAdded = function() {
|
||||
if (origOnAdded) origOnAdded.apply(this, arguments);
|
||||
const isV3 = checkIsV3();
|
||||
if (!isV3) {
|
||||
requestAnimationFrame(() => {
|
||||
const galleryY = galleryWidget.last_y || 40;
|
||||
const minOutputsHeight = (this.outputs ? this.outputs.length : 1) * 20;
|
||||
const paddingBottom = 25; // Apply extra 20px pad on V1
|
||||
const absoluteMinHeight = Math.max(galleryY + 250 + paddingBottom, minOutputsHeight + 40);
|
||||
// Force the node to snap to its absolute minimum size on initial drop
|
||||
if (this.size && this.size[1] > absoluteMinHeight + 5) {
|
||||
this.setSize([this.size[0], absoluteMinHeight]);
|
||||
if (app.graph) app.graph.setDirtyCanvas(true, true);
|
||||
}
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
setTimeout(() => refreshGallery(), 100);
|
||||
}
|
||||
});
|
||||
@@ -1,336 +0,0 @@
|
||||
import { app } from "../../scripts/app.js";
|
||||
import { ComfyWidgets } from "../../scripts/widgets.js";
|
||||
|
||||
// 1. DEFINE CSS STYLES GLOBALLY ONCE
|
||||
// We use DOM elements instead of Canvas drawing so it works flawlessly in ComfyUI V3
|
||||
const cssStyles = `
|
||||
.slc-ui-container {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
min-height: 260px; /* Increased from 220px to prevent V1 cropping */
|
||||
background-color: rgba(15, 15, 19, 0.7);
|
||||
border: 1px solid rgba(255, 255, 255, 0.1);
|
||||
border-radius: 8px;
|
||||
padding: 12px;
|
||||
box-sizing: border-box;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
font-family: sans-serif;
|
||||
color: #ffffff;
|
||||
overflow: hidden;
|
||||
pointer-events: auto; /* allows text selection */
|
||||
}
|
||||
.slc-empty {
|
||||
flex: 1;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
}
|
||||
.slc-empty-title { color: #aaaaaa; font-size: 14px; font-weight: 500; margin-bottom: 4px; }
|
||||
.slc-empty-sub { color: #777777; font-size: 11px; }
|
||||
.slc-headers {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
.slc-header-block {
|
||||
flex: 1;
|
||||
background-color: rgba(0, 0, 0, 0.4);
|
||||
border-radius: 6px;
|
||||
padding: 8px 0;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
}
|
||||
.slc-header-title { color: #888888; font-size: 9px; font-weight: bold; margin-bottom: 4px; }
|
||||
.slc-header-value { color: #ffffff; font-size: 16px; font-weight: bold; }
|
||||
.slc-cards {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 8px;
|
||||
}
|
||||
.slc-card {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
background-color: rgba(0, 0, 0, 0.25);
|
||||
border: 1px solid rgba(255, 255, 255, 0.05);
|
||||
border-radius: 4px;
|
||||
padding: 8px 12px 8px 16px;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
}
|
||||
.slc-card::before {
|
||||
content: '';
|
||||
position: absolute;
|
||||
left: 0;
|
||||
top: 0;
|
||||
bottom: 0;
|
||||
width: 4px;
|
||||
}
|
||||
.slc-card.slow::before { background-color: #93c5fd; }
|
||||
.slc-card.avg::before { background-color: #86efac; }
|
||||
.slc-card.fast::before { background-color: #fca5a5; }
|
||||
|
||||
.slc-card-left { display: flex; flex-direction: column; }
|
||||
.slc-card-speed { font-size: 12px; font-weight: bold; margin-bottom: 2px; }
|
||||
.slc-card.slow .slc-card-speed { color: #93c5fd; }
|
||||
.slc-card.avg .slc-card-speed { color: #86efac; }
|
||||
.slc-card.fast .slc-card-speed { color: #fca5a5; }
|
||||
.slc-card-wpm { font-size: 10px; color: #aaaaaa; }
|
||||
|
||||
.slc-card-right { display: flex; flex-direction: column; align-items: flex-end; }
|
||||
.slc-card-time { font-size: 13px; font-weight: bold; color: #ffffff; margin-bottom: 2px; }
|
||||
.slc-card-frames { font-size: 11px; font-family: monospace; color: #888888; }
|
||||
|
||||
.slc-legend {
|
||||
margin-top: auto;
|
||||
padding-top: 8px;
|
||||
color: #999999;
|
||||
font-size: 10px;
|
||||
font-weight: 500;
|
||||
}
|
||||
`;
|
||||
|
||||
// Inject styles to document head safely
|
||||
if (!document.getElementById("speech-length-calculator-styles")) {
|
||||
const styleEl = document.createElement("style");
|
||||
styleEl.id = "speech-length-calculator-styles";
|
||||
styleEl.textContent = cssStyles;
|
||||
document.head.appendChild(styleEl);
|
||||
}
|
||||
|
||||
// 2. HELPER TO GENERATE THE HTML CONTENT
|
||||
function buildUIHTML(statsData) {
|
||||
if (!statsData || statsData.empty) {
|
||||
return `
|
||||
<div class="slc-empty">
|
||||
<div class="slc-empty-title">Awaiting Script</div>
|
||||
<div class="slc-empty-sub">Wrap spoken text inside "quotes"</div>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
|
||||
return `
|
||||
<div class="slc-headers">
|
||||
<div class="slc-header-block">
|
||||
<div class="slc-header-title">SPOKEN WORDS</div>
|
||||
<div class="slc-header-value">${statsData.wordCount}</div>
|
||||
</div>
|
||||
<div class="slc-header-block">
|
||||
<div class="slc-header-title">ADDED TIME</div>
|
||||
<div class="slc-header-value">${statsData.additionalTime}s</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="slc-cards">
|
||||
<div class="slc-card slow">
|
||||
<div class="slc-card-left">
|
||||
<div class="slc-card-speed">SLOW</div>
|
||||
<div class="slc-card-wpm">100 WPM</div>
|
||||
</div>
|
||||
<div class="slc-card-right">
|
||||
<div class="slc-card-time">${statsData.slow.time}</div>
|
||||
<div class="slc-card-frames">${statsData.slow.frames} frames</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="slc-card avg">
|
||||
<div class="slc-card-left">
|
||||
<div class="slc-card-speed">AVG</div>
|
||||
<div class="slc-card-wpm">130 WPM</div>
|
||||
</div>
|
||||
<div class="slc-card-right">
|
||||
<div class="slc-card-time">${statsData.avg.time}</div>
|
||||
<div class="slc-card-frames">${statsData.avg.frames} frames</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="slc-card fast">
|
||||
<div class="slc-card-left">
|
||||
<div class="slc-card-speed">FAST</div>
|
||||
<div class="slc-card-wpm">160 WPM</div>
|
||||
</div>
|
||||
<div class="slc-card-right">
|
||||
<div class="slc-card-time">${statsData.fast.time}</div>
|
||||
<div class="slc-card-frames">${statsData.fast.frames} frames</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="slc-legend">WPM = Words Per Minute</div>
|
||||
`;
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "Comfy.SpeechLengthCalculator",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData, app) {
|
||||
if (nodeData.name === "SpeechLengthCalculator") {
|
||||
|
||||
const onNodeCreated = nodeType.prototype.onNodeCreated;
|
||||
nodeType.prototype.onNodeCreated = function () {
|
||||
const r = onNodeCreated ? onNodeCreated.apply(this, arguments) : undefined;
|
||||
|
||||
// 3. CREATE THE DOM WIDGET
|
||||
const uiContainer = document.createElement("div");
|
||||
uiContainer.className = "slc-ui-container";
|
||||
uiContainer.innerHTML = buildUIHTML({ empty: true });
|
||||
|
||||
// Attach to node: standard addDOMWidget works natively in both V1 and V3 Frontends
|
||||
let statsWidget;
|
||||
if (typeof this.addDOMWidget === "function") {
|
||||
statsWidget = this.addDOMWidget("Stats", "HTML", uiContainer, {
|
||||
serialize: false,
|
||||
hideOnZoom: false
|
||||
});
|
||||
} else {
|
||||
// Fallback for very old unpatched V1 installations
|
||||
statsWidget = ComfyWidgets["STRING"](this, "Stats", ["STRING", { multiline: true }], app).widget;
|
||||
if (statsWidget.inputEl) {
|
||||
statsWidget.inputEl.style.display = "none";
|
||||
if (statsWidget.inputEl.parentNode) {
|
||||
statsWidget.inputEl.parentNode.appendChild(uiContainer);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Tell LiteGraph exactly how much vertical space to hold for our UI
|
||||
statsWidget.computeSize = function() {
|
||||
return [0, 280]; // Increased from 240 to prevent bottom cropping in V1
|
||||
};
|
||||
|
||||
// Ensure the node is wide enough on initial creation
|
||||
requestAnimationFrame(() => {
|
||||
const minWidth = 340;
|
||||
if (this.size[0] < minWidth) this.size[0] = minWidth;
|
||||
|
||||
// Fine tune initial height to account for the larger computeSize and make text box taller
|
||||
if (this.size[1] < 500) this.size[1] = 500;
|
||||
|
||||
this.setDirtyCanvas(true, true);
|
||||
});
|
||||
|
||||
// Fetch text from input link OR widget
|
||||
this._getCurrentText = () => {
|
||||
const inputSlot = this.inputs && this.inputs.find(i => i.name === "text_input" || (i.name === "text" && i.link));
|
||||
if (inputSlot && inputSlot.link) {
|
||||
const link = app.graph.links[inputSlot.link];
|
||||
if (link) {
|
||||
const sourceNode = app.graph.getNodeById(link.origin_id);
|
||||
if (sourceNode && sourceNode.widgets) {
|
||||
const w = sourceNode.widgets.find(w => w.name === "value" || w.name === "text" || w.name === "Text" || w.type === "customtext" || w.type === "STRING");
|
||||
if (w && typeof w.value === "string") return w.value;
|
||||
}
|
||||
}
|
||||
}
|
||||
const textWidget = this.widgets && this.widgets.find(w => w.name === "text");
|
||||
return textWidget ? (textWidget.value || "") : "";
|
||||
};
|
||||
|
||||
this._lastState = { text: null, fps: null, addTime: null };
|
||||
|
||||
const updateStats = () => {
|
||||
const fpsWidget = this.widgets && this.widgets.find(w => w.name === "fps");
|
||||
const additionalTimeWidget = this.widgets && this.widgets.find(w => w.name === "additional_time");
|
||||
|
||||
if (!fpsWidget) return;
|
||||
|
||||
const text = this._getCurrentText();
|
||||
const fps = fpsWidget.value || 24;
|
||||
const additionalTime = additionalTimeWidget ? parseFloat(additionalTimeWidget.value) || 0 : 0;
|
||||
|
||||
// Skip expensive calculations if nothing changed
|
||||
if (this._lastState.text === text &&
|
||||
this._lastState.fps === fps &&
|
||||
this._lastState.addTime === additionalTime) {
|
||||
return;
|
||||
}
|
||||
|
||||
this._lastState.text = text;
|
||||
this._lastState.fps = fps;
|
||||
this._lastState.addTime = additionalTime;
|
||||
|
||||
const regex = /"([^"]*)"|'([^']*)'|“([^”]*)”|‘([^’]*)’/g;
|
||||
let match;
|
||||
let quotedText = "";
|
||||
while ((match = regex.exec(text)) !== null) {
|
||||
quotedText += (match[1] || match[2] || match[3] || match[4] || "") + " ";
|
||||
}
|
||||
|
||||
const words = quotedText.trim().split(/\s+/).filter(w => w.length > 0);
|
||||
const wordCount = words.length;
|
||||
|
||||
const formatTime = (wpm) => {
|
||||
const baseMins = wordCount / wpm;
|
||||
const totalSecs = (baseMins * 60) + additionalTime;
|
||||
|
||||
const mins = Math.floor(totalSecs / 60);
|
||||
let secs = totalSecs % 60;
|
||||
secs = Math.ceil(secs * 10) / 10;
|
||||
const frames = Math.ceil(totalSecs * fps);
|
||||
|
||||
const secsStr = secs.toFixed(1);
|
||||
const timeStr = mins > 0 ? `${mins}m ${secsStr}s` : `${secsStr}s`;
|
||||
|
||||
return {
|
||||
time: timeStr,
|
||||
frames: frames.toString()
|
||||
};
|
||||
};
|
||||
|
||||
const statsData = {
|
||||
empty: (wordCount === 0 && additionalTime === 0),
|
||||
wordCount: wordCount,
|
||||
additionalTime: additionalTime,
|
||||
slow: formatTime(100),
|
||||
avg: formatTime(130),
|
||||
fast: formatTime(160)
|
||||
};
|
||||
|
||||
// 4. INJECT HTML TO THE DOM
|
||||
// Replaces the heavy canvas redrawing!
|
||||
uiContainer.innerHTML = buildUIHTML(statsData);
|
||||
|
||||
this.setDirtyCanvas(true, false);
|
||||
};
|
||||
|
||||
// We still use onDrawBackground as an efficient silent trigger
|
||||
// to auto-update in case upstream nodes silently change their text
|
||||
const onDrawBackground = this.onDrawBackground;
|
||||
this.onDrawBackground = function(ctx) {
|
||||
if (onDrawBackground) onDrawBackground.apply(this, arguments);
|
||||
updateStats();
|
||||
};
|
||||
|
||||
// Bind events to update in real time based on node interactions
|
||||
setTimeout(() => {
|
||||
const textWidget = this.widgets && this.widgets.find(w => w.name === "text");
|
||||
const fpsWidget = this.widgets && this.widgets.find(w => w.name === "fps");
|
||||
const additionalTimeWidget = this.widgets && this.widgets.find(w => w.name === "additional_time");
|
||||
|
||||
if (textWidget) {
|
||||
const origCallback = textWidget.callback;
|
||||
textWidget.callback = function() {
|
||||
if (origCallback) origCallback.apply(this, arguments);
|
||||
updateStats();
|
||||
}
|
||||
}
|
||||
if (fpsWidget) {
|
||||
const origFpsCallback = fpsWidget.callback;
|
||||
fpsWidget.callback = function() {
|
||||
if (origFpsCallback) origFpsCallback.apply(this, arguments);
|
||||
updateStats();
|
||||
}
|
||||
}
|
||||
if (additionalTimeWidget) {
|
||||
const origAddCallback = additionalTimeWidget.callback;
|
||||
additionalTimeWidget.callback = function() {
|
||||
if (origAddCallback) origAddCallback.apply(this, arguments);
|
||||
updateStats();
|
||||
}
|
||||
}
|
||||
updateStats();
|
||||
}, 100);
|
||||
|
||||
return r;
|
||||
};
|
||||
}
|
||||
}
|
||||
});
|
||||
@@ -16,7 +16,7 @@ class CleanLatentSlice:
|
||||
RETURN_TYPES = ("LATENT",)
|
||||
RETURN_NAMES = ("latent",)
|
||||
FUNCTION = "slice_latent"
|
||||
CATEGORY = "WhatDreamsCost"
|
||||
CATEGORY = "WhatDreamsCost CS"
|
||||
DESCRIPTION = "Safely slices a video latent starting from an offset index for a specific length. Uses torch.narrow to bypass PyTorch NestedTensor slicing bugs."
|
||||
|
||||
def slice_latent(self, latent, start, length):
|
||||
@@ -65,11 +65,11 @@ class CleanLatentSlice:
|
||||
|
||||
# Register the node with ComfyUI
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"CleanLatentSlice": CleanLatentSlice
|
||||
"CleanLatentSliceCS": CleanLatentSlice
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"CleanLatentSlice": "Clean Latent Slice"
|
||||
"CleanLatentSliceCS": "Clean Latent Slice CS"
|
||||
}
|
||||
|
||||
__all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS']
|
||||
@@ -71,7 +71,7 @@ class LoadAudioUI:
|
||||
}
|
||||
}
|
||||
|
||||
CATEGORY = "WhatDreamsCost"
|
||||
CATEGORY = "WhatDreamsCost CS"
|
||||
RETURN_TYPES = ("AUDIO", "FLOAT")
|
||||
RETURN_NAMES = ("audio", "duration")
|
||||
FUNCTION = "load_audio"
|
||||
|
||||
412
load_video_ui.py
412
load_video_ui.py
@@ -1,412 +0,0 @@
|
||||
import os
|
||||
import torch
|
||||
import numpy as np
|
||||
import folder_paths
|
||||
import av
|
||||
from server import PromptServer
|
||||
from aiohttp import web
|
||||
import comfy.utils
|
||||
|
||||
# Custom API route to serve video files from anywhere on the user's system for the frontend preview
|
||||
@PromptServer.instance.routes.get("/video_ui_custom_view")
|
||||
async def custom_view(request):
|
||||
file_path = request.query.get("filename", "")
|
||||
if os.path.exists(file_path) and os.path.isfile(file_path):
|
||||
return web.FileResponse(file_path)
|
||||
return web.Response(status=404, text="File not found")
|
||||
|
||||
# Custom API route for Chunked Uploads to bypass the 413 Payload Too Large error
|
||||
@PromptServer.instance.routes.post("/video_ui_upload_chunk")
|
||||
async def upload_chunk(request):
|
||||
post = await request.post()
|
||||
file = post.get("file")
|
||||
filename = post.get("filename")
|
||||
chunk_index = int(post.get("chunk_index"))
|
||||
total_chunks = int(post.get("total_chunks"))
|
||||
|
||||
upload_dir = folder_paths.get_input_directory()
|
||||
file_path = os.path.join(upload_dir, filename)
|
||||
|
||||
# Append to file if it's not the first chunk, otherwise write new
|
||||
mode = "ab" if chunk_index > 0 else "wb"
|
||||
with open(file_path, mode) as f:
|
||||
f.write(file.file.read())
|
||||
|
||||
if chunk_index == total_chunks - 1:
|
||||
return web.json_response({"name": filename})
|
||||
return web.json_response({"status": "ok"})
|
||||
|
||||
|
||||
class LoadVideoUI:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"video": ("STRING", {"default": ""}),
|
||||
"start_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
|
||||
"end_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
|
||||
"duration": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
|
||||
"start_frame": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}),
|
||||
"end_frame": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}),
|
||||
"duration_frames": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}),
|
||||
"resize_method": (["maintain aspect ratio", "stretch to fit", "pad", "crop"], {"default": "maintain aspect ratio"}),
|
||||
"custom_width": ("INT", {"default": 0, "min": 0, "max": 100000, "step": 8, "tooltip": "Custom width. 0 means original width."}),
|
||||
"custom_height": ("INT", {"default": 0, "min": 0, "max": 100000, "step": 8, "tooltip": "Custom height. 0 means original height."}),
|
||||
"frame_rate": ("INT", {"default": 24, "min": 1, "max": 120, "step": 1, "tooltip": "Force the video to a specific frame rate for extraction."}),
|
||||
"display_mode": (["seconds", "frames"], {"default": "seconds"}),
|
||||
"crop_x": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}),
|
||||
"crop_y": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}),
|
||||
"crop_w": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}),
|
||||
"crop_h": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("IMAGE", "AUDIO", "FLOAT", "INT")
|
||||
RETURN_NAMES = ("images", "audio", "duration", "frame_count")
|
||||
FUNCTION = "load_video"
|
||||
CATEGORY = "WhatDreamsCost"
|
||||
|
||||
def load_video(self, video, frame_rate, display_mode, start_time, end_time, duration, start_frame, end_frame, duration_frames, custom_width=0, custom_height=0, resize_method="maintain aspect ratio", crop_x=0.0, crop_y=0.0, crop_w=1.0, crop_h=1.0, **kwargs):
|
||||
if not video:
|
||||
# Return blank defaults if no video is loaded
|
||||
empty_image = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
|
||||
empty_audio = {"waveform": torch.zeros((1, 1, 44100)), "sample_rate": 44100}
|
||||
return (empty_image, empty_audio, 0.0, 0)
|
||||
|
||||
# 1. Resolve path using ComfyUI standard paths or Absolute Path
|
||||
video_path = video # Try exact/absolute path first
|
||||
if not os.path.exists(video_path):
|
||||
video_path_annotated = folder_paths.get_annotated_filepath(video)
|
||||
if os.path.exists(video_path_annotated):
|
||||
video_path = video_path_annotated
|
||||
else:
|
||||
video_path_input = os.path.join(folder_paths.get_input_directory(), video)
|
||||
if os.path.exists(video_path_input):
|
||||
video_path = video_path_input
|
||||
else:
|
||||
raise FileNotFoundError(f"Video file not found: {video}")
|
||||
|
||||
# Open container to read streams and metadata
|
||||
container = av.open(video_path)
|
||||
|
||||
# Determine video stream and duration
|
||||
video_stream = container.streams.video[0] if len(container.streams.video) > 0 else None
|
||||
video_duration = 0
|
||||
if video_stream and video_stream.duration and video_stream.time_base:
|
||||
video_duration = float(video_stream.duration * video_stream.time_base)
|
||||
|
||||
orig_w = video_stream.codec_context.width if video_stream else 512
|
||||
orig_h = video_stream.codec_context.height if video_stream else 512
|
||||
|
||||
# Determine correct colorspace and color range for PyAV conversion to prevent color shift
|
||||
try:
|
||||
from av.video.reformatter import Colorspace, ColorRange
|
||||
# Improve fallback heuristic to check both dimensions (e.g. 720x1280 vertical video is HD)
|
||||
fallback_cs = Colorspace.ITU709 if max(orig_w, orig_h) >= 720 else Colorspace.ITU601
|
||||
fallback_cr = ColorRange.MPEG
|
||||
dst_range = ColorRange.JPEG # RGB should always be full range
|
||||
except ImportError:
|
||||
fallback_cs = "itu709" if max(orig_w, orig_h) >= 720 else "itu601"
|
||||
fallback_cr = "mpeg"
|
||||
dst_range = "jpeg"
|
||||
|
||||
src_colorspace = fallback_cs
|
||||
src_color_range = fallback_cr
|
||||
|
||||
if video_stream and video_stream.codec_context:
|
||||
cc = video_stream.codec_context
|
||||
|
||||
c_space = getattr(cc, 'colorspace', getattr(cc, 'color_space', None))
|
||||
if c_space and hasattr(c_space, 'name') and c_space.name != "UNSPECIFIED":
|
||||
src_colorspace = c_space
|
||||
elif c_space and isinstance(c_space, str) and "unspecified" not in c_space.lower():
|
||||
src_colorspace = c_space
|
||||
|
||||
c_range = getattr(cc, 'color_range', None)
|
||||
if c_range and hasattr(c_range, 'name') and c_range.name != "UNSPECIFIED":
|
||||
src_color_range = c_range
|
||||
elif c_range and isinstance(c_range, str) and "unspecified" not in c_range.lower():
|
||||
src_color_range = c_range
|
||||
|
||||
target_w = custom_width if custom_width > 0 else orig_w
|
||||
target_h = custom_height if custom_height > 0 else orig_h
|
||||
|
||||
target_w = target_w - (target_w % 2)
|
||||
target_h = target_h - (target_h % 2)
|
||||
|
||||
# Calculate manual crop from interactive UI first
|
||||
manual_crop_left = int(orig_w * crop_x)
|
||||
manual_crop_top = int(orig_h * crop_y)
|
||||
manual_crop_right = orig_w - int(orig_w * (crop_x + crop_w))
|
||||
manual_crop_bottom = orig_h - int(orig_h * (crop_y + crop_h))
|
||||
|
||||
# Ensure we don't crop more than the image
|
||||
manual_crop_left = max(0, min(manual_crop_left, orig_w - 1))
|
||||
manual_crop_top = max(0, min(manual_crop_top, orig_h - 1))
|
||||
manual_crop_right = max(0, min(manual_crop_right, orig_w - manual_crop_left - 1))
|
||||
manual_crop_bottom = max(0, min(manual_crop_bottom, orig_h - manual_crop_top - 1))
|
||||
|
||||
# After manual crop, the new original dimensions are:
|
||||
cropped_orig_w = orig_w - manual_crop_left - manual_crop_right
|
||||
cropped_orig_h = orig_h - manual_crop_top - manual_crop_bottom
|
||||
|
||||
# If no custom width/height is provided, use the cropped original dimensions
|
||||
if custom_width == 0:
|
||||
target_w = cropped_orig_w
|
||||
target_w = target_w - (target_w % 2)
|
||||
if custom_height == 0:
|
||||
target_h = cropped_orig_h
|
||||
target_h = target_h - (target_h % 2)
|
||||
|
||||
scale_w, scale_h = target_w, target_h
|
||||
pad_left = pad_right = pad_top = pad_bottom = 0
|
||||
crop_left = crop_right = crop_top = crop_bottom = 0
|
||||
|
||||
if custom_width > 0 or custom_height > 0:
|
||||
if resize_method == "maintain aspect ratio" or resize_method == "pad":
|
||||
ratio = min(target_w / cropped_orig_w, target_h / cropped_orig_h)
|
||||
scale_w = int(cropped_orig_w * ratio)
|
||||
scale_h = int(cropped_orig_h * ratio)
|
||||
scale_w = scale_w - (scale_w % 2)
|
||||
scale_h = scale_h - (scale_h % 2)
|
||||
|
||||
if resize_method == "pad":
|
||||
pad_x = target_w - scale_w
|
||||
pad_y = target_h - scale_h
|
||||
pad_left = pad_x // 2
|
||||
pad_right = pad_x - pad_left
|
||||
pad_top = pad_y // 2
|
||||
pad_bottom = pad_y - pad_top
|
||||
else:
|
||||
target_w, target_h = scale_w, scale_h
|
||||
|
||||
elif resize_method == "crop":
|
||||
ratio = max(target_w / cropped_orig_w, target_h / cropped_orig_h)
|
||||
scale_w = int(cropped_orig_w * ratio)
|
||||
scale_h = int(cropped_orig_h * ratio)
|
||||
scale_w = scale_w - (scale_w % 2)
|
||||
scale_h = scale_h - (scale_h % 2)
|
||||
|
||||
crop_x = scale_w - target_w
|
||||
crop_y = scale_h - target_h
|
||||
crop_left = crop_x // 2
|
||||
crop_right = crop_x - crop_left
|
||||
crop_top = crop_y // 2
|
||||
crop_bottom = crop_y - crop_top
|
||||
|
||||
elif resize_method == "stretch to fit":
|
||||
scale_w, scale_h = target_w, target_h
|
||||
|
||||
# Determine exact bounds based on frontend mode
|
||||
if display_mode == "frames":
|
||||
fr = float(frame_rate) if frame_rate > 0 else 24.0
|
||||
actual_start_time = float(start_frame) / fr
|
||||
actual_end_time = float(end_frame) / fr if (end_frame > 0 and end_frame > start_frame) else video_duration
|
||||
else:
|
||||
actual_start_time = start_time
|
||||
actual_end_time = end_time if (end_time > 0 and end_time > start_time) else video_duration
|
||||
|
||||
if actual_end_time <= 0:
|
||||
actual_end_time = float('inf') # Fallback if duration is unknown
|
||||
|
||||
# 2. Extract Video Frames (PyAV)
|
||||
frames = []
|
||||
image_tensor = None
|
||||
frames_loaded = 0
|
||||
|
||||
if video_stream:
|
||||
video_stream.thread_type = "AUTO" # Enable multithreaded decoding
|
||||
|
||||
# Efficiently seek backwards to the nearest keyframe
|
||||
if video_stream.time_base:
|
||||
seek_pts = int(actual_start_time / float(video_stream.time_base))
|
||||
else:
|
||||
seek_pts = int(actual_start_time * av.time_base)
|
||||
|
||||
container.seek(seek_pts, stream=video_stream, backward=True)
|
||||
|
||||
# Custom sampling to force specific framerate
|
||||
frame_interval = 1.0 / float(frame_rate) if frame_rate > 0 else 1.0/24.0
|
||||
expected_target_time = actual_start_time
|
||||
|
||||
# Pre-calculate expected frames
|
||||
alloc_end_time = actual_end_time if actual_end_time != float('inf') else video_duration
|
||||
expected_frames = 0
|
||||
if alloc_end_time > 0:
|
||||
duration_to_extract = alloc_end_time - actual_start_time
|
||||
if duration_to_extract > 0:
|
||||
expected_frames = int(np.ceil(duration_to_extract / frame_interval)) + 2
|
||||
|
||||
pbar = comfy.utils.ProgressBar(expected_frames) if expected_frames > 0 else None
|
||||
|
||||
for frame in container.decode(video_stream):
|
||||
frame_time = frame.time
|
||||
if frame_time is None:
|
||||
frame_time = float(frame.pts * float(video_stream.time_base)) if frame.pts and video_stream.time_base else 0.0
|
||||
|
||||
if frame_time < actual_start_time:
|
||||
continue
|
||||
|
||||
# Add a slight buffer (interval) to ensure we evaluate the boundary correctly
|
||||
if frame_time > actual_end_time + frame_interval:
|
||||
break
|
||||
|
||||
# Fix PyAV color shift by forcing proper colorspace and range conversion.
|
||||
# Omit dst_colorspace so swscale defaults naturally for RGB output
|
||||
# (passing it can cause the YUV matrix to be applied incorrectly).
|
||||
try:
|
||||
frame = frame.reformat(
|
||||
format="rgb24",
|
||||
src_colorspace=src_colorspace,
|
||||
src_color_range=src_color_range,
|
||||
dst_color_range=dst_range
|
||||
)
|
||||
frame_rgb = frame.to_ndarray(format='rgb24')
|
||||
except Exception as e:
|
||||
# Fallback: if explicit color reformat fails, use PyAV's default conversion
|
||||
print(f"[LoadVideoUI] Color reformat failed, using default: {e}")
|
||||
frame_rgb = frame.to_ndarray(format='rgb24')
|
||||
|
||||
# Apply interactive crop first
|
||||
if manual_crop_left > 0 or manual_crop_top > 0 or manual_crop_right > 0 or manual_crop_bottom > 0:
|
||||
frame_rgb = frame_rgb[manual_crop_top:orig_h-manual_crop_bottom, manual_crop_left:orig_w-manual_crop_right, :]
|
||||
|
||||
# Now resize to the scaled dimensions
|
||||
if scale_w != cropped_orig_w or scale_h != cropped_orig_h:
|
||||
import cv2
|
||||
frame_rgb = cv2.resize(frame_rgb, (scale_w, scale_h), interpolation=cv2.INTER_AREA)
|
||||
|
||||
if crop_left > 0 or crop_top > 0 or crop_right > 0 or crop_bottom > 0:
|
||||
frame_rgb = frame_rgb[crop_top:scale_h-crop_bottom, crop_left:scale_w-crop_right, :]
|
||||
if pad_left > 0 or pad_top > 0 or pad_right > 0 or pad_bottom > 0:
|
||||
frame_rgb = np.pad(frame_rgb, ((pad_top, pad_bottom), (pad_left, pad_right), (0, 0)), mode='constant', constant_values=0)
|
||||
|
||||
# Duplicate or skip frames perfectly based on timestamps to meet forced framerate.
|
||||
# FIX: Use strictly less than (<) for actual_end_time to prevent the loop from fetching an extra +1 frame
|
||||
# at the exact boundary of the duration slice!
|
||||
while expected_target_time <= frame_time and expected_target_time < actual_end_time - 1e-5:
|
||||
if image_tensor is None and expected_frames > 0:
|
||||
# First frame: allocate the tensor
|
||||
height, width = frame_rgb.shape[:2]
|
||||
alloc_frames = expected_frames + 50 # Add generous buffer to prevent reallocation
|
||||
try:
|
||||
image_tensor = torch.zeros((alloc_frames, height, width, 3), dtype=torch.float32)
|
||||
except Exception as e:
|
||||
print(f"[LoadVideoUI] Pre-allocation failed, falling back to list: {e}")
|
||||
expected_frames = 0 # Disable pre-allocation
|
||||
|
||||
if image_tensor is not None:
|
||||
# Check bounds (just in case)
|
||||
if frames_loaded >= image_tensor.shape[0]:
|
||||
# Extend tensor if we underestimated
|
||||
extension = torch.zeros((50, image_tensor.shape[1], image_tensor.shape[2], 3), dtype=torch.float32)
|
||||
image_tensor = torch.cat((image_tensor, extension), dim=0)
|
||||
|
||||
# Insert frame with minimal memory copy directly to tensor
|
||||
image_tensor[frames_loaded] = torch.from_numpy(frame_rgb).float().div_(255.0)
|
||||
frames_loaded += 1
|
||||
else:
|
||||
# Fallback list append if pre-allocation failed
|
||||
frames.append(frame_rgb)
|
||||
|
||||
if pbar:
|
||||
pbar.update(1)
|
||||
|
||||
expected_target_time += frame_interval
|
||||
|
||||
# Convert frames to ComfyUI Image standard format [N, H, W, C], float32, range 0.0-1.0
|
||||
if image_tensor is not None:
|
||||
if frames_loaded > 0:
|
||||
image_tensor = image_tensor[:frames_loaded]
|
||||
else:
|
||||
image_tensor = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
|
||||
elif len(frames) > 0:
|
||||
frames_np = np.array(frames, dtype=np.float32) / 255.0
|
||||
image_tensor = torch.from_numpy(frames_np)
|
||||
else:
|
||||
# Fallback for an empty slice
|
||||
image_tensor = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
|
||||
|
||||
# 3. Extract Audio (PyAV)
|
||||
audio_dict = {"waveform": torch.zeros((1, 1, 44100)), "sample_rate": 44100} # Default empty audio
|
||||
|
||||
if len(container.streams.audio) > 0:
|
||||
try:
|
||||
audio_stream = container.streams.audio[0]
|
||||
audio_stream.thread_type = "AUTO"
|
||||
sample_rate = getattr(audio_stream, 'rate', 44100) or 44100
|
||||
|
||||
# We must seek again on the container specifically for the audio stream
|
||||
if audio_stream.time_base:
|
||||
seek_pts = int(actual_start_time / float(audio_stream.time_base))
|
||||
else:
|
||||
seek_pts = int(actual_start_time * av.time_base)
|
||||
|
||||
container.seek(seek_pts, stream=audio_stream, backward=True)
|
||||
|
||||
# Resample to standard float planar format (fltp)
|
||||
resampler = av.AudioResampler(format='fltp')
|
||||
|
||||
audio_data = []
|
||||
first_frame_time = None
|
||||
|
||||
for frame in container.decode(audio_stream):
|
||||
frame_time = frame.time
|
||||
if frame_time is None:
|
||||
frame_time = float(frame.pts * float(audio_stream.time_base)) if frame.pts and audio_stream.time_base else 0.0
|
||||
|
||||
# Give a small 1-second buffer to ensure we catch end frames
|
||||
if frame_time > actual_end_time + 1.0:
|
||||
break
|
||||
|
||||
if first_frame_time is None:
|
||||
first_frame_time = frame_time
|
||||
|
||||
resampled_frames = resampler.resample(frame)
|
||||
for r_frame in resampled_frames:
|
||||
audio_data.append(r_frame.to_ndarray())
|
||||
|
||||
if audio_data:
|
||||
# Concatenate all frames horizontally along the sample axis
|
||||
waveform_np = np.concatenate(audio_data, axis=1)
|
||||
waveform = torch.from_numpy(waveform_np).float()
|
||||
|
||||
if first_frame_time is None:
|
||||
first_frame_time = 0.0
|
||||
|
||||
# Calculate exact slice points to trim precisely
|
||||
offset_sec = max(0.0, actual_start_time - first_frame_time)
|
||||
start_sample = int(offset_sec * sample_rate)
|
||||
|
||||
duration_sec_audio = actual_end_time - actual_start_time
|
||||
end_sample = start_sample + int(duration_sec_audio * sample_rate)
|
||||
|
||||
# Trim array bounds properly
|
||||
if end_sample > start_sample:
|
||||
waveform = waveform[:, start_sample:end_sample]
|
||||
else:
|
||||
waveform = waveform[:, start_sample:]
|
||||
|
||||
# Expand to ComfyUI Audio standard [batch_size, channels, samples]
|
||||
waveform = waveform.unsqueeze(0)
|
||||
audio_dict = {"waveform": waveform, "sample_rate": sample_rate}
|
||||
|
||||
except Exception as e:
|
||||
# Catch gracefully without breaking the pipeline execution
|
||||
print(f"[LoadVideoUI] Audio track extraction skipped or failed: {e}")
|
||||
|
||||
# Always close container to free up system memory lock
|
||||
container.close()
|
||||
|
||||
# Output accurate final duration in seconds
|
||||
final_duration_sec = float(max(0.0, actual_end_time - actual_start_time))
|
||||
|
||||
# Accurately output the true number of extracted frames
|
||||
# (Using the shape of the array provides exact 1:1 parity with the timeline's math)
|
||||
frame_count = image_tensor.shape[0] if (frames_loaded > 0 or len(frames) > 0) else 0
|
||||
if frame_count == 0 and final_duration_sec > 0:
|
||||
# Fallback estimation only if PyAV completely failed to decode a valid chunk
|
||||
calc_fr = float(frame_rate) if frame_rate > 0 else 24.0
|
||||
frame_count = int(np.floor(final_duration_sec * calc_fr))
|
||||
|
||||
return (image_tensor, audio_dict, final_duration_sec, frame_count)
|
||||
2301
ltx_director.py
2301
ltx_director.py
File diff suppressed because it is too large
Load Diff
@@ -1,106 +1,637 @@
|
||||
from comfy_extras.nodes_lt import LTXVAddGuide
|
||||
import logging
|
||||
import math
|
||||
import os
|
||||
import av
|
||||
import numpy as np
|
||||
import torch
|
||||
import comfy
|
||||
import comfy.sd
|
||||
import comfy.utils
|
||||
import folder_paths
|
||||
import node_helpers
|
||||
from comfy_extras import nodes_lt
|
||||
from comfy_api.latest import io
|
||||
from .ltx_director import GuideData
|
||||
from .ltx_director import GuideData, MotionGuideData, _resize_image, _execute_comfy_node, _unpack
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
class LTXDirectorGuide(LTXVAddGuide):
|
||||
@classmethod
|
||||
def define_schema(cls):
|
||||
return io.Schema(
|
||||
node_id="LTXDirectorGuide",
|
||||
display_name="LTX Director Guide",
|
||||
category="WhatDreamsCost",
|
||||
description=(
|
||||
"Applies guide images from a Prompt Relay Timeline node at the frame positions "
|
||||
"and strengths defined on the timeline. Connect guide_data from the timeline node."
|
||||
),
|
||||
inputs=[
|
||||
io.Conditioning.Input("positive", tooltip="Positive conditioning to add guide keyframe info to."),
|
||||
io.Conditioning.Input("negative", tooltip="Negative conditioning to add guide keyframe info to."),
|
||||
io.Vae.Input("vae", tooltip="Video VAE used to encode the guide images."),
|
||||
io.Latent.Input("latent", tooltip="Video latent — guides are inserted into this latent."),
|
||||
GuideData.Input("guide_data", tooltip="Guide data produced by Prompt Relay Encode (Timeline)."),
|
||||
io.Float.Input("scale_by", default=1.0, min=0.01, max=8.0, step=0.01, tooltip="Scale the latent by this factor."),
|
||||
io.Combo.Input("upscale_method", options=["nearest-exact", "bilinear", "area", "bicubic", "bislerp"], default="bicubic", tooltip="Method used to upscale/downscale the latent."),
|
||||
io.Float.Input("msr_strength", default=0.0, min=0.0, max=1.0, step=0.05, tooltip="Licon MSR only: per-stage reference strength override. 0 = use the Director's value (full pull, right for stage 1). On a refinement/upscale stage set ~0.4 to hold detail without the references repainting the opening (fixes stage-2 mist/ghosting)."),
|
||||
],
|
||||
outputs=[
|
||||
io.Conditioning.Output(display_name="positive"),
|
||||
io.Conditioning.Output(display_name="negative"),
|
||||
io.Latent.Output(display_name="latent", tooltip="Video latent with guide frames applied."),
|
||||
],
|
||||
# --- Helper Functions from Nghtdrp ---
|
||||
|
||||
def _get_guide_attention_entries(conditioning):
|
||||
for item in conditioning:
|
||||
entries = item[1].get("guide_attention_entries", None)
|
||||
if entries is not None:
|
||||
return entries
|
||||
return []
|
||||
|
||||
def _set_guide_attention_entries(conditioning, entries):
|
||||
return node_helpers.conditioning_set_values(
|
||||
conditioning, {"guide_attention_entries": entries}
|
||||
)
|
||||
|
||||
def _append_guide_attention_entry(
|
||||
conditioning,
|
||||
pre_filter_count,
|
||||
latent_shape,
|
||||
attention_strength=1.0,
|
||||
attention_mask=None,
|
||||
):
|
||||
entries = [*_get_guide_attention_entries(conditioning)]
|
||||
entries.append(
|
||||
{
|
||||
"pre_filter_count": int(pre_filter_count),
|
||||
"strength": float(attention_strength),
|
||||
"pixel_mask": attention_mask,
|
||||
"latent_shape": list(latent_shape),
|
||||
}
|
||||
)
|
||||
return _set_guide_attention_entries(conditioning, entries)
|
||||
|
||||
def _clone_noise_mask(latent, latent_image):
|
||||
if "noise_mask" in latent and latent["noise_mask"] is not None:
|
||||
return latent["noise_mask"].clone()
|
||||
batch, _, frames, _, _ = latent_image.shape
|
||||
return torch.ones(
|
||||
(batch, 1, frames, 1, 1),
|
||||
dtype=torch.float32,
|
||||
device=latent_image.device,
|
||||
)
|
||||
|
||||
def _resize_latent_spatial(latent_image, noise_mask, width, height, method):
|
||||
b, c, f, h, w = latent_image.shape
|
||||
if width == w and height == h:
|
||||
return latent_image, noise_mask
|
||||
|
||||
latent_4d = latent_image.permute(0, 2, 1, 3, 4).reshape(b * f, c, h, w)
|
||||
latent_4d = comfy.utils.common_upscale(latent_4d, width, height, method, "disabled")
|
||||
latent_image = latent_4d.reshape(b, f, c, height, width).permute(0, 2, 1, 3, 4)
|
||||
|
||||
if noise_mask is not None and (noise_mask.shape[-1] > 1 or noise_mask.shape[-2] > 1):
|
||||
mb, mc, mf, mh, mw = noise_mask.shape
|
||||
mask_4d = noise_mask.permute(0, 2, 1, 3, 4).reshape(mb * mf, mc, mh, mw)
|
||||
mask_4d = comfy.utils.common_upscale(mask_4d, width, height, method, "disabled")
|
||||
noise_mask = mask_4d.reshape(mb, mf, mc, height, width).permute(0, 2, 1, 3, 4)
|
||||
|
||||
return latent_image, noise_mask
|
||||
|
||||
def _ceil_to_multiple(value, multiple):
|
||||
multiple = max(1, int(multiple))
|
||||
return int(math.ceil(value / multiple) * multiple)
|
||||
|
||||
def _snap_latent_to_downscale(latent_image, noise_mask, downscale_factor, method):
|
||||
factor = int(max(1, round(float(downscale_factor))))
|
||||
if factor <= 1:
|
||||
return latent_image, noise_mask
|
||||
|
||||
_, _, _, h, w = latent_image.shape
|
||||
new_w = _ceil_to_multiple(w, factor)
|
||||
new_h = _ceil_to_multiple(h, factor)
|
||||
if new_w == w and new_h == h:
|
||||
return latent_image, noise_mask
|
||||
|
||||
log.warning(
|
||||
"[LTXDirectorGuide] Auto-snapping latent grid from %sx%s to %sx%s for IC-LoRA downscale factor %s.",
|
||||
w, h, new_w, new_h, factor,
|
||||
)
|
||||
return _resize_latent_spatial(latent_image, noise_mask, new_w, new_h, method)
|
||||
|
||||
def _load_lora_model_only(model, ic_lora_name, strength_model):
|
||||
lora_path = folder_paths.get_full_path_or_raise("loras", ic_lora_name)
|
||||
lora, metadata = comfy.utils.load_torch_file(
|
||||
lora_path, safe_load=True, return_metadata=True,
|
||||
)
|
||||
|
||||
try:
|
||||
latent_downscale_factor = float(metadata["reference_downscale_factor"])
|
||||
except Exception:
|
||||
latent_downscale_factor = 1.0
|
||||
log.warning(
|
||||
"[LTXDirectorGuide] Could not read reference_downscale_factor from %s, using 1.0.",
|
||||
ic_lora_name,
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(cls, positive, negative, vae, latent, guide_data, scale_by=1.0, upscale_method="bicubic", msr_strength=0.0) -> io.NodeOutput:
|
||||
scale_factors = vae.downscale_index_formula
|
||||
if strength_model == 0:
|
||||
return model, latent_downscale_factor
|
||||
|
||||
# Clone latents to avoid mutating upstream nodes
|
||||
latent_image = latent["samples"].clone()
|
||||
model_lora, _ = comfy.sd.load_lora_for_models(
|
||||
model, None, lora, strength_model, 0,
|
||||
)
|
||||
return model_lora, latent_downscale_factor
|
||||
|
||||
if "noise_mask" in latent:
|
||||
noise_mask = latent["noise_mask"].clone()
|
||||
def _encode_video_iclora_guide(vae, latent_width, latent_height, images, scale_factors, latent_downscale_factor, crop, use_tiled_encode, tile_size, tile_overlap, resize_method="crop"):
|
||||
time_scale_factor, width_scale_factor, height_scale_factor = scale_factors
|
||||
|
||||
num_frames_to_keep = ((images.shape[0] - 1) // time_scale_factor) * time_scale_factor + 1
|
||||
images = images[:num_frames_to_keep]
|
||||
|
||||
target_width = int(latent_width * width_scale_factor / latent_downscale_factor)
|
||||
target_height = int(latent_height * height_scale_factor / latent_downscale_factor)
|
||||
target_width = max(8, target_width)
|
||||
target_height = max(8, target_height)
|
||||
|
||||
# For guides, we must match the exact target latent shape.
|
||||
# "maintain aspect ratio" doesn't pad, which causes shape mismatch in VAE encode / concatenation.
|
||||
# So we fallback "maintain aspect ratio" to "pad".
|
||||
if resize_method == "maintain aspect ratio":
|
||||
resize_method = "pad"
|
||||
|
||||
pixels = _resize_image(images, target_width, target_height, resize_method, divisible_by=1)
|
||||
|
||||
encode_pixels = pixels[:, :, :, :3]
|
||||
if use_tiled_encode:
|
||||
guide_latent = vae.encode_tiled(encode_pixels, tile_x=tile_size, tile_y=tile_size, overlap=tile_overlap)
|
||||
else:
|
||||
guide_latent = vae.encode(encode_pixels)
|
||||
|
||||
return pixels, guide_latent
|
||||
|
||||
def _dilate_latent(latent: dict, horizontal_scale: int, vertical_scale: int) -> dict:
|
||||
if horizontal_scale == 1 and vertical_scale == 1:
|
||||
return latent
|
||||
|
||||
samples = latent["samples"]
|
||||
mask = latent.get("noise_mask", None)
|
||||
|
||||
dilated_shape = samples.shape[:3] + (
|
||||
samples.shape[3] * vertical_scale,
|
||||
samples.shape[4] * horizontal_scale,
|
||||
)
|
||||
dilated_samples = torch.zeros(dilated_shape, device=samples.device, dtype=samples.dtype, requires_grad=False)
|
||||
dilated_samples[..., ::vertical_scale, ::horizontal_scale] = samples
|
||||
|
||||
dilated_mask_shape = (
|
||||
dilated_samples.shape[0], 1, dilated_samples.shape[2],
|
||||
dilated_samples.shape[3], dilated_samples.shape[4],
|
||||
)
|
||||
dilated_mask = torch.full(dilated_mask_shape, -1.0, device=samples.device, dtype=samples.dtype, requires_grad=False)
|
||||
dilated_mask[..., ::vertical_scale, ::horizontal_scale] = (mask if mask is not None else 1.0)
|
||||
|
||||
return {"samples": dilated_samples, "noise_mask": dilated_mask}
|
||||
|
||||
def _resolve_input_video_path(video_file):
|
||||
if os.path.isabs(str(video_file)) and os.path.exists(str(video_file)):
|
||||
return str(video_file)
|
||||
input_dir = folder_paths.get_input_directory()
|
||||
candidate = os.path.join(input_dir, str(video_file))
|
||||
if os.path.exists(candidate):
|
||||
return candidate
|
||||
try:
|
||||
annotated = folder_paths.get_annotated_filepath(str(video_file))
|
||||
if annotated and os.path.exists(annotated):
|
||||
return annotated
|
||||
except Exception:
|
||||
pass
|
||||
raise FileNotFoundError(f"Could not find motion guide video: {video_file}")
|
||||
|
||||
class ResampleGuideFrames:
|
||||
def execute(self, images, source_fps, target_fps, target_num_frames, mode):
|
||||
if images is None: return images
|
||||
frames = images
|
||||
n = int(frames.shape[0])
|
||||
target_num_frames = int(target_num_frames)
|
||||
if n <= 1:
|
||||
if target_num_frames > 1 and n == 1:
|
||||
return frames.repeat(target_num_frames, 1, 1, 1)
|
||||
return frames
|
||||
source_fps = float(max(0.001, source_fps))
|
||||
target_fps = float(max(0.001, target_fps))
|
||||
if target_num_frames <= 0:
|
||||
duration = (n - 1) / source_fps
|
||||
target_num_frames = max(1, int(round(duration * target_fps)) + 1)
|
||||
if target_num_frames == n and abs(target_fps - source_fps) < 1e-6:
|
||||
return frames
|
||||
positions = torch.linspace(0, n - 1, target_num_frames, device=frames.device, dtype=torch.float32)
|
||||
if mode == "nearest":
|
||||
idx = torch.round(positions).long().clamp(0, n - 1)
|
||||
return frames.index_select(0, idx)
|
||||
idx0 = torch.floor(positions).long().clamp(0, n - 1)
|
||||
idx1 = torch.ceil(positions).long().clamp(0, n - 1)
|
||||
alpha = (positions - idx0.to(positions.dtype)).view(-1, 1, 1, 1)
|
||||
f0 = frames.index_select(0, idx0).to(torch.float32)
|
||||
f1 = frames.index_select(0, idx1).to(torch.float32)
|
||||
return (f0 * (1.0 - alpha) + f1 * alpha).to(frames.dtype)
|
||||
|
||||
def _load_motion_video_frames(video_file, trim_start_frames, length_frames, director_fps, resample_mode="nearest"):
|
||||
path = _resolve_input_video_path(video_file)
|
||||
target_fps = max(1.0, float(director_fps))
|
||||
start_s = max(0.0, float(trim_start_frames) / target_fps)
|
||||
dur_s = max(0.0, float(length_frames) / target_fps)
|
||||
end_s = start_s + dur_s if dur_s > 0 else None
|
||||
|
||||
log.info(f"[LTXDirectorGuide] Loading video frames from {path}. start_s: {start_s:.3f}, dur_s: {dur_s:.3f}, end_s: {end_s}")
|
||||
|
||||
container = av.open(path)
|
||||
stream = container.streams.video[0]
|
||||
stream.thread_type = "AUTO" # Enable multi-threaded decoding
|
||||
|
||||
try:
|
||||
source_fps = float(stream.average_rate) if stream.average_rate else float(stream.base_rate)
|
||||
except Exception:
|
||||
source_fps = target_fps
|
||||
if source_fps <= 0: source_fps = target_fps
|
||||
|
||||
# Seek to keyframe slightly before target start_s to optimize loading speed and memory
|
||||
if start_s > 0:
|
||||
try:
|
||||
if stream.time_base:
|
||||
seek_pts = int(max(0, start_s - 0.5) / float(stream.time_base))
|
||||
else:
|
||||
seek_pts = int(max(0, start_s - 0.5) * av.time_base)
|
||||
container.seek(seek_pts, stream=stream, backward=True)
|
||||
log.info(f"[LTXDirectorGuide] PyAV seeked stream to pts={seek_pts} (approx {max(0, start_s - 0.5):.3f}s)")
|
||||
except Exception as seek_err:
|
||||
log.warning(f"[LTXDirectorGuide] Seek failed: {seek_err}, decoding from beginning.")
|
||||
|
||||
frames = []
|
||||
decoded_count = 0
|
||||
for frame in container.decode(stream):
|
||||
if frame.time is not None:
|
||||
t = float(frame.time)
|
||||
elif frame.pts is not None and stream.time_base is not None:
|
||||
t = float(frame.pts * stream.time_base)
|
||||
else:
|
||||
batch, _, latent_frames, latent_height, latent_width = latent_image.shape
|
||||
noise_mask = torch.ones(
|
||||
(batch, 1, latent_frames, 1, 1),
|
||||
dtype=torch.float32,
|
||||
device=latent_image.device,
|
||||
)
|
||||
t = float(decoded_count / source_fps)
|
||||
decoded_count += 1
|
||||
|
||||
if t < start_s - 0.01: continue
|
||||
if end_s is not None and t >= end_s: break
|
||||
|
||||
# Append raw uint8 numpy arrays to minimize CPU allocation overhead
|
||||
frames.append(frame.to_ndarray(format="rgb24"))
|
||||
container.close()
|
||||
|
||||
if not frames: raise ValueError(f"No frames decoded for motion guide segment: {video_file}")
|
||||
|
||||
# Convert all frames to float32 at once to optimize memory allocation
|
||||
frames_np = np.array(frames, dtype=np.float32) / 255.0
|
||||
images = torch.from_numpy(frames_np)
|
||||
|
||||
target_count = max(1, int(round(float(length_frames))))
|
||||
images = ResampleGuideFrames().execute(images, source_fps, target_fps, target_count, resample_mode)
|
||||
return images
|
||||
|
||||
# --- Main Class ---
|
||||
|
||||
class LTXDirectorGuide:
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
loras = folder_paths.get_filename_list("loras")
|
||||
if not loras: loras = ["put_ic_lora_in_ComfyUI_models_loras"]
|
||||
return {
|
||||
"required": {
|
||||
"positive": ("CONDITIONING", {"tooltip": "Positive conditioning to add guide keyframe info to."}),
|
||||
"negative": ("CONDITIONING", {"tooltip": "Negative conditioning to add guide keyframe info to."}),
|
||||
"vae": ("VAE", {"tooltip": "Video VAE used to encode the guide images."}),
|
||||
"latent": ("LATENT", {"tooltip": "Video latent — guides are inserted into this latent."}),
|
||||
"guide_data": ("GUIDE_DATA", {"tooltip": "Guide data produced by Prompt Relay Encode (Timeline)."}),
|
||||
},
|
||||
"optional": {
|
||||
"motion_guide_data": ("MOTION_GUIDE_DATA", {"tooltip": "Connect motion guide data from the timeline node to use IC-LoRA video guidance."}),
|
||||
"model": ("MODEL", {"tooltip": "Connect model if using IC-LoRA for motion guidance."}),
|
||||
"ic_lora_name": (["None"] + loras, {"default": "None", "tooltip": "Select the IC-LoRA model to use for motion guidance."}),
|
||||
"ic_lora_strength": ("FLOAT", {"default": 1.0, "min": -100.0, "max": 100.0, "step": 0.01}),
|
||||
"scale_by": ("FLOAT", {"default": 0.5, "min": 0.01, "max": 8.0, "step": 0.01, "tooltip": "Scale the latent by this factor. 0.5 is recommended for IC-LoRA."}),
|
||||
"upscale_method": (["nearest-exact", "bilinear", "area", "bicubic", "bislerp"], {"default": "bicubic", "tooltip": "Method used to upscale/downscale the latent."}),
|
||||
"image_attention_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}),
|
||||
"crop": (["disabled", "center"], {"default": "center"}),
|
||||
"auto_snap_ic_grid": ("BOOLEAN", {"default": True}),
|
||||
"use_tiled_encode": ("BOOLEAN", {"default": False}),
|
||||
"tile_size": ("INT", {"default": 256, "min": 64, "max": 512, "step": 32}),
|
||||
"tile_overlap": ("INT", {"default": 64, "min": 16, "max": 256, "step": 16}),
|
||||
"retake_mode": ("BOOLEAN", {"default": False, "tooltip": "Force Retake Mode. If false, it will still auto-detect Retake Mode from the timeline data."}),
|
||||
"msr_strength": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.05, "tooltip": "Licon MSR only: per-stage reference strength override. 0 = use the Director's value (full pull, right for stage 1). On a refinement/upscale stage set ~0.4 to hold detail without the references repainting the opening (fixes stage-2 mist/ghosting)."}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT", "MODEL", "FLOAT")
|
||||
RETURN_NAMES = ("positive", "negative", "latent", "model", "latent_downscale_factor")
|
||||
FUNCTION = "execute"
|
||||
|
||||
@classmethod
|
||||
def execute(cls, positive, negative, vae, latent, guide_data, motion_guide_data=None, model=None, ic_lora_name="None", ic_lora_strength=1.0, scale_by=1.0, upscale_method="bicubic", image_attention_strength=1.0, crop="center", auto_snap_ic_grid=True, use_tiled_encode=False, tile_size=256, tile_overlap=64, retake_mode=False, msr_strength=0.0):
|
||||
motion_segments = (motion_guide_data or {}).get("segments", []) if motion_guide_data else []
|
||||
image_guides_count = len(guide_data.get("images", [])) if guide_data else 0
|
||||
print(f"[LTXDirectorGuide] execute started. motion_segments: {len(motion_segments)}, image_guides: {image_guides_count}, ic_lora_name: {ic_lora_name}, model connected: {model is not None}, retake_mode: {retake_mode}")
|
||||
|
||||
active_resize_method = guide_data.get("resize_method") if guide_data else None
|
||||
if not active_resize_method:
|
||||
active_resize_method = motion_guide_data.get("resize_method") if motion_guide_data else None
|
||||
if not active_resize_method:
|
||||
active_resize_method = "crop" if crop == "center" else "stretch to fit"
|
||||
|
||||
latent_downscale_factor = 1.0
|
||||
if model is not None and ic_lora_name != "None":
|
||||
model, latent_downscale_factor = _load_lora_model_only(model, ic_lora_name, ic_lora_strength)
|
||||
|
||||
scale_factors = vae.downscale_index_formula
|
||||
latent_image = latent["samples"].clone()
|
||||
noise_mask = _clone_noise_mask(latent, latent_image)
|
||||
|
||||
# Apply scale factor if not 1.0
|
||||
if scale_by != 1.0:
|
||||
B, C, F, H, W = latent_image.shape
|
||||
width = round(W * scale_by)
|
||||
height = round(H * scale_by)
|
||||
|
||||
# Reshape to 4D for common_upscale
|
||||
latent_4d = latent_image.permute(0, 2, 1, 3, 4).reshape(B * F, C, H, W)
|
||||
latent_resized_4d = comfy.utils.common_upscale(latent_4d, width, height, upscale_method, "disabled")
|
||||
latent_image = latent_resized_4d.reshape(B, F, C, height, width).permute(0, 2, 1, 3, 4)
|
||||
_, _, _, h, w = latent_image.shape
|
||||
target_w = max(1, round(w * scale_by))
|
||||
target_h = max(1, round(h * scale_by))
|
||||
latent_image, noise_mask = _resize_latent_spatial(latent_image, noise_mask, target_w, target_h, upscale_method)
|
||||
|
||||
# Also resize noise mask if it's not a broadcasted mask
|
||||
if noise_mask.shape[-1] > 1 or noise_mask.shape[-2] > 1:
|
||||
mask_4d = noise_mask.permute(0, 2, 1, 3, 4).reshape(B * F, 1, H, W)
|
||||
mask_resized_4d = comfy.utils.common_upscale(mask_4d, width, height, upscale_method, "disabled")
|
||||
noise_mask = mask_resized_4d.reshape(B, F, 1, height, width).permute(0, 2, 1, 3, 4)
|
||||
if auto_snap_ic_grid and model is not None and ic_lora_name != "None":
|
||||
latent_image, noise_mask = _snap_latent_to_downscale(latent_image, noise_mask, latent_downscale_factor, upscale_method)
|
||||
|
||||
# Ghost Mask: the Director kept the latent clean (so it stays length-matched with the audio
|
||||
# latent) and asked us to pre-extend it here by one hidden frame per reference. The tail
|
||||
# sits past the clean region; the reference images (in guide_data["images"], anchored at
|
||||
# (clean + i)) are written into it by the loop below, then dropped downstream by Clean
|
||||
# Latent Slice (length = clean_latent_frames).
|
||||
_ghost_pre_extend = int(guide_data.get("ghost_pre_extend", 0)) if guide_data else 0
|
||||
if _ghost_pre_extend > 0:
|
||||
gb, gc, gf, gh, gw = latent_image.shape
|
||||
latent_image = torch.cat(
|
||||
[latent_image, torch.zeros((gb, gc, _ghost_pre_extend, gh, gw), dtype=latent_image.dtype, device=latent_image.device)],
|
||||
dim=2,
|
||||
)
|
||||
mb, mc, mf, mh, mw = noise_mask.shape
|
||||
noise_mask = torch.cat(
|
||||
[noise_mask, torch.ones((mb, mc, _ghost_pre_extend, mh, mw), dtype=noise_mask.dtype, device=noise_mask.device)],
|
||||
dim=2,
|
||||
)
|
||||
|
||||
_, _, latent_length, latent_height, latent_width = latent_image.shape
|
||||
initial_latent_length = int(latent_length)
|
||||
|
||||
# MSR mode: the Director handed us the raw references; inject them at THIS node's
|
||||
# resolution (so a half-res Stage 1 and a full-res Stage 2 each stay self-consistent).
|
||||
msr = guide_data.get("msr")
|
||||
# Licon MSR (Prefix) mode: the Director handed us raw references via guide_data["msr"].
|
||||
# Inject them at THIS node's resolution (so a half-res Stage 1 and full-res Stage 2 each
|
||||
# stay self-consistent), then return early — the standard guide paths don't apply.
|
||||
msr = guide_data.get("msr") if guide_data else None
|
||||
if msr is not None:
|
||||
return cls._inject_msr(positive, negative, vae, latent_image, noise_mask, msr, msr_strength)
|
||||
return cls._inject_msr(positive, negative, vae, latent_image, noise_mask, msr, model, latent_downscale_factor, msr_strength)
|
||||
|
||||
images = guide_data.get("images", [])
|
||||
insert_frames = guide_data.get("insert_frames", [])
|
||||
strengths = guide_data.get("strengths", [])
|
||||
# Parse timeline JSON to see if retake mode is active in UI
|
||||
import json
|
||||
timeline_data_str = guide_data.get("timeline_data", "{}") if guide_data else "{}"
|
||||
try:
|
||||
tdata = json.loads(timeline_data_str)
|
||||
except Exception:
|
||||
tdata = {}
|
||||
|
||||
is_retake_active = bool(retake_mode) or tdata.get("retakeMode", False)
|
||||
is_empty_latent = (latent_image.abs().max().item() < 1e-5)
|
||||
|
||||
for idx, img_tensor in enumerate(images):
|
||||
f_idx = insert_frames[idx] if idx < len(insert_frames) else 0
|
||||
strength = strengths[idx] if idx < len(strengths) else 1.0
|
||||
# Director image guides and motion video segments
|
||||
images = guide_data.get("images", []) if guide_data else []
|
||||
insert_frames = guide_data.get("insert_frames", []) if guide_data else []
|
||||
strengths = guide_data.get("strengths", []) if guide_data else []
|
||||
|
||||
director_fps = float((motion_guide_data or {}).get("frame_rate", guide_data.get("frame_rate", 24) if guide_data else 24))
|
||||
segments = (motion_guide_data or {}).get("segments", [])
|
||||
|
||||
is_lora_active = (model is not None) and (ic_lora_name != "None")
|
||||
time_scale_factor = scale_factors[0]
|
||||
ltxv_length = (latent_length - 1) * time_scale_factor + 1
|
||||
|
||||
image_1, t = cls.encode(vae, latent_width, latent_height, img_tensor, scale_factors)
|
||||
frame_idx, latent_idx = cls.get_latent_index(positive, latent_length, len(image_1), f_idx, scale_factors)
|
||||
# -----------------------------------------------------------------------
|
||||
# Retake Mode Branch:
|
||||
# Load single base video, encode continuously, apply temporal mask.
|
||||
# -----------------------------------------------------------------------
|
||||
if is_retake_active:
|
||||
print(f"[LTXDirectorGuide] Retake Mode active. Preserving base latent, masking selected regions. is_empty_latent: {is_empty_latent}")
|
||||
target_width = latent_width * 32
|
||||
target_height = latent_height * 32
|
||||
|
||||
assert latent_idx + t.shape[2] <= latent_length, (
|
||||
f"Guide image {idx + 1}: conditioning frames exceed the length of the latent sequence."
|
||||
)
|
||||
# Calculate retake region latent indices first so we know what to copy/paste
|
||||
retake_start = int(tdata.get("retakeStart", 0))
|
||||
retake_len = int(tdata.get("retakeLength", 0))
|
||||
retake_strength = float(tdata.get("retakeStrength", 1.0))
|
||||
|
||||
positive, negative, latent_image, noise_mask = cls.append_keyframe(
|
||||
positive, negative, frame_idx, latent_image, noise_mask, t, strength, scale_factors,
|
||||
)
|
||||
start_frame = int(guide_data.get("start_frame", 0))
|
||||
relative_start = max(0, retake_start - start_frame)
|
||||
|
||||
l_start = relative_start // time_scale_factor
|
||||
l_end = int(math.ceil((relative_start + retake_len) / time_scale_factor))
|
||||
|
||||
l_start = min(l_start, latent_length)
|
||||
l_end = min(l_end, latent_length)
|
||||
|
||||
return io.NodeOutput(positive, negative, {"samples": latent_image, "noise_mask": noise_mask})
|
||||
# Stage 2 optimization: If the retake region covers the entire generation area,
|
||||
# there are no preserved regions to copy over in Stage 2. We can bypass video loading/encoding.
|
||||
need_base_video = True
|
||||
if not is_empty_latent and l_start == 0 and l_end >= latent_length:
|
||||
need_base_video = False
|
||||
print("[LTXDirectorGuide] Stage 2: Retake region covers the entire generated range. Skipping base video loading and VAE encoding.")
|
||||
|
||||
# 1. Try to load and encode base video from timeline data
|
||||
retake_vid_info = tdata.get("retakeVideo") or {}
|
||||
video_file = retake_vid_info.get("imageFile", "") if isinstance(retake_vid_info, dict) else ""
|
||||
|
||||
# Fallback to first segment only if it exists and we have no retake video info at all (old workflows)
|
||||
if not video_file and not retake_vid_info and len(segments) > 0:
|
||||
video_file = segments[0].get("videoFile", "")
|
||||
|
||||
if need_base_video:
|
||||
if not video_file:
|
||||
if retake_vid_info and not retake_vid_info.get("imageFile"):
|
||||
raise ValueError(
|
||||
"Retake Mode is active, but the base video file upload is still in progress (or failed). "
|
||||
"Please wait for the 'Uploading base video...' overlay on the timeline to disappear before queuing the prompt."
|
||||
)
|
||||
else:
|
||||
raise ValueError(
|
||||
"Retake Mode is active, but no base video has been selected on the timeline. "
|
||||
"Please drag and drop or upload a base video on the timeline first."
|
||||
)
|
||||
|
||||
if video_file and need_base_video:
|
||||
try:
|
||||
print(f"[LTXDirectorGuide] Loading and encoding base video file: {video_file} starting at frame {start_frame} for length {ltxv_length} at resolution {target_width}x{target_height}")
|
||||
video_frames = _load_motion_video_frames(
|
||||
video_file, trim_start_frames=start_frame, length_frames=ltxv_length, director_fps=director_fps, resample_mode="nearest"
|
||||
)
|
||||
|
||||
# Retake base video must match the exact target latent shape.
|
||||
# "maintain aspect ratio" doesn't pad, which causes shape mismatch in VAE encode.
|
||||
# So we fallback "maintain aspect ratio" to "pad" for retake base video.
|
||||
retake_resize_method = active_resize_method
|
||||
if retake_resize_method == "maintain aspect ratio":
|
||||
retake_resize_method = "pad"
|
||||
|
||||
pixels = _resize_image(video_frames, target_width, target_height, retake_resize_method, divisible_by=1)
|
||||
|
||||
num_clip_frames = pixels.shape[0]
|
||||
num_frames_to_keep = ((num_clip_frames - 1) // time_scale_factor) * time_scale_factor + 1
|
||||
encode_src = pixels[:num_frames_to_keep, :, :, :3]
|
||||
|
||||
if use_tiled_encode:
|
||||
base_latent = vae.encode_tiled(encode_src, tile_x=tile_size, tile_y=tile_size, overlap=tile_overlap)
|
||||
else:
|
||||
base_latent = vae.encode(encode_src)
|
||||
|
||||
base_latent = base_latent.to(device=latent_image.device, dtype=latent_image.dtype)
|
||||
|
||||
# Copy to latent_image
|
||||
paste_len = min(base_latent.shape[2], latent_length)
|
||||
if is_empty_latent:
|
||||
# Stage 1: Overwrite entire latent with base video VAE encode
|
||||
latent_image[:, :, :paste_len] = base_latent[:, :, :paste_len]
|
||||
else:
|
||||
# Stage 2: Overwrite only preserved regions (before l_start and after l_end)
|
||||
# leaving the generated retake region from Stage 1 untouched
|
||||
print(f"[LTXDirectorGuide] Stage 2: Copying high-resolution base latent for preserved regions (0-{l_start} and {l_end}-{paste_len})")
|
||||
if l_start > 0:
|
||||
latent_image[:, :, :l_start] = base_latent[:, :, :l_start]
|
||||
if l_end < paste_len:
|
||||
latent_image[:, :, l_end:paste_len] = base_latent[:, :, l_end:paste_len]
|
||||
except Exception as e:
|
||||
print(f"[LTXDirectorGuide] Failed to load/encode base video: {e}. Falling back to input latent.")
|
||||
|
||||
# 2. Build the temporal noise mask (0.0 = frozen, 1.0 = regenerate)
|
||||
noise_mask = torch.zeros_like(noise_mask) # Initialize fully frozen
|
||||
|
||||
l_start = min(l_start, latent_length)
|
||||
l_end = min(l_end, latent_length)
|
||||
|
||||
if l_end > l_start:
|
||||
noise_mask[:, :, l_start:l_end] = retake_strength
|
||||
|
||||
print(f"[LTXDirectorGuide] noise_mask slice: {noise_mask[0, 0, :, 0, 0].tolist()}")
|
||||
|
||||
# In retake mode, skip normal mode processing entirely and return immediately!
|
||||
exact_crop_frames = max(0, int(latent_image.shape[2]) - initial_latent_length)
|
||||
positive = node_helpers.conditioning_set_values(positive, {"nghtdrp_guide_crop_latent_frames": exact_crop_frames})
|
||||
negative = node_helpers.conditioning_set_values(negative, {"nghtdrp_guide_crop_latent_frames": exact_crop_frames})
|
||||
return (positive, negative, {"samples": latent_image, "noise_mask": noise_mask}, model, float(latent_downscale_factor))
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# Standard Timeline Keyframe Guidance:
|
||||
# Appends image segments as keyframes (and motion segments if present)
|
||||
# to the latent stream using standard LTX-Video cross-attention conditioning.
|
||||
# Registers guide attention entries if IC-LoRA is active.
|
||||
# -----------------------------------------------------------------------
|
||||
if len(images) > 0 or len(segments) > 0:
|
||||
print(f"[LTXDirectorGuide] Using Appended Keyframe Guidance. is_lora_active: {is_lora_active}")
|
||||
|
||||
# A. Process Image Guides
|
||||
for idx, img_tensor in enumerate(images):
|
||||
f_idx = insert_frames[idx] if idx < len(insert_frames) else 0
|
||||
strength = float(strengths[idx] if idx < len(strengths) else 1.0)
|
||||
if strength <= 0.0:
|
||||
continue
|
||||
|
||||
B_img, H_img, W_img, C_img = img_tensor.shape
|
||||
target_pix_w = int(latent_width * 32)
|
||||
target_pix_h = int(latent_height * 32)
|
||||
if target_pix_w != W_img or target_pix_h != H_img:
|
||||
img_nchw = img_tensor.permute(0, 3, 1, 2)
|
||||
img_resized = comfy.utils.common_upscale(img_nchw, target_pix_w, target_pix_h, upscale_method, "disabled")
|
||||
img_tensor = img_resized.permute(0, 2, 3, 1)
|
||||
|
||||
image_pixels, guide_latent = nodes_lt.LTXVAddGuide.encode(vae, latent_width, latent_height, img_tensor, scale_factors)
|
||||
frame_idx, latent_idx = nodes_lt.LTXVAddGuide.get_latent_index(positive, latent_length, len(image_pixels), int(f_idx), scale_factors)
|
||||
|
||||
if latent_idx >= latent_length:
|
||||
continue
|
||||
|
||||
max_frames = latent_length - latent_idx
|
||||
if guide_latent.shape[2] > max_frames:
|
||||
guide_latent = guide_latent[:, :, :max_frames]
|
||||
|
||||
tokens_added = guide_latent.shape[2] * guide_latent.shape[3] * guide_latent.shape[4]
|
||||
guide_orig_shape = list(guide_latent.shape[2:])
|
||||
|
||||
positive, negative, latent_image, noise_mask = nodes_lt.LTXVAddGuide.append_keyframe(
|
||||
positive, negative, frame_idx, latent_image, noise_mask, guide_latent, strength, scale_factors
|
||||
)
|
||||
if is_lora_active:
|
||||
positive = _append_guide_attention_entry(positive, tokens_added, guide_orig_shape, attention_strength=image_attention_strength)
|
||||
negative = _append_guide_attention_entry(negative, tokens_added, guide_orig_shape, attention_strength=image_attention_strength)
|
||||
|
||||
# B. Process Motion Video Segments
|
||||
for seg in segments:
|
||||
try:
|
||||
video_file = seg.get("videoFile")
|
||||
if not video_file:
|
||||
continue
|
||||
|
||||
start_frame = int(seg.get("start", 0))
|
||||
length_frames = int(seg.get("length", 1))
|
||||
trim_start = int(seg.get("trimStart", 0))
|
||||
video_strength = float(seg.get("videoStrength", 1.0))
|
||||
video_attention_strength = float(seg.get("videoAttentionStrength", 0.65))
|
||||
|
||||
if length_frames <= 0 or video_strength <= 0.0:
|
||||
continue
|
||||
|
||||
start_frame_aligned = start_frame
|
||||
video_frames = _load_motion_video_frames(video_file, trim_start, length_frames, director_fps, seg.get("resampleMode", "nearest"))
|
||||
|
||||
num_frames_to_keep = ((video_frames.shape[0] - 1) // time_scale_factor) * time_scale_factor + 1
|
||||
video_frames = video_frames[:num_frames_to_keep]
|
||||
causal_fix = int(start_frame_aligned) == 0 or num_frames_to_keep == 1
|
||||
encode_frames = video_frames if causal_fix else torch.cat([video_frames[:1], video_frames], dim=0)
|
||||
|
||||
_, guide_latent = _encode_video_iclora_guide(vae, latent_width, latent_height, encode_frames, scale_factors, latent_downscale_factor, crop, use_tiled_encode, tile_size, tile_overlap, resize_method=active_resize_method)
|
||||
|
||||
if not causal_fix:
|
||||
guide_latent = guide_latent[:, :, 1:, :, :]
|
||||
|
||||
frame_idx = start_frame_aligned
|
||||
latent_idx = (frame_idx + time_scale_factor - 1) // time_scale_factor if frame_idx > 0 else 0
|
||||
|
||||
if latent_idx >= latent_length:
|
||||
continue
|
||||
|
||||
if start_frame > 0 and guide_latent.shape[2] > 1:
|
||||
guide_latent = guide_latent[:, :, 1:, :, :]
|
||||
frame_idx += time_scale_factor
|
||||
latent_idx += 1
|
||||
if latent_idx >= latent_length:
|
||||
continue
|
||||
|
||||
max_frames = latent_length - latent_idx
|
||||
if guide_latent.shape[2] > max_frames:
|
||||
guide_latent = guide_latent[:, :, :max_frames]
|
||||
|
||||
guide_orig_shape = list(guide_latent.shape[2:])
|
||||
|
||||
B_g, C_g, F_g, H_g, W_g = guide_latent.shape
|
||||
guide_mask = torch.ones((B_g, 1, F_g, H_g, W_g), device=guide_latent.device, dtype=guide_latent.dtype)
|
||||
|
||||
if start_frame > 0:
|
||||
ramp_steps = [0.25, 0.65]
|
||||
for i, s in enumerate(ramp_steps):
|
||||
if i < F_g:
|
||||
guide_mask_val = 1.0 + video_strength * (1.0 - s)
|
||||
guide_mask[:, :, i, :, :] = guide_mask_val
|
||||
|
||||
ldf = int(max(1, round(float(latent_downscale_factor))))
|
||||
if ldf > 1:
|
||||
dilated = _dilate_latent({"samples": guide_latent, "noise_mask": guide_mask}, horizontal_scale=ldf, vertical_scale=ldf)
|
||||
guide_mask = dilated["noise_mask"]
|
||||
guide_latent = dilated["samples"]
|
||||
|
||||
tokens_added = guide_latent.shape[2] * guide_latent.shape[3] * guide_latent.shape[4]
|
||||
positive, negative, latent_image, noise_mask = nodes_lt.LTXVAddGuide.append_keyframe(
|
||||
positive, negative, frame_idx, latent_image, noise_mask, guide_latent, video_strength, scale_factors, guide_mask=guide_mask, latent_downscale_factor=float(latent_downscale_factor), causal_fix=causal_fix
|
||||
)
|
||||
if is_lora_active:
|
||||
positive = _append_guide_attention_entry(positive, tokens_added, guide_orig_shape, attention_strength=video_attention_strength)
|
||||
negative = _append_guide_attention_entry(negative, tokens_added, guide_orig_shape, attention_strength=video_attention_strength)
|
||||
except Exception as e:
|
||||
raise RuntimeError(f"LTX Director Guide motion segment failed for {seg}: {e}") from e
|
||||
|
||||
else:
|
||||
print("[LTXDirectorGuide] No timeline guides present. Passing through.")
|
||||
|
||||
exact_crop_frames = max(0, int(latent_image.shape[2]) - initial_latent_length)
|
||||
positive = node_helpers.conditioning_set_values(positive, {"nghtdrp_guide_crop_latent_frames": exact_crop_frames})
|
||||
negative = node_helpers.conditioning_set_values(negative, {"nghtdrp_guide_crop_latent_frames": exact_crop_frames})
|
||||
|
||||
return (positive, negative, {"samples": latent_image, "noise_mask": noise_mask}, model, float(latent_downscale_factor))
|
||||
|
||||
@classmethod
|
||||
def _inject_msr(cls, positive, negative, vae, latent_image, noise_mask, msr, msr_strength=0.0):
|
||||
"""Inject the Licon MSR references at this node's current resolution."""
|
||||
def _inject_msr(cls, positive, negative, vae, latent_image, noise_mask, msr, model, latent_downscale_factor, msr_strength=0.0):
|
||||
"""Inject the Licon MSR references at this node's current resolution.
|
||||
|
||||
Returns this node's 5-tuple (positive, negative, latent, model, latent_downscale_factor).
|
||||
"""
|
||||
from nodes import NODE_CLASS_MAPPINGS
|
||||
from .ltx_director import _execute_comfy_node, _unpack
|
||||
|
||||
IC = NODE_CLASS_MAPPINGS.get("LTXAddVideoICLoRAGuide")
|
||||
AG = NODE_CLASS_MAPPINGS.get("LTXVAddGuide")
|
||||
@@ -118,6 +649,8 @@ class LTXDirectorGuide(LTXVAddGuide):
|
||||
strength = clamp01(msr_strength) if float(msr_strength) > 0.0 else clamp01(msr["strength"])
|
||||
downscale = float(msr["downscale"])
|
||||
|
||||
initial_latent_length = int(latent_image.shape[2])
|
||||
|
||||
# Pad the generated region so (pad + keyframes) == prefix_latents, so the downstream crop
|
||||
# (which trims the slideshow's temporal footprint) lands on the true clean length.
|
||||
pad_latents = max(0, prefix_latents - len(keyframes))
|
||||
@@ -140,15 +673,95 @@ class LTXDirectorGuide(LTXVAddGuide):
|
||||
IC, positive=positive, negative=negative, vae=vae, latent=latent, image=slideshow,
|
||||
frame_idx=0, strength=strength, latent_downscale_factor=downscale,
|
||||
crop="center", use_tiled_encode=False, tile_size=256, tile_overlap=64,
|
||||
))
|
||||
|
||||
))[:3]
|
||||
positive, negative = cp, cn
|
||||
|
||||
# LATENT PATH: one frozen keyframe per reference at frame 0 -> keep latent, drop cond.
|
||||
for kf in keyframes:
|
||||
_p, _n, latent = _unpack(_execute_comfy_node(
|
||||
unpacked = _unpack(_execute_comfy_node(
|
||||
AG, positive=positive, negative=negative, vae=vae, latent=latent,
|
||||
image=kf, frame_idx=0, strength=strength,
|
||||
))
|
||||
latent = unpacked[2]
|
||||
|
||||
return io.NodeOutput(positive, negative, latent)
|
||||
# NOTE: MSR is a PREFIX. We deliberately do NOT set nghtdrp_guide_crop_latent_frames
|
||||
# (the new LTXDirectorCropGuides trims from the END, which would cut clean frames).
|
||||
# The IC/AG nodes write position-aware keyframe metadata into the conditioning, so the
|
||||
# MSR branch of the workflow should crop with the STOCK position-aware LTXVCropGuides
|
||||
# node (exactly as the old MSR pipeline did).
|
||||
return (positive, negative, latent, model, float(latent_downscale_factor))
|
||||
|
||||
|
||||
def _conditioning_get_any_value(conditioning, key, default=None):
|
||||
for item in conditioning:
|
||||
meta = item[1]
|
||||
if key in meta and meta.get(key) is not None:
|
||||
return meta.get(key)
|
||||
return default
|
||||
|
||||
def _get_exact_crop_count_from_conditioning(conditioning):
|
||||
value = _conditioning_get_any_value(conditioning, "nghtdrp_guide_crop_latent_frames", None)
|
||||
if value is not None:
|
||||
try:
|
||||
return max(0, int(value))
|
||||
except Exception:
|
||||
return 0
|
||||
keyframe_idxs = _conditioning_get_any_value(conditioning, "keyframe_idxs", None)
|
||||
if keyframe_idxs is None:
|
||||
return 0
|
||||
try:
|
||||
return int(torch.unique(keyframe_idxs[:, 0, :, 0]).shape[0])
|
||||
except Exception:
|
||||
return 0
|
||||
|
||||
def _get_noise_mask_for_crop(latent):
|
||||
latent_image = latent["samples"]
|
||||
noise_mask = latent.get("noise_mask", None)
|
||||
if noise_mask is None:
|
||||
batch, _, frames, _, _ = latent_image.shape
|
||||
return torch.ones((batch, 1, frames, 1, 1), dtype=torch.float32, device=latent_image.device)
|
||||
return noise_mask.clone()
|
||||
|
||||
|
||||
class LTXDirectorCropGuides:
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"positive": ("CONDITIONING",),
|
||||
"negative": ("CONDITIONING",),
|
||||
"latent": ("LATENT",),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT")
|
||||
RETURN_NAMES = ("positive", "negative", "latent")
|
||||
FUNCTION = "execute"
|
||||
CATEGORY = "WhatDreamsCost CS"
|
||||
|
||||
def execute(self, positive, negative, latent):
|
||||
latent_image = latent["samples"].clone()
|
||||
noise_mask = _get_noise_mask_for_crop(latent)
|
||||
|
||||
crop_frames = _get_exact_crop_count_from_conditioning(positive)
|
||||
if crop_frames <= 0:
|
||||
return (positive, negative, {"samples": latent_image, "noise_mask": noise_mask})
|
||||
|
||||
crop_frames = min(crop_frames, max(0, latent_image.shape[2] - 1))
|
||||
if crop_frames > 0:
|
||||
latent_image = latent_image[:, :, :-crop_frames]
|
||||
noise_mask = noise_mask[:, :, :-crop_frames]
|
||||
|
||||
clear_values = {
|
||||
"keyframe_idxs": None,
|
||||
"guide_attention_entries": None,
|
||||
"nghtdrp_guide_crop_latent_frames": None,
|
||||
}
|
||||
positive = node_helpers.conditioning_set_values(positive, clear_values)
|
||||
negative = node_helpers.conditioning_set_values(negative, clear_values)
|
||||
return (positive, negative, {"samples": latent_image, "noise_mask": noise_mask})
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"LTXDirectorGuideCS": LTXDirectorGuide,
|
||||
"LTXDirectorCropGuidesCS": LTXDirectorCropGuides,
|
||||
}
|
||||
@@ -1,4 +1,3 @@
|
||||
import json
|
||||
from comfy_extras.nodes_lt import get_noise_mask, LTXVAddGuide
|
||||
import torch
|
||||
import comfy.utils
|
||||
@@ -56,14 +55,12 @@ class LTXSequencer(LTXVAddGuide):
|
||||
node_id="LTXSequencer",
|
||||
display_name="LTX Sequencer",
|
||||
category="WhatDreamsCost",
|
||||
description="Add multiple guide images at specified frame indices or seconds. Auto-syncs to Prompt Relay invisibly via the positive wire.",
|
||||
description="Add multiple guide images at specified frame indices or seconds with strengths. Number of widgets is dynamically configured.",
|
||||
inputs=inputs,
|
||||
outputs=[
|
||||
io.Conditioning.Output(display_name="positive"),
|
||||
io.Conditioning.Output(display_name="negative"),
|
||||
io.Latent.Output(display_name="latent", tooltip="Video latent with added guides"),
|
||||
# ---> ADDED SYNC LOG OUTPUT <---
|
||||
io.String.Output(display_name="sync_log", tooltip="Outputs a readable text block showing the actual times used for each image"),
|
||||
],
|
||||
)
|
||||
|
||||
@@ -92,63 +89,29 @@ class LTXSequencer(LTXVAddGuide):
|
||||
insert_mode = kwargs.get("insert_mode", "frames")
|
||||
frame_rate = kwargs.get("frame_rate", 24)
|
||||
|
||||
# ---> EXTRACT GHOST SYNC DATA <---
|
||||
timeline_data = None
|
||||
if positive is not None and len(positive) > 0:
|
||||
if "prompt_relay_timeline" in positive[0][1]:
|
||||
try:
|
||||
timeline_data = json.loads(positive[0][1]["prompt_relay_timeline"])
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
# Prepare the log text
|
||||
sync_log_lines = []
|
||||
if timeline_data:
|
||||
sync_log_lines.append("=== TIMELINE SYNC ENABLED ===")
|
||||
sync_log_lines.append(f"Invisibly Synced via Positive Wire! Using {insert_mode.upper()}:")
|
||||
else:
|
||||
sync_log_lines.append("=== MANUAL MODE ===")
|
||||
sync_log_lines.append(f"No timeline detected, or Sync turned OFF. Using manual UI inputs ({insert_mode}):")
|
||||
|
||||
# Process inputs up to num_images, extracting dynamic frame/strength values from kwargs
|
||||
for i in range(1, num_images + 1):
|
||||
# Skip if this image index exceeds the batch
|
||||
if i > batch_size:
|
||||
sync_log_lines.append(f"Image #{i}: Skipped (No image loaded in batch)")
|
||||
continue
|
||||
|
||||
img = multi_input[i-1:i] # Extract the single image frame from the batch
|
||||
if img is None:
|
||||
continue
|
||||
|
||||
# Calculate the final frame index based on the chosen mode
|
||||
f_idx = None
|
||||
strength = kwargs.get(f"strength_{i}", 1.0)
|
||||
|
||||
# 1. AUTO-SYNC
|
||||
if timeline_data and "starts_frames" in timeline_data and (i - 1) < len(timeline_data["starts_frames"]):
|
||||
display_frame = timeline_data["starts_frames"][i - 1]
|
||||
display_sec = timeline_data["starts_seconds"][i - 1]
|
||||
|
||||
if insert_mode == "frames":
|
||||
f_idx = display_frame
|
||||
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Synced to Segment #{i} start @ {display_frame} frames")
|
||||
elif insert_mode == "seconds":
|
||||
f_idx = int(display_sec * frame_rate)
|
||||
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Synced to Segment #{i} start @ {display_sec:.2f} seconds (Frame {f_idx})")
|
||||
|
||||
# 2. MANUAL FALLBACK
|
||||
else:
|
||||
if insert_mode == "frames":
|
||||
f_idx = kwargs.get(f"insert_frame_{i}")
|
||||
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Manual input @ {f_idx} frames")
|
||||
elif insert_mode == "seconds":
|
||||
sec = kwargs.get(f"insert_second_{i}")
|
||||
if sec is not None:
|
||||
f_idx = int(sec * frame_rate)
|
||||
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Manual input @ {sec:.2f} seconds (Frame {f_idx})")
|
||||
if insert_mode == "frames":
|
||||
f_idx = kwargs.get(f"insert_frame_{i}")
|
||||
elif insert_mode == "seconds":
|
||||
sec = kwargs.get(f"insert_second_{i}")
|
||||
if sec is not None:
|
||||
f_idx = int(sec * frame_rate)
|
||||
|
||||
if f_idx is None:
|
||||
continue
|
||||
|
||||
strength = kwargs.get(f"strength_{i}", 1.0)
|
||||
|
||||
# Execution logic mirrored from LTXVAddGuideMulti
|
||||
image_1, t = cls.encode(vae, latent_width, latent_height, img, scale_factors)
|
||||
@@ -167,8 +130,4 @@ class LTXSequencer(LTXVAddGuide):
|
||||
scale_factors,
|
||||
)
|
||||
|
||||
sync_log_str = "\n".join(sync_log_lines)
|
||||
print(f"\n[LTX Sequencer]\n{sync_log_str}\n")
|
||||
|
||||
# Returned the newly formatted sync log string at the end of NodeOutput
|
||||
return io.NodeOutput(positive, negative, {"samples": latent_image, "noise_mask": noise_mask}, sync_log_str)
|
||||
return io.NodeOutput(positive, negative, {"samples": latent_image, "noise_mask": noise_mask})
|
||||
@@ -1,182 +0,0 @@
|
||||
import torch
|
||||
import torch.nn.functional as F
|
||||
import numpy as np
|
||||
from PIL import Image, ImageOps
|
||||
import os
|
||||
import folder_paths
|
||||
import io
|
||||
import comfy.utils
|
||||
|
||||
class MultiImageLoader:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"image_paths": ("STRING", {"default": "", "multiline": True}),
|
||||
"width": ("INT", {"default": 0, "min": 0, "max": 8192, "step": 1}),
|
||||
"height": ("INT", {"default": 0, "min": 0, "max": 8192, "step": 1}),
|
||||
"interpolation": (["lanczos", "nearest", "bilinear", "bicubic", "area", "nearest-exact"],),
|
||||
"resize_method": (["keep proportion", "stretch", "pad", "crop"],),
|
||||
"multiple_of": ("INT", {"default": 32, "min": 0, "max": 512, "step": 1}),
|
||||
"img_compression": ("INT", {"default": 18, "min": 0, "max": 100, "step": 1}),
|
||||
},
|
||||
}
|
||||
|
||||
# Added "IMAGE" at the beginning for multi_output + 50 individual outputs = 51 outputs
|
||||
RETURN_TYPES = ("IMAGE",) * 51
|
||||
RETURN_NAMES = ("multi_output",) + tuple(f"image_{i+1}" for i in range(50))
|
||||
FUNCTION = "load_images"
|
||||
CATEGORY = "WhatDreamsCost"
|
||||
|
||||
def resize_image(self, image, width, height, resize_method="keep proportion", interpolation="nearest", multiple_of=0):
|
||||
MAX_RESOLUTION = 8192
|
||||
_, oh, ow, _ = image.shape
|
||||
x = y = x2 = y2 = 0
|
||||
pad_left = pad_right = pad_top = pad_bottom = 0
|
||||
|
||||
if multiple_of > 1:
|
||||
width = width - (width % multiple_of)
|
||||
height = height - (height % multiple_of)
|
||||
|
||||
if resize_method == 'keep proportion' or resize_method == 'pad':
|
||||
if width == 0 and oh < height:
|
||||
width = MAX_RESOLUTION
|
||||
elif width == 0 and oh >= height:
|
||||
width = ow
|
||||
|
||||
if height == 0 and ow < width:
|
||||
height = MAX_RESOLUTION
|
||||
elif height == 0 and ow >= width:
|
||||
height = oh
|
||||
|
||||
ratio = min(width / ow, height / oh)
|
||||
new_width = round(ow * ratio)
|
||||
new_height = round(oh * ratio)
|
||||
|
||||
if resize_method == 'pad':
|
||||
pad_left = (width - new_width) // 2
|
||||
pad_right = width - new_width - pad_left
|
||||
pad_top = (height - new_height) // 2
|
||||
pad_bottom = height - new_height - pad_top
|
||||
|
||||
width = new_width
|
||||
height = new_height
|
||||
|
||||
elif resize_method == 'crop':
|
||||
width = width if width > 0 else ow
|
||||
height = height if height > 0 else oh
|
||||
|
||||
ratio = max(width / ow, height / oh)
|
||||
new_width = round(ow * ratio)
|
||||
new_height = round(oh * ratio)
|
||||
x = (new_width - width) // 2
|
||||
y = (new_height - height) // 2
|
||||
x2 = x + width
|
||||
y2 = y + height
|
||||
if x2 > new_width:
|
||||
x -= (x2 - new_width)
|
||||
if x < 0:
|
||||
x = 0
|
||||
if y2 > new_height:
|
||||
y -= (y2 - new_height)
|
||||
if y < 0:
|
||||
y = 0
|
||||
width = new_width
|
||||
height = new_height
|
||||
|
||||
else:
|
||||
width = width if width > 0 else ow
|
||||
height = height if height > 0 else oh
|
||||
|
||||
# Always apply resize logic
|
||||
outputs = image.permute(0, 3, 1, 2)
|
||||
|
||||
if interpolation == "lanczos":
|
||||
outputs = comfy.utils.lanczos(outputs, width, height)
|
||||
else:
|
||||
outputs = F.interpolate(outputs, size=(height, width), mode=interpolation)
|
||||
|
||||
if resize_method == 'pad':
|
||||
if pad_left > 0 or pad_right > 0 or pad_top > 0 or pad_bottom > 0:
|
||||
outputs = F.pad(outputs, (pad_left, pad_right, pad_top, pad_bottom), value=0)
|
||||
|
||||
outputs = outputs.permute(0, 2, 3, 1)
|
||||
|
||||
if resize_method == 'crop':
|
||||
if x > 0 or y > 0 or x2 > 0 or y2 > 0:
|
||||
outputs = outputs[:, y:y2, x:x2, :]
|
||||
|
||||
if multiple_of > 1 and (outputs.shape[2] % multiple_of != 0 or outputs.shape[1] % multiple_of != 0):
|
||||
width = outputs.shape[2]
|
||||
height = outputs.shape[1]
|
||||
x = (width % multiple_of) // 2
|
||||
y = (height % multiple_of) // 2
|
||||
x2 = width - ((width % multiple_of) - x)
|
||||
y2 = height - ((height % multiple_of) - y)
|
||||
outputs = outputs[:, y:y2, x:x2, :]
|
||||
|
||||
outputs = torch.clamp(outputs, 0, 1)
|
||||
|
||||
return outputs
|
||||
|
||||
def load_images(self, image_paths, width, height, interpolation, resize_method, multiple_of, img_compression):
|
||||
results = []
|
||||
valid_paths = [p.strip() for p in image_paths.split("\n") if p.strip()]
|
||||
|
||||
for path in valid_paths:
|
||||
try:
|
||||
# Resolve full path
|
||||
full_path = path
|
||||
if not os.path.exists(full_path):
|
||||
full_path = os.path.join(folder_paths.get_input_directory(), path)
|
||||
|
||||
if not os.path.exists(full_path):
|
||||
print(f"Warning: Image path not found: {path}")
|
||||
continue
|
||||
|
||||
# Load image
|
||||
image = Image.open(full_path)
|
||||
image = ImageOps.exif_transpose(image)
|
||||
image = image.convert("RGB")
|
||||
|
||||
# Convert to Torch Tensor to prepare for Advanced Resize Logic
|
||||
image_np = np.array(image).astype(np.float32) / 255.0
|
||||
image_tensor = torch.from_numpy(image_np)[None,]
|
||||
|
||||
# Apply Advanced Resize
|
||||
image_tensor = self.resize_image(image_tensor, width, height, resize_method, interpolation, multiple_of)
|
||||
|
||||
# Compression (Applied after resize to accurately maintain the effect)
|
||||
if img_compression > 0:
|
||||
img_np = (image_tensor[0].numpy() * 255).clip(0, 255).astype(np.uint8)
|
||||
img_pil = Image.fromarray(img_np)
|
||||
img_byte_arr = io.BytesIO()
|
||||
img_pil.save(img_byte_arr, format="JPEG", quality=max(1, 100 - img_compression))
|
||||
img_pil = Image.open(img_byte_arr)
|
||||
image_tensor = torch.from_numpy(np.array(img_pil).astype(np.float32) / 255.0)[None,]
|
||||
|
||||
results.append(image_tensor)
|
||||
except Exception as e:
|
||||
print(f"Error loading {path}: {e}")
|
||||
|
||||
# Combine all successfully loaded images into a single batched tensor for multi_output
|
||||
if len(results) > 0:
|
||||
# Safety Check: Advanced resize methods might output differently sized tensors (e.g., 'keep proportion')
|
||||
first_shape = results[0].shape
|
||||
all_same_shape = all(r.shape == first_shape for r in results)
|
||||
|
||||
if all_same_shape:
|
||||
multi_output = torch.cat(results, dim=0)
|
||||
else:
|
||||
print("MultiImageLoader Warning: Images have different dimensions due to resize settings. Cannot batch into multi_output. Outputting zero tensor for the batch, but individual output nodes will still work fine.")
|
||||
multi_output = torch.zeros((1, 64, 64, 3))
|
||||
else:
|
||||
# Fallback empty tensor if no valid paths
|
||||
multi_output = torch.zeros((1, 64, 64, 3))
|
||||
results = [multi_output]
|
||||
|
||||
# Pad individual outputs exactly to length 50 as defined in RETURN_TYPES
|
||||
padded_results = results + [torch.zeros((1, 64, 64, 3))] * (50 - len(results))
|
||||
|
||||
# Return the multi batch output first, followed by the individual padded items
|
||||
return (multi_output, *padded_results[:50])
|
||||
100
patches.py
100
patches.py
@@ -1,7 +1,10 @@
|
||||
import logging
|
||||
import types
|
||||
import torch
|
||||
|
||||
import comfy.ldm.modules.attention
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _masked_attention(q, k, v, heads, mask, transformer_options={}, **kwargs):
|
||||
# Bypass wrap_attn (sage/etc may ignore masks) by calling attention_pytorch directly.
|
||||
@@ -18,7 +21,7 @@ def _wan_t2v_forward(self, mask_fn, x, context, transformer_options={}, **kwargs
|
||||
k = self.norm_k(self.k(context))
|
||||
v = self.v(context)
|
||||
|
||||
mask = mask_fn(q, k, transformer_options)
|
||||
mask = mask_fn(q.shape[1], k.shape[1], q.dtype, q.device, transformer_options)
|
||||
if mask is not None:
|
||||
x = _masked_attention(q, k, v, heads=self.num_heads, mask=mask,
|
||||
transformer_options=transformer_options)
|
||||
@@ -44,7 +47,7 @@ def _wan_i2v_forward(self, mask_fn, x, context, context_img_len, transformer_opt
|
||||
k = self.norm_k(self.k(context_text))
|
||||
v = self.v(context_text)
|
||||
|
||||
mask = mask_fn(q, k, transformer_options)
|
||||
mask = mask_fn(q.shape[1], k.shape[1], q.dtype, q.device, transformer_options)
|
||||
if mask is not None:
|
||||
x = _masked_attention(q, k, v, heads=self.num_heads, mask=mask,
|
||||
transformer_options=transformer_options)
|
||||
@@ -56,47 +59,65 @@ def _wan_i2v_forward(self, mask_fn, x, context, context_img_len, transformer_opt
|
||||
return self.o(x + img_x)
|
||||
|
||||
|
||||
def _ltx_forward(self, mask_fn, x, context=None, mask=None, pe=None, k_pe=None, transformer_options={}):
|
||||
from comfy.ldm.lightricks.model import apply_rotary_emb
|
||||
def _make_masked_override(prev_override):
|
||||
"""transformer_options override that routes mask-bearing attention calls through
|
||||
attention_pytorch (sage/etc. drop arbitrary masks). Chains to a prior override
|
||||
when no mask is present so we don't clobber other backends."""
|
||||
def override(func, *args, **kwargs):
|
||||
if kwargs.get("mask") is not None:
|
||||
return comfy.ldm.modules.attention.attention_pytorch(*args, **kwargs)
|
||||
if prev_override is not None:
|
||||
return prev_override(func, *args, **kwargs)
|
||||
return func(*args, **kwargs)
|
||||
return override
|
||||
|
||||
is_self_attn = context is None
|
||||
context = x if is_self_attn else context
|
||||
|
||||
q = self.q_norm(self.to_q(x))
|
||||
k = self.k_norm(self.to_k(context))
|
||||
v = self.to_v(context)
|
||||
def debug_log(msg):
|
||||
try:
|
||||
import os
|
||||
log_path = os.path.join(os.path.dirname(__file__), "debug_prompt_relay.log")
|
||||
with open(log_path, "a", encoding="utf-8") as f:
|
||||
f.write(msg + "\n")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
if pe is not None:
|
||||
q = apply_rotary_emb(q, pe)
|
||||
k = apply_rotary_emb(k, pe if k_pe is None else k_pe)
|
||||
|
||||
if not is_self_attn:
|
||||
temporal_mask = mask_fn(q, k, transformer_options)
|
||||
if temporal_mask is not None:
|
||||
mask = temporal_mask if mask is None else mask + temporal_mask
|
||||
def _make_ltx_mask_wrapper(underlying, mask_fn, attr):
|
||||
"""Wrap an existing LTX cross-attn forward (the default `CrossAttention.forward`
|
||||
or another node's patch — e.g. KJNodes NAG), injecting PromptRelay's additive
|
||||
mask via the `mask` kwarg the upstream signature already accepts.
|
||||
|
||||
if mask is None:
|
||||
out = comfy.ldm.modules.attention.optimized_attention(
|
||||
q, k, v, self.heads, attn_precision=self.attn_precision,
|
||||
`underlying` must already be bound to its module — callable as
|
||||
`underlying(x, context=..., mask=..., ...)`.
|
||||
"""
|
||||
def wrapped(_self, x, context=None, mask=None, pe=None, k_pe=None, transformer_options={}):
|
||||
debug_log(f"wrapped called: x.shape={list(x.shape)} context.shape={list(context.shape) if context is not None else None} mask_is_none={mask is None}")
|
||||
if context is not None:
|
||||
opts = {**transformer_options, "promptrelay_attn_type": attr}
|
||||
pr_mask = mask_fn(x.shape[1], context.shape[1], x.dtype, x.device, opts)
|
||||
debug_log(f"mask_fn returned pr_mask_is_none={pr_mask is None}")
|
||||
if pr_mask is not None:
|
||||
debug_log(f"pr_mask info: shape={list(pr_mask.shape)} min={pr_mask.min().item()} max={pr_mask.max().item()} sum={pr_mask.sum().item()}")
|
||||
mask = pr_mask if mask is None else mask + pr_mask
|
||||
|
||||
if mask is not None:
|
||||
prev = transformer_options.get("optimized_attention_override")
|
||||
transformer_options = {
|
||||
**transformer_options,
|
||||
"optimized_attention_override": _make_masked_override(prev),
|
||||
}
|
||||
|
||||
return underlying(
|
||||
x, context=context, mask=mask, pe=pe, k_pe=k_pe,
|
||||
transformer_options=transformer_options,
|
||||
)
|
||||
else:
|
||||
out = _masked_attention(q, k, v, self.heads, mask=mask,
|
||||
attn_precision=self.attn_precision,
|
||||
transformer_options=transformer_options)
|
||||
|
||||
if self.to_gate_logits is not None:
|
||||
gate_logits = self.to_gate_logits(x)
|
||||
b, t, _ = out.shape
|
||||
out = out.view(b, t, self.heads, self.dim_head)
|
||||
out = out * (2.0 * torch.sigmoid(gate_logits)).unsqueeze(-1)
|
||||
out = out.view(b, t, self.heads * self.dim_head)
|
||||
|
||||
return self.to_out(out)
|
||||
wrapped._promptrelay_wrapper = True
|
||||
return wrapped
|
||||
|
||||
|
||||
class _CrossAttnPatch:
|
||||
"""Descriptor that binds (impl, mask_fn) as a method onto a cross-attn module."""
|
||||
"""Descriptor that binds (impl, mask_fn) as a method onto a Wan cross-attn module."""
|
||||
|
||||
def __init__(self, impl, mask_fn):
|
||||
self.impl = impl
|
||||
@@ -135,8 +156,8 @@ def _check_unpatched(model_clone, key):
|
||||
if key in getattr(model_clone, "object_patches", {}):
|
||||
raise RuntimeError(
|
||||
f"PromptRelay: cross-attention forward at '{key}' is already patched by "
|
||||
"another node (e.g. KJNodes NAG). Stacking is not supported — remove the "
|
||||
"conflicting node."
|
||||
"another node. Stacking is not supported for this architecture — remove "
|
||||
"the conflicting node."
|
||||
)
|
||||
|
||||
|
||||
@@ -154,14 +175,19 @@ def apply_patches(model_clone, arch, mask_fn):
|
||||
return
|
||||
|
||||
if arch == "ltx":
|
||||
to = model_clone.model_options["transformer_options"]
|
||||
to["promptrelay_mask_fn"] = mask_fn
|
||||
|
||||
for idx, block in enumerate(diffusion_model.transformer_blocks):
|
||||
for attr in ("attn2", "audio_attn2"):
|
||||
module = getattr(block, attr, None)
|
||||
if module is None:
|
||||
continue
|
||||
key = f"diffusion_model.transformer_blocks.{idx}.{attr}.forward"
|
||||
_check_unpatched(model_clone, key)
|
||||
model_clone.add_object_patch(key, _CrossAttnPatch(_ltx_forward, mask_fn).__get__(module, module.__class__))
|
||||
# get_model_object returns the prior patch if present, else the default bound forward.
|
||||
underlying = model_clone.get_model_object(key)
|
||||
wrapper = _make_ltx_mask_wrapper(underlying, mask_fn, attr)
|
||||
model_clone.add_object_patch(key, types.MethodType(wrapper, module))
|
||||
return
|
||||
|
||||
raise ValueError(f"Unknown model arch: {arch}")
|
||||
|
||||
@@ -20,7 +20,7 @@ def build_temporal_cost(q_token_idx, Lq, Lk, device, dtype, tokens_per_frame):
|
||||
return offset
|
||||
|
||||
|
||||
def build_temporal_cost_scaled(q_token_idx, Lq, Lk, device, dtype, latent_frames):
|
||||
def build_temporal_cost_scaled(q_token_idx, Lq, Lk, device, dtype, latent_frames, is_audio=False):
|
||||
"""Penalty matrix for queries that don't map to integer frames (e.g. LTXAV audio tokens)."""
|
||||
offset = torch.zeros(Lq, Lk, device=device, dtype=dtype)
|
||||
query_frames = torch.arange(Lq, device=device, dtype=torch.float32) * latent_frames / Lq
|
||||
@@ -28,54 +28,90 @@ def build_temporal_cost_scaled(q_token_idx, Lq, Lk, device, dtype, latent_frames
|
||||
for seg in q_token_idx:
|
||||
local = seg["local_token_idx"].to(device=device)
|
||||
d = (query_frames[:, None] - seg["midpoint"]).abs()
|
||||
sigma_a = seg.get("sigma_audio", seg["sigma"])
|
||||
window_a = seg.get("window_audio", seg["window"])
|
||||
strength_a = seg.get("strength_audio", 1.0)
|
||||
cost = strength_a * (torch.relu(d - window_a) ** 2) / (2 * sigma_a ** 2)
|
||||
if is_audio:
|
||||
sigma_val = seg.get("sigma_audio", seg["sigma"])
|
||||
window_val = seg.get("window_audio", seg["window"])
|
||||
strength_val = seg.get("strength_audio", 1.0)
|
||||
else:
|
||||
sigma_val = seg["sigma"]
|
||||
window_val = seg["window"]
|
||||
strength_val = seg.get("strength", 1.0)
|
||||
cost = strength_val * (torch.relu(d - window_val) ** 2) / (2 * sigma_val ** 2)
|
||||
offset[:, local] = cost.to(offset.dtype)
|
||||
|
||||
return offset
|
||||
|
||||
|
||||
def debug_log(msg):
|
||||
try:
|
||||
import os
|
||||
log_path = os.path.join(os.path.dirname(__file__), "debug_prompt_relay.log")
|
||||
with open(log_path, "a", encoding="utf-8") as f:
|
||||
f.write(msg + "\n")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def create_mask_fn(q_token_idx, fallback_tokens_per_frame, latent_frames):
|
||||
"""Closure: mask_fn(q, k, transformer_options) -> additive mask or None."""
|
||||
"""Closure: mask_fn(Lq, Lk, dtype, device, transformer_options) -> additive mask or None.
|
||||
|
||||
Takes shapes/dtype/device instead of tensors so callers can compute the mask
|
||||
without first materializing q/k projections — required so PromptRelay can
|
||||
wrap an existing cross-attn forward (e.g. KJNodes NAG) instead of replacing it.
|
||||
"""
|
||||
cache = {}
|
||||
max_token_idx = max(int(seg["local_token_idx"].max().item()) for seg in q_token_idx) + 1
|
||||
|
||||
def mask_fn(q, k, transformer_options):
|
||||
Lq, Lk = q.shape[1], k.shape[1]
|
||||
|
||||
def mask_fn(Lq, Lk, dtype, device, transformer_options):
|
||||
debug_log(f"mask_fn check: Lq={Lq} Lk={Lk} max_token_idx={max_token_idx} cond_or_uncond={transformer_options.get('cond_or_uncond', [])}")
|
||||
if Lq == Lk:
|
||||
debug_log("mask_fn: Lq == Lk, returning None")
|
||||
return None
|
||||
|
||||
# Only apply on conditional pass — not unconditional (negative prompt)
|
||||
cond_or_uncond = transformer_options.get("cond_or_uncond", [])
|
||||
if 1 in cond_or_uncond and 0 not in cond_or_uncond:
|
||||
debug_log("mask_fn: unconditional pass, returning None")
|
||||
return None
|
||||
|
||||
grid_sizes = transformer_options.get("grid_sizes", None)
|
||||
video_tpf = int(grid_sizes[1]) * int(grid_sizes[2]) if grid_sizes is not None else fallback_tokens_per_frame
|
||||
video_lq = latent_frames * video_tpf
|
||||
attn_type = transformer_options.get("promptrelay_attn_type", "attn2")
|
||||
is_audio = (attn_type == "audio_attn2")
|
||||
|
||||
# Skip cross-modal attention — text keys are padded to a fixed length ≥ max_token_idx and != video_lq
|
||||
if Lk == video_lq or Lk < max_token_idx:
|
||||
return None
|
||||
if is_audio:
|
||||
mode = "scaled"
|
||||
video_lq = -1
|
||||
else:
|
||||
if grid_sizes is not None:
|
||||
video_tpf = int(grid_sizes[1]) * int(grid_sizes[2])
|
||||
else:
|
||||
if Lq % latent_frames == 0:
|
||||
video_tpf = Lq // latent_frames
|
||||
else:
|
||||
video_tpf = fallback_tokens_per_frame
|
||||
video_lq = latent_frames * video_tpf
|
||||
|
||||
mode = "video" if Lq == video_lq else "scaled"
|
||||
# Skip cross-modal attention — text keys are padded to a fixed length ≥ max_token_idx and != video_lq
|
||||
if Lk == video_lq or Lk < max_token_idx:
|
||||
debug_log(f"mask_fn: Lk == video_lq ({Lk == video_lq}) or Lk < max_token_idx ({Lk < max_token_idx}), returning None")
|
||||
return None
|
||||
|
||||
key = (Lq, Lk, mode, q.device)
|
||||
mode = "video" if Lq == video_lq else "scaled"
|
||||
|
||||
key = (Lq, Lk, mode, device)
|
||||
if key not in cache:
|
||||
if mode == "video":
|
||||
cost = build_temporal_cost(q_token_idx, Lq, Lk, q.device, q.dtype, video_tpf)
|
||||
cost = build_temporal_cost(q_token_idx, Lq, Lk, device, dtype, video_tpf)
|
||||
else:
|
||||
cost = build_temporal_cost_scaled(q_token_idx, Lq, Lk, q.device, q.dtype, latent_frames)
|
||||
cost = build_temporal_cost_scaled(q_token_idx, Lq, Lk, device, dtype, latent_frames, is_audio=is_audio)
|
||||
log.info(
|
||||
"[PromptRelay] Built penalty matrix (%s): Lq=%d, Lk=%d, nonzero=%d/%d",
|
||||
mode, Lq, Lk, (cost > 0).sum().item(), cost.numel(),
|
||||
)
|
||||
debug_log(f"Built penalty matrix ({mode}): Lq={Lq}, Lk={Lk}, key={key}")
|
||||
cache[key] = -cost
|
||||
|
||||
return cache[key].to(q.dtype)
|
||||
return cache[key].to(dtype)
|
||||
|
||||
return mask_fn
|
||||
|
||||
@@ -160,7 +196,30 @@ def map_token_indices(raw_tokenizer, global_prompt, local_prompts):
|
||||
"""
|
||||
prefixed_locals = [" " + lp for lp in local_prompts]
|
||||
full_prompt = global_prompt + "".join(prefixed_locals)
|
||||
|
||||
# Detect if the tokenizer appends EOS dynamically
|
||||
has_eos = getattr(raw_tokenizer, "add_eos", False)
|
||||
if not has_eos:
|
||||
try:
|
||||
test_res = raw_tokenizer("test")
|
||||
if isinstance(test_res, dict) and "input_ids" in test_res:
|
||||
ids = test_res["input_ids"]
|
||||
elif hasattr(test_res, "input_ids"):
|
||||
ids = test_res.input_ids
|
||||
elif isinstance(test_res, list):
|
||||
ids = test_res
|
||||
else:
|
||||
ids = []
|
||||
|
||||
if ids:
|
||||
eos_id = getattr(raw_tokenizer, "eos_token_id", None)
|
||||
if eos_id is not None and ids[-1] == eos_id:
|
||||
has_eos = True
|
||||
elif ids[-1] == 1:
|
||||
has_eos = True
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
eos_adj = 1 if has_eos else 0
|
||||
|
||||
prev_len = len(raw_tokenizer(global_prompt)["input_ids"]) - eos_adj
|
||||
|
||||
@@ -1,36 +1,14 @@
|
||||
[project]
|
||||
name = "WhatDreamsCost-ComfyUI"
|
||||
description = "A variety of custom ComfyUI nodes and workflows for creatives."
|
||||
version = "1.3.9"
|
||||
license = {file = "LICENSE"}
|
||||
# classifiers = [
|
||||
# # For OS-independent nodes (works on all operating systems)
|
||||
# "Operating System :: OS Independent",
|
||||
#
|
||||
# # OR for OS-specific nodes, specify the supported systems:
|
||||
# "Operating System :: Microsoft :: Windows", # Windows specific
|
||||
# "Operating System :: POSIX :: Linux", # Linux specific
|
||||
# "Operating System :: MacOS", # macOS specific
|
||||
#
|
||||
# # GPU Accelerator support. Pick the ones that are supported by your extension.
|
||||
# "Environment :: GPU :: NVIDIA CUDA", # NVIDIA CUDA support
|
||||
# "Environment :: GPU :: AMD ROCm", # AMD ROCm support
|
||||
# "Environment :: GPU :: Intel Arc", # Intel Arc support
|
||||
# "Environment :: NPU :: Huawei Ascend", # Huawei Ascend support
|
||||
# "Environment :: GPU :: Apple Metal", # Apple Metal support
|
||||
# ]
|
||||
|
||||
|
||||
[project.urls]
|
||||
Repository = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI"
|
||||
# Used by Comfy Registry https://registry.comfy.org
|
||||
Documentation = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/wiki"
|
||||
"Bug Tracker" = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/issues"
|
||||
|
||||
[tool.comfy]
|
||||
PublisherId = "whatdreamscost"
|
||||
DisplayName = "WhatDreamsCost-ComfyUI"
|
||||
Icon = "https://raw.githubusercontent.com/WhatDreamsCost/MediaSyncer/refs/heads/main/Icon.png"
|
||||
includes = []
|
||||
# "requires-comfyui" = ">=1.0.0" # ComfyUI version compatibility
|
||||
|
||||
[project]
|
||||
name = "whatdreamscost-csglide"
|
||||
description = "LTX Director (CS) — modded LTX Director with reference-sheet / @char / MSR + provider Analyze backend. Coexists with the original WhatDreamsCost-ComfyUI without collisions."
|
||||
version = "2.0.0"
|
||||
license = {file = "LICENSE"}
|
||||
|
||||
[project.urls]
|
||||
Repository = "https://github.com/CGlide/WhatDreamsCost-CSGlide"
|
||||
|
||||
[tool.comfy]
|
||||
PublisherId = "REPLACE_WITH_YOUR_REGISTRY_PUBLISHER_ID"
|
||||
DisplayName = "WhatDreamsCost-CSGlide"
|
||||
Icon = ""
|
||||
includes = []
|
||||
|
||||
@@ -1,51 +0,0 @@
|
||||
import re
|
||||
import math
|
||||
|
||||
class SpeechLengthCalculator:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"text": ("STRING", {"multiline": True, "default": 'Enter your script here. "Make sure to put spoken words inside quotes!"'}),
|
||||
"fps": ("INT", {"default": 24, "min": 1, "max": 120, "step": 1}),
|
||||
"additional_time": ("FLOAT", {"default": 0.0, "min": 0.0, "step": 0.1}),
|
||||
},
|
||||
"optional": {
|
||||
"text_input": ("STRING", {"forceInput": True}),
|
||||
}
|
||||
}
|
||||
|
||||
# Added "STRING" to RETURN_TYPES
|
||||
RETURN_TYPES = ("INT", "INT", "INT", "STRING")
|
||||
# Added "text" to RETURN_NAMES
|
||||
RETURN_NAMES = ("slow_frame_count", "average_frame_count", "fast_frame_count", "text")
|
||||
FUNCTION = "calculate_speech"
|
||||
CATEGORY = "WhatDreamsCost"
|
||||
|
||||
def calculate_speech(self, text, fps, additional_time=0.0, text_input=None):
|
||||
# Prioritize the connected text_input if provided, otherwise fallback to the text widget
|
||||
active_text = text_input if (text_input is not None and isinstance(text_input, str) and text_input.strip() != "") else text
|
||||
|
||||
# Regex to find words inside double quotes, single quotes, or smart quotes
|
||||
matches = re.findall(r'"([^"]*)"|\'([^\']*)\'|“([^”]*)”|‘([^’]*)’', active_text)
|
||||
|
||||
# Extract matches, handling all possible captured groups from the regex
|
||||
quoted_text = " ".join([next((g for g in m if g), "") for m in matches])
|
||||
|
||||
# Split by whitespace to get words and count them
|
||||
words = quoted_text.split()
|
||||
word_count = len(words)
|
||||
|
||||
def calc_frames(wpm):
|
||||
if word_count == 0 and additional_time == 0:
|
||||
return 0
|
||||
minutes = word_count / wpm
|
||||
seconds = (minutes * 60) + additional_time
|
||||
return math.ceil(seconds * fps)
|
||||
|
||||
slow_frames = calc_frames(100)
|
||||
avg_frames = calc_frames(130)
|
||||
fast_frames = calc_frames(160)
|
||||
|
||||
# Added active_text as the 4th returned value
|
||||
return (slow_frames, avg_frames, fast_frames, active_text)
|
||||
Reference in New Issue
Block a user