25 Commits
0.13 ... 0.20

Author SHA1 Message Date
CGlide
6c6ead6f24 Update README.md 2026-06-27 22:44:28 +02:00
CGlide
fdc9dac5f1 Update README.md 2026-06-27 22:42:23 +02:00
CGlide
8c10e887c8 Delete example_workflows/LTX Director MSR_Ghost.json 2026-06-27 15:19:15 +02:00
CGlide
93ed99b9d7 Update README.md 2026-06-27 15:03:06 +02:00
CGlide
750c8961c9 Update README.md 2026-06-27 14:55:14 +02:00
CGlide
0d6d848a26 Update README.md 2026-06-27 14:47:33 +02:00
CGlide
2a684aaa4c Update README.md 2026-06-27 14:35:46 +02:00
CGlide
b67ad92eef v0.21 2026-06-27 14:27:53 +02:00
CGlide
722777cc13 Director 2CS v0.21
"WhatDreamsCost" Director node modded for Ref sheets
2026-06-27 14:21:50 +02:00
CGlide
21dd076689 Update README.md 2026-06-23 03:37:06 +02:00
CGlide
4411e71be4 Update README.md 2026-06-23 03:28:32 +02:00
CGlide
142cc0fdbc Update README.md 2026-06-23 03:26:33 +02:00
CGlide
d105088195 Update README.md 2026-06-23 03:25:08 +02:00
CGlide
6a63adc563 Update README.md 2026-06-23 03:23:17 +02:00
CGlide
fd164b480c 0.13 now co-exist with "vanilla" V2 HQ coming 2026-06-23 03:17:58 +02:00
CGlide
6542586cdc Delete js/multi_image_loader.js 2026-06-23 03:14:53 +02:00
CGlide
d12e1501b7 Delete js/ltx_sequencer.js 2026-06-23 03:14:37 +02:00
CGlide
c30e8963ef Delete js/ltx_keyframer.js 2026-06-23 03:13:44 +02:00
CGlide
6a2c0ad51f Delete js/speech_length_calculator.js 2026-06-23 03:12:54 +02:00
CGlide
32e8b9f898 Delete js/load_video_ui.js 2026-06-23 03:12:29 +02:00
CGlide
38835f62d3 Delete speech_length_calculator.py 2026-06-23 03:11:59 +02:00
CGlide
92e1f32859 Delete load_video_ui.py 2026-06-23 03:11:24 +02:00
CGlide
3fc4219a38 Delete multi_image_loader.py 2026-06-23 03:11:01 +02:00
CGlide
656a665705 Delete ltx_keyframer.py 2026-06-23 03:10:14 +02:00
CGlide
3337bf18ff Delete example_workflows directory 2026-06-23 03:08:57 +02:00
45 changed files with 391550 additions and 35935 deletions

151
README.md
View File

@@ -1,5 +1,14 @@
A Complete Timeline Editor For LTX 2.3. This is the sucessor of my previous nodes, and has loads of features in it. It was originally based off of [Kijai's Prompt Relay node](https://github.com/kijai/ComfyUI-PromptRelay) and my LTX Sequencer/Multi Image Loader nodes.
https://github.com/user-attachments/assets/68dc826f-c65f-4f1e-86cd-4f2df17bacd8
# Overview # Overview
Just added a couple of things from Director Node from "WhatdreamsCost" or Jonathan that I thought would be cool
This will be a collection of free resources for ComfyUI. This will be a collection of free resources for ComfyUI.
Hopefully it will make creating cool stuff easier. Hopefully it will make creating cool stuff easier.
@@ -9,8 +18,9 @@ All of my nodes are created with the help of AI, so there may or may not be redu
## ▶️ YouTube Tutorial Videos ## ▶️ YouTube Tutorial Videos
<a href="https://youtu.be/j28z5PZXkKk?si=TPeWKm2BW-uA3BAx"> <a href="https://youtu.be/j28z5PZXkKk?si=TPeWKm2BW-uA3BAx">
<img width="642" height="718" alt="Capture d&#39;écran 2026-06-17 234136" src="https://github.com/user-attachments/assets/89433110-8b78-4971-8098-3a9e9226f460" /> <p align="center">
<img width="620" height="675" alt="Capture d&#39;écran 2026-06-27 143149" src="https://github.com/user-attachments/assets/da3ded29-8c98-4f05-8246-7b9eb72b99f2" />
</p>
<table> <table>
<tr> <tr>
@@ -32,14 +42,13 @@ All of my nodes are created with the help of AI, so there may or may not be redu
## ❓ How to install nodes ## ❓ How to install nodes
- Navigate to your `/ComfyUI/custom_nodes/ folder` - Navigate to your `/ComfyUI/custom_nodes/ folder`
- Delete your old "WhatDreamsCost-ComfyUI" Folder - Run `git clone -b main_cs https://github.com/CGlide/WhatDreamsCost-CSGlide.git`
- Run `git clone -b main_cs https://github.com/CGlide/WhatDreamsCost-ComfyUI.git`
**❗❗IMPORTANT❗❗** **❗❗IMPORTANT❗❗**
This is a Modded LTX director node with some extra options to help you create videos with references sheets This is a Modded LTX director node from "WhatDreamsCost" with some extra options to help you create videos with references sheets
This node uses Ollama Locally : you will need to install it (very lightweight) and This node uses Ollama Locally : you will need to install it (very lightweight) and
@@ -79,81 +88,12 @@ If anyone finds anymore bugs or has idea for improvements please let me know!
* **LTX Director Example Workflow Fix** * **LTX Director Example Workflow Fix**
- Minor fix to the example workflow (i forgot to set the clip loader type to ltxv lol) - Minor fix to the example workflow (i forgot to set the clip loader type to ltxv lol)
**v1.3.0**
* **New nodes: LTX Director and LTX Director Guide**
- A complete timeline editor that can do almost everything. It's my most ambitious node so far and the successor to LTX Sequencer/Multi Image Loader.
**v1.2.9**
* **Fixed every known issue with Multi Image Loader and added text output to Speech Length Calculator**
- Removed the completely useless drag and drop animations (now it's snappy and no longer finicky)
- Fixed the node resizing on nodes 2.0
- Updated grid logic to fit images better
- Added ablity to right click images to copy/open/save images
- Fixed the "invisible hitbox" underneath node issue (actually this time).
Also added a text output to the Speech Length Calculator node (can't believe i didn't do this initially)
<details>
<summary>Click to view older Updates</summary>
**v1.2.8**
* **Updated Load Video UI and Color Conversion**
* Added crop mode, a simple interface to crop videos. It also include various aspect ratio presets.
* Updated color conversion to ensure colors are as accurate as possible. Will first check metadata for colorspace, and if metadata is missing then it will guess the colorspace based on video dimensions.
* Updated display mode toggle UI to be more understandable
**v1.2.7**
* **New Node: Load Video UI**
Custom Node to Trim, Resize, and Preview Videos in Realtime
**v1.2.6**
* **Updated Speech Length Calculator UI**
Also added duration output to the Load Audio UI node
**v1.2.5**
* **Updated Load Audio UI Node**
* Added Duration Setting
* Made the whole selection bar draggable
* Fixed Trimmed UI to show centiseconds
**v1.2.4**
* **New Node: Load Audio UI**
Overhaul of the load audio node. Features a simple interface to easily trim audio. Also allows dragging and dropping files (fixes the original node that doesn't allow dropping in videos). Also compatible with nodes 2.0.
**v1.2.3**
* **Workflow Update + Minor Bug Fix**
* Added new workflow that is compatible with the latest ComfyUI version (as of 4/27/26). The new workflow also included an option to include custom audio, and has minor improvements of the previous workflows.
* Fixed minor bug with Multi Image Loader that blocked mouse input in a small area under the node 🤷‍♂️
**v1.2.0**
* **New Node: Speech Length Calculator**
Automatically output in realtime how long a video should be based on the dialouge.
**v1.1.0**
* Added resize_method to the Multi Image Loader node for more resize options
* Added insert_mode which allows you to enter in seconds instead of frames on the LTX Sequencer node
* Updated workflows with more notes
* Re-added tiny vae to workflows
* Fixed various bugs
* more things i can't rememeber
**This update will change the node layouts, so be sure to update your workflows or else they won't work properly.**
❗❗❗ **New Tutorial on using these nodes available: https://www.youtube.com/watch?v=aXDIr8eNovI** ❗❗❗
</details>
# ⚙️ Custom Nodes
## LTX Director ## LTX Director
<img width="1481" height="833" alt="Clipboard Image (2)" src="https://github.com/user-attachments/assets/08f3fe53-9393-4f5d-9de5-58b229fbed47" /> <img width="1481" height="833" alt="Clipboard Image (2)" src="https://github.com/user-attachments/assets/08f3fe53-9393-4f5d-9de5-58b229fbed47" />
A Complete Timeline Editor For LTX 2.3. This is the sucessor of my previous nodes, and has loads of features in it. It was originally based off of [Kijai's Prompt Relay node](https://github.com/kijai/ComfyUI-PromptRelay) and my LTX Sequencer/Multi Image Loader nodes.
**Main Features:** **Main Features:**
- **Fully Functional Timeline Editor:** I spent hours studying various video editors and ended up with this design. If anyone has ideas for improvements let me know! I will adding documentation on all the functions soon. - **Fully Functional Timeline Editor:** I spent hours studying various video editors and ended up with this design. If anyone has ideas for improvements let me know! I will adding documentation on all the functions soon.
@@ -168,55 +108,7 @@ Download workflows here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyU
**Tutorial videos and documentation coming soon** **Tutorial videos and documentation coming soon**
## Multi Image Loader
<img width="1280" height="720" alt="Multi_Image_Loader_Wide_Gif" src="https://github.com/user-attachments/assets/99b6afd8-5197-4e6c-81da-a7bd156c42c7" />
An Image loader that features a built in gallery, allowing your to easily rearrange images and output them seperately or batched together. It also combines the image resize node and LTXVPreprocess node to reduce clutter in LTX workflows.
## LTX Sequencer
![LTX_Sequencer_GIF](https://github.com/user-attachments/assets/88f27155-f50e-4cb2-b937-ab173e6bdf0b)
An overhaul of the LTXVAddGuideMulti node. It allows you to quickly create FFLF (First Frame Last Frame) videos, shot sequences, supports any number of middle frames.
Connect the Multi Image Loader node's multi_output to automatically update the node's widgets.
It also has a sync feature that syncs all LTX Sequencer nodes together in realtime, removing the need to edit every single node manually every time you want to make a change to something.
## LTX Keyframer
<img width="1082" height="608" alt="LTX Keyframer Wide" src="https://github.com/user-attachments/assets/850ba4a2-dbca-4e5a-a580-1c271e9f0c41" />
An overhaul of the LTXVImgToVideoInplaceKJ node. It allows you to quickly create FFLF (First Frame Last Frame) videos and shot sequences. Also upports any number of middle frames.
Connect the Multi Image Loader node's multi_output to automatically update the node's widgets.
It also has a sync feature that syncs all LTX Keyframer nodes together in realtime, removing the need to edit every single node manually every time you want to make a change to something.
**I would recommend using the LTX Sequencer Node over this node, after further testing it seems superior in at pretty much everything. I'll leave it in just in case more people want to test it**
## Speech Length Calculator
<img width="1280" height="720" alt="Speech Length Calculator v2 Gif" src="https://github.com/user-attachments/assets/04b9a1cf-20e4-4b7b-a9c6-4a5a0825995b" />
<br>
<br>
This node calculates in realtime how long a video should be based on the dialogue. Any words in quotations will be considered as speech. The node updates in realtime without having to run the workflow, and outputs the length depending on how fast the speech is.
If you connect another string/text node to the text_input, it will still update in the length in realtime.
I kept having to play the guessing game on my own generations so I made this node to make it easier :man_shrugging:
## Load Video UI
<table width="100%">
<tr>
<td width="50%" align="center">
<p>Simple Controls</p>
<img src="https://github.com/user-attachments/assets/fb76ff03-a6ff-4837-bd63-7e429f5f3d37" width="100%" />
</td>
<td width="50%" align="center">
<p>New Crop Mode!</p>
<img src="https://github.com/user-attachments/assets/28cfb4ca-e42a-44da-9afb-f20cb01b9722" width="100%" />
</td>
</tr>
</table>
<br> <br>
<br> <br>
@@ -232,22 +124,9 @@ An upgraded Load Video node. It has the following features:
Please note that due to ComfyUI limitations (and the fact that this node doesn't use any addtional libraries), this node will not work well for outputting large videos. You can trim any length of video without a problem, but if the output is still large it will end up using a lot of RAM. I have implemented various optimizations though to make it use less memory. Please note that due to ComfyUI limitations (and the fact that this node doesn't use any addtional libraries), this node will not work well for outputting large videos. You can trim any length of video without a problem, but if the output is still large it will end up using a lot of RAM. I have implemented various optimizations though to make it use less memory.
## Load Audio UI
<img width="1280" height="720" alt="Load_Audio_UI_V2" src="https://github.com/user-attachments/assets/e3dc5c8d-d0b9-4336-8196-944204719239" />
<br>
<br>
An upgraded Load Audio node. Features a simple interface to easily trim audio. Also allows dragging and dropping files (fixes the original node that doesn't allow dropping in videos). Also compatible with nodes 2.0.
# 💡 Workflows
<img width="3120" height="990" alt="LTX I2V First Last Frame 3 Stage Workflow v6" src="https://github.com/user-attachments/assets/c993ef2f-ac4b-4091-a7f6-5ff1674c3718" />
<br>
<br>
This is a compact LTX 2.3 workflow for I2V and First Frame, Middle Frame, Last frame video generation.
I seperated and organized everything into subraphs to make things as clean as possible, and added toggles to customize the workflow quickly.
Download workflows here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/tree/main/example_workflows
Or drag and drop the image into ComfyUI to import workflow.
# ❗ Known Issues # ❗ Known Issues

View File

@@ -1,11 +1,5 @@
from .ltx_keyframer import LTXKeyframer
from .multi_image_loader import MultiImageLoader
from .ltx_sequencer import LTXSequencer
from .speech_length_calculator import SpeechLengthCalculator
from .load_audio_ui import LoadAudioUI
from .load_video_ui import LoadVideoUI
from .ltx_director import LTXDirector from .ltx_director import LTXDirector
from .ltx_director_guide import LTXDirectorGuide from .ltx_director_guide import LTXDirectorGuide, LTXDirectorCropGuides
from comfy_api.latest import ComfyExtension, io from comfy_api.latest import ComfyExtension, io
from typing_extensions import override from typing_extensions import override
from .latent_slice import CleanLatentSlice from .latent_slice import CleanLatentSlice
@@ -15,34 +9,23 @@ class PromptRelay(ComfyExtension):
async def get_node_list(self) -> list[type[io.ComfyNode]]: async def get_node_list(self) -> list[type[io.ComfyNode]]:
return [ return [
LTXDirector, LTXDirector,
LTXDirectorGuide
] ]
async def comfy_entrypoint() -> PromptRelay: async def comfy_entrypoint() -> PromptRelay:
return PromptRelay() return PromptRelay()
NODE_CLASS_MAPPINGS = { NODE_CLASS_MAPPINGS = {
"LTXKeyframer": LTXKeyframer, "LTXDirectorCS": LTXDirector,
"MultiImageLoader": MultiImageLoader, "LTXDirectorGuideCS": LTXDirectorGuide,
"LTXSequencer": LTXSequencer, "LTXDirectorCropGuidesCS": LTXDirectorCropGuides,
"SpeechLengthCalculator": SpeechLengthCalculator, "CleanLatentSliceCS": CleanLatentSlice,
"LoadAudioUI": LoadAudioUI,
"LoadVideoUI": LoadVideoUI,
"LTXDirector": LTXDirector,
"LTXDirectorGuide": LTXDirectorGuide,
"CleanLatentSlice": CleanLatentSlice,
} }
NODE_DISPLAY_NAME_MAPPINGS = { NODE_DISPLAY_NAME_MAPPINGS = {
"LTXKeyframer": "LTX Keyframer", "LTXDirectorCS": "LTX Director CS",
"MultiImageLoader": "Multi Image Loader", "LTXDirectorGuideCS": "LTX Director Guide CS",
"LTXSequencer": "LTX Sequencer", "LTXDirectorCropGuidesCS": "LTX Director Crop Guides CS",
"SpeechLengthCalculator": "Speech Length Calculator", "CleanLatentSliceCS": "Clean Latent Slice CS",
"LoadAudioUI": "Load Audio UI",
"LoadVideoUI": "Load Video UI",
"LTXDirector": "LTX Director",
"LTXDirectorGuide": "LTX Director Guide",
"CleanLatentSlice": "Clean Latent Slice",
} }
WEB_DIRECTORY = "./js" WEB_DIRECTORY = "./js"

Binary file not shown.

Binary file not shown.

379053
debug_prompt_relay.log Normal file

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

File diff suppressed because it is too large Load Diff

Binary file not shown.

Before

Width:  |  Height:  |  Size: 825 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 290 KiB

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -3,15 +3,22 @@ import { app } from "../../scripts/app.js";
// LTX Director Guide is a pure pass-through processor node. // LTX Director Guide is a pure pass-through processor node.
// All configuration (images, insert frames, strengths) comes from // All configuration (images, insert frames, strengths) comes from
// the guide_data output of Prompt Relay Encode (Timeline). // the guide_data output of Prompt Relay Encode (Timeline).
// No dynamic widgets or sync logic needed.
app.registerExtension({ app.registerExtension({
name: "Comfy.LTXDirectorGuide", name: "Comfy.LTXDirectorGuideCS",
async nodeCreated(node) { async nodeCreated(node) {
if (node.comfyClass !== "LTXDirectorGuide") return; if (node.comfyClass !== "LTXDirectorGuideCS") return;
// Nothing to initialize — the node has no configurable widgets.
},
});
// Nothing to initialize — the node has no configurable widgets. // Hide retake_mode widget on LiteGraph as it is dynamically auto-detected from the timeline data.
const w = node.widgets?.find(x => x.name === "retake_mode");
if (w) {
w.hidden = true;
if (!w.options) w.options = {};
w.options.hidden = true;
if (!window.LiteGraph || !window.LiteGraph.vueNodesMode) {
w.computeSize = () => [0, -4];
w.draw = () => { };
}
if (w.element) w.element.style.display = "none";
}
}, },
}); });

View File

@@ -1,419 +0,0 @@
import { app } from "../../scripts/app.js";
// Global registry to track all LTXKeyframer nodes across all subgraphs
window._LTXKeyframerGlobalNodes = window._LTXKeyframerGlobalNodes || new Set();
// ComfyUI native trick to cleanly hide/show widgets without deleting them
function toggleWidget(widget, visible) {
if (visible) {
if (widget.origType !== undefined) {
widget.type = widget.origType;
widget.computeSize = widget.origComputeSize;
delete widget.origType;
delete widget.origComputeSize;
}
} else {
if (widget.type !== "hidden") {
widget.origType = widget.type;
widget.origComputeSize = widget.computeSize;
widget.type = "hidden";
widget.computeSize = () => [0, -4];
}
}
}
// --- NEW SYNC HELPER FUNCTION ---
// Finds all other LTXKeyframer nodes globally and mirrors the value to them
function syncWidgetAcrossNodes(sourceNode, widgetName, value) {
if (!window._LTXKeyframerGlobalNodes) return;
for (const targetNode of window._LTXKeyframerGlobalNodes) {
// Target all OTHER LTXKeyframer nodes by direct object reference
if (targetNode !== sourceNode) {
// 1. Always update the hidden properties cache so it remembers the sync
// even if the widget isn't currently visible (e.g. fewer images loaded right now)
targetNode.properties[widgetName] = value;
// 2. If the widget is currently visible on the UI, update it visually
if (targetNode.widgets) {
const targetWidget = targetNode.widgets.find(w => w.name === widgetName);
if (targetWidget && targetWidget.value !== value) {
targetWidget.value = value;
targetNode.setDirtyCanvas(true, false);
}
}
}
}
}
app.registerExtension({
name: "Comfy.LTXKeyframer.DynamicInputs",
async nodeCreated(node) {
if (node.comfyClass !== "LTXKeyframer") return;
// Register this node instance globally
window._LTXKeyframerGlobalNodes.add(node);
node._currentImageCount = -1; // Force first update
// Initialize persistent properties cache
node.properties = node.properties || {};
// Add subtle separator line above images_loaded
node.addCustomWidget({
name: "num_images_separator",
type: "text",
draw(ctx, node, widget_width, y, widget_height) {
ctx.save();
ctx.strokeStyle = "#444";
ctx.lineWidth = 1;
ctx.beginPath();
ctx.moveTo(10, y + 5);
ctx.lineTo(widget_width - 10, y + 5);
ctx.stroke();
ctx.restore();
},
computeSize(width) {
return [width, 10];
}
});
// Move separator before num_images
const moveSeparator = () => {
const idx = node.widgets.findIndex(w => w.name === "num_images");
const sepIdx = node.widgets.findIndex(w => w.name === "num_images_separator");
if (idx !== -1 && sepIdx !== -1) {
const separator = node.widgets.splice(sepIdx, 1)[0];
node.widgets.splice(idx, 0, separator);
}
};
setTimeout(moveSeparator, 50); // Small delay to ensure num_images is present
// Core update: synchronize widget visibility to match imageCount
node._applyWidgetCount = function(count) {
const isInitialLoad = this._currentImageCount === -1;
if (this._currentImageCount === count && !isInitialLoad) return;
this._currentImageCount = count;
const initialWidth = this.size[0];
const numWidget = this.widgets?.find(w => w.name === "num_images");
if (numWidget) {
numWidget.label = "images_loaded";
numWidget.value = Math.max(0, Math.min(count || 0, 50));
}
// 1. Store current widget values in properties BEFORE removing them
// We skip reading from `this.widgets` on the initial load because it might be scrambling.
if (!isInitialLoad && this.widgets) {
this.widgets.forEach(w => {
if (w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) {
this.properties[w.name] = w.value;
}
});
}
// 2. Remove all existing dynamic insert_frame/strength/header widgets
if (this.widgets) {
this.widgets = this.widgets.filter(w =>
!w.name.startsWith("insert_frame_") &&
!w.name.startsWith("strength_") &&
!w.name.startsWith("header_")
);
} else {
this.widgets = [];
}
// 3. Add back exactly the right amount of widgets using the cached values
for (let i = 1; i <= count; i++) {
// Add header/separator widget for grouping
const headerName = `header_${i}`;
this.addCustomWidget({
name: headerName,
type: "text",
value: `Image #${i}`,
draw(ctx, node, widget_width, y, widget_height) {
ctx.save();
const margin = 10;
const topPadding = 15;
// Subtle separator line
ctx.strokeStyle = "#333";
ctx.lineWidth = 1;
ctx.beginPath();
ctx.moveTo(margin, y + 5);
ctx.lineTo(widget_width - margin, y + 5);
ctx.stroke();
// Text label
ctx.fillStyle = "#dddddd"; // Light gray
ctx.font = "bold 12px Arial";
ctx.textAlign = "left";
ctx.fillText(`Image #${i}`, margin, y + topPadding + 10);
ctx.restore();
},
computeSize(width) {
return [width, 35]; // Vertical gap + label height
}
});
const insertFrameWidgetName = `insert_frame_${i}`;
const strengthWidgetName = `strength_${i}`;
// Add insert_frame widget with Sync Callback
const savedInsertFrameValue = this.properties[insertFrameWidgetName];
this.addWidget("number", insertFrameWidgetName,
savedInsertFrameValue !== undefined ? savedInsertFrameValue : 0,
(value) => {
const rounded = Math.round(value);
this.properties[insertFrameWidgetName] = rounded;
syncWidgetAcrossNodes(this, insertFrameWidgetName, rounded); // Sync out
}, { min: -9999, max: 9999, step: 10, precision: 0 }
);
// Add strength widget with Sync Callback
const savedStrengthValue = this.properties[strengthWidgetName];
this.addWidget("number", strengthWidgetName,
savedStrengthValue !== undefined ? savedStrengthValue : 1.0,
(value) => {
this.properties[strengthWidgetName] = value;
syncWidgetAcrossNodes(this, strengthWidgetName, value); // Sync out
}, { min: 0.0, max: 1.0, step: 0.01 }
);
}
this.setDirtyCanvas(true, true);
requestAnimationFrame(() => {
if (this.computeSize) {
this.setSize(this.computeSize());
this.size[0] = initialWidth; // keep width fixed when restructuring
}
});
};
// --- STRICT ARRAY MAPPER: FIXES ALL SHIFTING FOREVER ---
// This runs the exact instant the node is loaded, before any UI widgets shift indices.
// It locks the perfectly mapped array values directly into our properties dictionary.
const origConfigure = node.configure;
node.configure = function(info) {
if (origConfigure) {
origConfigure.apply(this, arguments);
}
if (this.widgets) {
this.widgets.forEach(w => {
if (w.name === "num_images" || w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) {
this.properties[w.name] = w.value;
}
});
}
};
// Handle deserialization to load properties properly from JSON
const originalOnConfigure = node.onConfigure;
node.onConfigure = function(info) {
if (originalOnConfigure) {
originalOnConfigure.apply(this, arguments);
}
if (info.properties) {
this.properties = { ...this.properties, ...info.properties };
}
setTimeout(() => {
const count = readSourceImageCount(this);
// Fallback to properties.num_images if source node disconnected
let targetCount = count !== null ? count : (this.properties.num_images || 0);
this._applyWidgetCount(targetCount);
}, 100);
};
// --- STRICT ARRAY GENERATOR ---
// Completely detach from ComfyUI's blind visual array saving.
// We construct an exact 101-element strict array that Python expects.
// This makes your node 100% immune to UI/Header index shifting.
const originalOnSerialize = node.onSerialize;
node.onSerialize = function(info) {
// Ensure properties are strictly synced with current widget values before building
if (this.widgets) {
this.widgets.forEach(w => {
if (w.name === "num_images" || w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) {
this.properties[w.name] = w.value;
}
});
}
if (originalOnSerialize) {
originalOnSerialize.apply(this, arguments);
}
info.properties = { ...this.properties };
// Build the exact strict array that maps 1-to-1 to the Python backend
const strictArray = [];
const numWidgetVal = this.properties["num_images"];
strictArray.push(numWidgetVal !== undefined ? numWidgetVal : 1);
for (let i = 1; i <= 50; i++) {
const fVal = this.properties[`insert_frame_${i}`];
const sVal = this.properties[`strength_${i}`];
strictArray.push(fVal !== undefined ? fVal : 0);
strictArray.push(sVal !== undefined ? sVal : 1.0);
}
info.widgets_values = strictArray;
};
// Set up manual num_images widget callback
setTimeout(() => {
const numWidget = node.widgets?.find(w => w.name === "num_images");
if (numWidget) {
numWidget.callback = (val) => {
node.properties["num_images"] = val;
node._applyWidgetCount(val);
};
}
}, 100);
// Exposed receiver for push-based notifications
node._syncImageCount = function(count) {
this._applyWidgetCount(count);
};
// Helper: read image count from a connected MultiImageLoader node
function readSourceImageCount(self) {
const multiInput = self.inputs?.find(inp => inp.name === "multi_input");
if (!multiInput || !multiInput.link) return null;
const nodeGraph = self.graph || app.graph;
// Helper to safely trace back through Reroutes and ComfyUI Group Nodes/Subgraphs
function traceUpstream(graph, linkId, visited = new Set()) {
if (!linkId || visited.has(linkId)) return null;
visited.add(linkId);
const link = graph.links[linkId];
if (!link) return null;
const originNode = graph.getNodeById(link.origin_id);
if (!originNode) return null;
if (originNode.comfyClass === "MultiImageLoader") {
return originNode;
}
// Traverse Reroute nodes
if (originNode.type === "Reroute" || originNode.comfyClass === "Reroute") {
if (originNode.inputs && originNode.inputs.length > 0 && originNode.inputs[0].link) {
return traceUpstream(graph, originNode.inputs[0].link, visited);
}
}
// Traverse standard ComfyUI Group Nodes (subgraphs)
if (typeof originNode.getInnerNode === "function") {
try {
const innerNode = originNode.getInnerNode(link.origin_slot);
if (innerNode && innerNode.comfyClass === "MultiImageLoader") {
return innerNode;
}
} catch (e) {
console.warn("Could not trace inner node", e);
}
}
return null;
}
let sourceNode = traceUpstream(nodeGraph, multiInput.link);
// Helper to extract the count once we find a node
function getCountFromNode(n) {
if (typeof n._imageCount === "number") return n._imageCount;
const pathsWidget = n.widgets?.find(w => w.name === "image_paths");
if (pathsWidget) {
return (pathsWidget.value || "").split('\n').map(p => p.trim()).filter(p => p.length > 0).length;
}
return null;
}
if (sourceNode) {
return getCountFromNode(sourceNode);
}
// Fallback Strategy: If it is connected to something, but we couldn't resolve it
// directly (e.g. complex nested 3rd party subgraphs), scan the entire UI.
// If there is EXACTLY ONE MultiImageLoader in the workspace, safely assume that's the one.
let multiImageLoaders = [];
function findAllLoaders(nodes) {
if (!nodes) return;
for (let n of nodes) {
if (n.comfyClass === "MultiImageLoader") {
multiImageLoaders.push(n);
}
if (n.subgraph && n.subgraph._nodes) {
findAllLoaders(n.subgraph._nodes);
}
}
}
if (app.graph && app.graph._nodes) {
findAllLoaders(app.graph._nodes);
}
if (multiImageLoaders.length === 1) {
return getCountFromNode(multiImageLoaders[0]);
}
return null;
}
// --- Backup polling via setInterval (4 Hz) ---
const pollInterval = setInterval(() => {
if (!node.graph) {
clearInterval(pollInterval);
return;
}
const count = readSourceImageCount(node);
if (count !== null) {
node._applyWidgetCount(count);
}
}, 250);
// Clean up the interval when the node is deleted
const origOnRemoved = node.onRemoved;
node.onRemoved = function() {
// Unregister this node instance
window._LTXKeyframerGlobalNodes.delete(node);
clearInterval(pollInterval);
if (origOnRemoved) origOnRemoved.apply(this, arguments);
};
// --- Connection change handler ---
const onConnectionsChange = node.onConnectionsChange;
node.onConnectionsChange = function(type, index, connected, link_info) {
if (onConnectionsChange) onConnectionsChange.apply(this, arguments);
if (type === 1) { // 1 = Input
const input = this.inputs[index];
if (input && input.name === "multi_input") {
if (connected) {
setTimeout(() => {
const count = readSourceImageCount(this);
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
}, 100);
} else {
this._applyWidgetCount(0);
}
}
}
};
// --- Initial sync when first placed on canvas ---
const origOnAdded = node.onAdded;
node.onAdded = function() {
if (origOnAdded) origOnAdded.apply(this, arguments);
setTimeout(() => {
const count = readSourceImageCount(this);
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
}, 100);
};
}
});

View File

@@ -1,398 +0,0 @@
import { app } from "../../scripts/app.js";
// Global registry to track all LTXSequencer nodes across all subgraphs
window._LTXSequencerGlobalNodes = window._LTXSequencerGlobalNodes || new Set();
// ComfyUI native trick to cleanly hide/show widgets without deleting them
function toggleWidget(widget, visible) {
if (visible) {
if (widget.origType !== undefined) {
widget.type = widget.origType;
widget.computeSize = widget.origComputeSize;
delete widget.origType;
delete widget.origComputeSize;
}
} else {
if (widget.type !== "hidden") {
widget.origType = widget.type;
widget.origComputeSize = widget.computeSize;
widget.type = "hidden";
widget.computeSize = () => [0, -4];
}
}
}
/**
* FULL STATE SYNC:
* Instead of syncing one widget, we push the entire properties object
* to ensure no values are ever lost during subgraph transitions or deletions.
*/
function syncFullStateAcrossNodes(sourceNode) {
if (!window._LTXSequencerGlobalNodes) return;
for (const targetNode of window._LTXSequencerGlobalNodes) {
if (targetNode === sourceNode) continue;
// 1. Mirror the properties object completely
// We use a shallow copy to ensure we don't accidentally share object references
const newState = { ...sourceNode.properties };
targetNode.properties = { ...targetNode.properties, ...newState };
// 2. Check if we need to rebuild the widget list (if num_images changed)
const targetImageCount = targetNode.properties["num_images"] || 0;
const currentVisibleCount = targetNode._currentImageCount;
if (targetImageCount !== currentVisibleCount) {
targetNode._applyWidgetCount(targetImageCount);
}
// 3. Update all existing widget values visually
if (targetNode.widgets) {
let modeChanged = false;
targetNode.widgets.forEach(w => {
const newValue = targetNode.properties[w.name];
if (newValue !== undefined && w.value !== newValue) {
w.value = newValue;
if (w.name === "insert_mode") modeChanged = true;
}
});
if (modeChanged && targetNode._updateVisibility) {
targetNode._updateVisibility();
}
}
targetNode.setDirtyCanvas(true, false);
}
}
app.registerExtension({
name: "Comfy.LTXSequencer.DynamicInputs",
async nodeCreated(node) {
if (node.comfyClass !== "LTXSequencer") return;
// Register this node instance globally
window._LTXSequencerGlobalNodes.add(node);
node._currentImageCount = -1; // Force first update
// Initialize persistent properties cache
node.properties = node.properties || {};
// Add subtle separator line above images_loaded
node.addCustomWidget({
name: "num_images_separator",
type: "text",
draw(ctx, node, widget_width, y, widget_height) {
ctx.save();
ctx.strokeStyle = "#444";
ctx.lineWidth = 1;
ctx.beginPath();
ctx.moveTo(10, y + 5);
ctx.lineTo(widget_width - 10, y + 5);
ctx.stroke();
ctx.restore();
},
computeSize(width) {
return [width, 10];
}
});
// Move separator before num_images
const moveSeparator = () => {
const idx = node.widgets.findIndex(w => w.name === "num_images");
const sepIdx = node.widgets.findIndex(w => w.name === "num_images_separator");
if (idx !== -1 && sepIdx !== -1) {
const separator = node.widgets.splice(sepIdx, 1)[0];
node.widgets.splice(idx, 0, separator);
}
};
setTimeout(moveSeparator, 50);
// Binds custom callbacks to python-schema generated widgets
node._hookStaticWidgets = function() {
if (!this.widgets) return;
const staticNames = ["num_images", "insert_mode", "frame_rate"];
staticNames.forEach(name => {
const w = this.widgets.find(w => w.name === name);
if (w && !w._has_custom_callback) {
const orig = w.callback;
w.callback = (val) => {
this.properties[name] = val;
if (name === "num_images") {
this._applyWidgetCount(val);
}
// Push full state to siblings
syncFullStateAcrossNodes(this);
if (name === "insert_mode") {
this._updateVisibility();
}
if (orig) orig.apply(w, [val]);
};
w._has_custom_callback = true;
}
});
};
// Handles show/hiding widgets based on insertion method
node._updateVisibility = function() {
const mode = this.properties["insert_mode"] || "frames";
if (!this.widgets) return;
let changed = false;
for (const w of this.widgets) {
let shouldBeVisible = true;
if (w.name.startsWith("insert_frame_")) {
shouldBeVisible = (mode === "frames");
} else if (w.name.startsWith("insert_second_")) {
shouldBeVisible = (mode === "seconds");
}
const isHidden = (w.type === "hidden");
if (shouldBeVisible && isHidden) {
toggleWidget(w, true);
changed = true;
} else if (!shouldBeVisible && !isHidden) {
toggleWidget(w, false);
changed = true;
}
}
if (changed) {
this.setDirtyCanvas(true, true);
requestAnimationFrame(() => {
if (this.computeSize) {
this.setSize(this.computeSize());
}
});
}
};
// Core update: synchronize widget visibility to match imageCount
node._applyWidgetCount = function(count) {
this._hookStaticWidgets();
const isInitialLoad = this._currentImageCount === -1;
if (this._currentImageCount === count && !isInitialLoad) return;
this._currentImageCount = count;
const initialWidth = this.size[0];
const numWidget = this.widgets?.find(w => w.name === "num_images");
if (numWidget) {
numWidget.label = "images_loaded";
numWidget.value = Math.max(0, Math.min(count || 0, 50));
}
// 1. Update properties from current widget values before restructuring
if (this.widgets) {
this.widgets.forEach(w => {
if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) {
if (w.type !== "hidden" && w.type !== "button") this.properties[w.name] = w.value;
}
});
}
// 2. Clear dynamic widgets
if (this.widgets) {
this.widgets = this.widgets.filter(w =>
!w.name.startsWith("insert_frame_") &&
!w.name.startsWith("insert_second_") &&
!w.name.startsWith("strength_") &&
!w.name.startsWith("header_")
);
} else {
this.widgets = [];
}
// 3. Rebuild precisely
for (let i = 1; i <= count; i++) {
// Header
const headerName = `header_${i}`;
this.addCustomWidget({
name: headerName,
type: "text",
value: `Image #${i}`,
draw(ctx, node, widget_width, y, widget_height) {
ctx.save();
const margin = 10;
const topPadding = 15;
ctx.strokeStyle = "#333";
ctx.lineWidth = 1;
ctx.beginPath();
ctx.moveTo(margin, y + 5);
ctx.lineTo(widget_width - margin, y + 5);
ctx.stroke();
ctx.fillStyle = "#dddddd";
ctx.font = "bold 12px Arial";
ctx.textAlign = "left";
ctx.fillText(`Image #${i}`, margin, y + topPadding + 10);
ctx.restore();
},
computeSize(width) { return [width, 35]; }
});
// Helper to add widget with full sync
const addSyncedWidget = (type, name, def, options) => {
const saved = this.properties[name];
return this.addWidget(type, name, saved !== undefined ? saved : def, (val) => {
this.properties[name] = val;
syncFullStateAcrossNodes(this);
}, options);
};
addSyncedWidget("number", `insert_frame_${i}`, 0, { min: -9999, max: 9999, step: 10, precision: 0 });
addSyncedWidget("number", `insert_second_${i}`, 0.0, { min: 0.0, max: 9999.0, step: 0.1, precision: 2 });
addSyncedWidget("number", `strength_${i}`, 1.0, { min: 0.0, max: 1.0, step: 0.01 });
}
this._updateVisibility();
this.setDirtyCanvas(true, true);
requestAnimationFrame(() => {
if (this.computeSize) {
this.setSize(this.computeSize());
this.size[0] = initialWidth;
}
});
};
const origConfigure = node.configure;
node.configure = function(info) {
if (origConfigure) origConfigure.apply(this, arguments);
if (this.widgets) {
this.widgets.forEach(w => {
if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) {
if (w.type !== "button") this.properties[w.name] = w.value;
}
});
}
};
node.onConfigure = function(info) {
if (info.properties) {
this.properties = { ...this.properties, ...info.properties };
}
this._hookStaticWidgets();
setTimeout(() => {
const count = readSourceImageCount(this);
let targetCount = count !== null ? count : (this.properties.num_images || 0);
this._applyWidgetCount(targetCount);
this._updateVisibility();
}, 100);
};
const originalOnSerialize = node.onSerialize;
node.onSerialize = function(info) {
if (this.widgets) {
this.widgets.forEach(w => {
if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) {
if (w.type !== "hidden" && w.type !== "button") this.properties[w.name] = w.value;
}
});
}
if (originalOnSerialize) originalOnSerialize.apply(this, arguments);
info.properties = { ...this.properties };
const strictArray = [];
strictArray.push(this.properties["num_images"] !== undefined ? this.properties["num_images"] : 1);
strictArray.push(this.properties["insert_mode"] !== undefined ? this.properties["insert_mode"] : "frames");
strictArray.push(this.properties["frame_rate"] !== undefined ? this.properties["frame_rate"] : 24);
for (let i = 1; i <= 50; i++) {
strictArray.push(this.properties[`insert_frame_${i}`] !== undefined ? this.properties[`insert_frame_${i}`] : 0);
strictArray.push(this.properties[`insert_second_${i}`] !== undefined ? this.properties[`insert_second_${i}`] : 0.0);
strictArray.push(this.properties[`strength_${i}`] !== undefined ? this.properties[`strength_${i}`] : 1.0);
}
info.widgets_values = strictArray;
};
function readSourceImageCount(self) {
const multiInput = self.inputs?.find(inp => inp.name === "multi_input");
if (!multiInput || !multiInput.link) return null;
const nodeGraph = self.graph || app.graph;
function traceUpstream(graph, linkId, visited = new Set()) {
if (!linkId || visited.has(linkId)) return null;
visited.add(linkId);
const link = graph.links[linkId];
if (!link) return null;
const originNode = graph.getNodeById(link.origin_id);
if (!originNode) return null;
if (originNode.comfyClass === "MultiImageLoader") return originNode;
if (originNode.type === "Reroute" || originNode.comfyClass === "Reroute") {
if (originNode.inputs?.[0]?.link) return traceUpstream(graph, originNode.inputs[0].link, visited);
}
if (typeof originNode.getInnerNode === "function") {
try {
const innerNode = originNode.getInnerNode(link.origin_slot);
if (innerNode?.comfyClass === "MultiImageLoader") return innerNode;
} catch (e) {}
}
return null;
}
let sourceNode = traceUpstream(nodeGraph, multiInput.link);
function getCountFromNode(n) {
if (typeof n._imageCount === "number") return n._imageCount;
const pathsWidget = n.widgets?.find(w => w.name === "image_paths");
return pathsWidget ? (pathsWidget.value || "").split('\n').filter(p => p.trim()).length : null;
}
if (sourceNode) return getCountFromNode(sourceNode);
let multiImageLoaders = [];
function findAllLoaders(nodes) {
if (!nodes) return;
for (let n of nodes) {
if (n.comfyClass === "MultiImageLoader") multiImageLoaders.push(n);
if (n.subgraph?._nodes) findAllLoaders(n.subgraph._nodes);
}
}
if (app.graph?._nodes) findAllLoaders(app.graph._nodes);
if (multiImageLoaders.length === 1) return getCountFromNode(multiImageLoaders[0]);
return null;
}
const pollInterval = setInterval(() => {
if (!node.graph) {
clearInterval(pollInterval);
return;
}
const count = readSourceImageCount(node);
if (count !== null && count !== node._currentImageCount) {
node._applyWidgetCount(count);
syncFullStateAcrossNodes(node); // Sync the new count to others
}
}, 500);
const origOnRemoved = node.onRemoved;
node.onRemoved = function() {
window._LTXSequencerGlobalNodes.delete(node);
clearInterval(pollInterval);
if (origOnRemoved) origOnRemoved.apply(this, arguments);
};
node.onConnectionsChange = function(type, index, connected) {
if (type === 1 && this.inputs[index]?.name === "multi_input") {
if (connected) {
setTimeout(() => {
const count = readSourceImageCount(this);
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
}, 100);
} else {
this._applyWidgetCount(0);
}
}
};
node.onAdded = function() {
setTimeout(() => {
const count = readSourceImageCount(this);
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
}, 100);
};
}
});

View File

@@ -1,688 +0,0 @@
import { app } from "../../scripts/app.js";
import { api } from "../../scripts/api.js";
app.registerExtension({
name: "Comfy.MultiImageLoader",
async nodeCreated(node) {
if (node.comfyClass !== "MultiImageLoader") return;
// Helper to detect if we are in the new Nodes 2.0 / V3 Web Component frontend
let v3NodeElement = null;
function checkIsV3() {
if (v3NodeElement) return true;
let el = container.parentElement;
while (el) {
if ((el.tagName && el.tagName.toLowerCase().includes('comfy-node')) ||
(el.classList && el.classList.contains('comfy-node'))) {
v3NodeElement = el;
return true;
}
el = el.parentElement || (el.getRootNode ? el.getRootNode().host : null);
}
return false;
}
// --- 1. UI Setup: Main Container ---
const container = document.createElement("div");
container.style.cssText = `
width: 100%;
min-height: 250px;
min-width: 100px; /* Reduced from 400px to allow thin resizing in V3 */
background: #222222;
border: 1px solid #353545;
border-radius: 4px;
margin-top: 5px;
padding: 10px;
box-sizing: border-box;
display: flex;
flex-direction: column;
gap: 10px;
pointer-events: auto;
overflow: hidden;
`;
// Top Bar for Actions
const topBar = document.createElement("div");
// Added flex-wrap: wrap so buttons stack if the node gets extremely thin
topBar.style.cssText = "display: flex; flex-wrap: wrap; justify-content: flex-start; align-items: center; width: 100%; gap: 8px;";
const uploadBtn = document.createElement("button");
uploadBtn.innerText = "Upload Images";
uploadBtn.style.cssText = `
background: #3a3f4b; color: white; border: 1px solid #5a5f6b;
padding: 3px 8px; border-radius: 3px; cursor: pointer; font-size: 10px;
`;
const removeAllBtn = document.createElement("button");
removeAllBtn.innerText = "Remove All";
removeAllBtn.style.cssText = `
background: #cc2222; color: white; border: 1px solid #aa1111;
padding: 3px 8px; border-radius: 3px; cursor: pointer; font-size: 10px;
transition: background 0.2s;
`;
removeAllBtn.onmouseenter = () => { removeAllBtn.style.background = "#ff3333"; };
removeAllBtn.onmouseleave = () => { removeAllBtn.style.background = "#cc2222"; };
removeAllBtn.onclick = () => {
setWidgetValue([], false);
};
topBar.appendChild(uploadBtn);
topBar.appendChild(removeAllBtn);
container.appendChild(topBar);
const gridWrapper = document.createElement("div");
gridWrapper.style.cssText = `
position: relative;
flex-grow: 1;
width: 100%;
min-height: 0;
`;
const grid = document.createElement("div");
grid.style.cssText = `
position: absolute;
top: 0; left: 0; right: 0; bottom: 0;
display: grid;
gap: 8px;
justify-content: center;
align-content: center;
`;
gridWrapper.appendChild(grid);
container.appendChild(gridWrapper);
const fileInput = document.createElement("input");
fileInput.type = "file";
fileInput.multiple = true;
fileInput.accept = "image/*";
fileInput.style.display = "none";
container.appendChild(fileInput);
// Add the Widget to the Node
const galleryWidget = node.addDOMWidget("Gallery", "html_gallery", container, { serialize: false });
galleryWidget.computeSize = function() {
const galleryY = this.last_y || 40;
const minOutputsHeight = (node.outputs ? node.outputs.length : 1) * 20;
const requiredGalleryHeight = Math.max(250, minOutputsHeight + 40 - galleryY);
return [150, requiredGalleryHeight]; // Changed minimum theoretical widget width
};
// --- SAFELY HIDE THE IMAGE_PATHS WIDGET ---
const pathsWidget = node.widgets.find(w => w.name === "image_paths");
if (pathsWidget) {
// Forcefully lock the hidden state against V3's reactive redraws
Object.defineProperty(pathsWidget, 'hidden', {
get: () => true,
set: () => {} // Ignore attempts by V3 to unhide it
});
Object.defineProperty(pathsWidget, 'type', {
get: () => "hidden",
set: () => {} // Ignore attempts by V3 to reset the type
});
pathsWidget.computeSize = function() {
return [0, 0];
};
// Catch for V3 delayed DOM rendering to ensure no stubborn inputs appear
const hideInterval = setInterval(() => {
if (pathsWidget.element) {
pathsWidget.element.style.display = "none";
}
}, 50);
setTimeout(() => clearInterval(hideInterval), 1000);
}
const oldCallback = pathsWidget?.callback;
function setWidgetValue(newPathsArray, isRearranging = false) {
if (!pathsWidget) return;
const val = newPathsArray.join("\n");
const tempCallback = pathsWidget.callback;
pathsWidget.callback = null;
pathsWidget.value = val;
if (oldCallback) oldCallback.apply(pathsWidget, [val]);
pathsWidget.callback = tempCallback;
refreshGallery(isRearranging);
}
// --- 2. Logic: Output Syncing & Dynamic Packing ---
function syncOutputs(count) {
if (!node.outputs) return;
let changed = false;
const targetTotal = count + 1;
const wasFresh = node.outputs.length >= 50;
while (node.outputs.length > targetTotal && node.outputs.length > 1) {
node.removeOutput(node.outputs.length - 1);
changed = true;
}
for (let i = node.outputs.length; i < targetTotal; i++) {
node.addOutput(`image_${i}`, "IMAGE");
changed = true;
}
if (changed || wasFresh) {
updateLayout(wasFresh);
}
}
function notifyConnectedNodes(imageCount) {
if (!node.outputs) return;
for (const output of node.outputs) {
if (!output.links) continue;
for (const linkId of output.links) {
const link = app.graph.links[linkId];
if (!link) continue;
const targetNode = app.graph.getNodeById(link.target_id);
if (targetNode && typeof targetNode._syncImageCount === "function") {
targetNode._syncImageCount(imageCount);
}
}
}
}
function optimizeGrid(gridW, gridH) {
const paths = (pathsWidget?.value || "").split(/\n|,/).map(s => s.trim()).filter(s => s);
const N = paths.length;
if (N === 0) {
grid.style.gridTemplateColumns = 'repeat(auto-fit, minmax(75px, 1fr))';
grid.style.gridAutoRows = 'max-content';
return;
}
if (gridW <= 0 || gridH <= 0) return;
let bestS = 0;
let bestCols = 1;
for (let c = 1; c <= N; c++) {
const r = Math.ceil(N / c);
// Math.max guarantees we don't end up with negative max width in ultra-thin layouts
const maxW = Math.max(5, (gridW - (c - 1) * 8) / c);
const maxH = Math.max(5, (gridH - (r - 1) * 8) / r);
const size = Math.min(maxW, maxH);
// By using >= instead of strict > (with a tiny 0.1 buffer for float precision),
// if multiple column counts yield the exact same optimal cell size
// (which happens when height is the bottleneck), we aggressively pack
// more items horizontally onto the row to fill empty space.
if (size >= bestS - 0.1) {
bestS = size;
bestCols = c;
}
}
// Allow grid cells to shrink down to 10px instead of 15 to prevent horizontal overflow in V3
bestS = Math.max(10, Math.floor(bestS));
grid.style.gridTemplateColumns = `repeat(${bestCols}, ${bestS}px)`;
grid.style.gridAutoRows = `${bestS}px`;
}
let v3EventsAttached = false;
function enforceV3CSS() {
const isV3 = checkIsV3();
if (isV3 && v3NodeElement) {
const paddingBottom = 15;
const galleryY = galleryWidget.last_y || 40;
const minOutputsHeight = (node.outputs ? node.outputs.length : 1) * 20;
const absoluteMinHeight = Math.max(galleryY + 250 + paddingBottom, minOutputsHeight + 40);
// For Nodes 2.0 (V3), we remove the min-width constraint completely.
// We leave min-height so outputs don't bleed out vertically.
v3NodeElement.style.removeProperty('min-width');
v3NodeElement.style.setProperty('min-height', absoluteMinHeight + 'px', 'important');
// Attach drag & drop to the entire V3 Web Component
if (!v3EventsAttached) {
v3EventsAttached = true;
v3NodeElement.addEventListener("dragover", (e) => {
e.preventDefault();
});
v3NodeElement.addEventListener("drop", (e) => {
if (e.dataTransfer && e.dataTransfer.files) {
const files = Array.from(e.dataTransfer.files).filter(f => f.type.startsWith('image/'));
if (files.length > 0) {
e.preventDefault();
e.stopPropagation();
handleFiles(files);
}
}
});
}
}
}
let isLayouting = false;
function updateLayout(forceShrink = false) {
if (isLayouting) return;
isLayouting = true;
const isV3 = checkIsV3();
const minW = isV3 ? 100 : 200; // 100 in V3, 440 in V1
const paddingBottom = isV3 ? 15 : 25; // Apply extra 20px pad on V1
const galleryY = galleryWidget.last_y || 40;
const minOutputsHeight = (node.outputs ? node.outputs.length : 1) * 20;
const absoluteMinHeight = Math.max(galleryY + 250 + paddingBottom, minOutputsHeight + 40);
node.min_size = [minW, absoluteMinHeight];
enforceV3CSS();
let targetW = Math.max(node.size[0], minW);
let targetH = forceShrink ? absoluteMinHeight : node.size[1];
targetH = Math.max(targetH, absoluteMinHeight);
if (node.size[0] !== targetW || node.size[1] !== targetH) {
node.setSize([targetW, targetH]);
app.graph.setDirtyCanvas(true, true);
}
const availableGalleryHeight = targetH - galleryY - paddingBottom;
container.style.height = availableGalleryHeight + "px";
isLayouting = false;
}
// --- OVERRIDE LOGIC FOR RESIZING ---
const origOnResize = node.onResize;
node.onResize = function(size) {
const isV3 = checkIsV3();
const minW = isV3 ? 100 : 220; // Adjust limits based on frontend
const paddingBottom = isV3 ? 15 : 25; // Apply extra 20px pad on V1
const galleryY = galleryWidget.last_y || 40;
const minOutputsHeight = (this.outputs ? this.outputs.length : 1) * 20;
const absoluteMinHeight = Math.max(galleryY + 250 + paddingBottom, minOutputsHeight + 40);
size[0] = Math.max(size[0], minW);
size[1] = Math.max(size[1], absoluteMinHeight);
if (origOnResize) origOnResize.call(this, size);
if (isLayouting) return;
node.min_size = [minW, absoluteMinHeight];
enforceV3CSS();
const availableGalleryHeight = size[1] - galleryY - paddingBottom;
container.style.height = availableGalleryHeight + "px";
};
const origComputeSize = node.computeSize;
node.computeSize = function(out) {
const isV3 = checkIsV3();
const minW = isV3 ? 100 : 220;
const paddingBottom = isV3 ? 15 : 25;
let res = origComputeSize ? origComputeSize.apply(this, arguments) : [minW, 250];
const galleryY = galleryWidget.last_y || 40;
const minOutputsHeight = (this.outputs ? this.outputs.length : 1) * 20;
const absoluteMinHeight = Math.max(galleryY + 250 + paddingBottom, minOutputsHeight + 40);
this.min_size = [minW, absoluteMinHeight];
res[0] = Math.max(res[0], minW);
res[1] = Math.max(res[1], absoluteMinHeight);
enforceV3CSS();
return res;
};
const origSetSize = node.setSize;
node.setSize = function(size) {
const isV3 = checkIsV3();
const minW = isV3 ? 100 : 220;
const paddingBottom = isV3 ? 15 : 25;
const galleryY = galleryWidget.last_y || 40;
const minOutputsHeight = (this.outputs ? this.outputs.length : 1) * 20;
const absoluteMinHeight = Math.max(galleryY + 250 + paddingBottom, minOutputsHeight + 40);
size[0] = Math.max(size[0], minW);
size[1] = Math.max(size[1], absoluteMinHeight);
if (origSetSize) {
origSetSize.call(this, size);
} else {
this.size = size;
}
enforceV3CSS();
};
let lastObservedWidth = 0;
let lastObservedHeight = 0;
const resizeObserver = new ResizeObserver((entries) => {
enforceV3CSS();
for (const entry of entries) {
const w = Math.round(entry.contentRect.width);
const h = Math.round(entry.contentRect.height);
if (Math.abs(w - lastObservedWidth) > 1 || Math.abs(h - lastObservedHeight) > 1) {
lastObservedWidth = w;
lastObservedHeight = h;
if (h > 0) {
optimizeGrid(w, h);
}
}
}
});
resizeObserver.observe(gridWrapper);
// --- 3. Logic: Gallery Rendering ---
let draggedNode = null;
let lastSwapX = 0;
let lastSwapY = 0;
let lastSwapTime = 0;
function refreshGallery(isRearranging = false) {
grid.innerHTML = "";
const paths = (pathsWidget?.value || "").split(/\n|,/).map(s => s.trim()).filter(s => s);
if (!isRearranging) {
syncOutputs(paths.length);
}
node._imageCount = paths.length;
notifyConnectedNodes(paths.length);
paths.forEach((path, index) => {
const item = document.createElement("div");
item.dataset.path = path;
item.draggable = true;
item.style.cssText = `
position: relative;
width: 100%;
height: 100%;
aspect-ratio: 1 / 1;
background: #000000;
border-radius: 4px;
border: 1px solid #444;
overflow: hidden;
cursor: grab;
display: flex;
align-items: center;
justify-content: center;
`;
const img = document.createElement("img");
img.src = `/api/view?filename=${encodeURIComponent(path)}&type=input`;
// Allow pointer-events so context menu interacts directly with the image
img.style.cssText = "max-width: 100%; max-height: 100%; object-fit: contain; pointer-events: auto; display: block;";
img.draggable = false; // Prevent native browser ghost dragging on the image itself
const del = document.createElement("div");
del.style.cssText = `
position: absolute; top: 0; right: 0;
background: #cc2222; color: white;
width: 18px; height: 18px;
display: flex; align-items: center; justify-content: center;
font-size: 14px; cursor: pointer; z-index: 10;
font-family: Arial, sans-serif; font-weight: bold;
line-height: 1; border-bottom-left-radius: 4px;
transition: background 0.2s;
`;
del.innerHTML = `<svg width="10" height="10" viewBox="0 0 10 10" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M1 1L9 9M9 1L1 9" stroke="white" stroke-width="2" stroke-linecap="round"/>
</svg>`;
del.onmouseenter = () => { del.style.background = "#ff3333"; };
del.onmouseleave = () => { del.style.background = "#cc2222"; };
del.onclick = (e) => {
e.stopPropagation();
const newPaths = paths.filter((_, i) => i !== index);
setWidgetValue(newPaths, false);
};
const numBadge = document.createElement("div");
numBadge.style.cssText = `
position: absolute; bottom: 0; left: 0;
background: rgba(0, 0, 0, 0.75); color: #fff;
padding: 2px 6px; font-size: 11px; font-family: sans-serif;
font-weight: bold; border-top-right-radius: 4px; pointer-events: none;
z-index: 5;
`;
numBadge.innerText = (index + 1).toString();
// Prevent LiteGraph context menu and instead show standard browser context menu (Copy, Save, Open)
item.addEventListener("contextmenu", (e) => {
e.stopPropagation();
});
item.ondragstart = (e) => {
draggedNode = item;
e.dataTransfer.setData('text/plain', path);
e.dataTransfer.effectAllowed = "move";
setTimeout(() => {
if (draggedNode === item) {
// Style as an empty dashed placeholder
item.style.background = "transparent";
item.style.border = "2px dashed #666";
// Hide the visual children (image, delete button, badge)
Array.from(item.children).forEach(c => c.style.opacity = "0");
}
}, 0);
};
item.ondragend = () => {
if (draggedNode) {
// Restore original appearance
draggedNode.style.background = "#000000";
draggedNode.style.border = "1px solid #444";
Array.from(draggedNode.children).forEach(c => c.style.opacity = "1");
}
draggedNode = null;
const newPaths = Array.from(grid.children).map(n => n.dataset.path);
const currentVal = (pathsWidget?.value || "").trim();
if (newPaths.join("\n") !== currentVal) {
setWidgetValue(newPaths, true);
}
};
item.ondragover = (e) => {
e.preventDefault();
e.stopPropagation();
if (!draggedNode || draggedNode === item) return;
const distMoved = Math.hypot(e.clientX - lastSwapX, e.clientY - lastSwapY);
if (Date.now() - lastSwapTime < 50 && distMoved < 5) {
return;
}
const itemRect = item.getBoundingClientRect();
const bufferX = itemRect.width * 0.25;
const bufferY = itemRect.height * 0.25;
if (e.clientX < itemRect.left + bufferX || e.clientX > itemRect.right - bufferX ||
e.clientY < itemRect.top + bufferY || e.clientY > itemRect.bottom - bufferY) {
return;
}
const items = Array.from(grid.children);
const draggedIdx = items.indexOf(draggedNode);
const targetIdx = items.indexOf(item);
// Instantly snap the placeholder to its new position, moving items aside
if (draggedIdx < targetIdx) {
grid.insertBefore(draggedNode, item.nextSibling);
} else {
grid.insertBefore(draggedNode, item);
}
lastSwapX = e.clientX;
lastSwapY = e.clientY;
lastSwapTime = Date.now();
};
item.ondrop = (e) => {
e.preventDefault();
e.stopPropagation();
};
item.appendChild(img);
item.appendChild(del);
item.appendChild(numBadge);
grid.appendChild(item);
});
if (!isRearranging) {
requestAnimationFrame(() => {
updateLayout();
if (gridWrapper.offsetWidth > 0) optimizeGrid(gridWrapper.offsetWidth, gridWrapper.offsetHeight);
});
}
}
// --- 4. Logic: File Handling ---
async function handleFiles(files) {
const uploaded = [];
for (const file of files) {
const body = new FormData();
body.append("image", file);
try {
const resp = await api.fetchApi("/upload/image", { method: "POST", body });
if (resp.status === 200) {
const data = await resp.json();
let name = data.name;
if (data.subfolder) name = data.subfolder + "/" + name;
uploaded.push(name);
}
} catch (e) { console.error("Upload error", e); }
}
if (uploaded.length > 0) {
const current = (pathsWidget?.value || "").trim();
const allPaths = current ? current.split('\n').concat(uploaded) : uploaded;
setWidgetValue(allPaths, false);
}
}
// Apply drag & drop to LiteGraph node container bounds (V1)
const origOnDragDrop = node.onDragDrop;
node.onDragDrop = function(e) {
let handled = false;
if (e.dataTransfer && e.dataTransfer.files) {
const files = Array.from(e.dataTransfer.files).filter(f => f.type.startsWith('image/'));
if (files.length > 0) {
e.preventDefault();
handleFiles(files);
handled = true;
}
}
if (!handled && origOnDragDrop) {
return origOnDragDrop.apply(this, arguments);
}
return handled;
};
const origOnDragOver = node.onDragOver;
node.onDragOver = function(e) {
if (e.dataTransfer && e.dataTransfer.items) {
const hasImage = Array.from(e.dataTransfer.items).some(f => f.kind === 'file' && f.type.startsWith('image/'));
if (hasImage) {
e.preventDefault();
return true;
}
}
if (origOnDragOver) {
return origOnDragOver.apply(this, arguments);
}
return false;
};
uploadBtn.onclick = () => fileInput.click();
fileInput.onchange = (e) => handleFiles(e.target.files);
container.ondragover = (e) => {
e.preventDefault();
e.stopPropagation();
container.style.borderColor = "#4CAF50";
};
container.ondragleave = (e) => {
e.preventDefault();
e.stopPropagation();
container.style.borderColor = "#353545";
};
container.ondrop = (e) => {
e.preventDefault();
e.stopPropagation();
container.style.borderColor = "#353545";
if (e.dataTransfer.files.length > 0) handleFiles(e.dataTransfer.files);
};
// --- 5. Logic: Paste Handling ---
const pasteHandler = (e) => {
if (app.canvas.selected_nodes && app.canvas.selected_nodes[node.id]) {
const items = e.clipboardData?.items;
if (!items) return;
const files = [];
for (let i = 0; i < items.length; i++) {
if (items[i].kind === 'file' && items[i].type.startsWith('image/')) {
files.push(items[i].getAsFile());
}
}
if (files.length > 0) {
e.preventDefault();
e.stopImmediatePropagation();
handleFiles(files);
}
}
};
document.addEventListener("paste", pasteHandler, { capture: true });
const origOnRemoved = node.onRemoved;
node.onRemoved = function() {
document.removeEventListener("paste", pasteHandler, { capture: true });
resizeObserver.disconnect();
if (origOnRemoved) origOnRemoved.apply(this, arguments);
};
if (pathsWidget) {
pathsWidget.callback = (v) => {
if (oldCallback) oldCallback.apply(pathsWidget, [v]);
refreshGallery();
};
}
// Run immediately to trim blank outputs before LiteGraph does its first layout calculations
refreshGallery();
// Enforce the tightest possible packing explicitly upon being dropped into the graph (V1 specifically)
const origOnAdded = node.onAdded;
node.onAdded = function() {
if (origOnAdded) origOnAdded.apply(this, arguments);
const isV3 = checkIsV3();
if (!isV3) {
requestAnimationFrame(() => {
const galleryY = galleryWidget.last_y || 40;
const minOutputsHeight = (this.outputs ? this.outputs.length : 1) * 20;
const paddingBottom = 25; // Apply extra 20px pad on V1
const absoluteMinHeight = Math.max(galleryY + 250 + paddingBottom, minOutputsHeight + 40);
// Force the node to snap to its absolute minimum size on initial drop
if (this.size && this.size[1] > absoluteMinHeight + 5) {
this.setSize([this.size[0], absoluteMinHeight]);
if (app.graph) app.graph.setDirtyCanvas(true, true);
}
});
}
};
setTimeout(() => refreshGallery(), 100);
}
});

View File

@@ -1,336 +0,0 @@
import { app } from "../../scripts/app.js";
import { ComfyWidgets } from "../../scripts/widgets.js";
// 1. DEFINE CSS STYLES GLOBALLY ONCE
// We use DOM elements instead of Canvas drawing so it works flawlessly in ComfyUI V3
const cssStyles = `
.slc-ui-container {
width: 100%;
height: 100%;
min-height: 260px; /* Increased from 220px to prevent V1 cropping */
background-color: rgba(15, 15, 19, 0.7);
border: 1px solid rgba(255, 255, 255, 0.1);
border-radius: 8px;
padding: 12px;
box-sizing: border-box;
display: flex;
flex-direction: column;
font-family: sans-serif;
color: #ffffff;
overflow: hidden;
pointer-events: auto; /* allows text selection */
}
.slc-empty {
flex: 1;
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
}
.slc-empty-title { color: #aaaaaa; font-size: 14px; font-weight: 500; margin-bottom: 4px; }
.slc-empty-sub { color: #777777; font-size: 11px; }
.slc-headers {
display: flex;
gap: 8px;
margin-bottom: 12px;
}
.slc-header-block {
flex: 1;
background-color: rgba(0, 0, 0, 0.4);
border-radius: 6px;
padding: 8px 0;
display: flex;
flex-direction: column;
align-items: center;
}
.slc-header-title { color: #888888; font-size: 9px; font-weight: bold; margin-bottom: 4px; }
.slc-header-value { color: #ffffff; font-size: 16px; font-weight: bold; }
.slc-cards {
display: flex;
flex-direction: column;
gap: 8px;
}
.slc-card {
display: flex;
justify-content: space-between;
align-items: center;
background-color: rgba(0, 0, 0, 0.25);
border: 1px solid rgba(255, 255, 255, 0.05);
border-radius: 4px;
padding: 8px 12px 8px 16px;
position: relative;
overflow: hidden;
}
.slc-card::before {
content: '';
position: absolute;
left: 0;
top: 0;
bottom: 0;
width: 4px;
}
.slc-card.slow::before { background-color: #93c5fd; }
.slc-card.avg::before { background-color: #86efac; }
.slc-card.fast::before { background-color: #fca5a5; }
.slc-card-left { display: flex; flex-direction: column; }
.slc-card-speed { font-size: 12px; font-weight: bold; margin-bottom: 2px; }
.slc-card.slow .slc-card-speed { color: #93c5fd; }
.slc-card.avg .slc-card-speed { color: #86efac; }
.slc-card.fast .slc-card-speed { color: #fca5a5; }
.slc-card-wpm { font-size: 10px; color: #aaaaaa; }
.slc-card-right { display: flex; flex-direction: column; align-items: flex-end; }
.slc-card-time { font-size: 13px; font-weight: bold; color: #ffffff; margin-bottom: 2px; }
.slc-card-frames { font-size: 11px; font-family: monospace; color: #888888; }
.slc-legend {
margin-top: auto;
padding-top: 8px;
color: #999999;
font-size: 10px;
font-weight: 500;
}
`;
// Inject styles to document head safely
if (!document.getElementById("speech-length-calculator-styles")) {
const styleEl = document.createElement("style");
styleEl.id = "speech-length-calculator-styles";
styleEl.textContent = cssStyles;
document.head.appendChild(styleEl);
}
// 2. HELPER TO GENERATE THE HTML CONTENT
function buildUIHTML(statsData) {
if (!statsData || statsData.empty) {
return `
<div class="slc-empty">
<div class="slc-empty-title">Awaiting Script</div>
<div class="slc-empty-sub">Wrap spoken text inside "quotes"</div>
</div>
`;
}
return `
<div class="slc-headers">
<div class="slc-header-block">
<div class="slc-header-title">SPOKEN WORDS</div>
<div class="slc-header-value">${statsData.wordCount}</div>
</div>
<div class="slc-header-block">
<div class="slc-header-title">ADDED TIME</div>
<div class="slc-header-value">${statsData.additionalTime}s</div>
</div>
</div>
<div class="slc-cards">
<div class="slc-card slow">
<div class="slc-card-left">
<div class="slc-card-speed">SLOW</div>
<div class="slc-card-wpm">100 WPM</div>
</div>
<div class="slc-card-right">
<div class="slc-card-time">${statsData.slow.time}</div>
<div class="slc-card-frames">${statsData.slow.frames} frames</div>
</div>
</div>
<div class="slc-card avg">
<div class="slc-card-left">
<div class="slc-card-speed">AVG</div>
<div class="slc-card-wpm">130 WPM</div>
</div>
<div class="slc-card-right">
<div class="slc-card-time">${statsData.avg.time}</div>
<div class="slc-card-frames">${statsData.avg.frames} frames</div>
</div>
</div>
<div class="slc-card fast">
<div class="slc-card-left">
<div class="slc-card-speed">FAST</div>
<div class="slc-card-wpm">160 WPM</div>
</div>
<div class="slc-card-right">
<div class="slc-card-time">${statsData.fast.time}</div>
<div class="slc-card-frames">${statsData.fast.frames} frames</div>
</div>
</div>
</div>
<div class="slc-legend">WPM = Words Per Minute</div>
`;
}
app.registerExtension({
name: "Comfy.SpeechLengthCalculator",
async beforeRegisterNodeDef(nodeType, nodeData, app) {
if (nodeData.name === "SpeechLengthCalculator") {
const onNodeCreated = nodeType.prototype.onNodeCreated;
nodeType.prototype.onNodeCreated = function () {
const r = onNodeCreated ? onNodeCreated.apply(this, arguments) : undefined;
// 3. CREATE THE DOM WIDGET
const uiContainer = document.createElement("div");
uiContainer.className = "slc-ui-container";
uiContainer.innerHTML = buildUIHTML({ empty: true });
// Attach to node: standard addDOMWidget works natively in both V1 and V3 Frontends
let statsWidget;
if (typeof this.addDOMWidget === "function") {
statsWidget = this.addDOMWidget("Stats", "HTML", uiContainer, {
serialize: false,
hideOnZoom: false
});
} else {
// Fallback for very old unpatched V1 installations
statsWidget = ComfyWidgets["STRING"](this, "Stats", ["STRING", { multiline: true }], app).widget;
if (statsWidget.inputEl) {
statsWidget.inputEl.style.display = "none";
if (statsWidget.inputEl.parentNode) {
statsWidget.inputEl.parentNode.appendChild(uiContainer);
}
}
}
// Tell LiteGraph exactly how much vertical space to hold for our UI
statsWidget.computeSize = function() {
return [0, 280]; // Increased from 240 to prevent bottom cropping in V1
};
// Ensure the node is wide enough on initial creation
requestAnimationFrame(() => {
const minWidth = 340;
if (this.size[0] < minWidth) this.size[0] = minWidth;
// Fine tune initial height to account for the larger computeSize and make text box taller
if (this.size[1] < 500) this.size[1] = 500;
this.setDirtyCanvas(true, true);
});
// Fetch text from input link OR widget
this._getCurrentText = () => {
const inputSlot = this.inputs && this.inputs.find(i => i.name === "text_input" || (i.name === "text" && i.link));
if (inputSlot && inputSlot.link) {
const link = app.graph.links[inputSlot.link];
if (link) {
const sourceNode = app.graph.getNodeById(link.origin_id);
if (sourceNode && sourceNode.widgets) {
const w = sourceNode.widgets.find(w => w.name === "value" || w.name === "text" || w.name === "Text" || w.type === "customtext" || w.type === "STRING");
if (w && typeof w.value === "string") return w.value;
}
}
}
const textWidget = this.widgets && this.widgets.find(w => w.name === "text");
return textWidget ? (textWidget.value || "") : "";
};
this._lastState = { text: null, fps: null, addTime: null };
const updateStats = () => {
const fpsWidget = this.widgets && this.widgets.find(w => w.name === "fps");
const additionalTimeWidget = this.widgets && this.widgets.find(w => w.name === "additional_time");
if (!fpsWidget) return;
const text = this._getCurrentText();
const fps = fpsWidget.value || 24;
const additionalTime = additionalTimeWidget ? parseFloat(additionalTimeWidget.value) || 0 : 0;
// Skip expensive calculations if nothing changed
if (this._lastState.text === text &&
this._lastState.fps === fps &&
this._lastState.addTime === additionalTime) {
return;
}
this._lastState.text = text;
this._lastState.fps = fps;
this._lastState.addTime = additionalTime;
const regex = /"([^"]*)"|'([^']*)'|“([^”]*)”|([^]*)/g;
let match;
let quotedText = "";
while ((match = regex.exec(text)) !== null) {
quotedText += (match[1] || match[2] || match[3] || match[4] || "") + " ";
}
const words = quotedText.trim().split(/\s+/).filter(w => w.length > 0);
const wordCount = words.length;
const formatTime = (wpm) => {
const baseMins = wordCount / wpm;
const totalSecs = (baseMins * 60) + additionalTime;
const mins = Math.floor(totalSecs / 60);
let secs = totalSecs % 60;
secs = Math.ceil(secs * 10) / 10;
const frames = Math.ceil(totalSecs * fps);
const secsStr = secs.toFixed(1);
const timeStr = mins > 0 ? `${mins}m ${secsStr}s` : `${secsStr}s`;
return {
time: timeStr,
frames: frames.toString()
};
};
const statsData = {
empty: (wordCount === 0 && additionalTime === 0),
wordCount: wordCount,
additionalTime: additionalTime,
slow: formatTime(100),
avg: formatTime(130),
fast: formatTime(160)
};
// 4. INJECT HTML TO THE DOM
// Replaces the heavy canvas redrawing!
uiContainer.innerHTML = buildUIHTML(statsData);
this.setDirtyCanvas(true, false);
};
// We still use onDrawBackground as an efficient silent trigger
// to auto-update in case upstream nodes silently change their text
const onDrawBackground = this.onDrawBackground;
this.onDrawBackground = function(ctx) {
if (onDrawBackground) onDrawBackground.apply(this, arguments);
updateStats();
};
// Bind events to update in real time based on node interactions
setTimeout(() => {
const textWidget = this.widgets && this.widgets.find(w => w.name === "text");
const fpsWidget = this.widgets && this.widgets.find(w => w.name === "fps");
const additionalTimeWidget = this.widgets && this.widgets.find(w => w.name === "additional_time");
if (textWidget) {
const origCallback = textWidget.callback;
textWidget.callback = function() {
if (origCallback) origCallback.apply(this, arguments);
updateStats();
}
}
if (fpsWidget) {
const origFpsCallback = fpsWidget.callback;
fpsWidget.callback = function() {
if (origFpsCallback) origFpsCallback.apply(this, arguments);
updateStats();
}
}
if (additionalTimeWidget) {
const origAddCallback = additionalTimeWidget.callback;
additionalTimeWidget.callback = function() {
if (origAddCallback) origAddCallback.apply(this, arguments);
updateStats();
}
}
updateStats();
}, 100);
return r;
};
}
}
});

View File

@@ -16,7 +16,7 @@ class CleanLatentSlice:
RETURN_TYPES = ("LATENT",) RETURN_TYPES = ("LATENT",)
RETURN_NAMES = ("latent",) RETURN_NAMES = ("latent",)
FUNCTION = "slice_latent" FUNCTION = "slice_latent"
CATEGORY = "WhatDreamsCost" CATEGORY = "WhatDreamsCost CS"
DESCRIPTION = "Safely slices a video latent starting from an offset index for a specific length. Uses torch.narrow to bypass PyTorch NestedTensor slicing bugs." DESCRIPTION = "Safely slices a video latent starting from an offset index for a specific length. Uses torch.narrow to bypass PyTorch NestedTensor slicing bugs."
def slice_latent(self, latent, start, length): def slice_latent(self, latent, start, length):
@@ -65,11 +65,11 @@ class CleanLatentSlice:
# Register the node with ComfyUI # Register the node with ComfyUI
NODE_CLASS_MAPPINGS = { NODE_CLASS_MAPPINGS = {
"CleanLatentSlice": CleanLatentSlice "CleanLatentSliceCS": CleanLatentSlice
} }
NODE_DISPLAY_NAME_MAPPINGS = { NODE_DISPLAY_NAME_MAPPINGS = {
"CleanLatentSlice": "Clean Latent Slice" "CleanLatentSliceCS": "Clean Latent Slice CS"
} }
__all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS'] __all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS']

View File

@@ -71,7 +71,7 @@ class LoadAudioUI:
} }
} }
CATEGORY = "WhatDreamsCost" CATEGORY = "WhatDreamsCost CS"
RETURN_TYPES = ("AUDIO", "FLOAT") RETURN_TYPES = ("AUDIO", "FLOAT")
RETURN_NAMES = ("audio", "duration") RETURN_NAMES = ("audio", "duration")
FUNCTION = "load_audio" FUNCTION = "load_audio"

View File

@@ -1,412 +0,0 @@
import os
import torch
import numpy as np
import folder_paths
import av
from server import PromptServer
from aiohttp import web
import comfy.utils
# Custom API route to serve video files from anywhere on the user's system for the frontend preview
@PromptServer.instance.routes.get("/video_ui_custom_view")
async def custom_view(request):
file_path = request.query.get("filename", "")
if os.path.exists(file_path) and os.path.isfile(file_path):
return web.FileResponse(file_path)
return web.Response(status=404, text="File not found")
# Custom API route for Chunked Uploads to bypass the 413 Payload Too Large error
@PromptServer.instance.routes.post("/video_ui_upload_chunk")
async def upload_chunk(request):
post = await request.post()
file = post.get("file")
filename = post.get("filename")
chunk_index = int(post.get("chunk_index"))
total_chunks = int(post.get("total_chunks"))
upload_dir = folder_paths.get_input_directory()
file_path = os.path.join(upload_dir, filename)
# Append to file if it's not the first chunk, otherwise write new
mode = "ab" if chunk_index > 0 else "wb"
with open(file_path, mode) as f:
f.write(file.file.read())
if chunk_index == total_chunks - 1:
return web.json_response({"name": filename})
return web.json_response({"status": "ok"})
class LoadVideoUI:
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"video": ("STRING", {"default": ""}),
"start_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
"end_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
"duration": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
"start_frame": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}),
"end_frame": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}),
"duration_frames": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}),
"resize_method": (["maintain aspect ratio", "stretch to fit", "pad", "crop"], {"default": "maintain aspect ratio"}),
"custom_width": ("INT", {"default": 0, "min": 0, "max": 100000, "step": 8, "tooltip": "Custom width. 0 means original width."}),
"custom_height": ("INT", {"default": 0, "min": 0, "max": 100000, "step": 8, "tooltip": "Custom height. 0 means original height."}),
"frame_rate": ("INT", {"default": 24, "min": 1, "max": 120, "step": 1, "tooltip": "Force the video to a specific frame rate for extraction."}),
"display_mode": (["seconds", "frames"], {"default": "seconds"}),
"crop_x": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}),
"crop_y": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}),
"crop_w": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}),
"crop_h": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}),
}
}
RETURN_TYPES = ("IMAGE", "AUDIO", "FLOAT", "INT")
RETURN_NAMES = ("images", "audio", "duration", "frame_count")
FUNCTION = "load_video"
CATEGORY = "WhatDreamsCost"
def load_video(self, video, frame_rate, display_mode, start_time, end_time, duration, start_frame, end_frame, duration_frames, custom_width=0, custom_height=0, resize_method="maintain aspect ratio", crop_x=0.0, crop_y=0.0, crop_w=1.0, crop_h=1.0, **kwargs):
if not video:
# Return blank defaults if no video is loaded
empty_image = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
empty_audio = {"waveform": torch.zeros((1, 1, 44100)), "sample_rate": 44100}
return (empty_image, empty_audio, 0.0, 0)
# 1. Resolve path using ComfyUI standard paths or Absolute Path
video_path = video # Try exact/absolute path first
if not os.path.exists(video_path):
video_path_annotated = folder_paths.get_annotated_filepath(video)
if os.path.exists(video_path_annotated):
video_path = video_path_annotated
else:
video_path_input = os.path.join(folder_paths.get_input_directory(), video)
if os.path.exists(video_path_input):
video_path = video_path_input
else:
raise FileNotFoundError(f"Video file not found: {video}")
# Open container to read streams and metadata
container = av.open(video_path)
# Determine video stream and duration
video_stream = container.streams.video[0] if len(container.streams.video) > 0 else None
video_duration = 0
if video_stream and video_stream.duration and video_stream.time_base:
video_duration = float(video_stream.duration * video_stream.time_base)
orig_w = video_stream.codec_context.width if video_stream else 512
orig_h = video_stream.codec_context.height if video_stream else 512
# Determine correct colorspace and color range for PyAV conversion to prevent color shift
try:
from av.video.reformatter import Colorspace, ColorRange
# Improve fallback heuristic to check both dimensions (e.g. 720x1280 vertical video is HD)
fallback_cs = Colorspace.ITU709 if max(orig_w, orig_h) >= 720 else Colorspace.ITU601
fallback_cr = ColorRange.MPEG
dst_range = ColorRange.JPEG # RGB should always be full range
except ImportError:
fallback_cs = "itu709" if max(orig_w, orig_h) >= 720 else "itu601"
fallback_cr = "mpeg"
dst_range = "jpeg"
src_colorspace = fallback_cs
src_color_range = fallback_cr
if video_stream and video_stream.codec_context:
cc = video_stream.codec_context
c_space = getattr(cc, 'colorspace', getattr(cc, 'color_space', None))
if c_space and hasattr(c_space, 'name') and c_space.name != "UNSPECIFIED":
src_colorspace = c_space
elif c_space and isinstance(c_space, str) and "unspecified" not in c_space.lower():
src_colorspace = c_space
c_range = getattr(cc, 'color_range', None)
if c_range and hasattr(c_range, 'name') and c_range.name != "UNSPECIFIED":
src_color_range = c_range
elif c_range and isinstance(c_range, str) and "unspecified" not in c_range.lower():
src_color_range = c_range
target_w = custom_width if custom_width > 0 else orig_w
target_h = custom_height if custom_height > 0 else orig_h
target_w = target_w - (target_w % 2)
target_h = target_h - (target_h % 2)
# Calculate manual crop from interactive UI first
manual_crop_left = int(orig_w * crop_x)
manual_crop_top = int(orig_h * crop_y)
manual_crop_right = orig_w - int(orig_w * (crop_x + crop_w))
manual_crop_bottom = orig_h - int(orig_h * (crop_y + crop_h))
# Ensure we don't crop more than the image
manual_crop_left = max(0, min(manual_crop_left, orig_w - 1))
manual_crop_top = max(0, min(manual_crop_top, orig_h - 1))
manual_crop_right = max(0, min(manual_crop_right, orig_w - manual_crop_left - 1))
manual_crop_bottom = max(0, min(manual_crop_bottom, orig_h - manual_crop_top - 1))
# After manual crop, the new original dimensions are:
cropped_orig_w = orig_w - manual_crop_left - manual_crop_right
cropped_orig_h = orig_h - manual_crop_top - manual_crop_bottom
# If no custom width/height is provided, use the cropped original dimensions
if custom_width == 0:
target_w = cropped_orig_w
target_w = target_w - (target_w % 2)
if custom_height == 0:
target_h = cropped_orig_h
target_h = target_h - (target_h % 2)
scale_w, scale_h = target_w, target_h
pad_left = pad_right = pad_top = pad_bottom = 0
crop_left = crop_right = crop_top = crop_bottom = 0
if custom_width > 0 or custom_height > 0:
if resize_method == "maintain aspect ratio" or resize_method == "pad":
ratio = min(target_w / cropped_orig_w, target_h / cropped_orig_h)
scale_w = int(cropped_orig_w * ratio)
scale_h = int(cropped_orig_h * ratio)
scale_w = scale_w - (scale_w % 2)
scale_h = scale_h - (scale_h % 2)
if resize_method == "pad":
pad_x = target_w - scale_w
pad_y = target_h - scale_h
pad_left = pad_x // 2
pad_right = pad_x - pad_left
pad_top = pad_y // 2
pad_bottom = pad_y - pad_top
else:
target_w, target_h = scale_w, scale_h
elif resize_method == "crop":
ratio = max(target_w / cropped_orig_w, target_h / cropped_orig_h)
scale_w = int(cropped_orig_w * ratio)
scale_h = int(cropped_orig_h * ratio)
scale_w = scale_w - (scale_w % 2)
scale_h = scale_h - (scale_h % 2)
crop_x = scale_w - target_w
crop_y = scale_h - target_h
crop_left = crop_x // 2
crop_right = crop_x - crop_left
crop_top = crop_y // 2
crop_bottom = crop_y - crop_top
elif resize_method == "stretch to fit":
scale_w, scale_h = target_w, target_h
# Determine exact bounds based on frontend mode
if display_mode == "frames":
fr = float(frame_rate) if frame_rate > 0 else 24.0
actual_start_time = float(start_frame) / fr
actual_end_time = float(end_frame) / fr if (end_frame > 0 and end_frame > start_frame) else video_duration
else:
actual_start_time = start_time
actual_end_time = end_time if (end_time > 0 and end_time > start_time) else video_duration
if actual_end_time <= 0:
actual_end_time = float('inf') # Fallback if duration is unknown
# 2. Extract Video Frames (PyAV)
frames = []
image_tensor = None
frames_loaded = 0
if video_stream:
video_stream.thread_type = "AUTO" # Enable multithreaded decoding
# Efficiently seek backwards to the nearest keyframe
if video_stream.time_base:
seek_pts = int(actual_start_time / float(video_stream.time_base))
else:
seek_pts = int(actual_start_time * av.time_base)
container.seek(seek_pts, stream=video_stream, backward=True)
# Custom sampling to force specific framerate
frame_interval = 1.0 / float(frame_rate) if frame_rate > 0 else 1.0/24.0
expected_target_time = actual_start_time
# Pre-calculate expected frames
alloc_end_time = actual_end_time if actual_end_time != float('inf') else video_duration
expected_frames = 0
if alloc_end_time > 0:
duration_to_extract = alloc_end_time - actual_start_time
if duration_to_extract > 0:
expected_frames = int(np.ceil(duration_to_extract / frame_interval)) + 2
pbar = comfy.utils.ProgressBar(expected_frames) if expected_frames > 0 else None
for frame in container.decode(video_stream):
frame_time = frame.time
if frame_time is None:
frame_time = float(frame.pts * float(video_stream.time_base)) if frame.pts and video_stream.time_base else 0.0
if frame_time < actual_start_time:
continue
# Add a slight buffer (interval) to ensure we evaluate the boundary correctly
if frame_time > actual_end_time + frame_interval:
break
# Fix PyAV color shift by forcing proper colorspace and range conversion.
# Omit dst_colorspace so swscale defaults naturally for RGB output
# (passing it can cause the YUV matrix to be applied incorrectly).
try:
frame = frame.reformat(
format="rgb24",
src_colorspace=src_colorspace,
src_color_range=src_color_range,
dst_color_range=dst_range
)
frame_rgb = frame.to_ndarray(format='rgb24')
except Exception as e:
# Fallback: if explicit color reformat fails, use PyAV's default conversion
print(f"[LoadVideoUI] Color reformat failed, using default: {e}")
frame_rgb = frame.to_ndarray(format='rgb24')
# Apply interactive crop first
if manual_crop_left > 0 or manual_crop_top > 0 or manual_crop_right > 0 or manual_crop_bottom > 0:
frame_rgb = frame_rgb[manual_crop_top:orig_h-manual_crop_bottom, manual_crop_left:orig_w-manual_crop_right, :]
# Now resize to the scaled dimensions
if scale_w != cropped_orig_w or scale_h != cropped_orig_h:
import cv2
frame_rgb = cv2.resize(frame_rgb, (scale_w, scale_h), interpolation=cv2.INTER_AREA)
if crop_left > 0 or crop_top > 0 or crop_right > 0 or crop_bottom > 0:
frame_rgb = frame_rgb[crop_top:scale_h-crop_bottom, crop_left:scale_w-crop_right, :]
if pad_left > 0 or pad_top > 0 or pad_right > 0 or pad_bottom > 0:
frame_rgb = np.pad(frame_rgb, ((pad_top, pad_bottom), (pad_left, pad_right), (0, 0)), mode='constant', constant_values=0)
# Duplicate or skip frames perfectly based on timestamps to meet forced framerate.
# FIX: Use strictly less than (<) for actual_end_time to prevent the loop from fetching an extra +1 frame
# at the exact boundary of the duration slice!
while expected_target_time <= frame_time and expected_target_time < actual_end_time - 1e-5:
if image_tensor is None and expected_frames > 0:
# First frame: allocate the tensor
height, width = frame_rgb.shape[:2]
alloc_frames = expected_frames + 50 # Add generous buffer to prevent reallocation
try:
image_tensor = torch.zeros((alloc_frames, height, width, 3), dtype=torch.float32)
except Exception as e:
print(f"[LoadVideoUI] Pre-allocation failed, falling back to list: {e}")
expected_frames = 0 # Disable pre-allocation
if image_tensor is not None:
# Check bounds (just in case)
if frames_loaded >= image_tensor.shape[0]:
# Extend tensor if we underestimated
extension = torch.zeros((50, image_tensor.shape[1], image_tensor.shape[2], 3), dtype=torch.float32)
image_tensor = torch.cat((image_tensor, extension), dim=0)
# Insert frame with minimal memory copy directly to tensor
image_tensor[frames_loaded] = torch.from_numpy(frame_rgb).float().div_(255.0)
frames_loaded += 1
else:
# Fallback list append if pre-allocation failed
frames.append(frame_rgb)
if pbar:
pbar.update(1)
expected_target_time += frame_interval
# Convert frames to ComfyUI Image standard format [N, H, W, C], float32, range 0.0-1.0
if image_tensor is not None:
if frames_loaded > 0:
image_tensor = image_tensor[:frames_loaded]
else:
image_tensor = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
elif len(frames) > 0:
frames_np = np.array(frames, dtype=np.float32) / 255.0
image_tensor = torch.from_numpy(frames_np)
else:
# Fallback for an empty slice
image_tensor = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
# 3. Extract Audio (PyAV)
audio_dict = {"waveform": torch.zeros((1, 1, 44100)), "sample_rate": 44100} # Default empty audio
if len(container.streams.audio) > 0:
try:
audio_stream = container.streams.audio[0]
audio_stream.thread_type = "AUTO"
sample_rate = getattr(audio_stream, 'rate', 44100) or 44100
# We must seek again on the container specifically for the audio stream
if audio_stream.time_base:
seek_pts = int(actual_start_time / float(audio_stream.time_base))
else:
seek_pts = int(actual_start_time * av.time_base)
container.seek(seek_pts, stream=audio_stream, backward=True)
# Resample to standard float planar format (fltp)
resampler = av.AudioResampler(format='fltp')
audio_data = []
first_frame_time = None
for frame in container.decode(audio_stream):
frame_time = frame.time
if frame_time is None:
frame_time = float(frame.pts * float(audio_stream.time_base)) if frame.pts and audio_stream.time_base else 0.0
# Give a small 1-second buffer to ensure we catch end frames
if frame_time > actual_end_time + 1.0:
break
if first_frame_time is None:
first_frame_time = frame_time
resampled_frames = resampler.resample(frame)
for r_frame in resampled_frames:
audio_data.append(r_frame.to_ndarray())
if audio_data:
# Concatenate all frames horizontally along the sample axis
waveform_np = np.concatenate(audio_data, axis=1)
waveform = torch.from_numpy(waveform_np).float()
if first_frame_time is None:
first_frame_time = 0.0
# Calculate exact slice points to trim precisely
offset_sec = max(0.0, actual_start_time - first_frame_time)
start_sample = int(offset_sec * sample_rate)
duration_sec_audio = actual_end_time - actual_start_time
end_sample = start_sample + int(duration_sec_audio * sample_rate)
# Trim array bounds properly
if end_sample > start_sample:
waveform = waveform[:, start_sample:end_sample]
else:
waveform = waveform[:, start_sample:]
# Expand to ComfyUI Audio standard [batch_size, channels, samples]
waveform = waveform.unsqueeze(0)
audio_dict = {"waveform": waveform, "sample_rate": sample_rate}
except Exception as e:
# Catch gracefully without breaking the pipeline execution
print(f"[LoadVideoUI] Audio track extraction skipped or failed: {e}")
# Always close container to free up system memory lock
container.close()
# Output accurate final duration in seconds
final_duration_sec = float(max(0.0, actual_end_time - actual_start_time))
# Accurately output the true number of extracted frames
# (Using the shape of the array provides exact 1:1 parity with the timeline's math)
frame_count = image_tensor.shape[0] if (frames_loaded > 0 or len(frames) > 0) else 0
if frame_count == 0 and final_duration_sec > 0:
# Fallback estimation only if PyAV completely failed to decode a valid chunk
calc_fr = float(frame_rate) if frame_rate > 0 else 24.0
frame_count = int(np.floor(final_duration_sec * calc_fr))
return (image_tensor, audio_dict, final_duration_sec, frame_count)

File diff suppressed because it is too large Load Diff

View File

@@ -1,106 +1,637 @@
from comfy_extras.nodes_lt import LTXVAddGuide import logging
import math
import os
import av
import numpy as np
import torch import torch
import comfy
import comfy.sd
import comfy.utils import comfy.utils
import folder_paths
import node_helpers
from comfy_extras import nodes_lt
from comfy_api.latest import io from comfy_api.latest import io
from .ltx_director import GuideData from .ltx_director import GuideData, MotionGuideData, _resize_image, _execute_comfy_node, _unpack
log = logging.getLogger(__name__)
class LTXDirectorGuide(LTXVAddGuide): # --- Helper Functions from Nghtdrp ---
@classmethod
def define_schema(cls): def _get_guide_attention_entries(conditioning):
return io.Schema( for item in conditioning:
node_id="LTXDirectorGuide", entries = item[1].get("guide_attention_entries", None)
display_name="LTX Director Guide", if entries is not None:
category="WhatDreamsCost", return entries
description=( return []
"Applies guide images from a Prompt Relay Timeline node at the frame positions "
"and strengths defined on the timeline. Connect guide_data from the timeline node." def _set_guide_attention_entries(conditioning, entries):
), return node_helpers.conditioning_set_values(
inputs=[ conditioning, {"guide_attention_entries": entries}
io.Conditioning.Input("positive", tooltip="Positive conditioning to add guide keyframe info to."), )
io.Conditioning.Input("negative", tooltip="Negative conditioning to add guide keyframe info to."),
io.Vae.Input("vae", tooltip="Video VAE used to encode the guide images."), def _append_guide_attention_entry(
io.Latent.Input("latent", tooltip="Video latent — guides are inserted into this latent."), conditioning,
GuideData.Input("guide_data", tooltip="Guide data produced by Prompt Relay Encode (Timeline)."), pre_filter_count,
io.Float.Input("scale_by", default=1.0, min=0.01, max=8.0, step=0.01, tooltip="Scale the latent by this factor."), latent_shape,
io.Combo.Input("upscale_method", options=["nearest-exact", "bilinear", "area", "bicubic", "bislerp"], default="bicubic", tooltip="Method used to upscale/downscale the latent."), attention_strength=1.0,
io.Float.Input("msr_strength", default=0.0, min=0.0, max=1.0, step=0.05, tooltip="Licon MSR only: per-stage reference strength override. 0 = use the Director's value (full pull, right for stage 1). On a refinement/upscale stage set ~0.4 to hold detail without the references repainting the opening (fixes stage-2 mist/ghosting)."), attention_mask=None,
], ):
outputs=[ entries = [*_get_guide_attention_entries(conditioning)]
io.Conditioning.Output(display_name="positive"), entries.append(
io.Conditioning.Output(display_name="negative"), {
io.Latent.Output(display_name="latent", tooltip="Video latent with guide frames applied."), "pre_filter_count": int(pre_filter_count),
], "strength": float(attention_strength),
"pixel_mask": attention_mask,
"latent_shape": list(latent_shape),
}
)
return _set_guide_attention_entries(conditioning, entries)
def _clone_noise_mask(latent, latent_image):
if "noise_mask" in latent and latent["noise_mask"] is not None:
return latent["noise_mask"].clone()
batch, _, frames, _, _ = latent_image.shape
return torch.ones(
(batch, 1, frames, 1, 1),
dtype=torch.float32,
device=latent_image.device,
)
def _resize_latent_spatial(latent_image, noise_mask, width, height, method):
b, c, f, h, w = latent_image.shape
if width == w and height == h:
return latent_image, noise_mask
latent_4d = latent_image.permute(0, 2, 1, 3, 4).reshape(b * f, c, h, w)
latent_4d = comfy.utils.common_upscale(latent_4d, width, height, method, "disabled")
latent_image = latent_4d.reshape(b, f, c, height, width).permute(0, 2, 1, 3, 4)
if noise_mask is not None and (noise_mask.shape[-1] > 1 or noise_mask.shape[-2] > 1):
mb, mc, mf, mh, mw = noise_mask.shape
mask_4d = noise_mask.permute(0, 2, 1, 3, 4).reshape(mb * mf, mc, mh, mw)
mask_4d = comfy.utils.common_upscale(mask_4d, width, height, method, "disabled")
noise_mask = mask_4d.reshape(mb, mf, mc, height, width).permute(0, 2, 1, 3, 4)
return latent_image, noise_mask
def _ceil_to_multiple(value, multiple):
multiple = max(1, int(multiple))
return int(math.ceil(value / multiple) * multiple)
def _snap_latent_to_downscale(latent_image, noise_mask, downscale_factor, method):
factor = int(max(1, round(float(downscale_factor))))
if factor <= 1:
return latent_image, noise_mask
_, _, _, h, w = latent_image.shape
new_w = _ceil_to_multiple(w, factor)
new_h = _ceil_to_multiple(h, factor)
if new_w == w and new_h == h:
return latent_image, noise_mask
log.warning(
"[LTXDirectorGuide] Auto-snapping latent grid from %sx%s to %sx%s for IC-LoRA downscale factor %s.",
w, h, new_w, new_h, factor,
)
return _resize_latent_spatial(latent_image, noise_mask, new_w, new_h, method)
def _load_lora_model_only(model, ic_lora_name, strength_model):
lora_path = folder_paths.get_full_path_or_raise("loras", ic_lora_name)
lora, metadata = comfy.utils.load_torch_file(
lora_path, safe_load=True, return_metadata=True,
)
try:
latent_downscale_factor = float(metadata["reference_downscale_factor"])
except Exception:
latent_downscale_factor = 1.0
log.warning(
"[LTXDirectorGuide] Could not read reference_downscale_factor from %s, using 1.0.",
ic_lora_name,
) )
@classmethod if strength_model == 0:
def execute(cls, positive, negative, vae, latent, guide_data, scale_by=1.0, upscale_method="bicubic", msr_strength=0.0) -> io.NodeOutput: return model, latent_downscale_factor
scale_factors = vae.downscale_index_formula
# Clone latents to avoid mutating upstream nodes model_lora, _ = comfy.sd.load_lora_for_models(
latent_image = latent["samples"].clone() model, None, lora, strength_model, 0,
)
return model_lora, latent_downscale_factor
if "noise_mask" in latent: def _encode_video_iclora_guide(vae, latent_width, latent_height, images, scale_factors, latent_downscale_factor, crop, use_tiled_encode, tile_size, tile_overlap, resize_method="crop"):
noise_mask = latent["noise_mask"].clone() time_scale_factor, width_scale_factor, height_scale_factor = scale_factors
num_frames_to_keep = ((images.shape[0] - 1) // time_scale_factor) * time_scale_factor + 1
images = images[:num_frames_to_keep]
target_width = int(latent_width * width_scale_factor / latent_downscale_factor)
target_height = int(latent_height * height_scale_factor / latent_downscale_factor)
target_width = max(8, target_width)
target_height = max(8, target_height)
# For guides, we must match the exact target latent shape.
# "maintain aspect ratio" doesn't pad, which causes shape mismatch in VAE encode / concatenation.
# So we fallback "maintain aspect ratio" to "pad".
if resize_method == "maintain aspect ratio":
resize_method = "pad"
pixels = _resize_image(images, target_width, target_height, resize_method, divisible_by=1)
encode_pixels = pixels[:, :, :, :3]
if use_tiled_encode:
guide_latent = vae.encode_tiled(encode_pixels, tile_x=tile_size, tile_y=tile_size, overlap=tile_overlap)
else:
guide_latent = vae.encode(encode_pixels)
return pixels, guide_latent
def _dilate_latent(latent: dict, horizontal_scale: int, vertical_scale: int) -> dict:
if horizontal_scale == 1 and vertical_scale == 1:
return latent
samples = latent["samples"]
mask = latent.get("noise_mask", None)
dilated_shape = samples.shape[:3] + (
samples.shape[3] * vertical_scale,
samples.shape[4] * horizontal_scale,
)
dilated_samples = torch.zeros(dilated_shape, device=samples.device, dtype=samples.dtype, requires_grad=False)
dilated_samples[..., ::vertical_scale, ::horizontal_scale] = samples
dilated_mask_shape = (
dilated_samples.shape[0], 1, dilated_samples.shape[2],
dilated_samples.shape[3], dilated_samples.shape[4],
)
dilated_mask = torch.full(dilated_mask_shape, -1.0, device=samples.device, dtype=samples.dtype, requires_grad=False)
dilated_mask[..., ::vertical_scale, ::horizontal_scale] = (mask if mask is not None else 1.0)
return {"samples": dilated_samples, "noise_mask": dilated_mask}
def _resolve_input_video_path(video_file):
if os.path.isabs(str(video_file)) and os.path.exists(str(video_file)):
return str(video_file)
input_dir = folder_paths.get_input_directory()
candidate = os.path.join(input_dir, str(video_file))
if os.path.exists(candidate):
return candidate
try:
annotated = folder_paths.get_annotated_filepath(str(video_file))
if annotated and os.path.exists(annotated):
return annotated
except Exception:
pass
raise FileNotFoundError(f"Could not find motion guide video: {video_file}")
class ResampleGuideFrames:
def execute(self, images, source_fps, target_fps, target_num_frames, mode):
if images is None: return images
frames = images
n = int(frames.shape[0])
target_num_frames = int(target_num_frames)
if n <= 1:
if target_num_frames > 1 and n == 1:
return frames.repeat(target_num_frames, 1, 1, 1)
return frames
source_fps = float(max(0.001, source_fps))
target_fps = float(max(0.001, target_fps))
if target_num_frames <= 0:
duration = (n - 1) / source_fps
target_num_frames = max(1, int(round(duration * target_fps)) + 1)
if target_num_frames == n and abs(target_fps - source_fps) < 1e-6:
return frames
positions = torch.linspace(0, n - 1, target_num_frames, device=frames.device, dtype=torch.float32)
if mode == "nearest":
idx = torch.round(positions).long().clamp(0, n - 1)
return frames.index_select(0, idx)
idx0 = torch.floor(positions).long().clamp(0, n - 1)
idx1 = torch.ceil(positions).long().clamp(0, n - 1)
alpha = (positions - idx0.to(positions.dtype)).view(-1, 1, 1, 1)
f0 = frames.index_select(0, idx0).to(torch.float32)
f1 = frames.index_select(0, idx1).to(torch.float32)
return (f0 * (1.0 - alpha) + f1 * alpha).to(frames.dtype)
def _load_motion_video_frames(video_file, trim_start_frames, length_frames, director_fps, resample_mode="nearest"):
path = _resolve_input_video_path(video_file)
target_fps = max(1.0, float(director_fps))
start_s = max(0.0, float(trim_start_frames) / target_fps)
dur_s = max(0.0, float(length_frames) / target_fps)
end_s = start_s + dur_s if dur_s > 0 else None
log.info(f"[LTXDirectorGuide] Loading video frames from {path}. start_s: {start_s:.3f}, dur_s: {dur_s:.3f}, end_s: {end_s}")
container = av.open(path)
stream = container.streams.video[0]
stream.thread_type = "AUTO" # Enable multi-threaded decoding
try:
source_fps = float(stream.average_rate) if stream.average_rate else float(stream.base_rate)
except Exception:
source_fps = target_fps
if source_fps <= 0: source_fps = target_fps
# Seek to keyframe slightly before target start_s to optimize loading speed and memory
if start_s > 0:
try:
if stream.time_base:
seek_pts = int(max(0, start_s - 0.5) / float(stream.time_base))
else:
seek_pts = int(max(0, start_s - 0.5) * av.time_base)
container.seek(seek_pts, stream=stream, backward=True)
log.info(f"[LTXDirectorGuide] PyAV seeked stream to pts={seek_pts} (approx {max(0, start_s - 0.5):.3f}s)")
except Exception as seek_err:
log.warning(f"[LTXDirectorGuide] Seek failed: {seek_err}, decoding from beginning.")
frames = []
decoded_count = 0
for frame in container.decode(stream):
if frame.time is not None:
t = float(frame.time)
elif frame.pts is not None and stream.time_base is not None:
t = float(frame.pts * stream.time_base)
else: else:
batch, _, latent_frames, latent_height, latent_width = latent_image.shape t = float(decoded_count / source_fps)
noise_mask = torch.ones( decoded_count += 1
(batch, 1, latent_frames, 1, 1),
dtype=torch.float32, if t < start_s - 0.01: continue
device=latent_image.device, if end_s is not None and t >= end_s: break
)
# Append raw uint8 numpy arrays to minimize CPU allocation overhead
frames.append(frame.to_ndarray(format="rgb24"))
container.close()
if not frames: raise ValueError(f"No frames decoded for motion guide segment: {video_file}")
# Convert all frames to float32 at once to optimize memory allocation
frames_np = np.array(frames, dtype=np.float32) / 255.0
images = torch.from_numpy(frames_np)
target_count = max(1, int(round(float(length_frames))))
images = ResampleGuideFrames().execute(images, source_fps, target_fps, target_count, resample_mode)
return images
# --- Main Class ---
class LTXDirectorGuide:
@classmethod
def INPUT_TYPES(cls):
loras = folder_paths.get_filename_list("loras")
if not loras: loras = ["put_ic_lora_in_ComfyUI_models_loras"]
return {
"required": {
"positive": ("CONDITIONING", {"tooltip": "Positive conditioning to add guide keyframe info to."}),
"negative": ("CONDITIONING", {"tooltip": "Negative conditioning to add guide keyframe info to."}),
"vae": ("VAE", {"tooltip": "Video VAE used to encode the guide images."}),
"latent": ("LATENT", {"tooltip": "Video latent — guides are inserted into this latent."}),
"guide_data": ("GUIDE_DATA", {"tooltip": "Guide data produced by Prompt Relay Encode (Timeline)."}),
},
"optional": {
"motion_guide_data": ("MOTION_GUIDE_DATA", {"tooltip": "Connect motion guide data from the timeline node to use IC-LoRA video guidance."}),
"model": ("MODEL", {"tooltip": "Connect model if using IC-LoRA for motion guidance."}),
"ic_lora_name": (["None"] + loras, {"default": "None", "tooltip": "Select the IC-LoRA model to use for motion guidance."}),
"ic_lora_strength": ("FLOAT", {"default": 1.0, "min": -100.0, "max": 100.0, "step": 0.01}),
"scale_by": ("FLOAT", {"default": 0.5, "min": 0.01, "max": 8.0, "step": 0.01, "tooltip": "Scale the latent by this factor. 0.5 is recommended for IC-LoRA."}),
"upscale_method": (["nearest-exact", "bilinear", "area", "bicubic", "bislerp"], {"default": "bicubic", "tooltip": "Method used to upscale/downscale the latent."}),
"image_attention_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}),
"crop": (["disabled", "center"], {"default": "center"}),
"auto_snap_ic_grid": ("BOOLEAN", {"default": True}),
"use_tiled_encode": ("BOOLEAN", {"default": False}),
"tile_size": ("INT", {"default": 256, "min": 64, "max": 512, "step": 32}),
"tile_overlap": ("INT", {"default": 64, "min": 16, "max": 256, "step": 16}),
"retake_mode": ("BOOLEAN", {"default": False, "tooltip": "Force Retake Mode. If false, it will still auto-detect Retake Mode from the timeline data."}),
"msr_strength": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.05, "tooltip": "Licon MSR only: per-stage reference strength override. 0 = use the Director's value (full pull, right for stage 1). On a refinement/upscale stage set ~0.4 to hold detail without the references repainting the opening (fixes stage-2 mist/ghosting)."}),
}
}
RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT", "MODEL", "FLOAT")
RETURN_NAMES = ("positive", "negative", "latent", "model", "latent_downscale_factor")
FUNCTION = "execute"
@classmethod
def execute(cls, positive, negative, vae, latent, guide_data, motion_guide_data=None, model=None, ic_lora_name="None", ic_lora_strength=1.0, scale_by=1.0, upscale_method="bicubic", image_attention_strength=1.0, crop="center", auto_snap_ic_grid=True, use_tiled_encode=False, tile_size=256, tile_overlap=64, retake_mode=False, msr_strength=0.0):
motion_segments = (motion_guide_data or {}).get("segments", []) if motion_guide_data else []
image_guides_count = len(guide_data.get("images", [])) if guide_data else 0
print(f"[LTXDirectorGuide] execute started. motion_segments: {len(motion_segments)}, image_guides: {image_guides_count}, ic_lora_name: {ic_lora_name}, model connected: {model is not None}, retake_mode: {retake_mode}")
active_resize_method = guide_data.get("resize_method") if guide_data else None
if not active_resize_method:
active_resize_method = motion_guide_data.get("resize_method") if motion_guide_data else None
if not active_resize_method:
active_resize_method = "crop" if crop == "center" else "stretch to fit"
latent_downscale_factor = 1.0
if model is not None and ic_lora_name != "None":
model, latent_downscale_factor = _load_lora_model_only(model, ic_lora_name, ic_lora_strength)
scale_factors = vae.downscale_index_formula
latent_image = latent["samples"].clone()
noise_mask = _clone_noise_mask(latent, latent_image)
# Apply scale factor if not 1.0
if scale_by != 1.0: if scale_by != 1.0:
B, C, F, H, W = latent_image.shape _, _, _, h, w = latent_image.shape
width = round(W * scale_by) target_w = max(1, round(w * scale_by))
height = round(H * scale_by) target_h = max(1, round(h * scale_by))
latent_image, noise_mask = _resize_latent_spatial(latent_image, noise_mask, target_w, target_h, upscale_method)
# Reshape to 4D for common_upscale if auto_snap_ic_grid and model is not None and ic_lora_name != "None":
latent_4d = latent_image.permute(0, 2, 1, 3, 4).reshape(B * F, C, H, W) latent_image, noise_mask = _snap_latent_to_downscale(latent_image, noise_mask, latent_downscale_factor, upscale_method)
latent_resized_4d = comfy.utils.common_upscale(latent_4d, width, height, upscale_method, "disabled")
latent_image = latent_resized_4d.reshape(B, F, C, height, width).permute(0, 2, 1, 3, 4)
# Also resize noise mask if it's not a broadcasted mask # Ghost Mask: the Director kept the latent clean (so it stays length-matched with the audio
if noise_mask.shape[-1] > 1 or noise_mask.shape[-2] > 1: # latent) and asked us to pre-extend it here by one hidden frame per reference. The tail
mask_4d = noise_mask.permute(0, 2, 1, 3, 4).reshape(B * F, 1, H, W) # sits past the clean region; the reference images (in guide_data["images"], anchored at
mask_resized_4d = comfy.utils.common_upscale(mask_4d, width, height, upscale_method, "disabled") # (clean + i)) are written into it by the loop below, then dropped downstream by Clean
noise_mask = mask_resized_4d.reshape(B, F, 1, height, width).permute(0, 2, 1, 3, 4) # Latent Slice (length = clean_latent_frames).
_ghost_pre_extend = int(guide_data.get("ghost_pre_extend", 0)) if guide_data else 0
if _ghost_pre_extend > 0:
gb, gc, gf, gh, gw = latent_image.shape
latent_image = torch.cat(
[latent_image, torch.zeros((gb, gc, _ghost_pre_extend, gh, gw), dtype=latent_image.dtype, device=latent_image.device)],
dim=2,
)
mb, mc, mf, mh, mw = noise_mask.shape
noise_mask = torch.cat(
[noise_mask, torch.ones((mb, mc, _ghost_pre_extend, mh, mw), dtype=noise_mask.dtype, device=noise_mask.device)],
dim=2,
)
_, _, latent_length, latent_height, latent_width = latent_image.shape _, _, latent_length, latent_height, latent_width = latent_image.shape
initial_latent_length = int(latent_length)
# MSR mode: the Director handed us the raw references; inject them at THIS node's # Licon MSR (Prefix) mode: the Director handed us raw references via guide_data["msr"].
# resolution (so a half-res Stage 1 and a full-res Stage 2 each stay self-consistent). # Inject them at THIS node's resolution (so a half-res Stage 1 and full-res Stage 2 each
msr = guide_data.get("msr") # stay self-consistent), then return early — the standard guide paths don't apply.
msr = guide_data.get("msr") if guide_data else None
if msr is not None: if msr is not None:
return cls._inject_msr(positive, negative, vae, latent_image, noise_mask, msr, msr_strength) return cls._inject_msr(positive, negative, vae, latent_image, noise_mask, msr, model, latent_downscale_factor, msr_strength)
images = guide_data.get("images", []) # Parse timeline JSON to see if retake mode is active in UI
insert_frames = guide_data.get("insert_frames", []) import json
strengths = guide_data.get("strengths", []) timeline_data_str = guide_data.get("timeline_data", "{}") if guide_data else "{}"
try:
tdata = json.loads(timeline_data_str)
except Exception:
tdata = {}
for idx, img_tensor in enumerate(images): is_retake_active = bool(retake_mode) or tdata.get("retakeMode", False)
f_idx = insert_frames[idx] if idx < len(insert_frames) else 0 is_empty_latent = (latent_image.abs().max().item() < 1e-5)
strength = strengths[idx] if idx < len(strengths) else 1.0
image_1, t = cls.encode(vae, latent_width, latent_height, img_tensor, scale_factors) # Director image guides and motion video segments
frame_idx, latent_idx = cls.get_latent_index(positive, latent_length, len(image_1), f_idx, scale_factors) images = guide_data.get("images", []) if guide_data else []
insert_frames = guide_data.get("insert_frames", []) if guide_data else []
strengths = guide_data.get("strengths", []) if guide_data else []
assert latent_idx + t.shape[2] <= latent_length, ( director_fps = float((motion_guide_data or {}).get("frame_rate", guide_data.get("frame_rate", 24) if guide_data else 24))
f"Guide image {idx + 1}: conditioning frames exceed the length of the latent sequence." segments = (motion_guide_data or {}).get("segments", [])
)
positive, negative, latent_image, noise_mask = cls.append_keyframe( is_lora_active = (model is not None) and (ic_lora_name != "None")
positive, negative, frame_idx, latent_image, noise_mask, t, strength, scale_factors, time_scale_factor = scale_factors[0]
) ltxv_length = (latent_length - 1) * time_scale_factor + 1
return io.NodeOutput(positive, negative, {"samples": latent_image, "noise_mask": noise_mask}) # -----------------------------------------------------------------------
# Retake Mode Branch:
# Load single base video, encode continuously, apply temporal mask.
# -----------------------------------------------------------------------
if is_retake_active:
print(f"[LTXDirectorGuide] Retake Mode active. Preserving base latent, masking selected regions. is_empty_latent: {is_empty_latent}")
target_width = latent_width * 32
target_height = latent_height * 32
# Calculate retake region latent indices first so we know what to copy/paste
retake_start = int(tdata.get("retakeStart", 0))
retake_len = int(tdata.get("retakeLength", 0))
retake_strength = float(tdata.get("retakeStrength", 1.0))
start_frame = int(guide_data.get("start_frame", 0))
relative_start = max(0, retake_start - start_frame)
l_start = relative_start // time_scale_factor
l_end = int(math.ceil((relative_start + retake_len) / time_scale_factor))
l_start = min(l_start, latent_length)
l_end = min(l_end, latent_length)
# Stage 2 optimization: If the retake region covers the entire generation area,
# there are no preserved regions to copy over in Stage 2. We can bypass video loading/encoding.
need_base_video = True
if not is_empty_latent and l_start == 0 and l_end >= latent_length:
need_base_video = False
print("[LTXDirectorGuide] Stage 2: Retake region covers the entire generated range. Skipping base video loading and VAE encoding.")
# 1. Try to load and encode base video from timeline data
retake_vid_info = tdata.get("retakeVideo") or {}
video_file = retake_vid_info.get("imageFile", "") if isinstance(retake_vid_info, dict) else ""
# Fallback to first segment only if it exists and we have no retake video info at all (old workflows)
if not video_file and not retake_vid_info and len(segments) > 0:
video_file = segments[0].get("videoFile", "")
if need_base_video:
if not video_file:
if retake_vid_info and not retake_vid_info.get("imageFile"):
raise ValueError(
"Retake Mode is active, but the base video file upload is still in progress (or failed). "
"Please wait for the 'Uploading base video...' overlay on the timeline to disappear before queuing the prompt."
)
else:
raise ValueError(
"Retake Mode is active, but no base video has been selected on the timeline. "
"Please drag and drop or upload a base video on the timeline first."
)
if video_file and need_base_video:
try:
print(f"[LTXDirectorGuide] Loading and encoding base video file: {video_file} starting at frame {start_frame} for length {ltxv_length} at resolution {target_width}x{target_height}")
video_frames = _load_motion_video_frames(
video_file, trim_start_frames=start_frame, length_frames=ltxv_length, director_fps=director_fps, resample_mode="nearest"
)
# Retake base video must match the exact target latent shape.
# "maintain aspect ratio" doesn't pad, which causes shape mismatch in VAE encode.
# So we fallback "maintain aspect ratio" to "pad" for retake base video.
retake_resize_method = active_resize_method
if retake_resize_method == "maintain aspect ratio":
retake_resize_method = "pad"
pixels = _resize_image(video_frames, target_width, target_height, retake_resize_method, divisible_by=1)
num_clip_frames = pixels.shape[0]
num_frames_to_keep = ((num_clip_frames - 1) // time_scale_factor) * time_scale_factor + 1
encode_src = pixels[:num_frames_to_keep, :, :, :3]
if use_tiled_encode:
base_latent = vae.encode_tiled(encode_src, tile_x=tile_size, tile_y=tile_size, overlap=tile_overlap)
else:
base_latent = vae.encode(encode_src)
base_latent = base_latent.to(device=latent_image.device, dtype=latent_image.dtype)
# Copy to latent_image
paste_len = min(base_latent.shape[2], latent_length)
if is_empty_latent:
# Stage 1: Overwrite entire latent with base video VAE encode
latent_image[:, :, :paste_len] = base_latent[:, :, :paste_len]
else:
# Stage 2: Overwrite only preserved regions (before l_start and after l_end)
# leaving the generated retake region from Stage 1 untouched
print(f"[LTXDirectorGuide] Stage 2: Copying high-resolution base latent for preserved regions (0-{l_start} and {l_end}-{paste_len})")
if l_start > 0:
latent_image[:, :, :l_start] = base_latent[:, :, :l_start]
if l_end < paste_len:
latent_image[:, :, l_end:paste_len] = base_latent[:, :, l_end:paste_len]
except Exception as e:
print(f"[LTXDirectorGuide] Failed to load/encode base video: {e}. Falling back to input latent.")
# 2. Build the temporal noise mask (0.0 = frozen, 1.0 = regenerate)
noise_mask = torch.zeros_like(noise_mask) # Initialize fully frozen
l_start = min(l_start, latent_length)
l_end = min(l_end, latent_length)
if l_end > l_start:
noise_mask[:, :, l_start:l_end] = retake_strength
print(f"[LTXDirectorGuide] noise_mask slice: {noise_mask[0, 0, :, 0, 0].tolist()}")
# In retake mode, skip normal mode processing entirely and return immediately!
exact_crop_frames = max(0, int(latent_image.shape[2]) - initial_latent_length)
positive = node_helpers.conditioning_set_values(positive, {"nghtdrp_guide_crop_latent_frames": exact_crop_frames})
negative = node_helpers.conditioning_set_values(negative, {"nghtdrp_guide_crop_latent_frames": exact_crop_frames})
return (positive, negative, {"samples": latent_image, "noise_mask": noise_mask}, model, float(latent_downscale_factor))
# -----------------------------------------------------------------------
# Standard Timeline Keyframe Guidance:
# Appends image segments as keyframes (and motion segments if present)
# to the latent stream using standard LTX-Video cross-attention conditioning.
# Registers guide attention entries if IC-LoRA is active.
# -----------------------------------------------------------------------
if len(images) > 0 or len(segments) > 0:
print(f"[LTXDirectorGuide] Using Appended Keyframe Guidance. is_lora_active: {is_lora_active}")
# A. Process Image Guides
for idx, img_tensor in enumerate(images):
f_idx = insert_frames[idx] if idx < len(insert_frames) else 0
strength = float(strengths[idx] if idx < len(strengths) else 1.0)
if strength <= 0.0:
continue
B_img, H_img, W_img, C_img = img_tensor.shape
target_pix_w = int(latent_width * 32)
target_pix_h = int(latent_height * 32)
if target_pix_w != W_img or target_pix_h != H_img:
img_nchw = img_tensor.permute(0, 3, 1, 2)
img_resized = comfy.utils.common_upscale(img_nchw, target_pix_w, target_pix_h, upscale_method, "disabled")
img_tensor = img_resized.permute(0, 2, 3, 1)
image_pixels, guide_latent = nodes_lt.LTXVAddGuide.encode(vae, latent_width, latent_height, img_tensor, scale_factors)
frame_idx, latent_idx = nodes_lt.LTXVAddGuide.get_latent_index(positive, latent_length, len(image_pixels), int(f_idx), scale_factors)
if latent_idx >= latent_length:
continue
max_frames = latent_length - latent_idx
if guide_latent.shape[2] > max_frames:
guide_latent = guide_latent[:, :, :max_frames]
tokens_added = guide_latent.shape[2] * guide_latent.shape[3] * guide_latent.shape[4]
guide_orig_shape = list(guide_latent.shape[2:])
positive, negative, latent_image, noise_mask = nodes_lt.LTXVAddGuide.append_keyframe(
positive, negative, frame_idx, latent_image, noise_mask, guide_latent, strength, scale_factors
)
if is_lora_active:
positive = _append_guide_attention_entry(positive, tokens_added, guide_orig_shape, attention_strength=image_attention_strength)
negative = _append_guide_attention_entry(negative, tokens_added, guide_orig_shape, attention_strength=image_attention_strength)
# B. Process Motion Video Segments
for seg in segments:
try:
video_file = seg.get("videoFile")
if not video_file:
continue
start_frame = int(seg.get("start", 0))
length_frames = int(seg.get("length", 1))
trim_start = int(seg.get("trimStart", 0))
video_strength = float(seg.get("videoStrength", 1.0))
video_attention_strength = float(seg.get("videoAttentionStrength", 0.65))
if length_frames <= 0 or video_strength <= 0.0:
continue
start_frame_aligned = start_frame
video_frames = _load_motion_video_frames(video_file, trim_start, length_frames, director_fps, seg.get("resampleMode", "nearest"))
num_frames_to_keep = ((video_frames.shape[0] - 1) // time_scale_factor) * time_scale_factor + 1
video_frames = video_frames[:num_frames_to_keep]
causal_fix = int(start_frame_aligned) == 0 or num_frames_to_keep == 1
encode_frames = video_frames if causal_fix else torch.cat([video_frames[:1], video_frames], dim=0)
_, guide_latent = _encode_video_iclora_guide(vae, latent_width, latent_height, encode_frames, scale_factors, latent_downscale_factor, crop, use_tiled_encode, tile_size, tile_overlap, resize_method=active_resize_method)
if not causal_fix:
guide_latent = guide_latent[:, :, 1:, :, :]
frame_idx = start_frame_aligned
latent_idx = (frame_idx + time_scale_factor - 1) // time_scale_factor if frame_idx > 0 else 0
if latent_idx >= latent_length:
continue
if start_frame > 0 and guide_latent.shape[2] > 1:
guide_latent = guide_latent[:, :, 1:, :, :]
frame_idx += time_scale_factor
latent_idx += 1
if latent_idx >= latent_length:
continue
max_frames = latent_length - latent_idx
if guide_latent.shape[2] > max_frames:
guide_latent = guide_latent[:, :, :max_frames]
guide_orig_shape = list(guide_latent.shape[2:])
B_g, C_g, F_g, H_g, W_g = guide_latent.shape
guide_mask = torch.ones((B_g, 1, F_g, H_g, W_g), device=guide_latent.device, dtype=guide_latent.dtype)
if start_frame > 0:
ramp_steps = [0.25, 0.65]
for i, s in enumerate(ramp_steps):
if i < F_g:
guide_mask_val = 1.0 + video_strength * (1.0 - s)
guide_mask[:, :, i, :, :] = guide_mask_val
ldf = int(max(1, round(float(latent_downscale_factor))))
if ldf > 1:
dilated = _dilate_latent({"samples": guide_latent, "noise_mask": guide_mask}, horizontal_scale=ldf, vertical_scale=ldf)
guide_mask = dilated["noise_mask"]
guide_latent = dilated["samples"]
tokens_added = guide_latent.shape[2] * guide_latent.shape[3] * guide_latent.shape[4]
positive, negative, latent_image, noise_mask = nodes_lt.LTXVAddGuide.append_keyframe(
positive, negative, frame_idx, latent_image, noise_mask, guide_latent, video_strength, scale_factors, guide_mask=guide_mask, latent_downscale_factor=float(latent_downscale_factor), causal_fix=causal_fix
)
if is_lora_active:
positive = _append_guide_attention_entry(positive, tokens_added, guide_orig_shape, attention_strength=video_attention_strength)
negative = _append_guide_attention_entry(negative, tokens_added, guide_orig_shape, attention_strength=video_attention_strength)
except Exception as e:
raise RuntimeError(f"LTX Director Guide motion segment failed for {seg}: {e}") from e
else:
print("[LTXDirectorGuide] No timeline guides present. Passing through.")
exact_crop_frames = max(0, int(latent_image.shape[2]) - initial_latent_length)
positive = node_helpers.conditioning_set_values(positive, {"nghtdrp_guide_crop_latent_frames": exact_crop_frames})
negative = node_helpers.conditioning_set_values(negative, {"nghtdrp_guide_crop_latent_frames": exact_crop_frames})
return (positive, negative, {"samples": latent_image, "noise_mask": noise_mask}, model, float(latent_downscale_factor))
@classmethod @classmethod
def _inject_msr(cls, positive, negative, vae, latent_image, noise_mask, msr, msr_strength=0.0): def _inject_msr(cls, positive, negative, vae, latent_image, noise_mask, msr, model, latent_downscale_factor, msr_strength=0.0):
"""Inject the Licon MSR references at this node's current resolution.""" """Inject the Licon MSR references at this node's current resolution.
Returns this node's 5-tuple (positive, negative, latent, model, latent_downscale_factor).
"""
from nodes import NODE_CLASS_MAPPINGS from nodes import NODE_CLASS_MAPPINGS
from .ltx_director import _execute_comfy_node, _unpack
IC = NODE_CLASS_MAPPINGS.get("LTXAddVideoICLoRAGuide") IC = NODE_CLASS_MAPPINGS.get("LTXAddVideoICLoRAGuide")
AG = NODE_CLASS_MAPPINGS.get("LTXVAddGuide") AG = NODE_CLASS_MAPPINGS.get("LTXVAddGuide")
@@ -118,6 +649,8 @@ class LTXDirectorGuide(LTXVAddGuide):
strength = clamp01(msr_strength) if float(msr_strength) > 0.0 else clamp01(msr["strength"]) strength = clamp01(msr_strength) if float(msr_strength) > 0.0 else clamp01(msr["strength"])
downscale = float(msr["downscale"]) downscale = float(msr["downscale"])
initial_latent_length = int(latent_image.shape[2])
# Pad the generated region so (pad + keyframes) == prefix_latents, so the downstream crop # Pad the generated region so (pad + keyframes) == prefix_latents, so the downstream crop
# (which trims the slideshow's temporal footprint) lands on the true clean length. # (which trims the slideshow's temporal footprint) lands on the true clean length.
pad_latents = max(0, prefix_latents - len(keyframes)) pad_latents = max(0, prefix_latents - len(keyframes))
@@ -140,15 +673,95 @@ class LTXDirectorGuide(LTXVAddGuide):
IC, positive=positive, negative=negative, vae=vae, latent=latent, image=slideshow, IC, positive=positive, negative=negative, vae=vae, latent=latent, image=slideshow,
frame_idx=0, strength=strength, latent_downscale_factor=downscale, frame_idx=0, strength=strength, latent_downscale_factor=downscale,
crop="center", use_tiled_encode=False, tile_size=256, tile_overlap=64, crop="center", use_tiled_encode=False, tile_size=256, tile_overlap=64,
)) ))[:3]
positive, negative = cp, cn positive, negative = cp, cn
# LATENT PATH: one frozen keyframe per reference at frame 0 -> keep latent, drop cond. # LATENT PATH: one frozen keyframe per reference at frame 0 -> keep latent, drop cond.
for kf in keyframes: for kf in keyframes:
_p, _n, latent = _unpack(_execute_comfy_node( unpacked = _unpack(_execute_comfy_node(
AG, positive=positive, negative=negative, vae=vae, latent=latent, AG, positive=positive, negative=negative, vae=vae, latent=latent,
image=kf, frame_idx=0, strength=strength, image=kf, frame_idx=0, strength=strength,
)) ))
latent = unpacked[2]
return io.NodeOutput(positive, negative, latent) # NOTE: MSR is a PREFIX. We deliberately do NOT set nghtdrp_guide_crop_latent_frames
# (the new LTXDirectorCropGuides trims from the END, which would cut clean frames).
# The IC/AG nodes write position-aware keyframe metadata into the conditioning, so the
# MSR branch of the workflow should crop with the STOCK position-aware LTXVCropGuides
# node (exactly as the old MSR pipeline did).
return (positive, negative, latent, model, float(latent_downscale_factor))
def _conditioning_get_any_value(conditioning, key, default=None):
for item in conditioning:
meta = item[1]
if key in meta and meta.get(key) is not None:
return meta.get(key)
return default
def _get_exact_crop_count_from_conditioning(conditioning):
value = _conditioning_get_any_value(conditioning, "nghtdrp_guide_crop_latent_frames", None)
if value is not None:
try:
return max(0, int(value))
except Exception:
return 0
keyframe_idxs = _conditioning_get_any_value(conditioning, "keyframe_idxs", None)
if keyframe_idxs is None:
return 0
try:
return int(torch.unique(keyframe_idxs[:, 0, :, 0]).shape[0])
except Exception:
return 0
def _get_noise_mask_for_crop(latent):
latent_image = latent["samples"]
noise_mask = latent.get("noise_mask", None)
if noise_mask is None:
batch, _, frames, _, _ = latent_image.shape
return torch.ones((batch, 1, frames, 1, 1), dtype=torch.float32, device=latent_image.device)
return noise_mask.clone()
class LTXDirectorCropGuides:
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"positive": ("CONDITIONING",),
"negative": ("CONDITIONING",),
"latent": ("LATENT",),
}
}
RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT")
RETURN_NAMES = ("positive", "negative", "latent")
FUNCTION = "execute"
CATEGORY = "WhatDreamsCost CS"
def execute(self, positive, negative, latent):
latent_image = latent["samples"].clone()
noise_mask = _get_noise_mask_for_crop(latent)
crop_frames = _get_exact_crop_count_from_conditioning(positive)
if crop_frames <= 0:
return (positive, negative, {"samples": latent_image, "noise_mask": noise_mask})
crop_frames = min(crop_frames, max(0, latent_image.shape[2] - 1))
if crop_frames > 0:
latent_image = latent_image[:, :, :-crop_frames]
noise_mask = noise_mask[:, :, :-crop_frames]
clear_values = {
"keyframe_idxs": None,
"guide_attention_entries": None,
"nghtdrp_guide_crop_latent_frames": None,
}
positive = node_helpers.conditioning_set_values(positive, clear_values)
negative = node_helpers.conditioning_set_values(negative, clear_values)
return (positive, negative, {"samples": latent_image, "noise_mask": noise_mask})
NODE_CLASS_MAPPINGS = {
"LTXDirectorGuideCS": LTXDirectorGuide,
"LTXDirectorCropGuidesCS": LTXDirectorCropGuides,
}

View File

@@ -1,4 +1,3 @@
import json
from comfy_extras.nodes_lt import get_noise_mask, LTXVAddGuide from comfy_extras.nodes_lt import get_noise_mask, LTXVAddGuide
import torch import torch
import comfy.utils import comfy.utils
@@ -56,14 +55,12 @@ class LTXSequencer(LTXVAddGuide):
node_id="LTXSequencer", node_id="LTXSequencer",
display_name="LTX Sequencer", display_name="LTX Sequencer",
category="WhatDreamsCost", category="WhatDreamsCost",
description="Add multiple guide images at specified frame indices or seconds. Auto-syncs to Prompt Relay invisibly via the positive wire.", description="Add multiple guide images at specified frame indices or seconds with strengths. Number of widgets is dynamically configured.",
inputs=inputs, inputs=inputs,
outputs=[ outputs=[
io.Conditioning.Output(display_name="positive"), io.Conditioning.Output(display_name="positive"),
io.Conditioning.Output(display_name="negative"), io.Conditioning.Output(display_name="negative"),
io.Latent.Output(display_name="latent", tooltip="Video latent with added guides"), io.Latent.Output(display_name="latent", tooltip="Video latent with added guides"),
# ---> ADDED SYNC LOG OUTPUT <---
io.String.Output(display_name="sync_log", tooltip="Outputs a readable text block showing the actual times used for each image"),
], ],
) )
@@ -92,64 +89,30 @@ class LTXSequencer(LTXVAddGuide):
insert_mode = kwargs.get("insert_mode", "frames") insert_mode = kwargs.get("insert_mode", "frames")
frame_rate = kwargs.get("frame_rate", 24) frame_rate = kwargs.get("frame_rate", 24)
# ---> EXTRACT GHOST SYNC DATA <---
timeline_data = None
if positive is not None and len(positive) > 0:
if "prompt_relay_timeline" in positive[0][1]:
try:
timeline_data = json.loads(positive[0][1]["prompt_relay_timeline"])
except json.JSONDecodeError:
pass
# Prepare the log text
sync_log_lines = []
if timeline_data:
sync_log_lines.append("=== TIMELINE SYNC ENABLED ===")
sync_log_lines.append(f"Invisibly Synced via Positive Wire! Using {insert_mode.upper()}:")
else:
sync_log_lines.append("=== MANUAL MODE ===")
sync_log_lines.append(f"No timeline detected, or Sync turned OFF. Using manual UI inputs ({insert_mode}):")
# Process inputs up to num_images, extracting dynamic frame/strength values from kwargs # Process inputs up to num_images, extracting dynamic frame/strength values from kwargs
for i in range(1, num_images + 1): for i in range(1, num_images + 1):
# Skip if this image index exceeds the batch # Skip if this image index exceeds the batch
if i > batch_size: if i > batch_size:
sync_log_lines.append(f"Image #{i}: Skipped (No image loaded in batch)")
continue continue
img = multi_input[i-1:i] # Extract the single image frame from the batch img = multi_input[i-1:i] # Extract the single image frame from the batch
if img is None: if img is None:
continue continue
# Calculate the final frame index based on the chosen mode
f_idx = None f_idx = None
strength = kwargs.get(f"strength_{i}", 1.0) if insert_mode == "frames":
f_idx = kwargs.get(f"insert_frame_{i}")
# 1. AUTO-SYNC elif insert_mode == "seconds":
if timeline_data and "starts_frames" in timeline_data and (i - 1) < len(timeline_data["starts_frames"]): sec = kwargs.get(f"insert_second_{i}")
display_frame = timeline_data["starts_frames"][i - 1] if sec is not None:
display_sec = timeline_data["starts_seconds"][i - 1] f_idx = int(sec * frame_rate)
if insert_mode == "frames":
f_idx = display_frame
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Synced to Segment #{i} start @ {display_frame} frames")
elif insert_mode == "seconds":
f_idx = int(display_sec * frame_rate)
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Synced to Segment #{i} start @ {display_sec:.2f} seconds (Frame {f_idx})")
# 2. MANUAL FALLBACK
else:
if insert_mode == "frames":
f_idx = kwargs.get(f"insert_frame_{i}")
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Manual input @ {f_idx} frames")
elif insert_mode == "seconds":
sec = kwargs.get(f"insert_second_{i}")
if sec is not None:
f_idx = int(sec * frame_rate)
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Manual input @ {sec:.2f} seconds (Frame {f_idx})")
if f_idx is None: if f_idx is None:
continue continue
strength = kwargs.get(f"strength_{i}", 1.0)
# Execution logic mirrored from LTXVAddGuideMulti # Execution logic mirrored from LTXVAddGuideMulti
image_1, t = cls.encode(vae, latent_width, latent_height, img, scale_factors) image_1, t = cls.encode(vae, latent_width, latent_height, img, scale_factors)
@@ -167,8 +130,4 @@ class LTXSequencer(LTXVAddGuide):
scale_factors, scale_factors,
) )
sync_log_str = "\n".join(sync_log_lines) return io.NodeOutput(positive, negative, {"samples": latent_image, "noise_mask": noise_mask})
print(f"\n[LTX Sequencer]\n{sync_log_str}\n")
# Returned the newly formatted sync log string at the end of NodeOutput
return io.NodeOutput(positive, negative, {"samples": latent_image, "noise_mask": noise_mask}, sync_log_str)

View File

@@ -1,182 +0,0 @@
import torch
import torch.nn.functional as F
import numpy as np
from PIL import Image, ImageOps
import os
import folder_paths
import io
import comfy.utils
class MultiImageLoader:
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"image_paths": ("STRING", {"default": "", "multiline": True}),
"width": ("INT", {"default": 0, "min": 0, "max": 8192, "step": 1}),
"height": ("INT", {"default": 0, "min": 0, "max": 8192, "step": 1}),
"interpolation": (["lanczos", "nearest", "bilinear", "bicubic", "area", "nearest-exact"],),
"resize_method": (["keep proportion", "stretch", "pad", "crop"],),
"multiple_of": ("INT", {"default": 32, "min": 0, "max": 512, "step": 1}),
"img_compression": ("INT", {"default": 18, "min": 0, "max": 100, "step": 1}),
},
}
# Added "IMAGE" at the beginning for multi_output + 50 individual outputs = 51 outputs
RETURN_TYPES = ("IMAGE",) * 51
RETURN_NAMES = ("multi_output",) + tuple(f"image_{i+1}" for i in range(50))
FUNCTION = "load_images"
CATEGORY = "WhatDreamsCost"
def resize_image(self, image, width, height, resize_method="keep proportion", interpolation="nearest", multiple_of=0):
MAX_RESOLUTION = 8192
_, oh, ow, _ = image.shape
x = y = x2 = y2 = 0
pad_left = pad_right = pad_top = pad_bottom = 0
if multiple_of > 1:
width = width - (width % multiple_of)
height = height - (height % multiple_of)
if resize_method == 'keep proportion' or resize_method == 'pad':
if width == 0 and oh < height:
width = MAX_RESOLUTION
elif width == 0 and oh >= height:
width = ow
if height == 0 and ow < width:
height = MAX_RESOLUTION
elif height == 0 and ow >= width:
height = oh
ratio = min(width / ow, height / oh)
new_width = round(ow * ratio)
new_height = round(oh * ratio)
if resize_method == 'pad':
pad_left = (width - new_width) // 2
pad_right = width - new_width - pad_left
pad_top = (height - new_height) // 2
pad_bottom = height - new_height - pad_top
width = new_width
height = new_height
elif resize_method == 'crop':
width = width if width > 0 else ow
height = height if height > 0 else oh
ratio = max(width / ow, height / oh)
new_width = round(ow * ratio)
new_height = round(oh * ratio)
x = (new_width - width) // 2
y = (new_height - height) // 2
x2 = x + width
y2 = y + height
if x2 > new_width:
x -= (x2 - new_width)
if x < 0:
x = 0
if y2 > new_height:
y -= (y2 - new_height)
if y < 0:
y = 0
width = new_width
height = new_height
else:
width = width if width > 0 else ow
height = height if height > 0 else oh
# Always apply resize logic
outputs = image.permute(0, 3, 1, 2)
if interpolation == "lanczos":
outputs = comfy.utils.lanczos(outputs, width, height)
else:
outputs = F.interpolate(outputs, size=(height, width), mode=interpolation)
if resize_method == 'pad':
if pad_left > 0 or pad_right > 0 or pad_top > 0 or pad_bottom > 0:
outputs = F.pad(outputs, (pad_left, pad_right, pad_top, pad_bottom), value=0)
outputs = outputs.permute(0, 2, 3, 1)
if resize_method == 'crop':
if x > 0 or y > 0 or x2 > 0 or y2 > 0:
outputs = outputs[:, y:y2, x:x2, :]
if multiple_of > 1 and (outputs.shape[2] % multiple_of != 0 or outputs.shape[1] % multiple_of != 0):
width = outputs.shape[2]
height = outputs.shape[1]
x = (width % multiple_of) // 2
y = (height % multiple_of) // 2
x2 = width - ((width % multiple_of) - x)
y2 = height - ((height % multiple_of) - y)
outputs = outputs[:, y:y2, x:x2, :]
outputs = torch.clamp(outputs, 0, 1)
return outputs
def load_images(self, image_paths, width, height, interpolation, resize_method, multiple_of, img_compression):
results = []
valid_paths = [p.strip() for p in image_paths.split("\n") if p.strip()]
for path in valid_paths:
try:
# Resolve full path
full_path = path
if not os.path.exists(full_path):
full_path = os.path.join(folder_paths.get_input_directory(), path)
if not os.path.exists(full_path):
print(f"Warning: Image path not found: {path}")
continue
# Load image
image = Image.open(full_path)
image = ImageOps.exif_transpose(image)
image = image.convert("RGB")
# Convert to Torch Tensor to prepare for Advanced Resize Logic
image_np = np.array(image).astype(np.float32) / 255.0
image_tensor = torch.from_numpy(image_np)[None,]
# Apply Advanced Resize
image_tensor = self.resize_image(image_tensor, width, height, resize_method, interpolation, multiple_of)
# Compression (Applied after resize to accurately maintain the effect)
if img_compression > 0:
img_np = (image_tensor[0].numpy() * 255).clip(0, 255).astype(np.uint8)
img_pil = Image.fromarray(img_np)
img_byte_arr = io.BytesIO()
img_pil.save(img_byte_arr, format="JPEG", quality=max(1, 100 - img_compression))
img_pil = Image.open(img_byte_arr)
image_tensor = torch.from_numpy(np.array(img_pil).astype(np.float32) / 255.0)[None,]
results.append(image_tensor)
except Exception as e:
print(f"Error loading {path}: {e}")
# Combine all successfully loaded images into a single batched tensor for multi_output
if len(results) > 0:
# Safety Check: Advanced resize methods might output differently sized tensors (e.g., 'keep proportion')
first_shape = results[0].shape
all_same_shape = all(r.shape == first_shape for r in results)
if all_same_shape:
multi_output = torch.cat(results, dim=0)
else:
print("MultiImageLoader Warning: Images have different dimensions due to resize settings. Cannot batch into multi_output. Outputting zero tensor for the batch, but individual output nodes will still work fine.")
multi_output = torch.zeros((1, 64, 64, 3))
else:
# Fallback empty tensor if no valid paths
multi_output = torch.zeros((1, 64, 64, 3))
results = [multi_output]
# Pad individual outputs exactly to length 50 as defined in RETURN_TYPES
padded_results = results + [torch.zeros((1, 64, 64, 3))] * (50 - len(results))
# Return the multi batch output first, followed by the individual padded items
return (multi_output, *padded_results[:50])

View File

@@ -1,7 +1,10 @@
import logging
import types import types
import torch
import comfy.ldm.modules.attention import comfy.ldm.modules.attention
log = logging.getLogger(__name__)
def _masked_attention(q, k, v, heads, mask, transformer_options={}, **kwargs): def _masked_attention(q, k, v, heads, mask, transformer_options={}, **kwargs):
# Bypass wrap_attn (sage/etc may ignore masks) by calling attention_pytorch directly. # Bypass wrap_attn (sage/etc may ignore masks) by calling attention_pytorch directly.
@@ -18,7 +21,7 @@ def _wan_t2v_forward(self, mask_fn, x, context, transformer_options={}, **kwargs
k = self.norm_k(self.k(context)) k = self.norm_k(self.k(context))
v = self.v(context) v = self.v(context)
mask = mask_fn(q, k, transformer_options) mask = mask_fn(q.shape[1], k.shape[1], q.dtype, q.device, transformer_options)
if mask is not None: if mask is not None:
x = _masked_attention(q, k, v, heads=self.num_heads, mask=mask, x = _masked_attention(q, k, v, heads=self.num_heads, mask=mask,
transformer_options=transformer_options) transformer_options=transformer_options)
@@ -44,7 +47,7 @@ def _wan_i2v_forward(self, mask_fn, x, context, context_img_len, transformer_opt
k = self.norm_k(self.k(context_text)) k = self.norm_k(self.k(context_text))
v = self.v(context_text) v = self.v(context_text)
mask = mask_fn(q, k, transformer_options) mask = mask_fn(q.shape[1], k.shape[1], q.dtype, q.device, transformer_options)
if mask is not None: if mask is not None:
x = _masked_attention(q, k, v, heads=self.num_heads, mask=mask, x = _masked_attention(q, k, v, heads=self.num_heads, mask=mask,
transformer_options=transformer_options) transformer_options=transformer_options)
@@ -56,47 +59,65 @@ def _wan_i2v_forward(self, mask_fn, x, context, context_img_len, transformer_opt
return self.o(x + img_x) return self.o(x + img_x)
def _ltx_forward(self, mask_fn, x, context=None, mask=None, pe=None, k_pe=None, transformer_options={}): def _make_masked_override(prev_override):
from comfy.ldm.lightricks.model import apply_rotary_emb """transformer_options override that routes mask-bearing attention calls through
attention_pytorch (sage/etc. drop arbitrary masks). Chains to a prior override
when no mask is present so we don't clobber other backends."""
def override(func, *args, **kwargs):
if kwargs.get("mask") is not None:
return comfy.ldm.modules.attention.attention_pytorch(*args, **kwargs)
if prev_override is not None:
return prev_override(func, *args, **kwargs)
return func(*args, **kwargs)
return override
is_self_attn = context is None
context = x if is_self_attn else context
q = self.q_norm(self.to_q(x)) def debug_log(msg):
k = self.k_norm(self.to_k(context)) try:
v = self.to_v(context) import os
log_path = os.path.join(os.path.dirname(__file__), "debug_prompt_relay.log")
with open(log_path, "a", encoding="utf-8") as f:
f.write(msg + "\n")
except Exception:
pass
if pe is not None:
q = apply_rotary_emb(q, pe)
k = apply_rotary_emb(k, pe if k_pe is None else k_pe)
if not is_self_attn: def _make_ltx_mask_wrapper(underlying, mask_fn, attr):
temporal_mask = mask_fn(q, k, transformer_options) """Wrap an existing LTX cross-attn forward (the default `CrossAttention.forward`
if temporal_mask is not None: or another node's patch — e.g. KJNodes NAG), injecting PromptRelay's additive
mask = temporal_mask if mask is None else mask + temporal_mask mask via the `mask` kwarg the upstream signature already accepts.
if mask is None: `underlying` must already be bound to its module — callable as
out = comfy.ldm.modules.attention.optimized_attention( `underlying(x, context=..., mask=..., ...)`.
q, k, v, self.heads, attn_precision=self.attn_precision, """
def wrapped(_self, x, context=None, mask=None, pe=None, k_pe=None, transformer_options={}):
debug_log(f"wrapped called: x.shape={list(x.shape)} context.shape={list(context.shape) if context is not None else None} mask_is_none={mask is None}")
if context is not None:
opts = {**transformer_options, "promptrelay_attn_type": attr}
pr_mask = mask_fn(x.shape[1], context.shape[1], x.dtype, x.device, opts)
debug_log(f"mask_fn returned pr_mask_is_none={pr_mask is None}")
if pr_mask is not None:
debug_log(f"pr_mask info: shape={list(pr_mask.shape)} min={pr_mask.min().item()} max={pr_mask.max().item()} sum={pr_mask.sum().item()}")
mask = pr_mask if mask is None else mask + pr_mask
if mask is not None:
prev = transformer_options.get("optimized_attention_override")
transformer_options = {
**transformer_options,
"optimized_attention_override": _make_masked_override(prev),
}
return underlying(
x, context=context, mask=mask, pe=pe, k_pe=k_pe,
transformer_options=transformer_options, transformer_options=transformer_options,
) )
else:
out = _masked_attention(q, k, v, self.heads, mask=mask,
attn_precision=self.attn_precision,
transformer_options=transformer_options)
if self.to_gate_logits is not None: wrapped._promptrelay_wrapper = True
gate_logits = self.to_gate_logits(x) return wrapped
b, t, _ = out.shape
out = out.view(b, t, self.heads, self.dim_head)
out = out * (2.0 * torch.sigmoid(gate_logits)).unsqueeze(-1)
out = out.view(b, t, self.heads * self.dim_head)
return self.to_out(out)
class _CrossAttnPatch: class _CrossAttnPatch:
"""Descriptor that binds (impl, mask_fn) as a method onto a cross-attn module.""" """Descriptor that binds (impl, mask_fn) as a method onto a Wan cross-attn module."""
def __init__(self, impl, mask_fn): def __init__(self, impl, mask_fn):
self.impl = impl self.impl = impl
@@ -135,8 +156,8 @@ def _check_unpatched(model_clone, key):
if key in getattr(model_clone, "object_patches", {}): if key in getattr(model_clone, "object_patches", {}):
raise RuntimeError( raise RuntimeError(
f"PromptRelay: cross-attention forward at '{key}' is already patched by " f"PromptRelay: cross-attention forward at '{key}' is already patched by "
"another node (e.g. KJNodes NAG). Stacking is not supported — remove the " "another node. Stacking is not supported for this architecture — remove "
"conflicting node." "the conflicting node."
) )
@@ -154,14 +175,19 @@ def apply_patches(model_clone, arch, mask_fn):
return return
if arch == "ltx": if arch == "ltx":
to = model_clone.model_options["transformer_options"]
to["promptrelay_mask_fn"] = mask_fn
for idx, block in enumerate(diffusion_model.transformer_blocks): for idx, block in enumerate(diffusion_model.transformer_blocks):
for attr in ("attn2", "audio_attn2"): for attr in ("attn2", "audio_attn2"):
module = getattr(block, attr, None) module = getattr(block, attr, None)
if module is None: if module is None:
continue continue
key = f"diffusion_model.transformer_blocks.{idx}.{attr}.forward" key = f"diffusion_model.transformer_blocks.{idx}.{attr}.forward"
_check_unpatched(model_clone, key) # get_model_object returns the prior patch if present, else the default bound forward.
model_clone.add_object_patch(key, _CrossAttnPatch(_ltx_forward, mask_fn).__get__(module, module.__class__)) underlying = model_clone.get_model_object(key)
wrapper = _make_ltx_mask_wrapper(underlying, mask_fn, attr)
model_clone.add_object_patch(key, types.MethodType(wrapper, module))
return return
raise ValueError(f"Unknown model arch: {arch}") raise ValueError(f"Unknown model arch: {arch}")

View File

@@ -20,7 +20,7 @@ def build_temporal_cost(q_token_idx, Lq, Lk, device, dtype, tokens_per_frame):
return offset return offset
def build_temporal_cost_scaled(q_token_idx, Lq, Lk, device, dtype, latent_frames): def build_temporal_cost_scaled(q_token_idx, Lq, Lk, device, dtype, latent_frames, is_audio=False):
"""Penalty matrix for queries that don't map to integer frames (e.g. LTXAV audio tokens).""" """Penalty matrix for queries that don't map to integer frames (e.g. LTXAV audio tokens)."""
offset = torch.zeros(Lq, Lk, device=device, dtype=dtype) offset = torch.zeros(Lq, Lk, device=device, dtype=dtype)
query_frames = torch.arange(Lq, device=device, dtype=torch.float32) * latent_frames / Lq query_frames = torch.arange(Lq, device=device, dtype=torch.float32) * latent_frames / Lq
@@ -28,54 +28,90 @@ def build_temporal_cost_scaled(q_token_idx, Lq, Lk, device, dtype, latent_frames
for seg in q_token_idx: for seg in q_token_idx:
local = seg["local_token_idx"].to(device=device) local = seg["local_token_idx"].to(device=device)
d = (query_frames[:, None] - seg["midpoint"]).abs() d = (query_frames[:, None] - seg["midpoint"]).abs()
sigma_a = seg.get("sigma_audio", seg["sigma"]) if is_audio:
window_a = seg.get("window_audio", seg["window"]) sigma_val = seg.get("sigma_audio", seg["sigma"])
strength_a = seg.get("strength_audio", 1.0) window_val = seg.get("window_audio", seg["window"])
cost = strength_a * (torch.relu(d - window_a) ** 2) / (2 * sigma_a ** 2) strength_val = seg.get("strength_audio", 1.0)
else:
sigma_val = seg["sigma"]
window_val = seg["window"]
strength_val = seg.get("strength", 1.0)
cost = strength_val * (torch.relu(d - window_val) ** 2) / (2 * sigma_val ** 2)
offset[:, local] = cost.to(offset.dtype) offset[:, local] = cost.to(offset.dtype)
return offset return offset
def debug_log(msg):
try:
import os
log_path = os.path.join(os.path.dirname(__file__), "debug_prompt_relay.log")
with open(log_path, "a", encoding="utf-8") as f:
f.write(msg + "\n")
except Exception:
pass
def create_mask_fn(q_token_idx, fallback_tokens_per_frame, latent_frames): def create_mask_fn(q_token_idx, fallback_tokens_per_frame, latent_frames):
"""Closure: mask_fn(q, k, transformer_options) -> additive mask or None.""" """Closure: mask_fn(Lq, Lk, dtype, device, transformer_options) -> additive mask or None.
Takes shapes/dtype/device instead of tensors so callers can compute the mask
without first materializing q/k projections — required so PromptRelay can
wrap an existing cross-attn forward (e.g. KJNodes NAG) instead of replacing it.
"""
cache = {} cache = {}
max_token_idx = max(int(seg["local_token_idx"].max().item()) for seg in q_token_idx) + 1 max_token_idx = max(int(seg["local_token_idx"].max().item()) for seg in q_token_idx) + 1
def mask_fn(q, k, transformer_options): def mask_fn(Lq, Lk, dtype, device, transformer_options):
Lq, Lk = q.shape[1], k.shape[1] debug_log(f"mask_fn check: Lq={Lq} Lk={Lk} max_token_idx={max_token_idx} cond_or_uncond={transformer_options.get('cond_or_uncond', [])}")
if Lq == Lk: if Lq == Lk:
debug_log("mask_fn: Lq == Lk, returning None")
return None return None
# Only apply on conditional pass — not unconditional (negative prompt) # Only apply on conditional pass — not unconditional (negative prompt)
cond_or_uncond = transformer_options.get("cond_or_uncond", []) cond_or_uncond = transformer_options.get("cond_or_uncond", [])
if 1 in cond_or_uncond and 0 not in cond_or_uncond: if 1 in cond_or_uncond and 0 not in cond_or_uncond:
debug_log("mask_fn: unconditional pass, returning None")
return None return None
grid_sizes = transformer_options.get("grid_sizes", None) grid_sizes = transformer_options.get("grid_sizes", None)
video_tpf = int(grid_sizes[1]) * int(grid_sizes[2]) if grid_sizes is not None else fallback_tokens_per_frame attn_type = transformer_options.get("promptrelay_attn_type", "attn2")
video_lq = latent_frames * video_tpf is_audio = (attn_type == "audio_attn2")
# Skip cross-modal attention — text keys are padded to a fixed length ≥ max_token_idx and != video_lq if is_audio:
if Lk == video_lq or Lk < max_token_idx: mode = "scaled"
return None video_lq = -1
else:
if grid_sizes is not None:
video_tpf = int(grid_sizes[1]) * int(grid_sizes[2])
else:
if Lq % latent_frames == 0:
video_tpf = Lq // latent_frames
else:
video_tpf = fallback_tokens_per_frame
video_lq = latent_frames * video_tpf
mode = "video" if Lq == video_lq else "scaled" # Skip cross-modal attention — text keys are padded to a fixed length ≥ max_token_idx and != video_lq
if Lk == video_lq or Lk < max_token_idx:
debug_log(f"mask_fn: Lk == video_lq ({Lk == video_lq}) or Lk < max_token_idx ({Lk < max_token_idx}), returning None")
return None
key = (Lq, Lk, mode, q.device) mode = "video" if Lq == video_lq else "scaled"
key = (Lq, Lk, mode, device)
if key not in cache: if key not in cache:
if mode == "video": if mode == "video":
cost = build_temporal_cost(q_token_idx, Lq, Lk, q.device, q.dtype, video_tpf) cost = build_temporal_cost(q_token_idx, Lq, Lk, device, dtype, video_tpf)
else: else:
cost = build_temporal_cost_scaled(q_token_idx, Lq, Lk, q.device, q.dtype, latent_frames) cost = build_temporal_cost_scaled(q_token_idx, Lq, Lk, device, dtype, latent_frames, is_audio=is_audio)
log.info( log.info(
"[PromptRelay] Built penalty matrix (%s): Lq=%d, Lk=%d, nonzero=%d/%d", "[PromptRelay] Built penalty matrix (%s): Lq=%d, Lk=%d, nonzero=%d/%d",
mode, Lq, Lk, (cost > 0).sum().item(), cost.numel(), mode, Lq, Lk, (cost > 0).sum().item(), cost.numel(),
) )
debug_log(f"Built penalty matrix ({mode}): Lq={Lq}, Lk={Lk}, key={key}")
cache[key] = -cost cache[key] = -cost
return cache[key].to(q.dtype) return cache[key].to(dtype)
return mask_fn return mask_fn
@@ -160,7 +196,30 @@ def map_token_indices(raw_tokenizer, global_prompt, local_prompts):
""" """
prefixed_locals = [" " + lp for lp in local_prompts] prefixed_locals = [" " + lp for lp in local_prompts]
full_prompt = global_prompt + "".join(prefixed_locals) full_prompt = global_prompt + "".join(prefixed_locals)
# Detect if the tokenizer appends EOS dynamically
has_eos = getattr(raw_tokenizer, "add_eos", False) has_eos = getattr(raw_tokenizer, "add_eos", False)
if not has_eos:
try:
test_res = raw_tokenizer("test")
if isinstance(test_res, dict) and "input_ids" in test_res:
ids = test_res["input_ids"]
elif hasattr(test_res, "input_ids"):
ids = test_res.input_ids
elif isinstance(test_res, list):
ids = test_res
else:
ids = []
if ids:
eos_id = getattr(raw_tokenizer, "eos_token_id", None)
if eos_id is not None and ids[-1] == eos_id:
has_eos = True
elif ids[-1] == 1:
has_eos = True
except Exception:
pass
eos_adj = 1 if has_eos else 0 eos_adj = 1 if has_eos else 0
prev_len = len(raw_tokenizer(global_prompt)["input_ids"]) - eos_adj prev_len = len(raw_tokenizer(global_prompt)["input_ids"]) - eos_adj

View File

@@ -1,36 +1,14 @@
[project] [project]
name = "WhatDreamsCost-ComfyUI" name = "whatdreamscost-csglide"
description = "A variety of custom ComfyUI nodes and workflows for creatives." description = "LTX Director (CS) — modded LTX Director with reference-sheet / @char / MSR + provider Analyze backend. Coexists with the original WhatDreamsCost-ComfyUI without collisions."
version = "1.3.9" version = "2.0.0"
license = {file = "LICENSE"} license = {file = "LICENSE"}
# classifiers = [
# # For OS-independent nodes (works on all operating systems)
# "Operating System :: OS Independent",
#
# # OR for OS-specific nodes, specify the supported systems:
# "Operating System :: Microsoft :: Windows", # Windows specific
# "Operating System :: POSIX :: Linux", # Linux specific
# "Operating System :: MacOS", # macOS specific
#
# # GPU Accelerator support. Pick the ones that are supported by your extension.
# "Environment :: GPU :: NVIDIA CUDA", # NVIDIA CUDA support
# "Environment :: GPU :: AMD ROCm", # AMD ROCm support
# "Environment :: GPU :: Intel Arc", # Intel Arc support
# "Environment :: NPU :: Huawei Ascend", # Huawei Ascend support
# "Environment :: GPU :: Apple Metal", # Apple Metal support
# ]
[project.urls] [project.urls]
Repository = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI" Repository = "https://github.com/CGlide/WhatDreamsCost-CSGlide"
# Used by Comfy Registry https://registry.comfy.org
Documentation = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/wiki"
"Bug Tracker" = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/issues"
[tool.comfy] [tool.comfy]
PublisherId = "whatdreamscost" PublisherId = "REPLACE_WITH_YOUR_REGISTRY_PUBLISHER_ID"
DisplayName = "WhatDreamsCost-ComfyUI" DisplayName = "WhatDreamsCost-CSGlide"
Icon = "https://raw.githubusercontent.com/WhatDreamsCost/MediaSyncer/refs/heads/main/Icon.png" Icon = ""
includes = [] includes = []
# "requires-comfyui" = ">=1.0.0" # ComfyUI version compatibility

View File

@@ -1,51 +0,0 @@
import re
import math
class SpeechLengthCalculator:
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"text": ("STRING", {"multiline": True, "default": 'Enter your script here. "Make sure to put spoken words inside quotes!"'}),
"fps": ("INT", {"default": 24, "min": 1, "max": 120, "step": 1}),
"additional_time": ("FLOAT", {"default": 0.0, "min": 0.0, "step": 0.1}),
},
"optional": {
"text_input": ("STRING", {"forceInput": True}),
}
}
# Added "STRING" to RETURN_TYPES
RETURN_TYPES = ("INT", "INT", "INT", "STRING")
# Added "text" to RETURN_NAMES
RETURN_NAMES = ("slow_frame_count", "average_frame_count", "fast_frame_count", "text")
FUNCTION = "calculate_speech"
CATEGORY = "WhatDreamsCost"
def calculate_speech(self, text, fps, additional_time=0.0, text_input=None):
# Prioritize the connected text_input if provided, otherwise fallback to the text widget
active_text = text_input if (text_input is not None and isinstance(text_input, str) and text_input.strip() != "") else text
# Regex to find words inside double quotes, single quotes, or smart quotes
matches = re.findall(r'"([^"]*)"|\'([^\']*)\'|“([^”]*)”|([^]*)', active_text)
# Extract matches, handling all possible captured groups from the regex
quoted_text = " ".join([next((g for g in m if g), "") for m in matches])
# Split by whitespace to get words and count them
words = quoted_text.split()
word_count = len(words)
def calc_frames(wpm):
if word_count == 0 and additional_time == 0:
return 0
minutes = word_count / wpm
seconds = (minutes * 60) + additional_time
return math.ceil(seconds * fps)
slow_frames = calc_frames(100)
avg_frames = calc_frames(130)
fast_frames = calc_frames(160)
# Added active_text as the 4th returned value
return (slow_frames, avg_frames, fast_frames, active_text)