chris.dumas/WhatDreamsCost-Dumas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: chris.dumas:main
Branches Tags
chris.dumas:main_dumas chris.dumas:main_cs chris.dumas:main
chris.dumas:0.20 chris.dumas:0.13 chris.dumas:0.12 chris.dumas:0.11
...
compare: chris.dumas:0.11
Branches Tags
chris.dumas:main_dumas chris.dumas:main_cs chris.dumas:main
chris.dumas:0.20 chris.dumas:0.13 chris.dumas:0.12 chris.dumas:0.11

5 Commits
main ... 0.11

Author SHA1 Message Date
CGlide
32a8bfa511 Add files via upload 2026-06-02 12:22:23 +02:00
CGlide
0604d7fb0e Add files via upload 2026-05-24 16:03:13 +02:00
CGlide
0f90ddd5da Add files via upload 2026-05-24 16:02:32 +02:00
CGlide
04b51c81f2 Add files via upload 2026-05-23 18:44:58 +02:00
CGlide
9f2807826e Add files via upload 2026-05-22 16:29:52 +02:00
42 changed files with 40765 additions and 40249 deletions
Expand all files Collapse all files

1348
LICENSE
View File

File diff suppressed because it is too large Load Diff

480
README.md
View File

@@ -1,242 +1,238 @@
# Overview # Overview
This will be a collection of free resources for ComfyUI. This will be a collection of free resources for ComfyUI.
Hopefully it will make creating cool stuff easier. Hopefully it will make creating cool stuff easier.
All of my nodes are created with the help of AI, so there may or may not be redundant, messy code. All of my nodes are created with the help of AI, so there may or may not be redundant, messy code.
## ▶️ YouTube Tutorial Videos ## ▶️ YouTube Tutorial Videos
<table> <table>
<tr> <tr>
<td> <td>
<p align="center">LTX Director Trailer</p> <p align="center">LTX Director Trailer</p>
<a href="https://www.youtube.com/watch?v=fZgtkRcu4_k"> <a href="https://www.youtube.com/watch?v=fZgtkRcu4_k">
<img src="https://img.youtube.com/vi/fZgtkRcu4_k/0.jpg" alt="LTX Director Trailer" width="400"> <img src="https://img.youtube.com/vi/fZgtkRcu4_k/0.jpg" alt="LTX Director Trailer" width="400">
</a> </a>
</td> </td>
<td> <td>
<p align="center">LTX Director Tutorial</p> <p align="center">LTX Director Tutorial</p>
<a href="https://www.youtube.com/watch?v=vM60pJJqqEI"> <a href="https://www.youtube.com/watch?v=vM60pJJqqEI">
<img src="https://img.youtube.com/vi/vM60pJJqqEI/0.jpg" alt="LTX Director Tutorial" width="400"> <img src="https://img.youtube.com/vi/vM60pJJqqEI/0.jpg" alt="LTX Director Tutorial" width="400">
</a> </a>
</td> </td>
</tr> </tr>
</table> </table>
## ❓ How to install nodes ## ❓ How to install nodes
- Navigate to your `/ComfyUI/custom_nodes/ folder` - Navigate to your `/ComfyUI/custom_nodes/ folder`
- Run `git clone https://github.com/WhatDreamscost/WhatDreamsCost-ComfyUI` - Run `git clone https://github.com/WhatDreamscost/WhatDreamsCost-ComfyUI`
- Or download through the ComfyUI Manager. - Or download through the ComfyUI Manager.
**❗❗IMPORTANT❗❗** **❗❗IMPORTANT❗❗**
If you don't see the latest version (v1.3.9) yet in the manager then just downloaded the nightly version (or fetch the updates to update the list to see the latest version). If you don't see the latest version (v1.3.5) yet in the manager then just downloaded the nightly version (or fetch the updates to update the list to see the latest version).
Also you will need to update ComfyUI-LTXVideo and ComfyUI-KJNodes to the latest version as well. You cannot use this node without updating ComfyUI-LTXVideo! Also you will need to update ComfyUI-LTXVideo and ComfyUI-KJNodes to the latest version as well. You cannot use this node without updating ComfyUI-LTXVideo!
# 🔄 Recent Updates # 🔄 Recent Updates
**v1.3.9**
* **Fixed recent updates not showing in the manager** **v1.3.3**
* **LTX Director Hotfix 2**
It took like 5 tries but I finally got it working 🤦‍♂️ - Fixed duration_seconds input issue.
- Made both duration widgets visible at all times now
**v1.3.3** - Implemented audio latent fix to improve compatibility
* **LTX Director Hotfix 2**
- Fixed duration_seconds input issue.
- Made both duration widgets visible at all times now **v1.3.2**
- Implemented audio latent fix to improve compatibility * **LTX Director Hotfix**
- Fixed epsilon input overlapping custom_width input
- Fixed invisible widgets in nodes 2.0 when toggling widget visibility through settings menu
**v1.3.2**
* **LTX Director Hotfix** If anyone finds anymore bugs or has idea for improvements please let me know!
- Fixed epsilon input overlapping custom_width input
- Fixed invisible widgets in nodes 2.0 when toggling widget visibility through settings menu
**v1.3.1**
If anyone finds anymore bugs or has idea for improvements please let me know! * **LTX Director Example Workflow Fix**
- Minor fix to the example workflow (i forgot to set the clip loader type to ltxv lol)
**v1.3.1** **v1.3.0**
* **LTX Director Example Workflow Fix** * **New nodes: LTX Director and LTX Director Guide**
- Minor fix to the example workflow (i forgot to set the clip loader type to ltxv lol) - A complete timeline editor that can do almost everything. It's my most ambitious node so far and the successor to LTX Sequencer/Multi Image Loader.
**v1.3.0** **v1.2.9**
* **New nodes: LTX Director and LTX Director Guide** * **Fixed every known issue with Multi Image Loader and added text output to Speech Length Calculator**
- A complete timeline editor that can do almost everything. It's my most ambitious node so far and the successor to LTX Sequencer/Multi Image Loader.
- Removed the completely useless drag and drop animations (now it's snappy and no longer finicky)
**v1.2.9** - Fixed the node resizing on nodes 2.0
* **Fixed every known issue with Multi Image Loader and added text output to Speech Length Calculator** - Updated grid logic to fit images better
- Added ablity to right click images to copy/open/save images
- Removed the completely useless drag and drop animations (now it's snappy and no longer finicky) - Fixed the "invisible hitbox" underneath node issue (actually this time).
- Fixed the node resizing on nodes 2.0
- Updated grid logic to fit images better Also added a text output to the Speech Length Calculator node (can't believe i didn't do this initially)
- Added ablity to right click images to copy/open/save images
- Fixed the "invisible hitbox" underneath node issue (actually this time). <details>
<summary>Click to view older Updates</summary>
Also added a text output to the Speech Length Calculator node (can't believe i didn't do this initially)
**v1.2.8**
<details> * **Updated Load Video UI and Color Conversion**
<summary>Click to view older Updates</summary> * Added crop mode, a simple interface to crop videos. It also include various aspect ratio presets.
* Updated color conversion to ensure colors are as accurate as possible. Will first check metadata for colorspace, and if metadata is missing then it will guess the colorspace based on video dimensions.
**v1.2.8** * Updated display mode toggle UI to be more understandable
* **Updated Load Video UI and Color Conversion**
* Added crop mode, a simple interface to crop videos. It also include various aspect ratio presets. **v1.2.7**
* Updated color conversion to ensure colors are as accurate as possible. Will first check metadata for colorspace, and if metadata is missing then it will guess the colorspace based on video dimensions. * **New Node: Load Video UI**
* Updated display mode toggle UI to be more understandable
Custom Node to Trim, Resize, and Preview Videos in Realtime
**v1.2.7**
* **New Node: Load Video UI** **v1.2.6**
* **Updated Speech Length Calculator UI**
Custom Node to Trim, Resize, and Preview Videos in Realtime
Also added duration output to the Load Audio UI node
**v1.2.6**
* **Updated Speech Length Calculator UI** **v1.2.5**
* **Updated Load Audio UI Node**
Also added duration output to the Load Audio UI node * Added Duration Setting
* Made the whole selection bar draggable
**v1.2.5** * Fixed Trimmed UI to show centiseconds
* **Updated Load Audio UI Node**
* Added Duration Setting **v1.2.4**
* Made the whole selection bar draggable * **New Node: Load Audio UI**
* Fixed Trimmed UI to show centiseconds
Overhaul of the load audio node. Features a simple interface to easily trim audio. Also allows dragging and dropping files (fixes the original node that doesn't allow dropping in videos). Also compatible with nodes 2.0.
**v1.2.4**
* **New Node: Load Audio UI** **v1.2.3**
* **Workflow Update + Minor Bug Fix**
Overhaul of the load audio node. Features a simple interface to easily trim audio. Also allows dragging and dropping files (fixes the original node that doesn't allow dropping in videos). Also compatible with nodes 2.0. * Added new workflow that is compatible with the latest ComfyUI version (as of 4/27/26). The new workflow also included an option to include custom audio, and has minor improvements of the previous workflows.
* Fixed minor bug with Multi Image Loader that blocked mouse input in a small area under the node 🤷‍♂️
**v1.2.3**
* **Workflow Update + Minor Bug Fix** **v1.2.0**
* Added new workflow that is compatible with the latest ComfyUI version (as of 4/27/26). The new workflow also included an option to include custom audio, and has minor improvements of the previous workflows. * **New Node: Speech Length Calculator**
* Fixed minor bug with Multi Image Loader that blocked mouse input in a small area under the node 🤷‍♂️
Automatically output in realtime how long a video should be based on the dialouge.
**v1.2.0**
* **New Node: Speech Length Calculator** **v1.1.0**
* Added resize_method to the Multi Image Loader node for more resize options
Automatically output in realtime how long a video should be based on the dialouge. * Added insert_mode which allows you to enter in seconds instead of frames on the LTX Sequencer node
* Updated workflows with more notes
**v1.1.0** * Re-added tiny vae to workflows
* Added resize_method to the Multi Image Loader node for more resize options * Fixed various bugs
* Added insert_mode which allows you to enter in seconds instead of frames on the LTX Sequencer node * more things i can't rememeber
* Updated workflows with more notes
* Re-added tiny vae to workflows **This update will change the node layouts, so be sure to update your workflows or else they won't work properly.**
* Fixed various bugs
* more things i can't rememeber ❗❗❗ **New Tutorial on using these nodes available: https://www.youtube.com/watch?v=aXDIr8eNovI** ❗❗❗
</details>
**This update will change the node layouts, so be sure to update your workflows or else they won't work properly.**
# ⚙️ Custom Nodes
❗❗❗ **New Tutorial on using these nodes available: https://www.youtube.com/watch?v=aXDIr8eNovI** ❗❗❗
</details>
## LTX Director
# ⚙️ Custom Nodes <img width="1481" height="833" alt="Clipboard Image (2)" src="https://github.com/user-attachments/assets/08f3fe53-9393-4f5d-9de5-58b229fbed47" />
A Complete Timeline Editor For LTX 2.3. This is the sucessor of my previous nodes, and has loads of features in it. It was originally based off of [Kijai's Prompt Relay node](https://github.com/kijai/ComfyUI-PromptRelay) and my LTX Sequencer/Multi Image Loader nodes.
## LTX Director
<img width="1481" height="833" alt="Clipboard Image (2)" src="https://github.com/user-attachments/assets/08f3fe53-9393-4f5d-9de5-58b229fbed47" /> **Main Features:**
- **Fully Functional Timeline Editor:** I spent hours studying various video editors and ended up with this design. If anyone has ideas for improvements let me know! I will adding documentation on all the functions soon.
A Complete Timeline Editor For LTX 2.3. This is the sucessor of my previous nodes, and has loads of features in it. It was originally based off of [Kijai's Prompt Relay node](https://github.com/kijai/ComfyUI-PromptRelay) and my LTX Sequencer/Multi Image Loader nodes. - **Prompt Relay integrated:** This unlocks the ability to have granular control over video generation. For more information on Prompt Relay go here, https://gordonchen19.github.io/Prompt-Relay/
- **First, Middle, Last Frame Support:** This has by far the easiest method of creating first/last frames videos. It supports any number of keyframes, and will be the successor of my previous nodes.
**Main Features:** - **Custom Audio Support:** Import, trim, and combine your own audio clips in this node. Enabling custom audio is as simple as clicking 1 button. It is also compatible with every other feature in the node, include first/last frames, t2v, i2v, and prompt relay.
- **Fully Functional Timeline Editor:** I spent hours studying various video editors and ended up with this design. If anyone has ideas for improvements let me know! I will adding documentation on all the functions soon. - **Image to Video:** Part of the goal of this node was to make it easier to do everything, including Image to Video. It has built in resize functionality, and of course all the benifits of the prompt relay and custom audio integration.
- **Prompt Relay integrated:** This unlocks the ability to have granular control over video generation. For more information on Prompt Relay go here, https://gordonchen19.github.io/Prompt-Relay/ - **Text to Video:** Use text segments to create T2V videos. Compatible with all other features of the node.
- **First, Middle, Last Frame Support:** This has by far the easiest method of creating first/last frames videos. It supports any number of keyframes, and will be the successor of my previous nodes.
- **Custom Audio Support:** Import, trim, and combine your own audio clips in this node. Enabling custom audio is as simple as clicking 1 button. It is also compatible with every other feature in the node, include first/last frames, t2v, i2v, and prompt relay. Download workflows here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/tree/main/example_workflows
- **Image to Video:** Part of the goal of this node was to make it easier to do everything, including Image to Video. It has built in resize functionality, and of course all the benifits of the prompt relay and custom audio integration.
- **Text to Video:** Use text segments to create T2V videos. Compatible with all other features of the node. **Tutorial videos and documentation coming soon**
Download workflows here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/tree/main/example_workflows
## Multi Image Loader
**Tutorial videos and documentation coming soon** <img width="1280" height="720" alt="Multi_Image_Loader_Wide_Gif" src="https://github.com/user-attachments/assets/99b6afd8-5197-4e6c-81da-a7bd156c42c7" />
An Image loader that features a built in gallery, allowing your to easily rearrange images and output them seperately or batched together. It also combines the image resize node and LTXVPreprocess node to reduce clutter in LTX workflows.
## Multi Image Loader
<img width="1280" height="720" alt="Multi_Image_Loader_Wide_Gif" src="https://github.com/user-attachments/assets/99b6afd8-5197-4e6c-81da-a7bd156c42c7" /> ## LTX Sequencer
![LTX_Sequencer_GIF](https://github.com/user-attachments/assets/88f27155-f50e-4cb2-b937-ab173e6bdf0b)
An Image loader that features a built in gallery, allowing your to easily rearrange images and output them seperately or batched together. It also combines the image resize node and LTXVPreprocess node to reduce clutter in LTX workflows.
An overhaul of the LTXVAddGuideMulti node. It allows you to quickly create FFLF (First Frame Last Frame) videos, shot sequences, supports any number of middle frames.
## LTX Sequencer
![LTX_Sequencer_GIF](https://github.com/user-attachments/assets/88f27155-f50e-4cb2-b937-ab173e6bdf0b) Connect the Multi Image Loader node's multi_output to automatically update the node's widgets.
An overhaul of the LTXVAddGuideMulti node. It allows you to quickly create FFLF (First Frame Last Frame) videos, shot sequences, supports any number of middle frames. It also has a sync feature that syncs all LTX Sequencer nodes together in realtime, removing the need to edit every single node manually every time you want to make a change to something.
Connect the Multi Image Loader node's multi_output to automatically update the node's widgets.
## LTX Keyframer
It also has a sync feature that syncs all LTX Sequencer nodes together in realtime, removing the need to edit every single node manually every time you want to make a change to something. <img width="1082" height="608" alt="LTX Keyframer Wide" src="https://github.com/user-attachments/assets/850ba4a2-dbca-4e5a-a580-1c271e9f0c41" />
An overhaul of the LTXVImgToVideoInplaceKJ node. It allows you to quickly create FFLF (First Frame Last Frame) videos and shot sequences. Also upports any number of middle frames.
## LTX Keyframer
<img width="1082" height="608" alt="LTX Keyframer Wide" src="https://github.com/user-attachments/assets/850ba4a2-dbca-4e5a-a580-1c271e9f0c41" /> Connect the Multi Image Loader node's multi_output to automatically update the node's widgets.
An overhaul of the LTXVImgToVideoInplaceKJ node. It allows you to quickly create FFLF (First Frame Last Frame) videos and shot sequences. Also upports any number of middle frames. It also has a sync feature that syncs all LTX Keyframer nodes together in realtime, removing the need to edit every single node manually every time you want to make a change to something.
Connect the Multi Image Loader node's multi_output to automatically update the node's widgets. **I would recommend using the LTX Sequencer Node over this node, after further testing it seems superior in at pretty much everything. I'll leave it in just in case more people want to test it**
It also has a sync feature that syncs all LTX Keyframer nodes together in realtime, removing the need to edit every single node manually every time you want to make a change to something. ## Speech Length Calculator
<img width="1280" height="720" alt="Speech Length Calculator v2 Gif" src="https://github.com/user-attachments/assets/04b9a1cf-20e4-4b7b-a9c6-4a5a0825995b" />
**I would recommend using the LTX Sequencer Node over this node, after further testing it seems superior in at pretty much everything. I'll leave it in just in case more people want to test it** <br>
<br>
## Speech Length Calculator This node calculates in realtime how long a video should be based on the dialogue. Any words in quotations will be considered as speech. The node updates in realtime without having to run the workflow, and outputs the length depending on how fast the speech is.
<img width="1280" height="720" alt="Speech Length Calculator v2 Gif" src="https://github.com/user-attachments/assets/04b9a1cf-20e4-4b7b-a9c6-4a5a0825995b" />
<br> If you connect another string/text node to the text_input, it will still update in the length in realtime.
<br>
This node calculates in realtime how long a video should be based on the dialogue. Any words in quotations will be considered as speech. The node updates in realtime without having to run the workflow, and outputs the length depending on how fast the speech is. I kept having to play the guessing game on my own generations so I made this node to make it easier :man_shrugging:
If you connect another string/text node to the text_input, it will still update in the length in realtime. ## Load Video UI
<table width="100%">
I kept having to play the guessing game on my own generations so I made this node to make it easier :man_shrugging: <tr>
<td width="50%" align="center">
## Load Video UI <p>Simple Controls</p>
<table width="100%"> <img src="https://github.com/user-attachments/assets/fb76ff03-a6ff-4837-bd63-7e429f5f3d37" width="100%" />
<tr> </td>
<td width="50%" align="center"> <td width="50%" align="center">
<p>Simple Controls</p> <p>New Crop Mode!</p>
<img src="https://github.com/user-attachments/assets/fb76ff03-a6ff-4837-bd63-7e429f5f3d37" width="100%" /> <img src="https://github.com/user-attachments/assets/28cfb4ca-e42a-44da-9afb-f20cb01b9722" width="100%" />
</td> </td>
<td width="50%" align="center"> </tr>
<p>New Crop Mode!</p> </table>
<img src="https://github.com/user-attachments/assets/28cfb4ca-e42a-44da-9afb-f20cb01b9722" width="100%" />
</td> <br>
</tr> <br>
</table> An upgraded Load Video node. It has the following features:
<br> * Simple interface to quickly trim videos and preview them in realtime.
<br> * Ability to load any length of video into the node (the default load video node was limited to 100MB files)
An upgraded Load Video node. It has the following features: * Easily switch between showing seconds and frames with a toggle button. This will change the widgets as well as the interface.
* Multiple options for resizing the video (maintain aspect ratio, crop, stretch to fit, pad)
* Simple interface to quickly trim videos and preview them in realtime. * Allows dragging and dropping files into the node
* Ability to load any length of video into the node (the default load video node was limited to 100MB files) * Progress bar
* Easily switch between showing seconds and frames with a toggle button. This will change the widgets as well as the interface. * Optimized to use less RAM (still very limited due to ComfyUI limitations, but at least a little more efficient)
* Multiple options for resizing the video (maintain aspect ratio, crop, stretch to fit, pad)
* Allows dragging and dropping files into the node Please note that due to ComfyUI limitations (and the fact that this node doesn't use any addtional libraries), this node will not work well for outputting large videos. You can trim any length of video without a problem, but if the output is still large it will end up using a lot of RAM. I have implemented various optimizations though to make it use less memory.
* Progress bar
* Optimized to use less RAM (still very limited due to ComfyUI limitations, but at least a little more efficient) ## Load Audio UI
<img width="1280" height="720" alt="Load_Audio_UI_V2" src="https://github.com/user-attachments/assets/e3dc5c8d-d0b9-4336-8196-944204719239" />
Please note that due to ComfyUI limitations (and the fact that this node doesn't use any addtional libraries), this node will not work well for outputting large videos. You can trim any length of video without a problem, but if the output is still large it will end up using a lot of RAM. I have implemented various optimizations though to make it use less memory. <br>
<br>
## Load Audio UI An upgraded Load Audio node. Features a simple interface to easily trim audio. Also allows dragging and dropping files (fixes the original node that doesn't allow dropping in videos). Also compatible with nodes 2.0.
<img width="1280" height="720" alt="Load_Audio_UI_V2" src="https://github.com/user-attachments/assets/e3dc5c8d-d0b9-4336-8196-944204719239" />
<br> # 💡 Workflows
<br> <img width="3120" height="990" alt="LTX I2V First Last Frame 3 Stage Workflow v6" src="https://github.com/user-attachments/assets/c993ef2f-ac4b-4091-a7f6-5ff1674c3718" />
An upgraded Load Audio node. Features a simple interface to easily trim audio. Also allows dragging and dropping files (fixes the original node that doesn't allow dropping in videos). Also compatible with nodes 2.0. <br>
<br>
# 💡 Workflows This is a compact LTX 2.3 workflow for I2V and First Frame, Middle Frame, Last frame video generation.
<img width="3120" height="990" alt="LTX I2V First Last Frame 3 Stage Workflow v6" src="https://github.com/user-attachments/assets/c993ef2f-ac4b-4091-a7f6-5ff1674c3718" /> I seperated and organized everything into subraphs to make things as clean as possible, and added toggles to customize the workflow quickly.
<br>
<br> Download workflows here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/tree/main/example_workflows
This is a compact LTX 2.3 workflow for I2V and First Frame, Middle Frame, Last frame video generation.
I seperated and organized everything into subraphs to make things as clean as possible, and added toggles to customize the workflow quickly. Or drag and drop the image into ComfyUI to import workflow.
Download workflows here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/tree/main/example_workflows # ❗ Known Issues
Or drag and drop the image into ComfyUI to import workflow. Fixed everything so far. If there are any other issue or bugs you find please let me know!
# ❗ Known Issues # 💡 Additional Info
Fixed everything so far. If there are any other issue or bugs you find please let me know! I made these nodes knowing little about python and a beginner level understanding of javascript. Feel free to suggest improvements, and if you run into any bugs let me know.
# 💡 Additional Info For those asking, I mainly used gemini to create these nodes.
I made these nodes knowing little about python and a beginner level understanding of javascript. Feel free to suggest improvements, and if you run into any bugs let me know.
For those asking, I mainly used gemini to create these nodes.

95
__init__.py
View File

@@ -1,47 +1,50 @@
from .ltx_keyframer import LTXKeyframer from .ltx_keyframer import LTXKeyframer
from .multi_image_loader import MultiImageLoader from .multi_image_loader import MultiImageLoader
from .ltx_sequencer import LTXSequencer from .ltx_sequencer import LTXSequencer
from .speech_length_calculator import SpeechLengthCalculator from .speech_length_calculator import SpeechLengthCalculator
from .load_audio_ui import LoadAudioUI from .load_audio_ui import LoadAudioUI
from .load_video_ui import LoadVideoUI from .load_video_ui import LoadVideoUI
from .ltx_director import LTXDirector from .ltx_director import LTXDirector
from .ltx_director_guide import LTXDirectorGuide from .ltx_director_guide import LTXDirectorGuide
from comfy_api.latest import ComfyExtension, io from comfy_api.latest import ComfyExtension, io
from typing_extensions import override from typing_extensions import override
from .latent_slice import CleanLatentSlice
class PromptRelay(ComfyExtension):
@override class PromptRelay(ComfyExtension):
async def get_node_list(self) -> list[type[io.ComfyNode]]: @override
return [ async def get_node_list(self) -> list[type[io.ComfyNode]]:
LTXDirector, return [
LTXDirectorGuide LTXDirector,
] LTXDirectorGuide
]
async def comfy_entrypoint() -> PromptRelay:
return PromptRelay() async def comfy_entrypoint() -> PromptRelay:
return PromptRelay()
NODE_CLASS_MAPPINGS = {
"LTXKeyframer": LTXKeyframer, NODE_CLASS_MAPPINGS = {
"MultiImageLoader": MultiImageLoader, "LTXKeyframer": LTXKeyframer,
"LTXSequencer": LTXSequencer, "MultiImageLoader": MultiImageLoader,
"SpeechLengthCalculator": SpeechLengthCalculator, "LTXSequencer": LTXSequencer,
"LoadAudioUI": LoadAudioUI, "SpeechLengthCalculator": SpeechLengthCalculator,
"LoadVideoUI": LoadVideoUI, "LoadAudioUI": LoadAudioUI,
"LTXDirector": LTXDirector, "LoadVideoUI": LoadVideoUI,
"LTXDirectorGuide": LTXDirectorGuide, "LTXDirector": LTXDirector,
} "LTXDirectorGuide": LTXDirectorGuide,
"CleanLatentSlice": CleanLatentSlice,
NODE_DISPLAY_NAME_MAPPINGS = { }
"LTXKeyframer": "LTX Keyframer",
"MultiImageLoader": "Multi Image Loader", NODE_DISPLAY_NAME_MAPPINGS = {
"LTXSequencer": "LTX Sequencer", "LTXKeyframer": "LTX Keyframer",
"SpeechLengthCalculator": "Speech Length Calculator", "MultiImageLoader": "Multi Image Loader",
"LoadAudioUI": "Load Audio UI", "LTXSequencer": "LTX Sequencer",
"LoadVideoUI": "Load Video UI", "SpeechLengthCalculator": "Speech Length Calculator",
"LTXDirector": "LTX Director", "LoadAudioUI": "Load Audio UI",
"LTXDirectorGuide": "LTX Director Guide", "LoadVideoUI": "Load Video UI",
} "LTXDirector": "LTX Director",
"LTXDirectorGuide": "LTX Director Guide",
WEB_DIRECTORY = "./js" "CleanLatentSlice": "Clean Latent Slice",
}
WEB_DIRECTORY = "./js"
__all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS', 'WEB_DIRECTORY'] __all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS', 'WEB_DIRECTORY']

BIN
__pycache__/__init__.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
__pycache__/latent_slice.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
__pycache__/load_audio_ui.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
__pycache__/load_video_ui.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
__pycache__/ltx_director.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
__pycache__/ltx_director_guide.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
__pycache__/ltx_keyframer.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
__pycache__/ltx_sequencer.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
__pycache__/multi_image_loader.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
__pycache__/patches.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
__pycache__/prompt_relay.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
__pycache__/speech_length_calculator.cpython-312.pyc Normal file
View File

Binary file not shown.

4672
example_workflows/LTX Director Example Workflow (Fixed).json
View File

File diff suppressed because it is too large Load Diff

7326
example_workflows/LTX Director Example Workflow Subgraphs v2.json
View File

File diff suppressed because it is too large Load Diff

1
example_workflows/LTX Director Workflow_cs.json Normal file
View File

File diff suppressed because one or more lines are too long

1
example_workflows/LTX Director Workflow_v2.json Normal file
View File

File diff suppressed because one or more lines are too long

15026
example_workflows/LTX I2V FFLF Custom Audio Workflow - SUPPORTS LATEST COMFYUI VERSION - V3.json
View File

File diff suppressed because it is too large Load Diff

15828
example_workflows/LTX I2V First Last Frame 2 Stage Workflow v6.json
View File

File diff suppressed because it is too large Load Diff

15830
example_workflows/LTX I2V First Last Frame 3 Stage Workflow v6.json
View File

File diff suppressed because it is too large Load Diff

1042
js/load_audio_ui.js
View File

File diff suppressed because it is too large Load Diff

3026
js/load_video_ui.js
View File

File diff suppressed because it is too large Load Diff

7932
js/ltx_director.js
View File

File diff suppressed because it is too large Load Diff

32
js/ltx_director_guide.js
View File

@@ -1,17 +1,17 @@
import { app } from "../../scripts/app.js"; import { app } from "../../scripts/app.js";
// LTX Director Guide is a pure pass-through processor node. // LTX Director Guide is a pure pass-through processor node.
// All configuration (images, insert frames, strengths) comes from // All configuration (images, insert frames, strengths) comes from
// the guide_data output of Prompt Relay Encode (Timeline). // the guide_data output of Prompt Relay Encode (Timeline).
// No dynamic widgets or sync logic needed. // No dynamic widgets or sync logic needed.
app.registerExtension({ app.registerExtension({
name: "Comfy.LTXDirectorGuide", name: "Comfy.LTXDirectorGuide",
async nodeCreated(node) { async nodeCreated(node) {
if (node.comfyClass !== "LTXDirectorGuide") return; if (node.comfyClass !== "LTXDirectorGuide") return;
// Nothing to initialize — the node has no configurable widgets. // Nothing to initialize — the node has no configurable widgets.
}, },
}); });
// Nothing to initialize — the node has no configurable widgets. // Nothing to initialize — the node has no configurable widgets.
}, },
}); });

836
js/ltx_keyframer.js
View File

@@ -1,419 +1,419 @@
import { app } from "../../scripts/app.js"; import { app } from "../../scripts/app.js";
// Global registry to track all LTXKeyframer nodes across all subgraphs // Global registry to track all LTXKeyframer nodes across all subgraphs
window._LTXKeyframerGlobalNodes = window._LTXKeyframerGlobalNodes || new Set(); window._LTXKeyframerGlobalNodes = window._LTXKeyframerGlobalNodes || new Set();
// ComfyUI native trick to cleanly hide/show widgets without deleting them // ComfyUI native trick to cleanly hide/show widgets without deleting them
function toggleWidget(widget, visible) { function toggleWidget(widget, visible) {
if (visible) { if (visible) {
if (widget.origType !== undefined) { if (widget.origType !== undefined) {
widget.type = widget.origType; widget.type = widget.origType;
widget.computeSize = widget.origComputeSize; widget.computeSize = widget.origComputeSize;
delete widget.origType; delete widget.origType;
delete widget.origComputeSize; delete widget.origComputeSize;
} }
} else { } else {
if (widget.type !== "hidden") { if (widget.type !== "hidden") {
widget.origType = widget.type; widget.origType = widget.type;
widget.origComputeSize = widget.computeSize; widget.origComputeSize = widget.computeSize;
widget.type = "hidden"; widget.type = "hidden";
widget.computeSize = () => [0, -4]; widget.computeSize = () => [0, -4];
} }
} }
} }
// --- NEW SYNC HELPER FUNCTION --- // --- NEW SYNC HELPER FUNCTION ---
// Finds all other LTXKeyframer nodes globally and mirrors the value to them // Finds all other LTXKeyframer nodes globally and mirrors the value to them
function syncWidgetAcrossNodes(sourceNode, widgetName, value) { function syncWidgetAcrossNodes(sourceNode, widgetName, value) {
if (!window._LTXKeyframerGlobalNodes) return; if (!window._LTXKeyframerGlobalNodes) return;
for (const targetNode of window._LTXKeyframerGlobalNodes) { for (const targetNode of window._LTXKeyframerGlobalNodes) {
// Target all OTHER LTXKeyframer nodes by direct object reference // Target all OTHER LTXKeyframer nodes by direct object reference
if (targetNode !== sourceNode) { if (targetNode !== sourceNode) {
// 1. Always update the hidden properties cache so it remembers the sync // 1. Always update the hidden properties cache so it remembers the sync
// even if the widget isn't currently visible (e.g. fewer images loaded right now) // even if the widget isn't currently visible (e.g. fewer images loaded right now)
targetNode.properties[widgetName] = value; targetNode.properties[widgetName] = value;
// 2. If the widget is currently visible on the UI, update it visually // 2. If the widget is currently visible on the UI, update it visually
if (targetNode.widgets) { if (targetNode.widgets) {
const targetWidget = targetNode.widgets.find(w => w.name === widgetName); const targetWidget = targetNode.widgets.find(w => w.name === widgetName);
if (targetWidget && targetWidget.value !== value) { if (targetWidget && targetWidget.value !== value) {
targetWidget.value = value; targetWidget.value = value;
targetNode.setDirtyCanvas(true, false); targetNode.setDirtyCanvas(true, false);
} }
} }
} }
} }
} }
app.registerExtension({ app.registerExtension({
name: "Comfy.LTXKeyframer.DynamicInputs", name: "Comfy.LTXKeyframer.DynamicInputs",
async nodeCreated(node) { async nodeCreated(node) {
if (node.comfyClass !== "LTXKeyframer") return; if (node.comfyClass !== "LTXKeyframer") return;
// Register this node instance globally // Register this node instance globally
window._LTXKeyframerGlobalNodes.add(node); window._LTXKeyframerGlobalNodes.add(node);
node._currentImageCount = -1; // Force first update node._currentImageCount = -1; // Force first update
// Initialize persistent properties cache // Initialize persistent properties cache
node.properties = node.properties || {}; node.properties = node.properties || {};
// Add subtle separator line above images_loaded // Add subtle separator line above images_loaded
node.addCustomWidget({ node.addCustomWidget({
name: "num_images_separator", name: "num_images_separator",
type: "text", type: "text",
draw(ctx, node, widget_width, y, widget_height) { draw(ctx, node, widget_width, y, widget_height) {
ctx.save(); ctx.save();
ctx.strokeStyle = "#444"; ctx.strokeStyle = "#444";
ctx.lineWidth = 1; ctx.lineWidth = 1;
ctx.beginPath(); ctx.beginPath();
ctx.moveTo(10, y + 5); ctx.moveTo(10, y + 5);
ctx.lineTo(widget_width - 10, y + 5); ctx.lineTo(widget_width - 10, y + 5);
ctx.stroke(); ctx.stroke();
ctx.restore(); ctx.restore();
}, },
computeSize(width) { computeSize(width) {
return [width, 10]; return [width, 10];
} }
}); });
// Move separator before num_images // Move separator before num_images
const moveSeparator = () => { const moveSeparator = () => {
const idx = node.widgets.findIndex(w => w.name === "num_images"); const idx = node.widgets.findIndex(w => w.name === "num_images");
const sepIdx = node.widgets.findIndex(w => w.name === "num_images_separator"); const sepIdx = node.widgets.findIndex(w => w.name === "num_images_separator");
if (idx !== -1 && sepIdx !== -1) { if (idx !== -1 && sepIdx !== -1) {
const separator = node.widgets.splice(sepIdx, 1)[0]; const separator = node.widgets.splice(sepIdx, 1)[0];
node.widgets.splice(idx, 0, separator); node.widgets.splice(idx, 0, separator);
} }
}; };
setTimeout(moveSeparator, 50); // Small delay to ensure num_images is present setTimeout(moveSeparator, 50); // Small delay to ensure num_images is present
// Core update: synchronize widget visibility to match imageCount // Core update: synchronize widget visibility to match imageCount
node._applyWidgetCount = function(count) { node._applyWidgetCount = function(count) {
const isInitialLoad = this._currentImageCount === -1; const isInitialLoad = this._currentImageCount === -1;
if (this._currentImageCount === count && !isInitialLoad) return; if (this._currentImageCount === count && !isInitialLoad) return;
this._currentImageCount = count; this._currentImageCount = count;
const initialWidth = this.size[0]; const initialWidth = this.size[0];
const numWidget = this.widgets?.find(w => w.name === "num_images"); const numWidget = this.widgets?.find(w => w.name === "num_images");
if (numWidget) { if (numWidget) {
numWidget.label = "images_loaded"; numWidget.label = "images_loaded";
numWidget.value = Math.max(0, Math.min(count || 0, 50)); numWidget.value = Math.max(0, Math.min(count || 0, 50));
} }
// 1. Store current widget values in properties BEFORE removing them // 1. Store current widget values in properties BEFORE removing them
// We skip reading from `this.widgets` on the initial load because it might be scrambling. // We skip reading from `this.widgets` on the initial load because it might be scrambling.
if (!isInitialLoad && this.widgets) { if (!isInitialLoad && this.widgets) {
this.widgets.forEach(w => { this.widgets.forEach(w => {
if (w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) { if (w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) {
this.properties[w.name] = w.value; this.properties[w.name] = w.value;
} }
}); });
} }
// 2. Remove all existing dynamic insert_frame/strength/header widgets // 2. Remove all existing dynamic insert_frame/strength/header widgets
if (this.widgets) { if (this.widgets) {
this.widgets = this.widgets.filter(w => this.widgets = this.widgets.filter(w =>
!w.name.startsWith("insert_frame_") && !w.name.startsWith("insert_frame_") &&
!w.name.startsWith("strength_") && !w.name.startsWith("strength_") &&
!w.name.startsWith("header_") !w.name.startsWith("header_")
); );
} else { } else {
this.widgets = []; this.widgets = [];
} }
// 3. Add back exactly the right amount of widgets using the cached values // 3. Add back exactly the right amount of widgets using the cached values
for (let i = 1; i <= count; i++) { for (let i = 1; i <= count; i++) {
// Add header/separator widget for grouping // Add header/separator widget for grouping
const headerName = `header_${i}`; const headerName = `header_${i}`;
this.addCustomWidget({ this.addCustomWidget({
name: headerName, name: headerName,
type: "text", type: "text",
value: `Image #${i}`, value: `Image #${i}`,
draw(ctx, node, widget_width, y, widget_height) { draw(ctx, node, widget_width, y, widget_height) {
ctx.save(); ctx.save();
const margin = 10; const margin = 10;
const topPadding = 15; const topPadding = 15;
// Subtle separator line // Subtle separator line
ctx.strokeStyle = "#333"; ctx.strokeStyle = "#333";
ctx.lineWidth = 1; ctx.lineWidth = 1;
ctx.beginPath(); ctx.beginPath();
ctx.moveTo(margin, y + 5); ctx.moveTo(margin, y + 5);
ctx.lineTo(widget_width - margin, y + 5); ctx.lineTo(widget_width - margin, y + 5);
ctx.stroke(); ctx.stroke();
// Text label // Text label
ctx.fillStyle = "#dddddd"; // Light gray ctx.fillStyle = "#dddddd"; // Light gray
ctx.font = "bold 12px Arial"; ctx.font = "bold 12px Arial";
ctx.textAlign = "left"; ctx.textAlign = "left";
ctx.fillText(`Image #${i}`, margin, y + topPadding + 10); ctx.fillText(`Image #${i}`, margin, y + topPadding + 10);
ctx.restore(); ctx.restore();
}, },
computeSize(width) { computeSize(width) {
return [width, 35]; // Vertical gap + label height return [width, 35]; // Vertical gap + label height
} }
}); });
const insertFrameWidgetName = `insert_frame_${i}`; const insertFrameWidgetName = `insert_frame_${i}`;
const strengthWidgetName = `strength_${i}`; const strengthWidgetName = `strength_${i}`;
// Add insert_frame widget with Sync Callback // Add insert_frame widget with Sync Callback
const savedInsertFrameValue = this.properties[insertFrameWidgetName]; const savedInsertFrameValue = this.properties[insertFrameWidgetName];
this.addWidget("number", insertFrameWidgetName, this.addWidget("number", insertFrameWidgetName,
savedInsertFrameValue !== undefined ? savedInsertFrameValue : 0, savedInsertFrameValue !== undefined ? savedInsertFrameValue : 0,
(value) => { (value) => {
const rounded = Math.round(value); const rounded = Math.round(value);
this.properties[insertFrameWidgetName] = rounded; this.properties[insertFrameWidgetName] = rounded;
syncWidgetAcrossNodes(this, insertFrameWidgetName, rounded); // Sync out syncWidgetAcrossNodes(this, insertFrameWidgetName, rounded); // Sync out
}, { min: -9999, max: 9999, step: 10, precision: 0 } }, { min: -9999, max: 9999, step: 10, precision: 0 }
); );
// Add strength widget with Sync Callback // Add strength widget with Sync Callback
const savedStrengthValue = this.properties[strengthWidgetName]; const savedStrengthValue = this.properties[strengthWidgetName];
this.addWidget("number", strengthWidgetName, this.addWidget("number", strengthWidgetName,
savedStrengthValue !== undefined ? savedStrengthValue : 1.0, savedStrengthValue !== undefined ? savedStrengthValue : 1.0,
(value) => { (value) => {
this.properties[strengthWidgetName] = value; this.properties[strengthWidgetName] = value;
syncWidgetAcrossNodes(this, strengthWidgetName, value); // Sync out syncWidgetAcrossNodes(this, strengthWidgetName, value); // Sync out
}, { min: 0.0, max: 1.0, step: 0.01 } }, { min: 0.0, max: 1.0, step: 0.01 }
); );
} }
this.setDirtyCanvas(true, true); this.setDirtyCanvas(true, true);
requestAnimationFrame(() => { requestAnimationFrame(() => {
if (this.computeSize) { if (this.computeSize) {
this.setSize(this.computeSize()); this.setSize(this.computeSize());
this.size[0] = initialWidth; // keep width fixed when restructuring this.size[0] = initialWidth; // keep width fixed when restructuring
} }
}); });
}; };
// --- STRICT ARRAY MAPPER: FIXES ALL SHIFTING FOREVER --- // --- STRICT ARRAY MAPPER: FIXES ALL SHIFTING FOREVER ---
// This runs the exact instant the node is loaded, before any UI widgets shift indices. // This runs the exact instant the node is loaded, before any UI widgets shift indices.
// It locks the perfectly mapped array values directly into our properties dictionary. // It locks the perfectly mapped array values directly into our properties dictionary.
const origConfigure = node.configure; const origConfigure = node.configure;
node.configure = function(info) { node.configure = function(info) {
if (origConfigure) { if (origConfigure) {
origConfigure.apply(this, arguments); origConfigure.apply(this, arguments);
} }
if (this.widgets) { if (this.widgets) {
this.widgets.forEach(w => { this.widgets.forEach(w => {
if (w.name === "num_images" || w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) { if (w.name === "num_images" || w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) {
this.properties[w.name] = w.value; this.properties[w.name] = w.value;
} }
}); });
} }
}; };
// Handle deserialization to load properties properly from JSON // Handle deserialization to load properties properly from JSON
const originalOnConfigure = node.onConfigure; const originalOnConfigure = node.onConfigure;
node.onConfigure = function(info) { node.onConfigure = function(info) {
if (originalOnConfigure) { if (originalOnConfigure) {
originalOnConfigure.apply(this, arguments); originalOnConfigure.apply(this, arguments);
} }
if (info.properties) { if (info.properties) {
this.properties = { ...this.properties, ...info.properties }; this.properties = { ...this.properties, ...info.properties };
} }
setTimeout(() => { setTimeout(() => {
const count = readSourceImageCount(this); const count = readSourceImageCount(this);
// Fallback to properties.num_images if source node disconnected // Fallback to properties.num_images if source node disconnected
let targetCount = count !== null ? count : (this.properties.num_images || 0); let targetCount = count !== null ? count : (this.properties.num_images || 0);
this._applyWidgetCount(targetCount); this._applyWidgetCount(targetCount);
}, 100); }, 100);
}; };
// --- STRICT ARRAY GENERATOR --- // --- STRICT ARRAY GENERATOR ---
// Completely detach from ComfyUI's blind visual array saving. // Completely detach from ComfyUI's blind visual array saving.
// We construct an exact 101-element strict array that Python expects. // We construct an exact 101-element strict array that Python expects.
// This makes your node 100% immune to UI/Header index shifting. // This makes your node 100% immune to UI/Header index shifting.
const originalOnSerialize = node.onSerialize; const originalOnSerialize = node.onSerialize;
node.onSerialize = function(info) { node.onSerialize = function(info) {
// Ensure properties are strictly synced with current widget values before building // Ensure properties are strictly synced with current widget values before building
if (this.widgets) { if (this.widgets) {
this.widgets.forEach(w => { this.widgets.forEach(w => {
if (w.name === "num_images" || w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) { if (w.name === "num_images" || w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) {
this.properties[w.name] = w.value; this.properties[w.name] = w.value;
} }
}); });
} }
if (originalOnSerialize) { if (originalOnSerialize) {
originalOnSerialize.apply(this, arguments); originalOnSerialize.apply(this, arguments);
} }
info.properties = { ...this.properties }; info.properties = { ...this.properties };
// Build the exact strict array that maps 1-to-1 to the Python backend // Build the exact strict array that maps 1-to-1 to the Python backend
const strictArray = []; const strictArray = [];
const numWidgetVal = this.properties["num_images"]; const numWidgetVal = this.properties["num_images"];
strictArray.push(numWidgetVal !== undefined ? numWidgetVal : 1); strictArray.push(numWidgetVal !== undefined ? numWidgetVal : 1);
for (let i = 1; i <= 50; i++) { for (let i = 1; i <= 50; i++) {
const fVal = this.properties[`insert_frame_${i}`]; const fVal = this.properties[`insert_frame_${i}`];
const sVal = this.properties[`strength_${i}`]; const sVal = this.properties[`strength_${i}`];
strictArray.push(fVal !== undefined ? fVal : 0); strictArray.push(fVal !== undefined ? fVal : 0);
strictArray.push(sVal !== undefined ? sVal : 1.0); strictArray.push(sVal !== undefined ? sVal : 1.0);
} }
info.widgets_values = strictArray; info.widgets_values = strictArray;
}; };
// Set up manual num_images widget callback // Set up manual num_images widget callback
setTimeout(() => { setTimeout(() => {
const numWidget = node.widgets?.find(w => w.name === "num_images"); const numWidget = node.widgets?.find(w => w.name === "num_images");
if (numWidget) { if (numWidget) {
numWidget.callback = (val) => { numWidget.callback = (val) => {
node.properties["num_images"] = val; node.properties["num_images"] = val;
node._applyWidgetCount(val); node._applyWidgetCount(val);
}; };
} }
}, 100); }, 100);
// Exposed receiver for push-based notifications // Exposed receiver for push-based notifications
node._syncImageCount = function(count) { node._syncImageCount = function(count) {
this._applyWidgetCount(count); this._applyWidgetCount(count);
}; };
// Helper: read image count from a connected MultiImageLoader node // Helper: read image count from a connected MultiImageLoader node
function readSourceImageCount(self) { function readSourceImageCount(self) {
const multiInput = self.inputs?.find(inp => inp.name === "multi_input"); const multiInput = self.inputs?.find(inp => inp.name === "multi_input");
if (!multiInput || !multiInput.link) return null; if (!multiInput || !multiInput.link) return null;
const nodeGraph = self.graph || app.graph; const nodeGraph = self.graph || app.graph;
// Helper to safely trace back through Reroutes and ComfyUI Group Nodes/Subgraphs // Helper to safely trace back through Reroutes and ComfyUI Group Nodes/Subgraphs
function traceUpstream(graph, linkId, visited = new Set()) { function traceUpstream(graph, linkId, visited = new Set()) {
if (!linkId || visited.has(linkId)) return null; if (!linkId || visited.has(linkId)) return null;
visited.add(linkId); visited.add(linkId);
const link = graph.links[linkId]; const link = graph.links[linkId];
if (!link) return null; if (!link) return null;
const originNode = graph.getNodeById(link.origin_id); const originNode = graph.getNodeById(link.origin_id);
if (!originNode) return null; if (!originNode) return null;
if (originNode.comfyClass === "MultiImageLoader") { if (originNode.comfyClass === "MultiImageLoader") {
return originNode; return originNode;
} }
// Traverse Reroute nodes // Traverse Reroute nodes
if (originNode.type === "Reroute" || originNode.comfyClass === "Reroute") { if (originNode.type === "Reroute" || originNode.comfyClass === "Reroute") {
if (originNode.inputs && originNode.inputs.length > 0 && originNode.inputs[0].link) { if (originNode.inputs && originNode.inputs.length > 0 && originNode.inputs[0].link) {
return traceUpstream(graph, originNode.inputs[0].link, visited); return traceUpstream(graph, originNode.inputs[0].link, visited);
} }
} }
// Traverse standard ComfyUI Group Nodes (subgraphs) // Traverse standard ComfyUI Group Nodes (subgraphs)
if (typeof originNode.getInnerNode === "function") { if (typeof originNode.getInnerNode === "function") {
try { try {
const innerNode = originNode.getInnerNode(link.origin_slot); const innerNode = originNode.getInnerNode(link.origin_slot);
if (innerNode && innerNode.comfyClass === "MultiImageLoader") { if (innerNode && innerNode.comfyClass === "MultiImageLoader") {
return innerNode; return innerNode;
} }
} catch (e) { } catch (e) {
console.warn("Could not trace inner node", e); console.warn("Could not trace inner node", e);
} }
} }
return null; return null;
} }
let sourceNode = traceUpstream(nodeGraph, multiInput.link); let sourceNode = traceUpstream(nodeGraph, multiInput.link);
// Helper to extract the count once we find a node // Helper to extract the count once we find a node
function getCountFromNode(n) { function getCountFromNode(n) {
if (typeof n._imageCount === "number") return n._imageCount; if (typeof n._imageCount === "number") return n._imageCount;
const pathsWidget = n.widgets?.find(w => w.name === "image_paths"); const pathsWidget = n.widgets?.find(w => w.name === "image_paths");
if (pathsWidget) { if (pathsWidget) {
return (pathsWidget.value || "").split('\n').map(p => p.trim()).filter(p => p.length > 0).length; return (pathsWidget.value || "").split('\n').map(p => p.trim()).filter(p => p.length > 0).length;
} }
return null; return null;
} }
if (sourceNode) { if (sourceNode) {
return getCountFromNode(sourceNode); return getCountFromNode(sourceNode);
} }
// Fallback Strategy: If it is connected to something, but we couldn't resolve it // Fallback Strategy: If it is connected to something, but we couldn't resolve it
// directly (e.g. complex nested 3rd party subgraphs), scan the entire UI. // directly (e.g. complex nested 3rd party subgraphs), scan the entire UI.
// If there is EXACTLY ONE MultiImageLoader in the workspace, safely assume that's the one. // If there is EXACTLY ONE MultiImageLoader in the workspace, safely assume that's the one.
let multiImageLoaders = []; let multiImageLoaders = [];
function findAllLoaders(nodes) { function findAllLoaders(nodes) {
if (!nodes) return; if (!nodes) return;
for (let n of nodes) { for (let n of nodes) {
if (n.comfyClass === "MultiImageLoader") { if (n.comfyClass === "MultiImageLoader") {
multiImageLoaders.push(n); multiImageLoaders.push(n);
} }
if (n.subgraph && n.subgraph._nodes) { if (n.subgraph && n.subgraph._nodes) {
findAllLoaders(n.subgraph._nodes); findAllLoaders(n.subgraph._nodes);
} }
} }
} }
if (app.graph && app.graph._nodes) { if (app.graph && app.graph._nodes) {
findAllLoaders(app.graph._nodes); findAllLoaders(app.graph._nodes);
} }
if (multiImageLoaders.length === 1) { if (multiImageLoaders.length === 1) {
return getCountFromNode(multiImageLoaders[0]); return getCountFromNode(multiImageLoaders[0]);
} }
return null; return null;
} }
// --- Backup polling via setInterval (4 Hz) --- // --- Backup polling via setInterval (4 Hz) ---
const pollInterval = setInterval(() => { const pollInterval = setInterval(() => {
if (!node.graph) { if (!node.graph) {
clearInterval(pollInterval); clearInterval(pollInterval);
return; return;
} }
const count = readSourceImageCount(node); const count = readSourceImageCount(node);
if (count !== null) { if (count !== null) {
node._applyWidgetCount(count); node._applyWidgetCount(count);
} }
}, 250); }, 250);
// Clean up the interval when the node is deleted // Clean up the interval when the node is deleted
const origOnRemoved = node.onRemoved; const origOnRemoved = node.onRemoved;
node.onRemoved = function() { node.onRemoved = function() {
// Unregister this node instance // Unregister this node instance
window._LTXKeyframerGlobalNodes.delete(node); window._LTXKeyframerGlobalNodes.delete(node);
clearInterval(pollInterval); clearInterval(pollInterval);
if (origOnRemoved) origOnRemoved.apply(this, arguments); if (origOnRemoved) origOnRemoved.apply(this, arguments);
}; };
// --- Connection change handler --- // --- Connection change handler ---
const onConnectionsChange = node.onConnectionsChange; const onConnectionsChange = node.onConnectionsChange;
node.onConnectionsChange = function(type, index, connected, link_info) { node.onConnectionsChange = function(type, index, connected, link_info) {
if (onConnectionsChange) onConnectionsChange.apply(this, arguments); if (onConnectionsChange) onConnectionsChange.apply(this, arguments);
if (type === 1) { // 1 = Input if (type === 1) { // 1 = Input
const input = this.inputs[index]; const input = this.inputs[index];
if (input && input.name === "multi_input") { if (input && input.name === "multi_input") {
if (connected) { if (connected) {
setTimeout(() => { setTimeout(() => {
const count = readSourceImageCount(this); const count = readSourceImageCount(this);
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0)); this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
}, 100); }, 100);
} else { } else {
this._applyWidgetCount(0); this._applyWidgetCount(0);
} }
} }
} }
}; };
// --- Initial sync when first placed on canvas --- // --- Initial sync when first placed on canvas ---
const origOnAdded = node.onAdded; const origOnAdded = node.onAdded;
node.onAdded = function() { node.onAdded = function() {
if (origOnAdded) origOnAdded.apply(this, arguments); if (origOnAdded) origOnAdded.apply(this, arguments);
setTimeout(() => { setTimeout(() => {
const count = readSourceImageCount(this); const count = readSourceImageCount(this);
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0)); this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
}, 100); }, 100);
}; };
} }
}); });

794
js/ltx_sequencer.js
View File

@@ -1,398 +1,398 @@
import { app } from "../../scripts/app.js"; import { app } from "../../scripts/app.js";
// Global registry to track all LTXSequencer nodes across all subgraphs // Global registry to track all LTXSequencer nodes across all subgraphs
window._LTXSequencerGlobalNodes = window._LTXSequencerGlobalNodes || new Set(); window._LTXSequencerGlobalNodes = window._LTXSequencerGlobalNodes || new Set();
// ComfyUI native trick to cleanly hide/show widgets without deleting them // ComfyUI native trick to cleanly hide/show widgets without deleting them
function toggleWidget(widget, visible) { function toggleWidget(widget, visible) {
if (visible) { if (visible) {
if (widget.origType !== undefined) { if (widget.origType !== undefined) {
widget.type = widget.origType; widget.type = widget.origType;
widget.computeSize = widget.origComputeSize; widget.computeSize = widget.origComputeSize;
delete widget.origType; delete widget.origType;
delete widget.origComputeSize; delete widget.origComputeSize;
} }
} else { } else {
if (widget.type !== "hidden") { if (widget.type !== "hidden") {
widget.origType = widget.type; widget.origType = widget.type;
widget.origComputeSize = widget.computeSize; widget.origComputeSize = widget.computeSize;
widget.type = "hidden"; widget.type = "hidden";
widget.computeSize = () => [0, -4]; widget.computeSize = () => [0, -4];
} }
} }
} }
/** /**
* FULL STATE SYNC: * FULL STATE SYNC:
* Instead of syncing one widget, we push the entire properties object * Instead of syncing one widget, we push the entire properties object
* to ensure no values are ever lost during subgraph transitions or deletions. * to ensure no values are ever lost during subgraph transitions or deletions.
*/ */
function syncFullStateAcrossNodes(sourceNode) { function syncFullStateAcrossNodes(sourceNode) {
if (!window._LTXSequencerGlobalNodes) return; if (!window._LTXSequencerGlobalNodes) return;
for (const targetNode of window._LTXSequencerGlobalNodes) { for (const targetNode of window._LTXSequencerGlobalNodes) {
if (targetNode === sourceNode) continue; if (targetNode === sourceNode) continue;
// 1. Mirror the properties object completely // 1. Mirror the properties object completely
// We use a shallow copy to ensure we don't accidentally share object references // We use a shallow copy to ensure we don't accidentally share object references
const newState = { ...sourceNode.properties }; const newState = { ...sourceNode.properties };
targetNode.properties = { ...targetNode.properties, ...newState }; targetNode.properties = { ...targetNode.properties, ...newState };
// 2. Check if we need to rebuild the widget list (if num_images changed) // 2. Check if we need to rebuild the widget list (if num_images changed)
const targetImageCount = targetNode.properties["num_images"] || 0; const targetImageCount = targetNode.properties["num_images"] || 0;
const currentVisibleCount = targetNode._currentImageCount; const currentVisibleCount = targetNode._currentImageCount;
if (targetImageCount !== currentVisibleCount) { if (targetImageCount !== currentVisibleCount) {
targetNode._applyWidgetCount(targetImageCount); targetNode._applyWidgetCount(targetImageCount);
} }
// 3. Update all existing widget values visually // 3. Update all existing widget values visually
if (targetNode.widgets) { if (targetNode.widgets) {
let modeChanged = false; let modeChanged = false;
targetNode.widgets.forEach(w => { targetNode.widgets.forEach(w => {
const newValue = targetNode.properties[w.name]; const newValue = targetNode.properties[w.name];
if (newValue !== undefined && w.value !== newValue) { if (newValue !== undefined && w.value !== newValue) {
w.value = newValue; w.value = newValue;
if (w.name === "insert_mode") modeChanged = true; if (w.name === "insert_mode") modeChanged = true;
} }
}); });
if (modeChanged && targetNode._updateVisibility) { if (modeChanged && targetNode._updateVisibility) {
targetNode._updateVisibility(); targetNode._updateVisibility();
} }
} }
targetNode.setDirtyCanvas(true, false); targetNode.setDirtyCanvas(true, false);
} }
} }
app.registerExtension({ app.registerExtension({
name: "Comfy.LTXSequencer.DynamicInputs", name: "Comfy.LTXSequencer.DynamicInputs",
async nodeCreated(node) { async nodeCreated(node) {
if (node.comfyClass !== "LTXSequencer") return; if (node.comfyClass !== "LTXSequencer") return;
// Register this node instance globally // Register this node instance globally
window._LTXSequencerGlobalNodes.add(node); window._LTXSequencerGlobalNodes.add(node);
node._currentImageCount = -1; // Force first update node._currentImageCount = -1; // Force first update
// Initialize persistent properties cache // Initialize persistent properties cache
node.properties = node.properties || {}; node.properties = node.properties || {};
// Add subtle separator line above images_loaded // Add subtle separator line above images_loaded
node.addCustomWidget({ node.addCustomWidget({
name: "num_images_separator", name: "num_images_separator",
type: "text", type: "text",
draw(ctx, node, widget_width, y, widget_height) { draw(ctx, node, widget_width, y, widget_height) {
ctx.save(); ctx.save();
ctx.strokeStyle = "#444"; ctx.strokeStyle = "#444";
ctx.lineWidth = 1; ctx.lineWidth = 1;
ctx.beginPath(); ctx.beginPath();
ctx.moveTo(10, y + 5); ctx.moveTo(10, y + 5);
ctx.lineTo(widget_width - 10, y + 5); ctx.lineTo(widget_width - 10, y + 5);
ctx.stroke(); ctx.stroke();
ctx.restore(); ctx.restore();
}, },
computeSize(width) { computeSize(width) {
return [width, 10]; return [width, 10];
} }
}); });
// Move separator before num_images // Move separator before num_images
const moveSeparator = () => { const moveSeparator = () => {
const idx = node.widgets.findIndex(w => w.name === "num_images"); const idx = node.widgets.findIndex(w => w.name === "num_images");
const sepIdx = node.widgets.findIndex(w => w.name === "num_images_separator"); const sepIdx = node.widgets.findIndex(w => w.name === "num_images_separator");
if (idx !== -1 && sepIdx !== -1) { if (idx !== -1 && sepIdx !== -1) {
const separator = node.widgets.splice(sepIdx, 1)[0]; const separator = node.widgets.splice(sepIdx, 1)[0];
node.widgets.splice(idx, 0, separator); node.widgets.splice(idx, 0, separator);
} }
}; };
setTimeout(moveSeparator, 50); setTimeout(moveSeparator, 50);
// Binds custom callbacks to python-schema generated widgets // Binds custom callbacks to python-schema generated widgets
node._hookStaticWidgets = function() { node._hookStaticWidgets = function() {
if (!this.widgets) return; if (!this.widgets) return;
const staticNames = ["num_images", "insert_mode", "frame_rate"]; const staticNames = ["num_images", "insert_mode", "frame_rate"];
staticNames.forEach(name => { staticNames.forEach(name => {
const w = this.widgets.find(w => w.name === name); const w = this.widgets.find(w => w.name === name);
if (w && !w._has_custom_callback) { if (w && !w._has_custom_callback) {
const orig = w.callback; const orig = w.callback;
w.callback = (val) => { w.callback = (val) => {
this.properties[name] = val; this.properties[name] = val;
if (name === "num_images") { if (name === "num_images") {
this._applyWidgetCount(val); this._applyWidgetCount(val);
} }
// Push full state to siblings // Push full state to siblings
syncFullStateAcrossNodes(this); syncFullStateAcrossNodes(this);
if (name === "insert_mode") { if (name === "insert_mode") {
this._updateVisibility(); this._updateVisibility();
} }
if (orig) orig.apply(w, [val]); if (orig) orig.apply(w, [val]);
}; };
w._has_custom_callback = true; w._has_custom_callback = true;
} }
}); });
}; };
// Handles show/hiding widgets based on insertion method // Handles show/hiding widgets based on insertion method
node._updateVisibility = function() { node._updateVisibility = function() {
const mode = this.properties["insert_mode"] || "frames"; const mode = this.properties["insert_mode"] || "frames";
if (!this.widgets) return; if (!this.widgets) return;
let changed = false; let changed = false;
for (const w of this.widgets) { for (const w of this.widgets) {
let shouldBeVisible = true; let shouldBeVisible = true;
if (w.name.startsWith("insert_frame_")) { if (w.name.startsWith("insert_frame_")) {
shouldBeVisible = (mode === "frames"); shouldBeVisible = (mode === "frames");
} else if (w.name.startsWith("insert_second_")) { } else if (w.name.startsWith("insert_second_")) {
shouldBeVisible = (mode === "seconds"); shouldBeVisible = (mode === "seconds");
} }
const isHidden = (w.type === "hidden"); const isHidden = (w.type === "hidden");
if (shouldBeVisible && isHidden) { if (shouldBeVisible && isHidden) {
toggleWidget(w, true); toggleWidget(w, true);
changed = true; changed = true;
} else if (!shouldBeVisible && !isHidden) { } else if (!shouldBeVisible && !isHidden) {
toggleWidget(w, false); toggleWidget(w, false);
changed = true; changed = true;
} }
} }
if (changed) { if (changed) {
this.setDirtyCanvas(true, true); this.setDirtyCanvas(true, true);
requestAnimationFrame(() => { requestAnimationFrame(() => {
if (this.computeSize) { if (this.computeSize) {
this.setSize(this.computeSize()); this.setSize(this.computeSize());
} }
}); });
} }
}; };
// Core update: synchronize widget visibility to match imageCount // Core update: synchronize widget visibility to match imageCount
node._applyWidgetCount = function(count) { node._applyWidgetCount = function(count) {
this._hookStaticWidgets(); this._hookStaticWidgets();
const isInitialLoad = this._currentImageCount === -1; const isInitialLoad = this._currentImageCount === -1;
if (this._currentImageCount === count && !isInitialLoad) return; if (this._currentImageCount === count && !isInitialLoad) return;
this._currentImageCount = count; this._currentImageCount = count;
const initialWidth = this.size[0]; const initialWidth = this.size[0];
const numWidget = this.widgets?.find(w => w.name === "num_images"); const numWidget = this.widgets?.find(w => w.name === "num_images");
if (numWidget) { if (numWidget) {
numWidget.label = "images_loaded"; numWidget.label = "images_loaded";
numWidget.value = Math.max(0, Math.min(count || 0, 50)); numWidget.value = Math.max(0, Math.min(count || 0, 50));
} }
// 1. Update properties from current widget values before restructuring // 1. Update properties from current widget values before restructuring
if (this.widgets) { if (this.widgets) {
this.widgets.forEach(w => { this.widgets.forEach(w => {
if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) { if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) {
if (w.type !== "hidden" && w.type !== "button") this.properties[w.name] = w.value; if (w.type !== "hidden" && w.type !== "button") this.properties[w.name] = w.value;
} }
}); });
} }
// 2. Clear dynamic widgets // 2. Clear dynamic widgets
if (this.widgets) { if (this.widgets) {
this.widgets = this.widgets.filter(w => this.widgets = this.widgets.filter(w =>
!w.name.startsWith("insert_frame_") && !w.name.startsWith("insert_frame_") &&
!w.name.startsWith("insert_second_") && !w.name.startsWith("insert_second_") &&
!w.name.startsWith("strength_") && !w.name.startsWith("strength_") &&
!w.name.startsWith("header_") !w.name.startsWith("header_")
); );
} else { } else {
this.widgets = []; this.widgets = [];
} }
// 3. Rebuild precisely // 3. Rebuild precisely
for (let i = 1; i <= count; i++) { for (let i = 1; i <= count; i++) {
// Header // Header
const headerName = `header_${i}`; const headerName = `header_${i}`;
this.addCustomWidget({ this.addCustomWidget({
name: headerName, name: headerName,
type: "text", type: "text",
value: `Image #${i}`, value: `Image #${i}`,
draw(ctx, node, widget_width, y, widget_height) { draw(ctx, node, widget_width, y, widget_height) {
ctx.save(); ctx.save();
const margin = 10; const margin = 10;
const topPadding = 15; const topPadding = 15;
ctx.strokeStyle = "#333"; ctx.strokeStyle = "#333";
ctx.lineWidth = 1; ctx.lineWidth = 1;
ctx.beginPath(); ctx.beginPath();
ctx.moveTo(margin, y + 5); ctx.moveTo(margin, y + 5);
ctx.lineTo(widget_width - margin, y + 5); ctx.lineTo(widget_width - margin, y + 5);
ctx.stroke(); ctx.stroke();
ctx.fillStyle = "#dddddd"; ctx.fillStyle = "#dddddd";
ctx.font = "bold 12px Arial"; ctx.font = "bold 12px Arial";
ctx.textAlign = "left"; ctx.textAlign = "left";
ctx.fillText(`Image #${i}`, margin, y + topPadding + 10); ctx.fillText(`Image #${i}`, margin, y + topPadding + 10);
ctx.restore(); ctx.restore();
}, },
computeSize(width) { return [width, 35]; } computeSize(width) { return [width, 35]; }
}); });
// Helper to add widget with full sync // Helper to add widget with full sync
const addSyncedWidget = (type, name, def, options) => { const addSyncedWidget = (type, name, def, options) => {
const saved = this.properties[name]; const saved = this.properties[name];
return this.addWidget(type, name, saved !== undefined ? saved : def, (val) => { return this.addWidget(type, name, saved !== undefined ? saved : def, (val) => {
this.properties[name] = val; this.properties[name] = val;
syncFullStateAcrossNodes(this); syncFullStateAcrossNodes(this);
}, options); }, options);
}; };
addSyncedWidget("number", `insert_frame_${i}`, 0, { min: -9999, max: 9999, step: 10, precision: 0 }); addSyncedWidget("number", `insert_frame_${i}`, 0, { min: -9999, max: 9999, step: 10, precision: 0 });
addSyncedWidget("number", `insert_second_${i}`, 0.0, { min: 0.0, max: 9999.0, step: 0.1, precision: 2 }); addSyncedWidget("number", `insert_second_${i}`, 0.0, { min: 0.0, max: 9999.0, step: 0.1, precision: 2 });
addSyncedWidget("number", `strength_${i}`, 1.0, { min: 0.0, max: 1.0, step: 0.01 }); addSyncedWidget("number", `strength_${i}`, 1.0, { min: 0.0, max: 1.0, step: 0.01 });
} }
this._updateVisibility(); this._updateVisibility();
this.setDirtyCanvas(true, true); this.setDirtyCanvas(true, true);
requestAnimationFrame(() => { requestAnimationFrame(() => {
if (this.computeSize) { if (this.computeSize) {
this.setSize(this.computeSize()); this.setSize(this.computeSize());
this.size[0] = initialWidth; this.size[0] = initialWidth;
} }
}); });
}; };
const origConfigure = node.configure; const origConfigure = node.configure;
node.configure = function(info) { node.configure = function(info) {
if (origConfigure) origConfigure.apply(this, arguments); if (origConfigure) origConfigure.apply(this, arguments);
if (this.widgets) { if (this.widgets) {
this.widgets.forEach(w => { this.widgets.forEach(w => {
if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) { if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) {
if (w.type !== "button") this.properties[w.name] = w.value; if (w.type !== "button") this.properties[w.name] = w.value;
} }
}); });
} }
}; };
node.onConfigure = function(info) { node.onConfigure = function(info) {
if (info.properties) { if (info.properties) {
this.properties = { ...this.properties, ...info.properties }; this.properties = { ...this.properties, ...info.properties };
} }
this._hookStaticWidgets(); this._hookStaticWidgets();
setTimeout(() => { setTimeout(() => {
const count = readSourceImageCount(this); const count = readSourceImageCount(this);
let targetCount = count !== null ? count : (this.properties.num_images || 0); let targetCount = count !== null ? count : (this.properties.num_images || 0);
this._applyWidgetCount(targetCount); this._applyWidgetCount(targetCount);
this._updateVisibility(); this._updateVisibility();
}, 100); }, 100);
}; };
const originalOnSerialize = node.onSerialize; const originalOnSerialize = node.onSerialize;
node.onSerialize = function(info) { node.onSerialize = function(info) {
if (this.widgets) { if (this.widgets) {
this.widgets.forEach(w => { this.widgets.forEach(w => {
if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) { if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) {
if (w.type !== "hidden" && w.type !== "button") this.properties[w.name] = w.value; if (w.type !== "hidden" && w.type !== "button") this.properties[w.name] = w.value;
} }
}); });
} }
if (originalOnSerialize) originalOnSerialize.apply(this, arguments); if (originalOnSerialize) originalOnSerialize.apply(this, arguments);
info.properties = { ...this.properties }; info.properties = { ...this.properties };
const strictArray = []; const strictArray = [];
strictArray.push(this.properties["num_images"] !== undefined ? this.properties["num_images"] : 1); strictArray.push(this.properties["num_images"] !== undefined ? this.properties["num_images"] : 1);
strictArray.push(this.properties["insert_mode"] !== undefined ? this.properties["insert_mode"] : "frames"); strictArray.push(this.properties["insert_mode"] !== undefined ? this.properties["insert_mode"] : "frames");
strictArray.push(this.properties["frame_rate"] !== undefined ? this.properties["frame_rate"] : 24); strictArray.push(this.properties["frame_rate"] !== undefined ? this.properties["frame_rate"] : 24);
for (let i = 1; i <= 50; i++) { for (let i = 1; i <= 50; i++) {
strictArray.push(this.properties[`insert_frame_${i}`] !== undefined ? this.properties[`insert_frame_${i}`] : 0); strictArray.push(this.properties[`insert_frame_${i}`] !== undefined ? this.properties[`insert_frame_${i}`] : 0);
strictArray.push(this.properties[`insert_second_${i}`] !== undefined ? this.properties[`insert_second_${i}`] : 0.0); strictArray.push(this.properties[`insert_second_${i}`] !== undefined ? this.properties[`insert_second_${i}`] : 0.0);
strictArray.push(this.properties[`strength_${i}`] !== undefined ? this.properties[`strength_${i}`] : 1.0); strictArray.push(this.properties[`strength_${i}`] !== undefined ? this.properties[`strength_${i}`] : 1.0);
} }
info.widgets_values = strictArray; info.widgets_values = strictArray;
}; };
function readSourceImageCount(self) { function readSourceImageCount(self) {
const multiInput = self.inputs?.find(inp => inp.name === "multi_input"); const multiInput = self.inputs?.find(inp => inp.name === "multi_input");
if (!multiInput || !multiInput.link) return null; if (!multiInput || !multiInput.link) return null;
const nodeGraph = self.graph || app.graph; const nodeGraph = self.graph || app.graph;
function traceUpstream(graph, linkId, visited = new Set()) { function traceUpstream(graph, linkId, visited = new Set()) {
if (!linkId || visited.has(linkId)) return null; if (!linkId || visited.has(linkId)) return null;
visited.add(linkId); visited.add(linkId);
const link = graph.links[linkId]; const link = graph.links[linkId];
if (!link) return null; if (!link) return null;
const originNode = graph.getNodeById(link.origin_id); const originNode = graph.getNodeById(link.origin_id);
if (!originNode) return null; if (!originNode) return null;
if (originNode.comfyClass === "MultiImageLoader") return originNode; if (originNode.comfyClass === "MultiImageLoader") return originNode;
if (originNode.type === "Reroute" || originNode.comfyClass === "Reroute") { if (originNode.type === "Reroute" || originNode.comfyClass === "Reroute") {
if (originNode.inputs?.[0]?.link) return traceUpstream(graph, originNode.inputs[0].link, visited); if (originNode.inputs?.[0]?.link) return traceUpstream(graph, originNode.inputs[0].link, visited);
} }
if (typeof originNode.getInnerNode === "function") { if (typeof originNode.getInnerNode === "function") {
try { try {
const innerNode = originNode.getInnerNode(link.origin_slot); const innerNode = originNode.getInnerNode(link.origin_slot);
if (innerNode?.comfyClass === "MultiImageLoader") return innerNode; if (innerNode?.comfyClass === "MultiImageLoader") return innerNode;
} catch (e) {} } catch (e) {}
} }
return null; return null;
} }
let sourceNode = traceUpstream(nodeGraph, multiInput.link); let sourceNode = traceUpstream(nodeGraph, multiInput.link);
function getCountFromNode(n) { function getCountFromNode(n) {
if (typeof n._imageCount === "number") return n._imageCount; if (typeof n._imageCount === "number") return n._imageCount;
const pathsWidget = n.widgets?.find(w => w.name === "image_paths"); const pathsWidget = n.widgets?.find(w => w.name === "image_paths");
return pathsWidget ? (pathsWidget.value || "").split('\n').filter(p => p.trim()).length : null; return pathsWidget ? (pathsWidget.value || "").split('\n').filter(p => p.trim()).length : null;
} }
if (sourceNode) return getCountFromNode(sourceNode); if (sourceNode) return getCountFromNode(sourceNode);
let multiImageLoaders = []; let multiImageLoaders = [];
function findAllLoaders(nodes) { function findAllLoaders(nodes) {
if (!nodes) return; if (!nodes) return;
for (let n of nodes) { for (let n of nodes) {
if (n.comfyClass === "MultiImageLoader") multiImageLoaders.push(n); if (n.comfyClass === "MultiImageLoader") multiImageLoaders.push(n);
if (n.subgraph?._nodes) findAllLoaders(n.subgraph._nodes); if (n.subgraph?._nodes) findAllLoaders(n.subgraph._nodes);
} }
} }
if (app.graph?._nodes) findAllLoaders(app.graph._nodes); if (app.graph?._nodes) findAllLoaders(app.graph._nodes);
if (multiImageLoaders.length === 1) return getCountFromNode(multiImageLoaders[0]); if (multiImageLoaders.length === 1) return getCountFromNode(multiImageLoaders[0]);
return null; return null;
} }
const pollInterval = setInterval(() => { const pollInterval = setInterval(() => {
if (!node.graph) { if (!node.graph) {
clearInterval(pollInterval); clearInterval(pollInterval);
return; return;
} }
const count = readSourceImageCount(node); const count = readSourceImageCount(node);
if (count !== null && count !== node._currentImageCount) { if (count !== null && count !== node._currentImageCount) {
node._applyWidgetCount(count); node._applyWidgetCount(count);
syncFullStateAcrossNodes(node); // Sync the new count to others syncFullStateAcrossNodes(node); // Sync the new count to others
} }
}, 500); }, 500);
const origOnRemoved = node.onRemoved; const origOnRemoved = node.onRemoved;
node.onRemoved = function() { node.onRemoved = function() {
window._LTXSequencerGlobalNodes.delete(node); window._LTXSequencerGlobalNodes.delete(node);
clearInterval(pollInterval); clearInterval(pollInterval);
if (origOnRemoved) origOnRemoved.apply(this, arguments); if (origOnRemoved) origOnRemoved.apply(this, arguments);
}; };
node.onConnectionsChange = function(type, index, connected) { node.onConnectionsChange = function(type, index, connected) {
if (type === 1 && this.inputs[index]?.name === "multi_input") { if (type === 1 && this.inputs[index]?.name === "multi_input") {
if (connected) { if (connected) {
setTimeout(() => { setTimeout(() => {
const count = readSourceImageCount(this); const count = readSourceImageCount(this);
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0)); this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
}, 100); }, 100);
} else { } else {
this._applyWidgetCount(0); this._applyWidgetCount(0);
} }
} }
}; };
node.onAdded = function() { node.onAdded = function() {
setTimeout(() => { setTimeout(() => {
const count = readSourceImageCount(this); const count = readSourceImageCount(this);
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0)); this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
}, 100); }, 100);
}; };
} }
}); });

1374
js/multi_image_loader.js
View File

File diff suppressed because it is too large Load Diff

670
js/speech_length_calculator.js
View File

@@ -1,336 +1,336 @@
import { app } from "../../scripts/app.js"; import { app } from "../../scripts/app.js";
import { ComfyWidgets } from "../../scripts/widgets.js"; import { ComfyWidgets } from "../../scripts/widgets.js";
// 1. DEFINE CSS STYLES GLOBALLY ONCE // 1. DEFINE CSS STYLES GLOBALLY ONCE
// We use DOM elements instead of Canvas drawing so it works flawlessly in ComfyUI V3 // We use DOM elements instead of Canvas drawing so it works flawlessly in ComfyUI V3
const cssStyles = ` const cssStyles = `
.slc-ui-container { .slc-ui-container {
width: 100%; width: 100%;
height: 100%; height: 100%;
min-height: 260px; /* Increased from 220px to prevent V1 cropping */ min-height: 260px; /* Increased from 220px to prevent V1 cropping */
background-color: rgba(15, 15, 19, 0.7); background-color: rgba(15, 15, 19, 0.7);
border: 1px solid rgba(255, 255, 255, 0.1); border: 1px solid rgba(255, 255, 255, 0.1);
border-radius: 8px; border-radius: 8px;
padding: 12px; padding: 12px;
box-sizing: border-box; box-sizing: border-box;
display: flex; display: flex;
flex-direction: column; flex-direction: column;
font-family: sans-serif; font-family: sans-serif;
color: #ffffff; color: #ffffff;
overflow: hidden; overflow: hidden;
pointer-events: auto; /* allows text selection */ pointer-events: auto; /* allows text selection */
} }
.slc-empty { .slc-empty {
flex: 1; flex: 1;
display: flex; display: flex;
flex-direction: column; flex-direction: column;
align-items: center; align-items: center;
justify-content: center; justify-content: center;
} }
.slc-empty-title { color: #aaaaaa; font-size: 14px; font-weight: 500; margin-bottom: 4px; } .slc-empty-title { color: #aaaaaa; font-size: 14px; font-weight: 500; margin-bottom: 4px; }
.slc-empty-sub { color: #777777; font-size: 11px; } .slc-empty-sub { color: #777777; font-size: 11px; }
.slc-headers { .slc-headers {
display: flex; display: flex;
gap: 8px; gap: 8px;
margin-bottom: 12px; margin-bottom: 12px;
} }
.slc-header-block { .slc-header-block {
flex: 1; flex: 1;
background-color: rgba(0, 0, 0, 0.4); background-color: rgba(0, 0, 0, 0.4);
border-radius: 6px; border-radius: 6px;
padding: 8px 0; padding: 8px 0;
display: flex; display: flex;
flex-direction: column; flex-direction: column;
align-items: center; align-items: center;
} }
.slc-header-title { color: #888888; font-size: 9px; font-weight: bold; margin-bottom: 4px; } .slc-header-title { color: #888888; font-size: 9px; font-weight: bold; margin-bottom: 4px; }
.slc-header-value { color: #ffffff; font-size: 16px; font-weight: bold; } .slc-header-value { color: #ffffff; font-size: 16px; font-weight: bold; }
.slc-cards { .slc-cards {
display: flex; display: flex;
flex-direction: column; flex-direction: column;
gap: 8px; gap: 8px;
} }
.slc-card { .slc-card {
display: flex; display: flex;
justify-content: space-between; justify-content: space-between;
align-items: center; align-items: center;
background-color: rgba(0, 0, 0, 0.25); background-color: rgba(0, 0, 0, 0.25);
border: 1px solid rgba(255, 255, 255, 0.05); border: 1px solid rgba(255, 255, 255, 0.05);
border-radius: 4px; border-radius: 4px;
padding: 8px 12px 8px 16px; padding: 8px 12px 8px 16px;
position: relative; position: relative;
overflow: hidden; overflow: hidden;
} }
.slc-card::before { .slc-card::before {
content: ''; content: '';
position: absolute; position: absolute;
left: 0; left: 0;
top: 0; top: 0;
bottom: 0; bottom: 0;
width: 4px; width: 4px;
} }
.slc-card.slow::before { background-color: #93c5fd; } .slc-card.slow::before { background-color: #93c5fd; }
.slc-card.avg::before { background-color: #86efac; } .slc-card.avg::before { background-color: #86efac; }
.slc-card.fast::before { background-color: #fca5a5; } .slc-card.fast::before { background-color: #fca5a5; }
.slc-card-left { display: flex; flex-direction: column; } .slc-card-left { display: flex; flex-direction: column; }
.slc-card-speed { font-size: 12px; font-weight: bold; margin-bottom: 2px; } .slc-card-speed { font-size: 12px; font-weight: bold; margin-bottom: 2px; }
.slc-card.slow .slc-card-speed { color: #93c5fd; } .slc-card.slow .slc-card-speed { color: #93c5fd; }
.slc-card.avg .slc-card-speed { color: #86efac; } .slc-card.avg .slc-card-speed { color: #86efac; }
.slc-card.fast .slc-card-speed { color: #fca5a5; } .slc-card.fast .slc-card-speed { color: #fca5a5; }
.slc-card-wpm { font-size: 10px; color: #aaaaaa; } .slc-card-wpm { font-size: 10px; color: #aaaaaa; }
.slc-card-right { display: flex; flex-direction: column; align-items: flex-end; } .slc-card-right { display: flex; flex-direction: column; align-items: flex-end; }
.slc-card-time { font-size: 13px; font-weight: bold; color: #ffffff; margin-bottom: 2px; } .slc-card-time { font-size: 13px; font-weight: bold; color: #ffffff; margin-bottom: 2px; }
.slc-card-frames { font-size: 11px; font-family: monospace; color: #888888; } .slc-card-frames { font-size: 11px; font-family: monospace; color: #888888; }
.slc-legend { .slc-legend {
margin-top: auto; margin-top: auto;
padding-top: 8px; padding-top: 8px;
color: #999999; color: #999999;
font-size: 10px; font-size: 10px;
font-weight: 500; font-weight: 500;
} }
`; `;
// Inject styles to document head safely // Inject styles to document head safely
if (!document.getElementById("speech-length-calculator-styles")) { if (!document.getElementById("speech-length-calculator-styles")) {
const styleEl = document.createElement("style"); const styleEl = document.createElement("style");
styleEl.id = "speech-length-calculator-styles"; styleEl.id = "speech-length-calculator-styles";
styleEl.textContent = cssStyles; styleEl.textContent = cssStyles;
document.head.appendChild(styleEl); document.head.appendChild(styleEl);
} }
// 2. HELPER TO GENERATE THE HTML CONTENT // 2. HELPER TO GENERATE THE HTML CONTENT
function buildUIHTML(statsData) { function buildUIHTML(statsData) {
if (!statsData || statsData.empty) { if (!statsData || statsData.empty) {
return ` return `
<div class="slc-empty"> <div class="slc-empty">
<div class="slc-empty-title">Awaiting Script</div> <div class="slc-empty-title">Awaiting Script</div>
<div class="slc-empty-sub">Wrap spoken text inside "quotes"</div> <div class="slc-empty-sub">Wrap spoken text inside "quotes"</div>
</div> </div>
`; `;
} }
return ` return `
<div class="slc-headers"> <div class="slc-headers">
<div class="slc-header-block"> <div class="slc-header-block">
<div class="slc-header-title">SPOKEN WORDS</div> <div class="slc-header-title">SPOKEN WORDS</div>
<div class="slc-header-value">${statsData.wordCount}</div> <div class="slc-header-value">${statsData.wordCount}</div>
</div> </div>
<div class="slc-header-block"> <div class="slc-header-block">
<div class="slc-header-title">ADDED TIME</div> <div class="slc-header-title">ADDED TIME</div>
<div class="slc-header-value">${statsData.additionalTime}s</div> <div class="slc-header-value">${statsData.additionalTime}s</div>
</div> </div>
</div> </div>
<div class="slc-cards"> <div class="slc-cards">
<div class="slc-card slow"> <div class="slc-card slow">
<div class="slc-card-left"> <div class="slc-card-left">
<div class="slc-card-speed">SLOW</div> <div class="slc-card-speed">SLOW</div>
<div class="slc-card-wpm">100 WPM</div> <div class="slc-card-wpm">100 WPM</div>
</div> </div>
<div class="slc-card-right"> <div class="slc-card-right">
<div class="slc-card-time">${statsData.slow.time}</div> <div class="slc-card-time">${statsData.slow.time}</div>
<div class="slc-card-frames">${statsData.slow.frames} frames</div> <div class="slc-card-frames">${statsData.slow.frames} frames</div>
</div> </div>
</div> </div>
<div class="slc-card avg"> <div class="slc-card avg">
<div class="slc-card-left"> <div class="slc-card-left">
<div class="slc-card-speed">AVG</div> <div class="slc-card-speed">AVG</div>
<div class="slc-card-wpm">130 WPM</div> <div class="slc-card-wpm">130 WPM</div>
</div> </div>
<div class="slc-card-right"> <div class="slc-card-right">
<div class="slc-card-time">${statsData.avg.time}</div> <div class="slc-card-time">${statsData.avg.time}</div>
<div class="slc-card-frames">${statsData.avg.frames} frames</div> <div class="slc-card-frames">${statsData.avg.frames} frames</div>
</div> </div>
</div> </div>
<div class="slc-card fast"> <div class="slc-card fast">
<div class="slc-card-left"> <div class="slc-card-left">
<div class="slc-card-speed">FAST</div> <div class="slc-card-speed">FAST</div>
<div class="slc-card-wpm">160 WPM</div> <div class="slc-card-wpm">160 WPM</div>
</div> </div>
<div class="slc-card-right"> <div class="slc-card-right">
<div class="slc-card-time">${statsData.fast.time}</div> <div class="slc-card-time">${statsData.fast.time}</div>
<div class="slc-card-frames">${statsData.fast.frames} frames</div> <div class="slc-card-frames">${statsData.fast.frames} frames</div>
</div> </div>
</div> </div>
</div> </div>
<div class="slc-legend">WPM = Words Per Minute</div> <div class="slc-legend">WPM = Words Per Minute</div>
`; `;
} }
app.registerExtension({ app.registerExtension({
name: "Comfy.SpeechLengthCalculator", name: "Comfy.SpeechLengthCalculator",
async beforeRegisterNodeDef(nodeType, nodeData, app) { async beforeRegisterNodeDef(nodeType, nodeData, app) {
if (nodeData.name === "SpeechLengthCalculator") { if (nodeData.name === "SpeechLengthCalculator") {
const onNodeCreated = nodeType.prototype.onNodeCreated; const onNodeCreated = nodeType.prototype.onNodeCreated;
nodeType.prototype.onNodeCreated = function () { nodeType.prototype.onNodeCreated = function () {
const r = onNodeCreated ? onNodeCreated.apply(this, arguments) : undefined; const r = onNodeCreated ? onNodeCreated.apply(this, arguments) : undefined;
// 3. CREATE THE DOM WIDGET // 3. CREATE THE DOM WIDGET
const uiContainer = document.createElement("div"); const uiContainer = document.createElement("div");
uiContainer.className = "slc-ui-container"; uiContainer.className = "slc-ui-container";
uiContainer.innerHTML = buildUIHTML({ empty: true }); uiContainer.innerHTML = buildUIHTML({ empty: true });
// Attach to node: standard addDOMWidget works natively in both V1 and V3 Frontends // Attach to node: standard addDOMWidget works natively in both V1 and V3 Frontends
let statsWidget; let statsWidget;
if (typeof this.addDOMWidget === "function") { if (typeof this.addDOMWidget === "function") {
statsWidget = this.addDOMWidget("Stats", "HTML", uiContainer, { statsWidget = this.addDOMWidget("Stats", "HTML", uiContainer, {
serialize: false, serialize: false,
hideOnZoom: false hideOnZoom: false
}); });
} else { } else {
// Fallback for very old unpatched V1 installations // Fallback for very old unpatched V1 installations
statsWidget = ComfyWidgets["STRING"](this, "Stats", ["STRING", { multiline: true }], app).widget; statsWidget = ComfyWidgets["STRING"](this, "Stats", ["STRING", { multiline: true }], app).widget;
if (statsWidget.inputEl) { if (statsWidget.inputEl) {
statsWidget.inputEl.style.display = "none"; statsWidget.inputEl.style.display = "none";
if (statsWidget.inputEl.parentNode) { if (statsWidget.inputEl.parentNode) {
statsWidget.inputEl.parentNode.appendChild(uiContainer); statsWidget.inputEl.parentNode.appendChild(uiContainer);
} }
} }
} }
// Tell LiteGraph exactly how much vertical space to hold for our UI // Tell LiteGraph exactly how much vertical space to hold for our UI
statsWidget.computeSize = function() { statsWidget.computeSize = function() {
return [0, 280]; // Increased from 240 to prevent bottom cropping in V1 return [0, 280]; // Increased from 240 to prevent bottom cropping in V1
}; };
// Ensure the node is wide enough on initial creation // Ensure the node is wide enough on initial creation
requestAnimationFrame(() => { requestAnimationFrame(() => {
const minWidth = 340; const minWidth = 340;
if (this.size[0] < minWidth) this.size[0] = minWidth; if (this.size[0] < minWidth) this.size[0] = minWidth;
// Fine tune initial height to account for the larger computeSize and make text box taller // Fine tune initial height to account for the larger computeSize and make text box taller
if (this.size[1] < 500) this.size[1] = 500; if (this.size[1] < 500) this.size[1] = 500;
this.setDirtyCanvas(true, true); this.setDirtyCanvas(true, true);
}); });
// Fetch text from input link OR widget // Fetch text from input link OR widget
this._getCurrentText = () => { this._getCurrentText = () => {
const inputSlot = this.inputs && this.inputs.find(i => i.name === "text_input" || (i.name === "text" && i.link)); const inputSlot = this.inputs && this.inputs.find(i => i.name === "text_input" || (i.name === "text" && i.link));
if (inputSlot && inputSlot.link) { if (inputSlot && inputSlot.link) {
const link = app.graph.links[inputSlot.link]; const link = app.graph.links[inputSlot.link];
if (link) { if (link) {
const sourceNode = app.graph.getNodeById(link.origin_id); const sourceNode = app.graph.getNodeById(link.origin_id);
if (sourceNode && sourceNode.widgets) { if (sourceNode && sourceNode.widgets) {
const w = sourceNode.widgets.find(w => w.name === "value" || w.name === "text" || w.name === "Text" || w.type === "customtext" || w.type === "STRING"); const w = sourceNode.widgets.find(w => w.name === "value" || w.name === "text" || w.name === "Text" || w.type === "customtext" || w.type === "STRING");
if (w && typeof w.value === "string") return w.value; if (w && typeof w.value === "string") return w.value;
} }
} }
} }
const textWidget = this.widgets && this.widgets.find(w => w.name === "text"); const textWidget = this.widgets && this.widgets.find(w => w.name === "text");
return textWidget ? (textWidget.value || "") : ""; return textWidget ? (textWidget.value || "") : "";
}; };
this._lastState = { text: null, fps: null, addTime: null }; this._lastState = { text: null, fps: null, addTime: null };
const updateStats = () => { const updateStats = () => {
const fpsWidget = this.widgets && this.widgets.find(w => w.name === "fps"); const fpsWidget = this.widgets && this.widgets.find(w => w.name === "fps");
const additionalTimeWidget = this.widgets && this.widgets.find(w => w.name === "additional_time"); const additionalTimeWidget = this.widgets && this.widgets.find(w => w.name === "additional_time");
if (!fpsWidget) return; if (!fpsWidget) return;
const text = this._getCurrentText(); const text = this._getCurrentText();
const fps = fpsWidget.value || 24; const fps = fpsWidget.value || 24;
const additionalTime = additionalTimeWidget ? parseFloat(additionalTimeWidget.value) || 0 : 0; const additionalTime = additionalTimeWidget ? parseFloat(additionalTimeWidget.value) || 0 : 0;
// Skip expensive calculations if nothing changed // Skip expensive calculations if nothing changed
if (this._lastState.text === text && if (this._lastState.text === text &&
this._lastState.fps === fps && this._lastState.fps === fps &&
this._lastState.addTime === additionalTime) { this._lastState.addTime === additionalTime) {
return; return;
} }
this._lastState.text = text; this._lastState.text = text;
this._lastState.fps = fps; this._lastState.fps = fps;
this._lastState.addTime = additionalTime; this._lastState.addTime = additionalTime;
const regex = /"([^"]*)"|'([^']*)'|“([^”]*)”|‘([^’]*)’/g; const regex = /"([^"]*)"|'([^']*)'|“([^”]*)”|‘([^’]*)’/g;
let match; let match;
let quotedText = ""; let quotedText = "";
while ((match = regex.exec(text)) !== null) { while ((match = regex.exec(text)) !== null) {
quotedText += (match[1] || match[2] || match[3] || match[4] || "") + " "; quotedText += (match[1] || match[2] || match[3] || match[4] || "") + " ";
} }
const words = quotedText.trim().split(/\s+/).filter(w => w.length > 0); const words = quotedText.trim().split(/\s+/).filter(w => w.length > 0);
const wordCount = words.length; const wordCount = words.length;
const formatTime = (wpm) => { const formatTime = (wpm) => {
const baseMins = wordCount / wpm; const baseMins = wordCount / wpm;
const totalSecs = (baseMins * 60) + additionalTime; const totalSecs = (baseMins * 60) + additionalTime;
const mins = Math.floor(totalSecs / 60); const mins = Math.floor(totalSecs / 60);
let secs = totalSecs % 60; let secs = totalSecs % 60;
secs = Math.ceil(secs * 10) / 10; secs = Math.ceil(secs * 10) / 10;
const frames = Math.ceil(totalSecs * fps); const frames = Math.ceil(totalSecs * fps);
const secsStr = secs.toFixed(1); const secsStr = secs.toFixed(1);
const timeStr = mins > 0 ? `${mins}m ${secsStr}s` : `${secsStr}s`; const timeStr = mins > 0 ? `${mins}m ${secsStr}s` : `${secsStr}s`;
return { return {
time: timeStr, time: timeStr,
frames: frames.toString() frames: frames.toString()
}; };
}; };
const statsData = { const statsData = {
empty: (wordCount === 0 && additionalTime === 0), empty: (wordCount === 0 && additionalTime === 0),
wordCount: wordCount, wordCount: wordCount,
additionalTime: additionalTime, additionalTime: additionalTime,
slow: formatTime(100), slow: formatTime(100),
avg: formatTime(130), avg: formatTime(130),
fast: formatTime(160) fast: formatTime(160)
}; };
// 4. INJECT HTML TO THE DOM // 4. INJECT HTML TO THE DOM
// Replaces the heavy canvas redrawing! // Replaces the heavy canvas redrawing!
uiContainer.innerHTML = buildUIHTML(statsData); uiContainer.innerHTML = buildUIHTML(statsData);
this.setDirtyCanvas(true, false); this.setDirtyCanvas(true, false);
}; };
// We still use onDrawBackground as an efficient silent trigger // We still use onDrawBackground as an efficient silent trigger
// to auto-update in case upstream nodes silently change their text // to auto-update in case upstream nodes silently change their text
const onDrawBackground = this.onDrawBackground; const onDrawBackground = this.onDrawBackground;
this.onDrawBackground = function(ctx) { this.onDrawBackground = function(ctx) {
if (onDrawBackground) onDrawBackground.apply(this, arguments); if (onDrawBackground) onDrawBackground.apply(this, arguments);
updateStats(); updateStats();
}; };
// Bind events to update in real time based on node interactions // Bind events to update in real time based on node interactions
setTimeout(() => { setTimeout(() => {
const textWidget = this.widgets && this.widgets.find(w => w.name === "text"); const textWidget = this.widgets && this.widgets.find(w => w.name === "text");
const fpsWidget = this.widgets && this.widgets.find(w => w.name === "fps"); const fpsWidget = this.widgets && this.widgets.find(w => w.name === "fps");
const additionalTimeWidget = this.widgets && this.widgets.find(w => w.name === "additional_time"); const additionalTimeWidget = this.widgets && this.widgets.find(w => w.name === "additional_time");
if (textWidget) { if (textWidget) {
const origCallback = textWidget.callback; const origCallback = textWidget.callback;
textWidget.callback = function() { textWidget.callback = function() {
if (origCallback) origCallback.apply(this, arguments); if (origCallback) origCallback.apply(this, arguments);
updateStats(); updateStats();
} }
} }
if (fpsWidget) { if (fpsWidget) {
const origFpsCallback = fpsWidget.callback; const origFpsCallback = fpsWidget.callback;
fpsWidget.callback = function() { fpsWidget.callback = function() {
if (origFpsCallback) origFpsCallback.apply(this, arguments); if (origFpsCallback) origFpsCallback.apply(this, arguments);
updateStats(); updateStats();
} }
} }
if (additionalTimeWidget) { if (additionalTimeWidget) {
const origAddCallback = additionalTimeWidget.callback; const origAddCallback = additionalTimeWidget.callback;
additionalTimeWidget.callback = function() { additionalTimeWidget.callback = function() {
if (origAddCallback) origAddCallback.apply(this, arguments); if (origAddCallback) origAddCallback.apply(this, arguments);
updateStats(); updateStats();
} }
} }
updateStats(); updateStats();
}, 100); }, 100);
return r; return r;
}; };
} }
} }
}); });

75
latent_slice.py Normal file
View File

@@ -0,0 +1,75 @@
# --- START OF FILE latent_slice.py ---
import torch
class CleanLatentSlice:
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"latent": ("LATENT",),
"start": ("INT", {"default": 0, "min": 0, "max": 100000, "step": 1, "tooltip": "The starting frame index to slice from (plug 'latent_start_index' here)."}),
"length": ("INT", {"default": 1, "min": 1, "max": 100000, "step": 1, "tooltip": "The number of frames to keep (plug 'clean_latent_frames' here)."}),
}
}
RETURN_TYPES = ("LATENT",)
RETURN_NAMES = ("latent",)
FUNCTION = "slice_latent"
CATEGORY = "WhatDreamsCost"
DESCRIPTION = "Safely slices a video latent starting from an offset index for a specific length. Uses torch.narrow to bypass PyTorch NestedTensor slicing bugs."
def slice_latent(self, latent, start, length):
new_latent = latent.copy()
def safe_slice(tensor, target_start, target_len):
dims = tensor.ndim if hasattr(tensor, "ndim") else len(tensor.shape)
# Constrain starting index and length to the actual size of the tensor
max_size = tensor.size(2) if dims == 5 else tensor.size(0)
actual_start = min(target_start, max_size - 1) if max_size > 0 else 0
actual_len = min(target_len, max_size - actual_start)
try:
# torch.narrow is the safest low-level C++ slice method to bypass NestedTensor bugs
if dims == 5:
# [Batch, Channels, Frames, Height, Width] -> Slice dimension 2
return torch.narrow(tensor, 2, actual_start, actual_len)
elif dims == 4:
# [Frames, Channels, Height, Width] -> Slice dimension 0
return torch.narrow(tensor, 0, actual_start, actual_len)
elif dims == 3:
# [Frames, Height, Width] -> Slice dimension 0
return torch.narrow(tensor, 0, actual_start, actual_len)
except Exception as e:
# Fallback if narrow fails
if dims == 5:
return tensor[:, :, actual_start : actual_start + actual_len]
elif dims == 4:
return tensor[actual_start : actual_start + actual_len]
elif dims == 3:
return tensor[actual_start : actual_start + actual_len]
return tensor
# Safely slice video samples
if "samples" in new_latent:
new_latent["samples"] = safe_slice(new_latent["samples"], start, length)
# Safely slice video noise mask (if it exists)
if "noise_mask" in new_latent:
new_latent["noise_mask"] = safe_slice(new_latent["noise_mask"], start, length)
return (new_latent,)
# Register the node with ComfyUI
NODE_CLASS_MAPPINGS = {
"CleanLatentSlice": CleanLatentSlice
}
NODE_DISPLAY_NAME_MAPPINGS = {
"CleanLatentSlice": "Clean Latent Slice"
}
__all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS']

280
load_audio_ui.py
View File

@@ -1,141 +1,141 @@
import folder_paths import folder_paths
import os import os
import torch import torch
import av import av
def f32_pcm(wav: torch.Tensor) -> torch.Tensor: def f32_pcm(wav: torch.Tensor) -> torch.Tensor:
"""Convert audio to float 32 bits PCM format.""" """Convert audio to float 32 bits PCM format."""
if wav.dtype.is_floating_point: if wav.dtype.is_floating_point:
return wav return wav
elif wav.dtype == torch.int16: elif wav.dtype == torch.int16:
return wav.float() / (2 ** 15) return wav.float() / (2 ** 15)
elif wav.dtype == torch.int32: elif wav.dtype == torch.int32:
return wav.float() / (2 ** 31) return wav.float() / (2 ** 31)
raise ValueError(f"Unsupported wav dtype: {wav.dtype}") raise ValueError(f"Unsupported wav dtype: {wav.dtype}")
def load_audio_file(filepath: str) -> tuple[torch.Tensor, int]: def load_audio_file(filepath: str) -> tuple[torch.Tensor, int]:
"""Uses the latest ComfyUI av-based decoding for maximum compatibility.""" """Uses the latest ComfyUI av-based decoding for maximum compatibility."""
with av.open(filepath) as af: with av.open(filepath) as af:
if not af.streams.audio: if not af.streams.audio:
raise ValueError("No audio stream found in the file.") raise ValueError("No audio stream found in the file.")
stream = af.streams.audio[0] stream = af.streams.audio[0]
sr = stream.codec_context.sample_rate sr = stream.codec_context.sample_rate
n_channels = stream.channels n_channels = stream.channels
frames = [] frames = []
for frame in af.decode(streams=stream.index): for frame in af.decode(streams=stream.index):
buf = torch.from_numpy(frame.to_ndarray()) buf = torch.from_numpy(frame.to_ndarray())
if buf.shape[0] != n_channels: if buf.shape[0] != n_channels:
buf = buf.view(-1, n_channels).t() buf = buf.view(-1, n_channels).t()
frames.append(buf) frames.append(buf)
if not frames: if not frames:
raise ValueError("No audio frames decoded.") raise ValueError("No audio frames decoded.")
wav = torch.cat(frames, dim=1) wav = torch.cat(frames, dim=1)
wav = f32_pcm(wav) wav = f32_pcm(wav)
return wav, sr return wav, sr
class LoadAudioUI: class LoadAudioUI:
@classmethod @classmethod
def INPUT_TYPES(s): def INPUT_TYPES(s):
try: try:
files = folder_paths.get_filename_list("audio") files = folder_paths.get_filename_list("audio")
except: except:
files = [] files = []
if not files: if not files:
input_dir = folder_paths.get_input_directory() input_dir = folder_paths.get_input_directory()
if os.path.exists(input_dir): if os.path.exists(input_dir):
all_files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))] all_files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))]
try: try:
files = sorted(folder_paths.filter_files_content_types(all_files, ["audio", "video"])) files = sorted(folder_paths.filter_files_content_types(all_files, ["audio", "video"]))
except: except:
files = sorted(all_files) files = sorted(all_files)
if not files or len(files) == 0: if not files or len(files) == 0:
files = ["none"] files = ["none"]
return { return {
"required": { "required": {
"audio": (files, {"audio_upload": True}), # Moved to the top so it appears first "audio": (files, {"audio_upload": True}), # Moved to the top so it appears first
"start_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}), "start_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
"end_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}), "end_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
"duration": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}), "duration": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
}, },
"optional": { "optional": {
"audioUI": ("AUDIO_UI",) "audioUI": ("AUDIO_UI",)
} }
} }
CATEGORY = "WhatDreamsCost" CATEGORY = "WhatDreamsCost"
RETURN_TYPES = ("AUDIO", "FLOAT") RETURN_TYPES = ("AUDIO", "FLOAT")
RETURN_NAMES = ("audio", "duration") RETURN_NAMES = ("audio", "duration")
FUNCTION = "load_audio" FUNCTION = "load_audio"
@classmethod @classmethod
def VALIDATE_INPUTS(cls, audio, **kwargs): def VALIDATE_INPUTS(cls, audio, **kwargs):
# CRITICAL FIX: This bypasses the "Value not in list" error. # CRITICAL FIX: This bypasses the "Value not in list" error.
# By returning True, we tell ComfyUI to allow the 'audio' value even if it isn't in # By returning True, we tell ComfyUI to allow the 'audio' value even if it isn't in
# the current dropdown list. This allows the execution to reach load_audio(), # the current dropdown list. This allows the execution to reach load_audio(),
# where our fallback silence logic can handle the missing file gracefully. # where our fallback silence logic can handle the missing file gracefully.
return True return True
def load_audio(self, audio, start_time, end_time, duration, **kwargs): def load_audio(self, audio, start_time, end_time, duration, **kwargs):
# Determine the annotated file path if a file is actually selected # Determine the annotated file path if a file is actually selected
# We wrap this in a try/except because get_annotated_filepath can fail if # We wrap this in a try/except because get_annotated_filepath can fail if
# the input string is malformed or doesn't follow expected paths. # the input string is malformed or doesn't follow expected paths.
try: try:
audio_path = folder_paths.get_annotated_filepath(audio) if audio != "none" else "" audio_path = folder_paths.get_annotated_filepath(audio) if audio != "none" else ""
except: except:
audio_path = "" audio_path = ""
# --- FALLBACK LOGIC --- # --- FALLBACK LOGIC ---
# If the file is 'none' or doesn't exist on disk, provide 1 second of silence # If the file is 'none' or doesn't exist on disk, provide 1 second of silence
if audio == "none" or not audio_path or not os.path.exists(audio_path): if audio == "none" or not audio_path or not os.path.exists(audio_path):
missing_info = audio if audio != "none" else "None selected" missing_info = audio if audio != "none" else "None selected"
print(f"!!! [LoadAudioUI] Warning: Audio file '{missing_info}' not found. Outputting 1 second of silence.") print(f"!!! [LoadAudioUI] Warning: Audio file '{missing_info}' not found. Outputting 1 second of silence.")
sample_rate = 44100 sample_rate = 44100
# 1 second of silence (stereo) -> shape [channels, time] # 1 second of silence (stereo) -> shape [channels, time]
waveform = torch.zeros((2, 44100)) waveform = torch.zeros((2, 44100))
else: else:
try: try:
waveform, sample_rate = load_audio_file(audio_path) waveform, sample_rate = load_audio_file(audio_path)
except Exception as e: except Exception as e:
# If decoding fails for any reason, fallback to silence rather than crashing the workflow # If decoding fails for any reason, fallback to silence rather than crashing the workflow
print(f"!!! [LoadAudioUI] Error decoding {audio}: {e}. Falling back to silence.") print(f"!!! [LoadAudioUI] Error decoding {audio}: {e}. Falling back to silence.")
sample_rate = 44100 sample_rate = 44100
waveform = torch.zeros((2, 44100)) waveform = torch.zeros((2, 44100))
# Convert seconds to frames # Convert seconds to frames
start_frame = int(start_time * sample_rate) start_frame = int(start_time * sample_rate)
if end_time > 0: if end_time > 0:
end_frame = int(end_time * sample_rate) end_frame = int(end_time * sample_rate)
# Ensure the end_frame does not exceed the actual audio length # Ensure the end_frame does not exceed the actual audio length
end_frame = min(end_frame, waveform.shape[1]) end_frame = min(end_frame, waveform.shape[1])
else: else:
# 0 defaults to the end of the file # 0 defaults to the end of the file
end_frame = waveform.shape[1] end_frame = waveform.shape[1]
# Ensure start frame stays within bounds and doesn't pass the end frame # Ensure start frame stays within bounds and doesn't pass the end frame
start_frame = min(start_frame, end_frame) start_frame = min(start_frame, end_frame)
# Trim the waveform tensor -> shape: [channels, time] # Trim the waveform tensor -> shape: [channels, time]
trimmed_waveform = waveform[:, start_frame:end_frame] trimmed_waveform = waveform[:, start_frame:end_frame]
# Final safety check: if trimming resulted in zero length, give it a tiny bit of padding # Final safety check: if trimming resulted in zero length, give it a tiny bit of padding
# to prevent downstream nodes from crashing on empty tensors # to prevent downstream nodes from crashing on empty tensors
if trimmed_waveform.shape[1] == 0: if trimmed_waveform.shape[1] == 0:
trimmed_waveform = torch.zeros((waveform.shape[0], 1)) trimmed_waveform = torch.zeros((waveform.shape[0], 1))
# Format for ComfyUI's standard AUDIO type: [batch, channels, time] # Format for ComfyUI's standard AUDIO type: [batch, channels, time]
audio_output = {"waveform": trimmed_waveform.unsqueeze(0), "sample_rate": sample_rate} audio_output = {"waveform": trimmed_waveform.unsqueeze(0), "sample_rate": sample_rate}
# Calculate the final trimmed duration in seconds as a float # Calculate the final trimmed duration in seconds as a float
final_duration = float(trimmed_waveform.shape[1] / sample_rate) final_duration = float(trimmed_waveform.shape[1] / sample_rate)
return (audio_output, final_duration) return (audio_output, final_duration)

822
load_video_ui.py
View File

@@ -1,412 +1,412 @@
import os import os
import torch import torch
import numpy as np import numpy as np
import folder_paths import folder_paths
import av import av
from server import PromptServer from server import PromptServer
from aiohttp import web from aiohttp import web
import comfy.utils import comfy.utils
# Custom API route to serve video files from anywhere on the user's system for the frontend preview # Custom API route to serve video files from anywhere on the user's system for the frontend preview
@PromptServer.instance.routes.get("/video_ui_custom_view") @PromptServer.instance.routes.get("/video_ui_custom_view")
async def custom_view(request): async def custom_view(request):
file_path = request.query.get("filename", "") file_path = request.query.get("filename", "")
if os.path.exists(file_path) and os.path.isfile(file_path): if os.path.exists(file_path) and os.path.isfile(file_path):
return web.FileResponse(file_path) return web.FileResponse(file_path)
return web.Response(status=404, text="File not found") return web.Response(status=404, text="File not found")
# Custom API route for Chunked Uploads to bypass the 413 Payload Too Large error # Custom API route for Chunked Uploads to bypass the 413 Payload Too Large error
@PromptServer.instance.routes.post("/video_ui_upload_chunk") @PromptServer.instance.routes.post("/video_ui_upload_chunk")
async def upload_chunk(request): async def upload_chunk(request):
post = await request.post() post = await request.post()
file = post.get("file") file = post.get("file")
filename = post.get("filename") filename = post.get("filename")
chunk_index = int(post.get("chunk_index")) chunk_index = int(post.get("chunk_index"))
total_chunks = int(post.get("total_chunks")) total_chunks = int(post.get("total_chunks"))
upload_dir = folder_paths.get_input_directory() upload_dir = folder_paths.get_input_directory()
file_path = os.path.join(upload_dir, filename) file_path = os.path.join(upload_dir, filename)
# Append to file if it's not the first chunk, otherwise write new # Append to file if it's not the first chunk, otherwise write new
mode = "ab" if chunk_index > 0 else "wb" mode = "ab" if chunk_index > 0 else "wb"
with open(file_path, mode) as f: with open(file_path, mode) as f:
f.write(file.file.read()) f.write(file.file.read())
if chunk_index == total_chunks - 1: if chunk_index == total_chunks - 1:
return web.json_response({"name": filename}) return web.json_response({"name": filename})
return web.json_response({"status": "ok"}) return web.json_response({"status": "ok"})
class LoadVideoUI: class LoadVideoUI:
@classmethod @classmethod
def INPUT_TYPES(s): def INPUT_TYPES(s):
return { return {
"required": { "required": {
"video": ("STRING", {"default": ""}), "video": ("STRING", {"default": ""}),
"start_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}), "start_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
"end_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}), "end_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
"duration": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}), "duration": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
"start_frame": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}), "start_frame": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}),
"end_frame": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}), "end_frame": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}),
"duration_frames": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}), "duration_frames": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}),
"resize_method": (["maintain aspect ratio", "stretch to fit", "pad", "crop"], {"default": "maintain aspect ratio"}), "resize_method": (["maintain aspect ratio", "stretch to fit", "pad", "crop"], {"default": "maintain aspect ratio"}),
"custom_width": ("INT", {"default": 0, "min": 0, "max": 100000, "step": 8, "tooltip": "Custom width. 0 means original width."}), "custom_width": ("INT", {"default": 0, "min": 0, "max": 100000, "step": 8, "tooltip": "Custom width. 0 means original width."}),
"custom_height": ("INT", {"default": 0, "min": 0, "max": 100000, "step": 8, "tooltip": "Custom height. 0 means original height."}), "custom_height": ("INT", {"default": 0, "min": 0, "max": 100000, "step": 8, "tooltip": "Custom height. 0 means original height."}),
"frame_rate": ("INT", {"default": 24, "min": 1, "max": 120, "step": 1, "tooltip": "Force the video to a specific frame rate for extraction."}), "frame_rate": ("INT", {"default": 24, "min": 1, "max": 120, "step": 1, "tooltip": "Force the video to a specific frame rate for extraction."}),
"display_mode": (["seconds", "frames"], {"default": "seconds"}), "display_mode": (["seconds", "frames"], {"default": "seconds"}),
"crop_x": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}), "crop_x": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}),
"crop_y": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}), "crop_y": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}),
"crop_w": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}), "crop_w": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}),
"crop_h": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}), "crop_h": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}),
} }
} }
RETURN_TYPES = ("IMAGE", "AUDIO", "FLOAT", "INT") RETURN_TYPES = ("IMAGE", "AUDIO", "FLOAT", "INT")
RETURN_NAMES = ("images", "audio", "duration", "frame_count") RETURN_NAMES = ("images", "audio", "duration", "frame_count")
FUNCTION = "load_video" FUNCTION = "load_video"
CATEGORY = "WhatDreamsCost" CATEGORY = "WhatDreamsCost"
def load_video(self, video, frame_rate, display_mode, start_time, end_time, duration, start_frame, end_frame, duration_frames, custom_width=0, custom_height=0, resize_method="maintain aspect ratio", crop_x=0.0, crop_y=0.0, crop_w=1.0, crop_h=1.0, **kwargs): def load_video(self, video, frame_rate, display_mode, start_time, end_time, duration, start_frame, end_frame, duration_frames, custom_width=0, custom_height=0, resize_method="maintain aspect ratio", crop_x=0.0, crop_y=0.0, crop_w=1.0, crop_h=1.0, **kwargs):
if not video: if not video:
# Return blank defaults if no video is loaded # Return blank defaults if no video is loaded
empty_image = torch.zeros((1, 512, 512, 3), dtype=torch.float32) empty_image = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
empty_audio = {"waveform": torch.zeros((1, 1, 44100)), "sample_rate": 44100} empty_audio = {"waveform": torch.zeros((1, 1, 44100)), "sample_rate": 44100}
return (empty_image, empty_audio, 0.0, 0) return (empty_image, empty_audio, 0.0, 0)
# 1. Resolve path using ComfyUI standard paths or Absolute Path # 1. Resolve path using ComfyUI standard paths or Absolute Path
video_path = video # Try exact/absolute path first video_path = video # Try exact/absolute path first
if not os.path.exists(video_path): if not os.path.exists(video_path):
video_path_annotated = folder_paths.get_annotated_filepath(video) video_path_annotated = folder_paths.get_annotated_filepath(video)
if os.path.exists(video_path_annotated): if os.path.exists(video_path_annotated):
video_path = video_path_annotated video_path = video_path_annotated
else: else:
video_path_input = os.path.join(folder_paths.get_input_directory(), video) video_path_input = os.path.join(folder_paths.get_input_directory(), video)
if os.path.exists(video_path_input): if os.path.exists(video_path_input):
video_path = video_path_input video_path = video_path_input
else: else:
raise FileNotFoundError(f"Video file not found: {video}") raise FileNotFoundError(f"Video file not found: {video}")
# Open container to read streams and metadata # Open container to read streams and metadata
container = av.open(video_path) container = av.open(video_path)
# Determine video stream and duration # Determine video stream and duration
video_stream = container.streams.video[0] if len(container.streams.video) > 0 else None video_stream = container.streams.video[0] if len(container.streams.video) > 0 else None
video_duration = 0 video_duration = 0
if video_stream and video_stream.duration and video_stream.time_base: if video_stream and video_stream.duration and video_stream.time_base:
video_duration = float(video_stream.duration * video_stream.time_base) video_duration = float(video_stream.duration * video_stream.time_base)
orig_w = video_stream.codec_context.width if video_stream else 512 orig_w = video_stream.codec_context.width if video_stream else 512
orig_h = video_stream.codec_context.height if video_stream else 512 orig_h = video_stream.codec_context.height if video_stream else 512
# Determine correct colorspace and color range for PyAV conversion to prevent color shift # Determine correct colorspace and color range for PyAV conversion to prevent color shift
try: try:
from av.video.reformatter import Colorspace, ColorRange from av.video.reformatter import Colorspace, ColorRange
# Improve fallback heuristic to check both dimensions (e.g. 720x1280 vertical video is HD) # Improve fallback heuristic to check both dimensions (e.g. 720x1280 vertical video is HD)
fallback_cs = Colorspace.ITU709 if max(orig_w, orig_h) >= 720 else Colorspace.ITU601 fallback_cs = Colorspace.ITU709 if max(orig_w, orig_h) >= 720 else Colorspace.ITU601
fallback_cr = ColorRange.MPEG fallback_cr = ColorRange.MPEG
dst_range = ColorRange.JPEG # RGB should always be full range dst_range = ColorRange.JPEG # RGB should always be full range
except ImportError: except ImportError:
fallback_cs = "itu709" if max(orig_w, orig_h) >= 720 else "itu601" fallback_cs = "itu709" if max(orig_w, orig_h) >= 720 else "itu601"
fallback_cr = "mpeg" fallback_cr = "mpeg"
dst_range = "jpeg" dst_range = "jpeg"
src_colorspace = fallback_cs src_colorspace = fallback_cs
src_color_range = fallback_cr src_color_range = fallback_cr
if video_stream and video_stream.codec_context: if video_stream and video_stream.codec_context:
cc = video_stream.codec_context cc = video_stream.codec_context
c_space = getattr(cc, 'colorspace', getattr(cc, 'color_space', None)) c_space = getattr(cc, 'colorspace', getattr(cc, 'color_space', None))
if c_space and hasattr(c_space, 'name') and c_space.name != "UNSPECIFIED": if c_space and hasattr(c_space, 'name') and c_space.name != "UNSPECIFIED":
src_colorspace = c_space src_colorspace = c_space
elif c_space and isinstance(c_space, str) and "unspecified" not in c_space.lower(): elif c_space and isinstance(c_space, str) and "unspecified" not in c_space.lower():
src_colorspace = c_space src_colorspace = c_space
c_range = getattr(cc, 'color_range', None) c_range = getattr(cc, 'color_range', None)
if c_range and hasattr(c_range, 'name') and c_range.name != "UNSPECIFIED": if c_range and hasattr(c_range, 'name') and c_range.name != "UNSPECIFIED":
src_color_range = c_range src_color_range = c_range
elif c_range and isinstance(c_range, str) and "unspecified" not in c_range.lower(): elif c_range and isinstance(c_range, str) and "unspecified" not in c_range.lower():
src_color_range = c_range src_color_range = c_range
target_w = custom_width if custom_width > 0 else orig_w target_w = custom_width if custom_width > 0 else orig_w
target_h = custom_height if custom_height > 0 else orig_h target_h = custom_height if custom_height > 0 else orig_h
target_w = target_w - (target_w % 2) target_w = target_w - (target_w % 2)
target_h = target_h - (target_h % 2) target_h = target_h - (target_h % 2)
# Calculate manual crop from interactive UI first # Calculate manual crop from interactive UI first
manual_crop_left = int(orig_w * crop_x) manual_crop_left = int(orig_w * crop_x)
manual_crop_top = int(orig_h * crop_y) manual_crop_top = int(orig_h * crop_y)
manual_crop_right = orig_w - int(orig_w * (crop_x + crop_w)) manual_crop_right = orig_w - int(orig_w * (crop_x + crop_w))
manual_crop_bottom = orig_h - int(orig_h * (crop_y + crop_h)) manual_crop_bottom = orig_h - int(orig_h * (crop_y + crop_h))
# Ensure we don't crop more than the image # Ensure we don't crop more than the image
manual_crop_left = max(0, min(manual_crop_left, orig_w - 1)) manual_crop_left = max(0, min(manual_crop_left, orig_w - 1))
manual_crop_top = max(0, min(manual_crop_top, orig_h - 1)) manual_crop_top = max(0, min(manual_crop_top, orig_h - 1))
manual_crop_right = max(0, min(manual_crop_right, orig_w - manual_crop_left - 1)) manual_crop_right = max(0, min(manual_crop_right, orig_w - manual_crop_left - 1))
manual_crop_bottom = max(0, min(manual_crop_bottom, orig_h - manual_crop_top - 1)) manual_crop_bottom = max(0, min(manual_crop_bottom, orig_h - manual_crop_top - 1))
# After manual crop, the new original dimensions are: # After manual crop, the new original dimensions are:
cropped_orig_w = orig_w - manual_crop_left - manual_crop_right cropped_orig_w = orig_w - manual_crop_left - manual_crop_right
cropped_orig_h = orig_h - manual_crop_top - manual_crop_bottom cropped_orig_h = orig_h - manual_crop_top - manual_crop_bottom
# If no custom width/height is provided, use the cropped original dimensions # If no custom width/height is provided, use the cropped original dimensions
if custom_width == 0: if custom_width == 0:
target_w = cropped_orig_w target_w = cropped_orig_w
target_w = target_w - (target_w % 2) target_w = target_w - (target_w % 2)
if custom_height == 0: if custom_height == 0:
target_h = cropped_orig_h target_h = cropped_orig_h
target_h = target_h - (target_h % 2) target_h = target_h - (target_h % 2)
scale_w, scale_h = target_w, target_h scale_w, scale_h = target_w, target_h
pad_left = pad_right = pad_top = pad_bottom = 0 pad_left = pad_right = pad_top = pad_bottom = 0
crop_left = crop_right = crop_top = crop_bottom = 0 crop_left = crop_right = crop_top = crop_bottom = 0
if custom_width > 0 or custom_height > 0: if custom_width > 0 or custom_height > 0:
if resize_method == "maintain aspect ratio" or resize_method == "pad": if resize_method == "maintain aspect ratio" or resize_method == "pad":
ratio = min(target_w / cropped_orig_w, target_h / cropped_orig_h) ratio = min(target_w / cropped_orig_w, target_h / cropped_orig_h)
scale_w = int(cropped_orig_w * ratio) scale_w = int(cropped_orig_w * ratio)
scale_h = int(cropped_orig_h * ratio) scale_h = int(cropped_orig_h * ratio)
scale_w = scale_w - (scale_w % 2) scale_w = scale_w - (scale_w % 2)
scale_h = scale_h - (scale_h % 2) scale_h = scale_h - (scale_h % 2)
if resize_method == "pad": if resize_method == "pad":
pad_x = target_w - scale_w pad_x = target_w - scale_w
pad_y = target_h - scale_h pad_y = target_h - scale_h
pad_left = pad_x // 2 pad_left = pad_x // 2
pad_right = pad_x - pad_left pad_right = pad_x - pad_left
pad_top = pad_y // 2 pad_top = pad_y // 2
pad_bottom = pad_y - pad_top pad_bottom = pad_y - pad_top
else: else:
target_w, target_h = scale_w, scale_h target_w, target_h = scale_w, scale_h
elif resize_method == "crop": elif resize_method == "crop":
ratio = max(target_w / cropped_orig_w, target_h / cropped_orig_h) ratio = max(target_w / cropped_orig_w, target_h / cropped_orig_h)
scale_w = int(cropped_orig_w * ratio) scale_w = int(cropped_orig_w * ratio)
scale_h = int(cropped_orig_h * ratio) scale_h = int(cropped_orig_h * ratio)
scale_w = scale_w - (scale_w % 2) scale_w = scale_w - (scale_w % 2)
scale_h = scale_h - (scale_h % 2) scale_h = scale_h - (scale_h % 2)
crop_x = scale_w - target_w crop_x = scale_w - target_w
crop_y = scale_h - target_h crop_y = scale_h - target_h
crop_left = crop_x // 2 crop_left = crop_x // 2
crop_right = crop_x - crop_left crop_right = crop_x - crop_left
crop_top = crop_y // 2 crop_top = crop_y // 2
crop_bottom = crop_y - crop_top crop_bottom = crop_y - crop_top
elif resize_method == "stretch to fit": elif resize_method == "stretch to fit":
scale_w, scale_h = target_w, target_h scale_w, scale_h = target_w, target_h
# Determine exact bounds based on frontend mode # Determine exact bounds based on frontend mode
if display_mode == "frames": if display_mode == "frames":
fr = float(frame_rate) if frame_rate > 0 else 24.0 fr = float(frame_rate) if frame_rate > 0 else 24.0
actual_start_time = float(start_frame) / fr actual_start_time = float(start_frame) / fr
actual_end_time = float(end_frame) / fr if (end_frame > 0 and end_frame > start_frame) else video_duration actual_end_time = float(end_frame) / fr if (end_frame > 0 and end_frame > start_frame) else video_duration
else: else:
actual_start_time = start_time actual_start_time = start_time
actual_end_time = end_time if (end_time > 0 and end_time > start_time) else video_duration actual_end_time = end_time if (end_time > 0 and end_time > start_time) else video_duration
if actual_end_time <= 0: if actual_end_time <= 0:
actual_end_time = float('inf') # Fallback if duration is unknown actual_end_time = float('inf') # Fallback if duration is unknown
# 2. Extract Video Frames (PyAV) # 2. Extract Video Frames (PyAV)
frames = [] frames = []
image_tensor = None image_tensor = None
frames_loaded = 0 frames_loaded = 0
if video_stream: if video_stream:
video_stream.thread_type = "AUTO" # Enable multithreaded decoding video_stream.thread_type = "AUTO" # Enable multithreaded decoding
# Efficiently seek backwards to the nearest keyframe # Efficiently seek backwards to the nearest keyframe
if video_stream.time_base: if video_stream.time_base:
seek_pts = int(actual_start_time / float(video_stream.time_base)) seek_pts = int(actual_start_time / float(video_stream.time_base))
else: else:
seek_pts = int(actual_start_time * av.time_base) seek_pts = int(actual_start_time * av.time_base)
container.seek(seek_pts, stream=video_stream, backward=True) container.seek(seek_pts, stream=video_stream, backward=True)
# Custom sampling to force specific framerate # Custom sampling to force specific framerate
frame_interval = 1.0 / float(frame_rate) if frame_rate > 0 else 1.0/24.0 frame_interval = 1.0 / float(frame_rate) if frame_rate > 0 else 1.0/24.0
expected_target_time = actual_start_time expected_target_time = actual_start_time
# Pre-calculate expected frames # Pre-calculate expected frames
alloc_end_time = actual_end_time if actual_end_time != float('inf') else video_duration alloc_end_time = actual_end_time if actual_end_time != float('inf') else video_duration
expected_frames = 0 expected_frames = 0
if alloc_end_time > 0: if alloc_end_time > 0:
duration_to_extract = alloc_end_time - actual_start_time duration_to_extract = alloc_end_time - actual_start_time
if duration_to_extract > 0: if duration_to_extract > 0:
expected_frames = int(np.ceil(duration_to_extract / frame_interval)) + 2 expected_frames = int(np.ceil(duration_to_extract / frame_interval)) + 2
pbar = comfy.utils.ProgressBar(expected_frames) if expected_frames > 0 else None pbar = comfy.utils.ProgressBar(expected_frames) if expected_frames > 0 else None
for frame in container.decode(video_stream): for frame in container.decode(video_stream):
frame_time = frame.time frame_time = frame.time
if frame_time is None: if frame_time is None:
frame_time = float(frame.pts * float(video_stream.time_base)) if frame.pts and video_stream.time_base else 0.0 frame_time = float(frame.pts * float(video_stream.time_base)) if frame.pts and video_stream.time_base else 0.0
if frame_time < actual_start_time: if frame_time < actual_start_time:
continue continue
# Add a slight buffer (interval) to ensure we evaluate the boundary correctly # Add a slight buffer (interval) to ensure we evaluate the boundary correctly
if frame_time > actual_end_time + frame_interval: if frame_time > actual_end_time + frame_interval:
break break
# Fix PyAV color shift by forcing proper colorspace and range conversion. # Fix PyAV color shift by forcing proper colorspace and range conversion.
# Omit dst_colorspace so swscale defaults naturally for RGB output # Omit dst_colorspace so swscale defaults naturally for RGB output
# (passing it can cause the YUV matrix to be applied incorrectly). # (passing it can cause the YUV matrix to be applied incorrectly).
try: try:
frame = frame.reformat( frame = frame.reformat(
format="rgb24", format="rgb24",
src_colorspace=src_colorspace, src_colorspace=src_colorspace,
src_color_range=src_color_range, src_color_range=src_color_range,
dst_color_range=dst_range dst_color_range=dst_range
) )
frame_rgb = frame.to_ndarray(format='rgb24') frame_rgb = frame.to_ndarray(format='rgb24')
except Exception as e: except Exception as e:
# Fallback: if explicit color reformat fails, use PyAV's default conversion # Fallback: if explicit color reformat fails, use PyAV's default conversion
print(f"[LoadVideoUI] Color reformat failed, using default: {e}") print(f"[LoadVideoUI] Color reformat failed, using default: {e}")
frame_rgb = frame.to_ndarray(format='rgb24') frame_rgb = frame.to_ndarray(format='rgb24')
# Apply interactive crop first # Apply interactive crop first
if manual_crop_left > 0 or manual_crop_top > 0 or manual_crop_right > 0 or manual_crop_bottom > 0: if manual_crop_left > 0 or manual_crop_top > 0 or manual_crop_right > 0 or manual_crop_bottom > 0:
frame_rgb = frame_rgb[manual_crop_top:orig_h-manual_crop_bottom, manual_crop_left:orig_w-manual_crop_right, :] frame_rgb = frame_rgb[manual_crop_top:orig_h-manual_crop_bottom, manual_crop_left:orig_w-manual_crop_right, :]
# Now resize to the scaled dimensions # Now resize to the scaled dimensions
if scale_w != cropped_orig_w or scale_h != cropped_orig_h: if scale_w != cropped_orig_w or scale_h != cropped_orig_h:
import cv2 import cv2
frame_rgb = cv2.resize(frame_rgb, (scale_w, scale_h), interpolation=cv2.INTER_AREA) frame_rgb = cv2.resize(frame_rgb, (scale_w, scale_h), interpolation=cv2.INTER_AREA)
if crop_left > 0 or crop_top > 0 or crop_right > 0 or crop_bottom > 0: if crop_left > 0 or crop_top > 0 or crop_right > 0 or crop_bottom > 0:
frame_rgb = frame_rgb[crop_top:scale_h-crop_bottom, crop_left:scale_w-crop_right, :] frame_rgb = frame_rgb[crop_top:scale_h-crop_bottom, crop_left:scale_w-crop_right, :]
if pad_left > 0 or pad_top > 0 or pad_right > 0 or pad_bottom > 0: if pad_left > 0 or pad_top > 0 or pad_right > 0 or pad_bottom > 0:
frame_rgb = np.pad(frame_rgb, ((pad_top, pad_bottom), (pad_left, pad_right), (0, 0)), mode='constant', constant_values=0) frame_rgb = np.pad(frame_rgb, ((pad_top, pad_bottom), (pad_left, pad_right), (0, 0)), mode='constant', constant_values=0)
# Duplicate or skip frames perfectly based on timestamps to meet forced framerate. # Duplicate or skip frames perfectly based on timestamps to meet forced framerate.
# FIX: Use strictly less than (<) for actual_end_time to prevent the loop from fetching an extra +1 frame # FIX: Use strictly less than (<) for actual_end_time to prevent the loop from fetching an extra +1 frame
# at the exact boundary of the duration slice! # at the exact boundary of the duration slice!
while expected_target_time <= frame_time and expected_target_time < actual_end_time - 1e-5: while expected_target_time <= frame_time and expected_target_time < actual_end_time - 1e-5:
if image_tensor is None and expected_frames > 0: if image_tensor is None and expected_frames > 0:
# First frame: allocate the tensor # First frame: allocate the tensor
height, width = frame_rgb.shape[:2] height, width = frame_rgb.shape[:2]
alloc_frames = expected_frames + 50 # Add generous buffer to prevent reallocation alloc_frames = expected_frames + 50 # Add generous buffer to prevent reallocation
try: try:
image_tensor = torch.zeros((alloc_frames, height, width, 3), dtype=torch.float32) image_tensor = torch.zeros((alloc_frames, height, width, 3), dtype=torch.float32)
except Exception as e: except Exception as e:
print(f"[LoadVideoUI] Pre-allocation failed, falling back to list: {e}") print(f"[LoadVideoUI] Pre-allocation failed, falling back to list: {e}")
expected_frames = 0 # Disable pre-allocation expected_frames = 0 # Disable pre-allocation
if image_tensor is not None: if image_tensor is not None:
# Check bounds (just in case) # Check bounds (just in case)
if frames_loaded >= image_tensor.shape[0]: if frames_loaded >= image_tensor.shape[0]:
# Extend tensor if we underestimated # Extend tensor if we underestimated
extension = torch.zeros((50, image_tensor.shape[1], image_tensor.shape[2], 3), dtype=torch.float32) extension = torch.zeros((50, image_tensor.shape[1], image_tensor.shape[2], 3), dtype=torch.float32)
image_tensor = torch.cat((image_tensor, extension), dim=0) image_tensor = torch.cat((image_tensor, extension), dim=0)
# Insert frame with minimal memory copy directly to tensor # Insert frame with minimal memory copy directly to tensor
image_tensor[frames_loaded] = torch.from_numpy(frame_rgb).float().div_(255.0) image_tensor[frames_loaded] = torch.from_numpy(frame_rgb).float().div_(255.0)
frames_loaded += 1 frames_loaded += 1
else: else:
# Fallback list append if pre-allocation failed # Fallback list append if pre-allocation failed
frames.append(frame_rgb) frames.append(frame_rgb)
if pbar: if pbar:
pbar.update(1) pbar.update(1)
expected_target_time += frame_interval expected_target_time += frame_interval
# Convert frames to ComfyUI Image standard format [N, H, W, C], float32, range 0.0-1.0 # Convert frames to ComfyUI Image standard format [N, H, W, C], float32, range 0.0-1.0
if image_tensor is not None: if image_tensor is not None:
if frames_loaded > 0: if frames_loaded > 0:
image_tensor = image_tensor[:frames_loaded] image_tensor = image_tensor[:frames_loaded]
else: else:
image_tensor = torch.zeros((1, 512, 512, 3), dtype=torch.float32) image_tensor = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
elif len(frames) > 0: elif len(frames) > 0:
frames_np = np.array(frames, dtype=np.float32) / 255.0 frames_np = np.array(frames, dtype=np.float32) / 255.0
image_tensor = torch.from_numpy(frames_np) image_tensor = torch.from_numpy(frames_np)
else: else:
# Fallback for an empty slice # Fallback for an empty slice
image_tensor = torch.zeros((1, 512, 512, 3), dtype=torch.float32) image_tensor = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
# 3. Extract Audio (PyAV) # 3. Extract Audio (PyAV)
audio_dict = {"waveform": torch.zeros((1, 1, 44100)), "sample_rate": 44100} # Default empty audio audio_dict = {"waveform": torch.zeros((1, 1, 44100)), "sample_rate": 44100} # Default empty audio
if len(container.streams.audio) > 0: if len(container.streams.audio) > 0:
try: try:
audio_stream = container.streams.audio[0] audio_stream = container.streams.audio[0]
audio_stream.thread_type = "AUTO" audio_stream.thread_type = "AUTO"
sample_rate = getattr(audio_stream, 'rate', 44100) or 44100 sample_rate = getattr(audio_stream, 'rate', 44100) or 44100
# We must seek again on the container specifically for the audio stream # We must seek again on the container specifically for the audio stream
if audio_stream.time_base: if audio_stream.time_base:
seek_pts = int(actual_start_time / float(audio_stream.time_base)) seek_pts = int(actual_start_time / float(audio_stream.time_base))
else: else:
seek_pts = int(actual_start_time * av.time_base) seek_pts = int(actual_start_time * av.time_base)
container.seek(seek_pts, stream=audio_stream, backward=True) container.seek(seek_pts, stream=audio_stream, backward=True)
# Resample to standard float planar format (fltp) # Resample to standard float planar format (fltp)
resampler = av.AudioResampler(format='fltp') resampler = av.AudioResampler(format='fltp')
audio_data = [] audio_data = []
first_frame_time = None first_frame_time = None
for frame in container.decode(audio_stream): for frame in container.decode(audio_stream):
frame_time = frame.time frame_time = frame.time
if frame_time is None: if frame_time is None:
frame_time = float(frame.pts * float(audio_stream.time_base)) if frame.pts and audio_stream.time_base else 0.0 frame_time = float(frame.pts * float(audio_stream.time_base)) if frame.pts and audio_stream.time_base else 0.0
# Give a small 1-second buffer to ensure we catch end frames # Give a small 1-second buffer to ensure we catch end frames
if frame_time > actual_end_time + 1.0: if frame_time > actual_end_time + 1.0:
break break
if first_frame_time is None: if first_frame_time is None:
first_frame_time = frame_time first_frame_time = frame_time
resampled_frames = resampler.resample(frame) resampled_frames = resampler.resample(frame)
for r_frame in resampled_frames: for r_frame in resampled_frames:
audio_data.append(r_frame.to_ndarray()) audio_data.append(r_frame.to_ndarray())
if audio_data: if audio_data:
# Concatenate all frames horizontally along the sample axis # Concatenate all frames horizontally along the sample axis
waveform_np = np.concatenate(audio_data, axis=1) waveform_np = np.concatenate(audio_data, axis=1)
waveform = torch.from_numpy(waveform_np).float() waveform = torch.from_numpy(waveform_np).float()
if first_frame_time is None: if first_frame_time is None:
first_frame_time = 0.0 first_frame_time = 0.0
# Calculate exact slice points to trim precisely # Calculate exact slice points to trim precisely
offset_sec = max(0.0, actual_start_time - first_frame_time) offset_sec = max(0.0, actual_start_time - first_frame_time)
start_sample = int(offset_sec * sample_rate) start_sample = int(offset_sec * sample_rate)
duration_sec_audio = actual_end_time - actual_start_time duration_sec_audio = actual_end_time - actual_start_time
end_sample = start_sample + int(duration_sec_audio * sample_rate) end_sample = start_sample + int(duration_sec_audio * sample_rate)
# Trim array bounds properly # Trim array bounds properly
if end_sample > start_sample: if end_sample > start_sample:
waveform = waveform[:, start_sample:end_sample] waveform = waveform[:, start_sample:end_sample]
else: else:
waveform = waveform[:, start_sample:] waveform = waveform[:, start_sample:]
# Expand to ComfyUI Audio standard [batch_size, channels, samples] # Expand to ComfyUI Audio standard [batch_size, channels, samples]
waveform = waveform.unsqueeze(0) waveform = waveform.unsqueeze(0)
audio_dict = {"waveform": waveform, "sample_rate": sample_rate} audio_dict = {"waveform": waveform, "sample_rate": sample_rate}
except Exception as e: except Exception as e:
# Catch gracefully without breaking the pipeline execution # Catch gracefully without breaking the pipeline execution
print(f"[LoadVideoUI] Audio track extraction skipped or failed: {e}") print(f"[LoadVideoUI] Audio track extraction skipped or failed: {e}")
# Always close container to free up system memory lock # Always close container to free up system memory lock
container.close() container.close()
# Output accurate final duration in seconds # Output accurate final duration in seconds
final_duration_sec = float(max(0.0, actual_end_time - actual_start_time)) final_duration_sec = float(max(0.0, actual_end_time - actual_start_time))
# Accurately output the true number of extracted frames # Accurately output the true number of extracted frames
# (Using the shape of the array provides exact 1:1 parity with the timeline's math) # (Using the shape of the array provides exact 1:1 parity with the timeline's math)
frame_count = image_tensor.shape[0] if (frames_loaded > 0 or len(frames) > 0) else 0 frame_count = image_tensor.shape[0] if (frames_loaded > 0 or len(frames) > 0) else 0
if frame_count == 0 and final_duration_sec > 0: if frame_count == 0 and final_duration_sec > 0:
# Fallback estimation only if PyAV completely failed to decode a valid chunk # Fallback estimation only if PyAV completely failed to decode a valid chunk
calc_fr = float(frame_rate) if frame_rate > 0 else 24.0 calc_fr = float(frame_rate) if frame_rate > 0 else 24.0
frame_count = int(np.floor(final_duration_sec * calc_fr)) frame_count = int(np.floor(final_duration_sec * calc_fr))
return (image_tensor, audio_dict, final_duration_sec, frame_count) return (image_tensor, audio_dict, final_duration_sec, frame_count)

1537
ltx_director.py
View File

File diff suppressed because it is too large Load Diff

178
ltx_director_guide.py
View File

@@ -1,90 +1,90 @@
from comfy_extras.nodes_lt import LTXVAddGuide from comfy_extras.nodes_lt import LTXVAddGuide
import torch import torch
import comfy.utils import comfy.utils
from comfy_api.latest import io from comfy_api.latest import io
from .ltx_director import GuideData from .ltx_director import GuideData
class LTXDirectorGuide(LTXVAddGuide): class LTXDirectorGuide(LTXVAddGuide):
@classmethod @classmethod
def define_schema(cls): def define_schema(cls):
return io.Schema( return io.Schema(
node_id="LTXDirectorGuide", node_id="LTXDirectorGuide",
display_name="LTX Director Guide", display_name="LTX Director Guide",
category="WhatDreamsCost", category="WhatDreamsCost",
description=( description=(
"Applies guide images from a Prompt Relay Timeline node at the frame positions " "Applies guide images from a Prompt Relay Timeline node at the frame positions "
"and strengths defined on the timeline. Connect guide_data from the timeline node." "and strengths defined on the timeline. Connect guide_data from the timeline node."
), ),
inputs=[ inputs=[
io.Conditioning.Input("positive", tooltip="Positive conditioning to add guide keyframe info to."), io.Conditioning.Input("positive", tooltip="Positive conditioning to add guide keyframe info to."),
io.Conditioning.Input("negative", tooltip="Negative conditioning to add guide keyframe info to."), io.Conditioning.Input("negative", tooltip="Negative conditioning to add guide keyframe info to."),
io.Vae.Input("vae", tooltip="Video VAE used to encode the guide images."), io.Vae.Input("vae", tooltip="Video VAE used to encode the guide images."),
io.Latent.Input("latent", tooltip="Video latent — guides are inserted into this latent."), io.Latent.Input("latent", tooltip="Video latent — guides are inserted into this latent."),
GuideData.Input("guide_data", tooltip="Guide data produced by Prompt Relay Encode (Timeline)."), GuideData.Input("guide_data", tooltip="Guide data produced by Prompt Relay Encode (Timeline)."),
io.Float.Input("scale_by", default=1.0, min=0.01, max=8.0, step=0.01, tooltip="Scale the latent by this factor."), io.Float.Input("scale_by", default=1.0, min=0.01, max=8.0, step=0.01, tooltip="Scale the latent by this factor."),
io.Combo.Input("upscale_method", options=["nearest-exact", "bilinear", "area", "bicubic", "bislerp"], default="bicubic", tooltip="Method used to upscale/downscale the latent."), io.Combo.Input("upscale_method", options=["nearest-exact", "bilinear", "area", "bicubic", "bislerp"], default="bicubic", tooltip="Method used to upscale/downscale the latent."),
], ],
outputs=[ outputs=[
io.Conditioning.Output(display_name="positive"), io.Conditioning.Output(display_name="positive"),
io.Conditioning.Output(display_name="negative"), io.Conditioning.Output(display_name="negative"),
io.Latent.Output(display_name="latent", tooltip="Video latent with guide frames applied."), io.Latent.Output(display_name="latent", tooltip="Video latent with guide frames applied."),
], ],
) )
@classmethod @classmethod
def execute(cls, positive, negative, vae, latent, guide_data, scale_by=1.0, upscale_method="bicubic") -> io.NodeOutput: def execute(cls, positive, negative, vae, latent, guide_data, scale_by=1.0, upscale_method="bicubic") -> io.NodeOutput:
scale_factors = vae.downscale_index_formula scale_factors = vae.downscale_index_formula
# Clone latents to avoid mutating upstream nodes # Clone latents to avoid mutating upstream nodes
latent_image = latent["samples"].clone() latent_image = latent["samples"].clone()
if "noise_mask" in latent: if "noise_mask" in latent:
noise_mask = latent["noise_mask"].clone() noise_mask = latent["noise_mask"].clone()
else: else:
batch, _, latent_frames, latent_height, latent_width = latent_image.shape batch, _, latent_frames, latent_height, latent_width = latent_image.shape
noise_mask = torch.ones( noise_mask = torch.ones(
(batch, 1, latent_frames, 1, 1), (batch, 1, latent_frames, 1, 1),
dtype=torch.float32, dtype=torch.float32,
device=latent_image.device, device=latent_image.device,
) )
# Apply scale factor if not 1.0 # Apply scale factor if not 1.0
if scale_by != 1.0: if scale_by != 1.0:
B, C, F, H, W = latent_image.shape B, C, F, H, W = latent_image.shape
width = round(W * scale_by) width = round(W * scale_by)
height = round(H * scale_by) height = round(H * scale_by)
# Reshape to 4D for common_upscale # Reshape to 4D for common_upscale
latent_4d = latent_image.permute(0, 2, 1, 3, 4).reshape(B * F, C, H, W) latent_4d = latent_image.permute(0, 2, 1, 3, 4).reshape(B * F, C, H, W)
latent_resized_4d = comfy.utils.common_upscale(latent_4d, width, height, upscale_method, "disabled") latent_resized_4d = comfy.utils.common_upscale(latent_4d, width, height, upscale_method, "disabled")
latent_image = latent_resized_4d.reshape(B, F, C, height, width).permute(0, 2, 1, 3, 4) latent_image = latent_resized_4d.reshape(B, F, C, height, width).permute(0, 2, 1, 3, 4)
# Also resize noise mask if it's not a broadcasted mask # Also resize noise mask if it's not a broadcasted mask
if noise_mask.shape[-1] > 1 or noise_mask.shape[-2] > 1: if noise_mask.shape[-1] > 1 or noise_mask.shape[-2] > 1:
mask_4d = noise_mask.permute(0, 2, 1, 3, 4).reshape(B * F, 1, H, W) mask_4d = noise_mask.permute(0, 2, 1, 3, 4).reshape(B * F, 1, H, W)
mask_resized_4d = comfy.utils.common_upscale(mask_4d, width, height, upscale_method, "disabled") mask_resized_4d = comfy.utils.common_upscale(mask_4d, width, height, upscale_method, "disabled")
noise_mask = mask_resized_4d.reshape(B, F, 1, height, width).permute(0, 2, 1, 3, 4) noise_mask = mask_resized_4d.reshape(B, F, 1, height, width).permute(0, 2, 1, 3, 4)
_, _, latent_length, latent_height, latent_width = latent_image.shape _, _, latent_length, latent_height, latent_width = latent_image.shape
images = guide_data.get("images", []) images = guide_data.get("images", [])
insert_frames = guide_data.get("insert_frames", []) insert_frames = guide_data.get("insert_frames", [])
strengths = guide_data.get("strengths", []) strengths = guide_data.get("strengths", [])
for idx, img_tensor in enumerate(images): for idx, img_tensor in enumerate(images):
f_idx = insert_frames[idx] if idx < len(insert_frames) else 0 f_idx = insert_frames[idx] if idx < len(insert_frames) else 0
strength = strengths[idx] if idx < len(strengths) else 1.0 strength = strengths[idx] if idx < len(strengths) else 1.0
image_1, t = cls.encode(vae, latent_width, latent_height, img_tensor, scale_factors) image_1, t = cls.encode(vae, latent_width, latent_height, img_tensor, scale_factors)
frame_idx, latent_idx = cls.get_latent_index(positive, latent_length, len(image_1), f_idx, scale_factors) frame_idx, latent_idx = cls.get_latent_index(positive, latent_length, len(image_1), f_idx, scale_factors)
assert latent_idx + t.shape[2] <= latent_length, ( assert latent_idx + t.shape[2] <= latent_length, (
f"Guide image {idx + 1}: conditioning frames exceed the length of the latent sequence." f"Guide image {idx + 1}: conditioning frames exceed the length of the latent sequence."
) )
positive, negative, latent_image, noise_mask = cls.append_keyframe( positive, negative, latent_image, noise_mask = cls.append_keyframe(
positive, negative, frame_idx, latent_image, noise_mask, t, strength, scale_factors, positive, negative, frame_idx, latent_image, noise_mask, t, strength, scale_factors,
) )
return io.NodeOutput(positive, negative, {"samples": latent_image, "noise_mask": noise_mask}) return io.NodeOutput(positive, negative, {"samples": latent_image, "noise_mask": noise_mask})

232
ltx_keyframer.py
View File

@@ -1,117 +1,117 @@
import torch import torch
import comfy.utils import comfy.utils
from comfy_api.latest import io from comfy_api.latest import io
class LTXKeyframer(io.ComfyNode): class LTXKeyframer(io.ComfyNode):
@classmethod @classmethod
def define_schema(cls): def define_schema(cls):
inputs = [ inputs = [
io.Vae.Input("vae", tooltip="Video VAE used to encode the images"), io.Vae.Input("vae", tooltip="Video VAE used to encode the images"),
io.Latent.Input("latent", tooltip="Video latent to insert images into"), io.Latent.Input("latent", tooltip="Video latent to insert images into"),
io.Image.Input("multi_input", tooltip="Batched images from MultiImageLoader"), io.Image.Input("multi_input", tooltip="Batched images from MultiImageLoader"),
] ]
inputs.append(io.Int.Input("num_images", default=1, min=0, max=50, step=1, display_name="images_loaded", tooltip="Select how many index/strength widgets to configure.")) inputs.append(io.Int.Input("num_images", default=1, min=0, max=50, step=1, display_name="images_loaded", tooltip="Select how many index/strength widgets to configure."))
for i in range(1, 51): # 1 to 50 images for i in range(1, 51): # 1 to 50 images
inputs.extend([ inputs.extend([
io.Int.Input( io.Int.Input(
f"insert_frame_{i}", f"insert_frame_{i}",
default=0, default=0,
min=-9999, min=-9999,
max=9999, max=9999,
step=1, step=1,
tooltip=f"Frame insert_frame for image {i} (in pixel space).", tooltip=f"Frame insert_frame for image {i} (in pixel space).",
optional=True, optional=True,
), ),
io.Float.Input( io.Float.Input(
f"strength_{i}", f"strength_{i}",
default=1.0, default=1.0,
min=0.0, min=0.0,
max=1.0, max=1.0,
step=0.01, step=0.01,
tooltip=f"Strength for image {i}.", tooltip=f"Strength for image {i}.",
optional=True, optional=True,
), ),
]) ])
return io.Schema( return io.Schema(
node_id="LTXKeyframer", node_id="LTXKeyframer",
display_name="LTX Keyframer", display_name="LTX Keyframer",
category="WhatDreamsCost", category="WhatDreamsCost",
description="Replaces video latent frames with the encoded input images. Number of widgets is dynamically configured.", description="Replaces video latent frames with the encoded input images. Number of widgets is dynamically configured.",
inputs=inputs, inputs=inputs,
outputs=[ outputs=[
io.Latent.Output(display_name="latent", tooltip="The video latent with the images inserted and latent noise mask updated."), io.Latent.Output(display_name="latent", tooltip="The video latent with the images inserted and latent noise mask updated."),
], ],
) )
@classmethod @classmethod
def execute(cls, vae, latent, multi_input, num_images, **kwargs) -> io.NodeOutput: def execute(cls, vae, latent, multi_input, num_images, **kwargs) -> io.NodeOutput:
samples = latent["samples"].clone() samples = latent["samples"].clone()
scale_factors = vae.downscale_index_formula scale_factors = vae.downscale_index_formula
_, height_scale_factor, width_scale_factor = scale_factors _, height_scale_factor, width_scale_factor = scale_factors
batch, _, latent_frames, latent_height, latent_width = samples.shape batch, _, latent_frames, latent_height, latent_width = samples.shape
width = latent_width * width_scale_factor width = latent_width * width_scale_factor
height = latent_height * height_scale_factor height = latent_height * height_scale_factor
# Get existing noise mask if present, otherwise create new one # Get existing noise mask if present, otherwise create new one
if "noise_mask" in latent: if "noise_mask" in latent:
conditioning_latent_frames_mask = latent["noise_mask"].clone() conditioning_latent_frames_mask = latent["noise_mask"].clone()
else: else:
conditioning_latent_frames_mask = torch.ones( conditioning_latent_frames_mask = torch.ones(
(batch, 1, latent_frames, 1, 1), (batch, 1, latent_frames, 1, 1),
dtype=torch.float32, dtype=torch.float32,
device=samples.device, device=samples.device,
) )
batch_size = multi_input.shape[0] if multi_input is not None else 0 batch_size = multi_input.shape[0] if multi_input is not None else 0
# We process inputs up to num_images, extracting values from kwargs # We process inputs up to num_images, extracting values from kwargs
for i in range(1, num_images + 1): for i in range(1, num_images + 1):
# Skip if this image index exceeds the batch # Skip if this image index exceeds the batch
if i > batch_size: if i > batch_size:
continue continue
image = multi_input[i-1:i] # Extract the single image frame from the batch image = multi_input[i-1:i] # Extract the single image frame from the batch
if image is None: if image is None:
continue continue
insert_frame = kwargs.get(f"insert_frame_{i}") insert_frame = kwargs.get(f"insert_frame_{i}")
if insert_frame is None: if insert_frame is None:
continue continue
strength = kwargs.get(f"strength_{i}", 1.0) strength = kwargs.get(f"strength_{i}", 1.0)
if image.shape[1] != height or image.shape[2] != width: if image.shape[1] != height or image.shape[2] != width:
pixels = comfy.utils.common_upscale(image.movedim(-1, 1), width, height, "bilinear", "center").movedim(1, -1) pixels = comfy.utils.common_upscale(image.movedim(-1, 1), width, height, "bilinear", "center").movedim(1, -1)
else: else:
pixels = image pixels = image
encode_pixels = pixels[:, :, :, :3] encode_pixels = pixels[:, :, :, :3]
t = vae.encode(encode_pixels) t = vae.encode(encode_pixels)
# Convert pixel frame insert_frame to latent insert_frame # Convert pixel frame insert_frame to latent insert_frame
time_scale_factor = scale_factors[0] time_scale_factor = scale_factors[0]
# Handle negative indexing in pixel space # Handle negative indexing in pixel space
pixel_frame_count = (latent_frames - 1) * time_scale_factor + 1 pixel_frame_count = (latent_frames - 1) * time_scale_factor + 1
if insert_frame < 0: if insert_frame < 0:
insert_frame = pixel_frame_count + insert_frame insert_frame = pixel_frame_count + insert_frame
# Convert to latent insert_frame # Convert to latent insert_frame
latent_idx = insert_frame // time_scale_factor latent_idx = insert_frame // time_scale_factor
# Clamp to valid range # Clamp to valid range
latent_idx = max(0, min(latent_idx, latent_frames - 1)) latent_idx = max(0, min(latent_idx, latent_frames - 1))
# Calculate end insert_frame, ensuring we don't exceed latent_frames # Calculate end insert_frame, ensuring we don't exceed latent_frames
end_index = min(latent_idx + t.shape[2], latent_frames) end_index = min(latent_idx + t.shape[2], latent_frames)
# Replace samples at the specified insert_frame range # Replace samples at the specified insert_frame range
samples[:, :, latent_idx:end_index] = t[:, :, :end_index - latent_idx] samples[:, :, latent_idx:end_index] = t[:, :, :end_index - latent_idx]
# Update mask at the specified insert_frame range # Update mask at the specified insert_frame range
conditioning_latent_frames_mask[:, :, latent_idx:end_index] = 1.0 - strength conditioning_latent_frames_mask[:, :, latent_idx:end_index] = 1.0 - strength
return io.NodeOutput({"samples": samples, "noise_mask": conditioning_latent_frames_mask}) return io.NodeOutput({"samples": samples, "noise_mask": conditioning_latent_frames_mask})

307
ltx_sequencer.py
View File

@@ -1,133 +1,174 @@
from comfy_extras.nodes_lt import get_noise_mask, LTXVAddGuide import json
import torch from comfy_extras.nodes_lt import get_noise_mask, LTXVAddGuide
import comfy.utils import torch
from comfy_api.latest import io import comfy.utils
from comfy_api.latest import io
class LTXSequencer(LTXVAddGuide):
@classmethod class LTXSequencer(LTXVAddGuide):
def define_schema(cls): @classmethod
inputs = [ def define_schema(cls):
io.Conditioning.Input("positive", tooltip="Positive conditioning to which guide keyframe info will be added"), inputs = [
io.Conditioning.Input("negative", tooltip="Negative conditioning to which guide keyframe info will be added"), io.Conditioning.Input("positive", tooltip="Positive conditioning to which guide keyframe info will be added"),
io.Vae.Input("vae", tooltip="Video VAE used to encode the guide images"), io.Conditioning.Input("negative", tooltip="Negative conditioning to which guide keyframe info will be added"),
io.Latent.Input("latent", tooltip="Video latent, guides are added to the end of this latent"), io.Vae.Input("vae", tooltip="Video VAE used to encode the guide images"),
io.Image.Input("multi_input", tooltip="Batched images from MultiImageLoader"), io.Latent.Input("latent", tooltip="Video latent, guides are added to the end of this latent"),
] io.Image.Input("multi_input", tooltip="Batched images from MultiImageLoader"),
]
inputs.append(io.Int.Input("num_images", default=1, min=0, max=50, step=1, display_name="images_loaded", tooltip="Select how many index/strength widgets to configure."))
inputs.append(io.Int.Input("num_images", default=1, min=0, max=50, step=1, display_name="images_loaded", tooltip="Select how many index/strength widgets to configure."))
# New global settings widgets
inputs.append(io.Combo.Input("insert_mode", options=["frames", "seconds"], default="frames", tooltip="Select the method for determining insertion points.")) # New global settings widgets
inputs.append(io.Int.Input("frame_rate", default=24, min=1, max=120, step=1, tooltip="Video FPS (used for calculating second insertions).")) inputs.append(io.Combo.Input("insert_mode", options=["frames", "seconds"], default="frames", tooltip="Select the method for determining insertion points."))
inputs.append(io.Int.Input("frame_rate", default=24, min=1, max=120, step=1, tooltip="Video FPS (used for calculating second insertions)."))
for i in range(1, 51): # 1 to 50 images
inputs.extend([ for i in range(1, 51): # 1 to 50 images
io.Int.Input( inputs.extend([
f"insert_frame_{i}", io.Int.Input(
default=0, f"insert_frame_{i}",
min=-9999, default=0,
max=9999, min=-9999,
step=1, max=9999,
tooltip=f"Frame insert point for image {i} (in pixel space).", step=1,
optional=True, tooltip=f"Frame insert point for image {i} (in pixel space).",
), optional=True,
io.Float.Input( ),
f"insert_second_{i}", io.Float.Input(
default=0.0, f"insert_second_{i}",
min=0.0, default=0.0,
max=9999.0, min=0.0,
step=0.1, max=9999.0,
tooltip=f"Second insert point for image {i}.", step=0.1,
optional=True, tooltip=f"Second insert point for image {i}.",
), optional=True,
io.Float.Input( ),
f"strength_{i}", io.Float.Input(
default=1.0, f"strength_{i}",
min=0.0, default=1.0,
max=1.0, min=0.0,
step=0.01, max=1.0,
tooltip=f"Strength for image {i}.", step=0.01,
optional=True, tooltip=f"Strength for image {i}.",
), optional=True,
]) ),
])
return io.Schema(
node_id="LTXSequencer", return io.Schema(
display_name="LTX Sequencer", node_id="LTXSequencer",
category="WhatDreamsCost", display_name="LTX Sequencer",
description="Add multiple guide images at specified frame indices or seconds with strengths. Number of widgets is dynamically configured.", category="WhatDreamsCost",
inputs=inputs, description="Add multiple guide images at specified frame indices or seconds. Auto-syncs to Prompt Relay invisibly via the positive wire.",
outputs=[ inputs=inputs,
io.Conditioning.Output(display_name="positive"), outputs=[
io.Conditioning.Output(display_name="negative"), io.Conditioning.Output(display_name="positive"),
io.Latent.Output(display_name="latent", tooltip="Video latent with added guides"), io.Conditioning.Output(display_name="negative"),
], io.Latent.Output(display_name="latent", tooltip="Video latent with added guides"),
) # ---> ADDED SYNC LOG OUTPUT <---
io.String.Output(display_name="sync_log", tooltip="Outputs a readable text block showing the actual times used for each image"),
@classmethod ],
def execute(cls, positive, negative, vae, latent, multi_input, num_images, **kwargs) -> io.NodeOutput: )
scale_factors = vae.downscale_index_formula
@classmethod
# Clone latents to avoid overwriting previous nodes' operations def execute(cls, positive, negative, vae, latent, multi_input, num_images, **kwargs) -> io.NodeOutput:
latent_image = latent["samples"].clone() scale_factors = vae.downscale_index_formula
# Helper logic to fetch or generate a noise mask # Clone latents to avoid overwriting previous nodes' operations
if "noise_mask" in latent: latent_image = latent["samples"].clone()
noise_mask = latent["noise_mask"].clone()
else: # Helper logic to fetch or generate a noise mask
batch, _, latent_frames, latent_height, latent_width = latent_image.shape if "noise_mask" in latent:
noise_mask = torch.ones( noise_mask = latent["noise_mask"].clone()
(batch, 1, latent_frames, 1, 1), else:
dtype=torch.float32, batch, _, latent_frames, latent_height, latent_width = latent_image.shape
device=latent_image.device, noise_mask = torch.ones(
) (batch, 1, latent_frames, 1, 1),
dtype=torch.float32,
_, _, latent_length, latent_height, latent_width = latent_image.shape device=latent_image.device,
batch_size = multi_input.shape[0] if multi_input is not None else 0 )
# Retrieve selected insertion settings _, _, latent_length, latent_height, latent_width = latent_image.shape
insert_mode = kwargs.get("insert_mode", "frames") batch_size = multi_input.shape[0] if multi_input is not None else 0
frame_rate = kwargs.get("frame_rate", 24)
# Retrieve selected insertion settings
# Process inputs up to num_images, extracting dynamic frame/strength values from kwargs insert_mode = kwargs.get("insert_mode", "frames")
for i in range(1, num_images + 1): frame_rate = kwargs.get("frame_rate", 24)
# Skip if this image index exceeds the batch
if i > batch_size: # ---> EXTRACT GHOST SYNC DATA <---
continue timeline_data = None
if positive is not None and len(positive) > 0:
img = multi_input[i-1:i] # Extract the single image frame from the batch if "prompt_relay_timeline" in positive[0][1]:
if img is None: try:
continue timeline_data = json.loads(positive[0][1]["prompt_relay_timeline"])
except json.JSONDecodeError:
# Calculate the final frame index based on the chosen mode pass
f_idx = None
if insert_mode == "frames": # Prepare the log text
f_idx = kwargs.get(f"insert_frame_{i}") sync_log_lines = []
elif insert_mode == "seconds": if timeline_data:
sec = kwargs.get(f"insert_second_{i}") sync_log_lines.append("=== TIMELINE SYNC ENABLED ===")
if sec is not None: sync_log_lines.append(f"Invisibly Synced via Positive Wire! Using {insert_mode.upper()}:")
f_idx = int(sec * frame_rate) else:
sync_log_lines.append("=== MANUAL MODE ===")
if f_idx is None: sync_log_lines.append(f"No timeline detected, or Sync turned OFF. Using manual UI inputs ({insert_mode}):")
continue
# Process inputs up to num_images, extracting dynamic frame/strength values from kwargs
strength = kwargs.get(f"strength_{i}", 1.0) for i in range(1, num_images + 1):
# Skip if this image index exceeds the batch
# Execution logic mirrored from LTXVAddGuideMulti if i > batch_size:
image_1, t = cls.encode(vae, latent_width, latent_height, img, scale_factors) sync_log_lines.append(f"Image #{i}: Skipped (No image loaded in batch)")
continue
frame_idx, latent_idx = cls.get_latent_index(positive, latent_length, len(image_1), f_idx, scale_factors)
assert latent_idx + t.shape[2] <= latent_length, "Conditioning frames exceed the length of the latent sequence." img = multi_input[i-1:i] # Extract the single image frame from the batch
if img is None:
positive, negative, latent_image, noise_mask = cls.append_keyframe( continue
positive,
negative, f_idx = None
frame_idx, strength = kwargs.get(f"strength_{i}", 1.0)
latent_image,
noise_mask, # 1. AUTO-SYNC
t, if timeline_data and "starts_frames" in timeline_data and (i - 1) < len(timeline_data["starts_frames"]):
strength, display_frame = timeline_data["starts_frames"][i - 1]
scale_factors, display_sec = timeline_data["starts_seconds"][i - 1]
)
if insert_mode == "frames":
return io.NodeOutput(positive, negative, {"samples": latent_image, "noise_mask": noise_mask}) f_idx = display_frame
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Synced to Segment #{i} start @ {display_frame} frames")
elif insert_mode == "seconds":
f_idx = int(display_sec * frame_rate)
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Synced to Segment #{i} start @ {display_sec:.2f} seconds (Frame {f_idx})")
# 2. MANUAL FALLBACK
else:
if insert_mode == "frames":
f_idx = kwargs.get(f"insert_frame_{i}")
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Manual input @ {f_idx} frames")
elif insert_mode == "seconds":
sec = kwargs.get(f"insert_second_{i}")
if sec is not None:
f_idx = int(sec * frame_rate)
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Manual input @ {sec:.2f} seconds (Frame {f_idx})")
if f_idx is None:
continue
# Execution logic mirrored from LTXVAddGuideMulti
image_1, t = cls.encode(vae, latent_width, latent_height, img, scale_factors)
frame_idx, latent_idx = cls.get_latent_index(positive, latent_length, len(image_1), f_idx, scale_factors)
assert latent_idx + t.shape[2] <= latent_length, "Conditioning frames exceed the length of the latent sequence."
positive, negative, latent_image, noise_mask = cls.append_keyframe(
positive,
negative,
frame_idx,
latent_image,
noise_mask,
t,
strength,
scale_factors,
)
sync_log_str = "\n".join(sync_log_lines)
print(f"\n[LTX Sequencer]\n{sync_log_str}\n")
# Returned the newly formatted sync log string at the end of NodeOutput
return io.NodeOutput(positive, negative, {"samples": latent_image, "noise_mask": noise_mask}, sync_log_str)

362
multi_image_loader.py
View File

@@ -1,182 +1,182 @@
import torch import torch
import torch.nn.functional as F import torch.nn.functional as F
import numpy as np import numpy as np
from PIL import Image, ImageOps from PIL import Image, ImageOps
import os import os
import folder_paths import folder_paths
import io import io
import comfy.utils import comfy.utils
class MultiImageLoader: class MultiImageLoader:
@classmethod @classmethod
def INPUT_TYPES(s): def INPUT_TYPES(s):
return { return {
"required": { "required": {
"image_paths": ("STRING", {"default": "", "multiline": True}), "image_paths": ("STRING", {"default": "", "multiline": True}),
"width": ("INT", {"default": 0, "min": 0, "max": 8192, "step": 1}), "width": ("INT", {"default": 0, "min": 0, "max": 8192, "step": 1}),
"height": ("INT", {"default": 0, "min": 0, "max": 8192, "step": 1}), "height": ("INT", {"default": 0, "min": 0, "max": 8192, "step": 1}),
"interpolation": (["lanczos", "nearest", "bilinear", "bicubic", "area", "nearest-exact"],), "interpolation": (["lanczos", "nearest", "bilinear", "bicubic", "area", "nearest-exact"],),
"resize_method": (["keep proportion", "stretch", "pad", "crop"],), "resize_method": (["keep proportion", "stretch", "pad", "crop"],),
"multiple_of": ("INT", {"default": 32, "min": 0, "max": 512, "step": 1}), "multiple_of": ("INT", {"default": 32, "min": 0, "max": 512, "step": 1}),
"img_compression": ("INT", {"default": 18, "min": 0, "max": 100, "step": 1}), "img_compression": ("INT", {"default": 18, "min": 0, "max": 100, "step": 1}),
}, },
} }
# Added "IMAGE" at the beginning for multi_output + 50 individual outputs = 51 outputs # Added "IMAGE" at the beginning for multi_output + 50 individual outputs = 51 outputs
RETURN_TYPES = ("IMAGE",) * 51 RETURN_TYPES = ("IMAGE",) * 51
RETURN_NAMES = ("multi_output",) + tuple(f"image_{i+1}" for i in range(50)) RETURN_NAMES = ("multi_output",) + tuple(f"image_{i+1}" for i in range(50))
FUNCTION = "load_images" FUNCTION = "load_images"
CATEGORY = "WhatDreamsCost" CATEGORY = "WhatDreamsCost"
def resize_image(self, image, width, height, resize_method="keep proportion", interpolation="nearest", multiple_of=0): def resize_image(self, image, width, height, resize_method="keep proportion", interpolation="nearest", multiple_of=0):
MAX_RESOLUTION = 8192 MAX_RESOLUTION = 8192
_, oh, ow, _ = image.shape _, oh, ow, _ = image.shape
x = y = x2 = y2 = 0 x = y = x2 = y2 = 0
pad_left = pad_right = pad_top = pad_bottom = 0 pad_left = pad_right = pad_top = pad_bottom = 0
if multiple_of > 1: if multiple_of > 1:
width = width - (width % multiple_of) width = width - (width % multiple_of)
height = height - (height % multiple_of) height = height - (height % multiple_of)
if resize_method == 'keep proportion' or resize_method == 'pad': if resize_method == 'keep proportion' or resize_method == 'pad':
if width == 0 and oh < height: if width == 0 and oh < height:
width = MAX_RESOLUTION width = MAX_RESOLUTION
elif width == 0 and oh >= height: elif width == 0 and oh >= height:
width = ow width = ow
if height == 0 and ow < width: if height == 0 and ow < width:
height = MAX_RESOLUTION height = MAX_RESOLUTION
elif height == 0 and ow >= width: elif height == 0 and ow >= width:
height = oh height = oh
ratio = min(width / ow, height / oh) ratio = min(width / ow, height / oh)
new_width = round(ow * ratio) new_width = round(ow * ratio)
new_height = round(oh * ratio) new_height = round(oh * ratio)
if resize_method == 'pad': if resize_method == 'pad':
pad_left = (width - new_width) // 2 pad_left = (width - new_width) // 2
pad_right = width - new_width - pad_left pad_right = width - new_width - pad_left
pad_top = (height - new_height) // 2 pad_top = (height - new_height) // 2
pad_bottom = height - new_height - pad_top pad_bottom = height - new_height - pad_top
width = new_width width = new_width
height = new_height height = new_height
elif resize_method == 'crop': elif resize_method == 'crop':
width = width if width > 0 else ow width = width if width > 0 else ow
height = height if height > 0 else oh height = height if height > 0 else oh
ratio = max(width / ow, height / oh) ratio = max(width / ow, height / oh)
new_width = round(ow * ratio) new_width = round(ow * ratio)
new_height = round(oh * ratio) new_height = round(oh * ratio)
x = (new_width - width) // 2 x = (new_width - width) // 2
y = (new_height - height) // 2 y = (new_height - height) // 2
x2 = x + width x2 = x + width
y2 = y + height y2 = y + height
if x2 > new_width: if x2 > new_width:
x -= (x2 - new_width) x -= (x2 - new_width)
if x < 0: if x < 0:
x = 0 x = 0
if y2 > new_height: if y2 > new_height:
y -= (y2 - new_height) y -= (y2 - new_height)
if y < 0: if y < 0:
y = 0 y = 0
width = new_width width = new_width
height = new_height height = new_height
else: else:
width = width if width > 0 else ow width = width if width > 0 else ow
height = height if height > 0 else oh height = height if height > 0 else oh
# Always apply resize logic # Always apply resize logic
outputs = image.permute(0, 3, 1, 2) outputs = image.permute(0, 3, 1, 2)
if interpolation == "lanczos": if interpolation == "lanczos":
outputs = comfy.utils.lanczos(outputs, width, height) outputs = comfy.utils.lanczos(outputs, width, height)
else: else:
outputs = F.interpolate(outputs, size=(height, width), mode=interpolation) outputs = F.interpolate(outputs, size=(height, width), mode=interpolation)
if resize_method == 'pad': if resize_method == 'pad':
if pad_left > 0 or pad_right > 0 or pad_top > 0 or pad_bottom > 0: if pad_left > 0 or pad_right > 0 or pad_top > 0 or pad_bottom > 0:
outputs = F.pad(outputs, (pad_left, pad_right, pad_top, pad_bottom), value=0) outputs = F.pad(outputs, (pad_left, pad_right, pad_top, pad_bottom), value=0)
outputs = outputs.permute(0, 2, 3, 1) outputs = outputs.permute(0, 2, 3, 1)
if resize_method == 'crop': if resize_method == 'crop':
if x > 0 or y > 0 or x2 > 0 or y2 > 0: if x > 0 or y > 0 or x2 > 0 or y2 > 0:
outputs = outputs[:, y:y2, x:x2, :] outputs = outputs[:, y:y2, x:x2, :]
if multiple_of > 1 and (outputs.shape[2] % multiple_of != 0 or outputs.shape[1] % multiple_of != 0): if multiple_of > 1 and (outputs.shape[2] % multiple_of != 0 or outputs.shape[1] % multiple_of != 0):
width = outputs.shape[2] width = outputs.shape[2]
height = outputs.shape[1] height = outputs.shape[1]
x = (width % multiple_of) // 2 x = (width % multiple_of) // 2
y = (height % multiple_of) // 2 y = (height % multiple_of) // 2
x2 = width - ((width % multiple_of) - x) x2 = width - ((width % multiple_of) - x)
y2 = height - ((height % multiple_of) - y) y2 = height - ((height % multiple_of) - y)
outputs = outputs[:, y:y2, x:x2, :] outputs = outputs[:, y:y2, x:x2, :]
outputs = torch.clamp(outputs, 0, 1) outputs = torch.clamp(outputs, 0, 1)
return outputs return outputs
def load_images(self, image_paths, width, height, interpolation, resize_method, multiple_of, img_compression): def load_images(self, image_paths, width, height, interpolation, resize_method, multiple_of, img_compression):
results = [] results = []
valid_paths = [p.strip() for p in image_paths.split("\n") if p.strip()] valid_paths = [p.strip() for p in image_paths.split("\n") if p.strip()]
for path in valid_paths: for path in valid_paths:
try: try:
# Resolve full path # Resolve full path
full_path = path full_path = path
if not os.path.exists(full_path): if not os.path.exists(full_path):
full_path = os.path.join(folder_paths.get_input_directory(), path) full_path = os.path.join(folder_paths.get_input_directory(), path)
if not os.path.exists(full_path): if not os.path.exists(full_path):
print(f"Warning: Image path not found: {path}") print(f"Warning: Image path not found: {path}")
continue continue
# Load image # Load image
image = Image.open(full_path) image = Image.open(full_path)
image = ImageOps.exif_transpose(image) image = ImageOps.exif_transpose(image)
image = image.convert("RGB") image = image.convert("RGB")
# Convert to Torch Tensor to prepare for Advanced Resize Logic # Convert to Torch Tensor to prepare for Advanced Resize Logic
image_np = np.array(image).astype(np.float32) / 255.0 image_np = np.array(image).astype(np.float32) / 255.0
image_tensor = torch.from_numpy(image_np)[None,] image_tensor = torch.from_numpy(image_np)[None,]
# Apply Advanced Resize # Apply Advanced Resize
image_tensor = self.resize_image(image_tensor, width, height, resize_method, interpolation, multiple_of) image_tensor = self.resize_image(image_tensor, width, height, resize_method, interpolation, multiple_of)
# Compression (Applied after resize to accurately maintain the effect) # Compression (Applied after resize to accurately maintain the effect)
if img_compression > 0: if img_compression > 0:
img_np = (image_tensor[0].numpy() * 255).clip(0, 255).astype(np.uint8) img_np = (image_tensor[0].numpy() * 255).clip(0, 255).astype(np.uint8)
img_pil = Image.fromarray(img_np) img_pil = Image.fromarray(img_np)
img_byte_arr = io.BytesIO() img_byte_arr = io.BytesIO()
img_pil.save(img_byte_arr, format="JPEG", quality=max(1, 100 - img_compression)) img_pil.save(img_byte_arr, format="JPEG", quality=max(1, 100 - img_compression))
img_pil = Image.open(img_byte_arr) img_pil = Image.open(img_byte_arr)
image_tensor = torch.from_numpy(np.array(img_pil).astype(np.float32) / 255.0)[None,] image_tensor = torch.from_numpy(np.array(img_pil).astype(np.float32) / 255.0)[None,]
results.append(image_tensor) results.append(image_tensor)
except Exception as e: except Exception as e:
print(f"Error loading {path}: {e}") print(f"Error loading {path}: {e}")
# Combine all successfully loaded images into a single batched tensor for multi_output # Combine all successfully loaded images into a single batched tensor for multi_output
if len(results) > 0: if len(results) > 0:
# Safety Check: Advanced resize methods might output differently sized tensors (e.g., 'keep proportion') # Safety Check: Advanced resize methods might output differently sized tensors (e.g., 'keep proportion')
first_shape = results[0].shape first_shape = results[0].shape
all_same_shape = all(r.shape == first_shape for r in results) all_same_shape = all(r.shape == first_shape for r in results)
if all_same_shape: if all_same_shape:
multi_output = torch.cat(results, dim=0) multi_output = torch.cat(results, dim=0)
else: else:
print("MultiImageLoader Warning: Images have different dimensions due to resize settings. Cannot batch into multi_output. Outputting zero tensor for the batch, but individual output nodes will still work fine.") print("MultiImageLoader Warning: Images have different dimensions due to resize settings. Cannot batch into multi_output. Outputting zero tensor for the batch, but individual output nodes will still work fine.")
multi_output = torch.zeros((1, 64, 64, 3)) multi_output = torch.zeros((1, 64, 64, 3))
else: else:
# Fallback empty tensor if no valid paths # Fallback empty tensor if no valid paths
multi_output = torch.zeros((1, 64, 64, 3)) multi_output = torch.zeros((1, 64, 64, 3))
results = [multi_output] results = [multi_output]
# Pad individual outputs exactly to length 50 as defined in RETURN_TYPES # Pad individual outputs exactly to length 50 as defined in RETURN_TYPES
padded_results = results + [torch.zeros((1, 64, 64, 3))] * (50 - len(results)) padded_results = results + [torch.zeros((1, 64, 64, 3))] * (50 - len(results))
# Return the multi batch output first, followed by the individual padded items # Return the multi batch output first, followed by the individual padded items
return (multi_output, *padded_results[:50]) return (multi_output, *padded_results[:50])

334
patches.py
View File

@@ -1,167 +1,167 @@
import types import types
import torch import torch
import comfy.ldm.modules.attention import comfy.ldm.modules.attention
def _masked_attention(q, k, v, heads, mask, transformer_options={}, **kwargs): def _masked_attention(q, k, v, heads, mask, transformer_options={}, **kwargs):
# Bypass wrap_attn (sage/etc may ignore masks) by calling attention_pytorch directly. # Bypass wrap_attn (sage/etc may ignore masks) by calling attention_pytorch directly.
return comfy.ldm.modules.attention.attention_pytorch( return comfy.ldm.modules.attention.attention_pytorch(
q, k, v, heads, mask=mask, q, k, v, heads, mask=mask,
_inside_attn_wrapper=True, _inside_attn_wrapper=True,
transformer_options=transformer_options, transformer_options=transformer_options,
**kwargs, **kwargs,
) )
def _wan_t2v_forward(self, mask_fn, x, context, transformer_options={}, **kwargs): def _wan_t2v_forward(self, mask_fn, x, context, transformer_options={}, **kwargs):
q = self.norm_q(self.q(x)) q = self.norm_q(self.q(x))
k = self.norm_k(self.k(context)) k = self.norm_k(self.k(context))
v = self.v(context) v = self.v(context)
mask = mask_fn(q, k, transformer_options) mask = mask_fn(q, k, transformer_options)
if mask is not None: if mask is not None:
x = _masked_attention(q, k, v, heads=self.num_heads, mask=mask, x = _masked_attention(q, k, v, heads=self.num_heads, mask=mask,
transformer_options=transformer_options) transformer_options=transformer_options)
else: else:
x = comfy.ldm.modules.attention.optimized_attention( x = comfy.ldm.modules.attention.optimized_attention(
q, k, v, heads=self.num_heads, transformer_options=transformer_options, q, k, v, heads=self.num_heads, transformer_options=transformer_options,
) )
return self.o(x) return self.o(x)
def _wan_i2v_forward(self, mask_fn, x, context, context_img_len, transformer_options={}, **kwargs): def _wan_i2v_forward(self, mask_fn, x, context, context_img_len, transformer_options={}, **kwargs):
context_img = context[:, :context_img_len] context_img = context[:, :context_img_len]
context_text = context[:, context_img_len:] context_text = context[:, context_img_len:]
q = self.norm_q(self.q(x)) q = self.norm_q(self.q(x))
k_img = self.norm_k_img(self.k_img(context_img)) k_img = self.norm_k_img(self.k_img(context_img))
v_img = self.v_img(context_img) v_img = self.v_img(context_img)
img_x = comfy.ldm.modules.attention.optimized_attention( img_x = comfy.ldm.modules.attention.optimized_attention(
q, k_img, v_img, heads=self.num_heads, transformer_options=transformer_options, q, k_img, v_img, heads=self.num_heads, transformer_options=transformer_options,
) )
k = self.norm_k(self.k(context_text)) k = self.norm_k(self.k(context_text))
v = self.v(context_text) v = self.v(context_text)
mask = mask_fn(q, k, transformer_options) mask = mask_fn(q, k, transformer_options)
if mask is not None: if mask is not None:
x = _masked_attention(q, k, v, heads=self.num_heads, mask=mask, x = _masked_attention(q, k, v, heads=self.num_heads, mask=mask,
transformer_options=transformer_options) transformer_options=transformer_options)
else: else:
x = comfy.ldm.modules.attention.optimized_attention( x = comfy.ldm.modules.attention.optimized_attention(
q, k, v, heads=self.num_heads, transformer_options=transformer_options, q, k, v, heads=self.num_heads, transformer_options=transformer_options,
) )
return self.o(x + img_x) return self.o(x + img_x)
def _ltx_forward(self, mask_fn, x, context=None, mask=None, pe=None, k_pe=None, transformer_options={}): def _ltx_forward(self, mask_fn, x, context=None, mask=None, pe=None, k_pe=None, transformer_options={}):
from comfy.ldm.lightricks.model import apply_rotary_emb from comfy.ldm.lightricks.model import apply_rotary_emb
is_self_attn = context is None is_self_attn = context is None
context = x if is_self_attn else context context = x if is_self_attn else context
q = self.q_norm(self.to_q(x)) q = self.q_norm(self.to_q(x))
k = self.k_norm(self.to_k(context)) k = self.k_norm(self.to_k(context))
v = self.to_v(context) v = self.to_v(context)
if pe is not None: if pe is not None:
q = apply_rotary_emb(q, pe) q = apply_rotary_emb(q, pe)
k = apply_rotary_emb(k, pe if k_pe is None else k_pe) k = apply_rotary_emb(k, pe if k_pe is None else k_pe)
if not is_self_attn: if not is_self_attn:
temporal_mask = mask_fn(q, k, transformer_options) temporal_mask = mask_fn(q, k, transformer_options)
if temporal_mask is not None: if temporal_mask is not None:
mask = temporal_mask if mask is None else mask + temporal_mask mask = temporal_mask if mask is None else mask + temporal_mask
if mask is None: if mask is None:
out = comfy.ldm.modules.attention.optimized_attention( out = comfy.ldm.modules.attention.optimized_attention(
q, k, v, self.heads, attn_precision=self.attn_precision, q, k, v, self.heads, attn_precision=self.attn_precision,
transformer_options=transformer_options, transformer_options=transformer_options,
) )
else: else:
out = _masked_attention(q, k, v, self.heads, mask=mask, out = _masked_attention(q, k, v, self.heads, mask=mask,
attn_precision=self.attn_precision, attn_precision=self.attn_precision,
transformer_options=transformer_options) transformer_options=transformer_options)
if self.to_gate_logits is not None: if self.to_gate_logits is not None:
gate_logits = self.to_gate_logits(x) gate_logits = self.to_gate_logits(x)
b, t, _ = out.shape b, t, _ = out.shape
out = out.view(b, t, self.heads, self.dim_head) out = out.view(b, t, self.heads, self.dim_head)
out = out * (2.0 * torch.sigmoid(gate_logits)).unsqueeze(-1) out = out * (2.0 * torch.sigmoid(gate_logits)).unsqueeze(-1)
out = out.view(b, t, self.heads * self.dim_head) out = out.view(b, t, self.heads * self.dim_head)
return self.to_out(out) return self.to_out(out)
class _CrossAttnPatch: class _CrossAttnPatch:
"""Descriptor that binds (impl, mask_fn) as a method onto a cross-attn module.""" """Descriptor that binds (impl, mask_fn) as a method onto a cross-attn module."""
def __init__(self, impl, mask_fn): def __init__(self, impl, mask_fn):
self.impl = impl self.impl = impl
self.mask_fn = mask_fn self.mask_fn = mask_fn
def __get__(self, obj, objtype=None): def __get__(self, obj, objtype=None):
impl, mask_fn = self.impl, self.mask_fn impl, mask_fn = self.impl, self.mask_fn
def wrapped(self_module, *args, **kwargs): def wrapped(self_module, *args, **kwargs):
return impl(self_module, mask_fn, *args, **kwargs) return impl(self_module, mask_fn, *args, **kwargs)
return types.MethodType(wrapped, obj) return types.MethodType(wrapped, obj)
def detect_model_type(model): def detect_model_type(model):
"""Return (arch, patch_size, temporal_stride) for latent geometry. """Return (arch, patch_size, temporal_stride) for latent geometry.
temporal_stride is the VAE's pixel→latent temporal compression factor, temporal_stride is the VAE's pixel→latent temporal compression factor,
used to convert user-facing pixel frame counts to latent frames. used to convert user-facing pixel frame counts to latent frames.
""" """
diff_model = model.model.diffusion_model diff_model = model.model.diffusion_model
if hasattr(diff_model, "patch_size") and not hasattr(diff_model, "patchifier"): if hasattr(diff_model, "patch_size") and not hasattr(diff_model, "patchifier"):
return "wan", tuple(diff_model.patch_size), 4 return "wan", tuple(diff_model.patch_size), 4
if hasattr(diff_model, "patchifier"): if hasattr(diff_model, "patchifier"):
return "ltx", (1, 1, 1), int(diff_model.vae_scale_factors[0]) return "ltx", (1, 1, 1), int(diff_model.vae_scale_factors[0])
raise ValueError( raise ValueError(
f"Unsupported model type: {type(diff_model).__name__}. " f"Unsupported model type: {type(diff_model).__name__}. "
f"Currently supports Wan and LTX models." f"Currently supports Wan and LTX models."
) )
def _check_unpatched(model_clone, key): def _check_unpatched(model_clone, key):
if key in getattr(model_clone, "object_patches", {}): if key in getattr(model_clone, "object_patches", {}):
raise RuntimeError( raise RuntimeError(
f"PromptRelay: cross-attention forward at '{key}' is already patched by " f"PromptRelay: cross-attention forward at '{key}' is already patched by "
"another node (e.g. KJNodes NAG). Stacking is not supported — remove the " "another node (e.g. KJNodes NAG). Stacking is not supported — remove the "
"conflicting node." "conflicting node."
) )
def apply_patches(model_clone, arch, mask_fn): def apply_patches(model_clone, arch, mask_fn):
diffusion_model = model_clone.get_model_object("diffusion_model") diffusion_model = model_clone.get_model_object("diffusion_model")
if arch == "wan": if arch == "wan":
from comfy.ldm.wan.model import WanI2VCrossAttention from comfy.ldm.wan.model import WanI2VCrossAttention
for idx, block in enumerate(diffusion_model.blocks): for idx, block in enumerate(diffusion_model.blocks):
key = f"diffusion_model.blocks.{idx}.cross_attn.forward" key = f"diffusion_model.blocks.{idx}.cross_attn.forward"
_check_unpatched(model_clone, key) _check_unpatched(model_clone, key)
cross_attn = block.cross_attn cross_attn = block.cross_attn
impl = _wan_i2v_forward if isinstance(cross_attn, WanI2VCrossAttention) else _wan_t2v_forward impl = _wan_i2v_forward if isinstance(cross_attn, WanI2VCrossAttention) else _wan_t2v_forward
model_clone.add_object_patch(key, _CrossAttnPatch(impl, mask_fn).__get__(cross_attn, cross_attn.__class__)) model_clone.add_object_patch(key, _CrossAttnPatch(impl, mask_fn).__get__(cross_attn, cross_attn.__class__))
return return
if arch == "ltx": if arch == "ltx":
for idx, block in enumerate(diffusion_model.transformer_blocks): for idx, block in enumerate(diffusion_model.transformer_blocks):
for attr in ("attn2", "audio_attn2"): for attr in ("attn2", "audio_attn2"):
module = getattr(block, attr, None) module = getattr(block, attr, None)
if module is None: if module is None:
continue continue
key = f"diffusion_model.transformer_blocks.{idx}.{attr}.forward" key = f"diffusion_model.transformer_blocks.{idx}.{attr}.forward"
_check_unpatched(model_clone, key) _check_unpatched(model_clone, key)
model_clone.add_object_patch(key, _CrossAttnPatch(_ltx_forward, mask_fn).__get__(module, module.__class__)) model_clone.add_object_patch(key, _CrossAttnPatch(_ltx_forward, mask_fn).__get__(module, module.__class__))
return return
raise ValueError(f"Unknown model arch: {arch}") raise ValueError(f"Unknown model arch: {arch}")

402
prompt_relay.py
View File

@@ -1,201 +1,201 @@
import logging import logging
import math import math
import torch import torch
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
def build_temporal_cost(q_token_idx, Lq, Lk, device, dtype, tokens_per_frame): def build_temporal_cost(q_token_idx, Lq, Lk, device, dtype, tokens_per_frame):
"""Gaussian penalty matrix [Lq, Lk] for video cross-attention (integer frame indexing).""" """Gaussian penalty matrix [Lq, Lk] for video cross-attention (integer frame indexing)."""
offset = torch.zeros(Lq, Lk, device=device, dtype=dtype) offset = torch.zeros(Lq, Lk, device=device, dtype=dtype)
query_frames = torch.arange(Lq, device=device, dtype=torch.long) // tokens_per_frame query_frames = torch.arange(Lq, device=device, dtype=torch.long) // tokens_per_frame
for seg in q_token_idx: for seg in q_token_idx:
local = seg["local_token_idx"].to(device=device) local = seg["local_token_idx"].to(device=device)
d = (query_frames.float()[:, None] - seg["midpoint"]).abs() d = (query_frames.float()[:, None] - seg["midpoint"]).abs()
strength = seg.get("strength", 1.0) strength = seg.get("strength", 1.0)
cost = strength * (torch.relu(d - seg["window"]) ** 2) / (2 * seg["sigma"] ** 2) cost = strength * (torch.relu(d - seg["window"]) ** 2) / (2 * seg["sigma"] ** 2)
offset[:, local] = cost.to(offset.dtype) offset[:, local] = cost.to(offset.dtype)
return offset return offset
def build_temporal_cost_scaled(q_token_idx, Lq, Lk, device, dtype, latent_frames): def build_temporal_cost_scaled(q_token_idx, Lq, Lk, device, dtype, latent_frames):
"""Penalty matrix for queries that don't map to integer frames (e.g. LTXAV audio tokens).""" """Penalty matrix for queries that don't map to integer frames (e.g. LTXAV audio tokens)."""
offset = torch.zeros(Lq, Lk, device=device, dtype=dtype) offset = torch.zeros(Lq, Lk, device=device, dtype=dtype)
query_frames = torch.arange(Lq, device=device, dtype=torch.float32) * latent_frames / Lq query_frames = torch.arange(Lq, device=device, dtype=torch.float32) * latent_frames / Lq
for seg in q_token_idx: for seg in q_token_idx:
local = seg["local_token_idx"].to(device=device) local = seg["local_token_idx"].to(device=device)
d = (query_frames[:, None] - seg["midpoint"]).abs() d = (query_frames[:, None] - seg["midpoint"]).abs()
sigma_a = seg.get("sigma_audio", seg["sigma"]) sigma_a = seg.get("sigma_audio", seg["sigma"])
window_a = seg.get("window_audio", seg["window"]) window_a = seg.get("window_audio", seg["window"])
strength_a = seg.get("strength_audio", 1.0) strength_a = seg.get("strength_audio", 1.0)
cost = strength_a * (torch.relu(d - window_a) ** 2) / (2 * sigma_a ** 2) cost = strength_a * (torch.relu(d - window_a) ** 2) / (2 * sigma_a ** 2)
offset[:, local] = cost.to(offset.dtype) offset[:, local] = cost.to(offset.dtype)
return offset return offset
def create_mask_fn(q_token_idx, fallback_tokens_per_frame, latent_frames): def create_mask_fn(q_token_idx, fallback_tokens_per_frame, latent_frames):
"""Closure: mask_fn(q, k, transformer_options) -> additive mask or None.""" """Closure: mask_fn(q, k, transformer_options) -> additive mask or None."""
cache = {} cache = {}
max_token_idx = max(int(seg["local_token_idx"].max().item()) for seg in q_token_idx) + 1 max_token_idx = max(int(seg["local_token_idx"].max().item()) for seg in q_token_idx) + 1
def mask_fn(q, k, transformer_options): def mask_fn(q, k, transformer_options):
Lq, Lk = q.shape[1], k.shape[1] Lq, Lk = q.shape[1], k.shape[1]
if Lq == Lk: if Lq == Lk:
return None return None
# Only apply on conditional pass — not unconditional (negative prompt) # Only apply on conditional pass — not unconditional (negative prompt)
cond_or_uncond = transformer_options.get("cond_or_uncond", []) cond_or_uncond = transformer_options.get("cond_or_uncond", [])
if 1 in cond_or_uncond and 0 not in cond_or_uncond: if 1 in cond_or_uncond and 0 not in cond_or_uncond:
return None return None
grid_sizes = transformer_options.get("grid_sizes", None) grid_sizes = transformer_options.get("grid_sizes", None)
video_tpf = int(grid_sizes[1]) * int(grid_sizes[2]) if grid_sizes is not None else fallback_tokens_per_frame video_tpf = int(grid_sizes[1]) * int(grid_sizes[2]) if grid_sizes is not None else fallback_tokens_per_frame
video_lq = latent_frames * video_tpf video_lq = latent_frames * video_tpf
# Skip cross-modal attention — text keys are padded to a fixed length ≥ max_token_idx and != video_lq # Skip cross-modal attention — text keys are padded to a fixed length ≥ max_token_idx and != video_lq
if Lk == video_lq or Lk < max_token_idx: if Lk == video_lq or Lk < max_token_idx:
return None return None
mode = "video" if Lq == video_lq else "scaled" mode = "video" if Lq == video_lq else "scaled"
key = (Lq, Lk, mode, q.device) key = (Lq, Lk, mode, q.device)
if key not in cache: if key not in cache:
if mode == "video": if mode == "video":
cost = build_temporal_cost(q_token_idx, Lq, Lk, q.device, q.dtype, video_tpf) cost = build_temporal_cost(q_token_idx, Lq, Lk, q.device, q.dtype, video_tpf)
else: else:
cost = build_temporal_cost_scaled(q_token_idx, Lq, Lk, q.device, q.dtype, latent_frames) cost = build_temporal_cost_scaled(q_token_idx, Lq, Lk, q.device, q.dtype, latent_frames)
log.info( log.info(
"[PromptRelay] Built penalty matrix (%s): Lq=%d, Lk=%d, nonzero=%d/%d", "[PromptRelay] Built penalty matrix (%s): Lq=%d, Lk=%d, nonzero=%d/%d",
mode, Lq, Lk, (cost > 0).sum().item(), cost.numel(), mode, Lq, Lk, (cost > 0).sum().item(), cost.numel(),
) )
cache[key] = -cost cache[key] = -cost
return cache[key].to(q.dtype) return cache[key].to(q.dtype)
return mask_fn return mask_fn
def build_segments(token_ranges, segment_lengths, epsilon=1e-3, relay_options=None): def build_segments(token_ranges, segment_lengths, epsilon=1e-3, relay_options=None):
"""Per-segment metadata for the temporal penalty. """Per-segment metadata for the temporal penalty.
relay_options (optional dict) overrides per-stream knobs: relay_options (optional dict) overrides per-stream knobs:
video_strength, video_window_scale, video_strength, video_window_scale,
audio_epsilon, audio_strength, audio_window_scale audio_epsilon, audio_strength, audio_window_scale
Audio knobs only affect architectures whose cross-attention takes the scaled Audio knobs only affect architectures whose cross-attention takes the scaled
(non-integer-frame) path — currently LTX audio_attn2. (non-integer-frame) path — currently LTX audio_attn2.
""" """
# Paper uses constant sigma = 1/ln(1/epsilon) regardless of segment length # Paper uses constant sigma = 1/ln(1/epsilon) regardless of segment length
sigma = 1.0 / math.log(1.0 / epsilon) if 0 < epsilon < 1 else 0.1448 sigma = 1.0 / math.log(1.0 / epsilon) if 0 < epsilon < 1 else 0.1448
opts = relay_options or {} opts = relay_options or {}
v_strength = opts.get("video_strength", 1.0) v_strength = opts.get("video_strength", 1.0)
v_window_scale = opts.get("video_window_scale", 1.0) v_window_scale = opts.get("video_window_scale", 1.0)
a_epsilon = opts.get("audio_epsilon") a_epsilon = opts.get("audio_epsilon")
a_strength = opts.get("audio_strength", 1.0) a_strength = opts.get("audio_strength", 1.0)
a_window_scale = opts.get("audio_window_scale", 1.0) a_window_scale = opts.get("audio_window_scale", 1.0)
if a_epsilon is not None and 0 < a_epsilon < 1: if a_epsilon is not None and 0 < a_epsilon < 1:
sigma_audio = 1.0 / math.log(1.0 / a_epsilon) sigma_audio = 1.0 / math.log(1.0 / a_epsilon)
else: else:
sigma_audio = sigma sigma_audio = sigma
if relay_options: if relay_options:
log.info( log.info(
"[PromptRelay] Advanced options active — video: strength=%.3f window_scale=%.3f | " "[PromptRelay] Advanced options active — video: strength=%.3f window_scale=%.3f | "
"audio: epsilon=%s strength=%.3f window_scale=%.3f", "audio: epsilon=%s strength=%.3f window_scale=%.3f",
v_strength, v_window_scale, v_strength, v_window_scale,
f"{a_epsilon:.4f}" if a_epsilon is not None else "inherit", f"{a_epsilon:.4f}" if a_epsilon is not None else "inherit",
a_strength, a_window_scale, a_strength, a_window_scale,
) )
q_token_idx = [] q_token_idx = []
frame_cursor = 0 frame_cursor = 0
for (tok_start, tok_end), L in zip(token_ranges, segment_lengths): for (tok_start, tok_end), L in zip(token_ranges, segment_lengths):
if L <= 0: if L <= 0:
frame_cursor += L frame_cursor += L
continue continue
midpoint = (2 * frame_cursor + L) // 2 midpoint = (2 * frame_cursor + L) // 2
base_window = max(L // 2 - 2, 0) base_window = max(L // 2 - 2, 0)
q_token_idx.append({ q_token_idx.append({
"local_token_idx": torch.arange(tok_start, tok_end), "local_token_idx": torch.arange(tok_start, tok_end),
"midpoint": midpoint, "midpoint": midpoint,
"window": max(base_window * v_window_scale, 0.0), "window": max(base_window * v_window_scale, 0.0),
"sigma": sigma, "sigma": sigma,
"strength": v_strength, "strength": v_strength,
"window_audio": max(base_window * a_window_scale, 0.0), "window_audio": max(base_window * a_window_scale, 0.0),
"sigma_audio": sigma_audio, "sigma_audio": sigma_audio,
"strength_audio": a_strength, "strength_audio": a_strength,
}) })
frame_cursor += L frame_cursor += L
return q_token_idx return q_token_idx
def get_raw_tokenizer(clip): def get_raw_tokenizer(clip):
"""Extract the raw SPiece/HF tokenizer from a ComfyUI CLIP object.""" """Extract the raw SPiece/HF tokenizer from a ComfyUI CLIP object."""
tokenizer_wrapper = clip.tokenizer tokenizer_wrapper = clip.tokenizer
for attr_name in dir(tokenizer_wrapper): for attr_name in dir(tokenizer_wrapper):
if attr_name.startswith("_"): if attr_name.startswith("_"):
continue continue
inner = getattr(tokenizer_wrapper, attr_name, None) inner = getattr(tokenizer_wrapper, attr_name, None)
if inner is not None and hasattr(inner, "tokenizer"): if inner is not None and hasattr(inner, "tokenizer"):
return inner.tokenizer return inner.tokenizer
raise RuntimeError( raise RuntimeError(
f"Could not find raw tokenizer on CLIP object. " f"Could not find raw tokenizer on CLIP object. "
f"Known attributes: {[a for a in dir(tokenizer_wrapper) if not a.startswith('_')]}" f"Known attributes: {[a for a in dir(tokenizer_wrapper) if not a.startswith('_')]}"
) )
def map_token_indices(raw_tokenizer, global_prompt, local_prompts): def map_token_indices(raw_tokenizer, global_prompt, local_prompts):
"""Tokenize global + space-prefixed locals; return (full_prompt, per-local token ranges). """Tokenize global + space-prefixed locals; return (full_prompt, per-local token ranges).
Uses incremental tokenization to avoid SentencePiece context-dependency issues. Uses incremental tokenization to avoid SentencePiece context-dependency issues.
""" """
prefixed_locals = [" " + lp for lp in local_prompts] prefixed_locals = [" " + lp for lp in local_prompts]
full_prompt = global_prompt + "".join(prefixed_locals) full_prompt = global_prompt + "".join(prefixed_locals)
has_eos = getattr(raw_tokenizer, "add_eos", False) has_eos = getattr(raw_tokenizer, "add_eos", False)
eos_adj = 1 if has_eos else 0 eos_adj = 1 if has_eos else 0
prev_len = len(raw_tokenizer(global_prompt)["input_ids"]) - eos_adj prev_len = len(raw_tokenizer(global_prompt)["input_ids"]) - eos_adj
token_ranges = [] token_ranges = []
built = global_prompt built = global_prompt
for plp in prefixed_locals: for plp in prefixed_locals:
built += plp built += plp
cur_len = len(raw_tokenizer(built)["input_ids"]) - eos_adj cur_len = len(raw_tokenizer(built)["input_ids"]) - eos_adj
if cur_len <= prev_len: if cur_len <= prev_len:
raise ValueError(f"Local prompt produced no tokens: '{plp.strip()}'") raise ValueError(f"Local prompt produced no tokens: '{plp.strip()}'")
token_ranges.append((prev_len, cur_len)) token_ranges.append((prev_len, cur_len))
prev_len = cur_len prev_len = cur_len
return full_prompt, token_ranges return full_prompt, token_ranges
def distribute_segment_lengths(num_segments, latent_frames, specified_lengths=None): def distribute_segment_lengths(num_segments, latent_frames, specified_lengths=None):
"""Validate or auto-distribute segment frame counts, capped to fit within latent_frames.""" """Validate or auto-distribute segment frame counts, capped to fit within latent_frames."""
if specified_lengths: if specified_lengths:
if len(specified_lengths) != num_segments: if len(specified_lengths) != num_segments:
raise ValueError( raise ValueError(
f"Number of segment_lengths ({len(specified_lengths)}) " f"Number of segment_lengths ({len(specified_lengths)}) "
f"must match number of local prompts ({num_segments})" f"must match number of local prompts ({num_segments})"
) )
lengths = specified_lengths lengths = specified_lengths
else: else:
# ceil division — matches reference implementation # ceil division — matches reference implementation
step = -(-latent_frames // num_segments) step = -(-latent_frames // num_segments)
lengths = [step] * num_segments lengths = [step] * num_segments
effective = [] effective = []
cursor = 0 cursor = 0
for L in lengths: for L in lengths:
end = min(cursor + L, latent_frames) end = min(cursor + L, latent_frames)
effective.append(max(end - cursor, 0)) effective.append(max(end - cursor, 0))
cursor = end cursor = end
return effective return effective

72
pyproject.toml
View File

@@ -1,36 +1,36 @@
[project] [project]
name = "WhatDreamsCost-ComfyUI" name = "WhatDreamsCost-ComfyUI"
description = "A variety of custom ComfyUI nodes and workflows for creatives." description = "A variety of custom ComfyUI nodes and workflows for creatives."
version = "1.3.9" version = "1.3.9"
license = {file = "LICENSE"} license = {file = "LICENSE"}
# classifiers = [ # classifiers = [
# # For OS-independent nodes (works on all operating systems) # # For OS-independent nodes (works on all operating systems)
# "Operating System :: OS Independent", # "Operating System :: OS Independent",
# #
# # OR for OS-specific nodes, specify the supported systems: # # OR for OS-specific nodes, specify the supported systems:
# "Operating System :: Microsoft :: Windows", # Windows specific # "Operating System :: Microsoft :: Windows", # Windows specific
# "Operating System :: POSIX :: Linux", # Linux specific # "Operating System :: POSIX :: Linux", # Linux specific
# "Operating System :: MacOS", # macOS specific # "Operating System :: MacOS", # macOS specific
# #
# # GPU Accelerator support. Pick the ones that are supported by your extension. # # GPU Accelerator support. Pick the ones that are supported by your extension.
# "Environment :: GPU :: NVIDIA CUDA", # NVIDIA CUDA support # "Environment :: GPU :: NVIDIA CUDA", # NVIDIA CUDA support
# "Environment :: GPU :: AMD ROCm", # AMD ROCm support # "Environment :: GPU :: AMD ROCm", # AMD ROCm support
# "Environment :: GPU :: Intel Arc", # Intel Arc support # "Environment :: GPU :: Intel Arc", # Intel Arc support
# "Environment :: NPU :: Huawei Ascend", # Huawei Ascend support # "Environment :: NPU :: Huawei Ascend", # Huawei Ascend support
# "Environment :: GPU :: Apple Metal", # Apple Metal support # "Environment :: GPU :: Apple Metal", # Apple Metal support
# ] # ]
[project.urls] [project.urls]
Repository = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI" Repository = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI"
# Used by Comfy Registry https://registry.comfy.org # Used by Comfy Registry https://registry.comfy.org
Documentation = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/wiki" Documentation = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/wiki"
"Bug Tracker" = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/issues" "Bug Tracker" = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/issues"
[tool.comfy] [tool.comfy]
PublisherId = "whatdreamscost" PublisherId = "whatdreamscost"
DisplayName = "WhatDreamsCost-ComfyUI" DisplayName = "WhatDreamsCost-ComfyUI"
Icon = "https://raw.githubusercontent.com/WhatDreamsCost/MediaSyncer/refs/heads/main/Icon.png" Icon = "https://raw.githubusercontent.com/WhatDreamsCost/MediaSyncer/refs/heads/main/Icon.png"
includes = [] includes = []
# "requires-comfyui" = ">=1.0.0" # ComfyUI version compatibility # "requires-comfyui" = ">=1.0.0" # ComfyUI version compatibility

100
speech_length_calculator.py
View File

@@ -1,51 +1,51 @@
import re import re
import math import math
class SpeechLengthCalculator: class SpeechLengthCalculator:
@classmethod @classmethod
def INPUT_TYPES(s): def INPUT_TYPES(s):
return { return {
"required": { "required": {
"text": ("STRING", {"multiline": True, "default": 'Enter your script here. "Make sure to put spoken words inside quotes!"'}), "text": ("STRING", {"multiline": True, "default": 'Enter your script here. "Make sure to put spoken words inside quotes!"'}),
"fps": ("INT", {"default": 24, "min": 1, "max": 120, "step": 1}), "fps": ("INT", {"default": 24, "min": 1, "max": 120, "step": 1}),
"additional_time": ("FLOAT", {"default": 0.0, "min": 0.0, "step": 0.1}), "additional_time": ("FLOAT", {"default": 0.0, "min": 0.0, "step": 0.1}),
}, },
"optional": { "optional": {
"text_input": ("STRING", {"forceInput": True}), "text_input": ("STRING", {"forceInput": True}),
} }
} }
# Added "STRING" to RETURN_TYPES # Added "STRING" to RETURN_TYPES
RETURN_TYPES = ("INT", "INT", "INT", "STRING") RETURN_TYPES = ("INT", "INT", "INT", "STRING")
# Added "text" to RETURN_NAMES # Added "text" to RETURN_NAMES
RETURN_NAMES = ("slow_frame_count", "average_frame_count", "fast_frame_count", "text") RETURN_NAMES = ("slow_frame_count", "average_frame_count", "fast_frame_count", "text")
FUNCTION = "calculate_speech" FUNCTION = "calculate_speech"
CATEGORY = "WhatDreamsCost" CATEGORY = "WhatDreamsCost"
def calculate_speech(self, text, fps, additional_time=0.0, text_input=None): def calculate_speech(self, text, fps, additional_time=0.0, text_input=None):
# Prioritize the connected text_input if provided, otherwise fallback to the text widget # Prioritize the connected text_input if provided, otherwise fallback to the text widget
active_text = text_input if (text_input is not None and isinstance(text_input, str) and text_input.strip() != "") else text active_text = text_input if (text_input is not None and isinstance(text_input, str) and text_input.strip() != "") else text
# Regex to find words inside double quotes, single quotes, or smart quotes # Regex to find words inside double quotes, single quotes, or smart quotes
matches = re.findall(r'"([^"]*)"|\'([^\']*)\'|“([^”]*)”|‘([^’]*)’', active_text) matches = re.findall(r'"([^"]*)"|\'([^\']*)\'|“([^”]*)”|‘([^’]*)’', active_text)
# Extract matches, handling all possible captured groups from the regex # Extract matches, handling all possible captured groups from the regex
quoted_text = " ".join([next((g for g in m if g), "") for m in matches]) quoted_text = " ".join([next((g for g in m if g), "") for m in matches])
# Split by whitespace to get words and count them # Split by whitespace to get words and count them
words = quoted_text.split() words = quoted_text.split()
word_count = len(words) word_count = len(words)
def calc_frames(wpm): def calc_frames(wpm):
if word_count == 0 and additional_time == 0: if word_count == 0 and additional_time == 0:
return 0 return 0
minutes = word_count / wpm minutes = word_count / wpm
seconds = (minutes * 60) + additional_time seconds = (minutes * 60) + additional_time
return math.ceil(seconds * fps) return math.ceil(seconds * fps)
slow_frames = calc_frames(100) slow_frames = calc_frames(100)
avg_frames = calc_frames(130) avg_frames = calc_frames(130)
fast_frames = calc_frames(160) fast_frames = calc_frames(160)
# Added active_text as the 4th returned value # Added active_text as the 4th returned value
return (slow_frames, avg_frames, fast_frames, active_text) return (slow_frames, avg_frames, fast_frames, active_text)