Compare commits
17 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
38207e93e5 | ||
|
|
8bca327a07 | ||
|
|
23294fa8df | ||
|
|
f0cc7aee26 | ||
|
|
ba9e2f8d78 | ||
|
|
1b159d81f9 | ||
|
|
e6040e1bad | ||
|
|
4d98a77639 | ||
|
|
c1c4557096 | ||
|
|
df0a760c70 | ||
|
|
1cce96f64f | ||
|
|
b31d511a27 | ||
|
|
32a8bfa511 | ||
|
|
0604d7fb0e | ||
|
|
0f90ddd5da | ||
|
|
04b51c81f2 | ||
|
|
9f2807826e |
502
README.md
502
README.md
@@ -1,242 +1,260 @@
|
||||
# Overview
|
||||
|
||||
This will be a collection of free resources for ComfyUI.
|
||||
|
||||
Hopefully it will make creating cool stuff easier.
|
||||
|
||||
All of my nodes are created with the help of AI, so there may or may not be redundant, messy code.
|
||||
|
||||
## ▶️ YouTube Tutorial Videos
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td>
|
||||
<p align="center">LTX Director Trailer</p>
|
||||
<a href="https://www.youtube.com/watch?v=fZgtkRcu4_k">
|
||||
<img src="https://img.youtube.com/vi/fZgtkRcu4_k/0.jpg" alt="LTX Director Trailer" width="400">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<p align="center">LTX Director Tutorial</p>
|
||||
<a href="https://www.youtube.com/watch?v=vM60pJJqqEI">
|
||||
<img src="https://img.youtube.com/vi/vM60pJJqqEI/0.jpg" alt="LTX Director Tutorial" width="400">
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## ❓ How to install nodes
|
||||
|
||||
- Navigate to your `/ComfyUI/custom_nodes/ folder`
|
||||
- Run `git clone https://github.com/WhatDreamscost/WhatDreamsCost-ComfyUI`
|
||||
- Or download through the ComfyUI Manager.
|
||||
|
||||
**❗❗IMPORTANT❗❗**
|
||||
|
||||
If you don't see the latest version (v1.3.9) yet in the manager then just downloaded the nightly version (or fetch the updates to update the list to see the latest version).
|
||||
Also you will need to update ComfyUI-LTXVideo and ComfyUI-KJNodes to the latest version as well. You cannot use this node without updating ComfyUI-LTXVideo!
|
||||
|
||||
# 🔄 Recent Updates
|
||||
**v1.3.9**
|
||||
* **Fixed recent updates not showing in the manager**
|
||||
|
||||
It took like 5 tries but I finally got it working 🤦♂️
|
||||
|
||||
**v1.3.3**
|
||||
* **LTX Director Hotfix 2**
|
||||
- Fixed duration_seconds input issue.
|
||||
- Made both duration widgets visible at all times now
|
||||
- Implemented audio latent fix to improve compatibility
|
||||
|
||||
|
||||
**v1.3.2**
|
||||
* **LTX Director Hotfix**
|
||||
- Fixed epsilon input overlapping custom_width input
|
||||
- Fixed invisible widgets in nodes 2.0 when toggling widget visibility through settings menu
|
||||
|
||||
If anyone finds anymore bugs or has idea for improvements please let me know!
|
||||
|
||||
|
||||
**v1.3.1**
|
||||
* **LTX Director Example Workflow Fix**
|
||||
- Minor fix to the example workflow (i forgot to set the clip loader type to ltxv lol)
|
||||
|
||||
**v1.3.0**
|
||||
* **New nodes: LTX Director and LTX Director Guide**
|
||||
- A complete timeline editor that can do almost everything. It's my most ambitious node so far and the successor to LTX Sequencer/Multi Image Loader.
|
||||
|
||||
**v1.2.9**
|
||||
* **Fixed every known issue with Multi Image Loader and added text output to Speech Length Calculator**
|
||||
|
||||
- Removed the completely useless drag and drop animations (now it's snappy and no longer finicky)
|
||||
- Fixed the node resizing on nodes 2.0
|
||||
- Updated grid logic to fit images better
|
||||
- Added ablity to right click images to copy/open/save images
|
||||
- Fixed the "invisible hitbox" underneath node issue (actually this time).
|
||||
|
||||
Also added a text output to the Speech Length Calculator node (can't believe i didn't do this initially)
|
||||
|
||||
<details>
|
||||
<summary>Click to view older Updates</summary>
|
||||
|
||||
**v1.2.8**
|
||||
* **Updated Load Video UI and Color Conversion**
|
||||
* Added crop mode, a simple interface to crop videos. It also include various aspect ratio presets.
|
||||
* Updated color conversion to ensure colors are as accurate as possible. Will first check metadata for colorspace, and if metadata is missing then it will guess the colorspace based on video dimensions.
|
||||
* Updated display mode toggle UI to be more understandable
|
||||
|
||||
**v1.2.7**
|
||||
* **New Node: Load Video UI**
|
||||
|
||||
Custom Node to Trim, Resize, and Preview Videos in Realtime
|
||||
|
||||
**v1.2.6**
|
||||
* **Updated Speech Length Calculator UI**
|
||||
|
||||
Also added duration output to the Load Audio UI node
|
||||
|
||||
**v1.2.5**
|
||||
* **Updated Load Audio UI Node**
|
||||
* Added Duration Setting
|
||||
* Made the whole selection bar draggable
|
||||
* Fixed Trimmed UI to show centiseconds
|
||||
|
||||
**v1.2.4**
|
||||
* **New Node: Load Audio UI**
|
||||
|
||||
Overhaul of the load audio node. Features a simple interface to easily trim audio. Also allows dragging and dropping files (fixes the original node that doesn't allow dropping in videos). Also compatible with nodes 2.0.
|
||||
|
||||
**v1.2.3**
|
||||
* **Workflow Update + Minor Bug Fix**
|
||||
* Added new workflow that is compatible with the latest ComfyUI version (as of 4/27/26). The new workflow also included an option to include custom audio, and has minor improvements of the previous workflows.
|
||||
* Fixed minor bug with Multi Image Loader that blocked mouse input in a small area under the node 🤷♂️
|
||||
|
||||
**v1.2.0**
|
||||
* **New Node: Speech Length Calculator**
|
||||
|
||||
Automatically output in realtime how long a video should be based on the dialouge.
|
||||
|
||||
**v1.1.0**
|
||||
* Added resize_method to the Multi Image Loader node for more resize options
|
||||
* Added insert_mode which allows you to enter in seconds instead of frames on the LTX Sequencer node
|
||||
* Updated workflows with more notes
|
||||
* Re-added tiny vae to workflows
|
||||
* Fixed various bugs
|
||||
* more things i can't rememeber
|
||||
|
||||
**This update will change the node layouts, so be sure to update your workflows or else they won't work properly.**
|
||||
|
||||
❗❗❗ **New Tutorial on using these nodes available: https://www.youtube.com/watch?v=aXDIr8eNovI** ❗❗❗
|
||||
</details>
|
||||
|
||||
# ⚙️ Custom Nodes
|
||||
|
||||
|
||||
## LTX Director
|
||||
<img width="1481" height="833" alt="Clipboard Image (2)" src="https://github.com/user-attachments/assets/08f3fe53-9393-4f5d-9de5-58b229fbed47" />
|
||||
|
||||
A Complete Timeline Editor For LTX 2.3. This is the sucessor of my previous nodes, and has loads of features in it. It was originally based off of [Kijai's Prompt Relay node](https://github.com/kijai/ComfyUI-PromptRelay) and my LTX Sequencer/Multi Image Loader nodes.
|
||||
|
||||
**Main Features:**
|
||||
- **Fully Functional Timeline Editor:** I spent hours studying various video editors and ended up with this design. If anyone has ideas for improvements let me know! I will adding documentation on all the functions soon.
|
||||
- **Prompt Relay integrated:** This unlocks the ability to have granular control over video generation. For more information on Prompt Relay go here, https://gordonchen19.github.io/Prompt-Relay/
|
||||
- **First, Middle, Last Frame Support:** This has by far the easiest method of creating first/last frames videos. It supports any number of keyframes, and will be the successor of my previous nodes.
|
||||
- **Custom Audio Support:** Import, trim, and combine your own audio clips in this node. Enabling custom audio is as simple as clicking 1 button. It is also compatible with every other feature in the node, include first/last frames, t2v, i2v, and prompt relay.
|
||||
- **Image to Video:** Part of the goal of this node was to make it easier to do everything, including Image to Video. It has built in resize functionality, and of course all the benifits of the prompt relay and custom audio integration.
|
||||
- **Text to Video:** Use text segments to create T2V videos. Compatible with all other features of the node.
|
||||
|
||||
Download workflows here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/tree/main/example_workflows
|
||||
|
||||
**Tutorial videos and documentation coming soon**
|
||||
|
||||
|
||||
## Multi Image Loader
|
||||
<img width="1280" height="720" alt="Multi_Image_Loader_Wide_Gif" src="https://github.com/user-attachments/assets/99b6afd8-5197-4e6c-81da-a7bd156c42c7" />
|
||||
|
||||
An Image loader that features a built in gallery, allowing your to easily rearrange images and output them seperately or batched together. It also combines the image resize node and LTXVPreprocess node to reduce clutter in LTX workflows.
|
||||
|
||||
## LTX Sequencer
|
||||
|
||||
|
||||
An overhaul of the LTXVAddGuideMulti node. It allows you to quickly create FFLF (First Frame Last Frame) videos, shot sequences, supports any number of middle frames.
|
||||
|
||||
Connect the Multi Image Loader node's multi_output to automatically update the node's widgets.
|
||||
|
||||
It also has a sync feature that syncs all LTX Sequencer nodes together in realtime, removing the need to edit every single node manually every time you want to make a change to something.
|
||||
|
||||
|
||||
## LTX Keyframer
|
||||
<img width="1082" height="608" alt="LTX Keyframer Wide" src="https://github.com/user-attachments/assets/850ba4a2-dbca-4e5a-a580-1c271e9f0c41" />
|
||||
|
||||
An overhaul of the LTXVImgToVideoInplaceKJ node. It allows you to quickly create FFLF (First Frame Last Frame) videos and shot sequences. Also upports any number of middle frames.
|
||||
|
||||
Connect the Multi Image Loader node's multi_output to automatically update the node's widgets.
|
||||
|
||||
It also has a sync feature that syncs all LTX Keyframer nodes together in realtime, removing the need to edit every single node manually every time you want to make a change to something.
|
||||
|
||||
**I would recommend using the LTX Sequencer Node over this node, after further testing it seems superior in at pretty much everything. I'll leave it in just in case more people want to test it**
|
||||
|
||||
## Speech Length Calculator
|
||||
<img width="1280" height="720" alt="Speech Length Calculator v2 Gif" src="https://github.com/user-attachments/assets/04b9a1cf-20e4-4b7b-a9c6-4a5a0825995b" />
|
||||
<br>
|
||||
<br>
|
||||
This node calculates in realtime how long a video should be based on the dialogue. Any words in quotations will be considered as speech. The node updates in realtime without having to run the workflow, and outputs the length depending on how fast the speech is.
|
||||
|
||||
If you connect another string/text node to the text_input, it will still update in the length in realtime.
|
||||
|
||||
I kept having to play the guessing game on my own generations so I made this node to make it easier :man_shrugging:
|
||||
|
||||
## Load Video UI
|
||||
<table width="100%">
|
||||
<tr>
|
||||
<td width="50%" align="center">
|
||||
<p>Simple Controls</p>
|
||||
<img src="https://github.com/user-attachments/assets/fb76ff03-a6ff-4837-bd63-7e429f5f3d37" width="100%" />
|
||||
</td>
|
||||
<td width="50%" align="center">
|
||||
<p>New Crop Mode!</p>
|
||||
<img src="https://github.com/user-attachments/assets/28cfb4ca-e42a-44da-9afb-f20cb01b9722" width="100%" />
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<br>
|
||||
<br>
|
||||
An upgraded Load Video node. It has the following features:
|
||||
|
||||
* Simple interface to quickly trim videos and preview them in realtime.
|
||||
* Ability to load any length of video into the node (the default load video node was limited to 100MB files)
|
||||
* Easily switch between showing seconds and frames with a toggle button. This will change the widgets as well as the interface.
|
||||
* Multiple options for resizing the video (maintain aspect ratio, crop, stretch to fit, pad)
|
||||
* Allows dragging and dropping files into the node
|
||||
* Progress bar
|
||||
* Optimized to use less RAM (still very limited due to ComfyUI limitations, but at least a little more efficient)
|
||||
|
||||
Please note that due to ComfyUI limitations (and the fact that this node doesn't use any addtional libraries), this node will not work well for outputting large videos. You can trim any length of video without a problem, but if the output is still large it will end up using a lot of RAM. I have implemented various optimizations though to make it use less memory.
|
||||
|
||||
## Load Audio UI
|
||||
<img width="1280" height="720" alt="Load_Audio_UI_V2" src="https://github.com/user-attachments/assets/e3dc5c8d-d0b9-4336-8196-944204719239" />
|
||||
<br>
|
||||
<br>
|
||||
An upgraded Load Audio node. Features a simple interface to easily trim audio. Also allows dragging and dropping files (fixes the original node that doesn't allow dropping in videos). Also compatible with nodes 2.0.
|
||||
|
||||
# 💡 Workflows
|
||||
<img width="3120" height="990" alt="LTX I2V First Last Frame 3 Stage Workflow v6" src="https://github.com/user-attachments/assets/c993ef2f-ac4b-4091-a7f6-5ff1674c3718" />
|
||||
<br>
|
||||
<br>
|
||||
This is a compact LTX 2.3 workflow for I2V and First Frame, Middle Frame, Last frame video generation.
|
||||
I seperated and organized everything into subraphs to make things as clean as possible, and added toggles to customize the workflow quickly.
|
||||
|
||||
Download workflows here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/tree/main/example_workflows
|
||||
|
||||
Or drag and drop the image into ComfyUI to import workflow.
|
||||
|
||||
# ❗ Known Issues
|
||||
|
||||
Fixed everything so far. If there are any other issue or bugs you find please let me know!
|
||||
|
||||
# 💡 Additional Info
|
||||
|
||||
I made these nodes knowing little about python and a beginner level understanding of javascript. Feel free to suggest improvements, and if you run into any bugs let me know.
|
||||
|
||||
For those asking, I mainly used gemini to create these nodes.
|
||||
# Overview
|
||||
|
||||
This will be a collection of free resources for ComfyUI.
|
||||
|
||||
Hopefully it will make creating cool stuff easier.
|
||||
|
||||
All of my nodes are created with the help of AI, so there may or may not be redundant, messy code.
|
||||
|
||||
## ▶️ YouTube Tutorial Videos
|
||||
|
||||
<a href="https://youtu.be/j28z5PZXkKk?si=TPeWKm2BW-uA3BAx">
|
||||
<img width="642" height="718" alt="Capture d'écran 2026-06-17 234136" src="https://github.com/user-attachments/assets/89433110-8b78-4971-8098-3a9e9226f460" />
|
||||
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td>
|
||||
<p align="center">LTX Director Trailer</p>
|
||||
<a href="https://www.youtube.com/watch?v=fZgtkRcu4_k">
|
||||
<img src="https://img.youtube.com/vi/fZgtkRcu4_k/0.jpg" alt="LTX Director Trailer" width="400">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<p align="center">LTX Director Tutorial</p>
|
||||
<a href="https://www.youtube.com/watch?v=vM60pJJqqEI">
|
||||
<img src="https://img.youtube.com/vi/vM60pJJqqEI/0.jpg" alt="LTX Director Tutorial" width="400">
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## ❓ How to install nodes
|
||||
|
||||
- Navigate to your `/ComfyUI/custom_nodes/ folder`
|
||||
- Delete your old "WhatDreamsCost-ComfyUI" Folder
|
||||
- Run `git clone -b main_cs https://github.com/CGlide/WhatDreamsCost-ComfyUI.git`
|
||||
|
||||
|
||||
**❗❗IMPORTANT❗❗**
|
||||
|
||||
|
||||
This is a Modded LTX director node with some extra options to help you create videos with references sheets
|
||||
|
||||
This node uses Ollama Locally : you will need to install it (very lightweight) and
|
||||
|
||||
- In your Ollama model folder run this command to install qwen 3.5 2b q4 (1.9gb) "ollama run huihui_ai/qwen3.5-abliterated:2B"
|
||||
|
||||
- The model won't eat memory while generating since there is an auto clear VRAM when you hit Run or after 5min.
|
||||
|
||||
- You can still enter your description manually if you don't want to install it but it works very well!
|
||||
|
||||
- Both references mode work (fixed), Licon MSR and Ghost Mask.
|
||||
|
||||
- Use their Lora, that is an important step : https://huggingface.co/LiconStudio/LTX-2.3-Multiple-Subject-Reference/tree/main
|
||||
|
||||
- Enjoy!
|
||||
|
||||
<img width="596" height="668" alt="Capture d'écran 2026-06-13 234036" src="https://github.com/user-attachments/assets/2dfe7992-a72a-4eda-a103-41b3a8f77714" />
|
||||
|
||||
|
||||
# 🔄 Recent Updates
|
||||
|
||||
**v1.3.3**
|
||||
* **LTX Director Hotfix 2**
|
||||
- Fixed duration_seconds input issue.
|
||||
- Made both duration widgets visible at all times now
|
||||
- Implemented audio latent fix to improve compatibility
|
||||
|
||||
|
||||
**v1.3.2**
|
||||
* **LTX Director Hotfix**
|
||||
- Fixed epsilon input overlapping custom_width input
|
||||
- Fixed invisible widgets in nodes 2.0 when toggling widget visibility through settings menu
|
||||
|
||||
If anyone finds anymore bugs or has idea for improvements please let me know!
|
||||
|
||||
|
||||
**v1.3.1**
|
||||
* **LTX Director Example Workflow Fix**
|
||||
- Minor fix to the example workflow (i forgot to set the clip loader type to ltxv lol)
|
||||
|
||||
**v1.3.0**
|
||||
* **New nodes: LTX Director and LTX Director Guide**
|
||||
- A complete timeline editor that can do almost everything. It's my most ambitious node so far and the successor to LTX Sequencer/Multi Image Loader.
|
||||
|
||||
**v1.2.9**
|
||||
* **Fixed every known issue with Multi Image Loader and added text output to Speech Length Calculator**
|
||||
|
||||
- Removed the completely useless drag and drop animations (now it's snappy and no longer finicky)
|
||||
- Fixed the node resizing on nodes 2.0
|
||||
- Updated grid logic to fit images better
|
||||
- Added ablity to right click images to copy/open/save images
|
||||
- Fixed the "invisible hitbox" underneath node issue (actually this time).
|
||||
|
||||
Also added a text output to the Speech Length Calculator node (can't believe i didn't do this initially)
|
||||
|
||||
<details>
|
||||
<summary>Click to view older Updates</summary>
|
||||
|
||||
**v1.2.8**
|
||||
* **Updated Load Video UI and Color Conversion**
|
||||
* Added crop mode, a simple interface to crop videos. It also include various aspect ratio presets.
|
||||
* Updated color conversion to ensure colors are as accurate as possible. Will first check metadata for colorspace, and if metadata is missing then it will guess the colorspace based on video dimensions.
|
||||
* Updated display mode toggle UI to be more understandable
|
||||
|
||||
**v1.2.7**
|
||||
* **New Node: Load Video UI**
|
||||
|
||||
Custom Node to Trim, Resize, and Preview Videos in Realtime
|
||||
|
||||
**v1.2.6**
|
||||
* **Updated Speech Length Calculator UI**
|
||||
|
||||
Also added duration output to the Load Audio UI node
|
||||
|
||||
**v1.2.5**
|
||||
* **Updated Load Audio UI Node**
|
||||
* Added Duration Setting
|
||||
* Made the whole selection bar draggable
|
||||
* Fixed Trimmed UI to show centiseconds
|
||||
|
||||
**v1.2.4**
|
||||
* **New Node: Load Audio UI**
|
||||
|
||||
Overhaul of the load audio node. Features a simple interface to easily trim audio. Also allows dragging and dropping files (fixes the original node that doesn't allow dropping in videos). Also compatible with nodes 2.0.
|
||||
|
||||
**v1.2.3**
|
||||
* **Workflow Update + Minor Bug Fix**
|
||||
* Added new workflow that is compatible with the latest ComfyUI version (as of 4/27/26). The new workflow also included an option to include custom audio, and has minor improvements of the previous workflows.
|
||||
* Fixed minor bug with Multi Image Loader that blocked mouse input in a small area under the node 🤷♂️
|
||||
|
||||
**v1.2.0**
|
||||
* **New Node: Speech Length Calculator**
|
||||
|
||||
Automatically output in realtime how long a video should be based on the dialouge.
|
||||
|
||||
**v1.1.0**
|
||||
* Added resize_method to the Multi Image Loader node for more resize options
|
||||
* Added insert_mode which allows you to enter in seconds instead of frames on the LTX Sequencer node
|
||||
* Updated workflows with more notes
|
||||
* Re-added tiny vae to workflows
|
||||
* Fixed various bugs
|
||||
* more things i can't rememeber
|
||||
|
||||
**This update will change the node layouts, so be sure to update your workflows or else they won't work properly.**
|
||||
|
||||
❗❗❗ **New Tutorial on using these nodes available: https://www.youtube.com/watch?v=aXDIr8eNovI** ❗❗❗
|
||||
</details>
|
||||
|
||||
# ⚙️ Custom Nodes
|
||||
|
||||
|
||||
## LTX Director
|
||||
<img width="1481" height="833" alt="Clipboard Image (2)" src="https://github.com/user-attachments/assets/08f3fe53-9393-4f5d-9de5-58b229fbed47" />
|
||||
|
||||
A Complete Timeline Editor For LTX 2.3. This is the sucessor of my previous nodes, and has loads of features in it. It was originally based off of [Kijai's Prompt Relay node](https://github.com/kijai/ComfyUI-PromptRelay) and my LTX Sequencer/Multi Image Loader nodes.
|
||||
|
||||
**Main Features:**
|
||||
- **Fully Functional Timeline Editor:** I spent hours studying various video editors and ended up with this design. If anyone has ideas for improvements let me know! I will adding documentation on all the functions soon.
|
||||
- **Prompt Relay integrated:** This unlocks the ability to have granular control over video generation. For more information on Prompt Relay go here, https://gordonchen19.github.io/Prompt-Relay/
|
||||
- **First, Middle, Last Frame Support:** This has by far the easiest method of creating first/last frames videos. It supports any number of keyframes, and will be the successor of my previous nodes.
|
||||
- **Custom Audio Support:** Import, trim, and combine your own audio clips in this node. Enabling custom audio is as simple as clicking 1 button. It is also compatible with every other feature in the node, include first/last frames, t2v, i2v, and prompt relay.
|
||||
- **Image to Video:** Part of the goal of this node was to make it easier to do everything, including Image to Video. It has built in resize functionality, and of course all the benifits of the prompt relay and custom audio integration.
|
||||
- **Text to Video:** Use text segments to create T2V videos. Compatible with all other features of the node.
|
||||
|
||||
Download workflows here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/tree/main/example_workflows
|
||||
|
||||
**Tutorial videos and documentation coming soon**
|
||||
|
||||
|
||||
## Multi Image Loader
|
||||
<img width="1280" height="720" alt="Multi_Image_Loader_Wide_Gif" src="https://github.com/user-attachments/assets/99b6afd8-5197-4e6c-81da-a7bd156c42c7" />
|
||||
|
||||
An Image loader that features a built in gallery, allowing your to easily rearrange images and output them seperately or batched together. It also combines the image resize node and LTXVPreprocess node to reduce clutter in LTX workflows.
|
||||
|
||||
## LTX Sequencer
|
||||
|
||||
|
||||
An overhaul of the LTXVAddGuideMulti node. It allows you to quickly create FFLF (First Frame Last Frame) videos, shot sequences, supports any number of middle frames.
|
||||
|
||||
Connect the Multi Image Loader node's multi_output to automatically update the node's widgets.
|
||||
|
||||
It also has a sync feature that syncs all LTX Sequencer nodes together in realtime, removing the need to edit every single node manually every time you want to make a change to something.
|
||||
|
||||
|
||||
## LTX Keyframer
|
||||
<img width="1082" height="608" alt="LTX Keyframer Wide" src="https://github.com/user-attachments/assets/850ba4a2-dbca-4e5a-a580-1c271e9f0c41" />
|
||||
|
||||
An overhaul of the LTXVImgToVideoInplaceKJ node. It allows you to quickly create FFLF (First Frame Last Frame) videos and shot sequences. Also upports any number of middle frames.
|
||||
|
||||
Connect the Multi Image Loader node's multi_output to automatically update the node's widgets.
|
||||
|
||||
It also has a sync feature that syncs all LTX Keyframer nodes together in realtime, removing the need to edit every single node manually every time you want to make a change to something.
|
||||
|
||||
**I would recommend using the LTX Sequencer Node over this node, after further testing it seems superior in at pretty much everything. I'll leave it in just in case more people want to test it**
|
||||
|
||||
## Speech Length Calculator
|
||||
<img width="1280" height="720" alt="Speech Length Calculator v2 Gif" src="https://github.com/user-attachments/assets/04b9a1cf-20e4-4b7b-a9c6-4a5a0825995b" />
|
||||
<br>
|
||||
<br>
|
||||
This node calculates in realtime how long a video should be based on the dialogue. Any words in quotations will be considered as speech. The node updates in realtime without having to run the workflow, and outputs the length depending on how fast the speech is.
|
||||
|
||||
If you connect another string/text node to the text_input, it will still update in the length in realtime.
|
||||
|
||||
I kept having to play the guessing game on my own generations so I made this node to make it easier :man_shrugging:
|
||||
|
||||
## Load Video UI
|
||||
<table width="100%">
|
||||
<tr>
|
||||
<td width="50%" align="center">
|
||||
<p>Simple Controls</p>
|
||||
<img src="https://github.com/user-attachments/assets/fb76ff03-a6ff-4837-bd63-7e429f5f3d37" width="100%" />
|
||||
</td>
|
||||
<td width="50%" align="center">
|
||||
<p>New Crop Mode!</p>
|
||||
<img src="https://github.com/user-attachments/assets/28cfb4ca-e42a-44da-9afb-f20cb01b9722" width="100%" />
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<br>
|
||||
<br>
|
||||
An upgraded Load Video node. It has the following features:
|
||||
|
||||
* Simple interface to quickly trim videos and preview them in realtime.
|
||||
* Ability to load any length of video into the node (the default load video node was limited to 100MB files)
|
||||
* Easily switch between showing seconds and frames with a toggle button. This will change the widgets as well as the interface.
|
||||
* Multiple options for resizing the video (maintain aspect ratio, crop, stretch to fit, pad)
|
||||
* Allows dragging and dropping files into the node
|
||||
* Progress bar
|
||||
* Optimized to use less RAM (still very limited due to ComfyUI limitations, but at least a little more efficient)
|
||||
|
||||
Please note that due to ComfyUI limitations (and the fact that this node doesn't use any addtional libraries), this node will not work well for outputting large videos. You can trim any length of video without a problem, but if the output is still large it will end up using a lot of RAM. I have implemented various optimizations though to make it use less memory.
|
||||
|
||||
## Load Audio UI
|
||||
<img width="1280" height="720" alt="Load_Audio_UI_V2" src="https://github.com/user-attachments/assets/e3dc5c8d-d0b9-4336-8196-944204719239" />
|
||||
<br>
|
||||
<br>
|
||||
An upgraded Load Audio node. Features a simple interface to easily trim audio. Also allows dragging and dropping files (fixes the original node that doesn't allow dropping in videos). Also compatible with nodes 2.0.
|
||||
|
||||
# 💡 Workflows
|
||||
<img width="3120" height="990" alt="LTX I2V First Last Frame 3 Stage Workflow v6" src="https://github.com/user-attachments/assets/c993ef2f-ac4b-4091-a7f6-5ff1674c3718" />
|
||||
<br>
|
||||
<br>
|
||||
This is a compact LTX 2.3 workflow for I2V and First Frame, Middle Frame, Last frame video generation.
|
||||
I seperated and organized everything into subraphs to make things as clean as possible, and added toggles to customize the workflow quickly.
|
||||
|
||||
Download workflows here: https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/tree/main/example_workflows
|
||||
|
||||
Or drag and drop the image into ComfyUI to import workflow.
|
||||
|
||||
# ❗ Known Issues
|
||||
|
||||
Fixed everything so far. If there are any other issue or bugs you find please let me know!
|
||||
|
||||
# 💡 Additional Info
|
||||
|
||||
I made these nodes knowing little about python and a beginner level understanding of javascript. Feel free to suggest improvements, and if you run into any bugs let me know.
|
||||
|
||||
For those asking, I mainly used gemini to create these nodes.
|
||||
|
||||
95
__init__.py
95
__init__.py
@@ -1,47 +1,50 @@
|
||||
from .ltx_keyframer import LTXKeyframer
|
||||
from .multi_image_loader import MultiImageLoader
|
||||
from .ltx_sequencer import LTXSequencer
|
||||
from .speech_length_calculator import SpeechLengthCalculator
|
||||
from .load_audio_ui import LoadAudioUI
|
||||
from .load_video_ui import LoadVideoUI
|
||||
from .ltx_director import LTXDirector
|
||||
from .ltx_director_guide import LTXDirectorGuide
|
||||
from comfy_api.latest import ComfyExtension, io
|
||||
from typing_extensions import override
|
||||
|
||||
class PromptRelay(ComfyExtension):
|
||||
@override
|
||||
async def get_node_list(self) -> list[type[io.ComfyNode]]:
|
||||
return [
|
||||
LTXDirector,
|
||||
LTXDirectorGuide
|
||||
]
|
||||
|
||||
async def comfy_entrypoint() -> PromptRelay:
|
||||
return PromptRelay()
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"LTXKeyframer": LTXKeyframer,
|
||||
"MultiImageLoader": MultiImageLoader,
|
||||
"LTXSequencer": LTXSequencer,
|
||||
"SpeechLengthCalculator": SpeechLengthCalculator,
|
||||
"LoadAudioUI": LoadAudioUI,
|
||||
"LoadVideoUI": LoadVideoUI,
|
||||
"LTXDirector": LTXDirector,
|
||||
"LTXDirectorGuide": LTXDirectorGuide,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"LTXKeyframer": "LTX Keyframer",
|
||||
"MultiImageLoader": "Multi Image Loader",
|
||||
"LTXSequencer": "LTX Sequencer",
|
||||
"SpeechLengthCalculator": "Speech Length Calculator",
|
||||
"LoadAudioUI": "Load Audio UI",
|
||||
"LoadVideoUI": "Load Video UI",
|
||||
"LTXDirector": "LTX Director",
|
||||
"LTXDirectorGuide": "LTX Director Guide",
|
||||
}
|
||||
|
||||
WEB_DIRECTORY = "./js"
|
||||
|
||||
from .ltx_keyframer import LTXKeyframer
|
||||
from .multi_image_loader import MultiImageLoader
|
||||
from .ltx_sequencer import LTXSequencer
|
||||
from .speech_length_calculator import SpeechLengthCalculator
|
||||
from .load_audio_ui import LoadAudioUI
|
||||
from .load_video_ui import LoadVideoUI
|
||||
from .ltx_director import LTXDirector
|
||||
from .ltx_director_guide import LTXDirectorGuide
|
||||
from comfy_api.latest import ComfyExtension, io
|
||||
from typing_extensions import override
|
||||
from .latent_slice import CleanLatentSlice
|
||||
|
||||
class PromptRelay(ComfyExtension):
|
||||
@override
|
||||
async def get_node_list(self) -> list[type[io.ComfyNode]]:
|
||||
return [
|
||||
LTXDirector,
|
||||
LTXDirectorGuide
|
||||
]
|
||||
|
||||
async def comfy_entrypoint() -> PromptRelay:
|
||||
return PromptRelay()
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"LTXKeyframer": LTXKeyframer,
|
||||
"MultiImageLoader": MultiImageLoader,
|
||||
"LTXSequencer": LTXSequencer,
|
||||
"SpeechLengthCalculator": SpeechLengthCalculator,
|
||||
"LoadAudioUI": LoadAudioUI,
|
||||
"LoadVideoUI": LoadVideoUI,
|
||||
"LTXDirector": LTXDirector,
|
||||
"LTXDirectorGuide": LTXDirectorGuide,
|
||||
"CleanLatentSlice": CleanLatentSlice,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"LTXKeyframer": "LTX Keyframer",
|
||||
"MultiImageLoader": "Multi Image Loader",
|
||||
"LTXSequencer": "LTX Sequencer",
|
||||
"SpeechLengthCalculator": "Speech Length Calculator",
|
||||
"LoadAudioUI": "Load Audio UI",
|
||||
"LoadVideoUI": "Load Video UI",
|
||||
"LTXDirector": "LTX Director",
|
||||
"LTXDirectorGuide": "LTX Director Guide",
|
||||
"CleanLatentSlice": "Clean Latent Slice",
|
||||
}
|
||||
|
||||
WEB_DIRECTORY = "./js"
|
||||
|
||||
__all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS', 'WEB_DIRECTORY']
|
||||
BIN
__pycache__/__init__.cpython-312.pyc
Normal file
BIN
__pycache__/__init__.cpython-312.pyc
Normal file
Binary file not shown.
BIN
__pycache__/latent_slice.cpython-312.pyc
Normal file
BIN
__pycache__/latent_slice.cpython-312.pyc
Normal file
Binary file not shown.
BIN
__pycache__/load_audio_ui.cpython-312.pyc
Normal file
BIN
__pycache__/load_audio_ui.cpython-312.pyc
Normal file
Binary file not shown.
BIN
__pycache__/load_video_ui.cpython-312.pyc
Normal file
BIN
__pycache__/load_video_ui.cpython-312.pyc
Normal file
Binary file not shown.
BIN
__pycache__/ltx_director.cpython-312.pyc
Normal file
BIN
__pycache__/ltx_director.cpython-312.pyc
Normal file
Binary file not shown.
BIN
__pycache__/ltx_director_guide.cpython-312.pyc
Normal file
BIN
__pycache__/ltx_director_guide.cpython-312.pyc
Normal file
Binary file not shown.
BIN
__pycache__/ltx_keyframer.cpython-312.pyc
Normal file
BIN
__pycache__/ltx_keyframer.cpython-312.pyc
Normal file
Binary file not shown.
BIN
__pycache__/ltx_sequencer.cpython-312.pyc
Normal file
BIN
__pycache__/ltx_sequencer.cpython-312.pyc
Normal file
Binary file not shown.
BIN
__pycache__/multi_image_loader.cpython-312.pyc
Normal file
BIN
__pycache__/multi_image_loader.cpython-312.pyc
Normal file
Binary file not shown.
BIN
__pycache__/patches.cpython-312.pyc
Normal file
BIN
__pycache__/patches.cpython-312.pyc
Normal file
Binary file not shown.
BIN
__pycache__/prompt_relay.cpython-312.pyc
Normal file
BIN
__pycache__/prompt_relay.cpython-312.pyc
Normal file
Binary file not shown.
BIN
__pycache__/speech_length_calculator.cpython-312.pyc
Normal file
BIN
__pycache__/speech_length_calculator.cpython-312.pyc
Normal file
Binary file not shown.
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
1
example_workflows/LTX Director MSR_Ghost_V5.json
Normal file
1
example_workflows/LTX Director MSR_Ghost_V5.json
Normal file
File diff suppressed because one or more lines are too long
1
example_workflows/LTX Director Workflow_cs.json
Normal file
1
example_workflows/LTX Director Workflow_cs.json
Normal file
File diff suppressed because one or more lines are too long
1
example_workflows/LTX Director Workflow_v2.json
Normal file
1
example_workflows/LTX Director Workflow_v2.json
Normal file
File diff suppressed because one or more lines are too long
1
example_workflows/LTX Director Workflow_v4.json
Normal file
1
example_workflows/LTX Director Workflow_v4.json
Normal file
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
1042
js/load_audio_ui.js
1042
js/load_audio_ui.js
File diff suppressed because it is too large
Load Diff
3026
js/load_video_ui.js
3026
js/load_video_ui.js
File diff suppressed because it is too large
Load Diff
7692
js/ltx_director.js
7692
js/ltx_director.js
File diff suppressed because it is too large
Load Diff
@@ -1,17 +1,17 @@
|
||||
import { app } from "../../scripts/app.js";
|
||||
|
||||
// LTX Director Guide is a pure pass-through processor node.
|
||||
// All configuration (images, insert frames, strengths) comes from
|
||||
// the guide_data output of Prompt Relay Encode (Timeline).
|
||||
// No dynamic widgets or sync logic needed.
|
||||
app.registerExtension({
|
||||
name: "Comfy.LTXDirectorGuide",
|
||||
async nodeCreated(node) {
|
||||
if (node.comfyClass !== "LTXDirectorGuide") return;
|
||||
// Nothing to initialize — the node has no configurable widgets.
|
||||
},
|
||||
});
|
||||
|
||||
// Nothing to initialize — the node has no configurable widgets.
|
||||
},
|
||||
import { app } from "../../scripts/app.js";
|
||||
|
||||
// LTX Director Guide is a pure pass-through processor node.
|
||||
// All configuration (images, insert frames, strengths) comes from
|
||||
// the guide_data output of Prompt Relay Encode (Timeline).
|
||||
// No dynamic widgets or sync logic needed.
|
||||
app.registerExtension({
|
||||
name: "Comfy.LTXDirectorGuide",
|
||||
async nodeCreated(node) {
|
||||
if (node.comfyClass !== "LTXDirectorGuide") return;
|
||||
// Nothing to initialize — the node has no configurable widgets.
|
||||
},
|
||||
});
|
||||
|
||||
// Nothing to initialize — the node has no configurable widgets.
|
||||
},
|
||||
});
|
||||
@@ -1,419 +1,419 @@
|
||||
import { app } from "../../scripts/app.js";
|
||||
|
||||
// Global registry to track all LTXKeyframer nodes across all subgraphs
|
||||
window._LTXKeyframerGlobalNodes = window._LTXKeyframerGlobalNodes || new Set();
|
||||
|
||||
// ComfyUI native trick to cleanly hide/show widgets without deleting them
|
||||
function toggleWidget(widget, visible) {
|
||||
if (visible) {
|
||||
if (widget.origType !== undefined) {
|
||||
widget.type = widget.origType;
|
||||
widget.computeSize = widget.origComputeSize;
|
||||
delete widget.origType;
|
||||
delete widget.origComputeSize;
|
||||
}
|
||||
} else {
|
||||
if (widget.type !== "hidden") {
|
||||
widget.origType = widget.type;
|
||||
widget.origComputeSize = widget.computeSize;
|
||||
widget.type = "hidden";
|
||||
widget.computeSize = () => [0, -4];
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --- NEW SYNC HELPER FUNCTION ---
|
||||
// Finds all other LTXKeyframer nodes globally and mirrors the value to them
|
||||
function syncWidgetAcrossNodes(sourceNode, widgetName, value) {
|
||||
if (!window._LTXKeyframerGlobalNodes) return;
|
||||
|
||||
for (const targetNode of window._LTXKeyframerGlobalNodes) {
|
||||
// Target all OTHER LTXKeyframer nodes by direct object reference
|
||||
if (targetNode !== sourceNode) {
|
||||
|
||||
// 1. Always update the hidden properties cache so it remembers the sync
|
||||
// even if the widget isn't currently visible (e.g. fewer images loaded right now)
|
||||
targetNode.properties[widgetName] = value;
|
||||
|
||||
// 2. If the widget is currently visible on the UI, update it visually
|
||||
if (targetNode.widgets) {
|
||||
const targetWidget = targetNode.widgets.find(w => w.name === widgetName);
|
||||
if (targetWidget && targetWidget.value !== value) {
|
||||
targetWidget.value = value;
|
||||
targetNode.setDirtyCanvas(true, false);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "Comfy.LTXKeyframer.DynamicInputs",
|
||||
async nodeCreated(node) {
|
||||
if (node.comfyClass !== "LTXKeyframer") return;
|
||||
|
||||
// Register this node instance globally
|
||||
window._LTXKeyframerGlobalNodes.add(node);
|
||||
|
||||
node._currentImageCount = -1; // Force first update
|
||||
|
||||
// Initialize persistent properties cache
|
||||
node.properties = node.properties || {};
|
||||
|
||||
// Add subtle separator line above images_loaded
|
||||
node.addCustomWidget({
|
||||
name: "num_images_separator",
|
||||
type: "text",
|
||||
draw(ctx, node, widget_width, y, widget_height) {
|
||||
ctx.save();
|
||||
ctx.strokeStyle = "#444";
|
||||
ctx.lineWidth = 1;
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(10, y + 5);
|
||||
ctx.lineTo(widget_width - 10, y + 5);
|
||||
ctx.stroke();
|
||||
ctx.restore();
|
||||
},
|
||||
computeSize(width) {
|
||||
return [width, 10];
|
||||
}
|
||||
});
|
||||
|
||||
// Move separator before num_images
|
||||
const moveSeparator = () => {
|
||||
const idx = node.widgets.findIndex(w => w.name === "num_images");
|
||||
const sepIdx = node.widgets.findIndex(w => w.name === "num_images_separator");
|
||||
if (idx !== -1 && sepIdx !== -1) {
|
||||
const separator = node.widgets.splice(sepIdx, 1)[0];
|
||||
node.widgets.splice(idx, 0, separator);
|
||||
}
|
||||
};
|
||||
setTimeout(moveSeparator, 50); // Small delay to ensure num_images is present
|
||||
|
||||
// Core update: synchronize widget visibility to match imageCount
|
||||
node._applyWidgetCount = function(count) {
|
||||
const isInitialLoad = this._currentImageCount === -1;
|
||||
|
||||
if (this._currentImageCount === count && !isInitialLoad) return;
|
||||
this._currentImageCount = count;
|
||||
|
||||
const initialWidth = this.size[0];
|
||||
const numWidget = this.widgets?.find(w => w.name === "num_images");
|
||||
if (numWidget) {
|
||||
numWidget.label = "images_loaded";
|
||||
numWidget.value = Math.max(0, Math.min(count || 0, 50));
|
||||
}
|
||||
|
||||
// 1. Store current widget values in properties BEFORE removing them
|
||||
// We skip reading from `this.widgets` on the initial load because it might be scrambling.
|
||||
if (!isInitialLoad && this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) {
|
||||
this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// 2. Remove all existing dynamic insert_frame/strength/header widgets
|
||||
if (this.widgets) {
|
||||
this.widgets = this.widgets.filter(w =>
|
||||
!w.name.startsWith("insert_frame_") &&
|
||||
!w.name.startsWith("strength_") &&
|
||||
!w.name.startsWith("header_")
|
||||
);
|
||||
} else {
|
||||
this.widgets = [];
|
||||
}
|
||||
|
||||
// 3. Add back exactly the right amount of widgets using the cached values
|
||||
for (let i = 1; i <= count; i++) {
|
||||
// Add header/separator widget for grouping
|
||||
const headerName = `header_${i}`;
|
||||
this.addCustomWidget({
|
||||
name: headerName,
|
||||
type: "text",
|
||||
value: `Image #${i}`,
|
||||
draw(ctx, node, widget_width, y, widget_height) {
|
||||
ctx.save();
|
||||
const margin = 10;
|
||||
const topPadding = 15;
|
||||
|
||||
// Subtle separator line
|
||||
ctx.strokeStyle = "#333";
|
||||
ctx.lineWidth = 1;
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(margin, y + 5);
|
||||
ctx.lineTo(widget_width - margin, y + 5);
|
||||
ctx.stroke();
|
||||
|
||||
// Text label
|
||||
ctx.fillStyle = "#dddddd"; // Light gray
|
||||
ctx.font = "bold 12px Arial";
|
||||
ctx.textAlign = "left";
|
||||
ctx.fillText(`Image #${i}`, margin, y + topPadding + 10);
|
||||
ctx.restore();
|
||||
},
|
||||
computeSize(width) {
|
||||
return [width, 35]; // Vertical gap + label height
|
||||
}
|
||||
});
|
||||
|
||||
const insertFrameWidgetName = `insert_frame_${i}`;
|
||||
const strengthWidgetName = `strength_${i}`;
|
||||
|
||||
// Add insert_frame widget with Sync Callback
|
||||
const savedInsertFrameValue = this.properties[insertFrameWidgetName];
|
||||
this.addWidget("number", insertFrameWidgetName,
|
||||
savedInsertFrameValue !== undefined ? savedInsertFrameValue : 0,
|
||||
(value) => {
|
||||
const rounded = Math.round(value);
|
||||
this.properties[insertFrameWidgetName] = rounded;
|
||||
syncWidgetAcrossNodes(this, insertFrameWidgetName, rounded); // Sync out
|
||||
}, { min: -9999, max: 9999, step: 10, precision: 0 }
|
||||
);
|
||||
|
||||
// Add strength widget with Sync Callback
|
||||
const savedStrengthValue = this.properties[strengthWidgetName];
|
||||
this.addWidget("number", strengthWidgetName,
|
||||
savedStrengthValue !== undefined ? savedStrengthValue : 1.0,
|
||||
(value) => {
|
||||
this.properties[strengthWidgetName] = value;
|
||||
syncWidgetAcrossNodes(this, strengthWidgetName, value); // Sync out
|
||||
}, { min: 0.0, max: 1.0, step: 0.01 }
|
||||
);
|
||||
}
|
||||
|
||||
this.setDirtyCanvas(true, true);
|
||||
requestAnimationFrame(() => {
|
||||
if (this.computeSize) {
|
||||
this.setSize(this.computeSize());
|
||||
this.size[0] = initialWidth; // keep width fixed when restructuring
|
||||
}
|
||||
});
|
||||
};
|
||||
|
||||
// --- STRICT ARRAY MAPPER: FIXES ALL SHIFTING FOREVER ---
|
||||
// This runs the exact instant the node is loaded, before any UI widgets shift indices.
|
||||
// It locks the perfectly mapped array values directly into our properties dictionary.
|
||||
const origConfigure = node.configure;
|
||||
node.configure = function(info) {
|
||||
if (origConfigure) {
|
||||
origConfigure.apply(this, arguments);
|
||||
}
|
||||
if (this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name === "num_images" || w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) {
|
||||
this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
// Handle deserialization to load properties properly from JSON
|
||||
const originalOnConfigure = node.onConfigure;
|
||||
node.onConfigure = function(info) {
|
||||
if (originalOnConfigure) {
|
||||
originalOnConfigure.apply(this, arguments);
|
||||
}
|
||||
if (info.properties) {
|
||||
this.properties = { ...this.properties, ...info.properties };
|
||||
}
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
// Fallback to properties.num_images if source node disconnected
|
||||
let targetCount = count !== null ? count : (this.properties.num_images || 0);
|
||||
this._applyWidgetCount(targetCount);
|
||||
}, 100);
|
||||
};
|
||||
|
||||
// --- STRICT ARRAY GENERATOR ---
|
||||
// Completely detach from ComfyUI's blind visual array saving.
|
||||
// We construct an exact 101-element strict array that Python expects.
|
||||
// This makes your node 100% immune to UI/Header index shifting.
|
||||
const originalOnSerialize = node.onSerialize;
|
||||
node.onSerialize = function(info) {
|
||||
// Ensure properties are strictly synced with current widget values before building
|
||||
if (this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name === "num_images" || w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) {
|
||||
this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
if (originalOnSerialize) {
|
||||
originalOnSerialize.apply(this, arguments);
|
||||
}
|
||||
|
||||
info.properties = { ...this.properties };
|
||||
|
||||
// Build the exact strict array that maps 1-to-1 to the Python backend
|
||||
const strictArray = [];
|
||||
const numWidgetVal = this.properties["num_images"];
|
||||
strictArray.push(numWidgetVal !== undefined ? numWidgetVal : 1);
|
||||
|
||||
for (let i = 1; i <= 50; i++) {
|
||||
const fVal = this.properties[`insert_frame_${i}`];
|
||||
const sVal = this.properties[`strength_${i}`];
|
||||
strictArray.push(fVal !== undefined ? fVal : 0);
|
||||
strictArray.push(sVal !== undefined ? sVal : 1.0);
|
||||
}
|
||||
|
||||
info.widgets_values = strictArray;
|
||||
};
|
||||
|
||||
// Set up manual num_images widget callback
|
||||
setTimeout(() => {
|
||||
const numWidget = node.widgets?.find(w => w.name === "num_images");
|
||||
if (numWidget) {
|
||||
numWidget.callback = (val) => {
|
||||
node.properties["num_images"] = val;
|
||||
node._applyWidgetCount(val);
|
||||
};
|
||||
}
|
||||
}, 100);
|
||||
|
||||
// Exposed receiver for push-based notifications
|
||||
node._syncImageCount = function(count) {
|
||||
this._applyWidgetCount(count);
|
||||
};
|
||||
|
||||
// Helper: read image count from a connected MultiImageLoader node
|
||||
function readSourceImageCount(self) {
|
||||
const multiInput = self.inputs?.find(inp => inp.name === "multi_input");
|
||||
if (!multiInput || !multiInput.link) return null;
|
||||
|
||||
const nodeGraph = self.graph || app.graph;
|
||||
|
||||
// Helper to safely trace back through Reroutes and ComfyUI Group Nodes/Subgraphs
|
||||
function traceUpstream(graph, linkId, visited = new Set()) {
|
||||
if (!linkId || visited.has(linkId)) return null;
|
||||
visited.add(linkId);
|
||||
|
||||
const link = graph.links[linkId];
|
||||
if (!link) return null;
|
||||
|
||||
const originNode = graph.getNodeById(link.origin_id);
|
||||
if (!originNode) return null;
|
||||
|
||||
if (originNode.comfyClass === "MultiImageLoader") {
|
||||
return originNode;
|
||||
}
|
||||
|
||||
// Traverse Reroute nodes
|
||||
if (originNode.type === "Reroute" || originNode.comfyClass === "Reroute") {
|
||||
if (originNode.inputs && originNode.inputs.length > 0 && originNode.inputs[0].link) {
|
||||
return traceUpstream(graph, originNode.inputs[0].link, visited);
|
||||
}
|
||||
}
|
||||
|
||||
// Traverse standard ComfyUI Group Nodes (subgraphs)
|
||||
if (typeof originNode.getInnerNode === "function") {
|
||||
try {
|
||||
const innerNode = originNode.getInnerNode(link.origin_slot);
|
||||
if (innerNode && innerNode.comfyClass === "MultiImageLoader") {
|
||||
return innerNode;
|
||||
}
|
||||
} catch (e) {
|
||||
console.warn("Could not trace inner node", e);
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
let sourceNode = traceUpstream(nodeGraph, multiInput.link);
|
||||
|
||||
// Helper to extract the count once we find a node
|
||||
function getCountFromNode(n) {
|
||||
if (typeof n._imageCount === "number") return n._imageCount;
|
||||
const pathsWidget = n.widgets?.find(w => w.name === "image_paths");
|
||||
if (pathsWidget) {
|
||||
return (pathsWidget.value || "").split('\n').map(p => p.trim()).filter(p => p.length > 0).length;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
if (sourceNode) {
|
||||
return getCountFromNode(sourceNode);
|
||||
}
|
||||
|
||||
// Fallback Strategy: If it is connected to something, but we couldn't resolve it
|
||||
// directly (e.g. complex nested 3rd party subgraphs), scan the entire UI.
|
||||
// If there is EXACTLY ONE MultiImageLoader in the workspace, safely assume that's the one.
|
||||
let multiImageLoaders = [];
|
||||
function findAllLoaders(nodes) {
|
||||
if (!nodes) return;
|
||||
for (let n of nodes) {
|
||||
if (n.comfyClass === "MultiImageLoader") {
|
||||
multiImageLoaders.push(n);
|
||||
}
|
||||
if (n.subgraph && n.subgraph._nodes) {
|
||||
findAllLoaders(n.subgraph._nodes);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (app.graph && app.graph._nodes) {
|
||||
findAllLoaders(app.graph._nodes);
|
||||
}
|
||||
|
||||
if (multiImageLoaders.length === 1) {
|
||||
return getCountFromNode(multiImageLoaders[0]);
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
// --- Backup polling via setInterval (4 Hz) ---
|
||||
const pollInterval = setInterval(() => {
|
||||
if (!node.graph) {
|
||||
clearInterval(pollInterval);
|
||||
return;
|
||||
}
|
||||
const count = readSourceImageCount(node);
|
||||
if (count !== null) {
|
||||
node._applyWidgetCount(count);
|
||||
}
|
||||
}, 250);
|
||||
|
||||
// Clean up the interval when the node is deleted
|
||||
const origOnRemoved = node.onRemoved;
|
||||
node.onRemoved = function() {
|
||||
// Unregister this node instance
|
||||
window._LTXKeyframerGlobalNodes.delete(node);
|
||||
|
||||
clearInterval(pollInterval);
|
||||
if (origOnRemoved) origOnRemoved.apply(this, arguments);
|
||||
};
|
||||
|
||||
// --- Connection change handler ---
|
||||
const onConnectionsChange = node.onConnectionsChange;
|
||||
node.onConnectionsChange = function(type, index, connected, link_info) {
|
||||
if (onConnectionsChange) onConnectionsChange.apply(this, arguments);
|
||||
|
||||
if (type === 1) { // 1 = Input
|
||||
const input = this.inputs[index];
|
||||
if (input && input.name === "multi_input") {
|
||||
if (connected) {
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
|
||||
}, 100);
|
||||
} else {
|
||||
this._applyWidgetCount(0);
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// --- Initial sync when first placed on canvas ---
|
||||
const origOnAdded = node.onAdded;
|
||||
node.onAdded = function() {
|
||||
if (origOnAdded) origOnAdded.apply(this, arguments);
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
|
||||
}, 100);
|
||||
};
|
||||
}
|
||||
import { app } from "../../scripts/app.js";
|
||||
|
||||
// Global registry to track all LTXKeyframer nodes across all subgraphs
|
||||
window._LTXKeyframerGlobalNodes = window._LTXKeyframerGlobalNodes || new Set();
|
||||
|
||||
// ComfyUI native trick to cleanly hide/show widgets without deleting them
|
||||
function toggleWidget(widget, visible) {
|
||||
if (visible) {
|
||||
if (widget.origType !== undefined) {
|
||||
widget.type = widget.origType;
|
||||
widget.computeSize = widget.origComputeSize;
|
||||
delete widget.origType;
|
||||
delete widget.origComputeSize;
|
||||
}
|
||||
} else {
|
||||
if (widget.type !== "hidden") {
|
||||
widget.origType = widget.type;
|
||||
widget.origComputeSize = widget.computeSize;
|
||||
widget.type = "hidden";
|
||||
widget.computeSize = () => [0, -4];
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --- NEW SYNC HELPER FUNCTION ---
|
||||
// Finds all other LTXKeyframer nodes globally and mirrors the value to them
|
||||
function syncWidgetAcrossNodes(sourceNode, widgetName, value) {
|
||||
if (!window._LTXKeyframerGlobalNodes) return;
|
||||
|
||||
for (const targetNode of window._LTXKeyframerGlobalNodes) {
|
||||
// Target all OTHER LTXKeyframer nodes by direct object reference
|
||||
if (targetNode !== sourceNode) {
|
||||
|
||||
// 1. Always update the hidden properties cache so it remembers the sync
|
||||
// even if the widget isn't currently visible (e.g. fewer images loaded right now)
|
||||
targetNode.properties[widgetName] = value;
|
||||
|
||||
// 2. If the widget is currently visible on the UI, update it visually
|
||||
if (targetNode.widgets) {
|
||||
const targetWidget = targetNode.widgets.find(w => w.name === widgetName);
|
||||
if (targetWidget && targetWidget.value !== value) {
|
||||
targetWidget.value = value;
|
||||
targetNode.setDirtyCanvas(true, false);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "Comfy.LTXKeyframer.DynamicInputs",
|
||||
async nodeCreated(node) {
|
||||
if (node.comfyClass !== "LTXKeyframer") return;
|
||||
|
||||
// Register this node instance globally
|
||||
window._LTXKeyframerGlobalNodes.add(node);
|
||||
|
||||
node._currentImageCount = -1; // Force first update
|
||||
|
||||
// Initialize persistent properties cache
|
||||
node.properties = node.properties || {};
|
||||
|
||||
// Add subtle separator line above images_loaded
|
||||
node.addCustomWidget({
|
||||
name: "num_images_separator",
|
||||
type: "text",
|
||||
draw(ctx, node, widget_width, y, widget_height) {
|
||||
ctx.save();
|
||||
ctx.strokeStyle = "#444";
|
||||
ctx.lineWidth = 1;
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(10, y + 5);
|
||||
ctx.lineTo(widget_width - 10, y + 5);
|
||||
ctx.stroke();
|
||||
ctx.restore();
|
||||
},
|
||||
computeSize(width) {
|
||||
return [width, 10];
|
||||
}
|
||||
});
|
||||
|
||||
// Move separator before num_images
|
||||
const moveSeparator = () => {
|
||||
const idx = node.widgets.findIndex(w => w.name === "num_images");
|
||||
const sepIdx = node.widgets.findIndex(w => w.name === "num_images_separator");
|
||||
if (idx !== -1 && sepIdx !== -1) {
|
||||
const separator = node.widgets.splice(sepIdx, 1)[0];
|
||||
node.widgets.splice(idx, 0, separator);
|
||||
}
|
||||
};
|
||||
setTimeout(moveSeparator, 50); // Small delay to ensure num_images is present
|
||||
|
||||
// Core update: synchronize widget visibility to match imageCount
|
||||
node._applyWidgetCount = function(count) {
|
||||
const isInitialLoad = this._currentImageCount === -1;
|
||||
|
||||
if (this._currentImageCount === count && !isInitialLoad) return;
|
||||
this._currentImageCount = count;
|
||||
|
||||
const initialWidth = this.size[0];
|
||||
const numWidget = this.widgets?.find(w => w.name === "num_images");
|
||||
if (numWidget) {
|
||||
numWidget.label = "images_loaded";
|
||||
numWidget.value = Math.max(0, Math.min(count || 0, 50));
|
||||
}
|
||||
|
||||
// 1. Store current widget values in properties BEFORE removing them
|
||||
// We skip reading from `this.widgets` on the initial load because it might be scrambling.
|
||||
if (!isInitialLoad && this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) {
|
||||
this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// 2. Remove all existing dynamic insert_frame/strength/header widgets
|
||||
if (this.widgets) {
|
||||
this.widgets = this.widgets.filter(w =>
|
||||
!w.name.startsWith("insert_frame_") &&
|
||||
!w.name.startsWith("strength_") &&
|
||||
!w.name.startsWith("header_")
|
||||
);
|
||||
} else {
|
||||
this.widgets = [];
|
||||
}
|
||||
|
||||
// 3. Add back exactly the right amount of widgets using the cached values
|
||||
for (let i = 1; i <= count; i++) {
|
||||
// Add header/separator widget for grouping
|
||||
const headerName = `header_${i}`;
|
||||
this.addCustomWidget({
|
||||
name: headerName,
|
||||
type: "text",
|
||||
value: `Image #${i}`,
|
||||
draw(ctx, node, widget_width, y, widget_height) {
|
||||
ctx.save();
|
||||
const margin = 10;
|
||||
const topPadding = 15;
|
||||
|
||||
// Subtle separator line
|
||||
ctx.strokeStyle = "#333";
|
||||
ctx.lineWidth = 1;
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(margin, y + 5);
|
||||
ctx.lineTo(widget_width - margin, y + 5);
|
||||
ctx.stroke();
|
||||
|
||||
// Text label
|
||||
ctx.fillStyle = "#dddddd"; // Light gray
|
||||
ctx.font = "bold 12px Arial";
|
||||
ctx.textAlign = "left";
|
||||
ctx.fillText(`Image #${i}`, margin, y + topPadding + 10);
|
||||
ctx.restore();
|
||||
},
|
||||
computeSize(width) {
|
||||
return [width, 35]; // Vertical gap + label height
|
||||
}
|
||||
});
|
||||
|
||||
const insertFrameWidgetName = `insert_frame_${i}`;
|
||||
const strengthWidgetName = `strength_${i}`;
|
||||
|
||||
// Add insert_frame widget with Sync Callback
|
||||
const savedInsertFrameValue = this.properties[insertFrameWidgetName];
|
||||
this.addWidget("number", insertFrameWidgetName,
|
||||
savedInsertFrameValue !== undefined ? savedInsertFrameValue : 0,
|
||||
(value) => {
|
||||
const rounded = Math.round(value);
|
||||
this.properties[insertFrameWidgetName] = rounded;
|
||||
syncWidgetAcrossNodes(this, insertFrameWidgetName, rounded); // Sync out
|
||||
}, { min: -9999, max: 9999, step: 10, precision: 0 }
|
||||
);
|
||||
|
||||
// Add strength widget with Sync Callback
|
||||
const savedStrengthValue = this.properties[strengthWidgetName];
|
||||
this.addWidget("number", strengthWidgetName,
|
||||
savedStrengthValue !== undefined ? savedStrengthValue : 1.0,
|
||||
(value) => {
|
||||
this.properties[strengthWidgetName] = value;
|
||||
syncWidgetAcrossNodes(this, strengthWidgetName, value); // Sync out
|
||||
}, { min: 0.0, max: 1.0, step: 0.01 }
|
||||
);
|
||||
}
|
||||
|
||||
this.setDirtyCanvas(true, true);
|
||||
requestAnimationFrame(() => {
|
||||
if (this.computeSize) {
|
||||
this.setSize(this.computeSize());
|
||||
this.size[0] = initialWidth; // keep width fixed when restructuring
|
||||
}
|
||||
});
|
||||
};
|
||||
|
||||
// --- STRICT ARRAY MAPPER: FIXES ALL SHIFTING FOREVER ---
|
||||
// This runs the exact instant the node is loaded, before any UI widgets shift indices.
|
||||
// It locks the perfectly mapped array values directly into our properties dictionary.
|
||||
const origConfigure = node.configure;
|
||||
node.configure = function(info) {
|
||||
if (origConfigure) {
|
||||
origConfigure.apply(this, arguments);
|
||||
}
|
||||
if (this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name === "num_images" || w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) {
|
||||
this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
// Handle deserialization to load properties properly from JSON
|
||||
const originalOnConfigure = node.onConfigure;
|
||||
node.onConfigure = function(info) {
|
||||
if (originalOnConfigure) {
|
||||
originalOnConfigure.apply(this, arguments);
|
||||
}
|
||||
if (info.properties) {
|
||||
this.properties = { ...this.properties, ...info.properties };
|
||||
}
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
// Fallback to properties.num_images if source node disconnected
|
||||
let targetCount = count !== null ? count : (this.properties.num_images || 0);
|
||||
this._applyWidgetCount(targetCount);
|
||||
}, 100);
|
||||
};
|
||||
|
||||
// --- STRICT ARRAY GENERATOR ---
|
||||
// Completely detach from ComfyUI's blind visual array saving.
|
||||
// We construct an exact 101-element strict array that Python expects.
|
||||
// This makes your node 100% immune to UI/Header index shifting.
|
||||
const originalOnSerialize = node.onSerialize;
|
||||
node.onSerialize = function(info) {
|
||||
// Ensure properties are strictly synced with current widget values before building
|
||||
if (this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name === "num_images" || w.name.startsWith("insert_frame_") || w.name.startsWith("strength_")) {
|
||||
this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
if (originalOnSerialize) {
|
||||
originalOnSerialize.apply(this, arguments);
|
||||
}
|
||||
|
||||
info.properties = { ...this.properties };
|
||||
|
||||
// Build the exact strict array that maps 1-to-1 to the Python backend
|
||||
const strictArray = [];
|
||||
const numWidgetVal = this.properties["num_images"];
|
||||
strictArray.push(numWidgetVal !== undefined ? numWidgetVal : 1);
|
||||
|
||||
for (let i = 1; i <= 50; i++) {
|
||||
const fVal = this.properties[`insert_frame_${i}`];
|
||||
const sVal = this.properties[`strength_${i}`];
|
||||
strictArray.push(fVal !== undefined ? fVal : 0);
|
||||
strictArray.push(sVal !== undefined ? sVal : 1.0);
|
||||
}
|
||||
|
||||
info.widgets_values = strictArray;
|
||||
};
|
||||
|
||||
// Set up manual num_images widget callback
|
||||
setTimeout(() => {
|
||||
const numWidget = node.widgets?.find(w => w.name === "num_images");
|
||||
if (numWidget) {
|
||||
numWidget.callback = (val) => {
|
||||
node.properties["num_images"] = val;
|
||||
node._applyWidgetCount(val);
|
||||
};
|
||||
}
|
||||
}, 100);
|
||||
|
||||
// Exposed receiver for push-based notifications
|
||||
node._syncImageCount = function(count) {
|
||||
this._applyWidgetCount(count);
|
||||
};
|
||||
|
||||
// Helper: read image count from a connected MultiImageLoader node
|
||||
function readSourceImageCount(self) {
|
||||
const multiInput = self.inputs?.find(inp => inp.name === "multi_input");
|
||||
if (!multiInput || !multiInput.link) return null;
|
||||
|
||||
const nodeGraph = self.graph || app.graph;
|
||||
|
||||
// Helper to safely trace back through Reroutes and ComfyUI Group Nodes/Subgraphs
|
||||
function traceUpstream(graph, linkId, visited = new Set()) {
|
||||
if (!linkId || visited.has(linkId)) return null;
|
||||
visited.add(linkId);
|
||||
|
||||
const link = graph.links[linkId];
|
||||
if (!link) return null;
|
||||
|
||||
const originNode = graph.getNodeById(link.origin_id);
|
||||
if (!originNode) return null;
|
||||
|
||||
if (originNode.comfyClass === "MultiImageLoader") {
|
||||
return originNode;
|
||||
}
|
||||
|
||||
// Traverse Reroute nodes
|
||||
if (originNode.type === "Reroute" || originNode.comfyClass === "Reroute") {
|
||||
if (originNode.inputs && originNode.inputs.length > 0 && originNode.inputs[0].link) {
|
||||
return traceUpstream(graph, originNode.inputs[0].link, visited);
|
||||
}
|
||||
}
|
||||
|
||||
// Traverse standard ComfyUI Group Nodes (subgraphs)
|
||||
if (typeof originNode.getInnerNode === "function") {
|
||||
try {
|
||||
const innerNode = originNode.getInnerNode(link.origin_slot);
|
||||
if (innerNode && innerNode.comfyClass === "MultiImageLoader") {
|
||||
return innerNode;
|
||||
}
|
||||
} catch (e) {
|
||||
console.warn("Could not trace inner node", e);
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
let sourceNode = traceUpstream(nodeGraph, multiInput.link);
|
||||
|
||||
// Helper to extract the count once we find a node
|
||||
function getCountFromNode(n) {
|
||||
if (typeof n._imageCount === "number") return n._imageCount;
|
||||
const pathsWidget = n.widgets?.find(w => w.name === "image_paths");
|
||||
if (pathsWidget) {
|
||||
return (pathsWidget.value || "").split('\n').map(p => p.trim()).filter(p => p.length > 0).length;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
if (sourceNode) {
|
||||
return getCountFromNode(sourceNode);
|
||||
}
|
||||
|
||||
// Fallback Strategy: If it is connected to something, but we couldn't resolve it
|
||||
// directly (e.g. complex nested 3rd party subgraphs), scan the entire UI.
|
||||
// If there is EXACTLY ONE MultiImageLoader in the workspace, safely assume that's the one.
|
||||
let multiImageLoaders = [];
|
||||
function findAllLoaders(nodes) {
|
||||
if (!nodes) return;
|
||||
for (let n of nodes) {
|
||||
if (n.comfyClass === "MultiImageLoader") {
|
||||
multiImageLoaders.push(n);
|
||||
}
|
||||
if (n.subgraph && n.subgraph._nodes) {
|
||||
findAllLoaders(n.subgraph._nodes);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (app.graph && app.graph._nodes) {
|
||||
findAllLoaders(app.graph._nodes);
|
||||
}
|
||||
|
||||
if (multiImageLoaders.length === 1) {
|
||||
return getCountFromNode(multiImageLoaders[0]);
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
// --- Backup polling via setInterval (4 Hz) ---
|
||||
const pollInterval = setInterval(() => {
|
||||
if (!node.graph) {
|
||||
clearInterval(pollInterval);
|
||||
return;
|
||||
}
|
||||
const count = readSourceImageCount(node);
|
||||
if (count !== null) {
|
||||
node._applyWidgetCount(count);
|
||||
}
|
||||
}, 250);
|
||||
|
||||
// Clean up the interval when the node is deleted
|
||||
const origOnRemoved = node.onRemoved;
|
||||
node.onRemoved = function() {
|
||||
// Unregister this node instance
|
||||
window._LTXKeyframerGlobalNodes.delete(node);
|
||||
|
||||
clearInterval(pollInterval);
|
||||
if (origOnRemoved) origOnRemoved.apply(this, arguments);
|
||||
};
|
||||
|
||||
// --- Connection change handler ---
|
||||
const onConnectionsChange = node.onConnectionsChange;
|
||||
node.onConnectionsChange = function(type, index, connected, link_info) {
|
||||
if (onConnectionsChange) onConnectionsChange.apply(this, arguments);
|
||||
|
||||
if (type === 1) { // 1 = Input
|
||||
const input = this.inputs[index];
|
||||
if (input && input.name === "multi_input") {
|
||||
if (connected) {
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
|
||||
}, 100);
|
||||
} else {
|
||||
this._applyWidgetCount(0);
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// --- Initial sync when first placed on canvas ---
|
||||
const origOnAdded = node.onAdded;
|
||||
node.onAdded = function() {
|
||||
if (origOnAdded) origOnAdded.apply(this, arguments);
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
|
||||
}, 100);
|
||||
};
|
||||
}
|
||||
});
|
||||
@@ -1,398 +1,398 @@
|
||||
import { app } from "../../scripts/app.js";
|
||||
|
||||
// Global registry to track all LTXSequencer nodes across all subgraphs
|
||||
window._LTXSequencerGlobalNodes = window._LTXSequencerGlobalNodes || new Set();
|
||||
|
||||
// ComfyUI native trick to cleanly hide/show widgets without deleting them
|
||||
function toggleWidget(widget, visible) {
|
||||
if (visible) {
|
||||
if (widget.origType !== undefined) {
|
||||
widget.type = widget.origType;
|
||||
widget.computeSize = widget.origComputeSize;
|
||||
delete widget.origType;
|
||||
delete widget.origComputeSize;
|
||||
}
|
||||
} else {
|
||||
if (widget.type !== "hidden") {
|
||||
widget.origType = widget.type;
|
||||
widget.origComputeSize = widget.computeSize;
|
||||
widget.type = "hidden";
|
||||
widget.computeSize = () => [0, -4];
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* FULL STATE SYNC:
|
||||
* Instead of syncing one widget, we push the entire properties object
|
||||
* to ensure no values are ever lost during subgraph transitions or deletions.
|
||||
*/
|
||||
function syncFullStateAcrossNodes(sourceNode) {
|
||||
if (!window._LTXSequencerGlobalNodes) return;
|
||||
|
||||
for (const targetNode of window._LTXSequencerGlobalNodes) {
|
||||
if (targetNode === sourceNode) continue;
|
||||
|
||||
// 1. Mirror the properties object completely
|
||||
// We use a shallow copy to ensure we don't accidentally share object references
|
||||
const newState = { ...sourceNode.properties };
|
||||
targetNode.properties = { ...targetNode.properties, ...newState };
|
||||
|
||||
// 2. Check if we need to rebuild the widget list (if num_images changed)
|
||||
const targetImageCount = targetNode.properties["num_images"] || 0;
|
||||
const currentVisibleCount = targetNode._currentImageCount;
|
||||
|
||||
if (targetImageCount !== currentVisibleCount) {
|
||||
targetNode._applyWidgetCount(targetImageCount);
|
||||
}
|
||||
|
||||
// 3. Update all existing widget values visually
|
||||
if (targetNode.widgets) {
|
||||
let modeChanged = false;
|
||||
targetNode.widgets.forEach(w => {
|
||||
const newValue = targetNode.properties[w.name];
|
||||
if (newValue !== undefined && w.value !== newValue) {
|
||||
w.value = newValue;
|
||||
if (w.name === "insert_mode") modeChanged = true;
|
||||
}
|
||||
});
|
||||
|
||||
if (modeChanged && targetNode._updateVisibility) {
|
||||
targetNode._updateVisibility();
|
||||
}
|
||||
}
|
||||
|
||||
targetNode.setDirtyCanvas(true, false);
|
||||
}
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "Comfy.LTXSequencer.DynamicInputs",
|
||||
async nodeCreated(node) {
|
||||
if (node.comfyClass !== "LTXSequencer") return;
|
||||
|
||||
// Register this node instance globally
|
||||
window._LTXSequencerGlobalNodes.add(node);
|
||||
|
||||
node._currentImageCount = -1; // Force first update
|
||||
|
||||
// Initialize persistent properties cache
|
||||
node.properties = node.properties || {};
|
||||
|
||||
// Add subtle separator line above images_loaded
|
||||
node.addCustomWidget({
|
||||
name: "num_images_separator",
|
||||
type: "text",
|
||||
draw(ctx, node, widget_width, y, widget_height) {
|
||||
ctx.save();
|
||||
ctx.strokeStyle = "#444";
|
||||
ctx.lineWidth = 1;
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(10, y + 5);
|
||||
ctx.lineTo(widget_width - 10, y + 5);
|
||||
ctx.stroke();
|
||||
ctx.restore();
|
||||
},
|
||||
computeSize(width) {
|
||||
return [width, 10];
|
||||
}
|
||||
});
|
||||
|
||||
// Move separator before num_images
|
||||
const moveSeparator = () => {
|
||||
const idx = node.widgets.findIndex(w => w.name === "num_images");
|
||||
const sepIdx = node.widgets.findIndex(w => w.name === "num_images_separator");
|
||||
if (idx !== -1 && sepIdx !== -1) {
|
||||
const separator = node.widgets.splice(sepIdx, 1)[0];
|
||||
node.widgets.splice(idx, 0, separator);
|
||||
}
|
||||
};
|
||||
setTimeout(moveSeparator, 50);
|
||||
|
||||
// Binds custom callbacks to python-schema generated widgets
|
||||
node._hookStaticWidgets = function() {
|
||||
if (!this.widgets) return;
|
||||
const staticNames = ["num_images", "insert_mode", "frame_rate"];
|
||||
staticNames.forEach(name => {
|
||||
const w = this.widgets.find(w => w.name === name);
|
||||
if (w && !w._has_custom_callback) {
|
||||
const orig = w.callback;
|
||||
w.callback = (val) => {
|
||||
this.properties[name] = val;
|
||||
|
||||
if (name === "num_images") {
|
||||
this._applyWidgetCount(val);
|
||||
}
|
||||
|
||||
// Push full state to siblings
|
||||
syncFullStateAcrossNodes(this);
|
||||
|
||||
if (name === "insert_mode") {
|
||||
this._updateVisibility();
|
||||
}
|
||||
if (orig) orig.apply(w, [val]);
|
||||
};
|
||||
w._has_custom_callback = true;
|
||||
}
|
||||
});
|
||||
};
|
||||
|
||||
// Handles show/hiding widgets based on insertion method
|
||||
node._updateVisibility = function() {
|
||||
const mode = this.properties["insert_mode"] || "frames";
|
||||
if (!this.widgets) return;
|
||||
|
||||
let changed = false;
|
||||
for (const w of this.widgets) {
|
||||
let shouldBeVisible = true;
|
||||
|
||||
if (w.name.startsWith("insert_frame_")) {
|
||||
shouldBeVisible = (mode === "frames");
|
||||
} else if (w.name.startsWith("insert_second_")) {
|
||||
shouldBeVisible = (mode === "seconds");
|
||||
}
|
||||
|
||||
const isHidden = (w.type === "hidden");
|
||||
if (shouldBeVisible && isHidden) {
|
||||
toggleWidget(w, true);
|
||||
changed = true;
|
||||
} else if (!shouldBeVisible && !isHidden) {
|
||||
toggleWidget(w, false);
|
||||
changed = true;
|
||||
}
|
||||
}
|
||||
|
||||
if (changed) {
|
||||
this.setDirtyCanvas(true, true);
|
||||
requestAnimationFrame(() => {
|
||||
if (this.computeSize) {
|
||||
this.setSize(this.computeSize());
|
||||
}
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
// Core update: synchronize widget visibility to match imageCount
|
||||
node._applyWidgetCount = function(count) {
|
||||
this._hookStaticWidgets();
|
||||
|
||||
const isInitialLoad = this._currentImageCount === -1;
|
||||
if (this._currentImageCount === count && !isInitialLoad) return;
|
||||
this._currentImageCount = count;
|
||||
|
||||
const initialWidth = this.size[0];
|
||||
const numWidget = this.widgets?.find(w => w.name === "num_images");
|
||||
if (numWidget) {
|
||||
numWidget.label = "images_loaded";
|
||||
numWidget.value = Math.max(0, Math.min(count || 0, 50));
|
||||
}
|
||||
|
||||
// 1. Update properties from current widget values before restructuring
|
||||
if (this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) {
|
||||
if (w.type !== "hidden" && w.type !== "button") this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// 2. Clear dynamic widgets
|
||||
if (this.widgets) {
|
||||
this.widgets = this.widgets.filter(w =>
|
||||
!w.name.startsWith("insert_frame_") &&
|
||||
!w.name.startsWith("insert_second_") &&
|
||||
!w.name.startsWith("strength_") &&
|
||||
!w.name.startsWith("header_")
|
||||
);
|
||||
} else {
|
||||
this.widgets = [];
|
||||
}
|
||||
|
||||
// 3. Rebuild precisely
|
||||
for (let i = 1; i <= count; i++) {
|
||||
// Header
|
||||
const headerName = `header_${i}`;
|
||||
this.addCustomWidget({
|
||||
name: headerName,
|
||||
type: "text",
|
||||
value: `Image #${i}`,
|
||||
draw(ctx, node, widget_width, y, widget_height) {
|
||||
ctx.save();
|
||||
const margin = 10;
|
||||
const topPadding = 15;
|
||||
ctx.strokeStyle = "#333";
|
||||
ctx.lineWidth = 1;
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(margin, y + 5);
|
||||
ctx.lineTo(widget_width - margin, y + 5);
|
||||
ctx.stroke();
|
||||
ctx.fillStyle = "#dddddd";
|
||||
ctx.font = "bold 12px Arial";
|
||||
ctx.textAlign = "left";
|
||||
ctx.fillText(`Image #${i}`, margin, y + topPadding + 10);
|
||||
ctx.restore();
|
||||
},
|
||||
computeSize(width) { return [width, 35]; }
|
||||
});
|
||||
|
||||
// Helper to add widget with full sync
|
||||
const addSyncedWidget = (type, name, def, options) => {
|
||||
const saved = this.properties[name];
|
||||
return this.addWidget(type, name, saved !== undefined ? saved : def, (val) => {
|
||||
this.properties[name] = val;
|
||||
syncFullStateAcrossNodes(this);
|
||||
}, options);
|
||||
};
|
||||
|
||||
addSyncedWidget("number", `insert_frame_${i}`, 0, { min: -9999, max: 9999, step: 10, precision: 0 });
|
||||
addSyncedWidget("number", `insert_second_${i}`, 0.0, { min: 0.0, max: 9999.0, step: 0.1, precision: 2 });
|
||||
addSyncedWidget("number", `strength_${i}`, 1.0, { min: 0.0, max: 1.0, step: 0.01 });
|
||||
}
|
||||
|
||||
this._updateVisibility();
|
||||
this.setDirtyCanvas(true, true);
|
||||
requestAnimationFrame(() => {
|
||||
if (this.computeSize) {
|
||||
this.setSize(this.computeSize());
|
||||
this.size[0] = initialWidth;
|
||||
}
|
||||
});
|
||||
};
|
||||
|
||||
const origConfigure = node.configure;
|
||||
node.configure = function(info) {
|
||||
if (origConfigure) origConfigure.apply(this, arguments);
|
||||
if (this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) {
|
||||
if (w.type !== "button") this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
node.onConfigure = function(info) {
|
||||
if (info.properties) {
|
||||
this.properties = { ...this.properties, ...info.properties };
|
||||
}
|
||||
this._hookStaticWidgets();
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
let targetCount = count !== null ? count : (this.properties.num_images || 0);
|
||||
this._applyWidgetCount(targetCount);
|
||||
this._updateVisibility();
|
||||
}, 100);
|
||||
};
|
||||
|
||||
const originalOnSerialize = node.onSerialize;
|
||||
node.onSerialize = function(info) {
|
||||
if (this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) {
|
||||
if (w.type !== "hidden" && w.type !== "button") this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
if (originalOnSerialize) originalOnSerialize.apply(this, arguments);
|
||||
info.properties = { ...this.properties };
|
||||
|
||||
const strictArray = [];
|
||||
strictArray.push(this.properties["num_images"] !== undefined ? this.properties["num_images"] : 1);
|
||||
strictArray.push(this.properties["insert_mode"] !== undefined ? this.properties["insert_mode"] : "frames");
|
||||
strictArray.push(this.properties["frame_rate"] !== undefined ? this.properties["frame_rate"] : 24);
|
||||
|
||||
for (let i = 1; i <= 50; i++) {
|
||||
strictArray.push(this.properties[`insert_frame_${i}`] !== undefined ? this.properties[`insert_frame_${i}`] : 0);
|
||||
strictArray.push(this.properties[`insert_second_${i}`] !== undefined ? this.properties[`insert_second_${i}`] : 0.0);
|
||||
strictArray.push(this.properties[`strength_${i}`] !== undefined ? this.properties[`strength_${i}`] : 1.0);
|
||||
}
|
||||
info.widgets_values = strictArray;
|
||||
};
|
||||
|
||||
function readSourceImageCount(self) {
|
||||
const multiInput = self.inputs?.find(inp => inp.name === "multi_input");
|
||||
if (!multiInput || !multiInput.link) return null;
|
||||
const nodeGraph = self.graph || app.graph;
|
||||
|
||||
function traceUpstream(graph, linkId, visited = new Set()) {
|
||||
if (!linkId || visited.has(linkId)) return null;
|
||||
visited.add(linkId);
|
||||
const link = graph.links[linkId];
|
||||
if (!link) return null;
|
||||
const originNode = graph.getNodeById(link.origin_id);
|
||||
if (!originNode) return null;
|
||||
if (originNode.comfyClass === "MultiImageLoader") return originNode;
|
||||
if (originNode.type === "Reroute" || originNode.comfyClass === "Reroute") {
|
||||
if (originNode.inputs?.[0]?.link) return traceUpstream(graph, originNode.inputs[0].link, visited);
|
||||
}
|
||||
if (typeof originNode.getInnerNode === "function") {
|
||||
try {
|
||||
const innerNode = originNode.getInnerNode(link.origin_slot);
|
||||
if (innerNode?.comfyClass === "MultiImageLoader") return innerNode;
|
||||
} catch (e) {}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
let sourceNode = traceUpstream(nodeGraph, multiInput.link);
|
||||
function getCountFromNode(n) {
|
||||
if (typeof n._imageCount === "number") return n._imageCount;
|
||||
const pathsWidget = n.widgets?.find(w => w.name === "image_paths");
|
||||
return pathsWidget ? (pathsWidget.value || "").split('\n').filter(p => p.trim()).length : null;
|
||||
}
|
||||
|
||||
if (sourceNode) return getCountFromNode(sourceNode);
|
||||
|
||||
let multiImageLoaders = [];
|
||||
function findAllLoaders(nodes) {
|
||||
if (!nodes) return;
|
||||
for (let n of nodes) {
|
||||
if (n.comfyClass === "MultiImageLoader") multiImageLoaders.push(n);
|
||||
if (n.subgraph?._nodes) findAllLoaders(n.subgraph._nodes);
|
||||
}
|
||||
}
|
||||
if (app.graph?._nodes) findAllLoaders(app.graph._nodes);
|
||||
if (multiImageLoaders.length === 1) return getCountFromNode(multiImageLoaders[0]);
|
||||
return null;
|
||||
}
|
||||
|
||||
const pollInterval = setInterval(() => {
|
||||
if (!node.graph) {
|
||||
clearInterval(pollInterval);
|
||||
return;
|
||||
}
|
||||
const count = readSourceImageCount(node);
|
||||
if (count !== null && count !== node._currentImageCount) {
|
||||
node._applyWidgetCount(count);
|
||||
syncFullStateAcrossNodes(node); // Sync the new count to others
|
||||
}
|
||||
}, 500);
|
||||
|
||||
const origOnRemoved = node.onRemoved;
|
||||
node.onRemoved = function() {
|
||||
window._LTXSequencerGlobalNodes.delete(node);
|
||||
clearInterval(pollInterval);
|
||||
if (origOnRemoved) origOnRemoved.apply(this, arguments);
|
||||
};
|
||||
|
||||
node.onConnectionsChange = function(type, index, connected) {
|
||||
if (type === 1 && this.inputs[index]?.name === "multi_input") {
|
||||
if (connected) {
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
|
||||
}, 100);
|
||||
} else {
|
||||
this._applyWidgetCount(0);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
node.onAdded = function() {
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
|
||||
}, 100);
|
||||
};
|
||||
}
|
||||
import { app } from "../../scripts/app.js";
|
||||
|
||||
// Global registry to track all LTXSequencer nodes across all subgraphs
|
||||
window._LTXSequencerGlobalNodes = window._LTXSequencerGlobalNodes || new Set();
|
||||
|
||||
// ComfyUI native trick to cleanly hide/show widgets without deleting them
|
||||
function toggleWidget(widget, visible) {
|
||||
if (visible) {
|
||||
if (widget.origType !== undefined) {
|
||||
widget.type = widget.origType;
|
||||
widget.computeSize = widget.origComputeSize;
|
||||
delete widget.origType;
|
||||
delete widget.origComputeSize;
|
||||
}
|
||||
} else {
|
||||
if (widget.type !== "hidden") {
|
||||
widget.origType = widget.type;
|
||||
widget.origComputeSize = widget.computeSize;
|
||||
widget.type = "hidden";
|
||||
widget.computeSize = () => [0, -4];
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* FULL STATE SYNC:
|
||||
* Instead of syncing one widget, we push the entire properties object
|
||||
* to ensure no values are ever lost during subgraph transitions or deletions.
|
||||
*/
|
||||
function syncFullStateAcrossNodes(sourceNode) {
|
||||
if (!window._LTXSequencerGlobalNodes) return;
|
||||
|
||||
for (const targetNode of window._LTXSequencerGlobalNodes) {
|
||||
if (targetNode === sourceNode) continue;
|
||||
|
||||
// 1. Mirror the properties object completely
|
||||
// We use a shallow copy to ensure we don't accidentally share object references
|
||||
const newState = { ...sourceNode.properties };
|
||||
targetNode.properties = { ...targetNode.properties, ...newState };
|
||||
|
||||
// 2. Check if we need to rebuild the widget list (if num_images changed)
|
||||
const targetImageCount = targetNode.properties["num_images"] || 0;
|
||||
const currentVisibleCount = targetNode._currentImageCount;
|
||||
|
||||
if (targetImageCount !== currentVisibleCount) {
|
||||
targetNode._applyWidgetCount(targetImageCount);
|
||||
}
|
||||
|
||||
// 3. Update all existing widget values visually
|
||||
if (targetNode.widgets) {
|
||||
let modeChanged = false;
|
||||
targetNode.widgets.forEach(w => {
|
||||
const newValue = targetNode.properties[w.name];
|
||||
if (newValue !== undefined && w.value !== newValue) {
|
||||
w.value = newValue;
|
||||
if (w.name === "insert_mode") modeChanged = true;
|
||||
}
|
||||
});
|
||||
|
||||
if (modeChanged && targetNode._updateVisibility) {
|
||||
targetNode._updateVisibility();
|
||||
}
|
||||
}
|
||||
|
||||
targetNode.setDirtyCanvas(true, false);
|
||||
}
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "Comfy.LTXSequencer.DynamicInputs",
|
||||
async nodeCreated(node) {
|
||||
if (node.comfyClass !== "LTXSequencer") return;
|
||||
|
||||
// Register this node instance globally
|
||||
window._LTXSequencerGlobalNodes.add(node);
|
||||
|
||||
node._currentImageCount = -1; // Force first update
|
||||
|
||||
// Initialize persistent properties cache
|
||||
node.properties = node.properties || {};
|
||||
|
||||
// Add subtle separator line above images_loaded
|
||||
node.addCustomWidget({
|
||||
name: "num_images_separator",
|
||||
type: "text",
|
||||
draw(ctx, node, widget_width, y, widget_height) {
|
||||
ctx.save();
|
||||
ctx.strokeStyle = "#444";
|
||||
ctx.lineWidth = 1;
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(10, y + 5);
|
||||
ctx.lineTo(widget_width - 10, y + 5);
|
||||
ctx.stroke();
|
||||
ctx.restore();
|
||||
},
|
||||
computeSize(width) {
|
||||
return [width, 10];
|
||||
}
|
||||
});
|
||||
|
||||
// Move separator before num_images
|
||||
const moveSeparator = () => {
|
||||
const idx = node.widgets.findIndex(w => w.name === "num_images");
|
||||
const sepIdx = node.widgets.findIndex(w => w.name === "num_images_separator");
|
||||
if (idx !== -1 && sepIdx !== -1) {
|
||||
const separator = node.widgets.splice(sepIdx, 1)[0];
|
||||
node.widgets.splice(idx, 0, separator);
|
||||
}
|
||||
};
|
||||
setTimeout(moveSeparator, 50);
|
||||
|
||||
// Binds custom callbacks to python-schema generated widgets
|
||||
node._hookStaticWidgets = function() {
|
||||
if (!this.widgets) return;
|
||||
const staticNames = ["num_images", "insert_mode", "frame_rate"];
|
||||
staticNames.forEach(name => {
|
||||
const w = this.widgets.find(w => w.name === name);
|
||||
if (w && !w._has_custom_callback) {
|
||||
const orig = w.callback;
|
||||
w.callback = (val) => {
|
||||
this.properties[name] = val;
|
||||
|
||||
if (name === "num_images") {
|
||||
this._applyWidgetCount(val);
|
||||
}
|
||||
|
||||
// Push full state to siblings
|
||||
syncFullStateAcrossNodes(this);
|
||||
|
||||
if (name === "insert_mode") {
|
||||
this._updateVisibility();
|
||||
}
|
||||
if (orig) orig.apply(w, [val]);
|
||||
};
|
||||
w._has_custom_callback = true;
|
||||
}
|
||||
});
|
||||
};
|
||||
|
||||
// Handles show/hiding widgets based on insertion method
|
||||
node._updateVisibility = function() {
|
||||
const mode = this.properties["insert_mode"] || "frames";
|
||||
if (!this.widgets) return;
|
||||
|
||||
let changed = false;
|
||||
for (const w of this.widgets) {
|
||||
let shouldBeVisible = true;
|
||||
|
||||
if (w.name.startsWith("insert_frame_")) {
|
||||
shouldBeVisible = (mode === "frames");
|
||||
} else if (w.name.startsWith("insert_second_")) {
|
||||
shouldBeVisible = (mode === "seconds");
|
||||
}
|
||||
|
||||
const isHidden = (w.type === "hidden");
|
||||
if (shouldBeVisible && isHidden) {
|
||||
toggleWidget(w, true);
|
||||
changed = true;
|
||||
} else if (!shouldBeVisible && !isHidden) {
|
||||
toggleWidget(w, false);
|
||||
changed = true;
|
||||
}
|
||||
}
|
||||
|
||||
if (changed) {
|
||||
this.setDirtyCanvas(true, true);
|
||||
requestAnimationFrame(() => {
|
||||
if (this.computeSize) {
|
||||
this.setSize(this.computeSize());
|
||||
}
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
// Core update: synchronize widget visibility to match imageCount
|
||||
node._applyWidgetCount = function(count) {
|
||||
this._hookStaticWidgets();
|
||||
|
||||
const isInitialLoad = this._currentImageCount === -1;
|
||||
if (this._currentImageCount === count && !isInitialLoad) return;
|
||||
this._currentImageCount = count;
|
||||
|
||||
const initialWidth = this.size[0];
|
||||
const numWidget = this.widgets?.find(w => w.name === "num_images");
|
||||
if (numWidget) {
|
||||
numWidget.label = "images_loaded";
|
||||
numWidget.value = Math.max(0, Math.min(count || 0, 50));
|
||||
}
|
||||
|
||||
// 1. Update properties from current widget values before restructuring
|
||||
if (this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) {
|
||||
if (w.type !== "hidden" && w.type !== "button") this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// 2. Clear dynamic widgets
|
||||
if (this.widgets) {
|
||||
this.widgets = this.widgets.filter(w =>
|
||||
!w.name.startsWith("insert_frame_") &&
|
||||
!w.name.startsWith("insert_second_") &&
|
||||
!w.name.startsWith("strength_") &&
|
||||
!w.name.startsWith("header_")
|
||||
);
|
||||
} else {
|
||||
this.widgets = [];
|
||||
}
|
||||
|
||||
// 3. Rebuild precisely
|
||||
for (let i = 1; i <= count; i++) {
|
||||
// Header
|
||||
const headerName = `header_${i}`;
|
||||
this.addCustomWidget({
|
||||
name: headerName,
|
||||
type: "text",
|
||||
value: `Image #${i}`,
|
||||
draw(ctx, node, widget_width, y, widget_height) {
|
||||
ctx.save();
|
||||
const margin = 10;
|
||||
const topPadding = 15;
|
||||
ctx.strokeStyle = "#333";
|
||||
ctx.lineWidth = 1;
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(margin, y + 5);
|
||||
ctx.lineTo(widget_width - margin, y + 5);
|
||||
ctx.stroke();
|
||||
ctx.fillStyle = "#dddddd";
|
||||
ctx.font = "bold 12px Arial";
|
||||
ctx.textAlign = "left";
|
||||
ctx.fillText(`Image #${i}`, margin, y + topPadding + 10);
|
||||
ctx.restore();
|
||||
},
|
||||
computeSize(width) { return [width, 35]; }
|
||||
});
|
||||
|
||||
// Helper to add widget with full sync
|
||||
const addSyncedWidget = (type, name, def, options) => {
|
||||
const saved = this.properties[name];
|
||||
return this.addWidget(type, name, saved !== undefined ? saved : def, (val) => {
|
||||
this.properties[name] = val;
|
||||
syncFullStateAcrossNodes(this);
|
||||
}, options);
|
||||
};
|
||||
|
||||
addSyncedWidget("number", `insert_frame_${i}`, 0, { min: -9999, max: 9999, step: 10, precision: 0 });
|
||||
addSyncedWidget("number", `insert_second_${i}`, 0.0, { min: 0.0, max: 9999.0, step: 0.1, precision: 2 });
|
||||
addSyncedWidget("number", `strength_${i}`, 1.0, { min: 0.0, max: 1.0, step: 0.01 });
|
||||
}
|
||||
|
||||
this._updateVisibility();
|
||||
this.setDirtyCanvas(true, true);
|
||||
requestAnimationFrame(() => {
|
||||
if (this.computeSize) {
|
||||
this.setSize(this.computeSize());
|
||||
this.size[0] = initialWidth;
|
||||
}
|
||||
});
|
||||
};
|
||||
|
||||
const origConfigure = node.configure;
|
||||
node.configure = function(info) {
|
||||
if (origConfigure) origConfigure.apply(this, arguments);
|
||||
if (this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) {
|
||||
if (w.type !== "button") this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
node.onConfigure = function(info) {
|
||||
if (info.properties) {
|
||||
this.properties = { ...this.properties, ...info.properties };
|
||||
}
|
||||
this._hookStaticWidgets();
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
let targetCount = count !== null ? count : (this.properties.num_images || 0);
|
||||
this._applyWidgetCount(targetCount);
|
||||
this._updateVisibility();
|
||||
}, 100);
|
||||
};
|
||||
|
||||
const originalOnSerialize = node.onSerialize;
|
||||
node.onSerialize = function(info) {
|
||||
if (this.widgets) {
|
||||
this.widgets.forEach(w => {
|
||||
if (w.name.startsWith("insert_") || w.name.startsWith("strength_") || ["num_images", "insert_mode", "frame_rate"].includes(w.name)) {
|
||||
if (w.type !== "hidden" && w.type !== "button") this.properties[w.name] = w.value;
|
||||
}
|
||||
});
|
||||
}
|
||||
if (originalOnSerialize) originalOnSerialize.apply(this, arguments);
|
||||
info.properties = { ...this.properties };
|
||||
|
||||
const strictArray = [];
|
||||
strictArray.push(this.properties["num_images"] !== undefined ? this.properties["num_images"] : 1);
|
||||
strictArray.push(this.properties["insert_mode"] !== undefined ? this.properties["insert_mode"] : "frames");
|
||||
strictArray.push(this.properties["frame_rate"] !== undefined ? this.properties["frame_rate"] : 24);
|
||||
|
||||
for (let i = 1; i <= 50; i++) {
|
||||
strictArray.push(this.properties[`insert_frame_${i}`] !== undefined ? this.properties[`insert_frame_${i}`] : 0);
|
||||
strictArray.push(this.properties[`insert_second_${i}`] !== undefined ? this.properties[`insert_second_${i}`] : 0.0);
|
||||
strictArray.push(this.properties[`strength_${i}`] !== undefined ? this.properties[`strength_${i}`] : 1.0);
|
||||
}
|
||||
info.widgets_values = strictArray;
|
||||
};
|
||||
|
||||
function readSourceImageCount(self) {
|
||||
const multiInput = self.inputs?.find(inp => inp.name === "multi_input");
|
||||
if (!multiInput || !multiInput.link) return null;
|
||||
const nodeGraph = self.graph || app.graph;
|
||||
|
||||
function traceUpstream(graph, linkId, visited = new Set()) {
|
||||
if (!linkId || visited.has(linkId)) return null;
|
||||
visited.add(linkId);
|
||||
const link = graph.links[linkId];
|
||||
if (!link) return null;
|
||||
const originNode = graph.getNodeById(link.origin_id);
|
||||
if (!originNode) return null;
|
||||
if (originNode.comfyClass === "MultiImageLoader") return originNode;
|
||||
if (originNode.type === "Reroute" || originNode.comfyClass === "Reroute") {
|
||||
if (originNode.inputs?.[0]?.link) return traceUpstream(graph, originNode.inputs[0].link, visited);
|
||||
}
|
||||
if (typeof originNode.getInnerNode === "function") {
|
||||
try {
|
||||
const innerNode = originNode.getInnerNode(link.origin_slot);
|
||||
if (innerNode?.comfyClass === "MultiImageLoader") return innerNode;
|
||||
} catch (e) {}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
let sourceNode = traceUpstream(nodeGraph, multiInput.link);
|
||||
function getCountFromNode(n) {
|
||||
if (typeof n._imageCount === "number") return n._imageCount;
|
||||
const pathsWidget = n.widgets?.find(w => w.name === "image_paths");
|
||||
return pathsWidget ? (pathsWidget.value || "").split('\n').filter(p => p.trim()).length : null;
|
||||
}
|
||||
|
||||
if (sourceNode) return getCountFromNode(sourceNode);
|
||||
|
||||
let multiImageLoaders = [];
|
||||
function findAllLoaders(nodes) {
|
||||
if (!nodes) return;
|
||||
for (let n of nodes) {
|
||||
if (n.comfyClass === "MultiImageLoader") multiImageLoaders.push(n);
|
||||
if (n.subgraph?._nodes) findAllLoaders(n.subgraph._nodes);
|
||||
}
|
||||
}
|
||||
if (app.graph?._nodes) findAllLoaders(app.graph._nodes);
|
||||
if (multiImageLoaders.length === 1) return getCountFromNode(multiImageLoaders[0]);
|
||||
return null;
|
||||
}
|
||||
|
||||
const pollInterval = setInterval(() => {
|
||||
if (!node.graph) {
|
||||
clearInterval(pollInterval);
|
||||
return;
|
||||
}
|
||||
const count = readSourceImageCount(node);
|
||||
if (count !== null && count !== node._currentImageCount) {
|
||||
node._applyWidgetCount(count);
|
||||
syncFullStateAcrossNodes(node); // Sync the new count to others
|
||||
}
|
||||
}, 500);
|
||||
|
||||
const origOnRemoved = node.onRemoved;
|
||||
node.onRemoved = function() {
|
||||
window._LTXSequencerGlobalNodes.delete(node);
|
||||
clearInterval(pollInterval);
|
||||
if (origOnRemoved) origOnRemoved.apply(this, arguments);
|
||||
};
|
||||
|
||||
node.onConnectionsChange = function(type, index, connected) {
|
||||
if (type === 1 && this.inputs[index]?.name === "multi_input") {
|
||||
if (connected) {
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
|
||||
}, 100);
|
||||
} else {
|
||||
this._applyWidgetCount(0);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
node.onAdded = function() {
|
||||
setTimeout(() => {
|
||||
const count = readSourceImageCount(this);
|
||||
this._applyWidgetCount(count !== null ? count : (this.properties.num_images || 0));
|
||||
}, 100);
|
||||
};
|
||||
}
|
||||
});
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,336 +1,336 @@
|
||||
import { app } from "../../scripts/app.js";
|
||||
import { ComfyWidgets } from "../../scripts/widgets.js";
|
||||
|
||||
// 1. DEFINE CSS STYLES GLOBALLY ONCE
|
||||
// We use DOM elements instead of Canvas drawing so it works flawlessly in ComfyUI V3
|
||||
const cssStyles = `
|
||||
.slc-ui-container {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
min-height: 260px; /* Increased from 220px to prevent V1 cropping */
|
||||
background-color: rgba(15, 15, 19, 0.7);
|
||||
border: 1px solid rgba(255, 255, 255, 0.1);
|
||||
border-radius: 8px;
|
||||
padding: 12px;
|
||||
box-sizing: border-box;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
font-family: sans-serif;
|
||||
color: #ffffff;
|
||||
overflow: hidden;
|
||||
pointer-events: auto; /* allows text selection */
|
||||
}
|
||||
.slc-empty {
|
||||
flex: 1;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
}
|
||||
.slc-empty-title { color: #aaaaaa; font-size: 14px; font-weight: 500; margin-bottom: 4px; }
|
||||
.slc-empty-sub { color: #777777; font-size: 11px; }
|
||||
.slc-headers {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
.slc-header-block {
|
||||
flex: 1;
|
||||
background-color: rgba(0, 0, 0, 0.4);
|
||||
border-radius: 6px;
|
||||
padding: 8px 0;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
}
|
||||
.slc-header-title { color: #888888; font-size: 9px; font-weight: bold; margin-bottom: 4px; }
|
||||
.slc-header-value { color: #ffffff; font-size: 16px; font-weight: bold; }
|
||||
.slc-cards {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 8px;
|
||||
}
|
||||
.slc-card {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
background-color: rgba(0, 0, 0, 0.25);
|
||||
border: 1px solid rgba(255, 255, 255, 0.05);
|
||||
border-radius: 4px;
|
||||
padding: 8px 12px 8px 16px;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
}
|
||||
.slc-card::before {
|
||||
content: '';
|
||||
position: absolute;
|
||||
left: 0;
|
||||
top: 0;
|
||||
bottom: 0;
|
||||
width: 4px;
|
||||
}
|
||||
.slc-card.slow::before { background-color: #93c5fd; }
|
||||
.slc-card.avg::before { background-color: #86efac; }
|
||||
.slc-card.fast::before { background-color: #fca5a5; }
|
||||
|
||||
.slc-card-left { display: flex; flex-direction: column; }
|
||||
.slc-card-speed { font-size: 12px; font-weight: bold; margin-bottom: 2px; }
|
||||
.slc-card.slow .slc-card-speed { color: #93c5fd; }
|
||||
.slc-card.avg .slc-card-speed { color: #86efac; }
|
||||
.slc-card.fast .slc-card-speed { color: #fca5a5; }
|
||||
.slc-card-wpm { font-size: 10px; color: #aaaaaa; }
|
||||
|
||||
.slc-card-right { display: flex; flex-direction: column; align-items: flex-end; }
|
||||
.slc-card-time { font-size: 13px; font-weight: bold; color: #ffffff; margin-bottom: 2px; }
|
||||
.slc-card-frames { font-size: 11px; font-family: monospace; color: #888888; }
|
||||
|
||||
.slc-legend {
|
||||
margin-top: auto;
|
||||
padding-top: 8px;
|
||||
color: #999999;
|
||||
font-size: 10px;
|
||||
font-weight: 500;
|
||||
}
|
||||
`;
|
||||
|
||||
// Inject styles to document head safely
|
||||
if (!document.getElementById("speech-length-calculator-styles")) {
|
||||
const styleEl = document.createElement("style");
|
||||
styleEl.id = "speech-length-calculator-styles";
|
||||
styleEl.textContent = cssStyles;
|
||||
document.head.appendChild(styleEl);
|
||||
}
|
||||
|
||||
// 2. HELPER TO GENERATE THE HTML CONTENT
|
||||
function buildUIHTML(statsData) {
|
||||
if (!statsData || statsData.empty) {
|
||||
return `
|
||||
<div class="slc-empty">
|
||||
<div class="slc-empty-title">Awaiting Script</div>
|
||||
<div class="slc-empty-sub">Wrap spoken text inside "quotes"</div>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
|
||||
return `
|
||||
<div class="slc-headers">
|
||||
<div class="slc-header-block">
|
||||
<div class="slc-header-title">SPOKEN WORDS</div>
|
||||
<div class="slc-header-value">${statsData.wordCount}</div>
|
||||
</div>
|
||||
<div class="slc-header-block">
|
||||
<div class="slc-header-title">ADDED TIME</div>
|
||||
<div class="slc-header-value">${statsData.additionalTime}s</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="slc-cards">
|
||||
<div class="slc-card slow">
|
||||
<div class="slc-card-left">
|
||||
<div class="slc-card-speed">SLOW</div>
|
||||
<div class="slc-card-wpm">100 WPM</div>
|
||||
</div>
|
||||
<div class="slc-card-right">
|
||||
<div class="slc-card-time">${statsData.slow.time}</div>
|
||||
<div class="slc-card-frames">${statsData.slow.frames} frames</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="slc-card avg">
|
||||
<div class="slc-card-left">
|
||||
<div class="slc-card-speed">AVG</div>
|
||||
<div class="slc-card-wpm">130 WPM</div>
|
||||
</div>
|
||||
<div class="slc-card-right">
|
||||
<div class="slc-card-time">${statsData.avg.time}</div>
|
||||
<div class="slc-card-frames">${statsData.avg.frames} frames</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="slc-card fast">
|
||||
<div class="slc-card-left">
|
||||
<div class="slc-card-speed">FAST</div>
|
||||
<div class="slc-card-wpm">160 WPM</div>
|
||||
</div>
|
||||
<div class="slc-card-right">
|
||||
<div class="slc-card-time">${statsData.fast.time}</div>
|
||||
<div class="slc-card-frames">${statsData.fast.frames} frames</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="slc-legend">WPM = Words Per Minute</div>
|
||||
`;
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "Comfy.SpeechLengthCalculator",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData, app) {
|
||||
if (nodeData.name === "SpeechLengthCalculator") {
|
||||
|
||||
const onNodeCreated = nodeType.prototype.onNodeCreated;
|
||||
nodeType.prototype.onNodeCreated = function () {
|
||||
const r = onNodeCreated ? onNodeCreated.apply(this, arguments) : undefined;
|
||||
|
||||
// 3. CREATE THE DOM WIDGET
|
||||
const uiContainer = document.createElement("div");
|
||||
uiContainer.className = "slc-ui-container";
|
||||
uiContainer.innerHTML = buildUIHTML({ empty: true });
|
||||
|
||||
// Attach to node: standard addDOMWidget works natively in both V1 and V3 Frontends
|
||||
let statsWidget;
|
||||
if (typeof this.addDOMWidget === "function") {
|
||||
statsWidget = this.addDOMWidget("Stats", "HTML", uiContainer, {
|
||||
serialize: false,
|
||||
hideOnZoom: false
|
||||
});
|
||||
} else {
|
||||
// Fallback for very old unpatched V1 installations
|
||||
statsWidget = ComfyWidgets["STRING"](this, "Stats", ["STRING", { multiline: true }], app).widget;
|
||||
if (statsWidget.inputEl) {
|
||||
statsWidget.inputEl.style.display = "none";
|
||||
if (statsWidget.inputEl.parentNode) {
|
||||
statsWidget.inputEl.parentNode.appendChild(uiContainer);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Tell LiteGraph exactly how much vertical space to hold for our UI
|
||||
statsWidget.computeSize = function() {
|
||||
return [0, 280]; // Increased from 240 to prevent bottom cropping in V1
|
||||
};
|
||||
|
||||
// Ensure the node is wide enough on initial creation
|
||||
requestAnimationFrame(() => {
|
||||
const minWidth = 340;
|
||||
if (this.size[0] < minWidth) this.size[0] = minWidth;
|
||||
|
||||
// Fine tune initial height to account for the larger computeSize and make text box taller
|
||||
if (this.size[1] < 500) this.size[1] = 500;
|
||||
|
||||
this.setDirtyCanvas(true, true);
|
||||
});
|
||||
|
||||
// Fetch text from input link OR widget
|
||||
this._getCurrentText = () => {
|
||||
const inputSlot = this.inputs && this.inputs.find(i => i.name === "text_input" || (i.name === "text" && i.link));
|
||||
if (inputSlot && inputSlot.link) {
|
||||
const link = app.graph.links[inputSlot.link];
|
||||
if (link) {
|
||||
const sourceNode = app.graph.getNodeById(link.origin_id);
|
||||
if (sourceNode && sourceNode.widgets) {
|
||||
const w = sourceNode.widgets.find(w => w.name === "value" || w.name === "text" || w.name === "Text" || w.type === "customtext" || w.type === "STRING");
|
||||
if (w && typeof w.value === "string") return w.value;
|
||||
}
|
||||
}
|
||||
}
|
||||
const textWidget = this.widgets && this.widgets.find(w => w.name === "text");
|
||||
return textWidget ? (textWidget.value || "") : "";
|
||||
};
|
||||
|
||||
this._lastState = { text: null, fps: null, addTime: null };
|
||||
|
||||
const updateStats = () => {
|
||||
const fpsWidget = this.widgets && this.widgets.find(w => w.name === "fps");
|
||||
const additionalTimeWidget = this.widgets && this.widgets.find(w => w.name === "additional_time");
|
||||
|
||||
if (!fpsWidget) return;
|
||||
|
||||
const text = this._getCurrentText();
|
||||
const fps = fpsWidget.value || 24;
|
||||
const additionalTime = additionalTimeWidget ? parseFloat(additionalTimeWidget.value) || 0 : 0;
|
||||
|
||||
// Skip expensive calculations if nothing changed
|
||||
if (this._lastState.text === text &&
|
||||
this._lastState.fps === fps &&
|
||||
this._lastState.addTime === additionalTime) {
|
||||
return;
|
||||
}
|
||||
|
||||
this._lastState.text = text;
|
||||
this._lastState.fps = fps;
|
||||
this._lastState.addTime = additionalTime;
|
||||
|
||||
const regex = /"([^"]*)"|'([^']*)'|“([^”]*)”|‘([^’]*)’/g;
|
||||
let match;
|
||||
let quotedText = "";
|
||||
while ((match = regex.exec(text)) !== null) {
|
||||
quotedText += (match[1] || match[2] || match[3] || match[4] || "") + " ";
|
||||
}
|
||||
|
||||
const words = quotedText.trim().split(/\s+/).filter(w => w.length > 0);
|
||||
const wordCount = words.length;
|
||||
|
||||
const formatTime = (wpm) => {
|
||||
const baseMins = wordCount / wpm;
|
||||
const totalSecs = (baseMins * 60) + additionalTime;
|
||||
|
||||
const mins = Math.floor(totalSecs / 60);
|
||||
let secs = totalSecs % 60;
|
||||
secs = Math.ceil(secs * 10) / 10;
|
||||
const frames = Math.ceil(totalSecs * fps);
|
||||
|
||||
const secsStr = secs.toFixed(1);
|
||||
const timeStr = mins > 0 ? `${mins}m ${secsStr}s` : `${secsStr}s`;
|
||||
|
||||
return {
|
||||
time: timeStr,
|
||||
frames: frames.toString()
|
||||
};
|
||||
};
|
||||
|
||||
const statsData = {
|
||||
empty: (wordCount === 0 && additionalTime === 0),
|
||||
wordCount: wordCount,
|
||||
additionalTime: additionalTime,
|
||||
slow: formatTime(100),
|
||||
avg: formatTime(130),
|
||||
fast: formatTime(160)
|
||||
};
|
||||
|
||||
// 4. INJECT HTML TO THE DOM
|
||||
// Replaces the heavy canvas redrawing!
|
||||
uiContainer.innerHTML = buildUIHTML(statsData);
|
||||
|
||||
this.setDirtyCanvas(true, false);
|
||||
};
|
||||
|
||||
// We still use onDrawBackground as an efficient silent trigger
|
||||
// to auto-update in case upstream nodes silently change their text
|
||||
const onDrawBackground = this.onDrawBackground;
|
||||
this.onDrawBackground = function(ctx) {
|
||||
if (onDrawBackground) onDrawBackground.apply(this, arguments);
|
||||
updateStats();
|
||||
};
|
||||
|
||||
// Bind events to update in real time based on node interactions
|
||||
setTimeout(() => {
|
||||
const textWidget = this.widgets && this.widgets.find(w => w.name === "text");
|
||||
const fpsWidget = this.widgets && this.widgets.find(w => w.name === "fps");
|
||||
const additionalTimeWidget = this.widgets && this.widgets.find(w => w.name === "additional_time");
|
||||
|
||||
if (textWidget) {
|
||||
const origCallback = textWidget.callback;
|
||||
textWidget.callback = function() {
|
||||
if (origCallback) origCallback.apply(this, arguments);
|
||||
updateStats();
|
||||
}
|
||||
}
|
||||
if (fpsWidget) {
|
||||
const origFpsCallback = fpsWidget.callback;
|
||||
fpsWidget.callback = function() {
|
||||
if (origFpsCallback) origFpsCallback.apply(this, arguments);
|
||||
updateStats();
|
||||
}
|
||||
}
|
||||
if (additionalTimeWidget) {
|
||||
const origAddCallback = additionalTimeWidget.callback;
|
||||
additionalTimeWidget.callback = function() {
|
||||
if (origAddCallback) origAddCallback.apply(this, arguments);
|
||||
updateStats();
|
||||
}
|
||||
}
|
||||
updateStats();
|
||||
}, 100);
|
||||
|
||||
return r;
|
||||
};
|
||||
}
|
||||
}
|
||||
import { app } from "../../scripts/app.js";
|
||||
import { ComfyWidgets } from "../../scripts/widgets.js";
|
||||
|
||||
// 1. DEFINE CSS STYLES GLOBALLY ONCE
|
||||
// We use DOM elements instead of Canvas drawing so it works flawlessly in ComfyUI V3
|
||||
const cssStyles = `
|
||||
.slc-ui-container {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
min-height: 260px; /* Increased from 220px to prevent V1 cropping */
|
||||
background-color: rgba(15, 15, 19, 0.7);
|
||||
border: 1px solid rgba(255, 255, 255, 0.1);
|
||||
border-radius: 8px;
|
||||
padding: 12px;
|
||||
box-sizing: border-box;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
font-family: sans-serif;
|
||||
color: #ffffff;
|
||||
overflow: hidden;
|
||||
pointer-events: auto; /* allows text selection */
|
||||
}
|
||||
.slc-empty {
|
||||
flex: 1;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
}
|
||||
.slc-empty-title { color: #aaaaaa; font-size: 14px; font-weight: 500; margin-bottom: 4px; }
|
||||
.slc-empty-sub { color: #777777; font-size: 11px; }
|
||||
.slc-headers {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
.slc-header-block {
|
||||
flex: 1;
|
||||
background-color: rgba(0, 0, 0, 0.4);
|
||||
border-radius: 6px;
|
||||
padding: 8px 0;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
}
|
||||
.slc-header-title { color: #888888; font-size: 9px; font-weight: bold; margin-bottom: 4px; }
|
||||
.slc-header-value { color: #ffffff; font-size: 16px; font-weight: bold; }
|
||||
.slc-cards {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 8px;
|
||||
}
|
||||
.slc-card {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
background-color: rgba(0, 0, 0, 0.25);
|
||||
border: 1px solid rgba(255, 255, 255, 0.05);
|
||||
border-radius: 4px;
|
||||
padding: 8px 12px 8px 16px;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
}
|
||||
.slc-card::before {
|
||||
content: '';
|
||||
position: absolute;
|
||||
left: 0;
|
||||
top: 0;
|
||||
bottom: 0;
|
||||
width: 4px;
|
||||
}
|
||||
.slc-card.slow::before { background-color: #93c5fd; }
|
||||
.slc-card.avg::before { background-color: #86efac; }
|
||||
.slc-card.fast::before { background-color: #fca5a5; }
|
||||
|
||||
.slc-card-left { display: flex; flex-direction: column; }
|
||||
.slc-card-speed { font-size: 12px; font-weight: bold; margin-bottom: 2px; }
|
||||
.slc-card.slow .slc-card-speed { color: #93c5fd; }
|
||||
.slc-card.avg .slc-card-speed { color: #86efac; }
|
||||
.slc-card.fast .slc-card-speed { color: #fca5a5; }
|
||||
.slc-card-wpm { font-size: 10px; color: #aaaaaa; }
|
||||
|
||||
.slc-card-right { display: flex; flex-direction: column; align-items: flex-end; }
|
||||
.slc-card-time { font-size: 13px; font-weight: bold; color: #ffffff; margin-bottom: 2px; }
|
||||
.slc-card-frames { font-size: 11px; font-family: monospace; color: #888888; }
|
||||
|
||||
.slc-legend {
|
||||
margin-top: auto;
|
||||
padding-top: 8px;
|
||||
color: #999999;
|
||||
font-size: 10px;
|
||||
font-weight: 500;
|
||||
}
|
||||
`;
|
||||
|
||||
// Inject styles to document head safely
|
||||
if (!document.getElementById("speech-length-calculator-styles")) {
|
||||
const styleEl = document.createElement("style");
|
||||
styleEl.id = "speech-length-calculator-styles";
|
||||
styleEl.textContent = cssStyles;
|
||||
document.head.appendChild(styleEl);
|
||||
}
|
||||
|
||||
// 2. HELPER TO GENERATE THE HTML CONTENT
|
||||
function buildUIHTML(statsData) {
|
||||
if (!statsData || statsData.empty) {
|
||||
return `
|
||||
<div class="slc-empty">
|
||||
<div class="slc-empty-title">Awaiting Script</div>
|
||||
<div class="slc-empty-sub">Wrap spoken text inside "quotes"</div>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
|
||||
return `
|
||||
<div class="slc-headers">
|
||||
<div class="slc-header-block">
|
||||
<div class="slc-header-title">SPOKEN WORDS</div>
|
||||
<div class="slc-header-value">${statsData.wordCount}</div>
|
||||
</div>
|
||||
<div class="slc-header-block">
|
||||
<div class="slc-header-title">ADDED TIME</div>
|
||||
<div class="slc-header-value">${statsData.additionalTime}s</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="slc-cards">
|
||||
<div class="slc-card slow">
|
||||
<div class="slc-card-left">
|
||||
<div class="slc-card-speed">SLOW</div>
|
||||
<div class="slc-card-wpm">100 WPM</div>
|
||||
</div>
|
||||
<div class="slc-card-right">
|
||||
<div class="slc-card-time">${statsData.slow.time}</div>
|
||||
<div class="slc-card-frames">${statsData.slow.frames} frames</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="slc-card avg">
|
||||
<div class="slc-card-left">
|
||||
<div class="slc-card-speed">AVG</div>
|
||||
<div class="slc-card-wpm">130 WPM</div>
|
||||
</div>
|
||||
<div class="slc-card-right">
|
||||
<div class="slc-card-time">${statsData.avg.time}</div>
|
||||
<div class="slc-card-frames">${statsData.avg.frames} frames</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="slc-card fast">
|
||||
<div class="slc-card-left">
|
||||
<div class="slc-card-speed">FAST</div>
|
||||
<div class="slc-card-wpm">160 WPM</div>
|
||||
</div>
|
||||
<div class="slc-card-right">
|
||||
<div class="slc-card-time">${statsData.fast.time}</div>
|
||||
<div class="slc-card-frames">${statsData.fast.frames} frames</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="slc-legend">WPM = Words Per Minute</div>
|
||||
`;
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "Comfy.SpeechLengthCalculator",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData, app) {
|
||||
if (nodeData.name === "SpeechLengthCalculator") {
|
||||
|
||||
const onNodeCreated = nodeType.prototype.onNodeCreated;
|
||||
nodeType.prototype.onNodeCreated = function () {
|
||||
const r = onNodeCreated ? onNodeCreated.apply(this, arguments) : undefined;
|
||||
|
||||
// 3. CREATE THE DOM WIDGET
|
||||
const uiContainer = document.createElement("div");
|
||||
uiContainer.className = "slc-ui-container";
|
||||
uiContainer.innerHTML = buildUIHTML({ empty: true });
|
||||
|
||||
// Attach to node: standard addDOMWidget works natively in both V1 and V3 Frontends
|
||||
let statsWidget;
|
||||
if (typeof this.addDOMWidget === "function") {
|
||||
statsWidget = this.addDOMWidget("Stats", "HTML", uiContainer, {
|
||||
serialize: false,
|
||||
hideOnZoom: false
|
||||
});
|
||||
} else {
|
||||
// Fallback for very old unpatched V1 installations
|
||||
statsWidget = ComfyWidgets["STRING"](this, "Stats", ["STRING", { multiline: true }], app).widget;
|
||||
if (statsWidget.inputEl) {
|
||||
statsWidget.inputEl.style.display = "none";
|
||||
if (statsWidget.inputEl.parentNode) {
|
||||
statsWidget.inputEl.parentNode.appendChild(uiContainer);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Tell LiteGraph exactly how much vertical space to hold for our UI
|
||||
statsWidget.computeSize = function() {
|
||||
return [0, 280]; // Increased from 240 to prevent bottom cropping in V1
|
||||
};
|
||||
|
||||
// Ensure the node is wide enough on initial creation
|
||||
requestAnimationFrame(() => {
|
||||
const minWidth = 340;
|
||||
if (this.size[0] < minWidth) this.size[0] = minWidth;
|
||||
|
||||
// Fine tune initial height to account for the larger computeSize and make text box taller
|
||||
if (this.size[1] < 500) this.size[1] = 500;
|
||||
|
||||
this.setDirtyCanvas(true, true);
|
||||
});
|
||||
|
||||
// Fetch text from input link OR widget
|
||||
this._getCurrentText = () => {
|
||||
const inputSlot = this.inputs && this.inputs.find(i => i.name === "text_input" || (i.name === "text" && i.link));
|
||||
if (inputSlot && inputSlot.link) {
|
||||
const link = app.graph.links[inputSlot.link];
|
||||
if (link) {
|
||||
const sourceNode = app.graph.getNodeById(link.origin_id);
|
||||
if (sourceNode && sourceNode.widgets) {
|
||||
const w = sourceNode.widgets.find(w => w.name === "value" || w.name === "text" || w.name === "Text" || w.type === "customtext" || w.type === "STRING");
|
||||
if (w && typeof w.value === "string") return w.value;
|
||||
}
|
||||
}
|
||||
}
|
||||
const textWidget = this.widgets && this.widgets.find(w => w.name === "text");
|
||||
return textWidget ? (textWidget.value || "") : "";
|
||||
};
|
||||
|
||||
this._lastState = { text: null, fps: null, addTime: null };
|
||||
|
||||
const updateStats = () => {
|
||||
const fpsWidget = this.widgets && this.widgets.find(w => w.name === "fps");
|
||||
const additionalTimeWidget = this.widgets && this.widgets.find(w => w.name === "additional_time");
|
||||
|
||||
if (!fpsWidget) return;
|
||||
|
||||
const text = this._getCurrentText();
|
||||
const fps = fpsWidget.value || 24;
|
||||
const additionalTime = additionalTimeWidget ? parseFloat(additionalTimeWidget.value) || 0 : 0;
|
||||
|
||||
// Skip expensive calculations if nothing changed
|
||||
if (this._lastState.text === text &&
|
||||
this._lastState.fps === fps &&
|
||||
this._lastState.addTime === additionalTime) {
|
||||
return;
|
||||
}
|
||||
|
||||
this._lastState.text = text;
|
||||
this._lastState.fps = fps;
|
||||
this._lastState.addTime = additionalTime;
|
||||
|
||||
const regex = /"([^"]*)"|'([^']*)'|“([^”]*)”|‘([^’]*)’/g;
|
||||
let match;
|
||||
let quotedText = "";
|
||||
while ((match = regex.exec(text)) !== null) {
|
||||
quotedText += (match[1] || match[2] || match[3] || match[4] || "") + " ";
|
||||
}
|
||||
|
||||
const words = quotedText.trim().split(/\s+/).filter(w => w.length > 0);
|
||||
const wordCount = words.length;
|
||||
|
||||
const formatTime = (wpm) => {
|
||||
const baseMins = wordCount / wpm;
|
||||
const totalSecs = (baseMins * 60) + additionalTime;
|
||||
|
||||
const mins = Math.floor(totalSecs / 60);
|
||||
let secs = totalSecs % 60;
|
||||
secs = Math.ceil(secs * 10) / 10;
|
||||
const frames = Math.ceil(totalSecs * fps);
|
||||
|
||||
const secsStr = secs.toFixed(1);
|
||||
const timeStr = mins > 0 ? `${mins}m ${secsStr}s` : `${secsStr}s`;
|
||||
|
||||
return {
|
||||
time: timeStr,
|
||||
frames: frames.toString()
|
||||
};
|
||||
};
|
||||
|
||||
const statsData = {
|
||||
empty: (wordCount === 0 && additionalTime === 0),
|
||||
wordCount: wordCount,
|
||||
additionalTime: additionalTime,
|
||||
slow: formatTime(100),
|
||||
avg: formatTime(130),
|
||||
fast: formatTime(160)
|
||||
};
|
||||
|
||||
// 4. INJECT HTML TO THE DOM
|
||||
// Replaces the heavy canvas redrawing!
|
||||
uiContainer.innerHTML = buildUIHTML(statsData);
|
||||
|
||||
this.setDirtyCanvas(true, false);
|
||||
};
|
||||
|
||||
// We still use onDrawBackground as an efficient silent trigger
|
||||
// to auto-update in case upstream nodes silently change their text
|
||||
const onDrawBackground = this.onDrawBackground;
|
||||
this.onDrawBackground = function(ctx) {
|
||||
if (onDrawBackground) onDrawBackground.apply(this, arguments);
|
||||
updateStats();
|
||||
};
|
||||
|
||||
// Bind events to update in real time based on node interactions
|
||||
setTimeout(() => {
|
||||
const textWidget = this.widgets && this.widgets.find(w => w.name === "text");
|
||||
const fpsWidget = this.widgets && this.widgets.find(w => w.name === "fps");
|
||||
const additionalTimeWidget = this.widgets && this.widgets.find(w => w.name === "additional_time");
|
||||
|
||||
if (textWidget) {
|
||||
const origCallback = textWidget.callback;
|
||||
textWidget.callback = function() {
|
||||
if (origCallback) origCallback.apply(this, arguments);
|
||||
updateStats();
|
||||
}
|
||||
}
|
||||
if (fpsWidget) {
|
||||
const origFpsCallback = fpsWidget.callback;
|
||||
fpsWidget.callback = function() {
|
||||
if (origFpsCallback) origFpsCallback.apply(this, arguments);
|
||||
updateStats();
|
||||
}
|
||||
}
|
||||
if (additionalTimeWidget) {
|
||||
const origAddCallback = additionalTimeWidget.callback;
|
||||
additionalTimeWidget.callback = function() {
|
||||
if (origAddCallback) origAddCallback.apply(this, arguments);
|
||||
updateStats();
|
||||
}
|
||||
}
|
||||
updateStats();
|
||||
}, 100);
|
||||
|
||||
return r;
|
||||
};
|
||||
}
|
||||
}
|
||||
});
|
||||
75
latent_slice.py
Normal file
75
latent_slice.py
Normal file
@@ -0,0 +1,75 @@
|
||||
# --- START OF FILE latent_slice.py ---
|
||||
|
||||
import torch
|
||||
|
||||
class CleanLatentSlice:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"latent": ("LATENT",),
|
||||
"start": ("INT", {"default": 0, "min": 0, "max": 100000, "step": 1, "tooltip": "The starting frame index to slice from (plug 'latent_start_index' here)."}),
|
||||
"length": ("INT", {"default": 1, "min": 1, "max": 100000, "step": 1, "tooltip": "The number of frames to keep (plug 'clean_latent_frames' here)."}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("LATENT",)
|
||||
RETURN_NAMES = ("latent",)
|
||||
FUNCTION = "slice_latent"
|
||||
CATEGORY = "WhatDreamsCost"
|
||||
DESCRIPTION = "Safely slices a video latent starting from an offset index for a specific length. Uses torch.narrow to bypass PyTorch NestedTensor slicing bugs."
|
||||
|
||||
def slice_latent(self, latent, start, length):
|
||||
new_latent = latent.copy()
|
||||
|
||||
def safe_slice(tensor, target_start, target_len):
|
||||
dims = tensor.ndim if hasattr(tensor, "ndim") else len(tensor.shape)
|
||||
|
||||
# Constrain starting index and length to the actual size of the tensor
|
||||
max_size = tensor.size(2) if dims == 5 else tensor.size(0)
|
||||
actual_start = min(target_start, max_size - 1) if max_size > 0 else 0
|
||||
actual_len = min(target_len, max_size - actual_start)
|
||||
|
||||
try:
|
||||
# torch.narrow is the safest low-level C++ slice method to bypass NestedTensor bugs
|
||||
if dims == 5:
|
||||
# [Batch, Channels, Frames, Height, Width] -> Slice dimension 2
|
||||
return torch.narrow(tensor, 2, actual_start, actual_len)
|
||||
elif dims == 4:
|
||||
# [Frames, Channels, Height, Width] -> Slice dimension 0
|
||||
return torch.narrow(tensor, 0, actual_start, actual_len)
|
||||
elif dims == 3:
|
||||
# [Frames, Height, Width] -> Slice dimension 0
|
||||
return torch.narrow(tensor, 0, actual_start, actual_len)
|
||||
except Exception as e:
|
||||
# Fallback if narrow fails
|
||||
if dims == 5:
|
||||
return tensor[:, :, actual_start : actual_start + actual_len]
|
||||
elif dims == 4:
|
||||
return tensor[actual_start : actual_start + actual_len]
|
||||
elif dims == 3:
|
||||
return tensor[actual_start : actual_start + actual_len]
|
||||
|
||||
return tensor
|
||||
|
||||
# Safely slice video samples
|
||||
if "samples" in new_latent:
|
||||
new_latent["samples"] = safe_slice(new_latent["samples"], start, length)
|
||||
|
||||
# Safely slice video noise mask (if it exists)
|
||||
if "noise_mask" in new_latent:
|
||||
new_latent["noise_mask"] = safe_slice(new_latent["noise_mask"], start, length)
|
||||
|
||||
return (new_latent,)
|
||||
|
||||
|
||||
# Register the node with ComfyUI
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"CleanLatentSlice": CleanLatentSlice
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"CleanLatentSlice": "Clean Latent Slice"
|
||||
}
|
||||
|
||||
__all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS']
|
||||
280
load_audio_ui.py
280
load_audio_ui.py
@@ -1,141 +1,141 @@
|
||||
import folder_paths
|
||||
import os
|
||||
import torch
|
||||
import av
|
||||
|
||||
def f32_pcm(wav: torch.Tensor) -> torch.Tensor:
|
||||
"""Convert audio to float 32 bits PCM format."""
|
||||
if wav.dtype.is_floating_point:
|
||||
return wav
|
||||
elif wav.dtype == torch.int16:
|
||||
return wav.float() / (2 ** 15)
|
||||
elif wav.dtype == torch.int32:
|
||||
return wav.float() / (2 ** 31)
|
||||
raise ValueError(f"Unsupported wav dtype: {wav.dtype}")
|
||||
|
||||
def load_audio_file(filepath: str) -> tuple[torch.Tensor, int]:
|
||||
"""Uses the latest ComfyUI av-based decoding for maximum compatibility."""
|
||||
with av.open(filepath) as af:
|
||||
if not af.streams.audio:
|
||||
raise ValueError("No audio stream found in the file.")
|
||||
|
||||
stream = af.streams.audio[0]
|
||||
sr = stream.codec_context.sample_rate
|
||||
n_channels = stream.channels
|
||||
|
||||
frames = []
|
||||
for frame in af.decode(streams=stream.index):
|
||||
buf = torch.from_numpy(frame.to_ndarray())
|
||||
if buf.shape[0] != n_channels:
|
||||
buf = buf.view(-1, n_channels).t()
|
||||
|
||||
frames.append(buf)
|
||||
|
||||
if not frames:
|
||||
raise ValueError("No audio frames decoded.")
|
||||
|
||||
wav = torch.cat(frames, dim=1)
|
||||
wav = f32_pcm(wav)
|
||||
return wav, sr
|
||||
|
||||
|
||||
class LoadAudioUI:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
try:
|
||||
files = folder_paths.get_filename_list("audio")
|
||||
except:
|
||||
files = []
|
||||
|
||||
if not files:
|
||||
input_dir = folder_paths.get_input_directory()
|
||||
if os.path.exists(input_dir):
|
||||
all_files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))]
|
||||
try:
|
||||
files = sorted(folder_paths.filter_files_content_types(all_files, ["audio", "video"]))
|
||||
except:
|
||||
files = sorted(all_files)
|
||||
|
||||
if not files or len(files) == 0:
|
||||
files = ["none"]
|
||||
|
||||
return {
|
||||
"required": {
|
||||
"audio": (files, {"audio_upload": True}), # Moved to the top so it appears first
|
||||
"start_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
|
||||
"end_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
|
||||
"duration": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
|
||||
},
|
||||
"optional": {
|
||||
"audioUI": ("AUDIO_UI",)
|
||||
}
|
||||
}
|
||||
|
||||
CATEGORY = "WhatDreamsCost"
|
||||
RETURN_TYPES = ("AUDIO", "FLOAT")
|
||||
RETURN_NAMES = ("audio", "duration")
|
||||
FUNCTION = "load_audio"
|
||||
|
||||
@classmethod
|
||||
def VALIDATE_INPUTS(cls, audio, **kwargs):
|
||||
# CRITICAL FIX: This bypasses the "Value not in list" error.
|
||||
# By returning True, we tell ComfyUI to allow the 'audio' value even if it isn't in
|
||||
# the current dropdown list. This allows the execution to reach load_audio(),
|
||||
# where our fallback silence logic can handle the missing file gracefully.
|
||||
return True
|
||||
|
||||
def load_audio(self, audio, start_time, end_time, duration, **kwargs):
|
||||
# Determine the annotated file path if a file is actually selected
|
||||
# We wrap this in a try/except because get_annotated_filepath can fail if
|
||||
# the input string is malformed or doesn't follow expected paths.
|
||||
try:
|
||||
audio_path = folder_paths.get_annotated_filepath(audio) if audio != "none" else ""
|
||||
except:
|
||||
audio_path = ""
|
||||
|
||||
# --- FALLBACK LOGIC ---
|
||||
# If the file is 'none' or doesn't exist on disk, provide 1 second of silence
|
||||
if audio == "none" or not audio_path or not os.path.exists(audio_path):
|
||||
missing_info = audio if audio != "none" else "None selected"
|
||||
print(f"!!! [LoadAudioUI] Warning: Audio file '{missing_info}' not found. Outputting 1 second of silence.")
|
||||
|
||||
sample_rate = 44100
|
||||
# 1 second of silence (stereo) -> shape [channels, time]
|
||||
waveform = torch.zeros((2, 44100))
|
||||
else:
|
||||
try:
|
||||
waveform, sample_rate = load_audio_file(audio_path)
|
||||
except Exception as e:
|
||||
# If decoding fails for any reason, fallback to silence rather than crashing the workflow
|
||||
print(f"!!! [LoadAudioUI] Error decoding {audio}: {e}. Falling back to silence.")
|
||||
sample_rate = 44100
|
||||
waveform = torch.zeros((2, 44100))
|
||||
|
||||
# Convert seconds to frames
|
||||
start_frame = int(start_time * sample_rate)
|
||||
if end_time > 0:
|
||||
end_frame = int(end_time * sample_rate)
|
||||
# Ensure the end_frame does not exceed the actual audio length
|
||||
end_frame = min(end_frame, waveform.shape[1])
|
||||
else:
|
||||
# 0 defaults to the end of the file
|
||||
end_frame = waveform.shape[1]
|
||||
|
||||
# Ensure start frame stays within bounds and doesn't pass the end frame
|
||||
start_frame = min(start_frame, end_frame)
|
||||
|
||||
# Trim the waveform tensor -> shape: [channels, time]
|
||||
trimmed_waveform = waveform[:, start_frame:end_frame]
|
||||
|
||||
# Final safety check: if trimming resulted in zero length, give it a tiny bit of padding
|
||||
# to prevent downstream nodes from crashing on empty tensors
|
||||
if trimmed_waveform.shape[1] == 0:
|
||||
trimmed_waveform = torch.zeros((waveform.shape[0], 1))
|
||||
|
||||
# Format for ComfyUI's standard AUDIO type: [batch, channels, time]
|
||||
audio_output = {"waveform": trimmed_waveform.unsqueeze(0), "sample_rate": sample_rate}
|
||||
|
||||
# Calculate the final trimmed duration in seconds as a float
|
||||
final_duration = float(trimmed_waveform.shape[1] / sample_rate)
|
||||
|
||||
import folder_paths
|
||||
import os
|
||||
import torch
|
||||
import av
|
||||
|
||||
def f32_pcm(wav: torch.Tensor) -> torch.Tensor:
|
||||
"""Convert audio to float 32 bits PCM format."""
|
||||
if wav.dtype.is_floating_point:
|
||||
return wav
|
||||
elif wav.dtype == torch.int16:
|
||||
return wav.float() / (2 ** 15)
|
||||
elif wav.dtype == torch.int32:
|
||||
return wav.float() / (2 ** 31)
|
||||
raise ValueError(f"Unsupported wav dtype: {wav.dtype}")
|
||||
|
||||
def load_audio_file(filepath: str) -> tuple[torch.Tensor, int]:
|
||||
"""Uses the latest ComfyUI av-based decoding for maximum compatibility."""
|
||||
with av.open(filepath) as af:
|
||||
if not af.streams.audio:
|
||||
raise ValueError("No audio stream found in the file.")
|
||||
|
||||
stream = af.streams.audio[0]
|
||||
sr = stream.codec_context.sample_rate
|
||||
n_channels = stream.channels
|
||||
|
||||
frames = []
|
||||
for frame in af.decode(streams=stream.index):
|
||||
buf = torch.from_numpy(frame.to_ndarray())
|
||||
if buf.shape[0] != n_channels:
|
||||
buf = buf.view(-1, n_channels).t()
|
||||
|
||||
frames.append(buf)
|
||||
|
||||
if not frames:
|
||||
raise ValueError("No audio frames decoded.")
|
||||
|
||||
wav = torch.cat(frames, dim=1)
|
||||
wav = f32_pcm(wav)
|
||||
return wav, sr
|
||||
|
||||
|
||||
class LoadAudioUI:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
try:
|
||||
files = folder_paths.get_filename_list("audio")
|
||||
except:
|
||||
files = []
|
||||
|
||||
if not files:
|
||||
input_dir = folder_paths.get_input_directory()
|
||||
if os.path.exists(input_dir):
|
||||
all_files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))]
|
||||
try:
|
||||
files = sorted(folder_paths.filter_files_content_types(all_files, ["audio", "video"]))
|
||||
except:
|
||||
files = sorted(all_files)
|
||||
|
||||
if not files or len(files) == 0:
|
||||
files = ["none"]
|
||||
|
||||
return {
|
||||
"required": {
|
||||
"audio": (files, {"audio_upload": True}), # Moved to the top so it appears first
|
||||
"start_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
|
||||
"end_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
|
||||
"duration": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
|
||||
},
|
||||
"optional": {
|
||||
"audioUI": ("AUDIO_UI",)
|
||||
}
|
||||
}
|
||||
|
||||
CATEGORY = "WhatDreamsCost"
|
||||
RETURN_TYPES = ("AUDIO", "FLOAT")
|
||||
RETURN_NAMES = ("audio", "duration")
|
||||
FUNCTION = "load_audio"
|
||||
|
||||
@classmethod
|
||||
def VALIDATE_INPUTS(cls, audio, **kwargs):
|
||||
# CRITICAL FIX: This bypasses the "Value not in list" error.
|
||||
# By returning True, we tell ComfyUI to allow the 'audio' value even if it isn't in
|
||||
# the current dropdown list. This allows the execution to reach load_audio(),
|
||||
# where our fallback silence logic can handle the missing file gracefully.
|
||||
return True
|
||||
|
||||
def load_audio(self, audio, start_time, end_time, duration, **kwargs):
|
||||
# Determine the annotated file path if a file is actually selected
|
||||
# We wrap this in a try/except because get_annotated_filepath can fail if
|
||||
# the input string is malformed or doesn't follow expected paths.
|
||||
try:
|
||||
audio_path = folder_paths.get_annotated_filepath(audio) if audio != "none" else ""
|
||||
except:
|
||||
audio_path = ""
|
||||
|
||||
# --- FALLBACK LOGIC ---
|
||||
# If the file is 'none' or doesn't exist on disk, provide 1 second of silence
|
||||
if audio == "none" or not audio_path or not os.path.exists(audio_path):
|
||||
missing_info = audio if audio != "none" else "None selected"
|
||||
print(f"!!! [LoadAudioUI] Warning: Audio file '{missing_info}' not found. Outputting 1 second of silence.")
|
||||
|
||||
sample_rate = 44100
|
||||
# 1 second of silence (stereo) -> shape [channels, time]
|
||||
waveform = torch.zeros((2, 44100))
|
||||
else:
|
||||
try:
|
||||
waveform, sample_rate = load_audio_file(audio_path)
|
||||
except Exception as e:
|
||||
# If decoding fails for any reason, fallback to silence rather than crashing the workflow
|
||||
print(f"!!! [LoadAudioUI] Error decoding {audio}: {e}. Falling back to silence.")
|
||||
sample_rate = 44100
|
||||
waveform = torch.zeros((2, 44100))
|
||||
|
||||
# Convert seconds to frames
|
||||
start_frame = int(start_time * sample_rate)
|
||||
if end_time > 0:
|
||||
end_frame = int(end_time * sample_rate)
|
||||
# Ensure the end_frame does not exceed the actual audio length
|
||||
end_frame = min(end_frame, waveform.shape[1])
|
||||
else:
|
||||
# 0 defaults to the end of the file
|
||||
end_frame = waveform.shape[1]
|
||||
|
||||
# Ensure start frame stays within bounds and doesn't pass the end frame
|
||||
start_frame = min(start_frame, end_frame)
|
||||
|
||||
# Trim the waveform tensor -> shape: [channels, time]
|
||||
trimmed_waveform = waveform[:, start_frame:end_frame]
|
||||
|
||||
# Final safety check: if trimming resulted in zero length, give it a tiny bit of padding
|
||||
# to prevent downstream nodes from crashing on empty tensors
|
||||
if trimmed_waveform.shape[1] == 0:
|
||||
trimmed_waveform = torch.zeros((waveform.shape[0], 1))
|
||||
|
||||
# Format for ComfyUI's standard AUDIO type: [batch, channels, time]
|
||||
audio_output = {"waveform": trimmed_waveform.unsqueeze(0), "sample_rate": sample_rate}
|
||||
|
||||
# Calculate the final trimmed duration in seconds as a float
|
||||
final_duration = float(trimmed_waveform.shape[1] / sample_rate)
|
||||
|
||||
return (audio_output, final_duration)
|
||||
822
load_video_ui.py
822
load_video_ui.py
@@ -1,412 +1,412 @@
|
||||
import os
|
||||
import torch
|
||||
import numpy as np
|
||||
import folder_paths
|
||||
import av
|
||||
from server import PromptServer
|
||||
from aiohttp import web
|
||||
import comfy.utils
|
||||
|
||||
# Custom API route to serve video files from anywhere on the user's system for the frontend preview
|
||||
@PromptServer.instance.routes.get("/video_ui_custom_view")
|
||||
async def custom_view(request):
|
||||
file_path = request.query.get("filename", "")
|
||||
if os.path.exists(file_path) and os.path.isfile(file_path):
|
||||
return web.FileResponse(file_path)
|
||||
return web.Response(status=404, text="File not found")
|
||||
|
||||
# Custom API route for Chunked Uploads to bypass the 413 Payload Too Large error
|
||||
@PromptServer.instance.routes.post("/video_ui_upload_chunk")
|
||||
async def upload_chunk(request):
|
||||
post = await request.post()
|
||||
file = post.get("file")
|
||||
filename = post.get("filename")
|
||||
chunk_index = int(post.get("chunk_index"))
|
||||
total_chunks = int(post.get("total_chunks"))
|
||||
|
||||
upload_dir = folder_paths.get_input_directory()
|
||||
file_path = os.path.join(upload_dir, filename)
|
||||
|
||||
# Append to file if it's not the first chunk, otherwise write new
|
||||
mode = "ab" if chunk_index > 0 else "wb"
|
||||
with open(file_path, mode) as f:
|
||||
f.write(file.file.read())
|
||||
|
||||
if chunk_index == total_chunks - 1:
|
||||
return web.json_response({"name": filename})
|
||||
return web.json_response({"status": "ok"})
|
||||
|
||||
|
||||
class LoadVideoUI:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"video": ("STRING", {"default": ""}),
|
||||
"start_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
|
||||
"end_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
|
||||
"duration": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
|
||||
"start_frame": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}),
|
||||
"end_frame": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}),
|
||||
"duration_frames": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}),
|
||||
"resize_method": (["maintain aspect ratio", "stretch to fit", "pad", "crop"], {"default": "maintain aspect ratio"}),
|
||||
"custom_width": ("INT", {"default": 0, "min": 0, "max": 100000, "step": 8, "tooltip": "Custom width. 0 means original width."}),
|
||||
"custom_height": ("INT", {"default": 0, "min": 0, "max": 100000, "step": 8, "tooltip": "Custom height. 0 means original height."}),
|
||||
"frame_rate": ("INT", {"default": 24, "min": 1, "max": 120, "step": 1, "tooltip": "Force the video to a specific frame rate for extraction."}),
|
||||
"display_mode": (["seconds", "frames"], {"default": "seconds"}),
|
||||
"crop_x": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}),
|
||||
"crop_y": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}),
|
||||
"crop_w": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}),
|
||||
"crop_h": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("IMAGE", "AUDIO", "FLOAT", "INT")
|
||||
RETURN_NAMES = ("images", "audio", "duration", "frame_count")
|
||||
FUNCTION = "load_video"
|
||||
CATEGORY = "WhatDreamsCost"
|
||||
|
||||
def load_video(self, video, frame_rate, display_mode, start_time, end_time, duration, start_frame, end_frame, duration_frames, custom_width=0, custom_height=0, resize_method="maintain aspect ratio", crop_x=0.0, crop_y=0.0, crop_w=1.0, crop_h=1.0, **kwargs):
|
||||
if not video:
|
||||
# Return blank defaults if no video is loaded
|
||||
empty_image = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
|
||||
empty_audio = {"waveform": torch.zeros((1, 1, 44100)), "sample_rate": 44100}
|
||||
return (empty_image, empty_audio, 0.0, 0)
|
||||
|
||||
# 1. Resolve path using ComfyUI standard paths or Absolute Path
|
||||
video_path = video # Try exact/absolute path first
|
||||
if not os.path.exists(video_path):
|
||||
video_path_annotated = folder_paths.get_annotated_filepath(video)
|
||||
if os.path.exists(video_path_annotated):
|
||||
video_path = video_path_annotated
|
||||
else:
|
||||
video_path_input = os.path.join(folder_paths.get_input_directory(), video)
|
||||
if os.path.exists(video_path_input):
|
||||
video_path = video_path_input
|
||||
else:
|
||||
raise FileNotFoundError(f"Video file not found: {video}")
|
||||
|
||||
# Open container to read streams and metadata
|
||||
container = av.open(video_path)
|
||||
|
||||
# Determine video stream and duration
|
||||
video_stream = container.streams.video[0] if len(container.streams.video) > 0 else None
|
||||
video_duration = 0
|
||||
if video_stream and video_stream.duration and video_stream.time_base:
|
||||
video_duration = float(video_stream.duration * video_stream.time_base)
|
||||
|
||||
orig_w = video_stream.codec_context.width if video_stream else 512
|
||||
orig_h = video_stream.codec_context.height if video_stream else 512
|
||||
|
||||
# Determine correct colorspace and color range for PyAV conversion to prevent color shift
|
||||
try:
|
||||
from av.video.reformatter import Colorspace, ColorRange
|
||||
# Improve fallback heuristic to check both dimensions (e.g. 720x1280 vertical video is HD)
|
||||
fallback_cs = Colorspace.ITU709 if max(orig_w, orig_h) >= 720 else Colorspace.ITU601
|
||||
fallback_cr = ColorRange.MPEG
|
||||
dst_range = ColorRange.JPEG # RGB should always be full range
|
||||
except ImportError:
|
||||
fallback_cs = "itu709" if max(orig_w, orig_h) >= 720 else "itu601"
|
||||
fallback_cr = "mpeg"
|
||||
dst_range = "jpeg"
|
||||
|
||||
src_colorspace = fallback_cs
|
||||
src_color_range = fallback_cr
|
||||
|
||||
if video_stream and video_stream.codec_context:
|
||||
cc = video_stream.codec_context
|
||||
|
||||
c_space = getattr(cc, 'colorspace', getattr(cc, 'color_space', None))
|
||||
if c_space and hasattr(c_space, 'name') and c_space.name != "UNSPECIFIED":
|
||||
src_colorspace = c_space
|
||||
elif c_space and isinstance(c_space, str) and "unspecified" not in c_space.lower():
|
||||
src_colorspace = c_space
|
||||
|
||||
c_range = getattr(cc, 'color_range', None)
|
||||
if c_range and hasattr(c_range, 'name') and c_range.name != "UNSPECIFIED":
|
||||
src_color_range = c_range
|
||||
elif c_range and isinstance(c_range, str) and "unspecified" not in c_range.lower():
|
||||
src_color_range = c_range
|
||||
|
||||
target_w = custom_width if custom_width > 0 else orig_w
|
||||
target_h = custom_height if custom_height > 0 else orig_h
|
||||
|
||||
target_w = target_w - (target_w % 2)
|
||||
target_h = target_h - (target_h % 2)
|
||||
|
||||
# Calculate manual crop from interactive UI first
|
||||
manual_crop_left = int(orig_w * crop_x)
|
||||
manual_crop_top = int(orig_h * crop_y)
|
||||
manual_crop_right = orig_w - int(orig_w * (crop_x + crop_w))
|
||||
manual_crop_bottom = orig_h - int(orig_h * (crop_y + crop_h))
|
||||
|
||||
# Ensure we don't crop more than the image
|
||||
manual_crop_left = max(0, min(manual_crop_left, orig_w - 1))
|
||||
manual_crop_top = max(0, min(manual_crop_top, orig_h - 1))
|
||||
manual_crop_right = max(0, min(manual_crop_right, orig_w - manual_crop_left - 1))
|
||||
manual_crop_bottom = max(0, min(manual_crop_bottom, orig_h - manual_crop_top - 1))
|
||||
|
||||
# After manual crop, the new original dimensions are:
|
||||
cropped_orig_w = orig_w - manual_crop_left - manual_crop_right
|
||||
cropped_orig_h = orig_h - manual_crop_top - manual_crop_bottom
|
||||
|
||||
# If no custom width/height is provided, use the cropped original dimensions
|
||||
if custom_width == 0:
|
||||
target_w = cropped_orig_w
|
||||
target_w = target_w - (target_w % 2)
|
||||
if custom_height == 0:
|
||||
target_h = cropped_orig_h
|
||||
target_h = target_h - (target_h % 2)
|
||||
|
||||
scale_w, scale_h = target_w, target_h
|
||||
pad_left = pad_right = pad_top = pad_bottom = 0
|
||||
crop_left = crop_right = crop_top = crop_bottom = 0
|
||||
|
||||
if custom_width > 0 or custom_height > 0:
|
||||
if resize_method == "maintain aspect ratio" or resize_method == "pad":
|
||||
ratio = min(target_w / cropped_orig_w, target_h / cropped_orig_h)
|
||||
scale_w = int(cropped_orig_w * ratio)
|
||||
scale_h = int(cropped_orig_h * ratio)
|
||||
scale_w = scale_w - (scale_w % 2)
|
||||
scale_h = scale_h - (scale_h % 2)
|
||||
|
||||
if resize_method == "pad":
|
||||
pad_x = target_w - scale_w
|
||||
pad_y = target_h - scale_h
|
||||
pad_left = pad_x // 2
|
||||
pad_right = pad_x - pad_left
|
||||
pad_top = pad_y // 2
|
||||
pad_bottom = pad_y - pad_top
|
||||
else:
|
||||
target_w, target_h = scale_w, scale_h
|
||||
|
||||
elif resize_method == "crop":
|
||||
ratio = max(target_w / cropped_orig_w, target_h / cropped_orig_h)
|
||||
scale_w = int(cropped_orig_w * ratio)
|
||||
scale_h = int(cropped_orig_h * ratio)
|
||||
scale_w = scale_w - (scale_w % 2)
|
||||
scale_h = scale_h - (scale_h % 2)
|
||||
|
||||
crop_x = scale_w - target_w
|
||||
crop_y = scale_h - target_h
|
||||
crop_left = crop_x // 2
|
||||
crop_right = crop_x - crop_left
|
||||
crop_top = crop_y // 2
|
||||
crop_bottom = crop_y - crop_top
|
||||
|
||||
elif resize_method == "stretch to fit":
|
||||
scale_w, scale_h = target_w, target_h
|
||||
|
||||
# Determine exact bounds based on frontend mode
|
||||
if display_mode == "frames":
|
||||
fr = float(frame_rate) if frame_rate > 0 else 24.0
|
||||
actual_start_time = float(start_frame) / fr
|
||||
actual_end_time = float(end_frame) / fr if (end_frame > 0 and end_frame > start_frame) else video_duration
|
||||
else:
|
||||
actual_start_time = start_time
|
||||
actual_end_time = end_time if (end_time > 0 and end_time > start_time) else video_duration
|
||||
|
||||
if actual_end_time <= 0:
|
||||
actual_end_time = float('inf') # Fallback if duration is unknown
|
||||
|
||||
# 2. Extract Video Frames (PyAV)
|
||||
frames = []
|
||||
image_tensor = None
|
||||
frames_loaded = 0
|
||||
|
||||
if video_stream:
|
||||
video_stream.thread_type = "AUTO" # Enable multithreaded decoding
|
||||
|
||||
# Efficiently seek backwards to the nearest keyframe
|
||||
if video_stream.time_base:
|
||||
seek_pts = int(actual_start_time / float(video_stream.time_base))
|
||||
else:
|
||||
seek_pts = int(actual_start_time * av.time_base)
|
||||
|
||||
container.seek(seek_pts, stream=video_stream, backward=True)
|
||||
|
||||
# Custom sampling to force specific framerate
|
||||
frame_interval = 1.0 / float(frame_rate) if frame_rate > 0 else 1.0/24.0
|
||||
expected_target_time = actual_start_time
|
||||
|
||||
# Pre-calculate expected frames
|
||||
alloc_end_time = actual_end_time if actual_end_time != float('inf') else video_duration
|
||||
expected_frames = 0
|
||||
if alloc_end_time > 0:
|
||||
duration_to_extract = alloc_end_time - actual_start_time
|
||||
if duration_to_extract > 0:
|
||||
expected_frames = int(np.ceil(duration_to_extract / frame_interval)) + 2
|
||||
|
||||
pbar = comfy.utils.ProgressBar(expected_frames) if expected_frames > 0 else None
|
||||
|
||||
for frame in container.decode(video_stream):
|
||||
frame_time = frame.time
|
||||
if frame_time is None:
|
||||
frame_time = float(frame.pts * float(video_stream.time_base)) if frame.pts and video_stream.time_base else 0.0
|
||||
|
||||
if frame_time < actual_start_time:
|
||||
continue
|
||||
|
||||
# Add a slight buffer (interval) to ensure we evaluate the boundary correctly
|
||||
if frame_time > actual_end_time + frame_interval:
|
||||
break
|
||||
|
||||
# Fix PyAV color shift by forcing proper colorspace and range conversion.
|
||||
# Omit dst_colorspace so swscale defaults naturally for RGB output
|
||||
# (passing it can cause the YUV matrix to be applied incorrectly).
|
||||
try:
|
||||
frame = frame.reformat(
|
||||
format="rgb24",
|
||||
src_colorspace=src_colorspace,
|
||||
src_color_range=src_color_range,
|
||||
dst_color_range=dst_range
|
||||
)
|
||||
frame_rgb = frame.to_ndarray(format='rgb24')
|
||||
except Exception as e:
|
||||
# Fallback: if explicit color reformat fails, use PyAV's default conversion
|
||||
print(f"[LoadVideoUI] Color reformat failed, using default: {e}")
|
||||
frame_rgb = frame.to_ndarray(format='rgb24')
|
||||
|
||||
# Apply interactive crop first
|
||||
if manual_crop_left > 0 or manual_crop_top > 0 or manual_crop_right > 0 or manual_crop_bottom > 0:
|
||||
frame_rgb = frame_rgb[manual_crop_top:orig_h-manual_crop_bottom, manual_crop_left:orig_w-manual_crop_right, :]
|
||||
|
||||
# Now resize to the scaled dimensions
|
||||
if scale_w != cropped_orig_w or scale_h != cropped_orig_h:
|
||||
import cv2
|
||||
frame_rgb = cv2.resize(frame_rgb, (scale_w, scale_h), interpolation=cv2.INTER_AREA)
|
||||
|
||||
if crop_left > 0 or crop_top > 0 or crop_right > 0 or crop_bottom > 0:
|
||||
frame_rgb = frame_rgb[crop_top:scale_h-crop_bottom, crop_left:scale_w-crop_right, :]
|
||||
if pad_left > 0 or pad_top > 0 or pad_right > 0 or pad_bottom > 0:
|
||||
frame_rgb = np.pad(frame_rgb, ((pad_top, pad_bottom), (pad_left, pad_right), (0, 0)), mode='constant', constant_values=0)
|
||||
|
||||
# Duplicate or skip frames perfectly based on timestamps to meet forced framerate.
|
||||
# FIX: Use strictly less than (<) for actual_end_time to prevent the loop from fetching an extra +1 frame
|
||||
# at the exact boundary of the duration slice!
|
||||
while expected_target_time <= frame_time and expected_target_time < actual_end_time - 1e-5:
|
||||
if image_tensor is None and expected_frames > 0:
|
||||
# First frame: allocate the tensor
|
||||
height, width = frame_rgb.shape[:2]
|
||||
alloc_frames = expected_frames + 50 # Add generous buffer to prevent reallocation
|
||||
try:
|
||||
image_tensor = torch.zeros((alloc_frames, height, width, 3), dtype=torch.float32)
|
||||
except Exception as e:
|
||||
print(f"[LoadVideoUI] Pre-allocation failed, falling back to list: {e}")
|
||||
expected_frames = 0 # Disable pre-allocation
|
||||
|
||||
if image_tensor is not None:
|
||||
# Check bounds (just in case)
|
||||
if frames_loaded >= image_tensor.shape[0]:
|
||||
# Extend tensor if we underestimated
|
||||
extension = torch.zeros((50, image_tensor.shape[1], image_tensor.shape[2], 3), dtype=torch.float32)
|
||||
image_tensor = torch.cat((image_tensor, extension), dim=0)
|
||||
|
||||
# Insert frame with minimal memory copy directly to tensor
|
||||
image_tensor[frames_loaded] = torch.from_numpy(frame_rgb).float().div_(255.0)
|
||||
frames_loaded += 1
|
||||
else:
|
||||
# Fallback list append if pre-allocation failed
|
||||
frames.append(frame_rgb)
|
||||
|
||||
if pbar:
|
||||
pbar.update(1)
|
||||
|
||||
expected_target_time += frame_interval
|
||||
|
||||
# Convert frames to ComfyUI Image standard format [N, H, W, C], float32, range 0.0-1.0
|
||||
if image_tensor is not None:
|
||||
if frames_loaded > 0:
|
||||
image_tensor = image_tensor[:frames_loaded]
|
||||
else:
|
||||
image_tensor = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
|
||||
elif len(frames) > 0:
|
||||
frames_np = np.array(frames, dtype=np.float32) / 255.0
|
||||
image_tensor = torch.from_numpy(frames_np)
|
||||
else:
|
||||
# Fallback for an empty slice
|
||||
image_tensor = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
|
||||
|
||||
# 3. Extract Audio (PyAV)
|
||||
audio_dict = {"waveform": torch.zeros((1, 1, 44100)), "sample_rate": 44100} # Default empty audio
|
||||
|
||||
if len(container.streams.audio) > 0:
|
||||
try:
|
||||
audio_stream = container.streams.audio[0]
|
||||
audio_stream.thread_type = "AUTO"
|
||||
sample_rate = getattr(audio_stream, 'rate', 44100) or 44100
|
||||
|
||||
# We must seek again on the container specifically for the audio stream
|
||||
if audio_stream.time_base:
|
||||
seek_pts = int(actual_start_time / float(audio_stream.time_base))
|
||||
else:
|
||||
seek_pts = int(actual_start_time * av.time_base)
|
||||
|
||||
container.seek(seek_pts, stream=audio_stream, backward=True)
|
||||
|
||||
# Resample to standard float planar format (fltp)
|
||||
resampler = av.AudioResampler(format='fltp')
|
||||
|
||||
audio_data = []
|
||||
first_frame_time = None
|
||||
|
||||
for frame in container.decode(audio_stream):
|
||||
frame_time = frame.time
|
||||
if frame_time is None:
|
||||
frame_time = float(frame.pts * float(audio_stream.time_base)) if frame.pts and audio_stream.time_base else 0.0
|
||||
|
||||
# Give a small 1-second buffer to ensure we catch end frames
|
||||
if frame_time > actual_end_time + 1.0:
|
||||
break
|
||||
|
||||
if first_frame_time is None:
|
||||
first_frame_time = frame_time
|
||||
|
||||
resampled_frames = resampler.resample(frame)
|
||||
for r_frame in resampled_frames:
|
||||
audio_data.append(r_frame.to_ndarray())
|
||||
|
||||
if audio_data:
|
||||
# Concatenate all frames horizontally along the sample axis
|
||||
waveform_np = np.concatenate(audio_data, axis=1)
|
||||
waveform = torch.from_numpy(waveform_np).float()
|
||||
|
||||
if first_frame_time is None:
|
||||
first_frame_time = 0.0
|
||||
|
||||
# Calculate exact slice points to trim precisely
|
||||
offset_sec = max(0.0, actual_start_time - first_frame_time)
|
||||
start_sample = int(offset_sec * sample_rate)
|
||||
|
||||
duration_sec_audio = actual_end_time - actual_start_time
|
||||
end_sample = start_sample + int(duration_sec_audio * sample_rate)
|
||||
|
||||
# Trim array bounds properly
|
||||
if end_sample > start_sample:
|
||||
waveform = waveform[:, start_sample:end_sample]
|
||||
else:
|
||||
waveform = waveform[:, start_sample:]
|
||||
|
||||
# Expand to ComfyUI Audio standard [batch_size, channels, samples]
|
||||
waveform = waveform.unsqueeze(0)
|
||||
audio_dict = {"waveform": waveform, "sample_rate": sample_rate}
|
||||
|
||||
except Exception as e:
|
||||
# Catch gracefully without breaking the pipeline execution
|
||||
print(f"[LoadVideoUI] Audio track extraction skipped or failed: {e}")
|
||||
|
||||
# Always close container to free up system memory lock
|
||||
container.close()
|
||||
|
||||
# Output accurate final duration in seconds
|
||||
final_duration_sec = float(max(0.0, actual_end_time - actual_start_time))
|
||||
|
||||
# Accurately output the true number of extracted frames
|
||||
# (Using the shape of the array provides exact 1:1 parity with the timeline's math)
|
||||
frame_count = image_tensor.shape[0] if (frames_loaded > 0 or len(frames) > 0) else 0
|
||||
if frame_count == 0 and final_duration_sec > 0:
|
||||
# Fallback estimation only if PyAV completely failed to decode a valid chunk
|
||||
calc_fr = float(frame_rate) if frame_rate > 0 else 24.0
|
||||
frame_count = int(np.floor(final_duration_sec * calc_fr))
|
||||
|
||||
import os
|
||||
import torch
|
||||
import numpy as np
|
||||
import folder_paths
|
||||
import av
|
||||
from server import PromptServer
|
||||
from aiohttp import web
|
||||
import comfy.utils
|
||||
|
||||
# Custom API route to serve video files from anywhere on the user's system for the frontend preview
|
||||
@PromptServer.instance.routes.get("/video_ui_custom_view")
|
||||
async def custom_view(request):
|
||||
file_path = request.query.get("filename", "")
|
||||
if os.path.exists(file_path) and os.path.isfile(file_path):
|
||||
return web.FileResponse(file_path)
|
||||
return web.Response(status=404, text="File not found")
|
||||
|
||||
# Custom API route for Chunked Uploads to bypass the 413 Payload Too Large error
|
||||
@PromptServer.instance.routes.post("/video_ui_upload_chunk")
|
||||
async def upload_chunk(request):
|
||||
post = await request.post()
|
||||
file = post.get("file")
|
||||
filename = post.get("filename")
|
||||
chunk_index = int(post.get("chunk_index"))
|
||||
total_chunks = int(post.get("total_chunks"))
|
||||
|
||||
upload_dir = folder_paths.get_input_directory()
|
||||
file_path = os.path.join(upload_dir, filename)
|
||||
|
||||
# Append to file if it's not the first chunk, otherwise write new
|
||||
mode = "ab" if chunk_index > 0 else "wb"
|
||||
with open(file_path, mode) as f:
|
||||
f.write(file.file.read())
|
||||
|
||||
if chunk_index == total_chunks - 1:
|
||||
return web.json_response({"name": filename})
|
||||
return web.json_response({"status": "ok"})
|
||||
|
||||
|
||||
class LoadVideoUI:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"video": ("STRING", {"default": ""}),
|
||||
"start_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
|
||||
"end_time": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
|
||||
"duration": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 100000.0, "step": 0.01}),
|
||||
"start_frame": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}),
|
||||
"end_frame": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}),
|
||||
"duration_frames": ("INT", {"default": 0, "min": 0, "max": 10000000, "step": 1}),
|
||||
"resize_method": (["maintain aspect ratio", "stretch to fit", "pad", "crop"], {"default": "maintain aspect ratio"}),
|
||||
"custom_width": ("INT", {"default": 0, "min": 0, "max": 100000, "step": 8, "tooltip": "Custom width. 0 means original width."}),
|
||||
"custom_height": ("INT", {"default": 0, "min": 0, "max": 100000, "step": 8, "tooltip": "Custom height. 0 means original height."}),
|
||||
"frame_rate": ("INT", {"default": 24, "min": 1, "max": 120, "step": 1, "tooltip": "Force the video to a specific frame rate for extraction."}),
|
||||
"display_mode": (["seconds", "frames"], {"default": "seconds"}),
|
||||
"crop_x": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}),
|
||||
"crop_y": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}),
|
||||
"crop_w": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}),
|
||||
"crop_h": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("IMAGE", "AUDIO", "FLOAT", "INT")
|
||||
RETURN_NAMES = ("images", "audio", "duration", "frame_count")
|
||||
FUNCTION = "load_video"
|
||||
CATEGORY = "WhatDreamsCost"
|
||||
|
||||
def load_video(self, video, frame_rate, display_mode, start_time, end_time, duration, start_frame, end_frame, duration_frames, custom_width=0, custom_height=0, resize_method="maintain aspect ratio", crop_x=0.0, crop_y=0.0, crop_w=1.0, crop_h=1.0, **kwargs):
|
||||
if not video:
|
||||
# Return blank defaults if no video is loaded
|
||||
empty_image = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
|
||||
empty_audio = {"waveform": torch.zeros((1, 1, 44100)), "sample_rate": 44100}
|
||||
return (empty_image, empty_audio, 0.0, 0)
|
||||
|
||||
# 1. Resolve path using ComfyUI standard paths or Absolute Path
|
||||
video_path = video # Try exact/absolute path first
|
||||
if not os.path.exists(video_path):
|
||||
video_path_annotated = folder_paths.get_annotated_filepath(video)
|
||||
if os.path.exists(video_path_annotated):
|
||||
video_path = video_path_annotated
|
||||
else:
|
||||
video_path_input = os.path.join(folder_paths.get_input_directory(), video)
|
||||
if os.path.exists(video_path_input):
|
||||
video_path = video_path_input
|
||||
else:
|
||||
raise FileNotFoundError(f"Video file not found: {video}")
|
||||
|
||||
# Open container to read streams and metadata
|
||||
container = av.open(video_path)
|
||||
|
||||
# Determine video stream and duration
|
||||
video_stream = container.streams.video[0] if len(container.streams.video) > 0 else None
|
||||
video_duration = 0
|
||||
if video_stream and video_stream.duration and video_stream.time_base:
|
||||
video_duration = float(video_stream.duration * video_stream.time_base)
|
||||
|
||||
orig_w = video_stream.codec_context.width if video_stream else 512
|
||||
orig_h = video_stream.codec_context.height if video_stream else 512
|
||||
|
||||
# Determine correct colorspace and color range for PyAV conversion to prevent color shift
|
||||
try:
|
||||
from av.video.reformatter import Colorspace, ColorRange
|
||||
# Improve fallback heuristic to check both dimensions (e.g. 720x1280 vertical video is HD)
|
||||
fallback_cs = Colorspace.ITU709 if max(orig_w, orig_h) >= 720 else Colorspace.ITU601
|
||||
fallback_cr = ColorRange.MPEG
|
||||
dst_range = ColorRange.JPEG # RGB should always be full range
|
||||
except ImportError:
|
||||
fallback_cs = "itu709" if max(orig_w, orig_h) >= 720 else "itu601"
|
||||
fallback_cr = "mpeg"
|
||||
dst_range = "jpeg"
|
||||
|
||||
src_colorspace = fallback_cs
|
||||
src_color_range = fallback_cr
|
||||
|
||||
if video_stream and video_stream.codec_context:
|
||||
cc = video_stream.codec_context
|
||||
|
||||
c_space = getattr(cc, 'colorspace', getattr(cc, 'color_space', None))
|
||||
if c_space and hasattr(c_space, 'name') and c_space.name != "UNSPECIFIED":
|
||||
src_colorspace = c_space
|
||||
elif c_space and isinstance(c_space, str) and "unspecified" not in c_space.lower():
|
||||
src_colorspace = c_space
|
||||
|
||||
c_range = getattr(cc, 'color_range', None)
|
||||
if c_range and hasattr(c_range, 'name') and c_range.name != "UNSPECIFIED":
|
||||
src_color_range = c_range
|
||||
elif c_range and isinstance(c_range, str) and "unspecified" not in c_range.lower():
|
||||
src_color_range = c_range
|
||||
|
||||
target_w = custom_width if custom_width > 0 else orig_w
|
||||
target_h = custom_height if custom_height > 0 else orig_h
|
||||
|
||||
target_w = target_w - (target_w % 2)
|
||||
target_h = target_h - (target_h % 2)
|
||||
|
||||
# Calculate manual crop from interactive UI first
|
||||
manual_crop_left = int(orig_w * crop_x)
|
||||
manual_crop_top = int(orig_h * crop_y)
|
||||
manual_crop_right = orig_w - int(orig_w * (crop_x + crop_w))
|
||||
manual_crop_bottom = orig_h - int(orig_h * (crop_y + crop_h))
|
||||
|
||||
# Ensure we don't crop more than the image
|
||||
manual_crop_left = max(0, min(manual_crop_left, orig_w - 1))
|
||||
manual_crop_top = max(0, min(manual_crop_top, orig_h - 1))
|
||||
manual_crop_right = max(0, min(manual_crop_right, orig_w - manual_crop_left - 1))
|
||||
manual_crop_bottom = max(0, min(manual_crop_bottom, orig_h - manual_crop_top - 1))
|
||||
|
||||
# After manual crop, the new original dimensions are:
|
||||
cropped_orig_w = orig_w - manual_crop_left - manual_crop_right
|
||||
cropped_orig_h = orig_h - manual_crop_top - manual_crop_bottom
|
||||
|
||||
# If no custom width/height is provided, use the cropped original dimensions
|
||||
if custom_width == 0:
|
||||
target_w = cropped_orig_w
|
||||
target_w = target_w - (target_w % 2)
|
||||
if custom_height == 0:
|
||||
target_h = cropped_orig_h
|
||||
target_h = target_h - (target_h % 2)
|
||||
|
||||
scale_w, scale_h = target_w, target_h
|
||||
pad_left = pad_right = pad_top = pad_bottom = 0
|
||||
crop_left = crop_right = crop_top = crop_bottom = 0
|
||||
|
||||
if custom_width > 0 or custom_height > 0:
|
||||
if resize_method == "maintain aspect ratio" or resize_method == "pad":
|
||||
ratio = min(target_w / cropped_orig_w, target_h / cropped_orig_h)
|
||||
scale_w = int(cropped_orig_w * ratio)
|
||||
scale_h = int(cropped_orig_h * ratio)
|
||||
scale_w = scale_w - (scale_w % 2)
|
||||
scale_h = scale_h - (scale_h % 2)
|
||||
|
||||
if resize_method == "pad":
|
||||
pad_x = target_w - scale_w
|
||||
pad_y = target_h - scale_h
|
||||
pad_left = pad_x // 2
|
||||
pad_right = pad_x - pad_left
|
||||
pad_top = pad_y // 2
|
||||
pad_bottom = pad_y - pad_top
|
||||
else:
|
||||
target_w, target_h = scale_w, scale_h
|
||||
|
||||
elif resize_method == "crop":
|
||||
ratio = max(target_w / cropped_orig_w, target_h / cropped_orig_h)
|
||||
scale_w = int(cropped_orig_w * ratio)
|
||||
scale_h = int(cropped_orig_h * ratio)
|
||||
scale_w = scale_w - (scale_w % 2)
|
||||
scale_h = scale_h - (scale_h % 2)
|
||||
|
||||
crop_x = scale_w - target_w
|
||||
crop_y = scale_h - target_h
|
||||
crop_left = crop_x // 2
|
||||
crop_right = crop_x - crop_left
|
||||
crop_top = crop_y // 2
|
||||
crop_bottom = crop_y - crop_top
|
||||
|
||||
elif resize_method == "stretch to fit":
|
||||
scale_w, scale_h = target_w, target_h
|
||||
|
||||
# Determine exact bounds based on frontend mode
|
||||
if display_mode == "frames":
|
||||
fr = float(frame_rate) if frame_rate > 0 else 24.0
|
||||
actual_start_time = float(start_frame) / fr
|
||||
actual_end_time = float(end_frame) / fr if (end_frame > 0 and end_frame > start_frame) else video_duration
|
||||
else:
|
||||
actual_start_time = start_time
|
||||
actual_end_time = end_time if (end_time > 0 and end_time > start_time) else video_duration
|
||||
|
||||
if actual_end_time <= 0:
|
||||
actual_end_time = float('inf') # Fallback if duration is unknown
|
||||
|
||||
# 2. Extract Video Frames (PyAV)
|
||||
frames = []
|
||||
image_tensor = None
|
||||
frames_loaded = 0
|
||||
|
||||
if video_stream:
|
||||
video_stream.thread_type = "AUTO" # Enable multithreaded decoding
|
||||
|
||||
# Efficiently seek backwards to the nearest keyframe
|
||||
if video_stream.time_base:
|
||||
seek_pts = int(actual_start_time / float(video_stream.time_base))
|
||||
else:
|
||||
seek_pts = int(actual_start_time * av.time_base)
|
||||
|
||||
container.seek(seek_pts, stream=video_stream, backward=True)
|
||||
|
||||
# Custom sampling to force specific framerate
|
||||
frame_interval = 1.0 / float(frame_rate) if frame_rate > 0 else 1.0/24.0
|
||||
expected_target_time = actual_start_time
|
||||
|
||||
# Pre-calculate expected frames
|
||||
alloc_end_time = actual_end_time if actual_end_time != float('inf') else video_duration
|
||||
expected_frames = 0
|
||||
if alloc_end_time > 0:
|
||||
duration_to_extract = alloc_end_time - actual_start_time
|
||||
if duration_to_extract > 0:
|
||||
expected_frames = int(np.ceil(duration_to_extract / frame_interval)) + 2
|
||||
|
||||
pbar = comfy.utils.ProgressBar(expected_frames) if expected_frames > 0 else None
|
||||
|
||||
for frame in container.decode(video_stream):
|
||||
frame_time = frame.time
|
||||
if frame_time is None:
|
||||
frame_time = float(frame.pts * float(video_stream.time_base)) if frame.pts and video_stream.time_base else 0.0
|
||||
|
||||
if frame_time < actual_start_time:
|
||||
continue
|
||||
|
||||
# Add a slight buffer (interval) to ensure we evaluate the boundary correctly
|
||||
if frame_time > actual_end_time + frame_interval:
|
||||
break
|
||||
|
||||
# Fix PyAV color shift by forcing proper colorspace and range conversion.
|
||||
# Omit dst_colorspace so swscale defaults naturally for RGB output
|
||||
# (passing it can cause the YUV matrix to be applied incorrectly).
|
||||
try:
|
||||
frame = frame.reformat(
|
||||
format="rgb24",
|
||||
src_colorspace=src_colorspace,
|
||||
src_color_range=src_color_range,
|
||||
dst_color_range=dst_range
|
||||
)
|
||||
frame_rgb = frame.to_ndarray(format='rgb24')
|
||||
except Exception as e:
|
||||
# Fallback: if explicit color reformat fails, use PyAV's default conversion
|
||||
print(f"[LoadVideoUI] Color reformat failed, using default: {e}")
|
||||
frame_rgb = frame.to_ndarray(format='rgb24')
|
||||
|
||||
# Apply interactive crop first
|
||||
if manual_crop_left > 0 or manual_crop_top > 0 or manual_crop_right > 0 or manual_crop_bottom > 0:
|
||||
frame_rgb = frame_rgb[manual_crop_top:orig_h-manual_crop_bottom, manual_crop_left:orig_w-manual_crop_right, :]
|
||||
|
||||
# Now resize to the scaled dimensions
|
||||
if scale_w != cropped_orig_w or scale_h != cropped_orig_h:
|
||||
import cv2
|
||||
frame_rgb = cv2.resize(frame_rgb, (scale_w, scale_h), interpolation=cv2.INTER_AREA)
|
||||
|
||||
if crop_left > 0 or crop_top > 0 or crop_right > 0 or crop_bottom > 0:
|
||||
frame_rgb = frame_rgb[crop_top:scale_h-crop_bottom, crop_left:scale_w-crop_right, :]
|
||||
if pad_left > 0 or pad_top > 0 or pad_right > 0 or pad_bottom > 0:
|
||||
frame_rgb = np.pad(frame_rgb, ((pad_top, pad_bottom), (pad_left, pad_right), (0, 0)), mode='constant', constant_values=0)
|
||||
|
||||
# Duplicate or skip frames perfectly based on timestamps to meet forced framerate.
|
||||
# FIX: Use strictly less than (<) for actual_end_time to prevent the loop from fetching an extra +1 frame
|
||||
# at the exact boundary of the duration slice!
|
||||
while expected_target_time <= frame_time and expected_target_time < actual_end_time - 1e-5:
|
||||
if image_tensor is None and expected_frames > 0:
|
||||
# First frame: allocate the tensor
|
||||
height, width = frame_rgb.shape[:2]
|
||||
alloc_frames = expected_frames + 50 # Add generous buffer to prevent reallocation
|
||||
try:
|
||||
image_tensor = torch.zeros((alloc_frames, height, width, 3), dtype=torch.float32)
|
||||
except Exception as e:
|
||||
print(f"[LoadVideoUI] Pre-allocation failed, falling back to list: {e}")
|
||||
expected_frames = 0 # Disable pre-allocation
|
||||
|
||||
if image_tensor is not None:
|
||||
# Check bounds (just in case)
|
||||
if frames_loaded >= image_tensor.shape[0]:
|
||||
# Extend tensor if we underestimated
|
||||
extension = torch.zeros((50, image_tensor.shape[1], image_tensor.shape[2], 3), dtype=torch.float32)
|
||||
image_tensor = torch.cat((image_tensor, extension), dim=0)
|
||||
|
||||
# Insert frame with minimal memory copy directly to tensor
|
||||
image_tensor[frames_loaded] = torch.from_numpy(frame_rgb).float().div_(255.0)
|
||||
frames_loaded += 1
|
||||
else:
|
||||
# Fallback list append if pre-allocation failed
|
||||
frames.append(frame_rgb)
|
||||
|
||||
if pbar:
|
||||
pbar.update(1)
|
||||
|
||||
expected_target_time += frame_interval
|
||||
|
||||
# Convert frames to ComfyUI Image standard format [N, H, W, C], float32, range 0.0-1.0
|
||||
if image_tensor is not None:
|
||||
if frames_loaded > 0:
|
||||
image_tensor = image_tensor[:frames_loaded]
|
||||
else:
|
||||
image_tensor = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
|
||||
elif len(frames) > 0:
|
||||
frames_np = np.array(frames, dtype=np.float32) / 255.0
|
||||
image_tensor = torch.from_numpy(frames_np)
|
||||
else:
|
||||
# Fallback for an empty slice
|
||||
image_tensor = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
|
||||
|
||||
# 3. Extract Audio (PyAV)
|
||||
audio_dict = {"waveform": torch.zeros((1, 1, 44100)), "sample_rate": 44100} # Default empty audio
|
||||
|
||||
if len(container.streams.audio) > 0:
|
||||
try:
|
||||
audio_stream = container.streams.audio[0]
|
||||
audio_stream.thread_type = "AUTO"
|
||||
sample_rate = getattr(audio_stream, 'rate', 44100) or 44100
|
||||
|
||||
# We must seek again on the container specifically for the audio stream
|
||||
if audio_stream.time_base:
|
||||
seek_pts = int(actual_start_time / float(audio_stream.time_base))
|
||||
else:
|
||||
seek_pts = int(actual_start_time * av.time_base)
|
||||
|
||||
container.seek(seek_pts, stream=audio_stream, backward=True)
|
||||
|
||||
# Resample to standard float planar format (fltp)
|
||||
resampler = av.AudioResampler(format='fltp')
|
||||
|
||||
audio_data = []
|
||||
first_frame_time = None
|
||||
|
||||
for frame in container.decode(audio_stream):
|
||||
frame_time = frame.time
|
||||
if frame_time is None:
|
||||
frame_time = float(frame.pts * float(audio_stream.time_base)) if frame.pts and audio_stream.time_base else 0.0
|
||||
|
||||
# Give a small 1-second buffer to ensure we catch end frames
|
||||
if frame_time > actual_end_time + 1.0:
|
||||
break
|
||||
|
||||
if first_frame_time is None:
|
||||
first_frame_time = frame_time
|
||||
|
||||
resampled_frames = resampler.resample(frame)
|
||||
for r_frame in resampled_frames:
|
||||
audio_data.append(r_frame.to_ndarray())
|
||||
|
||||
if audio_data:
|
||||
# Concatenate all frames horizontally along the sample axis
|
||||
waveform_np = np.concatenate(audio_data, axis=1)
|
||||
waveform = torch.from_numpy(waveform_np).float()
|
||||
|
||||
if first_frame_time is None:
|
||||
first_frame_time = 0.0
|
||||
|
||||
# Calculate exact slice points to trim precisely
|
||||
offset_sec = max(0.0, actual_start_time - first_frame_time)
|
||||
start_sample = int(offset_sec * sample_rate)
|
||||
|
||||
duration_sec_audio = actual_end_time - actual_start_time
|
||||
end_sample = start_sample + int(duration_sec_audio * sample_rate)
|
||||
|
||||
# Trim array bounds properly
|
||||
if end_sample > start_sample:
|
||||
waveform = waveform[:, start_sample:end_sample]
|
||||
else:
|
||||
waveform = waveform[:, start_sample:]
|
||||
|
||||
# Expand to ComfyUI Audio standard [batch_size, channels, samples]
|
||||
waveform = waveform.unsqueeze(0)
|
||||
audio_dict = {"waveform": waveform, "sample_rate": sample_rate}
|
||||
|
||||
except Exception as e:
|
||||
# Catch gracefully without breaking the pipeline execution
|
||||
print(f"[LoadVideoUI] Audio track extraction skipped or failed: {e}")
|
||||
|
||||
# Always close container to free up system memory lock
|
||||
container.close()
|
||||
|
||||
# Output accurate final duration in seconds
|
||||
final_duration_sec = float(max(0.0, actual_end_time - actual_start_time))
|
||||
|
||||
# Accurately output the true number of extracted frames
|
||||
# (Using the shape of the array provides exact 1:1 parity with the timeline's math)
|
||||
frame_count = image_tensor.shape[0] if (frames_loaded > 0 or len(frames) > 0) else 0
|
||||
if frame_count == 0 and final_duration_sec > 0:
|
||||
# Fallback estimation only if PyAV completely failed to decode a valid chunk
|
||||
calc_fr = float(frame_rate) if frame_rate > 0 else 24.0
|
||||
frame_count = int(np.floor(final_duration_sec * calc_fr))
|
||||
|
||||
return (image_tensor, audio_dict, final_duration_sec, frame_count)
|
||||
1781
ltx_director.py
1781
ltx_director.py
File diff suppressed because it is too large
Load Diff
@@ -1,90 +1,154 @@
|
||||
from comfy_extras.nodes_lt import LTXVAddGuide
|
||||
import torch
|
||||
import comfy.utils
|
||||
from comfy_api.latest import io
|
||||
from .ltx_director import GuideData
|
||||
|
||||
|
||||
class LTXDirectorGuide(LTXVAddGuide):
|
||||
@classmethod
|
||||
def define_schema(cls):
|
||||
return io.Schema(
|
||||
node_id="LTXDirectorGuide",
|
||||
display_name="LTX Director Guide",
|
||||
category="WhatDreamsCost",
|
||||
description=(
|
||||
"Applies guide images from a Prompt Relay Timeline node at the frame positions "
|
||||
"and strengths defined on the timeline. Connect guide_data from the timeline node."
|
||||
),
|
||||
inputs=[
|
||||
io.Conditioning.Input("positive", tooltip="Positive conditioning to add guide keyframe info to."),
|
||||
io.Conditioning.Input("negative", tooltip="Negative conditioning to add guide keyframe info to."),
|
||||
io.Vae.Input("vae", tooltip="Video VAE used to encode the guide images."),
|
||||
io.Latent.Input("latent", tooltip="Video latent — guides are inserted into this latent."),
|
||||
GuideData.Input("guide_data", tooltip="Guide data produced by Prompt Relay Encode (Timeline)."),
|
||||
io.Float.Input("scale_by", default=1.0, min=0.01, max=8.0, step=0.01, tooltip="Scale the latent by this factor."),
|
||||
io.Combo.Input("upscale_method", options=["nearest-exact", "bilinear", "area", "bicubic", "bislerp"], default="bicubic", tooltip="Method used to upscale/downscale the latent."),
|
||||
],
|
||||
outputs=[
|
||||
io.Conditioning.Output(display_name="positive"),
|
||||
io.Conditioning.Output(display_name="negative"),
|
||||
io.Latent.Output(display_name="latent", tooltip="Video latent with guide frames applied."),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(cls, positive, negative, vae, latent, guide_data, scale_by=1.0, upscale_method="bicubic") -> io.NodeOutput:
|
||||
scale_factors = vae.downscale_index_formula
|
||||
|
||||
# Clone latents to avoid mutating upstream nodes
|
||||
latent_image = latent["samples"].clone()
|
||||
|
||||
if "noise_mask" in latent:
|
||||
noise_mask = latent["noise_mask"].clone()
|
||||
else:
|
||||
batch, _, latent_frames, latent_height, latent_width = latent_image.shape
|
||||
noise_mask = torch.ones(
|
||||
(batch, 1, latent_frames, 1, 1),
|
||||
dtype=torch.float32,
|
||||
device=latent_image.device,
|
||||
)
|
||||
|
||||
# Apply scale factor if not 1.0
|
||||
if scale_by != 1.0:
|
||||
B, C, F, H, W = latent_image.shape
|
||||
width = round(W * scale_by)
|
||||
height = round(H * scale_by)
|
||||
|
||||
# Reshape to 4D for common_upscale
|
||||
latent_4d = latent_image.permute(0, 2, 1, 3, 4).reshape(B * F, C, H, W)
|
||||
latent_resized_4d = comfy.utils.common_upscale(latent_4d, width, height, upscale_method, "disabled")
|
||||
latent_image = latent_resized_4d.reshape(B, F, C, height, width).permute(0, 2, 1, 3, 4)
|
||||
|
||||
# Also resize noise mask if it's not a broadcasted mask
|
||||
if noise_mask.shape[-1] > 1 or noise_mask.shape[-2] > 1:
|
||||
mask_4d = noise_mask.permute(0, 2, 1, 3, 4).reshape(B * F, 1, H, W)
|
||||
mask_resized_4d = comfy.utils.common_upscale(mask_4d, width, height, upscale_method, "disabled")
|
||||
noise_mask = mask_resized_4d.reshape(B, F, 1, height, width).permute(0, 2, 1, 3, 4)
|
||||
|
||||
_, _, latent_length, latent_height, latent_width = latent_image.shape
|
||||
|
||||
images = guide_data.get("images", [])
|
||||
insert_frames = guide_data.get("insert_frames", [])
|
||||
strengths = guide_data.get("strengths", [])
|
||||
|
||||
for idx, img_tensor in enumerate(images):
|
||||
f_idx = insert_frames[idx] if idx < len(insert_frames) else 0
|
||||
strength = strengths[idx] if idx < len(strengths) else 1.0
|
||||
|
||||
image_1, t = cls.encode(vae, latent_width, latent_height, img_tensor, scale_factors)
|
||||
frame_idx, latent_idx = cls.get_latent_index(positive, latent_length, len(image_1), f_idx, scale_factors)
|
||||
|
||||
assert latent_idx + t.shape[2] <= latent_length, (
|
||||
f"Guide image {idx + 1}: conditioning frames exceed the length of the latent sequence."
|
||||
)
|
||||
|
||||
positive, negative, latent_image, noise_mask = cls.append_keyframe(
|
||||
positive, negative, frame_idx, latent_image, noise_mask, t, strength, scale_factors,
|
||||
)
|
||||
|
||||
return io.NodeOutput(positive, negative, {"samples": latent_image, "noise_mask": noise_mask})
|
||||
from comfy_extras.nodes_lt import LTXVAddGuide
|
||||
import torch
|
||||
import comfy.utils
|
||||
from comfy_api.latest import io
|
||||
from .ltx_director import GuideData
|
||||
|
||||
|
||||
class LTXDirectorGuide(LTXVAddGuide):
|
||||
@classmethod
|
||||
def define_schema(cls):
|
||||
return io.Schema(
|
||||
node_id="LTXDirectorGuide",
|
||||
display_name="LTX Director Guide",
|
||||
category="WhatDreamsCost",
|
||||
description=(
|
||||
"Applies guide images from a Prompt Relay Timeline node at the frame positions "
|
||||
"and strengths defined on the timeline. Connect guide_data from the timeline node."
|
||||
),
|
||||
inputs=[
|
||||
io.Conditioning.Input("positive", tooltip="Positive conditioning to add guide keyframe info to."),
|
||||
io.Conditioning.Input("negative", tooltip="Negative conditioning to add guide keyframe info to."),
|
||||
io.Vae.Input("vae", tooltip="Video VAE used to encode the guide images."),
|
||||
io.Latent.Input("latent", tooltip="Video latent — guides are inserted into this latent."),
|
||||
GuideData.Input("guide_data", tooltip="Guide data produced by Prompt Relay Encode (Timeline)."),
|
||||
io.Float.Input("scale_by", default=1.0, min=0.01, max=8.0, step=0.01, tooltip="Scale the latent by this factor."),
|
||||
io.Combo.Input("upscale_method", options=["nearest-exact", "bilinear", "area", "bicubic", "bislerp"], default="bicubic", tooltip="Method used to upscale/downscale the latent."),
|
||||
io.Float.Input("msr_strength", default=0.0, min=0.0, max=1.0, step=0.05, tooltip="Licon MSR only: per-stage reference strength override. 0 = use the Director's value (full pull, right for stage 1). On a refinement/upscale stage set ~0.4 to hold detail without the references repainting the opening (fixes stage-2 mist/ghosting)."),
|
||||
],
|
||||
outputs=[
|
||||
io.Conditioning.Output(display_name="positive"),
|
||||
io.Conditioning.Output(display_name="negative"),
|
||||
io.Latent.Output(display_name="latent", tooltip="Video latent with guide frames applied."),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(cls, positive, negative, vae, latent, guide_data, scale_by=1.0, upscale_method="bicubic", msr_strength=0.0) -> io.NodeOutput:
|
||||
scale_factors = vae.downscale_index_formula
|
||||
|
||||
# Clone latents to avoid mutating upstream nodes
|
||||
latent_image = latent["samples"].clone()
|
||||
|
||||
if "noise_mask" in latent:
|
||||
noise_mask = latent["noise_mask"].clone()
|
||||
else:
|
||||
batch, _, latent_frames, latent_height, latent_width = latent_image.shape
|
||||
noise_mask = torch.ones(
|
||||
(batch, 1, latent_frames, 1, 1),
|
||||
dtype=torch.float32,
|
||||
device=latent_image.device,
|
||||
)
|
||||
|
||||
# Apply scale factor if not 1.0
|
||||
if scale_by != 1.0:
|
||||
B, C, F, H, W = latent_image.shape
|
||||
width = round(W * scale_by)
|
||||
height = round(H * scale_by)
|
||||
|
||||
# Reshape to 4D for common_upscale
|
||||
latent_4d = latent_image.permute(0, 2, 1, 3, 4).reshape(B * F, C, H, W)
|
||||
latent_resized_4d = comfy.utils.common_upscale(latent_4d, width, height, upscale_method, "disabled")
|
||||
latent_image = latent_resized_4d.reshape(B, F, C, height, width).permute(0, 2, 1, 3, 4)
|
||||
|
||||
# Also resize noise mask if it's not a broadcasted mask
|
||||
if noise_mask.shape[-1] > 1 or noise_mask.shape[-2] > 1:
|
||||
mask_4d = noise_mask.permute(0, 2, 1, 3, 4).reshape(B * F, 1, H, W)
|
||||
mask_resized_4d = comfy.utils.common_upscale(mask_4d, width, height, upscale_method, "disabled")
|
||||
noise_mask = mask_resized_4d.reshape(B, F, 1, height, width).permute(0, 2, 1, 3, 4)
|
||||
|
||||
_, _, latent_length, latent_height, latent_width = latent_image.shape
|
||||
|
||||
# MSR mode: the Director handed us the raw references; inject them at THIS node's
|
||||
# resolution (so a half-res Stage 1 and a full-res Stage 2 each stay self-consistent).
|
||||
msr = guide_data.get("msr")
|
||||
if msr is not None:
|
||||
return cls._inject_msr(positive, negative, vae, latent_image, noise_mask, msr, msr_strength)
|
||||
|
||||
images = guide_data.get("images", [])
|
||||
insert_frames = guide_data.get("insert_frames", [])
|
||||
strengths = guide_data.get("strengths", [])
|
||||
|
||||
for idx, img_tensor in enumerate(images):
|
||||
f_idx = insert_frames[idx] if idx < len(insert_frames) else 0
|
||||
strength = strengths[idx] if idx < len(strengths) else 1.0
|
||||
|
||||
image_1, t = cls.encode(vae, latent_width, latent_height, img_tensor, scale_factors)
|
||||
frame_idx, latent_idx = cls.get_latent_index(positive, latent_length, len(image_1), f_idx, scale_factors)
|
||||
|
||||
assert latent_idx + t.shape[2] <= latent_length, (
|
||||
f"Guide image {idx + 1}: conditioning frames exceed the length of the latent sequence."
|
||||
)
|
||||
|
||||
positive, negative, latent_image, noise_mask = cls.append_keyframe(
|
||||
positive, negative, frame_idx, latent_image, noise_mask, t, strength, scale_factors,
|
||||
)
|
||||
|
||||
return io.NodeOutput(positive, negative, {"samples": latent_image, "noise_mask": noise_mask})
|
||||
|
||||
@classmethod
|
||||
def _inject_msr(cls, positive, negative, vae, latent_image, noise_mask, msr, msr_strength=0.0):
|
||||
"""Inject the Licon MSR references at this node's current resolution."""
|
||||
from nodes import NODE_CLASS_MAPPINGS
|
||||
from .ltx_director import _execute_comfy_node, _unpack
|
||||
|
||||
IC = NODE_CLASS_MAPPINGS.get("LTXAddVideoICLoRAGuide")
|
||||
AG = NODE_CLASS_MAPPINGS.get("LTXVAddGuide")
|
||||
if IC is None or AG is None:
|
||||
raise ValueError("MSR mode needs LTXAddVideoICLoRAGuide (ComfyUI-LTXVideo) and LTXVAddGuide (core LTXV nodes).")
|
||||
|
||||
def clamp01(v):
|
||||
return max(0.0, min(1.0, float(v)))
|
||||
|
||||
slideshow = msr["slideshow"]
|
||||
keyframes = msr["keyframes"]
|
||||
prefix_latents = int(msr["prefix_latents"])
|
||||
# Per-stage override: msr_strength > 0 wins (set it low on a refinement stage so the
|
||||
# references hold detail without repainting the opening); 0 = use the Director's value.
|
||||
strength = clamp01(msr_strength) if float(msr_strength) > 0.0 else clamp01(msr["strength"])
|
||||
downscale = float(msr["downscale"])
|
||||
|
||||
# Pad the generated region so (pad + keyframes) == prefix_latents, so the downstream crop
|
||||
# (which trims the slideshow's temporal footprint) lands on the true clean length.
|
||||
pad_latents = max(0, prefix_latents - len(keyframes))
|
||||
if pad_latents > 0:
|
||||
B, C, F, H, W = latent_image.shape
|
||||
latent_image = torch.cat(
|
||||
[latent_image, torch.zeros((B, C, pad_latents, H, W), dtype=latent_image.dtype, device=latent_image.device)],
|
||||
dim=2,
|
||||
)
|
||||
mb, mc, mf, mh, mw = noise_mask.shape
|
||||
noise_mask = torch.cat(
|
||||
[noise_mask, torch.ones((mb, mc, pad_latents, mh, mw), dtype=noise_mask.dtype, device=noise_mask.device)],
|
||||
dim=2,
|
||||
)
|
||||
|
||||
latent = {"samples": latent_image, "noise_mask": noise_mask}
|
||||
|
||||
# CONDITIONING PATH: slideshow -> IC-LoRA -> keep conditioning, discard latent.
|
||||
cp, cn, _ = _unpack(_execute_comfy_node(
|
||||
IC, positive=positive, negative=negative, vae=vae, latent=latent, image=slideshow,
|
||||
frame_idx=0, strength=strength, latent_downscale_factor=downscale,
|
||||
crop="center", use_tiled_encode=False, tile_size=256, tile_overlap=64,
|
||||
))
|
||||
|
||||
positive, negative = cp, cn
|
||||
|
||||
# LATENT PATH: one frozen keyframe per reference at frame 0 -> keep latent, drop cond.
|
||||
for kf in keyframes:
|
||||
_p, _n, latent = _unpack(_execute_comfy_node(
|
||||
AG, positive=positive, negative=negative, vae=vae, latent=latent,
|
||||
image=kf, frame_idx=0, strength=strength,
|
||||
))
|
||||
|
||||
return io.NodeOutput(positive, negative, latent)
|
||||
232
ltx_keyframer.py
232
ltx_keyframer.py
@@ -1,117 +1,117 @@
|
||||
import torch
|
||||
import comfy.utils
|
||||
from comfy_api.latest import io
|
||||
|
||||
class LTXKeyframer(io.ComfyNode):
|
||||
@classmethod
|
||||
def define_schema(cls):
|
||||
inputs = [
|
||||
io.Vae.Input("vae", tooltip="Video VAE used to encode the images"),
|
||||
io.Latent.Input("latent", tooltip="Video latent to insert images into"),
|
||||
io.Image.Input("multi_input", tooltip="Batched images from MultiImageLoader"),
|
||||
]
|
||||
|
||||
inputs.append(io.Int.Input("num_images", default=1, min=0, max=50, step=1, display_name="images_loaded", tooltip="Select how many index/strength widgets to configure."))
|
||||
|
||||
for i in range(1, 51): # 1 to 50 images
|
||||
inputs.extend([
|
||||
io.Int.Input(
|
||||
f"insert_frame_{i}",
|
||||
default=0,
|
||||
min=-9999,
|
||||
max=9999,
|
||||
step=1,
|
||||
tooltip=f"Frame insert_frame for image {i} (in pixel space).",
|
||||
optional=True,
|
||||
),
|
||||
io.Float.Input(
|
||||
f"strength_{i}",
|
||||
default=1.0,
|
||||
min=0.0,
|
||||
max=1.0,
|
||||
step=0.01,
|
||||
tooltip=f"Strength for image {i}.",
|
||||
optional=True,
|
||||
),
|
||||
])
|
||||
|
||||
return io.Schema(
|
||||
node_id="LTXKeyframer",
|
||||
display_name="LTX Keyframer",
|
||||
category="WhatDreamsCost",
|
||||
description="Replaces video latent frames with the encoded input images. Number of widgets is dynamically configured.",
|
||||
inputs=inputs,
|
||||
outputs=[
|
||||
io.Latent.Output(display_name="latent", tooltip="The video latent with the images inserted and latent noise mask updated."),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(cls, vae, latent, multi_input, num_images, **kwargs) -> io.NodeOutput:
|
||||
|
||||
samples = latent["samples"].clone()
|
||||
scale_factors = vae.downscale_index_formula
|
||||
_, height_scale_factor, width_scale_factor = scale_factors
|
||||
|
||||
batch, _, latent_frames, latent_height, latent_width = samples.shape
|
||||
width = latent_width * width_scale_factor
|
||||
height = latent_height * height_scale_factor
|
||||
|
||||
# Get existing noise mask if present, otherwise create new one
|
||||
if "noise_mask" in latent:
|
||||
conditioning_latent_frames_mask = latent["noise_mask"].clone()
|
||||
else:
|
||||
conditioning_latent_frames_mask = torch.ones(
|
||||
(batch, 1, latent_frames, 1, 1),
|
||||
dtype=torch.float32,
|
||||
device=samples.device,
|
||||
)
|
||||
|
||||
batch_size = multi_input.shape[0] if multi_input is not None else 0
|
||||
|
||||
# We process inputs up to num_images, extracting values from kwargs
|
||||
for i in range(1, num_images + 1):
|
||||
# Skip if this image index exceeds the batch
|
||||
if i > batch_size:
|
||||
continue
|
||||
|
||||
image = multi_input[i-1:i] # Extract the single image frame from the batch
|
||||
if image is None:
|
||||
continue
|
||||
|
||||
insert_frame = kwargs.get(f"insert_frame_{i}")
|
||||
if insert_frame is None:
|
||||
continue
|
||||
strength = kwargs.get(f"strength_{i}", 1.0)
|
||||
|
||||
if image.shape[1] != height or image.shape[2] != width:
|
||||
pixels = comfy.utils.common_upscale(image.movedim(-1, 1), width, height, "bilinear", "center").movedim(1, -1)
|
||||
else:
|
||||
pixels = image
|
||||
encode_pixels = pixels[:, :, :, :3]
|
||||
t = vae.encode(encode_pixels)
|
||||
|
||||
# Convert pixel frame insert_frame to latent insert_frame
|
||||
time_scale_factor = scale_factors[0]
|
||||
|
||||
# Handle negative indexing in pixel space
|
||||
pixel_frame_count = (latent_frames - 1) * time_scale_factor + 1
|
||||
if insert_frame < 0:
|
||||
insert_frame = pixel_frame_count + insert_frame
|
||||
|
||||
# Convert to latent insert_frame
|
||||
latent_idx = insert_frame // time_scale_factor
|
||||
|
||||
# Clamp to valid range
|
||||
latent_idx = max(0, min(latent_idx, latent_frames - 1))
|
||||
|
||||
# Calculate end insert_frame, ensuring we don't exceed latent_frames
|
||||
end_index = min(latent_idx + t.shape[2], latent_frames)
|
||||
|
||||
# Replace samples at the specified insert_frame range
|
||||
samples[:, :, latent_idx:end_index] = t[:, :, :end_index - latent_idx]
|
||||
|
||||
# Update mask at the specified insert_frame range
|
||||
conditioning_latent_frames_mask[:, :, latent_idx:end_index] = 1.0 - strength
|
||||
|
||||
import torch
|
||||
import comfy.utils
|
||||
from comfy_api.latest import io
|
||||
|
||||
class LTXKeyframer(io.ComfyNode):
|
||||
@classmethod
|
||||
def define_schema(cls):
|
||||
inputs = [
|
||||
io.Vae.Input("vae", tooltip="Video VAE used to encode the images"),
|
||||
io.Latent.Input("latent", tooltip="Video latent to insert images into"),
|
||||
io.Image.Input("multi_input", tooltip="Batched images from MultiImageLoader"),
|
||||
]
|
||||
|
||||
inputs.append(io.Int.Input("num_images", default=1, min=0, max=50, step=1, display_name="images_loaded", tooltip="Select how many index/strength widgets to configure."))
|
||||
|
||||
for i in range(1, 51): # 1 to 50 images
|
||||
inputs.extend([
|
||||
io.Int.Input(
|
||||
f"insert_frame_{i}",
|
||||
default=0,
|
||||
min=-9999,
|
||||
max=9999,
|
||||
step=1,
|
||||
tooltip=f"Frame insert_frame for image {i} (in pixel space).",
|
||||
optional=True,
|
||||
),
|
||||
io.Float.Input(
|
||||
f"strength_{i}",
|
||||
default=1.0,
|
||||
min=0.0,
|
||||
max=1.0,
|
||||
step=0.01,
|
||||
tooltip=f"Strength for image {i}.",
|
||||
optional=True,
|
||||
),
|
||||
])
|
||||
|
||||
return io.Schema(
|
||||
node_id="LTXKeyframer",
|
||||
display_name="LTX Keyframer",
|
||||
category="WhatDreamsCost",
|
||||
description="Replaces video latent frames with the encoded input images. Number of widgets is dynamically configured.",
|
||||
inputs=inputs,
|
||||
outputs=[
|
||||
io.Latent.Output(display_name="latent", tooltip="The video latent with the images inserted and latent noise mask updated."),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(cls, vae, latent, multi_input, num_images, **kwargs) -> io.NodeOutput:
|
||||
|
||||
samples = latent["samples"].clone()
|
||||
scale_factors = vae.downscale_index_formula
|
||||
_, height_scale_factor, width_scale_factor = scale_factors
|
||||
|
||||
batch, _, latent_frames, latent_height, latent_width = samples.shape
|
||||
width = latent_width * width_scale_factor
|
||||
height = latent_height * height_scale_factor
|
||||
|
||||
# Get existing noise mask if present, otherwise create new one
|
||||
if "noise_mask" in latent:
|
||||
conditioning_latent_frames_mask = latent["noise_mask"].clone()
|
||||
else:
|
||||
conditioning_latent_frames_mask = torch.ones(
|
||||
(batch, 1, latent_frames, 1, 1),
|
||||
dtype=torch.float32,
|
||||
device=samples.device,
|
||||
)
|
||||
|
||||
batch_size = multi_input.shape[0] if multi_input is not None else 0
|
||||
|
||||
# We process inputs up to num_images, extracting values from kwargs
|
||||
for i in range(1, num_images + 1):
|
||||
# Skip if this image index exceeds the batch
|
||||
if i > batch_size:
|
||||
continue
|
||||
|
||||
image = multi_input[i-1:i] # Extract the single image frame from the batch
|
||||
if image is None:
|
||||
continue
|
||||
|
||||
insert_frame = kwargs.get(f"insert_frame_{i}")
|
||||
if insert_frame is None:
|
||||
continue
|
||||
strength = kwargs.get(f"strength_{i}", 1.0)
|
||||
|
||||
if image.shape[1] != height or image.shape[2] != width:
|
||||
pixels = comfy.utils.common_upscale(image.movedim(-1, 1), width, height, "bilinear", "center").movedim(1, -1)
|
||||
else:
|
||||
pixels = image
|
||||
encode_pixels = pixels[:, :, :, :3]
|
||||
t = vae.encode(encode_pixels)
|
||||
|
||||
# Convert pixel frame insert_frame to latent insert_frame
|
||||
time_scale_factor = scale_factors[0]
|
||||
|
||||
# Handle negative indexing in pixel space
|
||||
pixel_frame_count = (latent_frames - 1) * time_scale_factor + 1
|
||||
if insert_frame < 0:
|
||||
insert_frame = pixel_frame_count + insert_frame
|
||||
|
||||
# Convert to latent insert_frame
|
||||
latent_idx = insert_frame // time_scale_factor
|
||||
|
||||
# Clamp to valid range
|
||||
latent_idx = max(0, min(latent_idx, latent_frames - 1))
|
||||
|
||||
# Calculate end insert_frame, ensuring we don't exceed latent_frames
|
||||
end_index = min(latent_idx + t.shape[2], latent_frames)
|
||||
|
||||
# Replace samples at the specified insert_frame range
|
||||
samples[:, :, latent_idx:end_index] = t[:, :, :end_index - latent_idx]
|
||||
|
||||
# Update mask at the specified insert_frame range
|
||||
conditioning_latent_frames_mask[:, :, latent_idx:end_index] = 1.0 - strength
|
||||
|
||||
return io.NodeOutput({"samples": samples, "noise_mask": conditioning_latent_frames_mask})
|
||||
307
ltx_sequencer.py
307
ltx_sequencer.py
@@ -1,133 +1,174 @@
|
||||
from comfy_extras.nodes_lt import get_noise_mask, LTXVAddGuide
|
||||
import torch
|
||||
import comfy.utils
|
||||
from comfy_api.latest import io
|
||||
|
||||
class LTXSequencer(LTXVAddGuide):
|
||||
@classmethod
|
||||
def define_schema(cls):
|
||||
inputs = [
|
||||
io.Conditioning.Input("positive", tooltip="Positive conditioning to which guide keyframe info will be added"),
|
||||
io.Conditioning.Input("negative", tooltip="Negative conditioning to which guide keyframe info will be added"),
|
||||
io.Vae.Input("vae", tooltip="Video VAE used to encode the guide images"),
|
||||
io.Latent.Input("latent", tooltip="Video latent, guides are added to the end of this latent"),
|
||||
io.Image.Input("multi_input", tooltip="Batched images from MultiImageLoader"),
|
||||
]
|
||||
|
||||
inputs.append(io.Int.Input("num_images", default=1, min=0, max=50, step=1, display_name="images_loaded", tooltip="Select how many index/strength widgets to configure."))
|
||||
|
||||
# New global settings widgets
|
||||
inputs.append(io.Combo.Input("insert_mode", options=["frames", "seconds"], default="frames", tooltip="Select the method for determining insertion points."))
|
||||
inputs.append(io.Int.Input("frame_rate", default=24, min=1, max=120, step=1, tooltip="Video FPS (used for calculating second insertions)."))
|
||||
|
||||
for i in range(1, 51): # 1 to 50 images
|
||||
inputs.extend([
|
||||
io.Int.Input(
|
||||
f"insert_frame_{i}",
|
||||
default=0,
|
||||
min=-9999,
|
||||
max=9999,
|
||||
step=1,
|
||||
tooltip=f"Frame insert point for image {i} (in pixel space).",
|
||||
optional=True,
|
||||
),
|
||||
io.Float.Input(
|
||||
f"insert_second_{i}",
|
||||
default=0.0,
|
||||
min=0.0,
|
||||
max=9999.0,
|
||||
step=0.1,
|
||||
tooltip=f"Second insert point for image {i}.",
|
||||
optional=True,
|
||||
),
|
||||
io.Float.Input(
|
||||
f"strength_{i}",
|
||||
default=1.0,
|
||||
min=0.0,
|
||||
max=1.0,
|
||||
step=0.01,
|
||||
tooltip=f"Strength for image {i}.",
|
||||
optional=True,
|
||||
),
|
||||
])
|
||||
|
||||
return io.Schema(
|
||||
node_id="LTXSequencer",
|
||||
display_name="LTX Sequencer",
|
||||
category="WhatDreamsCost",
|
||||
description="Add multiple guide images at specified frame indices or seconds with strengths. Number of widgets is dynamically configured.",
|
||||
inputs=inputs,
|
||||
outputs=[
|
||||
io.Conditioning.Output(display_name="positive"),
|
||||
io.Conditioning.Output(display_name="negative"),
|
||||
io.Latent.Output(display_name="latent", tooltip="Video latent with added guides"),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(cls, positive, negative, vae, latent, multi_input, num_images, **kwargs) -> io.NodeOutput:
|
||||
scale_factors = vae.downscale_index_formula
|
||||
|
||||
# Clone latents to avoid overwriting previous nodes' operations
|
||||
latent_image = latent["samples"].clone()
|
||||
|
||||
# Helper logic to fetch or generate a noise mask
|
||||
if "noise_mask" in latent:
|
||||
noise_mask = latent["noise_mask"].clone()
|
||||
else:
|
||||
batch, _, latent_frames, latent_height, latent_width = latent_image.shape
|
||||
noise_mask = torch.ones(
|
||||
(batch, 1, latent_frames, 1, 1),
|
||||
dtype=torch.float32,
|
||||
device=latent_image.device,
|
||||
)
|
||||
|
||||
_, _, latent_length, latent_height, latent_width = latent_image.shape
|
||||
batch_size = multi_input.shape[0] if multi_input is not None else 0
|
||||
|
||||
# Retrieve selected insertion settings
|
||||
insert_mode = kwargs.get("insert_mode", "frames")
|
||||
frame_rate = kwargs.get("frame_rate", 24)
|
||||
|
||||
# Process inputs up to num_images, extracting dynamic frame/strength values from kwargs
|
||||
for i in range(1, num_images + 1):
|
||||
# Skip if this image index exceeds the batch
|
||||
if i > batch_size:
|
||||
continue
|
||||
|
||||
img = multi_input[i-1:i] # Extract the single image frame from the batch
|
||||
if img is None:
|
||||
continue
|
||||
|
||||
# Calculate the final frame index based on the chosen mode
|
||||
f_idx = None
|
||||
if insert_mode == "frames":
|
||||
f_idx = kwargs.get(f"insert_frame_{i}")
|
||||
elif insert_mode == "seconds":
|
||||
sec = kwargs.get(f"insert_second_{i}")
|
||||
if sec is not None:
|
||||
f_idx = int(sec * frame_rate)
|
||||
|
||||
if f_idx is None:
|
||||
continue
|
||||
|
||||
strength = kwargs.get(f"strength_{i}", 1.0)
|
||||
|
||||
# Execution logic mirrored from LTXVAddGuideMulti
|
||||
image_1, t = cls.encode(vae, latent_width, latent_height, img, scale_factors)
|
||||
|
||||
frame_idx, latent_idx = cls.get_latent_index(positive, latent_length, len(image_1), f_idx, scale_factors)
|
||||
assert latent_idx + t.shape[2] <= latent_length, "Conditioning frames exceed the length of the latent sequence."
|
||||
|
||||
positive, negative, latent_image, noise_mask = cls.append_keyframe(
|
||||
positive,
|
||||
negative,
|
||||
frame_idx,
|
||||
latent_image,
|
||||
noise_mask,
|
||||
t,
|
||||
strength,
|
||||
scale_factors,
|
||||
)
|
||||
|
||||
return io.NodeOutput(positive, negative, {"samples": latent_image, "noise_mask": noise_mask})
|
||||
import json
|
||||
from comfy_extras.nodes_lt import get_noise_mask, LTXVAddGuide
|
||||
import torch
|
||||
import comfy.utils
|
||||
from comfy_api.latest import io
|
||||
|
||||
class LTXSequencer(LTXVAddGuide):
|
||||
@classmethod
|
||||
def define_schema(cls):
|
||||
inputs = [
|
||||
io.Conditioning.Input("positive", tooltip="Positive conditioning to which guide keyframe info will be added"),
|
||||
io.Conditioning.Input("negative", tooltip="Negative conditioning to which guide keyframe info will be added"),
|
||||
io.Vae.Input("vae", tooltip="Video VAE used to encode the guide images"),
|
||||
io.Latent.Input("latent", tooltip="Video latent, guides are added to the end of this latent"),
|
||||
io.Image.Input("multi_input", tooltip="Batched images from MultiImageLoader"),
|
||||
]
|
||||
|
||||
inputs.append(io.Int.Input("num_images", default=1, min=0, max=50, step=1, display_name="images_loaded", tooltip="Select how many index/strength widgets to configure."))
|
||||
|
||||
# New global settings widgets
|
||||
inputs.append(io.Combo.Input("insert_mode", options=["frames", "seconds"], default="frames", tooltip="Select the method for determining insertion points."))
|
||||
inputs.append(io.Int.Input("frame_rate", default=24, min=1, max=120, step=1, tooltip="Video FPS (used for calculating second insertions)."))
|
||||
|
||||
for i in range(1, 51): # 1 to 50 images
|
||||
inputs.extend([
|
||||
io.Int.Input(
|
||||
f"insert_frame_{i}",
|
||||
default=0,
|
||||
min=-9999,
|
||||
max=9999,
|
||||
step=1,
|
||||
tooltip=f"Frame insert point for image {i} (in pixel space).",
|
||||
optional=True,
|
||||
),
|
||||
io.Float.Input(
|
||||
f"insert_second_{i}",
|
||||
default=0.0,
|
||||
min=0.0,
|
||||
max=9999.0,
|
||||
step=0.1,
|
||||
tooltip=f"Second insert point for image {i}.",
|
||||
optional=True,
|
||||
),
|
||||
io.Float.Input(
|
||||
f"strength_{i}",
|
||||
default=1.0,
|
||||
min=0.0,
|
||||
max=1.0,
|
||||
step=0.01,
|
||||
tooltip=f"Strength for image {i}.",
|
||||
optional=True,
|
||||
),
|
||||
])
|
||||
|
||||
return io.Schema(
|
||||
node_id="LTXSequencer",
|
||||
display_name="LTX Sequencer",
|
||||
category="WhatDreamsCost",
|
||||
description="Add multiple guide images at specified frame indices or seconds. Auto-syncs to Prompt Relay invisibly via the positive wire.",
|
||||
inputs=inputs,
|
||||
outputs=[
|
||||
io.Conditioning.Output(display_name="positive"),
|
||||
io.Conditioning.Output(display_name="negative"),
|
||||
io.Latent.Output(display_name="latent", tooltip="Video latent with added guides"),
|
||||
# ---> ADDED SYNC LOG OUTPUT <---
|
||||
io.String.Output(display_name="sync_log", tooltip="Outputs a readable text block showing the actual times used for each image"),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(cls, positive, negative, vae, latent, multi_input, num_images, **kwargs) -> io.NodeOutput:
|
||||
scale_factors = vae.downscale_index_formula
|
||||
|
||||
# Clone latents to avoid overwriting previous nodes' operations
|
||||
latent_image = latent["samples"].clone()
|
||||
|
||||
# Helper logic to fetch or generate a noise mask
|
||||
if "noise_mask" in latent:
|
||||
noise_mask = latent["noise_mask"].clone()
|
||||
else:
|
||||
batch, _, latent_frames, latent_height, latent_width = latent_image.shape
|
||||
noise_mask = torch.ones(
|
||||
(batch, 1, latent_frames, 1, 1),
|
||||
dtype=torch.float32,
|
||||
device=latent_image.device,
|
||||
)
|
||||
|
||||
_, _, latent_length, latent_height, latent_width = latent_image.shape
|
||||
batch_size = multi_input.shape[0] if multi_input is not None else 0
|
||||
|
||||
# Retrieve selected insertion settings
|
||||
insert_mode = kwargs.get("insert_mode", "frames")
|
||||
frame_rate = kwargs.get("frame_rate", 24)
|
||||
|
||||
# ---> EXTRACT GHOST SYNC DATA <---
|
||||
timeline_data = None
|
||||
if positive is not None and len(positive) > 0:
|
||||
if "prompt_relay_timeline" in positive[0][1]:
|
||||
try:
|
||||
timeline_data = json.loads(positive[0][1]["prompt_relay_timeline"])
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
# Prepare the log text
|
||||
sync_log_lines = []
|
||||
if timeline_data:
|
||||
sync_log_lines.append("=== TIMELINE SYNC ENABLED ===")
|
||||
sync_log_lines.append(f"Invisibly Synced via Positive Wire! Using {insert_mode.upper()}:")
|
||||
else:
|
||||
sync_log_lines.append("=== MANUAL MODE ===")
|
||||
sync_log_lines.append(f"No timeline detected, or Sync turned OFF. Using manual UI inputs ({insert_mode}):")
|
||||
|
||||
# Process inputs up to num_images, extracting dynamic frame/strength values from kwargs
|
||||
for i in range(1, num_images + 1):
|
||||
# Skip if this image index exceeds the batch
|
||||
if i > batch_size:
|
||||
sync_log_lines.append(f"Image #{i}: Skipped (No image loaded in batch)")
|
||||
continue
|
||||
|
||||
img = multi_input[i-1:i] # Extract the single image frame from the batch
|
||||
if img is None:
|
||||
continue
|
||||
|
||||
f_idx = None
|
||||
strength = kwargs.get(f"strength_{i}", 1.0)
|
||||
|
||||
# 1. AUTO-SYNC
|
||||
if timeline_data and "starts_frames" in timeline_data and (i - 1) < len(timeline_data["starts_frames"]):
|
||||
display_frame = timeline_data["starts_frames"][i - 1]
|
||||
display_sec = timeline_data["starts_seconds"][i - 1]
|
||||
|
||||
if insert_mode == "frames":
|
||||
f_idx = display_frame
|
||||
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Synced to Segment #{i} start @ {display_frame} frames")
|
||||
elif insert_mode == "seconds":
|
||||
f_idx = int(display_sec * frame_rate)
|
||||
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Synced to Segment #{i} start @ {display_sec:.2f} seconds (Frame {f_idx})")
|
||||
|
||||
# 2. MANUAL FALLBACK
|
||||
else:
|
||||
if insert_mode == "frames":
|
||||
f_idx = kwargs.get(f"insert_frame_{i}")
|
||||
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Manual input @ {f_idx} frames")
|
||||
elif insert_mode == "seconds":
|
||||
sec = kwargs.get(f"insert_second_{i}")
|
||||
if sec is not None:
|
||||
f_idx = int(sec * frame_rate)
|
||||
sync_log_lines.append(f"-> Image #{i} (Strength: {strength}): Manual input @ {sec:.2f} seconds (Frame {f_idx})")
|
||||
|
||||
if f_idx is None:
|
||||
continue
|
||||
|
||||
# Execution logic mirrored from LTXVAddGuideMulti
|
||||
image_1, t = cls.encode(vae, latent_width, latent_height, img, scale_factors)
|
||||
|
||||
frame_idx, latent_idx = cls.get_latent_index(positive, latent_length, len(image_1), f_idx, scale_factors)
|
||||
assert latent_idx + t.shape[2] <= latent_length, "Conditioning frames exceed the length of the latent sequence."
|
||||
|
||||
positive, negative, latent_image, noise_mask = cls.append_keyframe(
|
||||
positive,
|
||||
negative,
|
||||
frame_idx,
|
||||
latent_image,
|
||||
noise_mask,
|
||||
t,
|
||||
strength,
|
||||
scale_factors,
|
||||
)
|
||||
|
||||
sync_log_str = "\n".join(sync_log_lines)
|
||||
print(f"\n[LTX Sequencer]\n{sync_log_str}\n")
|
||||
|
||||
# Returned the newly formatted sync log string at the end of NodeOutput
|
||||
return io.NodeOutput(positive, negative, {"samples": latent_image, "noise_mask": noise_mask}, sync_log_str)
|
||||
@@ -1,182 +1,182 @@
|
||||
import torch
|
||||
import torch.nn.functional as F
|
||||
import numpy as np
|
||||
from PIL import Image, ImageOps
|
||||
import os
|
||||
import folder_paths
|
||||
import io
|
||||
import comfy.utils
|
||||
|
||||
class MultiImageLoader:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"image_paths": ("STRING", {"default": "", "multiline": True}),
|
||||
"width": ("INT", {"default": 0, "min": 0, "max": 8192, "step": 1}),
|
||||
"height": ("INT", {"default": 0, "min": 0, "max": 8192, "step": 1}),
|
||||
"interpolation": (["lanczos", "nearest", "bilinear", "bicubic", "area", "nearest-exact"],),
|
||||
"resize_method": (["keep proportion", "stretch", "pad", "crop"],),
|
||||
"multiple_of": ("INT", {"default": 32, "min": 0, "max": 512, "step": 1}),
|
||||
"img_compression": ("INT", {"default": 18, "min": 0, "max": 100, "step": 1}),
|
||||
},
|
||||
}
|
||||
|
||||
# Added "IMAGE" at the beginning for multi_output + 50 individual outputs = 51 outputs
|
||||
RETURN_TYPES = ("IMAGE",) * 51
|
||||
RETURN_NAMES = ("multi_output",) + tuple(f"image_{i+1}" for i in range(50))
|
||||
FUNCTION = "load_images"
|
||||
CATEGORY = "WhatDreamsCost"
|
||||
|
||||
def resize_image(self, image, width, height, resize_method="keep proportion", interpolation="nearest", multiple_of=0):
|
||||
MAX_RESOLUTION = 8192
|
||||
_, oh, ow, _ = image.shape
|
||||
x = y = x2 = y2 = 0
|
||||
pad_left = pad_right = pad_top = pad_bottom = 0
|
||||
|
||||
if multiple_of > 1:
|
||||
width = width - (width % multiple_of)
|
||||
height = height - (height % multiple_of)
|
||||
|
||||
if resize_method == 'keep proportion' or resize_method == 'pad':
|
||||
if width == 0 and oh < height:
|
||||
width = MAX_RESOLUTION
|
||||
elif width == 0 and oh >= height:
|
||||
width = ow
|
||||
|
||||
if height == 0 and ow < width:
|
||||
height = MAX_RESOLUTION
|
||||
elif height == 0 and ow >= width:
|
||||
height = oh
|
||||
|
||||
ratio = min(width / ow, height / oh)
|
||||
new_width = round(ow * ratio)
|
||||
new_height = round(oh * ratio)
|
||||
|
||||
if resize_method == 'pad':
|
||||
pad_left = (width - new_width) // 2
|
||||
pad_right = width - new_width - pad_left
|
||||
pad_top = (height - new_height) // 2
|
||||
pad_bottom = height - new_height - pad_top
|
||||
|
||||
width = new_width
|
||||
height = new_height
|
||||
|
||||
elif resize_method == 'crop':
|
||||
width = width if width > 0 else ow
|
||||
height = height if height > 0 else oh
|
||||
|
||||
ratio = max(width / ow, height / oh)
|
||||
new_width = round(ow * ratio)
|
||||
new_height = round(oh * ratio)
|
||||
x = (new_width - width) // 2
|
||||
y = (new_height - height) // 2
|
||||
x2 = x + width
|
||||
y2 = y + height
|
||||
if x2 > new_width:
|
||||
x -= (x2 - new_width)
|
||||
if x < 0:
|
||||
x = 0
|
||||
if y2 > new_height:
|
||||
y -= (y2 - new_height)
|
||||
if y < 0:
|
||||
y = 0
|
||||
width = new_width
|
||||
height = new_height
|
||||
|
||||
else:
|
||||
width = width if width > 0 else ow
|
||||
height = height if height > 0 else oh
|
||||
|
||||
# Always apply resize logic
|
||||
outputs = image.permute(0, 3, 1, 2)
|
||||
|
||||
if interpolation == "lanczos":
|
||||
outputs = comfy.utils.lanczos(outputs, width, height)
|
||||
else:
|
||||
outputs = F.interpolate(outputs, size=(height, width), mode=interpolation)
|
||||
|
||||
if resize_method == 'pad':
|
||||
if pad_left > 0 or pad_right > 0 or pad_top > 0 or pad_bottom > 0:
|
||||
outputs = F.pad(outputs, (pad_left, pad_right, pad_top, pad_bottom), value=0)
|
||||
|
||||
outputs = outputs.permute(0, 2, 3, 1)
|
||||
|
||||
if resize_method == 'crop':
|
||||
if x > 0 or y > 0 or x2 > 0 or y2 > 0:
|
||||
outputs = outputs[:, y:y2, x:x2, :]
|
||||
|
||||
if multiple_of > 1 and (outputs.shape[2] % multiple_of != 0 or outputs.shape[1] % multiple_of != 0):
|
||||
width = outputs.shape[2]
|
||||
height = outputs.shape[1]
|
||||
x = (width % multiple_of) // 2
|
||||
y = (height % multiple_of) // 2
|
||||
x2 = width - ((width % multiple_of) - x)
|
||||
y2 = height - ((height % multiple_of) - y)
|
||||
outputs = outputs[:, y:y2, x:x2, :]
|
||||
|
||||
outputs = torch.clamp(outputs, 0, 1)
|
||||
|
||||
return outputs
|
||||
|
||||
def load_images(self, image_paths, width, height, interpolation, resize_method, multiple_of, img_compression):
|
||||
results = []
|
||||
valid_paths = [p.strip() for p in image_paths.split("\n") if p.strip()]
|
||||
|
||||
for path in valid_paths:
|
||||
try:
|
||||
# Resolve full path
|
||||
full_path = path
|
||||
if not os.path.exists(full_path):
|
||||
full_path = os.path.join(folder_paths.get_input_directory(), path)
|
||||
|
||||
if not os.path.exists(full_path):
|
||||
print(f"Warning: Image path not found: {path}")
|
||||
continue
|
||||
|
||||
# Load image
|
||||
image = Image.open(full_path)
|
||||
image = ImageOps.exif_transpose(image)
|
||||
image = image.convert("RGB")
|
||||
|
||||
# Convert to Torch Tensor to prepare for Advanced Resize Logic
|
||||
image_np = np.array(image).astype(np.float32) / 255.0
|
||||
image_tensor = torch.from_numpy(image_np)[None,]
|
||||
|
||||
# Apply Advanced Resize
|
||||
image_tensor = self.resize_image(image_tensor, width, height, resize_method, interpolation, multiple_of)
|
||||
|
||||
# Compression (Applied after resize to accurately maintain the effect)
|
||||
if img_compression > 0:
|
||||
img_np = (image_tensor[0].numpy() * 255).clip(0, 255).astype(np.uint8)
|
||||
img_pil = Image.fromarray(img_np)
|
||||
img_byte_arr = io.BytesIO()
|
||||
img_pil.save(img_byte_arr, format="JPEG", quality=max(1, 100 - img_compression))
|
||||
img_pil = Image.open(img_byte_arr)
|
||||
image_tensor = torch.from_numpy(np.array(img_pil).astype(np.float32) / 255.0)[None,]
|
||||
|
||||
results.append(image_tensor)
|
||||
except Exception as e:
|
||||
print(f"Error loading {path}: {e}")
|
||||
|
||||
# Combine all successfully loaded images into a single batched tensor for multi_output
|
||||
if len(results) > 0:
|
||||
# Safety Check: Advanced resize methods might output differently sized tensors (e.g., 'keep proportion')
|
||||
first_shape = results[0].shape
|
||||
all_same_shape = all(r.shape == first_shape for r in results)
|
||||
|
||||
if all_same_shape:
|
||||
multi_output = torch.cat(results, dim=0)
|
||||
else:
|
||||
print("MultiImageLoader Warning: Images have different dimensions due to resize settings. Cannot batch into multi_output. Outputting zero tensor for the batch, but individual output nodes will still work fine.")
|
||||
multi_output = torch.zeros((1, 64, 64, 3))
|
||||
else:
|
||||
# Fallback empty tensor if no valid paths
|
||||
multi_output = torch.zeros((1, 64, 64, 3))
|
||||
results = [multi_output]
|
||||
|
||||
# Pad individual outputs exactly to length 50 as defined in RETURN_TYPES
|
||||
padded_results = results + [torch.zeros((1, 64, 64, 3))] * (50 - len(results))
|
||||
|
||||
# Return the multi batch output first, followed by the individual padded items
|
||||
import torch
|
||||
import torch.nn.functional as F
|
||||
import numpy as np
|
||||
from PIL import Image, ImageOps
|
||||
import os
|
||||
import folder_paths
|
||||
import io
|
||||
import comfy.utils
|
||||
|
||||
class MultiImageLoader:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"image_paths": ("STRING", {"default": "", "multiline": True}),
|
||||
"width": ("INT", {"default": 0, "min": 0, "max": 8192, "step": 1}),
|
||||
"height": ("INT", {"default": 0, "min": 0, "max": 8192, "step": 1}),
|
||||
"interpolation": (["lanczos", "nearest", "bilinear", "bicubic", "area", "nearest-exact"],),
|
||||
"resize_method": (["keep proportion", "stretch", "pad", "crop"],),
|
||||
"multiple_of": ("INT", {"default": 32, "min": 0, "max": 512, "step": 1}),
|
||||
"img_compression": ("INT", {"default": 18, "min": 0, "max": 100, "step": 1}),
|
||||
},
|
||||
}
|
||||
|
||||
# Added "IMAGE" at the beginning for multi_output + 50 individual outputs = 51 outputs
|
||||
RETURN_TYPES = ("IMAGE",) * 51
|
||||
RETURN_NAMES = ("multi_output",) + tuple(f"image_{i+1}" for i in range(50))
|
||||
FUNCTION = "load_images"
|
||||
CATEGORY = "WhatDreamsCost"
|
||||
|
||||
def resize_image(self, image, width, height, resize_method="keep proportion", interpolation="nearest", multiple_of=0):
|
||||
MAX_RESOLUTION = 8192
|
||||
_, oh, ow, _ = image.shape
|
||||
x = y = x2 = y2 = 0
|
||||
pad_left = pad_right = pad_top = pad_bottom = 0
|
||||
|
||||
if multiple_of > 1:
|
||||
width = width - (width % multiple_of)
|
||||
height = height - (height % multiple_of)
|
||||
|
||||
if resize_method == 'keep proportion' or resize_method == 'pad':
|
||||
if width == 0 and oh < height:
|
||||
width = MAX_RESOLUTION
|
||||
elif width == 0 and oh >= height:
|
||||
width = ow
|
||||
|
||||
if height == 0 and ow < width:
|
||||
height = MAX_RESOLUTION
|
||||
elif height == 0 and ow >= width:
|
||||
height = oh
|
||||
|
||||
ratio = min(width / ow, height / oh)
|
||||
new_width = round(ow * ratio)
|
||||
new_height = round(oh * ratio)
|
||||
|
||||
if resize_method == 'pad':
|
||||
pad_left = (width - new_width) // 2
|
||||
pad_right = width - new_width - pad_left
|
||||
pad_top = (height - new_height) // 2
|
||||
pad_bottom = height - new_height - pad_top
|
||||
|
||||
width = new_width
|
||||
height = new_height
|
||||
|
||||
elif resize_method == 'crop':
|
||||
width = width if width > 0 else ow
|
||||
height = height if height > 0 else oh
|
||||
|
||||
ratio = max(width / ow, height / oh)
|
||||
new_width = round(ow * ratio)
|
||||
new_height = round(oh * ratio)
|
||||
x = (new_width - width) // 2
|
||||
y = (new_height - height) // 2
|
||||
x2 = x + width
|
||||
y2 = y + height
|
||||
if x2 > new_width:
|
||||
x -= (x2 - new_width)
|
||||
if x < 0:
|
||||
x = 0
|
||||
if y2 > new_height:
|
||||
y -= (y2 - new_height)
|
||||
if y < 0:
|
||||
y = 0
|
||||
width = new_width
|
||||
height = new_height
|
||||
|
||||
else:
|
||||
width = width if width > 0 else ow
|
||||
height = height if height > 0 else oh
|
||||
|
||||
# Always apply resize logic
|
||||
outputs = image.permute(0, 3, 1, 2)
|
||||
|
||||
if interpolation == "lanczos":
|
||||
outputs = comfy.utils.lanczos(outputs, width, height)
|
||||
else:
|
||||
outputs = F.interpolate(outputs, size=(height, width), mode=interpolation)
|
||||
|
||||
if resize_method == 'pad':
|
||||
if pad_left > 0 or pad_right > 0 or pad_top > 0 or pad_bottom > 0:
|
||||
outputs = F.pad(outputs, (pad_left, pad_right, pad_top, pad_bottom), value=0)
|
||||
|
||||
outputs = outputs.permute(0, 2, 3, 1)
|
||||
|
||||
if resize_method == 'crop':
|
||||
if x > 0 or y > 0 or x2 > 0 or y2 > 0:
|
||||
outputs = outputs[:, y:y2, x:x2, :]
|
||||
|
||||
if multiple_of > 1 and (outputs.shape[2] % multiple_of != 0 or outputs.shape[1] % multiple_of != 0):
|
||||
width = outputs.shape[2]
|
||||
height = outputs.shape[1]
|
||||
x = (width % multiple_of) // 2
|
||||
y = (height % multiple_of) // 2
|
||||
x2 = width - ((width % multiple_of) - x)
|
||||
y2 = height - ((height % multiple_of) - y)
|
||||
outputs = outputs[:, y:y2, x:x2, :]
|
||||
|
||||
outputs = torch.clamp(outputs, 0, 1)
|
||||
|
||||
return outputs
|
||||
|
||||
def load_images(self, image_paths, width, height, interpolation, resize_method, multiple_of, img_compression):
|
||||
results = []
|
||||
valid_paths = [p.strip() for p in image_paths.split("\n") if p.strip()]
|
||||
|
||||
for path in valid_paths:
|
||||
try:
|
||||
# Resolve full path
|
||||
full_path = path
|
||||
if not os.path.exists(full_path):
|
||||
full_path = os.path.join(folder_paths.get_input_directory(), path)
|
||||
|
||||
if not os.path.exists(full_path):
|
||||
print(f"Warning: Image path not found: {path}")
|
||||
continue
|
||||
|
||||
# Load image
|
||||
image = Image.open(full_path)
|
||||
image = ImageOps.exif_transpose(image)
|
||||
image = image.convert("RGB")
|
||||
|
||||
# Convert to Torch Tensor to prepare for Advanced Resize Logic
|
||||
image_np = np.array(image).astype(np.float32) / 255.0
|
||||
image_tensor = torch.from_numpy(image_np)[None,]
|
||||
|
||||
# Apply Advanced Resize
|
||||
image_tensor = self.resize_image(image_tensor, width, height, resize_method, interpolation, multiple_of)
|
||||
|
||||
# Compression (Applied after resize to accurately maintain the effect)
|
||||
if img_compression > 0:
|
||||
img_np = (image_tensor[0].numpy() * 255).clip(0, 255).astype(np.uint8)
|
||||
img_pil = Image.fromarray(img_np)
|
||||
img_byte_arr = io.BytesIO()
|
||||
img_pil.save(img_byte_arr, format="JPEG", quality=max(1, 100 - img_compression))
|
||||
img_pil = Image.open(img_byte_arr)
|
||||
image_tensor = torch.from_numpy(np.array(img_pil).astype(np.float32) / 255.0)[None,]
|
||||
|
||||
results.append(image_tensor)
|
||||
except Exception as e:
|
||||
print(f"Error loading {path}: {e}")
|
||||
|
||||
# Combine all successfully loaded images into a single batched tensor for multi_output
|
||||
if len(results) > 0:
|
||||
# Safety Check: Advanced resize methods might output differently sized tensors (e.g., 'keep proportion')
|
||||
first_shape = results[0].shape
|
||||
all_same_shape = all(r.shape == first_shape for r in results)
|
||||
|
||||
if all_same_shape:
|
||||
multi_output = torch.cat(results, dim=0)
|
||||
else:
|
||||
print("MultiImageLoader Warning: Images have different dimensions due to resize settings. Cannot batch into multi_output. Outputting zero tensor for the batch, but individual output nodes will still work fine.")
|
||||
multi_output = torch.zeros((1, 64, 64, 3))
|
||||
else:
|
||||
# Fallback empty tensor if no valid paths
|
||||
multi_output = torch.zeros((1, 64, 64, 3))
|
||||
results = [multi_output]
|
||||
|
||||
# Pad individual outputs exactly to length 50 as defined in RETURN_TYPES
|
||||
padded_results = results + [torch.zeros((1, 64, 64, 3))] * (50 - len(results))
|
||||
|
||||
# Return the multi batch output first, followed by the individual padded items
|
||||
return (multi_output, *padded_results[:50])
|
||||
334
patches.py
334
patches.py
@@ -1,167 +1,167 @@
|
||||
import types
|
||||
import torch
|
||||
import comfy.ldm.modules.attention
|
||||
|
||||
|
||||
def _masked_attention(q, k, v, heads, mask, transformer_options={}, **kwargs):
|
||||
# Bypass wrap_attn (sage/etc may ignore masks) by calling attention_pytorch directly.
|
||||
return comfy.ldm.modules.attention.attention_pytorch(
|
||||
q, k, v, heads, mask=mask,
|
||||
_inside_attn_wrapper=True,
|
||||
transformer_options=transformer_options,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
|
||||
def _wan_t2v_forward(self, mask_fn, x, context, transformer_options={}, **kwargs):
|
||||
q = self.norm_q(self.q(x))
|
||||
k = self.norm_k(self.k(context))
|
||||
v = self.v(context)
|
||||
|
||||
mask = mask_fn(q, k, transformer_options)
|
||||
if mask is not None:
|
||||
x = _masked_attention(q, k, v, heads=self.num_heads, mask=mask,
|
||||
transformer_options=transformer_options)
|
||||
else:
|
||||
x = comfy.ldm.modules.attention.optimized_attention(
|
||||
q, k, v, heads=self.num_heads, transformer_options=transformer_options,
|
||||
)
|
||||
return self.o(x)
|
||||
|
||||
|
||||
def _wan_i2v_forward(self, mask_fn, x, context, context_img_len, transformer_options={}, **kwargs):
|
||||
context_img = context[:, :context_img_len]
|
||||
context_text = context[:, context_img_len:]
|
||||
|
||||
q = self.norm_q(self.q(x))
|
||||
|
||||
k_img = self.norm_k_img(self.k_img(context_img))
|
||||
v_img = self.v_img(context_img)
|
||||
img_x = comfy.ldm.modules.attention.optimized_attention(
|
||||
q, k_img, v_img, heads=self.num_heads, transformer_options=transformer_options,
|
||||
)
|
||||
|
||||
k = self.norm_k(self.k(context_text))
|
||||
v = self.v(context_text)
|
||||
|
||||
mask = mask_fn(q, k, transformer_options)
|
||||
if mask is not None:
|
||||
x = _masked_attention(q, k, v, heads=self.num_heads, mask=mask,
|
||||
transformer_options=transformer_options)
|
||||
else:
|
||||
x = comfy.ldm.modules.attention.optimized_attention(
|
||||
q, k, v, heads=self.num_heads, transformer_options=transformer_options,
|
||||
)
|
||||
|
||||
return self.o(x + img_x)
|
||||
|
||||
|
||||
def _ltx_forward(self, mask_fn, x, context=None, mask=None, pe=None, k_pe=None, transformer_options={}):
|
||||
from comfy.ldm.lightricks.model import apply_rotary_emb
|
||||
|
||||
is_self_attn = context is None
|
||||
context = x if is_self_attn else context
|
||||
|
||||
q = self.q_norm(self.to_q(x))
|
||||
k = self.k_norm(self.to_k(context))
|
||||
v = self.to_v(context)
|
||||
|
||||
if pe is not None:
|
||||
q = apply_rotary_emb(q, pe)
|
||||
k = apply_rotary_emb(k, pe if k_pe is None else k_pe)
|
||||
|
||||
if not is_self_attn:
|
||||
temporal_mask = mask_fn(q, k, transformer_options)
|
||||
if temporal_mask is not None:
|
||||
mask = temporal_mask if mask is None else mask + temporal_mask
|
||||
|
||||
if mask is None:
|
||||
out = comfy.ldm.modules.attention.optimized_attention(
|
||||
q, k, v, self.heads, attn_precision=self.attn_precision,
|
||||
transformer_options=transformer_options,
|
||||
)
|
||||
else:
|
||||
out = _masked_attention(q, k, v, self.heads, mask=mask,
|
||||
attn_precision=self.attn_precision,
|
||||
transformer_options=transformer_options)
|
||||
|
||||
if self.to_gate_logits is not None:
|
||||
gate_logits = self.to_gate_logits(x)
|
||||
b, t, _ = out.shape
|
||||
out = out.view(b, t, self.heads, self.dim_head)
|
||||
out = out * (2.0 * torch.sigmoid(gate_logits)).unsqueeze(-1)
|
||||
out = out.view(b, t, self.heads * self.dim_head)
|
||||
|
||||
return self.to_out(out)
|
||||
|
||||
|
||||
class _CrossAttnPatch:
|
||||
"""Descriptor that binds (impl, mask_fn) as a method onto a cross-attn module."""
|
||||
|
||||
def __init__(self, impl, mask_fn):
|
||||
self.impl = impl
|
||||
self.mask_fn = mask_fn
|
||||
|
||||
def __get__(self, obj, objtype=None):
|
||||
impl, mask_fn = self.impl, self.mask_fn
|
||||
|
||||
def wrapped(self_module, *args, **kwargs):
|
||||
return impl(self_module, mask_fn, *args, **kwargs)
|
||||
|
||||
return types.MethodType(wrapped, obj)
|
||||
|
||||
|
||||
def detect_model_type(model):
|
||||
"""Return (arch, patch_size, temporal_stride) for latent geometry.
|
||||
|
||||
temporal_stride is the VAE's pixel→latent temporal compression factor,
|
||||
used to convert user-facing pixel frame counts to latent frames.
|
||||
"""
|
||||
diff_model = model.model.diffusion_model
|
||||
|
||||
if hasattr(diff_model, "patch_size") and not hasattr(diff_model, "patchifier"):
|
||||
return "wan", tuple(diff_model.patch_size), 4
|
||||
|
||||
if hasattr(diff_model, "patchifier"):
|
||||
return "ltx", (1, 1, 1), int(diff_model.vae_scale_factors[0])
|
||||
|
||||
raise ValueError(
|
||||
f"Unsupported model type: {type(diff_model).__name__}. "
|
||||
f"Currently supports Wan and LTX models."
|
||||
)
|
||||
|
||||
|
||||
def _check_unpatched(model_clone, key):
|
||||
if key in getattr(model_clone, "object_patches", {}):
|
||||
raise RuntimeError(
|
||||
f"PromptRelay: cross-attention forward at '{key}' is already patched by "
|
||||
"another node (e.g. KJNodes NAG). Stacking is not supported — remove the "
|
||||
"conflicting node."
|
||||
)
|
||||
|
||||
|
||||
def apply_patches(model_clone, arch, mask_fn):
|
||||
diffusion_model = model_clone.get_model_object("diffusion_model")
|
||||
|
||||
if arch == "wan":
|
||||
from comfy.ldm.wan.model import WanI2VCrossAttention
|
||||
for idx, block in enumerate(diffusion_model.blocks):
|
||||
key = f"diffusion_model.blocks.{idx}.cross_attn.forward"
|
||||
_check_unpatched(model_clone, key)
|
||||
cross_attn = block.cross_attn
|
||||
impl = _wan_i2v_forward if isinstance(cross_attn, WanI2VCrossAttention) else _wan_t2v_forward
|
||||
model_clone.add_object_patch(key, _CrossAttnPatch(impl, mask_fn).__get__(cross_attn, cross_attn.__class__))
|
||||
return
|
||||
|
||||
if arch == "ltx":
|
||||
for idx, block in enumerate(diffusion_model.transformer_blocks):
|
||||
for attr in ("attn2", "audio_attn2"):
|
||||
module = getattr(block, attr, None)
|
||||
if module is None:
|
||||
continue
|
||||
key = f"diffusion_model.transformer_blocks.{idx}.{attr}.forward"
|
||||
_check_unpatched(model_clone, key)
|
||||
model_clone.add_object_patch(key, _CrossAttnPatch(_ltx_forward, mask_fn).__get__(module, module.__class__))
|
||||
return
|
||||
|
||||
raise ValueError(f"Unknown model arch: {arch}")
|
||||
import types
|
||||
import torch
|
||||
import comfy.ldm.modules.attention
|
||||
|
||||
|
||||
def _masked_attention(q, k, v, heads, mask, transformer_options={}, **kwargs):
|
||||
# Bypass wrap_attn (sage/etc may ignore masks) by calling attention_pytorch directly.
|
||||
return comfy.ldm.modules.attention.attention_pytorch(
|
||||
q, k, v, heads, mask=mask,
|
||||
_inside_attn_wrapper=True,
|
||||
transformer_options=transformer_options,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
|
||||
def _wan_t2v_forward(self, mask_fn, x, context, transformer_options={}, **kwargs):
|
||||
q = self.norm_q(self.q(x))
|
||||
k = self.norm_k(self.k(context))
|
||||
v = self.v(context)
|
||||
|
||||
mask = mask_fn(q, k, transformer_options)
|
||||
if mask is not None:
|
||||
x = _masked_attention(q, k, v, heads=self.num_heads, mask=mask,
|
||||
transformer_options=transformer_options)
|
||||
else:
|
||||
x = comfy.ldm.modules.attention.optimized_attention(
|
||||
q, k, v, heads=self.num_heads, transformer_options=transformer_options,
|
||||
)
|
||||
return self.o(x)
|
||||
|
||||
|
||||
def _wan_i2v_forward(self, mask_fn, x, context, context_img_len, transformer_options={}, **kwargs):
|
||||
context_img = context[:, :context_img_len]
|
||||
context_text = context[:, context_img_len:]
|
||||
|
||||
q = self.norm_q(self.q(x))
|
||||
|
||||
k_img = self.norm_k_img(self.k_img(context_img))
|
||||
v_img = self.v_img(context_img)
|
||||
img_x = comfy.ldm.modules.attention.optimized_attention(
|
||||
q, k_img, v_img, heads=self.num_heads, transformer_options=transformer_options,
|
||||
)
|
||||
|
||||
k = self.norm_k(self.k(context_text))
|
||||
v = self.v(context_text)
|
||||
|
||||
mask = mask_fn(q, k, transformer_options)
|
||||
if mask is not None:
|
||||
x = _masked_attention(q, k, v, heads=self.num_heads, mask=mask,
|
||||
transformer_options=transformer_options)
|
||||
else:
|
||||
x = comfy.ldm.modules.attention.optimized_attention(
|
||||
q, k, v, heads=self.num_heads, transformer_options=transformer_options,
|
||||
)
|
||||
|
||||
return self.o(x + img_x)
|
||||
|
||||
|
||||
def _ltx_forward(self, mask_fn, x, context=None, mask=None, pe=None, k_pe=None, transformer_options={}):
|
||||
from comfy.ldm.lightricks.model import apply_rotary_emb
|
||||
|
||||
is_self_attn = context is None
|
||||
context = x if is_self_attn else context
|
||||
|
||||
q = self.q_norm(self.to_q(x))
|
||||
k = self.k_norm(self.to_k(context))
|
||||
v = self.to_v(context)
|
||||
|
||||
if pe is not None:
|
||||
q = apply_rotary_emb(q, pe)
|
||||
k = apply_rotary_emb(k, pe if k_pe is None else k_pe)
|
||||
|
||||
if not is_self_attn:
|
||||
temporal_mask = mask_fn(q, k, transformer_options)
|
||||
if temporal_mask is not None:
|
||||
mask = temporal_mask if mask is None else mask + temporal_mask
|
||||
|
||||
if mask is None:
|
||||
out = comfy.ldm.modules.attention.optimized_attention(
|
||||
q, k, v, self.heads, attn_precision=self.attn_precision,
|
||||
transformer_options=transformer_options,
|
||||
)
|
||||
else:
|
||||
out = _masked_attention(q, k, v, self.heads, mask=mask,
|
||||
attn_precision=self.attn_precision,
|
||||
transformer_options=transformer_options)
|
||||
|
||||
if self.to_gate_logits is not None:
|
||||
gate_logits = self.to_gate_logits(x)
|
||||
b, t, _ = out.shape
|
||||
out = out.view(b, t, self.heads, self.dim_head)
|
||||
out = out * (2.0 * torch.sigmoid(gate_logits)).unsqueeze(-1)
|
||||
out = out.view(b, t, self.heads * self.dim_head)
|
||||
|
||||
return self.to_out(out)
|
||||
|
||||
|
||||
class _CrossAttnPatch:
|
||||
"""Descriptor that binds (impl, mask_fn) as a method onto a cross-attn module."""
|
||||
|
||||
def __init__(self, impl, mask_fn):
|
||||
self.impl = impl
|
||||
self.mask_fn = mask_fn
|
||||
|
||||
def __get__(self, obj, objtype=None):
|
||||
impl, mask_fn = self.impl, self.mask_fn
|
||||
|
||||
def wrapped(self_module, *args, **kwargs):
|
||||
return impl(self_module, mask_fn, *args, **kwargs)
|
||||
|
||||
return types.MethodType(wrapped, obj)
|
||||
|
||||
|
||||
def detect_model_type(model):
|
||||
"""Return (arch, patch_size, temporal_stride) for latent geometry.
|
||||
|
||||
temporal_stride is the VAE's pixel→latent temporal compression factor,
|
||||
used to convert user-facing pixel frame counts to latent frames.
|
||||
"""
|
||||
diff_model = model.model.diffusion_model
|
||||
|
||||
if hasattr(diff_model, "patch_size") and not hasattr(diff_model, "patchifier"):
|
||||
return "wan", tuple(diff_model.patch_size), 4
|
||||
|
||||
if hasattr(diff_model, "patchifier"):
|
||||
return "ltx", (1, 1, 1), int(diff_model.vae_scale_factors[0])
|
||||
|
||||
raise ValueError(
|
||||
f"Unsupported model type: {type(diff_model).__name__}. "
|
||||
f"Currently supports Wan and LTX models."
|
||||
)
|
||||
|
||||
|
||||
def _check_unpatched(model_clone, key):
|
||||
if key in getattr(model_clone, "object_patches", {}):
|
||||
raise RuntimeError(
|
||||
f"PromptRelay: cross-attention forward at '{key}' is already patched by "
|
||||
"another node (e.g. KJNodes NAG). Stacking is not supported — remove the "
|
||||
"conflicting node."
|
||||
)
|
||||
|
||||
|
||||
def apply_patches(model_clone, arch, mask_fn):
|
||||
diffusion_model = model_clone.get_model_object("diffusion_model")
|
||||
|
||||
if arch == "wan":
|
||||
from comfy.ldm.wan.model import WanI2VCrossAttention
|
||||
for idx, block in enumerate(diffusion_model.blocks):
|
||||
key = f"diffusion_model.blocks.{idx}.cross_attn.forward"
|
||||
_check_unpatched(model_clone, key)
|
||||
cross_attn = block.cross_attn
|
||||
impl = _wan_i2v_forward if isinstance(cross_attn, WanI2VCrossAttention) else _wan_t2v_forward
|
||||
model_clone.add_object_patch(key, _CrossAttnPatch(impl, mask_fn).__get__(cross_attn, cross_attn.__class__))
|
||||
return
|
||||
|
||||
if arch == "ltx":
|
||||
for idx, block in enumerate(diffusion_model.transformer_blocks):
|
||||
for attr in ("attn2", "audio_attn2"):
|
||||
module = getattr(block, attr, None)
|
||||
if module is None:
|
||||
continue
|
||||
key = f"diffusion_model.transformer_blocks.{idx}.{attr}.forward"
|
||||
_check_unpatched(model_clone, key)
|
||||
model_clone.add_object_patch(key, _CrossAttnPatch(_ltx_forward, mask_fn).__get__(module, module.__class__))
|
||||
return
|
||||
|
||||
raise ValueError(f"Unknown model arch: {arch}")
|
||||
|
||||
402
prompt_relay.py
402
prompt_relay.py
@@ -1,201 +1,201 @@
|
||||
import logging
|
||||
import math
|
||||
import torch
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def build_temporal_cost(q_token_idx, Lq, Lk, device, dtype, tokens_per_frame):
|
||||
"""Gaussian penalty matrix [Lq, Lk] for video cross-attention (integer frame indexing)."""
|
||||
offset = torch.zeros(Lq, Lk, device=device, dtype=dtype)
|
||||
query_frames = torch.arange(Lq, device=device, dtype=torch.long) // tokens_per_frame
|
||||
|
||||
for seg in q_token_idx:
|
||||
local = seg["local_token_idx"].to(device=device)
|
||||
d = (query_frames.float()[:, None] - seg["midpoint"]).abs()
|
||||
strength = seg.get("strength", 1.0)
|
||||
cost = strength * (torch.relu(d - seg["window"]) ** 2) / (2 * seg["sigma"] ** 2)
|
||||
offset[:, local] = cost.to(offset.dtype)
|
||||
|
||||
return offset
|
||||
|
||||
|
||||
def build_temporal_cost_scaled(q_token_idx, Lq, Lk, device, dtype, latent_frames):
|
||||
"""Penalty matrix for queries that don't map to integer frames (e.g. LTXAV audio tokens)."""
|
||||
offset = torch.zeros(Lq, Lk, device=device, dtype=dtype)
|
||||
query_frames = torch.arange(Lq, device=device, dtype=torch.float32) * latent_frames / Lq
|
||||
|
||||
for seg in q_token_idx:
|
||||
local = seg["local_token_idx"].to(device=device)
|
||||
d = (query_frames[:, None] - seg["midpoint"]).abs()
|
||||
sigma_a = seg.get("sigma_audio", seg["sigma"])
|
||||
window_a = seg.get("window_audio", seg["window"])
|
||||
strength_a = seg.get("strength_audio", 1.0)
|
||||
cost = strength_a * (torch.relu(d - window_a) ** 2) / (2 * sigma_a ** 2)
|
||||
offset[:, local] = cost.to(offset.dtype)
|
||||
|
||||
return offset
|
||||
|
||||
|
||||
def create_mask_fn(q_token_idx, fallback_tokens_per_frame, latent_frames):
|
||||
"""Closure: mask_fn(q, k, transformer_options) -> additive mask or None."""
|
||||
cache = {}
|
||||
max_token_idx = max(int(seg["local_token_idx"].max().item()) for seg in q_token_idx) + 1
|
||||
|
||||
def mask_fn(q, k, transformer_options):
|
||||
Lq, Lk = q.shape[1], k.shape[1]
|
||||
|
||||
if Lq == Lk:
|
||||
return None
|
||||
|
||||
# Only apply on conditional pass — not unconditional (negative prompt)
|
||||
cond_or_uncond = transformer_options.get("cond_or_uncond", [])
|
||||
if 1 in cond_or_uncond and 0 not in cond_or_uncond:
|
||||
return None
|
||||
|
||||
grid_sizes = transformer_options.get("grid_sizes", None)
|
||||
video_tpf = int(grid_sizes[1]) * int(grid_sizes[2]) if grid_sizes is not None else fallback_tokens_per_frame
|
||||
video_lq = latent_frames * video_tpf
|
||||
|
||||
# Skip cross-modal attention — text keys are padded to a fixed length ≥ max_token_idx and != video_lq
|
||||
if Lk == video_lq or Lk < max_token_idx:
|
||||
return None
|
||||
|
||||
mode = "video" if Lq == video_lq else "scaled"
|
||||
|
||||
key = (Lq, Lk, mode, q.device)
|
||||
if key not in cache:
|
||||
if mode == "video":
|
||||
cost = build_temporal_cost(q_token_idx, Lq, Lk, q.device, q.dtype, video_tpf)
|
||||
else:
|
||||
cost = build_temporal_cost_scaled(q_token_idx, Lq, Lk, q.device, q.dtype, latent_frames)
|
||||
log.info(
|
||||
"[PromptRelay] Built penalty matrix (%s): Lq=%d, Lk=%d, nonzero=%d/%d",
|
||||
mode, Lq, Lk, (cost > 0).sum().item(), cost.numel(),
|
||||
)
|
||||
cache[key] = -cost
|
||||
|
||||
return cache[key].to(q.dtype)
|
||||
|
||||
return mask_fn
|
||||
|
||||
|
||||
def build_segments(token_ranges, segment_lengths, epsilon=1e-3, relay_options=None):
|
||||
"""Per-segment metadata for the temporal penalty.
|
||||
|
||||
relay_options (optional dict) overrides per-stream knobs:
|
||||
video_strength, video_window_scale,
|
||||
audio_epsilon, audio_strength, audio_window_scale
|
||||
Audio knobs only affect architectures whose cross-attention takes the scaled
|
||||
(non-integer-frame) path — currently LTX audio_attn2.
|
||||
"""
|
||||
# Paper uses constant sigma = 1/ln(1/epsilon) regardless of segment length
|
||||
sigma = 1.0 / math.log(1.0 / epsilon) if 0 < epsilon < 1 else 0.1448
|
||||
|
||||
opts = relay_options or {}
|
||||
v_strength = opts.get("video_strength", 1.0)
|
||||
v_window_scale = opts.get("video_window_scale", 1.0)
|
||||
a_epsilon = opts.get("audio_epsilon")
|
||||
a_strength = opts.get("audio_strength", 1.0)
|
||||
a_window_scale = opts.get("audio_window_scale", 1.0)
|
||||
|
||||
if a_epsilon is not None and 0 < a_epsilon < 1:
|
||||
sigma_audio = 1.0 / math.log(1.0 / a_epsilon)
|
||||
else:
|
||||
sigma_audio = sigma
|
||||
|
||||
if relay_options:
|
||||
log.info(
|
||||
"[PromptRelay] Advanced options active — video: strength=%.3f window_scale=%.3f | "
|
||||
"audio: epsilon=%s strength=%.3f window_scale=%.3f",
|
||||
v_strength, v_window_scale,
|
||||
f"{a_epsilon:.4f}" if a_epsilon is not None else "inherit",
|
||||
a_strength, a_window_scale,
|
||||
)
|
||||
|
||||
q_token_idx = []
|
||||
frame_cursor = 0
|
||||
|
||||
for (tok_start, tok_end), L in zip(token_ranges, segment_lengths):
|
||||
if L <= 0:
|
||||
frame_cursor += L
|
||||
continue
|
||||
midpoint = (2 * frame_cursor + L) // 2
|
||||
base_window = max(L // 2 - 2, 0)
|
||||
q_token_idx.append({
|
||||
"local_token_idx": torch.arange(tok_start, tok_end),
|
||||
"midpoint": midpoint,
|
||||
"window": max(base_window * v_window_scale, 0.0),
|
||||
"sigma": sigma,
|
||||
"strength": v_strength,
|
||||
"window_audio": max(base_window * a_window_scale, 0.0),
|
||||
"sigma_audio": sigma_audio,
|
||||
"strength_audio": a_strength,
|
||||
})
|
||||
frame_cursor += L
|
||||
|
||||
return q_token_idx
|
||||
|
||||
|
||||
def get_raw_tokenizer(clip):
|
||||
"""Extract the raw SPiece/HF tokenizer from a ComfyUI CLIP object."""
|
||||
tokenizer_wrapper = clip.tokenizer
|
||||
for attr_name in dir(tokenizer_wrapper):
|
||||
if attr_name.startswith("_"):
|
||||
continue
|
||||
inner = getattr(tokenizer_wrapper, attr_name, None)
|
||||
if inner is not None and hasattr(inner, "tokenizer"):
|
||||
return inner.tokenizer
|
||||
|
||||
raise RuntimeError(
|
||||
f"Could not find raw tokenizer on CLIP object. "
|
||||
f"Known attributes: {[a for a in dir(tokenizer_wrapper) if not a.startswith('_')]}"
|
||||
)
|
||||
|
||||
|
||||
def map_token_indices(raw_tokenizer, global_prompt, local_prompts):
|
||||
"""Tokenize global + space-prefixed locals; return (full_prompt, per-local token ranges).
|
||||
|
||||
Uses incremental tokenization to avoid SentencePiece context-dependency issues.
|
||||
"""
|
||||
prefixed_locals = [" " + lp for lp in local_prompts]
|
||||
full_prompt = global_prompt + "".join(prefixed_locals)
|
||||
has_eos = getattr(raw_tokenizer, "add_eos", False)
|
||||
eos_adj = 1 if has_eos else 0
|
||||
|
||||
prev_len = len(raw_tokenizer(global_prompt)["input_ids"]) - eos_adj
|
||||
token_ranges = []
|
||||
built = global_prompt
|
||||
|
||||
for plp in prefixed_locals:
|
||||
built += plp
|
||||
cur_len = len(raw_tokenizer(built)["input_ids"]) - eos_adj
|
||||
if cur_len <= prev_len:
|
||||
raise ValueError(f"Local prompt produced no tokens: '{plp.strip()}'")
|
||||
token_ranges.append((prev_len, cur_len))
|
||||
prev_len = cur_len
|
||||
|
||||
return full_prompt, token_ranges
|
||||
|
||||
|
||||
def distribute_segment_lengths(num_segments, latent_frames, specified_lengths=None):
|
||||
"""Validate or auto-distribute segment frame counts, capped to fit within latent_frames."""
|
||||
if specified_lengths:
|
||||
if len(specified_lengths) != num_segments:
|
||||
raise ValueError(
|
||||
f"Number of segment_lengths ({len(specified_lengths)}) "
|
||||
f"must match number of local prompts ({num_segments})"
|
||||
)
|
||||
lengths = specified_lengths
|
||||
else:
|
||||
# ceil division — matches reference implementation
|
||||
step = -(-latent_frames // num_segments)
|
||||
lengths = [step] * num_segments
|
||||
|
||||
effective = []
|
||||
cursor = 0
|
||||
for L in lengths:
|
||||
end = min(cursor + L, latent_frames)
|
||||
effective.append(max(end - cursor, 0))
|
||||
cursor = end
|
||||
return effective
|
||||
import logging
|
||||
import math
|
||||
import torch
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def build_temporal_cost(q_token_idx, Lq, Lk, device, dtype, tokens_per_frame):
|
||||
"""Gaussian penalty matrix [Lq, Lk] for video cross-attention (integer frame indexing)."""
|
||||
offset = torch.zeros(Lq, Lk, device=device, dtype=dtype)
|
||||
query_frames = torch.arange(Lq, device=device, dtype=torch.long) // tokens_per_frame
|
||||
|
||||
for seg in q_token_idx:
|
||||
local = seg["local_token_idx"].to(device=device)
|
||||
d = (query_frames.float()[:, None] - seg["midpoint"]).abs()
|
||||
strength = seg.get("strength", 1.0)
|
||||
cost = strength * (torch.relu(d - seg["window"]) ** 2) / (2 * seg["sigma"] ** 2)
|
||||
offset[:, local] = cost.to(offset.dtype)
|
||||
|
||||
return offset
|
||||
|
||||
|
||||
def build_temporal_cost_scaled(q_token_idx, Lq, Lk, device, dtype, latent_frames):
|
||||
"""Penalty matrix for queries that don't map to integer frames (e.g. LTXAV audio tokens)."""
|
||||
offset = torch.zeros(Lq, Lk, device=device, dtype=dtype)
|
||||
query_frames = torch.arange(Lq, device=device, dtype=torch.float32) * latent_frames / Lq
|
||||
|
||||
for seg in q_token_idx:
|
||||
local = seg["local_token_idx"].to(device=device)
|
||||
d = (query_frames[:, None] - seg["midpoint"]).abs()
|
||||
sigma_a = seg.get("sigma_audio", seg["sigma"])
|
||||
window_a = seg.get("window_audio", seg["window"])
|
||||
strength_a = seg.get("strength_audio", 1.0)
|
||||
cost = strength_a * (torch.relu(d - window_a) ** 2) / (2 * sigma_a ** 2)
|
||||
offset[:, local] = cost.to(offset.dtype)
|
||||
|
||||
return offset
|
||||
|
||||
|
||||
def create_mask_fn(q_token_idx, fallback_tokens_per_frame, latent_frames):
|
||||
"""Closure: mask_fn(q, k, transformer_options) -> additive mask or None."""
|
||||
cache = {}
|
||||
max_token_idx = max(int(seg["local_token_idx"].max().item()) for seg in q_token_idx) + 1
|
||||
|
||||
def mask_fn(q, k, transformer_options):
|
||||
Lq, Lk = q.shape[1], k.shape[1]
|
||||
|
||||
if Lq == Lk:
|
||||
return None
|
||||
|
||||
# Only apply on conditional pass — not unconditional (negative prompt)
|
||||
cond_or_uncond = transformer_options.get("cond_or_uncond", [])
|
||||
if 1 in cond_or_uncond and 0 not in cond_or_uncond:
|
||||
return None
|
||||
|
||||
grid_sizes = transformer_options.get("grid_sizes", None)
|
||||
video_tpf = int(grid_sizes[1]) * int(grid_sizes[2]) if grid_sizes is not None else fallback_tokens_per_frame
|
||||
video_lq = latent_frames * video_tpf
|
||||
|
||||
# Skip cross-modal attention — text keys are padded to a fixed length ≥ max_token_idx and != video_lq
|
||||
if Lk == video_lq or Lk < max_token_idx:
|
||||
return None
|
||||
|
||||
mode = "video" if Lq == video_lq else "scaled"
|
||||
|
||||
key = (Lq, Lk, mode, q.device)
|
||||
if key not in cache:
|
||||
if mode == "video":
|
||||
cost = build_temporal_cost(q_token_idx, Lq, Lk, q.device, q.dtype, video_tpf)
|
||||
else:
|
||||
cost = build_temporal_cost_scaled(q_token_idx, Lq, Lk, q.device, q.dtype, latent_frames)
|
||||
log.info(
|
||||
"[PromptRelay] Built penalty matrix (%s): Lq=%d, Lk=%d, nonzero=%d/%d",
|
||||
mode, Lq, Lk, (cost > 0).sum().item(), cost.numel(),
|
||||
)
|
||||
cache[key] = -cost
|
||||
|
||||
return cache[key].to(q.dtype)
|
||||
|
||||
return mask_fn
|
||||
|
||||
|
||||
def build_segments(token_ranges, segment_lengths, epsilon=1e-3, relay_options=None):
|
||||
"""Per-segment metadata for the temporal penalty.
|
||||
|
||||
relay_options (optional dict) overrides per-stream knobs:
|
||||
video_strength, video_window_scale,
|
||||
audio_epsilon, audio_strength, audio_window_scale
|
||||
Audio knobs only affect architectures whose cross-attention takes the scaled
|
||||
(non-integer-frame) path — currently LTX audio_attn2.
|
||||
"""
|
||||
# Paper uses constant sigma = 1/ln(1/epsilon) regardless of segment length
|
||||
sigma = 1.0 / math.log(1.0 / epsilon) if 0 < epsilon < 1 else 0.1448
|
||||
|
||||
opts = relay_options or {}
|
||||
v_strength = opts.get("video_strength", 1.0)
|
||||
v_window_scale = opts.get("video_window_scale", 1.0)
|
||||
a_epsilon = opts.get("audio_epsilon")
|
||||
a_strength = opts.get("audio_strength", 1.0)
|
||||
a_window_scale = opts.get("audio_window_scale", 1.0)
|
||||
|
||||
if a_epsilon is not None and 0 < a_epsilon < 1:
|
||||
sigma_audio = 1.0 / math.log(1.0 / a_epsilon)
|
||||
else:
|
||||
sigma_audio = sigma
|
||||
|
||||
if relay_options:
|
||||
log.info(
|
||||
"[PromptRelay] Advanced options active — video: strength=%.3f window_scale=%.3f | "
|
||||
"audio: epsilon=%s strength=%.3f window_scale=%.3f",
|
||||
v_strength, v_window_scale,
|
||||
f"{a_epsilon:.4f}" if a_epsilon is not None else "inherit",
|
||||
a_strength, a_window_scale,
|
||||
)
|
||||
|
||||
q_token_idx = []
|
||||
frame_cursor = 0
|
||||
|
||||
for (tok_start, tok_end), L in zip(token_ranges, segment_lengths):
|
||||
if L <= 0:
|
||||
frame_cursor += L
|
||||
continue
|
||||
midpoint = (2 * frame_cursor + L) // 2
|
||||
base_window = max(L // 2 - 2, 0)
|
||||
q_token_idx.append({
|
||||
"local_token_idx": torch.arange(tok_start, tok_end),
|
||||
"midpoint": midpoint,
|
||||
"window": max(base_window * v_window_scale, 0.0),
|
||||
"sigma": sigma,
|
||||
"strength": v_strength,
|
||||
"window_audio": max(base_window * a_window_scale, 0.0),
|
||||
"sigma_audio": sigma_audio,
|
||||
"strength_audio": a_strength,
|
||||
})
|
||||
frame_cursor += L
|
||||
|
||||
return q_token_idx
|
||||
|
||||
|
||||
def get_raw_tokenizer(clip):
|
||||
"""Extract the raw SPiece/HF tokenizer from a ComfyUI CLIP object."""
|
||||
tokenizer_wrapper = clip.tokenizer
|
||||
for attr_name in dir(tokenizer_wrapper):
|
||||
if attr_name.startswith("_"):
|
||||
continue
|
||||
inner = getattr(tokenizer_wrapper, attr_name, None)
|
||||
if inner is not None and hasattr(inner, "tokenizer"):
|
||||
return inner.tokenizer
|
||||
|
||||
raise RuntimeError(
|
||||
f"Could not find raw tokenizer on CLIP object. "
|
||||
f"Known attributes: {[a for a in dir(tokenizer_wrapper) if not a.startswith('_')]}"
|
||||
)
|
||||
|
||||
|
||||
def map_token_indices(raw_tokenizer, global_prompt, local_prompts):
|
||||
"""Tokenize global + space-prefixed locals; return (full_prompt, per-local token ranges).
|
||||
|
||||
Uses incremental tokenization to avoid SentencePiece context-dependency issues.
|
||||
"""
|
||||
prefixed_locals = [" " + lp for lp in local_prompts]
|
||||
full_prompt = global_prompt + "".join(prefixed_locals)
|
||||
has_eos = getattr(raw_tokenizer, "add_eos", False)
|
||||
eos_adj = 1 if has_eos else 0
|
||||
|
||||
prev_len = len(raw_tokenizer(global_prompt)["input_ids"]) - eos_adj
|
||||
token_ranges = []
|
||||
built = global_prompt
|
||||
|
||||
for plp in prefixed_locals:
|
||||
built += plp
|
||||
cur_len = len(raw_tokenizer(built)["input_ids"]) - eos_adj
|
||||
if cur_len <= prev_len:
|
||||
raise ValueError(f"Local prompt produced no tokens: '{plp.strip()}'")
|
||||
token_ranges.append((prev_len, cur_len))
|
||||
prev_len = cur_len
|
||||
|
||||
return full_prompt, token_ranges
|
||||
|
||||
|
||||
def distribute_segment_lengths(num_segments, latent_frames, specified_lengths=None):
|
||||
"""Validate or auto-distribute segment frame counts, capped to fit within latent_frames."""
|
||||
if specified_lengths:
|
||||
if len(specified_lengths) != num_segments:
|
||||
raise ValueError(
|
||||
f"Number of segment_lengths ({len(specified_lengths)}) "
|
||||
f"must match number of local prompts ({num_segments})"
|
||||
)
|
||||
lengths = specified_lengths
|
||||
else:
|
||||
# ceil division — matches reference implementation
|
||||
step = -(-latent_frames // num_segments)
|
||||
lengths = [step] * num_segments
|
||||
|
||||
effective = []
|
||||
cursor = 0
|
||||
for L in lengths:
|
||||
end = min(cursor + L, latent_frames)
|
||||
effective.append(max(end - cursor, 0))
|
||||
cursor = end
|
||||
return effective
|
||||
|
||||
@@ -1,36 +1,36 @@
|
||||
[project]
|
||||
name = "WhatDreamsCost-ComfyUI"
|
||||
description = "A variety of custom ComfyUI nodes and workflows for creatives."
|
||||
version = "1.3.9"
|
||||
license = {file = "LICENSE"}
|
||||
# classifiers = [
|
||||
# # For OS-independent nodes (works on all operating systems)
|
||||
# "Operating System :: OS Independent",
|
||||
#
|
||||
# # OR for OS-specific nodes, specify the supported systems:
|
||||
# "Operating System :: Microsoft :: Windows", # Windows specific
|
||||
# "Operating System :: POSIX :: Linux", # Linux specific
|
||||
# "Operating System :: MacOS", # macOS specific
|
||||
#
|
||||
# # GPU Accelerator support. Pick the ones that are supported by your extension.
|
||||
# "Environment :: GPU :: NVIDIA CUDA", # NVIDIA CUDA support
|
||||
# "Environment :: GPU :: AMD ROCm", # AMD ROCm support
|
||||
# "Environment :: GPU :: Intel Arc", # Intel Arc support
|
||||
# "Environment :: NPU :: Huawei Ascend", # Huawei Ascend support
|
||||
# "Environment :: GPU :: Apple Metal", # Apple Metal support
|
||||
# ]
|
||||
|
||||
|
||||
[project.urls]
|
||||
Repository = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI"
|
||||
# Used by Comfy Registry https://registry.comfy.org
|
||||
Documentation = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/wiki"
|
||||
"Bug Tracker" = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/issues"
|
||||
|
||||
[tool.comfy]
|
||||
PublisherId = "whatdreamscost"
|
||||
DisplayName = "WhatDreamsCost-ComfyUI"
|
||||
Icon = "https://raw.githubusercontent.com/WhatDreamsCost/MediaSyncer/refs/heads/main/Icon.png"
|
||||
includes = []
|
||||
# "requires-comfyui" = ">=1.0.0" # ComfyUI version compatibility
|
||||
|
||||
[project]
|
||||
name = "WhatDreamsCost-ComfyUI"
|
||||
description = "A variety of custom ComfyUI nodes and workflows for creatives."
|
||||
version = "1.3.9"
|
||||
license = {file = "LICENSE"}
|
||||
# classifiers = [
|
||||
# # For OS-independent nodes (works on all operating systems)
|
||||
# "Operating System :: OS Independent",
|
||||
#
|
||||
# # OR for OS-specific nodes, specify the supported systems:
|
||||
# "Operating System :: Microsoft :: Windows", # Windows specific
|
||||
# "Operating System :: POSIX :: Linux", # Linux specific
|
||||
# "Operating System :: MacOS", # macOS specific
|
||||
#
|
||||
# # GPU Accelerator support. Pick the ones that are supported by your extension.
|
||||
# "Environment :: GPU :: NVIDIA CUDA", # NVIDIA CUDA support
|
||||
# "Environment :: GPU :: AMD ROCm", # AMD ROCm support
|
||||
# "Environment :: GPU :: Intel Arc", # Intel Arc support
|
||||
# "Environment :: NPU :: Huawei Ascend", # Huawei Ascend support
|
||||
# "Environment :: GPU :: Apple Metal", # Apple Metal support
|
||||
# ]
|
||||
|
||||
|
||||
[project.urls]
|
||||
Repository = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI"
|
||||
# Used by Comfy Registry https://registry.comfy.org
|
||||
Documentation = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/wiki"
|
||||
"Bug Tracker" = "https://github.com/WhatDreamsCost/WhatDreamsCost-ComfyUI/issues"
|
||||
|
||||
[tool.comfy]
|
||||
PublisherId = "whatdreamscost"
|
||||
DisplayName = "WhatDreamsCost-ComfyUI"
|
||||
Icon = "https://raw.githubusercontent.com/WhatDreamsCost/MediaSyncer/refs/heads/main/Icon.png"
|
||||
includes = []
|
||||
# "requires-comfyui" = ">=1.0.0" # ComfyUI version compatibility
|
||||
|
||||
|
||||
@@ -1,51 +1,51 @@
|
||||
import re
|
||||
import math
|
||||
|
||||
class SpeechLengthCalculator:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"text": ("STRING", {"multiline": True, "default": 'Enter your script here. "Make sure to put spoken words inside quotes!"'}),
|
||||
"fps": ("INT", {"default": 24, "min": 1, "max": 120, "step": 1}),
|
||||
"additional_time": ("FLOAT", {"default": 0.0, "min": 0.0, "step": 0.1}),
|
||||
},
|
||||
"optional": {
|
||||
"text_input": ("STRING", {"forceInput": True}),
|
||||
}
|
||||
}
|
||||
|
||||
# Added "STRING" to RETURN_TYPES
|
||||
RETURN_TYPES = ("INT", "INT", "INT", "STRING")
|
||||
# Added "text" to RETURN_NAMES
|
||||
RETURN_NAMES = ("slow_frame_count", "average_frame_count", "fast_frame_count", "text")
|
||||
FUNCTION = "calculate_speech"
|
||||
CATEGORY = "WhatDreamsCost"
|
||||
|
||||
def calculate_speech(self, text, fps, additional_time=0.0, text_input=None):
|
||||
# Prioritize the connected text_input if provided, otherwise fallback to the text widget
|
||||
active_text = text_input if (text_input is not None and isinstance(text_input, str) and text_input.strip() != "") else text
|
||||
|
||||
# Regex to find words inside double quotes, single quotes, or smart quotes
|
||||
matches = re.findall(r'"([^"]*)"|\'([^\']*)\'|“([^”]*)”|‘([^’]*)’', active_text)
|
||||
|
||||
# Extract matches, handling all possible captured groups from the regex
|
||||
quoted_text = " ".join([next((g for g in m if g), "") for m in matches])
|
||||
|
||||
# Split by whitespace to get words and count them
|
||||
words = quoted_text.split()
|
||||
word_count = len(words)
|
||||
|
||||
def calc_frames(wpm):
|
||||
if word_count == 0 and additional_time == 0:
|
||||
return 0
|
||||
minutes = word_count / wpm
|
||||
seconds = (minutes * 60) + additional_time
|
||||
return math.ceil(seconds * fps)
|
||||
|
||||
slow_frames = calc_frames(100)
|
||||
avg_frames = calc_frames(130)
|
||||
fast_frames = calc_frames(160)
|
||||
|
||||
# Added active_text as the 4th returned value
|
||||
import re
|
||||
import math
|
||||
|
||||
class SpeechLengthCalculator:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"text": ("STRING", {"multiline": True, "default": 'Enter your script here. "Make sure to put spoken words inside quotes!"'}),
|
||||
"fps": ("INT", {"default": 24, "min": 1, "max": 120, "step": 1}),
|
||||
"additional_time": ("FLOAT", {"default": 0.0, "min": 0.0, "step": 0.1}),
|
||||
},
|
||||
"optional": {
|
||||
"text_input": ("STRING", {"forceInput": True}),
|
||||
}
|
||||
}
|
||||
|
||||
# Added "STRING" to RETURN_TYPES
|
||||
RETURN_TYPES = ("INT", "INT", "INT", "STRING")
|
||||
# Added "text" to RETURN_NAMES
|
||||
RETURN_NAMES = ("slow_frame_count", "average_frame_count", "fast_frame_count", "text")
|
||||
FUNCTION = "calculate_speech"
|
||||
CATEGORY = "WhatDreamsCost"
|
||||
|
||||
def calculate_speech(self, text, fps, additional_time=0.0, text_input=None):
|
||||
# Prioritize the connected text_input if provided, otherwise fallback to the text widget
|
||||
active_text = text_input if (text_input is not None and isinstance(text_input, str) and text_input.strip() != "") else text
|
||||
|
||||
# Regex to find words inside double quotes, single quotes, or smart quotes
|
||||
matches = re.findall(r'"([^"]*)"|\'([^\']*)\'|“([^”]*)”|‘([^’]*)’', active_text)
|
||||
|
||||
# Extract matches, handling all possible captured groups from the regex
|
||||
quoted_text = " ".join([next((g for g in m if g), "") for m in matches])
|
||||
|
||||
# Split by whitespace to get words and count them
|
||||
words = quoted_text.split()
|
||||
word_count = len(words)
|
||||
|
||||
def calc_frames(wpm):
|
||||
if word_count == 0 and additional_time == 0:
|
||||
return 0
|
||||
minutes = word_count / wpm
|
||||
seconds = (minutes * 60) + additional_time
|
||||
return math.ceil(seconds * fps)
|
||||
|
||||
slow_frames = calc_frames(100)
|
||||
avg_frames = calc_frames(130)
|
||||
fast_frames = calc_frames(160)
|
||||
|
||||
# Added active_text as the 4th returned value
|
||||
return (slow_frames, avg_frames, fast_frames, active_text)
|
||||
Reference in New Issue
Block a user