Files
comfyui-workflows/optimized/T2I/ideogram/README.md
T

130 lines
6.5 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Ideogram 4 — author-informed T2I tests
This section follows the model card and author workflow: structured Ideogram 4
JSON captions, AuraFlow shift 5.25, StarSampler, and CFG 1.0. It is intended
for controlled quality/time comparisons on the RTX 3070 server.
## Current direction
- Main models: `ideogram4_int8_convrot.safetensors` and
`Fal_Ideo_4_NSFW-int8_convrot.safetensors`, tested separately
- Text encoder: `qwen3vl_8b_w4a8.safetensors`, type `ideogram4`
- VAE: `flux2-vae.safetensors`
- Prompt: Ideogram 4 structured JSON caption
- Sampling: AuraFlow shift `5.25`; StarSampler, Euler, simple scheduler
- Steps: author workflow 8; model-card reference 10
- CFG: `1.0`
- Seed: 20261012 for the controlled baseline
The Fal model is retained as a comparison only; it produced more stylised,
noisier results. The hosted `IdeogramV4` API node was not used because this
server is not authenticated for Comfy API partner nodes.
## Author-inspired comparison
The model author publishes a workflow at
[`Winnougan/Ideogram_Instant_NSFW`](https://huggingface.co/Winnougan/Ideogram_Instant_NSFW).
It suggests a useful comparison branch rather than a replacement for the
local baseline:
- `Ideogram4PromptBuilderKJ` for structured prompt construction
- `ModelSamplingAuraFlow` with `shift=5.25`
- `StarSampler` with 8 steps, Euler, simple schedule, and CFG 1.0
- Fal Ideogram checkpoint with the local `flux2-vae.safetensors`
The author's graph references `Boogu\\qwen3vl_8b_int8_convrot.safetensors`, which
is not installed here, so the comparison uses the available
`qwen3vl_8b_w4a8.safetensors`. Complete settings are recorded in
[`author-inspired-settings.json`](author-inspired-settings.json).
These settings remain comparison-only until a fixed prompt and seed are tested
at 768×1024 and reviewed for typography, anatomy, texture, composition, and
runtime.
The model card also states that the fine-tune expects a structured JSON caption
with `high_level_description` and `compositional_deconstruction` fields, and
gives 10 steps with guidance scale 1.0 as its reference usage. The published
ComfyUI graph uses 8 steps, so both settings are worth measuring separately.
The card contains several unfinished placeholder sections, so its training and
fine-tuning details should not be treated as verified claims.
## Initial observations
- The 512×512 local smoke test completed successfully.
- The 768×1024 standard Ideogram 4 run took about 6 minutes at 12 steps.
- The 768×1024 Fal comparison took about 7 minutes at 12 steps.
- The 8-step Fal run completed in about 5 minutes 18 seconds, but remained
painterly and noisy.
## Current smoke tests
The first smoke tests use the same 512×512 structured caption and seed before
full-size tests are queued. Exact prompt IDs are recorded in `smoke-tests.json`.
| ID | Model | Steps | Sampler | Scheduler | Shift | CFG | Status |
|---|---|---:|---|---|---:|---:|---|
| IDEO-SMOKE-FAL-10 | Fal Ideogram | 10 | Euler | simple | 5.25 | 1 | Complete |
| IDEO-SMOKE-LOCAL-08 | Local Ideogram | 8 | Euler | simple | 5.25 | 1 | Complete |
Smoke results: both completed without node errors. Fal took 45.483 seconds and
was visually strong. Local took 29.268 seconds and was coherent but notably
underexposed. The next full-size test should therefore start with Fal; local
lighting/shift changes should be isolated rather than mixed into sampler tests.
## Revised quality/time matrix
The previous free-form-prompt/dual-guider queue was stopped because its
outputs did not meet the quality bar. Those IDs remain historical in
`submissions.json`; they are not part of this new matrix.
Keep the prompt and seed fixed for sampler tests. Keep all other settings fixed
when changing one variable. A row is not promoted until detail, anatomy,
composition, text fidelity, artefacts, and runtime are reviewed.
| ID | Model | Resolution | Steps | Sampler | Scheduler | Shift | CFG | Purpose | Status |
|---|---|---:|---:|---|---|---:|---:|---|---|
| IDEO-AUTHOR-01 | Fal | 768×1024 | 8 | Euler | simple | 5.25 | 1 | Author sampling | Complete |
| IDEO-AUTHOR-02 | Fal | 768×1024 | 8 | Euler | simple | 5.25 | 1 | Fal author sampling | Complete |
| IDEO-CARD-10 | Fal | 768×1024 | 10 | Euler | simple | 5.25 | 1 | Model-card step count | Complete |
| IDEO-SAMPLER-01 | Fal | 768×1024 | 8 | res_multistep | simple | 5.25 | 1 | Sampler comparison | Complete |
| IDEO-SAMPLER-02 | Fal | 768×1024 | 8 | euler_ancestral | simple | 5.25 | 1 | Sampler comparison | Complete |
| IDEO-SCHED-01 | Fal | 768×1024 | 8 | Euler | beta | 5.25 | 1 | Scheduler comparison | Complete |
| IDEO-SHIFT-01 | Fal | 768×1024 | 8 | Euler | simple | 3.0 | 1 | Shift comparison | Complete |
| IDEO-SHIFT-02 | Fal | 768×1024 | 8 | Euler | simple | 7.0 | 1 | Shift comparison | Complete |
| IDEO-STEPS-01 | Fal | 768×1024 | 6 | Euler | simple | 5.25 | 1 | Speed lower bound | Complete |
Quality ranking: `IDEO-CARD-10`, `IDEO-SCHED-01`, `IDEO-SHIFT-02`,
`IDEO-AUTHOR-02`, `IDEO-AUTHOR-01`, `IDEO-SAMPLER-02`, `IDEO-STEPS-01`,
`IDEO-SHIFT-01`, `IDEO-SAMPLER-01`. Several lower-ranked outputs contain a
thin whisker/tendril artefact near the dragon's mouth.
Results belong in `results/<test-id>.png` with a matching JSON record containing
the exact prompt, seed, resolution, settings, runtime, and visual verdict.
## Reuse
Replace `STRUCTURED_JSON_CAPTION` in `workflow-author-inspired.json` with the
caption from `smoke-tests.json` or a new test caption. Keep the seed and all
other settings fixed while changing one matrix variable.
## Magazine-cover settings matrix
The earlier magazine batches were discarded because their prompts and seeds
varied between runs. The replacement matrix uses one fixed structured magazine
caption, seed `20261234`, the Fal checkpoint, and the same VAE/text encoder.
Only the listed technical variable changes per row. Prompt IDs and exact values
are tracked in [`magazine-cover-matrix.json`](magazine-cover-matrix.json).
| Group | Rows | Variable under test |
|---|---|---|
| Baseline/steps | 01–03 | 10, 8, and 6 steps |
| Sampler | 04–05 | `res_multistep`, `euler_ancestral` |
| Scheduler | 06–07 | `beta`, `sgm_uniform` |
| CFG | 08–09 | 1.5 and 2.0 versus baseline 1.0 |
| AuraFlow shift | 10–11 | 3.0 and 7.0 versus baseline 5.25 |
| Resolution | 12 | 512×768 versus 768×1024 |
Expanded rows 13–24 add `euler_cfg_pp`, `heun`, and `dpmpp_2m`; Karras and
exponential schedulers; CFG 0.8, 1.2, and 3.0; AuraFlow shifts 4.0 and 6.0;
12 steps; and a 1024×1024 resolution test. They use the same fixed caption
and seed as rows 01–12.