Reset Flux 3 Klein T2I results

This commit is contained in:
Morpheus
2026-10-28 17:07:46 +00:00
parent f5d3cddb8a
commit 4caccae494
18 changed files with 57 additions and 2594 deletions
+1 -1
View File
@@ -8,7 +8,7 @@ matrix; visual review must use the same category fixtures and resolution.
|---|---|---|---:|---|---|
| T2I | Flux.2 Klein | shared T2I prompt | 1088×1920 comparison target | Matrix/baseline recorded | Existing baseline |
| T2I | Qwen Image 2.1 | shared T2I prompt | 1088×1920 comparison target | 88/88 recorded | Existing best recorded in category README |
| T2I | FLUX.3 Klein | shared T2I prompt | 1088×1920 comparison target | 432-cell sampler matrix recorded | Visual cross-model ranking pending |
| T2I | FLUX.3 Klein | shared T2I prompt | 1088×1920 comparison target | Fresh 432-cell sampler matrix planned (0/432) | Results pending |
| T2I | Ideogram 4 | structured magazine prompt | varies by magazine matrix | 24-row matrix complete | Separate prompt family; not directly comparable to portrait rows |
| I2I | Flux.2 Klein | `character_reference.png` + shared I2I prompt | 1080×1920 category target | Matrix planned/untested | Candidate baseline |
| I2I | Qwen Image 2.1 | `character_reference.png` + shared I2I prompt | 1080×1920 category target | 34-row matrix recorded | Existing best recorded in category README |
+1 -1
View File
@@ -14,7 +14,7 @@ This summary is maintained across the model-specific test plans. A result is onl
|---|---|---:|---|---|
| Flux.2 Klein | F-T2I-03 | 220.3 s | [F-T2I-03.png](flux/results/F-T2I-03.png) | Current best; 1080×1920 target |
| Qwen Image 2.1 | Q-T2I-57 | 110.2 s | [Q-T2I-57.png](qwen/results/Q-T2I-57.png) | Current best; direct 1088×1920 latent; res_multistep, beta, 25 steps, CFG 2.5 |
| FLUX.3 Klein | F3I-T2I-01 / sampler matrix | recorded | [best5](flux3image/results/best5/) | Complete local matrix; canonical comparison resolution 1088×1920; visual cross-model ranking pending |
| FLUX.3 Klein | F3I-MATRIX | planned (0/432) | — | Fresh local matrix; canonical comparison resolution 1088×1920 |
| Ideogram 4 | MAG-MATRIX-24 | 30.154 s | [MAG-MATRIX-24.png](ideogram/results/magazine-matrix/MAG-MATRIX-24.png) | Complete magazine-cover matrix; different structured prompt, not directly comparable to the shared portrait prompt |
Store each T2I test under the selected model's `results/<test-id>.png` and `results/<test-id>.json` using the table ID as the label.
@@ -1,86 +0,0 @@
# FLUX 3 Image sampler/scheduler matrix
## Test conditions
- **Runs:** 432 (48 samplers × 9 schedulers)
- **Model:** `flux3-image-klein-int8-convrot.safetensors`
- **Text encoder:** `qwen3vl_4b_int8_convrot.safetensors` (`type=flux3`)
- **VAE:** `flux2-vae.safetensors`
- **Prompt:** [`prompt.txt`](prompt.txt), SHA-256 `5802909ebe894260b70a3977430e08aaa95732e0559749d8cb42e2f5ecf3ab6f`
- **Output:** 1088 × 1920 (the 1080 × 1920 target snapped to the model-compatible width)
- **Steps:** 4
- **Seed:** 310001
- **Result:** 432/432 successful
- **Total execution time:** 3726.104 seconds
- **Mean execution time:** 8.625 seconds
The complete per-cell record is in [`results/F3I-MATRIX.json`](results/F3I-MATRIX.json). Each cell contains the status, measured runtime, and the `F3I-M-SS-DD` output prefix used by the server, where `SS` is the sampler index and `DD` is the scheduler index.
## Runtime findings
Fastest sampler means across all nine schedulers:
- `TBG Flux2 Sampler` — 6.257 s
- `pid_sde` — 6.359 s
- `euler` — 6.407 s
- `dpm_fast` — 6.408 s
- `res_multistep` — 6.412 s
- `cfgpp_ud10_ab` — 6.415 s
- `ipndm` — 6.417 s
- `ipndm_v` — 6.424 s
- `ddim` — 6.428 s
- `dpmpp_2m` — 6.429 s
Scheduler means across all 48 samplers:
- `sgm_uniform` — 8.022 s
- `simple` — 8.063 s
- `linear_quadratic` — 8.265 s
- `beta` — 8.374 s
- `kl_optimal` — 8.816 s
- `karras` — 8.829 s
- `exponential` — 8.919 s
- `normal` — 8.921 s
- `ddim_uniform` — 9.417 s
`dpm_adaptive` is a clear runtime outlier. Its slowest cells were:
- `dpm_adaptive` + `ddim_uniform` — 87.590 s
- `dpm_adaptive` + `normal` — 61.760 s
- `dpm_adaptive` + `exponential` — 61.244 s
- `dpm_adaptive` + `kl_optimal` — 57.667 s
- `dpm_adaptive` + `karras` — 57.582 s
The fastest individual cell was `euler` + `exponential` at 6.199 s. Runtime alone is not a quality score.
## Visual spot-check
Representative generated images were inspected after the automated run:
- `F3I-M-01-01` (`euler` + `simple`) was a strong baseline: coherent subject, composition, anatomy, and useful prompt adherence.
- `F3I-M-01-04` (`euler` + `exponential`) was visibly degraded in the inspected sample, with garbled scoreboard text and smeared detail.
- `F3I-M-16-05` (`dpm_adaptive` + `ddim_uniform`) showed severe anatomy duplication/extra limbs, garbled text, and texture artifacts.
- `F3I-M-48-09` (`uni_pc_bh2` + `kl_optimal`) was overprocessed in the inspected sample, with distorted texture/anatomy and garbled text.
These are representative spot-checks, not a claim that four images fully rank all 432 outputs. The matrix proves that every combination executes successfully on the installed local model; a full perceptual ranking would require reviewing all 432 images or adding an image-quality scoring pass.
## Recommendation
Keep **Euler + Simple** as the default comparison baseline. It is fast, already aligned with the native workflow path, and produced the strongest inspected sample. Treat `sgm_uniform` as a similarly fast scheduler candidate for follow-up visual testing. Avoid `dpm_adaptive` for routine generation because its runtime is unpredictable and its inspected sample was poor. Keep the remaining combinations available as experimental options until they receive visual review.
No generated server files were deleted during collection. The matrix outputs remain under the server prefix `fluxExperiments/flux3image/matrix/` so the visual findings can be revisited.
## Best five reviewed outputs
These are the five strongest images from the visual review, ordered best first.
They are copied into [`results/best5/`](results/best5/).
1. [`01-euler-simple.png`](results/best5/01-euler-simple.png) — `euler` + `simple`
2. [`02-euler-sgm_uniform.png`](results/best5/02-euler-sgm_uniform.png) — `euler` + `sgm_uniform`
3. [`03-dpmpp_2m-simple.png`](results/best5/03-dpmpp_2m-simple.png) — `dpmpp_2m` + `simple`
4. [`04-res_multistep-simple.png`](results/best5/04-res_multistep-simple.png) — `res_multistep` + `simple`
5. [`05-ddim-simple.png`](results/best5/05-ddim-simple.png) — `ddim` + `simple`
The first two are very close in appearance because this low-step configuration
produced nearly identical results for those cells. The ranking is a visual
selection, not a statistically significant model-quality benchmark.
+33 -73
View File
@@ -1,84 +1,44 @@
# FLUX 3 Image / Klein
This is the canonical local FLUX.3 Klein text-to-image workflow and test
matrix. New reference-based tests belong in `optimized/I2I/flux3klein/` and
`optimized/MultipleRef2I/flux3klein/`; do not create a second T2I directory.
The canonical cross-model comparison row is `1088×1920`; the other recorded
resolution rows are explicitly labelled scaling studies.
This is the canonical local FLUX.3 Klein text-to-image workflow and clean
sampler/scheduler test plan. Reference-based tests belong in
`optimized/I2I/flux3klein/` and `optimized/MultipleRef2I/flux3klein/`.
This directory contains the first controlled workflow and test plan for the
server's locally installed FLUX 3 Image Klein model. The workflow does not use
No results have been retained for this fresh run. The workflow does not use
the remote `Flux3ImageNode` API.
## Workflow
## Fixed comparison inputs
- `workflow.json` — API-format prompt for `/prompt` automation.
- `prompt.txt` — fixed baseline prompt.
- `variables.json` — fixed values and allowed test variables.
- `results/` — output images and matching metadata records.
- Workflow: `workflow.json`
- Prompt: `prompt.txt`
- Model: `flux3-image-klein-int8-convrot.safetensors`
- Text encoder: `qwen3vl_4b_int8_convrot.safetensors` (`type=flux3`)
- VAE: `flux2-vae.safetensors`
- Resolution: **1088×1920**
- References: none
- Seed: `310001`
- Steps: `4`
- Flux Guidance: `3.5`
The local graph uses `flux3-image-klein-int8-convrot.safetensors`,
`qwen3vl_4b_int8_convrot.safetensors` with `type=flux3`, `flux2-vae.safetensors`,
Flux guidance, Euler sampling, and the native ComfyUI sampler chain. The
remote API node is not used because it requires a separate Comfy API login.
The graph uses `FluxGuidance`, not `CFGGuider`; therefore this T2I suite
records Flux Guidance rather than calling it CFG. Prompt, resolution, seed,
steps, guidance, and reference count stay fixed while sampler and scheduler
vary.
## Test matrix
## Fresh test matrix
The first pass changes one variable at a time. The prompt, aspect ratio,
grounding state, safety tolerance, and reference count remain fixed unless a
row explicitly changes them.
The matrix contains **48 samplers × 9 schedulers = 432 planned runs**. All
rows are currently **Planned**. The complete sampler and scheduler lists are
stored in `variables.json`; each executed cell must receive a result record in
`results/` with its exact settings, runtime, output path, dimensions, and
error or visual notes.
| ID | Purpose | Resolution | References | Grounding | Seed | Status | Runtime (s) | Verdict |
|---|---|---:|---:|---|---:|---|---:|---|
| F3I-T2I-01 | Shared-prompt local baseline | 1080×1920 target | 0 | Off | 310001 | **Passed** | 19.6 | 1088×1920 output generated |
| F3I-T2I-02 | Lower-cost portrait comparison | 0.75K target | 0 | Off | 310002 | **Passed** | 5.9 | 768×1360 output generated |
| F3I-T2I-03 | Detail/quality portrait comparison | 1.5K target | 0 | Off | 310003 | **Passed** | 13.9 | 1536×2720 output generated |
| F3I-T2I-04 | Higher-resolution portrait feasibility | 2K target | 0 | Off | 310004 | **Passed** | 47.2 | 2048×3648 output generated |
| F3I-T2I-05 | Grounded prompt comparison | 1K | 0 | On | 310005 | **N/A local** | — | Grounding belongs to remote API node |
| F3I-I2I-01 | Single-reference edit | 1K | 1 | Off | 310006 | **Passed** | 33.6 | 1024×1024 output generated |
| Matrix | Resolution | References | Seed | Guidance | Steps | Samplers | Schedulers | Status |
|---|---:|---:|---:|---:|---:|---:|---:|---|
| F3I-MATRIX | 1088×1920 | 0 | 310001 | 3.5 | 4 | 48 | 9 | Planned (0/432) |
## Evaluation
Record the exact prompt, seed, resolution, reference filename/hash, runtime,
output dimensions, success/failure, and visual notes in `results/<test-id>.json`.
Keep the first successful baseline as the comparison target. Do not promote a
larger resolution solely because it is larger; judge detail, composition,
prompt adherence, artifacts, and cost/runtime together.
The API node's seed controls rerun identity but the service may remain
nondeterministic. That is recorded as a property of the test, not treated as a
failure.
## Matrix results
The initial resolution and reference runs were calibration tests with a short
fox prompt. They remain useful for checking graph/model compatibility but are
superseded for model comparison by the shared prompt baseline. The native
reference conditioning path passed with `Base001.png`. The grounding row is not
run locally because grounding is a capability of the authenticated remote
`Flux3ImageNode`, which is intentionally excluded from this repository's
workflow.
## Initial run
The earlier API-node probe is retained as `results/F3I-T2I-01-remote-api.json`
as an environment diagnostic: it returned `Unauthorized` before generation.
The local workflow was then corrected to use the Flux 3-compatible
Qwen3-VL 4B encoder and 128-channel `flux2-vae.safetensors`; that run passed
and is recorded in `results/F3I-T2I-01.json`.
## Full sampler/scheduler matrix
The local model was tested with every available sampler and scheduler: 48 × 9
combinations, for 432 total runs. All 432 completed successfully at 1088 ×
1920, four steps, seed 310001, using the shared prompt. The complete cell-level
results, runtimes, and server output prefixes are recorded in
[`results/F3I-MATRIX.json`](results/F3I-MATRIX.json). The analysis and visual
spot-check findings are in [`MATRIX-ANALYSIS.md`](MATRIX-ANALYSIS.md).
The recommended baseline remains **Euler + Simple**. `dpm_adaptive` is not
recommended for routine use because it was an extreme runtime outlier; runtime
success does not by itself establish visual quality.
The five strongest visually reviewed outputs are collected in
[`results/best5/`](results/best5/), ordered best first.
Do not silently substitute a rejected sampler or scheduler. Record failures
as results, preserve the fixed comparison inputs, and update this table only
from the actual result records. The first successful `euler + simple` cell is
the compatibility baseline; no visual winner should be declared until the
outputs have been reviewed under the same prompt and resolution.
@@ -1,12 +0,0 @@
{
"test_id": "F3I-I2I-01",
"status": "passed",
"prompt_id": "80876b8b-afcb-4ba6-b6f8-eb0b4916f362",
"reference": "Base001.png",
"resolution": [1024, 1024],
"seed": 310006,
"steps": 16,
"alpha": 16.0,
"runtime_seconds": 33.6,
"output": {"filename": "F3I-I2I-01_00001_.png", "subfolder": "fluxExperiments\\flux3image", "width": 1024, "height": 1024}
}
File diff suppressed because it is too large Load Diff
@@ -1,10 +0,0 @@
{
"test_id": "F3I-T2I-01-remote-api",
"status": "failed",
"verdict": "Diagnostic only: remote Flux 3 API requires authentication",
"prompt_id": "65659b7d-a050-4076-8d8f-1abb461a94db",
"node_type": "Flux3ImageNode",
"error": "Unauthorized: Please login first to use this node.",
"runtime_seconds": 0.0,
"used_by_tracked_workflow": false
}
@@ -1,31 +0,0 @@
{
"test_id": "F3I-T2I-01",
"status": "passed",
"verdict": "Shared comparison prompt completed successfully with local native Klein graph",
"prompt_id": "dbc97923-d5c6-48a8-aeb9-aceac2e7396d",
"configuration": {
"resolution": "1080x1920 target; 1088x1920 actual",
"aspect_ratio": "1:1",
"grounding": false,
"safety_tolerance": 4,
"seed": 310001,
"references": 0
},
"error": {
"node_id": "1",
"node_type": "Flux3ImageNode",
"message": "Unauthorized: Please login first to use this node."
},
"runtime_seconds": 19.6,
"model": "flux3-image-klein-int8-convrot.safetensors",
"text_encoder": "qwen3vl_4b_int8_convrot.safetensors",
"vae": "flux2-vae.safetensors",
"prompt_sha256": "5802909ebe894260b70a3977430e08aaa95732e0559749d8cb42e2f5ecf3ab6f",
"output": {
"filename": "F3I-T2I-01_00002_.png",
"subfolder": "fluxExperiments\\flux3image",
"type": "output",
"width": 1088,
"height": 1920
}
}
@@ -1,10 +0,0 @@
{
"test_id": "F3I-T2I-02",
"status": "passed",
"prompt_id": "c60b883c-d40f-4c0e-a9e9-de8a9cd23c6b",
"resolution": [768, 1360],
"seed": 310002,
"runtime_seconds": 5.9,
"prompt_sha256": "5802909ebe894260b70a3977430e08aaa95732e0559749d8cb42e2f5ecf3ab6f",
"output": {"filename": "F3I-T2I-02_00002_.png", "subfolder": "fluxExperiments\\flux3image", "width": 768, "height": 1360}
}
@@ -1,10 +0,0 @@
{
"test_id": "F3I-T2I-03",
"status": "passed",
"prompt_id": "60c68b18-587f-40d1-9e59-1140a57761e5",
"resolution": [1536, 2720],
"seed": 310003,
"runtime_seconds": 13.9,
"prompt_sha256": "5802909ebe894260b70a3977430e08aaa95732e0559749d8cb42e2f5ecf3ab6f",
"output": {"filename": "F3I-T2I-03_00002_.png", "subfolder": "fluxExperiments\\flux3image", "width": 1536, "height": 2720}
}
@@ -1,10 +0,0 @@
{
"test_id": "F3I-T2I-04",
"status": "passed",
"prompt_id": "49938a50-d34c-4360-b2f0-9acef37048b4",
"resolution": [2048, 3648],
"seed": 310004,
"runtime_seconds": 47.2,
"prompt_sha256": "5802909ebe894260b70a3977430e08aaa95732e0559749d8cb42e2f5ecf3ab6f",
"output": {"filename": "F3I-T2I-04_00002_.png", "subfolder": "fluxExperiments\\flux3image", "width": 2048, "height": 3648}
}
Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.0 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.0 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.1 MiB

+22 -15
View File
@@ -4,25 +4,32 @@
"vae": "flux2-vae.safetensors (128-channel / 16x latent decoder)",
"fixed": {
"prompt_file": "prompt.txt",
"aspect_ratio": "1:1",
"width": 1088,
"height": 1920,
"guidance": 3.5,
"sampler": "euler",
"scheduler": "simple",
"steps": 4,
"seed": 310001,
"image_references": 0,
"remote_api": false
},
"test_ids": [
"F3I-T2I-01",
"F3I-T2I-02",
"F3I-T2I-03",
"F3I-T2I-04",
"F3I-T2I-05",
"F3I-I2I-01"
"samplers": [
"euler", "TBG Flux2 Sampler", "pid_sde", "pid_creative_sde",
"euler_cfg_pp", "euler_ancestral", "euler_ancestral_cfg_pp", "heun",
"heunpp2", "exp_heun_2_x0", "exp_heun_2_x0_sde", "dpm_2",
"dpm_2_ancestral", "lms", "dpm_fast", "dpm_adaptive",
"dpmpp_2s_ancestral", "dpmpp_2s_ancestral_cfg_pp", "dpmpp_sde",
"dpmpp_sde_gpu", "dpmpp_2m", "dpmpp_2m_cfg_pp", "dpmpp_2m_sde",
"dpmpp_2m_sde_gpu", "dpmpp_2m_sde_heun", "dpmpp_2m_sde_heun_gpu",
"dpmpp_3m_sde", "dpmpp_3m_sde_gpu", "ddpm", "lcm", "ipndm",
"ipndm_v", "deis", "cfgpp_ud10_ab", "res_multistep",
"res_multistep_cfg_pp", "res_multistep_ancestral",
"res_multistep_ancestral_cfg_pp", "gradient_estimation",
"gradient_estimation_cfg_pp", "er_sde", "seeds_2", "seeds_3",
"sa_solver", "sa_solver_pece", "ddim", "uni_pc", "uni_pc_bh2"
],
"allowed_variables": {
"resolution": ["0.75K", "1K", "1.5K", "2K", "4K"],
"seed": "controlled per test",
"reference_images": "0 or 1"
}
"schedulers": [
"simple", "sgm_uniform", "karras", "exponential", "ddim_uniform",
"beta", "normal", "linear_quadratic", "kl_optimal"
],
"matrix": {"sampler_count": 48, "scheduler_count": 9, "planned_runs": 432, "completed_runs": 0}
}