Reset Flux 3 Klein T2I results
This commit is contained in:
+1
-1
@@ -8,7 +8,7 @@ matrix; visual review must use the same category fixtures and resolution.
|
||||
|---|---|---|---:|---|---|
|
||||
| T2I | Flux.2 Klein | shared T2I prompt | 1088×1920 comparison target | Matrix/baseline recorded | Existing baseline |
|
||||
| T2I | Qwen Image 2.1 | shared T2I prompt | 1088×1920 comparison target | 88/88 recorded | Existing best recorded in category README |
|
||||
| T2I | FLUX.3 Klein | shared T2I prompt | 1088×1920 comparison target | 432-cell sampler matrix recorded | Visual cross-model ranking pending |
|
||||
| T2I | FLUX.3 Klein | shared T2I prompt | 1088×1920 comparison target | Fresh 432-cell sampler matrix planned (0/432) | Results pending |
|
||||
| T2I | Ideogram 4 | structured magazine prompt | varies by magazine matrix | 24-row matrix complete | Separate prompt family; not directly comparable to portrait rows |
|
||||
| I2I | Flux.2 Klein | `character_reference.png` + shared I2I prompt | 1080×1920 category target | Matrix planned/untested | Candidate baseline |
|
||||
| I2I | Qwen Image 2.1 | `character_reference.png` + shared I2I prompt | 1080×1920 category target | 34-row matrix recorded | Existing best recorded in category README |
|
||||
|
||||
@@ -14,7 +14,7 @@ This summary is maintained across the model-specific test plans. A result is onl
|
||||
|---|---|---:|---|---|
|
||||
| Flux.2 Klein | F-T2I-03 | 220.3 s | [F-T2I-03.png](flux/results/F-T2I-03.png) | Current best; 1080×1920 target |
|
||||
| Qwen Image 2.1 | Q-T2I-57 | 110.2 s | [Q-T2I-57.png](qwen/results/Q-T2I-57.png) | Current best; direct 1088×1920 latent; res_multistep, beta, 25 steps, CFG 2.5 |
|
||||
| FLUX.3 Klein | F3I-T2I-01 / sampler matrix | recorded | [best5](flux3image/results/best5/) | Complete local matrix; canonical comparison resolution 1088×1920; visual cross-model ranking pending |
|
||||
| FLUX.3 Klein | F3I-MATRIX | planned (0/432) | — | Fresh local matrix; canonical comparison resolution 1088×1920 |
|
||||
| Ideogram 4 | MAG-MATRIX-24 | 30.154 s | [MAG-MATRIX-24.png](ideogram/results/magazine-matrix/MAG-MATRIX-24.png) | Complete magazine-cover matrix; different structured prompt, not directly comparable to the shared portrait prompt |
|
||||
|
||||
Store each T2I test under the selected model's `results/<test-id>.png` and `results/<test-id>.json` using the table ID as the label.
|
||||
|
||||
@@ -1,86 +0,0 @@
|
||||
# FLUX 3 Image sampler/scheduler matrix
|
||||
|
||||
## Test conditions
|
||||
|
||||
- **Runs:** 432 (48 samplers × 9 schedulers)
|
||||
- **Model:** `flux3-image-klein-int8-convrot.safetensors`
|
||||
- **Text encoder:** `qwen3vl_4b_int8_convrot.safetensors` (`type=flux3`)
|
||||
- **VAE:** `flux2-vae.safetensors`
|
||||
- **Prompt:** [`prompt.txt`](prompt.txt), SHA-256 `5802909ebe894260b70a3977430e08aaa95732e0559749d8cb42e2f5ecf3ab6f`
|
||||
- **Output:** 1088 × 1920 (the 1080 × 1920 target snapped to the model-compatible width)
|
||||
- **Steps:** 4
|
||||
- **Seed:** 310001
|
||||
- **Result:** 432/432 successful
|
||||
- **Total execution time:** 3726.104 seconds
|
||||
- **Mean execution time:** 8.625 seconds
|
||||
|
||||
The complete per-cell record is in [`results/F3I-MATRIX.json`](results/F3I-MATRIX.json). Each cell contains the status, measured runtime, and the `F3I-M-SS-DD` output prefix used by the server, where `SS` is the sampler index and `DD` is the scheduler index.
|
||||
|
||||
## Runtime findings
|
||||
|
||||
Fastest sampler means across all nine schedulers:
|
||||
|
||||
- `TBG Flux2 Sampler` — 6.257 s
|
||||
- `pid_sde` — 6.359 s
|
||||
- `euler` — 6.407 s
|
||||
- `dpm_fast` — 6.408 s
|
||||
- `res_multistep` — 6.412 s
|
||||
- `cfgpp_ud10_ab` — 6.415 s
|
||||
- `ipndm` — 6.417 s
|
||||
- `ipndm_v` — 6.424 s
|
||||
- `ddim` — 6.428 s
|
||||
- `dpmpp_2m` — 6.429 s
|
||||
|
||||
Scheduler means across all 48 samplers:
|
||||
|
||||
- `sgm_uniform` — 8.022 s
|
||||
- `simple` — 8.063 s
|
||||
- `linear_quadratic` — 8.265 s
|
||||
- `beta` — 8.374 s
|
||||
- `kl_optimal` — 8.816 s
|
||||
- `karras` — 8.829 s
|
||||
- `exponential` — 8.919 s
|
||||
- `normal` — 8.921 s
|
||||
- `ddim_uniform` — 9.417 s
|
||||
|
||||
`dpm_adaptive` is a clear runtime outlier. Its slowest cells were:
|
||||
|
||||
- `dpm_adaptive` + `ddim_uniform` — 87.590 s
|
||||
- `dpm_adaptive` + `normal` — 61.760 s
|
||||
- `dpm_adaptive` + `exponential` — 61.244 s
|
||||
- `dpm_adaptive` + `kl_optimal` — 57.667 s
|
||||
- `dpm_adaptive` + `karras` — 57.582 s
|
||||
|
||||
The fastest individual cell was `euler` + `exponential` at 6.199 s. Runtime alone is not a quality score.
|
||||
|
||||
## Visual spot-check
|
||||
|
||||
Representative generated images were inspected after the automated run:
|
||||
|
||||
- `F3I-M-01-01` (`euler` + `simple`) was a strong baseline: coherent subject, composition, anatomy, and useful prompt adherence.
|
||||
- `F3I-M-01-04` (`euler` + `exponential`) was visibly degraded in the inspected sample, with garbled scoreboard text and smeared detail.
|
||||
- `F3I-M-16-05` (`dpm_adaptive` + `ddim_uniform`) showed severe anatomy duplication/extra limbs, garbled text, and texture artifacts.
|
||||
- `F3I-M-48-09` (`uni_pc_bh2` + `kl_optimal`) was overprocessed in the inspected sample, with distorted texture/anatomy and garbled text.
|
||||
|
||||
These are representative spot-checks, not a claim that four images fully rank all 432 outputs. The matrix proves that every combination executes successfully on the installed local model; a full perceptual ranking would require reviewing all 432 images or adding an image-quality scoring pass.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Keep **Euler + Simple** as the default comparison baseline. It is fast, already aligned with the native workflow path, and produced the strongest inspected sample. Treat `sgm_uniform` as a similarly fast scheduler candidate for follow-up visual testing. Avoid `dpm_adaptive` for routine generation because its runtime is unpredictable and its inspected sample was poor. Keep the remaining combinations available as experimental options until they receive visual review.
|
||||
|
||||
No generated server files were deleted during collection. The matrix outputs remain under the server prefix `fluxExperiments/flux3image/matrix/` so the visual findings can be revisited.
|
||||
|
||||
## Best five reviewed outputs
|
||||
|
||||
These are the five strongest images from the visual review, ordered best first.
|
||||
They are copied into [`results/best5/`](results/best5/).
|
||||
|
||||
1. [`01-euler-simple.png`](results/best5/01-euler-simple.png) — `euler` + `simple`
|
||||
2. [`02-euler-sgm_uniform.png`](results/best5/02-euler-sgm_uniform.png) — `euler` + `sgm_uniform`
|
||||
3. [`03-dpmpp_2m-simple.png`](results/best5/03-dpmpp_2m-simple.png) — `dpmpp_2m` + `simple`
|
||||
4. [`04-res_multistep-simple.png`](results/best5/04-res_multistep-simple.png) — `res_multistep` + `simple`
|
||||
5. [`05-ddim-simple.png`](results/best5/05-ddim-simple.png) — `ddim` + `simple`
|
||||
|
||||
The first two are very close in appearance because this low-step configuration
|
||||
produced nearly identical results for those cells. The ranking is a visual
|
||||
selection, not a statistically significant model-quality benchmark.
|
||||
@@ -1,84 +1,44 @@
|
||||
# FLUX 3 Image / Klein
|
||||
|
||||
This is the canonical local FLUX.3 Klein text-to-image workflow and test
|
||||
matrix. New reference-based tests belong in `optimized/I2I/flux3klein/` and
|
||||
`optimized/MultipleRef2I/flux3klein/`; do not create a second T2I directory.
|
||||
The canonical cross-model comparison row is `1088×1920`; the other recorded
|
||||
resolution rows are explicitly labelled scaling studies.
|
||||
This is the canonical local FLUX.3 Klein text-to-image workflow and clean
|
||||
sampler/scheduler test plan. Reference-based tests belong in
|
||||
`optimized/I2I/flux3klein/` and `optimized/MultipleRef2I/flux3klein/`.
|
||||
|
||||
This directory contains the first controlled workflow and test plan for the
|
||||
server's locally installed FLUX 3 Image Klein model. The workflow does not use
|
||||
No results have been retained for this fresh run. The workflow does not use
|
||||
the remote `Flux3ImageNode` API.
|
||||
|
||||
## Workflow
|
||||
## Fixed comparison inputs
|
||||
|
||||
- `workflow.json` — API-format prompt for `/prompt` automation.
|
||||
- `prompt.txt` — fixed baseline prompt.
|
||||
- `variables.json` — fixed values and allowed test variables.
|
||||
- `results/` — output images and matching metadata records.
|
||||
- Workflow: `workflow.json`
|
||||
- Prompt: `prompt.txt`
|
||||
- Model: `flux3-image-klein-int8-convrot.safetensors`
|
||||
- Text encoder: `qwen3vl_4b_int8_convrot.safetensors` (`type=flux3`)
|
||||
- VAE: `flux2-vae.safetensors`
|
||||
- Resolution: **1088×1920**
|
||||
- References: none
|
||||
- Seed: `310001`
|
||||
- Steps: `4`
|
||||
- Flux Guidance: `3.5`
|
||||
|
||||
The local graph uses `flux3-image-klein-int8-convrot.safetensors`,
|
||||
`qwen3vl_4b_int8_convrot.safetensors` with `type=flux3`, `flux2-vae.safetensors`,
|
||||
Flux guidance, Euler sampling, and the native ComfyUI sampler chain. The
|
||||
remote API node is not used because it requires a separate Comfy API login.
|
||||
The graph uses `FluxGuidance`, not `CFGGuider`; therefore this T2I suite
|
||||
records Flux Guidance rather than calling it CFG. Prompt, resolution, seed,
|
||||
steps, guidance, and reference count stay fixed while sampler and scheduler
|
||||
vary.
|
||||
|
||||
## Test matrix
|
||||
## Fresh test matrix
|
||||
|
||||
The first pass changes one variable at a time. The prompt, aspect ratio,
|
||||
grounding state, safety tolerance, and reference count remain fixed unless a
|
||||
row explicitly changes them.
|
||||
The matrix contains **48 samplers × 9 schedulers = 432 planned runs**. All
|
||||
rows are currently **Planned**. The complete sampler and scheduler lists are
|
||||
stored in `variables.json`; each executed cell must receive a result record in
|
||||
`results/` with its exact settings, runtime, output path, dimensions, and
|
||||
error or visual notes.
|
||||
|
||||
| ID | Purpose | Resolution | References | Grounding | Seed | Status | Runtime (s) | Verdict |
|
||||
|---|---|---:|---:|---|---:|---|---:|---|
|
||||
| F3I-T2I-01 | Shared-prompt local baseline | 1080×1920 target | 0 | Off | 310001 | **Passed** | 19.6 | 1088×1920 output generated |
|
||||
| F3I-T2I-02 | Lower-cost portrait comparison | 0.75K target | 0 | Off | 310002 | **Passed** | 5.9 | 768×1360 output generated |
|
||||
| F3I-T2I-03 | Detail/quality portrait comparison | 1.5K target | 0 | Off | 310003 | **Passed** | 13.9 | 1536×2720 output generated |
|
||||
| F3I-T2I-04 | Higher-resolution portrait feasibility | 2K target | 0 | Off | 310004 | **Passed** | 47.2 | 2048×3648 output generated |
|
||||
| F3I-T2I-05 | Grounded prompt comparison | 1K | 0 | On | 310005 | **N/A local** | — | Grounding belongs to remote API node |
|
||||
| F3I-I2I-01 | Single-reference edit | 1K | 1 | Off | 310006 | **Passed** | 33.6 | 1024×1024 output generated |
|
||||
| Matrix | Resolution | References | Seed | Guidance | Steps | Samplers | Schedulers | Status |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|---|
|
||||
| F3I-MATRIX | 1088×1920 | 0 | 310001 | 3.5 | 4 | 48 | 9 | Planned (0/432) |
|
||||
|
||||
## Evaluation
|
||||
|
||||
Record the exact prompt, seed, resolution, reference filename/hash, runtime,
|
||||
output dimensions, success/failure, and visual notes in `results/<test-id>.json`.
|
||||
Keep the first successful baseline as the comparison target. Do not promote a
|
||||
larger resolution solely because it is larger; judge detail, composition,
|
||||
prompt adherence, artifacts, and cost/runtime together.
|
||||
|
||||
The API node's seed controls rerun identity but the service may remain
|
||||
nondeterministic. That is recorded as a property of the test, not treated as a
|
||||
failure.
|
||||
|
||||
## Matrix results
|
||||
|
||||
The initial resolution and reference runs were calibration tests with a short
|
||||
fox prompt. They remain useful for checking graph/model compatibility but are
|
||||
superseded for model comparison by the shared prompt baseline. The native
|
||||
reference conditioning path passed with `Base001.png`. The grounding row is not
|
||||
run locally because grounding is a capability of the authenticated remote
|
||||
`Flux3ImageNode`, which is intentionally excluded from this repository's
|
||||
workflow.
|
||||
|
||||
## Initial run
|
||||
|
||||
The earlier API-node probe is retained as `results/F3I-T2I-01-remote-api.json`
|
||||
as an environment diagnostic: it returned `Unauthorized` before generation.
|
||||
The local workflow was then corrected to use the Flux 3-compatible
|
||||
Qwen3-VL 4B encoder and 128-channel `flux2-vae.safetensors`; that run passed
|
||||
and is recorded in `results/F3I-T2I-01.json`.
|
||||
|
||||
## Full sampler/scheduler matrix
|
||||
|
||||
The local model was tested with every available sampler and scheduler: 48 × 9
|
||||
combinations, for 432 total runs. All 432 completed successfully at 1088 ×
|
||||
1920, four steps, seed 310001, using the shared prompt. The complete cell-level
|
||||
results, runtimes, and server output prefixes are recorded in
|
||||
[`results/F3I-MATRIX.json`](results/F3I-MATRIX.json). The analysis and visual
|
||||
spot-check findings are in [`MATRIX-ANALYSIS.md`](MATRIX-ANALYSIS.md).
|
||||
|
||||
The recommended baseline remains **Euler + Simple**. `dpm_adaptive` is not
|
||||
recommended for routine use because it was an extreme runtime outlier; runtime
|
||||
success does not by itself establish visual quality.
|
||||
|
||||
The five strongest visually reviewed outputs are collected in
|
||||
[`results/best5/`](results/best5/), ordered best first.
|
||||
Do not silently substitute a rejected sampler or scheduler. Record failures
|
||||
as results, preserve the fixed comparison inputs, and update this table only
|
||||
from the actual result records. The first successful `euler + simple` cell is
|
||||
the compatibility baseline; no visual winner should be declared until the
|
||||
outputs have been reviewed under the same prompt and resolution.
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
{
|
||||
"test_id": "F3I-I2I-01",
|
||||
"status": "passed",
|
||||
"prompt_id": "80876b8b-afcb-4ba6-b6f8-eb0b4916f362",
|
||||
"reference": "Base001.png",
|
||||
"resolution": [1024, 1024],
|
||||
"seed": 310006,
|
||||
"steps": 16,
|
||||
"alpha": 16.0,
|
||||
"runtime_seconds": 33.6,
|
||||
"output": {"filename": "F3I-I2I-01_00001_.png", "subfolder": "fluxExperiments\\flux3image", "width": 1024, "height": 1024}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,10 +0,0 @@
|
||||
{
|
||||
"test_id": "F3I-T2I-01-remote-api",
|
||||
"status": "failed",
|
||||
"verdict": "Diagnostic only: remote Flux 3 API requires authentication",
|
||||
"prompt_id": "65659b7d-a050-4076-8d8f-1abb461a94db",
|
||||
"node_type": "Flux3ImageNode",
|
||||
"error": "Unauthorized: Please login first to use this node.",
|
||||
"runtime_seconds": 0.0,
|
||||
"used_by_tracked_workflow": false
|
||||
}
|
||||
@@ -1,31 +0,0 @@
|
||||
{
|
||||
"test_id": "F3I-T2I-01",
|
||||
"status": "passed",
|
||||
"verdict": "Shared comparison prompt completed successfully with local native Klein graph",
|
||||
"prompt_id": "dbc97923-d5c6-48a8-aeb9-aceac2e7396d",
|
||||
"configuration": {
|
||||
"resolution": "1080x1920 target; 1088x1920 actual",
|
||||
"aspect_ratio": "1:1",
|
||||
"grounding": false,
|
||||
"safety_tolerance": 4,
|
||||
"seed": 310001,
|
||||
"references": 0
|
||||
},
|
||||
"error": {
|
||||
"node_id": "1",
|
||||
"node_type": "Flux3ImageNode",
|
||||
"message": "Unauthorized: Please login first to use this node."
|
||||
},
|
||||
"runtime_seconds": 19.6,
|
||||
"model": "flux3-image-klein-int8-convrot.safetensors",
|
||||
"text_encoder": "qwen3vl_4b_int8_convrot.safetensors",
|
||||
"vae": "flux2-vae.safetensors",
|
||||
"prompt_sha256": "5802909ebe894260b70a3977430e08aaa95732e0559749d8cb42e2f5ecf3ab6f",
|
||||
"output": {
|
||||
"filename": "F3I-T2I-01_00002_.png",
|
||||
"subfolder": "fluxExperiments\\flux3image",
|
||||
"type": "output",
|
||||
"width": 1088,
|
||||
"height": 1920
|
||||
}
|
||||
}
|
||||
@@ -1,10 +0,0 @@
|
||||
{
|
||||
"test_id": "F3I-T2I-02",
|
||||
"status": "passed",
|
||||
"prompt_id": "c60b883c-d40f-4c0e-a9e9-de8a9cd23c6b",
|
||||
"resolution": [768, 1360],
|
||||
"seed": 310002,
|
||||
"runtime_seconds": 5.9,
|
||||
"prompt_sha256": "5802909ebe894260b70a3977430e08aaa95732e0559749d8cb42e2f5ecf3ab6f",
|
||||
"output": {"filename": "F3I-T2I-02_00002_.png", "subfolder": "fluxExperiments\\flux3image", "width": 768, "height": 1360}
|
||||
}
|
||||
@@ -1,10 +0,0 @@
|
||||
{
|
||||
"test_id": "F3I-T2I-03",
|
||||
"status": "passed",
|
||||
"prompt_id": "60c68b18-587f-40d1-9e59-1140a57761e5",
|
||||
"resolution": [1536, 2720],
|
||||
"seed": 310003,
|
||||
"runtime_seconds": 13.9,
|
||||
"prompt_sha256": "5802909ebe894260b70a3977430e08aaa95732e0559749d8cb42e2f5ecf3ab6f",
|
||||
"output": {"filename": "F3I-T2I-03_00002_.png", "subfolder": "fluxExperiments\\flux3image", "width": 1536, "height": 2720}
|
||||
}
|
||||
@@ -1,10 +0,0 @@
|
||||
{
|
||||
"test_id": "F3I-T2I-04",
|
||||
"status": "passed",
|
||||
"prompt_id": "49938a50-d34c-4360-b2f0-9acef37048b4",
|
||||
"resolution": [2048, 3648],
|
||||
"seed": 310004,
|
||||
"runtime_seconds": 47.2,
|
||||
"prompt_sha256": "5802909ebe894260b70a3977430e08aaa95732e0559749d8cb42e2f5ecf3ab6f",
|
||||
"output": {"filename": "F3I-T2I-04_00002_.png", "subfolder": "fluxExperiments\\flux3image", "width": 2048, "height": 3648}
|
||||
}
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 3.1 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 3.1 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 3.0 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 3.0 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 3.1 MiB |
@@ -4,25 +4,32 @@
|
||||
"vae": "flux2-vae.safetensors (128-channel / 16x latent decoder)",
|
||||
"fixed": {
|
||||
"prompt_file": "prompt.txt",
|
||||
"aspect_ratio": "1:1",
|
||||
"width": 1088,
|
||||
"height": 1920,
|
||||
"guidance": 3.5,
|
||||
"sampler": "euler",
|
||||
"scheduler": "simple",
|
||||
"steps": 4,
|
||||
"seed": 310001,
|
||||
"image_references": 0,
|
||||
"remote_api": false
|
||||
},
|
||||
"test_ids": [
|
||||
"F3I-T2I-01",
|
||||
"F3I-T2I-02",
|
||||
"F3I-T2I-03",
|
||||
"F3I-T2I-04",
|
||||
"F3I-T2I-05",
|
||||
"F3I-I2I-01"
|
||||
"samplers": [
|
||||
"euler", "TBG Flux2 Sampler", "pid_sde", "pid_creative_sde",
|
||||
"euler_cfg_pp", "euler_ancestral", "euler_ancestral_cfg_pp", "heun",
|
||||
"heunpp2", "exp_heun_2_x0", "exp_heun_2_x0_sde", "dpm_2",
|
||||
"dpm_2_ancestral", "lms", "dpm_fast", "dpm_adaptive",
|
||||
"dpmpp_2s_ancestral", "dpmpp_2s_ancestral_cfg_pp", "dpmpp_sde",
|
||||
"dpmpp_sde_gpu", "dpmpp_2m", "dpmpp_2m_cfg_pp", "dpmpp_2m_sde",
|
||||
"dpmpp_2m_sde_gpu", "dpmpp_2m_sde_heun", "dpmpp_2m_sde_heun_gpu",
|
||||
"dpmpp_3m_sde", "dpmpp_3m_sde_gpu", "ddpm", "lcm", "ipndm",
|
||||
"ipndm_v", "deis", "cfgpp_ud10_ab", "res_multistep",
|
||||
"res_multistep_cfg_pp", "res_multistep_ancestral",
|
||||
"res_multistep_ancestral_cfg_pp", "gradient_estimation",
|
||||
"gradient_estimation_cfg_pp", "er_sde", "seeds_2", "seeds_3",
|
||||
"sa_solver", "sa_solver_pece", "ddim", "uni_pc", "uni_pc_bh2"
|
||||
],
|
||||
"allowed_variables": {
|
||||
"resolution": ["0.75K", "1K", "1.5K", "2K", "4K"],
|
||||
"seed": "controlled per test",
|
||||
"reference_images": "0 or 1"
|
||||
}
|
||||
"schedulers": [
|
||||
"simple", "sgm_uniform", "karras", "exponential", "ddim_uniform",
|
||||
"beta", "normal", "linear_quadratic", "kl_optimal"
|
||||
],
|
||||
"matrix": {"sampler_count": 48, "scheduler_count": 9, "planned_runs": 432, "completed_runs": 0}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user