Record generation times in test tables

This commit is contained in:
Morpheus
2026-10-28 17:32:04 +00:00
parent 4f08340ea6
commit 6df04c2c6c
7 changed files with 120 additions and 120 deletions
+20 -20
View File
@@ -31,26 +31,26 @@ scheduler, and reference count stay fixed while sampler varies.
This follows the practical 18-row scope used by the Flux.2 Klein plan: focused
sampler comparisons with the standard settings held constant. The remaining rows are currently **Planned**.
| ID | Guidance | Steps | Sampler | Scheduler | Status | Purpose |
|---|---:|---:|---|---|---|---|
| F3I-T2I-01 | 1.0 | 16 | euler | simple | Failed (validation) | Baseline; requested encoder absent |
| F3I-T2I-02 | 1.0 | 16 | euler_cfg_pp | simple | Planned | CFG++ sampler |
| F3I-T2I-03 | 1.0 | 16 | euler_ancestral | simple | Planned | Ancestral sampler |
| F3I-T2I-04 | 1.0 | 16 | euler_ancestral_cfg_pp | simple | Planned | Ancestral CFG++ sampler |
| F3I-T2I-05 | 1.0 | 16 | res_multistep | simple | Planned | RES multistep sampler |
| F3I-T2I-06 | 1.0 | 16 | res_multistep_cfg_pp | simple | Planned | RES multistep CFG++ sampler |
| F3I-T2I-07 | 1.0 | 16 | dpmpp_2m | simple | Planned | DPM++ sampler |
| F3I-T2I-08 | 1.0 | 16 | ddim | simple | Planned | DDIM sampler |
| F3I-T2I-09 | 1.0 | 16 | uni_pc | simple | Planned | UniPC sampler |
| F3I-T2I-10 | 1.0 | 16 | heun | simple | Planned | Heun sampler |
| F3I-T2I-11 | 1.0 | 16 | dpmpp_sde | simple | Planned | DPM++ SDE sampler |
| F3I-T2I-12 | 1.0 | 16 | dpmpp_2m_cfg_pp | simple | Planned | DPM++ CFG++ sampler |
| F3I-T2I-13 | 1.0 | 16 | lcm | simple | Planned | LCM sampler |
| F3I-T2I-14 | 1.0 | 16 | deis | simple | Planned | DEIS sampler |
| F3I-T2I-15 | 1.0 | 16 | ipndm | simple | Planned | IPNDM sampler |
| F3I-T2I-16 | 1.0 | 16 | sa_solver | simple | Planned | SA solver |
| F3I-T2I-17 | 1.0 | 16 | uni_pc_bh2 | simple | Planned | UniPC BH2 sampler |
| F3I-T2I-18 | 1.0 | 16 | gradient_estimation | simple | Planned | Gradient-estimation sampler |
| ID | Guidance | Steps | Sampler | Scheduler | Generation time (s) | Status | Purpose |
|---|---:|---:|---|---|---:|---|---|
| F3I-T2I-01 | 1.0 | 16 | euler | simple | — | Failed (validation) | Baseline; requested encoder absent |
| F3I-T2I-02 | 1.0 | 16 | euler_cfg_pp | simple | — | Planned | CFG++ sampler |
| F3I-T2I-03 | 1.0 | 16 | euler_ancestral | simple | — | Planned | Ancestral sampler |
| F3I-T2I-04 | 1.0 | 16 | euler_ancestral_cfg_pp | simple | — | Planned | Ancestral CFG++ sampler |
| F3I-T2I-05 | 1.0 | 16 | res_multistep | simple | — | Planned | RES multistep sampler |
| F3I-T2I-06 | 1.0 | 16 | res_multistep_cfg_pp | simple | — | Planned | RES multistep CFG++ sampler |
| F3I-T2I-07 | 1.0 | 16 | dpmpp_2m | simple | — | Planned | DPM++ sampler |
| F3I-T2I-08 | 1.0 | 16 | ddim | simple | — | Planned | DDIM sampler |
| F3I-T2I-09 | 1.0 | 16 | uni_pc | simple | — | Planned | UniPC sampler |
| F3I-T2I-10 | 1.0 | 16 | heun | simple | — | Planned | Heun sampler |
| F3I-T2I-11 | 1.0 | 16 | dpmpp_sde | simple | — | Planned | DPM++ SDE sampler |
| F3I-T2I-12 | 1.0 | 16 | dpmpp_2m_cfg_pp | simple | — | Planned | DPM++ CFG++ sampler |
| F3I-T2I-13 | 1.0 | 16 | lcm | simple | — | Planned | LCM sampler |
| F3I-T2I-14 | 1.0 | 16 | deis | simple | — | Planned | DEIS sampler |
| F3I-T2I-15 | 1.0 | 16 | ipndm | simple | — | Planned | IPNDM sampler |
| F3I-T2I-16 | 1.0 | 16 | sa_solver | simple | — | Planned | SA solver |
| F3I-T2I-17 | 1.0 | 16 | uni_pc_bh2 | simple | — | Planned | UniPC BH2 sampler |
| F3I-T2I-18 | 1.0 | 16 | gradient_estimation | simple | — | Planned | Gradient-estimation sampler |
Record runtime, output dimensions, output path, exact settings, and visual
notes for every row under `results/`. Record failures rather than silently
+15 -15
View File
@@ -61,10 +61,10 @@ fine-tuning details should not be treated as verified claims.
The first smoke tests use the same 512×512 structured caption and seed before
full-size tests are queued. Exact prompt IDs are recorded in `smoke-tests.json`.
| ID | Model | Steps | Sampler | Scheduler | Shift | CFG | Status |
|---|---|---:|---|---|---:|---:|---|
| IDEO-SMOKE-FAL-10 | Fal Ideogram | 10 | Euler | simple | 5.25 | 1 | Complete |
| IDEO-SMOKE-LOCAL-08 | Local Ideogram | 8 | Euler | simple | 5.25 | 1 | Complete |
| ID | Model | Steps | Sampler | Scheduler | Shift | CFG | Generation time (s) | Status |
|---|---|---:|---|---|---:|---:|---:|---|
| IDEO-SMOKE-FAL-10 | Fal Ideogram | 10 | Euler | simple | 5.25 | 1 | 45.483 | Complete |
| IDEO-SMOKE-LOCAL-08 | Local Ideogram | 8 | Euler | simple | 5.25 | 1 | 29.268 | Complete |
Smoke results: both completed without node errors. Fal took 45.483 seconds and
was visually strong. Local took 29.268 seconds and was coherent but notably
@@ -81,17 +81,17 @@ Keep the prompt and seed fixed for sampler tests. Keep all other settings fixed
when changing one variable. A row is not promoted until detail, anatomy,
composition, text fidelity, artefacts, and runtime are reviewed.
| ID | Model | Resolution | Steps | Sampler | Scheduler | Shift | CFG | Purpose | Status |
|---|---|---:|---:|---|---|---:|---:|---|---|
| IDEO-AUTHOR-01 | Fal | 768×1024 | 8 | Euler | simple | 5.25 | 1 | Author sampling | Complete |
| IDEO-AUTHOR-02 | Fal | 768×1024 | 8 | Euler | simple | 5.25 | 1 | Fal author sampling | Complete |
| IDEO-CARD-10 | Fal | 768×1024 | 10 | Euler | simple | 5.25 | 1 | Model-card step count | Complete |
| IDEO-SAMPLER-01 | Fal | 768×1024 | 8 | res_multistep | simple | 5.25 | 1 | Sampler comparison | Complete |
| IDEO-SAMPLER-02 | Fal | 768×1024 | 8 | euler_ancestral | simple | 5.25 | 1 | Sampler comparison | Complete |
| IDEO-SCHED-01 | Fal | 768×1024 | 8 | Euler | beta | 5.25 | 1 | Scheduler comparison | Complete |
| IDEO-SHIFT-01 | Fal | 768×1024 | 8 | Euler | simple | 3.0 | 1 | Shift comparison | Complete |
| IDEO-SHIFT-02 | Fal | 768×1024 | 8 | Euler | simple | 7.0 | 1 | Shift comparison | Complete |
| IDEO-STEPS-01 | Fal | 768×1024 | 6 | Euler | simple | 5.25 | 1 | Speed lower bound | Complete |
| ID | Model | Resolution | Steps | Sampler | Scheduler | Shift | CFG | Generation time (s) | Purpose | Status |
|---|---|---:|---:|---|---|---:|---:|---:|---|---|
| IDEO-AUTHOR-01 | Fal | 768×1024 | 8 | Euler | simple | 5.25 | 1 | — | Author sampling | Complete |
| IDEO-AUTHOR-02 | Fal | 768×1024 | 8 | Euler | simple | 5.25 | 1 | — | Fal author sampling | Complete |
| IDEO-CARD-10 | Fal | 768×1024 | 10 | Euler | simple | 5.25 | 1 | — | Model-card step count | Complete |
| IDEO-SAMPLER-01 | Fal | 768×1024 | 8 | res_multistep | simple | 5.25 | 1 | — | Sampler comparison | Complete |
| IDEO-SAMPLER-02 | Fal | 768×1024 | 8 | euler_ancestral | simple | 5.25 | 1 | — | Sampler comparison | Complete |
| IDEO-SCHED-01 | Fal | 768×1024 | 8 | Euler | beta | 5.25 | 1 | — | Scheduler comparison | Complete |
| IDEO-SHIFT-01 | Fal | 768×1024 | 8 | Euler | simple | 3.0 | 1 | — | Shift comparison | Complete |
| IDEO-SHIFT-02 | Fal | 768×1024 | 8 | Euler | simple | 7.0 | 1 | — | Shift comparison | Complete |
| IDEO-STEPS-01 | Fal | 768×1024 | 6 | Euler | simple | 5.25 | 1 | — | Speed lower bound | Complete |
Quality ranking: `IDEO-CARD-10`, `IDEO-SCHED-01`, `IDEO-SHIFT-02`,
`IDEO-AUTHOR-02`, `IDEO-AUTHOR-01`, `IDEO-SAMPLER-02`, `IDEO-STEPS-01`,