Standardize cross-model comparison fixtures

This commit is contained in:
Morpheus
2026-10-28 16:51:55 +00:00
parent 83cadab350
commit 6dd16b8926
5 changed files with 33 additions and 4 deletions
+12
View File
@@ -93,3 +93,15 @@ Each optimized workflow should explain:
- why those changes were chosen
- what still needs real-world testing
- what run settings or node parameters define the current behavior
## Apples-to-apples model comparisons
Within each category, model matrices must use the category's canonical
fixtures. T2I comparisons use the shared prompt in `optimized/T2I/prompt.txt`;
I2I comparisons use `optimized/I2I/character_reference.png` and the shared
I2I prompt; MultipleRef2I comparisons use
`optimized/MultipleRef2I/person_reference.png`,
`optimized/MultipleRef2I/scene_reference.png`, and the shared multi-reference
prompt. A model-specific matrix may vary settings such as resolution, steps,
CFG, sampler, scheduler, or seed, but it must not silently change the prompt
or reference inputs when claiming a cross-model comparison.
+9 -1
View File
@@ -2,8 +2,16 @@
This is the FLUX.3 Klein image-to-image section. Keep one-reference tests here rather than in the text-to-image directory.
The canonical comparison fixture is `../character_reference.png`, and the
canonical prompt is the shared I2I prompt in `../flux/workflow.json` (node
`74`). These inputs must remain fixed when comparing FLUX.3 Klein with other
models; only the tested generation settings may vary.
| ID | Encoder | Resolution | References | Sampler / scheduler | Purpose |
|---|---|---:|---:|---|---|
| F3K-I2I-01 | `qwen3-vl-4b-heretic_int8.safetensors` | 1024×1024 | 1 | Euler / simple | Single-reference conditioning |
Use the shared prompt from `optimized/T2I/flux3image/prompt.txt`, the installed `flux3-image-klein-int8-convrot.safetensors`, seed `310101`, guidance `3.5`, and four steps. Use an existing repository reference image where available and record its path/hash. Stop and record the exact node-contract error if the local workflow cannot represent the reference path.
Use `../character_reference.png`, the canonical I2I prompt, the installed
`flux3-image-klein-int8-convrot.safetensors`, seed `310101`, guidance `3.5`,
and four steps. Record the reference path/hash. Stop and record the exact
node-contract error if the local workflow cannot represent the reference path.
+1 -1
View File
@@ -2,6 +2,6 @@
"model": "flux3-image-klein-int8-convrot.safetensors",
"text_encoder": "qwen3-vl-4b-heretic_int8.safetensors",
"vae": "flux2-vae.safetensors",
"fixed": {"prompt_file":"../../T2I/flux3image/prompt.txt","seed":310101,"guidance":3.5,"steps":4},
"fixed": {"prompt_source":"../flux/workflow.json#74","reference_image":"../character_reference.png","seed":310101,"guidance":3.5,"steps":4},
"tests": [{"id":"F3K-I2I-01","mode":"i2i","width":1024,"height":1024,"references":1,"sampler":"euler","scheduler":"simple"}]
}
+10 -1
View File
@@ -2,8 +2,17 @@
This is the multi-reference-to-image section. Keep tests with two or more input images here.
The canonical comparison fixtures are `../person_reference.png` and
`../scene_reference.png`, and the canonical prompt is the shared
MultipleRef2I prompt in `../flux/workflow.json` (node `74`). These inputs must
remain fixed for cross-model comparisons; only generation settings may vary.
| ID | Encoder | Resolution | References | Sampler / scheduler | Purpose |
|---|---|---:|---:|---|---|
| F3K-MI2I-01 | `qwen3-vl-4b-heretic_int8.safetensors` | 1024×1024 | 2 | Euler / simple | Two-reference conditioning |
Use the shared prompt from `optimized/T2I/flux3image/prompt.txt`, the installed `flux3-image-klein-int8-convrot.safetensors`, seed `310101`, guidance `3.5`, and four steps. Record each reference image path/hash and the exact result. Stop rather than silently reducing the reference count if the workflow or node contract does not support two references.
Use both canonical reference images, the shared MultipleRef2I prompt, the
installed `flux3-image-klein-int8-convrot.safetensors`, seed `310101`,
guidance `3.5`, and four steps. Record both reference paths/hashes and the
exact result. Stop rather than silently reducing the reference count if the
workflow or node contract does not support two references.
@@ -2,6 +2,6 @@
"model": "flux3-image-klein-int8-convrot.safetensors",
"text_encoder": "qwen3-vl-4b-heretic_int8.safetensors",
"vae": "flux2-vae.safetensors",
"fixed": {"prompt_file":"../../T2I/flux3image/prompt.txt","seed":310101,"guidance":3.5,"steps":4},
"fixed": {"prompt_source":"../flux/workflow.json#74","reference_images":["../person_reference.png","../scene_reference.png"],"seed":310101,"guidance":3.5,"steps":4},
"tests": [{"id":"F3K-MI2I-01","mode":"mi2i","width":1024,"height":1024,"references":2,"sampler":"euler","scheduler":"simple"}]
}