Standardize cross-model comparison fixtures
This commit is contained in:
@@ -93,3 +93,15 @@ Each optimized workflow should explain:
|
||||
- why those changes were chosen
|
||||
- what still needs real-world testing
|
||||
- what run settings or node parameters define the current behavior
|
||||
|
||||
## Apples-to-apples model comparisons
|
||||
|
||||
Within each category, model matrices must use the category's canonical
|
||||
fixtures. T2I comparisons use the shared prompt in `optimized/T2I/prompt.txt`;
|
||||
I2I comparisons use `optimized/I2I/character_reference.png` and the shared
|
||||
I2I prompt; MultipleRef2I comparisons use
|
||||
`optimized/MultipleRef2I/person_reference.png`,
|
||||
`optimized/MultipleRef2I/scene_reference.png`, and the shared multi-reference
|
||||
prompt. A model-specific matrix may vary settings such as resolution, steps,
|
||||
CFG, sampler, scheduler, or seed, but it must not silently change the prompt
|
||||
or reference inputs when claiming a cross-model comparison.
|
||||
|
||||
@@ -2,8 +2,16 @@
|
||||
|
||||
This is the FLUX.3 Klein image-to-image section. Keep one-reference tests here rather than in the text-to-image directory.
|
||||
|
||||
The canonical comparison fixture is `../character_reference.png`, and the
|
||||
canonical prompt is the shared I2I prompt in `../flux/workflow.json` (node
|
||||
`74`). These inputs must remain fixed when comparing FLUX.3 Klein with other
|
||||
models; only the tested generation settings may vary.
|
||||
|
||||
| ID | Encoder | Resolution | References | Sampler / scheduler | Purpose |
|
||||
|---|---|---:|---:|---|---|
|
||||
| F3K-I2I-01 | `qwen3-vl-4b-heretic_int8.safetensors` | 1024×1024 | 1 | Euler / simple | Single-reference conditioning |
|
||||
|
||||
Use the shared prompt from `optimized/T2I/flux3image/prompt.txt`, the installed `flux3-image-klein-int8-convrot.safetensors`, seed `310101`, guidance `3.5`, and four steps. Use an existing repository reference image where available and record its path/hash. Stop and record the exact node-contract error if the local workflow cannot represent the reference path.
|
||||
Use `../character_reference.png`, the canonical I2I prompt, the installed
|
||||
`flux3-image-klein-int8-convrot.safetensors`, seed `310101`, guidance `3.5`,
|
||||
and four steps. Record the reference path/hash. Stop and record the exact
|
||||
node-contract error if the local workflow cannot represent the reference path.
|
||||
|
||||
@@ -2,6 +2,6 @@
|
||||
"model": "flux3-image-klein-int8-convrot.safetensors",
|
||||
"text_encoder": "qwen3-vl-4b-heretic_int8.safetensors",
|
||||
"vae": "flux2-vae.safetensors",
|
||||
"fixed": {"prompt_file":"../../T2I/flux3image/prompt.txt","seed":310101,"guidance":3.5,"steps":4},
|
||||
"fixed": {"prompt_source":"../flux/workflow.json#74","reference_image":"../character_reference.png","seed":310101,"guidance":3.5,"steps":4},
|
||||
"tests": [{"id":"F3K-I2I-01","mode":"i2i","width":1024,"height":1024,"references":1,"sampler":"euler","scheduler":"simple"}]
|
||||
}
|
||||
|
||||
@@ -2,8 +2,17 @@
|
||||
|
||||
This is the multi-reference-to-image section. Keep tests with two or more input images here.
|
||||
|
||||
The canonical comparison fixtures are `../person_reference.png` and
|
||||
`../scene_reference.png`, and the canonical prompt is the shared
|
||||
MultipleRef2I prompt in `../flux/workflow.json` (node `74`). These inputs must
|
||||
remain fixed for cross-model comparisons; only generation settings may vary.
|
||||
|
||||
| ID | Encoder | Resolution | References | Sampler / scheduler | Purpose |
|
||||
|---|---|---:|---:|---|---|
|
||||
| F3K-MI2I-01 | `qwen3-vl-4b-heretic_int8.safetensors` | 1024×1024 | 2 | Euler / simple | Two-reference conditioning |
|
||||
|
||||
Use the shared prompt from `optimized/T2I/flux3image/prompt.txt`, the installed `flux3-image-klein-int8-convrot.safetensors`, seed `310101`, guidance `3.5`, and four steps. Record each reference image path/hash and the exact result. Stop rather than silently reducing the reference count if the workflow or node contract does not support two references.
|
||||
Use both canonical reference images, the shared MultipleRef2I prompt, the
|
||||
installed `flux3-image-klein-int8-convrot.safetensors`, seed `310101`,
|
||||
guidance `3.5`, and four steps. Record both reference paths/hashes and the
|
||||
exact result. Stop rather than silently reducing the reference count if the
|
||||
workflow or node contract does not support two references.
|
||||
|
||||
@@ -2,6 +2,6 @@
|
||||
"model": "flux3-image-klein-int8-convrot.safetensors",
|
||||
"text_encoder": "qwen3-vl-4b-heretic_int8.safetensors",
|
||||
"vae": "flux2-vae.safetensors",
|
||||
"fixed": {"prompt_file":"../../T2I/flux3image/prompt.txt","seed":310101,"guidance":3.5,"steps":4},
|
||||
"fixed": {"prompt_source":"../flux/workflow.json#74","reference_images":["../person_reference.png","../scene_reference.png"],"seed":310101,"guidance":3.5,"steps":4},
|
||||
"tests": [{"id":"F3K-MI2I-01","mode":"mi2i","width":1024,"height":1024,"references":2,"sampler":"euler","scheduler":"simple"}]
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user