Add overnight flux2klein smoke benchmark runner

This commit is contained in:
Morpheus
2026-08-20 20:34:56 +00:00
parent 69f3c0f9c3
commit 3e75113cee
5 changed files with 352 additions and 2 deletions
+38
View File
@@ -0,0 +1,38 @@
# FLUX.2-klein Benchmarks
This folder holds durable benchmark inputs, outputs, and state for low-cost `flux2klein` testing.
## Current intent
- Keep overnight tests cheap and reviewable.
- Prefer one or two single-image smoke tests over broad matrices.
- Log enough context to reproduce a result without trawling temporary folders.
## Layout
- `canonical-inputs/`
- durable reference inputs used for repeatable comparisons
- `runs/`
- dated run batches
- `overnight-state.json`
- handoff file between the `3am` optimisation pass and the `6am` review pass
## Batch contents
Each case folder should keep:
- `input-1.*`
- `resolved-prompt.json`
- `run-record.json`
- the saved output image
## Current nightly default
- family: `flux2klein`
- workflow: `models/flux2klein/image-edit-9b-distilled/workflow.json`
- output policy: single image only
- test style: smoke test, not broad sweep
## Review rule
Do not call a workflow fully optimal from one benchmark image. Use these batches to decide whether a path is promising enough for follow-up.
@@ -0,0 +1,5 @@
# Canonical Inputs
Put durable benchmark reference images here when a test input becomes part of the standing comparison set.
For now, the overnight runner can also copy an external source image into each run case so the used input is still captured in-repo even before it graduates into this folder.
@@ -0,0 +1,10 @@
{
"date": null,
"testsSet": false,
"serverBusy": false,
"reason": "No overnight run has written state yet.",
"cases": [],
"reviewPending": false,
"optimizedPaths": [],
"verdict": null
}