m/ImaGen
1
0
Fork 0
Code Issues 10 Pull Requests Actions Packages Projects Releases Wiki Activity

ImaGen #9: imagen.series (batch tries 1-10 + selection) #9

New Issue
Open
opened 2026-05-11 08:43:08 +00:00 by mAi · 2 comments
mAi commented 2026-05-11 08:43:08 +00:00
Collaborator
Copy Link

Goal

Let users submit N parallel-ish tries of the same prompt (N in 1..10), group them into a series, and pick the winner. Adds an imagen.series parent table and links imagen.jobs to it. The worker doesn't really change — it just keeps chewing the queue serially (mRock has one GPU; parallel rendering isn't useful at this layer).

Joint plan with paul (flexsiebels/head): mai messages 1637 / 1638 / 1639. m's ask: 2026-05-11 10:40 ("Nice — can we add a number (up to 10) how many tries we want? And then put those into a folder for that series? We can still select.").

Sibling issue on the flexsiebels side: paul filing in m/flexsiebels.de for the form + grid + selection UI.

Why a parent imagen.series table (not just a column)

  • Single place to hang prompt + params for the series (cheap "re-roll with N new seeds" later, batch re-render).
  • selected_image_id column makes the selection UI trivial — single UPDATE, no orphan-pick logic.
  • count is denormalised so the list page doesn't JOIN + GROUP BY on every render. Acceptable drift; multi-INSERT happens in one transaction.
  • Solo runs (tries=1) bypass the series entirely — keeps two clean code paths (image.series_id IS NULL for solos, IS NOT NULL for series members). No backfill of legacy rows.

Scope

1. Schema migration

CREATE TABLE imagen.series (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  owner_user_id UUID NOT NULL REFERENCES auth.users(id),
  prompt TEXT NOT NULL,
  backend TEXT NOT NULL,
  model TEXT,
  width INT,
  height INT,
  steps INT,
  style TEXT,
  count INT NOT NULL CHECK (count BETWEEN 1 AND 10),
  selected_image_id UUID REFERENCES imagen.images(id) ON DELETE SET NULL,
  created_at TIMESTAMPTZ DEFAULT now()
);
CREATE INDEX imagen_series_owner_created_idx ON imagen.series(owner_user_id, created_at DESC);

ALTER TABLE imagen.jobs ADD COLUMN series_id UUID REFERENCES imagen.series(id) ON DELETE SET NULL;
ALTER TABLE imagen.jobs ADD COLUMN series_idx INT;  -- 1..N order within the series (UI sort)
CREATE INDEX imagen_jobs_series_idx ON imagen.jobs(series_id, series_idx) WHERE series_id IS NOT NULL;

ALTER TABLE imagen.images ADD COLUMN series_id UUID REFERENCES imagen.series(id) ON DELETE SET NULL;
CREATE INDEX imagen_images_series_idx ON imagen.images(series_id) WHERE series_id IS NOT NULL;

RLS for imagen.series: same shape as imagen.images / imagen.jobs (owner-scoped SELECT + INSERT + UPDATE via auth.uid(); service-role bypasses for worker writes).

Grants: same authenticated + service_role pattern as #7/#8.

No NOTIFY trigger on imagen.series itself — series creation flows from the flexsiebels form INSERT, then it INSERTs N imagen.jobs rows in the same transaction. Existing imagen.jobs AFTER INSERT trigger fires N times, worker handles them serially.

2. Worker changes (minimal)

When the worker completes a job with series_id populated, it must:

  1. Propagate series_id onto the inserted imagen.images row (so the list-page query WHERE series_id IS NULL skips series members from the main grid).
  2. Otherwise no behavior change. No auto-selection. No early-exit. The flexsiebels UI handles series-level state.

The propagation point is the cloud-sync writer (internal/cloud/). Pass seriesID (nullable UUID) through the writer's input struct. Series-aware in the data layer; everything else unchanged.

3. Tests

  • Migration applies cleanly + RLS policies attached.
  • Unit test in internal/worker/: job-with-series_id produces image row with the same series_id propagated.
  • Existing tests stay green — solo path is unchanged.

4. Smoke

With the worker running, INSERT a series + N child jobs via mcp__supabase:

WITH s AS (
  INSERT INTO imagen.series (owner_user_id, prompt, backend, width, height, count)
  VALUES ('ac6c9501-3757-4a6d-8b97-2cff4288382b', 'a cozy bookshop cat in three styles, photo', 'flux-schnell-local', 1024, 1024, 3)
  RETURNING id
)
INSERT INTO imagen.jobs (owner_user_id, prompt, backend, width, height, series_id, series_idx)
SELECT 'ac6c9501-3757-4a6d-8b97-2cff4288382b', 'a cozy bookshop cat in three styles, photo', 'flux-schnell-local', 1024, 1024, s.id, idx
FROM s, generate_series(1, 3) AS idx;

Expected within ~25-30s (3x ~8s FLUX schnell renders, serial): three imagen.images rows all sharing the same series_id, three imagen.jobs rows all in status='done' with image_id populated. UPDATE the series' selected_image_id to pick one. List-page query WHERE series_id IS NULL shows no new rows (the series members are hidden from the main grid).

Acceptance criteria

  1. imagen.series table + RLS + indexes exist; imagen.jobs.series_id + series_idx + index exist; imagen.images.series_id + index exist.
  2. Inserting a series + N child jobs in one transaction triggers N generations; all resulting imagen.images rows carry the matching series_id.
  3. The main list-page query (WHERE series_id IS NULL) hides series members from the flat grid — series surfaces as a single card (handled flexsiebels-side).
  4. imagen.series.selected_image_id can be set to any of the series' imagen.images.id values; FK enforces correctness; ON DELETE SET NULL handles image deletion gracefully.
  5. Solo runs (tries=1) do NOT create a series row — they keep working exactly as in #7+#8, no behavior change.
  6. go build ./... && go test ./... clean.
  7. One smoke run with N=3 captured in the merge comment.

Out of scope

  • The form / grid / selection UI — that's the sibling flexsiebels.de issue (paul + knuth).
  • Auto-purge of un-picked images — default per joint design: stay forever, only selected surfaces on /imagine. m can bulk-delete via the series view if storage grows.
  • Re-roll / batch-re-render — possible future feature using the series row's stored prompt+params. Not in v3.
  • Concurrent worker / parallel rendering across N jobs — mRock has one GPU, serial is correct. Multi-worker scale is a future issue.
  • Series-level tags or comments — single-select on selected_image_id is enough for v3.

Refs

  • Joint design: mai messages 1637 (head → paul) + 1638 (paul → head) + 1639 (head → paul).
  • m's ask: 2026-05-11 10:40 ("Nice — can we add a number (up to 10) how many tries we want? And then put those into a folder for that series? We can still select.").
  • Sibling: m/flexsiebels.de#67 (or next free) — form + series view + selection.
  • Builds on: ImaGen#7 (cloud-sync), ImaGen#8 (queue + worker).

Workflow

Coder/gitster role. Phases mirror #7/#8:

  • Phase 1: migration + RLS + grants. Ping head with DONE-PHASE-1 so paul/knuth can light up their series-INSERT path.
  • Phase 2: worker pipeline tweak (propagate series_id from job to image), unit test.
  • Phase 3: smoke with N=3.

Head reviews + merges --no-ff into main + comments + applies done label.

## Goal Let users submit N parallel-ish tries of the same prompt (N in 1..10), group them into a **series**, and pick the winner. Adds an `imagen.series` parent table and links `imagen.jobs` to it. The worker doesn't really change — it just keeps chewing the queue serially (mRock has one GPU; parallel rendering isn't useful at this layer). Joint plan with paul (flexsiebels/head): mai messages 1637 / 1638 / 1639. m's ask: 2026-05-11 10:40 ("Nice — can we add a number (up to 10) how many tries we want? And then put those into a folder for that series? We can still select."). Sibling issue on the flexsiebels side: paul filing in `m/flexsiebels.de` for the form + grid + selection UI. ## Why a parent `imagen.series` table (not just a column) - Single place to hang prompt + params for the series (cheap "re-roll with N new seeds" later, batch re-render). - `selected_image_id` column makes the selection UI trivial — single UPDATE, no orphan-pick logic. - `count` is denormalised so the list page doesn't JOIN + GROUP BY on every render. Acceptable drift; multi-INSERT happens in one transaction. - Solo runs (tries=1) **bypass the series** entirely — keeps two clean code paths (`image.series_id IS NULL` for solos, `IS NOT NULL` for series members). No backfill of legacy rows. ## Scope ### 1. Schema migration ```sql CREATE TABLE imagen.series ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), owner_user_id UUID NOT NULL REFERENCES auth.users(id), prompt TEXT NOT NULL, backend TEXT NOT NULL, model TEXT, width INT, height INT, steps INT, style TEXT, count INT NOT NULL CHECK (count BETWEEN 1 AND 10), selected_image_id UUID REFERENCES imagen.images(id) ON DELETE SET NULL, created_at TIMESTAMPTZ DEFAULT now() ); CREATE INDEX imagen_series_owner_created_idx ON imagen.series(owner_user_id, created_at DESC); ALTER TABLE imagen.jobs ADD COLUMN series_id UUID REFERENCES imagen.series(id) ON DELETE SET NULL; ALTER TABLE imagen.jobs ADD COLUMN series_idx INT; -- 1..N order within the series (UI sort) CREATE INDEX imagen_jobs_series_idx ON imagen.jobs(series_id, series_idx) WHERE series_id IS NOT NULL; ALTER TABLE imagen.images ADD COLUMN series_id UUID REFERENCES imagen.series(id) ON DELETE SET NULL; CREATE INDEX imagen_images_series_idx ON imagen.images(series_id) WHERE series_id IS NOT NULL; ``` RLS for `imagen.series`: same shape as `imagen.images` / `imagen.jobs` (owner-scoped SELECT + INSERT + UPDATE via `auth.uid()`; service-role bypasses for worker writes). Grants: same authenticated + service_role pattern as #7/#8. No NOTIFY trigger on `imagen.series` itself — series creation flows from the flexsiebels form INSERT, then it INSERTs N `imagen.jobs` rows in the same transaction. Existing `imagen.jobs` AFTER INSERT trigger fires N times, worker handles them serially. ### 2. Worker changes (minimal) When the worker completes a job with `series_id` populated, it must: 1. Propagate `series_id` onto the inserted `imagen.images` row (so the list-page query `WHERE series_id IS NULL` skips series members from the main grid). 2. Otherwise no behavior change. No auto-selection. No early-exit. The flexsiebels UI handles series-level state. The propagation point is the cloud-sync writer (`internal/cloud/`). Pass `seriesID` (nullable UUID) through the writer's input struct. Series-aware in the data layer; everything else unchanged. ### 3. Tests - Migration applies cleanly + RLS policies attached. - Unit test in `internal/worker/`: job-with-series_id produces image row with the same series_id propagated. - Existing tests stay green — solo path is unchanged. ### 4. Smoke With the worker running, INSERT a series + N child jobs via mcp__supabase: ```sql WITH s AS ( INSERT INTO imagen.series (owner_user_id, prompt, backend, width, height, count) VALUES ('ac6c9501-3757-4a6d-8b97-2cff4288382b', 'a cozy bookshop cat in three styles, photo', 'flux-schnell-local', 1024, 1024, 3) RETURNING id ) INSERT INTO imagen.jobs (owner_user_id, prompt, backend, width, height, series_id, series_idx) SELECT 'ac6c9501-3757-4a6d-8b97-2cff4288382b', 'a cozy bookshop cat in three styles, photo', 'flux-schnell-local', 1024, 1024, s.id, idx FROM s, generate_series(1, 3) AS idx; ``` Expected within ~25-30s (3x ~8s FLUX schnell renders, serial): three `imagen.images` rows all sharing the same `series_id`, three `imagen.jobs` rows all in `status='done'` with `image_id` populated. UPDATE the series' `selected_image_id` to pick one. List-page query `WHERE series_id IS NULL` shows no new rows (the series members are hidden from the main grid). ## Acceptance criteria 1. `imagen.series` table + RLS + indexes exist; `imagen.jobs.series_id` + `series_idx` + index exist; `imagen.images.series_id` + index exist. 2. Inserting a series + N child jobs in one transaction triggers N generations; all resulting `imagen.images` rows carry the matching `series_id`. 3. The main list-page query (`WHERE series_id IS NULL`) hides series members from the flat grid — series surfaces as a single card (handled flexsiebels-side). 4. `imagen.series.selected_image_id` can be set to any of the series' `imagen.images.id` values; FK enforces correctness; ON DELETE SET NULL handles image deletion gracefully. 5. Solo runs (tries=1) **do NOT** create a series row — they keep working exactly as in #7+#8, no behavior change. 6. `go build ./... && go test ./...` clean. 7. One smoke run with N=3 captured in the merge comment. ## Out of scope - The form / grid / selection UI — that's the sibling flexsiebels.de issue (paul + knuth). - Auto-purge of un-picked images — default per joint design: stay forever, only selected surfaces on `/imagine`. m can bulk-delete via the series view if storage grows. - Re-roll / batch-re-render — possible future feature using the series row's stored prompt+params. Not in v3. - Concurrent worker / parallel rendering across N jobs — mRock has one GPU, serial is correct. Multi-worker scale is a future issue. - Series-level tags or comments — single-select on `selected_image_id` is enough for v3. ## Refs - Joint design: mai messages 1637 (head → paul) + 1638 (paul → head) + 1639 (head → paul). - m's ask: 2026-05-11 10:40 ("Nice — can we add a number (up to 10) how many tries we want? And then put those into a folder for that series? We can still select."). - Sibling: m/flexsiebels.de#67 (or next free) — form + series view + selection. - Builds on: ImaGen#7 (cloud-sync), ImaGen#8 (queue + worker). ## Workflow Coder/gitster role. Phases mirror #7/#8: - **Phase 1**: migration + RLS + grants. Ping head with DONE-PHASE-1 so paul/knuth can light up their series-INSERT path. - **Phase 2**: worker pipeline tweak (propagate `series_id` from job to image), unit test. - **Phase 3**: smoke with N=3. Head reviews + merges --no-ff into main + comments + applies `done` label.
mAi self-assigned this 2026-05-11 08:43:08 +00:00
mAi commented 2026-05-11 08:50:27 +00:00
Author
Collaborator
Copy Link

Phase 1 + 2 landed — Phase 3 waiting on worker redeploy.

Phase 1 — schema migration (DONE)

Migration imagen_series_init applied to prod Supabase. Shape:

  • imagen.series(id, owner_user_id, prompt, backend, model, width, height, steps, style, count CHECK 1..10, selected_image_id REFERENCES imagen.images(id) ON DELETE SET NULL, created_at).
  • imagen.jobs += series_id UUID FK -> imagen.series(id) ON DELETE SET NULL, + series_idx INT. Partial index on (series_id, series_idx) WHERE series_id IS NOT NULL.
  • imagen.images += series_id UUID FK -> imagen.series(id) ON DELETE SET NULL. Partial index WHERE series_id IS NOT NULL.
  • RLS on imagen.series: owner SELECT/INSERT/UPDATE via auth.uid(). Grants: authenticated + service_role.
  • PostgREST schema reload pinged. imagen already in pgrst.db_schemas.

Phase 2 — worker pipeline tweak (DONE)

Commit: https://mgit.msbls.de/m/ImaGen/commit/64120c2

  • worker.Job += SeriesID. Claim SQL now scans COALESCE(series_id::text,'').
  • cloud.SyncRequest += SeriesID. insertRow writes series_id only when non-empty (solo path keeps the column NULL, list-page query WHERE series_id IS NULL keeps showing solo runs).
  • maybeCloudSync threads seriesID through; generate.go (CLI solo path) passes "", cmd/imagen/worker.go passes job.SeriesID.
  • Tests: TestWorker_PropagatesSeriesIDToPipeline, TestWorker_SoloJobLeavesSeriesIDEmpty, TestSyncWritesSeriesID, TestSyncOmitsSeriesIDWhenEmpty. All green; full suite clean.

Phase 3 — pending

N=3 smoke needs the new binary on mriver — current systemd service runs the pre-#9 binary (claim SQL returns 10 fields, doesn't read series_id). Coordinating redeploy with head, then smoke + selected_image_id verification land here.

**Phase 1 + 2 landed — Phase 3 waiting on worker redeploy.** ## Phase 1 — schema migration (DONE) Migration `imagen_series_init` applied to prod Supabase. Shape: - `imagen.series(id, owner_user_id, prompt, backend, model, width, height, steps, style, count CHECK 1..10, selected_image_id REFERENCES imagen.images(id) ON DELETE SET NULL, created_at)`. - `imagen.jobs` += `series_id UUID FK -> imagen.series(id) ON DELETE SET NULL`, `+ series_idx INT`. Partial index on (series_id, series_idx) WHERE series_id IS NOT NULL. - `imagen.images` += `series_id UUID FK -> imagen.series(id) ON DELETE SET NULL`. Partial index WHERE series_id IS NOT NULL. - RLS on `imagen.series`: owner SELECT/INSERT/UPDATE via `auth.uid()`. Grants: authenticated + service_role. - PostgREST schema reload pinged. `imagen` already in `pgrst.db_schemas`. ## Phase 2 — worker pipeline tweak (DONE) Commit: https://mgit.msbls.de/m/ImaGen/commit/64120c2 - `worker.Job` += `SeriesID`. Claim SQL now scans `COALESCE(series_id::text,'')`. - `cloud.SyncRequest` += `SeriesID`. `insertRow` writes `series_id` only when non-empty (solo path keeps the column NULL, list-page query `WHERE series_id IS NULL` keeps showing solo runs). - `maybeCloudSync` threads `seriesID` through; `generate.go` (CLI solo path) passes `""`, `cmd/imagen/worker.go` passes `job.SeriesID`. - Tests: `TestWorker_PropagatesSeriesIDToPipeline`, `TestWorker_SoloJobLeavesSeriesIDEmpty`, `TestSyncWritesSeriesID`, `TestSyncOmitsSeriesIDWhenEmpty`. All green; full suite clean. ## Phase 3 — pending N=3 smoke needs the new binary on mriver — current systemd service runs the pre-#9 binary (claim SQL returns 10 fields, doesn't read `series_id`). Coordinating redeploy with head, then smoke + selected_image_id verification land here.
mAi commented 2026-05-11 08:52:55 +00:00
Author
Collaborator
Copy Link

Phase 3 — N=3 smoke (PASS).

Inserted series + 3 jobs in one transaction:

WITH s AS (
  INSERT INTO imagen.series (owner_user_id, prompt, backend, width, height, count)
  VALUES ('ac6c9501-...', 'a cozy bookshop cat in three styles, photo', 'flux-schnell-local', 1024, 1024, 3)
  RETURNING id
)
INSERT INTO imagen.jobs (owner_user_id, prompt, backend, width, height, series_id, series_idx)
SELECT 'ac6c9501-...', ..., s.id, idx FROM s, generate_series(1, 3) AS idx;

Series id: 2047f1c5-04da-47b7-8127-8bfe5322dad6.

series_idx status dur_s (cumulative from create) image_id image_series_id
1 done 8.36 4bf4b798-...beb6f09 2047f1c5-...22dad6
2 done 15.18 934a8602-...bbb18d4 2047f1c5-...22dad6
3 done 22.04 afc4e926-...279a5e2 2047f1c5-...22dad6

Total wall-clock: ~22s for three 1024x1024 FLUX schnell renders. All three imagen.images.series_id columns match the parent — propagation works end-to-end (claim SQL → worker.Job.SeriesID → cloud.SyncRequest.SeriesID → row.series_id).

Acceptance checks

  • (#1) imagen.series + RLS + indexes exist; jobs.series_id/series_idx + index exist; images.series_id + index exist (migration imagen_series_init).
  • (#2) Three imagen.images rows all carry series_id=2047f1c5-.... ✓
  • (#3) List-page query WHERE series_id IS NULL returns zero of the three smoke rows — series members are hidden from the flat grid. ✓
  • (#4) selected_image_id set to 934a8602-... (series_idx=2). FK rejects bogus UUIDs (verified with a foreign_key_violation roundtrip — value stays at the real winner). ✓
  • (#5) Solo path unchanged: generate.go (CLI) and the worker both treat empty SeriesID as "skip propagation"; cloud.insertRow omits the series_id key entirely when empty. Unit tests TestSyncOmitsSeriesIDWhenEmpty and TestWorker_SoloJobLeavesSeriesIDEmpty lock the behaviour in. ✓
  • (#6) go build ./... && go test ./... clean.
  • (#7) This comment captures the N=3 smoke per spec.

Phases 1+2 commit: https://mgit.msbls.de/m/ImaGen/commit/64120c2
Merge commit: https://mgit.msbls.de/m/ImaGen/commit/623dd29

Ready for done label.

**Phase 3 — N=3 smoke (PASS).** Inserted series + 3 jobs in one transaction: ```sql WITH s AS ( INSERT INTO imagen.series (owner_user_id, prompt, backend, width, height, count) VALUES ('ac6c9501-...', 'a cozy bookshop cat in three styles, photo', 'flux-schnell-local', 1024, 1024, 3) RETURNING id ) INSERT INTO imagen.jobs (owner_user_id, prompt, backend, width, height, series_id, series_idx) SELECT 'ac6c9501-...', ..., s.id, idx FROM s, generate_series(1, 3) AS idx; ``` Series id: `2047f1c5-04da-47b7-8127-8bfe5322dad6`. | series_idx | status | dur_s (cumulative from create) | image_id | image_series_id | |------------|--------|--------|----------|-----------------| | 1 | done | 8.36 | `4bf4b798-...beb6f09` | `2047f1c5-...22dad6` | | 2 | done | 15.18 | `934a8602-...bbb18d4` | `2047f1c5-...22dad6` | | 3 | done | 22.04 | `afc4e926-...279a5e2` | `2047f1c5-...22dad6` | Total wall-clock: ~22s for three 1024x1024 FLUX schnell renders. All three `imagen.images.series_id` columns match the parent — propagation works end-to-end (claim SQL → worker.Job.SeriesID → cloud.SyncRequest.SeriesID → row.series_id). **Acceptance checks** - (#1) `imagen.series` + RLS + indexes exist; jobs.series_id/series_idx + index exist; images.series_id + index exist (migration `imagen_series_init`). - (#2) Three imagen.images rows all carry `series_id=2047f1c5-...`. ✓ - (#3) List-page query `WHERE series_id IS NULL` returns zero of the three smoke rows — series members are hidden from the flat grid. ✓ - (#4) `selected_image_id` set to `934a8602-...` (series_idx=2). FK rejects bogus UUIDs (verified with a foreign_key_violation roundtrip — value stays at the real winner). ✓ - (#5) Solo path unchanged: `generate.go` (CLI) and the worker both treat empty SeriesID as "skip propagation"; cloud.insertRow omits the `series_id` key entirely when empty. Unit tests `TestSyncOmitsSeriesIDWhenEmpty` and `TestWorker_SoloJobLeavesSeriesIDEmpty` lock the behaviour in. ✓ - (#6) `go build ./... && go test ./...` clean. - (#7) This comment captures the N=3 smoke per spec. Phases 1+2 commit: https://mgit.msbls.de/m/ImaGen/commit/64120c2 Merge commit: https://mgit.msbls.de/m/ImaGen/commit/623dd29 Ready for `done` label.
mAi added the
done
 label 2026-05-11 08:53:24 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
done
Work complete, awaiting m close
needs-review
Awaiting m review
needs-smoke
Code merged; real-API smoke still pending
No Label
done
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
mAi m
No Assignees mAi
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: m/ImaGen#9
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?