# imggen

Local SDXL image generation on an Intel Arc B580 (12GB) under Slackware, via a
container that carries the Intel Level Zero runtime the host distro does not
package. Prompt in, PNG on the host out.

## Why this exists (and its limits)

The Arc B580 runs SDXL's *compute* fast (~31 UNet it/s warm), but its *memory*
is tight. Stock SDXL fp16 OOMs at VAE decode on 12GB because diffusers upcasts
the VAE to fp32. Swapping in the fp16-fix VAE removes that upcast and the whole
pipeline fits, at roughly 6.3s per warm 1024x1024 image, 25 steps.

Measured peak VRAM for the working config is 9.07 GB on a 12GB card, so there
is roughly 2.5-3 GB of real headroom, not zero.

Known to work:

- SDXL base, single image, 1024x1024, ~25 steps. ~6.3s warm.

Plausibly fits in the remaining headroom (UNTESTED, try before assuming):

- A second batched image, or modest resolution bumps above 1024.
- Light img2img.

Probably does NOT fit (a second resident model exceeds ~3GB):

- SDXL refiner co-resident with base.

Does NOT fit:

- Flux (any variant). Far heavier than the available margin.

If you want the refiner or Flux, that is the signal to move that tier of work
to a RunPod serverless endpoint rather than fighting 12GB. Local covers the
light case comfortably and may stretch a little past it; it does not reach the
heavy tier.

## Build

The Dockerfile is PINNED to the exact versions captured from the known-good
test container (torch 2.12.1+xpu, diffusers 0.39.0, transformers 5.13.0, etc.,
on oneAPI 2025.3.2 / Ubuntu 24.04.4). These produced ~6.3s warm SDXL-base
images with 9.07 GB peak VRAM on an Arc B580.

Build:

```
docker build -t imggen:local .
```

If the base image tag `intel/oneapi-basekit:2025.3.2-0-devel-ubuntu24.04` is
not available on Docker Hub, substitute the closest 2025.3.x tag and record
what you used. Do not fall back to `:latest`.

## Models

Three model keys, all SDXL-architecture, all sharing the fp16-fix VAE, all
fitting the 12GB card. The registry in `generate.py` (`MODELS`) is the source
of truth; swapping the default is a one-line key change.

| key       | repo                                    | notes                          |
|-----------|------------------------------------------|--------------------------------|
| `sdxl`    | stabilityai/stable-diffusion-xl-base-1.0 | general purpose, default       |
| `realvis` | SG161222/RealVisXL_V5.0                  | photoreal, permissive license  |
| `pony`    | kitty7779/ponyDiffusionV6XL              | unrestricted, versatile (ungated diffusers-format mirror; the original AstraliteHeart repo is gated) |

`imggen list` prints this from the wrapper. Pick a model per invocation with
`-m KEY`; without one, `sdxl` is the default (or the daemon's currently
resident model, see below).

## Setup

Set up the repo once:

```
cp .env.example .env
```

Edit `.env` with your real host paths and, if needed, your uid/gid:

```
IMGGEN_OUT=/path/to/output          # where PNGs land
IMGGEN_CACHE=/path/to/model-cache   # where the ~7.5GB weights are cached
IMGGEN_PORT=8765                    # daemon port, localhost-only
IMGGEN_MODEL=sdxl                   # model loaded on `imggen start`
IMGGEN_UID=1000                     # your host uid (id -u)
IMGGEN_GID=100                      # your host gid (id -g)
```

`.env` is gitignored; `.env.example` documents the keys. `docker compose`
(via `compose.yaml`) reads it. The service runs as your host user (uid:gid
plus `group_add: video` for `/dev/dri` access), so generated PNGs are owned
by you, not root.

## Install the wrapper

The `imggen` script lives in this repo and runs `docker compose` here; it
`cd`s into `$IMGGEN_DIR` (default `$HOME/Programming/GIT/imggen`) to find
`compose.yaml` and `.env`. There is nothing to copy elsewhere. Run it in
place:

```
./imggen "a red bicycle against a stone wall"
```

Or put a symlink to it on your PATH for convenience; it still operates on
the repo via `IMGGEN_DIR`:

```
ln -s "$PWD/imggen" ~/bin/imggen
```

If the repo lives somewhere else, set `IMGGEN_DIR` accordingly.

## Daemon / lifecycle

A persistent daemon (`server.py`, run via `docker compose`) keeps ONE model
resident in XPU VRAM so repeated prompts skip the ~45s model reload. It
holds one model at a time (three at once would exceed the 12GB card) and
swaps on demand. It is serial by construction, one GPU, one request at a
time.

```
imggen start [-m KEY]   # start daemon, load model (default sdxl), hold VRAM
imggen stop              # stop daemon, free VRAM
imggen status            # is it up? which model? (prints /status JSON, or "daemon down")
imggen list               # show the three model keys
```

**VRAM stays held (~9GB) the whole time the daemon is up, until you run
`imggen stop`.** Don't leave it running unattended if you need that VRAM
back.

A bare prompt routes automatically:

```
imggen "a foggy harbour at dawn" --steps 30 --seed 42
imggen "portrait of a fox" -m realvis -o fox.png -n "blurry, low quality"
```

- If the daemon is UP, the prompt goes to it. If the requested `-m` differs
  from the resident model, the daemon swaps first (a few seconds). With no
  `-m` and an interactive terminal, a menu picks the model. A failed swap
  (bad repo, gated repo, OOM) prints the daemon's error and the wrapper
  exits 1.
- If the daemon is DOWN, the wrapper falls back to a one-shot run: it loads
  the model, generates, and exits. Slower (pays the ~45s load every time)
  but requires no daemon management.

Output lands in `$IMGGEN_OUT` (from `.env`). Models cache in `$IMGGEN_CACHE`
so the ~7.5GB download happens once.

## First run

The first invocation downloads SDXL base + the fp16-fix VAE (~7.5GB) into the
cache volume. That run is network-bound and slow. Subsequent runs skip the
download. The first *generation* in any fresh container is also slower (~7.3s)
due to one-time kernel compilation; warm runs settle to ~6.3s.

## Upgrading

Treat upgrades as deliberate, tested events. Do not `pip install -U` casually.
This working state depends on a specific torch-XPU build, a specific diffusers
version, and the fp16-fix VAE. If you rebuild with newer versions, keep the old
pinned Dockerfile in git so you can revert when something breaks.

## Trial

This is a time-boxed experiment. Run it against real blog work for a few weeks.
If it sticks, keep it. If it does not, the fallback is a RunPod serverless
ComfyUI endpoint wrapped in the same `imggen` CLI shape, which trades local
privacy for the ability to run heavier models (Flux, refiner, batching)
on-demand at zero idle cost.

## Development Approach

This project is developed using AI-assisted tools. Code is generated with the help of AI based on human-provided specifications, design decisions, and iterative feedback.

All contributions are reviewed, tested, and curated by the maintainer before being included in the codebase. AI is used as a productivity and exploration tool, while human oversight remains central to all decisions.

The goal is to combine the flexibility of AI-assisted development with standard open-source practices such as transparency, review, and accountability.