aboutsummaryrefslogtreecommitdiffstats

imggen

Local SDXL image generation on an Intel Arc B580 (12GB) under Slackware, via a container that carries the Intel Level Zero runtime the host distro does not package. Prompt in, PNG on the host out.

Why this exists (and its limits)

The Arc B580 runs SDXL's compute fast (~31 UNet it/s warm), but its memory is tight. Stock SDXL fp16 OOMs at VAE decode on 12GB because diffusers upcasts the VAE to fp32. Swapping in the fp16-fix VAE removes that upcast and the whole pipeline fits, at roughly 6.3s per warm 1024x1024 image, 25 steps.

Measured peak VRAM for the working config is 9.07 GB on a 12GB card, so there is roughly 2.5-3 GB of real headroom, not zero.

Known to work:

  • SDXL base, single image, 1024x1024, ~25 steps. ~6.3s warm.

Plausibly fits in the remaining headroom (UNTESTED, try before assuming):

  • A second batched image, or modest resolution bumps above 1024.
  • Light img2img.

Probably does NOT fit (a second resident model exceeds ~3GB):

  • SDXL refiner co-resident with base.

Does NOT fit:

  • Flux (any variant). Far heavier than the available margin.

If you want the refiner or Flux, that is the signal to move that tier of work to a RunPod serverless endpoint rather than fighting 12GB. Local covers the light case comfortably and may stretch a little past it; it does not reach the heavy tier.

Build

The Dockerfile is PINNED to the exact versions captured from the known-good test container (torch 2.12.1+xpu, diffusers 0.39.0, transformers 5.13.0, etc., on oneAPI 2025.3.2 / Ubuntu 24.04.4). These produced ~6.3s warm SDXL-base images with 9.07 GB peak VRAM on an Arc B580.

Build:

docker build -t imggen:local .

If the base image tag intel/oneapi-basekit:2025.3.2-0-devel-ubuntu24.04 is not available on Docker Hub, substitute the closest 2025.3.x tag and record what you used. Do not fall back to :latest.

Models

Three model keys, all SDXL-architecture, all sharing the fp16-fix VAE, all fitting the 12GB card. The registry in generate.py (MODELS) is the source of truth; swapping the default is a one-line key change.

key repo notes
sdxl stabilityai/stable-diffusion-xl-base-1.0 general purpose, default
realvis SG161222/RealVisXL_V5.0 photoreal, permissive license
pony kitty7779/ponyDiffusionV6XL unrestricted, versatile (ungated diffusers-format mirror; the original AstraliteHeart repo is gated)

imggen list prints this from the wrapper. Pick a model per invocation with -m KEY; without one, sdxl is the default (or the daemon's currently resident model, see below).

Setup

Set up the repo once:

cp .env.example .env

Edit .env with your real host paths and, if needed, your uid/gid:

IMGGEN_OUT=/path/to/output          # where PNGs land
IMGGEN_CACHE=/path/to/model-cache   # where the ~7.5GB weights are cached
IMGGEN_PORT=8765                    # daemon port, localhost-only
IMGGEN_MODEL=sdxl                   # model loaded on `imggen start`
IMGGEN_WIDTH=1280                   # default image width (multiple of 8)
IMGGEN_HEIGHT=960                   # default image height (multiple of 8)
IMGGEN_UID=1000                     # your host uid (id -u)
IMGGEN_GID=100                      # your host gid (id -g)

.env is gitignored; .env.example documents the keys. docker compose (via compose.yaml) reads it. The service runs as your host user (uid:gid plus group_add: video for /dev/dri access), so generated PNGs are owned by you, not root.

Install the wrapper

The imggen script lives in this repo and runs docker compose here; it cds into $IMGGEN_DIR (default $HOME/Programming/GIT/imggen) to find compose.yaml and .env. There is nothing to copy elsewhere. Run it in place:

./imggen "a red bicycle against a stone wall"

Or put a symlink to it on your PATH for convenience; it still operates on the repo via IMGGEN_DIR:

ln -s "$PWD/imggen" ~/bin/imggen

If the repo lives somewhere else, set IMGGEN_DIR accordingly.

Daemon / lifecycle

A persistent daemon (server.py, run via docker compose) keeps ONE model resident in XPU VRAM so repeated prompts skip the ~45s model reload. It holds one model at a time (three at once would exceed the 12GB card) and swaps on demand. It is serial by construction, one GPU, one request at a time.

imggen start [-m KEY]   # start daemon, load model (default sdxl), hold VRAM
imggen stop              # stop daemon, free VRAM
imggen status            # is it up? which model? (prints /status JSON, or "daemon down")
imggen list               # show the three model keys

VRAM stays held (~9GB) the whole time the daemon is up, until you run imggen stop. Don't leave it running unattended if you need that VRAM back.

A bare prompt routes automatically:

imggen "a foggy harbour at dawn" --steps 30 --seed 42
imggen "portrait of a fox" -m realvis -o fox.png -n "blurry, low quality"
  • If the daemon is UP, the prompt goes to it. With no -m, it uses the currently resident model as-is (the one you chose at imggen start), so repeated prompts just iterate on it. Pass -m KEY to override; if that differs from the resident model the daemon swaps first (a few seconds). A failed swap (bad repo, gated repo, OOM) prints the daemon's error and the wrapper exits 1.
  • If the daemon is DOWN, the wrapper falls back to a one-shot run: it loads the model, generates, and exits. Slower (pays the ~45s load every time) but requires no daemon management.

Size defaults to IMGGEN_WIDTHxIMGGEN_HEIGHT from .env (1280x960 landscape in the example), falling back to 1024x1024 if unset. Override per run with -W/-H. Both must be multiples of 8 (SDXL constraint); the daemon and the one-shot path reject non-multiples.

Output lands in $IMGGEN_OUT (from .env). Models cache in $IMGGEN_CACHE so the ~7.5GB download happens once.

First run

The first invocation downloads SDXL base + the fp16-fix VAE (~7.5GB) into the cache volume. That run is network-bound and slow. Subsequent runs skip the download. The first generation in any fresh container is also slower (~7.3s) due to one-time kernel compilation; warm runs settle to ~6.3s.

Upgrading

Treat upgrades as deliberate, tested events. Do not pip install -U casually. This working state depends on a specific torch-XPU build, a specific diffusers version, and the fp16-fix VAE. If you rebuild with newer versions, keep the old pinned Dockerfile in git so you can revert when something breaks.

Trial

This is a time-boxed experiment. Run it against real blog work for a few weeks. If it sticks, keep it. If it does not, the fallback is a RunPod serverless ComfyUI endpoint wrapped in the same imggen CLI shape, which trades local privacy for the ability to run heavier models (Flux, refiner, batching) on-demand at zero idle cost.

Development Approach

This project is developed using AI-assisted tools. Code is generated with the help of AI based on human-provided specifications, design decisions, and iterative feedback.

All contributions are reviewed, tested, and curated by the maintainer before being included in the codebase. AI is used as a productivity and exploration tool, while human oversight remains central to all decisions.

The goal is to combine the flexibility of AI-assisted development with standard open-source practices such as transparency, review, and accountability.