diff options
| -rw-r--r-- | CLAUDE.md | 56 | ||||
| -rw-r--r-- | README.md | 102 |
2 files changed, 132 insertions, 26 deletions
@@ -4,19 +4,30 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## What this is -Local SDXL-base image generation on an Intel Arc B580 (12GB) under Slackware. Three files: -`imggen` (host bash wrapper) → `docker run` → `generate.py` (in-container, XPU). README.md carries -the full rationale and VRAM measurements; read it before changing behavior. +Local SDXL image generation on an Intel Arc B580 (12GB) under Slackware. Five source files: +`imggen` (host bash wrapper: subcommands + daemon client + one-shot fallback) → `docker compose` +→ either `server.py` (persistent daemon, in-container, XPU) or `generate.py` (one-shot CLI, model +registry + resolve_model + build_pipe + generate), plus `compose.yaml` (compose service) and +`.env.example` (documents the gitignored `.env`). There is also `test_generate.py` (pure-logic +unit tests). README.md carries the full rationale and VRAM measurements; read it before changing +behavior. ## Build and run ``` -docker build -t imggen:local . # image tag is hardcoded as imggen:local in the wrapper -cp imggen ~/bin/imggen && chmod +x ~/bin/imggen -imggen "a prompt" --steps 30 --seed 42 -o out.png -n "blurry" +docker build -t imggen:local . # image tag is hardcoded as imggen:local in compose.yaml +cp .env.example .env # set real IMGGEN_OUT / IMGGEN_CACHE / IMGGEN_UID / IMGGEN_GID +./imggen start # start daemon, load default model, hold VRAM +./imggen "a prompt" --steps 30 --seed 42 -o out.png -n "blurry" +./imggen stop # free VRAM ``` -No test suite, no lint config. Verification is running `imggen` end to end and checking the PNG. +The wrapper lives in the repo and `cd`s to `$IMGGEN_DIR` (default `$HOME/Programming/GIT/imggen`) +so it can find `compose.yaml` and `.env`. It is not copied to `~/bin`; symlink it onto PATH if +convenient, it still operates on the repo via `IMGGEN_DIR`. + +`test_generate.py` covers pure logic (model registry, resolve_model) with no GPU needed. Beyond +that, no lint config; verification is running `imggen` end to end and checking the PNG. ## Architecture constraints (the whole point of the project) @@ -24,11 +35,18 @@ No test suite, no lint config. Verification is running `imggen` end to end and c the VAE to fp32. The fix is the fp16-fix VAE (`madebyollin/sdxl-vae-fp16-fix`, hardcoded in `generate.py`). This one swap is why the pipeline fits. Do not remove it. Peak VRAM in the working config is 9.07 GB. -- **Scope is deliberately bounded:** SDXL base, single 1024x1024 image, 25 steps, no refiner, - no Flux, no batching. Refiner/Flux/heavy work is explicitly out of scope and belongs on a - RunPod endpoint, not here. Don't add them "to be helpful." +- **Scope is deliberately bounded:** SDXL-architecture models, single 1024x1024 image, 25 steps, + no refiner, no Flux, no batching. The persistent daemon (`server.py`) is IN SCOPE: it holds one + model resident in VRAM for fast local burst-iteration, and that does not change the limits. + Refiner/Flux/batching/heavy work stays explicitly out of scope and belongs on a RunPod endpoint, + not here. Don't add them "to be helpful." - **XPU only.** `build_pipe()` fails loud (`sys.exit(2)`) if `torch.xpu` is unavailable rather than falling back to CPU (which would be minutes per image). Keep that behavior. +- **The model registry (`MODELS` in `generate.py`) is the source of truth.** Three keys: `sdxl` + (stabilityai/stable-diffusion-xl-base-1.0, default), `realvis` (SG161222/RealVisXL_V5.0), `pony` + (kitty7779/ponyDiffusionV6XL, an ungated diffusers-format mirror; the original AstraliteHeart + repo is gated). All three share the fp16-fix VAE and fit 12GB. The daemon holds only ONE model + at a time and swaps on demand; it never holds two co-resident (would exceed 12GB). ## Version pinning is load-bearing @@ -40,9 +58,15 @@ keep the old Dockerfile in git to revert. ## Container boundaries -- `imggen` bind-mounts host `$IMGGEN_OUT` (default `~/imggen-out`) → `/out`, and - `$IMGGEN_CACHE` (default `~/.cache/imggen-models`) → `/models`. `HF_HOME=/models` so the - ~7.5GB weights download once. Output paths in `generate.py` are all under `/out`. -- All wrapper args pass straight through to `generate.py` argparse. Adding a CLI flag means - editing `generate.py` only; the wrapper needs no change. -- GPU passthrough is `--device /dev/dri`. +- `docker compose` (via `compose.yaml`) replaces raw `docker run`. The image tag is still + `imggen:local`. The service runs as the HOST user (`user: "${IMGGEN_UID:-1000}:${IMGGEN_GID:-100}"` + plus `group_add: video` for `/dev/dri` access), so generated PNGs are host-owned, not root. +- Real host paths live in the gitignored `.env` (copy `.env.example` to create it), NOT in + hardcoded wrapper defaults. `.env` sets `IMGGEN_OUT` → `/out` and `IMGGEN_CACHE` → `/models` + (bind mounts), plus optionally `IMGGEN_PORT` (daemon port, default 8765, localhost-only), + `IMGGEN_MODEL` (default sdxl), `IMGGEN_UID`/`IMGGEN_GID`. `HF_HOME=/models` so the ~7.5GB + weights download once. Output paths in `generate.py`/`server.py` are all under `/out`. +- Daemon args (`generate` endpoint) and one-shot CLI args (`generate.py` argparse) are kept in + sync by hand in the wrapper; adding a flag means updating `generate.py`, `server.py`, and the + wrapper's arg parsing together. +- GPU passthrough is `--device /dev/dri` (via compose `devices:`). @@ -53,30 +53,104 @@ If the base image tag `intel/oneapi-basekit:2025.3.2-0-devel-ubuntu24.04` is not available on Docker Hub, substitute the closest 2025.3.x tag and record what you used. Do not fall back to `:latest`. +## Models + +Three model keys, all SDXL-architecture, all sharing the fp16-fix VAE, all +fitting the 12GB card. The registry in `generate.py` (`MODELS`) is the source +of truth; swapping the default is a one-line key change. + +| key | repo | notes | +|-----------|------------------------------------------|--------------------------------| +| `sdxl` | stabilityai/stable-diffusion-xl-base-1.0 | general purpose, default | +| `realvis` | SG161222/RealVisXL_V5.0 | photoreal, permissive license | +| `pony` | kitty7779/ponyDiffusionV6XL | unrestricted, versatile (ungated diffusers-format mirror; the original AstraliteHeart repo is gated) | + +`imggen list` prints this from the wrapper. Pick a model per invocation with +`-m KEY`; without one, `sdxl` is the default (or the daemon's currently +resident model, see below). + +## Setup + +Set up the repo once: + +``` +cp .env.example .env +``` + +Edit `.env` with your real host paths and, if needed, your uid/gid: + +``` +IMGGEN_OUT=/path/to/output # where PNGs land +IMGGEN_CACHE=/path/to/model-cache # where the ~7.5GB weights are cached +IMGGEN_PORT=8765 # daemon port, localhost-only +IMGGEN_MODEL=sdxl # model loaded on `imggen start` +IMGGEN_UID=1000 # your host uid (id -u) +IMGGEN_GID=100 # your host gid (id -g) +``` + +`.env` is gitignored; `.env.example` documents the keys. `docker compose` +(via `compose.yaml`) reads it. The service runs as your host user (uid:gid +plus `group_add: video` for `/dev/dri` access), so generated PNGs are owned +by you, not root. + ## Install the wrapper +The `imggen` script lives in this repo and runs `docker compose` here; it +`cd`s into `$IMGGEN_DIR` (default `$HOME/Programming/GIT/imggen`) to find +`compose.yaml` and `.env`. There is nothing to copy elsewhere. Run it in +place: + ``` -cp imggen ~/bin/imggen # or anywhere on your PATH -chmod +x ~/bin/imggen +./imggen "a red bicycle against a stone wall" ``` -## Usage +Or put a symlink to it on your PATH for convenience; it still operates on +the repo via `IMGGEN_DIR`: ``` -imggen "a red bicycle against a stone wall" -imggen "a foggy harbour at dawn" --steps 30 --seed 42 -imggen "portrait of a fox" -o fox.png -n "blurry, low quality" +ln -s "$PWD/imggen" ~/bin/imggen +``` + +If the repo lives somewhere else, set `IMGGEN_DIR` accordingly. + +## Daemon / lifecycle + +A persistent daemon (`server.py`, run via `docker compose`) keeps ONE model +resident in XPU VRAM so repeated prompts skip the ~45s model reload. It +holds one model at a time (three at once would exceed the 12GB card) and +swaps on demand. It is serial by construction, one GPU, one request at a +time. + +``` +imggen start [-m KEY] # start daemon, load model (default sdxl), hold VRAM +imggen stop # stop daemon, free VRAM +imggen status # is it up? which model? (prints /status JSON, or "daemon down") +imggen list # show the three model keys ``` -Output lands in `~/imggen-out` by default. Models cache in -`~/.cache/imggen-models` so the ~7.5GB download happens once. +**VRAM stays held (~9GB) the whole time the daemon is up, until you run +`imggen stop`.** Don't leave it running unattended if you need that VRAM +back. -Override paths with env vars: +A bare prompt routes automatically: ``` -IMGGEN_OUT=/srv/blog/img IMGGEN_CACHE=/mnt/models imggen "a lighthouse" +imggen "a foggy harbour at dawn" --steps 30 --seed 42 +imggen "portrait of a fox" -m realvis -o fox.png -n "blurry, low quality" ``` +- If the daemon is UP, the prompt goes to it. If the requested `-m` differs + from the resident model, the daemon swaps first (a few seconds). With no + `-m` and an interactive terminal, a menu picks the model. A failed swap + (bad repo, gated repo, OOM) prints the daemon's error and the wrapper + exits 1. +- If the daemon is DOWN, the wrapper falls back to a one-shot run: it loads + the model, generates, and exits. Slower (pays the ~45s load every time) + but requires no daemon management. + +Output lands in `$IMGGEN_OUT` (from `.env`). Models cache in `$IMGGEN_CACHE` +so the ~7.5GB download happens once. + ## First run The first invocation downloads SDXL base + the fp16-fix VAE (~7.5GB) into the @@ -98,3 +172,11 @@ If it sticks, keep it. If it does not, the fallback is a RunPod serverless ComfyUI endpoint wrapped in the same `imggen` CLI shape, which trades local privacy for the ability to run heavier models (Flux, refiner, batching) on-demand at zero idle cost. + +## Development Approach + +This project is developed using AI-assisted tools. Code is generated with the help of AI based on human-provided specifications, design decisions, and iterative feedback. + +All contributions are reviewed, tested, and curated by the maintainer before being included in the codebase. AI is used as a productivity and exploration tool, while human oversight remains central to all decisions. + +The goal is to combine the flexibility of AI-assisted development with standard open-source practices such as transparency, review, and accountability. |