diff options
| -rw-r--r-- | CLAUDE.md | 48 | ||||
| -rw-r--r-- | Dockerfile | 54 | ||||
| -rw-r--r-- | README.md | 100 | ||||
| -rw-r--r-- | generate.py | 128 | ||||
| -rw-r--r-- | imggen | 38 |
5 files changed, 368 insertions, 0 deletions
diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..3c19091 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,48 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## What this is + +Local SDXL-base image generation on an Intel Arc B580 (12GB) under Slackware. Three files: +`imggen` (host bash wrapper) → `docker run` → `generate.py` (in-container, XPU). README.md carries +the full rationale and VRAM measurements; read it before changing behavior. + +## Build and run + +``` +docker build -t imggen:local . # image tag is hardcoded as imggen:local in the wrapper +cp imggen ~/bin/imggen && chmod +x ~/bin/imggen +imggen "a prompt" --steps 30 --seed 42 -o out.png -n "blurry" +``` + +No test suite, no lint config. Verification is running `imggen` end to end and checking the PNG. + +## Architecture constraints (the whole point of the project) + +- **12GB VRAM is the hard limit.** Stock SDXL fp16 OOMs at VAE decode because diffusers upcasts + the VAE to fp32. The fix is the fp16-fix VAE (`madebyollin/sdxl-vae-fp16-fix`, hardcoded in + `generate.py`). This one swap is why the pipeline fits. Do not remove it. Peak VRAM in the + working config is 9.07 GB. +- **Scope is deliberately bounded:** SDXL base, single 1024x1024 image, 25 steps, no refiner, + no Flux, no batching. Refiner/Flux/heavy work is explicitly out of scope and belongs on a + RunPod endpoint, not here. Don't add them "to be helpful." +- **XPU only.** `build_pipe()` fails loud (`sys.exit(2)`) if `torch.xpu` is unavailable rather + than falling back to CPU (which would be minutes per image). Keep that behavior. + +## Version pinning is load-bearing + +The Dockerfile pins EXACT versions (torch 2.12.1+xpu, diffusers 0.39.0, transformers 5.13.0, +etc.) captured from the known-good container. XPU-for-diffusion is young and fast-moving. Never +`pip install -U` or loosen a pin casually. torch comes from the Intel XPU wheel index +(`download.pytorch.org/whl/xpu`); everything else from PyPI. If you rebuild with newer versions, +keep the old Dockerfile in git to revert. + +## Container boundaries + +- `imggen` bind-mounts host `$IMGGEN_OUT` (default `~/imggen-out`) → `/out`, and + `$IMGGEN_CACHE` (default `~/.cache/imggen-models`) → `/models`. `HF_HOME=/models` so the + ~7.5GB weights download once. Output paths in `generate.py` are all under `/out`. +- All wrapper args pass straight through to `generate.py` argparse. Adding a CLI flag means + editing `generate.py` only; the wrapper needs no change. +- GPU passthrough is `--device /dev/dri`. diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..222f7d2 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,54 @@ +# Copyright (C) 2026 Danilo +# Licensed under the GNU General Public License version 2. +# +# Local SDXL image generation on Intel Arc B580 under Slackware, via a +# container that carries the Intel Level Zero runtime the host distro lacks. +# +# Base image: intel/oneapi-basekit ships the Level Zero loader + compute +# runtime that make torch.xpu see the GPU. This is the piece Slackware does +# not package, and the reason the whole approach is containerized rather +# than native. +# +# IMPORTANT - version pinning: +# The tags below are placeholders you MUST replace with the EXACT versions +# that worked in your test session. XPU-for-diffusion is a young, fast-moving +# stack (IPEX was retired in 2026); an unpinned rebuild can silently break a +# working setup. Capture your real versions with: +# python3 -m pip freeze | grep -Ei 'torch|diffusers|transformers|accelerate|huggingface|numpy|pillow|safetensors' +# then paste them into the pip install line below, replacing the loose names. + +# Confirmed from the working container: oneAPI 2025.3, Ubuntu 24.04.4 Noble. +# If this exact tag is not on Docker Hub, pick the closest oneapi-basekit tag +# whose oneAPI version is 2025.3.x and record what you used. Do NOT use :latest. +FROM intel/oneapi-basekit:2025.3.2-0-devel-ubuntu24.04 + +# --- system python + pip (base image ships python3 but not pip) --- +RUN apt-get update \ + && apt-get install -y --no-install-recommends python3-pip python3-venv \ + && rm -rf /var/lib/apt/lists/* + +# --- python deps (PINNED to the confirmed working environment) --- +# torch from the Intel XPU wheel index; everything else from PyPI. +# These are the exact versions that produced ~6.3s warm SDXL-base images +# with peak VRAM 9.07 GB on an Arc B580. Do not bump casually. +RUN python3 -m pip install --break-system-packages \ + --index-url https://download.pytorch.org/whl/xpu \ + torch==2.12.1+xpu \ + && python3 -m pip install --break-system-packages \ + diffusers==0.39.0 \ + transformers==5.13.0 \ + accelerate==1.14.0 \ + safetensors==0.8.0 \ + huggingface_hub==1.22.0 \ + numpy==2.5.1 \ + pillow==12.3.0 + +# --- app --- +WORKDIR /app +COPY generate.py /app/generate.py + +# Model cache lives here; mount a host volume at this path to avoid +# re-downloading ~7.5GB of weights on every container start. +ENV HF_HOME=/models + +ENTRYPOINT ["python3", "/app/generate.py"] diff --git a/README.md b/README.md new file mode 100644 index 0000000..d52dbeb --- /dev/null +++ b/README.md @@ -0,0 +1,100 @@ +# imggen + +Local SDXL image generation on an Intel Arc B580 (12GB) under Slackware, via a +container that carries the Intel Level Zero runtime the host distro does not +package. Prompt in, PNG on the host out. + +## Why this exists (and its limits) + +The Arc B580 runs SDXL's *compute* fast (~31 UNet it/s warm), but its *memory* +is tight. Stock SDXL fp16 OOMs at VAE decode on 12GB because diffusers upcasts +the VAE to fp32. Swapping in the fp16-fix VAE removes that upcast and the whole +pipeline fits, at roughly 6.3s per warm 1024x1024 image, 25 steps. + +Measured peak VRAM for the working config is 9.07 GB on a 12GB card, so there +is roughly 2.5-3 GB of real headroom, not zero. + +Known to work: + +- SDXL base, single image, 1024x1024, ~25 steps. ~6.3s warm. + +Plausibly fits in the remaining headroom (UNTESTED, try before assuming): + +- A second batched image, or modest resolution bumps above 1024. +- Light img2img. + +Probably does NOT fit (a second resident model exceeds ~3GB): + +- SDXL refiner co-resident with base. + +Does NOT fit: + +- Flux (any variant). Far heavier than the available margin. + +If you want the refiner or Flux, that is the signal to move that tier of work +to a RunPod serverless endpoint rather than fighting 12GB. Local covers the +light case comfortably and may stretch a little past it; it does not reach the +heavy tier. + +## Build + +The Dockerfile is PINNED to the exact versions captured from the known-good +test container (torch 2.12.1+xpu, diffusers 0.39.0, transformers 5.13.0, etc., +on oneAPI 2025.3.2 / Ubuntu 24.04.4). These produced ~6.3s warm SDXL-base +images with 9.07 GB peak VRAM on an Arc B580. + +Build: + +``` +docker build -t imggen:local . +``` + +If the base image tag `intel/oneapi-basekit:2025.3.2-0-devel-ubuntu24.04` is +not available on Docker Hub, substitute the closest 2025.3.x tag and record +what you used. Do not fall back to `:latest`. + +## Install the wrapper + +``` +cp imggen ~/bin/imggen # or anywhere on your PATH +chmod +x ~/bin/imggen +``` + +## Usage + +``` +imggen "a red bicycle against a stone wall" +imggen "a foggy harbour at dawn" --steps 30 --seed 42 +imggen "portrait of a fox" -o fox.png -n "blurry, low quality" +``` + +Output lands in `~/imggen-out` by default. Models cache in +`~/.cache/imggen-models` so the ~7.5GB download happens once. + +Override paths with env vars: + +``` +IMGGEN_OUT=/srv/blog/img IMGGEN_CACHE=/mnt/models imggen "a lighthouse" +``` + +## First run + +The first invocation downloads SDXL base + the fp16-fix VAE (~7.5GB) into the +cache volume. That run is network-bound and slow. Subsequent runs skip the +download. The first *generation* in any fresh container is also slower (~7.3s) +due to one-time kernel compilation; warm runs settle to ~6.3s. + +## Upgrading + +Treat upgrades as deliberate, tested events. Do not `pip install -U` casually. +This working state depends on a specific torch-XPU build, a specific diffusers +version, and the fp16-fix VAE. If you rebuild with newer versions, keep the old +pinned Dockerfile in git so you can revert when something breaks. + +## Trial + +This is a time-boxed experiment. Run it against real blog work for a few weeks. +If it sticks, keep it. If it does not, the fallback is a RunPod serverless +ComfyUI endpoint wrapped in the same `imggen` CLI shape, which trades local +privacy for the ability to run heavier models (Flux, refiner, batching) +on-demand at zero idle cost. diff --git a/generate.py b/generate.py new file mode 100644 index 0000000..ba437ae --- /dev/null +++ b/generate.py @@ -0,0 +1,128 @@ +#!/usr/bin/env python3 +# Copyright (C) 2026 Danilo +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# Local SDXL-base image generation for Intel Arc B580 (XPU backend). +# Bounded scope by design: SDXL base, single 1024x1024 image, no refiner, +# no Flux, no batching. This is what fits comfortably in 12GB VRAM once the +# fp32 VAE upcast is removed via the fp16-fix VAE. See README.md for limits. + +import argparse +import sys +import time +from datetime import datetime +from pathlib import Path + +import torch +from diffusers import StableDiffusionXLPipeline, AutoencoderKL + +MODEL = "stabilityai/stable-diffusion-xl-base-1.0" +# fp16-fix VAE avoids the fp32 upcast that OOMs a 12GB card at decode time. +VAE = "madebyollin/sdxl-vae-fp16-fix" +OUT_DIR = Path("/out") + + +def build_pipe(): + if not torch.xpu.is_available(): + # Fail loud and specific. A silent CPU fallback would run but take + # minutes per image, which is worse than a clear error. + print( + "ERROR: torch.xpu is not available. The Level Zero runtime is not " + "visible inside this container. Check that you passed --device " + "/dev/dri and that the base image ships the Intel GPU runtime.", + file=sys.stderr, + ) + sys.exit(2) + + vae = AutoencoderKL.from_pretrained(VAE, torch_dtype=torch.float16) + pipe = StableDiffusionXLPipeline.from_pretrained( + MODEL, + vae=vae, + torch_dtype=torch.float16, + use_safetensors=True, + variant="fp16", + ).to("xpu") + return pipe + + +def generate(pipe, prompt, negative, steps, guidance, seed): + generator = None + if seed is not None: + generator = torch.Generator(device="xpu").manual_seed(seed) + + t = time.time() + image = pipe( + prompt, + negative_prompt=negative, + num_inference_steps=steps, + guidance_scale=guidance, + generator=generator, + ).images[0] + torch.xpu.synchronize() + elapsed = time.time() - t + return image, elapsed + + +def main(): + ap = argparse.ArgumentParser( + description="Generate a single SDXL-base image locally on Intel Arc (XPU)." + ) + ap.add_argument("prompt", help="Text prompt for the image.") + ap.add_argument( + "-n", "--negative", default="", help="Negative prompt (default: none)." + ) + ap.add_argument( + "-s", "--steps", type=int, default=25, help="Inference steps (default: 25)." + ) + ap.add_argument( + "-g", + "--guidance", + type=float, + default=7.0, + help="Guidance scale / CFG (default: 7.0).", + ) + ap.add_argument( + "--seed", + type=int, + default=None, + help="Seed for reproducible output (default: random).", + ) + ap.add_argument( + "-o", + "--out", + default=None, + help="Output filename inside /out (default: timestamped).", + ) + args = ap.parse_args() + + OUT_DIR.mkdir(parents=True, exist_ok=True) + + if args.out: + out_path = OUT_DIR / args.out + else: + stamp = datetime.now().strftime("%Y%m%d-%H%M%S") + out_path = OUT_DIR / f"sdxl-{stamp}.png" + + print(f"Loading SDXL base + fp16-fix VAE on XPU ...", file=sys.stderr) + t0 = time.time() + pipe = build_pipe() + print(f"Loaded in {time.time() - t0:.1f}s", file=sys.stderr) + + image, elapsed = generate( + pipe, args.prompt, args.negative, args.steps, args.guidance, args.seed + ) + image.save(out_path) + print(f"Generated in {elapsed:.2f}s") + print(f"Saved {out_path}") + + +if __name__ == "__main__": + main() @@ -0,0 +1,38 @@ +#!/bin/bash +# Copyright (C) 2026 Danilo +# Licensed under the GNU General Public License version 2. +# +# Host-side wrapper for local SDXL image generation on Intel Arc. +# Handles GPU passthrough, the /out bind-mount (so output lands on the host, +# never trapped inside the container), and a persistent model cache (so the +# ~7.5GB of weights download once, not every run). +# +# Usage: +# imggen "a red bicycle against a stone wall" +# imggen "a foggy harbour at dawn" --steps 30 --seed 42 +# imggen "portrait of a fox" -o fox.png -n "blurry, low quality" +# +# All arguments are passed straight through to generate.py. + +set -euo pipefail + +IMAGE="imggen:local" + +# Where generated images land on the host. Override with IMGGEN_OUT. +OUT_DIR="${IMGGEN_OUT:-$HOME/imggen-out}" + +# Persistent model cache on the host. Override with IMGGEN_CACHE. +CACHE_DIR="${IMGGEN_CACHE:-$HOME/.cache/imggen-models}" + +mkdir -p "$OUT_DIR" "$CACHE_DIR" + +if [ "$#" -eq 0 ]; then + echo "usage: imggen \"<prompt>\" [--steps N] [--guidance G] [--seed S] [-n \"<negative>\"] [-o name.png]" >&2 + exit 1 +fi + +exec docker run --rm -it \ + --device /dev/dri \ + -v "$OUT_DIR:/out" \ + -v "$CACHE_DIR:/models" \ + "$IMAGE" "$@" |