1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
|
# imggen
Local SDXL image generation on an Intel Arc B580 (12GB) under Slackware, via a
container that carries the Intel Level Zero runtime the host distro does not
package. Prompt in, PNG on the host out.
## Why this exists (and its limits)
The Arc B580 runs SDXL's *compute* fast (~31 UNet it/s warm), but its *memory*
is tight. Stock SDXL fp16 OOMs at VAE decode on 12GB because diffusers upcasts
the VAE to fp32. Swapping in the fp16-fix VAE removes that upcast and the whole
pipeline fits, at roughly 6.3s per warm 1024x1024 image, 25 steps.
Measured peak VRAM for the working config is 9.07 GB on a 12GB card, so there
is roughly 2.5-3 GB of real headroom, not zero.
Known to work:
- SDXL base, single image, 1024x1024, ~25 steps. ~6.3s warm.
Plausibly fits in the remaining headroom (UNTESTED, try before assuming):
- A second batched image, or modest resolution bumps above 1024.
- Light img2img.
Probably does NOT fit (a second resident model exceeds ~3GB):
- SDXL refiner co-resident with base.
Does NOT fit:
- Flux (any variant). Far heavier than the available margin.
If you want the refiner or Flux, that is the signal to move that tier of work
to a RunPod serverless endpoint rather than fighting 12GB. Local covers the
light case comfortably and may stretch a little past it; it does not reach the
heavy tier.
## Build
The Dockerfile is PINNED to the exact versions captured from the known-good
test container (torch 2.12.1+xpu, diffusers 0.39.0, transformers 5.13.0, etc.,
on oneAPI 2025.3.2 / Ubuntu 24.04.4). These produced ~6.3s warm SDXL-base
images with 9.07 GB peak VRAM on an Arc B580.
Build:
```
docker build -t imggen:local .
```
If the base image tag `intel/oneapi-basekit:2025.3.2-0-devel-ubuntu24.04` is
not available on Docker Hub, substitute the closest 2025.3.x tag and record
what you used. Do not fall back to `:latest`.
## Models
Three model keys, all SDXL-architecture, all sharing the fp16-fix VAE, all
fitting the 12GB card. The registry in `generate.py` (`MODELS`) is the source
of truth; swapping the default is a one-line key change.
| key | repo | notes |
|-----------|------------------------------------------|--------------------------------|
| `sdxl` | stabilityai/stable-diffusion-xl-base-1.0 | general purpose, default |
| `realvis` | SG161222/RealVisXL_V5.0 | photoreal, permissive license |
| `pony` | kitty7779/ponyDiffusionV6XL | unrestricted, versatile (ungated diffusers-format mirror; the original AstraliteHeart repo is gated) |
`imggen list` prints this from the wrapper. Pick a model per invocation with
`-m KEY`; without one, `sdxl` is the default (or the daemon's currently
resident model, see below).
## Setup
Set up the repo once:
```
cp .env.example .env
```
Edit `.env` with your real host paths and, if needed, your uid/gid:
```
IMGGEN_OUT=/path/to/output # where PNGs land
IMGGEN_CACHE=/path/to/model-cache # where the ~7.5GB weights are cached
IMGGEN_PORT=8765 # daemon port, localhost-only
IMGGEN_MODEL=sdxl # model loaded on `imggen start`
IMGGEN_WIDTH=1280 # default image width (multiple of 8)
IMGGEN_HEIGHT=960 # default image height (multiple of 8)
IMGGEN_UID=1000 # your host uid (id -u)
IMGGEN_GID=100 # your host gid (id -g)
```
`.env` is gitignored; `.env.example` documents the keys. `docker compose`
(via `compose.yaml`) reads it. The service runs as your host user (uid:gid
plus `group_add: video` for `/dev/dri` access), so generated PNGs are owned
by you, not root.
## Install the wrapper
The `imggen` script lives in this repo and runs `docker compose` here; it
`cd`s into `$IMGGEN_DIR` (default `$HOME/Programming/GIT/imggen`) to find
`compose.yaml` and `.env`. There is nothing to copy elsewhere. Run it in
place:
```
./imggen "a red bicycle against a stone wall"
```
Or put a symlink to it on your PATH for convenience; it still operates on
the repo via `IMGGEN_DIR`:
```
ln -s "$PWD/imggen" ~/bin/imggen
```
If the repo lives somewhere else, set `IMGGEN_DIR` accordingly.
## Daemon / lifecycle
A persistent daemon (`server.py`, run via `docker compose`) keeps ONE model
resident in XPU VRAM so repeated prompts skip the ~45s model reload. It
holds one model at a time (three at once would exceed the 12GB card) and
swaps on demand. It is serial by construction, one GPU, one request at a
time.
```
imggen start [-m KEY] # start daemon, load model (default sdxl), hold VRAM
imggen stop # stop daemon, free VRAM
imggen status # is it up? which model? (prints /status JSON, or "daemon down")
imggen list # show the three model keys
```
**VRAM stays held (~9GB) the whole time the daemon is up, until you run
`imggen stop`.** Don't leave it running unattended if you need that VRAM
back.
A bare prompt routes automatically:
```
imggen "a foggy harbour at dawn" --steps 30 --seed 42
imggen "portrait of a fox" -m realvis -o fox.png -n "blurry, low quality"
```
- If the daemon is UP, the prompt goes to it. With no `-m`, it uses the
currently resident model as-is (the one you chose at `imggen start`), so
repeated prompts just iterate on it. Pass `-m KEY` to override; if that
differs from the resident model the daemon swaps first (a few seconds). A
failed swap (bad repo, gated repo, OOM) prints the daemon's error and the
wrapper exits 1.
- If the daemon is DOWN, the wrapper falls back to a one-shot run: it loads
the model, generates, and exits. Slower (pays the ~45s load every time)
but requires no daemon management.
Size defaults to `IMGGEN_WIDTH`x`IMGGEN_HEIGHT` from `.env` (1280x960 landscape
in the example), falling back to 1024x1024 if unset. Override per run with
`-W`/`-H`. Both must be multiples of 8 (SDXL constraint); the daemon and the
one-shot path reject non-multiples.
Output lands in `$IMGGEN_OUT` (from `.env`). Models cache in `$IMGGEN_CACHE`
so the ~7.5GB download happens once.
## First run
The first invocation downloads SDXL base + the fp16-fix VAE (~7.5GB) into the
cache volume. That run is network-bound and slow. Subsequent runs skip the
download. The first *generation* in any fresh container is also slower (~7.3s)
due to one-time kernel compilation; warm runs settle to ~6.3s.
## Upgrading
Treat upgrades as deliberate, tested events. Do not `pip install -U` casually.
This working state depends on a specific torch-XPU build, a specific diffusers
version, and the fp16-fix VAE. If you rebuild with newer versions, keep the old
pinned Dockerfile in git so you can revert when something breaks.
## Trial
This is a time-boxed experiment. Run it against real blog work for a few weeks.
If it sticks, keep it. If it does not, the fallback is a RunPod serverless
ComfyUI endpoint wrapped in the same `imggen` CLI shape, which trades local
privacy for the ability to run heavier models (Flux, refiner, batching)
on-demand at zero idle cost.
## Development Approach
This project is developed using AI-assisted tools. Code is generated with the help of AI based on human-provided specifications, design decisions, and iterative feedback.
All contributions are reviewed, tested, and curated by the maintainer before being included in the codebase. AI is used as a productivity and exploration tool, while human oversight remains central to all decisions.
The goal is to combine the flexibility of AI-assisted development with standard open-source practices such as transparency, review, and accountability.
|
