1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
|
# TODO / future ideas
Ideas for later, to brainstorm and spec before building. Nothing here is committed
to scope yet. Each entry should go through the brainstorming flow (like the model
menu + daemon work did) before implementation.
## Image editing (img2img)
Pass an existing image plus a prompt describing the edits, get a modified image back.
**Why it fits:** This is SDXL img2img. Same base model, same fp16-fix VAE, same 12GB
budget. `AutoPipelineForImage2Image.from_pipe(txt2img_pipe)` derives the img2img pipeline
from the already-resident model, so no second load and no extra VRAM. Still SDXL-arch,
single image, no refiner/Flux/batching, so it stays within the project's bounded scope.
**Rough shape (to be confirmed in brainstorming):**
- `generate.py`: a `generate_img2img(pipe, image, prompt, ..., strength)`; load the input
via PIL, resize to a multiple-of-8 dimension. `build_pipe` unchanged; derive the img2img
pipe from it.
- `server.py`: a new `/edit` endpoint. Image transport likely base64-in-JSON (simple,
stdlib-only, fits the jq-free / no-new-dep spirit; ~33% size bloat is fine for one local
image). Reuses the resident model.
- `imggen` wrapper: either a subcommand `imggen edit input.png "make it snowy" -o out.png
[--strength 0.6]`, or an `-i/--input` flag on the normal prompt path. Subcommand is
probably cleaner since start/stop/status/list already use subcommands.
- Registry unchanged: all three models (sdxl/realvis/pony) can do img2img.
- `strength` is the key new knob: 0 = ignore prompt (return input), 1 = ignore input
(pure txt2img); ~0.5-0.75 typical for edits. Leave it exposed for tuning.
**Open questions to resolve first:**
1. Interface: `edit` subcommand vs `-i/--input` flag.
2. Image transport to the daemon: base64-in-JSON vs multipart.
3. Inpainting (mask a region to edit) is a separate third pipeline
(`XLInpaintPipeline` + a mask input), more surface. Start with whole-image img2img;
add masked inpainting later only if wanted.
**VRAM:** img2img peak is roughly the same as txt2img (same UNet passes, often fewer steps
since it starts from a noised input), so it fits 12GB. Verify on the real Arc during build.
|
