From 333833ab80dcbd5434aae120c76fc7b60b897cd1 Mon Sep 17 00:00:00 2001 From: "Danilo M." Date: Fri, 16 Oct 2026 15:38:12 +0200 Subject: Docs: images past the ubatch kill the model; the preset fix Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index a02bce1..f265ffe 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -132,6 +132,17 @@ not import from it. - With the render fixed, the same scan (now 1250x625) plus the BLUE 42 image, sent together, were answered from both in one reply, first token at 8.4 s, ctx 1615 / 32768; the model stayed loaded. +- **An image larger than llama-server's ubatch kills the model process.** + Gemma 4's vision encoder is non-causal, and llama.cpp asserts + `n_ubatch >= n_tokens` per image (`llama-context.cpp:1784`, in + `/var/log/llama-server/server.log`). Tokens are 48 px patches: an A4 scan + rendered at 1132x1601 became 1152x1584, 24x33 = 792 tokens, past the + default ubatch of 512; 1250x625 (338) had worked, 2048x1024 (903) had + not. The fix is the preset, not the client, since a large photo crosses + 512 as easily: `ubatch-size = 2048` and `batch-size = 2048` in + `/etc/llama-server/presets.ini`, `[Gemma4-12B-qat-mtp]` (set 2026-10-16; + check `/v1/models` args for `--ubatch-size 2048`). After it, the same + two-page scan was answered, first token at 17.7 s, ctx 2782 / 32768. - tesseract is installed with no language data, so OCR is the model's vision, not tesseract. - A Unix socket path is limited to 108 bytes: a test socket under a deep -- cgit v1.2.3