diff options
| author | Danilo M. <danix@danix.xyz> | 2026-10-16 15:12:06 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-10-16 15:12:06 +0200 |
| commit | 056853313dd73975a8af52e151af5eed33253fc7 (patch) | |
| tree | 81e6b8eb932aedf77501e17dffc60e362c796297 | |
| parent | 39d1ffbbb7cdaacb3b653545461bf01e3d62ad67 (diff) | |
| download | desktop-assistant-056853313dd73975a8af52e151af5eed33253fc7.tar.gz desktop-assistant-056853313dd73975a8af52e151af5eed33253fc7.zip | |
Docs: attachments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| -rw-r--r-- | AGENTS.md | 14 | ||||
| -rw-r--r-- | README.md | 11 | ||||
| -rw-r--r-- | docs/superpowers/specs/2026-10-16-attachments-design.md | 4 |
3 files changed, 26 insertions, 3 deletions
@@ -120,6 +120,20 @@ not import from it. ## Findings +### Attachments, live check (2026-10-16) + +- An image reading "BLUE 42" (`magick`, 400x120) was answered `BLUE 42`, + first token at 6.1 s, ctx 1242 / 32768. +- A two-page PDF, a text page and an image-only page, was answered with + facts from both: the text page went inline, the image-only page as a + `pdftoppm` render. +- A page render needs a bound: at `-r 150` a 3000 pt page came out + 6250x6250 at 171 MB peak (review probe). Renders use `-scale-to 2048`. +- tesseract is installed with no language data, so OCR is the model's + vision, not tesseract. +- A Unix socket path is limited to 108 bytes: a test socket under a deep + scratch directory fails with `AF_UNIX path too long`. + ### Text daemon, live check (2026-10-16) Model Gemma4-12B-qat-mtp, ctx 32768, preset reasoning-budget 4096. @@ -17,6 +17,7 @@ The voice version (push-to-talk, whisper, Kokoro) is the tag `voice-v1`. - a running `llama-server` with an OpenAI-compatible API, router mode - Python 3 (stdlib only; SQLite with FTS5) +- poppler (`pdftotext`, `pdftoppm`) for PDF attachments - `curl` and `jq` for the weather and search example tools, `python3-dateutil` for the calendar one @@ -42,7 +43,7 @@ first answers. JSON lines both ways on `$XDG_RUNTIME_DIR/desktop-assistant.sock` (mode 0600). Every connected client sees every event. -In: `ask {text}`, `stop`, `new`, `list`, `open {id}`, `search {q}`, +In: `ask {text, files}`, `stop`, `new`, `list`, `open {id}`, `search {q}`, `delete {id}`. Out: `hello` on connect (state, model, context use, the current @@ -54,6 +55,14 @@ conversation and any reply in flight), then `user`, `think`, `token`, Ids are integers; a malformed command or argument answers `error` and the connection stays open. +`files` is a list of absolute paths, at most 8. Text and code files and +PDFs are read and inlined, sharing half the free context (a PDF page with +no text goes as an image, at most 4, rendered no larger than 2048 px); +images go as image parts to a model that reports seeing them. History +keeps the text and, for images, the path and hash: a file that changed or +moved is reported as no longer available when the conversation is +reopened, and a model that cannot see images gets a note instead. + Try it by hand: ```sh diff --git a/docs/superpowers/specs/2026-10-16-attachments-design.md b/docs/superpowers/specs/2026-10-16-attachments-design.md index d8a8092..2b56f7d 100644 --- a/docs/superpowers/specs/2026-10-16-attachments-design.md +++ b/docs/superpowers/specs/2026-10-16-attachments-design.md @@ -1,6 +1,6 @@ # Attachments: design -Date: 2026-10-16. Status: approved in brainstorming, not implemented. +Date: 2026-10-16. Status: implemented (2026-10-16), live-checked. ## Goal @@ -60,7 +60,7 @@ starts) and nothing is sent to the model: | --- | --- | --- | | text | a `text/*` mime type, or a code suffix (llamachat's list) | the file as UTF-8 (errors replaced) in a `--- file: name ---` block | | image | png, jpeg, webp, gif | an `image_url` part, a base64 data URL | -| pdf | `.pdf` | `pdftotext -layout` per page in one block; a page with no text is rendered with `pdftoppm -r 150 -png` and sent as an image, at most 4 such pages | +| pdf | `.pdf` | `pdftotext -layout` per page in one block; a page with no text is rendered with `pdftoppm -r 150 -scale-to 2048 -png` and sent as an image, at most 4 such pages (further ones are listed as not shown and the chip is truncated) | | other | anything else | refused, see above | Text blocks share a budget: half the free context, |
