aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorDanilo M. <danix@danix.xyz>2026-10-16 15:12:06 +0200
committerDanilo M. <danix@danix.xyz>2026-10-16 15:12:06 +0200
commit056853313dd73975a8af52e151af5eed33253fc7 (patch)
tree81e6b8eb932aedf77501e17dffc60e362c796297
parent39d1ffbbb7cdaacb3b653545461bf01e3d62ad67 (diff)
downloaddesktop-assistant-056853313dd73975a8af52e151af5eed33253fc7.tar.gz
desktop-assistant-056853313dd73975a8af52e151af5eed33253fc7.zip
Docs: attachments
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-rw-r--r--AGENTS.md14
-rw-r--r--README.md11
-rw-r--r--docs/superpowers/specs/2026-10-16-attachments-design.md4
3 files changed, 26 insertions, 3 deletions
diff --git a/AGENTS.md b/AGENTS.md
index e25ec1e..427fdaa 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -120,6 +120,20 @@ not import from it.
## Findings
+### Attachments, live check (2026-10-16)
+
+- An image reading "BLUE 42" (`magick`, 400x120) was answered `BLUE 42`,
+ first token at 6.1 s, ctx 1242 / 32768.
+- A two-page PDF, a text page and an image-only page, was answered with
+ facts from both: the text page went inline, the image-only page as a
+ `pdftoppm` render.
+- A page render needs a bound: at `-r 150` a 3000 pt page came out
+ 6250x6250 at 171 MB peak (review probe). Renders use `-scale-to 2048`.
+- tesseract is installed with no language data, so OCR is the model's
+ vision, not tesseract.
+- A Unix socket path is limited to 108 bytes: a test socket under a deep
+ scratch directory fails with `AF_UNIX path too long`.
+
### Text daemon, live check (2026-10-16)
Model Gemma4-12B-qat-mtp, ctx 32768, preset reasoning-budget 4096.
diff --git a/README.md b/README.md
index 61c2aa8..6a8c767 100644
--- a/README.md
+++ b/README.md
@@ -17,6 +17,7 @@ The voice version (push-to-talk, whisper, Kokoro) is the tag `voice-v1`.
- a running `llama-server` with an OpenAI-compatible API, router mode
- Python 3 (stdlib only; SQLite with FTS5)
+- poppler (`pdftotext`, `pdftoppm`) for PDF attachments
- `curl` and `jq` for the weather and search example tools,
`python3-dateutil` for the calendar one
@@ -42,7 +43,7 @@ first answers.
JSON lines both ways on `$XDG_RUNTIME_DIR/desktop-assistant.sock` (mode
0600). Every connected client sees every event.
-In: `ask {text}`, `stop`, `new`, `list`, `open {id}`, `search {q}`,
+In: `ask {text, files}`, `stop`, `new`, `list`, `open {id}`, `search {q}`,
`delete {id}`.
Out: `hello` on connect (state, model, context use, the current
@@ -54,6 +55,14 @@ conversation and any reply in flight), then `user`, `think`, `token`,
Ids are integers; a malformed command or argument answers `error` and the
connection stays open.
+`files` is a list of absolute paths, at most 8. Text and code files and
+PDFs are read and inlined, sharing half the free context (a PDF page with
+no text goes as an image, at most 4, rendered no larger than 2048 px);
+images go as image parts to a model that reports seeing them. History
+keeps the text and, for images, the path and hash: a file that changed or
+moved is reported as no longer available when the conversation is
+reopened, and a model that cannot see images gets a note instead.
+
Try it by hand:
```sh
diff --git a/docs/superpowers/specs/2026-10-16-attachments-design.md b/docs/superpowers/specs/2026-10-16-attachments-design.md
index d8a8092..2b56f7d 100644
--- a/docs/superpowers/specs/2026-10-16-attachments-design.md
+++ b/docs/superpowers/specs/2026-10-16-attachments-design.md
@@ -1,6 +1,6 @@
# Attachments: design
-Date: 2026-10-16. Status: approved in brainstorming, not implemented.
+Date: 2026-10-16. Status: implemented (2026-10-16), live-checked.
## Goal
@@ -60,7 +60,7 @@ starts) and nothing is sent to the model:
| --- | --- | --- |
| text | a `text/*` mime type, or a code suffix (llamachat's list) | the file as UTF-8 (errors replaced) in a `--- file: name ---` block |
| image | png, jpeg, webp, gif | an `image_url` part, a base64 data URL |
-| pdf | `.pdf` | `pdftotext -layout` per page in one block; a page with no text is rendered with `pdftoppm -r 150 -png` and sent as an image, at most 4 such pages |
+| pdf | `.pdf` | `pdftotext -layout` per page in one block; a page with no text is rendered with `pdftoppm -r 150 -scale-to 2048 -png` and sent as an image, at most 4 such pages (further ones are listed as not shown and the chip is truncated) |
| other | anything else | refused, see above |
Text blocks share a budget: half the free context,