# Attachments: design

Date: 2026-10-16. Status: implemented (2026-10-16), live-checked.

## Goal

Let a question carry files. The quickshell canvas stages files dropped on
the island and sends their paths with `ask`; the daemon reads them, gives
the model their content, and keeps enough in the history for a reopened
conversation to still talk about them. The canvas side is specified in the
quickshell repo, `docs/superpowers/specs/2026-10-16-file-drop-design.md`.

This is piece 1 of three. Piece 2 is per-type model prompt presets
(summarize, describe, explain) and piece 3 local file operations (PDF to
markdown, merge, video to audio or GIF, trackcrop); each gets its own spec.

## Verified before designing

- Gemma4-12B on the router reports `input_modalities` of
  `["text", "image", "audio"]` in `/v1/models`; Qwen3.8-9B reports
  `["text"]`. The field is how vision is detected.
- `pdftotext` and `pdftoppm` (poppler 26.09.0) are installed.
- `tesseract` is installed with **no language data** (`/usr/share/tessdata`
  lists 0 languages), so it cannot OCR. Scanned pages go to the model as
  images instead, which Gemma reads.
- llamachat's attachment code (`llamachat/backend.py`) is the model:
  `--- file: name ---` blocks for text, `image_url` data URLs for images,
  truncation to a character budget. Read for ideas; this is a separate
  implementation.

## Protocol

`ask` gains an optional `files`: a list of absolute paths, at most 8.

```json
{"type": "ask", "text": "What changed?", "files": ["/home/you/report.pdf"]}
```

The daemon runs as the user and reads the paths itself, so nothing is
copied over the socket. `text` may be empty when files are given; the
model then gets the files alone.

The `user` event and each shown turn gain `files`:
`[{name, kind, pages, truncated}]` (`pages` for a PDF, else absent), for
the canvas's chips.

Errors answer with `error` (with `turn`, since they end the reply before it
starts) and nothing is sent to the model:

- not an absolute path, missing, not a regular file, or unreadable;
- more than 8 files;
- an image over 20 MB;
- a kind chat does not take: "chat supports text, images and PDF";
- an image, or a PDF needing page images, for a model whose
  `input_modalities` lacks `image`: "<model> cannot see images".

## Extraction

| kind | detected by | the model gets |
| --- | --- | --- |
| text | a `text/*` mime type, or a code suffix (llamachat's list) | the file as UTF-8 (errors replaced) in a `--- file: name ---` block |
| image | png, jpeg, webp, gif | an `image_url` part, a base64 data URL |
| pdf | `.pdf` | `pdftotext -layout` per page in one block; a page with no text is rendered with `pdftoppm -png` at 150 dpi, lowered so the longest side stays within 1600 px (never raised), and sent as an image, at most 4 such pages (further ones are listed as not shown and the chip is truncated) |
| other | anything else | refused, see above |

Text blocks share a budget: half the free context,
`(ctx_max - ctx_used) / 2` tokens at 3 characters per token. A file cut to
fit is marked `truncated` in its block header and its chip. With
`ctx_max` unknown (0), the budget is 24000 characters.

## History

`turns` gains a `files` column (JSON, the chip list), added with
`ALTER TABLE turns ADD COLUMN files TEXT` when `PRAGMA table_info` shows it
missing, so an existing `history.db` keeps working.

In `turns.messages`, extracted text is stored inline: a reopened
conversation still has the content. Images are stored as a reference part
`{"type": "image_ref", "name", "path", "sha256"}`. When history is sent to
the model, each reference is resolved: if the path still holds a file with
that hash, it becomes the `image_url` again; otherwise the text
`[image <name> is no longer available]`. Rendered PDF pages are stored the
same way, with the PDF's path, its hash and the page number, and re-rendered
on resolve.

## Testing

`python3 assistant.py --test` gains:

- classification of a text file, a code suffix, an image, a PDF and an
  unknown file;
- a PDF built in the test with `magick` (one page of text, one blank), whose
  extraction gives the text and one page image;
- truncation at a small budget, with the block marked;
- an image reference that resolves after a save and reload, and becomes the
  unavailable text once the file changes;
- the vision error, against the fake server's `/models` reporting
  `["text"]`;
- each refusal above.