1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
|
# Text assistant daemon: design
Date: 2026-10-16. Status: approved in brainstorming, not implemented.
Supersedes `2026-09-26-overlay-design.md` (the voice overlay protocol).
## Goal
Turn desktop-assistant from a push-to-talk voice assistant into a local
text assistant daemon, the single brain behind a Luma-style conversation
canvas in the quickshell desktop (see the quickshell repo,
`docs/superpowers/specs/2026-10-16-assistant-canvas-design.md`). The canvas
is only a view: the daemon owns the model, the tools and the history.
Voice is dropped for now, not abandoned. The last voice commit is tagged
`voice-v1` before any deletion, and future voice work starts from the tag
rather than from dormant code behind a flag.
The canvas takes llamachat's place on SUPER+Home while it is tested.
llamachat keeps running, unbound, and its future is undecided.
## Out of scope for v1
- Voice in any form (STT, TTS, mic button, levels).
- A model picker. The daemon keeps using whichever model the router has
loaded, so it never swaps a model out under another client.
- Attachments, cloud providers, skills: llamachat features that may be
ported later.
- Model-written conversation titles.
- Importing llamachat's existing history. Its schema stores no tool calls.
## What is deleted
whisper start/stop, `stt`, `transcribe`, Kokoro and `speaker`, `arecord`,
`aplay`, the `amixer events` loop and `mic_on`, `user_levels`,
`level_emitter`, `envelope`, `pcm_level`, `scale`, the presentation gate
(`presenting`, `notify_ignored`), `announcement`, `split_sentences`,
`lang_of`, `latin`, and the `[overlay]` config table. Requirements drop to
a running llama-server, Python 3 (stdlib, plus whatever the user's tools
need) and `curl`/`jq` for the example tools.
## What is kept
`serve` and `emit`, the multi-client JSON-lines socket at
`$XDG_RUNTIME_DIR/desktop-assistant.sock` (mode 0600). `ask` streaming,
`run_tool`, `collect_calls`, the TOML tools and the example tools.
`llm_model()` with its never-swap rule, `context_size()`. The history rule:
tool calls are kept and tool results replaced by a placeholder, with the
measurements already recorded in `reply()`. The date stamp on the system
prompt and the question. The last 10 turns are sent to the model. Config
merging, `--test`, `assistant.sh`.
## What changes
- **Thinking.** `chat_template_kwargs.enable_thinking` becomes true.
`delta.reasoning_content` is streamed as its own `think` event, so the
canvas can collapse it. Reasoning is not stored and not sent back to the
model.
- **System prompt.** The default is rewritten for text: markdown is
allowed, there is no instruction to answer in short spoken sentences.
- **Input.** The mic loop is replaced by a reader thread per client. Each
line a client writes is one JSON command. `main()` serves the socket,
opens the database and waits for SIGTERM.
- **Turns.** A command `ask` starts `reply()` on a thread, exactly as a
button release did. A global `turn` counter still marks stale replies.
- **Errors.** llama-server unreachable, an HTTP error or a malformed
stream emit `error` and return the state to idle, instead of killing the
thread. Every exit path of `main()` and of a reply thread logs a line.
This closes the old debt of the assistant dying with nothing in the log.
## Protocol
JSON lines both ways on the existing socket. Every connected client sees
every outbound event, so a canvas reopened mid-reply catches up.
Inbound:
| Command | Effect |
| --- | --- |
| `{"type":"ask","text":"..."}` | starts a turn in the current conversation |
| `{"type":"stop"}` | bumps `turn`: the stream stops, a tool already running finishes and its result is dropped |
| `{"type":"new"}` | starts an empty current conversation (the old one is already saved) |
| `{"type":"list"}` | answers `conversations` |
| `{"type":"open","id":N}` | answers `conversation`; N becomes current |
| `{"type":"search","q":"..."}` | answers `results` |
| `{"type":"delete","id":N}` | deletes N, answers `conversations`; deleting the current one acts as `new` |
An `ask` while a reply is streaming stops the old reply first. An unknown
or malformed line is logged and answered with `error`; it never closes the
connection.
Outbound:
| Event | Fields |
| --- | --- |
| `hello` | `state`, `model`, `ctx_used`, `ctx_max`, `id`, `turns` (the current conversation) |
| `state` | `state`: `idle`, `thinking`, `tool` (with `tool`) |
| `user` | `text`: the question, echoed so every client draws it |
| `think` | `delta` |
| `token` | `delta` |
| `tool` | `name`, `done` (false when it starts, true when it returns) |
| `done` | `id`: the conversation the turn was saved in |
| `error` | `message` |
| `ctx` | `used`, `max` |
| `conversations` | `items`: `[{id, title, updated}]`, newest first |
| `conversation` | `id`, `turns`: `[{user, reply, tools}]` |
| `results` | `items`: `[{id, title, snippet}]`, best first |
`hello` is sent on connect. A reply that was stopped emits `done` with
whatever text it had, and that partial turn is not saved.
## Storage
SQLite with FTS5, stdlib `sqlite3`. Verified on the system interpreter:
SQLite 3.53.4, and `tokenize='unicode61 remove_diacritics 2'` matches
`citta` against `città`, which matters for Italian.
One file, `~/.local/share/desktop-assistant/history.db`
(`$XDG_DATA_HOME` honoured), directory 0700, file 0600, WAL mode. It is
personal data at rest and never goes in a repository.
```sql
CREATE TABLE conversations(id INTEGER PRIMARY KEY, title TEXT,
created INTEGER, updated INTEGER);
CREATE TABLE turns(id INTEGER PRIMARY KEY,
conv INTEGER REFERENCES conversations ON DELETE CASCADE,
seq INTEGER, user TEXT, reply TEXT,
tools TEXT, -- JSON list of tool names, for display
messages TEXT); -- JSON: what the model gets for this turn
CREATE VIRTUAL TABLE turns_fts USING fts5(user, reply, content='turns',
content_rowid='id', tokenize='unicode61 remove_diacritics 2');
```
Triggers on `turns` keep `turns_fts` in sync for insert and delete;
`PRAGMA foreign_keys=ON` makes the conversation delete cascade, and the
delete trigger cleans the index.
- A conversation row is created at its first `done`, so an empty `new`
leaves nothing behind. Its title is the first six words of the first
question.
- One `turns` row is inserted per `done`, and `updated` is bumped. A crash
loses at most the reply in flight.
- `open` loads the last 10 turns' `messages` as the model history and all
turns' `user`, `reply`, `tools` for the canvas. Tool results come back as
the stored placeholder.
- `search` wraps the user's text as an FTS5 phrase (double quotes doubled
inside), so FTS syntax in the query cannot break or widen it. Results are
ranked by `bm25` and grouped to one row per conversation, with
`snippet()` from its best turn.
- One connection guarded by a lock: the reply threads and the client
reader threads all go through it. Only the daemon touches the file.
## assistant.sh
The overlay handling goes: `start`, `stop`, `restart` and `status` cover
the daemon only. `status` reports the process, the socket and the log's
tail, and exits 1 when the daemon is down.
## Testing
`python3 assistant.py --test`, extended:
- the existing socketpair check, now in both directions;
- a fake LLM stream (a local HTTP server on an ephemeral port) that sends
reasoning, content and a tool call, checking the `think`, `token`,
`tool`, `done` order on the socket;
- `stop` in the middle of the stream: no further `token`, the partial turn
not saved;
- the schema against `:memory:`: two conversations, a search with an
accent-free query finding an accented word, a search containing `"` and
`*`, a delete that removes the conversation's rows from `turns_fts`.
Manual check with
`socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/desktop-assistant.sock` against the
real router before the canvas exists.
## Docs
README and AGENTS.md are rewritten around "local text assistant daemon".
The overlay spec gets a superseded line at the top pointing here. The
local-memory spec and plan are unaffected and still target this repo.
|
