Add design spec for the assistant overlay

A quickshell overlay fed by a JSON-lines Unix socket the assistant
owns: state, tool, context use and real audio levels. Ignored presses
send a notification instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

1 files changed, 170 insertions, 0 deletions

diff --git a/docs/superpowers/specs/2026-09-26-overlay-design.md b/docs/superpowers/specs/2026-09-26-overlay-design.md
new file mode 100644
index 0000000..e45819f
--- /dev/null
+++ b/docs/superpowers/specs/2026-09-26-overlay-design.md
@@ -0,0 +1,170 @@
+# Assistant overlay: design
+
+Date: 2026-09-26. Status: approved in brainstorming, not implemented.
+
+## Goal
+
+A visual companion for the voice assistant. When the mic button is
+pressed, a transparent overlay appears with a circle: the assistant's
+name and how full the model's context is inside it, the running tool's
+icon and name when one runs, and a ring of sound waves around it that
+follows the real audio, in one colour while the user talks and another
+while the assistant replies. It fades out a configurable delay after the
+reply. A press the assistant ignores (presentation or gaming mode) gets a
+notification saying why, instead of an overlay.
+
+The assistant must keep working exactly as now with no overlay running.
+
+## Verified before designing
+
+- Quickshell 0.3.1 is installed; `Quickshell.Io` has `Socket` with a
+  `SplitParser`, which reads a line-based stream.
+- The quickshell repo (`../quickshell`) holds standalone components, each
+  its own shell started from Hyprland's autostart. `shared/Theme.qml`
+  already reads unified-desktop-theme's `~/.cache/udt/udt-palette.qml`
+  and follows it live; the palette has `accent` and `green`.
+- llama-server reports the context size per model:
+  `GET /props?model=<id>` returns `default_generation_settings.n_ctx`
+  (32768 for the resident Gemma). Tokens used per request already come
+  back in the stream (`usage.prompt_tokens`).
+- Presentation mode turns DND on; the shell's notification daemon then
+  holds back low and normal notifications (listed in its drawer) and pops
+  only critical ones. `notify-send` is installed.
+
+## Architecture
+
+```
+assistant.py ──JSON lines──> $XDG_RUNTIME_DIR/desktop-assistant.sock ──> quickshell assistant/
+   (server)                                                           (client, reconnects)
+```
+
+The assistant owns a Unix socket and broadcasts events to every
+connected client. With no client connected, events are dropped: the
+voice loop never waits on the UI. The overlay connects, retries every
+2 s while the assistant is down, and restarts independently.
+
+Rejected: quickshell as the server (the assistant would depend on the UI
+being up); state files (fine for state, useless for a 30 Hz level
+stream).
+
+## Protocol
+
+One JSON object per line, UTF-8. The socket file is created with mode
+0600. Clients only read; anything they send is ignored.
+
+| `type` | Fields | When |
+|--------|--------|------|
+| `hello` | `name`, `position`, `hide_delay`, `user_color`, `assistant_color`, `ctx_max`, `state` | right after a client connects |
+| `state` | `state`, and `tool` when `state` is `tool` | on every change |
+| `ctx` | `used`, `max` | after every LLM round |
+| `level` | `who` (`user` or `assistant`), `value` (0 to 1) | ~30 per second while audio flows |
+
+States:
+
+| `state` | Entered when |
+|---------|--------------|
+| `listening` | the button goes on (not ignored), including a barge-in |
+| `transcribing` | the button goes off with a usable clip |
+| `thinking` | STT is done; also again after a tool round |
+| `tool` | a tool starts; `tool` names it. One event per tool |
+| `speaking` | the first audio of a reply starts playing |
+| `idle` | the reply's audio has finished, or the turn ended without one (stray press, blank audio, interrupted) |
+
+`speaking` can overlap `thinking` and `tool`: sentences play while later
+ones are still being generated, and the announcement plays while its
+tool runs. The state event reports the reply thread's stage; whether the
+assistant is audible is what `level` events with `who: assistant` show.
+The overlay treats them independently.
+
+`ctx_max` comes from `/props?model=<id>`, read when the model id changes
+(the router can swap models); `0` when unknown.
+
+A client connecting mid-turn gets the current state in `hello`, then the
+stream.
+
+## Assistant side (`assistant.py`)
+
+- **Socket thread.** Stdlib `socket` + `threading`. Accepts clients, keeps
+  a list, `emit(event)` serialises once and sends to each; a client that
+  errors is dropped. The socket file is removed on exit and replaced if
+  stale at startup.
+- **User level.** While recording, a thread reads the RAW file's new bytes
+  every 33 ms and emits the RMS of that window.
+- **Assistant level.** For each synthesised sentence the speaker computes
+  an envelope (RMS per 33 ms frame) and queues the frames with their due
+  time on a playback clock: the stream's start plus the lead silence plus
+  the samples already written, divided by the sample rate. One emitter
+  thread sends each frame at its due time. A barge-in drops the pending
+  frames of the stale turn. `ponytail:` no correction for the sound
+  server's own output latency; add a constant offset if the waves visibly
+  lead the voice.
+- **Level scale.** `value = clamp((20*log10(rms/32768) + 60) / 60, 0, 1)`:
+  -60 dBFS and below is 0, full scale is 1. Perceptual, so quiet speech
+  still moves the ring.
+- **Ignored press.** `presenting()` returns why (`presentation`, `gaming`
+  or nothing). An ignored press sends `notify-send -a desktop-assistant
+  -u normal` with the reason, reusing the previous notification's id
+  (`-p` prints it, `-r` replaces), so repeated presses during one game
+  make one entry. No overlay events for an ignored press.
+- **Config.** New `[overlay]` table in `config.toml`, all optional:
+
+```toml
+[overlay]
+name = "Assistant"              # shown in the circle
+position = "center"             # center, top, bottom, top-left, top-right, bottom-left, bottom-right
+hide_delay = 3.0                # seconds idle before it fades out
+user_color = "accent"           # udt palette names
+assistant_color = "green"
+```
+
+These reach the overlay in `hello`, so the assistant's config is the one
+place to change them.
+
+## Overlay side (`../quickshell/assistant/`)
+
+A new standalone component in the quickshell repo, built under that
+repo's AGENTS.md (keep-alive 1x1 window, `qs` process name, verification
+rules). Started from Hyprland's autostart like the others.
+
+- **Window.** A full-screen, transparent, click-through (`mask: Region
+  {}`) layer-shell window on the overlay layer, on the Hyprland-focused
+  monitor at the moment it appears (first screen as fallback). It never
+  takes keyboard focus.
+- **Visibility.** Appears on `listening`. Fades out (opacity, ~200 ms)
+  once `idle` has lasted `hide_delay` seconds with no assistant level
+  above silence; any other state cancels the fade. Nothing on an ignored
+  press, since the assistant sends nothing then.
+- **Circle.** Fixed size (~160 px), filled with a translucent `base`.
+  Inside: `name`, and below it `used / max` (for example `1115 / 32768`),
+  or only `used` when `max` is 0. In `tool` state, a tool icon and the
+  tool name replace the context line until the state changes.
+- **Waves.** A ring of radial bars around the circle drawn with `Canvas`
+  from a rolling history of the last ~64 level values, so the ring
+  moves like a waveform. Bars from `user` levels use `user_color`, from
+  `assistant` levels `assistant_color`, looked up by name in `Theme`, so
+  a udt theme change recolours them live. Unknown names fall back to
+  `accent`.
+- **Position.** From `hello.position`, with a fixed margin from the
+  screen edges for the edge and corner positions.
+- **Reconnect.** A `Socket` on the path; while disconnected a 2 s `Timer`
+  retries. A lost connection hides the overlay.
+
+## Testing
+
+- `assistant.py --test` gains: the level scale (silence, a full-scale
+  sine, a -30 dBFS sine), the envelope frame count for a known length,
+  `presenting()` returning the reason, and `emit` with a connected test
+  client reading back one line per event (a socketpair, no running
+  assistant needed).
+- Protocol by hand: `socat - UNIX-CONNECT:$XDG_RUNTIME_DIR/desktop-assistant.sock`
+  while asking a question shows the event sequence.
+- Overlay: visual, judged by the user (the quickshell repo's rule: for
+  anything visual, ask). Checks: appears on press, waves follow the voice
+  in `accent` and the reply in `green`, tool icon on a search, fade after
+  `hide_delay`, nothing but a notification during a game, reconnects after
+  an assistant restart.
+
+## Out of scope
+
+A transcript or history view, clickable controls (talk, stop, clear),
+multiple overlays per monitor, sound-server latency calibration.