From 746e8c05512675b1c059d73f5f54d977d5e8e8e3 Mon Sep 17 00:00:00 2001 From: "Danilo M." Date: Sat, 26 Sep 2026 09:56:08 +0200 Subject: Document the overlay and record its findings Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) (limited to 'AGENTS.md') diff --git a/AGENTS.md b/AGENTS.md index 77313ac..23fa55f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -393,6 +393,35 @@ phase 2. - Whisper wrote "build system" as two words; the model mapped it to the VM name `buildsystem` without help. +### Overlay (2026-09-26) + +The assistant owns `$XDG_RUNTIME_DIR/desktop-assistant.sock` (0600) and +broadcasts JSON lines: `hello`, `state`, `ctx`, `level`; the quickshell +component `assistant/` draws them. Spec and plan under `docs/superpowers/`. + +- Live event sequence for a search question: listening, transcribing, + thinking, ctx, tool (search), thinking, ctx, speaking, idle. Barge-in + shows as listening mid-reply; the overlay stays up. +- **arecord writes its file in periods**, ~125 ms by default: the user's + levels came at 8 per second. `--period-time=33333` gives ~31 (measured + as file-size changes over 2 s: 16 vs 63). +- Reply levels are emitted on a playback clock (lead silence + samples + written / rate), not at synthesis; measured 30 per second of speech. +- `speaking` comes from the reply thread, not the speaker: otherwise the + tool announcement replaced the `tool` state while the tool still ran. +- `ctx_max` from `GET /props?model=` (`n_ctx`), read at startup too, + else the first `hello` said 0. +- The overlay's ripple first ran on a 16 ms `Timer` and stuttered (by + eye); `FrameAnimation` (vsync, Qt 6.11) plus a GPU canvas fixed it. + Bars were replaced by three sine ripples at the user's request, 30% + larger (`zoom`), and pulsing dots while transcribing or thinking. +- Ignored presses: `notify-send -u normal -p`, then `-r ` so repeated + presses make one entry; DND holds it in the drawer (confirmed). +- Started from Hyprland autostart (hypr-theme repo), not an rc script: + audio, the session bus and the runtime dir only exist in the login + session. Log in `$XDG_RUNTIME_DIR/desktop-assistant.log` (transcripts, + so private and gone at reboot). + ### Context size (2026-09-26) Gemma's context is 32768 tokens. Measured against the running model: @@ -487,7 +516,6 @@ Steps 1-5 of the original plan are done (models, backend comparison, live mic, TTS, glue); push-to-talk came for free from the mic button. Open, in no particular order: -- **Run as a service/autostart** (it currently runs by hand). - **Latency:** STT ~1.2 s is the biggest share; the double encoder pass (detect + transcribe) is the target if it matters. - **Phase 2 tools:** tools that write or modify (events, messages, -- cgit v1.2.3