diff options
| author | Danilo M. <danix@danix.xyz> | 2026-09-26 09:56:08 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-09-26 09:56:08 +0200 |
| commit | 746e8c05512675b1c059d73f5f54d977d5e8e8e3 (patch) | |
| tree | 119b93af42a688a46120dab6b1eb1bb65541fe68 | |
| parent | 3b6c7bf876526717d940080b84ee58026b57e853 (diff) | |
| download | desktop-assistant-746e8c05512675b1c059d73f5f54d977d5e8e8e3.tar.gz desktop-assistant-746e8c05512675b1c059d73f5f54d977d5e8e8e3.zip | |
Document the overlay and record its findings
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| -rw-r--r-- | AGENTS.md | 30 | ||||
| -rw-r--r-- | README.md | 12 | ||||
| -rw-r--r-- | examples/config.toml | 9 |
3 files changed, 50 insertions, 1 deletions
@@ -393,6 +393,35 @@ phase 2. - Whisper wrote "build system" as two words; the model mapped it to the VM name `buildsystem` without help. +### Overlay (2026-09-26) + +The assistant owns `$XDG_RUNTIME_DIR/desktop-assistant.sock` (0600) and +broadcasts JSON lines: `hello`, `state`, `ctx`, `level`; the quickshell +component `assistant/` draws them. Spec and plan under `docs/superpowers/`. + +- Live event sequence for a search question: listening, transcribing, + thinking, ctx, tool (search), thinking, ctx, speaking, idle. Barge-in + shows as listening mid-reply; the overlay stays up. +- **arecord writes its file in periods**, ~125 ms by default: the user's + levels came at 8 per second. `--period-time=33333` gives ~31 (measured + as file-size changes over 2 s: 16 vs 63). +- Reply levels are emitted on a playback clock (lead silence + samples + written / rate), not at synthesis; measured 30 per second of speech. +- `speaking` comes from the reply thread, not the speaker: otherwise the + tool announcement replaced the `tool` state while the tool still ran. +- `ctx_max` from `GET /props?model=<id>` (`n_ctx`), read at startup too, + else the first `hello` said 0. +- The overlay's ripple first ran on a 16 ms `Timer` and stuttered (by + eye); `FrameAnimation` (vsync, Qt 6.11) plus a GPU canvas fixed it. + Bars were replaced by three sine ripples at the user's request, 30% + larger (`zoom`), and pulsing dots while transcribing or thinking. +- Ignored presses: `notify-send -u normal -p`, then `-r <id>` so repeated + presses make one entry; DND holds it in the drawer (confirmed). +- Started from Hyprland autostart (hypr-theme repo), not an rc script: + audio, the session bus and the runtime dir only exist in the login + session. Log in `$XDG_RUNTIME_DIR/desktop-assistant.log` (transcripts, + so private and gone at reboot). + ### Context size (2026-09-26) Gemma's context is 32768 tokens. Measured against the running model: @@ -487,7 +516,6 @@ Steps 1-5 of the original plan are done (models, backend comparison, live mic, TTS, glue); push-to-talk came for free from the mic button. Open, in no particular order: -- **Run as a service/autostart** (it currently runs by hand). - **Latency:** STT ~1.2 s is the biggest share; the double encoder pass (detect + transcribe) is the target if it matters. - **Phase 2 tools:** tools that write or modify (events, messages, @@ -91,6 +91,18 @@ The example tools only read: - `vms`: libvirt VMs and their state, or one VM's details, through `virsh` read commands +## Overlay + +The assistant broadcasts its state, context use, the running tool and the +audio levels as JSON lines on `$XDG_RUNTIME_DIR/desktop-assistant.sock` +(mode 0600). The overlay is the `assistant/` component of the quickshell +repo: a circle with the name, `used / max` context and the tool, pulsing +dots while it transcribes or thinks, and a ring of ripples following your +voice and the reply. It fades out after the reply. Name, position, delay +and colours are the `[overlay]` table in `config.toml`. With no overlay +running the assistant works the same. A press ignored in presentation or +gaming mode sends a normal-urgency notification instead. + ## License Copyright (C) 2026 Danilo M. <danix@danix.xyz> diff --git a/examples/config.toml b/examples/config.toml index 5e148b8..b0d2b49 100644 --- a/examples/config.toml +++ b/examples/config.toml @@ -25,6 +25,15 @@ # lang = "en-us" # blend = { af_heart = 1.0 } +# The quickshell overlay (quickshell repo, assistant/). udt palette names +# for the colours. +# [overlay] +# name = "Assistant" +# position = "center" # center, top, bottom, top-left, top-right, bottom-left, bottom-right +# hide_delay = 3.0 # seconds idle before it fades out +# user_color = "accent" +# assistant_color = "green" + # Added to every tool's environment. Personal values live here, not in # the tool files. [env] |
