aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorDanilo M. <danix@danix.xyz>2026-09-26 09:56:08 +0200
committerDanilo M. <danix@danix.xyz>2026-09-26 09:56:08 +0200
commit746e8c05512675b1c059d73f5f54d977d5e8e8e3 (patch)
tree119b93af42a688a46120dab6b1eb1bb65541fe68
parent3b6c7bf876526717d940080b84ee58026b57e853 (diff)
downloaddesktop-assistant-746e8c05512675b1c059d73f5f54d977d5e8e8e3.tar.gz
desktop-assistant-746e8c05512675b1c059d73f5f54d977d5e8e8e3.zip
Document the overlay and record its findings
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-rw-r--r--AGENTS.md30
-rw-r--r--README.md12
-rw-r--r--examples/config.toml9
3 files changed, 50 insertions, 1 deletions
diff --git a/AGENTS.md b/AGENTS.md
index 77313ac..23fa55f 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -393,6 +393,35 @@ phase 2.
- Whisper wrote "build system" as two words; the model mapped it to the
VM name `buildsystem` without help.
+### Overlay (2026-09-26)
+
+The assistant owns `$XDG_RUNTIME_DIR/desktop-assistant.sock` (0600) and
+broadcasts JSON lines: `hello`, `state`, `ctx`, `level`; the quickshell
+component `assistant/` draws them. Spec and plan under `docs/superpowers/`.
+
+- Live event sequence for a search question: listening, transcribing,
+ thinking, ctx, tool (search), thinking, ctx, speaking, idle. Barge-in
+ shows as listening mid-reply; the overlay stays up.
+- **arecord writes its file in periods**, ~125 ms by default: the user's
+ levels came at 8 per second. `--period-time=33333` gives ~31 (measured
+ as file-size changes over 2 s: 16 vs 63).
+- Reply levels are emitted on a playback clock (lead silence + samples
+ written / rate), not at synthesis; measured 30 per second of speech.
+- `speaking` comes from the reply thread, not the speaker: otherwise the
+ tool announcement replaced the `tool` state while the tool still ran.
+- `ctx_max` from `GET /props?model=<id>` (`n_ctx`), read at startup too,
+ else the first `hello` said 0.
+- The overlay's ripple first ran on a 16 ms `Timer` and stuttered (by
+ eye); `FrameAnimation` (vsync, Qt 6.11) plus a GPU canvas fixed it.
+ Bars were replaced by three sine ripples at the user's request, 30%
+ larger (`zoom`), and pulsing dots while transcribing or thinking.
+- Ignored presses: `notify-send -u normal -p`, then `-r <id>` so repeated
+ presses make one entry; DND holds it in the drawer (confirmed).
+- Started from Hyprland autostart (hypr-theme repo), not an rc script:
+ audio, the session bus and the runtime dir only exist in the login
+ session. Log in `$XDG_RUNTIME_DIR/desktop-assistant.log` (transcripts,
+ so private and gone at reboot).
+
### Context size (2026-09-26)
Gemma's context is 32768 tokens. Measured against the running model:
@@ -487,7 +516,6 @@ Steps 1-5 of the original plan are done (models, backend comparison, live
mic, TTS, glue); push-to-talk came for free from the mic button. Open, in
no particular order:
-- **Run as a service/autostart** (it currently runs by hand).
- **Latency:** STT ~1.2 s is the biggest share; the double encoder pass
(detect + transcribe) is the target if it matters.
- **Phase 2 tools:** tools that write or modify (events, messages,
diff --git a/README.md b/README.md
index de500c1..5a8cc92 100644
--- a/README.md
+++ b/README.md
@@ -91,6 +91,18 @@ The example tools only read:
- `vms`: libvirt VMs and their state, or one VM's details, through `virsh`
read commands
+## Overlay
+
+The assistant broadcasts its state, context use, the running tool and the
+audio levels as JSON lines on `$XDG_RUNTIME_DIR/desktop-assistant.sock`
+(mode 0600). The overlay is the `assistant/` component of the quickshell
+repo: a circle with the name, `used / max` context and the tool, pulsing
+dots while it transcribes or thinks, and a ring of ripples following your
+voice and the reply. It fades out after the reply. Name, position, delay
+and colours are the `[overlay]` table in `config.toml`. With no overlay
+running the assistant works the same. A press ignored in presentation or
+gaming mode sends a normal-urgency notification instead.
+
## License
Copyright (C) 2026 Danilo M. <danix@danix.xyz>
diff --git a/examples/config.toml b/examples/config.toml
index 5e148b8..b0d2b49 100644
--- a/examples/config.toml
+++ b/examples/config.toml
@@ -25,6 +25,15 @@
# lang = "en-us"
# blend = { af_heart = 1.0 }
+# The quickshell overlay (quickshell repo, assistant/). udt palette names
+# for the colours.
+# [overlay]
+# name = "Assistant"
+# position = "center" # center, top, bottom, top-left, top-right, bottom-left, bottom-right
+# hide_delay = 3.0 # seconds idle before it fades out
+# user_color = "accent"
+# assistant_color = "green"
+
# Added to every tool's environment. Personal values live here, not in
# the tool files.
[env]