From 5ce0cdf77d19e74d85de8674e88b25db213e8534 Mon Sep 17 00:00:00 2001 From: "Danilo M." Date: Mon, 19 Oct 2026 11:04:07 +0200 Subject: docs: spec the assistant canvas mic A mic button records with pw-record and sends the WAV's path as an audio ask; the daemon transcribes it. pw-record writes a right header when stopped by SIGTERM, but one stray kept recording in a trial, so every path out stops it and the checks look for a stray each time. Recording stops at 59 s: the daemon refuses over 60 s by file size. Co-Authored-By: Claude Opus 5.5 --- .../specs/2026-10-19-assistant-mic-design.md | 90 ++++++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 docs/superpowers/specs/2026-10-19-assistant-mic-design.md (limited to 'docs/superpowers') diff --git a/docs/superpowers/specs/2026-10-19-assistant-mic-design.md b/docs/superpowers/specs/2026-10-19-assistant-mic-design.md new file mode 100644 index 0000000..444c323 --- /dev/null +++ b/docs/superpowers/specs/2026-10-19-assistant-mic-design.md @@ -0,0 +1,90 @@ +# Assistant mic: design + +Date: 2026-10-19. Status: proposed. + +## Goal + +Ask the assistant by voice from the canvas: click the mic, speak, click +again, and the question is asked. The daemon does the hearing: an `ask` +with `audio` has the loaded model transcribe the clip, then answers the +words as a typed question. Its side is specified in desktop-assistant, +`docs/superpowers/specs/2026-10-19-voice-input-design.md`; the canvas +only records and sends the path. + +## Verified before designing + +- `pw-record --rate 16000 --channels 1 --format s16 .wav`, stopped + after 2 s with SIGINT or SIGTERM, leaves a WAV whose header is right: + 16000 Hz, mono, 16-bit, ~32000 frames. It records from PipeWire's + default source, the mic the rest of the desktop uses. +- A `pw-record` whose stop missed kept recording until killed by pid. + The canvas must stop it on every path out, and the check below looks + for a stray one each time. + +## Button + +A `Button` in the footer, left of the input box (after the paperclip +when that shows), in every card state, entry included: SUPER+Home opens +the card and the mic is one click away. Glyph nf-fa-microphone from +`Theme.iconFamily`, checked in Inconsolata Nerd Font. + +- Enabled while connected, not busy, no Do-row operation running, and no + staged file that chat does not take (`blocked`), as Send is. +- First click: start recording. The button turns `danger` (red) and its + text is the elapsed time, `0:07`, ticking once a second. The input's + placeholder reads "Listening… click the mic to send". +- Second click, or 59 s reached: stop and send. The daemon refuses over + 60 s by file size, and the stop lands a little after the tick. + +No keyboard shortcut and no level meter: just the button for now. + +## Recording + +A `Process` running `pw-record` into +`$XDG_RUNTIME_DIR/assistant-mic.wav`, one path overwritten by each +recording. The canvas never deletes it; the runtime dir goes at logout. + +- **Send:** SIGTERM, and on the process's exit + `{"type": "ask", "text": "", "audio": , "files": }`. + The send waits for the exit so the header is written. It goes through + `ask()`'s path: `chatting` on, `notice` cleared, staged chips kept until + the `user` event. +- **Cancel:** Escape, or the card closing, sends SIGTERM and sends + nothing. Escape while recording only cancels; the next one closes the + card as before (`back()` gains the recording as its first step). +- A `pw-record` that exits on its own (no mic, PipeWire down) ends the + recording with a `notice`: "recording failed". + +## While the daemon works + +The daemon's `state` becomes `transcribing` and then `thinking`. `busy` +is already `state !== "idle"`, so Send turns into Stop and Stop sends +`stop`, which ends the turn in the daemon. The placeholder reads +"Transcribing…" while the state is `transcribing`. There is no turn row +yet: the daemon's `user` event, carrying the transcript, adds it as for +a typed question. + +The daemon's refusals (" cannot hear audio", "heard nothing", +"recording too long", a file that is not a WAV) come back as `error` +events, which the canvas already shows. + +## Not included + +A shortcut or a bind that opens the card recording, a level meter, +hiding the mic when the loaded model has no audio input (the daemon's +error says so after the recording), and editing the transcript before +it is asked. + +## Testing + +By hand, `pgrep -a pw-record` empty after each: + +- record a question, send it: the transcript appears as the question and + the answer streams; +- record, then Escape: nothing sent, the card stays open; a second + Escape closes it; +- record, then close the card with SUPER+Home: nothing sent; +- record past 59 s: sent at 59 s, and not refused as too long; +- `./reload.sh desktop` while recording: no stray `pw-record`; +- Stop while transcribing: the turn ends, idle; +- with a file staged: the question goes with the file. -- cgit v1.2.3