aboutsummaryrefslogtreecommitdiffstats
path: root/docs/superpowers/specs/2026-10-19-assistant-mic-design.md
diff options
context:
space:
mode:
authorDanilo M. <danix@danix.xyz>2026-10-19 11:04:07 +0200
committerDanilo M. <danix@danix.xyz>2026-10-19 11:04:07 +0200
commit5ce0cdf77d19e74d85de8674e88b25db213e8534 (patch)
treec463cffea4a2de89cbfacb41a86b4a5d152b9d0c /docs/superpowers/specs/2026-10-19-assistant-mic-design.md
parentbb595d78ee63198f3e8d7fede62f88c54f4a77b5 (diff)
downloadquickshell-5ce0cdf77d19e74d85de8674e88b25db213e8534.tar.gz
quickshell-5ce0cdf77d19e74d85de8674e88b25db213e8534.zip
docs: spec the assistant canvas mic
A mic button records with pw-record and sends the WAV's path as an audio ask; the daemon transcribes it. pw-record writes a right header when stopped by SIGTERM, but one stray kept recording in a trial, so every path out stops it and the checks look for a stray each time. Recording stops at 59 s: the daemon refuses over 60 s by file size. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'docs/superpowers/specs/2026-10-19-assistant-mic-design.md')
-rw-r--r--docs/superpowers/specs/2026-10-19-assistant-mic-design.md90
1 files changed, 90 insertions, 0 deletions
diff --git a/docs/superpowers/specs/2026-10-19-assistant-mic-design.md b/docs/superpowers/specs/2026-10-19-assistant-mic-design.md
new file mode 100644
index 0000000..444c323
--- /dev/null
+++ b/docs/superpowers/specs/2026-10-19-assistant-mic-design.md
@@ -0,0 +1,90 @@
+# Assistant mic: design
+
+Date: 2026-10-19. Status: proposed.
+
+## Goal
+
+Ask the assistant by voice from the canvas: click the mic, speak, click
+again, and the question is asked. The daemon does the hearing: an `ask`
+with `audio` has the loaded model transcribe the clip, then answers the
+words as a typed question. Its side is specified in desktop-assistant,
+`docs/superpowers/specs/2026-10-19-voice-input-design.md`; the canvas
+only records and sends the path.
+
+## Verified before designing
+
+- `pw-record --rate 16000 --channels 1 --format s16 <file>.wav`, stopped
+ after 2 s with SIGINT or SIGTERM, leaves a WAV whose header is right:
+ 16000 Hz, mono, 16-bit, ~32000 frames. It records from PipeWire's
+ default source, the mic the rest of the desktop uses.
+- A `pw-record` whose stop missed kept recording until killed by pid.
+ The canvas must stop it on every path out, and the check below looks
+ for a stray one each time.
+
+## Button
+
+A `Button` in the footer, left of the input box (after the paperclip
+when that shows), in every card state, entry included: SUPER+Home opens
+the card and the mic is one click away. Glyph nf-fa-microphone from
+`Theme.iconFamily`, checked in Inconsolata Nerd Font.
+
+- Enabled while connected, not busy, no Do-row operation running, and no
+ staged file that chat does not take (`blocked`), as Send is.
+- First click: start recording. The button turns `danger` (red) and its
+ text is the elapsed time, `0:07`, ticking once a second. The input's
+ placeholder reads "Listening… click the mic to send".
+- Second click, or 59 s reached: stop and send. The daemon refuses over
+ 60 s by file size, and the stop lands a little after the tick.
+
+No keyboard shortcut and no level meter: just the button for now.
+
+## Recording
+
+A `Process` running `pw-record` into
+`$XDG_RUNTIME_DIR/assistant-mic.wav`, one path overwritten by each
+recording. The canvas never deletes it; the runtime dir goes at logout.
+
+- **Send:** SIGTERM, and on the process's exit
+ `{"type": "ask", "text": "", "audio": <path>, "files": <staged paths>}`.
+ The send waits for the exit so the header is written. It goes through
+ `ask()`'s path: `chatting` on, `notice` cleared, staged chips kept until
+ the `user` event.
+- **Cancel:** Escape, or the card closing, sends SIGTERM and sends
+ nothing. Escape while recording only cancels; the next one closes the
+ card as before (`back()` gains the recording as its first step).
+- A `pw-record` that exits on its own (no mic, PipeWire down) ends the
+ recording with a `notice`: "recording failed".
+
+## While the daemon works
+
+The daemon's `state` becomes `transcribing` and then `thinking`. `busy`
+is already `state !== "idle"`, so Send turns into Stop and Stop sends
+`stop`, which ends the turn in the daemon. The placeholder reads
+"Transcribing…" while the state is `transcribing`. There is no turn row
+yet: the daemon's `user` event, carrying the transcript, adds it as for
+a typed question.
+
+The daemon's refusals ("<model> cannot hear audio", "heard nothing",
+"recording too long", a file that is not a WAV) come back as `error`
+events, which the canvas already shows.
+
+## Not included
+
+A shortcut or a bind that opens the card recording, a level meter,
+hiding the mic when the loaded model has no audio input (the daemon's
+error says so after the recording), and editing the transcript before
+it is asked.
+
+## Testing
+
+By hand, `pgrep -a pw-record` empty after each:
+
+- record a question, send it: the transcript appears as the question and
+ the answer streams;
+- record, then Escape: nothing sent, the card stays open; a second
+ Escape closes it;
+- record, then close the card with SUPER+Home: nothing sent;
+- record past 59 s: sent at 59 s, and not refused as too long;
+- `./reload.sh desktop` while recording: no stray `pw-record`;
+- Stop while transcribing: the turn ends, idle;
+- with a file staged: the question goes with the file.