1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
|
# Assistant mic: design
Date: 2026-10-19. Status: proposed.
## Goal
Ask the assistant by voice from the canvas: click the mic, speak, click
again, and the question is asked. The daemon does the hearing: an `ask`
with `audio` has the loaded model transcribe the clip, then answers the
words as a typed question. Its side is specified in desktop-assistant,
`docs/superpowers/specs/2026-10-19-voice-input-design.md`; the canvas
only records and sends the path.
## Verified before designing
- `pw-record --rate 16000 --channels 1 --format s16 <file>.wav`, stopped
after 2 s with SIGINT or SIGTERM, leaves a WAV whose header is right:
16000 Hz, mono, 16-bit, ~32000 frames. It records from PipeWire's
default source, the mic the rest of the desktop uses.
- A `pw-record` whose stop missed kept recording until killed by pid.
The canvas must stop it on every path out, and the check below looks
for a stray one each time.
## Button
A `Button` in the footer, left of the input box (after the paperclip
when that shows), in every card state, entry included: SUPER+Home opens
the card and the mic is one click away. Glyph nf-fa-microphone from
`Theme.iconFamily`, checked in Inconsolata Nerd Font.
- Enabled while connected, not busy, no Do-row operation running, and no
staged file that chat does not take (`blocked`), as Send is.
- First click: start recording. The button turns `danger` (red) and its
text is the elapsed time, `0:07`, ticking once a second. The input's
placeholder reads "Listening… click the mic to send".
- Second click, or 59 s reached: stop and send. The daemon refuses over
60 s by file size, and the stop lands a little after the tick.
No keyboard shortcut and no level meter: just the button for now.
## Recording
A `Process` running `pw-record` into
`$XDG_RUNTIME_DIR/assistant-mic.wav`, one path overwritten by each
recording. The canvas never deletes it; the runtime dir goes at logout.
- **Send:** SIGTERM, and on the process's exit
`{"type": "ask", "text": "", "audio": <path>, "files": <staged paths>}`.
The send waits for the exit so the header is written. It goes through
`ask()`'s path: `chatting` on, `notice` cleared, staged chips kept until
the `user` event.
- **Cancel:** Escape, or the card closing, sends SIGTERM and sends
nothing. Escape while recording only cancels; the next one closes the
card as before (`back()` gains the recording as its first step).
- A `pw-record` that exits on its own (no mic, PipeWire down) ends the
recording with a `notice`: "recording failed".
## While the daemon works
The daemon's `state` becomes `transcribing` and then `thinking`. `busy`
is already `state !== "idle"`, so Send turns into Stop and Stop sends
`stop`, which ends the turn in the daemon. The placeholder reads
"Transcribing…" while the state is `transcribing`. There is no turn row
yet: the daemon's `user` event, carrying the transcript, adds it as for
a typed question.
The daemon's refusals ("<model> cannot hear audio", "heard nothing",
"recording too long", a file that is not a WAV) come back as `error`
events, which the canvas already shows.
## Not included
A shortcut or a bind that opens the card recording, a level meter,
hiding the mic when the loaded model has no audio input (the daemon's
error says so after the recording), and editing the transcript before
it is asked.
## Testing
By hand, `pgrep -a pw-record` empty after each:
- record a question, send it: the transcript appears as the question and
the answer streams;
- record, then Escape: nothing sent, the card stays open; a second
Escape closes it;
- record, then close the card with SUPER+Home: nothing sent;
- record past 59 s: sent at 59 s, and not refused as too long;
- `./reload.sh desktop` while recording: no stray `pw-record`;
- Stop while transcribing: the turn ends, idle;
- with a file staged: the question goes with the file.
|
