path: root/docs/superpowers/plans/2026-10-19-assistant-mic.md

# Assistant Mic Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** A mic button on the assistant canvas: click to record with `pw-record`, click again to ask the daemon with the recording.

**Architecture:** `Assistant.qml` owns the recording: one `Process` per recording, created from a `Component` like the Do row's `opRun`, so a stopped recording's late exit cannot be mistaken for the next one. Stopping sets `running = false` (SIGTERM; pw-record then writes the WAV header) and the ask goes out from the process's `exited`. `Card.qml` only draws the button and the placeholder lines. Spec: `docs/superpowers/specs/2026-10-19-assistant-mic-design.md`; daemon side: desktop-assistant `docs/superpowers/specs/2026-10-19-voice-input-design.md` (implemented, running).

**Tech Stack:** QML (Quickshell 0.3.1, `Quickshell.Io.Process`), PipeWire `pw-record`.

QML here has no unit tests; the check is the scratch-harness load check the canvas and file-drop plans used, then the user's hand checks.

---

### Task 1: Recording in `Assistant.qml`

**Files:**
- Modify: `desktop/assistant/Assistant.qml` (`onConnectedChanged` ~line 79, `close()`/`back()` ~line 110, new block after the `opRun` Component ~line 199)

- [ ] **Step 1: Stop a recording when the daemon goes away.** In `onConnectedChanged`, after `root.runningTool = "";`, add:

```qml
        root.endRecording(false);
```

- [ ] **Step 2: `close()` and `back()`.** Replace:

```qml
    function close() { root.open = false; root.historyOpen = false; root.pickerOpen = false; }
    // Escape: the picker, then the history panel, then the card.
    function back() {
        if (root.pickerOpen) root.pickerOpen = false;
```

with:

```qml
    function close() {
        root.endRecording(false);
        root.open = false; root.historyOpen = false; root.pickerOpen = false;
    }
    // Escape: a recording, the picker, then the history panel, then the card.
    function back() {
        if (root.recording) root.endRecording(false);
        else if (root.pickerOpen) root.pickerOpen = false;
```

- [ ] **Step 3: The recording.** After the `opRun` `Component { ... }` block, add:

```qml
    // The mic: pw-record into one file, overwritten by each recording and
    // never deleted (the runtime dir goes at logout). The daemon has the
    // model transcribe it. Spec: docs/superpowers/specs/2026-10-19-assistant-mic-design.md.
    readonly property string micPath: `${Quickshell.env("XDG_RUNTIME_DIR") || "/tmp"}/assistant-mic.wav`
    property var mic: null                // the recording's Process, null when not recording
    property int micSecs: 0
    readonly property bool recording: root.mic !== null

    function record() {
        if (root.recording || !root.connected || root.busy || root.opRunning || root.blocked) return;
        root.notice = "";
        root.micSecs = 0;
        root.mic = micRun.createObject(root);
    }

    // SIGTERM: pw-record finishes the header, then exited sends the ask.
    // ponytail: a recording started before the last one's exit shares the
    // file; pw-record exits within milliseconds of the signal.
    function endRecording(send) {
        if (!root.mic) return;
        root.mic.send = send;
        root.mic.running = false;
        root.mic = null;
    }

    // 59, not 60: the daemon refuses over 60 s by file size, and the stop
    // lands a little after the tick.
    Timer {
        running: root.recording
        interval: 1000
        repeat: true
        onTriggered: if (++root.micSecs >= 59) root.endRecording(true);
    }

    Component {
        id: micRun
        Process {
            id: rec
            property bool send: false
            running: true
            command: ["pw-record", "--rate", "16000", "--channels", "1", "--format", "s16", root.micPath]
            onExited: {
                if (root.mic === rec) {           // ended on its own: no mic, PipeWire down
                    root.mic = null;
                    root.notice = "recording failed";
                } else if (rec.send && root.connected) {
                    root.chatting = true;
                    root.send({ type: "ask", text: "", audio: root.micPath,
                                files: root.staged.map(f => f.path) });
                }
                rec.destroy();
            }
        }
    }
```

- [ ] **Step 4: Syntax check.** Run: `qmllint desktop/assistant/Assistant.qml 2>&1 | grep -i error || echo clean` (if `qmllint` is missing, the Task 3 load check catches errors). Expected: `clean`, or only the import warnings the file already had (compare with `git stash; qmllint ...; git stash pop`).

---

### Task 2: The button in `Card.qml`

**Files:**
- Modify: `desktop/assistant/Card.qml` (footer, ~lines 419-495)

- [ ] **Step 1: The mic button.** After the paperclip `Button { id: attach ... }`, add:

```qml
        // nf-fa-microphone, checked in Inconsolata Nerd Font. Every state,
        // entry included: SUPER+Home opens the card and the mic is one click away.
        Button {
            id: mic
            anchors { bottom: parent.bottom; left: attach.right; leftMargin: attach.visible ? 8 : 0 }
            height: box.height
            icon: ""
            text: card.canvas.recording
                ? `${Math.floor(card.canvas.micSecs / 60)}:${String(card.canvas.micSecs % 60).padStart(2, "0")}`
                : ""
            danger: card.canvas.recording
            enabled: card.canvas.connected && (card.canvas.recording
                || (!card.canvas.busy && !card.canvas.opRunning && !card.canvas.blocked))
            onClicked: {
                if (card.canvas.recording) card.canvas.endRecording(true);
                else card.canvas.record();
                input.forceActiveFocus();
            }
        }
```

- [ ] **Step 2: The input box sits after the mic.** In `Rectangle { id: box`, replace:

```qml
                bottom: parent.bottom; left: attach.right; leftMargin: attach.visible ? 8 : 0
```

with:

```qml
                bottom: parent.bottom; left: mic.right; leftMargin: 8
```

- [ ] **Step 3: Placeholder lines.** In the placeholder `Text`, replace:

```qml
                text: !card.canvas.connected ? "assistant not running"
                    : card.mode === "entry" ? "How can I help you?"
```

with:

```qml
                text: !card.canvas.connected ? "assistant not running"
                    : card.canvas.recording ? "Listening… click the mic to send"
                    : card.canvas.state === "transcribing" ? "Transcribing…"
                    : card.mode === "entry" ? "How can I help you?"
```

- [ ] **Step 4: Docs.** In `desktop/assistant/README.md`, replace:

```
Keys: Enter sends, Shift+Enter is a newline, Escape closes the picker, then
the history panel, then the card. Stop replaces Send while a reply streams.
```

with:

```
The mic records with `pw-record` into `$XDG_RUNTIME_DIR/assistant-mic.wav`
until clicked again (at most 59 s), then asks with it: the daemon has the
model transcribe it and answers the words.

Keys: Enter sends, Shift+Enter is a newline, Escape cancels a recording,
then closes the picker, the history panel, then the card. Stop replaces
Send while a reply streams.
```

---

### Task 3: Load check against a scratch daemon

The mic records the room for about 3 s; the clip goes only to the local model. A desktop shell restart is the user's (`./reload.sh desktop`), so this runs a separate `qs` instance.

- [ ] **Step 1: Scratch daemon.** Run in the background (harness-owned):

```bash
mkdir -p /tmp/claude-1000/dar
XDG_RUNTIME_DIR=/tmp/claude-1000/dar XDG_DATA_HOME=/tmp/claude-1000/dar/data \
  python3 <desktop-assistant>/assistant.py
```

Expected log line: `ready on /tmp/claude-1000/dar/desktop-assistant.sock`.

- [ ] **Step 2: Harness.** Mirror `desktop/` with links, then write `shell.qml`:

```bash
R=<this repo>; H=<scratchpad>/harness-mic
mkdir -p $H
ln -sf $R/shared/Theme.qml $H/Theme.qml
ln -sf $R/shared/Glow.qml $H/Glow.qml
ln -sf $R/shared/Notify.qml $H/Notify.qml
ln -sf $R/shared/NotifyConfig.js $H/NotifyConfig.js
ln -sf $R/desktop/Button.qml $H/Button.qml
ln -sfn $R/desktop/assistant $H/assistant
```

`$H/shell.qml`:

```qml
import Quickshell
import Quickshell.Wayland
import QtQuick
import "assistant"

ShellRoot {
    Assistant { id: a; socketPath: "/tmp/claude-1000/dar/desktop-assistant.sock" }

    PanelWindow {
        anchors { top: true; left: true }
        implicitWidth: 1
        implicitHeight: 1
        color: "transparent"
        mask: Region {}
        exclusionMode: ExclusionMode.Ignore
        WlrLayershell.keyboardFocus: WlrKeyboardFocus.None
        Card { id: c; width: 760; fullHeight: 700; canvas: a; turnModel: a.turnsModel }
    }

    property int step: 0
    property var states: []
    Connections { target: a; function onStateChanged() { states.push(a.state); } }
    Timer {
        interval: 500; repeat: true; running: true
        onTriggered: {
            if (!a.connected) return;
            step++;
            if (step === 1) { a.historyOpen = true; a.record(); console.log("PROTO cancel-start recording=" + a.recording); }
            else if (step === 3) { a.back(); console.log("PROTO cancelled recording=" + a.recording + " history=" + a.historyOpen); a.historyOpen = false; }
            else if (step === 5) { a.record(); console.log("PROTO send-start recording=" + a.recording); }
            else if (step === 11) { a.endRecording(true); console.log("PROTO sent secs=" + a.micSecs); }
            else if (step > 11 && !a.busy && (a.turnsModel.count || a.notice !== "")) {
                console.log("PROTO done states=" + JSON.stringify(states) + " turns=" + a.turnsModel.count
                    + " notice=" + JSON.stringify(a.notice)
                    + " error=" + JSON.stringify(a.turnsModel.count ? a.turnsModel.get(0).error : ""));
                step = -1000;
            }
        }
    }
}
```

Run: `qs -p <scratchpad>/harness-mic` in the background, read its output for `PROTO` lines.

Expected:
- `cancel-start recording=true`, `cancelled recording=false history=true` (Escape only cancelled; `open` stays false so the real card never shows and grabs the keyboard)
- `send-start recording=true`, `sent secs=3`
- `done states=[..."transcribing"...]`, with either one turn (the room's words, or a reply) or the daemon's `heard nothing` as the turn's error or notice
- no QML warnings or errors from `assistant/`.

- [ ] **Step 3: No stray recorder.** `pgrep -a pw-record` prints nothing. Then stop both (`qs kill -p <scratchpad>/harness-mic`, the daemon by PID) and `pgrep -a pw-record` again.

- [ ] **Step 4: Commit** (after asking the user):

```bash
git add desktop/assistant/Assistant.qml desktop/assistant/Card.qml desktop/assistant/README.md
git commit -m "assistant: a mic button asks by voice"
```

Body: pw-record per recording in its own Process, stop by SIGTERM then ask on exit (the header), 59 s cap and why, Escape and closing cancel, and the scratch load check's result.

---

### Task 4: The user's hand checks

`./reload.sh desktop` from the user's terminal (new properties in a loaded file). Then the spec's list, `pgrep -a pw-record` empty after each: ask by voice, Escape twice, close mid-recording, past 59 s, reload mid-recording, Stop while transcribing, with a file staged. Afterwards set the spec's status to implemented with the date, and commit with the hand-check results.