aboutsummaryrefslogtreecommitdiffstats
path: root/docs
diff options
context:
space:
mode:
authorDanilo M. <danix@danix.xyz>2026-10-19 10:52:42 +0200
committerDanilo M. <danix@danix.xyz>2026-10-19 10:52:42 +0200
commit1b3dca845f2b2866cc967e4f5fb5f21a6d866a4e (patch)
tree41686331f5d24725b2af0e8ec0049b117c0bd0ae /docs
parentea603369bd342ed67b802fb04febfde9632e91e2 (diff)
downloaddesktop-assistant-1b3dca845f2b2866cc967e4f5fb5f21a6d866a4e.tar.gz
desktop-assistant-1b3dca845f2b2866cc967e4f5fb5f21a6d866a4e.zip
voice input: ask takes a WAV, the model transcribes it
An ask with audio sends a transcribing state, has the loaded model transcribe the clip (thinking off, no tools, the prompt naming Italian or English), then answers the transcript as a typed question, so the user event, history and search hold the words. A stop while transcribing ends the turn like any reply. Refused: a relative path, a file that is not a WAV, over 60 s (the ubatch bound, untried for audio), a model without audio input, and an empty transcript. The length comes from the file size, since a recording stopped by a signal leaves the header's length unset. Live: 10/10 clips heard right, the right tool each time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'docs')
-rw-r--r--docs/superpowers/specs/2026-10-16-text-daemon-design.md2
-rw-r--r--docs/superpowers/specs/2026-10-19-voice-input-design.md19
2 files changed, 13 insertions, 8 deletions
diff --git a/docs/superpowers/specs/2026-10-16-text-daemon-design.md b/docs/superpowers/specs/2026-10-16-text-daemon-design.md
index 223af38..6ae4dd6 100644
--- a/docs/superpowers/specs/2026-10-16-text-daemon-design.md
+++ b/docs/superpowers/specs/2026-10-16-text-daemon-design.md
@@ -96,7 +96,7 @@ Outbound:
| Event | Fields |
| --- | --- |
| `hello` | `state`, `model`, `ctx_used`, `ctx_max`, `id`, `turns` (the current conversation), `pending` (`{user, reply, think, tools}` of a reply in flight, or null) |
-| `state` | `state`: `idle`, `thinking`, `tool` (with `tool`) |
+| `state` | `state`: `idle`, `transcribing` (an `audio` ask, see the voice input spec), `thinking`, `tool` (with `tool`) |
| `user` | `text`: the question, echoed so every client draws it |
| `think` | `delta` |
| `token` | `delta` |
diff --git a/docs/superpowers/specs/2026-10-19-voice-input-design.md b/docs/superpowers/specs/2026-10-19-voice-input-design.md
index 860967a..551ed60 100644
--- a/docs/superpowers/specs/2026-10-19-voice-input-design.md
+++ b/docs/superpowers/specs/2026-10-19-voice-input-design.md
@@ -1,6 +1,6 @@
# Voice input: design
-Date: 2026-10-19. Status: proposed.
+Date: 2026-10-19. Status: implemented (2026-10-19), live-checked.
## Goal
@@ -40,9 +40,13 @@ it the reply runs as a typed `ask` with the transcript as `text`: the
`user` event, history, search and the date stamp all see the words, not
the audio.
-Errors answer with `error` (with `turn`) and nothing more is sent:
+`audio` that is not a string, or comes with text, answers `bad audio`
+(no `turn`, like `bad files`). Otherwise errors answer with `error`
+(with `turn`) and nothing more is sent:
- not an absolute path, missing, not a readable WAV (stdlib `wave`);
+ the length comes from the file size, since a recording stopped by a
+ signal leaves the header's length unset;
- longer than 60 s: "recording too long". 60 s is about 1500 audio
tokens; past the 2048 ubatch an image crashed the model, and audio
has not been tried there;
@@ -52,17 +56,18 @@ Errors answer with `error` (with `turn`) and nothing more is sent:
## Transcription
-One non-streamed-to-the-client request, thinking off, temperature 0.6,
-no tools, no history, no system prompt:
+One non-streamed-to-the-client request, thinking off, the configured
+temperature (0.6 when tested), no tools, no history, no system prompt:
- a text part, "Transcribe this audio exactly, in the language spoken
(Italian or English). Output only the transcript.";
- an `input_audio` part, `{"data": <base64>, "format": "wav"}`, no
`data:` prefix (llamachat's finding).
-It goes through `ask()` with the reply's turn, so a stop while
-transcribing shuts the request like any other. The transcript is
-stripped and goes to `answer()` unchanged.
+It goes through `ask()` with the reply's turn. The turn is pending from
+the `transcribing` state on, so a stop while transcribing shuts the
+request and ends with `done` (stopped) and `idle`, like any reply. The
+transcript is stripped and goes to `answer()` unchanged.
## Testing