From 1b3dca845f2b2866cc967e4f5fb5f21a6d866a4e Mon Sep 17 00:00:00 2001 From: "Danilo M." Date: Mon, 19 Oct 2026 10:52:42 +0200 Subject: voice input: ask takes a WAV, the model transcribes it An ask with audio sends a transcribing state, has the loaded model transcribe the clip (thinking off, no tools, the prompt naming Italian or English), then answers the transcript as a typed question, so the user event, history and search hold the words. A stop while transcribing ends the turn like any reply. Refused: a relative path, a file that is not a WAV, over 60 s (the ubatch bound, untried for audio), a model without audio input, and an empty transcript. The length comes from the file size, since a recording stopped by a signal leaves the header's length unset. Live: 10/10 clips heard right, the right tool each time. Co-Authored-By: Claude Opus 5.5 --- .../specs/2026-10-16-text-daemon-design.md | 2 +- .../specs/2026-10-19-voice-input-design.md | 19 ++++++++++++------- 2 files changed, 13 insertions(+), 8 deletions(-) (limited to 'docs') diff --git a/docs/superpowers/specs/2026-10-16-text-daemon-design.md b/docs/superpowers/specs/2026-10-16-text-daemon-design.md index 223af38..6ae4dd6 100644 --- a/docs/superpowers/specs/2026-10-16-text-daemon-design.md +++ b/docs/superpowers/specs/2026-10-16-text-daemon-design.md @@ -96,7 +96,7 @@ Outbound: | Event | Fields | | --- | --- | | `hello` | `state`, `model`, `ctx_used`, `ctx_max`, `id`, `turns` (the current conversation), `pending` (`{user, reply, think, tools}` of a reply in flight, or null) | -| `state` | `state`: `idle`, `thinking`, `tool` (with `tool`) | +| `state` | `state`: `idle`, `transcribing` (an `audio` ask, see the voice input spec), `thinking`, `tool` (with `tool`) | | `user` | `text`: the question, echoed so every client draws it | | `think` | `delta` | | `token` | `delta` | diff --git a/docs/superpowers/specs/2026-10-19-voice-input-design.md b/docs/superpowers/specs/2026-10-19-voice-input-design.md index 860967a..551ed60 100644 --- a/docs/superpowers/specs/2026-10-19-voice-input-design.md +++ b/docs/superpowers/specs/2026-10-19-voice-input-design.md @@ -1,6 +1,6 @@ # Voice input: design -Date: 2026-10-19. Status: proposed. +Date: 2026-10-19. Status: implemented (2026-10-19), live-checked. ## Goal @@ -40,9 +40,13 @@ it the reply runs as a typed `ask` with the transcript as `text`: the `user` event, history, search and the date stamp all see the words, not the audio. -Errors answer with `error` (with `turn`) and nothing more is sent: +`audio` that is not a string, or comes with text, answers `bad audio` +(no `turn`, like `bad files`). Otherwise errors answer with `error` +(with `turn`) and nothing more is sent: - not an absolute path, missing, not a readable WAV (stdlib `wave`); + the length comes from the file size, since a recording stopped by a + signal leaves the header's length unset; - longer than 60 s: "recording too long". 60 s is about 1500 audio tokens; past the 2048 ubatch an image crashed the model, and audio has not been tried there; @@ -52,17 +56,18 @@ Errors answer with `error` (with `turn`) and nothing more is sent: ## Transcription -One non-streamed-to-the-client request, thinking off, temperature 0.6, -no tools, no history, no system prompt: +One non-streamed-to-the-client request, thinking off, the configured +temperature (0.6 when tested), no tools, no history, no system prompt: - a text part, "Transcribe this audio exactly, in the language spoken (Italian or English). Output only the transcript."; - an `input_audio` part, `{"data": , "format": "wav"}`, no `data:` prefix (llamachat's finding). -It goes through `ask()` with the reply's turn, so a stop while -transcribing shuts the request like any other. The transcript is -stripped and goes to `answer()` unchanged. +It goes through `ask()` with the reply's turn. The turn is pending from +the `transcribing` state on, so a stop while transcribing shuts the +request and ends with `done` (stopped) and `idle`, like any reply. The +transcript is stripped and goes to `answer()` unchanged. ## Testing -- cgit v1.2.3