From be9719e61feab28612499680be9efd7411c5254f Mon Sep 17 00:00:00 2001 From: "Danilo M." Date: Sat, 26 Sep 2026 08:56:08 +0200 Subject: Let a button press cut the current reply (barge-in) Every press bumps a turn counter and kills the running aplay; queued audio from an older turn is dropped, and reply() now runs on its own thread and returns once its turn is stale, so the button loop is never blocked. A cut-off exchange is left out of history so user and assistant messages still alternate. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) (limited to 'AGENTS.md') diff --git a/AGENTS.md b/AGENTS.md index bfc29f4..0e58c38 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -362,6 +362,28 @@ file: the model fills parameters but cannot reach `mailctl tag` or confirmation before acting for exactly this reason. - These tools speak personal data (mail subjects, amounts) out loud. +### Barge-in (2026-09-26) + +A button press cuts the current reply. A global `turn` counter goes up on +every press; queued audio items carry their turn and the speaker drops +stale ones; the press kills the running `aplay` at once (that also drops +the ~1 s already in its pipe). `reply()` runs on its own thread and +returns when its turn is stale: after STT, between streamed pieces, +before tool rounds. The button loop is never blocked by a reply any more. + +- Live: stopped instantly while speaking and while a search was running, + nothing of the old reply played afterwards (by ear). A correction + ("anzi, dimmi...") pressed during a search got its own clean answer. +- A reply whose text had already finished generating is cut in audio + only; the log shows no `interrupted` then, and the exchange stays in + history. A reply cut while generating is dropped from history whole, + so user and assistant messages still alternate. +- Known ceilings (`ponytail:` comments): a running tool is not killed, + its result is dropped; a press during the old reply's STT can + overwrite the WAV mid-upload. +- Unrelated but seen: whisper garbled "i Mille" into "i 1000.000e" and + the model invented an answer for it instead of searching. + ### TTS (2026-09-25) `piper-tts` on CPU (onnxruntime), ~5 s sentence per voice: @@ -418,8 +440,6 @@ mic, TTS, glue); push-to-talk came for free from the mic button. Open, in no particular order: - **Run as a service/autostart** (it currently runs by hand). -- **Barge-in:** pressing the button while it speaks does not stop the - reply yet. - **Check presentation mode during games:** `statusctl presentation get` reads the stored flag; confirm it is 1 while a game holds it on. - **Latency:** STT ~1.2 s is the biggest share; the double encoder pass -- cgit v1.2.3