aboutsummaryrefslogtreecommitdiffstats
path: root/AGENTS.md
diff options
context:
space:
mode:
authorDanilo M. <danix@danix.xyz>2026-09-26 08:56:08 +0200
committerDanilo M. <danix@danix.xyz>2026-09-26 08:56:08 +0200
commitbe9719e61feab28612499680be9efd7411c5254f (patch)
tree61d52badaea0c27aa58d0f595710199f6fdab0f9 /AGENTS.md
parent4a357b335f74ae64714800527ac3e95b7057a3ce (diff)
downloaddesktop-assistant-be9719e61feab28612499680be9efd7411c5254f.tar.gz
desktop-assistant-be9719e61feab28612499680be9efd7411c5254f.zip
Let a button press cut the current reply (barge-in)
Every press bumps a turn counter and kills the running aplay; queued audio from an older turn is dropped, and reply() now runs on its own thread and returns once its turn is stale, so the button loop is never blocked. A cut-off exchange is left out of history so user and assistant messages still alternate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'AGENTS.md')
-rw-r--r--AGENTS.md24
1 files changed, 22 insertions, 2 deletions
diff --git a/AGENTS.md b/AGENTS.md
index bfc29f4..0e58c38 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -362,6 +362,28 @@ file: the model fills parameters but cannot reach `mailctl tag` or
confirmation before acting for exactly this reason.
- These tools speak personal data (mail subjects, amounts) out loud.
+### Barge-in (2026-09-26)
+
+A button press cuts the current reply. A global `turn` counter goes up on
+every press; queued audio items carry their turn and the speaker drops
+stale ones; the press kills the running `aplay` at once (that also drops
+the ~1 s already in its pipe). `reply()` runs on its own thread and
+returns when its turn is stale: after STT, between streamed pieces,
+before tool rounds. The button loop is never blocked by a reply any more.
+
+- Live: stopped instantly while speaking and while a search was running,
+ nothing of the old reply played afterwards (by ear). A correction
+ ("anzi, dimmi...") pressed during a search got its own clean answer.
+- A reply whose text had already finished generating is cut in audio
+ only; the log shows no `interrupted` then, and the exchange stays in
+ history. A reply cut while generating is dropped from history whole,
+ so user and assistant messages still alternate.
+- Known ceilings (`ponytail:` comments): a running tool is not killed,
+ its result is dropped; a press during the old reply's STT can
+ overwrite the WAV mid-upload.
+- Unrelated but seen: whisper garbled "i Mille" into "i 1000.000e" and
+ the model invented an answer for it instead of searching.
+
### TTS (2026-09-25)
`piper-tts` on CPU (onnxruntime), ~5 s sentence per voice:
@@ -418,8 +440,6 @@ mic, TTS, glue); push-to-talk came for free from the mic button. Open, in
no particular order:
- **Run as a service/autostart** (it currently runs by hand).
-- **Barge-in:** pressing the button while it speaks does not stop the
- reply yet.
- **Check presentation mode during games:** `statusctl presentation get`
reads the stored flag; confirm it is 1 while a game holds it on.
- **Latency:** STT ~1.2 s is the biggest share; the double encoder pass