diff options
| -rw-r--r-- | AGENTS.md | 16 | ||||
| -rw-r--r-- | assistant.py | 4 |
2 files changed, 20 insertions, 0 deletions
@@ -343,6 +343,22 @@ distance, a person, next appointment twice). reads Italian digits right (13:37, 26 settembre 2026, 6549,67 euro). English decimals come out split ("six thousand ... forty nine. sixty seven"): the phonemizer takes the point as a full stop. +- **Misheard words send it to the wrong tool.** Whisper wrote "Che ori + sono?" (twice in one day); once the model answered with Rome's weather, + "Rome,IT" being the example in the weather tool's place description. + 10 runs of "Che ori sono?": temperature 1.0 4 stray tool calls, 0.6 + 1, 0.6 plus "the user's words come from speech recognition and may + contain misheard words: read them by how they sound" in the system + prompt 0 (2 of 10 gave the date only). Same hint: finance and weather + questions still 10/10 on the right tool. "Che ore sono?" heard right: + 20/20 at either temperature. +- **Beam search in whisper** (2026-09-26). 10 clips of the user's short + questions (it/en, 3 s, `/data/voice-models/samples/stt/`, personal voice + data, never in the repo), `language=auto`: greedy 8/10 (0.80 s), beam 5 + 9/10 (0.86 s), initial prompt of typical questions 9/10 but "Ho nuove + mail?" came back as "." (lost question), beam + prompt 9/10. **Beam 5 + chosen**, prompt rejected. "ori" did not reproduce in these clips, so + the misheard-word hint above stays. - **It gets weekdays wrong from dates** ("mercoledi 29" for a Tuesday). `agenda.py` prints the weekday itself; right since. - It searches for general knowledge too (a question about a well-known diff --git a/assistant.py b/assistant.py index bf6049a..991dfa0 100644 --- a/assistant.py +++ b/assistant.py @@ -52,6 +52,8 @@ DEFAULTS = { "briefly in plain sentences: no markdown, no lists, no emoji. " "Reply in the language the user speaks, Italian or English. " "Write numbers, times and dates in digits, never spelled out. " + "The user's words come from speech recognition and may contain " + "misheard words: read them by how they sound. " "Facts about the user's computer, devices, accounts, mail, calendar " "or money come only from a tool: call one, and if none can tell, " "say you do not know. Never guess them."), @@ -261,7 +263,9 @@ def envelope(audio, sr): def transcribe(lang="auto"): # Always send language: whisper-server's own default is en, not auto. + # Beam search: 9/10 test clips right vs 8/10 greedy, +0.06 s. out = subprocess.run(["curl", "-s", "-F", f"file=@{WAV}", "-F", f"language={lang}", + "-F", "beam_size=5", "-F", "response_format=verbose_json", WHISPER + "/inference"], capture_output=True, text=True).stdout d = json.loads(out) |