aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorDanilo M. <danix@danix.xyz>2026-09-28 12:18:06 +0200
committerDanilo M. <danix@danix.xyz>2026-09-28 12:18:06 +0200
commitc873e3ca7b51ce159cbfac5f9635d67e542c4c0a (patch)
treec9a2971014b19a536428c4e1fd09340e30746237
parent7a6190becf90364c6f31f401ccda745e4745081c (diff)
downloaddesktop-assistant-c873e3ca7b51ce159cbfac5f9635d67e542c4c0a.tar.gz
desktop-assistant-c873e3ca7b51ce159cbfac5f9635d67e542c4c0a.zip
Record the openWakeWord hey_jarvis trial
Logging-only trial, nothing wired into the assistant. The pretrained model catches every attempt in a quiet room and with quiet music, but only 3 of 8 under music at normal volume, and pipewire here has no echo canceller to remove it. The options are listed for a later decision. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-rw-r--r--AGENTS.md46
1 files changed, 46 insertions, 0 deletions
diff --git a/AGENTS.md b/AGENTS.md
index 9966849..2022166 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -547,6 +547,49 @@ pending. Parses the 2.1 MB ChangeLog in ~95 ms, so no announcement.
the quickshell repo (fresh `Socket` per retry), which also covers login,
where the overlay starts first.
+### Wake word trial, openWakeWord (2026-09-28, undecided)
+
+Logging-only trial of a wake word instead of the button, nothing wired
+into the assistant. openWakeWord 0.6.0 in a throwaway venv
+(`--system-site-packages`, `pip install --no-deps`), pretrained
+`hey_jarvis_v0.1` on the ONNX backend. Models in
+`/data/voice-models/openwakeword/` (3.6 MB, from the openWakeWord GitHub
+release v0.5.1): `melspectrogram.onnx`, `embedding_model.onnx`,
+`hey_jarvis_v0.1.onnx`. The logger read `arecord` 16 kHz mono in 80 ms
+chunks, threshold 0.5, and logged hits, near misses (0.2-0.5) and the
+mic switch state; the assistant was stopped meanwhile.
+
+- The package imports scikit-learn at init only for custom verifier
+ models; stubbing the module in `sys.modules` is enough.
+ `tflite-runtime` has no Python 3.12 wheels; ONNX needs neither.
+- The mic's mute switch gates the detector too: with it off ALSA
+ delivers silence, so a wake word means an open mic, and the button can
+ no longer be push-to-talk.
+- Quiet room, 2 min take: 8/8 hits, peaks 0.66-0.999, no false wakes.
+ Speech at -30 to -40 dBFS, noise floor -62 dBFS. The accent is fine.
+- Music at low volume (floor -54.5 dBFS): 6/6 live, 5/5 in a second
+ run, but peaks dropped to 0.54-0.96. The same utterance scored 0.465
+ offline and 0.919 live: the score depends on where the 80 ms chunks
+ fall, so near the threshold it is noisy.
+- Music at the user's usual volume (floor -38 dBFS, voice only ~5 dB
+ above it): **3/8**. Offline the misses peaked at 0.40-0.48; live two
+ of them stayed under 0.2. Earlier, casual use with music gave 2 of 10+.
+- No false wakes from music at either volume, nor from other background
+ sound (~-50 dBFS): nothing but the user's attempts scored above 0.2.
+- The score flickers under 0.2 for single frames mid-word: one utterance
+ needs ~0.5 s under the floor before it counts as ended.
+- **No echo cancellation available:** Slackware's pipewire 1.6.9 is built
+ without `webrtc-audio-processing`; only `libspa-aec-null.so` is
+ installed, and `pactl load-module module-echo-cancel` fails with "No
+ such entity".
+- Options left open by the user: a lower threshold (0.3 would have
+ caught 6/8 live under loud music, false-wake rate at 0.3 unmeasured),
+ maybe only while a player is playing; ducking the music on a near
+ miss; packaging `webrtc-audio-processing` and rebuilding pipewire with
+ it (a core package, decision for `my-slackbuilds`; would also cover
+ the assistant's own voice); or keeping push-to-talk and the wake word
+ only for quiet rooms.
+
### TTS (2026-09-25)
`piper-tts` on CPU (onnxruntime), ~5 s sentence per voice:
@@ -608,6 +651,9 @@ no particular order:
desktop), own spec, after phase 1 has been used for a while.
- OpenVINO in the whisper.cpp package: runs on CPU only here (no Intel
compute-runtime), decision goes to `my-slackbuilds`.
+- **Wake word:** `hey_jarvis` works in quiet rooms, fails under loud
+ music (3/8); no echo canceller in pipewire. Pick an option from
+ "Wake word trial" before building anything.
Open questions: whether an audio-capable LLM makes whisper unnecessary.
License: GPLv2 only (chosen 2026-09-25).