diff options
| author | Danilo M. <danix@danix.xyz> | 2026-09-28 12:18:06 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-09-28 12:18:06 +0200 |
| commit | c873e3ca7b51ce159cbfac5f9635d67e542c4c0a (patch) | |
| tree | c9a2971014b19a536428c4e1fd09340e30746237 | |
| parent | 7a6190becf90364c6f31f401ccda745e4745081c (diff) | |
| download | desktop-assistant-c873e3ca7b51ce159cbfac5f9635d67e542c4c0a.tar.gz desktop-assistant-c873e3ca7b51ce159cbfac5f9635d67e542c4c0a.zip | |
Record the openWakeWord hey_jarvis trial
Logging-only trial, nothing wired into the assistant. The pretrained
model catches every attempt in a quiet room and with quiet music, but
only 3 of 8 under music at normal volume, and pipewire here has no echo
canceller to remove it. The options are listed for a later decision.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| -rw-r--r-- | AGENTS.md | 46 |
1 files changed, 46 insertions, 0 deletions
@@ -547,6 +547,49 @@ pending. Parses the 2.1 MB ChangeLog in ~95 ms, so no announcement. the quickshell repo (fresh `Socket` per retry), which also covers login, where the overlay starts first. +### Wake word trial, openWakeWord (2026-09-28, undecided) + +Logging-only trial of a wake word instead of the button, nothing wired +into the assistant. openWakeWord 0.6.0 in a throwaway venv +(`--system-site-packages`, `pip install --no-deps`), pretrained +`hey_jarvis_v0.1` on the ONNX backend. Models in +`/data/voice-models/openwakeword/` (3.6 MB, from the openWakeWord GitHub +release v0.5.1): `melspectrogram.onnx`, `embedding_model.onnx`, +`hey_jarvis_v0.1.onnx`. The logger read `arecord` 16 kHz mono in 80 ms +chunks, threshold 0.5, and logged hits, near misses (0.2-0.5) and the +mic switch state; the assistant was stopped meanwhile. + +- The package imports scikit-learn at init only for custom verifier + models; stubbing the module in `sys.modules` is enough. + `tflite-runtime` has no Python 3.12 wheels; ONNX needs neither. +- The mic's mute switch gates the detector too: with it off ALSA + delivers silence, so a wake word means an open mic, and the button can + no longer be push-to-talk. +- Quiet room, 2 min take: 8/8 hits, peaks 0.66-0.999, no false wakes. + Speech at -30 to -40 dBFS, noise floor -62 dBFS. The accent is fine. +- Music at low volume (floor -54.5 dBFS): 6/6 live, 5/5 in a second + run, but peaks dropped to 0.54-0.96. The same utterance scored 0.465 + offline and 0.919 live: the score depends on where the 80 ms chunks + fall, so near the threshold it is noisy. +- Music at the user's usual volume (floor -38 dBFS, voice only ~5 dB + above it): **3/8**. Offline the misses peaked at 0.40-0.48; live two + of them stayed under 0.2. Earlier, casual use with music gave 2 of 10+. +- No false wakes from music at either volume, nor from other background + sound (~-50 dBFS): nothing but the user's attempts scored above 0.2. +- The score flickers under 0.2 for single frames mid-word: one utterance + needs ~0.5 s under the floor before it counts as ended. +- **No echo cancellation available:** Slackware's pipewire 1.6.9 is built + without `webrtc-audio-processing`; only `libspa-aec-null.so` is + installed, and `pactl load-module module-echo-cancel` fails with "No + such entity". +- Options left open by the user: a lower threshold (0.3 would have + caught 6/8 live under loud music, false-wake rate at 0.3 unmeasured), + maybe only while a player is playing; ducking the music on a near + miss; packaging `webrtc-audio-processing` and rebuilding pipewire with + it (a core package, decision for `my-slackbuilds`; would also cover + the assistant's own voice); or keeping push-to-talk and the wake word + only for quiet rooms. + ### TTS (2026-09-25) `piper-tts` on CPU (onnxruntime), ~5 s sentence per voice: @@ -608,6 +651,9 @@ no particular order: desktop), own spec, after phase 1 has been used for a while. - OpenVINO in the whisper.cpp package: runs on CPU only here (no Intel compute-runtime), decision goes to `my-slackbuilds`. +- **Wake word:** `hey_jarvis` works in quiet rooms, fails under loud + music (3/8); no echo canceller in pipewire. Pick an option from + "Wake word trial" before building anything. Open questions: whether an audio-capable LLM makes whisper unnecessary. License: GPLv2 only (chosen 2026-09-25). |
