aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--.gitignore2
-rw-r--r--AGENTS.md397
-rw-r--r--CLAUDE.md6
-rw-r--r--LICENSE338
-rw-r--r--README.md62
-rw-r--r--assistant.py281
6 files changed, 1086 insertions, 0 deletions
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..a230a78
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,2 @@
+.venv/
+__pycache__/
diff --git a/AGENTS.md b/AGENTS.md
new file mode 100644
index 0000000..899fd86
--- /dev/null
+++ b/AGENTS.md
@@ -0,0 +1,397 @@
+# desktop-assistant
+
+A local desktop voice assistant: speak, get a spoken answer, nothing leaves
+the machine. Everything runs on already-packaged local tools; this repo is
+the glue between them.
+
+Status: **first working loop** in `assistant.py` (2026-09-25). Mic button
+on/off is push-to-talk, whisper large-v3-turbo on the Arc transcribes,
+the resident llama-server model answers, Kokoro speaks it. Button-off to
+first spoken word ~2 s. Kokoro is not packaged yet: it runs from the
+repo's gitignored `.venv`. See "Findings" and "Next steps".
+
+Run: `.venv/bin/python assistant.py` (`--test` for the self-check). The
+venv is `python3 -m venv --system-site-packages .venv && .venv/bin/pip
+install kokoro-onnx`; system-site-packages gives it the system onnxruntime.
+
+---
+
+## Pipeline
+
+```
+mic button ──> arecord ──> whisper-server (STT) ──> llama-server (LLM) ──> Kokoro (TTS) ──> aplay
+```
+
+| Stage | Package (installed) | Binary | Notes |
+|-------|---------------------|--------|-------|
+| STT | `whisper.cpp` 1.9.4 | `whisper-server` (also `whisper-stream`, `whisper-cli`) | SDL2 mic capture, ffmpeg decoding, Vulkan + OpenVINO backends |
+| LLM | `llama.cpp-vulkan` 0.5.0 | `llama-server` | already running as a system service, see below |
+| TTS | none yet: `kokoro-onnx` from PyPI in `.venv` | | uses system `python3-onnxruntime` and `espeak-ng` |
+| (TTS, unused) | `piper-tts` 1.7.0 | `piper-tts` | voices rejected by ear, see Findings |
+
+The three packages are in the `my-slackbuilds` repo
+(`../slackbuilds-superrepo/my-slackbuilds/`). Package-level changes (build
+options, backends, bumps) happen there, under that repo's `AGENTS.md`, not
+here. `openvino` 2026.4.0 is also packaged there, solely to feed whisper.cpp.
+
+## Hardware
+
+- Discrete GPU: Intel Arc B580 (Battlemage). Target for both Vulkan and
+ OpenVINO GPU.
+- Integrated GPU: AMD Radeon 610M on a Ryzen 7 9700X, also exposed through
+ Vulkan (RADV). Vulkan enumeration order is Arc first, iGPU second, llvmpipe
+ last. Check device indices in the startup log before picking `-dev N`,
+ the iGPU is a plausible wrong pick.
+
+## whisper.cpp: what the package gives you
+
+Built from ggml-org upstream 1.9.4 (SBo still ships 1.6.2, which is why it
+is ours). Binaries only, libraries static: bundled ggml would clash with
+llama.cpp-vulkan's.
+
+Installed tools: `whisper-cli`, `whisper-server`, `whisper-bench`,
+`whisper-quantize`, `whisper-vad-speech-segments`, `parakeet-cli`,
+`parakeet-quantize`, and the SDL2 mic tools `whisper-stream` (live
+transcription), `whisper-command` (voice commands), `whisper-talk-llama`,
+`whisper-lsp`.
+
+Backends compiled in: CPU, BLAS (OpenBLAS, autodetected), Vulkan
+(`GGML_VULKAN`), ggml OpenVINO (`GGML_OPENVINO`, whole model through
+OpenVINO), plus the older `WHISPER_OPENVINO` encoder-only path.
+
+- Vulkan and ggml-openvino both register as GPU devices. whisper uses the
+ first GPU it finds; choose with `whisper-cli -dev N`.
+- `whisper-bench` has **no** device flag. Compare backends with
+ `whisper-cli -dev N` on the same sample and read the timings it prints at
+ exit.
+- The `WHISPER_OPENVINO` encoder path needs the encoder converted to
+ OpenVINO IR first (upstream README, "OpenVINO Support"). Not done yet.
+ The ggml-openvino device does not need that conversion.
+- OpenVINO libs are found through an rpath
+ (`/usr/share/openvino/runtime/lib/intel64`); no need to source
+ `setupvars.sh` at runtime. The OpenVINO GPU plugin
+ (`libopenvino_intel_gpu_plugin.so`) is installed.
+- Input goes through ffmpeg, so any format ffmpeg reads works.
+- Speech encoders expect 16 kHz mono.
+
+`whisper-talk-llama` already does mic, whisper, LLM in one binary, but it
+loads its own model rather than talking to the running `llama-server`. Worth
+reading as reference, probably not the final shape.
+
+## llama-server: already running
+
+A system service, `/etc/rc.d/rc.llama.cpp`, runs `llama-server` in **router
+mode** on `http://localhost:8181` with an OpenAI-compatible API.
+
+- Models are defined in `/etc/llama-server/presets.ini` (sections with
+ `model =` and optional `mmproj =`), GGUF files under `/data/LLM-models/`.
+- `--models-max 1`: the router keeps **one** model loaded. If the assistant
+ asks for a different model than the one in use, the router swaps it out,
+ under whoever else is using it (e.g. llamachat). Prefer reusing whatever
+ model is loaded, or accept the reload latency knowingly.
+- `/v1/models` lists models and each one's `architecture.input_modalities`.
+ A model reporting `"audio"` accepts `input_audio` content parts directly,
+ which is a possible shortcut that skips whisper entirely. Not evaluated.
+
+Prior art: `../local-chat` (llamachat, PySide6) is a client for the same
+router. It already records 16 kHz mono WAV and sends it as `input_audio`
+to audio-capable models, and parses `presets.ini`. Reuse its findings, do
+not import from it.
+
+## piper-tts
+
+Prebuilt upstream wheel (OHF-voice/piper1-gpl), console script `piper-tts`,
+Python module `piper` under the system python3.12. Runtime deps
+`python3-onnxruntime` and `python3-pathvalidate` are packaged separately.
+Voices are ONNX models plus a `.onnx.json` config; none downloaded yet.
+
+---
+
+## Findings
+
+### Models (2026-09-25)
+
+Everything under `/data/voice-models/` (outside the repo, kept apart from
+`/data/LLM-models/`):
+
+- `whisper/ggml-base.bin`, 142 MB, multilingual. SHA1 matches upstream
+ (`465707469ff3a37a2b9b8d8f89f2f99de7299dac`).
+- `piper/it_IT-paola-medium.onnx` and `piper/en_US-lessac-medium.onnx`,
+ 61 MB each, plus their `.onnx.json`. Both output 22050 Hz.
+- `samples/it.wav`, `samples/en.wav`: 10 s, 16 kHz mono mic recordings used
+ for the STT comparison. Personal voice data, never copy into the repo.
+
+### STT backends (2026-09-25)
+
+Same 10 s clip, `ggml-base`, `whisper-cli`:
+
+| `-dev` | Device | Total (warm) | Result |
+|--------|--------|--------------|--------|
+| `-ng` | CPU, 4 threads | ~370-420 ms | correct |
+| 0 | Vulkan0, Arc B580 | ~160-240 ms | correct |
+| 1 | Vulkan1, iGPU | ~900 ms | correct |
+| 2 | OPENVINO0 | 13.7 s | garbage |
+
+**Decision: Arc on Vulkan, `-dev 0`, `-l auto`.**
+
+- `-dev N` indexes GPU-type devices only, not the full device list:
+ 0 = Arc, 1 = iGPU, 2 = OpenVINO. An index past the last GPU silently falls
+ back to CPU.
+- The Arc's first run after a while costs 0.6-1.4 s (Vulkan pipeline
+ warm-up), later runs are fast. A long-running process pays it once.
+- `-l auto` detected both languages correctly (it, p=0.999) for ~40 ms extra
+ (one more encoder pass). One setup covers Italian and English.
+- ggml-openvino runs on **CPU**, not the Arc: the log says "OpenVINO: using
+ device CPU" even with `GGML_OPENVINO_DEVICE=GPU`, and its output is
+ garbage. Only OpenCL platform installed is Mesa `rusticl`; Intel
+ compute-runtime (NEO), which the OpenVINO GPU plugin needs, is not
+ installed. Likely cause, not confirmed. Decision on keeping OpenVINO in
+ the package is deferred to `my-slackbuilds`.
+- Every run tries to load `ggml-base-encoder-openvino.xml` (the
+ `WHISPER_OPENVINO` encoder path), fails, and continues. Harmless noise.
+- base accuracy is fine on full sentences, weak on short words: "ho" came
+ out as "a", a spoken "hi" came out as "5". Try `ggml-small` (~488 MB) if
+ that hurts short commands.
+
+### Live mic with whisper-stream (2026-09-25)
+
+`whisper-stream -m ggml-base.bin -l auto --step 0 --length 8000 -vth 0.6`
+(VAD mode), 40 s, mixed Italian/English utterances with pauses.
+
+- No `-dev` flag; it takes the first GPU, which is the Arc (Vulkan0). Default
+ SDL capture device is the USB mic, correct.
+- **Not usable as the assistant's STT entry point.** Its "VAD" is a crude
+ energy check: on speech activity it re-transcribes the whole last
+ `--length` window, not a segmented utterance. The same sentence comes out
+ 2-3 times, windows straddle two utterances in different languages (one
+ output mixed an English sentence with the next Italian one), and `-l auto`
+ detects per window, so mixed windows garble. A window with short words
+ ("hello, hi, ciao") came out as Greek gibberish.
+- Content of a clean window was mostly right ("ricordami di comprare
+ latte"), so the model is fine, the segmentation is the problem.
+- Conclusion: the loop must segment utterances itself (push-to-talk or a real
+ VAD) and send exactly one utterance to whisper, kept warm in a
+ long-running process (`whisper-server`) to avoid the Arc warm-up per call.
+
+### Push-to-talk via the mic's mute button (2026-09-25)
+
+The USB mic's hardware mute button toggles the ALSA control
+`numid=2 'Mic Capture Switch'` on card 0 (`amixer -c 0 cget numid=2` reads
+on/off, observed flipping with each press). So the button *is* the
+push-to-talk: switch on = start recording, switch off = stop and send the
+utterance to whisper. No keybind, no VAD.
+
+- Event-driven, no polling: `amixer -c 0 events` prints a line on each
+ control change ("Ready to listen..." then events).
+- Card index 0 and numid 2 are what this machine shows now; card numbering
+ can change with USB plug order, so resolve by card name, not number.
+- The mic also exposes a HID keyboard device (`/dev/input/event3`, group
+ `input`); not needed, the ALSA switch is enough.
+- **Conflict: calls.** In a call the same button unmutes the mic for the
+ call, and the assistant would record and answer the call. Decision: the
+ assistant is paused while **presentation mode** is on (the user turns it on
+ for calls). Presentation mode lives in the desktop status registry
+ `~/bin/statusctl` (quickshell repo): `statusctl presentation get` prints
+ `1`/`0`, backed by `$XDG_RUNTIME_DIR/status.presentation` (missing = off).
+ Check it at the moment the switch goes on, and ignore the press if `1`.
+ No watcher needed. Considered and not chosen: skipping when another app
+ already records from the mic (`pactl list source-outputs`, sound server is
+ PipeWire with the Pulse API); fallback if presentation mode gets forgotten.
+
+### Push-to-talk round trip with whisper-server (2026-09-25)
+
+`whisper-server -m ggml-base.bin -dev 0 -l auto --host 127.0.0.1 --port 8182`
+(port next to llama-server's 8181), throwaway bash loop: `amixer events`
+drives `arecord` on/off, clip POSTed to `/inference`.
+
+- Works end to end. Warm server: **0.10-0.23 s** per utterance, including the
+ very first request (no per-call Arc warm-up once the server is up).
+- `amixer events` piped into a loop is block-buffered and delivers nothing;
+ needs `stdbuf -oL amixer ...`.
+- Do **not** mute the mic from software (`amixer cset numid=2 off`): tried
+ at loop startup and the user found it unreliable (likely out of sync with
+ the button's own state/LED). The user drives mute/unmute on the button
+ only; the loop just reads the current state at start.
+- `amixer -c Microphone` (the card id) works, no need for the index.
+- Stopping `arecord` with SIGINT leaves the WAV header unfinalized (size
+ field reads as ~67108 s). whisper decodes it anyway; do not trust the
+ header for the duration.
+- **Language auto-detect is the weak spot with base on live speech:** an
+ English sentence came out as a Greek/Hebrew/Arabic mix, "ciao" came out as
+ "Tchau" (Portuguese). Italian sentence fine, "hi" fine. The same sentences
+ from the pre-recorded samples detected correctly, so short or accented
+ utterances push base's detector off. Only it/en will ever be used.
+- Correction: the pre-recorded English sample *also* detects as Greek
+ (el p=0.88, en 0.009, it 0.075), on server and CLI, CPU and GPU alike; the
+ earlier CLI run decoded English text in spite of the wrong language tag.
+ base's detector does not handle this speaker's English.
+- Tried and rejected for base: (1) pick it/en by `language_probabilities`
+ (verbose_json): picks it for English. (2) transcribe forced `it` and forced
+ `en`, keep higher token-weighted `avg_logprob`: forced-it turns English into
+ plausible Italian and scores *higher* (-0.164 vs -0.287). Moving to
+ `ggml-small`.
+- `ggml-small.bin` (466 MB, SHA1 `55356645c2b361a969dfd0ef2c5a50d530afd8d5`,
+ matches upstream) in `/data/voice-models/whisper/`. On the Arc via
+ whisper-server: **~0.35 s per pass** on a 10 s clip. Transcripts clean on
+ both samples (Italian and English self-introductions, all words right).
+- small's detector *still* mislabels the English sample (la 0.45, it 0.31,
+ en 0.04) though it decodes English text under auto. Not trustworthy.
+- **Picker works with small:** forced `it` vs forced `en`, keep higher
+ token-weighted `avg_logprob`. it.wav: it -0.196 vs en -0.426; en.wav:
+ en -0.127 vs it -0.243. Both correct, ~0.5 s for the two passes.
+ **STT decision: ggml-small + it/en picker.**
+- Live push-to-talk with small + picker, 5 utterances: Italian sentence,
+ English sentence, "hi", and a mixed it/en sentence all transcribed
+ perfectly, 0.42-0.54 s each (two passes). "ciao" failed: forced `en`
+ produced Chinese characters ("早, 早!") with the *better* logprob (-0.032
+ vs -0.312); forced `it` gave "Ciao!". An initial `prompt` ("Ciao. Hi.")
+ does not fix forced-en.
+- **Fix: reject any candidate containing non-Latin letters** (it and en are
+ both Latin script; `unicodedata.name(c).startswith("LATIN")` for every
+ alphabetic char). With that guard all 5 live clips come out right.
+- Mixed-language sentences: the picker chooses one language, but small still
+ transcribes both halves correctly under either forced language.
+- In the full loop the picker still failed: spoken English "What does HTML
+ stand for?" -> picked it ("Che stanno per HTML?", it -0.153 vs en -0.214)
+ though small's own detector said en 0.926 on that clip. Combining
+ `logprob + w*log(p_detect)` over 8 saved clips: 7/8 at every w, never 8/8.
+ small's two signals are each unreliable on this speaker, in different clips.
+
+### large-v3-turbo (2026-09-25, **chosen**, replaces small)
+
+`ggml-large-v3-turbo-q5_0.bin`, 548 MB, sha256 matches huggingface
+(`394221709cd5ad1f40c46e6031ca61bce88931e6e088c188294c6d5a55ffa7e2`).
+
+- Auto-detect correct on **all 7** single-language clips, including the
+ English ones small's detector called Latin/Greek; the mixed it/en clip
+ comes out "it". Transcripts good (spoken "HTML stands for" kept English).
+- Its forced-language logprob picker is *worse* than small's (it.wav picks en,
+ -0.225 vs -0.667): with turbo, trust the detector, drop the picker.
+- Cost on the Arc: ~0.95 s (whisper-cli) / ~1.2 s (whisper-server) per
+ clip. Encoder runs twice (~400 ms each): once for language detection, once
+ for transcription.
+- `-ac` (audio context) does not pay: 768 saves ~0.25 s but starts
+ repeating words, 512 repeats more, 384 is garbage. Keep 0 (full).
+
+### The loop, `assistant.py` (2026-09-25)
+
+- STT: one `language=auto` pass on turbo; the answer is whichever of it/en
+ the detector rates higher. Forced second pass only if detection lands on
+ a third language or the text is non-Latin. whisper-server's default
+ language is **en**, not auto: always send `language=auto` explicitly.
+- Live: turbo still tagged one English question "it" ("What is
+ slackware?"), but the text stayed English, so no harm, because:
+- **TTS voice follows the reply's language, not the user's.** The model may
+ answer in the other language (seen: user tagged en, reply Italian, read
+ by the English voice). Per-sentence stopword count picks it/en.
+- **Pad 0.5 s of silence** after each clip: whisper dropped the final word
+ ("... a cosa serve [python]") when the button was pressed mid-word.
+ Fixed by the padding. Recording is raw PCM (`arecord -t raw`), wrapped
+ into a WAV with stdlib `wave`, which also fixes the SIGINT header problem.
+- LLM: model = first one in `/v1/models` whose status is not `unloaded`
+ (resident is Gemma4-12B, status `sleeping` when idle). Warm first token
+ ~0.3 s; after `sleep-idle-seconds 1800` the first request waits ~45 s for
+ the reload. `chat_template_kwargs.enable_thinking=false` sent; only
+ `delta.content` is read.
+- Timings live: STT ~1.23 s, first LLM sentence 0.6-0.8 s.
+- Child processes (whisper-server, `amixer events`) get
+ `prctl(PR_SET_PDEATHSIG, SIGTERM)` via ctypes in `preexec_fn`: they die
+ with the assistant even on SIGKILL or a kill during startup (tested).
+ Python's default SIGTERM skips `finally`; a handler calls `sys.exit`.
+- Gemma confidently gave a wrong fact (a city-to-city distance off by
+ ~10x). No tools yet, so no time/weather/facts lookup.
+- Recordings live in `$XDG_RUNTIME_DIR` and are deleted on exit.
+- The reply's first word was often clipped: each reply opens a new `aplay`
+ stream and the idle sink needs a moment to wake. 0.3 s of leading silence
+ per reply fixed it (confirmed by ear).
+
+### TTS (2026-09-25)
+
+`piper-tts` on CPU (onnxruntime), ~5 s sentence per voice:
+
+- CLI to WAV: 0.64-0.76 s wall for ~5 s of audio, ~7x real time.
+- CLI `--output-raw`, time to first byte: ~0.58 s, nearly all voice load
+ (0.46 s). Python + `import piper` alone is 0.09 s.
+- Voice preloaded in-process (`PiperVoice.load` once, then
+ `voice.synthesize(text)` yields chunks per sentence): first chunk in
+ 0.035-0.069 s.
+- So: a per-reply `piper-tts` process costs ~0.5 s of load each time; a
+ long-running process holding the voice makes TTS latency negligible.
+- `--output-raw` is 16-bit mono PCM at the voice's rate (22050 Hz):
+ `piper-tts -m <voice> --output-raw | aplay -r 22050 -f S16_LE -c 1`.
+- **Quality rejected by ear** on a ~20 s paragraph: Italian `paola-medium`
+ flat with odd rhythm, English `lessac-medium` metallic and robotic.
+ Piper speed is fine, its voices are not. Looking at alternatives.
+
+### Kokoro-82M (2026-09-25, **chosen**)
+
+`kokoro-onnx` 0.6.1 in a throwaway venv (`--system-site-packages`, so it
+uses the system onnxruntime 1.30, CPU). Model `kokoro-v1.0.onnx` (311 MB)
+and `voices-v1.0.bin` (27 MB) in `/data/voice-models/kokoro/`, from the
+kokoro-onnx GitHub release `model-files-v1.0`.
+
+- The PyPI `espeakng-loader` wheel ships a lib with a hardcoded CI data path
+ and fails ("Error processing file .../phontab"). Point it at the system
+ espeak-ng instead:
+ `EspeakConfig(lib_path="/usr/lib64/libespeak-ng.so.1", data_path="/usr/share/espeak-ng-data")`.
+ A system package would do the same and drop `espeakng-loader`.
+- Load 0.30 s. `create()` on the ~20 s paragraph: ~2.6 s, ~7x real time,
+ same for `if_sara`, `im_nicola` (lang `it`), `af_heart` (`en-us`),
+ `bm_george` (`en-gb`). `create()` returns only after the whole text, so
+ a reply needs per-sentence synthesis (or `create_stream`) to start
+ speaking early.
+- **By ear: much better than piper. Kokoro chosen.** Voices: `if_sara`
+ (Italian) and `af_heart` (English), both female for consistency.
+ `bm_george` also liked (male, "Jarvis"-like) but not used. `if_sara` wants
+ a bit more color/expressiveness.
+- Kokoro's only knob is `speed`; color comes from blending voice style
+ vectors (`k.get_voice_style(name)` returns an array, `create(voice=...)`
+ accepts it). Compared by ear: sara alone, sara 80/heart 20, sara 65/heart
+ 35, sara 80/bella 20. **Italian voice chosen: `0.8*if_sara + 0.2*af_bella`,
+ lang `it`.**
+
+---
+
+## Next steps
+
+Steps 1-5 of the original plan are done (models, backend comparison, live
+mic, TTS, glue); push-to-talk came for free from the mic button. Open, in
+no particular order:
+
+- **Packaging Kokoro** in `my-slackbuilds` (`kokoro-onnx` without
+ `espeakng-loader`, pointed at system espeak-ng), then drop `.venv`.
+- **Run as a service/autostart** (it currently runs by hand).
+- **Barge-in:** pressing the button while it speaks does not stop the
+ reply yet.
+- **Check presentation mode during games:** `statusctl presentation get`
+ reads the stored flag; confirm it is 1 while a game holds it on.
+- **Latency:** STT ~1.2 s is the biggest share; the double encoder pass
+ (detect + transcribe) is the target if it matters.
+- **Tools** (time, weather, calendar) so the model stops guessing facts.
+- Piper leftovers: package unused, voices in `/data/voice-models/piper/`
+ (~120 MB) can go. ggml-base and ggml-small too, once turbo is settled.
+- OpenVINO in the whisper.cpp package: runs on CPU only here (no Intel
+ compute-runtime), decision goes to `my-slackbuilds`.
+
+Open questions: whether an audio-capable LLM makes whisper unnecessary.
+License: GPLv2 only (chosen 2026-09-25).
+
+## Package-side leftovers (tracked in my-slackbuilds, not here)
+
+- whisper.cpp not yet built on the buildsystem VM via slackrepo (the
+ installed package is from the local build).
+- `sbopkglint` on the openvino package not done.
+- OpenCL-CLHPP 2026.05.29 (needed by openvino and ggml-openvino) exists
+ only as a buildsystem hint; the Docker test-build needs a temporary local
+ copy when its cache is cold. Permanent fix undecided.
+- whisper.cpp and openvino cannot build on Slackware 15.0 (CMake 3.26,
+ glslc, old CLHPP). -current only.
+
+---
+
+## Agent rules
+
+- Ask before acting when anything is ambiguous, and before committing.
+- Package changes belong in `my-slackbuilds`, not here.
+- Never download models or voices without saying which and how big.
+- Everything stays local: no cloud STT/TTS/LLM fallbacks.
diff --git a/CLAUDE.md b/CLAUDE.md
new file mode 100644
index 0000000..dfe9f3f
--- /dev/null
+++ b/CLAUDE.md
@@ -0,0 +1,6 @@
+# desktop-assistant
+
+This file is intentionally thin. AGENTS.md is the single source of truth for
+every agent tool. Edit AGENTS.md instead.
+
+@AGENTS.md
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000..9efa6fb
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,338 @@
+ GNU GENERAL PUBLIC LICENSE
+ Version 2, June 1991
+
+ Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
+ <https://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+ Preamble
+
+ The licenses for most software are designed to take away your
+freedom to share and change it. By contrast, the GNU General Public
+License is intended to guarantee your freedom to share and change free
+software--to make sure the software is free for all its users. This
+General Public License applies to most of the Free Software
+Foundation's software and to any other program whose authors commit to
+using it. (Some other Free Software Foundation software is covered by
+the GNU Lesser General Public License instead.) You can apply it to
+your programs, too.
+
+ When we speak of free software, we are referring to freedom, not
+price. Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+this service if you wish), that you receive source code or can get it
+if you want it, that you can change the software or use pieces of it
+in new free programs; and that you know you can do these things.
+
+ To protect your rights, we need to make restrictions that forbid
+anyone to deny you these rights or to ask you to surrender the rights.
+These restrictions translate to certain responsibilities for you if you
+distribute copies of the software, or if you modify it.
+
+ For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must give the recipients all the rights that
+you have. You must make sure that they, too, receive or can get the
+source code. And you must show them these terms so they know their
+rights.
+
+ We protect your rights with two steps: (1) copyright the software, and
+(2) offer you this license which gives you legal permission to copy,
+distribute and/or modify the software.
+
+ Also, for each author's protection and ours, we want to make certain
+that everyone understands that there is no warranty for this free
+software. If the software is modified by someone else and passed on, we
+want its recipients to know that what they have is not the original, so
+that any problems introduced by others will not reflect on the original
+authors' reputations.
+
+ Finally, any free program is threatened constantly by software
+patents. We wish to avoid the danger that redistributors of a free
+program will individually obtain patent licenses, in effect making the
+program proprietary. To prevent this, we have made it clear that any
+patent must be licensed for everyone's free use or not licensed at all.
+
+ The precise terms and conditions for copying, distribution and
+modification follow.
+
+ GNU GENERAL PUBLIC LICENSE
+ TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+ 0. This License applies to any program or other work which contains
+a notice placed by the copyright holder saying it may be distributed
+under the terms of this General Public License. The "Program", below,
+refers to any such program or work, and a "work based on the Program"
+means either the Program or any derivative work under copyright law:
+that is to say, a work containing the Program or a portion of it,
+either verbatim or with modifications and/or translated into another
+language. (Hereinafter, translation is included without limitation in
+the term "modification".) Each licensee is addressed as "you".
+
+Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope. The act of
+running the Program is not restricted, and the output from the Program
+is covered only if its contents constitute a work based on the
+Program (independent of having been made by running the Program).
+Whether that is true depends on what the Program does.
+
+ 1. You may copy and distribute verbatim copies of the Program's
+source code as you receive it, in any medium, provided that you
+conspicuously and appropriately publish on each copy an appropriate
+copyright notice and disclaimer of warranty; keep intact all the
+notices that refer to this License and to the absence of any warranty;
+and give any other recipients of the Program a copy of this License
+along with the Program.
+
+You may charge a fee for the physical act of transferring a copy, and
+you may at your option offer warranty protection in exchange for a fee.
+
+ 2. You may modify your copy or copies of the Program or any portion
+of it, thus forming a work based on the Program, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+ a) You must cause the modified files to carry prominent notices
+ stating that you changed the files and the date of any change.
+
+ b) You must cause any work that you distribute or publish, that in
+ whole or in part contains or is derived from the Program or any
+ part thereof, to be licensed as a whole at no charge to all third
+ parties under the terms of this License.
+
+ c) If the modified program normally reads commands interactively
+ when run, you must cause it, when started running for such
+ interactive use in the most ordinary way, to print or display an
+ announcement including an appropriate copyright notice and a
+ notice that there is no warranty (or else, saying that you provide
+ a warranty) and that users may redistribute the program under
+ these conditions, and telling the user how to view a copy of this
+ License. (Exception: if the Program itself is interactive but
+ does not normally print such an announcement, your work based on
+ the Program is not required to print an announcement.)
+
+These requirements apply to the modified work as a whole. If
+identifiable sections of that work are not derived from the Program,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works. But when you
+distribute the same sections as part of a whole which is a work based
+on the Program, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Program.
+
+In addition, mere aggregation of another work not based on the Program
+with the Program (or with a work based on the Program) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+ 3. You may copy and distribute the Program (or a work based on it,
+under Section 2) in object code or executable form under the terms of
+Sections 1 and 2 above provided that you also do one of the following:
+
+ a) Accompany it with the complete corresponding machine-readable
+ source code, which must be distributed under the terms of Sections
+ 1 and 2 above on a medium customarily used for software interchange; or,
+
+ b) Accompany it with a written offer, valid for at least three
+ years, to give any third party, for a charge no more than your
+ cost of physically performing source distribution, a complete
+ machine-readable copy of the corresponding source code, to be
+ distributed under the terms of Sections 1 and 2 above on a medium
+ customarily used for software interchange; or,
+
+ c) Accompany it with the information you received as to the offer
+ to distribute corresponding source code. (This alternative is
+ allowed only for noncommercial distribution and only if you
+ received the program in object code or executable form with such
+ an offer, in accord with Subsection b above.)
+
+The source code for a work means the preferred form of the work for
+making modifications to it. For an executable work, complete source
+code means all the source code for all modules it contains, plus any
+associated interface definition files, plus the scripts used to
+control compilation and installation of the executable. However, as a
+special exception, the source code distributed need not include
+anything that is normally distributed (in either source or binary
+form) with the major components (compiler, kernel, and so on) of the
+operating system on which the executable runs, unless that component
+itself accompanies the executable.
+
+If distribution of executable or object code is made by offering
+access to copy from a designated place, then offering equivalent
+access to copy the source code from the same place counts as
+distribution of the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+ 4. You may not copy, modify, sublicense, or distribute the Program
+except as expressly provided under this License. Any attempt
+otherwise to copy, modify, sublicense or distribute the Program is
+void, and will automatically terminate your rights under this License.
+However, parties who have received copies, or rights, from you under
+this License will not have their licenses terminated so long as such
+parties remain in full compliance.
+
+ 5. You are not required to accept this License, since you have not
+signed it. However, nothing else grants you permission to modify or
+distribute the Program or its derivative works. These actions are
+prohibited by law if you do not accept this License. Therefore, by
+modifying or distributing the Program (or any work based on the
+Program), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Program or works based on it.
+
+ 6. Each time you redistribute the Program (or any work based on the
+Program), the recipient automatically receives a license from the
+original licensor to copy, distribute or modify the Program subject to
+these terms and conditions. You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties to
+this License.
+
+ 7. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License. If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Program at all. For example, if a patent
+license would not permit royalty-free redistribution of the Program by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under
+any particular circumstance, the balance of the section is intended to
+apply and the section as a whole is intended to apply in other
+circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system, which is
+implemented by public license practices. Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+ 8. If the distribution and/or use of the Program is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Program under this License
+may add an explicit geographical distribution limitation excluding
+those countries, so that distribution is permitted only in or among
+countries not thus excluded. In such case, this License incorporates
+the limitation as if written in the body of this License.
+
+ 9. The Free Software Foundation may publish revised and/or new versions
+of the General Public License from time to time. Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+Each version is given a distinguishing version number. If the Program
+specifies a version number of this License which applies to it and "any
+later version", you have the option of following the terms and conditions
+either of that version or of any later version published by the Free
+Software Foundation. If the Program does not specify a version number of
+this License, you may choose any version ever published by the Free Software
+Foundation.
+
+ 10. If you wish to incorporate parts of the Program into other free
+programs whose distribution conditions are different, write to the author
+to ask for permission. For software which is copyrighted by the Free
+Software Foundation, write to the Free Software Foundation; we sometimes
+make exceptions for this. Our decision will be guided by the two goals
+of preserving the free status of all derivatives of our free software and
+of promoting the sharing and reuse of software generally.
+
+ NO WARRANTY
+
+ 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
+FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
+OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
+PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
+OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
+TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
+PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
+REPAIR OR CORRECTION.
+
+ 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
+REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
+OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
+TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
+YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
+PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGES.
+
+ END OF TERMS AND CONDITIONS
+
+ How to Apply These Terms to Your New Programs
+
+ If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+ To do so, attach the following notices to the program. It is safest
+to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+ <one line to give the program's name and a brief idea of what it does.>
+ Copyright (C) <year> <name of author>
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 2 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License along
+ with this program; if not, see <https://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+If the program is interactive, make it output a short notice like this
+when it starts in an interactive mode:
+
+ Gnomovision version 69, Copyright (C) year name of author
+ Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+ This is free software, and you are welcome to redistribute it
+ under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License. Of course, the commands you use may
+be called something other than `show w' and `show c'; they could even be
+mouse-clicks or menu items--whatever suits your program.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary. Here is a sample; alter the names:
+
+ Yoyodyne, Inc., hereby disclaims all copyright interest in the program
+ `Gnomovision' (which makes passes at compilers) written by James Hacker.
+
+ <signature of Moe Ghoul>, 1 April 1989
+ Moe Ghoul, President of Vice
+
+This General Public License does not permit incorporating your program into
+proprietary programs. If your program is a subroutine library, you may
+consider it more useful to permit linking proprietary applications with the
+library. If this is what you want to do, use the GNU Lesser General
+Public License instead of this License.
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..d9f4d28
--- /dev/null
+++ b/README.md
@@ -0,0 +1,62 @@
+# desktop-assistant
+
+A local push-to-talk voice assistant. Unmute the microphone, speak, mute it:
+the assistant transcribes what you said, asks a local LLM, and speaks the
+answer. Italian and English. Nothing leaves the machine.
+
+```
+mic button ──> arecord ──> whisper-server (STT) ──> llama-server (LLM) ──> Kokoro (TTS) ──> aplay
+```
+
+## How it works
+
+- **Push-to-talk** is the microphone's own mute button: the script listens to
+ the ALSA `Mic Capture Switch` (`amixer events`). Switch on starts a
+ recording, switch off ends it.
+- **Speech to text**: whisper.cpp's `whisper-server` with the
+ `large-v3-turbo` model on the GPU (Vulkan). The script starts it if it is
+ not already running and stops it on exit.
+- **LLM**: an OpenAI-compatible `llama-server` on `localhost:8181`. The
+ script uses whichever model the server already has loaded, so it never
+ swaps models under other clients.
+- **Text to speech**: Kokoro-82M through `kokoro-onnx`, one voice per
+ language, chosen per sentence from the language of the reply. Speech
+ starts after the first sentence, while the rest is still being generated.
+- **Calls**: while the desktop's presentation mode is on
+ (`statusctl presentation get` returns `1`), button presses are ignored, so
+ a call can use the microphone.
+
+## Requirements
+
+- whisper.cpp (`whisper-server`) and a whisper ggml model
+- a running `llama-server` with an OpenAI-compatible API
+- Python 3 with `onnxruntime`, `numpy` and `kokoro-onnx`
+- `espeak-ng`, ALSA utils (`amixer`, `arecord`, `aplay`), `curl`
+- a USB microphone with a hardware mute button exposed as an ALSA switch
+
+Paths, the card name and ports are constants at the top of `assistant.py`.
+
+## Usage
+
+```sh
+python3 -m venv --system-site-packages .venv
+.venv/bin/pip install kokoro-onnx
+.venv/bin/python assistant.py # run
+.venv/bin/python assistant.py --test # self-check
+```
+
+## License
+
+Copyright (C) 2026 Danilo M. <danix@danix.xyz>
+
+This program is free software; you can redistribute it and/or modify it
+under the terms of the GNU General Public License version 2 as published by
+the Free Software Foundation. See [LICENSE](LICENSE).
+
+## Development Approach
+
+This project is developed using AI-assisted tools. Code is generated with the help of AI based on human-provided specifications, design decisions, and iterative feedback.
+
+All contributions are reviewed, tested, and curated by the maintainer before being included in the codebase. AI is used as a productivity and exploration tool, while human oversight remains central to all decisions.
+
+The goal is to combine the flexibility of AI-assisted development with standard open-source practices such as transparency, review, and accountability.
diff --git a/assistant.py b/assistant.py
new file mode 100644
index 0000000..49fb082
--- /dev/null
+++ b/assistant.py
@@ -0,0 +1,281 @@
+#!/usr/bin/env python3
+# desktop-assistant: push-to-talk local voice assistant.
+# Copyright (C) 2026 Danilo M. <danix@danix.xyz>
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 2 as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
+# Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+"""Mic button on: record. Mic button off: transcribe (whisper-server),
+ask the loaded llama-server model, speak the reply (Kokoro).
+
+Run: .venv/bin/python assistant.py (--test runs the self-check)
+"""
+
+import ctypes
+import json
+import os
+import queue
+import re
+import signal
+import subprocess
+import sys
+import threading
+import time
+import unicodedata
+import urllib.request
+import wave
+
+CARD = "Microphone" # ALSA card id of the USB mic
+MODELS = "/data/voice-models"
+WHISPER = "http://127.0.0.1:8182"
+LLM = "http://localhost:8181/v1"
+RAW = os.path.join(os.environ.get("XDG_RUNTIME_DIR", "/tmp"), "desktop-assistant.raw")
+WAV = RAW[:-3] + "wav"
+SYSTEM = ("You are a voice assistant. Your replies are spoken aloud, so answer "
+ "briefly in plain sentences: no markdown, no lists, no emoji. "
+ "Reply in the language the user speaks, Italian or English.")
+KLANG = {"it": "it", "en": "en-us"}
+
+
+def log(*a):
+ print(time.strftime("%H:%M:%S"), *a, flush=True)
+
+
+def mic_on():
+ out = subprocess.run(["amixer", "-c", CARD, "cget", "numid=2"],
+ capture_output=True, text=True).stdout
+ return out.rstrip().endswith("values=on")
+
+
+def presenting():
+ # During calls the user turns on presentation mode; the button then
+ # belongs to the call, not to us.
+ r = subprocess.run([os.path.expanduser("~/bin/statusctl"), "presentation", "get"],
+ capture_output=True, text=True)
+ return r.stdout.strip() == "1"
+
+
+def latin(text):
+ return all(unicodedata.name(c, "").startswith("LATIN") for c in text if c.isalpha())
+
+
+def transcribe(lang="auto"):
+ # Always send language: whisper-server's own default is en, not auto.
+ out = subprocess.run(["curl", "-s", "-F", f"file=@{WAV}", "-F", f"language={lang}",
+ "-F", "response_format=verbose_json", WHISPER + "/inference"],
+ capture_output=True, text=True).stdout
+ d = json.loads(out)
+ return d.get("language_probabilities", {}), " ".join(d["text"].split())
+
+
+def stt():
+ # large-v3-turbo's detector is reliable for this speaker (small's was
+ # not). Only it and en are spoken: if detection lands elsewhere, or the
+ # text is non-Latin (a hallucination, both are Latin script), redo the
+ # pass forced to the likelier of the two.
+ p, text = transcribe()
+ lang = max(KLANG, key=lambda l: p.get(l, 0))
+ if max(p, key=p.get, default=lang) != lang or not latin(text):
+ text = transcribe(lang)[1]
+ return lang, text
+
+
+IT_WORDS = set("il lo la gli le di che è non per una sono ho hai con mi ti ci del della "
+ "come questo questa anche ma più se nel alla puoi posso sei".split())
+EN_WORDS = set("the is are you i to of and it that for with not this what be have "
+ "can your my do there".split())
+
+
+def lang_of(sentence, fallback):
+ # The voice must follow the reply's language, not the user's: the model
+ # may answer in the other one. ponytail: stopword count, a real language
+ # identifier if this misfires.
+ words = re.findall(r"[a-zàèéìòù]+", sentence.lower())
+ it = sum(w in IT_WORDS for w in words)
+ en = sum(w in EN_WORDS for w in words)
+ return "it" if it > en else "en" if en > it else fallback
+
+
+def split_sentences(buf):
+ parts = re.split(r"(?<=[.!?…])\s+|\n+", buf)
+ return [p.strip() for p in parts[:-1] if p.strip()], parts[-1]
+
+
+def llm_model():
+ # The router keeps one model resident; asking for another would swap it
+ # out under other clients, so use whatever is not unloaded.
+ with urllib.request.urlopen(LLM + "/models") as r:
+ models = json.load(r)["data"]
+ return next((m["id"] for m in models if m["status"]["value"] != "unloaded"),
+ models[0]["id"])
+
+
+def ask(messages):
+ body = {"model": llm_model(), "messages": messages, "stream": True,
+ "chat_template_kwargs": {"enable_thinking": False}}
+ req = urllib.request.Request(LLM + "/chat/completions", json.dumps(body).encode(),
+ {"Content-Type": "application/json"})
+ with urllib.request.urlopen(req) as r:
+ for line in r:
+ if not line.startswith(b"data: ") or line.strip() == b"data: [DONE]":
+ continue
+ choices = json.loads(line[6:])["choices"]
+ if choices and choices[0]["delta"].get("content"):
+ yield choices[0]["delta"]["content"]
+
+
+def speaker(q):
+ from kokoro_onnx import Kokoro, EspeakConfig
+ # The espeakng-loader wheel's bundled lib has a broken data path; use
+ # the system espeak-ng.
+ k = Kokoro(f"{MODELS}/kokoro/kokoro-v1.0.onnx", f"{MODELS}/kokoro/voices-v1.0.bin",
+ espeak_config=EspeakConfig(lib_path="/usr/lib64/libespeak-ng.so.1",
+ data_path="/usr/share/espeak-ng-data"))
+ voices = {"it": 0.8 * k.get_voice_style("if_sara") + 0.2 * k.get_voice_style("af_bella"),
+ "en": k.get_voice_style("af_heart")}
+ play = None
+ while True:
+ item = q.get()
+ if item is None: # end of reply
+ if play:
+ play.stdin.close()
+ play.wait()
+ play = None
+ continue
+ text, lang = item
+ audio, sr = k.create(text, voice=voices[lang], lang=KLANG[lang])
+ if play is None:
+ play = subprocess.Popen(["aplay", "-q", "-r", str(sr), "-f", "S16_LE", "-c", "1"],
+ stdin=subprocess.PIPE)
+ # A new stream wakes the suspended sink and the first ~0.2 s is
+ # lost: lead with silence so it eats that, not the first word.
+ play.stdin.write(bytes(int(sr * 0.3) * 2))
+ # The pipe buffer lets synthesis of the next sentence overlap playback.
+ play.stdin.write((audio.clip(-1, 1) * 32767).astype("<i2").tobytes())
+
+
+def die_with_parent():
+ # Runs in the child before exec: the kernel sends SIGTERM when we die,
+ # however we die, so no whisper-server or amixer is ever orphaned.
+ ctypes.CDLL("libc.so.6").prctl(1, signal.SIGTERM) # PR_SET_PDEATHSIG
+
+
+def start_whisper():
+ def up():
+ try:
+ urllib.request.urlopen(WHISPER, timeout=1)
+ return True
+ except OSError:
+ return False
+ if up():
+ return
+ p = subprocess.Popen(["whisper-server", "-m", f"{MODELS}/whisper/ggml-large-v3-turbo-q5_0.bin",
+ "-dev", "0", "--host", "127.0.0.1", "--port", WHISPER.rsplit(":", 1)[1]],
+ stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
+ preexec_fn=die_with_parent)
+ for _ in range(60):
+ if up():
+ return
+ time.sleep(0.5)
+ p.kill()
+ sys.exit("whisper-server did not come up")
+
+
+def reply(history, q):
+ pcm = open(RAW, "rb").read()
+ if len(pcm) < 16000 * 2 * 0.3: # under 0.3 s: a stray press
+ return
+ # Trailing silence: whisper drops the last word when audio ends mid-word,
+ # which it does when the button is pressed while still talking.
+ with wave.open(WAV, "wb") as w:
+ w.setnchannels(1)
+ w.setsampwidth(2)
+ w.setframerate(16000)
+ w.writeframes(pcm + bytes(16000 * 2 // 2))
+ t = time.perf_counter()
+ lang, text = stt()
+ if not text or text.startswith("["): # [BLANK_AUDIO] and friends
+ return
+ log(f"you ({lang}, {time.perf_counter() - t:.2f}s): {text}")
+ history.append({"role": "user", "content": text})
+ t, said, buf = time.perf_counter(), [], ""
+ for delta in ask([{"role": "system", "content": SYSTEM}] + history[-20:]):
+ # ponytail: last 20 messages, summarise if long sessions need memory
+ buf += delta
+ done, buf = split_sentences(buf)
+ if not said and done:
+ log(f"first sentence after {time.perf_counter() - t:.2f}s")
+ for s in done:
+ said.append(s)
+ lang = lang_of(s, lang)
+ q.put((re.sub(r"[*_#`]", "", s), lang))
+ if buf.strip():
+ if not said:
+ log(f"first sentence after {time.perf_counter() - t:.2f}s")
+ said.append(buf.strip())
+ q.put((re.sub(r"[*_#`]", "", buf.strip()), lang_of(buf, lang)))
+ q.put(None)
+ history.append({"role": "assistant", "content": " ".join(said)})
+ log("assistant:", " ".join(said))
+
+
+def main():
+ # SIGTERM must run the cleanup below too, or the recording is left behind.
+ signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
+ start_whisper()
+ q = queue.Queue()
+ threading.Thread(target=speaker, args=(q,), daemon=True).start()
+ history, rec = [], None
+ events = subprocess.Popen(["stdbuf", "-oL", "amixer", "-c", CARD, "events"],
+ stdout=subprocess.PIPE, text=True, preexec_fn=die_with_parent)
+ prev = mic_on()
+ log(f"ready, mic is {'on' if prev else 'off'}")
+ try:
+ for line in events.stdout:
+ if "Mic Capture Switch" not in line:
+ continue
+ on = mic_on()
+ if on == prev:
+ continue
+ prev = on
+ if on and presenting():
+ log("presentation mode: ignored")
+ elif on:
+ # ponytail: no barge-in, a new press does not cut the reply
+ rec = subprocess.Popen(["arecord", "-q", "-t", "raw", "-f", "S16_LE", "-r", "16000",
+ "-c", "1", RAW], stderr=subprocess.DEVNULL)
+ elif rec:
+ rec.send_signal(signal.SIGINT)
+ rec.wait()
+ rec = None
+ reply(history, q)
+ except KeyboardInterrupt:
+ pass
+ finally:
+ for f in (RAW, WAV): # the user's voice: never keep it
+ if os.path.exists(f):
+ os.remove(f)
+
+
+def test():
+ assert split_sentences("Ciao! Come stai? Io") == (["Ciao!", "Come stai?"], "Io")
+ assert split_sentences("Pi is 3.14 today") == ([], "Pi is 3.14 today")
+ assert split_sentences("one\n\ntwo") == (["one"], "two")
+ assert latin("Perché no? Hi!") and not latin("早, 早!")
+ assert lang_of("Non posso fornire il meteo in tempo reale.", "en") == "it"
+ assert lang_of("That is a very exciting place to be.", "it") == "en"
+ assert lang_of("Tokyo!", "it") == "it"
+ print("ok")
+
+
+if __name__ == "__main__":
+ test() if sys.argv[1:] == ["--test"] else main()