desktop-assistant

A local push-to-talk voice assistant. Unmute the microphone, speak, mute it: the assistant transcribes what you said, asks a local LLM, and speaks the answer. Italian and English. Nothing leaves the machine.

mic button ──> arecord ──> whisper-server (STT) ──> llama-server (LLM) ──> Kokoro (TTS) ──> aplay

How it works

Push-to-talk is the microphone's own mute button: the script listens to the ALSA Mic Capture Switch (amixer events). Switch on starts a recording, switch off ends it.
Speech to text: whisper.cpp's whisper-server with the large-v3-turbo model on the GPU (Vulkan). The script starts it if it is not already running and stops it on exit.
LLM: an OpenAI-compatible llama-server on localhost:8181. The script uses whichever model the server already has loaded, so it never swaps models under other clients.
Text to speech: Kokoro-82M through kokoro-onnx, one voice per language, chosen per sentence from the language of the reply. Speech starts after the first sentence, while the rest is still being generated.
Calls: while the desktop's presentation mode is on (statusctl presentation get returns 1), button presses are ignored, so a call can use the microphone.

Requirements

whisper.cpp (whisper-server) and a whisper ggml model
a running llama-server with an OpenAI-compatible API
Python 3 with onnxruntime, numpy and kokoro-onnx
espeak-ng, ALSA utils (amixer, arecord, aplay), curl
a USB microphone with a hardware mute button exposed as an ALSA switch

Paths, the card name and ports are constants at the top of assistant.py.

Usage

python3 -m venv --system-site-packages .venv
.venv/bin/pip install kokoro-onnx
.venv/bin/python assistant.py          # run
.venv/bin/python assistant.py --test   # self-check

License

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License version 2 as published by the Free Software Foundation. See LICENSE.

Development Approach

This project is developed using AI-assisted tools. Code is generated with the help of AI based on human-provided specifications, design decisions, and iterative feedback.

All contributions are reviewed, tested, and curated by the maintainer before being included in the codebase. AI is used as a productivity and exploration tool, while human oversight remains central to all decisions.

The goal is to combine the flexibility of AI-assisted development with standard open-source practices such as transparency, review, and accountability.