# Tools and config folder: design Date: 2026-09-26. Status: approved in brainstorming, not implemented. Phase 1 of 2: read-only tools (see "Phasing"). ## Goal Give the assistant tools (date/time, calendar, weather, web search) so it stops guessing facts, and make it customizable: what it can do and how is decided by files in a config folder, not by editing `assistant.py`. ## Verified before designing - llama-server runs with `--jinja`; Gemma4-12B emits OpenAI-style `tool_calls` in the stream. Probe: "Che ore sono?" called `now()` in 0.63 s, "What's the weather like in Rome?" called `weather` with `{"place": "Rome"}` in 0.4 s, arguments streamed in fragments that must be concatenated per `index`. A general question called no tool. - The calendar is a vdirsyncer vdir: plain `.ics` files, one directory per calendar. Recurring events are mostly yearly; most `RRULE` lines in the files belong to `VTIMEZONE` blocks, not events. - `python3-dateutil` 2.9.0 is installed as its own package. - The SearXNG instance answers `/search?q=...&format=json`. - Weather in the sibling conky-theme-udt project comes from OpenWeatherMap: key and home city in an env file outside any repo, current conditions cached as JSON every 15 minutes. ## Config folder `${XDG_CONFIG_HOME:-~/.config}/desktop-assistant/`, read once at startup with stdlib `tomllib`. ``` desktop-assistant/ config.toml optional, every key optional tools/ .toml one tool each; the file name is the tool name scripts the tools call, if any ``` A missing folder, missing `config.toml` or empty `tools/` is valid: the assistant then behaves exactly as it does today, with no tools. A tool is enabled by its file being present; there is no enable flag. ### config.toml Replaces the constants at the top of `assistant.py`. The current values become the built-in defaults, merged key by key under whatever the file sets. ```toml card = "Microphone" # ALSA card id of the mic llm = "http://localhost:8181/v1" system = "You are a voice assistant..." tool_rounds = 3 # max tool round trips per question [whisper] model = "/data/voice-models/whisper/ggml-large-v3-turbo-q5_0.bin" device = 0 port = 8182 [kokoro] model = "/data/voice-models/kokoro/kokoro-v1.0.onnx" voices = "/data/voice-models/kokoro/voices-v1.0.bin" [voice.it] # style name = weight, blended lang = "it" blend = { if_sara = 0.8, af_bella = 0.2 } [voice.en] lang = "en-us" blend = { af_heart = 1.0 } [env] # added to every tool's environment # SEARXNG_URL = "http://searxng.example.org" # WEATHER_ENV = "/home/you/.config/udt/weather.env" ``` `[env]` is how personal values (instance URLs, key file paths) reach the tool scripts without appearing in the repo or in tool files. An unknown key is an error at startup, printed with its name: a typo must not silently fall back to a default. ### Tool file ```toml # tools/weather.toml description = "Current weather and short forecast. Leave place empty for home." command = ["weather.sh", "{place}", "{when}"] timeout = 10 announce.it = ["Controllo il meteo."] announce.en = ["Checking the weather."] [params.place] type = "string" description = "City name, empty for home" [params.when] type = "string" description = "now, today or tomorrow" ``` Top-level keys must come before the `[params.*]` tables (TOML assigns keys after a table header to that table). - `description`, `command`: required. `timeout` (seconds): default 10. - `params`: optional; each has `type` (`string`, `integer`, `number`, `boolean`) and `description`. Sent to the model as the JSON schema of the function. No `required` list: every parameter is optional, missing ones become empty strings. - `announce.`: optional list of phrases, see below. ## Running a tool - `command` is an argv list, run without a shell. Each `{name}` in an element is replaced by the argument's string value. A value can therefore never become a second argument or a shell command; a test covers a value like `x; rm -rf ~`. - `command[0]` without a slash is looked up in `tools/` first, then `PATH`. - Environment: the assistant's own, plus `[env]`. - stdout is returned to the model, stripped and cut to 2000 characters. - A non-zero exit, a timeout or a failure to start returns `error: ` (with stderr's last line when there is one) as the tool result. The model then says it could not find out; the assistant never crashes on a tool. - An unknown tool name or arguments that are not valid JSON also return `error: ...`. ## The loop `ask()` sends `tools` with every request when any are configured. While streaming: - `delta.content` is spoken as now, sentence by sentence. - `delta.tool_calls` fragments are collected per `index`: `id` and `function.name` from the first fragment, `function.arguments` concatenated. When the stream ends with tool calls, the assistant message with its `tool_calls` and one `{"role": "tool", "tool_call_id": ..., "content": ...}` per call are appended to the turn's messages, and the request is repeated. After `tool_rounds` rounds the request is sent without `tools`, which forces a text answer. History across turns keeps only the user text and the final spoken reply, as now. Tool calls and results live only inside the turn, which keeps the 20-message window for conversation. ## Announcements When a round has tool calls, before running them, the assistant speaks one phrase from the first called tool that has an `announce` entry for the language whisper detected for the question. The phrase is picked with `random.choice` so it does not repeat word for word. At most one announcement per question, even across several rounds. Tools without `announce` are silent. Rejected: speaking only if a tool runs past ~1 s. It needs a timer thread and the phrase would land at an arbitrary point. ## Example tools Shipped in the repo under `examples/`, copied by the user into the config folder. They contain no personal values; those come from `[env]`. | Tool | Command | Network | Announce | |------|---------|---------|----------| | `now` | `date "+%A %d %B %Y, %H:%M"` | none | silent | | `calendar` | `calendar.py {start} {days}` | none | silent | | `weather` | `weather.sh {place} {when}` | OpenWeatherMap, except home "now" | yes | | `search` | `search.sh {query}` | SearXNG | yes | ### calendar.py Reads the vdir named by `CALENDAR_DIR` (default `~/.local/share/calendars`). `start` is an ISO date (empty = today), `days` a count (empty = 1). Prints one line per event in range, sorted: `YYYY-MM-DD HH:MM summary` or `YYYY-MM-DD all day summary`, or `no events`. Parsing is stdlib: unfold continuation lines, take each `VEVENT` block (skipping `VTIMEZONE`), read `DTSTART`, `SUMMARY`, `RRULE`, `EXDATE`. Recurrence goes through `dateutil.rrule.rrulestr` on the event's own `DTSTART`/`RRULE`/`EXDATE` lines with `forceset=True`, so every RRULE form and exception date is handled by dateutil. Times with `TZID` are converted to local time. `RECURRENCE-ID` overrides (a moved single occurrence): the override replaces that occurrence. Its own runnable check: `calendar.py --test` against a generated temp vdir with a one-off, a yearly and an EXDATE'd event. ### weather.sh Sources the env file named by `WEATHER_ENV` (same format as conky-theme-udt's: `KEY`, `CITY`, `COUNTRY`, `UNITS`). - Empty `place` and `when` = now: reads conky-theme-udt's cache (`${XDG_CACHE_HOME:-~/.cache}/udt/weather.json`) if younger than 30 minutes, no network. Otherwise fetches `/data/2.5/weather`. - Another place, or `when` = today/tomorrow: `/data/2.5/weather` or the free `/data/2.5/forecast` (3-hourly, filtered to the day). - Prints a short plain-text summary (place, conditions, temperature, min/max, wind) for the model to phrase. - Uses `curl` and `jq`, both already used by conky-theme-udt. ### search.sh `curl` SearXNG `/search?q=&format=json` at `SEARXNG_URL`, `jq` to the top 5 results as `title: snippet`. The model answers from those. ## Testing `assistant.py --test` gains: - argv substitution, including an injection-shaped value that must stay one argument; - a tool that times out and one that exits non-zero both yield `error: ...`; - assembling streamed `tool_calls` fragments into complete calls; - config merge: defaults survive a partial file, an unknown key fails. Then a live run: time, a calendar question, home and remote weather, a search, and a plain question that must call nothing. ## Phasing This spec is **phase 1: read-only tools only**. Its purpose is to prove the mechanism (config folder, tool files, the loop, announcements) on tools that cannot do damage when the model calls them wrongly or with bad arguments. Every example tool only reads. **Phase 2**, a separate spec after phase 1 is tested live: tools that write, edit or otherwise change something (adding calendar events, sending messages, controlling the desktop). That spec decides what such tools need that read-only ones do not, for example a spoken confirmation before running, a marker in the tool file that says a tool modifies state, and what happens on a misheard command. Nothing in phase 1 builds toward those ahead of time. ## Out of scope for phase 1 Modifying tools (phase 2), contacts, barge-in. The loop does not stop a user from putting a modifying command in a tool file: the config folder is trusted like any user config, but phase 1 neither ships nor tests such a tool.