Document the config folder, the tools and the live test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

1 files changed, 38 insertions, 1 deletions

diff --git a/AGENTS.md b/AGENTS.md
index bc01b7e..51a46d3 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -9,6 +9,13 @@ on/off is push-to-talk, whisper large-v3-turbo on the Arc transcribes,
 the resident llama-server model answers, Kokoro speaks it. Button-off to
 first spoken word ~2 s. See "Findings" and "Next steps".
 
+Tools (2026-09-26, phase 1 of 2): the model can call read-only tools
+(time, calendar, weather, search) defined as shell-command TOML files in
+`~/.config/desktop-assistant/tools/`; `config.toml` there overrides the
+built-in settings. Examples in `examples/`. STT, LLM and TTS stay local;
+only the weather and search tools reach the network, and only when a
+question needs them. Spec: `docs/superpowers/specs/2026-09-26-tools-and-config-design.md`.
+
 Run: `python3 assistant.py` (`--test` for the self-check).
 
 ---
@@ -295,6 +302,35 @@ drives `arecord` on/off, clip POSTed to `/inference`.
   stream and the idle sink needs a moment to wake. 0.3 s of leading silence
   per reply fixed it (confirmed by ear).
 
+### Tools, live test (2026-09-26)
+
+Gemma4-12B through llama-server `--jinja`, streamed OpenAI `tool_calls`.
+Eight spoken questions (time, calendar, home weather, remote forecast,
+distance, a person, next appointment twice).
+
+- Tool choice and arguments were right every time. Weather for a small
+  town came with the country code the parameter description asks for.
+- Bare city names are ambiguous for OpenWeatherMap: "Rome" and "Roma"
+  both resolve to Rome, US. The place parameter asks for `City,CC` and
+  the output names the country.
+- **The model has no clock.** Asked for the next appointment it called the
+  calendar with a 2025 start date, then recovered by calling `now` (three
+  rounds, 4.0 s). With the date and time in the system prompt every turn:
+  one round, 2.0 s.
+- **It gets weekdays wrong from dates** ("mercoledi 29" for a Tuesday).
+  `agenda.py` prints the weekday itself; right since.
+- It searches for general knowledge too (a question about a well-known
+  person), because the search description says "anything you are not
+  sure about". Costs ~3 s (first sentence 6.7 s). **Kept on purpose** by
+  the user: facts over speed.
+- First sentence with one tool: 1.9-2.2 s for local tools, ~4 s with a
+  search, 6.7 s for a search whose results the model read at length.
+- Announcements heard on every slow tool, no audio glitch while the
+  stream waits for the tool. Straight into the answer it sounded rushed;
+  a 0.6 s pause after it fixed that (by ear).
+- `agenda.py` matched khal's listing over 7 and 60 days on the real vdir,
+  runs in ~60 ms.
+
 ### TTS (2026-09-25)
 
 `piper-tts` on CPU (onnxruntime), ~5 s sentence per voice:
@@ -357,7 +393,8 @@ no particular order:
   reads the stored flag; confirm it is 1 while a game holds it on.
 - **Latency:** STT ~1.2 s is the biggest share; the double encoder pass
   (detect + transcribe) is the target if it matters.
-- **Tools** (time, weather, calendar) so the model stops guessing facts.
+- **Phase 2 tools:** tools that write or modify (events, messages,
+  desktop), own spec, after phase 1 has been used for a while.
 - OpenVINO in the whisper.cpp package: runs on CPU only here (no Intel
   compute-runtime), decision goes to `my-slackbuilds`.