From 7f7bc1b00f991c1cba49414edd4377c44e97f571 Mon Sep 17 00:00:00 2001 From: "Danilo M." Date: Fri, 23 Oct 2026 16:55:24 +0200 Subject: cloud: one ctx event per request; docs: live check with Hetzner Hetzner sends usage on every chunk, which made the daemon emit a ctx event per token. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) (limited to 'AGENTS.md') diff --git a/AGENTS.md b/AGENTS.md index d501fad..a7cbc46 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -274,6 +274,34 @@ must not decide alone (unknown host, already cloned) are asked. Spec: - A tool added to `~/.config/desktop-assistant/tools/` needs a daemon restart: `TOOLS` is loaded once at import, not per request. +### Cloud models, live check (2026-10-23) + +Hetzner's inference API (`https://inference.hetzner.com/api/v1`, +`key_cmd = "pass hetzner.de/llamachat"`), model `Qwen3.8-27B`, through a +second daemon on a scratch socket, config and data dir; the real config +and history were untouched. + +- `--cloud-models hetzner`: 4 ids in 1.2 s, including the pass prompt. +- `models` listed the router's two and both configured cloud ones. +- "Che tempo fa domani a Roma?": `weather` with `Rome,IT`, first token + 4.6 s, ctx 775 then 953, `max` 0. +- "Quante email non lette ho?" in the same chat: no tool offered, it said + it has no way to read mail. Nothing personal went out. +- Voice clip (`05.wav`) in the cloud chat: transcribed by the local + Gemma (17 s: it was sleeping and reloaded), answered by the cloud with + `weather` for home. +- Same chat switched to Gemma: `ctx` max back to 32768, all tools + offered, mail answered (173 messages in 141 threads). Back to the + cloud: `weather` for Milan, 3.4 s; the mail result had gone back as + the history placeholder only. `open` reported the stored cloud model. +- **Hetzner sends `usage` on every chunk**, not once at the end: the + daemon sent ~60 `ctx` events per reply. Now one per request. +- Titles from the cloud took ~12 s: `enable_thinking` is not sent to + providers, so Qwen reasons over the title too. Harmless, after done. +- Qwen's replies start with a blank line or two after its reasoning; + Markdown drops them. +- The key appeared in no log line. + ### Text daemon, live check (2026-10-16) Model Gemma4-12B-qat-mtp, ctx 32768, preset reasoning-budget 4096. -- cgit v1.2.3