diff options
| author | Danilo M. <danix@danix.xyz> | 2026-10-23 16:55:24 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-10-23 16:55:24 +0200 |
| commit | 7f7bc1b00f991c1cba49414edd4377c44e97f571 (patch) | |
| tree | deb918b3d862ab9d27033bf00fdcf8216ffd7ed2 /AGENTS.md | |
| parent | ba07a1df91cd64aa1ef22a166bba6088daf67b37 (diff) | |
| download | desktop-assistant-7f7bc1b00f991c1cba49414edd4377c44e97f571.tar.gz desktop-assistant-7f7bc1b00f991c1cba49414edd4377c44e97f571.zip | |
cloud: one ctx event per request; docs: live check with Hetzner
Hetzner sends usage on every chunk, which made the daemon emit a ctx
event per token.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'AGENTS.md')
| -rw-r--r-- | AGENTS.md | 28 |
1 files changed, 28 insertions, 0 deletions
@@ -274,6 +274,34 @@ must not decide alone (unknown host, already cloned) are asked. Spec: - A tool added to `~/.config/desktop-assistant/tools/` needs a daemon restart: `TOOLS` is loaded once at import, not per request. +### Cloud models, live check (2026-10-23) + +Hetzner's inference API (`https://inference.hetzner.com/api/v1`, +`key_cmd = "pass hetzner.de/llamachat"`), model `Qwen3.8-27B`, through a +second daemon on a scratch socket, config and data dir; the real config +and history were untouched. + +- `--cloud-models hetzner`: 4 ids in 1.2 s, including the pass prompt. +- `models` listed the router's two and both configured cloud ones. +- "Che tempo fa domani a Roma?": `weather` with `Rome,IT`, first token + 4.6 s, ctx 775 then 953, `max` 0. +- "Quante email non lette ho?" in the same chat: no tool offered, it said + it has no way to read mail. Nothing personal went out. +- Voice clip (`05.wav`) in the cloud chat: transcribed by the local + Gemma (17 s: it was sleeping and reloaded), answered by the cloud with + `weather` for home. +- Same chat switched to Gemma: `ctx` max back to 32768, all tools + offered, mail answered (173 messages in 141 threads). Back to the + cloud: `weather` for Milan, 3.4 s; the mail result had gone back as + the history placeholder only. `open` reported the stored cloud model. +- **Hetzner sends `usage` on every chunk**, not once at the end: the + daemon sent ~60 `ctx` events per reply. Now one per request. +- Titles from the cloud took ~12 s: `enable_thinking` is not sent to + providers, so Qwen reasons over the title too. Harmless, after done. +- Qwen's replies start with a blank line or two after its reasoning; + Markdown drops them. +- The key appeared in no log line. + ### Text daemon, live check (2026-10-16) Model Gemma4-12B-qat-mtp, ctx 32768, preset reasoning-budget 4096. |
