diff options
| author | Danilo M. <danix@danix.xyz> | 2026-10-23 17:28:27 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-10-23 17:28:27 +0200 |
| commit | 985d538a5fdb1b58d35621687313ed7d74d80288 (patch) | |
| tree | b778a8836a8cfe59ffe6775bda289808953c7a40 /AGENTS.md | |
| parent | 83e1d30a4208c0e169a360fd6d6ce7124aadcc2e (diff) | |
| download | desktop-assistant-985d538a5fdb1b58d35621687313ed7d74d80288.tar.gz desktop-assistant-985d538a5fdb1b58d35621687313ed7d74d80288.zip | |
cloud: the context size from the provider's /models
Hetzner runs vLLM, whose /models reports max_model_len per model
(262144); OpenRouter names it context_length. Read when the chat's model
changes, as /props is for a local one, so the meter shows a cloud chat's
fill and its attachment budget is half of what is free instead of a
fixed 24000 characters. Image input already went through: a cloud
model's inputs count as unknown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'AGENTS.md')
| -rw-r--r-- | AGENTS.md | 8 |
1 files changed, 7 insertions, 1 deletions
@@ -59,6 +59,11 @@ socket (ask) ──> llama-server (LLM, tools) ──> socket (think, token, too - **Search** quotes the query as one FTS5 phrase with a prefix star. The self-check proves typed FTS syntax (`"`, `*`, `NOT`, `OR`, `)`) is matched as text and raises nothing. +- **Usage** (2026-10-23): `stream()` keeps each request's last `usage` + chunk (Hetzner sends running totals on every chunk) and `answer()` saves + them per turn in `turns.usage`, JSON `{ts, provider, model, rounds}`, + `provider` the cloud name or `local`. Read by llm-cost-dashboard's + `ingest/assistant_ingest.py`. Titles and transcriptions are not counted. ## Hardware @@ -284,7 +289,8 @@ and history were untouched. - `--cloud-models hetzner`: 4 ids in 1.2 s, including the pass prompt. - `models` listed the router's two and both configured cloud ones. - "Che tempo fa domani a Roma?": `weather` with `Rome,IT`, first token - 4.6 s, ctx 775 then 953, `max` 0. + 4.6 s, ctx 775 then 953, `max` 0 at the time. Hetzner's `/models` + reports `max_model_len` (262144); the meter reads it since. - "Quante email non lette ho?" in the same chat: no tool offered, it said it has no way to read mail. Nothing personal went out. - Voice clip (`05.wav`) in the cloud chat: transcribed by the local |