diff options
| author | Danilo M. <danix@danix.xyz> | 2026-10-23 17:28:27 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-10-23 17:28:27 +0200 |
| commit | 985d538a5fdb1b58d35621687313ed7d74d80288 (patch) | |
| tree | b778a8836a8cfe59ffe6775bda289808953c7a40 /docs | |
| parent | 83e1d30a4208c0e169a360fd6d6ce7124aadcc2e (diff) | |
| download | desktop-assistant-985d538a5fdb1b58d35621687313ed7d74d80288.tar.gz desktop-assistant-985d538a5fdb1b58d35621687313ed7d74d80288.zip | |
cloud: the context size from the provider's /models
Hetzner runs vLLM, whose /models reports max_model_len per model
(262144); OpenRouter names it context_length. Read when the chat's model
changes, as /props is for a local one, so the meter shows a cloud chat's
fill and its attachment budget is half of what is free instead of a
fixed 24000 characters. Image input already went through: a cloud
model's inputs count as unknown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/superpowers/specs/2026-10-23-cloud-models-design.md | 7 |
1 files changed, 5 insertions, 2 deletions
diff --git a/docs/superpowers/specs/2026-10-23-cloud-models-design.md b/docs/superpowers/specs/2026-10-23-cloud-models-design.md index 8a581ee..d2270af 100644 --- a/docs/superpowers/specs/2026-10-23-cloud-models-design.md +++ b/docs/superpowers/specs/2026-10-23-cloud-models-design.md @@ -69,8 +69,11 @@ ids contain `/` and sometimes `:` (OpenRouter's `:free`). asks for a model" becomes "only for a model the user picked for this chat". `ctx` still carries the model actually answering. - **Cloud model.** Same request body minus `chat_template_kwargs` (strict - providers reject unknown fields). No `/props`: `ctx` max is 0 and the - canvas shows used tokens only, from `usage`. Reasoning deltas are read + providers reject unknown fields). No `/props`: `ctx` max comes from the + provider's `/models`, `max_model_len` (vLLM, Hetzner) or + `context_length` (OpenRouter), read when the chat's model changes; 0 + when neither is there, and the canvas then shows used tokens only. A + chat's attachment budget uses the same size. Reasoning deltas are read from `reasoning_content` (llama-server) or `reasoning` (OpenRouter). `temperature` from config applies when set. Vision and audio are unknown for cloud models (`/models` does not report them), so |