1 files changed, 152 insertions, 0 deletions
diff --git a/docs/superpowers/specs/2026-10-23-cloud-models-design.md b/docs/superpowers/specs/2026-10-23-cloud-models-design.md
new file mode 100644
index 0000000..5747c22
--- /dev/null
+++ b/docs/superpowers/specs/2026-10-23-cloud-models-design.md
@@ -0,0 +1,152 @@
+# Cloud models: design
+
+Date: 2026-10-23. Status: approved, not implemented.
+
+## Goal
+
+Let a chat use a cloud model instead of the one llama-server has loaded.
+Providers and their models are set up in the quickshell settings window;
+the canvas header's model name becomes a dropdown that picks the model
+for that chat. The preferred model may be a cloud one.
+
+This changes the project's rule "Everything stays local: no cloud
+STT/TTS/LLM fallbacks". The new rule: no silent cloud fallback, ever. A
+cloud model is used only when the user picks it for a chat; tools opt in
+to cloud chats one by one; audio never leaves the machine.
+
+## What leaves the machine in a cloud chat
+
+The provider receives the system prompt, the date stamp, the history
+window (10 turns, tool calls with placeholder results), the question,
+attachments, and the results of the tools that opted in. That is the
+privacy cost, and the reason tools default to staying out.
+
+## Config
+
+One `[[cloud]]` table per provider in `config.toml`:
+
+```toml
+[[cloud]]
+name    = "openrouter"
+url     = "https://openrouter.ai/api/v1"
+key_cmd = "pass show llm/openrouter"
+models  = ["mistralai/mistral-large", "deepseek/deepseek-chat"]
+```
+
+- Providers are OpenAI-compatible only: `GET {url}/models` and
+  `POST {url}/chat/completions`, `Authorization: Bearer <key>`. That
+  covers OpenRouter, Mistral, Groq, DeepSeek, OpenAI, and the
+  OpenAI-compatible endpoints of Gemini and Anthropic.
+- `key_cmd` is an argv string (split with `shlex`, run without a shell);
+  its stdout, stripped, is the key. Run once per provider at startup. The
+  key is never logged, never sent on the socket, never written by the
+  settings page.
+- `name` must be unique and contain no `:`. `merge()` today handles
+  tables, not arrays of tables: `cloud` is validated on its own (list of
+  tables, exactly the four keys, all strings except `models`, a list of
+  strings). A typo fails loudly, as for every other key.
+- `key_cmd` failing (non-zero exit, empty output, 10 s timeout) drops
+  that provider: one log line, its models absent from `models`. The
+  daemon still starts.
+
+## Model ids
+
+`"<provider>:<model>"` for a cloud model, the bare router id for a local
+one. A prefix counts only when it names a configured provider, so a local
+id containing `:` stays local. Split on the first `:` only: cloud model
+ids contain `/` and sometimes `:` (OpenRouter's `:free`).
+
+## Daemon
+
+- **Per-chat model.** `conversations` gets a `model TEXT` column; an
+  existing database gains it with `ALTER TABLE` when missing. `ask` takes
+  an optional `model`; the first ask of a chat stores it, a different one
+  updates it. NULL (old chats, asks without `model`) means today's
+  behaviour: whatever the router has loaded. Opening a chat reports its
+  model (`open` answers with it) so the canvas can set the dropdown.
+- **Local model.** A chosen local id is sent to the router as is, and the
+  router swaps it in if another is loaded. The old rule "the daemon never
+  asks for a model" becomes "only for a model the user picked for this
+  chat". `ctx` still carries the model actually answering.
+- **Cloud model.** Same request body minus `chat_template_kwargs` (strict
+  providers reject unknown fields). No `/props`: `ctx` max is 0 and the
+  canvas shows used tokens only, from `usage`. Reasoning deltas are read
+  from `reasoning_content` (llama-server) or `reasoning` (OpenRouter).
+  `temperature` from config applies when set. Vision and audio are
+  unknown for cloud models (`/models` does not report them), so
+  attachments are sent and a provider refusal comes back as `error`.
+- **Tools.** A tool file may set `cloud = true`. A cloud chat is offered
+  only those; the default is false. A local chat is offered all of them,
+  as today. The example tools ship with the flag only where the output is
+  not personal (`calc`, `now`, `search`, `weather`); mail, finance,
+  calendar, playing, VMs, Bluetooth, updates, modes, clone and project
+  stay out.
+- **Voice.** Transcription always uses the local loaded model, whatever
+  the chat's model. The audio never leaves; only the transcript does, as
+  the text question. If the loaded model cannot hear, the ask is refused
+  as today.
+- **Titles.** Written by the chat's own model.
+- **`models` command.** Answers with the router's models (id, status)
+  plus every configured cloud model, each with `cloud: true|false` and,
+  for cloud, its provider name. Feeds the dropdown.
+- **`--cloud-models <name>`.** CLI mode: reads `config.toml`, runs that
+  provider's `key_cmd`, prints the provider's `/models` ids as JSON, one
+  array, sorted, and exits. The settings page runs it to fetch the list,
+  so the key stays in a short-lived daemon process and never reaches QML.
+- **Errors.** Provider HTTP error: `error` event with status and body,
+  capped at 2000 characters like tool errors, nothing written to history.
+  Timeouts as for the router.
+
+## Settings page (quickshell, `desktop/modules/assistant/`)
+
+A "Cloud" section under "Model": one card per provider with name, URL,
+key command, and Fetch. Fetch saves the draft, runs `assistant.py
+--cloud-models <name>`, and shows the ids as a checklist; the checked
+ones become `models`. Add and remove provider. Saved through the existing
+line-by-line `Config.js` path, still parsed with `tomllib` before the
+atomic replace. Restart offered as for every config change.
+
+Prefer extends to cloud models: `Prefs.assistantModel` holds the
+prefixed id. Opening the canvas with a cloud preference loads nothing
+locally.
+
+## Canvas header (quickshell, `desktop/assistant/`)
+
+The model name becomes a dropdown from the `models` command: local
+models, then cloud models grouped by provider, cloud ones marked with an
+icon. A new chat starts on the preferred model; a reopened chat on its
+stored one. Picking a model applies from the next ask.
+
+Switching a chat that already has turns from local to cloud sends its
+whole history window to the provider on the next ask, so the first such
+switch in a chat shows one line: "History in this chat goes to
+<provider>."
+
+## Testing
+
+`--test` gains a fake provider: a local `http.server` stub on a random
+port, `key_cmd` an `echo`. It checks: prefix routing (cloud id to the
+stub with the Bearer key, local id to the router stub), a local id with
+`:` stays local, `cloud = false` tools absent from the cloud request,
+no `chat_template_kwargs` in it, the `model` column stored and reported
+on reopen, the `ALTER TABLE` on an old database, a failing `key_cmd`
+dropping its provider, and the key absent from every socket event and
+log line.
+
+Live check after: one cloud chat with a tool-using question (`weather`),
+one asking for mail (must answer that it has no tool for it), a voice
+question in a cloud chat (transcribed locally), and switching a chat
+local to cloud and back.
+
+## Docs
+
+AGENTS.md: the opening line and the "Everything stays local" agent rule
+rewritten to the rule in Goal; a findings section after the live check.
+README: config example for `[[cloud]]` and the `cloud = true` tool flag.
+
+## Scope
+
+Two repos, one plan each: the daemon here first (config, routing, tool
+filter, `models`, `--cloud-models`, DB column), then quickshell (settings
+section, dropdown). Out: non-OpenAI-compatible APIs, per-model
+capability detection, cost tracking, cloud STT/TTS.