diff options
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 58 |
1 files changed, 57 insertions, 1 deletions
@@ -43,7 +43,7 @@ persistent process and toggles like a scratchpad from a Hyprland keybind. - **Tray icon** for show/hide/quit, hosted by waybar's tray module. Not in this version: RAG or embedding search over history, multi-user -support, any cloud or non-local backend, remote access. +support, remote access. ## Requirements @@ -446,6 +446,62 @@ None of that stops a poisoned snippet from influencing what the model says. If a top result asserts something false, a 9B may repeat it. That is why the sources stay visible rather than being hidden behind the answer. +### External providers + +llamachat can talk to OpenAI-compatible cloud providers alongside the local +router. Cloud models appear in the picker as `provider:model`. Local models +stay bare, so an existing local setup needs no changes. + +Providers live in `[providers.*]` tables. The generated config ships a +commented together.ai block; uncomment it and set `api_key` to enable it: + +```toml +[providers.together] +base_url = "https://api.together.xyz" +api_key = "pass:api/together" +filter = ["qwen", "deepseek"] +ctx_size = 32768 +price_in = 0.60 +price_out = 0.60 +``` + +`api_key` accepts three forms: + +- `pass:api/together` reads the key from the password store. This is + preferred, because the secret never sits in a config file that might be + committed or copied. +- `env:TOGETHER_KEY` reads the key from the environment. +- `sk-...` is the literal key, written into `config.toml` itself. Avoid + this: it puts a secret in a plain-text file. + +Keys are resolved lazily, on the first request to that provider. A session +that only touches local models never triggers a pinentry or reads an +environment variable. + +`filter` keeps only models whose id contains one of the given strings, +case-insensitively. Providers list hundreds of models, and without a filter +the picker is unusable. Omit `filter` to list everything. + +`ctx_size`, `vision`, `price_in` and `price_out` prefill the per-model +settings dialog. Prices are US dollars per million tokens. The dialog saves +anything you enter to `models.ini` beside `config.toml`, so the provider +block can stay minimal. `models.ini` is machine-written cache, not +user-editable config. + +The cost readout in the top bar shows what has been spent in the current +conversation plus what the next message would add. It is approximate, based +entirely on the hand-entered prices in the provider block or the per-model +dialog. It is useful for seeing when a long conversation is becoming +expensive. + +**A conversation resends its whole history every turn.** That is free with a +local router, but on a cloud provider every message is billed for the entire +conversation so far. The projection in the cost label exists to make that +visible before you send. + +Tool calling and reasoning output vary between providers. Web search on a +cloud model may not work as reliably as it does with a local router. + ### The context meter The bar in the top bar shows how much of the active model's context window |