aboutsummaryrefslogtreecommitdiffstats
path: root/README.md
diff options
context:
space:
mode:
authorDanilo M. <danix@danix.xyz>2026-08-10 21:02:01 +0200
committerDanilo M. <danix@danix.xyz>2026-08-10 21:02:01 +0200
commit3f984ccb8ec25ef3b06109d638eed4022db82d45 (patch)
tree9c1b5331ba85a91f7f6d4bb37ac701948639f7fc /README.md
parent1faadea207429ecfe1d4580fa4f6a4c1c40d0dd5 (diff)
downloadllamachat-3f984ccb8ec25ef3b06109d638eed4022db82d45.tar.gz
llamachat-3f984ccb8ec25ef3b06109d638eed4022db82d45.zip
docs: document external providers, keys and cost
The generated config ships the provider block commented out so a fresh install never reaches an endpoint nobody configured. The README states plainly that every turn resends the whole conversation, which is free locally and billed per message on a cloud provider.
Diffstat (limited to 'README.md')
-rw-r--r--README.md58
1 files changed, 57 insertions, 1 deletions
diff --git a/README.md b/README.md
index ada0d5a..5df70d6 100644
--- a/README.md
+++ b/README.md
@@ -43,7 +43,7 @@ persistent process and toggles like a scratchpad from a Hyprland keybind.
- **Tray icon** for show/hide/quit, hosted by waybar's tray module.
Not in this version: RAG or embedding search over history, multi-user
-support, any cloud or non-local backend, remote access.
+support, remote access.
## Requirements
@@ -446,6 +446,62 @@ None of that stops a poisoned snippet from influencing what the model says.
If a top result asserts something false, a 9B may repeat it. That is why the
sources stay visible rather than being hidden behind the answer.
+### External providers
+
+llamachat can talk to OpenAI-compatible cloud providers alongside the local
+router. Cloud models appear in the picker as `provider:model`. Local models
+stay bare, so an existing local setup needs no changes.
+
+Providers live in `[providers.*]` tables. The generated config ships a
+commented together.ai block; uncomment it and set `api_key` to enable it:
+
+```toml
+[providers.together]
+base_url = "https://api.together.xyz"
+api_key = "pass:api/together"
+filter = ["qwen", "deepseek"]
+ctx_size = 32768
+price_in = 0.60
+price_out = 0.60
+```
+
+`api_key` accepts three forms:
+
+- `pass:api/together` reads the key from the password store. This is
+ preferred, because the secret never sits in a config file that might be
+ committed or copied.
+- `env:TOGETHER_KEY` reads the key from the environment.
+- `sk-...` is the literal key, written into `config.toml` itself. Avoid
+ this: it puts a secret in a plain-text file.
+
+Keys are resolved lazily, on the first request to that provider. A session
+that only touches local models never triggers a pinentry or reads an
+environment variable.
+
+`filter` keeps only models whose id contains one of the given strings,
+case-insensitively. Providers list hundreds of models, and without a filter
+the picker is unusable. Omit `filter` to list everything.
+
+`ctx_size`, `vision`, `price_in` and `price_out` prefill the per-model
+settings dialog. Prices are US dollars per million tokens. The dialog saves
+anything you enter to `models.ini` beside `config.toml`, so the provider
+block can stay minimal. `models.ini` is machine-written cache, not
+user-editable config.
+
+The cost readout in the top bar shows what has been spent in the current
+conversation plus what the next message would add. It is approximate, based
+entirely on the hand-entered prices in the provider block or the per-model
+dialog. It is useful for seeing when a long conversation is becoming
+expensive.
+
+**A conversation resends its whole history every turn.** That is free with a
+local router, but on a cloud provider every message is billed for the entire
+conversation so far. The projection in the cost label exists to make that
+visible before you send.
+
+Tool calling and reasoning output vary between providers. Web search on a
+cloud model may not work as reliably as it does with a local router.
+
### The context meter
The bar in the top bar shows how much of the active model's context window