aboutsummaryrefslogtreecommitdiffstats
path: root/README.md
diff options
context:
space:
mode:
authorDanilo M. <danix@danix.xyz>2026-08-21 20:58:39 +0200
committerDanilo M. <danix@danix.xyz>2026-08-21 20:58:39 +0200
commita7422ddeb7771983e984350b31092fe4898897c6 (patch)
tree2833a9c8f5c2ab646ceee5f42b711b7d0c35ec47 /README.md
parent2cae7f19ac90e6df9d4008e6e364bae30e8b389a (diff)
downloadllamachat-feature/external-providers.tar.gz
llamachat-feature/external-providers.zip
fix: web search on DeepSeek and other reasoning modelsfeature/external-providers
A searched turn on a cloud reasoning model ended at the thinking: the model emitted the tool call, but finish_reason: "tool_calls" landed on the same SSE line as the include_usage block, so the single-event parser returned that line as a usage chunk and the loop never saw the tool finish. The parser now emits every event a line carries, so the search fires. Also in this change: - Replay each round's reasoning_content on the assistant tool-call message, which interleaved-thinking models require to keep going. - Add a per-provider replay_reasoning option (DeepSeek, SiliconFlow GLM-4.7+) to carry prior turns' reasoning_content when search is on. - Add an on-demand diagnostic log gated by $LLAMACHAT_DEBUG_LOG. - Record provider, usage_json and reported_cost_usd per reply, so a searched turn keeps every round's billed usage for external consumers.
Diffstat (limited to 'README.md')
-rw-r--r--README.md28
1 files changed, 27 insertions, 1 deletions
diff --git a/README.md b/README.md
index 5df70d6..416a84c 100644
--- a/README.md
+++ b/README.md
@@ -288,7 +288,9 @@ there is no second thread.
```sql
sessions (id, mode, title, model, prompt_name, prompt_custom,
created_at, updated_at)
-messages (id, session_id, role, content, reasoning, searches, created_at)
+messages (id, session_id, role, content, reasoning, searches,
+ prompt_tokens, completion_tokens, model, provider, usage_json,
+ reported_cost_usd, created_at)
attachments (id, message_id, path, kind, mime, size, sha256, thumb, truncated)
messages_fts -- FTS5 external-content table over messages.content
```
@@ -307,6 +309,19 @@ per search with its query, results and any error, or NULL when nothing was
searched. Like `reasoning` it stays out of the FTS index, or snippets from
web pages would compete with messages the user actually wrote.
+`provider` records which provider served the reply (`local` or a configured
+provider name), written when the reply finishes so history stays
+self-describing even if `config.toml` later renames or drops the provider.
+`usage_json` keeps every round's raw usage block from the stream as a JSON
+array — a searched turn makes one API call per search round, and each round
+is billed separately, so all of them are kept verbatim and token details
+beyond the two counts (e.g. cached or reasoning tokens on a cloud endpoint)
+are recoverable later without re-guessing a field mapping.
+`reported_cost_usd` is the GUI's hand-entered-price estimate for that one
+reply (summed across rounds), stored as a sanity check for external
+consumers — it is approximate and never a source of truth; it stays NULL for
+a model with no entered prices rather than claiming the reply cost nothing.
+
`prompt_name` records which system prompt a conversation was built with, and
`prompt_custom` holds the text when that prompt is a one-off rather than a
file. Storing the name rather than the resolved text means editing a preset
@@ -463,6 +478,8 @@ filter = ["qwen", "deepseek"]
ctx_size = 32768
price_in = 0.60
price_out = 0.60
+# thinking_budget = 8192 # per-provider chain-of-thought cap
+# replay_reasoning = false # see below
```
`api_key` accepts three forms:
@@ -499,6 +516,15 @@ local router, but on a cloud provider every message is billed for the entire
conversation so far. The projection in the cost label exists to make that
visible before you send.
+`thinking_budget` sets a provider-specific cap on chain-of-thought tokens
+(currently sent to SiliconFlow; other endpoints ignore it when unset).
+
+`replay_reasoning` makes each assistant turn's thinking be sent back on the
+next request while web search is enabled. DeepSeek's and SiliconFlow's
+interleaved-thinking models require this when a `tools` key is present, and
+return a 400 if it is missing. Leave it off elsewhere — replaying thinking
+costs context and input tokens for providers that ignore it.
+
Tool calling and reasoning output vary between providers. Web search on a
cloud model may not work as reliably as it does with a local router.