diff options
| author | Danilo M. <danix@danix.xyz> | 2026-08-21 20:58:39 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-08-21 20:58:39 +0200 |
| commit | a7422ddeb7771983e984350b31092fe4898897c6 (patch) | |
| tree | 2833a9c8f5c2ab646ceee5f42b711b7d0c35ec47 /README.md | |
| parent | 2cae7f19ac90e6df9d4008e6e364bae30e8b389a (diff) | |
| download | llamachat-a7422ddeb7771983e984350b31092fe4898897c6.tar.gz llamachat-a7422ddeb7771983e984350b31092fe4898897c6.zip | |
fix: web search on DeepSeek and other reasoning modelsfeature/external-providers
A searched turn on a cloud reasoning model ended at the thinking: the
model emitted the tool call, but finish_reason: "tool_calls" landed on
the same SSE line as the include_usage block, so the single-event parser
returned that line as a usage chunk and the loop never saw the tool
finish. The parser now emits every event a line carries, so the search
fires.
Also in this change:
- Replay each round's reasoning_content on the assistant tool-call
message, which interleaved-thinking models require to keep going.
- Add a per-provider replay_reasoning option (DeepSeek, SiliconFlow
GLM-4.7+) to carry prior turns' reasoning_content when search is on.
- Add an on-demand diagnostic log gated by $LLAMACHAT_DEBUG_LOG.
- Record provider, usage_json and reported_cost_usd per reply, so a
searched turn keeps every round's billed usage for external consumers.
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 28 |
1 files changed, 27 insertions, 1 deletions
@@ -288,7 +288,9 @@ there is no second thread. ```sql sessions (id, mode, title, model, prompt_name, prompt_custom, created_at, updated_at) -messages (id, session_id, role, content, reasoning, searches, created_at) +messages (id, session_id, role, content, reasoning, searches, + prompt_tokens, completion_tokens, model, provider, usage_json, + reported_cost_usd, created_at) attachments (id, message_id, path, kind, mime, size, sha256, thumb, truncated) messages_fts -- FTS5 external-content table over messages.content ``` @@ -307,6 +309,19 @@ per search with its query, results and any error, or NULL when nothing was searched. Like `reasoning` it stays out of the FTS index, or snippets from web pages would compete with messages the user actually wrote. +`provider` records which provider served the reply (`local` or a configured +provider name), written when the reply finishes so history stays +self-describing even if `config.toml` later renames or drops the provider. +`usage_json` keeps every round's raw usage block from the stream as a JSON +array — a searched turn makes one API call per search round, and each round +is billed separately, so all of them are kept verbatim and token details +beyond the two counts (e.g. cached or reasoning tokens on a cloud endpoint) +are recoverable later without re-guessing a field mapping. +`reported_cost_usd` is the GUI's hand-entered-price estimate for that one +reply (summed across rounds), stored as a sanity check for external +consumers — it is approximate and never a source of truth; it stays NULL for +a model with no entered prices rather than claiming the reply cost nothing. + `prompt_name` records which system prompt a conversation was built with, and `prompt_custom` holds the text when that prompt is a one-off rather than a file. Storing the name rather than the resolved text means editing a preset @@ -463,6 +478,8 @@ filter = ["qwen", "deepseek"] ctx_size = 32768 price_in = 0.60 price_out = 0.60 +# thinking_budget = 8192 # per-provider chain-of-thought cap +# replay_reasoning = false # see below ``` `api_key` accepts three forms: @@ -499,6 +516,15 @@ local router, but on a cloud provider every message is billed for the entire conversation so far. The projection in the cost label exists to make that visible before you send. +`thinking_budget` sets a provider-specific cap on chain-of-thought tokens +(currently sent to SiliconFlow; other endpoints ignore it when unset). + +`replay_reasoning` makes each assistant turn's thinking be sent back on the +next request while web search is enabled. DeepSeek's and SiliconFlow's +interleaved-thinking models require this when a `tools` key is present, and +return a 400 if it is missing. Leave it off elsewhere — replaying thinking +costs context and input tokens for providers that ignore it. + Tool calling and reasoning output vary between providers. Web search on a cloud model may not work as reliably as it does with a local router. |
