diff options
| author | Danilo M. <danix@danix.xyz> | 2026-07-31 19:09:44 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-07-31 19:09:44 +0200 |
| commit | 0e35416128e38f158af49935262ecbf8dbf585e6 (patch) | |
| tree | 2af059613853634dc8968ad700e98651bb383514 /README.md | |
| parent | 283fc9ae0371f4d318a539f021a4b88e05fac190 (diff) | |
| download | llamachat-0e35416128e38f158af49935262ecbf8dbf585e6.tar.gz llamachat-0e35416128e38f158af49935262ecbf8dbf585e6.zip | |
fix: cap searches at one per turn
A second tool call in the same turn, issued after the model has seen the
first set of results, does not arrive as a tool_calls delta. It comes back
as literal <tool_call><function=web_search> text inside reasoning_content,
with finish_reason stop, no tool_calls and empty content. There is no
structured call for the loop to act on, so the turn ends and the user is
shown a blank reply.
Reproduced 3/3 with Qwen3.5-9B through the router; the first call of a turn
parses correctly every time, so this is specific to a call that follows a
tool result.
A retry with the tool withdrawn was tried and rejected: it produced an
answer only some of the time and could surface the raw XML in the reply,
which is worse than the blank it replaces. Capping at one search avoids
reaching the broken state at all. Empty replies drop from 3/3 to about 1/3
and the XML no longer leaks, but the failure is not eliminated.
Raise max_searches once the chat template parses follow-up calls.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 12 |
1 files changed, 10 insertions, 2 deletions
@@ -144,7 +144,7 @@ search_url = "" search_results = 5 search_snippet_chars = 300 search_timeout = 10 -max_searches = 2 +max_searches = 1 ``` `presets.ini` is read for two things the API does not report: which models @@ -382,10 +382,18 @@ Expanding it lists each result's title, link and snippet. The queries are visible on purpose: when an answer is wrong it is usually the query that was wrong, and without seeing it a bad search and a bad answer look identical. -`max_searches` caps searches per turn, default 2. On the last round the tool +`max_searches` caps searches per turn, default 1. On the last round the tool is withdrawn from the request, which forces the model to answer instead of searching again. A local 9B will otherwise keep searching when it is unsure. +The default is 1 rather than 2 because of an upstream limitation. The first +tool call of a turn arrives as a proper `tool_calls` delta, but a second one, +issued after the model has seen the first set of results, comes back as +literal `<tool_call><function=web_search>` text inside the thinking instead. +There is no structured call to act on, so the turn ends with an empty reply. +Observed with Qwen3.5-9B through llama.cpp's router; raise the cap if your +model and build handle follow-up calls properly. + Failures do not abort the turn. A timeout, a refused connection, a non-JSON response or zero results all come back to the model as a tool result saying what happened, so it answers with that knowledge rather than silently |
