diff options
| author | Danilo M. <danix@danix.xyz> | 2026-07-31 19:28:16 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-07-31 19:28:16 +0200 |
| commit | fe335db0cdc72cc190c290479d24c6469e2c07ab (patch) | |
| tree | 43f1996795bbb9bf41ba793422d6581c5a60992c /README.md | |
| parent | 0e35416128e38f158af49935262ecbf8dbf585e6 (diff) | |
| download | llamachat-fe335db0cdc72cc190c290479d24c6469e2c07ab.tar.gz llamachat-fe335db0cdc72cc190c290479d24c6469e2c07ab.zip | |
docs: correct the follow-up tool call claim
The comments and README stated flatly that a second tool call always comes
back as literal <tool_call> XML. On llama.cpp b10208 that is no longer true:
forcing a follow-up by starving the first search gave 1/5 parsed as a
structured call, 1/5 leaking XML, 3/5 answered without searching again.
Earlier builds never parsed it.
Still not reliable enough to raise max_searches, since the failure mode is
a blank reply, but the wording should not read as permanent when it was
measured once against one build.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 10 |
1 files changed, 7 insertions, 3 deletions
@@ -388,11 +388,15 @@ searching again. A local 9B will otherwise keep searching when it is unsure. The default is 1 rather than 2 because of an upstream limitation. The first tool call of a turn arrives as a proper `tool_calls` delta, but a second one, -issued after the model has seen the first set of results, comes back as +issued after the model has seen the first set of results, often comes back as literal `<tool_call><function=web_search>` text inside the thinking instead. There is no structured call to act on, so the turn ends with an empty reply. -Observed with Qwen3.5-9B through llama.cpp's router; raise the cap if your -model and build handle follow-up calls properly. + +Observed with Qwen3.5-9B through llama.cpp's router. Newer builds parse +follow-up calls some of the time (1 in 5 on b10208) rather than never, so +this may be worth retesting; raise the cap if your model and build handle +them reliably. Enabling more search engines helps more than raising the cap, +since a first round that returns plenty removes the reason to search twice. Failures do not abort the turn. A timeout, a refused connection, a non-JSON response or zero results all come back to the model as a tool result saying |
