diff options
| -rw-r--r-- | README.md | 10 | ||||
| -rw-r--r-- | docs/superpowers/specs/2026-07-31-web-search-design.md | 12 | ||||
| -rw-r--r-- | llamachat/config.py | 7 |
3 files changed, 22 insertions, 7 deletions
@@ -388,11 +388,15 @@ searching again. A local 9B will otherwise keep searching when it is unsure. The default is 1 rather than 2 because of an upstream limitation. The first tool call of a turn arrives as a proper `tool_calls` delta, but a second one, -issued after the model has seen the first set of results, comes back as +issued after the model has seen the first set of results, often comes back as literal `<tool_call><function=web_search>` text inside the thinking instead. There is no structured call to act on, so the turn ends with an empty reply. -Observed with Qwen3.5-9B through llama.cpp's router; raise the cap if your -model and build handle follow-up calls properly. + +Observed with Qwen3.5-9B through llama.cpp's router. Newer builds parse +follow-up calls some of the time (1 in 5 on b10208) rather than never, so +this may be worth retesting; raise the cap if your model and build handle +them reliably. Enabling more search engines helps more than raising the cap, +since a first round that returns plenty removes the reason to search twice. Failures do not abort the turn. A timeout, a refused connection, a non-JSON response or zero results all come back to the model as a tool result saying diff --git a/docs/superpowers/specs/2026-07-31-web-search-design.md b/docs/superpowers/specs/2026-07-31-web-search-design.md index 46ebc91..36732f7 100644 --- a/docs/superpowers/specs/2026-07-31-web-search-design.md +++ b/docs/superpowers/specs/2026-07-31-web-search-design.md @@ -249,7 +249,17 @@ structured call for the loop to act on, so the turn ends with a blank reply. Reproduced 3/3 with Qwen3.5-9B through the router. A retry with the tool withdrawn was tried and rejected: it recovered an answer only some of the time and could surface the raw XML to the user, which is worse than the -blank it replaces. The cap is 1 until the template parses follow-up calls. +blank it replaces. + +Retested on llama.cpp b10208, forcing a follow-up by starving the first +search: 1/5 parsed as a structured call, 1/5 still leaked XML, 3/5 answered +without searching again. So newer builds do parse follow-up calls sometimes, +where earlier ones never did. Not reliable enough to raise the cap: the +failure mode is a blank reply, which is the worst available output. + +In practice this is rarer than those numbers suggest. With enough search +engines enabled the first round returns plenty, so the model seldom needs a +second call at all. ## Notes diff --git a/llamachat/config.py b/llamachat/config.py index 692dd79..ca03996 100644 --- a/llamachat/config.py +++ b/llamachat/config.py @@ -44,10 +44,11 @@ DEFAULTS = { "search_results": 5, "search_snippet_chars": 300, "search_timeout": 10, - # One search per turn. A second call is emitted by the model as literal + # One search per turn. A second call is often emitted as literal # <tool_call> XML rather than a tool_calls delta, which this code cannot - # act on and which leaves the reply empty. Raise it once the template - # parses follow-up calls. + # act on and which leaves the reply empty. It parses correctly some of + # the time (1 in 5 on llama.cpp b10208), so raise this once a build + # handles follow-up calls reliably. "max_searches": 1, } |