| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
The comments and README stated flatly that a second tool call always comes
back as literal <tool_call> XML. On llama.cpp b10208 that is no longer true:
forcing a follow-up by starving the first search gave 1/5 parsed as a
structured call, 1/5 leaking XML, 3/5 answered without searching again.
Earlier builds never parsed it.
Still not reliable enough to raise max_searches, since the failure mode is
a blank reply, but the wording should not read as permanent when it was
measured once against one build.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
|
A second tool call in the same turn, issued after the model has seen the
first set of results, does not arrive as a tool_calls delta. It comes back
as literal <tool_call><function=web_search> text inside reasoning_content,
with finish_reason stop, no tool_calls and empty content. There is no
structured call for the loop to act on, so the turn ends and the user is
shown a blank reply.
Reproduced 3/3 with Qwen3.5-9B through the router; the first call of a turn
parses correctly every time, so this is specific to a call that follows a
tool result.
A retry with the tool withdrawn was tried and rejected: it produced an
answer only some of the time and could surface the raw XML in the reply,
which is worse than the blank it replaces. Capping at one search avoids
reaching the broken state at all. Empty replies drop from 3/3 to about 1/3
and the XML no longer leaks, but the failure is not eliminated.
Raise max_searches once the chat template parses follow-up calls.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
|
The model is offered a web_search tool and decides when a question needs
current information. llamachat runs the query against SearXNG's JSON API,
feeds the results back, and the model answers from them. Off unless both
search_enabled and search_url are set, so an upgrade never starts talking
to the network on its own.
The loop lives in backend.stream_chat: a round ending in tool_calls is
searched for, the result appended, and the request re-sent. After
max_searches rounds the tool is withdrawn, which forces an answer rather
than letting an uncertain model search forever.
Searches show as a collapsible block above the thinking block, listing
each query and its sources. The queries are visible on purpose: when an
answer is wrong it is usually the query that was wrong, and without
seeing it a bad search and a bad answer look identical.
Two failures found while testing against the live stack shaped the
design:
The model has no clock, so it falls back on its training cutoff and
writes that year into the query itself ("latest kernel ... 2025"),
poisoning the results before they are fetched. The current date now goes
into the system prompt whenever search is on, with an instruction not to
date its own queries.
A SearXNG whose engines are all rate-limited or CAPTCHA'd returns a
valid response with zero results. Reporting that as "no results" tells
the model the web is empty and invites a confident answer from stale
training data, so a search where every engine failed is now an error
naming the engines.
Results are attacker-influenced text entering the model's context. Only
title, url and the snippet survive, snippets are truncated, result text
is escaped on display, result links are never fetched automatically, and
a reply forging the search block's URL scheme has it defused as the
reasoning scheme already was. None of that stops a poisoned snippet from
influencing the answer, which is why the sources stay visible.
Adds eight test groups, 23 to 31, all hermetic behind a fake HTTP layer.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
|
Model-decided tool calling against a SearXNG instance, with the loop
inside stream_chat, results shown as a collapsible block, and a cap of
two searches per turn.
Verified against the live stack before writing: Qwen3.5-9B through the
router emits well-formed tool_calls and answers correctly from a
role: "tool" result. No llama-server flags are involved; the shipped
--tools option runs server-side tools and none of them search the web.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
