summaryrefslogtreecommitdiffstats
path: root/README.md
diff options
context:
space:
mode:
authorDanilo M. <danix@danix.xyz>2026-07-31 18:36:13 +0200
committerDanilo M. <danix@danix.xyz>2026-07-31 18:36:13 +0200
commit283fc9ae0371f4d318a539f021a4b88e05fac190 (patch)
tree0bf03685df84c275cb7ab3af3e20d94781a078cc /README.md
parent8bda554af191c5749459cd258c793e4826c3f9c7 (diff)
downloadllamachat-283fc9ae0371f4d318a539f021a4b88e05fac190.tar.gz
llamachat-283fc9ae0371f4d318a539f021a4b88e05fac190.zip
feat: web search via SearXNG
The model is offered a web_search tool and decides when a question needs current information. llamachat runs the query against SearXNG's JSON API, feeds the results back, and the model answers from them. Off unless both search_enabled and search_url are set, so an upgrade never starts talking to the network on its own. The loop lives in backend.stream_chat: a round ending in tool_calls is searched for, the result appended, and the request re-sent. After max_searches rounds the tool is withdrawn, which forces an answer rather than letting an uncertain model search forever. Searches show as a collapsible block above the thinking block, listing each query and its sources. The queries are visible on purpose: when an answer is wrong it is usually the query that was wrong, and without seeing it a bad search and a bad answer look identical. Two failures found while testing against the live stack shaped the design: The model has no clock, so it falls back on its training cutoff and writes that year into the query itself ("latest kernel ... 2025"), poisoning the results before they are fetched. The current date now goes into the system prompt whenever search is on, with an instruction not to date its own queries. A SearXNG whose engines are all rate-limited or CAPTCHA'd returns a valid response with zero results. Reporting that as "no results" tells the model the web is empty and invites a confident answer from stale training data, so a search where every engine failed is now an error naming the engines. Results are attacker-influenced text entering the model's context. Only title, url and the snippet survive, snippets are truncated, result text is escaped on display, result links are never fetched automatically, and a reply forging the search block's URL scheme has it defused as the reasoning scheme already was. None of that stops a poisoned snippet from influencing the answer, which is why the sources stay visible. Adds eight test groups, 23 to 31, all hermetic behind a fake HTTP layer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Diffstat (limited to 'README.md')
-rw-r--r--README.md92
1 files changed, 90 insertions, 2 deletions
diff --git a/README.md b/README.md
index 778010f..546fa1b 100644
--- a/README.md
+++ b/README.md
@@ -33,6 +33,10 @@ persistent process and toggles like a scratchpad from a Hyprland keybind.
any conversation can take a one-off custom prompt or none at all. The
choice is stored per session, so reopening an old chat restores the prompt
it was built with.
+- **Web search** through a SearXNG instance, off by default. The model is
+ offered a `web_search` tool and calls it when a question needs current
+ information; the queries and their sources show as a collapsible block
+ above the reply. See [Web search](#web-search).
- **Tray icon** for show/hide/quit, hosted by waybar's tray module.
Not in this version: RAG or embedding search over history, multi-user
@@ -132,6 +136,15 @@ default_prompt = ""
attach_ctx_fraction = 0.5
chars_per_token = 3.5
+
+# Web search. Both keys are required: a flag with no URL stays off.
+search_enabled = false
+search_url = ""
+
+search_results = 5
+search_snippet_chars = 300
+search_timeout = 10
+max_searches = 2
```
`presets.ini` is read for two things the API does not report: which models
@@ -242,7 +255,8 @@ llamachat.py launcher, venv shebang, symlink target
llamachat/
__main__.py argument dispatch, daemon, tray, socket event loop
config.py config.toml and presets.ini parsing
- backend.py httpx streaming client, attachment loading
+ backend.py httpx streaming client, tool loop, attachment loading
+ search.py SearXNG queries, tool schema, result sanitising
db.py SQLite schema, FTS5 search
ui.py the window
test_llamachat.py self-checks for everything except the GUI
@@ -271,7 +285,7 @@ there is no second thread.
```sql
sessions (id, mode, title, model, prompt_name, prompt_custom,
created_at, updated_at)
-messages (id, session_id, role, content, reasoning, created_at)
+messages (id, session_id, role, content, reasoning, searches, created_at)
attachments (id, message_id, path, kind, mime, size, sha256, thumb, truncated)
messages_fts -- FTS5 external-content table over messages.content
```
@@ -285,6 +299,11 @@ reasons: the FTS index covers `content` only, so thinking text cannot bury
real search hits, and only `content` is replayed when continuing a
conversation, which is what these APIs expect.
+`searches` holds a JSON array of what was looked up for that reply, one entry
+per search with its query, results and any error, or NULL when nothing was
+searched. Like `reasoning` it stays out of the FTS index, or snippets from
+web pages would compete with messages the user actually wrote.
+
`prompt_name` records which system prompt a conversation was built with, and
`prompt_custom` holds the text when that prompt is a one-off rather than a
file. Storing the name rather than the resolved text means editing a preset
@@ -331,6 +350,75 @@ a preset name, `"none"`, or empty for `default.md`.
The choice is recorded per session, so reopening a chat restores the prompt
it was built with rather than silently applying whatever is selected now.
+### Web search
+
+Off unless both `search_enabled = true` and a `search_url` are set. A flag
+with no URL behind it resolves to disabled at startup, so an upgrade never
+starts making network requests on its own.
+
+The model decides when to search. It is offered an OpenAI-style `web_search`
+function, and when it judges a question needs current information it calls
+it; llamachat queries SearXNG's JSON API, hands back the results, and the
+model answers from them. Nothing needs enabling on the llama-server side:
+this is plain tool calling, not llama-server's `--tools` (which runs
+server-side tools, none of which search the web) and not `--ui-mcp-proxy`
+(a CORS shim that exists for browsers).
+
+The SearXNG instance must have its JSON output format enabled, which many
+installs turn off by default. Without it, queries fail with a message saying
+so.
+
+Searches show in the transcript as a collapsible block above the thinking
+block, with a status line while one runs:
+
+```
+▸ searched: latest stable linux kernel (5 results)
+▸ thinking (1441 chars, 38 lines)
+Model:
+...
+```
+
+Expanding it lists each result's title, link and snippet. The queries are
+visible on purpose: when an answer is wrong it is usually the query that was
+wrong, and without seeing it a bad search and a bad answer look identical.
+
+`max_searches` caps searches per turn, default 2. On the last round the tool
+is withdrawn from the request, which forces the model to answer instead of
+searching again. A local 9B will otherwise keep searching when it is unsure.
+
+Failures do not abort the turn. A timeout, a refused connection, a non-JSON
+response or zero results all come back to the model as a tool result saying
+what happened, so it answers with that knowledge rather than silently
+falling back on stale training data. The block then reads
+`▸ search failed: <query> — <reason>`.
+
+A search that returns nothing *because every engine failed* counts as a
+failure rather than an empty result, and the message names the engines. On
+a self-hosted SearXNG the usual way search stops working is engines getting
+rate-limited or CAPTCHA'd one by one, and reporting that as "no results"
+tells the model the web is empty, which invites exactly the confident wrong
+answer this feature exists to prevent.
+
+When search is on, the current date is appended to the system prompt. A
+model has no clock, so it falls back on its training cutoff and writes that
+year into the search query itself, poisoning the results before they are
+fetched. It is also told not to put a year in a query unless asked. This is
+added even when `no system prompt` is selected, since the model has no clock
+either way.
+
+**On trusting results.** Search snippets are attacker-influenced text going
+into the model's context, in an app whose output you may paste into a
+terminal. Structurally: only `title`, `url` and the snippet are kept and
+everything else SearXNG returns is dropped, snippets are truncated to
+`search_snippet_chars`, all result text is escaped when displayed, result
+links are shown but never fetched automatically, and a reply that forges the
+search block's internal URL scheme has it defused. The tool result is also
+labelled untrusted for the model, which helps inconsistently.
+
+None of that stops a poisoned snippet from influencing what the model says.
+If a top result asserts something false, a 9B may repeat it. That is why the
+sources stay visible rather than being hidden behind the answer.
+
### The context meter
The bar in the top bar shows how much of the active model's context window