llamachat

A small native desktop chat client for a local llama.cpp server running in router mode. PySide6, no browser, no Electron. It runs as a single persistent process and toggles like a scratchpad from a Hyprland keybind.

Features

Two modes. One-shot for a quick question with no context carried over, Chat for a resumable multi-turn conversation. The toggle sits in the top bar, not in a menu.
Model picker populated at runtime from the router's /v1/models. Models with an mmproj file in presets.ini are marked with an eye.
File attachment by drag-and-drop anywhere on the window or through the file dialog. Text and code files are inlined into the prompt, truncated to fit the model's context with a warning when that happens. Images require a vision model, and dropping one on a text model offers to switch.
History in SQLite with FTS5 full-text search. Past chat sessions reopen and continue with their context intact; one-shot entries reopen read-only.
Streaming replies rendered token by token, formatted as markdown: headings, bold and italic, bullet and numbered lists, tables, inline code and tinted fenced code blocks. What you type is shown exactly as typed, so the contents of an attached file are never reflowed.
Collapsed reasoning. Presets with a reasoning budget return the model's thinking separately from its answer. It shows as a one-line ▸ thinking summary above the reply, and clicking that expands it. Each reply collapses independently and the state is remembered per bubble.
Context meter in the top bar showing how much of the model's context window the next request will use. It estimates while you type and becomes exact after each reply, turning amber at 75% and red at 90%.
System prompts kept as markdown files in ~/.config/llamachat/prompts/. A global default.md applies to new chats, named presets replace it, and any conversation can take a one-off custom prompt or none at all. The choice is stored per session, so reopening an old chat restores the prompt it was built with.
Web search through a SearXNG instance, off by default. The model is offered a web_search tool and calls it when a question needs current information; the queries and their sources show as a collapsible block above the reply. See Web search.
Tray icon for show/hide/quit, hosted by waybar's tray module.

Not in this version: RAG or embedding search over history, multi-user support, any cloud or non-local backend, remote access.

Requirements

Python 3.11 or newer (3.12 tested)
PySide6, httpx, Pillow
A llama.cpp server in router mode with an OpenAI-compatible API

Pillow is only used for attachment thumbnails.

Installation

PySide6 is not in SlackBuilds, so the simplest route on Slackware is a venv. --system-site-packages lets httpx and Pillow come from the system packages instead of being duplicated:

python3 -m venv --system-site-packages ~/.local/share/llamachat-venv
~/.local/share/llamachat-venv/bin/pip install PySide6-Essentials

Then put the launcher on your PATH:

git clone <repo> ~/src/llamachat
ln -s ~/src/llamachat/llamachat.py ~/bin/llamachat

The launcher starts under the system Python and hands over to an interpreter that has PySide6, so no activation step is needed and the command works from any directory. It looks for .venv/ or venv/ beside the checkout, then ~/.local/share/llamachat-venv/. Set LLAMACHAT_PYTHON to name a specific interpreter, or LLAMACHAT_NO_REEXEC=1 to disable the handover entirely.

Install the desktop entry, which lets the portal resolve the application id and quiets a startup warning:

ln -s ~/src/llamachat/llamachat.desktop \
      ~/.local/share/applications/llamachat.desktop

Verify:

llamachat --help
cd ~/src/llamachat && ./test_llamachat.py

Qt theming from a venv

A pip-installed PySide6 bundles its own Qt build with almost no plugins, so QT_QPA_PLATFORMTHEME=qt6ct finds nothing to load and every window falls back to Fusion, ignoring qt6ct and Kvantum entirely.

llamachat handles this at startup: if the system Qt6 is the same version as the one PySide6 bundles, it points QT_PLUGIN_PATH at the system plugin directory so the real platform theme loads. The version check matters, because a plugin built against a different Qt would crash rather than merely look wrong. Setting QT_PLUGIN_PATH yourself disables this and your value is used unchanged.

Check what a mismatch would look like:

python3 -c 'from PySide6.QtCore import qVersion; print("bundled", qVersion())'
ls /usr/lib64/libQt6Core.so.6.*

If those differ, the theme will not be picked up. Either install a PySide6 matching the system Qt (pip install "PySide6-Essentials==6.11.*" for a 6.11 system) or build PySide6 against the system Qt.

Configuration

~/.config/llamachat/config.toml is written with defaults on first run.

base_url = "http://localhost:8181"
presets = "/etc/llama-server/presets.ini"

# Empty values take the XDG defaults:
#   socket -> $XDG_RUNTIME_DIR/llamachat.sock
#   db     -> $XDG_DATA_HOME/llamachat/history.db
socket = ""
db = ""

default_model = ""
request_timeout = 300

# System prompt for new conversations: a preset name from prompts/,
# "none" for no system prompt, or empty for the global default.md.
default_prompt = ""

attach_ctx_fraction = 0.5
chars_per_token = 3.5

# Web search. Both keys are required: a flag with no URL stays off.
search_enabled = false
search_url = ""

search_results = 5
search_snippet_chars = 300
search_timeout = 10
max_searches = 1

presets.ini is read for two things the API does not report: which models have an mmproj file, and each model's ctx-size. Both ; and # count as comment characters there, matching llama-server, so a commented-out mmproj line correctly reads as "no vision support".

An attachment may fill ctx_size * chars_per_token * attach_ctx_fraction characters before it is truncated. With the defaults and a 32k model that is roughly 57000 characters. Models the router offers but presets.ini does not describe fall back to a conservative 4096-token assumption.

Usage

llamachat --daemon    start the background instance
llamachat --toggle    show if hidden, hide if visible
llamachat --show      show the window
llamachat --hide      hide the window
llamachat --ping      exit 0 when an instance is running
llamachat --quit      stop the running instance

The control commands do not import Qt, so they return in well under a tenth of a second, which is what makes them usable behind a keybind.

In the window: Ctrl+Enter sends, Ctrl+\ shows or hides the history panel (the ☰ button does the same), Esc hides the window, New starts a fresh conversation, and right-clicking a history entry offers to delete it.

The panel's width and whether it was hidden are remembered in ~/.config/llamachat/state.ini between runs.

Hyprland configuration

For Hyprland 0.55+ with the Lua config system. Paths below assume the ~/.config/hypr/sections/ layout.

Autostart

In sections/autostart.lua, inside the existing hl.on("hyprland.start", ...) block:

hl.exec_cmd("llamachat --daemon")

Keybind

In sections/keybindings.lua:

hl.bind(mainMod .. " + Home",          hl.dsp.exec_cmd("llamachat --toggle"))

Window rules

In sections/look_and_feel.lua:

-- llamachat: floating scratchpad-style window
hl.window_rule({
    name  = "llamachat-float",
    match = { class = "llamachat" },
    float = true,
})
hl.window_rule({
    name  = "llamachat-size",
    match = { class = "llamachat" },
    size  = { 1000, 700 },
})
hl.window_rule({
    name   = "llamachat-center",
    match  = { class = "llamachat" },
    center = true,
})

If center is not supported by your Hyprland Lua plugin, pin the geometry instead. move is monitor-relative, so for a 2560x1080 DP-1 and a 1000x700 window the offsets are x=(2560-1000)/2=780 and y=(1080-700)/2=190:

hl.window_rule({
    name    = "llamachat-monitor",
    match   = { class = "llamachat" },
    monitor = "DP-1",
})
hl.window_rule({
    name  = "llamachat-move",
    match = { class = "llamachat" },
    move  = { 780, 190 },
})

The window class is llamachat. Class matching in this plugin is a regex/substring match, so no escaping is needed.

Reload with hyprctl reload after editing. Window rules apply at window-open time, so if the daemon was already running when you added them, restart it with llamachat --quit followed by llamachat --daemon.

Architecture

llamachat.py           launcher, venv shebang, symlink target
llamachat/
  __main__.py          argument dispatch, daemon, tray, socket event loop
  config.py            config.toml and presets.ini parsing
  backend.py           httpx streaming client, tool loop, attachment loading
  search.py            SearXNG queries, tool schema, result sanitising
  db.py                SQLite schema, FTS5 search
  ui.py                the window
test_llamachat.py      self-checks for everything except the GUI

Control protocol

One JSON object per line over a Unix socket, one response line per request. The socket is created mode 0600 in $XDG_RUNTIME_DIR.

-> {"cmd": "toggle"}    <- {"ok": true, "visible": false}
-> {"cmd": "show"}      <- {"ok": true, "visible": true}
-> {"cmd": "hide"}      <- {"ok": true, "visible": false}
-> {"cmd": "ping"}      <- {"ok": true}
-> {"cmd": "quit"}      <- {"ok": true}

Anything else answers {"ok": false, "error": "..."}. A socket left behind by a crashed instance is detected with a ping and removed at startup. The listening socket is serviced by a QSocketNotifier on the Qt event loop, so there is no second thread.

Database schema

sessions    (id, mode, title, model, prompt_name, prompt_custom,
             created_at, updated_at)
messages    (id, session_id, role, content, reasoning, searches, created_at)
attachments (id, message_id, path, kind, mime, size, sha256, thumb, truncated)
messages_fts -- FTS5 external-content table over messages.content

Three triggers keep the FTS index in step with inserts, updates and deletes, which matters because a streamed reply is inserted empty and updated once it finishes.

reasoning holds the model's thinking, kept in its own column for two reasons: the FTS index covers content only, so thinking text cannot bury real search hits, and only content is replayed when continuing a conversation, which is what these APIs expect.

searches holds a JSON array of what was looked up for that reply, one entry per search with its query, results and any error, or NULL when nothing was searched. Like reasoning it stays out of the FTS index, or snippets from web pages would compete with messages the user actually wrote.

prompt_name records which system prompt a conversation was built with, and prompt_custom holds the text when that prompt is a one-off rather than a file. Storing the name rather than the resolved text means editing a preset affects the conversations that use it, which is usually what you want; a conversation that needed frozen wording should use a custom prompt.

Databases created before any of these columns existed gain them automatically on open.

Attachment content lives inline in messages.content, since that is what was actually sent to the model. The attachments table records provenance: the original path, size, and a SHA-256 so a file can be identified even if it was later moved or renamed, plus a 128px JPEG thumbnail for images so a reopened conversation still shows what was sent even when the original file is gone. Full image bytes are not stored.

Search input is tokenised and quoted before it reaches FTS5, so punctuation that would otherwise be read as query syntax cannot cause an error.

System prompts

Prompts are markdown files in ~/.config/llamachat/prompts/, one per file, created on first run:

prompts/
  default.md      the global prompt, used unless something overrides it
  coding.md       a named preset
  terse.md        another

They are plain files on purpose, so they can be edited in an editor, diffed, and kept in version control. The … button beside the picker opens a dialog that does the same thing from inside the app.

The picker offers every file plus two extras. A named preset replaces the global prompt rather than being appended to it. custom for this chat… takes one-off text stored with that conversation, and no system prompt sends none at all.

default_prompt in config.toml chooses what new conversations start with: a preset name, "none", or empty for default.md.

The choice is recorded per session, so reopening a chat restores the prompt it was built with rather than silently applying whatever is selected now.

Web search

Off unless both search_enabled = true and a search_url are set. A flag with no URL behind it resolves to disabled at startup, so an upgrade never starts making network requests on its own.

The model decides when to search. It is offered an OpenAI-style web_search function, and when it judges a question needs current information it calls it; llamachat queries SearXNG's JSON API, hands back the results, and the model answers from them. Nothing needs enabling on the llama-server side: this is plain tool calling, not llama-server's --tools (which runs server-side tools, none of which search the web) and not --ui-mcp-proxy (a CORS shim that exists for browsers).

The SearXNG instance must have its JSON output format enabled, which many installs turn off by default. Without it, queries fail with a message saying so.

Searches show in the transcript as a collapsible block above the thinking block, with a status line while one runs:

▸ searched: latest stable linux kernel (5 results)
▸ thinking (1441 chars, 38 lines)
Model:
...

Expanding it lists each result's title, link and snippet. The queries are visible on purpose: when an answer is wrong it is usually the query that was wrong, and without seeing it a bad search and a bad answer look identical.

max_searches caps searches per turn, default 1. On the last round the tool is withdrawn from the request, which forces the model to answer instead of searching again. A local 9B will otherwise keep searching when it is unsure.

The default is 1 rather than 2 because of an upstream limitation. The first tool call of a turn arrives as a proper tool_calls delta, but a second one, issued after the model has seen the first set of results, often comes back as literal <tool_call><function=web_search> text inside the thinking instead. There is no structured call to act on, so the turn ends with an empty reply.

Observed with Qwen3.5-9B through llama.cpp's router. Newer builds parse follow-up calls some of the time (1 in 5 on b10208) rather than never, so this may be worth retesting; raise the cap if your model and build handle them reliably. Enabling more search engines helps more than raising the cap, since a first round that returns plenty removes the reason to search twice.

Failures do not abort the turn. A timeout, a refused connection, a non-JSON response or zero results all come back to the model as a tool result saying what happened, so it answers with that knowledge rather than silently falling back on stale training data. The block then reads ▸ search failed: <query> — <reason>.

A search that returns nothing because every engine failed counts as a failure rather than an empty result, and the message names the engines. On a self-hosted SearXNG the usual way search stops working is engines getting rate-limited or CAPTCHA'd one by one, and reporting that as "no results" tells the model the web is empty, which invites exactly the confident wrong answer this feature exists to prevent.

When search is on, the current date is appended to the system prompt. A model has no clock, so it falls back on its training cutoff and writes that year into the search query itself, poisoning the results before they are fetched. It is also told not to put a year in a query unless asked. This is added even when no system prompt is selected, since the model has no clock either way.

On trusting results. Search snippets are attacker-influenced text going into the model's context, in an app whose output you may paste into a terminal. Structurally: only title, url and the snippet are kept and everything else SearXNG returns is dropped, snippets are truncated to search_snippet_chars, all result text is escaped when displayed, result links are shown but never fetched automatically, and a reply that forges the search block's internal URL scheme has it defused. The tool result is also labelled untrusted for the model, which helps inconsistently.

None of that stops a poisoned snippet from influencing what the model says. If a top result asserts something false, a 9B may repeat it. That is why the sources stay visible rather than being hidden behind the answer.

The context meter

The bar in the top bar shows how much of the active model's context window the next request will occupy: system prompt, prior turns, attachments, and whatever is currently in the input box.

While you type it is an estimate derived from chars_per_token, shown with a leading ~. When a reply finishes, the server reports the real token counts in the stream and the meter switches to those, dropping the ~.

Expect the estimate to read low on models with a reasoning budget. Thinking tokens count against the context but cannot be known before the reply arrives, so a model that thinks for 2000 tokens will jump well past the estimate once it answers. The exact figure after each turn is the one to trust.

The limit comes from ctx-size in presets.ini. Models the router offers but that file does not describe show no limit.

Markdown rendering

Replies are parsed by QTextDocument.setMarkdown(), so there is no markdown dependency. It runs in the GitHub dialect with the MarkdownNoHTML flag, which means markup a model emits is displayed literally rather than being interpreted: a reply containing <img src=x onerror=...> shows that text and nothing executes.

Markdown links are still real links, so a reply cannot forge one that drives the interface: anchors using the internal x-llamachat-reasoning: scheme are rewritten before display, and only http, https and mailto links are handed to the desktop opener.

Qt renders a fenced block as one <pre> element per line with no background of its own, so llamachat adds the tint that makes a code block read as one.

Notes and limits

Switching models makes the router unload the previous one. The first token after a switch can take several seconds; the window shows a waiting state rather than pretending it is instant.
Reasoning is read from the reasoning_content field the router sends alongside content, not by parsing <think> tags out of the reply. How much of it a model produces is set by reasoning-budget in presets.ini; set that to 0 for a preset that should not think at all.
The attachment budget is an estimate based on a characters-per-token ratio, not a real tokeniser. It is deliberately conservative.
The tray icon needs a StatusNotifierItem host. With waybar, that means the tray module in your config. Without one, the icon silently does nothing and the keybind still works.

License

GPL-2.0-only. See LICENSE.

Development Approach

This project is developed using AI-assisted tools. Code is generated with the help of AI based on human-provided specifications, design decisions, and iterative feedback.

All contributions are reviewed, tested, and curated by the maintainer before being included in the codebase. AI is used as a productivity and exploration tool, while human oversight remains central to all decisions.

The goal is to combine the flexibility of AI-assisted development with standard open-source practices such as transparency, review, and accountability.