llamachat

A small native desktop chat client for a local llama.cpp server running in router mode. PySide6, no browser, no Electron. It runs as a single persistent process and toggles like a scratchpad from a Hyprland keybind.

Features

Two modes. One-shot for a quick question with no context carried over, Chat for a resumable multi-turn conversation. The toggle sits in the top bar, not in a menu.
Model picker populated at runtime from the router's /v1/models. Models with an mmproj file in presets.ini are marked with an eye.
File attachment by drag-and-drop anywhere on the window or through the file dialog. Text and code files are inlined into the prompt, truncated to fit the model's context with a warning when that happens. Images require a vision model, and dropping one on a text model offers to switch.
History in SQLite with FTS5 full-text search. Past chat sessions reopen and continue with their context intact; one-shot entries reopen read-only. Sessions are named by the model itself: once the first reply lands, it is asked to title the exchange in a few words, which replaces the opening words of the question the entry was created with.
Streaming replies rendered token by token, formatted as markdown: headings, bold and italic, bullet and numbered lists, tables, inline code and tinted fenced code blocks. Long lines wrap rather than scrolling the transcript sideways, and each code block carries a copy button. What you type is shown exactly as typed, so the contents of an attached file are never reflowed.
Collapsed reasoning. Presets with a reasoning budget return the model's thinking separately from its answer. It shows as a one-line ▸ thinking summary above the reply, and clicking that expands it. Each reply collapses independently and the state is remembered per bubble.
Context meter in the top bar showing how much of the model's context window the next request will use. It estimates while you type and becomes exact after each reply, turning amber at 75% and red at 90%.
System prompts kept as markdown files in ~/.config/llamachat/prompts/. A global default.md applies to new chats, named presets replace it, and any conversation can take a one-off custom prompt or none at all. The choice is stored per session, so reopening an old chat restores the prompt it was built with.
Web search through a SearXNG instance, off by default. The model is offered a web_search tool and calls it when a question needs current information; the queries and their sources show as a collapsible block above the reply. See Web search.
Tray icon for show/hide/quit, hosted by waybar's tray module.

Not in this version: RAG or embedding search over history, multi-user support, remote access.

Requirements

Python 3.11 or newer (3.12 tested)
PySide6, httpx, Pillow
A llama.cpp server in router mode with an OpenAI-compatible API

Pillow is only used for attachment thumbnails.

Installation

PySide6 is not in SlackBuilds, so the simplest route on Slackware is a venv. --system-site-packages lets httpx and Pillow come from the system packages instead of being duplicated:

python3 -m venv --system-site-packages ~/.local/share/llamachat-venv
~/.local/share/llamachat-venv/bin/pip install PySide6-Essentials

Then put the launcher on your PATH:

git clone <repo> ~/src/llamachat
ln -s ~/src/llamachat/llamachat.py ~/bin/llamachat

The launcher starts under the system Python and hands over to an interpreter that has PySide6, so no activation step is needed and the command works from any directory. It looks for .venv/ or venv/ beside the checkout, then ~/.local/share/llamachat-venv/. Set LLAMACHAT_PYTHON to name a specific interpreter, or LLAMACHAT_NO_REEXEC=1 to disable the handover entirely.

Install the desktop entry, which lets the portal resolve the application id and quiets a startup warning:

ln -s ~/src/llamachat/llamachat.desktop \
      ~/.local/share/applications/llamachat.desktop

Verify:

llamachat --help
cd ~/src/llamachat && ./test_llamachat.py

Qt theming from a venv

A pip-installed PySide6 bundles its own Qt build with almost no plugins, so QT_QPA_PLATFORMTHEME=qt6ct finds nothing to load and every window falls back to Fusion, ignoring qt6ct and Kvantum entirely.

llamachat handles this at startup: if the system Qt6 is the same version as the one PySide6 bundles, it points QT_PLUGIN_PATH at the system plugin directory so the real platform theme loads. The version check matters, because a plugin built against a different Qt would crash rather than merely look wrong. Setting QT_PLUGIN_PATH yourself disables this and your value is used unchanged.

Check what a mismatch would look like:

python3 -c 'from PySide6.QtCore import qVersion; print("bundled", qVersion())'
ls /usr/lib64/libQt6Core.so.6.*

If those differ, the theme will not be picked up. Either install a PySide6 matching the system Qt (pip install "PySide6-Essentials==6.11.*" for a 6.11 system) or build PySide6 against the system Qt.

Configuration

~/.config/llamachat/config.toml is written with defaults on first run.

base_url = "http://localhost:8181"
presets = "/etc/llama-server/presets.ini"

# Empty values take the XDG defaults:
#   socket -> $XDG_RUNTIME_DIR/llamachat.sock
#   db     -> $XDG_DATA_HOME/llamachat/history.db
socket = ""
db = ""

default_model = ""
request_timeout = 300

# System prompt for new conversations: a preset name from prompts/,
# "none" for no system prompt, or empty for the global default.md.
default_prompt = ""

attach_ctx_fraction = 0.5
chars_per_token = 3.5

# Web search. Both keys are required: a flag with no URL stays off.
search_enabled = false
search_url = ""

search_results = 5
search_snippet_chars = 300
search_timeout = 10
max_searches = 2

presets.ini is read for two things the API does not report: which models have an mmproj file, and each model's ctx-size. Both ; and # count as comment characters there, matching llama-server, so a commented-out mmproj line correctly reads as "no vision support".

An attachment may fill ctx_size * chars_per_token * attach_ctx_fraction characters before it is truncated. With the defaults and a 32k model that is roughly 57000 characters. Models the router offers but presets.ini does not describe fall back to a conservative 4096-token assumption.

Usage

llamachat --daemon    start the background instance
llamachat --toggle    show if hidden, hide if visible
llamachat --show      show the window
llamachat --hide      hide the window
llamachat --ping      exit 0 when an instance is running
llamachat --quit      stop the running instance

The control commands do not import Qt, so they return in well under a tenth of a second, which is what makes them usable behind a keybind.

In the window: Ctrl+Enter sends, Ctrl+\ shows or hides the history panel (the ☰ button does the same), Esc hides the window, New starts a fresh conversation, and right-clicking a history entry offers to delete it.

The panel's width and whether it was hidden are remembered in ~/.config/llamachat/state.ini between runs.

Hyprland configuration

For Hyprland 0.55+ with the Lua config system. Paths below assume the ~/.config/hypr/sections/ layout.

Autostart

In sections/autostart.lua, inside the existing hl.on("hyprland.start", ...) block:

hl.exec_cmd("llamachat --daemon")

Keybind

In sections/keybindings.lua:

hl.bind(mainMod .. " + Home",          hl.dsp.exec_cmd("llamachat --toggle"))

Window rules

In sections/look_and_feel.lua:

-- llamachat: floating scratchpad-style window
hl.window_rule({
    name  = "llamachat-float",
    match = { class = "llamachat" },
    float = true,
})
hl.window_rule({
    name  = "llamachat-size",
    match = { class = "llamachat" },
    size  = { 1000, 700 },
})
hl.window_rule({
    name   = "llamachat-center",
    match  = { class = "llamachat" },
    center = true,
})

If center is not supported by your Hyprland Lua plugin, pin the geometry instead. move is monitor-relative, so for a 2560x1080 DP-1 and a 1000x700 window the offsets are x=(2560-1000)/2=780 and y=(1080-700)/2=190:

hl.window_rule({
    name    = "llamachat-monitor",
    match   = { class = "llamachat" },
    monitor = "DP-1",
})
hl.window_rule({
    name  = "llamachat-move",
    match = { class = "llamachat" },
    move  = { 780, 190 },
})

The window class is llamachat. Class matching in this plugin is a regex/substring match, so no escaping is needed.

Reload with hyprctl reload after editing. Window rules apply at window-open time, so if the daemon was already running when you added them, restart it with llamachat --quit followed by llamachat --daemon.

Architecture

llamachat.py           launcher, venv shebang, symlink target
llamachat/
  __main__.py          argument dispatch, daemon, tray, socket event loop
  config.py            config.toml and presets.ini parsing
  backend.py           httpx streaming client, tool loop, attachment loading
  search.py            SearXNG queries, tool schema, result sanitising
  db.py                SQLite schema, FTS5 search
  ui.py                the window
test_llamachat.py      self-checks for everything except the GUI

Control protocol

One JSON object per line over a Unix socket, one response line per request. The socket is created mode 0600 in $XDG_RUNTIME_DIR.

-> {"cmd": "toggle"}    <- {"ok": true, "visible": false}
-> {"cmd": "show"}      <- {"ok": true, "visible": true}
-> {"cmd": "hide"}      <- {"ok": true, "visible": false}
-> {"cmd": "ping"}      <- {"ok": true}
-> {"cmd": "quit"}      <- {"ok": true}

Anything else answers {"ok": false, "error": "..."}. A socket left behind by a crashed instance is detected with a ping and removed at startup. The listening socket is serviced by a QSocketNotifier on the Qt event loop, so there is no second thread.

Database schema

sessions    (id, mode, title, model, prompt_name, prompt_custom,
             created_at, updated_at)
messages    (id, session_id, role, content, reasoning, searches,
             prompt_tokens, completion_tokens, model, provider, usage_json,
             reported_cost_usd, created_at)
attachments (id, message_id, path, kind, mime, size, sha256, thumb, truncated)
messages_fts -- FTS5 external-content table over messages.content

Three triggers keep the FTS index in step with inserts, updates and deletes, which matters because a streamed reply is inserted empty and updated once it finishes.

reasoning holds the model's thinking, kept in its own column for two reasons: the FTS index covers content only, so thinking text cannot bury real search hits, and only content is replayed when continuing a conversation, which is what these APIs expect.

searches holds a JSON array of what was looked up for that reply, one entry per search with its query, results and any error, or NULL when nothing was searched. Like reasoning it stays out of the FTS index, or snippets from web pages would compete with messages the user actually wrote.

provider records which provider served the reply (local or a configured provider name), written when the reply finishes so history stays self-describing even if config.toml later renames or drops the provider. prompt_tokens and completion_tokens hold the turn's totals summed across every round — a searched turn makes one API call per search round and all of them are billed, so the last round's counts alone would under-report it. usage_json keeps each round's raw usage block as a JSON array, verbatim, so token details beyond the two counts (e.g. cached or reasoning tokens on a cloud endpoint) are recoverable later without re-guessing a field mapping. reported_cost_usd is the GUI's hand-entered-price estimate for that one reply (summed across rounds), stored as a sanity check for external consumers — it is approximate and never a source of truth; it stays NULL for a model with no entered prices rather than claiming the reply cost nothing.

prompt_name records which system prompt a conversation was built with, and prompt_custom holds the text when that prompt is a one-off rather than a file. Storing the name rather than the resolved text means editing a preset affects the conversations that use it, which is usually what you want; a conversation that needed frozen wording should use a custom prompt.

Databases created before any of these columns existed gain them automatically on open.

Attachment content lives inline in messages.content, since that is what was actually sent to the model. The attachments table records provenance: the original path, size, and a SHA-256 so a file can be identified even if it was later moved or renamed, plus a 128px JPEG thumbnail for images so a reopened conversation still shows what was sent even when the original file is gone. Full image bytes are not stored.

Search input is tokenised and quoted before it reaches FTS5, so punctuation that would otherwise be read as query syntax cannot cause an error.

System prompts

Prompts are markdown files in ~/.config/llamachat/prompts/, one per file, created on first run:

prompts/
  default.md      the global prompt, used unless something overrides it
  coding.md       a named preset
  terse.md        another

They are plain files on purpose, so they can be edited in an editor, diffed, and kept in version control. The … button beside the picker opens a dialog that does the same thing from inside the app.

The picker offers every file plus two extras. A named preset replaces the global prompt rather than being appended to it. custom for this chat… takes one-off text stored with that conversation, and no system prompt sends none at all.

default_prompt in config.toml chooses what new conversations start with: a preset name, "none", or empty for default.md.

The choice is recorded per session, so reopening a chat restores the prompt it was built with rather than silently applying whatever is selected now.

Web search

Off unless both search_enabled = true and a search_url are set. A flag with no URL behind it resolves to disabled at startup, so an upgrade never starts making network requests on its own.

The model decides when to search. It is offered an OpenAI-style web_search function, and when it judges a question needs current information it calls it; llamachat queries SearXNG's JSON API, hands back the results, and the model answers from them. Nothing needs enabling on the llama-server side: this is plain tool calling, not llama-server's --tools (which runs server-side tools, none of which search the web) and not --ui-mcp-proxy (a CORS shim that exists for browsers).

The SearXNG instance must have its JSON output format enabled, which many installs turn off by default. Without it, queries fail with a message saying so.

Searches show in the transcript as a collapsible block above the thinking block, with a status line while one runs:

▸ searched: latest stable linux kernel (5 results)
▸ thinking (1441 chars, 38 lines)
Model:
...

Expanding it lists each result's title, link and snippet. The queries are visible on purpose: when an answer is wrong it is usually the query that was wrong, and without seeing it a bad search and a bad answer look identical.

max_searches caps searches per turn, default 2. On the last round the tool is withdrawn from the request and the tool result says so in words, which is what forces the model to answer instead of searching again. A local 9B will otherwise keep searching when it is unsure.

Withdrawing the schema on its own is not enough, because the model cannot see a schema disappear. It asks for another search regardless, and that request surfaces either as literal <tool_call><function=web_search> text in the reply or buried in the thinking block, leaving the reply itself empty. Saying "this was your last search" in the tool result is what fixes it. The note has to ride on the tool result rather than a trailing system message: Qwen3.5's chat template rejects those outright, raising System message must be at the beginning.

If replies still come back empty after a search, check the server's KV cache. llama.cpp's own function-calling documentation warns that quantized KV degrades tool calling, and it does: with cache-type-k/cache-type-v set to q8_0, three of five turns on one question came back broken, and dropping to the f16 default made the same question answer six times out of six.

Turns that search once are reliable. Turns that genuinely need two are not quite: on a question forcing two rounds, four and five of six succeeded across two runs, the rest coming back empty. Lower max_searches to 1 if you would rather never see an empty reply, or if you keep the quantized cache. Enabling more search engines helps either way, since a first round that returns plenty removes the reason to search twice.

Failures do not abort the turn. A timeout, a refused connection, a non-JSON response or zero results all come back to the model as a tool result saying what happened, so it answers with that knowledge rather than silently falling back on stale training data. The block then reads ▸ search failed: <query> — <reason>.

A search that returns nothing because every engine failed counts as a failure rather than an empty result, and the message names the engines. On a self-hosted SearXNG the usual way search stops working is engines getting rate-limited or CAPTCHA'd one by one, and reporting that as "no results" tells the model the web is empty, which invites exactly the confident wrong answer this feature exists to prevent.

When search is on, the current date is appended to the system prompt. A model has no clock, so it falls back on its training cutoff and writes that year into the search query itself, poisoning the results before they are fetched. It is also told not to put a year in a query unless asked. This is added even when no system prompt is selected, since the model has no clock either way.

On trusting results. Search snippets are attacker-influenced text going into the model's context, in an app whose output you may paste into a terminal. Structurally: only title, url and the snippet are kept and everything else SearXNG returns is dropped, snippets are truncated to search_snippet_chars, all result text is escaped when displayed, result links are shown but never fetched automatically, and a reply that forges the search block's internal URL scheme has it defused. The tool result is also labelled untrusted for the model, which helps inconsistently.

None of that stops a poisoned snippet from influencing what the model says. If a top result asserts something false, a 9B may repeat it. That is why the sources stay visible rather than being hidden behind the answer.

External providers

llamachat can talk to OpenAI-compatible cloud providers alongside the local router. Cloud models appear in the picker as provider:model. Local models stay bare, so an existing local setup needs no changes.

Providers live in [providers.*] tables. The generated config ships a commented together.ai block; uncomment it and set api_key to enable it:

[providers.together]
base_url = "https://api.together.xyz"
api_key = "pass:api/together"
filter = ["qwen", "deepseek"]
ctx_size = 32768
price_in = 0.60
price_out = 0.60
# thinking_budget = 8192     # per-provider chain-of-thought cap
# replay_reasoning = false   # see below

api_key accepts three forms:

pass:api/together reads the key from the password store. This is preferred, because the secret never sits in a config file that might be committed or copied.
env:TOGETHER_KEY reads the key from the environment.
sk-... is the literal key, written into config.toml itself. Avoid this: it puts a secret in a plain-text file.

Keys are resolved lazily, on the first request to that provider. A session that only touches local models never triggers a pinentry or reads an environment variable.

filter keeps only models whose id contains one of the given strings, case-insensitively. Providers list hundreds of models, and without a filter the picker is unusable. Omit filter to list everything.

ctx_size, vision, price_in and price_out prefill the per-model settings dialog. Prices are US dollars per million tokens. The dialog saves anything you enter to models.ini beside config.toml, so the provider block can stay minimal. models.ini is machine-written cache, not user-editable config.

The cost readout in the top bar shows what has been spent in the current conversation plus what the next message would add. It is approximate, based entirely on the hand-entered prices in the provider block or the per-model dialog. It is useful for seeing when a long conversation is becoming expensive.

A conversation resends its whole history every turn. That is free with a local router, but on a cloud provider every message is billed for the entire conversation so far. The projection in the cost label exists to make that visible before you send.

thinking_budget sets a provider-specific cap on chain-of-thought tokens (currently sent to SiliconFlow; other endpoints ignore it when unset).

replay_reasoning makes each assistant turn's thinking be sent back on the next request while web search is enabled. DeepSeek's and SiliconFlow's interleaved-thinking models require this when a tools key is present, and return a 400 if it is missing. Leave it off elsewhere — replaying thinking costs context and input tokens for providers that ignore it.

Tool calling and reasoning output vary between providers. Web search on a cloud model may not work as reliably as it does with a local router.

The context meter

The bar in the top bar shows how much of the active model's context window the next request will occupy: system prompt, prior turns, attachments, and whatever is currently in the input box.

While you type it is an estimate derived from chars_per_token, shown with a leading ~. When a reply finishes, the server reports the real token counts in the stream and the meter switches to those, dropping the ~.

Expect the estimate to read low on models with a reasoning budget. Thinking tokens count against the context but cannot be known before the reply arrives, so a model that thinks for 2000 tokens will jump well past the estimate once it answers. The exact figure after each turn is the one to trust.

The limit comes from ctx-size in presets.ini. Models the router offers but that file does not describe show no limit.

Markdown rendering

Replies are parsed by QTextDocument.setMarkdown(), so there is no markdown dependency. It runs in the GitHub dialect with the MarkdownNoHTML flag, which means markup a model emits is displayed literally rather than being interpreted: a reply containing <img src=x onerror=...> shows that text and nothing executes.

Markdown links are still real links, so a reply cannot forge one that drives the interface: anchors using the internal x-llamachat-reasoning: scheme are rewritten before display, and only http, https and mailto links are handed to the desktop opener.

Qt renders a fenced block as one <pre> element per line with no background of its own, so llamachat adds the tint that makes a code block read as one. Qt also ignores padding and vertical margins on <pre>, so the inset comes from left and right margins and the room above and below from a tinted spacer line. Without an explicit white-space:pre-wrap a long line would widen the whole transcript: Qt's own markdown stylesheet wraps p and li but not pre, and that stylesheet is dropped when only the body fragment is kept.

Each code block gets a copy button below it, on the right. Qt rich text has no widgets and no :hover, so the button is an anchor in the same internal scheme dispatch the thinking and search toggles use, and hovering is a re-render driven by QTextBrowser.highlighted(). It sits below the block because Qt cannot float it over the tint. The button is grey at rest, takes the highlight colour under the pointer, and becomes a green tick for a moment once used; the tick reverts on its own rather than standing indefinitely and implying the clipboard still holds that block. The text is copied from the source markdown, not the rendered HTML, which per-line <pre> elements would make unrecoverable.

Notes and limits

Switching models makes the router unload the previous one. The first token after a switch can take several seconds; the window shows a waiting state rather than pretending it is instant.
Reasoning is read from the reasoning_content field the router sends alongside content, not by parsing <think> tags out of the reply. How much of it a model produces is set by reasoning-budget in presets.ini; set that to 0 for a preset that should not think at all.
The attachment budget is an estimate based on a characters-per-token ratio, not a real tokeniser. It is deliberately conservative.
The tray icon needs a StatusNotifierItem host. With waybar, that means the tray module in your config. Without one, the icon silently does nothing and the keybind still works.

License

GPL-2.0-only. See LICENSE.

Development Approach

This project is developed using AI-assisted tools. Code is generated with the help of AI based on human-provided specifications, design decisions, and iterative feedback.

All contributions are reviewed, tested, and curated by the maintainer before being included in the codebase. AI is used as a productivity and exploration tool, while human oversight remains central to all decisions.

The goal is to combine the flexibility of AI-assisted development with standard open-source practices such as transparency, review, and accountability.