# llamachat

A small native desktop chat client for a local llama.cpp server running in
router mode. PySide6, no browser, no Electron. It runs as a single
persistent process and toggles like a scratchpad from a Hyprland keybind.

## Features

- **Two modes.** One-shot for a quick question with no context carried over,
  Chat for a resumable multi-turn conversation. The toggle sits in the top
  bar, not in a menu.
- **Model picker** populated at runtime from the router's `/v1/models`. Models
  with an `mmproj` file in `presets.ini` are marked with an eye.
- **File attachment** by drag-and-drop anywhere on the window or through the
  file dialog. Text and code files are inlined into the prompt, truncated to
  fit the model's context with a warning when that happens. Images require a
  vision model, and dropping one on a text model offers to switch.
- **History** in SQLite with FTS5 full-text search. Past chat sessions reopen
  and continue with their context intact; one-shot entries reopen read-only.
  Sessions are named by the model itself: once the first reply lands, it is
  asked to title the exchange in a few words, which replaces the opening
  words of the question the entry was created with.
- **Streaming replies** rendered token by token, formatted as markdown:
  headings, bold and italic, bullet and numbered lists, tables, inline code
  and tinted fenced code blocks. What you type is shown exactly as typed, so
  the contents of an attached file are never reflowed.
- **Collapsed reasoning.** Presets with a reasoning budget return the model's
  thinking separately from its answer. It shows as a one-line `▸ thinking`
  summary above the reply, and clicking that expands it. Each reply collapses
  independently and the state is remembered per bubble.
- **Context meter** in the top bar showing how much of the model's context
  window the next request will use. It estimates while you type and becomes
  exact after each reply, turning amber at 75% and red at 90%.
- **System prompts** kept as markdown files in `~/.config/llamachat/prompts/`.
  A global `default.md` applies to new chats, named presets replace it, and
  any conversation can take a one-off custom prompt or none at all. The
  choice is stored per session, so reopening an old chat restores the prompt
  it was built with.
- **Web search** through a SearXNG instance, off by default. The model is
  offered a `web_search` tool and calls it when a question needs current
  information; the queries and their sources show as a collapsible block
  above the reply. See [Web search](#web-search).
- **Tray icon** for show/hide/quit, hosted by waybar's tray module.

Not in this version: RAG or embedding search over history, multi-user
support, remote access.

## Requirements

- Python 3.11 or newer (3.12 tested)
- PySide6, httpx, Pillow
- A llama.cpp server in router mode with an OpenAI-compatible API

Pillow is only used for attachment thumbnails.

## Installation

PySide6 is not in SlackBuilds, so the simplest route on Slackware is a venv.
`--system-site-packages` lets httpx and Pillow come from the system packages
instead of being duplicated:

```sh
python3 -m venv --system-site-packages ~/.local/share/llamachat-venv
~/.local/share/llamachat-venv/bin/pip install PySide6-Essentials
```

Then put the launcher on your PATH:

```sh
git clone <repo> ~/src/llamachat
ln -s ~/src/llamachat/llamachat.py ~/bin/llamachat
```

The launcher starts under the system Python and hands over to an interpreter
that has PySide6, so no activation step is needed and the command works from
any directory. It looks for `.venv/` or `venv/` beside the checkout, then
`~/.local/share/llamachat-venv/`. Set `LLAMACHAT_PYTHON` to name a specific
interpreter, or `LLAMACHAT_NO_REEXEC=1` to disable the handover entirely.

Install the desktop entry, which lets the portal resolve the application id
and quiets a startup warning:

```sh
ln -s ~/src/llamachat/llamachat.desktop \
      ~/.local/share/applications/llamachat.desktop
```

Verify:

```sh
llamachat --help
cd ~/src/llamachat && ./test_llamachat.py
```

### Qt theming from a venv

A pip-installed PySide6 bundles its own Qt build with almost no plugins, so
`QT_QPA_PLATFORMTHEME=qt6ct` finds nothing to load and every window falls
back to Fusion, ignoring qt6ct and Kvantum entirely.

llamachat handles this at startup: if the system Qt6 is the *same version* as
the one PySide6 bundles, it points `QT_PLUGIN_PATH` at the system plugin
directory so the real platform theme loads. The version check matters,
because a plugin built against a different Qt would crash rather than merely
look wrong. Setting `QT_PLUGIN_PATH` yourself disables this and your value is
used unchanged.

Check what a mismatch would look like:

```sh
python3 -c 'from PySide6.QtCore import qVersion; print("bundled", qVersion())'
ls /usr/lib64/libQt6Core.so.6.*
```

If those differ, the theme will not be picked up. Either install a PySide6
matching the system Qt (`pip install "PySide6-Essentials==6.11.*"` for a
6.11 system) or build PySide6 against the system Qt.

## Configuration

`~/.config/llamachat/config.toml` is written with defaults on first run.

```toml
base_url = "http://localhost:8181"
presets = "/etc/llama-server/presets.ini"

# Empty values take the XDG defaults:
#   socket -> $XDG_RUNTIME_DIR/llamachat.sock
#   db     -> $XDG_DATA_HOME/llamachat/history.db
socket = ""
db = ""

default_model = ""
request_timeout = 300

# System prompt for new conversations: a preset name from prompts/,
# "none" for no system prompt, or empty for the global default.md.
default_prompt = ""

attach_ctx_fraction = 0.5
chars_per_token = 3.5

# Web search. Both keys are required: a flag with no URL stays off.
search_enabled = false
search_url = ""

search_results = 5
search_snippet_chars = 300
search_timeout = 10
max_searches = 2
```

`presets.ini` is read for two things the API does not report: which models
have an `mmproj` file, and each model's `ctx-size`. Both `;` and `#` count as
comment characters there, matching llama-server, so a commented-out `mmproj`
line correctly reads as "no vision support".

An attachment may fill `ctx_size * chars_per_token * attach_ctx_fraction`
characters before it is truncated. With the defaults and a 32k model that is
roughly 57000 characters. Models the router offers but `presets.ini` does not
describe fall back to a conservative 4096-token assumption.

## Usage

```
llamachat --daemon    start the background instance
llamachat --toggle    show if hidden, hide if visible
llamachat --show      show the window
llamachat --hide      hide the window
llamachat --ping      exit 0 when an instance is running
llamachat --quit      stop the running instance
```

The control commands do not import Qt, so they return in well under a tenth
of a second, which is what makes them usable behind a keybind.

In the window: **Ctrl+Enter** sends, **Ctrl+\\** shows or hides the history
panel (the `☰` button does the same), **Esc** hides the window, **New**
starts a fresh conversation, and right-clicking a history entry offers to
delete it.

The panel's width and whether it was hidden are remembered in
`~/.config/llamachat/state.ini` between runs.

## Hyprland configuration

For Hyprland 0.55+ with the Lua config system. Paths below assume the
`~/.config/hypr/sections/` layout.

### Autostart

In `sections/autostart.lua`, inside the existing `hl.on("hyprland.start", ...)`
block:

```lua
hl.exec_cmd("llamachat --daemon")
```

### Keybind

In `sections/keybindings.lua`:

```lua
hl.bind(mainMod .. " + Home",          hl.dsp.exec_cmd("llamachat --toggle"))
```

### Window rules

In `sections/look_and_feel.lua`:

```lua
-- llamachat: floating scratchpad-style window
hl.window_rule({
    name  = "llamachat-float",
    match = { class = "llamachat" },
    float = true,
})
hl.window_rule({
    name  = "llamachat-size",
    match = { class = "llamachat" },
    size  = { 1000, 700 },
})
hl.window_rule({
    name   = "llamachat-center",
    match  = { class = "llamachat" },
    center = true,
})
```

If `center` is not supported by your Hyprland Lua plugin, pin the geometry
instead. `move` is monitor-relative, so for a 2560x1080 DP-1 and a 1000x700
window the offsets are x=(2560-1000)/2=780 and y=(1080-700)/2=190:

```lua
hl.window_rule({
    name    = "llamachat-monitor",
    match   = { class = "llamachat" },
    monitor = "DP-1",
})
hl.window_rule({
    name  = "llamachat-move",
    match = { class = "llamachat" },
    move  = { 780, 190 },
})
```

The window class is `llamachat`. Class matching in this plugin is a
regex/substring match, so no escaping is needed.

Reload with `hyprctl reload` after editing. Window rules apply at window-open
time, so if the daemon was already running when you added them, restart it
with `llamachat --quit` followed by `llamachat --daemon`.

## Architecture

```
llamachat.py           launcher, venv shebang, symlink target
llamachat/
  __main__.py          argument dispatch, daemon, tray, socket event loop
  config.py            config.toml and presets.ini parsing
  backend.py           httpx streaming client, tool loop, attachment loading
  search.py            SearXNG queries, tool schema, result sanitising
  db.py                SQLite schema, FTS5 search
  ui.py                the window
test_llamachat.py      self-checks for everything except the GUI
```

### Control protocol

One JSON object per line over a Unix socket, one response line per request.
The socket is created mode 0600 in `$XDG_RUNTIME_DIR`.

```
-> {"cmd": "toggle"}    <- {"ok": true, "visible": false}
-> {"cmd": "show"}      <- {"ok": true, "visible": true}
-> {"cmd": "hide"}      <- {"ok": true, "visible": false}
-> {"cmd": "ping"}      <- {"ok": true}
-> {"cmd": "quit"}      <- {"ok": true}
```

Anything else answers `{"ok": false, "error": "..."}`. A socket left behind by
a crashed instance is detected with a ping and removed at startup. The
listening socket is serviced by a `QSocketNotifier` on the Qt event loop, so
there is no second thread.

### Database schema

```sql
sessions    (id, mode, title, model, prompt_name, prompt_custom,
             created_at, updated_at)
messages    (id, session_id, role, content, reasoning, searches, created_at)
attachments (id, message_id, path, kind, mime, size, sha256, thumb, truncated)
messages_fts -- FTS5 external-content table over messages.content
```

Three triggers keep the FTS index in step with inserts, updates and deletes,
which matters because a streamed reply is inserted empty and updated once it
finishes.

`reasoning` holds the model's thinking, kept in its own column for two
reasons: the FTS index covers `content` only, so thinking text cannot bury
real search hits, and only `content` is replayed when continuing a
conversation, which is what these APIs expect.

`searches` holds a JSON array of what was looked up for that reply, one entry
per search with its query, results and any error, or NULL when nothing was
searched. Like `reasoning` it stays out of the FTS index, or snippets from
web pages would compete with messages the user actually wrote.

`prompt_name` records which system prompt a conversation was built with, and
`prompt_custom` holds the text when that prompt is a one-off rather than a
file. Storing the name rather than the resolved text means editing a preset
affects the conversations that use it, which is usually what you want; a
conversation that needed frozen wording should use a custom prompt.

Databases created before any of these columns existed gain them
automatically on open.

Attachment *content* lives inline in `messages.content`, since that is what
was actually sent to the model. The `attachments` table records provenance:
the original path, size, and a SHA-256 so a file can be identified even if it
was later moved or renamed, plus a 128px JPEG thumbnail for images so a
reopened conversation still shows what was sent even when the original file
is gone. Full image bytes are not stored.

Search input is tokenised and quoted before it reaches FTS5, so punctuation
that would otherwise be read as query syntax cannot cause an error.

### System prompts

Prompts are markdown files in `~/.config/llamachat/prompts/`, one per file,
created on first run:

```
prompts/
  default.md      the global prompt, used unless something overrides it
  coding.md       a named preset
  terse.md        another
```

They are plain files on purpose, so they can be edited in an editor, diffed,
and kept in version control. The `…` button beside the picker opens a dialog
that does the same thing from inside the app.

The picker offers every file plus two extras. A named preset **replaces** the
global prompt rather than being appended to it. `custom for this chat…` takes
one-off text stored with that conversation, and `no system prompt` sends none
at all.

`default_prompt` in `config.toml` chooses what new conversations start with:
a preset name, `"none"`, or empty for `default.md`.

The choice is recorded per session, so reopening a chat restores the prompt
it was built with rather than silently applying whatever is selected now.

### Web search

Off unless both `search_enabled = true` and a `search_url` are set. A flag
with no URL behind it resolves to disabled at startup, so an upgrade never
starts making network requests on its own.

The model decides when to search. It is offered an OpenAI-style `web_search`
function, and when it judges a question needs current information it calls
it; llamachat queries SearXNG's JSON API, hands back the results, and the
model answers from them. Nothing needs enabling on the llama-server side:
this is plain tool calling, not llama-server's `--tools` (which runs
server-side tools, none of which search the web) and not `--ui-mcp-proxy`
(a CORS shim that exists for browsers).

The SearXNG instance must have its JSON output format enabled, which many
installs turn off by default. Without it, queries fail with a message saying
so.

Searches show in the transcript as a collapsible block above the thinking
block, with a status line while one runs:

```
▸ searched: latest stable linux kernel (5 results)
▸ thinking (1441 chars, 38 lines)
Model:
...
```

Expanding it lists each result's title, link and snippet. The queries are
visible on purpose: when an answer is wrong it is usually the query that was
wrong, and without seeing it a bad search and a bad answer look identical.

`max_searches` caps searches per turn, default 2. On the last round the tool
is withdrawn from the request and the tool result says so in words, which is
what forces the model to answer instead of searching again. A local 9B will
otherwise keep searching when it is unsure.

Withdrawing the schema on its own is not enough, because the model cannot
see a schema disappear. It asks for another search regardless, and that
request surfaces either as literal `<tool_call><function=web_search>` text
in the reply or buried in the thinking block, leaving the reply itself
empty. Saying "this was your last search" in the tool result is what fixes
it. The note has to ride on the tool result rather than a trailing system
message: Qwen3.5's chat template rejects those outright, raising `System
message must be at the beginning`.

If replies still come back empty after a search, check the server's KV
cache. llama.cpp's own function-calling documentation warns that quantized
KV degrades tool calling, and it does: with `cache-type-k`/`cache-type-v`
set to `q8_0`, three of five turns on one question came back broken, and
dropping to the `f16` default made the same question answer six times out
of six.

Turns that search once are reliable. Turns that genuinely need two are not
quite: on a question forcing two rounds, four and five of six succeeded
across two runs, the rest coming back empty. Lower `max_searches` to 1 if
you would rather never see an empty reply, or if you keep the quantized
cache. Enabling more search engines helps either way, since a first round
that returns plenty removes the reason to search twice.

Failures do not abort the turn. A timeout, a refused connection, a non-JSON
response or zero results all come back to the model as a tool result saying
what happened, so it answers with that knowledge rather than silently
falling back on stale training data. The block then reads
`▸ search failed: <query> — <reason>`.

A search that returns nothing *because every engine failed* counts as a
failure rather than an empty result, and the message names the engines. On
a self-hosted SearXNG the usual way search stops working is engines getting
rate-limited or CAPTCHA'd one by one, and reporting that as "no results"
tells the model the web is empty, which invites exactly the confident wrong
answer this feature exists to prevent.

When search is on, the current date is appended to the system prompt. A
model has no clock, so it falls back on its training cutoff and writes that
year into the search query itself, poisoning the results before they are
fetched. It is also told not to put a year in a query unless asked. This is
added even when `no system prompt` is selected, since the model has no clock
either way.

**On trusting results.** Search snippets are attacker-influenced text going
into the model's context, in an app whose output you may paste into a
terminal. Structurally: only `title`, `url` and the snippet are kept and
everything else SearXNG returns is dropped, snippets are truncated to
`search_snippet_chars`, all result text is escaped when displayed, result
links are shown but never fetched automatically, and a reply that forges the
search block's internal URL scheme has it defused. The tool result is also
labelled untrusted for the model, which helps inconsistently.

None of that stops a poisoned snippet from influencing what the model says.
If a top result asserts something false, a 9B may repeat it. That is why the
sources stay visible rather than being hidden behind the answer.

### External providers

llamachat can talk to OpenAI-compatible cloud providers alongside the local
router. Cloud models appear in the picker as `provider:model`. Local models
stay bare, so an existing local setup needs no changes.

Providers live in `[providers.*]` tables. The generated config ships a
commented together.ai block; uncomment it and set `api_key` to enable it:

```toml
[providers.together]
base_url = "https://api.together.xyz"
api_key = "pass:api/together"
filter = ["qwen", "deepseek"]
ctx_size = 32768
price_in = 0.60
price_out = 0.60
```

`api_key` accepts three forms:

- `pass:api/together` reads the key from the password store. This is
  preferred, because the secret never sits in a config file that might be
  committed or copied.
- `env:TOGETHER_KEY` reads the key from the environment.
- `sk-...` is the literal key, written into `config.toml` itself. Avoid
  this: it puts a secret in a plain-text file.

Keys are resolved lazily, on the first request to that provider. A session
that only touches local models never triggers a pinentry or reads an
environment variable.

`filter` keeps only models whose id contains one of the given strings,
case-insensitively. Providers list hundreds of models, and without a filter
the picker is unusable. Omit `filter` to list everything.

`ctx_size`, `vision`, `price_in` and `price_out` prefill the per-model
settings dialog. Prices are US dollars per million tokens. The dialog saves
anything you enter to `models.ini` beside `config.toml`, so the provider
block can stay minimal. `models.ini` is machine-written cache, not
user-editable config.

The cost readout in the top bar shows what has been spent in the current
conversation plus what the next message would add. It is approximate, based
entirely on the hand-entered prices in the provider block or the per-model
dialog. It is useful for seeing when a long conversation is becoming
expensive.

**A conversation resends its whole history every turn.** That is free with a
local router, but on a cloud provider every message is billed for the entire
conversation so far. The projection in the cost label exists to make that
visible before you send.

Tool calling and reasoning output vary between providers. Web search on a
cloud model may not work as reliably as it does with a local router.

### The context meter

The bar in the top bar shows how much of the active model's context window
the next request will occupy: system prompt, prior turns, attachments, and
whatever is currently in the input box.

While you type it is an estimate derived from `chars_per_token`, shown with a
leading `~`. When a reply finishes, the server reports the real token counts
in the stream and the meter switches to those, dropping the `~`.

Expect the estimate to read low on models with a reasoning budget. Thinking
tokens count against the context but cannot be known before the reply
arrives, so a model that thinks for 2000 tokens will jump well past the
estimate once it answers. The exact figure after each turn is the one to
trust.

The limit comes from `ctx-size` in `presets.ini`. Models the router offers
but that file does not describe show no limit.

### Markdown rendering

Replies are parsed by `QTextDocument.setMarkdown()`, so there is no markdown
dependency. It runs in the GitHub dialect with the `MarkdownNoHTML` flag,
which means markup a model emits is displayed literally rather than being
interpreted: a reply containing `<img src=x onerror=...>` shows that text and
nothing executes.

Markdown links are still real links, so a reply cannot forge one that drives
the interface: anchors using the internal `x-llamachat-reasoning:` scheme are
rewritten before display, and only `http`, `https` and `mailto` links are
handed to the desktop opener.

Qt renders a fenced block as one `<pre>` element per line with no background
of its own, so llamachat adds the tint that makes a code block read as one.

## Notes and limits

- Switching models makes the router unload the previous one. The first token
  after a switch can take several seconds; the window shows a waiting state
  rather than pretending it is instant.
- Reasoning is read from the `reasoning_content` field the router sends
  alongside `content`, not by parsing `<think>` tags out of the reply. How
  much of it a model produces is set by `reasoning-budget` in `presets.ini`;
  set that to `0` for a preset that should not think at all.
- The attachment budget is an estimate based on a characters-per-token ratio,
  not a real tokeniser. It is deliberately conservative.
- The tray icon needs a StatusNotifierItem host. With waybar, that means the
  `tray` module in your config. Without one, the icon silently does nothing
  and the keybind still works.

## License

GPL-2.0-only. See [LICENSE](LICENSE).

Copyright (C) 2026 Danilo M. <danix@danix.xyz>

## Development Approach

This project is developed using AI-assisted tools. Code is generated with the help of AI based on human-provided specifications, design decisions, and iterative feedback.

All contributions are reviewed, tested, and curated by the maintainer before being included in the codebase. AI is used as a productivity and exploration tool, while human oversight remains central to all decisions.

The goal is to combine the flexibility of AI-assisted development with standard open-source practices such as transparency, review, and accountability.
