path: root/README.md

# llamachat

A small native desktop chat client for a local llama.cpp server running in
router mode. PySide6, no browser, no Electron. It runs as a single
persistent process and toggles like a scratchpad from a Hyprland keybind.

## Features

- **Two modes.** One-shot for a quick question with no context carried over,
  Chat for a resumable multi-turn conversation. The toggle sits in the top
  bar, not in a menu.
- **Model picker** populated at runtime from the router's `/v1/models`. Models
  with an `mmproj` file in `presets.ini` are marked with an eye.
- **File attachment** by drag-and-drop anywhere on the window or through the
  file dialog. Text and code files are inlined into the prompt, truncated to
  fit the model's context with a warning when that happens. Images require a
  vision model, and dropping one on a text model offers to switch.
- **History** in SQLite with FTS5 full-text search. Past chat sessions reopen
  and continue with their context intact; one-shot entries reopen read-only.
- **Streaming replies** rendered token by token, formatted as markdown:
  headings, bold and italic, bullet and numbered lists, tables, inline code
  and tinted fenced code blocks. What you type is shown exactly as typed, so
  the contents of an attached file are never reflowed.
- **Collapsed reasoning.** Presets with a reasoning budget return the model's
  thinking separately from its answer. It shows as a one-line `▸ thinking`
  summary above the reply, and clicking that expands it. Each reply collapses
  independently and the state is remembered per bubble.
- **Context meter** in the top bar showing how much of the model's context
  window the next request will use. It estimates while you type and becomes
  exact after each reply, turning amber at 75% and red at 90%.
- **System prompts** kept as markdown files in `~/.config/llamachat/prompts/`.
  A global `default.md` applies to new chats, named presets replace it, and
  any conversation can take a one-off custom prompt or none at all. The
  choice is stored per session, so reopening an old chat restores the prompt
  it was built with.
- **Tray icon** for show/hide/quit, hosted by waybar's tray module.

Not in this version: RAG or embedding search over history, multi-user
support, any cloud or non-local backend, remote access.

## Requirements

- Python 3.11 or newer (3.12 tested)
- PySide6, httpx, Pillow
- A llama.cpp server in router mode with an OpenAI-compatible API

Pillow is only used for attachment thumbnails.

## Installation

PySide6 is not in SlackBuilds, so the simplest route on Slackware is a venv.
`--system-site-packages` lets httpx and Pillow come from the system packages
instead of being duplicated:

```sh
python3 -m venv --system-site-packages ~/.local/share/llamachat-venv
~/.local/share/llamachat-venv/bin/pip install PySide6-Essentials
```

Then put the launcher on your PATH:

```sh
git clone <repo> ~/src/llamachat
ln -s ~/src/llamachat/llamachat.py ~/bin/llamachat
```

The launcher starts under the system Python and hands over to an interpreter
that has PySide6, so no activation step is needed and the command works from
any directory. It looks for `.venv/` or `venv/` beside the checkout, then
`~/.local/share/llamachat-venv/`. Set `LLAMACHAT_PYTHON` to name a specific
interpreter, or `LLAMACHAT_NO_REEXEC=1` to disable the handover entirely.

Install the desktop entry, which lets the portal resolve the application id
and quiets a startup warning:

```sh
ln -s ~/src/llamachat/llamachat.desktop \
      ~/.local/share/applications/llamachat.desktop
```

Verify:

```sh
llamachat --help
cd ~/src/llamachat && ./test_llamachat.py
```

### Qt theming from a venv

A pip-installed PySide6 bundles its own Qt build with almost no plugins, so
`QT_QPA_PLATFORMTHEME=qt6ct` finds nothing to load and every window falls
back to Fusion, ignoring qt6ct and Kvantum entirely.

llamachat handles this at startup: if the system Qt6 is the *same version* as
the one PySide6 bundles, it points `QT_PLUGIN_PATH` at the system plugin
directory so the real platform theme loads. The version check matters,
because a plugin built against a different Qt would crash rather than merely
look wrong. Setting `QT_PLUGIN_PATH` yourself disables this and your value is
used unchanged.

Check what a mismatch would look like:

```sh
python3 -c 'from PySide6.QtCore import qVersion; print("bundled", qVersion())'
ls /usr/lib64/libQt6Core.so.6.*
```

If those differ, the theme will not be picked up. Either install a PySide6
matching the system Qt (`pip install "PySide6-Essentials==6.11.*"` for a
6.11 system) or build PySide6 against the system Qt.

## Configuration

`~/.config/llamachat/config.toml` is written with defaults on first run.

```toml
base_url = "http://localhost:8181"
presets = "/etc/llama-server/presets.ini"

# Empty values take the XDG defaults:
#   socket -> $XDG_RUNTIME_DIR/llamachat.sock
#   db     -> $XDG_DATA_HOME/llamachat/history.db
socket = ""
db = ""

default_model = ""
request_timeout = 300

# System prompt for new conversations: a preset name from prompts/,
# "none" for no system prompt, or empty for the global default.md.
default_prompt = ""

attach_ctx_fraction = 0.5
chars_per_token = 3.5
```

`presets.ini` is read for two things the API does not report: which models
have an `mmproj` file, and each model's `ctx-size`. Both `;` and `#` count as
comment characters there, matching llama-server, so a commented-out `mmproj`
line correctly reads as "no vision support".

An attachment may fill `ctx_size * chars_per_token * attach_ctx_fraction`
characters before it is truncated. With the defaults and a 32k model that is
roughly 57000 characters. Models the router offers but `presets.ini` does not
describe fall back to a conservative 4096-token assumption.

## Usage

```
llamachat --daemon    start the background instance
llamachat --toggle    show if hidden, hide if visible
llamachat --show      show the window
llamachat --hide      hide the window
llamachat --ping      exit 0 when an instance is running
llamachat --quit      stop the running instance
```

The control commands do not import Qt, so they return in well under a tenth
of a second, which is what makes them usable behind a keybind.

In the window: **Ctrl+Enter** sends, **Ctrl+\\** shows or hides the history
panel (the `☰` button does the same), **Esc** hides the window, **New**
starts a fresh conversation, and right-clicking a history entry offers to
delete it.

The panel's width and whether it was hidden are remembered in
`~/.config/llamachat/state.ini` between runs.

## Hyprland configuration

For Hyprland 0.55+ with the Lua config system. Paths below assume the
`~/.config/hypr/sections/` layout.

### Autostart

In `sections/autostart.lua`, inside the existing `hl.on("hyprland.start", ...)`
block:

```lua
hl.exec_cmd("llamachat --daemon")
```

### Keybind

In `sections/keybindings.lua`:

```lua
hl.bind(mainMod .. " + Home",          hl.dsp.exec_cmd("llamachat --toggle"))
```

### Window rules

In `sections/look_and_feel.lua`:

```lua
-- llamachat: floating scratchpad-style window
hl.window_rule({
    name  = "llamachat-float",
    match = { class = "llamachat" },
    float = true,
})
hl.window_rule({
    name  = "llamachat-size",
    match = { class = "llamachat" },
    size  = { 1000, 700 },
})
hl.window_rule({
    name   = "llamachat-center",
    match  = { class = "llamachat" },
    center = true,
})
```

If `center` is not supported by your Hyprland Lua plugin, pin the geometry
instead. `move` is monitor-relative, so for a 2560x1080 DP-1 and a 1000x700
window the offsets are x=(2560-1000)/2=780 and y=(1080-700)/2=190:

```lua
hl.window_rule({
    name    = "llamachat-monitor",
    match   = { class = "llamachat" },
    monitor = "DP-1",
})
hl.window_rule({
    name  = "llamachat-move",
    match = { class = "llamachat" },
    move  = { 780, 190 },
})
```

The window class is `llamachat`. Class matching in this plugin is a
regex/substring match, so no escaping is needed.

Reload with `hyprctl reload` after editing. Window rules apply at window-open
time, so if the daemon was already running when you added them, restart it
with `llamachat --quit` followed by `llamachat --daemon`.

## Architecture

```
llamachat.py           launcher, venv shebang, symlink target
llamachat/
  __main__.py          argument dispatch, daemon, tray, socket event loop
  config.py            config.toml and presets.ini parsing
  backend.py           httpx streaming client, attachment loading
  db.py                SQLite schema, FTS5 search
  ui.py                the window
test_llamachat.py      self-checks for everything except the GUI
```

### Control protocol

One JSON object per line over a Unix socket, one response line per request.
The socket is created mode 0600 in `$XDG_RUNTIME_DIR`.

```
-> {"cmd": "toggle"}    <- {"ok": true, "visible": false}
-> {"cmd": "show"}      <- {"ok": true, "visible": true}
-> {"cmd": "hide"}      <- {"ok": true, "visible": false}
-> {"cmd": "ping"}      <- {"ok": true}
-> {"cmd": "quit"}      <- {"ok": true}
```

Anything else answers `{"ok": false, "error": "..."}`. A socket left behind by
a crashed instance is detected with a ping and removed at startup. The
listening socket is serviced by a `QSocketNotifier` on the Qt event loop, so
there is no second thread.

### Database schema

```sql
sessions    (id, mode, title, model, prompt_name, prompt_custom,
             created_at, updated_at)
messages    (id, session_id, role, content, reasoning, created_at)
attachments (id, message_id, path, kind, mime, size, sha256, thumb, truncated)
messages_fts -- FTS5 external-content table over messages.content
```

Three triggers keep the FTS index in step with inserts, updates and deletes,
which matters because a streamed reply is inserted empty and updated once it
finishes.

`reasoning` holds the model's thinking, kept in its own column for two
reasons: the FTS index covers `content` only, so thinking text cannot bury
real search hits, and only `content` is replayed when continuing a
conversation, which is what these APIs expect.

`prompt_name` records which system prompt a conversation was built with, and
`prompt_custom` holds the text when that prompt is a one-off rather than a
file. Storing the name rather than the resolved text means editing a preset
affects the conversations that use it, which is usually what you want; a
conversation that needed frozen wording should use a custom prompt.

Databases created before any of these columns existed gain them
automatically on open.

Attachment *content* lives inline in `messages.content`, since that is what
was actually sent to the model. The `attachments` table records provenance:
the original path, size, and a SHA-256 so a file can be identified even if it
was later moved or renamed, plus a 128px JPEG thumbnail for images so a
reopened conversation still shows what was sent even when the original file
is gone. Full image bytes are not stored.

Search input is tokenised and quoted before it reaches FTS5, so punctuation
that would otherwise be read as query syntax cannot cause an error.

### System prompts

Prompts are markdown files in `~/.config/llamachat/prompts/`, one per file,
created on first run:

```
prompts/
  default.md      the global prompt, used unless something overrides it
  coding.md       a named preset
  terse.md        another
```

They are plain files on purpose, so they can be edited in an editor, diffed,
and kept in version control. The `…` button beside the picker opens a dialog
that does the same thing from inside the app.

The picker offers every file plus two extras. A named preset **replaces** the
global prompt rather than being appended to it. `custom for this chat…` takes
one-off text stored with that conversation, and `no system prompt` sends none
at all.

`default_prompt` in `config.toml` chooses what new conversations start with:
a preset name, `"none"`, or empty for `default.md`.

The choice is recorded per session, so reopening a chat restores the prompt
it was built with rather than silently applying whatever is selected now.

### The context meter

The bar in the top bar shows how much of the active model's context window
the next request will occupy: system prompt, prior turns, attachments, and
whatever is currently in the input box.

While you type it is an estimate derived from `chars_per_token`, shown with a
leading `~`. When a reply finishes, the server reports the real token counts
in the stream and the meter switches to those, dropping the `~`.

Expect the estimate to read low on models with a reasoning budget. Thinking
tokens count against the context but cannot be known before the reply
arrives, so a model that thinks for 2000 tokens will jump well past the
estimate once it answers. The exact figure after each turn is the one to
trust.

The limit comes from `ctx-size` in `presets.ini`. Models the router offers
but that file does not describe show no limit.

### Markdown rendering

Replies are parsed by `QTextDocument.setMarkdown()`, so there is no markdown
dependency. It runs in the GitHub dialect with the `MarkdownNoHTML` flag,
which means markup a model emits is displayed literally rather than being
interpreted: a reply containing `<img src=x onerror=...>` shows that text and
nothing executes.

Markdown links are still real links, so a reply cannot forge one that drives
the interface: anchors using the internal `x-llamachat-reasoning:` scheme are
rewritten before display, and only `http`, `https` and `mailto` links are
handed to the desktop opener.

Qt renders a fenced block as one `<pre>` element per line with no background
of its own, so llamachat adds the tint that makes a code block read as one.

## Notes and limits

- Switching models makes the router unload the previous one. The first token
  after a switch can take several seconds; the window shows a waiting state
  rather than pretending it is instant.
- Reasoning is read from the `reasoning_content` field the router sends
  alongside `content`, not by parsing `<think>` tags out of the reply. How
  much of it a model produces is set by `reasoning-budget` in `presets.ini`;
  set that to `0` for a preset that should not think at all.
- The attachment budget is an estimate based on a characters-per-token ratio,
  not a real tokeniser. It is deliberately conservative.
- The tray icon needs a StatusNotifierItem host. With waybar, that means the
  `tray` module in your config. Without one, the icon silently does nothing
  and the keybind still works.

## License

GPL-2.0-only. See [LICENSE](LICENSE).

Copyright (C) 2026 Danilo M. <danix@danix.xyz>

## Development Approach

This project is developed using AI-assisted tools. Code is generated with the help of AI based on human-provided specifications, design decisions, and iterative feedback.

All contributions are reviewed, tested, and curated by the maintainer before being included in the codebase. AI is used as a productivity and exploration tool, while human oversight remains central to all decisions.

The goal is to combine the flexibility of AI-assisted development with standard open-source practices such as transparency, review, and accountability.