diff options
| author | Danilo M. <danix@danix.xyz> | 2026-07-31 10:35:56 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-07-31 10:35:56 +0200 |
| commit | 325a6a7c12832d8b9fb9b67567e8a1c496dc1330 (patch) | |
| tree | e2128aa71ca2f739f1ddc78060a980fbe06eacbc /README.md | |
| download | llamachat-325a6a7c12832d8b9fb9b67567e8a1c496dc1330.tar.gz llamachat-325a6a7c12832d8b9fb9b67567e8a1c496dc1330.zip | |
feat: initial release of llamachat 0.1.0v0.1.0
A native PySide6 chat client for a local llama.cpp server in router mode.
Runs as a single persistent process with a Unix-socket control channel, so
a Hyprland keybind toggles the window with a socket round trip rather than
a process start. The control commands do not import Qt and answer in under
a tenth of a second.
Features: one-shot and multi-turn chat modes, runtime model discovery from
/v1/models, streaming replies rendered as markdown, collapsible reasoning
for models with a thinking budget, drag-and-drop file and image attachment
with context-aware truncation, and SQLite history with FTS5 search.
Markdown is parsed with MarkdownNoHTML so markup in a reply is displayed
rather than interpreted, and links a reply produces cannot drive the
interface.
Includes the Hyprland Lua snippets for autostart, keybind and window rules,
and a self-check suite covering everything except the GUI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 324 |
1 files changed, 324 insertions, 0 deletions
diff --git a/README.md b/README.md new file mode 100644 index 0000000..7c05beb --- /dev/null +++ b/README.md @@ -0,0 +1,324 @@ +# llamachat + +A small native desktop chat client for a local llama.cpp server running in +router mode. PySide6, no browser, no Electron. It runs as a single +persistent process and toggles like a scratchpad from a Hyprland keybind. + +## Features + +- **Two modes.** One-shot for a quick question with no context carried over, + Chat for a resumable multi-turn conversation. The toggle sits in the top + bar, not in a menu. +- **Model picker** populated at runtime from the router's `/v1/models`. Models + with an `mmproj` file in `presets.ini` are marked with an eye. +- **File attachment** by drag-and-drop anywhere on the window or through the + file dialog. Text and code files are inlined into the prompt, truncated to + fit the model's context with a warning when that happens. Images require a + vision model, and dropping one on a text model offers to switch. +- **History** in SQLite with FTS5 full-text search. Past chat sessions reopen + and continue with their context intact; one-shot entries reopen read-only. +- **Streaming replies** rendered token by token, formatted as markdown: + headings, bold and italic, bullet and numbered lists, tables, inline code + and tinted fenced code blocks. What you type is shown exactly as typed, so + the contents of an attached file are never reflowed. +- **Collapsed reasoning.** Presets with a reasoning budget return the model's + thinking separately from its answer. It shows as a one-line `▸ thinking` + summary above the reply, and clicking that expands it. Each reply collapses + independently and the state is remembered per bubble. +- **Tray icon** for show/hide/quit, hosted by waybar's tray module. + +Not in this version: RAG or embedding search over history, multi-user +support, any cloud or non-local backend, remote access. + +## Requirements + +- Python 3.11 or newer (3.12 tested) +- PySide6, httpx, Pillow +- A llama.cpp server in router mode with an OpenAI-compatible API + +Pillow is only used for attachment thumbnails. + +## Installation + +PySide6 is not in SlackBuilds, so the simplest route on Slackware is a venv. +`--system-site-packages` lets httpx and Pillow come from the system packages +instead of being duplicated: + +```sh +python3 -m venv --system-site-packages ~/.local/share/llamachat-venv +~/.local/share/llamachat-venv/bin/pip install PySide6-Essentials +``` + +Then put the launcher on your PATH: + +```sh +git clone <repo> ~/src/llamachat +ln -s ~/src/llamachat/llamachat.py ~/bin/llamachat +``` + +The launcher starts under the system Python and hands over to an interpreter +that has PySide6, so no activation step is needed and the command works from +any directory. It looks for `.venv/` or `venv/` beside the checkout, then +`~/.local/share/llamachat-venv/`. Set `LLAMACHAT_PYTHON` to name a specific +interpreter, or `LLAMACHAT_NO_REEXEC=1` to disable the handover entirely. + +Install the desktop entry, which lets the portal resolve the application id +and quiets a startup warning: + +```sh +ln -s ~/src/llamachat/llamachat.desktop \ + ~/.local/share/applications/llamachat.desktop +``` + +Verify: + +```sh +llamachat --help +cd ~/src/llamachat && ./test_llamachat.py +``` + +### Qt theming from a venv + +A pip-installed PySide6 bundles its own Qt build with almost no plugins, so +`QT_QPA_PLATFORMTHEME=qt6ct` finds nothing to load and every window falls +back to Fusion, ignoring qt6ct and Kvantum entirely. + +llamachat handles this at startup: if the system Qt6 is the *same version* as +the one PySide6 bundles, it points `QT_PLUGIN_PATH` at the system plugin +directory so the real platform theme loads. The version check matters, +because a plugin built against a different Qt would crash rather than merely +look wrong. Setting `QT_PLUGIN_PATH` yourself disables this and your value is +used unchanged. + +Check what a mismatch would look like: + +```sh +python3 -c 'from PySide6.QtCore import qVersion; print("bundled", qVersion())' +ls /usr/lib64/libQt6Core.so.6.* +``` + +If those differ, the theme will not be picked up. Either install a PySide6 +matching the system Qt (`pip install "PySide6-Essentials==6.11.*"` for a +6.11 system) or build PySide6 against the system Qt. + +## Configuration + +`~/.config/llamachat/config.toml` is written with defaults on first run. + +```toml +base_url = "http://localhost:8181" +presets = "/etc/llama-server/presets.ini" + +# Empty values take the XDG defaults: +# socket -> $XDG_RUNTIME_DIR/llamachat.sock +# db -> $XDG_DATA_HOME/llamachat/history.db +socket = "" +db = "" + +default_model = "" +request_timeout = 300 + +attach_ctx_fraction = 0.5 +chars_per_token = 3.5 +``` + +`presets.ini` is read for two things the API does not report: which models +have an `mmproj` file, and each model's `ctx-size`. Both `;` and `#` count as +comment characters there, matching llama-server, so a commented-out `mmproj` +line correctly reads as "no vision support". + +An attachment may fill `ctx_size * chars_per_token * attach_ctx_fraction` +characters before it is truncated. With the defaults and a 32k model that is +roughly 57000 characters. Models the router offers but `presets.ini` does not +describe fall back to a conservative 4096-token assumption. + +## Usage + +``` +llamachat --daemon start the background instance +llamachat --toggle show if hidden, hide if visible +llamachat --show show the window +llamachat --hide hide the window +llamachat --ping exit 0 when an instance is running +llamachat --quit stop the running instance +``` + +The control commands do not import Qt, so they return in well under a tenth +of a second, which is what makes them usable behind a keybind. + +In the window: **Ctrl+Enter** sends, **Esc** hides, **New** starts a fresh +conversation, right-clicking a history entry offers to delete it. + +## Hyprland configuration + +For Hyprland 0.55+ with the Lua config system. Paths below assume the +`~/.config/hypr/sections/` layout. + +### Autostart + +In `sections/autostart.lua`, inside the existing `hl.on("hyprland.start", ...)` +block: + +```lua +hl.exec_cmd("llamachat --daemon") +``` + +### Keybind + +In `sections/keybindings.lua`: + +```lua +hl.bind(mainMod .. " + Home", hl.dsp.exec_cmd("llamachat --toggle")) +``` + +### Window rules + +In `sections/look_and_feel.lua`: + +```lua +-- llamachat: floating scratchpad-style window +hl.window_rule({ + name = "llamachat-float", + match = { class = "llamachat" }, + float = true, +}) +hl.window_rule({ + name = "llamachat-size", + match = { class = "llamachat" }, + size = { 1000, 700 }, +}) +hl.window_rule({ + name = "llamachat-center", + match = { class = "llamachat" }, + center = true, +}) +``` + +If `center` is not supported by your Hyprland Lua plugin, pin the geometry +instead. `move` is monitor-relative, so for a 2560x1080 DP-1 and a 1000x700 +window the offsets are x=(2560-1000)/2=780 and y=(1080-700)/2=190: + +```lua +hl.window_rule({ + name = "llamachat-monitor", + match = { class = "llamachat" }, + monitor = "DP-1", +}) +hl.window_rule({ + name = "llamachat-move", + match = { class = "llamachat" }, + move = { 780, 190 }, +}) +``` + +The window class is `llamachat`. Class matching in this plugin is a +regex/substring match, so no escaping is needed. + +Reload with `hyprctl reload` after editing. Window rules apply at window-open +time, so if the daemon was already running when you added them, restart it +with `llamachat --quit` followed by `llamachat --daemon`. + +## Architecture + +``` +llamachat.py launcher, venv shebang, symlink target +llamachat/ + __main__.py argument dispatch, daemon, tray, socket event loop + config.py config.toml and presets.ini parsing + backend.py httpx streaming client, attachment loading + db.py SQLite schema, FTS5 search + ui.py the window +test_llamachat.py self-checks for everything except the GUI +``` + +### Control protocol + +One JSON object per line over a Unix socket, one response line per request. +The socket is created mode 0600 in `$XDG_RUNTIME_DIR`. + +``` +-> {"cmd": "toggle"} <- {"ok": true, "visible": false} +-> {"cmd": "show"} <- {"ok": true, "visible": true} +-> {"cmd": "hide"} <- {"ok": true, "visible": false} +-> {"cmd": "ping"} <- {"ok": true} +-> {"cmd": "quit"} <- {"ok": true} +``` + +Anything else answers `{"ok": false, "error": "..."}`. A socket left behind by +a crashed instance is detected with a ping and removed at startup. The +listening socket is serviced by a `QSocketNotifier` on the Qt event loop, so +there is no second thread. + +### Database schema + +```sql +sessions (id, mode, title, model, created_at, updated_at) +messages (id, session_id, role, content, reasoning, created_at) +attachments (id, message_id, path, kind, mime, size, sha256, thumb, truncated) +messages_fts -- FTS5 external-content table over messages.content +``` + +Three triggers keep the FTS index in step with inserts, updates and deletes, +which matters because a streamed reply is inserted empty and updated once it +finishes. + +`reasoning` holds the model's thinking, kept in its own column for two +reasons: the FTS index covers `content` only, so thinking text cannot bury +real search hits, and only `content` is replayed when continuing a +conversation, which is what these APIs expect. Databases created before this +column existed gain it automatically on open. + +Attachment *content* lives inline in `messages.content`, since that is what +was actually sent to the model. The `attachments` table records provenance: +the original path, size, and a SHA-256 so a file can be identified even if it +was later moved or renamed, plus a 128px JPEG thumbnail for images so a +reopened conversation still shows what was sent even when the original file +is gone. Full image bytes are not stored. + +Search input is tokenised and quoted before it reaches FTS5, so punctuation +that would otherwise be read as query syntax cannot cause an error. + +### Markdown rendering + +Replies are parsed by `QTextDocument.setMarkdown()`, so there is no markdown +dependency. It runs in the GitHub dialect with the `MarkdownNoHTML` flag, +which means markup a model emits is displayed literally rather than being +interpreted: a reply containing `<img src=x onerror=...>` shows that text and +nothing executes. + +Markdown links are still real links, so a reply cannot forge one that drives +the interface: anchors using the internal `x-llamachat-reasoning:` scheme are +rewritten before display, and only `http`, `https` and `mailto` links are +handed to the desktop opener. + +Qt renders a fenced block as one `<pre>` element per line with no background +of its own, so llamachat adds the tint that makes a code block read as one. + +## Notes and limits + +- Switching models makes the router unload the previous one. The first token + after a switch can take several seconds; the window shows a waiting state + rather than pretending it is instant. +- Reasoning is read from the `reasoning_content` field the router sends + alongside `content`, not by parsing `<think>` tags out of the reply. How + much of it a model produces is set by `reasoning-budget` in `presets.ini`; + set that to `0` for a preset that should not think at all. +- The attachment budget is an estimate based on a characters-per-token ratio, + not a real tokeniser. It is deliberately conservative. +- The tray icon needs a StatusNotifierItem host. With waybar, that means the + `tray` module in your config. Without one, the icon silently does nothing + and the keybind still works. + +## License + +GPL-2.0-only. See [LICENSE](LICENSE). + +Copyright (C) 2026 Danilo M. <danix@danix.xyz> + +## Development Approach + +This project is developed using AI-assisted tools. Code is generated with the help of AI based on human-provided specifications, design decisions, and iterative feedback. + +All contributions are reviewed, tested, and curated by the maintainer before being included in the codebase. AI is used as a productivity and exploration tool, while human oversight remains central to all decisions. + +The goal is to combine the flexibility of AI-assisted development with standard open-source practices such as transparency, review, and accountability. |
