aboutsummaryrefslogtreecommitdiffstats
path: root/README.md
diff options
context:
space:
mode:
authorDanilo M. <danix@danix.xyz>2026-07-31 10:35:56 +0200
committerDanilo M. <danix@danix.xyz>2026-07-31 10:35:56 +0200
commit325a6a7c12832d8b9fb9b67567e8a1c496dc1330 (patch)
treee2128aa71ca2f739f1ddc78060a980fbe06eacbc /README.md
downloadllamachat-325a6a7c12832d8b9fb9b67567e8a1c496dc1330.tar.gz
llamachat-325a6a7c12832d8b9fb9b67567e8a1c496dc1330.zip
feat: initial release of llamachat 0.1.0v0.1.0
A native PySide6 chat client for a local llama.cpp server in router mode. Runs as a single persistent process with a Unix-socket control channel, so a Hyprland keybind toggles the window with a socket round trip rather than a process start. The control commands do not import Qt and answer in under a tenth of a second. Features: one-shot and multi-turn chat modes, runtime model discovery from /v1/models, streaming replies rendered as markdown, collapsible reasoning for models with a thinking budget, drag-and-drop file and image attachment with context-aware truncation, and SQLite history with FTS5 search. Markdown is parsed with MarkdownNoHTML so markup in a reply is displayed rather than interpreted, and links a reply produces cannot drive the interface. Includes the Hyprland Lua snippets for autostart, keybind and window rules, and a self-check suite covering everything except the GUI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Diffstat (limited to 'README.md')
-rw-r--r--README.md324
1 files changed, 324 insertions, 0 deletions
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..7c05beb
--- /dev/null
+++ b/README.md
@@ -0,0 +1,324 @@
+# llamachat
+
+A small native desktop chat client for a local llama.cpp server running in
+router mode. PySide6, no browser, no Electron. It runs as a single
+persistent process and toggles like a scratchpad from a Hyprland keybind.
+
+## Features
+
+- **Two modes.** One-shot for a quick question with no context carried over,
+ Chat for a resumable multi-turn conversation. The toggle sits in the top
+ bar, not in a menu.
+- **Model picker** populated at runtime from the router's `/v1/models`. Models
+ with an `mmproj` file in `presets.ini` are marked with an eye.
+- **File attachment** by drag-and-drop anywhere on the window or through the
+ file dialog. Text and code files are inlined into the prompt, truncated to
+ fit the model's context with a warning when that happens. Images require a
+ vision model, and dropping one on a text model offers to switch.
+- **History** in SQLite with FTS5 full-text search. Past chat sessions reopen
+ and continue with their context intact; one-shot entries reopen read-only.
+- **Streaming replies** rendered token by token, formatted as markdown:
+ headings, bold and italic, bullet and numbered lists, tables, inline code
+ and tinted fenced code blocks. What you type is shown exactly as typed, so
+ the contents of an attached file are never reflowed.
+- **Collapsed reasoning.** Presets with a reasoning budget return the model's
+ thinking separately from its answer. It shows as a one-line `▸ thinking`
+ summary above the reply, and clicking that expands it. Each reply collapses
+ independently and the state is remembered per bubble.
+- **Tray icon** for show/hide/quit, hosted by waybar's tray module.
+
+Not in this version: RAG or embedding search over history, multi-user
+support, any cloud or non-local backend, remote access.
+
+## Requirements
+
+- Python 3.11 or newer (3.12 tested)
+- PySide6, httpx, Pillow
+- A llama.cpp server in router mode with an OpenAI-compatible API
+
+Pillow is only used for attachment thumbnails.
+
+## Installation
+
+PySide6 is not in SlackBuilds, so the simplest route on Slackware is a venv.
+`--system-site-packages` lets httpx and Pillow come from the system packages
+instead of being duplicated:
+
+```sh
+python3 -m venv --system-site-packages ~/.local/share/llamachat-venv
+~/.local/share/llamachat-venv/bin/pip install PySide6-Essentials
+```
+
+Then put the launcher on your PATH:
+
+```sh
+git clone <repo> ~/src/llamachat
+ln -s ~/src/llamachat/llamachat.py ~/bin/llamachat
+```
+
+The launcher starts under the system Python and hands over to an interpreter
+that has PySide6, so no activation step is needed and the command works from
+any directory. It looks for `.venv/` or `venv/` beside the checkout, then
+`~/.local/share/llamachat-venv/`. Set `LLAMACHAT_PYTHON` to name a specific
+interpreter, or `LLAMACHAT_NO_REEXEC=1` to disable the handover entirely.
+
+Install the desktop entry, which lets the portal resolve the application id
+and quiets a startup warning:
+
+```sh
+ln -s ~/src/llamachat/llamachat.desktop \
+ ~/.local/share/applications/llamachat.desktop
+```
+
+Verify:
+
+```sh
+llamachat --help
+cd ~/src/llamachat && ./test_llamachat.py
+```
+
+### Qt theming from a venv
+
+A pip-installed PySide6 bundles its own Qt build with almost no plugins, so
+`QT_QPA_PLATFORMTHEME=qt6ct` finds nothing to load and every window falls
+back to Fusion, ignoring qt6ct and Kvantum entirely.
+
+llamachat handles this at startup: if the system Qt6 is the *same version* as
+the one PySide6 bundles, it points `QT_PLUGIN_PATH` at the system plugin
+directory so the real platform theme loads. The version check matters,
+because a plugin built against a different Qt would crash rather than merely
+look wrong. Setting `QT_PLUGIN_PATH` yourself disables this and your value is
+used unchanged.
+
+Check what a mismatch would look like:
+
+```sh
+python3 -c 'from PySide6.QtCore import qVersion; print("bundled", qVersion())'
+ls /usr/lib64/libQt6Core.so.6.*
+```
+
+If those differ, the theme will not be picked up. Either install a PySide6
+matching the system Qt (`pip install "PySide6-Essentials==6.11.*"` for a
+6.11 system) or build PySide6 against the system Qt.
+
+## Configuration
+
+`~/.config/llamachat/config.toml` is written with defaults on first run.
+
+```toml
+base_url = "http://localhost:8181"
+presets = "/etc/llama-server/presets.ini"
+
+# Empty values take the XDG defaults:
+# socket -> $XDG_RUNTIME_DIR/llamachat.sock
+# db -> $XDG_DATA_HOME/llamachat/history.db
+socket = ""
+db = ""
+
+default_model = ""
+request_timeout = 300
+
+attach_ctx_fraction = 0.5
+chars_per_token = 3.5
+```
+
+`presets.ini` is read for two things the API does not report: which models
+have an `mmproj` file, and each model's `ctx-size`. Both `;` and `#` count as
+comment characters there, matching llama-server, so a commented-out `mmproj`
+line correctly reads as "no vision support".
+
+An attachment may fill `ctx_size * chars_per_token * attach_ctx_fraction`
+characters before it is truncated. With the defaults and a 32k model that is
+roughly 57000 characters. Models the router offers but `presets.ini` does not
+describe fall back to a conservative 4096-token assumption.
+
+## Usage
+
+```
+llamachat --daemon start the background instance
+llamachat --toggle show if hidden, hide if visible
+llamachat --show show the window
+llamachat --hide hide the window
+llamachat --ping exit 0 when an instance is running
+llamachat --quit stop the running instance
+```
+
+The control commands do not import Qt, so they return in well under a tenth
+of a second, which is what makes them usable behind a keybind.
+
+In the window: **Ctrl+Enter** sends, **Esc** hides, **New** starts a fresh
+conversation, right-clicking a history entry offers to delete it.
+
+## Hyprland configuration
+
+For Hyprland 0.55+ with the Lua config system. Paths below assume the
+`~/.config/hypr/sections/` layout.
+
+### Autostart
+
+In `sections/autostart.lua`, inside the existing `hl.on("hyprland.start", ...)`
+block:
+
+```lua
+hl.exec_cmd("llamachat --daemon")
+```
+
+### Keybind
+
+In `sections/keybindings.lua`:
+
+```lua
+hl.bind(mainMod .. " + Home", hl.dsp.exec_cmd("llamachat --toggle"))
+```
+
+### Window rules
+
+In `sections/look_and_feel.lua`:
+
+```lua
+-- llamachat: floating scratchpad-style window
+hl.window_rule({
+ name = "llamachat-float",
+ match = { class = "llamachat" },
+ float = true,
+})
+hl.window_rule({
+ name = "llamachat-size",
+ match = { class = "llamachat" },
+ size = { 1000, 700 },
+})
+hl.window_rule({
+ name = "llamachat-center",
+ match = { class = "llamachat" },
+ center = true,
+})
+```
+
+If `center` is not supported by your Hyprland Lua plugin, pin the geometry
+instead. `move` is monitor-relative, so for a 2560x1080 DP-1 and a 1000x700
+window the offsets are x=(2560-1000)/2=780 and y=(1080-700)/2=190:
+
+```lua
+hl.window_rule({
+ name = "llamachat-monitor",
+ match = { class = "llamachat" },
+ monitor = "DP-1",
+})
+hl.window_rule({
+ name = "llamachat-move",
+ match = { class = "llamachat" },
+ move = { 780, 190 },
+})
+```
+
+The window class is `llamachat`. Class matching in this plugin is a
+regex/substring match, so no escaping is needed.
+
+Reload with `hyprctl reload` after editing. Window rules apply at window-open
+time, so if the daemon was already running when you added them, restart it
+with `llamachat --quit` followed by `llamachat --daemon`.
+
+## Architecture
+
+```
+llamachat.py launcher, venv shebang, symlink target
+llamachat/
+ __main__.py argument dispatch, daemon, tray, socket event loop
+ config.py config.toml and presets.ini parsing
+ backend.py httpx streaming client, attachment loading
+ db.py SQLite schema, FTS5 search
+ ui.py the window
+test_llamachat.py self-checks for everything except the GUI
+```
+
+### Control protocol
+
+One JSON object per line over a Unix socket, one response line per request.
+The socket is created mode 0600 in `$XDG_RUNTIME_DIR`.
+
+```
+-> {"cmd": "toggle"} <- {"ok": true, "visible": false}
+-> {"cmd": "show"} <- {"ok": true, "visible": true}
+-> {"cmd": "hide"} <- {"ok": true, "visible": false}
+-> {"cmd": "ping"} <- {"ok": true}
+-> {"cmd": "quit"} <- {"ok": true}
+```
+
+Anything else answers `{"ok": false, "error": "..."}`. A socket left behind by
+a crashed instance is detected with a ping and removed at startup. The
+listening socket is serviced by a `QSocketNotifier` on the Qt event loop, so
+there is no second thread.
+
+### Database schema
+
+```sql
+sessions (id, mode, title, model, created_at, updated_at)
+messages (id, session_id, role, content, reasoning, created_at)
+attachments (id, message_id, path, kind, mime, size, sha256, thumb, truncated)
+messages_fts -- FTS5 external-content table over messages.content
+```
+
+Three triggers keep the FTS index in step with inserts, updates and deletes,
+which matters because a streamed reply is inserted empty and updated once it
+finishes.
+
+`reasoning` holds the model's thinking, kept in its own column for two
+reasons: the FTS index covers `content` only, so thinking text cannot bury
+real search hits, and only `content` is replayed when continuing a
+conversation, which is what these APIs expect. Databases created before this
+column existed gain it automatically on open.
+
+Attachment *content* lives inline in `messages.content`, since that is what
+was actually sent to the model. The `attachments` table records provenance:
+the original path, size, and a SHA-256 so a file can be identified even if it
+was later moved or renamed, plus a 128px JPEG thumbnail for images so a
+reopened conversation still shows what was sent even when the original file
+is gone. Full image bytes are not stored.
+
+Search input is tokenised and quoted before it reaches FTS5, so punctuation
+that would otherwise be read as query syntax cannot cause an error.
+
+### Markdown rendering
+
+Replies are parsed by `QTextDocument.setMarkdown()`, so there is no markdown
+dependency. It runs in the GitHub dialect with the `MarkdownNoHTML` flag,
+which means markup a model emits is displayed literally rather than being
+interpreted: a reply containing `<img src=x onerror=...>` shows that text and
+nothing executes.
+
+Markdown links are still real links, so a reply cannot forge one that drives
+the interface: anchors using the internal `x-llamachat-reasoning:` scheme are
+rewritten before display, and only `http`, `https` and `mailto` links are
+handed to the desktop opener.
+
+Qt renders a fenced block as one `<pre>` element per line with no background
+of its own, so llamachat adds the tint that makes a code block read as one.
+
+## Notes and limits
+
+- Switching models makes the router unload the previous one. The first token
+ after a switch can take several seconds; the window shows a waiting state
+ rather than pretending it is instant.
+- Reasoning is read from the `reasoning_content` field the router sends
+ alongside `content`, not by parsing `<think>` tags out of the reply. How
+ much of it a model produces is set by `reasoning-budget` in `presets.ini`;
+ set that to `0` for a preset that should not think at all.
+- The attachment budget is an estimate based on a characters-per-token ratio,
+ not a real tokeniser. It is deliberately conservative.
+- The tray icon needs a StatusNotifierItem host. With waybar, that means the
+ `tray` module in your config. Without one, the icon silently does nothing
+ and the keybind still works.
+
+## License
+
+GPL-2.0-only. See [LICENSE](LICENSE).
+
+Copyright (C) 2026 Danilo M. <danix@danix.xyz>
+
+## Development Approach
+
+This project is developed using AI-assisted tools. Code is generated with the help of AI based on human-provided specifications, design decisions, and iterative feedback.
+
+All contributions are reviewed, tested, and curated by the maintainer before being included in the codebase. AI is used as a productivity and exploration tool, while human oversight remains central to all decisions.
+
+The goal is to combine the flexibility of AI-assisted development with standard open-source practices such as transparency, review, and accountability.