diff options
| author | Danilo M. <danix@danix.xyz> | 2026-10-28 09:15:55 +0100 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-10-28 09:15:55 +0100 |
| commit | 779492e8b40084032e218c89f211e60527a28b98 (patch) | |
| tree | 30a99cfa626374a60a58688558516802a74aea7b /desktop/modules/ai/AiModule.qml | |
| parent | e59745a38b56d2517b8c006279ea6d07f8047678 (diff) | |
| download | quickshell-779492e8b40084032e218c89f211e60527a28b98.tar.gz quickshell-779492e8b40084032e218c89f211e60527a28b98.zip | |
ai: unload llama's model from the engine row
The llama.cpp row offers Unload while a model is loaded. It posts to the
router's /models/unload, which frees the VRAM and leaves llama-server up,
so its web UI stays available; the engines themselves are still never
stopped from here. The answer is not read: the next poll shows the row
without a model, or with it if the unload failed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'desktop/modules/ai/AiModule.qml')
| -rw-r--r-- | desktop/modules/ai/AiModule.qml | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/desktop/modules/ai/AiModule.qml b/desktop/modules/ai/AiModule.qml index 6df3d6b..d58ebfe 100644 --- a/desktop/modules/ai/AiModule.qml +++ b/desktop/modules/ai/AiModule.qml @@ -85,6 +85,13 @@ Module { exec(["sh", "-c", 'pkill -TERM -P "$1"; kill -TERM "$1"', "sh", app.detail]); } + // The router frees the model's VRAM; the next request for it loads it + // again. Its answer is not read, the next poll shows whether it worked. + function unload(model) { + exec(["curl", "-s", "-m", "30", "-X", "POST", "-H", "Content-Type: application/json", + "-d", JSON.stringify({ model: model }), "http://127.0.0.1:8181/models/unload"]); + } + // Detached, so a start outlives a quickshell reload. No exit code comes // back; the next poll shows whether it worked. function exec(cmd) { |
