diff options
Diffstat (limited to 'desktop')
| -rw-r--r-- | desktop/README.md | 3 | ||||
| -rw-r--r-- | desktop/modules/ai/README.md | 51 |
2 files changed, 53 insertions, 1 deletions
diff --git a/desktop/README.md b/desktop/README.md index 4d1e622..f51feb4 100644 --- a/desktop/README.md +++ b/desktop/README.md @@ -30,8 +30,9 @@ the IPC layer rejects. The page-less grid entry is therefore the zero-argument modules/mail/ unread per account, threads, the watcher dot modules/vm/ libvirt state, live stats, snapshots modules/appearance/ a tile that opens the separate appearance shell + modules/ai/ local AI engines, start/stop for the apps using them -Sound, mail, vm, network, bluetooth and kdeconnect each carry their own README. +Sound, mail, vm, network, bluetooth, kdeconnect and ai each carry their own README. Appearance is a tile onto the separate appearance shell, whose README covers the picker it opens. diff --git a/desktop/modules/ai/README.md b/desktop/modules/ai/README.md new file mode 100644 index 0000000..7498f06 --- /dev/null +++ b/desktop/modules/ai/README.md @@ -0,0 +1,51 @@ +# ai + +What is using the local AI stack, and start/stop for the projects built on it. + +The tile counts running apps and names any engine that is generating, or says +`Idle`. The page lists the two engines, then the apps. + +## Engines + +llama.cpp (`llama-server`, router mode, port 8181) and stable-diffusion.cpp +(`sd-server`, started by `~/bin/sd-webui`). Both rows are read-only: each +engine also serves a web UI, and stopping it would take that down. + +llama's state comes from its API: `/models` for the loaded model, then +`/slots?model=<id>&autoload=false` for whether a slot is processing. The +`autoload=false` matters: a plain `/slots?model=` on an unloaded model loads +it into VRAM, which is how this was found. + +The sd row covers two processes. A running `sd-cli`, the one-shot runs +fanfictioner makes, reads up and generating with its model marked `(sd-cli)`, +whatever the web UI is doing: it holds VRAM for as long as it lives. The first +version watched only `sd-server` and showed the engine down mid-run. + +sd-server has no progress endpoint. Its log, `~/.cache/sd-server.log`, streams +progress bars while it generates, so the row reads busy while the log was +written in the last 5s. A step slower than that reads idle. + +## Apps + +| App | Status | Start | Stop | +| --- | --- | --- | --- | +| desktop-assistant | `assistant.sh status` | `assistant.sh start` | `assistant.sh stop` | +| local chat | `llamachat --ping` | `llamachat --daemon` | `llamachat --quit` | +| imggen | `imggen status` | `imggen start` (default model) | `imggen stop` | +| fanfictioner | `pgrep` on `python3 .../fanfictioner` | none | TERM its children, then it, by pid | + +Each runs detached, so no exit code comes back; the next poll shows whether it +worked. `assistant.sh status` fails when either the assistant or its overlay +is down, so a dead overlay reads as stopped and Start restarts it. + +## Service lifetime + +`alwaysActive: false`. `ai-state.sh` runs every 3s while the drawer is open, +gated on the tile's lifetime the same way kdeconnect is, and 1s after any +action. + +## Test + + ./test-ai-state.sh + +Stubs every external command on PATH; touches nothing live. |