# Desktop companion: design Date: 2026-10-23. Status: designed, not implemented. ## Goal A small chibi 2B (NieR: Automata) who lives in the bottom-right corner of the focused monitor, follows focus between DP-1 and DP-3, and reacts to what happens on the desktop: notifications shove her, typing makes her watch, deleting makes her mock, the assistant gets a face, idling bores her, clicks make her blush and, at seven, self-destruct. She talks in short fixed English lines in a speech bubble. SUPER+\ shows and hides her. The idea comes from Modesty's window companion, read for ideas only (GPLv3; see AGENTS.md). Nothing here is taken from its code. The difference that shapes everything: this desktop runs mostly maximised windows, so she lives in a screen corner, not on a window edge. ## Verified before designing - **The art can be made locally.** Pony Diffusion V6 XL with two LoRAs makes the base frame, img2img makes expressions, FLUX.2 klein makes poses. Recipes, prompts and what failed are in `~/.local/share/keystone/companion/2b/recipe.txt`. Measured on the way: img2img at strength 0.75 changes a face and not an arm, and at 0.85 moves a hand halfway and starts to drift; klein moves limbs and keeps the character when every detail to keep is named. - **sd.cpp 929 cannot load an SDXL diffusers directory**: it detects it as SD 1.x (leejet/stable-diffusion.cpp#1938). sd-webui's Pony entry now uses the single-file checkpoint, which loads as SDXL. - **A resident llama.cpp model starves image generation.** Gemma left 1.1 GB of 12 GB free and the first unet segment failed; an img2img encode needed 4.5 GB. sd-webui unloads llama-server's models at start. - **The white background comes off cleanly.** ImageMagick, fuzz 6%, flood fill from the four corners, trim, scale to 256 px tall: the line art stops the fill, so the white hair and the white leotard survive. - **The look was checked on screen** with a throwaway shell: 128 px tall, 24 px from the right edge and 16 px from the bottom of DP-1, a dark translucent bubble up and to the left of her head. Approved by eye. - **Balloons are bottom-right of DP-1** (`~/.config/notifyd/config.json`, `"corner": "bottom-right"`), her corner. `shared/Notify.qml` already exposes the queue, DND, snooze and the drawer state; the check of which queued balloons are on screen lives in `notifications/`. - **Hyprland 0.56.2 has what the inputs need**, per `/usr/share/hypr/stubs/hl.meta.lua`: an `input.keyboard.key` event, a `non_consuming` bind flag, and an `above_fullscreen` layer rule. None has been tried on this machine yet; see Open checks. - **SUPER+\ is free**, and the keyboard carries two layouts, `it,us`. The `backslash` keysym sits on a different physical key in each, so the bind uses the physical key, `code:49`, left of 1. ## Where she lives A component of `desktop/`, a sibling of `island/` and `assistant/`, not a drawer module: she has no tile and no page. `desktop/` already hosts the assistant, so she reads its state directly. The cost, accepted: restarting `desktop/` restarts her. ``` desktop/companion/ Companion.qml the window, inputs, timers, IPC Figure.qml draws a frame; crossfade, bob, squash, explosion Mood.js which state wins, as a pure function (.pragma library) test_mood.js node check of Mood.js, like assistant/test_md.js Lines.js her English lines, grouped by reaction prep-frames bash + ImageMagick: generated images to frames/ README.md the component's notes ``` `desktop/shell.qml` creates one `Companion { asst: assistant }`, the way `Island` gets `asst: assistant`. ### The window A `PanelWindow` anchored bottom and right, on the focused monitor (`Hyprland.focusedMonitor`), layer `Top`, `ExclusionMode.Ignore`, `WlrKeyboardFocus.None`, namespace `keystone-companion`. Its `mask` is her body only, so clicks anywhere else reach the window below; the bubble is not in the mask either. It is the second window that exists for the whole session, alongside the keepalive. Hidden, it keeps existing with an empty mask and nothing drawn, so a toggle never creates or destroys a surface. Moving between monitors plays her exit on the old one and her entrance on the new: the window's screen changes only once the exit has finished. ### Art Never in the repo: 2B is Square Enix's character and the repo is public under GPLv2. Generated images and their recipe live in `~/.local/share/keystone/companion/2b/`; `prep-frames` writes the prepared PNGs, 256 px tall with a transparent background, to `frames/` inside it. Two outfits, the same frames in each: `frames/dress/.png` and `frames/leotard/.png`. The component loads the current outfit's `` and falls back to that outfit's `base`, never to the other outfit, so a missing frame cannot put the skirt back mid-scene. It works from the first commit with only `dress/base`; until `leotard/base` exists, the self-destruct ends in the dress. **She stands.** `base` is standing, facing the viewer, arms relaxed: lying down is what boredom ends in, so sitting as the rest pose made it no change at all. The first base was sitting; it stays in the art directory as `base-sitting.png`, the source the standing one is edited from. The `smile` and `wave` made from it are redone from the standing base, so every frame descends from one reference. | Method | Frames, per outfit | | --- | --- | | klein edit of `base-sitting.png` | `base`: "make her stand upright, facing the viewer, arms relaxed at her sides" | | img2img from `base`, strength 0.75 | `smile`, `mad`, `smirk`, `blush1`, `blush2`, `blush3`, `sleepy` | | klein edit of `base` | `wave`, `watch`, `point`, `yawn`, `lying`, `listen`, `think`, `muse`, `stretch`, `shoved`, `sing` | | klein edit of the dress frame | every `leotard/`: "her skirt is gone, revealing a plain white leotard; change nothing else" | | klein edit of `leotard/base` | `scorched`, leotard only: the moment after the explosion | Each leotard frame is made from its own dress frame rather than from `leotard/base`, so the pose comes along unchanged. 18 frames a set, 37 in all with `scorched`. Every prompt is in `desktop/companion/frames.txt`, and `gen-frames` makes them through the running sd-server. Dodging, shoving, the drop-in and the explosion are QML motion, not frames. She is drawn 128 px tall; both monitors are at scale 1. No size preference: one would need a control in the settings window, and nothing asks for it yet. ## Behaviour One state at a time, the highest that applies. `Mood.js` takes the inputs and the time and returns the state; `Companion.qml` owns the inputs and timers. | Priority | State | Trigger | Frame | Ends | | --- | --- | --- | --- | --- | | 1 | Self-destruct | 7th click of a burst | glow, flash, explosion, then `leotard/scorched` | see below | | 2 | Dodge | pointer rests on her 0.6 s with no click | slides out of the corner | returns after 5 s | | 3 | Shoved | a balloon is on screen on her monitor | `shoved`, slid left by the stack's width | balloons gone | | 4 | Assistant | recording / working / reply arrived | `listen` / `think` / `wave` | state ends; wave lasts 3 s | | 5 | Singing | Sony headphones connected | `sing` | 4 s | | 6 | Clicks | clicks 1-6 of a burst; a burst ends after 1.5 s without one | `smile`, `blush1` to `blush3` | 2 s after the last click | | 7 | Mock | BackSpace, at most once in 20 s | `smirk`, `point` | 2.5 s | | 8 | Watching | typing pulse in the last 2 s | `watch` | 2 s after typing stops | | 9 | Idle | no input for 1 / 3 / 5 min | `sleepy` / `yawn` / `lying` | any input: `stretch`, then base | | 10 | Base | nothing else | `base` | | - **Self-destruct, as in the game.** She glows, says "Self-destruct sequence initiated.", a white flash covers the sprite for about a second with particles and a shockwave ring, and she is back in place without her skirt (`leotard/scorched` for 3 s), saying "...That was unpleasant." The outfit is then `leotard` for 60 s, or until the next hide and show, whichever is first, and every state plays as usual from the leotard set. The outfit is the only thing the self-destruct changes: `Mood.js` picks the state, the outfit picks the folder. - **Dodge.** She cannot feel the pointer until it is over her: Quickshell has no global pointer and Hyprland sends no motion events. A click before the 0.6 s still reaches her, so clicking her keeps working. If the pointer is still there when she returns, she dodges again. - **Shoved** only on DP-1, where the balloons are. A balloon counts when the balloon shell would show it: not withheld by DND, snooze or the open drawer, not expired. That check moves into `shared/Notify.qml` as one function both shells call, so they cannot disagree. - **Assistant.** Recording is `asst.recording`; working is `asst.busy` (`thinking`, `tool`, `transcribing`); "reply arrived" is the reply ending, the same moment the canvas's done notification fires. - **Idle** is Quickshell's `IdleMonitor`, respecting inhibitors, so presentation mode keeps her awake. - **Hidden** while the focused window is fullscreen, while the screen is locked, and while toggled off. ### Speech Fixed English lines from `Lines.js`, picked at random per reaction, for example "...Operator, I'm bored.", "Typo again?", "Hey.", "Answer's ready.", "Glory to mankind.". A bubble shows 3 s. At most one line per 30 s, except clicks and the self-destruct. The game's opening monologue, verbatim: > Everything that lives is designed to end. We are perpetually trapped in > a never-ending spiral of life and death. Is this a curse? Or some kind > of punishment? I often think about the god who blessed us with this > cryptic puzzle...and wonder if we'll ever get the chance to kill him. It plays with a 1-in-4 chance on entering bored and 1-in-3 on being shown with SUPER+\, never more than once an hour. It types out at about 15 characters a second, one sentence per bubble, the last held 4 s, over the `muse` frame. Any state above idle cuts it short. ## Inputs from Hyprland In the live config, `~/.config/hypr/sections/`, not this repo: - `hl.bind("SUPER + code:49", ...)` runs `qs -p ipc call companion toggle`. - A hook on `input.keyboard.key` dispatches a custom `keystone-typing` event at most once a second. It reads no key data. - A `non_consuming` bind on BackSpace dispatches `keystone-delete`, so the key still reaches the focused window. BackSpace is the only key the companion learns about. `Companion.qml` reads both through `Hyprland.rawEvent`, the stream the island already uses. ## Testing - `node desktop/companion/test_mood.js`: the priority table, the click burst (1 smiles, 2-6 blush harder, 7 self-destructs, a 1.5 s gap resets), the 20 s mock cooldown, the 30 s line cooldown, the hourly monologue cap, the idle thresholds. - `prep-frames` checked on `base.png`: alpha present, 256 px tall, the hair not eaten (corner pixel transparent, a hair pixel opaque). - By hand, since it is visual: each reaction once, monitor hand-over, fullscreen and lock hiding, the dodge, and that BackSpace still deletes in a terminal and a browser with the bind active. ## Open checks Each is tested before the code that depends on it: 1. A `non_consuming` BackSpace bind still deletes in the focused window. 2. The `input.keyboard.key` hook fires and its dispatch reaches `Hyprland.rawEvent`. 3. A `Top` layer surface is covered by a fullscreen window, so the fullscreen rule may be the compositor's and not code. If not, hide on the focused toplevel's fullscreen flag. ## Out of scope - Generated lines (Gemma): slow, and it would hold VRAM against sd-webui. - A size preference or any other setting. - Following the focused window's edge, Modesty-style. - Italian lines.