diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/superpowers/specs/2026-10-23-companion-design.md | 215 |
1 files changed, 215 insertions, 0 deletions
diff --git a/docs/superpowers/specs/2026-10-23-companion-design.md b/docs/superpowers/specs/2026-10-23-companion-design.md new file mode 100644 index 0000000..b8910d8 --- /dev/null +++ b/docs/superpowers/specs/2026-10-23-companion-design.md @@ -0,0 +1,215 @@ +# Desktop companion: design + +Date: 2026-10-23. Status: designed, not implemented. + +## Goal + +A small chibi 2B (NieR: Automata) who lives in the bottom-right corner of +the focused monitor, follows focus between DP-1 and DP-3, and reacts to +what happens on the desktop: notifications shove her, typing makes her +watch, deleting makes her mock, the assistant gets a face, idling bores +her, clicks make her blush and, at seven, self-destruct. She talks in short +fixed English lines in a speech bubble. SUPER+\ shows and hides her. + +The idea comes from Modesty's window companion, read for ideas only +(GPLv3; see AGENTS.md). Nothing here is taken from its code. The +difference that shapes everything: this desktop runs mostly maximised +windows, so she lives in a screen corner, not on a window edge. + +## Verified before designing + +- **The art can be made locally.** Pony Diffusion V6 XL with two LoRAs + makes the base frame, img2img makes expressions, FLUX.2 klein makes + poses. Recipes, prompts and what failed are in + `~/.local/share/keystone/companion/2b/recipe.txt`. Measured on the way: + img2img at strength 0.75 changes a face and not an arm, and at 0.85 + moves a hand halfway and starts to drift; klein moves limbs and keeps + the character when every detail to keep is named. +- **sd.cpp 929 cannot load an SDXL diffusers directory**: it detects it + as SD 1.x (leejet/stable-diffusion.cpp#1938). sd-webui's Pony entry now + uses the single-file checkpoint, which loads as SDXL. +- **A resident llama.cpp model starves image generation.** Gemma left + 1.1 GB of 12 GB free and the first unet segment failed; an img2img + encode needed 4.5 GB. sd-webui unloads llama-server's models at start. +- **The white background comes off cleanly.** ImageMagick, fuzz 6%, + flood fill from the four corners, trim, scale to 256 px tall: the line + art stops the fill, so the white hair and the white leotard survive. +- **The look was checked on screen** with a throwaway shell: 128 px tall, + 24 px from the right edge and 16 px from the bottom of DP-1, a dark + translucent bubble up and to the left of her head. Approved by eye. +- **Balloons are bottom-right of DP-1** (`~/.config/notifyd/config.json`, + `"corner": "bottom-right"`), her corner. `shared/Notify.qml` already + exposes the queue, DND, snooze and the drawer state; the check of + which queued balloons are on screen lives in `notifications/`. +- **Hyprland 0.56.2 has what the inputs need**, per + `/usr/share/hypr/stubs/hl.meta.lua`: an `input.keyboard.key` event, a + `non_consuming` bind flag, and an `above_fullscreen` layer rule. None + has been tried on this machine yet; see Open checks. +- **SUPER+\ is free**, and the keyboard carries two layouts, `it,us`. + The `backslash` keysym sits on a different physical key in each, so the + bind uses the physical key, `code:49`, left of 1. + +## Where she lives + +A component of `desktop/`, a sibling of `island/` and `assistant/`, not a +drawer module: she has no tile and no page. `desktop/` already hosts the +assistant, so she reads its state directly. The cost, accepted: restarting +`desktop/` restarts her. + +``` +desktop/companion/ + Companion.qml the window, inputs, timers, IPC + Sprite.qml draws a frame; crossfade, bob, squash, explosion + Mood.js which state wins, as a pure function (.pragma library) + test_mood.js node check of Mood.js, like assistant/test_md.js + lines.js her English lines, grouped by reaction + prep-frames bash + ImageMagick: generated images to frames/ + README.md the component's notes +``` + +`desktop/shell.qml` creates one `Companion { asst: assistant }`, the way +`Island` gets `asst: assistant`. + +### The window + +A `PanelWindow` anchored bottom and right, on the focused monitor +(`Hyprland.focusedMonitor`), layer `Top`, `ExclusionMode.Ignore`, +`WlrKeyboardFocus.None`, namespace `keystone-companion`. Its `mask` is her +body only, so clicks anywhere else reach the window below; the bubble is +not in the mask either. + +It is the second window that exists for the whole session, alongside the +keepalive. Hidden, it keeps existing with an empty mask and nothing drawn, +so a toggle never creates or destroys a surface. + +Moving between monitors plays her exit on the old one and her entrance +on the new: the window's screen changes only once the exit has finished. + +### Art + +Never in the repo: 2B is Square Enix's character and the repo is public +under GPLv2. Generated images and their recipe live in +`~/.local/share/keystone/companion/2b/`; `prep-frames` writes the +prepared PNGs, 256 px tall with a transparent background, to `frames/` +inside it. The component loads `frames/<name>.png` and falls back to +`frames/base.png` when a frame is missing, so it works from the first +commit with the three frames that exist. + +| Method | Frames | +| --- | --- | +| Made | `base`, `smile`, `wave` | +| img2img from `base`, strength 0.75 | `mad`, `smirk`, `blush1`, `blush2`, `blush3`, `sleepy` | +| klein edit of `base` | `watch`, `point`, `yawn`, `lying`, `listen`, `think`, `muse`, `stretch`, `leotard`, `leotard-scorched` | + +Dodging, shoving, the drop-in and the explosion are QML motion, not +frames. She is drawn 128 px tall; both monitors are at scale 1. No size +preference: one would need a control in the settings window, and nothing +asks for it yet. + +## Behaviour + +One state at a time, the highest that applies. `Mood.js` takes the inputs +and the time and returns the state; `Companion.qml` owns the inputs and +timers. + +| Priority | State | Trigger | Frame | Ends | +| --- | --- | --- | --- | --- | +| 1 | Self-destruct | 7th click of a burst | glow, flash, explosion, then `leotard-scorched` | see below | +| 2 | Dodge | pointer rests on her 0.6 s with no click | slides out of the corner | returns after 5 s | +| 3 | Shoved | a balloon is on screen on her monitor | `mad`, slid left by the stack's width | balloons gone | +| 4 | Assistant | recording / working / reply arrived | `listen` / `think` / `wave` | state ends; wave lasts 3 s | +| 5 | Clicks | clicks 1-6 of a burst; a burst ends after 1.5 s without one | `smile`, `blush1` to `blush3` | 2 s after the last click | +| 6 | Mock | BackSpace, at most once in 20 s | `smirk`, `point` | 2.5 s | +| 7 | Watching | typing pulse in the last 2 s | `watch` | 2 s after typing stops | +| 8 | Idle | no input for 1 / 3 / 5 min | `sleepy` / `yawn` / `lying` | any input: `stretch`, then base | +| 9 | Base | nothing else | `base` | | + +- **Self-destruct, as in the game.** She glows, says "Self-destruct + sequence initiated.", a white flash covers the sprite for about a + second with particles and a shockwave ring, and she is back in place + without her skirt (`leotard-scorched`, then `leotard`), saying + "...That was unpleasant." The skirt returns after 60 s, or at the next + hide and show, whichever is first. Every other frame has the skirt, so + for those 60 s she stays on `leotard` whatever the state: reactions + still happen, as lines and motion, but the frame does not change. + ponytail: one skirtless frame; skirtless variants of the others only if + the 60 s feel frozen. +- **Dodge.** She cannot feel the pointer until it is over her: Quickshell + has no global pointer and Hyprland sends no motion events. A click + before the 0.6 s still reaches her, so clicking her keeps working. If + the pointer is still there when she returns, she dodges again. +- **Shoved** only on DP-1, where the balloons are. A balloon counts when + the balloon shell would show it: not withheld by DND, snooze or the + open drawer, not expired. That check moves into `shared/Notify.qml` + as one function both shells call, so they cannot disagree. +- **Assistant.** Recording is `asst.recording`; working is `asst.busy` + (`thinking`, `tool`, `transcribing`); "reply arrived" is the reply + ending, the same moment the canvas's done notification fires. +- **Idle** is Quickshell's `IdleMonitor`, respecting inhibitors, so + presentation mode keeps her awake. +- **Hidden** while the focused window is fullscreen, while the screen is + locked, and while toggled off. + +### Speech + +Fixed English lines from `lines.js`, picked at random per reaction, for +example "...Operator, I'm bored.", "Typo again?", "Hey.", "Answer's +ready.", "Glory to mankind.". A bubble shows 3 s. At most one line per +30 s, except clicks and the self-destruct. + +The game's opening monologue, verbatim: + +> Everything that lives is designed to end. We are perpetually trapped in +> a never-ending spiral of life and death. Is this a curse? Or some kind +> of punishment? I often think about the god who blessed us with this +> cryptic puzzle...and wonder if we'll ever get the chance to kill him. + +It plays with a 1-in-4 chance on entering bored and 1-in-3 on being shown +with SUPER+\, never more than once an hour. It types out at about 15 +characters a second, one sentence per bubble, the last held 4 s, over the +`muse` frame. Any state above idle cuts it short. + +## Inputs from Hyprland + +In the live config, `~/.config/hypr/sections/`, not this repo: + +- `hl.bind("SUPER + code:49", ...)` runs + `qs -p <desktop> ipc call companion toggle`. +- A hook on `input.keyboard.key` dispatches a custom `keystone-typing` + event at most once a second. It reads no key data. +- A `non_consuming` bind on BackSpace dispatches `keystone-delete`, so + the key still reaches the focused window. BackSpace is the only key + the companion learns about. + +`Companion.qml` reads both through `Hyprland.rawEvent`, the stream the +island already uses. + +## Testing + +- `node desktop/companion/test_mood.js`: the priority table, the click + burst (1 smiles, 2-6 blush harder, 7 self-destructs, a 1.5 s gap + resets), the 20 s mock cooldown, the 30 s line cooldown, the hourly + monologue cap, the idle thresholds. +- `prep-frames` checked on `base.png`: alpha present, 256 px tall, the + hair not eaten (corner pixel transparent, a hair pixel opaque). +- By hand, since it is visual: each reaction once, monitor hand-over, + fullscreen and lock hiding, the dodge, and that BackSpace still deletes + in a terminal and a browser with the bind active. + +## Open checks + +Each is tested before the code that depends on it: + +1. A `non_consuming` BackSpace bind still deletes in the focused window. +2. The `input.keyboard.key` hook fires and its dispatch reaches + `Hyprland.rawEvent`. +3. A `Top` layer surface is covered by a fullscreen window, so the + fullscreen rule may be the compositor's and not code. If not, hide on + the focused toplevel's fullscreen flag. + +## Out of scope + +- Generated lines (Gemma): slow, and it would hold VRAM against sd-webui. +- A size preference or any other setting. +- Following the focused window's edge, Modesty-style. +- Italian lines. |