diff options
| -rw-r--r-- | AGENTS.md | 5 | ||||
| -rw-r--r-- | desktop/companion/frames.txt | 99 | ||||
| -rwxr-xr-x | desktop/companion/gen-frames | 150 | ||||
| -rwxr-xr-x | desktop/companion/prep-frames | 45 | ||||
| -rw-r--r-- | desktop/modules/ai/AiModule.qml | 18 | ||||
| -rw-r--r-- | desktop/modules/ai/AiPage.qml | 2 | ||||
| -rw-r--r-- | desktop/modules/ai/AiRow.qml | 18 | ||||
| -rw-r--r-- | desktop/modules/ai/README.md | 16 | ||||
| -rw-r--r-- | docs/superpowers/plans/2026-10-23-companion.md | 1381 | ||||
| -rw-r--r-- | docs/superpowers/specs/2026-10-23-companion-design.md | 232 |
10 files changed, 1953 insertions, 13 deletions
@@ -383,7 +383,10 @@ changing that component. The ones that generalise: the loaded model answers 400 `model is already running`, and an unknown id 404; both measured. With `--models-max 1` a load swaps out the resident model for every client, so the assistant page does it only on a - click. + click. `POST /models/unload` with the same body frees it (answers + `{"success":true}`, measured); the next request for it loads it again. + `sd-webui` unloads whatever is loaded before starting sd-server: a resident + Gemma left Pony 1.1 GB of 12 GB and its first step failed. - **`busctl get-property` escapes non-ASCII as octal.** A name with an accented letter reads `s "Do\303\250r"`-style, which no shell builtin decodes. `busctl --json=short` returns real UTF-8; `userctl` pipes it diff --git a/desktop/companion/frames.txt b/desktop/companion/frames.txt new file mode 100644 index 0000000..0019db1 --- /dev/null +++ b/desktop/companion/frames.txt @@ -0,0 +1,99 @@ +# Copyright (C) 2026 Danilo M. <danix@danix.xyz> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# The companion's frames, for gen-frames. One frame per line: +# +# <output> <- <reference> : <prompt> +# +# Both are paths in the art directory (~/.local/share/keystone/companion/2b) +# without .png. The reference is the image FLUX.2 klein edits; the output is +# written over. Lines run top to bottom, so a frame made from another comes +# after it. ${NAME} expands a NAME = ... line defined above it. Blank lines +# and lines starting with # are skipped. +# +# dress/base is the one frame not made here: a klein edit of base-sitting.png, +# recorded in recipe.txt. Every other dress frame is made from it, and every +# leotard frame from its own dress frame, so the pose carries over. + +KEEP = She keeps her black gloves on both hands, her puffy black juliet sleeves, her black dress with its skirt, her black thighhigh boots, her black blindfold and hairband, her lips and the small mole under her mouth. Same character, chibi proportions, art style and white background. +POSE = keep her pose exactly, standing, arms relaxed at her sides. +# Strong expressions made klein turn the blindfold into an eye mask with the +# eyes showing through (mad, smirk, blush3, sleepy). The ones that read +# through her eyes take it off instead (BARE); the rest name it (EYES). +BARE = She has taken her blindfold off: both eyes are uncovered and show clearly, pale blue-grey, and carry the expression; no blindfold, no mask, nothing over her eyes; her black hairband stays. +KEEP_BARE = She keeps her black gloves on both hands, her puffy black juliet sleeves, her black dress with its skirt, her black thighhigh boots, her black hairband, her lips and the small mole under her mouth. Same character, chibi proportions, art style and white background. +EYES = Her black blindfold stays on as one solid band of cloth fully covering both eyes: no eyes and no eyeholes show; the expression is in her mouth, cheeks and brows above the band. +# After the self-destruct the blindfold is gone with the skirt: the whole +# leotard set shows her eyes. +LEOTARD = Her skirt is gone, revealing a plain white leotard underneath, and her blindfold is gone too: both eyes uncovered and clearly visible, pale blue-grey, matching her expression. Everything else stays exactly the same: pose, expression, black gloves, puffy black juliet sleeves, black bodice, black thighhigh boots, black hairband, lips, the small mole under her mouth, chibi proportions, art style and white background. + +# --- expressions +dress/smile <- dress/base : Change only her expression to a small warm closed-mouth smile; ${POSE} ${EYES} ${KEEP} +dress/mad <- dress/base : Change only her expression to annoyed and angry, a frown and a puffed cheek; ${POSE} ${BARE} ${KEEP_BARE} +dress/smirk <- dress/base : Change only her expression to a teasing one-sided smirk; ${POSE} Both eyes fully visible and uncovered; the blindfold is nowhere on her head or face, not pushed up, not over one eye. ${BARE} ${KEEP_BARE} +dress/blush1 <- dress/base : Change only her expression: a light pink blush on her cheeks and a shy small smile; ${POSE} ${EYES} ${KEEP} +dress/blush2 <- dress/base : Change only her expression: a strong red blush across her cheeks and a flustered wavy mouth; ${POSE} ${EYES} ${KEEP} +dress/blush3 <- dress/base : Change only her expression: her whole face bright red, small steam puffs above her head, very flustered; ${POSE} ${BARE} ${KEEP_BARE} +dress/sleepy <- dress/base : Change only her expression to sleepy and drowsy, head tilted slightly, mouth relaxed; ${POSE} ${BARE} ${KEEP_BARE} + +# --- poses +dress/wave <- dress/base : Raise only the hand on the left side of the image beside her head, waving, with a small smile; her other arm stays at her side. ${KEEP} +# Shown while typing: she faces into the screen, away from the viewer, as if +# reading over the user's shoulder. A profile read as posing, not watching. +dress/watch <- dress/base : Turn her completely around into a back view: we see her from behind, the back of her head with her white bob, crossed by her black hairband and the small black cloth knot of her black blindfold (black, not white, no bow), her back, the back of her dress and the backs of her legs, as if she is facing a screen and watching it; arms relaxed at her sides. Same character, black gloves, puffy black juliet sleeves, black dress with its skirt, black thighhigh boots, black hairband, chibi proportions, art style and white background. +dress/point <- dress/base : She points with the hand on the left side of the image toward the viewer, with a teasing smirk; her other hand rests on her hip. ${KEEP} +dress/yawn <- dress/base : A big yawn, mouth wide open, one gloved hand covering her mouth. ${KEEP} +dress/lying <- dress/base : Make her lie on her side on the ground, asleep, her head resting on her folded arms. ${KEEP} +dress/listen <- dress/base : She cups the hand on the left side of the image behind her ear, listening attentively. ${KEEP} +dress/think <- dress/base : One gloved hand at her chin, thinking, head tilted slightly. ${KEEP} +# The monologue's frame: the pose of her official character art, chibi. +# refs/2b-official-white is that art flattened on white, used only for the +# pose. A Virtuous Treaty beside her was tried and dropped: klein draws a +# generic longsword, nothing like the game's, and would not move it. +dress/muse <- dress/base + refs/2b-official-white : Redraw the chibi character from the first image, keeping her exact chibi proportions, face, outfit and art style, in the standing pose of the woman in the second image: body turned three-quarters, one arm hanging out low to the side with the hand open, the other hand raised near her shoulder, weight on one leg, head slightly lowered, calm and solemn. No weapon, nothing in her hands. ${KEEP} +# She stays frontal: a three-quarter turn, from the reference or as its own +# edit, only bent her knees. +dress/stretch <- dress/base : She stretches both arms up above her head, waking up. ${KEEP} +# A balloon shoved her out of the corner. Standing versions read as walking +# in four seeds, so this one is airborne, edited from the attempt whose hands +# and face were right: refs/shoved-7.png, made from dress/base with +# --seed 7 by the earlier standing prompt (see generated.log), kept in refs/. +dress/shoved <- refs/shoved-7 : Keep her face, her scared expression, her uncovered eyes and both raised hands exactly as they are. Change her body and legs: she is in the air, lifted completely off the ground, knocked flying toward the left side of the image by a strong blast from the right; both feet off the ground, legs trailing behind her toward the right side, her body tilted, her hair and skirt blown toward the left. No ground, no blast effect drawn. ${KEEP_BARE} + +# --- the leotard set: the same frames, after the self-destruct +leotard/base <- dress/base : ${LEOTARD} +leotard/smile <- dress/smile : ${LEOTARD} +# The generic line let the new eyes go soft: mad read sad, and yawn, whose +# dress frame has no eyes to copy, read as laughing. Name the mood they carry. +leotard/mad <- dress/mad : ${LEOTARD} Her expression stays angry and annoyed: furrowed brows, a sharp glaring stare, a frown and a puffed cheek; not sad, not tearful. +leotard/smirk <- dress/smirk : ${LEOTARD} +leotard/blush1 <- dress/blush1 : ${LEOTARD} +leotard/blush2 <- dress/blush2 : ${LEOTARD} +leotard/blush3 <- dress/blush3 : ${LEOTARD} +leotard/sleepy <- dress/sleepy : ${LEOTARD} +leotard/wave <- dress/wave : ${LEOTARD} +# From dress/watch, klein kept the blindfold's band at the back of her head +# through three prompts, by name and by position; from leotard/base, which +# has no blindfold, the back view is the usual pose edit. +leotard/watch <- leotard/base : Turn her completely around into a back view: we see her from behind, the back of her head with her white bob crossed only by her black hairband over the top, her back, the back of her black bodice, the white leotard and the backs of her legs, as if she is facing a screen and watching it; arms relaxed at her sides. No blindfold. Same character, black gloves, puffy black juliet sleeves, black thighhigh boots, chibi proportions, art style and white background. +leotard/point <- dress/point : ${LEOTARD} +leotard/yawn <- dress/yawn : ${LEOTARD} It is a big sleepy yawn, not a laugh: eyes squeezed shut and a little teary, brows relaxed and drowsy, her gloved hand over her wide open mouth. +# Lying face down and the back view have no eyes to uncover, so the generic +# line left the blindfold on them: say it is gone, band and knot. +# Asked to remove it from dress/lying, klein kept the band and only drew an +# eye under it; from leotard/base, which has none, it is the usual pose edit. +leotard/lying <- leotard/base : Make her lie on her side on the ground, asleep, her head resting on her folded arms, eyes closed. She keeps her plain white leotard, black gloves, puffy black juliet sleeves, black bodice, black thighhigh boots, black hairband, lips and the small mole under her mouth, and no blindfold. Same character, chibi proportions, art style and white background. +leotard/listen <- dress/listen : ${LEOTARD} +leotard/think <- dress/think : ${LEOTARD} +leotard/muse <- dress/muse : ${LEOTARD} +leotard/stretch <- dress/stretch : ${LEOTARD} +leotard/shoved <- dress/shoved : ${LEOTARD} +leotard/scorched <- leotard/base : Right after an explosion: her clothes lightly scorched and smudged with soot, a few wisps of smoke rising from her, hair slightly messy, a dazed flat expression; standing, arms at her sides. Her eyes stay uncovered, no blindfold. Her top stays black: the black bodice with its chest cutout and the puffy black juliet sleeves, exactly as in the image; the plain white leotard shows only below her waist, where the skirt was. She keeps her black gloves, black thighhigh boots, black hairband, lips and the small mole under her mouth. Same character, chibi proportions, art style and white background. diff --git a/desktop/companion/gen-frames b/desktop/companion/gen-frames new file mode 100755 index 0000000..df78cb9 --- /dev/null +++ b/desktop/companion/gen-frames @@ -0,0 +1,150 @@ +#!/bin/bash +# +# Copyright (C) 2026 Danilo M. <danix@danix.xyz> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# gen-frames: make the companion's frames listed in frames.txt. +# +# gen-frames [--seed N] [--list] [OUTPUT...] +# +# With no OUTPUT, makes every frame whose image does not exist yet. Naming +# outputs (dress/mad, leotard/mad) remakes those, existing or not; a dress +# frame remade leaves its leotard frame stale, so name both. --seed changes +# the seed (default 42) to reroll a result; --list prints what would be made. +# Then runs prep-frames. +# +# It drives a running sd-server rather than sd-cli, which would reload the +# model for every frame: start sd-webui and pick FLUX.2 klein first. sd-webui +# also unloads llama-server's model, which otherwise leaves too little VRAM. +# The model is checked from sd-server's command line, since its API does not +# name it. + +set -euo pipefail + +here=$(cd "$(dirname "$0")" && pwd) +art=${ART_DIR:-$HOME/.local/share/keystone/companion/2b} +url=${SD_URL:-http://127.0.0.1:7860} +seed=42 +list=0 +want=() +while (($#)); do + case $1 in + --seed) seed=$2; shift 2 ;; + --list) list=1; shift ;; + -h|--help) sed -n '14,28p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; + *) want+=("$1"); shift ;; + esac +done + +# frames.txt to "output<TAB>references<TAB>prompt" lines, variables expanded. +# References are joined with + and no spaces: the first is the image edited, +# any after it are extra images the prompt can point at ("the second image"). +parse() { + local line name out ref prompt + local frame='^[[:space:]]*([a-z0-9/_-]+)[[:space:]]*<-[[:space:]]*([a-z0-9/_-]+([[:space:]]*\+[[:space:]]*[a-z0-9/_-]+)*)[[:space:]]*:[[:space:]]*(.*)$' + declare -A vars=() + while IFS= read -r line || [[ -n $line ]]; do + [[ $line =~ ^[[:space:]]*(#|$) ]] && continue + if [[ $line =~ ^([A-Z_]+)[[:space:]]*=[[:space:]]*(.*)$ ]]; then + vars[${BASH_REMATCH[1]}]=${BASH_REMATCH[2]} + continue + fi + if [[ ! $line =~ $frame ]]; then + echo "gen-frames: frames.txt: cannot read: $line" >&2 + exit 1 + fi + out=${BASH_REMATCH[1]} ref=${BASH_REMATCH[2]// /} prompt=${BASH_REMATCH[4]} + for name in "${!vars[@]}"; do prompt=${prompt//\$\{$name\}/${vars[$name]}}; done + if [[ $prompt == *'${'* ]]; then + echo "gen-frames: frames.txt: $out: undefined variable in: $prompt" >&2 + exit 1 + fi + printf '%s\t%s\t%s\n' "$out" "$ref" "$prompt" + done <"$here/frames.txt" +} + +# What to make: the named outputs, else every missing one, in file order. +# (No grep -q in a pipe: under pipefail its early exit fails the printf.) +todo=() +rows=$(parse) || exit 1 +mapfile -t rows <<<"$rows" +declare -A known=() wanted=() +for row in "${rows[@]}"; do known[${row%%$'\t'*}]=1; done +for name in "${want[@]}"; do + [[ -n ${known[$name]:-} ]] || { echo "gen-frames: $name is not in frames.txt" >&2; exit 1; } + wanted[$name]=1 +done +for row in "${rows[@]}"; do + out=${row%%$'\t'*} + if ((${#want[@]})); then + [[ -n ${wanted[$out]:-} ]] && todo+=("$row") + elif [[ ! -e $art/$out.png ]]; then + todo+=("$row") + fi +done +if ((list)); then + for row in "${todo[@]}"; do printf '%s <- %s\n' "${row%%$'\t'*}" "$(cut -f2 <<<"$row")"; done + exit 0 +fi +if ((${#todo[@]} == 0)); then + echo "gen-frames: nothing to make" + exec "$here/prep-frames" "$art" +fi + +pgrep -ax sd-server | grep -q klein || + { echo "gen-frames: sd-server is not running FLUX.2 klein; start sd-webui and pick it" >&2; exit 1; } + +tmp=$(mktemp -d) +trap 'rm -rf "$tmp"' EXIT + +made=0 +for row in "${todo[@]}"; do + IFS=$'\t' read -r out ref prompt <<<"$row" + IFS=+ read -ra refs <<<"$ref" + missing="" + for r in "${refs[@]}"; do [[ -e $art/$r.png ]] || missing=$r; done + if [[ -n $missing ]]; then + echo "gen-frames: $out: its reference $missing.png does not exist yet, skipped" >&2 + continue + fi + echo "gen-frames: $out <- ${ref//+/ + }" + # One JSON string per reference, as data URLs, slurped into the array. + for r in "${refs[@]}"; do + { printf 'data:image/png;base64,'; base64 -w0 "$art/$r.png"; } | jq -Rs . + done >"$tmp/refs.json" + # The settings the frames were tuned with in sd-webui: recipe.txt. + jq -n --arg prompt "$prompt" --slurpfile refs "$tmp/refs.json" --argjson seed "$seed" '{ + prompt: $prompt, negative_prompt: "", width: 1024, height: 1024, seed: $seed, + batch_count: 1, ref_images: $refs, auto_resize_ref_image: true, + increase_ref_index: false, lora: [], output_format: "png", output_compression: 100, + sample_params: { sample_steps: 4, sample_method: "euler", scheduler: "default", + guidance: { txt_cfg: 1.0 } } + }' >"$tmp/body" + id=$(curl -sf -m 30 -X POST "$url/sdcpp/v1/img_gen" -H 'Content-Type: application/json' \ + --data-binary @"$tmp/body" | jq -r '.id // empty') || id="" + [[ -n $id ]] || { echo "gen-frames: $out: sd-server refused the job" >&2; exit 1; } + while :; do + curl -sf -m 30 "$url/sdcpp/v1/jobs/$id" >"$tmp/job" || { echo "gen-frames: $out: lost the job" >&2; exit 1; } + status=$(jq -r '.status' "$tmp/job") + case $status in + queued|generating) sleep 2 ;; + completed) break ;; + *) echo "gen-frames: $out: $status: $(jq -r '.error.message // "no message"' "$tmp/job")" >&2; exit 1 ;; + esac + done + mkdir -p "$art/$(dirname "$out")" + jq -r '.result.images[0].b64_json' "$tmp/job" | base64 -d >"$tmp/out.png" + mv "$tmp/out.png" "$art/$out.png" + printf '%s seed %s %s <- %s\n' "$(date +%F)" "$seed" "$out" "$ref" >>"$art/generated.log" + made=$((made + 1)) +done +echo "gen-frames: made $made" +exec "$here/prep-frames" "$art" diff --git a/desktop/companion/prep-frames b/desktop/companion/prep-frames new file mode 100755 index 0000000..462657e --- /dev/null +++ b/desktop/companion/prep-frames @@ -0,0 +1,45 @@ +#!/bin/bash +# +# Copyright (C) 2026 Danilo M. <danix@danix.xyz> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# prep-frames: the companion's generated images to the frames she shows. +# +# prep-frames [ART_DIR] default: ~/.local/share/keystone/companion/2b +# +# Reads ART_DIR/dress/*.png and ART_DIR/leotard/*.png, 1024x1024 on white, +# and writes ART_DIR/frames/<outfit>/<name>.png: background removed, trimmed +# to her, scaled by a quarter. One scale for every frame keeps her the same +# size whatever the pose, which scaling each to a fixed height would not: +# lying down would come out huge. Sprite.qml draws them at half size and +# stands each on its bottom edge. +# +# The background goes by flood fill from the four corners: the dark line art +# stops the fill, so the white hair and the white leotard inside it stay. + +set -euo pipefail + +art=${1:-$HOME/.local/share/keystone/companion/2b} +n=0 +for outfit in dress leotard; do + [[ -d $art/$outfit ]] || continue + mkdir -p "$art/frames/$outfit" + for src in "$art/$outfit"/*.png; do + [[ -e $src ]] || continue + read -r w h < <(magick identify -format '%w %h\n' "$src") + magick "$src" -alpha set -fuzz 6% -fill none \ + -draw "color 0,0 floodfill" -draw "color $((w - 1)),0 floodfill" \ + -draw "color 0,$((h - 1)) floodfill" -draw "color $((w - 1)),$((h - 1)) floodfill" \ + -trim +repage -resize 25% "$art/frames/$outfit/$(basename "$src")" + n=$((n + 1)) + done +done +echo "prep-frames: $n frames in $art/frames" diff --git a/desktop/modules/ai/AiModule.qml b/desktop/modules/ai/AiModule.qml index a96f484..6df3d6b 100644 --- a/desktop/modules/ai/AiModule.qml +++ b/desktop/modules/ai/AiModule.qml @@ -49,10 +49,16 @@ Module { Quickshell.env("HOME") + "/Programming/GIT/desktop-assistant/assistant.sh" // Each project's own control. Fanfictioner has no start, a run needs a - // story file, and its stop needs the pid, see stop(). + // story file, and its stop needs the pid, see stop(). llamachat has no + // restart, and --quit answers before the app is gone, so a --daemon + // straight after would find it still running and do nothing: wait for + // --ping to fail first, up to 5s. readonly property var controls: ({ - assistant: { start: [assistantSh, "start"], stop: [assistantSh, "stop"] }, - chat: { start: ["llamachat", "--daemon"], stop: ["llamachat", "--quit"] }, + assistant: { start: [assistantSh, "start"], stop: [assistantSh, "stop"], + restart: [assistantSh, "restart"] }, + chat: { start: ["llamachat", "--daemon"], stop: ["llamachat", "--quit"], + restart: ["sh", "-c", "llamachat --quit; for _ in $(seq 50); do " + + "llamachat --ping || break; sleep 0.1; done; exec llamachat --daemon"] }, imggen: { start: ["imggen", "start"], stop: ["imggen", "stop"] }, }) @@ -62,6 +68,12 @@ Module { if (canStart(app)) exec(controls[app.name].start); } + function canRestart(app) { return controls[app.name]?.restart !== undefined; } + + function restart(app) { + if (canRestart(app)) exec(controls[app.name].restart); + } + function stop(app) { if (app.name !== "fanfic") { exec(controls[app.name].stop); diff --git a/desktop/modules/ai/AiPage.qml b/desktop/modules/ai/AiPage.qml index c8e39e9..e89132a 100644 --- a/desktop/modules/ai/AiPage.qml +++ b/desktop/modules/ai/AiPage.qml @@ -69,6 +69,8 @@ Column { } action: modelData.running ? "Stop" : page.ai.canStart(modelData) ? "Start" : "" danger: modelData.running + restartable: modelData.running && page.ai.canRestart(modelData) + onRestart: page.ai.restart(modelData) onTriggered: modelData.running ? page.ai.stop(modelData) : page.ai.start(modelData) } } diff --git a/desktop/modules/ai/AiRow.qml b/desktop/modules/ai/AiRow.qml index ac4a180..2950759 100644 --- a/desktop/modules/ai/AiRow.qml +++ b/desktop/modules/ai/AiRow.qml @@ -12,8 +12,9 @@ import QtQuick import "../.." -// One engine or app: a state dot, a name, a detail line, and at most one -// button. An empty action means no button, which is every engine row. +// One engine or app: a state dot, a name, a detail line, at most one +// button, and a Restart before it when restartable. An empty action means no +// button, which is every engine row. Rectangle { id: row @@ -23,7 +24,9 @@ Rectangle { property bool busy: false property string action: "" property bool danger: false + property bool restartable: false signal triggered + signal restart implicitHeight: Math.max(texts.implicitHeight, btn.visible ? btn.implicitHeight : 0) + 16 radius: 8 @@ -40,7 +43,8 @@ Rectangle { id: texts anchors { left: dot.right; leftMargin: 10 - right: btn.visible ? btn.left : parent.right; rightMargin: 10 + right: restartBtn.visible ? restartBtn.left : btn.visible ? btn.left : parent.right + rightMargin: 10 verticalCenter: parent.verticalCenter } spacing: 2 @@ -63,6 +67,14 @@ Rectangle { } Button { + id: restartBtn + anchors { right: btn.left; rightMargin: 6; verticalCenter: parent.verticalCenter } + visible: row.restartable + text: "Restart" + onClicked: row.restart() + } + + Button { id: btn anchors { right: parent.right; rightMargin: 10; verticalCenter: parent.verticalCenter } visible: row.action !== "" diff --git a/desktop/modules/ai/README.md b/desktop/modules/ai/README.md index 5e204ca..5aec615 100644 --- a/desktop/modules/ai/README.md +++ b/desktop/modules/ai/README.md @@ -27,12 +27,16 @@ written in the last 5s. A step slower than that reads idle. ## Apps -| App | Status | Start | Stop | -| --- | --- | --- | --- | -| desktop-assistant | `assistant.sh status` | `assistant.sh start` | `assistant.sh stop` | -| local chat | `llamachat --ping` | `llamachat --daemon` | `llamachat --quit` | -| imggen | `imggen status` | `imggen start` (default model) | `imggen stop` | -| fanfictioner | `pgrep` on `python3 .../fanfictioner` | none | TERM its children, then it, by pid | +| App | Status | Start | Stop | Restart | +| --- | --- | --- | --- | --- | +| desktop-assistant | `assistant.sh status` | `assistant.sh start` | `assistant.sh stop` | `assistant.sh restart` | +| local chat | `llamachat --ping` | `llamachat --daemon` | `llamachat --quit` | quit, wait for `--ping` to fail (5s at most), `--daemon` | +| imggen | `imggen status` | `imggen start` (default model) | `imggen stop` | none | +| fanfictioner | `pgrep` on `python3 .../fanfictioner` | none | TERM its children, then it, by pid | none | + +Restart shows only while an app is running. llamachat's `--quit` answers +before the app has exited, so a `--daemon` sent straight after finds it still +running and does nothing; hence the wait. Each runs detached, so no exit code comes back; the next poll shows whether it worked. `assistant.sh status` fails when the daemon is down. diff --git a/docs/superpowers/plans/2026-10-23-companion.md b/docs/superpowers/plans/2026-10-23-companion.md new file mode 100644 index 0000000..b6fb24a --- /dev/null +++ b/docs/superpowers/plans/2026-10-23-companion.md @@ -0,0 +1,1381 @@ +# Desktop Companion Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** A chibi 2B in the bottom-right corner of the focused monitor who reacts to notifications, typing, deleting, the assistant, idling and clicks, as specified in `docs/superpowers/specs/2026-10-23-companion-design.md`. + +**Architecture:** A component of `desktop/` (`desktop/companion/`), created once in `desktop/shell.qml` and handed the assistant. The state decision is a pure JS library (`Mood.js`) checked with node; `Companion.qml` owns the window, inputs and timers; `Sprite.qml` draws and animates a frame. Typing and BackSpace arrive as Hyprland custom events from the live config in the `hypr-theme` repo. The art lives outside the repo, in `~/.local/share/keystone/companion/2b/`. + +**Tech Stack:** Quickshell 0.3.1 (QML, `Quickshell.Hyprland`, `Quickshell.Wayland`), node for the JS checks, bash + ImageMagick 7 for frame preparation, Hyprland 0.56.2 Lua config. + +--- + +## Ground rules for whoever runs this + +- Read `AGENTS.md` first. Three of its traps apply directly: the process is `qs`, never `quickshell`; never `pkill -f`; a `qs` started from an agent's tool call dies with the call, so **never start or restart the desktop shell yourself**. The running `desktop` shell hot-reloads on save; check it with `qs log -p ~/Programming/GIT/quickshell/desktop --log-times | tail`. +- Hot reload does not follow the `shared/` symlinks. After Task 2 the user restarts with `./reload.sh desktop notifications` from their own terminal (Task 10). +- A new directory under `desktop/` is only indexed when a file importing it changes; Task 8 changes `desktop/shell.qml`, which does that. +- Every new source file carries the GPLv2 header used across the repo (copy it from `desktop/assistant/Md.js`, lines 1-10, or the `#` form from `desktop/modules/ai/ai-state.sh`, lines 1-12). +- No home paths in committed files (a gitleaks hook blocks them): write `~` in docs, `Quickshell.env("HOME")` in QML. +- No em dashes in any prose, comment or commit message. +- Commits are GPG-signed (the default here; never disable signing) and end with: + `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>` + +## File map + +| File | Status | Responsibility | +| --- | --- | --- | +| `~/Programming/GIT/hypr-theme/sections/companion.lua` | create | typing pulse and BackSpace event | +| `~/Programming/GIT/hypr-theme/hyprland.lua` | modify | require the new section | +| `~/Programming/GIT/hypr-theme/sections/keybindings.lua` | modify | SUPER+\ toggle | +| `shared/NotifyConfig.js` | modify | `shown()`: is a balloon on screen | +| `shared/test_notify_config.js` | modify | checks for `shown()` | +| `notifications/NotificationBalloon.qml` | modify | use `shown()` | +| `desktop/companion/Mood.js` | create | which state wins; timings; outfit; cooldowns | +| `desktop/companion/test_mood.js` | create | node check of `Mood.js` | +| `desktop/companion/Lines.js` | create | her English lines and the monologue | +| `desktop/companion/prep-frames` | done | generated images to display frames | +| `desktop/companion/gen-frames`, `frames.txt` | done | make the frames through sd-server | +| `desktop/assistant/Assistant.qml` | modify | `signal replied()` | +| `desktop/companion/Sprite.qml` | create | frame, crossfade, bob, drop, explosion | +| `desktop/companion/Companion.qml` | create | window, inputs, timers, speech, IPC | +| `desktop/shell.qml` | modify | create the companion | +| `desktop/companion/README.md` | create | component notes | +| `AGENTS.md` | modify | list the component | +| `docs/superpowers/specs/2026-10-23-companion-design.md` | modify | `lines.js` became `Lines.js` | + +--- + +### Task 1: Hyprland inputs (open checks 1 and 2) + +The spec's first two open checks, tested before any QML depends on them. This is the live config, the `hypr-theme` repo that `~/.config/hypr` links to. It has an unrelated uncommitted change in `sections/autostart.lua`: leave it alone and stage only the files below. + +**Files:** +- Create: `~/Programming/GIT/hypr-theme/sections/companion.lua` +- Modify: `~/Programming/GIT/hypr-theme/hyprland.lua` (after `require("sections.keybindings")`, line 23) +- Modify: `~/Programming/GIT/hypr-theme/sections/keybindings.lua` (after the `mainMod .. " + Home"` bind, line 68) + +- [ ] **Step 1: Write `sections/companion.lua`** + +```lua +-- The desktop companion (quickshell/desktop/companion) reacts to typing and +-- to deleting. Neither reaches a layer surface that never takes the +-- keyboard, so the compositor tells it, as custom events on socket2. + +-- Typing: a pulse at most once a second, carrying no key data at all. +if _G.keystoneTyping then _G.keystoneTyping:remove() end +local lastPulse = 0 +_G.keystoneTyping = hl.on("input.keyboard.key", function() + local now = os.time() + if now == lastPulse then return end + lastPulse = now + hl.dispatch(hl.dsp.event("keystone-typing")) +end) + +-- Deleting: BackSpace only, and non_consuming so the key still reaches the +-- focused window. It is the one key the companion learns about. +hl.bind("BackSpace", hl.dsp.event("keystone-delete"), { non_consuming = true, description = "Desktop: Companion notices a delete" }) +``` + +- [ ] **Step 2: Require it** + +In `hyprland.lua`, after line 23 (`require("sections.keybindings")`), add: + +```lua +require("sections.companion") +``` + +- [ ] **Step 3: Add the toggle bind** + +In `sections/keybindings.lua`, after the `mainMod .. " + Home"` line, add: + +```lua +-- code:49, the physical key left of 1: `\` on the Italian layout, a backtick +-- on the US one. Binding the keysym would move the shortcut with the layout. +hl.bind(mainMod .. " + code:49", hl.dsp.exec_cmd("qs -p ~/Programming/GIT/quickshell/desktop ipc call companion toggle"), { description = "Desktop: Companion" }) +``` + +- [ ] **Step 4: Reload Hyprland and read its errors** + +Run: `hyprctl reload && sleep 1 && hyprctl configerrors` +Expected: `ok`, then no error line that mentions `companion`, `code:49`, `BackSpace` or `input.keyboard.key`. Run `hyprctl configerrors` once before Step 1 too, so errors that were already there are not blamed on this task. + +- [ ] **Step 5: Watch the events arrive** + +Run (it listens 15 s; ask the user to type a few words and press BackSpace in any window meanwhile): + +```bash +timeout 15 socat -u UNIX-CONNECT:"$XDG_RUNTIME_DIR/hypr/$HYPRLAND_INSTANCE_SIGNATURE/.socket2.sock" - | grep '^custom>>' +``` + +Expected: lines `custom>>keystone-typing`, at most one a second while typing, and `custom>>keystone-delete` per BackSpace. If `socat` is missing, use `nc -U` with the same path. If no `custom>>` line arrives, stop: the QML in Task 8 depends on this exact form, so report what socket2 printed instead. + +- [ ] **Step 6: Ask the user to confirm BackSpace still deletes** + +In a terminal and in the browser. This is open check 1; it cannot be measured from here. If BackSpace stops deleting, remove the `hl.bind("BackSpace", ...)` line, reload, and report: the mock reaction then needs another source. + +- [ ] **Step 7: Commit in hypr-theme** + +```bash +cd ~/Programming/GIT/hypr-theme +git add sections/companion.lua hyprland.lua sections/keybindings.lua +git commit -F - <<'EOF' +companion: typing and delete events, SUPER+\ toggle + +The quickshell desktop companion reacts to typing and to BackSpace, and +a layer surface that never takes the keyboard sees neither, so the +compositor sends them as custom events: a typing pulse at most once a +second with no key data, and a non_consuming BackSpace bind so the key +still deletes. The toggle binds code:49 rather than the backslash +keysym, which sits on different keys in the it and us layouts. + +Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> +EOF +``` + +--- + +### Task 2: One "balloon on screen" check for both shells + +**Files:** +- Modify: `shared/NotifyConfig.js` (add after `withheld`, which ends at line 72) +- Modify: `shared/test_notify_config.js` (the `new Function` line 21, and new asserts before `console.log("ok")`) +- Modify: `notifications/NotificationBalloon.qml:35-41` + +- [ ] **Step 1: Write the failing test** + +In `shared/test_notify_config.js`, change line 21 to: + +```js +const C = new Function(src + "\nreturn { defaults, resolve, withheld, soundFor, shown };")(); +``` + +and insert before the final `console.log("ok");`: + +```js +// On screen: the drawer, snooze, mute, DND and expiry, in one check that +// the balloons and the desktop companion share. +const live = { app: "ok", urgency: "normal", expires: 0 }; +assert.strictEqual(C.shown(cfg, live, false, false, 0, 5000), true); +assert.strictEqual(C.shown(cfg, live, false, true, 0, 5000), false); // drawer open +assert.strictEqual(C.shown(cfg, live, false, false, 6000, 5000), false); // snoozed +assert.strictEqual(C.shown(cfg, { app: "spam", urgency: "normal", expires: 0 }, false, false, 0, 5000), false); +assert.strictEqual(C.shown(cfg, live, true, false, 0, 5000), false); // DND +const timed = { app: "ok", urgency: "normal", expires: 5000 }; +assert.strictEqual(C.shown(cfg, timed, false, false, 0, 4999), true); +assert.strictEqual(C.shown(cfg, timed, false, false, 0, 5000), false); // expired +``` + +- [ ] **Step 2: Run it to see it fail** + +Run: `node shared/test_notify_config.js` +Expected: FAIL, `ReferenceError: shown is not defined`. + +- [ ] **Step 3: Implement `shown`** + +In `shared/NotifyConfig.js`, after the closing `}` of `withheld`: + +```js +// Whether a queued notification's balloon is on screen at `now`: the drawer +// and snooze withhold everything, then mute and DND, then its own expiry. +// The balloon shell and the desktop companion both ask this, so they cannot +// disagree about what is showing. +function shown(cfg, n, dnd, drawerOpen, snoozeUntil, now) { + if (drawerOpen || snoozeUntil > now) return false; + if (withheld(cfg, n, dnd)) return false; + return n.expires === 0 || now < n.expires; +} +``` + +- [ ] **Step 4: Run it to see it pass** + +Run: `node shared/test_notify_config.js` +Expected: `ok` + +- [ ] **Step 5: Use it in the balloon** + +In `notifications/NotificationBalloon.qml`, replace the `visible: { ... }` block (lines 35-41) with: + +```qml + visible: NC.shown(Notify.config, b.notification, Notify.dnd, Notify.drawerOpen, + Notify.snoozeUntil, b.now) +``` + +Keep the comment above it unchanged; it still describes the rule. + +- [ ] **Step 6: Commit** + +```bash +git add shared/NotifyConfig.js shared/test_notify_config.js notifications/NotificationBalloon.qml +git commit -F - <<'EOF' +notify: one "balloon on screen" check, NotifyConfig.shown + +The desktop companion needs to know when a balloon is showing in her +corner, and the rule lived inline in NotificationBalloon's visible +binding. It moves to the shared library so both shells ask the same +function: drawer, snooze, mute, DND, then expiry. + +Shared files do not hot-reload through their links: restart the +notifications and desktop shells after pulling this. + +Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> +EOF +``` + +--- + +### Task 3: Mood.js, the state decision + +**Files:** +- Create: `desktop/companion/Mood.js` +- Test: `desktop/companion/test_mood.js` + +- [ ] **Step 1: Write the failing test** + +`desktop/companion/test_mood.js` (GPLv2 header first, as in `desktop/assistant/test_md.js`): + +```js +// Run: node desktop/companion/test_mood.js +// Mood.js is a QML library, so its .pragma line is stripped before node +// evaluates it. + +const fs = require("fs"); +const assert = require("assert"); + +const src = fs.readFileSync(__dirname + "/Mood.js", "utf8") + .replace(/^\.pragma library$/m, ""); +const M = new Function(src + + "\nreturn { T, click, clickFrame, pick, outfit, mockAllowed, canSpeak, monologue };")(); + +const NOW = 10000000; +// A snapshot with nothing going on, overridden per case. +const calm = over => Object.assign({ + now: NOW, destructAt: 0, dodging: false, shoved: false, recording: false, busy: false, + repliedAt: 0, clicks: { count: 0, at: 0 }, mockAt: 0, typingAt: 0, wakeAt: 0, + monologue: false, idle: 0, +}, over); +const state = over => M.pick(calm(over)).state; +const frame = over => M.pick(calm(over)).frame; + +// Nothing going on: the rest pose. +assert.deepStrictEqual(M.pick(calm({})), { state: "base", frame: "base" }); + +// The priority table, top to bottom: with every trigger from a row down +// active at once, that row wins. Merged so a higher row's values win. +const ladder = [ + ["destruct", { destructAt: NOW - 10 }], + ["scorched", { destructAt: NOW - M.T.destruct - 10 }], + ["dodge", { dodging: true }], + ["shoved", { shoved: true }], + ["listen", { recording: true }], + ["think", { busy: true }], + ["wave", { repliedAt: NOW - 10 }], + ["clicks", { clicks: { count: 2, at: NOW - 10 } }], + ["mock", { mockAt: NOW - 10 }], + ["watch", { typingAt: NOW - 10 }], + ["stretch", { wakeAt: NOW - 10 }], + ["muse", { monologue: true }], + ["lying", { idle: 3 }], +]; +for (let i = 0; i < ladder.length; i++) { + const all = Object.assign({}, ...ladder.slice(i).map(l => l[1]).reverse()); + assert.strictEqual(state(all), ladder[i][0], "everything from " + ladder[i][0] + " down"); +} + +// Shoved has a frame of its own: knocked off balance, glaring at the balloon. +assert.deepStrictEqual(M.pick(calm({ shoved: true })), { state: "shoved", frame: "shoved" }); + +// Idle levels and their frames. +assert.deepStrictEqual(M.pick(calm({ idle: 1 })), { state: "bored", frame: "sleepy" }); +assert.deepStrictEqual(M.pick(calm({ idle: 2 })), { state: "yawn", frame: "yawn" }); +assert.deepStrictEqual(M.pick(calm({ idle: 3 })), { state: "lying", frame: "lying" }); + +// Holds end on time. +assert.strictEqual(state({ repliedAt: NOW - M.T.waveHold }), "base"); +assert.strictEqual(state({ typingAt: NOW - M.T.typingHold }), "base"); +assert.strictEqual(state({ wakeAt: NOW - M.T.wakeHold }), "base"); +assert.strictEqual(state({ clicks: { count: 3, at: NOW - M.T.clickHold } }), "base"); +assert.strictEqual(state({ destructAt: NOW - M.T.destruct - M.T.scorched }), "base"); +// The mock points first, then smirks. +assert.strictEqual(frame({ mockAt: NOW - 10 }), "point"); +assert.strictEqual(frame({ mockAt: NOW - M.T.mockHold / 2 - 10 }), "smirk"); +assert.strictEqual(state({ mockAt: NOW - M.T.mockHold }), "base"); + +// A click burst: within burstGap it grows, past it it starts over. +let c = { count: 0, at: 0 }; +c = M.click(c, 1000); +assert.deepStrictEqual(c, { count: 1, at: 1000 }); +c = M.click(c, 1000 + M.T.burstGap); +assert.strictEqual(c.count, 2); +c = M.click(c, c.at + M.T.burstGap + 1); +assert.strictEqual(c.count, 1); +c = { count: 0, at: 0 }; +for (let i = 0; i < 7; i++) c = M.click(c, 5000 + i * 300); +assert.strictEqual(c.count, 7); +// 1 smiles, 2-6 blush harder. +assert.deepStrictEqual([1, 2, 3, 4, 5, 6].map(M.clickFrame), + ["smile", "blush1", "blush1", "blush2", "blush2", "blush3"]); + +// Outfit: the dress until the flash is over, then the leotard for +// scorched + skirtless, then the dress again. +assert.strictEqual(M.outfit(0, NOW), "dress"); +assert.strictEqual(M.outfit(NOW - 10, NOW), "dress"); +assert.strictEqual(M.outfit(NOW - M.T.destruct, NOW), "leotard"); +const bare = M.T.destruct + M.T.scorched + M.T.skirtless; +assert.strictEqual(M.outfit(NOW - bare + 10, NOW), "leotard"); +assert.strictEqual(M.outfit(NOW - bare, NOW), "dress"); + +// Cooldowns. +assert.strictEqual(M.mockAllowed(0, NOW), true); +assert.strictEqual(M.mockAllowed(NOW - M.T.mockCooldown + 1, NOW), false); +assert.strictEqual(M.mockAllowed(NOW - M.T.mockCooldown, NOW), true); +assert.strictEqual(M.canSpeak(0, NOW), true); +assert.strictEqual(M.canSpeak(NOW - M.T.lineCooldown + 1, NOW), false); +assert.strictEqual(M.canSpeak(NOW - M.T.lineCooldown, NOW), true); + +// The monologue: 1 in 4 on boredom, 1 in 3 on being shown, once an hour. +assert.strictEqual(M.monologue("bored", 0, NOW, 0.24), true); +assert.strictEqual(M.monologue("bored", 0, NOW, 0.25), false); +assert.strictEqual(M.monologue("shown", 0, NOW, 0.33), true); +assert.strictEqual(M.monologue("shown", 0, NOW, 0.34), false); +assert.strictEqual(M.monologue("other", 0, NOW, 0), false); +assert.strictEqual(M.monologue("bored", NOW - M.T.monologueCooldown + 1, NOW, 0), false); +assert.strictEqual(M.monologue("bored", NOW - M.T.monologueCooldown, NOW, 0), true); + +console.log("ok"); +``` + +- [ ] **Step 2: Run it to see it fail** + +Run: `node desktop/companion/test_mood.js` +Expected: FAIL, `ENOENT: no such file or directory, open '.../Mood.js'`. + +- [ ] **Step 3: Write `Mood.js`** + +`desktop/companion/Mood.js` (GPLv2 header first): + +```js +.pragma library + +// Which state the companion is in, from a snapshot of her inputs. Pure, so +// test_mood.js drives it with plain objects and a fake clock. Companion.qml +// owns the inputs and the timers; this only decides. Times in milliseconds. + +var T = { + burstGap: 1500, // a click burst ends after this long without a click + clickHold: 2000, // a click reaction lasts this long after the last click + mockHold: 2500, + mockCooldown: 20000, + typingHold: 2000, + waveHold: 3000, + wakeHold: 1500, // the stretch after idling + lineCooldown: 30000, + monologueCooldown: 3600000, + destruct: 1000, // the glow and the flash; the sprite is hidden under it + scorched: 3000, + skirtless: 60000, +}; + +function st(state, frame) { return { state: state, frame: frame }; } + +// The burst after a click at now. +function click(prev, now) { + if (prev.count > 0 && now - prev.at <= T.burstGap) return { count: prev.count + 1, at: now }; + return { count: 1, at: now }; +} + +// Click 1 smiles; 2 to 6 blush harder, in three steps. The 7th is the +// self-destruct, which Companion.qml starts rather than shows. +function clickFrame(count) { + if (count <= 1) return "smile"; + if (count <= 3) return "blush1"; + if (count <= 5) return "blush2"; + return "blush3"; +} + +// The spec's priority table, top to bottom: the first that applies wins. +// idle is 0 awake, 1 bored, 2 yawning, 3 lying down. +function pick(s) { + const since = s.destructAt ? s.now - s.destructAt : Infinity; + if (since < T.destruct) return st("destruct", "base"); + if (since < T.destruct + T.scorched) return st("scorched", "scorched"); + if (s.dodging) return st("dodge", "base"); + if (s.shoved) return st("shoved", "shoved"); + if (s.recording) return st("listen", "listen"); + if (s.busy) return st("think", "think"); + if (s.now - s.repliedAt < T.waveHold) return st("wave", "wave"); + if (s.clicks.count > 0 && s.now - s.clicks.at < T.clickHold) + return st("clicks", clickFrame(s.clicks.count)); + if (s.now - s.mockAt < T.mockHold) + return st("mock", s.now - s.mockAt < T.mockHold / 2 ? "point" : "smirk"); + if (s.now - s.typingAt < T.typingHold) return st("watch", "watch"); + if (s.now - s.wakeAt < T.wakeHold) return st("stretch", "stretch"); + if (s.monologue) return st("muse", "muse"); + if (s.idle >= 3) return st("lying", "lying"); + if (s.idle === 2) return st("yawn", "yawn"); + if (s.idle === 1) return st("bored", "sleepy"); + return st("base", "base"); +} + +// The self-destruct takes the skirt off once the flash is over, for the +// scorched moment and the skirtless minute after it. Showing her again +// sets destructAt back to 0, which restores it early. +function outfit(destructAt, now) { + if (!destructAt) return "dress"; + const since = now - destructAt; + return since >= T.destruct && since < T.destruct + T.scorched + T.skirtless ? "leotard" : "dress"; +} + +function mockAllowed(lastMockAt, now) { return now - lastMockAt >= T.mockCooldown; } + +function canSpeak(lastLineAt, now) { return now - lastLineAt >= T.lineCooldown; } + +// Whether the opening monologue plays: a chance per trigger, and never +// twice within the hour. roll is Math.random(), passed in for the test. +function monologue(trigger, lastAt, now, roll) { + if (now - lastAt < T.monologueCooldown) return false; + const chance = trigger === "bored" ? 1 / 4 : trigger === "shown" ? 1 / 3 : 0; + return roll < chance; +} +``` + +- [ ] **Step 4: Run it to see it pass** + +Run: `node desktop/companion/test_mood.js` +Expected: `ok` + +- [ ] **Step 5: Commit** + +```bash +git add desktop/companion/Mood.js desktop/companion/test_mood.js +git commit -F - <<'EOF' +companion: Mood.js, which state wins + +The spec's priority table as a pure function over a snapshot of her +inputs, with the click burst, the outfit after the self-destruct, the +mock and line cooldowns and the monologue's odds beside it. Pure so the +node check can drive it with a fake clock; Companion.qml will own the +inputs and timers. + +Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> +EOF +``` + +--- + +### Task 4: Lines.js, what she says + +**Files:** +- Create: `desktop/companion/Lines.js` + +- [ ] **Step 1: Write `Lines.js`** + +GPLv2 header first, then: + +```js +.pragma library + +// What she says, in English, grouped by the reaction that says it. +// Companion.qml picks one at random per reaction and keeps to one line in +// 30 s, except for clicks and the self-destruct (Mood.canSpeak). + +var lines = { + shown: ["Glory to mankind.", "2B, reporting.", "...Back on duty."], + bored: ["...Operator, I'm bored.", "Nothing to report.", "Awaiting orders."], + wake: ["I'm awake.", "Systems online."], + mock: ["Typo again?", "Delete, delete, delete.", "Were you aiming for that key?"], + shoved: ["Hey.", "Rude.", "I was standing here."], + ready: ["Answer's ready.", "Your reply came in."], + click: ["...Yes?", "Can I help you?"], + blush: ["S-stop that.", "...Operator."], + dodge: ["...Fine.", "I'll move."], + destruct: ["Self-destruct sequence initiated."], + aftermath: ["...That was unpleasant."], +}; + +// The game's opening, one sentence per bubble. +var monologue = [ + "Everything that lives is designed to end.", + "We are perpetually trapped in a never-ending spiral of life and death.", + "Is this a curse?", + "Or some kind of punishment?", + "I often think about the god who blessed us with this cryptic puzzle...and wonder if we'll ever get the chance to kill him.", +]; + +// A line for kind; roll is Math.random(). "" for a kind with no lines. +function pick(kind, roll) { + const l = lines[kind] || []; + return l.length ? l[Math.min(l.length - 1, Math.floor(roll * l.length))] : ""; +} +``` + +- [ ] **Step 2: Check it evaluates** + +Run: +```bash +node -e 'const s=require("fs").readFileSync("desktop/companion/Lines.js","utf8").replace(/^\.pragma library$/m,""); const L=new Function(s+"\nreturn {lines,monologue,pick};")(); console.log(L.pick("mock",0), "|", L.pick("mock",0.999), "|", JSON.stringify(L.pick("nope",0)), "|", L.monologue.length)' +``` +Expected: `Typo again? | Were you aiming for that key? | "" | 5` + +- [ ] **Step 3: Commit** + +```bash +git add desktop/companion/Lines.js +git commit -F - <<'EOF' +companion: her lines and the opening monologue + +Fixed English lines per reaction, terse in the game's register, and the +opening monologue split at its sentences so each fills one bubble. + +Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> +EOF +``` + +--- + +### Task 5: prep-frames, and a placeholder base + +> **Done before implementation (2026-10-23), skip it.** `prep-frames` was committed with `gen-frames` and `frames.txt` (14a2c75), and every frame exists: 18 dress, 19 leotard, in `~/.local/share/keystone/companion/2b/frames/`. Run `desktop/companion/prep-frames` once to confirm `37 frames`. + +**Files:** +- Create: `desktop/companion/prep-frames` (executable) + +- [ ] **Step 1: Write the script** + +```bash +#!/bin/bash +# +# (GPLv2 header, as in desktop/modules/ai/ai-state.sh lines 2-12) +# +# prep-frames: the companion's generated images to the frames she shows. +# +# prep-frames [ART_DIR] default: ~/.local/share/keystone/companion/2b +# +# Reads ART_DIR/dress/*.png and ART_DIR/leotard/*.png, 1024x1024 on white, +# and writes ART_DIR/frames/<outfit>/<name>.png: background removed, trimmed +# to her, scaled by a quarter. One scale for every frame keeps her the same +# size whatever the pose, which scaling each to a fixed height would not: +# lying down would come out huge. Sprite.qml draws them at half size and +# stands each on its bottom edge. +# +# The background goes by flood fill from the four corners: the dark line art +# stops the fill, so the white hair and the white leotard inside it stay. + +set -euo pipefail + +art=${1:-$HOME/.local/share/keystone/companion/2b} +n=0 +for outfit in dress leotard; do + [[ -d $art/$outfit ]] || continue + mkdir -p "$art/frames/$outfit" + for src in "$art/$outfit"/*.png; do + [[ -e $src ]] || continue + read -r w h < <(magick identify -format '%w %h\n' "$src") + magick "$src" -alpha set -fuzz 6% -fill none \ + -draw "color 0,0 floodfill" -draw "color $((w - 1)),0 floodfill" \ + -draw "color 0,$((h - 1)) floodfill" -draw "color $((w - 1)),$((h - 1)) floodfill" \ + -trim +repage -resize 25% "$art/frames/$outfit/$(basename "$src")" + n=$((n + 1)) + done +done +echo "prep-frames: $n frames in $art/frames" +``` + +Then `chmod +x desktop/companion/prep-frames`. + +- [ ] **Step 2: Check it on a copy of the sitting base** + +Run: +```bash +T=$(mktemp -d) && mkdir "$T/dress" && cp ~/.local/share/keystone/companion/2b/base-sitting.png "$T/dress/base.png" \ + && desktop/companion/prep-frames "$T" \ + && magick "$T/frames/dress/base.png" -format '%w %h corner=%[fx:p{0,0}.a] hair=%[fx:p{w/2,h*0.15}.a]\n' info: ; rm -rf "$T" +``` +Expected: `prep-frames: 1 frames in ...`, then about `173 255 corner=0 hair=1`: transparent corner, opaque hair. A `hair=0` means the fill ate her hair: stop and report. + +- [ ] **Step 3: Put a placeholder base in place** + +The standing base is still being generated (spec, "She stands"). Until it exists the sitting one stands in, so the component has a frame to show: + +```bash +A=~/.local/share/keystone/companion/2b +mkdir -p "$A/dress" && [ -e "$A/dress/base.png" ] || cp "$A/base-sitting.png" "$A/dress/base.png" +desktop/companion/prep-frames +ls "$A/frames/dress" +``` +Expected: `base.png` listed. If the user has already delivered the standing base, it is used instead: the `[ -e ]` keeps it. + +- [ ] **Step 4: Commit** + +```bash +git add desktop/companion/prep-frames +git commit -F - <<'EOF' +companion: prep-frames, generated images to display frames + +Background removed by flood fill from the corners, which the line art +stops, so white hair survives; trimmed; scaled by one quarter for every +frame, so she stays one size across poses. The art itself stays outside +the repo. + +Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> +EOF +``` + +--- + +### Task 6: The assistant says when a reply arrives + +**Files:** +- Modify: `desktop/assistant/Assistant.qml` (after line 76, `readonly property bool busy: ...`; and the `case "done":` branch, line ~408) + +- [ ] **Step 1: Add the signal** + +After `readonly property bool busy: root.state !== "idle"`: + +```qml + // A reply finished (not stopped). The canvas notifies through notifyDone; + // the desktop companion waves. + signal replied() +``` + +- [ ] **Step 2: Emit it** + +In `case "done":`, replace + +```qml + if (!e.stopped && i >= 0) root.notifyDone(turns.get(i).reply); +``` + +with + +```qml + if (!e.stopped && i >= 0) { + root.notifyDone(turns.get(i).reply); + root.replied(); + } +``` + +- [ ] **Step 3: Check the reload** + +Run: `sleep 2; qs log -p ~/Programming/GIT/quickshell/desktop --log-times | tail -5` +Expected: a `Reloading configuration...` after the edit's mtime and `Configuration Loaded`, no error. If no reload line follows, see AGENTS.md, Reloading: force one by changing a file that imports the directory. + +- [ ] **Step 4: Commit** + +```bash +git add desktop/assistant/Assistant.qml +git commit -F - <<'EOF' +assistant: replied() when a reply finishes + +The desktop companion waves when an answer is ready. The moment already +existed as the done branch that calls notifyDone; it is now a signal +too, so the companion connects to it instead of reading the turn list. + +Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> +EOF +``` + +--- + +### Task 7: Sprite.qml + +**Files:** +- Create: `desktop/companion/Sprite.qml` + +- [ ] **Step 1: Write it** + +GPLv2 header first, then: + +```qml +import QtQuick +import ".." + +// One frame of the companion, drawn at half its pixel size and standing on +// its bottom edge: a crossfade between frames, a slow bob, the drop out of +// view and back, and the self-destruct flash. Which frame is Mood.js's +// business; this only draws it. +Item { + id: s + + // The frame, and what to show when it is missing: the same outfit's + // base, never the other outfit's, so a gap in the art cannot put the + // skirt back mid-scene. + property string source + property string fallback + // Dropped below the bottom edge and faded out. + property bool out: false + signal settled(bool out) + + // Two images: the front one stays until the back one has loaded, then + // they swap, so a frame change never flashes empty. + property bool flip: false + readonly property Image front: s.flip ? b : a + width: s.front.implicitWidth / 2 + height: s.front.implicitHeight / 2 + + function load(url) { + const front = s.flip ? b : a, back = s.flip ? a : b; + if (String(front.source) === url) { back.source = ""; return; } + if (String(back.source) === url && back.status === Image.Ready) { s.flip = !s.flip; return; } + back.source = url; + } + onSourceChanged: s.load(s.source) + Component.onCompleted: s.load(s.source) + + function ready(img) { + if (img.status === Image.Error && String(img.source) !== s.fallback) { + img.source = s.fallback; + return; + } + if (img.status === Image.Ready && img !== s.front) s.flip = !s.flip; + } + + property real drop: 0 + property real bob: 0 + NumberAnimation { + id: dropAnim + target: s; property: "drop"; duration: 320 + onFinished: s.settled(s.out) + } + onOutChanged: { + dropAnim.stop(); + dropAnim.to = s.out ? s.height + 24 : 0; + dropAnim.easing.type = s.out ? Easing.InCubic : Easing.OutBack; + dropAnim.start(); + } + SequentialAnimation on bob { + running: !s.out + loops: Animation.Infinite + NumberAnimation { to: 2; duration: 1900; easing.type: Easing.InOutSine } + NumberAnimation { to: 0; duration: 2300; easing.type: Easing.InOutSine } + } + + Item { + anchors.fill: parent + opacity: 1 - s.drop / (s.height + 24) + transform: Translate { y: s.drop - s.bob } + + Image { + id: a + width: implicitWidth / 2; height: implicitHeight / 2 + anchors { bottom: parent.bottom; horizontalCenter: parent.horizontalCenter } + smooth: true; mipmap: true; asynchronous: true + opacity: s.flip ? 0 : 1 + Behavior on opacity { NumberAnimation { duration: 120 } } + onStatusChanged: s.ready(a) + } + Image { + id: b + width: implicitWidth / 2; height: implicitHeight / 2 + anchors { bottom: parent.bottom; horizontalCenter: parent.horizontalCenter } + smooth: true; mipmap: true; asynchronous: true + opacity: s.flip ? 1 : 0 + Behavior on opacity { NumberAnimation { duration: 120 } } + onStatusChanged: s.ready(b) + } + } + + // --- self-destruct: the glow swells into a flash, then a ring and + // sparks burst out. About 1.1 s, matching Mood.T.destruct, during which + // the frame swaps to the leotard under the flash. + + function explode() { boom.restart(); } + + property real burst: 0 + Rectangle { + id: flash + anchors.centerIn: parent + width: s.height * 1.6; height: width; radius: width / 2 + color: "white"; opacity: 0; scale: 0 + } + Rectangle { + id: ring + anchors.centerIn: parent + width: s.height * 2; height: width; radius: width / 2 + color: "transparent" + border { width: 3; color: Theme.peach } + opacity: 0; scale: 0 + } + Repeater { + model: 8 + Rectangle { + required property int index + readonly property real angle: index * Math.PI / 4 + width: 6; height: 6; radius: 3 + color: index % 2 ? Theme.peach : Theme.yellow + x: s.width / 2 - 3 + Math.cos(angle) * s.burst * s.height + y: s.height / 2 - 3 + Math.sin(angle) * s.burst * s.height + opacity: s.burst > 0 && s.burst < 1 ? 1 - s.burst : 0 + } + } + SequentialAnimation { + id: boom + ParallelAnimation { + NumberAnimation { target: flash; property: "scale"; from: 0; to: 1; duration: 600; easing.type: Easing.InQuad } + NumberAnimation { target: flash; property: "opacity"; from: 0; to: 0.95; duration: 600 } + } + ParallelAnimation { + NumberAnimation { target: flash; property: "opacity"; to: 0; duration: 400 } + NumberAnimation { target: ring; property: "scale"; from: 0.2; to: 1.4; duration: 400; easing.type: Easing.OutCubic } + NumberAnimation { target: ring; property: "opacity"; from: 1; to: 0; duration: 400 } + NumberAnimation { target: s; property: "burst"; from: 0; to: 1; duration: 500; easing.type: Easing.OutCubic } + } + PropertyAction { target: s; property: "burst"; value: 0 } + } +} +``` + +No commit yet: nothing loads it until Task 8, and they are committed together. + +--- + +### Task 8: Companion.qml and the wiring + +**Files:** +- Create: `desktop/companion/Companion.qml` +- Modify: `desktop/shell.qml` (imports at line 42; the `Assistant { ... }` line ~124) + +- [ ] **Step 1: Write `Companion.qml`** + +GPLv2 header first, then: + +```qml +import Quickshell +import Quickshell.Io +import Quickshell.Wayland +import Quickshell.Hyprland +import QtQuick +import ".." +import "../NotifyConfig.js" as NC +import "Mood.js" as Mood +import "Lines.js" as Lines + +// The desktop companion: a chibi 2B in the bottom-right corner of the +// focused monitor. Inputs and timers live here, the decision in Mood.js, +// the drawing in Sprite.qml. See README.md here and +// docs/superpowers/specs/2026-10-23-companion-design.md. +PanelWindow { + id: root + + required property var asst + + readonly property string art: Quickshell.env("HOME") + "/.local/share/keystone/companion/2b/frames" + + // SUPER+\, through the IPC below. + property bool present: true + + // The monitor she is on trails the focused one: a change plays her exit + // here first, and only then moves the window (settled, below). + readonly property string focused: Hyprland.focusedMonitor?.name ?? "" + property string here: "" + property bool hopping: false + onFocusedChanged: { + if (root.here === "") root.here = root.focused; + else if (root.focused !== root.here) root.hopping = true; + } + Component.onCompleted: root.here = root.focused + + screen: Quickshell.screens.find(s => s.name === root.here) ?? Quickshell.screens[0] + anchors { right: true; bottom: true } + margins { right: 24; bottom: 16 } + implicitWidth: 760 + implicitHeight: 260 + color: "transparent" + exclusionMode: ExclusionMode.Ignore + WlrLayershell.layer: WlrLayer.Top + WlrLayershell.namespace: "keystone-companion" + WlrLayershell.keyboardFocus: WlrKeyboardFocus.None + // Her body only: clicks anywhere else, the bubble included, reach the + // window below. Out of view the region is empty. + mask: Region { item: hit } + + // --- inputs --------------------------------------------------------- + + property double now: Date.now() + Timer { interval: 200; repeat: true; running: root.present; onTriggered: root.now = Date.now() } + + property var clicks: ({ count: 0, at: 0 }) + property double destructAt: 0 + property bool dodging: false + property double repliedAt: 0 + property double mockAt: 0 + property double typingAt: 0 + property double wakeAt: 0 + property double lastLineAt: 0 + property double lastMonologueAt: 0 + property bool monologue: false + + // Bored, yawning, lying down at 1, 3 and 5 minutes. Inhibitors count, + // so presentation mode keeps her awake. enabled is set explicitly: left + // out, it read false on creation. + IdleMonitor { id: idle1; enabled: true; respectInhibitors: true; timeout: 60 } + IdleMonitor { id: idle2; enabled: true; respectInhibitors: true; timeout: 180 } + IdleMonitor { id: idle3; enabled: true; respectInhibitors: true; timeout: 300 } + readonly property int idle: idle3.isIdle ? 3 : idle2.isIdle ? 2 : idle1.isIdle ? 1 : 0 + property int wasIdle: 0 + onIdleChanged: { + const t = Date.now(); + if (root.idle === 0 && root.wasIdle > 0) { + root.wakeAt = t; + root.say("wake"); + } else if (root.idle === 1 && root.wasIdle === 0) { + if (Mood.monologue("bored", root.lastMonologueAt, t, Math.random())) root.startMonologue(); + else root.say("bored"); + } + root.wasIdle = root.idle; + } + + // 2 is real fullscreen; 1 is maximised, which is most windows here and + // must not hide her. lastIpcObject is only current after a refresh, and + // the island measured that it settles a moment after the event. + property int fullscreen: 0 + Timer { + id: fsSettle + interval: 60 + onTriggered: root.fullscreen = Hyprland.activeToplevel?.lastIpcObject?.fullscreen ?? 0 + } + + Connections { + target: Hyprland + function onRawEvent(event) { + if (event.name === "custom") { + // From hypr-theme's sections/companion.lua. + const what = event.data.trim(); + const t = Date.now(); + if (what === "keystone-typing") { + root.typingAt = t; + } else if (what === "keystone-delete") { + root.typingAt = t; + if (Mood.mockAllowed(root.mockAt, t)) { + root.mockAt = t; + root.say("mock"); + } + } + } else if (["fullscreen", "activewindowv2", "workspacev2"].includes(event.name)) { + Hyprland.refreshToplevels(); + fsSettle.restart(); + } + } + } + + Connections { + target: root.asst + function onReplied() { + root.repliedAt = Date.now(); + root.say("ready"); + } + } + + // Balloons stack bottom-right of their screen. When that is her corner, + // any balloon on screen shoves her left past the stack. + readonly property bool shoved: { + const place = Notify.config.balloons; + if (place.screen !== root.here || place.corner !== "bottom-right") return false; + return (Notify.queue || []).some(n => NC.shown(Notify.config, n, Notify.dnd, + Notify.drawerOpen, Notify.snoozeUntil, root.now)); + } + onShovedChanged: if (root.shoved) root.say("shoved") + + // --- state ---------------------------------------------------------- + + readonly property var mood: Mood.pick({ + now: root.now, destructAt: root.destructAt, dodging: root.dodging, shoved: root.shoved, + recording: root.asst.recording, busy: root.asst.busy, repliedAt: root.repliedAt, + clicks: root.clicks, mockAt: root.mockAt, typingAt: root.typingAt, wakeAt: root.wakeAt, + monologue: root.monologue, idle: root.idle, + }) + readonly property string outfit: Mood.outfit(root.destructAt, root.now) + // Anything above the monologue cuts it short. + onMoodChanged: if (root.monologue && root.mood.state !== "muse") root.stopMonologue() + + // Hidden: no sprite, no bubble. Away adds the dodge, which hides the + // sprite but lets its line show. + readonly property bool hidden: !root.present || root.fullscreen === 2 || root.hopping + readonly property bool away: root.hidden || root.dodging + + // --- speech --------------------------------------------------------- + + function say(kind, force) { + const t = Date.now(); + if (root.hidden || (!force && !Mood.canSpeak(root.lastLineAt, t))) return; + const line = Lines.pick(kind, Math.random()); + if (line === "") return; + root.lastLineAt = t; + bubble.show(line, 3000); + } + + // The opening monologue: about 15 characters a second, one sentence + // per bubble, 1.5 s between sentences, the last held 4 s. + property int monoLine: 0 + function startMonologue() { + if (root.hidden) return; + root.monologue = true; + root.lastMonologueAt = Date.now(); + root.lastLineAt = root.lastMonologueAt; + root.monoLine = 0; + typer.begin(Lines.monologue[0]); + } + function stopMonologue() { + root.monologue = false; + typer.stop(); + typerHold.stop(); + bubble.hide(); + } + Timer { + id: typer + property string full: "" + interval: 66 + repeat: true + function begin(text) { typer.full = text; bubble.typed(""); typer.restart(); } + onTriggered: { + const n = bubble.text.length + 1; + bubble.typed(typer.full.slice(0, n)); + if (n < typer.full.length) return; + typer.stop(); + typerHold.interval = root.monoLine === Lines.monologue.length - 1 ? 4000 : 1500; + typerHold.restart(); + } + } + Timer { + id: typerHold + onTriggered: { + root.monoLine++; + if (root.monoLine >= Lines.monologue.length) root.stopMonologue(); + else typer.begin(Lines.monologue[root.monoLine]); + } + } + + // --- clicks and the dodge ------------------------------------------- + + function clicked() { + const t = Date.now(); + root.now = t; + root.clicks = Mood.click(root.clicks, t); + if (root.clicks.count >= 7) { + root.clicks = { count: 0, at: 0 }; + root.stopMonologue(); + root.destructAt = t; + sprite.explode(); + root.say("destruct", true); + aftermath.restart(); + } else if (root.clicks.count === 1) { + root.say("click", true); + } else if (root.clicks.count === 4) { + root.say("blush", true); + } + } + Timer { id: aftermath; interval: Mood.T.destruct + 400; onTriggered: root.say("aftermath", true) } + + // She cannot see the pointer coming, only feel it on her: Quickshell has + // no global pointer and Hyprland sends no motion events. A click before + // the 0.6 s still reaches her. + Timer { + id: hover + interval: 600 + onTriggered: { + if (Math.random() < 0.5) root.say("dodge"); + root.dodging = true; + dodgeBack.restart(); + } + } + Timer { id: dodgeBack; interval: 5000; onTriggered: root.dodging = false } + + // --- showing and hiding ---------------------------------------------- + + function appear() { + if (root.present) return; + root.present = true; + root.destructAt = 0; // the skirt is back + root.now = Date.now(); + greet.restart(); + } + function dismiss() { + if (!root.present) return; + root.stopMonologue(); + bubble.hide(); + root.present = false; + } + // After the drop-in has landed. + Timer { + id: greet + interval: 450 + onTriggered: { + if (Mood.monologue("shown", root.lastMonologueAt, Date.now(), Math.random())) root.startMonologue(); + else root.say("shown", true); + } + } + + // qs -p <desktop> ipc call companion toggle -> SUPER+\ + IpcHandler { + target: "companion" + function toggle() { if (root.present) root.dismiss(); else root.appear(); } + function show() { root.appear(); } + function hide() { root.dismiss(); } + } + + // The exit has played on the old monitor: move, and the entrance plays. + function settled(out) { + if (out && root.hopping) { + root.here = root.focused; + root.hopping = false; + } + } + + // --- drawing -------------------------------------------------------- + + // Where she stands: the bottom-right of the window, or left of the + // balloon stack. 336 = the stack's 340 px plus its 12 px margin + // (notifications/Balloons.qml), minus this window's 24 px right margin, + // plus 8 px of air. + Item { + id: seat + width: sprite.width + height: sprite.height + anchors.bottom: parent.bottom + x: root.width - width - (root.shoved ? 336 : 0) + Behavior on x { NumberAnimation { duration: 260; easing.type: Easing.OutCubic } } + } + + Sprite { + id: sprite + x: seat.x + y: seat.y + source: `file://${root.art}/${root.outfit}/${root.mood.frame}.png` + fallback: `file://${root.art}/${root.outfit}/base.png` + out: root.away + onSettled: out => root.settled(out) + } + + Item { + id: hit + x: sprite.x + y: sprite.y + width: root.away ? 0 : sprite.width + height: root.away ? 0 : sprite.height + MouseArea { + anchors.fill: parent + hoverEnabled: true + cursorShape: Qt.PointingHandCursor + onEntered: hover.restart() + onExited: hover.stop() + onClicked: { hover.stop(); root.clicked(); } + } + } + + // Up and to the left of her head, anchored to where she stands rather + // than to the sprite, so a dodge line stays where she was. + Rectangle { + id: bubble + property alias text: label.text + function show(t, ms) { label.text = t; bubble.opacity = 1; hideTimer.interval = ms; hideTimer.restart(); } + function typed(t) { label.text = t; bubble.opacity = 1; hideTimer.stop(); } + function hide() { hideTimer.stop(); bubble.opacity = 0; } + + visible: !root.hidden && opacity > 0 && label.text !== "" + anchors { right: seat.left; rightMargin: -14; bottom: seat.top; bottomMargin: -22 } + width: Math.min(260, label.implicitWidth + 24) + height: label.implicitHeight + 16 + radius: 12 + color: Qt.alpha(Theme.base, 0.92) + border { width: 1; color: Qt.alpha(Theme.text, 0.25) } + opacity: 0 + Behavior on opacity { NumberAnimation { duration: 180 } } + + Timer { id: hideTimer; onTriggered: bubble.opacity = 0 } + + Text { + id: label + anchors.centerIn: parent + width: Math.min(236, implicitWidth) + wrapMode: Text.WordWrap + color: Theme.text + font { family: Theme.fontFamily; pixelSize: 13 } + } + Rectangle { + width: 10; height: 10; rotation: 45 + color: bubble.color + anchors { right: parent.right; rightMargin: 18; bottom: parent.bottom; bottomMargin: -5 } + } + } +} +``` + +- [ ] **Step 2: Wire it into the shell** + +In `desktop/shell.qml`, after `import "assistant"` (line 42): + +```qml +import "companion" +``` + +and after the `Assistant { id: assistant; ... }` line: + +```qml + + // The desktop companion, bottom-right of the focused monitor, on SUPER+\. + // She reads the assistant's state. See companion/README.md. + Companion { asst: assistant } +``` + +- [ ] **Step 3: Check the reload and the layer** + +Run: +```bash +sleep 2; qs log -p ~/Programming/GIT/quickshell/desktop --log-times | tail -8 +hyprctl layers -j | jq -r '.[] | .levels[]?[] | select(.namespace=="keystone-companion") | "x=\(.x) y=\(.y) w=\(.w) h=\(.h)"' +``` +Expected: `Configuration Loaded` with no `companion` error or warning after it, and one layer line, `w=760 h=260`, at the bottom-right of the focused monitor. A `ReferenceError` naming `Companion` means the new directory was not indexed: see AGENTS.md, Reloading, and ask the user to run `./reload.sh desktop`. + +- [ ] **Step 4: Run both JS checks again** + +Run: `node desktop/companion/test_mood.js && node shared/test_notify_config.js` +Expected: `ok` twice. + +- [ ] **Step 5: Commit** + +```bash +git add desktop/companion/Sprite.qml desktop/companion/Companion.qml desktop/shell.qml +git commit -F - <<'EOF' +companion: the window, her inputs and her drawing + +Companion.qml is a Top-layer PanelWindow bottom-right of the focused +monitor, masked to her body so every other click passes through. It +gathers the inputs (Hyprland's custom typing and delete events, the +assistant's state and replied signal, the balloons on her screen, +three IdleMonitors, clicks and a hover), hands a snapshot to Mood.js, +and speaks from Lines.js. Sprite.qml crossfades frames, bobs, drops +out of view and back, and plays the self-destruct. + +Maximised windows report fullscreen 1 and most windows here are, so +only 2 hides her. She only feels the pointer once it is on her, so the +dodge waits 0.6 s and a quick click still lands. + +Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> +EOF +``` + +--- + +### Task 9: Notes + +**Files:** +- Create: `desktop/companion/README.md` +- Modify: `AGENTS.md` (the directory listing under "What this is") +- Modify: `docs/superpowers/specs/2026-10-23-companion-design.md` (`lines.js` to `Lines.js`) + +- [ ] **Step 1: Write the README** + +```markdown +# companion + +A chibi 2B in the bottom-right corner of the focused monitor, on SUPER+\. +The design, with what was measured on the way, is +`docs/superpowers/specs/2026-10-23-companion-design.md`. + +## Files + +| File | | +| --- | --- | +| `Companion.qml` | the window, inputs, timers, speech, IPC (`companion toggle/show/hide`) | +| `Sprite.qml` | draws a frame: crossfade, bob, drop out and back, explosion | +| `Mood.js` | which state wins, from a snapshot; all the timings are its `T` | +| `Lines.js` | her lines and the opening monologue | +| `prep-frames` | generated images to display frames | + +## Art + +Not in the repo: the character is Square Enix's and this repo is GPLv2. +It lives in `~/.local/share/keystone/companion/2b/`: + + dress/<name>.png, leotard/<name>.png generated, 1024x1024 on white + frames/<outfit>/<name>.png what she shows, from prep-frames + recipe.txt how each was made, and what failed + +Run `prep-frames` after adding or replacing any generated image. A missing +frame falls back to its outfit's `base`. + +## Inputs + +- Typing and BackSpace come from `sections/companion.lua` in the + hypr-theme repo, as `custom>>keystone-typing` and + `custom>>keystone-delete` on socket2. Nothing else about the keyboard + reaches her. +- She only feels the pointer while it is over her: Quickshell has no + global pointer and Hyprland sends no motion events. Hence the 0.6 s + dodge rather than moving away as the pointer approaches. +- `IdleMonitor.enabled` read false on creation when left out, so each of + the three sets it. +- Fullscreen is `lastIpcObject.fullscreen === 2`. Maximised is 1, and + most windows here are maximised. +- The lock screen covers every layer, so locking needs no code. + +## Check + + node desktop/companion/test_mood.js +``` + +- [ ] **Step 2: List the component in AGENTS.md** + +In the directory listing under "What this is", after the `assistant/` line, add: + +``` + companion/ a chibi 2B, bottom-right of the focused monitor, on SUPER+\ +``` + +- [ ] **Step 3: Fix the spec's file name** + +In `docs/superpowers/specs/2026-10-23-companion-design.md`, replace `lines.js` with `Lines.js` (two places: the file tree and "Fixed English lines from `lines.js`"). + +- [ ] **Step 4: Commit** + +```bash +git add desktop/companion/README.md AGENTS.md docs/superpowers/specs/2026-10-23-companion-design.md +git commit -F - <<'EOF' +companion: README, AGENTS listing, Lines.js in the spec + +The README records the measurements the code depends on: where typing +and BackSpace come from, why the dodge waits for the pointer, the +IdleMonitor enabled default, and fullscreen 2 against maximised 1. + +Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> +EOF +``` + +--- + +### Task 10: Restart and check by hand + +Visual, so the user does it. Ask them to run, from their own terminal: + +``` +./reload.sh desktop notifications +``` + +(both link the edited `shared/NotifyConfig.js`; hot reload does not follow the link). Then go through, one at a time, and record the results in the README if anything differs: + +- [ ] She appears bottom-right of the focused monitor; SUPER+\ hides and shows her, with a greeting line. +- [ ] Focus DP-3, then DP-1: she drops out on one and in on the other. +- [ ] `notify-send test hello`: she slides left of the balloon, looks mad, says a line; slides back when it expires. +- [ ] Type in a terminal: she watches. BackSpace: she points, then smirks; again within 20 s: no second mock. +- [ ] Ask the assistant something with the canvas closed: think while it works, wave when it replies. +- [ ] Rest the pointer on her: after about 0.6 s she drops away and returns 5 s later. A quick click instead: a smile. +- [ ] Seven quick clicks: glow, flash, sparks, "Self-destruct sequence initiated.", then "...That was unpleasant." +- [ ] A fullscreen video (F11 in the browser) hides her; a maximised window does not. This is the spec's open check 3. +- [ ] Leave the machine 1 minute: bored (or the monologue); 3: yawn; 5: lying down; touch the mouse: stretch. +- [ ] Clicks anywhere beside her and on the bubble reach the window underneath. + +Until the frames are made, every state shows the base: that is the fallback working, not a bug. + +--- + +### Task 11: The frames (the user's, with sd-webui) + +> **Done (2026-10-23).** Made with `desktop/companion/gen-frames` from `frames.txt`, not by hand in the web UI; `shoved` was added as an 18th frame per set. The list below is kept as the record of what the set contains. + +Not code: a checklist for generating the art with the recipes in `recipe.txt`. Each image goes into `~/.local/share/keystone/companion/2b/dress/` or `leotard/` under the frame's name, then `desktop/companion/prep-frames`. + +- [x] `dress/base.png`: klein edit of `base-sitting.png`, the standing prompt (delivered 2026-10-23, before Task 5, so its placeholder step keeps it) +- [ ] img2img from `dress/base.png`, strength 0.75: `smile`, `mad`, `smirk`, `blush1`, `blush2`, `blush3`, `sleepy` +- [ ] klein edits of `dress/base.png`: `wave`, `watch`, `point`, `yawn`, `lying`, `listen`, `think`, `muse`, `stretch` +- [ ] klein edit of each dress frame into `leotard/<same name>`: "her skirt is gone, revealing a plain white leotard; change nothing else" +- [ ] klein edit of `leotard/base.png`: `leotard/scorched` diff --git a/docs/superpowers/specs/2026-10-23-companion-design.md b/docs/superpowers/specs/2026-10-23-companion-design.md new file mode 100644 index 0000000..8acffbf --- /dev/null +++ b/docs/superpowers/specs/2026-10-23-companion-design.md @@ -0,0 +1,232 @@ +# Desktop companion: design + +Date: 2026-10-23. Status: designed, not implemented. + +## Goal + +A small chibi 2B (NieR: Automata) who lives in the bottom-right corner of +the focused monitor, follows focus between DP-1 and DP-3, and reacts to +what happens on the desktop: notifications shove her, typing makes her +watch, deleting makes her mock, the assistant gets a face, idling bores +her, clicks make her blush and, at seven, self-destruct. She talks in short +fixed English lines in a speech bubble. SUPER+\ shows and hides her. + +The idea comes from Modesty's window companion, read for ideas only +(GPLv3; see AGENTS.md). Nothing here is taken from its code. The +difference that shapes everything: this desktop runs mostly maximised +windows, so she lives in a screen corner, not on a window edge. + +## Verified before designing + +- **The art can be made locally.** Pony Diffusion V6 XL with two LoRAs + makes the base frame, img2img makes expressions, FLUX.2 klein makes + poses. Recipes, prompts and what failed are in + `~/.local/share/keystone/companion/2b/recipe.txt`. Measured on the way: + img2img at strength 0.75 changes a face and not an arm, and at 0.85 + moves a hand halfway and starts to drift; klein moves limbs and keeps + the character when every detail to keep is named. +- **sd.cpp 929 cannot load an SDXL diffusers directory**: it detects it + as SD 1.x (leejet/stable-diffusion.cpp#1938). sd-webui's Pony entry now + uses the single-file checkpoint, which loads as SDXL. +- **A resident llama.cpp model starves image generation.** Gemma left + 1.1 GB of 12 GB free and the first unet segment failed; an img2img + encode needed 4.5 GB. sd-webui unloads llama-server's models at start. +- **The white background comes off cleanly.** ImageMagick, fuzz 6%, + flood fill from the four corners, trim, scale to 256 px tall: the line + art stops the fill, so the white hair and the white leotard survive. +- **The look was checked on screen** with a throwaway shell: 128 px tall, + 24 px from the right edge and 16 px from the bottom of DP-1, a dark + translucent bubble up and to the left of her head. Approved by eye. +- **Balloons are bottom-right of DP-1** (`~/.config/notifyd/config.json`, + `"corner": "bottom-right"`), her corner. `shared/Notify.qml` already + exposes the queue, DND, snooze and the drawer state; the check of + which queued balloons are on screen lives in `notifications/`. +- **Hyprland 0.56.2 has what the inputs need**, per + `/usr/share/hypr/stubs/hl.meta.lua`: an `input.keyboard.key` event, a + `non_consuming` bind flag, and an `above_fullscreen` layer rule. None + has been tried on this machine yet; see Open checks. +- **SUPER+\ is free**, and the keyboard carries two layouts, `it,us`. + The `backslash` keysym sits on a different physical key in each, so the + bind uses the physical key, `code:49`, left of 1. + +## Where she lives + +A component of `desktop/`, a sibling of `island/` and `assistant/`, not a +drawer module: she has no tile and no page. `desktop/` already hosts the +assistant, so she reads its state directly. The cost, accepted: restarting +`desktop/` restarts her. + +``` +desktop/companion/ + Companion.qml the window, inputs, timers, IPC + Sprite.qml draws a frame; crossfade, bob, squash, explosion + Mood.js which state wins, as a pure function (.pragma library) + test_mood.js node check of Mood.js, like assistant/test_md.js + lines.js her English lines, grouped by reaction + prep-frames bash + ImageMagick: generated images to frames/ + README.md the component's notes +``` + +`desktop/shell.qml` creates one `Companion { asst: assistant }`, the way +`Island` gets `asst: assistant`. + +### The window + +A `PanelWindow` anchored bottom and right, on the focused monitor +(`Hyprland.focusedMonitor`), layer `Top`, `ExclusionMode.Ignore`, +`WlrKeyboardFocus.None`, namespace `keystone-companion`. Its `mask` is her +body only, so clicks anywhere else reach the window below; the bubble is +not in the mask either. + +It is the second window that exists for the whole session, alongside the +keepalive. Hidden, it keeps existing with an empty mask and nothing drawn, +so a toggle never creates or destroys a surface. + +Moving between monitors plays her exit on the old one and her entrance +on the new: the window's screen changes only once the exit has finished. + +### Art + +Never in the repo: 2B is Square Enix's character and the repo is public +under GPLv2. Generated images and their recipe live in +`~/.local/share/keystone/companion/2b/`; `prep-frames` writes the +prepared PNGs, 256 px tall with a transparent background, to `frames/` +inside it. + +Two outfits, the same frames in each: `frames/dress/<name>.png` and +`frames/leotard/<name>.png`. The component loads the current outfit's +`<name>` and falls back to that outfit's `base`, never to the other +outfit, so a missing frame cannot put the skirt back mid-scene. It works +from the first commit with only `dress/base`; until `leotard/base` +exists, the self-destruct ends in the dress. + +**She stands.** `base` is standing, facing the viewer, arms relaxed: +lying down is what boredom ends in, so sitting as the rest pose made it +no change at all. The first base was sitting; it stays in the art +directory as `base-sitting.png`, the source the standing one is edited +from. The `smile` and `wave` made from it are redone from the standing +base, so every frame descends from one reference. + +| Method | Frames, per outfit | +| --- | --- | +| klein edit of `base-sitting.png` | `base`: "make her stand upright, facing the viewer, arms relaxed at her sides" | +| img2img from `base`, strength 0.75 | `smile`, `mad`, `smirk`, `blush1`, `blush2`, `blush3`, `sleepy` | +| klein edit of `base` | `wave`, `watch`, `point`, `yawn`, `lying`, `listen`, `think`, `muse`, `stretch`, `shoved` | +| klein edit of the dress frame | every `leotard/<name>`: "her skirt is gone, revealing a plain white leotard; change nothing else" | +| klein edit of `leotard/base` | `scorched`, leotard only: the moment after the explosion | + +Each leotard frame is made from its own dress frame rather than from +`leotard/base`, so the pose comes along unchanged. 18 frames a set, 37 in +all with `scorched`. Every prompt is in `desktop/companion/frames.txt`, +and `gen-frames` makes them through the running sd-server. + +Dodging, shoving, the drop-in and the explosion are QML motion, not +frames. She is drawn 128 px tall; both monitors are at scale 1. No size +preference: one would need a control in the settings window, and nothing +asks for it yet. + +## Behaviour + +One state at a time, the highest that applies. `Mood.js` takes the inputs +and the time and returns the state; `Companion.qml` owns the inputs and +timers. + +| Priority | State | Trigger | Frame | Ends | +| --- | --- | --- | --- | --- | +| 1 | Self-destruct | 7th click of a burst | glow, flash, explosion, then `leotard/scorched` | see below | +| 2 | Dodge | pointer rests on her 0.6 s with no click | slides out of the corner | returns after 5 s | +| 3 | Shoved | a balloon is on screen on her monitor | `shoved`, slid left by the stack's width | balloons gone | +| 4 | Assistant | recording / working / reply arrived | `listen` / `think` / `wave` | state ends; wave lasts 3 s | +| 5 | Clicks | clicks 1-6 of a burst; a burst ends after 1.5 s without one | `smile`, `blush1` to `blush3` | 2 s after the last click | +| 6 | Mock | BackSpace, at most once in 20 s | `smirk`, `point` | 2.5 s | +| 7 | Watching | typing pulse in the last 2 s | `watch` | 2 s after typing stops | +| 8 | Idle | no input for 1 / 3 / 5 min | `sleepy` / `yawn` / `lying` | any input: `stretch`, then base | +| 9 | Base | nothing else | `base` | | + +- **Self-destruct, as in the game.** She glows, says "Self-destruct + sequence initiated.", a white flash covers the sprite for about a + second with particles and a shockwave ring, and she is back in place + without her skirt (`leotard/scorched` for 3 s), saying "...That was + unpleasant." The outfit is then `leotard` for 60 s, or until the next + hide and show, whichever is first, and every state plays as usual from + the leotard set. The outfit is the only thing the self-destruct + changes: `Mood.js` picks the state, the outfit picks the folder. +- **Dodge.** She cannot feel the pointer until it is over her: Quickshell + has no global pointer and Hyprland sends no motion events. A click + before the 0.6 s still reaches her, so clicking her keeps working. If + the pointer is still there when she returns, she dodges again. +- **Shoved** only on DP-1, where the balloons are. A balloon counts when + the balloon shell would show it: not withheld by DND, snooze or the + open drawer, not expired. That check moves into `shared/Notify.qml` + as one function both shells call, so they cannot disagree. +- **Assistant.** Recording is `asst.recording`; working is `asst.busy` + (`thinking`, `tool`, `transcribing`); "reply arrived" is the reply + ending, the same moment the canvas's done notification fires. +- **Idle** is Quickshell's `IdleMonitor`, respecting inhibitors, so + presentation mode keeps her awake. +- **Hidden** while the focused window is fullscreen, while the screen is + locked, and while toggled off. + +### Speech + +Fixed English lines from `lines.js`, picked at random per reaction, for +example "...Operator, I'm bored.", "Typo again?", "Hey.", "Answer's +ready.", "Glory to mankind.". A bubble shows 3 s. At most one line per +30 s, except clicks and the self-destruct. + +The game's opening monologue, verbatim: + +> Everything that lives is designed to end. We are perpetually trapped in +> a never-ending spiral of life and death. Is this a curse? Or some kind +> of punishment? I often think about the god who blessed us with this +> cryptic puzzle...and wonder if we'll ever get the chance to kill him. + +It plays with a 1-in-4 chance on entering bored and 1-in-3 on being shown +with SUPER+\, never more than once an hour. It types out at about 15 +characters a second, one sentence per bubble, the last held 4 s, over the +`muse` frame. Any state above idle cuts it short. + +## Inputs from Hyprland + +In the live config, `~/.config/hypr/sections/`, not this repo: + +- `hl.bind("SUPER + code:49", ...)` runs + `qs -p <desktop> ipc call companion toggle`. +- A hook on `input.keyboard.key` dispatches a custom `keystone-typing` + event at most once a second. It reads no key data. +- A `non_consuming` bind on BackSpace dispatches `keystone-delete`, so + the key still reaches the focused window. BackSpace is the only key + the companion learns about. + +`Companion.qml` reads both through `Hyprland.rawEvent`, the stream the +island already uses. + +## Testing + +- `node desktop/companion/test_mood.js`: the priority table, the click + burst (1 smiles, 2-6 blush harder, 7 self-destructs, a 1.5 s gap + resets), the 20 s mock cooldown, the 30 s line cooldown, the hourly + monologue cap, the idle thresholds. +- `prep-frames` checked on `base.png`: alpha present, 256 px tall, the + hair not eaten (corner pixel transparent, a hair pixel opaque). +- By hand, since it is visual: each reaction once, monitor hand-over, + fullscreen and lock hiding, the dodge, and that BackSpace still deletes + in a terminal and a browser with the bind active. + +## Open checks + +Each is tested before the code that depends on it: + +1. A `non_consuming` BackSpace bind still deletes in the focused window. +2. The `input.keyboard.key` hook fires and its dispatch reaches + `Hyprland.rawEvent`. +3. A `Top` layer surface is covered by a fullscreen window, so the + fullscreen rule may be the compositor's and not code. If not, hide on + the focused toplevel's fullscreen flag. + +## Out of scope + +- Generated lines (Gemma): slow, and it would hold VRAM against sd-webui. +- A size preference or any other setting. +- Following the focused window's edge, Modesty-style. +- Italian lines. |