diff options
| -rw-r--r-- | desktop/companion/frames.txt | 47 | ||||
| -rwxr-xr-x | desktop/companion/gen-frames | 26 | ||||
| -rw-r--r-- | docs/superpowers/plans/2026-10-23-companion.md | 5 | ||||
| -rw-r--r-- | docs/superpowers/specs/2026-10-23-companion-design.md | 9 |
4 files changed, 61 insertions, 26 deletions
diff --git a/desktop/companion/frames.txt b/desktop/companion/frames.txt index e97391c..d85dde6 100644 --- a/desktop/companion/frames.txt +++ b/desktop/companion/frames.txt @@ -25,27 +25,48 @@ KEEP = She keeps her black gloves on both hands, her puffy black juliet sleeves, her black dress with its skirt, her black thighhigh boots, her black blindfold and hairband, her lips and the small mole under her mouth. Same character, chibi proportions, art style and white background. POSE = keep her pose exactly, standing, arms relaxed at her sides. -LEOTARD = Her skirt is gone, revealing a plain white leotard underneath; everything else stays exactly the same: pose, expression, black gloves, puffy black juliet sleeves, black bodice, black thighhigh boots, blindfold, hairband, lips, the small mole under her mouth, chibi proportions, art style and white background. +# Strong expressions made klein turn the blindfold into an eye mask with the +# eyes showing through (mad, smirk, blush3, sleepy). The ones that read +# through her eyes take it off instead (BARE); the rest name it (EYES). +BARE = She has taken her blindfold off: both eyes are uncovered and show clearly, pale blue-grey, and carry the expression; no blindfold, no mask, nothing over her eyes; her black hairband stays. +KEEP_BARE = She keeps her black gloves on both hands, her puffy black juliet sleeves, her black dress with its skirt, her black thighhigh boots, her black hairband, her lips and the small mole under her mouth. Same character, chibi proportions, art style and white background. +EYES = Her black blindfold stays on as one solid band of cloth fully covering both eyes: no eyes and no eyeholes show; the expression is in her mouth, cheeks and brows above the band. +# After the self-destruct the blindfold is gone with the skirt: the whole +# leotard set shows her eyes. +LEOTARD = Her skirt is gone, revealing a plain white leotard underneath, and her blindfold is gone too: both eyes uncovered and clearly visible, pale blue-grey, matching her expression. Everything else stays exactly the same: pose, expression, black gloves, puffy black juliet sleeves, black bodice, black thighhigh boots, black hairband, lips, the small mole under her mouth, chibi proportions, art style and white background. # --- expressions -dress/smile <- dress/base : Change only her expression to a small warm closed-mouth smile; ${POSE} ${KEEP} -dress/mad <- dress/base : Change only her expression to annoyed and angry, a frown and a puffed cheek; ${POSE} ${KEEP} -dress/smirk <- dress/base : Change only her expression to a teasing one-sided smirk; ${POSE} ${KEEP} -dress/blush1 <- dress/base : Change only her expression: a light pink blush on her cheeks and a shy small smile; ${POSE} ${KEEP} -dress/blush2 <- dress/base : Change only her expression: a strong red blush across her cheeks and a flustered wavy mouth; ${POSE} ${KEEP} -dress/blush3 <- dress/base : Change only her expression: her whole face bright red, small steam puffs above her head, very flustered; ${POSE} ${KEEP} -dress/sleepy <- dress/base : Change only her expression to sleepy and drowsy, head tilted slightly, mouth relaxed; ${POSE} ${KEEP} +dress/smile <- dress/base : Change only her expression to a small warm closed-mouth smile; ${POSE} ${EYES} ${KEEP} +dress/mad <- dress/base : Change only her expression to annoyed and angry, a frown and a puffed cheek; ${POSE} ${BARE} ${KEEP_BARE} +dress/smirk <- dress/base : Change only her expression to a teasing one-sided smirk; ${POSE} Both eyes fully visible and uncovered; the blindfold is nowhere on her head or face, not pushed up, not over one eye. ${BARE} ${KEEP_BARE} +dress/blush1 <- dress/base : Change only her expression: a light pink blush on her cheeks and a shy small smile; ${POSE} ${EYES} ${KEEP} +dress/blush2 <- dress/base : Change only her expression: a strong red blush across her cheeks and a flustered wavy mouth; ${POSE} ${EYES} ${KEEP} +dress/blush3 <- dress/base : Change only her expression: her whole face bright red, small steam puffs above her head, very flustered; ${POSE} ${BARE} ${KEEP_BARE} +dress/sleepy <- dress/base : Change only her expression to sleepy and drowsy, head tilted slightly, mouth relaxed; ${POSE} ${BARE} ${KEEP_BARE} # --- poses dress/wave <- dress/base : Raise only the hand on the left side of the image beside her head, waving, with a small smile; her other arm stays at her side. ${KEEP} -dress/watch <- dress/base : Turn her head and upper body toward the left side of the image, as if watching a screen beside her. ${KEEP} +# Shown while typing: she faces into the screen, away from the viewer, as if +# reading over the user's shoulder. A profile read as posing, not watching. +dress/watch <- dress/base : Turn her completely around into a back view: we see her from behind, the back of her head with her white bob, crossed by her black hairband and the small black cloth knot of her black blindfold (black, not white, no bow), her back, the back of her dress and the backs of her legs, as if she is facing a screen and watching it; arms relaxed at her sides. Same character, black gloves, puffy black juliet sleeves, black dress with its skirt, black thighhigh boots, black hairband, chibi proportions, art style and white background. dress/point <- dress/base : She points with the hand on the left side of the image toward the viewer, with a teasing smirk; her other hand rests on her hip. ${KEEP} dress/yawn <- dress/base : A big yawn, mouth wide open, one gloved hand covering her mouth. ${KEEP} dress/lying <- dress/base : Make her lie on her side on the ground, asleep, her head resting on her folded arms. ${KEEP} dress/listen <- dress/base : She cups the hand on the left side of the image behind her ear, listening attentively. ${KEEP} dress/think <- dress/base : One gloved hand at her chin, thinking, head tilted slightly. ${KEEP} -dress/muse <- dress/base : One hand on her chest, head turned slightly away and down, contemplative and solemn. ${KEEP} +# The monologue's frame: the pose of her official character art, chibi. +# refs/2b-official-white is that art flattened on white, used only for the +# pose. A Virtuous Treaty beside her was tried and dropped: klein draws a +# generic longsword, nothing like the game's, and would not move it. +dress/muse <- dress/base + refs/2b-official-white : Redraw the chibi character from the first image, keeping her exact chibi proportions, face, outfit and art style, in the standing pose of the woman in the second image: body turned three-quarters, one arm hanging out low to the side with the hand open, the other hand raised near her shoulder, weight on one leg, head slightly lowered, calm and solemn. No weapon, nothing in her hands. ${KEEP} +# She stays frontal: a three-quarter turn, from the reference or as its own +# edit, only bent her knees. dress/stretch <- dress/base : She stretches both arms up above her head, waking up. ${KEEP} +# A balloon shoved her out of the corner. Standing versions read as walking +# in four seeds, so this one is airborne, edited from the attempt whose hands +# and face were right: candidates/shoved-7.png, made from dress/base with +# --seed 7 by the earlier standing prompt (see generated.log). Keep that file. +dress/shoved <- candidates/shoved-7 : Keep her face, her scared expression, her uncovered eyes and both raised hands exactly as they are. Change her body and legs: she is in the air, lifted completely off the ground, knocked flying toward the left side of the image by a strong blast from the right; both feet off the ground, legs trailing behind her toward the right side, her body tilted, her hair and skirt blown toward the left. No ground, no blast effect drawn. ${KEEP_BARE} # --- the leotard set: the same frames, after the self-destruct leotard/base <- dress/base : ${LEOTARD} @@ -57,7 +78,8 @@ leotard/blush2 <- dress/blush2 : ${LEOTARD} leotard/blush3 <- dress/blush3 : ${LEOTARD} leotard/sleepy <- dress/sleepy : ${LEOTARD} leotard/wave <- dress/wave : ${LEOTARD} -leotard/watch <- dress/watch : ${LEOTARD} +# A back view has no eyes to uncover; LEOTARD asking for them would turn her round. +leotard/watch <- dress/watch : Her skirt is gone, revealing a plain white leotard underneath, and the blindfold knot is gone from the back of her head. She stays in exactly the same back view, seen from behind, facing away. Everything else the same: black gloves, puffy black juliet sleeves, black bodice, black thighhigh boots, black hairband, chibi proportions, art style and white background. leotard/point <- dress/point : ${LEOTARD} leotard/yawn <- dress/yawn : ${LEOTARD} leotard/lying <- dress/lying : ${LEOTARD} @@ -65,4 +87,5 @@ leotard/listen <- dress/listen : ${LEOTARD} leotard/think <- dress/think : ${LEOTARD} leotard/muse <- dress/muse : ${LEOTARD} leotard/stretch <- dress/stretch : ${LEOTARD} -leotard/scorched <- leotard/base : Right after an explosion: her clothes lightly scorched and smudged with soot, a few wisps of smoke rising from her, hair slightly messy, a dazed flat expression; standing, arms at her sides. She keeps her plain white leotard, black gloves, puffy black juliet sleeves, black thighhigh boots, blindfold, hairband, lips and the small mole under her mouth. Same character, chibi proportions, art style and white background. +leotard/shoved <- dress/shoved : ${LEOTARD} +leotard/scorched <- leotard/base : Right after an explosion: her clothes lightly scorched and smudged with soot, a few wisps of smoke rising from her, hair slightly messy, a dazed flat expression; standing, arms at her sides. Her eyes stay uncovered, no blindfold. She keeps her plain white leotard, black gloves, puffy black juliet sleeves, black thighhigh boots, black hairband, lips and the small mole under her mouth. Same character, chibi proportions, art style and white background. diff --git a/desktop/companion/gen-frames b/desktop/companion/gen-frames index 75e6655..df78cb9 100755 --- a/desktop/companion/gen-frames +++ b/desktop/companion/gen-frames @@ -44,10 +44,12 @@ while (($#)); do esac done -# frames.txt to "output<TAB>reference<TAB>prompt" lines, variables expanded. +# frames.txt to "output<TAB>references<TAB>prompt" lines, variables expanded. +# References are joined with + and no spaces: the first is the image edited, +# any after it are extra images the prompt can point at ("the second image"). parse() { local line name out ref prompt - local frame='^[[:space:]]*([a-z0-9/_-]+)[[:space:]]*<-[[:space:]]*([a-z0-9/_-]+)[[:space:]]*:[[:space:]]*(.*)$' + local frame='^[[:space:]]*([a-z0-9/_-]+)[[:space:]]*<-[[:space:]]*([a-z0-9/_-]+([[:space:]]*\+[[:space:]]*[a-z0-9/_-]+)*)[[:space:]]*:[[:space:]]*(.*)$' declare -A vars=() while IFS= read -r line || [[ -n $line ]]; do [[ $line =~ ^[[:space:]]*(#|$) ]] && continue @@ -59,7 +61,7 @@ parse() { echo "gen-frames: frames.txt: cannot read: $line" >&2 exit 1 fi - out=${BASH_REMATCH[1]} ref=${BASH_REMATCH[2]} prompt=${BASH_REMATCH[3]} + out=${BASH_REMATCH[1]} ref=${BASH_REMATCH[2]// /} prompt=${BASH_REMATCH[4]} for name in "${!vars[@]}"; do prompt=${prompt//\$\{$name\}/${vars[$name]}}; done if [[ $prompt == *'${'* ]]; then echo "gen-frames: frames.txt: $out: undefined variable in: $prompt" >&2 @@ -106,16 +108,22 @@ trap 'rm -rf "$tmp"' EXIT made=0 for row in "${todo[@]}"; do IFS=$'\t' read -r out ref prompt <<<"$row" - if [[ ! -e $art/$ref.png ]]; then - echo "gen-frames: $out: its reference $ref.png does not exist yet, skipped" >&2 + IFS=+ read -ra refs <<<"$ref" + missing="" + for r in "${refs[@]}"; do [[ -e $art/$r.png ]] || missing=$r; done + if [[ -n $missing ]]; then + echo "gen-frames: $out: its reference $missing.png does not exist yet, skipped" >&2 continue fi - echo "gen-frames: $out <- $ref" - { printf 'data:image/png;base64,'; base64 -w0 "$art/$ref.png"; } >"$tmp/ref" + echo "gen-frames: $out <- ${ref//+/ + }" + # One JSON string per reference, as data URLs, slurped into the array. + for r in "${refs[@]}"; do + { printf 'data:image/png;base64,'; base64 -w0 "$art/$r.png"; } | jq -Rs . + done >"$tmp/refs.json" # The settings the frames were tuned with in sd-webui: recipe.txt. - jq -n --arg prompt "$prompt" --rawfile ref "$tmp/ref" --argjson seed "$seed" '{ + jq -n --arg prompt "$prompt" --slurpfile refs "$tmp/refs.json" --argjson seed "$seed" '{ prompt: $prompt, negative_prompt: "", width: 1024, height: 1024, seed: $seed, - batch_count: 1, ref_images: [$ref], auto_resize_ref_image: true, + batch_count: 1, ref_images: $refs, auto_resize_ref_image: true, increase_ref_index: false, lora: [], output_format: "png", output_compression: 100, sample_params: { sample_steps: 4, sample_method: "euler", scheduler: "default", guidance: { txt_cfg: 1.0 } } diff --git a/docs/superpowers/plans/2026-10-23-companion.md b/docs/superpowers/plans/2026-10-23-companion.md index 371e9fa..3543b88 100644 --- a/docs/superpowers/plans/2026-10-23-companion.md +++ b/docs/superpowers/plans/2026-10-23-companion.md @@ -281,6 +281,9 @@ for (let i = 0; i < ladder.length; i++) { assert.strictEqual(state(all), ladder[i][0], "everything from " + ladder[i][0] + " down"); } +// Shoved has a frame of its own: knocked off balance, glaring at the balloon. +assert.deepStrictEqual(M.pick(calm({ shoved: true })), { state: "shoved", frame: "shoved" }); + // Idle levels and their frames. assert.deepStrictEqual(M.pick(calm({ idle: 1 })), { state: "bored", frame: "sleepy" }); assert.deepStrictEqual(M.pick(calm({ idle: 2 })), { state: "yawn", frame: "yawn" }); @@ -396,7 +399,7 @@ function pick(s) { if (since < T.destruct) return st("destruct", "base"); if (since < T.destruct + T.scorched) return st("scorched", "scorched"); if (s.dodging) return st("dodge", "base"); - if (s.shoved) return st("shoved", "mad"); + if (s.shoved) return st("shoved", "shoved"); if (s.recording) return st("listen", "listen"); if (s.busy) return st("think", "think"); if (s.now - s.repliedAt < T.waveHold) return st("wave", "wave"); diff --git a/docs/superpowers/specs/2026-10-23-companion-design.md b/docs/superpowers/specs/2026-10-23-companion-design.md index 8af7b69..8acffbf 100644 --- a/docs/superpowers/specs/2026-10-23-companion-design.md +++ b/docs/superpowers/specs/2026-10-23-companion-design.md @@ -111,13 +111,14 @@ base, so every frame descends from one reference. | --- | --- | | klein edit of `base-sitting.png` | `base`: "make her stand upright, facing the viewer, arms relaxed at her sides" | | img2img from `base`, strength 0.75 | `smile`, `mad`, `smirk`, `blush1`, `blush2`, `blush3`, `sleepy` | -| klein edit of `base` | `wave`, `watch`, `point`, `yawn`, `lying`, `listen`, `think`, `muse`, `stretch` | +| klein edit of `base` | `wave`, `watch`, `point`, `yawn`, `lying`, `listen`, `think`, `muse`, `stretch`, `shoved` | | klein edit of the dress frame | every `leotard/<name>`: "her skirt is gone, revealing a plain white leotard; change nothing else" | | klein edit of `leotard/base` | `scorched`, leotard only: the moment after the explosion | Each leotard frame is made from its own dress frame rather than from -`leotard/base`, so the pose comes along unchanged. 17 frames a set, 35 in -all with `scorched`. +`leotard/base`, so the pose comes along unchanged. 18 frames a set, 37 in +all with `scorched`. Every prompt is in `desktop/companion/frames.txt`, +and `gen-frames` makes them through the running sd-server. Dodging, shoving, the drop-in and the explosion are QML motion, not frames. She is drawn 128 px tall; both monitors are at scale 1. No size @@ -134,7 +135,7 @@ timers. | --- | --- | --- | --- | --- | | 1 | Self-destruct | 7th click of a burst | glow, flash, explosion, then `leotard/scorched` | see below | | 2 | Dodge | pointer rests on her 0.6 s with no click | slides out of the corner | returns after 5 s | -| 3 | Shoved | a balloon is on screen on her monitor | `mad`, slid left by the stack's width | balloons gone | +| 3 | Shoved | a balloon is on screen on her monitor | `shoved`, slid left by the stack's width | balloons gone | | 4 | Assistant | recording / working / reply arrived | `listen` / `think` / `wave` | state ends; wave lasts 3 s | | 5 | Clicks | clicks 1-6 of a burst; a burst ends after 1.5 s without one | `smile`, `blush1` to `blush3` | 2 s after the last click | | 6 | Mock | BackSpace, at most once in 20 s | `smirk`, `point` | 2.5 s | |