docs: OBS pill enhancements design

1 files changed, 279 insertions, 0 deletions

diff --git a/docs/superpowers/specs/2026-10-17-obs-pill-enhancements-design.md b/docs/superpowers/specs/2026-10-17-obs-pill-enhancements-design.md
new file mode 100644
index 0000000..35d22a9
--- /dev/null
+++ b/docs/superpowers/specs/2026-10-17-obs-pill-enhancements-design.md
@@ -0,0 +1,279 @@
+# OBS pill: desktop audio mute, webcam corners, scene grid
+
+Extends the OBS pill from `2026-10-17-obs-pill-design.md`. Two new controls, a
+refreshed panel, and a settings page for the scene icon map. This is a design
+document; the plan follows.
+
+## Intent
+
+While OBS records, control two more things from the island pill: the desktop
+audio (mute/unmute) and the webcam's on-canvas position (snap to a corner). At
+the same time, make the panel's buttons icon-first so the panel is faster to
+scan mid-recording.
+
+## Goals
+
+- Toggle desktop audio mute from the panel; state follows OBS.
+- Snap the webcam to any of the 4 corners of the recorded canvas.
+- Panel buttons become icon + short label.
+- Scenes become an icon grid whose glyphs are mapped by name in a watched
+  `~/.config` file, editable from the settings window, with a generic glyph for
+  unmapped scenes.
+
+## Non-goals / anti-goals
+
+- Mic mute: the user has a physical button (unchanged from the original spec).
+- Camera hardware settings, replay buffer, streaming, virtual camera.
+- Scene creation or reordering from the pill.
+- Persistent "current corner" state: corner placement is fire-and-forget.
+- Echo cancellation: see "Mic bleed" below.
+- A general source-transform editor: only the webcam is moved, only to corners.
+- A glyph palette or font browser inside the settings page: glyphs are pasted
+  from the Nerd Fonts cheat sheet.
+
+## Mic bleed (setup, not code)
+
+With desktop audio captured and the mic open, the mic also hears the speakers,
+so the recording carries the app's audio twice. This is acoustic, not a routing
+bug, and OBS cannot subtract the speaker signal from the mic. The chosen fix is
+**headphones** while recording with the mic: no bleed, no pipeline change. The
+pill does not touch the mic, and no code is added for this.
+
+## Verified current state (2026-10-17, read from OBS's own files and live probes)
+
+- Desktop audio is a **global** device, not a scene source: a
+  `pulse_output_capture` of the monitor output, named `Desktop Audio`, kind
+  `pulse_output_capture`, currently unmuted. `GetSpecialInputs.desktop1` is
+  `Desktop Audio`, and `GetInputMute` resolves by that name.
+- Base canvas is 1920x1080.
+- Scenes present: `Desktop`, `Webcam Only`, `Window Only`, and `Intervallo`
+  (Image Slide Show, the pause scene). `Intervallo` is the current program
+  scene, so the panel opens without webcam controls until the user switches to
+  `Desktop` (cosmetic; not a bug).
+- Webcam sources: `camera` (Desktop) and `Camera` (Webcam Only), both
+  `v4l2_input`. The existing per-scene webcam toggle already finds the first
+  `v4l2_input`, so no name-based matching is needed.
+- The live webcam item in `Desktop` is scaled 0.4656: source 640x360 renders as
+  298x168 at position (1572, 861), `alignment: 5` (top-left).
+
+## `Obs.qml` additions
+
+New state:
+
+- `desktopAudio`: string, the desktop-audio input name, `""` when none.
+- `desktopMuted`: bool.
+- `baseWidth`, `baseHeight`: real, the base canvas size.
+- `camW`, `camH`: real, the webcam item's **rendered** size in the current
+  scene.
+
+On `Identified` (op 2), in addition to the existing `GetRecordStatus` and
+`GetSceneList`:
+
+- `GetSpecialInputs` -> `desktopAudio = desktop1` (the first non-empty desktop
+  input name).
+- `GetVideoSettings` -> `baseWidth` / `baseHeight`.
+- If `desktopAudio` is non-empty, `GetInputMute { inputName: desktopAudio }` ->
+  `desktopMuted`.
+
+`identify()` must also subscribe to the Inputs event category:
+`eventSubscriptions` becomes `4 | 8 | 64 | 128` = **204** (Scenes | Inputs |
+Outputs | SceneItems). Without Inputs, `InputMuteStateChanged` never arrives and
+`desktopMuted` would only ever be set by the initial `GetInputMute`.
+
+New requests:
+
+- `toggleAudio()`: if `desktopAudio`, `SetInputMute { inputName: desktopAudio,
+  inputMuted: !desktopMuted }`. State is not set optimistically; it follows the
+  `InputMuteStateChanged` event.
+- `moveCam(corner)`: if `camId >= 0` and `baseWidth > 0`, compute the position
+  and send `SetSceneItemTransform { sceneName: scene, sceneItemId: camId,
+  sceneItemTransform: { positionX, positionY } }`. Send **position only**, never
+  the full echoed transform: for an item with `boundsType:
+  OBS_BOUNDS_NONE` (the live webcam), the echoed `boundsWidth: 0` fails the
+  server's minimum of 1.
+
+  Corners use a margin `m` (~16px) and the rendered size `camW`/`camH`:
+  - `tl` `(m, m)`
+  - `tr` `(baseWidth - camW - m, m)`
+  - `bl` `(m, baseHeight - camH - m)`
+  - `br` `(baseWidth - camW - m, baseHeight - camH - m)`
+
+  The math assumes `alignment: 5` (top-left), which holds on the live webcam
+  item; state that assumption in a code comment rather than relying on it
+  silently.
+
+New event:
+
+- `InputMuteStateChanged`: if `d.inputName === desktopAudio`, set
+  `desktopMuted = d.inputMuted`.
+
+In the `GetSceneItemList` response (already handled), set `camW`/`camH` from the
+webcam item's `sceneItemTransform.width` / `.height` (rendered, 298x168 on the
+live item). Never use `sourceWidth`/`sourceHeight` (640x360); the item is scaled
+and the corner math would land about 340px off-canvas.
+
+## `ObsPill.qml` changes
+
+Buttons switch from text-only to icon + short label:
+
+- Record / Stop: record-circle / stop-circle glyph + "Record"/"Stop".
+- Pause / Resume (visible only while recording): pause / play glyph +
+  "Pause"/"Resume".
+- Webcam (visible when `camId >= 0`): video / video-off glyph + "Webcam".
+- Audio (visible when `desktopAudio !== ""`): volume-high / volume-off glyph +
+  "Audio".
+- Corner move: four icon-only corner-arrow buttons, visible when `camId >= 0`,
+  calling `Obs.moveCam("tl" | "tr" | "bl" | "br")`. Icon-only because the arrow
+  glyph is the label.
+
+Unchanged:
+
+- Status line (`Not recording`, `Recording 03:21`, `Paused 03:21`).
+- At-rest pill dot and elapsed time.
+
+### Scene grid
+
+Scenes are square icon tiles in a `Grid` with `columns: 4`, reserved at the
+bottom of the panel and always visible (not collapsible). Up to 8 tiles fit in
+two rows; more scroll inside a fixed-height `Flickable`. Selection is click-only,
+so scrolling never switches a scene. The current scene is highlighted as now.
+
+Each scene's glyph comes from a config file,
+`$XDG_CONFIG_HOME/quickshell/obs-scene-icons.json` (default
+`~/.config/quickshell/obs-scene-icons.json`), watched and reloaded on change:
+
+```json
+{
+  "generic": "\udb80\uded7",
+  "scenes": {
+    "Desktop": "\udb80\udf79",
+    "Webcam Only": "\udb81\udd67",
+    "Window Only": "\uf2d0",
+    "Intervallo": "\udb80\udd76"
+  }
+}
+```
+
+(The `md-*` glyphs are supplementary-plane, so their JSON/JS form is a surrogate
+pair; `JSON.stringify` also writes the glyph itself, which is equivalent.)
+
+That starter map is also the built-in default in
+`desktop/island/SceneIcons.js` (`pragma library`), used when the file is missing
+or unreadable, so the pill works before the settings page is ever opened.
+`SceneIcons.js` holds the pure `defaults`, `parse(text)`, `serialize(map)`,
+`iconFor(map, name)` and the one-code-point check; a small `SceneIcons.qml`
+singleton owns the watched `FileView`, exposes `iconFor(name)` and `save(map)`.
+Unmapped scenes, including any added in OBS later, show `generic`. A missing or
+unparseable file yields the starter map; a valid file is used as-is and is not
+merged with the defaults, so deleting a key makes that scene `generic` again.
+
+The file is user-editable and the settings page writes it; there is no new state
+in `Obs.qml`.
+
+Tiles are about 36px square: 4 across is ~168px with gaps, so the current 260px
+panel width holds them without horizontal growth. The control-button row is the
+width driver; the builder may widen the panel or wrap that row.
+
+Panel height follows `controls.implicitHeight` as before. The `Flickable` has a
+fixed two-row height, so the panel does not grow as scenes are added.
+
+### Webcam Only
+
+The corner buttons also appear in `Webcam Only`, where the webcam fills the
+canvas; moving it there shifts it slightly off-centre. That is harmless and
+accepted rather than adding scene-name or display-source detection.
+
+## Settings page: OBS scenes
+
+A new module `desktop/modules/obs/ObsScenesModule.qml`, `settings: true`, group
+"Desktop", label "OBS scenes", so it joins the settings window's sidebar and
+search. It imports `../../island` for `Obs` (the live scene list) and
+`SceneIcons` (read and save). Its page lists one row per scene: the scene name
+and a paste field holding that scene's glyph, rendered with `Theme.iconFamily`.
+A final "Default for unmapped scenes" row edits `generic`.
+
+Picking a glyph is paste-only: the user finds an icon on the Nerd Fonts cheat
+sheet and pastes the character into the field (e.g. U+F02DC, `md-home`). No
+palette and no built-in font browser; the cheat sheet already is one.
+
+Rows are the union of the file's `scenes` keys and the scenes `Obs` currently
+reports, so the page works with OBS closed (configured rows) and offers newly
+created scenes once OBS is connected. A scene that exists in neither cannot be
+added from the page; add it in OBS, or add a key to the JSON by hand.
+
+Each field accepts exactly one code point, surrogate pairs included (a
+supplementary-plane glyph such as U+F02DC is two UTF-16 units). An empty field
+means the scene falls back to `generic`. The `generic` row itself must also hold
+one code point; empty falls back to the built-in `md-help_circle` default, so
+there is always a visible tile. Invalid input (zero or more than one code
+point) gets a red border and blocks Save. Edits are a draft behind the page's
+`SaveBar`, as with the other settings pages; Save serialises `{generic, scenes}`
+through `SceneIcons.save()` and writes the file atomically. A glyph missing from
+the installed font renders as tofu in the field, which is the visible sign to
+pick another.
+
+## Icons
+
+Codepoints verified 2026-10-17 against the installed Inconsolata Nerd Font
+Regular's cmap (all present). The control glyphs and the default scene glyphs
+below are fixed in code; the scene glyphs actually rendered are user-supplied
+from the config file. Re-check at build time if the font changes:
+
+| Use | Glyph | U+ | JS/JSON escape |
+| --- | --- | --- | --- |
+| Record | `md-record_circle` | U+F0EC2 | `\udb83\udec2` |
+| Stop | `md-stop_circle` | U+F0666 | `\udb81\ude66` |
+| Pause | `md-pause` | U+F03E4 | `\udb80\udfe4` |
+| Resume | `md-play` | U+F040A | `\udb81\udc0a` |
+| Webcam on | `md-video` | U+F0567 | `\udb81\udd67` |
+| Webcam off | `md-video_off` | U+F0568 | `\udb81\udd68` |
+| Audio on | `md-volume_high` | U+F057E | `\udb81\udd7e` |
+| Audio off | `md-volume_off` | U+F0581 | `\udb81\udd81` |
+| Corner TL | `md-arrow_top_left` | U+F005B | `\udb80\udc5b` |
+| Corner TR | `md-arrow_top_right` | U+F005C | `\udb80\udc5c` |
+| Corner BL | `md-arrow_bottom_left` | U+F0042 | `\udb80\udc42` |
+| Corner BR | `md-arrow_bottom_right` | U+F0043 | `\udb80\udc43` |
+| Scene: Desktop | `md-monitor` | U+F0379 | `\udb80\udf79` |
+| Scene: Webcam Only | `md-video` | U+F0567 | `\udb81\udd67` |
+| Scene: Window Only | `fa-window_maximize` | U+F2D0 | `\uf2d0` |
+| Scene: Intervallo | `md-coffee` | U+F0176 | `\udb80\udd76` |
+| Scene: unmapped | `md-help_circle` | U+F02D7 | `\udb80\uded7` |
+
+The `md-*` codepoints are supplementary-plane: their `\u` form is a surrogate
+pair, never a five-digit escape. `\uf0ec2` would parse as `\uf0ec` + `"2"`, a
+different glyph plus a stray digit.
+
+All render with `Theme.iconFamily`.
+
+## Errors and edge cases
+
+- `desktop1` empty (no desktop audio configured): hide the Audio button.
+- Audio toggle semantics: muting `Desktop Audio` removes system audio from the
+  recording; it does not affect the user's live speakers.
+- `camId === -1` (no webcam in the current scene): hide Webcam and corner
+  buttons (existing gate).
+- A failed request is already logged by `response()`; state follows events, so a
+  refused mute or transform does not desynchronise the pill.
+
+## Verification (no state-changing probes)
+
+Do not script state changes against the live OBS during development: a corner
+move is written into the saved scene collection, and a mute during a take is not
+restorable. Verify by hand instead: move the webcam to each corner and confirm
+the framing, mute and unmute and confirm the pill follows (including a mute
+triggered from OBS's own mixer), click scene tiles to switch, and with more than
+eight scenes confirm scrolling the grid never changes the scene.
+
+For the icon map: paste a glyph into a settings row, Save, and confirm the pill's
+tile changes without a restart; edit the JSON by hand and confirm the pill
+follows; remove the file and confirm the starter defaults return; create a scene
+in OBS and confirm its tile is `generic` until mapped.
+
+## Open items to verify at build time
+
+1. `SetSceneItemTransform` with position only (settled by live probe; confirm
+   once by hand).
+2. `GetSpecialInputs.desktop1` round-trips through `SetInputMute` (settled;
+   confirm with the hand check above).
+3. `SceneIcons.js` `parse`/`serialize`/one-code-point check covered in node
+   before the QML that uses them is written.