diff options
| -rw-r--r-- | docs/superpowers/specs/2026-10-17-obs-pill-enhancements-design.md | 279 |
1 files changed, 279 insertions, 0 deletions
diff --git a/docs/superpowers/specs/2026-10-17-obs-pill-enhancements-design.md b/docs/superpowers/specs/2026-10-17-obs-pill-enhancements-design.md new file mode 100644 index 0000000..35d22a9 --- /dev/null +++ b/docs/superpowers/specs/2026-10-17-obs-pill-enhancements-design.md @@ -0,0 +1,279 @@ +# OBS pill: desktop audio mute, webcam corners, scene grid + +Extends the OBS pill from `2026-10-17-obs-pill-design.md`. Two new controls, a +refreshed panel, and a settings page for the scene icon map. This is a design +document; the plan follows. + +## Intent + +While OBS records, control two more things from the island pill: the desktop +audio (mute/unmute) and the webcam's on-canvas position (snap to a corner). At +the same time, make the panel's buttons icon-first so the panel is faster to +scan mid-recording. + +## Goals + +- Toggle desktop audio mute from the panel; state follows OBS. +- Snap the webcam to any of the 4 corners of the recorded canvas. +- Panel buttons become icon + short label. +- Scenes become an icon grid whose glyphs are mapped by name in a watched + `~/.config` file, editable from the settings window, with a generic glyph for + unmapped scenes. + +## Non-goals / anti-goals + +- Mic mute: the user has a physical button (unchanged from the original spec). +- Camera hardware settings, replay buffer, streaming, virtual camera. +- Scene creation or reordering from the pill. +- Persistent "current corner" state: corner placement is fire-and-forget. +- Echo cancellation: see "Mic bleed" below. +- A general source-transform editor: only the webcam is moved, only to corners. +- A glyph palette or font browser inside the settings page: glyphs are pasted + from the Nerd Fonts cheat sheet. + +## Mic bleed (setup, not code) + +With desktop audio captured and the mic open, the mic also hears the speakers, +so the recording carries the app's audio twice. This is acoustic, not a routing +bug, and OBS cannot subtract the speaker signal from the mic. The chosen fix is +**headphones** while recording with the mic: no bleed, no pipeline change. The +pill does not touch the mic, and no code is added for this. + +## Verified current state (2026-10-17, read from OBS's own files and live probes) + +- Desktop audio is a **global** device, not a scene source: a + `pulse_output_capture` of the monitor output, named `Desktop Audio`, kind + `pulse_output_capture`, currently unmuted. `GetSpecialInputs.desktop1` is + `Desktop Audio`, and `GetInputMute` resolves by that name. +- Base canvas is 1920x1080. +- Scenes present: `Desktop`, `Webcam Only`, `Window Only`, and `Intervallo` + (Image Slide Show, the pause scene). `Intervallo` is the current program + scene, so the panel opens without webcam controls until the user switches to + `Desktop` (cosmetic; not a bug). +- Webcam sources: `camera` (Desktop) and `Camera` (Webcam Only), both + `v4l2_input`. The existing per-scene webcam toggle already finds the first + `v4l2_input`, so no name-based matching is needed. +- The live webcam item in `Desktop` is scaled 0.4656: source 640x360 renders as + 298x168 at position (1572, 861), `alignment: 5` (top-left). + +## `Obs.qml` additions + +New state: + +- `desktopAudio`: string, the desktop-audio input name, `""` when none. +- `desktopMuted`: bool. +- `baseWidth`, `baseHeight`: real, the base canvas size. +- `camW`, `camH`: real, the webcam item's **rendered** size in the current + scene. + +On `Identified` (op 2), in addition to the existing `GetRecordStatus` and +`GetSceneList`: + +- `GetSpecialInputs` -> `desktopAudio = desktop1` (the first non-empty desktop + input name). +- `GetVideoSettings` -> `baseWidth` / `baseHeight`. +- If `desktopAudio` is non-empty, `GetInputMute { inputName: desktopAudio }` -> + `desktopMuted`. + +`identify()` must also subscribe to the Inputs event category: +`eventSubscriptions` becomes `4 | 8 | 64 | 128` = **204** (Scenes | Inputs | +Outputs | SceneItems). Without Inputs, `InputMuteStateChanged` never arrives and +`desktopMuted` would only ever be set by the initial `GetInputMute`. + +New requests: + +- `toggleAudio()`: if `desktopAudio`, `SetInputMute { inputName: desktopAudio, + inputMuted: !desktopMuted }`. State is not set optimistically; it follows the + `InputMuteStateChanged` event. +- `moveCam(corner)`: if `camId >= 0` and `baseWidth > 0`, compute the position + and send `SetSceneItemTransform { sceneName: scene, sceneItemId: camId, + sceneItemTransform: { positionX, positionY } }`. Send **position only**, never + the full echoed transform: for an item with `boundsType: + OBS_BOUNDS_NONE` (the live webcam), the echoed `boundsWidth: 0` fails the + server's minimum of 1. + + Corners use a margin `m` (~16px) and the rendered size `camW`/`camH`: + - `tl` `(m, m)` + - `tr` `(baseWidth - camW - m, m)` + - `bl` `(m, baseHeight - camH - m)` + - `br` `(baseWidth - camW - m, baseHeight - camH - m)` + + The math assumes `alignment: 5` (top-left), which holds on the live webcam + item; state that assumption in a code comment rather than relying on it + silently. + +New event: + +- `InputMuteStateChanged`: if `d.inputName === desktopAudio`, set + `desktopMuted = d.inputMuted`. + +In the `GetSceneItemList` response (already handled), set `camW`/`camH` from the +webcam item's `sceneItemTransform.width` / `.height` (rendered, 298x168 on the +live item). Never use `sourceWidth`/`sourceHeight` (640x360); the item is scaled +and the corner math would land about 340px off-canvas. + +## `ObsPill.qml` changes + +Buttons switch from text-only to icon + short label: + +- Record / Stop: record-circle / stop-circle glyph + "Record"/"Stop". +- Pause / Resume (visible only while recording): pause / play glyph + + "Pause"/"Resume". +- Webcam (visible when `camId >= 0`): video / video-off glyph + "Webcam". +- Audio (visible when `desktopAudio !== ""`): volume-high / volume-off glyph + + "Audio". +- Corner move: four icon-only corner-arrow buttons, visible when `camId >= 0`, + calling `Obs.moveCam("tl" | "tr" | "bl" | "br")`. Icon-only because the arrow + glyph is the label. + +Unchanged: + +- Status line (`Not recording`, `Recording 03:21`, `Paused 03:21`). +- At-rest pill dot and elapsed time. + +### Scene grid + +Scenes are square icon tiles in a `Grid` with `columns: 4`, reserved at the +bottom of the panel and always visible (not collapsible). Up to 8 tiles fit in +two rows; more scroll inside a fixed-height `Flickable`. Selection is click-only, +so scrolling never switches a scene. The current scene is highlighted as now. + +Each scene's glyph comes from a config file, +`$XDG_CONFIG_HOME/quickshell/obs-scene-icons.json` (default +`~/.config/quickshell/obs-scene-icons.json`), watched and reloaded on change: + +```json +{ + "generic": "\udb80\uded7", + "scenes": { + "Desktop": "\udb80\udf79", + "Webcam Only": "\udb81\udd67", + "Window Only": "\uf2d0", + "Intervallo": "\udb80\udd76" + } +} +``` + +(The `md-*` glyphs are supplementary-plane, so their JSON/JS form is a surrogate +pair; `JSON.stringify` also writes the glyph itself, which is equivalent.) + +That starter map is also the built-in default in +`desktop/island/SceneIcons.js` (`pragma library`), used when the file is missing +or unreadable, so the pill works before the settings page is ever opened. +`SceneIcons.js` holds the pure `defaults`, `parse(text)`, `serialize(map)`, +`iconFor(map, name)` and the one-code-point check; a small `SceneIcons.qml` +singleton owns the watched `FileView`, exposes `iconFor(name)` and `save(map)`. +Unmapped scenes, including any added in OBS later, show `generic`. A missing or +unparseable file yields the starter map; a valid file is used as-is and is not +merged with the defaults, so deleting a key makes that scene `generic` again. + +The file is user-editable and the settings page writes it; there is no new state +in `Obs.qml`. + +Tiles are about 36px square: 4 across is ~168px with gaps, so the current 260px +panel width holds them without horizontal growth. The control-button row is the +width driver; the builder may widen the panel or wrap that row. + +Panel height follows `controls.implicitHeight` as before. The `Flickable` has a +fixed two-row height, so the panel does not grow as scenes are added. + +### Webcam Only + +The corner buttons also appear in `Webcam Only`, where the webcam fills the +canvas; moving it there shifts it slightly off-centre. That is harmless and +accepted rather than adding scene-name or display-source detection. + +## Settings page: OBS scenes + +A new module `desktop/modules/obs/ObsScenesModule.qml`, `settings: true`, group +"Desktop", label "OBS scenes", so it joins the settings window's sidebar and +search. It imports `../../island` for `Obs` (the live scene list) and +`SceneIcons` (read and save). Its page lists one row per scene: the scene name +and a paste field holding that scene's glyph, rendered with `Theme.iconFamily`. +A final "Default for unmapped scenes" row edits `generic`. + +Picking a glyph is paste-only: the user finds an icon on the Nerd Fonts cheat +sheet and pastes the character into the field (e.g. U+F02DC, `md-home`). No +palette and no built-in font browser; the cheat sheet already is one. + +Rows are the union of the file's `scenes` keys and the scenes `Obs` currently +reports, so the page works with OBS closed (configured rows) and offers newly +created scenes once OBS is connected. A scene that exists in neither cannot be +added from the page; add it in OBS, or add a key to the JSON by hand. + +Each field accepts exactly one code point, surrogate pairs included (a +supplementary-plane glyph such as U+F02DC is two UTF-16 units). An empty field +means the scene falls back to `generic`. The `generic` row itself must also hold +one code point; empty falls back to the built-in `md-help_circle` default, so +there is always a visible tile. Invalid input (zero or more than one code +point) gets a red border and blocks Save. Edits are a draft behind the page's +`SaveBar`, as with the other settings pages; Save serialises `{generic, scenes}` +through `SceneIcons.save()` and writes the file atomically. A glyph missing from +the installed font renders as tofu in the field, which is the visible sign to +pick another. + +## Icons + +Codepoints verified 2026-10-17 against the installed Inconsolata Nerd Font +Regular's cmap (all present). The control glyphs and the default scene glyphs +below are fixed in code; the scene glyphs actually rendered are user-supplied +from the config file. Re-check at build time if the font changes: + +| Use | Glyph | U+ | JS/JSON escape | +| --- | --- | --- | --- | +| Record | `md-record_circle` | U+F0EC2 | `\udb83\udec2` | +| Stop | `md-stop_circle` | U+F0666 | `\udb81\ude66` | +| Pause | `md-pause` | U+F03E4 | `\udb80\udfe4` | +| Resume | `md-play` | U+F040A | `\udb81\udc0a` | +| Webcam on | `md-video` | U+F0567 | `\udb81\udd67` | +| Webcam off | `md-video_off` | U+F0568 | `\udb81\udd68` | +| Audio on | `md-volume_high` | U+F057E | `\udb81\udd7e` | +| Audio off | `md-volume_off` | U+F0581 | `\udb81\udd81` | +| Corner TL | `md-arrow_top_left` | U+F005B | `\udb80\udc5b` | +| Corner TR | `md-arrow_top_right` | U+F005C | `\udb80\udc5c` | +| Corner BL | `md-arrow_bottom_left` | U+F0042 | `\udb80\udc42` | +| Corner BR | `md-arrow_bottom_right` | U+F0043 | `\udb80\udc43` | +| Scene: Desktop | `md-monitor` | U+F0379 | `\udb80\udf79` | +| Scene: Webcam Only | `md-video` | U+F0567 | `\udb81\udd67` | +| Scene: Window Only | `fa-window_maximize` | U+F2D0 | `\uf2d0` | +| Scene: Intervallo | `md-coffee` | U+F0176 | `\udb80\udd76` | +| Scene: unmapped | `md-help_circle` | U+F02D7 | `\udb80\uded7` | + +The `md-*` codepoints are supplementary-plane: their `\u` form is a surrogate +pair, never a five-digit escape. `\uf0ec2` would parse as `\uf0ec` + `"2"`, a +different glyph plus a stray digit. + +All render with `Theme.iconFamily`. + +## Errors and edge cases + +- `desktop1` empty (no desktop audio configured): hide the Audio button. +- Audio toggle semantics: muting `Desktop Audio` removes system audio from the + recording; it does not affect the user's live speakers. +- `camId === -1` (no webcam in the current scene): hide Webcam and corner + buttons (existing gate). +- A failed request is already logged by `response()`; state follows events, so a + refused mute or transform does not desynchronise the pill. + +## Verification (no state-changing probes) + +Do not script state changes against the live OBS during development: a corner +move is written into the saved scene collection, and a mute during a take is not +restorable. Verify by hand instead: move the webcam to each corner and confirm +the framing, mute and unmute and confirm the pill follows (including a mute +triggered from OBS's own mixer), click scene tiles to switch, and with more than +eight scenes confirm scrolling the grid never changes the scene. + +For the icon map: paste a glyph into a settings row, Save, and confirm the pill's +tile changes without a restart; edit the JSON by hand and confirm the pill +follows; remove the file and confirm the starter defaults return; create a scene +in OBS and confirm its tile is `generic` until mapped. + +## Open items to verify at build time + +1. `SetSceneItemTransform` with position only (settled by live probe; confirm + once by hand). +2. `GetSpecialInputs.desktop1` round-trips through `SetInputMute` (settled; + confirm with the hand check above). +3. `SceneIcons.js` `parse`/`serialize`/one-code-point check covered in node + before the QML that uses them is written. |