path: root/docs/superpowers/specs/2026-10-17-obs-pill-enhancements-design.md

# OBS pill: desktop audio mute, webcam corners, scene grid

Extends the OBS pill from `2026-10-17-obs-pill-design.md`. Two new controls, a
refreshed panel, and a settings page for the scene icon map. This is a design
document; the plan follows.

## Intent

While OBS records, control two more things from the island pill: the desktop
audio (mute/unmute) and the webcam's on-canvas position (snap to a corner). At
the same time, make the panel's buttons icon-first so the panel is faster to
scan mid-recording.

## Goals

- Toggle desktop audio mute from the panel; state follows OBS.
- Snap the webcam to any of the 4 corners of the recorded canvas.
- Panel buttons become icon + short label.
- Scenes become an icon grid whose glyphs are mapped by name in a watched
  `~/.config` file, editable from the settings window, with a generic glyph for
  unmapped scenes.

## Non-goals / anti-goals

- Mic mute: the user has a physical button (unchanged from the original spec).
- Camera hardware settings, replay buffer, streaming, virtual camera.
- Scene creation or reordering from the pill.
- Persistent "current corner" state: corner placement is fire-and-forget.
- Echo cancellation: see "Mic bleed" below.
- A general source-transform editor: only the webcam is moved, only to corners.
- A glyph palette or font browser inside the settings page: glyphs are pasted
  from the Nerd Fonts cheat sheet.

## Mic bleed (setup, not code)

With desktop audio captured and the mic open, the mic also hears the speakers,
so the recording carries the app's audio twice. This is acoustic, not a routing
bug, and OBS cannot subtract the speaker signal from the mic. The chosen fix is
**headphones** while recording with the mic: no bleed, no pipeline change. The
pill does not touch the mic, and no code is added for this.

## Verified current state (2026-10-17, read from OBS's own files and live probes)

- Desktop audio is a **global** device, not a scene source: a
  `pulse_output_capture` of the monitor output, named `Desktop Audio`, kind
  `pulse_output_capture`, currently unmuted. `GetSpecialInputs.desktop1` is
  `Desktop Audio`, and `GetInputMute` resolves by that name.
- Base canvas is 1920x1080.
- Scenes present: `Desktop`, `Webcam Only`, `Window Only`, and `Intervallo`
  (Image Slide Show, the pause scene). `Intervallo` is the current program
  scene, so the panel opens without webcam controls until the user switches to
  `Desktop` (cosmetic; not a bug).
- Webcam sources: `camera` (Desktop) and `Camera` (Webcam Only), both
  `v4l2_input`. The existing per-scene webcam toggle already finds the first
  `v4l2_input`, so no name-based matching is needed.
- The live webcam item in `Desktop` is scaled 0.4656: source 640x360 renders as
  298x168 at position (1572, 861), `alignment: 5` (top-left).

## `Obs.qml` additions

New state:

- `desktopAudio`: string, the desktop-audio input name, `""` when none.
- `desktopMuted`: bool.
- `baseWidth`, `baseHeight`: real, the base canvas size.
- `camW`, `camH`: real, the webcam item's **rendered** size in the current
  scene.

On `Identified` (op 2), in addition to the existing `GetRecordStatus` and
`GetSceneList`:

- `GetSpecialInputs` -> `desktopAudio = desktop1` (the first non-empty desktop
  input name).
- `GetVideoSettings` -> `baseWidth` / `baseHeight`.
- If `desktopAudio` is non-empty, `GetInputMute { inputName: desktopAudio }` ->
  `desktopMuted`.

`identify()` must also subscribe to the Inputs event category:
`eventSubscriptions` becomes `4 | 8 | 64 | 128` = **204** (Scenes | Inputs |
Outputs | SceneItems). Without Inputs, `InputMuteStateChanged` never arrives and
`desktopMuted` would only ever be set by the initial `GetInputMute`.

New requests:

- `toggleAudio()`: if `desktopAudio`, `SetInputMute { inputName: desktopAudio,
  inputMuted: !desktopMuted }`. State is not set optimistically; it follows the
  `InputMuteStateChanged` event.
- `moveCam(corner)`: if `camId >= 0` and `baseWidth > 0`, compute the position
  and send `SetSceneItemTransform { sceneName: scene, sceneItemId: camId,
  sceneItemTransform: { positionX, positionY } }`. Send **position only**, never
  the full echoed transform: for an item with `boundsType:
  OBS_BOUNDS_NONE` (the live webcam), the echoed `boundsWidth: 0` fails the
  server's minimum of 1.

  Corners use a margin `m` (~16px) and the rendered size `camW`/`camH`:
  - `tl` `(m, m)`
  - `tr` `(baseWidth - camW - m, m)`
  - `bl` `(m, baseHeight - camH - m)`
  - `br` `(baseWidth - camW - m, baseHeight - camH - m)`

  The math assumes `alignment: 5` (top-left), which holds on the live webcam
  item; state that assumption in a code comment rather than relying on it
  silently.

New event:

- `InputMuteStateChanged`: if `d.inputName === desktopAudio`, set
  `desktopMuted = d.inputMuted`.

In the `GetSceneItemList` response (already handled), set `camW`/`camH` from the
webcam item's `sceneItemTransform.width` / `.height` (rendered, 298x168 on the
live item). Never use `sourceWidth`/`sourceHeight` (640x360); the item is scaled
and the corner math would land about 340px off-canvas.

## `ObsPill.qml` changes

Buttons switch from text-only to icon + short label:

- Record / Stop: record-circle / stop-circle glyph + "Record"/"Stop".
- Pause / Resume (visible only while recording): pause / play glyph +
  "Pause"/"Resume".
- Webcam (visible when `camId >= 0`): video / video-off glyph + "Webcam".
- Audio (visible when `desktopAudio !== ""`): volume-high / volume-off glyph +
  "Audio".
- Corner move: four icon-only corner-arrow buttons, visible when `camId >= 0`,
  calling `Obs.moveCam("tl" | "tr" | "bl" | "br")`. Icon-only because the arrow
  glyph is the label.

Unchanged:

- Status line (`Not recording`, `Recording 03:21`, `Paused 03:21`).
- At-rest pill dot and elapsed time.

### Scene grid

Scenes are square icon tiles in a `Grid` with `columns: 4`, reserved at the
bottom of the panel and always visible (not collapsible). Up to 8 tiles fit in
two rows; more scroll inside a fixed-height `Flickable`. Selection is click-only,
so scrolling never switches a scene. The current scene is highlighted as now.

Each scene's glyph comes from a config file,
`$XDG_CONFIG_HOME/quickshell/obs-scene-icons.json` (default
`~/.config/quickshell/obs-scene-icons.json`), watched and reloaded on change:

```json
{
  "generic": "\udb80\uded7",
  "scenes": {
    "Desktop": "\udb80\udf79",
    "Webcam Only": "\udb81\udd67",
    "Window Only": "\uf2d0",
    "Intervallo": "\udb80\udd76"
  }
}
```

(The `md-*` glyphs are supplementary-plane, so their JSON/JS form is a surrogate
pair; `JSON.stringify` also writes the glyph itself, which is equivalent.)

That starter map is also the built-in default in
`desktop/island/SceneIcons.js` (`pragma library`), used when the file is missing
or unreadable, so the pill works before the settings page is ever opened.
`SceneIcons.js` holds the pure `defaults`, `parse(text)`, `serialize(map)`,
`iconFor(map, name)` and the one-code-point check; a small `SceneIcons.qml`
singleton owns the watched `FileView`, exposes `iconFor(name)` and `save(map)`.
Unmapped scenes, including any added in OBS later, show `generic`. A missing or
unparseable file yields the starter map; a valid file is used as-is and is not
merged with the defaults, so deleting a key makes that scene `generic` again.

The file is user-editable and the settings page writes it; there is no new state
in `Obs.qml`.

Tiles are about 36px square: 4 across is ~168px with gaps, so the current 260px
panel width holds them without horizontal growth. The control-button row is the
width driver; the builder may widen the panel or wrap that row.

Panel height follows `controls.implicitHeight` as before. The `Flickable` has a
fixed two-row height, so the panel does not grow as scenes are added.

### Webcam Only

The corner buttons also appear in `Webcam Only`, where the webcam fills the
canvas; moving it there shifts it slightly off-centre. That is harmless and
accepted rather than adding scene-name or display-source detection.

## Settings page: OBS scenes

A new module `desktop/modules/obs/ObsScenesModule.qml`, `settings: true`, group
"Desktop", label "OBS scenes", so it joins the settings window's sidebar and
search. It imports `../../island` for `Obs` (the live scene list) and
`SceneIcons` (read and save). Its page lists one row per scene: the scene name
and a paste field holding that scene's glyph, rendered with `Theme.iconFamily`.
A final "Default for unmapped scenes" row edits `generic`.

Picking a glyph is paste-only: the user finds an icon on the Nerd Fonts cheat
sheet and pastes the character into the field (e.g. U+F02DC, `md-home`). No
palette and no built-in font browser; the cheat sheet already is one.

Rows are the union of the file's `scenes` keys and the scenes `Obs` currently
reports, so the page works with OBS closed (configured rows) and offers newly
created scenes once OBS is connected. A scene that exists in neither cannot be
added from the page; add it in OBS, or add a key to the JSON by hand.

Each field accepts exactly one code point, surrogate pairs included (a
supplementary-plane glyph such as U+F02DC is two UTF-16 units). An empty field
means the scene falls back to `generic`. The `generic` row itself must also hold
one code point; empty falls back to the built-in `md-help_circle` default, so
there is always a visible tile. Invalid input (zero or more than one code
point) gets a red border and blocks Save. Edits are a draft behind the page's
`SaveBar`, as with the other settings pages; Save serialises `{generic, scenes}`
through `SceneIcons.save()` and writes the file atomically. A glyph missing from
the installed font renders as tofu in the field, which is the visible sign to
pick another.

## Icons

Codepoints verified 2026-10-17 against the installed Inconsolata Nerd Font
Regular's cmap (all present). The control glyphs and the default scene glyphs
below are fixed in code; the scene glyphs actually rendered are user-supplied
from the config file. Re-check at build time if the font changes:

| Use | Glyph | U+ | JS/JSON escape |
| --- | --- | --- | --- |
| Record | `md-record_circle` | U+F0EC2 | `\udb83\udec2` |
| Stop | `md-stop_circle` | U+F0666 | `\udb81\ude66` |
| Pause | `md-pause` | U+F03E4 | `\udb80\udfe4` |
| Resume | `md-play` | U+F040A | `\udb81\udc0a` |
| Webcam on | `md-video` | U+F0567 | `\udb81\udd67` |
| Webcam off | `md-video_off` | U+F0568 | `\udb81\udd68` |
| Audio on | `md-volume_high` | U+F057E | `\udb81\udd7e` |
| Audio off | `md-volume_off` | U+F0581 | `\udb81\udd81` |
| Corner TL | `md-arrow_top_left` | U+F005B | `\udb80\udc5b` |
| Corner TR | `md-arrow_top_right` | U+F005C | `\udb80\udc5c` |
| Corner BL | `md-arrow_bottom_left` | U+F0042 | `\udb80\udc42` |
| Corner BR | `md-arrow_bottom_right` | U+F0043 | `\udb80\udc43` |
| Scene: Desktop | `md-monitor` | U+F0379 | `\udb80\udf79` |
| Scene: Webcam Only | `md-video` | U+F0567 | `\udb81\udd67` |
| Scene: Window Only | `fa-window_maximize` | U+F2D0 | `\uf2d0` |
| Scene: Intervallo | `md-coffee` | U+F0176 | `\udb80\udd76` |
| Scene: unmapped | `md-help_circle` | U+F02D7 | `\udb80\uded7` |

The `md-*` codepoints are supplementary-plane: their `\u` form is a surrogate
pair, never a five-digit escape. `\uf0ec2` would parse as `\uf0ec` + `"2"`, a
different glyph plus a stray digit.

All render with `Theme.iconFamily`.

## Errors and edge cases

- `desktop1` empty (no desktop audio configured): hide the Audio button.
- Audio toggle semantics: muting `Desktop Audio` removes system audio from the
  recording; it does not affect the user's live speakers.
- `camId === -1` (no webcam in the current scene): hide Webcam and corner
  buttons (existing gate).
- A failed request is already logged by `response()`; state follows events, so a
  refused mute or transform does not desynchronise the pill.

## Verification (no state-changing probes)

Do not script state changes against the live OBS during development: a corner
move is written into the saved scene collection, and a mute during a take is not
restorable. Verify by hand instead: move the webcam to each corner and confirm
the framing, mute and unmute and confirm the pill follows (including a mute
triggered from OBS's own mixer), click scene tiles to switch, and with more than
eight scenes confirm scrolling the grid never changes the scene.

For the icon map: paste a glyph into a settings row, Save, and confirm the pill's
tile changes without a restart; edit the JSON by hand and confirm the pill
follows; remove the file and confirm the starter defaults return; create a scene
in OBS and confirm its tile is `generic` until mapped.

## Open items to verify at build time

1. `SetSceneItemTransform` with position only (settled by live probe; confirm
   once by hand).
2. `GetSpecialInputs.desktop1` round-trips through `SetInputMute` (settled;
   confirm with the hand check above).
3. `SceneIcons.js` `parse`/`serialize`/one-code-point check covered in node
   before the QML that uses them is written.