path: root/docs/superpowers/specs/2026-10-19-sb-builder-phase2-design.md

# sb-builder phase 2: packages and slackrepo builds

Date: 2026-10-19
Status: approved
Builds on: `2026-10-19-sb-builder-phase1-design.md` (helper, job model, GUI shell)

## Goal

Browse and search the ~10k packages of the SBo-danix tree, select packages
and run slackrepo `build`, `rebuild`, `update` and `remove` on them, plus
`update` of everything, as jobs. Every run is previewed first. Publishing
(the rsync push of the package repository) can be held back per run and
done later with a separate Publish action. Package signing works from the
GUI although jobs have no terminal.

## Facts this design rests on

Verified on the VM and in the slackrepo source (`~/Programming/GIT/GITHUB/slackrepo`):

- slackrepo usage: `slackrepo {build|rebuild|update|remove} [--preview|--dry-run] [item...]`,
  `--repo=SBo-danix`. Items may be bare package names.
- `--preview` on build/update does **not** set `OPT_DRY_RUN` (only revert and
  remove do, `libexec/functions.d/cmdfunctions.sh:231,398`). Finish hooks that
  test only `OPT_DRY_RUN` still run during a preview: today `gotify_hook`
  sends a message and `manage_packages_hook` runs without `--dry-run`.
- The live config `/etc/slackrepo/slackrepo_SBo-danix.conf` defines start hook
  `danix_gitfetch_hook` (fetch, rebase `danix-current` onto upstream; runs on
  every invocation, previews included) and finish hooks
  `manage_packages_hook bell_hook genrepos_hook gen_web_hook dep_template_hook rsync_push_hook recap_hook gotify_hook`.
  It contains a Gotify key and site hostnames. Stale copies exist in
  `repo-html-structure/` and in the upstream slackrepo clone.
- `genrepos_hook` runs `gen_repos_files.sh`, which signs with gpg.
  `/root/.genreprc` sets `USE_GPGAGENT=1` and `REPOSOWNERGPG=<fingerprint>`.
  The agent uses `pinentry-curses` (needs a tty) with
  `default-cache-ttl 14400`, `max-cache-ttl 21600`. With the key uncached
  and no tty, `gen_repos_files.sh` aborts ("GPG test failed").
- slackrepo DB (`database_SBo-danix_x86_64.sqlite3`): `revisions` row with
  `dep='/'` holds the built `version` and `built` time per itemid
  (`category/pkg`); `buildresults` holds the last `result` per itemid
  (`ok`, `failed`, `bad`, `aborted`, `removed`). 213 of 799 `buildresults`
  rows have legacy keys without a `/` and match no package.
- Hints: 623 in `/etc/slackrepo/SBo-danix/hintfiles/`; the default hint dir
  (`default_hintfiles/15.0`) is empty. Real usage: 594 `DELREQUIRES`, 13 each
  of `VERSION`, `DOWNLOAD`, `MD5SUM`, `ARCH`, 1 `OPTIONS`.
- Hint grammar (`libexec/functions.d/parsefunctions.sh:405-640`,
  `man/man5/slackrepo.hint.5`): slackrepo searches `<SBREPO>/<cat>/<pkg>/`,
  `HINTDIR/`, `HINTDIR/<cat>/<pkg>/`, `DEFAULT_HINTDIR/`,
  `DEFAULT_HINTDIR/<cat>/<pkg>/` for `<pkg>.hint`, first match wins, and
  sources it. Recognised keys: `SKIP VERSION OPTIONS GROUPADD USERADD
  NOWARNING NOWARNINGS ADDREQUIRES DELREQUIRES BUILDTIME CONFLICTS INSTALL
  NUMJOBS ANSWER CLEANUP PRAGMA SPECIAL ARCH DOWNLOAD MD5SUM SHA256SUM`, plus
  `DOWNLOAD_<arch>`, `MD5SUM_<arch>`, `SHA256SUM_<arch>` resolved through
  `ARCH`. The man page says `ANSWERS`; the code reads `ANSWER`.
- mkhint `--no-dl` writes `NODOWNLOAD=yes`, which slackrepo does not read
  (its switch is `DOWNLOAD="no"`). No current hint has it. Phase 4 changes
  mkhint to write `DOWNLOAD="no"`.

## Part 1: VM side, in the slackrepo_setup repo

slackrepo_setup becomes the source of truth for the slackrepo config.

- `slackrepo_SBo-danix.conf`: imported from the VM. Site values move out to
  `/etc/slackrepo/SBo-danix.local` (root-only, mode 600, not in git), which the
  config sources: rsync target, rsync password and excludes files, Gotify URL
  and key. The repo ships `SBo-danix.local.example` with placeholders.
- `sb-publish`: new script, the rsync push of `PKGREPO` that
  `rsync_push_hook` does today. Reads its settings from the `.local` file.
  `rsync_push_hook` calls it.
- Gates in the config:
  - `rsync_push_hook`: return 0 when `SBB_PUBLISH=n`.
  - `gotify_hook`: return 0 when `OPT_PREVIEW=y` (as well as `OPT_DRY_RUN=y`).
  - `manage_packages_hook`: pass `--dry-run` when `OPT_PREVIEW=y` too.
- `gpg-agent.conf` for root, managed here: today's content plus
  `allow-preset-passphrase`. Deploy runs `gpgconf --reload gpg-agent`.
- Terminal runs without `SBB_PUBLISH` behave as today, except that previews
  no longer notify Gotify or touch packages.
- The stale config copy in `repo-html-structure` (untracked, may still hold
  the Gotify key) is deleted after asking the user.
- Deploy (asks first): config to `/etc/slackrepo/`, `.local` created from the
  current live values only if missing, `sb-publish` to `/usr/local/bin`,
  `gpg-agent.conf` to `/root/.gnupg/`.

## Part 2: helper (`sbb-remote` 0.2.0)

### New config keys

```toml
slackrepo_repo = "SBo-danix"
hintdir        = "/etc/slackrepo/SBo-danix/hintfiles"
default_hintdir = ""          # optional, searched after hintdir
genreprc       = "/root/.genreprc"
```

### `inventory`

One JSON object `{"columns": [...], "rows": [[...], ...]}`, a row per
package directory `<cat>/<pkg>/<pkg>.SlackBuild` in `repo_dir`:

| column | source |
|---|---|
| `path` | `cat/pkg` |
| `version` | hint `VERSION` if the hint has one, else `.info` `VERSION` |
| `info_version` | `.info` `VERSION` |
| `hint_version` | hint `VERSION` or `null` |
| `built_version`, `built_time` | `revisions`, `dep='/'`, by path; `null` if none |
| `result` | `buildresults` by path; `null` if none |
| `maintainer`, `requires` | `.info` `MAINTAINER`, `REQUIRES` (list) |
| `summary` | `slack-desc` line 1, text inside the parentheses |
| `hint_path` | the hint found by slackrepo's search order (repo package dir, `hintdir`, `hintdir/<cat>/<pkg>`, `default_hintdir`, `default_hintdir/<cat>/<pkg>`), or `null` |
| `hints` | `{key: raw value}` of every assignment in that hint, or `null` |

`.info` and hint files are parsed, never sourced: continuation lines joined,
then `shlex.split(line, comments=True)`, keeping `NAME=value` tokens. A
value containing `$` is kept as written and flagged `"shell": true` for the
GUI, since it would be expanded by the shell. A missing DB yields `null` build
fields, not an error.

### Job kinds

| kind | argv | env |
|---|---|---|
| `slackrepo <action> [--preview] [--no-publish] [pkg...]` | `slackrepo --repo=<slackrepo_repo> <action> [--preview] pkg...` | `SBB_PUBLISH=n` with `--no-publish` |
| `publish` | `sb-publish` | |

- `action` is one of `build`, `rebuild`, `update`, `remove`.
- Package names match `^[A-Za-z0-9._+-]+$` and must name a package in the
  tree. Duplicates are dropped.
- `build`, `rebuild`, `remove` need at least one package; `update` with none
  means all.
- Kinds now return `(argv, env)`; `_run` merges `env` into the environment.
  Existing kinds return an empty env. `meta.json` records `env`.
- While the signing key is locked, `job-start` refuses a non-preview
  `slackrepo` job with `{"error": "signing key locked"}`. Previews and
  `publish` (which does not sign) are not affected.

### Signing key

- Key: `REPOSOWNERGPG` read from `genreprc` with a regex over
  `REPOSOWNERGPG="..."` (never sourced). Keygrips from
  `gpg --with-colons --with-keygrip --list-secret-keys <key>` (`grp` lines).
- `gpg-status`: `gpg-connect-agent "keyinfo <grip>" /bye` per keygrip; the key
  is unlocked when every keygrip reports cached. Returns
  `{"unlocked": bool, "key": "<fingerprint>"}`; `{"unlocked": null}` when no
  key is configured.
- `gpg-unlock`: reads the passphrase from stdin, runs
  `gpg-preset-passphrase --preset <grip>` per keygrip with the passphrase on
  its stdin, then verifies by signing a temp file with
  `gpg --batch --pinentry-mode error -u <key> -bas`. On failure it runs
  `gpg-preset-passphrase --forget` per keygrip and returns
  `{"error": "bad passphrase"}`. The passphrase is never put on a command
  line, in the environment, in a log or on disk.
- To verify in planning: whether a preset passphrase expires with the cache
  TTLs or lasts until the agent restarts. The GUI checks state before every
  real run either way.

### Version

`VERSION = "0.2.0"` in both helper and GUI; the existing banner asks for
`./install.sh --remote` until the VM is updated.

## Part 3: GUI

### Packages tab (`sb_builder/packages.py`, `Open > Packages`)

Layout B from brainstorming: query bar, chip rows, table, detail pane, action
bar.

- Model: `QAbstractTableModel` over the cached inventory, with a
  `QSortFilterProxyModel` subclass doing the filtering. Columns: check, name,
  category, version, built, status, maintainer. Sortable.
- Status, derived in the GUI:

  | status | rule |
  |---|---|
  | `never` | no `built_version` and no `result` |
  | `failed` | `result` in `failed`, `bad`, `aborted` |
  | `outdated` | `built_version` != `version` |
  | `ok` | otherwise |

  Shown as a colored glyph plus the word (status colors from the Dashboard).
  `# ponytail:` outdated compares versions only; slackrepo also rebuilds on
  SlackBuild or dependency changes, which its preview shows.
- Query bar: whitespace-separated terms, all must match, case-insensitive
  substring. Plain term: name or summary. `cat:`, `maint:`, `req:`,
  `status:` match that field.
- Chips: overlay group (`personal`, `pentesting`, `upstream` = any other
  category) and status group (`ok`, `failed`, `never`, `outdated`). OR within
  a group, AND across groups and with the query; an empty group does not
  filter.
- Detail pane for the current row: path, summary, `.info`, hint and built
  versions, built time, last result, maintainer, and the hint view below.

### Hint view (`sb_builder/hints.py`, in the detail pane)

Read-only and structured; never the raw file. `hints.interpret(hints, info)`
returns sections, the detail pane renders only those that apply:

| section | keys | shown as |
|---|---|---|
| Skipped (banner) | `SKIP` | `no` = no banner; `yes` = always skipped; other text = skipped, the text as reason |
| Version override | `VERSION`, `ARCH`, `DOWNLOAD[_arch]`, `MD5SUM[_arch]`, `SHA256SUM[_arch]` | `.info` version -> hint version; URLs after `ARCH` resolution (`DOWNLOAD_<ARCH>` wins when set); "checksums not checked" when `MD5SUM="no"` or `VERSION` set without `MD5SUM` (slackrepo forces `no`); "no download, source must be in the cache" for `DOWNLOAD="no"` and for `NODOWNLOAD=yes` (shown identically, by user decision) |
| Dependencies | `.info` `REQUIRES`, `DELREQUIRES`, `ADDREQUIRES`, `BUILDTIME`, `PRAGMA python3`, `CONFLICTS` | one chip row computed as slackrepo does: `.info` deps minus `DELREQUIRES` and `%README%` (struck through), `python3` added by the pragma, `ADDREQUIRES` added (+), `BUILDTIME` added and marked build-time only (not in package metadata); `CONFLICTS` listed apart. With no `.info` `REQUIRES`, `ADDREQUIRES` alone defines the deps |
| Build settings | `OPTIONS`, `NUMJOBS`, `INSTALL`, `ANSWER`, `NOWARNING`/`NOWARNINGS` | rows; `OPTIONS` split into `OPT=value` pairs; `INSTALL` as yes/no by its first letter (`Y`/`1`, `N`/`n`/`0`); `ANSWER` with `\n` shown as a return glyph; `NOWARNING` labelled as a regex |
| Pragmas | `PRAGMA`, `SPECIAL` | one row per pragma with its meaning from `slackrepo.hint(5)`; `x86arch=<arch>` and `unset_<VAR>` parsed; unknown pragmas shown as unknown |
| Host changes | `GROUPADD`, `USERADD` | table rows: gid, name; uid, name, group, home, shell, extra args, with slackrepo's defaults filled in (name = package, group = user, home `/dev/null`, shell `/bin/false`) |
| On uninstall | `CLEANUP` | the command |
| Ignored | any other key | "not read by slackrepo" (except `NODOWNLOAD`, see above) |

Values flagged `shell` show as written, marked "expanded by the shell". The
footer shows `hint_path`.
- Selection: check column, "Select visible", "Clear", "N selected" label.
  Actions apply to the checked packages, or to the current row when none is
  checked.
- Action bar: Build, Update, Rebuild, Remove, Update all, and Publish
  (confirm dialog, starts the `publish` job).
- Build/Update/Rebuild/Remove/Update all start the preview job
  (`slackrepo <action> --preview ...`) and open it in the Jobs tab.
- Inventory: fetched when the tab opens, on its Refresh button, and after any
  `slackrepo` or `publish` job ends. While loading, the old rows stay and the
  status line says "loading...".

### Jobs tab additions

- A finished `ok` preview of kind `slackrepo` shows [Run it] and a ☑ publish
  checkbox (initial state from config `builds.publish`). Run it starts the
  same action and packages without `--preview`, adding `--no-publish` when the
  box is unticked.
- Before starting any non-preview `slackrepo` job (Run it, or anywhere else)
  the GUI calls `gpg-status`; when locked it asks for the passphrase
  (`QInputDialog` in password mode), calls `gpg-unlock` with the passphrase on
  stdin, repeats on `bad passphrase`, then starts the job. Cancel aborts.
- `remote.call` gains an optional `stdin` bytes argument, written to the
  QProcess and closed.

### Main window and Dashboard

- Status bar: signing key state, 🔒 locked / 🔓 unlocked, refreshed with the
  stats poll; hidden when the helper reports no key.
- Dashboard: clicking the Packages tile opens the Packages tab, clicking the
  Recent jobs card opens the Jobs tab.
- `Open` menu: Packages, Jobs.

### GUI config

```toml
[builds]
publish = true   # initial state of the publish checkbox
```

## Error handling

| case | behaviour |
|---|---|
| inventory slow (cold disk ~7s) | async; old rows stay, "loading..." |
| inventory call fails | status bar message, old rows stay |
| preview fails | no [Run it]; the log says why |
| key locked | password dialog; `bad passphrase` asks again; Cancel aborts |
| `job-start` refuses (locked) | same dialog, then retry once |
| busy | existing busy dialog |
| `genrepos` fails anyway | job `failed`, notification, slackrepo's message in the log |
| unknown package | helper rejects by name |

## Testing

stdlib `unittest`, headless, no VM.

- Helper: `inventory` on a fake tree, hint dirs and sqlite DB (hint version
  wins, hint search order precedence, shlex parsing with quotes, comments and
  continuation lines, `$` values flagged, legacy DB keys ignored, missing DB
  gives nulls);
  `slackrepo` kind (unknown package, `build` without packages, `update`
  without packages, duplicate names, `--no-publish` sets
  `SBB_PUBLISH=n` in env and in the child process); locked key refuses a real
  run, not a preview (gpg check patched); `gpg-unlock` feeds stdin to a fake
  `gpg-preset-passphrase` on `PATH` and forgets on failed verification.
- GUI: `hints.interpret` table-driven per key and derived rule (ARCH
  resolution, forced checksum skip, dependency computation incl. python3
  pragma and `ADDREQUIRES`-only, USERADD/GROUPADD defaults, SKIP forms,
  INSTALL letters, `NODOWNLOAD` shown as `DOWNLOAD="no"`, ignored keys,
  shell-flagged values); query parser and filter (prefixes, chips, OR/AND); status
  derivation; action target is checked rows else current row; [Run it] after
  an ok preview, with `--no-publish` when unticked; locked key opens the
  dialog before the real job (dialog patched); Dashboard tile clicks open
  tabs.
- slackrepo_setup: `bash -n` on config and `sb-publish`; source the config
  with stub `rsync`, `curl` and `manage-packages.sh` on `PATH` and check the
  gates with and without `SBB_PUBLISH=n` and `OPT_PREVIEW=y`.
- First real run with the user: unlock, preview an update of one package,
  run it without publish, then Publish.