diff options
| author | Danilo M. <danix@danix.xyz> | 2026-10-20 20:30:57 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-10-20 20:30:57 +0200 |
| commit | 0602623806b7c0984544913d663804ed5f54e07f (patch) | |
| tree | fdd838b21d4dd0775f98760ca11d9e91e92d3745 /docs/design | |
| parent | cd9ca18cf12ab24e0d1532d01c0f621b06869545 (diff) | |
| download | sb-builder-master.tar.gz sb-builder-master.zip | |
README says what the GUI does today and what is planned, which defaults
are the author's setup, and how to set up the VM. The specs move to
docs/design/; the implementation plans are dropped (working files, now
kept out of git). AGENTS.md links slackrepo upstream instead of a local
clone path.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'docs/design')
| -rw-r--r-- | docs/design/2026-10-19-sb-builder-phase1-design.md | 335 | ||||
| -rw-r--r-- | docs/design/2026-10-19-sb-builder-phase2-design.md | 292 | ||||
| -rw-r--r-- | docs/design/2026-10-20-sb-builder-bundled-deps-design.md | 115 | ||||
| -rw-r--r-- | docs/design/2026-10-20-sb-builder-phantom-deps-design.md | 74 | ||||
| -rw-r--r-- | docs/design/2026-10-20-sb-builder-phase4-design.md | 266 | ||||
| -rw-r--r-- | docs/design/2026-10-20-sb-builder-ui-design.md | 163 |
6 files changed, 1245 insertions, 0 deletions
diff --git a/docs/design/2026-10-19-sb-builder-phase1-design.md b/docs/design/2026-10-19-sb-builder-phase1-design.md new file mode 100644 index 0000000..3ec3fc0 --- /dev/null +++ b/docs/design/2026-10-19-sb-builder-phase1-design.md @@ -0,0 +1,335 @@ +# sb-builder phase 1: core design + +Date: 2026-10-19 +Status: approved + +## Goal + +One Qt6 desktop GUI to drive the SlackBuild tooling on the build VM +(`buildsystem`): slackrepo, mkhint, overlay sync (sbpull, sbshadow), full repo +rebuild, and SBo orphan discovery. Long-running work runs as jobs on the VM +that survive the GUI disconnecting, and the GUI can reattach to them later. + +## Phases + +Each phase gets its own spec, plan and implementation cycle. + +1. **Core** (this spec): remote helper with the job model, GUI shell with + tabs, Dashboard, Jobs tab, VM control, config, notifications. Job kinds: + `sbpull`, `sbshadow`. +2. **Packages + slackrepo**: package browser over the large SBo-danix tree + (search and filter by name, category, overlay, maintainer, build status); + Build, Rebuild, Bulk update screens. +3. **Full repo rebuild**: `slackrepo_setup_SBo-danix.sh`, destructive, guarded. +4. **Hints**: mkhint browse, edit, update, create, `-F`, diff before save. + Requires non-interactive flags in mkhint (it has about 10 `read -p` + prompts and wraps slackrepo at `mkhint:1798`). Also: `--no-dl` must write + `DOWNLOAD="no"`, which slackrepo reads, instead of `NODOWNLOAD=yes`, + which it does not (keep reading old `NODOWNLOAD` lines for the NoDL column), + and existing hints on the VM get migrated: every `NODOWNLOAD=yes` line in + any hint slackrepo searches becomes `DOWNLOAD="no"` (an existing `DOWNLOAD` + line is replaced, since `DOWNLOAD="no"` overrides it anyway), backed up to + `.bak` first, idempotent. As of 2026-10-19 no hint carries the line. + Superseded by `2026-10-20-sb-builder-phase4-design.md`: the hint logic + moves into the helper instead of extending mkhint. +5. **Orphans**: SBo orphan discovery (`EMAIL="NOEMAIL"` in `.info`, the + `sbo-orphans` logic from slackbuilds-superrepo), including orphaned deps + in REQUIRES. Needs an SBo-stable tree, which is not on the VM today. + +Dashboard tiles become entry points as their screens land: Packages opens +the package browser (phase 2, select packages to build there), Built the +build screen, Last pull the sync actions, Recent jobs the Jobs tab, SBo +orphans the orphans screen (phase 5). + +Out of scope for all phases so far: overlay git operations (status, commit, +push in my-slackbuilds and friends). + +## Stack + +- GUI: Python 3 + PyQt6 (stock in Slackware -current, 6.11), including + `PyQt6.QtDBus` for notifications. No third-party packages. +- Helper: Python 3 stdlib only (VM runs 3.12, so `tomllib` is available). +- Config: TOML, read with `tomllib`. +- License: GPLv2-only. + +## Architecture + +``` +desktop buildsystem (root) ++----------------------+ ssh (ControlMaster) +---------------------------+ +| sb-builder (PyQt6) | ----------------------> | sbb-remote (py3 stdlib) | +| tabs: Dashboard, | sbb-remote <cmd> | stats / jobs / | +| Jobs (+later tabs) | <---------------------- | job-start / job-follow / | +| QProcess per call | JSON on stdout | job-kill, flock, setsid | ++----------------------+ | /var/lib/sb-builder/ | + | +---------------------------+ + | virsh (local libvirt) + v + domain "buildsystem" +``` + +### Repository layout + +``` +sb_builder/__main__.py entry point, config load +sb_builder/config.py GUI config: defaults, load, validate +sb_builder/remote.py the only code that runs ssh; also virsh +sb_builder/mainwindow.py tabs, menu, status bar, notifications +sb_builder/dashboard.py Dashboard tab +sb_builder/jobs.py Jobs tab +sb_builder/fmt.py text formatting shared by the tabs +sb-builder launcher (python3 -m sb_builder) +remote/sbb-remote VM helper, single file +config.example.toml GUI config example +remote/config.example.toml helper config example +install.sh local install; --remote deploys helper to VM +tests/test_remote.py +tests/test_gui.py +``` + +### Transport + +Every GUI call is a QProcess running: + +``` +ssh -o ControlMaster=auto -o ControlPath=<runtime dir>/sbb-%C -o ControlPersist=10m \ + <ssh_options...> <target> sbb-remote <cmd> [args...] +``` + +Calls are async. ssh exit 255 means the connection failed. Exit 127 means the +helper is not installed. + +### Trust boundary + +The helper runs only a fixed table of job kinds. The GUI sends a kind name +plus arguments; the helper validates each argument and builds argv itself. +No shell strings and no paths cross ssh. Validation rules: + +- unknown kind: rejected +- `sbpull` target: must be listed in the helper config `overlays` +- flags: only the ones the kind declares (`--yes`, `--no`) +- package names (phase 2 onward): `^[A-Za-z0-9._+-]+$` + +### Version check + +On connect the GUI runs `sbb-remote version`, which also returns the +configured `overlays` (the GUI builds its sbpull menu from them). A mismatch with the GUI's own +version shows a banner telling the user to run `./install.sh --remote`. + +## Helper: `sbb-remote` + +### State + +``` +<state_dir>/lock flock target, held for the whole job +<state_dir>/jobs/<id>/meta.json kind, args, argv, pid, start, end, exit, status +<state_dir>/jobs/<id>/log combined stdout+stderr, raw bytes +``` + +`<id>` is `YYYYMMDD-HHMMSS-<kind>`. `status` is one of `running`, `ok`, +`failed`, `killed`, `lost`, `busy`. + +Logs are kept forever (`# ponytail:` comment in code; add pruning once disk +usage is visible on the Dashboard). + +### Commands + +All print one JSON object on stdout, except `job-follow`. + +| command | behaviour | +|---|---| +| `version` | `{"version": "...", "overlays": [...]}` | +| `stats` | Dashboard data: load, uptime, disk free for repo and state dirs, package counts (total, personal, pentesting, shadowed = overlay packages also present in the pristine `current` branch, as sbshadow computes it), last sync per overlay (time of the newest commit touching `<overlay>/` in `repo_dir`), slackrepo summary from its sqlite DB (built, failed, never built). Any missing source yields `null` for that field, never an error | +| `jobs` | list of all jobs' meta. A job whose meta says `running` but whose pid is gone (`os.kill(pid, 0)` fails) is reported and rewritten as `lost` | +| `job-start <kind> [args]` | validate; spawn `sbb-remote _run <id>` with `start_new_session=True`, `stdin=/dev/null`, stdout/stderr to `/dev/null`; wait up to 3s for `_run` to write status; return `{"id": ...}`, or `{"busy": "<running id>"}` after deleting the refused job's dir | +| `job-follow <id> <offset>` | write log bytes from `offset` to stdout, keep following until the job is no longer `running`, then exit 0 | +| `job-kill <id>` | SIGTERM to the job's process group; `_run` records `killed` | +| `_run <id>` | internal, see below | + +### `_run` + +1. Take a non-blocking `fcntl.flock` on `<state_dir>/lock`. If held, write + status `busy` and exit. +2. Start the job argv with `stdin=/dev/null`, stdout and stderr to the log + file, in its own process group. Write `pid` and status `running`. +3. Wait. Write `end`, `exit` and final status (`ok` on 0, `killed` on + SIGTERM, else `failed`). + +Jobs never read input. A tool that prompts anyway gets EOF on `read` and, +under `set -e`, exits non-zero: the job fails, it never hangs and it never +answers yes by accident. Destructive steps are confirmed in the GUI. + +### Phase 1 job kinds + +| kind | argv built by helper | +|---|---| +| `sbpull <overlay> [--yes\|--no]` | `bash -c '. "$1" && sbpull "$2" "$3"' _ <sbpull_script> <overlay> <flag>` | +| `sbshadow [--yes\|--no]` | `sbshadow <flag>` | + +The flag is required in both kinds (a job has no stdin). + +### Helper config: `/etc/sb-builder/config.toml` + +```toml +state_dir = "/var/lib/sb-builder" +repo_dir = "/var/lib/sbopkg/SBo-danix" +upstream_branch = "current" +slackrepo_db = "/var/lib/slackrepo/SBo-danix/database_SBo-danix_x86_64.sqlite3" +sbpull_script = "/usr/local/share/slackrepo_setup/sbpull.bash" +overlays = ["personal", "pentesting"] +``` + +Missing file: these defaults. Unknown key or wrong type: every command +returns `{"error": "config: <key>: <reason>"}` and exits 1. + +### Errors + +Top-level handler catches any exception, prints `{"error": "<message>"}`, +exits 1. + +## Changes in slackrepo_setup + +Done in that repo, its own commit, redeployed to the VM. + +- `sbshadow --yes | --no`: no flag keeps the current interactive prompt. + `--yes` applies without asking. `--no` prints the plan and leaves the tree + unchanged (preview). `mkhint -F` runs in all cases, as it does today on + abort. +- `sbshadow` honours `SBO_REPO` to override the repo path, so it can be + tested against a throwaway repo. +- `sbpull <target> [--yes|--no]`: passes the flag to `sbshadow`, and returns + `sbshadow`'s exit status (today it returns `popd`'s, so a failure would + show as a successful job). + +GUI flow: `sbpull personal --no` pulls and shows the shadow plan in the log; +the Jobs tab offers "Apply shadowing", which starts `sbshadow --yes`. + +## GUI + +### Main window + +- QTabWidget. Dashboard is pinned. Other tabs are closable and + single-instance. +- Menu: `File > Reload config`, `File > Quit`, `Open > Jobs` (later phases + add entries). +- Status bar: connection state (ok, down, VM off, VM paused, VM running but + ssh unreachable, helper version mismatch), running job if any (click opens + it in Jobs), transient messages. + +### Dashboard + +Grid of QGroupBox tiles filled from `stats`, plus `jobs` for the Jobs tile: + +- VM: load, uptime, disk free +- Repo: total packages, personal, pentesting, shadowed +- slackrepo: built, failed, never built +- Sync: last pull per overlay +- Jobs: running job and last 10 (status, duration) +- Orphans: placeholder tile, filled in phase 5 + +Refreshes every `refresh.dashboard` seconds while visible, plus a Refresh +button. `null` fields show "n/a". When the connection is down the last +values stay, greyed, with "stale since HH:MM". + +### Jobs tab + +- Left: job list (status icon, kind and args, start, duration), refreshed + every `refresh.jobs` seconds. +- Right: log view (QPlainTextEdit, ANSI escapes stripped, `\r` rewrites the + current line), Kill button with confirm. +- Toolbar `Start`: one entry per overlay for `sbpull <overlay> --no`, plus + `sbshadow --no`, `sbshadow --yes` (confirm dialog). +- After a `--no` run whose log lists `shadow:` or `unshadow:` lines, an + "Apply shadowing" button starts `sbshadow --yes`. +- Selecting a job starts `job-follow` from offset 0. If the stream drops + while the job is still running, the next successful `jobs` poll restarts + it from the last received offset (the poll itself backs off while the VM + is down, see Error handling). +- `job-start` returning busy: dialog "busy: <id>" with [Show job]. + +### Notifications + +freedesktop `org.freedesktop.Notifications.Notify` via QtDBus. Events: +`finished`, `failed`, `killed`, `lost`, and `vm-up` (VM started and ssh +answering). Clicking a notification (`ActionInvoked`) raises the window and +opens the job. The GUI tracks the job states it has seen, so each job end +notifies once. + +### VM control + +When ssh fails and `vm.domain` is set, the GUI runs +`virsh -c <libvirt_uri> domstate <domain>` locally: + +| domstate | behaviour | +|---|---| +| `shut off` | status "VM off", [Start VM] (automatic when `vm.autostart`): `virsh start`, then poll ssh with backoff, notify `vm-up` | +| `paused` | status "VM paused", [Resume]: `virsh resume` | +| `running` | status "VM running, ssh unreachable", keep retrying | +| virsh missing or error | plain "down" | + +No shutdown control. + +### GUI config: `$XDG_CONFIG_HOME/sb-builder/config.toml` + +```toml +[remote] +target = "buildsystem" # any ssh destination +ssh_options = [] # extra ssh args; ControlMaster options always added + +[refresh] +dashboard = 30 +jobs = 5 + +[notify] +enabled = true +events = ["finished", "failed", "killed", "lost", "vm-up"] + +[vm] +libvirt_uri = "qemu:///system" +domain = "buildsystem" # empty disables VM control +autostart = false +``` + +Missing file: these defaults. Invalid key or type: dialog naming the key, +then defaults are used. Read at startup and on `File > Reload config`. No +settings dialog yet. + +## Error handling summary + +| case | behaviour | +|---|---| +| ssh down | status bar, VM check, stale tiles; the `jobs` poll interval doubles from `refresh.jobs` up to 30s, and resets once ssh answers | +| helper missing or version mismatch | banner with `./install.sh --remote` | +| busy | dialog with [Show job] | +| follow drop | reconnect from offset, job unaffected | +| helper exception | `{"error"}` shown in status bar | +| `_run` died, VM rebooted mid-job | `lost`, red in list, notification | +| config invalid | dialog naming key, defaults used | + +## Testing + +stdlib `unittest`, no frameworks. + +- `tests/test_remote.py`: imports `remote/sbb-remote` as a module with a + temp `state_dir`. Covers argument validation (unknown kind, overlay not in + config, unknown flag), lock busy, `lost` detection, `job-follow` from an + offset, and a full `_run` of a fake kind patched into the kind table + (ok, failed, killed). +- `tests/test_gui.py`: config load and validation, `remote.parse`, GUI and + helper versions equal, both example configs equal the defaults. Smoke tests + under `QT_QPA_PLATFORM=offscreen` with a fake Remote: tabs, Dashboard from + a canned `stats` dict with `null` fields, job-end notification sent once, + VM-off offer, carriage-return handling in the log view. +- slackrepo_setup: sbshadow `--yes` and `--no` verified in a throwaway git + repo, the same way the current behaviour was tested. +- First run against the real VM: manual, with the user. + +## Install + +- `install.sh`: symlinks the `sb-builder` launcher into `~/bin` (`BINDIR` + overrides), so repo edits are live; copies `config.example.toml` to the + config path only if no config exists. +- `install.sh --remote`: scp `remote/sbb-remote` to `<target>:/usr/local/bin/` + (`SBB_TARGET`, default `buildsystem`), + create `/etc/sb-builder/config.toml` from the example only if missing. + Asks before deploying. A SlackBuild comes later. diff --git a/docs/design/2026-10-19-sb-builder-phase2-design.md b/docs/design/2026-10-19-sb-builder-phase2-design.md new file mode 100644 index 0000000..e0eb794 --- /dev/null +++ b/docs/design/2026-10-19-sb-builder-phase2-design.md @@ -0,0 +1,292 @@ +# sb-builder phase 2: packages and slackrepo builds + +Date: 2026-10-19 +Status: approved +Builds on: `2026-10-19-sb-builder-phase1-design.md` (helper, job model, GUI shell) + +## Goal + +Browse and search the ~10k packages of the SBo-danix tree, select packages +and run slackrepo `build`, `rebuild`, `update` and `remove` on them, plus +`update` of everything, as jobs. Every run is previewed first. Publishing +(the rsync push of the package repository) can be held back per run and +done later with a separate Publish action. Package signing works from the +GUI although jobs have no terminal. + +## Facts this design rests on + +Verified on the VM and in the slackrepo source ([slackrepo](https://github.com/aclemons/slackrepo)): + +- slackrepo usage: `slackrepo {build|rebuild|update|remove} [--preview|--dry-run] [item...]`, + `--repo=SBo-danix`. Items may be bare package names. +- `--preview` on build/update does **not** set `OPT_DRY_RUN` (only revert and + remove do, `libexec/functions.d/cmdfunctions.sh:231,398`). Finish hooks that + test only `OPT_DRY_RUN` still run during a preview: today `gotify_hook` + sends a message and `manage_packages_hook` runs without `--dry-run`. +- The live config `/etc/slackrepo/slackrepo_SBo-danix.conf` defines start hook + `danix_gitfetch_hook` (fetch, rebase `danix-current` onto upstream; runs on + every invocation, previews included) and finish hooks + `manage_packages_hook bell_hook genrepos_hook gen_web_hook dep_template_hook rsync_push_hook recap_hook gotify_hook`. +- `genrepos_hook` runs `gen_repos_files.sh`, which signs with gpg. + `/root/.genreprc` sets `USE_GPGAGENT=1` and `REPOSOWNERGPG=<fingerprint>`. + The agent uses `pinentry-curses` (needs a tty) with + `default-cache-ttl 14400`, `max-cache-ttl 21600`. With the key uncached + and no tty, `gen_repos_files.sh` aborts ("GPG test failed"). +- slackrepo DB (`database_SBo-danix_x86_64.sqlite3`): `revisions` row with + `dep='/'` holds the built `version` and `built` time per itemid + (`category/pkg`); `buildresults` holds the last `result` per itemid + (`ok`, `failed`, `bad`, `aborted`, `removed`). 213 of 799 `buildresults` + rows have legacy keys without a `/` and match no package. +- Hints: 623 in `/etc/slackrepo/SBo-danix/hintfiles/`; the default hint dir + (`default_hintfiles/15.0`) is empty. Real usage: 594 `DELREQUIRES`, 13 each + of `VERSION`, `DOWNLOAD`, `MD5SUM`, `ARCH`, 1 `OPTIONS`. +- Hint grammar (`libexec/functions.d/parsefunctions.sh:405-640`, + `man/man5/slackrepo.hint.5`): slackrepo searches `<SBREPO>/<cat>/<pkg>/`, + `HINTDIR/`, `HINTDIR/<cat>/<pkg>/`, `DEFAULT_HINTDIR/`, + `DEFAULT_HINTDIR/<cat>/<pkg>/` for `<pkg>.hint`, first match wins, and + sources it. Recognised keys: `SKIP VERSION OPTIONS GROUPADD USERADD + NOWARNING NOWARNINGS ADDREQUIRES DELREQUIRES BUILDTIME CONFLICTS INSTALL + NUMJOBS ANSWER CLEANUP PRAGMA SPECIAL ARCH DOWNLOAD MD5SUM SHA256SUM`, plus + `DOWNLOAD_<arch>`, `MD5SUM_<arch>`, `SHA256SUM_<arch>` resolved through + `ARCH`. The man page says `ANSWERS`; the code reads `ANSWER`. +- mkhint `--no-dl` writes `NODOWNLOAD=yes`, which slackrepo does not read + (its switch is `DOWNLOAD="no"`). No current hint has it. Phase 4 changes + mkhint to write `DOWNLOAD="no"`. + +## Part 1: VM side, in the slackrepo_setup repo + +slackrepo_setup becomes the source of truth for the slackrepo config. + +- `slackrepo_SBo-danix.conf`: imported from the VM. Site values move out to + `/etc/slackrepo/SBo-danix.local` (root-only, mode 600, not in git), which the + config sources: rsync target, rsync password and excludes files, Gotify URL + and key. The repo ships `SBo-danix.local.example` with placeholders. +- `sb-publish`: new script, the rsync push of `PKGREPO` that + `rsync_push_hook` does today. Reads its settings from the `.local` file. + `rsync_push_hook` calls it. +- Gates in the config: + - `rsync_push_hook`: return 0 when `SBB_PUBLISH=n`. + - `gotify_hook`: return 0 when `OPT_PREVIEW=y` (as well as `OPT_DRY_RUN=y`). + - `manage_packages_hook`: pass `--dry-run` when `OPT_PREVIEW=y` too. +- `gpg-agent.conf` for root, managed here: today's content plus + `allow-preset-passphrase`. Deploy runs `gpgconf --reload gpg-agent`. +- Terminal runs without `SBB_PUBLISH` behave as today, except that previews + no longer notify Gotify or touch packages. +- A stale untracked copy of the old config is deleted after asking the user. +- Deploy (asks first): config to `/etc/slackrepo/`, `.local` created from the + current live values only if missing, `sb-publish` to `/usr/local/bin`, + `gpg-agent.conf` to `/root/.gnupg/`. + +## Part 2: helper (`sbb-remote` 0.2.0) + +### New config keys + +```toml +slackrepo_repo = "SBo-danix" +hintdir = "/etc/slackrepo/SBo-danix/hintfiles" +default_hintdir = "" # optional, searched after hintdir +genreprc = "/root/.genreprc" +``` + +### `inventory` + +One JSON object `{"columns": [...], "rows": [[...], ...]}`, a row per +package directory `<cat>/<pkg>/<pkg>.SlackBuild` in `repo_dir`: + +| column | source | +|---|---| +| `path` | `cat/pkg` | +| `version` | hint `VERSION` if the hint has one, else `.info` `VERSION` | +| `info_version` | `.info` `VERSION` | +| `hint_version` | hint `VERSION` or `null` | +| `built_version`, `built_time` | `revisions`, `dep='/'`, by path; `null` if none | +| `result` | `buildresults` by path; `null` if none | +| `maintainer`, `requires` | `.info` `MAINTAINER`, `REQUIRES` (list) | +| `summary` | `slack-desc` line 1, text inside the parentheses | +| `hint_path` | the hint found by slackrepo's search order (repo package dir, `hintdir`, `hintdir/<cat>/<pkg>`, `default_hintdir`, `default_hintdir/<cat>/<pkg>`), or `null` | +| `hints` | `{key: raw value}` of every assignment in that hint, or `null` | + +`.info` and hint files are parsed, never sourced: continuation lines joined, +then `shlex.split(line, comments=True)`, keeping `NAME=value` tokens. A +value containing `$` is kept as written and flagged `"shell": true` for the +GUI, since it would be expanded by the shell. A missing DB yields `null` build +fields, not an error. + +### Job kinds + +| kind | argv | env | +|---|---|---| +| `slackrepo <action> [--preview] [--no-publish] [pkg...]` | `slackrepo --repo=<slackrepo_repo> <action> [--preview] pkg...` | `SBB_PUBLISH=n` with `--no-publish` | +| `publish` | `sb-publish` | | + +- `action` is one of `build`, `rebuild`, `update`, `remove`. +- Package names match `^[A-Za-z0-9._+-]+$` and must name a package in the + tree. Duplicates are dropped. +- `build`, `rebuild`, `remove` need at least one package; `update` with none + means all. +- Kinds now return `(argv, env)`; `_run` merges `env` into the environment. + Existing kinds return an empty env. `meta.json` records `env`. +- While the signing key is locked, `job-start` refuses a non-preview + `slackrepo` job with `{"error": "signing key locked"}`. Previews and + `publish` (which does not sign) are not affected. + +### Signing key + +- Key: `REPOSOWNERGPG` read from `genreprc` with a regex over + `REPOSOWNERGPG="..."` (never sourced). Keygrips from + `gpg --with-colons --with-keygrip --list-secret-keys <key>` (`grp` lines). +- `gpg-status`: `gpg-connect-agent "keyinfo <grip>" /bye` per keygrip; the key + is unlocked when every keygrip reports cached. Returns + `{"unlocked": bool, "key": "<fingerprint>"}`; `{"unlocked": null}` when no + key is configured. +- `gpg-unlock`: reads the passphrase from stdin, runs + `gpg-preset-passphrase --preset <grip>` per keygrip with the passphrase on + its stdin, then verifies by signing a temp file with + `gpg --batch --pinentry-mode error -u <key> -bas`. On failure it runs + `gpg-preset-passphrase --forget` per keygrip and returns + `{"error": "bad passphrase"}`. The passphrase is never put on a command + line, in the environment, in a log or on disk. +- Found 2026-10-20: the preset passphrase is lost when gpg-agent exits, and + it exits when root's last login session ends. elogind then removes + `/run/user/0` (root does not linger), where the agent's sockets live; the + agent notices its socket is gone and terminates. The GUI's ssh + ControlMaster keeps a session open while sb-builder runs, so the key stays + unlocked as long as the GUI does, and locks about 10 minutes after it + quits (ControlPersist). Accepted as is; `loginctl enable-linger root` + would keep the agent across sessions. The GUI checks state before every + real run either way. + +### Version + +`VERSION = "0.2.0"` in both helper and GUI; the existing banner asks for +`./install.sh --remote` until the VM is updated. + +## Part 3: GUI + +### Packages tab (`sb_builder/packages.py`, `Open > Packages`) + +Layout B from brainstorming: query bar, chip rows, table, detail pane, action +bar. + +- Model: `QAbstractTableModel` over the cached inventory, with a + `QSortFilterProxyModel` subclass doing the filtering. Columns: check, name, + category, version, built, status, maintainer. Sortable. +- Status, derived in the GUI: + + | status | rule | + |---|---| + | `never` | no `built_version` and no `result` | + | `failed` | `result` in `failed`, `bad`, `aborted` | + | `outdated` | `built_version` != `version` | + | `ok` | otherwise | + + Shown as a colored glyph plus the word (status colors from the Dashboard). + `# ponytail:` outdated compares versions only; slackrepo also rebuilds on + SlackBuild or dependency changes, which its preview shows. +- Query bar: whitespace-separated terms, all must match, case-insensitive + substring. Plain term: name or summary. `cat:`, `maint:`, `req:`, + `status:` match that field. +- Chips: overlay group (`personal`, `pentesting`, `upstream` = any other + category) and status group (`ok`, `failed`, `never`, `outdated`). OR within + a group, AND across groups and with the query; an empty group does not + filter. +- Detail pane for the current row: path, summary, `.info`, hint and built + versions, built time, last result, maintainer, and the hint view below. + +### Hint view (`sb_builder/hints.py`, in the detail pane) + +Read-only and structured; never the raw file. `hints.interpret(hints, info)` +returns sections, the detail pane renders only those that apply: + +| section | keys | shown as | +|---|---|---| +| Skipped (banner) | `SKIP` | `no` = no banner; `yes` = always skipped; other text = skipped, the text as reason | +| Version override | `VERSION`, `ARCH`, `DOWNLOAD[_arch]`, `MD5SUM[_arch]`, `SHA256SUM[_arch]` | `.info` version -> hint version; URLs after `ARCH` resolution (`DOWNLOAD_<ARCH>` wins when set); "checksums not checked" when `MD5SUM="no"` or `VERSION` set without `MD5SUM` (slackrepo forces `no`); "no download, source must be in the cache" for `DOWNLOAD="no"` and for `NODOWNLOAD=yes` (shown identically, by user decision) | +| Dependencies | `.info` `REQUIRES`, `DELREQUIRES`, `ADDREQUIRES`, `BUILDTIME`, `PRAGMA python3`, `CONFLICTS` | one chip row computed as slackrepo does: `.info` deps minus `DELREQUIRES` and `%README%` (struck through), `python3` added by the pragma, `ADDREQUIRES` added (+), `BUILDTIME` added and marked build-time only (not in package metadata); `CONFLICTS` listed apart. With no `.info` `REQUIRES`, `ADDREQUIRES` alone defines the deps | +| Build settings | `OPTIONS`, `NUMJOBS`, `INSTALL`, `ANSWER`, `NOWARNING`/`NOWARNINGS` | rows; `OPTIONS` split into `OPT=value` pairs; `INSTALL` as yes/no by its first letter (`Y`/`1`, `N`/`n`/`0`); `ANSWER` with `\n` shown as a return glyph; `NOWARNING` labelled as a regex | +| Pragmas | `PRAGMA`, `SPECIAL` | one row per pragma with its meaning from `slackrepo.hint(5)`; `x86arch=<arch>` and `unset_<VAR>` parsed; unknown pragmas shown as unknown | +| Host changes | `GROUPADD`, `USERADD` | table rows: gid, name; uid, name, group, home, shell, extra args, with slackrepo's defaults filled in (name = package, group = user, home `/dev/null`, shell `/bin/false`) | +| On uninstall | `CLEANUP` | the command | +| Ignored | any other key | "not read by slackrepo" (except `NODOWNLOAD`, see above) | + +Values flagged `shell` show as written, marked "expanded by the shell". The +footer shows `hint_path`. +- Selection: check column, "Select visible", "Clear", "N selected" label. + Actions apply to the checked packages, or to the current row when none is + checked. +- Action bar: Build, Update, Rebuild, Remove, Update all, and Publish + (confirm dialog, starts the `publish` job). +- Build/Update/Rebuild/Remove/Update all start the preview job + (`slackrepo <action> --preview ...`) and open it in the Jobs tab. +- Inventory: fetched when the tab opens, on its Refresh button, and after any + `slackrepo` or `publish` job ends. While loading, the old rows stay and the + status line says "loading...". + +### Jobs tab additions + +- A finished `ok` preview of kind `slackrepo` shows [Run it] and a ☑ publish + checkbox (initial state from config `builds.publish`). Run it starts the + same action and packages without `--preview`, adding `--no-publish` when the + box is unticked. +- Before starting any non-preview `slackrepo` job (Run it, or anywhere else) + the GUI calls `gpg-status`; when locked it asks for the passphrase + (`QInputDialog` in password mode), calls `gpg-unlock` with the passphrase on + stdin, repeats on `bad passphrase`, then starts the job. Cancel aborts. +- `remote.call` gains an optional `stdin` bytes argument, written to the + QProcess and closed. + +### Main window and Dashboard + +- Status bar: signing key state, 🔒 locked / 🔓 unlocked, refreshed with the + stats poll; hidden when the helper reports no key. +- Dashboard: clicking the Packages tile opens the Packages tab, clicking the + Recent jobs card opens the Jobs tab. +- `Open` menu: Packages, Jobs. + +### GUI config + +```toml +[builds] +publish = true # initial state of the publish checkbox +``` + +## Error handling + +| case | behaviour | +|---|---| +| inventory slow (cold disk ~7s) | async; old rows stay, "loading..." | +| inventory call fails | status bar message, old rows stay | +| preview fails | no [Run it]; the log says why | +| key locked | password dialog; `bad passphrase` asks again; Cancel aborts | +| `job-start` refuses (locked) | same dialog, then retry once | +| busy | existing busy dialog | +| `genrepos` fails anyway | job `failed`, notification, slackrepo's message in the log | +| unknown package | helper rejects by name | + +## Testing + +stdlib `unittest`, headless, no VM. + +- Helper: `inventory` on a fake tree, hint dirs and sqlite DB (hint version + wins, hint search order precedence, shlex parsing with quotes, comments and + continuation lines, `$` values flagged, legacy DB keys ignored, missing DB + gives nulls); + `slackrepo` kind (unknown package, `build` without packages, `update` + without packages, duplicate names, `--no-publish` sets + `SBB_PUBLISH=n` in env and in the child process); locked key refuses a real + run, not a preview (gpg check patched); `gpg-unlock` feeds stdin to a fake + `gpg-preset-passphrase` on `PATH` and forgets on failed verification. +- GUI: `hints.interpret` table-driven per key and derived rule (ARCH + resolution, forced checksum skip, dependency computation incl. python3 + pragma and `ADDREQUIRES`-only, USERADD/GROUPADD defaults, SKIP forms, + INSTALL letters, `NODOWNLOAD` shown as `DOWNLOAD="no"`, ignored keys, + shell-flagged values); query parser and filter (prefixes, chips, OR/AND); status + derivation; action target is checked rows else current row; [Run it] after + an ok preview, with `--no-publish` when unticked; locked key opens the + dialog before the real job (dialog patched); Dashboard tile clicks open + tabs. +- slackrepo_setup: `bash -n` on config and `sb-publish`; source the config + with stub `rsync`, `curl` and `manage-packages.sh` on `PATH` and check the + gates with and without `SBB_PUBLISH=n` and `OPT_PREVIEW=y`. +- First real run with the user: unlock, preview an update of one package, + run it without publish, then Publish. diff --git a/docs/design/2026-10-20-sb-builder-bundled-deps-design.md b/docs/design/2026-10-20-sb-builder-bundled-deps-design.md new file mode 100644 index 0000000..de6ad1d --- /dev/null +++ b/docs/design/2026-10-20-sb-builder-bundled-deps-design.md @@ -0,0 +1,115 @@ +# sb-builder: bundled deps (phase 4b, part 2) + +Date: 2026-10-20 +Status: approved +Builds on: `2026-10-20-sb-builder-phase4-design.md`, `2026-10-20-sb-builder-phantom-deps-design.md` + +## Goal + +`hint-update` handles packages with several downloads, so mkhint's bundled-dep +code (url mode) is no longer needed and part 3 can remove mkhint from the VM. + +## Facts + +- 64 packages are tracked in `nvchecker.toml`; 4 have more than one download + (686 in the whole tree): + - neovim, 13 extra: upstream lists them in `cmake.deps/deps.txt` + (`NAME_URL <url>` lines), at a path fixed per tag. + - openvino, 9 extra: git submodules pinned by commit. Lives in the personal + overlay, its `.info` is edited there; not hint-managed. + - feroxbuster, 1 extra: a self-hosted vendor tarball carrying the main + version (`feroxbuster-<ver>-vendor.tar.gz`). + - exploitdb, 1 extra: `exploitdb-bin-sploits`, its own version, unchanged + since 2022. +- `apply_version` swaps the version through the whole hint text but + downloads only the first URL of each `DOWNLOAD*` for its md5; other URLs + keep their old md5. A feroxbuster bump today writes the new vendor URL + with the old md5. +- mkhint's url mode (`/usr/local/bin/mkhint`, `reconcile_bundle_deps_url` and + helpers, about 200 lines of bash) fetches the manifest at the new version, + takes the `_URL` lines, matches each extra hint URL to one, compares + versions and swaps the changed ones, downloading them for the md5. + +## Scope + +In: md5 for every changed URL (all packages), manifest reconcile (packages +in `[manifests]`). + +Out: git submodule mode (openvino, not hint-managed), separately versioned +extras with no manifest (exploitdb), a GUI for the manifest map, adding deps +the manifest has and the hint lacks (logged only), part 3 (removing mkhint, +nvchecker and their config from the VM). + +## Helper (`sbb-remote` 0.8.0) + +### Config + +`[manifests]` table, package -> manifest URL template, `{VERSION}` replaced +by the new version. The default holds neovim's, so the VM needs no new line. +Replaces `manifest_packages`, and with it the refusal of manifest packages by +`hint-update` and `hint-new`. `remote/config.example.toml` shows the default: + + [manifests] + neovim = "https://raw.githubusercontent.com/neovim/neovim/v{VERSION}/cmake.deps/deps.txt" + +`manifest_packages` becomes an unknown key: the deploy drops its line from +the VM's `/etc/sb-builder/config.toml`, or the helper refuses to start. + +### `apply_version`, every package + +After the version swap, for each `DOWNLOAD*`/`MD5SUM*` pair, download every +URL that differs from the one at the same position before the swap, plus the +first URL as today, and set its md5. Unchanged continuation URLs keep their +md5. `SKIP_URLS` (`UNSUPPORTED`, `UNTESTED`) and `--no-dl` behave as today. + +### `apply_version`, packages in `[manifests]` + +Runs after the version swap and before the downloads, only when a new +version is given and `--no-dl` is not: + +1. Fetch the manifest at the new version. +2. Extract every `https?://` URL from it (regex, any file format). +3. For each extra URL (position 1 and later) of `DOWNLOAD`, find its match: + first by GitHub `owner/repo`, lowercased, skipping the shared blob repo + `neovim/deps`; then by stem, the basename without archive extension and + without a trailing `-<version>` (`[vV]?` digit, or 7+ hex). +4. Version of a URL: basename without archive extension; on GitHub, strip a + leading `<repo>-` or `<repo>_` (case-insensitive) and the rest is the + version; elsewhere, the trailing `-<version>` or `-<sha>`. Then drop a + leading `v` and turn `-` into `_`. The first URL's manifest counterpart is + never reconciled. +5. A match whose version differs replaces the hint URL; the download step + above then gives it its md5. + +Log lines: `<pkg>: <dep> <old> -> <new>` per changed dep, `<pkg>: <dep> no +manifest match, left as is`, `<pkg>: manifest has N deps not in the hint`. + +### Errors + +A manifest fetch failure, or a download failure, fails that package: its +hint and `.bak` stay untouched, `<pkg>: failed: <why>` is logged, the job +moves on, as for the main download today. An empty manifest (no URLs) is a +failure too: a moved file must not pass as "all current". + +## GUI + +None. The hint diff dialog already shows the changed `DOWNLOAD`/`MD5SUM` +lines. + +## Testing + +`test_remote.py`, no network (the existing `Served` localhost http server): + +- URL version and match against neovim's real URL shapes: github archive + `v<ver>/<name>-<ver>`, LuaJIT by commit, luv release asset, lpeg off + GitHub, mixed-case owner (JuliaStrings). +- manifest reconcile: one dep changed, one unchanged, one unmatched; the + changed one gets the new URL and md5, the rest keep theirs. +- a continuation URL carrying the main version gets a new md5 without a + manifest (feroxbuster). +- manifest fetch failure leaves the hint untouched. + +## Version + +0.8.0 in `remote/sbb-remote` and `sb_builder/__init__.py`; deploy with +`./install.sh --remote` after asking. diff --git a/docs/design/2026-10-20-sb-builder-phantom-deps-design.md b/docs/design/2026-10-20-sb-builder-phantom-deps-design.md new file mode 100644 index 0000000..46bc41c --- /dev/null +++ b/docs/design/2026-10-20-sb-builder-phantom-deps-design.md @@ -0,0 +1,74 @@ +# sb-builder: phantom deps (phase 4b, part 1) + +Date: 2026-10-20 +Status: approved +Builds on: `2026-10-20-sb-builder-phase4-design.md`, `2026-10-20-sb-builder-ui-design.md` + +## Goal + +Manage the -current phantom deps (deps SBo packages still require but +Slackware -current ships) from sb-builder: the list, suggestions from the +tree, and the `DELREQUIRES` sweep that mkhint's `-F` does today. Found when +-current added zenity: 8 SBo packages require it, nothing in the GUI could +mark it. + +## Facts + +- When a package enters the official tree, SBo removes it. A `REQUIRES` + name that matches no package of the repo is then left behind. On + 2026-10-20 the tree has 140 such names, zenity among them; the 17 on + mkhint's list (`/root/.config/mkhint/phantom-deps`) are all among them. +- sbshadow ends with `mkhint -F` (skipped on `--no`). mkhint's `-F` merges + the phantom deps found in each `.info` `REQUIRES` into the hint's + `DELREQUIRES`, `.bak` first, a minimal hint when there is none; it never + removes entries. +- openvino moved into the personal overlay; only neovim still needs mkhint's + bundled-dep logic. + +## Helper (`sbb-remote` 0.7.0) + +- Config: `phantom_deps_file`, default `/etc/sb-builder/phantom-deps`: one + name per line, `#` comments, mkhint's format. Missing file = empty list. + `manifest_packages` default becomes `["neovim"]`. +- `phantom-list`: `{"deps": [...], "suggestions": [{"name", "needed_by": + [...]}]}`. Suggestions: every `REQUIRES` name (without `%README%`) that + matches no package of the repo and is not on the list, sorted, with the + packages requiring it. +- `phantom-add <name>...`, `phantom-remove <name>...`: names must match + `PKGNAME`, need not exist in the repo. Rewrites the file atomically, + comments kept, one name per line, no duplicates. Returns the new list. +- Job kind `fix-current`: for every package whose `.info` `REQUIRES` names a + listed dep, merge those deps into `DELREQUIRES` of `<hintdir>/<pkg>.hint` + (union, first seen order; `.bak` only when it changes; a new hint has just + that line and no `.bak`). Unmanaged hint paths fail that package. Log one + line per changed hint, then `SBB-RESULT`. Add only, like mkhint. +- `fix-current-now`: the same sweep run inline, its log on stdout, for + sbshadow. + +## sbshadow (slackrepo_setup) + +The end of a non-preview run calls `sbb-remote fix-current-now` in place of +`mkhint -F`; without sbb-remote it says so and skips, as it does for mkhint +today. + +## GUI + +Hints ribbon: **Phantom deps…** (between Delete hint and the end). A +non-modal dialog: + +- the list, one name per row, with **Remove** for the selected rows; +- a name field with **Add**; +- **Suggestions**: tickable rows `name needed by a, b, c`, **Add ticked**; +- **Apply**: starts `fix-current` (the list itself is saved by each Add or + Remove at once); the usual diff dialog follows, with Revert per package. + +## Deploy (asks first) + +1. Helper 0.7.0. +2. `/etc/sb-builder/phantom-deps` from mkhint's file (17 entries). +3. sbshadow. + +## Out + +neovim bundled deps (phase 4b, part 2), removing mkhint and nvchecker from +the VM (part 3), removing `DELREQUIRES` entries when a dep leaves the list. diff --git a/docs/design/2026-10-20-sb-builder-phase4-design.md b/docs/design/2026-10-20-sb-builder-phase4-design.md new file mode 100644 index 0000000..62b8d53 --- /dev/null +++ b/docs/design/2026-10-20-sb-builder-phase4-design.md @@ -0,0 +1,266 @@ +# sb-builder phase 4: hints + +Date: 2026-10-20 +Status: approved +Builds on: `2026-10-19-sb-builder-phase1-design.md` (helper, job model, GUI shell), +`2026-10-19-sb-builder-phase2-design.md` (inventory, hint view, slackrepo jobs) + +## Goal + +Manage slackrepo hint files from sb-builder: check upstream versions, update +version hints, create and delete hints, and see a diff of every change +with a way to revert it. + +Direction set by the user: the repo is run from the desktop, the VM shell is +for upgrading Slackware only. The VM-side tools were built for a terminal; a +tool may lose its in-VM usability if that makes sb-builder simpler. So the +hint logic of mkhint moves into `sbb-remote` (Python, JSON answers) and the +upstream check moves to the desktop. mkhint is not extended. + +## Facts this design rests on + +Verified on the VM, in the mkhint source (`~/Programming/GIT/mkhintfile/mkhint`, +1.4.0, 2291 lines) and in slackrepo ([slackrepo](https://github.com/aclemons/slackrepo)): + +- The VM has 607 hints, all in `hintdir`, none in subdirectories and none in + the SBo tree. 594 carry only `DELREQUIRES`, 13 carry + `VERSION` + `ARCH` + `DOWNLOAD` + `MD5SUM`, one carries `OPTIONS`. No hint + carries `NODOWNLOAD`. Pinned hints carry `DOWNLOAD_x86_64`/`MD5SUM_x86_64` + lines, mostly empty (`claude-code`: `DOWNLOAD="UNSUPPORTED"` with the URL in + `DOWNLOAD_x86_64`), and `ARCH="x86_64"`. +- slackrepo reads `DOWNLOAD="no"` as "no download" and ignores `NODOWNLOAD`. + `DOWNLOAD_$ARCH` replaces `DOWNLOAD` when set + (`libexec/functions.d/parsefunctions.sh:457`), so `DOWNLOAD="no"` next to a + `DOWNLOAD_x86_64` line has no effect. +- mkhint's version update: back up to `.bak`, replace the old version by the + new one everywhere in the file, also the `_` -> `-` form when either version + contains `_`, also the dotless form of a dotted version inside URLs as a + whole number (`_swap_dotless_url_version`), then for `DOWNLOAD`/`MD5SUM` and + `DOWNLOAD_x86_64`/`MD5SUM_x86_64`: download the first URL and recompute its + md5, keep the md5 of continuation URLs unless the user typed a new URL. + `UNSUPPORTED`/`UNTESTED` values are skipped. +- mkhint's new hint from `.info`: copy the `.info`, drop `PRGNAM`, `HOMEPAGE`, + `MAINTAINER`, `EMAIL`, comment out `REQUIRES`, set `ARCH="x86_64"`, append + `DELREQUIRES` for phantom deps in `REQUIRES`, then apply a version as above + when one is given. +- mkhint's `-F`: for every `.info` in the repo whose `REQUIRES` contains a + phantom dep, merge those deps into the hint's `DELREQUIRES` (union, first + seen order, `.bak` only when something changes, minimal hint created when + absent). It edits only the `DELREQUIRES` line. sbshadow + (slackrepo_setup) runs it after every sync: that is where the 594 + DELREQUIRES-only hints come from, and why they exist (slackrepo must not + build deps -current does not need). Until now sbshadow ran it on `--no` + previews too. +- The VM's nvchecker config has 64 stanzas named after the packages, plus a + `[__config__]` with oldver, newver and keyfile. nvchecker v2.22 is + installed on the desktop, with a GitHub token in + `~/.config/nvchecker/keys.toml`. With `--logger json` it prints one JSON + object per line on stdout: `{"name", "version", "event": "updated"}` per + result, `{"name", "error", "level": "error"}` per failure; it exits 0 when + entries fail and needs no oldver or newver to run. +- A DELREQUIRES-only hint has no `VERSION`; mkhint's update cannot bump it. + Updating one pins it from the `.info` first; the next `mkhint -F` keeps the + pin. A package with a phantom dep always has such a hint, so `hint-new` + never sees one. +- slackrepo `update <item>` builds an item that exists, built or not + (`update_command` calls `build_command`). +- The phantom deps list on the VM (`/root/.config/mkhint/phantom-deps`) has + 18 entries. +- Two packages use bundled-dep manifests: neovim (url mode) and openvino + (sha mode, GitHub submodules). That code is about 470 lines of mkhint. + +## Scope + +In: upstream check, version update, new hint, delete hint, +`--no-dl` as `DOWNLOAD="no"`, diff and revert. + +Out, for good (the Hints screen covers them): mkhint `--list`, `--versions`, +`--review`, `--strip-version`, `--info`, `--clean`. + +Out, later (phase 4b, which then removes mkhint and nvchecker from the VM): +bundled-dep reconcile for neovim and openvino, and a helper `fix-current` +that sbshadow calls in place of `mkhint -F`, with the phantom deps list +moved into the helper config as its single source. Also out: generating nvchecker stanzas +for new packages (add them to `nvchecker.toml` by hand), free text hint +editing, migration of `NODOWNLOAD=yes` lines (none exist). + +## Part 1: upstream check, on the desktop + +- `nvchecker.toml` in the repo root: the VM's 64 stanzas, without + `[__config__]`. Stanza name = package name. Git-tracked, edited by hand. +- `sb_builder/upstream.py` runs nvchecker through `QProcess` (as `remote.py` + runs `virsh`) with a fixed argv: `nvchecker -c <tmp>/nvchecker.toml + --logger json`. The temp dir holds a generated config: a `[__config__]` + with `keyfile` (from the GUI config, when the file exists), followed by the + repo file's text. No oldver, newver or `nvtake`: the comparison is against + the hints. +- Result: `{pkg: version}` and `{pkg: error}` from the JSON log. A package + without a result, or a failed run (exit status, stderr), shows as + `nvchecker error` with the reason in the tooltip. +- Upstream versions are stored with `-` as `_` (mkhint's `_normalize_version`): + `-` is illegal in a SlackBuild version, so upstream `2026-10-14` is + packaged `2026_10_14`. The comparison and `hint-update` use that form. +- A package is outdated when its upstream version differs from the hint + `VERSION`, or from the `.info` `VERSION` when the hint has none. Versions + are compared as strings; no ordering is attempted. + +GUI config gains `[hints] nvchecker_keyfile`, default +`~/.config/nvchecker/keys.toml` (expanded). A missing keyfile is not an +error: nvchecker runs without a token. + +## Part 2: helper (`sbb-remote` 0.5.0) + +### Config + +New key `manifest_packages` (below). `remote/config.example.toml` shows it. + +### Arguments + +- package: `PKGNAME` and present in the repo (as for slackrepo jobs). +- version: `^[A-Za-z0-9._+-]+$`, at most 64 characters. +- hint path: always `<hintdir>/<pkg>.hint`. If `find_hint` returns a + different file for the package (a hint earlier in slackrepo's search + order), every hint command refuses: "hint lives at <path>, not managed". +- packages with a bundled-dep manifest are named in config key + `manifest_packages`, default `["neovim", "openvino"]`. `hint-update` and + `hint-new` with a version refuse them: "bundled deps: use mkhint until + phase 4b". + +### Job kinds + +- `hint-update <pkg>=<ver>... [--no-dl]`: per package, in order: compute the + new text (apply the version, below), then back the hint up to `.bak` and + write atomically. A failure (download, missing `.info` VERSION) leaves the + hint and its old `.bak` untouched, logs `<pkg>: failed: <why>` and + continues with the next package. A package without a hint gets a new one + built as `hint-new` does. The last log line is + `SBB-RESULT {"ok": [pkg...], "failed": [[pkg, why]...]}`. Exit status 1 + when any package failed. +- `hint-new <pkg> [<ver>] [--no-dl]` (since 0.6.0 `hint-new <pkg>... [--no-dl]`, + no version, see the UI design): refuses when the hint exists. Builds + the hint from the `.info` as mkhint does, without phantom deps (see the + facts), then applies the version when given. No `.bak` (a stale + one is removed); revert means delete. + +These jobs never sign, so they do not check the signing key. Their argv is +the helper itself: `python3 sbb-remote _hint <kind> <args>`, validated again +there. Job ids now allow a hyphenated kind (`...-hint-update`). + +`job-result <id>`: the last `SBB-RESULT` object of a job's log. + +### Applying a version + +1. A hint without `VERSION` is first pinned from the `.info`: its + `VERSION`, `DOWNLOAD*` and `MD5SUM*` lines, plus `ARCH="x86_64"` when the + hint has no `ARCH`, are put in front of the hint's text. Old version = the + hint's `VERSION`. With no version given (`hint-new <pkg>`), steps 2 and 3 + are skipped. +2. Replace old by new everywhere in the file except inside `MD5SUM*` + values. When either contains `_`, also the `-` forms. When old contains + `.`, also its dotless form inside URLs, as a whole number (no digit before + or after). +3. Without `--no-dl`: for `DOWNLOAD`/`MD5SUM` and + `DOWNLOAD_x86_64`/`MD5SUM_x86_64`, split the values on whitespace (hint + values may continue over lines with `\`). Skip a pair whose value is empty, + `UNSUPPORTED` or `UNTESTED`. Download the first URL with `urllib` + (timeout 60 s, streamed into `hashlib.md5`, nothing kept on disk) and + replace its md5; keep the md5s of the other URLs. Write values back one + URL per line with ` \` continuation, as mkhint does. +4. With `--no-dl`: nothing is downloaded. Every `DOWNLOAD*` and `MD5SUM*` + assignment is removed (whole multi-line values) and `DOWNLOAD="no"` is + written, because `DOWNLOAD_$ARCH` would override it. + +Hint text is edited as text, not regenerated, so comments, order and other +keys stay as they are. An assignment is `KEY="..."` (the value may span +lines inside the quotes) or `KEY=word`, at line start. + +### Commands (answer at once, not jobs) + +- `hint-diff <pkg>`: `{"bak": text|null, "current": text|null}`. Both null + is an error ("no backup"). +- `hint-revert <pkg>`: when a `.bak` exists, move it over the hint (also + recreating a deleted hint); when there is no `.bak` and the hint exists, + delete the hint (reverting a `hint-new`). Returns `{"ok": true}`. +- `hint-delete <pkg>`: move the hint to `.bak`. Errors when there is no hint. + +`.bak` is one level deep and overwritten by each change, the mkhint +convention. + +### Version + +`VERSION` 0.5.0 in `remote/sbb-remote` and `sb_builder/__init__.py`. + +## Part 3: GUI + +### Hints screen (`sb_builder/hints_tab.py`, ribbon entry and `Open > Hints`) + +- Rows: every package with a stanza in `nvchecker.toml` or a hint in + `hintdir`. Columns: checkbox, package, hint version, `.info` version, + upstream, state. Versions come from the existing `inventory` call. +- States: `outdated`, `current`, `no stanza`, `not checked`, + `nvchecker error` (stderr in the tooltip). Filter chips per state and a + text filter on the name, as in the Packages tab. +- Buttons: + - **Check upstream**: runs Part 1 over the whole file, fills the upstream + column, checks the outdated rows. Results live for the session. + - **Update selected**: starts `hint-update pkg=ver ...` for the checked + rows that have an upstream version; a **No download** checkbox adds + `--no-dl`. + - **Delete hint**: confirm dialog, then `hint-delete` for the current row. +- Jobs go through the Jobs tab like every other job. + +### After a hint job + +When a `hint-update` or `hint-new` job ends, the GUI reads +its `SBB-RESULT` line and opens a dialog with one entry per changed package: +a unified diff (`difflib`) of `hint-diff`, and a **Revert** button per +package. Failed packages are listed with their reason. The dialog ends with +**Build these**, which starts `slackrepo update --preview` for the changed +packages that were not reverted (update builds unbuilt items too), and +**Close**. Revert is offered once per package: a second one would delete +the restored hint. After Revert or Delete the inventory is reloaded. + +Each diff is fetched when its package is selected. + +### Packages detail pane + +- No hint: **New hint**, with an optional version field and a **No + download** checkbox, starts `hint-new`. +- Hint present: **Open in Hints** selects the package in the Hints screen. + +## Error handling + +- Bad arguments, unmanaged hint paths and manifest packages are refused by + the helper before a job starts; the GUI shows the message. +- A download failure (HTTP status, timeout, DNS) fails that package only and + leaves its hint as it was. +- Writes are atomic (temp file in the same directory, `os.replace`). +- nvchecker missing or failing on the desktop affects only the upstream + column; the rest of the screen works. +- A job without an `SBB-RESULT` line (killed, lost) opens no dialog; the + log shows what happened and `hint-diff` still works per package. + +## Testing + +- `test_remote.py`: an `http.server` thread serves fixture files for md5. + Version swap forms (plain, `_`/`-`, dotless in URL only, md5 hex not + touched), multi-line `DOWNLOAD`/`MD5SUM` with continuation md5s kept, + `UNSUPPORTED` skipped, `--no-dl` removing `DOWNLOAD_x86_64` and writing + `DOWNLOAD="no"`, a download failure restoring the hint while the next + package still updates, `SBB-RESULT` contents, `hint-new` from `.info`, + a DELREQUIRES-only hint pinned from the `.info`, new -> revert, update -> diff -> + revert, delete -> revert, manifest package refusal, unmanaged hint path + refusal, bad version and package rejection. +- `test_upstream.py`: generated config, JSON log parsing, states (hint + version, `.info` fallback), the repo file has no `[__config__]`, `Checker` + against a fake `nvchecker` on `PATH` and a missing one. No network. +- `test_gui.py`: Hints screen with a fake remote and injected upstream + results (states, checked rows, Update selected argv), diff dialog Revert + call. + +## Deploy (asks first) + +1. `./install.sh --remote` with helper 0.5.0. +2. slackrepo_setup's sbshadow, which no longer runs `mkhint -F` on `--no`. +3. mkhint, nvchecker and `/root/.config/{mkhint,nvchecker}` stay on the VM + until phase 4b. diff --git a/docs/design/2026-10-20-sb-builder-ui-design.md b/docs/design/2026-10-20-sb-builder-ui-design.md new file mode 100644 index 0000000..8a6844d --- /dev/null +++ b/docs/design/2026-10-20-sb-builder-ui-design.md @@ -0,0 +1,163 @@ +# sb-builder: one UI frame for every tab + +Date: 2026-10-20 +Status: approved +Builds on: phases 1, 2 and 4 (tabs, Jobs, Packages, Results, Hints) + +## Goal + +Every tab looks and works the same, and a multi-step task (check upstream, +update hints, preview, build) never sends the user hunting for the next +button in another place. Rules go into AGENTS.md so later tabs follow them. + +## What is wrong today + +- Action buttons sit at the bottom in Packages and Hints, in a side pane in + Results, at the top in Jobs. +- The ribbon opens tabs; tabs open and close. +- A step's follow-up lives elsewhere: Packages action -> auto switch to Jobs + -> Run it at the top of Jobs. Hints: Check upstream (bottom) -> tick -> + Update selected (bottom) -> diff dialog -> Build these -> Jobs -> Run it + (top). +- The Jobs Start menu name says nothing of what it holds. + +## Window frame + +Top to bottom: main ribbon, small ribbon, tab bar, the tab's view (its +filter bar first), status bar. + +- **Permanent tabs**: Dashboard (default), Packages, Hints, Results, Jobs, + created at startup, not closable. Each loads its data the first time it is + shown (Packages, Hints and Results call `inventory`/`results` then, not at + startup). Navigation: tab bar, Ctrl+PgUp/PgDn, Dashboard tiles (Packages, + Recent jobs, slackrepo results as today). The ribbon does not open tabs. + `File` menu unchanged. +- **Main ribbon**: `QToolBar`, 32 px icons, text under icon, the current + tab's actions in workflow order. Refilled on tab change. +- **Small ribbon**: `QToolBar` right under it, 16 px icons, text beside + icon, the tab's view controls. Hidden when the tab has none. +- Each tab exposes `actions` and `view_actions`, lists of `QAction` it owns, + so enabled and checked state survive tab switches. Icons are freedesktop + theme names with a `QStyle` fallback, as the ribbon does now. +- An action that does not apply is disabled with a tooltip saying why; + never hidden, so the ribbon does not shift. +- Filter bars (search, chips, `N shown · M selected` or `N of M`) stay at + the top of the view; their Refresh buttons move to the small ribbon. + +## Selection + +- Change actions act on the **ticked** rows; view actions (Show build log) + on the **highlighted** row. +- A click anywhere on a row toggles its tick (done for Packages and Hints). +- Ctrl+A ticks the visible rows, Ctrl+Shift+A unticks all (done); the + small ribbon's Select visible and Clear carry the same shortcuts. + +## Ribbons per tab + +| Tab | Main ribbon | Small ribbon | +|---|---|---| +| Dashboard | Refresh | none | +| Packages | Build, Update, Rebuild, Remove, Update all, New hint, Publish | Select visible, Clear, Refresh | +| Hints | Check upstream, Update selected, Delete hint | Select visible, Clear, No download (checkable), Refresh | +| Results | Show build log | Refresh | +| Jobs | Pull overlay (menu: one entry per overlay), Check shadowing, Run it, Kill | none | + +Disabled reasons (tooltips): + +- Build, Update, Rebuild, Remove: "tick packages first". +- New hint: "none of the ticked packages is without a hint". +- Update selected: "tick outdated packages after Check upstream". +- Delete hint: "none of the ticked packages has a hint". +- Show build log: "select a result with a log". +- Run it: "select a finished preview". +- Kill: "select a running job". +- Check upstream while running: "checking…". + +Update all, Publish, Pull overlay and Check shadowing are always enabled +(the helper refuses a second job with the busy answer, as today). + +Packages' detail pane keeps **Open in Hints** as a link (navigation, not an +action). Its New hint button moves to the ribbon. + +### New hint on ticked packages + +`hint-new` takes one or more packages and no version: `hint-new <pkg>... +[--no-dl]`. Each gets the `.info` pin; Update selected from upstream sets a +version later. The GUI sends no `--no-dl` here (Update selected has the +toggle); the helper keeps the flag. Every package must be without a hint, else the job is +refused before it starts, as today for one. Helper `VERSION` 0.6.0 in both +files, VM redeploy. The New hint dialog goes away. + +## Next-step dialogs + +- Shown only for jobs started from this GUI session: the main window keeps + the ids `job-start` returned. Jobs seen on reconnect or started elsewhere + only notify, as today. +- Non-modal, one per job, never reopened by polling. +- **Preview ended ok** (`slackrepo` with `--preview`): dialog titled + `Preview: <kind and args without --preview>`, with the whole job log + (`job-follow` from offset 0; the job has ended, so the stream closes by + itself), a **publish** checkbox (default `builds.publish`), **Run it** + (default button) and **Not now**. Run it goes through the signing-key + check (`start_signed`) and starts the real run. +- **sbpull or sbshadow preview ended ok with a shadow plan** (`shadow:` or + `unshadow:` lines, as `SHADOW_PLAN` today): same dialog, **Apply + shadowing** (`sbshadow --yes`, after a confirm: it changes the repo) and + **Not now**. Without a plan, no dialog. +- **Hint job ended** (`hint-update`, `hint-new`) or a delete: the existing + diff dialog; Build these starts `slackrepo update --preview`, whose end + opens the preview dialog. +- A real run ending, a failed or killed job: notification only. +- **Run it** in the Jobs ribbon, with a finished preview selected, reopens + that preview's dialog (also for a shadowing preview), so Not now loses + nothing. The Jobs top bar (Start, Apply shadowing, Run it, publish, Kill) + goes away. + +The hugo flow becomes: Hints, Check upstream -> Update selected -> diff +dialog, Build these -> preview dialog, Run it. No tab switch. + +## Jobs tab signal + +- Starting a job no longer switches to Jobs; the user stays on the tab. +- While a job runs, the Jobs tab shows a running icon (`system-run`) and + reads "Jobs (running)": the tab style sheet's text color would override a + per-tab color. +- When it ends, the icon becomes the end status (`✔` ok, `✘` failed, + killed, lost) and stays until the Jobs tab is shown, then clears. +- The status bar job button stays. + +## AGENTS.md: UI guidelines + +A new section, binding for every tab: + +1. Tabs are permanent; each loads lazily on first show. +2. Main ribbon: the tab's actions in workflow order. Small ribbon: view + controls. Nothing actionable lives inside a view except navigation links. +3. Change actions act on ticked rows, view actions on the highlighted row. + A row click toggles its tick; Ctrl+A / Ctrl+Shift+A select visible / + clear. +4. An inapplicable action is disabled with a tooltip saying why, never + hidden. +5. When a job this session started ends, its next step opens as a + non-modal dialog where the user is; the Jobs tab shows activity; a + dismissed step can be taken again from the ribbon. +6. A modal confirm only for destructive or outward steps (delete hint, + publish, apply shadowing). + +## Testing + +`test_gui.py` with the fake remote, offscreen: + +- ribbon contents per tab (action texts in order) after switching tabs; +- disabled states and tooltips (nothing ticked, no preview selected, no job + running); +- tabs are not closable and present from the start; data loaded on first + show only (no `inventory` call before Packages or Hints is shown); +- a preview this session started ends -> dialog -> Run it starts the real + run through the signing check; Not now, then Jobs' Run it reopens it; +- a preview not started by this session ends -> no dialog; +- Jobs tab icon running -> end status -> cleared on showing Jobs; +- New hint on two ticked packages -> `hint-new a b`. + +`test_remote.py`: `hint-new` with several packages, refusing a version and +a package that has a hint. |
