diff options
Diffstat (limited to 'docs/design/2026-10-19-sb-builder-phase1-design.md')
| -rw-r--r-- | docs/design/2026-10-19-sb-builder-phase1-design.md | 335 |
1 files changed, 335 insertions, 0 deletions
diff --git a/docs/design/2026-10-19-sb-builder-phase1-design.md b/docs/design/2026-10-19-sb-builder-phase1-design.md new file mode 100644 index 0000000..3ec3fc0 --- /dev/null +++ b/docs/design/2026-10-19-sb-builder-phase1-design.md @@ -0,0 +1,335 @@ +# sb-builder phase 1: core design + +Date: 2026-10-19 +Status: approved + +## Goal + +One Qt6 desktop GUI to drive the SlackBuild tooling on the build VM +(`buildsystem`): slackrepo, mkhint, overlay sync (sbpull, sbshadow), full repo +rebuild, and SBo orphan discovery. Long-running work runs as jobs on the VM +that survive the GUI disconnecting, and the GUI can reattach to them later. + +## Phases + +Each phase gets its own spec, plan and implementation cycle. + +1. **Core** (this spec): remote helper with the job model, GUI shell with + tabs, Dashboard, Jobs tab, VM control, config, notifications. Job kinds: + `sbpull`, `sbshadow`. +2. **Packages + slackrepo**: package browser over the large SBo-danix tree + (search and filter by name, category, overlay, maintainer, build status); + Build, Rebuild, Bulk update screens. +3. **Full repo rebuild**: `slackrepo_setup_SBo-danix.sh`, destructive, guarded. +4. **Hints**: mkhint browse, edit, update, create, `-F`, diff before save. + Requires non-interactive flags in mkhint (it has about 10 `read -p` + prompts and wraps slackrepo at `mkhint:1798`). Also: `--no-dl` must write + `DOWNLOAD="no"`, which slackrepo reads, instead of `NODOWNLOAD=yes`, + which it does not (keep reading old `NODOWNLOAD` lines for the NoDL column), + and existing hints on the VM get migrated: every `NODOWNLOAD=yes` line in + any hint slackrepo searches becomes `DOWNLOAD="no"` (an existing `DOWNLOAD` + line is replaced, since `DOWNLOAD="no"` overrides it anyway), backed up to + `.bak` first, idempotent. As of 2026-10-19 no hint carries the line. + Superseded by `2026-10-20-sb-builder-phase4-design.md`: the hint logic + moves into the helper instead of extending mkhint. +5. **Orphans**: SBo orphan discovery (`EMAIL="NOEMAIL"` in `.info`, the + `sbo-orphans` logic from slackbuilds-superrepo), including orphaned deps + in REQUIRES. Needs an SBo-stable tree, which is not on the VM today. + +Dashboard tiles become entry points as their screens land: Packages opens +the package browser (phase 2, select packages to build there), Built the +build screen, Last pull the sync actions, Recent jobs the Jobs tab, SBo +orphans the orphans screen (phase 5). + +Out of scope for all phases so far: overlay git operations (status, commit, +push in my-slackbuilds and friends). + +## Stack + +- GUI: Python 3 + PyQt6 (stock in Slackware -current, 6.11), including + `PyQt6.QtDBus` for notifications. No third-party packages. +- Helper: Python 3 stdlib only (VM runs 3.12, so `tomllib` is available). +- Config: TOML, read with `tomllib`. +- License: GPLv2-only. + +## Architecture + +``` +desktop buildsystem (root) ++----------------------+ ssh (ControlMaster) +---------------------------+ +| sb-builder (PyQt6) | ----------------------> | sbb-remote (py3 stdlib) | +| tabs: Dashboard, | sbb-remote <cmd> | stats / jobs / | +| Jobs (+later tabs) | <---------------------- | job-start / job-follow / | +| QProcess per call | JSON on stdout | job-kill, flock, setsid | ++----------------------+ | /var/lib/sb-builder/ | + | +---------------------------+ + | virsh (local libvirt) + v + domain "buildsystem" +``` + +### Repository layout + +``` +sb_builder/__main__.py entry point, config load +sb_builder/config.py GUI config: defaults, load, validate +sb_builder/remote.py the only code that runs ssh; also virsh +sb_builder/mainwindow.py tabs, menu, status bar, notifications +sb_builder/dashboard.py Dashboard tab +sb_builder/jobs.py Jobs tab +sb_builder/fmt.py text formatting shared by the tabs +sb-builder launcher (python3 -m sb_builder) +remote/sbb-remote VM helper, single file +config.example.toml GUI config example +remote/config.example.toml helper config example +install.sh local install; --remote deploys helper to VM +tests/test_remote.py +tests/test_gui.py +``` + +### Transport + +Every GUI call is a QProcess running: + +``` +ssh -o ControlMaster=auto -o ControlPath=<runtime dir>/sbb-%C -o ControlPersist=10m \ + <ssh_options...> <target> sbb-remote <cmd> [args...] +``` + +Calls are async. ssh exit 255 means the connection failed. Exit 127 means the +helper is not installed. + +### Trust boundary + +The helper runs only a fixed table of job kinds. The GUI sends a kind name +plus arguments; the helper validates each argument and builds argv itself. +No shell strings and no paths cross ssh. Validation rules: + +- unknown kind: rejected +- `sbpull` target: must be listed in the helper config `overlays` +- flags: only the ones the kind declares (`--yes`, `--no`) +- package names (phase 2 onward): `^[A-Za-z0-9._+-]+$` + +### Version check + +On connect the GUI runs `sbb-remote version`, which also returns the +configured `overlays` (the GUI builds its sbpull menu from them). A mismatch with the GUI's own +version shows a banner telling the user to run `./install.sh --remote`. + +## Helper: `sbb-remote` + +### State + +``` +<state_dir>/lock flock target, held for the whole job +<state_dir>/jobs/<id>/meta.json kind, args, argv, pid, start, end, exit, status +<state_dir>/jobs/<id>/log combined stdout+stderr, raw bytes +``` + +`<id>` is `YYYYMMDD-HHMMSS-<kind>`. `status` is one of `running`, `ok`, +`failed`, `killed`, `lost`, `busy`. + +Logs are kept forever (`# ponytail:` comment in code; add pruning once disk +usage is visible on the Dashboard). + +### Commands + +All print one JSON object on stdout, except `job-follow`. + +| command | behaviour | +|---|---| +| `version` | `{"version": "...", "overlays": [...]}` | +| `stats` | Dashboard data: load, uptime, disk free for repo and state dirs, package counts (total, personal, pentesting, shadowed = overlay packages also present in the pristine `current` branch, as sbshadow computes it), last sync per overlay (time of the newest commit touching `<overlay>/` in `repo_dir`), slackrepo summary from its sqlite DB (built, failed, never built). Any missing source yields `null` for that field, never an error | +| `jobs` | list of all jobs' meta. A job whose meta says `running` but whose pid is gone (`os.kill(pid, 0)` fails) is reported and rewritten as `lost` | +| `job-start <kind> [args]` | validate; spawn `sbb-remote _run <id>` with `start_new_session=True`, `stdin=/dev/null`, stdout/stderr to `/dev/null`; wait up to 3s for `_run` to write status; return `{"id": ...}`, or `{"busy": "<running id>"}` after deleting the refused job's dir | +| `job-follow <id> <offset>` | write log bytes from `offset` to stdout, keep following until the job is no longer `running`, then exit 0 | +| `job-kill <id>` | SIGTERM to the job's process group; `_run` records `killed` | +| `_run <id>` | internal, see below | + +### `_run` + +1. Take a non-blocking `fcntl.flock` on `<state_dir>/lock`. If held, write + status `busy` and exit. +2. Start the job argv with `stdin=/dev/null`, stdout and stderr to the log + file, in its own process group. Write `pid` and status `running`. +3. Wait. Write `end`, `exit` and final status (`ok` on 0, `killed` on + SIGTERM, else `failed`). + +Jobs never read input. A tool that prompts anyway gets EOF on `read` and, +under `set -e`, exits non-zero: the job fails, it never hangs and it never +answers yes by accident. Destructive steps are confirmed in the GUI. + +### Phase 1 job kinds + +| kind | argv built by helper | +|---|---| +| `sbpull <overlay> [--yes\|--no]` | `bash -c '. "$1" && sbpull "$2" "$3"' _ <sbpull_script> <overlay> <flag>` | +| `sbshadow [--yes\|--no]` | `sbshadow <flag>` | + +The flag is required in both kinds (a job has no stdin). + +### Helper config: `/etc/sb-builder/config.toml` + +```toml +state_dir = "/var/lib/sb-builder" +repo_dir = "/var/lib/sbopkg/SBo-danix" +upstream_branch = "current" +slackrepo_db = "/var/lib/slackrepo/SBo-danix/database_SBo-danix_x86_64.sqlite3" +sbpull_script = "/usr/local/share/slackrepo_setup/sbpull.bash" +overlays = ["personal", "pentesting"] +``` + +Missing file: these defaults. Unknown key or wrong type: every command +returns `{"error": "config: <key>: <reason>"}` and exits 1. + +### Errors + +Top-level handler catches any exception, prints `{"error": "<message>"}`, +exits 1. + +## Changes in slackrepo_setup + +Done in that repo, its own commit, redeployed to the VM. + +- `sbshadow --yes | --no`: no flag keeps the current interactive prompt. + `--yes` applies without asking. `--no` prints the plan and leaves the tree + unchanged (preview). `mkhint -F` runs in all cases, as it does today on + abort. +- `sbshadow` honours `SBO_REPO` to override the repo path, so it can be + tested against a throwaway repo. +- `sbpull <target> [--yes|--no]`: passes the flag to `sbshadow`, and returns + `sbshadow`'s exit status (today it returns `popd`'s, so a failure would + show as a successful job). + +GUI flow: `sbpull personal --no` pulls and shows the shadow plan in the log; +the Jobs tab offers "Apply shadowing", which starts `sbshadow --yes`. + +## GUI + +### Main window + +- QTabWidget. Dashboard is pinned. Other tabs are closable and + single-instance. +- Menu: `File > Reload config`, `File > Quit`, `Open > Jobs` (later phases + add entries). +- Status bar: connection state (ok, down, VM off, VM paused, VM running but + ssh unreachable, helper version mismatch), running job if any (click opens + it in Jobs), transient messages. + +### Dashboard + +Grid of QGroupBox tiles filled from `stats`, plus `jobs` for the Jobs tile: + +- VM: load, uptime, disk free +- Repo: total packages, personal, pentesting, shadowed +- slackrepo: built, failed, never built +- Sync: last pull per overlay +- Jobs: running job and last 10 (status, duration) +- Orphans: placeholder tile, filled in phase 5 + +Refreshes every `refresh.dashboard` seconds while visible, plus a Refresh +button. `null` fields show "n/a". When the connection is down the last +values stay, greyed, with "stale since HH:MM". + +### Jobs tab + +- Left: job list (status icon, kind and args, start, duration), refreshed + every `refresh.jobs` seconds. +- Right: log view (QPlainTextEdit, ANSI escapes stripped, `\r` rewrites the + current line), Kill button with confirm. +- Toolbar `Start`: one entry per overlay for `sbpull <overlay> --no`, plus + `sbshadow --no`, `sbshadow --yes` (confirm dialog). +- After a `--no` run whose log lists `shadow:` or `unshadow:` lines, an + "Apply shadowing" button starts `sbshadow --yes`. +- Selecting a job starts `job-follow` from offset 0. If the stream drops + while the job is still running, the next successful `jobs` poll restarts + it from the last received offset (the poll itself backs off while the VM + is down, see Error handling). +- `job-start` returning busy: dialog "busy: <id>" with [Show job]. + +### Notifications + +freedesktop `org.freedesktop.Notifications.Notify` via QtDBus. Events: +`finished`, `failed`, `killed`, `lost`, and `vm-up` (VM started and ssh +answering). Clicking a notification (`ActionInvoked`) raises the window and +opens the job. The GUI tracks the job states it has seen, so each job end +notifies once. + +### VM control + +When ssh fails and `vm.domain` is set, the GUI runs +`virsh -c <libvirt_uri> domstate <domain>` locally: + +| domstate | behaviour | +|---|---| +| `shut off` | status "VM off", [Start VM] (automatic when `vm.autostart`): `virsh start`, then poll ssh with backoff, notify `vm-up` | +| `paused` | status "VM paused", [Resume]: `virsh resume` | +| `running` | status "VM running, ssh unreachable", keep retrying | +| virsh missing or error | plain "down" | + +No shutdown control. + +### GUI config: `$XDG_CONFIG_HOME/sb-builder/config.toml` + +```toml +[remote] +target = "buildsystem" # any ssh destination +ssh_options = [] # extra ssh args; ControlMaster options always added + +[refresh] +dashboard = 30 +jobs = 5 + +[notify] +enabled = true +events = ["finished", "failed", "killed", "lost", "vm-up"] + +[vm] +libvirt_uri = "qemu:///system" +domain = "buildsystem" # empty disables VM control +autostart = false +``` + +Missing file: these defaults. Invalid key or type: dialog naming the key, +then defaults are used. Read at startup and on `File > Reload config`. No +settings dialog yet. + +## Error handling summary + +| case | behaviour | +|---|---| +| ssh down | status bar, VM check, stale tiles; the `jobs` poll interval doubles from `refresh.jobs` up to 30s, and resets once ssh answers | +| helper missing or version mismatch | banner with `./install.sh --remote` | +| busy | dialog with [Show job] | +| follow drop | reconnect from offset, job unaffected | +| helper exception | `{"error"}` shown in status bar | +| `_run` died, VM rebooted mid-job | `lost`, red in list, notification | +| config invalid | dialog naming key, defaults used | + +## Testing + +stdlib `unittest`, no frameworks. + +- `tests/test_remote.py`: imports `remote/sbb-remote` as a module with a + temp `state_dir`. Covers argument validation (unknown kind, overlay not in + config, unknown flag), lock busy, `lost` detection, `job-follow` from an + offset, and a full `_run` of a fake kind patched into the kind table + (ok, failed, killed). +- `tests/test_gui.py`: config load and validation, `remote.parse`, GUI and + helper versions equal, both example configs equal the defaults. Smoke tests + under `QT_QPA_PLATFORM=offscreen` with a fake Remote: tabs, Dashboard from + a canned `stats` dict with `null` fields, job-end notification sent once, + VM-off offer, carriage-return handling in the log view. +- slackrepo_setup: sbshadow `--yes` and `--no` verified in a throwaway git + repo, the same way the current behaviour was tested. +- First run against the real VM: manual, with the user. + +## Install + +- `install.sh`: symlinks the `sb-builder` launcher into `~/bin` (`BINDIR` + overrides), so repo edits are live; copies `config.example.toml` to the + config path only if no config exists. +- `install.sh --remote`: scp `remote/sbb-remote` to `<target>:/usr/local/bin/` + (`SBB_TARGET`, default `buildsystem`), + create `/etc/sb-builder/config.toml` from the example only if missing. + Asks before deploying. A SlackBuild comes later. |