diff options
Diffstat (limited to 'docs/specs/2026-07-13-image-builder-design.md')
| -rw-r--r-- | docs/specs/2026-07-13-image-builder-design.md | 216 |
1 files changed, 216 insertions, 0 deletions
diff --git a/docs/specs/2026-07-13-image-builder-design.md b/docs/specs/2026-07-13-image-builder-design.md new file mode 100644 index 0000000..9ee1a11 --- /dev/null +++ b/docs/specs/2026-07-13-image-builder-design.md @@ -0,0 +1,216 @@ +# sbo-testbuild image builder — design + +Date: 2026-07-13 +Status: approved (design) + +## Purpose + +Produce and serve the docker images the `.extras/test-build` tool consumes. +`.extras/test-build` verifies a published SBo package still builds on a target +Slackware version inside a throwaway container; it does **not** build the +images, it only checks image presence by tag and errors if absent. This is the +missing image-builder job. + +Images produced (x86_64 only for now): + +- `sbo-testbuild:current` — Slackware -current, full install + sbopkg + + sbo-maintainer-tools +- `sbo-testbuild:15.0` — Slackware 15.0, same + +Served from a LAN docker registry so `.extras/test-build`'s `require_image` +can `docker pull` them, and so `installed_in_base` can later query the image's +package db. + +## Infrastructure + +Self-hosted on a Slackware VM (proxmox node), fully offline from the public +internet for the Slackware bits: + +- **Host**: `docker.noland.dnx` (LAN DNS), Slackware x86_64, 4 vCPU / 4 GB RAM + / 80 GB disk. +- **NAS trees**: two standard Slackware mirror trees NFS-mounted read-only: + - `/mnt/nas/slackware64-current` + - `/mnt/nas/slackware64-15.0` + Each is a full mirror tree (`PACKAGES.TXT`, `ChangeLog.txt`, `slackware64/` + series dirs, `patches/`, `extra/`). +- **Registry**: `registry:2` container on the VM, port 5000, plain HTTP (LAN + only). Pull path `docker.noland.dnx:5000/sbo-testbuild:{tag}`. +- **Local .txz drop**: `/opt/sbo-testbuild/pkgs/` holds `sbopkg-*.txz` and + `sbo-maintainer-tools-*.txz`, provided by the user (sbopkg packaged from the + user's SBo repo). Installed into the image, not built in it (hermetic, no + build-time deps). + +Both `docker.noland.dnx` and every client that pulls (this dev box, the +`buildsystem` VM) need the registry marked insecure in +`/etc/docker/daemon.json`: +`{"insecure-registries":["docker.noland.dnx:5000"]}`, then `docker` restarted. + +i586 is out of scope for now; the variant list is written to extend to it +later without restructuring. + +## Architecture + +Three scripts, chained, adapted from the forge repo +`slackware/docker-images` (`scripts/bootstrap.sh`, `build-full-image.sh`, +`build-builder-image.sh`). The forge `build-builder-image.sh` is **not** used: +the full image already ships the toolchain (series d, kde, x, etc.), so a +separate builder layer is redundant here. + +``` +bootstrap.sh FROM scratch; installpkg base pkgs read from NAS + → sbo-base:{ver} via file://; configure rootfs for container use. + +build-full-image.sh FROM sbo-base:{ver}; slackpkg install every series + → sbo-full:{ver} (a ap d e f k kde l n t tcl x xap xfce y). + +build-sbo-testbuild.sh FROM sbo-full:{ver}; installpkg sbopkg + + → sbo-testbuild:{ver} sbo-maintainer-tools from /opt/sbo-testbuild/pkgs. +``` + +All three push to `docker.noland.dnx:5000`. + +### Namespacing + +Forge scripts push to `registry.slackware.nl/slackware/slackware*`. Re-namespace +to the LAN registry with our own image names: + +- `docker.noland.dnx:5000/sbo-base:{ver}` +- `docker.noland.dnx:5000/sbo-full:{ver}` +- `docker.noland.dnx:5000/sbo-testbuild:{ver}` + +`sbo-base` and `sbo-full` are intermediate; `.extras/test-build` only pulls +`sbo-testbuild`. + +## Configuration + +Single shared file `config`, sourced by all scripts and the test: + +```sh +REGISTRY=docker.noland.dnx:5000 +MIRROR=file:///mnt/nas +VARIANTS=(current 15.0) # x86_64 only for now +PKGDIR=/opt/sbo-testbuild/pkgs # sbopkg + sbo-maintainer-tools .txz +``` + +The forge scripts hardcode `MIRROR=https://slackware.nl/slackware` and +`REGISTRY=registry.slackware.nl/...`; both move to the shared config. + +### NAS path mapping + +bootstrap derives the per-variant repo dir as `slackware64-${VERSION}` for +x86_64 (`DIRSUFFIX=64`). With `MIRROR=file:///mnt/nas` this resolves to +`file:///mnt/nas/slackware64-current/...` and `.../slackware64-15.0/...`, +matching the mount points. Mounts must be named to match. + +## Patches to the forge scripts + +1. **`file://`-aware fetch.** Forge downloads `PACKAGES.TXT` and packages with + `curl` over http. Replace with one helper in `lib.sh` that detects a + `file://` (or bare-local) MIRROR and uses `cp`, otherwise `curl`. curl's + own `file://` support is unreliable across builds, so branch explicitly. +2. **Drop i586.** `ALL_VARIANTS` becomes the config `VARIANTS` (x86_64 only). + All the `ARCH_SUFFIX`/`-i586` machinery collapses to the x86_64 path. +3. **Re-namespace + LAN registry** (see Namespacing). +4. **MIRROR → `file:///mnt/nas`** (see config). + +## Data flow and rebuild gating + +Nightly cron on the VM (root; `bootstrap` needs `installpkg`): + +```cron + 0 4 * * * bootstrap.sh +15 4 * * * build-full-image.sh +30 4 * * * build-sbo-testbuild.sh +``` + +Rebuild gating (kept from the forge scripts, extended): + +- **bootstrap**: sha256 of the NAS `ChangeLog.txt` (read via `file://`) + compared to a stored hash under `HASH_DIR`; skip the variant if unchanged. +- **build-full-image**: base image digest recorded as a label on the full + image; pull base, compare, skip if unchanged. +- **build-sbo-testbuild**: full image digest label, same pattern; **plus** a + hash of the `.txz` set in `PKGDIR`, so a sbopkg / sbo-maintainer-tools bump + forces a rebuild even when the full image is unchanged. + +Net effect: a quiet NAS tree makes the whole chain no-op in seconds; a +-current churn cascades base → full → testbuild. + +### Local-registry digest consistency + +The forge scripts read the base/full digest via `docker inspect RepoDigests` +after `docker pull`. Self-hosted, those intermediate images live in the LAN +registry, so pull-then-inspect must target `docker.noland.dnx:5000`. Since +every image name is derived from `REGISTRY`, re-namespacing handles this with +no extra logic. + +## Error handling + +- `set -euo pipefail` in every script (forge already has it). +- **Missing NAS mount**: guard `[[ -d /mnt/nas/slackware64-$ver ]]` first; + `_err` if absent. +- **Missing `.txz`**: `build-sbo-testbuild` errors before building if + `PKGDIR` lacks a `sbopkg-*.txz` or `sbo-maintainer-tools-*.txz` — never push + a testbuild image missing sbopkg. +- **Registry unreachable**: `docker push` fails under `set -e`; the chain + aborts. The base/full may already be built locally; cron retries next night. + Acceptable. +- **Per-variant isolation**: one variant failing (e.g. current) must not block + the other (15.0). Each variant runs wrapped; log the failure, continue, and + exit non-zero at the end so cron logs surface it. + +## Testing + +Pure-logic pieces get a `test-image-builder.sh` self-check, matching the +existing `.extras/test-build` pattern (`.extras/test-logic.sh`, assert-based, +no framework). No docker daemon in tests — hermetic. + +- **`file://` fetch helper**: local file resolves via cp; missing local file + errors; an http MIRROR still routes to curl. +- **`.txz`-set hash**: stable across file reordering; changes when a file's + content changes. +- **variant guard**: a missing mount dir yields non-zero. + +The real end-to-end chain is verified by a manual first run on the VM (the user +runs it and reports back), the same division of labor as `.extras/test-build` +Task 10. + +## Layout + +``` +.extras/image-builder/ +├── config REGISTRY, MIRROR, VARIANTS, PKGDIR +├── lib.sh shared: _log/_warn/_err, fetch, txz_hash, guards +├── bootstrap.sh base image (root; installpkg from NAS) +├── build-full-image.sh full image (slackpkg all series) +├── build-sbo-testbuild.sh testbuild image (installpkg sbopkg + tools) +├── test-image-builder.sh pure-logic self-check +└── README VM-side setup, reproducible +``` + +Lives in `.extras/` for now (per the user), relocatable later. + +The README carries the user-side checklist so it is reproducible and not +trapped in chat: + +- VM specs (4/4/80), Slackware x86_64, docker installed. +- Two NFS mounts (ro), root-readable, named `slackware64-current` / + `slackware64-15.0` under `/mnt/nas`. +- `registry:2` container: `-v /opt/registry/data:/var/lib/registry`, + `--restart=always`, port 5000. +- `/etc/docker/daemon.json` insecure-registry entry on the VM **and** on every + pulling client (this dev box, `buildsystem`), docker restarted. +- Drop `sbopkg-*.txz` and `sbo-maintainer-tools-*.txz` into + `/opt/sbo-testbuild/pkgs/`. +- Install the three cron lines. +- VM static IP / LAN DNS for `docker.noland.dnx`. + +## Out of scope + +- i586 images (extensible later via `VARIANTS` / arch machinery). +- Building sbopkg / sbo-maintainer-tools from source in the image (installed + from prebuilt `.txz`). +- The forge `build-builder-image.sh` toolchain layer (redundant with full). +- `installed_in_base` implementation in `.extras/test-build` — unblocked by + this work (queries the built image's package db) but tracked separately. +- Registry auth / TLS (LAN-only plain HTTP). |