diff options
| author | Danilo M. <danix@danix.xyz> | 2026-07-13 09:56:51 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-07-13 09:56:51 +0200 |
| commit | b4c286efff30dc6667fc49d5f7895ae6af8c296c (patch) | |
| tree | 8e982c0d43a3c1b8a11f6167d4c7052c8bfae2df | |
| parent | fa86f5b48d09f7367ae20434bf8d007354496e14 (diff) | |
| download | sbo-slackbuilds-b4c286efff30dc6667fc49d5f7895ae6af8c296c.tar.gz sbo-slackbuilds-b4c286efff30dc6667fc49d5f7895ae6af8c296c.zip | |
docs: test-build design spec; refine CLAUDE.md purpose and docker workflow
| -rw-r--r-- | .extras/docs/specs/2026-07-13-docker-test-build-design.md | 258 | ||||
| -rw-r--r-- | CLAUDE.md | 94 |
2 files changed, 340 insertions, 12 deletions
diff --git a/.extras/docs/specs/2026-07-13-docker-test-build-design.md b/.extras/docs/specs/2026-07-13-docker-test-build-design.md new file mode 100644 index 0000000..9f14d1e --- /dev/null +++ b/.extras/docs/specs/2026-07-13-docker-test-build-design.md @@ -0,0 +1,258 @@ +# Docker test-build script - design + +Date: 2026-07-13 + +## Purpose + +A single bash script that verifies an already-published SBo package still +builds cleanly on a target Slackware version, inside a throwaway docker +container. This is the "test-build" step of this repo's maintenance loop +(track upstream -> bump -> **test-build** -> make tarball). + +The container is disposable, so unlike a real `slackrepo build` this runs +directly (no host install), and Claude may run it (per the repo's +`Who builds SlackBuilds` exception). + +## Scope and non-goals + +- One target package per run: `test-build <pkg>`. +- Two Slackware targets: `-current` (default) and `15.0` (`--stable`). +- Resolves the target's SBo dependency tree from the LOCAL SBo tree, builds + deps then target in the container, reports, discards everything. +- NOT a redistributable-package builder. The container's built `.txz` is + throwaway; the SBo submission tarball is made separately (post-commit hook) + only after a green run. +- Does NOT wrap slackrepo or sbo-batch-test. It REUSES sbo-batch-test's + resolver + build logic, adapted, but is an independent script. +- Shares the host kernel (docker). No kernel-module build claims. + +## Why not the overlay approach (sbo-batch-test) + +`sbo-batch-test` already resolves deps + builds + reports, but against a +LOCAL overlayfs over a 15.0 base. In docker the container IS the disposable +base, and overlayfs-over-docker-storage (itself overlayfs) is fragile. So we +drop the overlay layer and build directly in the container. We port +sbo-batch-test's dep resolver and its `build_one` chroot heredoc +(download -> md5 -> build -> installpkg -> status token) into a `docker run`. + +## Architecture + +Single self-contained bash script (`test-build`, location TBD in +implementation - likely `.extras/`). Host does resolution + orchestration; +the container does the actual building. + +``` +host: parse args, load config +host: pick version -> image tag + SBo tree +host: resolve dep tree (local tree, topo sort) [ported from sbo-batch-test] +host: apply current-vs-stable overrides +host: print final build order, Y/n confirm (--yes: still prints first) + docker run --rm -v <tree>:ro -v <pkg>:ro -v <cache/digest>:ro <image>: + for each dep in order: + cache hit (version match) -> installpkg from cache (CACHED) + else -> download/md5/build/installpkg + cache + target: download/md5/build/installpkg (always fresh) + target: sbopkglint on the built .txz +host: collect per-package results, print color summary +container discarded (--rm) +``` + +### Images (external input, not built by this script) + +The script does NOT build images. It consumes a ready image per version, +tagged locally: + +- current: `sbo-testbuild:current` +- 15.0: `sbo-testbuild:15.0` + +Each image is a FULL, patched Slackware install (SBo tests against a full +install; a minimal base causes false "missing dependency" results, a lesson +from sbo-batch-tester) plus `sbo-maintainer-tools` (for `sbopkglint`) and +`sbopkg` configured to that version's SBo repo (`:current` -> -current repo, +`:15.0` -> 15.0 repo). + +**How the images are produced is out of scope for this script.** A separate +nightly job (a LAN machine, images stored on the NAS, later served from a +local registry) builds and refreshes them. That builder is its own tool, +specced separately. + +For now the script assumes the tagged image is present locally: it runs +`docker run` against the tag and errors with a clear message if the image is +missing (pointing at the image-builder job). When the local registry exists, +a `docker pull` of the tag slots in ahead of the run via the same config, +no other change. + +Per-run cost is just the SlackBuild's own build time + ~1s container start. + +### Config (external file) + +`~/.config/sbo-testbuild/config`, empty defaults in-script, sourced if +present (same pattern as sbo-batch-test). Keys: + +```sh +SBO_TREE_CURRENT=/path/to/SBo-current # local SBo tree, -current +SBO_TREE_STABLE=/path/to/SBo-15.0 # local SBo tree, 15.0 +IMAGE_CURRENT=sbo-testbuild:current # ready image tag (built elsewhere) +IMAGE_STABLE=sbo-testbuild:15.0 +LOG_ROOT=/path/to/logs +PKG_CACHE=/path/to/cache # dep cache; empty = disabled +``` + +The version flag selects BOTH the image and the tree together (default +current; `--stable` / `15.0` switches both), so tree and image never mismatch. + +### current-vs-stable overrides + +A data file next to the config, `~/.config/sbo-testbuild/overrides`. It +encodes the known deltas between building on 15.0 and on -current. Applied +ONLY when the target version is `-current` (15.0 is the SBo baseline). Three +rule kinds: + +``` +# dep present in 15.0 but already in -current base -> drop from order +drop: rust +# dep renamed on -current -> map old -> new before lookup +rename: python3-foo -> foo +# dep removed from the -current tree -> fetch from SBo instead of local tree +fetch: somelib +``` + +Format: one rule per line, `kind: value` (rename uses `old -> new`), `#` +comments and blank lines ignored. + +### Dependency resolution + unknown deps + +Resolver is ported from sbo-batch-test (`resolve_target` / `_resolve_visit`, +DFS topo sort + cycle detection, `installed_in_base` check). Runs on the host +against the selected tree. + +After resolving and applying overrides, every dep must be accounted for: +in base, in the tree, or covered by a `fetch:` rule. If a dep is none of +those (and no rule covers it), the script **stops before building**, reports +it as UNMET, and asks the user to add an override rule and rerun, or abort. +Never silently skip or guess. + +### Confirmation + +Always print the final build order before building, with overrides marked +(dropped / renamed / fetch). Then `Y/n` to proceed. `--yes` skips the prompt +for non-interactive/agent runs but STILL prints the order first. + +`--dry-run` resolves, applies overrides, prints the order, and exits without +building (no confirm). + +### Build in container (ported from build_one) + +Deps then target, in order. For each package, inside the container: + +1. download sources (arch-specific `DOWNLOAD_x86_64`/`MD5SUM_x86_64` when + present, else `DOWNLOAD`/`MD5SUM`) +2. verify md5 +3. `bash <pkg>.SlackBuild` with `OUTPUT` forced to a known dir +4. `installpkg` the resulting `.txz` +5. write a status token read back by the host + +`fetch:` deps (removed from the -current tree) are pulled via the baked-in +`sbopkg`, which points at the image's matching SBo repo; everything else +builds from the mounted local tree. Start light: handle only the few known +`fetch:` cases, add rules as real packages need them. + +### Dependency cache + +Ported from sbo-batch-tester (`cache_decision`/`cache_path`/`cache_store`/ +`version_of`). A host dir caches built dep packages so consecutive runs reuse +them instead of rebuilding the whole tree. + +- Config `PKG_CACHE=/path` enables it; empty (default) disables. `--no-cache` + forces a full rebuild for one run. +- Bind-mounted read-only into the container; a cached dep is `installpkg`ed + instead of built (status `CACHED`). Storing is host-side: a freshly built + package is written to `OUTPUT` (a mounted dir the host reads back), and the + host copies it into the cache after the build (`cache_store`), so the cache + mount stays read-only. The **target always builds fresh** and refreshes its + own cache entry; it is never `CACHED`. +- Key = prog + version (build/arch/tag ignored). Layout mirrors the SBo tree: + `$PKG_CACHE/<image-digest>/<category>/<prog>/<prog>-<ver>-...txz`, one .txz + per prog. +- **Invalidation by image digest.** The cache is namespaced by the running + image's digest. When the digest changes (image updated by the nightly job), + the old namespace is stale: its deps were built against the previous image, + so they are ignored and rebuilt once under the new digest. This also keeps + the `:current` and `:15.0` caches separate (different images, different + digests). Old digest namespaces can be pruned. +- A per-dep version bump (`bump: OLD -> NEW`) rebuilds and re-caches that dep. +- Build-order line shows the outcome per dep: `cached (1.1)`, + `rebuild: 1.0 -> 1.1`, or `build (new)`; `--dry-run` shows the same without + building. + +### Target lint + +After the target builds, run `sbopkglint` on its `.txz` (tool baked into the +image). Fail-soft: findings reported but do not change SUCCESS; missing tool +prints a skip note. Target only (deps are vetted). + +### Report + +Per-package status + a color summary (screen) and a plain-text log under +`LOG_ROOT/<timestamp>/`: + +``` +<timestamp>/ + <prog>.log per-package full build/install output + summary.log plain recap + build-order.txt the resolved+overridden order actually used +``` + +Status values (from sbo-batch-test): +`SUCCESS CACHED DOWNLOAD-FAILED MD5-MISMATCH BUILD-FAILED INSTALL-FAILED +BLOCKED-BY-DEP UNMET-DEP`. `CACHED` = a dep installed from the cache instead +of rebuilt (target is never CACHED). `%README%` deps flagged as a reminder, +not built. + +A fully green run tells the user it is safe to make the SBo submission tarball +on the host (the container is no longer needed). + +## Options (planned) + +| Option | Effect | +|--------|--------| +| `--stable` / `15.0` | Target 15.0 (image + tree). Default is -current. | +| `--dry-run` | Resolve + apply overrides + print order, no build. | +| `--yes` | Skip the Y/n confirm (still prints the order). | +| `--no-cache` | Rebuild all deps this run, ignore/refresh the cache. | +| `--no-color` | Disable ANSI (auto-off when not a TTY). | +| `-h`, `--help` | Usage. | + +## Self-check + +A `test-logic.sh` (host, no docker) covering the pure logic: + +- dep topo order + cycle detection (ported tests from sbo-batch-test) +- override application: drop removes, rename maps, fetch marks source +- unknown-dep -> UNMET stop +- BLOCKED-BY-DEP propagation +- cache decision: `cached`/`rebuild: OLD -> NEW`/`build (new)`, digest + namespacing, version-bump eviction (ported from sbo-batch-tester's tests) + +Docker/build/installpkg paths are out of the self-check's reach (verified by +running a real target), same boundary as sbo-batch-test. + +## Resolved decisions + +- **Image**: a ready FULL-install image per version (full pkg set + + `sbo-maintainer-tools` + `sbopkg` on that version's repo), built by a + separate nightly job, NOT by this script. Script consumes it by tag. +- **Missing image**: error clearly, point at the image-builder job. A local + registry `docker pull` slots in later via the same tag, no script change. +- **`fetch:`**: via the image's `sbopkg` (per-version repo). Start with only + the few known cases; grow the override rules as needed. +- **Script location**: `.extras/` (a tool, not a package). + +## Out of scope / later + +- The nightly image-builder (full-install + tools + sbopkg per version, NAS + storage, local registry). Its own tool, specced when the LAN machine is set + up. +- Local registry pull step in this script (slots into the same config tag). +``` + @@ -1,10 +1,27 @@ # SBo SlackBuilds Repository -SlackBuild scripts maintained by Danilo (danix) and submitted to -[SlackBuilds.org (SBo)](https://slackbuilds.org). This repo holds ONLY the -packages for which danix is the **official SBo maintainer**. Lead and -experimental packages not on SBo under danix live in the separate -`my-slackbuilds` repository. +SlackBuild scripts for which danix is the **official SBo maintainer** on +[SlackBuilds.org (SBo)](https://slackbuilds.org). + +## Purpose: tracking, not development + +This repo is a **maintenance mirror of already-published SBo packages**. No new +packages are developed here. Its job is the ongoing upkeep of packages already +on SBo: + +1. Track upstream version changes (via nvchecker). +2. Reflect a new upstream version in the package's `.SlackBuild` / `.info`. +3. Test-build in a throwaway docker image (see below). +4. Generate the SBo submission tarball to upload. + +New-package development happens in **other** repos, not here: + +- `../GITHUB/my-slackbuilds/` — regular packages, plus SBo packages whose + upstream moves faster than SBo review (worked ahead of the SBo copy). +- `../GITHUB/Slackware-Pentesting-Suite/` — cybersecurity packages. + +When a new package is uploaded to SBo and approved, it **moves here** and this +repo takes over its maintenance. --- @@ -31,7 +48,9 @@ Version tracking is handled by a single file: ### `.extras/` holds all non-package repo files Anything that is not a SlackBuild package must live under `.extras/`, never at -the repo root. +the repo root. Exception: a top-level `docs/` for design docs and specs is fine +in this repo (the subtree-leak concern that bans it applies to the development +repos, not here). Existing specs live under `.extras/docs/specs/`. --- @@ -122,9 +141,52 @@ compiled binaries. No stripping needed. --- +## Test-building in docker + +Because these packages are already published on SBo, the build here is a +**verification step** on a version bump, not development. It runs in a +throwaway Slackware container, so it is hermetic and disposable: unlike a real +`slackrepo build` on the buildsystem VM, **Claude may run it directly**. + +Images (Slackware official, pull fresh each time — they are throwaway): + +- `-current`: `registry.slackware.nl/slackware/slackware-builder:current` +- `15.0` (stable): `registry.slackware.nl/slackware/slackware-builder:15.0` + +Mount the package dir into a `--rm` container and run its SlackBuild. Anything +built is discarded with the container; nothing lands in the working tree. + +No wrapper script exists yet — write the `docker run` invocation inline, or add +one under `.extras/` if this gets repetitive. The build's only outputs that +matter are the exit status and lint result; the package tarball it produces is +throwaway. + +The one artifact worth keeping from a clean bump is the **SBo submission +tarball** (`SBo/<pkg>.tar.gz`), produced by the `post-commit` hook — that is +what gets uploaded to SlackBuilds.org. + +--- + ## SlackBuild Scripting Guidelines -- Follow the [SBo template](https://slackbuilds.org/templates/) as the base for all scripts +### Templates (local clone) + +A clone of the SBo templates lives at `~/Templates/SlackBuilds/sbo/` (refreshed +by `~/Templates/SlackBuilds/update-templates.sh`). **Start from the matching +file there** rather than from memory: + +- Per-build-type SlackBuilds: `autotools-`, `cmake-`, `meson-`, `python-`, + `perl-`, `rubygem-`, `haskell-template.SlackBuild`. +- Shared support files: `slack-desc`, `template.info`, `doinst.sh`, + `douninst.sh`. + +The [online SBo template](https://slackbuilds.org/templates/) remains the +authority; the local clone mirrors it. Refresh the clone if it looks stale +before relying on it. + +### General + +- Follow the SBo template (local clone above) as the base for all scripts - Use `set -e` to abort on errors - Honor `$TMP`, `$BUILD`, `$TAG`, `$OUTPUT` variables; default values must be set if unset - Use `$ARCH` detection with proper `SLKCFLAGS` and `LIBDIRSUFFIX` @@ -215,11 +277,19 @@ be committed. ## Who builds SlackBuilds -Only the user builds SlackBuilds, on their own infra. Claude authors/edits the -sources (`.SlackBuild`, `.info`, `README`, `slack-desc`, `doinst.sh`, nvchecker -stanzas) and runs only the non-building checks (`sbolint`, `sbodl`, -`sbofixinfo`); the user builds and reports back. Never run or offer -`bash <pkg>.SlackBuild`, `sbopkglint`, or `slackrepo build`. +Claude authors/edits the sources (`.SlackBuild`, `.info`, `README`, +`slack-desc`, `doinst.sh`, nvchecker stanzas) and runs the non-building checks +(`sbolint`, `sbodl`, `sbofixinfo`). + +**The one exception in this repo** is the throwaway-docker test-build (see +[Test-building in docker](#test-building-in-docker)): it is hermetic and +disposable, so Claude may run it to verify a bump, including `sbopkglint` on +the throwaway package it produces. + +Everything else stays with the user: never run or offer a **real**, persistent +build — `slackrepo build`, or `bash <pkg>.SlackBuild` / `sudo bash ...` on the +host (as opposed to inside the throwaway container). Those install to the +user's system or the buildsystem VM; the user runs them and reports back. --- |
