diff options
| -rw-r--r-- | CLAUDE.md | 129 | ||||
| -rw-r--r-- | README.md | 161 | ||||
| -rw-r--r-- | claude-code-prompt-sbo-batch-tester.txt | 289 | ||||
| -rw-r--r-- | overlay-chroot.sh | 257 | ||||
| -rwxr-xr-x | sbo-batch-test | 668 | ||||
| -rwxr-xr-x | test-logic.sh | 111 |
6 files changed, 1615 insertions, 0 deletions
diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..8484a50 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,129 @@ +# CLAUDE.md + +Working notes for Claude Code on this project. Read before editing. + +## What this is + +`sbo-batch-test`: a single self-contained bash script that batch-tests +SlackBuilds against a clean Slackware 15.0 overlay chroot. The user is an SBo +(SlackBuilds.org) maintainer on Slackware64-current who needs to verify builds +against 15.0 stable without booting a VM. + +Full spec lives in `claude-code-prompt-sbo-batch-tester.txt`. Reference mount +machinery in `overlay-chroot.sh` (Jeremy Hansen / bassmadrigal). User-facing +docs in `README.md`. + +## Conventions (the user cares about these) + +- **No em dash character** in any prose, comments, README, or commit messages. + Use commas or periods. This is a hard rule. +- This session runs with ponytail (laziest working solution) and caveman + (terse replies) modes. Code stays normal; chat is terse. +- Build incrementally. The user wants to review the dep resolver and the + mount/teardown core before trusting the rest. Do not balloon scope. +- Do NOT default to agreement. If a choice is fragile or wrong, say so and + propose better with reasoning. + +## Hard constraints (do not violate) + +- **Resolution is LOCAL-tree-only.** Never add network/sbopkg resolution. The + local SBo tree is the single source of truth (contains unpublished + personal/pentesting packages). Settled, not open for revisiting. +- **overlay lowerdir must be LOCAL.** `SLACKWARE_BASE` is a local fs path. + `LOCAL_MIRROR_15` is NFS and is a package SOURCE only. overlayfs over NFS + lowerdir is fragile. The script guards against pointing base under the mirror. +- **Do not wrap/drive slackrepo.** Separate pipeline, keep independent. +- **Shares host kernel.** No kernel-module testing claims. +- Runs as root only (overlay + chroot). +- Per-target disposable overlay: each category target tests against pristine + 15.0. Built packages are throwaway. + +## Architecture (single file: sbo-batch-test) + +Top-to-bottom layout: + +1. **CONFIG block** - `SLACKWARE_BASE`, `LOCAL_MIRROR_15`, `SBO_TREE_ROOTS[]`, + `CHROOT_LOCATION`, `LOG_ROOT`, `VERSION`. Currently placeholder paths (we are + not in the VM, no real values yet). User fills these in. +2. **Globals / flags** - `USE_COLOR DRY_RUN WITH_X JOBS`, status maps + (`ST_STATUS ST_REASON ST_TIME ST_README`), `ACTIVE_MOUNTS[]`. +3. **usage / init_color / parse_args / validate_env** - fail-fast startup + checks with copy-pasteable hints. +4. **update_base** - patches local base from mirror's `patches/packages/` when + ChangeLog head differs (reused from reference script). +5. **SBo tree lookup** - `find_slackbuild_dir`, `category_of`, `pkg_key`, + `read_requires`. +6. **Dependency resolution** - `resolve_target` -> `_resolve_visit` (DFS topo + sort + cycle detection). Outputs `RESOLVED_ORDER[]`, records `UNMET[]`, + `CYCLES[]`, `HAS_README[]`. `installed_in_base` checks base package db. +7. **Overlay lifecycle** - `setup_overlay` (echoes tmpdir, mounts overlay + + binds), `teardown_overlay` (idempotent, correct reverse order), `cleanup_trap` + (EXIT/INT/TERM, unwinds all live overlays). +8. **build_one** - copies SlackBuild into overlay, runs download/md5/build/ + installpkg INSIDE the chroot non-interactively via heredoc, reads back a + status token file, sets status maps. Logs resolved .info env up front and + the installed file list (from /var/log/packages) after installpkg, so the + overlay is fully disposable. +9. **run_target** - fresh overlay per target, builds resolved chain in order, + marks `BLOCKED-BY-DEP` on dependents of failures, tears down. +10. **print_summary** - color screen recap + plain `summary.log`. +11. **main** - parse, validate, make `RUN_DIR`, update base, collect targets + (single-package or category-folder), run each. + +## Teardown order (do not reorder) + +pts -> dev/proc/sys -> resolv.conf -> dbus machine-id -> overlay last. +Matches the reference script. Idempotent (mountpoint-guarded), trap-registered +so a mid-target abort still unwinds. `ACTIVE_MOUNTS[]` tracks live overlays. + +## Status values + +`SUCCESS DOWNLOAD-FAILED MD5-MISMATCH BUILD-FAILED INSTALL-FAILED +BLOCKED-BY-DEP UNMET-DEP`. `%README%` recorded separately as a reminder flag, +not a status. + +## What is verified vs not + +- **Verified by self-check** (`test-logic.sh` in repo, `bash test-logic.sh`, no + VM needed): topo order, `%README%` recording, unmet-dep, cycle detection, and + BLOCKED-BY-DEP propagation (`depends_on_failed`, including the transitive + one-hop cascade). The check builds a fake SBo tree and sources the script with + config overridden AFTER sourcing (sourcing re-runs the CONFIG block, so test + vars must be set after the `source`). Gotchas baked in: do not name the dead + list `failed` (collides with `depends_on_failed`'s `local -n failed`), and + `ok`/`bad` must `return 0` (`((x++))` returns nonzero when x was 0). +- **NOT runnable-tested** (needs the VM): overlay mounts, chroot build flow, + base patching, installpkg. Logic mirrors the reference script. + +## Known shortcuts (ponytail: comments in source) + +- `depends_on_failed`: direct-REQUIRES check only. Transitive blocking still + works because the loop runs in topo order, so a failure propagates one hop per + package. + +## Open items / TODO + +- **`--keep` removed (resolved).** No keep-overlay option. Overlay is always + torn down. Rationale: per-package logs now capture full build/install output, + resolved `.info` env, and the installed file list, so the overlay holds + nothing worth retaining. Do not re-add `--keep` without a reason logs cannot + cover. +- **Package-output cache** (deferred, user wants it next): keep built `.tgz` + outside the overlay, `installpkg` a cached dep instead of rebuilding. Fixes + the "shared deps rebuilt per target" cost in category mode. Likely shape: a + per-run stash in `$RUN_DIR/pkgcache` keyed `prog-version`, reused across + targets within one run. NOT implemented yet. +- **"all" mode**: build every package across all SBo roots. Extension point in + `main` where `targets` is populated. TODO marker in source. +- **queue/list-file mode**: build a named list. Same extension point. +- **`-j` parallelism**: flag parsed, currently no-op. TODO marker in source. + +## How to run the self-check + +```sh +bash test-logic.sh +``` + +Covers resolution + BLOCKED-BY-DEP. Extend it (not /tmp scratch files) when +adding logic. Anything VM-dependent (overlay, chroot, installpkg) is out of its +reach by design. diff --git a/README.md b/README.md new file mode 100644 index 0000000..b98197a --- /dev/null +++ b/README.md @@ -0,0 +1,161 @@ +# sbo-batch-test + +Batch-test SlackBuilds against a clean Slackware 15.0 overlay chroot. + +Resolves the full SBo dependency tree (locally, never network), topologically +sorts it, builds and installs every package in a fresh disposable overlay over +a read-only 15.0 base, captures persistent per-package logs, and prints a +color-coded summary. + +Built for an SBo maintainer whose daily driver is Slackware64-current but whose +packages target 15.0 stable. It verifies that SlackBuilds BUILD and install +cleanly against the 15.0 userland/toolchain/libraries, which is where +current-vs-15.0 drift bites. + +## Scope and limits + +- Shares the host kernel (the current VM's kernel, not a 15.0 kernel). It does + NOT test kernel-module packages or anything tied to the running kernel + version. Those still want a real 15.0 VM. +- Resolution is LOCAL-tree-only. No network, no sbopkg. The local SBo tree is + the single source of truth (it contains unpublished personal/pentesting + packages that do not exist upstream). +- Does not integrate with, wrap, or drive slackrepo. Separate, independent tool. +- Built packages are throwaway. The point is "does it build on clean 15.0", + not producing redistributable output. + +## Prerequisites + +You need three things in place, all configured in the CONFIG block at the top +of `sbo-batch-test`: + +1. **`SLACKWARE_BASE`** - a LOCAL (non-NFS) full Slackware 15.0 install tree. + This is the overlay lowerdir. It MUST be local (ext4/xfs). overlayfs over an + NFS lowerdir is fragile and a known source of intermittent failures. + +2. **`LOCAL_MIRROR_15`** - the NFS-mounted Slackware 15.0 mirror. Used only as a + package SOURCE to populate and patch `SLACKWARE_BASE`. Read-only is fine. + +3. **`SBO_TREE_ROOTS`** - one or more LOCAL SBo tree roots, resolved in order + (first match wins). Standard SBo layout + `<root>/<category>/<prog>/{prog.SlackBuild,prog.info,...}`. Read in place, + never copied or synced. + +Also: run as root (overlay + chroot require it). + +## Populating SLACKWARE_BASE + +Populate from the NFS mirror with the FULL package set, not a minimal install. +A minimal base causes false "missing dependency" results that would not happen +on a normal user's full Slackware install. + +```sh +mkdir -p /sbo-base/15.0 +for p in /mnt/nfs/slackware64-15.0/slackware64/*/*.t?z; do + installpkg --root /sbo-base/15.0 "$p" +done +``` + +Adjust the two paths to your `SLACKWARE_BASE` and `LOCAL_MIRROR_15`. The tool +will not auto-bootstrap the base unattended; if the base is missing or +incomplete it fails fast with this exact hint. + +The base is kept patched automatically on each run: when the mirror's +ChangeLog head differs from the recorded marker, new packages from +`patches/packages/` are applied to the local base via `upgradepkg`. + +## Usage + +``` +sbo-batch-test [OPTIONS] <program-name> +sbo-batch-test [OPTIONS] <category-folder> +``` + +### Single-package mode + +Resolve a program's full dep tree, build+install every dep in order, then the +target. The whole chain shares ONE overlay, torn down at the end. + +```sh +sbo-batch-test playwright-cli +``` + +### Category-folder mode + +Every SlackBuild dir inside the folder is an independent target. Each target +gets its OWN fresh overlay against pristine 15.0, so one target's installed +deps never leak into the next. Deps may live in other categories. + +```sh +sbo-batch-test ./network +sbo-batch-test /path/to/SBo-danix/pentesting +``` + +### Dry run + +Resolve and print the build order, build nothing. + +```sh +sbo-batch-test --dry-run playwright-cli +``` + +### Options + +| Option | Effect | +|-------------------|--------------------------------------------------------------| +| `-h`, `--help` | Usage text. | +| `--no-color` | Disable ANSI color (auto-disabled when stdout is not a TTY). | +| `--dry-run` | Resolve and print build order, do not build. | +| `--with-x` | Enable X passthrough (`xhost +local:hosts`). Headless by default. Security caveat: allows local non-network connections to your X server. | +| `-j`, `--jobs N` | Reserved. No-op stub today (builds are serial). | + +## Per-package status values + +| Status | Meaning | +|------------------|-----------------------------------------------------------| +| `SUCCESS` | Built and installed. | +| `DOWNLOAD-FAILED`| Source download failed. | +| `MD5-MISMATCH` | Source MD5 did not match the .info. Hard failure. | +| `BUILD-FAILED` | The .SlackBuild failed or produced no package. | +| `INSTALL-FAILED` | `installpkg` of the resulting package failed. | +| `BLOCKED-BY-DEP` | A dependency failed, so this was not attempted. | +| `UNMET-DEP` | A required package is neither in the SBo tree nor in base. Also covers dependency cycles. | + +Packages whose REQUIRES carry `%README%` are flagged in the summary as a +reminder to check manual/optional configuration steps. `%README%` is not a +package and is skipped for build ordering. + +## Logs + +Written outside the overlay so they survive teardown, under `LOG_ROOT`: + +``` +$LOG_ROOT/2026-06-22_14-30-05/ + <category>_<prog>.log per-package full build/install output + summary.log plain-text recap (same facts as the screen summary, no color) + build-order.txt the resolved topological order actually used +``` + +Each per-package log captures the full build/install output, the resolved +`.info` build context (PRGNAM/VERSION/BUILD/TAG/REQUIRES), and the installed +file list from the package db. That is everything worth inspecting, so the +overlay is always disposed and there is no keep-overlay option. + +## Design tradeoffs (on the record) + +- **Shared deps are rebuilt per target.** Each category target starts from a + clean base, so a dep shared by N targets is rebuilt N times. Correct for + isolation, slow for heavy shared deps (qt5, boost). A future optional + package-output cache (keep built `.tgz` outside the overlay, `installpkg` a + cached dep when its SlackBuild + version are unchanged) is marked TODO in the + source, not yet implemented. +- **overlayfs sharp edges.** A build that fails ONLY in the overlay but works on + bare 15.0 is more likely an overlayfs quirk (rename/whiteout) than a real + build bug. Noted near the build step in the source. + +## Reference + +Overlay/chroot mount + teardown machinery follows the patterns in +`overlay-chroot.sh` by Jeremy Hansen (bassmadrigal). The teardown ordering +(pts, dev/proc/sys, resolv.conf, dbus machine-id, overlay last) is preserved +deliberately. diff --git a/claude-code-prompt-sbo-batch-tester.txt b/claude-code-prompt-sbo-batch-tester.txt new file mode 100644 index 0000000..7a2d0e5 --- /dev/null +++ b/claude-code-prompt-sbo-batch-tester.txt @@ -0,0 +1,289 @@ +================================================================================ +PROMPT FOR CLAUDE CODE +Build a batch SlackBuild testing tool for Slackware 15.0 stable +================================================================================ + +CONTEXT +------- +I am an SBo (SlackBuilds.org) package maintainer. My daily-driver machine runs +Slackware64-current, but SBo targets Slackware 15.0 stable. I need to test my +SlackBuilds against a clean 15.0 environment WITHOUT booting a VM, because a +build that works on -current can fail on 15.0 (different toolchain, library +sonames, Python version, etc.). + +There is an existing reference script by Jeremy Hansen (bassmadrigal) called +overlay-chroot.sh that solves the isolation problem using an overlayfs-backed +chroot: it keeps a read-only 15.0 base tree as the overlay lowerdir, layers a +disposable writable upper dir on top, binds the needed system dirs (dev, proc, +sys, /dev/pts, resolv.conf, dbus machine-id), drops the user into an interactive +chroot, and on exit unmounts everything and discards the upper layer. I want to +REUSE that proven mounting/teardown machinery but build a NEW, purpose-built +tool around it. Treat overlay-chroot.sh as a reference for the mechanics, not as +a base to lightly patch. The new tool's job is different: non-interactive, +automated, batch testing with dependency resolution, persistent logs, and a +color summary. + +NOTE ON ISOLATION SCOPE (so expectations are correct): the chroot shares the +host kernel, which here is the -current VM's kernel, not a 15.0 kernel. This is +acceptable and expected: the tool verifies that SlackBuilds BUILD and install +cleanly against the 15.0 userland/toolchain/libraries, which is where -current +vs 15.0 drift actually bites. It does NOT claim to test kernel-module packages +or anything depending on the running kernel version; those few cases still want +a real 15.0 VM and are out of scope for this tool. + +I will provide overlay-chroot.sh alongside this prompt (or you can ask me to +paste it). Study its overlay setup, bind mounts, X-access handling, and its +robust cleanup/unmount ordering and reuse those patterns faithfully. Its cleanup +ordering in particular is correct and worth preserving (pts before dev, overlay +last, etc.). + +================================================================================ +GOAL +================================================================================ +A single root-run bash script that takes either ONE package name or a CATEGORY +FOLDER, resolves and builds the full SBo dependency tree in correct order inside +a clean, disposable Slackware 15.0 overlay chroot, installs each resulting +package, captures persistent per-package logs, and prints a color-coded summary +at the end recapping exactly what happened. + +================================================================================ +INVOCATION / ARGUMENTS +================================================================================ +The script must accept: + + 1. A single program name, e.g.: + sbo-batch-test playwright-cli + -> Resolve its full SBo dependency tree, topologically sort it, build and + install every dependency first (in order), then build/install the + target. Test as a unit; if a dependency fails, the target is reported + as blocked/failed and the reason is the failed dep. + + 2. A category folder from my LOCAL SBo repository, e.g.: + sbo-batch-test ./network + sbo-batch-test /path/to/SBo-danix/pentesting + -> Treat every SlackBuild directory inside that folder as a target. For + EACH target, resolve+build its dep tree (deps may live in OTHER + categories) then build the target. Each target is an independent test + unit; one target failing must not abort the whole run. + +Design the argument parsing so additional modes can be added later (e.g. "all", +a queue/list file). Do not implement those now, but leave a clean extension +point and mention it in --help. + +Standard niceties: + --help / -h usage text + --keep do not destroy the overlay on completion (for inspection) + --no-color disable ANSI color in summary (auto-disable if stdout is + not a TTY) + --jobs / -j N reserved/optional; if non-trivial, stub it and note it + A dry-run flag that resolves and PRINTS the build order without building. + +================================================================================ +DEPENDENCY RESOLUTION (this is the hard part — get it right) +================================================================================ +- Parse REQUIRES from each package's .info file. +- Resolve TRANSITIVELY: a dep's deps must also be built first. +- TOPOLOGICALLY SORT the resulting graph so every package is built only after + all its dependencies. Detect and clearly report dependency CYCLES instead of + looping forever. +- The "%README%" token in REQUIRES is NOT a package. It is a marker meaning the + user must read the README for manual configuration/optional deps. Skip it for + build ordering but RECORD that the package carries a %README% so it shows in + the summary (maintainers should be reminded). +- A dependency may already be satisfied by the base Slackware install (i.e. it + is part of stock 15.0, not an SBo package). Such deps will NOT be found in the + SBo tree. Do NOT treat "not in SBo tree" as a hard failure by default; check + whether it is already installed in the chroot (look in + /var/log/packages). If it is neither in the SBo tree nor installed, THAT is a + real unmet-dependency error — report it. +- SOURCE OF TRUTH for resolution: my LOCAL on-disk SBo tree (my SBo-danix + checkout and/or a local SBo 15.0 clone), NOT the network. The script should + take the SBo tree root as a configurable variable at the top. If a needed + SlackBuild directory cannot be located in the configured tree(s), report it as + a resolution failure with the missing package name — do not silently skip. + This is settled: resolution is LOCAL-tree-only (the SBO_TREE_ROOTS in the VM). + Do NOT add network/sbopkg resolution. My SBo-danix tree contains unpublished + personal and pentesting packages that do not exist upstream, so the local tree + is the only correct source of truth. + +================================================================================ +BUILD + INSTALL FLOW (per package, inside the chroot) +================================================================================ +For each package in sorted order: + 1. Copy its SlackBuild directory into a working location inside the overlay. + 2. Source the .info; download the source(s) to the build dir. + 3. Verify the MD5SUM from the .info. A mismatch is a hard failure for that + package (and blocks anything depending on it). + 4. Run the .SlackBuild non-interactively. Capture full stdout+stderr to that + package's log file. + 5. installpkg the resulting package from /tmp (or $OUTPUT). + 6. Record: SUCCESS / BUILD-FAILED / DOWNLOAD-FAILED / MD5-MISMATCH / + INSTALL-FAILED / BLOCKED-BY-DEP / UNMET-DEP, plus elapsed build time. + 7. If a package fails, mark every not-yet-built package that depends on it as + BLOCKED-BY-DEP (do not attempt them) and continue with independent targets. + +The chroot must run the build NON-INTERACTIVELY (no dropping into a shell). +Reuse the reference script's chroot/env/PS1 mounting approach but execute a +command string instead of an interactive login shell. Preserve and propagate +exit codes correctly. + +OVERLAY LIFECYCLE — THIS IS THE INTENDED DESIGN, NOT A QUESTION: +The purpose of this tool is purely to verify that SlackBuilds BUILD cleanly +against stable 15.0. I do NOT keep or redistribute the resulting packages for +stable, so installed packages are throwaway. Therefore: + - Use a FRESH, disposable overlay PER TARGET, destroyed after that target's + build chain completes (success or failure). + - Single-package mode: one overlay covers the whole dependency-tree-plus-target + chain, then is torn down. + - Category-folder mode: EACH target package gets its OWN fresh overlay against + a pristine 15.0 base. A previous target's built/installed deps must NOT leak + into the next target. This guarantees every target is tested against clean + stable, which is the whole point. + - --keep should suppress teardown ONLY for the final/most-recent overlay (or + make its exact behavior clear in --help); by default everything is destroyed. + +Accepted tradeoff (document it, do not optimize it away now): shared +dependencies will be rebuilt for every target that requires them, since each +target starts from a clean base. This is correct for isolation. Leave a TODO +marker for a future optional package-output cache (keep built .tgz outside the +overlay, installpkg a cached dep when its SlackBuild + version are unchanged) so +this optimization is on record but NOT implemented now. + +================================================================================ +ENVIRONMENT / BASE TREE (my actual setup — wire these as config vars) +================================================================================ +DEPLOYMENT CONTEXT — read this carefully, it constrains the design: +This tool runs INSIDE my Slackware64-CURRENT VM, the same VM where my SBo-danix +SlackBuild repo lives and where I use slackrepo to build my redistributable +-current packages. The SBo tree is therefore a LOCAL path in the VM and is the +single source of truth. There is NO second copy of the repo and the tool must +NOT clone, sync, or duplicate it. It reads the local tree in place. + +The Slackware 15.0 STABLE material (mirror) is provided over NFS, mounted into +the VM. This keeps the big generic distro bits out of the VM's own storage. + +CRITICAL OVERLAYFS CONSTRAINT — do not get this wrong: +The overlay LOWERDIR (SLACKWARE_BASE) must be a LOCAL filesystem path in the VM +(ext4/xfs/etc), NOT the NFS mount. overlayfs over an NFS lowerdir is fragile and +a known source of intermittent, hard-to-debug failures. Therefore: + - NFS provides the 15.0 MIRROR only (a read-only package SOURCE). + - SLACKWARE_BASE is a LOCAL directory in the VM, populated FROM that mirror + with `installpkg -root`. The overlay lowerdir points at this local tree. +Make this separation explicit in the config comments so the lowerdir is never +accidentally pointed at the NFS path. + +Expose clear, well-commented config variables at the top: + SLACKWARE_BASE -> LOCAL (non-NFS) read-only 15.0 base install tree; + this is the overlay lowerdir. MUST be local. + LOCAL_MIRROR_15 -> the NFS-mounted 15.0 mirror (package source for + populating/patching the base). Read-only is fine. + SBO_TREE_ROOTS -> one or more LOCAL SBo tree roots in the VM to + resolve from (my SBo-danix checkout, plus optionally + a local SBo 15.0 clone for upstream deps). In-place, + never copied or synced. + CHROOT_LOCATION -> where overlays are created (default /tmp), LOCAL. + LOG_ROOT -> where persistent logs are written (see below). +- Keep the base patched: reuse the reference script's logic that upgrades the + base from the mirror's patches/ when the ChangeLog head differs. Point it at + LOCAL_MIRROR_15 (the NFS mirror). The base tree it WRITES to is the local + SLACKWARE_BASE. +- STARTUP VALIDATION (fail fast, clear messages): + * Verify SLACKWARE_BASE is a LOCAL path and looks like a real Slackware + install (/var/log/packages present, key dirs exist). If it does not exist + or is incomplete, FAIL with a precise, copy-pasteable hint for populating + it from LOCAL_MIRROR_15 via `installpkg -root` (full package set, not + minimal — a minimal base yields false "missing dependency" results that + would not occur on a normal user's full Slackware install). + * Verify LOCAL_MIRROR_15 is actually mounted/reachable (it is NFS); if the + mount is absent, fail with a clear message rather than proceeding against + an empty path. + * Verify each SBO_TREE_ROOTS path exists locally. + Do NOT attempt to auto-bootstrap the base unattended. + +DO NOT integrate with, wrap, or drive slackrepo. slackrepo is my separate +release pipeline (clean reproducible builds, upstream revision tracking, +slackpkg+ output for redistribution). THIS tool is only a disposable pre-flight +"does it build on clean 15.0 stable" check. Keep them fully independent. + +================================================================================ +LOGGING (non-volatile — must survive overlay teardown) +================================================================================ +- Logs MUST be written OUTSIDE the overlay so they persist after cleanup. +- Per RUN: create a timestamped directory under LOG_ROOT, e.g. + $LOG_ROOT/2026-06-22_14-30-05/ +- Inside it: + <category>_<prog>.log per-package full build/install log + summary.log plain-text recap of the whole run + build-order.txt the resolved topological order actually used +- The plain-text summary.log must be greppable and contain the same facts as + the on-screen color summary (no color codes in the file). + +================================================================================ +END-OF-RUN COLOR SUMMARY (on screen) +================================================================================ +Print a clear, color-coded recap to the terminal: + - GREEN = success + - RED = failed (with the failure category) + - YELLOW = blocked-by-dep, unmet-dep, or skipped + - Note packages carrying a %README% (so I remember to check manual steps). +Include, per package: status, failure reason if any, and build time. +End with totals: X succeeded, Y failed, Z blocked, total wall time, and the +path to the run's log directory. +Auto-disable color when stdout is not a TTY or when --no-color is passed. + +================================================================================ +SAFETY, CORRECTNESS, ROBUSTNESS +================================================================================ +- Must run as root (overlay + chroot). Check EUID and exit cleanly if not. +- Use `set -euo pipefail` thoughtfully, but DO NOT let one package's build + failure kill the whole batch — isolate per-package execution so the run + continues. A failed build is an expected, handled outcome, not a script crash. +- Cleanup must be bulletproof and idempotent: trap EXIT/INT/TERM and unmount in + the correct reverse order (pts, dev/proc/sys, resolv.conf, dbus machine-id, + overlay last), exactly as the reference script does, even if the run aborts + mid-way. Never leave dangling mounts or overlay dirs (unless --keep). Because + overlays are created and destroyed PER TARGET, the teardown routine must be a + reusable function called after each target, AND registered in the trap so an + abort mid-target still unmounts cleanly. Track all active overlay mounts so the + trap can unwind whichever one is live when an interrupt hits. +- Be aware overlayfs has occasional sharp edges with some builds; if a build + fails ONLY in the overlay, that is worth a comment in the code near the build + step, but do not try to work around it speculatively. +- X server access: default to NO X (these are headless batch builds). Only wire + optional X passthrough behind a flag, and comment the xhost security caveat. +- Quote variables, handle paths with spaces, use mktemp for temp dirs. +- Comment generously. Put all user-tunable config in one clearly marked block at + the top. + +================================================================================ +DELIVERABLES +================================================================================ +1. The new script (suggest name: sbo-batch-test). Single self-contained bash + file, heavily commented. +2. A short README section (can be a comment block or separate .md) covering: + prerequisites (the 15.0 base tree, the mirror, the SBo tree roots), how to + populate SLACKWARE_BASE, and usage examples for both the single-package and + category-folder modes, plus the dry-run. +3. Inline TODO markers for the deferred features (e.g. "all" mode, queue-file + mode, -j parallelism) at the extension points. + +================================================================================ +WORKING STYLE I WANT FROM YOU +================================================================================ +- ASK ME QUESTIONS before coding if anything is ambiguous — especially the + dependency-resolution source-of-truth assumption above, the exact layout of my + SBo tree, and whether my SlackBuilds follow stock SBo conventions. +- Do NOT default to agreement. If any choice in this prompt is wrong, suboptimal, + or fragile (e.g. overlay quirks, dep-resolution edge cases, the + build-then-install-then-leave-installed question), tell me and propose a + better alternative with reasoning. +- Build incrementally: I'd rather review the dependency resolver and the + mount/teardown core first, confirm they're solid, then layer on logging and + the summary. + +A note on my conventions: when writing any prose, comments, README text, or +commit messages, do NOT use the em dash character. Use commas or periods. + +================================================================================ +END OF PROMPT +================================================================================ diff --git a/overlay-chroot.sh b/overlay-chroot.sh new file mode 100644 index 0000000..332311b --- /dev/null +++ b/overlay-chroot.sh @@ -0,0 +1,257 @@ +#!/bin/bash +# +# Copyright 2023-2025 Jeremy Hansen <jebrhansen -at- gmail.com> +# All rights reserved. +# +# Redistribution and use of this script, with or without modification, is +# permitted provided that the following conditions are met: +# +# 1. Redistributions of this script must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ''AS IS'' AND ANY EXPRESS OR IMPLIED +# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO +# EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; +# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR +# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF +# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +# ----------------------------------------------------------------------------- + +# Create a chroot from a maintained and updated Slackware stable base to allow +# easy testing of SlackBuild scripts in a clean environment. Ensure that base +# is updated every time this script is ran along with updating sbopkg and +# running sqg to update all queues. Offer to remove chroot files when exiting +# the chroot. + +# Supports passing "cleanup" to remove any existing chroots and their files. + +# Supports passing "update" to update the base image with slackpkg and sbopkg +# and then exiting without starting the chroot. + +# TODO +# Currently empty + +# ----------------------------------------------------------------------------- + +# --------------------------Global Settings Beginning-------------------------- + +# Set where you want the chroot located and the base name of the folder +CHROOT_LOCATION=/tmp/ +CHROOT_TEMPLATE_BASE="chroot" + +# Set variables for the base image and location of the local Slackware mirror +VERSION=15.0 +SLACKWARE_BASE=/share/gothrough/sbo-build/$VERSION +LOCAL_MIRROR=/share/gothrough/slackware-mirrors/slackware64-$VERSION/ + +# To allow you to open GUI programs from within the chroot, you need to +# allow "remote" access to the x server. This could possibly open up +# security issues, but it is limited to non-network local connections. +# Change to "no" if you want this disabled or pass ACCESS=no to the script. +ACCESS=${ACCESS:-yes} + +# ---------------------------Global Settings Ending---------------------------- + +# --------------------------Custom Commands Beginning-------------------------- +# This will allow you to add custom aliases or functions into the chroot +# Make sure you escape single quotes and variables and that all commands are on +# their own lines +custom_cmd=' +# Check all listed deps in a .info to see if they are installed +alias checkdeps=". *.info; for i in \$REQUIRES; do ls /var/log/packages/*SBo* | cut -d/ -f5 | rev | cut -d- -f4- | rev | grep ^\$i$; done" +' +# ---------------------------Custom Commands Ending---------------------------- + +# Check that we're root +if [ "$EUID" -ne 0 ]; then + echo "Please run as root" + exit +fi + +# Provide an easy cleanup for older tmp files and exit +if [ "$1" == "cleanup" ]; then + + for i in "$CHROOT_LOCATION"/"$CHROOT_TEMPLATE_BASE".*; do + if [ -d "$i" ]; then + echo "Found $i" + else + echo "No chroots to clean up." + exit 2 + fi + + if mountpoint -q "$i"/chroot/dev/pts; then + printf "\tUnmounting %s/chroot/dev/pts\n" "$i" + umount "$i"/chroot/dev/pts + fi + for j in dev proc sys; do + if mountpoint -q "$i"/chroot/$j; then + printf "\tUnmounting %s/chroot/%s\n" "$i" "$j" + umount "$i"/chroot/$j + fi + done + + if mountpoint -q "$i"/chroot/etc/resolv.conf; then + printf "\tUnmounting %s/chroot/etc/resolv.conf\n" "$i" + umount "$i"/chroot/etc/resolv.conf + fi + + if mountpoint -q "$i"/chroot/var/lib/dbus/machine-id; then + printf "\tUnmounting %s/chroot/var/lib/dbus/machine-id\n" "$i" + umount "$i"/chroot/var/lib/dbus/machine-id + fi + + # umount overlayfs + if mountpoint -q "$i"/chroot; then + printf "\tUnmounting %s/chroot/%s/chroot\n" "$i" "$j" + umount "$i"/chroot + fi + + # Remove dirs + if [ -d "$i" ]; then + printf "\tRemoving %s.\n" "$i" + rm -r "$i" + fi + done + echo "Cleanup complete" + exit +fi + +# Track the latest updates to prevent attempting to update system +# packages and rebuilding sbopkg's queues +touch $SLACKWARE_BASE/last-base-update + +# Make sure the base image is up-to-date +if [ "$(head -n1 $LOCAL_MIRROR/ChangeLog.txt)" != "$(cat $SLACKWARE_BASE/last-base-update)" ]; then + for i in "$LOCAL_MIRROR"/patches/packages/*.t?z; do + if [ ! -e "$SLACKWARE_BASE"/var/lib/pkgtools/packages/"$(basename "${i%.*}")" ]; then + ROOT=$SLACKWARE_BASE upgradepkg --install-new "$i" + fi + done + echo "Slackware has been updated with local mirror." + head -n1 $LOCAL_MIRROR/ChangeLog.txt > $SLACKWARE_BASE/last-base-update +else + echo "Slackware is up-to-date with the local mirror." +fi + +# Set up directories for the chroot +echo "Creating required directories for the overlay" +TMPDIR=$(mktemp -d "$CHROOT_LOCATION"/"$CHROOT_TEMPLATE_BASE".XXXXX) +mkdir "$TMPDIR"/{changes,tmp,chroot} + +# Mount the overlayfs +echo "Mounting the overlay" +mount -t overlay overlay -olowerdir="$SLACKWARE_BASE",upperdir="$TMPDIR"/changes,workdir="$TMPDIR"/tmp "$TMPDIR"/chroot + +# Bind mount the pertinent system dirs +echo "Binding required directories" +mkdir -p "$TMPDIR"/changes/{dev,proc,sys} +for i in dev proc sys; do + mount -o bind /$i "$TMPDIR"/chroot/$i +done + +# Mount /dev/pts for sudo +mkdir -p "$TMPDIR"/changes/dev/pts +mount -o bind /dev/pts "$TMPDIR"/chroot/dev/pts + +# Give the chroot internet +echo "Setting up internet" +mount -o bind /etc/resolv.conf "$TMPDIR"/chroot/etc/resolv.conf +chroot "$TMPDIR"/chroot /bin/bash -c "/usr/sbin/update-ca-certificates --fresh > /dev/null" + +# Setting up DBUS binding required for certain apps +echo "Binding DBUS to local machine" +touch "$TMPDIR"/chroot/var/lib/dbus/machine-id +mount -o bind /var/lib/dbus/machine-id "$TMPDIR"/chroot/var/lib/dbus/machine-id + +# Update sbopkg (if installed) and queues +# Do it in the chroot to prevent GPG errors, but copy files back to the +# base image so we only need to do it during updates. +if [ -e "$SLACKWARE_BASE"/usr/sbin/sbopkg ]; then + echo "Checking for SBo updates for sbopkg" + # Get the latest changelog date from server + SERVDATE="$(wget -qO- https://slackbuilds.org/slackbuilds/15.0/ChangeLog.txt | head -n1)" + if [ -z "$SERVDATE" ]; then + echo "Upstream address did not provide a changelog." + echo "Please validate internet is working and address is correct" + echo "This will continue in 5 seconds. Ctrl+C if you'd like to exit." + sleep 5 + fi + # Get latest changelog date on local copy + LOCALDATE="$(head -n1 $SLACKWARE_BASE/var/lib/sbopkg/SBo/15.0/ChangeLog.txt)" + # If they don't match, update sbopkg and run sqg. Copy updates back to base image. + if [ "$SERVDATE" != "$LOCALDATE" ]; then + chroot "$TMPDIR"/chroot /bin/bash -c "/usr/sbin/sbopkg -r; /usr/sbin/sqg -a" + rsync -a --delete "$TMPDIR"/chroot/var/lib/sbopkg/ "$SLACKWARE_BASE"/var/lib/sbopkg + rsync -a --delete "$TMPDIR"/chroot/root/.gnupg "$SLACKWARE_BASE"/root/ + else + echo "sbopkg is up-to-date." + fi +fi + +# Only set up X server access and launch the chroot if update isn't passed +if [ "$1" != "update" ]; then + # Checking if we can add local connection access + if [ "$ACCESS" == "yes" ]; then + echo "Setting up X server access" + echo "STATUS: $(xhost +local:hosts)" + fi + + # Check if there are custom commands to add by checking for text + if [[ "$custom_cmd" =~ [a-zA-Z] ]]; then + echo "Adding custom commands to chroot's /etc/profile.d/chroot_custom_cmds.sh" + echo "$custom_cmd" > "$TMPDIR"/chroot/etc/profile.d/chroot_custom_cmds.sh + chmod +x "$TMPDIR"/chroot/etc/profile.d/chroot_custom_cmds.sh + fi + + # Let's save the following in the root of the chroot structure to allow + # the user to enter into that chroot from another prompt and/or if they + # accidentally leave the chroot. + # Then we'll just execute the file to actually enter the chroot. + cat << EOH > "$TMPDIR"/start-chroot.sh +#!/bin/bash +# Time to actually chroot and do our work +# Need to type 'exit' to leave the chroot and start the cleanup +# Use custom PS1 so we know we're in the chroot +echo "Entering chroot. Please type \"exit\" to exit it." +echo "You can add files to the chroot by placing them in $TMPDIR/chroot/" +chroot "$TMPDIR"/chroot env PS1="\[\e[41m\]\u\[\e[49m\]@\[\e[33m\]$(basename "$TMPDIR")\[\e[0m\]:\w$ " bash -l +EOH + + # Start the chroot + bash "$TMPDIR"/start-chroot.sh +fi + +# Start cleanup + +# Undo bind mounts +umount "$TMPDIR"/chroot/dev/pts +for i in dev proc sys; do + umount "$TMPDIR"/chroot/$i +done + +umount "$TMPDIR"/chroot/etc/resolv.conf +umount "$TMPDIR"/chroot/var/lib/dbus/machine-id + +# umount overlayfs +umount "$TMPDIR"/chroot + +# Only ask to delete chroot if update isn't passed, otherwise delete without asking +if [ "$1" != "update" ]; then + # Ask if tmp dirs should be removed + # Could be kept to review filesystem changes + echo -n "Would you like to remove the unneeded overlay directories? y/N " + read -r answer + # If anything other than y, rm them + if ! /usr/bin/grep -qi "y" <<< "$answer"; then + echo "Temp overlay dirs will not be removed. They can be found at $TMPDIR." + else + rm -r "$TMPDIR" + fi +else + rm -r "$TMPDIR" +fi diff --git a/sbo-batch-test b/sbo-batch-test new file mode 100755 index 0000000..d6ca1ef --- /dev/null +++ b/sbo-batch-test @@ -0,0 +1,668 @@ +#!/bin/bash +# +# sbo-batch-test - batch-test SlackBuilds against a clean Slackware 15.0 +# overlay chroot. Non-interactive, dependency-resolving, per-target disposable +# overlay, persistent logs, color summary. +# +# Overlay/chroot mount + teardown machinery follows the patterns in +# overlay-chroot.sh by Jeremy Hansen (bassmadrigal). The teardown ordering +# (pts, dev/proc/sys, resolv.conf, dbus machine-id, overlay last) is preserved +# deliberately, it is the correct unwind order. +# +# Runs INSIDE a Slackware64-current VM. Verifies SlackBuilds BUILD and install +# cleanly against 15.0 userland. Does NOT test kernel modules (shares host +# kernel). Resolution is LOCAL-tree-only, never network. Does not touch +# slackrepo. +# +# No em dashes in prose by author convention. + +# ============================================================================= +# CONFIG (edit these for your VM) +# ============================================================================= + +# LOCAL (non-NFS) read-only 15.0 base install tree. This is the overlay +# lowerdir. MUST be a local filesystem (ext4/xfs). NEVER point this at the NFS +# mirror: overlayfs over NFS lowerdir is fragile and fails intermittently. +SLACKWARE_BASE="/sbo-base/15.0" + +# NFS-mounted 15.0 mirror. Package SOURCE only, used to populate/patch +# SLACKWARE_BASE. Read-only is fine. Never used as the overlay lowerdir. +LOCAL_MIRROR_15="/mnt/nfs/slackware64-15.0" + +# One or more LOCAL SBo tree roots, resolved in order (first match wins). +# Standard SBo layout: <root>/<category>/<prog>/{prog.SlackBuild,prog.info,...}. +# Read in place, never copied or synced. +SBO_TREE_ROOTS=( + "/home/danix/SBo-danix" + "/home/danix/slackbuilds-15.0" +) + +# Where overlays are created. LOCAL. One disposable overlay per target lives here. +CHROOT_LOCATION="/tmp" + +# Where persistent logs are written (outside the overlay, survives teardown). +LOG_ROOT="/var/log/sbo-batch-test" + +# Slackware version, used for the mirror ChangeLog / patches path. +VERSION="15.0" + +# ============================================================================= +# END CONFIG +# ============================================================================= + +set -uo pipefail +# Note: NOT using -e globally. One package build failing is an expected, +# handled outcome, not a script crash. Per-package execution is isolated. + +# ---- flags / globals -------------------------------------------------------- +USE_COLOR=1 # --no-color or non-TTY disables +DRY_RUN=0 # resolve + print build order, do not build +WITH_X=0 # --with-x: optional X passthrough (default headless) +JOBS=1 # -j: reserved, see TODO +TARGET_ARG="" + +RUN_DIR="" # timestamped log dir for this run +declare -a ACTIVE_MOUNTS=() # overlay chroot roots currently mounted (for trap unwind) +CURRENT_OVERLAY="" # the $TMPDIR of the overlay being built in now + +# Status tracking. Keyed by "category/prog". Parallel assoc arrays. +declare -A ST_STATUS=() # SUCCESS|BUILD-FAILED|DOWNLOAD-FAILED|MD5-MISMATCH|INSTALL-FAILED|BLOCKED-BY-DEP|UNMET-DEP +declare -A ST_REASON=() +declare -A ST_TIME=() # elapsed seconds +declare -A ST_README=() # 1 if package carries %README% + +# ============================================================================= +# usage +# ============================================================================= +usage() { + cat <<'EOF' +sbo-batch-test - batch-test SlackBuilds against a clean Slackware 15.0 overlay chroot + +USAGE: + sbo-batch-test [OPTIONS] <program-name> + sbo-batch-test [OPTIONS] <category-folder> + +MODES: + <program-name> Resolve its full SBo dependency tree, build+install every + dep in topological order, then build+install the target. + <category-folder> Treat every SlackBuild dir inside as an independent target. + Each target gets its OWN fresh overlay against pristine 15.0. + +OPTIONS: + -h, --help This text. + --no-color Disable ANSI color (auto-disabled when stdout is not a TTY). + --dry-run Resolve and print the build order, do not build. + --with-x Enable X passthrough via xhost +local:hosts (security caveat: + allows local non-network connections to your X server). + Default is headless (no X). + -j, --jobs N Reserved. Currently a no-op stub (builds are serial). + +EXTENSION POINTS (not implemented, see TODOs in source): + - "all" mode (build every package across all SBo roots). + - queue/list-file mode (build a named list of targets). + - -j parallelism. + +EOF +} + +# ============================================================================= +# color helpers +# ============================================================================= +init_color() { + if [[ $USE_COLOR -eq 1 && -t 1 ]]; then + C_RED=$'\e[31m'; C_GRN=$'\e[32m'; C_YEL=$'\e[33m'; C_RST=$'\e[0m' + else + C_RED=""; C_GRN=""; C_YEL=""; C_RST="" + fi +} + +# ============================================================================= +# arg parsing +# ============================================================================= +parse_args() { + while [[ $# -gt 0 ]]; do + case "$1" in + -h|--help) usage; exit 0 ;; + --no-color) USE_COLOR=0; shift ;; + --dry-run) DRY_RUN=1; shift ;; + --with-x) WITH_X=1; shift ;; + -j|--jobs) JOBS="${2:-1}"; shift 2 ;; # TODO: implement parallelism + -*) echo "Unknown option: $1" >&2; usage >&2; exit 2 ;; + *) + if [[ -n "$TARGET_ARG" ]]; then + echo "Only one target argument accepted (got '$TARGET_ARG' and '$1')." >&2 + exit 2 + fi + TARGET_ARG="$1"; shift ;; + esac + done + if [[ -z "$TARGET_ARG" ]]; then + echo "No target given." >&2; usage >&2; exit 2 + fi +} + +# ============================================================================= +# startup validation (fail fast, copy-pasteable hints) +# ============================================================================= +validate_env() { + if [[ $EUID -ne 0 ]]; then + echo "This tool must run as root (overlay + chroot)." >&2 + exit 1 + fi + + # SLACKWARE_BASE: local, looks like a real Slackware install. + if [[ ! -d "$SLACKWARE_BASE" || ! -d "$SLACKWARE_BASE/var/log/packages" ]]; then + cat >&2 <<EOF +SLACKWARE_BASE is missing or incomplete: $SLACKWARE_BASE +It must be a LOCAL (non-NFS) full Slackware 15.0 install tree. +Populate it from the mirror with the FULL package set (not minimal, +a minimal base causes false "missing dependency" results): + + mkdir -p "$SLACKWARE_BASE" + for p in "$LOCAL_MIRROR_15"/slackware64/*/*.t?z; do + installpkg --root "$SLACKWARE_BASE" "\$p" + done +EOF + exit 1 + fi + # Guard against the classic mistake: base sitting on the NFS mirror path. + case "$SLACKWARE_BASE" in + "$LOCAL_MIRROR_15"*) + echo "SLACKWARE_BASE must NOT live under LOCAL_MIRROR_15 (NFS). overlayfs over NFS lowerdir is unsupported here." >&2 + exit 1 ;; + esac + + # LOCAL_MIRROR_15: NFS, must be mounted/reachable. + if [[ ! -d "$LOCAL_MIRROR_15" || ! -f "$LOCAL_MIRROR_15/ChangeLog.txt" ]]; then + echo "LOCAL_MIRROR_15 not reachable (NFS mount absent?): $LOCAL_MIRROR_15" >&2 + echo "Expected a Slackware 15.0 mirror with ChangeLog.txt at its root." >&2 + exit 1 + fi + + # SBo roots exist locally. + local r found=0 + for r in "${SBO_TREE_ROOTS[@]}"; do + if [[ -d "$r" ]]; then found=1; else + echo "SBO_TREE_ROOTS path does not exist: $r" >&2 + fi + done + if [[ $found -eq 0 ]]; then + echo "No valid SBo tree root found. Fix SBO_TREE_ROOTS." >&2 + exit 1 + fi +} + +# ============================================================================= +# keep base patched from the mirror (reuse reference logic, point at NFS mirror, +# write to local base). Skipped on --dry-run. +# ============================================================================= +update_base() { + local marker="$SLACKWARE_BASE/last-base-update" + touch "$marker" + local head_now; head_now="$(head -n1 "$LOCAL_MIRROR_15/ChangeLog.txt")" + if [[ "$head_now" == "$(cat "$marker")" ]]; then + echo "Base is up-to-date with the mirror." + return + fi + echo "Patching base from mirror..." + local p + for p in "$LOCAL_MIRROR_15"/patches/packages/*.t?z; do + [[ -e "$p" ]] || continue + if [[ ! -e "$SLACKWARE_BASE/var/lib/pkgtools/packages/$(basename "${p%.*}")" ]]; then + ROOT="$SLACKWARE_BASE" upgradepkg --install-new "$p" + fi + done + echo "$head_now" > "$marker" + echo "Base patched." +} + +# ============================================================================= +# SBo tree lookup. Find the SlackBuild dir for a prog name across roots. +# Echoes the dir path, returns 0 if found, 1 otherwise. +# ============================================================================= +find_slackbuild_dir() { + local prog="$1" root d + for root in "${SBO_TREE_ROOTS[@]}"; do + [[ -d "$root" ]] || continue + # <root>/<category>/<prog> + for d in "$root"/*/"$prog"; do + if [[ -d "$d" && -f "$d/$prog.info" ]]; then + echo "$d"; return 0 + fi + done + done + return 1 +} + +# Category of a SlackBuild dir = its parent dir name. +category_of() { basename "$(dirname "$1")"; } +# Key used in status maps and log filenames. +pkg_key() { echo "$(category_of "$1")/$(basename "$1")"; } + +# Read REQUIRES from a .info, stripped. Echoes space-separated tokens. +read_requires() { + local info="$1" + # shellcheck disable=SC1090 + ( set +u; source "$info"; echo "${REQUIRES:-}" ) +} + +# ============================================================================= +# DEPENDENCY RESOLUTION +# Builds a topological order over the local SBo tree. +# - transitive REQUIRES +# - %README% recorded, not built +# - deps already in base (installed in chroot base) are satisfied, not built +# - deps neither in SBo tree nor installed => UNMET-DEP +# - cycles detected and reported, no infinite loop +# +# Outputs the order into the global array RESOLVED_ORDER (dir paths). +# Records unmet deps into UNMET (prog -> requiring pkg) and cycle errors. +# Returns 1 if any hard resolution failure (unmet dep, cycle) for the target set. +# ============================================================================= +declare -a RESOLVED_ORDER=() +declare -A UNMET=() # prog -> "needed by X" +declare -a CYCLES=() # human-readable cycle descriptions +declare -A HAS_README=() # dir -> 1 if its REQUIRES carried %README% + +# Is a prog already installed in the base tree? (stock 15.0 or otherwise present) +installed_in_base() { + local prog="$1" + # Package db entries look like prog-version-arch-build[ tag] + ls "$SLACKWARE_BASE"/var/log/packages/ 2>/dev/null \ + | grep -qE "^${prog}-[^-]+-[^-]+-[^-]+" +} + +# DFS topo sort with cycle detection. +# visit_state: dir -> 0 visiting (on stack), 1 done +declare -A _vstate=() +_resolve_visit() { + local dir="$1" parent="$2" + local key; key="$(basename "$dir")" + + if [[ "${_vstate[$dir]:-}" == "1" ]]; then return 0; fi + if [[ "${_vstate[$dir]:-}" == "0" ]]; then + CYCLES+=("cycle involving $key (pulled in via $parent)") + return 1 + fi + _vstate["$dir"]=0 + + local info="$dir/$(basename "$dir").info" + local req tok depdir rc=0 + req="$(read_requires "$info")" + for tok in $req; do + if [[ "$tok" == "%README%" ]]; then + HAS_README["$dir"]=1 + continue + fi + if depdir="$(find_slackbuild_dir "$tok")"; then + _resolve_visit "$depdir" "$key" || rc=1 + elif installed_in_base "$tok"; then + : # satisfied by base, nothing to build + else + UNMET["$tok"]="needed by $key" + rc=1 + fi + done + + _vstate["$dir"]=1 + RESOLVED_ORDER+=("$dir") + return $rc +} + +# Resolve a single target dir into RESOLVED_ORDER (deps first, target last). +resolve_target() { + local dir="$1" + RESOLVED_ORDER=() + CYCLES=() + UNMET=() + _vstate=() + _resolve_visit "$dir" "(top)" +} + +# ============================================================================= +# OVERLAY LIFECYCLE (per target). Patterns from overlay-chroot.sh. +# setup_overlay -> echoes the chroot root path, sets CURRENT_OVERLAY. +# teardown_overlay <tmpdir> -> idempotent unwind in correct reverse order. +# ============================================================================= +setup_overlay() { + local tmpdir; tmpdir="$(mktemp -d "$CHROOT_LOCATION"/sbo-bt.XXXXXX)" + mkdir "$tmpdir"/{changes,tmp,chroot} + + # overlayfs: read-only 15.0 base as lowerdir, disposable upper on top. + mount -t overlay overlay \ + -olowerdir="$SLACKWARE_BASE",upperdir="$tmpdir/changes",workdir="$tmpdir/tmp" \ + "$tmpdir/chroot" + + # bind system dirs + mkdir -p "$tmpdir"/changes/{dev,proc,sys} + local i + for i in dev proc sys; do + mount -o bind "/$i" "$tmpdir/chroot/$i" + done + # /dev/pts (sudo/pty) + mkdir -p "$tmpdir"/changes/dev/pts + mount -o bind /dev/pts "$tmpdir/chroot/dev/pts" + # internet + mount -o bind /etc/resolv.conf "$tmpdir/chroot/etc/resolv.conf" + chroot "$tmpdir/chroot" /bin/bash -c "/usr/sbin/update-ca-certificates --fresh >/dev/null 2>&1" || true + # dbus machine-id + touch "$tmpdir/chroot/var/lib/dbus/machine-id" + mount -o bind /var/lib/dbus/machine-id "$tmpdir/chroot/var/lib/dbus/machine-id" + + CURRENT_OVERLAY="$tmpdir" + ACTIVE_MOUNTS+=("$tmpdir") + echo "$tmpdir" +} + +# Idempotent teardown. Safe to call twice, safe mid-abort. +teardown_overlay() { + local tmpdir="$1" + [[ -n "$tmpdir" && -d "$tmpdir" ]] || return 0 + local c="$tmpdir/chroot" + + mountpoint -q "$c/dev/pts" && umount "$c/dev/pts" + local i + for i in dev proc sys; do + mountpoint -q "$c/$i" && umount "$c/$i" + done + mountpoint -q "$c/etc/resolv.conf" && umount "$c/etc/resolv.conf" + mountpoint -q "$c/var/lib/dbus/machine-id" && umount "$c/var/lib/dbus/machine-id" + mountpoint -q "$c" && umount "$c" + + # Remove from ACTIVE_MOUNTS + local n=() m + for m in "${ACTIVE_MOUNTS[@]}"; do [[ "$m" != "$tmpdir" ]] && n+=("$m"); done + ACTIVE_MOUNTS=("${n[@]:-}") + + rm -rf "$tmpdir" +} + +# Trap: unwind every live overlay on abort. Reverse order of creation. +cleanup_trap() { + [[ $WITH_X -eq 1 ]] && xhost -local:hosts >/dev/null 2>&1 || true + local i + for (( i=${#ACTIVE_MOUNTS[@]}-1; i>=0; i-- )); do + teardown_overlay "${ACTIVE_MOUNTS[$i]}" + done +} +trap cleanup_trap EXIT INT TERM + +# ============================================================================= +# BUILD + INSTALL one package inside an existing overlay chroot. +# Args: <chroot-tmpdir> <slackbuild-dir> +# Sets ST_STATUS / ST_REASON / ST_TIME for the package key. +# Returns 0 on SUCCESS, 1 otherwise. +# ============================================================================= +build_one() { + local tmpdir="$1" dir="$2" + local c="$tmpdir/chroot" + local prog cat key + prog="$(basename "$dir")"; cat="$(category_of "$dir")"; key="$cat/$prog" + local logf="$RUN_DIR/${cat}_${prog}.log" + local start; start=$(date +%s) + + # 1. copy SlackBuild dir into the overlay + local workroot="/sbo-work" + mkdir -p "$c$workroot" + cp -a "$dir" "$c$workroot/$prog" + + # %README% reminder carries through to summary + [[ "${HAS_README[$dir]:-}" == "1" ]] && ST_README["$key"]=1 + + # 2-5. download, md5, build, install all run INSIDE the chroot non-interactively. + # The heredoc script writes a status token to a known file we read back out. + # overlayfs note: if a build fails ONLY here and works on bare 15.0, suspect + # an overlayfs sharp edge (rename/whiteout quirks) rather than a real build bug. + local statf="$c$workroot/$prog.status" + chroot "$c" /bin/bash -s <<CHROOT_EOF >>"$logf" 2>&1 +set -uo pipefail +cd "$workroot/$prog" || { echo "BUILD-FAILED: cannot cd"; echo BUILD-FAILED > "$workroot/$prog.status"; exit 1; } +. ./$prog.info + +# Log resolved build context so the overlay never needs to be kept. +echo "===== sbo-batch-test: $prog =====" +echo "PRGNAM=\${PRGNAM:-$prog} VERSION=\${VERSION:-?} BUILD=\${BUILD:-?} TAG=\${TAG:-?}" +echo "uname -m: \$(uname -m) OUTPUT=\${OUTPUT:-/tmp}" +echo "REQUIRES=\${REQUIRES:-}" +echo "=================================" + +# pick arch-specific download/md5 if present (x86_64 VM) +if [ "\$(uname -m)" = "x86_64" ] && [ -n "\${DOWNLOAD_x86_64:-}" ] && [ "\${DOWNLOAD_x86_64}" != "UNSUPPORTED" ] && [ "\${DOWNLOAD_x86_64}" != "UNTESTED" ]; then + DL="\$DOWNLOAD_x86_64"; MD="\$MD5SUM_x86_64" +else + DL="\$DOWNLOAD"; MD="\$MD5SUM" +fi + +# 2. download +for u in \$DL; do + wget -c --tries=3 "\$u" || { echo DOWNLOAD-FAILED > "$workroot/$prog.status"; exit 1; } +done + +# 3. verify md5 +set -- \$MD +for u in \$DL; do + f="\$(basename "\$u")" + want="\$1"; shift + got="\$(md5sum "\$f" | cut -d' ' -f1)" + if [ "\$got" != "\$want" ]; then + echo "MD5 mismatch on \$f: want \$want got \$got" + echo MD5-MISMATCH > "$workroot/$prog.status"; exit 1 + fi +done + +# 4. build non-interactively +chmod +x ./$prog.SlackBuild +if ! ./$prog.SlackBuild; then + echo BUILD-FAILED > "$workroot/$prog.status"; exit 1 +fi + +# 5. installpkg the resulting package (SlackBuilds write to \${OUTPUT:-/tmp}) +out="\${OUTPUT:-/tmp}" +pkg="\$(ls -t "\$out"/${prog}-*.t?z 2>/dev/null | head -n1)" +if [ -z "\$pkg" ]; then + echo "No package produced in \$out" + echo BUILD-FAILED > "$workroot/$prog.status"; exit 1 +fi +if ! installpkg "\$pkg"; then + echo INSTALL-FAILED > "$workroot/$prog.status"; exit 1 +fi +# Log the installed file list (from the package db) so the overlay is disposable. +echo "===== installed files: \$(basename "\$pkg") =====" +pkgname="\$(basename "\$pkg")"; pkgname="\${pkgname%.t?z}" +cat "/var/log/packages/\$pkgname" 2>/dev/null || echo "(package db entry not found: \$pkgname)" +echo "=================================" +echo SUCCESS > "$workroot/$prog.status" +CHROOT_EOF + + local status="BUILD-FAILED" + [[ -f "$statf" ]] && status="$(cat "$statf")" + local end; end=$(date +%s) + ST_TIME["$key"]=$(( end - start )) + ST_STATUS["$key"]="$status" + [[ "$status" == "SUCCESS" ]] && return 0 + ST_REASON["$key"]="see $(basename "$logf")" + return 1 +} + +# ============================================================================= +# Run one target: fresh overlay, build its resolved chain, teardown. +# Args: <target-slackbuild-dir> +# ============================================================================= +run_target() { + local target_dir="$1" + local tkey; tkey="$(pkg_key "$target_dir")" + + echo + echo "=== Target: $tkey ===" + resolve_target "$target_dir" + local resolve_rc=$? + + # Hard resolution failures: mark target and (newly seen) deps, skip building. + if [[ ${#CYCLES[@]} -gt 0 || ${#UNMET[@]} -gt 0 ]]; then + local why="" + if [[ ${#UNMET[@]} -gt 0 ]]; then + local u + for u in "${!UNMET[@]}"; do why+="unmet:$u(${UNMET[$u]}) "; done + fi + [[ ${#CYCLES[@]} -gt 0 ]] && why+="${CYCLES[*]}" + ST_STATUS["$tkey"]="UNMET-DEP" + ST_REASON["$tkey"]="$why" + echo " resolution failed: $why" + return + fi + + if [[ $DRY_RUN -eq 1 ]]; then + echo " build order:" + local d + for d in "${RESOLVED_ORDER[@]}"; do + echo " $(pkg_key "$d")$([[ "${HAS_README[$d]:-}" == 1 ]] && echo " [%README%]")" + echo "$(pkg_key "$d")" >> "$RUN_DIR/build-order.txt" + done + return + fi + + echo " build order: ${RESOLVED_ORDER[*]##*/}" + for d in "${RESOLVED_ORDER[@]}"; do echo "$(pkg_key "$d")" >> "$RUN_DIR/build-order.txt"; done + + local tmpdir; tmpdir="$(setup_overlay)" + + # Build chain in order. On a failure, mark dependents BLOCKED-BY-DEP. + local d failed_progs=() + for d in "${RESOLVED_ORDER[@]}"; do + local key; key="$(pkg_key "$d")" + local prog; prog="$(basename "$d")" + + # already blocked because an earlier dep it needs failed? + if depends_on_failed "$d" failed_progs; then + ST_STATUS["$key"]="BLOCKED-BY-DEP" + ST_REASON["$key"]="blocked by failed dep" + [[ "${HAS_README[$d]:-}" == "1" ]] && ST_README["$key"]=1 + echo " $key: BLOCKED-BY-DEP" + failed_progs+=("$prog") + continue + fi + + echo " building $key ..." + if build_one "$tmpdir" "$d"; then + echo " $key: ${ST_STATUS[$key]} (${ST_TIME[$key]}s)" + else + echo " $key: ${ST_STATUS[$key]} (${ST_TIME[$key]}s)" + failed_progs+=("$prog") + fi + done + + # Overlay is always torn down: the per-package logs capture everything worth + # inspecting (full build/install output, resolved .info env, installed file + # list), so there is no reason to retain the filesystem. + teardown_overlay "$tmpdir" +} + +# Does SlackBuild dir $1 directly require any prog in failed list (nameref $2)? +# ponytail: direct REQUIRES check only; transitive blocking still works because +# this runs in topo order, so a dep's failure propagates up one hop at a time. +depends_on_failed() { + local dir="$1"; local -n failed="$2" + local info="$dir/$(basename "$dir").info" + local req tok f + req="$(read_requires "$info")" + for tok in $req; do + [[ "$tok" == "%README%" ]] && continue + for f in "${failed[@]:-}"; do + [[ "$tok" == "$f" ]] && return 0 + done + done + return 1 +} + +# ============================================================================= +# SUMMARY (screen color + plain summary.log) +# ============================================================================= +print_summary() { + local total=$SECONDS + local succ=0 fail=0 blocked=0 + local summary="$RUN_DIR/summary.log" + + { + echo "sbo-batch-test run summary" + echo "target: $TARGET_ARG" + echo + } > "$summary" + + echo + echo "================ SUMMARY ================" + local key + for key in "${!ST_STATUS[@]}"; do + local st="${ST_STATUS[$key]}" rsn="${ST_REASON[$key]:-}" t="${ST_TIME[$key]:-0}" + local rd=""; [[ "${ST_README[$key]:-}" == "1" ]] && rd=" [%README%]" + local col="$C_YEL" + case "$st" in + SUCCESS) col="$C_GRN"; ((succ++)) ;; + BLOCKED-BY-DEP|UNMET-DEP) col="$C_YEL"; ((blocked++)) ;; + *) col="$C_RED"; ((fail++)) ;; + esac + printf "%s%-30s %-16s%s %s%s (%ss)\n" "$col" "$key" "$st" "$C_RST" "$rsn" "$rd" "$t" + printf "%-30s %-16s %s%s (%ss)\n" "$key" "$st" "$rsn" "$rd" "$t" >> "$summary" + done + echo "----------------------------------------" + printf "%s%d succeeded%s, %s%d failed%s, %s%d blocked%s, total %ss\n" \ + "$C_GRN" "$succ" "$C_RST" "$C_RED" "$fail" "$C_RST" "$C_YEL" "$blocked" "$C_RST" "$total" + echo "logs: $RUN_DIR" + { + echo + echo "$succ succeeded, $fail failed, $blocked blocked, total ${total}s" + echo "logs: $RUN_DIR" + } >> "$summary" +} + +# ============================================================================= +# main +# ============================================================================= +main() { + parse_args "$@" + init_color + validate_env + + RUN_DIR="$LOG_ROOT/$(date +%Y-%m-%d_%H-%M-%S)" + mkdir -p "$RUN_DIR" + : > "$RUN_DIR/build-order.txt" + + [[ $DRY_RUN -eq 0 ]] && update_base + + [[ $WITH_X -eq 1 ]] && { echo "X passthrough on (xhost +local:hosts). Security: allows local connections to your X server."; xhost +local:hosts >/dev/null; } + + # Collect targets. + local -a targets=() + if [[ -d "$TARGET_ARG" ]]; then + # category-folder mode: every subdir with a matching .info is a target + local d prog + for d in "$TARGET_ARG"/*; do + [[ -d "$d" ]] || continue + prog="$(basename "$d")" + [[ -f "$d/$prog.info" ]] && targets+=("$d") + done + if [[ ${#targets[@]} -eq 0 ]]; then + echo "No SlackBuild targets found in folder: $TARGET_ARG" >&2 + exit 1 + fi + else + # single-package mode: resolve the name in the tree + local tdir + if tdir="$(find_slackbuild_dir "$TARGET_ARG")"; then + targets+=("$tdir") + else + echo "Program not found in any SBo tree root: $TARGET_ARG" >&2 + exit 1 + fi + fi + + # TODO: "all" mode and queue/list-file mode plug in here (populate `targets`). + + local t + for t in "${targets[@]}"; do + run_target "$t" + done + + print_summary +} + +main "$@" diff --git a/test-logic.sh b/test-logic.sh new file mode 100755 index 0000000..13d97f8 --- /dev/null +++ b/test-logic.sh @@ -0,0 +1,111 @@ +#!/bin/bash +# +# Logic self-check for sbo-batch-test. Covers the pure, VM-independent parts: +# dependency resolution (topo order, %README%, unmet-dep, cycles) and +# BLOCKED-BY-DEP propagation (depends_on_failed). No overlay, no chroot. +# +# Run: bash test-logic.sh +# +set -uo pipefail + +SCRIPT="$(dirname "$0")/sbo-batch-test" +T=$(mktemp -d) +BASE=$(mktemp -d); mkdir -p "$BASE/var/log/packages" + +cleanup() { rm -rf "$T" "$BASE"; } +trap cleanup EXIT + +# Fake SBo tree under one category. mk <prog> "<REQUIRES>". +mk() { mkdir -p "$T/cat/$1"; echo "REQUIRES=\"$2\"" > "$T/cat/$1/$1.info"; } + +# Graph: +# c (no deps) +# b -> c +# a -> b, %README% +# d -> nonexistentpkg (unmet) +# e -> f, f -> e (cycle) +# g -> b (for blocked-by-dep: if b fails, g blocks) +mk c "" +mk b "c" +mk a "b %README%" +mk d "nonexistentpkg" +mk e "f" +mk f "e" +mk g "b" + +# Source the script without running main(), then override config AFTER the +# source (sourcing re-runs the CONFIG block, which would clobber test vars). +LIB=$(mktemp) +sed '/^main "\$@"$/d' "$SCRIPT" > "$LIB" +# shellcheck disable=SC1090 +source "$LIB" 2>/dev/null +rm -f "$LIB" +SBO_TREE_ROOTS=("$T") +SLACKWARE_BASE="$BASE" + +pass=0; fail=0 +ok() { echo " ok: $1"; ((pass++)); return 0; } +bad() { echo " FAIL: $1"; ((fail++)); return 0; } + +# --- resolution ------------------------------------------------------------- +resolve_target "$T/cat/a" +order=""; for x in "${RESOLVED_ORDER[@]}"; do order+="$(basename "$x") "; done +order="${order% }" # trim trailing space +[[ "$order" == "c b a" ]] && ok "topo order c b a" || bad "topo order, got: [$order]" +[[ "${HAS_README[$T/cat/a]:-}" == "1" ]] && ok "%README% recorded" || bad "%README% not recorded" +[[ ${#UNMET[@]} -eq 0 ]] && ok "no false unmet" || bad "unexpected unmet" + +resolve_target "$T/cat/d" +[[ ${#UNMET[@]} -eq 1 ]] && ok "unmet-dep caught" || bad "unmet-dep missed" + +resolve_target "$T/cat/e" +[[ ${#CYCLES[@]} -ge 1 ]] && ok "cycle caught" || bad "cycle missed" + +# --- BLOCKED-BY-DEP (depends_on_failed) ------------------------------------- +# depends_on_failed <slackbuild-dir> <nameref-to-dead-prog-array> +# returns 0 if the dir directly REQUIRES any prog in the dead list. + +dead=(b) +if depends_on_failed "$T/cat/g" dead; then ok "g blocked when b dead"; else bad "g should block on b"; fi +if depends_on_failed "$T/cat/a" dead; then ok "a blocked when b dead (direct dep)"; else bad "a should block on b"; fi + +dead=(c) +# a does NOT directly require c (a->b->c). One-hop check must say no here. +if depends_on_failed "$T/cat/a" dead; then bad "a wrongly blocked on c (not a direct dep)"; else ok "a not directly blocked by c"; fi +# but b DOES directly require c. +if depends_on_failed "$T/cat/b" dead; then ok "b blocked when c dead"; else bad "b should block on c"; fi + +dead=() +if depends_on_failed "$T/cat/a" dead; then bad "a blocked with empty dead list"; else ok "no block when nothing dead"; fi + +# %README% token in REQUIRES must not be treated as a dead dep. +dead=("%README%") +if depends_on_failed "$T/cat/a" dead; then bad "%README% treated as dep"; else ok "%README% not treated as dep"; fi + +# Propagation invariant note: depends_on_failed is a DIRECT-requires check only. +# Transitive blocking works because run_target iterates in topo order and adds +# each blocked package's own prog name to `dead`, so the failure of c blocks b +# (direct), then b's name enters `dead`, which then blocks a (direct on b). +# Simulate that one-hop cascade here: +dead=(c) +# topo order for a is: c b a. c "fails" -> seed dead=(c). +chain=(c b a) +declare -A blocked=() +for p in "${chain[@]}"; do + [[ "$p" == "c" ]] && continue # c is the original failure, already in dead + if depends_on_failed "$T/cat/$p" dead; then + blocked[$p]=1 + dead+=("$p") # mark dependents-of-this as blockable next hop + fi +done +if [[ "${blocked[b]:-}" == "1" && "${blocked[a]:-}" == "1" ]]; then + ok "transitive cascade c->b->a via one-hop propagation" +else + bad "cascade dead: b=${blocked[b]:-0} a=${blocked[a]:-0}" +fi + +# --- result ----------------------------------------------------------------- +echo +echo "$pass passed, $fail failed" +[[ $fail -eq 0 ]] || exit 1 +echo "ALL LOGIC CHECKS PASS" |