diff options
| -rw-r--r-- | .extras/docs/plans/2026-07-13-docker-test-build.md | 1579 | ||||
| -rw-r--r-- | .extras/docs/plans/2026-07-13-image-builder.md | 790 | ||||
| -rw-r--r-- | .extras/docs/specs/2026-07-13-docker-test-build-design.md | 258 | ||||
| -rw-r--r-- | .extras/docs/specs/2026-07-13-image-builder-design.md | 216 | ||||
| -rw-r--r-- | .extras/image-builder/README | 69 | ||||
| -rwxr-xr-x | .extras/image-builder/bootstrap.sh | 552 | ||||
| -rwxr-xr-x | .extras/image-builder/build-full-image.sh | 233 | ||||
| -rwxr-xr-x | .extras/image-builder/build-sbo-testbuild.sh | 131 | ||||
| -rw-r--r-- | .extras/image-builder/config | 15 | ||||
| -rw-r--r-- | .extras/image-builder/lib.sh | 72 | ||||
| -rwxr-xr-x | .extras/image-builder/test-image-builder.sh | 68 | ||||
| -rw-r--r-- | .extras/overrides.example | 16 | ||||
| -rwxr-xr-x | .extras/test-build | 864 | ||||
| -rw-r--r-- | .extras/test-build-config.example | 18 | ||||
| -rw-r--r-- | .extras/test-logic.sh | 215 | ||||
| -rw-r--r-- | CLAUDE.md | 43 |
16 files changed, 23 insertions, 5116 deletions
diff --git a/.extras/docs/plans/2026-07-13-docker-test-build.md b/.extras/docs/plans/2026-07-13-docker-test-build.md deleted file mode 100644 index 6195de9..0000000 --- a/.extras/docs/plans/2026-07-13-docker-test-build.md +++ /dev/null @@ -1,1579 +0,0 @@ -# Docker test-build Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** A single bash script `.extras/test-build` that verifies an already-published SBo package still builds cleanly on `-current` or `15.0` inside a throwaway docker container, resolving and building its SBo deps from the local tree, caching built deps per image digest, and reporting per-package status + lint. - -**Architecture:** Host resolves the dep tree from the local SBo tree (logic ported from `sbo-batch-test`), applies current-vs-stable override rules, prints the build order and confirms, then runs one `docker run --rm` that builds deps (or installs them from a host cache) and the target, lints the target, and streams results back. The container is the disposable environment (no overlayfs). Images are built by a separate job and consumed by tag. - -**Tech Stack:** Bash, docker, Slackware `installpkg`/`makepkg`, `sbo-maintainer-tools` (`sbopkglint`). Self-check is plain bash (`test-logic.sh`), no framework. - ---- - -## Reference source - -Port from `/home/danix/Programming/GIT/sbo-batch-tester/sbo-batch-test`. These -functions are lifted with minimal change (they are pure or host-side): - -- `find_slackbuild_dir`, `category_of`, `pkg_key`, `read_requires`, - `version_of`, `installed_in_base` -- resolver: `_resolve_visit`, `resolve_target`, globals `RESOLVED_ORDER`, - `UNMET`, `CYCLES`, `HAS_README`, `_vstate` -- cache: `_cache_ver_of`, `cache_decision`, `cache_path`, `cache_store`, - `cache_label` -- `depends_on_failed`, `print_summary`, `lint_pkg`, `init_color` -- self-check patterns from - `/home/danix/Programming/GIT/sbo-batch-tester/test-logic.sh` - -What is NEW or CHANGED in this repo: - -- overlay lifecycle (`setup_overlay`/`teardown_overlay`/`cleanup_trap`) is - REMOVED. The container replaces it. -- `build_one` is rewritten to run its build steps via `docker run` instead of - `chroot`. -- image + tree selection by version (`--stable`). -- override engine (drop / rename / fetch) applied to the resolved order. -- cache namespaced by image digest. -- config keys differ (see Task 2). - -## File Structure - -- Create: `/home/danix/Programming/GIT/sbo-slackbuilds/.extras/test-build` - (the script, executable) -- Create: - `/home/danix/Programming/GIT/sbo-slackbuilds/.extras/test-build-config.example` - (config template) -- Create: - `/home/danix/Programming/GIT/sbo-slackbuilds/.extras/overrides.example` - (override-rules template) -- Create: - `/home/danix/Programming/GIT/sbo-slackbuilds/.extras/test-logic.sh` - (self-check) - -All work happens in `.extras/`. Run every command from the repo root -`/home/danix/Programming/GIT/sbo-slackbuilds`. - -Note on committing: this repo has a pre-commit hook that runs `sbolint` on -staged packages. `.extras/` files are not packages, but to be safe every -commit in this plan uses `SBOLINT=no git commit ...`. Commits are GPG-signed -automatically; do not disable signing. - ---- - -## Task 1: Scaffold the script + smoke self-check - -**Files:** -- Create: `.extras/test-build` -- Create: `.extras/test-logic.sh` - -- [ ] **Step 1: Write the failing self-check** - -Create `.extras/test-logic.sh`: - -```bash -#!/bin/bash -# -# Logic self-check for .extras/test-build. Covers the pure, host-side parts: -# dependency resolution, override application, unknown-dep -> UNMET, cache -# decisions, and BLOCKED-BY-DEP propagation. No docker. -# -# Run: bash .extras/test-logic.sh -# -set -uo pipefail - -SCRIPT="$(dirname "$0")/test-build" -T=$(mktemp -d) -BASE_DB=$(mktemp -d); mkdir -p "$BASE_DB" - -cleanup() { rm -rf "$T" "$BASE_DB"; } -trap cleanup EXIT - -# Source the script without running main(). Sourcing re-runs the CONFIG block, -# so set test vars AFTER the source. -LIB=$(mktemp) -sed '/^main "\$@"$/d' "$SCRIPT" > "$LIB" -# shellcheck disable=SC1090 -source "$LIB" 2>/dev/null -rm -f "$LIB" - -pass=0; fail=0 -ok() { echo " ok: $1"; ((pass++)); return 0; } -bad() { echo " FAIL: $1"; ((fail++)); return 0; } - -# Placeholder assertion so this file runs before any logic exists. -ok "script sources without executing main" - -echo -echo "$pass passed, $fail failed" -[[ $fail -eq 0 ]] || exit 1 -echo "ALL LOGIC CHECKS PASS" -``` - -- [ ] **Step 2: Run it to verify it fails** - -Run: `bash .extras/test-logic.sh` -Expected: FAIL. `$SCRIPT` does not exist yet, `sed`/`source` on a missing file -errors, no "ALL LOGIC CHECKS PASS". - -- [ ] **Step 3: Write the minimal script skeleton** - -Create `.extras/test-build`: - -```bash -#!/bin/bash -# -# test-build - verify an already-published SBo package still builds on a target -# Slackware version inside a throwaway docker container. Resolves + builds its -# SBo deps from the local tree, caches built deps per image digest, reports -# per-package status and lints the target. -# -# Dependency-resolution, cache, and summary logic are adapted from sbo-batch-test -# (github: danixland). The overlay chroot is replaced by a docker container: the -# container IS the disposable environment, so no overlayfs. -# -# No em dashes in prose by author convention. - -# ============================================================================= -# CONFIG (do not edit here; real values live in the external config file) -# ============================================================================= -SBO_TREE_CURRENT="" -SBO_TREE_STABLE="" -IMAGE_CURRENT="" -IMAGE_STABLE="" -LOG_ROOT="/var/log/sbo-test-build" -PKG_CACHE="" - -TB_CONFIG="${TB_CONFIG:-$HOME/.config/sbo-testbuild/config}" -if [[ -f "$TB_CONFIG" ]]; then - # shellcheck disable=SC1090 - source "$TB_CONFIG" -fi - -TB_OVERRIDES="${TB_OVERRIDES:-$HOME/.config/sbo-testbuild/overrides}" - -# ============================================================================= -set -uo pipefail -# Not -e: a package build failing is a handled outcome, not a script crash. - -main() { - : -} - -main "$@" -``` - -- [ ] **Step 4: Run the self-check to verify it passes** - -Run: `bash .extras/test-logic.sh` -Expected: PASS, ending `ALL LOGIC CHECKS PASS`. - -- [ ] **Step 5: Commit** - -```bash -chmod +x .extras/test-build -git add .extras/test-build .extras/test-logic.sh -SBOLINT=no git commit -m 'test-build: scaffold script + self-check' -``` - ---- - -## Task 2: Config, arg parsing, version selection - -**Files:** -- Modify: `.extras/test-build` -- Create: `.extras/test-build-config.example` -- Test: `.extras/test-logic.sh` - -- [ ] **Step 1: Write the failing test** - -Append to `.extras/test-logic.sh` before the `--- result ---` block (the -`echo; "$pass passed..."` lines at the end): - -```bash -# --- version selection ------------------------------------------------------ -SBO_TREE_CURRENT="/trees/current" -SBO_TREE_STABLE="/trees/stable" -IMAGE_CURRENT="sbo-testbuild:current" -IMAGE_STABLE="sbo-testbuild:15.0" - -VERSION_ID="current"; select_version_paths -[[ "$ACTIVE_TREE" == "/trees/current" ]] && ok "current -> current tree" || bad "current tree wrong: [$ACTIVE_TREE]" -[[ "$ACTIVE_IMAGE" == "sbo-testbuild:current" ]] && ok "current -> current image" || bad "current image wrong: [$ACTIVE_IMAGE]" - -VERSION_ID="15.0"; select_version_paths -[[ "$ACTIVE_TREE" == "/trees/stable" ]] && ok "15.0 -> stable tree" || bad "stable tree wrong: [$ACTIVE_TREE]" -[[ "$ACTIVE_IMAGE" == "sbo-testbuild:15.0" ]] && ok "15.0 -> stable image" || bad "stable image wrong: [$ACTIVE_IMAGE]" -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `bash .extras/test-logic.sh` -Expected: FAIL, `select_version_paths: command not found`. - -- [ ] **Step 3: Implement config globals, flags, parse_args, select_version_paths** - -In `.extras/test-build`, replace the `main() { : }` line and everything up to -`main "$@"` with the flags/globals and functions below (keep the CONFIG block -above untouched): - -```bash -# ---- flags / globals -------------------------------------------------------- -USE_COLOR=1 # --no-color or non-TTY disables -DRY_RUN=0 # --dry-run: resolve + print order, do not build -ASSUME_YES=0 # --yes: skip the confirm prompt (still prints the order) -USE_CACHE=1 # --no-cache disables the dep cache for one run -VERSION_ID="current" # "current" | "15.0"; set by --stable -TARGET_ARG="" - -ACTIVE_TREE="" # selected SBo tree (by version) -ACTIVE_IMAGE="" # selected image tag (by version) -RUN_DIR="" # timestamped log dir for this run - -# Status tracking. Keyed by "category/prog". Parallel assoc arrays. -declare -A ST_STATUS=() -declare -A ST_REASON=() -declare -A ST_TIME=() -declare -A ST_README=() - -usage() { - cat <<'EOF' -test-build - verify an SBo package builds on a target Slackware in docker - -USAGE: - test-build [OPTIONS] <program-name> - -OPTIONS: - -h, --help This text. - --stable Target Slackware 15.0 (image + tree). Default is -current. - --dry-run Resolve, apply overrides, print the build order, do not build. - --yes Skip the confirm prompt (the order is still printed first). - --no-cache Rebuild all deps this run, ignore/refresh the cache. - --no-color Disable ANSI color (auto-disabled when stdout is not a TTY). -EOF -} - -parse_args() { - while [[ $# -gt 0 ]]; do - case "$1" in - -h|--help) usage; exit 0 ;; - --stable|15.0) VERSION_ID="15.0"; shift ;; - --dry-run) DRY_RUN=1; shift ;; - --yes) ASSUME_YES=1; shift ;; - --no-cache) USE_CACHE=0; shift ;; - --no-color) USE_COLOR=0; shift ;; - -*) echo "Unknown option: $1" >&2; usage >&2; exit 2 ;; - *) - if [[ -n "$TARGET_ARG" ]]; then - echo "Only one target accepted (got '$TARGET_ARG' and '$1')." >&2 - exit 2 - fi - TARGET_ARG="$1"; shift ;; - esac - done - if [[ -z "$TARGET_ARG" ]]; then - echo "No target given." >&2; usage >&2; exit 2 - fi -} - -init_color() { - if [[ $USE_COLOR -eq 1 && -t 1 ]]; then - C_RED=$'\e[31m'; C_GRN=$'\e[32m'; C_YEL=$'\e[33m'; C_RST=$'\e[0m' - else - C_RED=""; C_GRN=""; C_YEL=""; C_RST="" - fi -} - -# Map VERSION_ID to the active tree + image. No I/O, unit-testable. -select_version_paths() { - if [[ "$VERSION_ID" == "15.0" ]]; then - ACTIVE_TREE="$SBO_TREE_STABLE"; ACTIVE_IMAGE="$IMAGE_STABLE" - else - ACTIVE_TREE="$SBO_TREE_CURRENT"; ACTIVE_IMAGE="$IMAGE_CURRENT" - fi -} - -# require_config: the external config must exist and set the version's paths. -require_config() { - if [[ ! -f "$TB_CONFIG" ]]; then - cat >&2 <<EOF -No config file: $TB_CONFIG -Copy the example and edit it: - - mkdir -p "\$(dirname "$TB_CONFIG")" - cp .extras/test-build-config.example "$TB_CONFIG" - \${EDITOR:-vi} "$TB_CONFIG" -EOF - exit 1 - fi - select_version_paths - if [[ -z "$ACTIVE_TREE" || -z "$ACTIVE_IMAGE" ]]; then - echo "Config $TB_CONFIG is missing the tree/image for version '$VERSION_ID'." >&2 - exit 1 - fi - if [[ ! -d "$ACTIVE_TREE" ]]; then - echo "SBo tree for '$VERSION_ID' does not exist: $ACTIVE_TREE" >&2 - exit 1 - fi -} - -main() { - parse_args "$@" - init_color - require_config -} - -main "$@" -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `bash .extras/test-logic.sh` -Expected: PASS (the new version-selection checks included). - -- [ ] **Step 5: Write the config example** - -Create `.extras/test-build-config.example`: - -```bash -# test-build config. Copy to ~/.config/sbo-testbuild/config and edit. -# Override the path with the TB_CONFIG environment variable. - -# Local SBo trees, one per target Slackware version. -SBO_TREE_CURRENT="/home/danix/SBo/current" -SBO_TREE_STABLE="/home/danix/SBo/15.0" - -# Ready image tags (built by a separate job, consumed here by tag). -IMAGE_CURRENT="sbo-testbuild:current" -IMAGE_STABLE="sbo-testbuild:15.0" - -# Where per-run logs land. -LOG_ROOT="/home/danix/.cache/sbo-test-build" - -# Dependency package cache (host dir). Empty disables caching. -# Namespaced internally by image digest, so it self-invalidates on image update -# and keeps current vs 15.0 separate. -PKG_CACHE="/home/danix/.cache/sbo-test-build/pkgcache" -``` - -- [ ] **Step 6: Commit** - -```bash -git add .extras/test-build .extras/test-build-config.example .extras/test-logic.sh -SBOLINT=no git commit -m 'test-build: config, arg parsing, version selection' -``` - ---- - -## Task 3: Port the SBo-tree lookup + resolver + self-check - -**Files:** -- Modify: `.extras/test-build` -- Test: `.extras/test-logic.sh` - -- [ ] **Step 1: Write the failing test** - -Append to `.extras/test-logic.sh` before the result block: - -```bash -# --- resolution ------------------------------------------------------------- -# Fake SBo tree under one category. mk <prog> "<REQUIRES>". -mk() { mkdir -p "$T/cat/$1"; echo "REQUIRES=\"$2\"" > "$T/cat/$1/$1.info"; } -mk c "" -mk b "c" -mk a "b %README%" -mk d "nonexistentpkg" -mk e "f" -mk f "e" -mk g "b" - -SBO_TREE_ROOTS=("$T") # resolver reads this global (set by select in real runs) - -resolve_target "$T/cat/a" -order=""; for x in "${RESOLVED_ORDER[@]}"; do order+="$(basename "$x") "; done -order="${order% }" -[[ "$order" == "c b a" ]] && ok "topo order c b a" || bad "topo order, got: [$order]" -[[ "${HAS_README[$T/cat/a]:-}" == "1" ]] && ok "%README% recorded" || bad "%README% not recorded" -[[ ${#UNMET[@]} -eq 0 ]] && ok "no false unmet" || bad "unexpected unmet" - -resolve_target "$T/cat/d" -[[ ${#UNMET[@]} -eq 1 ]] && ok "unmet-dep caught" || bad "unmet-dep missed" - -resolve_target "$T/cat/e" -[[ ${#CYCLES[@]} -ge 1 ]] && ok "cycle caught" || bad "cycle missed" -``` - -Note: the base-installed check (`installed_in_base`) is exercised in real runs -against the container, not here. In the self-check `BASE_DB` is empty, so a dep -that is neither in the tree nor a known override reads as UNMET (which is the -behavior we test with `d`). - -- [ ] **Step 2: Run test to verify it fails** - -Run: `bash .extras/test-logic.sh` -Expected: FAIL, `resolve_target: command not found`. - -- [ ] **Step 3: Implement lookup + resolver (ported)** - -Add to `.extras/test-build` above `main()`: - -```bash -# ============================================================================= -# SBo tree lookup -# ============================================================================= -# In this tool there is one active tree per run, but the resolver reads an array -# named SBO_TREE_ROOTS so the ported logic and its self-check match sbo-batch-test. -# main() sets SBO_TREE_ROOTS=("$ACTIVE_TREE") after require_config. -declare -a SBO_TREE_ROOTS=() - -find_slackbuild_dir() { - local prog="$1" root d - for root in "${SBO_TREE_ROOTS[@]}"; do - [[ -d "$root" ]] || continue - for d in "$root"/*/"$prog"; do - if [[ -d "$d" && -f "$d/$prog.info" ]]; then - echo "$d"; return 0 - fi - done - done - return 1 -} - -category_of() { basename "$(dirname "$1")"; } -pkg_key() { echo "$(category_of "$1")/$(basename "$1")"; } - -read_requires() { - local info="$1" - # shellcheck disable=SC1090 - ( set +u; source "$info"; echo "${REQUIRES:-}" ) -} - -version_of() { - local dir="$1" - local info="$dir/$(basename "$dir").info" - [[ -f "$info" ]] || return - local v; v="$(grep -m1 '^VERSION=' "$info" | cut -d'"' -f2)" - echo "$v" -} - -# ============================================================================= -# DEPENDENCY RESOLUTION (topo sort + cycle detection, LOCAL tree only) -# ============================================================================= -declare -a RESOLVED_ORDER=() -declare -A UNMET=() -declare -a CYCLES=() -declare -A HAS_README=() -declare -A _vstate=() - -# Is a prog already present in the container base? Overridden at build time to -# consult the image; in resolution we treat "in base" as a callback so the pure -# topo logic stays testable. Default: not in base (self-check has no base db). -installed_in_base() { return 1; } - -_resolve_visit() { - local dir="$1" parent="$2" - local key; key="$(basename "$dir")" - - if [[ "${_vstate[$dir]:-}" == "1" ]]; then return 0; fi - if [[ "${_vstate[$dir]:-}" == "0" ]]; then - CYCLES+=("cycle involving $key (pulled in via $parent)") - return 1 - fi - _vstate["$dir"]=0 - - local info="$dir/$(basename "$dir").info" - local req tok depdir rc=0 - req="$(read_requires "$info")" - for tok in $req; do - if [[ "$tok" == "%README%" ]]; then - HAS_README["$dir"]=1 - continue - fi - if depdir="$(find_slackbuild_dir "$tok")"; then - _resolve_visit "$depdir" "$key" || rc=1 - elif installed_in_base "$tok"; then - : - else - UNMET["$tok"]="needed by $key" - rc=1 - fi - done - - _vstate["$dir"]=1 - RESOLVED_ORDER+=("$dir") - return $rc -} - -resolve_target() { - local dir="$1" - RESOLVED_ORDER=() - CYCLES=() - UNMET=() - _vstate=() - _resolve_visit "$dir" "(top)" -} -``` - -Update `main()` to set the resolver's tree array after `require_config`: - -```bash -main() { - parse_args "$@" - init_color - require_config - SBO_TREE_ROOTS=("$ACTIVE_TREE") -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `bash .extras/test-logic.sh` -Expected: PASS (topo order, %README%, unmet, cycle checks green). - -- [ ] **Step 5: Commit** - -```bash -git add .extras/test-build .extras/test-logic.sh -SBOLINT=no git commit -m 'test-build: port SBo lookup + dependency resolver' -``` - ---- - -## Task 4: Override engine (drop / rename / fetch) + UNMET stop - -**Files:** -- Modify: `.extras/test-build` -- Create: `.extras/overrides.example` -- Test: `.extras/test-logic.sh` - -The override file encodes current-vs-stable deltas. It is applied ONLY when -`VERSION_ID == current` (15.0 is the SBo baseline). Rules: - -- `drop: <prog>` remove from the build order (already in -current base) -- `rename: <old> -> <new>` rewrite the dep name before it is looked up -- `fetch: <prog>` mark this dep to come from sbopkg in-container, not the local tree - -- [ ] **Step 1: Write the failing test** - -Append to `.extras/test-logic.sh` before the result block: - -```bash -# --- overrides -------------------------------------------------------------- -OV=$(mktemp) -cat > "$OV" <<'EOF' -# comment ignored -drop: dropme -rename: oldname -> newname -fetch: fetchme -EOF -TB_OVERRIDES="$OV" -load_overrides - -[[ "${OV_DROP[dropme]:-}" == "1" ]] && ok "override drop parsed" || bad "drop not parsed" -[[ "${OV_RENAME[oldname]:-}" == "newname" ]] && ok "override rename parsed" || bad "rename not parsed" -[[ "${OV_FETCH[fetchme]:-}" == "1" ]] && ok "override fetch parsed" || bad "fetch not parsed" - -# rename maps a token -[[ "$(apply_rename oldname)" == "newname" ]] && ok "apply_rename maps" || bad "apply_rename wrong: [$(apply_rename oldname)]" -[[ "$(apply_rename untouched)" == "untouched" ]] && ok "apply_rename passthrough" || bad "apply_rename mangled untouched" - -# drop filters the order (build a fake order of dirs; dropme is removed) -mk dropme "" -mk keepme "" -RESOLVED_ORDER=("$T/cat/dropme" "$T/cat/keepme") -apply_overrides_to_order -kept=""; for x in "${RESOLVED_ORDER[@]}"; do kept+="$(basename "$x") "; done -kept="${kept% }" -[[ "$kept" == "keepme" ]] && ok "drop removes from order" || bad "drop failed, order=[$kept]" -rm -f "$OV" - -# overrides only apply on current; on 15.0 they are a no-op -VERSION_ID="15.0" -TB_OVERRIDES="$OV" # file gone; must not error, must clear maps -load_overrides -[[ ${#OV_DROP[@]} -eq 0 ]] && ok "overrides inert on 15.0" || bad "overrides applied on 15.0" -VERSION_ID="current" -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `bash .extras/test-logic.sh` -Expected: FAIL, `load_overrides: command not found`. - -- [ ] **Step 3: Implement the override engine** - -Add to `.extras/test-build` above `main()`: - -```bash -# ============================================================================= -# current-vs-stable overrides. Parsed from $TB_OVERRIDES. Applied only when -# targeting -current (15.0 is the SBo baseline, no deltas). -# ============================================================================= -declare -A OV_DROP=() # prog -> 1 -declare -A OV_RENAME=() # old -> new -declare -A OV_FETCH=() # prog -> 1 - -load_overrides() { - OV_DROP=(); OV_RENAME=(); OV_FETCH=() - # 15.0 is the baseline: no overrides. - [[ "$VERSION_ID" == "15.0" ]] && return - [[ -f "$TB_OVERRIDES" ]] || return - local line kind rest - while IFS= read -r line; do - line="${line%%#*}" # strip comments - line="${line#"${line%%[![:space:]]*}"}" # ltrim - [[ -z "$line" ]] && continue - kind="${line%%:*}"; rest="${line#*:}" - kind="${kind//[[:space:]]/}" - rest="${rest#"${rest%%[![:space:]]*}"}" # ltrim value - case "$kind" in - drop) OV_DROP["${rest//[[:space:]]/}"]=1 ;; - fetch) OV_FETCH["${rest//[[:space:]]/}"]=1 ;; - rename) - # rest is "old -> new" - local old new - old="${rest%%->*}"; new="${rest##*->}" - old="${old//[[:space:]]/}"; new="${new//[[:space:]]/}" - [[ -n "$old" && -n "$new" ]] && OV_RENAME["$old"]="$new" ;; - *) echo "WARN: unknown override rule: $line" >&2 ;; - esac - done < "$TB_OVERRIDES" -} - -# Map a dep token through rename rules (identity if no rule). -apply_rename() { - local tok="$1" - echo "${OV_RENAME[$tok]:-$tok}" -} - -# Remove dropped packages from RESOLVED_ORDER in place. -apply_overrides_to_order() { - local d prog keep=() - for d in "${RESOLVED_ORDER[@]}"; do - prog="$(basename "$d")" - [[ "${OV_DROP[$prog]:-}" == "1" ]] && continue - keep+=("$d") - done - RESOLVED_ORDER=("${keep[@]:-}") -} -``` - -Wire `apply_rename` into the resolver so renamed deps look up under the new -name. In `_resolve_visit`, change the dep-lookup line: - -```bash - for tok in $req; do - if [[ "$tok" == "%README%" ]]; then - HAS_README["$dir"]=1 - continue - fi - tok="$(apply_rename "$tok")" # <-- add this line - if [[ "${OV_FETCH[$tok]:-}" == "1" ]]; then - FETCH_DEPS["$tok"]=1 # satisfied via sbopkg at build time - continue - fi - if depdir="$(find_slackbuild_dir "$tok")"; then - _resolve_visit "$depdir" "$key" || rc=1 - elif installed_in_base "$tok"; then - : - else - UNMET["$tok"]="needed by $key" - rc=1 - fi - done -``` - -Add the `FETCH_DEPS` global with the other resolver globals: - -```bash -declare -A FETCH_DEPS=() # prog -> 1: resolve via sbopkg in-container -``` - -And reset it in `resolve_target`: - -```bash -resolve_target() { - local dir="$1" - RESOLVED_ORDER=() - CYCLES=() - UNMET=() - FETCH_DEPS=() - _vstate=() - _resolve_visit "$dir" "(top)" -} -``` - -Load overrides in `main()` (before any resolve happens, after tree setup): - -```bash -main() { - parse_args "$@" - init_color - require_config - SBO_TREE_ROOTS=("$ACTIVE_TREE") - load_overrides -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `bash .extras/test-logic.sh` -Expected: PASS (all override checks green). - -- [ ] **Step 5: Write the overrides example** - -Create `.extras/overrides.example`: - -``` -# current-vs-stable dep overrides for test-build. -# Copy to ~/.config/sbo-testbuild/overrides and edit. Applied only when -# targeting -current (15.0 is the SBo baseline). One rule per line. -# -# drop: <prog> dep is in 15.0 but already in the -current base -# rename: <old> -> <new> dep was renamed on -current -# fetch: <prog> dep was removed from the -current tree; pull via sbopkg -# -# Start light: add rules only when a real package needs them. - -# drop: rust -# rename: python3-foo -> foo -# fetch: somelib -``` - -- [ ] **Step 6: Commit** - -```bash -git add .extras/test-build .extras/overrides.example .extras/test-logic.sh -SBOLINT=no git commit -m 'test-build: current-vs-stable override engine' -``` - ---- - -## Task 5: Port the dep cache, namespaced by image digest - -**Files:** -- Modify: `.extras/test-build` -- Test: `.extras/test-logic.sh` - -The cache is the ported sbo-batch-test cache with one change: the cache root -seen by `cache_*` is `$PKG_CACHE/<image-digest>`, computed once per run. A new -digest gives a fresh namespace (old deps ignored), which is the invalidation. - -- [ ] **Step 1: Write the failing test** - -Append to `.extras/test-logic.sh` before the result block: - -```bash -# --- cache ------------------------------------------------------------------ -PKG_CACHE=$(mktemp -d) -CACHE_ROOT="$PKG_CACHE/sha256-deadbeef" # simulate a resolved digest namespace -mkc() { mkdir -p "$CACHE_ROOT/$1/$2"; : > "$CACHE_ROOT/$1/$2/$3"; } - -mkc net libfoo "libfoo-1.1-x86_64-1_danix.txz" -[[ "$(cache_decision net libfoo 1.1)" == "cached" ]] && ok "cache hit on version match" || bad "cache_decision got [$(cache_decision net libfoo 1.1)]" -[[ "$(cache_decision net libfoo 1.2)" == "bump:1.1:1.2" ]] && ok "cache bump reported" || bad "cache bump got [$(cache_decision net libfoo 1.2)]" -[[ "$(cache_decision net libbar 1.0)" == "new" ]] && ok "cache new for absent" || bad "cache new got [$(cache_decision net libbar 1.0)]" - -hit="$(cache_path net libfoo 1.1)" -[[ "$hit" == "$CACHE_ROOT/net/libfoo/libfoo-1.1-x86_64-1_danix.txz" ]] && ok "cache_path returns hit" || bad "cache_path got [$hit]" -[[ -z "$(cache_path net libfoo 9.9)" ]] && ok "cache_path empty on miss" || bad "cache_path not empty on miss" - -srctmp=$(mktemp -d); : > "$srctmp/libfoo-1.2-x86_64-1_danix.txz" -cache_store net libfoo "$srctmp/libfoo-1.2-x86_64-1_danix.txz" -count=$(find "$CACHE_ROOT/net/libfoo" -name '*.t?z' | wc -l) -[[ "$count" -eq 1 ]] && ok "cache_store evicts to one file" || bad "cache_store left $count files" -[[ -e "$CACHE_ROOT/net/libfoo/libfoo-1.2-x86_64-1_danix.txz" ]] && ok "cache_store stored new file" || bad "cache_store did not store" -rm -rf "$srctmp" - -# disabled cache -> new -CACHE_ROOT_SAVE="$CACHE_ROOT"; USE_CACHE=0 -[[ "$(cache_decision net libfoo 1.2)" == "new" ]] && ok "--no-cache disables (new)" || bad "disabled cache got [$(cache_decision net libfoo 1.2)]" -USE_CACHE=1; CACHE_ROOT="$CACHE_ROOT_SAVE" -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `bash .extras/test-logic.sh` -Expected: FAIL, `cache_decision: command not found`. - -- [ ] **Step 3: Implement the cache (ported, keyed on CACHE_ROOT + USE_CACHE)** - -Add to `.extras/test-build` above `main()`: - -```bash -# ============================================================================= -# Dependency cache. Layout: $CACHE_ROOT/<cat>/<prog>/<prog>-<ver>-...txz where -# CACHE_ROOT = $PKG_CACHE/<image-digest> (set per run by resolve_cache_root). -# Key is prog+version. --no-cache (USE_CACHE=0) or empty PKG_CACHE disables. -# ============================================================================= -CACHE_ROOT="" # set by resolve_cache_root once the image digest is known - -# True when the cache is usable this run. -_cache_on() { [[ $USE_CACHE -eq 1 && -n "$PKG_CACHE" && -n "$CACHE_ROOT" ]]; } - -_cache_ver_of() { - local prog="$1" base="$2" - base="${base#"$prog"-}" - echo "${base%%-*}" -} - -# cache_decision <cat> <prog> <version> -> cached | bump:OLD:NEW | new -cache_decision() { - local cat="$1" prog="$2" version="$3" - _cache_on || { echo new; return; } - local dir="$CACHE_ROOT/$cat/$prog" - local f newest="" - for f in "$dir/$prog"-*.t?z; do - [[ -e "$f" ]] || continue - [[ -z "$newest" || "$f" -nt "$newest" ]] && newest="$f" - done - [[ -z "$newest" ]] && { echo new; return; } - local have; have="$(_cache_ver_of "$prog" "$(basename "$newest")")" - if [[ "$have" == "$version" ]]; then echo cached; else echo "bump:$have:$version"; fi -} - -cache_path() { - local cat="$1" prog="$2" version="$3" - _cache_on || return - local dir="$CACHE_ROOT/$cat/$prog" - local f newest="" - for f in "$dir/$prog"-*.t?z; do - [[ -e "$f" ]] || continue - [[ -z "$newest" || "$f" -nt "$newest" ]] && newest="$f" - done - [[ -z "$newest" ]] && return - [[ "$(_cache_ver_of "$prog" "$(basename "$newest")")" == "$version" ]] && echo "$newest" -} - -cache_store() { - local cat="$1" prog="$2" src="$3" - _cache_on || return - local dir="$CACHE_ROOT/$cat/$prog" - mkdir -p "$dir" - rm -f "$dir"/*.t?z - cp -a "$src" "$dir/" -} - -cache_label() { - local dir="$1" is_target="$2" - local cat prog ver dec - cat="$(category_of "$dir")"; prog="$(basename "$dir")"; ver="$(version_of "$dir")" - dec="$(cache_decision "$cat" "$prog" "$ver")" - local label - case "$dec" in - cached) label="cached ($ver)" ;; - bump:*) label="rebuild: ${dec#bump:}"; label="${label/:/ -> }" ;; - *) label="build (new)" ;; - esac - if [[ "$is_target" == "1" ]]; then - case "$dec" in - cached) label="build (cached $ver, rebuilt as target)" ;; - esac - echo "target, $label" - else - echo "$label" - fi -} - -# Compute CACHE_ROOT from the image's digest. Falls back to the tag if the -# digest cannot be read (still isolates per image reference). -resolve_cache_root() { - [[ -z "$PKG_CACHE" ]] && { CACHE_ROOT=""; return; } - local digest - digest="$(docker image inspect --format '{{index .Id}}' "$ACTIVE_IMAGE" 2>/dev/null)" - [[ -z "$digest" ]] && digest="tag-${ACTIVE_IMAGE//[^a-zA-Z0-9._-]/_}" - digest="${digest//[^a-zA-Z0-9._-]/_}" - CACHE_ROOT="$PKG_CACHE/$digest" - mkdir -p "$CACHE_ROOT" -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `bash .extras/test-logic.sh` -Expected: PASS (all cache checks green). - -- [ ] **Step 5: Commit** - -```bash -git add .extras/test-build .extras/test-logic.sh -SBOLINT=no git commit -m 'test-build: port dep cache, namespaced by image digest' -``` - ---- - -## Task 6: BLOCKED-BY-DEP propagation (ported) - -**Files:** -- Modify: `.extras/test-build` -- Test: `.extras/test-logic.sh` - -- [ ] **Step 1: Write the failing test** - -Append to `.extras/test-logic.sh` before the result block: - -```bash -# --- BLOCKED-BY-DEP (depends_on_failed) ------------------------------------- -dead=(b) -if depends_on_failed "$T/cat/g" dead; then ok "g blocked when b dead"; else bad "g should block on b"; fi -if depends_on_failed "$T/cat/a" dead; then ok "a blocked when b dead (direct)"; else bad "a should block on b"; fi - -dead=(c) -if depends_on_failed "$T/cat/a" dead; then bad "a wrongly blocked on c"; else ok "a not directly blocked by c"; fi -if depends_on_failed "$T/cat/b" dead; then ok "b blocked when c dead"; else bad "b should block on c"; fi - -dead=() -if depends_on_failed "$T/cat/a" dead; then bad "a blocked with empty dead"; else ok "no block when nothing dead"; fi - -dead=("%README%") -if depends_on_failed "$T/cat/a" dead; then bad "%README% treated as dep"; else ok "%README% not treated as dep"; fi -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `bash .extras/test-logic.sh` -Expected: FAIL, `depends_on_failed: command not found`. - -- [ ] **Step 3: Implement depends_on_failed (ported)** - -Add to `.extras/test-build` above `main()`: - -```bash -# Does SlackBuild dir $1 directly require any prog in the dead list (nameref $2)? -# Direct-requires check only; transitive blocking works because run_target -# iterates in topo order, propagating a failure one hop per package. -depends_on_failed() { - local dir="$1"; local -n failed="$2" - local info="$dir/$(basename "$dir").info" - local req tok f - req="$(read_requires "$info")" - for tok in $req; do - [[ "$tok" == "%README%" ]] && continue - for f in "${failed[@]:-}"; do - [[ "$tok" == "$f" ]] && return 0 - done - done - return 1 -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `bash .extras/test-logic.sh` -Expected: PASS. - -- [ ] **Step 5: Commit** - -```bash -git add .extras/test-build .extras/test-logic.sh -SBOLINT=no git commit -m 'test-build: port BLOCKED-BY-DEP propagation' -``` - ---- - -## Task 7: The container build step (build_one via docker) - -**Files:** -- Modify: `.extras/test-build` - -No self-check here (docker/build is out of the pure-logic reach, same boundary -as sbo-batch-test). Verified by a real run in Task 10. - -- [ ] **Step 1: Implement build_one + lint_pkg** - -Add to `.extras/test-build` above `main()`. - -`lint_pkg` is ported verbatim: - -```bash -# lint_pkg <txz> <logf> -> run sbopkglint on a built package. Host-side, target -# only, fail-soft (skip if absent, never change SUCCESS). -lint_pkg() { - local txz="$1" logf="$2" - if ! command -v sbopkglint >/dev/null 2>&1; then - echo " sbopkglint not installed on host, skipping lint" - return 0 - fi - echo " sbopkglint $(basename "$txz") ..." - echo "===== sbopkglint: $(basename "$txz") =====" >> "$logf" - local out rc - out="$(sbopkglint "$txz" 2>&1)"; rc=$? - printf '%s\n' "$out" >> "$logf" - if [[ $rc -eq 0 ]]; then - echo " lint: ${C_GRN}clean${C_RST}" - else - echo " lint: ${C_RED}findings${C_RST} (see $(basename "$logf")):" - printf '%s\n' "$out" | sed 's/^/ /' - fi -} -``` - -`build_one` runs the same download/md5/build/installpkg steps as sbo-batch-test, -but inside a container via `docker run` instead of `chroot`. The container gets: -the SlackBuild dir mounted read-only, a work dir for output, and (for deps) the -cached .txz mounted in. `sbopkglint` runs host-side on the copied-out package. - -```bash -# build_one <slackbuild-dir> <is_target 0|1> [container-name] -# Runs the build in a throwaway container. Sets ST_STATUS/ST_REASON/ST_TIME. -# Successful builds copy their package to a host workdir; deps are cached and -# the target is linted. Returns 0 on SUCCESS/CACHED, 1 otherwise. -build_one() { - local dir="$1" is_target="${2:-0}" - local prog cat key - prog="$(basename "$dir")"; cat="$(category_of "$dir")"; key="$cat/$prog" - local logf="$RUN_DIR/${cat}_${prog}.log" - local start; start=$(date +%s) - local version; version="$(version_of "$dir")" - - [[ "${HAS_README[$dir]:-}" == "1" ]] && ST_README["$key"]=1 - - # Dep with a version-matching cached package: installpkg it into the shared - # dep-package dir; no build. The target never takes this path. - if [[ "$is_target" != "1" ]]; then - local cached; cached="$(cache_path "$cat" "$prog" "$version")" - if [[ -n "$cached" ]]; then - cp -a "$cached" "$DEPS_DIR/" - { - echo "===== test-build: $prog (from cache) =====" - echo "cached package: $(basename "$cached")" - } >> "$logf" - ST_TIME["$key"]=$(( $(date +%s) - start )) - ST_STATUS["$key"]="CACHED" - return 0 - fi - fi - - # Build in a container. Mounts: - # $dir -> /sbo/pkg (ro, the SlackBuild) - # $DEPS_DIR -> /sbo/deps (rw, already-built dep .txz to installpkg first) - # $BUILD_OUT -> /sbo/out (rw, where the built package is copied out) - # The in-container script installs any deps present, then builds the target, - # writes a status token to /sbo/out/$prog.status, and copies the package out. - local statf="$BUILD_OUT/$prog.status" - rm -f "$statf" - - # fetch deps (removed from -current tree): let the container's sbopkg build - # them first. FETCH_DEPS is the set collected during resolution. - local fetch_list="" - local fp - for fp in "${!FETCH_DEPS[@]}"; do fetch_list+="$fp "; done - - docker run --rm \ - -v "$dir":/sbo/pkg:ro \ - -v "$DEPS_DIR":/sbo/deps \ - -v "$BUILD_OUT":/sbo/out \ - -e PROG="$prog" \ - -e FETCH_LIST="$fetch_list" \ - "$ACTIVE_IMAGE" /bin/bash -s >>"$logf" 2>&1 <<'CONTAINER_EOF' -set -uo pipefail -prog="$PROG" -statf="/sbo/out/$prog.status" - -# 0. install already-built dependency packages (order guaranteed by the host). -for d in /sbo/deps/*.t?z; do - [[ -e "$d" ]] || continue - installpkg --terse "$d" || { echo "INSTALL-FAILED (dep $d)"; echo INSTALL-FAILED > "$statf"; exit 1; } -done - -# 0b. fetch-from-SBo deps via sbopkg (removed from the -current tree). -for f in $FETCH_LIST; do - echo "sbopkg-building fetch dep: $f" - sbopkg -B -i "$f" || { echo "BUILD-FAILED (fetch dep $f)"; echo BUILD-FAILED > "$statf"; exit 1; } -done - -# copy the SlackBuild out of the read-only mount so it can write there. -cp -a /sbo/pkg /sbo/build -cd /sbo/build || { echo BUILD-FAILED > "$statf"; exit 1; } -. ./"$prog".info - -export OUTPUT=/sbo/out -mkdir -p "$OUTPUT" - -echo "===== test-build: $prog =====" -echo "PRGNAM=${PRGNAM:-$prog} VERSION=${VERSION:-?} BUILD=${BUILD:-?} TAG=${TAG:-?}" -echo "uname -m: $(uname -m) OUTPUT=$OUTPUT" -echo "REQUIRES=${REQUIRES:-}" -echo "=================================" - -if [ "$(uname -m)" = "x86_64" ] && [ -n "${DOWNLOAD_x86_64:-}" ] && [ "${DOWNLOAD_x86_64}" != "UNSUPPORTED" ] && [ "${DOWNLOAD_x86_64}" != "UNTESTED" ]; then - DL="$DOWNLOAD_x86_64"; MD="$MD5SUM_x86_64" -else - DL="$DOWNLOAD"; MD="$MD5SUM" -fi - -for u in $DL; do - wget -c --tries=3 "$u" || { echo DOWNLOAD-FAILED > "$statf"; exit 1; } -done - -set -- $MD -for u in $DL; do - f="$(basename "$u")" - want="$1"; shift - got="$(md5sum "$f" | cut -d' ' -f1)" - if [ "$got" != "$want" ]; then - echo "MD5 mismatch on $f: want $want got $got" - echo MD5-MISMATCH > "$statf"; exit 1 - fi -done - -chmod +x ./"$prog".SlackBuild -if ! ./"$prog".SlackBuild; then - echo BUILD-FAILED > "$statf"; exit 1 -fi - -pkg="$(ls -t "$OUTPUT"/"$prog"-*.t?z 2>/dev/null | head -n1)" -if [ -z "$pkg" ]; then - echo "No package produced in $OUTPUT" - echo BUILD-FAILED > "$statf"; exit 1 -fi -if ! installpkg --terse "$pkg"; then - echo INSTALL-FAILED > "$statf"; exit 1 -fi -echo "===== installed files: $(basename "$pkg") =====" -pkgname="$(basename "$pkg")"; pkgname="${pkgname%.t?z}" -cat "/var/log/packages/$pkgname" 2>/dev/null || echo "(package db entry not found)" -echo "=================================" -echo SUCCESS > "$statf" -CONTAINER_EOF - - local status="BUILD-FAILED" - [[ -f "$statf" ]] && status="$(cat "$statf")" - ST_TIME["$key"]=$(( $(date +%s) - start )) - ST_STATUS["$key"]="$status" - - if [[ "$status" == "SUCCESS" ]]; then - # locate the built package copied to the host workdir - local built newest="" - for built in "$BUILD_OUT/${prog}"-*.t?z; do - [[ -e "$built" ]] || continue - [[ -z "$newest" || "$built" -nt "$newest" ]] && newest="$built" - done - if [[ -n "$newest" ]]; then - if [[ "$is_target" != "1" ]]; then - cache_store "$cat" "$prog" "$newest" - # make the dep available to later builds in this run - cp -a "$newest" "$DEPS_DIR/" - else - lint_pkg "$newest" "$logf" - fi - fi - return 0 - fi - ST_REASON["$key"]="see $(basename "$logf")" - return 1 -} -``` - -- [ ] **Step 2: Sanity-check bash syntax** - -Run: `bash -n .extras/test-build` -Expected: no output (syntax OK). - -- [ ] **Step 3: Run the self-check still passes (no regression)** - -Run: `bash .extras/test-logic.sh` -Expected: PASS (build_one is not exercised, but sourcing must not break). - -- [ ] **Step 4: Commit** - -```bash -git add .extras/test-build -SBOLINT=no git commit -m 'test-build: container build step (build_one via docker)' -``` - ---- - -## Task 8: run_target orchestration + summary - -**Files:** -- Modify: `.extras/test-build` - -- [ ] **Step 1: Implement run_target + print_summary + confirm** - -Add to `.extras/test-build` above `main()`. `print_summary` is ported verbatim -(status set is the same). `run_target` replaces the overlay lifecycle with the -per-run docker workdirs and adds the printed-order + Y/n confirm. - -```bash -# confirm_order: print the resolved order (overrides marked) and ask to proceed. -# --yes skips the prompt but the order is still printed. --dry-run never reaches -# here. Returns 0 to proceed, 1 to abort. -confirm_order() { - local target_dir="$1" - echo " build order (${VERSION_ID}):" - local d - for d in "${RESOLVED_ORDER[@]}"; do - local it=0; [[ "$d" == "$target_dir" ]] && it=1 - local rdm=""; [[ "${HAS_README[$d]:-}" == 1 ]] && rdm=" [%README%]" - printf " %-30s %s%s\n" "$(pkg_key "$d")" "$(cache_label "$d" "$it")" "$rdm" - echo "$(pkg_key "$d")" >> "$RUN_DIR/build-order.txt" - done - # note fetch deps (built via sbopkg in-container, not in the order list) - local fp - for fp in "${!FETCH_DEPS[@]}"; do - printf " %-30s %s\n" "$fp" "fetch (sbopkg in container)" - done - [[ $ASSUME_YES -eq 1 ]] && return 0 - local reply - read -rp " Proceed? [Y/n] " reply - [[ -z "$reply" || "$reply" =~ ^[Yy]$ ]] -} - -# run_target <target-slackbuild-dir> -run_target() { - local target_dir="$1" - local tkey; tkey="$(pkg_key "$target_dir")" - - echo - echo "=== Target: $tkey (${VERSION_ID}) ===" - resolve_target "$target_dir" - apply_overrides_to_order - - # Hard resolution failures: report and stop, do not build. - if [[ ${#CYCLES[@]} -gt 0 || ${#UNMET[@]} -gt 0 ]]; then - local why="" - if [[ ${#UNMET[@]} -gt 0 ]]; then - local u - for u in "${!UNMET[@]}"; do why+="unmet:$u(${UNMET[$u]}) "; done - fi - [[ ${#CYCLES[@]} -gt 0 ]] && why+="${CYCLES[*]}" - ST_STATUS["$tkey"]="UNMET-DEP" - ST_REASON["$tkey"]="$why" - echo " resolution failed: $why" - echo " add an override rule ($TB_OVERRIDES) and rerun, or fix the tree." >&2 - return - fi - - if [[ $DRY_RUN -eq 1 ]]; then - echo " build order (dry-run):" - local d - for d in "${RESOLVED_ORDER[@]}"; do - local it=0; [[ "$d" == "$target_dir" ]] && it=1 - local rdm=""; [[ "${HAS_README[$d]:-}" == 1 ]] && rdm=" [%README%]" - printf " %-30s %s%s\n" "$(pkg_key "$d")" "$(cache_label "$d" "$it")" "$rdm" - echo "$(pkg_key "$d")" >> "$RUN_DIR/build-order.txt" - done - local fp - for fp in "${!FETCH_DEPS[@]}"; do - printf " %-30s %s\n" "$fp" "fetch (sbopkg in container)" - done - return - fi - - if ! confirm_order "$target_dir"; then - echo " aborted." - ST_STATUS["$tkey"]="ABORTED" - return - fi - - # Per-run docker workdirs (host side, discarded after the run). - DEPS_DIR="$RUN_DIR/deps"; BUILD_OUT="$RUN_DIR/out" - mkdir -p "$DEPS_DIR" "$BUILD_OUT" - - local d failed_progs=() - for d in "${RESOLVED_ORDER[@]}"; do - local key; key="$(pkg_key "$d")" - local prog; prog="$(basename "$d")" - - if depends_on_failed "$d" failed_progs; then - ST_STATUS["$key"]="BLOCKED-BY-DEP" - ST_REASON["$key"]="blocked by failed dep" - [[ "${HAS_README[$d]:-}" == "1" ]] && ST_README["$key"]=1 - echo " $key: BLOCKED-BY-DEP" - failed_progs+=("$prog") - continue - fi - - local it=0; [[ "$d" == "$target_dir" ]] && it=1 - echo " building $key ..." - if build_one "$d" "$it"; then - echo " $key: ${ST_STATUS[$key]} (${ST_TIME[$key]}s)" - else - echo " $key: ${ST_STATUS[$key]} (${ST_TIME[$key]}s)" - failed_progs+=("$prog") - fi - done -} - -# ============================================================================= -# SUMMARY (ported) -# ============================================================================= -print_summary() { - local total=$SECONDS - local succ=0 fail=0 blocked=0 cached=0 - local summary="$RUN_DIR/summary.log" - - { - echo "test-build run summary" - echo "target: $TARGET_ARG version: $VERSION_ID" - echo - } > "$summary" - - echo - echo "================ SUMMARY ================" - local key - for key in "${!ST_STATUS[@]}"; do - local st="${ST_STATUS[$key]}" rsn="${ST_REASON[$key]:-}" t="${ST_TIME[$key]:-0}" - local rd=""; [[ "${ST_README[$key]:-}" == "1" ]] && rd=" [%README%]" - local col="$C_YEL" - case "$st" in - SUCCESS) col="$C_GRN"; ((succ++)) ;; - CACHED) col="$C_GRN"; ((cached++)) ;; - BLOCKED-BY-DEP|UNMET-DEP|ABORTED) col="$C_YEL"; ((blocked++)) ;; - *) col="$C_RED"; ((fail++)) ;; - esac - printf "%s%-30s %-16s%s %s%s (%ss)\n" "$col" "$key" "$st" "$C_RST" "$rsn" "$rd" "$t" - printf "%-30s %-16s %s%s (%ss)\n" "$key" "$st" "$rsn" "$rd" "$t" >> "$summary" - done - echo "----------------------------------------" - printf "%s%d succeeded%s, %s%d failed%s, %s%d blocked%s, %s%d cached%s, total %ss\n" \ - "$C_GRN" "$succ" "$C_RST" "$C_RED" "$fail" "$C_RST" "$C_YEL" "$blocked" "$C_RST" \ - "$C_GRN" "$cached" "$C_RST" "$total" - echo "logs: $RUN_DIR" - if [[ $fail -eq 0 && $blocked -eq 0 ]]; then - echo "${C_GRN}All green.${C_RST} Safe to build the SBo submission tarball on the host." - fi - { - echo - echo "$succ succeeded, $fail failed, $blocked blocked, $cached cached, total ${total}s" - echo "logs: $RUN_DIR" - } >> "$summary" -} -``` - -Add the two workdir globals near the other globals (top of the flags block): - -```bash -DEPS_DIR="" # per-run host dir of built dep .txz, mounted into the container -BUILD_OUT="" # per-run host dir where built packages are copied out -``` - -- [ ] **Step 2: Sanity-check syntax** - -Run: `bash -n .extras/test-build` -Expected: no output. - -- [ ] **Step 3: Self-check no regression** - -Run: `bash .extras/test-logic.sh` -Expected: PASS. - -- [ ] **Step 4: Commit** - -```bash -git add .extras/test-build -SBOLINT=no git commit -m 'test-build: run_target orchestration + summary + confirm' -``` - ---- - -## Task 9: Wire main(), image presence check, target lookup - -**Files:** -- Modify: `.extras/test-build` - -- [ ] **Step 1: Implement the full main()** - -Replace the current `main()` in `.extras/test-build` with: - -```bash -# Verify the selected image exists locally. It is built by a separate job; this -# script only consumes it. (Later: a local-registry docker pull slots in here.) -require_image() { - if ! docker image inspect "$ACTIVE_IMAGE" >/dev/null 2>&1; then - cat >&2 <<EOF -Image not found locally: $ACTIVE_IMAGE -It is produced by the separate image-builder job (full Slackware $VERSION_ID + -sbo-maintainer-tools + sbopkg), stored on the NAS / local registry. Load or -pull it, then rerun. This script does not build images. -EOF - exit 1 - fi -} - -main() { - parse_args "$@" - init_color - require_config - SBO_TREE_ROOTS=("$ACTIVE_TREE") - load_overrides - - # image + cache namespace (skip the image check on a pure dry-run so the order - # can be inspected without the image present). - if [[ $DRY_RUN -eq 0 ]]; then - command -v docker >/dev/null 2>&1 || { echo "docker not found in PATH." >&2; exit 1; } - require_image - fi - resolve_cache_root - - RUN_DIR="$LOG_ROOT/$(date +%Y-%m-%d_%H-%M-%S)" - mkdir -p "$RUN_DIR" - : > "$RUN_DIR/build-order.txt" - - # Single-package mode: resolve the name in the active tree. - local tdir - if ! tdir="$(find_slackbuild_dir "$TARGET_ARG")"; then - echo "Program not found in the SBo tree ($ACTIVE_TREE): $TARGET_ARG" >&2 - exit 1 - fi - - run_target "$tdir" - print_summary -} -``` - -- [ ] **Step 2: Sanity-check syntax** - -Run: `bash -n .extras/test-build` -Expected: no output. - -- [ ] **Step 3: Dry-run smoke test against the real tree (no docker needed)** - -This needs a real config. If `~/.config/sbo-testbuild/config` is not set up, -create a throwaway one pointing at an existing local SBo tree, then: - -Run: `.extras/test-build --dry-run <some-pkg-in-that-tree>` -Expected: prints `=== Target: ... ===`, a build order, exits 0, no docker call. - -- [ ] **Step 4: Self-check no regression** - -Run: `bash .extras/test-logic.sh` -Expected: PASS. - -- [ ] **Step 5: Commit** - -```bash -git add .extras/test-build -SBOLINT=no git commit -m 'test-build: wire main, image-presence check, target lookup' -``` - ---- - -## Task 10: Real end-to-end build (user-run) + README - -**Files:** -- Modify: `.extras/test-build` (only if the real run surfaces a bug) -- Create: `.extras/README.test-build` (short usage note) - -This task needs a real image and the buildsystem. Per repo rules the throwaway -docker build is Claude-OK, but the IMAGE must exist first (separate job, not yet -built). So this task is: document usage, and record the end-to-end run as a -manual verification step the user performs once an image is available. - -- [ ] **Step 1: Write the usage README** - -Create `.extras/README.test-build`: - -```markdown -# test-build - -Verify an already-published SBo package still builds on a target Slackware -version inside a throwaway docker container. - -## Setup - - mkdir -p ~/.config/sbo-testbuild - cp .extras/test-build-config.example ~/.config/sbo-testbuild/config - cp .extras/overrides.example ~/.config/sbo-testbuild/overrides - $EDITOR ~/.config/sbo-testbuild/config - -Set the two SBo tree paths, the two image tags, LOG_ROOT, and PKG_CACHE. - -The images (`sbo-testbuild:current` / `:15.0`) are FULL Slackware installs plus -`sbo-maintainer-tools` and `sbopkg`, built by a separate job (not this script). -The script errors if the tagged image is not present locally. - -## Usage - - .extras/test-build <pkg> # build on -current (default) - .extras/test-build --stable <pkg> # build on 15.0 - .extras/test-build --dry-run <pkg> # show the build order, no build - .extras/test-build --yes <pkg> # skip the confirm (order still shown) - .extras/test-build --no-cache <pkg> # rebuild all deps this run - -A green run means it is safe to build the SBo submission tarball on the host. - -## current-vs-stable overrides - -`~/.config/sbo-testbuild/overrides` (see `overrides.example`) encodes deltas -that only apply when building on -current: - - drop: <prog> already in the -current base - rename: <old> -> <new> renamed on -current - fetch: <prog> removed from the -current tree; built via sbopkg - -Start light; add rules only when a real package needs them. - -## Self-check - - bash .extras/test-logic.sh -``` - -- [ ] **Step 2: Commit the docs** - -```bash -git add .extras/README.test-build -SBOLINT=no git commit -m 'test-build: usage README' -``` - -- [ ] **Step 3: Manual end-to-end verification (user, once an image exists)** - -Documented for the user to run when a `sbo-testbuild:*` image is available: - -```bash -# pick a small already-published package with a dep, e.g. one already in the repo -.extras/test-build --yes <pkg> -``` - -Expected: build order printed, deps built (or CACHED on a second run), target -built, `sbopkglint` result shown, summary green, `All green.` line printed. -A second run of the same package should show its deps as `CACHED`. - -Record the outcome. If a real bug surfaces (mount path, sbopkg invocation, -package glob), fix it in `.extras/test-build` and re-run; add a self-check case -if the bug was in pure logic. - ---- - -## Self-Review - -Run this checklist against the spec after the plan is written: - -**1. Spec coverage:** -- purpose / one-target-per-run: Task 2 (`parse_args`), Task 9 (`main`) - covered -- two versions, current default: Task 2 (`--stable`, `select_version_paths`) - covered -- resolve local tree + topo: Task 3 - covered -- overrides drop/rename/fetch + UNMET stop: Task 4, stop in Task 8 `run_target` - covered -- external config + overrides file: Task 2, Task 4 - covered -- external image, presence check, no bake: Task 9 `require_image` - covered -- container build (download/md5/build/installpkg): Task 7 - covered -- fetch via sbopkg: Task 7 (`FETCH_LIST` loop) - covered -- dep cache, digest-namespaced, --no-cache, target always fresh: Task 5, wired Task 7/8 - covered -- target lint: Task 7 `lint_pkg`, called in Task 7 build_one - covered -- confirm order + --yes still prints: Task 8 `confirm_order` - covered -- --dry-run: Task 8 `run_target`, Task 9 skips image check - covered -- report + logs (per-pkg, summary, build-order): Task 8 `print_summary`, Task 7 logs - covered -- status values incl CACHED: Task 8 `print_summary` - covered -- self-check parity: Tasks 1-6 - covered - -**2. Placeholder scan:** no TBD/TODO in code steps; every code step shows full -code. The one deferred item (real end-to-end run) is explicit and gated on an -external image, not a placeholder. - -**3. Type consistency:** function + global names match across tasks: -`RESOLVED_ORDER`, `FETCH_DEPS`, `OV_DROP/OV_RENAME/OV_FETCH`, `CACHE_ROOT`, -`ACTIVE_TREE/ACTIVE_IMAGE`, `DEPS_DIR/BUILD_OUT`, `cache_decision/cache_path/ -cache_store/cache_label`, `select_version_paths`, `apply_overrides_to_order`, -`apply_rename`, `load_overrides`, `resolve_cache_root`, `require_image`, -`build_one(dir,is_target)`, `run_target`, `confirm_order`, `print_summary`. -Consistent. - -## Notes on deliberate simplifications - -- Single-package mode only (no category-folder / all / queue modes). This repo - bumps one package at a time. Extension point is `main`'s target lookup. -- `-j` parallelism dropped entirely (was a no-op stub upstream). Builds serial. -- fetch deps are not topo-ordered among themselves; sbopkg resolves their own - deps. Start light per the spec; revisit if a fetch dep needs another fetch dep. -``` - diff --git a/.extras/docs/plans/2026-07-13-image-builder.md b/.extras/docs/plans/2026-07-13-image-builder.md deleted file mode 100644 index 0c61051..0000000 --- a/.extras/docs/plans/2026-07-13-image-builder.md +++ /dev/null @@ -1,790 +0,0 @@ -# sbo-testbuild image builder — Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Build the three chained bash scripts that produce `sbo-testbuild:current` / `sbo-testbuild:15.0` docker images from NAS Slackware trees and push them to a LAN registry, consumed by `.extras/test-build`. - -**Architecture:** Adapt the forge `slackware/docker-images` scripts (`bootstrap.sh`, `build-full-image.sh`) into `.extras/image-builder/`. Add a third `build-sbo-testbuild.sh` layer. Share config + helpers via `config` and `lib.sh`. Pure-logic helpers are unit-tested with an assert-based self-check; the docker chain is verified manually on the VM. - -**Tech Stack:** bash, docker, slackpkg/installpkg (Slackware), `registry:2`. - -**Reference:** design spec at `.extras/docs/specs/2026-07-13-image-builder-design.md`. Forge originals cloned at (scratchpad) `docker-images/scripts/` — re-clone with `git clone slackware_forge:slackware/docker-images.git` if gone. - ---- - -## File Structure - -``` -.extras/image-builder/ -├── config REGISTRY, MIRROR, VARIANTS, PKGDIR (sourced by all) -├── lib.sh _log/_warn/_err, fetch(), txz_hash(), require_mount() -├── bootstrap.sh base image FROM scratch (root; installpkg from NAS) -├── build-full-image.sh full image FROM base (slackpkg all series) -├── build-sbo-testbuild.sh testbuild image FROM full (installpkg sbopkg + tools) -├── test-image-builder.sh assert-based self-check for lib.sh logic -└── README VM-side setup checklist -``` - -Responsibilities: -- **config** — the only place hostnames/paths/variants live. No script hardcodes them. -- **lib.sh** — all shared, testable logic. Sourced by the three build scripts and by the test. No side effects at source time (only function defs + nothing else). -- **bootstrap/full/testbuild** — one image each, thin: guard, gate, generate Dockerfile, build, push. -- **test-image-builder.sh** — sources lib.sh, exercises fetch/txz_hash/require_mount with asserts. No docker. - ---- - -## Task 1: config + lib.sh skeleton - -**Files:** -- Create: `.extras/image-builder/config` -- Create: `.extras/image-builder/lib.sh` - -- [ ] **Step 1: Write config** - -Create `.extras/image-builder/config`: - -```sh -# Shared configuration for the sbo-testbuild image builder. -# Sourced by bootstrap.sh, build-full-image.sh, build-sbo-testbuild.sh, tests. - -REGISTRY="docker.noland.dnx:5000" -MIRROR="file:///mnt/nas" -VARIANTS=(current 15.0) # x86_64 only for now -PKGDIR="/opt/sbo-testbuild/pkgs" # sbopkg + sbo-maintainer-tools .txz -HASH_DIR="/var/cache/sbo-testbuild" # ChangeLog hashes for rebuild gating -``` - -- [ ] **Step 2: Write lib.sh with logging + require_mount** - -Create `.extras/image-builder/lib.sh`: - -```sh -# Shared helpers for the image builder. Source, do not execute. -# Sourcing must have no side effects beyond defining functions. - -_log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] [${LOG_TAG:-image-builder}] $*"; } -_warn() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] [${LOG_TAG:-image-builder}] WARNING: $*" >&2; } -_err() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] [${LOG_TAG:-image-builder}] ERROR: $*" >&2; exit 1; } - -# mirror_path MIRROR RELPATH -# Join a MIRROR (file:///... or http(s)://...) with a relative path. -mirror_path() { - local mirror="$1" rel="$2" - printf '%s/%s' "${mirror%/}" "${rel#/}" -} - -# is_local_mirror MIRROR -> 0 if file:// or bare absolute path, else 1 -is_local_mirror() { - case "$1" in - file://*) return 0 ;; - http://*|https://*) return 1 ;; - /*) return 0 ;; - *) return 1 ;; - esac -} - -# strip_scheme URL -> filesystem path for a file:// or bare-local URL -strip_scheme() { - local u="$1" - case "$u" in - file://*) printf '%s' "${u#file://}" ;; - *) printf '%s' "$u" ;; - esac -} - -# fetch SRC DEST -# Copy SRC (a full mirror URL) to DEST. Local mirror -> cp, http -> curl. -# Returns non-zero on failure; caller decides whether that is fatal. -fetch() { - local src="$1" dest="$2" - if is_local_mirror "$src"; then - local path; path="$(strip_scheme "$src")" - [[ -f "$path" ]] || return 1 - cp "$path" "$dest" - else - curl -sSfL --connect-timeout 60 "$src" -o "$dest" - fi -} - -# txz_hash DIR -# Stable hash of the set of .txz files in DIR (names + contents), order -# independent. Empty/absent dir -> empty string. -txz_hash() { - local dir="$1" - [[ -d "$dir" ]] || { printf ''; return 0; } - local files; files=$(find "$dir" -maxdepth 1 -name '*.txz' -type f | sort) - [[ -n "$files" ]] || { printf ''; return 0; } - # Hash sorted "name sha256" lines so reordering does not change output. - while IFS= read -r f; do - printf '%s %s\n' "$(basename "$f")" "$(sha256sum "$f" | cut -d' ' -f1)" - done <<< "$files" | sha256sum | cut -d' ' -f1 -} - -# require_mount VERSION -# Assert the NAS tree for VERSION is mounted (file:// mirror only). -# x86_64 tree dir is slackware64-${VERSION}. -require_mount() { - local version="$1" - is_local_mirror "$MIRROR" || return 0 # http mirror: nothing to check - local base; base="$(strip_scheme "$MIRROR")" - local dir="${base}/slackware64-${version}" - [[ -d "$dir" ]] || _err "NAS tree not mounted: ${dir}" -} -``` - -- [ ] **Step 3: Commit** - -```bash -git add .extras/image-builder/config .extras/image-builder/lib.sh -git commit -m 'image-builder: config + lib.sh helpers' -``` - ---- - -## Task 2: lib.sh self-check - -**Files:** -- Create: `.extras/image-builder/test-image-builder.sh` - -- [ ] **Step 1: Write the test** - -Create `.extras/image-builder/test-image-builder.sh`: - -```sh -#!/bin/bash -# Assert-based self-check for lib.sh pure logic. No docker, no network. -# Run: bash .extras/image-builder/test-image-builder.sh -set -u -cd "$(dirname "$0")" -source ./lib.sh - -pass=0 fail=0 -check() { # DESC EXPECTED ACTUAL - if [[ "$2" == "$3" ]]; then pass=$((pass+1)); - else fail=$((fail+1)); echo "FAIL: $1"; echo " expected: [$2]"; echo " actual: [$3]"; fi -} -check_rc() { # DESC EXPECTED_RC ACTUAL_RC - if [[ "$2" == "$3" ]]; then pass=$((pass+1)); - else fail=$((fail+1)); echo "FAIL: $1 (rc expected $2 got $3)"; fi -} - -tmp="$(mktemp -d)" -trap 'rm -rf "$tmp"' EXIT - -# --- mirror_path --- -check "mirror_path trailing slash" \ - "file:///mnt/nas/slackware64-current/PACKAGES.TXT" \ - "$(mirror_path 'file:///mnt/nas/' 'slackware64-current/PACKAGES.TXT')" -check "mirror_path no trailing slash" \ - "file:///mnt/nas/x/y" "$(mirror_path 'file:///mnt/nas' '/x/y')" - -# --- is_local_mirror --- -is_local_mirror "file:///mnt/nas"; check_rc "is_local file://" 0 $? -is_local_mirror "/mnt/nas"; check_rc "is_local bare abs" 0 $? -is_local_mirror "https://x/y"; check_rc "is_local https" 1 $? - -# --- strip_scheme --- -check "strip_scheme file://" "/mnt/nas" "$(strip_scheme 'file:///mnt/nas')" -check "strip_scheme bare" "/mnt/nas" "$(strip_scheme '/mnt/nas')" - -# --- fetch: local file resolves --- -echo hello > "$tmp/src.txt" -fetch "file://$tmp/src.txt" "$tmp/dst.txt"; check_rc "fetch local ok" 0 $? -check "fetch local content" "hello" "$(cat "$tmp/dst.txt" 2>/dev/null)" - -# --- fetch: missing local file errors --- -fetch "file://$tmp/nope.txt" "$tmp/x.txt"; check_rc "fetch local missing" 1 $? - -# --- txz_hash: stable + content-sensitive --- -mkdir -p "$tmp/pkgs" -printf a > "$tmp/pkgs/sbopkg-1.txz" -printf b > "$tmp/pkgs/tools-1.txz" -h1="$(txz_hash "$tmp/pkgs")" -h2="$(txz_hash "$tmp/pkgs")" -check "txz_hash stable" "$h1" "$h2" -printf c > "$tmp/pkgs/tools-1.txz" # change content -h3="$(txz_hash "$tmp/pkgs")" -check_rc "txz_hash changes on content" 0 "$([[ "$h1" != "$h3" ]] && echo 0 || echo 1)" -check "txz_hash empty dir" "" "$(txz_hash "$tmp/empty-nope")" - -# --- require_mount: missing mount errors (subshell to catch _err exit) --- -MIRROR="file://$tmp/no-such-root" -( require_mount current ) 2>/dev/null; check_rc "require_mount missing" 1 $? -mkdir -p "$tmp/mnt/slackware64-current" -MIRROR="file://$tmp/mnt" -( require_mount current ) 2>/dev/null; check_rc "require_mount present" 0 $? -MIRROR="https://example/x" -( require_mount current ) 2>/dev/null; check_rc "require_mount http skips" 0 $? - -echo "----" -echo "PASS: $pass FAIL: $fail" -[[ "$fail" -eq 0 ]] -``` - -- [ ] **Step 2: Run — expect PASS** - -Run: `bash .extras/image-builder/test-image-builder.sh` -Expected: final line `PASS: N FAIL: 0`, exit 0. - -If any FAIL, fix `lib.sh` (the test encodes the intended behavior). - -- [ ] **Step 3: Commit** - -```bash -git add .extras/image-builder/test-image-builder.sh -git commit -m 'image-builder: lib.sh self-check' -``` - ---- - -## Task 3: bootstrap.sh (base image) - -**Files:** -- Create: `.extras/image-builder/bootstrap.sh` -- Reference: forge `docker-images/scripts/bootstrap.sh` - -This is the forge `bootstrap.sh` adapted: source `config`+`lib.sh`, drop i586, -`file://` fetch via `fetch()`, LAN registry, `sbo-base` name, mount guard. - -- [ ] **Step 1: Write bootstrap.sh** - -Create `.extras/image-builder/bootstrap.sh`. Start from the forge original and apply exactly these changes; everything else (the `download_pkgtxt`, `find_package`, rootfs configuration, cleanup, Dockerfile-from-scratch blocks) is copied verbatim from the forge script: - -1. Header block: - -```sh -#!/bin/bash -# bootstrap.sh — build the sbo-base:{ver} image FROM scratch from NAS trees. -# Run as root (installpkg). Adapted from forge slackware/docker-images. -set -euo pipefail -HERE="$(cd "$(dirname "$0")" && pwd)" -source "${HERE}/config" -LOG_TAG=bootstrap -source "${HERE}/lib.sh" - -REGISTRY_IMAGE="${REGISTRY}/sbo-base" -[[ "${EUID}" -eq 0 ]] || _err "run as root." -mkdir -p "${HASH_DIR}" -``` - -2. Argument parsing: keep `--force` and `--version` only; **remove `--arch`** - (x86_64 fixed). Variant list comes from config: - -```sh -FORCE=false -OPT_VERSION="" -while [[ $# -gt 0 ]]; do - case "$1" in - --version) OPT_VERSION="$2"; shift 2 ;; - --force) FORCE=true; shift ;; - *) _err "unknown argument: $1" ;; - esac -done -if [[ -n "${OPT_VERSION}" ]]; then - BUILD_VARIANTS=("${OPT_VERSION}") -else - BUILD_VARIANTS=("${VARIANTS[@]}") -fi -``` - -3. Copy the forge `PACKAGES=( ... )` array verbatim (lines 80–187 of the forge - original — the full base package list). - -4. Replace `changelog_changed`: read ChangeLog via `fetch` from the NAS, - x86_64-only repo key: - -```sh -# changelog_changed VERSION -> 0 if changed (or --force), else 1 -changelog_changed() { - local version="$1" - local repo_key="slackware64-${version}" - local hash_file="${HASH_DIR}/${repo_key}.sha256" - local url; url="$(mirror_path "${MIRROR}" "${repo_key}/ChangeLog.txt")" - local tmp; tmp="$(mktemp)" - if ! fetch "${url}" "${tmp}"; then - rm -f "${tmp}" - _warn "${repo_key}: cannot read ChangeLog (${url}); skipping." - return 1 - fi - local live; live=$(sha256sum "${tmp}" | cut -d' ' -f1); rm -f "${tmp}" - if [[ "${FORCE}" == "true" ]]; then - echo "${live}" > "${hash_file}"; return 0 - fi - local stored=""; [[ -f "${hash_file}" ]] && stored=$(cat "${hash_file}") - if [[ "${live}" == "${stored}" ]]; then - _log "${repo_key}: ChangeLog unchanged; skipping."; return 1 - fi - _log "${repo_key}: ChangeLog changed."; echo "${live}" > "${hash_file}"; return 0 -} -``` - -5. Copy `download_pkgtxt` and `find_package` verbatim from the forge original, - but change the two `curl -sfL ... "${URL}" -o "${DEST}"` download lines in - `download_pkgtxt` to `fetch "${URL}" "${DEST}"` (drops the http assumption). - -6. `build_variant`: take `VERSION` only (no ARCH). Set: - -```sh -build_variant() { - local VERSION="$1" - require_mount "${VERSION}" - local REPO_KEY="slackware64-${VERSION}" - local PKG_PATH; PKG_PATH="$(mirror_path "${MIRROR}" "${REPO_KEY}")" - local TAG="${REGISTRY_IMAGE}:${VERSION}" - local SLACKPKG_MIRROR="https://slackware.nl/slackware/slackware64-${VERSION}/" - ... -} -``` - - The `SLACKPKG_MIRROR` written into the image stays an https slackware.nl URL - (it is used later by slackpkg inside the container, which has internet; the - NAS `file://` mount only feeds the host-side build). Keep the rest of the - forge `build_variant` body verbatim, except: - - the package **download loop** uses `fetch "${URL}" "${PKGCACHE}/${FILENAME}"` - instead of `curl`. - - the final `docker build ... && docker push` targets `${TAG}` as above. - -7. Main loop over `BUILD_VARIANTS`, one arg: - -```sh -for VERSION in "${BUILD_VARIANTS[@]}"; do - if changelog_changed "${VERSION}"; then - build_variant "${VERSION}" - fi -done -``` - -- [ ] **Step 2: Syntax check** - -Run: `bash -n .extras/image-builder/bootstrap.sh` -Expected: no output, exit 0. - -- [ ] **Step 3: shellcheck (if available)** - -Run: `shellcheck -x .extras/image-builder/bootstrap.sh || true` -Expected: no errors (warnings about unreachable `_err` exit are fine). If shellcheck absent, skip. - -- [ ] **Step 4: Commit** - -```bash -git add .extras/image-builder/bootstrap.sh -git commit -m 'image-builder: bootstrap.sh base image from NAS' -``` - ---- - -## Task 4: build-full-image.sh - -**Files:** -- Create: `.extras/image-builder/build-full-image.sh` -- Reference: forge `docker-images/scripts/build-full-image.sh` - -Forge `build-full-image.sh` adapted: source config+lib, drop i586, LAN -registry, `sbo-base`→`sbo-full` names. - -- [ ] **Step 1: Write build-full-image.sh** - -Create `.extras/image-builder/build-full-image.sh`. From the forge original, apply exactly: - -1. Header: - -```sh -#!/bin/bash -# build-full-image.sh — sbo-full:{ver} FROM sbo-base:{ver}, all series. -# Adapted from forge slackware/docker-images. No root needed. -set -euo pipefail -HERE="$(cd "$(dirname "$0")" && pwd)" -source "${HERE}/config" -LOG_TAG=full -source "${HERE}/lib.sh" - -BASE_IMAGE="${REGISTRY}/sbo-base" -FULL_IMAGE="${REGISTRY}/sbo-full" -DIGEST_LABEL="sbo.full.base-digest" -``` - -2. Arg parsing: `--force` and `--version` only (no `--arch`), variant list from - config — identical shape to bootstrap Task 3 Step 1.2, but `BUILD_VARIANTS`. - -3. `derive_tags VERSION`: - -```sh -derive_tags() { - local VERSION="$1" - BASE_TAG="${BASE_IMAGE}:${VERSION}" - FULL_TAG="${FULL_IMAGE}:${VERSION}" -} -``` - -4. Keep `needs_rebuild` verbatim from the forge original (it already takes - BASE_TAG/FULL_TAG args and uses `_log`/`_warn`/`FORCE`). - -5. `build_variant VERSION`: drop the ARCH/ARCH_SUFFIX lines; call - `derive_tags "${VERSION}"`. Keep the forge Dockerfile heredoc **verbatim** - (the `slackpkg update` + all-series install + cleanup + labels). Only the - `LABEL org.opencontainers.image.title/description` strings lose the - `(${ARCH})` suffix; leave them or hardcode `(x86_64)`. - -6. Main loop over `BUILD_VARIANTS`: - -```sh -for VERSION in "${BUILD_VARIANTS[@]}"; do - build_variant "${VERSION}" -done -``` - -- [ ] **Step 2: Syntax check** - -Run: `bash -n .extras/image-builder/build-full-image.sh` -Expected: exit 0. - -- [ ] **Step 3: shellcheck (if available)** - -Run: `shellcheck -x .extras/image-builder/build-full-image.sh || true` - -- [ ] **Step 4: Commit** - -```bash -git add .extras/image-builder/build-full-image.sh -git commit -m 'image-builder: build-full-image.sh' -``` - ---- - -## Task 5: build-sbo-testbuild.sh - -**Files:** -- Create: `.extras/image-builder/build-sbo-testbuild.sh` - -New script (no forge equivalent). FROM sbo-full, installpkg the two .txz, -gate on full digest AND .txz-set hash. - -- [ ] **Step 1: Write build-sbo-testbuild.sh** - -Create `.extras/image-builder/build-sbo-testbuild.sh`: - -```sh -#!/bin/bash -# build-sbo-testbuild.sh — sbo-testbuild:{ver} FROM sbo-full:{ver}. -# Adds sbopkg + sbo-maintainer-tools from prebuilt .txz in PKGDIR. -# Rebuilds when the full image OR the .txz set changes. -set -euo pipefail -HERE="$(cd "$(dirname "$0")" && pwd)" -source "${HERE}/config" -LOG_TAG=testbuild -source "${HERE}/lib.sh" - -FULL_IMAGE="${REGISTRY}/sbo-full" -TB_IMAGE="${REGISTRY}/sbo-testbuild" -DIGEST_LABEL="sbo.testbuild.full-digest" -PKGS_LABEL="sbo.testbuild.pkgs-hash" - -FORCE=false -OPT_VERSION="" -while [[ $# -gt 0 ]]; do - case "$1" in - --version) OPT_VERSION="$2"; shift 2 ;; - --force) FORCE=true; shift ;; - *) _err "unknown argument: $1" ;; - esac -done -if [[ -n "${OPT_VERSION}" ]]; then - BUILD_VARIANTS=("${OPT_VERSION}") -else - BUILD_VARIANTS=("${VARIANTS[@]}") -fi - -# Verify the required .txz are present before doing anything. -require_pkgs() { - [[ -d "${PKGDIR}" ]] || _err "PKGDIR missing: ${PKGDIR}" - compgen -G "${PKGDIR}/sbopkg-*.txz" >/dev/null \ - || _err "no sbopkg-*.txz in ${PKGDIR}" - compgen -G "${PKGDIR}/sbo-maintainer-tools-*.txz" >/dev/null \ - || _err "no sbo-maintainer-tools-*.txz in ${PKGDIR}" -} - -# needs_rebuild FULL_TAG TB_TAG PKGS_HASH -> 0 if rebuild needed -needs_rebuild() { - local full_tag="$1" tb_tag="$2" pkgs_hash="$3" - [[ "${FORCE}" == "true" ]] && { _log " --force."; return 0; } - - local full_digest - full_digest=$(docker inspect --format '{{index .RepoDigests 0}}' \ - "${full_tag}" 2>/dev/null || echo "") - [[ -n "${full_digest}" ]] || { _warn " no full digest; rebuilding."; return 0; } - - local rec_digest rec_pkgs - rec_digest=$(docker inspect \ - --format "{{index .Config.Labels \"${DIGEST_LABEL}\"}}" \ - "${tb_tag}" 2>/dev/null || echo "") - rec_pkgs=$(docker inspect \ - --format "{{index .Config.Labels \"${PKGS_LABEL}\"}}" \ - "${tb_tag}" 2>/dev/null || echo "") - - if [[ "${full_digest}" == "${rec_digest}" && "${pkgs_hash}" == "${rec_pkgs}" ]]; then - _log " full image and .txz set unchanged; skipping." - return 1 - fi - _log " full image or .txz set changed; rebuilding." - return 0 -} - -build_variant() { - local VERSION="$1" - local FULL_TAG="${FULL_IMAGE}:${VERSION}" - local TB_TAG="${TB_IMAGE}:${VERSION}" - _log "=== ${TB_TAG} ===" - - _log " Pulling ${FULL_TAG}..." - docker pull "${FULL_TAG}" - - local PKGS_HASH; PKGS_HASH="$(txz_hash "${PKGDIR}")" - - if ! needs_rebuild "${FULL_TAG}" "${TB_TAG}" "${PKGS_HASH}"; then - return 0 - fi - - local FULL_DIGEST - FULL_DIGEST=$(docker inspect --format '{{index .RepoDigests 0}}' "${FULL_TAG}") - - local WORKDIR; WORKDIR="$(mktemp -d /tmp/sbo-testbuild.XXXXXX)" - trap "rm -rf '${WORKDIR}'" RETURN - mkdir -p "${WORKDIR}/pkgs" - cp "${PKGDIR}"/sbopkg-*.txz "${PKGDIR}"/sbo-maintainer-tools-*.txz "${WORKDIR}/pkgs/" - - cat > "${WORKDIR}/Dockerfile" <<DOCKERFILE -FROM ${FULL_TAG} -LABEL maintainer="danix <danix@danix.xyz>" -COPY pkgs/*.txz /tmp/pkgs/ -RUN installpkg /tmp/pkgs/*.txz && rm -rf /tmp/pkgs -RUN /sbin/ldconfig -LABEL org.opencontainers.image.created="$(date -u +%Y-%m-%dT%H:%M:%SZ)" -LABEL org.opencontainers.image.title="SBo test-build env ${VERSION} (x86_64)" -LABEL org.opencontainers.image.description="Slackware ${VERSION} full + sbopkg + sbo-maintainer-tools" -LABEL slackware.version="${VERSION}" -LABEL ${DIGEST_LABEL}="${FULL_DIGEST}" -LABEL ${PKGS_LABEL}="${PKGS_HASH}" -CMD ["/bin/bash"] -DOCKERFILE - - local BUILD_FLAGS=() - [[ "${FORCE}" == "true" ]] && BUILD_FLAGS+=(--no-cache) - _log " Building ${TB_TAG}..." - docker build "${BUILD_FLAGS[@]}" -t "${TB_TAG}" "${WORKDIR}" - _log " Pushing ${TB_TAG}..." - docker push "${TB_TAG}" - _log "=== Done: ${TB_TAG} ===" -} - -require_pkgs -for VERSION in "${BUILD_VARIANTS[@]}"; do - build_variant "${VERSION}" -done -``` - -- [ ] **Step 2: Syntax check** - -Run: `bash -n .extras/image-builder/build-sbo-testbuild.sh` -Expected: exit 0. - -- [ ] **Step 3: shellcheck (if available)** - -Run: `shellcheck -x .extras/image-builder/build-sbo-testbuild.sh || true` - -- [ ] **Step 4: Commit** - -```bash -git add .extras/image-builder/build-sbo-testbuild.sh -git commit -m 'image-builder: build-sbo-testbuild.sh' -``` - ---- - -## Task 6: per-variant isolation - -The spec requires one variant failing not to block the other, non-zero exit at -the end. Add this to all three scripts' main loops. - -**Files:** -- Modify: `.extras/image-builder/bootstrap.sh` (main loop) -- Modify: `.extras/image-builder/build-full-image.sh` (main loop) -- Modify: `.extras/image-builder/build-sbo-testbuild.sh` (main loop) - -- [ ] **Step 1: Wrap each main loop** - -In each script, replace the plain `for VERSION ...` main loop with an -isolation wrapper. For **bootstrap.sh** (which also has the changelog gate): - -```sh -rc=0 -for VERSION in "${BUILD_VARIANTS[@]}"; do - if changelog_changed "${VERSION}"; then - if ! ( build_variant "${VERSION}" ); then - _warn "variant ${VERSION} failed; continuing." - rc=1 - fi - fi -done -exit "${rc}" -``` - -For **build-full-image.sh** and **build-sbo-testbuild.sh** (no changelog gate): - -```sh -rc=0 -for VERSION in "${BUILD_VARIANTS[@]}"; do - if ! ( build_variant "${VERSION}" ); then - _warn "variant ${VERSION} failed; continuing." - rc=1 - fi -done -exit "${rc}" -``` - -Note: `build_variant` runs in a subshell `( )` so a `set -e` abort inside it -(or an `_err` exit) is contained and the loop proceeds to the next variant. -For build-sbo-testbuild.sh keep the `require_pkgs` call **before** the loop -(a missing .txz is fatal for all variants, not per-variant). - -- [ ] **Step 2: Syntax check all three** - -Run: `for f in bootstrap build-full-image build-sbo-testbuild; do bash -n .extras/image-builder/$f.sh; done` -Expected: exit 0, no output. - -- [ ] **Step 3: Re-run lib self-check (unaffected, sanity)** - -Run: `bash .extras/image-builder/test-image-builder.sh` -Expected: `FAIL: 0`. - -- [ ] **Step 4: Commit** - -```bash -git add .extras/image-builder/bootstrap.sh .extras/image-builder/build-full-image.sh .extras/image-builder/build-sbo-testbuild.sh -git commit -m 'image-builder: per-variant isolation, non-zero exit on failure' -``` - ---- - -## Task 7: README - -**Files:** -- Create: `.extras/image-builder/README` - -- [ ] **Step 1: Write README** - -Create `.extras/image-builder/README` with the VM-side setup, verbatim: - -``` -sbo-testbuild image builder -=========================== - -Builds the docker images that .extras/test-build consumes: - docker.noland.dnx:5000/sbo-testbuild:current - docker.noland.dnx:5000/sbo-testbuild:15.0 - -Three scripts, chained (see .extras/docs/specs/2026-07-13-image-builder-design.md): - bootstrap.sh sbo-base:{ver} FROM scratch, base pkgs from NAS - build-full-image.sh sbo-full:{ver} FROM base, all series - build-sbo-testbuild.sh sbo-testbuild:{ver} FROM full, + sbopkg + tools - -All settings live in ./config. - -VM setup (docker.noland.dnx, Slackware x86_64, 4 vCPU / 4 GB / 80 GB) --------------------------------------------------------------------- -1. Install docker; enable the daemon. - -2. NFS-mount the two NAS trees read-only, named to match: - /mnt/nas/slackware64-current -> -current mirror tree - /mnt/nas/slackware64-15.0 -> 15.0 mirror tree - Each is a full mirror (PACKAGES.TXT, ChangeLog.txt, slackware64/, patches/, - extra/). Root must be able to read them (bootstrap runs installpkg as root). - -3. Run a LAN registry: - docker run -d --restart=always -p 5000:5000 \ - -v /opt/registry/data:/var/lib/registry --name registry registry:2 - -4. Mark the registry insecure (plain HTTP) on the VM AND every pulling client - (this dev box, the buildsystem VM). In /etc/docker/daemon.json: - { "insecure-registries": ["docker.noland.dnx:5000"] } - then restart docker. - -5. Drop the two prebuilt packages (built once, re-drop on upstream bumps): - /opt/sbo-testbuild/pkgs/sbopkg-*.txz - /opt/sbo-testbuild/pkgs/sbo-maintainer-tools-*.txz - -6. Install the nightly cron (root): - 0 4 * * * /path/to/.extras/image-builder/bootstrap.sh >> /var/log/sbo-testbuild.log 2>&1 - 15 4 * * * /path/to/.extras/image-builder/build-full-image.sh >> /var/log/sbo-testbuild.log 2>&1 - 30 4 * * * /path/to/.extras/image-builder/build-sbo-testbuild.sh >> /var/log/sbo-testbuild.log 2>&1 - -7. Ensure docker.noland.dnx resolves on the LAN (static IP or DNS). - -Manual first run ----------------- - ./bootstrap.sh --version current --force - ./build-full-image.sh --version current --force - ./build-sbo-testbuild.sh --version current --force -Then confirm: - docker pull docker.noland.dnx:5000/sbo-testbuild:current - docker run --rm docker.noland.dnx:5000/sbo-testbuild:current sbopkg -V - -Flags: --force (rebuild unconditionally), --version <current|15.0> (one variant). - -Tests ------ - bash test-image-builder.sh # pure-logic self-check, no docker -``` - -- [ ] **Step 2: Commit** - -```bash -git add .extras/image-builder/README -git commit -m 'image-builder: README with VM setup checklist' -``` - ---- - -## Task 8: executable bits + final check - -**Files:** -- Modify (mode): the four `.sh` scripts - -- [ ] **Step 1: chmod +x the scripts** - -```bash -chmod +x .extras/image-builder/bootstrap.sh \ - .extras/image-builder/build-full-image.sh \ - .extras/image-builder/build-sbo-testbuild.sh \ - .extras/image-builder/test-image-builder.sh -``` - -- [ ] **Step 2: Run the self-check once more** - -Run: `bash .extras/image-builder/test-image-builder.sh` -Expected: `PASS: N FAIL: 0`, exit 0. - -- [ ] **Step 3: Syntax-check all four** - -Run: `for f in .extras/image-builder/*.sh; do bash -n "$f" || echo "SYNTAX FAIL: $f"; done` -Expected: no `SYNTAX FAIL` lines. - -- [ ] **Step 4: Commit** - -```bash -git add .extras/image-builder -git commit -m 'image-builder: mark scripts executable' -``` - ---- - -## Deferred (not in this plan) - -- **Real end-to-end chain** on the VM (bootstrap→full→testbuild producing a - pushed `sbo-testbuild:current`): the user runs it once the VM/registry/mounts - exist, and reports back. Mirrors `.extras/test-build` Task 10. -- **`installed_in_base`** in `.extras/test-build`: now unblockable (query the - built image's package db) but tracked separately in the project memory - `test-build-image-builder-todo`. -- **i586** variants. -- **Registry auth/TLS** (LAN plain HTTP for now). -``` diff --git a/.extras/docs/specs/2026-07-13-docker-test-build-design.md b/.extras/docs/specs/2026-07-13-docker-test-build-design.md deleted file mode 100644 index 9f14d1e..0000000 --- a/.extras/docs/specs/2026-07-13-docker-test-build-design.md +++ /dev/null @@ -1,258 +0,0 @@ -# Docker test-build script - design - -Date: 2026-07-13 - -## Purpose - -A single bash script that verifies an already-published SBo package still -builds cleanly on a target Slackware version, inside a throwaway docker -container. This is the "test-build" step of this repo's maintenance loop -(track upstream -> bump -> **test-build** -> make tarball). - -The container is disposable, so unlike a real `slackrepo build` this runs -directly (no host install), and Claude may run it (per the repo's -`Who builds SlackBuilds` exception). - -## Scope and non-goals - -- One target package per run: `test-build <pkg>`. -- Two Slackware targets: `-current` (default) and `15.0` (`--stable`). -- Resolves the target's SBo dependency tree from the LOCAL SBo tree, builds - deps then target in the container, reports, discards everything. -- NOT a redistributable-package builder. The container's built `.txz` is - throwaway; the SBo submission tarball is made separately (post-commit hook) - only after a green run. -- Does NOT wrap slackrepo or sbo-batch-test. It REUSES sbo-batch-test's - resolver + build logic, adapted, but is an independent script. -- Shares the host kernel (docker). No kernel-module build claims. - -## Why not the overlay approach (sbo-batch-test) - -`sbo-batch-test` already resolves deps + builds + reports, but against a -LOCAL overlayfs over a 15.0 base. In docker the container IS the disposable -base, and overlayfs-over-docker-storage (itself overlayfs) is fragile. So we -drop the overlay layer and build directly in the container. We port -sbo-batch-test's dep resolver and its `build_one` chroot heredoc -(download -> md5 -> build -> installpkg -> status token) into a `docker run`. - -## Architecture - -Single self-contained bash script (`test-build`, location TBD in -implementation - likely `.extras/`). Host does resolution + orchestration; -the container does the actual building. - -``` -host: parse args, load config -host: pick version -> image tag + SBo tree -host: resolve dep tree (local tree, topo sort) [ported from sbo-batch-test] -host: apply current-vs-stable overrides -host: print final build order, Y/n confirm (--yes: still prints first) - docker run --rm -v <tree>:ro -v <pkg>:ro -v <cache/digest>:ro <image>: - for each dep in order: - cache hit (version match) -> installpkg from cache (CACHED) - else -> download/md5/build/installpkg + cache - target: download/md5/build/installpkg (always fresh) - target: sbopkglint on the built .txz -host: collect per-package results, print color summary -container discarded (--rm) -``` - -### Images (external input, not built by this script) - -The script does NOT build images. It consumes a ready image per version, -tagged locally: - -- current: `sbo-testbuild:current` -- 15.0: `sbo-testbuild:15.0` - -Each image is a FULL, patched Slackware install (SBo tests against a full -install; a minimal base causes false "missing dependency" results, a lesson -from sbo-batch-tester) plus `sbo-maintainer-tools` (for `sbopkglint`) and -`sbopkg` configured to that version's SBo repo (`:current` -> -current repo, -`:15.0` -> 15.0 repo). - -**How the images are produced is out of scope for this script.** A separate -nightly job (a LAN machine, images stored on the NAS, later served from a -local registry) builds and refreshes them. That builder is its own tool, -specced separately. - -For now the script assumes the tagged image is present locally: it runs -`docker run` against the tag and errors with a clear message if the image is -missing (pointing at the image-builder job). When the local registry exists, -a `docker pull` of the tag slots in ahead of the run via the same config, -no other change. - -Per-run cost is just the SlackBuild's own build time + ~1s container start. - -### Config (external file) - -`~/.config/sbo-testbuild/config`, empty defaults in-script, sourced if -present (same pattern as sbo-batch-test). Keys: - -```sh -SBO_TREE_CURRENT=/path/to/SBo-current # local SBo tree, -current -SBO_TREE_STABLE=/path/to/SBo-15.0 # local SBo tree, 15.0 -IMAGE_CURRENT=sbo-testbuild:current # ready image tag (built elsewhere) -IMAGE_STABLE=sbo-testbuild:15.0 -LOG_ROOT=/path/to/logs -PKG_CACHE=/path/to/cache # dep cache; empty = disabled -``` - -The version flag selects BOTH the image and the tree together (default -current; `--stable` / `15.0` switches both), so tree and image never mismatch. - -### current-vs-stable overrides - -A data file next to the config, `~/.config/sbo-testbuild/overrides`. It -encodes the known deltas between building on 15.0 and on -current. Applied -ONLY when the target version is `-current` (15.0 is the SBo baseline). Three -rule kinds: - -``` -# dep present in 15.0 but already in -current base -> drop from order -drop: rust -# dep renamed on -current -> map old -> new before lookup -rename: python3-foo -> foo -# dep removed from the -current tree -> fetch from SBo instead of local tree -fetch: somelib -``` - -Format: one rule per line, `kind: value` (rename uses `old -> new`), `#` -comments and blank lines ignored. - -### Dependency resolution + unknown deps - -Resolver is ported from sbo-batch-test (`resolve_target` / `_resolve_visit`, -DFS topo sort + cycle detection, `installed_in_base` check). Runs on the host -against the selected tree. - -After resolving and applying overrides, every dep must be accounted for: -in base, in the tree, or covered by a `fetch:` rule. If a dep is none of -those (and no rule covers it), the script **stops before building**, reports -it as UNMET, and asks the user to add an override rule and rerun, or abort. -Never silently skip or guess. - -### Confirmation - -Always print the final build order before building, with overrides marked -(dropped / renamed / fetch). Then `Y/n` to proceed. `--yes` skips the prompt -for non-interactive/agent runs but STILL prints the order first. - -`--dry-run` resolves, applies overrides, prints the order, and exits without -building (no confirm). - -### Build in container (ported from build_one) - -Deps then target, in order. For each package, inside the container: - -1. download sources (arch-specific `DOWNLOAD_x86_64`/`MD5SUM_x86_64` when - present, else `DOWNLOAD`/`MD5SUM`) -2. verify md5 -3. `bash <pkg>.SlackBuild` with `OUTPUT` forced to a known dir -4. `installpkg` the resulting `.txz` -5. write a status token read back by the host - -`fetch:` deps (removed from the -current tree) are pulled via the baked-in -`sbopkg`, which points at the image's matching SBo repo; everything else -builds from the mounted local tree. Start light: handle only the few known -`fetch:` cases, add rules as real packages need them. - -### Dependency cache - -Ported from sbo-batch-tester (`cache_decision`/`cache_path`/`cache_store`/ -`version_of`). A host dir caches built dep packages so consecutive runs reuse -them instead of rebuilding the whole tree. - -- Config `PKG_CACHE=/path` enables it; empty (default) disables. `--no-cache` - forces a full rebuild for one run. -- Bind-mounted read-only into the container; a cached dep is `installpkg`ed - instead of built (status `CACHED`). Storing is host-side: a freshly built - package is written to `OUTPUT` (a mounted dir the host reads back), and the - host copies it into the cache after the build (`cache_store`), so the cache - mount stays read-only. The **target always builds fresh** and refreshes its - own cache entry; it is never `CACHED`. -- Key = prog + version (build/arch/tag ignored). Layout mirrors the SBo tree: - `$PKG_CACHE/<image-digest>/<category>/<prog>/<prog>-<ver>-...txz`, one .txz - per prog. -- **Invalidation by image digest.** The cache is namespaced by the running - image's digest. When the digest changes (image updated by the nightly job), - the old namespace is stale: its deps were built against the previous image, - so they are ignored and rebuilt once under the new digest. This also keeps - the `:current` and `:15.0` caches separate (different images, different - digests). Old digest namespaces can be pruned. -- A per-dep version bump (`bump: OLD -> NEW`) rebuilds and re-caches that dep. -- Build-order line shows the outcome per dep: `cached (1.1)`, - `rebuild: 1.0 -> 1.1`, or `build (new)`; `--dry-run` shows the same without - building. - -### Target lint - -After the target builds, run `sbopkglint` on its `.txz` (tool baked into the -image). Fail-soft: findings reported but do not change SUCCESS; missing tool -prints a skip note. Target only (deps are vetted). - -### Report - -Per-package status + a color summary (screen) and a plain-text log under -`LOG_ROOT/<timestamp>/`: - -``` -<timestamp>/ - <prog>.log per-package full build/install output - summary.log plain recap - build-order.txt the resolved+overridden order actually used -``` - -Status values (from sbo-batch-test): -`SUCCESS CACHED DOWNLOAD-FAILED MD5-MISMATCH BUILD-FAILED INSTALL-FAILED -BLOCKED-BY-DEP UNMET-DEP`. `CACHED` = a dep installed from the cache instead -of rebuilt (target is never CACHED). `%README%` deps flagged as a reminder, -not built. - -A fully green run tells the user it is safe to make the SBo submission tarball -on the host (the container is no longer needed). - -## Options (planned) - -| Option | Effect | -|--------|--------| -| `--stable` / `15.0` | Target 15.0 (image + tree). Default is -current. | -| `--dry-run` | Resolve + apply overrides + print order, no build. | -| `--yes` | Skip the Y/n confirm (still prints the order). | -| `--no-cache` | Rebuild all deps this run, ignore/refresh the cache. | -| `--no-color` | Disable ANSI (auto-off when not a TTY). | -| `-h`, `--help` | Usage. | - -## Self-check - -A `test-logic.sh` (host, no docker) covering the pure logic: - -- dep topo order + cycle detection (ported tests from sbo-batch-test) -- override application: drop removes, rename maps, fetch marks source -- unknown-dep -> UNMET stop -- BLOCKED-BY-DEP propagation -- cache decision: `cached`/`rebuild: OLD -> NEW`/`build (new)`, digest - namespacing, version-bump eviction (ported from sbo-batch-tester's tests) - -Docker/build/installpkg paths are out of the self-check's reach (verified by -running a real target), same boundary as sbo-batch-test. - -## Resolved decisions - -- **Image**: a ready FULL-install image per version (full pkg set + - `sbo-maintainer-tools` + `sbopkg` on that version's repo), built by a - separate nightly job, NOT by this script. Script consumes it by tag. -- **Missing image**: error clearly, point at the image-builder job. A local - registry `docker pull` slots in later via the same tag, no script change. -- **`fetch:`**: via the image's `sbopkg` (per-version repo). Start with only - the few known cases; grow the override rules as needed. -- **Script location**: `.extras/` (a tool, not a package). - -## Out of scope / later - -- The nightly image-builder (full-install + tools + sbopkg per version, NAS - storage, local registry). Its own tool, specced when the LAN machine is set - up. -- Local registry pull step in this script (slots into the same config tag). -``` - diff --git a/.extras/docs/specs/2026-07-13-image-builder-design.md b/.extras/docs/specs/2026-07-13-image-builder-design.md deleted file mode 100644 index 9ee1a11..0000000 --- a/.extras/docs/specs/2026-07-13-image-builder-design.md +++ /dev/null @@ -1,216 +0,0 @@ -# sbo-testbuild image builder — design - -Date: 2026-07-13 -Status: approved (design) - -## Purpose - -Produce and serve the docker images the `.extras/test-build` tool consumes. -`.extras/test-build` verifies a published SBo package still builds on a target -Slackware version inside a throwaway container; it does **not** build the -images, it only checks image presence by tag and errors if absent. This is the -missing image-builder job. - -Images produced (x86_64 only for now): - -- `sbo-testbuild:current` — Slackware -current, full install + sbopkg + - sbo-maintainer-tools -- `sbo-testbuild:15.0` — Slackware 15.0, same - -Served from a LAN docker registry so `.extras/test-build`'s `require_image` -can `docker pull` them, and so `installed_in_base` can later query the image's -package db. - -## Infrastructure - -Self-hosted on a Slackware VM (proxmox node), fully offline from the public -internet for the Slackware bits: - -- **Host**: `docker.noland.dnx` (LAN DNS), Slackware x86_64, 4 vCPU / 4 GB RAM - / 80 GB disk. -- **NAS trees**: two standard Slackware mirror trees NFS-mounted read-only: - - `/mnt/nas/slackware64-current` - - `/mnt/nas/slackware64-15.0` - Each is a full mirror tree (`PACKAGES.TXT`, `ChangeLog.txt`, `slackware64/` - series dirs, `patches/`, `extra/`). -- **Registry**: `registry:2` container on the VM, port 5000, plain HTTP (LAN - only). Pull path `docker.noland.dnx:5000/sbo-testbuild:{tag}`. -- **Local .txz drop**: `/opt/sbo-testbuild/pkgs/` holds `sbopkg-*.txz` and - `sbo-maintainer-tools-*.txz`, provided by the user (sbopkg packaged from the - user's SBo repo). Installed into the image, not built in it (hermetic, no - build-time deps). - -Both `docker.noland.dnx` and every client that pulls (this dev box, the -`buildsystem` VM) need the registry marked insecure in -`/etc/docker/daemon.json`: -`{"insecure-registries":["docker.noland.dnx:5000"]}`, then `docker` restarted. - -i586 is out of scope for now; the variant list is written to extend to it -later without restructuring. - -## Architecture - -Three scripts, chained, adapted from the forge repo -`slackware/docker-images` (`scripts/bootstrap.sh`, `build-full-image.sh`, -`build-builder-image.sh`). The forge `build-builder-image.sh` is **not** used: -the full image already ships the toolchain (series d, kde, x, etc.), so a -separate builder layer is redundant here. - -``` -bootstrap.sh FROM scratch; installpkg base pkgs read from NAS - → sbo-base:{ver} via file://; configure rootfs for container use. - -build-full-image.sh FROM sbo-base:{ver}; slackpkg install every series - → sbo-full:{ver} (a ap d e f k kde l n t tcl x xap xfce y). - -build-sbo-testbuild.sh FROM sbo-full:{ver}; installpkg sbopkg + - → sbo-testbuild:{ver} sbo-maintainer-tools from /opt/sbo-testbuild/pkgs. -``` - -All three push to `docker.noland.dnx:5000`. - -### Namespacing - -Forge scripts push to `registry.slackware.nl/slackware/slackware*`. Re-namespace -to the LAN registry with our own image names: - -- `docker.noland.dnx:5000/sbo-base:{ver}` -- `docker.noland.dnx:5000/sbo-full:{ver}` -- `docker.noland.dnx:5000/sbo-testbuild:{ver}` - -`sbo-base` and `sbo-full` are intermediate; `.extras/test-build` only pulls -`sbo-testbuild`. - -## Configuration - -Single shared file `config`, sourced by all scripts and the test: - -```sh -REGISTRY=docker.noland.dnx:5000 -MIRROR=file:///mnt/nas -VARIANTS=(current 15.0) # x86_64 only for now -PKGDIR=/opt/sbo-testbuild/pkgs # sbopkg + sbo-maintainer-tools .txz -``` - -The forge scripts hardcode `MIRROR=https://slackware.nl/slackware` and -`REGISTRY=registry.slackware.nl/...`; both move to the shared config. - -### NAS path mapping - -bootstrap derives the per-variant repo dir as `slackware64-${VERSION}` for -x86_64 (`DIRSUFFIX=64`). With `MIRROR=file:///mnt/nas` this resolves to -`file:///mnt/nas/slackware64-current/...` and `.../slackware64-15.0/...`, -matching the mount points. Mounts must be named to match. - -## Patches to the forge scripts - -1. **`file://`-aware fetch.** Forge downloads `PACKAGES.TXT` and packages with - `curl` over http. Replace with one helper in `lib.sh` that detects a - `file://` (or bare-local) MIRROR and uses `cp`, otherwise `curl`. curl's - own `file://` support is unreliable across builds, so branch explicitly. -2. **Drop i586.** `ALL_VARIANTS` becomes the config `VARIANTS` (x86_64 only). - All the `ARCH_SUFFIX`/`-i586` machinery collapses to the x86_64 path. -3. **Re-namespace + LAN registry** (see Namespacing). -4. **MIRROR → `file:///mnt/nas`** (see config). - -## Data flow and rebuild gating - -Nightly cron on the VM (root; `bootstrap` needs `installpkg`): - -```cron - 0 4 * * * bootstrap.sh -15 4 * * * build-full-image.sh -30 4 * * * build-sbo-testbuild.sh -``` - -Rebuild gating (kept from the forge scripts, extended): - -- **bootstrap**: sha256 of the NAS `ChangeLog.txt` (read via `file://`) - compared to a stored hash under `HASH_DIR`; skip the variant if unchanged. -- **build-full-image**: base image digest recorded as a label on the full - image; pull base, compare, skip if unchanged. -- **build-sbo-testbuild**: full image digest label, same pattern; **plus** a - hash of the `.txz` set in `PKGDIR`, so a sbopkg / sbo-maintainer-tools bump - forces a rebuild even when the full image is unchanged. - -Net effect: a quiet NAS tree makes the whole chain no-op in seconds; a --current churn cascades base → full → testbuild. - -### Local-registry digest consistency - -The forge scripts read the base/full digest via `docker inspect RepoDigests` -after `docker pull`. Self-hosted, those intermediate images live in the LAN -registry, so pull-then-inspect must target `docker.noland.dnx:5000`. Since -every image name is derived from `REGISTRY`, re-namespacing handles this with -no extra logic. - -## Error handling - -- `set -euo pipefail` in every script (forge already has it). -- **Missing NAS mount**: guard `[[ -d /mnt/nas/slackware64-$ver ]]` first; - `_err` if absent. -- **Missing `.txz`**: `build-sbo-testbuild` errors before building if - `PKGDIR` lacks a `sbopkg-*.txz` or `sbo-maintainer-tools-*.txz` — never push - a testbuild image missing sbopkg. -- **Registry unreachable**: `docker push` fails under `set -e`; the chain - aborts. The base/full may already be built locally; cron retries next night. - Acceptable. -- **Per-variant isolation**: one variant failing (e.g. current) must not block - the other (15.0). Each variant runs wrapped; log the failure, continue, and - exit non-zero at the end so cron logs surface it. - -## Testing - -Pure-logic pieces get a `test-image-builder.sh` self-check, matching the -existing `.extras/test-build` pattern (`.extras/test-logic.sh`, assert-based, -no framework). No docker daemon in tests — hermetic. - -- **`file://` fetch helper**: local file resolves via cp; missing local file - errors; an http MIRROR still routes to curl. -- **`.txz`-set hash**: stable across file reordering; changes when a file's - content changes. -- **variant guard**: a missing mount dir yields non-zero. - -The real end-to-end chain is verified by a manual first run on the VM (the user -runs it and reports back), the same division of labor as `.extras/test-build` -Task 10. - -## Layout - -``` -.extras/image-builder/ -├── config REGISTRY, MIRROR, VARIANTS, PKGDIR -├── lib.sh shared: _log/_warn/_err, fetch, txz_hash, guards -├── bootstrap.sh base image (root; installpkg from NAS) -├── build-full-image.sh full image (slackpkg all series) -├── build-sbo-testbuild.sh testbuild image (installpkg sbopkg + tools) -├── test-image-builder.sh pure-logic self-check -└── README VM-side setup, reproducible -``` - -Lives in `.extras/` for now (per the user), relocatable later. - -The README carries the user-side checklist so it is reproducible and not -trapped in chat: - -- VM specs (4/4/80), Slackware x86_64, docker installed. -- Two NFS mounts (ro), root-readable, named `slackware64-current` / - `slackware64-15.0` under `/mnt/nas`. -- `registry:2` container: `-v /opt/registry/data:/var/lib/registry`, - `--restart=always`, port 5000. -- `/etc/docker/daemon.json` insecure-registry entry on the VM **and** on every - pulling client (this dev box, `buildsystem`), docker restarted. -- Drop `sbopkg-*.txz` and `sbo-maintainer-tools-*.txz` into - `/opt/sbo-testbuild/pkgs/`. -- Install the three cron lines. -- VM static IP / LAN DNS for `docker.noland.dnx`. - -## Out of scope - -- i586 images (extensible later via `VARIANTS` / arch machinery). -- Building sbopkg / sbo-maintainer-tools from source in the image (installed - from prebuilt `.txz`). -- The forge `build-builder-image.sh` toolchain layer (redundant with full). -- `installed_in_base` implementation in `.extras/test-build` — unblocked by - this work (queries the built image's package db) but tracked separately. -- Registry auth / TLS (LAN-only plain HTTP). diff --git a/.extras/image-builder/README b/.extras/image-builder/README deleted file mode 100644 index fccab11..0000000 --- a/.extras/image-builder/README +++ /dev/null @@ -1,69 +0,0 @@ -sbo-testbuild image builder -=========================== - -Builds the docker images that .extras/test-build consumes: - docker.noland.dnx:5000/sbo-testbuild:current - docker.noland.dnx:5000/sbo-testbuild:15.0 - -Three scripts, chained (see .extras/docs/specs/2026-07-13-image-builder-design.md): - bootstrap.sh sbo-base:{ver} FROM scratch, base pkgs from NAS - build-full-image.sh sbo-full:{ver} FROM base, all series - build-sbo-testbuild.sh sbo-testbuild:{ver} FROM full, + sbopkg + tools - -All settings live in ./config. - -VM setup (docker.noland.dnx, Slackware x86_64, 4 vCPU / 4 GB / 80 GB) --------------------------------------------------------------------- -1. Install docker; enable the daemon. - -2. NFS-mount the two NAS trees read-only, named to match: - /mnt/nas/slackware64-current -> -current mirror tree - /mnt/nas/slackware64-15.0 -> 15.0 mirror tree - Each is a full mirror (PACKAGES.TXT, ChangeLog.txt, slackware64/, patches/, - extra/). Root must be able to read them (bootstrap runs installpkg as root). - -3. Run a LAN registry: - docker run -d --restart=always -p 5000:5000 \ - -v /opt/registry/data:/var/lib/registry --name registry registry:2 - -4. Mark the registry insecure (plain HTTP) on the VM AND every pulling client - (this dev box, the buildsystem VM). In /etc/docker/daemon.json: - { "insecure-registries": ["docker.noland.dnx:5000"] } - then restart docker. - -5. Drop the two prebuilt packages (built once, re-drop on upstream bumps): - /opt/sbo-testbuild/pkgs/sbopkg-*.txz - /opt/sbo-testbuild/pkgs/sbo-maintainer-tools-*.txz - -6. Install the nightly cron (root). The NAS repos sync at 01:00 and 02:00, so - the chain runs after and both variants are ready well before the ~09:00 work - start. No --force: each script self-gates (bootstrap on the ChangeLog hash, - full-image on the base-image digest, build-sbo-testbuild on the full-image - digest + tools .txz hash), so an unchanged night is a cheap no-op. -current - moves daily and rebuilds most nights; 15.0 is frozen stable and rebuilds only - on a real repo update. Deployed schedule on docker.noland.dnx: - # -current (ready ~04:35) - 0 3 * * * /path/to/.extras/image-builder/bootstrap.sh --version current >> /var/log/sbo-testbuild.log 2>&1 - 20 3 * * * /path/to/.extras/image-builder/build-full-image.sh --version current >> /var/log/sbo-testbuild.log 2>&1 - 30 4 * * * /path/to/.extras/image-builder/build-sbo-testbuild.sh --version current >> /var/log/sbo-testbuild.log 2>&1 - # 15.0 (ready ~06:35) - 0 5 * * * /path/to/.extras/image-builder/bootstrap.sh --version 15.0 >> /var/log/sbo-testbuild.log 2>&1 - 20 5 * * * /path/to/.extras/image-builder/build-full-image.sh --version 15.0 >> /var/log/sbo-testbuild.log 2>&1 - 30 6 * * * /path/to/.extras/image-builder/build-sbo-testbuild.sh --version 15.0 >> /var/log/sbo-testbuild.log 2>&1 - -7. Ensure docker.noland.dnx resolves on the LAN (static IP or DNS). - -Manual first run ----------------- - ./bootstrap.sh --version current --force - ./build-full-image.sh --version current --force - ./build-sbo-testbuild.sh --version current --force -Then confirm: - docker pull docker.noland.dnx:5000/sbo-testbuild:current - docker run --rm docker.noland.dnx:5000/sbo-testbuild:current sbopkg -V - -Flags: --force (rebuild unconditionally), --version <current|15.0> (one variant). - -Tests ------ - bash test-image-builder.sh # pure-logic self-check, no docker diff --git a/.extras/image-builder/bootstrap.sh b/.extras/image-builder/bootstrap.sh deleted file mode 100755 index 9893e66..0000000 --- a/.extras/image-builder/bootstrap.sh +++ /dev/null @@ -1,552 +0,0 @@ -#!/bin/bash -# bootstrap.sh — build the sbo-base:{ver} image FROM scratch from NAS trees. -# Run as root (installpkg). Adapted from forge slackware/docker-images. -set -euo pipefail -HERE="$(cd "$(dirname "$0")" && pwd)" -source "${HERE}/config" -LOG_TAG=bootstrap -source "${HERE}/lib.sh" - -REGISTRY_IMAGE="${REGISTRY}/sbo-base" -[[ "${EUID}" -eq 0 ]] || _err "run as root." -mkdir -p "${HASH_DIR}" - -# ============================================================================ -# Package list -# -# Names only - the version and build suffix are resolved automatically -# from PACKAGES.TXT for the x86_64 target architecture. -# -# slackpkg and its runtime dependencies (perl, gnupg2, gpgme) are included -# so users can run slackpkg inside the container without extra setup. -# ============================================================================ - -PACKAGES=( - # Base system - aaa_base - coreutils - aaa_glibc-solibs - aaa_libraries - aaa_terminfo - pam - cracklib - libpwquality - acl - attr - bash - bin - brotli - bzip2 - c-ares - cpio - curl - cyrus-sasl - devs - dialog - diffutils - duktape - elvis - etc - file - findutils - flex - gawk - gnutls - grep - groff - gzip - iproute2 - iptables - jansson - less - libcgroup - libpsl - librsvg - libtermcap - mtr - ncurses - net-tools - network-scripts - nghttp2 - nghttp3 - ngtcp2 - nvi - openssh - patch - pcre2 - pinentry - pkgtools - polkit - procps-ng - quota - rsync - screen - sed - elogind - shadow - sharutils - strace - sudo - sysfsutils - sysvinit - sysvinit-scripts - tar - time - tree - eudev - usbutils - utempter - util-linux - wget - which - whois - xz - zlib - # Package management - series ap - slackpkg - # Perl runtime required by slackpkg - series d - perl - # e2fsprogs provides libcom_err.so.2 required by perl on 15.0 - e2fsprogs - # libunistring - Unicode string library required by gnupg2 dependencies - libunistring - # gnupg2 runtime libraries (series l) - required for gpg to start at all - libgpg-error - libgcrypt - libassuan - libksba - npth - sqlite - icu4c - # GPG/PKCS support for slackpkg (series l) - gpgme - # GnuPG binaries (series n) - # gnupg = version 1, called by slackpkg on Slackware 15.0 - # gnupg2 = version 2, called by slackpkg on -current - gnupg - gnupg2 - # Full OpenSSL and CA bundle - required for HTTPS inside the container - openssl - ca-certificates -) - -# ============================================================================ -# Argument parsing -# ============================================================================ - -FORCE=false -OPT_VERSION="" -while [[ $# -gt 0 ]]; do - case "$1" in - --version) OPT_VERSION="$2"; shift 2 ;; - --force) FORCE=true; shift ;; - *) _err "unknown argument: $1" ;; - esac -done -if [[ -n "${OPT_VERSION}" ]]; then - BUILD_VARIANTS=("${OPT_VERSION}") -else - BUILD_VARIANTS=("${VARIANTS[@]}") -fi - -# ============================================================================ -# ChangeLog tracking -# ============================================================================ - -# changelog_changed VERSION -> 0 if changed (or --force), else 1 -changelog_changed() { - local version="$1" - require_mount "${version}" # loud fail if NAS unmounted, not silent skip - local repo_key="slackware64-${version}" - local hash_file="${HASH_DIR}/${repo_key}.sha256" - local url; url="$(mirror_path "${MIRROR}" "${repo_key}/ChangeLog.txt")" - local tmp; tmp="$(mktemp)" - if ! fetch "${url}" "${tmp}"; then - rm -f "${tmp}" - _warn "${repo_key}: cannot read ChangeLog (${url}); skipping." - return 1 - fi - local live; live=$(sha256sum "${tmp}" | cut -d' ' -f1); rm -f "${tmp}" - if [[ "${FORCE}" == "true" ]]; then - echo "${live}" > "${hash_file}"; return 0 - fi - local stored=""; [[ -f "${hash_file}" ]] && stored=$(cat "${hash_file}") - if [[ "${live}" == "${stored}" ]]; then - _log "${repo_key}: ChangeLog unchanged; skipping."; return 1 - fi - _log "${repo_key}: ChangeLog changed."; echo "${live}" > "${hash_file}"; return 0 -} - -# ============================================================================ -# Package discovery via PACKAGES.TXT -# -# Search order: patches/ -> main repo -> extra/ -# This mirrors Slackware's own install priority: a patched package in -# patches/ always takes precedence over the same package in the main tree. -# ============================================================================ - -# download_pkgtxt PKGTXT_DIR PKG_PATH -# Downloads PACKAGES.TXT from patches/, the repo root, and extra/ into -# PKGTXT_DIR. Sets global associative array PKGTXT[loc] -> local file path. -# Uses -L to follow HTTP redirects (some mirrors use them) and checks that -# the downloaded file is non-empty before marking it as available. -declare -gA PKGTXT=() - -download_pkgtxt() { - local PKGTXT_DIR="$1" - local PKG_PATH="$2" - mkdir -p "${PKGTXT_DIR}" - PKGTXT=() - local LOC URL DEST - for LOC in patches main extra; do - case "${LOC}" in - patches) URL="${PKG_PATH}/patches/PACKAGES.TXT" ;; - main) URL="${PKG_PATH}/PACKAGES.TXT" ;; - extra) URL="${PKG_PATH}/extra/PACKAGES.TXT" ;; - esac - DEST="${PKGTXT_DIR}/${LOC}.txt" - if fetch "${URL}" "${DEST}" && [[ -s "${DEST}" ]]; then - PKGTXT["${LOC}"]="${DEST}" - _log " PACKAGES.TXT [${LOC}]: $(wc -l < "${DEST}") lines" - else - rm -f "${DEST}" - _log " PACKAGES.TXT [${LOC}]: not available at ${URL}" - fi - done -} - -# find_package NAME -# Searches PKGTXT files in order: patches -> main -> extra. -# Prints the path relative to PKG_PATH, e.g.: -# slackware64/a/bash-5.2-x86_64-1.txz -# -# Base name extraction uses the Slackware convention: the filename has the -# form PKGBASE-VERSION-ARCH-BUILD.txz where VERSION, ARCH and BUILD are -# guaranteed to contain no dashes. Reversing, cutting the last three fields, -# and reversing again always yields the correct PKGBASE: -# echo aaa_glibc-solibs-2.42-x86_64-1.txz | rev | cut -d- -f4- | rev -# -> aaa_glibc-solibs -find_package() { - local PKG="$1" - [[ "${#PKGTXT[@]}" -gt 0 ]] || { - _warn "find_package called with empty PKGTXT array - PACKAGES.TXT not loaded" - return 1 - } - for LOC in patches main extra; do - [[ -v "PKGTXT[${LOC}]" ]] || continue - local RESULT - RESULT=$(awk -v want="${PKG}" ' - /^PACKAGE NAME:/ { - filename = $NF - # PKGBASE = all fields except the last three (VERSION ARCH BUILD) - n = split(filename, parts, "-") - base = parts[1] - for (i = 2; i <= n-3; i++) base = base "-" parts[i] - if (base == want) found = filename - } - /^PACKAGE LOCATION:/ && found { - loc = $NF; sub(/^\.\//, "", loc) - print loc "/" found - found = ""; exit - } - ' "${PKGTXT[${LOC}]}") - if [[ -n "${RESULT}" ]]; then - echo "${RESULT}" - return 0 - fi - done - return 1 -} - -# ============================================================================ -# Build one variant -# ============================================================================ - -build_variant() { - local VERSION="$1" - require_mount "${VERSION}" - local REPO_KEY="slackware64-${VERSION}" - local PKG_PATH; PKG_PATH="$(mirror_path "${MIRROR}" "${REPO_KEY}")" - local TAG="${REGISTRY_IMAGE}:${VERSION}" - # Bake the LAN HTTP mirror into the base image: the full-image build serves - # the NFS mirror over HTTP on the bridge (see build-full-image.sh), rooted - # at the variant's mirror tree, so slackpkg pulls from LAN not the internet. - # The URL is variant-agnostic (the server's --directory selects the tree). - local SLACKPKG_MIRROR="http://${HTTP_MIRROR_HOST}:${HTTP_MIRROR_PORT}/" - - _log "=== Building ${TAG} from ${PKG_PATH} ===" - - local WORKDIR - WORKDIR="$(mktemp -d /tmp/slackware-bootstrap.XXXXXX)" - # shellcheck disable=SC2064 - trap "rm -rf '${WORKDIR}'" EXIT - - local ROOTFS="${WORKDIR}/rootfs" - local PKGCACHE="${WORKDIR}/packages" - local PKGTXT_DIR="${WORKDIR}/pkgtxt" - mkdir -p "${ROOTFS}" "${PKGCACHE}" - - # --- Download PACKAGES.TXT index files --- - download_pkgtxt "${PKGTXT_DIR}" "${PKG_PATH}" - - # --- Download packages --- - _log "Locating and downloading packages..." - local PKG RELPATH FILENAME URL - for PKG in "${PACKAGES[@]}"; do - RELPATH=$(find_package "${PKG}") || { - _warn " '${PKG}' not found in any PACKAGES.TXT - skipping" - continue - } - FILENAME="${RELPATH##*/}" - URL="${PKG_PATH}/${RELPATH}" - _log " ${FILENAME}" - fetch "${URL}" "${PKGCACHE}/${FILENAME}" || { - _warn " Failed to download ${URL}" - } - done - - # --- Install packages --- - _log "Installing packages..." - for PKG in "${PKGCACHE}"/*.txz; do - [[ -f "${PKG}" ]] || continue - ROOT="${ROOTFS}" installpkg --terse "${PKG}" - done - - # --- Run ldconfig to create library symlinks --- - # Some packages (notably aaa_libraries on stable releases) install - # versioned library files (e.g. libtinfo.so.6.3) and rely on ldconfig - # to create the unversioned symlinks (e.g. libtinfo.so.6). On -current - # the symlinks are included in the package tarballs directly; on 15.0 - # they are not. Running ldconfig here ensures the symlinks exist before - # any chroot operations and before the dependency check below. - _log "Running ldconfig..." - chroot "${ROOTFS}" /sbin/ldconfig - - # --- Check for missing library dependencies --- - # Run ldd on key binaries inside the chroot and report anything missing - # before it surfaces as a runtime failure inside the container. - _log "Checking for missing library dependencies in key binaries..." - local DEPCHECK_OK=true - for BIN in /usr/bin/gpg /usr/bin/gpg2 /usr/sbin/slackpkg /usr/bin/perl /bin/wget /usr/bin/openssl; do - [[ -f "${ROOTFS}${BIN}" ]] || continue - local MISSING_LIBS - MISSING_LIBS=$(chroot "${ROOTFS}" ldd "${BIN}" 2>/dev/null | grep "not found" || true) - if [[ -n "${MISSING_LIBS}" ]]; then - _warn "${BIN} has unresolved dependencies:" - while IFS= read -r lib; do - _warn " ${lib}" - done <<< "${MISSING_LIBS}" - DEPCHECK_OK=false - fi - done - if [[ "${DEPCHECK_OK}" == "true" ]]; then - _log "Dependency check passed." - else - _warn "Add the missing packages to PACKAGES and rebuild with --force." - fi - - # --- Prepare rootfs for container use --- - _log "Configuring rootfs for container use..." - - # Remove the empty update.d hook directory so that update-ca-certificates - # does not look for run-parts (which ships with dcron, not installed here). - rm -rf "${ROOTFS}/etc/ca-certificates/update.d" - - # Disable init scripts that manage hardware absent in a container - local SVC - for SVC in rc.acpid rc.pcmcia rc.setterm rc.udev; do - [[ -f "${ROOTFS}/etc/rc.d/${SVC}" ]] && chmod -x "${ROOTFS}/etc/rc.d/${SVC}" - done - - # Each container generates its own SSH host keys on first start - rm -f "${ROOTFS}"/etc/ssh/*key* - - # /etc/mtab should reflect the container's actual mounts - rm -f "${ROOTFS}/etc/mtab" - ln -s /proc/mounts "${ROOTFS}/etc/mtab" - - # No hardware clock in a container - sed -i -e '/^if \[ -x \/sbin\/hwclock/,/^fi$/s/^/#/' \ - "${ROOTFS}/etc/rc.d/rc.S" 2>/dev/null || true - - # The filesystem write-check would drop the container into a recovery shell - sed -i -e '/^if touch \/fsrwtestfile/,/^fi$/s/^/#/' \ - "${ROOTFS}/etc/rc.d/rc.S" 2>/dev/null || true - - # Skip all filesystem checks at boot - touch "${ROOTFS}/etc/fastboot" - - # setterm is not useful in a container - sed -i -e '/\/bin\/setterm/s/^/# /' \ - "${ROOTFS}/etc/rc.d/rc.M" 2>/dev/null || true - - # Cannot write to the hardware clock at shutdown - sed -i -e '/systohc/s/^/# /' \ - "${ROOTFS}/etc/rc.d/rc.6" 2>/dev/null || true - - # Container-appropriate fstab - cat > "${ROOTFS}/etc/fstab" <<'FSTAB' -devtmpfs /dev devtmpfs defaults 0 0 -devpts /dev/pts devpts gid=5,mode=620 0 0 -tmpfs /dev/shm tmpfs defaults,nodev,nosuid,mode=1777 0 0 -FSTAB - - # No TTYs in a container; disable agetty and reduce console definitions - sed -i -e '/agetty/s/^c/#c/' \ - "${ROOTFS}/etc/inittab" 2>/dev/null || true - sed -i -e '/^c3\|^c4\|^c5\|^c6/s/^/# /' \ - "${ROOTFS}/etc/inittab" 2>/dev/null || true - - # Invalidate the root password (containers use other means of access) - sed -i -e '/^root/s/^root::/root:!:/' \ - "${ROOTFS}/etc/shadow" 2>/dev/null || true - - # Terminal configuration - printf 'export TERM=linux\n' > "${ROOTFS}/etc/profile.d/term.sh" - chmod +x "${ROOTFS}/etc/profile.d/term.sh" - - # Source /etc/profile from .bashrc so interactive sessions are fully - # initialised (Docker sets / as the home directory for root in containers) - printf '. /etc/profile\n' > "${ROOTFS}/.bashrc" - - # DNS resolver - printf 'nameserver 1.1.1.1\nnameserver 8.8.4.4\n' \ - >> "${ROOTFS}/etc/resolv.conf" - - # Configure slackpkg for silent non-interactive use in a container - if [[ -f "${ROOTFS}/etc/slackpkg/slackpkg.conf" ]]; then - sed -i 's/DIALOG=on/DIALOG=off/' "${ROOTFS}/etc/slackpkg/slackpkg.conf" - sed -i 's/POSTINST=on/POSTINST=off/' "${ROOTFS}/etc/slackpkg/slackpkg.conf" - sed -i 's/SPINNING=on/SPINNING=off/' "${ROOTFS}/etc/slackpkg/slackpkg.conf" - sed -i 's/WGETFLAGS="--passive-ftp/& --no-verbose/' \ - "${ROOTFS}/etc/slackpkg/slackpkg.conf" - fi - - # Configure the slackpkg mirror - [[ -f "${ROOTFS}/etc/slackpkg/mirrors" ]] && \ - printf '%s\n' "${SLACKPKG_MIRROR}" >> "${ROOTFS}/etc/slackpkg/mirrors" - - # slackpkg requires an explicit opt-in for -current - if [[ "${VERSION}" == "current" ]]; then - mkdir -p "${ROOTFS}/var/lib/slackpkg" - touch "${ROOTFS}/var/lib/slackpkg/current" - fi - - # Set a predictable locale for all chroot operations. - # Without this, perl emits locale warnings when the host locale - # (e.g. en_US.UTF-8) is not installed in the minimal rootfs. - export LC_ALL=C - - # Rebuild the CA certificate database inside the rootfs. - # Without this, wget cannot verify Let's Encrypt certificates and every - # HTTPS download by slackpkg fails with an SSL error. That SSL error in - # turn causes the interactive "do you want to import the GPG key?" prompt - # even when -batch=on -default_answer=y is set. - _log "Refreshing CA certificate database..." - chroot "${ROOTFS}" /usr/sbin/update-ca-certificates --fresh 1>/dev/null || _warn "CA certificate update failed; SSL verification may not work correctly." - - # Import the Slackware GPG key and update the package list. - # 'yes YES |' feeds a pre-emptive answer to the interactive import prompt - # as a safety net; with the CA database now valid the prompt should not - # appear because the download succeeds on the first attempt. - _log "Importing Slackware GPG key via slackpkg..." - # Check slackpkg's own exit status, not the pipeline's: 'yes' takes SIGPIPE - # when slackpkg exits first, which can make the pipeline non-zero even on - # success (false "GPG update failed" warning). Wrapping in 'if' exempts the - # pipeline from set -e/pipefail so we can read PIPESTATUS[1] (slackpkg's own - # status) instead of the SIGPIPE-poisoned pipeline status. - local gpg_rc - if yes YES | chroot "${ROOTFS}" /usr/sbin/slackpkg -batch=on -default_answer=y update gpg - then gpg_rc="${PIPESTATUS[1]}"; else gpg_rc="${PIPESTATUS[1]}"; fi - [[ "${gpg_rc}" -eq 0 ]] || _warn "slackpkg GPG update failed; check the output above." - - _log "Updating slackpkg package list..." - chroot "${ROOTFS}" /usr/sbin/slackpkg -batch=on -default_answer=y update || _warn "slackpkg update failed; cache will build on first container run." - - # --- Clean up --- - _log "Cleaning up rootfs..." - rm -f "${ROOTFS}"/boot/* - rm -f "${ROOTFS}"/tmp/[A-Za-z]* - rm -f "${ROOTFS}"/var/mail/* - rm -rf "${ROOTFS}"/dev/* - rm -rf "${ROOTFS}"/usr/share/locale/* - rm -rf "${ROOTFS}"/usr/info/* - rm -rf "${ROOTFS}"/usr/man/* - (cd "${ROOTFS}/usr/doc" 2>/dev/null && \ - find . -type d -mindepth 2 -maxdepth 2 | grep -v '/cups-' | \ - xargs rm -rf) || true - rm -rf "${ROOTFS}"/usr/doc/*/html - rm -f "${ROOTFS}"/usr/doc/*/*.{pdf,db,gz,bz2,xz,txt,TXT} 2>/dev/null || true - rm -rf "${ROOTFS}"/usr/share/gtk-doc - rm -rf "${ROOTFS}"/usr/share/help - find "${ROOTFS}"/usr/share/ -type d -name doc | xargs rm -rf - # Keep only the terminfo entries the container actually needs - find "${ROOTFS}"/usr/share/terminfo/ -type f \ - ! -name 'linux' ! -name 'xterm' ! -name 'screen.linux' \ - -delete 2>/dev/null || true - find "${ROOTFS}"/usr/share/terminfo/ -xtype l -delete 2>/dev/null || true - # gpg-agent socket files left behind by slackpkg in the chroot. - # tar cannot archive Unix domain sockets; removing them eliminates - # the "socket ignored" warnings when the image tarball is created. - rm -f "${ROOTFS}"/root/.gnupg/S.* - - # Kernel-specific content has no meaning in a container - rm -rf "${ROOTFS}"/usr/src - rm -rf "${ROOTFS}"/lib/modules - rm -rf "${ROOTFS}"/lib64/modules 2>/dev/null || true - - # --- Build Docker image --- - _log "Creating image tarball..." - tar -C "${ROOTFS}" -czf "${WORKDIR}/rootfs.tar.gz" . - - local BUILD_DATE - BUILD_DATE="$(date -u +%Y-%m-%dT%H:%M:%SZ)" - - cat > "${WORKDIR}/Dockerfile" <<DOCKERFILE -FROM scratch -LABEL maintainer="Eric Hameleers <alien@slackware.com>" -ADD rootfs.tar.gz / -ENV container=docker -ENV PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin -LABEL org.opencontainers.image.created="${BUILD_DATE}" -LABEL org.opencontainers.image.title="Slackware Linux ${VERSION} (x86_64)" -LABEL org.opencontainers.image.description="Slackware Linux ${VERSION} base container image (x86_64)" -LABEL org.opencontainers.image.vendor="Slackware Linux" -LABEL org.opencontainers.image.licenses="GPL-2.0-or-later AND LGPL-2.1-or-later" -LABEL org.opencontainers.image.url="http://www.slackware.com" -LABEL org.opencontainers.image.source="https://forge.slackware.nl/slackware/docker-images" -LABEL org.opencontainers.image.revision="${VERSION}" -LABEL slackware.version="${VERSION}" -LABEL slackware.arch="x86_64" -CMD ["/bin/bash"] -DOCKERFILE - - # Explicit exit checks: build_variant runs in an `if ! (...)` condition - # (see Main), which suppresses `set -e` in this subshell, so a failed build - # would otherwise push a nonexistent tag and log "Done". - _log "Building and pushing ${TAG}..." - if ! docker build --no-cache -t "${TAG}" "${WORKDIR}"; then - _warn "build failed for ${TAG}; not pushing." - return 1 - fi - if ! docker push "${TAG}"; then - _warn "push failed for ${TAG}." - return 1 - fi - _log "=== Done: ${TAG} ===" -} - -# ============================================================================ -# Main -# ============================================================================ - -rc=0 -for VERSION in "${BUILD_VARIANTS[@]}"; do - if changelog_changed "${VERSION}"; then - if ! ( build_variant "${VERSION}" ); then - _warn "variant ${VERSION} failed; continuing." - rc=1 - fi - fi -done -exit "${rc}" diff --git a/.extras/image-builder/build-full-image.sh b/.extras/image-builder/build-full-image.sh deleted file mode 100755 index 24cbe1d..0000000 --- a/.extras/image-builder/build-full-image.sh +++ /dev/null @@ -1,233 +0,0 @@ -#!/bin/bash -# build-full-image.sh — sbo-full:{ver} FROM sbo-base:{ver}, all series. -# Adapted from forge slackware/docker-images. No root needed. -set -euo pipefail -HERE="$(cd "$(dirname "$0")" && pwd)" -source "${HERE}/config" -LOG_TAG=full -source "${HERE}/lib.sh" - -BASE_IMAGE="${REGISTRY}/sbo-base" -FULL_IMAGE="${REGISTRY}/sbo-full" -DIGEST_LABEL="sbo.full.base-digest" - -# ============================================================================ -# Argument parsing -# ============================================================================ - -FORCE=false -OPT_VERSION="" -while [[ $# -gt 0 ]]; do - case "$1" in - --version) OPT_VERSION="$2"; shift 2 ;; - --force) FORCE=true; shift ;; - *) _err "unknown argument: $1" ;; - esac -done -if [[ -n "${OPT_VERSION}" ]]; then - BUILD_VARIANTS=("${OPT_VERSION}") -else - BUILD_VARIANTS=("${VARIANTS[@]}") -fi - -# ============================================================================ -# Tag derivation -# ============================================================================ - -# derive_tags VERSION -# Sets globals BASE_TAG and FULL_TAG for the given variant. -derive_tags() { - local VERSION="$1" - BASE_TAG="${BASE_IMAGE}:${VERSION}" - FULL_TAG="${FULL_IMAGE}:${VERSION}" -} - -# ============================================================================ -# Rebuild detection -# -# After pulling the latest base image, its digest is compared with the -# digest stored in the DIGEST_LABEL on the current full image. If -# they differ the base image has been updated and the full must be -# rebuilt to incorporate slackpkg package updates shipped with the new base. -# ============================================================================ - -needs_rebuild() { - local BASE_TAG="$1" FULL_TAG="$2" - - if [[ "${FORCE}" == "true" ]]; then - _log " --force specified; rebuilding unconditionally." - return 0 - fi - - # Digest of the base image we just pulled - local BASE_DIGEST - BASE_DIGEST=$(docker inspect --format '{{index .RepoDigests 0}}' \ - "${BASE_TAG}" 2>/dev/null || echo "") - if [[ -z "${BASE_DIGEST}" ]]; then - _warn " Could not determine base image digest; rebuilding to be safe." - return 0 - fi - - # Digest recorded in the full image's label (empty if image doesn't exist) - local RECORDED_DIGEST - RECORDED_DIGEST=$(docker inspect \ - --format "{{index .Config.Labels \"${DIGEST_LABEL}\"}}" \ - "${FULL_TAG}" 2>/dev/null || echo "") - - if [[ "${BASE_DIGEST}" == "${RECORDED_DIGEST}" ]]; then - _log " Base image unchanged (${BASE_DIGEST:0:40}...); skipping." - return 1 - fi - - _log " Base image changed; rebuilding." - _log " Was : ${RECORDED_DIGEST:0:60}..." - _log " Now : ${BASE_DIGEST:0:60}..." - return 0 -} - -# ============================================================================ -# Build one variant -# ============================================================================ - -build_variant() { - local VERSION="$1" - local REPO_KEY="slackware64-${VERSION}" - require_mount "${VERSION}" # NFS must be up: we serve it over HTTP below - derive_tags "${VERSION}" - - _log "=== ${FULL_TAG} ===" - - # Pull the latest base image so digest comparison is against the current - # state of the registry, not a stale local cache. - _log " Pulling base image ${BASE_TAG}..." - docker pull "${BASE_TAG}" - - if ! needs_rebuild "${BASE_TAG}" "${FULL_TAG}"; then - return 0 - fi - - # Record the base image digest for the label - local BASE_DIGEST - BASE_DIGEST=$(docker inspect --format '{{index .RepoDigests 0}}' "${BASE_TAG}") - - local WORKDIR - WORKDIR="$(mktemp -d /tmp/slackware-full.XXXXXX)" - - # Serve the NFS mirror over HTTP on the docker bridge for the duration of - # the build, so slackpkg in the build container fetches packages from LAN - # instead of the internet. Killed (with WORKDIR cleanup) on any exit. - local MIRROR_ROOT; MIRROR_ROOT="$(strip_scheme "$(mirror_path "${MIRROR}" "${REPO_KEY}")")" - [[ -f "${MIRROR_ROOT}/PACKAGES.TXT" ]] \ - || _err "mirror root has no PACKAGES.TXT: ${MIRROR_ROOT}" - python3 -m http.server "${HTTP_MIRROR_PORT}" \ - --bind "${HTTP_MIRROR_HOST}" --directory "${MIRROR_ROOT}" \ - >/dev/null 2>&1 & - local HTTP_PID=$! - # shellcheck disable=SC2064 - trap "kill ${HTTP_PID} 2>/dev/null; rm -rf '${WORKDIR}'" EXIT - # Wait for the server to accept connections before building. - local i - for i in 1 2 3 4 5 6 7 8 9 10; do - curl -sf -o /dev/null "http://${HTTP_MIRROR_HOST}:${HTTP_MIRROR_PORT}/PACKAGES.TXT" && break - [[ $i -eq 10 ]] && _err "HTTP mirror did not come up on ${HTTP_MIRROR_HOST}:${HTTP_MIRROR_PORT}" - sleep 0.5 - done - - # Generate the Dockerfile for this variant - cat > "${WORKDIR}/Dockerfile" <<DOCKERFILE -FROM ${BASE_TAG} -LABEL maintainer="Eric Hameleers <alien@slackware.com>" - -# Import the Slackware GPG key against THIS mirror first: without it slackpkg -# cannot verify CHECKSUMS.md5 and silently installs almost nothing (only the -# packages already in base). 'yes YES |' answers the import prompt. -RUN yes YES | LC_ALL=C slackpkg -batch=on -default_answer=y update gpg - -# Update the slackpkg package list before installing so that we always -# get the versions current at build time, not the versions cached in the -# base image. -RUN LC_ALL=C slackpkg -batch=on -default_answer=y update - -# Download and install each individual package (thanks aclemons): -RUN sed -i 's/DOWNLOAD_ALL=on/DOWNLOAD_ALL=off/' /etc/slackpkg/slackpkg.conf - -# We don't care about a big firmware package: -RUN mv /etc/slackpkg/blacklist{,.keep} && \\ - echo kernel-firmware > /etc/slackpkg/blacklist - -# Install the full Slackware: -# All packages are installed in a single RUN layer to keep the image lean. -RUN export LC_ALL=C && \\ - for series in a ap d e f k kde l n t tcl x xap xfce y ; do \\ - slackpkg -delall=on -batch=on -default_answer=y install "\$series"/* ; done - -# And back to defaults: -RUN sed -i 's/DOWNLOAD_ALL=off/DOWNLOAD_ALL=on/' /etc/slackpkg/slackpkg.conf -RUN mv /etc/slackpkg/blacklist{.keep,} - - -# Delete package cache: -RUN rm -rf /var/cache/packages/* - -# Rebuild the linker cache after adding new libraries -RUN /sbin/ldconfig - -LABEL org.opencontainers.image.created="$(date -u +%Y-%m-%dT%H:%M:%SZ)" -LABEL org.opencontainers.image.title="Slackware Linux ${VERSION} full (x86_64)" -LABEL org.opencontainers.image.description="Slackware Linux ${VERSION} full image with all packages installed (x86_64)" -LABEL org.opencontainers.image.vendor="Slackware Linux" -LABEL org.opencontainers.image.licenses="GPL-2.0-or-later AND LGPL-2.1-or-later" -LABEL org.opencontainers.image.url="http://www.slackware.com" -LABEL org.opencontainers.image.source="https://forge.slackware.nl/slackware/docker-images" -LABEL org.opencontainers.image.revision="${VERSION}" -LABEL slackware.version="${VERSION}" -LABEL slackware.arch="x86_64" -LABEL ${DIGEST_LABEL}="${BASE_DIGEST}" -CMD ["/bin/bash"] -DOCKERFILE - - local BUILD_FLAGS=() - [[ "${FORCE}" == "true" ]] && BUILD_FLAGS+=(--no-cache) - - # Explicit exit checks: build_variant runs in an `if ! (...)` condition - # (see Main), which suppresses `set -e` inside this subshell, so a failed - # docker build would otherwise fall through to push a nonexistent tag and - # log "Done". Check each step and return non-zero on failure. - _log " Building ${FULL_TAG}..." - if ! docker build "${BUILD_FLAGS[@]}" \ - -t "${FULL_TAG}" \ - "${WORKDIR}"; then - _warn " build failed for ${FULL_TAG}; not pushing." - return 1 - fi - - _log " Pushing ${FULL_TAG}..." - if ! docker push "${FULL_TAG}"; then - _warn " push failed for ${FULL_TAG}." - return 1 - fi - - _log "=== Done: ${FULL_TAG} ===" -} - -# ============================================================================ -# Main -# ============================================================================ - -# A --force build passes --no-cache, so the stale build cache is dead weight: -# it still loads the containerd snapshotter's lease/lock bookkeeping and has -# raced the layer export ("failed to open writer: ref ... locked ... unavailable"). -# Prune it first to shrink that surface. Best-effort; never fail the build on it. -if [[ "${FORCE}" == "true" ]]; then - _log " --force: pruning build cache before rebuild" - docker builder prune -f >/dev/null 2>&1 || _warn " builder prune failed (ignored)" -fi - -rc=0 -for VERSION in "${BUILD_VARIANTS[@]}"; do - if ! ( build_variant "${VERSION}" ); then - _warn "variant ${VERSION} failed; continuing." - rc=1 - fi -done -exit "${rc}" diff --git a/.extras/image-builder/build-sbo-testbuild.sh b/.extras/image-builder/build-sbo-testbuild.sh deleted file mode 100755 index d04c426..0000000 --- a/.extras/image-builder/build-sbo-testbuild.sh +++ /dev/null @@ -1,131 +0,0 @@ -#!/bin/bash -# build-sbo-testbuild.sh — sbo-testbuild:{ver} FROM sbo-full:{ver}. -# Adds sbopkg + sbo-maintainer-tools from prebuilt .txz in PKGDIR. -# Rebuilds when the full image OR the .txz set changes. -set -euo pipefail -HERE="$(cd "$(dirname "$0")" && pwd)" -source "${HERE}/config" -LOG_TAG=testbuild -source "${HERE}/lib.sh" - -FULL_IMAGE="${REGISTRY}/sbo-full" -TB_IMAGE="${REGISTRY}/sbo-testbuild" -DIGEST_LABEL="sbo.testbuild.full-digest" -PKGS_LABEL="sbo.testbuild.pkgs-hash" - -FORCE=false -OPT_VERSION="" -while [[ $# -gt 0 ]]; do - case "$1" in - --version) OPT_VERSION="$2"; shift 2 ;; - --force) FORCE=true; shift ;; - *) _err "unknown argument: $1" ;; - esac -done -if [[ -n "${OPT_VERSION}" ]]; then - BUILD_VARIANTS=("${OPT_VERSION}") -else - BUILD_VARIANTS=("${VARIANTS[@]}") -fi - -# Verify the required packages (*.t?z) are present before doing anything. -require_pkgs() { - [[ -d "${PKGDIR}" ]] || _err "PKGDIR missing: ${PKGDIR}" - compgen -G "${PKGDIR}/sbopkg-*.t?z" >/dev/null \ - || _err "no sbopkg-*.t?z in ${PKGDIR}" - compgen -G "${PKGDIR}/sbo-maintainer-tools-*.t?z" >/dev/null \ - || _err "no sbo-maintainer-tools-*.t?z in ${PKGDIR}" -} - -# needs_rebuild FULL_TAG TB_TAG PKGS_HASH -> 0 if rebuild needed -needs_rebuild() { - local full_tag="$1" tb_tag="$2" pkgs_hash="$3" - [[ "${FORCE}" == "true" ]] && { _log " --force."; return 0; } - - local full_digest - full_digest=$(docker inspect --format '{{index .RepoDigests 0}}' \ - "${full_tag}" 2>/dev/null || echo "") - [[ -n "${full_digest}" ]] || { _warn " no full digest; rebuilding."; return 0; } - - local rec_digest rec_pkgs - rec_digest=$(docker inspect \ - --format "{{index .Config.Labels \"${DIGEST_LABEL}\"}}" \ - "${tb_tag}" 2>/dev/null || echo "") - rec_pkgs=$(docker inspect \ - --format "{{index .Config.Labels \"${PKGS_LABEL}\"}}" \ - "${tb_tag}" 2>/dev/null || echo "") - - if [[ "${full_digest}" == "${rec_digest}" && "${pkgs_hash}" == "${rec_pkgs}" ]]; then - _log " full image and .txz set unchanged; skipping." - return 1 - fi - _log " full image or .txz set changed; rebuilding." - return 0 -} - -build_variant() { - local VERSION="$1" - local FULL_TAG="${FULL_IMAGE}:${VERSION}" - local TB_TAG="${TB_IMAGE}:${VERSION}" - _log "=== ${TB_TAG} ===" - - _log " Pulling ${FULL_TAG}..." - docker pull "${FULL_TAG}" - - local PKGS_HASH; PKGS_HASH="$(txz_hash "${PKGDIR}")" - - if ! needs_rebuild "${FULL_TAG}" "${TB_TAG}" "${PKGS_HASH}"; then - return 0 - fi - - local FULL_DIGEST - FULL_DIGEST=$(docker inspect --format '{{index .RepoDigests 0}}' "${FULL_TAG}" 2>/dev/null || echo "") - - local WORKDIR; WORKDIR="$(mktemp -d /tmp/sbo-testbuild.XXXXXX)" - # shellcheck disable=SC2064 - trap "rm -rf '${WORKDIR}'" EXIT - mkdir -p "${WORKDIR}/pkgs" - cp "${PKGDIR}"/sbopkg-*.t?z "${PKGDIR}"/sbo-maintainer-tools-*.t?z "${WORKDIR}/pkgs/" - - cat > "${WORKDIR}/Dockerfile" <<DOCKERFILE -FROM ${FULL_TAG} -LABEL maintainer="danix <danix@danix.xyz>" -COPY pkgs/ /tmp/pkgs/ -RUN installpkg /tmp/pkgs/* && rm -rf /tmp/pkgs -RUN /sbin/ldconfig -LABEL org.opencontainers.image.created="$(date -u +%Y-%m-%dT%H:%M:%SZ)" -LABEL org.opencontainers.image.title="SBo test-build env ${VERSION} (x86_64)" -LABEL org.opencontainers.image.description="Slackware ${VERSION} full + sbopkg + sbo-maintainer-tools" -LABEL slackware.version="${VERSION}" -LABEL ${DIGEST_LABEL}="${FULL_DIGEST}" -LABEL ${PKGS_LABEL}="${PKGS_HASH}" -CMD ["/bin/bash"] -DOCKERFILE - - local BUILD_FLAGS=() - [[ "${FORCE}" == "true" ]] && BUILD_FLAGS+=(--no-cache) - # Explicit exit checks: build_variant runs in an `if ! (...)` condition - # (see Main), which suppresses `set -e` in this subshell, so a failed build - # would otherwise push a nonexistent tag and log "Done". - _log " Building ${TB_TAG}..." - if ! docker build "${BUILD_FLAGS[@]}" -t "${TB_TAG}" "${WORKDIR}"; then - _warn " build failed for ${TB_TAG}; not pushing." - return 1 - fi - _log " Pushing ${TB_TAG}..." - if ! docker push "${TB_TAG}"; then - _warn " push failed for ${TB_TAG}." - return 1 - fi - _log "=== Done: ${TB_TAG} ===" -} - -require_pkgs -rc=0 -for VERSION in "${BUILD_VARIANTS[@]}"; do - if ! ( build_variant "${VERSION}" ); then - _warn "variant ${VERSION} failed; continuing." - rc=1 - fi -done -exit "${rc}" diff --git a/.extras/image-builder/config b/.extras/image-builder/config deleted file mode 100644 index 8b52313..0000000 --- a/.extras/image-builder/config +++ /dev/null @@ -1,15 +0,0 @@ -# Shared configuration for the sbo-testbuild image builder. -# Sourced by bootstrap.sh, build-full-image.sh, build-sbo-testbuild.sh, tests. - -REGISTRY="docker.noland.dnx:5000" -MIRROR="file:///mnt/nas" -VARIANTS=(current 15.0) # x86_64 only for now -PKGDIR="/opt/sbo-testbuild/pkgs" # sbopkg + sbo-maintainer-tools .txz -HASH_DIR="/var/cache/sbo-testbuild" # ChangeLog hashes for rebuild gating - -# During the full-image build, packages are served to the build container off -# the local NFS mirror (MIRROR) over HTTP on the docker bridge, so slackpkg -# fetches from LAN instead of the internet. The bridge-gateway IP is what the -# build container sees as the host; the port is ephemeral to this host. -HTTP_MIRROR_HOST="172.17.0.1" # docker0 bridge gateway (host from container) -HTTP_MIRROR_PORT="8099" diff --git a/.extras/image-builder/lib.sh b/.extras/image-builder/lib.sh deleted file mode 100644 index da12fd0..0000000 --- a/.extras/image-builder/lib.sh +++ /dev/null @@ -1,72 +0,0 @@ -# Shared helpers for the image builder. Source, do not execute. -# Sourcing must have no side effects beyond defining functions. - -_log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] [${LOG_TAG:-image-builder}] $*"; } -_warn() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] [${LOG_TAG:-image-builder}] WARNING: $*" >&2; } -_err() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] [${LOG_TAG:-image-builder}] ERROR: $*" >&2; exit 1; } - -# mirror_path MIRROR RELPATH -# Join a MIRROR (file:///... or http(s)://...) with a relative path. -mirror_path() { - local mirror="$1" rel="$2" - printf '%s/%s' "${mirror%/}" "${rel#/}" -} - -# is_local_mirror MIRROR -> 0 if file:// or bare absolute path, else 1 -is_local_mirror() { - case "$1" in - file://*) return 0 ;; - http://*|https://*) return 1 ;; - /*) return 0 ;; - *) return 1 ;; - esac -} - -# strip_scheme URL -> filesystem path for a file:// or bare-local URL -strip_scheme() { - local u="$1" - case "$u" in - file://*) printf '%s' "${u#file://}" ;; - *) printf '%s' "$u" ;; - esac -} - -# fetch SRC DEST -# Copy SRC (a full mirror URL) to DEST. Local mirror -> cp, http -> curl. -# Returns non-zero on failure; caller decides whether that is fatal. -fetch() { - local src="$1" dest="$2" - if is_local_mirror "$src"; then - local path; path="$(strip_scheme "$src")" - [[ -f "$path" ]] || return 1 - cp "$path" "$dest" - else - curl -sSfL --connect-timeout 60 "$src" -o "$dest" - fi -} - -# txz_hash DIR -# Stable hash of the set of Slackware package files (*.t?z) in DIR (names + -# contents), order -# independent. Empty/absent dir -> empty string. -txz_hash() { - local dir="$1" - [[ -d "$dir" ]] || { printf ''; return 0; } - local files; files=$(find "$dir" -maxdepth 1 -name '*.t?z' -type f | sort) - [[ -n "$files" ]] || { printf ''; return 0; } - # Hash sorted "name sha256" lines so reordering does not change output. - while IFS= read -r f; do - printf '%s %s\n' "$(basename "$f")" "$(sha256sum "$f" | cut -d' ' -f1)" - done <<< "$files" | sha256sum | cut -d' ' -f1 -} - -# require_mount VERSION -# Assert the NAS tree for VERSION is mounted (file:// mirror only). -# x86_64 tree dir is slackware64-${VERSION}. -require_mount() { - local version="$1" - is_local_mirror "$MIRROR" || return 0 # http mirror: nothing to check - local base; base="$(strip_scheme "$MIRROR")" - local dir="${base}/slackware64-${version}" - [[ -d "$dir" ]] || _err "NAS tree not mounted: ${dir}" -} diff --git a/.extras/image-builder/test-image-builder.sh b/.extras/image-builder/test-image-builder.sh deleted file mode 100755 index 39d3f1c..0000000 --- a/.extras/image-builder/test-image-builder.sh +++ /dev/null @@ -1,68 +0,0 @@ -#!/bin/bash -# Assert-based self-check for lib.sh pure logic. No docker, no network. -# Run: bash .extras/image-builder/test-image-builder.sh -set -u -cd "$(dirname "$0")" -source ./lib.sh - -pass=0 fail=0 -check() { # DESC EXPECTED ACTUAL - if [[ "$2" == "$3" ]]; then pass=$((pass+1)); - else fail=$((fail+1)); echo "FAIL: $1"; echo " expected: [$2]"; echo " actual: [$3]"; fi -} -check_rc() { # DESC EXPECTED_RC ACTUAL_RC - if [[ "$2" == "$3" ]]; then pass=$((pass+1)); - else fail=$((fail+1)); echo "FAIL: $1 (rc expected $2 got $3)"; fi -} - -tmp="$(mktemp -d)" -trap 'rm -rf "$tmp"' EXIT - -# --- mirror_path --- -check "mirror_path trailing slash" \ - "file:///mnt/nas/slackware64-current/PACKAGES.TXT" \ - "$(mirror_path 'file:///mnt/nas/' 'slackware64-current/PACKAGES.TXT')" -check "mirror_path no trailing slash" \ - "file:///mnt/nas/x/y" "$(mirror_path 'file:///mnt/nas' '/x/y')" - -# --- is_local_mirror --- -is_local_mirror "file:///mnt/nas"; check_rc "is_local file://" 0 $? -is_local_mirror "/mnt/nas"; check_rc "is_local bare abs" 0 $? -is_local_mirror "https://x/y"; check_rc "is_local https" 1 $? - -# --- strip_scheme --- -check "strip_scheme file://" "/mnt/nas" "$(strip_scheme 'file:///mnt/nas')" -check "strip_scheme bare" "/mnt/nas" "$(strip_scheme '/mnt/nas')" - -# --- fetch: local file resolves --- -echo hello > "$tmp/src.txt" -fetch "file://$tmp/src.txt" "$tmp/dst.txt"; check_rc "fetch local ok" 0 $? -check "fetch local content" "hello" "$(cat "$tmp/dst.txt" 2>/dev/null)" - -# --- fetch: missing local file errors --- -fetch "file://$tmp/nope.txt" "$tmp/x.txt"; check_rc "fetch local missing" 1 $? - -# --- txz_hash: stable + content-sensitive --- -mkdir -p "$tmp/pkgs" -printf a > "$tmp/pkgs/sbopkg-1.txz" -printf b > "$tmp/pkgs/tools-1.txz" -h1="$(txz_hash "$tmp/pkgs")" -h2="$(txz_hash "$tmp/pkgs")" -check "txz_hash stable" "$h1" "$h2" -printf c > "$tmp/pkgs/tools-1.txz" # change content -h3="$(txz_hash "$tmp/pkgs")" -check_rc "txz_hash changes on content" 0 "$([[ "$h1" != "$h3" ]] && echo 0 || echo 1)" -check "txz_hash empty dir" "" "$(txz_hash "$tmp/empty-nope")" - -# --- require_mount: missing mount errors (subshell to catch _err exit) --- -MIRROR="file://$tmp/no-such-root" -( require_mount current ) 2>/dev/null; check_rc "require_mount missing" 1 $? -mkdir -p "$tmp/mnt/slackware64-current" -MIRROR="file://$tmp/mnt" -( require_mount current ) 2>/dev/null; check_rc "require_mount present" 0 $? -MIRROR="https://example/x" -( require_mount current ) 2>/dev/null; check_rc "require_mount http skips" 0 $? - -echo "----" -echo "PASS: $pass FAIL: $fail" -[[ "$fail" -eq 0 ]] diff --git a/.extras/overrides.example b/.extras/overrides.example deleted file mode 100644 index e9d89e9..0000000 --- a/.extras/overrides.example +++ /dev/null @@ -1,16 +0,0 @@ -# current-vs-stable dep overrides for test-build. -# Copy to ~/.config/sbo-testbuild/overrides and edit. Applied only when -# targeting -current (15.0 is the SBo baseline). One rule per line. -# -# drop: <prog> dep is a 15.0-only SBo package whose payload the -# -current base already ships; skipped at resolution -# (not built, not flagged UNMET). e.g. rust-opt: -# 15.0 needs the SBo rust-opt, current has rust in base. -# rename: <old> -> <new> dep was renamed on -current -# fetch: <prog> dep was removed from the -current tree; pull via sbopkg -# -# Start light: add rules only when a real package needs them. - -# drop: rust-opt -# rename: python3-foo -> foo -# fetch: somelib diff --git a/.extras/test-build b/.extras/test-build deleted file mode 100755 index f245aed..0000000 --- a/.extras/test-build +++ /dev/null @@ -1,864 +0,0 @@ -#!/bin/bash -# -# test-build - verify an already-published SBo package still builds on a target -# Slackware version inside a throwaway docker container. Resolves + builds its -# SBo deps from the local tree, caches built deps per image digest, reports -# per-package status and lints the target. -# -# Dependency-resolution, cache, and summary logic are adapted from sbo-batch-test -# (github: danixland). The overlay chroot is replaced by a docker container: the -# container IS the disposable environment, so no overlayfs. -# -# No em dashes in prose by author convention. - -# ============================================================================= -# CONFIG (do not edit here; real values live in the external config file) -# ============================================================================= -SBO_TREE_CURRENT="" -SBO_TREE_STABLE="" -IMAGE_CURRENT="" -IMAGE_STABLE="" -LOG_ROOT="/var/log/sbo-test-build" -PKG_CACHE="" - -TB_CONFIG="${TB_CONFIG:-$HOME/.config/sbo-testbuild/config}" -if [[ -f "$TB_CONFIG" ]]; then - # shellcheck disable=SC1090 - source "$TB_CONFIG" -fi - -TB_OVERRIDES="${TB_OVERRIDES:-$HOME/.config/sbo-testbuild/overrides}" - -# ============================================================================= -set -uo pipefail -# Not -e: a package build failing is a handled outcome, not a script crash. - -# ---- flags / globals -------------------------------------------------------- -USE_COLOR=1 # --no-color or non-TTY disables -DRY_RUN=0 # --dry-run: resolve + print order, do not build -ASSUME_YES=0 # --yes: skip the confirm prompt (still prints the order) -USE_CACHE=1 # --no-cache disables the dep cache for one run -KEEP_TARGET=0 # --keep: copy the built target package out to KEEP_DIR -VERSION_ID="current" # "current" | "15.0"; set by --stable -TARGET_ARG="" - -ACTIVE_TREE="" # selected SBo tree (by version) -ACTIVE_IMAGE="" # selected image tag (by version) -RUN_DIR="" # timestamped log dir for this run -DEPS_DIR="" # per-run host dir of built dep .txz, mounted into the container -BUILD_OUT="" # per-run host dir where built packages are copied out - -# Status tracking. Keyed by "category/prog". Parallel assoc arrays. -declare -A ST_STATUS=() -declare -A ST_REASON=() -declare -A ST_TIME=() -declare -A ST_README=() - -usage() { - cat <<'EOF' -test-build - verify an SBo package builds on a target Slackware in docker - -USAGE: - test-build [OPTIONS] <program-name> - -OPTIONS: - -h, --help This text. - --stable Target Slackware 15.0 (image + tree). Default is -current. - --dry-run Resolve, apply overrides, print the build order, do not build. - --yes Skip the confirm prompt (the order is still printed first). - --no-cache Rebuild all deps this run, ignore/refresh the cache. - --keep Copy the built target package out to a kept/ dir (path is - printed), so it can be installed on the host. The build is - otherwise throwaway. - --no-color Disable ANSI color (auto-disabled when stdout is not a TTY). -EOF -} - -parse_args() { - while [[ $# -gt 0 ]]; do - case "$1" in - -h|--help) usage; exit 0 ;; - --stable|15.0) VERSION_ID="15.0"; shift ;; - --dry-run) DRY_RUN=1; shift ;; - --yes) ASSUME_YES=1; shift ;; - --no-cache) USE_CACHE=0; shift ;; - --keep) KEEP_TARGET=1; shift ;; - --no-color) USE_COLOR=0; shift ;; - -*) echo "Unknown option: $1" >&2; usage >&2; exit 2 ;; - *) - if [[ -n "$TARGET_ARG" ]]; then - echo "Only one target accepted (got '$TARGET_ARG' and '$1')." >&2 - exit 2 - fi - TARGET_ARG="$1"; shift ;; - esac - done - if [[ -z "$TARGET_ARG" ]]; then - echo "No target given." >&2; usage >&2; exit 2 - fi -} - -init_color() { - if [[ $USE_COLOR -eq 1 && -t 1 ]]; then - C_RED=$'\e[31m'; C_GRN=$'\e[32m'; C_YEL=$'\e[33m'; C_RST=$'\e[0m' - else - C_RED=""; C_GRN=""; C_YEL=""; C_RST="" - fi -} - -# Map VERSION_ID to the active tree + image. No I/O, unit-testable. -select_version_paths() { - if [[ "$VERSION_ID" == "15.0" ]]; then - ACTIVE_TREE="$SBO_TREE_STABLE"; ACTIVE_IMAGE="$IMAGE_STABLE" - else - ACTIVE_TREE="$SBO_TREE_CURRENT"; ACTIVE_IMAGE="$IMAGE_CURRENT" - fi -} - -# require_config: the external config must exist and set the version's paths. -require_config() { - if [[ ! -f "$TB_CONFIG" ]]; then - cat >&2 <<EOF -No config file: $TB_CONFIG -Copy the example and edit it: - - mkdir -p "\$(dirname "$TB_CONFIG")" - cp .extras/test-build-config.example "$TB_CONFIG" - \${EDITOR:-vi} "$TB_CONFIG" -EOF - exit 1 - fi - select_version_paths - if [[ -z "$ACTIVE_TREE" || -z "$ACTIVE_IMAGE" ]]; then - echo "Config $TB_CONFIG is missing the tree/image for version '$VERSION_ID'." >&2 - exit 1 - fi - if [[ ! -d "$ACTIVE_TREE" ]]; then - echo "SBo tree for '$VERSION_ID' does not exist: $ACTIVE_TREE" >&2 - exit 1 - fi -} - -# ============================================================================= -# SBo tree lookup -# ============================================================================= -# In this tool there is one active tree per run, but the resolver reads an array -# named SBO_TREE_ROOTS so the ported logic and its self-check match sbo-batch-test. -# main() sets SBO_TREE_ROOTS=("$ACTIVE_TREE") after require_config. -declare -a SBO_TREE_ROOTS=() - -find_slackbuild_dir() { - local prog="$1" root d - for root in "${SBO_TREE_ROOTS[@]}"; do - [[ -d "$root" ]] || continue - for d in "$root"/*/"$prog"; do - if [[ -d "$d" && -f "$d/$prog.info" ]]; then - echo "$d"; return 0 - fi - done - done - return 1 -} - -category_of() { basename "$(dirname "$1")"; } -pkg_key() { echo "$(category_of "$1")/$(basename "$1")"; } - -# Resolve the TARGET to test from CWD or a path (repo-agnostic). The target is -# the SlackBuild you are editing, wherever it lives; its deps come from the -# configured SBo tree (see find_slackbuild_dir), NOT the other way around. -# - a path (absolute, or containing '/', or '.'/'./x'): use that dir directly -# - a bare name: ./<name>/ under CWD, else CWD itself if it IS <name>/ -# Prints the resolved absolute dir on success; returns 1 (with a message) if the -# dir has no matching <name>.info. -resolve_target_dir() { - local arg="$1" dir prog - if [[ "$arg" == /* || "$arg" == .* || "$arg" == */* ]]; then - dir="${arg%/}"; prog="$(basename "$dir")" - elif [[ -f "./$arg/$arg.info" ]]; then - dir="./$arg"; prog="$arg" - elif [[ "$(basename "$PWD")" == "$arg" && -f "./$arg.info" ]]; then - dir="."; prog="$arg" - else - echo "Target '$arg' not found: no ./$arg/$arg.info, and CWD is not $arg/." >&2 - return 1 - fi - if [[ ! -f "$dir/$prog.info" ]]; then - echo "Target dir '$dir' has no $prog.info (expected a SlackBuild package dir)." >&2 - return 1 - fi - ( cd "$dir" && pwd ) # emit absolute path -} - -read_requires() { - local info="$1" - # shellcheck disable=SC1090 - ( set +u; source "$info"; echo "${REQUIRES:-}" ) -} - -version_of() { - local dir="$1" - local info="$dir/$(basename "$dir").info" - [[ -f "$info" ]] || return - local v; v="$(grep -m1 '^VERSION=' "$info" | cut -d'"' -f2)" - echo "$v" -} - -# ============================================================================= -# DEPENDENCY RESOLUTION (topo sort + cycle detection, LOCAL tree only) -# ============================================================================= -declare -a RESOLVED_ORDER=() -declare -A UNMET=() -declare -a CYCLES=() -declare -A HAS_README=() -declare -A _vstate=() -declare -A FETCH_DEPS=() # prog -> 1: resolve via sbopkg in-container - -# Is a prog already present in the container base? Kept as a callback so the -# pure topo logic stays testable. -# -# Populated lazily from the active image's package db on first call: each entry -# in /var/log/packages is named <name>-<ver>-<arch>-<build>, so strip the last -# three dash-fields to get the bare package name and match the SBo token -# against that. TB_BASE_PKGS can be pre-seeded by tests to bypass docker. -declare -A TB_BASE_PKGS=() -TB_BASE_LOADED="${TB_BASE_LOADED:-0}" - -load_base_pkgs() { - [[ "$TB_BASE_LOADED" == "1" ]] && return 0 - TB_BASE_LOADED=1 - [[ -n "$ACTIVE_IMAGE" ]] || return 0 - local entry name - while IFS= read -r entry; do - [[ -n "$entry" ]] || continue - # strip trailing -<ver>-<arch>-<build> - name="${entry%-*-*-*}" - TB_BASE_PKGS["$name"]=1 - done < <(docker run --rm "$ACTIVE_IMAGE" \ - /bin/sh -c 'ls /var/log/packages' 2>/dev/null) -} - -installed_in_base() { - load_base_pkgs - [[ -n "${TB_BASE_PKGS[$1]:-}" ]] -} - -_resolve_visit() { - local dir="$1" parent="$2" - local key; key="$(basename "$dir")" - - if [[ "${_vstate[$dir]:-}" == "1" ]]; then return 0; fi - if [[ "${_vstate[$dir]:-}" == "0" ]]; then - CYCLES+=("cycle involving $key (pulled in via $parent)") - return 1 - fi - _vstate["$dir"]=0 - - local info="$dir/$(basename "$dir").info" - local req tok depdir rc=0 - req="$(read_requires "$info")" - for tok in $req; do - if [[ "$tok" == "%README%" ]]; then - HAS_README["$dir"]=1 - continue - fi - tok="$(apply_rename "$tok")" - # drop: satisfied by the -current base (e.g. a 15.0-only dep like rust-opt - # whose payload ships in current's full-install image). Skip it entirely: - # no recurse, no UNMET, no order entry. Inert on 15.0 (overrides are). - if [[ "${OV_DROP[$tok]:-}" == "1" ]]; then - continue - fi - if [[ "${OV_FETCH[$tok]:-}" == "1" ]]; then - FETCH_DEPS["$tok"]=1 - continue - fi - if depdir="$(find_slackbuild_dir "$tok")"; then - _resolve_visit "$depdir" "$key" || rc=1 - elif installed_in_base "$tok"; then - : - else - UNMET["$tok"]="needed by $key" - rc=1 - fi - done - - _vstate["$dir"]=1 - RESOLVED_ORDER+=("$dir") - return $rc -} - -resolve_target() { - local dir="$1" - RESOLVED_ORDER=() - CYCLES=() - UNMET=() - FETCH_DEPS=() - _vstate=() - _resolve_visit "$dir" "(top)" -} - -# ============================================================================= -# current-vs-stable overrides. Parsed from $TB_OVERRIDES. Applied only when -# targeting -current (15.0 is the SBo baseline, no deltas). -# ============================================================================= -declare -A OV_DROP=() # prog -> 1 -declare -A OV_RENAME=() # old -> new -declare -A OV_FETCH=() # prog -> 1 - -load_overrides() { - OV_DROP=(); OV_RENAME=(); OV_FETCH=() - # 15.0 is the baseline: no overrides. - [[ "$VERSION_ID" == "15.0" ]] && return - [[ -f "$TB_OVERRIDES" ]] || return - local line kind rest - while IFS= read -r line; do - line="${line%%#*}" # strip comments - line="${line#"${line%%[![:space:]]*}"}" # ltrim - [[ -z "$line" ]] && continue - kind="${line%%:*}"; rest="${line#*:}" - kind="${kind//[[:space:]]/}" - rest="${rest#"${rest%%[![:space:]]*}"}" # ltrim value - case "$kind" in - drop) OV_DROP["${rest//[[:space:]]/}"]=1 ;; - fetch) OV_FETCH["${rest//[[:space:]]/}"]=1 ;; - rename) - # rest is "old -> new" - local old new - old="${rest%%->*}"; new="${rest##*->}" - old="${old//[[:space:]]/}"; new="${new//[[:space:]]/}" - [[ -n "$old" && -n "$new" ]] && OV_RENAME["$old"]="$new" ;; - *) echo "WARN: unknown override rule: $line" >&2 ;; - esac - done < "$TB_OVERRIDES" -} - -# Map a dep token through rename rules (identity if no rule). -apply_rename() { - local tok="$1" - echo "${OV_RENAME[$tok]:-$tok}" -} - -# Remove dropped packages from RESOLVED_ORDER in place. -apply_overrides_to_order() { - local d prog keep=() - for d in "${RESOLVED_ORDER[@]}"; do - prog="$(basename "$d")" - [[ "${OV_DROP[$prog]:-}" == "1" ]] && continue - keep+=("$d") - done - # Assign without "${keep[@]:-}": under set -u that fallback yields a - # one-element array holding an empty string when keep is empty (all deps - # dropped), which the build loop would then iterate over as d="". Guard it. - if [[ ${#keep[@]} -eq 0 ]]; then - RESOLVED_ORDER=() - else - RESOLVED_ORDER=("${keep[@]}") - fi -} - -# ============================================================================= -# Dependency cache. Layout: $CACHE_ROOT/<cat>/<prog>/<prog>-<ver>-...txz where -# CACHE_ROOT = $PKG_CACHE/<image-digest> (set per run by resolve_cache_root). -# Key is prog+version. --no-cache (USE_CACHE=0) or empty PKG_CACHE disables. -# ============================================================================= -CACHE_ROOT="" # set by resolve_cache_root once the image digest is known - -# True when the cache is usable this run. -_cache_on() { [[ $USE_CACHE -eq 1 && -n "$PKG_CACHE" && -n "$CACHE_ROOT" ]]; } - -_cache_ver_of() { - local prog="$1" base="$2" - base="${base#"$prog"-}" - echo "${base%%-*}" -} - -# cache_decision <cat> <prog> <version> -> cached | bump:OLD:NEW | new -cache_decision() { - local cat="$1" prog="$2" version="$3" - _cache_on || { echo new; return; } - local dir="$CACHE_ROOT/$cat/$prog" - local f newest="" - for f in "$dir/$prog"-*.t?z; do - [[ -e "$f" ]] || continue - [[ -z "$newest" || "$f" -nt "$newest" ]] && newest="$f" - done - [[ -z "$newest" ]] && { echo new; return; } - local have; have="$(_cache_ver_of "$prog" "$(basename "$newest")")" - if [[ "$have" == "$version" ]]; then echo cached; else echo "bump:$have:$version"; fi -} - -cache_path() { - local cat="$1" prog="$2" version="$3" - _cache_on || return - local dir="$CACHE_ROOT/$cat/$prog" - local f newest="" - for f in "$dir/$prog"-*.t?z; do - [[ -e "$f" ]] || continue - [[ -z "$newest" || "$f" -nt "$newest" ]] && newest="$f" - done - [[ -z "$newest" ]] && return - [[ "$(_cache_ver_of "$prog" "$(basename "$newest")")" == "$version" ]] && echo "$newest" -} - -cache_store() { - local cat="$1" prog="$2" src="$3" - _cache_on || return - local dir="$CACHE_ROOT/$cat/$prog" - mkdir -p "$dir" - rm -f "$dir"/*.t?z - cp -a "$src" "$dir/" -} - -cache_label() { - local dir="$1" is_target="$2" - local cat prog ver dec - cat="$(category_of "$dir")"; prog="$(basename "$dir")"; ver="$(version_of "$dir")" - dec="$(cache_decision "$cat" "$prog" "$ver")" - local label - case "$dec" in - cached) label="cached ($ver)" ;; - bump:*) label="rebuild: ${dec#bump:}"; label="${label/:/ -> }" ;; - *) label="build (new)" ;; - esac - if [[ "$is_target" == "1" ]]; then - case "$dec" in - cached) label="build (cached $ver, rebuilt as target)" ;; - esac - echo "target, $label" - else - echo "$label" - fi -} - -# Compute CACHE_ROOT from the image's digest, namespaced by variant. Falls back -# to the tag if the digest cannot be read (still isolates per image reference). -# The dir is "<variant>-<digest>", so an image rebuild changes the digest and -# self-invalidates the cache (deps built against the old base are never reused). -# -# Prune stale digests of THIS variant on the way in: a base update leaves the -# old <variant>-<olddigest> dir orphaned, so drop it. Scoped to the variant so a -# --current run does not wipe the 15.0 cache (both live caches survive, and -# testing both trees back-to-back still reuses built deps). -resolve_cache_root() { - [[ -z "$PKG_CACHE" ]] && { CACHE_ROOT=""; return; } - local digest - digest="$(docker image inspect --format '{{index .Id}}' "$ACTIVE_IMAGE" 2>/dev/null)" - [[ -z "$digest" ]] && digest="tag-${ACTIVE_IMAGE//[^a-zA-Z0-9._-]/_}" - digest="${digest//[^a-zA-Z0-9._-]/_}" - local name="${VERSION_ID}-${digest}" - CACHE_ROOT="$PKG_CACHE/$name" - mkdir -p "$CACHE_ROOT" - - # drop superseded caches for this variant only - local d - for d in "$PKG_CACHE/${VERSION_ID}-"*; do - [[ -d "$d" ]] || continue - [[ "$(basename "$d")" == "$name" ]] && continue - rm -rf "$d" - done -} - -# Does SlackBuild dir $1 directly require any prog in the dead list (nameref $2)? -# Direct-requires check only; transitive blocking works because run_target -# iterates in topo order, propagating a failure one hop per package. -depends_on_failed() { - local dir="$1"; local -n failed="$2" - local info="$dir/$(basename "$dir").info" - local req tok f - req="$(read_requires "$info")" - for tok in $req; do - [[ "$tok" == "%README%" ]] && continue - for f in "${failed[@]:-}"; do - [[ "$tok" == "$f" ]] && return 0 - done - done - return 1 -} - -# build_one <slackbuild-dir> <is_target 0|1> [container-name] -# Runs the build in a throwaway container. Sets ST_STATUS/ST_REASON/ST_TIME. -# Successful builds copy their package to a host workdir; deps are cached and -# the target is linted. Returns 0 on SUCCESS/CACHED, 1 otherwise. -build_one() { - local dir="$1" is_target="${2:-0}" - local prog cat key - prog="$(basename "$dir")"; cat="$(category_of "$dir")"; key="$cat/$prog" - local logf="$RUN_DIR/${cat}_${prog}.log" - local start; start=$(date +%s) - local version; version="$(version_of "$dir")" - - [[ "${HAS_README[$dir]:-}" == "1" ]] && ST_README["$key"]=1 - - # Dep with a version-matching cached package: installpkg it into the shared - # dep-package dir; no build. The target never takes this path. - if [[ "$is_target" != "1" ]]; then - local cached; cached="$(cache_path "$cat" "$prog" "$version")" - if [[ -n "$cached" ]]; then - cp -a "$cached" "$DEPS_DIR/" - { - echo "===== test-build: $prog (from cache) =====" - echo "cached package: $(basename "$cached")" - } >> "$logf" - ST_TIME["$key"]=$(( $(date +%s) - start )) - ST_STATUS["$key"]="CACHED" - return 0 - fi - fi - - # Build in a container. Mounts: - # $dir -> /sbo/pkg (ro, the SlackBuild) - # $DEPS_DIR -> /sbo/deps (rw, already-built dep .txz to installpkg first) - # $BUILD_OUT -> /sbo/out (rw, where the built package is copied out) - # The in-container script installs any deps present, then builds the target, - # writes a status token to /sbo/out/$prog.status, and copies the package out. - local statf="$BUILD_OUT/$prog.status" - rm -f "$statf" - - # fetch deps (removed from -current tree): let the container's sbopkg build - # them first. FETCH_DEPS is the set collected during resolution. - local fetch_list="" - local fp - for fp in "${!FETCH_DEPS[@]}"; do fetch_list+="$fp "; done - - # -i is required: without it docker does not attach stdin, so `bash -s` reads - # nothing and the heredoc script is silently discarded (exit 0, empty log). - docker run --rm -i \ - -v "$dir":/sbo/pkg:ro \ - -v "$DEPS_DIR":/sbo/deps \ - -v "$BUILD_OUT":/sbo/out \ - -e PROG="$prog" \ - -e FETCH_LIST="$fetch_list" \ - -e IS_TARGET="$is_target" \ - "$ACTIVE_IMAGE" /bin/bash -s >>"$logf" 2>&1 <<'CONTAINER_EOF' -set -uo pipefail -prog="$PROG" -statf="/sbo/out/$prog.status" - -# 0. install already-built dependency packages (order guaranteed by the host). -for d in /sbo/deps/*.t?z; do - [[ -e "$d" ]] || continue - installpkg --terse "$d" || { echo "INSTALL-FAILED (dep $d)"; echo INSTALL-FAILED > "$statf"; exit 1; } -done - -# 0b. fetch-from-SBo deps via sbopkg (removed from the -current tree). -for f in $FETCH_LIST; do - echo "sbopkg-building fetch dep: $f" - sbopkg -B -i "$f" || { echo "BUILD-FAILED (fetch dep $f)"; echo BUILD-FAILED > "$statf"; exit 1; } -done - -# copy the SlackBuild out of the read-only mount so it can write there. -cp -a /sbo/pkg /sbo/build -cd /sbo/build || { echo BUILD-FAILED > "$statf"; exit 1; } -. ./"$prog".info - -export OUTPUT=/sbo/out -mkdir -p "$OUTPUT" - -echo "===== test-build: $prog =====" -echo "PRGNAM=${PRGNAM:-$prog} VERSION=${VERSION:-?} BUILD=${BUILD:-?} TAG=${TAG:-?}" -echo "uname -m: $(uname -m) OUTPUT=$OUTPUT" -echo "REQUIRES=${REQUIRES:-}" -echo "=================================" - -if [ "$(uname -m)" = "x86_64" ] && [ -n "${DOWNLOAD_x86_64:-}" ] && [ "${DOWNLOAD_x86_64}" != "UNSUPPORTED" ] && [ "${DOWNLOAD_x86_64}" != "UNTESTED" ]; then - DL="$DOWNLOAD_x86_64"; MD="$MD5SUM_x86_64" -else - DL="$DOWNLOAD"; MD="$MD5SUM" -fi - -for u in $DL; do - wget -c --tries=3 "$u" || { echo DOWNLOAD-FAILED > "$statf"; exit 1; } -done - -set -- $MD -for u in $DL; do - f="$(basename "$u")" - want="$1"; shift - got="$(md5sum "$f" | cut -d' ' -f1)" - if [ "$got" != "$want" ]; then - echo "MD5 mismatch on $f: want $want got $got" - echo MD5-MISMATCH > "$statf"; exit 1 - fi -done - -# Source /etc/profile.d so dep-provided env is live (google-go-lang sets GOROOT -# + PATH to its go here, rust-opt sets cargo, etc.). This heredoc runs in a -# non-login shell, which does NOT read profile.d, so a bare `go build` would -# otherwise pick the system gccgo instead of the installed google-go-lang. -set +u # profile.d scripts routinely reference unset vars -for pf in /etc/profile.d/*.sh; do - [ -r "$pf" ] && . "$pf" -done -set -u - -chmod +x ./"$prog".SlackBuild -if ! ./"$prog".SlackBuild; then - echo BUILD-FAILED > "$statf"; exit 1 -fi - -pkg="$(ls -t "$OUTPUT"/"$prog"-*.t?z 2>/dev/null | head -n1)" -if [ -z "$pkg" ]; then - echo "No package produced in $OUTPUT" - echo BUILD-FAILED > "$statf"; exit 1 -fi -if ! installpkg --terse "$pkg"; then - echo INSTALL-FAILED > "$statf"; exit 1 -fi -echo "===== installed files: $(basename "$pkg") =====" -pkgname="$(basename "$pkg")"; pkgname="${pkgname%.t?z}" -cat "/var/log/packages/$pkgname" 2>/dev/null || echo "(package db entry not found)" -echo "=================================" - -# lint the target here in the container: sbopkglint is baked into the image and -# runs as root, so it needs no host sudo. Fail-soft: findings never fail the build. -if [ "${IS_TARGET:-0}" = "1" ] && command -v sbopkglint >/dev/null 2>&1; then - echo "===== sbopkglint: $(basename "$pkg") =====" - if sbopkglint "$pkg"; then - echo "LINT-CLEAN" - else - echo "LINT-FINDINGS" - fi - echo "=================================" -fi -echo SUCCESS > "$statf" -CONTAINER_EOF - - local status="BUILD-FAILED" - [[ -f "$statf" ]] && status="$(cat "$statf")" - ST_TIME["$key"]=$(( $(date +%s) - start )) - ST_STATUS["$key"]="$status" - - if [[ "$status" == "SUCCESS" ]]; then - # locate the built package copied to the host workdir - local built newest="" - for built in "$BUILD_OUT/${prog}"-*.t?z; do - [[ -e "$built" ]] || continue - [[ -z "$newest" || "$built" -nt "$newest" ]] && newest="$built" - done - if [[ -n "$newest" ]]; then - if [[ "$is_target" != "1" ]]; then - cache_store "$cat" "$prog" "$newest" - # make the dep available to later builds in this run - cp -a "$newest" "$DEPS_DIR/" - else - # lint ran in-container (see IS_TARGET block); surface its verdict from - # the log so the host summary can show clean/findings. - if grep -q '^LINT-FINDINGS$' "$logf" 2>/dev/null; then - echo " lint: ${C_RED}findings${C_RST} (see $(basename "$logf"))" - elif grep -q '^LINT-CLEAN$' "$logf" 2>/dev/null; then - echo " lint: ${C_GRN}clean${C_RST}" - fi - # --keep: copy the built target package to a durable dir (sibling of the - # logs tree, not inside a throwaway run dir) so it can be installed on - # the host, e.g. to regenerate post-install artifacts. - if [[ $KEEP_TARGET -eq 1 ]]; then - local keepdir; keepdir="$(dirname "$LOG_ROOT")/kept" - mkdir -p "$keepdir" - cp -a "$newest" "$keepdir/" - echo " kept: $keepdir/$(basename "$newest")" - fi - fi - fi - return 0 - fi - ST_REASON["$key"]="see $(basename "$logf")" - return 1 -} - -# confirm_order: print the resolved order (overrides marked) and ask to proceed. -# --yes skips the prompt but the order is still printed. --dry-run never reaches -# here. Returns 0 to proceed, 1 to abort. -confirm_order() { - local target_dir="$1" - echo " build order (${VERSION_ID}):" - local d - for d in "${RESOLVED_ORDER[@]}"; do - local it=0; [[ "$d" == "$target_dir" ]] && it=1 - local rdm=""; [[ "${HAS_README[$d]:-}" == 1 ]] && rdm=" [%README%]" - printf " %-30s %s%s\n" "$(pkg_key "$d")" "$(cache_label "$d" "$it")" "$rdm" - echo "$(pkg_key "$d")" >> "$RUN_DIR/build-order.txt" - done - # note fetch deps (built via sbopkg in-container, not in the order list) - local fp - for fp in "${!FETCH_DEPS[@]}"; do - printf " %-30s %s\n" "$fp" "fetch (sbopkg in container)" - done - [[ $ASSUME_YES -eq 1 ]] && return 0 - local reply - read -rp " Proceed? [Y/n] " reply - [[ -z "$reply" || "$reply" =~ ^[Yy]$ ]] -} - -# run_target <target-slackbuild-dir> -run_target() { - local target_dir="$1" - local tkey; tkey="$(pkg_key "$target_dir")" - - echo - echo "=== Target: $tkey (${VERSION_ID}) ===" - resolve_target "$target_dir" - apply_overrides_to_order - - # Hard resolution failures: report and stop, do not build. - if [[ ${#CYCLES[@]} -gt 0 || ${#UNMET[@]} -gt 0 ]]; then - local why="" - if [[ ${#UNMET[@]} -gt 0 ]]; then - local u - for u in "${!UNMET[@]}"; do why+="unmet:$u(${UNMET[$u]}) "; done - fi - [[ ${#CYCLES[@]} -gt 0 ]] && why+="${CYCLES[*]}" - ST_STATUS["$tkey"]="UNMET-DEP" - ST_REASON["$tkey"]="$why" - echo " resolution failed: $why" - echo " add an override rule ($TB_OVERRIDES) and rerun, or fix the tree." >&2 - return - fi - - if [[ $DRY_RUN -eq 1 ]]; then - echo " build order (dry-run):" - local d - for d in "${RESOLVED_ORDER[@]}"; do - local it=0; [[ "$d" == "$target_dir" ]] && it=1 - local rdm=""; [[ "${HAS_README[$d]:-}" == 1 ]] && rdm=" [%README%]" - printf " %-30s %s%s\n" "$(pkg_key "$d")" "$(cache_label "$d" "$it")" "$rdm" - echo "$(pkg_key "$d")" >> "$RUN_DIR/build-order.txt" - done - local fp - for fp in "${!FETCH_DEPS[@]}"; do - printf " %-30s %s\n" "$fp" "fetch (sbopkg in container)" - done - return - fi - - if ! confirm_order "$target_dir"; then - echo " aborted." - ST_STATUS["$tkey"]="ABORTED" - return - fi - - # Per-run docker workdirs (host side, discarded after the run). - DEPS_DIR="$RUN_DIR/deps"; BUILD_OUT="$RUN_DIR/out" - mkdir -p "$DEPS_DIR" "$BUILD_OUT" - - local d failed_progs=() - for d in "${RESOLVED_ORDER[@]}"; do - local key; key="$(pkg_key "$d")" - local prog; prog="$(basename "$d")" - - if depends_on_failed "$d" failed_progs; then - ST_STATUS["$key"]="BLOCKED-BY-DEP" - ST_REASON["$key"]="blocked by failed dep" - [[ "${HAS_README[$d]:-}" == "1" ]] && ST_README["$key"]=1 - echo " $key: BLOCKED-BY-DEP" - failed_progs+=("$prog") - continue - fi - - local it=0; [[ "$d" == "$target_dir" ]] && it=1 - echo " building $key ..." - if build_one "$d" "$it"; then - echo " $key: ${ST_STATUS[$key]} (${ST_TIME[$key]}s)" - else - echo " $key: ${ST_STATUS[$key]} (${ST_TIME[$key]}s)" - failed_progs+=("$prog") - fi - done -} - -# ============================================================================= -# SUMMARY (ported) -# ============================================================================= -print_summary() { - local total=$SECONDS - local succ=0 fail=0 blocked=0 cached=0 - local summary="$RUN_DIR/summary.log" - - { - echo "test-build run summary" - echo "target: $TARGET_ARG version: $VERSION_ID" - echo - } > "$summary" - - echo - echo "================ SUMMARY ================" - local key - for key in "${!ST_STATUS[@]}"; do - local st="${ST_STATUS[$key]}" rsn="${ST_REASON[$key]:-}" t="${ST_TIME[$key]:-0}" - local rd=""; [[ "${ST_README[$key]:-}" == "1" ]] && rd=" [%README%]" - local col="$C_YEL" - case "$st" in - SUCCESS) col="$C_GRN"; ((succ++)) ;; - CACHED) col="$C_GRN"; ((cached++)) ;; - BLOCKED-BY-DEP|UNMET-DEP|ABORTED) col="$C_YEL"; ((blocked++)) ;; - *) col="$C_RED"; ((fail++)) ;; - esac - printf "%s%-30s %-16s%s %s%s (%ss)\n" "$col" "$key" "$st" "$C_RST" "$rsn" "$rd" "$t" - printf "%-30s %-16s %s%s (%ss)\n" "$key" "$st" "$rsn" "$rd" "$t" >> "$summary" - done - echo "----------------------------------------" - printf "%s%d succeeded%s, %s%d failed%s, %s%d blocked%s, %s%d cached%s, total %ss\n" \ - "$C_GRN" "$succ" "$C_RST" "$C_RED" "$fail" "$C_RST" "$C_YEL" "$blocked" "$C_RST" \ - "$C_GRN" "$cached" "$C_RST" "$total" - echo "logs: $RUN_DIR" - if [[ $fail -eq 0 && $blocked -eq 0 ]]; then - echo "${C_GRN}All green.${C_RST} Safe to build the SBo submission tarball on the host." - fi - { - echo - echo "$succ succeeded, $fail failed, $blocked blocked, $cached cached, total ${total}s" - echo "logs: $RUN_DIR" - } >> "$summary" -} - - -# Ensure the selected image is available locally. It is built by a separate job -# (the image-builder) and pushed to the LAN registry; this script only consumes -# it. If it is not already local, pull it once (ACTIVE_IMAGE is a fully-qualified -# registry ref). A stale local tag is not refreshed here. -# ponytail: pull-if-missing only. If the registry's :current is rebuilt, the -# local copy goes stale silently. Add a --pull force-flag if that bites. -require_image() { - docker image inspect "$ACTIVE_IMAGE" >/dev/null 2>&1 && return 0 - echo "Image not present locally, pulling: $ACTIVE_IMAGE" >&2 - docker pull "$ACTIVE_IMAGE" >&2 && return 0 - cat >&2 <<EOF -Image not found and pull failed: $ACTIVE_IMAGE -It is produced by the image-builder job (full Slackware $VERSION_ID + -sbo-maintainer-tools + sbopkg) and pushed to the LAN registry. Check the -registry is reachable and the tag exists, then rerun. This script does not -build images. -EOF - exit 1 -} - -main() { - parse_args "$@" - init_color - require_config - SBO_TREE_ROOTS=("$ACTIVE_TREE") - load_overrides - - # image + cache namespace (skip the image check on a pure dry-run so the order - # can be inspected without the image present). - if [[ $DRY_RUN -eq 0 ]]; then - command -v docker >/dev/null 2>&1 || { echo "docker not found in PATH." >&2; exit 1; } - require_image - fi - resolve_cache_root - - RUN_DIR="$LOG_ROOT/$(date +%Y-%m-%d_%H-%M-%S)" - mkdir -p "$RUN_DIR" - : > "$RUN_DIR/build-order.txt" - - # The target is the local SlackBuild under test (CWD or a path); its deps are - # resolved from the configured SBo tree during _resolve_visit. - local tdir - if ! tdir="$(resolve_target_dir "$TARGET_ARG")"; then - exit 1 - fi - - run_target "$tdir" - print_summary -} - -main "$@" diff --git a/.extras/test-build-config.example b/.extras/test-build-config.example deleted file mode 100644 index a07b2c1..0000000 --- a/.extras/test-build-config.example +++ /dev/null @@ -1,18 +0,0 @@ -# test-build config. Copy to ~/.config/sbo-testbuild/config and edit. -# Override the path with the TB_CONFIG environment variable. - -# Local SBo trees, one per target Slackware version. -SBO_TREE_CURRENT="/home/danix/SBo/current" -SBO_TREE_STABLE="/home/danix/SBo/15.0" - -# Ready image tags (built by a separate job, consumed here by tag). -IMAGE_CURRENT="sbo-testbuild:current" -IMAGE_STABLE="sbo-testbuild:15.0" - -# Where per-run logs land. -LOG_ROOT="/home/danix/.cache/sbo-test-build" - -# Dependency package cache (host dir). Empty disables caching. -# Namespaced internally by image digest, so it self-invalidates on image update -# and keeps current vs 15.0 separate. -PKG_CACHE="/home/danix/.cache/sbo-test-build/pkgcache" diff --git a/.extras/test-logic.sh b/.extras/test-logic.sh deleted file mode 100644 index 7dc7387..0000000 --- a/.extras/test-logic.sh +++ /dev/null @@ -1,215 +0,0 @@ -#!/bin/bash -# -# Logic self-check for .extras/test-build. Covers the pure, host-side parts: -# dependency resolution, override application, unknown-dep -> UNMET, cache -# decisions, and BLOCKED-BY-DEP propagation. No docker. -# -# Run: bash .extras/test-logic.sh -# -set -uo pipefail - -SCRIPT="$(dirname "$0")/test-build" -T=$(mktemp -d) -BASE_DB=$(mktemp -d); mkdir -p "$BASE_DB" - -cleanup() { rm -rf "$T" "$BASE_DB"; } -trap cleanup EXIT - -# Source the script without running main(). Sourcing re-runs the CONFIG block, -# so set test vars AFTER the source. -LIB=$(mktemp) -sed '/^main "\$@"$/d' "$SCRIPT" > "$LIB" -# shellcheck disable=SC1090 -source "$LIB" 2>/dev/null -rm -f "$LIB" - -pass=0; fail=0 -ok() { echo " ok: $1"; ((pass++)); return 0; } -bad() { echo " FAIL: $1"; ((fail++)); return 0; } - -# Placeholder assertion so this file runs before any logic exists. -ok "script sources without executing main" - -# --- version selection ------------------------------------------------------ -SBO_TREE_CURRENT="/trees/current" -SBO_TREE_STABLE="/trees/stable" -IMAGE_CURRENT="sbo-testbuild:current" -IMAGE_STABLE="sbo-testbuild:15.0" - -VERSION_ID="current"; select_version_paths -[[ "$ACTIVE_TREE" == "/trees/current" ]] && ok "current -> current tree" || bad "current tree wrong: [$ACTIVE_TREE]" -[[ "$ACTIVE_IMAGE" == "sbo-testbuild:current" ]] && ok "current -> current image" || bad "current image wrong: [$ACTIVE_IMAGE]" - -VERSION_ID="15.0"; select_version_paths -[[ "$ACTIVE_TREE" == "/trees/stable" ]] && ok "15.0 -> stable tree" || bad "stable tree wrong: [$ACTIVE_TREE]" -[[ "$ACTIVE_IMAGE" == "sbo-testbuild:15.0" ]] && ok "15.0 -> stable image" || bad "stable image wrong: [$ACTIVE_IMAGE]" - -# --- resolution ------------------------------------------------------------- -# Fake SBo tree under one category. mk <prog> "<REQUIRES>". -mk() { mkdir -p "$T/cat/$1"; echo "REQUIRES=\"$2\"" > "$T/cat/$1/$1.info"; } -mk c "" -mk b "c" -mk a "b %README%" -mk d "nonexistentpkg" -mk e "f" -mk f "e" -mk g "b" - -SBO_TREE_ROOTS=("$T") # resolver reads this global (set by select in real runs) - -resolve_target "$T/cat/a" -order=""; for x in "${RESOLVED_ORDER[@]}"; do order+="$(basename "$x") "; done -order="${order% }" -[[ "$order" == "c b a" ]] && ok "topo order c b a" || bad "topo order, got: [$order]" -[[ "${HAS_README[$T/cat/a]:-}" == "1" ]] && ok "%README% recorded" || bad "%README% not recorded" -[[ ${#UNMET[@]} -eq 0 ]] && ok "no false unmet" || bad "unexpected unmet" - -resolve_target "$T/cat/d" -[[ ${#UNMET[@]} -eq 1 ]] && ok "unmet-dep caught" || bad "unmet-dep missed" - -# --- resolve_target_dir (target from CWD/path, NOT the SBo tree) ------------- -# Local package dir, separate from the fake SBo tree above. -LOCALREPO=$(mktemp -d); mkdir -p "$LOCALREPO/mypkg" -echo 'REQUIRES=""' > "$LOCALREPO/mypkg/mypkg.info" -# 1. absolute path -got="$(resolve_target_dir "$LOCALREPO/mypkg" 2>/dev/null)" -[[ "$got" == "$LOCALREPO/mypkg" ]] && ok "target: absolute path" || bad "abs path got [$got]" -# 2. bare name resolved under CWD (./mypkg/) -got="$(cd "$LOCALREPO" && resolve_target_dir "mypkg" 2>/dev/null)" -[[ "$got" == "$LOCALREPO/mypkg" ]] && ok "target: bare name in CWD" || bad "bare-in-cwd got [$got]" -# 3. CWD itself is the package -got="$(cd "$LOCALREPO/mypkg" && resolve_target_dir "mypkg" 2>/dev/null)" -[[ "$got" == "$LOCALREPO/mypkg" ]] && ok "target: CWD is the package" || bad "cwd-is-pkg got [$got]" -# 4. missing -> failure -resolve_target_dir "nosuchpkg" >/dev/null 2>&1 && bad "missing target should fail" || ok "target: missing fails" -rm -rf "$LOCALREPO" - -resolve_target "$T/cat/e" -[[ ${#CYCLES[@]} -ge 1 ]] && ok "cycle caught" || bad "cycle missed" - -# --- installed_in_base ------------------------------------------------------ -# Pre-seed the base pkg set (TB_BASE_LOADED=1 bypasses docker). Names are the -# bare package name; installed_in_base must match a REQUIRES token against it. -TB_BASE_LOADED=1 -TB_BASE_PKGS=([python3]=1 [libfoo]=1) -installed_in_base python3 && ok "base pkg matched" || bad "base pkg not matched" -installed_in_base notthere && bad "absent pkg wrongly matched" || ok "absent pkg not matched" - -# a dep present in the base resolves as met, not UNMET -mk h "python3" -resolve_target "$T/cat/h" -[[ ${#UNMET[@]} -eq 0 ]] && ok "base-provided dep not unmet" || bad "base dep flagged unmet" -TB_BASE_LOADED=0; TB_BASE_PKGS=() - -# --- overrides -------------------------------------------------------------- -VERSION_ID="current" # reset: left at 15.0 by the version-selection checks above -OV=$(mktemp) -cat > "$OV" <<'EOF' -# comment ignored -drop: dropme -rename: oldname -> newname -fetch: fetchme -EOF -TB_OVERRIDES="$OV" -load_overrides - -[[ "${OV_DROP[dropme]:-}" == "1" ]] && ok "override drop parsed" || bad "drop not parsed" -[[ "${OV_RENAME[oldname]:-}" == "newname" ]] && ok "override rename parsed" || bad "rename not parsed" -[[ "${OV_FETCH[fetchme]:-}" == "1" ]] && ok "override fetch parsed" || bad "fetch not parsed" - -# rename maps a token -[[ "$(apply_rename oldname)" == "newname" ]] && ok "apply_rename maps" || bad "apply_rename wrong: [$(apply_rename oldname)]" -[[ "$(apply_rename untouched)" == "untouched" ]] && ok "apply_rename passthrough" || bad "apply_rename mangled untouched" - -# drop satisfies an external (no-dir) dep during resolution: not UNMET, absent -# from the order. mk k requires a dropme that has no dir in the tree. -OV_DROP=([extdrop]=1) -mk k "extdrop" -resolve_target "$T/cat/k" -[[ ${#UNMET[@]} -eq 0 ]] && ok "drop satisfies external dep (not unmet)" || bad "dropped external dep still unmet" -seen=""; for x in "${RESOLVED_ORDER[@]}"; do seen+="$(basename "$x") "; done -[[ "$seen" != *extdrop* ]] && ok "dropped dep absent from order" || bad "dropped dep in order: [$seen]" -load_overrides # restore OV_DROP[dropme] for the order-filter test below - -# drop filters the order (build a fake order of dirs; dropme is removed) -mk dropme "" -mk keepme "" -RESOLVED_ORDER=("$T/cat/dropme" "$T/cat/keepme") -apply_overrides_to_order -kept=""; for x in "${RESOLVED_ORDER[@]}"; do kept+="$(basename "$x") "; done -kept="${kept% }" -[[ "$kept" == "keepme" ]] && ok "drop removes from order" || bad "drop failed, order=[$kept]" - -# all entries dropped -> RESOLVED_ORDER must be truly empty (not one "" element) -OV_DROP=([dropme]=1 [keepme]=1) -RESOLVED_ORDER=("$T/cat/dropme" "$T/cat/keepme") -apply_overrides_to_order -[[ ${#RESOLVED_ORDER[@]} -eq 0 ]] && ok "all-dropped -> empty order" || bad "all-dropped left ${#RESOLVED_ORDER[@]} elem(s): [${RESOLVED_ORDER[*]}]" -OV_DROP=() -rm -f "$OV" - -# overrides only apply on current; on 15.0 they are a no-op -VERSION_ID="15.0" -TB_OVERRIDES="$OV" # file gone; must not error, must clear maps -load_overrides -[[ ${#OV_DROP[@]} -eq 0 ]] && ok "overrides inert on 15.0" || bad "overrides applied on 15.0" -VERSION_ID="current" - -# --- cache ------------------------------------------------------------------ -PKG_CACHE=$(mktemp -d) -CACHE_ROOT="$PKG_CACHE/sha256-deadbeef" # simulate a resolved digest namespace -mkc() { mkdir -p "$CACHE_ROOT/$1/$2"; : > "$CACHE_ROOT/$1/$2/$3"; } - -mkc net libfoo "libfoo-1.1-x86_64-1_danix.txz" -[[ "$(cache_decision net libfoo 1.1)" == "cached" ]] && ok "cache hit on version match" || bad "cache_decision got [$(cache_decision net libfoo 1.1)]" -[[ "$(cache_decision net libfoo 1.2)" == "bump:1.1:1.2" ]] && ok "cache bump reported" || bad "cache bump got [$(cache_decision net libfoo 1.2)]" -[[ "$(cache_decision net libbar 1.0)" == "new" ]] && ok "cache new for absent" || bad "cache new got [$(cache_decision net libbar 1.0)]" - -hit="$(cache_path net libfoo 1.1)" -[[ "$hit" == "$CACHE_ROOT/net/libfoo/libfoo-1.1-x86_64-1_danix.txz" ]] && ok "cache_path returns hit" || bad "cache_path got [$hit]" -[[ -z "$(cache_path net libfoo 9.9)" ]] && ok "cache_path empty on miss" || bad "cache_path not empty on miss" - -srctmp=$(mktemp -d); : > "$srctmp/libfoo-1.2-x86_64-1_danix.txz" -cache_store net libfoo "$srctmp/libfoo-1.2-x86_64-1_danix.txz" -count=$(find "$CACHE_ROOT/net/libfoo" -name '*.t?z' | wc -l) -[[ "$count" -eq 1 ]] && ok "cache_store evicts to one file" || bad "cache_store left $count files" -[[ -e "$CACHE_ROOT/net/libfoo/libfoo-1.2-x86_64-1_danix.txz" ]] && ok "cache_store stored new file" || bad "cache_store did not store" -rm -rf "$srctmp" - -# disabled cache -> new -CACHE_ROOT_SAVE="$CACHE_ROOT"; USE_CACHE=0 -[[ "$(cache_decision net libfoo 1.2)" == "new" ]] && ok "--no-cache disables (new)" || bad "disabled cache got [$(cache_decision net libfoo 1.2)]" -USE_CACHE=1; CACHE_ROOT="$CACHE_ROOT_SAVE" - -# resolve_cache_root prunes stale digests of the SAME variant, keeps the other. -# No docker here, so the digest falls back to the tag-derived name; seed sibling -# dirs and confirm only the active-variant orphan is removed. -PRUNE_CACHE=$(mktemp -d) -PKG_CACHE="$PRUNE_CACHE"; ACTIVE_IMAGE="sbo-testbuild:current"; VERSION_ID="current" -mkdir -p "$PRUNE_CACHE/current-oldstale" "$PRUNE_CACHE/15.0-keepme" -resolve_cache_root -[[ -d "$CACHE_ROOT" ]] && ok "cache root created" || bad "cache root missing" -[[ ! -e "$PRUNE_CACHE/current-oldstale" ]] && ok "prune drops stale same-variant digest" || bad "stale current digest survived" -[[ -d "$PRUNE_CACHE/15.0-keepme" ]] && ok "prune keeps other-variant cache" || bad "15.0 cache wrongly pruned" -rm -rf "$PRUNE_CACHE" -VERSION_ID="current"; CACHE_ROOT="$CACHE_ROOT_SAVE" - -# --- BLOCKED-BY-DEP (depends_on_failed) ------------------------------------- -dead=(b) -if depends_on_failed "$T/cat/g" dead; then ok "g blocked when b dead"; else bad "g should block on b"; fi -if depends_on_failed "$T/cat/a" dead; then ok "a blocked when b dead (direct)"; else bad "a should block on b"; fi - -dead=(c) -if depends_on_failed "$T/cat/a" dead; then bad "a wrongly blocked on c"; else ok "a not directly blocked by c"; fi -if depends_on_failed "$T/cat/b" dead; then ok "b blocked when c dead"; else bad "b should block on c"; fi - -dead=() -if depends_on_failed "$T/cat/a" dead; then bad "a blocked with empty dead"; else ok "no block when nothing dead"; fi - -dead=("%README%") -if depends_on_failed "$T/cat/a" dead; then bad "%README% treated as dep"; else ok "%README% not treated as dep"; fi - -echo -echo "$pass passed, $fail failed" -[[ $fail -eq 0 ]] || exit 1 -echo "ALL LOGIC CHECKS PASS" @@ -11,7 +11,7 @@ on SBo: 1. Track upstream version changes (via nvchecker). 2. Reflect a new upstream version in the package's `.SlackBuild` / `.info`. -3. Test-build in a throwaway docker image (see below). +3. Test-build with the `test-build` tool (see below). 4. Generate the SBo submission tarball to upload. New-package development happens in **other** repos, not here: @@ -48,9 +48,8 @@ Version tracking is handled by a single file: ### `.extras/` holds all non-package repo files Anything that is not a SlackBuild package must live under `.extras/`, never at -the repo root. Exception: a top-level `docs/` for design docs and specs is fine -in this repo (the subtree-leak concern that bans it applies to the development -repos, not here). Existing specs live under `.extras/docs/specs/`. +the repo root. Currently: `nvchecker.toml` (version tracking), `hooks/` (git +hooks), `assets/`. --- @@ -141,25 +140,29 @@ compiled binaries. No stripping needed. --- -## Test-building in docker +## Test-building Because these packages are already published on SBo, the build here is a -**verification step** on a version bump, not development. It runs in a -throwaway Slackware container, so it is hermetic and disposable: unlike a real -`slackrepo build` on the buildsystem VM, **Claude may run it directly**. +**verification step** on a version bump, not development. It runs in a throwaway +Slackware container via the **`test-build`** tool, which lives in its own repo, +`../sbo-dockerbuild` (installed to `~/bin/test-build`). Being hermetic and +disposable, **Claude may run it directly** (unlike a real `slackrepo build` on +the buildsystem VM). -Images (Slackware official, pull fresh each time — they are throwaway): +Workflow for a bump: edit the `.SlackBuild`/`.info`, then verify on **both** +trees (these are all SBo-published packages, so both are mandatory): -- `-current`: `registry.slackware.nl/slackware/slackware-builder:current` -- `15.0` (stable): `registry.slackware.nl/slackware/slackware-builder:15.0` - -Mount the package dir into a `--rm` container and run its SlackBuild. Anything -built is discarded with the container; nothing lands in the working tree. +```bash +test-build <pkg> # -current +test-build --stable <pkg> # 15.0 +``` -No wrapper script exists yet — write the `docker run` invocation inline, or add -one under `.extras/` if this gets repetitive. The build's only outputs that -matter are the exit status and lint result; the package tarball it produces is -throwaway. +`test-build` resolves the target from the local dir (CWD or a path) and its deps +from the configured SBo tree, builds in a `--rm` container, and lints with +`sbopkglint`. Use `--keep` to copy the built `.txz` out (e.g. a binary repack +like `metasploit-framework-bin` that needs a host install to regenerate +post-install artifacts). See `../sbo-dockerbuild` for the full tool and its +image-builder chain. The one artifact worth keeping from a clean bump is the **SBo submission tarball** (`SBo/<pkg>.tar.gz`), produced by the `post-commit` hook — that is @@ -281,8 +284,8 @@ Claude authors/edits the sources (`.SlackBuild`, `.info`, `README`, `slack-desc`, `doinst.sh`, nvchecker stanzas) and runs the non-building checks (`sbolint`, `sbodl`, `sbofixinfo`). -**The one exception in this repo** is the throwaway-docker test-build (see -[Test-building in docker](#test-building-in-docker)): it is hermetic and +**The one exception in this repo** is the throwaway-docker test-build via the +`test-build` tool (see [Test-building](#test-building)): it is hermetic and disposable, so Claude may run it to verify a bump, including `sbopkglint` on the throwaway package it produces. |