diff options
Diffstat (limited to 'docs/superpowers/plans/2026-06-24-package-cache.md')
| -rw-r--r-- | docs/superpowers/plans/2026-06-24-package-cache.md | 996 |
1 files changed, 996 insertions, 0 deletions
diff --git a/docs/superpowers/plans/2026-06-24-package-cache.md b/docs/superpowers/plans/2026-06-24-package-cache.md new file mode 100644 index 0000000..736dd08 --- /dev/null +++ b/docs/superpowers/plans/2026-06-24-package-cache.md @@ -0,0 +1,996 @@ +# Package Cache Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a persistent on-disk package cache so unchanged dependencies are installed from cache instead of rebuilt, while the named target always builds fresh. + +**Architecture:** A new `PKG_CACHE` config var points at a local directory laid out as `<cat>/<prog>/<prog>-<ver>-<arch>-<build>_<tag>.txz`, one `.txz` per prog. Three pure helper functions (`cache_decision`, `cache_path`, `cache_store`) hold all cache logic and are unit-tested in `test-logic.sh`. `build_one` gains an `is_target` argument: the target builds fresh and refreshes the cache; deps install from cache on a version match, else build and cache. `update_base` wipes the cache when it patches the base. + +**Tech Stack:** Plain bash (single file `sbo-batch-test`), `installpkg`/`upgradepkg` (pkgtools), overlayfs chroot. Self-check in `test-logic.sh` (no VM). + +Spec: `docs/superpowers/specs/2026-06-24-package-cache-design.md`. + +Note: work directly on master (user preference for this project). Commits are GPG-signed (`-S`). No em dash in any message or comment. + +--- + +### Task 1: Add PKG_CACHE config var and example + +**Files:** +- Modify: `sbo-batch-test` (CONFIG defaults block, around lines 41-50) +- Modify: `config.example` + +- [ ] **Step 1: Add the default in the script CONFIG block** + +In `sbo-batch-test`, the CONFIG block sets empty/default vars before sourcing the +external config. Add `PKG_CACHE` after `LOG_ROOT`. Find: + +```sh +LOG_ROOT="/var/log/sbo-batch-test" +VERSION="15.0" +``` + +Replace with: + +```sh +LOG_ROOT="/var/log/sbo-batch-test" + +# Persistent package cache (local disk). Built dependency packages are stored +# here and reused across runs while their version is unchanged. Empty disables +# the cache (every package builds fresh). Wiped automatically when the base is +# patched. Layout mirrors the SBo tree: <cat>/<prog>/<prog>-<ver>-...txz +PKG_CACHE="" + +VERSION="15.0" +``` + +- [ ] **Step 2: Document it in config.example** + +In `config.example`, after the `LOG_ROOT` block, add: + +```sh +# Persistent package cache. Local disk, survives across runs. Built dependency +# packages are stored here and reused when their version is unchanged. Wiped +# automatically when the base is patched. Leave empty to disable caching. +PKG_CACHE="/var/cache/sbo-batch-test" +``` + +- [ ] **Step 3: Verify syntax** + +Run: `bash -n sbo-batch-test` +Expected: no output, exit 0. + +- [ ] **Step 4: Verify self-check still passes** + +Run: `bash test-logic.sh` +Expected: ends with `ALL LOGIC CHECKS PASS`. + +- [ ] **Step 5: Commit** + +```bash +git add sbo-batch-test config.example +git commit -S -m "Add PKG_CACHE config var (empty = disabled) + +Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>" +``` + +--- + +### Task 2: cache_decision (pure) + tests + +**Files:** +- Modify: `sbo-batch-test` (new function, place it just before the "SBo tree lookup" section, i.e. before `find_slackbuild_dir`, around line 302) +- Test: `test-logic.sh` + +`cache_decision <cat> <prog> <version>` globs the prog's cache dir and echoes one +token: `cached`, `bump:<oldver>:<newver>`, or `new`. + +- [ ] **Step 1: Write the failing tests** + +In `test-logic.sh`, after the BLOCKED-BY-DEP block and before `# --- result`, +add: + +```bash +# --- package cache ---------------------------------------------------------- +PKG_CACHE=$(mktemp -d) +mkc() { mkdir -p "$PKG_CACHE/$1/$2"; : > "$PKG_CACHE/$1/$2/$3"; } + +# exact version match -> cached +mkc net libfoo "libfoo-1.1-x86_64-1_danix.txz" +[[ "$(cache_decision net libfoo 1.1)" == "cached" ]] && ok "cache hit on version match" || bad "cache_decision should be cached, got [$(cache_decision net libfoo 1.1)]" + +# different version cached -> bump:old:new +[[ "$(cache_decision net libfoo 1.2)" == "bump:1.1:1.2" ]] && ok "cache bump reported" || bad "cache_decision bump wrong, got [$(cache_decision net libfoo 1.2)]" + +# nothing cached -> new +[[ "$(cache_decision net libbar 1.0)" == "new" ]] && ok "cache new for absent prog" || bad "cache_decision should be new, got [$(cache_decision net libbar 1.0)]" + +# empty PKG_CACHE disables -> new +PKG_CACHE_SAVE="$PKG_CACHE"; PKG_CACHE="" +[[ "$(cache_decision net libfoo 1.1)" == "new" ]] && ok "empty PKG_CACHE disables (new)" || bad "disabled cache should be new, got [$(cache_decision net libfoo 1.1)]" +PKG_CACHE="$PKG_CACHE_SAVE" +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `bash test-logic.sh` +Expected: FAIL lines for the cache checks (function not defined -> empty output mismatches), script exits 1. + +- [ ] **Step 3: Implement cache_decision** + +In `sbo-batch-test`, just before the `# === SBo tree lookup` section header (before +`find_slackbuild_dir`), add: + +```sh +# ============================================================================= +# Package cache. Persistent, on-disk store of built packages so unchanged deps +# are installed from cache instead of rebuilt. Layout mirrors the SBo tree: +# $PKG_CACHE/<cat>/<prog>/<prog>-<ver>-<arch>-<build>_<tag>.txz +# One .txz per prog dir (the latest tested build). Key is prog+version; build +# number, arch, and tag do not affect a hit. Empty PKG_CACHE disables caching. +# ============================================================================= + +# Echo the version field of a cached package filename. <file> is a basename like +# prog-1.2-x86_64-1_danix.txz; prog may itself contain dashes, so strip the known +# prog prefix first, then take the field up to the next dash. +_cache_ver_of() { + local prog="$1" base="$2" + base="${base#"$prog"-}" # drop "prog-" + echo "${base%%-*}" # version is up to the next dash +} + +# cache_decision <cat> <prog> <version> -> echoes: cached | bump:OLD:NEW | new +cache_decision() { + local cat="$1" prog="$2" version="$3" + [[ -z "$PKG_CACHE" ]] && { echo new; return; } + local dir="$PKG_CACHE/$cat/$prog" + local f newest="" + for f in "$dir/$prog"-*.t?z; do + [[ -e "$f" ]] || continue + [[ -z "$newest" || "$f" -nt "$newest" ]] && newest="$f" + done + [[ -z "$newest" ]] && { echo new; return; } + local have; have="$(_cache_ver_of "$prog" "$(basename "$newest")")" + if [[ "$have" == "$version" ]]; then + echo cached + else + echo "bump:$have:$version" + fi +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `bash test-logic.sh` +Expected: the four new `ok:` lines, ends with `ALL LOGIC CHECKS PASS`. + +- [ ] **Step 5: Commit** + +```bash +git add sbo-batch-test test-logic.sh +git commit -S -m "Add cache_decision + tests + +Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>" +``` + +--- + +### Task 3: cache_path + cache_store (pure) + tests + +**Files:** +- Modify: `sbo-batch-test` (append both functions right after `cache_decision`) +- Test: `test-logic.sh` + +`cache_path <cat> <prog> <version>` echoes the cached file for a matching version +(empty if none). `cache_store <cat> <prog> <src>` clears the prog dir and copies +`<src>` in (no-op if PKG_CACHE empty). + +- [ ] **Step 1: Write the failing tests** + +In `test-logic.sh`, append to the package-cache block (after the cache_decision +tests, before restoring/!cleanup): + +```bash +# cache_path returns the hit file for a matching version +hit="$(cache_path net libfoo 1.1)" +[[ "$hit" == "$PKG_CACHE/net/libfoo/libfoo-1.1-x86_64-1_danix.txz" ]] && ok "cache_path returns hit" || bad "cache_path wrong, got [$hit]" + +# cache_path empty when version does not match +[[ -z "$(cache_path net libfoo 9.9)" ]] && ok "cache_path empty on miss" || bad "cache_path should be empty on miss" + +# cache_store evicts: prog dir holds exactly the new file +srctmp=$(mktemp -d); : > "$srctmp/libfoo-1.2-x86_64-1_danix.txz" +cache_store net libfoo "$srctmp/libfoo-1.2-x86_64-1_danix.txz" +count=$(find "$PKG_CACHE/net/libfoo" -name '*.t?z' | wc -l) +[[ "$count" -eq 1 ]] && ok "cache_store evicts to one file" || bad "cache_store left $count files" +[[ -e "$PKG_CACHE/net/libfoo/libfoo-1.2-x86_64-1_danix.txz" ]] && ok "cache_store stored new file" || bad "cache_store did not store new file" +rm -rf "$srctmp" +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `bash test-logic.sh` +Expected: FAIL lines for cache_path/cache_store (functions not defined), exit 1. + +- [ ] **Step 3: Implement cache_path and cache_store** + +In `sbo-batch-test`, immediately after the `cache_decision` function, add: + +```sh +# cache_path <cat> <prog> <version> -> echoes the cached .txz path for that +# version, or nothing. Used to installpkg a cached dep. +cache_path() { + local cat="$1" prog="$2" version="$3" + [[ -z "$PKG_CACHE" ]] && return + local dir="$PKG_CACHE/$cat/$prog" + local f newest="" + for f in "$dir/$prog"-*.t?z; do + [[ -e "$f" ]] || continue + [[ -z "$newest" || "$f" -nt "$newest" ]] && newest="$f" + done + [[ -z "$newest" ]] && return + [[ "$(_cache_ver_of "$prog" "$(basename "$newest")")" == "$version" ]] && echo "$newest" +} + +# cache_store <cat> <prog> <src-txz> -> clear the prog dir, copy src in. +cache_store() { + local cat="$1" prog="$2" src="$3" + [[ -z "$PKG_CACHE" ]] && return + local dir="$PKG_CACHE/$cat/$prog" + mkdir -p "$dir" + rm -f "$dir"/*.t?z + cp -a "$src" "$dir/" +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `bash test-logic.sh` +Expected: the four new `ok:` lines, ends with `ALL LOGIC CHECKS PASS`. + +- [ ] **Step 5: Commit** + +```bash +git add sbo-batch-test test-logic.sh +git commit -S -m "Add cache_path + cache_store + tests + +Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>" +``` + +--- + +### Task 4: Read each package version for cache lookups (helper) + +**Files:** +- Modify: `sbo-batch-test` (new helper near the other SBo-tree lookups, after `read_requires`) + +The build order printing and `build_one` need a package's VERSION from its +`.info`. Add a small reader so it is not duplicated. + +- [ ] **Step 1: Implement version_of** + +In `sbo-batch-test`, after the `read_requires` function (SBo tree lookup +section), add: + +```sh +# Echo the VERSION field from a SlackBuild dir's .info (empty if unreadable). +# Used as the cache key's version component. +version_of() { + local dir="$1" info="$dir/$(basename "$dir").info" + [[ -f "$info" ]] || return + # .info lines look like: VERSION="1.2.3" + local v; v="$(grep -m1 '^VERSION=' "$info" | cut -d'"' -f2)" + echo "$v" +} +``` + +- [ ] **Step 2: Add a test** + +In `test-logic.sh`, the `mk` helper writes only REQUIRES. Extend the package-cache +block with a version_of check using a purpose-built .info: + +```bash +# version_of reads VERSION from .info +mkdir -p "$T/cat/verpkg" +printf 'PRGNAM="verpkg"\nVERSION="3.4.5"\nREQUIRES=""\n' > "$T/cat/verpkg/verpkg.info" +[[ "$(version_of "$T/cat/verpkg")" == "3.4.5" ]] && ok "version_of reads VERSION" || bad "version_of wrong, got [$(version_of "$T/cat/verpkg")]" +``` + +- [ ] **Step 3: Run tests to verify pass** + +Run: `bash test-logic.sh` +Expected: new `ok: version_of reads VERSION`, ends with `ALL LOGIC CHECKS PASS`. + +- [ ] **Step 4: Commit** + +```bash +git add sbo-batch-test test-logic.sh +git commit -S -m "Add version_of .info reader + test + +Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>" +``` + +--- + +### Task 5: Annotate build order with cache outcome (real run + dry-run) + +**Files:** +- Modify: `sbo-batch-test` (`run_target`, dry-run branch around lines 597-605 and real-run order print around lines 607-608) + +Show per-package why it will build or reuse, in both dry-run and real run. The +target is the package whose dir equals the run's target dir (`target_dir` in +`run_target`). + +- [ ] **Step 1: Add a shared annotator helper** + +In `sbo-batch-test`, just before `run_target`, add: + +```sh +# Echo a human label for a package's cache outcome, given its SlackBuild dir and +# whether it is the run's target. Mirrors cache_decision but renders text. +# target, build (new) | target, rebuild: A -> B | cached (V) | rebuild: A -> B | build (new) +cache_label() { + local dir="$1" is_target="$2" + local cat prog ver dec + cat="$(category_of "$dir")"; prog="$(basename "$dir")"; ver="$(version_of "$dir")" + dec="$(cache_decision "$cat" "$prog" "$ver")" + local label + case "$dec" in + cached) label="cached ($ver)" ;; + bump:*) label="rebuild: ${dec#bump:}"; label="${label/:/ -> }" ;; + *) label="build (new)" ;; + esac + # The target always builds; never show it as plain "cached". + if [[ "$is_target" == "1" ]]; then + case "$dec" in + cached) label="build (cached $ver, rebuilt as target)" ;; + esac + echo "target, $label" + else + echo "$label" + fi +} +``` + +- [ ] **Step 2: Use it in the dry-run branch** + +In `run_target`, replace the dry-run print loop. Find: + +```sh + if [[ $DRY_RUN -eq 1 ]]; then + echo " build order:" + local d + for d in "${RESOLVED_ORDER[@]}"; do + echo " $(pkg_key "$d")$([[ "${HAS_README[$d]:-}" == 1 ]] && echo " [%README%]")" + echo "$(pkg_key "$d")" >> "$RUN_DIR/build-order.txt" + done + return + fi +``` + +Replace with: + +```sh + if [[ $DRY_RUN -eq 1 ]]; then + echo " build order:" + local d + for d in "${RESOLVED_ORDER[@]}"; do + local it=0; [[ "$d" == "$target_dir" ]] && it=1 + local rdm=""; [[ "${HAS_README[$d]:-}" == 1 ]] && rdm=" [%README%]" + printf " %-30s %s%s\n" "$(pkg_key "$d")" "$(cache_label "$d" "$it")" "$rdm" + echo "$(pkg_key "$d")" >> "$RUN_DIR/build-order.txt" + done + return + fi +``` + +- [ ] **Step 3: Use it in the real-run order print** + +In `run_target`, find: + +```sh + echo " build order: ${RESOLVED_ORDER[*]##*/}" + for d in "${RESOLVED_ORDER[@]}"; do echo "$(pkg_key "$d")" >> "$RUN_DIR/build-order.txt"; done +``` + +Replace with: + +```sh + echo " build order:" + for d in "${RESOLVED_ORDER[@]}"; do + local it=0; [[ "$d" == "$target_dir" ]] && it=1 + printf " %-30s %s\n" "$(pkg_key "$d")" "$(cache_label "$d" "$it")" + echo "$(pkg_key "$d")" >> "$RUN_DIR/build-order.txt" + done +``` + +- [ ] **Step 4: Verify syntax and self-check** + +Run: `bash -n sbo-batch-test && bash test-logic.sh` +Expected: no syntax error; ends with `ALL LOGIC CHECKS PASS`. + +- [ ] **Step 5: Commit** + +```bash +git add sbo-batch-test +git commit -S -m "Annotate build order with cache outcome (run + dry-run) + +Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>" +``` + +--- + +### Task 6: Force a controlled OUTPUT path in the build heredoc + +**Files:** +- Modify: `sbo-batch-test` (`build_one` chroot heredoc, lines 500-558) + +The harness must control where SlackBuilds write the produced package, instead +of inheriting `OUTPUT` from the chroot environment. The verified run showed +`OUTPUT=/repo` leaking in (that is the user's -current repository convention and +must NOT be used). Force `OUTPUT` to a known overlay path so the build is +deterministic and the package can be found and cached reliably. + +- [ ] **Step 1: Export a fixed OUTPUT inside the heredoc** + +In `sbo-batch-test`, find (inside the chroot heredoc, near the top after the +`. ./$prog.info` line): + +```sh +. ./$prog.info + +# Log resolved build context so the overlay never needs to be kept. +echo "===== sbo-batch-test: $prog =====" +echo "PRGNAM=\${PRGNAM:-$prog} VERSION=\${VERSION:-?} BUILD=\${BUILD:-?} TAG=\${TAG:-?}" +echo "uname -m: \$(uname -m) OUTPUT=\${OUTPUT:-/tmp}" +``` + +Replace with: + +```sh +. ./$prog.info + +# Force a controlled output dir so the harness, not the SlackBuild's environment, +# decides where the package lands. Never inherit OUTPUT (the user's -current repo +# convention sets it to /repo, which must not be touched here). +export OUTPUT=/sbo-work/output +mkdir -p "\$OUTPUT" + +# Log resolved build context so the overlay never needs to be kept. +echo "===== sbo-batch-test: $prog =====" +echo "PRGNAM=\${PRGNAM:-$prog} VERSION=\${VERSION:-?} BUILD=\${BUILD:-?} TAG=\${TAG:-?}" +echo "uname -m: \$(uname -m) OUTPUT=\$OUTPUT" +``` + +- [ ] **Step 2: Read the produced package from the forced OUTPUT** + +In the heredoc's install section, find: + +```sh +# 5. installpkg the resulting package (SlackBuilds write to \${OUTPUT:-/tmp}) +out="\${OUTPUT:-/tmp}" +pkg="\$(ls -t "\$out"/${prog}-*.t?z 2>/dev/null | head -n1)" +``` + +Replace with: + +```sh +# 5. installpkg the resulting package (written to the forced OUTPUT above) +out="\$OUTPUT" +pkg="\$(ls -t "\$out"/${prog}-*.t?z 2>/dev/null | head -n1)" +``` + +- [ ] **Step 3: Verify syntax and self-check** + +Run: `bash -n sbo-batch-test && bash test-logic.sh` +Expected: no syntax error; ends with `ALL LOGIC CHECKS PASS`. + +- [ ] **Step 4: Commit** + +```bash +git add sbo-batch-test +git commit -S -m "Force controlled OUTPUT in build, do not inherit /repo + +Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>" +``` + +--- + +### Task 7: Wire cache into build_one (install-from-cache, store-on-build) + +**Files:** +- Modify: `sbo-batch-test` (`build_one` signature + body, lines 479-568; `run_target` call site, line 629) + +`build_one` gains a third arg `is_target`. Target -> always build, then store. +Dep -> on `cached`, installpkg the cached .txz into the overlay and set status +`CACHED`; otherwise build, then store. + +- [ ] **Step 1: Change the run_target call site to pass is_target** + +In `run_target`, find: + +```sh + echo " building $key ..." + if build_one "$tmpdir" "$d"; then + echo " $key: ${ST_STATUS[$key]} (${ST_TIME[$key]}s)" + else + echo " $key: ${ST_STATUS[$key]} (${ST_TIME[$key]}s)" + failed_progs+=("$prog") + fi +``` + +Replace with: + +```sh + local it=0; [[ "$d" == "$target_dir" ]] && it=1 + echo " building $key ..." + if build_one "$tmpdir" "$d" "$it"; then + echo " $key: ${ST_STATUS[$key]} (${ST_TIME[$key]}s)" + else + echo " $key: ${ST_STATUS[$key]} (${ST_TIME[$key]}s)" + failed_progs+=("$prog") + fi +``` + +- [ ] **Step 2: Add is_target arg and the cache-install short-circuit in build_one** + +In `sbo-batch-test`, find the start of `build_one`: + +```sh +build_one() { + local tmpdir="$1" dir="$2" + local c="$tmpdir/chroot" + local prog cat key + prog="$(basename "$dir")"; cat="$(category_of "$dir")"; key="$cat/$prog" + local logf="$RUN_DIR/${cat}_${prog}.log" + local start; start=$(date +%s) + + # 1. copy SlackBuild dir into the overlay + local workroot="/sbo-work" + mkdir -p "$c$workroot" + cp -a "$dir" "$c$workroot/$prog" + + # %README% reminder carries through to summary + [[ "${HAS_README[$dir]:-}" == "1" ]] && ST_README["$key"]=1 +``` + +Replace with: + +```sh +build_one() { + local tmpdir="$1" dir="$2" is_target="${3:-0}" + local c="$tmpdir/chroot" + local prog cat key + prog="$(basename "$dir")"; cat="$(category_of "$dir")"; key="$cat/$prog" + local logf="$RUN_DIR/${cat}_${prog}.log" + local start; start=$(date +%s) + local version; version="$(version_of "$dir")" + + # %README% reminder carries through to summary + [[ "${HAS_README[$dir]:-}" == "1" ]] && ST_README["$key"]=1 + + # Dependency with a version-matching cached package: installpkg it into the + # overlay and skip the build entirely. The target never takes this path. + if [[ "$is_target" != "1" ]]; then + local cached; cached="$(cache_path "$cat" "$prog" "$version")" + if [[ -n "$cached" ]]; then + local workroot="/sbo-work" + mkdir -p "$c$workroot" + cp -a "$cached" "$c$workroot/" + { + echo "===== sbo-batch-test: $prog (from cache) =====" + echo "cached package: $(basename "$cached")" + } >> "$logf" + if chroot "$c" /bin/bash -c "installpkg --terse '$workroot/$(basename "$cached")'" >>"$logf" 2>&1; then + ST_TIME["$key"]=$(( $(date +%s) - start )) + ST_STATUS["$key"]="CACHED" + return 0 + fi + # Cache install failed: fall through and build fresh. + echo "cache install failed, building fresh" >> "$logf" + fi + fi + + # 1. copy SlackBuild dir into the overlay + local workroot="/sbo-work" + mkdir -p "$c$workroot" + cp -a "$dir" "$c$workroot/$prog" +``` + +- [ ] **Step 3: Store the produced package after a successful build** + +In `build_one`, find the tail that reads the status token: + +```sh + local status="BUILD-FAILED" + [[ -f "$statf" ]] && status="$(cat "$statf")" + local end; end=$(date +%s) + ST_TIME["$key"]=$(( end - start )) + ST_STATUS["$key"]="$status" + [[ "$status" == "SUCCESS" ]] && return 0 + ST_REASON["$key"]="see $(basename "$logf")" + return 1 +} +``` + +Replace with: + +```sh + local status="BUILD-FAILED" + [[ -f "$statf" ]] && status="$(cat "$statf")" + local end; end=$(date +%s) + ST_TIME["$key"]=$(( end - start )) + ST_STATUS["$key"]="$status" + if [[ "$status" == "SUCCESS" ]]; then + # Copy the produced package out of the overlay into the persistent cache + # before teardown, so it can be reused as a dep in this or a later run. The + # build wrote to the forced OUTPUT=/sbo-work/output (see the heredoc). + local built newest="" + for built in "$c"/sbo-work/output/${prog}-*.t?z; do + [[ -e "$built" ]] || continue + [[ -z "$newest" || "$built" -nt "$newest" ]] && newest="$built" + done + [[ -n "$newest" ]] && cache_store "$cat" "$prog" "$newest" + return 0 + fi + ST_REASON["$key"]="see $(basename "$logf")" + return 1 +} +``` + +- [ ] **Step 4: Verify syntax and self-check** + +Run: `bash -n sbo-batch-test && bash test-logic.sh` +Expected: no syntax error; ends with `ALL LOGIC CHECKS PASS`. + +- [ ] **Step 5: Commit** + +```bash +git add sbo-batch-test +git commit -S -m "Wire cache into build_one: install cached deps, store builds + +Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>" +``` + +--- + +### Task 8: CACHED status in summary + cached count + +**Files:** +- Modify: `sbo-batch-test` (`print_summary`, lines 663-698) + +`CACHED` must be counted and colored as a non-failure, and the summary line gets +a cached count. + +- [ ] **Step 1: Count and color CACHED** + +In `print_summary`, find: + +```sh + local total=$SECONDS + local succ=0 fail=0 blocked=0 +``` + +Replace with: + +```sh + local total=$SECONDS + local succ=0 fail=0 blocked=0 cached=0 +``` + +Then find the status case: + +```sh + case "$st" in + SUCCESS) col="$C_GRN"; ((succ++)) ;; + BLOCKED-BY-DEP|UNMET-DEP) col="$C_YEL"; ((blocked++)) ;; + *) col="$C_RED"; ((fail++)) ;; + esac +``` + +Replace with: + +```sh + case "$st" in + SUCCESS) col="$C_GRN"; ((succ++)) ;; + CACHED) col="$C_GRN"; ((cached++)) ;; + BLOCKED-BY-DEP|UNMET-DEP) col="$C_YEL"; ((blocked++)) ;; + *) col="$C_RED"; ((fail++)) ;; + esac +``` + +- [ ] **Step 2: Add cached to both summary lines** + +Find the screen summary line: + +```sh + printf "%s%d succeeded%s, %s%d failed%s, %s%d blocked%s, total %ss\n" \ + "$C_GRN" "$succ" "$C_RST" "$C_RED" "$fail" "$C_RST" "$C_YEL" "$blocked" "$C_RST" "$total" +``` + +Replace with: + +```sh + printf "%s%d succeeded%s, %s%d failed%s, %s%d blocked%s, %s%d cached%s, total %ss\n" \ + "$C_GRN" "$succ" "$C_RST" "$C_RED" "$fail" "$C_RST" "$C_YEL" "$blocked" "$C_RST" \ + "$C_GRN" "$cached" "$C_RST" "$total" +``` + +Find the summary.log line: + +```sh + echo "$succ succeeded, $fail failed, $blocked blocked, total ${total}s" +``` + +Replace with: + +```sh + echo "$succ succeeded, $fail failed, $blocked blocked, $cached cached, total ${total}s" +``` + +- [ ] **Step 3: Verify syntax and self-check** + +Run: `bash -n sbo-batch-test && bash test-logic.sh` +Expected: no syntax error; ends with `ALL LOGIC CHECKS PASS`. + +- [ ] **Step 4: Commit** + +```bash +git add sbo-batch-test +git commit -S -m "Report CACHED status and cached count in summary + +Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>" +``` + +--- + +### Task 9: Wipe cache on base patch + +**Files:** +- Modify: `sbo-batch-test` (`update_base`, patching branch, lines 290-299) + +When `update_base` actually patches, cached deps were built against the old base +and must be dropped. + +- [ ] **Step 1: Add the wipe after patching** + +In `update_base`, find: + +```sh + echo "$head_now" > "$marker" + echo "Base patched." +} +``` + +Replace with: + +```sh + echo "$head_now" > "$marker" + # Base changed: cached deps were built against the old base, drop them all. + if [[ -n "$PKG_CACHE" && -d "$PKG_CACHE" ]]; then + rm -rf "${PKG_CACHE:?}"/* + echo "Package cache cleared (base changed)." + fi + echo "Base patched." +} +``` + +- [ ] **Step 2: Verify syntax and self-check** + +Run: `bash -n sbo-batch-test && bash test-logic.sh` +Expected: no syntax error; ends with `ALL LOGIC CHECKS PASS`. + +- [ ] **Step 3: Commit** + +```bash +git add sbo-batch-test +git commit -S -m "Wipe package cache when base is patched + +Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>" +``` + +--- + +### Task 10: Update docs (CLAUDE.md, README.md) + +**Files:** +- Modify: `CLAUDE.md` (Status values; Architecture CONFIG entry; Open items/TODO; verified list) +- Modify: `README.md` (config var list; a cache section) + +- [ ] **Step 1: CLAUDE.md status values** + +In `CLAUDE.md`, find the Status values block: + +``` +`SUCCESS DOWNLOAD-FAILED MD5-MISMATCH BUILD-FAILED INSTALL-FAILED +BLOCKED-BY-DEP UNMET-DEP`. `%README%` recorded separately as a reminder flag, +not a status. +``` + +Replace with: + +``` +`SUCCESS CACHED DOWNLOAD-FAILED MD5-MISMATCH BUILD-FAILED INSTALL-FAILED +BLOCKED-BY-DEP UNMET-DEP`. `CACHED` = a dependency installed from the persistent +package cache instead of being rebuilt (target is never CACHED). `%README%` +recorded separately as a reminder flag, not a status. +``` + +- [ ] **Step 2: CLAUDE.md TODO entry (package cache is now done)** + +In `CLAUDE.md`, find the deferred package-output cache TODO: + +``` +- **Package-output cache** (deferred, user wants it next): keep built `.tgz` + outside the overlay, `installpkg` a cached dep instead of rebuilding. Fixes + the "shared deps rebuilt per target" cost in category mode. Likely shape: a + per-run stash in `$RUN_DIR/pkgcache` keyed `prog-version`, reused across + targets within one run. NOT implemented yet. +``` + +Replace with: + +``` +- **Package cache (DONE).** Persistent `$PKG_CACHE` (config var, empty = + disabled), SBo-tree layout `<cat>/<prog>/<prog>-<ver>-...txz`, one .txz per + prog. Key = prog+version (build/arch/tag ignored). The named target always + builds fresh and refreshes the cache; deps install from cache on a version + match (status `CACHED`), else build and cache. Cache wiped when `update_base` + patches. Pure `cache_decision`/`cache_path`/`cache_store`/`version_of` covered + by `test-logic.sh`. Spec: docs/superpowers/specs/2026-06-24-package-cache-design.md. +``` + +- [ ] **Step 3: CLAUDE.md CONFIG architecture entry** + +In `CLAUDE.md`, the CONFIG block architecture item lists the vars. Find the +opening of item 1 (`1. **CONFIG block**`) and add `PKG_CACHE` to the var list at +its start. Find: + +``` +1. **CONFIG block** - empty defaults plus a `source` of the external config +``` + +(do not change that line). Then find within the same item the sentence listing +the block's vars: + +``` + defaults (`SLACKWARE_BASE LOCAL_MIRROR_15 SBO_TREE_ROOTS`) plus + `CHROOT_LOCATION=/tmp LOG_ROOT VERSION=15.0`, sources the config if present +``` + +Replace with: + +``` + defaults (`SLACKWARE_BASE LOCAL_MIRROR_15 SBO_TREE_ROOTS`) plus + `CHROOT_LOCATION=/tmp LOG_ROOT PKG_CACHE='' VERSION=15.0`, sources the config + if present +``` + +- [ ] **Step 4: CLAUDE.md verified list** + +In `CLAUDE.md`, find the "Verified by self-check" item and append the cache +functions to its coverage list. Find: + +``` + BLOCKED-BY-DEP propagation (`depends_on_failed`, including the transitive + one-hop cascade). The check builds a fake SBo tree and sources the script with +``` + +Replace with: + +``` + BLOCKED-BY-DEP propagation (`depends_on_failed`, including the transitive + one-hop cascade), and the package-cache logic + (`cache_decision`/`cache_path`/`cache_store`/`version_of`). The check builds a + fake SBo tree and sources the script with +``` + +- [ ] **Step 5: README config var + cache section** + +In `README.md`, the config prerequisites list documents the externally-set vars. +After the `SBO_TREE_ROOTS` item (item 3) add a short note, then add a cache +section after "Populating SLACKWARE_BASE". First, find the populate section's end +(the paragraph beginning "The base is kept patched automatically"). After that +paragraph, add: + +```markdown +## Package cache + +Set `PKG_CACHE` in your config to a local directory to cache built packages +across runs (leave empty to disable). Layout mirrors the SBo tree +(`<category>/<prog>/<prog>-<version>-...txz`), one package per prog. + +The named target always builds fresh, it is the package under test. Its +dependencies are installed from the cache when their version is unchanged +(reported as `CACHED`), otherwise they are built and cached. A build order line +shows the outcome per package: `cached (1.1)`, `rebuild: 1.0 -> 1.1`, or +`build (new)`; `--dry-run` shows the same without building. The whole cache is +wiped automatically when the base is patched, since cached packages were built +against the previous base. +``` + +- [ ] **Step 6: Verify docs have no em dash and self-check still green** + +Run: `grep -n "—" CLAUDE.md README.md || echo "no em dash"` +Expected: `no em dash`. +Run: `bash test-logic.sh` +Expected: `ALL LOGIC CHECKS PASS`. + +- [ ] **Step 7: Commit** + +```bash +git add CLAUDE.md README.md +git commit -S -m "Document package cache (CLAUDE.md, README) + +Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>" +``` + +--- + +### Task 11: VM verification on buildsystem + +**Files:** none (manual verification on the build VM) + +This exercises the VM-only paths the self-check cannot reach. Run on +buildsystem as root, with `PKG_CACHE` set in `/root/.config/sbo-batch-tester/config`. + +- [ ] **Step 1: Copy the updated script + example to buildsystem** + +Run (from the repo on the workstation): +`scp sbo-batch-test config.example buildsystem:` + +- [ ] **Step 2: Set PKG_CACHE in the buildsystem config** + +On buildsystem, add to `/root/.config/sbo-batch-tester/config`: +`PKG_CACHE="/var/cache/sbo-batch-test"` + +- [ ] **Step 3: Dry-run a package with deps** + +Run: `sbo-batch-test --dry-run <prog-with-deps>` +Expected: build order lists each package with `build (new)` (cache empty), target +line prefixed `target,`. + +- [ ] **Step 4: Real run, first time (populates cache)** + +Run: `sbo-batch-test <prog-with-deps>` +Expected: deps build and are cached; target builds. Summary shows +`N succeeded, 0 failed, 0 blocked, 0 cached`. Check +`ls -R /var/cache/sbo-batch-test` shows one `.txz` per built prog. + +- [ ] **Step 5: Second run reuses cached deps** + +Run a DIFFERENT target that shares a dep, or re-run the same target. +Expected: shared deps report `CACHED`; summary cached count > 0; the target +still builds fresh (never CACHED). Per-dep log shows `(from cache)`. + +- [ ] **Step 6: Confirm eviction on version bump** + +Bump a dep's VERSION in its `.info` (or use a package whose version changed), +run a target needing it. +Expected: build order shows `rebuild: OLD -> NEW` for that dep; after the run its +cache dir holds exactly one `.txz` at the new version. + +- [ ] **Step 7: Confirm base-patch wipe (optional, only if a patch is pending)** + +If the mirror ChangeLog head has advanced, a run triggers `update_base` +patching, which prints `Package cache cleared (base changed)` and empties +`/var/cache/sbo-batch-test`. Skip if no patch is pending. + +- [ ] **Step 8: Record results** + +No commit. Note any deviation (especially the OUTPUT path in Task 6 Step 4: if +the produced package was not cached, check where the SlackBuild wrote it and +adjust the store glob in `build_one`). + +--- + +## Notes for the implementer + +- Single file `sbo-batch-test`, plain bash. Match the existing comment style and + the `ponytail:` convention for deliberate shortcuts. +- No em dash anywhere (prose, comments, commit messages). Hard rule. +- Commits are GPG-signed (`-S`). Work on master. +- `set -uo pipefail` is on: guard array/glob expansions (`[[ -e "$f" ]]` before + use), the existing code does this. +- The self-check sources the script with config set AFTER the source. The cache + tests set `PKG_CACHE` after the source too (in the package-cache block). |
