# Persistent package cache design Date: 2026-06-24 Project: sbo-batch-test ## Problem In a build run, the named target's dependency chain is built one package at a time in a fresh overlay. Dependencies are rebuilt from scratch every run even when they have not changed since a previous run. Building a leaf dep can take minutes; rebuilding unchanged deps is wasted time. The fix is a persistent, on-disk cache of built packages. A dependency whose version has not changed is installed from the cache instead of being rebuilt. The package under test is always built fresh (it is the thing being verified). ## Scope - Single-package runs: `sbo-batch-test `. The named prog is the target; every other package in its resolved chain is a dependency. - Category / "all" / queue modes remain stubbed TODOs and are out of scope. The same `build_one` cache logic will apply when they are built later, with no redesign needed (each invocation's CLI target builds fresh). ## Decisions (settled) - Cache is persistent across runs, on local disk. - Cache key is `prog + version`. Build number, arch, and tag are NOT part of the match (single-arch repo, fixed tag; build number auto-increments per build and is therefore noise for matching). - The target (the CLI argument) ALWAYS builds fresh, even on a cache hit, and its fresh output refreshes the cache. - A dependency uses the cache on a version match (hit), otherwise it builds fresh and is then cached (miss). - Across runs, a cached package is reused as a dependency as long as its version is unchanged. - On store, the prog's cache directory is cleared first, so it holds exactly one `.txz`: the latest tested build of the latest tested version. - When `update_base` actually patches the base, the entire cache is wiped (cached deps were built against the old base). ## Configuration New config variable, set in the external config file (see `config.example`): ```sh # Persistent package cache. Local disk, survives across runs. Built dependency # packages are stored here and reused when their version is unchanged. Wiped # automatically when the base is patched. PKG_CACHE="/var/cache/sbo-batch-test" ``` Defaults to empty in the script; if unset/empty the cache is disabled (every package builds fresh, current behavior). `validate_env` does not hard-require it. ## Cache layout Mirrors the SBo tree: ``` $PKG_CACHE///---_.txz ``` Example: ``` /var/cache/sbo-batch-test/personal/claude-code-bin/claude-code-bin-2.1.140-x86_64-1_danix.txz ``` Each `//` directory holds exactly one `.txz` (the latest tested build). Eviction is therefore "clear the prog directory, then copy the new `.txz` in", no sibling matching needed. ## Core logic (pure, unit-testable) Three small functions, free of chroot/installpkg side effects so `test-logic.sh` can cover them: ### `cache_decision ` Globs `$PKG_CACHE///-*.t?z` (any version) and echoes one token: - `cached` — a `.txz` for exactly `` is present. - `bump::` — a `.txz` is present but for a different version (`` parsed from the cached filename, `` = requested version). - `new` — nothing cached for this prog. Version is parsed from the filename by stripping the `-` prefix and taking the field up to the next `-`. Glob is nullglob-safe (`[[ -e ]]` guard). Missing `PKG_CACHE` or prog dir is treated as `new`. If `PKG_CACHE` is empty (disabled), always returns `new`. ### `cache_store ` `mkdir -p` the prog dir, remove its existing contents, copy `` in. No-op if `PKG_CACHE` is empty. ### `cache_path ` Echoes the path of the cached `.txz` for `/` at `` (the file that made `cache_decision` return `cached`), for the installpkg-from-cache path. Empty output if no hit. ## Build-flow integration `build_one` gains an `is_target` argument (true only for the package whose dir equals the run's target dir). `run_target` passes it: for each package in the resolved chain, `is_target` is true iff its dir is the CLI target's dir. Decision at the top of `build_one`, given `cat`, `prog`, `version` (from `.info`): - **Target (`is_target` true):** always take the build path. After a successful build, `cache_store` the produced `.txz`. - **Dependency (`is_target` false):** - `cache_decision` == `cached` → cache-install path: copy the cached `.txz` from the host into the overlay (`$c/sbo-work/`), `installpkg --terse` it in the chroot, log it, set status `CACHED`. No download/build. - `cache_decision` == `bump:OLD:NEW` or `new` → build path. After success, `cache_store` the produced `.txz`. Caching across the overlay boundary: - The build runs inside the chroot; the produced `.txz` lands in the overlay's `$OUTPUT`. To cache it, the host side copies it OUT of the overlay (from the known overlay path) before teardown, into `PKG_CACHE` via `cache_store`. - A cache hit copies the host-side cached `.txz` INTO the overlay, then `installpkg --terse` runs in the chroot (same as a freshly built package). ## Reporting ### Per-package outcome (build order + per-package line) Annotate each package in the resolved order so the reason for build vs reuse is visible. Shown in BOTH a real run and `--dry-run`: ``` build order: libfoo cached (1.1) libbar rebuild: 1.0 -> 1.1 libbaz build (new) progX target, rebuild: 1.0 -> 1.1 ``` - `cached ()` — dep hit, will be / was installed from cache. - `rebuild: -> ` — version bump; old cache evicted, built fresh. - `build (new)` — nothing cached, built fresh. - The target is prefixed `target,` and always shows a build outcome (it never uses the cache); if a different version was cached it still reports the bump so the version change is visible. This derives from `cache_decision` for every package, target included. ### Status value New status `CACHED`, distinct from `SUCCESS`, set for a dependency installed from the cache. The target is never `CACHED`. Added to the status list. ### Summary Add a cached count to the summary line: ``` 3 succeeded, 0 failed, 0 blocked, 2 cached, total 41s ``` ## Dry-run `--dry-run` resolves and prints the annotated build order (above) using `cache_decision` only. It performs NO installpkg, NO build, and NO cache write. Read-only. Lets the user preview reuse vs rebuild before a real run. ## Base-patch invalidation In `update_base`, inside the existing "patching" branch (taken only when the mirror ChangeLog head differs from the recorded marker), after patching succeeds, wipe the cache: ```sh rm -rf "${PKG_CACHE:?}"/* # base changed; cached deps are suspect ``` Guarded so it never runs with an empty `PKG_CACHE`. Normal runs (base up-to-date) do not touch the cache. ## Edge cases - `PKG_CACHE` empty/unset: cache disabled, every package builds fresh (current behavior). No errors. - Cache dir missing: treated as all-miss, created on first `cache_store`. - Multiple `.txz` ever present in a prog dir (should not happen given eviction): `cache_decision` and `cache_path` take the newest by mtime. - Package extensions: stored and matched as `*.t?z` (covers txz/tgz/tbz/tlz), consistent with the existing build flow. - A dynamic SlackBuild that produces an unexpected version: the produced `.txz` is cached under whatever version it actually built; the next run's `cache_decision` compares against `.info`, so a mismatch simply rebuilds. Safe, never a false hit. ## Testing Extend `test-logic.sh` (no VM needed) to cover the pure functions against a seeded fake `$PKG_CACHE` tree: - `cache_decision` returns `cached` on exact version match. - `cache_decision` returns `bump:OLD:NEW` when a different version is cached. - `cache_decision` returns `new` for an empty/absent prog dir. - `cache_decision` returns `new` when `PKG_CACHE` is empty (disabled). - `cache_store` results in exactly one `.txz` in the prog dir (eviction). - `cache_path` returns the hit file for a matching version. VM-only (out of self-check reach, by design, same boundary as the rest of the build flow): installpkg of a cached `.txz` in the chroot, copy-out-of-overlay on store, and the base-patch wipe firing. These mirror the reference build flow. ## Out of scope / not changed - No network resolution (hard constraint). - No slackrepo interaction; the slackrepo hintfiles under `/etc/slackrepo` are a separate workflow and are not read by this tool. - Category/"all"/queue modes and `-j` parallelism remain stubbed. - `--keep` is not reintroduced.