aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorDanilo M. <danix@danix.xyz>2026-06-24 16:21:55 +0200
committerDanilo M. <danix@danix.xyz>2026-06-24 16:21:55 +0200
commitd22d96ff832f421925e1d5ee5591df776fbcbd1f (patch)
tree315b34aa493cf9e870cb6facc0960dcdcc7e51f2
parent198499d28f69abd20584a2eb88d9e1dd29e9dbec (diff)
downloadsbo-batch-tester-d22d96ff832f421925e1d5ee5591df776fbcbd1f.tar.gz
sbo-batch-tester-d22d96ff832f421925e1d5ee5591df776fbcbd1f.zip
Add package-cache design spec
Persistent on-disk cache of built dependency packages, keyed prog+version, SBo-tree layout, reused across runs while version is unchanged. Target always builds fresh; cache wiped on base patch. Pure cache_decision/store/path functions for self-check coverage. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
-rw-r--r--docs/superpowers/specs/2026-06-24-package-cache-design.md222
1 files changed, 222 insertions, 0 deletions
diff --git a/docs/superpowers/specs/2026-06-24-package-cache-design.md b/docs/superpowers/specs/2026-06-24-package-cache-design.md
new file mode 100644
index 0000000..70a073f
--- /dev/null
+++ b/docs/superpowers/specs/2026-06-24-package-cache-design.md
@@ -0,0 +1,222 @@
+# Persistent package cache design
+
+Date: 2026-06-24
+Project: sbo-batch-test
+
+## Problem
+
+In a build run, the named target's dependency chain is built one package at a
+time in a fresh overlay. Dependencies are rebuilt from scratch every run even
+when they have not changed since a previous run. Building a leaf dep can take
+minutes; rebuilding unchanged deps is wasted time.
+
+The fix is a persistent, on-disk cache of built packages. A dependency whose
+version has not changed is installed from the cache instead of being rebuilt.
+The package under test is always built fresh (it is the thing being verified).
+
+## Scope
+
+- Single-package runs: `sbo-batch-test <prog>`. The named prog is the target;
+ every other package in its resolved chain is a dependency.
+- Category / "all" / queue modes remain stubbed TODOs and are out of scope. The
+ same `build_one` cache logic will apply when they are built later, with no
+ redesign needed (each invocation's CLI target builds fresh).
+
+## Decisions (settled)
+
+- Cache is persistent across runs, on local disk.
+- Cache key is `prog + version`. Build number, arch, and tag are NOT part of the
+ match (single-arch repo, fixed tag; build number auto-increments per build and
+ is therefore noise for matching).
+- The target (the CLI argument) ALWAYS builds fresh, even on a cache hit, and
+ its fresh output refreshes the cache.
+- A dependency uses the cache on a version match (hit), otherwise it builds
+ fresh and is then cached (miss).
+- Across runs, a cached package is reused as a dependency as long as its version
+ is unchanged.
+- On store, the prog's cache directory is cleared first, so it holds exactly one
+ `.txz`: the latest tested build of the latest tested version.
+- When `update_base` actually patches the base, the entire cache is wiped (cached
+ deps were built against the old base).
+
+## Configuration
+
+New config variable, set in the external config file (see `config.example`):
+
+```sh
+# Persistent package cache. Local disk, survives across runs. Built dependency
+# packages are stored here and reused when their version is unchanged. Wiped
+# automatically when the base is patched.
+PKG_CACHE="/var/cache/sbo-batch-test"
+```
+
+Defaults to empty in the script; if unset/empty the cache is disabled (every
+package builds fresh, current behavior). `validate_env` does not hard-require it.
+
+## Cache layout
+
+Mirrors the SBo tree:
+
+```
+$PKG_CACHE/<category>/<prog>/<prog>-<version>-<arch>-<build>_<tag>.txz
+```
+
+Example:
+
+```
+/var/cache/sbo-batch-test/personal/claude-code-bin/claude-code-bin-2.1.140-x86_64-1_danix.txz
+```
+
+Each `<cat>/<prog>/` directory holds exactly one `.txz` (the latest tested
+build). Eviction is therefore "clear the prog directory, then copy the new
+`.txz` in", no sibling matching needed.
+
+## Core logic (pure, unit-testable)
+
+Three small functions, free of chroot/installpkg side effects so `test-logic.sh`
+can cover them:
+
+### `cache_decision <cat> <prog> <version>`
+
+Globs `$PKG_CACHE/<cat>/<prog>/<prog>-*.t?z` (any version) and echoes one token:
+
+- `cached` — a `.txz` for exactly `<version>` is present.
+- `bump:<oldver>:<newver>` — a `.txz` is present but for a different version
+ (`<oldver>` parsed from the cached filename, `<newver>` = requested version).
+- `new` — nothing cached for this prog.
+
+Version is parsed from the filename by stripping the `<prog>-` prefix and taking
+the field up to the next `-`. Glob is nullglob-safe (`[[ -e ]]` guard). Missing
+`PKG_CACHE` or prog dir is treated as `new`. If `PKG_CACHE` is empty (disabled),
+always returns `new`.
+
+### `cache_store <cat> <prog> <src_txz>`
+
+`mkdir -p` the prog dir, remove its existing contents, copy `<src_txz>` in. No-op
+if `PKG_CACHE` is empty.
+
+### `cache_path <cat> <prog> <version>`
+
+Echoes the path of the cached `.txz` for `<cat>/<prog>` at `<version>` (the file
+that made `cache_decision` return `cached`), for the installpkg-from-cache path.
+Empty output if no hit.
+
+## Build-flow integration
+
+`build_one` gains an `is_target` argument (true only for the package whose dir
+equals the run's target dir). `run_target` passes it: for each package in the
+resolved chain, `is_target` is true iff its dir is the CLI target's dir.
+
+Decision at the top of `build_one`, given `cat`, `prog`, `version` (from
+`.info`):
+
+- **Target (`is_target` true):** always take the build path. After a successful
+ build, `cache_store` the produced `.txz`.
+- **Dependency (`is_target` false):**
+ - `cache_decision` == `cached` → cache-install path: copy the cached `.txz`
+ from the host into the overlay (`$c/sbo-work/`), `installpkg --terse` it in
+ the chroot, log it, set status `CACHED`. No download/build.
+ - `cache_decision` == `bump:OLD:NEW` or `new` → build path. After success,
+ `cache_store` the produced `.txz`.
+
+Caching across the overlay boundary:
+
+- The build runs inside the chroot; the produced `.txz` lands in the overlay's
+ `$OUTPUT`. To cache it, the host side copies it OUT of the overlay (from the
+ known overlay path) before teardown, into `PKG_CACHE` via `cache_store`.
+- A cache hit copies the host-side cached `.txz` INTO the overlay, then
+ `installpkg --terse` runs in the chroot (same as a freshly built package).
+
+## Reporting
+
+### Per-package outcome (build order + per-package line)
+
+Annotate each package in the resolved order so the reason for build vs reuse is
+visible. Shown in BOTH a real run and `--dry-run`:
+
+```
+build order:
+ libfoo cached (1.1)
+ libbar rebuild: 1.0 -> 1.1
+ libbaz build (new)
+ progX target, rebuild: 1.0 -> 1.1
+```
+
+- `cached (<ver>)` — dep hit, will be / was installed from cache.
+- `rebuild: <old> -> <new>` — version bump; old cache evicted, built fresh.
+- `build (new)` — nothing cached, built fresh.
+- The target is prefixed `target,` and always shows a build outcome (it never
+ uses the cache); if a different version was cached it still reports the bump so
+ the version change is visible.
+
+This derives from `cache_decision` for every package, target included.
+
+### Status value
+
+New status `CACHED`, distinct from `SUCCESS`, set for a dependency installed from
+the cache. The target is never `CACHED`. Added to the status list.
+
+### Summary
+
+Add a cached count to the summary line:
+
+```
+3 succeeded, 0 failed, 0 blocked, 2 cached, total 41s
+```
+
+## Dry-run
+
+`--dry-run` resolves and prints the annotated build order (above) using
+`cache_decision` only. It performs NO installpkg, NO build, and NO cache write.
+Read-only. Lets the user preview reuse vs rebuild before a real run.
+
+## Base-patch invalidation
+
+In `update_base`, inside the existing "patching" branch (taken only when the
+mirror ChangeLog head differs from the recorded marker), after patching
+succeeds, wipe the cache:
+
+```sh
+rm -rf "${PKG_CACHE:?}"/* # base changed; cached deps are suspect
+```
+
+Guarded so it never runs with an empty `PKG_CACHE`. Normal runs (base
+up-to-date) do not touch the cache.
+
+## Edge cases
+
+- `PKG_CACHE` empty/unset: cache disabled, every package builds fresh (current
+ behavior). No errors.
+- Cache dir missing: treated as all-miss, created on first `cache_store`.
+- Multiple `.txz` ever present in a prog dir (should not happen given eviction):
+ `cache_decision` and `cache_path` take the newest by mtime.
+- Package extensions: stored and matched as `*.t?z` (covers txz/tgz/tbz/tlz),
+ consistent with the existing build flow.
+- A dynamic SlackBuild that produces an unexpected version: the produced `.txz`
+ is cached under whatever version it actually built; the next run's
+ `cache_decision` compares against `.info`, so a mismatch simply rebuilds. Safe,
+ never a false hit.
+
+## Testing
+
+Extend `test-logic.sh` (no VM needed) to cover the pure functions against a
+seeded fake `$PKG_CACHE` tree:
+
+- `cache_decision` returns `cached` on exact version match.
+- `cache_decision` returns `bump:OLD:NEW` when a different version is cached.
+- `cache_decision` returns `new` for an empty/absent prog dir.
+- `cache_decision` returns `new` when `PKG_CACHE` is empty (disabled).
+- `cache_store` results in exactly one `.txz` in the prog dir (eviction).
+- `cache_path` returns the hit file for a matching version.
+
+VM-only (out of self-check reach, by design, same boundary as the rest of the
+build flow): installpkg of a cached `.txz` in the chroot, copy-out-of-overlay on
+store, and the base-patch wipe firing. These mirror the reference build flow.
+
+## Out of scope / not changed
+
+- No network resolution (hard constraint).
+- No slackrepo interaction; the slackrepo hintfiles under `/etc/slackrepo` are a
+ separate workflow and are not read by this tool.
+- Category/"all"/queue modes and `-j` parallelism remain stubbed.
+- `--keep` is not reintroduced.