diff options
| -rw-r--r-- | docs/superpowers/plans/2026-07-11-repo-setup-update.md | 568 |
1 files changed, 568 insertions, 0 deletions
diff --git a/docs/superpowers/plans/2026-07-11-repo-setup-update.md b/docs/superpowers/plans/2026-07-11-repo-setup-update.md new file mode 100644 index 0000000..4392d88 --- /dev/null +++ b/docs/superpowers/plans/2026-07-11-repo-setup-update.md @@ -0,0 +1,568 @@ +# Repo setup + update (`--setup-repo` / `--update`) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add two modes to `sbo-batch-test`: `--setup-repo` (rm+rsync the SBo tree, clone personal subtrees, shadow upstream dups) and `--update` (rsync-with-delete to unshadow, git-pull subtrees, re-shadow) before a build. + +**Architecture:** One bash file, `sbo-batch-test`. New config `SBO_RSYNC_URL` (string) and `SBO_PERSONAL_TREES` (assoc array dir->URL). Four new functions: two pure/testable helpers (`collect_shadows`, `rsync_excludes`) and two orchestrators (`setup_repo`, `update_personal_trees`) plus a shared `shadow_scan` and `rsync_tree`. `--setup-repo` is a standalone exit mode like `--init-base`; `--update` runs inline before build. Personal subtrees are excluded from rsync `--delete` so they are never wiped. + +**Tech Stack:** bash, rsync, git, `test-logic.sh` self-check (assert-style, runs as normal user). + +Spec: `docs/superpowers/specs/2026-07-11-repo-setup-update-design.md`. + +--- + +### Task 1: Config vars (defaults + example + config-block source) + +**Files:** +- Modify: `sbo-batch-test:45-55` (CONFIG block defaults + source) +- Modify: `config.example` + +- [ ] **Step 1: Add empty defaults to the CONFIG block** + +In `sbo-batch-test`, after the `VERSION="15.0"` line (currently line 47) and BEFORE the `SBO_BATCH_CONFIG=` source block (line 49+), add: + +```bash +# rsync source for the SBo tree (same source sbopkg uses). --setup-repo rsyncs +# it fresh; --update rsyncs with --delete (personal subtrees excluded) to +# unshadow. Empty defaults; real value in the external config. +SBO_RSYNC_URL="" + +# Personal SBo subtrees: dir name (under SBO_TREE_ROOTS[0]) -> git clone URL. +# declare -A here so the external config's assignment lands as a global assoc +# array. --setup-repo clones each; --update git-pulls each; both then shadow. +declare -A SBO_PERSONAL_TREES=() +``` + +- [ ] **Step 2: Verify the script still parses** + +Run: `bash -n sbo-batch-test` +Expected: no output, exit 0. + +- [ ] **Step 3: Add the documented vars to config.example** + +Append to `config.example` (after the `VERSION="15.0"` line): + +```bash + +# rsync source for the SBo tree (same source sbopkg uses). --setup-repo rm -rf's +# SBO_TREE_ROOTS[0] and rsyncs this into it fresh; --update rsyncs it with +# --delete (personal subtrees excluded) to refresh upstream and unshadow. +# Empty: --setup-repo refuses to run; --update skips the rsync and only pulls + +# shadows. Package SOURCE for the tree only; unrelated to LOCAL_MIRROR_15 (the +# NFS package mirror). +SBO_RSYNC_URL="slackbuilds.org::slackbuilds/15.0" + +# Personal SBo subtrees: directory name (under SBO_TREE_ROOTS[0]) -> git clone +# URL. --setup-repo clones each after the rsync; --update git-pulls each in +# place. Both then shadow any same-named upstream package found elsewhere in the +# tree (the personal version wins, the upstream duplicate is rm -r'd). Empty = +# no personal trees. +declare -A SBO_PERSONAL_TREES=( + [personal]="https://github.com/danixland/my-slackbuilds.git" + [pentesting]="https://github.com/danixland/Slackware-Pentesting-Suite.git" +) +``` + +- [ ] **Step 4: Commit** + +```bash +git add sbo-batch-test config.example +git commit -S -m "Add SBO_RSYNC_URL and SBO_PERSONAL_TREES config vars" +``` + +--- + +### Task 2: `rsync_excludes` helper (pure, tested) + +**Files:** +- Modify: `sbo-batch-test` (new function near the other tree helpers, before `main`) +- Test: `test-logic.sh` + +- [ ] **Step 1: Write the failing test** + +Append to `test-logic.sh` BEFORE the `# --- result` block: + +```bash +# --- rsync_excludes: one anchored --exclude per personal tree key ------------ +declare -A SBO_PERSONAL_TREES=([personal]=url1 [pentesting]=url2) +ex="$(rsync_excludes)" +echo "$ex" | grep -qx -- "--exclude=/personal/" && ok "rsync_excludes personal" || bad "missing --exclude=/personal/, got [$ex]" +echo "$ex" | grep -qx -- "--exclude=/pentesting/" && ok "rsync_excludes pentesting" || bad "missing --exclude=/pentesting/, got [$ex]" + +declare -A SBO_PERSONAL_TREES=() +[[ -z "$(rsync_excludes)" ]] && ok "rsync_excludes empty for no trees" || bad "rsync_excludes should be empty, got [$(rsync_excludes)]" +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `bash test-logic.sh` +Expected: FAIL, `rsync_excludes: command not found` / a `bad` line. + +- [ ] **Step 3: Write minimal implementation** + +In `sbo-batch-test`, add before `main()` (e.g. just after `update_base`, near line 312): + +```bash +# Echo one anchored `--exclude=/<dir>/` per SBO_PERSONAL_TREES key, one per +# line. Anchored to the tree root so rsync --delete never wipes a personal +# clone (they are not on the rsync source). Empty when no personal trees. +rsync_excludes() { + local dir + for dir in "${!SBO_PERSONAL_TREES[@]}"; do + echo "--exclude=/$dir/" + done +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `bash test-logic.sh` +Expected: the three new `ok` lines, no `bad`. + +- [ ] **Step 5: Commit** + +```bash +git add sbo-batch-test test-logic.sh +git commit -S -m "Add rsync_excludes helper + self-check" +``` + +--- + +### Task 3: `rsync_tree` (shared rsync with personal excludes) + +**Files:** +- Modify: `sbo-batch-test` (new function after `rsync_excludes`) + +No new test: pure I/O against rsync, mirrors the `update_base`/`lint_pkg` no-stub precedent. The branchy piece (excludes) is already tested in Task 2. + +- [ ] **Step 1: Write the implementation** + +In `sbo-batch-test`, add directly after `rsync_excludes`: + +```bash +# rsync the SBo tree from SBO_RSYNC_URL into <root>, excluding every personal +# subtree from transfer AND deletion. Flags match sbopkg's own rsync (files +# root:root). Returns rsync's exit status; caller decides hard-fail vs warn. +rsync_tree() { + local root="$1" + local excludes=() + mapfile -t excludes < <(rsync_excludes) + rsync -a --delete --no-owner --no-group \ + "${excludes[@]}" \ + "$SBO_RSYNC_URL/" "$root/" +} +``` + +- [ ] **Step 2: Verify the script still parses** + +Run: `bash -n sbo-batch-test` +Expected: no output, exit 0. + +- [ ] **Step 3: Commit** + +```bash +git add sbo-batch-test +git commit -S -m "Add rsync_tree helper (personal subtrees excluded from --delete)" +``` + +--- + +### Task 4: `collect_shadows` (pure dup detection, tested) + +**Files:** +- Modify: `sbo-batch-test` (new function after `rsync_tree`) +- Test: `test-logic.sh` + +- [ ] **Step 1: Write the failing test** + +Append to `test-logic.sh` BEFORE the `# --- result` block: + +```bash +# --- collect_shadows: upstream dups of personal packages ------------------- +S=$(mktemp -d) +mkslb() { mkdir -p "$S/$1/$2"; : > "$S/$1/$2/$2.SlackBuild"; } +# upstream copies +mkslb academic foo +mkslb network bar +# personal copies (win) +mkslb personal foo +mkslb pentesting bar +# decoy: personal pkg with no upstream twin +mkslb personal nomatch +# decoy: personal subdir without a .SlackBuild (ignored) +mkdir -p "$S/personal/noslb" +declare -A SBO_PERSONAL_TREES=([personal]=u1 [pentesting]=u2) + +got="$(collect_shadows "$S" | sort)" +want="$(printf 'academic/foo\nnetwork/bar\n' | sort)" +[[ "$got" == "$want" ]] && ok "collect_shadows finds upstream dups only" || bad "collect_shadows wrong; got [$got] want [$want]" +echo "$got" | grep -q "personal/" && bad "collect_shadows returned a personal path" || ok "collect_shadows excludes personal trees" +echo "$got" | grep -q "nomatch" && bad "collect_shadows invented a match for nomatch" || ok "collect_shadows ignores no-twin pkg" +rm -rf "$S" +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `bash test-logic.sh` +Expected: FAIL, `collect_shadows: command not found`. + +- [ ] **Step 3: Write minimal implementation** + +In `sbo-batch-test`, add after `rsync_tree`. Adapted from +`slackrepo_setup_SBo-danix.sh:38-55`, generalized over `SBO_PERSONAL_TREES` keys: + +```bash +# Echo (one per line) the <root>-relative dirs of upstream packages that +# duplicate a package in a personal subtree. Reads only, never removes. A +# personal package is a subdir with a matching <name>.SlackBuild; an upstream +# dup is a same-named depth-2 dir anywhere under <root> outside the personal +# trees. +collect_shadows() { + local root="$1" + local -a personal_dirs=() prune=() + local dir + for dir in "${!SBO_PERSONAL_TREES[@]}"; do + [[ -d "$root/$dir" ]] || continue + personal_dirs+=("$root/$dir") + prune+=(! -path "$root/$dir/*") + done + [[ ${#personal_dirs[@]} -eq 0 ]] && return + local pkgdir pkg match + while IFS= read -r pkgdir; do + pkg=$(basename "$pkgdir") + [[ -f "$pkgdir/$pkg.SlackBuild" ]] || continue + while IFS= read -r match; do + echo "${match#"$root"/}" + done < <(find "$root" -mindepth 2 -maxdepth 2 -type d \ + -name "$pkg" "${prune[@]}") + done < <(find "${personal_dirs[@]}" -mindepth 1 -maxdepth 1 -type d) +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `bash test-logic.sh` +Expected: the new `ok` lines, no `bad`, `ALL LOGIC CHECKS PASS`. + +- [ ] **Step 5: Commit** + +```bash +git add sbo-batch-test test-logic.sh +git commit -S -m "Add collect_shadows dup detection + self-check" +``` + +--- + +### Task 5: `shadow_scan` (interactive removal, no test) + +**Files:** +- Modify: `sbo-batch-test` (new function after `collect_shadows`) + +No new test: `collect_shadows` is the branchy piece and is tested; this wraps it with print + confirm + `rm -r` (I/O + interaction). + +- [ ] **Step 1: Write the implementation** + +In `sbo-batch-test`, add after `collect_shadows`: + +```bash +# Show the upstream dups of personal packages, confirm, and rm -r them. Reads +# the set from collect_shadows. Interactive by design (the user wants to see +# removals before they happen). No git (the 15.0 tree is rsync'd, not git). +shadow_scan() { + local root="$1" + local -a to_shadow=() + mapfile -t to_shadow < <(collect_shadows "$root") + if [[ ${#to_shadow[@]} -eq 0 ]]; then + echo "No upstream packages match your personal subtrees. Nothing to shadow." + return + fi + echo + echo "The following upstream packages will be shadowed (rm -r):" + local item + for item in "${to_shadow[@]}"; do + echo " - $item" + done + echo + read -rp "Proceed? [y/N] " confirm + if [[ ! "$confirm" =~ ^[Yy]$ ]]; then + echo "Aborted shadowing; no packages were removed." + return + fi + for item in "${to_shadow[@]}"; do + echo "Shadowing: $item" + rm -r "${root:?}/$item" + done +} +``` + +- [ ] **Step 2: Verify the script still parses** + +Run: `bash -n sbo-batch-test` +Expected: no output, exit 0. + +- [ ] **Step 3: Commit** + +```bash +git add sbo-batch-test +git commit -S -m "Add shadow_scan (confirm + rm -r upstream dups)" +``` + +--- + +### Task 6: `setup_repo` (standalone exit mode) + +**Files:** +- Modify: `sbo-batch-test` (new function after `shadow_scan`) + +- [ ] **Step 1: Write the implementation** + +In `sbo-batch-test`, add after `shadow_scan`: + +```bash +# --setup-repo: first-time full build of the SBo tree. rm -rf the tree root, +# rsync it fresh, clone the personal subtrees in, shadow upstream dups. Runs its +# own checks (the tree may not exist yet). Standalone: exits, never builds. +setup_repo() { + [[ $EUID -eq 0 ]] || { echo "--setup-repo must run as root (rsync/rm to the tree)." >&2; exit 1; } + require_config + [[ -n "$SBO_RSYNC_URL" ]] || { echo "SBO_RSYNC_URL is unset; set it in $SBO_BATCH_CONFIG." >&2; exit 1; } + local root="${SBO_TREE_ROOTS[0]}" + [[ -n "$root" ]] || { echo "SBO_TREE_ROOTS[0] is empty." >&2; exit 1; } + + echo "This will REMOVE $root and rsync it fresh from $SBO_RSYNC_URL." + read -rp "Proceed? [y/N] " confirm + [[ "$confirm" =~ ^[Yy]$ ]] || { echo "Aborted."; exit 0; } + + rm -rf "${root:?}" + echo "Rsyncing tree..." + rsync_tree "$root" || { echo "rsync failed." >&2; exit 1; } + + local dir + for dir in "${!SBO_PERSONAL_TREES[@]}"; do + echo "Cloning $dir..." + git clone "${SBO_PERSONAL_TREES[$dir]}" "$root/$dir" \ + || { echo "clone of $dir failed." >&2; exit 1; } + done + + shadow_scan "$root" + echo "Repository ready at $root" + exit 0 +} +``` + +- [ ] **Step 2: Verify the script still parses** + +Run: `bash -n sbo-batch-test` +Expected: no output, exit 0. + +- [ ] **Step 3: Commit** + +```bash +git add sbo-batch-test +git commit -S -m "Add setup_repo (rm+rsync+clone+shadow, exit mode)" +``` + +--- + +### Task 7: `update_personal_trees` (inline, fail-soft) + +**Files:** +- Modify: `sbo-batch-test` (new function after `setup_repo`) + +- [ ] **Step 1: Write the implementation** + +In `sbo-batch-test`, add after `setup_repo`: + +```bash +# --update: per-run refresh. rsync the tree (--delete, personal excluded) to +# unshadow + refresh upstream, git-pull each personal subtree, re-shadow. Inline +# and fail-soft: a network hiccup warns but never kills the build session. +update_personal_trees() { + local root="${SBO_TREE_ROOTS[0]}" + [[ -n "$root" ]] || { echo "SBO_TREE_ROOTS[0] is empty; skipping --update." >&2; return; } + + if [[ -n "$SBO_RSYNC_URL" ]]; then + echo "Refreshing tree (rsync, personal subtrees preserved)..." + rsync_tree "$root" || echo "WARN: rsync failed; continuing." >&2 + else + echo "SBO_RSYNC_URL unset; skipping tree rsync (no unshadow)." >&2 + fi + + local dir d + for dir in "${!SBO_PERSONAL_TREES[@]}"; do + d="$root/$dir" + if [[ ! -d "$d/.git" ]]; then + echo "WARN: $d is not a git clone, skipping pull." >&2 + continue + fi + echo "Pulling $dir..." + git -C "$d" pull || echo "WARN: git pull failed in $dir; continuing." >&2 + done + + shadow_scan "$root" +} +``` + +- [ ] **Step 2: Verify the script still parses** + +Run: `bash -n sbo-batch-test` +Expected: no output, exit 0. + +- [ ] **Step 3: Commit** + +```bash +git add sbo-batch-test +git commit -S -m "Add update_personal_trees (rsync-unshadow + pull + shadow)" +``` + +--- + +### Task 8: Flags, usage, and main wiring + +**Files:** +- Modify: `sbo-batch-test` globals (near line 75), `usage` (near 113), `parse_args` (139-160), `main` (894-901) + +- [ ] **Step 1: Add flag globals** + +In `sbo-batch-test`, next to `INIT_BASE=0` (line 75), add: + +```bash +SETUP_REPO=0 # --setup-repo: rm+rsync tree, clone subtrees, shadow, then exit +DO_UPDATE=0 # --update: rsync-unshadow + pull subtrees + shadow before build +``` + +- [ ] **Step 2: Document both in usage** + +In `usage()`, after the `--init-base` help block (around line 113-138), add two entries in the same style: + +``` + --setup-repo First-time build of the SBo tree: REMOVE SBO_TREE_ROOTS[0], + rsync it fresh from SBO_RSYNC_URL, clone the SBO_PERSONAL_TREES + subtrees in, shadow duplicated upstream packages, then exit. + Does not build. Runs as root. + --update Before building: rsync the tree (--delete, personal subtrees + preserved) to unshadow and refresh upstream, git-pull each + personal subtree, re-shadow duplicated upstream packages. +``` + +- [ ] **Step 3: Parse both flags** + +In `parse_args`, add cases next to `--init-base` (line 147): + +```bash + --setup-repo) SETUP_REPO=1; shift ;; + --update) DO_UPDATE=1; shift ;; +``` + +Then relax the "no target" guard (line 157) so `--setup-repo` needs no target: + +```bash + if [[ -z "$TARGET_ARG" && $INIT_BASE -eq 0 && $SETUP_REPO -eq 0 ]]; then + echo "No target given." >&2; usage >&2; exit 2 + fi +``` + +- [ ] **Step 4: Wire into main** + +In `main` (lines 894-901), after `parse_args "$@"` and the `INIT_BASE` block, add `setup_repo` as an exit mode, and run `update_personal_trees` after `validate_env`. Result: + +```bash +main() { + parse_args "$@" + init_color + if [[ $INIT_BASE -eq 1 ]]; then + init_base + fi + if [[ $SETUP_REPO -eq 1 ]]; then + setup_repo + fi + validate_env + [[ $DO_UPDATE -eq 1 ]] && update_personal_trees + ... +``` + +Note: match the EXACT existing order around `init_color`/`init_base`/`validate_env` in the file; insert `setup_repo` right after the `init_base` block (both are pre-`validate_env` exit modes) and `update_personal_trees` right after `validate_env`. Read lines 894-910 first and splice in place. + +- [ ] **Step 5: Verify parse + self-check** + +Run: `bash -n sbo-batch-test && bash test-logic.sh` +Expected: parse clean; `ALL LOGIC CHECKS PASS`. + +- [ ] **Step 6: Smoke the flags (no side effects)** + +Run: `bash sbo-batch-test --help` +Expected: usage shows `--setup-repo` and `--update`. + +Run: `SBO_BATCH_CONFIG=/nonexistent bash sbo-batch-test --setup-repo` +Expected: fails fast (not root, or missing config), does NOT rm anything. + +- [ ] **Step 7: Commit** + +```bash +git add sbo-batch-test +git commit -S -m "Wire --setup-repo and --update flags into main" +``` + +--- + +### Task 9: Docs (README + CLAUDE.md) + +**Files:** +- Modify: `README.md`, `CLAUDE.md` + +- [ ] **Step 1: README section** + +Add a "Repository setup and refresh" section to `README.md` (near the other mode docs). Cover: +- `--setup-repo`: first-time, REMOVES and rsyncs the tree, clones personal + subtrees, shadows upstream dups, exits. Root. Interactive confirm. +- `--update`: before a build, rsyncs (--delete, personal preserved) to unshadow, + pulls personal subtrees, re-shadows. Fail-soft. +- The two config vars (`SBO_RSYNC_URL`, `SBO_PERSONAL_TREES`), noting the rsync + source is public slackbuilds.org, distinct from the NFS `LOCAL_MIRROR_15`. +- Shadowing: personal packages win; same-named upstream dup is rm -r'd. Dropping + a package from a personal repo self-heals on the next `--update` (rsync brings + the upstream copy back). + +- [ ] **Step 2: CLAUDE.md updates** + +In `CLAUDE.md`: +- Architecture list: add `rsync_excludes`, `rsync_tree`, `collect_shadows`, + `shadow_scan`, `setup_repo`, `update_personal_trees` with one-line roles. +- CONFIG-block description: add `SBO_RSYNC_URL` and `SBO_PERSONAL_TREES`. +- "What is verified vs not": note `collect_shadows` + `rsync_excludes` are + self-checked; rsync/clone/pull and the interactive shadow removal are NOT yet + run on hardware. Note the personal-exclude safety invariant. + +- [ ] **Step 3: Verify no em dash** + +Run: `grep -nP '\x{2014}' README.md CLAUDE.md config.example sbo-batch-test` +Expected: no output (no em dash anywhere). + +- [ ] **Step 4: Final self-check + parse** + +Run: `bash -n sbo-batch-test && bash test-logic.sh` +Expected: `ALL LOGIC CHECKS PASS`. + +- [ ] **Step 5: Commit** + +```bash +git add README.md CLAUDE.md +git commit -S -m "Document --setup-repo and --update" +``` + +--- + +## Notes for the implementer + +- All commits GPG-signed (`-S`); the environment does not prompt. Verify with + `git log --format='%h %G? %s'` (want `G`). +- No em dash character anywhere (hard rule). Use commas/periods. +- Do NOT run any build (`bash *.SlackBuild`, `slackrepo`, `sbopkglint` on a + package). The user builds on their infra. You only edit + run `bash -n` and + `bash test-logic.sh`. +- Do NOT run `--setup-repo` / `--update` for real here: they rsync/rm/clone + against the VM tree and need root + network. Live-testing is the user's, on + buildsystem. The self-check covers the pure logic. +- After the plan lands, the script gets scp'd to buildsystem + (`/usr/local/bin/sbo-batch-test`) by the user. |