path: root/docs/superpowers/plans/2026-07-11-repo-setup-update.md

# Repo setup + update (`--setup-repo` / `--update`) Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Add two modes to `sbo-batch-test`: `--setup-repo` (rm+rsync the SBo tree, clone personal subtrees, shadow upstream dups) and `--update` (rsync-with-delete to unshadow, git-pull subtrees, re-shadow) before a build.

**Architecture:** One bash file, `sbo-batch-test`. New config `SBO_RSYNC_URL` (string) and `SBO_PERSONAL_TREES` (assoc array dir->URL). Four new functions: two pure/testable helpers (`collect_shadows`, `rsync_excludes`) and two orchestrators (`setup_repo`, `update_personal_trees`) plus a shared `shadow_scan` and `rsync_tree`. `--setup-repo` is a standalone exit mode like `--init-base`; `--update` runs inline before build. Personal subtrees are excluded from rsync `--delete` so they are never wiped.

**Tech Stack:** bash, rsync, git, `test-logic.sh` self-check (assert-style, runs as normal user).

Spec: `docs/superpowers/specs/2026-07-11-repo-setup-update-design.md`.

---

### Task 1: Config vars (defaults + example + config-block source)

**Files:**
- Modify: `sbo-batch-test:45-55` (CONFIG block defaults + source)
- Modify: `config.example`

- [ ] **Step 1: Add empty defaults to the CONFIG block**

In `sbo-batch-test`, after the `VERSION="15.0"` line (currently line 47) and BEFORE the `SBO_BATCH_CONFIG=` source block (line 49+), add:

```bash
# rsync source for the SBo tree (same source sbopkg uses). --setup-repo rsyncs
# it fresh; --update rsyncs with --delete (personal subtrees excluded) to
# unshadow. Empty defaults; real value in the external config.
SBO_RSYNC_URL=""

# Personal SBo subtrees: dir name (under SBO_TREE_ROOTS[0]) -> git clone URL.
# declare -A here so the external config's assignment lands as a global assoc
# array. --setup-repo clones each; --update git-pulls each; both then shadow.
declare -A SBO_PERSONAL_TREES=()
```

- [ ] **Step 2: Verify the script still parses**

Run: `bash -n sbo-batch-test`
Expected: no output, exit 0.

- [ ] **Step 3: Add the documented vars to config.example**

Append to `config.example` (after the `VERSION="15.0"` line):

```bash

# rsync source for the SBo tree (same source sbopkg uses). --setup-repo rm -rf's
# SBO_TREE_ROOTS[0] and rsyncs this into it fresh; --update rsyncs it with
# --delete (personal subtrees excluded) to refresh upstream and unshadow.
# Empty: --setup-repo refuses to run; --update skips the rsync and only pulls +
# shadows. Package SOURCE for the tree only; unrelated to LOCAL_MIRROR_15 (the
# NFS package mirror).
SBO_RSYNC_URL="slackbuilds.org::slackbuilds/15.0"

# Personal SBo subtrees: directory name (under SBO_TREE_ROOTS[0]) -> git clone
# URL. --setup-repo clones each after the rsync; --update git-pulls each in
# place. Both then shadow any same-named upstream package found elsewhere in the
# tree (the personal version wins, the upstream duplicate is rm -r'd). Empty =
# no personal trees.
declare -A SBO_PERSONAL_TREES=(
  [personal]="https://github.com/danixland/my-slackbuilds.git"
  [pentesting]="https://github.com/danixland/Slackware-Pentesting-Suite.git"
)
```

- [ ] **Step 4: Commit**

```bash
git add sbo-batch-test config.example
git commit -S -m "Add SBO_RSYNC_URL and SBO_PERSONAL_TREES config vars"
```

---

### Task 2: `rsync_excludes` helper (pure, tested)

**Files:**
- Modify: `sbo-batch-test` (new function near the other tree helpers, before `main`)
- Test: `test-logic.sh`

- [ ] **Step 1: Write the failing test**

Append to `test-logic.sh` BEFORE the `# --- result` block:

```bash
# --- rsync_excludes: one anchored --exclude per personal tree key ------------
declare -A SBO_PERSONAL_TREES=([personal]=url1 [pentesting]=url2)
ex="$(rsync_excludes)"
echo "$ex" | grep -qx -- "--exclude=/personal/"   && ok "rsync_excludes personal"   || bad "missing --exclude=/personal/, got [$ex]"
echo "$ex" | grep -qx -- "--exclude=/pentesting/" && ok "rsync_excludes pentesting" || bad "missing --exclude=/pentesting/, got [$ex]"

declare -A SBO_PERSONAL_TREES=()
[[ -z "$(rsync_excludes)" ]] && ok "rsync_excludes empty for no trees" || bad "rsync_excludes should be empty, got [$(rsync_excludes)]"
```

- [ ] **Step 2: Run test to verify it fails**

Run: `bash test-logic.sh`
Expected: FAIL, `rsync_excludes: command not found` / a `bad` line.

- [ ] **Step 3: Write minimal implementation**

In `sbo-batch-test`, add before `main()` (e.g. just after `update_base`, near line 312):

```bash
# Echo one anchored `--exclude=/<dir>/` per SBO_PERSONAL_TREES key, one per
# line. Anchored to the tree root so rsync --delete never wipes a personal
# clone (they are not on the rsync source). Empty when no personal trees.
rsync_excludes() {
  local dir
  for dir in "${!SBO_PERSONAL_TREES[@]}"; do
    echo "--exclude=/$dir/"
  done
}
```

- [ ] **Step 4: Run test to verify it passes**

Run: `bash test-logic.sh`
Expected: the three new `ok` lines, no `bad`.

- [ ] **Step 5: Commit**

```bash
git add sbo-batch-test test-logic.sh
git commit -S -m "Add rsync_excludes helper + self-check"
```

---

### Task 3: `rsync_tree` (shared rsync with personal excludes)

**Files:**
- Modify: `sbo-batch-test` (new function after `rsync_excludes`)

No new test: pure I/O against rsync, mirrors the `update_base`/`lint_pkg` no-stub precedent. The branchy piece (excludes) is already tested in Task 2.

- [ ] **Step 1: Write the implementation**

In `sbo-batch-test`, add directly after `rsync_excludes`:

```bash
# rsync the SBo tree from SBO_RSYNC_URL into <root>, excluding every personal
# subtree from transfer AND deletion. Flags match sbopkg's own rsync (files
# root:root). Returns rsync's exit status; caller decides hard-fail vs warn.
rsync_tree() {
  local root="$1"
  local excludes=()
  mapfile -t excludes < <(rsync_excludes)
  rsync -a --delete --no-owner --no-group \
    "${excludes[@]}" \
    "$SBO_RSYNC_URL/" "$root/"
}
```

- [ ] **Step 2: Verify the script still parses**

Run: `bash -n sbo-batch-test`
Expected: no output, exit 0.

- [ ] **Step 3: Commit**

```bash
git add sbo-batch-test
git commit -S -m "Add rsync_tree helper (personal subtrees excluded from --delete)"
```

---

### Task 4: `collect_shadows` (pure dup detection, tested)

**Files:**
- Modify: `sbo-batch-test` (new function after `rsync_tree`)
- Test: `test-logic.sh`

- [ ] **Step 1: Write the failing test**

Append to `test-logic.sh` BEFORE the `# --- result` block:

```bash
# --- collect_shadows: upstream dups of personal packages -------------------
S=$(mktemp -d)
mkslb() { mkdir -p "$S/$1/$2"; : > "$S/$1/$2/$2.SlackBuild"; }
# upstream copies
mkslb academic foo
mkslb network  bar
# personal copies (win)
mkslb personal   foo
mkslb pentesting bar
# decoy: personal pkg with no upstream twin
mkslb personal nomatch
# decoy: personal subdir without a .SlackBuild (ignored)
mkdir -p "$S/personal/noslb"
declare -A SBO_PERSONAL_TREES=([personal]=u1 [pentesting]=u2)

got="$(collect_shadows "$S" | sort)"
want="$(printf 'academic/foo\nnetwork/bar\n' | sort)"
[[ "$got" == "$want" ]] && ok "collect_shadows finds upstream dups only" || bad "collect_shadows wrong; got [$got] want [$want]"
echo "$got" | grep -q "personal/"   && bad "collect_shadows returned a personal path" || ok "collect_shadows excludes personal trees"
echo "$got" | grep -q "nomatch"     && bad "collect_shadows invented a match for nomatch" || ok "collect_shadows ignores no-twin pkg"
rm -rf "$S"
```

- [ ] **Step 2: Run test to verify it fails**

Run: `bash test-logic.sh`
Expected: FAIL, `collect_shadows: command not found`.

- [ ] **Step 3: Write minimal implementation**

In `sbo-batch-test`, add after `rsync_tree`. Adapted from
`slackrepo_setup_SBo-danix.sh:38-55`, generalized over `SBO_PERSONAL_TREES` keys:

```bash
# Echo (one per line) the <root>-relative dirs of upstream packages that
# duplicate a package in a personal subtree. Reads only, never removes. A
# personal package is a subdir with a matching <name>.SlackBuild; an upstream
# dup is a same-named depth-2 dir anywhere under <root> outside the personal
# trees.
collect_shadows() {
  local root="$1"
  local -a personal_dirs=() prune=()
  local dir
  for dir in "${!SBO_PERSONAL_TREES[@]}"; do
    [[ -d "$root/$dir" ]] || continue
    personal_dirs+=("$root/$dir")
    prune+=(! -path "$root/$dir/*")
  done
  [[ ${#personal_dirs[@]} -eq 0 ]] && return
  local pkgdir pkg match
  while IFS= read -r pkgdir; do
    pkg=$(basename "$pkgdir")
    [[ -f "$pkgdir/$pkg.SlackBuild" ]] || continue
    while IFS= read -r match; do
      echo "${match#"$root"/}"
    done < <(find "$root" -mindepth 2 -maxdepth 2 -type d \
        -name "$pkg" "${prune[@]}")
  done < <(find "${personal_dirs[@]}" -mindepth 1 -maxdepth 1 -type d)
}
```

- [ ] **Step 4: Run test to verify it passes**

Run: `bash test-logic.sh`
Expected: the new `ok` lines, no `bad`, `ALL LOGIC CHECKS PASS`.

- [ ] **Step 5: Commit**

```bash
git add sbo-batch-test test-logic.sh
git commit -S -m "Add collect_shadows dup detection + self-check"
```

---

### Task 5: `shadow_scan` (interactive removal, no test)

**Files:**
- Modify: `sbo-batch-test` (new function after `collect_shadows`)

No new test: `collect_shadows` is the branchy piece and is tested; this wraps it with print + confirm + `rm -r` (I/O + interaction).

- [ ] **Step 1: Write the implementation**

In `sbo-batch-test`, add after `collect_shadows`:

```bash
# Show the upstream dups of personal packages, confirm, and rm -r them. Reads
# the set from collect_shadows. Interactive by design (the user wants to see
# removals before they happen). No git (the 15.0 tree is rsync'd, not git).
shadow_scan() {
  local root="$1"
  local -a to_shadow=()
  mapfile -t to_shadow < <(collect_shadows "$root")
  if [[ ${#to_shadow[@]} -eq 0 ]]; then
    echo "No upstream packages match your personal subtrees. Nothing to shadow."
    return
  fi
  echo
  echo "The following upstream packages will be shadowed (rm -r):"
  local item
  for item in "${to_shadow[@]}"; do
    echo "  - $item"
  done
  echo
  read -rp "Proceed? [y/N] " confirm
  if [[ ! "$confirm" =~ ^[Yy]$ ]]; then
    echo "Aborted shadowing; no packages were removed."
    return
  fi
  for item in "${to_shadow[@]}"; do
    echo "Shadowing: $item"
    rm -r "${root:?}/$item"
  done
}
```

- [ ] **Step 2: Verify the script still parses**

Run: `bash -n sbo-batch-test`
Expected: no output, exit 0.

- [ ] **Step 3: Commit**

```bash
git add sbo-batch-test
git commit -S -m "Add shadow_scan (confirm + rm -r upstream dups)"
```

---

### Task 6: `setup_repo` (standalone exit mode)

**Files:**
- Modify: `sbo-batch-test` (new function after `shadow_scan`)

- [ ] **Step 1: Write the implementation**

In `sbo-batch-test`, add after `shadow_scan`:

```bash
# --setup-repo: first-time full build of the SBo tree. rm -rf the tree root,
# rsync it fresh, clone the personal subtrees in, shadow upstream dups. Runs its
# own checks (the tree may not exist yet). Standalone: exits, never builds.
setup_repo() {
  [[ $EUID -eq 0 ]] || { echo "--setup-repo must run as root (rsync/rm to the tree)." >&2; exit 1; }
  require_config
  [[ -n "$SBO_RSYNC_URL" ]] || { echo "SBO_RSYNC_URL is unset; set it in $SBO_BATCH_CONFIG." >&2; exit 1; }
  local root="${SBO_TREE_ROOTS[0]}"
  [[ -n "$root" ]] || { echo "SBO_TREE_ROOTS[0] is empty." >&2; exit 1; }

  echo "This will REMOVE $root and rsync it fresh from $SBO_RSYNC_URL."
  read -rp "Proceed? [y/N] " confirm
  [[ "$confirm" =~ ^[Yy]$ ]] || { echo "Aborted."; exit 0; }

  rm -rf "${root:?}"
  echo "Rsyncing tree..."
  rsync_tree "$root" || { echo "rsync failed." >&2; exit 1; }

  local dir
  for dir in "${!SBO_PERSONAL_TREES[@]}"; do
    echo "Cloning $dir..."
    git clone "${SBO_PERSONAL_TREES[$dir]}" "$root/$dir" \
      || { echo "clone of $dir failed." >&2; exit 1; }
  done

  shadow_scan "$root"
  echo "Repository ready at $root"
  exit 0
}
```

- [ ] **Step 2: Verify the script still parses**

Run: `bash -n sbo-batch-test`
Expected: no output, exit 0.

- [ ] **Step 3: Commit**

```bash
git add sbo-batch-test
git commit -S -m "Add setup_repo (rm+rsync+clone+shadow, exit mode)"
```

---

### Task 7: `update_personal_trees` (inline, fail-soft)

**Files:**
- Modify: `sbo-batch-test` (new function after `setup_repo`)

- [ ] **Step 1: Write the implementation**

In `sbo-batch-test`, add after `setup_repo`:

```bash
# --update: per-run refresh. rsync the tree (--delete, personal excluded) to
# unshadow + refresh upstream, git-pull each personal subtree, re-shadow. Inline
# and fail-soft: a network hiccup warns but never kills the build session.
update_personal_trees() {
  local root="${SBO_TREE_ROOTS[0]}"
  [[ -n "$root" ]] || { echo "SBO_TREE_ROOTS[0] is empty; skipping --update." >&2; return; }

  if [[ -n "$SBO_RSYNC_URL" ]]; then
    echo "Refreshing tree (rsync, personal subtrees preserved)..."
    rsync_tree "$root" || echo "WARN: rsync failed; continuing." >&2
  else
    echo "SBO_RSYNC_URL unset; skipping tree rsync (no unshadow)." >&2
  fi

  local dir d
  for dir in "${!SBO_PERSONAL_TREES[@]}"; do
    d="$root/$dir"
    if [[ ! -d "$d/.git" ]]; then
      echo "WARN: $d is not a git clone, skipping pull." >&2
      continue
    fi
    echo "Pulling $dir..."
    git -C "$d" pull || echo "WARN: git pull failed in $dir; continuing." >&2
  done

  shadow_scan "$root"
}
```

- [ ] **Step 2: Verify the script still parses**

Run: `bash -n sbo-batch-test`
Expected: no output, exit 0.

- [ ] **Step 3: Commit**

```bash
git add sbo-batch-test
git commit -S -m "Add update_personal_trees (rsync-unshadow + pull + shadow)"
```

---

### Task 8: Flags, usage, and main wiring

**Files:**
- Modify: `sbo-batch-test` globals (near line 75), `usage` (near 113), `parse_args` (139-160), `main` (894-901)

- [ ] **Step 1: Add flag globals**

In `sbo-batch-test`, next to `INIT_BASE=0` (line 75), add:

```bash
SETUP_REPO=0    # --setup-repo: rm+rsync tree, clone subtrees, shadow, then exit
DO_UPDATE=0     # --update: rsync-unshadow + pull subtrees + shadow before build
```

- [ ] **Step 2: Document both in usage**

In `usage()`, after the `--init-base` help block (around line 113-138), add two entries in the same style:

```
  --setup-repo     First-time build of the SBo tree: REMOVE SBO_TREE_ROOTS[0],
                   rsync it fresh from SBO_RSYNC_URL, clone the SBO_PERSONAL_TREES
                   subtrees in, shadow duplicated upstream packages, then exit.
                   Does not build. Runs as root.
  --update         Before building: rsync the tree (--delete, personal subtrees
                   preserved) to unshadow and refresh upstream, git-pull each
                   personal subtree, re-shadow duplicated upstream packages.
```

- [ ] **Step 3: Parse both flags**

In `parse_args`, add cases next to `--init-base` (line 147):

```bash
      --setup-repo) SETUP_REPO=1; shift ;;
      --update) DO_UPDATE=1; shift ;;
```

Then relax the "no target" guard (line 157) so `--setup-repo` needs no target:

```bash
  if [[ -z "$TARGET_ARG" && $INIT_BASE -eq 0 && $SETUP_REPO -eq 0 ]]; then
    echo "No target given." >&2; usage >&2; exit 2
  fi
```

- [ ] **Step 4: Wire into main**

In `main` (lines 894-901), after `parse_args "$@"` and the `INIT_BASE` block, add `setup_repo` as an exit mode, and run `update_personal_trees` after `validate_env`. Result:

```bash
main() {
  parse_args "$@"
  init_color
  if [[ $INIT_BASE -eq 1 ]]; then
    init_base
  fi
  if [[ $SETUP_REPO -eq 1 ]]; then
    setup_repo
  fi
  validate_env
  [[ $DO_UPDATE -eq 1 ]] && update_personal_trees
  ...
```

Note: match the EXACT existing order around `init_color`/`init_base`/`validate_env` in the file; insert `setup_repo` right after the `init_base` block (both are pre-`validate_env` exit modes) and `update_personal_trees` right after `validate_env`. Read lines 894-910 first and splice in place.

- [ ] **Step 5: Verify parse + self-check**

Run: `bash -n sbo-batch-test && bash test-logic.sh`
Expected: parse clean; `ALL LOGIC CHECKS PASS`.

- [ ] **Step 6: Smoke the flags (no side effects)**

Run: `bash sbo-batch-test --help`
Expected: usage shows `--setup-repo` and `--update`.

Run: `SBO_BATCH_CONFIG=/nonexistent bash sbo-batch-test --setup-repo`
Expected: fails fast (not root, or missing config), does NOT rm anything.

- [ ] **Step 7: Commit**

```bash
git add sbo-batch-test
git commit -S -m "Wire --setup-repo and --update flags into main"
```

---

### Task 9: Docs (README + CLAUDE.md)

**Files:**
- Modify: `README.md`, `CLAUDE.md`

- [ ] **Step 1: README section**

Add a "Repository setup and refresh" section to `README.md` (near the other mode docs). Cover:
- `--setup-repo`: first-time, REMOVES and rsyncs the tree, clones personal
  subtrees, shadows upstream dups, exits. Root. Interactive confirm.
- `--update`: before a build, rsyncs (--delete, personal preserved) to unshadow,
  pulls personal subtrees, re-shadows. Fail-soft.
- The two config vars (`SBO_RSYNC_URL`, `SBO_PERSONAL_TREES`), noting the rsync
  source is public slackbuilds.org, distinct from the NFS `LOCAL_MIRROR_15`.
- Shadowing: personal packages win; same-named upstream dup is rm -r'd. Dropping
  a package from a personal repo self-heals on the next `--update` (rsync brings
  the upstream copy back).

- [ ] **Step 2: CLAUDE.md updates**

In `CLAUDE.md`:
- Architecture list: add `rsync_excludes`, `rsync_tree`, `collect_shadows`,
  `shadow_scan`, `setup_repo`, `update_personal_trees` with one-line roles.
- CONFIG-block description: add `SBO_RSYNC_URL` and `SBO_PERSONAL_TREES`.
- "What is verified vs not": note `collect_shadows` + `rsync_excludes` are
  self-checked; rsync/clone/pull and the interactive shadow removal are NOT yet
  run on hardware. Note the personal-exclude safety invariant.

- [ ] **Step 3: Verify no em dash**

Run: `grep -nP '\x{2014}' README.md CLAUDE.md config.example sbo-batch-test`
Expected: no output (no em dash anywhere).

- [ ] **Step 4: Final self-check + parse**

Run: `bash -n sbo-batch-test && bash test-logic.sh`
Expected: `ALL LOGIC CHECKS PASS`.

- [ ] **Step 5: Commit**

```bash
git add README.md CLAUDE.md
git commit -S -m "Document --setup-repo and --update"
```

---

## Notes for the implementer

- All commits GPG-signed (`-S`); the environment does not prompt. Verify with
  `git log --format='%h %G? %s'` (want `G`).
- No em dash character anywhere (hard rule). Use commas/periods.
- Do NOT run any build (`bash *.SlackBuild`, `slackrepo`, `sbopkglint` on a
  package). The user builds on their infra. You only edit + run `bash -n` and
  `bash test-logic.sh`.
- Do NOT run `--setup-repo` / `--update` for real here: they rsync/rm/clone
  against the VM tree and need root + network. Live-testing is the user's, on
  buildsystem. The self-check covers the pure logic.
- After the plan lands, the script gets scp'd to buildsystem
  (`/usr/local/bin/sbo-batch-test`) by the user.