# Repo setup + update: `--setup-repo` and `--update`

Date: 2026-07-11
Status: approved, ready for implementation

## Problem

`sbo-batch-test` builds against a local SBo tree (`SBO_TREE_ROOTS[0]`). On the
build VM that tree is `/var/lib/sbopkg/SBo/15.0`, kept fresh with rsync from
slackbuilds.org (NOT a git repo). The maintainer's own packages live in two
personal subtrees inside that tree, `personal/` and `pentesting/`, cloned from
GitHub. Those subtrees must:

- be created from scratch on first setup (tree rsync'd fresh, repos cloned in),
- be refreshed per run when wanted (git pull in place),
- shadow any same-named upstream package elsewhere in the tree (the personal
  version wins, the upstream duplicate is removed),
- UNSHADOW automatically: when a package leaves a personal repo, the upstream
  copy that was previously removed must come back. Because the tree is rsync'd,
  `--update` re-rsyncs the tree (with `--delete`) before pulling and re-
  shadowing, so any previously-removed upstream package reappears and only
  currently-duplicated packages get shadowed again. The personal subtrees are
  EXCLUDED from that rsync so `--delete` never wipes them.

Today the tool does none of this; setup is a separate hand-run script written
for the -current git repo (`slackrepo_setup_SBo-danix.sh`), whose `git rm`
shadow logic does not apply to the rsync'd 15.0 tree.

This adds two modes: `--setup-repo` (full first-time build of the tree) and
`--update` (per-run refresh), sharing one shadow routine.

## Scope

In scope:
- `--setup-repo`: rm -rf the tree root, rsync fresh, clone the two personal
  repos in, shadow upstream duplicates. Standalone exit mode (like
  `--init-base`), never combined with a build.
- `--update`: rsync the tree (with `--delete`, personal subtrees excluded) to
  restore/refresh upstream and unshadow, git pull each personal subtree, shadow.
  Runs inline before a normal build.
- A shared `shadow_scan` routine (interactive confirm, `rm -r`, no git).
- A shared `rsync_tree` routine that rsyncs the SBo tree with the personal
  subtrees excluded from transfer and deletion.
- New config: `SBO_RSYNC_URL`, `SBO_PERSONAL_TREES` (assoc array dir -> URL).
- Self-check for the shadow-detection logic.
- README + CLAUDE.md docs.

Out of scope (explicitly):
- No `git subtree` on 15.0 (tree is not git; subtrees are plain clones).
- No `mkhint -F` (that is -current-only territory).
- No touching `LOCAL_MIRROR_15` (the NFS package source) or the overlay base.
  `SBO_RSYNC_URL` (public slackbuilds.org) is a distinct thing from the NFS
  mirror; the two never interact.
- Resolution stays LOCAL-tree-only (unchanged hard constraint).

## Config

Two new variables. CONFIG block sets empty defaults; `config.example` documents
real values.

CONFIG block (empty defaults, alongside the existing empties):
```bash
SBO_RSYNC_URL=""
declare -A SBO_PERSONAL_TREES=()
```

`config.example` (real values):
```bash
# rsync source for the SBo tree (same source sbopkg uses). --setup-repo rm -rf's
# SBO_TREE_ROOTS[0] and rsyncs this into it fresh; --update rsyncs it with
# --delete (personal subtrees excluded) to refresh upstream and unshadow.
# Empty: --setup-repo refuses to run; --update skips the rsync and only pulls +
# shadows. Package SOURCE for the tree only; unrelated to LOCAL_MIRROR_15 (the
# NFS package mirror).
SBO_RSYNC_URL="slackbuilds.org::slackbuilds/15.0"

# Personal SBo subtrees: directory name (under SBO_TREE_ROOTS[0]) -> git clone
# URL. --setup-repo clones each after the rsync; --update git-pulls each in
# place. Both then shadow any same-named upstream package found elsewhere in the
# tree (the personal version wins, the upstream duplicate is rm -r'd). Empty =
# no personal trees, both modes become no-ops on the subtree/shadow steps.
declare -A SBO_PERSONAL_TREES=(
  [personal]="https://github.com/danixland/my-slackbuilds.git"
  [pentesting]="https://github.com/danixland/Slackware-Pentesting-Suite.git"
)
```

Note: `SBO_PERSONAL_TREES` is an associative array. The CONFIG block must
`declare -A` it before sourcing the external config so the config's `declare -A`
(or bare assignment) lands as global. Both modes resolve dir names under
`SBO_TREE_ROOTS[0]` only (single-root VM setup, settled).

## Flags and wiring

- `--setup-repo` sets `DO_SETUP=1`. Standalone exit mode. Wired in `main`
  BEFORE `validate_env` (like `--init-base`): runs its own checks, does its
  work, exits 0. Mutually exclusive with a build; if a target is also given,
  `--setup-repo` still just sets up and exits (document this, do not build).
- `--update` sets `DO_UPDATE=1`. Inline, not an exit mode. Wired in `main`
  after `validate_env` and near `update_base`, before targets are collected and
  run. Falls through to the normal build.

`parse_args` learns both flags; `usage` documents both.

## `--setup-repo` (first-time full build of the tree)

Function `setup_repo`. Runs its own preconditions since the tree may not exist:

1. Require root.
2. `require_config`; refuse if `SBO_RSYNC_URL` is empty (print the config hint).
3. `root="${SBO_TREE_ROOTS[0]}"`. Refuse if empty.
4. Destructive confirm: print `root` and warn it will be removed and rsynced
   fresh; `read -rp "Proceed? [y/N]"`. Abort on anything but y/Y.
5. `rm -rf "$root"`.
6. `rsync_tree "$root"` (see below). On a clean-slate setup the personal dirs
   do not exist yet, so the excludes are harmless no-ops; the same helper is
   reused so both modes rsync identically. Hard-fail on rsync nonzero.
7. For each `dir` in `SBO_PERSONAL_TREES` keys:
   `git clone "${SBO_PERSONAL_TREES[$dir]}" "$root/$dir"`. Hard-fail on clone
   nonzero (a half-set-up tree is worse than none).
8. `shadow_scan "$root"`.
9. Exit 0.

No mkhint. No build.

## `--update` (per-run refresh)

Function `update_personal_trees`. Assumes the tree already exists:

1. `root="${SBO_TREE_ROOTS[0]}"`.
2. `rsync_tree "$root"` (see below). This is the UNSHADOW step: `--delete`
   restores any upstream package removed in a previous shadow, and refreshes
   upstream to current. The personal subtrees are excluded, so `--delete` never
   touches them. Requires `SBO_RSYNC_URL`; if empty, warn and skip the rsync
   (fall through to pull + shadow so `--update` still refreshes personal repos).
   On rsync nonzero: warn and continue (fail-soft; do not kill a build session).
3. For each `dir` in `SBO_PERSONAL_TREES` keys:
   - `d="$root/$dir"`. If `! -d "$d/.git"`: warn "not a git clone, skipping"
     and continue (fail-soft; do not abort the whole run).
   - `git -C "$d" pull`. On nonzero: warn and continue (fail-soft; a pull
     failure should not kill a build session).
4. `shadow_scan "$root"`.
5. Return; `main` continues into the normal build.

Plain `git pull` (subtrees are read-only mirrors, no local commits, nothing to
rebase or fast-forward-guard; a merge can never happen because the working tree
is never modified locally). Documented so no one "improves" it to `--ff-only`
without cause.

## `rsync_tree` (shared)

Signature: `rsync_tree <root>`. rsyncs the SBo tree, protecting the personal
subtrees:

```bash
rsync -a --delete --no-owner --no-group \
  <one --exclude=/<dir>/ per key in SBO_PERSONAL_TREES> \
  "$SBO_RSYNC_URL/" "$root/"
```

- `-a --delete --no-owner --no-group` match sbopkg's own rsync (files root:root,
  no ownership churn, tree made to match upstream exactly).
- One `--exclude=/<dir>/` per `SBO_PERSONAL_TREES` key, built from the actual
  keys (not literals). The leading `/` anchors each exclude to the tree root, so
  a personal dir is skipped for BOTH transfer and deletion. Without this,
  `--delete` would wipe the personal clones (they are not on the rsync source).
  This is the critical safety property.
- Returns rsync's exit status; callers decide hard-fail (setup) vs warn
  (update).

## `shadow_scan` (shared)

Adapted from `slackrepo_setup_SBo-danix.sh` lines 38-80, with two changes for
the non-git 15.0 tree: `rm -r` instead of `git rm -r`, and no `git commit`.

Signature: `shadow_scan <root>`.

1. Build `TO_SHADOW=()`:
   - Split the shadow-detection into a pure helper `collect_shadows <root>`
     that ECHOES the newline-separated list of tree-relative dirs to remove, so
     it is unit-testable without touching the filesystem destructively. It only
     reads (find), never removes.
   - For each personal tree dir present under `root` (from
     `SBO_PERSONAL_TREES` keys, skip if the dir is absent): for each immediate
     subdir `pkgdir` that has a matching `pkgdir/<pkg>.SlackBuild`, find
     same-named dirs elsewhere under `root` at depth 2, excluding every
     personal tree path. Emit each as a `root`-relative path.
   - Exclusions are built from the actual `SBO_PERSONAL_TREES` keys (not the
     literals `personal`/`pentesting`), so adding a third tree Just Works.
2. If `TO_SHADOW` empty: print "No upstream packages match your personal
   subtrees. Nothing to shadow." and return.
3. Else: print the list under "The following upstream packages will be shadowed
   (rm -r):", one indented per line. `read -rp "Proceed? [y/N]"`. On non-y:
   print "Aborted shadowing; no packages were removed." and return. On y: for
   each, print "Shadowing: <item>" and `rm -r "$root/$item"`.

Interactive confirm is required (user wants to see removals before they happen)
in both `--setup-repo` and `--update`.

## Precedence and ordering (settled)

- Personal packages win. An upstream package with the same name as one in a
  personal tree is the one removed.
- Shadow runs AFTER clone (`--setup-repo`) or pull (`--update`), so the shadow
  set reflects freshly-fetched personal packages.

## Error handling

- `--setup-repo`: rsync and clone failures are hard (exit nonzero) — a
  partially built tree is unusable. The destructive `rm -rf` is gated behind an
  interactive confirm.
- `--update`: the rsync, git-pull, and non-git-dir problems are all fail-soft
  (warn, continue), so a transient network hiccup does not abort a build
  session. An empty `SBO_RSYNC_URL` in `--update` skips the rsync (no unshadow)
  and proceeds to pull + shadow. Shadow's `rm -r` is gated behind the confirm.
- Personal-subtree safety: `rsync_tree` excludes every personal dir from
  `--delete`, so the rsync can never wipe a personal clone. This is a hard
  invariant, not a nicety.
- Missing config (`SBO_RSYNC_URL` for setup) fails fast with the config hint.

## Testing

`test-logic.sh` gains one case for `collect_shadows`:
- Build a fake tree: `academic/foo`, `network/bar` (upstream), plus
  `personal/foo` and `pentesting/bar` (each with a matching `.SlackBuild`), and
  a decoy `personal/nomatch` with no upstream twin.
- Assert `collect_shadows <root>` returns exactly `academic/foo` and
  `network/bar` (order-insensitive), and does NOT return anything under the
  personal trees, and does NOT invent a match for `nomatch`.
- Assert a personal subdir without a `.SlackBuild` is ignored.

Plus one case for the rsync exclude construction (the personal-subtree safety
invariant), without running rsync:
- Factor the `--exclude=/<dir>/` list into a pure helper
  `rsync_excludes` that echoes the flags from `SBO_PERSONAL_TREES` keys.
- Assert it emits `--exclude=/personal/` and `--exclude=/pentesting/` for the
  two-key config, and nothing for an empty config.
This guards the invariant that `--delete` can never wipe a personal clone.

rsync/clone/pull/rm themselves are I/O against external tools, no stub (mirrors
the existing `update_base` / `lint_pkg` precedent: pure host I/O, no branch
logic worth a fake). `collect_shadows` and `rsync_excludes` are the branchy/
constructed pieces, so they are what gets tested.

Self-check must stay runnable as a normal user: the fake tree is built under a
temp dir, no root, no real rsync/git.

## Docs

- `config.example`: the two new vars, documented as above.
- `README.md`: a "Repository setup and refresh" section covering `--setup-repo`
  (first-time) and `--update` (per-run), the shadow behavior, and the config
  vars. Note the rsync source is public slackbuilds.org, distinct from the NFS
  mirror.
- `CLAUDE.md`: add `setup_repo`, `update_personal_trees`, `rsync_tree`,
  `shadow_scan`, `collect_shadows` to the architecture list; add the two config
  vars to the CONFIG-block description; document the unshadow-via-rsync behavior
  and the personal-exclude safety invariant; add a verified/not-verified note
  (shadow logic self-checked; rsync/clone/pull not yet run on hardware).

## What is NOT verified until hardware

- The actual rsync of the tree, the two clones, and a real git pull (network +
  root, VM-only). Logic mirrors the reference setup script and sbopkg's rsync
  flags. `collect_shadows` is self-checked; the destructive `rm -r` path and the
  interactive confirm are not (trivial, no branch logic beyond the confirm).