diff options
Diffstat (limited to 'docs/superpowers/specs')
| -rw-r--r-- | docs/superpowers/specs/2026-07-11-repo-setup-update-design.md | 99 |
1 files changed, 75 insertions, 24 deletions
diff --git a/docs/superpowers/specs/2026-07-11-repo-setup-update-design.md b/docs/superpowers/specs/2026-07-11-repo-setup-update-design.md index 0902d42..e108850 100644 --- a/docs/superpowers/specs/2026-07-11-repo-setup-update-design.md +++ b/docs/superpowers/specs/2026-07-11-repo-setup-update-design.md @@ -14,7 +14,13 @@ GitHub. Those subtrees must: - be created from scratch on first setup (tree rsync'd fresh, repos cloned in), - be refreshed per run when wanted (git pull in place), - shadow any same-named upstream package elsewhere in the tree (the personal - version wins, the upstream duplicate is removed). + version wins, the upstream duplicate is removed), +- UNSHADOW automatically: when a package leaves a personal repo, the upstream + copy that was previously removed must come back. Because the tree is rsync'd, + `--update` re-rsyncs the tree (with `--delete`) before pulling and re- + shadowing, so any previously-removed upstream package reappears and only + currently-duplicated packages get shadowed again. The personal subtrees are + EXCLUDED from that rsync so `--delete` never wipes them. Today the tool does none of this; setup is a separate hand-run script written for the -current git repo (`slackrepo_setup_SBo-danix.sh`), whose `git rm` @@ -29,9 +35,12 @@ In scope: - `--setup-repo`: rm -rf the tree root, rsync fresh, clone the two personal repos in, shadow upstream duplicates. Standalone exit mode (like `--init-base`), never combined with a build. -- `--update`: git pull each personal subtree in place, shadow. Runs inline - before a normal build. +- `--update`: rsync the tree (with `--delete`, personal subtrees excluded) to + restore/refresh upstream and unshadow, git pull each personal subtree, shadow. + Runs inline before a normal build. - A shared `shadow_scan` routine (interactive confirm, `rm -r`, no git). +- A shared `rsync_tree` routine that rsyncs the SBo tree with the personal + subtrees excluded from transfer and deletion. - New config: `SBO_RSYNC_URL`, `SBO_PERSONAL_TREES` (assoc array dir -> URL). - Self-check for the shadow-detection logic. - README + CLAUDE.md docs. @@ -57,10 +66,12 @@ declare -A SBO_PERSONAL_TREES=() `config.example` (real values): ```bash -# rsync source for the SBo tree. --setup-repo rm -rf's SBO_TREE_ROOTS[0] and -# rsyncs this into it fresh (same source sbopkg uses). Empty disables -# --setup-repo's rsync step (it will refuse to run). Package SOURCE for the -# tree only; unrelated to LOCAL_MIRROR_15 (the NFS package mirror). +# rsync source for the SBo tree (same source sbopkg uses). --setup-repo rm -rf's +# SBO_TREE_ROOTS[0] and rsyncs this into it fresh; --update rsyncs it with +# --delete (personal subtrees excluded) to refresh upstream and unshadow. +# Empty: --setup-repo refuses to run; --update skips the rsync and only pulls + +# shadows. Package SOURCE for the tree only; unrelated to LOCAL_MIRROR_15 (the +# NFS package mirror). SBO_RSYNC_URL="slackbuilds.org::slackbuilds/15.0" # Personal SBo subtrees: directory name (under SBO_TREE_ROOTS[0]) -> git clone @@ -101,9 +112,9 @@ Function `setup_repo`. Runs its own preconditions since the tree may not exist: 4. Destructive confirm: print `root` and warn it will be removed and rsynced fresh; `read -rp "Proceed? [y/N]"`. Abort on anything but y/Y. 5. `rm -rf "$root"`. -6. `rsync -a --delete --no-owner --no-group "$SBO_RSYNC_URL/" "$root/"`. - Flags match sbopkg's own rsync (keeps files root:root, avoids ownership - churn). Hard-fail on rsync nonzero. +6. `rsync_tree "$root"` (see below). On a clean-slate setup the personal dirs + do not exist yet, so the excludes are harmless no-ops; the same helper is + reused so both modes rsync identically. Hard-fail on rsync nonzero. 7. For each `dir` in `SBO_PERSONAL_TREES` keys: `git clone "${SBO_PERSONAL_TREES[$dir]}" "$root/$dir"`. Hard-fail on clone nonzero (a half-set-up tree is worse than none). @@ -117,19 +128,46 @@ No mkhint. No build. Function `update_personal_trees`. Assumes the tree already exists: 1. `root="${SBO_TREE_ROOTS[0]}"`. -2. For each `dir` in `SBO_PERSONAL_TREES` keys: +2. `rsync_tree "$root"` (see below). This is the UNSHADOW step: `--delete` + restores any upstream package removed in a previous shadow, and refreshes + upstream to current. The personal subtrees are excluded, so `--delete` never + touches them. Requires `SBO_RSYNC_URL`; if empty, warn and skip the rsync + (fall through to pull + shadow so `--update` still refreshes personal repos). + On rsync nonzero: warn and continue (fail-soft; do not kill a build session). +3. For each `dir` in `SBO_PERSONAL_TREES` keys: - `d="$root/$dir"`. If `! -d "$d/.git"`: warn "not a git clone, skipping" and continue (fail-soft; do not abort the whole run). - `git -C "$d" pull`. On nonzero: warn and continue (fail-soft; a pull failure should not kill a build session). -3. `shadow_scan "$root"`. -4. Return; `main` continues into the normal build. +4. `shadow_scan "$root"`. +5. Return; `main` continues into the normal build. Plain `git pull` (subtrees are read-only mirrors, no local commits, nothing to rebase or fast-forward-guard; a merge can never happen because the working tree is never modified locally). Documented so no one "improves" it to `--ff-only` without cause. +## `rsync_tree` (shared) + +Signature: `rsync_tree <root>`. rsyncs the SBo tree, protecting the personal +subtrees: + +```bash +rsync -a --delete --no-owner --no-group \ + <one --exclude=/<dir>/ per key in SBO_PERSONAL_TREES> \ + "$SBO_RSYNC_URL/" "$root/" +``` + +- `-a --delete --no-owner --no-group` match sbopkg's own rsync (files root:root, + no ownership churn, tree made to match upstream exactly). +- One `--exclude=/<dir>/` per `SBO_PERSONAL_TREES` key, built from the actual + keys (not literals). The leading `/` anchors each exclude to the tree root, so + a personal dir is skipped for BOTH transfer and deletion. Without this, + `--delete` would wipe the personal clones (they are not on the rsync source). + This is the critical safety property. +- Returns rsync's exit status; callers decide hard-fail (setup) vs warn + (update). + ## `shadow_scan` (shared) Adapted from `slackrepo_setup_SBo-danix.sh` lines 38-80, with two changes for @@ -171,9 +209,13 @@ in both `--setup-repo` and `--update`. - `--setup-repo`: rsync and clone failures are hard (exit nonzero) — a partially built tree is unusable. The destructive `rm -rf` is gated behind an interactive confirm. -- `--update`: git-pull and non-git-dir problems are fail-soft (warn, continue), - so a transient GitHub hiccup does not abort a build session. Shadow's `rm -r` - is gated behind the confirm. +- `--update`: the rsync, git-pull, and non-git-dir problems are all fail-soft + (warn, continue), so a transient network hiccup does not abort a build + session. An empty `SBO_RSYNC_URL` in `--update` skips the rsync (no unshadow) + and proceeds to pull + shadow. Shadow's `rm -r` is gated behind the confirm. +- Personal-subtree safety: `rsync_tree` excludes every personal dir from + `--delete`, so the rsync can never wipe a personal clone. This is a hard + invariant, not a nicety. - Missing config (`SBO_RSYNC_URL` for setup) fails fast with the config hint. ## Testing @@ -187,10 +229,18 @@ in both `--setup-repo` and `--update`. personal trees, and does NOT invent a match for `nomatch`. - Assert a personal subdir without a `.SlackBuild` is ignored. -rsync/clone/pull/rm are I/O against external tools, no stub (mirrors the -existing `update_base` / `lint_pkg` precedent: pure host I/O, no branch logic -worth a fake). `collect_shadows` is the only branchy piece, so it is the only -thing tested. +Plus one case for the rsync exclude construction (the personal-subtree safety +invariant), without running rsync: +- Factor the `--exclude=/<dir>/` list into a pure helper + `rsync_excludes` that echoes the flags from `SBO_PERSONAL_TREES` keys. +- Assert it emits `--exclude=/personal/` and `--exclude=/pentesting/` for the + two-key config, and nothing for an empty config. +This guards the invariant that `--delete` can never wipe a personal clone. + +rsync/clone/pull/rm themselves are I/O against external tools, no stub (mirrors +the existing `update_base` / `lint_pkg` precedent: pure host I/O, no branch +logic worth a fake). `collect_shadows` and `rsync_excludes` are the branchy/ +constructed pieces, so they are what gets tested. Self-check must stay runnable as a normal user: the fake tree is built under a temp dir, no root, no real rsync/git. @@ -202,10 +252,11 @@ temp dir, no root, no real rsync/git. (first-time) and `--update` (per-run), the shadow behavior, and the config vars. Note the rsync source is public slackbuilds.org, distinct from the NFS mirror. -- `CLAUDE.md`: add `setup_repo`, `update_personal_trees`, `shadow_scan`, - `collect_shadows` to the architecture list; add the two config vars to the - CONFIG-block description; add a verified/not-verified note (shadow logic - self-checked; rsync/clone/pull not yet run on hardware). +- `CLAUDE.md`: add `setup_repo`, `update_personal_trees`, `rsync_tree`, + `shadow_scan`, `collect_shadows` to the architecture list; add the two config + vars to the CONFIG-block description; document the unshadow-via-rsync behavior + and the personal-exclude safety invariant; add a verified/not-verified note + (shadow logic self-checked; rsync/clone/pull not yet run on hardware). ## What is NOT verified until hardware |