1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
|
# Repo setup + update: `--setup-repo` and `--update`
Date: 2026-07-11
Status: approved, ready for implementation
## Problem
`sbo-batch-test` builds against a local SBo tree (`SBO_TREE_ROOTS[0]`). On the
build VM that tree is `/var/lib/sbopkg/SBo/15.0`, kept fresh with rsync from
slackbuilds.org (NOT a git repo). The maintainer's own packages live in two
personal subtrees inside that tree, `personal/` and `pentesting/`, cloned from
GitHub. Those subtrees must:
- be created from scratch on first setup (tree rsync'd fresh, repos cloned in),
- be refreshed per run when wanted (git pull in place),
- shadow any same-named upstream package elsewhere in the tree (the personal
version wins, the upstream duplicate is removed),
- UNSHADOW automatically: when a package leaves a personal repo, the upstream
copy that was previously removed must come back. Because the tree is rsync'd,
`--update` re-rsyncs the tree (with `--delete`) before pulling and re-
shadowing, so any previously-removed upstream package reappears and only
currently-duplicated packages get shadowed again. The personal subtrees are
EXCLUDED from that rsync so `--delete` never wipes them.
Today the tool does none of this; setup is a separate hand-run script written
for the -current git repo (`slackrepo_setup_SBo-danix.sh`), whose `git rm`
shadow logic does not apply to the rsync'd 15.0 tree.
This adds two modes: `--setup-repo` (full first-time build of the tree) and
`--update` (per-run refresh), sharing one shadow routine.
## Scope
In scope:
- `--setup-repo`: rm -rf the tree root, rsync fresh, clone the two personal
repos in, shadow upstream duplicates. Standalone exit mode (like
`--init-base`), never combined with a build.
- `--update`: rsync the tree (with `--delete`, personal subtrees excluded) to
restore/refresh upstream and unshadow, git pull each personal subtree, shadow.
Runs inline before a normal build.
- A shared `shadow_scan` routine (interactive confirm, `rm -r`, no git).
- A shared `rsync_tree` routine that rsyncs the SBo tree with the personal
subtrees excluded from transfer and deletion.
- New config: `SBO_RSYNC_URL`, `SBO_PERSONAL_TREES` (assoc array dir -> URL).
- Self-check for the shadow-detection logic.
- README + CLAUDE.md docs.
Out of scope (explicitly):
- No `git subtree` on 15.0 (tree is not git; subtrees are plain clones).
- No `mkhint -F` (that is -current-only territory).
- No touching `LOCAL_MIRROR_15` (the NFS package source) or the overlay base.
`SBO_RSYNC_URL` (public slackbuilds.org) is a distinct thing from the NFS
mirror; the two never interact.
- Resolution stays LOCAL-tree-only (unchanged hard constraint).
## Config
Two new variables. CONFIG block sets empty defaults; `config.example` documents
real values.
CONFIG block (empty defaults, alongside the existing empties):
```bash
SBO_RSYNC_URL=""
declare -A SBO_PERSONAL_TREES=()
```
`config.example` (real values):
```bash
# rsync source for the SBo tree (same source sbopkg uses). --setup-repo rm -rf's
# SBO_TREE_ROOTS[0] and rsyncs this into it fresh; --update rsyncs it with
# --delete (personal subtrees excluded) to refresh upstream and unshadow.
# Empty: --setup-repo refuses to run; --update skips the rsync and only pulls +
# shadows. Package SOURCE for the tree only; unrelated to LOCAL_MIRROR_15 (the
# NFS package mirror).
SBO_RSYNC_URL="slackbuilds.org::slackbuilds/15.0"
# Personal SBo subtrees: directory name (under SBO_TREE_ROOTS[0]) -> git clone
# URL. --setup-repo clones each after the rsync; --update git-pulls each in
# place. Both then shadow any same-named upstream package found elsewhere in the
# tree (the personal version wins, the upstream duplicate is rm -r'd). Empty =
# no personal trees, both modes become no-ops on the subtree/shadow steps.
declare -A SBO_PERSONAL_TREES=(
[personal]="https://github.com/danixland/my-slackbuilds.git"
[pentesting]="https://github.com/danixland/Slackware-Pentesting-Suite.git"
)
```
Note: `SBO_PERSONAL_TREES` is an associative array. The CONFIG block must
`declare -A` it before sourcing the external config so the config's `declare -A`
(or bare assignment) lands as global. Both modes resolve dir names under
`SBO_TREE_ROOTS[0]` only (single-root VM setup, settled).
## Flags and wiring
- `--setup-repo` sets `DO_SETUP=1`. Standalone exit mode. Wired in `main`
BEFORE `validate_env` (like `--init-base`): runs its own checks, does its
work, exits 0. Mutually exclusive with a build; if a target is also given,
`--setup-repo` still just sets up and exits (document this, do not build).
- `--update` sets `DO_UPDATE=1`. Inline, not an exit mode. Wired in `main`
after `validate_env` and near `update_base`, before targets are collected and
run. Falls through to the normal build.
`parse_args` learns both flags; `usage` documents both.
## `--setup-repo` (first-time full build of the tree)
Function `setup_repo`. Runs its own preconditions since the tree may not exist:
1. Require root.
2. `require_config`; refuse if `SBO_RSYNC_URL` is empty (print the config hint).
3. `root="${SBO_TREE_ROOTS[0]}"`. Refuse if empty.
4. Destructive confirm: print `root` and warn it will be removed and rsynced
fresh; `read -rp "Proceed? [y/N]"`. Abort on anything but y/Y.
5. `rm -rf "$root"`.
6. `rsync_tree "$root"` (see below). On a clean-slate setup the personal dirs
do not exist yet, so the excludes are harmless no-ops; the same helper is
reused so both modes rsync identically. Hard-fail on rsync nonzero.
7. For each `dir` in `SBO_PERSONAL_TREES` keys:
`git clone "${SBO_PERSONAL_TREES[$dir]}" "$root/$dir"`. Hard-fail on clone
nonzero (a half-set-up tree is worse than none).
8. `shadow_scan "$root"`.
9. Exit 0.
No mkhint. No build.
## `--update` (per-run refresh)
Function `update_personal_trees`. Assumes the tree already exists:
1. `root="${SBO_TREE_ROOTS[0]}"`.
2. `rsync_tree "$root"` (see below). This is the UNSHADOW step: `--delete`
restores any upstream package removed in a previous shadow, and refreshes
upstream to current. The personal subtrees are excluded, so `--delete` never
touches them. Requires `SBO_RSYNC_URL`; if empty, warn and skip the rsync
(fall through to pull + shadow so `--update` still refreshes personal repos).
On rsync nonzero: warn and continue (fail-soft; do not kill a build session).
3. For each `dir` in `SBO_PERSONAL_TREES` keys:
- `d="$root/$dir"`. If `! -d "$d/.git"`: warn "not a git clone, skipping"
and continue (fail-soft; do not abort the whole run).
- `git -C "$d" pull`. On nonzero: warn and continue (fail-soft; a pull
failure should not kill a build session).
4. `shadow_scan "$root"`.
5. Return; `main` continues into the normal build.
Plain `git pull` (subtrees are read-only mirrors, no local commits, nothing to
rebase or fast-forward-guard; a merge can never happen because the working tree
is never modified locally). Documented so no one "improves" it to `--ff-only`
without cause.
## `rsync_tree` (shared)
Signature: `rsync_tree <root>`. rsyncs the SBo tree, protecting the personal
subtrees:
```bash
rsync -a --delete --no-owner --no-group \
<one --exclude=/<dir>/ per key in SBO_PERSONAL_TREES> \
"$SBO_RSYNC_URL/" "$root/"
```
- `-a --delete --no-owner --no-group` match sbopkg's own rsync (files root:root,
no ownership churn, tree made to match upstream exactly).
- One `--exclude=/<dir>/` per `SBO_PERSONAL_TREES` key, built from the actual
keys (not literals). The leading `/` anchors each exclude to the tree root, so
a personal dir is skipped for BOTH transfer and deletion. Without this,
`--delete` would wipe the personal clones (they are not on the rsync source).
This is the critical safety property.
- Returns rsync's exit status; callers decide hard-fail (setup) vs warn
(update).
## `shadow_scan` (shared)
Adapted from `slackrepo_setup_SBo-danix.sh` lines 38-80, with two changes for
the non-git 15.0 tree: `rm -r` instead of `git rm -r`, and no `git commit`.
Signature: `shadow_scan <root>`.
1. Build `TO_SHADOW=()`:
- Split the shadow-detection into a pure helper `collect_shadows <root>`
that ECHOES the newline-separated list of tree-relative dirs to remove, so
it is unit-testable without touching the filesystem destructively. It only
reads (find), never removes.
- For each personal tree dir present under `root` (from
`SBO_PERSONAL_TREES` keys, skip if the dir is absent): for each immediate
subdir `pkgdir` that has a matching `pkgdir/<pkg>.SlackBuild`, find
same-named dirs elsewhere under `root` at depth 2, excluding every
personal tree path. Emit each as a `root`-relative path.
- Exclusions are built from the actual `SBO_PERSONAL_TREES` keys (not the
literals `personal`/`pentesting`), so adding a third tree Just Works.
2. If `TO_SHADOW` empty: print "No upstream packages match your personal
subtrees. Nothing to shadow." and return.
3. Else: print the list under "The following upstream packages will be shadowed
(rm -r):", one indented per line. `read -rp "Proceed? [y/N]"`. On non-y:
print "Aborted shadowing; no packages were removed." and return. On y: for
each, print "Shadowing: <item>" and `rm -r "$root/$item"`.
Interactive confirm is required (user wants to see removals before they happen)
in both `--setup-repo` and `--update`.
## Precedence and ordering (settled)
- Personal packages win. An upstream package with the same name as one in a
personal tree is the one removed.
- Shadow runs AFTER clone (`--setup-repo`) or pull (`--update`), so the shadow
set reflects freshly-fetched personal packages.
## Error handling
- `--setup-repo`: rsync and clone failures are hard (exit nonzero) — a
partially built tree is unusable. The destructive `rm -rf` is gated behind an
interactive confirm.
- `--update`: the rsync, git-pull, and non-git-dir problems are all fail-soft
(warn, continue), so a transient network hiccup does not abort a build
session. An empty `SBO_RSYNC_URL` in `--update` skips the rsync (no unshadow)
and proceeds to pull + shadow. Shadow's `rm -r` is gated behind the confirm.
- Personal-subtree safety: `rsync_tree` excludes every personal dir from
`--delete`, so the rsync can never wipe a personal clone. This is a hard
invariant, not a nicety.
- Missing config (`SBO_RSYNC_URL` for setup) fails fast with the config hint.
## Testing
`test-logic.sh` gains one case for `collect_shadows`:
- Build a fake tree: `academic/foo`, `network/bar` (upstream), plus
`personal/foo` and `pentesting/bar` (each with a matching `.SlackBuild`), and
a decoy `personal/nomatch` with no upstream twin.
- Assert `collect_shadows <root>` returns exactly `academic/foo` and
`network/bar` (order-insensitive), and does NOT return anything under the
personal trees, and does NOT invent a match for `nomatch`.
- Assert a personal subdir without a `.SlackBuild` is ignored.
Plus one case for the rsync exclude construction (the personal-subtree safety
invariant), without running rsync:
- Factor the `--exclude=/<dir>/` list into a pure helper
`rsync_excludes` that echoes the flags from `SBO_PERSONAL_TREES` keys.
- Assert it emits `--exclude=/personal/` and `--exclude=/pentesting/` for the
two-key config, and nothing for an empty config.
This guards the invariant that `--delete` can never wipe a personal clone.
rsync/clone/pull/rm themselves are I/O against external tools, no stub (mirrors
the existing `update_base` / `lint_pkg` precedent: pure host I/O, no branch
logic worth a fake). `collect_shadows` and `rsync_excludes` are the branchy/
constructed pieces, so they are what gets tested.
Self-check must stay runnable as a normal user: the fake tree is built under a
temp dir, no root, no real rsync/git.
## Docs
- `config.example`: the two new vars, documented as above.
- `README.md`: a "Repository setup and refresh" section covering `--setup-repo`
(first-time) and `--update` (per-run), the shadow behavior, and the config
vars. Note the rsync source is public slackbuilds.org, distinct from the NFS
mirror.
- `CLAUDE.md`: add `setup_repo`, `update_personal_trees`, `rsync_tree`,
`shadow_scan`, `collect_shadows` to the architecture list; add the two config
vars to the CONFIG-block description; document the unshadow-via-rsync behavior
and the personal-exclude safety invariant; add a verified/not-verified note
(shadow logic self-checked; rsync/clone/pull not yet run on hardware).
## What is NOT verified until hardware
- The actual rsync of the tree, the two clones, and a real git pull (network +
root, VM-only). Logic mirrors the reference setup script and sbopkg's rsync
flags. `collect_shadows` is self-checked; the destructive `rm -r` path and the
interactive confirm are not (trivial, no branch logic beyond the confirm).
|
