1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
|
# sb-builder: bundled deps (phase 4b, part 2)
Date: 2026-10-20
Status: approved
Builds on: `2026-10-20-sb-builder-phase4-design.md`, `2026-10-20-sb-builder-phantom-deps-design.md`
## Goal
`hint-update` handles packages with several downloads, so mkhint's bundled-dep
code (url mode) is no longer needed and part 3 can remove mkhint from the VM.
## Facts
- 64 packages are tracked in `nvchecker.toml`; 4 have more than one download
(686 in the whole tree):
- neovim, 13 extra: upstream lists them in `cmake.deps/deps.txt`
(`NAME_URL <url>` lines), at a path fixed per tag.
- openvino, 9 extra: git submodules pinned by commit. Lives in the personal
overlay, its `.info` is edited there; not hint-managed.
- feroxbuster, 1 extra: a self-hosted vendor tarball carrying the main
version (`feroxbuster-<ver>-vendor.tar.gz`).
- exploitdb, 1 extra: `exploitdb-bin-sploits`, its own version, unchanged
since 2022.
- `apply_version` swaps the version through the whole hint text but
downloads only the first URL of each `DOWNLOAD*` for its md5; other URLs
keep their old md5. A feroxbuster bump today writes the new vendor URL
with the old md5.
- mkhint's url mode (`/usr/local/bin/mkhint`, `reconcile_bundle_deps_url` and
helpers, about 200 lines of bash) fetches the manifest at the new version,
takes the `_URL` lines, matches each extra hint URL to one, compares
versions and swaps the changed ones, downloading them for the md5.
## Scope
In: md5 for every changed URL (all packages), manifest reconcile (packages
in `[manifests]`).
Out: git submodule mode (openvino, not hint-managed), separately versioned
extras with no manifest (exploitdb), a GUI for the manifest map, adding deps
the manifest has and the hint lacks (logged only), part 3 (removing mkhint,
nvchecker and their config from the VM).
## Helper (`sbb-remote` 0.8.0)
### Config
`[manifests]` table, package -> manifest URL template, `{VERSION}` replaced
by the new version. Default empty. Replaces `manifest_packages`, and with it
the refusal of manifest packages by `hint-update` and `hint-new`.
`remote/config.example.toml` shows neovim's:
[manifests]
neovim = "https://raw.githubusercontent.com/neovim/neovim/v{VERSION}/cmake.deps/deps.txt"
The VM's `/etc/sb-builder/config.toml` gets that line by hand.
### `apply_version`, every package
After the version swap, for each `DOWNLOAD*`/`MD5SUM*` pair, download every
URL that differs from the one at the same position before the swap, plus the
first URL as today, and set its md5. Unchanged continuation URLs keep their
md5. `SKIP_URLS` (`UNSUPPORTED`, `UNTESTED`) and `--no-dl` behave as today.
### `apply_version`, packages in `[manifests]`
Runs after the version swap and before the downloads, only when a new
version is given and `--no-dl` is not:
1. Fetch the manifest at the new version.
2. Extract every `https?://` URL from it (regex, any file format).
3. For each extra URL (position 1 and later) of `DOWNLOAD`, find its match:
first by GitHub `owner/repo`, lowercased, skipping the shared blob repo
`neovim/deps`; then by stem, the basename without archive extension and
without a trailing `-<version>` (`[vV]?` digit, or 7+ hex).
4. Version of a URL: basename without archive extension; on GitHub, strip a
leading `<repo>-` or `<repo>_` (case-insensitive) and the rest is the
version; elsewhere, the trailing `-<version>` or `-<sha>`. Then drop a
leading `v` and turn `-` into `_`. The first URL's manifest counterpart is
never reconciled.
5. A match whose version differs replaces the hint URL; the download step
above then gives it its md5.
Log lines: `<pkg>: <dep> <old> -> <new>` per changed dep, `<pkg>: <dep> no
manifest match, left as is`, `<pkg>: manifest has N deps not in the hint`.
### Errors
A manifest fetch failure, or a download failure, fails that package: its
hint and `.bak` stay untouched, `<pkg>: failed: <why>` is logged, the job
moves on, as for the main download today. An empty manifest (no URLs) is a
failure too: a moved file must not pass as "all current".
## GUI
None. The hint diff dialog already shows the changed `DOWNLOAD`/`MD5SUM`
lines.
## Testing
`test_remote.py`, no network (`file://` URLs in a temp dir):
- URL version and match against neovim's real URL shapes: github archive
`v<ver>/<name>-<ver>`, LuaJIT by commit, luv release asset, lpeg off
GitHub, mixed-case owner (JuliaStrings).
- manifest reconcile: one dep changed, one unchanged, one unmatched; the
changed one gets the new URL and md5, the rest keep theirs.
- a continuation URL carrying the main version gets a new md5 without a
manifest (feroxbuster).
- manifest fetch failure leaves the hint untouched.
## Version
0.8.0 in `remote/sbb-remote` and `sb_builder/__init__.py`; deploy with
`./install.sh --remote` after asking.
|
