diff options
| author | Danilo M. <danix@danix.xyz> | 2026-10-20 20:30:57 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-10-20 20:30:57 +0200 |
| commit | 0602623806b7c0984544913d663804ed5f54e07f (patch) | |
| tree | fdd838b21d4dd0775f98760ca11d9e91e92d3745 /docs/design/2026-10-20-sb-builder-bundled-deps-design.md | |
| parent | cd9ca18cf12ab24e0d1532d01c0f621b06869545 (diff) | |
| download | sb-builder-master.tar.gz sb-builder-master.zip | |
README says what the GUI does today and what is planned, which defaults
are the author's setup, and how to set up the VM. The specs move to
docs/design/; the implementation plans are dropped (working files, now
kept out of git). AGENTS.md links slackrepo upstream instead of a local
clone path.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'docs/design/2026-10-20-sb-builder-bundled-deps-design.md')
| -rw-r--r-- | docs/design/2026-10-20-sb-builder-bundled-deps-design.md | 115 |
1 files changed, 115 insertions, 0 deletions
diff --git a/docs/design/2026-10-20-sb-builder-bundled-deps-design.md b/docs/design/2026-10-20-sb-builder-bundled-deps-design.md new file mode 100644 index 0000000..de6ad1d --- /dev/null +++ b/docs/design/2026-10-20-sb-builder-bundled-deps-design.md @@ -0,0 +1,115 @@ +# sb-builder: bundled deps (phase 4b, part 2) + +Date: 2026-10-20 +Status: approved +Builds on: `2026-10-20-sb-builder-phase4-design.md`, `2026-10-20-sb-builder-phantom-deps-design.md` + +## Goal + +`hint-update` handles packages with several downloads, so mkhint's bundled-dep +code (url mode) is no longer needed and part 3 can remove mkhint from the VM. + +## Facts + +- 64 packages are tracked in `nvchecker.toml`; 4 have more than one download + (686 in the whole tree): + - neovim, 13 extra: upstream lists them in `cmake.deps/deps.txt` + (`NAME_URL <url>` lines), at a path fixed per tag. + - openvino, 9 extra: git submodules pinned by commit. Lives in the personal + overlay, its `.info` is edited there; not hint-managed. + - feroxbuster, 1 extra: a self-hosted vendor tarball carrying the main + version (`feroxbuster-<ver>-vendor.tar.gz`). + - exploitdb, 1 extra: `exploitdb-bin-sploits`, its own version, unchanged + since 2022. +- `apply_version` swaps the version through the whole hint text but + downloads only the first URL of each `DOWNLOAD*` for its md5; other URLs + keep their old md5. A feroxbuster bump today writes the new vendor URL + with the old md5. +- mkhint's url mode (`/usr/local/bin/mkhint`, `reconcile_bundle_deps_url` and + helpers, about 200 lines of bash) fetches the manifest at the new version, + takes the `_URL` lines, matches each extra hint URL to one, compares + versions and swaps the changed ones, downloading them for the md5. + +## Scope + +In: md5 for every changed URL (all packages), manifest reconcile (packages +in `[manifests]`). + +Out: git submodule mode (openvino, not hint-managed), separately versioned +extras with no manifest (exploitdb), a GUI for the manifest map, adding deps +the manifest has and the hint lacks (logged only), part 3 (removing mkhint, +nvchecker and their config from the VM). + +## Helper (`sbb-remote` 0.8.0) + +### Config + +`[manifests]` table, package -> manifest URL template, `{VERSION}` replaced +by the new version. The default holds neovim's, so the VM needs no new line. +Replaces `manifest_packages`, and with it the refusal of manifest packages by +`hint-update` and `hint-new`. `remote/config.example.toml` shows the default: + + [manifests] + neovim = "https://raw.githubusercontent.com/neovim/neovim/v{VERSION}/cmake.deps/deps.txt" + +`manifest_packages` becomes an unknown key: the deploy drops its line from +the VM's `/etc/sb-builder/config.toml`, or the helper refuses to start. + +### `apply_version`, every package + +After the version swap, for each `DOWNLOAD*`/`MD5SUM*` pair, download every +URL that differs from the one at the same position before the swap, plus the +first URL as today, and set its md5. Unchanged continuation URLs keep their +md5. `SKIP_URLS` (`UNSUPPORTED`, `UNTESTED`) and `--no-dl` behave as today. + +### `apply_version`, packages in `[manifests]` + +Runs after the version swap and before the downloads, only when a new +version is given and `--no-dl` is not: + +1. Fetch the manifest at the new version. +2. Extract every `https?://` URL from it (regex, any file format). +3. For each extra URL (position 1 and later) of `DOWNLOAD`, find its match: + first by GitHub `owner/repo`, lowercased, skipping the shared blob repo + `neovim/deps`; then by stem, the basename without archive extension and + without a trailing `-<version>` (`[vV]?` digit, or 7+ hex). +4. Version of a URL: basename without archive extension; on GitHub, strip a + leading `<repo>-` or `<repo>_` (case-insensitive) and the rest is the + version; elsewhere, the trailing `-<version>` or `-<sha>`. Then drop a + leading `v` and turn `-` into `_`. The first URL's manifest counterpart is + never reconciled. +5. A match whose version differs replaces the hint URL; the download step + above then gives it its md5. + +Log lines: `<pkg>: <dep> <old> -> <new>` per changed dep, `<pkg>: <dep> no +manifest match, left as is`, `<pkg>: manifest has N deps not in the hint`. + +### Errors + +A manifest fetch failure, or a download failure, fails that package: its +hint and `.bak` stay untouched, `<pkg>: failed: <why>` is logged, the job +moves on, as for the main download today. An empty manifest (no URLs) is a +failure too: a moved file must not pass as "all current". + +## GUI + +None. The hint diff dialog already shows the changed `DOWNLOAD`/`MD5SUM` +lines. + +## Testing + +`test_remote.py`, no network (the existing `Served` localhost http server): + +- URL version and match against neovim's real URL shapes: github archive + `v<ver>/<name>-<ver>`, LuaJIT by commit, luv release asset, lpeg off + GitHub, mixed-case owner (JuliaStrings). +- manifest reconcile: one dep changed, one unchanged, one unmatched; the + changed one gets the new URL and md5, the rest keep theirs. +- a continuation URL carrying the main version gets a new md5 without a + manifest (feroxbuster). +- manifest fetch failure leaves the hint untouched. + +## Version + +0.8.0 in `remote/sbb-remote` and `sb_builder/__init__.py`; deploy with +`./install.sh --remote` after asking. |
