aboutsummaryrefslogtreecommitdiffstats
path: root/docs/design/2026-10-20-sb-builder-bundled-deps-design.md
diff options
context:
space:
mode:
authorDanilo M. <danix@danix.xyz>2026-10-20 20:30:57 +0200
committerDanilo M. <danix@danix.xyz>2026-10-20 20:30:57 +0200
commit0602623806b7c0984544913d663804ed5f54e07f (patch)
treefdd838b21d4dd0775f98760ca11d9e91e92d3745 /docs/design/2026-10-20-sb-builder-bundled-deps-design.md
parentcd9ca18cf12ab24e0d1532d01c0f621b06869545 (diff)
downloadsb-builder-master.tar.gz
sb-builder-master.zip
Docs ready to publish: VM setup, specs in docs/designHEADmaster
README says what the GUI does today and what is planned, which defaults are the author's setup, and how to set up the VM. The specs move to docs/design/; the implementation plans are dropped (working files, now kept out of git). AGENTS.md links slackrepo upstream instead of a local clone path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'docs/design/2026-10-20-sb-builder-bundled-deps-design.md')
-rw-r--r--docs/design/2026-10-20-sb-builder-bundled-deps-design.md115
1 files changed, 115 insertions, 0 deletions
diff --git a/docs/design/2026-10-20-sb-builder-bundled-deps-design.md b/docs/design/2026-10-20-sb-builder-bundled-deps-design.md
new file mode 100644
index 0000000..de6ad1d
--- /dev/null
+++ b/docs/design/2026-10-20-sb-builder-bundled-deps-design.md
@@ -0,0 +1,115 @@
+# sb-builder: bundled deps (phase 4b, part 2)
+
+Date: 2026-10-20
+Status: approved
+Builds on: `2026-10-20-sb-builder-phase4-design.md`, `2026-10-20-sb-builder-phantom-deps-design.md`
+
+## Goal
+
+`hint-update` handles packages with several downloads, so mkhint's bundled-dep
+code (url mode) is no longer needed and part 3 can remove mkhint from the VM.
+
+## Facts
+
+- 64 packages are tracked in `nvchecker.toml`; 4 have more than one download
+ (686 in the whole tree):
+ - neovim, 13 extra: upstream lists them in `cmake.deps/deps.txt`
+ (`NAME_URL <url>` lines), at a path fixed per tag.
+ - openvino, 9 extra: git submodules pinned by commit. Lives in the personal
+ overlay, its `.info` is edited there; not hint-managed.
+ - feroxbuster, 1 extra: a self-hosted vendor tarball carrying the main
+ version (`feroxbuster-<ver>-vendor.tar.gz`).
+ - exploitdb, 1 extra: `exploitdb-bin-sploits`, its own version, unchanged
+ since 2022.
+- `apply_version` swaps the version through the whole hint text but
+ downloads only the first URL of each `DOWNLOAD*` for its md5; other URLs
+ keep their old md5. A feroxbuster bump today writes the new vendor URL
+ with the old md5.
+- mkhint's url mode (`/usr/local/bin/mkhint`, `reconcile_bundle_deps_url` and
+ helpers, about 200 lines of bash) fetches the manifest at the new version,
+ takes the `_URL` lines, matches each extra hint URL to one, compares
+ versions and swaps the changed ones, downloading them for the md5.
+
+## Scope
+
+In: md5 for every changed URL (all packages), manifest reconcile (packages
+in `[manifests]`).
+
+Out: git submodule mode (openvino, not hint-managed), separately versioned
+extras with no manifest (exploitdb), a GUI for the manifest map, adding deps
+the manifest has and the hint lacks (logged only), part 3 (removing mkhint,
+nvchecker and their config from the VM).
+
+## Helper (`sbb-remote` 0.8.0)
+
+### Config
+
+`[manifests]` table, package -> manifest URL template, `{VERSION}` replaced
+by the new version. The default holds neovim's, so the VM needs no new line.
+Replaces `manifest_packages`, and with it the refusal of manifest packages by
+`hint-update` and `hint-new`. `remote/config.example.toml` shows the default:
+
+ [manifests]
+ neovim = "https://raw.githubusercontent.com/neovim/neovim/v{VERSION}/cmake.deps/deps.txt"
+
+`manifest_packages` becomes an unknown key: the deploy drops its line from
+the VM's `/etc/sb-builder/config.toml`, or the helper refuses to start.
+
+### `apply_version`, every package
+
+After the version swap, for each `DOWNLOAD*`/`MD5SUM*` pair, download every
+URL that differs from the one at the same position before the swap, plus the
+first URL as today, and set its md5. Unchanged continuation URLs keep their
+md5. `SKIP_URLS` (`UNSUPPORTED`, `UNTESTED`) and `--no-dl` behave as today.
+
+### `apply_version`, packages in `[manifests]`
+
+Runs after the version swap and before the downloads, only when a new
+version is given and `--no-dl` is not:
+
+1. Fetch the manifest at the new version.
+2. Extract every `https?://` URL from it (regex, any file format).
+3. For each extra URL (position 1 and later) of `DOWNLOAD`, find its match:
+ first by GitHub `owner/repo`, lowercased, skipping the shared blob repo
+ `neovim/deps`; then by stem, the basename without archive extension and
+ without a trailing `-<version>` (`[vV]?` digit, or 7+ hex).
+4. Version of a URL: basename without archive extension; on GitHub, strip a
+ leading `<repo>-` or `<repo>_` (case-insensitive) and the rest is the
+ version; elsewhere, the trailing `-<version>` or `-<sha>`. Then drop a
+ leading `v` and turn `-` into `_`. The first URL's manifest counterpart is
+ never reconciled.
+5. A match whose version differs replaces the hint URL; the download step
+ above then gives it its md5.
+
+Log lines: `<pkg>: <dep> <old> -> <new>` per changed dep, `<pkg>: <dep> no
+manifest match, left as is`, `<pkg>: manifest has N deps not in the hint`.
+
+### Errors
+
+A manifest fetch failure, or a download failure, fails that package: its
+hint and `.bak` stay untouched, `<pkg>: failed: <why>` is logged, the job
+moves on, as for the main download today. An empty manifest (no URLs) is a
+failure too: a moved file must not pass as "all current".
+
+## GUI
+
+None. The hint diff dialog already shows the changed `DOWNLOAD`/`MD5SUM`
+lines.
+
+## Testing
+
+`test_remote.py`, no network (the existing `Served` localhost http server):
+
+- URL version and match against neovim's real URL shapes: github archive
+ `v<ver>/<name>-<ver>`, LuaJIT by commit, luv release asset, lpeg off
+ GitHub, mixed-case owner (JuliaStrings).
+- manifest reconcile: one dep changed, one unchanged, one unmatched; the
+ changed one gets the new URL and md5, the rest keep theirs.
+- a continuation URL carrying the main version gets a new md5 without a
+ manifest (feroxbuster).
+- manifest fetch failure leaves the hint untouched.
+
+## Version
+
+0.8.0 in `remote/sbb-remote` and `sb_builder/__init__.py`; deploy with
+`./install.sh --remote` after asking.