From d71a10b8a10e04d9a1cd5683034f3f94d4a81a3a Mon Sep 17 00:00:00 2001 From: "Danilo M." Date: Fri, 26 Jun 2026 13:33:50 +0200 Subject: mkwheels: add gh source mode (pypi/gh subcommands) Vendor GitHub source releases that are not on PyPI (e.g. NetExec, which also pulls git deps). New flag-based CLI with pypi/gh mode selectors: mkwheels pypi --name PKG --ver VER [--epoch N] mkwheels gh --repo OWNER/REPO --ver VER [--name PKG] [--tag TAG] [--epoch N] gh mode downloads the tagged source and uses `pip wheel` to build the project plus its whole dependency tree (PyPI + git deps) into wheels; `pip download ` is wrong for a local source since it only resolves metadata. Epoch auto-derives from the release published_at. selftest now covers both modes. Co-Authored-By: Claude Opus 4.8 --- README.md | 55 ++++++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 42 insertions(+), 13 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index 8823a3b..530a85c 100644 --- a/README.md +++ b/README.md @@ -1,20 +1,42 @@ # mkwheels Build a reproducible, pinned Python wheels tarball for vendoring into a -SlackBuild (or any offline `pip install`). Generic over package + version. +SlackBuild (or any offline `pip install`). Generic over package + version, with +two source modes: PyPI packages and GitHub source releases. ## Usage ``` -mkwheels [epoch] +mkwheels pypi --name PKG --ver VER [--epoch N] +mkwheels gh --repo OWNER/REPO --ver VER [--name PKG] [--tag TAG] [--epoch N] ``` -- ` ` — the PyPI package and exact version to vendor. -- `[epoch]` — optional `SOURCE_DATE_EPOCH`. Omitted → auto-derived from the - PyPI release upload time (a warning is printed). Pass it to override. +Common flags: + +- `--ver VER` — version for the output filename. A leading `v` is stripped. +- `--epoch N` — optional `SOURCE_DATE_EPOCH`. Omitted → auto-derived (see each + mode). Pass it to override. - `OUTPUT` env var overrides the output directory (default: current dir). -Outputs `-wheels-.tar.gz` and `requirements.txt` (pinned + hashed). +### pypi mode + +- `--name PKG` — the PyPI package, downloaded at exactly `--ver`. +- Epoch auto-derived from the PyPI release upload time. + +### gh mode + +For packages not on PyPI (GitHub source, possibly with git dependencies). The +tagged source is downloaded and pip builds the project plus its whole +dependency tree (PyPI deps and any `git+` deps) into wheels. + +- `--repo OWNER/REPO` — the GitHub repository. +- `--name PKG` — output name; defaults to the repo basename, lowercased. +- `--tag TAG` — git tag to fetch; a leading `v` is stripped for naming, and the + real ref is resolved by trying `` then `v`. Defaults to `--ver`. +- Epoch auto-derived from the GitHub release `published_at` (the repo must + publish a GitHub Release for the tag). + +Outputs `-wheels-.tar.gz` and `requirements.txt` (pinned + hashed). Prints the md5sum and the resolved epoch. The `requirements.txt` is an audit record of the resolved versions, not the install input: the SlackBuild installs straight from the wheel files (`--find-links`), it does not re-resolve the @@ -30,21 +52,28 @@ PyPI releases are immutable, so the wheel set for a fixed version is deterministic. The tarball normalizes tar metadata (sorted entries, fixed mtime/owner, `gzip -n`) so it is byte-identical for the same inputs + epoch. -Git-sourced dependencies (packages whose upstream pins a git URL) are frozen -at download time: `pip download` resolves whatever is current, and the emitted -`requirements.txt` records the exact resolved versions. Once built, the -tarball is the source of truth. +In `gh` mode the project (and any source-only deps) are built from source. +With a fixed epoch this is byte-identical on the same machine, which is what +vendoring needs: build once, upload, pin the md5. Wheels containing compiled +extensions may differ across machines/toolchains, so build the vendored tarball +on the target platform. + +Git-sourced dependencies (packages whose upstream pins a git URL) are frozen at +build time: pip resolves whatever is current, and the emitted `requirements.txt` +records the exact resolved versions. Once built, the tarball is the source of +truth. ## SBo integration -Run `mkwheels `, upload the tarball to your package host, and set +Run `mkwheels`, upload the tarball to your package host, and set `DOWNLOAD_x86_64` / `MD5SUM_x86_64` in the SlackBuild `.info` to point at it. The SlackBuild then `pip install --no-index --find-links=` into a venv. ## Test -`./selftest` builds `six` twice with a fixed epoch and asserts the two wheels -tarballs are byte-identical. Run it after changing the tar/packing logic. +`./selftest` builds twice with a fixed epoch in both modes (`pypi` six, +`gh` pyparsing) and asserts each pair of wheels tarballs is byte-identical. Run +it after changing the tar/packing or mode-resolution logic. ## License -- cgit v1.2.3