diff options
| -rw-r--r-- | docs/superpowers/specs/2026-06-26-mkwheels-design.md | 115 |
1 files changed, 115 insertions, 0 deletions
diff --git a/docs/superpowers/specs/2026-06-26-mkwheels-design.md b/docs/superpowers/specs/2026-06-26-mkwheels-design.md new file mode 100644 index 0000000..cc424ea --- /dev/null +++ b/docs/superpowers/specs/2026-06-26-mkwheels-design.md @@ -0,0 +1,115 @@ +# mkwheels — Design + +Date: 2026-06-26 +Author: danix <danix@danix.xyz> +License: GPLv2 (v2-only) + +## Goal + +A standalone bash CLI that generates a **reproducible, pinned Python wheels +tarball** for a given package + version. Primary use: vendoring a Python +project's full dependency tree into a SlackBuild so the build installs into a +venv from local wheels with no network (the feroxbuster vendored-crates +pattern, applied to Python). Generic over package/version, so it serves any +future Python SBo, not just netexec. + +## Repo + +`~/Programming/GIT/mkwheels`, its own git repo. GPLv2 (v2-only): full `LICENSE` +text, a per-file header notice `Copyright (C) 2026 Danilo M. <danix@danix.xyz>`, +and a License section in the README. + +## Files + +``` +mkwheels/ +├── mkwheels # the script (single-file bash) +├── LICENSE # GPLv2 full text (from gnu.org) +└── README.md # usage, reproducibility rationale, SBo integration +``` + +## Dependencies + +Runtime tools the script requires (documented in README, checked at startup): + +- `bash` +- `python3` + `pip` (the venv + download engine) +- `jq` (parse the PyPI JSON API) +- `tar`, `gzip`, `md5sum`, `curl` (or `wget`) + +No third-party Python packages; the tool only drives `pip`. + +## Interface + +``` +mkwheels <pkg> <ver> [epoch] +``` + +- `-h` / `--help` → usage. +- `<pkg>` `<ver>` required. +- `[epoch]` optional SOURCE_DATE_EPOCH. If omitted, auto-derived from the + PyPI release upload time (see below) and a warning is printed. Pass it + explicitly to override. +- `OUTPUT` env var overrides the output directory (defaults to `$PWD`), + matching the mksboarchive convention. + +## Flow + +1. Parse args; handle `-h`. Require `<pkg>` and `<ver>`. Check required tools + are on PATH; fail clearly if not. +2. **Resolve epoch.** If `[epoch]` was passed, use it. Otherwise fetch + `https://pypi.org/pypi/<pkg>/<ver>/json`, take the earliest file's + `upload_time_iso_8601`, convert to epoch seconds with `jq` + `date`, and + warn that it was auto-derived. This is a real, reproducible, + package-tied timestamp. +3. Create a throwaway temp workdir; `trap` cleanup on exit. +4. `python3 -m venv` a throwaway build env (isolates pip from the host). +5. `pip download <pkg>==<ver> -d wheels/` — resolves the full tree including + git-sourced deps, building sdists to wheels. +6. Emit `requirements.txt`: every resolved distribution pinned to its exact + version with `--hash=sha256:...` entries computed from the downloaded + files. Pinned + hashed lockfile, shippable into the SlackBuild package dir. +7. **Reproducible tar.** Archive `wheels/` with normalized metadata so the + output is byte-identical for the same inputs + epoch: + - sorted entry order, + - `--mtime=@$EPOCH --owner=0 --group=0 --numeric-owner`, + - gzip `-n` (no embedded timestamp). + Output: `$OUTPUT/<pkg>-wheels-<ver>.tar.gz`. +8. Print the md5sum, the resolved epoch, and the output paths + (tarball + requirements.txt). + +## Reproducibility + +- PyPI releases are immutable, so the wheel set for a fixed version is + deterministic. +- **Git-sourced deps** (e.g. NetExec's impacket / certipy-ad / oscrypto / + pynfsclient) are unversioned upstream; `pip download` freezes whatever is + current at download time. Once the tarball is built it is the source of + truth; re-running months later may resolve newer git deps. The emitted + `requirements.txt` records the exact resolved versions for audit. This is + the same caveat the feroxbuster crates tarball carries. +- Tar normalization (step 7) makes the archive byte-identical for identical + inputs + epoch. + +## Error handling + +- `set -eu`. +- `trap` removes the temp workdir on any exit. +- Fail with a clear message if: a required tool is missing; the package/version + is not found on PyPI (HTTP error or empty JSON); `pip download` fails. + +## Test (one runnable check) + +A `selftest` path (or a small `test` script invoked manually) that runs +mkwheels twice on `six` (tiny, pure-Python, stable) and asserts the two +tarballs are byte-identical (md5 match). This is the smallest check that fails +if the reproducibility normalization breaks. No test framework. + +## Out of scope (YAGNI) + +- Resolving from an arbitrary input requirements.txt (the tool resolves from + `<pkg> <ver>`). +- Uploading the tarball anywhere (the maintainer uploads to + packages.danix.xyz by hand, as with the crates tarball). +- Non-PyPI indexes / private indexes. +- Caching or incremental rebuilds. |
