From 18b0611d3d1917c112672eae40d0a276af1336ea Mon Sep 17 00:00:00 2001 From: "Danilo M." Date: Fri, 26 Jun 2026 13:25:34 +0200 Subject: docs: add GitHub source mode design spec Co-Authored-By: Claude Opus 4.8 --- .../2026-06-26-mkwheels-gh-source-mode-design.md | 93 ++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-26-mkwheels-gh-source-mode-design.md (limited to 'docs/superpowers/specs') diff --git a/docs/superpowers/specs/2026-06-26-mkwheels-gh-source-mode-design.md b/docs/superpowers/specs/2026-06-26-mkwheels-gh-source-mode-design.md new file mode 100644 index 0000000..f2b6a7a --- /dev/null +++ b/docs/superpowers/specs/2026-06-26-mkwheels-gh-source-mode-design.md @@ -0,0 +1,93 @@ +# mkwheels — GitHub source mode + +Date: 2026-06-26 +Status: approved + +## Problem + +mkwheels currently vendors a single PyPI package and its dependency tree into +a reproducible, pinned wheels tarball. Some packages are not on PyPI. NetExec +(the motivating case) ships only from GitHub source and pulls in four git +dependencies (impacket, certipy-ad, oscrypto, pynfsclient). The existing +PyPI-only interface cannot vendor it. + +We want a second mode that takes a GitHub repo + version, downloads the tagged +source, and lets pip resolve the full dependency tree (PyPI + git deps) into +the same reproducible wheels tarball. + +## CLI + +Mode selector first, all options as explicit flags, no positionals. This is a +breaking change to the current positional interface; acceptable because the +only consumer (the netexec SlackBuild) is not yet written. + +``` +mkwheels --pypi --name PKG --ver VER [--epoch N] +mkwheels --gh --repo OWNER/REPO --ver VER [--name PKG] [--tag TAG] [--epoch N] +``` + +### Normalization + +- `--ver` and `--tag` each strip a single leading `v` if present. +- The normalized version is what appears in the output filename, always + without a leading `v`: `-wheels-.tar.gz`. + +### `--gh` defaults + +- `--name` → repo basename, lowercased (e.g. `Pennyw0rth/NetExec` → `netexec`). +- `--tag` → the normalized `--ver`. +- `--epoch` → auto-derived from the GitHub release `published_at` (below). + +## `--gh` flow + +1. **Resolve the release / ref and epoch.** + GET `https://api.github.com/repos//releases/tags/`. + On 404, retry with `v`. Use whichever resolves; remember the real ref + string (`` or `v`). Take `published_at` from the response and + convert to epoch — unless `--epoch` was given, which wins. + - NetExec example: tag normalizes to `1.5.1`; `releases/tags/1.5.1` 404s, + `releases/tags/v1.5.1` resolves; ref is `v1.5.1`. + +2. **Download and unpack the source.** + Fetch `https://github.com//archive/refs/tags/.tar.gz`, + unpack into the throwaway workdir. + +3. **Resolve the tree with pip.** + `pip download --dest "$wheels"`. pip reads the + project's metadata, resolves PyPI deps, and clones+builds the git deps into + wheels. This is the only step that differs from PyPI mode. + +4. **Emit outputs (shared with `--pypi`).** + Generate the pinned, hashed `requirements.txt` from the wheels dir, pack the + normalized reproducible tarball, print epoch + md5. Identical to the current + path. + +## `--pypi` flow + +Unchanged behavior from the current tool: resolve `==` via +`pip download`, auto-derive epoch from PyPI `upload_time_iso_8601` when +`--epoch` is omitted. Only the surface changes: gated behind the `--pypi` +selector and switched from positionals to `--name` / `--ver` / `--epoch`. + +## Shared internals + +The requirements.txt generation and the reproducible tar packing already +operate on a populated wheels directory regardless of how it was filled. Both +modes converge on that directory; no duplication of the packaging logic. + +`requirements.txt` remains an audit record (pinned + hashed), not the install +input — the SlackBuild installs from the wheels via `--no-index --find-links`. + +## Selftest + +Keep the existing `--pypi` reproducibility check (two builds at a fixed epoch +must be byte-identical). Add a `--gh` reproducibility check against a small, +pure-Python, GitHub-tagged package so the run stays fast. Two builds at a fixed +epoch must be byte-identical. + +## Out of scope + +- Caching git clones between runs. +- Private repos / authenticated GitHub access. +- Non-GitHub forges (GitLab, Codeberg, etc.). +- Using requirements.txt as a pip install input. -- cgit v1.2.3