docs: add mkwheels implementation plan

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

1 files changed, 479 insertions, 0 deletions

diff --git a/docs/superpowers/plans/2026-06-26-mkwheels.md b/docs/superpowers/plans/2026-06-26-mkwheels.md
new file mode 100644
index 0000000..6197431
--- /dev/null
+++ b/docs/superpowers/plans/2026-06-26-mkwheels.md
@@ -0,0 +1,479 @@
+# mkwheels Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** A standalone bash CLI that produces a reproducible, pinned Python wheels tarball (`<pkg>-wheels-<ver>.tar.gz`) plus a hashed `requirements.txt` for vendoring into SlackBuilds.
+
+**Architecture:** Single bash script drives a throwaway `python3 -m venv` + `pip download` to resolve a package's full dependency tree into wheels, emits a hashed lockfile, and packs the wheels into a byte-reproducible tarball (normalized tar metadata + gzip `-n`, mtime pinned to a SOURCE_DATE_EPOCH that defaults to the PyPI release upload time). A selftest builds `six` twice and asserts the tarballs are byte-identical.
+
+**Tech Stack:** bash, python3 + pip, jq, curl, tar/gzip/md5sum. GPLv2 (v2-only).
+
+---
+
+## File Structure
+
+```
+~/Programming/GIT/mkwheels/
+├── mkwheels          # the script (single-file bash, executable)
+├── selftest          # reproducibility check (bash)
+├── LICENSE           # GPLv2 full text
+├── README.md         # usage, reproducibility rationale, SBo integration
+└── .gitignore        # ignore scratch output (*.tar.gz, requirements.txt at root)
+```
+
+Responsibilities:
+- `mkwheels` — the whole CLI: arg parse, epoch resolution, venv+download, lockfile, reproducible tar.
+- `selftest` — runs `mkwheels six <ver>` twice, asserts md5 of the two tarballs match.
+- `LICENSE` / `README.md` — licensing + docs per global preference.
+
+The script is small enough to stay one file; the selftest is separated so the
+tool itself carries no test scaffolding.
+
+---
+
+### Task 1: Repo scaffolding (LICENSE, .gitignore, README skeleton)
+
+**Files:**
+- Create: `LICENSE`
+- Create: `.gitignore`
+- Create: `README.md`
+
+- [ ] **Step 1: Add the GPLv2 LICENSE**
+
+Copy the official GPLv2 text into `LICENSE`. Fetch it (already cached at
+`/tmp/.../scratchpad/gpl-2.0.txt` during planning; re-fetch if absent):
+
+```bash
+curl -fsSL https://www.gnu.org/licenses/old-licenses/gpl-2.0.txt -o LICENSE
+head -2 LICENSE   # "GNU GENERAL PUBLIC LICENSE / Version 2, June 1991"
+```
+
+- [ ] **Step 2: Add .gitignore**
+
+```
+# scratch output from running mkwheels in the repo root
+/*.tar.gz
+/requirements.txt
+```
+
+- [ ] **Step 3: Add README skeleton**
+
+`README.md`:
+
+````markdown
+# mkwheels
+
+Build a reproducible, pinned Python wheels tarball for vendoring into a
+SlackBuild (or any offline `pip install`). Generic over package + version.
+
+## Usage
+
+```
+mkwheels <pkg> <ver> [epoch]
+```
+
+- `<pkg> <ver>` — the PyPI package and exact version to vendor.
+- `[epoch]` — optional `SOURCE_DATE_EPOCH`. Omitted → auto-derived from the
+  PyPI release upload time (a warning is printed). Pass it to override.
+- `OUTPUT` env var overrides the output directory (default: current dir).
+
+Outputs `<pkg>-wheels-<ver>.tar.gz` and `requirements.txt` (pinned + hashed).
+Prints the md5sum and the resolved epoch.
+
+## Requirements
+
+`bash`, `python3` + `pip`, `jq`, `curl`, `tar`, `gzip`, `md5sum`.
+
+## Reproducibility
+
+PyPI releases are immutable, so the wheel set for a fixed version is
+deterministic. The tarball normalizes tar metadata (sorted entries, fixed
+mtime/owner, `gzip -n`) so it is byte-identical for the same inputs + epoch.
+
+Git-sourced dependencies (packages whose upstream pins a git URL) are frozen
+at download time: `pip download` resolves whatever is current, and the emitted
+`requirements.txt` records the exact resolved versions. Once built, the
+tarball is the source of truth.
+
+## SBo integration
+
+Run `mkwheels <pkg> <ver>`, upload the tarball to your package host, and set
+`DOWNLOAD_x86_64` / `MD5SUM_x86_64` in the SlackBuild `.info` to point at it.
+The SlackBuild then `pip install --no-index --find-links=<wheels>` into a venv.
+
+## License
+
+GPLv2 (v2-only). See `LICENSE`. Copyright (C) 2026 Danilo M. <danix@danix.xyz>.
+````
+
+- [ ] **Step 4: Commit**
+
+```bash
+git add LICENSE .gitignore README.md
+git commit -S -m "mkwheels: add LICENSE, gitignore, README skeleton"
+```
+
+---
+
+### Task 2: Script skeleton — header, usage, arg parse, tool checks
+
+**Files:**
+- Create: `mkwheels`
+
+- [ ] **Step 1: Write the script skeleton**
+
+Create `mkwheels`, `chmod +x` it after. Content:
+
+```bash
+#!/bin/bash
+# mkwheels — build a reproducible, pinned Python wheels tarball for a package.
+#
+# Copyright (C) 2026 Danilo M. <danix@danix.xyz>
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 2 as published by
+# the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program; if not, see <https://www.gnu.org/licenses/>.
+set -eu
+
+usage() {
+    cat <<EOF
+usage: ${0##*/} <pkg> <ver> [epoch]
+
+Build a reproducible pinned Python wheels tarball <pkg>-wheels-<ver>.tar.gz
+plus a hashed requirements.txt, for vendoring into a SlackBuild.
+
+  <pkg> <ver>  PyPI package name and exact version to vendor.
+  [epoch]      SOURCE_DATE_EPOCH for the tarball mtime. Omitted -> auto-derived
+               from the PyPI release upload time (a warning is printed).
+
+  OUTPUT       env var: output directory (default: current dir).
+
+Requires: python3+pip, jq, curl, tar, gzip, md5sum.
+EOF
+}
+
+case "${1:-}" in
+    -h|--help) usage; exit 0 ;;
+esac
+[ $# -ge 2 ] && [ $# -le 3 ] || { usage >&2; exit 2; }
+
+pkg=$1
+ver=$2
+epoch=${3:-}
+OUTPUT=${OUTPUT:-$PWD}
+
+# Check required tools up front.
+for tool in python3 jq curl tar gzip md5sum; do
+    command -v "$tool" >/dev/null 2>&1 || {
+        echo "error: required tool not found: $tool" >&2
+        exit 1
+    }
+done
+python3 -m pip --version >/dev/null 2>&1 || {
+    echo "error: python3 pip module not available" >&2
+    exit 1
+}
+
+echo "mkwheels: $pkg $ver -> $OUTPUT/$pkg-wheels-$ver.tar.gz"
+```
+
+- [ ] **Step 2: Verify usage and arg validation**
+
+```bash
+chmod +x mkwheels
+./mkwheels -h            # prints usage, exit 0
+./mkwheels; echo $?      # usage to stderr, exit 2
+./mkwheels onlyone; echo $?   # exit 2
+```
+Expected: `-h` prints usage; no-arg and one-arg print usage and exit 2.
+
+- [ ] **Step 3: Commit**
+
+```bash
+git add mkwheels
+git commit -S -m "mkwheels: add script skeleton with arg parse and tool checks"
+```
+
+---
+
+### Task 3: Epoch resolution from PyPI
+
+**Files:**
+- Modify: `mkwheels` (append epoch-resolution block after the tool checks)
+
+- [ ] **Step 1: Add the epoch resolution block**
+
+Insert after the `echo "mkwheels: ..."` line:
+
+```bash
+# Resolve SOURCE_DATE_EPOCH. Explicit arg wins; otherwise derive it from the
+# earliest file upload time of this version on PyPI (a real, reproducible,
+# release-tied timestamp).
+if [ -z "$epoch" ]; then
+    meta=$(curl -fsSL "https://pypi.org/pypi/$pkg/$ver/json") || {
+        echo "error: cannot fetch PyPI metadata for $pkg $ver" >&2
+        exit 1
+    }
+    iso=$(printf '%s' "$meta" \
+        | jq -r '[.urls[].upload_time_iso_8601] | sort | .[0] // empty')
+    [ -n "$iso" ] || {
+        echo "error: no upload time found for $pkg $ver on PyPI" >&2
+        exit 1
+    }
+    epoch=$(date -u -d "$iso" +%s)
+    echo "warning: epoch not given; using PyPI upload time $iso (epoch $epoch)" >&2
+fi
+export SOURCE_DATE_EPOCH="$epoch"
+```
+
+- [ ] **Step 2: Verify epoch derivation against a known release**
+
+```bash
+# six 1.16.0 was uploaded 2021-05-05; check we get a stable epoch and warning.
+./mkwheels six 1.16.0 2>&1 | grep -i "using PyPI upload time"
+```
+Expected: a warning line naming a 2021 ISO timestamp and an epoch integer.
+(The run will continue past this point only once Task 4 is implemented; for
+now it is fine if it errors after printing the warning.)
+
+- [ ] **Step 3: Verify explicit epoch suppresses the warning**
+
+```bash
+./mkwheels six 1.16.0 1620000000 2>&1 | grep -i "using PyPI" && echo UNEXPECTED || echo "ok: no auto-derive"
+```
+Expected: `ok: no auto-derive`.
+
+- [ ] **Step 4: Commit**
+
+```bash
+git add mkwheels
+git commit -S -m "mkwheels: resolve SOURCE_DATE_EPOCH from PyPI upload time"
+```
+
+---
+
+### Task 4: Download wheels + emit hashed requirements.txt
+
+**Files:**
+- Modify: `mkwheels` (append download + lockfile block)
+
+- [ ] **Step 1: Add the temp workdir, venv, download, and lockfile block**
+
+Insert after the epoch block:
+
+```bash
+# Throwaway workdir, cleaned on exit.
+work=$(mktemp -d)
+trap 'rm -rf "$work"' EXIT
+
+wheels="$work/wheels"
+mkdir -p "$wheels"
+
+# Isolated build env so host pip config / installed pkgs don't leak in.
+python3 -m venv "$work/venv"
+"$work/venv/bin/pip" install --quiet --upgrade pip wheel >/dev/null
+
+# Resolve the full tree into $wheels (sdists are built to wheels).
+"$work/venv/bin/pip" download "$pkg==$ver" --dest "$wheels"
+
+# Emit a pinned, hashed requirements.txt from the downloaded files. Each
+# distribution is pinned to its version with a sha256 hash per file.
+req="$work/requirements.txt"
+: > "$req"
+for f in "$wheels"/*; do
+    base=$(basename "$f")
+    # name-version from the wheel/sdist filename: split on first two '-' fields
+    # wheels: name-version-...; sdists: name-version.tar.gz
+    name=${base%%-*}
+    rest=${base#*-}
+    version=${rest%%-*}
+    version=${version%.tar.gz}
+    hash=$(python3 -c "import hashlib,sys;print(hashlib.sha256(open(sys.argv[1],'rb').read()).hexdigest())" "$f")
+    printf '%s==%s --hash=sha256:%s\n' "$name" "$version" "$hash" >> "$req"
+done
+sort -o "$req" "$req"
+```
+
+- [ ] **Step 2: Verify a small download produces wheels + a hashed lockfile**
+
+```bash
+./mkwheels six 1.16.0 1620000000 2>/dev/null || true
+# (no output yet — tarball step is Task 5; this just checks it runs without error)
+echo $?
+```
+Expected: exit 0 (download + lockfile build succeed; six is pure-python with
+no deps, so exactly one entry would be produced internally).
+
+- [ ] **Step 3: Commit**
+
+```bash
+git add mkwheels
+git commit -S -m "mkwheels: download wheels and emit hashed requirements.txt"
+```
+
+---
+
+### Task 5: Reproducible tar + final output
+
+**Files:**
+- Modify: `mkwheels` (append tar/output block)
+
+- [ ] **Step 1: Add the reproducible tar and output block**
+
+Insert after the lockfile block:
+
+```bash
+mkdir -p "$OUTPUT"
+tarball="$OUTPUT/$pkg-wheels-$ver.tar.gz"
+
+# Reproducible archive: sorted entries, normalized ownership/mtime, gzip -n.
+# Run from $work so the archive holds a top-level 'wheels/' dir.
+( cd "$work" \
+  && find wheels -print0 | LC_ALL=C sort -z \
+     | tar --null --files-from=- \
+           --mtime="@$SOURCE_DATE_EPOCH" \
+           --owner=0 --group=0 --numeric-owner \
+           --no-recursion -cf - \
+     | gzip -n > "$tarball" )
+
+cp "$work/requirements.txt" "$OUTPUT/requirements.txt"
+
+md5=$(md5sum "$tarball" | cut -d' ' -f1)
+echo "wheels tarball: $tarball"
+echo "requirements:   $OUTPUT/requirements.txt"
+echo "epoch:          $SOURCE_DATE_EPOCH"
+echo "md5sum:         $md5"
+```
+
+- [ ] **Step 2: Verify a full run emits tarball, lockfile, md5**
+
+```bash
+cd /tmp && OUTPUT=/tmp/mkw-test /home/danix/Programming/GIT/mkwheels/mkwheels six 1.16.0 1620000000
+ls -l /tmp/mkw-test/six-wheels-1.16.0.tar.gz /tmp/mkw-test/requirements.txt
+```
+Expected: both files exist; final output prints `md5sum: <hex>` and the
+resolved epoch `1620000000`.
+
+- [ ] **Step 3: Verify the lockfile is pinned + hashed**
+
+```bash
+cat /tmp/mkw-test/requirements.txt
+```
+Expected: a line like `six==1.16.0 --hash=sha256:<64 hex>`.
+
+- [ ] **Step 4: Commit**
+
+```bash
+git add mkwheels
+git commit -S -m "mkwheels: pack reproducible tarball and print md5"
+```
+
+---
+
+### Task 6: selftest — byte-identical reproducibility check
+
+**Files:**
+- Create: `selftest`
+
+- [ ] **Step 1: Write the selftest**
+
+Create `selftest`, `chmod +x` after:
+
+```bash
+#!/bin/bash
+# selftest — build six twice and assert the wheels tarballs are byte-identical.
+# The smallest check that fails if the reproducible-tar normalization breaks.
+set -eu
+
+here=$(cd "$(dirname "$0")" && pwd)
+tmp=$(mktemp -d)
+trap 'rm -rf "$tmp"' EXIT
+
+# Fixed epoch so both runs use the same mtime (we are testing tar determinism,
+# not epoch derivation).
+OUTPUT="$tmp/a" "$here/mkwheels" six 1.16.0 1620000000 >/dev/null
+OUTPUT="$tmp/b" "$here/mkwheels" six 1.16.0 1620000000 >/dev/null
+
+a=$(md5sum "$tmp/a/six-wheels-1.16.0.tar.gz" | cut -d' ' -f1)
+b=$(md5sum "$tmp/b/six-wheels-1.16.0.tar.gz" | cut -d' ' -f1)
+
+if [ "$a" = "$b" ]; then
+    echo "PASS: reproducible ($a)"
+else
+    echo "FAIL: tarballs differ ($a != $b)" >&2
+    exit 1
+fi
+```
+
+- [ ] **Step 2: Run the selftest**
+
+```bash
+chmod +x selftest
+./selftest
+```
+Expected: `PASS: reproducible (<md5>)`.
+
+- [ ] **Step 3: Commit**
+
+```bash
+git add selftest
+git commit -S -m "mkwheels: add selftest asserting byte-identical tarballs"
+```
+
+---
+
+### Task 7: README selftest note + final review
+
+**Files:**
+- Modify: `README.md` (add a Test section)
+
+- [ ] **Step 1: Add a Test section to the README**
+
+Append to `README.md`:
+
+````markdown
+## Test
+
+`./selftest` builds `six` twice with a fixed epoch and asserts the two wheels
+tarballs are byte-identical. Run it after changing the tar/packing logic.
+````
+
+- [ ] **Step 2: Commit**
+
+```bash
+git add README.md
+git commit -S -m "mkwheels: document selftest in README"
+```
+
+---
+
+## Self-Review notes
+
+- **Spec coverage:** interface (`<pkg> <ver> [epoch]`, OUTPUT) → Task 2;
+  epoch auto-derive from PyPI upload_time + warning → Task 3; venv + pip
+  download full tree → Task 4; hashed requirements.txt → Task 4; reproducible
+  tar (sorted, mtime/owner, gzip -n) → Task 5; error handling (set -eu, trap,
+  clear failures) → Tasks 2/3/4; jq used for JSON → Task 3; selftest on six →
+  Task 6; GPLv2 LICENSE + header + README License section → Tasks 1/2.
+- **Out-of-scope items** (arbitrary requirements.txt input, uploading,
+  private indexes, caching) are intentionally not in any task.
+- **Name consistency:** `$work`, `$wheels`, `$req`, `$tarball`, `$epoch`,
+  `SOURCE_DATE_EPOCH`, `OUTPUT` used consistently across Tasks 2–6.
+
+## Caveat to verify during execution
+
+The `requirements.txt` name/version parse in Task 4 splits on `-` from the
+filename. PEP 503 normalizes some names (underscores vs hyphens); for the
+netexec tree most filenames are well-behaved, but if a wheel filename's
+project name itself contains a hyphen-mapped char the parsed `name` may differ
+from the canonical PyPI name. This only affects the cosmetic lockfile, not the
+tarball contents or the build (the SlackBuild installs from the wheel files
+directly, not by re-resolving the lockfile). If it matters, switch the parse
+to read `WHEEL`/`METADATA` instead. Noted, not blocking.