From 59849d458b6bb71ed25efedde03d46acbab04d08 Mon Sep 17 00:00:00 2001 From: "Danilo M." Date: Wed, 1 Jul 2026 09:46:31 +0200 Subject: Install from release zip; fix Passport migration collision Two root causes made the composer-based update produce a broken install: - composer create-project ships source only, without the compiled frontend bundles (public/v1/js/app.js etc.), leaving the UI broken (no graphs, 404s). Switch to downloading and extracting the official GitHub release zip, which is prebuilt. Normalize the version so both "6.6.5" and "v6.6.5" resolve to the correct v-prefixed release asset. - Laravel Passport renames its migration files between versions, so a carried-over DB has the oauth_* tables while the new filenames look pending, and migrate aborts trying to recreate existing tables. Reconcile the migrations ledger before migrate: record already-present tables, let missing ones be created. Also add an ERR trap so a failed run names the failing line and cannot masquerade as success (a silent mid-run failure left a half-built dir that looked done), a dependency check (curl/unzip/php), and mkdir guards for the storage carry-over. Document all of it in README.md and CLAUDE.md. Co-Authored-By: Claude Opus 4.8 --- CLAUDE.md | 47 ++++++++++++++++++++++++++++++++++++++++++++ README.md | 33 ++++++++++++++++++++++--------- firefly-update | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++------- 3 files changed, 125 insertions(+), 16 deletions(-) create mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..c1c103b --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,47 @@ +# CLAUDE.md + +Single bash script, `firefly-update`, that updates a Firefly III instance on +Debian. See README.md for usage; this file is the non-obvious context. + +## Hard-won gotchas (do not relearn these) + +- **Install from the GitHub release zip, never `composer create-project`.** The + composer/Packagist dist ships source only, no compiled frontend bundles + (`public/v1/js/app.js`, `app_vue.js`, ...). A composer install looks fine but + the UI is broken: no graphs, `404 /v1/js/app.js`, and the v2 error/login + pages throw `Vite manifest not found (public/build/manifest.json)` as a + secondary symptom. The zip is prebuilt and is the supported artifact. + +- **Release tag carries a leading `v`** (`v6.6.5`). Asset name is + `FireflyIII-.zip` with the `v` included. `--version` is normalized so + both `6.6.5` and `v6.6.5` work; don't reintroduce a double-`v` bug. + +- **Passport OAuth migration collision.** Laravel Passport renames its + migration files between versions, so a carried-over DB has the `oauth_*` + tables while the new filenames look pending; a naive `migrate` dies with + `table "oauth_auth_codes" already exists` (then `oauth_device_codes`, ...). + The script reconciles the `migrations` ledger before migrating (records + already-present tables, lets missing ones create). Do NOT "fix" a collision + by dropping tables, it is whack-a-mole and can lose + `oauth_personal_access_clients`. + +- **The Vite/`public/build` error is a red herring for graphs.** It only fires + on the v2 error and login pages. Graphs breaking = missing `public/v1/js` + bundles (see the zip point above), a different cause. + +## Conventions + +- Config via env vars (`WORKDIR`, `INSTANCE`, `BACKUPDIR`), matching the + existing style; no new CLI flags unless asked. +- Keep it a single file. `set -euo pipefail` + `ERR` trap are load-bearing: + the script must abort before the live swap on any failure and never leave a + half-update looking successful. +- Any change to the install/migrate flow: test the download+extract and the + migrate path before claiming it works. The script touches a live personal + finance instance; a bad run is not cheap. + +## Testing without a target machine + +There is no Firefly instance in this repo. Test URL construction and +download/extract against the real GitHub releases (the asset URLs are public); +the artisan/migrate steps can only be exercised on the actual host. diff --git a/README.md b/README.md index 49ab6ef..caf5f8f 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,17 @@ Update a [Firefly III](https://www.firefly-iii.org/) instance on Debian. -Installs the latest (or a pinned) release via Composer, carries over your -`.env`, uploads, exports, and SQLite database, runs the migrations and -upgrade steps, then swaps the new version in and restarts Apache. A -timestamped database backup is taken before anything is touched, and -`--restore` rolls back to the previous version. +Downloads the latest (or a pinned) release, carries over your `.env`, uploads, +exports, and SQLite database, runs the migrations and upgrade steps, then swaps +the new version in and restarts Apache. A timestamped database backup is taken +before anything is touched, and `--restore` rolls back to the previous version. + +Installs from the **official GitHub release zip**, not `composer +create-project`. The composer/Packagist dist ships source only and lacks the +compiled frontend bundles, which leaves the UI broken (no graphs, 404s on +`/v1/js/app.js`). The release zip is prebuilt. + +Requires `curl`, `unzip`, `php`, and `sqlite3` on the host. ## Usage @@ -14,7 +20,7 @@ Run as root (it chowns files and restarts Apache): ```bash sudo ./firefly-update # update to latest release -sudo ./firefly-update -v 6.1.0 # update to a specific tag +sudo ./firefly-update -v 6.6.5 # update to a specific tag (v-prefix optional) sudo ./firefly-update --restore # roll back to the previous version sudo ./firefly-update --help ``` @@ -36,9 +42,18 @@ WORKDIR=/srv INSTANCE=ff sudo -E ./firefly-update ## Notes - Assumes an SQLite database at `storage/database/database.sqlite`. -- `set -euo pipefail`: any failed step aborts before the live instance is - swapped, so a broken build never replaces a working one. -- The previous version is kept at `$INSTANCE-old` until the next run. +- `set -euo pipefail` plus an `ERR` trap: any failed step aborts before the + live instance is swapped (a broken build never replaces a working one) and + prints the failing line, so a partial run cannot masquerade as success. +- Before migrating, the script reconciles Laravel Passport's OAuth migrations: + Passport renames its migration files between versions, so a carried-over DB + has the `oauth_*` tables while the new filenames look pending, and a naive + `migrate` would fail trying to recreate existing tables. The script records + those already-present tables as migrated and lets genuinely-missing ones be + created. +- The previous version is kept at `$INSTANCE-old` until the next run; + `--restore` swaps it back and overlays the newest DB backup. A failed restore + leaves the broken version at `$INSTANCE-broken`. ## License diff --git a/firefly-update b/firefly-update index 608a265..fe46d4a 100644 --- a/firefly-update +++ b/firefly-update @@ -16,6 +16,9 @@ # along with this program; if not, see . set -euo pipefail +# Loud failure: without this the script died silently mid-update and left a +# half-built dir that looked like success. Now it names the failing line. +trap 'echo "FAILED at line $LINENO (last command exited $?)" >&2' ERR WORKDIR=${WORKDIR:-/var/www} # Directory name of the live instance under $WORKDIR. Override with INSTANCE=... @@ -100,6 +103,10 @@ if [ -n "$dorestore" ]; then exit 0 fi +for dep in curl unzip php; do + command -v "$dep" >/dev/null || { echo "required command not found: $dep" >&2; exit 1; } +done + # Modify next line to where your firefly-iii instance is installed to. cd "$WORKDIR" || { echo "cannot cd to $WORKDIR" >&2; exit 1; } @@ -121,16 +128,30 @@ if [ -z "$latestversion" ]; then exit 1 fi fi -echo "installing firefly-iii $latestversion" - -# Install latest version. COMPOSER_ALLOW_SUPERUSER silences the root warning -# (this script runs as root); --no-interaction skips all prompts. +# Release tags carry a leading "v" (e.g. v6.6.5). Normalize so both -v 6.6.5 +# and -v v6.6.5 work, and so the autodiscovered tag matches the asset name. +case "$latestversion" in v*) tag="$latestversion" ;; *) tag="v$latestversion" ;; esac +echo "installing firefly-iii $tag" + +# Install from the official release zip, NOT composer create-project. The +# composer/Packagist dist ships source only: it lacks the webpack-compiled +# frontend bundles (public/v1/js/app.js etc.), which breaks the UI (no graphs, +# broken pages). The release zip is prebuilt and is the artifact Firefly's +# install docs point at. Asset name is FireflyIII-.zip, tag incl. the "v". rm -rf "$UPDATED" -COMPOSER_ALLOW_SUPERUSER=1 composer create-project --no-interaction --no-dev --prefer-dist grumpydictator/firefly-iii "$UPDATED" "$latestversion" +mkdir -p "$UPDATED" +zip="${WORKDIR}/firefly-${tag}.zip" +url="https://github.com/firefly-iii/firefly-iii/releases/download/${tag}/FireflyIII-${tag}.zip" +echo "downloading $url" +curl -fL -o "$zip" "$url" +unzip -q "$zip" -d "$UPDATED" +rm -f "$zip" # Carry over config and user data. cp -a of dir/. copies contents incl. dotfiles, -# and does not fail on an empty source directory. +# and does not fail on an empty source directory. mkdir -p in case the zip +# ships these dirs gitignored/absent. cp "${LIVE}/.env" "${UPDATED}/.env" +mkdir -p "${UPDATED}/storage/upload" "${UPDATED}/storage/export" "${UPDATED}/storage/database" cp -a "${LIVE}/storage/upload/." "${UPDATED}/storage/upload/" cp -a "${LIVE}/storage/export/." "${UPDATED}/storage/export/" # SQLite DB is a file; copy it so migrate runs on real data, not the fresh @@ -140,7 +161,33 @@ cp "${LIVE}/storage/database/database.sqlite" "${UPDATED}/storage/database/datab cd "$UPDATED" || { echo "cannot cd to $UPDATED" >&2; exit 1; } rm -rf bootstrap/cache/* php artisan cache:clear -php artisan migrate --seed --force + +# Reconcile Passport OAuth migrations. Passport renames its migration files +# between versions (e.g. 2018_..._create_oauth_auth_codes -> 2016_...), so a +# carried-over DB has the TABLES but the new filenames look "pending", and +# migrate then tries CREATE TABLE on an existing table and aborts. For every +# oauth migration whose table already exists but whose filename isn't recorded, +# insert a migrations row so migrate skips it. Missing tables are left for +# migrate to create normally. Requires sqlite3. +db="${UPDATED}/storage/database/database.sqlite" +if command -v sqlite3 >/dev/null; then + batch=$(( $(sqlite3 "$db" "SELECT COALESCE(MAX(batch),0) FROM migrations;") + 1 )) + for f in database/migrations/*oauth*; do + [ -e "$f" ] || continue + name=$(basename "$f" .php) + tbl=$(grep -oP "Schema::create\('\K[^']+" "$f" | head -1) + [ -n "$tbl" ] || continue + [ -n "$(sqlite3 "$db" "SELECT 1 FROM migrations WHERE migration='$name';")" ] && continue + if [ -n "$(sqlite3 "$db" "SELECT 1 FROM sqlite_master WHERE type='table' AND name='$tbl';")" ]; then + sqlite3 "$db" "INSERT INTO migrations (migration,batch) VALUES ('$name',$batch);" + echo "reconciled existing oauth table: $tbl ($name)" + fi + done +else + echo "warning: sqlite3 not found, skipping oauth migration reconcile" >&2 +fi + +php artisan migrate --force php artisan firefly-iii:upgrade-database php artisan passport:install php artisan cache:clear -- cgit v1.2.3