diff options
| author | Danilo M. <danix@danix.xyz> | 2026-10-05 09:59:51 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-10-05 09:59:51 +0200 |
| commit | 889daacdf8e3a9f1f3a41fe3b546deff1bfbd0ae (patch) | |
| tree | 337909e697bced54fb8026d3115b677464722073 /image-builder/crontab.example | |
| parent | f66db7142686bcc8069a50da3925c098d595c5a5 (diff) | |
| download | sbo-dockerbuild-889daacdf8e3a9f1f3a41fe3b546deff1bfbd0ae.tar.gz sbo-dockerbuild-889daacdf8e3a9f1f3a41fe3b546deff1bfbd0ae.zip | |
On sda2 the registry grew 13G -> 53G between weekly GCs and filled /.
That broke /tmp, the build log, buildx state and the GC itself, so every
build failed from 2026-10-04. Run registry-gc.sh daily at 07:30 instead
of Sundays only.
The store has since moved to a dedicated disk with backup=0, out of
vzdump. Record that in fstab.example along with two traps hit during
the fix: `crontab -` on a full disk silently writes a 0-byte crontab,
and a plain docker stop/start leaves the registry serving the old,
deleted directory.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'image-builder/crontab.example')
| -rw-r--r-- | image-builder/crontab.example | 8 |
1 files changed, 5 insertions, 3 deletions
diff --git a/image-builder/crontab.example b/image-builder/crontab.example index 28f72d1..2b2cc24 100644 --- a/image-builder/crontab.example +++ b/image-builder/crontab.example @@ -17,7 +17,7 @@ # 03:00 -current bootstrap -> full -> testbuild # 05:00 15.0 bootstrap -> full -> testbuild # 07:00 reclaim dangling images (catches both variants) -# 08:00 registry blob GC, Sundays only +# 07:30 registry blob GC, daily # 09:00 staleness alert if any tag stopped moving # # Repos sync at 01:00/02:00, so the chain starts after that and the images are @@ -68,9 +68,11 @@ # build). Cheap when there is nothing to do. 0 7 * * * docker image prune -f >> /var/log/sbo-testbuild.log 2>&1 # The registry never reclaims on its own: every push adds blobs and nothing -# removes them, so its store grows until the disk fills. Weekly is enough. +# removes them, so its store grows until the disk fills. Weekly was enough +# while the store lived on sdb1; on sda2 (since 2026-09-22) it has ~55G and +# grew 13G -> 53G in one week, filling / and failing every build. Daily. # registry-gc.sh has its own safety gates; see the script. -0 8 * * 0 /opt/sbo-testbuild/image-builder/notify.sh run "registry GC" /opt/sbo-testbuild/image-builder/registry-gc.sh >> /var/log/sbo-testbuild.log 2>&1 +30 7 * * * /opt/sbo-testbuild/image-builder/notify.sh run "registry GC" /opt/sbo-testbuild/image-builder/registry-gc.sh >> /var/log/sbo-testbuild.log 2>&1 # --------------------------------------------------------------------------- # Staleness check |