feat: log permanently failed downloads to failures.json

State advances past a download that exhausts MAX_RETRIES so one broken
file cannot wedge the backup, which until now left a silent gap in the
archive. Failures are appended to failures.json in the archive dir as
JSONL, one {id, file, error} per line; appending is a single write, so a
crash costs the last record rather than the file.

download_with_retry now returns the last exception or None on success,
replacing the True/False return, so the caller has an error to log.

Nothing reads or prunes the file: it is a record, not a retry queue.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

1 files changed, 6 insertions, 2 deletions

diff --git a/CLAUDE.md b/CLAUDE.md
index 551fc5f..c0af417 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -28,7 +28,7 @@ Three things are load-bearing and easy to break:
 
 1. **`reverse=True` on `iter_messages`** (`tg_backup.py:75`). Processing must be oldest-to-newest. `state['last_id']` is a high-water mark used as `min_id` on resume; iterating newest-first would make it skip unfetched history permanently.
 2. **State saved after every message**, not at the end, via temp-file + `os.replace`. Cheap insurance against interruption; do not "optimize" into a batched write without keeping resume correctness, and keep the write atomic. A truncated `state.json` makes the next run unstartable.
-3. **Two-level retry handling.** The inner `download_with_retry` retries a single file up to `MAX_RETRIES`: FloodWait sleeps the server-requested interval, other exceptions (network drops, expired file references, disk errors) get exponential backoff so one flaky socket can't kill an overnight run. The outer `while True` in `run_backup` catches FloodWait on history fetching, saves state, sleeps, and restarts the iterator from the saved id. A permanently failed download still advances state on purpose, so one broken file can't wedge the backup forever.
+3. **Two-level retry handling.** The inner `download_with_retry` retries a single file up to `MAX_RETRIES`: FloodWait sleeps the server-requested interval, other exceptions (network drops, expired file references, disk errors) get exponential backoff so one flaky socket can't kill an overnight run. The outer `while True` in `run_backup` catches FloodWait on history fetching, saves state, sleeps, and restarts the iterator from the saved id. A permanently failed download still advances state on purpose, so one broken file can't wedge the backup forever; the gap is recorded in `failures.json` (see below) rather than lost. `download_with_retry` returns the last exception, or `None` on success, so the caller has something to log.
 4. **Media downloads to `{name}.part`, renamed on success.** The dedup check is `filepath.exists()`, so a truncated file at the final path would be skipped forever as if complete. The `.part` file is unlinked on failure and on interrupt, leaving the message to be retried on the next run.
 
 Resume/dedup is also file-based: media is named `{message.id}_{sanitized_name}` (see `safe_filename`, which strips sender-controlled path components) and an existing path is skipped, so state loss degrades to a slow rescan rather than re-downloading everything.
@@ -39,7 +39,11 @@ Resume/dedup is also file-based: media is named `{message.id}_{sanitized_name}`
 
 ## Layout
 
-`--archive-dir` holds media files and `state.json`, and is created if missing (`parents=True`). It is per-chat: one dir per backed-up chat.
+`--archive-dir` holds media files, `state.json`, and `failures.json`, and is created if missing (`parents=True`). It is per-chat: one dir per backed-up chat.
+
+`state.json` also stores the `target` the dir was last backed up with, so subsequent runs need only `--archive-dir`. An explicit `--target` still wins; it is only required for a dir with no saved target (a first run, or one created before this was added). Nothing guards against pointing an existing dir at a *different* chat: `last_id` from the old chat would be applied as `min_id` to the new one and skip its history.
+
+`failures.json` is append-only JSONL, one `{"id", "file", "error"}` object per line, written only when a download exhausts `MAX_RETRIES`. JSONL rather than a JSON array so appending is a single write with no read-modify-write: a crash costs the last line, not the file. Read it with `jq -s`. Nothing reads or prunes it, and a message failing on two runs is logged twice.
 
 Auth lives in `~/.config/telegram_backup/` (mode 0700): `config.json` with the api_id/api_hash, and `session.session`, a single Telethon session **shared by every archive dir**, so one login covers all chats. `migrate_session` relocates a pre-existing `<archive_dir>/session.session` to the shared path on first run rather than forcing a re-login; it is a no-op once the shared session exists. It uses `shutil.move`, not `os.replace`, because the two paths can be on different filesystems.