From bc909d9886e0c679fec1589734199de8873f9469 Mon Sep 17 00:00:00 2001 From: "Danilo M." Date: Fri, 31 Jul 2026 08:57:50 +0200 Subject: feat: log permanently failed downloads to failures.json State advances past a download that exhausts MAX_RETRIES so one broken file cannot wedge the backup, which until now left a silent gap in the archive. Failures are appended to failures.json in the archive dir as JSONL, one {id, file, error} per line; appending is a single write, so a crash costs the last record rather than the file. download_with_retry now returns the last exception or None on success, replacing the True/False return, so the caller has an error to log. Nothing reads or prunes the file: it is a record, not a retry queue. Co-Authored-By: Claude Opus 5 --- README.md | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index 32a7fe2..ee917fd 100644 --- a/README.md +++ b/README.md @@ -60,11 +60,19 @@ Numeric IDs are more durable than usernames, which their owners can change. Chat `--target` also accepts `me` (Saved Messages), t.me links, and invite links for groups you have already joined. +Update it later with just the directory: + +```bash +tg_backup.py --archive-dir ~/backups/examplegroup +``` + +The target is recorded in the archive directory on the first run, so repeat runs do not need it. Passing `--target` again still overrides what was saved. Point a directory at a different chat and the resume position carries over, which will skip that chat's history; use a fresh directory instead. + ### Options | Option | Default | Meaning | |---|---|---| -| `--target` | required | Chat to back up: `@username`, numeric ID, t.me link, or `me` | +| `--target` | first run only | Chat to back up: `@username`, numeric ID, t.me link, or `me`. Reused from the archive directory afterwards | | `--archive-dir` | `./tg_archive` | Where media and resume state are written; created if missing | | `--list-chats` | | Print your dialogs with IDs and usernames, then exit | | `--self-check` | | Run internal assertions and exit. No network, no login | @@ -82,11 +90,17 @@ Without `=`, argparse reads the leading `-` as the start of another flag. ## How it works -Messages are processed oldest to newest, and the last message ID seen is written to `state.json` in the archive directory after every message. An interrupted run resumes from there instead of rescanning the whole chat. +Messages are processed oldest to newest, and the last message ID seen is written to `state.json` in the archive directory after every message, alongside the chat it belongs to. An interrupted run resumes from there instead of rescanning the whole chat. Media is saved as `{message_id}_{original_name}`. Downloads land in a temporary `.part` file and are renamed only on success, so a killed run never leaves a truncated file that a later run would mistake for a complete one. Files that already exist are skipped, so even a lost `state.json` degrades to a slow rescan rather than re-downloading everything. -Rate limits are handled at two levels: Telegram's requested wait is honoured for both individual downloads and history fetches, and other errors (dropped connections, expired file references) are retried with exponential backoff. A download that fails permanently is logged and skipped rather than stalling the run forever. +Rate limits are handled at two levels: Telegram's requested wait is honoured for both individual downloads and history fetches, and other errors (dropped connections, expired file references) are retried with exponential backoff. A download that fails permanently is skipped rather than stalling the run forever, and its message ID is appended to `failures.json` so the gap is auditable: + +```bash +jq -s 'map(.id)' ~/backups/examplegroup/failures.json +``` + +One JSON object per line, written only after all retries are exhausted. Nothing retries or prunes these; the file is a record, not a queue. Only media is saved. Message text and captions are not. @@ -96,7 +110,8 @@ Only media is saved. Message text and captions are not. |---|---| | `~/.config/telegram_backup/config.json` | API credentials | | `~/.config/telegram_backup/session.session` | Login session, shared across all archive directories | -| `/state.json` | Resume position for that chat | +| `/state.json` | Resume position and saved target for that chat | +| `/failures.json` | Message IDs whose media never downloaded | | `/*` | Downloaded media | The session file is account credentials: anyone holding it can read your Telegram. It is created mode `0600` inside a `0700` directory. Keep it out of version control and off shared storage. -- cgit v1.2.3