diff options
Diffstat (limited to 'CLAUDE.md')
| -rw-r--r-- | CLAUDE.md | 8 |
1 files changed, 6 insertions, 2 deletions
@@ -28,7 +28,7 @@ Three things are load-bearing and easy to break: 1. **`reverse=True` on `iter_messages`** (`tg_backup.py:75`). Processing must be oldest-to-newest. `state['last_id']` is a high-water mark used as `min_id` on resume; iterating newest-first would make it skip unfetched history permanently. 2. **State saved after every message**, not at the end, via temp-file + `os.replace`. Cheap insurance against interruption; do not "optimize" into a batched write without keeping resume correctness, and keep the write atomic. A truncated `state.json` makes the next run unstartable. -3. **Two-level retry handling.** The inner `download_with_retry` retries a single file up to `MAX_RETRIES`: FloodWait sleeps the server-requested interval, other exceptions (network drops, expired file references, disk errors) get exponential backoff so one flaky socket can't kill an overnight run. The outer `while True` in `run_backup` catches FloodWait on history fetching, saves state, sleeps, and restarts the iterator from the saved id. A permanently failed download still advances state on purpose, so one broken file can't wedge the backup forever. +3. **Two-level retry handling.** The inner `download_with_retry` retries a single file up to `MAX_RETRIES`: FloodWait sleeps the server-requested interval, other exceptions (network drops, expired file references, disk errors) get exponential backoff so one flaky socket can't kill an overnight run. The outer `while True` in `run_backup` catches FloodWait on history fetching, saves state, sleeps, and restarts the iterator from the saved id. A permanently failed download still advances state on purpose, so one broken file can't wedge the backup forever; the gap is recorded in `failures.json` (see below) rather than lost. `download_with_retry` returns the last exception, or `None` on success, so the caller has something to log. 4. **Media downloads to `{name}.part`, renamed on success.** The dedup check is `filepath.exists()`, so a truncated file at the final path would be skipped forever as if complete. The `.part` file is unlinked on failure and on interrupt, leaving the message to be retried on the next run. Resume/dedup is also file-based: media is named `{message.id}_{sanitized_name}` (see `safe_filename`, which strips sender-controlled path components) and an existing path is skipped, so state loss degrades to a slow rescan rather than re-downloading everything. @@ -39,7 +39,11 @@ Resume/dedup is also file-based: media is named `{message.id}_{sanitized_name}` ## Layout -`--archive-dir` holds media files and `state.json`, and is created if missing (`parents=True`). It is per-chat: one dir per backed-up chat. +`--archive-dir` holds media files, `state.json`, and `failures.json`, and is created if missing (`parents=True`). It is per-chat: one dir per backed-up chat. + +`state.json` also stores the `target` the dir was last backed up with, so subsequent runs need only `--archive-dir`. An explicit `--target` still wins; it is only required for a dir with no saved target (a first run, or one created before this was added). Nothing guards against pointing an existing dir at a *different* chat: `last_id` from the old chat would be applied as `min_id` to the new one and skip its history. + +`failures.json` is append-only JSONL, one `{"id", "file", "error"}` object per line, written only when a download exhausts `MAX_RETRIES`. JSONL rather than a JSON array so appending is a single write with no read-modify-write: a crash costs the last line, not the file. Read it with `jq -s`. Nothing reads or prunes it, and a message failing on two runs is logged twice. Auth lives in `~/.config/telegram_backup/` (mode 0700): `config.json` with the api_id/api_hash, and `session.session`, a single Telethon session **shared by every archive dir**, so one login covers all chats. `migrate_session` relocates a pre-existing `<archive_dir>/session.session` to the shared path on first run rather than forcing a re-login; it is a no-op once the shared session exists. It uses `shutil.move`, not `os.replace`, because the two paths can be on different filesystems. |