# AGENTS.md This file is the single source of truth for how to work on this repository, shared across every agent tool (Claude Code, opencode, etc). `CLAUDE.md` beside it is a pointer to this file and holds no content of its own. ## What this is `abusectl` parses a phishing email, extracts indicators of compromise, resolves who to report each one to, and files the result to a MISP instance and to public abuse channels. It exists because the author is a security consultant who has been phished and wants to act quickly on a campaign that targets them. It is a **sidecar to qtmaildir**, not part of it. qtmaildir does no network protocol work at all by design, and this needs RDAP, three vendor APIs and mail to abuse desks. qtmaildir invokes this tool by name, the way it invokes `mailsync.sh`, and hosts the review dialog. The two repositories are coupled only by the manifest format and a command name in qtmaildir's config; there is no submodule and there should not be one. Backlog item 194 in qtmaildir's `docs/superpowers/plans/2026-08-03-post-0.1.0-usability.md` carries the qtmaildir half. ## Build and test ```bash python3 -m unittest discover tests ``` Stdlib only, no framework, no network, no fixtures directory beyond `tests/fixtures/*.eml`. A single module: ```bash python3 -m unittest tests.test_parse -v ``` Run the tool from the checkout with `python3 -m abusectl`. ## THREE PROPERTIES THAT ARE NOT NEGOTIABLE Each has a concrete victim. Do not weaken one for convenience, and do not "simplify" the code that enforces it without reading this section first. ### 1. Recipient identifiers must never reach a report Reports go to third parties and to public feeds. A phishing URL routinely carries the recipient's identity in its query string (`?e=
`, `?u=`, `?id=`), so publishing a URL verbatim leaks the victim's address, and it **deanonymises the reporter to the attacker**, since abuse desks forward reports to the abused customer and URLhaus is public. The guarantee is STRUCTURAL: the tool cannot disclose an identifier it was never given. `parse.py` does not read `To`, `Cc`, `Delivered-To` or `X-Original-To` at all, and `redact.py` blanks values before anything is stored. `redact.url()` keeps parameter NAMES (they fingerprint the kit) and blanks VALUES (they identify the recipient). It covers the query, the fragment AND userinfo. The fragment matters even though a browser never sends it, because what gets published is the URL TEXT from the message. Userinfo is worse than the rest: it carries an address and a credential together. A token is unique per recipient by design, so keeping it would make correlation WORSE, not better: two messages from one campaign would look like different URLs. **The one exception is the rule's own logic, not a hole in it.** A redirector carries its destination in a parameter. That destination is an indicator rather than a recipient identifier, so `redact.url_valued_parameters()` recovers it and it becomes an IOC in its own right, still redacted itself. Every value that is not a URL stays blanked. Path segments are FLAGGED, never redacted (`suspect_path_segments`), because unlike a query value a path segment may be the thing being reported. Flagged is visible to the user during review; silent was the bug. ### 2. Nothing is ever fetched or resolved Not URLs, not redirect chains, not remote images, no DNS. Following a link confirms to the sender that the address is live and fires exactly the tracker the message wanted. A redirect chain is read from what the message DECLARES. `redirect_chains()` recurses into recovered targets, bounded by `_MAX_REDIRECT_DEPTH` and a `seen` set, because a nested value is attacker-supplied. **This is verified, not asserted.** The suite passes with `socket.socket`, `socket.create_connection` and `socket.getaddrinfo` all raising. Keep that possible: if a later part needs the network, it goes in its own module with an injected transport, never in the parse path. ### 3. The trust boundary is configured, never guessed `Received` headers are prepended, so the list runs newest first: our own infrastructure at the top, the sender below. **Everything below our own servers was written by whoever was talking to them and can be forged wholesale.** Attackers routinely prepend headers naming innocent third parties. `sending_ip()` returns the FIRST hop outside the configured networks, walking outermost inward. Walking to the last one instead reports whoever the attacker chose. `tests/fixtures/forged-chain.eml` exists for exactly this: the correct answer is `203.0.113.99`, and `198.51.100.7` is a planted innocent party. **That test is mutation-checked and must stay that way.** Change `sending_ip()` to keep walking and `test_a_forged_chain_stops_at_the_first_untrusted_hop` must fail with `'198.51.100.8' != '203.0.113.99'`. A test that cannot fail is not protecting anything, and this is the one standing between the tool and reporting an innocent party. With no boundary configured, `parse` REFUSES rather than guessing the outermost public IP. Refusing is only defensible because `abusectl init` is the route out, which is why that command exists at all. ## Architecture ``` abusectl/ cli.py argparse dispatch, exit codes, prompts. No logic. init.py first-run config: pure builder + prompt helpers + provider table config.py reads ~/.config/abusectl/config.toml via tomllib parse.py .eml -> IOCs pure, offline redact.py the safety rule, alone and testable case.py case directory: create, manifest read/write, atomic ``` Planned, each needing its own spec first: `contacts` (RDAP), `report` (X-ARF), `submit` (MISP then vendors), `retry` (cron). See the design document. **`redact.py` is separate from `parse.py` deliberately.** It is the safety property, and a module of its own gets tests that name it rather than tests that reach it incidentally. **`parse.py` takes the trust boundary as an ARGUMENT, never a config read.** That is what keeps it pure and testable with no files on disk. `cli.py` reads the config and passes it in. ## The case directory ``` /2026-09-08-a3f1/ source.eml the original, UNREDACTED manifest.json IOCs, and later contacts and per-destination status bodies/ report bodies, once `report` exists ``` `case.py` is the ONLY writer. The manifest is written atomically, temp file plus rename, because a half-written manifest during a review is a corrupted evidence record. A failed write must leave the existing manifest untouched and no stray temp file; there is a test for both. **Nothing deletes a case.** No cleanup function exists and none should be added: these are the user's working evidence. **`source.eml` is unredacted**, so a case directory is sensitive at rest. The redaction rules are about what may be PUBLISHED. The submit path must never attach `source.eml` wholesale. `load()` refuses a manifest whose `format` it does not know, rather than silently dropping fields a newer writer added. ## Config `~/.config/abusectl/config.toml`, read with stdlib `tomllib`, written by `init.py`. Mode `0600`, and so are the backups: the file holds API keys as later parts land. **A skipped answer is ABSENT from the file, never an empty string.** `api_key = ""` reads as configured-and-broken and produces a confusing auth error much later; absent reads as not-configured and the part that wants it can say so plainly. `config.load()` enforces the same rule reading back: an empty `cases` falls back to the default rather than becoming `Path("")`, which would scatter evidence into whatever directory the command ran from. Two traps found by hand-testing and now covered: - `trusted_relays = "192.0.2.0/24"` without brackets is valid TOML and iterates CHARACTERS, validating `'1'`, `'9'`, `'2'`. Rejected explicitly now. - **`ipaddress.ip_network(42)` does not raise**, it returns `0.0.0.42/32`. A typo'd config would have produced a valid-looking boundary trusting an address the user never named. Non-string entries are rejected. `init` never overwrites silently: it refuses without `--force`, the interactive path shows what is configured and asks, and either route backs the old file up first. **Sections `build()` does not produce are carried across verbatim**, so an init that only sets the relays cannot discard a `[misp]` key set earlier. ## The provider table `init.PROVIDERS` holds each provider's published sending ranges, transcribed from their own SPF records on 2026-09-08. The transcription commands are in a comment above it. **Do not edit these from memory.** The first draft was written from memory and every single range was wrong: the table claimed eleven IPv4 ranges for Gmail where `_spf.google.com` publishes two. A wrong range means a hop is treated as the user's own and the real sender is never reported. Google's `goog.json` looks authoritative and is the WRONG source: it lists all Google infrastructure, over a hundred ranges including `8.8.8.8`, and using it would trust every Google-hosted service as part of the user's mail path. `_spf.google.com` is the mail-sending answer. IPv4 only, deliberately. An IPv6 hop from a listed provider simply does not match and the user is asked instead, which is the safe direction. ## Testing TDD. Write the failing test, watch it fail, implement, watch it pass. **Test what has a right answer.** The geometry of a manifest, the generated TOML, the redaction of a URL, the hop a chain resolves to. Do not write a test that passes regardless of what the code does. **The interactive prompts are HAND-TESTED by the user, not unit-tested.** Whether a question reads clearly has no assertion, and a test driving stdin asserts the wording it was written against and breaks on a rewording that improved it. `cli.py`'s prompt helpers are covered by the user running them; its dispatch is covered by `tests/test_cli.py`. That hand test found five defects in one pass, four of them the same mistake: an answer validated somewhere other than where it was given, so a typo cost the whole run. **Validate each answer at the prompt that asked for it** and re-ask, rather than erroring after the next question. **Fixtures use `example.org`, `.invalid` and RFC 5737 documentation ranges only.** A real phishing sample carries the identifiers this tool exists to keep out of reports, and a repository is potentially public. Check a fixture's weekday with `date -d +%A` rather than writing it from memory: an RFC2822 parser validates the day against the date and a wrong one reads as a malformed header. ## Working on this repo Work directly on `master`, no PR flow. Commits are GPG-signed (`git commit -S`); never pass `--no-verify`. Global git hooks scan for personal data and a rejection is correct until proven otherwise. `HANDOFF.md` is local-only and gitignored; never stage or commit it. Design first for anything unbuilt: `contacts`, `report`, `submit` and `retry` each need their own spec before code, because each has real unknowns. The umbrella design settles only what they share. ## Documents - `docs/specs/2026-09-08-abusectl-design.md` — the umbrella design. Read it before changing anything about the case format, the redaction rule, or the ordering between MISP and the vendors. - `docs/plans/2026-09-08-parse.md` — the plan `init` and `parse` were built from. Historical once built, but it records why each test exists.