1 files changed, 345 insertions, 0 deletions
diff --git a/docs/specs/2026-09-09-contacts.md b/docs/specs/2026-09-09-contacts.md
new file mode 100644
index 0000000..faeef58
--- /dev/null
+++ b/docs/specs/2026-09-09-contacts.md
@@ -0,0 +1,345 @@
+# abusectl `contacts`: IOCs to abuse contacts, over RDAP
+
+Status: **agreed 2026-09-09**, in one brainstorming session with the user.
+This spec settles the part the umbrella design left open, named there as
+"RDAP bootstrap and referral chasing, caching policy, rate limits, and what
+happens for a netblock that publishes no abuse contact".
+
+Read `docs/specs/2026-09-08-abusectl-design.md` first. This document assumes
+its manifest format, its case directory and its ordering between the parts,
+and changes none of them.
+
+## What it does
+
+```
+abusectl contacts <case>
+```
+
+Reads a case manifest, resolves an abuse contact for every IP and domain
+indicator via RDAP, and rewrites the manifest with a `contacts[]` array. It
+is read-only against the network: it queries registries and sends no mail,
+files no report and changes nothing a third party can observe beyond the
+queries themselves.
+
+It is the FIRST network module in this tool. Everything below that looks
+defensive is defensive because of that.
+
+## Modules
+
+```
+abusectl/
+  contacts.py     IOCs -> contacts[]            network, read-only
+  rdap.py         bootstrap, query, jCard       network, read-only
+```
+
+`rdap.py` is the PROTOCOL: fetching and caching the IANA bootstrap files,
+selecting a server for an address or a TLD, issuing a query, and pulling an
+abuse address out of a jCard. `contacts.py` is the POLICY: which indicators
+are resolvable, how hosts fold together, what is written into the manifest.
+
+**The split is the same one that puts `redact.py` beside `parse.py`.** The
+protocol dirt is where the defects will be, and a module of its own gets
+tests that name it rather than tests that reach it incidentally through case
+handling.
+
+### The offline guarantee survives, for the whole suite
+
+Both modules take a `fetch` callable as an ARGUMENT, defaulting to the real
+urllib transport. Tests pass a fake and never construct the real one.
+
+The suite must keep passing with `socket.socket`, `socket.create_connection`
+and `socket.getaddrinfo` all raising, exactly as it does today, and that
+property stays a property of the WHOLE suite. Not "the whole suite except
+contacts". A network module that can only be tested with a network is a
+module that stops being tested.
+
+This is the pattern the umbrella design promised for exactly this moment: if
+a later part needs the network, it goes in its own module with an injected
+transport.
+
+## THE FOURTH PROPERTY
+
+The umbrella design states three properties that are not negotiable. This
+part adds a fourth, because it is the first part that can violate it.
+
+### Never disclose more than the host under investigation
+
+An RDAP query tells a third party what the user is looking at. Queries carry
+BARE HOSTS AND IP ADDRESSES ONLY, never a full URL.
+
+A URL path can carry recipient identity. `parse.suspect_path_segments()`
+FLAGS those rather than redacting them, deliberately, because a path segment
+may be the thing being reported. That decision is safe only while the URL
+stays local. Sending a URL to a registry would leak precisely what property 1
+exists to prevent, through a channel property 1 does not cover: property 1
+governs what is PUBLISHED, and a query is a disclosure that never appears in
+any report.
+
+Concretely: a `url` indicator contributes its HOST to the domain worklist and
+nothing else. Path, query and fragment never leave the machine.
+
+**This is a trap, not a theoretical concern.** The obvious implementation
+resolves "a contact for each indicator" by reading each indicator's `value`,
+and for a `url` indicator that value is an entire URL. The sweep described
+under Testing exists to catch exactly that mistake against real mail.
+
+## What gets resolved
+
+| Indicator | Query | Why |
+|---|---|---|
+| `ipv4`, `ipv6` | RDAP IP lookup | the sending IP is the primary takedown target |
+| `domain` | RDAP domain lookup | the registrar holds the abuse desk |
+| `url` | its HOST, as a domain | folded into the domain worklist, see the fourth property |
+| `sha256` | none | no registry owns a hash |
+| `observation` | none | not a network object |
+
+Indicators carrying `confidence: untrusted-hop` ARE resolved. A forged chain's
+IP may still be the real sender's, and refusing to resolve it would discard
+the case the tool exists for. The confidence marker stays in the manifest, so
+review shows the user that a contact came from an attacker-supplied hop
+before any report goes out.
+
+**Hosts fold.** Twenty URLs on one host produce one worklist entry and one
+query. The resulting contact entry lists every indicator id that contributed,
+so nothing is lost by folding.
+
+## RDAP mechanics
+
+### Bootstrap
+
+Fetch `ipv4.json`, `ipv6.json` and `dns.json` from `data.iana.org/rdap/`.
+Cache them under `$XDG_CACHE_HOME/abusectl/rdap/`, defaulting to
+`~/.cache/abusectl/rdap/`. TTL 7 days.
+
+The cache is NOT in the case directory. It is not evidence, it is a copy of a
+public map.
+
+**A failed refetch falls back to the stale copy**, with a warning, rather
+than failing the run. Last week's map is almost certainly still correct, and
+IANA being unreachable should not stop the user filing a report. A stale
+bootstrap is safe to serve: the worst case is querying a server that has
+moved, which fails and reads as "no contact".
+
+### Server selection
+
+- IP: longest-prefix match over the bootstrap ranges, using `ipaddress`.
+- Domain: exact TLD match in `dns.json`.
+- No match: `abuse: []` with the reason recorded. Many TLDs publish no RDAP
+  service at all, and that is a normal outcome rather than a defect.
+
+### The registrable domain, by walking up
+
+RDAP domain lookup wants the registrable domain. `mail.deep.example.invalid`
+is not registrable and most registries answer it with a 404.
+
+Query the full host, then drop one label and retry, until a server answers or
+the labels run out. Cap at 5 attempts and NEVER query a bare TLD.
+
+**This replaces a Public Suffix List deliberately.** Taking the last two
+labels is wrong for every multi-part suffix, `co.uk` and `com.au` among them,
+and a bundled PSL snapshot is a transcribed table that goes stale weekly,
+which is the failure `init.PROVIDERS` already documents at length. The
+registry is the authority on what is registrable, so the walk asks it instead
+of approximating it.
+
+The in-memory dedup below makes the walk cheap in the common case: a
+campaign's twenty subdomains under one registrable domain collapse to one
+successful query plus a few cheap misses.
+
+### Transport
+
+`urllib.request` from the standard library. `requirements.txt` stays empty.
+
+- `Accept: application/rdap+json`
+- **timeout mandatory**, 10 seconds. `urllib` with no timeout blocks forever,
+  and a hung registry would hang a review.
+- **redirects capped at 5 hops.** RIRs redirect to each other and the default
+  handler follows without a limit of ours.
+- **an https to http downgrade is refused.**
+
+The cap and the downgrade refusal follow the discipline `_MAX_REDIRECT_DEPTH`
+already sets in `parse.py`, and for the same reason: this is remote data
+directing our next move.
+
+`requests` was considered and rejected. It is present in slackware64-current,
+so packaging was not the deciding factor. It simply buys little here: the two
+things this module needs beyond a plain GET are the redirect cap and the
+downgrade refusal, and neither is a knob on `requests` either, both needing a
+`Session` plus custom wiring of comparable size. An empty `requirements.txt`
+means the tool runs and tests with no venv step at all.
+
+### Caching policy
+
+Bootstrap: on disk, 7 day TTL, as above.
+
+Responses: **in memory only, keyed by query URL, for the duration of one
+run.** Nothing on disk.
+
+The asymmetry is deliberate and is about which staleness hurts. A stale
+bootstrap fails loudly enough, producing a miss. **A stale abuse address
+sends a report into a mailbox nobody reads, silently**, and the user learns
+nothing came of it only by never hearing back. RDAP rate limits are generous
+for a human-paced tool filing a handful of reports, so the saving is not
+worth that.
+
+In-run dedup is free and still worth having: it is what stops a case with
+forty URLs on one host from issuing forty identical queries.
+
+## Extracting the address
+
+Strict role match, and nothing else. An entity whose `roles` contains `abuse`
+yields the `email` entries of its jCard. Recurse into nested `entities`,
+depth capped at 4, because the abuse entity is usually a child of the
+organisation entity.
+
+Four rules, each with a victim:
+
+**1. No fallback to a non-abuse entity.** A `technical` or `registrant`
+contact is a named human who never volunteered to receive abuse mail. Mailing
+them is useless and is a small privacy harm to an uninvolved third party.
+
+**2. No `abuse@<domain>` by convention.** RFC 2142 says the mailbox should
+exist; for a phishing domain it belongs to the ATTACKER. Constructing it
+would mail the attacker to tell them their campaign was caught and that the
+user's address is live, which is the exact outcome property 2 exists to
+prevent. This is the single worst thing this module could do, and it looks
+like a helpful fallback, which is why it is written down.
+
+**3. Never follow a `links` referral to fetch a contact.** Some responses
+point elsewhere for an entity. Following that is an outbound fetch to a URL
+the response chose for us, which is SSRF-shaped. If the address is not in the
+response, there is no address.
+
+**4. Validate the address before storing it.** `email.utils.parseaddr`,
+exactly one address, and reject any control character or newline. It becomes
+a mail recipient in `report` and `submit`; a CRLF there is header injection
+into mail this tool sends.
+
+**Multiple abuse addresses are all kept.** Some netblocks publish two, and
+picking one arbitrarily can drop the desk that would have answered.
+
+**A missing contact is a normal outcome, not a failure.** The umbrella design
+already settles this: the indicator keeps an empty `abuse` with the reason,
+no email destination is created for it, and review shows what could not be
+resolved. The indicator still reaches MISP and the vendor feeds.
+
+## The manifest
+
+```json
+"contacts": [
+  { "iocs": ["ioc-1"], "query": "198.51.100.7",
+    "abuse": ["abuse@example.invalid"],
+    "source": "rdap", "handle": "AS64496",
+    "server": "rdap.example.invalid" },
+  { "iocs": ["ioc-3", "ioc-7"], "query": "example.invalid",
+    "abuse": [], "source": "rdap",
+    "error": "no abuse role published" }
+]
+```
+
+Two departures from the sketch in the umbrella design, both deliberate:
+
+- **`iocs` is a list**, because hosts fold and one contact can serve several
+  indicators.
+- **`abuse` is a list**, because multiple desks are real.
+
+`query` records what was actually asked. Review can then see that a URL
+indicator was resolved by its host, which is the fourth property made visible
+rather than merely promised.
+
+`case.py` remains the only writer, and the write stays atomic.
+
+### Re-running
+
+**A re-run overwrites `contacts[]` wholesale**, in a single atomic write
+after every lookup has finished.
+
+Never a merge. A merge lets a contact resolved a week ago survive into a
+report filed today, which is the stale-address hazard the response caching
+policy already refuses. Overwriting makes a re-run always safe and always
+current, which matters because a partial network failure makes re-running the
+natural next step.
+
+### Failure is per query
+
+A timeout or an error on one indicator records that reason against that entry
+and the run continues. The manifest is written once at the end regardless. A
+run that resolved nothing at all still writes, so that review shows why
+rather than showing an absent section.
+
+## Testing
+
+TDD, as everywhere in this repository. Write the failing test, watch it fail,
+implement, watch it pass.
+
+`rdap.py` is tested with a fake `fetch` returning canned responses in real RIR
+SHAPES, with every address, handle and range replaced by `example.invalid`
+and RFC 5737 documentation values, per the fixture rule in AGENTS.md.
+
+Tests that have a right answer:
+
+- longest-prefix server selection, including an address matching two ranges
+- the label walk stops at the first server that answers, and never queries a
+  bare TLD
+- nested entity recursion finds an abuse entity under an organisation
+- a response carrying only a `technical` role yields no address, not the
+  technical one
+- a `links` referral is NOT fetched: the fake transport asserts it was never
+  called for that URL
+- a CRLF in an email field is rejected
+- **a `url` indicator contributes only its host**: the fake transport asserts
+  no query ever contained the path
+- a stale bootstrap is used when the refetch fails
+- the whole suite still passes with sockets raising
+
+### Two sweeps of the user's real spam
+
+The `parse` sweep found a header the parser never read and a spoofed
+`Reply-To` no fixture had. The same corpus is worth sweeping here, but this
+module sends traffic, so it splits in two.
+
+**Sweep A, offline, the whole corpus.** Build the worklist from all messages
+with a fake transport that RECORDS queries and returns canned responses. No
+packets. This carries the assertion:
+
+> no query string ever contained anything but a bare host or IP address
+
+That is the fourth property checked against real mail rather than against
+fixtures, over the shapes a synthetic fixture misses: a URL with a path, with
+userinfo, an IDN, a trailing dot, a bracketed IPv6 literal, a host that is
+itself an address. It also yields free counts: unique hosts, how many
+indicators fold per host, how deep the label walks go.
+
+**Sweep B, online, a deliberate handful.** Ten to twenty indicators picked by
+hand across distinct netblocks and TLDs, against the real registries. This
+confirms that real jCard shapes parse, that the walk terminates, and that the
+selection works. Twenty queries is polite and unremarkable.
+
+Sweep A carries the assertion. Sweep B is a shape check.
+
+**Ask before either, and sweep B needs an explicit go-ahead**, because unlike
+every test before it in this repository it is OUTWARD-FACING: it discloses to
+registries, and to a registrar who may be the attacker's, which netblocks and
+domains the user is investigating, from their address, at a known time. That
+cannot be undone, which is why it stays small and hand-picked rather than
+being run over the corpus.
+
+What may leave the sweep scripts: counts, tallies, TLDs, error reasons, walk
+depths, and whether any query was ever malformed. What may not: an address, a
+real abuse contact, a real domain written into a test, or a URL from a real
+message. A defect found by a sweep is reproduced as a synthetic fixture and
+THAT is committed. The real messages stay in the scratchpad, which is
+per-session and outside the repository.
+
+## Deliberately absent
+
+- **No response cache on disk.** See the caching policy above.
+- **No PSL.** See the label walk above.
+- **No retry or backoff here.** Rate-limit handling belongs to `submit` and
+  `retry`, which own the deadline logic the umbrella design specifies. A
+  read-only lookup that fails is simply re-run.
+- **No `whois` fallback.** It is a different protocol with an unparseable
+  free-text response per registry, and its absence costs a contact rather
+  than a report.
+- **No contact for `sha256` or `observation` indicators.** No registry owns
+  them.