From 00c68e841c416c7c069869a52d64e8c15a72e1de Mon Sep 17 00:00:00 2001 From: "Danilo M." Date: Tue, 8 Sep 2026 16:28:48 +0200 Subject: docs: record the real-mail sweep and open a backlog A synthetic fixture only ever holds the shapes someone thought of, and the shapes nobody thought of are the ones that leak. Sweeping the user's own tag:spam is the only source of those, and it earned its place: the first message found a header the parser did not read, and the corpus exercised a spoofed Reply-To display name no fixture had. AGENTS.md records it under Testing, beside the fixture rule it is the deliberate exception to: ask first, work from the scratchpad, compare every address in the raw source against the whole ioc output, and let only counts and stripped domains out. A finding becomes a synthetic fixture; the real message never enters the repository. BACKLOG.md opens with the one thing the sweep found that was not worth fixing blind: boilerplate namespace urls (w3.org doctypes) reported as indicators. Noise rather than a defect, and the argument for leaving it alone is real, so it is written down with both sides rather than built. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019NHaqA1Rz5ybed7wFUeQbK --- docs/BACKLOG.md | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 docs/BACKLOG.md (limited to 'docs/BACKLOG.md') diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md new file mode 100644 index 0000000..25793e1 --- /dev/null +++ b/docs/BACKLOG.md @@ -0,0 +1,40 @@ +# Backlog + +Open items, newest last. One numbering sequence; a closed item keeps its +number and gains a status rather than being renumbered. + +| # | Item | Size | Status | +|---|------|------|--------| +| 1 | Skip boilerplate namespace URLs | XS | open | + +## 1. Skip boilerplate namespace URLs + +**Observed.** A sweep of 82 real spam messages reported 8 URL IOCs pointing +at `www.w3.org`, from four distinct values: + +``` +http://www.w3.org/1999/xhtml +http://www.w3.org/TR/html4/loose.dtd +http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd +https://www.w3.org/1999/xhtml +``` + +**Cause.** `_scan_urls()` scans the body text for anything URL-shaped, and an +XHTML doctype or namespace declaration is URL-shaped. The spammer did not put +those there; the HTML boilerplate did. + +**Approach.** A hostname skip-list, checked after the URL is extracted. Four +hosts cover everything the corpus produced: `www.w3.org`, `schemas.microsoft.com`, +`purl.org`, `ns.adobe.com`. Keep it a list of HOSTS, not a regex over the URL: +a hostile URL can put any string in a path or query, so matching on anything +but the host lets an attacker suppress their own indicator. + +**Constraints.** Not a leak and not a correctness defect: it is noise in a +report a human reads, and the cost is filing `w3.org` as phishing +infrastructure with an abuse desk. Weigh against the argument for leaving it +alone, which is real: a parser that reports exactly what the message contained +is easier to defend than one that decides what to omit, and every entry on a +skip-list is a thing an attacker knows will not be reported. If it is built, +the skipped URLs should still be visible somewhere during review rather than +silently dropped, on the same reasoning that makes `suspect_path_segments` +flag rather than redact. -- cgit v1.2.3