aboutsummaryrefslogtreecommitdiffstats
path: root/docs
diff options
context:
space:
mode:
Diffstat (limited to 'docs')
-rw-r--r--docs/BACKLOG.md40
1 files changed, 40 insertions, 0 deletions
diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md
new file mode 100644
index 0000000..25793e1
--- /dev/null
+++ b/docs/BACKLOG.md
@@ -0,0 +1,40 @@
+# Backlog
+
+Open items, newest last. One numbering sequence; a closed item keeps its
+number and gains a status rather than being renumbered.
+
+| # | Item | Size | Status |
+|---|------|------|--------|
+| 1 | Skip boilerplate namespace URLs | XS | open |
+
+## 1. Skip boilerplate namespace URLs
+
+**Observed.** A sweep of 82 real spam messages reported 8 URL IOCs pointing
+at `www.w3.org`, from four distinct values:
+
+```
+http://www.w3.org/1999/xhtml
+http://www.w3.org/TR/html4/loose.dtd
+http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd
+https://www.w3.org/1999/xhtml
+```
+
+**Cause.** `_scan_urls()` scans the body text for anything URL-shaped, and an
+XHTML doctype or namespace declaration is URL-shaped. The spammer did not put
+those there; the HTML boilerplate did.
+
+**Approach.** A hostname skip-list, checked after the URL is extracted. Four
+hosts cover everything the corpus produced: `www.w3.org`, `schemas.microsoft.com`,
+`purl.org`, `ns.adobe.com`. Keep it a list of HOSTS, not a regex over the URL:
+a hostile URL can put any string in a path or query, so matching on anything
+but the host lets an attacker suppress their own indicator.
+
+**Constraints.** Not a leak and not a correctness defect: it is noise in a
+report a human reads, and the cost is filing `w3.org` as phishing
+infrastructure with an abuse desk. Weigh against the argument for leaving it
+alone, which is real: a parser that reports exactly what the message contained
+is easier to defend than one that decides what to omit, and every entry on a
+skip-list is a thing an attacker knows will not be reported. If it is built,
+the skipped URLs should still be visible somewhere during review rather than
+silently dropped, on the same reasoning that makes `suspect_path_segments`
+flag rather than redact.