1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
|
# Backlog
Open items, newest last. One numbering sequence; a closed item keeps its
number and gains a status rather than being renumbered.
| # | Item | Size | Status |
|---|------|------|--------|
| 1 | Skip boilerplate namespace URLs | XS | open |
## 1. Skip boilerplate namespace URLs
**Observed.** A sweep of 82 real spam messages reported 8 URL IOCs pointing
at `www.w3.org`, from four distinct values:
```
http://www.w3.org/1999/xhtml
http://www.w3.org/TR/html4/loose.dtd
http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd
https://www.w3.org/1999/xhtml
```
**Cause.** `_scan_urls()` scans the body text for anything URL-shaped, and an
XHTML doctype or namespace declaration is URL-shaped. The spammer did not put
those there; the HTML boilerplate did.
**Approach.** A hostname skip-list, checked after the URL is extracted. Four
hosts cover everything the corpus produced: `www.w3.org`, `schemas.microsoft.com`,
`purl.org`, `ns.adobe.com`. Keep it a list of HOSTS, not a regex over the URL:
a hostile URL can put any string in a path or query, so matching on anything
but the host lets an attacker suppress their own indicator.
**Constraints.** Not a leak and not a correctness defect: it is noise in a
report a human reads, and the cost is filing `w3.org` as phishing
infrastructure with an abuse desk. Weigh against the argument for leaving it
alone, which is real: a parser that reports exactly what the message contained
is easier to defend than one that decides what to omit, and every entry on a
skip-list is a thing an attacker knows will not be reported. If it is built,
the skipped URLs should still be visible somewhere during review rather than
silently dropped, on the same reasoning that makes `suspect_path_segments`
flag rather than redact.
|
