| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Every other field in the machine-readable part was checked against RFC 5965
itself. This one follows the design document's worked example and no primary
x-arf source was reachable to confirm it, so it is recorded rather than left
as an assumption in the code.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xj1ayFRSUQ2u7cwb3S4axE
|
|
The for-clause rule anchored on the clause terminator, which is strictly
stronger than RFC 5321 4.4: For sits inside Opt-info, so With, ID, Via or a
CFWS comment may legitimately follow it, and its ABNF is 1*( Path / Mailbox )
where Mailbox carries no angle brackets. Only the tidiest shape was stripped.
Four ordinary ones published the victim's address in the one header a report
reproduces verbatim, including "for <a@b> (envelope-from <c@d>);", which is
routine Exim and Sendmail output.
Anchored on the address instead. The envelope SENDER survives the cut, since
that is what the report is about, and the leftover whitespace, orphan
separator and emptied comment are tidied because the line is published to a
third party verbatim. Table-driven over all five shapes; reverting the regex
fails six of nine tests in the class.
Two accepted disclosures are now named in the spec's "Where the headers come
from" rather than living in a test comment: our own relay's hostname, which
the by and authserv-id clauses carry by construction, and attacker-controlled
free text in Subject and the From display name, which the whitelist does not
and should not filter. The second is asserted as a documented limit so it
reads as deliberate, and the manifest assertion now bars the obfuscated
you%40example.org spelling alongside the plain one.
case.create() seeds an empty headers block so an unparsed case has the same
shape as a parsed one. Additive, so FORMAT_VERSION is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xj1ayFRSUQ2u7cwb3S4axE
|
|
Thirteen tasks, TDD throughout. Task 1 changes parse.py to store a whitelist
of publishable headers, which is what lets report never open source.eml, and
Task 13 re-runs both sweeps with a third assertion covering generated bodies.
Two spec items deliberately have no task and are recorded as gaps: api
destination bodies belong to the submit spec, and writing the frozen marker
is submit's work.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xj1ayFRSUQ2u7cwb3S4axE
|
|
Settles the three questions the umbrella design left open for report: the
X-ARF schema, which fields the reporting identity fills, and the plain-text
alternative.
The message is not attached. RFC 5965's text/rfc822-headers carries a
whitelist of headers instead, and parse stores that whitelist in the manifest
so report never opens source.eml, which is the first property applied one
level down.
Also settles destination grouping per abuse address, an explicit unreportable
list, and a freeze rule: any sent destination freezes the whole case, marked
by a write-once field, with body integrity checked by content hash rather
than mtime.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xj1ayFRSUQ2u7cwb3S4axE
|
|
The contacts spec's manifest example carried a "server" field that no code
ever wrote, while resolve() writes a "queried" key the spec never defined.
They are not the same fact under two names: "queried" is what was asked
about, the name the label walk landed on, and "server" is who was asked,
the RDAP endpoint the bootstrap selected.
Document "queried" as shipped, and record "server" as specified but not
built, with the condition that would earn it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xj1ayFRSUQ2u7cwb3S4axE
|
|
The author keeps ideas in an Obsidian note outside this repository. Nothing
in the repo pointed at it, so a fresh session would never find it and the
backlog would look complete while ideas sat unread. AGENTS.md now names the
path and the rule: the note is the inbox, the backlog is the tracked list,
and they are reconciled in both directions.
Item 3 is that note's one open idea, an API letting qtmaildir tag incoming
mail by resemblance to a kept case. It is recorded unsized and explicitly
marked as having no verified cause, because unlike items 1 and 2 there is
nothing built to verify. The entry records the real tension rather than just
the request: this tool has been deliberately mechanical, reporting what a
message declared and refusing rather than guessing, and a resemblance score
would be its first opinion. It also notes that source.eml is unredacted, so
matching must not become a route by which a stored recipient identifier
reaches a comparison.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wrfqr2xqQfhtXCscU7zrdz
|
|
An RDAP query discloses what the user is looking at, and property 1
covers only what is published, so the query rule needed stating in its
own right beside the other three. The section records the three leaks it
came from, because each one was the same shape: validation applied per
branch, forgotten on the next.
Also moves rdap.py's imports into one block at the top. Pure move, the
suite is 192 either side of it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wrfqr2xqQfhtXCscU7zrdz
|
|
The spec named a strict Accept: application/rdap+json while the plan, and
so the code, sends application/rdap+json with application/json at q=0.9.
A spec review caught the disagreement. The wider header is the right
behaviour, since some registries serve plain application/json and a
strict header would lose a contact that was published, so the spec moves
to match the code and now records why.
Both documents also claimed requirements.txt stays empty. There is no
requirements.txt in this repository and there never has been, so the
claim was true only vacuously. They now say the tool declares no
dependencies, which is the thing actually being promised.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wrfqr2xqQfhtXCscU7zrdz
|
|
Two corrections found while executing it.
The offline proof in task 9 patched socket.socket itself, which breaks
the ssl module at import time and produces false failures unrelated to
network use. It now patches socket.socket.connect, which is the right
target.
Task 12 gains a step moving the imports in rdap.py to the top of the
file. The module was built one task at a time, so several imports landed
mid-file after function definitions. Doing it in the documentation pass
keeps it out of the diff that contacts.py builds on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wrfqr2xqQfhtXCscU7zrdz
|
|
Twelve tasks, TDD throughout, building bottom up: transport, bootstrap
cache, server selection, jCard extraction, querying with the label walk,
the worklist, resolution, the subcommand, an offline proof, two sweeps
and the documentation.
The code in the plan was assembled and run before committing it, against
the plan's own tests, 44 of which pass. That found two defects in the
plan itself: a parameter that shadowed the cache_dir function and forced
a globals() lookup, now renamed to cache_root, and a hand-rolled fake
request in the redirect test that lacked the attributes
HTTPRedirectHandler reads, now a real urllib Request.
Also drops the em dashes from the Documents list in AGENTS.md, which the
no-em-dash writing rule covers and which the contacts entry had matched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wrfqr2xqQfhtXCscU7zrdz
|
|
Settles what the umbrella design left open for this part: RDAP bootstrap,
caching policy, and what happens for a netblock that publishes no abuse
contact.
Scope is IPs and domains, with a URL contributing only its host. The
bootstrap files are fetched from IANA and cached with a long TTL, while
responses are cached in memory for one run only: a stale bootstrap merely
misses, but a stale abuse address sends a report into a dead mailbox
silently. The registrable domain is found by walking up the labels and
asking the registry rather than by bundling a Public Suffix List, which
would be a transcribed table that goes stale weekly.
Address extraction is a strict abuse-role match with no fallback. In
particular it never constructs abuse@<domain>, because for a phishing
domain that mailbox belongs to the attacker and mailing it would confirm
both the catch and that the address is live.
Adds a fourth non-negotiable property: a query carries a bare host or IP
and never a URL. Property 1 governs what is published, and a query is a
disclosure that appears in no report, so a path segment that parse
deliberately flags rather than redacts would otherwise leak.
Transport is urllib with an injected fetch, so requirements.txt stays
empty and the suite keeps passing with sockets raising, for the whole
suite rather than for everything except contacts.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wrfqr2xqQfhtXCscU7zrdz
|
|
A synthetic fixture only ever holds the shapes someone thought of, and
the shapes nobody thought of are the ones that leak. Sweeping the user's
own tag:spam is the only source of those, and it earned its place: the
first message found a header the parser did not read, and the corpus
exercised a spoofed Reply-To display name no fixture had.
AGENTS.md records it under Testing, beside the fixture rule it is the
deliberate exception to: ask first, work from the scratchpad, compare
every address in the raw source against the whole ioc output, and let
only counts and stripped domains out. A finding becomes a synthetic
fixture; the real message never enters the repository.
BACKLOG.md opens with the one thing the sweep found that was not worth
fixing blind: boilerplate namespace urls (w3.org doctypes) reported as
indicators. Noise rather than a defect, and the argument for leaving it
alone is real, so it is written down with both sides rather than built.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019NHaqA1Rz5ybed7wFUeQbK
|
|
Its SPF is a tree of includes rather than a flat list: spf.privateemail.com
carries no addresses at all, only includes, and one branch nests a further
level. Two of the branches live on jellyfish.systems. The entry here is the
flattened union of the five leaf records, deduplicated, 18 networks.
The re-verification command lists the leaves rather than the top-level
name, and a note says why: querying spf.privateemail.com and finding no
ip4 entries looks like a stale record and is not one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KphFXTc2QajxXsHWyvGJ4R
|
|
Every range in the first draft was wrong. They had been written from
memory, and the correct source is each provider's own SPF record, which
is the list of addresses it declares it sends from.
Gmail is the clearest case: the table claimed eleven IPv4 ranges and
_spf.google.com publishes two. Fastmail, Proton and the rest were wrong
in the same way. Outlook and Zoho are added since they were queried
anyway, and the transcription commands are recorded in a comment so the
next check is a copy-paste rather than a search.
Google's goog.json is deliberately NOT the source, though it looks like
one: it lists all Google infrastructure, over a hundred ranges, and using
it would trust every Google-hosted service as part of the user's own mail
path. _spf.google.com is the mail-sending answer.
The table is IPv4 only. An IPv6 hop from one of these providers does not
match and the user is asked instead, which is the safe direction: a hop
wrongly trusted means the real sender is never reported.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KphFXTc2QajxXsHWyvGJ4R
|
|
Thirteen tasks, TDD throughout, stdlib only. parse is pure and offline:
the trusted-relay boundary arrives as an argument rather than a config
read, so the whole extractor is testable against fixtures with no setup.
The plan carries three checks that are not ordinary unit tests. The
Received-chain task has a mutation step, because walking one hop too far
reports an innocent third party named in a header the attacker wrote, and
a test that cannot fail would not protect against it. The URL task runs
the suite with sockets refused, so the never-fetch rule is verified
rather than read. And every fixture is asserted to leave no recipient
address anywhere in the manifest.
init exists because parse refuses to guess the trust boundary. It asks
for CIDRs, offers a static table of known provider ranges, or reads the
chain of a known-good sample and lets the user pick their own hops. A
pure builder with the prompts and the flags as two front ends over it, so
--non-interactive covers agent-driven setup and the config writing is
tested without a terminal. Re-running shows what is already configured
and asks; either route backs the old file up first and preserves sections
this run does not set, so a later init cannot silently drop a MISP key.
The prompts themselves are hand-tested rather than driven from stdin: a
test there would assert the wording it was written against and break on a
rewording that improved it. Task 12 is the checklist, weighted towards
wrong answers.
Redirect chains were missing from the first draft of the plan and are now
specified: a parameter whose value is itself a URL is recovered as an
indicator while every other value stays redacted, which resolves the
conflict between reporting the destination and never publishing a
tracking token.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KphFXTc2QajxXsHWyvGJ4R
|
|
A rate limit tells us when, not merely that: a 429 carries Retry-After or
the vendor's reset headers. Recording the deadline and asking the user to
run submit again throws that away and relies on them remembering.
So `abusectl retry` scans every case for destinations whose retry_after
has passed and sends only those, as one unattended cron line beside
mailsync.sh. No inline retry: submit never sleeps waiting for a window,
because a daily quota resets in hours and a process killed while
sleeping is back to the user remembering. One mechanism, not two.
Unattended retry makes three properties load-bearing, since a retry that
re-sends is a duplicate abuse report and cannot be withdrawn. Status is
written before the attempt, so a crash mid-send leaves in-flight, which
is honest, rather than looking like it never happened; retry never
touches in-flight. Attempts are capped, so a dead abuse mailbox stops
being retried. A soft failure with no server deadline gets exponential
backoff.
deferred and failed are separate statuses: deferred means the tool will
handle it, failed means the user must. Collapsing them either strands a
rate-limited report forever or retries a dead mailbox indefinitely.
The qtmaildir dialog accordingly grows no retry button. A deferred
destination belongs to cron, and a button beside it would race the
scheduled run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KphFXTc2QajxXsHWyvGJ4R
|
|
An abuse reporting sidecar for phishing mail: parse a flagged message,
extract its indicators, resolve abuse contacts, and file the result to a
MISP instance and to public abuse channels.
This is the umbrella spec, agreed in one design session. Each part gets
its own spec before it is built; this settles what the parts share and
what would be expensive to change later: the case directory and its
manifest format, the redaction rule, the ordering between MISP and the
vendors, and how partial failure is recorded.
It exists as a separate tool because qtmaildir does no network protocol
work by design, and this needs RDAP, three vendor APIs and mail to abuse
desks. qtmaildir invokes it by name the way it invokes mailsync.sh, and
hosts the review dialog; the two are coupled only by the manifest format
and a command name in config.
Two properties are recorded as safety properties rather than
preferences. Recipient identifiers are never captured, at extraction
rather than at submission, so the tool cannot disclose an identifier it
was never given; tracking tokens inside URLs are covered, since a
parameter value is frequently the recipient's address. And nothing
remote is fetched while parsing, because following a link confirms the
address is live and fires the tracker.
parse is the first part to build: stdlib only, no network, no config,
and its output is the format every other part reads.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KphFXTc2QajxXsHWyvGJ4R
|
