# Vendor and MISP Destinations Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Make `report` write the `misp` and `api` destination rows its spec has always required, so a case shows every destination it will reach before anything is sent. **Architecture:** A static table in `report.py` names each destination and the IOC types it accepts, transcribed from vendor documentation the way `init.PROVIDERS` is transcribed from SPF records. `config.py` learns which destinations are configured and reports that as data rather than raising. `report.generate()` appends vendor rows to the email rows it already builds. `init.py` and `cli.py` grow four skippable prompts. **Tech Stack:** Python 3.11+ stdlib only (`tomllib`, `unittest`). No new dependency: nothing here sends anything. **Read first:** `docs/specs/2026-09-09-report.md`, the addendum dated 2026-09-10 at the end. It settles every decision below and records why. `AGENTS.md` for the four non-negotiable properties. --- ## Context an engineer needs before starting **This repository is TDD and the tests are the point.** Write the failing test, run it, watch it fail for the RIGHT reason, then implement. A test that passes before the implementation is a test that proves nothing. **Clear `__pycache__` between any mutation check.** A stale `.pyc` shadowed edited source in the last session and made a mutation check report nonsense. `python3 -B` does not help; it only stops bytecode being written. Use `find . -name __pycache__ -type d -exec rm -rf {} +`. **Run the suite with** `python3 -m unittest discover tests`. One module: `python3 -m unittest tests.test_report -v`. **Commits are GPG-signed.** `git commit -S`. Never `--no-verify`. Global hooks scan for personal data and a rejection is correct until proven otherwise. **Fixtures use `example.org`, `.invalid` and RFC 5737 documentation ranges only.** Never a real address, domain, or API key, in a test or a commit message. **What "configured" means throughout this plan:** a config section that is present AND carries every key it requires. `[abusedb]` and `[urlhaus]` require `api_key`. `[misp]` requires BOTH `url` and `api_key`; exactly one of the two is an error, not a skip, and Task 6 handles that separately from Task 3. **The five IOC type strings `parse.py` actually emits** are `ipv4`, `ipv6`, `domain`, `url` and `sha256`. Verified in `abusectl/parse.py:624`, `:631`, `:647` and `:658`. Do not invent a sixth. --- ## File structure | File | Responsibility | Change | |---|---|---| | `abusectl/report.py` | the destination table, vendor row building, `generate()` | modify | | `abusectl/config.py` | read which destinations are configured | modify | | `abusectl/init.py` | render the three new sections, fix the footer | modify | | `abusectl/cli.py` | the four prompts, the no-destination warning | modify | | `tests/test_report.py` | vendor rows | modify | | `tests/test_config.py` | configured-destination reading | modify | | `tests/test_init.py` | rendering the new sections | modify | | `tests/test_cli.py` | the warning, the MISP pair refusal | modify | No new files. The table belongs beside the code that consumes it, as `PROVIDERS` sits in `init.py`. --- ### Task 1: The destination table **Files:** - Modify: `abusectl/report.py` (add near the top, after the existing module constants) - Test: `tests/test_report.py` - [ ] **Step 1: Write the failing test** Add to `tests/test_report.py`: ```python class DestinationTable(unittest.TestCase): def test_every_destination_names_the_types_it_accepts(self): self.assertEqual(report.DESTINATIONS["abusedb"]["accepts"], ("ipv4", "ipv6")) self.assertEqual(report.DESTINATIONS["urlhaus"]["accepts"], ("url",)) self.assertEqual(report.DESTINATIONS["misp"]["accepts"], report.ALL_TYPES) def test_misp_is_its_own_kind_and_the_vendors_are_api(self): self.assertEqual(report.DESTINATIONS["misp"]["kind"], "misp") self.assertEqual(report.DESTINATIONS["abusedb"]["kind"], "api") self.assertEqual(report.DESTINATIONS["urlhaus"]["kind"], "api") def test_all_types_covers_what_parse_emits(self): # The five type strings parse.py writes. A sixth arriving without a # decision about which destinations accept it would silently reach # MISP only, so this test names them rather than deriving them. self.assertEqual(report.ALL_TYPES, ("ipv4", "ipv6", "domain", "url", "sha256")) ``` - [ ] **Step 2: Run test to verify it fails** Run: `python3 -m unittest tests.test_report.DestinationTable -v` Expected: FAIL with `AttributeError: module 'abusectl.report' has no attribute 'DESTINATIONS'` - [ ] **Step 3: Write minimal implementation** Add to `abusectl/report.py`, after the existing module-level constants: ```python # Every IOC type parse.py emits. Named rather than derived, so a new type # arriving without a decision about which destinations accept it is a test # failure rather than an indicator that quietly reaches MISP alone. ALL_TYPES = ("ipv4", "ipv6", "domain", "url", "sha256") # What each destination accepts, read from its own API documentation on # 2026-09-10: # # misp MISP book, circl.lu/doc/misp/automation. Any attribute type. # abusedb docs.abuseipdb.com, POST /api/v2/report: "a valid IPv4 or # IPv6 address", and nothing else. # urlhaus abuse.ch's own submit_url.py, POST urlhaus.abuse.ch/api/, # whose submission array carries a url. The full docs are # behind a login at auth.abuse.ch; re-verify the PAYLOAD there # when submit is built. The accepted type is unambiguous. # # VirusTotal is deliberately absent: its only submission endpoints are # POST /urls and file upload, so it accepts exactly what urlhaus accepts # and takes no verdict with a submission. See backlog item 6. # # DO NOT EDIT THESE FROM MEMORY. init.PROVIDERS records what that costs: # the first draft was written from memory and every range was wrong. A # wrong "accepts" here means a row promising a submission the endpoint # will refuse, or a missing row for something the vendor would have taken. DESTINATIONS: dict[str, dict] = { "misp": {"kind": "misp", "accepts": ALL_TYPES}, "abusedb": {"kind": "api", "accepts": ("ipv4", "ipv6")}, "urlhaus": {"kind": "api", "accepts": ("url",)}, } ``` - [ ] **Step 4: Run test to verify it passes** Run: `python3 -m unittest tests.test_report.DestinationTable -v` Expected: PASS, 3 tests - [ ] **Step 5: Commit** ```bash git add abusectl/report.py tests/test_report.py git commit -S -m "feat: add the destination table, transcribed from vendor docs" ``` --- ### Task 2: Building vendor rows **Files:** - Modify: `abusectl/report.py` (new function after `email_destinations`) - Test: `tests/test_report.py` - [ ] **Step 1: Write the failing test** Add to `tests/test_report.py`: ```python class VendorDestinations(unittest.TestCase): IOCS = [ {"id": "ioc-1", "type": "ipv4", "value": "198.51.100.7"}, {"id": "ioc-2", "type": "domain", "value": "example.invalid"}, {"id": "ioc-3", "type": "url", "value": "http://example.invalid/a"}, ] def test_a_row_carries_only_the_types_its_destination_accepts(self): rows = report.vendor_destinations(self.IOCS, {"abusedb", "urlhaus"}) by_id = {row["id"]: row for row in rows} self.assertEqual(by_id["abusedb"]["iocs"], ["ioc-1"]) self.assertEqual(by_id["urlhaus"]["iocs"], ["ioc-3"]) def test_misp_carries_every_ioc_including_what_no_vendor_takes(self): rows = report.vendor_destinations(self.IOCS, {"misp"}) self.assertEqual(rows[0]["iocs"], ["ioc-1", "ioc-2", "ioc-3"]) def test_a_destination_with_no_acceptable_ioc_gets_no_row(self): # The rule's whole point: a case with no URL must not promise a # urlhaus submission that would have nothing to submit. no_urls = [ioc for ioc in self.IOCS if ioc["type"] != "url"] rows = report.vendor_destinations(no_urls, {"abusedb", "urlhaus"}) self.assertEqual([row["id"] for row in rows], ["abusedb"]) def test_an_unconfigured_destination_gets_no_row(self): rows = report.vendor_destinations(self.IOCS, set()) self.assertEqual(rows, []) def test_the_row_shape_is_the_shape_submit_iterates(self): rows = report.vendor_destinations(self.IOCS, {"abusedb"}) self.assertEqual(rows[0], { "id": "abusedb", "kind": "api", "iocs": ["ioc-1"], "body": None, "status": "pending", }) def test_no_row_carries_a_target(self): # An endpoint is a property of the vendor, not of the case. Writing # one into the evidence record would let a hand-edit redirect a # submission somewhere the user never named. rows = report.vendor_destinations(self.IOCS, {"misp", "abusedb"}) for row in rows: self.assertNotIn("target", row) def test_rows_come_out_in_table_order_not_set_order(self): # The configured set is a set, so a stable order has to come from # somewhere else or the manifest churns between runs. first = report.vendor_destinations(self.IOCS, {"urlhaus", "misp", "abusedb"}) second = report.vendor_destinations(self.IOCS, {"abusedb", "misp", "urlhaus"}) self.assertEqual([row["id"] for row in first], [row["id"] for row in second]) self.assertEqual([row["id"] for row in first], ["misp", "abusedb", "urlhaus"]) ``` - [ ] **Step 2: Run test to verify it fails** Run: `python3 -m unittest tests.test_report.VendorDestinations -v` Expected: FAIL with `AttributeError: module 'abusectl.report' has no attribute 'vendor_destinations'` - [ ] **Step 3: Write minimal implementation** Add to `abusectl/report.py`, after `email_destinations`: ```python def vendor_destinations(iocs: list[dict], configured: set) -> list[dict]: """Build one row per configured destination that has something to send. BOTH conditions have to hold. A row for an unconfigured destination is a promise that can only fail; a row for a configured one with nothing it accepts is a promise with no content, an AbuseIPDB submission with nothing to put in its `ip` parameter. Each row carries ONLY the IOCs its own destination accepts. A row is what submit iterates, so a urlhaus row listing an IP is a submission that gets built wrong or dropped at send time, whichever the implementer notices first. Order comes from DESTINATIONS rather than from `configured`, which is a set and therefore has no order worth writing into a manifest twice. """ rows = [] for name, spec in DESTINATIONS.items(): if name not in configured: continue accepted = [ioc["id"] for ioc in iocs if ioc.get("type") in spec["accepts"]] if not accepted: continue rows.append({ "id": name, "kind": spec["kind"], "iocs": accepted, # Null until submit settles each payload shape. Writing a # vendor's JSON now would mean guessing an endpoint's contract. # No body_sha256 either: that hash records what was disclosed, # and nothing has been. "body": None, "status": "pending", }) return rows ``` - [ ] **Step 4: Run test to verify it passes** Run: `python3 -m unittest tests.test_report.VendorDestinations -v` Expected: PASS, 7 tests - [ ] **Step 5: Commit** ```bash git add abusectl/report.py tests/test_report.py git commit -S -m "feat: build vendor and MISP destination rows" ``` --- ### Task 3: Reading which destinations are configured **Files:** - Modify: `abusectl/config.py` (add to the `Config` dataclass and `load`) - Test: `tests/test_config.py` **Read before starting:** the addendum's "Config" section. The MISP pair is NOT validated here; Task 6 does that in `report`. `config.load()` reports what it found. Raising here would make `parse` and `contacts` refuse to run over a section neither reads, and `cli.py` catches only `NotConfigured`, so the user would get a traceback. - [ ] **Step 1: Write the failing test** Add to `tests/test_config.py`: ```python class ConfiguredDestinations(unittest.TestCase): def _load(self, text): with tempfile.TemporaryDirectory() as tmp: path = pathlib.Path(tmp) / "config.toml" path.write_text( '[general]\ntrusted_relays = ["192.0.2.0/24"]\n' + text ) return config.load(path) def test_no_sections_means_nothing_configured(self): self.assertEqual(self._load("").destinations, set()) def test_a_vendor_with_a_key_is_configured(self): settings = self._load('[abusedb]\napi_key = "k"\n') self.assertEqual(settings.destinations, {"abusedb"}) def test_a_vendor_section_with_no_key_is_not_configured(self): # Absent reads as not-configured, which is the whole convention. self.assertEqual(self._load("[urlhaus]\n").destinations, set()) def test_an_empty_key_is_not_configured(self): # api_key = "" is the trap this file exists to refuse: it reads as # configured-and-broken and produces an auth error much later. self.assertEqual(self._load('[urlhaus]\napi_key = ""\n').destinations, set()) def test_misp_needs_both_halves(self): both = '[misp]\nurl = "https://misp.example.invalid"\napi_key = "k"\n' self.assertEqual(self._load(both).destinations, {"misp"}) def test_half_a_misp_pair_is_reported_separately_not_configured(self): # Not an exception here: parse and contacts load this same file and # read neither key. report refuses instead, with a sentence. settings = self._load('[misp]\nurl = "https://misp.example.invalid"\n') self.assertEqual(settings.destinations, set()) self.assertEqual(settings.incomplete, {"misp": "api_key"}) def test_the_other_half_is_reported_too(self): settings = self._load('[misp]\napi_key = "k"\n') self.assertEqual(settings.destinations, set()) self.assertEqual(settings.incomplete, {"misp": "url"}) def test_a_non_string_key_is_rejected_naming_the_file(self): with self.assertRaises(ValueError) as caught: self._load("[abusedb]\napi_key = 42\n") self.assertIn("abusedb.api_key", str(caught.exception)) self.assertIn("config.toml", str(caught.exception)) def test_a_non_table_section_is_rejected(self): with self.assertRaises(ValueError) as caught: self._load('abusedb = "nope"\n') self.assertIn("abusedb", str(caught.exception)) ``` - [ ] **Step 2: Run test to verify it fails** Run: `python3 -m unittest tests.test_config.ConfiguredDestinations -v` Expected: FAIL with `AttributeError: 'Config' object has no attribute 'destinations'` - [ ] **Step 3: Write minimal implementation** In `abusectl/config.py`, add the required-keys table after `REPORTER_KEYS`: ```python # What each reporting destination's section must carry to count as # configured. MISP takes a url because it is self-hosted and has no default # endpoint; the vendors' endpoints are fixed and belong to their own # destination modules. Everything ELSE inside these tables is carried # through untouched: submit will want timeouts and category codes, and this # reader deliberately does not design that. DESTINATION_KEYS = { "misp": ("url", "api_key"), "abusedb": ("api_key",), "urlhaus": ("api_key",), } ``` Add two fields to the `Config` dataclass: ```python # Which destinations are fully configured, and which are half-filled. # Reported as data rather than raised: every subcommand loads this file # and most of them read neither key. destinations: set = field(default_factory=set) incomplete: dict = field(default_factory=dict) ``` Add to `load()`, before the `return`: ```python destinations = set() incomplete = {} for name, required in DESTINATION_KEYS.items(): section = data.get(name) if section is None: continue if not isinstance(section, dict): raise ValueError( f"{name} in {source} must be a table, got " f"{type(section).__name__}" ) present = [] for key in required: if key not in section: continue value = section[key] # Rejected, not dropped, for the reason [reporter] records: a # wrong value that reads as plausible is worse than an error. if not isinstance(value, str): raise ValueError( f"{name}.{key} in {source} must be a string, got " f"{type(value).__name__}: {value!r}" ) if value.strip(): present.append(key) if len(present) == len(required): destinations.add(name) elif present: # Some but not all: configured-and-broken rather than skipped. # Absence of the WHOLE section is a skip; absence of one half of # a pair is a mistake, and the command that acts on it says so. missing = [key for key in required if key not in present] incomplete[name] = missing[0] return Config(trusted_relays=trusted_relays, cases=cases, reporter=reporter, destinations=destinations, incomplete=incomplete) ``` Replace the existing `return Config(...)` line with the one above. - [ ] **Step 4: Run test to verify it passes** Run: `python3 -m unittest tests.test_config -v` Expected: PASS, including the 9 new tests and every existing one - [ ] **Step 5: Commit** ```bash git add abusectl/config.py tests/test_config.py git commit -S -m "feat: read which reporting destinations are configured" ``` --- ### Task 4: Wiring vendor rows into `generate()` **Files:** - Modify: `abusectl/report.py:1019` (`generate` signature and body) - Modify: `abusectl/cli.py:445` (the `generate` call) - Test: `tests/test_report.py` **The seam:** `generate()` takes the configured set as an ARGUMENT, the way it already takes the identity and `parse.py` takes the trust boundary. It must not read the config itself. That is what keeps the module pure and testable with no files on disk. - [ ] **Step 1: Write the failing test** Add to `tests/test_report.py`: ```python class GenerateWritesVendorRows(unittest.TestCase): def _manifest(self): return { "format": 1, "case_id": "2026-09-10-test", "headers": [["From", "phish@example.invalid"], ["Subject", "test"], ["Date", "Wed, 09 Sep 2026 09:12:44 +0000"]], "iocs": [ {"id": "ioc-1", "type": "ipv4", "value": "198.51.100.7"}, {"id": "ioc-2", "type": "url", "value": "http://example.invalid/a"}, ], "contacts": [{"ioc": "ioc-1", "iocs": ["ioc-1"], "abuse": ["abuse@example.invalid"]}], } def test_configured_destinations_appear_alongside_the_email_rows(self): with tempfile.TemporaryDirectory() as tmp: out = report.generate(self._manifest(), pathlib.Path(tmp), {"email": "r@example.org"}, configured={"misp", "abusedb"}) kinds = [row["kind"] for row in out["destinations"]] self.assertIn("email", kinds) self.assertIn("misp", kinds) self.assertIn("api", kinds) def test_nothing_configured_leaves_the_email_rows_untouched(self): with tempfile.TemporaryDirectory() as tmp: out = report.generate(self._manifest(), pathlib.Path(tmp), {"email": "r@example.org"}, configured=set()) self.assertEqual([row["kind"] for row in out["destinations"]], ["email"]) def test_a_second_run_produces_the_same_rows(self): with tempfile.TemporaryDirectory() as tmp: first = report.generate(self._manifest(), pathlib.Path(tmp), {"email": "r@example.org"}, configured={"misp"}) ids = [row["id"] for row in first["destinations"]] second = report.generate(first, pathlib.Path(tmp), {"email": "r@example.org"}, configured={"misp"}) self.assertEqual([row["id"] for row in second["destinations"]], ids) def test_vendor_rows_get_no_body_written_to_disk(self): with tempfile.TemporaryDirectory() as tmp: out = report.generate(self._manifest(), pathlib.Path(tmp), {"email": "r@example.org"}, configured={"misp", "abusedb"}) written = sorted(p.name for p in (pathlib.Path(tmp) / "bodies").iterdir()) self.assertTrue(all(name.endswith(".xarf") for name in written)) for row in out["destinations"]: if row["kind"] != "email": self.assertIsNone(row["body"]) self.assertNotIn("body_sha256", row) def test_configured_defaults_to_nothing(self): # An older caller that does not pass it must not crash, and must not # silently gain destinations it never configured. with tempfile.TemporaryDirectory() as tmp: out = report.generate(self._manifest(), pathlib.Path(tmp), {"email": "r@example.org"}) self.assertEqual([row["kind"] for row in out["destinations"]], ["email"]) ``` - [ ] **Step 2: Run test to verify it fails** Run: `python3 -m unittest tests.test_report.GenerateWritesVendorRows -v` Expected: FAIL with `TypeError: generate() got an unexpected keyword argument 'configured'` - [ ] **Step 3: Write minimal implementation** In `abusectl/report.py`, change the `generate` signature: ```python def generate(manifest: dict, case_path: Path, identity: dict, force: bool = False, configured: set | None = None) -> dict: ``` Add to the docstring, after the existing "ORDER MATTERS" paragraph: ``` `configured` is the set of destinations the config carries keys for, an ARGUMENT rather than a config read, so this module stays pure and testable with no files on disk. Defaulting to nothing configured is the safe direction: a caller that forgets it writes no vendor row rather than promising a submission nobody set up. destinations[] is rebuilt WHOLESALE every run, vendor rows included. That is safe only while the freeze rule holds: once a destination has landed the case is frozen and this function refuses to run at all. The freeze marker is written by submit, so this is a constraint on that spec, not merely a description of this one. ``` Replace the two lines at the end that build and assign destinations: ```python destinations = email_destinations(manifest.get("contacts", [])) for destination in destinations: text = build(manifest, destination, identity) relative = f"bodies/{destination['id']}.xarf" _write_body(case_path / relative, text) destination["body"] = relative destination["body_sha256"] = body_hash(text) destinations.extend( vendor_destinations(manifest.get("iocs", []), configured or set()) ) manifest["destinations"] = destinations ``` Then in `abusectl/cli.py`, in `_cmd_report`, change the `generate` call: ```python manifest = report_module.generate( manifest, args.case, identity, force=args.force, configured=settings.destinations, ) ``` - [ ] **Step 4: Run test to verify it passes** Run: `python3 -m unittest discover tests` Expected: PASS, everything - [ ] **Step 5: Mutation check** ```bash find . -name __pycache__ -type d -exec rm -rf {} + cp abusectl/report.py /tmp/report.py.bak ``` Change `configured or set()` to `set()` in `generate`, then: ```bash find . -name __pycache__ -type d -exec rm -rf {} + python3 -m unittest tests.test_report.GenerateWritesVendorRows -v ``` Expected: FAIL on `test_configured_destinations_appear_alongside_the_email_rows`. Restore with `cp /tmp/report.py.bak abusectl/report.py` and clear `__pycache__` again. **Restore from the copy, never with `git checkout`**, which would discard the whole file's work if anything else were uncommitted. - [ ] **Step 6: Commit** ```bash git add abusectl/report.py abusectl/cli.py tests/test_report.py git commit -S -m "feat: write vendor rows into the manifest" ``` --- ### Task 5: The no-destination warning **Files:** - Modify: `abusectl/cli.py` (`_cmd_report`, after the identity check) - Test: `tests/test_cli.py` **Warn and continue, exit zero.** `report` "exits non-zero only when it could not write", per the spec. A case with abuse desks and no vendors is a good report, and the tool is specified to be useful with no API key configured anywhere. - [ ] **Step 1: Write the failing test** Add to `tests/test_cli.py`, inside the existing report-command test class (the one with `_write_config`, `_make_case` and `_run`, around line 275). **Reuse those helpers; do not write new ones.** Note that `--config` is a GLOBAL option and goes BEFORE the subcommand. ```python def test_no_configured_destination_warns_and_still_exits_zero(self): self._write_config('[reporter]\nemail = "r@example.org"\n') case_path = self._make_case() code, _, err = self._run( "--config", str(self.config), "report", str(case_path) ) self.assertEqual(code, 0) self.assertIn("no reporting destinations", err) self.assertIn(str(self.config), err) def test_a_configured_destination_produces_no_warning(self): self._write_config( '[reporter]\nemail = "r@example.org"\n' '\n[abusedb]\napi_key = "k"\n' ) case_path = self._make_case() code, _, err = self._run( "--config", str(self.config), "report", str(case_path) ) self.assertEqual(code, 0) self.assertNotIn("no reporting destinations", err) ``` - [ ] **Step 2: Run test to verify it fails** Run: `python3 -m unittest tests.test_cli.ReportWarnsWithNoDestinations -v` Expected: FAIL on the first test, `AssertionError: 'no reporting destinations' not found in ''` - [ ] **Step 3: Write minimal implementation** In `abusectl/cli.py`, in `_cmd_report`, after the identity check and before `case.load`: ```python # Warned, not refused: a case with abuse-desk destinations and no # vendors is a perfectly good report, and this tool is specified to be # useful with no API key configured anywhere. Not silent either. A user # who believes MISP is configured and finds no misp row has a typo'd # section name, and MISP is the gate for everything irreversible: a case # built with no gate, discovered at submit time, is discovered a step # too late. if not settings.destinations: print( "abusectl report: no reporting destinations configured, so this " "case will reach abuse desks only. Add [misp], [abusedb] or " f"[urlhaus] to {config_path}, or run `abusectl init`.", file=sys.stderr, ) ``` - [ ] **Step 4: Run test to verify it passes** Run: `python3 -m unittest tests.test_cli -v` Expected: PASS, everything - [ ] **Step 5: Commit** ```bash git add abusectl/cli.py tests/test_cli.py git commit -S -m "feat: warn when no reporting destination is configured" ``` --- ### Task 6: Refusing a half-filled MISP pair **Files:** - Modify: `abusectl/cli.py` (`_cmd_report`, beside the warning from Task 5) - Test: `tests/test_cli.py` **Refused by `report`, not by `config.load()`.** The test that matters most here is the second one: `parse` must still run. - [ ] **Step 1: Write the failing test** Add to the same report-command test class in `tests/test_cli.py`, reusing its helpers: ```python def test_a_misp_url_with_no_key_is_refused_naming_the_missing_key(self): self._write_config( '[reporter]\nemail = "r@example.org"\n' '\n[misp]\nurl = "https://misp.example.invalid"\n' ) case_path = self._make_case() code, _, err = self._run( "--config", str(self.config), "report", str(case_path) ) self.assertEqual(code, 3) self.assertIn("api_key", err) self.assertIn(str(self.config), err) def test_half_a_pair_does_not_stop_parse(self): # The half that would regress silently. Every subcommand loads this # file; parse reads neither MISP key and must not be refused over # one, nor traceback, since cli catches only NotConfigured. self.config.write_text( '[general]\ntrusted_relays = ["192.0.2.0/24"]\n' f'cases = "{self.root / "cases"}"\n' '\n[misp]\napi_key = "k"\n', encoding="utf-8", ) code, _, _ = self._run( "--config", str(self.config), "parse", str(FIXTURES / "forged-chain.eml"), ) self.assertEqual(code, 0) ``` `forged-chain.eml` resolves against `trusted_relays = ["192.0.2.0/24"]`, which is what `tests/test_parse.py:42` uses for that fixture. The config here sets `cases` too, since `parse` writes one. - [ ] **Step 2: Run test to verify it fails** Run: `python3 -m unittest tests.test_cli.ReportRefusesHalfAMispPair -v` Expected: FAIL on the first test with `AssertionError: 0 != 3` - [ ] **Step 3: Write minimal implementation** In `abusectl/cli.py`, in `_cmd_report`, BEFORE the no-destination warning from Task 5: ```python # Refused HERE rather than in config.load(), for the reason backlog # item 5 already records one level up: an answer reported somewhere # other than where it is used. Every subcommand loads this file, so # raising in the reader would make `parse` refuse over a section it # never reads, and cli catches only NotConfigured, so the user would # get a traceback rather than a sentence. if settings.incomplete: for name, missing in sorted(settings.incomplete.items()): print( f"abusectl report: [{name}] in {config_path} is missing " f"{missing}, so it is configured-and-broken rather than " "skipped. Add it, or remove the section entirely.", file=sys.stderr, ) return EXIT_NOT_CONFIGURED ``` - [ ] **Step 4: Run test to verify it passes** Run: `python3 -m unittest discover tests` Expected: PASS, everything - [ ] **Step 5: Commit** ```bash git add abusectl/cli.py tests/test_cli.py git commit -S -m "feat: refuse a half-configured destination at report" ``` --- ### Task 7: Rendering the new sections in `init.build()` **Files:** - Modify: `abusectl/init.py` (`build`, `FOOTER`) - Test: `tests/test_init.py` **The footer is wrong today.** It promises `[vendors]` as one table; the spec settles three separate sections. Fix it in this task or it documents a shape that never arrives. - [ ] **Step 1: Write the failing test** Add to `tests/test_init.py`: ```python class BuildsDestinationSections(unittest.TestCase): ANSWERS = {"trusted_relays": ["192.0.2.0/24"]} def test_a_misp_pair_is_rendered(self): text = init.build({**self.ANSWERS, "misp_url": "https://misp.example.invalid", "misp_api_key": "k"}) data = tomllib.loads(text) self.assertEqual(data["misp"]["url"], "https://misp.example.invalid") self.assertEqual(data["misp"]["api_key"], "k") def test_a_vendor_key_is_rendered(self): text = init.build({**self.ANSWERS, "abusedb_api_key": "k"}) self.assertEqual(tomllib.loads(text)["abusedb"]["api_key"], "k") def test_a_skipped_destination_is_absent_not_empty(self): # api_key = "" reads as configured-and-broken. Absent reads as # not-configured and report can say so plainly. text = init.build({**self.ANSWERS, "urlhaus_api_key": ""}) self.assertNotIn("[urlhaus]", text) self.assertNotIn("urlhaus", tomllib.loads(text)) def test_nothing_answered_renders_no_destination_section(self): data = tomllib.loads(init.build(self.ANSWERS)) for name in ("misp", "abusedb", "urlhaus"): self.assertNotIn(name, data) def test_the_footer_no_longer_promises_a_vendors_table(self): # It named [vendors] as one table; the spec settles three sections, # and a footer describing a shape that never arrives is worse than # no footer. self.assertNotIn("[vendors]", init.build(self.ANSWERS)) def test_every_rendered_section_parses(self): text = init.build({**self.ANSWERS, "misp_url": "https://misp.example.invalid", "misp_api_key": "k", "abusedb_api_key": "a", "urlhaus_api_key": "u"}) data = tomllib.loads(text) self.assertEqual(set(data) & {"misp", "abusedb", "urlhaus"}, {"misp", "abusedb", "urlhaus"}) ``` - [ ] **Step 2: Run test to verify it fails** Run: `python3 -m unittest tests.test_init.BuildsDestinationSections -v` Expected: FAIL with `KeyError: 'misp'` on the first test - [ ] **Step 3: Write minimal implementation** In `abusectl/init.py`, replace `FOOTER`: ```python FOOTER = "\n".join([ "", "# Later parts of abusectl add further sections here:", "# [reporting] - abuse-desk reporting defaults", ]) + "\n" ``` Add the destination field table after `_REPORTER_FIELDS`: ```python # Each destination's config keys and the answer key each is asked under. # Rendered in this order so a written file is stable between runs. _DESTINATION_FIELDS = ( ("misp", (("url", "misp_url"), ("api_key", "misp_api_key"))), ("abusedb", (("api_key", "abusedb_api_key"),)), ("urlhaus", (("api_key", "urlhaus_api_key"),)), ) ``` In `build()`, before the final `return`: ```python # A section is emitted only when something was answered for it, the same # rule [reporter] follows: an empty [abusedb] reads as # configured-with-nothing, which is the api_key = "" trap wearing a # different coat. for section, fields in _DESTINATION_FIELDS: rendered = [] for key, answer_key in fields: value = answers.get(answer_key, "") if isinstance(value, str) and value.strip(): rendered.append(f"{key} = {_quoted(value.strip(), key)}") if rendered: lines.extend(["", f"[{section}]"]) lines.extend(rendered) ``` - [ ] **Step 4: Run test to verify it passes** Run: `python3 -m unittest tests.test_init -v` Expected: PASS, everything. The existing carry-across tests (`test_an_unsupplied_section_survives_a_rewrite` and `test_a_rewrite_still_loads`, `tests/test_init.py:261` and `:280`) still pass unchanged: both call `init.write` with only `trusted_relays`, so `build()` renders no `[misp]` and the old section is preserved exactly as before. The behaviour only changes when a run ANSWERS a MISP question, and then the new answer correctly wins, because `_rendered_sections` reads section names off the rendered text. - [ ] **Step 5: Commit** ```bash git add abusectl/init.py tests/test_init.py git commit -S -m "feat: render the destination sections at init" ``` --- ### Task 8: The four prompts **Files:** - Modify: `abusectl/cli.py` (`_prompt_answers`, new `_ask_destinations`) - Test: none. **These are hand-tested by the user**, per `AGENTS.md`: whether a question reads clearly has no assertion, and a test driving stdin asserts the wording it was written against and breaks on a rewording that improved it. - [ ] **Step 1: Write the implementation** In `abusectl/cli.py`, add after `_ask_reporter`: ```python def _ask_destinations() -> dict: """Ask for the reporting destinations, all skippable. The MISP pair is validated TOGETHER and here, at the prompt that asked: a url with no key is configured-and-broken rather than skipped, and finding that out at `report` time costs the whole run. That is this repository's standing prompt rule, learned from a hand test that found four instances of one mistake. Skippable, and they say so, because the relay question and the hop picker now say they are REQUIRED and a user reads the difference. """ print("\nabusectl can also file each case to a threat-intel platform.") print("MISP is your own instance and is written FIRST: it is the record") print("that gates everything irreversible, so a failure there stops the") print("rest. The vendors are public feeds. Leave any blank to skip it.") answers = {} while True: url = _ask("\nMISP instance URL (Enter to skip): ") if not url: answers["misp_url"] = "" answers["misp_api_key"] = "" break if not url.startswith(("http://", "https://")): print(" Expected a URL such as https://misp.example.org.") continue key = _ask("MISP API key (Enter to skip MISP entirely): ") if not key: # Half a pair is worse than none: it reads as configured and # fails at submit. Ask the pair again rather than writing it. print(" A URL with no key cannot be used, so MISP is skipped.") answers["misp_url"] = "" answers["misp_api_key"] = "" break answers["misp_url"] = url answers["misp_api_key"] = key break answers["abusedb_api_key"] = _ask( "\nAbuseIPDB API key, for reporting IPs (Enter to skip): " ) answers["urlhaus_api_key"] = _ask( "URLhaus Auth-Key, for reporting URLs (Enter to skip): " ) if not any(answers.values()): print("\n No destinations set. Cases will reach abuse desks only,") print(" which is a complete report; add them later if you want one.") return answers ``` Change `_prompt_answers` to include them: ```python return {"trusted_relays": trusted_relays, "cases": cases, **_ask_reporter(), **_ask_destinations()} ``` - [ ] **Step 2: Verify the suite still passes** Run: `python3 -m unittest discover tests` Expected: PASS, everything. `tests/test_cli.py` covers dispatch, not prompts. - [ ] **Step 3: Hand-test, and ask the user to run it** ```bash python3 -m abusectl init --config /tmp/handtest.toml ``` Walk every path: skip everything; answer MISP fully; answer a MISP url then skip the key (must skip MISP, not write half); type a bare hostname for the MISP url (must re-ask). Then `cat /tmp/handtest.toml` and confirm no `api_key = ""` anywhere, and `python3 -c "import tomllib; tomllib.load(open('/tmp/handtest.toml','rb'))"` parses. **Ask the user to do this pass themselves.** The last hand test found five defects the suite could not, four of them the same mistake. - [ ] **Step 4: Commit** ```bash git add abusectl/cli.py git commit -S -m "feat: ask for the reporting destinations at init" ``` --- ### Task 9: Documentation **Files:** - Modify: `AGENTS.md` (the architecture block and the config section) - Modify: `README.md` (config example, if it carries one) - [ ] **Step 1: Update AGENTS.md** In the "Config" section, after the two traps, add: ```markdown **A reporting destination is configured only when its section carries every key it needs.** `[abusedb]` and `[urlhaus]` need `api_key`; `[misp]` needs `url` AND `api_key`. Exactly one half of the MISP pair is configured-and-broken rather than skipped, and `config.load()` reports it in `incomplete` rather than raising: every subcommand loads this file, and `parse` must not be refused over a section it never reads. `report` refuses it, because that is the command that acts on it. ``` In the architecture block, extend the `report.py` line: ``` report.py IOCs + contacts -> bodies + destinations[] pure ``` And add after the block: ```markdown **`report.DESTINATIONS` is transcribed from vendor documentation**, read 2026-09-10, the same rule as `init.PROVIDERS`: do not edit it from memory. A wrong `accepts` means a row promising a submission the endpoint will refuse, or a missing row for something the vendor would have taken. VirusTotal is deliberately absent; backlog item 6 records the findings. ``` - [ ] **Step 2: Check the README** ```bash grep -n 'trusted_relays\|\[reporter\]\|config.toml' README.md ``` If it shows a config example, add the three sections to it as commented-out optional entries. If it does not, skip this step. - [ ] **Step 3: Verify the suite one final time** ```bash find . -name __pycache__ -type d -exec rm -rf {} + python3 -m unittest discover tests ``` Expected: OK, with the new tests counted. Note the number: it was 366 before this plan. - [ ] **Step 4: Commit** ```bash git add AGENTS.md README.md git commit -S -m "docs: record the destination table and the configured rule" ``` --- ## What this plan does NOT do Named so nobody adds them unasked: - **No vendor payloads.** Bodies stay null. The shapes belong to the `submit` spec, against each vendor's real documentation, and URLhaus's needs an `auth.abuse.ch` account to read properly. - **No network.** Nothing here opens a socket. `tests/test_offline.py` enumerates modules by import and must keep passing unchanged. - **No sweep.** Both sweeps prove no recipient identifier reaches an IOC or a body. This touches neither `parse.py` nor body content; it adds rows from values the user typed into their own config. Stated because `AGENTS.md` requires a sweep when `parse.py` changes, and a reader should see the question was asked. - **No `submit`, no `retry`, no freeze marker.** The freeze is named as a constraint on the submit spec in the addendum, and stays there.