1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
|
# Vendor and MISP Destinations Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make `report` write the `misp` and `api` destination rows its spec has always required, so a case shows every destination it will reach before anything is sent.
**Architecture:** A static table in `report.py` names each destination and the IOC types it accepts, transcribed from vendor documentation the way `init.PROVIDERS` is transcribed from SPF records. `config.py` learns which destinations are configured and reports that as data rather than raising. `report.generate()` appends vendor rows to the email rows it already builds. `init.py` and `cli.py` grow four skippable prompts.
**Tech Stack:** Python 3.11+ stdlib only (`tomllib`, `unittest`). No new dependency: nothing here sends anything.
**Read first:** `docs/specs/2026-09-09-report.md`, the addendum dated 2026-09-10 at the end. It settles every decision below and records why. `AGENTS.md` for the four non-negotiable properties.
---
## Context an engineer needs before starting
**This repository is TDD and the tests are the point.** Write the failing test, run it, watch it fail for the RIGHT reason, then implement. A test that passes before the implementation is a test that proves nothing.
**Clear `__pycache__` between any mutation check.** A stale `.pyc` shadowed edited source in the last session and made a mutation check report nonsense. `python3 -B` does not help; it only stops bytecode being written. Use `find . -name __pycache__ -type d -exec rm -rf {} +`.
**Run the suite with** `python3 -m unittest discover tests`. One module: `python3 -m unittest tests.test_report -v`.
**Commits are GPG-signed.** `git commit -S`. Never `--no-verify`. Global hooks scan for personal data and a rejection is correct until proven otherwise.
**Fixtures use `example.org`, `.invalid` and RFC 5737 documentation ranges only.** Never a real address, domain, or API key, in a test or a commit message.
**What "configured" means throughout this plan:** a config section that is present AND carries every key it requires. `[abusedb]` and `[urlhaus]` require `api_key`. `[misp]` requires BOTH `url` and `api_key`; exactly one of the two is an error, not a skip, and Task 6 handles that separately from Task 3.
**The five IOC type strings `parse.py` actually emits** are `ipv4`, `ipv6`, `domain`, `url` and `sha256`. Verified in `abusectl/parse.py:624`, `:631`, `:647` and `:658`. Do not invent a sixth.
---
## File structure
| File | Responsibility | Change |
|---|---|---|
| `abusectl/report.py` | the destination table, vendor row building, `generate()` | modify |
| `abusectl/config.py` | read which destinations are configured | modify |
| `abusectl/init.py` | render the three new sections, fix the footer | modify |
| `abusectl/cli.py` | the four prompts, the no-destination warning | modify |
| `tests/test_report.py` | vendor rows | modify |
| `tests/test_config.py` | configured-destination reading | modify |
| `tests/test_init.py` | rendering the new sections | modify |
| `tests/test_cli.py` | the warning, the MISP pair refusal | modify |
No new files. The table belongs beside the code that consumes it, as `PROVIDERS` sits in `init.py`.
---
### Task 1: The destination table
**Files:**
- Modify: `abusectl/report.py` (add near the top, after the existing module constants)
- Test: `tests/test_report.py`
- [ ] **Step 1: Write the failing test**
Add to `tests/test_report.py`:
```python
class DestinationTable(unittest.TestCase):
def test_every_destination_names_the_types_it_accepts(self):
self.assertEqual(report.DESTINATIONS["abusedb"]["accepts"],
("ipv4", "ipv6"))
self.assertEqual(report.DESTINATIONS["urlhaus"]["accepts"], ("url",))
self.assertEqual(report.DESTINATIONS["misp"]["accepts"], report.ALL_TYPES)
def test_misp_is_its_own_kind_and_the_vendors_are_api(self):
self.assertEqual(report.DESTINATIONS["misp"]["kind"], "misp")
self.assertEqual(report.DESTINATIONS["abusedb"]["kind"], "api")
self.assertEqual(report.DESTINATIONS["urlhaus"]["kind"], "api")
def test_all_types_covers_what_parse_emits(self):
# The five type strings parse.py writes. A sixth arriving without a
# decision about which destinations accept it would silently reach
# MISP only, so this test names them rather than deriving them.
self.assertEqual(report.ALL_TYPES,
("ipv4", "ipv6", "domain", "url", "sha256"))
```
- [ ] **Step 2: Run test to verify it fails**
Run: `python3 -m unittest tests.test_report.DestinationTable -v`
Expected: FAIL with `AttributeError: module 'abusectl.report' has no attribute 'DESTINATIONS'`
- [ ] **Step 3: Write minimal implementation**
Add to `abusectl/report.py`, after the existing module-level constants:
```python
# Every IOC type parse.py emits. Named rather than derived, so a new type
# arriving without a decision about which destinations accept it is a test
# failure rather than an indicator that quietly reaches MISP alone.
ALL_TYPES = ("ipv4", "ipv6", "domain", "url", "sha256")
# What each destination accepts, read from its own API documentation on
# 2026-09-10:
#
# misp MISP book, circl.lu/doc/misp/automation. Any attribute type.
# abusedb docs.abuseipdb.com, POST /api/v2/report: "a valid IPv4 or
# IPv6 address", and nothing else.
# urlhaus abuse.ch's own submit_url.py, POST urlhaus.abuse.ch/api/,
# whose submission array carries a url. The full docs are
# behind a login at auth.abuse.ch; re-verify the PAYLOAD there
# when submit is built. The accepted type is unambiguous.
#
# VirusTotal is deliberately absent: its only submission endpoints are
# POST /urls and file upload, so it accepts exactly what urlhaus accepts
# and takes no verdict with a submission. See backlog item 6.
#
# DO NOT EDIT THESE FROM MEMORY. init.PROVIDERS records what that costs:
# the first draft was written from memory and every range was wrong. A
# wrong "accepts" here means a row promising a submission the endpoint
# will refuse, or a missing row for something the vendor would have taken.
DESTINATIONS: dict[str, dict] = {
"misp": {"kind": "misp", "accepts": ALL_TYPES},
"abusedb": {"kind": "api", "accepts": ("ipv4", "ipv6")},
"urlhaus": {"kind": "api", "accepts": ("url",)},
}
```
- [ ] **Step 4: Run test to verify it passes**
Run: `python3 -m unittest tests.test_report.DestinationTable -v`
Expected: PASS, 3 tests
- [ ] **Step 5: Commit**
```bash
git add abusectl/report.py tests/test_report.py
git commit -S -m "feat: add the destination table, transcribed from vendor docs"
```
---
### Task 2: Building vendor rows
**Files:**
- Modify: `abusectl/report.py` (new function after `email_destinations`)
- Test: `tests/test_report.py`
- [ ] **Step 1: Write the failing test**
Add to `tests/test_report.py`:
```python
class VendorDestinations(unittest.TestCase):
IOCS = [
{"id": "ioc-1", "type": "ipv4", "value": "198.51.100.7"},
{"id": "ioc-2", "type": "domain", "value": "example.invalid"},
{"id": "ioc-3", "type": "url", "value": "http://example.invalid/a"},
]
def test_a_row_carries_only_the_types_its_destination_accepts(self):
rows = report.vendor_destinations(self.IOCS, {"abusedb", "urlhaus"})
by_id = {row["id"]: row for row in rows}
self.assertEqual(by_id["abusedb"]["iocs"], ["ioc-1"])
self.assertEqual(by_id["urlhaus"]["iocs"], ["ioc-3"])
def test_misp_carries_every_ioc_including_what_no_vendor_takes(self):
rows = report.vendor_destinations(self.IOCS, {"misp"})
self.assertEqual(rows[0]["iocs"], ["ioc-1", "ioc-2", "ioc-3"])
def test_a_destination_with_no_acceptable_ioc_gets_no_row(self):
# The rule's whole point: a case with no URL must not promise a
# urlhaus submission that would have nothing to submit.
no_urls = [ioc for ioc in self.IOCS if ioc["type"] != "url"]
rows = report.vendor_destinations(no_urls, {"abusedb", "urlhaus"})
self.assertEqual([row["id"] for row in rows], ["abusedb"])
def test_an_unconfigured_destination_gets_no_row(self):
rows = report.vendor_destinations(self.IOCS, set())
self.assertEqual(rows, [])
def test_the_row_shape_is_the_shape_submit_iterates(self):
rows = report.vendor_destinations(self.IOCS, {"abusedb"})
self.assertEqual(rows[0], {
"id": "abusedb",
"kind": "api",
"iocs": ["ioc-1"],
"body": None,
"status": "pending",
})
def test_no_row_carries_a_target(self):
# An endpoint is a property of the vendor, not of the case. Writing
# one into the evidence record would let a hand-edit redirect a
# submission somewhere the user never named.
rows = report.vendor_destinations(self.IOCS, {"misp", "abusedb"})
for row in rows:
self.assertNotIn("target", row)
def test_rows_come_out_in_table_order_not_set_order(self):
# The configured set is a set, so a stable order has to come from
# somewhere else or the manifest churns between runs.
first = report.vendor_destinations(self.IOCS, {"urlhaus", "misp", "abusedb"})
second = report.vendor_destinations(self.IOCS, {"abusedb", "misp", "urlhaus"})
self.assertEqual([row["id"] for row in first],
[row["id"] for row in second])
self.assertEqual([row["id"] for row in first],
["misp", "abusedb", "urlhaus"])
```
- [ ] **Step 2: Run test to verify it fails**
Run: `python3 -m unittest tests.test_report.VendorDestinations -v`
Expected: FAIL with `AttributeError: module 'abusectl.report' has no attribute 'vendor_destinations'`
- [ ] **Step 3: Write minimal implementation**
Add to `abusectl/report.py`, after `email_destinations`:
```python
def vendor_destinations(iocs: list[dict], configured: set) -> list[dict]:
"""Build one row per configured destination that has something to send.
BOTH conditions have to hold. A row for an unconfigured destination is a
promise that can only fail; a row for a configured one with nothing it
accepts is a promise with no content, an AbuseIPDB submission with
nothing to put in its `ip` parameter.
Each row carries ONLY the IOCs its own destination accepts. A row is
what submit iterates, so a urlhaus row listing an IP is a submission
that gets built wrong or dropped at send time, whichever the
implementer notices first.
Order comes from DESTINATIONS rather than from `configured`, which is a
set and therefore has no order worth writing into a manifest twice.
"""
rows = []
for name, spec in DESTINATIONS.items():
if name not in configured:
continue
accepted = [ioc["id"] for ioc in iocs if ioc.get("type") in spec["accepts"]]
if not accepted:
continue
rows.append({
"id": name,
"kind": spec["kind"],
"iocs": accepted,
# Null until submit settles each payload shape. Writing a
# vendor's JSON now would mean guessing an endpoint's contract.
# No body_sha256 either: that hash records what was disclosed,
# and nothing has been.
"body": None,
"status": "pending",
})
return rows
```
- [ ] **Step 4: Run test to verify it passes**
Run: `python3 -m unittest tests.test_report.VendorDestinations -v`
Expected: PASS, 7 tests
- [ ] **Step 5: Commit**
```bash
git add abusectl/report.py tests/test_report.py
git commit -S -m "feat: build vendor and MISP destination rows"
```
---
### Task 3: Reading which destinations are configured
**Files:**
- Modify: `abusectl/config.py` (add to the `Config` dataclass and `load`)
- Test: `tests/test_config.py`
**Read before starting:** the addendum's "Config" section. The MISP pair is NOT validated here; Task 6 does that in `report`. `config.load()` reports what it found. Raising here would make `parse` and `contacts` refuse to run over a section neither reads, and `cli.py` catches only `NotConfigured`, so the user would get a traceback.
- [ ] **Step 1: Write the failing test**
Add to `tests/test_config.py`:
```python
class ConfiguredDestinations(unittest.TestCase):
def _load(self, text):
with tempfile.TemporaryDirectory() as tmp:
path = pathlib.Path(tmp) / "config.toml"
path.write_text(
'[general]\ntrusted_relays = ["192.0.2.0/24"]\n' + text
)
return config.load(path)
def test_no_sections_means_nothing_configured(self):
self.assertEqual(self._load("").destinations, set())
def test_a_vendor_with_a_key_is_configured(self):
settings = self._load('[abusedb]\napi_key = "k"\n')
self.assertEqual(settings.destinations, {"abusedb"})
def test_a_vendor_section_with_no_key_is_not_configured(self):
# Absent reads as not-configured, which is the whole convention.
self.assertEqual(self._load("[urlhaus]\n").destinations, set())
def test_an_empty_key_is_not_configured(self):
# api_key = "" is the trap this file exists to refuse: it reads as
# configured-and-broken and produces an auth error much later.
self.assertEqual(self._load('[urlhaus]\napi_key = ""\n').destinations,
set())
def test_misp_needs_both_halves(self):
both = '[misp]\nurl = "https://misp.example.invalid"\napi_key = "k"\n'
self.assertEqual(self._load(both).destinations, {"misp"})
def test_half_a_misp_pair_is_reported_separately_not_configured(self):
# Not an exception here: parse and contacts load this same file and
# read neither key. report refuses instead, with a sentence.
settings = self._load('[misp]\nurl = "https://misp.example.invalid"\n')
self.assertEqual(settings.destinations, set())
self.assertEqual(settings.incomplete, {"misp": "api_key"})
def test_the_other_half_is_reported_too(self):
settings = self._load('[misp]\napi_key = "k"\n')
self.assertEqual(settings.destinations, set())
self.assertEqual(settings.incomplete, {"misp": "url"})
def test_a_non_string_key_is_rejected_naming_the_file(self):
with self.assertRaises(ValueError) as caught:
self._load("[abusedb]\napi_key = 42\n")
self.assertIn("abusedb.api_key", str(caught.exception))
self.assertIn("config.toml", str(caught.exception))
def test_a_non_table_section_is_rejected(self):
with self.assertRaises(ValueError) as caught:
self._load('abusedb = "nope"\n')
self.assertIn("abusedb", str(caught.exception))
```
- [ ] **Step 2: Run test to verify it fails**
Run: `python3 -m unittest tests.test_config.ConfiguredDestinations -v`
Expected: FAIL with `AttributeError: 'Config' object has no attribute 'destinations'`
- [ ] **Step 3: Write minimal implementation**
In `abusectl/config.py`, add the required-keys table after `REPORTER_KEYS`:
```python
# What each reporting destination's section must carry to count as
# configured. MISP takes a url because it is self-hosted and has no default
# endpoint; the vendors' endpoints are fixed and belong to their own
# destination modules. Everything ELSE inside these tables is carried
# through untouched: submit will want timeouts and category codes, and this
# reader deliberately does not design that.
DESTINATION_KEYS = {
"misp": ("url", "api_key"),
"abusedb": ("api_key",),
"urlhaus": ("api_key",),
}
```
Add two fields to the `Config` dataclass:
```python
# Which destinations are fully configured, and which are half-filled.
# Reported as data rather than raised: every subcommand loads this file
# and most of them read neither key.
destinations: set = field(default_factory=set)
incomplete: dict = field(default_factory=dict)
```
Add to `load()`, before the `return`:
```python
destinations = set()
incomplete = {}
for name, required in DESTINATION_KEYS.items():
section = data.get(name)
if section is None:
continue
if not isinstance(section, dict):
raise ValueError(
f"{name} in {source} must be a table, got "
f"{type(section).__name__}"
)
present = []
for key in required:
if key not in section:
continue
value = section[key]
# Rejected, not dropped, for the reason [reporter] records: a
# wrong value that reads as plausible is worse than an error.
if not isinstance(value, str):
raise ValueError(
f"{name}.{key} in {source} must be a string, got "
f"{type(value).__name__}: {value!r}"
)
if value.strip():
present.append(key)
if len(present) == len(required):
destinations.add(name)
elif present:
# Some but not all: configured-and-broken rather than skipped.
# Absence of the WHOLE section is a skip; absence of one half of
# a pair is a mistake, and the command that acts on it says so.
missing = [key for key in required if key not in present]
incomplete[name] = missing[0]
return Config(trusted_relays=trusted_relays, cases=cases,
reporter=reporter, destinations=destinations,
incomplete=incomplete)
```
Replace the existing `return Config(...)` line with the one above.
- [ ] **Step 4: Run test to verify it passes**
Run: `python3 -m unittest tests.test_config -v`
Expected: PASS, including the 9 new tests and every existing one
- [ ] **Step 5: Commit**
```bash
git add abusectl/config.py tests/test_config.py
git commit -S -m "feat: read which reporting destinations are configured"
```
---
### Task 4: Wiring vendor rows into `generate()`
**Files:**
- Modify: `abusectl/report.py:1019` (`generate` signature and body)
- Modify: `abusectl/cli.py:445` (the `generate` call)
- Test: `tests/test_report.py`
**The seam:** `generate()` takes the configured set as an ARGUMENT, the way it already takes the identity and `parse.py` takes the trust boundary. It must not read the config itself. That is what keeps the module pure and testable with no files on disk.
- [ ] **Step 1: Write the failing test**
Add to `tests/test_report.py`:
```python
class GenerateWritesVendorRows(unittest.TestCase):
def _manifest(self):
return {
"format": 1,
"case_id": "2026-09-10-test",
"headers": [["From", "phish@example.invalid"],
["Subject", "test"],
["Date", "Wed, 09 Sep 2026 09:12:44 +0000"]],
"iocs": [
{"id": "ioc-1", "type": "ipv4", "value": "198.51.100.7"},
{"id": "ioc-2", "type": "url",
"value": "http://example.invalid/a"},
],
"contacts": [{"ioc": "ioc-1", "iocs": ["ioc-1"],
"abuse": ["abuse@example.invalid"]}],
}
def test_configured_destinations_appear_alongside_the_email_rows(self):
with tempfile.TemporaryDirectory() as tmp:
out = report.generate(self._manifest(), pathlib.Path(tmp),
{"email": "r@example.org"},
configured={"misp", "abusedb"})
kinds = [row["kind"] for row in out["destinations"]]
self.assertIn("email", kinds)
self.assertIn("misp", kinds)
self.assertIn("api", kinds)
def test_nothing_configured_leaves_the_email_rows_untouched(self):
with tempfile.TemporaryDirectory() as tmp:
out = report.generate(self._manifest(), pathlib.Path(tmp),
{"email": "r@example.org"},
configured=set())
self.assertEqual([row["kind"] for row in out["destinations"]], ["email"])
def test_a_second_run_produces_the_same_rows(self):
with tempfile.TemporaryDirectory() as tmp:
first = report.generate(self._manifest(), pathlib.Path(tmp),
{"email": "r@example.org"},
configured={"misp"})
ids = [row["id"] for row in first["destinations"]]
second = report.generate(first, pathlib.Path(tmp),
{"email": "r@example.org"},
configured={"misp"})
self.assertEqual([row["id"] for row in second["destinations"]], ids)
def test_vendor_rows_get_no_body_written_to_disk(self):
with tempfile.TemporaryDirectory() as tmp:
out = report.generate(self._manifest(), pathlib.Path(tmp),
{"email": "r@example.org"},
configured={"misp", "abusedb"})
written = sorted(p.name for p in (pathlib.Path(tmp) / "bodies").iterdir())
self.assertTrue(all(name.endswith(".xarf") for name in written))
for row in out["destinations"]:
if row["kind"] != "email":
self.assertIsNone(row["body"])
self.assertNotIn("body_sha256", row)
def test_configured_defaults_to_nothing(self):
# An older caller that does not pass it must not crash, and must not
# silently gain destinations it never configured.
with tempfile.TemporaryDirectory() as tmp:
out = report.generate(self._manifest(), pathlib.Path(tmp),
{"email": "r@example.org"})
self.assertEqual([row["kind"] for row in out["destinations"]], ["email"])
```
- [ ] **Step 2: Run test to verify it fails**
Run: `python3 -m unittest tests.test_report.GenerateWritesVendorRows -v`
Expected: FAIL with `TypeError: generate() got an unexpected keyword argument 'configured'`
- [ ] **Step 3: Write minimal implementation**
In `abusectl/report.py`, change the `generate` signature:
```python
def generate(manifest: dict, case_path: Path, identity: dict,
force: bool = False, configured: set | None = None) -> dict:
```
Add to the docstring, after the existing "ORDER MATTERS" paragraph:
```
`configured` is the set of destinations the config carries keys for, an
ARGUMENT rather than a config read, so this module stays pure and
testable with no files on disk. Defaulting to nothing configured is the
safe direction: a caller that forgets it writes no vendor row rather
than promising a submission nobody set up.
destinations[] is rebuilt WHOLESALE every run, vendor rows included.
That is safe only while the freeze rule holds: once a destination has
landed the case is frozen and this function refuses to run at all. The
freeze marker is written by submit, so this is a constraint on that
spec, not merely a description of this one.
```
Replace the two lines at the end that build and assign destinations:
```python
destinations = email_destinations(manifest.get("contacts", []))
for destination in destinations:
text = build(manifest, destination, identity)
relative = f"bodies/{destination['id']}.xarf"
_write_body(case_path / relative, text)
destination["body"] = relative
destination["body_sha256"] = body_hash(text)
destinations.extend(
vendor_destinations(manifest.get("iocs", []), configured or set())
)
manifest["destinations"] = destinations
```
Then in `abusectl/cli.py`, in `_cmd_report`, change the `generate` call:
```python
manifest = report_module.generate(
manifest, args.case, identity, force=args.force,
configured=settings.destinations,
)
```
- [ ] **Step 4: Run test to verify it passes**
Run: `python3 -m unittest discover tests`
Expected: PASS, everything
- [ ] **Step 5: Mutation check**
```bash
find . -name __pycache__ -type d -exec rm -rf {} +
cp abusectl/report.py /tmp/report.py.bak
```
Change `configured or set()` to `set()` in `generate`, then:
```bash
find . -name __pycache__ -type d -exec rm -rf {} +
python3 -m unittest tests.test_report.GenerateWritesVendorRows -v
```
Expected: FAIL on `test_configured_destinations_appear_alongside_the_email_rows`. Restore with `cp /tmp/report.py.bak abusectl/report.py` and clear `__pycache__` again. **Restore from the copy, never with `git checkout`**, which would discard the whole file's work if anything else were uncommitted.
- [ ] **Step 6: Commit**
```bash
git add abusectl/report.py abusectl/cli.py tests/test_report.py
git commit -S -m "feat: write vendor rows into the manifest"
```
---
### Task 5: The no-destination warning
**Files:**
- Modify: `abusectl/cli.py` (`_cmd_report`, after the identity check)
- Test: `tests/test_cli.py`
**Warn and continue, exit zero.** `report` "exits non-zero only when it could not write", per the spec. A case with abuse desks and no vendors is a good report, and the tool is specified to be useful with no API key configured anywhere.
- [ ] **Step 1: Write the failing test**
Add to `tests/test_cli.py`, inside the existing report-command test class (the one with `_write_config`, `_make_case` and `_run`, around line 275). **Reuse those helpers; do not write new ones.** Note that `--config` is a GLOBAL option and goes BEFORE the subcommand.
```python
def test_no_configured_destination_warns_and_still_exits_zero(self):
self._write_config('[reporter]\nemail = "r@example.org"\n')
case_path = self._make_case()
code, _, err = self._run(
"--config", str(self.config), "report", str(case_path)
)
self.assertEqual(code, 0)
self.assertIn("no reporting destinations", err)
self.assertIn(str(self.config), err)
def test_a_configured_destination_produces_no_warning(self):
self._write_config(
'[reporter]\nemail = "r@example.org"\n'
'\n[abusedb]\napi_key = "k"\n'
)
case_path = self._make_case()
code, _, err = self._run(
"--config", str(self.config), "report", str(case_path)
)
self.assertEqual(code, 0)
self.assertNotIn("no reporting destinations", err)
```
- [ ] **Step 2: Run test to verify it fails**
Run: `python3 -m unittest tests.test_cli.ReportWarnsWithNoDestinations -v`
Expected: FAIL on the first test, `AssertionError: 'no reporting destinations' not found in ''`
- [ ] **Step 3: Write minimal implementation**
In `abusectl/cli.py`, in `_cmd_report`, after the identity check and before `case.load`:
```python
# Warned, not refused: a case with abuse-desk destinations and no
# vendors is a perfectly good report, and this tool is specified to be
# useful with no API key configured anywhere. Not silent either. A user
# who believes MISP is configured and finds no misp row has a typo'd
# section name, and MISP is the gate for everything irreversible: a case
# built with no gate, discovered at submit time, is discovered a step
# too late.
if not settings.destinations:
print(
"abusectl report: no reporting destinations configured, so this "
"case will reach abuse desks only. Add [misp], [abusedb] or "
f"[urlhaus] to {config_path}, or run `abusectl init`.",
file=sys.stderr,
)
```
- [ ] **Step 4: Run test to verify it passes**
Run: `python3 -m unittest tests.test_cli -v`
Expected: PASS, everything
- [ ] **Step 5: Commit**
```bash
git add abusectl/cli.py tests/test_cli.py
git commit -S -m "feat: warn when no reporting destination is configured"
```
---
### Task 6: Refusing a half-filled MISP pair
**Files:**
- Modify: `abusectl/cli.py` (`_cmd_report`, beside the warning from Task 5)
- Test: `tests/test_cli.py`
**Refused by `report`, not by `config.load()`.** The test that matters most here is the second one: `parse` must still run.
- [ ] **Step 1: Write the failing test**
Add to the same report-command test class in `tests/test_cli.py`, reusing its helpers:
```python
def test_a_misp_url_with_no_key_is_refused_naming_the_missing_key(self):
self._write_config(
'[reporter]\nemail = "r@example.org"\n'
'\n[misp]\nurl = "https://misp.example.invalid"\n'
)
case_path = self._make_case()
code, _, err = self._run(
"--config", str(self.config), "report", str(case_path)
)
self.assertEqual(code, 3)
self.assertIn("api_key", err)
self.assertIn(str(self.config), err)
def test_half_a_pair_does_not_stop_parse(self):
# The half that would regress silently. Every subcommand loads this
# file; parse reads neither MISP key and must not be refused over
# one, nor traceback, since cli catches only NotConfigured.
self.config.write_text(
'[general]\ntrusted_relays = ["192.0.2.0/24"]\n'
f'cases = "{self.root / "cases"}"\n'
'\n[misp]\napi_key = "k"\n',
encoding="utf-8",
)
code, _, _ = self._run(
"--config", str(self.config), "parse",
str(FIXTURES / "forged-chain.eml"),
)
self.assertEqual(code, 0)
```
`forged-chain.eml` resolves against `trusted_relays = ["192.0.2.0/24"]`, which is what `tests/test_parse.py:42` uses for that fixture. The config here sets `cases` too, since `parse` writes one.
- [ ] **Step 2: Run test to verify it fails**
Run: `python3 -m unittest tests.test_cli.ReportRefusesHalfAMispPair -v`
Expected: FAIL on the first test with `AssertionError: 0 != 3`
- [ ] **Step 3: Write minimal implementation**
In `abusectl/cli.py`, in `_cmd_report`, BEFORE the no-destination warning from Task 5:
```python
# Refused HERE rather than in config.load(), for the reason backlog
# item 5 already records one level up: an answer reported somewhere
# other than where it is used. Every subcommand loads this file, so
# raising in the reader would make `parse` refuse over a section it
# never reads, and cli catches only NotConfigured, so the user would
# get a traceback rather than a sentence.
if settings.incomplete:
for name, missing in sorted(settings.incomplete.items()):
print(
f"abusectl report: [{name}] in {config_path} is missing "
f"{missing}, so it is configured-and-broken rather than "
"skipped. Add it, or remove the section entirely.",
file=sys.stderr,
)
return EXIT_NOT_CONFIGURED
```
- [ ] **Step 4: Run test to verify it passes**
Run: `python3 -m unittest discover tests`
Expected: PASS, everything
- [ ] **Step 5: Commit**
```bash
git add abusectl/cli.py tests/test_cli.py
git commit -S -m "feat: refuse a half-configured destination at report"
```
---
### Task 7: Rendering the new sections in `init.build()`
**Files:**
- Modify: `abusectl/init.py` (`build`, `FOOTER`)
- Test: `tests/test_init.py`
**The footer is wrong today.** It promises `[vendors]` as one table; the spec settles three separate sections. Fix it in this task or it documents a shape that never arrives.
- [ ] **Step 1: Write the failing test**
Add to `tests/test_init.py`:
```python
class BuildsDestinationSections(unittest.TestCase):
ANSWERS = {"trusted_relays": ["192.0.2.0/24"]}
def test_a_misp_pair_is_rendered(self):
text = init.build({**self.ANSWERS,
"misp_url": "https://misp.example.invalid",
"misp_api_key": "k"})
data = tomllib.loads(text)
self.assertEqual(data["misp"]["url"], "https://misp.example.invalid")
self.assertEqual(data["misp"]["api_key"], "k")
def test_a_vendor_key_is_rendered(self):
text = init.build({**self.ANSWERS, "abusedb_api_key": "k"})
self.assertEqual(tomllib.loads(text)["abusedb"]["api_key"], "k")
def test_a_skipped_destination_is_absent_not_empty(self):
# api_key = "" reads as configured-and-broken. Absent reads as
# not-configured and report can say so plainly.
text = init.build({**self.ANSWERS, "urlhaus_api_key": ""})
self.assertNotIn("[urlhaus]", text)
self.assertNotIn("urlhaus", tomllib.loads(text))
def test_nothing_answered_renders_no_destination_section(self):
data = tomllib.loads(init.build(self.ANSWERS))
for name in ("misp", "abusedb", "urlhaus"):
self.assertNotIn(name, data)
def test_the_footer_no_longer_promises_a_vendors_table(self):
# It named [vendors] as one table; the spec settles three sections,
# and a footer describing a shape that never arrives is worse than
# no footer.
self.assertNotIn("[vendors]", init.build(self.ANSWERS))
def test_every_rendered_section_parses(self):
text = init.build({**self.ANSWERS,
"misp_url": "https://misp.example.invalid",
"misp_api_key": "k",
"abusedb_api_key": "a",
"urlhaus_api_key": "u"})
data = tomllib.loads(text)
self.assertEqual(set(data) & {"misp", "abusedb", "urlhaus"},
{"misp", "abusedb", "urlhaus"})
```
- [ ] **Step 2: Run test to verify it fails**
Run: `python3 -m unittest tests.test_init.BuildsDestinationSections -v`
Expected: FAIL with `KeyError: 'misp'` on the first test
- [ ] **Step 3: Write minimal implementation**
In `abusectl/init.py`, replace `FOOTER`:
```python
FOOTER = "\n".join([
"",
"# Later parts of abusectl add further sections here:",
"# [reporting] - abuse-desk reporting defaults",
]) + "\n"
```
Add the destination field table after `_REPORTER_FIELDS`:
```python
# Each destination's config keys and the answer key each is asked under.
# Rendered in this order so a written file is stable between runs.
_DESTINATION_FIELDS = (
("misp", (("url", "misp_url"), ("api_key", "misp_api_key"))),
("abusedb", (("api_key", "abusedb_api_key"),)),
("urlhaus", (("api_key", "urlhaus_api_key"),)),
)
```
In `build()`, before the final `return`:
```python
# A section is emitted only when something was answered for it, the same
# rule [reporter] follows: an empty [abusedb] reads as
# configured-with-nothing, which is the api_key = "" trap wearing a
# different coat.
for section, fields in _DESTINATION_FIELDS:
rendered = []
for key, answer_key in fields:
value = answers.get(answer_key, "")
if isinstance(value, str) and value.strip():
rendered.append(f"{key} = {_quoted(value.strip(), key)}")
if rendered:
lines.extend(["", f"[{section}]"])
lines.extend(rendered)
```
- [ ] **Step 4: Run test to verify it passes**
Run: `python3 -m unittest tests.test_init -v`
Expected: PASS, everything.
The existing carry-across tests (`test_an_unsupplied_section_survives_a_rewrite` and `test_a_rewrite_still_loads`, `tests/test_init.py:261` and `:280`) still pass unchanged: both call `init.write` with only `trusted_relays`, so `build()` renders no `[misp]` and the old section is preserved exactly as before. The behaviour only changes when a run ANSWERS a MISP question, and then the new answer correctly wins, because `_rendered_sections` reads section names off the rendered text.
- [ ] **Step 5: Commit**
```bash
git add abusectl/init.py tests/test_init.py
git commit -S -m "feat: render the destination sections at init"
```
---
### Task 8: The four prompts
**Files:**
- Modify: `abusectl/cli.py` (`_prompt_answers`, new `_ask_destinations`)
- Test: none. **These are hand-tested by the user**, per `AGENTS.md`: whether a question reads clearly has no assertion, and a test driving stdin asserts the wording it was written against and breaks on a rewording that improved it.
- [ ] **Step 1: Write the implementation**
In `abusectl/cli.py`, add after `_ask_reporter`:
```python
def _ask_destinations() -> dict:
"""Ask for the reporting destinations, all skippable.
The MISP pair is validated TOGETHER and here, at the prompt that asked:
a url with no key is configured-and-broken rather than skipped, and
finding that out at `report` time costs the whole run. That is this
repository's standing prompt rule, learned from a hand test that found
four instances of one mistake.
Skippable, and they say so, because the relay question and the hop
picker now say they are REQUIRED and a user reads the difference.
"""
print("\nabusectl can also file each case to a threat-intel platform.")
print("MISP is your own instance and is written FIRST: it is the record")
print("that gates everything irreversible, so a failure there stops the")
print("rest. The vendors are public feeds. Leave any blank to skip it.")
answers = {}
while True:
url = _ask("\nMISP instance URL (Enter to skip): ")
if not url:
answers["misp_url"] = ""
answers["misp_api_key"] = ""
break
if not url.startswith(("http://", "https://")):
print(" Expected a URL such as https://misp.example.org.")
continue
key = _ask("MISP API key (Enter to skip MISP entirely): ")
if not key:
# Half a pair is worse than none: it reads as configured and
# fails at submit. Ask the pair again rather than writing it.
print(" A URL with no key cannot be used, so MISP is skipped.")
answers["misp_url"] = ""
answers["misp_api_key"] = ""
break
answers["misp_url"] = url
answers["misp_api_key"] = key
break
answers["abusedb_api_key"] = _ask(
"\nAbuseIPDB API key, for reporting IPs (Enter to skip): "
)
answers["urlhaus_api_key"] = _ask(
"URLhaus Auth-Key, for reporting URLs (Enter to skip): "
)
if not any(answers.values()):
print("\n No destinations set. Cases will reach abuse desks only,")
print(" which is a complete report; add them later if you want one.")
return answers
```
Change `_prompt_answers` to include them:
```python
return {"trusted_relays": trusted_relays, "cases": cases,
**_ask_reporter(), **_ask_destinations()}
```
- [ ] **Step 2: Verify the suite still passes**
Run: `python3 -m unittest discover tests`
Expected: PASS, everything. `tests/test_cli.py` covers dispatch, not prompts.
- [ ] **Step 3: Hand-test, and ask the user to run it**
```bash
python3 -m abusectl init --config /tmp/handtest.toml
```
Walk every path: skip everything; answer MISP fully; answer a MISP url then skip the key (must skip MISP, not write half); type a bare hostname for the MISP url (must re-ask). Then `cat /tmp/handtest.toml` and confirm no `api_key = ""` anywhere, and `python3 -c "import tomllib; tomllib.load(open('/tmp/handtest.toml','rb'))"` parses.
**Ask the user to do this pass themselves.** The last hand test found five defects the suite could not, four of them the same mistake.
- [ ] **Step 4: Commit**
```bash
git add abusectl/cli.py
git commit -S -m "feat: ask for the reporting destinations at init"
```
---
### Task 9: Documentation
**Files:**
- Modify: `AGENTS.md` (the architecture block and the config section)
- Modify: `README.md` (config example, if it carries one)
- [ ] **Step 1: Update AGENTS.md**
In the "Config" section, after the two traps, add:
```markdown
**A reporting destination is configured only when its section carries every
key it needs.** `[abusedb]` and `[urlhaus]` need `api_key`; `[misp]` needs
`url` AND `api_key`. Exactly one half of the MISP pair is
configured-and-broken rather than skipped, and `config.load()` reports it in
`incomplete` rather than raising: every subcommand loads this file, and
`parse` must not be refused over a section it never reads. `report` refuses
it, because that is the command that acts on it.
```
In the architecture block, extend the `report.py` line:
```
report.py IOCs + contacts -> bodies + destinations[] pure
```
And add after the block:
```markdown
**`report.DESTINATIONS` is transcribed from vendor documentation**, read
2026-09-10, the same rule as `init.PROVIDERS`: do not edit it from memory. A
wrong `accepts` means a row promising a submission the endpoint will refuse,
or a missing row for something the vendor would have taken. VirusTotal is
deliberately absent; backlog item 6 records the findings.
```
- [ ] **Step 2: Check the README**
```bash
grep -n 'trusted_relays\|\[reporter\]\|config.toml' README.md
```
If it shows a config example, add the three sections to it as commented-out optional entries. If it does not, skip this step.
- [ ] **Step 3: Verify the suite one final time**
```bash
find . -name __pycache__ -type d -exec rm -rf {} +
python3 -m unittest discover tests
```
Expected: OK, with the new tests counted. Note the number: it was 366 before this plan.
- [ ] **Step 4: Commit**
```bash
git add AGENTS.md README.md
git commit -S -m "docs: record the destination table and the configured rule"
```
---
## What this plan does NOT do
Named so nobody adds them unasked:
- **No vendor payloads.** Bodies stay null. The shapes belong to the `submit` spec, against each vendor's real documentation, and URLhaus's needs an `auth.abuse.ch` account to read properly.
- **No network.** Nothing here opens a socket. `tests/test_offline.py` enumerates modules by import and must keep passing unchanged.
- **No sweep.** Both sweeps prove no recipient identifier reaches an IOC or a body. This touches neither `parse.py` nor body content; it adds rows from values the user typed into their own config. Stated because `AGENTS.md` requires a sweep when `parse.py` changes, and a reader should see the question was asked.
- **No `submit`, no `retry`, no freeze marker.** The freeze is named as a constraint on the submit spec in the addendum, and stays there.
|