aboutsummaryrefslogtreecommitdiffstats
path: root/docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md
diff options
context:
space:
mode:
Diffstat (limited to 'docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md')
-rw-r--r--docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md144
1 files changed, 144 insertions, 0 deletions
diff --git a/docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md b/docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md
index fe212e2..4556b48 100644
--- a/docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md
+++ b/docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md
@@ -8791,3 +8791,147 @@ flag set when its own comment said the point was the draft flag "not left
bare". It therefore failed against the corrected behaviour. An
over-specified assertion of this shape blocks the fix rather than the bug.
+## 164. A draft this application saved keeps `inbox`
+
+**Observed (developer, 2026-08-25):** `notmuch search --output=tags` on a
+draft this application had just written reported `draft inbox unread`.
+
+**The first cause recorded here was WRONG, and the correction is the useful
+part.** It said `strip_inbox_from_sent()` reads a sent-only folder list and
+that `qtmaildirconf.py` has no drafts equivalent. Neither is true:
+
+- `NOT_ARRIVALS` is `("sent", "drafts")`, so `sent_folders()` already returns
+ both. The name says "sent" and the contents do not, which is what made the
+ wrong reading plausible.
+- Run against the real config it returns every account's drafts folder.
+- `notmuch count "(<carve-out query>) and id:<the draft>"` returns **1**. The
+ query the hook builds MATCHES the affected message.
+
+So the folder list and the query are correct, and the fix is not there.
+
+**What is actually established.**
+
+- The carve-out is scoped to `SCOPE = "tag:new"` (`post-new:106`).
+- The affected draft carries `inbox`, and `notmuch count tag:new` is **0**.
+- The installed hooks are SYMLINKS into this repository, so the code read is
+ the code that runs. Verified rather than assumed.
+- An mbsync-style rename does **not** re-apply `new.tags`: measured in a
+ throwaway database, a file renamed to add `,U=4` and reindexed kept the tags
+ it had. The "the rename retags it" theory is therefore also out.
+
+**What is NOT established, and must be before any code is written:** which
+pass put `inbox` on this file, and why it was not carrying `tag:new` when the
+hook's carve-out ran. The likely shape is an ordering one, since item 158
+indexes a draft from the application itself, outside `notmuch new`, and a file
+already known to the database is not a new file on the next pass. But that is
+a hypothesis and the last two hypotheses here were both wrong.
+
+**The reproducer was built (2026-08-25) and it settles the mechanism.** Seven
+variants were driven in throwaway databases, modelling `indexDraftFile()` with
+a real `notmuch_database_index_file` call rather than the CLI, because no CLI
+command indexes an untracked path without applying `new.tags`.
+
+What the sweep established, each measured rather than reasoned:
+
+- `index_file` applies **no tags at all**. A draft the application indexes is
+ therefore never in `tag:new` scope, and the hook has nothing to carve out.
+- Whenever the file IS in `tag:new` scope, the carve-out strips `inbox`
+ correctly, in every filename shape tried: `:2,DS`, `:2,D`, no info suffix,
+ in `cur/` and in `new/`, with and without the `,U=4` infix. The real file's
+ shape (`,U=4:2,D`) is among them.
+- It survives the orderings too: `notmuch new` first then the app's index,
+ the app's index first then the rename, an autosave landing between
+ `notmuch new` and the hook, and the stale-path `remove_message` that makes
+ the renamed file arrive as new mail. All six left the draft clean.
+- The `D` flag is what puts `draft` on the message (`synchronize_flags`), and
+ the `S` flag is what removes `unread`. The affected file is `:2,D`, which is
+ why it carries `unread`, and that matches the reported tag set exactly.
+
+**The one variant that reproduces it** is the general shape rather than a
+filename detail: a pass where `inbox` is applied while `tag:new` has ALREADY
+been consumed. Modelled as a file indexed at a path the carve-out does not
+cover and moved into the drafts folder afterwards, it ends in precisely the
+live end state, `draft inbox unread` in Drafts with `,U=4` and `tag:new` at 0.
+Nothing revisits a message once the marker is gone, so the tag is permanent.
+
+**What is still NOT established, and the next step.** The affected account
+writes drafts straight to `<account>/Drafts`, which the carve-out
+covers (verified against the live config and the live query, which matches the
+message by id today), so the reproducing variant's premise does not hold for
+it as written. The live log for the pass that added it reads
+
+ 10:10:52 Added 1 new message to the database. Detected 9 file renames.
+ 10:10:52 post-new: sent-folder carve-out applied over 9 folder(s)
+
+so the hook DID run on that pass, over a path the query covers, and logged
+success. The remaining candidates are all about what the path or the marker
+looked like at that instant, not about the query text: the carve-out logs
+"applied" on a `notmuch tag` that matched zero messages, so a successful log
+line is not evidence the message was in scope. Instrumenting the hook to log
+the carve-out's MATCH COUNT, and leaving it to run until the next draft, is
+the cheapest way to close it, and is a log-only change to code that tags real
+mail unattended.
+
+The filename also rules one thing in: `1787645266.M802P16149Q3.<host>` is
+exactly `MaildirName::fresh()` output, so the application wrote this file. It
+is not a draft another client left behind.
+
+The reproducer scripts are throwaway and were not kept; `indexfile.c` is
+fifteen lines around one `notmuch_database_index_file` call and is trivial to
+rebuild from this entry if the instrumentation points back at the hook.
+
+**Constraints.**
+
+- **The hook tags real mail unattended every ten minutes.** Nothing here is
+ worth a speculative change.
+- The 0.27.0 changelog claims sent mail and drafts both stay out of the inbox.
+ Whatever the cause, that claim is currently false for drafts and the entry
+ needs correcting with the fix.
+- Only `inbox` may be touched. A draft legitimately carries `draft` and
+ `unread`, and `maildir.synchronize_flags` means removing `unread` rewrites
+ the filename and reaches the server.
+- The hook must keep refusing to consume `tag:new` when a carve-out fails.
+- `test_post_new.py` and `test_qtmaildirconf.py` both live beside the hook and
+ have sent-carve-out tests to copy.
+
+---
+
+**RE-MEASURED 2026-08-27, and the item is DROPPED: there was never an `inbox`
+tag on a draft.** Everything above this line is the investigation of a defect
+that did not exist, and it is kept because the way it went wrong is worth more
+than the conclusion.
+
+The premise came from `notmuch search --output=tags`, which reports the union
+over a THREAD. A draft replying to an arrived message sits in that message's
+thread, so the union reads `draft inbox unread` while the two tags live on two
+different messages. Measured today on the thread that produced the original
+report:
+
+- the arrived mail: `['account-<acct>', 'inbox']`
+- the draft reply: `['draft', 'unread']`
+
+Neither carries both. Across the whole index, `notmuch count --output=messages
+'tag:draft and tag:inbox'` is **0** against 12 drafts, nine of which were
+written on or before 2026-08-25 and so were present when this was filed.
+
+**The trap has a second half that makes it much easier to fall into.** A
+thread-level `notmuch count 'tag:draft and tag:inbox'` ALSO returns 0, because
+search terms match per message even in a thread query. So the count and the
+displayed tag list disagree, and the displayed list is the one that looks like
+evidence. Use `--output=messages` and `notmuch show` when asking what tags a
+message carries; `--output=tags` answers a different question than it appears
+to.
+
+This is the same union recorded for `ThreadSummary::tags` under item 110, where
+it made a card claim a tag its message did not have. It cost this item a week
+open, two wrong causes, and a seven-variant reproducer built to explain an end
+state that a union produces for free. It also caught a fresh reader of this
+backlog on 2026-08-27, who read the same union and reported that drafts were
+carrying `inbox` before measuring at message level.
+
+**The `unread` half of the original observation WAS real** and is item 172: the
+app wrote drafts as `:2,D`, and notmuch tags anything without `S` as `unread`.
+That is fixed. The reported tag set `draft inbox unread` is fully explained:
+`unread` from the missing `S` flag on the draft, `inbox` from the arrived
+message sharing its thread.
+