diff options
| author | Danilo M. <danix@danix.xyz> | 2026-08-27 11:55:59 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-08-27 11:55:59 +0200 |
| commit | 12e841b8e2c4c225ea79de87dc7bb50f0404ee69 (patch) | |
| tree | deec92b59b77ea7c85c11c351bf1d67ed0195137 /docs/superpowers/plans | |
| parent | edbf393cb79d20764fd224d32a195f59d19ef743 (diff) | |
| download | qtmaildir-12e841b8e2c4c225ea79de87dc7bb50f0404ee69.tar.gz qtmaildir-12e841b8e2c4c225ea79de87dc7bb50f0404ee69.zip | |
docs: drop item 164, the inbox tag on a draft was never there
Re-measured at message level: 0 of 12 drafts carry `inbox`, including
nine written on or before 2026-08-25, when the item was filed. The
thread that produced the original report splits into an arrived message
tagged `inbox` and a draft reply tagged `draft unread`; neither carries
both.
The premise came from `notmuch search --output=tags`, which displays the
union over a thread. The trap has a second half: a thread-level
`notmuch count 'tag:draft and tag:inbox'` also returns 0, because search
terms match per message even in a thread query, so the count and the
displayed tag list disagree and the displayed list is the one that looks
like evidence.
The investigation is kept above the correction rather than deleted: it
cost a week open, two wrong causes and a seven-variant reproducer built
to explain an end state a union produces for free, and it caught a fresh
reader again on 2026-08-27.
The `unread` half of the original observation was real and is item 172,
fixed in edbf393. The reported `draft inbox unread` is fully explained:
`unread` from the missing S flag, `inbox` from the arrived message
sharing the thread.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AtUzfNjMD8fiYfamDd3ywW
Diffstat (limited to 'docs/superpowers/plans')
| -rw-r--r-- | docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md | 144 | ||||
| -rw-r--r-- | docs/superpowers/plans/2026-08-03-post-0.1.0-usability.md | 105 |
2 files changed, 145 insertions, 104 deletions
diff --git a/docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md b/docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md index fe212e2..4556b48 100644 --- a/docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md +++ b/docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md @@ -8791,3 +8791,147 @@ flag set when its own comment said the point was the draft flag "not left bare". It therefore failed against the corrected behaviour. An over-specified assertion of this shape blocks the fix rather than the bug. +## 164. A draft this application saved keeps `inbox` + +**Observed (developer, 2026-08-25):** `notmuch search --output=tags` on a +draft this application had just written reported `draft inbox unread`. + +**The first cause recorded here was WRONG, and the correction is the useful +part.** It said `strip_inbox_from_sent()` reads a sent-only folder list and +that `qtmaildirconf.py` has no drafts equivalent. Neither is true: + +- `NOT_ARRIVALS` is `("sent", "drafts")`, so `sent_folders()` already returns + both. The name says "sent" and the contents do not, which is what made the + wrong reading plausible. +- Run against the real config it returns every account's drafts folder. +- `notmuch count "(<carve-out query>) and id:<the draft>"` returns **1**. The + query the hook builds MATCHES the affected message. + +So the folder list and the query are correct, and the fix is not there. + +**What is actually established.** + +- The carve-out is scoped to `SCOPE = "tag:new"` (`post-new:106`). +- The affected draft carries `inbox`, and `notmuch count tag:new` is **0**. +- The installed hooks are SYMLINKS into this repository, so the code read is + the code that runs. Verified rather than assumed. +- An mbsync-style rename does **not** re-apply `new.tags`: measured in a + throwaway database, a file renamed to add `,U=4` and reindexed kept the tags + it had. The "the rename retags it" theory is therefore also out. + +**What is NOT established, and must be before any code is written:** which +pass put `inbox` on this file, and why it was not carrying `tag:new` when the +hook's carve-out ran. The likely shape is an ordering one, since item 158 +indexes a draft from the application itself, outside `notmuch new`, and a file +already known to the database is not a new file on the next pass. But that is +a hypothesis and the last two hypotheses here were both wrong. + +**The reproducer was built (2026-08-25) and it settles the mechanism.** Seven +variants were driven in throwaway databases, modelling `indexDraftFile()` with +a real `notmuch_database_index_file` call rather than the CLI, because no CLI +command indexes an untracked path without applying `new.tags`. + +What the sweep established, each measured rather than reasoned: + +- `index_file` applies **no tags at all**. A draft the application indexes is + therefore never in `tag:new` scope, and the hook has nothing to carve out. +- Whenever the file IS in `tag:new` scope, the carve-out strips `inbox` + correctly, in every filename shape tried: `:2,DS`, `:2,D`, no info suffix, + in `cur/` and in `new/`, with and without the `,U=4` infix. The real file's + shape (`,U=4:2,D`) is among them. +- It survives the orderings too: `notmuch new` first then the app's index, + the app's index first then the rename, an autosave landing between + `notmuch new` and the hook, and the stale-path `remove_message` that makes + the renamed file arrive as new mail. All six left the draft clean. +- The `D` flag is what puts `draft` on the message (`synchronize_flags`), and + the `S` flag is what removes `unread`. The affected file is `:2,D`, which is + why it carries `unread`, and that matches the reported tag set exactly. + +**The one variant that reproduces it** is the general shape rather than a +filename detail: a pass where `inbox` is applied while `tag:new` has ALREADY +been consumed. Modelled as a file indexed at a path the carve-out does not +cover and moved into the drafts folder afterwards, it ends in precisely the +live end state, `draft inbox unread` in Drafts with `,U=4` and `tag:new` at 0. +Nothing revisits a message once the marker is gone, so the tag is permanent. + +**What is still NOT established, and the next step.** The affected account +writes drafts straight to `<account>/Drafts`, which the carve-out +covers (verified against the live config and the live query, which matches the +message by id today), so the reproducing variant's premise does not hold for +it as written. The live log for the pass that added it reads + + 10:10:52 Added 1 new message to the database. Detected 9 file renames. + 10:10:52 post-new: sent-folder carve-out applied over 9 folder(s) + +so the hook DID run on that pass, over a path the query covers, and logged +success. The remaining candidates are all about what the path or the marker +looked like at that instant, not about the query text: the carve-out logs +"applied" on a `notmuch tag` that matched zero messages, so a successful log +line is not evidence the message was in scope. Instrumenting the hook to log +the carve-out's MATCH COUNT, and leaving it to run until the next draft, is +the cheapest way to close it, and is a log-only change to code that tags real +mail unattended. + +The filename also rules one thing in: `1787645266.M802P16149Q3.<host>` is +exactly `MaildirName::fresh()` output, so the application wrote this file. It +is not a draft another client left behind. + +The reproducer scripts are throwaway and were not kept; `indexfile.c` is +fifteen lines around one `notmuch_database_index_file` call and is trivial to +rebuild from this entry if the instrumentation points back at the hook. + +**Constraints.** + +- **The hook tags real mail unattended every ten minutes.** Nothing here is + worth a speculative change. +- The 0.27.0 changelog claims sent mail and drafts both stay out of the inbox. + Whatever the cause, that claim is currently false for drafts and the entry + needs correcting with the fix. +- Only `inbox` may be touched. A draft legitimately carries `draft` and + `unread`, and `maildir.synchronize_flags` means removing `unread` rewrites + the filename and reaches the server. +- The hook must keep refusing to consume `tag:new` when a carve-out fails. +- `test_post_new.py` and `test_qtmaildirconf.py` both live beside the hook and + have sent-carve-out tests to copy. + +--- + +**RE-MEASURED 2026-08-27, and the item is DROPPED: there was never an `inbox` +tag on a draft.** Everything above this line is the investigation of a defect +that did not exist, and it is kept because the way it went wrong is worth more +than the conclusion. + +The premise came from `notmuch search --output=tags`, which reports the union +over a THREAD. A draft replying to an arrived message sits in that message's +thread, so the union reads `draft inbox unread` while the two tags live on two +different messages. Measured today on the thread that produced the original +report: + +- the arrived mail: `['account-<acct>', 'inbox']` +- the draft reply: `['draft', 'unread']` + +Neither carries both. Across the whole index, `notmuch count --output=messages +'tag:draft and tag:inbox'` is **0** against 12 drafts, nine of which were +written on or before 2026-08-25 and so were present when this was filed. + +**The trap has a second half that makes it much easier to fall into.** A +thread-level `notmuch count 'tag:draft and tag:inbox'` ALSO returns 0, because +search terms match per message even in a thread query. So the count and the +displayed tag list disagree, and the displayed list is the one that looks like +evidence. Use `--output=messages` and `notmuch show` when asking what tags a +message carries; `--output=tags` answers a different question than it appears +to. + +This is the same union recorded for `ThreadSummary::tags` under item 110, where +it made a card claim a tag its message did not have. It cost this item a week +open, two wrong causes, and a seven-variant reproducer built to explain an end +state that a union produces for free. It also caught a fresh reader of this +backlog on 2026-08-27, who read the same union and reported that drafts were +carrying `inbox` before measuring at message level. + +**The `unread` half of the original observation WAS real** and is item 172: the +app wrote drafts as `:2,D`, and notmuch tags anything without `S` as `unread`. +That is fixed. The reported tag set `draft inbox unread` is fully explained: +`unread` from the missing `S` flag on the draft, `inbox` from the arrived +message sharing its thread. + diff --git a/docs/superpowers/plans/2026-08-03-post-0.1.0-usability.md b/docs/superpowers/plans/2026-08-03-post-0.1.0-usability.md index fb659ec..fddb746 100644 --- a/docs/superpowers/plans/2026-08-03-post-0.1.0-usability.md +++ b/docs/superpowers/plans/2026-08-03-post-0.1.0-usability.md @@ -237,7 +237,7 @@ taking that too literally. | 162 | Delete fails while a sync is renaming the file underneath it | defect | S | **done, 2026-08-25.** mbsync renames an uploaded file to add its `,U=<uid>` infix and notmuch keeps the pre-`U=` name until that sync's `notmuch new` runs, so `moveMessages` renamed a path that no longer existed and Delete silently did nothing while blaming the destination folder. `moveMessages` now re-resolves by MESSAGE ID when the recorded path is gone: one reindex of that directory, then the filename that exists on disk. Bounded to one retry, so a file genuinely gone still reports. Holding the move during a sync was the other candidate and is NOT the fix: `sendMove` already refuses on notmuch's write lock, but this window sits between mbsync's rename and that sync's `notmuch new`, which touches no lock | | 163 | The message pane shows a stale path, and the composer forks the draft | defect | S | **done, 2026-08-25.** mbsync renames an uploaded file to add its `,U=<uid>` infix while the model still holds the name the query returned. `MaildirName::resolveRenamed()` returns the path unchanged when it exists, else finds the file in that one directory whose unique stem matches; it refuses an ambiguous match and yields nothing for a genuinely missing file. Wired into all THREE read sites: the pane, Reply/Forward, and the draft reopen. The reopen was the one that cost data, forking a draft into two files with two Message-IDs, both reaching the server | -| 164 | A draft this application saved keeps `inbox` | defect | S | open, 2026-08-25, **cause corrected 2026-08-25**. The first diagnosis blamed a missing drafts helper and was WRONG: `NOT_ARRIVALS` in `qtmaildirconf.py` is `("sent", "drafts")`, the folder list includes every account's drafts folder, and `notmuch count` confirms the carve-out query MATCHES the affected draft. The carve-out is scoped to `tag:new`, and the draft carries `inbox` while `tag:new` is 0, so it was never in scope when the hook ran. Measured separately: an mbsync-style rename does NOT re-add `new.tags`, so the retag theory is out too. What remains unestablished is WHICH pass tagged it; establish that before writing code | +| 164 | A draft this application saved keeps `inbox` | defect | S | **dropped** 2026-08-27, NOT A DEFECT. The premise was a measurement artifact: its evidence was `notmuch search --output=tags`, which DISPLAYS the union over a thread, and a reply-draft under an arrived message reads `draft inbox unread` while no message carries both. Re-measured at message level: 0 of 12 drafts carry `inbox`, including nine written on or before 2026-08-25. The `unread` half was real and is item 172 | | 165 | A draft gets a new Message-ID on every autosave | enhancement | ? | open, 2026-08-25, found while hand-testing 163 and 164. `MessageBuilder::build()` generates an id unconditionally and every autosave calls it, so each revision is a distinct MESSAGE to notmuch and to the server rather than a new version of one. Invisible while the file is replaced correctly, which item 163's fix restores; it is what turned that fork into two messages rather than one duplicated file. Needs a DECISION on what a draft's identity is before any code: a stable id reused at send, a stable id discarded at send, or the status quo. Neither `ComposeContext` nor `OutgoingMessage` has a field to carry an id, so it is not a changed call site | | 166 | Mail you send to your own other account loses `inbox` | defect | S | **done 2026-08-25**, unreleased. `sent_only()` keeps a message only when EVERY file is inside a sent folder, which is what the carve-out's docstring already claimed. No query can express it, measured; the root comes from `database.mail_root`, with a split-index fixture the ordinary layout cannot provide. Verified read-only against the live index: 780 of 807 still stripped, 27 spared, no arrival affected | | 167 | No way to tell one build of an unreleased version from another | enhancement | XS | **done 2026-08-25**, unreleased. The user chose a counter over a git description: `QTMAILDIR_BUILD_NUMBER`, a cmake option ON by default, increments a counter in the BUILD directory on every build and writes `buildnumber.h`. `QTMAILDIR_VERSION_DISPLAY` carries it; `QTMAILDIR_VERSION` stays clean and is what the window title, `applicationVersion` and the release procedure use | @@ -1092,109 +1092,6 @@ The 70-second duration recorded above fits a `QTRY_*` waiting for a file that is never going to appear, which is consistent with a wrong destination rather than a slow one. -## 164. A draft this application saved keeps `inbox` - -**Observed (developer, 2026-08-25):** `notmuch search --output=tags` on a -draft this application had just written reported `draft inbox unread`. - -**The first cause recorded here was WRONG, and the correction is the useful -part.** It said `strip_inbox_from_sent()` reads a sent-only folder list and -that `qtmaildirconf.py` has no drafts equivalent. Neither is true: - -- `NOT_ARRIVALS` is `("sent", "drafts")`, so `sent_folders()` already returns - both. The name says "sent" and the contents do not, which is what made the - wrong reading plausible. -- Run against the real config it returns every account's drafts folder. -- `notmuch count "(<carve-out query>) and id:<the draft>"` returns **1**. The - query the hook builds MATCHES the affected message. - -So the folder list and the query are correct, and the fix is not there. - -**What is actually established.** - -- The carve-out is scoped to `SCOPE = "tag:new"` (`post-new:106`). -- The affected draft carries `inbox`, and `notmuch count tag:new` is **0**. -- The installed hooks are SYMLINKS into this repository, so the code read is - the code that runs. Verified rather than assumed. -- An mbsync-style rename does **not** re-apply `new.tags`: measured in a - throwaway database, a file renamed to add `,U=4` and reindexed kept the tags - it had. The "the rename retags it" theory is therefore also out. - -**What is NOT established, and must be before any code is written:** which -pass put `inbox` on this file, and why it was not carrying `tag:new` when the -hook's carve-out ran. The likely shape is an ordering one, since item 158 -indexes a draft from the application itself, outside `notmuch new`, and a file -already known to the database is not a new file on the next pass. But that is -a hypothesis and the last two hypotheses here were both wrong. - -**The reproducer was built (2026-08-25) and it settles the mechanism.** Seven -variants were driven in throwaway databases, modelling `indexDraftFile()` with -a real `notmuch_database_index_file` call rather than the CLI, because no CLI -command indexes an untracked path without applying `new.tags`. - -What the sweep established, each measured rather than reasoned: - -- `index_file` applies **no tags at all**. A draft the application indexes is - therefore never in `tag:new` scope, and the hook has nothing to carve out. -- Whenever the file IS in `tag:new` scope, the carve-out strips `inbox` - correctly, in every filename shape tried: `:2,DS`, `:2,D`, no info suffix, - in `cur/` and in `new/`, with and without the `,U=4` infix. The real file's - shape (`,U=4:2,D`) is among them. -- It survives the orderings too: `notmuch new` first then the app's index, - the app's index first then the rename, an autosave landing between - `notmuch new` and the hook, and the stale-path `remove_message` that makes - the renamed file arrive as new mail. All six left the draft clean. -- The `D` flag is what puts `draft` on the message (`synchronize_flags`), and - the `S` flag is what removes `unread`. The affected file is `:2,D`, which is - why it carries `unread`, and that matches the reported tag set exactly. - -**The one variant that reproduces it** is the general shape rather than a -filename detail: a pass where `inbox` is applied while `tag:new` has ALREADY -been consumed. Modelled as a file indexed at a path the carve-out does not -cover and moved into the drafts folder afterwards, it ends in precisely the -live end state, `draft inbox unread` in Drafts with `,U=4` and `tag:new` at 0. -Nothing revisits a message once the marker is gone, so the tag is permanent. - -**What is still NOT established, and the next step.** The affected account -writes drafts straight to `<account>/Drafts`, which the carve-out -covers (verified against the live config and the live query, which matches the -message by id today), so the reproducing variant's premise does not hold for -it as written. The live log for the pass that added it reads - - 10:10:52 Added 1 new message to the database. Detected 9 file renames. - 10:10:52 post-new: sent-folder carve-out applied over 9 folder(s) - -so the hook DID run on that pass, over a path the query covers, and logged -success. The remaining candidates are all about what the path or the marker -looked like at that instant, not about the query text: the carve-out logs -"applied" on a `notmuch tag` that matched zero messages, so a successful log -line is not evidence the message was in scope. Instrumenting the hook to log -the carve-out's MATCH COUNT, and leaving it to run until the next draft, is -the cheapest way to close it, and is a log-only change to code that tags real -mail unattended. - -The filename also rules one thing in: `1787645266.M802P16149Q3.<host>` is -exactly `MaildirName::fresh()` output, so the application wrote this file. It -is not a draft another client left behind. - -The reproducer scripts are throwaway and were not kept; `indexfile.c` is -fifteen lines around one `notmuch_database_index_file` call and is trivial to -rebuild from this entry if the instrumentation points back at the hook. - -**Constraints.** - -- **The hook tags real mail unattended every ten minutes.** Nothing here is - worth a speculative change. -- The 0.27.0 changelog claims sent mail and drafts both stay out of the inbox. - Whatever the cause, that claim is currently false for drafts and the entry - needs correcting with the fix. -- Only `inbox` may be touched. A draft legitimately carries `draft` and - `unread`, and `maildir.synchronize_flags` means removing `unread` rewrites - the filename and reaches the server. -- The hook must keep refusing to consume `tag:new` when a carve-out fails. -- `test_post_new.py` and `test_qtmaildirconf.py` both live beside the hook and - have sent-carve-out tests to copy. - ## 165. A draft gets a new Message-ID on every autosave **Observed (developer, 2026-08-25), while hand-testing items 163 and 164.** |
