diff options
Diffstat (limited to 'docs/superpowers')
| -rw-r--r-- | docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md | 144 | ||||
| -rw-r--r-- | docs/superpowers/plans/2026-08-03-post-0.1.0-usability.md | 105 |
2 files changed, 145 insertions, 104 deletions
diff --git a/docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md b/docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md index fe212e2..4556b48 100644 --- a/docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md +++ b/docs/superpowers/plans/2026-08-03-post-0.1.0-usability-closed.md @@ -8791,3 +8791,147 @@ flag set when its own comment said the point was the draft flag "not left bare". It therefore failed against the corrected behaviour. An over-specified assertion of this shape blocks the fix rather than the bug. +## 164. A draft this application saved keeps `inbox` + +**Observed (developer, 2026-08-25):** `notmuch search --output=tags` on a +draft this application had just written reported `draft inbox unread`. + +**The first cause recorded here was WRONG, and the correction is the useful +part.** It said `strip_inbox_from_sent()` reads a sent-only folder list and +that `qtmaildirconf.py` has no drafts equivalent. Neither is true: + +- `NOT_ARRIVALS` is `("sent", "drafts")`, so `sent_folders()` already returns + both. The name says "sent" and the contents do not, which is what made the + wrong reading plausible. +- Run against the real config it returns every account's drafts folder. +- `notmuch count "(<carve-out query>) and id:<the draft>"` returns **1**. The + query the hook builds MATCHES the affected message. + +So the folder list and the query are correct, and the fix is not there. + +**What is actually established.** + +- The carve-out is scoped to `SCOPE = "tag:new"` (`post-new:106`). +- The affected draft carries `inbox`, and `notmuch count tag:new` is **0**. +- The installed hooks are SYMLINKS into this repository, so the code read is + the code that runs. Verified rather than assumed. +- An mbsync-style rename does **not** re-apply `new.tags`: measured in a + throwaway database, a file renamed to add `,U=4` and reindexed kept the tags + it had. The "the rename retags it" theory is therefore also out. + +**What is NOT established, and must be before any code is written:** which +pass put `inbox` on this file, and why it was not carrying `tag:new` when the +hook's carve-out ran. The likely shape is an ordering one, since item 158 +indexes a draft from the application itself, outside `notmuch new`, and a file +already known to the database is not a new file on the next pass. But that is +a hypothesis and the last two hypotheses here were both wrong. + +**The reproducer was built (2026-08-25) and it settles the mechanism.** Seven +variants were driven in throwaway databases, modelling `indexDraftFile()` with +a real `notmuch_database_index_file` call rather than the CLI, because no CLI +command indexes an untracked path without applying `new.tags`. + +What the sweep established, each measured rather than reasoned: + +- `index_file` applies **no tags at all**. A draft the application indexes is + therefore never in `tag:new` scope, and the hook has nothing to carve out. +- Whenever the file IS in `tag:new` scope, the carve-out strips `inbox` + correctly, in every filename shape tried: `:2,DS`, `:2,D`, no info suffix, + in `cur/` and in `new/`, with and without the `,U=4` infix. The real file's + shape (`,U=4:2,D`) is among them. +- It survives the orderings too: `notmuch new` first then the app's index, + the app's index first then the rename, an autosave landing between + `notmuch new` and the hook, and the stale-path `remove_message` that makes + the renamed file arrive as new mail. All six left the draft clean. +- The `D` flag is what puts `draft` on the message (`synchronize_flags`), and + the `S` flag is what removes `unread`. The affected file is `:2,D`, which is + why it carries `unread`, and that matches the reported tag set exactly. + +**The one variant that reproduces it** is the general shape rather than a +filename detail: a pass where `inbox` is applied while `tag:new` has ALREADY +been consumed. Modelled as a file indexed at a path the carve-out does not +cover and moved into the drafts folder afterwards, it ends in precisely the +live end state, `draft inbox unread` in Drafts with `,U=4` and `tag:new` at 0. +Nothing revisits a message once the marker is gone, so the tag is permanent. + +**What is still NOT established, and the next step.** The affected account +writes drafts straight to `<account>/Drafts`, which the carve-out +covers (verified against the live config and the live query, which matches the +message by id today), so the reproducing variant's premise does not hold for +it as written. The live log for the pass that added it reads + + 10:10:52 Added 1 new message to the database. Detected 9 file renames. + 10:10:52 post-new: sent-folder carve-out applied over 9 folder(s) + +so the hook DID run on that pass, over a path the query covers, and logged +success. The remaining candidates are all about what the path or the marker +looked like at that instant, not about the query text: the carve-out logs +"applied" on a `notmuch tag` that matched zero messages, so a successful log +line is not evidence the message was in scope. Instrumenting the hook to log +the carve-out's MATCH COUNT, and leaving it to run until the next draft, is +the cheapest way to close it, and is a log-only change to code that tags real +mail unattended. + +The filename also rules one thing in: `1787645266.M802P16149Q3.<host>` is +exactly `MaildirName::fresh()` output, so the application wrote this file. It +is not a draft another client left behind. + +The reproducer scripts are throwaway and were not kept; `indexfile.c` is +fifteen lines around one `notmuch_database_index_file` call and is trivial to +rebuild from this entry if the instrumentation points back at the hook. + +**Constraints.** + +- **The hook tags real mail unattended every ten minutes.** Nothing here is + worth a speculative change. +- The 0.27.0 changelog claims sent mail and drafts both stay out of the inbox. + Whatever the cause, that claim is currently false for drafts and the entry + needs correcting with the fix. +- Only `inbox` may be touched. A draft legitimately carries `draft` and + `unread`, and `maildir.synchronize_flags` means removing `unread` rewrites + the filename and reaches the server. +- The hook must keep refusing to consume `tag:new` when a carve-out fails. +- `test_post_new.py` and `test_qtmaildirconf.py` both live beside the hook and + have sent-carve-out tests to copy. + +--- + +**RE-MEASURED 2026-08-27, and the item is DROPPED: there was never an `inbox` +tag on a draft.** Everything above this line is the investigation of a defect +that did not exist, and it is kept because the way it went wrong is worth more +than the conclusion. + +The premise came from `notmuch search --output=tags`, which reports the union +over a THREAD. A draft replying to an arrived message sits in that message's +thread, so the union reads `draft inbox unread` while the two tags live on two +different messages. Measured today on the thread that produced the original +report: + +- the arrived mail: `['account-<acct>', 'inbox']` +- the draft reply: `['draft', 'unread']` + +Neither carries both. Across the whole index, `notmuch count --output=messages +'tag:draft and tag:inbox'` is **0** against 12 drafts, nine of which were +written on or before 2026-08-25 and so were present when this was filed. + +**The trap has a second half that makes it much easier to fall into.** A +thread-level `notmuch count 'tag:draft and tag:inbox'` ALSO returns 0, because +search terms match per message even in a thread query. So the count and the +displayed tag list disagree, and the displayed list is the one that looks like +evidence. Use `--output=messages` and `notmuch show` when asking what tags a +message carries; `--output=tags` answers a different question than it appears +to. + +This is the same union recorded for `ThreadSummary::tags` under item 110, where +it made a card claim a tag its message did not have. It cost this item a week +open, two wrong causes, and a seven-variant reproducer built to explain an end +state that a union produces for free. It also caught a fresh reader of this +backlog on 2026-08-27, who read the same union and reported that drafts were +carrying `inbox` before measuring at message level. + +**The `unread` half of the original observation WAS real** and is item 172: the +app wrote drafts as `:2,D`, and notmuch tags anything without `S` as `unread`. +That is fixed. The reported tag set `draft inbox unread` is fully explained: +`unread` from the missing `S` flag on the draft, `inbox` from the arrived +message sharing its thread. + diff --git a/docs/superpowers/plans/2026-08-03-post-0.1.0-usability.md b/docs/superpowers/plans/2026-08-03-post-0.1.0-usability.md index fb659ec..fddb746 100644 --- a/docs/superpowers/plans/2026-08-03-post-0.1.0-usability.md +++ b/docs/superpowers/plans/2026-08-03-post-0.1.0-usability.md @@ -237,7 +237,7 @@ taking that too literally. | 162 | Delete fails while a sync is renaming the file underneath it | defect | S | **done, 2026-08-25.** mbsync renames an uploaded file to add its `,U=<uid>` infix and notmuch keeps the pre-`U=` name until that sync's `notmuch new` runs, so `moveMessages` renamed a path that no longer existed and Delete silently did nothing while blaming the destination folder. `moveMessages` now re-resolves by MESSAGE ID when the recorded path is gone: one reindex of that directory, then the filename that exists on disk. Bounded to one retry, so a file genuinely gone still reports. Holding the move during a sync was the other candidate and is NOT the fix: `sendMove` already refuses on notmuch's write lock, but this window sits between mbsync's rename and that sync's `notmuch new`, which touches no lock | | 163 | The message pane shows a stale path, and the composer forks the draft | defect | S | **done, 2026-08-25.** mbsync renames an uploaded file to add its `,U=<uid>` infix while the model still holds the name the query returned. `MaildirName::resolveRenamed()` returns the path unchanged when it exists, else finds the file in that one directory whose unique stem matches; it refuses an ambiguous match and yields nothing for a genuinely missing file. Wired into all THREE read sites: the pane, Reply/Forward, and the draft reopen. The reopen was the one that cost data, forking a draft into two files with two Message-IDs, both reaching the server | -| 164 | A draft this application saved keeps `inbox` | defect | S | open, 2026-08-25, **cause corrected 2026-08-25**. The first diagnosis blamed a missing drafts helper and was WRONG: `NOT_ARRIVALS` in `qtmaildirconf.py` is `("sent", "drafts")`, the folder list includes every account's drafts folder, and `notmuch count` confirms the carve-out query MATCHES the affected draft. The carve-out is scoped to `tag:new`, and the draft carries `inbox` while `tag:new` is 0, so it was never in scope when the hook ran. Measured separately: an mbsync-style rename does NOT re-add `new.tags`, so the retag theory is out too. What remains unestablished is WHICH pass tagged it; establish that before writing code | +| 164 | A draft this application saved keeps `inbox` | defect | S | **dropped** 2026-08-27, NOT A DEFECT. The premise was a measurement artifact: its evidence was `notmuch search --output=tags`, which DISPLAYS the union over a thread, and a reply-draft under an arrived message reads `draft inbox unread` while no message carries both. Re-measured at message level: 0 of 12 drafts carry `inbox`, including nine written on or before 2026-08-25. The `unread` half was real and is item 172 | | 165 | A draft gets a new Message-ID on every autosave | enhancement | ? | open, 2026-08-25, found while hand-testing 163 and 164. `MessageBuilder::build()` generates an id unconditionally and every autosave calls it, so each revision is a distinct MESSAGE to notmuch and to the server rather than a new version of one. Invisible while the file is replaced correctly, which item 163's fix restores; it is what turned that fork into two messages rather than one duplicated file. Needs a DECISION on what a draft's identity is before any code: a stable id reused at send, a stable id discarded at send, or the status quo. Neither `ComposeContext` nor `OutgoingMessage` has a field to carry an id, so it is not a changed call site | | 166 | Mail you send to your own other account loses `inbox` | defect | S | **done 2026-08-25**, unreleased. `sent_only()` keeps a message only when EVERY file is inside a sent folder, which is what the carve-out's docstring already claimed. No query can express it, measured; the root comes from `database.mail_root`, with a split-index fixture the ordinary layout cannot provide. Verified read-only against the live index: 780 of 807 still stripped, 27 spared, no arrival affected | | 167 | No way to tell one build of an unreleased version from another | enhancement | XS | **done 2026-08-25**, unreleased. The user chose a counter over a git description: `QTMAILDIR_BUILD_NUMBER`, a cmake option ON by default, increments a counter in the BUILD directory on every build and writes `buildnumber.h`. `QTMAILDIR_VERSION_DISPLAY` carries it; `QTMAILDIR_VERSION` stays clean and is what the window title, `applicationVersion` and the release procedure use | @@ -1092,109 +1092,6 @@ The 70-second duration recorded above fits a `QTRY_*` waiting for a file that is never going to appear, which is consistent with a wrong destination rather than a slow one. -## 164. A draft this application saved keeps `inbox` - -**Observed (developer, 2026-08-25):** `notmuch search --output=tags` on a -draft this application had just written reported `draft inbox unread`. - -**The first cause recorded here was WRONG, and the correction is the useful -part.** It said `strip_inbox_from_sent()` reads a sent-only folder list and -that `qtmaildirconf.py` has no drafts equivalent. Neither is true: - -- `NOT_ARRIVALS` is `("sent", "drafts")`, so `sent_folders()` already returns - both. The name says "sent" and the contents do not, which is what made the - wrong reading plausible. -- Run against the real config it returns every account's drafts folder. -- `notmuch count "(<carve-out query>) and id:<the draft>"` returns **1**. The - query the hook builds MATCHES the affected message. - -So the folder list and the query are correct, and the fix is not there. - -**What is actually established.** - -- The carve-out is scoped to `SCOPE = "tag:new"` (`post-new:106`). -- The affected draft carries `inbox`, and `notmuch count tag:new` is **0**. -- The installed hooks are SYMLINKS into this repository, so the code read is - the code that runs. Verified rather than assumed. -- An mbsync-style rename does **not** re-apply `new.tags`: measured in a - throwaway database, a file renamed to add `,U=4` and reindexed kept the tags - it had. The "the rename retags it" theory is therefore also out. - -**What is NOT established, and must be before any code is written:** which -pass put `inbox` on this file, and why it was not carrying `tag:new` when the -hook's carve-out ran. The likely shape is an ordering one, since item 158 -indexes a draft from the application itself, outside `notmuch new`, and a file -already known to the database is not a new file on the next pass. But that is -a hypothesis and the last two hypotheses here were both wrong. - -**The reproducer was built (2026-08-25) and it settles the mechanism.** Seven -variants were driven in throwaway databases, modelling `indexDraftFile()` with -a real `notmuch_database_index_file` call rather than the CLI, because no CLI -command indexes an untracked path without applying `new.tags`. - -What the sweep established, each measured rather than reasoned: - -- `index_file` applies **no tags at all**. A draft the application indexes is - therefore never in `tag:new` scope, and the hook has nothing to carve out. -- Whenever the file IS in `tag:new` scope, the carve-out strips `inbox` - correctly, in every filename shape tried: `:2,DS`, `:2,D`, no info suffix, - in `cur/` and in `new/`, with and without the `,U=4` infix. The real file's - shape (`,U=4:2,D`) is among them. -- It survives the orderings too: `notmuch new` first then the app's index, - the app's index first then the rename, an autosave landing between - `notmuch new` and the hook, and the stale-path `remove_message` that makes - the renamed file arrive as new mail. All six left the draft clean. -- The `D` flag is what puts `draft` on the message (`synchronize_flags`), and - the `S` flag is what removes `unread`. The affected file is `:2,D`, which is - why it carries `unread`, and that matches the reported tag set exactly. - -**The one variant that reproduces it** is the general shape rather than a -filename detail: a pass where `inbox` is applied while `tag:new` has ALREADY -been consumed. Modelled as a file indexed at a path the carve-out does not -cover and moved into the drafts folder afterwards, it ends in precisely the -live end state, `draft inbox unread` in Drafts with `,U=4` and `tag:new` at 0. -Nothing revisits a message once the marker is gone, so the tag is permanent. - -**What is still NOT established, and the next step.** The affected account -writes drafts straight to `<account>/Drafts`, which the carve-out -covers (verified against the live config and the live query, which matches the -message by id today), so the reproducing variant's premise does not hold for -it as written. The live log for the pass that added it reads - - 10:10:52 Added 1 new message to the database. Detected 9 file renames. - 10:10:52 post-new: sent-folder carve-out applied over 9 folder(s) - -so the hook DID run on that pass, over a path the query covers, and logged -success. The remaining candidates are all about what the path or the marker -looked like at that instant, not about the query text: the carve-out logs -"applied" on a `notmuch tag` that matched zero messages, so a successful log -line is not evidence the message was in scope. Instrumenting the hook to log -the carve-out's MATCH COUNT, and leaving it to run until the next draft, is -the cheapest way to close it, and is a log-only change to code that tags real -mail unattended. - -The filename also rules one thing in: `1787645266.M802P16149Q3.<host>` is -exactly `MaildirName::fresh()` output, so the application wrote this file. It -is not a draft another client left behind. - -The reproducer scripts are throwaway and were not kept; `indexfile.c` is -fifteen lines around one `notmuch_database_index_file` call and is trivial to -rebuild from this entry if the instrumentation points back at the hook. - -**Constraints.** - -- **The hook tags real mail unattended every ten minutes.** Nothing here is - worth a speculative change. -- The 0.27.0 changelog claims sent mail and drafts both stay out of the inbox. - Whatever the cause, that claim is currently false for drafts and the entry - needs correcting with the fix. -- Only `inbox` may be touched. A draft legitimately carries `draft` and - `unread`, and `maildir.synchronize_flags` means removing `unread` rewrites - the filename and reaches the server. -- The hook must keep refusing to consume `tag:new` when a carve-out fails. -- `test_post_new.py` and `test_qtmaildirconf.py` both live beside the hook and - have sent-carve-out tests to copy. - ## 165. A draft gets a new Message-ID on every autosave **Observed (developer, 2026-08-25), while hand-testing items 163 and 164.** |