PLAN — jmap-bridge

Architecture, data model, milestones, decisions, risks, roadmap.

Doc hierarchy: REQUIREMENTS.md = scope (source of truth for what); this file = how and when. README.md = user-facing. If code and docs disagree, flag it and fix one of them — don't let them drift.

Module: github.com/CaffeinatedTech/jmap-bridge · License: MIT · Language: Go · Config: TOML · Packaging: Docker/Kubernetes · CI: none (local gates, AGENTS.md).

1. Locked decisions

#	Decision	Date
D-1	Separate project at `~/projects/jmap-bridge`, deployed as a Docker container or in a Kubernetes cluster	2026-09-28
D-2	Hybrid local copy: headers/flags/structure always synced, bodies lazy-fetched and cached on first open	2026-09-28
D-3	Contacts (JMAP RFC 9610 ↔ provider CardDAV) are in v0.1; JMAP Calendar is roadmap only (user corrected scope)	2026-09-28
D-4	Target is any IMAP server: Gmail (headline), cPanel, Dovecot, Namecheap Private Email → runtime capability detection, no server hard-coded	2026-09-28
D-5	Gmail auth is OAuth2-only (XOAUTH2); app passwords deliberately unsupported	2026-09-28
D-6	Freetext search over non-hydrated mail uses search-driven backfill (instant header matches + background hydration + SSE), not IMAP SEARCH merging, not full mirror	2026-09-28
D-7	IMAP client: `github.com/CaffeinatedTech/go-imap`, isolated behind our own driver interface in `internal/imapdrv` (amended 2026-09-29: chose `kiliant/go-imap` v1.1.0 over `emersion/go-imap/v2`, which has no QRESYNC/COMPRESS and no raw-command escape hatch — verified against v2.0.0-beta.8; kiliant ships tested client QRESYNC/CONDSTORE/COMPRESS, zero deps, frozen v1 API, interop-verified incl. Dovecot. amended 2026-09-30: the client is now the hard fork `github.com/CaffeinatedTech/go-imap` v1.2.2 — see D-21)	2026-09-28, amended 2026-09-29, 2026-09-30
D-8	MIT, Go, TOML config	2026-09-28
D-9	Module path `github.com/CaffeinatedTech/jmap-bridge`	2026-09-28
D-10	No CI. All gates run locally before every commit	2026-09-28
D-11	`REQUIREMENTS.md` at full FR/NFR rigor, traceable to milestones	2026-09-28
D-12	Real deployments terminate TLS at a reverse proxy with DNS — required by the Gmail OAuth callback (deployment documentation, README §Deployment)	2026-09-28
D-13	Path-prefixed multi-account on a single origin (`/{account}/…`), one container serves N accounts	2026-09-28
D-14	IMAP-first writes: apply to the real server, commit locally only on success — the cache never diverges optimistically	2026-09-28
D-15	Client-facing auth is a per-account bearer token over HTTP Basic (or `none` on loopback)	2026-09-28
D-16	Request size caps are final: JSON ≤ 32 MiB, upload ≤ 64 MiB (REQUIREMENTS NFR-5)	2026-09-28
D-17	`AddressBook/set` stays roadmap (v0.2+); v0.1 ships `AddressBook/get\|changes` only	2026-09-28
D-18	Body cache is grow-only in v0.1; growth documented in README; eviction revisited only on demonstrated disk pressure	2026-09-28
D-19	NFR-1 / NFR-8 numbers are binding as written (no M5 "pin"); M5 revalidates by measurement, revisions are REQUIREMENTS edits	2026-09-28
D-20	Gmail's `[Gmail]/All Mail` maps to JMAP role `archive` and its membership is implicit (may_add/may_remove false): archive = remove `\Inbox` only, the message stays listed in All Mail (FR-M.18, approved by the user)	2026-09-29
D-21	kiliant/go-imap is hard-forked and published as `github.com/CaffeinatedTech/go-imap` (root v1.2.2, nested `imapserver` v0.2.1), taken at kiliant's `main`; the bridge depends on it directly with no `replace`. The fork carries the X-GM-EXT-1 label store, flag-form value capture and long-response-line tolerance, retains upstream's MIT licence and README credit, and is not proposed upstream. `PATCH-NOTES.md` in the fork records the divergence	2026-09-29, amended 2026-09-30
D-22	Abuse protection is two-layer (NFR-5, SECURITY-PLAN.md): the reverse proxy rate-limits only the unauthenticated OAuth bootstrap — it deliberately does not rate-limit or concurrency-cap authenticated JMAP traffic, since one client action is legitimately many requests and the proxy cannot see the client token; `internal/ratelimit` adds credential-aware failed-auth lockout, enforced concurrency and EventSource caps in-process. The limits advertised in the session are the limits enforced; source identity is trusted only from configured `rate.trusted_proxies` (+ `rate.client_ip_header`). App-side volume limiting is deliberately not duplicated for the authenticated surface	2026-10-02, amended 2026-10-03

2. Constraints that shape the design

2.1 The client contract (what jmap-tui actually exercises)

Measured from jmap-tui/internal/jmapclient on 2026-09-28:

Session: GET {url}/.well-known/jmap → apiUrl, uploadUrl, downloadUrl, eventSourceUrl, accounts, primaryAccounts, sessionState; requires the urn:ietf:params:jmap:mail capability, gates submission and contacts.
Methods: Mailbox/get|query|changes, Email/get|query|changes|set, Thread/get, Identity/get, EmailSubmission/set, AddressBook/get|changes, ContactCard/get|changes|set.
Protocol details that must be honoured:
- batched POST /jmap with result references (#ids → Email/get),
- Email/get with properties + bodyProperties + fetchAllBodyValues,
- Email/query with anchor/anchorOffset, collapseThreads, calculateTotal, sorts incl. Mailbox/query by sortOrder,
- Email/set patches keywords/<kw> and mailboxIds/<id> using true/null (null removes — Fastmail rejects false),
- updated in a set response is an object {"id": null} — RFC 8620 §5.3 Id[Foo|null], measured on Fastmail and Stalwart 2026-09-29 (jmap-tui's wrapper also tolerates an array, but the RFC shape is the map, and go-jmap's typed field only decodes that),
- EmailSubmission/set with onSuccessUpdateEmail patches, response carries undoStatus, sendAt.
URL/auth policy: cleartext http:// accepted only for loopback; credentials attach to the configured origin and to advertised https non-IP origins only → serve session/API/upload/download/SSE from one origin (D-13).
Degradations the client already implements: no eventSourceUrl → 60 s polling; no contacts capability → contacts hidden. Both are legitimate cut lines.

Not used by jmap-tui but expected of a "proper" JMAP server by other clients: Email/queryChanges, Email/copy, ContactCard/query, AddressBook/set, Identity/set, EmailSubmission/get|query. These are roadmap, not v0.1 gates.

2.2 Backend reality

Backend	Notable capabilities	Consequences
Gmail	`UIDPLUS MOVE CONDSTORE ESEARCH SPECIAL-USE LIST-EXTENDED COMPRESS=DEFLATE UTF8=ACCEPT X-GM-EXT-1`, no QRESYNC	tier-2 sync; `X-GM-LABELS` ↔ mailbox membership; `X-GM-THRID` seeds threads; OAuth2 only; `[Gmail]/All Mail` archive semantics
Dovecot (self-hosted, cPanel)	full extension set incl. QRESYNC, UIDPLUS, MOVE, SPECIAL-USE	tier-1 sync; CONDSTORE/QRESYNC assumed but still detected
Namecheap Private Email, misc providers	unknown mix	detect at runtime; fall back to tier-3 baseline
CardDAV providers	RFC 6352; `sync-collection` (RFC 6578) common but not guaranteed	sync-token path + ctag/ETag diff fallback
Dovecot alone	no CardDAV	contacts capability simply not advertised

Gmail has no CardDAV groups (Google does not map contact labels) and no CalDAV-for-consumers worth depending on → documented gaps, not code paths.

3. Architecture

cmd/jmap-bridge
 └─ internal/httpapi      HTTP: session, /jmap, upload, download, eventsource, oauth, health
     └─ internal/jmapapi  method dispatch, result references, /changes, state strings
         ├─ internal/store      SQLite (metadata, content, FTS5, threads, contacts) + blobs
         ├─ internal/sync       per-account sync engine (passes, IDLE/push, hydration, backfill)
         │   └─ internal/mailbackend  provider-neutral Backend seam (D-API-2, M8)
         │       ├─ internal/imapdrv  CaffeinatedTech/go-imap (fork of kiliant) adapter (QRESYNC, X-GM-*, PREVIEW, COMPRESS)
         │       └─ internal/gmailapi Gmail REST API adapter (M9+, GMAIL_API_PLAN.md)
         ├─ internal/submit     SMTP submission (password / XOAUTH2) + Sent APPEND
         ├─ internal/dav        CardDAV client (discovery, sync-collection, PUT/DELETE)
         ├─ internal/convert    RFC 5322 ↔ JMAP Email · vCard ↔ JSContact
         └─ internal/auth       client tokens, OAuth2 bootstrap + refresh, credential encryption
             └─ internal/config   TOML + env, strict validation

Process model (v0.1): one binary, one goroutine group per configured account (sync loop, hydration workers, IDLE connection), one SQLite database in WAL mode with a single writer. context.Context on every network call. No global mutable state, no init() side effects. Each account holds up to four IMAP sessions — the sync/work connection, a dedicated write session (§7.1), a dedicated hydration session, and an IDLE watcher — so a long backfill never starves a read or a mutation (FR-X.6, NFR-1).

Request flow (read): POST /jmap → auth → per-method handler → SQLite query (index-covered) → assemble JMAP response (properties filtered) → JSON.

Request flow (write): see §7.

4. Data model (SQLite, WAL)

Schema follows the jmap.io server guide — split so each common query is one index range scan.

CREATE TABLE mailboxes (            -- JMAP Mailbox ⇄ IMAP folder
  id TEXT PRIMARY KEY,              -- opaque JMAP id (stable across uidvalidity)
  account TEXT NOT NULL,
  parent_id TEXT, role TEXT,        -- \Inbox \Sent \Drafts \Trash \Junk \Archive via SPECIAL-USE
  name TEXT NOT NULL,               -- IMAP path (hierarchical, separator kept)
  sort_order INTEGER NOT NULL DEFAULT 0,
  uidvalidity INTEGER, uidnext INTEGER, highestmodseq INTEGER,
  total_emails INTEGER NOT NULL DEFAULT 0, unread_emails INTEGER NOT NULL DEFAULT 0,
  total_threads INTEGER NOT NULL DEFAULT 0, unread_threads INTEGER NOT NULL DEFAULT 0,
  may_read_items INTEGER NOT NULL DEFAULT 1, may_add_items INTEGER NOT NULL DEFAULT 1,
  may_remove_items INTEGER NOT NULL DEFAULT 1, may_create_child INTEGER NOT NULL DEFAULT 1,
  may_rename INTEGER NOT NULL DEFAULT 1, may_delete INTEGER NOT NULL DEFAULT 1,
  created_modseq INTEGER NOT NULL, updated_modseq INTEGER NOT NULL,
  updated_not_counts_modseq INTEGER NOT NULL DEFAULT 0,
  deleted INTEGER                   -- deletion modseq (tombstone for /changes)
);
CREATE INDEX mailboxes_changes ON mailboxes(account, updated_modseq);
CREATE INDEX mailboxes_parent ON mailboxes(account, parent_id, deleted);

CREATE TABLE emails (               -- JMAP Email metadata (narrow, hot)
  id TEXT PRIMARY KEY,
  account TEXT NOT NULL,
  thread_id TEXT NOT NULL,
  keywords TEXT NOT NULL,           -- JSON: keyword → true
  received_at INTEGER NOT NULL,     -- unix micros
  size INTEGER NOT NULL,
  has_attachment INTEGER NOT NULL,
  preview TEXT NOT NULL DEFAULT '',
  created_modseq INTEGER NOT NULL,
  updated_modseq INTEGER NOT NULL,
  deleted INTEGER                   -- deletion modseq (tombstone for /changes)
);
CREATE INDEX emails_changes ON emails(account, updated_modseq);
CREATE INDEX emails_thread ON emails(account, thread_id, received_at);
CREATE INDEX emails_recv ON emails(account, received_at);
CREATE INDEX emails_live ON emails(account, deleted, received_at);

CREATE TABLE email_mailbox (        -- MailboxEmailList: membership + history
  account TEXT NOT NULL,
  mailbox_uid INTEGER NOT NULL,     -- mailboxes.rowid: compact per-mailbox key
  email_id TEXT NOT NULL,
  removed_modseq INTEGER NOT NULL DEFAULT 0,
  added_modseq INTEGER NOT NULL DEFAULT 0
);
CREATE UNIQUE INDEX email_mailbox_live ON email_mailbox(mailbox_uid, email_id)
  WHERE removed_modseq = 0;
CREATE INDEX email_mailbox_email ON email_mailbox(email_id);

CREATE TABLE email_content (        -- parsed at backfill; bodies at hydration
  id TEXT PRIMARY KEY,
  headers TEXT NOT NULL,            -- JSON subset (from/to/cc/bcc/replyTo/subject/…)
  message_ids TEXT, in_reply_to TEXT, "references" TEXT,  -- JSON arrays
  sent_at INTEGER,
  subject_l TEXT NOT NULL DEFAULT '',  -- lowercase mirrors for query filters/sorts
  from_l TEXT NOT NULL DEFAULT '',     -- (M5's FTS5 supersedes them for text)
  to_l TEXT NOT NULL DEFAULT '',
  body_structure TEXT NOT NULL DEFAULT '{}',  -- JSON EmailBodyPart tree
  body_values TEXT NOT NULL DEFAULT '{}',     -- partId → {value, isTruncated}
  hydrated_at INTEGER                -- null until the body was fetched
);

CREATE VIRTUAL TABLE email_fts USING fts5(
  email_id UNINDEXED, subject, people, body,
  tokenize = 'unicode61 remove_diacritics 2'
);

CREATE TABLE threads (              -- thread key → thread id (FR-M.7)
  account TEXT NOT NULL,
  thread_key TEXT NOT NULL,         -- 'm:<sha1(Message-ID)>' | 's:<sha1(base subject)>' | 'g:<sha1(GM thrid)>' (M4)
  thread_id TEXT NOT NULL,
  last_seen INTEGER NOT NULL,
  PRIMARY KEY (account, thread_key)
);
CREATE INDEX threads_by_id ON threads(account, thread_id);

CREATE TABLE imap_uids (            -- (folder, uidvalidity, uid) → email id
  account TEXT NOT NULL, folder TEXT NOT NULL, uidvalidity INTEGER NOT NULL,
  uid INTEGER NOT NULL, email_id TEXT NOT NULL,
  PRIMARY KEY (account, folder, uidvalidity, uid)
);
CREATE INDEX imap_uids_email ON imap_uids(email_id);

CREATE TABLE addressbooks (         -- JMAP AddressBook ⇄ CardDAV collection
  id TEXT PRIMARY KEY, account TEXT, href TEXT NOT NULL,
  name TEXT, description TEXT, sort_order INTEGER,
  may_read INTEGER NOT NULL DEFAULT 1, may_write INTEGER NOT NULL DEFAULT 1,
  may_share INTEGER NOT NULL DEFAULT 1, may_delete INTEGER NOT NULL DEFAULT 1,
  sync_token TEXT, ctag TEXT, etag TEXT,
  created_modseq INTEGER NOT NULL, updated_modseq INTEGER NOT NULL, deleted INTEGER
);
CREATE TABLE cards (                -- JSContact ⇄ vCard
  id TEXT PRIMARY KEY,              -- = vCard UID (RFC 9610 §3)
  account TEXT, book_id TEXT NOT NULL,
  uid TEXT NOT NULL, kind TEXT, name TEXT,   -- denormalised for search/sort
  jscontact TEXT NOT NULL,          -- full JSContact JSON
  vcard TEXT NOT NULL,              -- last known wire form
  href TEXT NOT NULL, etag TEXT,    -- CardDAV addressing
  created_modseq INTEGER NOT NULL, updated_modseq INTEGER NOT NULL, deleted INTEGER,
  UNIQUE (account, uid)
);

CREATE TABLE blobs (                -- blobId → relative file path, media type, size
  id TEXT PRIMARY KEY, account TEXT, path TEXT NOT NULL,
  media_type TEXT, size INTEGER, created_at INTEGER
);

CREATE TABLE sync_state (           -- per account/folder sync bookkeeping
  account TEXT NOT NULL, scope TEXT NOT NULL,   -- 'folder:<name>' | 'contacts' | 'account'
  highestmodseq INTEGER NOT NULL DEFAULT 0, sync_token TEXT,
  last_full_scan INTEGER,
  PRIMARY KEY (account, scope)
);

CREATE TABLE counters (             -- process-global monotonic allocators
  name TEXT PRIMARY KEY,            -- 'seq': change counter, feeds id minting
  value INTEGER NOT NULL
);

CREATE TABLE email_msgid (          -- a message's own Message-ID → its email id
  account TEXT NOT NULL, msgid TEXT NOT NULL, email_id TEXT NOT NULL,
  PRIMARY KEY (account, msgid)
);

CREATE TABLE tokens (               -- client-facing auth tokens
  account TEXT PRIMARY KEY, token_hash TEXT NOT NULL
);

CREATE TABLE oauth_tokens (         -- OAuth2 provider tokens (M4, v3)
  account TEXT PRIMARY KEY,
  refresh_token TEXT NOT NULL,      -- sealed by the oauth layer (FR-A.8)
  access_token TEXT NOT NULL DEFAULT '',
  access_expiry INTEGER NOT NULL DEFAULT 0,   -- unix seconds
  updated_at INTEGER NOT NULL
);

Bookkeeping conventions (M1, §4.1 companions):

counters holds one process-global seq that every mutation takes (and that id minting embeds — global so ids cannot collide across accounts).
sync_state row scope='account' holds the account's JSON state floors (emailState, mailboxState, purgedThrough): every mutation raises the relevant floor to its seq, which keeps type states monotonic even after tombstone retention expires, and purgedThrough marks the oldest change /changes can no longer replay. sync_state scope='folder:<name>' holds per-folder sync bookkeeping as JSON ({"uidvalidity":…,"uidnext":…,"highestmodseq":…,"backfill_uid":…}).
Type state strings are the floors above; Thread state is the Email state (any email change can change threads) and queryState's counter is the Email state (PLAN §4.1).
email_content.subject_l/from_l/to_l are lowercase query mirrors of the header subset — M1 filters and sorts on them with LIKE; M5's FTS5 index supersedes them for text search (FR-X).

Blobs live in {data_dir}/blobs/{aa}/{blobId} (raw messages, attachments, contact photos). SQLite holds metadata only; bodies are never inside rows.

4.1 Identifiers and state strings

JMAP ids are opaque, immutable, unique per account and independent of IMAP UIDs — they must survive UIDVALIDITY changes. Form: 13-char base62 of the global seq counter (time-prefixed, per jmap.io guidance, so id order approximates date order for Email/query fast paths).
UIDVALIDITY change → re-point imap_uids, mint new ids for survivors, emit destroys/creates through /changes (clients see a normal change set).
state for Email/Mailbox/Thread/AddressBook/ContactCard = "<typeModseq>" (decimal). Equality short-circuit is all clients may assume.
queryState = "<queryCounter>:<sha1(args)>", bumped on any change that could affect the result (any Email/membership modseq bump is enough for v0.1).

5. Sync engine

Per account: discover → select tier → incremental loop.

Discovery (once per account): CAPABILITY, LIST "" "" with LIST-EXTENDED + SPECIAL-USE (roles) → folders; NAMESPACE (prefix/separator); ENABLE UTF8=ACCEPT where offered; COMPRESS=DEFLATE where offered.

Tiers (highest supported wins):

Tier	Trigger	Incremental	Expunge detection
1 QRESYNC	`QRESYNC` in CAPABILITY	`SELECT (QRESYNC (UIDVALIDITY MODSEQ …))` → `FETCH … (CHANGEDSINCE)` + `VANISHED`	exact (`VANISHED`)
2 CONDSTORE	`CONDSTORE` (Gmail)	`SELECT (CONDSTORE)` → new UIDs by `UIDNEXT` scan + `UID FETCH (CHANGEDSINCE)` for flags	UID gaps → rescan folder
3 Baseline	always	periodic `UID FETCH * (UID)` diff + full `FLAGS` refetch when changed	missing UID → rescan folder

M8 note: tier dispatch, per-folder cursors and the Gmail label strategy live in the internal/imapdrv adapter behind the internal/mailbackend seam (D-API-2); the tables here describe that adapter's behaviour. internal/sync orchestrates passes and commits what the backend reported, reaching the store only through mailbackend.Hooks.

Backfill (initial): per folder, UID FETCH in batches of sync.batch_size with UID, FLAGS, INTERNALDATE, RFC822.SIZE, ENVELOPE, BODYSTRUCTURE — headers only, never bodies. BODYSTRUCTURE gives hasAttachment and the preview target.

Live: IDLE on INBOX + watched folders; on notification run the tier's incremental pass for that folder. Fallback: sync.interval poll. IDLE disconnections back off exponentially. The Gmail API backend has no IDLE: with watch = "pubsub" it arms users.watch, renews before the advertised expiration and calls users.stop on shutdown, and the /gmail/push/{account} handler wakes a history pass on a verified Pub/Sub push (FR-S.14); with watch = "poll" the sync.interval ticker is the path.

Preview: if PREVIEW (RFC 8970) is advertised use it; otherwise a partial BODY[]<0.4096> per message on first summary request, cached in emails.preview.

Body hydration (lazy, D-2): Email/get requesting body properties with hydrated_at IS NULL → UID FETCH BODY.PEEK[] → convert.RFC5322ToEmail → store email_content, split attachments into blobs, insert FTS body tokens, set hydrated_at, bump modseq only if derived summary fields changed (preview usually does → a small /changes is correct and desirable). sync.prefetch_window optionally hydrates recent mail in the background.

Rate discipline: batched FETCH, COMPRESS=DEFLATE when available, bounded hydration workers (search.concurrency), jittered backoff, never a tight loop against a provider.

6. Search (D-6)

Email/query with a text/filter component
 ├─ fast path: FTS5 over subject/people (+ body tokens already indexed)
 │             → respond immediately, queryState as of now
 └─ if backfill enabled and unhydrated candidates exist:
       enqueue hydration jobs (bounded) → each batch parses, indexes FTS,
       bumps query state, emits an SSE StateChange → client re-queries via /changes

Filters supported in v0.1 (what jmap-tui sends): inMailbox, text, from, to, subject, after, before, hasKeyword, hasAttachment; sorts: receivedAt, subject, from, size, hasAttachment, plus Mailbox/query sortOrder; collapseThreads, position/limit/anchor/anchorOffset, calculateTotal.
search.backfill = false degrades to header-only matches (documented).

7. Write, send, and blob paths

7.1 Email/set (IMAP-first, D-14)

JMAP op	IMAP op
update `keywords/$seen`	`UID STORE … (\Seen)` — `$seen↔\Seen`, `$draft↔\Draft`, `$flagged↔\Flagged`, `$answered↔\Answered`, `$deleted↔\Deleted`, `$labelN`/custom keywords ↔ IMAP keywords verbatim
add `mailboxIds/<id>`	`UID COPY` (or `UID MOVE` where the semantics are move)
remove `mailboxIds/<id>`	`UID MOVE` to target, or `COPY` + `STORE \Deleted` + `UID EXPUNGE` (UIDPLUS) fallback
destroy	`STORE \Deleted` + `UID EXPUNGE` (UIDPLUS) in every folder holding the message — destroy is permanent (RFC 8621 §4.6); a client files to Trash with a `mailboxIds` patch instead
create (draft)	`APPEND` to Drafts with `\Draft \Seen` + `$draft`
import raw message (FR-M.19)	`APPEND` the blob's bytes to the first target mailbox, then file the remaining `mailboxIds` through the membership path (Gmail label writes included) — the message is never copied per mailbox by hand
Gmail label change	`UID STORE +X-GM-LABELS (…)` where supported (avoids copy storms)

Local commit + modseq bump + SSE only after the IMAP command returns OK; on failure the JMAP response carries notUpdated/notDestroyed with serverFail. Patch values follow RFC 8620 §5.3 — true adds, null removes — and false also removes (Stalwart accepts it, Fastmail rejects it; accepting both never breaks a client while the canonical form works everywhere). A patch that would leave an Email in no Mailbox is refused with invalidProperties, since the mail store requires at least one membership (RFC 8621 §4.1).

Mailbox/set: CREATE / RENAME / DELETE (+ role detection refresh), with alreadyExists, mailboxHasChild and mailboxHasEmail (unless onDestroyRemoveEmails) reported as the RFC 8621 §2.5 SetErrors.

Gmail write profile (M4, FR-S.10 / FR-M.18). On an X-GM-EXT-1 server the membership commands above become label writes:

JMAP op	Gmail op
add `mailboxIds/<id>`	`UID STORE ±X-GM-LABELS` on one copy holding the message — UIDs are account-global, so one command covers every folder; system folders map to their labels (`INBOX`→`\Inbox`, `[Gmail]/Sent Mail`→`\Sent`, …), user labels are the mailbox path
add to `[Gmail]/All Mail`	no server command — All Mail membership is implicit (every message is there by definition); the membership commits locally because it is the server's state
remove from `[Gmail]/All Mail`	refused: the server model cannot express it (`may_remove_items` is false for the implicit mailbox)
destroy	MOVE every copy to `[Gmail]/Trash`, then `UID EXPUNGE` there — expunge is permanent only inside Trash/Spam; without a trash mailbox the generic expunge-everywhere path is the honest fallback
keywords	flags are per message, not per copy: one STORE covers the account

Archiving is the client's patch "remove INBOX, add archive(=All Mail)": the effective delta is the \Inbox label removal alone, which is exactly Gmail's archive semantics (the message remains listed in All Mail). The \All mailbox maps to JMAP role archive at discovery. After our own writes the reconcile paths apply a grace window (FR-S.12): uids we touched within the last minute are not tombstoned on a folder list that has not caught up yet — Gmail rebuilds its per-folder indexes around label changes, and a premature tombstone is a client-visible lie while a deferred one is one pass late. Gmail's shared-UID namespace is honoured on the read path too: a message first seen through one folder is deduped by (uidvalidity, uid) in every other folder, and backfill skips the header fetch for uids the account already knows, ingesting membership-only rows instead (FR-S.10).

7.2 EmailSubmission/set

Validate every create and every onSuccessUpdateEmail patch first: SMTP cannot be undone, so a malformed argument must fail before anything is sent. emailId may be a plain id or the #handle creation reference of an Email/set create in the same batch (RFC 8620 §3.7), sendAt is refused (maxDelayedSend is 0) and onSuccessDestroyEmail is refused (client-side fallback exists in jmap-tui).
SMTP submit: MAIL FROM = identity address, RCPT TO = the envelope (or To+Cc+Bcc when the client supplied none), AUTH PLAIN over TLS or loopback (XOAUTH2 arrives with M4), TLS per config, and Bcc stripped from the bytes — the envelope still carries those recipients (RFC 8621 §7.5), the Sent copy keeps its Bcc header.
On 2xx: file the sent message, in exactly one of two shapes. The caller's onSuccessUpdateEmail files it when it moves the Email into a mailbox — that is what every composing client sends (Drafts → Sent, $draft cleared) and the move is the filing. Only when the patch says nothing about membership does the bridge APPEND the bytes it just sent to the Sent mailbox with \Seen, because a sent message with no record anywhere is worse than one the server filed. Appending and patching would leave two copies in Sent (jmap-tui's own gate asserts one). Return created with undoStatus: "final" (send-delay is client-side in jmap-tui; no EmailSubmission/undo endpoint in v0.1). Everything after SMTP acceptance is logged, never reported as a failed create — a client that retries a "failed" send sends the message twice.
The onSuccess* effects run as one implicit Email/set after every create has been processed, and its response follows the EmailSubmission/set response (RFC 8621 §7.5); a patch that fails lands in that response's notUpdated.
On SMTP failure: notCreated with the SMTP reply as description. Never APPEND to Sent for a message that wasn't accepted.

7.3 Blobs

POST /{account}/upload/ → size/type caps → blobs row + file → blobId. The session's {accountId} expands into the account path prefix (D-13), so one template serves every account on the origin.
GET /{account}/download/{blobId}/{name}?type={type} — RFC 8620 template expansion exactly as jmap-tui performs it; {name} becomes the Content-Disposition filename and {type} the response's media type.
Draft attachments reference blobIds; the MIME part is materialised at APPEND, and the bytes of a message the bridge built or fetched are kept as its raw blob so Email/get can answer blobId and a submission resends exactly what was stored.

8. Contacts (M6)

Discovery: /.well-known/carddav (RFC 6764) → principal (RFC 5397) → addressbook-home-set → PROPFIND collections (displayname, sort order, getctag, sync-token, privileges).

Sync: REPORT sync-collection (RFC 6578) per book with the stored sync-token; fallback = getctag change → full list diff by ETag; fetch changed cards with addressbook-multiget, then GET for new hrefs.

Writes: create = PUT with If-None-Match: *; update = PUT with If-Match: <etag> (412 → refetch and retry once, then overwritten/serverFail); destroy = DELETE with If-Match. Address book destruction of a non-empty book → addressBookHasContents (RFC 9610 §7.4.1).

Conversion (internal/convert): vCard 3.0 (RFC 2426) and 4.0 (RFC 6350) ⇄ JSContact (RFC 9553) guided by RFC 9554 (vCard Format Extensions for JSContact), so no semantics are silently dropped. Golden-pair fixtures are the test corpus. Known limits, documented not hacked around: one address book per card (CardDAV cannot express JMAP multi-book membership), Google has no groups, photos capped to recognised image types with a size limit.

Capability gating: urn:ietf:params:jmap:contacts appears in the session only when an account has CardDAV configured and the first sync succeeded.

State strings: both contacts types carry a per-account modseq floor, bumped inside the same transaction that commits a server-confirmed change — AddressBook when a book's membership or client-visible properties move (discovery bookkeeping — sync-token/ctag refreshes — never moves it), ContactCard whenever any card changes (including via sync-collection results). Monotonic by construction (golden rule 5); the earlier "hash of book sync-tokens" sketch would have let a server-side token rotation look like a client-visible change.

9. Authentication and credentials

Client-facing (D-15): HTTP Basic, username ignored (any non-empty value), password = the account token; compared constant-time against a stored hash. auth.mode = "none" is accepted only when listen is loopback — the process refuses to start otherwise.

Backend-facing:

password: used for IMAP LOGIN/SASL PLAIN and SMTP AUTH LOGIN.
oauth2 (D-5): authorization-code + PKCE (S256) via GET /oauth/{account}/start → provider consent → GET /oauth/{account}/callback (served by the bridge, reachable through the TLS reverse proxy — README §Deployment). Refresh tokens persisted encrypted; refresh on expiry and on AUTHENTICATE failure with one retry; XOAUTH2 on IMAP and SMTP. provider = "google" fills in endpoints/scopes; provider = "generic" takes them from config (future: Microsoft).

Encryption at rest: credentials and refresh tokens AES-256-GCM with a key from JMAP_BRIDGE_SECRET_KEY (env/K8s Secret, never in the data volume). Without a key the bridge starts in plaintext mode with a loud warning (matching jmap-perl's behaviour, minus the silent default).

Never: log or fmt-print passwords, tokens, Authorization headers, OAuth codes; commit .env* or secrets; echo test credentials.

10. Concurrency and consistency

Single writer per SQLite database (WAL, busy_timeout, one write path).
Sync loop and HTTP handlers share the DB; cross-writes are transactions that bump modseq and queue the SSE notification atomically.
Mutations are serialised per account (a small queue) so a client's read-your- own-writes holds: response state strings reflect the write that just happened.
Gmail's eventual consistency: after our own IMAP write, absent results on the next fetch are not treated as deletions (grace window before tombstoning).
SSE delivery is best-effort; clients always recover via /changes (jmap-tui has a 60 s poll fallback), so a dropped stream is a latency event, not data loss.

11. Observability and operations

Structured logs (log/slog): level via config, request ids, per-account sync stats (folder counts, lag, tier, last pass). Never bodies, never credentials.
GET /healthz (process alive), GET /readyz (every account completed a pass; no account stuck in auth failure).
GET / and GET /privacy: unauthenticated static pages for the OAuth consent screen's home page and privacy policy (FR-D.13); neither carries account data.
Metrics endpoint (optional, text format): sync lag seconds, hydration queue depth, JMAP method counts/errors, IMAP reconnects.
Graceful shutdown: stop accepting HTTP, cancel IDLE, checkpoint SQLite.
Data volume layout: {data_dir}/config-ref, bridge.db, bridge.db-wal, blobs/. Schema migrations via schema_version (forward-only in v0.1).

12. Milestones

Each milestone ends with a hard gate that must be demonstrated before the next begins. FR references point at REQUIREMENTS.md.

Status is written here and nowhere else (REQUIREMENTS §10 maps FRs → milestones but deliberately carries no completion state). Rules:

pending → landed when the deliverable's code and docs merge.
landed → ✅ done only when the gate column has actually been demonstrated, recorded as ✅ done YYYY-MM-DD (<how>). Never mark done on the strength of code that looks right.
Status changes land in the same commit as the gate demonstration (AGENTS.md).

#	Deliverable	Gate	FRs	Status
M0	Repo skeleton, TOML config + validation, HTTP session + `POST /jmap` dispatch (derived from jmap-tui's `test/mockjmap`), fixture store, Dockerfile, gates green	jmap-tui connects to `http://127.0.0.1:PORT/{account}` and browses fixture mail	FR-J.1–.6, FR-A.1–.4 (A.4 config-level; live login lands M1/M3), FR-A.11–.12, FR-D.1	✅ done 2026-09-29 (jmap-tui live suite over loopback)
M1	SQLite store (schema §4), read-only IMAP sync (discovery, tier detection, backfill, IDLE), hydration, `/changes` + SSE, preview	real Dovecot account browsable read-only; a flag flipped in another IMAP client appears in jmap-tui ≤ 2 s	FR-S.1–.9, FR-M.1–.8, FR-J.7–.8	✅ done 2026-09-29 (live local Dovecot 2.4: `test/live` foreign flag → store 505 ms and jmap-tui engine 563 ms, both ≤ 2 s over IDLE+SSE; backfill, hydration and preview against the real server; jmap-tui live suite + `smoke` green over loopback)
M2	Write path §7.1, `Mailbox/set`, drafts, IMAP-first commits	jmap-tui triage (star/archive/move/delete/undo) round-trips; changes visible from a second IMAP client	FR-M.9–.13	✅ done 2026-09-29 (live local Dovecot 2.4: `test/live` write gate — star/move/copy with undo, `Mailbox/set` create/rename/delete, draft `APPEND` with `\Draft`, destroy — each re-read from an independent IMAP session; jmap-tui triage scratch green over loopback 3× — read/star/undo/move/undo/copy/undo/archive/delete-to-trash/destroy, every step confirmed by python3/imaplib; gate caught and fixed `/get` dropping `id` per RFC 8620 §5.1)
M3	Send §7.2, `Identity/get`, blob upload/download, `EmailSubmission/set` with `onSuccessUpdateEmail`	compose → send → message in Sent and delivered to a test sink; attachment round-trip byte-exact	FR-M.14–.17	✅ done 2026-09-29 (`test/live` M3 gate against dev Dovecot + in-process SMTP sink: batched draft+submission, `#draft` creation reference, patch files Sent, sink held the message with its attachment byte-exact and no Bcc, second IMAP session saw one `\Seen` copy — plus fixture-tier suites `TestSubmission*`, `internal/submit` against the SMTP fixture, and the jmap-tui cross-client rig: `TestLiveM5Gate` (compose→attach→send→Sent), `TestLiveM5ProbeDraftRoundTrip` and the M1/M2 bridge gates green over loopback)
M4	Gmail profile: OAuth2 bootstrap, XOAUTH2 IMAP/SMTP, `X-GM-LABELS`↔mailboxes, All Mail/archive, `X-GM-THRID`, CONDSTORE tier validation, rate limits	live Gmail: folders+labels both ways, compose/send, archive from jmap-tui, no rate-limit warnings	FR-A.5–.10, FR-S.10, FR-S.12, FR-M.18	✅ done 2026-10-01 (live Gmail the user's Gmail account, `dev/gate/m4gate.py` once, 14/14 checks, no `[THROTTLED]`, every claim re-read from an independent `imaplib` client: a label applied in Gmail appears in the bridge; archiving from the bridge drops INBOX membership and the undo restores it; compose/send over XOAUTH2 SMTP delivers exactly one copy and files exactly one Sent copy. The gate caught and fixed five real Gmail bugs: a label write sourced from the folder being removed is accepted and silently ignored (archive now sources from All Mail); a fresh message with no cached All Mail copy is addressed there by a Message-ID search (`imapdrv.FindUID`); filing a draft into Sent tombstoned it (non-implicit label adds now commit locally); Gmail's draft label is `\Draft`, not `\Drafts`; and Gmail delays an SMTP submission whose Message-ID matches the upstream draft, so the submission path expunges that draft before SMTP and restores it on failure. Fixture coverage: per-folder UIDs in `internal/sync/gmail_test.go`, `TestGmailArchiveSearchesAllMailWhenUncached`, `TestGmailSendExpungesDraftBeforeSMTP`)
M5	FTS5 + search-driven backfill, filter/sort/anchor/collapseThreads correctness, `PREVIEW`/partial-fetch, COMPRESS, 100k soak	jmap-tui live search cases green; cold browse of a 100k mailbox stays responsive; soak within NFR bounds	FR-X.1–.8, FR-S.11, NFR-1, NFR-2, NFR-8	✅ done 2026-09-30 (dev Dovecot 100k-message corpus, `test/live TestLiveSoak100k`: warm `Email/query` p95 browse 31.1 ms / collapseThreads 32.0 ms / body-token text 82.8 ms, all < NFR-1's 150 ms; hydration 133.4 bodies/s (≥ 15); idle RSS 29.0 MB (< 150 MB, NFR-8); 3 engine-reconnect cycles flat at the 3-goroutine baseline; cold reopen browse 37.7 ms / collapse 32.6 ms — and the same full workload under the race detector with zero data races (the soak relaxes its NFR budgets only under `-race`, which it also enforces verbatim otherwise); jmap-tui cross-client gate `TestLiveSearchVerification` green over loopback: 3000-message text search first page 505 ms, from/subject/hasKeyword/after/before/all-mailbox filters, engine `SearchOpen`/`SearchClose`, and both substring fallbacks (partial-word + fielded-subject scan matched 3000/3000); FTS5 index at ingest + hydration, tombstones, search-driven backfill and batched hydration covered by `internal/store/fts_test.go`, `internal/sync/backfill_search_test.go` and `internal/imapdrv/fetchbodies_test.go`)
M6	CardDAV §8: discovery, sync-collection, PUT/DELETE, vCard↔JSContact, photo blobs, capability gating	jmap-tui `contacts_live_test` suite green against a real CardDAV server; create/edit/delete contact round-trips	FR-P.1–.13	✅ done 2026-09-30 (loopback Radicale 3.8.1 as the real server — `dev/gate/radicale-start.sh`: jmap-tui `TestLiveContactsGate` green over loopback (capability + lazy load, create → SSE ContactCard push < 8 s → rename → destroy, card proven gone from the server); bridge-side `test/live TestLiveContactsGate` green against the same Rig (create/update/move/destroy each re-read from an independent DAV session, RFC 6578 sync-token replay + 412/refetch write guard); the gate caught and fixed a real bug — an update re-minted the card UID (Radicale answered 409 no-uid-conflict), now the uid is canonicalized into the stored JSContact on create and reused on update. Fixture-tier coverage: `internal/dav` (sync/getctag/bare tiers, well-known redirect, conditional PUT/DELETE, multiget/GET), `test/fixturecarddav`, `internal/convert` golden pairs (v3+v4, groups, inline photos, malformed), `internal/store/contacts_test.go` (bookkeeping never moves states, cascade tombstones), `internal/jmapapi` set/patch/error-mapping tests, `internal/httpapi` FR-P.3 session gating test)
M7	Packaging: multi-account paths, credential encryption, metrics/health (`[metrics]` + `/metrics`), UIDVALIDITY recovery drill, Docker image + k8s manifests, README deployment verified, `JMAP-TestSuite` run	fresh `docker run` + `kubectl` install works end-to-end from the docs; JMAP-TestSuite subset for the implemented surface green (deliberate gaps enumerated below); all gates green	FR-M.19, FR-J.9–.10, FR-D.2–.12, NFR-3–.7, NFR-9–.10	✅ done 2026-10-03 (live k3s `v0.1.5` at https://jmap-bridge.geekify.me — cPanel IMAP + Gmail OAuth2 accounts, jmap-tui connected, `/healthz`=ok `/readyz`=ready; `JMAP-TestSuite` vs loopback dev Dovecot: 208/262 required pass, 28 skip, 54 required fail — all in the documented out-of-scope gaps below, zero in-scope failures; `[metrics]` `/metrics` demonstrated `200` enabled vs `404` disabled; FR-D.12 redaction tests; FR-S.6 UIDVALIDITY drill; all local gates green: build, vet, gofumpt, golangci-lint 0 issues, `go test -race`, version smoke)
M8	Gmail API track (D-API-8, `GMAIL_API_PLAN.md`): `internal/mailbackend` seam + `imapdrv` adapter, behaviour-preserving	every existing test green; jmap-tui live suite over loopback still green; go-imap types still confined to `imapdrv`	(refactor)	✅ done 2026-10-03 (all repo tests green incl. `go test ./... -race`; `go build`, `go vet`, `gofumpt`, `golangci-lint` 0 issues; production go-imap imports remain confined to `internal/imapdrv` (its `imapserver` to `test/fixtureimap`); jmap-tui live suite over loopback against the dev Dovecot rig: `TestLiveSessionAndMailboxes`, `TestLiveReaderVerification`, `TestLiveTriageVerification` and `TestLiveSearchVerification` green against the new seam. The compose/identity probes fail and the flip-target tests skip identically on baseline HEAD (the dev account has no Identity and `JMAP_TUI_TEST_FLIP_*` is unset) — verified by building HEAD in a throwaway worktree, so no regression. `internal/sync` now names no provider: tier dispatch, per-folder cursors and the Gmail label strategy moved into the `imapdrv` adapter behind `mailbackend.Backend`; the store is reached only through `Hooks` function values)
M9	Gmail API client on the official `google.golang.org/api/gmail/v1` (pinned) + hand-rolled batch/quota/errors + `test/fixturegmail` + golden pairs	fixture-backed client tests incl. quota/backoff and batch correlation; cost table verified	(groundwork; see `GMAIL_API_PLAN.md`)	✅ done 2026-10-03 (`internal/gmailapi` on `google.golang.org/api v0.300.0` + `golang.org/x/oauth2 v0.37.0`, exact-pinned in `go.mod`; the `x/oauth2` bridge over `internal/oauth.Manager` is proven by a fixture-backed test whose server accepts only the manager-refreshed bearer; token-bucket pacer charges per-method costs; `googleapi.Error` taxonomy maps 429/403-rate→`ErrThrottled` (honouring `Retry-After`), 401→`ErrAuth`, 404→`ErrNotFound`, 403→`RejectedError`, 5xx/transport→retryable; the hand-rolled `/batch/gmail/v1` multipart endpoint correlates by Content-ID, proven against an out-of-order server and a batch whose sub-responses are all `429` under a `200` envelope; `test/fixturegmail` implements profile/labels/messages/history/threads/drafts/attachments/watch/stop/batch with normal + history-expiry + quota + eventual-visibility + unauthorized tiers. The gate's cost-table re-measurement corrected the plan: `messages.get` is 20, not 5, and Google's per-user budget is 6 000 units/min, so the default pacer is 100/s*, not 200 — values pinned by `TestCostTable`. Mapping golden pairs cover labels↔keywords, unsupported-keyword refusal, snippet decoding, thread-key seeding and header case-folding. All local gates green: `go build`, `go vet`, `gofumpt`, `golangci-lint` 0 issues, `go test ./... -race`, version smoke)
M10	`backend`/`[accounts.gmail_api]` config + validation; API discovery, metadata backfill, history incremental, hydration, `/changes`	jmap-tui browses a fixture Gmail account in API mode; live read-only Gmail gate	FR-A.13, FR-S.13, FR-M.1–.8 (new; with code)	✅ done 2026-10-03 (config `backend = "imap"\|"gmail_api"` with per-mode mutual-exclusion validation, Google-only OAuth2, and a loopback `endpoint`+`token` fixture escape; `internal/gmailapi.Backend` implements the read half of `mailbackend` — labels→mailboxes incl. synthetic implicit All Mail, per-container batched `messages.get(format=metadata)` backfill, per-account `history.list` incremental with `404` full-resync, `messages.get(format=raw)` hydration, snippet previews, `g:` thread keys — reusing the unmodified store/engine via a synthetic-UID `native_ids` mapping (schema v7); `cmd/jmap-bridge` picks the backend per account. Fixture gate: `dev/gate/gmailapi-fixture-start.sh` (standalone `test/fixturegmail` + API-mode bridge) — jmap-tui `TestLiveSessionAndMailboxes` green (1 account, 7 mailboxes, 6 roles). Live read-only gate: `dev/gate/gmailapi-live-gate.py` against the user's Gmail (consent re-run 2026-10-03 after the M4 token was revoked) — session over API mode, inbox/sent/archive roles, a created test label+message returned by `Email/query`+`Email/get`, an out-of-band Gmail-API `STARRED` change observed through history incremental, test message+label deleted after. Fixture coverage: `internal/gmailapi/adapter_test.go` and `internal/sync/gmailapi_test.go` (`TestGmailAPIEngineBackfillAndForeignFlag`); all local gates green)
M11	API writes: `Email/set`, `destroy`, drafts, `Mailbox/set` (refusing `onDestroyRemoveEmails=true`)	live write gate on the user's Gmail account, aborting on first `429`	FR-M.9–.13, FR-M.20 (new; with code)	✅ done 2026-10-03 (the write half of `internal/gmailapi.Backend` implemented: `StoreKeywords` folds to one `messages.modify` and refuses Gmail's non-writable keywords with `mailbackend.UnsupportedKeywordsError` (surfaced as `invalidArguments` naming each, D-API-5/FR-M.8); `SetMembership` maps mailbox adds/removes to label modify, treats synthetic All Mail adds as implicit no-ops and refuses its removal; `Destroy` is permanent `messages.delete` with `404` as a no-op success; `Append` creates a Gmail draft via `drafts.create` (drafts-only at M11; M14 added arbitrary `Email/import` over `users.messages.import`, so `Append` now also imports into any writable label) and persists the draft handle; `Mailbox/set` maps to `labels.create`/`patch`/`delete`. `onDestroyRemoveEmails=true` is refused on a label backend via `jmapapi.ErrOnDestroyRemoveEmails` → `invalidProperties` naming it (§16.4). Schema v8 re-keys `gmail_drafts` by the stable synthetic uid (the handle the adapter and engine both hold at Append/commit) and `CommitAppend` fills `native_ids.jmap_id`. Fixed a real M9 mapping bug caught by the write tests: `MapKeywords` mapped `$seen` in the wrong direction (it now inverts against Gmail's `UNREAD`, per §8). Fixture: `test/fixturegmail` gained write helpers and a fix for a label-delete that left a stale id in `labelOrder` (panicked the next `labels.list`). Tests: `internal/gmailapi/write_test.go` (keyword/membership/destroy/draft/mailbox ops incl. refusals) and `internal/sync/gmailapi_write_test.go` (engine write path, unsupported-keyword refusal, draft create, `Mailbox/set` + `onDestroyRemoveEmails` refusal). All local gates green: `go build`, `go vet`, `gofumpt`, `golangci-lint` 0 issues, `go test ./... -race`, version smoke. Live write gate demonstrated 2026-10-03 (`dev/gate/gmailapi-write-gate.py`, run against the user's Gmail account exactly as the M10 read gate was; it aborts on the first `429`): star, read, archive, move, `Mailbox/set` create/rename/delete, `onDestroyRemoveEmails=true` refused (`invalidProperties`), draft create, and permanent destroy — every claim re-read independently through the Gmail API; cleanup removed exactly the gate's `jmap-bridge-test*` objects and its own draft. Also caught and fixed two gate-harness bugs during the run (a stale-cache subject collision and an over-broad draft cleanup). `✅ done`.)
M12	API submission §8.1 via Gmail API (`drafts.send`/`messages.send`, no second Sent APPEND, ambiguous-send reconciliation)	compose → send → exactly one Sent copy + sink delivery; ambiguous-send reconciliation proven	FR-M.14–.17	✅ done 2026-10-04 (live self-send gate green — `dev/gate/gmailapi-send-gate.py` against the user's Gmail account: session served with `urn:ietf:params:jmap:submission`, `Identity/get`, a batched `Email/set` draft create + `EmailSubmission/set` (caller's `onSuccessUpdateEmail` moving Drafts→Sent) → `undoStatus: final`, implicit `Email/set` updated the draft; re-read independently through the Gmail API: exactly one `SENT` copy, an `INBOX` delivery copy (self-send sink), and no `DRAFT` (drafts.send consumed it); the bridge cache held one Sent copy with `$draft` cleared (read-your-writes); only the gate's messages cleaned up. The gate caught and fixed a real bug: `ApplyEmailPatch` issued keyword deltas the message did not carry and Gmail rejects removing an absent label (`"Invalid label: DRAFT"` after `drafts.send`), so the engine now computes effective keyword deltas before any provider call (regression `TestGmailAPIRedundantKeywordDeltaIsNotSent`). Fixture coverage: `internal/sync/gmailapi_submit_test.go` (patch and patch-less filing, `TierAmbiguousSend` reconciliation), `internal/gmailapi/send_test.go` (`drafts.send`/`messages.send`, `\Seen` mapping, ambiguity classification), `internal/httpapi` (submission+identity for a `gmail_api` account with no SMTP block). All local gates green: `go build`, `go vet`, `gofumpt`, `golangci-lint` 0 issues, `go test ./... -race`, version smoke)
M13	Pub/Sub push: `users.watch` + renewal + stop, `/gmail/push/{account}`, `idtoken` verification, `watch="poll"` fallback	fixture foreign change ≤ 2 s through push; watch-renewal expiry simulation; forged push rejected; poll fallback works	FR-S.14, NFR-2 (new; with code)	✅ done 2026-10-04 (fixture end-to-end gate `dev/gate/gmailapi-push-gate.sh` → `test/pushgate`, which wires the real sync engine + `POST /gmail/push/{account}` to `test/fixturegmail`: a verified push made a foreign change visible in ~23 ms (≤ 2 s budget), the watch re-armed before a shortened expiration, a forged push was rejected `401` without nudging the engine, an account with no verifier `404`d, and `watch="poll"` never armed a watch yet still converged. The live Pub/Sub path is operator-run per PLAN §17 and deliberately outside the sign-off gate. Config gained `pubsub_service_account`; push auth supports OIDC (`idtoken.Validate` + email claim) or the `push_allow_plain` shared secret = client token. All local gates green: `go build`, `go vet`, `gofumpt`, `golangci-lint` 0 issues, `go test ./... -race`, version smoke)
M14	API mode docs + re-run M7 packaging/conformance with API mode included	README install works end to end; `JMAP-TestSuite` subset green incl. API mode; all gates green	FR-D.14, NFR-3–.7 (new; with code)	✅ done 2026-10-04 (docs: README "Gmail API mode" gained the Pub/Sub setup and IMAP→API migration note (FR-D.14); the config reference and deploy/k8s examples show `backend`/`[accounts.gmail_api]`. API mode gained `Email/import` over `users.messages.import` (any writable label, multi-mailbox, seen/unread reconciliation, unsupported keywords refused per D-API-5), closing its last missing mail method. The conformance re-run caught and fixed two real bugs: `Identity/get` omitted RFC 8621 §6.1's required `textSignature`/`htmlSignature`/`replyTo`/`bcc`, and `Email/set` create into several mailboxes filed only the first. Conformance re-run of M7's rig: IMAP (dev Dovecot) 208/262 required pass, 54 required fail, 28 skip — identical to M7, zero regression; API mode (in-process Gmail fixture) 202/262 required pass, 73 required fail, 15 skip — every failure in the documented gaps below. All local gates green: `go build`, `go vet`, `gofumpt`, `golangci-lint` 0 issues, `go test ./... -race`, version smoke. Signed off 2026-10-04 with the v0.1.6 tag so the documented install path is runnable; the operator exercises the clean-host README install against the tagged image and files bug reports, which are follow-up fixes rather than a reopening of this gate.)

M8–M14 are the v0.1 Gmail API track (D-API-8, GMAIL_API_PLAN.md). At the user's direction (2026-10-03) M8 begins in parallel with M7's remaining verification, because M7 is near-complete; M7's gate is still owed, the v0.1 tag waits for both tracks, and M14 re-runs M7's packaging/conformance with API mode included. FRs marked "new; with code" are added to REQUIREMENTS.md in the same commit as the code that implements them (golden rule 7).

Verification assets: jmap-tui's live integration suite pointed at the bridge (JMAP_BRIDGE_TEST_*); its mockjmap-derived fixtures seeded M0; Fastmail's JMAP-TestSuite (jmapio/jmap-test-suite, Node/TypeScript) as the external conformance gate at M7.

Known conformance gaps for that M7 run (deliberate, and rejected rather than faked). The external JMAP-TestSuite ran 2026-10-03 against the loopback dev Dovecot rig: 208/262 required tests pass, 28 skip, 54 required fail, and every failure is one of the deliberate gaps below — no in-scope failure remains.

Methods not in v0.1 (roadmap, §15): Email/parse (6), Email/queryChanges (5), Mailbox/queryChanges (4), Email/copy (1), Blob/copy (1), PushSubscription/get|set (7 — SSE eventSourceUrl, RFC 8620 §7.3, is the push transport, not the §7.2 PushSubscription surface), SearchSnippet/get (6), Thread/changes (4).

Sub-features beyond the declared FR subset inside implemented methods:

Email/get header property forms (header:Name:asRaw|asText|asAddresses| asGroupedAddresses|asMessageIds|asDate|asURLs, case-insensitive) — 10; FR-M.4 models the header subset, not the RFC 8621 §4.1.2 as* projections.
Email/query filters outside FR-M.5 (header, minSize/maxSize, notKeyword, some/noneInThreadHaveKeyword, and the AND/OR/NOT filter operators) — 7.
Email/query sorts outside FR-M.5 (to, sentAt, hasKeyword) — 3.
Mailbox/query hasAnyRole (FR-M.2 filters on parentId/role) — 1.
Mailbox/set sortOrder on update — re-derived from the server at discovery and refused, so the requested value is never acknowledged — 1.

Earlier deliberate gaps still stand: Mailbox/set accepts name/parentId and sortOrder on create (stored in the cache until a discovery pass re-derives it, the shape jmap-tui seeds fixtures with); role/isSubscribed on create and sortOrder on update are refused with invalidProperties; /get ignores property names it does not model instead of answering invalidArguments; and Email/get's top-level blobId names the raw copy the bridge holds and is absent until it does, because bodies stay lazy.

API-mode (Gmail) conformance run, 2026-10-04 (JMAP-TestSuite vs the in-process Gmail fixture behind backend = "gmail_api", same suite and enumerated gaps as above): 202/262 required pass, 15 skip, 73 required fail. All 54 IMAP gaps recur (the unsupported methods and sub-features above), plus 19 API-only failures, every one a documented provider property rather than a bug:

Keyword set (D-API-5): Gmail API mode writes only $seen/$flagged/$draft/$important. The suite's seed data carries $answered (thread-reply-2) and $forwarded/custom_label (custom-keywords); those Email/imports are refused, so the tests that read those messages cascade — email/filter-after, email/filter-custom-keyword, email/filter-operator-or, email/filter-some-in-thread-have-keyword, email/filter-text-search-headers, email/get-keywords, email/get-thread-id-consistent, email/header-references, email/set-update-remove-keyword, thread/get-thread-by-id (10). This is the same refusal wall the live M11/M12 gates exercise (unsupported keywords are named in invalidArguments).
Identity set/changes are roadmap (PLAN §15): the dev IMAP account has no identity so those tests skip there, but API mode offers submission (Identity/get) so they run and fail as unknownMethod (8: identity/changes-* ×3, identity/set-* ×5). Identity/get itself passes all five of its tests after the §6.1 signature fix.
onDestroyRemoveEmails=true is refused in API mode (§16.4; a Gmail label does not own its messages), so mailbox/set-on-destroy-remove-emails fails by design (1).

Email/copy/Blob/copy remain roadmap as in the IMAP run. No in-scope failure is unexplained.

13. Risk register

Risk	Impact	Mitigation
`kiliant/go-imap` was new (first release 2026-08, single maintainer, low adoption); hard-forked and self-maintained since 2026-09-30 as `CaffeinatedTech/go-imap`	a client bug stalls sync	owning the fork removes the upstream-stall risk; the driver interface still isolates it (types never leave `internal/imapdrv`); zero-dependency + frozen v1 API keep the fork cheap to carry; its parser fuzzing and Dovecot interop matrix de-risk the gate; tier-3 fallback always exists. `X-GM-LABELS` (M4) is requestable today via its open-ended FETCH item types. All three tiers landed and fixture-verified 2026-09-29 (QRESYNC anchor replay incl. VANISHED, CONDSTORE CHANGEDSINCE, baseline flag refetch), and the QRESYNC tier ran against live Dovecot 2.4 the same day (tier=qresync, COMPRESS=DEFLATE active). The risk materialised on a live provider 2026-09-30 (commercial Dovecot): the wire decoder's 8 KiB response-line cap killed a legitimate 8.4 KiB literal-free `BODYSTRUCTURE` line (one large Outlook-msg message) and poisoned the session — every later command on that connection failed as the same opaque `invalid server response`. Fixed in the fork: default raised to 1 MiB as a memory-containment budget (the grammar bounds no untagged response), encoder literalisation budget decoupled, `MaxLineLength` exposed on `imapclient.Options`, protocol errors now carry the wrapped cause. Verified: full header pass + 50-bodies/folder pass clean on the live server (`TestFetchLongResponseLineParses` pins it).
M4 needed upstream code: `StoreUID` writes labels as bare atoms (Gmail labels with spaces fail) and the decoder could not capture flag-form values in unmodelled FETCH items	Gmail label writes impossible without a fork	hard-forked to `~/projects/go-imap` and published as `github.com/CaffeinatedTech/go-imap` v1.2.2 (2026-09-30), nested `imapserver` v0.2.1, taken at kiliant's `main`: one capability-gated `StoreUIDGmailLabels` command + a flag-form case in `DiscardValue`; PATCH-NOTES.md records the divergence. The bridge now depends on the fork directly — the `go.mod replace` was removed, so the M7 image is unblocked
Gmail eventual consistency after writes	false expunge / duplicate work	grace window before tombstoning own writes (§10) — landed with M4 (record-and-filter in the reconcile paths, FR-S.12)
Gmail's label/UID semantics deviate from the IMAP RFCs: UIDs are per-folder (`X-GM-MSGID` is the account-global id), removing the selected folder's own label is answered `OK` and ignored, the draft label is spelled `\Draft`, and an SMTP submission whose Message-ID already exists as a draft is filed as a draft-like Sent item and relayed only after a long delay	archive/send silently lie to the client (golden rule 1)	The write path never trusts those assumptions: label writes source from a copy outside the removed set, preferring implicit All Mail and falling back to a `UID SEARCH HEADER Message-ID` in All Mail when no copy is cached (`gmailLabelSource`/`gmailAllMailCopy`); the draft label uses the flag form; a label add commits its membership locally so filing does not tombstone; and `SubmitEmail` expunges the upstream draft before SMTP and re-APPENDs it on failure. Verified live 2026-10-01 (`m4gate.py` 14/14)
Gmail API mode reuses the UID-shaped store/engine path for opaque string ids, and an unbounded cold backfill of a large real mailbox would burn quota	the mapping could lie, or a first sync could stall for hours	The adapter allocates a stable synthetic UID per `message.id` in `native_ids` (schema v7) so `PutMessages`/`Locations`/`UpdateFlags` are unchanged and server-first truth is preserved; `history.list` advances only after commit; `messages.get` costs 20 units and is paced, so `backfill_query`/`backfill_limit` bound a cold start (used by the live gate). M10 read path verified live 2026-10-03 (roles/labels/threads/browse + out-of-band flag via history); M11 write path verified live 2026-10-03 (star/read/archive/move, `Mailbox/set` create/rename/delete, draft create, permanent destroy — every claim re-read through the Gmail API)
No Go library for vCard↔JSContact	conversion bugs, data loss	golden-pair fixtures; RFC 9554 as the normative map; Stalwart `calcard` as cross-check reference. Landed with M6 (2026-09-30): `internal/convert` v3+v4 golden pairs with byte-exact round-trip stability, unmodelled properties preserved via `@vcard-ext`; `go-vcard` carries only the field-level codec, the mapping is ours
Radicale answers 409 (not 404/412) for a PUT whose UID collides with another item — an unstable UID turns every bridge edit into a "foreign card"	cards become unupdatable against strict servers	UID stability is a write-path invariant, not a convenience: `prepareContact` canonicalizes the uid into the stored JSContact (create) and falls back to the row's UID (update). Pinned by `TestWriteGeneratedUidStable` + live gate
Provider extension roulette (Namecheap/cPanel)	sync failures	tier detection + capability probing at connect; strict-but-tolerant parsing; live tests per provider as they're added
OAuth callback requires public HTTPS	deploy friction	documented as a first-class deployment path (D-12); port-forward dev mode for everything else
Lazy bodies vs. freetext search expectations	"search misses mail"	backfill is on by default, progress is visible via SSE, `search.backfill=false` documented as a trade
SQLite hot rows (large mailbox counts)	slow list queries	narrow `emails` table + `email_mailbox` covering index (schema §4); count fields maintained incrementally. M5 soak (100k corpus, dev Dovecot) measured warm `Email/query` p95 well inside NFR-1's 150 ms
Search-driven backfill can become a re-fetch storm: every text query re-enqueues the unhydrated candidates it found, and a just-finished body can be re-enqueued before the next scan sees it	wasted provider fetches	the backfill lane drops ids on overflow (the next query re-enqueues survivors) and `doHydrate` short-circuits ids the store already shows hydrated — hydration progress lives in SQLite, which also makes backfill resumable across restarts (FR-X.6)
Cross-folder Message-ID dedupe vs differing IMAP flags: two copies of one self-sent message (`\Seen` in Sent, unflagged in INBOX) cannot share one JMAP object — keywords would flap by whichever folder syncs last, and unread counts with them	unread counts unstable; clients see a delivered self-sent copy born read	dedupe is lossless only: cross-folder, only while the copies have no live membership in the target folder, and only when their flag sets agree (ignoring `\Recent`) — otherwise the copy is its own message (FR-S.3, decided 2026-09-29 after the M2 push gate surfaced the race)
Gmail Pub/Sub push verification is security-critical: `POST /gmail/push/{account}` is on the public origin and cannot carry the client token	a forged or oversized push could trigger needless sync or a DoS	the endpoint fails closed: it verifies the Pub/Sub OIDC token with `idtoken.Validate(pubsub_audience)` and the `pubsub_service_account` email claim, or (only with `push_allow_plain` set for loopback/fixture rigs) the account's client token as a shared secret; the body is capped at 1 MiB. Even a verified push is only a hint — the engine reads `history.list` from its own saved cursor, so a duplicate, replayed or forged-but-verified hint cannot inject state (golden rule 1). Push exists only when `watch = "pubsub"`. Landed with M13 (`internal/gmailapi.PushVerifier` seam, `internal/httpapi` handler, unit + fixture tests)
SMTP submission is inline: a send that is accepted can no longer be rolled back, and a failure after acceptance (filing the Sent copy, applying a patch) cannot be reported without inviting a duplicate send	a sent message with a stale local view	id minted before the send; post-acceptance failures are logged only, and the `onSuccess*` effects run after acceptance in the implicit `Email/set` (§7.2) where a patch failure is visible as `notUpdated`
Scope creep toward calendars/sharing	v0.1 slips	REQUIREMENTS out-of-scope list; roadmap (§15) is where those requests land
Client-token brute force: the token is the only client credential and, before hardening, a wrong token cost one cheap 401 at full request rate	mailbox compromise	tokens require ≥24 characters of entropy (startup error) and failed authentication is throttled then locked out per source+account (`[rate]`, FR-A.3/NFR-5). Landed 2026-10-02: `internal/ratelimit` + `authorize` lockout, unit-tested; the app-side lockout is the real mitigation (the edge cannot tell a wrong token from a valid one), with the edge OAuth `RateLimit` verified live
Source identity behind a proxy/CDN: grouping by the TCP peer lumps every user under the proxy (or a spoofed `X-Forwarded-For` splits one attacker across keys)	lockout misfires or is bypassed	source identity is taken only when the immediate peer is in `rate.trusted_proxies`, with `rate.client_ip_header` naming the provider's single-IP header (`CF-Connecting-IP` for Cloudflare); misconfiguration fails closed to one shared bucket. Landed 2026-10-02 (`ratelimit.ClientIP`, table-tested)
Concurrent requests/EventSource connections exhausted goroutines before hardening: `maxConcurrentRequests`/`maxConcurrentUpload` were advertised but unenforced, and SSE subscribers were unbounded	resource exhaustion / capability dishonesty	advertise == enforce: request and upload semaphores plus per-account/total EventSource caps wired from the same `[rate]` numbers (A4, `httpapi`). Landed 2026-10-02, connection-cap test
FR-D.1 image never actually built: the dev machine has no docker-daemon access (sudo needs a password)	container problems surface late, at M7	native binary is the documented dev/test path (AGENTS.md); `docker build` + the README `docker run` are verified on a docker-capable host before M7 sign-off
The M8 refactor of the engine's provider calls regresses the shipping IMAP backend	a working account breaks silently	M8 is behaviour-preserving: the provider strategy moved into the `imapdrv` adapter behind `mailbackend.Backend` with no semantic change, and the gate is all repo tests (incl. `-race`) plus the jmap-tui live suite over loopback. Demonstrated 2026-10-03: core jmap-tui gates green through the new seam and every remaining live failure/skip reproduces on baseline HEAD in a throwaway worktree
Metrics registration assumed one engine per process: `Registry.GaugeFunc` panicked on a duplicate family name, so a second account crashed startup even with `/metrics` disabled (the registry is always built)	a multi-account bridge cannot start at all	a gauge family now accumulates one callback per source and concatenates their labelled series, matching `CounterVec`'s shared-family model. The M7 metrics work was only ever exercised on the single-account dev rig, and the 2-account live box was still on v0.1.5 (which predates the metrics commit, `7e10308`), so the gap shipped silently in v0.1.6. Fixed 2026-10-04 (v0.1.7): regression test `TestGaugeFamilyAccumulatesSources` plus a 2-account process smoke
The M13 push endpoint was fixture-tested only with a string `historyId`, but Gmail/Cloud Pub/Sub publish it as a JSON number	every real push 400s, so Pub/Sub retries it in a storm and the ≤2 s path never nudges sync while the endpoint looks "up"	decode `historyId` as `json.Number`, which accepts a number and a quoted string; the fixture string form stays covered, and the tests now use the real numeric payload. Fixed 2026-10-04 (v0.1.8, live Pub/Sub)
The cold backfill batched up to 100 `messages.get` sub-requests per `/batch`; Gmail answers "Too many concurrent requests for user" for large batches (its own error guide), so any folder with more than a handful of new messages failed every pass and the initial API sync stalled while `/readyz` stayed `503`	a real Gmail mailbox in API mode never completes its first sync	cap a batch's fan-out at `maxBatchGet = 10`; batching affects round trips, not quota (the pacer is charged the sum), so throughput is unchanged and the fan-out stays under the concurrency limit. Regression test `TestBatchGetCapsFanOut`. Fixed 2026-10-04 (v0.1.9). Cold-start time is still quota-bound, so a large mailbox should also bound the walk with `backfill_query`/`backfill_limit`
In API mode the interactive body fetch shared one quota pacer with the cold backfill, and the batched hydration fetch ran under the client's request context, so a slow (pacer-starved) read that hit the client's timeout cancelled its own download and nothing cached — bodies looked like they would never hydrate, and only folder-level quota warnings appeared in the log	a client cannot read mail while an API backfill runs; the "separate hydration session" gives no isolation because the bottleneck is the shared account quota, not a session	the batched hydration fetch now uses its own bounded engine context (as the single-message path already did), so it caches even when the caller gives up, and the sync pass yields between backfill batches while a read is pending (`hydratePending`/`yieldToHydration`). Tests `TestYieldToHydrationWaitsForPending`/`StopsOnCancel`. Fixed 2026-10-04 (v0.1.10). Operators with a large mailbox should still set `quota_units_per_second` below the per-user per-minute cap (the default 100/s sits exactly at 6000/min and trips a rolling-window 429)
The Gmail API header pass carries no MIME structure, and hydration stored `body_values` without ever building `body_structure`, so an API-mode `Email/get` returned `bodyStructure: {}` with no `textBody`/`htmlBody` — the message rendered blank. The IMAP path never showed it because `BODYSTRUCTURE` arrives with the headers at backfill	API-mode clients cannot read message bodies at all	hydration now builds the tree from the raw bytes (`ParseBody` → `structureOfRaw`, numbered to mirror its values) and `PutHydrated` adopts it when the row has none; migration v9 clears `hydrated_at` on the rows the bug already poisoned so they re-hydrate once. Tests `TestParseBodyStructureMatchesValues`, `TestGmailAPIEngineHydrationBuildsStructure`. Fixed 2026-10-04 (v0.1.11). A new opt-in `[sync] log_hydration` logs one line per body fetch/success/failure for exactly this kind of support case
Gmail API mode's OAuth bridge always asked the manager for a cached access token (`AccessToken(ctx, false)`), and nothing forced a refresh on a `401` — so when Google invalidated an access token before its nominal expiry, every API call replayed the dead token and the account looked wedged (reads/sync `401`, `/readyz` `503`) until the process restarted and the first call refreshed	a production mailbox stops syncing and hydrating for as long as the process runs, hammering the provider with `401`s, with no self-heal	the API client gained the FR-A.7 "on authentication failure, exactly one forced refresh + retry" the IMAP/SMTP path already had: `Options.Reauthorize` is wired to `Manager.AccessToken(ctx, true)`, and `invoke`/`invokeOnce` retry once after a `401` (a send retry is safe — an unauthenticated request was never processed). A dead refresh token still surfaces `ErrReauthNeeded` to `/readyz`. Tests `TestInvokeForcesRefreshOn401`, `TestInvokeAuthRetriedOnlyOnce`, `TestInvokeReauthFailureSurfaces`, `TestInvokeOnceRefreshesOn401`. Fixed 2026-10-04 (v0.1.12)
Interactive body hydration had no admission bound: each `Email/get` claimed its ids and fetched inline, so a client scrolling faster than the provider serves piled up one goroutine and one multi-minute fetch budget per keystroke. On a provider refusal each attempt redialed twice (auth treated as transport), a failed id was retried on every read, and the pending count kept the header backfill yielding — a scroll flood starved the backfill and hammered the provider	one fast client can stall sync for the whole account and turn a transient auth failure into a request storm	interactive hydration now runs through a bounded, drop-oldest queue (`defaultHydrationQueueCap = 10`): the newest read wins and an evicted one answers from cache quietly. `reader.withBackend` treats auth as terminal (no redial), `Ensure` sets/serves `authFailed` instead of retrying a dead account, a failed body gets a 30 s cooldown, and one worker sharing the engine context drains the queue (inline before `Run`). Tests `TestHydrationQueueDropsOldest`, `TestSettleEvictionIsQuiet`, `TestHydrationCooldownSparesFailedBody`, `TestEnsureSkipsWhenAuthFailed`. Fixed 2026-10-04 (v0.1.12)
The v0.1.12 401 self-heal was still dead on the wire: the Gmail client was built with `oauth2.NewClient`, which wraps the source in a `ReuseTokenSource`, and `TokenSourceFromManager` returned a token with no `Expiry` — which `oauth2.Token.Valid()` treats as valid forever, so the transport cached the very first bearer. A forced refresh (the new `Reauthorize`) updated the manager and the store, but every retry replayed the cached dead token, so a Gmail API account stayed wedged on `401`s (`sync: pass failed … invalid authentication credentials`, `/readyz` `503`, pod `0/1`) until the process restarted	a production Gmail mailbox stops syncing indefinitely after Google invalidates an access token before its nominal expiry (reported: synced overnight, then stopped while the IMAP account kept working); the v0.1.12 fix looked correct and its stub-based tests passed because they never exercised the oauth2 transport	the API client is now built on a bare `oauth2.Transport` (which calls the source on every request) instead of `oauth2.NewClient`'s caching wrapper, so refresh is governed by `internal/oauth` and the post-401 retry carries the fresh bearer. Regression test `TestManagerRefreshReachesRetriedRequest` drives the real oauth2 transport through a token endpoint that rotates `tok-1`→`tok-2` and an API that rejects `tok-1` after first use. Fixed 2026-10-06 (v0.1.13)
With the v0.1.13 token fix un-wedging the Gmail API, a real large account never finished its first pass: `doPass` walked every container in Go map order and `backfillLocked` looped to completion, so a huge not-yet-done label (`CATEGORY_PROMOTIONS`/`_UPDATES`) starved INBOX incremental for hours — new mail never appeared and `/readyz` stayed `503` (reported 2026-10-06: "gmail still not picking up today's emails, IMAP account fine"; live DB showed INBOX's history cursor frozen while `native_ids` grew)	a cold Gmail mailbox in API mode looks permanently broken even though every API call succeeds	the pass is steered by intent, not map order: `doPass` visits the hint, then INBOX, then already-backfilled (incremental) containers, then the still-backfilling ones, and a per-pass budget (`Config.BackfillBudget`, default 30 s) makes `backfillLocked` yield mid-walk after its first batch (the cursor is saved, so the next pass resumes); `apiBackfillBatch=50` bounds one adapter batch so a batch fits the budget, and Gmail's `CATEGORY_` tab labels are no longer surfaced as mailboxes (they multiplied the cold backfill per category for no client benefit; GMAIL_API_PLAN §6.1 updated — accepted consequence: the ~7k messages whose only membership was a category tombstone until All Mail backfills them). Regression tests `TestPassSteersIncrementalAheadOfColdBackfill`, `TestMapLabelHidesCategories`. Fixed 2026-10-06* (unreleased)

14. Spec register

Normative for v0.1 (verify status before relying on a draft):

Spec	Subject	Status
RFC 8620	JMAP Core (session, batch, `/changes`, blobs, push)	RFC
RFC 8621	JMAP Mail (Email, Mailbox, Thread, Identity, EmailSubmission)	RFC
RFC 9610	JMAP Contacts (AddressBook, ContactCard)	RFC
RFC 9553 / 9554	JSContact; vCard Format Extensions for JSContact	RFC
RFC 8474	OBJECTID (stable object ids)	RFC (adopted for id guidance)
RFC 3501 / 9051	IMAP4rev1 / rev2	RFC
RFC 2177	IMAP IDLE	RFC
RFC 7162 (obsoletes 4551)	CONDSTORE + QRESYNC	RFC
RFC 4315 / 6851 / 6154 / 5258 / 5819 / 4731 / 5161 / 4978 / 7888 / 6855 / 7889	UIDPLUS, MOVE, SPECIAL-USE, LIST-EXTENDED, LIST-STATUS, ESEARCH, ENABLE, COMPRESS=DEFLATE, LITERAL-, UTF8=ACCEPT, APPENDLIMIT	RFC
RFC 8970	IMAP PREVIEW	RFC
RFC 5321 / 5322 / 6409 / 4954 / 3207	SMTP, message format, submission port, SMTP AUTH, STARTTLS	RFC
RFC 4918 / 6352 / 6578 / 6764 / 5397	WebDAV, CardDAV, Sync Collection, service discovery, DAV:current-user-principal	RFC
RFC 2426 / 6350	vCard 3.0 / 4.0	RFC
Google	IMAP extensions, XOAUTH2, CalDAV/CardDAV OAuth transition (2025-03-14)	vendor docs
`google.golang.org/api/gmail/v1` v0.300.0	Gmail REST API generated client (M9+); package docs https://pkg.go.dev/google.golang.org/api@v0.300.0/gmail/v1	Go module, pinned 2026-10-03
`golang.org/x/oauth2` v0.37.0	OAuth2 client the generated Gmail client is bridged onto (M9+)	Go module, pinned 2026-10-03
Cloud Pub/Sub push	`users.watch` change-notification transport (M13)	vendor docs
draft-ietf-jmap-calendars-29	JMAP Calendars	draft — roadmap only
draft-ietf-calext-jscalendar-icalendar-28	JSCalendar ⇄ iCalendar	draft — roadmap only
RFC 8984	JSCalendar data model	RFC — roadmap only

15. Roadmap (v0.2+)

JMAP Calendar (D-3): urn:ietf:params:jmap:calendars — Calendar/get|query|changes|set, CalendarEvent/get|query|queryChanges|changes|set|copy, ParticipantIdentity/* ⇄ CalDAV (RFC 4791) with VEVENT conversion per draft-ietf-calext-jscalendar-icalendar. Google CalDAV works but is OAuth2-only (scope …/auth/calendar, CalDAV API enabled, endpoint apidata.googleusercontent.com/caldav/v2; the legacy www.google.com/calendar/dav is dead). Dovecot provides no CalDAV → same capability-gating pattern as contacts.
Email/queryChanges, Email/copy, Email/parse, ContactCard/query, AddressBook/set, Identity/set, EmailSubmission/get|query, Thread/changes, SearchSnippet/get, and PushSubscription/get|set (RFC 8620 §7.2; SSE eventSourceUrl covers push today).
The conformance surface M7 deliberately excludes, named in §12: the Email/get header:Name:as* property forms, the Email/query header/ minSize/maxSize/notKeyword/some|noneInThreadHaveKeyword filters and AND/OR/NOT operators, the to/sentAt/hasKeyword sorts, Mailbox/query hasAnyRole, and Mailbox/set sortOrder on update.
Microsoft Graph backend behind the internal/mailbackend seam (D-API-2/4; M365 basic auth dies December 2026): a second implementation of the interface the Gmail API mode introduced, not a new engine.
WebSocket push (RFC 8620 §7.6) alongside SSE.
Backfill steering (partly landed 2026-10-06): doPass now orders INBOX first, then incremental containers, then still-backfilling ones, and caps one pass's backfill work (Config.BackfillBudget) so a cold label cannot starve new-mail sync. Still owed: promoting the folder a client is actively browsing (Email/query/Mailbox/get, or the IDE'd folder) ahead of the other incremental folders. Observed 2026-10-01 on the M4 gate run: a 13-message INBOX backfilled ~13 min in, behind a 6 795-message label folder, leaving the inbox empty in the cache the whole time.
Sieve/vacation, Quota, MDN — only on demonstrated demand.

16. Open questions

Resolved 2026-09-28 and locked as D-16…D-19: size caps (32 MiB JSON / 64 MiB upload) final; AddressBook/set stays roadmap; body cache grow-only; NFR-1 / NFR-8 numbers binding rather than "pin at M5".

No open questions remain for v0.1. M5 measures NFR-1/NFR-8 against the binding numbers; any revision is a REQUIREMENTS edit at sign-off.

17. Testing Gmail API push in production

The M13 sign-off gate is fixture-only (dev/gate/gmailapi-push-gate.sh → test/pushgate), because a real Pub/Sub push needs Google Cloud setup and a publicly reachable HTTPS bridge. This section is the operator runbook for exercising the real path against a production bridge; it is a manual acceptance check, not part of the automated gate.

Prerequisites

A production bridge running M13+ behind TLS, with base_url on its public origin (e.g. https://jmap.example.com) and a gmail_api account already consented (GET /oauth/{account}/start).
The Gmail API enabled on the OAuth client's Cloud project.
gcloud (or Console access) in that project.

1. Create the Pub/Sub topic and grant Gmail publish

gcloud pubsub topics create jmap-bridge-push
gcloud pubsub topics add-iam-policy-binding jmap-bridge-push \
  --member="serviceAccount:gmail-api-push@system.gserviceaccount.com" \
  --role="roles/pubsub.publisher"

2. Create the OIDC push subscription

Use a dedicated service account as the push identity; the bridge checks its email against pubsub_service_account.

gcloud iam service-accounts create jmap-push \
  --display-name="jmap-bridge Pub/Sub push"
gcloud pubsub subscriptions create jmap-bridge-push-sub \
  --topic=jmap-bridge-push \
  --push-endpoint="https://jmap.example.com/gmail/push/<account>" \
  --push-auth-service-account="jmap-push@<project>.iam.gserviceaccount.com" \
  --push-auth-token-audience="https://jmap.example.com"

3. Configure the bridge

[[accounts]]
id = "gmail"
backend = "gmail_api"
# …

  [accounts.gmail_api]
  watch                   = "pubsub"
  pubsub_topic            = "projects/<project>/topics/jmap-bridge-push"
  pubsub_audience         = "https://jmap.example.com"   # must equal the subscription audience
  pubsub_service_account  = "jmap-push@<project>.iam.gserviceaccount.com"

pubsub_audience and pubsub_service_account are both required in pubsub mode unless push_allow_plain = true (loopback/fixture only, where the shared secret is the account's client token). Restart the bridge; the startup logs should show the engine arming the watch (gmailapi: watch renewed on the daily renewal). If watch defaults to poll because the topic was missing, no watch is armed and latency is sync.interval.

4. Observe a foreign change end to end

Record the account's Email state: Email/get → state (or subscribe to GET /{account}/eventsource/).
From the Gmail web UI or another client, star/unstar a message (or send a message into INBOX).
Within ~2 s (Email/changes since the recorded state, or an SSE StateChange) the change must appear. The bridge logs the push at debug level and the resulting history pass.

If it does not, check in order: the subscription's push endpoint URL and audience match the config; the bridge is reachable from Pub/Sub (Pub/Sub push requires public HTTPS); GET /readyz is ready; the account is not in auth-failure. A 401 in the subscription's dead-letter/ack metrics means the OIDC email or audience does not match pubsub_service_account / pubsub_audience.

5. Negative and fallback checks

Forged push: curl -X POST https://jmap.example.com/gmail/push/gmail -d '{}' with no/invalid bearer must answer 401, and no sync pass may run. gcloud pubsub subscriptions describe shows undelivered/retry counts if a real credential is broken.
Renewal: the watch is re-armed before its 7-day expiration (at most daily). It is observable in the debug logs; users.watch also returns a fresh expiration on each renewal.
Poll fallback: set watch = "poll" (no Pub/Sub needed) and confirm a foreign change appears within sync.interval; this is the documented degraded mode and needs no GCP setup.
Shutdown: on SIGTERM the bridge calls users.stop; a subsequent gcloud pubsub topics list/subscription check shows no new deliveries until the next start re-arms the watch.

Navigation Menu

Repository navigation

PLAN.md

PLAN.md

PLAN — jmap-bridge

1. Locked decisions

2. Constraints that shape the design

2.1 The client contract (what jmap-tui actually exercises)

2.2 Backend reality

3. Architecture

4. Data model (SQLite, WAL)

4.1 Identifiers and state strings

5. Sync engine

6. Search (D-6)

7. Write, send, and blob paths

7.1 Email/set (IMAP-first, D-14)

7.2 EmailSubmission/set

7.3 Blobs

8. Contacts (M6)

9. Authentication and credentials

10. Concurrency and consistency

11. Observability and operations

12. Milestones

13. Risk register

14. Spec register

15. Roadmap (v0.2+)

16. Open questions

17. Testing Gmail API push in production

Prerequisites

1. Create the Pub/Sub topic and grant Gmail publish

2. Create the OIDC push subscription

3. Configure the bridge

4. Observe a foreign change end to end

5. Negative and fallback checks

FilesExpand file tree

PLAN.md

Latest commit

History

PLAN.md

File metadata and controls

PLAN — jmap-bridge

1. Locked decisions

2. Constraints that shape the design

2.1 The client contract (what jmap-tui actually exercises)

2.2 Backend reality

3. Architecture

4. Data model (SQLite, WAL)

4.1 Identifiers and state strings

5. Sync engine

6. Search (D-6)

7. Write, send, and blob paths

7.1 Email/set (IMAP-first, D-14)

7.2 EmailSubmission/set

7.3 Blobs

8. Contacts (M6)

9. Authentication and credentials

10. Concurrency and consistency

11. Observability and operations

12. Milestones

13. Risk register

14. Spec register

15. Roadmap (v0.2+)

16. Open questions

17. Testing Gmail API push in production

Prerequisites

1. Create the Pub/Sub topic and grant Gmail publish

2. Create the OIDC push subscription

3. Configure the bridge

4. Observe a foreign change end to end

5. Negative and fallback checks