obsidian-vault/work/projects/cpm-project-summary-2026-11-03.md

tags, updated, status

tags	updated	status
work cpm summary	2026-11-03	draft

CPM stuck-messaging project: where we are, Nov/3

Internal metrics, keep out of public PRs. Task: CPM: root-cause the stuck embedded extension messaging. Details: cpm-post-release-analysis-2026-10-27, cpm-status-2026-10-15.

What the project has achieved

Goal: permanent stuck reports below 1% of DAU (0.2-0.5% = success). Permanent = no answer for 5 minutes, repeated or needed a reload.

Now (macOS 1.214, iOS 7.244, Nov/2):

• iOS: 0.44% of DAU (0.43% before). Inside the target.
• macOS: 1.89%. Above the target.

Fixes shipped:

• P3, OS<26.6 limit – expected: the extension starts without delay after restarts (up to 32 s delay without it in lab tests). Flag on in 7.244+/1.214+. iOS 26.6+, transient: users with a failure 14.6% → 8.3% of DAU, "background missing" 8.2% → 0.3% (12x higher than older iOS before, now equal). Permanent: no improvement (0.29% → 0.48%). Early-adopter sample.
• macOS: pinned tab lost on window close, tab merging – real bugs fixed, UI tests added.
• P1/P2/P4 – recovery after wake and startup failures. Triggered 2,026 times on iOS, 89 on macOS (Oct/26). Permanent: no visible drop.
• Network process crash – restart after the networking process crashes. Triggered 476 times on iOS, 277 on macOS (Oct/26). Permanent: no visible drop.
• State restoration race – extension failing to start while tabs restore. Effect not measured.

Causes proved by reproduction:

• iOS: after "error 6" the background stays broken and every wake fails again (harness). Fix goes public Nov/9.
• macOS: a tab the extension is not connected to never gets an answer. 0.42% of DAU, no fix yet.

Hypotheses rejected:

• Deallocation race (A/B, flag) – fix shipped behind a flag never turned on, no relation in the data.
• Reload on the stuck signal – 46% of macOS cases still got no answer. Late page reload never triggered.
• Lost message after wake – 0 of ~1,500 wakes in tests (Reproduce the issue).
• Hung background process – under 1% of failures.
• Memory pressure, flag toggling, "failed load blocks forever" – not reproduced.

Fixes going public next week (Nov/9)

• Upgrade before loading – load the new extension version first on launch. Removes part of the permanent failures right after an update (macOS 0.26% of DAU, iOS 0.05%).
• Error 6 restart – restart the extension when a load error leaves it broken. Up to 0.08% of DAU permanent on iOS, negligible on macOS.
• Together: macOS 1.89% → about 1.6%, iOS 0.44% → about 0.3-0.35%. Estimates, upper bounds.

Data now (% of DAU, Nov/2)

One day, early adopters, macOS about 9k DAU: a rough picture.

• macOS: permanent 1.89% (browser-wide 1.22%, one tab 0.68%).
  • Failures right after a wake 0.68%, tab not connected 0.42%, other 0.30%, right after an update 0.26%, right after a load 0.19%.
• iOS: permanent 0.44% (browser-wide 0.26%, one tab 0.18%).
  • Right after a load 0.18%, error 6 0.08%, right after a wake 0.06%, right after an update 0.05%, other 0.04%.
• No fix in flight for macOS wake failures (35% of permanent) and tab not connected (22%).

Plan

This week, cleanup:

• Feature flags cleanup:
  • Remove fix and flag (did not work or never ran): cpmBackgroundGraveyardTreatment, cpmLateBackgroundDocumentRecovery, cpmMessagingHangRecovery, cpmBackgroundLoadRecovery.
  • Remove with the diagnostics: cpmDiagnosticsRecorder, cpmBackgroundDelegateProxy.
  • Keep the fix, remove the failsafe flag: cpmNetworkProcessRecovery, cpmStableBaseURL, webExtensionSharedPinnedTabWindowClose, webExtensionStateRestorationGate, cpmBackgroundLoadRestart (after Nov/9 data).
• Pixel data cleanup: remove the temporary pixels and episode parameters that only served the investigation.

Next, metrics:

• Permanent breakage metrics: a small fixed set, rates against DAU, added to the health indicators dashboard. Task to create.
• Extension lifecycle metrics: find where the startup message stops, to diagnose the wake failures. Needs privacy review and releases, data in 1-2 release cycles. Task to create.
• Data check of the Nov/9 fixes: Analyze attribution data after stage 3.

Open:

• No fix for macOS wake failures and tab not connected. If the new metrics show nothing actionable, close the project and reset the target.