[2026-10-28] eagle: work/projects/cpm-post-release-analysis-2026-10-27.md

work/projects/cpm-post-release-analysis-2026-10-27.md

@@ -10,6 +10,47 @@ tags:
 
 Internal metrics; keep these figures out of public PRs. Previous analysis: [[cpm-metrics-causes-and-mitigations-2026-10-16]], [[cpm-status-2026-10-15]] (§3–5, 12–14 October cut).
 
+## Priorities — macOS permanent failures first
+
+macOS fails far more (28.5% of DAU with a failure, 2.87% permanent, vs iOS 9.1% / 0.43%), so fixes are ordered by their share of **macOS permanent failures** (26–27 Oct, 598 users). Sources: idle wake 38%, P6 22%, extension updated 15%, other 14%, recent context load 7%, no background view 3%, error 6 0.5%.
+
+1. **Origin change → release P3 (#7493).** Cuts across groups: the macOS idle-wake patterns with the highest repeat rates follow `base_url_changed` (58–66% of their episodes permanent, 13–27 episodes per user, small groups). Every M4 reload and launch on released builds changes the origin, with a reproduced 1.5–32 s blackout. Already implemented; verify the effect on internal 1.214 data.
+2. **P6, tab not connected (22% of macOS permanent; 39% of P6 users end up permanent).** Trigger unknown; the flag is not toggled in production. ~37% of P6 users show `extension_updated` before the failure, so investigate the mid-session upgrade path first, then fix.
+3. **Extension updated (15%).** Load the new bundled version before the old one at launch, and keep the origin across upgrades. Repeats after every app update and likely feeds P6.
+4. **Idle wake (38%, cause unknown).**
+   - Check Network-recovery false positives: recovery applies on 7.5% of macOS DAU vs 0.9% on iOS, and `died_client` precedes 16% of macOS idle-wake failures.
+   - Add telemetry: events after the first failure, a healthy-navigation baseline, retry/recovery applied in the episode.
+   - Keep-warm experiment behind a flag (also a causal test).
+5. **Other (14%):** needs the telemetry above.
+
+Then iOS: error-6 loop (20% of iOS permanent; restart the stale background and re-send `init`), recent context load (40%; origin change, covered by P3), extension updated (13%; same fix as macOS step 3).
+
+## Summary for the team — 26–27 Oct
+
+**CPM post-release analysis: iOS 7.243.0 / macOS 1.213.0, 26–27 Oct (2 full days)**
+
+Definitions: failure = no CPM dashboard response within 4 s after a page finished. Episode = one failure until resolution or the 5-minute deadline. Permanent = no response for 5 minutes, confirmed by a repeat failure or a reload. Affected users = users who reported a failure that day (daily pixel); all percentages are of DAU. Episodes = every failure report (count pixel).
+
+**Size**
+- iOS: 9.1% of DAU hit a failure, 0.43% permanently. About 3× fewer failures than the previous release.
+- macOS: 28.5% of DAU hit a failure, 2.87% permanently. No change vs the previous release.
+- Most failures recover by themselves: 89% on iOS, 63% on macOS.
+
+**Where permanent failures come from** (share of permanent users)
+- iOS (0.43% of DAU): recent context load 40%, error 6 20%, idle wake 16%, extension updated 13%, other 7%.
+- macOS (2.87% of DAU): idle wake 38%, P6 (tab not connected) 22%, extension updated 15%, other 14%.
+
+**Persistent mechanisms** (repeat episodes from the same users confirm a stuck state)
+- **iOS error-6 loop:** error 6 hits 0.86% of iOS DAU, and 10% of those users stay stuck. Stuck users average 7.4 episodes, recovered ones 1.2. In 38% of error-6 episodes the background keeps failing to load on every wake. Matches the stale-failed-background state reproduced in the harness. *Fix: restart the stale background on error 6 and re-send `init` to the page.*
+- **Failures after an origin change (`base_url_changed`):** present in 1,733 iOS users (14% permanent, 3× baseline) and 193 macOS users (16%). On macOS these installs fail repeatedly (13–27 episodes per user in the idle-wake patterns, from small groups). The origin change itself causes a 1.5–32 s blackout depending on cache size (reproduced); P3 removes it. *Fix: release P3; verify on internal 1.214 / 7.244 data.*
+- **macOS P6:** 1.6% of DAU, 39% of them permanent, about 2 episodes per user. A stuck tab, not a loop. ~37% of P6 users show `extension_updated` before the failure, so the mid-session upgrade is a likely trigger. *Fix: find the trigger (upgrade path first), then fix.*
+
+**Transient, not a stuck state**
+- Extension updated: one episode per user, at upgrade time. Still worth fixing (load the new version first, keep the origin), since it repeats after every app update.
+- Idle wake: mostly recovers (iOS 88%, macOS 58%). Its repeat-heavy part overlaps the origin-change patterns. Cause unknown; it may just be the state most macOS navigations are in.
+
+**Caveats:** two days after release, mostly early updaters (macOS ~17k DAU). Loop figures come from small groups. Not measured yet: retry attempts, events after the first failure, a healthy-navigation baseline.
+
 ## What shipped
 
 - iOS **7.243.0** (tag 26 Oct 19:23 UTC), macOS **1.213.0** (tag 26 Oct 16:23 UTC).
@@ -221,6 +262,9 @@ Remaining untested production ingredients: the real macOS `Tab` stack (DDG user
 - **Extension updated: yes, the outdated version is loaded.** `WebExtensionLifecycleCoordinator.loadAndSync` runs `loadInstalledExtensions()` (the previous, now outdated version) before `syncEmbeddedExtensions`, which installs the bundled version under a new identifier, loads it alongside, and unloads the old one. Early pages after an app update run against the old version, then the context is swapped with a new origin and the migration blackout.
 
 ### Next steps
+
+Order superseded by **Priorities — macOS permanent failures first** at the top of this note (28 Oct, two-day data); the items below remain the detailed list.
+
 1. **Ship what's understood:**
    - P3 release.
    - Extension updated: upgrade before load at launch; carry the origin across upgrades.