[2026-10-30] eagle: work/projects/cpm-post-release-analysis-2026-10-27.md work/projects/cpm-status-2026-10-15.md work/projects/cpm-web-extension-breakage-findings.md work/projects/cpm-web-extension-code-guide-2026-10-09.md
This commit is contained in:
1 parent
558832b4fb
commit
9623dd1e5d
4 files changed
+74
-16
No files matched your search
@@ -1,15 +1,45 @@
|
||||
---
|
||||
title: 'CPM — first post-release data (iOS 7.243.0 / macOS 1.213.0)'
|
||||
title: CPM — first post-release data (iOS 7.243.0 / macOS 1.213.0)
|
||||
date: '2026-10-27'
|
||||
tags:
|
||||
- work
|
||||
- cpm
|
||||
- analysis
|
||||
updated: '2026-10-30'
|
||||
---
|
||||
# CPM: first post-release data, 26 October 2026
|
||||
|
||||
Internal metrics; keep these figures out of public PRs. Previous analysis: [[cpm-metrics-causes-and-mitigations-2026-10-16]], [[cpm-status-2026-10-15]] (§3–5, 12–14 October cut).
|
||||
|
||||
## Release update — 30 October 2026
|
||||
|
||||
- **Merged, not yet verified as publicly released:** [iOS #7753](https://github.com/duckduckgo/apple-browsers/pull/7753) into `release/ios/7.244.0` (merge `26a461e30e73608e75e6d84e09bca0ced842993c`) and [macOS #7752](https://github.com/duckduckgo/apple-browsers/pull/7752) into `release/macos/1.214.0` (merge `1e392bac233d7e3e5ccef340ebc82ad7aee72922`). Latest public GitHub releases checked on 30 Oct are still 7.243.0 / 1.213.0. Merge does not establish rollout or production effect.
|
||||
- **Final stable-origin behavior:** set `baseURL` only on macOS/iOS 26.6+ with the feature flag enabled. Creating a context with an unsupported OS or disabled flag rotates the stored CPM origin and returns `nil` (WebKit chooses the context's origin). Toggling the flag alone does not change a live context; an off→on interval with no intervening context creation does not rotate anything. User reports the remote flag disabled; live remote configuration was not independently audited.
|
||||
- **Only a storage-key rename:** property `cpmOriginIdentifier` now encodes/decodes JSON key `cpmStableOriginIdentifier`. Old `cpmOriginIdentifier` values are ignored. There is no migration, UUID prefill, OS-version marker or reset on every OS upgrade. Missing new-key value falls back to the already persisted installation `uniqueIdentifier`; this is reuse of that UUID, not creation of a fresh UUID. Subsequent launches reuse it. A later normal rotation saves a replacement under the new key and reuses that replacement.
|
||||
- **Public-build transition:** 7.243 / 1.213 persist the installation UUID but do not assign it as the extension's baseURL. WebKit chooses a random origin for each new context. The first enabled stable-origin load therefore changes the origin once. An internal build with a replacement saved under the old key falls back to the installation UUID after this key change.
|
||||
- **Error-page fix:** the health monitor skips error-page commit/finish; macOS maps `.alternateHtmlLoad` to `.errorPage`. These documents no longer start timeout measurements, episodes or M4 reloads. This does not establish what fraction of production wake episodes were error pages.
|
||||
- **Validation:** the key-only revision passed all 1,025 WebExtensions package tests and strict lint; the final simplified policy suite passed 25 tests, including injected unsupported-OS coverage, with strict lint on both branches. The original release-base error-page change passed three macOS app integration tests (DNS, SSL and malicious-site warning), with a failing negative control when the mapping was removed. Those app tests were not rerun after the key/test-only edits; iOS runtime behavior was not tested. Final branch heads: iOS `fef5f9cdd35977b0cb848ac81f4feaca3e6a9ba6`, macOS `551cb2edd00c02e038b2b46a07a8c32d0b33e666`.
|
||||
- **Mock follow-up:** the local `StubWebExtensionManager` stays in the release tests by reviewer agreement. Moving them to the shared mock is a [main-branch follow-up](https://app.asana.com/1/137249556945/project/1202406491309510/task/1220003778559454?focus=true).
|
||||
|
||||
### Stable-origin evidence and limits
|
||||
|
||||
The demonstrated benefit is avoiding WebKit's origin-change data cleanup before injection/background startup: real-cache reproductions measured about 1.5 / 10.9 / 32.2 seconds at 20k / 200k / 600k cache entries with changing origins, versus about 0.1 seconds with a stable origin. See [WebKit bug 320812](https://bugs.webkit.org/show_bug.cgi?id=320812) and the reproduction notes below. Production symptom cohorts are not proof that every matching episode has this cause.
|
||||
|
||||
No test demonstrated an extension staying permanently broken after an OS upgrade because its baseURL was saved. A live-session background-process termination did reproduce an error-6 loop that origin rotation recovered. In the real two-process restart baseline, a full app restart restored messaging even while reusing the previous origin. Evidence: `alex/cpm-p3-integration-tests/macOS/IntegrationTests/CPM/README.md`, “two-process” prepare/resume results. Do not turn the live-session failure into a proven persistent-upgrade failure.
|
||||
|
||||
The earlier claim that extension workers were *never* persisted was incorrect. [WebKit commit eff9f9231ad90e31e6ae2b128d2269c589550c8d](https://github.com/WebKit/WebKit/commit/eff9f9231ad90e31e6ae2b128d2269c589550c8d) (312316@main, 29 April 2026) stops new extension service-worker registrations being written to disk; that change does not purge existing records. Historical records are therefore possible, but this does not demonstrate the proposed upgrade failure.
|
||||
|
||||
Pixels can show no response within five minutes, cross-tab failures, error 6 and reload outcomes. They cannot uniquely identify old-origin reuse after an OS upgrade; `base_url_changed` compares contexts within the current process, not across launches. “Permanent” in this analysis means the five-minute reporting outcome, not proof of an unrecoverable extension.
|
||||
|
||||
### OS split of the no-background-view symptom — query on 29 October
|
||||
|
||||
26–28 Oct (metrics date in Eastern time), released iOS 7.243.x / macOS 1.213.x without P3. Count-leg episode rows only, not users: `context_loaded=true AND bg_view_alive=false`, excluding a nonempty `bg_load_error_after` or an `error_[0-9]` event in `bg_events`.
|
||||
|
||||
- iOS: OS <26.6 — 695 of 14,537 failure episodes (4.78%); OS 26.6+ — 27,670 of 97,982 (28.24%). Thus 97.55% of this symptom cohort is on 26.6+. Confirmed no-response subset: 43 / 347 respectively.
|
||||
- macOS: OS <26.6 — 274 of 13,206 (2.07%); OS 26.6+ — 961 of 29,770 (3.23%). Thus 77.81% of this symptom cohort is on 26.6+. Confirmed no-response subset: 29 / 98 respectively.
|
||||
|
||||
Source: `metrics.pixels`, `pixel_id='web'`, `web.extension.cpm.episode.count%`, agents `ddg_ios` / `ddg_mac_desktop`; OS split by major/minor tuple at (26,6). These are symptom counts, not a causal P3 share or a stable-origin safety comparison. The query was run on 29 Oct; it was not rerun for this documentation update.
|
||||
|
||||
## Priorities — macOS browser-wide breakage first
|
||||
|
||||
macOS fails far more than iOS, so priorities follow **cross-tab permanent failures on macOS**: CPM not answering in two or more tabs for 5 minutes, confirmed by a repeat or reload. These are browser-wide breakage, not one stuck tab. 26–27 Oct, macOS 1.213: **386 users, 1.85% of DAU**.
|
||||
@@ -25,11 +55,11 @@ Share of macOS cross-tab permanent: idle wake 174 (45%), other 78 (20%), extensi
|
||||
3. **Other (20%).** No distinguishing signal yet; needs the telemetry from step 1.
|
||||
|
||||
Not priorities:
|
||||
- **P3 / origin change:** the baseURL fix is released. Verify its effect on recent context load, post-reload silence and the origin-change patterns (`base_url_changed`) once the build reaches users; no further work planned.
|
||||
- **P3 / origin change:** the baseURL fix is in the upcoming 7.244 / 1.214 release branches. Public rollout and an enabled flag are prerequisites for measuring its effect; recent context load is not by itself proof of origin-change cleanup.
|
||||
- **P6:** 96% of P6 users have a single failing tab; 122 of its 131 permanent users are single-tab, only 9 cross-tab. One stuck tab, not browser-wide breakage. Low priority; find the trigger when convenient (~37% show `extension_updated` before the failure).
|
||||
- **Error 6 on macOS:** 2 cross-tab users.
|
||||
|
||||
iOS (cross-tab permanent 861 users, 0.26% of DAU): recent context load 38% (origin change; covered by the released P3), error 6 20% (error-6 loop: restart the stale background and re-send `init`), extension updated 14% (same fix as macOS step 2), idle wake 14%, other 8%, no background view 5%.
|
||||
iOS (cross-tab permanent 861 users, 0.26% of DAU): recent context load 38% (a symptom targeted in part by upcoming P3; P3 is absent from this measured release), error 6 20% (error-6 loop: restart the stale background and re-send `init`), extension updated 14% (same fix as macOS step 2), idle wake 14%, other 8%, no background view 5%.
|
||||
|
||||
## Summary for the team — 26–27 Oct
|
||||
|
||||
@@ -48,7 +78,7 @@ Definitions: failure = no CPM dashboard response within 4 s after a page finishe
|
||||
|
||||
**Persistent mechanisms** (repeat episodes from the same users confirm a stuck state)
|
||||
- **iOS error-6 loop:** error 6 hits 0.86% of iOS DAU, and 10% of those users stay stuck. Stuck users average 7.4 episodes, recovered ones 1.2. In 38% of error-6 episodes the background keeps failing to load on every wake. Matches the stale-failed-background state reproduced in the harness. *Fix: restart the stale background on error 6 and re-send `init` to the page.*
|
||||
- **Failures after an origin change (`base_url_changed`):** present in 1,733 iOS users (14% permanent, 3× baseline) and 193 macOS users (16%). On macOS these installs fail repeatedly (13–27 episodes per user in the idle-wake patterns, from small groups). The origin change itself causes a 1.5–32 s blackout depending on cache size (reproduced); P3 removes it. *P3 is released; verify its effect.*
|
||||
- **Failures after an origin change (`base_url_changed`):** present in 1,733 iOS users (14% permanent, 3× baseline) and 193 macOS users (16%). On macOS these installs fail repeatedly (13–27 episodes per user in the idle-wake patterns, from small groups). The origin change itself causes a 1.5–32 s blackout depending on cache size (reproduced); P3 removes it. *P3 is merged into upcoming release branches; verify its effect after public rollout and flag enablement.*
|
||||
- **macOS P6:** 1.6% of DAU, 39% of them permanent, about 2 episodes per user. 96% are a single stuck tab (122 of 131 permanent are single-tab), not browser-wide breakage. ~37% of P6 users show `extension_updated` before the failure, so the mid-session upgrade is a likely trigger. *Fix: find the trigger (upgrade path first), then fix.*
|
||||
|
||||
**Transient, not a stuck state**
|
||||
@@ -57,12 +87,12 @@ Definitions: failure = no CPM dashboard response within 4 s after a page finishe
|
||||
|
||||
**Caveats:** two days after release, mostly early updaters (macOS ~17k DAU). Loop figures come from small groups. Not measured yet: retry attempts, events after the first failure, a healthy-navigation baseline.
|
||||
|
||||
## What shipped
|
||||
## What shipped in the measured 7.243.0 / 1.213.0 releases
|
||||
|
||||
- iOS **7.243.0** (tag 26 Oct 19:23 UTC), macOS **1.213.0** (tag 26 Oct 16:23 UTC).
|
||||
- Both: episode telemetry #7376/#7484, episode tab scope #7520/#7521, P1/P2/P4 recovery #7379/#7536, Network-process recovery #7459/#7531.
|
||||
- macOS only: P5 shared-pinned window close #7473, remote/debug disable #7498.
|
||||
- **Not shipped:** P3 stable origin #7493 (on main only). **No P6 fix exists in either release**: no PR implements the controller rebind.
|
||||
- **Absent from these measured releases:** P3 stable origin #7493; it is now in the upcoming 7.244 / 1.214 release branches. **No P6 fix exists in either release**: no PR implements the controller rebind.
|
||||
- Shipped episode contract is the rolled-back variant: `was_stuck` (bool), single-phase `bg_events` (no `;`), plus `tabs_affected`, `repeat_failures`, `interrupted`. No `bg_view_creations`, `context_errors` or `network_restarted`. Old cohort SQL needs adapting (below).
|
||||
- Monitor timing: 4 s response grace; suspended time no longer counts (8 s after resume, 30 s extension-load wait, 5 min deadline).
|
||||
|
||||
@@ -377,7 +407,7 @@ Re-cut of the post-release data (macOS 1.213.x, 26–27 Oct, matured since the s
|
||||
- **Hypothesis, not verified:** pages that fail to load are shown on every tab after wake from sleep or a network drop. Each would count as a CPM failure right after an idle unload (fresh `dealloc>view>proxy_on`, background ready in under 1 s, healthy diagnostics), in several tabs (cross-tab) and with no answer until a normal navigation (permanent-looking, and M4 would not help). A same-tab second timeout also triggers M4 on a healthy extension (see `was_stuck` note above), which then adds the origin-change blackout. The pixel has no URL, so data cannot separate this; the check has to be an app run (debug build, Wi-Fi off, several tabs, wake) with the monitor log, or filtering `.alternateHtmlLoad` / `duck://error` and comparing wake rates.
|
||||
|
||||
### Notes
|
||||
- The "P3" lines above disagreed. Checked on 29 Oct: `CPMExtensionBaseURLPolicy` is absent from the 1.213.0 and 7.243.0 release tags and present in `release/macos/1.214.0` and `release/ios/7.244.0`. So P3 is not in 1.213 / 7.243 ("What shipped" is right) and ships with 1.214.0 / 7.244.0, which also makes the 26.6+ gate in #7752 / #7753 relevant to those releases.
|
||||
- **Release status corrected 30 Oct:** P3 is absent from the measured public 1.213.0 / 7.243.0 tags. #7752 / #7753 are merged into the upcoming 1.214.0 / 7.244.0 release branches; public release and remote enablement remain separate checks (see the release update above).
|
||||
- Scratch SQL runner and queries are in the session scratchpad; the harness test is not committed or pushed.
|
||||
|
||||
### Addendum 28 Oct — error pages produce the wake signature (replaces the first version of this addendum)
|
||||
@@ -389,10 +419,10 @@ Re-cut of the post-release data (macOS 1.213.x, 26–27 Oct, matured since the s
|
||||
- **Weak evidence against, kept for the record:** per-OS correlation of error pages per install with wake episodes per install is r = 0.02 (n = 9); hourly r = 0.88 for both wake and non-wake (rollout trend, uninformative).
|
||||
- **Unverified:** the running app (the scenario with `log stream --level debug --predicate 'subsystem == "WebExtensions" AND eventMessage CONTAINS "CPM Health Monitor"'`: three tabs idle >30 s, load a dead host in each, expect `timeoutElapsed` without `Dashboard response attributed`); iOS error pages; the share of wake episodes that are error pages (the episode pixel has no URL or error code).
|
||||
- **Decisive options:** (a) macOS build where the monitor ignores error documents (`.alternateHtmlLoad` / failed provisional navigations) and M4 never fires for them; compare wake share and permanent rate with the current build; (b) a flag parameter on the episode pixel (needs the privacy review you wanted to avoid).
|
||||
- **Fix, tests and PRs (state on 29 Oct):**
|
||||
- **PRs, same commit:** [#7752](https://github.com/duckduckgo/apple-browsers/pull/7752) into `release/macos/1.214.0` (branch `alex/cpm-error-page-filter`) and [#7753](https://github.com/duckduckgo/apple-browsers/pull/7753) into `release/ios/7.244.0` (branch `alex/cpm-error-page-filter-ios`); reviewer `miasma13`; CI had not started when they were opened. Worktrees `apple-browsers.git/cpm-error-page-filter` and `…-ios`.
|
||||
- **Commit content:** `CPMNavigationKind.errorPage` and a monitor that does not measure commit/finish of such a navigation (so no timeout, no episode, no M4); macOS maps `.alternateHtmlLoad` to `.errorPage` in `PrivacyDashboardTabExtension.cpmMessagingNavigationKind`; `CPMExtensionBaseURLPolicy.isSupportedOS` (`#available(macOS 26.6, iOS 26.6, *)`) keeps WebKit's random origin below 26.6 without rotating the stored origin; two monitor unit tests; macOS integration tests `CPMErrorPageMonitoringTests` (both Integration Tests targets in the pbxproj). Harness tests and the reproduction suite are not part of it (the harness is not in the release branches).
|
||||
- **Run results (macOS release base):** 125 package tests (103 monitor, 22 base URL policy) and the 3 integration tests pass; on the iOS release base the same 125 package tests pass. iOS app and iOS tests were not built (no iOS source changed, no exhaustive `switch` on `CPMNavigationKind` there). The old-OS branch of the base URL gate has no test (hosts run 26.6+).
|
||||
- **Fix, tests and PRs (updated 30 Oct):**
|
||||
- **PRs merged:** #7752 into macOS 1.214.0 and #7753 into iOS 7.244.0; merge commits and final branch heads are recorded in the release update above. The sibling branches contain the same intended WebExtensions/error-page changes on their respective release bases.
|
||||
- **Commit content:** error-page monitor filtering and macOS alternate-HTML mapping; OS 26.6+ and flag gate; stored-origin rotation on a disabled/unsupported context load; JSON key rename to `cpmStableOriginIdentifier` with installation-UUID fallback. No migration or OS-version tracking. Harness/reproduction suites are not included.
|
||||
- **Run results:** key-only revision: all 1,025 package tests passed. Final simplified policy suite: 25 passed, including unsupported-OS coverage; strict lint passed on both branches. Three macOS error-page app integration tests passed before the key/test-only edits, with a negative control; no iOS runtime run. See release update for the exact scope.
|
||||
- **Integration tests** (real `Tab`, `macOS Integration Tests`, run locally with the signed Debug build): ordinary load error (`-1003`), SSL (`NSURLErrorServerCertificateUntrusted`), malicious-site warning (`MaliciousSiteError` via the `duck://error` redirect) all pass. Negative control with the macOS mapping removed: the test fails, the monitor receives `["started:other", "failed", "started:other", "committed:https://testhost.com/", "finished:https://testhost.com/"]`, the error page as an ordinary navigation of the failed URL (the false positive in the app's own `Tab`/`Navigation` path); mapping restored. They use their own `StubWebExtensionManager`; `WebExtensionsTestSupport/MockWebExtensionManaging` exists but that product is not linked to the Integration Tests targets.
|
||||
- **Running locally:** do not build with `CODE_SIGNING_ALLOWED=NO`: the host app then lacks the keychain entitlement (`-34018`) and macOS asks for keychain access. Use the normal signed Debug build, `xcodebuild … test-without-building -only-testing:"Integration Tests/CPMErrorPageMonitoringTests"`.
|
||||
- **Corner cases checked (29 Oct), no further false-positive source found:**
|
||||
|
||||
Reference in new issue
Block a user