obsidian-vault/work/projects/cpm-post-release-analysis-2026-10-27.md

---
title: 'CPM — first post-release data (iOS 7.243.0 / macOS 1.213.0)'
date: '2026-10-27'
tags:
  - work
  - cpm
  - analysis
---
# CPM: first post-release data, 26 October 2026

Internal metrics; keep these figures out of public PRs. Previous analysis: [[cpm-metrics-causes-and-mitigations-2026-10-16]], [[cpm-status-2026-10-15]] (§3–5, 12–14 October cut).

## What shipped

- iOS **7.243.0** (tag 26 Oct 19:23 UTC), macOS **1.213.0** (tag 26 Oct 16:23 UTC).
- Both: episode telemetry #7376/#7484, episode tab scope #7520/#7521, P1/P2/P4 recovery #7379/#7536, Network-process recovery #7459/#7531.
- macOS only: P5 shared-pinned window close #7473, remote/debug disable #7498.
- **Not shipped:** P3 stable origin #7493 (on main only). **No P6 fix exists in either release**: no PR implements the controller rebind.
- Shipped episode contract is the rolled-back variant: `was_stuck` (bool), single-phase `bg_events` (no `;`), plus `tabs_affected`, `repeat_failures`, `interrupted`. No `bg_view_creations`, `context_errors` or `network_restarted`. Old cohort SQL needs adapting (below).
- Monitor timing: 4 s response grace; suspended time no longer counts (8 s after resume, 30 s extension-load wait, 5 min deadline).

## How to query

- New pixels have **no `m_`/`debug_` prefix**: ClickHouse `pixel_id = 'web'`, `pixel LIKE 'web.extension.cpm.%'` (iOS adds `.ios.phone|tablet`). Querying `pixel_id='m'` without an agent filter and a one-day window exceeds the 7B row limit.
- Denominator: `pixel_id='m'`, `m.web.extension.loaded.daily%` / `m.mac.web.extension.loaded.daily`, split by `appVersion`.
- Old-contract attribution was **remotely switched off from 21 Oct** (old versions keep sending bare `debug.web.extension.cpm.*` without parameters). The last fully attributed old-version window is **18–20 Oct, iOS 7.241 / macOS 1.211.x**.
- Data lag: on 27 Oct the table ended at ~01:10 UTC; `date` is US-Eastern. **26 Oct is the only post-release day and it is partial for new adopters** (exposure starts at update time).
- Runner: `~/Developer/grafana-data-tools` reader; ad-hoc SQL and an `ARRAY JOIN extractURLParameters(request)` parameter dump were used. `extractURLParameter` needs a constant key.

## Rates per extension-loaded install-day (26 Oct)

| | iOS 7.242 (old) | iOS 7.243 (new) | macOS 1.212 (old) | macOS 1.213 (new) |
|---|---|---|---|---|
| loaded.daily | 4,257,839 | 52,979 | 93,030 | 2,566 |
| old init_failed.daily (sum of reasons) | 21.3% | — | 36.8% | — |
| old stuck.daily | 3.3% | — | 12.5% | — |
| episode.daily | — | **7.05%** (3,735) | — | **30.0%** (771) |
| episode with `reload_state≠none` (hang confirmed, ≈ old stuck), count leg | — | 23% of episodes | — | 26% of episodes |
| `was_stuck=true` (second distinct tab), daily leg | — | 8.1% | — | 11.2% |
| network_process_recovery_applied.daily | — | 432 (0.8%) | — | 232 (**9.0%**) |

Not like-for-like: the episode unit is one report per episode, the old init number sums four reasons, suspended time is now excluded, and day-1 adopters have short exposure. Directionally, **iOS fell ~3× (21% → 7%); macOS barely moved (37% → 30%).**

`was_stuck` only becomes true when a *different* tab fails; M4 reload already runs on a same-tab second timeout. Use `reload_state≠none` as the old "stuck" proxy.

## Cohorts (count leg, adapted classification)

Priority: error 6 (`bg_load_error_after` present or `error_` token) → no view (`bg_view_alive=false`) → tab not connected (`tab_has_ext_scripts=false`) → extension_updated ≤2 min → wake (`bg_view_age` < 30 s and a `dealloc` token) → recent load/base_url_changed ≤2 min → other. There is no Network-replaced cohort any more.

| Cohort | iOS share / of stuck / no-response | macOS share / of stuck / no-response |
|---|---|---|
| error 6 | 7.5% / 24.7% / 39% | 0.6% / 1.7% / 44% |
| no view (P3) | 7.7% / 6.8% / 7% | 2.9% / 0.9% / 25% |
| tab not connected (P6) | — | 5.8% / 1.3% / **70%** |
| extension_updated ≤2 m | 3.9% / 6.2% / 37% | 14.6% / 11.7% / 28% |
| **wake, view <30 s** | 22.8% / 22.0% / 9% | **43.9% / 58.4% / 48%** |
| recent context load | 22.8% / 17.2% / 12% | 11.9% / 6.9% / 25% |
| other | 35.3% / 23.2% / 5% | 20.4% / 19.0% / 27% |

Baseline with last week's exact SQL, 18–20 Oct (init → stuck shares). **iOS 7.241**: error6 6.4→22.0, no view 47.5→11.5, network 16.5→38.2, wake 7.7→9.2, recent load 16.7→6.6, other 5.2→12.5. **macOS 1.211.x**: error6 1.2→2.2, no view 3.6→1.9, tab not connected 5.9→8.7, network 13.2→25.8, wake 33.2→33.3, recent load 19.4→6.3, other 23.6→21.6. This matches the 12–14 Oct cut, so the baseline was stable.

Per-install rough rates (cohort share × failure rate), old init → new episode:

- **iOS:** no view 10% → 0.5%, recent load 3.5% → 1.6%, error 6 1.3% → 0.5%; the network cohort is absorbed. **Wake stays at 1.6% → 1.6%.** Other rises 1.1% → 2.5%; it now absorbs former network cases and has no network field.
- **macOS:** wake ≈12% → ≈13%. Unchanged.

## Findings

1. **P1 (wake) fix had no measurable effect on either platform.** In the wake cohort `bg_ready_after=under_1s` dominates (macOS 93% overall), so the background is *not* slow. The `background_wake` pixel fires only for ≥1 s callbacks (macOS 89/day vs ~670 wake episodes). The remaining failure is message/response loss after a fast idle wake, not readiness. On macOS 48% of wake episodes see no response for 5 min; tabs=3+ → 68%. Typical tail: `…dealloc@15m,view@15m,proxy_on@15m,dealloc@5m,view@30s,proxy_on@30s`, `bg_view_age=4s_to_8s` (view created around the navigation). The pattern appears on every macOS version, 15.6–27.2.
2. **iOS error 6 changed shape.** All attributed error 6 is now `bg_load_error_after=under_5s`; the 30 s timer bucket is gone.
   - **A** (`ready=failed`, app_state active): `dealloc → view → proxy_on → interruption → error_6`, often after `died_client`/`died_crash`/`died_crash_limit`. The background load spans an app suspension and fails at resume.
   - **B** (`ready=pending`): an older error 6, then a new view whose load is still pending 8–30 s later.
   - The P2 retry fires only while foregrounded with a 60 s cooldown (277/day). **Foreground-resume recovery is the missing piece.**
3. **macOS P6 is untreated**: 5.8% of episodes, 70% never answered. iOS now shows `tab_has_ext_scripts=false` in 5.3% of episodes, concentrated on iOS 26.7 (16%) and 27.0 (8%), with 0 on 27.2. Previously iOS was ≈0.
4. **macOS Network replacement is frequent**: the applied pixel is on 9% of 1.213 install-days. With recovery applied it no longer appears as a cohort, but `other` stays at 20%.
5. **After M4 on macOS**: `reload_state=loaded` → 46% no response by deadline; `response_timed_out` → 57%. The reload often does not restore messaging, or no qualifying navigation follows.
6. Recovery pixels only ever report `background_ready`. `response_received` needs late-document reload, which is default OFF.
7. Day-1 effect: macOS `extension_updated ≤2m` is 14.6% (embedded upgrade on update). Re-check after 2–3 full days.

## Next measurements

- Repeat for 27–29 Oct (full days, larger adoption). Track the wake cohort and macOS no-response by OS.
- `bg_events` per-phase budget and `network_replaced_since_view` remain missing (see [[cpm-status-2026-10-15#6. Телеметрия]]).

## Repro targets

| Issue | Evidence | Hypothesis to test | Candidate fix |
|---|---|---|---|
| R1 wake message loss | macOS 44% eps / 58% stuck; iOS 23%; ready <1 s | message to a woken non-persistent SW is acknowledged with no reply: (a) `processes(RuntimeOnMessage)` empty when `_loadServiceWorker` completion runs before listener IPC; (b) listener set loses `RuntimeOnMessage` so `wakeUpBackgroundContentIfNecessaryToFireEvents` resolves `{}` without waking; (c) background unloaded while a reply is pending | keep the background warm while CPM pages are active, or re-send init on empty reply (native/JS handshake) |
| R2 iOS error 6 across suspension | 7.5% eps / 25% stuck, all under 5 s, interruption between view and error | WebKit load timer spans suspension; nothing retries on resume | on foreground resume, retry `loadBackgroundContent` when the last load failed or is pending |
| R3 P6 controller mismatch | macOS 5.8%, 70% unanswered | known main-flag manager replacement | rebind/recreate tab web views to the current controller |

WebKit source pointers (`~/Downloads/WebKit_sparse`): `WebExtensionContext.cpp` `wakeUpBackgroundContentIfNecessaryToFireEvents` (drops events when the listener set lacks the type after the first load), `scheduleBackgroundContentToUnload` (30 s, reset only on dispatch), `WebExtensionContextCocoa.mm` `loadBackgroundWebView` / `performTasksAfterBackgroundContentLoads`, `WebExtensionContextAPIRuntimeCocoa.mm` `runtimeSendMessage` (empty `processes()` → `{}` reply). 324743 (`9bcf5840ef`) holds runtime messages until SW activation; it is not in shipped OS builds.

## Repro log — 27 October 2026

Worktree `apple-browsers.git/cpm-remaining-failures-repro`, branch `alex/cpm-remaining-failures-repro` (from `origin/main` `8023e4cb5c`, which includes the released P1/P2/P4/Network code plus P3). The package real-WebKit harness was imported from `origin/alex/cpm-repro-harness`. It compiles unchanged on current main, and its CI-tier tests pass. Local tier: `CPM_REPRO=1 swift test --filter <Suite>` from `SharedPackages/WebExtensions`. System WebKit is the host macOS; no app windows are opened.

- **R1 idle wake — not reproduced organically.** `CPMReproIdleWakeTests.testRepeatedIdleWakeCyclesDeliverInit`: 3 tabs, 8 idle-unload cycles (WebKit unloads 32.0 s after the last answered navigation), single and double-tab wakes, 0 of 11 lost. Every wake answered in 2–5 ms after didFinish. `testNavigationAtIdleUnloadBoundaryDeliversInit`: offsets −0.6…+0.2 s around the calibrated unload, 0 of 7 lost. A plain idle wake on an in-memory fixture page is not the production mechanism. The missing ingredient is still unknown: real page loads, a long-lived data store with abandoned registrations, Network replacement, or listener-set loss.
- **P3 (#7493) only in internal builds** (macOS 1.214.0-860, iOS 7.244.0-x). On 26 Oct: macOS 1.214 had 20 episodes / 9 daily of ~36 loaded installs, with 0 no-view and 0 error 6; iOS 7.244 had none. That is too small to evaluate. Relevance: released builds get a new random extension origin on every context load (launch and M4 reload), and WebKit migrates data before injecting. This may explain the iOS no-view cohort and the macOS "reload loaded → still no response" share (46%). Re-check when 7.244 reaches users.
- **R2 suspended wake — macOS analogue does not stay broken.** `CPMReproSuspendedWakeTests`: idle unload, then a navigation wakes the background, and the new background process is SIGSTOPped for 45 s while another tab navigates every 10 s. Two navigations timed out (+9 s, +14 s). Then the background was replaced by a new process (same view slot, new PID) and every later navigation answered in 1–14 ms, including 10 after thaw. No error 6 was recorded on the live context. On macOS the shipped recovery restores messaging within ~14 s of the fault. The harness cannot reproduce iOS suspension, which pauses the UI process's own timers. Likely production sequence: the overdue 30 s unload timer fires at resume, closes the loading view, and records error 6 "under 5 s" (pattern A). Pattern B (new view pending 8–30 s) needs a device or simulator test.
- WebKit hypothesis not hit yet: an SW load failure that **keeps** the background view (`loadBackgroundWebView` completion `!success` without unload). Later wakes would only queue, and every dispatch restarts the 30 s unload timer, so the public `loadBackgroundContent`/P2 retry cannot escape. A candidate mitigation is to restart the background web content process when error 6 is recorded while a background view still exists. Not implemented, because the state has not been reproduced.

## Treatment pixels, P3 ceiling and permanent hangs — 27 October 2026

### Treatment pixels received (26 Oct, count leg, iOS 7.243.0 / macOS 1.213.0)

| Fix | Pixel (`pixel_id='web'`) | iOS | macOS |
|---|---|---|---|
| P1 wake | `web_extension_cpm_background_wake` | 2,026 | 89 |
| P2 error-6 retry | `web_extension_cpm_background_load_retry` | 277 | 12 |
| Install / upgrade readiness | `…_install_readiness` / `…_upgrade_readiness` | 3 / 24 | 0 / 1 |
| Network recovery | `web_extension_cpm_network_process_recovery_applied` | 476 | 277 |
| P4 detection (flag-independent) | `web_extension_cpm_startup_response_missing` | 450 (daily 327) | 20 (daily 12) |
| Late page reload | `web_extension_cpm_late_document_reload` | 0 | 0 |

- Every recovery pixel only reports `outcome=background_ready`, meaning the fix was applied and the background came up. `reload_started` / `response_received` exist only for the late page reload, which is default OFF, so it has zero rows on every version including internal builds. **No pixel shows whether the page got its answer after a treatment; the effect has to be read from episode rates.**
- `background_wake` fires only when the wake callback took ≥1 s and a navigation waited, so it is not a count of all wakes.
- **Detection gap for the late reload.** `startup_response_missing` fires with the flag off but requires `startup.isFirstLoad`: the first background load of that context in the session (launch, install, upgrade). `attemptRecovery` (the reload) has no `isFirstLoad` condition and would also run after later wakes. With the flag off, **wake-time cases the reload would treat are not counted anywhere**; 327 / 12 is a lower bound of what it could fix. Both detection and reload only arm when the page commits while the startup is still pending. In the wake cohort the background is ready in under 1 s, so many wake failures are outside both. Counting the would-be-reload wake cases needs a pixel change: drop `isFirstLoad` from detection or add a parameter. Not done.
- Upper bound on the late reload: ~0.6% of iOS / 0.5% of macOS extension-loaded installs; ~9% of iOS / ~2% of macOS installs with a failure episode.

### P3 (baseURL) ceiling on the new data

- The earlier "≈half" was the pre-release no-view share: **48% of iOS initial failures**, but only 11% of stuck, mostly self-healing.
- On 7.243.0 without P3, no-view is already **7.7% of iOS episodes / 6.8% of stuck** (≈10% → 0.5% of loaded installs); macOS 2.9% / 0.9%. The drop comes from the new measurement (suspended time excluded, one report per episode) and the shipped readiness gates.
- Direct P3 ceiling ≈ **8% of iOS episodes**, less on macOS. A possible indirect effect: released builds get a new random origin on every M4 reload, and 46% of macOS reload-loaded episodes still got no response. That link is unproven; internal 1.214 / 7.244 data is too thin.

### Users affected by permanent hangs (26 Oct)

Denominators were refreshed after more 26 Oct data arrived: extension-loaded installs **iOS 69,088 / macOS 3,045**. "Permanent" = `outcome=no_response_before_deadline` (no qualifying response for 5 min). Single = `tabs_affected=1`, cross = `2`/`3+`. Confirmed = `repeat_failures≠0` or `reload_state≠none`. Daily leg = affected installs.

| Installs/day | iOS | macOS |
|---|---|---|
| Any failure episode | 4,655 (6.7%) | 876 (28.8%) |
| Single-tab, no response | 354 (0.51%) | 217 (7.1%) |
| ↳ confirmed | 76 (0.11%) | 25 (0.82%) |
| Cross-tab, no response | 129 (0.19%) | 58 (1.9%) |
| ↳ confirmed | 123 (0.18%) | 49 (1.6%) |

- **Cross-tab is the reliable permanent-hang figure:** ~0.2% of iOS and 1.6–1.9% of macOS users per day; 95% / 84% confirmed.
- Unconfirmed single-tab rows (iOS 278, macOS 192) may be stuck tabs or users who didn't navigate within 5 min.
- **Strict permanent (confirmed single + cross): ~0.3% iOS, ~2.4% macOS users/day.** Upper bound including unconfirmed single-tab: 0.7% / 9%.
- Lower bounds for users: the daily leg describes only each install's *first* completed episode of the day, and episodes lost when the process ends before the deadline are not reported. Count leg (episodes per install-day), confirmed cross-tab: iOS 0.41%, macOS 5.2%; confirmed single-tab: iOS 0.26%, macOS 2.3%.
- One partial day; macOS has ~3k installs on 1.213.0. Repeat on 27–29 Oct full days.

### Repro statistics note

18 wake navigations cannot rule out a rare loss. At p=8% a clean run happens 22% of the time. The per-wake production rate is likely ≪1% (many wakes per install-day), which needs ~1,000 wakes for 95% power. `CPMReproIdleWakeTests` now has `CPM_REPRO_FAST_UNLOAD=1`: it calls `+[WKWebView _setApplicationBundleIdentifier:]` with `com.apple.WebKit.TestWebKitAPI`, so WebKit unloads an idle background after 3 s instead of 30 s (~5 s per cycle). It can change other test-runner-only WebKit behavior, so confirm any hit with the 30 s delay. A 1,000-cycle run (~1,500 wake navigations) was started on 27 Oct.