Files
obsidian-vault/work/projects/cpm-post-release-analysis-2026-10-27.md
T

10 KiB
Raw Blame History

title, date, tags
title date tags
CPM — first post-release data (iOS 7.243.0 / macOS 1.213.0) 2026-10-27
work
cpm
analysis

CPM: first post-release data, 26 October 2026

Internal metrics; keep these figures out of public PRs. Previous analysis: cpm-metrics-causes-and-mitigations-2026-10-16, cpm-status-2026-10-15 (§3–5, 12–14 October cut).

What shipped

  • iOS 7.243.0 (tag 26 Oct 19:23 UTC), macOS 1.213.0 (tag 26 Oct 16:23 UTC).
  • Both: episode telemetry #7376/#7484, episode tab scope #7520/#7521, P1/P2/P4 recovery #7379/#7536, Network-process recovery #7459/#7531.
  • macOS only: P5 shared-pinned window close #7473, remote/debug disable #7498.
  • Not shipped: P3 stable origin #7493 (on main only). No P6 fix exists in either release: no PR implements the controller rebind.
  • Shipped episode contract is the rolled-back variant: was_stuck (bool), single-phase bg_events (no ;), plus tabs_affected, repeat_failures, interrupted. No bg_view_creations, context_errors or network_restarted. Old cohort SQL needs adapting (below).
  • Monitor timing: 4 s response grace; suspended time no longer counts (8 s after resume, 30 s extension-load wait, 5 min deadline).

How to query

  • New pixels have no m_/debug_ prefix: ClickHouse pixel_id = 'web', pixel LIKE 'web.extension.cpm.%' (iOS adds .ios.phone|tablet). Querying pixel_id='m' without an agent filter and a one-day window exceeds the 7B row limit.
  • Denominator: pixel_id='m', m.web.extension.loaded.daily% / m.mac.web.extension.loaded.daily, split by appVersion.
  • Old-contract attribution was remotely switched off from 21 Oct (old versions keep sending bare debug.web.extension.cpm.* without parameters). The last fully attributed old-version window is 18–20 Oct, iOS 7.241 / macOS 1.211.x.
  • Data lag: on 27 Oct the table ended at ~01:10 UTC; date is US-Eastern. 26 Oct is the only post-release day and it is partial for new adopters (exposure starts at update time).
  • Runner: ~/Developer/grafana-data-tools reader; ad-hoc SQL and an ARRAY JOIN extractURLParameters(request) parameter dump were used. extractURLParameter needs a constant key.

Rates per extension-loaded install-day (26 Oct)

iOS 7.242 (old) iOS 7.243 (new) macOS 1.212 (old) macOS 1.213 (new)
loaded.daily 4,257,839 52,979 93,030 2,566
old init_failed.daily (sum of reasons) 21.3% — 36.8% —
old stuck.daily 3.3% — 12.5% —
episode.daily — 7.05% (3,735) — 30.0% (771)
episode with reload_state≠none (hang confirmed, ≈ old stuck), count leg — 23% of episodes — 26% of episodes
was_stuck=true (second distinct tab), daily leg — 8.1% — 11.2%
network_process_recovery_applied.daily — 432 (0.8%) — 232 (9.0%)

Not like-for-like: the episode unit is one report per episode, the old init number sums four reasons, suspended time is now excluded, and day-1 adopters have short exposure. Directionally, iOS fell ~3× (21% → 7%); macOS barely moved (37% → 30%).

was_stuck only becomes true when a different tab fails; M4 reload already runs on a same-tab second timeout. Use reload_state≠none as the old "stuck" proxy.

Cohorts (count leg, adapted classification)

Priority: error 6 (bg_load_error_after present or error_ token) → no view (bg_view_alive=false) → tab not connected (tab_has_ext_scripts=false) → extension_updated ≤2 min → wake (bg_view_age < 30 s and a dealloc token) → recent load/base_url_changed ≤2 min → other. There is no Network-replaced cohort any more.

Cohort iOS share / of stuck / no-response macOS share / of stuck / no-response
error 6 7.5% / 24.7% / 39% 0.6% / 1.7% / 44%
no view (P3) 7.7% / 6.8% / 7% 2.9% / 0.9% / 25%
tab not connected (P6) — 5.8% / 1.3% / 70%
extension_updated ≤2 m 3.9% / 6.2% / 37% 14.6% / 11.7% / 28%
wake, view <30 s 22.8% / 22.0% / 9% 43.9% / 58.4% / 48%
recent context load 22.8% / 17.2% / 12% 11.9% / 6.9% / 25%
other 35.3% / 23.2% / 5% 20.4% / 19.0% / 27%

Baseline with last week's exact SQL, 18–20 Oct (init → stuck shares). iOS 7.241: error6 6.4→22.0, no view 47.5→11.5, network 16.5→38.2, wake 7.7→9.2, recent load 16.7→6.6, other 5.2→12.5. macOS 1.211.x: error6 1.2→2.2, no view 3.6→1.9, tab not connected 5.9→8.7, network 13.2→25.8, wake 33.2→33.3, recent load 19.4→6.3, other 23.6→21.6. This matches the 12–14 Oct cut, so the baseline was stable.

Per-install rough rates (cohort share × failure rate), old init → new episode:

  • iOS: no view 10% → 0.5%, recent load 3.5% → 1.6%, error 6 1.3% → 0.5%; the network cohort is absorbed. Wake stays at 1.6% → 1.6%. Other rises 1.1% → 2.5%; it now absorbs former network cases and has no network field.
  • macOS: wake ≈12% → ≈13%. Unchanged.

Findings

  1. P1 (wake) fix had no measurable effect on either platform. In the wake cohort bg_ready_after=under_1s dominates (macOS 93% overall), so the background is not slow. The background_wake pixel fires only for ≥1 s callbacks (macOS 89/day vs ~670 wake episodes). The remaining failure is message/response loss after a fast idle wake, not readiness. On macOS 48% of wake episodes see no response for 5 min; tabs=3+ → 68%. Typical tail: …dealloc@15m,view@15m,proxy_on@15m,dealloc@5m,view@30s,proxy_on@30s, bg_view_age=4s_to_8s (view created around the navigation). The pattern appears on every macOS version, 15.6–27.2.
  2. iOS error 6 changed shape. All attributed error 6 is now bg_load_error_after=under_5s; the 30 s timer bucket is gone.
    • A (ready=failed, app_state active): dealloc → view → proxy_on → interruption → error_6, often after died_client/died_crash/died_crash_limit. The background load spans an app suspension and fails at resume.
    • B (ready=pending): an older error 6, then a new view whose load is still pending 8–30 s later.
    • The P2 retry fires only while foregrounded with a 60 s cooldown (277/day). Foreground-resume recovery is the missing piece.
  3. macOS P6 is untreated: 5.8% of episodes, 70% never answered. iOS now shows tab_has_ext_scripts=false in 5.3% of episodes, concentrated on iOS 26.7 (16%) and 27.0 (8%), with 0 on 27.2. Previously iOS was ≈0.
  4. macOS Network replacement is frequent: the applied pixel is on 9% of 1.213 install-days. With recovery applied it no longer appears as a cohort, but other stays at 20%.
  5. After M4 on macOS: reload_state=loaded → 46% no response by deadline; response_timed_out → 57%. The reload often does not restore messaging, or no qualifying navigation follows.
  6. Recovery pixels only ever report background_ready. response_received needs late-document reload, which is default OFF.
  7. Day-1 effect: macOS extension_updated ≤2m is 14.6% (embedded upgrade on update). Re-check after 2–3 full days.

Next measurements

  • Repeat for 27–29 Oct (full days, larger adoption). Track the wake cohort and macOS no-response by OS.
  • bg_events per-phase budget and network_replaced_since_view remain missing (see cpm-status-2026-10-15#6. Телеметрия).

Repro targets

Issue Evidence Hypothesis to test Candidate fix
R1 wake message loss macOS 44% eps / 58% stuck; iOS 23%; ready <1 s message to a woken non-persistent SW is acknowledged with no reply: (a) processes(RuntimeOnMessage) empty when _loadServiceWorker completion runs before listener IPC; (b) listener set loses RuntimeOnMessage so wakeUpBackgroundContentIfNecessaryToFireEvents resolves {} without waking; (c) background unloaded while a reply is pending keep the background warm while CPM pages are active, or re-send init on empty reply (native/JS handshake)
R2 iOS error 6 across suspension 7.5% eps / 25% stuck, all under 5 s, interruption between view and error WebKit load timer spans suspension; nothing retries on resume on foreground resume, retry loadBackgroundContent when the last load failed or is pending
R3 P6 controller mismatch macOS 5.8%, 70% unanswered known main-flag manager replacement rebind/recreate tab web views to the current controller

WebKit source pointers (~/Downloads/WebKit_sparse): WebExtensionContext.cpp wakeUpBackgroundContentIfNecessaryToFireEvents (drops events when the listener set lacks the type after the first load), scheduleBackgroundContentToUnload (30 s, reset only on dispatch), WebExtensionContextCocoa.mm loadBackgroundWebView / performTasksAfterBackgroundContentLoads, WebExtensionContextAPIRuntimeCocoa.mm runtimeSendMessage (empty processes() → {} reply). 324743 (9bcf5840ef) holds runtime messages until SW activation; it is not in shipped OS builds.

Repro log — 27 October 2026

Worktree apple-browsers.git/cpm-remaining-failures-repro, branch alex/cpm-remaining-failures-repro (from origin/main 8023e4cb5c, which includes the released P1/P2/P4/Network code plus P3). The package real-WebKit harness was imported from origin/alex/cpm-repro-harness. It compiles unchanged on current main, and its CI-tier tests pass. Local tier: CPM_REPRO=1 swift test --filter <Suite> from SharedPackages/WebExtensions. System WebKit is the host macOS; no app windows are opened.

  • R1 idle wake — not reproduced organically. CPMReproIdleWakeTests.testRepeatedIdleWakeCyclesDeliverInit: 3 tabs, 8 idle-unload cycles (WebKit unloads 32.0 s after the last answered navigation), single and double-tab wakes, 0 of 11 lost. Every wake answered in 2–5 ms after didFinish. testNavigationAtIdleUnloadBoundaryDeliversInit: offsets −0.6…+0.2 s around the calibrated unload, 0 of 7 lost. A plain idle wake on an in-memory fixture page is not the production mechanism. The missing ingredient is still unknown: real page loads, a long-lived data store with abandoned registrations, Network replacement, or listener-set loss.
  • P3 (#7493) only in internal builds (macOS 1.214.0-860, iOS 7.244.0-x). On 26 Oct: macOS 1.214 had 20 episodes / 9 daily of ~36 loaded installs, with 0 no-view and 0 error 6; iOS 7.244 had none. That is too small to evaluate. Relevance: released builds get a new random extension origin on every context load (launch and M4 reload), and WebKit migrates data before injecting. This may explain the iOS no-view cohort and the macOS "reload loaded → still no response" share (46%). Re-check when 7.244 reaches users.