[2026-10-27] eagle: work/projects/cpm-post-release-analysis-2026-10-27.md
This commit is contained in:
Alexey Martemyanov committed
1 parent
27fcc2109a
commit
9da4728978
1 file changed
+93
@@ -0,0 +1,93 @@
|
||||
---
|
||||
title: 'CPM — first post-release data (iOS 7.243.0 / macOS 1.213.0)'
|
||||
date: '2026-10-27'
|
||||
tags:
|
||||
- work
|
||||
- cpm
|
||||
- analysis
|
||||
---
|
||||
# CPM: first post-release data, 26 October 2026
|
||||
|
||||
Internal metrics; keep these figures out of public PRs. Previous analysis: [[cpm-metrics-causes-and-mitigations-2026-10-16]], [[cpm-status-2026-10-15]] (§3–5, 12–14 October cut).
|
||||
|
||||
## What shipped
|
||||
|
||||
- iOS **7.243.0** (tag 26 Oct 19:23 UTC), macOS **1.213.0** (tag 26 Oct 16:23 UTC).
|
||||
- Both: episode telemetry #7376/#7484, episode tab scope #7520/#7521, P1/P2/P4 recovery #7379/#7536, Network-process recovery #7459/#7531.
|
||||
- macOS only: P5 shared-pinned window close #7473, remote/debug disable #7498.
|
||||
- **Not shipped:** P3 stable origin #7493 (on main only). **No P6 fix exists in either release**: no PR implements the controller rebind.
|
||||
- Shipped episode contract is the rolled-back variant: `was_stuck` (bool), single-phase `bg_events` (no `;`), plus `tabs_affected`, `repeat_failures`, `interrupted`. No `bg_view_creations`, `context_errors` or `network_restarted`. Old cohort SQL needs adapting (below).
|
||||
- Monitor timing: 4 s response grace; suspended time no longer counts (8 s after resume, 30 s extension-load wait, 5 min deadline).
|
||||
|
||||
## How to query
|
||||
|
||||
- New pixels have **no `m_`/`debug_` prefix**: ClickHouse `pixel_id = 'web'`, `pixel LIKE 'web.extension.cpm.%'` (iOS adds `.ios.phone|tablet`). Querying `pixel_id='m'` without an agent filter and a one-day window exceeds the 7B row limit.
|
||||
- Denominator: `pixel_id='m'`, `m.web.extension.loaded.daily%` / `m.mac.web.extension.loaded.daily`, split by `appVersion`.
|
||||
- Old-contract attribution was **remotely switched off from 21 Oct** (old versions keep sending bare `debug.web.extension.cpm.*` without parameters). The last fully attributed old-version window is **18–20 Oct, iOS 7.241 / macOS 1.211.x**.
|
||||
- Data lag: on 27 Oct the table ended at ~01:10 UTC; `date` is US-Eastern. **26 Oct is the only post-release day and it is partial for new adopters** (exposure starts at update time).
|
||||
- Runner: `~/Developer/grafana-data-tools` reader; ad-hoc SQL and an `ARRAY JOIN extractURLParameters(request)` parameter dump were used. `extractURLParameter` needs a constant key.
|
||||
|
||||
## Rates per extension-loaded install-day (26 Oct)
|
||||
|
||||
| | iOS 7.242 (old) | iOS 7.243 (new) | macOS 1.212 (old) | macOS 1.213 (new) |
|
||||
|---|---|---|---|---|
|
||||
| loaded.daily | 4,257,839 | 52,979 | 93,030 | 2,566 |
|
||||
| old init_failed.daily (sum of reasons) | 21.3% | — | 36.8% | — |
|
||||
| old stuck.daily | 3.3% | — | 12.5% | — |
|
||||
| episode.daily | — | **7.05%** (3,735) | — | **30.0%** (771) |
|
||||
| episode with `reload_state≠none` (hang confirmed, ≈ old stuck), count leg | — | 23% of episodes | — | 26% of episodes |
|
||||
| `was_stuck=true` (second distinct tab), daily leg | — | 8.1% | — | 11.2% |
|
||||
| network_process_recovery_applied.daily | — | 432 (0.8%) | — | 232 (**9.0%**) |
|
||||
|
||||
Not like-for-like: the episode unit is one report per episode, the old init number sums four reasons, suspended time is now excluded, and day-1 adopters have short exposure. Directionally, **iOS fell ~3× (21% → 7%); macOS barely moved (37% → 30%).**
|
||||
|
||||
`was_stuck` only becomes true when a *different* tab fails; M4 reload already runs on a same-tab second timeout. Use `reload_state≠none` as the old "stuck" proxy.
|
||||
|
||||
## Cohorts (count leg, adapted classification)
|
||||
|
||||
Priority: error 6 (`bg_load_error_after` present or `error_` token) → no view (`bg_view_alive=false`) → tab not connected (`tab_has_ext_scripts=false`) → extension_updated ≤2 min → wake (`bg_view_age` < 30 s and a `dealloc` token) → recent load/base_url_changed ≤2 min → other. There is no Network-replaced cohort any more.
|
||||
|
||||
| Cohort | iOS share / of stuck / no-response | macOS share / of stuck / no-response |
|
||||
|---|---|---|
|
||||
| error 6 | 7.5% / 24.7% / 39% | 0.6% / 1.7% / 44% |
|
||||
| no view (P3) | 7.7% / 6.8% / 7% | 2.9% / 0.9% / 25% |
|
||||
| tab not connected (P6) | — | 5.8% / 1.3% / **70%** |
|
||||
| extension_updated ≤2 m | 3.9% / 6.2% / 37% | 14.6% / 11.7% / 28% |
|
||||
| **wake, view <30 s** | 22.8% / 22.0% / 9% | **43.9% / 58.4% / 48%** |
|
||||
| recent context load | 22.8% / 17.2% / 12% | 11.9% / 6.9% / 25% |
|
||||
| other | 35.3% / 23.2% / 5% | 20.4% / 19.0% / 27% |
|
||||
|
||||
Baseline with last week's exact SQL, 18–20 Oct (init → stuck shares). **iOS 7.241**: error6 6.4→22.0, no view 47.5→11.5, network 16.5→38.2, wake 7.7→9.2, recent load 16.7→6.6, other 5.2→12.5. **macOS 1.211.x**: error6 1.2→2.2, no view 3.6→1.9, tab not connected 5.9→8.7, network 13.2→25.8, wake 33.2→33.3, recent load 19.4→6.3, other 23.6→21.6. This matches the 12–14 Oct cut, so the baseline was stable.
|
||||
|
||||
Per-install rough rates (cohort share × failure rate), old init → new episode:
|
||||
|
||||
- **iOS:** no view 10% → 0.5%, recent load 3.5% → 1.6%, error 6 1.3% → 0.5%; the network cohort is absorbed. **Wake stays at 1.6% → 1.6%.** Other rises 1.1% → 2.5%; it now absorbs former network cases and has no network field.
|
||||
- **macOS:** wake ≈12% → ≈13%. Unchanged.
|
||||
|
||||
## Findings
|
||||
|
||||
1. **P1 (wake) fix had no measurable effect on either platform.** In the wake cohort `bg_ready_after=under_1s` dominates (macOS 93% overall), so the background is *not* slow. The `background_wake` pixel fires only for ≥1 s callbacks (macOS 89/day vs ~670 wake episodes). The remaining failure is message/response loss after a fast idle wake, not readiness. On macOS 48% of wake episodes see no response for 5 min; tabs=3+ → 68%. Typical tail: `…dealloc@15m,view@15m,proxy_on@15m,dealloc@5m,view@30s,proxy_on@30s`, `bg_view_age=4s_to_8s` (view created around the navigation). The pattern appears on every macOS version, 15.6–27.2.
|
||||
2. **iOS error 6 changed shape.** All attributed error 6 is now `bg_load_error_after=under_5s`; the 30 s timer bucket is gone.
|
||||
- **A** (`ready=failed`, app_state active): `dealloc → view → proxy_on → interruption → error_6`, often after `died_client`/`died_crash`/`died_crash_limit`. The background load spans an app suspension and fails at resume.
|
||||
- **B** (`ready=pending`): an older error 6, then a new view whose load is still pending 8–30 s later.
|
||||
- The P2 retry fires only while foregrounded with a 60 s cooldown (277/day). **Foreground-resume recovery is the missing piece.**
|
||||
3. **macOS P6 is untreated**: 5.8% of episodes, 70% never answered. iOS now shows `tab_has_ext_scripts=false` in 5.3% of episodes, concentrated on iOS 26.7 (16%) and 27.0 (8%), with 0 on 27.2. Previously iOS was ≈0.
|
||||
4. **macOS Network replacement is frequent**: the applied pixel is on 9% of 1.213 install-days. With recovery applied it no longer appears as a cohort, but `other` stays at 20%.
|
||||
5. **After M4 on macOS**: `reload_state=loaded` → 46% no response by deadline; `response_timed_out` → 57%. The reload often does not restore messaging, or no qualifying navigation follows.
|
||||
6. Recovery pixels only ever report `background_ready`. `response_received` needs late-document reload, which is default OFF.
|
||||
7. Day-1 effect: macOS `extension_updated ≤2m` is 14.6% (embedded upgrade on update). Re-check after 2–3 full days.
|
||||
|
||||
## Next measurements
|
||||
|
||||
- Repeat for 27–29 Oct (full days, larger adoption). Track the wake cohort and macOS no-response by OS.
|
||||
- `bg_events` per-phase budget and `network_replaced_since_view` remain missing (see [[cpm-status-2026-10-15#6. Телеметрия]]).
|
||||
|
||||
## Repro targets
|
||||
|
||||
| Issue | Evidence | Hypothesis to test | Candidate fix |
|
||||
|---|---|---|---|
|
||||
| R1 wake message loss | macOS 44% eps / 58% stuck; iOS 23%; ready <1 s | message to a woken non-persistent SW is acknowledged with no reply: (a) `processes(RuntimeOnMessage)` empty when `_loadServiceWorker` completion runs before listener IPC; (b) listener set loses `RuntimeOnMessage` so `wakeUpBackgroundContentIfNecessaryToFireEvents` resolves `{}` without waking; (c) background unloaded while a reply is pending | keep the background warm while CPM pages are active, or re-send init on empty reply (native/JS handshake) |
|
||||
| R2 iOS error 6 across suspension | 7.5% eps / 25% stuck, all under 5 s, interruption between view and error | WebKit load timer spans suspension; nothing retries on resume | on foreground resume, retry `loadBackgroundContent` when the last load failed or is pending |
|
||||
| R3 P6 controller mismatch | macOS 5.8%, 70% unanswered | known main-flag manager replacement | rebind/recreate tab web views to the current controller |
|
||||
|
||||
WebKit source pointers (`~/Downloads/WebKit_sparse`): `WebExtensionContext.cpp` `wakeUpBackgroundContentIfNecessaryToFireEvents` (drops events when the listener set lacks the type after the first load), `scheduleBackgroundContentToUnload` (30 s, reset only on dispatch), `WebExtensionContextCocoa.mm` `loadBackgroundWebView` / `performTasksAfterBackgroundContentLoads`, `WebExtensionContextAPIRuntimeCocoa.mm` `runtimeSendMessage` (empty `processes()` → `{}` reply). 324743 (`9bcf5840ef`) holds runtime messages until SW activation; it is not in shipped OS builds.
|
||||
Reference in new issue
Block a user