[2026-10-28] eagle: work/projects/cpm-post-release-analysis-2026-10-27.md

This commit is contained in:
Alexey Martemyanov committed 2026-10-28 22:11:00 +06:00
1 parent df3f05e53d
commit a860ac9805
1 file changed
+9 -5
@@ -363,8 +363,12 @@ Re-cut of the post-release data (macOS 1.213.x, 26–27 Oct, matured since the s
- The "P3" lines above disagree: priorities say the baseURL fix is released; "What shipped" says P3 (#7493) is on main only and not in 1.213. Re-check before relying on either.
- Scratch SQL runner and queries are in the session scratchpad; the harness test is not committed or pushed.
### Addendum 28 Oct — error-page pixel and the sleep hypothesis
- There is an error-page pixel: `m.mac.errorpageshown.other` (+ `.daily`, `pixel_id='m'`, params `d`/`e` = error domain/code, `pixelSource`), fired in `Tab.swift` when the error page is shown, plus `m.mac.errorpageshown.webkittermination`. macOS 1.213, 26–27 Oct: 7,511 shows (2,284 install-days; ≈0.4 per loaded install-day). Codes: DNS `-1003` 29%, TLS `-1200/-1202…` 26%, connection lost `-1005` 10%, `-1004` and `-1001` a few % each; **offline `-1009` only 1.6% (122)**. So error pages are mostly dead hosts and certificate problems, not "no network after wake".
- Correlation with the wake group does not show up. Hourly counts correlate 0.88 with wake episodes and 0.88 with non-wake ones (both follow the rollout, so it says nothing). Per OS version (9 points) error pages per loaded install vs wake episodes per loaded install: r = 0.02 (vs 0.32 with non-wake). The pixel has no install id, so no per-install join.
- Reasoning against the wake link: the error document never sends `init`, so it does not wake the background. A failure counted on an error page would carry the background state as it was (after >30 s idle: no view; otherwise an old view), not a view created in the last 30 s. If error pages matter, they would sit in the "no view" and "other" cohorts rather than in wake. Not tested.
- The sleep/network-drop hypothesis above is therefore weak. The harness result (error pages are measured and counted as failures) stands as a monitor false-positive source; its size is bounded by the error-page rate (≤0.4 per install-day, most of it unrelated to wake). Filtering `.alternateHtmlLoad` in the monitor would remove it.
### Addendum 28 Oct — error pages produce the wake signature (replaces the first version of this addendum)
- **Pixel:** `m.mac.errorpageshown.other` (+ `.daily`, `pixel_id='m'`, params `d`/`e` = error domain/code, `pixelSource`; also `…webkittermination`), fired in `Tab.swift` when the error page is shown. macOS 1.213, 26–27 Oct: 7,511 shows, 2,284 install-days (≈3.3 per affected install-day). Codes: DNS `-1003` 29%, TLS `-1200/-1202…` 26%, `-1005` 10%, offline `-1009` only 1.6%. **Any** error page counts, not only "no network"; the sleep / offline story from the first version is not needed.
- **The wake happens at the navigation, the error comes later.** The navigation gate asks `initialLoadWaiter`, which calls `manager.prepareEmbeddedBackgroundForNavigation()` → `context.loadBackgroundContent` at policy time of every main-frame http(s) navigation (`WebExtensionLifecycleCoordinator.swift` ~80–105, `WebExtensionManager.swift` ~993; macOS caller `PrivacyDashboardTabExtension.decidePolicy`). So a navigation to a dead host after an idle unload recreates the background first and fails afterwards.
- **Reproduced in the harness** (`CPMReproErrorPageTests.testErrorPageAfterIdleUnloadDoesNotWakeOrFail`, production 30 s unload, real navigation to an unresolvable host, then `loadAlternateHTML(… forUnreachableURL:)` as the app does; worktree `cpm-remaining-failures-repro`, uncommitted): the background view was created ~2 ms after the policy decision, the request failed 0.15 s later with the view present, the error document sent no `init`, the monitor counted the 4 s timeout and closed `other / failed_tab_response / tabs=1` on the next healthy navigation. That is the wake signature: `dealloc>view>proxy_on` with a view younger than 30 s, `bg_ready_after` under 1 s, healthy diagnostics, `reason=other`, one tab. The snapshot is taken ≥4 s after the error page finishes, so `bg_view_age` lands in 4s_to_8s (66% of wake) or 8s_to_30s (33%), as in the data. (A first test without the real navigation did not wake the background.) The harness feeds the monitor `webView.url`; in the app, `PageSignalsTabExtension` skips `.alternateHtmlLoad` in `navigationDidFinish`, which shows that handler also runs for error pages, and the alternate-load action URL is `webView.url` (`NavigationAction.swift:183`).
- **Magnitude, install-days (separate pixels, not joined by install):** error-page `.daily` on CPM-capable macOS (15.4+): 1.213 **10.3%** of `loaded.daily` (1,891 / 18,299); 1.211.1 **10.2%** (10,507 / 102,806, 19 Oct). Wake-signature install-days: 1.213 **13.3%** (2,443 `episode.daily` rows), 1.211.1 **14.6%** (15,058 first-failure `.daily` rows; 5.5% for confirmed `stuck`). Error pages per affected install-day ≈3.3, wake episodes per wake install-day ≈3.2. This is a ceiling, not a proof: only an error page after an idle unload carries the wake tag (a warm background gives `other`), and some error pages may not take the alternate-load path.
- **Why it looks permanent, cross-tab and immune to M4:** a second timeout in a tab runs M4, which reloads a healthy extension (new origin, 1.5–32 s blackout). Error pages in two or more tabs with no healthy navigation within 5 minutes end as `no_response_before_deadline`, `was_stuck=true`, `reload_state=loaded/response_timed_out`, with healthy diagnostics. That matches the cross-tab permanent wake group without any CPM breakage. Not checked: how many permanent cross-tab episodes are such loops.
- **Weak evidence against, kept for the record:** per-OS correlation of error pages per install with wake episodes per install is r = 0.02 (n = 9); hourly r = 0.88 for both wake and non-wake (rollout trend, uninformative).
- **Unverified:** the running app (the scenario with `log stream --level debug --predicate 'subsystem == "WebExtensions" AND eventMessage CONTAINS "CPM Health Monitor"'`: three tabs idle >30 s, load a dead host in each, expect `timeoutElapsed` without `Dashboard response attributed`); iOS error pages; the share of wake episodes that are error pages (the episode pixel has no URL or error code).
- **Decisive options:** (a) macOS build where the monitor ignores error documents (`.alternateHtmlLoad` / failed provisional navigations) and M4 never fires for them; compare wake share and permanent rate with the current build; (b) a flag parameter on the episode pixel (needs the privacy review you wanted to avoid).