diff --git a/work/projects/cpm-pr0-episode-telemetry-summary-2026-10-16.md b/work/projects/cpm-pr0-episode-telemetry-summary-2026-10-16.md index 8d15b93f..ba7fbb5f 100755 --- a/work/projects/cpm-pr0-episode-telemetry-summary-2026-10-16.md +++ b/work/projects/cpm-pr0-episode-telemetry-summary-2026-10-16.md @@ -1,5 +1,5 @@ --- -title: CPM PR0 — one terminal report per failure episode +title: CPM PR0 — complete telemetry change date: '2026-10-16' tags: - work @@ -8,26 +8,41 @@ tags: --- # CPM PR0: one terminal report per failure episode -Local changes for [PR0 #7376](https://github.com/duckduckgo/apple-browsers/pull/7376), in `alex/cpm-telemetry-pr0`, uncommitted. +This describes the uncommitted replacement in `alex/cpm-telemetry-pr0`, based on `origin/main` (`874b465856ca`). It is a proposed revision of [PR0 #7376](https://github.com/duckduckgo/apple-browsers/pull/7376), not a description of that PR's currently published diff. -**What and why.** Replace separate `init_failed`, `stuck`, `recovery` and CPM reload-failed events with `web_extension_cpm_episode`. Each report contains the initial failure and its observed outcome, so diagnosis no longer requires joining a sequence of events. +**Change and purpose.** Replace the initialization-failed, stuck, recovered-with/without-reload and CPM post-reload-failure pixels with `web_extension_cpm_episode`. Put the initial failure and observed outcome together, avoiding reconstruction of a failure/recovery chain from separate events. -**When it fires.** The first timeout opens an episode and captures one diagnostic snapshot. Later failures and reloads update local state. A qualifying dashboard response or a five-minute deadline closes the episode and emits one logical report. The deadline works without further navigations. Existing stuck detection and automatic reload behaviour are preserved. +The first timeout opens an existing monitor episode and captures one diagnostic snapshot. Later failures update local stuck/reload state. A qualifying dashboard response or a five-minute deadline emits the terminal report. The new timer closes quiet episodes without waiting for another navigation; closing an episode cancels its timer and prevents duplicate reports. Stuck detection and automatic reload triggers retain their existing rules. -**What it contains.** Initial failure reason, stuck confirmation, latest reload state and observed outcome. Responses distinguish the original measurement, a later measurement on a failing tab, another tab and ambiguous attribution. Response delay uses three buckets: under 30 seconds, 30–under 60 seconds, and one minute to the deadline. +**Five added summary parameters.** -The initial snapshot records fixed facts about the extension context, background view/process, tab and native-handler registration, controller match and extension scripts. +- `reason`: the first timeout's context — `session_restoration`, `tab_crash`, `extension_reload` or `other`. Previously encoded in failure pixel names. +- `was_stuck`: whether a later eligible non-restoration navigation, begun after the first failure, also timed out. +- `outcome`: `failed_measurement_response` for the measurement that timed out; `failed_tab_response` for a later measurement on a failing tab; `other_tab_response` for another tab; `unattributed_response` for an ambiguous current-document match; or `no_response_before_deadline`. +- `reload_state`: `none` (no reload observed), `pending` (replacement started), `failed` (replacement failed), `loaded` (replacement loaded), or `response_timed_out` (its first eligible messaging measurement timed out). Loaded alone does not prove messaging recovery. +- `response_after`: elapsed time since the first failure, bucketed as `under_30s`, `30s_to_1m` or `1m_to_deadline`. Omitted for the no-response outcome. -**Privacy and counting.** Exclude `bg_events`, raw errors, identifiers, diagnostic counts and app/memory-pressure ages. Rare parameter combinations may still be correlatable. +**Eight retained diagnostic parameters.** All are optional booleans, captured at the first failure; unavailable facts are omitted. -Keep `daily+count`: daily estimates installations with a completed episode that day; count measures completed episodes. The first report each day sends both legs with the same payload; subsequent reports send count only. +- `context_loaded` — extension context loaded; `bg_view_alive` — current background view exists. +- `bg_process_alive` — its process is alive; `bg_process_responsive` — WebKit considers it responsive. +- `tab_in_context` — failing tab registered in WebKit; `handler_registered` — autoconsent native handler registered. +- `tab_controller_match` — tab and context controllers match; `tab_has_ext_scripts` — extension scripts remain registered. -**Fix attribution.** Graveyard attribution is removed from the episode and its diagnostic payload. Add fix-specific application/outcome metrics in the relevant fix PRs when needed. Preventive fixes must also count eligible attempts that succeed without a failure episode. Current diagnostic facts do not distinguish all five failure groups. +Standard metadata remains: `appVersion` on both platforms, plus `pixelSource` and `channel` on macOS. -**Limits.** A dashboard response does not prove document-level `initResp` delivery or recovery of every failing tab. A deadline means no qualifying response was observed. Unfinished episodes can be lost when the process exits. +**Parameters removed from transmission.** `bg_events`, `context_errors`, `network_restarted`, `critical_memory_age`, `bg_view_creations`, `bg_view_leaked_count`, `app_version_change`, `app_launch_age`, `background_graveyard_enabled` and `recovery_from`. Episode state and response attribution replace `recovery_from`. Graveyard attribution is also removed from the diagnostic model and monitor. -Retired pixel definitions and unused parameters are removed on both platforms. The manager still sends the separate `debug_web_extension_reload_failed` event. +For `bg_events`, removal means exclusion from this pixel's payload, not deletion of local lifecycle collection or its existing encoder. This reduces transmitted history but loses event order. The retained booleans are not a lossless replacement for lifecycle signatures and cannot distinguish all five failure groups. Rare combinations of the remaining fields may still be correlatable. -**Validation.** All 742 WebExtensions tests passed after removing graveyard attribution, including a regression test proving the monitor neither reads nor reports its treatment state. macOS PixelKitMock verification, both platform definition validators and strict SwiftLint also passed. +**Frequency and platform wiring.** Both adapters use the same base event name, with no added `_macos` suffix. Keep `daily+count`: the first completed episode of the day sends both legs with identical parameters; subsequent episodes send count only. Daily deduplication is by event name, not reason/outcome, so daily estimates overall affected installations rather than unique installations for every outcome. -Related: [[cpm-status-2026-10-15|CPM status]], [[cpm-readiness-telemetry-tech-design-2026-10-15|Readiness telemetry design]]. +**Definition cleanup.** Remove 11 retired CPM pixel definitions and 10 unused parameter dictionary entries per platform. The separate `debug_web_extension_reload_failed` remains emitted by the extension manager. + +**Scope and limits.** This local revision does not include the original #7376 additions: `bg_ready_after`, `bg_view_age`, `bg_load_error_after`, `bg_load_error_app_state`, `app_state`, `open_tabs`, `loading_tabs`, `recovery_document` or `recovery_after`. Missing diagnostic facts need an explicit follow-up decision; fix-specific application/outcome metrics belong in the relevant fix PRs. Preventive-fix metrics must include eligible attempts that succeed without a failure episode. + +A dashboard response proves neither document-level `initResp` delivery nor recovery of every failing tab. A deadline reports absence of qualifying evidence, not that all tabs died. Unfinished episodes can be lost when the monitor is released or the process exits. + +**Validation.** All 742 WebExtensions package tests passed on macOS. Coverage includes first-snapshot preservation, response attribution, reload generations, deadline-only reporting, duplicate prevention, payload allowlisting and absence of graveyard attribution. macOS PixelKitMock checks, both platform definition validators, wide-event consistency checks and strict SwiftLint passed. + +Related: [[cpm-status-2026-10-15|CPM status]], [[cpm-readiness-telemetry-tech-design-2026-10-15|Original readiness telemetry design]].