git_admin/obsidian-vault
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[2026-10-15] eagle: work/projects/cpm-readiness-telemetry-tech-design-2026-10-15.md

Browse Source
This commit is contained in:
Alexey Martemyanov committed 2026-10-15 14:28:11 +06:00
1 parent feb725f04a
commit fb5215cabc
1 file changed
+123
Download Patch File Download Diff File Expand all files Collapse all files
work/projects/cpm-readiness-telemetry-tech-design-2026-10-15.md
+123
View File
@@ -0,0 +1,123 @@
# TD: Pixel Definition: CPM Readiness Telemetry (macOS/iOS)
**Author:** Alex M
**Reviewer:** TBD (previous reviewer: Michal)
**Stakeholders:** Michal, Konrad, Russell (from the previous Tech Designs; confirm for this proposal)
**Project:** [CPM: root-cause the stuck embedded extension messaging (macOS/iOS)](https://app.asana.com/1/137249556945/task/1216761517055116)
**Task:** [Improve telemetry](https://app.asana.com/1/137249556945/project/1202406491309510/task/1219531810004069)
**Previous Tech Designs:** [CPM breakage pixels](https://app.asana.com/1/137249556945/task/1218005985662983) · [CPM Failure Diagnostics](https://app.asana.com/1/137249556945/task/1218619301158188) ([[cpm-m3-telemetry-privacy-triage]])
**Privacy Triage:** not requested yet; see Privacy below.
**Status:** [PR #7376](https://github.com/duckduckgo/apple-browsers/pull/7376) open; CI running at the time of writing. Not merged, not shipped.
**Analysis this builds on:** [[cpm-web-extension-public-attribution-2026-10-05]], [[cpm-web-extension-breakage-session-summary-2026-10-07]], [[cpm-web-extension-code-guide-2026-10-09]] (section 3).
## Background & Requirements
CPM Failure Diagnostics added process, extension and tab state to the existing failure pixels. The first release with that data (iOS 7.240.0, macOS 1.210.0) showed what it cannot tell apart:
- Most failure snapshots look healthy: context loaded, background view alive, process responsive. The largest group (about 56% of first failures on iOS, 66% on macOS) has no visible error. A live view does not show whether the background script has run.
- A background that is still starting (a wake that takes seconds) looks the same as one that is gone.
- Background load error 6 (`BackgroundContentFailedToLoad`) is recorded, but not when it happened relative to the view, nor in which app state.
- Recovery pixels say that CPM answered again, not whether the **late answer to the failed navigation** arrived, a later navigation answered, or the answer could not be attributed.
Add facts that separate slow starts from lost messages, to the existing failure and recovery pixels.
## Problem Statement
Tell apart, per failure and per recovery:
1. A background still loading when the failure was recorded, from one that finished loading or failed to load.
2. A failure during a background wake (the view is seconds old) from one long after it.
3. A background load error cut off by WebKit's 30 s unload timer from one that failed quickly, and the app state it happened in.
4. A late response from a lost one: did the failed navigation answer after its grace period, did a later navigation, or neither.
5. A failure with many loading tabs (a busy session restoration) from a quiet one.
Collect no page content, URLs or persistent identifiers.
This proposal covers diagnostic collection only. The failure detector, recovery behavior and reload logic do not change. No experiment.
## Recommended Approach
Keep the existing detection. Extend the existing recorder and pixels with new optional parameters. No new pixel names.
**Background readiness.** When WebKit creates the background view, an observer records that the load is `pending`. On the next run loop turn it calls `WKWebExtensionContext.loadBackgroundContent`. The view already exists, so the call joins the load in progress and does not wake a background. Its completion fires once the background script has run, and records `loaded(seconds since view creation)` or `failed`. A response for a replaced view is ignored. `readiness(of:)` answers only for the current view. The observer is attached by the same switch as the delegate proxy (`cpmBackgroundDelegateProxy`).
**Error 6 timing.** When error 6 is recorded, the recorder also records the app state and the seconds since the failing view was created. Values near 30 s point at WebKit's unload timer.
**Snapshot additions.** When a failure pixel fires, the recorder adds app state, background view age, background readiness and tab counts, read synchronously on the main actor. The app supplies the app state (`UIApplication.applicationState` on iOS; `NSApp.isActive` mapped to `active`/`inactive` on macOS). Facts that cannot be read are omitted.
**Recovery details.** The health monitor already tracks an episode. On recovery it now says which navigation answered, and the time since the episode's first failure, bucketed. Detector behavior is unchanged.
### Pixels
No new pixels. Parameters are added to existing ones.
| Pixel | Change |
|---|---|
| `debug_web_extension_cpm_initialization_failed_after_<reason>` (4 reasons × daily) | Failure parameters below. |
| `debug_web_extension_cpm_messaging_stuck_<reason>` (4 reasons × daily + count) | Failure parameters below. |
| `debug_web_extension_cpm_messaging_recovered_without_extension_reload` | Recovery parameters below. |
| `debug_web_extension_cpm_messaging_recovered_after_extension_reload` | Recovery parameters below. |
`<reason>`: `session_restoration`, `tab_crash`, `extension_reload`, `other`. Eight failure pixels and two recovery pixels per platform. Names, triggers and frequencies are unchanged. iOS defines them in `debug_pixels.json5`, macOS in `web_extensions.json5`; both add the parameters to `params_dictionary.json5`.
### Failure / stuck parameters
Unavailable optional facts are omitted. PIDs, URLs and tab identifiers are never sent.
| Parameter | Values |
|---|---|
| `app_state` | `active`, `inactive`, `background`. macOS reports only `active` / `inactive`. Omitted if the app does not report it. |
| `bg_view_age` | Time since the current background view was created: `under_4s`, `4s_to_8s`, `8s_to_30s`, `30s_to_2m`, `over_2m`. Omitted if there is no background view. |
| `bg_ready_after` | `none` (no background view), `pending` (still loading), `failed`, or time from view creation to load completion: `under_1s`, `1s_to_4s`, `4s_to_10s`, `10s_to_30s`, `over_30s`. Omitted if readiness is not observed (observer flag off). |
| `open_tabs` | Tabs in `WKWebExtensionContext.openTabs`: `0`, `1`, `2_to_5`, `6_to_10`, `11_to_20`, `21_to_50`, `over_50`. |
| `loading_tabs` | How many of those tabs were loading: `0`, `1`, `2`, `3_to_5`, `over_5`. |
| `bg_load_error_app_state` | App state when error 6 was last recorded for this context: `none` (no error 6), `active`, `inactive`, `background`. Omitted if unknown. |
| `bg_load_error_after` | Time from view creation to that error 6: `none`, `under_5s`, `5s_to_25s`, `25s_to_35s`, `over_35s`. The `25s_to_35s` bucket surrounds WebKit's 30 s timer. Omitted if unknown. |
`bg_view_age` and `bg_ready_after` use buckets around the monitor's 4 s grace period and WebKit's 30 s timer. They are boundaries chosen for the questions above, not measured thresholds.
### Recovery parameters
| Parameter | Values |
|---|---|
| `recovery_document` | `failed_navigation` (the navigation that failed answered after its grace period: late, not lost), `later_navigation`, `unattributed` (a response matched an open document but no single tab). |
| `recovery_after` | Time from the episode's first failure to recovery: `under_4s`, `4s_to_10s`, `10s_to_30s`, `30s_to_2m`, `over_2m`. A failure is reported 4 s after the navigation finished. |
Existing `recovery_from` is unchanged.
## Testing
- Unit tests: bucket boundaries, parameter serialization and omission rules, error 6 timing (including `none` versus unknown), readiness (loaded, failed, replaced view, observation off), app state, open and loading tabs on a real controller with a navigation that does not finish, recovery details and buckets in the health monitor. 115 tests in the affected suites pass locally.
- Pixel validation: both platforms' schema validators run in CI.
- Real WebKit: during development the harness was temporarily copied into this branch. A background start slowed to 5 s produced `bg_ready_after=pending`, `bg_view_age=4s_to_8s`, then `recovery_document=failed_navigation` with `recovery_after=under_4s`. This was a manual check; it is not in the PR. The CPM repro harness ([PR #7378](https://github.com/duckduckgo/apple-browsers/pull/7378)) can reproduce it in CI once it lands.
- Privacy validation: inspect the real payloads and confirm no URLs, identifiers or message content. Not done yet.
## Additional Considerations (if applicable)
**Privacy**
Adds enumerated, bucketed parameters to existing pixels; no page content, URLs, search queries, message payloads, tab/context identifiers or PIDs. App state, tab counts and timings are bucketed. These are new parameters on existing pixels, so Privacy Triage is required before release. I have not requested it.
**Security**
No new private WebKit selectors. `loadBackgroundContent` is a public API.
**Site Breakage**
No intended page-behavior changes. The one behavioral addition is the `loadBackgroundContent` call on a background view that already exists. It should join the load in progress and wake nothing, but the earlier delegate proxy changed the reproduction of this bug once, so check the observer with its flag on and off. A background wake the observer causes would also corrupt `bg_ready_after`.
**Experimentation**
None.
**Operational**
- The observer shares the `cpmBackgroundDelegateProxy` switch; the recorder and its snapshot additions follow `cpmDiagnosticsRecorder`. No new flag. Whether sharing is acceptable needs a decision from the flag owner, because turning the proxy off also turns readiness off and the parameter disappears.
- Update the CPM Grafana board for the new fields.
- Diagnostic removal is still planned for approximately November 2026; no removal date is enforced in code.
- The limits of the previous Tech Design stand: failure-only snapshots cannot measure failure rates across all process deaths or prove that every tab failed.
- `bg_ready_after` tells whether the background script ran. It does not tell whether the `init` message for a given document reached it. That needs per-message timings (content init sent, background init received, `initResp` sent and received, dashboard refresh sent and received), which is the next telemetry candidate and out of scope here.
**Localization / Internationalization**
None.