obsidian-vault/work/projects/cpm-readiness-telemetry-tech-design-2026-10-15.md

TD: Pixel Definition: CPM Readiness Telemetry (macOS/iOS)

Internal documentation update — 2026-10-15, 17:26 UTC. Current evidence, unresolved review decisions and PR stack: cpm-status-2026-10-15. This update does not approve or implement the proposed schema changes.

PR #7376 is OPEN at 619869ee84c6, review required, with no failing/pending checks; #7384/#7385 remain open and each has a failing Danger status. Their skipped checks are not completed tests. JS #3727 is open, checks successful, review required.

Source-linked P5/shared-window loss and conditional P6/main-flag replacement are now reproduced; natural metadata loss, P5/tab=true and actual production flag transitions remain unproven. The main status contains the full 14-test matrix, page-receipt controls, cold Network PID caveat, 494/499 denominator distinction and pending telemetry simplification decision. No new pixel or parameter was added by those tests. Internal ClickHouse figures stay in Obsidian, not public PR comments or issues.

Author: Alex M
Reviewer: TBD (previous reviewer: Michal)
Stakeholders: Michal, Konrad, Russell (from the previous Tech Designs; confirm for this proposal)
Project: CPM: root-cause the stuck embedded extension messaging (macOS/iOS)
Task: Improve telemetry
Previous Tech Designs: CPM breakage pixels · CPM Failure Diagnostics (cpm-m3-telemetry-privacy-triage)
Privacy Triage: not requested yet; see Privacy below.
Status (checked 2026-10-15, 17:26 UTC): PR #7376 OPEN at 619869ee84c6; REVIEW_REQUIRED; no failing/pending published checks (39 success, 18 skipped entries). Not merged; this check does not establish shipping.
Analysis this builds on: cpm-web-extension-public-attribution-2026-10-05, cpm-web-extension-breakage-session-summary-2026-10-07, cpm-web-extension-code-guide-2026-10-09 (section 3).

Background & Requirements

CPM Failure Diagnostics added process, extension and tab state to the existing failure pixels. The first release with that data (iOS 7.240.0, macOS 1.210.0) showed what it cannot tell apart:

• Most failure snapshots look healthy: context loaded, background view alive, process responsive. The largest group (about 56% of first failures on iOS, 66% on macOS) has no visible error. A live view does not show whether the background script has run.
• A background that is still starting (a wake that takes seconds) looks the same as one that is gone.
• Background load error 6 (BackgroundContentFailedToLoad) is recorded, but not when it happened relative to the view, nor in which app state.
• Recovery pixels say that CPM answered again, not whether the late answer to the failed navigation arrived, a later navigation answered, or the answer could not be attributed.

Add facts that separate slow starts from lost messages, to the existing failure and recovery pixels.

Problem Statement

Tell apart, per failure and per recovery:

1. A background still loading when the failure was recorded, from one that finished loading or failed to load.
2. A failure during a background wake (the view is seconds old) from one long after it.
3. A background load error cut off by WebKit's 30 s unload timer from one that failed quickly, and the app state it happened in.
4. A late response from a lost one: did the failed navigation answer after its grace period, did a later navigation, or neither.
5. A failure with many loading tabs (a busy session restoration) from a quiet one.

Collect no page content, URLs or persistent identifiers.

This proposal covers diagnostic collection only. The failure detector, recovery behavior and reload logic do not change. No experiment.

Recommended Approach

Keep the existing detection. Extend the existing recorder and pixels with new optional parameters. No new pixel names.

Background readiness. When WebKit creates the background view, an observer records that the load is pending. On the next run loop turn it calls WKWebExtensionContext.loadBackgroundContent. The view already exists, so the call joins the load in progress and does not wake a background. Its completion fires once the background script has run, and records loaded(seconds since view creation) or failed. A response for a replaced view is ignored. readiness(of:) answers only for the current view. The observer is attached by the same switch as the delegate proxy (cpmBackgroundDelegateProxy).

Error 6 timing. When error 6 is recorded, the recorder also records the app state and the seconds since the failing view was created. Values near 30 s point at WebKit's unload timer.

Snapshot additions. When a failure pixel fires, the recorder adds app state, background view age, background readiness and tab counts, read synchronously on the main actor. The app supplies the app state (UIApplication.applicationState on iOS; NSApp.isActive mapped to active/inactive on macOS). Facts that cannot be read are omitted.

Recovery details. The health monitor already tracks an episode. On recovery it now says which navigation answered, and the time since the episode's first failure, bucketed. Detector behavior is unchanged.

Pixels

No new pixels. Parameters are added to existing ones.

Pixel	Change
debug_web_extension_cpm_initialization_failed_after_<reason> (4 reasons × daily)	Failure parameters below.
debug_web_extension_cpm_messaging_stuck_<reason> (4 reasons × daily + count)	Failure parameters below.
debug_web_extension_cpm_messaging_recovered_without_extension_reload	Recovery parameters below.
debug_web_extension_cpm_messaging_recovered_after_extension_reload	Recovery parameters below.

<reason>: session_restoration, tab_crash, extension_reload, other. Eight failure pixels and two recovery pixels per platform. Names, triggers and frequencies are unchanged. iOS defines them in debug_pixels.json5, macOS in web_extensions.json5; both add the parameters to params_dictionary.json5.

Failure / stuck parameters

Unavailable optional facts are omitted. PIDs, URLs and tab identifiers are never sent.

Parameter	Values
app_state	active, inactive, background. macOS reports only active / inactive. Omitted if the app does not report it.
bg_view_age	Time since the current background view was created: under_4s, 4s_to_8s, 8s_to_30s, 30s_to_2m, over_2m. Omitted if there is no background view.
bg_ready_after	none (no background view), pending (still loading), failed, or time from view creation to load completion: under_1s, 1s_to_4s, 4s_to_10s, 10s_to_30s, over_30s. Omitted if readiness is not observed (observer flag off).
open_tabs	Tabs in WKWebExtensionContext.openTabs: 0, 1, 2_to_5, 6_to_10, 11_to_20, 21_to_50, over_50.
loading_tabs	How many of those tabs were loading: 0, 1, 2, 3_to_5, over_5.
bg_load_error_app_state	App state when error 6 was last recorded for this context: none (no error 6), active, inactive, background. Omitted if unknown.
bg_load_error_after	Time from view creation to that error 6: none, under_5s, 5s_to_25s, 25s_to_35s, over_35s. The 25s_to_35s bucket surrounds WebKit's 30 s timer. Omitted if unknown.

bg_view_age and bg_ready_after use buckets around the monitor's 4 s grace period and WebKit's 30 s timer. They are boundaries chosen for the questions above, not measured thresholds.

Recovery parameters

Parameter	Values
recovery_document	failed_navigation (the navigation that failed answered after its grace period: late, not lost), later_navigation, unattributed (a response matched an open document but no single tab).
recovery_after	Time from the episode's first failure to recovery: under_4s, 4s_to_10s, 10s_to_30s, 30s_to_2m, over_2m. A failure is reported 4 s after the navigation finished.

Existing recovery_from is unchanged.

Testing

• Unit tests: bucket boundaries, parameter serialization and omission rules, error 6 timing (including none versus unknown), readiness (loaded, failed, replaced view, observation off), app state, open and loading tabs on a real controller with a navigation that does not finish, recovery details and buckets in the health monitor. 115 tests in the affected suites pass locally.
• Pixel validation: both platforms' schema validators run in CI.
• Real WebKit: during development the harness was temporarily copied into this branch. A background start slowed to 5 s produced bg_ready_after=pending, bg_view_age=4s_to_8s, then recovery_document=failed_navigation with recovery_after=under_4s. This was a manual check; it is not in the PR. The CPM repro harness (PR #7378) can reproduce it in CI once it lands.
• Privacy validation: inspect the real payloads and confirm no URLs, identifiers or message content. Not done yet.

Additional Considerations (if applicable)

Privacy

Adds enumerated, bucketed parameters to existing pixels; no page content, URLs, search queries, message payloads, tab/context identifiers or PIDs. App state, tab counts and timings are bucketed. These are new parameters on existing pixels, so Privacy Triage is required before release. I have not requested it.

Security

No new private WebKit selectors. loadBackgroundContent is a public API.

Site Breakage

No intended page-behavior changes. The one behavioral addition is the loadBackgroundContent call on a background view that already exists. It should join the load in progress and wake nothing, but the earlier delegate proxy changed the reproduction of this bug once, so check the observer with its flag on and off. A background wake the observer causes would also corrupt bg_ready_after.

Experimentation

None.

Operational

• The observer shares the cpmBackgroundDelegateProxy switch; the recorder and its snapshot additions follow cpmDiagnosticsRecorder. No new flag. Whether sharing is acceptable needs a decision from the flag owner, because turning the proxy off also turns readiness off and the parameter disappears.
• Update the CPM Grafana board for the new fields.
• Diagnostic removal is still planned for approximately November 2026; no removal date is enforced in code.
• The limits of the previous Tech Design stand: failure-only snapshots cannot measure failure rates across all process deaths or prove that every tab failed.
• bg_ready_after tells whether the background script ran. It does not tell whether the init message for a given document reached it. That needs per-message timings (content init sent, background init received, initResp sent and received, dashboard refresh sent and received), which is the next telemetry candidate and out of scope here.

Localization / Internationalization

None.

App-hosted validation, 16 October 2026

A separate local suite now runs inside the signed DuckDuckGo.app, using actual AppDelegate lifecycle/flag callbacks, Tab/MainWindowController/WindowsManager, macOS WebExtension providers/native delegate, production monitor/recorder and system WebKit. Worktree: cpm-e2e-init, macOS/IntegrationTests/CPM/; scheme CPM Integration Tests, without profile-cleaning pre-actions. No commits or PR changes.

The ten core methods passed together (signed-core.xcresult, 0 failures): healthy baseline, slow HTTP document, slow native settings, one failed navigation followed by recovery, persistent single-tab failure, several tabs in one window, all windows including a newly opened tab, automatic M4 recovery, dashboard-transport negative control, and a runtime diagnostic-proxy flag cycle. Healthy checks require page-side initResp as well as a forwarded real native dashboard message. Fault scopes are injected into a temporary extracted copy at the extension messaging boundary; navigation/monitor events are not fabricated.

Observed library metadata: message loss gives init_stage=none; delayed settings gives received, then recovery on failed_navigation (4s_to_10s); a missing dashboard confirmation despite page receipt gives response_sent; later-document recovery is later_navigation. These are metadata captured before the platform pixel adapter/daily deduplication, not proof of an actual production HTTP payload or production causal attribution. JSON artifacts contain only owned fixture page URLs and test observations; pixel parameters remain the existing bucketed metadata.

Additional app/WebKit characterization tests cover real shared-pinned-window closure, main/embedded flag transitions, startup race, process termination, slow idle wake and the real error-6 load timeout. Their final outcomes and the preserved red experiments are tracked in cpm-status-2026-10-15#13. Integration tests в настоящем приложении — реализация и проверки. iOS lifecycle and natural listener-metadata loss remain unproven by the macOS suite. The initial unsigned CLI workaround was unnecessary; the original project signing configuration builds successfully and is used for the reported results. The temporary vault bypass has been removed.

Scope and source-linked characterization follow-up

The known-path methods also execute the real AppDelegate main/embedded flag transitions (including a first main enable), actual shared-pin window closure, the owned test host's WebKit process faults and cache reload. P5's final app method passed (signed-pinned-single-probe.xcresult, 73.806 s): after two healthy/idle cycles, real window closure causes one failure with no page initResp, tab_in_context=false, matching controller and background creation bucket 2; regular and new-tab controls remain healthy. Longer app attempts reached the enabled DEBUG MainWindow lifetime guard during the second timeout and remain documented as red experiments. Persistent P5/stuck proof belongs to the separate SPM suite, not this narrower app method.

The persistent-cache characterization now completes cleanup (signed-large-cache-session-end.xcresult, 47.519 s). A fresh UUID website/controller profile actually receives 20,000 cacheable resources and has disk-cache records; real reload changes the extension baseURL. In that run, immediate post-reload messaging was healthy and p3NoBackgroundFailure=false. Passing this test means the experiment and recovery/cleanup controls succeeded; P3 itself was not reproduced. _terminateNetworkProcess is used only in teardown of the owned test host, after evidence capture/data clearing, to permit removal of that UUID store. It adds no selector to production diagnostics and changes no telemetry schema.

P1 slow idle wake and P2 real WebKit load timeout have signed app passes; the latter records actual error 6 near the 30-second deadline after stopping the owned worker. These are macOS mechanisms, not proof of the causal attribution of iOS production groups. Runtime listener-metadata loss, iOS suspend/resume/device errors and actual production flag changes remain unresolved. The final current-source regression outcome will be recorded in the main status; original CI/PR and ClickHouse snapshots above are dated 15 October and have not been refreshed by these local app tests.