obsidian-vault/work/projects/cpm-permanent-breakage-metric-tech-design-2026-11-10.md

tags, updated, status

tags	updated	status
work cpm tech-design pixel	2026-11-10	draft

TD: Pixel Definition: CPM Permanent Breakage Metric (macOS/iOS)

Internal metrics, keep out of public PRs. Companion: cpm-wake-init-detection-tech-design-2026-11-10 (adds last_stage). Where we are: cpm-project-summary-2026-11-03.

Author: Alex M Reviewer: TBD (previous CPM pixel designs: Michal) Stakeholders: Michal, Konrad, Russell (from the previous Tech Designs; confirm) Project: CPM: root-cause the stuck embedded extension messaging (macOS/iOS) Task: to create ("Permanent breakage metrics") Previous Tech Designs: cpm-readiness-telemetry-tech-design-2026-10-15, cpm-m3-telemetry-privacy-triage Privacy Triage: to request (new pixel names)

Background & Requirements

The goal is permanent CPM failures below 1% of DAU. There is no pixel for it; the number is derived from web_extension_cpm_episode with a long SQL. The episode is wrong in two ways:

• A timer decides. A four-second navigation timeout opens an episode. On iOS 7.244, 6.0% of episodes closed with a late answer on the same page, 42% of those later than 30 s. Slow answers count as failures.
• Episodes are lost. An open episode lives in memory and is dropped when the app exits.

Requirements: a metric without a timer, as a rate against DAU; no page content, URLs or persistent identifiers. End state: the only CPM pixels are the two defined here.

Problem Statement

Report pages that really never got the extension's answer, and tell persistent breakage from a single page.

Recommended Approach

A page is healthy when an answer reaches it at any time while its document lives. A page that never got an answer is judged when its document ends. There are no navigation timers.

Event	Effect
Navigation finished, extension loaded	The document becomes judgeable and its clock starts. If the extension loads later, the clock starts then.
Attributed answer (dashboard refresh)	Healthy. Resets the streak.
Document ends: new navigation or tab close	If unanswered and alive at least 30 s: breakage. Shorter silence is not judged (it may be a slow wake).
Web content process terminated	Not judged. The tab's next navigation gets reason=tab_crash.
Extension signals cpmStage = config_unavailable or settings_missing	Breakage at once, no lifetime limit. The signal normally arrives while the page is still loading, so it is remembered and applied when the page finishes.
Answer that cannot be tied to one document	That document is not judged.
App suspended	Nothing is judged while suspended.

Judged documents are never re-judged. Measurement state never expires (the old five-minute drop is gone).

Series: reported pages that started after the latest answered page, any tab, in memory. Pages are ordered by navigation start, not by the time of the verdict, so a page judged late (tab closed long after) does not join a series that an answered newer page has already ended. At 3 the persistent pixel fires once for the series.

Pixels

Pixel	Trigger	Frequency
web_extension_cpm_breakage	A document is judged broken.	Daily + count
web_extension_cpm_breakage_persistent	The streak reaches 3. Once per streak.	Daily + count

_daily is installs per day; _count is events. Metadata follows the existing episode pixel on each platform: appVersion on both; macOS also pixelSource, channel, suffix daily_count; iOS keeps its suffix set. The macOS definition must match what is sent.

web_extension_cpm_breakage parameters:

Parameter	Values
reason	session_restoration, tab_crash, extension_reload, other (as in the episode pixel).
background	woken (a background view was created during this navigation), warm (views were seen, none created for it), unknown (no background view seen yet). From the manager's didCreateBackgroundWebView callback.

web_extension_cpm_breakage_persistent has no parameters. last_stage is added by the companion design.

Dashboard: persistent installs / DAU from breakage_persistent_daily (the success metric); any-breakage installs / DAU from breakage_daily; breakages per day from breakage_count by reason and background.

Removed in the same PR

The episode pixel web_extension_cpm_episode with its timers and episode state, web_extension_cpm_background_load_restart (and its observation), and web_extension_cpm_network_process_recovery_applied: definitions, events and firing code. The recovery behavior (error-6 restart, network recovery) stays; only the pixels go.

First attempt (e6419bb910): defects fixed here

Measurements older than five minutes dropped; no minimum lifetime; tab process crash counted; background from load duration and one app-wide signal; macOS definition mismatch; navigationWaitContexts leak (the wait-context set is deleted); no cpmStage test.

PR stack

PR	Content	State
1, #8025	Remove ineffective recovery fixes and flags	pushed, in review
2, alex/cpm-pixel-cleanup	Remove diagnostics, trim the episode pixel	local, tests pass, not pushed
3, alex/cpm-breakage-metric (b8a4361c26)	This design: new pixels and removal of the old ones, one commit	local; package tests 1013 pass; both pixel validators pass; macOS app and tests build; iOS build running; not pushed
4, alex/cpm-init-marks (7bb259e9cf)	Marks and last_stage (cpm-wake-init-detection-tech-design-2026-11-10)	local; package tests 1023 pass; validators pass; not pushed. Extension: alex/cpm-init-progress has the new commit 908f9a62, not pushed

Known limits

• A page that never reaches navigationFinished, or a document alive when the app is killed, is not judged.
• If a new navigation fails after navigationStarted, the old document was already judged and reset; its late answer is not attributed.
• A mark or answer for a reloaded page with the same URL cannot be told from the old document's.
• The dashboard refresh is sent independently of initResp, so a page whose initResp was never delivered counts as answered unless the extension reports response_failed.
• page_init_sent travels through the same background, so last_stage=none cannot tell a lost mark from an init that never started.

Cons

• Documents alive when the app exits or suspends are never judged; breakage on pages left open is missed.
• 30 s hides breakage on short pages.
• The series is new: no overlap with the episode series, so comparison is by app version against the previous derived number.
• Streak resets on relaunch.

Risks

• A page that works but whose answer is not attributed counts as breakage (other_tab_response was 37–53% of outcomes). Documents with an unattributable answer are not judged.
• Pixel volume for a persistently broken install: one count request per page.
• The Debug "Simulate CPM Messaging Breakage" switch must keep hiding answers.

Testing

• Unit tests per row of the table, streak at 3 once and reset, reason values, background values, simulation switch, cpmStage parsing, extension reload.
• Pixel definitions pass both validators; parameters match what is sent.
• Manual: CPM Breakage Testing Steps with the Debug switch.
• Real WebKit (CPM repro harness, PR #7378): not run.

Additional Considerations (if applicable)

Privacy: two new pixel names with enumerated parameters; no URLs, identifiers or page content. Privacy Triage required, not requested yet.

Security: no new private WebKit selectors.

Site Breakage: none; the detector only observes events and recovery is unchanged.

Experimentation: none.

Operational: pixel expiry 2027-01-15 as existing definitions. Danger label for the removed pixels. Data checks: metrics.pixels_aggregation and metrics._pixels_new (pixel_id = 'web').

Localization / Internationalization: none.