obsidian-vault/work/projects/cpm-permanent-breakage-metric-tech-design-2026-11-10.md

tags, updated, status

tags	updated	status
work cpm tech-design pixel	2026-11-11	draft

TD: Pixel Definition: CPM Permanent Breakage Metric (macOS/iOS)

Internal metrics, keep out of public PRs. Companion: cpm-wake-init-detection-tech-design-2026-11-10 (adds last_stage). Where we are: cpm-project-summary-2026-11-03. Reworked on 11 Nov from "judge a page when its document ends" to "a tab is silent"; the first design is gone.

Author: Alex M Reviewer: TBD (previous CPM pixel designs: Michal) Stakeholders: Michal, Konrad, Russell (from the previous Tech Designs; confirm) Project: CPM: root-cause the stuck embedded extension messaging (macOS/iOS) Task: to create ("Permanent breakage metrics") Previous Tech Designs: cpm-readiness-telemetry-tech-design-2026-10-15, cpm-m3-telemetry-privacy-triage Privacy Triage: to request (new pixel names)

Background & Requirements

The goal is permanent CPM failures below 1% of DAU. There is no pixel for it; the number is derived from web_extension_cpm_episode with a long SQL. The episode is wrong in two ways:

• A timer decides. A four-second navigation timeout opens an episode. On iOS 7.244, 6.0% of episodes closed with a late answer on the same page, 42% of those later than 30 s. Slow answers count as failures.
• Episodes are lost. An open episode lives in memory and is dropped when the app exits.

Requirements: a metric without a navigation timer, as a rate against DAU; no page content, URLs or persistent identifiers; a minimal layer next to the product code (no private selectors, no per-navigation state machine). End state: the only CPM pixels are the two defined here.

Problem Statement

Report tabs whose pages never hear from the extension, and tell one broken page from a broken extension.

Recommended Approach

The extension sends a message from a tab while a page initializes. A tab that commits an eligible page and gets no message for 30 s of app time is silent. A message that arrives later still counts as an answer.

The monitor keeps, per tab: the time of the latest eligible commit (until a message answers it) and the time of the latest message. Nothing runs on a timer: the check happens when any event arrives (a commit, a tab closing, a message, the app waking).

Event	Effect
Commit of an http(s) page, extension loaded, not an error page, not a history restore, not a restored session	The tab waits for a message. A message up to 2 s before the commit answers it (the extension can message a tab before the app reports the commit), also for a tab whose extension tab is not learned yet.
Message from the current extension (dashboard refresh)	Answers the tab's page, even when late. Ends the series of silent tabs.
config_unavailable or settings_missing refresh	Not an answer: the extension said it cannot answer the page's init. It still shows the extension is alive (restart budget reset).
Any event, with a commit older than 30 s and no message	The tab is silent: reported once. Tabs found silent in the same check are all many.
New commit or tab close	A page that already waited 30 s is reported first; a younger one is dropped.
Web content process terminated	The tab's page is dropped, not reported.
App suspended (iOS background, macOS sleep)	Nothing is checked, and the suspended time is added back to every wait; a page committed while suspended starts waiting at resume. The clock is time since boot, so a wall-clock change cannot make tabs overdue.
Extension reload	All waits, mappings and series are forgotten; the open pages stay, so tabs are learned again.

Tying a message to a tab

A message carries the extension's tab identifier and the page URL. WebKit creates its extension tab object lazily, and neither public nor private API exposes its identifier, so the app learns it: the first message from an extension tab that matches exactly one committed page (host and normalized path, the same match the privacy dashboard uses) fixes that tab's identifier; later messages use it directly. A message teaches a tab only while that tab still waits for its first message: a message about an already answered page may belong to the page before. A message that cannot be tied to one tab (two tabs on the same URL, no tab identifier, no match) answers every tab it could belong to, so the metric prefers missing a breakage to reporting a working tab. A tab closing forgets its identifier.

Pixels

Pixel	Trigger	Frequency
web_extension_cpm_breakage	A tab is silent.	Daily + count
web_extension_cpm_breakage_persistent	A second tab is silent with no message from the extension in any tab since the first one was reported. Once per series.	Daily + count

_daily is installs per day; _count is events. Metadata follows the existing episode pixel on each platform: appVersion on both; macOS also pixelSource, channel, suffix daily_count; iOS keeps its suffix set. The macOS definition must match what is sent.

web_extension_cpm_breakage parameter:

Parameter	Values
tabs	one: no other tab was silent since the extension last sent a message; the page is the likely cause. many: another tab was silent too, before or in the same check; the extension is the likely cause.

web_extension_cpm_breakage_persistent has no parameters. last_stage is added by the companion design.

A series ends with any message from the extension, in any tab. A silent tab that is closed still counts for its series. A single tab that keeps going silent is tabs=one each time and never persistent.

Dashboard: persistent installs / DAU from breakage_persistent_daily (the success metric); any-breakage installs / DAU from breakage_daily; breakages per day from breakage_count by tabs.

Removed in the same stack

The episode pixel web_extension_cpm_episode with its timers, episode state and failure reason, web_extension_cpm_background_load_restart (and its observation), and web_extension_cpm_network_process_recovery_applied: definitions, events and firing code are removed in PR 2. The recovery behavior (error-6 restart, network recovery) stays; only the pixels go. Between PR 2 and PR 3 no CPM breakage pixel exists, so they must ship in the same release.

PR stack

PR	Content	State, 11 Nov
1, #8025	Remove ineffective recovery fixes and flags	pushed, in review
2, #8054, alex/cpm-pixel-cleanup (fac1c98edc)	All removal. The monitor shrinks to learning which extension tab belongs to which app tab and letting the manager reset its restart budget. Tabs report one commit event with the kind of navigation (ordinary, error page, history restore, restored session); the monitor decides from the kind and the scheme whether the page can run the extension. Navigation start, finish and failure events are gone.	pushed, in review (stacked on #8025); 1004 package tests pass. iOS error pages are ineligible; back/forward is read at navigation start
3, alex/cpm-breakage-metric (a696254ca3)	Per-tab silence, the two pixels, tabs, definitions, adapters	local, one commit on PR 2; 1048 package tests pass; validators pass; not pushed
4, alex/cpm-init-marks (8751a745ab)	Init progress marks and last_stage; extension #3727	local, one commit on PR 3; 1071 package tests pass; not pushed

Known limits

• Evaluation happens only on events. A silent tab in an app that then sits idle until it is killed is not reported.
• The first message of a tab can only be tied to a tab by URL. Two tabs on the same URL, or a page whose path changed before the first message, stay unmapped; their messages answer every candidate tab, which hides some real silence.
• A single-page app changing its path can break the host and path match for a tab that is not mapped yet.
• Pages the extension does not inject into (some non-document types) send no message and look silent. The MIME type is not available at commit, so these are not filtered.
• A message within 2 s before a commit is counted as that commit's answer even when it belonged to the previous page.
• A message from an extension tab that already belongs to another app tab is never counted as an early answer (or mark) for a new tab's commit. A late refresh from the previous page of a mapped tab, arriving after the next commit, still answers the new page; marks are checked against the current page, answers are not.
• A reload of the same page within 2 s of its last mark starts with that mark's stage; last_stage can then show the previous load's furthest stage instead of none.
• A first message can teach the wrong tab when another tab sits on the same host and path but is not in the committed pages (back/forward or restored page) inside the 2 s window; that tab's later commits then count as silent. Low likelihood, not guarded.
• The dashboard refresh is sent independently of initResp, so a page whose initResp was never delivered counts as answered.
• The first series has no overlap with the episode series: comparison is by app version against the previous derived number.
• The restart budget now resets on any dashboard refresh from the current embedded extension, not on a message matched to a page, and without the old http(s) and generation filters.
• A page that navigates before the 30 s window ends is dropped, so a user who navigates every 20 s with a dead extension is never reported.
• If the native channel itself is lost, every tab looks silent and tabs=many follows: the metric cannot tell a lost channel from a broken extension.

Cons

• Silence of tabs left open until the app exits is never reported.
• 30 s hides breakage on short pages.
• Streak and series reset on relaunch.
• The tab mapping is a heuristic bootstrap; its error direction is missing a breakage, not inventing one.

Risks

• A page that works but whose message is not attributed counts as silent when the mapping is wrong. Mitigation: ambiguous messages answer all candidates.
• Pixel volume for a persistently broken install: one count request per silent tab.
• The Debug "Simulate CPM Messaging Breakage" switch must keep hiding messages (it does; tests cover it).
• Both definitions have no expires: they are the permanent metric (REVIEW-pixels.md).

Testing

• Unit tests per row of the table (window boundary, no timer, late answer, tolerance before commit, suspension, ineligible commits, crash, close, reload, mapping learned, kept and ambiguous, simulation, old extension, tabs one/many, persistent once and again after a message, closed silent tab). Mutation checks: answer applied after the check, no suspension shift, persistent on one tab, no tolerance each make a test fail.
• Pixel definitions pass both validators; adapter tests check names, frequency and parameters.
• App-hosted macOS and iOS tests: not run in this stack; package tests, validators, SwiftLint and the macOS build-for-testing are the checks.
• Manual: CPM Breakage Testing Steps with the Debug switch.
• iOS build-for-testing (iOS Unit Tests, simulator) passes on PR 4.
• Real WebKit (CPM repro harness, PR #7378): not run.

Additional Considerations (if applicable)

Privacy: two new pixel names with enumerated parameters; no URLs, identifiers or page content. Privacy Triage required, not requested yet.

Security: no new private WebKit selectors.

Site Breakage: none; the detector only observes events and recovery is unchanged.

Experimentation: none.

Operational: no pixel expiry (permanent metric). Danger label for the removed pixels. Data checks: metrics.pixels_aggregation and metrics._pixels_new (pixel_id = 'web').

Localization / Internationalization: none.