obsidian-vault/work/projects/cpm-permanent-breakage-metric-tech-design-2026-11-10.md

tags, updated, status

tags	updated	status
work cpm tech-design pixel	2026-11-11	draft

TD: Pixel Definition: CPM Permanent Breakage Metric (macOS/iOS)

Internal metrics, keep out of public PRs. Companion: cpm-wake-init-detection-tech-design-2026-11-10 (adds last_stage). Where we are: cpm-project-summary-2026-11-03. Reworked on 11 Nov from "judge a page when its document ends" to "a tab is silent"; the first design is gone.

Author: Alex M Reviewer: TBD (previous CPM pixel designs: Michal) Stakeholders: Michal, Konrad, Russell (from the previous Tech Designs; confirm) Project: CPM: root-cause the stuck embedded extension messaging (macOS/iOS) Task: to create ("Permanent breakage metrics") Previous Tech Designs: cpm-readiness-telemetry-tech-design-2026-10-15, cpm-m3-telemetry-privacy-triage Privacy Triage: to request (new pixel names)

Background & Requirements

The goal is permanent CPM failures below 1% of DAU. There is no pixel for it; the number is derived from web_extension_cpm_episode with a long SQL. The episode is wrong in two ways:

• A timer decides. A four-second navigation timeout opens an episode. On iOS 7.244, 6.0% of episodes closed with a late answer on the same page, 42% of those later than 30 s. Slow answers count as failures.
• Episodes are lost. An open episode lives in memory and is dropped when the app exits.

Requirements: a metric without a navigation timer, as a rate against DAU; no page content, URLs or persistent identifiers; a minimal layer next to the product code (no private selectors, no per-navigation state machine). End state: the only CPM pixels are the two defined here.

Problem Statement

Report tabs whose pages never hear from the extension, and tell one broken page from a broken extension.

Recommended Approach

The extension sends a message from a tab while a page initializes. A tab that commits an eligible page and gets no message for 30 s of app time is silent. A message that arrives later still counts as an answer.

The monitor keeps, per tab: the time of the latest eligible commit (until a message answers it) and the time of the latest message. Nothing runs on a timer: the check happens when any event arrives (a commit, a tab closing, a message, the app waking).

Event	Effect
Commit of an http(s) page, extension loaded, not an error page, not a history restore, not a restored session (tabs send the kind of navigation, the monitor decides)	The tab waits for a message. A message up to 2 s before the commit answers it (the extension can message a tab before the app reports the commit), also for a tab whose extension tab is not learned yet.
Message from the current extension (dashboard refresh)	Answers the tab's page, even when late. Ends the series of silent tabs.
config_unavailable or settings_missing refresh	Not an answer: the extension said it cannot answer the page's init. It still shows the extension is alive (restart budget reset).
Any event, with a commit older than 30 s and no message	The tab is silent: reported once. Tabs found silent in the same check are all many.
New commit or tab close	A page that already waited 30 s is reported first; a younger one is dropped.
Web content process terminated	The tab's page is dropped, not reported.
App suspended (iOS background, macOS sleep)	Silent tabs are checked as the suspension begins, so a tab already past 30 s is reported when iOS goes to the background. While suspended nothing is checked, and the suspended time is added back to every wait; a page committed while suspended starts waiting at resume. The clock is time since boot, so a wall-clock change cannot make tabs overdue.
Extension reload	All waits, mappings and series are forgotten; the open pages stay, so tabs are learned again.

Tying a message to a tab

A message carries the extension's tab identifier and the page URL. WebKit creates its extension tab object lazily, and neither public nor private API exposes its identifier, so the app learns it: the first message from an extension tab that matches exactly one committed page (host and normalized path, the same match the privacy dashboard uses) fixes that tab's identifier; later messages use it directly. A message teaches a tab only while that tab still waits for its first message: a message about an already answered page may belong to the page before. A message that cannot be tied to one tab (two tabs on the same URL, no tab identifier, no match) answers every tab it could belong to, so the metric prefers missing a breakage to reporting a working tab. A tab closing forgets its identifier.

Pixels

Pixel	Trigger	Frequency
web_extension_cpm_breakage	A tab is silent.	Daily + count
web_extension_cpm_breakage_persistent	A second tab is silent with no message from the extension in any tab since the first one was reported. Once per series.	Daily + count

_daily is installs per day; _count is events. Metadata follows the existing episode pixel on each platform: appVersion on both; macOS also pixelSource, channel, suffix daily_count; iOS keeps its suffix set. The macOS definition must match what is sent.

web_extension_cpm_breakage parameter:

Parameter	Values
tabs	one: no other tab was silent since the extension last sent a message; the page is the likely cause. many: another tab was silent too, before or in the same check; the extension is the likely cause.

web_extension_cpm_breakage_persistent has no parameters. last_stage is added by the companion design.

A series ends with any message from the extension, in any tab. A silent tab that is closed still counts for its series. A single tab that keeps going silent is tabs=one each time and never persistent.

Dashboard: persistent installs / DAU from breakage_persistent_daily (the success metric); any-breakage installs / DAU from breakage_daily; breakages per day from breakage_count by tabs.

Removed in the same stack

The episode pixel web_extension_cpm_episode with its timers, episode state and failure reason, web_extension_cpm_background_load_restart (and its observation), and web_extension_cpm_network_process_recovery_applied: definitions, events and firing code are removed in PR 2. The recovery behavior (error-6 restart, network recovery) stays; only the pixels go. Between PR 2 and PR 3 no CPM breakage pixel exists, so they must ship in the same release.

PR stack

PR	Content	State, 11 Nov
1, #8025	Remove ineffective recovery fixes and flags	pushed, in review
2, #8054, alex/cpm-pixel-cleanup (fac1c98edc)	All removal. The monitor shrinks to learning which extension tab belongs to which app tab and letting the manager reset its restart budget. Tabs report one commit event with the kind of navigation (ordinary, error page, history restore, restored session); the monitor decides from the kind and the scheme whether the page can run the extension. Navigation start, finish and failure events are gone.	pushed, in review (stacked on #8025); 1004 package tests pass. iOS error pages are ineligible; back/forward is read at navigation start
3, alex/cpm-breakage-metric (a696254ca3)	Per-tab silence, the two pixels, tabs, definitions, adapters	local, one commit on PR 2; 1048 package tests pass; validators pass; not pushed
4, alex/cpm-init-marks (8751a745ab)	Init progress marks and last_stage; extension #3727	local, one commit on PR 3; 1071 package tests pass; not pushed

Reporting cases

Each case was run against the monitor (PR 3, 30 s window, 2 s tolerance) with an injected clock on 11 Nov; the outcome column is what the code did. "Hidden" is a missed breakage, "wrong" is a report for a working extension.

#	Case	Outcome
1	Page commits, the extension answers within 30 s	Nothing.
2	Answer arrives after 30 s with no event in between	Nothing: the answer is applied before the check.
3	Answer arrives after 30 s but another event came first	Tab reported one; the late answer then ends the series.
4	One page silent, the user stays on it, then any event	one.
5	Same tab, three silent pages in a row (30 s+ each)	one, one, one; never persistent (one tab).
6	Two tabs silent one after the other	one, then many and persistent once.
7	Two tabs silent in the same check	many, many and persistent once.
8	A third tab gets an answer, then two more tabs go silent	many, many, persistent again (the answer ended the series).
9	Dead extension, the user navigates every 20 s	Nothing: pages younger than 30 s are dropped.
10	iOS goes to the background with a tab already past 30 s	Reported at the suspension.
11	Suspended for 100 s with 10 s already waited	Not silent at 29 s of app time, silent at 31 s: suspended time does not count.
12	config_unavailable / settings_missing refresh	Page reported one: not an answer.
13	Two tabs on the same host and path (for example two duckduckgo.com/?q= result pages), one message	Both answered. Hidden if one was silent.
14	Late refresh of the previous page on a mapped tab, after the next commit	Answers the new page. Hidden.
15	Page path changes (single-page app) before the first message	No tab matches, nothing answered, tab reported one. Wrong.
16	Message from tab C (back/forward commit, not tracked) while tab A waits on the same host and path	C's extension tab is mapped to A. When C then loads an eligible page and the extension answers, C is reported one. Wrong.

Cases 15 and 16 are the only paths that report a working extension; the design's "ambiguity answers every candidate" does not cover them. A possible fix, not implemented: treat a mapping as invalid when the message's host and path match a different committed page than the mapped tab's, and learn again.

Known limits

• Evaluation happens only on events. A silent tab in an app that then sits idle until it is killed is not reported; iOS reports it at the background transition, macOS at sleep.
• The page match ignores the query, so every result page of a search site, or every watch page of a video site, has the same identity. Tabs on such pages stay unmapped and their messages answer every candidate (case 13).
• A page whose path changed before the first message matches no tab and is reported silent although the extension answered (case 15).
• A message from an untracked tab (back/forward or restored page) can teach the wrong tab when exactly one other tab waits on the same host and path; that tab's later commits then count as silent (case 16). Low likelihood, not guarded.
• Pages the extension does not inject into (some non-document types) send no message and look silent. The MIME type is not available at commit, so these are not filtered. Whether the extension messages on every page the app counts as eligible (search result pages, duck.ai, non-document types) is not verified: the extension source is not in this repo.
• A message within 2 s before a commit is counted as that commit's answer even when it belonged to the previous page, in the same tab or another one that is not mapped yet.
• A message from an extension tab that already belongs to another app tab is never counted as an early answer (or mark) for a new tab's commit. A late refresh from the previous page of a mapped tab, arriving after the next commit, still answers the new page (case 14); marks are checked against the current page, answers are not.
• A reload of the same page within 2 s of its last mark starts with that mark's stage; last_stage can then show the previous load's furthest stage instead of none.
• The dashboard refresh is sent independently of initResp, so a page whose initResp was never delivered counts as answered.
• The first series has no overlap with the episode series: comparison is by app version against the previous derived number.
• The restart budget now resets on any dashboard refresh from the current embedded extension, not on a message matched to a page, and without the old http(s) and generation filters.
• A page that navigates before the 30 s window ends is dropped, so a user who navigates every 20 s with a dead extension is never reported (case 9).
• A single tab can never make the persistent pixel: it needs two distinct tabs. A user with a dead extension who browses in one tab sends breakage only.
• If the native channel itself is lost, every tab looks silent and tabs=many follows: the metric cannot tell a lost channel from a broken extension.

Cons

• Silence of tabs left open until the app exits is never reported.
• 30 s hides breakage on short pages.
• Streak and series reset on relaunch.
• The tab mapping is a heuristic bootstrap. It mostly errs towards missing a breakage, but two cases report a working extension (cases 15 and 16).

Risks

• A page that works but whose message is not attributed counts as silent when the mapping is wrong. Mitigation: ambiguous messages answer all candidates; an unambiguous wrong match (case 16) and a path change (case 15) are not covered.
• Fire (burner) tabs are measured: the embedded extension is trusted and gets private browsing access, so it runs in them (checked in the loader and the permission controller).
• Pixel volume for a persistently broken install: one count request per silent tab.
• The Debug "Simulate CPM Messaging Breakage" switch must keep hiding messages (it does; tests cover it).
• Both definitions have no expires: they are the permanent metric (REVIEW-pixels.md).

Testing

• Unit tests per row of the table (window boundary, no timer, late answer, tolerance before commit, suspension, ineligible commits, crash, close, reload, mapping learned, kept and ambiguous, simulation, old extension, tabs one/many, persistent once and again after a message, closed silent tab). Mutation checks: answer applied after the check, no suspension shift, persistent on one tab, no tolerance each make a test fail.
• Pixel definitions pass both validators; adapter tests check names, frequency and parameters.
• Package tests, validators, SwiftLint and build-for-testing on both platforms. App-hosted, signed: iOS (simulator) TabTests and WebExtensionPixelFiringTests 61 tests, macOS unit tests 16 (MacOSWebExtensionPixelFiringTests, WebExtensionsDebugMenuTests, NavigationPixelNavigationResponderTests) and the integration CPMErrorPageMonitoringTests 3 tests pass.
• The reporting cases above were run once as a throwaway test, which is not kept. Existing unit tests cover cases 1, 2, 4 to 13; cases 3, 14, 15 and 16 have no test of their own.
• Manual: CPM Breakage Testing Steps with the Debug switch.
• Real WebKit (CPM repro harness, PR #7378): not run.

Additional Considerations (if applicable)

Privacy: two new pixel names with enumerated parameters; no URLs, identifiers or page content. Privacy Triage required, not requested yet.

Security: no new private WebKit selectors.

Site Breakage: none; the detector only observes events and recovery is unchanged.

Experimentation: none.

Operational: no pixel expiry (permanent metric). Danger label for the removed pixels. Data checks: metrics.pixels_aggregation and metrics._pixels_new (pixel_id = 'web').

Localization / Internationalization: none.