TD: Pixel Definition: CPM Permanent Breakage Metric (macOS/iOS + embedded web extension)

Author: Alex M Reviewer: TBD Stakeholders: Michal, Konrad, Russell Project: CPM: root-cause the stuck embedded extension messaging (macOS/iOS)

Background & Requirements

Cookie pop-up management (CPM) runs in an embedded web extension. It can stop answering and stay broken even after page reloads and new tabs. Our goal is to keep these permanent failures below 1% of daily active users, but nothing measures it directly: the number is derived from the web_extension_cpm_episode pixel, and that is a weak basis. An episode starts after a four-second navigation timeout, so a slow but correct answer counts as a failure, and an episode still open when the app closes is lost.

We need a measure that does not depend on a navigation timer, can be expressed as a share of daily users, and collects no URLs, page content or persistent identifiers.

Problem Statement

Rework the health monitor so that it finds the tabs whose pages never get a message from the extension, and tells a broken page from a broken extension. That requires knowing which tab of the app each message belongs to, and the extension names a tab only by an id that WebKit assigns and the app cannot see.

Recommended Approach

When a tab is counted as silent. While a page initializes, the extension sends the app a message from that tab. When a page loads in a tab, the app waits 30 seconds for that message, provided that:
- the address is http or https;
- the navigation is a normal one: not the browser's error page, not a back/forward navigation, not a restored session (the extension may not run there);
- the extension is loaded.
If no message about the page arrives within 30 seconds, the tab is silent and one web_extension_cpm_breakage pixel is sent for it.
When a tab is not counted.
- A message that arrives late but before the app checks still counts as an answer, as does a message sent up to 2 seconds before the page loaded.
- A message saying that the extension cannot answer (no configuration or settings) is not an answer.
- The wait is dropped when the tab closes, when its page process crashes, when the tab moves on to a page that does not qualify, and when the extension reloads. A new qualifying page in the same tab starts a new wait.
- Time the app spends suspended (iOS in the background, macOS asleep) is not counted.
- There is no timer: the app checks for expired waits whenever another event comes in (a page loads, a tab closes, a message arrives, the app goes to the background).
Matching problem. A message carries the extension's tab id and the page URL. The app cannot see the tab id, so it can only match the message to a tab by the URL (host and path). Two tabs on the same page cannot be told apart: the message counts as an answer for both, and a silent tab hides behind a working one.
Solution. The extension adds to each message the tab's index in its window, the id of that window and the ids of all windows, as the browser reports them. The app numbers its windows in the order it hands them to WebKit, and WebKit gives each window its id from a growing counter in that same order. So the k-th smallest of the window ids is the k-th window of the app, and the tab is the one at the index in that window. The app remembers which tab the extension's tab id belongs to until the tab closes or the extension reloads. If the app cannot tell the window (an older extension, or the app and the extension count different numbers of windows), it matches by the URL as before.
Two pixels:

Pixel	When it is sent	Frequency
`web_extension_cpm_breakage`	A tab becomes silent. Parameter `tabs`: `one` if no other tab was silent since the extension last sent a message (the page is the likely cause), `many` if another one was (the extension is the likely cause).	Daily + count
`web_extension_cpm_breakage_persistent`	A second tab becomes silent and the extension sent no message in between. Once per such series.	Daily + count

Remove web_extension_cpm_episode, web_extension_cpm_background_load_restart and web_extension_cpm_network_process_recovery_applied, together with the code that only fed them. The new pixels must ship in the same release, otherwise there is no CPM breakage pixel at all.
The success metric is the number of installs per day with the persistent pixel, divided by DAU.

Testing

Unit tests for each rule above, including windows and tabs that are opened, closed, reordered, pinned and moved between windows.

Additional Considerations (if applicable)

Privacy: Two new pixels. Only web_extension_cpm_breakage has a parameter, tabs: one or many. The tab index and window ids go only from the extension to the app on the device, never into a pixel.
Security: None.
Site Breakage: None.
Experimentation: None.
Operational: None.
Localization / Internationalization: None.

5.7 KiB Raw Blame History