git_admin/obsidian-vault
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[2026-11-11] eagle: work/projects/cpm-permanent-breakage-metric-tech-design-2026-11-10.md work/projects/cpm-wake-init-detection-tech-design-2026-11-10.md

Browse Source
This commit is contained in:
Alexey Martemyanov committed 2026-11-11 18:51:37 +06:00
1 parent 8ecb8cc4d9
commit cc911f076a
2 files changed
+79 -249
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

work/projects/cpm-permanent-breakage-metric-tech-design-2026-11-10.md
+33 -112
View File
@@ -14,153 +14,74 @@ status: draft
**Stakeholders:** Michal, Konrad, Russell (from the previous Tech Designs; confirm) **Stakeholders:** Michal, Konrad, Russell (from the previous Tech Designs; confirm)
**Project:** [CPM: root-cause the stuck embedded extension messaging (macOS/iOS)](https://app.asana.com/1/137249556945/task/1216761517055116) **Project:** [CPM: root-cause the stuck embedded extension messaging (macOS/iOS)](https://app.asana.com/1/137249556945/task/1216761517055116)
**Task:** to create ("Permanent breakage metrics") **Task:** to create ("Permanent breakage metrics")
**Previous Tech Designs:** [[cpm-readiness-telemetry-tech-design-2026-10-15]], [[cpm-m3-telemetry-privacy-triage]] **Previous Tech Designs:** [CPM breakage pixels](https://app.asana.com/1/137249556945/task/1218005985662983), [CPM failure diagnostics](https://app.asana.com/1/137249556945/task/1218619301158188)
**Privacy Triage:** to request (new pixel names) **Privacy Triage:** to request (new pixel names)
## Background & Requirements ## Background & Requirements
The goal is permanent CPM failures below 1% of DAU. There is no pixel for it; the number is derived from `web_extension_cpm_episode` with a long SQL. The episode is wrong in two ways: The goal is permanent CPM failures below 1% of DAU. There is no pixel for it: the number is derived from `web_extension_cpm_episode` with a long SQL query, and the episode is a poor basis:
- **A timer decides.** A four-second navigation timeout opens an episode. On iOS 7.244, 6.0% of episodes closed with a late answer on the same page, 42% of those later than 30 s. Slow answers count as failures. - **A timer decides.** A four-second navigation timeout opens an episode. On iOS 7.244, 6.0% of episodes closed with a late answer on the same page, 42% of those later than 30 s. Slow answers are counted as failures.
- **Episodes are lost.** An open episode lives in memory and is dropped when the app exits. - **Episodes are lost.** An open episode lives in memory and is dropped when the app exits.
Requirements: a metric without a navigation timer, as a rate against DAU; no page content, URLs or persistent identifiers; a minimal layer next to the product code (no private selectors, no per-navigation state machine). End state: the only CPM pixels are the two defined here. Requirements: a metric without a navigation timer, expressed as a rate against DAU; no page content, URLs or persistent identifiers; a small layer next to the product code (no private WebKit selectors). End state: the only CPM pixels are the two below.
## Problem Statement ## Problem Statement
Report tabs whose pages never hear from the extension, and tell one broken page from a broken extension. Report the tabs whose pages never hear from the extension, and tell one broken page from a broken extension.
## Recommended Approach ## Recommended Approach
**The extension sends a message from a tab while a page initializes. A tab that commits an eligible page and gets no message for 30 s of app time is silent. A message that arrives later still counts as an answer.** While a page initializes, the extension sends the app a message from that tab. The app treats a tab as **silent** when it committed a page that should get a message and heard nothing for 30 seconds of app time. A message that arrives later still counts as an answer. There is no timer: the check runs whenever any other event arrives (a page commit, a tab closing, a message, the app going to the background or waking).
The monitor keeps, per tab: the time of the latest eligible commit (until a message answers it) and the time of the latest message. Nothing runs on a timer: the check happens when any event arrives (a commit, a tab closing, a message, the app waking). - **Pages that count:** http(s), extension loaded, not the browser's error page, not a back/forward restore, not a restored session.
- **Time that counts:** time the app is suspended is not counted. A page that is replaced or closed before 30 s, and a page whose process crashed, is not reported.
| Event | Effect | - **Which tab a message belongs to:** the message carries the extension's tab number and the page URL. The app learns the tab number from the first message that matches exactly one open page (host and path). A message that cannot be tied to one tab answers every tab it could belong to, so the metric prefers missing a breakage to reporting a working tab.
|---|---| - **Not an answer:** a refresh saying the extension has no configuration or no settings. It shows the extension is alive, but the page stays unanswered.
| Commit of an http(s) page, extension loaded, not an error page, not a history restore, not a restored session (tabs send the kind of navigation, the monitor decides) | The tab waits for a message. A message up to 2 s before the commit answers it (the extension can message a tab before the app reports the commit), also for a tab whose extension tab is not learned yet. |
| Message from the current extension (dashboard refresh) | Answers the tab's page, even when late. Ends the series of silent tabs. |
| `config_unavailable` or `settings_missing` refresh | Not an answer: the extension said it cannot answer the page's init. It still shows the extension is alive (restart budget reset). |
| Any event, with a commit older than 30 s and no message | The tab is silent: reported once. Tabs found silent in the same check are all `many`. |
| New commit or tab close | A page that already waited 30 s is reported first; a younger one is dropped. |
| Web content process terminated | The tab's page is dropped, not reported. |
| App suspended (iOS background, macOS sleep) | Silent tabs are checked as the suspension begins, so a tab already past 30 s is reported when iOS goes to the background. While suspended nothing is checked, and the suspended time is added back to every wait; a page committed while suspended starts waiting at resume. The clock is time since boot, so a wall-clock change cannot make tabs overdue. |
| Extension reload | All waits, mappings and series are forgotten; the open pages stay, so tabs are learned again. |
### Tying a message to a tab
A message carries the extension's tab identifier and the page URL. WebKit creates its extension tab object lazily, and neither public nor private API exposes its identifier, so the app learns it: the first message from an extension tab that matches exactly one committed page (host and normalized path, the same match the privacy dashboard uses) fixes that tab's identifier; later messages use it directly. A message teaches a tab only while that tab still waits for its first message: a message about an already answered page may belong to the page before. A message that cannot be tied to one tab (two tabs on the same URL, no tab identifier) answers every tab it could belong to; when no page matches by host and path (the page changed its path first), it answers the tabs still waiting on that host, without learning a mapping, so the metric prefers missing a breakage to reporting a working tab. A tab closing forgets its identifier.
### Pixels
| Pixel | Trigger | Frequency | | Pixel | Trigger | Frequency |
|---|---|---| |---|---|---|
| `web_extension_cpm_breakage` | A tab is silent. | Daily + count | | `web_extension_cpm_breakage` | A tab is silent. | Daily + count |
| `web_extension_cpm_breakage_persistent` | A second tab is silent with no message from the extension in any tab since the first one was reported. Once per series. | Daily + count | | `web_extension_cpm_breakage_persistent` | A second tab is silent and no tab heard from the extension in between. Once per series. | Daily + count |
`_daily` is installs per day; `_count` is events. Metadata follows the existing episode pixel on each platform: `appVersion` on both; macOS also `pixelSource`, `channel`, suffix `daily_count`; iOS keeps its suffix set. The macOS definition must match what is sent. `web_extension_cpm_breakage` has the parameter `tabs`: `one` (no other tab was silent since the extension last sent a message: the page is the likely cause) or `many` (another tab was silent too: the extension is the likely cause). The persistent pixel has no parameters. A series ends when the extension sends any message from any tab. A second parameter, `last_stage`, is defined in the wake init detection design.
`web_extension_cpm_breakage` parameter: Dashboard: installs per day with the persistent pixel / DAU is the success metric; installs with any breakage / DAU and breakages per day by `tabs` complete it.
| Parameter | Values | The episode pixel (`web_extension_cpm_episode`), `web_extension_cpm_background_load_restart` and `web_extension_cpm_network_process_recovery_applied` are removed together with the code that only fed them. The recoveries themselves are unchanged.
|---|---|
| `tabs` | `one`: no other tab was silent since the extension last sent a message; the page is the likely cause. `many`: another tab was silent too, before or in the same check; the extension is the likely cause. |
`web_extension_cpm_breakage_persistent` has no parameters. `last_stage` is added by the companion design. ## Pros
A series ends with any message from the extension, in any tab. A silent tab that is closed still counts for its series. A single tab that keeps going silent is `tabs=one` each time and never persistent. - No navigation timer, and a slow answer is no longer a failure.
- A rate against DAU straight from the pixel, without a query chain.
- Small: one monitor that counts events, no per-navigation state machine.
Dashboard: persistent installs / DAU from `breakage_persistent_daily` (the success metric); any-breakage installs / DAU from `breakage_daily`; breakages per day from `breakage_count` by `tabs`. ## Cons and known limits
### Removed in the same stack - A tab left open and silent until the app exits is not reported (iOS reports it when going to the background).
- A page left before 30 s is not reported, so a user who moves on every 20 s with a broken extension is never seen.
The episode pixel `web_extension_cpm_episode` with its timers, episode state and failure reason, `web_extension_cpm_background_load_restart` (and its observation), and `web_extension_cpm_network_process_recovery_applied`: definitions, events and firing code are removed in PR 2. The recovery behavior (error-6 restart, network recovery) stays; only the pixels go. Between PR 2 and PR 3 no CPM breakage pixel exists, so they must ship in the same release. - Tabs on the same host and path (for example two search result pages) cannot be told apart: a silent one can hide behind its twin.
- The tab mapping is a heuristic. In rare cases it attributes a message to the wrong tab and a working extension is reported.
### PR stack - Pages the extension does not inject into look silent. The page type is not known at commit, so they are not filtered.
- One tab can never make the persistent pixel: it needs two distinct tabs. If the native channel itself is lost, every tab looks silent and `tabs=many` follows.
| PR | Content | State, 11 Nov |
|---|---|---|
| 1, [#8025](https://github.com/duckduckgo/apple-browsers/pull/8025) | Remove ineffective recovery fixes and flags | pushed, in review |
| 2, [#8054](https://github.com/duckduckgo/apple-browsers/pull/8054), `alex/cpm-pixel-cleanup` (`fac1c98edc`) | All removal. The monitor shrinks to learning which extension tab belongs to which app tab and letting the manager reset its restart budget. Tabs report one commit event with the kind of navigation (ordinary, error page, history restore, restored session); the monitor decides from the kind and the scheme whether the page can run the extension. Navigation start, finish and failure events are gone. | pushed, in review (stacked on #8025); 1004 package tests pass. iOS error pages are ineligible; back/forward is read at navigation start |
| 3, `alex/cpm-breakage-metric` (`eea85b0812`) | Per-tab silence, the two pixels, `tabs`, definitions, adapters | local, one commit on PR 2; 1050 package tests pass; validators pass; not pushed |
| 4, `alex/cpm-init-marks` (`afa2bf2289`) | Init progress marks and `last_stage`; extension #3727 | local, one commit on PR 3; 1073 package tests pass; not pushed |
### Reporting cases
Each case was run against the monitor (PR 3, 30 s window, 2 s tolerance) with an injected clock on 11 Nov; the outcome column is what the code did. "Hidden" is a missed breakage, "wrong" is a report for a working extension.
| # | Case | Outcome |
|---|---|---|
| 1 | Page commits, the extension answers within 30 s | Nothing. |
| 2 | Answer arrives after 30 s with no event in between | Nothing: the answer is applied before the check. |
| 3 | Answer arrives after 30 s but another event came first | Tab reported `one`; the late answer then ends the series. |
| 4 | One page silent, the user stays on it, then any event | `one`. |
| 5 | Same tab, three silent pages in a row (30 s+ each) | `one`, `one`, `one`; never persistent (one tab). |
| 6 | Two tabs silent one after the other | `one`, then `many` and persistent once. |
| 7 | Two tabs silent in the same check | `many`, `many` and persistent once. |
| 8 | A third tab gets an answer, then two more tabs go silent | `many`, `many`, persistent again (the answer ended the series). |
| 9 | Dead extension, the user navigates every 20 s | Nothing: pages younger than 30 s are dropped. |
| 10 | iOS goes to the background with a tab already past 30 s | Reported at the suspension. |
| 11 | Suspended for 100 s with 10 s already waited | Not silent at 29 s of app time, silent at 31 s: suspended time does not count. |
| 12 | `config_unavailable` / `settings_missing` refresh | Page reported `one`: not an answer. |
| 13 | Two tabs on the same host and path (for example two `duckduckgo.com/?q=` result pages), one message | Both answered. Hidden if one was silent. |
| 14 | Late refresh of the previous page on a mapped tab, after the next commit | Answers the new page. Hidden. |
| 15 | Page path changes (single-page app) before the first message | The tabs still waiting on that host are answered; no tab is learned. Before this was fixed the tab was reported `one` although the extension had answered. |
| 16 | Message from tab C (back/forward commit, not tracked) while tab A waits on the same host and path | C's extension tab is mapped to A. When C then loads an eligible page and the extension answers, C is reported `one`. Wrong. |
Case 16 is the only path left that reports a working extension; the design's "ambiguity answers every candidate" does not cover it. A possible fix, not implemented: treat a mapping as invalid when the message's host and path match a different committed page than the mapped tab's, and learn again.
### Known limits
- Evaluation happens only on events. A silent tab in an app that then sits idle until it is killed is not reported; iOS reports it at the background transition, macOS at sleep.
- The page match ignores the query, so every result page of a search site, or every watch page of a video site, has the same identity. Tabs on such pages stay unmapped and their messages answer every candidate (case 13).
- A page whose path changed before the first message matches no page by host and path. The message then answers the tabs still waiting on that host, which can hide a silent tab on the same host (case 15), and teaches no mapping.
- A message from an untracked tab (back/forward or restored page) can teach the wrong tab when exactly one other tab waits on the same host and path; that tab's later commits then count as silent (case 16). Low likelihood, not guarded.
- Pages the extension does not inject into (some non-document types) send no message and look silent. The MIME type is not available at commit, so these are not filtered. Whether the extension messages on every page the app counts as eligible (search result pages, `duck.ai`, non-document types) is not verified: the extension source is not in this repo.
- A message within 2 s before a commit is counted as that commit's answer even when it belonged to the previous page, in the same tab or another one that is not mapped yet.
- A message from an extension tab that already belongs to another app tab is never counted as an early answer (or mark) for a new tab's commit. A late refresh from the previous page of a mapped tab, arriving after the next commit, still answers the new page (case 14); marks are checked against the current page, answers are not.
- A reload of the same page within 2 s of its last mark starts with that mark's stage; `last_stage` can then show the previous load's furthest stage instead of `none`.
- The dashboard refresh is sent independently of `initResp`, so a page whose `initResp` was never delivered counts as answered.
- The first series has no overlap with the episode series: comparison is by app version against the previous derived number.
- The restart budget now resets on any dashboard refresh from the current embedded extension, not on a message matched to a page, and without the old http(s) and generation filters.
- A page that navigates before the 30 s window ends is dropped, so a user who navigates every 20 s with a dead extension is never reported (case 9).
- A single tab can never make the persistent pixel: it needs two distinct tabs. A user with a dead extension who browses in one tab sends `breakage` only.
- If the native channel itself is lost, every tab looks silent and `tabs=many` follows: the metric cannot tell a lost channel from a broken extension.
### Cons
- Silence of tabs left open until the app exits is never reported.
- 30 s hides breakage on short pages.
- Streak and series reset on relaunch.
- The tab mapping is a heuristic bootstrap. It errs towards missing a breakage, except one case that reports a working extension (case 16).
### Risks
- A page that works but whose message is not attributed counts as silent when the mapping is wrong. Mitigation: ambiguous messages answer all candidates; an unambiguous wrong match (case 16) is not covered.
- Fire (burner) tabs are measured: the embedded extension is trusted and gets private browsing access, so it runs in them (checked in the loader and the permission controller).
- Pixel volume for a persistently broken install: one count request per silent tab.
- The Debug "Simulate CPM Messaging Breakage" switch must keep hiding messages (it does; tests cover it).
- Both definitions have no `expires`: they are the permanent metric (`REVIEW-pixels.md`).
## Testing ## Testing
- Unit tests per row of the table (window boundary, no timer, late answer, tolerance before commit, suspension, ineligible commits, crash, close, reload, mapping learned, kept and ambiguous, simulation, old extension, `tabs` one/many, persistent once and again after a message, closed silent tab). Mutation checks: answer applied after the check, no suspension shift, persistent on one tab, no tolerance each make a test fail. - Unit tests for each rule above, including the 30 s boundary, late answers, suspension, crashes, reload, tab mapping and the `one`/`many`/persistent logic.
- Pixel definitions pass both validators; adapter tests check names, frequency and parameters. - Both platforms' pixel definition validators and the adapter tests for names, frequency and parameters.
- Package tests, validators, SwiftLint and `build-for-testing` on both platforms. App-hosted, signed: iOS (simulator) `TabTests` and `WebExtensionPixelFiringTests` 61 tests, macOS unit tests 16 (`MacOSWebExtensionPixelFiringTests`, `WebExtensionsDebugMenuTests`, `NavigationPixelNavigationResponderTests`) and the integration `CPMErrorPageMonitoringTests` 3 tests pass. - App-hosted macOS and iOS tests for the touched code, including an integration test that the browser's error page is not measured.
- The reporting cases above were run once as a throwaway test, which is not kept. Existing unit tests cover cases 1, 2, 4 to 13; cases 3, 14 and 16 have no test of their own; case 15 has two. - Not done: a real-WebKit run, and a check that the extension messages on every page counted as eligible (search result pages, `duck.ai`, non-document types).
- Manual: [[CPM Breakage Testing Steps]] with the Debug switch.
- Real WebKit (CPM repro harness, [PR #7378](https://github.com/duckduckgo/apple-browsers/pull/7378)): not run.
## Additional Considerations (if applicable) ## Additional Considerations
**Privacy:** two new pixel names with enumerated parameters; no URLs, identifiers or page content. Privacy Triage required, not requested yet. **Privacy:** two new pixel names with an enumerated parameter; no URLs, identifiers or page content. Privacy Triage is required and not requested yet.
**Security:** no new private WebKit selectors. **Security:** no new private WebKit selectors.
**Site Breakage:** none; the detector only observes events and recovery is unchanged. **Site Breakage:** none. The monitor only observes events; recovery is unchanged.
**Experimentation:** none. **Experimentation:** none.
**Operational:** no pixel expiry (permanent metric). Danger label for the removed pixels. Data checks: `metrics.pixels_aggregation` and `metrics._pixels_new` (`pixel_id = 'web'`). **Operational:** the work is a stack of PRs: removal of the old diagnostics and pixels ([#8054](https://github.com/duckduckgo/apple-browsers/pull/8054), on top of [#8025](https://github.com/duckduckgo/apple-browsers/pull/8025)), the metric, and the `last_stage` addition. The removal and the metric must ship in the same release, otherwise no CPM breakage pixel exists in between. The new definitions have no expiry date (permanent metric). The first persistent series cannot be compared with the old episode numbers directly: compare by app version against the previous derived number.
**Localization / Internationalization:** none. **Localization / Internationalization:** none.
work/projects/cpm-wake-init-detection-tech-design-2026-11-10.md
+46 -137
View File
@@ -14,173 +14,82 @@ status: draft
**Reviewer:** TBD (previous CPM pixel designs: Michal) **Reviewer:** TBD (previous CPM pixel designs: Michal)
**Stakeholders:** Michal, Konrad, Russell (from the previous Tech Designs; confirm). Extension change: the privacy-extension owners. **Stakeholders:** Michal, Konrad, Russell (from the previous Tech Designs; confirm). Extension change: the privacy-extension owners.
**Project:** [CPM: root-cause the stuck embedded extension messaging (macOS/iOS)](https://app.asana.com/1/137249556945/task/1216761517055116) **Project:** [CPM: root-cause the stuck embedded extension messaging (macOS/iOS)](https://app.asana.com/1/137249556945/task/1216761517055116)
**Task:** to create ("Extension lifecycle metrics", from the plan in [[cpm-project-summary-2026-11-03]]) **Task:** to create ("Extension lifecycle metrics")
**Previous Tech Designs:** [[cpm-readiness-telemetry-tech-design-2026-10-15]] (background readiness, removed), [[cpm-m3-telemetry-privacy-triage]] **Previous Tech Designs:** [CPM breakage pixels](https://app.asana.com/1/137249556945/task/1218005985662983), [CPM failure diagnostics](https://app.asana.com/1/137249556945/task/1218619301158188)
**Privacy Triage:** to request together with [[cpm-permanent-breakage-metric-tech-design-2026-11-10]] **Privacy Triage:** to request together with the CPM permanent breakage metric design, which this one extends
**Extension PR:** [duckduckgo-privacy-extension#3727](https://github.com/duckduckgo/duckduckgo-privacy-extension/pull/3727), branch `alex/cpm-init-progress` **Extension PR:** [duckduckgo-privacy-extension#3727](https://github.com/duckduckgo/duckduckgo-privacy-extension/pull/3727)
## Background & Requirements ## Background & Requirements
Failures after a background wake are the largest group that has no fix. On macOS 1.214 it is 39% of all failures and 36% of permanent ones (0.68% of DAU); on iOS 7.244, 22% and 15% (0.06% of DAU). The data excludes the usual causes: Failures after a background wake are the largest group with no fix: on macOS 1.214 they are 39% of all failures and 36% of permanent ones (0.68% of DAU); on iOS 7.244, 22% and 15% (0.06% of DAU). The usual causes are ruled out: the background is ready (under one second in 93% of macOS wake episodes), and a plain idle-unload and wake loses no messages in the real-WebKit harness.
- The background is ready: `bg_ready_after` was under one second in 93% of macOS wake episodes. The earlier readiness fix (P1) had no measurable effect on either platform. What the app cannot see is where the startup message stops. For one page the chain is: the page's script sends `init` → the background receives it → reads the configuration → reads the user setting → checks the site → answers the page and refreshes the dashboard in the app. The app sees only the last step, so every failure looks the same: no answer.
- A bare idle-unload and wake does not lose messages in the real-WebKit harness: 0 of 1,500 wake navigations in 1,000 cycles on a fixture page.
- A live view, a responsive process and a loaded context describe the container, not whether the init message was processed.
What the app cannot see is where the startup message stops. The chain for one main-frame document is: Requirements: tell where the chain stopped for a page that got no answer, and whether the page really received its answer; no change to what the extension does for the page; no URLs, page content or persistent identifiers collected; an older app must tolerate a newer extension and the other way round.
`content script sends init → message reaches the background listener → config → user setting → site check → initResp to the page + dashboard refresh to native`.
The app only sees the last step (dashboard refresh), and that step is a separate branch that does not prove the page received `initResp`. So today every one of these looks the same: no answer.
Requirements:
- Tell where the init chain stopped for a page that never got its answer.
- Know whether the page actually received `initResp`, not only whether the dashboard refreshed.
- No change in what the extension does for the page, and no extra work on the path that is suspected to be fragile beyond the marks themselves.
- Backward compatible in both directions: an older app must tolerate a newer extension, and a newer app an older extension.
## Problem Statement ## Problem Statement
For every tab reported silent by the permanent breakage metric, record the furthest init step the extension reported for its page. With that: For every tab the permanent breakage metric reports as silent, record the furthest init step the extension reported for its page, so the next fix is picked from data.
1. Separate "init never reached the background" from "background stalled on its own storage or native round trip" from "answer could not be delivered to the page".
2. Join the result with `background` (woken, warm) so wake failures are split from the rest.
3. Pick the next fix from data, not from the hypotheses list (H1–H5 and R1 in [[cpm-post-release-analysis-2026-10-27]]).
Collect no page content, URLs or persistent identifiers.
## Recommended Approach ## Recommended Approach
The extension reports **progress marks** for each main-frame init as fire-and-forget native notifications. The app keeps the furthest mark per tab and adds `last_stage` of the silent tab to `web_extension_cpm_breakage`. A mark is never an answer: only the dashboard refresh answers a page, as in the companion design. A report is made only when a tab is silent, so marks never create a report and `response_failed` does not report anything by itself. The extension sends a small fire-and-forget notification to the app at each step of init. The app keeps the furthest step per tab and adds it as `last_stage` to the `web_extension_cpm_breakage` pixel. A notification never counts as an answer: only the dashboard refresh does. Notifications never create a report; they only describe a tab that is already silent.
### Marks | `last_stage` | The chain stopped |
| Stage | Reported by | When | Rank |
|---|---|---|---|
| `page_init_sent` | content script, relayed by the background | the content script is about to send `init` | 0 |
| `received` | background | first thing when `init` arrives, before any await | 1 |
| `config_read` | background | remote config available | 2 |
| `settings_checked` | background | user setting read from native | 3 |
| `site_checked` | background | per-site check answered by native | 4 |
| `response_sent` | background | `tabs.sendMessage(initResp)` resolved | 5 |
| `response_failed` | background | the same call rejected | 6 |
Marks are best effort: they carry `{ tabId, url, stage }`, never wait and never reject, never write to the diagnostics error list, and cannot change CPM state or what the page gets. The native side keeps the highest rank seen for the tab's page; an out-of-order arrival never lowers it. A mark is tied to a tab the same way a dashboard refresh is (the learned extension tab identifier, or the one committed page its URL matches); a mark that cannot be tied to exactly one tab is dropped. A mark that arrives within 2 s before the commit is kept for the new page.
### Native side
- `AutoconsentWebExtensionMessageHandler` gets a `cpmInitProgress` method. Unknown stage names are ignored without an error, so a newer extension cannot make an older app answer with failures.
- `CPMMessagingHealthEvent.initProgress` carries the mark; the monitor stores the furthest stage per tab for the tab's current page only (a mark about another page is not counted) and keeps marks for 2 s so a mark that arrives just before the commit counts for the new page; the stage is reset at every commit, when the tab closes or crashes and on an extension reload.
- A mark does not answer a page and does not end a series. The breakage report takes `last_stage` from the tab it reports, not from any other tab.
- The Debug "Simulate CPM Messaging Breakage" switch hides marks as well as dashboard refreshes, and marks from an old embedded extension are ignored.
### Pixel change
`web_extension_cpm_breakage` (defined in [[cpm-permanent-breakage-metric-tech-design-2026-11-10]]) gains one parameter:
| Parameter | Values |
|---|---| |---|---|
| `last_stage` | `none` (no mark reached the app or none could be tied to the tab), `page_init_sent`, `received`, `config_read`, `settings_checked`, `site_checked`. `response_sent` can appear: the page got `initResp` but no dashboard refresh answered the tab. `response_failed` appears when delivering `initResp` failed. | | `none` | nothing reached the app (script not injected, or the message or notification was lost) |
| `page_init_sent` | the page's script sent `init`; the background never processed it |
| `received` | `init` arrived; the configuration or storage read did not finish |
| `config_read` | waiting for the user setting from the app |
| `settings_checked` | waiting for the site check |
| `site_checked` | all checks passed; no answer and no dashboard refresh arrived |
| `response_sent` | the answer was sent to the page, but no dashboard refresh followed |
| `response_failed` | sending the answer to the page failed |
`web_extension_cpm_breakage_persistent` is unchanged. No new pixel. Names, frequency and the other parameters are unchanged. Both platforms' definitions and `params_dictionary.json5` get the parameter. The app keeps only the highest step, so a late or out-of-order notification never lowers it, and only notifications for the tab's current page count. Unknown step names are ignored, so a newer extension cannot break an older app. The Debug "Simulate CPM Messaging Breakage" switch hides these notifications as well.
### How to read `last_stage` **Extension change.** Notifications are sent inside the existing `init` handling: before any wait for the first step, after each check for the next, and around the answer to the page. They never wait, never throw and never change what the page receives; a failure to answer is still reported as before.
The marks are relayed by the background, so a dead background cannot report. `none` therefore cannot tell a missing content script, a lost message to the background and a lost notification apart. **Compatibility.** The extension ships inside the app, so it reaches users with an app release after its PR merges. The app side can ship first: with no notifications, `last_stage` is `none`. A newer extension on an older app is expected to be harmless; this is to be confirmed against the real message router. The page's script is one bundle for every extension build, so `page_init_sent` also reaches builds that ignore it, at the cost of one extra message per page.
| `last_stage` on a breakage | What it says | Next action | ## Pros
|---|---|---|
| `none` | nothing reached native: content script not injected, `init` and the mark lost on the way to the background, or the background cannot notify native | check injection (`tab_has_extension_user_scripts` is gone); WebKit-side loss on wake (R1, H2) |
| `page_init_sent` | the content script sent; the background never processed `init` | message lost between content script and worker listener after a wake: content-script retry of `init`, WebKit bug report |
| `received` | `init` arrived; remote config or session storage did not finish | H1 (unbounded config/storage reads, shared state queue): bound the awaits with ordering-safe semantics |
| `config_read` | stuck on the user setting round trip to native | native round-trip stall, H5: answer from cached config first |
| `settings_checked` | stuck on the site check | same, per-site call |
| `site_checked` | all checks passed, but neither `initResp` nor a dashboard refresh arrived | the page's tab is gone or unreachable (tab not connected) |
| `response_failed` | `tabs.sendMessage` rejected | tab not found or content script gone: the page's context changed under the answer |
A site or a setting that disables CPM legitimately ends the chain at `settings_checked` or `site_checked`, but then the extension still sends the dashboard refresh, so those pages are healthy and never appear here. - Turns "no answer" into a position in a known chain, which maps to the open hypotheses and to concrete fixes.
- Shows cases the dashboard refresh hides, such as an answer that was sent but not received.
### Extension change - Out of the page's way: no waits, no state changes.
All changes are in `duckduckgo-privacy-extension`, branch `alex/cpm-init-progress`. Commit `b00cc227` (pushed, PR #3727) has `received`, `response_sent` and `response_failed`; the remaining marks are an uncommitted working-tree change that has not been tested.
| File | Change |
|---|---|
| `shared/js/background/components/cpm-embedded-messaging.js` | `notifyInitProgress(tabId, url, stage)`: `nativeMessaging.notify('cpmInitProgress', …)` inside the existing timeout helper; catches every error; never records a diagnostics error. |
| `shared/js/background/components/cookie-prompt-management.js` | `received` at the top of the main-frame `init` branch, before any await; `config_read` after the config is available; `settings_checked` after `checkAutoconsentSetting`; `site_checked` after `checkAutoconsentEnabledForSite`; `notifyInitResponseProgress` wraps `tabs.sendMessage(initResp)` to report `response_sent` or `response_failed`, and rethrows so the failure is still an unhandled rejection reported as `extensionException`. A page-side `cpmInitProgress` message is accepted only for the main frame and only for `page_init_sent`; unknown stages and sub-frames are ignored. |
| `shared/js/cpm.js` | `reportInitProgress('page_init_sent')` right before the content script sends `init`; wrapped in try/catch, never awaited, never delays init. |
| `unit-test/background/cookie-prompt-management.js` | order of marks, no await before `received`, `received` before the state reset, dispatch when the remote config is unavailable, a site that disables CPM ends at `site_checked`, `response_failed` keeps the rejection unhandled, sub-frame and unknown-stage marks ignored. |
Compatibility and sequencing:
- The embedded extension ships inside the app bundle, so the extension change reaches users only with an app release after the extension PR merges and the embedded files are updated. The native handler can ship first: it ignores a missing mark, and `last_stage=none` is the value for an app that has no marks yet.
- Older app, newer extension: the handler ignores unknown stages. An app without the `cpmInitProgress` handler is expected to answer with an error that `notifyInitProgress` swallows; to confirm against the real router before merge.
- `cpm.js` is one bundle for every build (`shared/js/cpm.js` goes to the Chromium, Chromium-embedded and embedded builds). `page_init_sent` therefore also reaches the desktop-extension background, where `notifyInitProgress` is absent and the message is dropped by an early return. Cost: one extra `runtime.sendMessage` per main-frame document in those builds. To confirm before merge: that this does not wake a sleeping MV3 worker earlier than the real `init` would, and whether the message should be limited to the embedded build.
- Rollout is by release channel: internal builds carry the new extension first, public builds follow. No remote switch inside the extension: `received` is sent before the config is read, so the config cannot gate it.
### Pros
- Turns "no answer" into a position in a known chain, which maps directly to the hypotheses and to concrete fixes.
- `response_sent` and `response_failed` show cases the dashboard refresh hides (the page did not get `initResp` although the refresh arrived); a tab is still reported only when the refresh is also missing.
- Stays out of the page's way: fire-and-forget, no waits, no state changes.
- Both sides tolerate the other being older. - Both sides tolerate the other being older.
### Cons ## Cons and risks
- Marks pass through the same background whose failure we study, so the earliest failures collapse into `none`. - Notifications travel through the same background whose failure we study, so the earliest failures end up as `none`.
- Six notifications per main-frame document (five stages before the answer, then `response_sent` or `response_failed`), on top of the native round trips init already makes. The load on the path suspected to be fragile is not zero; its size has not been measured. - Up to seven extra notifications per page on a path suspected to be fragile. The extra load has not been measured. Mitigation: ship to internal builds first, compare breakage rates and the `last_stage` mix with and without notifications, and drop `config_read` and `site_checked` if the cost shows.
- The extension part reaches users only with an app release; the first data comes about two release cycles after the work is done (data in 1–2 release cycles, per the plan). - `response_sent` can be misleading: WebKit may report a send as done when no one was listening. It never answers a page, so the metric is not affected, only how `last_stage=response_sent` is read.
- A change to the page-visible content script (an extra message) on every build. - The extension part reaches users only with an app release, so the first data comes about two release cycles after the work.
- A page cannot forge notifications: the script runs in an isolated world, and the background accepts a page-side notification only for `page_init_sent` and only from the main frame.
### Risks
- **The marks change the behavior they measure** (a heisenbug). Mitigation: internal builds first, compare breakage rates and the `last_stage` mix between releases with and without marks; if the cost is visible, drop `config_read` and `site_checked` in the next extension release.
- **`response_sent` can be a false positive.** If `tabs.sendMessage` resolves without a listener (WebKit can resolve `undefined` for a message with no receiver), a page without the content script would show `response_sent`. It never answers a page, so the metric is not affected; only the reading of `last_stage=response_sent` is. A real-WebKit test with no listener should settle it.
- **Marks arrive out of order or late.** Handled by rank; a tab reported before its last mark arrives carries an earlier `last_stage`. Marks arriving after the report are ignored.
- **A mark for a tab that is not mapped yet** is dropped when its URL matches two tabs (a mark before the first commit is kept for 2 s and counted for the page that commits), so `none` also covers marks lost to attribution.
- **A hostile page forges marks.** The content script runs in an isolated world; the page cannot call `chrome.runtime`. The background accepts a page-side mark only for `page_init_sent` and only from the main frame.
## Testing ## Testing
- Extension: `make unit-test` (the cookie-prompt-management suite), `npm run tsc`, `npm run lint`. The uncommitted part has not been run. - App: the pixel carries the furthest step of the silent tab and not of another tab; no notifications give `none`; a notification is not an answer; unknown steps are ignored; stale notifications are forgotten at the next page; the parameter appears in both platforms' pixel definitions.
- Native unit tests (PR 4, package suite 1073 tests): the report carries the furthest stage of the silent tab and not of another tab; no marks give `none`; a mark is not an answer; unknown stage ignored by the handler; a mark that cannot be tied to one tab dropped; a mark just before the commit belongs to the new page and an older one is forgotten; old extension and simulation ignored; closed tab forgets its marks; `last_stage` in the pixel on both platforms. Mutation checks (a mark answers, rank lowers, stage kept forever, stage always cleared) each fail a test. App-hosted tests were not run. - Extension: the existing unit suite for the cookie prompt management, extended for the order of notifications, a site that disables CPM ending at `site_checked`, and sub-frame and unknown-step notifications being ignored. Part of the extension change has not been run yet.
- Real WebKit (CPM repro harness, [PR #7378](https://github.com/duckduckgo/apple-browsers/pull/7378)): the full chain shows every mark in order on a healthy page; a page without the content script yields no `response_sent`; an injected stall at each await shows the matching `last_stage`; a delay on the native round trips for `config_read` and `settings_checked`. These runs have not been done. - Not done: a real-WebKit run showing every step in order on a healthy page, the matching `last_stage` for an injected stall at each step, and a page without the script not yielding `response_sent`.
- Pixel definitions pass both platforms' validators in CI.
- Privacy check of real payloads: the marks contain `url` on the extension-to-app channel only; the pixel carries the enum only.
## Additional Considerations (if applicable) ## Additional Considerations
**Privacy** **Privacy:** one new enumerated parameter on a pixel that is new itself. The notifications carry the tab number and the page URL from the extension to the app, inside the app (the dashboard refresh already does); neither enters a pixel or a log. Privacy Triage is to be requested once for both designs.
One new enumerated parameter on a pixel that is itself new in [[cpm-permanent-breakage-metric-tech-design-2026-11-10]]. The marks carry the tab ID and the URL from the extension to the app inside the app (the dashboard refresh already does), but neither enters a pixel or a log (the monitor's log descriptions omit URLs). Privacy Triage to be requested once for both designs. Not requested yet. **Security:** no new private WebKit selectors. The new message is read-only: the app parses a step name and a tab and answers with nothing.
**Security** **Site Breakage:** no intended change for pages.
No new private WebKit selectors. A new native message method, `cpmInitProgress`, is read-only: it parses a stage name and a tab, never executes content and never answers with data. Page-origin spoofing is covered in Risks. **Experimentation:** none. Rollout is by release channel, internal builds first.
**Site Breakage** **Operational:** the app side lands first, the extension after its review and the usual update of the embedded files. After the first public release with notifications, look at `last_stage` by platform and wake state over the breakage count and pick the fix from the table above. The notifications and `last_stage` are investigation tools: remove them when the wake group has a fix or is closed. Not in scope: per-step durations and the background's own lifecycle metrics.
No intended page-behavior change. The only extension behavior change is the notifications; the `initResp` call keeps its arguments and its rejection behavior (`notifyInitResponseProgress` rethrows). **Localization / Internationalization:** none.
**Experimentation** ## Open Questions
None. The comparison is across releases and channels (see the heisenbug risk). 1. Seven notifications per page or fewer: are `config_read` and `site_checked` worth it for the first read?
2. Send `page_init_sent` only in the embedded build, or accept the extra message everywhere?
**Operational** 3. Is a switch inside the extension needed, or is release-channel rollout enough?
4. Who reviews the extension change, and can the update of the embedded files be timed with the app release that carries the app side?
- Order of landing: native handler and `last_stage` (PR 4, `alex/cpm-init-marks`, `afa2bf2289`, on top of PR 3 `eea85b0812`) can ship before the extension; the extension PR merges after review and enters the app through the usual embedded-files update.
- After the first public release with marks: `last_stage × background × platform` over the breakage count, and the share of breakages by stage, to pick the fix according to the table above. Data checks as in the companion design.
- The marks and the parameter are temporary investigation tools. Remove `config_read`, `settings_checked`, `site_checked` and `last_stage` when the wake group has a fix or is closed (project plan: close the project and reset the target if the new metrics show nothing actionable). The definitions have no expiry date, so the removal is a follow-up PR.
- Not in scope: per-stage durations (a time bucket between marks), the background's own lifecycle metrics, and the idle-unload distance. They are the next candidates if `last_stage` leaves the wake group unexplained.
**Localization / Internationalization**
None.
## Open questions
1. Six marks per page or fewer: is `config_read` and `site_checked` worth the extra notifications, or are `received`, `settings_checked` and the response enough for the first read?
2. Limit `page_init_sent` to the embedded build, or accept the extra message on every build?
3. Is an extension-side switch needed (a remote-config setting read before sending `received`), or is release-channel rollout enough?
4. Who owns the extension review, and can the embedded-files update be timed to the app release that carries the native handler?