diff --git a/work/projects/cpm-permanent-breakage-metric-tech-design-2026-11-10.md b/work/projects/cpm-permanent-breakage-metric-tech-design-2026-11-10.md new file mode 100644 index 00000000..428f9150 --- /dev/null +++ b/work/projects/cpm-permanent-breakage-metric-tech-design-2026-11-10.md @@ -0,0 +1,212 @@ +--- +tags: + - work + - cpm + - tech-design + - pixel +updated: '2026-11-10' +status: draft +--- +# TD: Pixel Definition: CPM Permanent Breakage Metric (macOS/iOS) + +> Internal metrics, keep out of public PRs. Companion: [[cpm-wake-init-detection-tech-design-2026-11-10]] (adds `last_stage` to the pixel defined here). Where we are: [[cpm-project-summary-2026-11-03]] (section "Состояние на 10 ноября"). Review of the earlier attempt that this design replaces: same section. + +**Author:** Alex M +**Reviewer:** TBD (previous CPM pixel designs: Michal) +**Stakeholders:** Michal, Konrad, Russell (from the previous Tech Designs; confirm) +**Project:** [CPM: root-cause the stuck embedded extension messaging (macOS/iOS)](https://app.asana.com/1/137249556945/task/1216761517055116) +**Task:** to create ("Permanent breakage metrics", from the plan in [[cpm-project-summary-2026-11-03]]) +**Previous Tech Designs:** [[cpm-readiness-telemetry-tech-design-2026-10-15]], [[cpm-m3-telemetry-privacy-triage]] (both describe pixels this design supersedes) +**Privacy Triage:** to request (new pixel names, see Additional Considerations) + +## Background & Requirements + +The project goal is permanent CPM failures below 1% of DAU (0.2–0.5% counts as success). Today that number does not exist as a pixel. It is derived from `web_extension_cpm_episode` with a long SQL: a four-second navigation timeout opens an episode, a later timeout or reload confirms it, a five-minute deadline or a response closes it. Three problems: + +- **The timer measures slowness, not breakage.** On iOS 7.244, 6.0% of episodes closed with a late answer on the same page, 42% of those later than 30 s. A page that answers after 40 s is counted as a failure. +- **Episodes are lost.** An unfinished episode lives in memory; closing or killing the app drops it. The pages that stay broken longest are the ones that end with the app. +- **The pixel carries what the investigation needed** (diagnostic booleans, `bg_events`, load fields) and is being trimmed to five parameters (PR 2 in the stack below). What remains cannot be turned into a small, fixed dashboard set. + +Requirements: + +- A small fixed set of metrics, as rates against DAU, for the health indicators dashboard. +- Detection that does not depend on a timer and does not depend on reaching a deadline before the app exits. +- No page content, URLs or persistent identifiers. +- Runs next to the episode pixel for one release, so the two series can be compared before the old one is removed. + +## Problem Statement + +Define and measure, per platform and version: + +1. **Breakage:** an http(s) page that finished loading with the extension loaded, and never got the extension's answer while its document lived. +2. **Persistent breakage:** several documents in a row with no answer in between, meaning the extension is broken for the user, not for one page. +3. How many installs per day are affected, and how many breakages happen. + +The wake investigation (which init step stops) is separate: [[cpm-wake-init-detection-tech-design-2026-11-10]]. + +## Recommended Approach + +A page is **healthy** as soon as an answer for its document arrives, at any time while the document lives (a late answer is not a failure). A page that never got the answer is **judged when its document ends**. There are no navigation timers. The approach is additive: the existing monitor, its timers and the episode pixel stay unchanged; the new logic is a second consumer of the same events. + +### What was wrong with the first attempt + +The first attempt (branch `alex/cpm-breakage-pixel`, commit `e6419bb910`) already used page-end judging, but combined it with deleting the episode pixel and had these defects, all addressed below: + +1. Unanswered measurements older than five minutes were dropped, so the longest-silent pages were never reported. +2. No minimum lifetime: a page left 200 ms after loading counted as broken. +3. A web content process crash counted as extension breakage. +4. `background` was derived from "background load took at least one second" and from one app-wide signal, not from the page's navigation. +5. Pages alive when the app exits were never judged. +6. The macOS pixel definition omitted `pixelSource` and `channel` and used iOS-only suffixes. +7. `navigationWaitContexts` leaked an entry when the context was replaced, and `cpmStage` parsing had no test. + +### Detection rules + +Per eligible document: http(s), not a browser error page, not a history restore from cache (existing eligibility rules, unchanged). + +| Event | Effect | +|---|---| +| Navigation finished, extension loaded | Document becomes **judgeable** and its clock starts. If the extension is not loaded yet, it becomes judgeable when the extension loads while the page lives. | +| Attributed dashboard response (later also `response_sent`, see the companion design) | Healthy. Ends judging for this document. Resets the streak. | +| Response that cannot be tied to one document | The document is not judgeable (neither outcome provable). | +| Document ends by a new navigation or tab close | Judge now. | +| App suspends (iOS to background, macOS sleep) or macOS terminates | Judge every pending judgeable document now, once. The processes still run at this moment; nothing is judged while suspended. | +| Web content process terminated | **Not judged.** The silence is the crash's, not the extension's. The tab's next navigation is reported with `reason=tab_crash`. | +| Extension signals `cpmStage` = `config_unavailable` or `settings_missing` | Breakage at once (`ended_by=explicit_signal`), no lifetime limit. | +| Extension reload | Existing rule: evidence from the replaced context is discarded. | + +A document is reported only if it lived at least **30 s** after it became judgeable (`minimum lifetime`). A shorter silence is ambiguous: it may be a slow wake. The 30 s value follows WebKit's unload timer and the late-answer data above; it is a constant in the first release. Shorter and longer lifetimes are told apart in analysis by the `lifetime` parameter. + +A judged document is never judged again and never retracted: an answer after the report changes nothing. Documents whose process or app ends without a suspension event (kill, crash of the app) are lost; this is the known bias, now limited to the time between the last suspension and the kill. + +Measurement state does not expire after five minutes. The old five-minute lifetime stays only for the restart observation, which is unrelated. + +### Persistence + +The monitor keeps a streak: the number of consecutive judged-broken documents since the last healthy one (any tab). When it reaches **3** the persistent pixel fires once for that streak. A healthy answer resets the streak. The streak lives in memory for the process lifetime. A relaunch resets it (see open questions). + +`tabs` on the persistent pixel records whether the streak's documents were in one tab or several. This keeps the cross-tab distinction the earlier analysis found important (cross-tab permanent was the figure used against the 1% target). + +### Pixels + +| Pixel | Trigger | Frequency | +|---|---|---| +| `web_extension_cpm_breakage` | A document is judged broken. Sent at that moment, once per document. | Daily + count (`dailyAndCount`) | +| `web_extension_cpm_breakage_persistent` | A streak reaches 3 documents. Once per streak. | Daily + count (`dailyAndCount`) | + +`_daily` estimates installs per day, `_count` counts events. Both metadata sets follow the existing episode pixel on each platform: `appVersion` on both; macOS also `pixelSource` and `channel`, suffix `daily_count`; iOS keeps its existing suffix set. The macOS definition must match what is sent (item 6 above). + +`web_extension_cpm_breakage` parameters: + +| Parameter | Values | +|---|---| +| `reason` | `session_restoration`, `tab_crash`, `extension_reload`, `other`. Context of the document's navigation. Same values as the episode pixel. | +| `background` | `woken` (the extension's background view was created during this navigation), `warm` (a background view existed and none was created), `unknown` (no background view known). Comes from the manager's `didCreateBackgroundWebView` delegate callback, correlated to the navigation, not from load duration. | +| `ended_by` | `navigation`, `tab_closed`, `app_suspended`, `explicit_signal`. | +| `lifetime` | Time from judgeable to judged: `30s_to_2m`, `2m_to_10m`, `over_10m`. Omitted for `explicit_signal`. | +| `streak` | Breakages in a row including this one: `1`, `2`, `3_plus`. | + +`web_extension_cpm_breakage_persistent` parameters: + +| Parameter | Values | +|---|---| +| `tabs` | `one` or `many`: whether the streak's documents were in one tab or several. | + +The `last_stage` parameter for wake diagnosis is added by [[cpm-wake-init-detection-tech-design-2026-11-10]], not here. No URL, document, tab or process identifier is sent. + +### The metric set for the dashboard + +| Metric | Source | Meaning | +|---|---|---| +| Installs with persistent breakage / DAU | `breakage_persistent_daily`, split by `tabs` | The success metric (target below 1%). | +| Installs with any breakage / DAU | `breakage_daily` | Upper bound; includes single-page failures. | +| Breakages per day | `breakage_count`, split by `reason`, `background`, `ended_by` | Volume and mix, for diagnosis. | + +Everything is per platform and app version. Rates use the existing DAU series. + +### Where it lives in the code + +- `CPMMessagingHealthMonitor` (existing, per-tab document state already tracks response, judgeable and crash state) gains a per-document record independent of the timeout measurement: when it became judgeable, whether it was judged, the navigation kind. Judging and the streak are small additions to existing handlers; the correlation logic is reused, not duplicated. +- New event `CPMMessagingHealthEvent.backgroundViewCreated`, sent by the manager from `didCreateBackgroundWebView`. It replaces `backgroundLoaded(wasWoken:)` from the first attempt. +- macOS also sends the existing suspension event on termination (`applicationWillTerminate`); iOS and macOS sleep already send `interruptionBegan`. +- `AutoconsentWebExtensionMessageHandler` already separates the `cpmStage` signals; they get a test. +- `CPMMessagingBreakageReport` and the two pixel events in `WebExtensionPixelFiring`. Definitions in both platforms' `web_extensions.json5` and `params_dictionary.json5`. + +### PR stack + +| PR | Branch | Content | State, 10 Nov | +|---|---|---|---| +| 1 | `alex/cpm-episode-metrics`, [#8025](https://github.com/duckduckgo/apple-browsers/pull/8025) | Remove ineffective recovery fixes, their flags and pixels | pushed, in review | +| 2 | `alex/cpm-pixel-cleanup` (`f04c93730f`) | Remove the diagnostics, trim the episode pixel to five parameters. No new pixel, no detection change. Needs the Danger label (existing pixel parameters change). | local; 1104 package tests pass, SwiftLint clean on the changed core files; app builds and pixel validation not run; not pushed | +| 3 | new, on top of 2 | This design: two pixels, document-end judging, streak | not started | +| 4 | new, on top of 3 | Init progress marks, `last_stage` ([[cpm-wake-init-detection-tech-design-2026-11-10]]) plus extension [#3727](https://github.com/duckduckgo/duckduckgo-privacy-extension/pull/3727) | native part drafted in `alex/cpm-init-diagnostics` (`adb28e643d`), needs the same fixes | +| 5 | later | Remove the episode pixel and its timers and states, one release after PR 3 | after comparison | + +Old branches `alex/cpm-breakage-pixel` and `alex/cpm-init-diagnostics` stay as sources only. + +### Pros + +- Measures the user-facing fact (this page never got an answer), not an elapsed time. +- A late answer is never a failure, which removes the 6% late-answer error class and its tail. +- Detected breakages are sent at once, so closing the app loses only documents that were never judged. +- Persistent breakage has its own daily pixel: the success metric is one series, not a join. +- Additive: the episode pixel and the recovery logic keep working, so the new numbers can be validated against the old. + +### Cons + +- Pages that live until a kill are not judged. Persistent breakage that a user never ends (leaves the page open for hours) appears only at suspension. +- 30 s minimum lifetime hides breakage on short pages. If most breakage is on short pages the metric underestimates; `lifetime` shows how much sits near the limit. +- A page-based streak does not equal the old "no answer for five minutes". The two series will differ, and the comparison needs interpretation. +- The streak resets on relaunch; iOS terminates apps often, so persistent breakage across launches is undercounted. + +### Risks + +- **Pixel volume.** A user with persistent breakage fires one count request per page. Mitigation: soft cap of 20 `breakage` count requests per process; the persistent pixel is unaffected by the cap. Volume to be estimated from `web_extension_cpm_episode` counts before release. +- **Healthy evidence that is missing in a working page** (dashboard refresh not delivered, wrong attribution). Counted as breakage. The earlier analysis found `other_tab_response` at 37–53% of outcomes, an attribution problem. Mitigation: documents with an unattributable response are not judged, and `unknown` attribution is visible only locally for now. Compare with the episode series. +- **Judging at suspension** assumes silence for 30 s before suspension means breakage. A wake that began just before suspension could be reported. Mitigation: the lifetime limit; the suspension case is separated by `ended_by`. +- **Simulation switch.** The Debug "Simulate CPM Messaging Breakage" switch hides dashboard responses from the monitor and must keep doing so for both detectors. + +## Testing + +- Unit tests for each rule in the table: late answer before document end is healthy; unanswered document shorter than 30 s is not reported and does not touch the streak; unanswered longer document is reported on navigation, tab close, suspension; process termination is not judged and tags the next navigation; judged once; extension not loaded at finish becomes judgeable on load; unattributable response; explicit signals; streak reaches 3 once, resets on a healthy answer; `tabs` value; `background` from the view-created event inside and outside the navigation window; simulation switch hides answers; extension reload evidence. +- Handler test for `cpmStage` parsing; test for `navigationWaitContexts` after a context replacement. +- Pixel definitions: schema validators of both platforms in CI; parameter lists match what is sent (compare the macOS definition against the event). +- Manual: updated steps in [[CPM Breakage Testing Steps]] (Debug simulation switch: load a page for 35 s without an answer, navigate away, expect one `breakage`; three in a row, expect `persistent`). +- Real WebKit: the CPM repro harness ([PR #7378](https://github.com/duckduckgo/apple-browsers/pull/7378)) with the injected error 6 and idle-wake fixtures to see both pixels with the expected values. Not run yet. + +## Additional Considerations (if applicable) + +**Privacy** + +Two new pixel names with enumerated, bucketed parameters. No URLs, page content, tab or document identifiers, or process IDs. The streak is a local integer that is never sent; only `streak` as a bucket. Because the pixel names are new, Privacy Triage is required before release. Not requested yet. Combined distinguishability of `reason`, `background`, `ended_by`, `lifetime`, `streak` and the platform metadata is small (five low-cardinality enums). + +**Security** + +No new private WebKit selectors. `didCreateBackgroundWebView` is already implemented in the manager. + +**Site Breakage** + +None intended. The detector only observes events; it sends no messages and changes no recovery behavior. Existing recovery (error-6 restart, network recovery, upgrade before load) is unchanged. + +**Experimentation** + +No experiment. The comparison is observational: the new and old series for the same versions during the overlap release. + +**Operational** + +- Overlap: ship PR 3 next to the episode pixel for one release; compare "installs with persistent breakage / DAU" with the existing permanent-failure series by platform. Expected: new is lower where late answers dominated, higher where the episode never reached its deadline. Remove the episode pixel (PR 5) when the difference is explained. +- Pixel expiry follows the existing definitions (2027-01-15). +- Add the three metrics to the health indicators dashboard. Data checks use ClickHouse: pixel counts from `metrics.pixels_aggregation` (`pixel_id = 'web'`), parameters from `metrics._pixels_new` (`pixel_id = 'web'`, one day). +- No remote kill switch is proposed. If volume surprises, the soft cap and the next release are the lever. **Question for reviewers:** is a feature flag required for a new pixel here? + +**Localization / Internationalization** + +None. + +## Open questions + +1. Minimum lifetime 30 s and streak 3: acceptable as first constants, or tune from the episode data first? +2. Persist the streak across launches (a local counter)? It makes persistent breakage visible across iOS terminations at the price of local state. +3. Flag or no flag for a new pixel. +4. Judge at suspension: accept, or only at document end? +5. Soft cap value. diff --git a/work/projects/cpm-project-summary-2026-11-03.md b/work/projects/cpm-project-summary-2026-11-03.md index a996f6f3..92c81c7f 100644 --- a/work/projects/cpm-project-summary-2026-11-03.md +++ b/work/projects/cpm-project-summary-2026-11-03.md @@ -101,3 +101,16 @@ Open: **Решения по данным (ClickHouse, 8–9 ноября, счёт `count`).** P2 убран: пикселя результата нет (115 тысяч повторов в день на iOS), при оставшемся view повтор не завершается (фон это service worker). Restart 9 ноября: iOS 7.245 118 перезапусков, 81% ответов страницы; macOS 1.215 54 перезапуска, 65%, выборка маленькая, контроля нет. Network recovery оставлен: iOS 388 тысяч срабатываний в день, 59% числа эпизодов, эффект не измерен (`network_restarted` убрали из отчёта в том же релизе), измерять выключением удалённого флага. Таймер 4 с: на iOS 7.244 6,0% эпизодов закрылись поздним ответом на ту же страницу, 42% из них позже 30 с, 19,4% без ответа за 5 минут. Доступа к статусу расширения из приложения нет (в SDK нет доступа к миру расширения, `evaluateJavaScript` на фоне не достаёт до воркера); порт или heartbeat как проверка здоровья не годятся (открытый порт откладывает выгрузку фона). **Данные из сессии.** `~/Developer/grafana-data-tools/` и запуск SQL через Grafana с SSO через системный Chrome (`grafana-auth --channel chrome`). Счёт пикселей: `metrics.pixels_aggregation` (`pixel_id = 'web'`); параметры: `metrics._pixels_new` только с `pixel_id = 'web'` и одним днём. + + +## 10 ноября, вечер: разрез PR и tech design + +**Разрез.** Ветка `alex/cpm-breakage-pixel` (`e6419bb910`) смешивала чистку с новым пикселем и удаляла эпизодный пиксель сразу, против плана. Новая раскладка: PR 1 [#8025](https://github.com/duckduckgo/apple-browsers/pull/8025) без изменений. **PR 2** `alex/cpm-pixel-cleanup` (`f04c93730f`, поверх `6928847174`) уже содержит чистую чистку по плану: удалены recorder, `CPMMessagingDiagnostics`, delegate proxy, readiness observer и два флага с проводкой и пунктом debug-меню macOS; эпизодный пиксель урезан до `reason`, `outcome`, `tabs_affected`, `repeat_failures`, `reload_state`, без нового пикселя и без изменения таймеров. Проверено: `swift test` пакета WebExtensions 1104 теста, 0 падений; SwiftLint strict без замечаний на `CPMMessagingHealthMonitor`, `WebExtensionPixelFiring`, `WebExtensionManager`. Не проверено: сборка приложений iOS и macOS, валидатор определений пикселей (CI). Не запушена, PR не создан (нужна ссылка на задачу Asana и ревьюер; метка Danger, потому что параметры существующего пикселя меняются). **PR 3** (перманентный пиксель) и **PR 4** (метки init и `last_stage`, плюс расширение) ещё не начаты, ждут ревью tech design. Старые `alex/cpm-breakage-pixel` и `alex/cpm-init-diagnostics` остаются только источником. + +**Что неверно в `e6419bb910`** (проверено по коду): тихие замеры старше 5 минут сбрасываются (`discardExpiredState`); нет минимального времени жизни страницы; падение процесса вкладки засчитывается поломкой; `background` берётся из «загрузка ≥ 1 с» и одного общего сигнала; страницы, живущие до выхода приложения, не оцениваются; определение macOS без `pixelSource` и `channel`, суффиксы от iOS; утечка `navigationWaitContexts`; нет теста на разбор `cpmStage`. + +**Tech design (черновики, ревьюер не назначен):** +- [[cpm-permanent-breakage-metric-tech-design-2026-11-10]]: два пикселя (`web_extension_cpm_breakage`, `web_extension_cpm_breakage_persistent`), оценка страницы по концу документа, минимум 30 с жизни, серия из 3 подряд, суждение при приостановке приложения, `background` по `didCreateBackgroundWebView`. Работает рядом с эпизодным пикселем один релиз. +- [[cpm-wake-init-detection-tech-design-2026-11-10]]: метки `cpmInitProgress` из расширения, `last_stage` в пикселе, `response_sent` как здоровый сигнал, изменение расширения (PR #3727, ветка `alex/cpm-init-progress`; метки `page_init_sent`, `config_read`, `settings_checked`, `site_checked` в рабочем дереве без коммита и без запуска тестов). + +Открытые вопросы для ревью: порог 30 с и серия 3; сохранять серию между запусками; считать ли `response_sent` здоровым до проверки без слушателя; шесть меток или меньше; нужен ли флаг для нового пикселя. diff --git a/work/projects/cpm-wake-init-detection-tech-design-2026-11-10.md b/work/projects/cpm-wake-init-detection-tech-design-2026-11-10.md new file mode 100644 index 00000000..772aa4a9 --- /dev/null +++ b/work/projects/cpm-wake-init-detection-tech-design-2026-11-10.md @@ -0,0 +1,188 @@ +--- +tags: + - work + - cpm + - tech-design + - pixel + - web-extension +updated: '2026-11-10' +status: draft +--- +# TD: Pixel Definition: CPM Wake Init Detection (macOS/iOS + embedded web extension) + +> Internal metrics, keep out of public PRs. Builds on [[cpm-permanent-breakage-metric-tech-design-2026-11-10]] (the pixel this design extends). Where we are: [[cpm-project-summary-2026-11-03]]. Wake background and what was ruled out: [[cpm-ios-idle-wake-investigation-2026-10-30]], [[cpm-post-release-analysis-2026-10-27]]. + +**Author:** Alex M +**Reviewer:** TBD (previous CPM pixel designs: Michal) +**Stakeholders:** Michal, Konrad, Russell (from the previous Tech Designs; confirm). Extension change: the privacy-extension owners. +**Project:** [CPM: root-cause the stuck embedded extension messaging (macOS/iOS)](https://app.asana.com/1/137249556945/task/1216761517055116) +**Task:** to create ("Extension lifecycle metrics", from the plan in [[cpm-project-summary-2026-11-03]]) +**Previous Tech Designs:** [[cpm-readiness-telemetry-tech-design-2026-10-15]] (background readiness, removed), [[cpm-m3-telemetry-privacy-triage]] +**Privacy Triage:** to request together with [[cpm-permanent-breakage-metric-tech-design-2026-11-10]] +**Extension PR:** [duckduckgo-privacy-extension#3727](https://github.com/duckduckgo/duckduckgo-privacy-extension/pull/3727), branch `alex/cpm-init-progress` + +## Background & Requirements + +Failures after a background wake are the largest group that has no fix. On macOS 1.214 it is 39% of all failures and 36% of permanent ones (0.68% of DAU); on iOS 7.244, 22% and 15% (0.06% of DAU). The data excludes the usual causes: + +- The background is ready: `bg_ready_after` was under one second in 93% of macOS wake episodes. The earlier readiness fix (P1) had no measurable effect on either platform. +- A bare idle-unload and wake does not lose messages in the real-WebKit harness: 0 of 1,500 wake navigations in 1,000 cycles on a fixture page. +- A live view, a responsive process and a loaded context describe the container, not whether the init message was processed. + +What the app cannot see is where the startup message stops. The chain for one main-frame document is: + +`content script sends init → message reaches the background listener → config → user setting → site check → initResp to the page + dashboard refresh to native`. + +The app only sees the last step (dashboard refresh), and that step is a separate branch that does not prove the page received `initResp`. So today every one of these looks the same: no answer. + +Requirements: + +- Tell where the init chain stopped for a page that never got its answer. +- Know whether the page actually received `initResp`, not only whether the dashboard refreshed. +- No change in what the extension does for the page, and no extra work on the path that is suspected to be fragile beyond the marks themselves. +- Backward compatible in both directions: an older app must tolerate a newer extension, and a newer app an older extension. + +## Problem Statement + +For every document judged broken by the permanent breakage metric, record the furthest init step the extension reported for it, and use `initResp` delivery as healthy evidence. With that: + +1. Separate "init never reached the background" from "background stalled on its own storage or native round trip" from "answer could not be delivered to the page". +2. Join the result with `background` (woken, warm) so wake failures are split from the rest. +3. Pick the next fix from data, not from the hypotheses list (H1–H5 and R1 in [[cpm-post-release-analysis-2026-10-27]]). + +Collect no page content, URLs or persistent identifiers. + +## Recommended Approach + +The extension reports **progress marks** for each main-frame init as fire-and-forget native notifications. The app keeps the furthest mark per document, treats `response_sent` as healthy evidence, treats `response_failed` as an immediate breakage, and adds `last_stage` to `web_extension_cpm_breakage`. + +### Marks + +| Stage | Reported by | When | Rank | +|---|---|---|---| +| `page_init_sent` | content script, relayed by the background | the content script is about to send `init` | 0 | +| `received` | background | first thing when `init` arrives, before any await | 1 | +| `config_read` | background | remote config available | 2 | +| `settings_checked` | background | user setting read from native | 3 | +| `site_checked` | background | per-site check answered by native | 4 | +| `response_sent` | background | `tabs.sendMessage(initResp)` resolved | 5 | +| `response_failed` | background | the same call rejected | 6 | + +Marks are best effort: they carry `{ tabId, url, stage }`, never wait and never reject, never write to the diagnostics error list, and cannot change CPM state or what the page gets. The native side keeps the highest rank seen for the document; an out-of-order arrival never lowers it. Marks about a document the monitor cannot tie to exactly one tab are dropped. + +### Native side + +- `AutoconsentWebExtensionMessageHandler` gets a `cpmInitProgress` method. Unknown stage names are ignored without an error, so a newer extension cannot make an older app answer with failures. +- `CPMMessagingHealthEvent.initProgress` carries the mark; the monitor stores `lastStage` per document and clears it on a new navigation, a cancelled navigation and an extension reload. +- `response_sent` counts as healthy evidence for the document, like a dashboard response. `response_failed` judges a pending document at once (`ended_by=explicit_signal`). Healthy evidence is either the dashboard refresh or `response_sent`; the open question below asks whether `response_sent` should count before it is verified. +- The Debug "Simulate CPM Messaging Breakage" switch hides marks as well as dashboard responses. + +### Pixel change + +`web_extension_cpm_breakage` (defined in [[cpm-permanent-breakage-metric-tech-design-2026-11-10]]) gains one parameter: + +| Parameter | Values | +|---|---| +| `last_stage` | `none` (no mark reached the app), `page_init_sent`, `received`, `config_read`, `settings_checked`, `site_checked`. `response_sent` cannot appear on a breakage, because it is healthy evidence. `response_failed` appears when it was the reason. | + +No new pixel. Names, frequency and the other parameters are unchanged. Both platforms' definitions and `params_dictionary.json5` get the parameter. + +### How to read `last_stage` + +The marks are relayed by the background, so a dead background cannot report. `none` therefore cannot tell a missing content script, a lost message to the background and a lost notification apart. + +| `last_stage` on a breakage | What it says | Next action | +|---|---|---| +| `none` | nothing reached native: content script not injected, `init` and the mark lost on the way to the background, or the background cannot notify native | check injection (`tab_has_extension_user_scripts` is gone); WebKit-side loss on wake (R1, H2) | +| `page_init_sent` | the content script sent; the background never processed `init` | message lost between content script and worker listener after a wake: content-script retry of `init`, WebKit bug report | +| `received` | `init` arrived; remote config or session storage did not finish | H1 (unbounded config/storage reads, shared state queue): bound the awaits with ordering-safe semantics | +| `config_read` | stuck on the user setting round trip to native | native round-trip stall, H5: answer from cached config first | +| `settings_checked` | stuck on the site check | same, per-site call | +| `site_checked` | all checks passed, but neither `initResp` nor a dashboard refresh arrived | the page's tab is gone or unreachable (tab not connected) | +| `response_failed` | `tabs.sendMessage` rejected | tab not found or content script gone: the page's context changed under the answer | + +A site or a setting that disables CPM legitimately ends the chain at `settings_checked` or `site_checked`, but then the extension still sends the dashboard refresh, so those pages are healthy and never appear here. + +### Extension change + +All changes are in `duckduckgo-privacy-extension`, branch `alex/cpm-init-progress`. Commit `b00cc227` (pushed, PR #3727) has `received`, `response_sent` and `response_failed`; the remaining marks are an uncommitted working-tree change that has not been tested. + +| File | Change | +|---|---| +| `shared/js/background/components/cpm-embedded-messaging.js` | `notifyInitProgress(tabId, url, stage)`: `nativeMessaging.notify('cpmInitProgress', …)` inside the existing timeout helper; catches every error; never records a diagnostics error. | +| `shared/js/background/components/cookie-prompt-management.js` | `received` at the top of the main-frame `init` branch, before any await; `config_read` after the config is available; `settings_checked` after `checkAutoconsentSetting`; `site_checked` after `checkAutoconsentEnabledForSite`; `notifyInitResponseProgress` wraps `tabs.sendMessage(initResp)` to report `response_sent` or `response_failed`, and rethrows so the failure is still an unhandled rejection reported as `extensionException`. A page-side `cpmInitProgress` message is accepted only for the main frame and only for `page_init_sent`; unknown stages and sub-frames are ignored. | +| `shared/js/cpm.js` | `reportInitProgress('page_init_sent')` right before the content script sends `init`; wrapped in try/catch, never awaited, never delays init. | +| `unit-test/background/cookie-prompt-management.js` | order of marks, no await before `received`, `received` before the state reset, dispatch when the remote config is unavailable, a site that disables CPM ends at `site_checked`, `response_failed` keeps the rejection unhandled, sub-frame and unknown-stage marks ignored. | + +Compatibility and sequencing: + +- The embedded extension ships inside the app bundle, so the extension change reaches users only with an app release after the extension PR merges and the embedded files are updated. The native handler can ship first: it ignores a missing mark, and `last_stage=none` is the value for an app that has no marks yet. +- Older app, newer extension: the handler ignores unknown stages. An app without the `cpmInitProgress` handler is expected to answer with an error that `notifyInitProgress` swallows; to confirm against the real router before merge. +- `cpm.js` is one bundle for every build (`shared/js/cpm.js` goes to the Chromium, Chromium-embedded and embedded builds). `page_init_sent` therefore also reaches the desktop-extension background, where `notifyInitProgress` is absent and the message is dropped by an early return. Cost: one extra `runtime.sendMessage` per main-frame document in those builds. To confirm before merge: that this does not wake a sleeping MV3 worker earlier than the real `init` would, and whether the message should be limited to the embedded build. +- Rollout is by release channel: internal builds carry the new extension first, public builds follow. No remote switch inside the extension: `received` is sent before the config is read, so the config cannot gate it. + +### Pros + +- Turns "no answer" into a position in a known chain, which maps directly to the hypotheses and to concrete fixes. +- `response_sent` is closer to the real criterion (the page got `initResp`) than a dashboard refresh. +- Stays out of the page's way: fire-and-forget, no waits, no state changes. +- Both sides tolerate the other being older. + +### Cons + +- Marks pass through the same background whose failure we study, so the earliest failures collapse into `none`. +- Six notifications per main-frame document (five stages before the answer, then `response_sent` or `response_failed`), on top of the native round trips init already makes. The load on the path suspected to be fragile is not zero; its size has not been measured. +- The extension part reaches users only with an app release; the first data comes about two release cycles after the work is done (data in 1–2 release cycles, per the plan). +- A change to the page-visible content script (an extra message) on every build. + +### Risks + +- **The marks change the behavior they measure** (a heisenbug). Mitigation: internal builds first, compare breakage rates and the `last_stage` mix between releases with and without marks; if the cost is visible, drop `config_read` and `site_checked` in the next extension release. +- **`response_sent` is a false positive.** If `tabs.sendMessage` resolves without a listener (WebKit can resolve `undefined` for a message with no receiver, and the shipped router already returns `undefined` for a successful `runtime.sendMessage(init)`), a page without the content script would be counted healthy and breakage underestimated. Mitigation: a real-WebKit test with no listener must show `response_failed` or a missing `response_sent`; until that passes, `response_sent` is recorded but not counted (open question 1). +- **Marks arrive out of order or late.** Handled by rank; a document judged before its last mark arrives would carry an earlier `last_stage`. Marks arriving after the report are ignored. +- **A hostile page forges marks.** The content script runs in an isolated world; the page cannot call `chrome.runtime`. The background accepts a page-side mark only for `page_init_sent` and only from the main frame. + +## Testing + +- Extension: `make unit-test` (the cookie-prompt-management suite), `npm run tsc`, `npm run lint`. The uncommitted part has not been run. +- Native unit tests: stage ranking and not lowering; `response_sent` as healthy; `response_failed` as an immediate judgement; unknown stage ignored; a mark that cannot be tied to one tab dropped; `lastStage` cleared on a new navigation and on reload; `last_stage` in the pixel; simulation hides marks. +- Real WebKit (CPM repro harness, [PR #7378](https://github.com/duckduckgo/apple-browsers/pull/7378)): the full chain shows every mark in order on a healthy page; a page without the content script yields no `response_sent`; an injected stall at each await shows the matching `last_stage`; a delay on the native round trips for `config_read` and `settings_checked`. These runs have not been done. +- Pixel definitions pass both platforms' validators in CI. +- Privacy check of real payloads: the marks contain `url` on the extension-to-app channel only; the pixel carries the enum only. + +## Additional Considerations (if applicable) + +**Privacy** + +One new enumerated parameter on a pixel that is itself new in [[cpm-permanent-breakage-metric-tech-design-2026-11-10]]. The marks carry the tab ID and the URL from the extension to the app inside the app (the dashboard refresh already does), but neither enters a pixel or a log (the monitor's log descriptions omit URLs). Privacy Triage to be requested once for both designs. Not requested yet. + +**Security** + +No new private WebKit selectors. A new native message method, `cpmInitProgress`, is read-only: it parses a stage name and a tab, never executes content and never answers with data. Page-origin spoofing is covered in Risks. + +**Site Breakage** + +No intended page-behavior change. The only extension behavior change is the notifications; the `initResp` call keeps its arguments and its rejection behavior (`notifyInitResponseProgress` rethrows). + +**Experimentation** + +None. The comparison is across releases and channels (see the heisenbug risk). + +**Operational** + +- Order of landing: native handler and `last_stage` (PR 4 in the stack of the companion design) can ship before the extension; the extension PR merges after review and enters the app through the usual embedded-files update. +- After the first public release with marks: `last_stage × background × platform` over the breakage count, and the share of breakages by stage, to pick the fix according to the table above. Data checks as in the companion design. +- The marks and the parameter are temporary investigation tools. Remove `config_read`, `settings_checked`, `site_checked` and `last_stage` when the wake group has a fix or is closed (project plan: close the project and reset the target if the new metrics show nothing actionable). The expiry date follows the pixel definition (2027-01-15). +- Not in scope: per-stage durations (a time bucket between marks), the background's own lifecycle metrics, and the idle-unload distance. They are the next candidates if `last_stage` leaves the wake group unexplained. + +**Localization / Internationalization** + +None. + +## Open questions + +1. Count `response_sent` as healthy in the first release, or only record it until the no-listener test proves it cannot be a false positive? +2. Six marks per page or fewer: is `config_read` and `site_checked` worth the extra notifications, or are `received`, `settings_checked` and the response enough for the first read? +3. Limit `page_init_sent` to the embedded build, or accept the extra message on every build? +4. Is an extension-side switch needed (a remote-config setting read before sending `received`), or is release-channel rollout enough? +5. Who owns the extension review, and can the embedded-files update be timed to the app release that carries the native handler?