git_admin/obsidian-vault
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[2026-11-11] eagle: work/projects/cpm-permanent-breakage-metric-tech-design-2026-11-10.md work/projects/cpm-wake-init-detection-tech-design-2026-11-10.md

Browse Source
This commit is contained in:
Alexey Martemyanov committed 2026-11-11 19:06:47 +06:00
1 parent 48083620d5
commit 006ded56a8
2 files changed
+51 -45
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

work/projects/cpm-permanent-breakage-metric-tech-design-2026-11-10.md
+23 -19
View File
@@ -13,13 +13,10 @@ status: draft
**Reviewer:** TBD (previous CPM pixel designs: Michal)
**Stakeholders:** Michal, Konrad, Russell (from the previous Tech Designs; confirm)
**Project:** [CPM: root-cause the stuck embedded extension messaging (macOS/iOS)](https://app.asana.com/1/137249556945/task/1216761517055116)
**Task:** to create ("Permanent breakage metrics")
**Previous Tech Designs:** [CPM breakage pixels](https://app.asana.com/1/137249556945/task/1218005985662983), [CPM failure diagnostics](https://app.asana.com/1/137249556945/task/1218619301158188)
**Privacy Triage:** to request (new pixel names)
## Background & Requirements
The goal is permanent CPM failures below 1% of DAU, but no pixel measures it: the number comes from `web_extension_cpm_episode` through a long query, and the episode is a poor basis. A four-second navigation timer opens it (on iOS 7.244, 6.0% of episodes closed with a late answer on the same page, 42% of those after 30 s), and an open episode is lost when the app exits.
The goal is permanent CPM failures below 1% of DAU, but no pixel measures it. The number is derived from `web_extension_cpm_episode`, which is a poor basis: a four-second navigation timer opens an episode (on iOS 7.244, 6.0% of episodes closed with a late answer on the same page), and an open episode is lost when the app exits.
Requirements: no navigation timer, a rate against DAU, no URLs, page content or persistent identifiers.
@@ -29,32 +26,39 @@ Report the tabs whose pages never hear from the extension, and tell one broken p
## Recommended Approach
While a page initializes, the extension sends the app a message from that tab. A tab is **silent** when it loaded a page that should get a message and heard nothing for 30 s of app time. A late message still counts as an answer. There is no timer: the check runs when another event arrives (page load, tab close, message, app to background or wake).
- Pages that count: http(s), extension loaded, not an error page, not a back/forward restore, not a restored session. Suspended time is not counted. A page replaced or closed within 30 s is not reported.
- A message carries the extension's tab number and the page URL. The app learns which tab it is from the first message that matches exactly one open page (host and path). A message it cannot place answers every tab it could belong to: the metric prefers missing a breakage to reporting a working tab.
1. The extension already sends the app a message from a tab while a page initializes. A tab is **silent** when it loaded a page that should get a message and heard nothing for 30 s of app time; a late message still counts as an answer. There is no timer: the check runs when another event arrives (page load, tab close, message, app to background or wake) [1].
2. Pages that count: http(s), extension loaded, not an error page, not a back/forward restore, not a restored session. Suspended time is not counted.
3. A message carries the extension's tab number and the page URL. The app learns which tab it is from the first message that matches exactly one open page (host and path). A message it cannot place answers every tab it could belong to [2].
4. Two pixels, no expiry date (permanent metric):
| Pixel | Trigger | Frequency |
|---|---|---|
| `web_extension_cpm_breakage` | A tab is silent. Parameter `tabs`: `one` (no other tab was silent since the extension last answered: likely the page) or `many` (another tab was too: likely the extension). | Daily + count |
| `web_extension_cpm_breakage_persistent` | A second tab is silent and the extension answered nothing in between. Once per series. | Daily + count |
Success metric: installs with the persistent pixel / DAU. The episode pixel, `web_extension_cpm_background_load_restart` and `web_extension_cpm_network_process_recovery_applied` are removed with the code that only fed them; the recoveries themselves stay.
5. Remove `web_extension_cpm_episode`, `web_extension_cpm_background_load_restart` and `web_extension_cpm_network_process_recovery_applied` with the code that only fed them, in the same release. Success metric: installs with the persistent pixel / DAU.
## Pros and Cons
## Notes
- Pros: no timer, a slow answer is not a failure, a rate straight from the pixel.
- Cons: a silent tab left open until exit is not reported (iOS reports it on going to the background); pages left within 30 s are never reported; tabs on the same host and path hide each other; the tab mapping is a heuristic and rarely blames a working extension; one tab alone never triggers the persistent pixel.
[1] A silent tab left open until the app exits is not reported (iOS reports it when going to the background); a page left within 30 s is not reported.
[2] The mapping is a heuristic: tabs on the same host and path hide each other, and rarely a working extension is reported. An alternative is to have the extension send its tab index; it needs an extension change.
## Testing
Unit tests for each rule, pixel definition validators, adapter tests, app-hosted macOS and iOS tests. Not done: a real-WebKit run, and a check that the extension messages on every eligible page (search results, `duck.ai`, non-document types).
Unit tests for each rule, pixel definition validators, app-hosted macOS and iOS tests. Not done: a real-WebKit run.
## Additional Considerations
**Privacy:** two new pixel names, one enumerated parameter, no URLs or identifiers. Privacy Triage not requested yet.
**Security:** no new private WebKit selectors.
**Site Breakage:** none.
**Experimentation:** none.
**Operational:** a stack of PRs ([#8054](https://github.com/duckduckgo/apple-browsers/pull/8054) removal on top of [#8025](https://github.com/duckduckgo/apple-browsers/pull/8025), then the metric). The removal and the metric ship in the same release. No expiry date: permanent metric.
**Localization:** none.
1. Privacy
1. Two new pixel names, one enumerated parameter, no URLs or identifiers. Privacy Triage not filed yet.
2. Security
1. No new private WebKit selectors.
3. Site Breakage
1. None.
4. Experimentation
1. None.
5. Operational
1. No new infrastructure.
6. Localization / Internationalization
1. Not affected.
work/projects/cpm-wake-init-detection-tech-design-2026-11-10.md
+28 -26
View File
@@ -14,59 +14,61 @@ status: draft
**Reviewer:** TBD (previous CPM pixel designs: Michal)
**Stakeholders:** Michal, Konrad, Russell (from the previous Tech Designs; confirm). Extension change: the privacy-extension owners.
**Project:** [CPM: root-cause the stuck embedded extension messaging (macOS/iOS)](https://app.asana.com/1/137249556945/task/1216761517055116)
**Task:** to create ("Extension lifecycle metrics")
**Previous Tech Designs:** [CPM breakage pixels](https://app.asana.com/1/137249556945/task/1218005985662983), [CPM failure diagnostics](https://app.asana.com/1/137249556945/task/1218619301158188)
**Privacy Triage:** to request together with the CPM permanent breakage metric design, which this one extends
**Extension PR:** [duckduckgo-privacy-extension#3727](https://github.com/duckduckgo/duckduckgo-privacy-extension/pull/3727)
## Background & Requirements
Failures after a background wake are the largest group with no fix: on macOS 1.214, 39% of all failures (0.68% of DAU); on iOS 7.244, 22% (0.06% of DAU).
The app cannot see where the startup message stops. For one page: page script sends `init` → background receives it → reads configuration → reads the user setting → checks the site → answers the page and refreshes the app's dashboard. The app sees only the last step, so every failure looks the same: no answer.
For one page, startup is: page script sends `init` → background receives it → reads configuration → reads the user setting → checks the site → answers the page and refreshes the app's dashboard. The app sees only the last step, so every failure looks the same: no answer.
Requirements: find where the chain stopped, without changing what the extension does for the page, without collecting URLs or persistent identifiers, and with older and newer app and extension tolerating each other.
Requirements: find where the chain stopped, without changing what the extension does for the page or collecting URLs or persistent identifiers.
## Problem Statement
For every tab the permanent breakage metric reports as silent, record the furthest init step the extension reported, so the next fix is picked from data.
For every tab the permanent breakage metric reports as silent, record the furthest init step the extension reported.
## Recommended Approach
The extension sends the app a fire-and-forget notification at each init step. The app keeps the furthest step per tab (for its current page only) and adds it as `last_stage` to `web_extension_cpm_breakage`. A notification is never an answer and never creates a report. Unknown step names are ignored.
1. The extension sends the app a fire-and-forget notification at each init step ([PR #3727](https://github.com/duckduckgo/duckduckgo-privacy-extension/pull/3727)). It never waits, throws or changes what the page receives.
2. The app keeps the furthest step per tab, for its current page only. A notification does not count as the extension answering: the tab is still reported silent unless the dashboard refresh arrives.
3. `web_extension_cpm_breakage` gets one parameter, `last_stage` [1]:
| `last_stage` | The chain stopped |
|---|---|
| `none` | nothing reached the app (script not injected, or a message was lost) |
| `none` | nothing reached the app [3] |
| `page_init_sent` | the page script sent `init`; the background never processed it |
| `received` | `init` arrived; configuration or storage read did not finish |
| `config_read` | waiting for the user setting from the app |
| `settings_checked` | waiting for the site check |
| `site_checked` | all checks passed; no answer arrived |
| `response_sent` | the answer was sent to the page, but no dashboard refresh followed |
| `response_sent` | the answer was sent to the page, but no dashboard refresh followed [2] |
| `response_failed` | sending the answer to the page failed |
The extension change only adds these notifications inside the existing `init` handling; they never wait, throw or change what the page receives. The extension ships inside the app, so it reaches users with an app release; the app side can ship first (then `last_stage` is `none`).
4. The extension ships inside the app, so the app side can ship first. The notifications and `last_stage` are removed when the wake group has a fix or is closed.
## Pros and Cons
## Notes
- Pros: turns "no answer" into a position in a known chain that maps to concrete fixes; stays out of the page's way.
- Cons: notifications pass through the background we study, so the earliest failures end as `none`; up to seven extra notifications per page, load not measured (internal builds first); first data comes about two release cycles after the work.
[1] Each stage follows a separate wait in the extension's `init` handling: the cached configuration, the user setting and the site check (the last two are native round trips). A stall at each has a different cause and fix.
[2] WebKit can report a send as done when nobody was listening, so `response_sent` does not prove the page got the answer.
[3] Notifications pass through the background we study, so the earliest failures end as `none`.
## Testing
App unit tests for the furthest step per tab, `none` without notifications, ignored unknown steps and the pixel definitions. Extension unit tests for the order of notifications (part not run yet). Not done: a real-WebKit run showing every step in order and the right `last_stage` for a stall at each step.
Unit tests in the app and in the extension. Not done: a real-WebKit run showing the right `last_stage` for a stall at each step.
## Additional Considerations
**Privacy:** one new enumerated parameter. Tab number and URL travel inside the app only; neither enters a pixel or a log. Privacy Triage is requested once for both designs.
**Security:** no new private WebKit selectors; the new message is read-only.
**Site Breakage:** no intended change.
**Experimentation:** none; rollout by release channel.
**Operational:** app side first, extension after review and the embedded-files update. Removed when the wake group has a fix or is closed.
**Localization:** none.
## Open Questions
1. Are `config_read` and `site_checked` worth the extra notifications for the first read?
2. Send `page_init_sent` only in the embedded build, or in every build?
1. Privacy
1. One new enumerated parameter. Tab number and URL stay inside the app. Privacy Triage not filed yet.
2. Security
1. No new private WebKit selectors; the new message is read-only.
3. Site Breakage
1. No intended change.
4. Experimentation
1. None.
5. Operational
1. First data comes about two release cycles after the work, because the extension ships with the app.
6. Localization / Internationalization
1. Not affected.