[2026-11-11] eagle: BuyerWatch/2026-11-10-0001.md BuyerWatch/2026-11-10-0002.md BuyerWatch/2026-11-10-0003.md BuyerWatch/2026-11-10-0004.md BuyerWatch/2026-11-10-0005.md BuyerWatch/2026-11-10-0006.md BuyerWatch/2026-11-10-0007.md BuyerWatch/2026-11-10-0008.md BuyerWatch/2026-11-11-0009.md BuyerWatch/2026-11-11-0010.md
This commit is contained in:
Alexey Martemyanov committed
1 parent
5f60361522
commit
a5d51aa22f
18 files changed
+996
-1468
No files matched your search
@@ -4,12 +4,12 @@ tags:
|
||||
- cpm
|
||||
- tech-design
|
||||
- pixel
|
||||
updated: '2026-11-10'
|
||||
updated: '2026-11-11'
|
||||
status: draft
|
||||
---
|
||||
# TD: Pixel Definition: CPM Permanent Breakage Metric (macOS/iOS)
|
||||
|
||||
> Internal metrics, keep out of public PRs. Companion: [[cpm-wake-init-detection-tech-design-2026-11-10]] (adds `last_stage`). Where we are: [[cpm-project-summary-2026-11-03]].
|
||||
> Internal metrics, keep out of public PRs. Companion: [[cpm-wake-init-detection-tech-design-2026-11-10]] (adds `last_stage`). Where we are: [[cpm-project-summary-2026-11-03]]. Reworked on 11 Nov from "judge a page when its document ends" to "a tab is silent"; the first design is gone.
|
||||
|
||||
**Author:** Alex M
|
||||
**Reviewer:** TBD (previous CPM pixel designs: Michal)
|
||||
@@ -26,93 +26,98 @@ The goal is permanent CPM failures below 1% of DAU. There is no pixel for it; th
|
||||
- **A timer decides.** A four-second navigation timeout opens an episode. On iOS 7.244, 6.0% of episodes closed with a late answer on the same page, 42% of those later than 30 s. Slow answers count as failures.
|
||||
- **Episodes are lost.** An open episode lives in memory and is dropped when the app exits.
|
||||
|
||||
Requirements: a metric without a timer, as a rate against DAU; no page content, URLs or persistent identifiers. End state: the only CPM pixels are the two defined here.
|
||||
Requirements: a metric without a navigation timer, as a rate against DAU; no page content, URLs or persistent identifiers; a minimal layer next to the product code (no private selectors, no per-navigation state machine). End state: the only CPM pixels are the two defined here.
|
||||
|
||||
## Problem Statement
|
||||
|
||||
Report pages that really never got the extension's answer, and tell persistent breakage from a single page.
|
||||
Report tabs whose pages never hear from the extension, and tell one broken page from a broken extension.
|
||||
|
||||
## Recommended Approach
|
||||
|
||||
**A page is healthy when an answer reaches it at any time while its document lives. A page that never got an answer is judged when its document ends. There are no navigation timers.**
|
||||
**The extension sends a message from a tab while a page initializes. A tab that commits an eligible page and gets no message for 30 s of app time is silent. A message that arrives later still counts as an answer.**
|
||||
|
||||
The monitor keeps, per tab: the time of the latest eligible commit (until a message answers it) and the time of the latest message. Nothing runs on a timer: the check happens when any event arrives (a commit, a tab closing, a message, the app waking).
|
||||
|
||||
| Event | Effect |
|
||||
|---|---|
|
||||
| Navigation finished, extension loaded | The document becomes judgeable and its clock starts. If the extension loads later, the clock starts then. |
|
||||
| Attributed answer (dashboard refresh) | Healthy. Resets the streak. |
|
||||
| Document ends: new navigation or tab close | If unanswered and alive at least **30 s**: breakage. Shorter silence is not judged (it may be a slow wake). |
|
||||
| Web content process terminated | Not judged. |
|
||||
| Extension signals `cpmStage` = `config_unavailable` or `settings_missing` | Breakage at once, no lifetime limit. The signal normally arrives while the page is still loading, so it is remembered and applied when the page finishes. |
|
||||
| Answer that cannot be tied to one document | That document is not judged. |
|
||||
| App suspended | Nothing is judged while suspended. |
|
||||
| Commit of an http(s) page, extension loaded, not an error page, not a history restore, not a restored session | The tab waits for a message. A message up to 2 s before the commit answers it (the extension can message a tab before the app reports the commit). |
|
||||
| Message from the current extension (dashboard refresh) | Answers the tab's page, even when late. Ends the series of silent tabs. |
|
||||
| `config_unavailable` or `settings_missing` refresh | Not an answer: the extension said it cannot answer the page's init. |
|
||||
| Any event, with a commit older than 30 s and no message | The tab is silent: reported once. |
|
||||
| New commit or tab close | A page that already waited 30 s is reported first; a younger one is dropped. |
|
||||
| Web content process terminated | The tab's page is dropped, not reported. |
|
||||
| App suspended (iOS background, macOS sleep) | Nothing is checked, and the suspended time is added back to every wait. |
|
||||
| Extension reload | All waits, mappings and series are forgotten. |
|
||||
|
||||
Judged documents are never re-judged. Measurement state never expires (the old five-minute drop is gone).
|
||||
### Tying a message to a tab
|
||||
|
||||
**Series:** reported pages that started after the latest answered page, any tab, in memory. Pages are ordered by navigation start, not by the time of the verdict, so a page judged late (tab closed long after) does not join a series that an answered newer page has already ended. At 3 the persistent pixel fires once for the series.
|
||||
A message carries the extension's tab identifier and the page URL. WebKit creates its extension tab object lazily, and neither public nor private API exposes its identifier, so the app learns it: the first message from an extension tab that matches exactly one committed page (host and normalized path, the same match the privacy dashboard uses) fixes that tab's identifier; later messages use it directly. A message that cannot be tied to one tab (two tabs on the same URL, no tab identifier, no match) answers every tab it could belong to, so the metric prefers missing a breakage to reporting a working tab. A tab closing forgets its identifier.
|
||||
|
||||
### Pixels
|
||||
|
||||
| Pixel | Trigger | Frequency |
|
||||
|---|---|---|
|
||||
| `web_extension_cpm_breakage` | A document is judged broken. | Daily + count |
|
||||
| `web_extension_cpm_breakage_persistent` | The streak reaches 3. Once per streak. | Daily + count |
|
||||
| `web_extension_cpm_breakage` | A tab is silent. | Daily + count |
|
||||
| `web_extension_cpm_breakage_persistent` | A second tab is silent with no message from the extension in any tab since the first one was reported. Once per series. | Daily + count |
|
||||
|
||||
`_daily` is installs per day; `_count` is events. Metadata follows the existing episode pixel on each platform: `appVersion` on both; macOS also `pixelSource`, `channel`, suffix `daily_count`; iOS keeps its suffix set. The macOS definition must match what is sent.
|
||||
|
||||
`web_extension_cpm_breakage` parameters:
|
||||
`web_extension_cpm_breakage` parameter:
|
||||
|
||||
| Parameter | Values |
|
||||
|---|---|
|
||||
| `background` | `woken` (a background view was created during this navigation), `warm` (views were seen, none created for it), `unknown` (no background view seen yet). From the manager's `didCreateBackgroundWebView` callback. |
|
||||
| `tabs` | `one`: no other tab was silent since the extension last sent a message; the page is the likely cause. `many`: another tab was already silent; the extension is the likely cause. |
|
||||
|
||||
`web_extension_cpm_breakage_persistent` has no parameters. `last_stage` is added by the companion design.
|
||||
|
||||
Dashboard: persistent installs / DAU from `breakage_persistent_daily` (the success metric); any-breakage installs / DAU from `breakage_daily`; breakages per day from `breakage_count` by `background`.
|
||||
A series ends with any message from the extension, in any tab. A silent tab that is closed still counts for its series. A single tab that keeps going silent is `tabs=one` each time and never persistent.
|
||||
|
||||
### Removed in the same PR
|
||||
Dashboard: persistent installs / DAU from `breakage_persistent_daily` (the success metric); any-breakage installs / DAU from `breakage_daily`; breakages per day from `breakage_count` by `tabs`.
|
||||
|
||||
The episode pixel `web_extension_cpm_episode` with its timers and episode state, `web_extension_cpm_background_load_restart` (and its observation), and `web_extension_cpm_network_process_recovery_applied`: definitions, events and firing code. The recovery behavior (error-6 restart, network recovery) stays; only the pixels go.
|
||||
### Removed in the same stack
|
||||
|
||||
### First attempt (`e6419bb910`): defects fixed here
|
||||
|
||||
Measurements older than five minutes dropped; no minimum lifetime; tab process crash counted; `background` from load duration and one app-wide signal; macOS definition mismatch; `navigationWaitContexts` leak (the wait-context set is deleted); no `cpmStage` test.
|
||||
The episode pixel `web_extension_cpm_episode` with its timers, episode state and failure reason, `web_extension_cpm_background_load_restart` (and its observation), and `web_extension_cpm_network_process_recovery_applied`: definitions, events and firing code are removed in PR 2. The recovery behavior (error-6 restart, network recovery) stays; only the pixels go. Between PR 2 and PR 3 no CPM breakage pixel exists, so they must ship in the same release.
|
||||
|
||||
### PR stack
|
||||
|
||||
| PR | Content | State, 11 Nov |
|
||||
|---|---|---|
|
||||
| 1, [#8025](https://github.com/duckduckgo/apple-browsers/pull/8025) | Remove ineffective recovery fixes and flags | pushed, in review |
|
||||
| 2, `alex/cpm-pixel-cleanup` (`ae725e6b12`) | All removal: diagnostics, the episode pixel with its timers and the failure reason, the restart and network-recovery pixels, orphaned helpers. The monitor shrinks to matching a page with the extension's answer. | local; 999 package tests pass; macOS build running; not pushed |
|
||||
| 3, `alex/cpm-breakage-metric` (`82d29f412a`) | All addition: verdict at the end of the document, series, buffered responses, `background`, the two pixels | local; 1017 package tests pass; validators pass; not pushed |
|
||||
| 4, `alex/cpm-init-marks` (`e8dc96dec2`) | Init progress marks and `last_stage`; extension #3727 | local; 1023 package tests pass; not pushed |
|
||||
|
||||
Between PR 2 and PR 3 no CPM breakage pixel exists, so they must ship in the same release.
|
||||
| 2, `alex/cpm-pixel-cleanup` (`73bdf742b0`) | All removal. The monitor shrinks to learning which extension tab belongs to which app tab and letting the manager reset its restart budget. Tabs report one commit event with an eligibility flag; navigation start, finish and failure events are gone. | local, one commit on `6928847174`; 1004 package tests pass; not pushed |
|
||||
| 3, `alex/cpm-breakage-metric` (`cf83e35c12`) | Per-tab silence, the two pixels, `tabs`, definitions, adapters | local, one commit on PR 2; 1034 package tests pass; validators pass; not pushed |
|
||||
| 4, `alex/cpm-init-marks` (`PR4HASH`) | Init progress marks and `last_stage`; extension #3727 | local, one commit on PR 3; 1044 package tests pass; not pushed |
|
||||
|
||||
### Known limits
|
||||
|
||||
- A page that never reaches `navigationFinished`, or a document alive when the app is killed, is not judged.
|
||||
- If a new navigation fails after `navigationStarted`, the old document was already judged and reset; its late answer is not attributed.
|
||||
- A mark or answer for a reloaded page with the same URL cannot be told from the old document's.
|
||||
- The dashboard refresh is sent independently of `initResp`, so a page whose `initResp` was never delivered counts as answered unless the extension reports `response_failed`.
|
||||
- `page_init_sent` travels through the same background, so `last_stage=none` cannot tell a lost mark from an init that never started.
|
||||
- Evaluation happens only on events. A silent tab in an app that then sits idle until it is killed is not reported.
|
||||
- The first message of a tab can only be tied to a tab by URL. Two tabs on the same URL, or a page whose path changed before the first message, stay unmapped; their messages answer every candidate tab, which hides some real silence.
|
||||
- A single-page app changing its path can break the host and path match for a tab that is not mapped yet.
|
||||
- Pages the extension does not inject into (some non-document types) send no message and look silent. The MIME type is not available at commit, so these are not filtered.
|
||||
- A message within 2 s before a commit is counted as that commit's answer even when it belonged to the previous page.
|
||||
- The dashboard refresh is sent independently of `initResp`, so a page whose `initResp` was never delivered counts as answered.
|
||||
- The first series has no overlap with the episode series: comparison is by app version against the previous derived number.
|
||||
- The restart budget now resets on any message from the current embedded extension, not on a message matched to a page.
|
||||
- If the native channel itself is lost, every tab looks silent and `tabs=many` follows: the metric cannot tell a lost channel from a broken extension.
|
||||
|
||||
### Cons
|
||||
|
||||
- Documents alive when the app exits or suspends are never judged; breakage on pages left open is missed.
|
||||
- Silence of tabs left open until the app exits is never reported.
|
||||
- 30 s hides breakage on short pages.
|
||||
- The series is new: no overlap with the episode series, so comparison is by app version against the previous derived number.
|
||||
- Streak resets on relaunch.
|
||||
- Streak and series reset on relaunch.
|
||||
- The tab mapping is a heuristic bootstrap; its error direction is missing a breakage, not inventing one.
|
||||
|
||||
### Risks
|
||||
|
||||
- A page that works but whose answer is not attributed counts as breakage (`other_tab_response` was 37–53% of outcomes). Documents with an unattributable answer are not judged.
|
||||
- Pixel volume for a persistently broken install: one count request per page.
|
||||
- The Debug "Simulate CPM Messaging Breakage" switch must keep hiding answers.
|
||||
- A page that works but whose message is not attributed counts as silent when the mapping is wrong. Mitigation: ambiguous messages answer all candidates.
|
||||
- Pixel volume for a persistently broken install: one count request per silent tab.
|
||||
- The Debug "Simulate CPM Messaging Breakage" switch must keep hiding messages (it does; tests cover it).
|
||||
- `expires` on the two definitions follows the existing episode definition (2027-01-15), but `REVIEW-pixels.md` says a permanent pixel has no `expires`. Decide before merge.
|
||||
|
||||
## Testing
|
||||
|
||||
- Unit tests per row of the table, streak at 3 once and reset, `background` values, simulation switch, `cpmStage` parsing, extension reload.
|
||||
- Pixel definitions pass both validators; parameters match what is sent.
|
||||
- Unit tests per row of the table (window boundary, no timer, late answer, tolerance before commit, suspension, ineligible commits, crash, close, reload, mapping learned, kept and ambiguous, simulation, old extension, `tabs` one/many, persistent once and again after a message, closed silent tab). Mutation checks: answer applied after the check, no suspension shift, persistent on one tab, no tolerance each make a test fail.
|
||||
- Pixel definitions pass both validators; adapter tests check names, frequency and parameters.
|
||||
- App-hosted macOS and iOS tests: not run in this stack; package tests, validators, SwiftLint and the macOS `build-for-testing` are the checks.
|
||||
- Manual: [[CPM Breakage Testing Steps]] with the Debug switch.
|
||||
- Real WebKit (CPM repro harness, [PR #7378](https://github.com/duckduckgo/apple-browsers/pull/7378)): not run.
|
||||
|
||||
@@ -126,6 +131,6 @@ Between PR 2 and PR 3 no CPM breakage pixel exists, so they must ship in the sam
|
||||
|
||||
**Experimentation:** none.
|
||||
|
||||
**Operational:** pixel expiry 2027-01-15 as existing definitions. Danger label for the removed pixels. Data checks: `metrics.pixels_aggregation` and `metrics._pixels_new` (`pixel_id = 'web'`).
|
||||
**Operational:** pixel expiry 2027-01-15 as existing definitions (see Risks). Danger label for the removed pixels. Data checks: `metrics.pixels_aggregation` and `metrics._pixels_new` (`pixel_id = 'web'`).
|
||||
|
||||
**Localization / Internationalization:** none.
|
||||
@@ -114,3 +114,20 @@ Open:
|
||||
- [[cpm-wake-init-detection-tech-design-2026-11-10]]: метки `cpmInitProgress` из расширения, `last_stage` в пикселе, `response_sent` как здоровый сигнал, изменение расширения (PR #3727, ветка `alex/cpm-init-progress`; метки `page_init_sent`, `config_read`, `settings_checked`, `site_checked` в рабочем дереве без коммита и без запуска тестов).
|
||||
|
||||
Открытые вопросы для ревью: порог 30 с и серия 3; сохранять серию между запусками; считать ли `response_sent` здоровым до проверки без слушателя; шесть меток или меньше; нужен ли флаг для нового пикселя.
|
||||
|
||||
|
||||
## 11 ноября: переделка стопки на детект по молчанию вкладки
|
||||
|
||||
**Решение.** Детект «страница оценивается по концу документа» (серия из 3, буфер ответов, `background`) заменён на «вкладка молчит»: после коммита подходящей http(s) страницы с загруженным расширением вкладка, от которой расширение не прислало сообщение 30 с времени приложения, считается молчащей и репортится один раз. Поздний ответ всё равно считается ответом, ответ за 2 с до коммита засчитывается, время приостановки приложения не считается, таймеров нет, проверка идёт на событиях. Сообщение привязывается к вкладке по идентификатору вкладки расширения, который приложение узнаёт один раз по единственной совпавшей странице (host + path); сообщение, которое не привязать, отвечает за все подходящие вкладки (лучше пропустить поломку, чем записать рабочую вкладку). Публичного и приватного API для id вкладки расширения нет (проверено по заголовкам SDK и исходникам WebKit).
|
||||
|
||||
**Пиксели.** Остаются два: `web_extension_cpm_breakage` (параметр `tabs`: `one` или `many`, то есть была ли уже молчащая вкладка с последнего сообщения расширения) и `web_extension_cpm_breakage_persistent` (вторая молчащая вкладка в серии, один раз на серию, без параметров). Старые `web_extension_cpm_episode`, `web_extension_cpm_background_load_restart`, `web_extension_cpm_network_process_recovery_applied` удалены в PR 2, отдельного PR 5 нет. Метки `cpmInitProgress` и `last_stage` (PR 4) привязаны к молчащей вкладке; метка никогда не считается ответом.
|
||||
|
||||
**Раскладка (локально, по одному коммиту на ворктри, не запушено, PR не создавались).**
|
||||
- PR 2 `alex/cpm-pixel-cleanup`: `73bdf742b0` на `6928847174`. Вся чистка, монитор урезан до сопоставления вкладок и сброса бюджета перезапусков; вкладки шлют одно событие коммита с флагом `isEligible`, события старта/финиша/ошибки навигации удалены. 1004 теста пакета.
|
||||
- PR 3 `alex/cpm-breakage-metric`: `cf83e35c12` на PR 2. Молчание по вкладкам, два пикселя, определения обеих платформ, адаптеры. 1034 теста пакета.
|
||||
- PR 4 `alex/cpm-init-marks`: `PR4HASH` на PR 3. Метки init по вкладкам и `last_stage`. 1044 теста пакета. Расширение `#3727` не менялось.
|
||||
Старые коммиты сохранены как `refs/backup/cpm-old-pr3` (`82d29f412a`) и `refs/backup/cpm-old-pr4` (`e8dc96dec2`) в репозитории `apple-browsers.git`.
|
||||
|
||||
**Проверено:** `swift test` пакета на каждой ветке, мутационные проверки (ответ после проверки, нет сдвига на приостановку, persistent на одной вкладке, нет допуска, метка считается ответом, ранг понижается), валидаторы определений пикселей обеих платформ, SwiftLint strict на изменённых файлах, `xcodebuild build-for-testing` macOS. **Не запускалось:** app-hosted тесты macOS и iOS (кроме трёх тестов `NavigationPixelNavigationResponderTests`, запущенных до запрета), сборка iOS (падает на двух script phases, окружение), реальный WebKit.
|
||||
|
||||
**Открытые проблемы подхода:** привязка первого сообщения по URL (две вкладки с одним URL, смена path в SPA), страницы без инъекции расширения выглядят молчащими, оценка только на событиях (молчащая вкладка в простаивающем приложении, которое потом убили, не репортится), допуск 2 с может засчитать сообщение предыдущей страницы, потеря нативного канала даёт `tabs=many`, бюджет перезапусков теперь сбрасывается любым сообщением текущего расширения, нет базовой серии после удаления старых пикселей, `expires` на новых определениях противоречит правилу REVIEW-pixels для постоянных пикселей.
|
||||
@@ -5,7 +5,7 @@ tags:
|
||||
- tech-design
|
||||
- pixel
|
||||
- web-extension
|
||||
updated: '2026-11-10'
|
||||
updated: '2026-11-11'
|
||||
status: draft
|
||||
---
|
||||
# TD: Pixel Definition: CPM Wake Init Detection (macOS/iOS + embedded web extension)
|
||||
@@ -44,7 +44,7 @@ Requirements:
|
||||
|
||||
## Problem Statement
|
||||
|
||||
For every document judged broken by the permanent breakage metric, record the furthest init step the extension reported for it. With that:
|
||||
For every tab reported silent by the permanent breakage metric, record the furthest init step the extension reported for its page. With that:
|
||||
|
||||
1. Separate "init never reached the background" from "background stalled on its own storage or native round trip" from "answer could not be delivered to the page".
|
||||
2. Join the result with `background` (woken, warm) so wake failures are split from the rest.
|
||||
@@ -54,7 +54,7 @@ Collect no page content, URLs or persistent identifiers.
|
||||
|
||||
## Recommended Approach
|
||||
|
||||
The extension reports **progress marks** for each main-frame init as fire-and-forget native notifications. The app keeps the furthest mark per document, reports the page at once on `response_failed`, and adds `last_stage` to `web_extension_cpm_breakage`. `response_sent` is recorded but not counted as the page's answer until it is verified against real WebKit.
|
||||
The extension reports **progress marks** for each main-frame init as fire-and-forget native notifications. The app keeps the furthest mark per tab and adds `last_stage` of the silent tab to `web_extension_cpm_breakage`. A mark is never an answer: only the dashboard refresh answers a page, as in the companion design. A report is made only when a tab is silent, so marks never create a report and `response_failed` does not report anything by itself.
|
||||
|
||||
### Marks
|
||||
|
||||
@@ -68,14 +68,14 @@ The extension reports **progress marks** for each main-frame init as fire-and-fo
|
||||
| `response_sent` | background | `tabs.sendMessage(initResp)` resolved | 5 |
|
||||
| `response_failed` | background | the same call rejected | 6 |
|
||||
|
||||
Marks are best effort: they carry `{ tabId, url, stage }`, never wait and never reject, never write to the diagnostics error list, and cannot change CPM state or what the page gets. The native side keeps the highest rank seen for the document; an out-of-order arrival never lowers it. Marks about a document the monitor cannot tie to exactly one tab are dropped.
|
||||
Marks are best effort: they carry `{ tabId, url, stage }`, never wait and never reject, never write to the diagnostics error list, and cannot change CPM state or what the page gets. The native side keeps the highest rank seen for the tab's page; an out-of-order arrival never lowers it. A mark is tied to a tab the same way a dashboard refresh is (the learned extension tab identifier, or the one committed page its URL matches); a mark that cannot be tied to exactly one tab is dropped. A mark that arrives within 2 s before the commit is kept for the new page.
|
||||
|
||||
### Native side
|
||||
|
||||
- `AutoconsentWebExtensionMessageHandler` gets a `cpmInitProgress` method. Unknown stage names are ignored without an error, so a newer extension cannot make an older app answer with failures.
|
||||
- `CPMMessagingHealthEvent.initProgress` carries the mark; the monitor stores `lastStage` per document and clears it on a new navigation, a cancelled navigation and an extension reload.
|
||||
- Only the dashboard refresh counts as the page's answer. `response_sent` is recorded as a stage; `response_failed` reports a pending page at once.
|
||||
- The Debug "Simulate CPM Messaging Breakage" switch hides marks as well as dashboard responses.
|
||||
- `CPMMessagingHealthEvent.initProgress` carries the mark; the monitor stores the furthest stage per tab and forgets it at the next commit (unless it is within the 2 s tolerance), when the tab closes or crashes and on an extension reload.
|
||||
- A mark does not answer a page and does not end a series. The breakage report takes `last_stage` from the tab it reports, not from any other tab.
|
||||
- The Debug "Simulate CPM Messaging Breakage" switch hides marks as well as dashboard refreshes, and marks from an old embedded extension are ignored.
|
||||
|
||||
### Pixel change
|
||||
|
||||
@@ -83,9 +83,9 @@ Marks are best effort: they carry `{ tabId, url, stage }`, never wait and never
|
||||
|
||||
| Parameter | Values |
|
||||
|---|---|
|
||||
| `last_stage` | `none` (no mark reached the app), `page_init_sent`, `received`, `config_read`, `settings_checked`, `site_checked`. `response_sent` can appear: the page got `initResp` but no dashboard refresh was attributed. `response_failed` appears when it was the reason. |
|
||||
| `last_stage` | `none` (no mark reached the app or none could be tied to the tab), `page_init_sent`, `received`, `config_read`, `settings_checked`, `site_checked`. `response_sent` can appear: the page got `initResp` but no dashboard refresh answered the tab. `response_failed` appears when delivering `initResp` failed. |
|
||||
|
||||
No new pixel. Names, frequency and the other parameters are unchanged. Both platforms' definitions and `params_dictionary.json5` get the parameter.
|
||||
`web_extension_cpm_breakage_persistent` is unchanged. No new pixel. Names, frequency and the other parameters are unchanged. Both platforms' definitions and `params_dictionary.json5` get the parameter.
|
||||
|
||||
### How to read `last_stage`
|
||||
|
||||
@@ -124,7 +124,7 @@ Compatibility and sequencing:
|
||||
### Pros
|
||||
|
||||
- Turns "no answer" into a position in a known chain, which maps directly to the hypotheses and to concrete fixes.
|
||||
- `response_sent` is closer to the real criterion (the page got `initResp`) than a dashboard refresh.
|
||||
- `response_sent` and `response_failed` show cases the dashboard refresh hides (the page did not get `initResp` although the refresh arrived); a tab is still reported only when the refresh is also missing.
|
||||
- Stays out of the page's way: fire-and-forget, no waits, no state changes.
|
||||
- Both sides tolerate the other being older.
|
||||
|
||||
@@ -138,14 +138,15 @@ Compatibility and sequencing:
|
||||
### Risks
|
||||
|
||||
- **The marks change the behavior they measure** (a heisenbug). Mitigation: internal builds first, compare breakage rates and the `last_stage` mix between releases with and without marks; if the cost is visible, drop `config_read` and `site_checked` in the next extension release.
|
||||
- **`response_sent` is a false positive.** If `tabs.sendMessage` resolves without a listener (WebKit can resolve `undefined` for a message with no receiver, and the shipped router already returns `undefined` for a successful `runtime.sendMessage(init)`), a page without the content script would be counted healthy and breakage underestimated. Mitigation: a real-WebKit test with no listener must show `response_failed` or a missing `response_sent`; until that passes, `response_sent` is recorded but not counted (open question 1).
|
||||
- **Marks arrive out of order or late.** Handled by rank; a document judged before its last mark arrives would carry an earlier `last_stage`. Marks arriving after the report are ignored.
|
||||
- **`response_sent` can be a false positive.** If `tabs.sendMessage` resolves without a listener (WebKit can resolve `undefined` for a message with no receiver), a page without the content script would show `response_sent`. It never answers a page, so the metric is not affected; only the reading of `last_stage=response_sent` is. A real-WebKit test with no listener should settle it.
|
||||
- **Marks arrive out of order or late.** Handled by rank; a tab reported before its last mark arrives carries an earlier `last_stage`. Marks arriving after the report are ignored.
|
||||
- **A mark for a tab that is not mapped yet** is dropped when its URL matches two tabs, so `none` also covers marks lost to attribution.
|
||||
- **A hostile page forges marks.** The content script runs in an isolated world; the page cannot call `chrome.runtime`. The background accepts a page-side mark only for `page_init_sent` and only from the main frame.
|
||||
|
||||
## Testing
|
||||
|
||||
- Extension: `make unit-test` (the cookie-prompt-management suite), `npm run tsc`, `npm run lint`. The uncommitted part has not been run.
|
||||
- Native unit tests: stage ranking and not lowering; `response_sent` as healthy; `response_failed` as an immediate judgement; unknown stage ignored; a mark that cannot be tied to one tab dropped; `lastStage` cleared on a new navigation and on reload; `last_stage` in the pixel; simulation hides marks.
|
||||
- Native unit tests (PR 4, package suite 1044 tests): the report carries the furthest stage of the silent tab and not of another tab; no marks give `none`; a mark is not an answer; unknown stage ignored by the handler; a mark that cannot be tied to one tab dropped; a mark just before the commit belongs to the new page and an older one is forgotten; old extension and simulation ignored; closed tab forgets its marks; `last_stage` in the pixel on both platforms. Mutation checks (a mark answers, rank lowers, stage kept forever, stage always cleared) each fail a test. App-hosted tests were not run.
|
||||
- Real WebKit (CPM repro harness, [PR #7378](https://github.com/duckduckgo/apple-browsers/pull/7378)): the full chain shows every mark in order on a healthy page; a page without the content script yields no `response_sent`; an injected stall at each await shows the matching `last_stage`; a delay on the native round trips for `config_read` and `settings_checked`. These runs have not been done.
|
||||
- Pixel definitions pass both platforms' validators in CI.
|
||||
- Privacy check of real payloads: the marks contain `url` on the extension-to-app channel only; the pixel carries the enum only.
|
||||
@@ -170,7 +171,7 @@ None. The comparison is across releases and channels (see the heisenbug risk).
|
||||
|
||||
**Operational**
|
||||
|
||||
- Order of landing: native handler and `last_stage` (PR 4 in the stack of the companion design) can ship before the extension; the extension PR merges after review and enters the app through the usual embedded-files update.
|
||||
- Order of landing: native handler and `last_stage` (PR 4, `alex/cpm-init-marks`, `PR4HASH`, on top of PR 3 `cf83e35c12`) can ship before the extension; the extension PR merges after review and enters the app through the usual embedded-files update.
|
||||
- After the first public release with marks: `last_stage × background × platform` over the breakage count, and the share of breakages by stage, to pick the fix according to the table above. Data checks as in the companion design.
|
||||
- The marks and the parameter are temporary investigation tools. Remove `config_read`, `settings_checked`, `site_checked` and `last_stage` when the wake group has a fix or is closed (project plan: close the project and reset the target if the new metrics show nothing actionable). The expiry date follows the pixel definition (2027-01-15).
|
||||
- Not in scope: per-stage durations (a time bucket between marks), the background's own lifecycle metrics, and the idle-unload distance. They are the next candidates if `last_stage` leaves the wake group unexplained.
|
||||
@@ -181,8 +182,7 @@ None.
|
||||
|
||||
## Open questions
|
||||
|
||||
1. Count `response_sent` as healthy in the first release, or only record it until the no-listener test proves it cannot be a false positive?
|
||||
2. Six marks per page or fewer: is `config_read` and `site_checked` worth the extra notifications, or are `received`, `settings_checked` and the response enough for the first read?
|
||||
3. Limit `page_init_sent` to the embedded build, or accept the extra message on every build?
|
||||
4. Is an extension-side switch needed (a remote-config setting read before sending `received`), or is release-channel rollout enough?
|
||||
5. Who owns the extension review, and can the embedded-files update be timed to the app release that carries the native handler?
|
||||
1. Six marks per page or fewer: is `config_read` and `site_checked` worth the extra notifications, or are `received`, `settings_checked` and the response enough for the first read?
|
||||
2. Limit `page_init_sent` to the embedded build, or accept the extra message on every build?
|
||||
3. Is an extension-side switch needed (a remote-config setting read before sending `received`), or is release-channel rollout enough?
|
||||
4. Who owns the extension review, and can the embedded-files update be timed to the app release that carries the native handler?
|
||||
Reference in new issue
Block a user