git_admin/obsidian-vault
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[2026-11-03] eagle: work/projects/cpm-project-summary-2026-11-03.md

Browse Source
This commit is contained in:
Alexey Martemyanov committed 2026-11-03 15:55:06 +06:00
1 parent 7b68a7bf58
commit 70e68145b8
1 file changed
+63
Download Patch File Download Diff File Expand all files Collapse all files
work/projects/cpm-project-summary-2026-11-03.md
+63
View File
@@ -0,0 +1,63 @@
---
tags:
- work
- cpm
- summary
updated: '2026-11-03'
status: draft
---
# CPM stuck-messaging project: where we are, Nov/3
Internal metrics, keep out of public PRs. Task: [CPM: root-cause the stuck embedded extension messaging](https://app.asana.com/1/137249556945/project/72649045549333/task/1216761517055116?focus=true). Details: [[cpm-post-release-analysis-2026-10-27]], [[cpm-status-2026-10-15]].
Goal: permanent stuck reports below 1% of DAU (0.2-0.5% counts as success).
## What the project has achieved
- Telemetry that attributes failures (public since Oct/5). We can now see what state the extension was in when a page got no response.
- Fixed the deterministic per-tab bugs: tab merging, Close Other Tabs, pinned tabs and window close (macOS).
- Reproduced the main failure patterns on real WebKit, with the event sequences matched to production.
- Oct/26 release (macOS 1.213 / iOS 7.243): new episode telemetry, recovery for slow wake, error 6 retry and slow startup, restart after a networking process crash.
- Nov/2 release (1.214 / 7.244): stable extension origin (P3, only on OS 26.6+), and the fix for browser error pages being counted as CPM failures.
- Result so far: the noise is gone, permanent failures are not. On iOS 26.6+, startup-missing and no-view reports are down by over 90% against 7.243 on the same day. Permanent share of DAU is about the same as before.
## Hypotheses and fixes that did not work out
- Deallocation race fix and its A/B test: cancelled because the metrics were too blurry, flag never activated. Data showed no relation to the released fixes.
- Slow wake fix (P1): shipped, no measurable effect on the wake group. The background is ready in under 1s, so the loss happens after that.
- Hung or crashed background process: not the cause, under 1% of failures show it.
- Memory pressure, feature flag toggling, "failed load blocks forever": not reproduced or ruled out as a global cause.
- Lost message after wake: could not be reproduced in WebKit tests (0 lost in ~1,500 wakes, also with real pages and the ad blocker running).
- Late page reload fix: default off and never fired.
- Graveyard mitigation: not supported by the data, not pursued.
## Fixes going public next week (Nov/9)
- Upgrade-before-load ([#7767](https://github.com/duckduckgo/apple-browsers/pull/7767)): load the new extension version first on launch, instead of an outdated one.
- Expected: removes part of the "extension updated" group (macOS 0.26% DAU, iOS 0.05%). Launch upgrades only, so partial.
- Error 6 restart ([#7730](https://github.com/duckduckgo/apple-browsers/pull/7730)): restart the stuck background when the failed view is still retained.
- Expected: up to 0.08% DAU on iOS, negligible on macOS. Partial, most error 6 reports only have it in history.
- Together: macOS 1.89% to about 1.6%, iOS 0.44% to about 0.3-0.35%. Upper bounds, not measured.
## Data now (% of DAU, 1.214 / 7.244, Nov/2)
One day, early adopters, macOS has about 9k DAU. Treat as a rough picture.
- Permanent stuck = no response for 5 minutes, confirmed by a repeat or a reload. Browser-wide = 2+ tabs, one-tab = single tab.
- macOS: any failure 15.1% (28.5% on 1.213 days 1-2), permanent 1.89% (2.87% on 1.213 days 1-2, but the mature 1.213 population is also at 1.9%).
- Browser-wide 1.22%, one-tab 0.68%.
- Idle wake 0.68%, tab not connected (P6) 0.42%, other 0.30%, extension updated 0.26%, recent load 0.19%.
- iOS: any failure 8.0%, permanent 0.44% (0.39% on 7.243 days 1-2).
- Browser-wide 0.26%, one-tab 0.18%.
- Recent load 0.18%, error 6 0.08%, idle wake 0.06%, extension updated 0.05%, other 0.04%.
- iOS is already under the 1% target. macOS stays above it after the Nov/9 fixes.
- Unaddressed on macOS: idle wake (35% of permanent) and P6 (22%). Nothing in flight for either.
- Extension install and load errors are rare (thousandths of a percent), not worth the time.
## Plan
- Clean up what did not work: remove the never-activated flags and experiment code (deallocation race, graveyard, late reload), the unused readiness pixels and the old debug pixels. Keep the kill-switch flags on fixes that change behavior until their effect is measured.
- Rebuild the metrics as a permanent small set: one episode report with a fixed format, rates against DAU, a sample of healthy navigations as baseline. Drop the tokens and flags that did not help. Publish old and new formats for one release to keep the trend.
- Add extension lifecycle metrics to diagnose the wake problem: where the init message stops (page, background, native checks, dashboard update, answer back to the page). Plus a check of the background state when a failure is reported. Needs a privacy review and extension and app releases, so data in 1-2 release cycles.
- Mitigations to decide on, not planned yet: retry the init from the page when no answer arrives, and a keep-warm A/B for the background. Both work without knowing the cause.
- Open: P6 has no fix, and the wake group will not be solved without the new metrics. If they show nothing actionable, close the project and reset the target.