[2026-11-03] eagle: work/projects/cpm-project-summary-2026-11-03.md

This commit is contained in:
Alexey Martemyanov committed 2026-11-03 18:22:34 +06:00
1 parent 68e617a17f
commit b6381351be
1 file changed
+17 -24
+17 -24
View File
@@ -10,25 +10,18 @@ status: draft
Internal metrics, keep out of public PRs. Task: [CPM: root-cause the stuck embedded extension messaging](https://app.asana.com/1/137249556945/project/72649045549333/task/1216761517055116?focus=true). Details: [[cpm-post-release-analysis-2026-10-27]], [[cpm-status-2026-10-15]]. Internal metrics, keep out of public PRs. Task: [CPM: root-cause the stuck embedded extension messaging](https://app.asana.com/1/137249556945/project/72649045549333/task/1216761517055116?focus=true). Details: [[cpm-post-release-analysis-2026-10-27]], [[cpm-status-2026-10-15]].
Goal: permanent stuck reports below 1% of DAU (0.2-0.5% counts as success).
## What the project has achieved ## What the project has achieved
Success criteria: permanent stuck reports below 1% of DAU (0.2-0.5% counts as success). Permanent = no response for 5 minutes, confirmed by a repeat or a reload. Success criteria: permanent stuck reports below 1% of DAU (0.2-0.5% counts as success). Permanent = no response for 5 minutes, confirmed by a repeat or a reload.
Progress against it (1.213 / 7.243 days 1-2 → 1.214 / 7.244 day 1): Progress against it (now = 1.214 / 7.244, Nov/2):
- iOS: 0.43% → 0.44% of DAU. Inside the target. - iOS: 0.44% of DAU (0.43% on 7.243). Inside the target.
- macOS: 2.87% → 1.89% of DAU (any failure 28.5% → 15.1%). Target not reached yet. Nov/9 fixes bring it to about 1.6%. - macOS: 1.89% of DAU. Target not reached yet. Nov/9 fixes should bring it to about 1.6% (estimate, upper bound).
Breakage measurement, built and corrected:
- Production telemetry for the stuck extension: one episode report with failure groups, tab scope, repeat count, and rates against DAU. Before it we could not separate stuck from recovered failures or attribute a cause. [Improve telemetry](https://app.asana.com/1/137249556945/task/1219531810004069?focus=true), [TD: failure pixels attribution](https://app.asana.com/1/137249556945/task/1218619301158188?focus=true).
- Error pages were reported as breakages on macOS. Fixed, so the macOS figure is now accurate: any failure 28.5% → 15.1%, permanent 2.87% → 1.89% of DAU. [Fix false-positive breakage reports for error pages](https://app.asana.com/1/137249556945/task/1220003778559452?focus=true).
Breakage fixes shipped, with what each did: Breakage fixes shipped, with what each did:
- Stable extension origin ([P3: stable baseURL fix](https://app.asana.com/1/137249556945/task/1219643764290826?focus=true), [Disable stable baseURL fix for OS<26.6](https://app.asana.com/1/137249556945/task/1220003778559448?focus=true), OS 26.6+): removes a 1.5 to 32 s startup blackout after an origin change. iOS 26.6+ startup-missing and no-view reports down over 90%. - Stable extension origin ([P3: stable baseURL fix](https://app.asana.com/1/137249556945/task/1219643764290826?focus=true), [Disable stable baseURL fix for OS<26.6](https://app.asana.com/1/137249556945/task/1220003778559448?focus=true), OS 26.6+): in a lab reproduction an origin change delays extension startup by 1.5 to 32 s depending on cache size, a stable origin by 0.1 s. Not measured in production. iOS 26.6+ startup-missing and no-view reports are down over 90% vs 7.243 (cohort comparison, P3 state not visible in the pixels).
- macOS pinned tab lost on window close ([P5: pinned tab after windowDidClose](https://app.asana.com/1/137249556945/task/1219643712648486?focus=true)) and tab merging ([Fix Tab Merging causing CPM failure](https://app.asana.com/1/137249556945/task/1218356681060823?focus=true)): reproduced and fixed. P5 tests 9 pass with fix → 5 expected failures with it off → 27 pass. - macOS pinned tab lost on window close ([P5: pinned tab after windowDidClose](https://app.asana.com/1/137249556945/task/1219643712648486?focus=true)) and tab merging ([Fix Tab Merging causing CPM failure](https://app.asana.com/1/137249556945/task/1218356681060823?focus=true)): reproduced and fixed. P5 tests 9 pass with fix → 5 expected failures with it off → 27 pass.
- Wake and startup recovery ([P1&P2[&P4]: Fix init loss on background unload](https://app.asana.com/1/137249556945/task/1219531810004072?focus=true)): applied to 2,026 iOS / 89 macOS episodes on Oct/26. - Wake and startup recovery ([P1&P2[&P4]: Fix init loss on background unload](https://app.asana.com/1/137249556945/task/1219531810004072?focus=true)): applied to 2,026 iOS / 89 macOS episodes on Oct/26.
- Network process restart ([Fix CPM breakage on network process crash](https://app.asana.com/1/137249556945/task/1219531810004073?focus=true)): applied 476 iOS / 277 macOS times on Oct/26. - Network process restart ([Fix CPM breakage on network process crash](https://app.asana.com/1/137249556945/task/1219531810004073?focus=true)): applied 476 iOS / 277 macOS times on Oct/26.
@@ -37,7 +30,7 @@ Breakage fixes shipped, with what each did:
Causes proved by reproduction: Causes proved by reproduction:
- iOS error-6 loop: stale failed background ([WebExtensions breakage harness](https://app.asana.com/1/137249556945/task/1219531810004071?focus=true)). Fixed by the Nov/9 error 6 restart. - iOS error-6 loop: stale failed background ([WebExtensions breakage harness](https://app.asana.com/1/137249556945/task/1219531810004071?focus=true)). Fixed by the Nov/9 error 6 restart.
- Origin change cleaning the cache before startup (fixed by P3). - Origin change delaying extension startup (lab reproduction with real cache sizes; production impact not measured).
- macOS P6: a tab without a controller never answers. 0.42% of DAU, no fix yet. - macOS P6: a tab without a controller never answers. 0.42% of DAU, no fix yet.
Hypotheses tested and rejected: Hypotheses tested and rejected:
@@ -62,7 +55,7 @@ Hypotheses tested and rejected:
One day, early adopters, macOS has about 9k DAU. Treat as a rough picture. One day, early adopters, macOS has about 9k DAU. Treat as a rough picture.
- Permanent stuck = no response for 5 minutes, confirmed by a repeat or a reload. Browser-wide = 2+ tabs, one-tab = single tab. - Permanent stuck = no response for 5 minutes, confirmed by a repeat or a reload. Browser-wide = 2+ tabs, one-tab = single tab.
- macOS: any failure 15.1%, permanent 1.89%. The earlier 28.5% / 2.87% on 1.213 included error pages, so the drop is the metric fix. Mature 1.213 is also at ~1.9%. - macOS: any failure 15.1%, permanent 1.89%.
- Browser-wide 1.22%, one-tab 0.68%. - Browser-wide 1.22%, one-tab 0.68%.
- Idle wake 0.68%, tab not connected (P6) 0.42%, other 0.30%, extension updated 0.26%, recent load 0.19%. - Idle wake 0.68%, tab not connected (P6) 0.42%, other 0.30%, extension updated 0.26%, recent load 0.19%.
- iOS: any failure 8.0%, permanent 0.44% (0.39% on 7.243 days 1-2). - iOS: any failure 8.0%, permanent 0.44% (0.39% on 7.243 days 1-2).
@@ -78,19 +71,19 @@ Cleanup, this week:
- [Feature flags cleanup](https://app.asana.com/1/137249556945/task/1220159402040629?focus=true) (task is empty in Asana, content below). Registry tasks per flag: - [Feature flags cleanup](https://app.asana.com/1/137249556945/task/1220159402040629?focus=true) (task is empty in Asana, content below). Registry tasks per flag:
- Remove the fix and the flag, they did not work or never ran: - Remove the fix and the flag, they did not work or never ran:
- [cpmBackgroundGraveyardTreatment](https://app.asana.com/1/137249556945/task/1218860893478440?focus=true): graveyard mitigation, not supported by data. - [cpmBackgroundGraveyardTreatment](https://app.asana.com/1/137249556945/task/1218860893478440?focus=true): graveyard mitigation, not supported by data.
- [cpmLateBackgroundDocumentRecovery](https://app.asana.com/1/137249556945/task/1219588497257482?focus=true): late page reload, default off, 0 fires. - [cpmLateBackgroundDocumentRecovery](https://app.asana.com/1/137249556945/task/1219588497257482?focus=true): late page reload, default off, 0 fires.
- [cpmMessagingHangRecovery](https://app.asana.com/1/137249556945/task/1218855001659655?focus=true): reload on the stuck signal, 46% still no response; decide keep or remove. - [cpmMessagingHangRecovery](https://app.asana.com/1/137249556945/task/1218855001659655?focus=true): reload on the stuck signal, 46% still no response; decide keep or remove.
- [cpmBackgroundLoadRecovery](https://app.asana.com/1/137249556945/task/1219701603183698?focus=true): P1/P2 wake and retry, no measurable effect. - [cpmBackgroundLoadRecovery](https://app.asana.com/1/137249556945/task/1219701603183698?focus=true): P1/P2 wake and retry, no measurable effect.
- Remove the diagnostics flags together with the diagnostics layer: - Remove the diagnostics flags together with the diagnostics layer:
- [cpmDiagnosticsRecorder](https://app.asana.com/1/137249556945/task/1218785444896174?focus=true): recorder and unresponsive token. - [cpmDiagnosticsRecorder](https://app.asana.com/1/137249556945/task/1218785444896174?focus=true): recorder and unresponsive token.
- [cpmBackgroundDelegateProxy](https://app.asana.com/1/137249556945/task/1218782530802146?focus=true): delegate proxy for diagnostics. - [cpmBackgroundDelegateProxy](https://app.asana.com/1/137249556945/task/1218782530802146?focus=true): delegate proxy for diagnostics.
- Keep the fix, remove the failsafe flag once its effect is measured: - Keep the fix, remove the failsafe flag once its effect is measured:
- [cpmNetworkProcessRecovery](https://app.asana.com/1/137249556945/task/1219701603183695?focus=true): network restart. - [cpmNetworkProcessRecovery](https://app.asana.com/1/137249556945/task/1219701603183695?focus=true): network restart.
- [cpmStableBaseURL](https://app.asana.com/1/137249556945/task/1219728649844711?focus=true): stable origin; flag state is not visible in pixels, check the remote config first. - [cpmStableBaseURL](https://app.asana.com/1/137249556945/task/1219728649844711?focus=true): stable origin; flag state is not visible in pixels, check the remote config first.
- [webExtensionSharedPinnedTabWindowClose](https://app.asana.com/1/137249556945/task/1219660990760563?focus=true): P5. - [webExtensionSharedPinnedTabWindowClose](https://app.asana.com/1/137249556945/task/1219660990760563?focus=true): P5.
- [webExtensionStateRestorationGate](https://app.asana.com/1/137249556945/task/1218931841045393?focus=true): session restoration race. - [webExtensionStateRestorationGate](https://app.asana.com/1/137249556945/task/1218931841045393?focus=true): session restoration race.
- [cpmBackgroundLoadRestart](https://app.asana.com/1/137249556945/task/1219970564251181?focus=true): error 6 restart, after Nov/9 data. - [cpmBackgroundLoadRestart](https://app.asana.com/1/137249556945/task/1219970564251181?focus=true): error 6 restart, after Nov/9 data.
- [Pixel data cleanup](https://app.asana.com/1/137249556945/task/1220159402040630?focus=true) (task is empty in Asana, content below): - [Pixel data cleanup](https://app.asana.com/1/137249556945/task/1220159402040630?focus=true) (task is empty in Asana, content below):
- Remove the treatment pixels that only say "applied": background_wake, background_load_retry, install/upgrade readiness, late_document_reload, network_process_recovery_applied, startup_response_missing. - Remove the treatment pixels that only say "applied": background_wake, background_load_retry, install/upgrade readiness, late_document_reload, network_process_recovery_applied, startup_response_missing.
- Remove the temporary episode parameters that only served the investigation (bg_events and the other debug fields), after the new metrics are live. - Remove the temporary episode parameters that only served the investigation (bg_events and the other debug fields), after the new metrics are live.