From 7f92af3f6010a669d7494178130086d8ec7a1492 Mon Sep 17 00:00:00 2001 From: Alexey Martemyanov Date: Tue, 3 Nov 2026 20:19:31 +0600 Subject: [PATCH] [2026-11-03] eagle: work/projects/cpm-project-summary-2026-11-03.md --- .../cpm-project-summary-2026-11-03.md | 48 +++++++++---------- 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/work/projects/cpm-project-summary-2026-11-03.md b/work/projects/cpm-project-summary-2026-11-03.md index 16f1bd87..b81e38ac 100644 --- a/work/projects/cpm-project-summary-2026-11-03.md +++ b/work/projects/cpm-project-summary-2026-11-03.md @@ -10,16 +10,16 @@ status: draft Internal metrics, keep out of public PRs. Task: [CPM: root-cause the stuck embedded extension messaging](https://app.asana.com/1/137249556945/project/72649045549333/task/1216761517055116?focus=true). Details: [[cpm-post-release-analysis-2026-10-27]], [[cpm-status-2026-10-15]]. -## What the project has achieved +**What the project has achieved so far** Goal: permanent failures below 1% of DAU (0.2-0.5% = success). Permanent = no answer for 5 minutes, repeated or needed a reload. -### Now (macOS 1.214, iOS 7.244, Nov/2) +**Now (macOS 1.214, iOS 7.244, Nov/2)** -- iOS: 0.44% of DAU (0.39% on 7.243, same day). Inside the target. -- macOS: 1.89%. Above the target. +- iOS: 0.44% of DAU (0.39% on 7.243, same day). Inside the target <<1%. +- macOS: 1.89%. Above the target <<1%. -### Fixes shipped +**Fixes shipped** - [P3](https://app.asana.com/1/137249556945/task/1219643764290826?focus=true), [OS<26.6 limit](https://app.asana.com/1/137249556945/task/1220003778559448?focus=true) – improves extension startup after a restart from up to 32 s → 0.1 s on iOS/macOS 26.6+ (lab tests). iOS 26.6+: "background missing" failures 8.2% → 0.3% of DAU (users with any failure 14.6% → 8.3%). Effect on permanent not measurable yet (one day, 90k users). - macOS: [pinned tab lost on window close](https://app.asana.com/1/137249556945/task/1219643712648486?focus=true), [tab merging](https://app.asana.com/1/137249556945/task/1218356681060823?focus=true) – real bugs fixed, UI tests added. They caused under 1% of permanent failures, about 3x fewer after the fix (macOS 1.211 → 1.213). @@ -28,19 +28,19 @@ Goal: permanent failures below 1% of DAU (0.2-0.5% = success). Permanent = no an - [Network process crash](https://app.asana.com/1/137249556945/task/1219531810004073?focus=true) – restart after the networking process crashes. Triggers for 6.6% of iOS and 4.2% of macOS daily users (Nov/2). - [State restoration race](https://app.asana.com/1/137249556945/task/1218744813699966?focus=true) – restored tabs wait up to 5 s for the extension to start. Init failures on restore: iOS ~20% → ~8.5% of DAU, macOS no clear change. -### Hypotheses rejected +**Hypotheses rejected** - [Deallocation race](https://app.asana.com/1/137249556945/task/1217907325352233?focus=true) ([A/B](https://app.asana.com/1/137249556945/task/1218356681060819?focus=true), [flag](https://app.asana.com/1/137249556945/task/1219036128743441?focus=true)) – fix shipped behind a flag never turned on, no relation in the data. - [Reload on the stuck signal](https://app.asana.com/1/137249556945/task/1218744813699968?focus=true) – 46% of macOS cases still got no answer. - [P4](https://app.asana.com/1/137249556945/task/1219531810004072?focus=true) – reload the page when the extension starts late. Off by default, never triggered in production. -## Fixes going public next week (Nov/9) +**Fixes going public next week (Nov/9)** - [Upgrade before loading](https://app.asana.com/1/137249556945/task/1220004977087625?focus=true) – load the new extension version first on launch. Removes part of the permanent failures right after an update (macOS 0.26% of DAU, iOS 0.05%). - [Error 6 restart](https://app.asana.com/1/137249556945/task/1219939142983214?focus=true) – restart the extension when a load error leaves it broken. Up to 0.08% of DAU permanent on iOS, negligible on macOS. - Together: macOS 1.89% → about 1.6%, iOS 0.44% → about 0.3-0.35%. Estimates, upper bounds. -## Data now (% of DAU) +**Data now (% of DAU)** Permanent failures, daily trend (macOS 1.213 / iOS 7.243: Oct/26 → Nov/2; 1.214 / 7.244: Nov/2): @@ -49,25 +49,25 @@ Permanent failures, daily trend (macOS 1.213 / iOS 7.243: Oct/26 → Nov/2; 1.21 By group on the new version (1.214 / 7.244, Nov/2), % of DAU. Trend is failures from the first day of the previous version (1.213 / 7.243, Oct/26) to the new version. -| Group | Platform | Failures, % of DAU | % of all failures | Permanent, % of DAU | % of all permanent | Failures trend, % of DAU | -| --------------------------- | -------- | --------- | ----------------- | --------- | ------------------ | ----------------- | -| Init on wake | macOS | 5.89% | 39% | 0.68% | 36% | 8.80% → 5.89% | -| Init on wake | iOS | 1.74% | 22% | 0.06% | 15% | 1.56% → 1.74% | -| Tab not connected | macOS | 1.26% | 8% | 0.42% | 22% | 1.18% → 1.26% | -| Init after extension update | macOS | 2.94% | 20% | 0.26% | 14% | 5.40% → 2.94% | -| Init after extension update | iOS | 0.34% | 4% | 0.05% | 12% | 0.37% → 0.34% | -| Init after load | macOS | 1.73% | 11% | 0.19% | 10% | 2.73% → 1.73% | -| Init after load | iOS | 2.54% | 32% | 0.18% | 42% | 1.86% → 2.54% | -| Error 6 | macOS | 0.11% | 1% | 0.01% | 1% | 0.14% → 0.11% | -| Error 6 | iOS | 0.67% | 8% | 0.08% | 18% | 0.50% → 0.67% | -| Background missing | macOS | **0.20%** | 1% | 0.03% | 2% | 1.06% → **0.20%** | -| Background missing | iOS | **0.27%** | 3% | 0.01% | 3% | 0.63% → **0.27%** | -| Other | macOS | 2.92% | 19% | 0.30% | 16% | 4.75% → 2.92% | -| Other | iOS | 2.45% | 31% | 0.04% | 10% | 1.95% → 2.45% | +| Group | Platform | Failures, % of DAU | % of all failures | Permanent, % of DAU | % of all permanent | Failures trend, % of DAU | +| --------------------------- | -------- | ------------------ | ----------------- | ------------------- | ------------------ | ------------------------ | +| Init on wake | macOS | 5.89% | 39% | 0.68% | 36% | 8.80% → 5.89% | +| Init on wake | iOS | 1.74% | 22% | 0.06% | 15% | 1.56% → 1.74% | +| Tab not connected | macOS | 1.26% | 8% | 0.42% | 22% | 1.18% → 1.26% | +| Init after extension update | macOS | 2.94% | 20% | 0.26% | 14% | 5.40% → 2.94% | +| Init after extension update | iOS | 0.34% | 4% | 0.05% | 12% | 0.37% → 0.34% | +| Init after load | macOS | 1.73% | 11% | 0.19% | 10% | 2.73% → 1.73% | +| Init after load | iOS | 2.54% | 32% | 0.18% | 42% | 1.86% → 2.54% | +| Error 6 | macOS | 0.11% | 1% | 0.01% | 1% | 0.14% → 0.11% | +| Error 6 | iOS | 0.67% | 8% | 0.08% | 18% | 0.50% → 0.67% | +| Background missing | macOS | **0.20%** | 1% | 0.03% | 2% | 1.06% → **0.20%** | +| Background missing | iOS | **0.27%** | 3% | 0.01% | 3% | 0.63% → **0.27%** | +| Other | macOS | 2.92% | 19% | 0.30% | 16% | 4.75% → 2.92% | +| Other | iOS | 2.45% | 31% | 0.04% | 10% | 1.95% → 2.45% | 1.214 / 7.244 is one day of early adopters (macOS about 9k DAU, iOS 110k), a rough picture. -## Plan +**Plan** This week, cleanup: