---
tags:
  - cpm
  - experiment
  - ab-testing
status: ready-for-review
updated: '2026-09-22'
---
# CPM Background Graveyard — Experiment Definition

> Status: **ready for review; not posted to Asana**  
> Updated: 2026-09-22  
> Code milestone: M6 `alex/cpm-graveyard-ab`  
> Depends on: M3 telemetry → M5 mitigation

## Asana task fields

| Field | Value |
|---|---|
| Name — iOS | `iOS Mobile Experiment: [control, treatment] CPM background graveyard` |
| Name — macOS | `macOS Desktop Experiment: [control, treatment] CPM background graveyard` |
| Platform | Separate experiment-log task for iOS and macOS |
| Experiment Name | `cpmBackgroundGraveyardExperiment` |
| Cohorts | `control`, `treatment` |
| Start / end | TBD after sample-size review |
| Status | Draft until Privacy Config and privacy/schema approvals are complete |
| Project | CPM Web Extension breakage |
| Related CPM task | [1216761517055116](https://app.asana.com/1/137249556945/project/1211834678943996/task/1216761517055116) |
| Technical design | [1218005985662983](https://app.asana.com/1/137249556945/project/1211834678943996/task/1218005985662983) |
| A/B framework | [1208680035673270](https://app.asana.com/1/137249556945/project/1208889145294658/task/1208680035673270) |

## Background

A forced termination of the Web Extension background WebContent process reproduces a persistent all-tabs CPM outage. Reloading a tab or opening a new tab does not recover it; reloading the embedded extension does.

M5 retains at most one terminated background `WKWebView` for 3 seconds. The hold keeps the dead page/process proxy reachable while WebKit establishes a replacement service-worker context connection. The mitigation is disabled by default and has a separate runtime kill switch.

Production incidence of natural process termination and its causal connection to CPM outage must first be confirmed by M3 telemetry.

## Hypothesis

For eligible users whose CPM background WebContent process terminates, the treatment reduces the proportion of users with a subsequent CPM `initialization_failed` outcome, without increasing crashes, hangs, or unmeasured outcomes.

## Eligibility and enrollment

A user is enrolled only when all conditions are true:

1. Web Extensions are available: iOS 18.4+ or macOS 15.4+.
2. M5 kill switch `cpmBackgroundGraveyard` is enabled.
3. Experiment subfeature `cpmBackgroundGraveyardExperiment` is enabled and its target/rollout matches.
4. The current CPM background WebContent process actually terminates.
5. `resolveCohort` returns `control` or `treatment`.

Enrollment occurs at the first real exposure event—the process termination—not at app startup or privacy-config parsing. Assignment is stable through the native A/B framework.

## Variants

| Cohort | Behavior |
|---|---|
| `control` | Enroll and emit the same denominator/outcome telemetry; do not retain the terminated view. |
| `treatment` | Retain at most one terminated background view for 3 seconds; replace an older held view; release on timeout, context load/unload, or runtime kill-switch disable. |

Both cohorts share all code except the bounded hold. Pending outcome correlation is capped at 10 entries; overflow is finalized as `not_measured`.

## Metrics

### Primary experiment metric

Framework pixel:

- Experiment: `cpmBackgroundGraveyardExperiment`
- Metric: `cpmBackgroundGraveyardOutcome`
- Conversion window: `0...1` days from enrollment
- Predefined values: `healthy`, `initialization_failed`, `not_measured`
- Primary comparison: share of enrolled users with value `initialization_failed`, treatment vs control

The framework makes this user-level and unique by experiment/cohort/metric/window/value. A user may legitimately emit both `healthy` and `initialization_failed`; analysis must treat the primary result as “ever failed in the conversion window,” not an event rate.

### Operational diagnostics

- `debug_web_extension_cpm_background_graveyard_termination`: countable denominator by bounded termination reason and cohort.
- `debug_web_extension_cpm_background_graveyard_outcome`: countable per-termination outcome by reason, cohort, and outcome.
- Existing initialization-failure and stuck pixels add `background_graveyard_cohort`.
- Standard framework enrollment, app-use retention, and search retention pixels are automatic.

Operational pixels are required for event-rate and correlation diagnosis; the framework metric is required for the standard experiment analysis pipeline.

### Guardrails

1. `not_measured` coverage and termination/outcome mismatch.
2. Crash and hang rate by platform/cohort.
3. Treatment-only graveyard lifecycle: one held view maximum and bounded release.
4. Cohort balance and enrollment volume.
5. Standard app-use/search retention metrics.

## Success and stop criteria

Success criteria must be frozen before rollout:

- statistically meaningful reduction in user-level `initialization_failed`;
- no material regression in crash/hang or app-use/search retention;
- no unexplained cohort imbalance;
- acceptable `not_measured` rate;
- operational denominator/outcome coverage is sufficient for interpretation.

Data Science must set minimum sample size, minimum detectable effect, significance/power, and observation duration before public rollout.

To stop new enrollment, set both cohort weights to `0`. Do not delete a cohort unless reassignment is intentional.

Emergency rollback requires disabling **both**:

1. `cpmBackgroundGraveyardExperiment` — stops experiment activity/retention reporting.
2. `cpmBackgroundGraveyard` — immediately releases a held view and finalizes pending outcomes as `not_measured`.

Disabling only the M5 kill switch prevents new exposure enrollment but previously assigned users can remain active in the framework while the experiment subfeature is enabled.

## Privacy Config draft

Names are framework-compliant: alphanumeric camelCase, no underscores. Version and rollout values remain explicit TODOs; do not invent them in the config PR.

```json
{
  "features": {
    "cpmBackgroundGraveyard": {
      "state": "enabled",
      "description": "Emergency kill switch for bounded CPM background-view retention",
      "minSupportedVersion": "<platform-specific release version>"
    },
    "cpmBackgroundGraveyardExperiment": {
      "state": "internal",
      "description": "Control/treatment experiment for CPM recovery after background process termination",
      "minSupportedVersion": "<platform-specific release version>",
      "rollout": {
        "steps": [
          {
            "percent": "<approved enrollment percent>"
          }
        ]
      },
      "cohorts": [
        {
          "name": "control",
          "weight": 1
        },
        {
          "name": "treatment",
          "weight": 1
        }
      ]
    }
  }
}
```

Create separate iOS and macOS config/review paths because release versions, rollout timing, and analysis are platform-specific.

## Rollout

1. Ship M3 and validate natural termination → CPM failure correlation for one production release.
2. Land M5/M6 with both runtime flags off; verify pixels and forced-kill recovery internally.
3. Enable experiment for internal/review users at 50/50; verify assignment, enrollment, outcome coverage, and rollback.
4. Start public rollout only after Data Science freezes sample size/duration and privacy/schema review is approved.
5. At target sample, set cohort weights to 0; keep configuration active through the observation window, then decide treatment rollout or removal.

## Tasks to create — do not create from this note automatically

1. iOS experiment-log task from template [1209007672204094](https://app.asana.com/1/137249556945/project/1208889145294658/task/1209007672204094).
2. macOS experiment-log task from the same template.
3. Privacy Configuration change/review for the M5 kill switch and M6 experiment subfeature.
4. Data Science sample-size/design review plus Native Apps Experiment Analysis request in project [1209801922076585](https://app.asana.com/1/137249556945/project/1209801922076585).
5. Privacy triage/schema review for operational CPM pixels and the custom framework metric.

## Implementation readiness

- Lazy enrollment at actual exposure: implemented.
- Stable native cohort assignment: implemented.
- Control/treatment behavior isolation: implemented.
- Standard framework outcome metric: implemented.
- Countable operational diagnostics: implemented.
- Runtime kill switch and bounded cleanup: implemented.
- iOS/macOS compile, WebExtensions tests, and pixel schema validation: passed locally.
- Remaining before rollout: PRs, the five tasks above, config values, approvals, and production M3 evidence.