obsidian-vault/work/projects/cpm-service-worker-architecture-and-diagnostics.md

CPM Service Worker — architecture, diagnostics, failure gaps

Follow-up evidence — 2026-10-15. Current status, test outcomes and scope: cpm-status-2026-10-15 (§10–12); current worktree map: cpm-web-extension-code-guide-2026-10-09 (§0). The source audit below remains the September 25 baseline; its bundle version and worktree/milestone readiness are historical.

• The October harness uses bundled extension 2026.9.29, real production manager/coordinator/monitor/recorder/native handler and system WebKit on macOS 26.7.1. A temporary extracted bundle copy carries trace and DOM initResp marks; the production zip is unchanged. It does not run the entire browser app/AppKit.
• Browser window lifecycle is now tied to a measured WebKit side effect: the closing window still exposes a shared pinned tab, so didCloseWindow internally forgets it despite the browser filtering explicit didCloseTab calls. Subsequent messages fail tab resolution before wake; exact P5/tab=false/stuck is reproduced. This does not explain P5/tab=true or iOS.
• Main webExtensions flag cycling through the production handler recreates the controller and produces P6 on the old view. Actual production flag transitions remain unproven; repeated true and embedded-only cycling preserve controller identity. Embedded reinstall can still lose its first message before ready (known A); the healthy binding control explicitly waits for background readiness.
• The no-listener-from-first-start experiment produces P5 with healthy-looking tab flags, but only under test-only suppression. One lost registration on a later wake recovered after restoring the script. The current bundled bootstrap creates MessageRouter and its runtime listener synchronously before its trailing Promise.all diagnostics; failure of those later async diagnostics alone is not evidence that listener registration was skipped.
• Two slow 35-second documents receive initResp; report/eval at finish wake the background again. Suppressing dashboard delivery yields actual detector stuck despite page receipt, with a live background. Two 35-second host main-actor stalls delay failure snapshots but also leave a live view; this is not a background-worker busy loop or real iOS suspension.
• All 14 current P5/P6 methods have passing outcomes across separate runs; no single final 14-test run. Latest ordinary package: 478 XCTest / 28 opt-in skipped / 0 failures. Priorities involving real iOS lifecycle, stable-origin teardown, natural metadata loss, request cancellation and stale-document replies are not closed by these checks. Native init_stage and its JS counterpart remain open PRs (#7385 / privacy-extension#3727), not assumed shipped.

2026-09-25 · source audit, not a new device reproduction. Start with §4: the five validation priorities. Current assessment supersedes the architecture/diagnostics assumptions in cpm-web-extension-breakage-findings. Milestone status remains in cpm-web-extension-breakage-validation.

Answer: no, we do not measure every useful signal. Existing telemetry describes the extension context and hosting process reasonably well, but cannot establish worker readiness, successful event delivery, native-request progress, or delivery of initResp to the current document. Some missing signals are implementable; others are WebKit internals or test-only facilities.

1. What the Service Worker is in CPM

The bundle audited on September 25 was MV3, version 2026.9.2; the October 15 harness uses 2026.9.29. Its classic service worker is public/js/background-embedded.js; public/js/content-scripts/cpm.js runs at document start in an isolated world in all matching frames. A background worker is shared by the tabs using that extension context; it is not one worker per tab and not necessarily one across every profile/data store.

It is the extension's background JavaScript environment: message router, native-settings/configuration access, CPM rule selection and replies. It has a worker global, not the page DOM. CPM uses extension messaging—not the website Service Worker fetch/offline-cache path.

Page/frame: isolated CPM content script
  runtime.sendMessage(init)
    → WebKit UI process: resolve tab → check listener/wake background
      → background Service Worker: runtime.onMessage
        ↔ Swift native-messaging handlers: settings, configuration, dashboard
      → tabs.sendMessage(initResp, frameId)
    → current page/frame must actually receive and apply the reply

Network process: SW registrations, jobs, clients and worker-process connections
Hidden WKWebView: host page connecting the extension context to its worker global

Important WebKit-specific detail: the hidden view registers the worker through a generated navigator.serviceWorker.register(...) page. For workers backed by such a page, this checkout selects WorkerThreadMode::UseMainThread. JavaScript executes on the hosting WebContent process's main thread, despite having a separate worker global. Extension bindings are installed by locating that host page inside the same process.

Consequently, these are different states:

Observable	What it actually proves
context.isLoaded	Context is associated with its controller; asynchronous setup may still be pending.
Hidden view exists / nonzero PID	A host object/process identifier exists—not a usable worker.
SW registration is active	Registration lifecycle state—not live event-loop progress or extension bindings.
Background load completed	WebKit's registration/load path settled—not successful CPM round-trip.
CPM dashboard response	Background/native path answered—not necessarily initResp delivery or consent handling.

Lifecycle: normal nonpersistent background eviction is scheduled after 30 seconds of idle time. Repeated wake-ups, ports, permission requests or inspection can prevent/delay it. Process termination closes the host; a nonpersistent background is recreated on demand. Failed registration records error 6 and can leave the host and pending actions retained until eviction. A retry only helps if its underlying cause is gone.

Sources: background host lifecycle, registration page, main-thread selection, page-dependent worker binding.

2. What we measure, what is missing

Existing coverage: useful but mostly indirect

Area	Current signal	Limitation
Context/errors	context_loaded, context_errors including immediate underlying domain/code	Errors combine historical and current state. WebKit deduplicates errors; neither array length nor notifications count failures reliably.
Background lifecycle	bg_view_creations, bg_view_alive, bg_view_leaked_count, bg_process_alive, bg_process_responsive, bg_events	Object/PID and cached responsiveness are not worker readiness. Timeline records load failures, deaths, responsiveness and retention, not message progress.
Routing prerequisites	tab_in_context, handler_registered, tab_controller_match, tab_has_ext_scripts	Membership/registration/script metadata do not prove actual injection, handler execution or reply delivery.
Environment	app_version_change, app_launch_age, critical_memory_age, network_restarted	Correlation, not causation. Network PID comparison is not a worker-connection health check.
Experiment/outcome	Cohort, termination/outcome events, init-failure/stuck/recovery events	Current “healthy” means a valid dashboard response. It is not an end-to-end CPM-success metric.

Two measurement confounds are source-confirmed:

• Intentional retention can count as a leak. The leak calculation is live weak-tracked views minus one expected current view. During overlap with a replacement, the graveyard's deliberately retained old view can increase bg_view_leaked_count.
• The timeline cannot resolve the 3-second race. Its first age bucket is 30 seconds; it retains at most 12 events/255 characters. Ordering helps, but local monotonic timestamps and context/view generations are needed for causal timing.

The recorder's weak “current view” also survives while another owner retains it. Comparing it with WebKit's actual current _backgroundWebView would distinguish an intentionally retained, closed host from the context's active host.

Useful additions within our control

Missing evidence	Proposed measurement	Boundary
Worker boot / listener readiness	Worker-generated boot + listener-installed marker, associated locally with a context/worker generation	An application protocol addition, not an existing native WebKit readiness callback. Absence alone cannot distinguish failed boot from failed native delivery.
Request progress	Host-side arrival/completion/error, outstanding-count and oldest-request-age buckets per allowlisted native method	Independent of the JS diagnostic channel. No URLs, payloads, arbitrary method strings or raw error text in pixels.
Actual initialization delivery	Document-scoped init/reply acknowledgement; distinguish background-received, reply-sent and page-acknowledged	Must reject late responses from previous navigation/context generations. Do not rename this metric “consent handled.”
Script/load failure phase	Separate current vs previously observed error codes; classify known boot/runtime/native stages	Code 6 is generic load failure. Code 7 is script execution error in this checkout; OS support needs verification.
Worker exception contract	Handle the bundle's autoconsent.extensionException with bounded, privacy-reviewed categories	Confirmed mismatch: bundle emits it; Swift Method has no matching case and returns unknown-method failure.

The exception mismatch does not mean all errors are invisible: WebKit's own error channel may report some of them. Nor will fixing it catch everything: the bundle's extension polyfill can throw before its global error/rejection listeners are installed, and the exception notification itself uses the same potentially failing native channel.

WebKit diagnostics that are not drop-in production telemetry

Facility	What it offers	Why it needs care
Public loadBackgroundContent(completionHandler:)	Explicit attempt to wake/load background	Active intervention. Changes idle timing and can repair/perturb the episode. Completion queues behind load and may remain pending on failure. Use only a deliberate debug probe with timeout/generation protection.
Private _backgroundWebView, _backgroundContentURL; context error updates	Current host identity and failure information	Availability-gate SPIs. Do not upload raw URLs/descriptions/stacks. Preserve WebKit's delegate behavior.
Worker Inspector + ServiceWorker/Extensions logs	Worker global, exceptions, breakpoints, registration/job/connection/PID evidence	Inspector changes worker lifetime. Reproduce detached; use inspection afterward or in a separate run. This checkout's Inspector ServiceWorker protocol is not Chromium's full lifecycle protocol.
_runningOrTerminatingServiceWorkerCountForTesting; datastore console callback	Aggregate test count; conditional console forwarding	Count is not CPM-specific. Console forwarding is disabled by default; the enabling call found in this tree is in ServiceWorkerInternals testing code. Implementing the datastore delegate alone does not enable a production worker-console stream.
SWServer maps/queues, registration-worker-host affinity, listener destinations, queued wake-up actions	The decisive internal state for several failures	No passive public native accessor found. Requires existing internal logs or an instrumented WebKit build; do not invent KVC access to C++ state.

Two tempting probes are invalid:

• The generic worker heartbeat is disabled for WorkerMainRunLoop. It cannot serve as CPM's extra 60-second watchdog. Existing UI-side process-responsiveness monitoring is a different mechanism.
• evaluateJavaScript on the hidden view evaluates a frame/document global, not the Service Worker's global. Missing browser there does not prove missing worker bindings. A registration query likewise does not prove the worker can answer.

Sources: error recording, worker script-error callback, active load API implementation, heartbeat exclusion, conditional console forwarding, test-only enablement.

3. The process-death failure and what graveyard actually changes

Historical evidence: the September 10 investigation recorded forced background-process death causing the all-tabs registration-error loop. This audit checked the supporting source chain; it did not repeat that device test or establish natural production incidence.

Background process dies
 → Network closes SW connection before unregistering that process's clients
 → remaining clients cause a connection request referencing the old host page
 → if old WebProcessProxy is gone, process selection may create a standalone worker process
 → new registration reuses that domain's existing connection
 → worker cannot find its host page in this other process
 → browser/chrome bindings are not installed
 → bundle's initial polyfill throws
 → registration fails → error 6 → later retries may repeat

An existing SW connection is selected by domain/COEP rather than by the new host page's process affinity. Merely creating a fresh background view therefore does not necessarily repair the connection.

Graveyard treatment retains the terminated WKWebView and its native proxy object graph. The old process proxy can still resolve when WebKit services the stale connection request. In this source branch, the initial host-page process lookup does not exclude a terminated proxy. The intended effect is to make that stale request fail/cancel instead of taking the standalone-process fallback while old clients drain. It does not revive the worker or fix its JavaScript/native handler.

The exact winning timing remains an experimental hypothesis, not a WebKit guarantee. Current implementation holds one view with a 3-second scheduled release. A second death replaces it immediately; main-queue blockage/app suspension can delay release beyond three wall-clock seconds. Context transitions also release it.

M2 changes the recovery assumption. The current M5 loader still uses the default changing origin; separate M2 assigns webkit-extension://com.duckduckgo.web-extension.embedded/. After M2, extension reload cannot rely on escaping a poisoned connection via a new origin. Fire may also clear data, so its behavior must be tested separately—not assumed equivalent to reload.

Sources: connection teardown order, connection recreation, process selection, existing connection reuse.

4. Five validation priorities

These are under-tested paths, not five newly proven root causes. Several broad hypotheses existed in the old note; this audit adds concrete mechanisms and discriminating tests.

1. Stable-origin reload and old-registration teardown — highest integration risk

Source-confirmed paths; combined race unproven. Registration can immediately reuse an existing same-script/type/cache registration without rebinding its host-page identifier in that branch. Old host-client removal separately clears its associated registration. A rapid same-origin replacement may therefore reuse a registration still owned by the old page, which later gets cleared. Separately, the poisoned domain connection can survive reload.

Test: combine M2 with M5+M6; force background death in control/treatment, then test extension reload and Fire separately. Also rapidly unload/reload a healthy context without killing it. Capture register/reuse/unregister order, host/worker PID affinity and page acknowledgement. Require no late old-generation teardown to invalidate the new worker.

Sources: reusable registration fast path, old page unregister.

2. Main-thread stall, event flood and missing startup deadline

Heartbeat exclusion confirmed; production trigger unproven. A genuine synchronous background stall cannot be detected by the disabled worker heartbeat. Main-thread JS/native work, repeated all-frame init traffic, or a registration that never settles can leave a live-looking host with no progress. Repeated events can prevent idle eviction; retained load actions have no CPM-owned deadline. An unresolved Promise alone is not a blocked event loop.

Test: separately inject a controlled background busy loop, delayed registration and sustained frame churn. Stop the producer and check drain/recovery. Capture process responsiveness, startup age, native outstanding work and page acknowledgements. Test iOS suspend/resume separately from foreground hangs. Ordinary page JS is only relevant if it actually shares the affected process; do not assume it blocks the extension.

3. Native errors/timeouts hidden by the diagnostic transport

Contract mismatch and 20-second timeout confirmed; natural stall unproven. The worker races native requests against 20 seconds, but does not cancel native work. Failure can become undefined and disabled settings. Exception/diagnostic notifications travel over that same native channel, and extensionException is currently unsupported. Session-storage/configuration operations can also fail before a usable init reply.

Test: independently fail/delay settings, feature and config calls; reject session-storage access; inject a runtime exception. Assert host-side arrival/completion/timeout evidence, bounded outstanding work and explicit failure stages. Repeated frame churn must not leave unlimited old native requests.

4. “Healthy” before response delivery; stale document replies

Source-confirmed measurement gap; user-visible race needs reproduction. The background schedules dashboard stage init_received before rule filtering and tabs.sendMessage(initResp). Swift credits a valid dashboard response as healthy without requiring that stage to prove page receipt. The reply send is not awaited/caught here and targets frameId, not a document identifier. Navigation while native work is pending can drop the reply or target a newer document.

Test: delay configuration, navigate the same frame, then release the reply; separately reject the reply send or throw during rule filtering. Distinguish “background answered” from “current document initialized.” Require document/generation attribution before claiming end-to-end recovery.

5. Registration failure vs listener-routing failure

Both source paths exist; no new production transition demonstrated. Generic error 6 merges resource/script startup failures with wrong-process binding failure. Separately, persisted listener metadata and live listener-frame destinations can disagree: WebKit can skip wake-up or have no usable receiver despite a loaded-looking context.

Test: independently inject a missing/throwing background resource and a controlled no-listener case; compare cold launch, normal idle eviction and context reload. Capture the load result, boot/listener marker and request arrival. In an instrumented WebKit build inspect listener sets/destinations and pending actions. Do not infer a listener bug from PID/alive alone.

5. What not to conclude; next implementation boundary

Deprioritized or disproven explanations: this WebKit checkout explicitly does not persist extension-backed SW registrations to disk, so ordinary website SW-database corruption across relaunch is not the leading explanation. Extension listener-state/session/local storage are separate and still deserve validation. removeAllUserScripts() excludes WebExtension scripts. Earlier negative memory-pressure/Network-restart tests remain negative on their tested OS—not universal proof those paths cannot fail.

Recommended order: fix the exception-reporting contract and define request/document-stage evidence first; run priority 1 with stable origin before relying on reload as recovery; then exercise priorities 2–5. Instrumentation must preserve control/treatment behavior, avoid unsolicited wake-ups, keep raw data local and receive normal pixel/privacy review.

Scope and evidence: WebKit checkout 0c7e7ad97b9c50e549387914d31ecc2908ae0171 (2026-08-23), tracked sources clean. App audit used the current dirty cpm-graveyard-mitigation worktree (HEAD 4de02983986c47769e4e5ce0d267c23f3ca8d48c) and inspected M2 separately in cpm-stable-base-url. The old cpm-breakage-pixel worktree is not the current M5+M6 implementation. Exact behavior/SPI availability must be checked on supported iOS and macOS releases; trunk source is not proof of the shipped OS binary.

App evidence is in SharedPackages/WebExtensions/Sources/WebExtensions/:

Source	Relevant evidence
BundledWebExtensions/duckduckgo-embedded-web-extension.zip	Manifest; background JS lines 64, 2645–2693, 2955–2999, 3300–3347: early polyfill, dashboard-before-reply, native timeout, exception notifications.
Messaging/MessageHandlers/AutoconsentWebExtensionMessageHandler.swift	Methods 30–41 omit extensionException; 68 rejects unknown method; dashboard response forwarded without requiring initResp acknowledgement.
Autoconsent/CPMMessagingHealthMonitor.swift	Healthy outcome at 442/680 represents dashboard response, with existing tab/generation safeguards.
Autoconsent/CPMMessagingDiagnostics{,Recorder}.swift	Pixel coverage, 30-second minimum timeline age bucket, historical error accumulation, leak-count calculation at recorder 506.
Autoconsent/CPMBackgroundWebViewGraveyard.swift	Dedicated retention entity, one retained view, 3-second scheduling, replacement/reset behavior and experiment outcomes.

Additional source: extension workers are not persisted.

Validation status of the September 25 audit: source tracing and bundle/Swift contract comparison completed. No new device reproduction, build, unit test or app-code change was performed for this documentation audit.