diff --git a/work/projects/cpm-web-extension-breakage-validation.md b/work/projects/cpm-web-extension-breakage-validation.md index 0e2d54eb..64ebbc87 100644 --- a/work/projects/cpm-web-extension-breakage-validation.md +++ b/work/projects/cpm-web-extension-breakage-validation.md @@ -37,7 +37,7 @@ M4 перезагружает Web Extensions при первом сигнале | M3 attribution telemetry | `alex/cpm-failure-telemetry` — `/Users/admin/DuckDuckGo/apple-browsers.git/cpm-failure-telemetry` | **Merged 2026-09-23:** [PR #6844](https://github.com/duckduckgo/apple-browsers/pull/6844), CI green. Bucketed diagnostics, app launch/version attribution, `extension_updated`, `base_url_changed`, `recovery_from`, reload-error pixel; recorder и proxy имеют отдельные flags. [[cpm-m3-telemetry-privacy-triage]]. | | M4 hung recovery | `codex/cpm-hung-recovery` — `/Users/admin/DuckDuckGo/apple-browsers.git/cpm-hung-recovery` | **In progress:** первый `messaging init failure` открывает diagnostic episode; первый последующий `hung` перезагружает Web Extensions. | | M5+M6 graveyard mitigation + A/B | `alex/cpm-graveyard-mitigation` — `/Users/admin/DuckDuckGo/apple-browsers.git/cpm-graveyard-mitigation` | **Production-ready, default off; one PR over merged M3:** one retained view maximum, 3-second bounded hold, weak scheduled release, runtime cleanup, lazy stable cohort enrollment, control/treatment isolation, bounded pending outcomes, operational and experiment pixels. 431/431 WebExtensions tests; iOS/macOS builds and pixel validators pass; earlier real forced-kill iOS E2E recovered with release at ≈3.15 s. | -| M7 state-restoration readiness gate | `alex/cpm-state-restoration-race` — `/Users/admin/DuckDuckGo/apple-browsers.git/cpm-state-restoration-race` | **Production-ready locally from `origin/main` (`a18fca47c3`):** restored main-frame HTTP(S) navigation now waits for initial Web Extension `loadAndSync`; Swift package and full `macOS Browser` builds pass. Targeted tests pass: 19/19 coordinator suite (including 3 new M7 cases) and 3/3 navigation-gate tests. | +| M7 state-restoration readiness gate | `alex/cpm-state-restoration-race` — `/Users/admin/DuckDuckGo/apple-browsers.git/cpm-state-restoration-race` | **Implemented on macOS and iOS from current `origin/main` (`b93091e769`):** initial main-frame HTTP(S) navigation waits for Web Extension `loadAndSync`, bounded by shared `withTimeout(3)`. Timeout stops only the cancellation-aware waiter; the underlying load continues and its retained task is cleared on completion/cancellation. WebExtensions lifecycle suite: 20/20. iOS Swift compilation reached the app target successfully; the generic-simulator build then stopped in existing environment-dependent scripts because `TARGET_DEVICE_OS_VERSION` and a signing identity were unavailable. The 100-cycle macOS app-restart/session-restore UI regression was imported from the WIP worktree; its run was skipped by request. SwiftLint and diff/project-file checks pass. | All milestone diffs pass `git diff --check`; changed Swift lines are within 150 columns. @@ -45,11 +45,13 @@ All milestone diffs pass `git diff --check`; changed Swift lines are within 150 **Confirmed race:** `setupWebExtensions()` creates the manager/controller before state restoration, but starts `WebExtensionLifecycleCoordinator.loadAndSync()` without awaiting it. `stateRestorationManager.applicationDidFinishLaunching()` runs immediately afterward. A restored tab can therefore begin its main-frame navigation while the CPM `WKWebExtensionContext` is not loaded, so WebKit cannot inject the CPM content script at document start. The existing Content Blocking navigation responder already waits for internal user scripts and rule lists; M7 adds the equivalent readiness boundary for Web Extensions. -1. Add an awaitable initial-load readiness API to `WebExtensionLifecycleCoordinator`; it must reuse the existing coalesced `loadAndSync` task and complete on success, empty/disabled embedded set, or cancellation. -2. Inject a narrow readiness waiter into tab extensions and await it from the navigation policy path before main-frame HTTP(S) navigation proceeds. Do not poll `isAutoconsentExtensionLoaded`, because a disabled/failed CPM extension must not deadlock navigation. -3. Keep the wait startup-only: once initial `loadAndSync` finishes, later navigations pass immediately. Non-web and subframe navigations do not wait. -4. Cover coordinator blocking/release/cancellation and navigation-gate scope with unit tests; run targeted WebExtensions/macOS tests after explicit approval. -5. Acceptance: a restored page cannot start before the initial extension load/sync settles; disabled Web Extensions and cancelled startup work never hold a tab; normal post-launch navigation has no added asynchronous work. +1. Retain only the first coalesced `loadAndSync` task in `WebExtensionLifecycleCoordinator`, clear it on completion/cancellation, and expose a cancellation-aware waiter that never cancels the underlying load. **Done.** +2. Inject the waiter into both macOS `PrivacyDashboardTabExtension` and iOS `TabViewController`. Gate initial main-frame HTTP(S) navigation with the shared `WebExtensionNavigationGate`, which uses project `withTimeout(3)`. **Done.** +3. Keep the wait startup-only: after initial `loadAndSync`, the provider returns `nil`, so later navigations avoid an async hop. Non-web and subframe navigations do not wait. **Done.** +4. Cover blocking/release/cancellation, timeout without cancellation of the underlying load, late completion, default timeout, and navigation scope. WebExtensions lifecycle suite is 20/20; SwiftLint and structural checks pass. **Done.** +5. Add the existing WIP UI regression that restarts the macOS app 100 times, restores the Autoconsent page, and verifies CPM injection after every restart. **Included; execution intentionally skipped by request.** + +Technical decision: awaiting `initialLoadAndSync.value` directly inside structured `withTimeout` was hang-tested and rejected. The timeout cancels its child operation, but a child suspended on `Task.value` does not finish from that cancellation, so the task group cannot return. The small cancellation-aware waiter gives `withTimeout` a cancellable wait without cancelling the retained load task. ## Stack and Git state diff --git a/work/tech-design/coredata-code-defined-models.md b/work/tech-design/coredata-code-defined-models.md new file mode 100644 index 00000000..2377bd9d --- /dev/null +++ b/work/tech-design/coredata-code-defined-models.md @@ -0,0 +1,79 @@ +--- +title: 'Tech Design: Code-defined Core Data models' +created: '2026-09-25' +tags: + - tech-design + - apple + - core-data + - persistence + - spm + - build +status: draft +chosen-approach: 'Schemas defined in Swift with full version history; explicit lightweight migration' +asana: TODO +--- +**Author:** Alexey Martemyanov +**Reviewer:** Sam Symons +**Stakeholders:** Bartek, Dominik, Federico Cappelli, Michal Smaga +**Project:** Build the apps and packages outside Xcode (VS Code / `swift build`) + +--- + +## Background & Requirements + +We're moving development and CI to VS Code and SwiftPM builds that don't go through Xcode. + +Core Data schemas live in `.xcdatamodeld` files. Only Xcode compiles them into the `.momd` the app loads, so SwiftPM builds and tests outside Xcode have no model. BSK tests work around this by running `momc` at runtime. + +Requirements: +- existing user stores open unchanged +- upgrades from any older version keep data +- minimum OS stays at iOS 15 / macOS 12.3. + +## Problem Statement + +Core Data models can't be built outside Xcode. + +## Recommended Approach + +Define each model in Swift instead of `.xcdatamodeld`, one file per model, containing every shipped schema version. A small `Persistence` helper builds the model at runtime and migrates stores created by older versions.[1] + +Existing stores are unaffected: Core Data matches a store to a model by schema hashes, and the code-defined versions produce the same hashes as the compiled `.momd`. The hashes are pinned in tests. + +Steps (one PR each): + +1. `Persistence` support + PrivacyStats and BrowsingHistory. +2. Bookmarks + SyncMetadata. +3. Stores combining several models: HTTPSUpgrade, RemoteMessaging, AppRatingPrompt.[2] +4. macOS app models: History, Downloads, legacy Bookmark. +5. Remove the `momc` workaround. + +Each step is revertible: the on-disk format doesn't change. + +## Notes + +[1] Old versions are needed because migration looks for the old schema in the compiled models, which will no longer exist. `NSStagedMigrationManager` handles code-defined versions but requires iOS 17 / macOS 14. + +[2] A combined store needs the old versions of all its models, so they're converted together. + +Alternatives: +- Remove only the generated dynamic CoreData property accessors: `momc` is still needed and compiled as a separate Swift Build System Plugin step. +- Commit precompiled `.momd` files: binary artifacts that can drift from the source. + +Trade-off: we lose Xcode's visual model editor (but we get standardized, structured, great&shiny code definitions!) + +## Testing + +- Unit tests pin the schema hashes of every shipped version. +- Migration tests upgrade old-version stores and check the data. +- Package tests run with plain `swift test`. +- Manual check on iOS and macOS: upgrade from the App Store build and confirm the data is intact. + +## Additional Considerations + +- **Privacy** — No change. +- **Security** — No change. +- **Site Breakage** — N/A. +- **Experimentation** — N/A. The schema can't be feature-flagged; rollback is a revert. +- **Operational** — Package builds no longer need Xcode‘s momd compiler. Migration failures are reported through the existing store-loading errors. +- **Localization / Internationalization** — N/A.