Merge remote-tracking branch 'origin/main'

work/projects/cpm-web-extension-breakage-validation.md

@@ -37,7 +37,7 @@ M4 перезагружает Web Extensions при первом сигнале
 | M3 attribution telemetry | `alex/cpm-failure-telemetry` — `/Users/admin/DuckDuckGo/apple-browsers.git/cpm-failure-telemetry` | **Merged 2026-09-23:** [PR #6844](https://github.com/duckduckgo/apple-browsers/pull/6844), CI green. Bucketed diagnostics, app launch/version attribution, `extension_updated`, `base_url_changed`, `recovery_from`, reload-error pixel; recorder и proxy имеют отдельные flags. [[cpm-m3-telemetry-privacy-triage]]. |
 | M4 hung recovery | `codex/cpm-hung-recovery` — `/Users/admin/DuckDuckGo/apple-browsers.git/cpm-hung-recovery` | **In progress:** первый `messaging init failure` открывает diagnostic episode; первый последующий `hung` перезагружает Web Extensions. |
 | M5+M6 graveyard mitigation + A/B | `alex/cpm-graveyard-mitigation` — `/Users/admin/DuckDuckGo/apple-browsers.git/cpm-graveyard-mitigation` | **Production-ready, default off; one PR over merged M3:** one retained view maximum, 3-second bounded hold, weak scheduled release, runtime cleanup, lazy stable cohort enrollment, control/treatment isolation, bounded pending outcomes, operational and experiment pixels. 431/431 WebExtensions tests; iOS/macOS builds and pixel validators pass; earlier real forced-kill iOS E2E recovered with release at ≈3.15 s. |
-| M7 state-restoration readiness gate | `alex/cpm-state-restoration-race` — `/Users/admin/DuckDuckGo/apple-browsers.git/cpm-state-restoration-race` | **Production-ready locally from `origin/main` (`a18fca47c3`):** restored main-frame HTTP(S) navigation now waits for initial Web Extension `loadAndSync`; Swift package and full `macOS Browser` builds pass. Targeted tests pass: 19/19 coordinator suite (including 3 new M7 cases) and 3/3 navigation-gate tests. |
+| M7 state-restoration readiness gate | `alex/cpm-state-restoration-race` — `/Users/admin/DuckDuckGo/apple-browsers.git/cpm-state-restoration-race` | **Implemented on macOS and iOS from current `origin/main` (`b93091e769`):** initial main-frame HTTP(S) navigation waits for Web Extension `loadAndSync`, bounded by shared `withTimeout(3)`. Timeout stops only the cancellation-aware waiter; the underlying load continues and its retained task is cleared on completion/cancellation. WebExtensions lifecycle suite: 20/20. iOS Swift compilation reached the app target successfully; the generic-simulator build then stopped in existing environment-dependent scripts because `TARGET_DEVICE_OS_VERSION` and a signing identity were unavailable. The 100-cycle macOS app-restart/session-restore UI regression was imported from the WIP worktree; its run was skipped by request. SwiftLint and diff/project-file checks pass. |
 
 All milestone diffs pass `git diff --check`; changed Swift lines are within 150 columns.
 
@@ -45,11 +45,13 @@ All milestone diffs pass `git diff --check`; changed Swift lines are within 150
 
 **Confirmed race:** `setupWebExtensions()` creates the manager/controller before state restoration, but starts `WebExtensionLifecycleCoordinator.loadAndSync()` without awaiting it. `stateRestorationManager.applicationDidFinishLaunching()` runs immediately afterward. A restored tab can therefore begin its main-frame navigation while the CPM `WKWebExtensionContext` is not loaded, so WebKit cannot inject the CPM content script at document start. The existing Content Blocking navigation responder already waits for internal user scripts and rule lists; M7 adds the equivalent readiness boundary for Web Extensions.
 
-1. Add an awaitable initial-load readiness API to `WebExtensionLifecycleCoordinator`; it must reuse the existing coalesced `loadAndSync` task and complete on success, empty/disabled embedded set, or cancellation.
+1. Retain only the first coalesced `loadAndSync` task in `WebExtensionLifecycleCoordinator`, clear it on completion/cancellation, and expose a cancellation-aware waiter that never cancels the underlying load. **Done.**
-2. Inject a narrow readiness waiter into tab extensions and await it from the navigation policy path before main-frame HTTP(S) navigation proceeds. Do not poll `isAutoconsentExtensionLoaded`, because a disabled/failed CPM extension must not deadlock navigation.
+2. Inject the waiter into both macOS `PrivacyDashboardTabExtension` and iOS `TabViewController`. Gate initial main-frame HTTP(S) navigation with the shared `WebExtensionNavigationGate`, which uses project `withTimeout(3)`. **Done.**
-3. Keep the wait startup-only: once initial `loadAndSync` finishes, later navigations pass immediately. Non-web and subframe navigations do not wait.
+3. Keep the wait startup-only: after initial `loadAndSync`, the provider returns `nil`, so later navigations avoid an async hop. Non-web and subframe navigations do not wait. **Done.**
-4. Cover coordinator blocking/release/cancellation and navigation-gate scope with unit tests; run targeted WebExtensions/macOS tests after explicit approval.
+4. Cover blocking/release/cancellation, timeout without cancellation of the underlying load, late completion, default timeout, and navigation scope. WebExtensions lifecycle suite is 20/20; SwiftLint and structural checks pass. **Done.**
-5. Acceptance: a restored page cannot start before the initial extension load/sync settles; disabled Web Extensions and cancelled startup work never hold a tab; normal post-launch navigation has no added asynchronous work.
+5. Add the existing WIP UI regression that restarts the macOS app 100 times, restores the Autoconsent page, and verifies CPM injection after every restart. **Included; execution intentionally skipped by request.**
+
+Technical decision: awaiting `initialLoadAndSync.value` directly inside structured `withTimeout` was hang-tested and rejected. The timeout cancels its child operation, but a child suspended on `Task.value` does not finish from that cancellation, so the task group cannot return. The small cancellation-aware waiter gives `withTimeout` a cancellable wait without cancelling the retained load task.
 
 ## Stack and Git state
 

work/tech-design/coredata-code-defined-models.md

@@ -0,0 +1,79 @@
+---
+title: 'Tech Design: Code-defined Core Data models'
+created: '2026-09-25'
+tags:
+  - tech-design
+  - apple
+  - core-data
+  - persistence
+  - spm
+  - build
+status: draft
+chosen-approach: 'Schemas defined in Swift with full version history; explicit lightweight migration'
+asana: TODO
+---
+**Author:** Alexey Martemyanov
+**Reviewer:** Sam Symons
+**Stakeholders:** Bartek, Dominik, Federico Cappelli, Michal Smaga
+**Project:** Build the apps and packages outside Xcode (VS Code / `swift build`)
+
+---
+
+## Background & Requirements
+
+We're moving development and CI to VS Code and SwiftPM builds that don't go through Xcode.
+
+Core Data schemas live in `.xcdatamodeld` files. Only Xcode compiles them into the `.momd` the app loads, so SwiftPM builds and tests outside Xcode have no model. BSK tests work around this by running `momc` at runtime.
+
+Requirements: 
+- existing user stores open unchanged
+- upgrades from any older version keep data
+- minimum OS stays at iOS 15 / macOS 12.3.
+
+## Problem Statement
+
+Core Data models can't be built outside Xcode.
+
+## Recommended Approach
+
+Define each model in Swift instead of `.xcdatamodeld`, one file per model, containing every shipped schema version. A small `Persistence` helper builds the model at runtime and migrates stores created by older versions.[1]
+
+Existing stores are unaffected: Core Data matches a store to a model by schema hashes, and the code-defined versions produce the same hashes as the compiled `.momd`. The hashes are pinned in tests.
+
+Steps (one PR each):
+
+1. `Persistence` support + PrivacyStats and BrowsingHistory.
+2. Bookmarks + SyncMetadata.
+3. Stores combining several models: HTTPSUpgrade, RemoteMessaging, AppRatingPrompt.[2]
+4. macOS app models: History, Downloads, legacy Bookmark.
+5. Remove the `momc` workaround.
+
+Each step is revertible: the on-disk format doesn't change.
+
+## Notes
+
+[1] Old versions are needed because migration looks for the old schema in the compiled models, which will no longer exist. `NSStagedMigrationManager` handles code-defined versions but requires iOS 17 / macOS 14.
+
+[2] A combined store needs the old versions of all its models, so they're converted together.
+
+Alternatives:
+- Remove only the generated dynamic CoreData property accessors: `momc` is still needed and compiled as a separate Swift Build System Plugin step.
+- Commit precompiled `.momd` files: binary artifacts that can drift from the source.
+
+Trade-off: we lose Xcode's visual model editor (but we get standardized, structured, great&shiny code definitions!)
+
+## Testing
+
+- Unit tests pin the schema hashes of every shipped version.
+- Migration tests upgrade old-version stores and check the data.
+- Package tests run with plain `swift test`.
+- Manual check on iOS and macOS: upgrade from the App Store build and confirm the data is intact.
+
+## Additional Considerations
+
+- **Privacy** — No change.
+- **Security** — No change.
+- **Site Breakage** — N/A.
+- **Experimentation** — N/A. The schema can't be feature-flagged; rollback is a revert.
+- **Operational** — Package builds no longer need Xcode‘s momd compiler. Migration failures are reported through the existing store-loading errors.
+- **Localization / Internationalization** — N/A.