Files
obsidian-vault/work/projects/vscode-migration/tech-design-spm-browser-package.md
2026-10-15 09:50:58 +00:00

228 lines
15 KiB
Markdown
Executable File

# Tech Design: macOS browser code as a Swift package
**Author:** Alex M
**Reviewer:** TBD (reviewer of the previous design: Dominik)
**Stakeholders:** macOS team, plus the stakeholders of [Tech Design: convert all macOS modules to Swift Package](https://app.asana.com/1/137249556945/project/1202406491309510/task/1211728521308688)
**Project:** [Migration to fully VSCode-supported builds&debugging](https://app.asana.com/1/137249556945/project/1203301625297703/task/1209808486380252) · implementation: [PR #7080](https://github.com/duckduckgo/apple-browsers/pull/7080) (draft)
## Background & Requirements
- The macOS browser code lives in `macOS/DuckDuckGo/`: 1,406 Swift files, ~271k lines.
- Today the DMG and App Store app targets compile it.
- Some files are also compiled into the helper targets (VPN agent, PIR agent, NetP extensions).
- The [previous design](https://app.asana.com/1/137249556945/project/1202406491309510/task/1211728521308688) (approved Oct 2025) proposed:
- a static library linked by the app targets;
- a SwiftGen plugin for Core Data, storyboards and assets.
Its blocker was files shared between targets and per-target `#if` flags. That blocker has since been removed:
- ObjC/C code moved to packages;
- `SPARKLE`/`APPSTORE`/`ALPHA` and the other per-target flags dropped (#6983);
- storyboards dropped;
- helper targets no longer compile app files, apart from a short tracked list (#7029, see Notes [2]);
- asset lookups use `NSImage(resource:)`.
- Packages built with Xcode 27 process asset catalogs, Core Data models and string catalogs natively. So the SwiftGen plugin and the code-defined Core Data models are no longer needed; both were cancelled.
- **Requirements:**
- All existing schemes, configurations, tests and release lanes keep working unchanged.
- The shipped binary keeps the same structure: browser code statically linked into the app executable, nothing loaded at runtime.
- No user-visible change and no data migration.
## Problem Statement
The browser code is part of the Xcode app targets, so:
- only the Xcode project can build, index or test it;
- file membership, dependencies and resources are maintained by hand in `project.pbxproj` (23k lines);
- the code can't be split into modules or shared with iOS without first leaving the app target.
We want the browser code to be a regular Swift package target that the app targets link, with no change to the app users get.
## Recommended Approach
Make `macOS/DuckDuckGo/` a Swift package in place. One library target holds all browser code and its resources. The DMG and App Store targets become thin shells that link it.
**Layout**
- `macOS/DuckDuckGo/Package.swift` (new):
- target `DuckDuckGo_Privacy_Browser` with `path: "."`, so files don't move and history is kept;
- product `DuckDuckGoBrowser` (static), linked by both app targets;
- product `DuckDuckGoBrowserDynamic`, for builds outside Xcode only (Notes [1]).
- `macOS/DuckDuckGoAppBundle/` is a folder, not a target. It holds what used to sit next to the code:
- `main.swift`;
- `Info.plist`;
- entitlements;
- Info.plist and App Store string catalogs.
- DMG and App Store app targets, both using that folder:
- sources: `main.swift` only (`AppMain.main()`);
- bundle resources: app icons, `macos-config.json`, plist strings;
- link: `DuckDuckGoBrowser`.
**Steps**
1. **Manifest.**
- Dependencies: the 82 package products the app targets already use.
- Resources declared with `.process(...)`:
- asset catalogs;
- `Localizable.xcstrings` (1,980 keys);
- JSON/JS/HTML data;
- the 8 Core Data models.
- Files that belong to the app or helper bundles are `exclude`d.
2. **Keep the module name.** The package target keeps the old app module name, `DuckDuckGo_Privacy_Browser`. So these keep working:
- archived class names;
- `@objc` runtime names;
- `@testable import` in ~600 test files.
The app targets' own module becomes `DuckDuckGoApp`, which contains only `main.swift`.
3. **Thin app targets.** Their Sources phase is `main.swift` only. ~11k lines of file references leave `project.pbxproj`.
4. **Resources through `Bundle.module`.**
- Core Data models, named images and Lottie assets are loaded from `.module`.
- Strings: a module-level `NSLocalizedString(…, bundle: Bundle = #bundle, …)` shadows Foundation's, so `UserText` (275 files) doesn't change.
- SwiftUI `Text("…")` literals get `bundle: #bundle`.
5. **Compilation conditions.**
- `DEBUG` comes from the package configuration.
- Xcode builds packages in release for the custom CI configuration, so CI sets `SPM_FORCE_DEBUG=1`, which `Package.swift` reads. `AppKitExtensions`/`SyncUI` already work this way.
- `REVIEW`, the last target flag, gated one debug menu item; it's now a runtime check.
6. **Guard rails.** These are part of this change; see Risks.
- `InputFilesChecker` fails the build if an Xcode target bundles a file from the package folder, except a short allowlist (app icons, `macos-config.json`, NetP strings).
- It also gets a new check (to add): every resource in the package folder is either declared in `Package.swift` or excluded.
- SwiftLint rules (to add), applied to `macOS/DuckDuckGo/`:
- no `NSImage(named:)`/`NSColor(named:)`;
- no resource lookups on `Bundle.main`. Identity uses such as `bundleIdentifier` and app group stay allowed.
- `Text("…")`, `String(localized:)` and `LocalizedStringKey` must pass `bundle: #bundle`.
7. **Follow-ups (separate PRs).**
- Drop the now-duplicated 82 product dependencies from the app targets, and check that framework embedding is unchanged.
- Keep `macOS/DuckDuckGo/Package.resolved` in sync with the project's, with a CI check.
The change is reversible: deleting `Package.swift` and restoring the app targets' Sources phases brings back the old structure, since no source files moved.
**Further splitting.** The details are in the modularization roadmap. New modules become targets of the same `Package.swift` (`path: "Bookmarks"` plus an `exclude` in the main target), so files still don't move. Order:
1. Foundation modules: utilities, design system, pixels, per-module string catalogs and assets, and one Core Data model set per module.
2. Shared contracts and constructor injection in place of the `Application.appDelegate` service locator (273 files). Settings models split from the Settings window.
3. Leaf features first: CrashReports, Geolocation, Sharing, FindInPage, AppHealth, …
4. Tab core, with tab extensions owned by their features.
5. Mid-tier features: Bookmarks, History, Permissions, Downloads, Fire, Autofill, Sync, VPN UI, …
6. Shell: what's left in the app module, target ≤ 30k lines.
Along the way, logic duplicated with iOS moves into `SharedPackages`/BrowserServicesKit instead of a macOS-only module. About 15 areas qualify: Sync adapters, content-blocking providers, Autoconsent, unified feedback, TipKit, data import, Duck.ai stores, subscription web flows, statistics/ATB, …
### Pros
- **Dependencies declared once.** They live in `Package.swift`, not twice as product dependencies in `project.pbxproj`.
- **No file membership management.**
- New files are compiled automatically, so they can't be added to the wrong target or forgotten.
- DMG and App Store can't drift apart, since both link one product.
- `project.pbxproj` loses ~11k lines, so there are far fewer merge conflicts.
- **Standard SwiftPM tooling.** `swift build`, `swift test` and SourceKit-LSP work on the browser code without the Xcode project, so any SwiftPM-aware editor or CI step can use them.
- **Test dependencies.**
- Test targets can depend on the package product instead of only the app host.
- Extracted modules get their own `testTarget`s with exactly the dependencies they need.
- Logic tests can run without launching the app. Existing host-app tests keep working unchanged.
- **Foundation for modularization and for sharing code with iOS** (see above).
- **Same shipped binary.** Static linking, one executable, the same module name for persisted names.
### Cons: what we lose
Already given up in earlier steps of this project:
- **Per-target compile-time flags** (`SPARKLE`, `APPSTORE`, `ALPHA`, `NETWORK_EXTENSION` in app files). Differences are now runtime checks and injected services.
- **Storyboards and xibs.** Their module binding breaks outside the app module. Views are built in code or SwiftUI.
- **Generated asset symbols** (`NSImage.foo`). Xcode doesn't generate them for packages; we use `NSImage(resource: .foo)`, which is still type-checked.
- **ObjC/C in the app code and the bridging header.** A Swift package target can't mix languages, so ObjC/C code lives in separate packages.
New with this step:
- **Cross-linking a file into several targets.** Code shared with helper targets has to go into a shared package. The few remaining files are referenced by path and tracked by `InputFilesChecker` (Notes [2]).
- **Implicit resource membership.** Every resource must be declared in `Package.swift`; SwiftPM only warns about unhandled files. Mitigation: the `InputFilesChecker` check from step 6.
- **`Bundle.main` for browser resources.** A lookup that's missed returns nil or shows an untranslated string at runtime, with no build error. Three such cases were found and fixed on the branch. Mitigation: SwiftLint rules and resource smoke tests.
- **Per-configuration build settings.**
- Packages only know debug and release, so the custom Review and CI configurations don't reach the package.
- Swift flags from the xcconfigs don't apply to package targets either. That includes testability in CI and the slow type-check warnings (`DDG_SLOW_COMPILE_CHECK`).
- Mitigation: `SPM_FORCE_DEBUG` for CI. Set the same flags in `Package.swift` (to do; allowed for local packages).
- SwiftPM caches the evaluated manifest regardless of environment variables, so switching `SPM_FORCE_DEBUG` locally may need a manifest change. CI runners are clean.
- **Building the app requires Xcode 27,** for package asset catalogs, Core Data code generation and string catalogs. The project already targets Xcode 27.
- **UI test runners don't need Xcode 27.** They download the prebuilt app and build only the UI test bundle, which doesn't link the browser package. The macOS 14 and 15 legs keep Xcode 16.2 and 26.2.
- **A second `Package.resolved`,** used for standalone SwiftPM builds. CI checks that it matches the project's.
- **Build time unknown.** One big package target replaces the app target, so clean and incremental times may change either way. We benchmark before merging.
### Risks
- **Resources missing from the app.**
- A new resource that isn't declared in `Package.swift` isn't bundled.
- A resource bundled by an Xcode target instead of the package ends up in the wrong bundle.
- Mitigation: the `InputFilesChecker` build phase fails on any file in the package folder that isn't a Swift source, isn't declared as a package resource, isn't excluded and isn't on the Xcode-target allowlist. Resource smoke tests load every bundled asset, JSON file and Core Data model from `Bundle.module`.
- **User data loss.**
- The data models are unchanged: same `.xcdatamodeld` files and versions, same store file, same encryption key. Only the bundle they load from changes.
- Still, a **manual update test of a release build is mandatory**: install the current release, create data, update to the new build, and check every model:
- `Bookmark` (legacy bookmarks, migration only);
- `Favicons`;
- `Downloads`;
- `FireproofDomains`;
- `History`;
- `Permissions`;
- `PixelDataModel` (pixel/ATB state);
- `UnprotectedDomains`;
- `HTTPSUpgrade` (BrowserServicesKit, part of the same merged store);
- the BrowserServicesKit stores the app opens: bookmarks (`BookmarksModel`), Sync metadata, remote messaging, privacy stats.
- Also check **keychain access** after the update:
- the Core Data encryption key — if it can't be read, the stores can't be opened;
- SecureVault (passwords, cards, identities);
- Sync account;
- subscription tokens;
- the VPN and PIR auth that the helpers share.
Entitlements and access groups are unchanged, so we expect no change.
- **Localization.** Strings now come from the package's catalog through `#bundle`. A missed `bundle:` silently shows English.
- Mitigation: the SwiftLint rule.
- Test a **release build with a non-English locale**.
- Test the **localization flow manually**: export, translation import and the translation PR checks.
- **Framework embedding** when the duplicated product dependencies are dropped (follow-up). Mitigation: compare the app bundle's `Frameworks/` before and after.
## Notes
- [1] The dynamic product is not used by Xcode or release builds. It lets the browser code be built as a dylib and run from a small host app outside Xcode. That developer tooling is covered separately and never ships.
- [2] Files still compiled by both the package and the helper targets:
- subscription pixels and environment;
- TipKit;
- VPN IPC, error recorder and country labels;
- shared VPN strings;
- `UserDefaultsWrapper`.
They move to shared packages during modularization; `UserDefaultsWrapper` is replaced by `KeyedStoring`.
- [3] Alternatives considered:
- **A parallel `Package.swift` for tooling only, keeping the app target:** two sources of truth for membership and dependencies.
- **SwiftGen plugin or code-defined Core Data models** (previous design): not needed with Xcode 27.
- **Many packages right away:** blocked by dependency cycles inside the code; this follows incrementally as targets of the same package.
## Testing
- **Builds** (local, Xcode 27.2, all passing):
- every product scheme in Debug;
- `macOS Browser` in Review;
- unit and integration `build-for-testing` in Debug and CI.
CI runs all existing PR checks, including the App Store sandbox UI tests and the UI test legs on older macOS.
- **Existing tests** run unchanged; the test utilities now load Core Data models from the package bundle.
- **Resource smoke tests:** Lottie/data assets (added). Named images, JSON resources and Core Data models (to add).
- **Manual, DMG and App Store builds:**
- Fire and onboarding animations;
- tracker icons;
- Duck.ai sidebar;
- app icon switching;
- the update and keychain checks from Risks;
- a non-English locale.
- **Release:**
- notarized DMG;
- App Store Connect validation (the nested resource bundle must pass);
- Sparkle update from the previous version;
- symbolication of a test crash;
- code coverage includes the package files.
- **Benchmark:** clean and incremental (one-file change) builds in Debug and Release, before and after.
## Additional Considerations
- **Privacy:** No change. Persisted data, store files and defaults keys stay the same.
- **Security:** No change. No new entitlements; nothing is loaded at runtime; the dynamic product isn't shipped.
- **Site Breakage:** None expected. No change to WebKit, content blocking or user scripts, and content-blocking data is still bundled (covered by the smoke tests).
- **Experimentation:** Not applicable.
- **Operational:**
- CI sets `SPM_FORCE_DEBUG=1` for the CI configuration and checks `Package.resolved` sync. No new infrastructure.
- A missed resource lookup fails at runtime, not at build time, hence the lint rules, `InputFilesChecker` and the smoke tests.
- **Localization / Internationalization:**
- `Localizable.xcstrings` moves into the package; translations are unchanged.
- `InfoPlist`, `AppStore` and `DeveloperID` string catalogs stay with the app bundle.
- Testing covers a non-English release build and the full export/import flow.