obsidian-vault/work/tech-design/coredata-code-defined-models.md

title, created, tags, status, chosen-approach, asana

title	created	tags	status	chosen-approach	asana
Tech Design: Code-defined Core Data models	2026-09-25	tech-design apple core-data persistence spm build	draft	Schemas defined in Swift with full version history; explicit lightweight migration	TODO

Author: Alexey Martemyanov Reviewer: Sam Symons Stakeholders: Dominik Kapusta (Bookmarks, Sync), Federico Cappelli (Persistence, Remote Messaging), Michal Smaga (History) Project: Build the apps and packages outside Xcode (VS Code / swift build)

---

Background & Requirements

The iOS and macOS browsers keep local data in Core Data: history, bookmarks, sync metadata, privacy stats, remote messages, downloads and a few smaller stores. Each store's schema lives in an Xcode model file (.xcdatamodeld). At build time Xcode compiles it into a binary model (.momd), and the app loads that at runtime.

A model file holds every schema version that has shipped, not just the current one. When an app update changes the schema, Core Data opens the existing store, finds the old version among the compiled models, and upgrades the store in place ("lightweight migration").

There are 10 production models; 7 of them have more than one version (History has 7, Bookmarks 6). Two stores, the main Database on iOS and macOS, each combine several models into one file.

We are moving development and CI to builds that don't go through Xcode. Swift Package Manager doesn't compile model files outside Xcode. Today this is patched with a debug-only hack that runs Xcode's model compiler at runtime from inside tests. That only works on a machine with Xcode installed, and only on macOS.

Requirements

• Apps and packages build and test without Xcode's model compiler.
• On-disk data is unchanged: existing stores open as they do today.
• Users upgrading from any older app version keep their data.
• Schema changes are readable in code review.
• No change to minimum OS versions (iOS 15, macOS 12.3).

Problem Statement

Core Data schemas are defined in Xcode-only model files, so any build that doesn't go through Xcode is missing them. This blocks moving packages and the apps off Xcode and keeps a fragile runtime workaround in place.

Recommended Approach

Define every schema in Swift, one file per model. Each file contains all versions the model has ever shipped with[1], and each version is written as a change to the previous one. A small helper in the Persistence package builds the model at runtime and, when a store was created by an older version, upgrades it with the same lightweight migration Core Data uses today[2].

Safety rests on one property: Core Data decides whether a store matches a model by comparing per-entity version hashes. These are derived from the schema's structure, not from how the model was loaded. A code-defined version that is structurally identical to the shipped one opens existing stores untouched. Each version is:

• compared field by field against the compiled Xcode model before the model file is deleted, and
• pinned by its hashes in unit tests, so any accidental change to a shipped version fails CI.

Implementation, one PR per step. Steps 2–4 are stacked on step 1.

1. Foundation + pilots. Schema description types and versioned-model support in Persistence, with migration integrated into CoreDataDatabase. Convert PrivacyStats (1 version) and BrowsingHistory (3 versions); each has its own store. (PR ready for review.)
2. Bookmarks + SyncMetadata (6 and 4 versions), including the custom bookmarks form-factor migration and the test database builders[3].
3. Combined stores. Allow one store to be composed of several models, then convert HTTPSUpgrade, RemoteMessaging and AppRatingPrompt (the iOS main Database)[4].
4. macOS app models. Convert History (7 versions), Downloads, legacy Bookmark, and the macOS main Database, including its encrypted value transformers.
5. Cleanup. Remove model loading from bundles and the runtime model-compiler workaround.

Every step can be undone by reverting it: the on-disk format never changes, so a store written by a converted build still opens with the model-file build and vice versa.

Notes

[1] Why keep old versions in code. The source model for a migration used to come from the compiled model files. Without them there is nothing for Core Data to search. Core Data also keeps a copy of the model inside each SQLite store and can migrate from that; I confirmed this experimentally, but it is not a contract we should rely on. Keeping versions explicit makes migration deterministic, and it's needed for the custom bookmarks migration and for building test fixtures. The store-embedded copy stays as a fallback for stores that match none of the listed versions.

[2] Migration mechanics are unchanged. We upgrade in one step from the store's version to the current one, using an inferred mapping, exactly as automatic migration does today. It runs only when a store is outdated, and writes to a copy that replaces the original only on success. NSStagedMigrationManager, Apple's built-in support for code-defined versions, requires iOS 17 / macOS 14, so it isn't an option.

[3] Bookmarks has a custom migration that inspects the store's current version. It moves from reading compiled model files to the in-code version list.

[4] Combined stores are why HTTPSUpgrade isn't in step 1. Migrating a combined store needs the old version of every model in it, so all its models have to be converted together.

Alternatives considered

• Drop only the generated classes and keep the model files — the model compiler is still required, so the problem isn't solved.
• Commit precompiled .momd files — binary artifacts in the repo that can silently drift from the source model files.
• Keep only the current version — works today only because of the store-embedded model copy [1]. It breaks the custom bookmarks migration and loses the ability to build old-version fixtures.

Trade-off: we lose Xcode's visual model editor. In exchange, schema changes are plain Swift diffs.

Testing

• Schema fidelity: each shipped version's entity hashes are pinned in unit tests; they were captured from the Xcode-compiled models.
• Migration: existing fixture stores from old versions (e.g. BrowsingHistory v1) are migrated to the current version and their data verified. Persistence unit tests cover migration, no-op load of an up-to-date store, a new store, an unknown version, and cleanup of temporary files.
• Outside Xcode: affected package tests run with plain swift test (verified for step 1: 102 tests).
• Manual, per PR: install the current App Store build on iOS and macOS, create data, upgrade to the branch build, and confirm the data is intact.

Additional Considerations

• Privacy — No change to what data is collected, used or retained. Same stores, same locations.
• Security — No new dependencies or entitlements. The encrypted value transformers on macOS keep their current registration, verified in step 4.
• Site Breakage — None; there are no web-facing changes.
• Experimentation — Not applicable. There's no user-visible change, and a store schema can't sit behind a remote feature flag because the model is needed before configuration loads. Rollback is a revert (see Recommended Approach).
• Operational — Removes Xcode's model compiler as a build requirement for packages; no CI or release changes are needed. Migration runs as rarely as today (only for outdated stores). Failures surface through the existing store-loading error handling and pixels.
• Localization / Internationalization — Not applicable.