git_admin/obsidian-vault
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[2026-10-16] eagle: work/projects/vscode-migration/tech-design-xcode-build-server.md work/projects/vscode-migration/vscode-migration.md work/projects/vscode-migration/xcode-build-server.md

Browse Source
This commit is contained in:
Alexey Martemyanov committed 2026-10-16 00:57:55 +06:00
1 parent d333e88688
commit 0ff32c89a7
3 files changed
+142 -10
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

work/projects/vscode-migration/xcode-build-server.md
+129 -7
View File
@@ -1,11 +1,11 @@
# VS Code via Xcode build server (sourcekit-xcode-bsp)
Status: **ready for early adopters, uncommitted** · 2026-10-14
Status: **local setup deployed to all 97 registered worktrees; app-repository setup remains uncommitted** · 2026-10-16
**Parent:** [[projects/vscode-migration/vscode-migration]]
**Tech design:** [[projects/vscode-migration/tech-design-xcode-build-server]]
An alternative to the [[projects/vscode-migration/tech-design-spm-browser-package|SwiftPM browser package]]: keep the Xcode projects as they are and give VS Code/Cursor what it needs from them.
- Code intelligence: sourcekit-lsp ← `buildServer.json` ← [sourcekit-xcode-bsp](https://github.com/slime-studio/sourcekit-xcode-bsp) (BSP server on top of swift-build, Xcode's build engine; early-stage 0.1.0, Apache 2.0), patched by us.
- Code intelligence: sourcekit-lsp ← `buildServer.json` ← [our sourcekit-xcode-bsp fork](https://github.com/mallexxx/sourcekit-xcode-bsp) ([upstream](https://github.com/slime-studio/sourcekit-xcode-bsp); BSP server on top of swift-build, Xcode's build engine; early-stage 0.1.0, Apache 2.0).
- Build/test/debug: `xcodebuild` tasks + lldb-dap launch configs, macOS and iOS simulator.
- Builds and index shared with Xcode through the worktree's DerivedData folder.
@@ -13,9 +13,104 @@ An alternative to the [[projects/vscode-migration/tech-design-spm-browser-packag
| What | Where |
|---|---|
| VS Code setup | worktree `vscode-xcode-bsp`, branch `alex/vscode-xcode-bsp` — `DuckDuckGo.code-workspace`, `scripts/vscode/` (uncommitted) |
| BSP fork | `~/Developer/sourcekit-xcode-bsp`, branch `configuration-name` (uncommitted), base upstream `726ce706`; identical to `scripts/vscode/sourcekit-xcode-bsp.patch` |
| Installed BSP | `~/.local/opt/sourcekit-xcode-bsp` (by `install-build-server.sh`, with `patch.sha256` stamp); brew 0.1.0 also installed but unused |
| Original migration worktree | `/Users/admin/DuckDuckGo/apple-browsers.git/vscode-xcode-bsp`, branch `alex/vscode-xcode-bsp`; original uncommitted setup preserved |
| Local bootstrap | `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/worktrees.py` |
| Master templates | `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/template/` |
| Active shared Git hook | `/Users/admin/DuckDuckGo/apple-browsers.git/.git/hooks/post-checkout` |
| Local ignore rules | `/Users/admin/DuckDuckGo/apple-browsers.git/.git/info/exclude` |
| Published BSP fork | [mallexxx/sourcekit-xcode-bsp](https://github.com/mallexxx/sourcekit-xcode-bsp), default branch `xcode-workspace-support`; pinned revision `afd0e3dd026b3ee3619e2271809debee25fec96f` |
| Fork checkout | `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/sourcekit-xcode-bsp`; the earlier draft at `/Users/admin/Developer/sourcekit-xcode-bsp` was preserved |
| Installed BSP binaries | `/Users/admin/.local/opt/sourcekit-xcode-bsp/`, shared by worktrees; existing patched installation was not replaced during the rollout |
## Local hook and all-worktree rollout — 2026-10-16
**Deployed and verified:** the real shared Git hook is active, and the setup has been applied to **all 97 registered Apple browser worktrees**, including the two checkouts beside `apple-browsers.git`. This is a local deployment on Alex's machine; no app-repository commit, PR, or upstream issue was created.
### Trigger and ownership
Git's repository-local `core.hooksPath` is already `/Users/admin/DuckDuckGo/apple-browsers.git/.git/hooks`. Its executable `post-checkout` hook runs after a normal `git worktree add`, `git checkout`, or `git switch` with the branch-checkout flag. It finds that worktree's root and runs:
```sh
python3 /Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/worktrees.py apply "$root"
```
The bootstrap accepts worktrees sharing this repository's Git common directory, including paths outside the bare-repository container. It copies central templates rather than linking worktrees to each other. It generates each worktree's absolute workspace/launcher paths and Xcode DerivedData location, preserves the selected macOS/iOS platform, and uses hashes to update files it owns while preserving custom edits and tracked files.
A normal fresh-worktree creation was tested with the installed hook, without a temporary `core.hooksPath` override: all 17 files appeared automatically, were ignored, and left Git status clean. The hook itself runs no network requests, installer, build, or indexing job. Its failure is reported without failing checkout.
Git does not run this hook for `worktree add --no-checkout`; a later checkout does. Tools that disable or bypass Git hooks need an explicit apply command. No cron, launchd job, or filesystem watcher was installed for bootstrapping.
### Files inside each worktree
```text
<worktree>/
DuckDuckGo.code-workspace
buildServer.json
.vscode/
settings.json
tasks.json
launch.json
extensions.json
scripts/vscode/
.build/vscode-local/files.json
```
`files.json` records ownership hashes. Each newly bootstrapped worktree receives 17 setup files; existing custom files may be retained instead. Open that worktree's `DuckDuckGo.code-workspace` in VS Code/Cursor. The binaries remain shared at `/Users/admin/.local/opt/sourcekit-xcode-bsp/`.
The shared local `.git/info/exclude` contains:
```gitignore
/DuckDuckGo.code-workspace
/scripts/vscode/
/.vscode/
/buildServer.json
/.vscode-local/
```
No tracked `.gitignore` was edited. Existing tracked files remain tracked despite these exclusions.
### Commands and rollback
The following install and backfill commands have already been run:
```sh
python3 /Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/worktrees.py install
python3 /Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/worktrees.py apply --all
```
For an individual worktree or a preview:
```sh
python3 /Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/worktrees.py apply /absolute/worktree/path
python3 /Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/worktrees.py apply /absolute/worktree/path --dry-run
```
The installer refuses to replace an existing different hook. To disable future automatic runs, move the installed `post-checkout` file out of the hooks directory (for example to `.vscode-local/post-checkout.disabled`); existing generated files remain. There is no dedicated uninstall command.
### Rollout verification and legacy exceptions
The backfill wrote 1,526 files across 97 worktrees and preserved 45 existing setup/editor-file entries. Structural verification found **zero failures**: expected files existed, server names and absolute paths matched each worktree, wrappers were executable, generated editor JSON parsed, and newly created files were Git-ignored. This was not a new app build, indexing run, or debugger test in every worktree.
The old `xcode-build-server` configurations in `exp` and `sweetpad` were migrated after backing up their originals:
- `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/backups/backfill-20261015T185058Z/exp/buildServer.json`
- `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/backups/backfill-20261015T185058Z/sweetpad/buildServer.json`
**Tracked-file exception:** the `exp` branch already tracks `buildServer.json`. Its migration is therefore an intentional **uncommitted local modification**, not an ignored file. All other previously tracked setup files were preserved unchanged. The hook skips tracked files, so a later branch checkout can restore that branch's tracked legacy config; the existing folder-open setup task regenerates the selected BSP config when opening the workspace.
Evidence:
- `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/backfill-report.json`
- `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/backfill.log`
- `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/backfill-before.json`
- `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/backfill-migrations.json`
- `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/research.txt`
### Publishing scope
The BSP fixes are public in the personal fork, with a committed dependency lockfile. All 58 tests and strict SwiftLint on the changed Swift files passed. The local template installer pins `afd0e3dd026b3ee3619e2271809debee25fec96f`, uses `xcrun swift` and `--force-resolved-versions`, and no longer applies a patch during installation. The old patch snapshot remains for compatibility with the installed fingerprint check; the original migration worktree's differing installer/README were preserved.
The bootstrap is still specific to this project: workspace/project names, schemes, app paths and target filters are hardcoded. Its installer also assumes an existing absolute `core.hooksPath`. A general public bootstrap needs configuration, Git layout/hook handling, packaging, removal support, and tests against another Xcode project. A DuckDuckGo-owned fork, team rollout and upstream PRs remain separate work.
## Files (`scripts/vscode/`)
@@ -25,7 +120,7 @@ An alternative to the [[projects/vscode-migration/tech-design-spm-browser-packag
- `xcodebuild.sh <build|test> <scheme> [destination]` — Debug into Xcode's DerivedData with Xcode's settings (no `-derivedDataPath`: passing it changes build parameters and makes Xcode and VS Code rebuild after each other). `DDG_WORKTREE_CACHE=1`: `.build/xcodebuild` + shared CAS with `/^wt` `/^dd` maps. `.build/products` → products folder. xcbeautify + raw `path:line: error:` lines for failed tests.
- `simulator.sh [--build-only|--no-build] [device]` — builds iOS Browser for a concrete simulator, installs, `simctl launch --wait-for-debugger`, writes `process attach --pid N` to `.build/simulator-attach.lldb`.
- `index-in-background.py [--all] [--if-idle]` — headless indexing; `background-indexing.sh install` = launchd agent every 30 min.
- `install-build-server.sh` + `sourcekit-xcode-bsp.patch` + `sourcekit-xcode-bsp.Package.resolved` — reproducible build of the patched BSP.
- `install-build-server.sh` — the central template builds the published fork at the pinned commit using its committed `Package.resolved`; the archival patch/lockfile snapshots remain for compatibility with earlier local installations. The original migration worktree's differing installer was preserved.
## Findings
@@ -45,6 +140,30 @@ An alternative to the [[projects/vscode-migration/tech-design-spm-browser-packag
13. swift-build service runs in its own process group → stop children first.
14. Unrelated: main's `SharedPackages/BrowserServicesKit/Package.resolved` is stale (C-S-S 17.20 vs `exact: 17.26.0`) — separate task spawned.
## Swift Build fork and cache findings — 2026-10-16
The existing fork is `/Users/admin/Developer/swift-build-custom`, origin `mallexxx/swift-build`, branch `cache-provisioning-inputs`, commit [0d23a4dd5eb536fcdbc10ef83bfdad57e6a7a022](https://github.com/mallexxx/swift-build/commit/0d23a4dd5eb536fcdbc10ef83bfdad57e6a7a022) from June 2025. Its patch serializes `ProvisioningTaskInputs` to `<OBJROOT>/ProvisioningTaskInputs/<target-name>.provisioningTaskInputs.json` and reuses them during planning. **It caches signing/provisioning inputs, not package resolution or downloaded package artifacts.**
That checkout has no ready `.build` service binary. The old cache has no configuration/profile/entitlement freshness key, and its Xcode 27.1 compatibility has not been validated. It was not enabled. After porting, building and checking compatibility, the existing wrapper can select a service for one invocation without a global shell export:
```sh
XCBBUILDSERVICE_PATH=/absolute/products/path/SWBBuildServiceBundle \
scripts/vscode/xcodebuild.sh build 'macOS Browser'
```
The fork's `Plugins/run-xcodebuild/run-xcodebuild.swift` demonstrates the override. Keep service resources alongside the executable and match service/client versions. No provisioning-cache speedup was measured.
**Current provisioning cost:** exact `GatherProvisioningInputs` section timestamps were extracted from five actual October 15 `macOS Browser` Debug build activity logs. All logged `XCODE_PRODUCT_BUILD_VERSION=27A9275`, matching the currently selected **Xcode 27.1 (27A9275)**. Durations were 0.621, 0.255, 1.111, 1.093 and 1.340 seconds: **median 1.09 s, range 0.25–1.34 s**. These are archived real-build measurements, not fresh full-build benchmarks or stdout approximations; they do not characterize cold signing/account setup or online provisioning updates. Evidence: `/Users/admin/DuckDuckGo/apple-browsers.git/vscode-local-setup/.build/.build-benchmark/provisioning-20261016/provisioning-summary.json`.
**Package/artifact reuse plan (not automatically enabled):** keep per-worktree `SourcePackages` and Xcode-compatible DerivedData; share standard download/repository caches, and optionally seed a new worktree using independent APFS copies from a matching prepared cache before normal resolution. Avoid one shared mutable checkout directory across branches. A future fast path should fingerprint the toolchain, workspace/project files, manifests, lockfiles and package configuration, verify cache completeness, then use `-skipPackageUpdates -onlyUsePackageVersionsFromResolvedFile` only when valid. On changed/missing inputs, resolve normally before updating the stamp. These flags still evaluate the package graph in each new xcodebuild process. Existing wrapper arguments already permit an explicit warm-cache trial:
```sh
scripts/vscode/xcodebuild.sh build 'macOS Browser' macOS \
-skipPackageUpdates -onlyUsePackageVersionsFromResolvedFile
```
The existing `DDG_WORKTREE_CACHE=1` mode is a separate compilation/CAS optimization; its earlier 179 s → 90–120 s measurements were not re-run during this rollout.
## Review (independent agent, 2026-10-14)
Verdict: "close, not ready" → fixed: iOS platform reset by setup/indexer, unpinned `swift-tools-protocols` (now Package.resolved + `--force-resolved-versions`), purpose not in cache key, empty `include`, sources not generation-checked, test gaps (58 tests now), editor/indexer race (refuse + wait), stop() killing unchecked pids, indexer hanging 1 h, spaces in prefix maps, in-place install over running binaries, stale-install detection, test failures not in Problems, Xcode 26/27 wording, `re.escape`, `as_uri`, editor detection, xcbeautify missing warning, `files.watcherExclude`, lldb-dap pinned, risky `waitFor` attach removed, mismatched scheme/destination task.
@@ -63,6 +182,9 @@ Not verified: VS Code/Cursor UI itself; Xcode IDE reading our index / no-rebuild
- [ ] Walk through VS Code and Cursor UI (tasks, Problems, launch/attach, workspace trust prompt).
- [ ] Check Xcode IDE: open a worktree indexed/built from VS Code — no reindex, no rebuild.
- [ ] Upstream PRs to slime-studio/sourcekit-xcode-bsp (public repo, needs OK); or a DDG fork.
- [x] Publish tested fixes to [mallexxx/sourcekit-xcode-bsp](https://github.com/mallexxx/sourcekit-xcode-bsp), pinned at `afd0e3dd026b3ee3619e2271809debee25fec96f`.
- [x] Deploy shared local hook and backfill all 97 registered worktrees (2026-10-16).
- [ ] Establish a DuckDuckGo-owned fork/team rollout if approved.
- [ ] Offer the build-server fixes upstream (no upstream PR opened).
- [ ] Upstream issue: duplicate outputs/stray `.o` when all targets are configured; lldb-dap attach with `program`/`attachCommands` to simulator processes.
- [ ] PR (needs Asana task link).