obsidian-vault/work/projects/vscode-migration/xcode-build-server.md

# VS Code via Xcode build server (sourcekit-xcode-bsp)

Status: **generic harness published at mallexxx/xcode-editor-tools; deployed to 97 existing worktrees plus a verified fresh experiment** · 2026-10-16
**Parent:** [[projects/vscode-migration/vscode-migration]]
**Tech design:** [[projects/vscode-migration/tech-design-xcode-build-server]]

An alternative to the [[projects/vscode-migration/tech-design-spm-browser-package|SwiftPM browser package]]: keep the Xcode projects as they are and give VS Code/Cursor what it needs from them.
- Code intelligence: sourcekit-lsp ← `buildServer.json` ← [our sourcekit-xcode-bsp fork](https://github.com/mallexxx/sourcekit-xcode-bsp) ([upstream](https://github.com/slime-studio/sourcekit-xcode-bsp); BSP server on top of swift-build, Xcode's build engine; early-stage 0.1.0, Apache 2.0).
- Build/test/debug: `xcodebuild` tasks + lldb-dap launch configs, macOS and iOS simulator.
- Builds and index shared with Xcode through the worktree's DerivedData folder.

## Where things are

| What | Where |
|---|---|
| Standalone harness repository | `/Users/admin/Developer/xcode-editor-tools` — generic source, README, configuration reference, examples, tests, MIT license, CI workflow; published at [mallexxx/xcode-editor-tools](https://github.com/mallexxx/xcode-editor-tools), branch `main`, commit `b74c333e580c6bc3ba8a8b48557446e4f8fe87a7`; local checkout clean and tracking `origin/main` |
| Integration instructions | `/Users/admin/Developer/xcode-editor-tools/README.md` and `CONFIGURATION.md` |
| Private Apple browser profile | `/Users/admin/DuckDuckGo/apple-browsers.git/.git/xcode-editor/config.json` |
| Active shared Git hook | `/Users/admin/DuckDuckGo/apple-browsers.git/.git/xcode-editor/hooks/post-checkout` |
| Hook installation/previous settings | `/Users/admin/DuckDuckGo/apple-browsers.git/.git/xcode-editor/installation.json` |
| Local ignore rules | `/Users/admin/DuckDuckGo/apple-browsers.git/.git/info/exclude` |
| Compatibility entry point | `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/worktrees.py` — forwards old commands to the new CLI |
| Previous harness source/templates | `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/extraction-20261016/legacy-harness/` — rollback archive |
| Previous hook/config/excludes | `/Users/admin/DuckDuckGo/apple-browsers.git/.git/xcode-editor/backups/extraction-20261016/` |
| Original migration worktree | `/Users/admin/DuckDuckGo/apple-browsers.git/vscode-xcode-bsp`, branch `alex/vscode-xcode-bsp`; custom installer/README preserved |
| Published BSP fork | [mallexxx/sourcekit-xcode-bsp](https://github.com/mallexxx/sourcekit-xcode-bsp), default branch `xcode-workspace-support`; pin `afd0e3dd026b3ee3619e2271809debee25fec96f` |
| BSP fork checkout | `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/sourcekit-xcode-bsp`; earlier draft in `~/Developer/sourcekit-xcode-bsp` preserved |
| Shared installed BSP | `/Users/admin/.local/opt/sourcekit-xcode-bsp/`; existing patched executable/service/resources retained |

## Standalone harness extraction and deployment — 2026-10-16

The harness now lives in **`~/Developer/xcode-editor-tools`**, published as [mallexxx/xcode-editor-tools](https://github.com/mallexxx/xcode-editor-tools) on 2026-10-16 at commit `b4a839b92f60ec09cd33982014f4b593161d2673`. Its source contains no workstation paths, Apple browser names, schemes, product paths or private target filters. Those values are in the private Git profile above. The default editor workspace filename is derived from the configured Xcode container; an ordinary project, bare repository and externally located linked worktrees are supported. There are no third-party patches, binaries, package checkouts or local logs in the new repository. The separate BSP fork remains the public pinned dependency.

The deployed hook now runs that repository's CLI. All **97 existing worktrees** were refreshed from the generic implementation. A fresh experimental worktree, `/Users/admin/DuckDuckGo/apple-browsers.git/vscode-editor-tools-check`, was then created through ordinary `git worktree add --detach … main` and bootstrapped automatically: **19 generated files**, clean Git status, and zero rewrites on another apply. There are now **98 registered checked-out worktrees** including that experiment.

### Trigger, hooks and preservation

This is a **Git `post-checkout` hook**, selected by repository-local `core.hooksPath = /Users/admin/DuckDuckGo/apple-browsers.git/.git/xcode-editor/hooks`. It runs after a normal `git worktree add`, `git checkout` or `git switch` when Git's branch-checkout flag is `1`. File-only checkouts are ignored. The hook discovers Git's common directory and applies the private profile to the worktree.

Existing executable hooks are forwarded with their arguments, environment, working directory, standard streams and exit status. Default, absolute custom and relative hook paths are supported. The previous bootstrap hook was adopted only after verifying its exact SHA-256, then archived; it does not run a second bootstrap. Re-run `install` if adding another hook to the previous hook directory later.

Tracked files, symlinks and locally customized generated files are preserved. Ownership remains in `.build/vscode-local/files.json` for this installation to adopt the earlier hashes. Newly installed repositories default to `.build/xcode-editor/files.json`. Generated runtime scripts are self-contained; the source checkout is needed by the Git hook for future generation.

The hook runs no network requests, installer, build or indexing job. Its own failure is printed without failing an otherwise successful checkout; a prior hook's failure status is retained. `worktree add --no-checkout`, per-worktree hook overrides and tools that bypass hooks require an explicit apply. No cron, watcher, launchd job or filesystem monitor was installed for bootstrap.

### Files inside each worktree

```text
<worktree>/
  DuckDuckGo.code-workspace
  buildServer.json
  .vscode/{settings,tasks,launch,extensions}.json
  scripts/vscode/                  generated runtime and local usage README
  .build/xcode-editor/config.json  private runtime profile
  .build/vscode-local/files.json   adopted ownership hashes
```

The shared `.git/info/exclude` retains its previous rules and adds a managed `xcode-editor-tools` block covering the generated editor workspace, scripts, `.vscode`, discovery file and runtime/ownership state. The old `/.vscode-local/` exclusion remains. No tracked `.gitignore` was changed.

Open the same worktree-local `DuckDuckGo.code-workspace`. macOS/iOS schemes, test selection, simulator launch/attach, regular Xcode DerivedData and target filters are preserved. Tasks use process arguments to handle spaces/quotes safely. `XCODE_EDITOR_WORKTREE_CACHE=1` is the generic opt-in CAS flag; the private profile also retains `DDG_WORKTREE_CACHE` and the old background-indexer flag for compatibility. Regular builds still omit `-derivedDataPath`; no new cache mode was enabled.

### Maintenance and removal

```sh
tools=/Users/admin/Developer/xcode-editor-tools/xcode_editor.py
repo=/Users/admin/DuckDuckGo/apple-browsers.git

python3 "$tools" status --repo "$repo"
python3 "$tools" apply --repo "$repo" --all
python3 "$tools" apply --repo "$repo" /absolute/worktree/path --dry-run
python3 "$tools" install --repo "$repo"
python3 "$tools" uninstall --repo "$repo"
```

Edit the private profile, then run `apply --all` to propagate profile changes. `init` refuses to replace an existing profile. After moving the harness checkout, run `install` from its new location; after moving the target repository, also run `apply --all`. Absolute paths in the private installed hook and generated BSP files are derived at installation time.

`uninstall` restores the earlier local `core.hooksPath` (here, `…/.git/hooks`) and leaves generated files, ownership and exclusions intact. Because the superseded bootstrap hook has been archived, automatic setup stops. The old `.vscode-local/worktrees.py apply …` command remains a forwarding compatibility shim. For full rollback, the preceding harness, hook, Git config and exclude file are retained at the backup paths above; restoring a full old config should be reviewed against any intervening Git changes.

### Verification and existing exceptions

- All **97 existing BSP configurations were identical** to their pre-extraction JSON, including platform, target filters, worktree paths and DerivedData.
- All **23 tracked integration files** and the customized settings file were unchanged. The new installer continues to preserve original migration-worktree custom files.
- **1,992 ignored-file checks** passed across existing ownership manifests; no structural failures.
- **47 unit tests** passed across the bootstrap and runtime suites. They cover ordinary/bare repositories, external and relocated worktrees, hook chaining/removal/adoption, ownership, path quoting, ignore metacharacters, platform persistence, build parameters, exit status, CAS/debug mapping and the original helper API.
- An unrelated minimal **OrbitSample** macOS application built and ran through the new wrapper; the actual installed BSP discovered its target. This used Xcode 27.1 and exposed/fixed Foundation's `/private` temporary-path spelling difference when locating DerivedData.
- The actual deployed hook generated the fresh browser experiment automatically; its folder-open setup command also verified `buildServer.json` was already current.
- Public source scan: no local paths or project names; Python/JSON/shell syntax and whitespace checks passed. CI was configured during extraction; hosted results are available under [GitHub Actions](https://github.com/mallexxx/xcode-editor-tools/actions). No new full browser build, editor UI walkthrough or indexing run was performed in every worktree.

The first rollout had written 1,526 files across 97 worktrees and preserved 45 existing/tracked entries. Its diagnostics remain in `.vscode-local/backfill-*.json` and `backfill.log`. The old configurations in `exp` and `sweetpad` were backed up at `.vscode-local/backups/backfill-20261015T185058Z/{exp,sweetpad}/buildServer.json` before migration. **`exp/buildServer.json` was already tracked** and remains its earlier intentional uncommitted modification. A branch checkout may restore its tracked legacy config; the explicit folder-open setup command refreshes it. The extraction introduced no new tracked changes.

Extraction evidence: `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/extraction-20261016/`: `before.json`, `verification.json`, `apply.log`, `fresh-worktree.log`, private profile snapshot and legacy harness backup.

### Publication scope

The new generic harness has README integration instructions, a configuration reference, minimal and two-platform examples, MIT license, dependency/license notice, contributor instructions, automated tests and a CI workflow. At the user's request, it is now **public at [mallexxx/xcode-editor-tools](https://github.com/mallexxx/xcode-editor-tools)**. The 35 audited source files were committed and pushed to `main` as `b4a839b92f60ec09cd33982014f4b593161d2673`; GitHub's remote SHA and public visibility were verified. The public commit uses `365498+mallexxx@users.noreply.github.com` for author/committer, with no private workstation identity. Local `main` tracks `origin/main` and is clean. README includes the actual clone command. The source/privacy audit found no local paths, private project identifiers, embedded credentials or secret patterns. The separate BSP fork is already public; its earlier 58 tests and strict SwiftLint validation remain recorded. A DuckDuckGo-owned fork, team deployment and upstream PRs remain proposed.

## Persistent local build logs — 2026-10-16

The wrapper now keeps the latest complete, unformatted xcodebuild stdout/stderr in `<worktree>/.build/xcode-editor/xcodebuild.log` and prints its path before the command. It retains successful and failed output, including when the build succeeds but launching the debugger fails afterwards. Only the next actual wrapper build/test replaces the file; `--help` leaves it untouched. The log is locally ignored, owner-readable/writable only (`0600`), and a symlink at that path is refused. Debug-adapter errors remain in VS Code's Swift/LLDB-DAP Output and Debug Console.

Applied and hash-verified in all **100 currently registered worktrees**, including `lantean-prevents-offscreen-animation`; all log paths are ignored. The file will first appear on the next wrapper build/test. **22 runtime tests passed**, covering raw stdout/stderr retention through formatting, failure status, replacement by the next successful run, help preservation, permissions, ignore rules and symlink protection. At the user's request, the log-retention fix, tests and documentation were committed and pushed to public `main` as [`b74c333e580c6bc3ba8a8b48557446e4f8fe87a7`](https://github.com/mallexxx/xcode-editor-tools/commit/b74c333e580c6bc3ba8a8b48557446e4f8fe87a7). The exact remote SHA and clean local checkout were verified; GitHub's noreply author/committer identity was used.

## Files (`scripts/vscode/`)

- `editor_support.py` — self-contained configuration, Git, BSP rendering, environment compatibility, DerivedData and debugger helpers.
- `setup-build-server.py [macos|ios]` — refresh discovery on folder open, preserving the existing platform without an argument; checks the installed BSP supports target selection.
- `xcode_derived_data.py` — print the configured container's Xcode DerivedData; retains the original one-argument helper for preserved scripts.
- `sourcekit-xcode-bsp.sh` — launch the editor's BSP; coordinate per-worktree PID files with the optional headless indexer.
- `xcodebuild.sh <action> <scheme> [destination] [args…]` — regular Xcode-compatible parameters, optional compilation CAS, products symlink, debugger init, formatted diagnostics and original build status.
- `simulator.sh [--build-only|--no-build|--test SCHEME] [--only-testing TARGET] [device]` — choose a concrete simulator, build/run or test; launch suspended and generate an attach command.
- `index-in-background.py [--all] [--if-idle]` and `background-indexing.sh install|status|uninstall` — optional headless indexing and per-repository launchd agent. No agent was installed.
- `install-build-server.sh` / `install_server.py` — explicit pinned fork installation with `--force-resolved-versions`, co-located service/resources and a retained previous installation. No patch is vendored by the generic harness; old worktree snapshots remain only for preserved legacy scripts.
- `manage_background.py` and `README.md` — launchd management and local usage.

## Findings

1. **Works for app code out of the box**, no Package.swift. Swift extension 2.16.7 activates on `buildServer.json`.
2. **Tests: nothing.** BSP had no configuration → swift-build used `defaultConfigurationName = Release` → no testability → `@testable import` fails. Fix: `configuration` (default Debug; swift-build's own index build defaults to Debug too).
3. **Stray `.o`, failed links, duplicate outputs.** BSP configured all 977 targets in one build (DMG + App Store, iOS + macOS, each package library 3–4 times). Fix: `targets` filter → 86 roots, 288 in the graph, 0 failures.
4. **Indexing ~2 h.** swift-build computes `sourceKitOptions` per file (~0.8 s, single thread, whole build request each time). Swift args of a target differ only in the trailing `-index-unit-output-path`. Fix: per-target cache keyed by (target, purpose) + single flight. Fresh worktree full index ~5.5 min (M4 Max, low load).
5. **Preparation** (~250 serialized prepare builds) is now the critical path.
6. **Shared CAS between worktrees (xcodebuild):** needs `SWIFT/CLANG_ENABLE_PREFIX_MAPPING` + `*_OTHER_PREFIX_MAPPINGS` = `<DerivedData>=/^dd <worktree>=/^wt`. Clean build 179 s → 90–120 s, 0 misses in a new worktree. Xcode's own project prefix mapping (`/^src` per project) breaks lldb sources; `/^wt` + `sourceMap` works. Opt-in now, because it can't be shared with Xcode (unconfirmed how Xcode's debugger handles mapped paths).
7. **CAS doesn't help the language server** (prepare-for-index doesn't cache; cold vs warm the same).
8. **Index shared with Xcode works:** with Xcode's DerivedData layout (index store = `Index.noindex/DataStore`, `INDEX_REGULAR_BUILD_*` = `Build/…`), units written by an xcodebuild build count as up to date: after building `macOS Browser`, indexing skipped all app files (2,150 files left vs 3,972). BSP prepares in `Index.noindex/SourceKitBuild` so it never shares a build DB with an open Xcode.
9. **Xcode-compatible builds:** `xcodebuild` without `-derivedDataPath` and the wrapper don't rebuild after each other (no-op 26–31 s). With `-derivedDataPath` pointing to the same folder they do.
10. **iOS simulator:** generic destination fails (`fix_libswiftWebkit_simulator.sh` needs `TARGET_DEVICE_OS_VERSION`) → build for a concrete simulator. Debugging in one launch: `preLaunchTask` runs `simulator.sh` (launch `--wait-for-debugger`, writes the attach command), the config attaches with `attachCommands: command source .build/simulator-attach.lldb` (read at attach time, after the task). Earlier failures of `attachCommands`/`program` were caused by the wrong `sourceMap` (10a), not by lldb-dap.
10a. **Gray breakpoints in VS Code (macOS app):** the launch configs had `sourceMap` `/^wt → worktree` for all builds. lldb also applies it in reverse when setting breakpoints, so `<worktree>/…/File.swift` became `/^wt/…/File.swift`, which regular (Xcode-mode) debug info doesn't contain → "no locations". Fix: no `sourceMap`; `xcodebuild.sh` writes `.build/debug.lldbinit` per build (source map only for `DDG_WORKTREE_CACHE=1`), launch configs `command source` it in `initCommands`. Confirmed fixed in VS Code 2026-10-15.
11. Index sharing between worktrees: rejected (absolute paths + mtime-based staleness).
12. `workspace/_pollIndex` isn't in release sourcekit-lsp → indexer waits for "Indexing" progress end + 60 s quiet; gives up after 5 min without progress.
13. swift-build service runs in its own process group → stop children first.
14. Unrelated: main's `SharedPackages/BrowserServicesKit/Package.resolved` is stale (C-S-S 17.20 vs `exact: 17.26.0`) — separate task spawned.

## Swift Build fork and cache findings — 2026-10-16

The existing fork is `/Users/admin/Developer/swift-build-custom`, origin `mallexxx/swift-build`, branch `cache-provisioning-inputs`, commit [0d23a4dd5eb536fcdbc10ef83bfdad57e6a7a022](https://github.com/mallexxx/swift-build/commit/0d23a4dd5eb536fcdbc10ef83bfdad57e6a7a022) from June 2025. Its patch serializes `ProvisioningTaskInputs` to `<OBJROOT>/ProvisioningTaskInputs/<target-name>.provisioningTaskInputs.json` and reuses them during planning. **It caches signing/provisioning inputs, not package resolution or downloaded package artifacts.**

That checkout has no ready `.build` service binary. The old cache has no configuration/profile/entitlement freshness key, and its Xcode 27.1 compatibility has not been validated. It was not enabled. After porting, building and checking compatibility, the existing wrapper can select a service for one invocation without a global shell export:

```sh
XCBBUILDSERVICE_PATH=/absolute/products/path/SWBBuildServiceBundle \
  scripts/vscode/xcodebuild.sh build 'macOS Browser'
```

The fork's `Plugins/run-xcodebuild/run-xcodebuild.swift` demonstrates the override. Keep service resources alongside the executable and match service/client versions. No provisioning-cache speedup was measured.

**Current provisioning cost:** exact `GatherProvisioningInputs` section timestamps were extracted from five actual October 15 `macOS Browser` Debug build activity logs. All logged `XCODE_PRODUCT_BUILD_VERSION=27A9275`, matching the currently selected **Xcode 27.1 (27A9275)**. Durations were 0.621, 0.255, 1.111, 1.093 and 1.340 seconds: **median 1.09 s, range 0.25–1.34 s**. These are archived real-build measurements, not fresh full-build benchmarks or stdout approximations; they do not characterize cold signing/account setup or online provisioning updates. Evidence: `/Users/admin/DuckDuckGo/apple-browsers.git/vscode-local-setup/.build/.build-benchmark/provisioning-20261016/provisioning-summary.json`.

**Package/artifact reuse plan (not automatically enabled):** keep per-worktree `SourcePackages` and Xcode-compatible DerivedData; share standard download/repository caches, and optionally seed a new worktree using independent APFS copies from a matching prepared cache before normal resolution. Avoid one shared mutable checkout directory across branches. A future fast path should fingerprint the toolchain, workspace/project files, manifests, lockfiles and package configuration, verify cache completeness, then use `-skipPackageUpdates -onlyUsePackageVersionsFromResolvedFile` only when valid. On changed/missing inputs, resolve normally before updating the stamp. These flags still evaluate the package graph in each new xcodebuild process. Existing wrapper arguments already permit an explicit warm-cache trial:

```sh
scripts/vscode/xcodebuild.sh build 'macOS Browser' macOS \
  -skipPackageUpdates -onlyUsePackageVersionsFromResolvedFile
```

The existing `DDG_WORKTREE_CACHE=1` mode is a separate compilation/CAS optimization; its earlier 179 s → 90–120 s measurements were not re-run during this rollout.

## Review (independent agent, 2026-10-14)

Verdict: "close, not ready" → fixed: iOS platform reset by setup/indexer, unpinned `swift-tools-protocols` (now Package.resolved + `--force-resolved-versions`), purpose not in cache key, empty `include`, sources not generation-checked, test gaps (58 tests now), editor/indexer race (refuse + wait), stop() killing unchecked pids, indexer hanging 1 h, spaces in prefix maps, in-place install over running binaries, stale-install detection, test failures not in Problems, Xcode 26/27 wording, `re.escape`, `as_uri`, editor detection, xcbeautify missing warning, `files.watcherExclude`, lldb-dap pinned, risky `waitFor` attach removed, mismatched scheme/destination task.
Not changed: `Debug` default (documented; matches swift-build's index default).

## Verified (Xcode 27.1)

- Code intelligence: macOS app, macOS unit test (`@testable`, jump into app), package test, iOS app (ios mode) — in both layouts.
- `Test macOS Unit Tests` task (33 tests pass), `Build macOS Browser`, iOS build for simulator, install, launch, lldb-dap attach + breakpoint; macOS launch + breakpoint (CLI lldb).
- Background indexer: full run, skip when unchanged, editor takeover, no orphans.
- Installer from scratch: patch applies, pinned build, stamp.

Not verified: VS Code/Cursor UI itself; Xcode IDE reading our index / no-rebuild after VS Code builds (xcodebuild was used as Xcode's stand-in).

## Open items

- [ ] Walk through VS Code and Cursor UI (tasks, Problems, launch/attach, workspace trust prompt).
- [ ] Check Xcode IDE: open a worktree indexed/built from VS Code — no reindex, no rebuild.
- [x] Publish tested fixes to [mallexxx/sourcekit-xcode-bsp](https://github.com/mallexxx/sourcekit-xcode-bsp), pinned at `afd0e3dd026b3ee3619e2271809debee25fec96f`.
- [x] Deploy shared local hook and backfill all 97 registered worktrees (2026-10-16).
- [x] Extract reusable harness into `~/Developer/xcode-editor-tools`, migrate all existing worktrees and verify an unrelated Xcode app plus a fresh deployed-hook experiment (2026-10-16); published to [mallexxx/xcode-editor-tools](https://github.com/mallexxx/xcode-editor-tools) at the user's request, commit `b4a839b`.
- [ ] Establish a DuckDuckGo-owned fork/team rollout if approved.
- [ ] Offer the build-server fixes upstream (no upstream PR opened).
- [ ] Upstream issue: duplicate outputs/stray `.o` when all targets are configured; lldb-dap attach with `program`/`attachCommands` to simulator processes.
- [ ] PR (needs Asana task link).