69 lines
8.4 KiB
Markdown
69 lines
8.4 KiB
Markdown
# VS Code via Xcode build server (sourcekit-xcode-bsp)
|
||
|
||
Status: **ready for early adopters, uncommitted** · 2026-10-14
|
||
**Parent:** [[projects/vscode-migration/vscode-migration]]
|
||
**Tech design:** [[projects/vscode-migration/tech-design-xcode-build-server]]
|
||
|
||
An alternative to the [[projects/vscode-migration/tech-design-spm-browser-package|SwiftPM browser package]]: keep the Xcode projects as they are and give VS Code/Cursor what it needs from them.
|
||
- Code intelligence: sourcekit-lsp ← `buildServer.json` ← [sourcekit-xcode-bsp](https://github.com/slime-studio/sourcekit-xcode-bsp) (BSP server on top of swift-build, Xcode's build engine; early-stage 0.1.0, Apache 2.0), patched by us.
|
||
- Build/test/debug: `xcodebuild` tasks + lldb-dap launch configs, macOS and iOS simulator.
|
||
- Builds and index shared with Xcode through the worktree's DerivedData folder.
|
||
|
||
## Where things are
|
||
|
||
| What | Where |
|
||
|---|---|
|
||
| VS Code setup | worktree `vscode-xcode-bsp`, branch `alex/vscode-xcode-bsp` — `DuckDuckGo.code-workspace`, `scripts/vscode/` (uncommitted) |
|
||
| BSP fork | `~/Developer/sourcekit-xcode-bsp`, branch `configuration-name` (uncommitted), base upstream `726ce706`; identical to `scripts/vscode/sourcekit-xcode-bsp.patch` |
|
||
| Installed BSP | `~/.local/opt/sourcekit-xcode-bsp` (by `install-build-server.sh`, with `patch.sha256` stamp); brew 0.1.0 also installed but unused |
|
||
|
||
## Files (`scripts/vscode/`)
|
||
|
||
- `setup-build-server.py [macos|ios]` — writes `buildServer.json` on folder open. Keeps the current platform without an argument. Build root = the worktree's Xcode DerivedData, `derivedDataLayout: xcode`, `configuration: Debug`, target filter (app project + `*Tests-product`, no App Store variants, no other-platform app/local packages). Warns when the installed BSP was built from an older patch.
|
||
- `xcode_derived_data.py` — Xcode's DerivedData folder for the worktree (MD5 → 28 letters; honors a custom location).
|
||
- `sourcekit-xcode-bsp.sh` — `argv` of buildServer.json. Editor's server stops a running background indexer and waits for it; the indexer's server refuses to start next to an editor's. Pid files in `.build/bsp/`, checked to be sourcekit processes.
|
||
- `xcodebuild.sh <build|test> <scheme> [destination]` — Debug into Xcode's DerivedData with Xcode's settings (no `-derivedDataPath`: passing it changes build parameters and makes Xcode and VS Code rebuild after each other). `DDG_WORKTREE_CACHE=1`: `.build/xcodebuild` + shared CAS with `/^wt` `/^dd` maps. `.build/products` → products folder. xcbeautify + raw `path:line: error:` lines for failed tests.
|
||
- `simulator.sh [--build-only|--no-build] [device]` — builds iOS Browser for a concrete simulator, installs, `simctl launch --wait-for-debugger`, pid to `.build/simulator.pid`.
|
||
- `index-in-background.py [--all] [--if-idle]` — headless indexing; `background-indexing.sh install` = launchd agent every 30 min.
|
||
- `install-build-server.sh` + `sourcekit-xcode-bsp.patch` + `sourcekit-xcode-bsp.Package.resolved` — reproducible build of the patched BSP.
|
||
|
||
## Findings
|
||
|
||
1. **Works for app code out of the box**, no Package.swift. Swift extension 2.16.7 activates on `buildServer.json`.
|
||
2. **Tests: nothing.** BSP had no configuration → swift-build used `defaultConfigurationName = Release` → no testability → `@testable import` fails. Fix: `configuration` (default Debug; swift-build's own index build defaults to Debug too).
|
||
3. **Stray `.o`, failed links, duplicate outputs.** BSP configured all 977 targets in one build (DMG + App Store, iOS + macOS, each package library 3–4 times). Fix: `targets` filter → 86 roots, 288 in the graph, 0 failures.
|
||
4. **Indexing ~2 h.** swift-build computes `sourceKitOptions` per file (~0.8 s, single thread, whole build request each time). Swift args of a target differ only in the trailing `-index-unit-output-path`. Fix: per-target cache keyed by (target, purpose) + single flight. Fresh worktree full index ~5.5 min (M4 Max, low load).
|
||
5. **Preparation** (~250 serialized prepare builds) is now the critical path.
|
||
6. **Shared CAS between worktrees (xcodebuild):** needs `SWIFT/CLANG_ENABLE_PREFIX_MAPPING` + `*_OTHER_PREFIX_MAPPINGS` = `<DerivedData>=/^dd <worktree>=/^wt`. Clean build 179 s → 90–120 s, 0 misses in a new worktree. Xcode's own project prefix mapping (`/^src` per project) breaks lldb sources; `/^wt` + `sourceMap` works. Opt-in now, because it can't be shared with Xcode (unconfirmed how Xcode's debugger handles mapped paths).
|
||
7. **CAS doesn't help the language server** (prepare-for-index doesn't cache; cold vs warm the same).
|
||
8. **Index shared with Xcode works:** with Xcode's DerivedData layout (index store = `Index.noindex/DataStore`, `INDEX_REGULAR_BUILD_*` = `Build/…`), units written by an xcodebuild build count as up to date: after building `macOS Browser`, indexing skipped all app files (2,150 files left vs 3,972). BSP prepares in `Index.noindex/SourceKitBuild` so it never shares a build DB with an open Xcode.
|
||
9. **Xcode-compatible builds:** `xcodebuild` without `-derivedDataPath` and the wrapper don't rebuild after each other (no-op 26–31 s). With `-derivedDataPath` pointing to the same folder they do.
|
||
10. **iOS simulator:** generic destination fails (`fix_libswiftWebkit_simulator.sh` needs `TARGET_DEVICE_OS_VERSION`) → build for a concrete simulator. Debugging: `simctl launch --wait-for-debugger` + lldb-dap attach by **pid** works (breakpoint in `AppDelegate`). lldb-dap with `attachCommands` or with `program` + pid never stops at breakpoints (CLI lldb does) → launch config uses `${command:pickProcess}` (newest first). VS Code substitutes variables before `preLaunchTask`, so run the task first.
|
||
10a. **Gray breakpoints in VS Code (macOS app):** the launch configs had `sourceMap` `/^wt → worktree` for all builds. lldb also applies it in reverse when setting breakpoints, so `<worktree>/…/File.swift` became `/^wt/…/File.swift`, which regular (Xcode-mode) debug info doesn't contain → "no locations". Fix: no `sourceMap`; `xcodebuild.sh` writes `.build/debug.lldbinit` per build (source map only for `DDG_WORKTREE_CACHE=1`), launch configs `command source` it in `initCommands`. Confirmed fixed in VS Code 2026-10-15.
|
||
11. Index sharing between worktrees: rejected (absolute paths + mtime-based staleness).
|
||
12. `workspace/_pollIndex` isn't in release sourcekit-lsp → indexer waits for "Indexing" progress end + 60 s quiet; gives up after 5 min without progress.
|
||
13. swift-build service runs in its own process group → stop children first.
|
||
14. Unrelated: main's `SharedPackages/BrowserServicesKit/Package.resolved` is stale (C-S-S 17.20 vs `exact: 17.26.0`) — separate task spawned.
|
||
|
||
## Review (independent agent, 2026-10-14)
|
||
|
||
Verdict: "close, not ready" → fixed: iOS platform reset by setup/indexer, unpinned `swift-tools-protocols` (now Package.resolved + `--force-resolved-versions`), purpose not in cache key, empty `include`, sources not generation-checked, test gaps (58 tests now), editor/indexer race (refuse + wait), stop() killing unchecked pids, indexer hanging 1 h, spaces in prefix maps, in-place install over running binaries, stale-install detection, test failures not in Problems, Xcode 26/27 wording, `re.escape`, `as_uri`, editor detection, xcbeautify missing warning, `files.watcherExclude`, lldb-dap pinned, risky `waitFor` attach removed, mismatched scheme/destination task.
|
||
Not changed: `Debug` default (documented; matches swift-build's index default).
|
||
|
||
## Verified (Xcode 27.1)
|
||
|
||
- Code intelligence: macOS app, macOS unit test (`@testable`, jump into app), package test, iOS app (ios mode) — in both layouts.
|
||
- `Test macOS Unit Tests` task (33 tests pass), `Build macOS Browser`, iOS build for simulator, install, launch, lldb-dap attach + breakpoint; macOS launch + breakpoint (CLI lldb).
|
||
- Background indexer: full run, skip when unchanged, editor takeover, no orphans.
|
||
- Installer from scratch: patch applies, pinned build, stamp.
|
||
|
||
Not verified: VS Code/Cursor UI itself; Xcode IDE reading our index / no-rebuild after VS Code builds (xcodebuild was used as Xcode's stand-in).
|
||
|
||
## Open items
|
||
|
||
- [ ] Walk through VS Code and Cursor UI (tasks, Problems, launch/attach, workspace trust prompt).
|
||
- [ ] Check Xcode IDE: open a worktree indexed/built from VS Code — no reindex, no rebuild.
|
||
- [ ] Upstream PRs to slime-studio/sourcekit-xcode-bsp (public repo, needs OK); or a DDG fork.
|
||
- [ ] Upstream issue: duplicate outputs/stray `.o` when all targets are configured; lldb-dap attach with `program`/`attachCommands` to simulator processes.
|
||
- [ ] PR (needs Asana task link).
|