git_admin/obsidian-vault
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[2026-10-16] eagle: work/projects/vscode-migration/tech-design-xcode-build-server.md work/projects/vscode-migration/vscode-migration.md work/projects/vscode-migration/xcode-build-server.md

Browse Source
This commit is contained in:
Alexey Martemyanov committed 2026-10-16 11:55:30 +06:00
1 parent d7ffe19f34
commit 37f97cb127
3 files changed
+64 -77
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

work/projects/vscode-migration/xcode-build-server.md
+60 -75
View File
@@ -1,6 +1,6 @@
# VS Code via Xcode build server (sourcekit-xcode-bsp)
Status: **local setup deployed to all 97 registered worktrees; app-repository setup remains uncommitted** · 2026-10-16
Status: **generic harness extracted to ~/Developer/xcode-editor-tools and deployed; 97 existing worktrees preserved + fresh hook experiment verified; nothing committed or published** · 2026-10-16
**Parent:** [[projects/vscode-migration/vscode-migration]]
**Tech design:** [[projects/vscode-migration/tech-design-xcode-build-server]]
@@ -13,32 +13,35 @@ An alternative to the [[projects/vscode-migration/tech-design-spm-browser-packag
| What | Where |
|---|---|
| Original migration worktree | `/Users/admin/DuckDuckGo/apple-browsers.git/vscode-xcode-bsp`, branch `alex/vscode-xcode-bsp`; original uncommitted setup preserved |
| Local bootstrap | `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/worktrees.py` |
| Master templates | `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/template/` |
| Active shared Git hook | `/Users/admin/DuckDuckGo/apple-browsers.git/.git/hooks/post-checkout` |
| Standalone harness repository | `/Users/admin/Developer/xcode-editor-tools` — generic source, README, configuration reference, examples, tests, MIT license, CI workflow; initialized on `main`, no commits, staged files, remote or publication |
| Integration instructions | `/Users/admin/Developer/xcode-editor-tools/README.md` and `CONFIGURATION.md` |
| Private Apple browser profile | `/Users/admin/DuckDuckGo/apple-browsers.git/.git/xcode-editor/config.json` |
| Active shared Git hook | `/Users/admin/DuckDuckGo/apple-browsers.git/.git/xcode-editor/hooks/post-checkout` |
| Hook installation/previous settings | `/Users/admin/DuckDuckGo/apple-browsers.git/.git/xcode-editor/installation.json` |
| Local ignore rules | `/Users/admin/DuckDuckGo/apple-browsers.git/.git/info/exclude` |
| Published BSP fork | [mallexxx/sourcekit-xcode-bsp](https://github.com/mallexxx/sourcekit-xcode-bsp), default branch `xcode-workspace-support`; pinned revision `afd0e3dd026b3ee3619e2271809debee25fec96f` |
| Fork checkout | `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/sourcekit-xcode-bsp`; the earlier draft at `/Users/admin/Developer/sourcekit-xcode-bsp` was preserved |
| Installed BSP binaries | `/Users/admin/.local/opt/sourcekit-xcode-bsp/`, shared by worktrees; existing patched installation was not replaced during the rollout |
| Compatibility entry point | `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/worktrees.py` — forwards old commands to the new CLI |
| Previous harness source/templates | `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/extraction-20261016/legacy-harness/` — rollback archive |
| Previous hook/config/excludes | `/Users/admin/DuckDuckGo/apple-browsers.git/.git/xcode-editor/backups/extraction-20261016/` |
| Original migration worktree | `/Users/admin/DuckDuckGo/apple-browsers.git/vscode-xcode-bsp`, branch `alex/vscode-xcode-bsp`; custom installer/README preserved |
| Published BSP fork | [mallexxx/sourcekit-xcode-bsp](https://github.com/mallexxx/sourcekit-xcode-bsp), default branch `xcode-workspace-support`; pin `afd0e3dd026b3ee3619e2271809debee25fec96f` |
| BSP fork checkout | `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/sourcekit-xcode-bsp`; earlier draft in `~/Developer/sourcekit-xcode-bsp` preserved |
| Shared installed BSP | `/Users/admin/.local/opt/sourcekit-xcode-bsp/`; existing patched executable/service/resources retained |
## Local hook and all-worktree rollout — 2026-10-16
## Standalone harness extraction and deployment — 2026-10-16
**Deployed and verified:** the real shared Git hook is active, and the setup has been applied to **all 97 registered Apple browser worktrees**, including the two checkouts beside `apple-browsers.git`. This is a local deployment on Alex's machine; no app-repository commit, PR, or upstream issue was created.
The harness now lives in **`~/Developer/xcode-editor-tools`**, ready for public publication as source but **not committed or published**. Its source contains no workstation paths, Apple browser names, schemes, product paths or private target filters. Those values are in the private Git profile above. The default editor workspace filename is derived from the configured Xcode container; an ordinary project, bare repository and externally located linked worktrees are supported. There are no third-party patches, binaries, package checkouts or local logs in the new repository. The separate BSP fork remains the public pinned dependency.
### Trigger and ownership
The deployed hook now runs that repository's CLI. All **97 existing worktrees** were refreshed from the generic implementation. A fresh experimental worktree, `/Users/admin/DuckDuckGo/apple-browsers.git/vscode-editor-tools-check`, was then created through ordinary `git worktree add --detach … main` and bootstrapped automatically: **19 generated files**, clean Git status, and zero rewrites on another apply. There are now **98 registered checked-out worktrees** including that experiment.
Git's repository-local `core.hooksPath` is already `/Users/admin/DuckDuckGo/apple-browsers.git/.git/hooks`. Its executable `post-checkout` hook runs after a normal `git worktree add`, `git checkout`, or `git switch` with the branch-checkout flag. It finds that worktree's root and runs:
### Trigger, hooks and preservation
```sh
python3 /Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/worktrees.py apply "$root"
```
This is a **Git `post-checkout` hook**, selected by repository-local `core.hooksPath = /Users/admin/DuckDuckGo/apple-browsers.git/.git/xcode-editor/hooks`. It runs after a normal `git worktree add`, `git checkout` or `git switch` when Git's branch-checkout flag is `1`. File-only checkouts are ignored. The hook discovers Git's common directory and applies the private profile to the worktree.
The bootstrap accepts worktrees sharing this repository's Git common directory, including paths outside the bare-repository container. It copies central templates rather than linking worktrees to each other. It generates each worktree's absolute workspace/launcher paths and Xcode DerivedData location, preserves the selected macOS/iOS platform, and uses hashes to update files it owns while preserving custom edits and tracked files.
Existing executable hooks are forwarded with their arguments, environment, working directory, standard streams and exit status. Default, absolute custom and relative hook paths are supported. The previous bootstrap hook was adopted only after verifying its exact SHA-256, then archived; it does not run a second bootstrap. Re-run `install` if adding another hook to the previous hook directory later.
A normal fresh-worktree creation was tested with the installed hook, without a temporary `core.hooksPath` override: all 17 files appeared automatically, were ignored, and left Git status clean. The hook itself runs no network requests, installer, build, or indexing job. Its failure is reported without failing checkout.
Tracked files, symlinks and locally customized generated files are preserved. Ownership remains in `.build/vscode-local/files.json` for this installation to adopt the earlier hashes. Newly installed repositories default to `.build/xcode-editor/files.json`. Generated runtime scripts are self-contained; the source checkout is needed by the Git hook for future generation.
Git does not run this hook for `worktree add --no-checkout`; a later checkout does. Tools that disable or bypass Git hooks need an explicit apply command. No cron, launchd job, or filesystem watcher was installed for bootstrapping.
The hook runs no network requests, installer, build or indexing job. Its own failure is printed without failing an otherwise successful checkout; a prior hook's failure status is retained. `worktree add --no-checkout`, per-worktree hook overrides and tools that bypass hooks require an explicit apply. No cron, watcher, launchd job or filesystem monitor was installed for bootstrap.
### Files inside each worktree
@@ -46,81 +49,62 @@ Git does not run this hook for `worktree add --no-checkout`; a later checkout do
<worktree>/
DuckDuckGo.code-workspace
buildServer.json
.vscode/
settings.json
tasks.json
launch.json
extensions.json
scripts/vscode/
.build/vscode-local/files.json
.vscode/{settings,tasks,launch,extensions}.json
scripts/vscode/ generated runtime and local usage README
.build/xcode-editor/config.json private runtime profile
.build/vscode-local/files.json adopted ownership hashes
```
`files.json` records ownership hashes. Each newly bootstrapped worktree receives 17 setup files; existing custom files may be retained instead. Open that worktree's `DuckDuckGo.code-workspace` in VS Code/Cursor. The binaries remain shared at `/Users/admin/.local/opt/sourcekit-xcode-bsp/`.
The shared `.git/info/exclude` retains its previous rules and adds a managed `xcode-editor-tools` block covering the generated editor workspace, scripts, `.vscode`, discovery file and runtime/ownership state. The old `/.vscode-local/` exclusion remains. No tracked `.gitignore` was changed.
The shared local `.git/info/exclude` contains:
Open the same worktree-local `DuckDuckGo.code-workspace`. macOS/iOS schemes, test selection, simulator launch/attach, regular Xcode DerivedData and target filters are preserved. Tasks use process arguments to handle spaces/quotes safely. `XCODE_EDITOR_WORKTREE_CACHE=1` is the generic opt-in CAS flag; the private profile also retains `DDG_WORKTREE_CACHE` and the old background-indexer flag for compatibility. Regular builds still omit `-derivedDataPath`; no new cache mode was enabled.
```gitignore
/DuckDuckGo.code-workspace
/scripts/vscode/
/.vscode/
/buildServer.json
/.vscode-local/
```
No tracked `.gitignore` was edited. Existing tracked files remain tracked despite these exclusions.
### Commands and rollback
The following install and backfill commands have already been run:
### Maintenance and removal
```sh
python3 /Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/worktrees.py install
python3 /Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/worktrees.py apply --all
tools=/Users/admin/Developer/xcode-editor-tools/xcode_editor.py
repo=/Users/admin/DuckDuckGo/apple-browsers.git
python3 "$tools" status --repo "$repo"
python3 "$tools" apply --repo "$repo" --all
python3 "$tools" apply --repo "$repo" /absolute/worktree/path --dry-run
python3 "$tools" install --repo "$repo"
python3 "$tools" uninstall --repo "$repo"
```
For an individual worktree or a preview:
Edit the private profile, then run `apply --all` to propagate profile changes. `init` refuses to replace an existing profile. After moving the harness checkout, run `install` from its new location; after moving the target repository, also run `apply --all`. Absolute paths in the private installed hook and generated BSP files are derived at installation time.
```sh
python3 /Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/worktrees.py apply /absolute/worktree/path
python3 /Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/worktrees.py apply /absolute/worktree/path --dry-run
```
`uninstall` restores the earlier local `core.hooksPath` (here, `…/.git/hooks`) and leaves generated files, ownership and exclusions intact. Because the superseded bootstrap hook has been archived, automatic setup stops. The old `.vscode-local/worktrees.py apply …` command remains a forwarding compatibility shim. For full rollback, the preceding harness, hook, Git config and exclude file are retained at the backup paths above; restoring a full old config should be reviewed against any intervening Git changes.
The installer refuses to replace an existing different hook. To disable future automatic runs, move the installed `post-checkout` file out of the hooks directory (for example to `.vscode-local/post-checkout.disabled`); existing generated files remain. There is no dedicated uninstall command.
### Verification and existing exceptions
### Rollout verification and legacy exceptions
- All **97 existing BSP configurations were identical** to their pre-extraction JSON, including platform, target filters, worktree paths and DerivedData.
- All **23 tracked integration files** and the customized settings file were unchanged. The new installer continues to preserve original migration-worktree custom files.
- **1,992 ignored-file checks** passed across existing ownership manifests; no structural failures.
- **47 unit tests** passed across the bootstrap and runtime suites. They cover ordinary/bare repositories, external and relocated worktrees, hook chaining/removal/adoption, ownership, path quoting, ignore metacharacters, platform persistence, build parameters, exit status, CAS/debug mapping and the original helper API.
- An unrelated minimal **OrbitSample** macOS application built and ran through the new wrapper; the actual installed BSP discovered its target. This used Xcode 27.1 and exposed/fixed Foundation's `/private` temporary-path spelling difference when locating DerivedData.
- The actual deployed hook generated the fresh browser experiment automatically; its folder-open setup command also verified `buildServer.json` was already current.
- Public source scan: no local paths or project names; Python/JSON/shell syntax and whitespace checks passed. CI is configured but has not run remotely. No new full browser build, editor UI walkthrough or indexing run was performed in every worktree.
The backfill wrote 1,526 files across 97 worktrees and preserved 45 existing setup/editor-file entries. Structural verification found **zero failures**: expected files existed, server names and absolute paths matched each worktree, wrappers were executable, generated editor JSON parsed, and newly created files were Git-ignored. This was not a new app build, indexing run, or debugger test in every worktree.
The first rollout had written 1,526 files across 97 worktrees and preserved 45 existing/tracked entries. Its diagnostics remain in `.vscode-local/backfill-*.json` and `backfill.log`. The old configurations in `exp` and `sweetpad` were backed up at `.vscode-local/backups/backfill-20261015T185058Z/{exp,sweetpad}/buildServer.json` before migration. **`exp/buildServer.json` was already tracked** and remains its earlier intentional uncommitted modification. A branch checkout may restore its tracked legacy config; the explicit folder-open setup command refreshes it. The extraction introduced no new tracked changes.
The old `xcode-build-server` configurations in `exp` and `sweetpad` were migrated after backing up their originals:
Extraction evidence: `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/extraction-20261016/`: `before.json`, `verification.json`, `apply.log`, `fresh-worktree.log`, private profile snapshot and legacy harness backup.
- `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/backups/backfill-20261015T185058Z/exp/buildServer.json`
- `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/backups/backfill-20261015T185058Z/sweetpad/buildServer.json`
### Publication scope
**Tracked-file exception:** the `exp` branch already tracks `buildServer.json`. Its migration is therefore an intentional **uncommitted local modification**, not an ignored file. All other previously tracked setup files were preserved unchanged. The hook skips tracked files, so a later branch checkout can restore that branch's tracked legacy config; the existing folder-open setup task regenerates the selected BSP config when opening the workspace.
Evidence:
- `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/backfill-report.json`
- `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/backfill.log`
- `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/backfill-before.json`
- `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/backfill-migrations.json`
- `/Users/admin/DuckDuckGo/apple-browsers.git/.vscode-local/research.txt`
### Publishing scope
The BSP fixes are public in the personal fork, with a committed dependency lockfile. All 58 tests and strict SwiftLint on the changed Swift files passed. The local template installer pins `afd0e3dd026b3ee3619e2271809debee25fec96f`, uses `xcrun swift` and `--force-resolved-versions`, and no longer applies a patch during installation. The old patch snapshot remains for compatibility with the installed fingerprint check; the original migration worktree's differing installer/README were preserved.
The bootstrap is still specific to this project: workspace/project names, schemes, app paths and target filters are hardcoded. Its installer also assumes an existing absolute `core.hooksPath`. A general public bootstrap needs configuration, Git layout/hook handling, packaging, removal support, and tests against another Xcode project. A DuckDuckGo-owned fork, team rollout and upstream PRs remain separate work.
The new generic harness has README integration instructions, a configuration reference, minimal and two-platform examples, MIT license, dependency/license notice, contributor instructions, automated tests and a CI workflow. It has **no commits, staged changes or Git remote**; nothing was posted or published. The separate BSP fork is already public; its earlier 58 tests and strict SwiftLint validation remain recorded. A DuckDuckGo-owned fork, team deployment and upstream PRs remain proposed.
## Files (`scripts/vscode/`)
- `setup-build-server.py [macos|ios]` — writes `buildServer.json` on folder open. Keeps the current platform without an argument. Build root = the worktree's Xcode DerivedData, `derivedDataLayout: xcode`, `configuration: Debug`, target filter (app project + `*Tests-product`, no App Store variants, no other-platform app/local packages). Warns when the installed BSP was built from an older patch.
- `xcode_derived_data.py` — Xcode's DerivedData folder for the worktree (MD5 → 28 letters; honors a custom location).
- `sourcekit-xcode-bsp.sh` — `argv` of buildServer.json. Editor's server stops a running background indexer and waits for it; the indexer's server refuses to start next to an editor's. Pid files in `.build/bsp/`, checked to be sourcekit processes.
- `xcodebuild.sh <build|test> <scheme> [destination]` — Debug into Xcode's DerivedData with Xcode's settings (no `-derivedDataPath`: passing it changes build parameters and makes Xcode and VS Code rebuild after each other). `DDG_WORKTREE_CACHE=1`: `.build/xcodebuild` + shared CAS with `/^wt` `/^dd` maps. `.build/products` → products folder. xcbeautify + raw `path:line: error:` lines for failed tests.
- `simulator.sh [--build-only|--no-build] [device]` — builds iOS Browser for a concrete simulator, installs, `simctl launch --wait-for-debugger`, writes `process attach --pid N` to `.build/simulator-attach.lldb`.
- `index-in-background.py [--all] [--if-idle]` — headless indexing; `background-indexing.sh install` = launchd agent every 30 min.
- `install-build-server.sh` — the central template builds the published fork at the pinned commit using its committed `Package.resolved`; the archival patch/lockfile snapshots remain for compatibility with earlier local installations. The original migration worktree's differing installer was preserved.
- `editor_support.py` — self-contained configuration, Git, BSP rendering, environment compatibility, DerivedData and debugger helpers.
- `setup-build-server.py [macos|ios]` — refresh discovery on folder open, preserving the existing platform without an argument; checks the installed BSP supports target selection.
- `xcode_derived_data.py` — print the configured container's Xcode DerivedData; retains the original one-argument helper for preserved scripts.
- `sourcekit-xcode-bsp.sh` — launch the editor's BSP; coordinate per-worktree PID files with the optional headless indexer.
- `xcodebuild.sh <action> <scheme> [destination] [args…]` — regular Xcode-compatible parameters, optional compilation CAS, products symlink, debugger init, formatted diagnostics and original build status.
- `simulator.sh [--build-only|--no-build|--test SCHEME] [--only-testing TARGET] [device]` — choose a concrete simulator, build/run or test; launch suspended and generate an attach command.
- `index-in-background.py [--all] [--if-idle]` and `background-indexing.sh install|status|uninstall` — optional headless indexing and per-repository launchd agent. No agent was installed.
- `install-build-server.sh` / `install_server.py` — explicit pinned fork installation with `--force-resolved-versions`, co-located service/resources and a retained previous installation. No patch is vendored by the generic harness; old worktree snapshots remain only for preserved legacy scripts.
- `manage_background.py` and `README.md` — launchd management and local usage.
## Findings
@@ -184,6 +168,7 @@ Not verified: VS Code/Cursor UI itself; Xcode IDE reading our index / no-rebuild
- [ ] Check Xcode IDE: open a worktree indexed/built from VS Code — no reindex, no rebuild.
- [x] Publish tested fixes to [mallexxx/sourcekit-xcode-bsp](https://github.com/mallexxx/sourcekit-xcode-bsp), pinned at `afd0e3dd026b3ee3619e2271809debee25fec96f`.
- [x] Deploy shared local hook and backfill all 97 registered worktrees (2026-10-16).
- [x] Extract reusable harness into `~/Developer/xcode-editor-tools`, migrate all existing worktrees and verify an unrelated Xcode app plus a fresh deployed-hook experiment (2026-10-16); source remains uncommitted/unpublished.
- [ ] Establish a DuckDuckGo-owned fork/team rollout if approved.
- [ ] Offer the build-server fixes upstream (no upstream PR opened).
- [ ] Upstream issue: duplicate outputs/stray `.o` when all targets are configured; lldb-dap attach with `program`/`attachCommands` to simulator processes.