Legacy XCTest runner

Runs DDG macOS UI tests built with the current Xcode (27.x) on a Mac that can only run an old Xcode, e.g. the MBP 2014 node (Monterey 12.6.8, Intel, Xcode 13.4.1).

Location: ~/Developer/legacy-xctest-runner/ (not committed anywhere).

Why it is needed

New Xcode's XCTest.framework and UI Tests-Runner.app require macOS 14+.
The test bundle itself targets the app's deployment target (12.3), so it can run if hosted by the old Xcode's runner.
The bundle imports 2 Swift symbols missing from Xcode 13 XCTest: XCUIElement.typeKey(_:modifierFlags:) (mangled under XCUIAutomation) and XCTIssue.init(…severity:). XCTestCompat.swift forwards them.

How it works

package.sh <derived-data> <out.tgz> — collects DuckDuckGo Review.app, tests-server, UI Tests.xctest + runner files.
run.sh [--gui] <TestClass[/testMethod]> [result-name] on the target Mac:
- builds XCTestCompat.framework (re-exports old XCTest + libXCTestSwiftSupport, adds forwarders);
- hosts the bundle in Xcode 13's XCTRunner.app, install_name_tool repoints the bundle's XCTest/libXCTestSwiftSupport imports at XCTestCompat;
- repeats macOS UI Tests CI scheme pre-actions (clean-app review, suppress Move-to-Applications, start tests-server);
- xcodebuild test-without-building with UITests.xctestrun (FormatVersion 1), then kills tests-server and the app (old XCTest doesn't terminate it).
- --gui: over SSH xcodebuild can't reach the runner; relaunches itself via a Terminal .command in the console session, waits, closes finished windows.

Build on the host (spell flags out — zsh doesn't word-split $var):

xcodebuild build-for-testing -workspace DuckDuckGo.xcworkspace -scheme "macOS UI Tests CI" -destination "platform=macOS,arch=x86_64" -derivedDataPath <dd> ONLY_ACTIVE_ARCH=YES -skipPackagePluginValidation -skipMacroValidation
xcodebuild build -workspace DuckDuckGo.xcworkspace -scheme "macOS Browser" -configuration Review -destination "platform=macOS,arch=x86_64" -derivedDataPath <dd> ONLY_ACTIVE_ARCH=YES -skipPackagePluginValidation -skipMacroValidation

Target Mac prerequisites (once per boot volume, needs sudo)

sudo DevToolsSecurity -enable
sudo automationmodetool enable-automationmode-without-authentication

Without them testmanagerd shows an admin-password prompt on the display ("Timed out while enabling automation mode"). Virfield's node probe now reports ui_testing (ready / devtools_disabled / automation_mode_requires_auth / no_console_session); setup script /Users/admin/Developer/virfield/deploy/node-ui-testing.sh.

Gotchas found

Repo Logger (macOS/UITests/Common/UITests.swift) calls private -[XCTContext _recordActivityMessageWithFormat:]; old XCTest lacks it → guard with responds(to:) (done on branch alex/fix-save-credentials-tooltip-crash-macos12).
Remote shell is zsh: log is a builtin — use /usr/bin/log show.
MBP has a different SSH host key per boot volume; use the Virfield lease's generated ssh-config (pins the booted volume's key).
Hover tracking in the app uses .activeInKeyWindow; popovers that open without focus don't get hover events.

First used: 2026-10-12, reproducing Sentry crash NSInternalInconsistencyException SaveCredentialsViewController.loadView (isfinite(newOrigin.x) on macOS 12) — reproduced before fix on the MBP. See also virfield.

3.6 KiB Executable File Raw Blame History

Legacy XCTest runner

Why it is needed

How it works

Target Mac prerequisites (once per boot volume, needs sudo)

Gotchas found

3.6 KiB

Executable File

Raw Blame History