obsidian-vault/work/wiki/apple-browsers/ci-pipeline.md

source, confidence, namespace, last_updated, tags

source	confidence	namespace	last_updated	tags
~/DuckDuckGo/apple-browsers.git/main/.github/workflows/	1.0	work	2026-04-28	ci github-actions xcodebuild testing build

CI Pipeline — macOS Browser

Reference for the GH Actions CI setup and how to run equivalent checks locally.

Xcode version: 26.4 (from .xcode-version)
Working directory for all macOS jobs: macOS/ inside the repo

---

Workflows Overview

Workflow file	Trigger	What it does
macos_pr_checks.yml	PR + push to main/release/hotfix	Unit tests, integration tests, release build, translations
macos_ui_tests.yml	PR (BSK changes) + push + schedule (3AM UTC)	Full UITest suite across macOS 14/15/26
macos_performance_tests.yml	PR + push + daily 5AM UTC	Performance benchmarks (notarized build required)
macos_pir_end_to_end_tests.yml	PR + push + schedule	PIR feature E2E tests
macos_build_notarized.yml	Called by other workflows	Builds notarized Review .app + optional DMG
macos_release.yml	Manual + release process	Full release build and publish
macos_check_sparkle_update.yml	Post-release	Validates Sparkle update feed

---

PR Checks (macos_pr_checks.yml)

Runs on every PR. Three parallel tracks:

Track 1: Unit + Integration Tests

Two matrix flavors run simultaneously:

Flavor	Scheme	Runner	active-arch
Non-Sandbox	macOS Browser	macos-26	YES
Sandbox	macOS Browser App Store	macos-26-xlarge	NO

Unit tests command:

cd macOS
set -o pipefail && xcodebuild test \
  -scheme "macOS Browser" \
  -derivedDataPath "DerivedData" \
  -configuration "CI" \
  -skipPackagePluginValidation -skipMacroValidation \
  -test-timeouts-enabled YES \
  -default-test-execution-time-allowance 60 \
  -maximum-test-execution-time-allowance 120 \
  ENABLE_TESTABILITY=true \
  ONLY_ACTIVE_ARCH=YES \
  COMPILATION_CACHE_ENABLE_CACHING=YES \
  "-skip-testing:Integration Tests" \
  2>&1 | xcbeautify

Integration tests command (same flags, opposite -testing: flag):

  "-only-testing:Integration Tests" \
  -retry-tests-on-failure \

Track 2: Release Build

cd macOS
set -o pipefail && xcodebuild \
  -scheme "macOS Browser" \
  -derivedDataPath "DerivedData" \
  -configuration "Release" \
  -skipPackagePluginValidation -skipMacroValidation \
  COMPILATION_CACHE_ENABLE_CACHING=YES \
  2>&1 | xcbeautify

Track 3: ShellCheck + Bats (shell scripts)

# Bats shell tests
brew install bats-core
cd macOS && bats --formatter junit scripts/tests/*

---

UI Tests (macos_ui_tests.yml)

How it works in CI

1. Build notarized app first (macos_build_notarized.yml) — runs as a dependency
2. Setup: lists all UITestCase-derived classes from UITests/ directory
3. Matrix: each test class × each macOS version runs as a separate job
4. Retry: -test-iterations 2 -retry-tests-on-failure -test-repetition-relaunch-enabled YES

Running locally (two approaches)

Option A — Direct xcodebuild (host only, no VM):

cd ~/DuckDuckGo/apple-browsers.git/main/macOS

# Build for testing first
set -o pipefail && xcodebuild build-for-testing \
  -scheme "macOS UI Tests CI" \
  -derivedDataPath DerivedData \
  -skipPackagePluginValidation -skipMacroValidation \
  2>&1 | xcbeautify

# Run a specific test class
defaults write com.duckduckgo.macos.browser.review moveToApplicationsFolderAlertSuppress 1
set -o pipefail && xcodebuild test-without-building \
  -scheme "macOS UI Tests CI" \
  -derivedDataPath DerivedData \
  -skipPackagePluginValidation -skipMacroValidation \
  '-only-testing:UI Tests/AutocompleteTests' \
  2>&1 | xcbeautify

Option B — Via VM (virfield/ddg-vm MCP, matches CI environment):

# 1. Build on host first (DerivedData must exist)
xcodebuild build-for-testing -scheme "macOS UI Tests CI" -derivedDataPath DerivedData ...

# 2. Start VM session
vm_prepare_session()  → { vmName, ip }

# 3. Run tests in VM
run_tests(vm_id: vmName, scheme: "DuckDuckGo macOS", workspace: "<VMShare path>")

# 4. Poll
get_test_run_status(vm_id: vmName, run_id: run_id)
get_test_results(vm_id: vmName, run_id: run_id)

# 5. On failure: inspect
peekaboo_image(vm_id: vmName)
get_crash_reports(vm_id: vmName)

# 6. Always clean up
vm_stop(vm_id: vmName)
vm_delete(vm_id: vmName)

See vm-ui-testing.md for full ddg-vm MCP reference.

Key notes

• All UI test classes must inherit from UITestCase, not XCTestCase — CI enforces this
• CI uses a notarized Review build (DuckDuckGo Review.app), not Debug
• Screen resolution is set to 1920×1080 in CI (macOS 15+) — VM does the same
• Build on host before running in VM — run_tests uses the pre-built xctestrun from DerivedData

---

Executor Build Command (for Executor agent)

The Executor uses Debug configuration without code signing or notarization:

cd ~/DuckDuckGo/apple-browsers.git/.claude/worktrees/executor-{gid}-{slug}/macOS
set -e -o pipefail && xcodebuild \
  -workspace DuckDuckGo-macOS.xcworkspace \
  -scheme "DuckDuckGo macOS" \
  -configuration Debug \
  build \
  2>&1 | xcbeautify

Key differences from CI:

• No -derivedDataPath override — uses default DerivedData in worktree
• No ONLY_ACTIVE_ARCH — defaults to YES for Debug, faster
• No COMPILATION_CACHE_ENABLE_CACHING=YES — not needed for single-run
• No -skipPackagePluginValidation unless you hit plugin errors (add if needed)

---

Xcode Scheme Reference

Scheme	Purpose	Used for
macOS Browser	Non-Sandbox (direct distribution)	Unit tests, release build
macOS Browser App Store	Sandbox variant	Unit tests (sandbox)
macOS UI Tests CI	UI test runner	UI tests (requires notarized Review app)
DuckDuckGo macOS	Development scheme	Executor Debug build

---

CI Runner Reference

Runner	Used for
macos-26	Unit tests (Non-Sandbox), release build
macos-26-xlarge	Unit tests (Sandbox), UI tests (latest macOS)
macos-15-xlarge	UI tests on macOS 15
macos-14-xlarge	UI tests on macOS 14
ubuntu-latest	ShellCheck, translation checks, Asana tasks

---

Common Local Equivalents

CI step	Local command
Select Xcode version	sudo xcode-select -s /Applications/Xcode_26.4.app (check .xcode-version)
Sync code signing	cd macOS && bundle exec fastlane sync_signing_ci
Build + unit test	See "Unit tests command" above
Build for UI test	xcodebuild build-for-testing -scheme "macOS UI Tests CI" ...
Run one UI test class	xcodebuild test-without-building ... '-only-testing:UI Tests/ClassName'
Check private API usage	cd macOS && ./scripts/find_private_symbols.sh "DerivedData/Build/Products/..."
Verify npm bundles	npm run rebuild-autoconsent --workspace=macOS

---

Cross-Reference

• vm-ui-testing.md — full virfield/ddg-vm MCP reference for VM-based UI tests
• testing.mdc → testing.md — UITestCase patterns, feature flag setup in tests
• development-commands.mdc → development-commands.md — team-approved xcodebuild flags