Files
obsidian-vault/work/wiki/apple-browsers/ui-testing.md
T

1786 lines
62 KiB
Markdown

---
source: ~/DuckDuckGo/apple-browsers.git/main/.cursor/rules/ui-testing.mdc
confidence: 0.9
namespace: work
last_synced: 2026-04-28
alwaysApply: false
---
# UI Testing Guidelines & Best Practices
*This guide covers UI testing practices and patterns specifically for the DuckDuckGo macOS browser.*
## Overview
UI Tests verify the end-to-end user experience and interface behavior. They test user workflows, navigation patterns, window management, and complex interactions across the entire application.
## Setting Up UI Tests
### ❗Always Use UITestCase Base Class
```swift
class FeatureUITests: UITestCase { // ✅ Use UITestCase, not XCTestCase
private var app: XCUIApplication!
override func setUpWithError() throws {
continueAfterFailure = false
app = XCUIApplication.setUp() // ✅ Use setUp(), never launch() directly
// app is already launched and configured by setUp()
}
}
```
**Why UITestCase is Required**:
- Provides proper app lifecycle management
- Handles feature flag configuration
- Sets up test server environment
- Manages window state and cleanup
- Provides debugging utilities
### Feature Flag Configuration
```swift
// Configure feature flags during test setup
override func setUpWithError() throws {
app = XCUIApplication.setUp(featureFlags: [
"contextualOnboarding": true,
"visualUpdates": false,
"duckPlayer": true
])
// Feature flags are automatically applied via FEATURE_FLAGS environment variable
}
// Alternative: Custom environment
app = XCUIApplication.setUp(environment: [
"UITEST_MODE_ONBOARDING": "1"
], featureFlags: [
"newTabPageSections": true
])
```
#### Privacy Subfeature Configuration
```swift
// Configure privacy subfeatures (separate from feature flags)
override func setUpWithError() throws {
app = XCUIApplication.setUp(privacySubfeatures: [
"autoconsent-filterlist": true,
"tracker-allowlist": true
])
// Privacy subfeatures are applied via PRIVACY_SUBFEATURES environment variable
}
// Combined feature flags and privacy subfeatures
app = XCUIApplication.setUp(
featureFlags: [
"contextualOnboarding": true
],
privacySubfeatures: [
"autoconsent-filterlist": true
]
)
```
**❗Why Feature Flag and Privacy Subfeature Configuration is Critical**:
- UI tests run against notarized builds - feature flags can't be changed at runtime
- MockFeatureFlagger is NOT available in UI tests (only real DefaultFeatureFlagger)
- Feature flags must be configured via FEATURE_FLAGS environment variable before app launch
- **Privacy subfeatures are controlled by PrivacyConfiguration, not feature flags**
- Privacy subfeatures must be configured via PRIVACY_SUBFEATURES environment variable
- Incorrect feature/subfeature state will cause UI tests to fail when expected UI elements don't appear
**Key Differences**:
- **Feature Flags**: Control app features (e.g., `contextualOnboarding`, `duckPlayer`)
- **Privacy Subfeatures**: Control privacy functionality (e.g., `autoconsent-filterlist`, `tracker-allowlist`)
- Both use separate environment variables and configuration systems
### File Management in UI Tests
The `UITestCase` base class provides built-in file management capabilities for handling downloads, temporary files, and other file operations during testing.
**Important**: UI tests run in a sandboxed environment and cannot directly read or delete files from user directories using standard FileManager calls. For non-temp directories, use the `filesToCleanup` pattern that's handled in the base class `tearDown()`.
#### Automatic File Cleanup
All UI test classes automatically clean up tracked files after test completion:
```swift
class DownloadsUITests: UITestCase {
func testFileDownload() {
let downloadsDir = FileManager.default.urls(for: .downloadsDirectory, in: .userDomainMask)[0]
let fileName = "test-file.json"
let filePath = downloadsDir.appendingPathComponent(fileName).path
// Track file for automatic cleanup
trackForCleanup(filePath)
// Perform download test...
// File will be automatically cleaned up after test completes
}
}
```
#### Reading Files via Local Server
Use `readFileViaLocalServer()` to read files that may have permission restrictions:
```swift
func testJSONFileContent() throws {
let filePath = "/Users/admin/Downloads/test-results.json"
// Read file via local test server (bypasses permission issues)
let jsonData = try readFileViaLocalServer(filePath: filePath)
let results = try JSONDecoder().decode(TestResults.self, from: jsonData)
// Validate file contents
XCTAssertFalse(results.items.isEmpty)
}
```
#### File Management Best Practices
```swift
class FileBasedUITests: UITestCase {
func testCompleteFileWorkflow() throws {
// 1. Track all files that will be created
let tempDir = FileManager.default.temporaryDirectory.appendingPathComponent("test-files")
trackForCleanup(tempDir.path)
let downloadedFile = "/Users/admin/Downloads/results.json"
trackForCleanup(downloadedFile)
// 2. Perform file operations
try FileManager.default.createDirectory(at: tempDir, withIntermediateDirectories: true)
// 3. Read files via server if needed
let fileData = try readFileViaLocalServer(filePath: downloadedFile)
// 4. Files are automatically cleaned up in tearDown()
}
}
```
#### Available Methods
- **`trackForCleanup(_ path: String)`**: Track a file/directory for automatic cleanup
- **`readFileViaLocalServer(filePath: String) throws -> Data`**: Read file via local test server
- **Automatic cleanup**: All tracked files are cleaned up after each test via the base class `tearDown()`
## Element Access Patterns
### Accessibility IDs - The Golden Standard
**ALWAYS prefer accessibility IDs for element access**. They provide the most reliable and maintainable element identification.
#### Finding Existing Accessibility IDs
1. **Check the actual browser code** for assigned accessibility IDs:
```swift
// In browser code - look for patterns like:
button.accessibilityIdentifier = "AddressBarViewController.addressBarButton"
textField.identifier = "PreferencesGeneralView.switchToNewTabImmediately"
```
2. **Check XCUIApplication/XCUIElement extensions** for existing quick-accessor variables:
```swift
// Check Common/XCUIApplicationExtension.swift
extension XCUIApplication {
var addressBar: XCUIElement {
windows.textFields["AddressBarViewController.addressBarTextField"]
}
var backButton: XCUIElement {
buttons["NavigationBarViewController.backButton"]
}
}
```
3. **Use accessibility identifiers consistently**:
```swift
// ✅ CORRECT: Using accessibility IDs
let addressBar = app.textFields["AddressBarViewController.addressBarTextField"]
let bookmarksMenu = app.menuItems["BookmarksMenu.showBookmarks"]
let downloadButton = app.buttons["DownloadsViewController.downloadButton"]
// ❌ INCORRECT: Using text-based selectors
let addressBar = app.textFields["Enter search or URL"] // Fragile - breaks with localization
let bookmarksMenu = app.menuItems["Bookmarks"] // Fragile - breaks with text changes
```
#### When to Add New Accessibility IDs
**ALWAYS validate with user before modifying main app code** to add missing accessibility IDs:
```swift
// Before adding to main app code, ask:
// "I need to add accessibility ID 'TabBarViewController.newTabButton'
// to the new tab button in TabBarViewController.swift. Should I proceed?"
// Then add to the main app:
newTabButton.accessibilityIdentifier = "TabBarViewController.newTabButton"
```
#### Element Variable Guidelines
**Only create element variables in test cases if they are very specific to the testable area**:
```swift
// ✅ CORRECT: Test-specific elements
func testSpecificFeatureWorkflow() {
let featureSpecificButton = app.buttons["FeatureViewController.specialActionButton"]
let uniqueDialog = app.dialogs["FeatureDialog.confirmationDialog"]
// Use directly in test
}
```
**If elements are generic, add accessors to XCUIApplication/XCUIElement extensions**:
```swift
// ✅ CORRECT: Add to XCUIApplicationExtension.swift
extension XCUIApplication {
var downloadButton: XCUIElement {
buttons["DownloadsViewController.downloadButton"]
}
var preferencesWindow: XCUIElement {
windows["PreferencesWindow"]
}
}
// Then use in tests:
func testDownloadFlow() {
app.downloadButton.click()
XCTAssertTrue(app.downloadButton.exists)
}
```
### Timeout Constants Usage
**MANDATORY**: Always use `UITests.Timeouts` constants instead of hardcoded timeout values.
```swift
// ✅ CORRECT: Use semantic timeout constants
XCTAssertTrue(button.waitForExistence(timeout: UITests.Timeouts.elementExistence), "Button should appear")
XCTAssertTrue(pageContent.waitForExistence(timeout: UITests.Timeouts.navigation), "Page should load")
XCTAssertTrue(localContent.waitForExistence(timeout: UITests.Timeouts.localTestServer), "Local server content should load")
// ❌ INCORRECT: Hardcoded timeout values
XCTAssertTrue(button.waitForExistence(timeout: 5.0), "Button should appear")
XCTAssertTrue(pageContent.waitForExistence(timeout: 30.0), "Page should load")
XCTAssertTrue(localContent.waitForExistence(timeout: 15.0), "Local server content should load")
```
**Available Timeout Constants**:
- `UITests.Timeouts.elementExistence` (5 sec) - UI elements, buttons, text fields, dialogs
- `UITests.Timeouts.navigation` (30 sec) - Page loads, network requests, external sites
- `UITests.Timeouts.localTestServer` (15 sec) - Localhost connections, test server content
- `UITests.Timeouts.fireAnimation` (30 sec) - Fire animation completion
### Address Bar Validation Rules
**MANDATORY**: Always use `app.addressBarValueActivatingIfNeeded()` for address bar validation and prefer exact matches over contains checks.
```swift
// ✅ CORRECT: Use helper method with exact match for known URLs
XCTAssertEqual(app.addressBarValueActivatingIfNeeded(), "https://example.com/", "Should navigate to example.com")
XCTAssertEqual(app.addressBarValueActivatingIfNeeded(), "https://duckduckgo.com/", "Should be on DuckDuckGo")
// ❌ INCORRECT: Manual address bar access, partial comparison
app.activateAddressBar()
let addressBarValue = addressBarTextField.value as? String ?? ""
XCTAssertTrue(addressBarValue.contains("example.com"), "Should be on example.com")
// ❌ INCORRECT: Contains check for known exact URLs
let addressBarValue = app.addressBarValueActivatingIfNeeded() ?? ""
XCTAssertTrue(addressBarValue.contains("example.com"), "Should be on example.com") // Use XCTAssertEqual instead
```
**Address Bar Validation Guidelines**:
- **Use exact matches** (`XCTAssertEqual`) for known static URLs
- **Use contains checks** only for dynamic URLs (search results, localhost with ports)
- **Always use** `app.addressBarValueActivatingIfNeeded()` helper method
- **Never manually** call `app.activateAddressBar()` + `addressBarTextField.value`
### CRITICAL: XCUIElement Queries Are Always Live
**XCUIElement queries are always valid and re-query the UI when accessed (e.g., `exists`, `waitForExistence`). No need to create "fresh" element references:**
```swift
// ✅ CORRECT: Reuse the same element reference
class FeatureUITests: UITestCase {
private var addressBarTextField: XCUIElement!
override func setUpWithError() throws {
continueAfterFailure = false
app = XCUIApplication.setUp()
// Get address bar reference once
addressBarTextField = app.addressBar
}
func testAddressBarNavigation() throws {
// Type URL and navigate
addressBarTextField.typeText("example.com")
addressBarTextField.typeKey(.enter, modifierFlags: [])
// Wait for navigation - validate specific content, not generic webView existence
let webView = app.webViews.firstMatch
let pageContent = webView.staticTexts.containing(NSPredicate(format: "value CONTAINS 'Example Domain'")).firstMatch
XCTAssertTrue(pageContent.waitForExistence(timeout: 30.0), "Should navigate to example.com and show page content")
// ✅ CORRECT: Reuse original reference - XCUIElement queries are live
app.activateAddressBar() // Use helper method instead of manual Cmd+L
XCTAssertTrue(addressBarTextField.exists, "Address bar should still be accessible")
// ✅ CORRECT: The same element reference works after navigation
addressBarTextField.typeText("another-site.com")
}
}
// ❌ INCORRECT: Creating "fresh" element references unnecessarily
func testBadPattern() {
addressBarTextField.typeText("example.com")
// ❌ Wrong: No need to create fresh reference
let freshAddressBar = app.textFields["AddressBarViewController.addressBarTextField"]
let currentAddressBar = app.textFields["AddressBarViewController.addressBarTextField"]
// The original addressBarTextField reference is still valid!
}
```
### Element Access Hierarchy
1. **Accessibility IDs** (most reliable)
2. **Extension-provided accessors** (for common elements)
3. **Stable attributes** (for dynamic content)
4. **Text-based selectors** (last resort, fragile)
## Window and Tab Management
### Essential Window/Tab Validation Patterns
Based on our tab navigation testing improvements, always validate both UI state and browser state:
#### Single-Window Tab Operations
```swift
func testTabOperation() {
// Perform action that opens new tab
link.click()
// ✅ CORRECT: Wait first, then validate counts
XCTAssertTrue(app.tabs["New Tab Page"].waitForExistence(timeout: UITests.Timeouts.elementExistence))
XCTAssertEqual(app.windows.count, 1) // Still single window
XCTAssertEqual(app.tabs.count, 2) // Original + new tab
// Validate both tabs exist
XCTAssertTrue(app.tabs["Original Page"].exists)
XCTAssertTrue(app.tabs["New Tab Page"].exists)
// Validate webview state
XCTAssertTrue(app.webViews["New Tab Page"].exists)
}
```
#### Multi-Window Operations
```swift
func testWindowOperation() {
// Perform action that opens new window
XCUIElement.perform(withKeyModifiers: [.command, .option]) {
link.click()
}
// ✅ CORRECT: Wait for new window, then validate structure
let mainWindow = app.windows.firstMatch
let backgroundWindow = app.windows.element(boundBy: 1)
XCTAssertTrue(backgroundWindow.waitForExistence(timeout: UITests.Timeouts.elementExistence))
XCTAssertEqual(app.windows.count, 2)
// Validate content in correct windows
XCTAssertTrue(backgroundWindow.webViews["New Window Page"].exists)
XCTAssertFalse(mainWindow.webViews["New Window Page"].exists)
XCTAssertTrue(mainWindow.webViews["Original Page"].exists)
// Validate tab counts per window
XCTAssertEqual(mainWindow.tabs.count, 1)
XCTAssertEqual(backgroundWindow.tabs.count, 1)
XCTAssertTrue(mainWindow.tabs["Original Page"].exists)
XCTAssertTrue(backgroundWindow.tabs["New Window Page"].exists)
}
```
#### Window Focus Behavior
**Important**: When windows are activated/deactivated, the `firstMatch` window changes:
```swift
func testWindowActivation() {
// Initially: Window A is active (firstMatch), Window B is background
let initialActiveWindow = app.windows.firstMatch
let backgroundWindow = app.windows.element(boundBy: 1)
// Click on background window to activate it
backgroundWindow.click()
// Now: Window B becomes firstMatch, Window A becomes background
let newActiveWindow = app.windows.firstMatch // This is now Window B
let newBackgroundWindow = app.windows.element(boundBy: 1) // This is now Window A
// Validate the swap occurred
XCTAssertNotEqual(initialActiveWindow, newActiveWindow)
}
```
### Navigation Modifier Patterns
Understanding modifier key behaviors for comprehensive testing:
#### Tab Opening Modifiers
- **Command Click**: Opens in background tab
- **Command+Shift Click**: Opens in foreground tab (switches to it)
- **Middle Click**: Opens in background tab
- **Middle+Shift Click**: Opens in foreground tab
#### Window Opening Modifiers
- **Command+Option Click**: Opens in background window
- **Command+Option+Shift Click**: Opens in foreground window (switches to it)
- **Middle+Option Click**: Opens in background window
- **Middle+Option+Shift Click**: Opens in foreground window
#### Validation Patterns by Modifier
```swift
// Background tab (Command click)
func testCommandClickOpensBackgroundTab() {
XCUIElement.perform(withKeyModifiers: [.command]) {
link.click()
}
XCTAssertTrue(app.tabs["New Tab"].waitForExistence(timeout: UITests.Timeouts.elementExistence))
XCTAssertEqual(app.windows.count, 1)
XCTAssertEqual(app.tabs.count, 2)
// Original page still visible (background tab behavior)
XCTAssertTrue(app.webViews["Original Page"].exists)
XCTAssertTrue(app.tabs["Original Page"].exists)
XCTAssertTrue(app.tabs["New Tab"].exists)
}
// Foreground tab (Command+Shift click)
func testCommandShiftClickOpensActiveTab() {
XCUIElement.perform(withKeyModifiers: [.command, .shift]) {
link.click()
}
XCTAssertTrue(app.webViews["New Tab"].waitForExistence(timeout: UITests.Timeouts.elementExistence))
XCTAssertEqual(app.windows.count, 1)
XCTAssertEqual(app.tabs.count, 2)
// New page visible, original page in background (foreground tab behavior)
XCTAssertFalse(app.webViews["Original Page"].exists)
XCTAssertTrue(app.tabs["Original Page"].exists) // Tab still exists
XCTAssertTrue(app.tabs["New Tab"].exists)
}
```
## Element Interaction Patterns
### Timing and Existence Best Practices
**Critical**: Always wait for elements before checking counts or states:
```swift
// ❌ BAD: Checking counts before content appears
action()
XCTAssertEqual(app.windows.count, 1) // Race condition!
XCTAssertTrue(element.waitForExistence(...))
// ✅ GOOD: Wait first, then validate
action()
XCTAssertTrue(element.waitForExistence(timeout: UITests.Timeouts.elementExistence))
XCTAssertEqual(app.windows.count, 1) // Now safe to check
```
### Interaction Methods
#### Safe Element Interaction
```swift
// ✅ GOOD - Use existing helper methods
element.clickAfterExistenceTestSucceeds()
element.hoverAfterExistenceTestSucceeds()
element.typeURLAfterExistenceTestSucceeds(testURL)
// ✅ GOOD - URL handling with colon workaround
element.typeURL(url, pressingEnter: true) // Handles colon typing issues
element.pasteURL(url, pressingEnter: true) // Faster than typing
// ✅ GOOD - Element disappearance tracking
element.waitForNonExistence(timeout: UITests.Timeouts.elementExistence)
```
#### Helper Method Organization
**CRITICAL: Move common helper methods to `XCUIApplication` extensions instead of duplicating them across test classes:**
```swift
// ❌ BAD: Duplicating helper methods across test classes
class FeatureUITests: UITestCase {
private func setupSingleWindow() {
app.typeKey("w", modifierFlags: [.command, .option, .shift])
app.typeKey("n", modifierFlags: .command)
}
}
class AnotherFeatureUITests: UITestCase {
private func setupSingleWindow() { // ❌ Duplicate!
app.typeKey("w", modifierFlags: [.command, .option, .shift])
app.typeKey("n", modifierFlags: .command)
}
}
// ✅ GOOD: Use existing extension methods or add new ones to XCUIApplicationExtension.swift
class FeatureUITests: UITestCase {
override func setUpWithError() throws {
continueAfterFailure = false
app = XCUIApplication.setUp()
// Use existing extension method
app.enforceSingleWindow() // ✅ Already exists in XCUIApplicationExtension.swift
}
}
// ✅ GOOD: Add new helper methods to the extension for reuse
extension XCUIApplication {
/// Navigate to a URL and wait for page load
/// - Parameter url: The URL to navigate to
/// - Parameter timeout: Timeout for page load wait
/// - Parameter isNewTab: Whether this is happening on a new tab (affects address bar activation)
func navigateToURL(_ url: String, timeout: TimeInterval = 10.0, isNewTab: Bool = false) -> Bool {
guard addressBar.waitForExistence(timeout: 5.0) else { return false }
// Only activate address bar if not on a new tab (new tabs have address bar pre-activated)
if !isNewTab {
activateAddressBar()
}
addressBar.typeText(url)
addressBar.typeKey(.enter, modifierFlags: [])
// Wait for specific content rather than generic webView existence
let webView = webViews.firstMatch
let pageContent = webView.staticTexts.firstMatch
return pageContent.waitForExistence(timeout: timeout)
}
/// Open downloads popup and verify it appears
func openDownloadsPopup() -> Bool {
typeKey("j", modifierFlags: [.command])
let downloadsPopup = windows.containing(.any).firstMatch
return downloadsPopup.waitForExistence(timeout: 5.0)
}
}
```
**Available Extension Properties and Methods:**
- `app.addressBar` → Address bar text field element (replaces manual `app.textFields["AddressBarViewController.addressBarTextField"]`)
- `app.addressBarValueActivatingIfNeeded()` → Activate address bar and return its current value as String?
- `app.enforceSingleWindow()` → Close all windows and open new one (replaces `setupSingleWindow()`)
- `app.activateAddressBar()` → Activate address bar for input (replaces direct `Cmd+L`)
- `app.openNewTab()` → Open new tab via `Cmd+T`
- `app.resetBookmarks()` → Reset bookmarks for testing
- `app.openBookmarksManager()` → Open bookmarks manager
- `app.openBookmarksPanel()` → Show bookmarks panel
**Common patterns that should be moved to extensions:**
- `setupSingleWindow()` → Use existing `enforceSingleWindow()`
- Manual address bar references → Use `app.addressBar`
- Direct `Cmd+L` usage → Use `app.activateAddressBar()`
- URL navigation helpers → `navigateToURL(_:timeout:)`
- Downloads popup helpers → `openDownloadsPopup()`
- Common assertion patterns → Extension methods
**Benefits of using extension methods:**
-**No duplication** - Write once, use everywhere
-**Consistent behavior** - Same implementation across all tests
-**Easier maintenance** - Fix bugs in one place
-**Better discoverability** - Other developers can find and reuse helpers
-**Cleaner test files** - Focus on test logic, not boilerplate
#### Existence Checking Patterns
```swift
// ✅ GOOD - Proper existence checking
XCTAssertTrue(mainElement.waitForExistence(timeout: UITests.Timeouts.elementExistence))
XCTAssertTrue(relatedButton.exists) // Good after waitForExistence passed
XCTAssertTrue(anotherComponent.exists) // Good for checking multiple components
// ❌ BAD - Avoid these patterns
XCTAssertTrue(element.exists) // Without waitForExistence first
XCTAssertTrue(element1.waitForExistence(timeout: 5))
XCTAssertTrue(element2.waitForExistence(timeout: 5)) // Consecutive waits slow tests
Thread.sleep(forTimeInterval: 2.0) // Unreliable
Task.sleep(nanoseconds: 2_000_000_000) // Same issue
```
#### XCUIElement Property Waiting Extensions
**NEW: Type-safe property waiting methods for more reliable UI tests:**
```swift
// ✅ EXCELLENT: Wait for element properties using key paths
let addressBar = app.addressBar
let button = app.buttons["TestButton"]
// Wait for property to contain substring (case-insensitive)
XCTAssertTrue(addressBar.wait(for: \.value, contains: "example.com", timeout: 10.0))
XCTAssertTrue(button.wait(for: \.label, contains: "Submit"))
// Wait for property to equal specific value
XCTAssertTrue(button.wait(for: \.isEnabled, equals: true, timeout: 5.0))
XCTAssertTrue(addressBar.wait(for: \.value, equals: "https://duckduckgo.com"))
// Use in assertions with descriptive failure messages
XCTAssertTrue(statusField.wait(for: \.value, equals: "1 of 4"),
"Status field should show '1 of 4', but got: \(statusField.value ?? "nil")")
```
**Benefits of property waiting extensions:**
- **Type-safe**: Uses Swift key paths instead of string predicates
- **Flexible**: Works with any property (\.value, \.label, \.title, \.isEnabled, etc.)
- **Reliable**: Built on XCTNSPredicateExpectation for proper waiting
- **Debuggable**: Easy to add current value to failure messages
#### XCUIElementQuery Filtering Extensions
**Type-safe element filtering using key paths:**
```swift
// ✅ CORRECT: Filter elements using key paths
let webView = app.webViews.firstMatch
// Filter by substring (case-insensitive)
let pageContent = webView.staticTexts.containing(\.value, containing: "Example Domain").firstMatch
let submitButtons = app.buttons.containing(\.label, containing: "Submit")
// Filter by exact value
let enabledButtons = app.buttons.containing(\.isEnabled, equalTo: true)
let specificText = webView.staticTexts.containing(\.value, equalTo: "Welcome").firstMatch
let settingsWindow = app.windows.containing(\.title, equalTo: "Settings").firstMatch
// Element matching patterns
let stopMenuItem = app.menuItems.containing(\.title, equalTo: "Stop").firstMatch
let backgroundTab = app.radioButtons.containing(\.title, equalTo: "Background Download").firstMatch
// Replace old NSPredicate format strings
// ❌ OLD: webView.staticTexts.containing(NSPredicate(format: "value CONTAINS 'Example Domain'"))
// ✅ NEW: webView.staticTexts.containing(\.value, containing: "Example Domain")
```
**Available XCUIElementQuery Methods**:
- `containing(_:containing:)` - Filter elements where property contains substring
- `containing(_:equalTo:)` - Filter elements where property equals value
- `containing(_:where:)` - Filter elements containing specific element type with predicate
- `matching(_:containing:)` - Alternative filtering method for contains
- `matching(_:equalTo:)` - Alternative filtering method for equals
- `element(matching:containing:)` - Get single element matching contains criteria
- `element(matching:equalTo:)` - Get single element matching equals criteria
#### NSPredicate KeyPath Extensions
**Type-safe predicate construction for complex filtering:**
```swift
// ✅ CORRECT: Using .keyPath() method for predicate construction
let webView = app.webViews.firstMatch
// Complex element filtering with compound predicates
let pdfElement = app.groups.containing(.staticText, where: .keyPath(\.value, beginsWith: "TestPDF")).firstMatch
// Compound predicates combining multiple conditions
let summaryGroup = webView.groups.containing(.keyPath(\.value, beginsWith: "1p navigation -")).firstMatch
let headerGroup = summaryGroup.groups.containing(.staticText, where: .keyPath(\.value, beginsWith: "Blocked")).firstMatch
// Advanced predicate construction with chaining
let complexPredicate = NSPredicate.keyPath(\.elementType, equalTo: XCUIElement.ElementType.staticText.rawValue)
.and(.keyPath(\.value, beginsWith: "Expected"))
let pathCell = tables.cells.containing(NSPredicate { element, _ in
guard let id = (element as? NSObject)?.value(forKey: #keyPath(XCUIElement.identifier)) as? String,
id.hasPrefix("/"),
URL(fileURLWithPath: id).standardizedFileURL.path == standardizedPath else { return false }
return true
}).firstMatch
// Window filtering patterns
let namedWindow = app.windows.containing(NSPredicate(format: "title == %@", "Page Title")).firstMatch
// Replace manual NSPredicate format strings
// ❌ OLD: NSPredicate(format: "value CONTAINS %@ AND isEnabled == %@", "text", true)
// ✅ NEW: .keyPath(\.value, contains: "text").and(.keyPath(\.isEnabled, equalTo: true))
```
**Available NSPredicate Static Methods**:
**Equality and Membership**:
- `.keyPath(_:equalTo:)` - Property equals specific value
- `.keyPath(_:in: [values])` - Property in collection of values
- `.keyPath(_:in: range)` - Property in numeric range
**String Operations**:
- `.keyPath(_:contains:)` - Property contains substring (case-insensitive)
- `.keyPath(_:like:)` - Pattern matching with wildcards (* and ?)
- `.keyPath(_:beginsWith:)` - Property starts with prefix
- `.keyPath(_:endsWith:)` - Property ends with suffix
- `.keyPath(_:matchingRegex:)` - Property matches regular expression
**Numeric Range Operations**:
- `.keyPath(_:in: 1...10)` - Closed range (inclusive)
- `.keyPath(_:in: 1..<10)` - Half-open range
- `.keyPath(_:in: 5...)` - Greater than or equal (>= 5)
- `.keyPath(_:in: ..<10)` - Less than (< 10)
- `.keyPath(_:in: ...10)` - Less than or equal (<= 10)
**Compound Operations**:
- `predicate.and(otherPredicate)` - AND combination (instance method)
- `predicate.or(otherPredicate)` - OR combination (instance method)
- `.and(pred1, pred2, ...)` - AND multiple predicates (static)
- `.or(pred1, pred2, ...)` - OR multiple predicates (static)
- `predicate.inverted` - NOT predicate (property)
**Benefits of NSPredicate KeyPath Extensions**:
- **Type Safety**: Compile-time KeyPath validation
- **Automatic Format Specifiers**: Handles %@, %d, %f automatically based on type
- **Composable**: Easy compound predicate construction with and/or/not
- **Reusable**: Store predicates as variables for reuse across tests
#### XCUIElementQuery Waiting Extensions
**Wait for conditions on element queries (e.g., count changes):**
```swift
// ✅ CORRECT: Wait for element count conditions
let table = app.tables.firstMatch
let cells = table.cells
// Wait for exact count
XCTAssertTrue(cells.wait(for: \.count, equals: 5, timeout: UITests.Timeouts.localTestServer), "Should have exactly 5 cells")
// Wait for range conditions
XCTAssertTrue(cells.wait(for: \.count, in: 1...10, timeout: UITests.Timeouts.elementExistence), "Should have 1-10 cells")
XCTAssertTrue(cells.wait(for: \.count, in: 2..., timeout: UITests.Timeouts.elementExistence), "Should have at least 2 cells")
XCTAssertTrue(cells.wait(for: \.count, in: ..<10, timeout: UITests.Timeouts.elementExistence), "Should have less than 10 cells")
// Wait with custom predicate
let countPredicate = NSPredicate.keyPath(\.count, in: 1...5)
XCTAssertTrue(cells.wait(for: countPredicate, timeout: UITests.Timeouts.elementExistence), "Should have 1-5 cells")
// Replace old XCTNSPredicateExpectation patterns
// ❌ OLD: Manual XCTNSPredicateExpectation creation
// let expectation = XCTNSPredicateExpectation(predicate: NSPredicate(format: "count == %d", 2), object: table.cells)
// XCTAssertEqual(XCTWaiter.wait(for: [expectation], timeout: 15.0), .completed)
// ✅ NEW: Direct query waiting methods
// XCTAssertTrue(table.cells.wait(for: \.count, equals: 2, timeout: UITests.Timeouts.localTestServer))
```
**Available XCUIElementQuery Wait Methods**:
- `wait(for: NSPredicate, timeout:)` - Wait for custom NSPredicate condition
- `wait(for: \.count, equals: value, timeout:)` - Wait for count to equal specific value
- `wait(for: \.count, in: ClosedRange, timeout:)` - Wait for count in inclusive range (1...10)
- `wait(for: \.count, in: Range, timeout:)` - Wait for count in half-open range (1..<10)
- `wait(for: \.count, in: PartialRangeFrom, timeout:)` - Wait for count >= value (5...)
- `wait(for: \.count, in: PartialRangeUpTo, timeout:)` - Wait for count < value (..<10)
- `wait(for: \.count, in: PartialRangeThrough, timeout:)` - Wait for count <= value (...10)
**Benefits of query waiting extensions:**
- **Type-safe**: Uses Swift key paths with compile-time validation
- **Range support**: Native Swift range syntax for numeric conditions
- **Simplified**: Replaces verbose XCTNSPredicateExpectation patterns
- **Consistent**: Same predicate-based API as filtering methods
- **Maintainable**: Compiler catches property and range type errors
#### Middle Click Special Handling
```swift
// ❌ NEVER: Use synthesized CGEvent approach
let mouseDownEvent = CGEvent(mouseEventSource: nil,
mouseType: .otherMouseDown,
mouseCursorPosition: point,
mouseButton: .center)!
let mouseUpEvent = CGEvent(mouseEventSource: nil,
mouseType: .otherMouseUp,
mouseCursorPosition: point,
mouseButton: .center)!
mouseDownEvent.post(tap: .cghidEventTap)
mouseUpEvent.post(tap: .cghidEventTap)
// ✅ CORRECT: Use extension method directly
element.middleClick() // Extension handles middle-click properly
// ✅ CORRECT: For middle-click with modifiers, use separate perform block
XCUIElement.perform(withKeyModifiers: [.option]) {
element.middleClick() // This doesn't work correctly
}
```
### Test Assertions Must Be Precise
**CRITICAL RULE**: All test checks must be predictable and precise. Avoid OR-conditions, Thread.sleep(), and vague checks.
```swift
// ❌ WRONG - Vague OR-conditions with sleep
Thread.sleep(forTimeInterval: 1.0)
let someUIVisible = element1.exists || element2.exists || element3.exists
XCTAssertTrue(someUIVisible, "Some UI should be accessible")
// ❌ WRONG - Using XCTNSPredicateExpectation for simple element waiting
let webView = app.webViews.firstMatch
let pageLoaded = XCTNSPredicateExpectation(
predicate: NSPredicate(format: "exists == true"),
object: webView
)
XCTAssertEqual(XCTWaiter.wait(for: [pageLoaded], timeout: 15.0), .completed)
// ❌ WRONG - Using 'if' statements for button waiting
if runButton.waitForExistence(timeout: 5.0) {
runButton.click()
}
// ✅ CORRECT - Use waitForExistence with assertion for simple element waiting
XCTAssertTrue(runButton.waitForExistence(timeout: 15.0), "Run button should be available")
runButton.click()
// ✅ BEST PRACTICE - Wait for the actual element you need, not its container
// Don't wait for webView if you need a button inside it - button existence implies page loaded
let runButton = app.webViews.buttons["run"]
XCTAssertTrue(runButton.waitForExistence(timeout: 15.0), "Run button should be available")
runButton.click()
// ✅ CORRECT - Use XCTNSPredicateExpectation only for complex conditions
let complexCondition = XCTNSPredicateExpectation(
predicate: NSPredicate(format: "count > 2"),
object: app.tables.cells
)
XCTAssertEqual(XCTWaiter.wait(for: [complexCondition], timeout: 5.0), .completed)
```
**Prohibited Patterns:**
- `Thread.sleep()` - Use `waitForExistence` or `XCTNSPredicateExpectation` instead
- OR-conditions (`||`) in assertions - Test one specific state
- Vague "should be accessible" - Test specific elements and values
- Fallback checks - If primary check fails, test should fail clearly
- `if button.waitForExistence()` - Use `XCTAssertTrue(button.waitForExistence())` instead
- Complex `XCTNSPredicateExpectation` for simple existence checks - Use `waitForExistence` directly
#### Address Bar Usage Pattern
**Use the extension property and follow activation rules:**
```swift
func testAddressBarInteraction() {
// ✅ CORRECT: Use extension property
let addressBar = app.addressBar
// ✅ CORRECT: On new tab page, address bar is already activated - no Cmd+L needed
addressBar.pasteURL("example.com")
// Wait for navigation
let pageContent = webView.staticTexts.containing(NSPredicate(format: "value CONTAINS 'Example Domain'")).firstMatch
XCTAssertTrue(pageContent.waitForExistence(timeout: 30.0), "Example.com should load")
// ✅ REQUIRED: After navigation, activate address bar before further interaction
app.activateAddressBar() // Use extension method instead of direct Cmd+L
// Now the address bar is ready for input
addressBar.typeText("another-site.com")
addressBar.typeKey(.enter, modifierFlags: [])
}
// ❌ INCORRECT: Don't create your own addressBar reference
func testBadPattern() {
let addressBarTextField = app.textFields["AddressBarViewController.addressBarTextField"] // ❌ Use app.addressBar instead
// ❌ INCORRECT: Don't use Cmd+L on new tab pages
app.typeKey("l", modifierFlags: [.command]) // Address bar is already activated on new tabs
addressBarTextField.typeText("example.com")
}
```
**Address Bar Activation Rules**:
-**NEW TAB PAGE**: Address bar is already activated - do NOT use `Cmd+L`
-**AFTER NAVIGATION**: Address bar becomes read-only - USE `app.activateAddressBar()` before typing
-**USE EXTENSION**: Always use `app.addressBar` instead of creating your own reference
-**USE HELPER METHOD**: Use `app.activateAddressBar()` instead of direct `Cmd+L`
#### Element Query Guidelines
```swift
// ✅ CORRECT: Use element references efficiently
class MyUITests: UITestCase {
private var addressBar: XCUIElement!
private var webView: XCUIElement!
override func setUpWithError() throws {
continueAfterFailure = false
app = XCUIApplication.setUp()
// Create element references once
addressBar = app.textFields["AddressBarViewController.addressBarTextField"]
webView = app.webViews.firstMatch
}
func testNavigation() {
// Use the same references throughout the test
XCTAssertTrue(addressBar.waitForExistence(timeout: 5.0))
addressBar.typeText("example.com")
addressBar.typeKey(.enter, modifierFlags: [])
// Wait for specific page content, not generic webView existence
let pageContent = webView.staticTexts.containing(NSPredicate(format: "value CONTAINS 'Example Domain'")).firstMatch
XCTAssertTrue(pageContent.waitForExistence(timeout: 10.0), "Should show example.com page content")
// After navigation, activate address bar for new input
app.activateAddressBar()
// Same addressBar reference is still valid
addressBar.typeText("duckduckgo.com")
}
}
// ❌ INCORRECT: Don't create multiple references to same element
func testBadElementHandling() {
let addressBar1 = app.textFields["AddressBarViewController.addressBarTextField"]
addressBar1.typeText("example.com")
// ❌ Unnecessary - addressBar1 is still valid
let addressBar2 = app.textFields["AddressBarViewController.addressBarTextField"]
let freshAddressBar = app.textFields["AddressBarViewController.addressBarTextField"]
// All three references point to the same element!
}
```
### Context Menu Interaction
```swift
// Use coordinate-based context menu clicking for reliability across macOS versions
func testContextMenuAction() {
let webView = app.webViews.firstMatch
webView.rightClick()
// ✅ Use the coordinate-based context menu hack
try app.clickContextMenuItem(matching: { $0.identifier == "PDFContextMenu.print" })
// This method uses coordinate-based clicking instead of direct element interaction
// because context menu detection fails on older macOS systems (13/14) in CI
}
```
## Local Test Server Integration
UI tests use a local test server running on `http://localhost:8085/` for reliable page loading and content simulation.
### Test Server Architecture
- **Port**: 8085 (not 8080)
- **Integration**: Uses `TestsURLExtension.swift` shared with Integration Tests
- **APIs**: Provides `URL.testsServer` and `.appendingTestParameters()` methods
- **Content**: Supports dynamic content generation via query parameters
### Creating Test Content
#### Static Content
```swift
// Static content from test files
let url = URL.testsServer.appendingPathComponent("test-page.html")
```
#### Dynamic Content
```swift
// Dynamic content with custom HTML
let url = UITests.simpleServedPage(titled: "Test Page")
// Creates: http://localhost:8085/?data=<html>...<title>Test Page</title>...</html>
// With custom body content
let url = UITests.simpleServedPage(titled: "Test Page") {
"<a href='http://example.com'>Test Link</a>"
}
```
#### Custom HTTP Responses
```swift
let url = URL.testsServer
.appendingPathComponent("test-endpoint")
.appendingTestParameters(
status: 404,
reason: "Not Found",
headers: ["Content-Type": "application/json"]
)
```
### URL Parameter Options
- `status`: HTTP status code (default: 200)
- `reason`: HTTP status string (default: "OK")
- `data`: Response body (Data or String, base64 encoded if binary)
- `headers`: HTTP response headers
### Test Server Best Practices
- Use `UITests.simpleServedPage(titled:)` for basic HTML pages
- Use `URL.testsServer.appendingTestParameters()` for custom responses
- Test various HTTP status codes and response types
- Keep test content simple and predictable
- Always validate server responses in tests
## Performance Optimizations
### Using Pasteboard for Speed
```swift
func testAddressBarInput() {
// Instead of typing character by character
let testURL = "https://example.com"
UIPasteboard.general.string = testURL
addressBar.press(forDuration: 1.0)
app.menuItems["Paste"].tap()
// Much faster than: addressBar.typeText(testURL)
}
```
### Avoiding Slow Operations
```swift
// ✅ FAST: Use paste for long URLs: it manages pasting into the pasteboard and restores its contents afterwards
addressBar.pasteURL(longURL)
// ❌ SLOW: Character-by-character typing
addressBar.typeText(longURL)
// ✅ FAST: Single waitForExistence
XCTAssertTrue(mainElement.waitForExistence(timeout: 5))
XCTAssertTrue(relatedElement.exists)
// ❌ SLOW: Multiple consecutive waits
XCTAssertTrue(element1.waitForExistence(timeout: 5))
XCTAssertTrue(element2.waitForExistence(timeout: 5))
```
## UI Test Build Architecture
UI tests use a unique build architecture that affects compatibility:
### Build Process
- **App Binary**: Built using notarized build action (latest Xcode/toolchain)
- **UI Test Bundle**: Built on target macOS version (macOS 13/14/15 runners)
- **Testing**: UI Test bundle tests the notarized app binary across macOS 13/14/15
### Code Compatibility Requirements
```swift
// ✅ GOOD - UI Test code must compile on older Xcode versions
func testFeature() {
let app = XCUIApplication.setUp()
app.addressBar.typeText("test")
// Uses APIs available in minimum supported Xcode
}
// ❌ BAD - Don't use newest APIs that aren't available on older Xcode
@available(macOS 14.0, *)
func testNewAPI() {
// This won't compile on macOS 13 UI test runners
}
```
### Compatibility Guidelines
- UI Test code must compile on oldest supported Xcode version (for macOS 13 runner)
- App code can use latest Swift/Xcode features (built with latest toolchain)
- Test only stable APIs - avoid beta/preview APIs in UI test code
- Use `@available` checks carefully - must work across all test runners
## Advanced Testing Patterns
### Tab Extension Testing in UI Tests
For testing Tab Extension behavior through the UI:
```swift
func testTabExtensionUIBehavior() {
// Load page that triggers tab extension
openTestPage("Extension Test Page")
// Test extension UI elements appear
XCTAssertTrue(app.buttons["TabExtension.actionButton"].waitForExistence(timeout: 5))
// Test extension interactions
app.buttons["TabExtension.actionButton"].click()
XCTAssertTrue(app.alerts["TabExtension.confirmationAlert"].exists)
}
```
### Settings and Preferences Testing
```swift
func testPreferencesImpactOnBehavior() {
// Navigate to preferences
navigateToGeneralPreferences()
// Change setting
let toggle = app.checkBoxes["PreferencesGeneralView.switchToNewTabImmediately"]
XCTAssertTrue(toggle.waitForExistence(timeout: UITests.Timeouts.elementExistence))
if (toggle.value as? Bool) != true {
toggle.click()
}
// Test behavior change
openTestPage("Test Page")
XCUIElement.perform(withKeyModifiers: [.command]) {
link.click()
}
// Validate inverted behavior due to setting
XCTAssertTrue(app.webViews["New Page"].waitForExistence(timeout: UITests.Timeouts.elementExistence))
// Reset setting
if (toggle.value as? Bool) == true {
toggle.click()
}
}
```
### Bookmark and History Testing
#### Bookmark Testing Best Practices
**Always reset bookmarks before testing** to ensure a clean state:
```swift
func testBookmarkNavigation() {
// ✅ CORRECT: Reset bookmarks for clean test state
app.resetBookmarks()
// Add bookmark
openTestPage("Test Page")
app.mainMenuAddBookmarkMenuItem.click()
app.addBookmarkAlertAddButton.click()
// Test bookmark interactions
app.bookmarksMenu.click()
// ✅ CORRECT: Use bookmarksMenu.menuItems for bookmark items
let bookmarkItem = app.bookmarksMenu.menuItems["Test Page"]
XCTAssertTrue(bookmarkItem.waitForExistence(timeout: UITests.Timeouts.elementExistence))
// Test modifier behaviors
XCUIElement.perform(withKeyModifiers: [.command]) {
bookmarkItem.click()
}
XCTAssertTrue(app.webViews["Test Page"].waitForExistence(timeout: UITests.Timeouts.elementExistence))
XCTAssertEqual(app.windows.count, 1)
XCTAssertEqual(app.tabs.count, 2)
}
```
#### Bookmark Testing Setup Patterns
**For single bookmark tests**: Call `app.resetBookmarks()` at the beginning of each test:
```swift
func testSingleBookmarkBehavior() {
app.resetBookmarks() // Clean state for this test
// Test bookmark functionality...
}
```
**For test suites focused on bookmark management**: Reset in `setUp` if all tests involve bookmarks:
```swift
class BookmarkManagementUITests: UITestCase {
override func setUpWithError() throws {
super.setUpWithError()
app.resetBookmarks() // Clean state for all bookmark tests
}
func testAddBookmark() {
// No need to reset here - already done in setUp
}
func testDeleteBookmark() {
// No need to reset here - already done in setUp
}
}
```
#### Bookmark Menu Item Access
**Always use `app.bookmarksMenu.menuItems` for bookmark menu items**:
```swift
// ✅ CORRECT: Specific bookmark menu item access
app.bookmarksMenu.click()
let bookmarkItem = app.bookmarksMenu.menuItems["My Bookmark"]
// ❌ INCORRECT: Generic menu item access (may conflict with other menus)
let bookmarkItem = app.menuItems["My Bookmark"]
```
## Screenshot and Debugging
### Automatic Screenshots
Screenshots are taken automatically on test failures for debugging.
### Manual Screenshots
```swift
func takeScreenshot(name: String) {
let screenshot = XCUIScreen.main.screenshot()
let attachment = XCTAttachment(screenshot: screenshot)
attachment.name = name
attachment.lifetime = .keepAlways
add(attachment)
}
```
### UI Tests Logging
**For UI tests, NEVER use `print()`. ALWAYS use `Logger.log()` for debug output:**
```swift
// GOOD: Use Logger.log() for UI test debugging
class FeatureUITests: UITestCase {
func testComplexFlow() {
Logger.log("Starting complex UI flow test")
Logger.log("Setting up test data with \(testData.count) items")
Logger.log("DEBUG: currentState = \(app.addressBarValueActivatingIfNeeded())")
// Perform UI test operations
Logger.log("UI test completed successfully")
}
}
// BAD: Using print() statements in UI tests
func testComplexFlow() {
print("Starting test") // Never use print()
print("DEBUG: addressBar = \(addressBar.value)") // Use Logger.log() instead
}
```
**Benefits of Logger.log():**
- Integrates with XCTest's internal logging system
- Appears in test logs alongside other XCTest debug output
- Better performance and integration than print() statements
- Proper formatting for CI log collection
- Uses XCTest's private debug logging infrastructure for better integration
**Usage Examples:**
```swift
class FeatureUITests: UITestCase {
func testComplexInteraction() {
Logger.log("Starting test with \(elements.count) elements")
let currentURL = app.addressBarValueActivatingIfNeeded()
Logger.log("Current URL: \(currentURL ?? "nil")")
if !button.waitForExistence(timeout: 5.0) {
Logger.log("Button not found, taking screenshot for debugging")
takeScreenshot("button-not-found")
}
}
}
```
### 🔍 Debug Operator: `???` for Optional String Conversion
The `???` operator provides safe string conversion for debugging:
```swift
// ✅ CORRECT: Debug string conversion with ??? operator (UI tests)
Logger.log("event received: \(event ??? "<nil>")")
XCTAssertTrue(element.exists, "Element should exist: \(optionalValue ??? "missing")")
// ✅ CORRECT: Debug string conversion with ??? operator (unit tests)
Logger.log("event received: \(event ??? "<nil>")")
```
**What it does:**
- **Converts any optional to String** for debugging/logging
- **Uses `String(describing:)` if value exists**
- **Falls back to provided default string if nil**
- **Safer than force unwrapping** for debug output
### 🔍 Debugging Pattern: UI Snapshot Logging
When UI tests fail and you need to see the actual element hierarchy:
```swift
// ✅ CORRECT: UI snapshot debugging for failed assertions
XCTAssertTrue(element.waitForExistence(timeout: 5.0),
"Element should exist: \((try? parentElement.snapshot().toDictionary()) ??? "snapshot failed")")
```
**When to use:**
- **UI tests fail unexpectedly** and you need to see what's actually there
- **Element queries don't find expected elements**
- **Debugging privacy dashboard content**, context menus, or complex UI
- **Only during debugging** - remove before committing
**Key Points:**
- Use `(try? element.snapshot().toDictionary())` to safely get UI hierarchy
- Use `??? "fallback"` to handle snapshot failures
- Provides complete element tree structure when tests fail
- Remove debugging code before final commit
### UI Test Debugging with View Hierarchy Snapshots
When debugging UI test failures, use the `toDictionary()` helper method to capture and inspect the complete view hierarchy:
```swift
// ✅ CORRECT: Debug view hierarchy when UI elements aren't found as expected
func testComplexUIInteraction() {
let webView = app.webViews.firstMatch
let runButton = webView.buttons["Start"]
if !runButton.waitForExistence(timeout: 5.0) {
// Capture view hierarchy for debugging
let snapshot = try! webView.snapshot().toDictionary()
Logger.log("WebView hierarchy:\n\(snapshot)")
XCTFail("Start button not found in webView")
}
runButton.click()
}
// ✅ CORRECT: Include hierarchy in assertion failure messages
func testAddressBarBehavior() {
app.activateAddressBar()
let addressBarValue = addressBarTextField.value as? String ?? ""
if addressBarValue.isEmpty {
// Include snapshot in failure message for debugging
let snapshot = try! app.snapshot().toDictionary()
XCTAssertFalse(addressBarValue.isEmpty,
"Address bar should have content, got: \(addressBarValue)\n\(snapshot)")
}
}
```
**Available snapshot properties** (customizable via `keys` parameter):
- `elementType`: UI element type (button, textField, etc.)
- `identifier`: Accessibility identifier
- `label`: Accessibility label
- `title`: Element title
- `value`: Current value
- `isEnabled`: Whether element is enabled
- `frame`: Element position and size
- `children`: Nested elements (automatically included)
```swift
// ✅ CORRECT: Custom properties for specific debugging needs
let snapshot = try! element.snapshot().toDictionary(keys: [
"elementType", "identifier", "label", "isEnabled"
])
// ✅ CORRECT: Full default properties for comprehensive debugging
let snapshot = try! element.snapshot().toDictionary()
```
**When to use view hierarchy debugging**:
- Element not found when expected to exist
- UI interaction failing unexpectedly
- Need to understand complex nested view structures
- Debugging test flakiness related to UI timing
- Adding detailed context to assertion failure messages
**Best practices**:
- Use sparingly in production tests (only for debugging)
- Include snapshots in assertion failure messages for context
- Remove debug snapshots once issues are resolved
- Use custom `keys` parameter to focus on relevant properties
### Debug Information
```swift
func debugElementHierarchy() {
// Print element hierarchy for debugging
Logger.log("Current window hierarchy: \(app.windows.debugDescription)")
Logger.log("Current tab structure: \(app.tabs.debugDescription)")
}
```
## Code Reading Requirements
### Always Read Actual Code
**Never assume how UI implementation is done**. Always read the actual browser code related to the tested area:
1. **Find the relevant view controller** or UI component
2. **Check for existing accessibility identifiers**
3. **Understand the UI hierarchy and structure**
4. **Identify interaction patterns and state management**
5. **Verify element lifecycle and timing**
```swift
// Example: Before testing address bar, read:
// - AddressBarViewController.swift
// - Check how textField.accessibilityIdentifier is set
// - Understand the view hierarchy
// - Look for existing test accessors in extensions
```
### Extension Integration
Before creating element accessors, check existing extensions:
```swift
// Check XCUIApplicationExtension.swift for patterns like:
extension XCUIApplication {
var addressBar: XCUIElement {
windows.textFields["AddressBarViewController.addressBarTextField"]
}
}
// Check XCUIElementExtension.swift for helper methods like:
extension XCUIElement {
func clickAfterExistenceTestSucceeds() {
// Implementation
}
}
```
## Test Execution Guidelines
### ❗Never Run Tests Yourself
**DO NOT attempt to run UI tests unless explicitly asked by the user.** UI tests:
- Take significant time to execute
- Require specific setup and environment
- Can interfere with system state
- Should only be run when specifically requested for debugging
### Local Testing Recommendations
When users want to run tests locally:
```bash
# Use xcodebuild for consistency with CI
xcodebuild -project macOS/DuckDuckGo.xcodeproj \
-scheme 'DuckDuckGo (macOS)' \
-configuration Debug \
-destination 'platform=macOS' \
test \
-only-testing:DuckDuckGo_Privacy_BrowserTests/TabNavigationTests
# For specific test methods
xcodebuild -project macOS/DuckDuckGo.xcodeproj \
-scheme 'DuckDuckGo (macOS)' \
-configuration Debug \
-destination 'platform=macOS' \
test \
-only-testing:DuckDuckGo_Privacy_BrowserTests/TabNavigationTests/testCommandClickOpensBackgroundTab
```
## 🚫 CRITICAL: UI Testing Anti-Patterns - NEVER USE THESE
#### ❌ Anti-Pattern #1: Thread.sleep and Arbitrary Delays
**NEVER USE:**
```swift
// ❌ FORBIDDEN: Thread.sleep()
Thread.sleep(forTimeInterval: 2.0)
// ❌ FORBIDDEN: DispatchQueue.main.asyncAfter
DispatchQueue.main.asyncAfter(deadline: .now() + 2.0) {
// test logic
}
// ❌ FORBIDDEN: Any fixed time delays
usleep(2000000)
```
**✅ USE INSTEAD:**
```swift
// ✅ CORRECT: Use waitForExistence with appropriate timeout
XCTAssertTrue(element.waitForExistence(timeout: 10.0), "Element should appear")
// ✅ CORRECT: Use waitForNonExistence for disappearing elements
XCTAssertTrue(element.waitForNonExistence(timeout: 5.0), "Element should disappear")
// ✅ CORRECT: Use XCTNSPredicateExpectation for complex conditions
let webView = app.webViews.firstMatch
let pageLoaded = XCTNSPredicateExpectation(
predicate: NSPredicate(format: "exists == true"),
object: webView
)
let result = XCTWaiter.wait(for: [pageLoaded], timeout: 30.0)
```
#### ❌ Anti-Pattern #2: Validation Branching (if/else Logic)
**NEVER USE:**
```swift
// ❌ FORBIDDEN: if/else validation branching
if button.waitForExistence(timeout: 5.0) {
button.click()
// test logic
} else {
XCTAssertTrue(true, "Button not available")
}
// ❌ FORBIDDEN: Combined conditional checks
if button.waitForExistence(timeout: 5.0) && button.isEnabled {
// test logic
} else {
XCTAssertTrue(true, "fallback message")
}
```
**✅ USE INSTEAD:**
```swift
// ✅ CORRECT: Direct assertions that fail clearly
XCTAssertTrue(button.waitForExistence(timeout: 5.0), "Button should be available")
XCTAssertTrue(button.isEnabled, "Button should be enabled")
button.click()
// ✅ CORRECT: Use XCTFail for impossible conditions
XCTAssertTrue(element.waitForExistence(timeout: 10.0), "Element should exist")
// If element doesn't exist, test fails clearly - no fallback needed
```
#### ❌ Anti-Pattern #3: Cop-Out Assertions
**NEVER USE:**
```swift
// ❌ FORBIDDEN: XCTAssertTrue(true) cop-outs
XCTAssertTrue(true, "Test completed - implementation may vary")
// ❌ FORBIDDEN: print() instead of assertions
if condition {
// test logic
} else {
print("Feature not available in test environment") // ❌ NO! Use Logger.log() instead
}
```
**✅ USE INSTEAD:**
```swift
// ✅ CORRECT: Meaningful assertions that can fail
XCTAssertEqual(actualCount, expectedCount, "Should have exact number of elements")
// ✅ CORRECT: Use XCTFail for impossible conditions
if !element.waitForExistence(timeout: 10.0) {
XCTFail("Critical element should always be available")
}
```
#### ❌ Anti-Pattern #4: Generic WebView Existence Checks
**NEVER USE:**
```swift
// ❌ FORBIDDEN: Pointless webView.waitForExistence()
XCTAssertTrue(webView.waitForExistence(timeout: 30.0), "Page should load")
```
**✅ USE INSTEAD:**
```swift
// ✅ CORRECT: Wait for specific content
let expectedContent = webView.staticTexts.containing(NSPredicate(format: "value CONTAINS 'expected text'")).firstMatch
XCTAssertTrue(expectedContent.waitForExistence(timeout: 30.0), "Page should show expected content")
// ✅ CORRECT: Validate specific page elements
let pageTitle = webView.staticTexts.containing(NSPredicate(format: "value CONTAINS 'Welcome'")).firstMatch
XCTAssertTrue(pageTitle.waitForExistence(timeout: 15.0), "Welcome page should load")
```
#### ❌ Anti-Pattern #5: typeURL() Usage
**NEVER USE:**
```swift
// ❌ FORBIDDEN: typeURL() is unreliable
addressBarTextField.typeURL(url)
```
**✅ USE INSTEAD:**
```swift
// ✅ CORRECT: Use pasteURL() with pressingEnter
addressBarTextField.pasteURL(url, pressingEnter: true)
```
#### ❌ Anti-Pattern #6: Manual Cmd+L for Address Bar Activation
**NEVER USE:**
```swift
// ❌ FORBIDDEN: Manual keyboard shortcuts
app.typeKey("l", modifierFlags: [.command])
```
**✅ USE INSTEAD:**
```swift
// ✅ CORRECT: Use dedicated helper method
app.activateAddressBar()
```
#### ❌ Anti-Pattern #7: Incorrect NSPredicate Usage
**NEVER USE:**
```swift
// ❌ FORBIDDEN: label CONTAINS in predicates
NSPredicate(format: "label CONTAINS 'search-text'")
```
**✅ USE INSTEAD:**
```swift
// ✅ CORRECT: value CONTAINS for UI elements
NSPredicate(format: "value CONTAINS 'search-text'")
```
#### ❌ Anti-Pattern #8: Windows for Tab Counting
**NEVER USE:**
```swift
// ❌ FORBIDDEN: Using windows.count for tabs in tabbed browser
let tabCount = app.windows.count
```
**✅ USE INSTEAD:**
```swift
// ✅ CORRECT: Use tabGroups for counting tabs
let tabCount = app.tabGroups.count
```
#### ❌ Anti-Pattern #9: Incorrect Modifier Clicks
**NEVER USE:**
```swift
// ❌ FORBIDDEN: These don't work or are unreliable
element.click(forDuration: 0.1, thenDragTo: element)
element.tap()
element.rightClick() // for modifier clicks
```
**✅ USE INSTEAD:**
```swift
// ✅ CORRECT: Use perform(withKeyModifiers:)
element.perform(withKeyModifiers: [.option]) {
element.click()
}
```
### Privacy Button Access
```swift
// ✅ CORRECT: Use the proper accessibility identifier
let privacyButton = app.buttons.matching(identifier: "AddressBarButtonsViewController.privacyDashboardButton").firstMatch
```
### ⚠️ ENFORCEMENT: These Rules Are MANDATORY
- **Every UI test MUST follow these patterns**
- **No exceptions for "quick fixes" or "temporary solutions"**
- **All existing tests MUST be refactored to follow these patterns**
- **Code reviews MUST check for these anti-patterns**
### Common Anti-Patterns to Avoid
#### ❌ Don't Use Generic Element Access
```swift
// ❌ BAD: Generic, fragile selectors
app.buttons.firstMatch
app.textFields["Search"]
app.menuItems.element(boundBy: 2)
```
#### ❌ Don't Create Redundant Element Variables
```swift
// ❌ BAD: Test-specific variables for common elements
func testSomething() {
let addressBar = app.textFields["AddressBarViewController.addressBarTextField"]
let backButton = app.buttons["NavigationBarViewController.backButton"]
// These should be in extensions
}
```
#### ❌ Don't Check State Before Waiting
```swift
// ❌ BAD: Race conditions
action()
XCTAssertEqual(app.tabs.count, 2) // Too early!
XCTAssertTrue(newElement.waitForExistence(...))
```
#### ❌ Don't Use Text-Based Selectors for Stable Elements
```swift
// ❌ BAD: Fragile to localization/text changes
app.buttons["Download"]
app.menuItems["Open in New Tab"]
// ✅ GOOD: Use accessibility IDs
app.buttons["DownloadsViewController.downloadButton"]
app.menuItems["ContextMenu.openInNewTab"]
```
#### ❌ Don't Test Bookmarks Without Clean State
```swift
// ❌ BAD: Testing without resetting bookmarks (may conflict with existing bookmarks)
func testBookmarkBehavior() {
openTestPage("Test Page")
app.mainMenuAddBookmarkMenuItem.click()
// Test may fail if bookmark already exists
}
// ❌ BAD: Using generic menu item access for bookmarks
app.bookmarksMenu.click()
let bookmarkItem = app.menuItems["My Bookmark"] // May conflict with other menus
// ✅ GOOD: Clean state and specific bookmark menu access
func testBookmarkBehavior() {
app.resetBookmarks() // Ensure clean state
openTestPage("Test Page")
app.mainMenuAddBookmarkMenuItem.click()
app.bookmarksMenu.click()
let bookmarkItem = app.bookmarksMenu.menuItems["Test Page"]
}
```
## Running UI Tests
```bash
# Run iOS UI tests
xcodebuild test \
-scheme "iOS Browser" \
-workspace DuckDuckGo.xcworkspace \
-destination "platform=iOS Simulator,name=iPhone 15 Pro" \
-only-testing:UITests
# Run macOS UI tests
xcodebuild test \
-scheme "macOS UI Tests" \
-configuration "Review" \
-skipPackagePluginValidation \
-skipMacroValidation \
-allowProvisioningUpdates=NO \
-only-testing:UI\ Tests
```
# Run specific macOS UI test case
xcodebuild test \
-scheme "macOS UI Tests" \
-configuration "Review" \
-skipPackagePluginValidation \
-skipMacroValidation \
-allowProvisioningUpdates=NO \
-only-testing:UI\ Tests/DownloadsUITests
```
## Best Practices Summary
1. **Always read actual browser code** before writing tests
2. **Use accessibility IDs** for element access
3. **Wait for elements before checking counts** or states
4. **Validate both UI state and browser state** (webViews + tabs)
5. **Use existing extensions** and helper methods
6. **Test modifier key behaviors** comprehensively
7. **Handle multi-window scenarios** with proper window indexing
8. **Reset bookmarks with `app.resetBookmarks()`** before bookmark tests or in setUp for bookmark-focused test suites
9. **Use `app.bookmarksMenu.menuItems`** for accessing bookmark menu items
10. **Never run tests unless explicitly requested**
11. **Ask permission before modifying main app code** for accessibility IDs
12. **Use middle-click extension methods** properly
---
**For questions about UI testing patterns or to request test execution, please reach out to the iOS/macOS team.**