127 lines
4.3 KiB
Markdown
127 lines
4.3 KiB
Markdown
---
|
|
source: ~/Developer/virfield/
|
|
confidence: 1.0
|
|
namespace: work
|
|
last_updated: 2026-04-28
|
|
tags: [ui-testing, vm, virfield, ddg-vm, xcode]
|
|
---
|
|
|
|
# VM UI Testing — virfield / ddg-vm MCP
|
|
|
|
**What it is**: macOS VM lifecycle console + MCP server for AI-driven UI test automation.
|
|
**How it runs**: Golden image (built once) → APFS CoW clone per test session → run → delete.
|
|
**MCP config**: `~/Developer/virfield/server/mcp-server.ts` (configured in `.claude/.mcp.json` as `ddg-vm`)
|
|
**Web UI**: `http://localhost:5173` (requires `npm run dev` in `~/Developer/virfield/`)
|
|
|
|
---
|
|
|
|
## Standard Agent Flow for UI Tests
|
|
|
|
### Step 1 — Start a session (single call)
|
|
|
|
```
|
|
vm_prepare_session()
|
|
```
|
|
|
|
This does everything in sequence: clone golden → start VM → wait for SSH → wait for Peekaboo ready.
|
|
Returns `{ vmName, ip }`. Use `vmName` in all subsequent calls.
|
|
|
|
### Step 2 — Run XCUITests
|
|
|
|
```
|
|
run_tests(
|
|
vm_id: vmName,
|
|
scheme: "DuckDuckGo macOS",
|
|
workspace: "<VMShare path to DuckDuckGo-macOS.xcworkspace>"
|
|
)
|
|
```
|
|
|
|
Returns `run_id` immediately. Poll with:
|
|
|
|
```
|
|
get_test_run_status(vm_id: vmName, run_id: run_id)
|
|
get_test_results(vm_id: vmName, run_id: run_id)
|
|
```
|
|
|
|
Uses pre-built `xctestrun` from `VMShare/DerivedData` — build must happen on host first.
|
|
|
|
### Step 3 — Inspect failures
|
|
|
|
```
|
|
peekaboo_image(vm_id: vmName) # screenshot
|
|
peekaboo_see(vm_id: vmName, app: "...") # AX tree
|
|
ax_snapshot(vm_id: vmName, app: "...", name: "before")
|
|
ax_diff_last(vm_id: vmName, app: "...") # what changed
|
|
get_crash_reports(vm_id: vmName) # crash logs
|
|
get_log_stream(vm_id: vmName, path: "...", lines: 50)
|
|
```
|
|
|
|
### Step 4 — Cleanup (ALWAYS — especially from Executor)
|
|
|
|
```
|
|
vm_stop(vm_id: vmName)
|
|
vm_delete(vm_id: vmName)
|
|
```
|
|
|
|
**Executor must call these before removing its worktree.** Never leave orphaned VMs.
|
|
|
|
---
|
|
|
|
## All MCP Tools
|
|
|
|
| Tool | Purpose |
|
|
|------|---------|
|
|
| `vm_list` | List all VMs with state, IP, checklist status |
|
|
| `vm_prepare_session` | **Main entry point** — clone + start + wait for ready |
|
|
| `vm_start` | Start a VM (optionally clone from golden first) |
|
|
| `vm_stop` | Stop a running VM |
|
|
| `vm_clone_golden` | APFS CoW clone from golden image |
|
|
| `vm_delete` | Delete a VM (run VMs only, never golden) |
|
|
| `vm_get_ip` | Get IP of running VM |
|
|
| `vm_status` | Full status including Peekaboo connection state |
|
|
| `vm_ssh_exec` | Run a shell command in VM via SSH |
|
|
| `run_tests` | Launch XCUITests in VM (returns run_id immediately) |
|
|
| `get_test_run_status` | Poll whether xcodebuild is still running |
|
|
| `get_test_results` | Parse test result artifacts for a run_id |
|
|
| `get_crash_reports` | Read crash reports from VM |
|
|
| `get_log_stream` | Tail a log file from VM |
|
|
| `peekaboo_see` | Dump AX accessibility tree for an app |
|
|
| `peekaboo_image` | Screenshot from VM |
|
|
| `peekaboo_click` | Click by query / element ID / coords |
|
|
| `peekaboo_type` | Type text in VM |
|
|
| `peekaboo_hotkey` | Send hotkey (e.g. cmd+s) |
|
|
| `peekaboo_scroll` | Scroll in VM |
|
|
| `peekaboo_list_apps` | List running apps in VM |
|
|
| `peekaboo_permissions` | Check Screen Recording / Accessibility permissions |
|
|
| `ax_snapshot` | Take named AX snapshot for later diff |
|
|
| `ax_diff_last` | Diff current AX state vs last snapshot |
|
|
| `ax_diff` | Diff two named snapshots by ID |
|
|
| `ax_query` | Query elements from latest snapshot |
|
|
| `vm_build_golden` | Build golden VM from IPSW (full 4-phase pipeline) |
|
|
| `vm_get_build_state` | Poll golden build progress |
|
|
| `vm_list_recordings` | List VNC recordings from build logs |
|
|
| `vm_list_screenshots` | List screenshots from build logs |
|
|
|
|
---
|
|
|
|
## Executor Integration
|
|
|
|
When the Executor needs UI tests (bug requires visual verification):
|
|
|
|
1. Call `vm_prepare_session()` after build succeeds
|
|
2. Call `run_tests()` with the scheme
|
|
3. Poll `get_test_run_status()` / `get_test_results()`
|
|
4. On failure: capture `peekaboo_image()` + `get_crash_reports()` for diagnosis
|
|
5. **Always** `vm_stop()` + `vm_delete()` before worktree cleanup
|
|
|
|
See also: `ui-testing.md` — Swift UITestCase patterns, feature flag setup, test structure.
|
|
|
|
---
|
|
|
|
## Notes
|
|
|
|
- VMs require Apple Silicon + lume (`brew install trycua/tap/lume`)
|
|
- Golden image must be built and provisioned before test runs work
|
|
- `run_tests` uses pre-built xctestrun — **build on host before running in VM**
|
|
- Peekaboo (AX/screenshot proxy) must be running inside the VM — `vm_prepare_session` handles this
|