docs: consolidate current Virfield release and remove legacy notes
This commit is contained in:
Alexey Martemyanov committed
1 parent
0e4202c5de
commit
3c0895e62c
11 files changed
+152
-272
No files matched your search
@@ -3,6 +3,7 @@ source: ~/DuckDuckGo/apple-browsers.git/main/.github/workflows/
|
||||
confidence: 1.0
|
||||
namespace: work
|
||||
last_updated: 2026-04-28
|
||||
virfield_updated: 2026-09-30
|
||||
tags: [ci, github-actions, xcodebuild, testing, build]
|
||||
---
|
||||
|
||||
@@ -121,31 +122,16 @@ set -o pipefail && xcodebuild test-without-building \
|
||||
2>&1 | xcbeautify
|
||||
```
|
||||
|
||||
**Option B — Via VM (virfield/ddg-vm MCP, matches CI environment):**
|
||||
```
|
||||
# 1. Build on host first (DerivedData must exist)
|
||||
xcodebuild build-for-testing -scheme "macOS UI Tests CI" -derivedDataPath DerivedData ...
|
||||
**Option B — Via a Virfield VM lease:**
|
||||
|
||||
# 2. Start VM session
|
||||
vm_prepare_session() → { vmName, ip }
|
||||
Virfield v2 provides a prepared VM, scoped SSH and lifecycle cleanup. Build/test
|
||||
execution and result collection belong to Runner/Executor, using the VM's local
|
||||
disk. See [[work/wiki/apple-browsers/vm-ui-testing]] for the current flow and
|
||||
[[personal/projects/virfield]] for the release and installation reference.
|
||||
|
||||
# 3. Run tests in VM
|
||||
run_tests(vm_id: vmName, scheme: "DuckDuckGo macOS", workspace: "<VMShare path>")
|
||||
|
||||
# 4. Poll
|
||||
get_test_run_status(vm_id: vmName, run_id: run_id)
|
||||
get_test_results(vm_id: vmName, run_id: run_id)
|
||||
|
||||
# 5. On failure: inspect
|
||||
peekaboo_image(vm_id: vmName)
|
||||
get_crash_reports(vm_id: vmName)
|
||||
|
||||
# 6. Always clean up
|
||||
vm_stop(vm_id: vmName)
|
||||
vm_delete(vm_id: vmName)
|
||||
```
|
||||
|
||||
See `vm-ui-testing.md` for full ddg-vm MCP reference.
|
||||
The new Broker/Runner → apple-browsers UI-test integration is not yet accepted.
|
||||
The current Virfield recipe targets macOS 27/Xcode 27+, so equivalence with the
|
||||
project's CI/Xcode version must be checked explicitly for the chosen branch.
|
||||
|
||||
### Key notes
|
||||
|
||||
@@ -217,6 +203,6 @@ Key differences from CI:
|
||||
|
||||
## Cross-Reference
|
||||
|
||||
- `vm-ui-testing.md` — full virfield/ddg-vm MCP reference for VM-based UI tests
|
||||
- [[work/wiki/apple-browsers/vm-ui-testing]] — current Virfield lease flow and UI-test integration boundary
|
||||
- `testing.mdc` → `testing.md` — UITestCase patterns, feature flag setup in tests
|
||||
- `development-commands.mdc` → `development-commands.md` — team-approved xcodebuild flags
|
||||
@@ -1,126 +1,34 @@
|
||||
---
|
||||
source: ~/Developer/virfield/
|
||||
source: https://github.com/mallexxx/virfield/tree/v2.0.0
|
||||
confidence: 1.0
|
||||
namespace: work
|
||||
last_updated: 2026-04-28
|
||||
tags: [ui-testing, vm, virfield, ddg-vm, xcode]
|
||||
last_updated: 2026-09-30
|
||||
tags: [ui-testing, vm, virfield, xcode]
|
||||
---
|
||||
# VM UI Testing — Virfield
|
||||
|
||||
# VM UI Testing — virfield / ddg-vm MCP
|
||||
Текущий host manager — Virfield v2.0.0 на Go. Карточка проекта: [[personal/projects/virfield]]. Эта заметка описывает границу UI-test integration; установка, API/MCP и команды поддерживаются в [Operations](https://github.com/mallexxx/virfield/blob/main/docs/OPERATIONS.md).
|
||||
|
||||
**What it is**: macOS VM lifecycle console + MCP server for AI-driven UI test automation.
|
||||
**How it runs**: Golden image (built once) → APFS CoW clone per test session → run → delete.
|
||||
**MCP config**: `~/Developer/virfield/server/mcp-server.ts` (configured in `.claude/.mcp.json` as `ddg-vm`)
|
||||
**Web UI**: `http://localhost:5173` (requires `npm run dev` in `~/Developer/virfield/`)
|
||||
## Поток UI-теста
|
||||
|
||||
---
|
||||
1. Получить lease подготовленного image с отдельным caller-owned SSH public key через CLI/API или MCP `vm_acquire`; дождаться состояния `ready`.
|
||||
2. По pinned SSH перенести/клонировать исходники на локальный диск VM и запустить build/test внутри гостя. Runner/Executor отвечает за команды, Xcode scheme и сбор результатов.
|
||||
3. Выгрузить `.xcresult`, логи и screenshots до освобождения lease.
|
||||
4. Вызвать `vm_release` и проверить завершение cleanup; expiry также запускает cleanup. Освобождение lease удаляет disposable VM.
|
||||
|
||||
## Standard Agent Flow for UI Tests
|
||||
Virfield управляет VM и соединением, а не выполняет XCUITest workflow через MCP. Протокол Runner/Broker и полный автоматический UI-test flow требуют отдельной интеграционной приёмки. Гостевые Peekaboo/tools устанавливаются full image profile, но host MCP не экспортирует их UI-команды.
|
||||
|
||||
### Step 1 — Start a session (single call)
|
||||
## Ограничения
|
||||
|
||||
```
|
||||
vm_prepare_session()
|
||||
```
|
||||
- Максимум две macOS VM с учётом внешних VM и reservations; `capacity_exhausted` — повод поставить run в очередь Broker.
|
||||
- Успешный acquire означает принятый job, не готовность гостя. Использовать lease только после `ready`.
|
||||
- Recipe не монтирует host shared folders. Build workspace находится в VM; личные host SSH keys и provider credentials не запекаются в golden.
|
||||
- Поддерживается macOS 27 build `26A428`; full UI-test profile требует Xcode 27+. Совместимость этого guest/toolchain с конкретной веткой apple-browsers проверяется отдельно.
|
||||
- Реальный golden/clone E2E Virfield прошёл на одном Mac. Это не подтверждение XCUITest run для apple-browsers через новый Broker/Runner.
|
||||
|
||||
This does everything in sequence: clone golden → start VM → wait for SSH → wait for Peekaboo ready.
|
||||
Returns `{ vmName, ip }`. Use `vmName` in all subsequent calls.
|
||||
## Ссылки
|
||||
|
||||
### Step 2 — Run XCUITests
|
||||
|
||||
```
|
||||
run_tests(
|
||||
vm_id: vmName,
|
||||
scheme: "DuckDuckGo macOS",
|
||||
workspace: "<VMShare path to DuckDuckGo-macOS.xcworkspace>"
|
||||
)
|
||||
```
|
||||
|
||||
Returns `run_id` immediately. Poll with:
|
||||
|
||||
```
|
||||
get_test_run_status(vm_id: vmName, run_id: run_id)
|
||||
get_test_results(vm_id: vmName, run_id: run_id)
|
||||
```
|
||||
|
||||
Uses pre-built `xctestrun` from `VMShare/DerivedData` — build must happen on host first.
|
||||
|
||||
### Step 3 — Inspect failures
|
||||
|
||||
```
|
||||
peekaboo_image(vm_id: vmName) # screenshot
|
||||
peekaboo_see(vm_id: vmName, app: "...") # AX tree
|
||||
ax_snapshot(vm_id: vmName, app: "...", name: "before")
|
||||
ax_diff_last(vm_id: vmName, app: "...") # what changed
|
||||
get_crash_reports(vm_id: vmName) # crash logs
|
||||
get_log_stream(vm_id: vmName, path: "...", lines: 50)
|
||||
```
|
||||
|
||||
### Step 4 — Cleanup (ALWAYS — especially from Executor)
|
||||
|
||||
```
|
||||
vm_stop(vm_id: vmName)
|
||||
vm_delete(vm_id: vmName)
|
||||
```
|
||||
|
||||
**Executor must call these before removing its worktree.** Never leave orphaned VMs.
|
||||
|
||||
---
|
||||
|
||||
## All MCP Tools
|
||||
|
||||
| Tool | Purpose |
|
||||
|------|---------|
|
||||
| `vm_list` | List all VMs with state, IP, checklist status |
|
||||
| `vm_prepare_session` | **Main entry point** — clone + start + wait for ready |
|
||||
| `vm_start` | Start a VM (optionally clone from golden first) |
|
||||
| `vm_stop` | Stop a running VM |
|
||||
| `vm_clone_golden` | APFS CoW clone from golden image |
|
||||
| `vm_delete` | Delete a VM (run VMs only, never golden) |
|
||||
| `vm_get_ip` | Get IP of running VM |
|
||||
| `vm_status` | Full status including Peekaboo connection state |
|
||||
| `vm_ssh_exec` | Run a shell command in VM via SSH |
|
||||
| `run_tests` | Launch XCUITests in VM (returns run_id immediately) |
|
||||
| `get_test_run_status` | Poll whether xcodebuild is still running |
|
||||
| `get_test_results` | Parse test result artifacts for a run_id |
|
||||
| `get_crash_reports` | Read crash reports from VM |
|
||||
| `get_log_stream` | Tail a log file from VM |
|
||||
| `peekaboo_see` | Dump AX accessibility tree for an app |
|
||||
| `peekaboo_image` | Screenshot from VM |
|
||||
| `peekaboo_click` | Click by query / element ID / coords |
|
||||
| `peekaboo_type` | Type text in VM |
|
||||
| `peekaboo_hotkey` | Send hotkey (e.g. cmd+s) |
|
||||
| `peekaboo_scroll` | Scroll in VM |
|
||||
| `peekaboo_list_apps` | List running apps in VM |
|
||||
| `peekaboo_permissions` | Check Screen Recording / Accessibility permissions |
|
||||
| `ax_snapshot` | Take named AX snapshot for later diff |
|
||||
| `ax_diff_last` | Diff current AX state vs last snapshot |
|
||||
| `ax_diff` | Diff two named snapshots by ID |
|
||||
| `ax_query` | Query elements from latest snapshot |
|
||||
| `vm_build_golden` | Build golden VM from IPSW (full 4-phase pipeline) |
|
||||
| `vm_get_build_state` | Poll golden build progress |
|
||||
| `vm_list_recordings` | List VNC recordings from build logs |
|
||||
| `vm_list_screenshots` | List screenshots from build logs |
|
||||
|
||||
---
|
||||
|
||||
## Executor Integration
|
||||
|
||||
When the Executor needs UI tests (bug requires visual verification):
|
||||
|
||||
1. Call `vm_prepare_session()` after build succeeds
|
||||
2. Call `run_tests()` with the scheme
|
||||
3. Poll `get_test_run_status()` / `get_test_results()`
|
||||
4. On failure: capture `peekaboo_image()` + `get_crash_reports()` for diagnosis
|
||||
5. **Always** `vm_stop()` + `vm_delete()` before worktree cleanup
|
||||
|
||||
See also: `ui-testing.md` — Swift UITestCase patterns, feature flag setup, test structure.
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- VMs require Apple Silicon + lume (`brew install trycua/tap/lume`)
|
||||
- Golden image must be built and provisioned before test runs work
|
||||
- `run_tests` uses pre-built xctestrun — **build on host before running in VM**
|
||||
- Peekaboo (AX/screenshot proxy) must be running inside the VM — `vm_prepare_session` handles this
|
||||
- [Image pipeline](https://github.com/mallexxx/virfield/blob/main/docs/IMAGE-PIPELINE.md) — golden preparation и guest policy.
|
||||
- [Verification](https://github.com/mallexxx/virfield/blob/main/docs/VERIFICATION.md) — фактическая приёмка и оставшиеся проверки.
|
||||
- [[work/wiki/apple-browsers/ci-pipeline]] — CI проекта и требования к Xcode.
|
||||
- [[work/wiki/apple-browsers/ui-testing]] — правила самих UI-тестов.
|
||||
Reference in new issue
Block a user