152 lines
8.4 KiB
Markdown
Executable File
152 lines
8.4 KiB
Markdown
Executable File
# Архив исходника Broker
|
|
|
|
Историческая копия до очистки репозитория 2026-10-16. Источник: `mallexxx/execution-broker`, commit `ae92c15d31c8396789765cebf82f7971d6f4cb12`, файл `docs/testing.md`. Старые статусы и deployment pins ниже не заменяют актуальную [[personal/projects/balda/execution-broker|эксплуатационную карточку]].
|
|
|
|
---
|
|
|
|
# Testing and acceptance
|
|
|
|
`scripts/check-runner-temporary.py --worker-image IMAGE` uses an executor image
|
|
containing Git and Go. It performs Runner checkout, compiles and executes Go in
|
|
the private run temporary directory while the shared `/tmp` stays `noexec`.
|
|
|
|
Run `make check` before changing a deployment. It executes all Go tests with the
|
|
race detector, `go vet`, packaged macOS bootstrap checks and coordinated backup
|
|
failure-path tests. `scripts/check-callee.sh` validates the pinned Callee adapter.
|
|
These checks do not require provider credentials or post chat messages.
|
|
|
|
The native macOS race suite includes a real zombie-only process-group fixture:
|
|
it reproduces XNU's `EPERM` and requires independent stopped-process evidence.
|
|
The Callee gate loads the actual Codex role files and injects edit, execute and
|
|
other permission requests; every read-only role must reject them. When a shared
|
|
workflow changes, compare Broker, gateway and each selected worker's actual
|
|
pack and digest before admitting a new run.
|
|
|
|
Run `python3 scripts/check-tooling-stacks.py` for real Make/C, Python, Node and
|
|
JVM profile acceptance. An installed JDK can be selected with `--java-home`.
|
|
For an isolated Linux run, build `fixtures/tooling/Dockerfile.linux` and pass
|
|
its image using `--docker-image`. Both modes retain actual versions and compile
|
|
real fixture projects; neither installs a compiler on the operator host.
|
|
The ordinary unit suite also checks failed probes, approved sources, checksums,
|
|
cache quota, archive path boundaries and durable pre-execution restart.
|
|
|
|
`scripts/check-tooling-handoff.py --base-image execution-broker-tooling:ci`
|
|
requires the prepared tooling fixture image, Linux binaries and Callee. It runs
|
|
five real executions through the image builder/gateway, proves cache hits,
|
|
changes the actual Node SDK through a new profile, and checks retained provenance
|
|
and project output. A mid-run checkpoint captures changed/untracked source,
|
|
restarts Broker twice, requires a new environment approval and resumes only the
|
|
remaining verification. An independent HTTP counter proves the earlier external
|
|
mutation happened once. All five runs share one immutable repository bundle.
|
|
Workers use an internal network and scoped Broker routes;
|
|
provider credentials and model inference are absent. This gate also catches a
|
|
builder that drops the selected profile from Runner configuration.
|
|
|
|
## Service integration
|
|
|
|
Build the Linux worker and pinned Beads binary using the README instructions.
|
|
Use a new test-only Beads workspace, then run:
|
|
|
|
```sh
|
|
EXECUTION_WORKER_IMAGE="$(docker image inspect execution-broker-worker:local --format '{{.Id}}')" \
|
|
BEADS_BINARY="$PWD/.build/linux/bd" BEADS_WORKSPACE=/absolute/test/beads \
|
|
go test -tags=integration -count=1 -v ./tests/integration
|
|
```
|
|
|
|
Use a Beads binary matching the machine that runs `go test`. The Linux binary is
|
|
appropriate in Linux CI; use the native binary on macOS. Missing Docker or Beads
|
|
prerequisites produce explicit skips. A skipped integration is not acceptance.
|
|
The remote test confines its commands to a temporary directory; it does not
|
|
certify administration of a real managed host.
|
|
|
|
The required input gate builds a static Linux test fixture, creates a private
|
|
Docker network with its own Broker alias, and runs a real producer/consumer pair
|
|
through Callee. It verifies digest-bound input bytes, original admission after
|
|
Broker restart, retained provenance and cleanup. It mounts no production state,
|
|
uses no provider credentials and publishes no ports:
|
|
|
|
```sh
|
|
python3 scripts/check-artifact-inputs.py --worker-image execution-broker-worker:local
|
|
```
|
|
|
|
Run from a source checkout; build the Linux services/Callee and worker image
|
|
first. The test fixture image
|
|
contains Docker administrative tooling and receives the socket only for its own
|
|
acceptance resources; leased workers keep the normal isolation. The static
|
|
cross-OS fixture complements the source/race checks; it does not claim to run a
|
|
race detector itself. CI runs this gate explicitly; a skipped ordinary input
|
|
integration without its private network is not acceptance.
|
|
|
|
## Native archives
|
|
|
|
`scripts/check-release.py` extracts a trusted archive into fresh private state
|
|
and verifies provenance, seven binary versions, synthetic credential import and
|
|
revocation, MCP authentication, SQLite backup and graceful shutdown. It uses no
|
|
live provider credentials. Host OS and CPU architecture must match the target.
|
|
`--allow-emulation` labels a local compatibility check as emulated; CI never uses
|
|
that flag as evidence for a native target.
|
|
|
|
The checker supports Python 3.8 and newer, including the stock developer Python
|
|
on Monterey. It validates every archive member before copying regular files,
|
|
rejects links and special files, and keeps extracted permissions owner-only.
|
|
|
|
The CI matrix runs Linux/macOS on ARM64 and AMD64. Passing a cross-build alone
|
|
is insufficient. Release promotion requires the native checks and service
|
|
integration to pass for the exact commit being distributed.
|
|
|
|
## Deployment acceptance
|
|
|
|
Use a dedicated repository, channel and credential identity. Confirm:
|
|
|
|
1. A fresh Balda session can discover the installed plugin, create a Story and
|
|
start an allowed workflow without direct Docker/SSH access.
|
|
2. The Human question arrives only in the configured approval channel: Telegram
|
|
when enabled for the session, otherwise the original conversation. A Telegram
|
|
request sends one readable message; full details are sent only after an
|
|
explicit button click, which never resumes the worker. Only the user's actual
|
|
decision resumes the matching request; an identical answer is idempotent.
|
|
3. Restart Broker, Prism and Balda during the pause. The run remains recoverable,
|
|
and no duplicate run or Story is created.
|
|
4. Check the terminal state, download and hash every artifact, confirm cleanup,
|
|
and verify Story closure independently of the model's prose.
|
|
5. Exercise failure, cancellation and recovery. Review duplicate chat responses,
|
|
resource leftovers, credential ownership and failed-turn monitoring.
|
|
|
|
Run `scripts/macos-smoke.py` only against a dedicated Virfield profile.
|
|
Record the exact installed Virfield version and its native GUI/new SSH key
|
|
storage and console qualification before bootstrap. If preparation returns
|
|
`guest_keychain_failed`, record a failed prerequisite and confirmed cleanup;
|
|
no workload or provider acceptance occurred. Use the [readiness diagnostic
|
|
procedure](operations.md#vm-readiness-failures) and keep the
|
|
[current production boundary](support.md#runtime-matrix) separate from earlier
|
|
successful OS/profile fixtures.
|
|
|
|
`scripts/codex-smoke.py` makes real provider calls and approves only its fixed test
|
|
fixture. Inspect their `--help` before execution. Store logs and raw proof in a
|
|
private operator directory; neither belongs in the public package.
|
|
|
|
Service probe scripts accept `--balda-container`, `--broker-container`,
|
|
`--prism-container`, `--broker-url`, `--probe-image` and `--executionctl`.
|
|
They mount the Linux probe binary and one token read-only into a disposable
|
|
container sharing Balda's network namespace. They do not copy secrets or
|
|
executables into Balda. Set `--executionctl` to a binary matching Docker's CPU;
|
|
keep proof files in a private working directory.
|
|
|
|
## Binary-only Docker installation
|
|
|
|
On each Linux architecture, CI extracts the release archive, builds all four
|
|
service/worker images from that bundle, and runs the real Callee/Beads fixture
|
|
through an isolated Compose project. The test covers Human responses, Broker
|
|
and Prism restarts, task completion, artifact hashes and worker cleanup. It
|
|
uses fresh secrets and random loopback ports, and removes only its own Docker
|
|
resources afterward. No live Balda instance or provider identity is used.
|
|
|
|
```sh
|
|
python3 scripts/check-binary-deployment.py \
|
|
--root /absolute/path/to/extracted-linux-bundle \
|
|
--state-dir /absolute/path/to/new-private-test-directory
|
|
```
|
|
|
|
Both paths must be visible to the Docker daemon. The test retains its private
|
|
configuration and diagnostic logs in the new directory; do not publish them.
|