8.4 KiB
Архив исходника Broker
Историческая копия до очистки репозитория 2026-10-16. Источник: mallexxx/execution-broker, commit ae92c15d31c8396789765cebf82f7971d6f4cb12, файл docs/testing.md. Старые статусы и deployment pins ниже не заменяют актуальную personal/projects/balda/execution-broker.
Testing and acceptance
scripts/check-runner-temporary.py --worker-image IMAGE uses an executor image
containing Git and Go. It performs Runner checkout, compiles and executes Go in
the private run temporary directory while the shared /tmp stays noexec.
Run make check before changing a deployment. It executes all Go tests with the
race detector, go vet, packaged macOS bootstrap checks and coordinated backup
failure-path tests. scripts/check-callee.sh validates the pinned Callee adapter.
These checks do not require provider credentials or post chat messages.
The native macOS race suite includes a real zombie-only process-group fixture:
it reproduces XNU's EPERM and requires independent stopped-process evidence.
The Callee gate loads the actual Codex role files and injects edit, execute and
other permission requests; every read-only role must reject them. When a shared
workflow changes, compare Broker, gateway and each selected worker's actual
pack and digest before admitting a new run.
Run python3 scripts/check-tooling-stacks.py for real Make/C, Python, Node and
JVM profile acceptance. An installed JDK can be selected with --java-home.
For an isolated Linux run, build fixtures/tooling/Dockerfile.linux and pass
its image using --docker-image. Both modes retain actual versions and compile
real fixture projects; neither installs a compiler on the operator host.
The ordinary unit suite also checks failed probes, approved sources, checksums,
cache quota, archive path boundaries and durable pre-execution restart.
scripts/check-tooling-handoff.py --base-image execution-broker-tooling:ci
requires the prepared tooling fixture image, Linux binaries and Callee. It runs
five real executions through the image builder/gateway, proves cache hits,
changes the actual Node SDK through a new profile, and checks retained provenance
and project output. A mid-run checkpoint captures changed/untracked source,
restarts Broker twice, requires a new environment approval and resumes only the
remaining verification. An independent HTTP counter proves the earlier external
mutation happened once. All five runs share one immutable repository bundle.
Workers use an internal network and scoped Broker routes;
provider credentials and model inference are absent. This gate also catches a
builder that drops the selected profile from Runner configuration.
Service integration
Build the Linux worker and pinned Beads binary using the README instructions. Use a new test-only Beads workspace, then run:
EXECUTION_WORKER_IMAGE="$(docker image inspect execution-broker-worker:local --format '{{.Id}}')" \
BEADS_BINARY="$PWD/.build/linux/bd" BEADS_WORKSPACE=/absolute/test/beads \
go test -tags=integration -count=1 -v ./tests/integration
Use a Beads binary matching the machine that runs go test. The Linux binary is
appropriate in Linux CI; use the native binary on macOS. Missing Docker or Beads
prerequisites produce explicit skips. A skipped integration is not acceptance.
The remote test confines its commands to a temporary directory; it does not
certify administration of a real managed host.
The required input gate builds a static Linux test fixture, creates a private Docker network with its own Broker alias, and runs a real producer/consumer pair through Callee. It verifies digest-bound input bytes, original admission after Broker restart, retained provenance and cleanup. It mounts no production state, uses no provider credentials and publishes no ports:
python3 scripts/check-artifact-inputs.py --worker-image execution-broker-worker:local
Run from a source checkout; build the Linux services/Callee and worker image first. The test fixture image contains Docker administrative tooling and receives the socket only for its own acceptance resources; leased workers keep the normal isolation. The static cross-OS fixture complements the source/race checks; it does not claim to run a race detector itself. CI runs this gate explicitly; a skipped ordinary input integration without its private network is not acceptance.
Native archives
scripts/check-release.py extracts a trusted archive into fresh private state
and verifies provenance, seven binary versions, synthetic credential import and
revocation, MCP authentication, SQLite backup and graceful shutdown. It uses no
live provider credentials. Host OS and CPU architecture must match the target.
--allow-emulation labels a local compatibility check as emulated; CI never uses
that flag as evidence for a native target.
The checker supports Python 3.8 and newer, including the stock developer Python on Monterey. It validates every archive member before copying regular files, rejects links and special files, and keeps extracted permissions owner-only.
The CI matrix runs Linux/macOS on ARM64 and AMD64. Passing a cross-build alone is insufficient. Release promotion requires the native checks and service integration to pass for the exact commit being distributed.
Deployment acceptance
Use a dedicated repository, channel and credential identity. Confirm:
- A fresh Balda session can discover the installed plugin, create a Story and start an allowed workflow without direct Docker/SSH access.
- The Human question arrives only in the configured approval channel: Telegram when enabled for the session, otherwise the original conversation. A Telegram request sends one readable message; full details are sent only after an explicit button click, which never resumes the worker. Only the user's actual decision resumes the matching request; an identical answer is idempotent.
- Restart Broker, Prism and Balda during the pause. The run remains recoverable, and no duplicate run or Story is created.
- Check the terminal state, download and hash every artifact, confirm cleanup, and verify Story closure independently of the model's prose.
- Exercise failure, cancellation and recovery. Review duplicate chat responses, resource leftovers, credential ownership and failed-turn monitoring.
Run scripts/macos-smoke.py only against a dedicated Virfield profile.
Record the exact installed Virfield version and its native GUI/new SSH key
storage and console qualification before bootstrap. If preparation returns
guest_keychain_failed, record a failed prerequisite and confirmed cleanup;
no workload or provider acceptance occurred. Use the readiness diagnostic
procedure and keep the
current production boundary separate from earlier
successful OS/profile fixtures.
scripts/codex-smoke.py makes real provider calls and approves only its fixed test
fixture. Inspect their --help before execution. Store logs and raw proof in a
private operator directory; neither belongs in the public package.
Service probe scripts accept --balda-container, --broker-container,
--prism-container, --broker-url, --probe-image and --executionctl.
They mount the Linux probe binary and one token read-only into a disposable
container sharing Balda's network namespace. They do not copy secrets or
executables into Balda. Set --executionctl to a binary matching Docker's CPU;
keep proof files in a private working directory.
Binary-only Docker installation
On each Linux architecture, CI extracts the release archive, builds all four service/worker images from that bundle, and runs the real Callee/Beads fixture through an isolated Compose project. The test covers Human responses, Broker and Prism restarts, task completion, artifact hashes and worker cleanup. It uses fresh secrets and random loopback ports, and removes only its own Docker resources afterward. No live Balda instance or provider identity is used.
python3 scripts/check-binary-deployment.py \
--root /absolute/path/to/extracted-linux-bundle \
--state-dir /absolute/path/to/new-private-test-directory
Both paths must be visible to the Docker daemon. The test retains its private configuration and diagnostic logs in the new directory; do not publish them.