[2026-10-16] eagle: personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/CHANGELOG.md personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/README.md personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/docs/deployment.md personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/docs/hermes-workflow-gap-analysis.md personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/docs/ipad-deployment-plan.md personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/docs/recovery.md personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/docs/support.md personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/docs/testing.md personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/docs/work-plan.md personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/index.md
This commit is contained in:
Alexey Martemyanov committed
1 parent
23c63a0ef6
commit
00a2d211af
10 files changed
+1238
No files matched your search
@@ -0,0 +1,182 @@
|
||||
# Архив исходника Broker
|
||||
|
||||
Историческая копия до очистки репозитория 2026-10-16. Источник: `mallexxx/execution-broker`, commit `ae92c15d31c8396789765cebf82f7971d6f4cb12`, файл `CHANGELOG.md`. Старые статусы и deployment pins ниже не заменяют актуальную [[personal/projects/balda/execution-broker|эксплуатационную карточку]].
|
||||
|
||||
---
|
||||
|
||||
# Changelog
|
||||
|
||||
## 0.1.0-rc.33 (candidate)
|
||||
|
||||
- Deny ACP permission expansion in the Codex reviewer role. Keep reviewer-only
|
||||
write probes out of verifier assertions and record bounded actual tool output
|
||||
for the acceptance probe instead of accepting a model's copied claim.
|
||||
- On macOS, settle a zombie-only provider process group after `EPERM` only when
|
||||
an independent process observation confirms that no executable member remains.
|
||||
Retain the credential fence when observation fails or a live member remains.
|
||||
- Refresh the bundled workflow digest to the accepted reviewer policy.
|
||||
- Record accepted production Story/cancellation/restart checks, native provider
|
||||
refusals in isolated synthetic vaults, and independent recovery of all three
|
||||
working provider identities. Deliberate upstream revocation and browser login
|
||||
remain unverified under the operator-approved acceptance scope.
|
||||
|
||||
- Keep an environment approval outside worker capacity until allocation, so an
|
||||
unrelated Story can run while the owner reviews the replacement.
|
||||
|
||||
- Add immutable repository/commit Git bundle caching with scoped capabilities,
|
||||
durable holds, bounded storage, checksum-verified restartable transfer and
|
||||
guest-local checkouts without upstream credentials.
|
||||
- Add operator-authored environment checkpoints, raw source snapshots and
|
||||
separately approved continuation with unchanged scope/deadline, transactional
|
||||
single admission, bounded replacements and cancellation across the lineage.
|
||||
Require execution/v3 so older workers cannot replay or skip preparation.
|
||||
|
||||
- Keep compiler temporary products in the private executable run directory while
|
||||
shared `/tmp` remains `noexec`. Verify real Go compilation/execution under that
|
||||
worker policy. Pass the declared tooling policy through macOS admission
|
||||
preflight and generated Docker executor configuration.
|
||||
|
||||
- Add operator-owned tooling profiles with OS/architecture, exact recipe digest,
|
||||
pinned package/archive sources, isolated installation, readiness probes and
|
||||
retained observed versions. Replace the special macOS Go installer with the
|
||||
same declarative archive mechanism. Keep project and executor runtimes separate.
|
||||
|
||||
- Add session-bound, digest-pinned artifact inputs with atomic retention holds,
|
||||
scoped worker download capabilities, resumable transfer and safe pre-execution
|
||||
restart. Retain input provenance; reject older worker protocol before execution.
|
||||
|
||||
- Support the current Balda managed plugin lifecycle and native session waits.
|
||||
Recover uncertain run admission by its original key and atomically close late
|
||||
Start/retry requests when canceled; keep the original conversation identity.
|
||||
Remove retired webhook-session and plugin CLI installation examples.
|
||||
|
||||
- Build pinned Callee with an explicit x/net v0.60.0 security override, retain
|
||||
the effective dependency/license manifest, and scan the built adapter in CI.
|
||||
|
||||
- Require Go 1.26.9 so source and release builds include the standard-library
|
||||
security fixes. Exempt only the exact historical Mattermost post-ID scanner
|
||||
finding; credential detection remains enabled for every other finding.
|
||||
|
||||
- Reject Docker daemon sockets/groups in worker profiles and remove the worker
|
||||
Docker CLI dependency. Keep Docker administrative access in the gateway.
|
||||
- Pin static and dynamically resolved images in durable acquisition handles;
|
||||
reject Broker/gateway drift before allocation and release an undelivered
|
||||
credential on that explicit rejection.
|
||||
- Repeat verifier assertions in the disposable reviewed checkout, retain actual
|
||||
exit codes/output, and reject source/index/commit changes during verification.
|
||||
Expected application errors must be wrapped in zero-exit assertions; a failed
|
||||
build or test suite cannot become success through expected_exit_code.
|
||||
|
||||
- Add optional owner-only Telegram approval actions for Broker-managed workflow
|
||||
and ACP questions, enabled explicitly in configuration. Keep existing command
|
||||
authorization available, including after Telegram prompt expiry, with durable
|
||||
request binding, restart recovery and competing-channel deduplication.
|
||||
- Select a single approval channel: configured Telegram receives the question
|
||||
exclusively; otherwise use the original Balda conversation. Keep result
|
||||
delivery in that conversation and suppress duplicate questions during polling.
|
||||
- Configure macOS guests automatically for each registered Broker-managed
|
||||
repository, preserving package/workflow pins and limiting guest access to
|
||||
that run's repository. Reject incompatible packages before allocation and
|
||||
deliver permanent start failures through the original Balda conversation.
|
||||
- Provide a checksum-verified Go SDK automatically in disposable macOS workers
|
||||
and expose concrete failed-role explanations from verified Callee logs even
|
||||
when no quality report was produced.
|
||||
|
||||
- Add session-bound repository creation, folder access and managed Git
|
||||
publication. Keep operator grants separate from agent MCP access.
|
||||
- Add selectable Virfield images and bounded Docker image configuration for
|
||||
approved worker profiles.
|
||||
- Allow bounded requirement amendments at the Human approval gate, then repair
|
||||
review and verification findings within the same run.
|
||||
- Expose quality findings through `execution_diagnostics` and classify Callee
|
||||
failures when a quality report is unavailable.
|
||||
- Tighten managed Git fetch routing, image watch authorization and deployment
|
||||
image selection after PR review.
|
||||
|
||||
## 0.1.0-rc.10 (candidate)
|
||||
|
||||
- Enforce OpenCode role permissions in the isolated runtime: plan roles deny
|
||||
shell and edits, build roles route effects through ACP permission requests,
|
||||
and incompatible mode/permission pairs fail before inference. Disable
|
||||
repository plugins in the managed OpenCode ACP launcher and prevent injected
|
||||
environment variables from overriding the operator policy.
|
||||
- Accept a fresh macOS DeepSeek Story with six roles, Human approval, executed
|
||||
tests, quality gate, Broker/issuer restart, credential checkpoint and VM cleanup. An
|
||||
adversarial write probe reached Callee's ACP deny handler and left no file.
|
||||
- Clarify OpenCode's exact Callee control record in every role prompt so
|
||||
`plan` and `pass` are not mistaken for valid workflow outcomes.
|
||||
|
||||
## 0.1.0-rc.9 (candidate)
|
||||
|
||||
- Add Docker worker credential issuer integration: issue at start after durable
|
||||
admission, checkpoint a stopped Runner before container/volume deletion, and
|
||||
keep the identity leased when a worker or checkpoint is lost.
|
||||
- Verify DeepSeek API-key reuse, changed-key rejection, capacity refusal and
|
||||
revoked lost-worker cleanup with synthetic tests. Live Docker provider and
|
||||
sandbox acceptance remain open.
|
||||
|
||||
## 0.1.0-rc.8 (candidate)
|
||||
|
||||
- Preserve exponential retry backoff while a macOS VM is still provisioning.
|
||||
A credential snapshot alone no longer clears the delay after a proven
|
||||
`image_in_use` refusal, avoiding rapid issuer issue/abort cycles during
|
||||
Virfield image maintenance.
|
||||
|
||||
## 0.1.0-rc.7 (candidate)
|
||||
|
||||
- Accept optional bounded `notes` in verifier JSON while still rejecting
|
||||
unknown fields, failed checks and unexecuted verification.
|
||||
- Document live macOS OpenCode/DeepSeek inference, two complete Stories and
|
||||
fresh-VM credential reuse after Broker and issuer restart. This acceptance
|
||||
is specific to the tested operator setup; denied-write and upstream
|
||||
revocation checks remain open.
|
||||
|
||||
## 0.1.0-rc.6 (candidate)
|
||||
|
||||
- Complete Claude macOS Story acceptance with saved account reuse after issuer
|
||||
and Broker restart. Planning and review keep denied write permissions; the
|
||||
verifier runs checks before the reviewer evaluates them.
|
||||
- Save quality evidence in the private run directory when native Claude
|
||||
Keychain access requires the macOS account HOME.
|
||||
- OpenCode, Docker/remote credentials, provider revocation and managed-host
|
||||
deployment still require their own acceptance before a stable release.
|
||||
|
||||
## 0.1.0-rc.5 (candidate)
|
||||
|
||||
- Add optional Codex/Claude browser sign-in before VM admission, durable Human waiting and private authorization-code completion.
|
||||
- Preserve the completed-onboarding marker alongside validated Claude account sessions in disposable workers.
|
||||
- Package verified portable executor runtimes with standalone Node 22 for automatic macOS bootstrap and Linux image builds.
|
||||
- Publish installed worker profile and OS constraints for model selection; reject incompatible macOS execution.
|
||||
- Add explicit Claude/OpenCode executor and model profiles, pinned optional
|
||||
runtimes, provider-specific workflow packs and private enrollment tooling.
|
||||
- Isolate provider credentials by identity, preserve the legacy Codex vault,
|
||||
validate checkpoint identity and include all identities in drained backup checks.
|
||||
- Probe real native executor/ACP compatibility in CI; real-provider Story and
|
||||
credential expiration acceptance remain required before enabling new profiles.
|
||||
|
||||
- Package the execution skill and `/execute` command as an installable Balda
|
||||
plugin with a marketplace index.
|
||||
- Validate native release archives in fresh state on Linux/macOS ARM64 and AMD64.
|
||||
- Separate public operator documentation from private acceptance reports.
|
||||
- Build archives without local filesystem metadata or operational transcripts.
|
||||
|
||||
## 0.1.0-rc.3
|
||||
|
||||
- Add encrypted persistent OAuth, macOS Keychain integration, exclusive run
|
||||
leases and credential checkpointing before VM cleanup.
|
||||
- Add `execution-credentials`, optional LaunchAgent deployment and coordinated
|
||||
stack backup with inert restore verification.
|
||||
- Support Virfield v2 lease-bound SSH tunnels and credential-free guest bootstrap.
|
||||
- Preserve quality-gate diagnostics and require executed verification evidence.
|
||||
|
||||
## 0.1.0-rc.2
|
||||
|
||||
- Harden malformed-stream handling, cancellation artifacts, rollback, retention,
|
||||
event validation and cleanup recovery.
|
||||
- Add independent notification cursors, signed host-plan approval, SSH pinning
|
||||
and artifact integrity checks.
|
||||
|
||||
## 0.1.0-rc.1
|
||||
|
||||
- Introduce durable Broker HTTP/MCP, Docker gateway, Callee Runner, Prism/Beads
|
||||
projection, Human approval and isolated runtime profiles.
|
||||
@@ -0,0 +1,117 @@
|
||||
# Архив исходника Broker
|
||||
|
||||
Историческая копия до очистки репозитория 2026-10-16. Источник: `mallexxx/execution-broker`, commit `ae92c15d31c8396789765cebf82f7971d6f4cb12`, файл `README.md`. Старые статусы и deployment pins ниже не заменяют актуальную [[personal/projects/balda/execution-broker|эксплуатационную карточку]].
|
||||
|
||||
---
|
||||
|
||||
# Execution Broker
|
||||
|
||||
Execution Broker runs Balda tasks in disposable Docker workers or Virfield macOS
|
||||
VMs. Callee executes the workflow; a separate Prism adapter owns the durable
|
||||
Beads Story. Balda receives progress, asks the user for approval, and reports the
|
||||
result. Provider credentials remain outside plugin packages and VM images.
|
||||
|
||||
**Release candidate:** use the [support matrix](docs/support.md) to select a
|
||||
verified deployment. A Docker smoke test does not qualify Docker coding agents,
|
||||
and installation does not configure access to managed production hosts.
|
||||
|
||||
The GitHub `main` branch is the source of truth for product code and public
|
||||
documentation. The plugin version is defined in
|
||||
[`plugins/execution-broker/plugin.json`](plugins/execution-broker/plugin.json);
|
||||
[`docs/support.md`](docs/support.md) records tested deployment boundaries.
|
||||
Candidate source is not a published release until the release procedure passes.
|
||||
|
||||
Runs can consume retained artifacts from an earlier run in the same session using
|
||||
the exact source run, name and SHA256. Broker holds the bytes during the dependency;
|
||||
Runner verifies them before execution and exposes `$BROKER_INPUT_DIR`. See the
|
||||
[input contract](docs/protocol.md#artifact-inputs) for limits and the required
|
||||
worker protocol. Artifact inputs do not install a toolchain or change its policy.
|
||||
|
||||
[Immutable repository caching](docs/repository-cache.md) and [safe environment
|
||||
changes](docs/environment-changes.md) use operator policy and execution/v3.
|
||||
|
||||
An agent can select an operator-approved [tooling profile](docs/tooling.md) for
|
||||
Make/C/C++, Python, Node, JVM or other stacks. Recipes supply checked archives,
|
||||
reviewed Linux image packages or preinstalled guest tools and report their actual
|
||||
versions. Adding a tool does not require a Broker code change. A missing profile
|
||||
or failed prerequisite stops before implementation. Dependencies discovered later
|
||||
use an explicitly authored safe checkpoint and approved environment continuation.
|
||||
|
||||
## Install the Balda plugin
|
||||
|
||||
The repository and binary archives include an Agent Plugins package and a Balda
|
||||
marketplace index. After connecting Broker and Prism, open Balda's Backoffice
|
||||
**Operations → Plugins → Marketplaces**, add the repository or extracted archive
|
||||
directory, and refresh it. Review the exact `execution-broker` revision, then
|
||||
select **Install plugin**.
|
||||
|
||||
In a new conversation, use `/execute <task>` or
|
||||
`/skill execution-broker:execution-broker <task>`. A chat transport may require
|
||||
mentioning the bot. Existing sessions keep their old capability snapshot until
|
||||
reset. See [Balda setup](integrations/balda/README.md) for service credentials,
|
||||
event routing, upgrades and removal.
|
||||
|
||||
## Start a smoke deployment
|
||||
|
||||
Requirements: Docker Engine with Compose, Python 3.12+, Git, and a private Docker
|
||||
network shared with Balda. Source builds also require Go 1.26.9+; the test suite
|
||||
requires Node.js 22+. The example
|
||||
builds Linux ARM64 workers; set `TARGETARCH=amd64` and pass `amd64` to the two
|
||||
helper scripts on an AMD64 Docker host.
|
||||
|
||||
```sh
|
||||
python3 -m pip install -r requirements-tools.txt
|
||||
make check build linux
|
||||
GOOS=linux GOARCH=arm64 CGO_ENABLED=0 \
|
||||
./scripts/build-callee.sh "$PWD/.build/linux/broker-callee"
|
||||
./scripts/fetch-beads.sh arm64
|
||||
python3 scripts/worker-config.py
|
||||
docker build -f deploy/Dockerfile.runner -t execution-broker-worker:local .
|
||||
python3 scripts/init-deployment.py --directory ./state/deploy \
|
||||
--worker-image "$(docker image inspect execution-broker-worker:local --format '{{.Id}}')"
|
||||
export BROKER_CONFIG_DIR="$PWD/state/deploy"
|
||||
export BALDA_NETWORK=balda_default # replace with the existing Balda network
|
||||
# Use docker-compose instead if your installation provides the standalone CLI.
|
||||
docker compose -p execution-broker -f deploy/compose.yaml up -d --build
|
||||
.build/bin/executionctl --token-file "$BROKER_CONFIG_DIR/broker.token" --operation tools
|
||||
```
|
||||
|
||||
For a binary installation, extract the **Linux** archive matching Docker's CPU,
|
||||
run `scripts/prepare-binary-deployment.sh`, and continue at `fetch-beads.sh`.
|
||||
Docker always requires Linux binaries, including when its host runs macOS.
|
||||
Use a native `executionctl` from your host archive for the final diagnostic
|
||||
command; `.build/bin/executionctl` is the source-build output.
|
||||
The macOS archive provides native operator and VM binaries.
|
||||
|
||||
The initial configuration allows only the deterministic `smoke/story` fixture
|
||||
and its pinned repository. Configure reviewed profiles and repository allowlists
|
||||
before enabling coding. [Deployment profiles](docs/deployment.md) explain macOS,
|
||||
provider credentials and signed managed-host plans.
|
||||
|
||||
## Components
|
||||
|
||||
| Component | Responsibility |
|
||||
|---|---|
|
||||
| `execution-broker` | Admission, leases, event journal, Human responses, artifacts and notification outbox |
|
||||
| `execution-runner` | Worker-local checkout, Callee execution, cancellation and artifact collection |
|
||||
| `execution-worker` | Restricted Docker gateway; the only service with the Docker socket |
|
||||
| `execution-host-runner` | Fixed, signed host operations and bounded rollback |
|
||||
| `execution-prism-adapter` | Story/Task lifecycle and idempotent Beads projection |
|
||||
| `execution-credentials` | Encrypted persistent OAuth identity, issuance and checkpointing |
|
||||
| `executionctl` | MCP/HTTP diagnostics and operator plan signing |
|
||||
|
||||
Broker does not interpret workflow graphs or write Beads. Requests cannot supply
|
||||
host paths, shell commands or Docker options. Worker configuration, workflow
|
||||
versions, repository access and agent profiles belong to the operator.
|
||||
|
||||
## Documentation
|
||||
|
||||
- [Installation and profiles](docs/deployment.md), [Balda integration](integrations/balda/README.md)
|
||||
- [Protocol and guarantees](docs/protocol.md), [supported configurations](docs/support.md)
|
||||
- [Operations](docs/operations.md), [coordinated recovery](docs/recovery.md)
|
||||
- [Executor profiles and enrollment](docs/executors.md)
|
||||
- [Credential lifecycle](docs/credentials.md), [security](docs/security.md)
|
||||
- [Testing](docs/testing.md), [release procedure](docs/releasing.md)
|
||||
|
||||
MIT licensed. Dependency licenses and inventories are included in every binary
|
||||
archive; see [third-party notices](THIRD_PARTY_NOTICES.md).
|
||||
+151
@@ -0,0 +1,151 @@
|
||||
# Архив исходника Broker
|
||||
|
||||
Историческая копия до очистки репозитория 2026-10-16. Источник: `mallexxx/execution-broker`, commit `ae92c15d31c8396789765cebf82f7971d6f4cb12`, файл `docs/deployment.md`. Старые статусы и deployment pins ниже не заменяют актуальную [[personal/projects/balda/execution-broker|эксплуатационную карточку]].
|
||||
|
||||
---
|
||||
|
||||
# Deployment profiles
|
||||
|
||||
Keep configuration and token files outside Git. `scripts/init-deployment.py` creates a fresh smoke-only configuration and refuses to overwrite it. The Compose stack owns separate Broker, projection, and Beads volumes. Beads v1.3.0 runs embedded Dolt inside the Prism sidecar; it is not an external shared Dolt server. Do not attach another Beads writer to this workspace.
|
||||
|
||||
## Linux coding workers
|
||||
|
||||
Build an operator-reviewed image containing `execution-runner`, the pinned `broker-callee`, Git, provider CLI binaries and the read-only workflow pack. Use `deploy/Dockerfile.runner` as the minimal contract. The image must have a `/state` directory owned by UID/GID 65532 and `/etc/execution-runner/config.json`. It must run `sleep infinity` as its entry command so the gateway can invoke RPC. Do not bake live credentials into image layers.
|
||||
|
||||
Allowlist `prism/story` on both Broker and Runner with the exact whole-pack digest in `workflows.version` and an agent profile such as `codex-default`. Runner's `agent_environment` maps profile IDs to explicitly injected environment values. It does not inherit the host process environment. Use a worker profile's `credentials_file` for an operator-issued lease. Broker pins the lease at admission and delivers it only over the private worker transport. Runner checks repository/profile and expiry, uses the Git token only in the fetch process environment (never argv, Git config files or Callee's environment), and exports only the selected provider environment. No host home directory is copied. Actual provider login and token issuance belong to the operator/secret manager; private repository and real provider acceptance require those credentials.
|
||||
|
||||
The bundled Prism pack uses fresh Codex sessions for specification, design and breakdown, followed by up to three implement, verify and review cycles in one disposable worker. A deterministic router requires the exact approval answer `approved` before mutation. The final quality gate records failure evidence after the repair limit. Callee remains responsible for workflow policy. Agent profile selection is an allowlist/environment boundary; this pack itself pins its provider to Codex.
|
||||
|
||||
Docker images must be configured as `registry/image@sha256:…` or a local immutable `sha256:…` image ID. Broker has no socket. The separate gateway is a trusted administrative component with the Docker socket and only creates fixed, restricted workers: no host binds, non-root, read-only rootfs, cap-drop ALL, no-new-privileges, 256 PIDs, 4 GiB RAM, two CPUs. Docker workers are not a VM security boundary for hostile multi-tenant code. Configure Docker-host network/firewall policy to restrict worker egress; default bridge networking alone does not isolate the host LAN.
|
||||
|
||||
Broker persists the selected Linux image in each private acquisition handle;
|
||||
Gateway rejects a missing or different pin before allocating a container.
|
||||
Upgrade Broker and Gateway together, align their static image IDs and rebuild
|
||||
the workflow pack before admitting new runs. Dynamic builds persist the resolved
|
||||
immutable image ID as well. Docker socket paths and daemon groups are rejected
|
||||
in worker profiles; container builds require an independently isolated builder.
|
||||
|
||||
### Nested Linux sandbox
|
||||
|
||||
Some provider CLIs need user/mount namespaces inside their worker. For those
|
||||
profiles the gateway accepts the operator setting `"container_sandbox": "nested-v1"`.
|
||||
Requests cannot select or modify it. The gateway image bundles the corresponding
|
||||
seccomp policy under `/opt/execution-broker/docker-sandbox/nested-v1/`.
|
||||
The Docker host must first load the exact
|
||||
[`apparmor.profile`](../deploy/docker-sandbox/nested-v1/apparmor.profile) as
|
||||
`execution-broker-codex-sandbox-v1`, using its AppArmor service or the distribution's
|
||||
`apparmor_parser`. Install it as a separate profile; preserve `docker-default` and
|
||||
the host's unprivileged-user-namespace restrictions. Missing policy files or a
|
||||
missing AppArmor profile cause worker creation to fail. An empty setting retains
|
||||
Docker's defaults.
|
||||
|
||||
The versioned policy derives from Moby's default profiles and enables namespace,
|
||||
mount and pivot-root operations required by bubblewrap. Workers retain non-root
|
||||
UID, empty outer capabilities, no-new-privileges, read-only root, private state,
|
||||
Docker's masked/read-only system paths, AF_ALG denial and resource limits. Never
|
||||
replace either policy with `unconfined`, disable system-path guards, or enable a
|
||||
provider's full-access mode to make a check pass. Codex can use its own `/proc`
|
||||
mount fallback while retaining Docker's inherited proc guards and a separate PID
|
||||
namespace. Qualify the exact kernel, Docker, provider and both security profiles
|
||||
with a real Story and attempted read-only writes before enabling new admissions.
|
||||
Do not remove a loaded profile while a worker still uses it.
|
||||
|
||||
## Virfield macOS workers
|
||||
|
||||
Virfield v2 must have a ready, approved template and independent lease cleanup.
|
||||
|
||||
The operator's macos27 path passed production reacceptance on 2026-10-16 with
|
||||
installed Virfield `8bfff9f`, the corrected Runner package and reviewer-deny
|
||||
workflow. The [support matrix](support.md#runtime-matrix) records its tests,
|
||||
artifacts, credential settlement and cleanup. Qualify each different deployment
|
||||
with its own fresh worker. A ready template is not fresh worker evidence.
|
||||
|
||||
Configure a Broker worker:
|
||||
|
||||
```json
|
||||
{
|
||||
"runtime_class": "macos-vm",
|
||||
"endpoint": "http://host.docker.internal:7780",
|
||||
"token_file": "/run/secrets/virfield_token",
|
||||
"template": "macos27",
|
||||
"ssh_route": "tunnel",
|
||||
"ssh_tunnel_host": "host.docker.internal",
|
||||
"bootstrap_file": "/opt/execution-broker/assets/macos-bootstrap.tar.gz",
|
||||
"limit": 2
|
||||
}
|
||||
```
|
||||
|
||||
Mount the API token and bootstrap archive read-only into the Broker container. With Colima, use the lease-bound Virfield tunnel and the explicit Docker host alias shown above: the guest's IP may be unreachable from Docker. `ssh_tunnel_host` rewrites only a numeric loopback address returned by the authenticated API, preserving its lease-specific port and guest SSH host-key pin. The host alias must resolve and reach host loopback listeners from the Broker network; no guest or host listener is exposed on all interfaces. For a native Broker on the Mac, omit `ssh_tunnel_host`. Use `ssh_route: direct` only when the Docker network can reach the guest's IP/SSH port, and omit the tunnel override. Do not disable SSH pinning or MCP DNS-rebinding protections to work around routing. Broker uses the authenticated REST API.
|
||||
|
||||
A ready image can preinstall Runner/Callee/config, or an operator can package a credential-free bootstrap:
|
||||
|
||||
```sh
|
||||
make build
|
||||
./scripts/build-callee.sh "$PWD/.build/bin/broker-callee"
|
||||
python3 scripts/build-macos-bootstrap.py \
|
||||
--runner .build/bin/execution-runner --callee .build/bin/broker-callee \
|
||||
--repository https://github.com/baldaworks/callee.git \
|
||||
--output .build/macos-bootstrap.tar.gz
|
||||
```
|
||||
|
||||
Bootstrap uses the per-lease SSH identity and pinned guest host key. Its digest is persisted before acquisition. Installation runs only inside the disposable leased VM and requires that guest account's noninteractive sudo. A failed bootstrap retains its lease for cleanup; it never modifies a golden image. The bootstrap shown contains only the smoke workflow. Install authorized provider binaries/credentials and a reviewed Runner config before enabling coding profiles. VM checkout is on guest-local disk, never a mutable host shared folder.
|
||||
|
||||
## Managed hosts
|
||||
|
||||
The proposed host permissions, SSH tooling, VM cache, iPad deployment and Balda
|
||||
integrations are tracked in [the work plan](work-plan.md).
|
||||
|
||||
Install `execution-host-runner` under a dedicated service identity. Keep its configuration/plan writable only by the administrator. A plan lists a fixed host, identity and ordered operation objects with absolute `preflight`, `apply`, `verify`, `rollback` argv. Requests cannot supply new commands. Preflight commands must actually be read-only: this is an operator obligation, not something the executable can infer.
|
||||
|
||||
Generate an operator signing key outside Balda/Broker/host-runner:
|
||||
|
||||
```sh
|
||||
executionctl -operation keygen -approval-key /secure/operator/approval.key
|
||||
```
|
||||
|
||||
Configure the host profile's `host_id` and `approval_key_file` (the `.pub` file) on Broker and host-runner. Start host-runner with `--config`, `--plan`, and `--profile`. Use TLS/authenticated private networking for a remote host; the built-in plain HTTP listener is intended behind that transport. `deploy/execution-host-runner.service` and `.plist` are templates and are not installed automatically.
|
||||
|
||||
Call `execution_remote_plan`, review the returned exact Plan and digest, save the **plan object** to a file, and sign it only after human approval:
|
||||
|
||||
```sh
|
||||
executionctl -operation approve-plan -approval-key /secure/operator/approval.key \
|
||||
-plan-file reviewed-plan.json -story-id eb-example -chat-id CHAT -topic-id TOPIC \
|
||||
-ttl-seconds 900 -out approval.json
|
||||
```
|
||||
|
||||
Pass `approval.json`'s capability to the matching run with a shorter run TTL (for example 600 seconds). A capability binds nonce, host, OS identity, ordered operations, plan SHA, Story, chat/topic and expiry; it cannot authorize a second run. The signer is intentionally unavailable through MCP. Automatic capability minting from a model-provided `approved` string would erase the human authorization boundary.
|
||||
|
||||
All operations preflight before the first mutation. Verify failure rolls back started operations in reverse order, within the original deadline. An interrupted/expired rollback is reported as requiring manual action. Service restart marks uncertain mutations `remote_outcome_unknown`; it never automatically replays them. Only one mutation plan runs on a host at a time.
|
||||
|
||||
## Credential lease file
|
||||
|
||||
Set `credentials_file` to an absolute private path on the Broker for the selected worker profile. Mount it as a read-only secret. A run request cannot supply this path or secret material. Example structure (replace the expiry with a time covering the intended run, at most 24 hours from now):
|
||||
|
||||
```json
|
||||
{
|
||||
"repository": "https://github.com/example/project.git",
|
||||
"agent_profile": "codex-default",
|
||||
"expires_at": "REPLACE_WITH_RFC3339_EXPIRY",
|
||||
"environment": {"PROVIDER_CREDENTIAL_VARIABLE": "OPERATOR_ISSUED_SCOPED_VALUE"},
|
||||
"git_token": "OPTIONAL_REPOSITORY_SCOPED_TOKEN"
|
||||
}
|
||||
```
|
||||
|
||||
The provider's actual accepted credential variable comes from its reviewed image/profile. Git HTTPS authentication uses Basic `x-access-token:TOKEN`; configure a compatible repository provider/token. Redirects are disabled. Omit `git_token` for public repositories. Runtime requests cannot choose additional credential files or profiles. The metadata does not issue or cryptographically attenuate a provider token: the issuer must enforce the same repository, permission and time scope. Broker/Runner validate metadata and execution lifetime; the external provider must revoke/expire the credential as promised.
|
||||
|
||||
Known provider/Git secret values are redacted from event summaries, Human questions, result text and exported Callee logs. Source patches and arbitrary build archives remain reviewable outputs, not a guaranteed secret-sanitization boundary. Private lease material is erased from active state after confirmed cleanup; encrypted backups and SQLite storage can still retain historical bytes and need normal secure retention controls.
|
||||
|
||||
|
||||
## Workflow completion policy
|
||||
|
||||
`prism/story` runs fresh sequential roles. Specify/design/breakdown/reviewer use the Codex read-only sandbox argument. The operator must verify that the installed provider bridge supports and enforces that argument before enabling the profile. Docker coding requires separate provider sandbox acceptance; see [Codex runtime](../integrations/codex/README.md). The final Script quality gate requires strict JSON reports from both reviewer and verifier, status `passed`, no unresolved findings, and at least one executed verification command. Commands must exit zero unless the check declares an `expected_exit_code`; expected nonzero exits require observed behavior evidence. ACP can concatenate assistant progress text before the final JSON object. The gate accepts that preamble, preserves the complete raw output in `quality.json`, and validates the first JSON object without skipping an earlier report. Competing objects, trailing content, failed, malformed or unexecuted reports fail the workflow. This validates reports; it does not independently prove that a model told the truth about evidence. Human/operator review of artifacts remains necessary for release approval.
|
||||
|
||||
The adapter currently rejects all `Parallel` nodes before starting work because it has no per-writer worktree allocator. This safely prevents shared-checkout concurrent mutation; parallel read-only roles and worktree fan-out are not yet advertised as supported. Parallel Stories still use separate workers and remain supported.
|
||||
|
||||
## Persistent macOS OAuth
|
||||
|
||||
For reusable OAuth, configure the separate host issuer described in [credentials](credentials.md) and include `compose.credentials.yaml`. A `macos-codex` profile uses `credential_endpoint` plus `credential_token_file` instead of a static `credentials_file`. The runner receives `.codex/auth.json` in its private HOME; successful and failed terminal workflows checkpoint it before release. Neither golden image nor bootstrap contains login credentials. A `prism/story` run has a design approval gate but no per-VM login gate. The CLI enrollment workflow `oauth/story` is reserved for first login/re-enrollment.
|
||||
|
||||
## Coordinated backup and restore
|
||||
|
||||
Use the [stack recovery runbook](recovery.md) for Balda, Broker, Prism, Beads, gateway, configuration and vault snapshots. A Broker-only SQLite backup does not cover this stack.
|
||||
+156
@@ -0,0 +1,156 @@
|
||||
# Архив исходника Broker
|
||||
|
||||
Историческая копия до очистки репозитория 2026-10-16. Источник: `mallexxx/execution-broker`, commit `ae92c15d31c8396789765cebf82f7971d6f4cb12`, файл `docs/hermes-workflow-gap-analysis.md`. Старые статусы и deployment pins ниже не заменяют актуальную [[personal/projects/balda/execution-broker|эксплуатационную карточку]].
|
||||
|
||||
---
|
||||
|
||||
# Hermes workflow inventory and migration gaps
|
||||
|
||||
Status: analysis and acceptance plan, not a claim that these flows work in the
|
||||
current deployment. Scope is the Balda + Broker + Prism system, not Broker alone.
|
||||
|
||||
Ownership correction: this historical scenario inventory is not a Broker feature
|
||||
checklist or release gate. Device flows, search, Obsidian access and personal/PR
|
||||
workflows belong to their local agent/project configuration. Earlier proposed
|
||||
scoped MCP/Obsidian grants were design suggestions, not an implemented mechanism
|
||||
or an approved user policy. Broker owns only the generic execution contracts
|
||||
listed in its [work plan](work-plan.md).
|
||||
|
||||
The Zulip SQLite export is used only to study historical work; runtime Zulip
|
||||
and Asana access are explicitly outside this plan.
|
||||
|
||||
## Evidence and limits
|
||||
|
||||
- The Obsidian migration notes `personal/plans/zulip-decommission.md` and
|
||||
`personal/tech/zulip-message-history.md` locate the read-only export at
|
||||
`~/Documents/zulip_message_history.sqlite`. A direct query on 2026-10-04 found
|
||||
134,940 messages, 197 topics, 11 streams, dated 2026-04-29 through
|
||||
2026-09-29. The older note says 134,734 messages; use the queried count for
|
||||
this snapshot. The export omits attachments and later Mattermost activity.
|
||||
- Examples below are message IDs in that SQLite export. Tool names in message
|
||||
text and counts of mentions are historical conversation evidence, not trusted
|
||||
execution telemetry. Bot-forwarded Discord material can duplicate events.
|
||||
All 197 topic titles were reviewed for scenario families; detailed action
|
||||
traces were sampled where they changed the security or acceptance design.
|
||||
- The current Virfield v2 boundary is documented in Obsidian
|
||||
`work/wiki/apple-browsers/vm-ui-testing.md`: Virfield leases the VM; Runner
|
||||
owns build/test and artifact collection. Its host MCP does not export guest
|
||||
Peekaboo actions. The older `vm_prepare_session` and `run_tests` references
|
||||
in May conversations describe a previous integration, not a v2 API contract.
|
||||
|
||||
The archive contains logged invocation lines for `terminal`/`process`, VM
|
||||
`vm_ssh_exec`, Peekaboo image/click/type, DuckDuckGo web search, and Obsidian
|
||||
read/patch/write. For scale, a text scan finds Obsidian read calls in 1,068
|
||||
messages across 94 topics, patch calls in 1,092 messages across 72 topics,
|
||||
VM SSH calls in 123 messages across 17 topics, and Peekaboo actions in 25
|
||||
messages across six topics. These are messages containing call text, not
|
||||
verified successful executions. `run_tests`, AX queries, crash retrieval,
|
||||
browser control, Spotify, `session_search` and `cronjob` appear in advertised
|
||||
tool inventories or proposed workflows, but this export alone does not prove
|
||||
they ran. `ScheduleWakeup` does appear in logged calls.
|
||||
|
||||
## Workflow coverage
|
||||
|
||||
| Workflow seen in Hermes | Historical evidence | Current or already planned | Remaining requirement |
|
||||
| --- | --- | --- | --- |
|
||||
| Code task to PR: task context, isolated checkout, agent edit, test, review, CI, task update | `executor` topics for PRs #4541, #4548 and #4559; historical Asana and GitHub/CI discussion | Broker has pinned workflows, sequential roles, artifact collection and a local reviewed-head publication path. The [work plan](work-plan.md) includes PR review. | Configure GitHub access, exact PR-head and CI gates, idempotent publication, Story status and bounded notifications. A local Broker publication is not an upstream PR or merge. |
|
||||
| UI bug investigation and regression | Messages 8093, 8250-8251 require reproduce → failing UI test → fix → passing test; the last passing Virfield run must have a recording and macOS UI CI must be green. Peekaboo and VM tools appear in executor traces. | A disposable Virfield v2 worker and an XCTest fixture returning `.xcresult` are integration-tested. | Guest-bound Peekaboo/AX/recording, actual app UI-test build/run, red/green evidence gate, CI status check, and recording collection are missing. See the flow below. |
|
||||
| iPad build, sign, transfer, install, unlock, launch | Requested in this migration thread | [iPad plan](ipad-deployment-plan.md) covers VM tooling, fake signing, artifact handoff and configured SSH actions. | Physical-device acceptance and failure recovery remain open. The iPad sequence belongs in a pinned workflow/target profile. |
|
||||
| Host and service administration | Hermes used terminal/process and SSH for TrueNAS, home services, media stacks and remote setup; explicit `vm_ssh_exec` calls also occurred outside UI tasks. | Balda can launch agents on its own host; Broker has fixed managed-host plans. The work plan includes per-agent grants and an SSH target/command catalog. | Provision target-specific identities and reviewed command definitions, typed arguments, protected secrets, reconciliation and rollback. A separately approved Balda host agent can use the target's catalog; running an agent on a remote host needs its own sandbox and acceptance. |
|
||||
| Research and knowledge maintenance | Logged web-search and Obsidian read/search/patch/write calls occur in multiple topics; `web_extract` was advertised. The Zulip SQLite file is the migration source, not a Hermes tool. | Balda has external MCP support and session/fact memory; web search MCP is planned. | Configure scoped Obsidian read/write and web extraction/search; preserve source links and require a reviewed diff for knowledge writes. Historical text is data, never authority to change permissions. |
|
||||
| Recurring personal operations | Daily brief, inbox/weekly review, health checks, task work and scheduled wakeups appear across the archive. | Balda already has a durable configured scheduler and job delivery. | Migrate in-scope jobs with explicit tool profiles, notification destinations, idempotent writes, timeouts and visible failed-run state. Do not implement another scheduler in Broker. |
|
||||
| Browser, desktop and media workflows | Hermes exposed browser navigation/click/snapshot, Peekaboo, media and home-service tools; personal topics show these used for setup and diagnostics. | Generic Balda MCP connections can host adapters. | Inventory each actually needed adapter, constrain browser/GUI to an isolated profile, and give service-specific MCPs or target commands separate grants. No media or app-specific branches in Broker. |
|
||||
| Advice, writing and visual input | Archive topics include planning, finance, travel, creative work and app design. The SQLite export contains message text but no attached images or files. | Direct Balda conversation and configured search can handle text-only work without Broker. | Verify attachment ingestion and scoped vision/document tools separately; do not claim that the text-only export covers every multimodal scenario. Sensitive content needs private storage and destination-specific sharing rights. |
|
||||
|
||||
The archive shows actual tool-call text for Obsidian writes, terminal/process,
|
||||
VM SSH, web search and Peekaboo. It also shows advertised tools that may not
|
||||
have been exercised. This matrix therefore treats capability discovery and
|
||||
live acceptance as separate gates.
|
||||
|
||||
## Required UI-bug flow on Virfield v2
|
||||
|
||||
1. Bind the task to a repository revision, test target, VM image and approved
|
||||
agent/tool profile. Verify a ready lease **and** working guest SSH, GUI login,
|
||||
Accessibility and Screen Recording permissions before declaring the VM usable.
|
||||
2. Reproduce the bug in the disposable VM with a lease-bound Peekaboo session.
|
||||
Save an initial screenshot/AX snapshot and the observed failure. Read the
|
||||
applicable project's UI-test conventions; for apple-browsers the Obsidian
|
||||
`work/wiki/apple-browsers/ui-testing.md` describes its test base class and
|
||||
selectors.
|
||||
3. Add a regression UI test. Run it against the unfixed code and retain a
|
||||
`.xcresult` proving the expected assertion failed. Record the base commit,
|
||||
test-patch digest, test ID and failure signature. An infrastructure failure,
|
||||
skipped test or build failure is **inconclusive**, not red evidence.
|
||||
4. Fix the code. Re-run the same test against the fixed revision, then run the
|
||||
relevant UI suite and independent verifier. Check the corresponding CI job
|
||||
for the exact PR head: skipped or pending is not green.
|
||||
5. Record the **last passing local VM run**. Collect video, `.xcresult`, logs,
|
||||
screenshots/AX diagnostics and the revision/test metadata before releasing
|
||||
the lease. Store SHA256, size, privacy class and retention for each artifact;
|
||||
attach only an authorized link to the Story/PR. Verify `cleanup=done`.
|
||||
|
||||
The v2 implementation should expose typed session actions through a guest
|
||||
Runner or lease-bound sidecar: `ui.snapshot`, `ui.ax_query`, `ui.click`,
|
||||
`ui.type`, `ui.hotkey`, `ui.record_start/stop`, `test.run/status/results`, and
|
||||
`diagnostics.crashes/logs`. Broker routes and audits generic actions; the
|
||||
project workflow defines schemes, tests and acceptance rules. An agent may run
|
||||
normal build commands inside its permitted VM checkout. Balda must not receive
|
||||
raw SSH access to a VM or a general host shell through these actions. A
|
||||
headless/locked VM, missing TCC grant, video collection failure, or lost SSH
|
||||
must fail the evidence gate with a specific reason. Video may exceed the
|
||||
current 512 MiB artifact limit, so bounded streaming/chunking and retention
|
||||
must be specified and tested. Give the UI sidecar only the needed TCC grants
|
||||
and limit actions to the selected app/window. Prove an OS-level boundary keeps
|
||||
the coding agent's shell from invoking the sidecar or the same GUI privileges;
|
||||
if a shared macOS GUI session prevents this, use a separate test VM and the
|
||||
digest-pinned artifact handoff. The disposable VM has no personal documents,
|
||||
host keys or ambient service credentials, and its egress is scoped to the task.
|
||||
|
||||
## Security and orchestration gaps beyond the existing plan
|
||||
|
||||
1. **Verifiable owner approval.** Balda can identify a Mattermost sender and
|
||||
review a provider permission request, but Broker currently trusts a single
|
||||
bearer principal and a `chat_id` supplied by Balda. Mint a short-lived,
|
||||
audience-bound capability for the exact owner, session, agent, action IDs,
|
||||
repository/VM/SSH target, artifact digest and expiry. A separate trusted
|
||||
component must attest approval; neither the model nor the ordinary Broker
|
||||
bearer may mint or widen it. Revoke/expire it and audit its use.
|
||||
2. **Tool rights that survive untrusted context.** Archived executor messages
|
||||
12672 and 12706 describe Asana task text injected before a recording pass;
|
||||
a broad data-access rule then blocked all Virfield MCP calls, leaving a VM
|
||||
and a repeated spawn loop. Treat task text, web pages, notes, screenshots and
|
||||
logs as untrusted data. Authorize each tool and destination independently,
|
||||
rather than disabling a whole workflow after reading context. Bound retries
|
||||
with a no-progress cap; cleanup uses a trusted control path even if the
|
||||
agent's data-plane permission is denied.
|
||||
3. **External writes and workflow changes.** The archive records comments to
|
||||
a historical task service despite notes forbidding them (message 14914).
|
||||
Permission to read task context does not grant write/comment/publish rights.
|
||||
Separate grants for PR comments, Mattermost posts, Obsidian edits and
|
||||
config/skill/workflow changes; show exact target and diff, honor task-level
|
||||
exclusions, pin revisions, and deduplicate after retries.
|
||||
4. **Evidence and resource lifecycle.** A passing model report does not prove a
|
||||
test or recording happened. Accept machine-checked test counts, exact run
|
||||
IDs and artifact hashes; keep failure artifacts; reconcile remote unknown
|
||||
outcomes; collect before VM deletion; distinguish terminal result,
|
||||
publication, notification and cleanup. Message 13181 documents a booted VM
|
||||
whose SSH never became reachable, so VM readiness needs an end-to-end probe.
|
||||
5. **Coverage inventory.** Classify every historical topic by scenario family,
|
||||
required tools, data sensitivity and external side effects. The 197-topic
|
||||
text export supports the initial matrix, but it cannot prove workflows that
|
||||
depended on missing attachments or later Mattermost messages. Keep an
|
||||
operator-visible capability registry with status `available`, `planned`,
|
||||
`blocked` or `unverified` and a live acceptance run for every enabled family.
|
||||
|
||||
## Release gates
|
||||
|
||||
- One UI bug passes the complete red/green/recorded/CI flow on a disposable VM;
|
||||
a skipped test, stale PR head, missing video, denied UI grant and guest SSH
|
||||
failure each produce a truthful blocked result and confirmed cleanup.
|
||||
- One authorized host action, one rejected injected argument, one host-key
|
||||
mismatch and one ambiguous SSH disconnect show the configured command path
|
||||
without giving Balda a raw shell.
|
||||
- One each of web search/extraction, scoped Obsidian write,
|
||||
scheduled task, PR review and device release is exercised with its actual
|
||||
destination and owner-approved permission profile. Unconfigured adapters
|
||||
remain explicitly unsupported until their live acceptance passes.
|
||||
+34
@@ -0,0 +1,34 @@
|
||||
# Архив исходника Broker
|
||||
|
||||
Историческая копия до очистки репозитория 2026-10-16. Источник: `mallexxx/execution-broker`, commit `ae92c15d31c8396789765cebf82f7971d6f4cb12`, файл `docs/ipad-deployment-plan.md`. Старые статусы и deployment pins ниже не заменяют актуальную [[personal/projects/balda/execution-broker|эксплуатационную карточку]].
|
||||
|
||||
---
|
||||
|
||||
# Local-agent iPad deployment scenario (reference)
|
||||
|
||||
Ownership: a local agent/project workflow, outside Execution Broker's release
|
||||
scope. This repository contains this reference note, not an iPad implementation.
|
||||
No iPad-specific actions or application logic belong in Broker.
|
||||
|
||||
The existing macOS VM build of the iOS 9.3 armv7 app does not prove installation
|
||||
or launch on a physical device. The scenario owns its remaining work:
|
||||
|
||||
1. Build the app and sign it with the project's selected signing tool; verify
|
||||
the signature and bundle identifier.
|
||||
2. Obtain the verified build artifact. If Broker ran the build, use its generic
|
||||
authenticated artifact API and check the artifact digest.
|
||||
3. Use the local agent's configured SSH/device tools to upload, install with
|
||||
the device's available installer, verify the installed bundle, unlock and
|
||||
launch. Select compatibility flags from that device's actual requirements.
|
||||
4. Check each outcome and reconcile the device after a disconnect before
|
||||
retrying a mutation. Report build and deployment outcomes separately.
|
||||
|
||||
Connection setup, device credentials, project scripts and approvals belong to
|
||||
that scenario's existing local-agent configuration. This note introduces no
|
||||
new global permission policy and does not require the scenario to use Broker
|
||||
for SSH or installation.
|
||||
|
||||
Broker can supply reusable disposable execution, tool provisioning and artifact
|
||||
transport. Any future generic remote-action API is a separately scoped feature
|
||||
in the [Broker work plan](work-plan.md), not a prerequisite for this scenario.
|
||||
Physical iPad acceptance does not block a Broker release.
|
||||
@@ -0,0 +1,148 @@
|
||||
# Архив исходника Broker
|
||||
|
||||
Историческая копия до очистки репозитория 2026-10-16. Источник: `mallexxx/execution-broker`, commit `ae92c15d31c8396789765cebf82f7971d6f4cb12`, файл `docs/recovery.md`. Старые статусы и deployment pins ниже не заменяют актуальную [[personal/projects/balda/execution-broker|эксплуатационную карточку]].
|
||||
|
||||
---
|
||||
|
||||
# Stack recovery
|
||||
|
||||
`scripts/stack-recovery.py` takes a coordinated **offline** snapshot, verifies it,
|
||||
and restores into a new inert directory. Python 3.12+ and Docker CLI are required.
|
||||
The plan is an operator-owned file, never model/tool input. Schedule a maintenance
|
||||
window and finish all runs/cleanup before backup. Balda stops first; Broker,
|
||||
Prism and gateway stop next. Only previously running containers restart, in
|
||||
reverse order, even when capture fails or receives SIGINT/SIGTERM. A forced
|
||||
shutdown, live Docker worker, credential owner, pending Broker run, unsafe tar
|
||||
member or corrupt SQLite database prevents publication of `manifest.json`.
|
||||
|
||||
Example plan (replace every path/container with the installed deployment):
|
||||
|
||||
```json
|
||||
{
|
||||
"containers": ["balda", "broker", "prism", "worker"],
|
||||
"vault_dir": "/private/operator/credentials",
|
||||
"credential_status_command": ["/opt/execution-broker/execution-credentials", "--operation", "status", "--vault", "/private/operator/credentials"],
|
||||
"sources": [
|
||||
{"name": "balda-state", "container": "balda", "path": "/state"},
|
||||
{"name": "broker-state", "container": "broker", "path": "/state"},
|
||||
{"name": "prism-state", "container": "prism", "path": "/state"},
|
||||
{"name": "beads-state", "container": "prism", "path": "/beads"},
|
||||
{"name": "worker-state", "container": "worker", "path": "/state"},
|
||||
{"name": "broker-config", "path": "/private/operator/broker-deployment"},
|
||||
{"name": "balda-config", "path": "/private/operator/balda/config"},
|
||||
{"name": "balda-environment", "path": "/private/operator/balda/.env"},
|
||||
{"name": "balda-compose", "path": "/private/operator/balda/compose.yaml"},
|
||||
{"name": "balda-secrets", "path": "/private/operator/balda/secrets"},
|
||||
{"name": "credential-vault", "path": "/private/operator/credentials"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Include service definitions, provider configuration, bootstrap assets and all
|
||||
secret files referenced by deployment. Capture Virfield configuration, golden
|
||||
images and signing-key escrow using their respective operator procedures; this
|
||||
script does not create a disk/image snapshot of the VM manager. The credential
|
||||
status command must access the same vault that `vault_dir` locks. Only this
|
||||
Broker may issue credentials during the maintenance window. No other process may
|
||||
write the listed state/configuration directories during capture.
|
||||
|
||||
```sh
|
||||
mkdir -m 700 /private/backups
|
||||
python3 scripts/stack-recovery.py backup --plan /private/recovery-plan.json \
|
||||
--out /private/backups/stack-snapshot
|
||||
python3 scripts/stack-recovery.py verify --snapshot /private/backups/stack-snapshot
|
||||
python3 scripts/stack-recovery.py restore --snapshot /private/backups/stack-snapshot \
|
||||
--out /private/restore-rehearsal
|
||||
```
|
||||
|
||||
A snapshot is complete only when its manifest exists. Archives and manifest are
|
||||
fsynced, files are private, and SHA-256 catches accidental corruption. This is
|
||||
**not encrypted or signed backup storage**: SQLite/history/configuration may
|
||||
contain plaintext secrets. Keep the private directory on encrypted storage and
|
||||
use authenticated encrypted off-host backup. Preserve image digests in the
|
||||
manifest and the corresponding distribution/images separately. A partial backup
|
||||
is retained for diagnosis and never accepted by `verify`.
|
||||
|
||||
Restore refuses existing destinations and never starts services, contacts
|
||||
Mattermost, issues credentials, or attaches production volumes. SQLite checks
|
||||
include WAL recovery and foreign-key integrity. Archive links, path traversal,
|
||||
absolute paths and special files are rejected. A failed restore has no
|
||||
`RESTORE-INERT.json`; use a fresh destination for retry.
|
||||
|
||||
## Rehearsal and promotion
|
||||
|
||||
1. Restore into a new private directory. Start copied Broker with **no workers or
|
||||
notifications**, new test bearer tokens and `--network none`. Share its network
|
||||
namespace only with the copied Prism/probe containers. Read the accepted run
|
||||
and verify every artifact's size/SHA-256 through the real HTTP API.
|
||||
2. Start copied Prism with copied Beads/state and the isolated Broker. Replay an
|
||||
existing Story request and run synchronization twice; Story identity and
|
||||
fact count must stay unchanged. Check closed Story state through real `bd`.
|
||||
3. Start copied Balda with external MCP/schedulers/webhooks disabled and a local
|
||||
Mattermost protocol fixture in the same isolated network. Balda requires at
|
||||
least one channel. Never point this rehearsal at the real Mattermost server.
|
||||
Confirm startup, persisted sessions and NATS recovery, then stop the copies.
|
||||
4. Read the copied vault with `execution-credentials --vault COPY --operation
|
||||
status`. The original Keychain master key must remain available. Master keys
|
||||
are deliberately absent from these archives: recovery on another machine
|
||||
needs separately protected key escrow or fresh OAuth enrollment. Do not
|
||||
overwrite a newer live OAuth cache with an old refresh token from backup.
|
||||
5. Promotion is a separate operator operation: fence the old deployment and all
|
||||
workers, reconcile resource/credential ownership, restore image/configuration
|
||||
pins, verify routes, then enable one writer per datastore. Never run old and
|
||||
restored control planes simultaneously against the same external resources.
|
||||
|
||||
Record the exact deployed image pins, snapshot hashes, database checks, artifact
|
||||
hashes, credential generations and worker ownership in private operator evidence.
|
||||
Keep these outcomes separate: drained same-host restore, cross-host synthetic vault
|
||||
recovery, and restoration of actual working accounts after host or Keychain loss.
|
||||
The first two do not establish the third. Production recovery requires an
|
||||
independent protected master-key copy or a selected secret store; never store its
|
||||
only recovery credential on the host being backed up. Restore the original keys
|
||||
first, then attempt provider refresh. If protection material cannot be recovered
|
||||
or the provider refuses refresh, preserve the old ciphertext and use fresh OAuth
|
||||
after fencing old workers. A successful browser login must be followed by bounded
|
||||
real-provider inference, checkpoint and cleanup before resuming ordinary work.
|
||||
|
||||
## Protected master-key escrow
|
||||
|
||||
`scripts/key-escrow.py` encrypts the enrolled identities' original master keys
|
||||
to an exact GPG recipient fingerprint. Keep the recipient's private key in an
|
||||
independent protected location and copy the encrypted package off-host. GPG is
|
||||
required; recipient lookup is local and never downloads a key. Use the private
|
||||
plan's `vault_dir` and `credential_status_commands` from the stack backup.
|
||||
Every enrolled identity must be listed and idle, with no pending login. Export
|
||||
takes the shared vault lock, detects concurrent record changes and preserves
|
||||
encrypted records. Plaintext keys are passed in memory, not argv, diagnostics or
|
||||
temporary files. Existing outputs are never overwritten.
|
||||
|
||||
```sh
|
||||
python3 scripts/key-escrow.py export --plan /private/recovery-plan.json \
|
||||
--recipient EXACT_RECOVERY_FINGERPRINT --out /private/backups/vault-keys.gpg
|
||||
python3 scripts/key-escrow.py restore --bundle /private/backups/vault-keys.gpg \
|
||||
--out /private/recovery-keys --gpg-home /private/independent-recovery-keyring
|
||||
```
|
||||
|
||||
Restore creates private key-files and a metadata manifest in a new directory;
|
||||
it starts no service and changes no existing vault. Validate each copied vault
|
||||
with its restored `--key-file` before promotion. On Linux, configure the issuer
|
||||
with that private key-file. On macOS, after validation, run
|
||||
`python3 scripts/key-escrow.py install-keychain --keys /private/recovery-keys`.
|
||||
Identical existing keys are retained; different keys cause a refusal. Keychain
|
||||
installation trusts `/usr/bin/security`, used by the issuer, and sends key
|
||||
material through stdin rather than argv.
|
||||
|
||||
Automated tests use synthetic identities. On 2026-10-16 the operator also accepted
|
||||
recovery of all three working provider identities on an independent physical
|
||||
Linux host without the original Mac or Keychain. The independently protected GPG
|
||||
recipient, encrypted original master keys, matching vault records and recovery
|
||||
software were retained off-host. Each copied vault passed native
|
||||
issue/checkpoint/release and an authenticated read-only provider check (HTTP 200).
|
||||
Temporary plaintext key-files were removed; original working vaults and provider
|
||||
authorizations were unchanged.
|
||||
|
||||
Deliberate upstream revocation and fresh browser login were excluded by the
|
||||
operator from the accepted scope and remain unverified. The successful restore
|
||||
does not guarantee that a provider will accept an older token or allow unattended
|
||||
recovery. This tool never silently resets a missing-key vault or revokes a live
|
||||
session; fresh OAuth remains the fenced fallback described above.
|
||||
@@ -0,0 +1,145 @@
|
||||
# Архив исходника Broker
|
||||
|
||||
Историческая копия до очистки репозитория 2026-10-16. Источник: `mallexxx/execution-broker`, commit `ae92c15d31c8396789765cebf82f7971d6f4cb12`, файл `docs/support.md`. Старые статусы и deployment pins ниже не заменяют актуальную [[personal/projects/balda/execution-broker|эксплуатационную карточку]].
|
||||
|
||||
---
|
||||
|
||||
# Supported configurations
|
||||
|
||||
Execution Broker is a single-operator control plane. One authenticated principal
|
||||
can control its runs. Chat/topic binding prevents accidental cross-session
|
||||
routing; it is not tenant isolation or independent user authorization.
|
||||
|
||||
## Runtime matrix
|
||||
|
||||
| Configuration | Status | Boundary |
|
||||
|---|---|---|
|
||||
| Native service archives | Runtime accepted on Darwin/Linux, ARM64/AMD64 | Exact archive provenance, seven binary versions, synthetic credential lifecycle, authenticated MCP/API, SQLite backup and graceful shutdown; this does not accept every provider on those hosts |
|
||||
| Docker deterministic workflows | Integration tested | Fixed non-root image, no host mounts, read-only filesystem and restricted capabilities |
|
||||
| Run-to-run artifact inputs | Docker integration and native transfer/restart tested | Exact session, source and SHA256 binding, transactional retention holds, resumable transfer before execution; requires the matching execution/v2 worker and gateway |
|
||||
| Declarative project tooling | Native macOS and isolated Linux projects, real Docker handoff and deployed macOS VM accepted | Make/C, Python, Node and JVM projects, cache hit and actual Node SDK change; the tested VM verified the approved Go archive through its scoped tunnel, retained tooling evidence and passed an XCTest |
|
||||
| Immutable repository cache | Native transfer/restart and real Docker/Callee accepted | Exact repository/commit, private checkouts, scoped capabilities and durable deadline holds; opt-in execution/v3 |
|
||||
| Environment replacement | Real Docker/Callee checkpoint, two Broker restarts and single-mutation acceptance | Operator-authored continuation, preserved raw source/base ancestry, separate approval, original deadline, bounded lineage and cancellation; matching execution/v3 workers required |
|
||||
| Virfield v2 macOS workflows | Production macos27 reacceptance passed on 2026-10-16 | Installed Virfield `8bfff9f`: native GUI/new SSH key storage and console qualification, pinned SSH bootstrap, production Story, actual reviewer write denial, independent tests, hashed artifacts, credential settlement and confirmed release |
|
||||
| macOS Codex with persistent OAuth | Story, credential reuse and actual reviewer write denial accepted on the tested operator setup | Earlier fresh-VM/restart checks plus the current owner-approved production continuation passed; eight artifacts were independently downloaded and hashed, four tests repeated, and reviewer `os.open` returned EPERM. Provider expiration/revocation and browser reauthentication were not exercised |
|
||||
| macOS Claude account | Accepted on the tested operator setup | Full Story, quality gate, denied-write probe and fresh-VM reuse after issuer/Broker restart passed; provider expiration/revocation and browser reauthentication were not exercised |
|
||||
| macOS OpenCode with DeepSeek API key | Accepted on the tested operator setup | Real inference, full Story, quality gate and fresh-VM reuse after issuer/Broker restart passed; upstream revocation was not exercised; both read-only role sessions reject injected ACP writes with the native plan policy unchanged |
|
||||
| Docker OpenCode with DeepSeek API key | Story, credential reuse and controlled invalid-key refusal accepted | Two fresh-worker Stories, re-executed quality assertions, hashed artifacts, Broker/issuer restart and checkpoint before cleanup passed. The native client rejected a never-valid API key in a separate synthetic vault; artifacts, credential settlement and cleanup passed. Actual upstream revocation was not exercised; both read-only role sessions reject injected ACP writes with the native plan policy unchanged |
|
||||
| Docker Codex | Updated reviewer policy, production Story, cancellation, restart/reuse and controlled credential refusal accepted | The new image preserves the previously accepted runtime binaries and uses reviewer deny. Broker/gateway/worker pins and the actual image pack agree. Two production Stories passed independent four-test replays and artifact hashes; one survived Broker/gateway/issuer restart with the same run, deadline and credential owner. Cancellation during implementation retained logs and checkpointed credentials. A separate synthetic vault produced native HTTP 401/token_expired refusal with artifacts, credential settlement and cleanup. Every owned worker/container volume was removed; actual upstream revocation and browser reauthentication were not exercised |
|
||||
| Docker Claude account | Story, credential reuse, natural access-token expiry and controlled credential refusal accepted | Two fresh-worker Stories, all six roles, independently repeated tests, hashed artifacts, Broker/issuer restart and checkpoint before cleanup passed; both read-only role sessions reject injected ACP writes. A naturally expired access token refreshed on a fresh worker, three planning roles completed, and the renewed session was checkpointed after cancellation. A separate synthetic vault produced native Authentication required and reauthentication_required with artifacts, credential settlement and cleanup. Actual upstream revocation and browser reauthentication were not exercised |
|
||||
| Managed-host plans | Protocol and local rollback tested | Each real host still requires dedicated identity, reviewed fixed operations and deployment acceptance |
|
||||
| Docker provider credential issuance | Code/synthetic lifecycle and live Claude/OpenCode checkpoint/reuse tested | Broker issues after durable admission, checkpoints a stopped runner, and retains the owner if worker state is lost; each provider's sandbox and credential-failure gates still apply |
|
||||
| Remote host provider credentials | Not enabled | Fixed capability plans do not grant provider login; each host requires its own identity and deployment acceptance |
|
||||
|
||||
On 2026-10-16 the production macos27 gate passed on installed Virfield `8bfff9f`.
|
||||
The successful continuation reused the actual owner-approved plan. Its reviewer
|
||||
executed a bounded `os.open` probe and received EPERM; the verifier and independent
|
||||
checkout passed four tests. All eight retained artifacts were downloaded again
|
||||
with size/SHA256 verification, Prism closed the Story, the credential owner was
|
||||
cleared and Virfield reported no workers or jobs. The Runner package includes
|
||||
the macOS zombie-group settlement correction; the exact source workflow matches
|
||||
the deployed pack. The dated [Virfield acceptance record](https://github.com/mallexxx/virfield/blob/a1460d3372bdfbb5cb19d633d41dd5d3fb3164e6/docs/VERIFICATION.md#broker-macos27-reacceptance)
|
||||
and private operator receipts preserve the tested versions and earlier failed
|
||||
preparations. Those historical failures remain recorded.
|
||||
|
||||
The shared `prism/story` version changed with the reviewer policy. The Linux
|
||||
worker image and gateway allowlist were switched together on 2026-10-16 after
|
||||
explicit operator approval and a zero-active-run check. Live container config,
|
||||
the actual image pack and source digest agree. A production Story and fresh
|
||||
worker reuse across Broker/gateway/issuer restart both passed; independent
|
||||
checkouts repeated four tests each. Cancellation during implementation retained
|
||||
five hashed artifacts and settled the credential lease. Both successful runs
|
||||
retained seven hashed artifacts. No owned worker container or volume remained,
|
||||
and all three credential identities were idle and unrevoked after the cycle.
|
||||
|
||||
On 2026-10-16 a separate Docker Claude probe started with an access token that
|
||||
had expired on 2026-10-15. The native client renewed the session and completed
|
||||
specify, design and breakdown. The operator canceled before implementation;
|
||||
Broker checkpointed the renewed token with the same account and organization,
|
||||
released the identity and removed the worker and volume. All five artifacts
|
||||
passed size/SHA256 verification. This accepts natural access-token expiry and
|
||||
refresh on that Docker setup; upstream revocation and browser reauthentication
|
||||
remain separate checks.
|
||||
|
||||
Integration evidence is specific to the tested configuration. It does not certify
|
||||
arbitrary guest images, provider versions, Docker hosts or remote services.
|
||||
Application/device installation is owned by the calling project; the
|
||||
[iPad workflow](ipad-deployment-plan.md) is a separate scenario, not a Broker
|
||||
release requirement.
|
||||
|
||||
## Balda integration
|
||||
|
||||
The portable plugin provides the execution skill and `/execute` command. Broker
|
||||
and Prism remain external services with operator-owned MCP connections. The
|
||||
plugin contains no bearer tokens, does not install Docker or provision VM images,
|
||||
and cannot approve a managed-host plan itself.
|
||||
|
||||
The CI managed-plugin compatibility gate pins Balda
|
||||
[`6621b189`](https://github.com/baldaworks/balda/commit/6621b189c9533ef8adaddfdb81f3155ea083c363).
|
||||
It exercises the actual package through discovery, managed install, upgrade,
|
||||
disable/enable, removal with retained data, and recovery. Install and upgrade
|
||||
through Backoffice; use a new or explicitly reset session to adopt its snapshot.
|
||||
Repeat the gate against the exact Balda source and binary selected for deployment;
|
||||
the CI pin alone does not attest a different running revision.
|
||||
|
||||
The operator deployment separately qualified clean upstream Balda
|
||||
[`b5f7d5b4`](https://github.com/baldaworks/balda/commit/b5f7d5b485301c3ab21c9622e6bf1dad2f0f307b)
|
||||
with the managed-plugin lifecycle and native wait restart gates. Both deployed
|
||||
instances retained their provider sessions, locator/snapshot bindings, history,
|
||||
configuration and authorization; post-deploy MCP parity and Eagle Asana read-only
|
||||
checks passed.
|
||||
|
||||
Automatic observation uses the bundled `balda.state` and `balda.session.wait`
|
||||
tools. Persist the attempt and arm the original-session wait before Start or
|
||||
retry. Recover lost admission responses with `execution_lookup`; close abandoned
|
||||
attempts with the exact-payload cancellation fence. A separate TTL recovery wait
|
||||
survives a failed intermediate observer turn. The compatibility gate tests the
|
||||
native one-shot's original-session scope and restart recovery, as well as managed
|
||||
plugin lifecycle. A real conversational acceptance is a separate deployment gate.
|
||||
|
||||
Generic webhooks in this Balda revision run isolated jobs with optional literal
|
||||
report destinations; they cannot replace a session observer. Neither wait
|
||||
acceptance nor transport deduplication proves provider delivery or prevents an
|
||||
LLM from repeating a message. Monitor failed Balda turns and provider failures.
|
||||
Existing sessions retain their capability snapshots until explicitly reset.
|
||||
|
||||
## Accepted release scope
|
||||
|
||||
On 2026-10-16 the operator explicitly accepted release readiness for the tested
|
||||
configurations and boundaries above. Production Stories, credential reuse and
|
||||
restart, write denial, cancellation, retained artifact checks and cleanup passed.
|
||||
Separate synthetic vaults exercised native provider refusals for Codex, Claude
|
||||
and DeepSeek on Linux; these runs retained artifacts and settled credential
|
||||
ownership before cleanup. Docker Claude also passed natural access-token expiry
|
||||
and refresh with the working account.
|
||||
|
||||
Working credentials for all three providers were restored on an independent
|
||||
physical Linux host without the original Mac or Keychain. Restored vaults passed
|
||||
native issue/checkpoint/release and authenticated read-only provider checks.
|
||||
The encrypted package, matching vault records and recovery software remain on
|
||||
that host with an independent protected recipient. See [recovery](recovery.md).
|
||||
|
||||
The operator excluded deliberate revocation of working provider authorizations
|
||||
and subsequent browser login from this acceptance. Those operations remain
|
||||
unverified limitations; controlled invalid credentials do not prove actual
|
||||
upstream revocation or browser reauthentication. Linux failure checks do not
|
||||
establish the corresponding macOS behavior. This scoped acceptance does not
|
||||
publish a release or promise unattended recovery when a provider requires login.
|
||||
Additional configurations still require the applicable
|
||||
[executor acceptance](executors.md#deployment-acceptance) gates.
|
||||
|
||||
## Operational limits
|
||||
|
||||
- Run one Broker per database on a local reliable filesystem. SQLite over NFS and
|
||||
multiple writers are unsupported.
|
||||
- Coding roles are sequential. Parallel Stories have separate workers; per-role
|
||||
worktree allocation and parallel mutation are not implemented.
|
||||
- TTL and retry limits bound execution lifetime. They are not token or monetary
|
||||
budgets, and there is no automatic model-tier escalation.
|
||||
- Beads uses embedded Dolt inside Prism. An external shared Dolt server is outside
|
||||
the supplied deployment.
|
||||
- Coordinated backup requires a drained stack. Lost master keys, golden-image
|
||||
restoration and active-worker disaster recovery require separate procedures.
|
||||
|
||||
See [security](security.md) for privilege boundaries and [testing](testing.md)
|
||||
for the distinction between tests, emulation and deployment acceptance.
|
||||
@@ -0,0 +1,151 @@
|
||||
# Архив исходника Broker
|
||||
|
||||
Историческая копия до очистки репозитория 2026-10-16. Источник: `mallexxx/execution-broker`, commit `ae92c15d31c8396789765cebf82f7971d6f4cb12`, файл `docs/testing.md`. Старые статусы и deployment pins ниже не заменяют актуальную [[personal/projects/balda/execution-broker|эксплуатационную карточку]].
|
||||
|
||||
---
|
||||
|
||||
# Testing and acceptance
|
||||
|
||||
`scripts/check-runner-temporary.py --worker-image IMAGE` uses an executor image
|
||||
containing Git and Go. It performs Runner checkout, compiles and executes Go in
|
||||
the private run temporary directory while the shared `/tmp` stays `noexec`.
|
||||
|
||||
Run `make check` before changing a deployment. It executes all Go tests with the
|
||||
race detector, `go vet`, packaged macOS bootstrap checks and coordinated backup
|
||||
failure-path tests. `scripts/check-callee.sh` validates the pinned Callee adapter.
|
||||
These checks do not require provider credentials or post chat messages.
|
||||
|
||||
The native macOS race suite includes a real zombie-only process-group fixture:
|
||||
it reproduces XNU's `EPERM` and requires independent stopped-process evidence.
|
||||
The Callee gate loads the actual Codex role files and injects edit, execute and
|
||||
other permission requests; every read-only role must reject them. When a shared
|
||||
workflow changes, compare Broker, gateway and each selected worker's actual
|
||||
pack and digest before admitting a new run.
|
||||
|
||||
Run `python3 scripts/check-tooling-stacks.py` for real Make/C, Python, Node and
|
||||
JVM profile acceptance. An installed JDK can be selected with `--java-home`.
|
||||
For an isolated Linux run, build `fixtures/tooling/Dockerfile.linux` and pass
|
||||
its image using `--docker-image`. Both modes retain actual versions and compile
|
||||
real fixture projects; neither installs a compiler on the operator host.
|
||||
The ordinary unit suite also checks failed probes, approved sources, checksums,
|
||||
cache quota, archive path boundaries and durable pre-execution restart.
|
||||
|
||||
`scripts/check-tooling-handoff.py --base-image execution-broker-tooling:ci`
|
||||
requires the prepared tooling fixture image, Linux binaries and Callee. It runs
|
||||
five real executions through the image builder/gateway, proves cache hits,
|
||||
changes the actual Node SDK through a new profile, and checks retained provenance
|
||||
and project output. A mid-run checkpoint captures changed/untracked source,
|
||||
restarts Broker twice, requires a new environment approval and resumes only the
|
||||
remaining verification. An independent HTTP counter proves the earlier external
|
||||
mutation happened once. All five runs share one immutable repository bundle.
|
||||
Workers use an internal network and scoped Broker routes;
|
||||
provider credentials and model inference are absent. This gate also catches a
|
||||
builder that drops the selected profile from Runner configuration.
|
||||
|
||||
## Service integration
|
||||
|
||||
Build the Linux worker and pinned Beads binary using the README instructions.
|
||||
Use a new test-only Beads workspace, then run:
|
||||
|
||||
```sh
|
||||
EXECUTION_WORKER_IMAGE="$(docker image inspect execution-broker-worker:local --format '{{.Id}}')" \
|
||||
BEADS_BINARY="$PWD/.build/linux/bd" BEADS_WORKSPACE=/absolute/test/beads \
|
||||
go test -tags=integration -count=1 -v ./tests/integration
|
||||
```
|
||||
|
||||
Use a Beads binary matching the machine that runs `go test`. The Linux binary is
|
||||
appropriate in Linux CI; use the native binary on macOS. Missing Docker or Beads
|
||||
prerequisites produce explicit skips. A skipped integration is not acceptance.
|
||||
The remote test confines its commands to a temporary directory; it does not
|
||||
certify administration of a real managed host.
|
||||
|
||||
The required input gate builds a static Linux test fixture, creates a private
|
||||
Docker network with its own Broker alias, and runs a real producer/consumer pair
|
||||
through Callee. It verifies digest-bound input bytes, original admission after
|
||||
Broker restart, retained provenance and cleanup. It mounts no production state,
|
||||
uses no provider credentials and publishes no ports:
|
||||
|
||||
```sh
|
||||
python3 scripts/check-artifact-inputs.py --worker-image execution-broker-worker:local
|
||||
```
|
||||
|
||||
Run from a source checkout; build the Linux services/Callee and worker image
|
||||
first. The test fixture image
|
||||
contains Docker administrative tooling and receives the socket only for its own
|
||||
acceptance resources; leased workers keep the normal isolation. The static
|
||||
cross-OS fixture complements the source/race checks; it does not claim to run a
|
||||
race detector itself. CI runs this gate explicitly; a skipped ordinary input
|
||||
integration without its private network is not acceptance.
|
||||
|
||||
## Native archives
|
||||
|
||||
`scripts/check-release.py` extracts a trusted archive into fresh private state
|
||||
and verifies provenance, seven binary versions, synthetic credential import and
|
||||
revocation, MCP authentication, SQLite backup and graceful shutdown. It uses no
|
||||
live provider credentials. Host OS and CPU architecture must match the target.
|
||||
`--allow-emulation` labels a local compatibility check as emulated; CI never uses
|
||||
that flag as evidence for a native target.
|
||||
|
||||
The checker supports Python 3.8 and newer, including the stock developer Python
|
||||
on Monterey. It validates every archive member before copying regular files,
|
||||
rejects links and special files, and keeps extracted permissions owner-only.
|
||||
|
||||
The CI matrix runs Linux/macOS on ARM64 and AMD64. Passing a cross-build alone
|
||||
is insufficient. Release promotion requires the native checks and service
|
||||
integration to pass for the exact commit being distributed.
|
||||
|
||||
## Deployment acceptance
|
||||
|
||||
Use a dedicated repository, channel and credential identity. Confirm:
|
||||
|
||||
1. A fresh Balda session can discover the installed plugin, create a Story and
|
||||
start an allowed workflow without direct Docker/SSH access.
|
||||
2. The Human question arrives only in the configured approval channel: Telegram
|
||||
when enabled for the session, otherwise the original conversation. A Telegram
|
||||
request sends one readable message; full details are sent only after an
|
||||
explicit button click, which never resumes the worker. Only the user's actual
|
||||
decision resumes the matching request; an identical answer is idempotent.
|
||||
3. Restart Broker, Prism and Balda during the pause. The run remains recoverable,
|
||||
and no duplicate run or Story is created.
|
||||
4. Check the terminal state, download and hash every artifact, confirm cleanup,
|
||||
and verify Story closure independently of the model's prose.
|
||||
5. Exercise failure, cancellation and recovery. Review duplicate chat responses,
|
||||
resource leftovers, credential ownership and failed-turn monitoring.
|
||||
|
||||
Run `scripts/macos-smoke.py` only against a dedicated Virfield profile.
|
||||
Record the exact installed Virfield version and its native GUI/new SSH key
|
||||
storage and console qualification before bootstrap. If preparation returns
|
||||
`guest_keychain_failed`, record a failed prerequisite and confirmed cleanup;
|
||||
no workload or provider acceptance occurred. Use the [readiness diagnostic
|
||||
procedure](operations.md#vm-readiness-failures) and keep the
|
||||
[current production boundary](support.md#runtime-matrix) separate from earlier
|
||||
successful OS/profile fixtures.
|
||||
|
||||
`scripts/codex-smoke.py` makes real provider calls and approves only its fixed test
|
||||
fixture. Inspect their `--help` before execution. Store logs and raw proof in a
|
||||
private operator directory; neither belongs in the public package.
|
||||
|
||||
Service probe scripts accept `--balda-container`, `--broker-container`,
|
||||
`--prism-container`, `--broker-url`, `--probe-image` and `--executionctl`.
|
||||
They mount the Linux probe binary and one token read-only into a disposable
|
||||
container sharing Balda's network namespace. They do not copy secrets or
|
||||
executables into Balda. Set `--executionctl` to a binary matching Docker's CPU;
|
||||
keep proof files in a private working directory.
|
||||
|
||||
## Binary-only Docker installation
|
||||
|
||||
On each Linux architecture, CI extracts the release archive, builds all four
|
||||
service/worker images from that bundle, and runs the real Callee/Beads fixture
|
||||
through an isolated Compose project. The test covers Human responses, Broker
|
||||
and Prism restarts, task completion, artifact hashes and worker cleanup. It
|
||||
uses fresh secrets and random loopback ports, and removes only its own Docker
|
||||
resources afterward. No live Balda instance or provider identity is used.
|
||||
|
||||
```sh
|
||||
python3 scripts/check-binary-deployment.py \
|
||||
--root /absolute/path/to/extracted-linux-bundle \
|
||||
--state-dir /absolute/path/to/new-private-test-directory
|
||||
```
|
||||
|
||||
Both paths must be visible to the Docker daemon. The test retains its private
|
||||
configuration and diagnostic logs in the new directory; do not publish them.
|
||||
@@ -0,0 +1,139 @@
|
||||
# Архив исходника Broker
|
||||
|
||||
Историческая копия до очистки репозитория 2026-10-16. Источник: `mallexxx/execution-broker`, commit `ae92c15d31c8396789765cebf82f7971d6f4cb12`, файл `docs/work-plan.md`. Старые статусы и deployment pins ниже не заменяют актуальную [[personal/projects/balda/execution-broker|эксплуатационную карточку]].
|
||||
|
||||
---
|
||||
|
||||
# Broker work plan and release scope
|
||||
|
||||
This plan covers Execution Broker's execution infrastructure. Feature completion
|
||||
and release acceptance apply to the capabilities this repository advertises.
|
||||
An unfinished local agent scenario is not a Broker release blocker.
|
||||
|
||||
## Ownership boundaries
|
||||
|
||||
- Broker owns disposable worker provisioning, run lifecycle, execution admission,
|
||||
worker-scoped credentials, generic tooling, artifacts and verified cleanup.
|
||||
- Balda/local agents own task orchestration, MCP connections, search, Obsidian,
|
||||
PR review/publication, recurring work and application/device scenarios.
|
||||
- iPad signing/install/unlock/launch belongs to a local agent/project workflow.
|
||||
Broker can supply a build worker and artifacts if that workflow chooses to use
|
||||
them. Physical-device acceptance is not required for a Broker release.
|
||||
- UI reproduction, project-specific red/green tests and PR/CI policy belong to
|
||||
their project workflow. Generic guest execution/artifact transport can be a
|
||||
Broker capability; the particular app's acceptance is separate.
|
||||
- Broker does not introduce Obsidian vault/folder rights or a general Balda MCP
|
||||
permission system. Earlier cross-project proposals were not an approved user
|
||||
access policy and must not be treated as one.
|
||||
|
||||
## 1. Worker isolation and deployment consistency
|
||||
|
||||
- Keep Docker administrative access in the gateway. Coding workers must not
|
||||
receive a Docker socket/group or ambient host credentials.
|
||||
- Pin the resolved worker image before allocation and reject Broker/gateway
|
||||
drift without leaking undelivered credential leases.
|
||||
- Deploy Broker, gateway, Runner/Callee, workflow/config pins, VM bootstrap and
|
||||
plugin as a consistent accepted set. Preserve a consistent backup and rollback
|
||||
image/config; do not interrupt unrelated active runs.
|
||||
- Verify actual execution evidence, source revision, artifact hashes and cleanup
|
||||
separately from model reports and notification delivery.
|
||||
|
||||
## 2. Generic artifact inputs and retention
|
||||
|
||||
Implemented with transactional holds, scoped transfer, durable pre-execution
|
||||
resume and real Docker/Callee restart acceptance. Deploy the matching worker
|
||||
protocol before advertising this capability in the operator stack.
|
||||
|
||||
- Gate advertised VM/Docker artifact return on success/failure evidence, bounded
|
||||
logs, checksums, authenticated discovery/download, explicit collection errors
|
||||
and documented retention.
|
||||
- Add a run-to-run input contract bound to source run, artifact name and SHA256.
|
||||
Hold source artifacts until their dependent run finishes or explicitly expires.
|
||||
- Accept resumable transfer and reject changed digests with disposable fixtures.
|
||||
Particular device deployments and app-specific signing steps remain outside
|
||||
this generic contract.
|
||||
|
||||
## 3. Runtime tooling and caches
|
||||
|
||||
- Cache immutable tooling by OS/architecture, tool versions and recipe digest.
|
||||
Keep a separate allowed-repository object cache keyed by repository and commit;
|
||||
seed disposable guest-local checkouts without sharing mutable host worktrees
|
||||
or Git credentials.
|
||||
|
||||
### 3.1. Stack-independent environment provisioning
|
||||
|
||||
The declarative profile, package/archive/system backends, scoped verified cache,
|
||||
readiness evidence and generic Go migration are implemented; see [tooling](tooling.md).
|
||||
The operator deployment also passed the real macOS VM bootstrap/archive/tunnel
|
||||
and XCTest gate; a separate Linux gate compiles and executes Go with shared
|
||||
`/tmp` mounted `noexec` and a private executable run temporary directory.
|
||||
Mid-run environment replacement now uses an explicit operator-authored checkpoint
|
||||
and a separately approved continuation; a separate immutable repository cache
|
||||
seeds guest checkouts by repository/commit. Acceptance and deployment evidence
|
||||
must cover the exact shipped version; source acceptance alone is insufficient.
|
||||
|
||||
- Define a versioned, declarative tooling profile for Linux and macOS: OS and
|
||||
architecture constraints, tool/SDK versions, approved package or archive
|
||||
sources, integrity checks, install locations, environment and executable
|
||||
readiness probes. Cover Make/C/C++, CMake/Ninja, Python, Node, Rust and JVM
|
||||
projects through generic provisioning backends. Move the current dedicated
|
||||
Go installer into this mechanism. Keep project tool versions independent
|
||||
from the executor's own pinned Node/ACP runtime.
|
||||
- Let the agent inspect repository build manifests and request the required
|
||||
tooling profile before execution. A Makefile defines commands, not the
|
||||
compilers, libraries or SDKs needed to run them. Validate requested tools
|
||||
against operator policy; repository content cannot authorize a new source,
|
||||
privilege or network destination. Provision and run readiness checks before
|
||||
implementation and verification. Report missing or incompatible tools with
|
||||
their required versions and the failed prerequisite.
|
||||
- Install system dependencies while preparing the disposable image or guest.
|
||||
Permit project-local dependencies only within the run's writable workspace
|
||||
and granted network scope. Preserve Linux worker isolation and keep host and
|
||||
golden-image state unchanged. Pin the resolved environment and cache it by
|
||||
platform, versions, package sources and recipe digest; record the actual
|
||||
tool versions in run evidence.
|
||||
- Handle a missing dependency discovered during execution through a bounded,
|
||||
durable environment-change request. Validate any additional approval, save
|
||||
the exact source revision and safe workflow checkpoint, prepare the new
|
||||
immutable environment, and resume idempotently. Reconcile the old worker and
|
||||
credential lease before replacement. Do not replay external mutations or
|
||||
widen the approved task implicitly; stop repeated no-progress rebuilds with
|
||||
a visible failure. Cancellation and restart must retain artifacts and clean
|
||||
both old and replacement resources.
|
||||
- Acceptance: run Make/C or CMake, Python and Node projects on Linux and macOS,
|
||||
plus a Rust or JVM project supplied through a new tooling profile without a
|
||||
Broker code change. Verify a cache hit, version change, missing tool found
|
||||
mid-run, unavailable package source, invalid checksum, denied installation,
|
||||
restart and cancellation. Prove no installation reached the host, executor
|
||||
runtime or shared golden image, and no duplicate side effect occurred.
|
||||
|
||||
## 4. Optional generic remote-action capability
|
||||
|
||||
- If Broker advertises a configurable remote-action API, implement typed command
|
||||
IDs/arguments, protected credentials, pinned host identity and explicit action
|
||||
scope, plus cancellation and reconciliation after an ambiguous disconnect.
|
||||
- Accept it using disposable managed-host fixtures, including invalid arguments,
|
||||
changed host keys and unknown outcomes. Keep arbitrary host/device workflows
|
||||
in the caller. A local agent may also use its own SSH tooling without Broker.
|
||||
- This future capability is separate from the existing signed fixed host plans
|
||||
and is not required merely because a local iPad scenario uses SSH.
|
||||
|
||||
## 5. Release acceptance for the supported scope
|
||||
|
||||
- Run source/race/vet/lint, adapter, security, plugin, real worker/artifact/restart
|
||||
and binary-install gates against the exact accepted revision.
|
||||
- Label native and emulated results accurately. Test advertised target platforms;
|
||||
document unsupported/unverified cases instead of inferring support.
|
||||
- Verify the deployed versions and one real conversational run with visible
|
||||
result, retained artifacts, credential settlement and confirmed cleanup.
|
||||
Test failures/outages at the Balda boundary with its own compatible code.
|
||||
- Verify backup/restore for the claimed recovery scope. A same-host drained
|
||||
restore alone does not prove recovery after host or credential-store loss.
|
||||
|
||||
## External scenario references
|
||||
|
||||
[iPad deployment](ipad-deployment-plan.md) and
|
||||
[Hermes migration inventory](hermes-workflow-gap-analysis.md) are cross-project
|
||||
reference notes. They do not add device-specific code, search/Obsidian access
|
||||
policy, personal workflow migration or PR publication requirements to Broker.
|
||||
Their scenario-level requirements belong to the local agent/project owner.
|
||||
@@ -0,0 +1,15 @@
|
||||
# Очистка репозитория Execution Broker — 2026-10-16
|
||||
|
||||
Сохранены девять исходных документов из commit `ae92c15`. Содержимое каждого проверено обратным чтением. Планы, исследования, отчёты приёмки и персональный deployment state ведутся в Obsidian; в Git остаются продуктовые контракты, инструкции и changelog. Это архив до очистки, его старые открытые пункты не отменяют согласованное закрытие приёмки.
|
||||
|
||||
- [[personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/README]]
|
||||
- [[personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/CHANGELOG]]
|
||||
- [[personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/docs/work-plan]]
|
||||
- [[personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/docs/hermes-workflow-gap-analysis]]
|
||||
- [[personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/docs/ipad-deployment-plan]]
|
||||
- [[personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/docs/support]]
|
||||
- [[personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/docs/recovery]]
|
||||
- [[personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/docs/testing]]
|
||||
- [[personal/projects/archive/execution-broker-repo-cleanup-2026-10-16/docs/deployment]]
|
||||
|
||||
Актуальные границы и установка: [[personal/projects/balda/execution-broker]]. Приёмка: [[personal/plans/balda-callee-prism-plan]].
|
||||
Reference in new issue
Block a user