diff --git a/personal/projects/eagle-dashboard.md b/personal/projects/eagle-dashboard.md new file mode 100644 index 00000000..1114b9b7 --- /dev/null +++ b/personal/projects/eagle-dashboard.md @@ -0,0 +1,1085 @@ +# Eagle Dashboard — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace `localhost` with a visual control panel that shows health, enables management (start/stop/restart) of all 25+ Eagle LaunchAgent services, and displays network security posture per service. + +**Architecture:** FastAPI backend reads a YAML service registry and checks health via HTTP/port/process checks every 30s. HTMX + Tailwind CDN frontend renders live cards with no build step. `launchctl` integration enables lifecycle control. Runs as its own LaunchAgent on port 3000. + +**Tech Stack:** Python 3.11, FastAPI, uvicorn, HTMX 2, Tailwind CDN, Alpine.js, PyYAML, psutil, httpx + +--- + +## Scope note + +Two independent subsystems — build them separately: +- **Plan A (this doc):** Service registry + health API + dashboard UI + LaunchAgent control +- **Plan B (future):** Network security panel — cloudflared tunnel management, pf rules, per-service firewall toggling + +--- + +## File Structure + +``` +~/Developer/eagle-dash/ +├── main.py # FastAPI app — routes, startup, SSE +├── registry.py # Service registry loader + YAML parser +├── health.py # Health check engine (http/port/process/launchctl) +├── control.py # launchctl start/stop/restart wrappers +├── services.yaml # Service definitions (source of truth) +├── templates/ +│ └── index.html # Dashboard — HTMX + Tailwind CDN + Alpine.js +├── requirements.txt +├── justfile +└── com.eagle.dashboard.plist # LaunchAgent plist +``` + +--- + +## Task 1: Project Bootstrap + +**Files:** +- Create: `~/Developer/eagle-dash/requirements.txt` +- Create: `~/Developer/eagle-dash/justfile` + +- [ ] **Step 1: Create project directory and requirements** + +```bash +mkdir -p ~/Developer/eagle-dash/templates +``` + +`~/Developer/eagle-dash/requirements.txt`: +``` +fastapi==0.115.0 +uvicorn[standard]==0.30.6 +pyyaml==6.0.2 +httpx==0.27.2 +psutil==6.0.0 +jinja2==3.1.4 +``` + +- [ ] **Step 2: Create justfile** + +`~/Developer/eagle-dash/justfile`: +```just +# vim: set ft=just ts=2 +[private] +default: + @just --list + +install: + pip install -r requirements.txt + +dev: + uvicorn main:app --reload --host 127.0.0.1 --port 3000 + +prod: + uvicorn main:app --host 127.0.0.1 --port 3000 --workers 1 + +deploy: + cp com.eagle.dashboard.plist ~/Library/LaunchAgents/ + launchctl load ~/Library/LaunchAgents/com.eagle.dashboard.plist + +undeploy: + launchctl unload ~/Library/LaunchAgents/com.eagle.dashboard.plist +``` + +- [ ] **Step 3: Install deps** + +```bash +cd ~/Developer/eagle-dash +pip install -r requirements.txt +``` + +Expected: all packages installed without error. + +- [ ] **Step 4: Commit** + +```bash +cd ~/Developer/eagle-dash +git init && git add . && git commit -m "feat: project bootstrap" +``` + +--- + +## Task 2: Service Registry YAML + +**Files:** +- Create: `~/Developer/eagle-dash/services.yaml` + +This is the source of truth. Every service gets: name, launchd label, health check config, network access level, category, log path. + +- [ ] **Step 1: Write services.yaml** + +```yaml +# Network access levels: +# localhost — binds to 127.0.0.1 only ✅ +# lan — binds to 0.0.0.0, home-network-only ⚠️ +# tunnel — exposed via cloudflared 🌐 +# public — directly internet-facing 🔴 + +services: + + # ── AI / ML ────────────────────────────────────────────── + claude_proxy_python: + name: "Claude Proxy (Python)" + label: "com.openclaw.claude-proxy" + port: 8090 + health: + type: http + url: "http://localhost:8090/v1/models" + access: localhost + critical: true + category: ai + log: "/tmp/claude-proxy.log" + + claude_proxy_node: + name: "Claude Proxy (Node.js)" + label: "ai.claude-proxy" + port: 3456 + health: + type: port + access: localhost + critical: false + category: ai + log: "/tmp/claude-proxy-node.log" + + mlx_lm: + name: "MLX LM Server" + label: "com.mlx-lm-server" + port: 8080 + health: + type: http + url: "http://localhost:8080/v1/models" + access: lan + critical: false + category: ai + log: "/tmp/mlx-lm.log" + + ollama: + name: "Ollama" + label: "com.ollama.server" + port: 11434 + health: + type: http + url: "http://localhost:11434/api/tags" + access: lan + critical: false + category: ai + log: "/tmp/ollama.log" + + hermes: + name: "Hermes Gateway" + label: "ai.hermes.gateway" + health: + type: process + name: "hermes" + access: localhost + critical: true + category: ai + log: "/tmp/hermes.log" + + hermes_whale: + name: "Hermes Whale" + label: "ai.hermes.whale-gateway" + health: + type: process + name: "hermes-whale" + access: localhost + critical: false + category: ai + log: "/tmp/hermes-whale.log" + + asana_mcp: + name: "Asana MCP Proxy" + label: "com.asana-mcp-proxy" + health: + type: launchctl + access: localhost + critical: false + category: ai + log: "/tmp/asana-mcp.log" + + virfield: + name: "Virfield VM MCP" + label: "com.virfield.server" + health: + type: launchctl + access: localhost + critical: false + category: ai + + # ── Database ───────────────────────────────────────────── + postgres: + name: "PostgreSQL 17" + label: "homebrew.mxcl.postgresql@17" + port: 5432 + health: + type: port + access: localhost + critical: true + category: database + log: "/opt/homebrew/var/log/postgresql@17.log" + + # ── Monitoring ─────────────────────────────────────────── + aw_ddg: + name: "AW Watcher DDG" + label: "com.personalos.aw-watcher-ddg" + health: + type: process + name: "aw-watcher-ddg" + access: localhost + critical: false + category: monitoring + log: "/tmp/aw-watcher-ddg.log" + + aw_xcode: + name: "AW Watcher Xcode" + label: "com.personalos.aw-watcher-xcode" + health: + type: process + name: "aw-watcher-xcode" + access: localhost + critical: false + category: monitoring + log: "/tmp/aw-watcher-xcode.log" + + # ── Orchestration ──────────────────────────────────────── + heartbeat: + name: "Personal OS Heartbeat" + label: "com.personalos.heartbeat" + health: + type: launchctl + access: localhost + critical: true + category: orchestration + log: "~/Library/Logs/personal-os/heartbeat.log" + + # ── Infrastructure ─────────────────────────────────────── + colima: + name: "Colima (Docker)" + label: "com.colima.start" + health: + type: process + name: "colima" + access: localhost + critical: false + category: infra + + zulip: + name: "Zulip" + label: "top.qentra.zulip" + port: 8080 + health: + type: launchctl + access: localhost + critical: false + category: infra +``` + +- [ ] **Step 2: Commit** + +```bash +cd ~/Developer/eagle-dash +git add services.yaml && git commit -m "feat: service registry YAML" +``` + +--- + +## Task 3: Registry Loader + +**Files:** +- Create: `~/Developer/eagle-dash/registry.py` +- Create: `~/Developer/eagle-dash/tests/test_registry.py` + +- [ ] **Step 1: Write the failing test** + +`~/Developer/eagle-dash/tests/test_registry.py`: +```python +import pytest +from registry import load_registry, Service + +def test_load_registry_returns_services(): + services = load_registry("services.yaml") + assert len(services) > 0 + +def test_critical_services_present(): + services = load_registry("services.yaml") + names = {s.id for s in services} + assert "postgres" in names + assert "hermes" in names + assert "claude_proxy_python" in names + +def test_service_has_required_fields(): + services = load_registry("services.yaml") + svc = next(s for s in services if s.id == "postgres") + assert svc.name == "PostgreSQL 17" + assert svc.label == "homebrew.mxcl.postgresql@17" + assert svc.access == "localhost" + assert svc.critical is True + assert svc.category == "database" +``` + +- [ ] **Step 2: Run test to verify it fails** + +```bash +cd ~/Developer/eagle-dash +python -m pytest tests/test_registry.py -v +``` + +Expected: FAIL with `ModuleNotFoundError: No module named 'registry'` + +- [ ] **Step 3: Write registry.py** + +`~/Developer/eagle-dash/registry.py`: +```python +from dataclasses import dataclass, field +from typing import Optional +import yaml + +@dataclass +class HealthConfig: + type: str # http | port | process | launchctl + url: Optional[str] = None + name: Optional[str] = None # process name + +@dataclass +class Service: + id: str + name: str + label: str + health: HealthConfig + access: str # localhost | lan | tunnel | public + critical: bool + category: str + port: Optional[int] = None + log: Optional[str] = None + +def load_registry(path: str = "services.yaml") -> list[Service]: + with open(path) as f: + raw = yaml.safe_load(f) + services = [] + for svc_id, cfg in raw["services"].items(): + hcfg = cfg["health"] + health = HealthConfig( + type=hcfg["type"], + url=hcfg.get("url"), + name=hcfg.get("name"), + ) + svc = Service( + id=svc_id, + name=cfg["name"], + label=cfg["label"], + health=health, + access=cfg.get("access", "localhost"), + critical=cfg.get("critical", False), + category=cfg.get("category", "other"), + port=cfg.get("port"), + log=cfg.get("log"), + ) + services.append(svc) + return services +``` + +- [ ] **Step 4: Run test to verify it passes** + +```bash +cd ~/Developer/eagle-dash +python -m pytest tests/test_registry.py -v +``` + +Expected: PASS all 3 tests. + +- [ ] **Step 5: Commit** + +```bash +cd ~/Developer/eagle-dash +git add registry.py tests/ && git commit -m "feat: service registry loader" +``` + +--- + +## Task 4: Health Check Engine + +**Files:** +- Create: `~/Developer/eagle-dash/health.py` +- Create: `~/Developer/eagle-dash/tests/test_health.py` + +- [ ] **Step 1: Write the failing test** + +`~/Developer/eagle-dash/tests/test_health.py`: +```python +import pytest +from unittest.mock import patch, MagicMock +from health import check_port, check_process, ServiceStatus + +def test_port_check_open(): + # Port 5432 (postgres) should be open on this machine + result = check_port(5432) + assert result is True + +def test_port_check_closed(): + result = check_port(19999) # nothing runs here + assert result is False + +def test_process_check_existing(): + result = check_process("python3") + assert result is True + +def test_process_check_missing(): + result = check_process("definitely-not-running-xyz") + assert result is False + +def test_service_status_dataclass(): + s = ServiceStatus(id="test", healthy=True, detail="ok") + assert s.id == "test" + assert s.healthy is True +``` + +- [ ] **Step 2: Run test to verify it fails** + +```bash +cd ~/Developer/eagle-dash +python -m pytest tests/test_health.py -v +``` + +Expected: FAIL with `ModuleNotFoundError: No module named 'health'` + +- [ ] **Step 3: Write health.py** + +`~/Developer/eagle-dash/health.py`: +```python +import socket +import subprocess +from dataclasses import dataclass +from typing import Optional +import httpx +import psutil +from registry import Service + +@dataclass +class ServiceStatus: + id: str + healthy: bool + detail: str = "" + latency_ms: Optional[float] = None + +def check_port(port: int, host: str = "127.0.0.1", timeout: float = 1.0) -> bool: + try: + with socket.create_connection((host, port), timeout=timeout): + return True + except OSError: + return False + +def check_process(name: str) -> bool: + for proc in psutil.process_iter(["name", "cmdline"]): + try: + cmdline = " ".join(proc.info["cmdline"] or []) + if name in proc.info["name"] or name in cmdline: + return True + except (psutil.NoSuchProcess, psutil.AccessDenied): + pass + return False + +def check_launchctl(label: str) -> bool: + result = subprocess.run( + ["launchctl", "list", label], + capture_output=True, text=True + ) + return result.returncode == 0 + +def check_http(url: str, timeout: float = 3.0) -> tuple[bool, float]: + import time + try: + t0 = time.monotonic() + resp = httpx.get(url, timeout=timeout) + latency = (time.monotonic() - t0) * 1000 + return resp.status_code < 500, latency + except Exception: + return False, 0.0 + +async def check_service(svc: Service) -> ServiceStatus: + h = svc.health + if h.type == "http": + ok, ms = check_http(h.url) + return ServiceStatus(id=svc.id, healthy=ok, latency_ms=ms, + detail="HTTP ok" if ok else "HTTP failed") + if h.type == "port": + ok = check_port(svc.port) + return ServiceStatus(id=svc.id, healthy=ok, + detail=f"port {svc.port} {'open' if ok else 'closed'}") + if h.type == "process": + ok = check_process(h.name) + return ServiceStatus(id=svc.id, healthy=ok, + detail=f"process '{h.name}' {'found' if ok else 'not found'}") + if h.type == "launchctl": + ok = check_launchctl(svc.label) + return ServiceStatus(id=svc.id, healthy=ok, + detail="loaded" if ok else "not loaded") + return ServiceStatus(id=svc.id, healthy=False, detail=f"unknown check type: {h.type}") +``` + +- [ ] **Step 4: Run tests to verify they pass** + +```bash +cd ~/Developer/eagle-dash +python -m pytest tests/test_health.py -v +``` + +Expected: PASS all 5 tests. (Port 5432 must be open — postgres must be running.) + +- [ ] **Step 5: Commit** + +```bash +cd ~/Developer/eagle-dash +git add health.py tests/test_health.py && git commit -m "feat: health check engine" +``` + +--- + +## Task 5: FastAPI Backend + +**Files:** +- Create: `~/Developer/eagle-dash/main.py` +- Create: `~/Developer/eagle-dash/control.py` + +- [ ] **Step 1: Write control.py (launchctl wrappers)** + +`~/Developer/eagle-dash/control.py`: +```python +import subprocess +import os + +def _launchctl(action: str, label_or_plist: str) -> tuple[bool, str]: + cmd = ["launchctl", action, label_or_plist] + result = subprocess.run(cmd, capture_output=True, text=True) + return result.returncode == 0, result.stderr.strip() or result.stdout.strip() + +def stop_service(label: str) -> tuple[bool, str]: + return _launchctl("stop", label) + +def start_service(label: str) -> tuple[bool, str]: + return _launchctl("start", label) + +def kickstart_service(label: str) -> tuple[bool, str]: + domain = f"gui/{os.getuid()}/{label}" + result = subprocess.run( + ["launchctl", "kickstart", "-k", domain], + capture_output=True, text=True + ) + return result.returncode == 0, result.stderr.strip() or result.stdout.strip() +``` + +- [ ] **Step 2: Write main.py** + +`~/Developer/eagle-dash/main.py`: +```python +import asyncio +import os +from pathlib import Path +from contextlib import asynccontextmanager + +from fastapi import FastAPI, HTTPException, Response +from fastapi.responses import HTMLResponse, StreamingResponse +from fastapi.templating import Jinja2Templates +from fastapi import Request + +from registry import load_registry, Service +from health import check_service, ServiceStatus +from control import stop_service, start_service, kickstart_service + +REGISTRY_PATH = Path(__file__).parent / "services.yaml" +TEMPLATES = Jinja2Templates(directory=str(Path(__file__).parent / "templates")) + +_services: list[Service] = [] +_status_cache: dict[str, ServiceStatus] = {} + +@asynccontextmanager +async def lifespan(app: FastAPI): + global _services + _services = load_registry(str(REGISTRY_PATH)) + asyncio.create_task(_poll_loop()) + yield + +app = FastAPI(title="Eagle Dashboard", lifespan=lifespan) + +async def _poll_loop(): + while True: + results = await asyncio.gather( + *[check_service(s) for s in _services], + return_exceptions=True + ) + for svc, result in zip(_services, results): + if isinstance(result, ServiceStatus): + _status_cache[svc.id] = result + await asyncio.sleep(30) + +def _svc_by_id(svc_id: str) -> Service: + for s in _services: + if s.id == svc_id: + return s + raise HTTPException(status_code=404, detail=f"Service '{svc_id}' not in registry") + +@app.get("/", response_class=HTMLResponse) +async def dashboard(request: Request): + return TEMPLATES.TemplateResponse("index.html", {"request": request}) + +@app.get("/api/services") +async def list_services(): + result = [] + for svc in _services: + status = _status_cache.get(svc.id) + result.append({ + "id": svc.id, + "name": svc.name, + "label": svc.label, + "category": svc.category, + "access": svc.access, + "critical": svc.critical, + "port": svc.port, + "healthy": status.healthy if status else None, + "detail": status.detail if status else "pending", + "latency_ms": status.latency_ms if status else None, + }) + return result + +@app.post("/api/services/{svc_id}/restart") +async def restart_service(svc_id: str): + svc = _svc_by_id(svc_id) + ok, msg = kickstart_service(svc.label) + return {"ok": ok, "message": msg} + +@app.post("/api/services/{svc_id}/stop") +async def stop(svc_id: str): + svc = _svc_by_id(svc_id) + ok, msg = stop_service(svc.label) + return {"ok": ok, "message": msg} + +@app.post("/api/services/{svc_id}/start") +async def start(svc_id: str): + svc = _svc_by_id(svc_id) + ok, msg = start_service(svc.label) + return {"ok": ok, "message": msg} + +@app.get("/api/services/{svc_id}/logs") +async def tail_log(svc_id: str, lines: int = 100): + svc = _svc_by_id(svc_id) + if not svc.log: + raise HTTPException(status_code=404, detail="No log configured for this service") + log_path = Path(svc.log).expanduser() + if not log_path.exists(): + return {"lines": [], "error": "log file not found"} + with open(log_path, "r", errors="replace") as f: + all_lines = f.readlines() + return {"lines": all_lines[-lines:]} + +@app.get("/api/health") +async def self_health(): + return {"status": "ok", "services_registered": len(_services)} +``` + +- [ ] **Step 3: Smoke-test the API manually** + +```bash +cd ~/Developer/eagle-dash +just dev & +sleep 2 +curl -s http://localhost:3000/api/health +curl -s http://localhost:3000/api/services | python3 -m json.tool | head -40 +``` + +Expected: `{"status":"ok","services_registered":14}` and a JSON array of services. + +- [ ] **Step 4: Commit** + +```bash +cd ~/Developer/eagle-dash +git add main.py control.py && git commit -m "feat: FastAPI backend with health API and control endpoints" +``` + +--- + +## Task 6: Dashboard HTML Template + +**Files:** +- Create: `~/Developer/eagle-dash/templates/index.html` + +The UI uses Tailwind CDN (no build step), HTMX for polling, Alpine.js for interactivity. + +- [ ] **Step 1: Write index.html** + +`~/Developer/eagle-dash/templates/index.html`: +```html + + + + + + Eagle Dashboard + + + + + + + + + +
+
+

Eagle Dashboard

+

+ services · + healthy · + unhealthy +

+
+
+ auto-refresh 30s + +
+
+ + + + +

+ Last updated: +

+ + + + +``` + +- [ ] **Step 2: Open the dashboard in browser** + +```bash +cd ~/Developer/eagle-dash +just dev +open http://localhost:3000 +``` + +Expected: Dark dashboard with service cards grouped by category, green/red dots, network badges, restart/stop buttons. + +- [ ] **Step 3: Verify all categories appear** + +Expected categories: `ai`, `database`, `monitoring`, `orchestration`, `infra` + +- [ ] **Step 4: Commit** + +```bash +cd ~/Developer/eagle-dash +git add templates/index.html && git commit -m "feat: dashboard UI with HTMX, Tailwind, Alpine.js" +``` + +--- + +## Task 7: Log Tail UI Panel + +**Files:** +- Modify: `~/Developer/eagle-dash/templates/index.html` (add log drawer) + +- [ ] **Step 1: Add log panel to index.html** + +Add this section inside ``, after the main grid and before the `