diff --git a/personal/projects/balda/balda.md b/personal/projects/balda/balda.md new file mode 100644 index 00000000..80adbf55 --- /dev/null +++ b/personal/projects/balda/balda.md @@ -0,0 +1,99 @@ +# Balda (Валера) — AI Worker Bot + +Аналог Hermes. Go-сервис: берёт задачи из чата (Telegram или Zulip), запускает AI CLI и отвечает. + +## Репо + +- Upstream: https://github.com/normahq/balda +- Fork: https://github.com/mallexxx/balda +- Local: ~/Developer/balda +- Ветка: `feat/zulip-transport` + +## Архитектура + +- NATS JetStream — event bus (embedded) +- SQLite — состояние (owner, sessions, tasks) +- Telegram — через polling или webhook +- Zulip — через outgoing webhook bot +- Providers: codex, opencode, copilot, gemini, claude (ACP protocol) + +## Zulip Integration + +Полностью реализована. Webhook-based (не polling). +Тип бота: **Outgoing webhook** (не Generic bot). + +### Credentials (zulip.qentra.top) + +- Bot email: `balda-bot@zulip.qentra.top` +- API key: `AtC9FypdIl6F0hNICW0lcOzehrVrWhPr` +- Server URL: `https://zulip.qentra.top` +- Webhook token: берётся из Settings → Bots → balda-bot → Edit → Token + +### Config (env vars) + +``` +BALDA_ZULIP_BOT_EMAIL=balda-bot@zulip.qentra.top +BALDA_ZULIP_API_KEY=AtC9FypdIl6F0hNICW0lcOzehrVrWhPr +BALDA_ZULIP_SERVER_URL=https://zulip.qentra.top +BALDA_ZULIP_WEBHOOK_TOKEN=<из настроек бота> +BALDA_ZULIP_WEBHOOK_ENABLED=true +``` + +### Webhook setup в Zulip + +1. Settings → Bots → balda-bot → Edit +2. Bot type: Outgoing webhook +3. Endpoint URL: `http://:8090/zulip/webhook` +4. Скопировать token → `BALDA_ZULIP_WEBHOOK_TOKEN` + +### Авторизация + +После запуска написать боту в DM: +``` +/start owner= +``` + +## Реализованные команды (Telegram parity) + +| Команда | Статус | +|---------|--------| +| `/start owner=` | ✅ | +| `/start invite=` | ✅ | +| `/goal ` / `/goal clear` | ✅ | +| `/topic` | ✅ (создаёт сессию для нового топика) | +| `/user add ` | ✅ | +| `/user list` | ✅ (с инвайтами + expiry) | +| Typing indicator | ✅ (isDM-aware: true в DM, false в стримах) | +| plan_updates / SendDraftPlain | ➖ no-op (нет аналога в Zulip) | + +## Upcoming: mention-based thread takeover + +Когда `@balda-bot` упоминается в топике без активной сессии — бот автоматически берёт топик под контроль. + +Механизм (аналог whale-thread-guard в Hermes): +- Webhook payload поле `trigger == "mention"` → auto-create session +- `allowed_owners` список в конфиге: Zulip email без токена +- После захвата топик отвечает на все сообщения (обычная сессия) + +## Конфиг и инструкции + +- [`balda/soul.md`](balda/soul.md) — soul (global_instruction) — редактировать здесь, синхронизировать в config.yaml +- [`balda/workspace-context.md`](balda/workspace-context.md) — per-provider system_instructions (рабочий проект/стек) +- `~/Developer/balda/.config/balda/config.yaml` — основной конфиг (runtime + balda) +- `~/Developer/balda/.env` — секреты (Zulip API key, webhook token) + +### Запуск + +```bash +cd ~/Developer/balda && ~/.local/bin/balda start +``` + +После первого запуска: из логов взять `owner_token` и отправить боту в Zulip DM: +``` +/start owner= +``` + +## Docs + +- `docs/zulip-webhook.md` — полное руководство по настройке +- `docs/balda.md` — техническая спецификация diff --git a/personal/projects/personal-os/claude-proxy-node-internals.md b/personal/projects/personal-os/claude-proxy-node-internals.md new file mode 100644 index 00000000..ba1abdd8 --- /dev/null +++ b/personal/projects/personal-os/claude-proxy-node-internals.md @@ -0,0 +1,121 @@ +--- +title: openclaw-claude-proxy (Node.js) — внутреннее устройство и проблемы +type: reference +namespace: personal +tags: + - hermes + - mac + - eagle + - claude-proxy + - nodejs + - llm-backend + - pitfalls + - docker +created: '2026-06-15' +updated: '2026-06-15' +confidence: 0.9 +--- +# openclaw-claude-proxy (Node.js) — внутреннее устройство и проблемы + +## Обзор + +**npm пакет:** `openclaw-claude-proxy` v1.0.8+ +**Порт:** 3456 +**Язык:** TypeScript (компилируется в JS) +**Репозиторий:** [mehdic/openclaw-claude-proxy](https://github.com/mehdic/openclaw-claude-proxy) +**Форк:** `mnemon-dev/claude-max-api-proxy` + OpenClaw compatibility PR +**Текущий статус:** fallback (активен Python wrapper на порту 8090) + +## Архитектура + +``` +Hermes → POST /v1/chat/completions (порт 3456, Node.js Express) + └── proxy преобразует OpenAI-формат → stream-json протокол + └── спавнит claude CLI как child process + └── claude --output-format stream-json --verbose + └── читает/пишет JSON через stdin/stdout pipe +``` + +### Runtime-модели (CLAUDE_PROXY_RUNTIME) + +**stream-json** (default): +- init pool: несколько долгоживущих `claude` subprocess'ов +- session pool: переиспользование сессий между запросами +- промпт-кэш между вызовами сохраняется +- использует `--output-format stream-json --input-format stream-json` + +**print** (fallback): +- на каждый запрос spawn'ит свежий `claude --print` +- медленнее, но изолированно +- не требует совместимости stream-json протокола + +## Контейнерный сетап + +`~/Docker/claude-proxy-node/Dockerfile` — образ на node:20-alpine: +``` +FROM node:20-alpine +RUN npm install -g openclaw-claude-proxy @anthropic-ai/claude-code +EXPOSE 3456 +CMD ["claude-proxy", "3456"] +``` + +Образ не собран — в работе. + +## Проблемы + +### 1. Orphaned claude subprocess'ы + +**Механизм:** `stream-json` runtime держит пул долгоживущих `claude` child process'ов через `child_process.spawn()`. Proxy управляет их жизненным циклом: при нормальном shutdown (SIGTERM) должен завершить pool. Но: + +- **SIGKILL** родительского proxy (kill -9, docker stop --time=0, launchd force quit) — Node.js не успевает выполнить cleanup +- **SIGTERM с таймаутом** — если graceful shutdown превышает лимит времени, Node.js процесс убивается до завершения pool +- **launchd KeepAlive** — при падении proxy по любой причине launchd убивает и перезапускает, но старые `claude` процессы с PPID 1 остаются в системе +- **docker restart** — если контейнер перезапускается без `--time` (grace period), старые процессы переживают контейнер + +На macOS orphan'ы выглядят как `claude` с PPID=1. Их количество растёт со временем. + +### 2. Нет живого вывода claude CLI + +**Причина:** proxy читает stdout `claude` исключительно как JSON pipe. Человеческий вывод markdown/thinking/прогресса `claude` пишет в **stderr**, но proxy его не форвардит. + +По умолчанию stderr `claude` либо: +- не пипруется вообще (pipe не открывается) — stderr уходит в /dev/null контейнера +- пипруется и выбрасывается + +В режиме `print` проблема та же — proxy ждёт завершения `claude --print` и возвращает результат целиком, не транслируя промежуточный вывод. + +### 3. claude spawn'ит свой bash без контекста Hermes + +**Механизм:** proxy запускает `claude` через `child_process.spawn('claude', args, {env: process.env})`. Ниже по цепочке: + +``` +launchd → claude-proxy-start.sh → claude-proxy (node) → claude CLI (subprocess) + └── bash -c "..." +``` + +Claude Code CLI для выполнения инструментов (Bash, Write, Edit) внутри себя спавнит `bash -c "..."`. Этот bash: +- наследует env от `claude` процесса +- НЕ имеет `HERMES_HOME`, кастомного `PATH`, `SHELL=/bin/zsh` с профилем Hermes +- использует системный `/bin/bash` или `/bin/sh` + +В контейнере (alpine) bash вообще может отсутствовать — claude CLI использует `/bin/sh`. + +## Предложенные подходы + +### Против orphaned процессов + +1. **Docker `--init` флаг:** запускать контейнер с `init: true` в docker-compose (или `docker run --init`). Использует `tini` как PID 1 — корректно форвардит SIGTERM и перезахоранивает orphan'ов. +2. **SIGTERM handler в Node.js:** добавить `process.on('SIGTERM', ...)` который форсированно убивает весь child process pool перед exit. +3. **Health check + restart policy:** не `always`, а `unless-stopped` с `--time=30` grace period. + +### Против отсутствия вывода CLI + +1. **Проброс stderr:** pipe'ить stderr `claude` в stderr proxy — видно в `docker logs` или launchd логах. +2. **Режим `print`:** переключить `CLAUDE_PROXY_RUNTIME=print` — теряется производительность stream-json, но логов больше. +3. **Node.js `stream-json` мониторинг:** логировать каждую N-ную JSON строку из stdout claude для отладки. + +### Против изоляции shell + +1. **В Docker:** установить bash в контейнер (`apk add bash`), установить `SHELL=/bin/bash` в env. +2. **Передача `HERMES_HOME`:** смонтировать `~/.hermes` в контейнер и передать `HERMES_HOME` в env claude CLI. +3. **wrapper-скрипт для claude:** заменить бинарник `claude` в контейнере на shell wrapper, который ставит профиль Hermes перед вызовом реального `claude`. diff --git a/personal/projects/personal-os/claude-python-cli-proxy.md b/personal/projects/personal-os/claude-python-cli-proxy.md new file mode 100644 index 00000000..3a840971 --- /dev/null +++ b/personal/projects/personal-os/claude-python-cli-proxy.md @@ -0,0 +1,219 @@ +--- +title: claude-code-openai-wrapper — Python Claude CLI Proxy +type: reference +namespace: work +tags: + - hermes + - mac + - eagle + - claude-proxy + - python + - llm-backend + - pitfalls +created: 2026-06-03 +updated: 2026-06-03 +confidence: 0.95 +--- + +# claude-code-openai-wrapper — Python Claude CLI Proxy + +OpenAI-compatible FastAPI server that wraps the `claude` CLI via the official +`claude-agent-sdk`. Runs at **port 8090** on Eagle. Hermes uses it as its LLM +backend (`base_url: http://localhost:8090/v1`). + +## Why this exists + +Hermes needs an OpenAI-compatible endpoint that routes calls through the local +`claude` CLI session (Max/OAuth subscription, no API key). Two wrappers were +evaluated — see below. The Python one is currently active. + +### Node.js wrapper (port 3456) — status: broken streaming, kept as fallback + +**Package:** `openclaw-claude-proxy` v1.0.8 (npm) +**Repo:** [mehdic/openclaw-claude-proxy](https://github.com/mehdic/openclaw-claude-proxy) +**LaunchAgent:** `ai.claude-proxy.plist` → `~/.local/bin/claude-proxy-start.sh` + +**Issue:** uses a reverse-engineered `stream-json` protocol to talk to `claude` +as a long-lived subprocess. This protocol broke with claude CLI ≥ 2.1.141. +Streaming requests stall after the initial role chunk — only keepalives come +through. Non-streaming (`--print` fallback path) still works. + +The proxy has `CLAUDE_PROXY_RUNTIME=stream-json` (default) and +`CLAUDE_PROXY_RUNTIME=print` (fallback). Switching to `print` fixes +hangs but removes true incremental streaming. + +--- + +### Python wrapper (port 8090) — status: **active** + +**Repo:** [RichardAtCT/claude-code-openai-wrapper](https://github.com/RichardAtCT/claude-code-openai-wrapper) +**Local path:** `/Users/admin/openclaw/claude-proxy/` +**Version (pyproject.toml):** 2.2.0 +**SDK:** `claude-agent-sdk` 0.2.88 (updated June 2026 from 0.1.56) +**Bundled CLI:** 2.1.161 (inside SDK venv, used instead of system CLI — see pitfalls) +**LaunchAgent:** `com.openclaw.claude-proxy.plist` → `~/.local/bin/claude-python-proxy-start.sh` + +#### How it works + +``` +Hermes → POST /v1/chat/completions (port 8090) + └── FastAPI (uvicorn) + └── claude-agent-sdk query() + └── spawns .venv/…/_bundled/claude 2.1.161 + --output-format stream-json --verbose + (reads CLAUDE_CODE_OAUTH_TOKEN from env) + → streams AssistantMessage events → SSE chunks +``` + +Key: the SDK uses its own **bundled** `claude` binary (not the system one at +`/opt/homebrew/bin/claude`) because the SDK's stream-json protocol must match +the exact CLI version it was built against. + +#### Hermes config + +`~/.hermes/config.yaml`: +```yaml +model: + default: claude-sonnet-4-6 + provider: custom + base_url: 'http://localhost:8090/v1' +``` + +--- + +## What was fixed (June 2026) + +### 1. Venv Python version mismatch + +Venv was created with Python 3.14.3, Homebrew had upgraded to 3.14.5. +Compiled C extensions (`.so` files) were incompatible → `pip install` crashed +with `ImportError: Symbol not found: _XML_SetAllocTrackerActivationThreshold`. + +**Fix:** rebuilt venv with `uv`: +```bash +cd /Users/admin/openclaw/claude-proxy +uv venv .venv --python python3.14 --clear +uv pip install fastapi "uvicorn[standard]" pydantic python-dotenv httpx \ + sse-starlette python-multipart claude-agent-sdk slowapi +``` + +### 2. SDK too old — bundled CLI mismatch + +`claude-agent-sdk` 0.1.56 bundled `claude` 2.1.92. Between 2.1.92 and +2.1.141+, Anthropic changed how the stream-json protocol is initiated: +- Old: inferred from `--output-format stream-json` +- New: requires `--input-format stream-json` flag explicitly + +Result: SDK sent `control_request` on stdin; new CLI ignored it (no output, +exited code 1). + +**Fix:** `uv pip install claude-agent-sdk` → upgraded to 0.2.88, bundled CLI +2.1.161 which matches the updated SDK protocol. + +### 3. `cli_path=SYSTEM_CLAUDE_PATH` override + +The wrapper hard-coded `cli_path = shutil.which("claude")` (system 2.1.145) +in `ClaudeAgentOptions`. This forced the SDK to use the system CLI instead +of its own bundled binary, breaking the protocol handshake. + +**Fix:** set `SYSTEM_CLAUDE_PATH = None` in `src/claude_cli.py` so the SDK +auto-discovers and uses its bundled CLI. + +### 4. LaunchAgent missing auth token + +The original plist ran `uvicorn` directly and only set `CLAUDE_AUTH_METHOD=cli`. +The `claude` subprocess (spawned by the SDK) inherits the process env, which +had no `CLAUDE_CODE_OAUTH_TOKEN`. CLI responded: *"Not logged in"*. + +**Fix:** added wrapper script `~/.local/bin/claude-python-proxy-start.sh` that +sources `~/.hermes/.env` before starting uvicorn (same pattern as the Node.js +proxy). Updated plist `ProgramArguments` to call the wrapper instead of +uvicorn directly. + +### 5. Expired OAuth token (root cause of 401s) + +`CLAUDE_CODE_OAUTH_TOKEN` in `~/.hermes/.env` expires (access tokens are +short-lived). When the stored token is expired and passed to the CLI +explicitly via env, the CLI uses it as-is → 401 from Anthropic API. The +CLI does **not** auto-refresh when `CLAUDE_CODE_OAUTH_TOKEN` is set to an +expired value in the env. + +**Symptom:** both proxies return `401 Invalid authentication credentials` +even though `claude` works fine in the terminal (terminal has a fresh token +from a previous session). + +**Immediate fix:** copy fresh token from terminal into `~/.hermes/.env`: +```bash +# In the terminal that has a working claude session: +grep CLAUDE_CODE_OAUTH_TOKEN <(env) +# Then update: +sed -i '' "s|CLAUDE_CODE_OAUTH_TOKEN=.*|CLAUDE_CODE_OAUTH_TOKEN=|" \ + ~/.hermes/.env +# Restart both proxies: +launchctl unload ~/Library/LaunchAgents/ai.claude-proxy.plist +launchctl load ~/Library/LaunchAgents/ai.claude-proxy.plist +launchctl unload ~/Library/LaunchAgents/com.openclaw.claude-proxy.plist +launchctl load ~/Library/LaunchAgents/com.openclaw.claude-proxy.plist +``` + +**Long-term:** a token-refresh cron/daemon that keeps `~/.hermes/.env` updated +is needed. Not yet implemented. + +--- + +## Wrapper script + +`~/.local/bin/claude-python-proxy-start.sh`: +```bash +#!/bin/zsh +set -a +source /Users/admin/.hermes/.env 2>/dev/null +set +a +export PATH="/Users/admin/.local/bin:/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:$PATH" +cd /Users/admin/openclaw/claude-proxy +exec .venv/bin/python -m uvicorn src.main:app --host 127.0.0.1 --port 8090 +``` + +## LaunchAgent + +`~/Library/LaunchAgents/com.openclaw.claude-proxy.plist`: +```xml +ProgramArguments + + /bin/zsh + /Users/admin/.local/bin/claude-python-proxy-start.sh + +WorkingDirectory +/Users/admin/openclaw/claude-proxy +StandardOutPath/tmp/claude-proxy.log +StandardErrorPath/tmp/claude-proxy.err +``` + +## Pitfall summary + +| Pitfall | Symptom | Fix | +|---------|---------|-----| +| Token expired in `.env` | 401 from both proxies, terminal claude works | Copy fresh token from terminal to `~/.hermes/.env`, restart proxies | +| Venv Python version mismatch | `pip` crashes, `ImportError` on `.so` | Rebuild with `uv venv --clear`, reinstall deps | +| `cli_path` set to system claude | SDK exits code 1, no response | Set `SYSTEM_CLAUDE_PATH = None` in `src/claude_cli.py` | +| LaunchAgent has no OAuth token | "Not logged in" | Wrapper script must `source ~/.hermes/.env` | +| SDK version too old | CLI protocol mismatch, exit code 1 | `uv pip install claude-agent-sdk` to update | +| Node.js proxy stream-json | Streaming hangs after role chunk | Use Python proxy on 8090 instead | + +## Health check + +```bash +curl http://localhost:8090/health +curl http://localhost:8090/v1/auth/status +# Test streaming: +curl -s -X POST http://localhost:8090/v1/chat/completions \ + -H 'Content-Type: application/json' \ + -d '{"model":"claude-haiku-4-5-20251001","messages":[{"role":"user","content":"hi"}],"stream":true}' +``` + +--- + +## Related + +- [[hermes-eagle-mac]] — full Eagle setup including Zulip, MCP, launchd +- [[hermes-deployment-patterns]] — comparison of Eagle/Kraken deployment models diff --git a/personal/projects/personal-os/eagle-dashboard.md b/personal/projects/personal-os/eagle-dashboard.md new file mode 100644 index 00000000..8c9cb8c3 --- /dev/null +++ b/personal/projects/personal-os/eagle-dashboard.md @@ -0,0 +1,166 @@ +# Eagle Dashboard + +Process supervisor and control panel for all Eagle local services. + +**URL:** `http://localhost:8880` (internal) · `https://dashboard.qentra.top` (external, token-required) +**Source:** `~/Developer/eagle-dash/` +**LaunchAgent:** `com.eagle.dashboard` (single Login Item, KeepAlive=true) + +--- + +## Architecture + +FastAPI + HTMX + Tailwind CDN + Alpine.js. No build step, CDN-only frontend. + +``` +~/Developer/eagle-dash/ +├── main.py # FastAPI app, lifespan, SSE +├── auth.py # Origin-aware auth: 127.0.0.1 → free; cloudflared → token +├── supervisor.py # PID-file-based process supervisor (asyncio, setsid) +├── registry.py # Service registry + YAML loader +├── health.py # Health checks: http / port / process / launchctl +├── control.py # /restart /stop /start endpoints +├── memory.py # RSS tracking (main process + children) +├── loader.py # services.yaml loader +├── services.yaml # Service definitions (source of truth) +├── .env # EAGLE_TOKEN= (not committed) +├── templates/ +│ └── index.html # Dashboard UI — 3 tabs: Services, Pages, Files +└── com.eagle.dashboard.plist +``` + +### Auth + +- `127.0.0.1` direct access → no auth +- Requests via cloudflared (have `X-Forwarded-For`) → require `Authorization: Bearer ` + or `eagle_token` cookie (set on first login, 30-day expiry) +- Token stored in `.env` + +### Supervisor (supervisor.py) + +PID-file-based — survives Dashboard restarts without crashing. + +- Services launched with `start_new_session=True` (setsid) → run in own session +- PID written to `~/.eagle-dash/pids/.pid` immediately on launch +- On Dashboard restart: reads PID file → if process alive → adopts without re-launching +- If process dead → launches fresh +- Auto-restart with exponential backoff (2s → 60s max) + +### macOS integration + +- `pf` anchor: `localhost:80` → `127.0.0.1:8880` (setup-pf-redirect.sh) +- cloudflared tunnel: `dashboard.qentra.top` → `localhost:8880` +- FileBrowser: `http://127.0.0.1:8181` (FB_NOAUTH=true, auth delegated to Dashboard) + +--- + +## Services + +24 services in `services.yaml`, 10 with `autostart: true`. Секция **Docker** — сервисы, запускающиеся через docker-compose — отображается первой в дашборде. + +### Docker + +| ID | Name | Access | Type | +|----|------|--------|------| +| zulip | Zulip Connector | localhost | container | +| filebrowser | FileBrowser | localhost | container | + +### AI + +| ID | Name | Access | Type | +|----|------|--------|------| +| hermes | Hermes Eagle | localhost | daemon | +| hermes_whale | Hermes Whale | localhost | daemon | +| claude_proxy_node | Claude Proxy (node) | localhost | daemon | +| claude_proxy_python | Claude Proxy (python) | localhost | daemon | +| asana_mcp | Asana MCP | localhost | daemon | +| virfield | Virfield | localhost | daemon | + +### ML + +| ID | Name | Access | Type | +|----|------|--------|------| +| mlx_lm | MLX LM Server | lan (0.0.0.0:8080) | on-demand | +| ollama | Ollama | lan (0.0.0.0:11434) | on-demand | + +### Reflect + +| ID | Name | Access | Type | +|----|------|--------|------| +| reflect_proxy | Reflect Proxy | lan | daemon | +| reflect_fdroid | Reflect F-Droid Server | lan | daemon | + +### Monitoring + +| ID | Name | Access | Type | +|----|------|--------|------| +| aw_ddg | AW Watcher DDG | localhost | daemon | +| aw_xcode | AW Watcher Xcode | localhost | daemon | + +### Orchestration + +| ID | Name | Access | Type | +|----|------|--------|------| +| wiki_ingest | Wiki Ingest | localhost | cron | + +**Remaining LaunchAgents (not under Dashboard):** +- `com.colima.start` — Homebrew managed +- `homebrew.mxcl.postgresql@17` — Homebrew managed +- `personal.os.wiki-ingest` — cron schedule +- `com.personalos.heartbeat` — cron schedule + +--- + +## Network Access Badges + +- 🟢 `localhost` — bound to 127.0.0.1, safe +- 🟡 `lan` — bound to 0.0.0.0, accessible on home network (mlx_lm, ollama) +- 🔵 `tunnel` — cloudflared-managed, token-protected + +--- + +## Tabs + +**Services** — health cards with: status badge, PID, uptime, memory (RSS + children), start/stop/restart controls, log tail drawer (last 200 lines). + +**Pages** — URL input + iframe for quick local page testing. + +**Files** — FileBrowser iframe at `http://localhost:8181`. + +--- + +## Deployment + +```bash +cd ~/Developer/eagle-dash + +# Install deps +uv sync + +# Generate token +python3 -c "import secrets; print('EAGLE_TOKEN='+secrets.token_hex(32))" > .env + +# Setup pf redirect (localhost:80 → :8880) +sudo bash setup-pf-redirect.sh + +# Load LaunchAgent +launchctl load ~/Library/LaunchAgents/com.eagle.dashboard.plist +``` + +--- + +## Troubleshooting + +**Services show `running: null`** — check `supervisor.supervisor_status` field in `/api/services` (not top-level `running`). The supervisor status is in the nested `supervisor` object. + +**PID files stale** — `ls ~/.eagle-dash/pids/` and compare with `ps`. Supervisor adopts automatically on next poll. + +**Ghost launchctl entries** — after removing plists, use `launchctl remove