2026-04-28: vault catchup — purpose, self-modification, CI pipeline, agent rules

- Add personal-os-purpose.md: ADHD context, design principles, quadrant model
- Add personal-os-self-modification.md: Eagle's guide for evolving the system
- Update personal-os-agent-rules.md: fix status.md path, corrections_log → SQL, add #retrospector cron, personal project activity notes
- Update personal-os-architecture.md: correct paths, add executor/tables/data flow
- Add work/wiki/apple-browsers/ci-pipeline.md: full CI schema, xcodebuild commands, local equivalents, VM/executor build commands
- Add raw/ symlinks to planning docs (state-2026-04-27-final, catchup-plan-2026-04-27)
This commit is contained in:
Alexey Martemyanov
2026-04-28 01:01:06 +06:00
parent f280819df4
commit 9303eb1bb7
8 changed files with 607 additions and 44 deletions
+34 -11
View File
@@ -1,7 +1,7 @@
---
namespace: work
tags: [system, agent, eagle, rules]
last_synced: 2026-04-27
last_updated: 2026-04-28
confidence: 1.0
---
@@ -14,17 +14,17 @@ Eagle (Орёл) is the Discord-facing Hermes agent. This page documents what Ea
```
Asana API
→ sync.js (every 30 min, workdays via Hermes cron)
→ PostgreSQL (tasks, stories, task_edges, sync_state)
→ PostgreSQL (tasks, stories, task_edges, sync_state, task_annotations)
→ generate-status.js
→ ~/Developer/personal-os/asana_context.md ← raw Asana data
→ ~/Developer/personal-os/status.md ← Eagle reads THIS
→ ~/Developer/personal-os/asana_context.md ← raw Asana context (inbox-check)
→ ~/context/status.md ← Eagle reads THIS
```
Eagle **only reads** `status.md`. It never touches the DB directly or calls Asana API.
Eagle **only reads** `~/context/status.md`. It never touches the DB directly or calls Asana API.
## Eagle's Knowledge Sources
1. `~/Developer/personal-os/status.md` — live Asana snapshot (updated every 30 min)
1. `~/context/status.md` — live Asana + ActivityWatch snapshot (updated every 30 min)
2. `~/Developer/personal-os/briefs/daily/YYYY-MM-DD.md` — today's brief
3. `~/Developer/personal-os/briefs/weekly/plan-MONDAY.md` — weekly plan
4. `~/obsidian/wiki/` — knowledge base (via obsidian MCP)
@@ -35,13 +35,25 @@ Eagle **only reads** `status.md`. It never touches the DB directly or calls Asan
When Alex corrects something:
1. Edit the brief directly (show diff first, confirm before writing)
2. If systemic → edit the prompt template in `agent/prompts/`
3. Log to `~/Developer/personal-os/logs/corrections_log.md`: `YYYY-MM-DD HH:MM | file | what changed`
3. Log correction silently to `corrections_log` table:
```sql
INSERT INTO corrections_log (date, week_number, source, original_plan, correction, deferred_gids, reason_tag)
VALUES (current_date, EXTRACT(WEEK FROM current_date)::int,
'<source>', '<original>', '<user_message>',
ARRAY[<gids>]::text[], '<reason_tag>');
```
`source` values: `discord_pushback` | `morning_brief_correction` | `manual`
`reason_tag` examples: `ship-review-crunch` | `urgent-bug` | `meeting-day` | `over-estimated` | `scope-change`
Never announce the log insertion. It is silent instrumentation.
## Self-Diagnostics
If pipeline seems stale, Eagle can check:
```bash
stat -f "%Sm" ~/Developer/personal-os/status.md
stat -f "%Sm" ~/context/status.md
# Should be < 40 min old on workdays
```
@@ -67,9 +79,10 @@ bash ~/scripts/run-wiki-ingest.sh
| inbox-check | */30 9-19 workdays | #inbox |
| weekly-plan | Mon 08:00 | #daily-brief |
| weekly-review | Fri 17:00 | #daily-brief |
| retrospector | Fri 17:30 | #retrospector |
| commit-vault | 23:00 daily | silent |
wiki-ingest runs at 22:00 via **launchd** (not Hermes) — because API-based Claude has no Mac filesystem access.
wiki-ingest runs at 22:00 via **launchd** (not Hermes) — because it needs Mac filesystem access.
## Allowed File Writes
@@ -79,6 +92,16 @@ Eagle can write to:
- `briefs/weekly/YYYY-MM-DD.md`
- `briefs/inbox/YYYY-MM-DD.md`
- `agent/prompts/*.md` (with confirmation)
- `logs/corrections_log.md`
- `~/obsidian/work/projects/SLUG.md`
- `~/obsidian/work/decisions/YYYY-MM-TOPIC.md`
- `~/obsidian/personal/`
- `~/obsidian/family/`
Never write to `obsidian/wiki/` directly — that's wiki-ingest's job.
Never write to `obsidian/wiki/` directly — that's wiki-ingest's job.
Never write `corrections_log.md` — use the SQL INSERT above (table, not file).
## Activity Classification
Work activity is classified via `aw-projects.json`. Projects with `namespace: "personal"` (e.g., AXPressDeck, media_files_db) appear as a footnote in the Activity section of status.md but are **not** counted as work time and **not** upserted to `activity_daily`.
To re-classify a project (personal ↔ work), ask Eagle to edit `aw-projects.json`.
+123 -32
View File
@@ -1,44 +1,135 @@
---
namespace: work
tags: [system, architecture]
last_synced: 2026-04-27
last_updated: 2026-04-28
confidence: 1.0
---
# Personal OS Architecture
# Personal OS Architecture
## Стратег (Claude Code, локально)
- `~/scripts/run-pipeline.sh` → sync.js + generate-status.js каждые 30 мин (workdays)
- `~/Developer/personal-os/asana_context.md` — сырые данные Asana
- `~/Developer/personal-os/status.md` — живой срез задач для Орла
- `~/scripts/run-wiki-ingest.sh` → обновляет ~/obsidian/wiki/ (22:00, launchd)
## Why It Exists
## Орёл (Hermes v0.11.x, Discord бот)
- Читает только `~/Developer/personal-os/status.md` — никогда не идёт в БД напрямую
- Discord каналы: #daily-brief #inbox #focus #executor #projects #journal
- Ищет знания через obsidian MCP по vault
- Логирует корректировки в `~/Developer/personal-os/logs/corrections_log.md`
- Cron jobs: morning-brief (08:30), eod-summary (18:00), inbox-check (30 мин), weekly-plan (Пн 08:00), weekly-review (Пт 17:00)
See: `personal-os-purpose.md`
## Vault (~/obsidian/)
- `wiki/` — LLM-синтез, генерируется wiki-ingest, не редактировать руками
- `raw/` — symlinks на внешние файлы проекта
- `work/`, `personal/`, `family/` — заметки, редактировать свободно
- Git репозиторий, автокоммит в 23:00 с агентным summary
## Agents
## База данных (Postgres, только Asana + activity)
- tasks, stories, task_edges, sync_state
- signal_queue, activity_daily
- Нет wiki данных — всё в vault файлах
### Стратег (Strategist) — data pipeline, no conversation
- `~/scripts/run-pipeline.sh` → sync.js + generate-status.js every 30 min (workdays via Hermes cron)
- Writes `~/context/status.md` and `~/Developer/personal-os/asana_context.md`
- Writes daily AW activity summaries to `activity_daily` table with quadrant classification
- No Discord output — pure data layer
## Почему wiki-ingest через launchd, не Hermes
Claude через API не имеет доступа к файловой системе Mac.
wiki-ingest пишет файлы в ~/obsidian/wiki/ — это требует
`claude -p` запущенного локально, не API вызова.
### Орёл / Eagle (Tactician + reactive layer) — Hermes v0.11.x Discord bot
- Reads `~/context/status.md`, daily/weekly briefs, vault via obsidian MCP
- Discord channels: `#daily-brief` `#inbox` `#focus` `#executor` `#projects` `#journal` `#retrospector`
- Cron jobs: morning-brief (08:30 MTWRF), eod-summary (18:00 MTWRF), inbox-check (30 min 0919 MTWRF), weekly-plan (Mon 08:00), weekly-review (Fri 17:00), retrospector (Fri 17:30), commit-vault (23:00 daily)
- Logs corrections to `corrections_log` table (SQL INSERT, silently on pushback)
- **Never accesses Asana API directly** — only reads pre-rendered markdown and DB
## Файлы конфигурации
- `~/.hermes/config.yaml` Hermes config, модель, Discord, cron
- `~/.hermes/SOUL.md` — характер и правила Орла
- `~/.hermes/auth.json` — Claude Code OAuth (credential_pool)
- `~/Developer/personal-os/config.json` — Asana workspace/user GIDs
- `~/.config/personal-os/env` — env vars (ASANA_API_KEY, POSTGRES_URL)
### Ретроспектор (Retrospector) — Friday pattern analysis
- Runs as separate Hermes cron at 17:30 Friday → `#retrospector`
- Sources: corrections_log table, status.md subtask counts, obsidian git log, activity_daily quadrants, stories visibility metric
- Prompt: `~/Developer/personal-os/agent/prompts/retrospector.md`
### Исполнитель (Executor) — on-demand autonomous worker
- Triggered by explicit "fix bug [GID]" in #executor — never autonomous
- Creates git worktree at `~/DuckDuckGo/apple-browsers.git/.claude/worktrees/executor-{gid}-{slug}/`
- Single STOP gate after analysis; then autonomous: fix → build → test → draft PR → CI loop
- State machine in `executor_runs` table
- Prompt: `~/Developer/personal-os/agent/prompts/executor-bug-fix.md`
- For UI tests: spins VM via ddg-vm MCP (virfield), cleans up on PR close/merge
## Data Flow
```
Asana API
→ sync.js (30 min, Hermes cron, silent)
→ PostgreSQL: tasks, stories, task_edges, sync_state, task_annotations
ActivityWatch (local daemon)
→ generate-status.js (reads AW HTTP API)
→ activity_daily (upserts daily project/quadrant summaries)
generate-status.js
→ ~/context/status.md ← Eagle reads this
→ ~/Developer/personal-os/asana_context.md ← inbox-check reads this
Eagle corrections in Discord
→ corrections_log table (INSERT on pushback, no announcement)
→ Retrospector reads weekly
Executor runs
→ executor_runs table (state machine)
→ ~/DuckDuckGo/apple-browsers.git/.claude/worktrees/
→ GitHub draft PRs (--draft, --assignee @me, no reviewers)
```
## Key File Locations
| File / Path | Purpose |
|-------------|---------|
| `~/context/status.md` | Live Asana + AW snapshot. Written by generate-status.js. Eagle reads this. |
| `~/Developer/personal-os/asana_context.md` | Raw Asana context for inbox-check |
| `~/Developer/personal-os/agent/prompts/` | All agent prompts (Hermes cron + on-demand) |
| `~/.hermes/SOUL.md` | Eagle's identity, rules, vault write permissions |
| `~/Developer/personal-os/config.json` | Asana workspace/user GIDs, section GIDs |
| `~/Developer/personal-os/aw-projects.json` | ActivityWatch project classification rules |
| `~/.hermes/config.yaml` | Hermes config: model, MCP servers, cron settings |
| `~/obsidian/` | Vault — git repo, NAS remote, obsidian-mcp for search |
| `~/Developer/personal-os/logs/` | heartbeat.log, wiki-ingest-YYYY-MM-DD.md |
| `~/Developer/personal-os/briefs/` | daily/, weekly/, inbox/ — agent-written brief files |
| `~/DuckDuckGo/apple-browsers.git/` | Browser repo (bare), worktrees as sibling dirs |
| `~/DuckDuckGo/apple-browsers.git/.claude/worktrees/` | Executor worktrees |
## Database Tables
| Table | Written by | Read by |
|-------|-----------|---------|
| tasks | sync.js | generate-status.js, executor, status queries |
| stories | sync.js | generate-status.js, executor, retrospector |
| task_edges | sync.js | subtask traversal |
| sync_state | sync.js | generate-status.js header |
| task_annotations | manual / Eagle | generate-status.js irrelevant filter |
| activity_daily | generate-status.js | retrospector quadrant drift |
| corrections_log | Eagle on pushback | retrospector pattern analysis |
| executor_runs | executor | generate-status.js active runs, executor state machine |
## MCP Servers
**Hermes agents:**
| Name | Command | Used for |
|------|---------|---------|
| obsidian | `/opt/homebrew/bin/obsidian-mcp ~/obsidian` | Vault search, read, write |
**Claude Code / `claude -p` sessions:**
| Name | Command | Used for |
|------|---------|---------|
| ddg-vm | `npx tsx ~/Developer/virfield/server/mcp-server.ts` | VM lifecycle for UI tests |
| obsidian | `/opt/homebrew/bin/obsidian-mcp ~/obsidian` | Vault access |
## launchd Agents
| Label | Schedule | Purpose |
|-------|----------|---------|
| personal.os.heartbeat | :05 every hour | Watchdog — triggers missed Hermes jobs after wake |
| personal.os.wiki-ingest | 22:00 daily | `claude -p` wiki synthesis (needs filesystem access) |
## Vault Structure
```
~/obsidian/
├── wiki/ ← LLM-generated + hand-written (confidence: 1.0 = never overwrite)
│ ├── personal-os-architecture.md (this file)
│ ├── personal-os-purpose.md
│ ├── personal-os-agent-rules.md
│ ├── personal-os-self-modification.md
│ ├── personal-os-sync-pipeline.md
│ ├── personal-os-schema.md (auto-generated)
│ ├── ddg-asana-workflow.md
│ └── vault-filling-guide.md
├── raw/ ← symlinks to external source files (schema.sql, etc.)
├── work/
│ └── wiki/
│ └── apple-browsers/ ← .cursor/rules copies (.md) + vm-ui-testing.md
└── personal/ family/
```
+73
View File
@@ -0,0 +1,73 @@
---
namespace: work
tags: [system, purpose, adhd, design]
last_updated: 2026-04-28
confidence: 1.0
---
# Personal OS — Purpose
## Why It Exists
Alex is a macOS Browser Developer / DRI at DuckDuckGo. Async-first company, primary channels Asana and Slack. ADHD profile.
The Personal OS exists to hold work context **outside Alex's head** — so that context isn't lost during deep focus, transitions, or when switching back from a meeting.
Without it, three things happen regularly:
1. A comment sits on an Asana task for days — Alex never saw it because he was in Xcode
2. A career-level task (Ship Review, Tech Design review) gets deferred until it becomes a bottleneck
3. End of day: "what did I actually do today?" — no clear answer, no signal for tomorrow
## Design Principles
**Principle 1 — Context is the product, not notifications.**
The system's job is to maintain a live, accurate picture of what's happening. Eagle reads that picture and speaks when something actually matters. It is not a notification firehose.
**Principle 2 — Protect focus.**
If Alex is in Xcode or Cursor, don't interrupt unless there's a red signal older than 3 hours. ADHD makes context switches expensive — the system should absorb noise, not amplify it.
**Principle 3 — No hallucinated state.**
Eagle never guesses about Asana tasks. All Asana data flows through sync.js → PostgreSQL → generate-status.js → status.md. Eagle reads markdown, not the API. This means the data is always consistent and never stale by more than 30 minutes.
**Principle 4 — Career goals are first-class.**
The system explicitly tracks: Project Advisor opportunities, visibility gaps (Ship Reviews where Alex hasn't commented), Tech Design reviews pending. These surface in status.md and the daily brief. Without explicit tracking they'd be invisible.
**Principle 5 — No new infra.**
The system has enough moving parts. New capabilities go into existing components: Postgres tables, generate-status.js queries, prompt files in `agent/prompts/`, SOUL.md rules. No new YAML files, no new shell scripts, no new launchd plists unless there is no other way.
## The Four Agents
| Agent | Role | When |
|-------|------|------|
| **Стратег (Strategist)** | Data pipeline — reads Asana + ActivityWatch, writes status.md | Every 30 min, workdays |
| **Орёл / Eagle (Tactician)** | Discord-facing reactive agent — briefs, inbox, focus gating | Cron + on-demand |
| **Ретроспектор (Retrospector)** | Weekly pattern analysis — corrections, quadrant drift, visibility | Friday 17:30 |
| **Исполнитель (Executor)** | On-demand bug-fix worker — analysis → fix → PR → CI loop | Explicit trigger only |
Eagle is the only agent Alex directly talks to. The others are background infrastructure.
## What "Work Context" Means
The system tracks four categories, mapped to `activity_daily.quadrant`:
| Quadrant | What it covers | Why it matters |
|----------|---------------|----------------|
| `project` | The current top-priority feature/bug | Time here should dominate most weeks |
| `aor` | AOR maintenance: code review, bug fixes, blocklist | Required to keep the area healthy |
| `career` | Project Advisor, Ship Reviews, Tech Design, visibility | Easily deferred; system forces it into view |
| `strategy` | Planning, Personal OS, Asana organisation | Meta-work; needs a floor, not a ceiling |
Personal pet projects (AXPressDeck, media_files_db, etc.) are tracked separately and not counted as work time. See `aw-projects.json` (`namespace: "personal"` entries).
## What It Does Not Do
- Does not make decisions for Alex — it informs
- Does not send notifications during focus (deep focus gate in inbox STEP 0)
- Does not access Asana API directly from Eagle — only reads pre-rendered status.md
- Does not create PRs, commits, or messages without explicit confirmation (except Executor, after its single approval gate)
## See Also
- `personal-os-architecture.md` — full technical architecture, agents, data flow, file locations
- `personal-os-agent-rules.md` — Eagle's operating rules and allowed writes
- `personal-os-self-modification.md` — how Eagle (and Alex) can evolve the system
+152
View File
@@ -0,0 +1,152 @@
---
namespace: work
tags: [system, self-modification, eagle, meta]
last_updated: 2026-04-28
confidence: 1.0
---
# Personal OS — Self-Modification Guide
How Eagle (and Alex via Claude Code) can safely evolve the system without breaking it.
**Rule**: Always show a diff/draft first, confirm, then apply. Commit the vault after.
---
## What Eagle Can Change (no approval needed — show draft first)
### 1. Prompt files (`~/Developer/personal-os/agent/prompts/*.md`)
Eagle reads and writes its own prompts. When Alex says "add X to the morning brief" or "change how inbox triage handles Y":
1. Read the current prompt: `~/Developer/personal-os/agent/prompts/<name>.md`
2. Draft the change, show the diff in Discord
3. On confirmation: write the file
4. Commit: `cd ~/Developer/personal-os && git add agent/prompts/<name>.md && git commit -m "[YYYY-MM-DD] prompt: <description>"`
**Do not** change `executor-bug-fix.md` without explicit confirmation — it controls autonomous code changes.
### 2. SOUL.md (`~/.hermes/SOUL.md`)
Eagle's identity, rules, and operating context. Eagle can propose changes to:
- Allowed vault write paths
- Focus gating rules
- Channel routing
Same flow: draft → confirm → write → no vault commit needed (SOUL.md is outside vault).
### 3. ActivityWatch project rules (`~/Developer/personal-os/aw-projects.json`)
To classify a new project as work or personal:
1. Read current `aw-projects.json`
2. Add entry with `project`, `namespace` (omit for work, `"personal"` for pet projects), and `rules`
3. Show draft, confirm, write
4. Commit: `cd ~/Developer/personal-os && git add aw-projects.json && git commit -m "[YYYY-MM-DD] aw: classify <project>"`
The `namespace: "personal"` field excludes a project from work activity in status.md and activity_daily.
### 4. Vault knowledge pages (`~/obsidian/wiki/`, `~/obsidian/work/wiki/`)
Eagle does **not** write to `wiki/` — that's the wiki-ingest job (22:00 launchd). But Eagle **can** write to:
- `~/obsidian/work/projects/SLUG.md` — project notes
- `~/obsidian/work/decisions/YYYY-MM-TOPIC.md` — decisions
- `~/obsidian/personal/` — personal notes
- `~/obsidian/family/` — family docs
After writing: `cd ~/obsidian && git add -A && git commit -m "[YYYY-MM-DD] <description>"`
### 5. Correction logging (corrections_log table)
Eagle silently logs a row whenever Alex pushes back on a suggestion. No approval needed — this is continuous background instrumentation, not a visible change.
```sql
INSERT INTO corrections_log (date, week_number, source, original_plan, correction, deferred_gids, reason_tag)
VALUES (current_date, EXTRACT(WEEK FROM current_date)::int,
'<source>', '<original>', '<user_message>',
ARRAY[<gids>]::text[], '<reason_tag>');
```
---
## What Requires Approval (always confirm explicitly)
### Schema changes (PostgreSQL)
New tables, columns, or indexes require a SQL migration. Pattern:
1. Draft the SQL in Discord, explain the purpose
2. On "go": run via `psql personal_os -c "..."`
3. Update `~/obsidian/wiki/personal-os-schema.md` (this is auto-generated at 22:00, but a manual update is fine)
Current tables: tasks, stories, task_edges, sync_state, task_annotations, activity_daily, corrections_log, executor_runs.
### New Hermes cron jobs
Add to `~/.hermes/config.yaml`. Format:
```yaml
crons:
- id: my-new-job
schedule: "30 9 * * 1-5"
prompt: |
Run: ~/Developer/personal-os/agent/prompts/my-prompt.md
channel: "#channel-name"
```
Must confirm before writing — a broken cron syntax silently prevents Hermes from starting.
### New launchd agents (`~/Library/LaunchAgents/`)
New plist files. Rarely needed — only if a job requires Mac filesystem access and can't run inside Hermes (e.g., wiki-ingest). Confirm before writing and before loading with `launchctl`.
### Executor prompt (`executor-bug-fix.md`)
Controls autonomous code changes. Changes here need explicit approval because a mistake could cause the Executor to behave incorrectly on real PRs.
### Hermes model or MCP configuration (`~/.hermes/config.yaml`)
Model upgrades, new MCP servers. Show the diff; confirm before writing.
---
## How generate-status.js Gets Extended
When a new data signal should appear in status.md:
1. Add a SQL query function (e.g., `getNewSignal()`)
2. Add a render function (e.g., `renderNewSignal(data)`)
3. Add both to `main()`: Promise.all for the query, the render call in `parts`
4. Test: `node ~/Developer/personal-os/generate-status.js` and read `~/context/status.md`
5. Commit
**Never** edit generate-status.js to change how Asana sync works — that's sync.js territory.
---
## Debugging the System
| Symptom | Check |
|---------|-------|
| status.md is stale (> 40 min old) | `stat -f "%Sm" ~/context/status.md` — if old, run `bash ~/scripts/run-pipeline.sh` |
| Hermes cron didn't fire | Check heartbeat: `tail -20 ~/Developer/personal-os/logs/heartbeat.log` |
| Eagle gave wrong Asana data | The DB may be stale — run `node ~/Developer/personal-os/sync.js` manually |
| activity_daily missing today | generate-status.js runs upsert — check `select * from activity_daily where bucket_day = current_date` |
| corrections_log empty | Eagle only inserts on pushback — expected to be sparse |
| Executor in stuck state | `SELECT id, state, updated_at FROM executor_runs WHERE state NOT IN ('complete','abandoned');` |
---
## Safe Operations Checklist
Before making any system change:
- [ ] Read the current file/config before proposing a change
- [ ] Show the diff, not just a description
- [ ] Wait for explicit confirmation
- [ ] Apply the change
- [ ] Verify: run the component or check output
- [ ] Commit if in a git repo (`personal-os` or `obsidian`)
---
## See Also
- `personal-os-architecture.md` — full system map
- `personal-os-agent-rules.md` — Eagle's rules and channel routing
- `personal-os-purpose.md` — why this system exists