From 5c9c895d831f85d91d9f6e709ed53ffabe5c9638 Mon Sep 17 00:00:00 2001 From: Alexey Martemyanov Date: Sat, 30 May 2026 23:00:25 +0600 Subject: [PATCH] [2026-05-30] personal os schema, state snapshot, catchup plan, wiki ingest process --- wiki/personal-os-catchup-plan-2026-04-27.md | 141 ++++++++++++++++++++ wiki/personal-os-schema.md | 115 ++++++++++++++++ wiki/personal-os-state-2026-04-27.md | 134 +++++++++++++++++++ wiki/wiki-ingest-process.md | 114 ++++++++++++++++ 4 files changed, 504 insertions(+) create mode 100644 wiki/personal-os-catchup-plan-2026-04-27.md create mode 100644 wiki/personal-os-schema.md create mode 100644 wiki/personal-os-state-2026-04-27.md create mode 100644 wiki/wiki-ingest-process.md diff --git a/wiki/personal-os-catchup-plan-2026-04-27.md b/wiki/personal-os-catchup-plan-2026-04-27.md new file mode 100644 index 00000000..ecee532d --- /dev/null +++ b/wiki/personal-os-catchup-plan-2026-04-27.md @@ -0,0 +1,141 @@ +--- +source: raw/personal-os-catchup-plan-2026-04-27.md +content_hash: 5081ceb719104e1b7c6b5edd642c1daf38b3fd7cf1a96442a00516e7cdb15379 +namespace: work +last_synced: 2026-05-30 +confidence: 0.9 +tags: [personal-os, plan, executor, catchup, 2026-04-27] +--- + +# Personal OS — Catch-Up Plan (2026-04-27) + +Handoff document for the executor agent: what to build next, in what +order, and why. Companion to [[personal-os-state-2026-04-27]] which +tracks live state. By 2026-04-28 every item below shipped except the +partial MS365 MCP wiring (item 1.7). + +## Context + +Alex is a macOS Browser DRI at DuckDuckGo with an ADHD profile and a +fully-async workflow (Asana + Slack). The Personal OS holds work context +outside Alex's head — preventing lost comments, runaway projects, and +silent collapse of the career quadrant. Original four-agent design from +`personal-os-design-1.md`: + +1. **Стратег** — scheduled Claude reads Asana, writes `status.md`. +2. **Тактик** — reactive layer over `status.md` + ActivityWatch + + Calendar; protects focus. +3. **Ретроспектор** — Friday-only, multi-week patterns. +4. **Исполнитель** — on-demand worker that drafts diffs and comments. + +Pre-plan status: Стратег ✅, Тактик 🟡 (no focus gating, no Calendar), +Ретроспектор ✅, Исполнитель 🟡 (v1 only, manual STOP gates). + +**Architectural principle (re-confirmed):** no new YAML, no new shell +scripts, no new launchd plists. Everything extends what already exists — +Postgres tables, `generate-status.js` queries, prompt files in +`agent/prompts/`, Eagle's SOUL.md rules. + +## Deliverables (dependency-ordered) + +### 1.1 Schema migration (single SQL file) +Adds `corrections_log` (date, week_number, source, original_plan, +correction, deferred_gids[], reason_tag), `executor_runs` (state machine +for autonomous executor: `analyzing → awaiting_analysis_approval → fixing +→ pr_open → ci_running → ci_failed → awaiting_user → complete`), and a +`quadrant TEXT` column on `activity_daily` +(`project|aor|career|strategy|other`). Tables over markdown because +Retrospector needs aggregates, not narrative. See [[personal-os-schema]]. + +### 1.2 `generate-status.js` — three new query blocks +Career advisor pipeline (open tasks where notes mention "Project +Advisor: Set the custom field"), visibility gaps (Assessments / O-N / +O-L tasks where Alex hasn't commented in 7+ days), and quadrant +classification at AW write time (project→quadrant map; top-priority +project is dynamic via config). Empty sections are omitted, not labelled +"No opportunities" — noise discipline. + +### 1.3 `inbox-triage.md` STEP 0 — focus gating + scope filter +Prompt-level gate (not a wrapper script) so the agent retains override +judgment. Reads latest `activity_daily` focus_score / current_app, plus +MS365 calendar for active meetings. Skips when in a meeting, or when +focus_score > 60 in Xcode/Cursor/VSCode for 20+ minutes (unless +`status.md` has 🔴 sections older than 3h). Scope filter: inbox = NEW +signal only; overdue stays in daily-brief. + +### 1.4 `retrospector.md` — three SQL blocks +Correction patterns (`reason_tag` counts over 4 weeks ≥ 3), +quadrant drift (4-week × 4-quadrant pivot; flag career under 5% for 3+ +weeks), and visibility-this-week (count of comments in +Assessment/Objective tasks vs prior 4 weeks). + +### 1.5 Eagle SOUL.md — `corrections_log` writeback +On any pushback in any channel: Eagle identifies pushback type and a +hyphen-tagged `reason_tag`, then silently `INSERT`s into +`corrections_log` before responding. Capture happens at the moment of +correction; no separate hook needed. + +### 1.6 Executor v2 — draft-PR autonomous flow +Replaces v1 (STOP gates everywhere) with a **single** STOP gate after +the Analysis phase, then autonomous through build → test → branch → +push → `gh pr create --draft --assignee @me` (no reviewers) → +self-review → CI loop. Hard prohibitions: never push to +main/develop/release/*, never open a non-draft PR, never `--reviewer`, +never `gh pr merge`, never touch `.github/` or CI configs without +explicit thread confirmation. CI loop polls every 10 min; max 3 fix +attempts before state→`awaiting_user`. + +### 1.7 MS365 MCP in Hermes +Wire `outlook_calendar_search` into Hermes for `inbox-triage`, +`weekly-plan`, and `daily-brief`. STEP 0 query: events with start ≤ now ≤ +end + 1h. By 04-28 this was still 🟡 — cloud connector works in Claude +Code, no local npm package yet. + +### 1.8 Vault consolidation +Process docs + UI testing skill + `apple-browsers/.cursor/*.mdc` files +all moved into `~/obsidian/work/wiki/`. Rule of thumb: process docs → +vault, runtime configs → original location. `.mdc` files are **copied** +(not moved) so Cursor IDE still finds them at the expected path. Each +moved file is verified by re-running the agent prompt or skill that +references it. + +### 1.9 Executor-specific PR template (Option B inline) +Team's `.cursor/*.mdc` template requires manual confirmation of task / +reviewer / description — all known by definition in the autonomous flow. +Executor generates the PR body inline: Task (Asana link from +`executor_runs.task_gid`), Root Cause (from analysis), Fix Summary, +Testing block (build + unit + UI), Self-review checklist, Notes for +review. Draft state IS the "needs polish" signal. + +## Order of execution + +1, 2 → 3 → 4, 5 → 6, 7 → 8 → 9 → 10. Items 1-9 are independent in +practice; 10 (Executor v2 prompt) needs the schema (1), vault +consolidation (8), and PR template (9). **No new Discord channels +required** — existing seven channels cover every surface. + +## Why these choices + +- **Postgres over markdown / YAML / shell**: extends existing components, + aggregable, recoverable after crashes. +- **STEP 0 in the prompt, not a wrapper**: ADHD-aware override at the + edge cases ("focus is high but Dominik is tagging you directly"). +- **Career as a forcing function**: when not surfaced every week, the + career quadrant predictably collapses to 0%. +- **Draft PR > Discord thread**: durable surface that doesn't scroll away. +- **Single STOP gate**: root-cause identification is the consequential + decision; mechanical CI re-pushes don't need human approval. +- **`executor_runs` table**: enables crash recovery, brief surfacing, + and Retrospector throughput counts. + +## Definition of done + +Six observable signals: targeted Retrospector callouts, signal-dense +inbox channel, populated Career section in `status.md`, single-row +`corrections_log` writes on pushback, autonomous draft PRs from +`#executor` within 30 min, and vault-resident process docs reachable +via obsidian-mcp. + +## Related +[[personal-os-state-2026-04-27]] [[personal-os-schema]] +[[wiki-ingest-process]] [[ddg-asana-workflow]] diff --git a/wiki/personal-os-schema.md b/wiki/personal-os-schema.md new file mode 100644 index 00000000..a4f28758 --- /dev/null +++ b/wiki/personal-os-schema.md @@ -0,0 +1,115 @@ +--- +source: raw/schema.sql +content_hash: 71b47e47b483834c887de14c76d5b16506d90ad64198d76a92a7b3731132dab8 +namespace: work +last_synced: 2026-05-30 +confidence: 0.9 +tags: [personal-os, postgres, schema, asana] +--- + +# Personal OS — Postgres Schema + +The Personal OS Postgres database holds Asana sync state, agent annotations, +file references for wiki ingest, the LLM-synthesised wiki, and a semantic +memory store. Extensions required: `vector` (pgvector) and `pg_trgm`. + +## Core Asana tables + +### `tasks` +One row per Asana task GID; upserted on every sync. Tracks identity, due +dates, completion, assignee, project, and a `source` enum that records how +the task was discovered: +- `my_tasks` — assigned to me, lives in a My Tasks section +- `following` — I'm a follower (commented, CC'd) +- `delegated` — I created it, someone else owns it +- `project` — visible via a project I'm on, none of the above + +Bookkeeping columns: `fetched_at`, `stories_fetched_before` (cursor used so +we only pull new stories), `raw_json` (fallback for unmodeled fields). +Indexes cover `modified_at`, `due_on` (partial — incomplete only), +`assignee_gid`, `source`, `completed`, and a trigram index on `name` for +fuzzy match. + +### `stories` +Raw Asana event log per task. The key resource subtypes are +`comment_added`, `assigned`, `due_date_changed`, `section_changed`, +`dependency_added`, `attachment_added`, `marked_complete`. Both plain text +and raw HTML are stored — HTML is needed to extract embedded task GIDs. + +### `task_edges` +Directed graph of relationships between tasks. `relation_type` covers +`subtask`, `dependency` (blocked by), `dependent` (blocking), `project_sibling`, +and `mention` (referenced inside a story). `related_gid` may not yet +exist in `tasks`, so it's not a FK. + +### `task_annotations` +Agent-written notes about tasks. `annotation_type` is one of `irrelevant`, +`watching`, `needs_action`, `snoozed`. `annotated_by` is `agent` or `user`. + +### `sync_state` +One row per logical sync stream (`my_tasks`, `following`, `delegated`, +`project:{gid}`, `workspace_events`). Stores `cursor` (ISO timestamp or +events `sync_token`) and `full_sync_at` so partial syncs can fall back to +a full pull when needed. + +### `task_embeddings` +1024-dim Voyage embedding per task, indexed via HNSW with cosine ops. + +### `active_tasks` view +Convenience view: non-completed tasks that aren't annotated `irrelevant` +or currently snoozed. Ordered by source priority (my_tasks → delegated → +following → project), then `due_on`, then `modified_at`. + +## Phase 0 additions (April 2026) + +All core tables gained a `namespace TEXT NOT NULL DEFAULT 'work'` column — +the three allowed values are `work`, `personal`, `family`. Applied via +ALTER TABLE; documented as commented-out statements at the bottom of the +file for fresh installs. + +`tasks` also gained `possibly_deleted BOOLEAN` and `last_seen_in_full_sync +TIMESTAMPTZ` — used by `generate-status.js` to filter out tasks that have +disappeared from Asana between full syncs. + +## Wiki + file ingest tables + +### `file_references` +Tracks files and URLs that should be ingested into the wiki. Holds a macOS +security-scoped `bookmark_data` BYTEA, `last_known_path` (resolved +cache), optional `url`, `content_hash` (SHA256), `mime_type`, `title`, and +a `modification_log` JSONB of `{ts, hash_before, hash_after}` entries. +The `wiki_stale` flag drives whether the wiki-ingest prompt processes the +file on its next run. + +### `wiki_pages` +LLM-synthesised wiki content (not raw copies). Columns: `namespace`, +`title`, `content` (markdown), `sources` JSONB pointing back at +`file_references`, `confidence`, `superseded_by` (chains old versions), +`stale`, `last_synced_hash`, and a 1536-dim `embedding`. Tagged for +ivfflat index but the index is held off until the table has 1000+ rows +for usable recall. + +> Note: per Decision 8 in [[personal-os-state-2026-04-27]], the live +> Personal OS no longer uses these DB tables for wiki ingest — the vault +> is the source of truth and SHA256 lives in frontmatter. The tables +> remain in the schema for compatibility / future re-use. + +### `memory_store` +Semantic memory accumulated from Discord, Claude sessions, and manual +entries. `type` ∈ {`fact`, `preference`, `decision`, `person`}; `source` ∈ +{`discord`, `claude`, `manual`}. Includes `entities` JSONB, `confidence`, +`embedding` (1536), and an optional `expires_at` (NULL = permanent). + +## Key Points + +- All namespaces gated to `work | personal | family` via CHECK constraints. +- pgvector embeddings live on three tables: `task_embeddings`, `wiki_pages`, + `memory_store`. Only `task_embeddings` ships with an HNSW index; + the wiki/memory ivfflat indexes are commented out until data volume + justifies them. +- The `active_tasks` view is the canonical "what should I look at" surface + for agents. + +## Related +[[personal-os-state-2026-04-27]] [[personal-os-catchup-plan-2026-04-27]] +[[wiki-ingest-process]] [[ddg-asana-workflow]] diff --git a/wiki/personal-os-state-2026-04-27.md b/wiki/personal-os-state-2026-04-27.md new file mode 100644 index 00000000..7710b7ff --- /dev/null +++ b/wiki/personal-os-state-2026-04-27.md @@ -0,0 +1,134 @@ +--- +source: raw/personal-os-state-2026-04-27.md +content_hash: 1640cb63145b49263b9476e2c39fe443ac561fdf6c27faccc50cfc8568552614 +namespace: work +last_synced: 2026-05-30 +confidence: 0.9 +tags: [personal-os, state, snapshot, executor, hermes] +--- + +# Personal OS — State Snapshot (2026-04-27) + +Current-state snapshot of the Personal OS as of 2026-04-27 (updated 04-28). +Supersedes earlier `state-2026-04-27.md` and `personal-os-plan-v3.md`. The +catch-up plan from the same date has been executed — see +[[personal-os-catchup-plan-2026-04-27]] for what was scoped, and the +status table below for what landed. + +## What's running + +### Infrastructure +- PostgreSQL (Mac) with the [[personal-os-schema]] tables. +- `sync.js` — 4-source Asana fetcher with incremental deltas and + `sweepRecentlyCompleted()`. +- `generate-status.js` — DB → `status.md` + `asana_context.md`, + `possibly_deleted` filter fixed. +- ActivityWatch watchers for DDG browser + Xcode. +- Hermes v0.11.x — launchd daemon `ai.hermes.gateway`, Claude Code OAuth, + model `claude-sonnet-4-6`. +- Discord bot **Орёл#0898** — no @-mention required. Channels: + `#daily-brief #inbox #focus #executor #projects #journal #retrospector`. + +### Hermes cron jobs +| Job | Schedule | Channel | +|-----|----------|---------| +| data-pipeline | */30 7-21 workdays | silent | +| morning-brief | 08:30 workdays | #daily-brief | +| eod-summary | 18:00 workdays | #daily-brief | +| inbox-check | */30 9-19 workdays | #inbox | +| weekly-plan | Mon 08:00 | #daily-brief | +| weekly-review | Fri 17:00 | #daily-brief | +| retrospector | Fri 17:30 | #retrospector | +| commit-vault | 23:00 daily | silent | + +### launchd agents (fire after wake) +- `personal.os.heartbeat` — :05 hourly, watchdog for missed jobs. +- `personal.os.wiki-ingest` — 22:00 daily, runs the + [[wiki-ingest-process]] via local `claude -p`. + +### Vault layout (~/obsidian/) +NAS git remote: `ssh://truenas_admin@mallexxx.duckdns.org/mnt/RED_2TB/storage/git/obsidian-vault.git`. +Directories: `wiki/` (LLM-generated + hand-written confidence-1.0 pages), +`raw/` (symlinks: schema.sql, wiki-ingest-prompt.md, etc.), `work/`, +`personal/`, `family/`. Hand-written pages include +[[ddg-asana-workflow]], `personal-os-architecture.md`, `agent-rules`, +`sync-pipeline`, `vault-filling-guide`. + +## Catch-up plan execution status + +All 10 items from the 2026-04-27 catch-up plan landed except for the +MS365 MCP wiring, which is partial (cloud connector confirmed in Claude +Code; no local npm package, so Hermes `claude -p` jobs can only use it +once it reaches the Claude CLI environment). See +[[personal-os-catchup-plan-2026-04-27]] for the detailed scope. + +| # | Item | Status | +|---|------|--------| +| 1 | Schema migration (corrections_log, executor_runs, quadrant) | ✅ | +| 2 | MS365 MCP in Hermes | 🟡 partial | +| 3 | inbox-triage STEP 0 (focus + overdue exclusion) | ✅ | +| 4 | generate-status.js: advisor + visibility + executor queue | ✅ | +| 5 | quadrant classification + activity_daily upsert | ✅ | +| 6 | retrospector SQL blocks | ✅ | +| 7 | SOUL.md corrections_log writeback | ✅ | +| 8 | Vault consolidation (43 .cursor/rules/*.mdc) | ✅ | +| 9 | Executor PR template (inline) | ✅ | +| 10 | Executor v2 prompt rewrite | ✅ | + +## Executor v2 design + +On-demand worker that evaluates the macOS task backlog, prioritises, +spawns a sub-agent under a Discord thread in `#executor`, and drives a +draft PR + autonomous CI loop. Trigger modes: user-initiated in +`#executor`, scheduled Friday weekly-review suggestion, or status-driven +via the `## ⚡ Executor Queue` section of `status.md`. + +Task qualification: assigned to me, not completed, no stories in 14+ +days, due within 30 days. macOS-browser backlog evaluated separately. +Handled types: bug with clear repro, bug requiring UI (VM via ddg-vm +MCP), stalled tech design, stalled ship review, planned-but-unmoved. + +Worker flow runs inside a sibling worktree at +`~/DuckDuckGo/apple-browsers.git/.claude/worktrees/executor-{gid}-{slug}/`, +posts streaming progress to its Discord thread, opens a `--draft` PR +assigned to `@me` with no human reviewers, self-reviews, and polls `gh +pr checks` every 10 min. Max 3 auto-fix attempts on red CI before +escalating with state `awaiting_user`. + +Phases: 4a (evaluation engine) → 4b (worker spawning + threads) → 4c +(PR + CI loop) → 4d (feedback loop) → 4e (scheduled evaluation). + +## Decisions vs original plan v3 + +Eight documented divergences (see source for full text): + +1. wiki-ingest moved from Hermes cron → launchd subprocess (filesystem I/O). +2. Custom `personal-wiki` MCP replaced with `obsidian-mcp` package. +3. Per-job launchd plists collapsed into one heartbeat watchdog. +4. Retrospector extracted to its own prompt + cron + channel. +5. NAS git path moved from `/home/` (boot pool, wiped on update) to + `/mnt/RED_2TB/storage/git/`. +6. Phase 2-family deferred until after Phase 4. +7. Executor v1 prompts kept manual-only with STOP gates; v2 adds the + autonomous PR/CI loop. +8. wiki-ingest rewritten from DB-first to file-based — SHA256 in + frontmatter, no DB tables involved. + +## Known issues / tech debt +- Morning brief never smoke-tested before this date (first real run was + 2026-04-28 08:30). +- `work/projects/` vault directory still empty. +- wiki-ingest runs `claude -p` without `--allowedTools` — may attempt + unavailable tools. +- NAS SSH key authorized_keys still on boot-pool `/home/` — needs to + move under `/mnt/RED_2TB/` before next TrueNAS update (High severity). +- `~/.hermes/memory/` not in vault or NAS backup. + +## Deferred +Phase 2-family (VPS Ubuntu 24.04, family Hermes, iCloud sync, +`~/obsidian → iCloud` symlink, hourly NAS rsync) — postponed indefinitely +until Executor v2 is fully live. + +## Related +[[personal-os-catchup-plan-2026-04-27]] [[personal-os-schema]] +[[wiki-ingest-process]] [[ddg-asana-workflow]] diff --git a/wiki/wiki-ingest-process.md b/wiki/wiki-ingest-process.md new file mode 100644 index 00000000..d24ed99b --- /dev/null +++ b/wiki/wiki-ingest-process.md @@ -0,0 +1,114 @@ +--- +source: raw/wiki-ingest-prompt.md +content_hash: 29f6a0914d428cf62eeb2a832d6b3048edf3e1d16aa7714c09f686377394d128 +namespace: work +last_synced: 2026-05-30 +confidence: 0.9 +tags: [personal-os, wiki, ingest, agent-prompt] +--- + +# Wiki Ingest Process + +The wiki-ingest agent maintains `~/obsidian/wiki/` by synthesising +markdown pages from source files exposed as symlinks under +`~/obsidian/raw/`. It is a local `claude -p` subprocess invoked nightly +by the `personal.os.wiki-ingest` launchd agent (22:00 daily) — see +[[personal-os-state-2026-04-27]] for the runtime layout. + +## Inputs + +Each entry under `~/obsidian/raw/` is a symlink to an external project +file. The wiki keeps a synthesised page per source, identified by a +frontmatter `source: raw/` line and tracked for staleness via a +`content_hash` (SHA256) in the same frontmatter. + +## Workflow + +### Step 0 — Discover changes +For every file in `~/obsidian/raw/`: +1. Read the file content. +2. Compute SHA256 (`shasum -a 256 | awk '{print $1}'`). +3. Search `~/obsidian/wiki/` for any `.md` whose frontmatter has + `source: raw/`. +4. If a page exists and `content_hash` matches → skip. +5. Otherwise add to the work queue. + +If the queue is empty, output `Wiki is up to date. Nothing to ingest.` +and stop. + +### Step 1 — Process each changed file +- **Find or create**: update the existing page if found, otherwise + create `~/obsidian/wiki/.md`. Filenames follow the topic, + not the source filename — e.g. `personal-os-schema.md`, not `schema.md`. +- **Synthesise**: extract key facts, decisions, and structure. Never + copy the source verbatim. Keep each page under 800 words; split into + linked pages if the topic is too large. +- **Cross-reference**: use `[[double brackets]]` to link related wiki + pages. + +### Step 2 — Report +After processing, print a summary with counts of created, updated, and +skipped pages, followed by the list of pages touched and their sources. + +## Frontmatter contract + +```yaml +source: raw/ +content_hash: +namespace: work +last_synced: +confidence: 0.8 +tags: [tag1, tag2] +``` + +`confidence` is a coarse trust signal: + +| Value | Meaning | +|-------|--------------------------------------------------------| +| 1.0 | Reserved for human-written notes (`work/`, `personal/`, `family/`). | +| 0.9 | Highly structured, authoritative source. | +| 0.8 | Default for clear single-source synthesis. | +| 0.6 | Inferred or partial content. | + +## Hard rules + +- Never copy source files verbatim — always synthesise. +- Never edit a wiki page with `confidence: 1.0` — those are + human-authored. +- Never process files belonging to `namespace: family` unless explicitly + instructed. +- Always update `content_hash` and `last_synced` after rewriting a page. +- If a symlink in `raw/` is broken (target missing), skip it — do not + create a stub wiki page. + +## Why this lives in the prompt, not a DB + +Per Decision 8 in [[personal-os-state-2026-04-27]], the original v3 design +used `file_references` + `wiki_pages` Postgres tables. That was replaced +with file-only state: SHA256 in frontmatter is enough for incremental +sync, the vault is self-contained and portable, and Obsidian (via the +`obsidian-mcp` package) becomes the single source of truth. The DB +tables still exist in [[personal-os-schema]] but are unused in the live +flow. + +## Operational notes + +- Runs once a day under launchd, so the prompt is allowed to be slow. +- Invoked without `--allowedTools` in v1 (tracked as low-severity tech + debt in [[personal-os-state-2026-04-27]] §6). The agent may try to + call tools it doesn't have — monitor stderr. +- The `commit-vault.sh` script (23:00 daily) picks up any new wiki + pages and pushes them to the NAS git remote. + +## Key Points + +- Source-of-truth is the filesystem; SHA256 in YAML frontmatter is the + staleness signal. +- Filenames follow topic, not source filename. +- Synthesis (≤ 800 words), never verbatim. +- Confidence 1.0 pages are human-owned and untouchable. +- Family namespace requires explicit user opt-in per run. + +## Related +[[personal-os-state-2026-04-27]] [[personal-os-schema]] +[[personal-os-catchup-plan-2026-04-27]] [[ddg-asana-workflow]]