[2026-05-29] personal os catchup plan, personal os state, wiki ingest process
This commit is contained in:
Alexey Martemyanov
@@ -0,0 +1,92 @@
|
||||
---
|
||||
source: raw/personal-os-catchup-plan-2026-04-27.md
|
||||
content_hash: 5081ceb719104e1b7c6b5edd642c1daf38b3fd7cf1a96442a00516e7cdb15379
|
||||
namespace: work
|
||||
last_synced: 2026-05-29
|
||||
confidence: 0.8
|
||||
tags: [personal-os, catchup, plan, executor, retrospector, schema]
|
||||
---
|
||||
|
||||
# Personal OS — Catch-Up Plan (2026-04-27)
|
||||
|
||||
Handoff for the executor agent: what to build next, in what order, why. Companion to [[personal-os-state-2026-04-27]].
|
||||
|
||||
## Context
|
||||
|
||||
Alex is a macOS Browser developer / DRI at DuckDuckGo. ADHD profile, fully async, primary channels Asana + Slack. The Personal OS holds work context outside Alex's head so comments aren't lost, daily focus survives, and career goals don't get crushed by the current project.
|
||||
|
||||
Four-agent design from the original spec: **Стратег** (strategist — status writer, ✅), **Тактик** (reactive, ADHD-aware, 🟡 partial), **Ретроспектор** (Friday-only patterns, ✅), **Исполнитель** (executor, 🟡 v1 only).
|
||||
|
||||
What's being caught up: career tracking, inbox hygiene, focus-aware ping gating, structured corrections (table not file), quadrants, Executor v2 (draft-PR + CI loop), MS365 wiring, vault consolidation, executor PR template variant.
|
||||
|
||||
**Architectural principle (hard):** no new YAML files, shell scripts, or launchd plists. Extend the existing four components — Postgres, `generate-status.js`, prompt files, Eagle's SOUL.md. See [[personal-os-state-2026-04-27]].
|
||||
|
||||
## Deliverables
|
||||
|
||||
### 1.1 Schema migration
|
||||
Three changes: new `corrections_log` (date, week, source, original_plan, correction, deferred_gids, reason_tag); new `executor_runs` (state machine: `analyzing | awaiting_analysis_approval | fixing | pr_open | ci_running | ci_failed | awaiting_user | complete | abandoned`); `activity_daily.quadrant TEXT` (project | aor | career | strategy | other). Tables not markdown because Retrospector needs aggregates.
|
||||
|
||||
### 1.2 `generate-status.js` blocks
|
||||
- **Career — Advisor pipeline**: filter on `notes ILIKE '%Project Advisor: Set the custom field%'`, recent 14d. Omit section if empty (no noise).
|
||||
- **Visibility — Active Assessments without Alex's input**: 14d-active tasks named Assessment/O-N/O-L where Alex hasn't commented in 7d.
|
||||
- **Quadrant classification at AW write time**: static `PROJECT_TO_QUADRANT` map plus dynamic "current top priority project" read from config/status (not hardcoded — top priority shifts).
|
||||
|
||||
### 1.3 `inbox-triage.md` STEP 0
|
||||
Two-part gating in the prompt itself (not a wrapper — agent retains override): focus check (`focus_score > 60` + Xcode/Cursor + 20 min stable → exit unless 🔴 sections older than 3h; active calendar meeting → exit silently); scope filter (NEW SIGNAL ONLY — no overdue, no system events, no Alex-already-responded). Empty → write "Inbox quiet — nothing new."
|
||||
|
||||
### 1.4 `retrospector.md` SQL blocks
|
||||
- **Correction patterns** — `reason_tag` grouped by 4w, threshold ≥3.
|
||||
- **Quadrant drift** — 4w × 4-quadrant pivot from `activity_daily`. Flag career quadrant <5% for 3+ weeks.
|
||||
- **Visibility this week** — count Alex's strategic comments via `stories` joined on Assessment/Objective/project tasks.
|
||||
|
||||
### 1.5 Eagle SOUL.md — corrections writeback
|
||||
On any user pushback in Discord, Eagle infers `reason_tag`, runs `INSERT INTO corrections_log` silently, then replies. Replaces markdown log. Natural capture moment is in-conversation, not via a separate hook.
|
||||
|
||||
### 1.6 Executor v2 — draft-PR autonomous flow
|
||||
**Single STOP gate: Analysis approval.** After approval, autonomously: fix → build → test → branch `executor/<gid>-<slug>` → `gh pr create --draft --assignee @me` (no `--reviewer`!) → self-review comment → CI polling loop (10 min, max 3 auto-retry attempts → `awaiting_user`).
|
||||
|
||||
Hard prohibitions: never push to main/develop/release; always `--draft`; never `--reviewer`; never `gh pr merge`; no force-push outside own branch; no edits to `.github/` or dependency manifests without thread confirmation.
|
||||
|
||||
Existing Discord channels suffice — no new ones.
|
||||
|
||||
Rationale: draft PRs are durable surface (vs. ephemeral Discord threads), don't trigger reviewer pings, easy one-click flip to ready. ADHD-friendly.
|
||||
|
||||
### 1.7 MS365 MCP into Hermes
|
||||
Calendar gating needs `outlook_calendar_search(afterDateTime: "now", beforeDateTime: "now + 1h")`. Wire once at Hermes config layer so inbox-triage, weekly-plan, daily-brief all benefit. Microsoft 365 (not Google) — that's what's in the workspace.
|
||||
|
||||
### 1.8 Vault consolidation
|
||||
Move process docs from Personal OS → `~/obsidian/work/wiki/`. Move/copy UI testing skill and 43 `apple-browsers/.cursor/*.mdc` files (copy `.mdc` — Cursor needs originals in place). Verification: invoke each moved skill/prompt end-to-end after move.
|
||||
|
||||
Decision rule: "process/convention/workflow → vault; read-at-runtime → original location." Both → canonical in vault, stub or symlink at runtime path.
|
||||
|
||||
### 1.9 Executor PR template (Option B)
|
||||
Executor inlines the template — task link (auto from `executor_runs.task_gid`), reviewer = Alex, description auto-generated from analysis + diff. Draft state is the "needs polish" signal. Sections: Task / Root Cause / Fix Summary / Testing (build, unit, UI) / Self-review / Notes. Footer: "Generated by Executor — DRAFT until Alex flips to ready-for-review."
|
||||
|
||||
## Execution order
|
||||
|
||||
Dependency-ordered. Items 1–9 can run in parallel; item 10 (Executor v2 prompt rewrite) needs #1 (executor_runs), #8 (vault has .mdc + UI skill), #9 (PR template).
|
||||
|
||||
The implementing agent does one-time discovery (reads `.cursor/*.mdc` + GHA workflows + UI testing skill in vault) and **hardcodes** Xcode scheme + test command + UI runner specifics into config — Hermes does not re-discover every run.
|
||||
|
||||
## Open questions
|
||||
All resolved. Repo at `~/DuckDuckGo/apple-browsers/main`. Worktrees as siblings. Scheme `DuckDuckGo macOS`. No CODEOWNERS. Team OK with `[executor]`-tagged drafts on Alex only.
|
||||
|
||||
Postponed: NAS migration (done), personal, Phase 2-family.
|
||||
|
||||
## "Done" looks like
|
||||
1. Friday retrospector says specific things ("deferred 4× for ship-review-crunch", "career <5% three weeks").
|
||||
2. Inbox = only new human signal; empty during meetings.
|
||||
3. `status.md` has `## ⚡ Career` (empty when nothing).
|
||||
4. Pushback → one `corrections_log` row, no ceremony.
|
||||
5. "fix [task]" → thread → analysis → approval → draft PR within 30 min for trivial bugs; CI runs solo.
|
||||
6. Vault has UI skill + `.mdc` + process docs, all searchable via obsidian-mcp.
|
||||
|
||||
## Key Points
|
||||
- Extend existing components; add no new shell scripts, plists, or YAML.
|
||||
- Postgres tables (not markdown) for anything Retrospector needs to aggregate.
|
||||
- STEP 0 gating lives in the prompt, not a wrapper — agent keeps judgment.
|
||||
- Executor v2 has exactly one human gate (Analysis approval); rest is autonomous to draft PR + CI green.
|
||||
- Draft PRs > Discord-only handoff — durable surface, no reviewer pings.
|
||||
|
||||
## Related
|
||||
[[personal-os-state-2026-04-27]] [[personal-os-schema]] [[wiki-ingest-process]]
|
||||
@@ -0,0 +1,98 @@
|
||||
---
|
||||
source: raw/personal-os-state-2026-04-27.md
|
||||
content_hash: 1640cb63145b49263b9476e2c39fe443ac561fdf6c27faccc50cfc8568552614
|
||||
namespace: work
|
||||
last_synced: 2026-05-29
|
||||
confidence: 0.8
|
||||
tags: [personal-os, state, architecture, hermes, executor]
|
||||
---
|
||||
|
||||
# Personal OS — State Snapshot (2026-04-27)
|
||||
|
||||
Supersedes `state-2026-04-27.md` and `personal-os-plan-v3.md`. Reflects the catch-up plan executed on 2026-04-27. See [[personal-os-catchup-plan-2026-04-27]] for the deliverables that produced this state.
|
||||
|
||||
## What's built
|
||||
|
||||
### Infrastructure
|
||||
- **PostgreSQL** on Mac — see [[personal-os-schema]] for tables.
|
||||
- **`sync.js`** — 4-source Asana fetch (`my_tasks`, `following`, `delegated`, `project:*`), incremental deltas, `sweepRecentlyCompleted()`.
|
||||
- **`generate-status.js`** — DB → `status.md` + `asana_context.md`. `possibly_deleted` filter fixed; now also writes advisor + visibility + executor queue blocks, and the quadrant column on `activity_daily`.
|
||||
- **ActivityWatch** — DDG browser watcher + Xcode watcher running.
|
||||
- **Hermes v0.11.x** — launchd daemon `ai.hermes.gateway`, Claude Code OAuth, model `claude-sonnet-4-6`.
|
||||
- **Discord bot Орёл#0898** — no @-mention needed. Channels: `#daily-brief #inbox #focus #executor #projects #journal #retrospector`.
|
||||
|
||||
### Hermes cron schedule
|
||||
| Job | Schedule | Channel |
|
||||
|-----|----------|---------|
|
||||
| data-pipeline | */30 7–21 workdays | silent |
|
||||
| morning-brief | 08:30 workdays | #daily-brief |
|
||||
| eod-summary | 18:00 workdays | #daily-brief |
|
||||
| inbox-check | */30 9–19 workdays | #inbox |
|
||||
| weekly-plan | Mon 08:00 | #daily-brief |
|
||||
| weekly-review | Fri 17:00 | #daily-brief |
|
||||
| retrospector | Fri 17:30 | #retrospector |
|
||||
| commit-vault | 23:00 daily | silent |
|
||||
|
||||
### launchd agents (fire on wake)
|
||||
- `personal.os.heartbeat` — `:05` hourly. Reads `jobs.json`, fires any missed once-per-day Hermes jobs.
|
||||
- `personal.os.wiki-ingest` — 22:00 daily. `claude -p` subprocess writes `~/obsidian/wiki/`. See [[wiki-ingest-process]].
|
||||
|
||||
### Scripts
|
||||
- `run-pipeline.sh`, `run-wiki-ingest.sh`, `commit-vault.sh`, `hermes-heartbeat.sh`.
|
||||
|
||||
### Vault layout (`~/obsidian/`)
|
||||
- `wiki/` — mix of LLM-generated and `confidence: 1.0` human notes.
|
||||
- `raw/` — symlinks to source files for ingest.
|
||||
- `work/`, `personal/` — Eagle write-allowed; `family/` exists.
|
||||
- Git remote on ZFS data pool: `ssh://truenas_admin@mallexxx.duckdns.org/mnt/RED_2TB/storage/git/obsidian-vault.git`.
|
||||
|
||||
### Agent prompts (`~/Developer/personal-os/agent/prompts/`)
|
||||
`daily-brief.md`, `weekly-plan.md`, `weekly-review.md`, `inbox-triage.md`, `generate-status.md`, `wiki-ingest.md`, `executor-bug-fix.md`, `executor-td-review.md`.
|
||||
|
||||
### Retrospector (Phase 3 — live)
|
||||
Friday-only. Surfaces: correction patterns (3+ times), velocity vs scope, vault git activity, exactly one decision question.
|
||||
|
||||
## What's not done
|
||||
|
||||
| Item | Status |
|
||||
|------|--------|
|
||||
| Phase 2-family (VPS, family Hermes, iCloud sync) | Deferred until after Phase 4 |
|
||||
| `work/projects/` notes | Empty — Eagle fills on first use |
|
||||
| Executor v2 end-to-end live run | Prompt written, not tested on real task |
|
||||
| MS365 MCP in Hermes | Cloud connector works in Claude Code; no local npm package |
|
||||
|
||||
Catch-up plan 2026-04-27 items 1, 3–10 ✅; item 2 (MS365) 🟡.
|
||||
|
||||
## Executor v2 — architecture summary
|
||||
|
||||
User-initiated in `#executor` ("fix [task]"), scheduled via Friday weekly-review, or status-driven via `## ⚡ Executor Queue`. Worker spawns in a Discord thread, streams progress, opens a **draft** PR assigned to Alex (never tags humans), runs a CI polling loop (every 10 min, max 3 auto-fix attempts). Full state machine in `executor_runs` table. See [[personal-os-catchup-plan-2026-04-27]] for design rationale.
|
||||
|
||||
Worktrees live at `~/DuckDuckGo/apple-browsers.git/.claude/worktrees/executor-{gid}-{slug}/`. Scheme: `DuckDuckGo macOS` in `.xcworkspace`, `xcbeautify` required. UI tests via `ddg-vm` MCP (VirtualBuddy).
|
||||
|
||||
## Decisions divergent from plan v3
|
||||
|
||||
1. Wiki-ingest moved from Hermes cron → launchd subprocess (needs local FS access).
|
||||
2. `personal-wiki` MCP replaced with `obsidian-mcp` — DB tables dropped.
|
||||
3. Single `personal.os.heartbeat` instead of per-job launchd plists.
|
||||
4. Retrospector extracted from weekly-review → own prompt, cron, channel.
|
||||
5. NAS git path moved to `/mnt/RED_2TB/storage/git/` (ZFS data pool, not boot pool).
|
||||
6. Phase 2-family deferred until after Phase 4.
|
||||
7. Executor v1 prompts manual-only with STOP gates (v2 adds autonomy).
|
||||
8. wiki-ingest rewritten file-based (SHA256 in frontmatter) — DB tables superseded.
|
||||
|
||||
## Known issues / tech debt
|
||||
|
||||
- Morning brief never smoke-tested (medium).
|
||||
- Executor v1 prompts manual-only — replace with v2 (medium).
|
||||
- `wiki-ingest` uses `claude -p` without `--allowedTools` (low).
|
||||
- NAS SSH key still in boot pool `/home/` — must move to `/mnt/RED_2TB/` before next TrueNAS update (**high**).
|
||||
- Hermes memory `~/.hermes/memory/` not backed up.
|
||||
|
||||
## Key Points
|
||||
- Postgres + Hermes + launchd + Obsidian vault are the four pillars; everything extends one of those.
|
||||
- All 10 catch-up items shipped except MS365 MCP in Hermes (cloud-only).
|
||||
- Executor v2 = single STOP gate at Analysis, then autonomous draft PR + CI loop.
|
||||
- Boot-pool-vs-data-pool distinction matters: persistent data must live under `/mnt/RED_2TB/`.
|
||||
|
||||
## Related
|
||||
[[personal-os-catchup-plan-2026-04-27]] [[personal-os-schema]] [[wiki-ingest-process]]
|
||||
@@ -0,0 +1,97 @@
|
||||
---
|
||||
source: raw/wiki-ingest-prompt.md
|
||||
content_hash: 29f6a0914d428cf62eeb2a832d6b3048edf3e1d16aa7714c09f686377394d128
|
||||
namespace: work
|
||||
last_synced: 2026-05-29
|
||||
confidence: 0.9
|
||||
tags: [wiki, ingest, process, obsidian, automation]
|
||||
---
|
||||
|
||||
# Wiki Ingest — Process
|
||||
|
||||
The wiki ingest job keeps `~/obsidian/wiki/` synchronised with source files referenced under `~/obsidian/raw/`. Each entry in `raw/` is a symlink to an external project file. The job runs as a launchd subprocess invoking `claude -p` (see [[personal-os-state-2026-04-27]] — needs local filesystem access, can't run in Hermes).
|
||||
|
||||
## Trigger
|
||||
|
||||
`personal.os.wiki-ingest` launchd agent fires daily at 22:00. Runs `~/scripts/run-wiki-ingest.sh`, which has a one-per-day guard.
|
||||
|
||||
## Algorithm
|
||||
|
||||
### Discover changes
|
||||
For every file under `~/obsidian/raw/`:
|
||||
1. Read file content (follows symlink).
|
||||
2. Compute SHA256 via `shasum -a 256 <path>`.
|
||||
3. Look in `~/obsidian/wiki/` for a `.md` file whose frontmatter has `source: raw/<filename>`.
|
||||
4. If a wiki page exists AND its `content_hash` matches the freshly computed one — skip (up-to-date).
|
||||
5. Otherwise — queue for synthesis.
|
||||
6. Broken symlinks (target missing) — skipped entirely, no wiki page is created.
|
||||
|
||||
If queue empty → emit `Wiki is up to date. Nothing to ingest.` and stop.
|
||||
|
||||
### Synthesise each queued file
|
||||
- Update existing wiki page if one references this source; otherwise create `~/obsidian/wiki/<topic-name>.md` (name by topic, not by source filename — e.g. `personal-os-schema.md`, not `schema.md`).
|
||||
- Write a synthesis, **not** a verbatim copy. Extract facts, decisions, structure.
|
||||
- Keep under 800 words per page; split large topics into linked pages using `[[double brackets]]`.
|
||||
|
||||
### Frontmatter contract
|
||||
```yaml
|
||||
---
|
||||
source: raw/<filename>
|
||||
content_hash: <sha256>
|
||||
namespace: work
|
||||
last_synced: <YYYY-MM-DD>
|
||||
confidence: 0.8
|
||||
tags: [tag1, tag2]
|
||||
---
|
||||
```
|
||||
|
||||
### Confidence levels
|
||||
- `0.9` — highly structured, authoritative source (e.g. SQL schema)
|
||||
- `0.8` — clear single source (default)
|
||||
- `0.6` — inferred or partial content
|
||||
- `1.0` — reserved for human-written notes; the ingester **must not** touch these
|
||||
|
||||
### Output report
|
||||
```
|
||||
Wiki Ingest — <date>
|
||||
|
||||
Processed: N files
|
||||
Created: N new pages
|
||||
Updated: N pages
|
||||
Skipped: N (unchanged)
|
||||
|
||||
Pages updated:
|
||||
- wiki/<filename> (source: raw/<source>)
|
||||
```
|
||||
|
||||
## Rules (hard)
|
||||
|
||||
- Never copy source content verbatim — always synthesise.
|
||||
- Never edit any `wiki/*.md` with `confidence: 1.0` — those are human-written.
|
||||
- Never process files whose namespace is `family` unless explicitly instructed.
|
||||
- Always update both `content_hash` and `last_synced` in frontmatter after writing.
|
||||
- Always skip broken symlinks; never create a wiki page for a missing target.
|
||||
|
||||
## Why file-based, not DB-based
|
||||
|
||||
Earlier designs (see [[personal-os-state-2026-04-27]] Decision 8) had wiki-ingest read `file_references` + `wiki_pages` Postgres tables. That was dropped:
|
||||
- Source of truth shifted to Obsidian vault filesystem.
|
||||
- SHA256 in frontmatter gives incremental sync without DB dependency.
|
||||
- Vault is self-contained — no DB schema migrations required.
|
||||
- `obsidian-mcp` gives agents native vault access (Decision 2).
|
||||
|
||||
## Operational notes
|
||||
|
||||
- `personal.os.wiki-ingest` uses `claude -p` without `--allowedTools` — it may try unavailable tools; monitor logs (low-severity tech debt per state snapshot).
|
||||
- Commit vault: `commit-vault.sh` runs nightly at 23:00, git-adds with LLM summary, pushes to NAS remote.
|
||||
- NAS remote lives on ZFS data pool, not the boot pool, to survive TrueNAS updates: `ssh://truenas_admin@mallexxx.duckdns.org/mnt/RED_2TB/storage/git/obsidian-vault.git`.
|
||||
|
||||
## Key Points
|
||||
- File-based incremental sync via SHA256 in frontmatter — no DB tables involved.
|
||||
- One wiki page per source, named by topic.
|
||||
- `confidence: 1.0` pages are off-limits to the ingester.
|
||||
- Output report tells the user exactly what was processed/created/updated/skipped.
|
||||
- Synthesis (~800 words), not transcription — split large sources.
|
||||
|
||||
## Related
|
||||
[[personal-os-schema]] [[personal-os-state-2026-04-27]] [[personal-os-catchup-plan-2026-04-27]]
|
||||
Reference in New Issue
Block a user