[2026-06-01] personal-os schema, state, catchup plan, wiki ingest process

2026-06-01 23:00:53 +06:00
parent 8dec68071e
commit dc6826e5a6
4 changed files with 445 additions and 0 deletions
@@ -0,0 +1,89 @@
+---
+source: raw/wiki-ingest-prompt.md
+content_hash: 29f6a0914d428cf62eeb2a832d6b3048edf3e1d16aa7714c09f686377394d128
+namespace: work
+last_synced: 2026-06-01
+confidence: 0.9
+tags: [personal-os, wiki, agent-prompt, obsidian]
+---
+
+# Wiki Ingest — Process
+
+Daily process that keeps `~/obsidian/wiki/` synchronised with external
+source files surfaced via symlinks in `~/obsidian/raw/`. Runs as a
+launchd subprocess (`personal.os.wiki-ingest`) invoking `claude -p` —
+not a Hermes cron job, because it needs local filesystem access to read
+sources and write wiki pages.
+
+## Flow
+
+### STEP 0 — Discover changed files
+For each entry in `~/obsidian/raw/`:
+1. Read the symlink target.
+2. Compute `shasum -a 256`.
+3. Scan `~/obsidian/wiki/` for any `.md` whose frontmatter has
+   `source: raw/<filename>`.
+4. If page exists AND `content_hash` matches → **skip** (unchanged).
+5. Otherwise → queue for processing.
+6. If a symlink is broken, skip silently — never create a page for
+   missing content.
+
+Empty queue → print "Wiki is up to date. Nothing to ingest." and exit.
+
+### STEP 1 — Synthesise (per file)
+- Update an existing wiki page if one references the source, otherwise
+  create `~/obsidian/wiki/<topic-name>.md` named by **topic, not source
+  filename** (e.g. `personal-os-schema.md`, not `schema.md`).
+- Write a synthesis: extract key facts, decisions, structure — **never
+  copy verbatim**.
+- Cap at ~800 words; split big topics into linked pages.
+- Use `[[wiki-links]]` for cross-references.
+
+### STEP 2 — Report
+Print processed / created / updated / skipped counts and a list of
+touched pages.
+
+## Frontmatter Schema
+
+```yaml
+source: raw/<filename>
+content_hash: <sha256>
+namespace: work
+last_synced: <YYYY-MM-DD>
+confidence: 0.8
+tags: [tag1, tag2]
+```
+
+`content_hash` and `last_synced` are always updated after a write — they
+are how the next ingest decides what changed.
+
+## Confidence Scale
+
+| Value | Meaning |
+|-------|---------|
+| 0.6 | inferred or partial content |
+| 0.8 | clear single source (default) |
+| 0.9 | highly structured, authoritative source (e.g. SQL schema) |
+| 1.0 | reserved for human-written notes in `work/`, `personal/`, `family/` |
+
+## Hard Rules
+
+- **Never** verbatim-copy source files; always synthesise.
+- **Never** edit files marked `confidence: 1.0` — those are
+  human-authored.
+- **Never** process `namespace: family` files unless explicitly told.
+- Always update `content_hash` and `last_synced` after writing.
+- Broken symlink → skip, do not create.
+
+## Why File-Based, Not DB-Backed
+
+Earlier design (v3) had `wiki_pages` / `file_references` tables drive
+ingest via SQL. That was replaced by file-based ingest because
+[[obsidian-mcp]] now exposes the vault directly — the filesystem is the
+source of truth and the DB tables would have been a duplicate index of
+the same content. The Postgres tables for [[personal-os-schema|wiki_pages]]
+still exist but the agent doesn't read or write them anymore (see
+[[personal-os-state-2026-04-27|Decision 8]]).
+
+## Related
+[[personal-os-schema]] [[personal-os-state-2026-04-27]] [[personal-os-catchup-plan-2026-04-27]]