Wiki Ingest — Process & Prompt

The wiki-ingest agent keeps ~/obsidian/wiki/ synchronised with external source files exposed as symlinks in ~/obsidian/raw/. It runs daily at 22:00 via the personal.os.wiki-ingest launchd agent, which invokes claude -p as a local subprocess (Hermes can't reach the local filesystem). See personal-os-state-2026-04-27 Decision 1.

How sync works

For every file in ~/obsidian/raw/, compute SHA256 via shasum -a 256 <path> | awk '{print $1}'.
Search ~/obsidian/wiki/ for any .md with frontmatter source: raw/<filename>.
If a page exists AND content_hash in frontmatter matches → skip.
Otherwise → queue for synthesis.
Broken symlinks → skip silently, do not create a page.

If the queue is empty after the scan, output "Wiki is up to date. Nothing to ingest." and stop.

Synthesis rules

Never copy source files verbatim. Extract key facts, decisions, and structure. Keep each page under 800 words.
Name the wiki file by topic, not by source filename (e.g. personal-os-schema.md, not schema.md).
If a topic is large, split into multiple linked pages.
Cross-reference related pages with [[double brackets]].
Existing page found → update in place (preserving the file path).

Frontmatter format

---
source: raw/<filename>
content_hash: <sha256>
namespace: work
last_synced: <YYYY-MM-DD>
confidence: 0.8
tags: [tag1, tag2]
---

last_synced and content_hash MUST be updated after every successful write.

Confidence scale

Value	Meaning
1.0	Human-written notes in `work/`, `personal/`, `family/`. Never edit these.
0.9	Highly structured, authoritative source (e.g. schema files)
0.8	Clear single source (default)
0.6	Inferred or partial content

Hard rules

Do not edit any wiki page with confidence: 1.0 — those are human-authored.
Do not process namespace: family files unless explicitly instructed.
Broken raw/ symlinks are skipped, not removed.
Always update content_hash and last_synced after writing.

Output format

At the end of a run, print:

Wiki Ingest — <date>

Processed: N files
Created: N new pages
Updated: N pages
Skipped: N (unchanged)

Pages updated:
- wiki/<filename> (source: raw/<source>)

Why file-based, not DB-based

Original design (v3) used file_references + wiki_pages Postgres tables with embeddings. That was dropped in favour of file-frontmatter hashing once Obsidian became the canonical vault. The vault is now self-contained, portable, and version-controlled via the NAS git remote without any DB dependency. See personal-os-state-2026-04-27 Decision 8 and Decision 2 (obsidian-mcp replaced personal-wiki MCP).

personal-os-schema personal-os-state-2026-04-27 personal-os-catchup-plan-2026-04-27

3.1 KiB Raw Blame History