115 lines
4.2 KiB
Markdown
115 lines
4.2 KiB
Markdown
---
|
|
source: raw/wiki-ingest-prompt.md
|
|
content_hash: 29f6a0914d428cf62eeb2a832d6b3048edf3e1d16aa7714c09f686377394d128
|
|
namespace: work
|
|
last_synced: 2026-05-30
|
|
confidence: 0.9
|
|
tags: [personal-os, wiki, ingest, agent-prompt]
|
|
---
|
|
|
|
# Wiki Ingest Process
|
|
|
|
The wiki-ingest agent maintains `~/obsidian/wiki/` by synthesising
|
|
markdown pages from source files exposed as symlinks under
|
|
`~/obsidian/raw/`. It is a local `claude -p` subprocess invoked nightly
|
|
by the `personal.os.wiki-ingest` launchd agent (22:00 daily) — see
|
|
[[personal-os-state-2026-04-27]] for the runtime layout.
|
|
|
|
## Inputs
|
|
|
|
Each entry under `~/obsidian/raw/` is a symlink to an external project
|
|
file. The wiki keeps a synthesised page per source, identified by a
|
|
frontmatter `source: raw/<filename>` line and tracked for staleness via a
|
|
`content_hash` (SHA256) in the same frontmatter.
|
|
|
|
## Workflow
|
|
|
|
### Step 0 — Discover changes
|
|
For every file in `~/obsidian/raw/`:
|
|
1. Read the file content.
|
|
2. Compute SHA256 (`shasum -a 256 <path> | awk '{print $1}'`).
|
|
3. Search `~/obsidian/wiki/` for any `.md` whose frontmatter has
|
|
`source: raw/<filename>`.
|
|
4. If a page exists and `content_hash` matches → skip.
|
|
5. Otherwise add to the work queue.
|
|
|
|
If the queue is empty, output `Wiki is up to date. Nothing to ingest.`
|
|
and stop.
|
|
|
|
### Step 1 — Process each changed file
|
|
- **Find or create**: update the existing page if found, otherwise
|
|
create `~/obsidian/wiki/<topic-name>.md`. Filenames follow the topic,
|
|
not the source filename — e.g. `personal-os-schema.md`, not `schema.md`.
|
|
- **Synthesise**: extract key facts, decisions, and structure. Never
|
|
copy the source verbatim. Keep each page under 800 words; split into
|
|
linked pages if the topic is too large.
|
|
- **Cross-reference**: use `[[double brackets]]` to link related wiki
|
|
pages.
|
|
|
|
### Step 2 — Report
|
|
After processing, print a summary with counts of created, updated, and
|
|
skipped pages, followed by the list of pages touched and their sources.
|
|
|
|
## Frontmatter contract
|
|
|
|
```yaml
|
|
source: raw/<filename>
|
|
content_hash: <sha256>
|
|
namespace: work
|
|
last_synced: <YYYY-MM-DD>
|
|
confidence: 0.8
|
|
tags: [tag1, tag2]
|
|
```
|
|
|
|
`confidence` is a coarse trust signal:
|
|
|
|
| Value | Meaning |
|
|
|-------|--------------------------------------------------------|
|
|
| 1.0 | Reserved for human-written notes (`work/`, `personal/`, `family/`). |
|
|
| 0.9 | Highly structured, authoritative source. |
|
|
| 0.8 | Default for clear single-source synthesis. |
|
|
| 0.6 | Inferred or partial content. |
|
|
|
|
## Hard rules
|
|
|
|
- Never copy source files verbatim — always synthesise.
|
|
- Never edit a wiki page with `confidence: 1.0` — those are
|
|
human-authored.
|
|
- Never process files belonging to `namespace: family` unless explicitly
|
|
instructed.
|
|
- Always update `content_hash` and `last_synced` after rewriting a page.
|
|
- If a symlink in `raw/` is broken (target missing), skip it — do not
|
|
create a stub wiki page.
|
|
|
|
## Why this lives in the prompt, not a DB
|
|
|
|
Per Decision 8 in [[personal-os-state-2026-04-27]], the original v3 design
|
|
used `file_references` + `wiki_pages` Postgres tables. That was replaced
|
|
with file-only state: SHA256 in frontmatter is enough for incremental
|
|
sync, the vault is self-contained and portable, and Obsidian (via the
|
|
`obsidian-mcp` package) becomes the single source of truth. The DB
|
|
tables still exist in [[personal-os-schema]] but are unused in the live
|
|
flow.
|
|
|
|
## Operational notes
|
|
|
|
- Runs once a day under launchd, so the prompt is allowed to be slow.
|
|
- Invoked without `--allowedTools` in v1 (tracked as low-severity tech
|
|
debt in [[personal-os-state-2026-04-27]] §6). The agent may try to
|
|
call tools it doesn't have — monitor stderr.
|
|
- The `commit-vault.sh` script (23:00 daily) picks up any new wiki
|
|
pages and pushes them to the NAS git remote.
|
|
|
|
## Key Points
|
|
|
|
- Source-of-truth is the filesystem; SHA256 in YAML frontmatter is the
|
|
staleness signal.
|
|
- Filenames follow topic, not source filename.
|
|
- Synthesis (≤ 800 words), never verbatim.
|
|
- Confidence 1.0 pages are human-owned and untouchable.
|
|
- Family namespace requires explicit user opt-in per run.
|
|
|
|
## Related
|
|
[[personal-os-state-2026-04-27]] [[personal-os-schema]]
|
|
[[personal-os-catchup-plan-2026-04-27]] [[ddg-asana-workflow]]
|