[2026-05-30] taiga sync: .stfolder/syncthing-folder-940f59.txt wiki/personal-os-catchup-plan-2026-04-27.md wiki/personal-os-schema.md wiki/personal-os-state-2026-04-27.md wiki/wiki-ingest-process.md
This commit is contained in:
@@ -1,114 +0,0 @@
|
||||
---
|
||||
source: raw/wiki-ingest-prompt.md
|
||||
content_hash: 29f6a0914d428cf62eeb2a832d6b3048edf3e1d16aa7714c09f686377394d128
|
||||
namespace: work
|
||||
last_synced: 2026-05-30
|
||||
confidence: 0.9
|
||||
tags: [personal-os, wiki, ingest, agent-prompt]
|
||||
---
|
||||
|
||||
# Wiki Ingest Process
|
||||
|
||||
The wiki-ingest agent maintains `~/obsidian/wiki/` by synthesising
|
||||
markdown pages from source files exposed as symlinks under
|
||||
`~/obsidian/raw/`. It is a local `claude -p` subprocess invoked nightly
|
||||
by the `personal.os.wiki-ingest` launchd agent (22:00 daily) — see
|
||||
[[personal-os-state-2026-04-27]] for the runtime layout.
|
||||
|
||||
## Inputs
|
||||
|
||||
Each entry under `~/obsidian/raw/` is a symlink to an external project
|
||||
file. The wiki keeps a synthesised page per source, identified by a
|
||||
frontmatter `source: raw/<filename>` line and tracked for staleness via a
|
||||
`content_hash` (SHA256) in the same frontmatter.
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 0 — Discover changes
|
||||
For every file in `~/obsidian/raw/`:
|
||||
1. Read the file content.
|
||||
2. Compute SHA256 (`shasum -a 256 <path> | awk '{print $1}'`).
|
||||
3. Search `~/obsidian/wiki/` for any `.md` whose frontmatter has
|
||||
`source: raw/<filename>`.
|
||||
4. If a page exists and `content_hash` matches → skip.
|
||||
5. Otherwise add to the work queue.
|
||||
|
||||
If the queue is empty, output `Wiki is up to date. Nothing to ingest.`
|
||||
and stop.
|
||||
|
||||
### Step 1 — Process each changed file
|
||||
- **Find or create**: update the existing page if found, otherwise
|
||||
create `~/obsidian/wiki/<topic-name>.md`. Filenames follow the topic,
|
||||
not the source filename — e.g. `personal-os-schema.md`, not `schema.md`.
|
||||
- **Synthesise**: extract key facts, decisions, and structure. Never
|
||||
copy the source verbatim. Keep each page under 800 words; split into
|
||||
linked pages if the topic is too large.
|
||||
- **Cross-reference**: use `[[double brackets]]` to link related wiki
|
||||
pages.
|
||||
|
||||
### Step 2 — Report
|
||||
After processing, print a summary with counts of created, updated, and
|
||||
skipped pages, followed by the list of pages touched and their sources.
|
||||
|
||||
## Frontmatter contract
|
||||
|
||||
```yaml
|
||||
source: raw/<filename>
|
||||
content_hash: <sha256>
|
||||
namespace: work
|
||||
last_synced: <YYYY-MM-DD>
|
||||
confidence: 0.8
|
||||
tags: [tag1, tag2]
|
||||
```
|
||||
|
||||
`confidence` is a coarse trust signal:
|
||||
|
||||
| Value | Meaning |
|
||||
|-------|--------------------------------------------------------|
|
||||
| 1.0 | Reserved for human-written notes (`work/`, `personal/`, `family/`). |
|
||||
| 0.9 | Highly structured, authoritative source. |
|
||||
| 0.8 | Default for clear single-source synthesis. |
|
||||
| 0.6 | Inferred or partial content. |
|
||||
|
||||
## Hard rules
|
||||
|
||||
- Never copy source files verbatim — always synthesise.
|
||||
- Never edit a wiki page with `confidence: 1.0` — those are
|
||||
human-authored.
|
||||
- Never process files belonging to `namespace: family` unless explicitly
|
||||
instructed.
|
||||
- Always update `content_hash` and `last_synced` after rewriting a page.
|
||||
- If a symlink in `raw/` is broken (target missing), skip it — do not
|
||||
create a stub wiki page.
|
||||
|
||||
## Why this lives in the prompt, not a DB
|
||||
|
||||
Per Decision 8 in [[personal-os-state-2026-04-27]], the original v3 design
|
||||
used `file_references` + `wiki_pages` Postgres tables. That was replaced
|
||||
with file-only state: SHA256 in frontmatter is enough for incremental
|
||||
sync, the vault is self-contained and portable, and Obsidian (via the
|
||||
`obsidian-mcp` package) becomes the single source of truth. The DB
|
||||
tables still exist in [[personal-os-schema]] but are unused in the live
|
||||
flow.
|
||||
|
||||
## Operational notes
|
||||
|
||||
- Runs once a day under launchd, so the prompt is allowed to be slow.
|
||||
- Invoked without `--allowedTools` in v1 (tracked as low-severity tech
|
||||
debt in [[personal-os-state-2026-04-27]] §6). The agent may try to
|
||||
call tools it doesn't have — monitor stderr.
|
||||
- The `commit-vault.sh` script (23:00 daily) picks up any new wiki
|
||||
pages and pushes them to the NAS git remote.
|
||||
|
||||
## Key Points
|
||||
|
||||
- Source-of-truth is the filesystem; SHA256 in YAML frontmatter is the
|
||||
staleness signal.
|
||||
- Filenames follow topic, not source filename.
|
||||
- Synthesis (≤ 800 words), never verbatim.
|
||||
- Confidence 1.0 pages are human-owned and untouchable.
|
||||
- Family namespace requires explicit user opt-in per run.
|
||||
|
||||
## Related
|
||||
[[personal-os-state-2026-04-27]] [[personal-os-schema]]
|
||||
[[personal-os-catchup-plan-2026-04-27]] [[ddg-asana-workflow]]
|
||||
Reference in New Issue
Block a user