diff --git a/family/how-to/vault-git-sync.md b/family/how-to/vault-git-sync.md index 8c93bb2f..9a836fc1 100644 --- a/family/how-to/vault-git-sync.md +++ b/family/how-to/vault-git-sync.md @@ -1,94 +1,129 @@ --- -title: Vault Git Sync — Setup & Pitfalls -created: '2026-05-23' -updated: '2026-05-23' +title: Vault Git Sync — Architecture & Scripts +updated: '2026-06-02' type: tech -namespace: wiki -tags: [vault, git, sync, infra, obsidian] +tags: + - vault + - git + - sync + - infra + - obsidian --- # Vault Git Sync -Obsidian vault is a bare git repo on TrueNAS (`mallexxx.duckdns.org:/mnt/RED_2TB/storage/git/obsidian-vault.git`). -Three hosts sync to it: Eagle, Kraken, Taiga. +Obsidian vault is a bare git repo on TrueNAS: +`/mnt/RED_2TB/storage/git/obsidian-vault.git` +Also mirrored in Gitea: `https://git.mallexxx.duckdns.org/git_admin/obsidian-vault` -## Sync Strategy (correct) +Four clients sync to it: Eagle, Kraken, Taiga (vault), Taiga (syncthing/phone). -All hosts must follow this order: +## Architecture ``` -1. stash local changes (git stash) -2. pull from remote (git fetch + git merge) -3. pop stash (git stash pop) -4. commit if anything new (git add -A + git commit) -5. push (git push) +bare repo (obsidian-vault.git) — source of truth + ↑↓ ↑↓ ↑↓ ↑↓ +Eagle Kraken Taiga /vault Taiga /obsidian-syncthing +full clone sparse clone sparse clone full clone +~/obsidian ~/obsidian personal/family/ phone ↔ Syncthing + personal/family/ .obsidian/ + .obsidian/ ``` -**Why this order matters:** `git add -A` before pull stages deletions of files -that exist on remote but not locally. This caused Taiga to delete 32+ wiki files -on 2026-05-23 (two incidents: 3a44a55, c9020c4). +Taiga runs 3-phase sync (order matters): +1. **syncthing** → phone changes land in bare repo first +2. **vault** → agent (mcpvault) changes land in bare repo +3. **syncthing** → agent changes propagate to phone + +## Sync Algorithm (same for all clients) + +```bash +git add -A && git commit # commit local changes +git fetch origin main # get remote state +git merge origin/main --no-edit # 3-way merge (conflict → commit markers) +git push origin main # push back +``` + +No stash. No GIT_DIR/GIT_WORK_TREE workarounds. ## Scripts -| Host | Script | Status | -|--------|------------------------------------|----------------| -| Eagle | `~/scripts/sync-vault.sh` | ✅ Correct | -| Kraken | `~/scripts/sync-vault-partial.sh` | ✅ Correct | -| Taiga | `/opt/data/sync-vault.sh` | ✅ Fixed 2026-05-23 | +| Host | Script | Type | Trigger | +|--------|-------------------------------|---------------|----------------------| +| Eagle | `~/scripts/sync-vault.sh` | full clone | Hermes cron */5 | +| Kraken | `~/scripts/sync-vault.sh` | sparse clone | host crontab */5 | +| Taiga | `/opt/data/sync-vault.sh` | 3-phase orch. | Hermes cron */5 | -All scripts now use scoped `git add personal/ family/ .obsidian/` (not `git add -A`) before stash. -Taiga additionally uses sparse checkout (only `personal/` and `family/`) — double safeguard: -even if a bug reintroduces `git add -A`, sparse checkout means wiki/ is never checked out locally. +All clients share a common library: `sync-vault-lib.sh` (same dir as sync-vault.sh). +Source of truth for scripts: `~/Developer/vault-sync-test/scripts/` on Eagle. -## Taiga Architecture +## Taiga Container Mounts -Taiga is TrueNAS running Hermes in Docker (`/mnt/RED_2TB/docker/hermes/`). -The sync script runs **inside the container** with: -- `GIT_WORK_TREE=/vault` → mounted from `/mnt/RED_2TB/storage/obsidian` -- `GIT_DIR=/vault.git` → mounted from `/mnt/RED_2TB/storage/git/obsidian-vault.git` (the bare repo itself) +| Container path | Host path | Purpose | +|----------------------|----------------------------------------------------|--------------------------| +| `/vault` | `/mnt/RED_2TB/storage/obsidian` | sparse clone (agent rw) | +| `/vault.git` | `/mnt/RED_2TB/storage/git/obsidian-vault.git` | bare repo (git remote) | +| `/obsidian-syncthing`| `/mnt/RED_2TB/storage/obsidian-syncthing` | full clone (phone sync) | +| `/opt/data` | `/mnt/RED_2TB/docker/hermes/config` | Hermes config + scripts | -Taiga IS the repo — no remote push needed. Eagle pushes to `nas/main`, which is this same bare repo. +`/vault` is a sparse clone with `remote = file:///vault.git`. +`/vault.git` is hidden from the agent (mcpvault points to `/vault`). -Script location: `/mnt/RED_2TB/docker/hermes/config/sync-vault.sh` (= `/opt/data/sync-vault.sh` inside container) +## Sparse Checkout Paths -SSH access: `ssh taiga` (alias in ~/.ssh/config → `truenas_admin@mallexxx.duckdns.org`) - -To deploy a script fix: -```bash -scp ~/scripts/sync-vault-taiga.sh taiga:/mnt/RED_2TB/docker/hermes/config/sync-vault.sh +Taiga and Kraken check out only: ``` +personal/ +family/ +.obsidian/ +``` +`work/` and `wiki/` are never checked out on sparse clients — they never touch those paths. -**NEVER use local IP for TrueNAS** — always `mallexxx.duckdns.org` or `ssh taiga`. +## Deploy a Script Fix + +```bash +# Edit on Eagle: +~/Developer/vault-sync-test/scripts/sync-vault-lib.sh # common algorithm +~/Developer/vault-sync-test/scripts/sync-vault.sh # Taiga 3-phase orchestrator + +# Deploy to Taiga: +scp ~/Developer/vault-sync-test/scripts/sync-vault-lib.sh \ + ~/Developer/vault-sync-test/scripts/sync-vault.sh \ + truenas_admin@mallexxx.duckdns.org:/mnt/RED_2TB/docker/hermes/config/ + +# Deploy to Kraken (via sudo): +cat ~/scripts/sync-vault-lib.sh | ssh kraken "sudo tee ~/scripts/sync-vault-lib.sh > /dev/null" +cat ~/scripts/sync-vault.sh | ssh kraken "sudo tee ~/scripts/sync-vault.sh > /dev/null" +``` ## .gitignore -Plugin binaries are excluded to prevent cross-device obsidian-git version conflicts: - +Plugin binaries excluded to avoid cross-device conflicts: ``` .obsidian/plugins/obsidian-git/main.js .obsidian/plugins/obsidian-git/styles.css .obsidian/plugins/obsidian-git/manifest.json ``` +`data.json` (plugin config) is tracked — shared settings. -Each device manages its own plugin binaries via Obsidian's built-in update mechanism. -`data.json` (plugin config) IS tracked — shared settings across devices. +## Regression Tests -## Incident: 2026-05-23 +`~/Developer/vault-sync-test/test-sync.sh` — 16 scenarios (A–P): +covers Cyrillic filenames, moves, deletes, conflicts, work/ phantom-delete, 3-phase sync. -**Root cause:** Taiga's `/opt/data/sync-vault.sh` did `git add -A` before `git pull`. - -**Sequence:** -- 22:02 UTC Eagle created 15 wiki files (d757029, 6dc4b89) -- 00:00 UTC Taiga ran sync → staged deletions (local didn't have wiki/) → commit 3a44a55 -- 03:02 UTC Taiga ran again → deleted 8 more files (c9020c4) - -**Recovery:** `git checkout -- ` for each file from last-good commits. -32 files restored in commit 620e2df. - -## Stash Cleanup - -Orphaned stashes from obsidian-git mobile syncs accumulate. Safe to drop: ```bash -git stash drop stash@{N} # drop specific, or: -git stash clear # drop all (only if no unrecovered work) +cd ~/Developer/vault-sync-test && bash test-sync.sh ``` + +## Pitfalls + +- `git add pathspec` fails with `fatal: pathspec did not match any files` when a directory doesn't exist on disk — use `git add -A` instead +- `git add -A` in sparse worktree still stages deletions of out-of-cone files tracked in index — need to unstage them (fixed in old script; irrelevant with proper clone) +- `core.quotePath=true` (default) escapes Cyrillic paths in `git ls-files` output — use `-c core.quotePath=false` +- ZFS on TrueNAS blocks `chmod` — `git init` fails from host; must run from inside Docker container or create `.git` structure manually + +## History + +- **2026-05-23**: Taiga deleted 32+ wiki files. Root cause: `git add -A` before merge staged deletions of files outside sparse cone. Fixed with scoped `git add`. +- **2026-06-01**: "Керамика" lost from gift-ideas. Root cause: `git add $SPARSE_PATHS` failed silently when `.obsidian/` absent — patch+move not committed. +- **2026-06-02**: Full architecture migration. Taiga `/vault` converted from `GIT_DIR=bare GIT_WORK_TREE` workaround to proper sparse clone. All clients unified on same `commit→fetch→merge→push` algorithm. Sync interval: every 5 min.