[2026-06-02] taiga-vault: family/how-to/vault-git-sync.md
This commit is contained in:
Taiga
@@ -1,94 +1,129 @@
|
|||||||
---
|
---
|
||||||
title: Vault Git Sync — Setup & Pitfalls
|
title: Vault Git Sync — Architecture & Scripts
|
||||||
created: '2026-05-23'
|
updated: '2026-06-02'
|
||||||
updated: '2026-05-23'
|
|
||||||
type: tech
|
type: tech
|
||||||
namespace: wiki
|
tags:
|
||||||
tags: [vault, git, sync, infra, obsidian]
|
- vault
|
||||||
|
- git
|
||||||
|
- sync
|
||||||
|
- infra
|
||||||
|
- obsidian
|
||||||
---
|
---
|
||||||
|
|
||||||
# Vault Git Sync
|
# Vault Git Sync
|
||||||
|
|
||||||
Obsidian vault is a bare git repo on TrueNAS (`mallexxx.duckdns.org:/mnt/RED_2TB/storage/git/obsidian-vault.git`).
|
Obsidian vault is a bare git repo on TrueNAS:
|
||||||
Three hosts sync to it: Eagle, Kraken, Taiga.
|
`/mnt/RED_2TB/storage/git/obsidian-vault.git`
|
||||||
|
Also mirrored in Gitea: `https://git.mallexxx.duckdns.org/git_admin/obsidian-vault`
|
||||||
|
|
||||||
## Sync Strategy (correct)
|
Four clients sync to it: Eagle, Kraken, Taiga (vault), Taiga (syncthing/phone).
|
||||||
|
|
||||||
All hosts must follow this order:
|
## Architecture
|
||||||
|
|
||||||
```
|
```
|
||||||
1. stash local changes (git stash)
|
bare repo (obsidian-vault.git) — source of truth
|
||||||
2. pull from remote (git fetch + git merge)
|
↑↓ ↑↓ ↑↓ ↑↓
|
||||||
3. pop stash (git stash pop)
|
Eagle Kraken Taiga /vault Taiga /obsidian-syncthing
|
||||||
4. commit if anything new (git add -A + git commit)
|
full clone sparse clone sparse clone full clone
|
||||||
5. push (git push)
|
~/obsidian ~/obsidian personal/family/ phone ↔ Syncthing
|
||||||
|
personal/family/ .obsidian/
|
||||||
|
.obsidian/
|
||||||
```
|
```
|
||||||
|
|
||||||
**Why this order matters:** `git add -A` before pull stages deletions of files
|
Taiga runs 3-phase sync (order matters):
|
||||||
that exist on remote but not locally. This caused Taiga to delete 32+ wiki files
|
1. **syncthing** → phone changes land in bare repo first
|
||||||
on 2026-05-23 (two incidents: 3a44a55, c9020c4).
|
2. **vault** → agent (mcpvault) changes land in bare repo
|
||||||
|
3. **syncthing** → agent changes propagate to phone
|
||||||
|
|
||||||
|
## Sync Algorithm (same for all clients)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add -A && git commit # commit local changes
|
||||||
|
git fetch origin main # get remote state
|
||||||
|
git merge origin/main --no-edit # 3-way merge (conflict → commit markers)
|
||||||
|
git push origin main # push back
|
||||||
|
```
|
||||||
|
|
||||||
|
No stash. No GIT_DIR/GIT_WORK_TREE workarounds.
|
||||||
|
|
||||||
## Scripts
|
## Scripts
|
||||||
|
|
||||||
| Host | Script | Status |
|
| Host | Script | Type | Trigger |
|
||||||
|--------|------------------------------------|----------------|
|
|--------|-------------------------------|---------------|----------------------|
|
||||||
| Eagle | `~/scripts/sync-vault.sh` | ✅ Correct |
|
| Eagle | `~/scripts/sync-vault.sh` | full clone | Hermes cron */5 |
|
||||||
| Kraken | `~/scripts/sync-vault-partial.sh` | ✅ Correct |
|
| Kraken | `~/scripts/sync-vault.sh` | sparse clone | host crontab */5 |
|
||||||
| Taiga | `/opt/data/sync-vault.sh` | ✅ Fixed 2026-05-23 |
|
| Taiga | `/opt/data/sync-vault.sh` | 3-phase orch. | Hermes cron */5 |
|
||||||
|
|
||||||
All scripts now use scoped `git add personal/ family/ .obsidian/` (not `git add -A`) before stash.
|
All clients share a common library: `sync-vault-lib.sh` (same dir as sync-vault.sh).
|
||||||
Taiga additionally uses sparse checkout (only `personal/` and `family/`) — double safeguard:
|
Source of truth for scripts: `~/Developer/vault-sync-test/scripts/` on Eagle.
|
||||||
even if a bug reintroduces `git add -A`, sparse checkout means wiki/ is never checked out locally.
|
|
||||||
|
|
||||||
## Taiga Architecture
|
## Taiga Container Mounts
|
||||||
|
|
||||||
Taiga is TrueNAS running Hermes in Docker (`/mnt/RED_2TB/docker/hermes/`).
|
| Container path | Host path | Purpose |
|
||||||
The sync script runs **inside the container** with:
|
|----------------------|----------------------------------------------------|--------------------------|
|
||||||
- `GIT_WORK_TREE=/vault` → mounted from `/mnt/RED_2TB/storage/obsidian`
|
| `/vault` | `/mnt/RED_2TB/storage/obsidian` | sparse clone (agent rw) |
|
||||||
- `GIT_DIR=/vault.git` → mounted from `/mnt/RED_2TB/storage/git/obsidian-vault.git` (the bare repo itself)
|
| `/vault.git` | `/mnt/RED_2TB/storage/git/obsidian-vault.git` | bare repo (git remote) |
|
||||||
|
| `/obsidian-syncthing`| `/mnt/RED_2TB/storage/obsidian-syncthing` | full clone (phone sync) |
|
||||||
|
| `/opt/data` | `/mnt/RED_2TB/docker/hermes/config` | Hermes config + scripts |
|
||||||
|
|
||||||
Taiga IS the repo — no remote push needed. Eagle pushes to `nas/main`, which is this same bare repo.
|
`/vault` is a sparse clone with `remote = file:///vault.git`.
|
||||||
|
`/vault.git` is hidden from the agent (mcpvault points to `/vault`).
|
||||||
|
|
||||||
Script location: `/mnt/RED_2TB/docker/hermes/config/sync-vault.sh` (= `/opt/data/sync-vault.sh` inside container)
|
## Sparse Checkout Paths
|
||||||
|
|
||||||
SSH access: `ssh taiga` (alias in ~/.ssh/config → `truenas_admin@mallexxx.duckdns.org`)
|
Taiga and Kraken check out only:
|
||||||
|
|
||||||
To deploy a script fix:
|
|
||||||
```bash
|
|
||||||
scp ~/scripts/sync-vault-taiga.sh taiga:/mnt/RED_2TB/docker/hermes/config/sync-vault.sh
|
|
||||||
```
|
```
|
||||||
|
personal/
|
||||||
|
family/
|
||||||
|
.obsidian/
|
||||||
|
```
|
||||||
|
`work/` and `wiki/` are never checked out on sparse clients — they never touch those paths.
|
||||||
|
|
||||||
**NEVER use local IP for TrueNAS** — always `mallexxx.duckdns.org` or `ssh taiga`.
|
## Deploy a Script Fix
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Edit on Eagle:
|
||||||
|
~/Developer/vault-sync-test/scripts/sync-vault-lib.sh # common algorithm
|
||||||
|
~/Developer/vault-sync-test/scripts/sync-vault.sh # Taiga 3-phase orchestrator
|
||||||
|
|
||||||
|
# Deploy to Taiga:
|
||||||
|
scp ~/Developer/vault-sync-test/scripts/sync-vault-lib.sh \
|
||||||
|
~/Developer/vault-sync-test/scripts/sync-vault.sh \
|
||||||
|
truenas_admin@mallexxx.duckdns.org:/mnt/RED_2TB/docker/hermes/config/
|
||||||
|
|
||||||
|
# Deploy to Kraken (via sudo):
|
||||||
|
cat ~/scripts/sync-vault-lib.sh | ssh kraken "sudo tee ~/scripts/sync-vault-lib.sh > /dev/null"
|
||||||
|
cat ~/scripts/sync-vault.sh | ssh kraken "sudo tee ~/scripts/sync-vault.sh > /dev/null"
|
||||||
|
```
|
||||||
|
|
||||||
## .gitignore
|
## .gitignore
|
||||||
|
|
||||||
Plugin binaries are excluded to prevent cross-device obsidian-git version conflicts:
|
Plugin binaries excluded to avoid cross-device conflicts:
|
||||||
|
|
||||||
```
|
```
|
||||||
.obsidian/plugins/obsidian-git/main.js
|
.obsidian/plugins/obsidian-git/main.js
|
||||||
.obsidian/plugins/obsidian-git/styles.css
|
.obsidian/plugins/obsidian-git/styles.css
|
||||||
.obsidian/plugins/obsidian-git/manifest.json
|
.obsidian/plugins/obsidian-git/manifest.json
|
||||||
```
|
```
|
||||||
|
`data.json` (plugin config) is tracked — shared settings.
|
||||||
|
|
||||||
Each device manages its own plugin binaries via Obsidian's built-in update mechanism.
|
## Regression Tests
|
||||||
`data.json` (plugin config) IS tracked — shared settings across devices.
|
|
||||||
|
|
||||||
## Incident: 2026-05-23
|
`~/Developer/vault-sync-test/test-sync.sh` — 16 scenarios (A–P):
|
||||||
|
covers Cyrillic filenames, moves, deletes, conflicts, work/ phantom-delete, 3-phase sync.
|
||||||
|
|
||||||
**Root cause:** Taiga's `/opt/data/sync-vault.sh` did `git add -A` before `git pull`.
|
|
||||||
|
|
||||||
**Sequence:**
|
|
||||||
- 22:02 UTC Eagle created 15 wiki files (d757029, 6dc4b89)
|
|
||||||
- 00:00 UTC Taiga ran sync → staged deletions (local didn't have wiki/) → commit 3a44a55
|
|
||||||
- 03:02 UTC Taiga ran again → deleted 8 more files (c9020c4)
|
|
||||||
|
|
||||||
**Recovery:** `git checkout <hash> -- <file>` for each file from last-good commits.
|
|
||||||
32 files restored in commit 620e2df.
|
|
||||||
|
|
||||||
## Stash Cleanup
|
|
||||||
|
|
||||||
Orphaned stashes from obsidian-git mobile syncs accumulate. Safe to drop:
|
|
||||||
```bash
|
```bash
|
||||||
git stash drop stash@{N} # drop specific, or:
|
cd ~/Developer/vault-sync-test && bash test-sync.sh
|
||||||
git stash clear # drop all (only if no unrecovered work)
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Pitfalls
|
||||||
|
|
||||||
|
- `git add pathspec` fails with `fatal: pathspec did not match any files` when a directory doesn't exist on disk — use `git add -A` instead
|
||||||
|
- `git add -A` in sparse worktree still stages deletions of out-of-cone files tracked in index — need to unstage them (fixed in old script; irrelevant with proper clone)
|
||||||
|
- `core.quotePath=true` (default) escapes Cyrillic paths in `git ls-files` output — use `-c core.quotePath=false`
|
||||||
|
- ZFS on TrueNAS blocks `chmod` — `git init` fails from host; must run from inside Docker container or create `.git` structure manually
|
||||||
|
|
||||||
|
## History
|
||||||
|
|
||||||
|
- **2026-05-23**: Taiga deleted 32+ wiki files. Root cause: `git add -A` before merge staged deletions of files outside sparse cone. Fixed with scoped `git add`.
|
||||||
|
- **2026-06-01**: "Керамика" lost from gift-ideas. Root cause: `git add $SPARSE_PATHS` failed silently when `.obsidian/` absent — patch+move not committed.
|
||||||
|
- **2026-06-02**: Full architecture migration. Taiga `/vault` converted from `GIT_DIR=bare GIT_WORK_TREE` workaround to proper sparse clone. All clients unified on same `commit→fetch→merge→push` algorithm. Sync interval: every 5 min.
|
||||||
|
|||||||
Reference in New Issue
Block a user