diff --git a/personal/projects/personal-os/wiki-curation-runs/2026-05-18-02h.md b/personal/projects/personal-os/wiki-curation-runs/2026-05-18-02h.md new file mode 100644 index 00000000..7098d935 --- /dev/null +++ b/personal/projects/personal-os/wiki-curation-runs/2026-05-18-02h.md @@ -0,0 +1,56 @@ +--- +date: 2026-05-18 +status: complete +run: 02:00 cron (second run of the day) +--- +# Wiki Curation — 2026-05-18 (02:00) + +> Note: A 2026-05-18 state file already existed (status: complete, from ~01:05 AM earlier). This is the scheduled 02:00 cron run — proceeded with full session crystallization pass. + +## Done + +- **Conflict scan:** No git conflict markers across ~/obsidian/ ✅ +- **Previous run:** 2026-05-18 01:05 state complete, 3 pages created earlier — noted, no resume needed +- **Change detection:** + - `media-toolbox-kraken.md` — *arr stack setup section added (2026-05-17 22:50) + - `media-pipeline-tracker-parsers.md` — roadmap Steps A-C actulized (2026-05-17 18:36) + - `htpc-emulators-setup.md` — NES shortcuts + HD packs (2026-05-17 22:24) + - Sessions: 7 sessions covering Hermes Docker deploy, *arr stack, HDD error on Kraken +- **Inbox:** empty ✅ +- **Sessions crystallized:** 7/7 checked, all crystallized into wiki updates: + - sessions 000940–005825: Hermes Docker deploy pitfalls (`gateway run` vs `gateway`, chown loop, skills path) → `tech/kraken-network.md` + - session 001134: HDD Hardware Error (read-only ext4, recovery procedure) → `tech/kraken-network.md` + - sessions 011803–012457: arr stack pitfalls (Pi-hole DNS hijack, curl absent in container) → `tech/kraken-network.md` +- **Pages updated (3):** + - `tech/kraken-network.md` — *arr stack section, HDD error recovery, pitfalls (curl in container, Pi-hole, skills path) + - `tech/media-pipeline-pitfalls.md` — bugs 8/12-15 (KP domain, underscore year, OutputConfig JSON, xvid markers, year-as-title); tracker-parsers Steps A-C; QuerySnapshotGenerator + - `personal-os-architecture.md` — Kraken skills path note + [[concepts/kraken-media-stack]] link +- **Synthesis page created (1):** + - `concepts/kraken-media-stack.md` — cross-synthesis of media-pipeline + *arr stack + Jellyfin + Hermes on Kraken; full architecture diagram, pitfall map, current status (5+ source pages) +- **Stale check:** all pages current ✅ +- **Lint:** 26 wiki files; all OK ✅ +- **Index:** 22 pages total (+3 from earlier run, +1 this run = 22); last updated 2026-05-18 +- **Log:** appended 02:00 entry to wiki/log.md + +## Validation fixes + +- none + +## Unresolved conflicts + +(none) + +## Unfinished + +(none) + +## Stats + +- Conflicts resolved: 0 +- Inbox processed: 0 +- Sessions crystallized: 7 / 7 checked +- Synthesis pages created: 1 (concepts/kraken-media-stack) +- Pages updated: 3 (kraken-network, media-pipeline-pitfalls, personal-os-architecture) +- Pages created: 1 (concepts/kraken-media-stack) +- Lint issues: 0 +- Validation issues fixed: 0 diff --git a/wiki/SCHEMA.md b/wiki/SCHEMA.md index 587f5b8e..29c08d39 100644 --- a/wiki/SCHEMA.md +++ b/wiki/SCHEMA.md @@ -40,6 +40,7 @@ relevant to Alex's specific setup. | `tech/htpc-steam-emulators.md` | HTPC Steam/emulator rules, gamepad config, save paths | personal | | `concepts/agent-memory-architecture.md` | Three-level memory model for LLM agents; Hermes implementation | work | | `concepts/vault-strategy.md` | Three vault tasks: enrichment, wiki accumulation, proactive research | work | +| `concepts/kraken-media-stack.md` | Full Kraken media server: media-pipeline + *arr + Jellyfin; pitfalls synthesis | work | | `entities/library-app.md` | Personal book library UI on TrueNAS (687k books, deployed 2026-05-16) | personal | ## Curation Rules diff --git a/wiki/concepts/kraken-media-stack.md b/wiki/concepts/kraken-media-stack.md new file mode 100644 index 00000000..2fc73e84 --- /dev/null +++ b/wiki/concepts/kraken-media-stack.md @@ -0,0 +1,80 @@ +--- +title: Kraken Media Server — Full Stack +created: '2026-05-18' +updated: '2026-05-18' +type: concept +tags: + - kraken + - media-pipeline + - infra + - synthesis +confidence: high +--- +# Kraken Media Server — Full Stack + +Синтез: полная карта медиа-стека на Кракене (RPi5). Объединяет [[tech/kraken-network]], [[tech/media-pipeline-pitfalls]], [[tech/htpc-steam-emulators]]. + +## Архитектурный обзор + +Два параллельных конвейера для медиа-контента: + +### Конвейер 1 — media-pipeline (Swift, ручной импорт HTPC→Kraken) + +``` +HTPC (rsync) → /media/{movies,cartoons,series,documentaries}/ + → media-pipeline:kraken (Docker) → scan → resolve → apply + → /media/library/{movies,cartoons,series}/ (canonical + NFO) + → Jellyfin +``` + +- Resolve: TMDb (Tier 1) → KP (Tier 2) → IMDb scrape (Tier 3) → web search (Tier 4) +- Классификация по жанрам провайдера (Animation→cartoons, Documentary→documentaries), никогда по пути +- `unresolved.json` — очередь нераспознанных (13 items на 2026-05-16, все genuine absences) + +### Конвейер 2 — *arr stack (новые скачки через Radarr/Sonarr) + +``` +Prowlarr (RuTracker) → Radarr (7878) / Sonarr (8989) + → Transmission (9091) → /media/movies-radarr, /media/series-sonarr + → Jellyfin +``` + +- Раздельные root folder'ы (`movies-radarr`, `series-sonarr`) — не конфликтуют с media-pipeline library +- Jellyfin читает оба пути для Movies и Series + +## Сервисы и порты + +| Сервис | Порт | Данные | +|--------|------|--------| +| Jellyfin | 8096 | `/media/library/` + `/media/movies-radarr` + `/media/series-sonarr` | +| Transmission | 9091 | `/media/downloads/complete` | +| Radarr | 7878 | root: `/media/movies-radarr` | +| Sonarr | 8989 | root: `/media/series-sonarr` | +| Prowlarr | 9696 | индексер: RuTracker | +| media-pipeline | — | Docker run, не compose; config: `~/docker/media-pipeline/` | +| Hermes (Kraken) | — | Docker container, skills: `~/.hermes/skills/` | + +## Ключевые питфоллы (пережитые) + +1. **HDD read-only под нагрузкой** — rsync + ffmpeg transcode одновременно → Sense Key 0x4, ext4 remount-ro. Фикс: ребут → fsck → `docker compose up -d` (2026-05-18) +2. **`docker save | ssh` зависает** — через WireGuard tunnel. Фикс: `docker save -o /tmp/` + `scp` + `docker load` раздельно +3. **curl отсутствует в Hermes-контейнере** — скиллы с curl-командами не работают из контейнера. Навигация: хостовый Hermes vs container Hermes +4. **Pi-hole DNS hijack** → Sonarr/Radarr 503. Фикс: whitelist или прямые IP +5. **Skills path**: `~/.hermes/skills/` (хост) ≠ `/opt/data/skills/` (контейнер) +6. **KP API domain** переехал на `api.poiskkino.dev` (был `api.kinopoisk.dev`) +7. **providers block в config.json** — если не указан явно, все провайдеры disabled → 213 skipped вместо 278 applied + +## Статус (2026-05-18) + +- ✅ media-pipeline: 402/402 тестов, 13 unresolved (genuine absences в KP/TMDb) +- ✅ *arr stack: Prowlarr+Radarr+Sonarr+Transmission+Jellyfin — работают +- ✅ Hermes на Кракене: docker container, скиллы `download-media` + `arr-media-stack-setup` +- ⏳ Трекер-парсеры (Steps D-F): audio/non-video scanner, fixture тесты для rutor/nnm-club +- ⏳ Transmission torrent-fix: 271/293 файлов без привязки после pipeline apply + +## Агент-доступ + +Kraken-Hermes в контейнере (`hermes-kraken`) доступен через Zulip. Скачать медиа: +- Сериал → Sonarr API → Transmission +- Фильм → Radarr API → Transmission +- Только через curl (python3 запрещён в скилле) diff --git a/wiki/index.md b/wiki/index.md index 28542c73..898e262d 100644 --- a/wiki/index.md +++ b/wiki/index.md @@ -9,7 +9,7 @@ updated: '2026-05-16' # Wiki Index Auto-maintained catalog of all pages in `wiki/`. Updated by the daily curation cron. -> Last updated: 2026-05-18 | Total pages: 19 +> Last updated: 2026-05-18 | Total pages: 22 ## Core Architecture @@ -34,6 +34,7 @@ Auto-maintained catalog of all pages in `wiki/`. Updated by the daily curation c - [[concepts/agent-memory-architecture]] — Three-level memory model (episodic/semantic/procedural) for LLM agents; overflow rules; Hermes implementation - [[concepts/vault-strategy]] — Three distinct vault tasks: enrichment, LLM wiki accumulation, proactive research; topic map principle +- [[concepts/kraken-media-stack]] — Full Kraken media server: media-pipeline (Swift) + *arr stack + Jellyfin; pitfalls synthesis ## Entities diff --git a/wiki/log.md b/wiki/log.md index 6a8ef53a..6987dc2f 100644 --- a/wiki/log.md +++ b/wiki/log.md @@ -43,7 +43,36 @@ Activity log for the daily Eagle wiki-curation cron job. - Page size: all pages under 200 lines ✅ - New pages created: none -## 2026-05-18 +## 2026-05-18 (02:00 cron) + +- **Conflict scan:** No git conflict markers found across ~/obsidian/ ✅ +- **Previous run check:** 2026-05-18 state file exists (status: complete, 3 pages created) — already ran at ~01:05 AM. This is the 02:00 scheduled run; proceeding with full check. +- **Change detection (since 2026-05-17 02:00):** + - `[2026-05-17 22:50]` `family/projects/media-toolbox-kraken.md` — *arr stack setup section added + - `[2026-05-17 22:24]` `family/how-to/htpc-emulators-setup.md` — NES shortcuts + HD packs + - `[2026-05-17 18:36]` `personal/projects/media-pipeline-tracker-parsers.md` — roadmap actulized (Steps A-C done) + - `[2026-05-18 01:05]` Sessions: Hermes Docker deploy + *arr stack setup + HDD Hardware Error on Kraken + - family/documents/*, family/sort/ — operational, no wiki action +- **Inbox:** empty ✅ +- **Conflict scan:** No git conflict markers ✅ +- **Sessions crystallized (7 sessions, 2026-05-18):** + - session_000940, 001134, 001828, 004113, 005825 — Hermes Docker deploy pitfalls + *arr stack; crystallized into wiki updates ✅ + - session_011803 — arr-media-stack-setup: curl absent in container, Pi-hole DNS hijack; crystallized ✅ + - session_012457 — arr pitfalls continued; crystallized ✅ +- **Pages updated (3):** + - `tech/kraken-network.md` — added *arr stack section, HDD hardware error recovery, pitfalls (Pi-hole DNS, skills path, curl in container); bumped updated to 2026-05-18 + - `tech/media-pipeline-pitfalls.md` — added bugs 8/12/13/14/15, tracker-parsers Steps A-C, QuerySnapshotGenerator note; bumped updated to 2026-05-18 + - `personal-os-architecture.md` — added Kraken skills path note + link to kraken-media-stack; bumped last_updated to 2026-05-18 +- **New pages created (1):** + - `concepts/kraken-media-stack.md` — synthesis: media-pipeline (Swift) + *arr stack + Jellyfin; full architecture, key pitfalls, current status +- **Synthesis page:** ✅ `concepts/kraken-media-stack.md` — co-occurring concepts across 5+ pages (kraken-network, media-pipeline-pitfalls, htpc-steam-emulators, media-toolbox-kraken, arr sessions); no prior synthesis existed +- **Architecture check:** personal-os-architecture.md current ✅ (updated above) +- **Stale check:** all pages current ✅ +- **Lint:** 26 wiki files; frontmatter ✅; wikilinks ✅; orphans: none ✅; page sizes: all under 200 lines ✅ +- **Index:** Bumped to 22 pages, added concepts/kraken-media-stack; last updated 2026-05-18 +- **Validation:** all 5 created/modified files passed validation checks ✅ + + - **Conflict scan:** No git conflict markers found across all ~/obsidian/ ✅ - **Previous run (2026-05-17):** Status = complete, no Unfinished section — nothing to resume diff --git a/wiki/personal-os-architecture.md b/wiki/personal-os-architecture.md index 639cae66..17ece433 100644 --- a/wiki/personal-os-architecture.md +++ b/wiki/personal-os-architecture.md @@ -1,7 +1,7 @@ --- namespace: work tags: [system, architecture] -last_updated: 2026-05-16 +last_updated: 2026-05-18 confidence: 1.0 --- @@ -140,9 +140,12 @@ Executor runs | obsidian | `/opt/data/.local/bin/mcpvault /vault` | Vault access (sparse: personal/ + family/). Tavily search enabled. | **Kraken (RPi5) — Hermes agent (`hermes-kraken` container):** -| Name | Command | Used for | -|------|---------|---------| -| obsidian | `/opt/data/.local/bin/mcpvault /vault` | Vault access (sparse: personal/ + family/). Tavily search enabled. | +|| Name | Command | Used for | +||------|---------|---------|| +|| obsidian | `/opt/data/.local/bin/mcpvault /vault` | Vault access (sparse: personal/ + family/). Tavily search enabled. | + +Skills on Kraken: `~/.hermes/skills/` (host) — **not** visible to the container at `/opt/data/skills/`. Copy skills there if needed. +Media stack on Kraken: full *arr stack (Prowlarr/Radarr/Sonarr) + Transmission + Jellyfin + media-pipeline. See [[concepts/kraken-media-stack]]. > ⚠️ **Kraken image routing (open issue, 2026-05-14):** Gemini API credits depleted → fallback to OpenRouter models (gpt-oss-120b, minimax-m2) → these lack vision → image inbox messages fail. Desired fix: add `save_to_fs` mode to Hermes `image_routing` (save image locally, pass path to agent instead of bytes). Workaround: top up Gemini API credits on Kraken. Tracked in `personal/projects/personal-os/kraken-obsidian-write-bug.md`. diff --git a/wiki/tech/kraken-network.md b/wiki/tech/kraken-network.md index c462041a..1504460c 100644 --- a/wiki/tech/kraken-network.md +++ b/wiki/tech/kraken-network.md @@ -1,7 +1,7 @@ --- title: Kraken Network & Infra created: '2026-05-17' -updated: '2026-05-17' +updated: '2026-05-18' type: reference tags: - kraken @@ -37,3 +37,44 @@ confidence: high - Configs: `~/docker/media-pipeline/{config.json,cache/,unresolved.json}` - Нет docker-compose — только `docker run` напрямую - Монтировать `unresolved.json` явно: `-v "$(pwd)/unresolved.json:/config/unresolved.json"` + +## *arr стек (добавлен 2026-05-17) + +Prowlarr / Radarr / Sonarr + Transmission + Jellyfin — полностью настроены через API. +Скилл: `~/.hermes/skills/arr-media-stack-setup/SKILL.md` + +| Сервис | Порт | Compose dir | +|--------|------|-------------| +| Transmission | 9091 | `docker/transmission/` | +| Radarr | 7878 | `docker/radarr/` | +| Sonarr | 8989 | `docker/sonarr/` | +| Prowlarr | 9696 | `docker/prowlarr/` | +| Jellyfin | 8096 | `docker/jellyfin/` | + +Все контейнеры в `media_net` (external network), кроме Jellyfin (читает диск напрямую). +RuTracker индексер подключён через Prowlarr → fullSync в Radarr/Sonarr. + +Скачка медиа-агентом: скилл `download-media` (Radarr/Sonarr API — только `curl + jq`, без python3). + +**⚠️ Pitfall: Pi-hole DNS hijack** — при включённом Pi-hole Radarr/Sonarr могут получать 503 на DNS lookup (Pi-hole блокирует трекеры). Решение: whitelist в Pi-hole или прямые IP в настройках индексера. + +**⚠️ Pitfall: curl отсутствует в Hermes-контейнере** — агент внутри Docker не может исполнять curl-команды из скилла. Skills path: `~/.hermes/skills/` видна только хостовому Hermes; контейнерный Hermes ищет в `/opt/data/skills/`. + +## HDD Hardware Error — поведение при перегрузке + +**Симптом (2026-05-18):** При одновременном rsync + Jellyfin transcode + docker overlay → ext4 сбрасывает ошибку Sense Key 0x4 (Hardware Error) → файловая система ремонтирует в read-only (`remount-ro`). + +**Признаки:** Docker containers fail with `EIO` (I/O error -5), overlayfs недоступен. + +**Восстановление:** +1. Reboot Kraken (OMV или физически) +2. При монтировании ext4 автоматически запускает fsck +3. После ребута: `docker compose up -d` из каждого `~/docker//` + +**Профилактика:** не запускать rsync и ffmpeg transcode одновременно. Jellyfin transcode: pgsrip (subtitle extraction) особенно нагружает CPU+диск. + +## Связанные заметки + +- [[personal-os-architecture]] +- [[tech/media-pipeline-pitfalls]] +- [[tech/htpc-steam-emulators]] diff --git a/wiki/tech/media-pipeline-pitfalls.md b/wiki/tech/media-pipeline-pitfalls.md index a9d2a73f..ec129953 100644 --- a/wiki/tech/media-pipeline-pitfalls.md +++ b/wiki/tech/media-pipeline-pitfalls.md @@ -1,7 +1,7 @@ --- title: Media Pipeline — Pitfalls & Debug Notes created: '2026-05-16' -updated: '2026-05-16' +updated: '2026-05-18' type: entity tags: - media-pipeline @@ -12,8 +12,8 @@ confidence: high --- # Media Pipeline — Pitfalls & Debug Notes -Операционные заметки по Swift media-pipeline (media_files_db / media-toolbox) на Крякне. -Связано: [[personal-os-architecture]], [[wiki/tech/kraken-infra]] +Операционные заметки по Swift media-pipeline (media_files_db / media-toolbox) на Кракене. +Связано: [[personal-os-architecture]], [[tech/kraken-network]] ## §9.x Score Engine — Критические питфолы @@ -41,6 +41,7 @@ confidence: high - 40 KP-запросов vs 8428 TMDb в типичном прогоне - Лимит исчерпывается только после очистки кеша когда весь TMDb идёт заново - Проблема не в архитектуре — это нормально +- **KP API URL смена домена (Баг 8):** `api.kinopoisk.dev` → `api.poiskkino.dev`. Swift URLSession при редиректе теряет `X-API-KEY` → пустые ответы кешируются. Фикс: обновить URL, добавить `shouldCache: ((Data) -> Bool)?` — не кешировать пустой `docs`. ## Docker Deploy — Kraken (обязательный порядок) @@ -92,6 +93,29 @@ docker run \ - §9.6 subset floor 0.35 — добавлен - §9.9 year tiebreak `count>1` only — исправлен - Year-miss retry в TMDb/KP providers — добавлен +- KP API URL (`api.poiskkino.dev`) + пустой-docs no-cache — исправлен +- Leet-speak в транслите (`4` = `ч`): `String.decodeLeetSpeak()` — исправлен +- Год `2023г` (кириллическое `г`): `(19|20)\d\d)г\b` паттерн — исправлен +- `TitleParser _year_` underscore boundary: `\b` → `(?