TrueNAS NFS4 ACL repair + cross-container media PUID

Hard-won lessons from the 2026-08-31/09-01/09-02 TrueNAS restore-incident remediation. Mirror of skill truenas-nfs4-acl-and-arrmultiuser. Detailed incident log: family/documents/vault-sync/2026-09-02-restore-privilege-scope.md.

Trigger

Any "permission denied" / file not readable / git loose object ... corrupt / empty Jellyfin library on TrueNAS SCALE (acltype=nfsv4) after a zfs restore, OR wiring Prowlarr/Radarr/Sonarr/Jellyfin/Transmission so container-written files are readable by other containers.

Core facts

TrueNAS SCALE uses NFSv4 ACL, not plain POSIX. ls -la shows a POSIX mask that can mislead.
Read real ACL: midclt call filesystem.getacl <path>; write: midclt call filesystem.setacl.
truenas_admin is FULL_ADMIN in midclt → can chown/setacl without passwordless root.
Web UI ACL editing only works on zfs datasets, not arbitrary subdirs under one dataset → use midclt/CLI.
Post-restore a tree may be owned by wrong uid (e.g. 921 transmission) with files mode 0000.

The two big gotchas

1. setacl takes ONE JSON with `path` INSIDE it

midclt call filesystem.setacl /path {…} → [EFAULT] Too many arguments (expected 1, found 2). Correct form:

midclt call filesystem.setacl '{"path":"/x","uid":950,"gid":950,"acltype":"NFS4","dacl":[...],"options":{...}}'

Jobs run async → returns a job id; verify midclt call core.get_jobs (state SUCCESS).

2. stripacl does NOT remove DENY; full dacl replacement does

options.stripacl:true reported SUCCESS but left the owner@ DENY ACE intact. Only passing a complete dacl array works — setacl treats it as a full REPLACE of the ACL.

NFSv4 DENY overrides ALLOW (the corrupt-object trap)

A file owned by uid 950 carrying owner@ DENY READ_DATA=True cannot be read by uid 950 itself. Git then reports loose object ... corrupt — this is ACL, not data damage (data intact; root reads fine). Diagnostic trap: git fsck under root is clean, but fetch/push under the owning uid fails → ACL, not corruption. Broken files often show POSIX mode 40 (r--------); healthy objects 750.

Repair / cross-container recipe (linuxserver media stack)

Goal: all containers run under ONE uid (950) so transmission→radarr/sonarr→jellyfin all read each other's files.

transmission: add PUID=950 PGID=950 to env in docker-compose.yml. It mounts storage→/mnt/storage, download-dir /mnt/storage/Downloads.
radarr/sonarr/prowlarr/jellyfin compose: add PUID=950 PGID=950 to each (retain TZ).
linuxserver s6 images (Entrypoint /init, /etc/s6-overlay) honor PUID/PGID natively. Default (unset) = runs root → creates root-owned files that 950-services can't read → always set PUID/PGID.
Recreate from proper folder: cd /mnt/RED_2TB/docker/<svc> && docker compose up -d (+--force-recreate if env changed). Verify daemon uid via ps aux inside container (process as abc/950 = applied). NOTE docker exec id ≠ daemon uid.
Simple datasets: recursive POSIX chown -R 950:950 + chmod under root SUFFICE (ACLs were trivial). Use midclt/ACL only when non-trivial ACEs present (check filesystem.getacl → trivial:true).
Bare git repo with owner@ DENY on objects → full dacl replacement recursively (perms JSON like above).

GOTCHA 3 — root dataset traverse (Permission denied despite clean file ACL)

After chown-ing all leaf dirs/files to 950, if a service still gets Permission denied reading /storage/Movies/... (or jellyfin FFmpeg exit 243), check the root of the dataset /mnt/RED_2TB/storage itself. In the restore it stayed 921:921 with owner@/group@ ALLOW but everyone@ EXECUTE=False. uid 950 (not owner, not in gid-921-group) falls under everyone@ → cannot traverse past the root into the tree, even though every nested ACL is clean. Fix (root):

chown 950:950 /mnt/RED_2TB/storage
chmod 750 /mnt/RED_2TB/storage    # rwxr-x---: gid 950 = truenas_admin (containers' group) gets r-x traverse

Kernel needs EXEC (traverse) on EVERY path component. Diagnostic via setpriv --reuid=950 ... ls shows Permission denied on a dir whose getacl looks clean → suspect root/parent traverse. stat via root shows clean 770 but uid 950 can't ls.

GOTCHA 4 — transmission "all torrents No Data Found" after data migration

If every transmission torrent shows error 3: No Data Found right after a container recreate/pool migration:

Check: docker exec -u fine to read? downloadDir & files present? If files ARE there and readable as uid 950, the error is likely stale from daemon startup while traverse was blocked (see GOTCHA 3).
torrent-verify on a single torrent does NOT clear it. Fix is trivial & non-destructive:

docker restart transmission

On restart the daemon re-validates → torrents clear (e.g. 254/255 instantly). Leftover #1 → Verify Local Data in Web UI.

Docker network persistency (per-service compose)

docker network connect <net> <container> is LOST on recreate → declare <net> in the service's compose networks:.
Caddy must co-own a network with its reverse-proxy target. transmission Web UI works because caddy is ALSO in transmission_default; radarr↔transmission works because both in media_net.
external: true networks (media_net/caddy_default) aren't created by compose — if they vanish (pool rebuild), docker network create media_net.

Containers / uid summary (as of 2026-09-02, TrueNAS "Taiga")

prowlarr/radarr/sonarr/jellyfin: all running, all daemons under uid 950, project arr.
transmission: PUID/PGID=950, in networks media_net + transmission_default (both external).
Radarr/sonarr/jellyfin mount /mnt/RED_2TB/storage as /storage (NOT /media as on Kraken) → set radarr root-folder accordingly.

6.1 KiB Raw Blame History