Media folder fix: live read-back on demo-hp 9201 (hardlink, live catalog bc54c01); R-1014/1015/1016/1030 closed, R-1019 narrowed, R-1039/1040 filed; report

2026-10-14 22:01:09 +02:00
parent adc9dee0e3
commit 5a03fd4a57
10 changed files with 241 additions and 11 deletions
@@ -15,8 +15,8 @@ import json, os, secrets, sys, time
 sys.path.insert(0, "/mnt/5_hdd/felhom.eu/git/app-catalog-felhom.eu/scripts")
 import box_walk as w

-HDD = "/mnt/felhom-drives/media_a"
-SEC = os.path.expanduser("~/.felhom-tester1/secrets.json")
+HDD = os.environ.get("HDD", "/mnt/felhom-drives/media_a")
+SEC = os.path.join(os.environ.get("SC", os.path.expanduser("~/.felhom-tester1")), "secrets.json")


 def load():
@@ -0,0 +1,13 @@
+radarr: delete movie 1 with files -> 200
+qbittorrent: torrents deleted with their data
+left in movies: 
+left in torrents: .temp 
+
+radarr stop -> 200
+radarr remove WITHOUT delete data -> 200 {'ok': True, 'data': {'removed': 'radarr', 'volumes_removed': ['radarr_radarr_config'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'userdata_kept': ['/mnt/felhom-drives/hdd_1/userdata/media (
+qbittorrent stop -> 200
+qbittorrent remove WITHOUT delete data -> 200 {'ok': True, 'data': {'removed': 'qbittorrent', 'volumes_removed': ['qbittorrent_qbittorrent_config'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'userdata_kept': ['/mnt/felhom-drives/hdd_1/u
+0
+/opt/docker/stacks/qbittorrent
+/opt/docker/stacks/radarr
+audiobooks books comics grimmory metube movies music nzbget photos podcasts torrents transmission tv usenet 
@@ -0,0 +1,22 @@
+21:41:40 qbittorrent: POST deploy {"DOMAIN": "enkisfelhom.hu", "SUBDOMAIN": "torrent", "HDD_PATH": "/mnt/felhom-drives/hdd_1", "QBT_ADMIN_PASSWORD": "<generated, kept in secrets.json>"} -> 202 {"ok": true, "message": "The install has started – the card shows how it is going"}
+21:41:56 qbittorrent: deployed, state=running, pinned={'qbittorrent': 'qbittorrentofficial/qbittorrent-nox:5.2.4-1'}
+running /var/lib/docker/volumes/qbittorrent_qbittorrent_config/_data->/config:true /mnt/felhom-drives/hdd_1/userdata/media/torrents->/downloads:true /mnt/felhom-drives/hdd_1/userdata/media/torrents->/mnt/felhom-drives/hdd_1/userdata/media/torrents:true 
+2026/10/14 19:41:40 router.go:449: [INFO] [api] Deploy requested for stack: qbittorrent
+2026/10/14 19:41:40 deploy.go:526: [INFO] [stacks] Deploying stack qbittorrent with 4 env vars: [DOMAIN, SUBDOMAIN, HDD_PATH, QBT_ADMIN_PASSWORD]
+2026/10/14 19:41:40 manager.go:1773: [INFO] [stacks] Deploying stack qbittorrent — checking 1 images...
+2026/10/14 19:41:41 notifier.go:353: [INFO] Event pushed: app_deploy_started (info) [+household(en)] — Alkalmazás telepítése elindult: qBittorrent
+2026/10/14 19:41:46 deploy.go:616: [INFO] [stacks] Stack qbittorrent deployed successfully (took 5.3s)
+2026/10/14 19:41:46 notifier.go:353: [INFO] Event pushed: app_deployed (info) [+household(en)] — Alkalmazás telepítve: qBittorrent
+2026/10/14 19:41:51 after_install.go:184: [INFO] [stacks] after_install qbittorrent: qbittorrent [/bin/sh -c curl -s -c /tmp/fc -o /dev/null -X POST -d "username=admin&password=${QBT_ADMIN_PASSWORD}" -H 'Referer: http://127.0.0.1:8080' http://127.0.0.1:8080/api/v2/auth/login && curl -s -b /tmp/fc -o /dev/null -X POST --data-urlencode 'json={"upnp":false,"web_ui_domain_list":"${SUBDOMAIN}.${DOMAIN};qbittorrent;localhost;127.0.0.1"}' -H 'Referer: http://127.0.0.1:8080' http://127.0.0.1:8080/api/v2/app/setPreferences && curl -s -b /tmp/fc http://127.0.0.1:8080/api/v2/app/preferences | grep -q '"upnp":false' && curl -s -o /dev/null -w '%{http_code}' -H 'Host: qbittorrent:8080' http://127.0.0.1:8080/ | grep -qE '^(200|401|403)$' && echo FELHOM_AFTER_INSTALL_OK
+2026/10/14 19:41:51 install_hold.go:135: [INFO] [stacks] qbittorrent: install hold OPENED by after_install — the app is reached as without a hold
+
+21:42:00 radarr: POST deploy {"DOMAIN": "enkisfelhom.hu", "SUBDOMAIN": "radarr", "HDD_PATH": "/mnt/felhom-drives/hdd_1"} -> 202 {"ok": true, "message": "The install has started – the card shows how it is going"}
+21:42:30 radarr: deployed, state=running, pinned={'radarr': 'lscr.io/linuxserver/radarr:6.4.4'}
+running /var/lib/docker/volumes/radarr_radarr_config/_data->/config:true /mnt/felhom-drives/hdd_1/userdata/media->/mnt/felhom-drives/hdd_1/userdata/media:true 
+2026/10/14 19:41:59 router.go:449: [INFO] [api] Deploy requested for stack: radarr
+2026/10/14 19:42:00 deploy.go:526: [INFO] [stacks] Deploying stack radarr with 3 env vars: [DOMAIN, SUBDOMAIN, HDD_PATH]
+2026/10/14 19:42:00 manager.go:1773: [INFO] [stacks] Deploying stack radarr — checking 1 images...
+2026/10/14 19:42:00 notifier.go:353: [INFO] Event pushed: app_deploy_started (info) [+household(en)] — Alkalmazás telepítése elindult: Radarr
+2026/10/14 19:42:10 deploy.go:616: [INFO] [stacks] Stack radarr deployed successfully (took 10.0s)
+2026/10/14 19:42:10 notifier.go:353: [INFO] Event pushed: app_deployed (info) [+household(en)] — Alkalmazás telepítve: Radarr
+
@@ -0,0 +1,27 @@
+== Radarr: what the first steps say
+drwxrwsr-x 2 root users 4096 Jul 21 19:08 /mnt/felhom-drives/hdd_1/userdata/media/movies
+drwxrwsr-x 2 abc  users 4096 Oct 13 21:26 /mnt/felhom-drives/hdd_1/userdata/media/torrents
+audiobooks books comics grimmory metube movies music nzbget photos podcasts torrents transmission tv usenet
+  root folder /mnt/felhom-drives/hdd_1/userdata/media/movies -> 201
+  download client qBittorrent (its own connection test) -> 201
+  movie added: Night of the Living Dead (1968) path=/mnt/felhom-drives/hdd_1/userdata/media/movies/Night of the Living Dead (1968)
+  release push -> 200 approved=True rejections=[]
+  grabbed by qBittorrent
+  +65s queue: downloading downloading left=594823937 output=None
+  +125s queue: downloading downloading left=300174081 output=None
+  +185s queue: downloading downloading left=300174081 output=None
+  +245s queue: downloading downloading left=173673216 output=None
+  +305s queue: downloading downloading left=72043264 output=None
+  +365s queue: downloading downloading left=72043264 output=None
+  +425s queue: downloading downloading left=35585 output=None
+  +485s queue: warning downloading left=35585 output=None
+  +545s queue: warning downloading left=35585 output=None
+  +605s queue: warning downloading left=35585 output=None
+  +665s queue: warning downloading left=35585 output=None
+  +725s queue: warning downloading left=35585 output=None
+  +735s history downloadFolderImported: {"downloadClientName": "qBittorrent", "droppedPath": "/mnt/felhom-drives/hdd_1/userdata/media/torrents/night-of-the-living-dead-1968_202508/Night of the Living Dead 1968.mp4", "importedPath": "/mnt/felhom-drives/hdd_1/userdata/media/movies/Night of the Living Dead (1968)/night-of-the-living-dead-1968_202508.mp4"}
+  download: links=2 inode=9568271  library: links=2 inode=9568271
+  RESULT: HARDLINK — one file, two names
+# NOTE: qBittorrent stalled on the LAST piece (archive.org's web seed; the same fault as on Tester 1 that afternoon). At
+# 21:53 a throwaway Transmission on the bench (192.168.0.118, LXC 9401) fetched the same public-domain torrent and was
+# given to qBittorrent as a peer (addPeers); qBittorrent completed at 21:54:43 and Radarr imported at +735 s.
@@ -0,0 +1,98 @@
+#!/usr/bin/env python3
+"""live_readback.py — Part B6: the one-drive chain on a box with the LIVE templates, run INSIDE the guest.
+
+Does what the new first steps tell the household to do — Radarr: root folder `.../media/movies`; one qBittorrent
+entry (host qbittorrent, port 8080, admin, the install's password, category `movies`) — then hands ONE public-domain
+archive.org film to Radarr by its release push and waits for the import. Proof: download path, library path, link
+count, inode, Radarr's history entry. Keys and passwords are read at runtime and never printed.
+
+    run_in_guest-style:  python3 live_readback.py <drive mount, e.g. /mnt/felhom-drives/hdd_1>
+"""
+import datetime, json, os, re, subprocess, sys, time, urllib.error, urllib.request
+
+HDD = sys.argv[1]
+LIB = f"{HDD}/userdata/media/movies"
+ITEM = "night-of-the-living-dead-1968_202508"   # CC Public Domain Mark, 597 MB
+SECRETS = []
+
+
+def red(s):
+    s = str(s)
+    for x in SECRETS:
+        s = s.replace(x, "<redacted>") if x else s
+    return s
+
+
+def say(*a):
+    print(red(" ".join(str(x) for x in a)), flush=True)
+
+
+def sh(c):
+    return subprocess.run(c, shell=True, capture_output=True, text=True).stdout.strip()
+
+
+def ip(c):
+    return sh(f"docker inspect {c} -f '{{{{range .NetworkSettings.Networks}}}}{{{{.IPAddress}}}}{{{{end}}}}'")
+
+
+key = re.search(r"<ApiKey>([^<]+)", open("/var/lib/docker/volumes/radarr_radarr_config/_data/config.xml").read()).group(1)
+pw = sh("docker exec qbittorrent printenv QBT_ADMIN_PASSWORD")
+SECRETS += [key, pw]
+RA = f"http://{ip('radarr')}:7878/api/v3"
+H = {"X-Api-Key": key, "Host": "radarr", "Content-Type": "application/json"}
+
+
+def ra(m, p, body=None):
+    r = urllib.request.Request(RA + p, data=json.dumps(body).encode() if body is not None else None, headers=H, method=m)
+    try:
+        with urllib.request.urlopen(r, timeout=120) as x:
+            t = x.read(); return x.status, (json.loads(t) if t else None)
+    except urllib.error.HTTPError as e:
+        b = e.read().decode()[:400]; say(f"  !! {m} {p} -> {e.code} {b}"); return e.code, None
+
+
+say("== Radarr: what the first steps say")
+print(sh(f"docker exec radarr sh -c 'ls -ld {LIB} {HDD}/userdata/media/torrents; ls {HDD}/userdata/media | tr \"\\n\" \" \"'"))
+if not any(r["path"].rstrip("/") == LIB for r in ra("GET", "/rootfolder")[1]):
+    c, r = ra("POST", "/rootfolder", {"path": LIB}); say(f"  root folder {LIB} -> {c}")
+sch = {s["implementation"]: s for s in ra("GET", "/downloadclient/schema")[1]}
+if not ra("GET", "/downloadclient")[1]:
+    b = json.loads(json.dumps(sch["QBittorrent"]))
+    vals = {"host": "qbittorrent", "port": 8080, "username": "admin", "password": pw, "movieCategory": "movies"}
+    for f in b["fields"]:
+        if f["name"] in vals:
+            f["value"] = vals[f["name"]]
+    b.update({"name": "qBittorrent", "enable": True})
+    c, r = ra("POST", "/downloadclient", b); say(f"  download client qBittorrent (its own connection test) -> {c}")
+c, lk = ra("GET", "/movie/lookup/tmdb?tmdbId=10331")
+qp = ra("GET", "/qualityprofile")[1][0]
+c, m = ra("POST", "/movie", dict(lk, qualityProfileId=qp["id"], rootFolderPath=LIB, monitored=True, tags=[],
+                                 minimumAvailability="released", addOptions={"searchForMovie": False}))
+say(f"  movie added: {m['title']} ({m['year']}) path={m['path']}")
+rel = {"title": "Night.of.the.Living.Dead.1968.480p.WEB-DL.x264-ARCHIVEORG", "protocol": "torrent", "size": 596645320,
+       "downloadUrl": f"https://archive.org/download/{ITEM}/{ITEM}_archive.torrent", "indexer": "archive.org (release push)",
+       "publishDate": datetime.datetime.now(datetime.timezone.utc).isoformat()}
+t0 = time.time()
+c, d = ra("POST", "/release/push", rel)
+dd = d[0] if isinstance(d, list) else (d or {})
+say(f"  release push -> {c} approved={dd.get('approved')} rejections={dd.get('rejections')}")
+imp = None
+for i in range(150):
+    time.sleep(10)
+    for e in ra("GET", f"/history/movie?movieId={m['id']}")[1] or []:
+        if e["eventType"] == "grabbed" and i == 0:
+            say(f"  grabbed by {e['data'].get('downloadClientName')}")
+        if e["eventType"] == "downloadFolderImported":
+            imp = e
+    if imp:
+        break
+    if i % 6 == 5:
+        for r in (ra("GET", f"/queue?movieIds={m['id']}")[1] or {}).get("records", []):
+            say(f"  +{int(time.time()-t0)}s queue: {r.get('status')} {r.get('trackedDownloadState')} left={r.get('sizeleft')} output={r.get('outputPath')}")
+if not imp:
+    say(f"  RESULT: not imported within {int(time.time()-t0)}s"); sys.exit(1)
+e = imp["data"]
+say(f"  +{int(time.time()-t0)}s history downloadFolderImported: {json.dumps({k: e.get(k) for k in ('downloadClientName', 'droppedPath', 'importedPath')})}")
+a, b = os.stat(e["droppedPath"]), os.stat(e["importedPath"])
+say(f"  download: links={a.st_nlink} inode={a.st_ino}  library: links={b.st_nlink} inode={b.st_ino}")
+say("  RESULT:", "HARDLINK — one file, two names" if (a.st_ino, a.st_dev) == (b.st_ino, b.st_dev) else "COPY")
@@ -26,6 +26,15 @@

 ---

+## 2026-10-14 — the media folders meet in the live catalog
+
+| Row | What | Closed | Evidence |
+|---|---|---|---|
+| **R-1014** | **The media chain could not work: the download folders did not meet, and the hardlink comment was false** (P2) | CLOSED 2026-10-14 — FIXED in the live catalog (decision 242, D9 = A with a limit): Sonarr/Radarr bind `media` at the same path, the clients only their own folder at the same path; the false comment replaced by the measured reason. Live read-back on demo-hp 9201: one public-domain film qBittorrent → Radarr imported as a HARDLINK (links 2, one inode). Installed clients keep their old torrents (bench). | app-catalog `bc54c01`; `audits/media-folder-fix-2026-10-14/``C-live-readback-9201.txt`, `B2-B3-*.txt` |
+| **R-1015** | **Three apps told the household to use `/media/shows`** (P3) | CLOSED 2026-10-14 — FIXED: Sonarr, Plex and Jellyfin first steps (hu + en) name `media/tv`; the copy freeze re-captured for those lines only. | app-catalog `bc54c01` |
+| **R-1016** | **qBittorrent's onboarding record named the wrong download folder** (P4) | CLOSED 2026-10-14 — FIXED: `onboarding/qbittorrent.md` and `transmission.md` row 2.2 name the real folders, with the bench evidence. | app-catalog `bc54c01` |
+| **R-1030** | **Sonarr's and Radarr's first admin must default to „Enabled”** (P2) | CLOSED 2026-10-14 — FIXED for new installs: the first start seeds `AuthenticationRequired=Enabled` into `config.xml`; the first-admin form then reads „enabled”; an existing `config.xml` is never touched (bench). The window BEFORE the household makes its admin stays open to other containers — filed as R-1040. | app-catalog `bc54c01`; `audits/media-folder-fix-2026-10-14/``B2-sonarr-radarr.txt` |
+
 ## 2026-10-14 — the Immich repair on every box; the shared checkout cleaned

 | Row | What | Closed | Evidence |