Media all drives, S5/S6 on Tester 1: a pulled drive stops nothing, the guest keeps a dead mount; on return nothing sees it until stop+start, FileBrowser cannot be restarted (R-1038); 15 §5 written; final report

documentation/backlog/OPEN-ITEMS.md

@@ -193,14 +193,15 @@ stopping line that lies.
 | **R-878** | Backup & restore | P4 | **A catch-up (R-871) runs the database-dump leg in the DAY, and that leg stops an app with a volume for its copy — the household may notice the stop, and a large volume makes it longer.** MEASURED 2026-10-05 on demo-felhom: the catch-up at 08:25:02 stopped opengist, copied 182.5 KB, started it again — about 1 s, then a few seconds of `health: starting`; the night does exactly the same, unseen. Nothing measured for a large volume. Fix direction (if it matters): skip the volume copy of a running app in a DAYTIME catch-up and leave it to the next night, or warn. `audits/catchup-2026-10-05/partA/live-demo-felhom.txt` | **READY — owner: CC** **2026-10-05 (burn-down night): NEEDS A LIVE MEASUREMENT** — a timed catch-up on 9202 for an app with a multi-GB volume, then the skip-or-warn choice. | — | measure a large volume first | CC |
 | **R-32** | Backup & restore | P4 | **[P2-HIGH] RESET must purge the customer base dir; the orphan card must stay honest; unattributed bytes must be visible.** The rehearsal's S7 said in advance that an orphan card would BE a finding — and one appeared (16:58:14). Cause: RESET's `"hetzner":"ok"` leg destroys the sub-account, but **a Hetzner sub-account is an access-control object, not a data object** — its directory survives, so re-enabling offsite recreated an account over the previous lifecycle's ciphertext, encrypted under a key that same RESET had destroyed. **MIGRATED FROM `ROADMAP.md` 2026-08-22 (R-369) — originally filed 2026-07-21, size M, roadmap state `idea`.** Moved verbatim; nothing added or reinterpreted. The roadmap keeps its copy as history, marked moved. | **OPEN — migrated from ROADMAP 2026-08-22, rank unchanged** **2026-10-06 night: design written** (`audits/night-burndown-2026-10-06/design-R-32.md`) — pick A: purge the repo + set-aside copies through the sub-account's own password login BEFORE deleting it (no main-account credential); the „repo data destroyed" comment at `hub/internal/offsite/offsite.go:276-280` is false for the shared tier; part (3) waits on a read-only measurement (`du` on port 23). Operator: agree to route A. **2026-10-07 07:58: `09` §3 decision 167 — option A yes, and delete the old RESET leftovers once.** **2026-10-07 (Part D step 2, read only):** the hub shows bx11 holding 3.3 GB of data while the four live customers account for ~0.6 GB → **~2.7 GB belongs to no live customer**; no live customer shows a set-aside folder, so the leftovers are in homes whose sub-account is gone, reachable only by the pool box's MAIN account. **The one-time clean-up is STOPPED:** no main-account login for bx11 is held on DooPlex (the API token in the credentials file sees no box). Needs: the main-account login, or the operator lists and deletes in the Hetzner console (`audits/day-2026-10-07/D/D2-listing-readonly.txt`). **2026-10-07: option A DELIVERED (hub 0.142.0):** RESET purges the repo and every `.orphaned-*` through the sub-account's own login before deleting it, and keeps the sub-account if the purge fails (red-proved, `audits/day-2026-10-07/D/`). Not yet seen live (no scratch RESET today). **The one-time clean-up of the ~2.7 GB of leftovers is STOPPED:** it needs the pool box's main-account login (see above). **2026-10-07 14:43 — `09` §3 decision 173: leave the ~2.7 GB of old leftovers for now; re-ranked P2 → P4.** Option A itself is delivered (hub 0.142.0). What stays open is ONLY the leftovers: they were computed (box total minus live customers), never listed, and deleting them needs the pool box's MAIN login, which reaches every customer's data. **Blocker: a safe main-account path.** | — | **Ruling from the run (three parts, deliberately separate):** (1) because RESET destroys custody, the ciphertext it leaves behind is unrecoverable **BY DESIGN** → RESET gains a **main-account purge of the customer base dir** (the existing operator ack already covers it); (2) the **move-aside guard STAYS** for reinstall-*without*-RESET — there custody survives and the card's "history recoverable" promise is true (R-26 depends on exactly that); (3) the operator **Restic tab shows per-customer directory bytes vs attributed snapshot bytes**, so dead data cannot hide. Measured on the pool box that night: **49 M attributed** (2 snapshots, 48.717 MiB) against **1.4 G + 3.0 M unattributed** across TWO `.orphaned-*` dirs. Evidence `restic-and-pool.txt` | CC |
 
-## Storage & devices — 7 rows (P2 4, P3 2, P4 1)
+## Storage & devices — 8 rows (P2 5, P3 2, P4 1)
 
 | ID | Category | Sev | What | State | Blocked on | Next action | Owner |
 |---|---|---|---|---|---|---|---|
 | **R-1027** | Storage & devices | P2 | **„Delete my data too” deletes every bind under the app's drive — read-only and shared ones included.** MEASURED 2026-10-14 on Tester 1: removing a throwaway Plex (drill template: `/mnt/felhom-drives/media_a/userdata/media` bound `:ro`, `HDD_PATH=/mnt/felhom-drives/media_a`) with `remove_hdd_data:true` logged „Removed HDD data: /mnt/felhom-drives/media_a/userdata/media (80K)” — the WHOLE media folder of the drive, which four other apps were using. `ParseComposeHDDMounts` counts a literal path under `HDD_PATH` as the app's own data, and `ProtectedHDDPaths` does not protect `userdata/media` (R-1032). Only an empty test skeleton was lost; it was rebuilt by the controller's next file-browser sync and the apps restarted (`audits/media-all-drives-2026-10-14/``D-plex-remove-deleted.txt`). Today's LIVE templates bind `${USERDATA_PATH}` and are not hit (R-800); **the media-library build (R-1023) would be, unless removal learns that a shared library is never one app's data.** The note „literal binds are never removed” in this session's first draft was WRONG and is retracted. | **READY** — filed 2026-10-14 | — | Removal: never delete a `:ro` bind nor anything under `userdata/`; a test that removes a reader app and asserts the shared folder survives (red-proof first); `15` §8 | CC |
-| **R-1023** | Storage & devices | P2 | **Build: media apps see every drive, and the box says when they must restart (operator rulings 2026-10-14, `09` §3).** Scope from `architecture/15-media-library.md`: one bind per drive of `userdata/media/<library folders>` at the same path; the drive-absent gate per bind (today it keys on the app's one `HDD_PATH`, and the boot gate waits only for THAT drive — measured after the 17:25 power cut); the restart notice after a drive is added, removed or comes back, with „restart now” / „later” and a per-drive „apps restarted since” mark; removal, export and backup classes taught that a shared library is no app's own (R-1027, R-1019). | **OPEN** — filed 2026-10-14; design in `15` (S5/S6 results pending the operator's test) | the operator's test on Tester 1 („done”), then S5/S6 | Finish `15` with S5/S6; then a build task | CC |
+| **R-1023** | Storage & devices | P2 | **Build: media apps see every drive, and the box says when they must restart (operator rulings 2026-10-14, `09` §3).** Scope from `architecture/15-media-library.md`: one bind per drive of `userdata/media/<library folders>` at the same path; the drive-absent gate per bind (today it keys on the app's one `HDD_PATH`, and the boot gate waits only for THAT drive — measured after the 17:25 power cut); the restart notice after a drive is added, removed or comes back, with „restart now” / „later” and a per-drive „apps restarted since” mark; removal, export and backup classes taught that a shared library is no app's own (R-1027, R-1019). **S5/S6 measured 2026-10-14:** a pulled drive stops none of the media apps (they key on another drive), they keep a dead view; on return nothing restarts them — so the restart notice MUST also cover „a drive came back”, the restart must be stop+start, and Sonarr/Radarr's own health page says nothing about an unreachable root folder, so the box must say it. | **OPEN** — filed 2026-10-14; design in `15` (S5/S6 results pending the operator's test) | the operator's test on Tester 1 („done”), then S5/S6 | Finish `15` with S5/S6; then a build task | CC |
 | **R-1031** | Storage & devices | P2 | **The build must not ask a media app for one drive.** Today every media template carries a required `HDD_PATH` field („Médiatár útvonal”, `locked_after_deploy`), so the wizard makes the household pick ONE drive — the opposite of `09` §3 decision 237 — and the drive gate, removal and backup key on that one drive (`15` §4, §8). The test rig kept the field only because the deploy needs it. | **READY** — filed 2026-10-14 | R-1023 | In the build: a media app (`15` §3 marker) has no drive field; its own config stays on the system SSD; its library binds come from the enrolled drives | CC |
 | **R-1032** | Storage & devices | P2 | **The delete safety list still names the old folder layout.** `ProtectedHDDPaths` (`felhom-controller/controller/internal/stacks/delete.go:240`) protects `<drive>/media` and `<drive>/Dokumentumok` — the layout before `userdata/` — and NOT `<drive>/userdata`, `userdata/media` or `userdata/documents`. That is why removing a Plex deleted `userdata/media` (R-1027). It touches every app with a drive bind, not only media apps. | **READY** — filed 2026-10-14 | — | Protect the current skeleton roots (`userdata` and its top folders) and keep the old names; a test that fails when a skeleton root is missing from the list (red-proof first). Fix with R-1027 | CC |
+| **R-1038** | Storage & devices | P2 | **After a drive comes back, the file browser (and every app that binds it) still shows the DEAD drive: „I/O error”.** MEASURED 2026-10-14 on Tester 1 (`audits/media-all-drives-2026-10-14/S5-S6/`): Media B's disk was pulled at 20:10:23 and re-attached at 20:41:00. The guest kept a dead ext4 mount of `/mnt/felhom-drives/media_b` the whole time (`mountpoint` = yes, `ls` = Input/output error); on return the agent bound the new disk ON TOP of it (two stacked mounts), the controller logged „drive RETURNED … restarted gate-stopped apps” and „FileBrowser sync — no config/compose change, ensured running without recreate” — so FileBrowser kept the dead mount: `ls /srv/media_b/media/tv` → I/O error, 4 minutes after the drive was back. The same was true for all six media apps until they were stopped and started. During the pull the box also tried to rebuild the skeleton ON the dead mount („chmod … input/output error”), because the drive-absent gate asks „is it a mountpoint?” and a dead mount answers yes. Today's product: any household whose USB drive drops and returns hits this in FileBrowser. **And the household cannot fix it:** FileBrowser is a protected stack — the dashboard answers `POST /api/stacks/filebrowser/stop` and `/start` with 403 „cannot stop protected stack filebrowser” (measured 20:46) — so only a restart of the whole box clears it. | **READY** — filed 2026-10-14 | — | On drive return, recreate FileBrowser (and every container that binds the drive) — stop+start, not `up -d` (R-1026); on drive loss, make the guest's mount go away (or treat an I/O-error mount as absent) so the gate and the skeleton see the truth; a test with a dead mount | CC |
 | **R-330** | Storage & devices | P3 | **Disk health Phase 2 — the three SMART attributes the wire does not carry.** The failing drive's most telling counter was **187 `Reported_Uncorrect`**, sitting at normalized **1** against threshold **0** with a raw count of **1001** — one point from failing and structurally unable to get there. Also wanted: **199 `UDMA_CRC_Error_Count`** (cabling) and **188 `Command_Timeout`**. None are on the agent→controller wire today, so v0.215.0's ladder could not use them. Phase 2 also persists periodic SMART **samples** (the right home is `metrics.MetricsStore`, NOT the Phase-1 state file, which is one record per disk and must stay that way). **This is a declared WIRE change, so under the G-1 gate the hub must model the new fields in the SAME session** — that is precisely why it was kept out of Phase 1, where it would have turned a one-word severity fix into a three-repo change | **READY** — **2026-10-12: the wire half is CLOSED in this row** — released in agent v0.150.0 (boxes run 0.154.0), controller and hub decode 187/188/199. **Only open work:** the verdict slice (use the three counters in the disk verdict and the household's words) and persisting samples. **READY (M) — NEW 2026-08-14** **2026-10-06 night: the wire half BUILT on main (unreleased):** the agent sends SMART 187 `reported_uncorrect`, 188 `command_timeout` (raw, vendor-packed on some drives — carried as reported), 199 `udma_crc_errors` (pointer + omitempty: unknown is absent, never 0); the controller decodes them and the hub's host-report mirror models them (G-1 wire gate green; it convicted the agent half alone). **Carried only** — no verdict, banner, mail or alarm reads them (`TestR330_CountersChangeNoVerdictYet`); using them changes what a household is told and is the next slice, with persisting samples. Red-proofs `audits/night-burndown-2026-10-06/r330/`. Ships with agent v0.150.0 and the next controller and hub releases. | R-328 (closed) | The verdict slice: 187/188/199 into the disk verdict with tests; then persist samples | CC |
 | **R-332** | Storage & devices | P3 | **The new Hiba-from-counters path has never fired on real hardware.** v0.215.0's whole point is a verdict the product could not previously reach, and it is proven only against the committed fixture's values in unit tests (12 scenario groups, 11 of 12 red-proofs failing as required). The live validation on demo-hp proved the **negative** — three healthy disks still read Rendben across the deploy, no false alert — and the **severity wire** end to end, but no live disk has actually reached Hiba. **This is the honest gap and it must not be closed by pointing at the fixture tests**: the drive that produced the fixture is in DooPlex, which is Tier 2 and never a drill target, and the demo boxes are all-flash and healthy | **WATCHING — NEW 2026-08-14, NARROWED same day.** One item originally in this gap is now PROVEN LIVE: the **persisted state surviving a controller restart**. The v0.215.0→v0.216.0 redeploy destroyed and rebuilt the container, and the new one read back a `changed_at` written by the PREVIOUS version (`2026-08-14T07:23:14.640216851Z`, still intact at 09:31:35Z) instead of re-baselining — Scenario L on real hardware, not just the production-path unit test. **What remains unproven is the verdict itself, plus the stronger restart half: an already-ALERTED disk not re-alerting** | a real degrading disk, or an injection harness | **Closing condition:** a live disk reaching Hiba from counters, OR a deliberate injection through the REAL pipeline (agent `/disks` → controller check → hub event), not a hand-set verdict | CC |
 | **R-331** | Storage & devices | P4 | **Disk health Phase 3 — growth-rate detection, and retiring the static 64.** The v0.215.0 count backstop (64 unreadable sectors → Hiba) is **a judgement from ONE drive**: the observed benign excursion peaked at 16 and cleared inside an hour, and the terminal run passed 64 at 13 Aug 11:28 and never came back. It is deliberately a backstop BEHIND the sustain rule, not the primary signal, but it is still a magic number tuned on a single sample and it will be wrong for some drive. With Phase 2's history the box can ask the question that actually matters — *is this count climbing, and how fast* — which distinguishes a drive with eight stable aging sectors from one adding forty a day, something no static threshold can do. Revisit 64 when that exists | **READY (M) — NEW 2026-08-14** | R-330 | Growth-rate rule over persisted samples; re-derive or delete the static 64 | CC |