fixes 2026-10-21: register (7 closed, R-1205/R-1206 opened, R-1183 VERIFY, R-1192 diagnosed), STATUS (not ready: one step — the agent swap fix), A12/A13 proven live, CONTEXT, report, evidence

documentation/backlog/CLOSED-ITEMS.md

@@ -26,6 +26,18 @@
 
 ---
 
+## 2026-10-21 (day brief, before Tester 2) — the rebuilt box gets its backups back, the new-link page tells the truth (full text of each row: `git show 7ad17be6:documentation/backlog/OPEN-ITEMS.md`)
+
+| Row | What | Closed | Evidence |
+|---|---|---|---|
+| **R-1176** | **A box rebuilt after „delete the old host first" could not reach the household's off-site history: no code was asked for, a new key was minted, the history set aside on night one.** (P1) | CLOSED 2026-10-21 — **hub v0.163.0** (operator yes, deployed 11:46 UTC; `03` §8a „the CARRY", `07` §7.0): the hub moves the household's sealed package to the rebuilt host (`store.ReattachRetainedEscrow`; one live host, no package held, the newest escrow-acknowledged deletion; never opened or logged); controller 0.337.0 records the escrow before the floor (belt). **PROVEN LIVE on demo-hp** (Tester4, public ISO 1.29.0, no operator press before the install): carried at the agent's first report (12:04:21 UTC); the box logged „NOT minting a repository password"; the dashboard offered „A korábbi, házon kívüli mentéseid visszaszerezhetők"; a wrong code was refused („Semmi nem változott"), the right one unlocked 9 groups; Nextcloud reinstalled and „Teljes visszaállítás" → **3 031 of 3 033 manifest files byte-identical** (the 2 never reached the old box: the refused 1.5 GB file and a symlink); off-site runs appended to the same store, **63 → 64**, nothing set aside. Tests `TestR1176_*` (store 7, api 3), each wiring red-proved. | `audits/fixes-2026-10-21/P1-live/` |
+| **R-1203** | **A household whose old box was still linked was promised a new-link mail that never came.** (P2) | CLOSED 2026-10-21 — **hub v0.163.0**: the holder of the household's REAL used or expired link reads „A régi dobozod még össze van kötve a fiókoddal. Ezért most nem küldünk új linket. Írj a Felhom ügyfélszolgálatának: …" (EN twin), no link is mailed, and the operator gets `bind_old_host_linked` with the host name (once per hour per customer); an unknown token still gets the generic page. **PROVEN LIVE** 2026-10-21 12:45 UTC on Tester4's spent link (host Tester4-f6b67c linked). `TestR1203_AHouseholdWhoseOldBoxIsStillLinkedIsToldSo` (red-proved). | `audits/fixes-2026-10-21/P1-live/16-r1203-oldbox-page.txt` |
+| **R-1175** | **Shares on two drives that use one second drive lost each other's copies every night; a dead drive's only copy was deleted.** (P1) | CLOSED 2026-10-21 — **controller 0.337.0** (global floor 15:54 UTC, D25): each share group reconciles only its own drive key; a copy whose source is gone is kept and listed („Megőrzött másolatok", „Eltávolítás" by record + device/inode). `TestR1175_*` red-proved (the old reconcile line: both copies tests fail). | controller `internal/backup/tier2_shares.go`, `tier2_kept.go`; `07` §8.2a |
+| **R-1179** | **A drive marked „away" could be chosen as the second-drive target.** (P2) | CLOSED 2026-10-21 — controller 0.337.0: an away drive is skipped like a decommissioned one. `TestR1179_AnAwayDriveIsNotChosenAsTarget` red-proved (the copy landed on the away drive). | controller `internal/backup/tier2.go` |
+| **R-1181** | **A missing or emptied app folder deleted its second-drive copy within one night.** (P2) | CLOSED 2026-10-21 — the operator's ruling **D25** answered the row's question (keep until the household releases it); controller 0.337.0: a missing declared folder's copy is kept and listed; an emptied folder is not mirrored over a full copy (the run warns). `TestR1181_*` red-proved (the emptied case with the real `rsync -a --delete`). | controller `internal/backup/tier2_capture.go`, `tier2_kept.go`; `07` §8.2a |
+| **R-1182** | **Abandoning ONE set-aside off-site copy deleted EVERY retained key package of the customer.** (P2) | CLOSED 2026-10-21 — hub v0.163.0 (deployed): retained rows carry `repo_tag`, tagged with the set-aside path at the hub's move-aside; the purge removes only packages of a path the hub DELETED (legacy rows are kept, never purged by guess); controller 0.337.0 closes the box's declaration on `abandon_purge_done`. `TestR1182_*` (hub, red-proved: `purge removed 3, want exactly 1`; controller, red-proved). | hub `internal/store/store.go` (`PurgeAbandonedEscrowForCustomer`, `TagRetainedEscrowSetAside`) |
+| **R-1191** | **Importing a `.fab` bundle made a family-only app open to anyone.** (P1) | CLOSED 2026-10-21 — controller 0.337.0: the import builds the record like the unit restore and writes the family gate and the sign-up lock before the start (`stacks.SaveImportedAppConfig` / `PrepareImportedApp`). **Proven live on 9202 (rc.1):** export → remove → import → the record carries `family_gate`, the gate file exists, a stranger gets **401** (was 200). `TestR1191_*` (Glance, MeTube, Grimmory) red-proved. | `audits/fixes-2026-10-21/P3-fab/01-run.log` |
+
 ## 2026-10-21 — the system poster refreshed (full text of the row: `git show 3decdf94:documentation/backlog/OPEN-ITEMS.md`)
 
 | Row | What | Closed | Evidence |

documentation/backlog/OPEN-ITEMS.md

@@ -122,7 +122,6 @@ stopping line that lies.
 | **R-991** | Install & onboarding | P3 | **Nothing a buyer can read says which machines Felhom installs on, and a quarter of the forum beginners own only machines it cannot install on.** FOUND 2026-10-12 (forum research, `audits/forum-research-2026-10-12/FELHOM.md` §2). Searched `documentation/architecture/`, `website/` (`gyik`, `technologiak`, `letoltes`) and `scripts/felhom-host-install.sh`: no CPU-architecture, RAM, disk or network floor is written anywhere. The download page says only that the installer is built on Proxmox VE 9.2 — x86-64 only (inferred from Proxmox, not stated on the page). Of the 33 forum beginners who name what they own, 8 own only a Synology/QNAP/other NAS box, a router with a USB disk or a Raspberry Pi. The selfh.st survey reading reached the same gap independently (R-989, its recommendation 1). **Needs (operator):** the floor itself (RAM and disk minimum; whether thin clients and old laptops with 4–8 GB are accepted) — then one short section on the download page and in the FAQ. Not changed here: the research task forbade website edits. | **WAITING-ON-OPERATOR** — **2026-10-20: R-993 merged here** (one decision, D5): the proposal to approve is `audits/sizing-2026-10-13/SIZING.md` §5 — minimum x86-64, 4 cores (N100 class), 8 GB, 256 GB SSD; recommended 16 GB, 512 GB SSD; out: ARM / Raspberry Pi, Synology-QNAP, routers, cloud VPS. R-993 also measured that the installer has no floor on the HOST (`autosize_guest_caps` limits only the guest), so whatever minimum is approved, nothing enforces it at install today (full text: `git show 5c487313:documentation/backlog/OPEN-ITEMS.md`). filed 2026-10-12 | the operator naming the floor | Operator: approve or change `SIZING.md` §5 (STATUS D5); then CC writes the section on `letoltes.html` and `gyik.html` (both languages) and files the install-time host floor as its own row if it is wanted. If nothing is done: a buyer with a Pi or a Synology downloads an ISO that cannot run | operator |
 | **R-494** | Install & onboarding | P4 | **NARROWED 2026-09-14 by operator ruling → [P3-LOW] the hub COULD create the tunnel at customer creation, for a domain already on Cloudflare. Not blocking: every customer has their own domain and the operator creates the tunnel per day-0 A.1 (`architecture/01-topology-and-trust.md`).** *Original finding, kept:* **[P1-HIGH] A new customer's dashboard has NO reachable address unless the operator hand-makes a Cloudflare tunnel — the link in the setup-code mail is dead.** MEASURED 2026-09-14 on a fresh install from the public ISO (drill intervention **I1**): the claim mail points at `https://felhom.drill0242.felhom.eu`; that name has **no A and no AAAA** record (`dig @1.1.1.1`, control `felhom.enkisfelhom.hu` resolves); the hub has **no tunnel- or DNS-creation code** (`hub/internal/cloudflare/` holds only geo-rule removal; `cf_tunnel_token` is a pasted, optional form field, `configs.go:1478`) — day-0 runbook A.1 makes it a manual Cloudflare-dashboard step that nothing on the customer-create page asks for; the box's own split-horizon resolver on the appliance LAN IP answered `google.com` but not the dashboard name at 13:27:39Z; the agent applied the record at **13:27:44Z** (`lanresolver: applied split-horizon record … ip=192.168.0.158`, 3 m 46 s after the controller started), so the box CAN answer the name — **but only to a device that uses the box as its DNS server, and no document, screen or mail tells a household to do that**; the router and the installer-offered DNS answer nothing. The page was reachable only at the guest's LAN address with the name forced (`curl --resolve …:443:192.168.0.158`). **A volunteer could not have done that.** **What it needs:** an operator ruling — the hub creates the tunnel and DNS at customer creation, or the product gives a household a LAN address that works with no DNS change. | **READY — rank P3-LOW; owner: CC** **Re-ranked 2026-10-03: P3→P4: operator-side automation; the operator creates the tunnel by hand per the day-0 runbook.** | — | — | CC |
 | **R-504** | Install & onboarding | P4 | **[P3-LOW] `iso.felhom.eu` cannot show an index page on its own — its root returns 404, and the download page lives on the website instead.** MEASURED 2026-09-14: `https://iso.felhom.eu/` and `/index.html` → 404; only named objects answer. The host is an R2 bucket behind a custom domain; whether R2 would serve an uploaded `index.html` at `/` was **not measured** (uploading anything to the public bucket is a publication). The ISO v1.27.0 task puts the Hungarian download page at `felhom.eu/letoltes` (published with the ISO, after the operator's yes). **Remaining:** a redirect from `iso.felhom.eu/` to that page needs a Cloudflare rule the session has no credential for. | **WAITING-ON-OPERATOR — rank P3-LOW; owner: operator (Cloudflare rule)** **Re-ranked 2026-10-03: P3→P4: households are sent to the website's download page; the bare address is cosmetic.** | — | — | operator |
-| **R-1203** | Install & onboarding | P2 | **When a box dies and the household reinstalls before the operator deletes the old host, the household is promised a new link mail that never comes, and nothing tells it to ask Felhom.** Rehearsal 2026-10-20 B7 (`audits/rehearsal-2026-10-20/B7-died/`, screens `b7-install/b7-newlink-*`): the old box destroyed 08:44Z, the new one installed from the public ISO and waiting with a pairing code at 08:53Z; the household's spent bind link says „Ez a hivatkozás már fel lett használva … Új linket kérek"; pressed → „Kész. Ha … a dobozod még nincs összekötve, néhány percen belül új e-mailt kapsz". No mail came (mailbox read 4 min later, names only; control: older mails to the same address are found). The hub still counts the dead box as linked, so the self-service path is a dead end; the console keeps waiting. The guide's „delete the old host first" is an OPERATOR step the household cannot see. Related: R-1176 (what happens after the delete), D19. | **OPEN** | — | when the customer already has a host, the new-link page should say so in plain words („a régi dobozod még össze van kötve — írj az ügyfélszolgálatnak") and the operator should get a mail with the host name to delete | CC |
 
 ## Apps & catalog — 14 rows (P3 6, P4 8)
 
@@ -157,17 +156,12 @@ stopping line that lies.
 
 | ID | Category | Sev | What | State | Blocked on | Next action | Owner |
 |---|---|---|---|---|---|---|---|
-| **R-1175** | Backup & restore | P1 | **Shared folders on two drives that use the same second drive: every night one drive's shares lose their second copy, and the night a drive dies its share copy is deleted.** The shares second-drive job runs one group per source drive; each group's clean-up walks EVERY drive key on the target and removes what is not its own (`internal/backup/tier2_shares.go:201-230`, `:266-267` → `tier2.go:299-329`). PROVEN against production code (fake copy seam): shares on hdd_2 and hdd_3 both target hdd_1 — run 1 logs `removed stale dest dir hdd_2/b_share`, so after EVERY run only the last group's shares have a copy while the status says ok; then hdd_3 goes away — run 2 logs `removed stale dest dir hdd_3/csaladi_fotok`: the dead drive's only second copy is deleted that night. Data-loss hunt 2026-10-20 (C1 reviewer 2, C2, C3: `audits/dataloss-hunt-2026-10-20/C3-D1-shares-tier2/`). | OPEN | — | Fix: reconcile only the group's OWN key dir (and keep a key whose drive is away/decommissioned until the household releases it); test with the C3 cases as red-proofs; release by the normal train | CC |
-| **R-1176** | Backup & restore | P1 | **A box rebuilt after „delete the old host first" cannot reach the household's off-site history: the recovery code is never asked for, a new key is minted, and the old history is set aside on night one.** D19 (`hub/internal/api/passphrase_scope.go:58`) makes the reinstall stop until the operator deletes the old host; the delete demotes the escrow under the OLD host id and deletes the host row (`hub/internal/store/store.go:3164-3229`); the new box's status query joins on live hosts only (`store.go:3499-3527`), so it reports no package and no retained package (both flags false) → the controller mints a fresh repo password (`offbox.go:542-555`), offers no recovery (`:1788`) and on its first night sets the old repository aside (`:905-906`); the retained-package API 404s for a deleted host (`handler.go:1691`). The reset and purge paths DO read `host_deletions` (`customer_reset.go:48,155`; `store.go:4189`) — the status path does not. PROVEN at the hub store against production code (`audits/dataloss-hunt-2026-10-20/C3-K1-escrow-after-host-delete/`: control = present; after delete + new host = nil). 03 §8a and 07 §7.0 promise this route; D19 + R-1154 removed it without a recorded choice. Data-loss hunt 2026-10-20 (C1 reviewer 3). | OPEN **PROVEN LIVE 2026-10-21 (rehearsal B7, `audits/rehearsal-2026-10-20/B7-died/README.md`):** Tester4 destroyed, reinstalled from the public ISO, the operator deleted the old host, bind + claim OK; the rebuilt dashboard shows only „A távoli mentés szünetel, amíg nem hozod létre a helyreállítási kódot. Helyreállítási kód létrehozása" (create, never enter), not the product's „A korábbi házon kívüli mentéseid visszaszerezhetők" sentence; the restore page: „Nem tudjuk elolvasni a távoli tárolót". The new code was not created and the box was torn down before its first night, so the history stays on the Storage Box. | the rehearsal's B7 is the live proof (needs Tester4 off-site ON) | Decide the shape (status query includes `host_deletions` for the same customer, and the retained GET answers for a deleted host of the caller's customer); red-proof with the C3 test; hub + controller release by the normal train | CC |
 | **R-232** | Backup & restore | P2 | **DooPlex's backup makes every copy inside the same box — and nothing tells anyone when it fails.** Surveyed read-only 2026-08-06 (`audits/RECON-dooplex-backup-2026-08-06.md`). **What works:** five sets, 14/14 successful runs in 14 days; a file was restored from the `data` repo and matched the live original **byte for byte**; every set except two is cross-disk; k3s is integrity-checked on every run. **What the matrix exposes, ranked:** (a) **`notify_failure` is a no-op** — `NOTIFY_ON_FAILURE=true` but `NOTIFY_WEBHOOK_URL` is commented out, so a failed backup notifies **nobody**; the project already has a working Resend path that CI uses. Cheapest item, and it makes every other failure visible. (b) **Nothing leaves the box** — no rclone, no remote repo, no off-site target anywhere; Longhorn's target is `nfs://192.168.0.180:` pointing at DooPlex itself, and the only outbound-looking cron pulls *inbound* from Hetzner for a different project. The machine that runs the hub managing the customers' off-site chain has no off-site copy of its own. (c) **The backup tree is a single writable path** and the restic repos are not append-only — one bad script or ransomware destroys every copy at once. (d) **Two same-disk sets**: `.claude-memory` and the PostgreSQL dumps, whose source directory sits *inside* the backup tree. (e) **Longhorn `retain=1`** — one generation per volume, so a corruption noticed a day late has no earlier copy. (f) **`/opt/backup/docs/BACKUP-RESTORE.md` does not exist** though the systemd unit advertises it. (g) **`secrets/restic-repo` has never held a snapshot** — `backup-secrets.sh` contains no `restic` call; the secrets are GPG files on `sda1` only. (h) **No restore has ever been run** beyond today's single-file probe — the matrix's "ever demonstrated?" column is otherwise entirely empty. **Not a finding:** the restic passphrase. The on-box copy is on `sdb1`, a different disk from the backups, and the **operator holds an offline copy out of band** — so a disk loss is recoverable. The narrow residual is that it is operator-held rather than system-held, unlike the customer case's hub-vaulted escrow, so it should be confirmed current and findable by someone else. **Nothing was changed by the recon.** | **NARROWED 2026-10-08 — owner Viktor.** (a) DONE with the operator's yes in chat: `notify_failure` now mails admin@felhom.eu through Resend; proven by one test mail that reached the inbox (`audits/day-2026-10-08/r232/`; no backup was started). (b) partly: the hub database leaves DooPlex nightly to ep0 (R-173); everything else stays on the box. (c)–(h) unchanged. **2026-10-09 (operator yes in chat): (b) and (h) DONE for Gitea and the secrets.** Nightly 00:20 `felhom-dooplex-offsite` pushes Gitea (614 MB of repositories, the `gitea` database dump taken first, `app.ini`) and the nightly GPG secrets export, encrypted with a NEW key, to ep0 `operator` (`host/dooplex-gitea`) on the hub-DB write-only token — no change on ep0; first push 57 s; restore test weekly (manifest, `git fsck` every repo, `pg_restore --list`), run once: 27 805 files, 10 repos OK; `DooplexGiteaOffsiteStale`/`DooplexGiteaRestoreTestStale` in force (`absent()` included, promtool + 2 red-proofs); every backup unit now mails on failure (`OnFailure=`, dry run reached the inbox). **First real Gitea restore:** into a throwaway on the bench, 10/10 repos, product `main` = live, a file byte for byte, a login (`runbooks/gitea-restore.md`). The hub came back too (R-173, closed). **The registry is left out on purpose:** its images rebuild from the code. **Correction to the recon §7:** the API mirror holds ONE repo (`homelab-manifests`), not all — Gitea had a single copy, on DooPlex. `audits/dooplex-survival-2026-10-09/` **LEFT:** (c) append-only for the on-box tree, (d)–(g) as written; the paper copy of the new key (operator). **READY** for the rest | — | — | operator |
 | **R-895** | Backup & restore | P2 | **The hub's clean-up-window check trusts the snapshot counts the box sends, so a broken-into box (or past-dated fakes added through the add-only key) can shrink the real off-site history without an alarm.** READ 2026-10-06 night in source (R-822's design): the before/after comparison uses counts the box itself reports (`hub/internal/offsitekeys/service.go:284`, `:343`); new fakes keep the count level. Decision 68 already accepts a box-trusted count. | **NARROWED 2026-10-18 — slice 2 built, not deployed** (hub v0.158.0): the 07:10 audit compares the `snapshots/` listing with yesterday's by name — files gone that no clean-up window explains → `offsite_snapshots_vanished` (error), files no reported box run explains → `offsite_snapshots_unexplained` (warning); operator-only, hub-origin (not in `allowedEventTypes`). First listing = baseline; a move-aside or RESET drops it; vanishing judged on a pinned tier only; files younger than 1 h judged the next day. Red-proofed twice (`snapshot_watch_test.go`). **Not built:** the window's before/after on the hub's own count (design §3.1) and refusing a window while unexplained files exist (§3.2). Unmeasured: the provider shell's `ls -l` time zone (± 3 h slack until then). **Deployed 2026-10-18 12:23 (hub v0.158.0).** The first listing (tomorrow 07:10) is a baseline; the first verdict is 2026-10-20 07:10. **Known limit (independent review 2026-10-18, F2):** `_unexplained` is explained by the box's OWN reported `offsite.last_run`, so whoever holds a box's key can silence it; `_vanished` is judged on the hub's own listing and is not affected. **2026-10-20 03:5x UTC: due date moved 10-20 → 10-21** — the first verdict lands at 07:10 hub time (05:10 UTC), after the night session that reached it ended; that brief did not include this read. Nothing is lost: the hub keeps its listing; read the 2026-10-20 07:10 lines on 10-21 (or later on 10-20). **FIRST VERDICT READ 2026-10-21 (rehearsal session, read-only):** the 07:10 CEST run judged three tiers against yesterday — demo-felhom 5, demo-hp 14, tester-1 4 new snapshot files, „each inside a reported box run"; nothing vanished, nothing unexplained; Tester4 stored its first baseline (44 files); Tester-2's own listing failed (no `snapshots/` dir — the box has never pushed). No `offsite_snapshots_vanished` / `_unexplained` mail in the operator mailbox (second channel). Evidence `audits/due-checks-2026-10-21/R-895-hub-0710.txt`. The positive alarm path is still proven only by the unit red-proofs. | the operator's yes to deploy hub v0.158.0 | Deploy; read the first 07:10 lines (newest upload vs the box's last run = the time-zone calibration), shrink the slack; then decide §3.1/§3.2 | CC |
 | **R-923** | Backup & restore | P2 | **The password manager (Vaultwarden) runs on DooPlex and holds the keys that open DooPlex's own off-site copies — a loss of DooPlex loses the keys with the copies.** Operator finding, 2026-10-09 11:19: "the password manager also runs on DooPlex (Vaultwarden), so I probably need to back up its DB somewhere too." Measured: `vaultwarden-system/vaultwarden` (1.37.3), PVC `vaultwarden-data` 7.4 MB (`db.sqlite3` + WAL, `rsa_key.pem`, no attachments); its only backups were Longhorn on DooPlex. **A key in a password manager that runs on DooPlex is not off DooPlex** — R-173's "both keys are off DooPlex" (2026-10-05) and the hub-DB runbook Step 0 were wrong in that sense. Also seen: Vaultwarden warns its `ADMIN_TOKEN` is plain text (an Argon2 hash is its guidance). | **NARROWED 2026-10-09 (operator yes in chat):** Vaultwarden's own `backup` copy + `rsa_key.pem` ride the nightly encrypted DooPlex→ep0 job (`scripts/dooplex-offsite/`), checked on every push and every Sunday (integrity, users, items — rows only). That copy opens only with the DooPlex off-site key, and the vault only with the master password — so the circle is broken only once both are on paper outside DooPlex: `runbooks/break-glass-sheet.md`, `runbooks/total-loss-of-dooplex.md`. **LEFT:** the operator prints the sheet and stores it away from home; the `ADMIN_TOKEN` hash (a DooPlex change, operator's word). | operator: print the sheet | Operator: print + store the sheet; then close | operator |
 | **R-1177** | Backup & restore | P2 | **A local restore that fails destroys the household's live data and keeps no copy of it: the app is left broken.** PROVEN LIVE on scratch 9202 (`audits/dataloss-hunt-2026-10-20/C3-R1-local-restore-9202/`): Rallly + Postgres; control restore OK; then a live row C, the unit's DB volume tar damaged (half), the household's restore → the live volume removed, `tar: short read`, the DB replay correctly skipped, the app started → Postgres crash-loops (`initdb: directory … exists but is not empty`); row C gone; NO undo or safety copy anywhere; the household reads „Visszaállítás sikertelen: starting rallly after restore from unit: starting stack rallly: exit code 1 stderr: …" (raw English Docker output). Mechanism `internal/backup/restore_unit.go:428-489`, `restore.go:215-246`. The OFF-SITE path got „put back exactly as it was" in v0.336.0 (D20 = A, R-893); the LOCAL path — 07 §3's main customer restore — did not. Two reviewers found it independently (R1, D4). | WAITING-ON-OPERATOR | decision: extend D20's put-back (safety dump + live-volume copy before the first change, rollback on failure) to the local and second-drive unit restores? | Operator: answer the D-question in STATUS; then CC builds it with the R-893 harness | operator |
 | **R-1178** | Backup & restore | P2 | **The second drive's „Teljes visszaállítás" can change files, then refuse the database half, leave the app STOPPED, and tell the household „the app and its data are untouched".** `tier2_whole.go:301-340` stops the app and merges the files first, then calls the unit restore, whose refusals (cut-off dump, mixed versions, allow-list, data key — `restore_unit.go:366-423`) all return before its own stop/start. PROVEN against production code (`audits/dataloss-hunt-2026-10-20/C3-R5-whole-restore/`): 1 file restored + 1 replaced (the live one kept beside), calls `[stop]`, no start, message „…A visszaállítás biztonsági okból nem indult el — az alkalmazás és az adatai érintetlenek." Data-loss hunt 2026-10-20 (C1 reviewer 1). | OPEN | — | Fix: run the unit's pre-checks BEFORE the file half (or start the app and say what changed on a refusal); red-proof with the C3 test | CC |
-| **R-1179** | Backup & restore | P2 | **A drive marked „away" can still be chosen as the second-drive backup target.** `tier2.go:151-180` loops `GetSchedulableStoragePaths()`, which drops only decommissioned drives (`settings.go:1593-1603`); nothing checks `Disconnected`. Seen in the R-1175 unit test (a share copied to a drive marked disconnected). On a real box the away drive's mount point is an empty folder on the guest's root disk (or the host's root via the bind), so the night's copy fills the wrong disk and is not where the household thinks. CONFIRMED in code; not run live. Data-loss hunt 2026-10-20. | OPEN | — | Fix: skip a disconnected target like a decommissioned one; live check on a scratch box with a removed drive | CC |
 | **R-1180** | Backup & restore | P2 | **The nightly second-drive mirror (`rsync --delete`) can run DURING a restore from that same mirror and delete files not yet copied back.** `RunAllTier2` (`tier2.go:479-520`) takes no backup flag and does not ask about a running restore; the whole restore holds only the backup flag (`tier2_whole.go:301`), the night leg only `LegLock` (`cmd/controller/main.go:1348-1352`). A long restore of a photo library at 03:30, or 15 min after a missed night, overlaps. PROVEN against production code with the real `rsync -a --delete` mirror (`audits/dataloss-hunt-2026-10-20/C3-T1-mirror-during-restore/`): with the backup flag held by a restore, `RunAllTier2` still mirrors, and the mirror goes from [a,b,c,d] to [a,b] — the two files the restore was bringing back are gone from the only copy. Data-loss hunt 2026-10-20 (C1 reviewer 4, T1). | OPEN | — | Fix: the mirror leg skips (and says so) while any restore runs; test with the existing tier-2 seams | CC |
-| **R-1181** | Backup & restore | P2 | **A missing or emptied app folder deletes its second-drive copy within one night; for OPTIONAL folders (audiobooks, comics, ROMs, Immich's external library) that copy was the only one.** A missing declared leg is skipped (`tier2_capture.go:55-61`, silent for optional legs) and then reconciled away (`tier2.go:454`); an emptied one is mirrored with `rsync -a --delete` (`:744-751`). PROVEN for a missing folder against production code (`audits/dataloss-hunt-2026-10-20/C3-D2-missing-leg/`): the mirror's `photos` deleted, run „ok", warning only „nem kerültek a másodlagos mentésbe". 07 §8.2 chose „data legs are not guarded" for a leg that LEAVES the class set; a declared leg that is MISSING is not that case. Optional legs are in no off-site tier (07 §6.2). | WAITING-ON-OPERATOR | decision: keep a missing leg's mirror (and say so) until the household releases it? And: does an accidental folder deletion need a second-drive history for optional legs? | Operator: answer in STATUS | operator |
-| **R-1182** | Backup & restore | P2 | **Abandoning ONE set-aside off-site copy deletes EVERY retained key package of the customer, including ones that open other set-aside copies still stored.** hub `api/handler.go:626-627` → `PurgeSupersededEscrowForCustomer` (`store.go:4185-4190`) deletes customer-wide; the comment right above (`:4176-4179`) states the invariant this breaks. CONFIRMED in code (the SQL deletes every retained row of the customer, live and deleted hosts alike; nothing ties a row to the abandoned path). The R-1176 reinstall makes a second set-aside copy likely. Data-loss hunt 2026-10-20 (C1 reviewer 3, K2). | OPEN | — | Fix: purge only the package whose key opens the abandoned path (record the path or repo id with each retained row); hub store test as red-proof | CC |
 | **R-1195** | Backup & restore | P2 | **A power cut in the middle of a full off-site restore can leave the app running on the BACKUP's data while the page says only „the restore was interrupted, start it again" — the household's newer data is gone and nothing says so.** PROVEN LIVE in the rehearsal 2026-10-20 (`audits/rehearsal-2026-10-20/B6-failures/s2-power-cut-mid-restore/`): BookStack, a page added after the backup; „Teljes visszaállítás"; power cut at 14 s (the app stopped at 8 s, the database starting for the replay) → after power-on the crash recovery restarted the app (`appstop_marker.go`), recorded „A visszaállítás megszakadt (a doboz újraindult) — indítsd el újra." and pushed `restore_interrupted`; BookStack ran on the backup's 5 pages, the newer page gone; no hold; the undo copy written before the change (`offbox_restore_undo.go`) was not used. A cut at 6 s (before data moved) behaved correctly. This is R-1188 (a) proven; R-1188 keeps its other items. | OPEN | — | On start after an interrupted restore: put the undo copy back (D20's „exactly as it was") or hold the app and say „a visszaállítás félbeszakadt — az adatok a mentés állapotában lehetnek"; a test that kills the controller between the volume step and the replay | CC |
 | **R-49** | Backup & restore | P3 | **[P2] The offsite capture set is ~90% cache and duplication — 1.1 GB of a 1.2 GB immich "photo backup".** Measured 2026-07-19: `immich_ml_cache.tar` **823 660 032 B (~60%)** — re-downloadable ML model weights; `immich_postgres_data.tar` **308 251 136 B (~23%)** — a raw tar of the postgres data dir that DUPLICATES the logical `.sql` dump captured beside it; `upload/backups/` **18 MB** — immich's own nightly dump, a backup inside the backup, growing daily; plus the stranded pre-v3 `dccc13fe…` tree (~36 MB) no DB has ever referenced. Actual irreplaceable content: **72 MB of originals**. **MIGRATED FROM `ROADMAP.md` 2026-08-22 (R-369) — originally filed 2026-07-19, size S–M, roadmap state `idea`.** Moved verbatim; nothing added or reinterpreted. The roadmap keeps its copy as history, marked moved. | **OPEN — migrated from ROADMAP 2026-08-22, rank unchanged** **Re-ranked 2026-10-03: P2→P3: wasted space and transfer, no data risk; needs a capture-set ruling, not a sale blocker.** **READY 2026-10-20 — measured and planned, nothing changed; the decision is the operator's (STATUS D21).** Re-measured on 9202 (Immich v3.2.4): off-site ~1.33 GB = model cache 824 MB (62 %), raw database folder 283 MB (21 %), Immich's own backups 115 MB (9 %), logical dump 54 MB, originals 49 MB, thumbnails 2 MB — 102 MB must survive. A: leave out all three re-fetchable parts (−92 %; needs a never-run restore from the dump alone on the vectorchord image). B: only the model cache (−62 %; needs one restore with an empty cache). Pick: B now, A's other parts after their proof (`audits/night2-2026-10-20/P4/immich-offsite-plan.md`). | — | **Evidence: `audits/DIAG-immich-restore-round2-2026-07-19.md` §4 (full byte breakdown).** This is the customer's offsite quota and transfer cost, and it lands on the Hetzner sub-account they are billed for. **Recorded, deliberately not changed** — a capture-set exclusion is a data-loss-shaped decision and gets its own ruling, not a drive-by edit. Candidates in priority order: (a) `immich_ml_cache` — pure cache, strongest case; (b) the `postgres_data` volume tar where a logical dump of the same DB is already captured (the dump is what the restore path actually replays); (c) `upload/backups/`. Likely generalises past immich into a template-classification rule about cache volumes and self-backup directories, so it should be specified against the catalog, not one app | CC |
 | **R-314** | Backup & restore | P3 | **`StopAbandon` has no web route — a customer who telephones is served by a command line.** `--abandon-stop` exists on the controller binary (`cmd/controller/main.go:86`) and refuses rather than silently no-opping, which is right. But there is no handler: the only in-product way to cancel a countdown is the customer finding their recovery code (Scenario G cancels it at the moment the code proves they still have it). **An operator who is telephoned instead has to reach a shell on the customer's machine.** Used this session on the operator's ruling, container stopped first so the running controller could not overwrite `settings.json` from memory — a sequencing subtlety that is itself an argument for a route **-- 2026-10-08 design:** `audits/day-2026-10-08/design-R-314-279-177.md` — the box's own 14-day countdown has no operator door (the hub's 7-day phase does, decision 74); one door for R-314 + R-279: a fixed list of operator actions carried in the report reply. Question D1 on STATUS's decision sheet. **-- 2026-10-08 14:16 operator ruling D1 (`09` §3 decision 185):** yes — option A, a fixed list of operator actions carried in the box's report reply. | **VERIFY — 2026-10-09: DELIVERED (hub 0.144.0 + controller 0.304.0); `run_job fill-watch` proven live on the Tester 1 box (hub row „done — ran to its end"; control: the box log „checked 3 filesystem(s)"; an unknown action is refused 400). `abandon_stop` / `abandon_extend` NOT yet proven — no box has a deletion countdown; prove them on the first one.** **VERIFY — built on main 2026-10-08 (D1, decision 185), ships with tomorrow's controller + hub releases; closes after the live proof on 9202** (`abandon_stop` / `abandon_extend` through the hub's buttons). **READY (S) — NEW 2026-08-12, RANK 3** **2026-10-05 (burn-down night): NEEDS A DESIGN.** There is no operator door into a RUNNING controller (its HTTP surface is the household's session; the CLI runs in a separate process and cannot safely trigger a job in the running one). Same design as R-177 and R-279. | R-241, R-307 | An operator-authenticated POST that calls the same `StopAbandon`, so the telephone path and the code path converge | CC |
@@ -195,12 +189,14 @@ stopping line that lies.
 | **R-1185** | Backup & restore | P3 | **A database dump that fails its own content check has already replaced last night's good dump, and the second drive copies it an hour later.** `appbackup/dbdump.go:388-422`: size check, rename over the good file (`:399`), then `ValidateDump` (`:419`). Volume tars are promoted with no content check either (`backup.go:902`). Tier 1 is one point per app by design (07 §6.1); the ORDER is the defect. PROVEN against production code (`audits/dataloss-hunt-2026-10-20/C3-D5-dump-replaced/`, a docker stub prints a header-only dump): `ValidateDump` says invalid („dump file too small"), `DumpOneTo` returns no error, and last night's good dump (with its `CREATE TABLE`) is gone from disk. Data-loss hunt 2026-10-20 (D5). | OPEN | — | Fix: validate the temp file before the rename; keep the last good dump on a failed check | CC |
 | **R-1186** | Backup & restore | P3 | **The nightly off-site copy is skipped silently when another backup holds the flag, and the catch-up never re-runs it.** `offbox.go:981-996` returns nil with an INFO line on the nightly path; `withLeg` marks the leg ended anyway (`cmd/controller/main.go:1246`). The 05:30 proof and 06:00 integrity check take the same flag, so a backup time near them collides every night. Only the hub's 48 h `offsite_stale` notices. CONFIRMED in code. Data-loss hunt 2026-10-20 (T5). | OPEN | — | Fix: mark the leg NOT ended on a busy skip so the catch-up retries; say it on the backups page | CC |
 | **R-1187** | Backup & restore | P3 | **`.fab` export/import candidates from the data-loss hunt, not yet proven:** (a) an export silently drops a drive folder that holds a symlink (`appexport/fabplan.go:292`, `export.go:714`); (b) the import never sets file owners and drops symlinks (`restore.go:1157-1176`); (c) import into a STOPPED app merges into the existing volumes, and the default export (stop ticked) carries no SQL dump (`restore.go:361-374`, `export.go:262,366`, `appbackup/dbdump.go:142`); (d) no concurrency guard, no undo, no record (`restore.go:181-186`, `web/handler_export.go:312-353`); (e) PROVEN and split out as R-1191 (the import drops the family gate); (f) it trusts the bundle's HDD_PATH (`restore.go:589-631`); (g) the DB import runs psql without ON_ERROR_STOP and misses postgis (`restore.go:1052-1053, 928-937`). Read by reviewer 1; main session spot-checked (a); (e) proven live → R-1191. Evidence `audits/dataloss-hunt-2026-10-20/C1-restore-rollback-candidates.md`. | VERIFY | — | Prove (c) first on 9202 with a throwaway app; split into rows only what reproduces | CC |
-| **R-1188** | Backup & restore | P3 | **Restore-path candidates from the data-loss hunt, not yet proven (one confirmed in code):** (a) PROVEN LIVE → R-1195 (a power cut mid-restore restarts the app on the backup's data; the undo copy is not used); (b) a failed `volume rm` is ignored before the untar (`restore.go:218`); (c) PROVEN against production code (`audits/dataloss-hunt-2026-10-20/C3-R12-leftover-temp/`): an interrupted second-drive whole restore leaves `<file>.felhom-restoring`, and EVERY retry fails „fájlmásolás sikertelen: … file exists" with the file never back (`tier2_whole.go:86-87`, O_EXCL); (d) the full off-site download has a 30-min cap per press and the household is not told to retry (`web/offbox_handlers.go:455`); (e) place and full restore re-read the LATEST snapshot instead of the downloaded one (`offbox_restore.go:675-679`). Evidence `audits/dataloss-hunt-2026-10-20/C1-restore-rollback-candidates.md`. | VERIFY | — | Fix (c) now (remove a stale temp of the same name before the copy); prove (a) with the B6 power-cut-mid-restore scenario | CC |
+| **R-1188** | Backup & restore | P3 | **Restore-path candidates from the data-loss hunt, not yet proven (one confirmed in code):** (a) PROVEN LIVE → R-1195 (a power cut mid-restore restarts the app on the backup's data; the undo copy is not used); (b) a failed `volume rm` is ignored before the untar (`restore.go:218`); (c) **FIXED in controller 0.337.0** (a leftover temp of the same name is removed before the copy; `TestR1188c_ALeftoverTempDoesNotBlockTheRetry`, the hunt's C3 case, red-proved) — was PROVEN against production code (`audits/dataloss-hunt-2026-10-20/C3-R12-leftover-temp/`): an interrupted second-drive whole restore leaves `<file>.felhom-restoring`, and EVERY retry fails „fájlmásolás sikertelen: … file exists" with the file never back (`tier2_whole.go:86-87`, O_EXCL); (d) the full off-site download has a 30-min cap per press and the household is not told to retry (`web/offbox_handlers.go:455`); (e) place and full restore re-read the LATEST snapshot instead of the downloaded one (`offbox_restore.go:675-679`). Evidence `audits/dataloss-hunt-2026-10-20/C1-restore-rollback-candidates.md`. | VERIFY | — | Prove (a) with the B6 power-cut-mid-restore scenario (R-1195 holds it); (b), (d), (e) still candidates | CC |
 | **R-1189** | Backup & restore | P3 | **Time and concurrency candidates from the data-loss hunt, not yet proven:** (a) a host clock ahead during one whole-guest backup silences that tier until real time passes it, and the hub reads the same clock (agent `localapi/server.go:1178-1192`, `backup/runner.go:607-629`, hub `monitor/deadline.go:243`; R-973 cap counts future runs `backup_cap.go:61`); (b) the whole-guest start creeps ~5 min/night until a night is skipped silently (`server.go:1184`, `quiesce.go:150, 807-818`) — PARTLY SUPPORTED by demo-hp's agent journal 2026-09-30…10-20: completions drift 04:36→04:50 between resets, and THREE nights had no local whole-box backup (Oct 2, 6, 9), each right after an off-cycle daytime/evening run (Oct 1 22:15, Oct 5 11:24, Oct 8 22:25) — the 24-hour „not due yet" rule skips the next night; nothing tells anyone; (c) the whole-guest quiesce stops apps mid-restore or mid-update (`quiesce.go:213-275, 565`); (d) CONFIRMED in code: the off-site proof and integrity check are plain daily jobs outside the catch-up (`nightchain.go` header: the catch-up runs only the three backup legs), so they never run on a box that is off at 05:30–06:00, while the comment at `cmd/controller/main.go:1575-1577` says the opposite; (e) CONFIRMED in code: the hub's 05:00 check wants a dump „since Budapest midnight" (`hub/internal/monitor/deadline.go:416-419, 556-565`), so an evening backup time (which the banner suggests) raises a false `expected_dbdump_missed` error every morning — and on 2026-10-25 the check also runs at 04:00 (R-1183); (f) the OS leg and kernel reboot follow a daytime valve backup (agent `server.go:982-986`, `kernel.go:101-114`) — design question; (g) spring forward 2027-03-28 fires dump and second copy together; (h) self-update busy check before the pull only (`updater.go:532-545, 764`); (i) a double-clicked restore shows failed while it runs (`web/handlers.go:1853-1874`). Evidence `audits/dataloss-hunt-2026-10-20/C1-time-concurrency-candidates.md`. | VERIFY **(a) half-live 2026-10-21 (B6 #8):** the box wrote a local whole-box archive dated 2028-10-21 while its clock was 2 years ahead (`s8-wrong-clock/README.md`); by `localapi/server.go:1187` the local tier is „within cadence" until then. The due answer itself was not read live (the box died in B7 before its night window). The same run found R-1204 (the ep0 tunnel). | — | (a) is B6 #8's live test; (b) read the agent journal on demo-hp; the rest by unit tests | CC |
 | **R-1190** | Backup & restore | P3 | **Key and retention candidates from the data-loss hunt, not yet proven:** (a) a recovery ceremony while off-site is not configured seals a package with no repo password and supersedes the good one (`escrow_handlers.go:217`, agent `main.go:2975-2981`, hub `store.go:3392`); (b) the global „create your recovery code" bar links to the ceremony the backups page hides during a recovery offer (`escrow_banner.go:23-33` vs `backups_remote.html:144-160`); (c) after a reuse-path reinstall the new PBS key K is never escrowed and nothing compares fingerprints (`felhom-pbs-apply:99`, `escrow_confirm.go:182-205`); (d) whole-guest off-site copies run before any escrow exists (design question vs 07 §6.1's Tier-3 gate); (e) the escrow PUT is two statements; (f) the operator's off-site re-issue resets the provider password before saving it (hub `offsite.go:217-245`); (g) „Megőrzött adatok" can list a running app's live folder when its app.yaml is unreadable (`kept.go:220-235, 330-402`); (h) a unit restore can point an app back at an old drive and F5 then prunes the current drive's unit (`backup.go:1517-1574`). Evidence `audits/dataloss-hunt-2026-10-20/C1-keys-candidates.md`, `C1-delete-retention-candidates.md`. | VERIFY | — | Unit tests first for (a), (e), (g) | CC |
 | **R-1198** | Backup & restore | P3 | **A backup disk the family adds later is never offered as the place for the whole-box backup; the page keeps offering the disk that holds the apps' own data.** Rehearsal 2026-10-20 B6 #13 (`audits/rehearsal-2026-10-20/B6-failures/s13-second-drive/`): a 50 GB disk plugged in and initialised as „Mentolemez" → the backups page still offers only „Családi adatlemez — Kijelölöm"; `internal/web/backup_target_offer.go:101` `firstOfferableDrive` offers the FIRST eligible drive, never a choice, and the drive's card has no „use for backups". | OPEN | — | Offer every eligible drive (largest empty first) and let the household pick | CC |
 | **R-1200** | Backup & restore | P3 | **A box installed in the afternoon gets no whole-box backup on its first night: the install's own backup starts the 24 h clock, so the family's whole first day reaches a whole-box copy only on the SECOND night (~38 h after install).** Rehearsal 2026-10-20 B5 (`audits/rehearsal-2026-10-20/B5-night/README.md`): Tester4's install backup at 12:09Z; at the 02:30–06:30Z window the agent answered „within cadence window" (`felhom-agent internal/localapi/server.go:1187`, `age >= tier.Cadence`, default 24 h). R-899 (closed) fixed the same shape for a daytime PRESS only. The other tiers did run that night (database copy, second drive, off-site — all PASS), so app data was covered; the whole-box (DR) tier was not. | **OPEN** — a design question: should the first scheduled night always run (like a press), or is one tier enough on night one? | — | decide; if „always run": treat the install backup like a press in the due rule, with a test on the first night | CC |
 | **R-1204** | Backup & restore | P2 | **A box that runs once with its clock ahead loses its WireGuard tunnel to ep0 — the PBS whole-box copies and the operator's door — until the real date passes the wrong one, and nothing says why.** Rehearsal 2026-10-20 B6 #8 (`audits/rehearsal-2026-10-20/B6-failures/s8-wrong-clock/`): Tester4 booted with its clock at 2028-10-21 for 13 min, then the real time. ep0's `wg show` (read-only): Tester4's last handshake 07:40:47Z (clock at 2028), none in the 14 min after the return, the other peers 21–96 s. WireGuard refuses a handshake initiation whose TAI64N timestamp is not newer than the last one accepted from that peer, so the peer stays dead until 2028 or until ep0 resets it. Effects seen: every agent PBS call times out (15 s), the controller's due-check errors, the backups page says „A host-ügynök jelenleg nem elérhető". Likely triggers in the field: a dead CMOS battery, a hand-set BIOS clock, a VM restored with a wrong RTC. | **OPEN** — reproduced once live; not reset on ep0 (read-only in this run; the peer still shows the stale handshake). The box is destroyed in B7, which ends it for Tester4. | — | design: detect it (the hub sees the peer's handshake age) and heal it (re-key the peer, or ep0 resets the stored timestamp on a hub-signed request); a test with a fake clock | CC |
+| **R-1205** | Backup & restore | P2 | **After a box death, a SECOND install that is deleted before it makes a key leaves the whole-box (PBS) tier waiting for the operator's re-issue.** Seen live 2026-10-21 (fix-1 proof, `audits/fixes-2026-10-21/P1-live/`): the rebuilt Tester4 box enrolled and the hub logged `pbsdr auto-provision for Tester4 (WG-registration hook): the endpoint already holds a PBS token for Tester4 but the hub has no descriptor — use the explicit "Re-issue PBS credentials" action`. The F-14 auto-re-issue reads the customer's LATEST host deletion, which was the short-lived rehearsal box (B7: deleted un-acked — it never held an escrow row); the R-1176 carry reads the newest ACKED deletion and worked. A household whose first reinstall attempt is abandoned gets no whole-box copy until the operator presses. | OPEN | — | Decide: may the F-14 gate skip an un-acked deletion that held no escrow row (it holds no key — the carry's own reasoning, `store/escrow_carry.go`), or keep the manual re-issue and mail the operator the press? Hub store test either way | CC |
+| **R-1206** | Backup & restore | P3 | **Backups written while the box's clock ran ahead stay „the latest" after the clock is fixed.** The rehearsal's scenario #10 (clock set to 2026-10-28) left off-site snapshots dated 2026-10-28; on 2026-10-21 the rebuilt box's recovery page listed every app's „Legutóbbi mentés: 2026. 10. 28. 07:39", and the full restore said „mentés: 2026. 10. 28. 07:37" — a date in the future (`audits/fixes-2026-10-21/P1-live/07-unlock-right.txt`, `11-nextcloud-full-restore.txt`). restic orders snapshots by their own time, so the future ones are „newest" to every retention rule for a week. NOT measured: whether the box's keep-daily prune then drops a real snapshot. Related R-1202, R-1204 (clock). | OPEN | — | Measure on a scratch repository: the box's retention policy with one future-dated snapshot — is a real one dropped? If yes: refuse to write a snapshot while the clock is outside the hub's time by more than N, or drop future-dated snapshots first | CC |
 
 ## Storage & devices — 10 rows (P3 4, P4 6)
 
@@ -234,7 +230,6 @@ stopping line that lies.
 | **R-908** | Security & access | P3 | **An old Resend API key is still in the history of the `homelab-manifests` repository; it was replaced on 2026-06-29, but whether it is also revoked at Resend is unknown.** FOUND 2026-10-08 by the contact-mailer source search (`audits/mailer-source-2026-10-08/SEARCH.md`, „A side finding"): commit c9648cd (2026-02-05, „added mailer pod") put a key literal in the old `felhom-system/contact-mailer.yaml` comment; the file was deleted in ee93b50 but the history keeps it. Compared by hash only, never printed: it is NOT today's key (`Secret/resend-api`, rotated 2026-06-29, felhom.eu feea0606). If the old key still works at Resend, anyone with read access to that repository can send mail as felhom.eu. | **OPEN — owner: operator** **Note 2026-10-12 (security review):** gitleaks 8.24.3 with its default rules did NOT flag this key in the full history scan, so a scanner gate alone would not have caught it (R-986). | — | In the Resend console, check that the old key is revoked (revoke it if not); rewriting the repository's history is NOT needed once it is revoked | operator |
 | **R-525** | Security & access | P4 | **[P3-LOW] FileBrowser has its own login; putting it behind the dashboard session (traefik forwardAuth or Quantum proxy auth) is a new mechanism nobody has measured.** Filed 2026-09-15 by the P1-fixes task (B.5). R-513 closed the default-password hole with a generated password; a household still has two logins. **What it needs:** a spike on a scratch guest — forwardAuth to the controller session, and what FileBrowser Quantum does with a trusted header. | **READY — rank P3-LOW; owner: CC (spike)** **Re-ranked 2026-10-03: P3->P4: comfort feature needing a new unmeasured mechanism; the default password hole is closed.** | — | — | CC |
 | **R-925** | Security & access | P1 | **`manifests/felhom.secret.yaml` carries REAL secret values, and the Gitea repo that holds it is readable by anyone on the internet with no login.** FOUND 2026-10-09 by a background security review of the legal-pages commit, which flagged the served `<!-- source -->` comments; chasing what those comments pointed at found this instead. **MEASURED, in this order:** (1) `scripts/manifest_bearer_gate.py` itself prints `manifests/felhom.secret.yaml:39 KNOWN-BACKLOG committed secret 65cee3c4…7a86`; (2) the file holds **live-shaped values, not placeholders** — `SECRET_KEY` (69 chars), `SUPERUSER_EMAIL`/`SUPERUSER_PASSWORD` (18), Umami `APP_SECRET` (66) and `POSTGRES_PASSWORD` (34), and a `username`/`password` pair (18); values were never printed, only measured by length; (3) `gitea.dooplex.hu` resolves to **37.191.56.193**, the same public address as the website; (4) an **anonymous** `curl` returns HTTP 200 and 1686 bytes for that exact file, and 200 for `hub/internal/store/store.go` and `manifests/hub.yaml`; (5) **the off-network control**, which is what makes (4) mean anything — fetched from outside the operator's network entirely: a real path returns the file's first line, a nonsense path returns 404, so the 200 is genuine anonymous read from the internet and not a LAN-only ACL. **This contradicts the project's own written model**: `documentation/runbooks/secrets.md:3-4` says *“Secret values are never committed to git. The manifests in `manifests/` carry only placeholders + comments that point here.”* That promise is false today, which is the P1 wording in this register's own scale — see the ranking note below. The same runbook already states the rule that governs the fix: *“The git-history copy stays alive until the value is ROTATED — de-git alone kills nothing.”* **Known neighbours, none of which cover this:** R-887 records that an outside crawler walks the public Gitea pages and leaves *“Gitea's exposure to the crawler”* as the operator's undone item; R-580 still owes a Gitea admin token rotation. Neither says a secret file is world-readable. **Why P2 and not P1, stated so the operator can overrule it:** the exposed credentials guard **analytics and an undeployed healthchecks instance, not household data** — measured: `umami-db` is a **ClusterIP** service on 5432 with no external IP, so the Postgres password is not reachable from the internet; the realistic harm is session forgery against the public `stats.felhom.eu` via `APP_SECRET`, and reuse of those passwords anywhere else. No customer box, hub token or escrow key is in this file. **CC did NOT change anything**: making the repo private could break the public day-0 path (the installer is fetched from a public tag, R-110), and rotation plus repository visibility are operator decisions on production infrastructure. **PARTLY FIXED 2026-10-09, and RE-RANKED P2 -> P1 on what it turned out to be.** The exposed value was not an analytics password: it was the **Gitea `admin` account password** (`is_admin: true`; `/api/v1/admin/users` answered 200), published in a repo `gitea.dooplex.hu` serves anonymously to the internet. That is push access to every repo — including the one whose `website/` is git-synced live and whose `scripts/` is published by tag to **every new box installer** (R-110), i.e. a supply-chain path onto customer hardware. My first ranking said P2 because I had only measured the analytics blast radius; the credential test is what corrected it. **DONE (CC, operator-instructed):** (1) `umami-config` `APP_SECRET` + `POSTGRES_PASSWORD` rotated — the password changed **inside Postgres** too (`ALTER USER`), because the env var is only read at first init; verified by a real beacon returning 200 with a bogus-site-id control still returning 400. (2) `healthchecks-config` `SECRET_KEY` + `SUPERUSER_PASSWORD` rotated (nothing consumes them — there is no healthchecks Deployment). (3) `gitea-creds` no longer holds the admin password at all: it holds a **scoped token** (`read:package` + `read:repository`). **Both scopes are load-bearing and the second was found by breaking it** — a package-only token made the hub log `Template fetch: unexpected status 403`, because the template fetcher reads a raw file out of the `felhom-controller` repo. (4) The Gitea **admin password** was changed and `gitea-system/gitea-admin` updated; **verified: new password 200, old published password 401 — the leak is dead.** (5) The file is removed from git and `.gitignore`'s `*secret*` now applies to it; `manifest_bearer_gate.py`'s `KNOWN_BACKLOG` carve-out is gone, red-proofed with a decoy (exit 1 with, 0 without). **AN INCIDENT CAUSED BY THE FIX, recorded because it is the useful part:** the `rollout restart` needed to pick up the new umami secret put umami into **CrashLoopBackOff and took `stats.felhom.eu` down (503)**. Cause was not the rotation — at `memory: 512Mi` the pod runs for months but **cannot restarLine truncated
-| **R-1191** | Security & access | P1 | **Importing a `.fab` bundle makes a family-only app open to anyone: the family gate is dropped.** PROVEN LIVE on scratch 9202 (`audits/dataloss-hunt-2026-10-20/C3-R1187e-fab-family-gate/`): Glance installed → a stranger gets 401 and the gate file exists; export → remove → the household's import → the app runs, its record has no `family_gate`, the gate file is gone, a stranger gets **200**. The import writes a bare app record (`cmd/controller/main.go:3517-3530` `SaveEncryptedAppConfig`: Deployed, DeployedAt, Env, DesiredState) and the gate reconcile then removes the file of any app with no `FamilyGate` (`stacks/family_gate.go:173-190`); the R-773 carry-over exists only on the unit-restore path (`stacks/deploy.go:840-870`). Family-gated apps: Glance, Grimmory (books), MeTube (videos); on a household box the address is reachable through the tunnel. The sign-up lock and the version pin are dropped the same way (not measured). Data-loss hunt 2026-10-20 (C1 reviewer 1, R8). | OPEN | — | Fix: the import rebuilds the app record from the catalog's metadata (family gate, setup gate, pin) like the unit restore does; red-proof with the C3 sequence; release by the normal train | CC |
 | **R-779** | Security & access | P4 | **[P3-LOW] Part A's "two outside addresses seen as two" is proven through the simulated tunnel only; on the REAL tunnel the second outside address (ep0, one request allowed) was refused by Cloudflare's edge with 403 and never reached the box.** Measured 2026-10-01 19:51 UTC (`audits/visitors-2026-10-01/A/L2-demo-hp-real-tunnel.txt`): no log line on demo-hp; demo-hp's box has no geo restriction in its settings, so a Cloudflare ZONE rule (country or bot, not read) refused a German datacenter address. DooPlex's own address on the real tunnel was seen as itself. **Needs:** one sign-in from a second Hungarian address (the operator's phone off wifi) while DooPlex is locked out — 2 minutes; and say which Cloudflare rule refused ep0. | **WAITING-ON-OPERATOR — rank P3-LOW; owner: operator (a phone), CC reads the logs** **Re-ranked 2026-10-03: P3→P4: a proof gap on the real tunnel; operator-only follow-up.** | — | — | CC + operator |
 | **R-904** | Security & access | P4 | **Cloudflare can read every household's app traffic; replacing it with our own relay is a later item.** Facts (reviewer discussion 2026-10-08; `01`): app traffic and the dashboard reach the box through the Cloudflare Tunnel (`01` §5 trust table, rows end-user ↔ apps and customer ↔ controller UI; §7), and the tunnel's public end is Cloudflare's edge, where TLS ends — so Cloudflare can technically read that traffic (the FAQ says so since 2026-10-08, R-900; „TLS ends at the edge" is not written in `01` — add it there). What Cloudflare gives today, free: inbound reach with no router setup, the CGNAT answer (`01` §4, §7); certificates (`01` §7, the free tier covers one level below a zone); the geo-WAF the hub enforces (`01` §5 last row, §7); flood protection (not in `01`). The alternative named: our own EU relay over WireGuard with TLS passthrough by SNI, certificates on the box, the geo-block on the relay. Its costs: one more machine the operator keeps up, and a single point of reach for every box; weaker flood protection; about a week of work after a spike. Operator ruling 2026-10-08 09:07 (`09` §3 decision 184): a later item. | **DEFERRED** — **2026-10-10: the asked sentence is in `01` §7** („TLS ends at Cloudflare's edge …”); the spike stays deferred. **DEFERRED — after the first customers (operator ruling 2026-10-08 09:07)** | — | A spike after the first customers (the relay's reach, cost and flood behaviour, measured) | operator |
 | **R-1040** | Security & access | P4 | **Until the household makes Sonarr's or Radarr's first admin, any container on the box can read the app's API key.** Measured on the bench 2026-10-14 with the R-1030 seed in place: `AuthenticationRequired=Enabled`, `AuthenticationMethod=None` — `/initialize.json` from another container answers 200 with `apiKey`. Outsiders are blocked by the box's setup gate in that window; a compromised other app is not. | **OPEN** — filed 2026-10-14 | — | Seed the API key at install (`SONARR__AUTH__APIKEY`, measured to work) so the box knows it, and close the window by telling the household in the first steps; or seed a Forms user | CC |
@@ -246,7 +241,7 @@ stopping line that lies.
 
 | ID | Category | Sev | What | State | Blocked on | Next action | Owner |
 |---|---|---|---|---|---|---|---|
-| **R-1192** | Box system & updates | P1 | **On the minimum machine (8 GB), the household's first afternoon froze the box for about an hour: the dashboard and every app stopped answering, and only pulling the plug brought it back.** Rehearsal 2026-10-20 on a fresh box (controller 0.336.0, agent 0.161.0): the install page let the household book 8 apps to 3638 of 3712 MB (98 %) — the guest has 4096 MB in all — and told it on every page „Az alkalmazások csúcsterhelése meghaladhatja a rendelkezésre álló memóriát. **Normál használat mellett ez nem okoz problémát.**" Then the first uploads (53 PDFs to Paperless, photos to Immich, a file tree to Nextcloud — sequential per app) → from ~13:05 UTC `/api/health` gave no answer in 15 s; the VM read ~4 GB/min from its disk (158 → 476 GB) with the network still; the box's own history shows load 139–152 and no samples for 36 min; it came back at 14:00 for minutes and froze again; the plug pulled at 14:11 → dashboard in 2 min 22 s, stable after. Lost on the way: 8 Paperless documents failed (5 ordinary invoices among them), the Immich upload cut at photo 150, Nextcloud at 251 of 3 393 files. Cause NOT diagnosed (no shell in the box; memory read ~2.3 GB while load was 150 — not plain „memory full"); the one hint: the hub's `app_oom` at 13:34 — immich-machine-learning killed by its memory limit. Reproduced a SECOND time at 15:27:55 with only one Immich upload + one chunked Nextcloud upload running (~35 min, then froze again; plug pulled 16:13); per-app memory now logged every 30 s (`B4-use/memwatch.txt`). Evidence `audits/rehearsal-2026-10-20/journal.md` (12:56–14:23), `B4-use/box-metrics-after-plug.txt`, `B4-use/vm-counters-before-plug.txt`, `B4-use/after-plug-watch.txt`. | OPEN | a diagnosis on a box with a shell (rerun on a scratch nested VM with the vaulted password) | Diagnose (per-container memory.events, PSI, swap, IO wait during the same uploads); then either lower the admission ceiling well below 100 % of the guest, raise the guest's share of the host, or cap the apps' background work; and drop the „nem okoz problémát" sentence until it is true | CC |
+| **R-1192** | Box system & updates | P1 | **On the minimum machine (8 GB), the household's first afternoon froze the box for about an hour: the dashboard and every app stopped answering, and only pulling the plug brought it back.** Rehearsal 2026-10-20 on a fresh box (controller 0.336.0, agent 0.161.0): the install page let the household book 8 apps to 3638 of 3712 MB (98 %) — the guest has 4096 MB in all — and told it on every page „Az alkalmazások csúcsterhelése meghaladhatja a rendelkezésre álló memóriát. **Normál használat mellett ez nem okoz problémát.**" Then the first uploads (53 PDFs to Paperless, photos to Immich, a file tree to Nextcloud — sequential per app) → from ~13:05 UTC `/api/health` gave no answer in 15 s; the VM read ~4 GB/min from its disk (158 → 476 GB) with the network still; the box's own history shows load 139–152 and no samples for 36 min; it came back at 14:00 for minutes and froze again; the plug pulled at 14:11 → dashboard in 2 min 22 s, stable after. Lost on the way: 8 Paperless documents failed (5 ordinary invoices among them), the Immich upload cut at photo 150, Nextcloud at 251 of 3 393 files. Cause NOT diagnosed (no shell in the box; memory read ~2.3 GB while load was 150 — not plain „memory full"); the one hint: the hub's `app_oom` at 13:34 — immich-machine-learning killed by its memory limit. Reproduced a SECOND time at 15:27:55 with only one Immich upload + one chunked Nextcloud upload running (~35 min, then froze again; plug pulled 16:13); per-app memory now logged every 30 s (`B4-use/memwatch.txt`). Evidence `audits/rehearsal-2026-10-20/journal.md` (12:56–14:23), `B4-use/box-metrics-after-plug.txt`, `B4-use/vm-counters-before-plug.txt`, `B4-use/after-plug-watch.txt`. | WAITING-ON-OPERATOR **DIAGNOSED 2026-10-21 with numbers, fix committed (agent `286a7e3`), proven by applying the same setting by hand** (`audits/fixes-2026-10-21/P4-freeze/`). Reproduced on demo-hp: a nested 8 GB VM, Tester4 from the public ISO, the same 8 apps, the same uploads (generator seed 20261020) → the dashboard dead 13:28:18–13:34:48 UTC. **Mechanism:** the guest's swap is capped at Proxmox's LXC default of 512 MB (the agent never set `swap`); Immich's analysis loaded its models (+1.15 GB, `immich-machine-learning` 70 → 1 224 MB) and the server grew (+0.4 GB) on top of 8 apps' 1.75 GB, so the 4096 MB guest filled (anon 3 585 MB, swap 511/512 MB) and could evict only file pages: ~360 MB left for ~25 containers' code → 150 000–2 700 000 refaults and up to 12 GB of reads per 10 s, memory „full" pressure 96 %, IO 97 %, load 162, 0 OOM kills; the host had 2.2 GB RAM and 7.4 GB swap free. **Experiment:** `pct set 9201 --swap 4096` live at 13:34:06 → „full" pressure 96 % → 1.7 % in 2 min, dashboard back 42 s later, uploads running on. **Proof (2 h, 13:43–15:43, swap = RAM by hand):** the same uploads repeated as fresh work (Paperless 110 cycles × 53 PDFs, Immich 111 new users × 306 photos, two Nextcloud trees): 720 samples, max „full" pressure 12 %, load 33, 0 limit hits, every dashboard probe answered (slowest 1.26 s). Controller 0.337.0: the false „Normál használat mellett ez nem okoz problémát." replaced. Nextcloud's small-file WebDAV upload ran ~17 files/min under this load — slow, not lost. | the operator's agent release + vouch | Release agent with the swap fix (swap = RAM at bring-up, at RAM resize, once at start); after the update read one box's `pct config` (swap = memory) and its `guest-swap: raised` log line; then close | operator |
 | **R-984** | Box system & updates | P4 | **Host and agent hardening, low risk each: the backup-target route has no role check; the unsigned guest spec has no size bounds; signed ops have no maximum validity; the guarded mkfs does not refuse LVs, zvols or RAID devices; the local API is reachable from the LAN by a static route; a sudoers inspect pattern is wider than needed; install passes secrets on the command line; guest settings files are world-readable; the host offers rpcbind and DNS on the LAN.** Fix each in its own small change. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN | — | Batch into hardening work | CC |
 | **R-982** | Box system & updates | P3 | **The Proxmox host accepts root password login over SSH from the LAN (measured on demo-hp), so anyone on the household network can try the root password.** Design-shaped: the break-glass password exists for the console. Fix: key-only SSH for root; the password stays for the console. Evidence: `audits/security-review-2026-10-12/evidence/demo-hp-host-reads.txt`. | OPEN — design-proposal | — | Decide; then set it in the installer and the config bundle | CC |
 | **R-979** | Box system & updates | P3 | **Controller updates are trusted by registry tag alone (no digest, no signature), and the Update button offers any newer tag, even an unvouched one; the built-in traefik, cloudflared and FileBrowser images are also pinned by tag only.** Whoever can push to the registry decides what runs. Fix: pin by digest from the hub's vouched list; offer only vouched versions. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN — design-proposal | — | Design note, then build | CC |
@@ -267,7 +262,7 @@ stopping line that lies.
 | **R-1020** | Monitoring & notifications | P4 | **`app_repaired_after_crash` reports a duration of 0 s, and the controller log says „in 0s".** `RunAfterUncleanBoot` sets `res.Duration` in a deferred func after the result is already returned (no named result), so the caller sees 0; the `app.yaml` record is right (10,172 ms on demo-hp 2026-10-14, `audits/immich-repair-2026-10-14/D/`). Small, but the controller release for this session is spent (v0.311.0). | **OPEN** — filed 2026-10-14 | — | CC: a named result (or set the duration before return), with a test on the returned value; ride the next controller release | CC |
 | **R-528** | Monitoring & notifications | P2 | **[P2-MEDIUM] Docker does not report an OOM kill inside a Felhom LXC guest: `OOMKilled` stays false and no `oom` event fires, so the v0.243.0 OOM line is not proven live.** MEASURED 2026-09-15 on scratch 9202 (Docker 29.8.0): Paperless capped at 128M restarted 11 times with `OOMKilled=false` and zero `docker events --filter event=oom`; a memory hog inside the running container was killed (rc 137) with the same silence (`E2-oom-signal-measure-9202.txt`). BIGNIGHT VM 333 did read `oomkilled=true`, so the shape differs by case. **Fix shape:** the agent reads the guest container cgroups' `memory.events oom_kill` counters (host-side, reliable), or the controller alarms on a restart-count trend **RE-MEASURED 2026-09-16 on the DRILL box (fresh install, nested VM 334, Docker in an LXC guest, controller 0.243.0), so the finding is not a property of one machine:** the Paperless webserver was capped at 128 M with `docker update --memory`; it restarted 9-10 times, and all three signals stayed silent - `OOMKilled=false` on every inspect, `docker events --filter event=oom` EMPTY for the whole window, the container's cgroup not visible from inside the guest, and `dmesg` unreadable there. Identical to scratch 9202. So the v0.243.0 OOM line cannot fire on ANY Felhom box as shipped, on either host. Evidence: `audits/evidence-drill-0243-2026-09-16/phase2-m1-oom.txt`. | **WATCHING** — **2026-10-12: the built part is released** (agent v0.150.0 carries the Docker-approval memory-kill check; the boxes run 0.154.0; hub 0.140.0+ requires a passing `oom_check` before a Docker set is approved). Per decision 157 nothing else is built; the row watches for a box whose Docker reports a false OOM flag. Dated look: 2026-11-12. **READY — rank P2-MEDIUM; owner: CC** **2026-09-17 (chaos night): an OOM WAS detected on a fresh box, and named precisely.** On `tester-1-022354` (controller 0.245.0, guest 9201, 6 GB RAM) immich’s Postgres was killed by the memory limit during its reverse-geocoding import, and the controller pushed `app_oom` (warning, operator-only): „Alkalmazás memóriája elfogyott: immich (immich-postgres) — egy folyamatát a memóriakorlát leállította” — naming the app AND the exact container. The visible consequence was `write CONNECTION_CLOSED immich-postgres:5432` and twelve restarts of immich-server. So on THIS box the OOM scan works and was the fastest route to the diagnosis; recorded here rather than filed as a new row. Evidence: `audits/evidence-chaos-night-2026-09-17/round-2.txt`. **2026-10-05 (burn-down night): a one-page design proposal (no code) is in `audits/night-burndown-2026-10-05/design-R-528.md`** — for the operator. **2026-10-06: STOPPED BEFORE ANY CODE — the measurement contradicted the design** (decision 155 said A then C). On scratch 9202, Docker 29.8.2, the `OOMKilled` flag was TRUE in all four shapes tried: a child process killed while the container kept running (`oom_kill` 0 → 3, flag true) and three main-process kills (exit 137, flag true); an `oom` event each time. The false flags of 2026-09-15 were on Docker 29.8.0. Cost read for the record: one exec read of 21 containers on demo-hp = 1.6 s. `audits/design-build-2026-10-06/`C/. **2026-10-06 18:24: operator ruling, option A (decision 157):** nothing is built; the row stays open as a WATCH for a box whose Docker reports a false flag. **Added:** a Docker engine set may not be approved until it reports a memory kill correctly on the boxes that ran it (built in this row's next session). **2026-10-06 (night): the Docker-approval memory-kill check is BUILT** — hub v0.140.0 (LIVE: the approval waits for a passing `oom_check` on every ring-0 box; a failed, errored or missing one blocks) and the agent's wrapper (`felhom-agent` `acccb66`, unreleased — ships as v0.150.0 after the 2026-10-07 read-back). Proven by hand on demo-hp's guest: `OOMKilled=true`, exit 137, the `oom` event — seen only with an events window ending after the run (the wrapper waits 2 s). `audits/readback-2026-10-07/`F/. | — | 2026-11-12: read the hub for any failed `oom_check` or an `app_oom` with no restart reason since 2026-10-12; none → close | CC |
 | **R-1199** | Monitoring & notifications | P2 | **The household's notification page shows disk-warning, drive-disconnected and backup-failed mails switched ON, but on the rehearsal day only the OPERATOR got them — the household's mailbox received one alarm (a failed off-site run) out of nine.** Rehearsal 2026-10-20, read from the Gmail catch-all (the second channel): to `tester4@` only the setup code and „A biztonsági mentés sikertelen!" (17:01); to `admin@` also `disk_warning` 89 % (18:36 — the box logged „notifying the customer"), `storage_disconnected` ×2, `storage_reconnected` ×2, `backup_run_failures` (3 of 8 apps), `app_oom`, `host/node_stale`. The box's page (`/settings/notifications`) has `event_disk_warning`, `event_storage_disconnected`, `event_backup_failed`, `event_crossdrive_failed` … ticked; the hub sends household mail only by ITS OWN stored prefs (`hub/internal/notify/dispatcher.go` `processCustomer`: `isEventEnabled(prefs.EnabledEvents, …)`) — whether the two lists disagree is not read (the token cannot read the customer record). Evidence `audits/rehearsal-2026-10-20/B6-failures/notifications-defaults.txt`, `journal.md`. | VERIFY | an operator read of Tester4's hub notification prefs and notification log | Compare the hub's `notification_prefs` / `notification_log` for Tester4 with the box's page; if they differ, make one the source of truth | CC |
-| **R-1193** | Monitoring & notifications | P3 | **When the box froze (R-1192), the operator heard only „no report for 45 m" — 28 minutes in — and the household heard nothing at all.** Measured 2026-10-20 against the operator mailbox (the second channel): the freeze began ~13:05 UTC; the hub page read ONLINE / OK at 13:22 (correct inside the 45-minute threshold, R-549); `host_stale` at 13:33, `app_oom` at 13:34 (immich-machine-learning killed by its memory limit), `host_crash_restart` at 14:26 and 16:29 (the two plug pulls), `node_stale` 15:59 / `node_recovered` 16:03. No mail or banner tells the household that its box is not answering, and no operator mail says „the box answers ping but not its dashboard". CORRECTED the same day: the first version of this row said the hub showed OK „throughout" and „nobody was told" — that read only the hub page, not the mailbox. Evidence `audits/rehearsal-2026-10-20/journal.md`. | OPEN | — | Decide whether a dashboard that does not answer for N minutes while the host still reports should alarm sooner and in its own words (guest-side liveness), and whether the household is told | CC |
+| **R-1193** | Monitoring & notifications | P3 | **When the box froze (R-1192), the operator heard only „no report for 45 m" — 28 minutes in — and the household heard nothing at all.** Measured 2026-10-20 against the operator mailbox (the second channel): the freeze began ~13:05 UTC; the hub page read ONLINE / OK at 13:22 (correct inside the 45-minute threshold, R-549); `host_stale` at 13:33, `app_oom` at 13:34 (immich-machine-learning killed by its memory limit), `host_crash_restart` at 14:26 and 16:29 (the two plug pulls), `node_stale` 15:59 / `node_recovered` 16:03. No mail or banner tells the household that its box is not answering, and no operator mail says „the box answers ping but not its dashboard". CORRECTED the same day: the first version of this row said the hub showed OK „throughout" and „nobody was told" — that read only the hub page, not the mailbox. Evidence `audits/rehearsal-2026-10-20/journal.md`. | OPEN | — | Design note (2026-10-21, after R-1192's diagnosis): the freeze's signature is cheap to see from the HOST side — the guest cgroup's memory PSI „full" above ~50 % for a minute, or its `memory.events max` climbing, while the host still reports; the agent could raise `guest_memory_thrash` (operator mail in its own words, not „no report for 45 m") and the household banner „A dobozod most nagyon lassú…". With the swap fix (R-1192) the measured thrash ends within a minute, so this alarm should be built only if a freeze is seen again after the agent release; until then it stays a design. Decide whether a dashboard that does not answer for N minutes while the host still reports should alarm sooner and in its own words (guest-side liveness), and whether the household is told | CC |
 | **R-1202** | Monitoring & notifications | P3 | **A box whose clock is behind blames the hub and stops an app without telling anyone.** Rehearsal 2026-10-20 B6 #8: RTC set 2 years back (the OS raised it to 2026-04-13): the dashboard shows only „Hub nem elérhető" while the log says „x509: certificate … not yet valid: current time 2026-04-13…"; Paperless crash-looped and the box stopped it with a restore hold; the `app_stopped_unhealthy` event failed (hub unreachable), so neither the household nor the operator heard. After the real time came back the app stayed stopped until a Start (clear sentence, 1 min). | **OPEN** | — | say „the box's clock is wrong (it says <date>)" when TLS fails with not-yet-valid/expired against a hub that answers; keep the stop event queued until the hub is reachable | CC |
 | **R-340** | Monitoring & notifications | P3 | **The new reachability check does not touch the surface that actually failed.** R-339 reports when the hub cannot READ ep0 — but the read it performs is the `usage` op, which is `proxmox-backup-manager` plus `df` over SSH, and therefore rides the **local API daemon**. The 2026-08-18 incident explicitly CLEARED that daemon: `proxmox-backup.service` was healthy throughout, and it was the **HTTPS proxy on 8007** that was wedged with a full accept queue. **So R-339's check would have returned green for all 9 h 37 m of that outage.** It closes the case where ep0 is unreachable *as a host*; it does not close the case that actually happened. This is not a defect in R-339 — it is the honest boundary of what it watches, recorded so a future reader does not mistake a green box gauge for a working off-site tier | **READY (M) — NEW 2026-08-18** | a tenantsync endpoint-script version bump (the op is added on ep0, so it needs the same version-gated rollout `ErrUsageUnsupported` already models) | Add a **health op** to `scripts/felhom-tenantsync.sh` that probes `https://127.0.0.1:8007/` on ep0 and reports the proxy's fd count and listen-queue depth, then surface it as a third signal. **Overlaps the connections spike (R-336's remaining half): both want the same observations from ep0, so whichever runs SECOND must reuse the first's evidence rather than re-measuring a protected machine twice**  **REUSE, per this row's own instruction — the connections spike ran FIRST (2026-08-20) and already produced most of what the health op wants; do not re-measure a protected machine a third time.** Available in `audits/evidence-ep0-established-connections-2026-08-20/`: the proxy **fd count** and its type breakdown (`lsof` + `/proc/<pid>/fd`), the **listen-queue depth** (`ss -lnt` — `Recv-Q 0`, `Send-Q 1024`), the **ESTAB/CLOSE-WAIT split**, the **per-peer** connection histogram, a **31-minute persistence diff** of full 4-tuples, and a **46.18 h** slope with Poisson bounds. What the health op would still add beyond these: a loopback `GET https://127.0.0.1:8007/` probe — the observation that distinguished "process problem" from "network problem" on 2026-08-18 and the one thing this spike did NOT take, because it is the surface R-339 cannot see. **And this spike sharpens what the op should report:** a rising **ESTAB** count is the live signal (CLOSE-WAIT was **0**, not merely flat), and per **R-344** the fd ceiling that matters may be the **agent's**, not only ep0's. | CC |
 | **R-388** | Monitoring & notifications | P3 | **PRODUCT DECISION (not a defect): the customer notification model is the wrong shape, and the settings page grows by one toggle per detector.** The operator's framing, recorded verbatim 2026-08-23: *"A customer should be notified only about things they can act on or are responsible for — the drive they unplugged, the storage they filled. **A failed backup is our incident, not theirs.** The intended shape is that we detect it, we tell them we noticed and are dealing with it, and they are not handed an error they cannot solve. The subscription should feel like being looked after, not like being on call."* Today's page is the opposite shape — one switch per detector, and it **grew from 12 to 15 in a single session** (one new alarm plus two compound toggles split into four). That growth is the argument, not an aside: a page that grows per detector keeps asking a household to make engineering decisions. | **READY** — **2026-10-12 evening: slice 1 delivered** (operator Q1/Q2 „as picked", `09` §3 decisions 234–235): the page shows three group switches; the per-detector switches and the cooldown sit under „Részletek"; an „egyedi" group is saved back unchanged; the default list fills groups 1 and 3 (households with no saved list), group 2 stays „egyedi" (`app_start_failed` off, R-329). Tests red-proved. controller 0.309.0, delivered 2026-10-12 18:08 to demo-hp, demo-felhom and Tester 1 (`controller-0.309.0-delivery.txt`). **READY** — **2026-10-12: the design is written** (`08` §8.1): three group switches („A gépem és a meghajtóim", „Az alkalmazásaim", „Amit mi intézünk") mapped onto today's 18 detectors; option A picked; slice 1 changes no wire. Two questions for the operator (group 3 default; the cooldown field). **READY** — operator ruling 2026-10-12 (S5, `09` §3 decision 200): design now (one page: options, a pick, the first slice), build in slices later. **OPEN — DIRECTION, operator's call** | a decision on scope; nothing here is a bug | Slice 2 (hub): group 3's mails say „we noticed X and are handling it" instead of the raw error | CC (+ operator: two answers) |
@@ -287,7 +282,7 @@ stopping line that lies.
 | **R-31** | Hub & operator | P3 | **[P2-HIGH] Offsite provisioning is synchronous with no status affordance.** Save runs the Hetzner sync in-request, so the request can hit the nginx 504 **while succeeding server-side**: the operator cannot tell failed from slow, and a retry races the first attempt. **MIGRATED FROM `ROADMAP.md` 2026-08-22 (R-369) — originally filed 2026-07-21, size M, roadmap state `idea`.** Moved verbatim; nothing added or reinterpreted. The roadmap keeps its copy as history, marked moved. | **OPEN — migrated from ROADMAP 2026-08-22, rank unchanged** **Re-ranked 2026-10-03: P2→P3: operator-only; a known workaround (click once, wait, verify) exists.** **2026-10-06 night: the race half fixed on felhom.eu main (hub, unreleased):** a second Save while the first still provisions is refused with 409 („already running — wait about a minute, then reload; do not save again"); nothing saved, nothing created; per customer, in memory. `TestProvision_R31_*`, red-proof `audits/night-burndown-2026-10-06/hub/R-31-red.txt`. LEFT: the async save with a status card (the escrow-card idiom). | — | Direction: make it async + a status card, reusing the proven **awaiting-card/poll idiom** (v0.138.0 escrow card). **Interim mitigation belongs in R-3 as an operator note: click once, wait, verify — do not re-click.** | CC |
 | **R-882** | Hub & operator | P3 | **Longhorn on DooPlex could not grow a volume online: its `instance-manager` (116 days up) called a host process that no longer existed** — `nsenter: cannot open /host/proc/196610/ns/mnt` on every expansion retry, and an offline growth was blocked by the expansion's own attachment ticket (found 2026-10-05 growing `hub-data` to 2 Gi). A restart of the instance-manager (operator-approved) fixed it: 77/77 volumes back `attached/healthy` in 110 s. **Why the cached PID went stale was not established** (likely a containerd/k3s or iscsid restart after the instance-manager started), so it will recur after the next such restart and stay invisible until a volume needs to grow. `audits/hub-db-offsite-2026-10-05/partA/step1-*.txt` | **OPEN** | — | Find which host process the PID was and whether Longhorn 1.10.x re-resolves it; until then, before growing any volume, check the instance-manager's age against the last k3s/containerd restart | operator |
 | **R-883** | Hub & operator | P3 | **8 DooPlex workloads run an image by a moving tag (`:latest` or none), so any pod restart is a silent upgrade.** Measured 2026-10-05: the Longhorn restart restarted zipline on `ghcr.io/diced/zipline:latest` (pull Always), which pulled 4.8.0; 4.8.0 refused its database (`cannot safely migrate from prisma to drizzle: expected migration 20260508022000 … was not applied`) and crash-looped. Fixed for zipline by pinning `4.7.0` (homelab-manifests `90f60e4`, `4c8ec7a`; 4.7.0 applied the four missing migrations; a dump from before is kept out-of-band). The other 7 were counted, not named or changed. **Checked from source 2026-10-05 (burn-down round 2):** homelab-manifests@87dfc29 still has moving tags (grep image lines without a numeric tag): admin-system/toolbox.yaml:12 nicolaka/netshoot:latest (a bare Pod); calibre-system/cwa.yaml:826 calibre-web-automated:dev; outline-system/outline.yaml:270 minio/minio:latest; tandoor-system/recipe-importer.yaml:26 gitea.dooplex.hu/admin/recipe-importer:latest (pull Always :27); adventurelog-system/adventurelog.yaml:100 and :256 adventurelog-backend/frontend:latest (pull Always :101,:257); jarrs-system/jarr- | **OPEN** | — | List the 7 (`kubectl get deploy,sts -A` images without a fixed tag), pin each to the running version, and let Renovate move them | operator |
-| **R-1183** | Hub & operator | P3 | **On Sunday 2026-10-25 (the clocks go back) every hub daily job after 02:00 runs TWICE: the alarm check at 04:00 and 05:00, the hub database snapshot twice (keep 2 → the previous day's copy pruned a day early), the prune and the key audit twice.** `hub/cmd/hub/main.go:1057-1062` adds 24 h to an absolute time; the controller fixed this (`scheduler.go:511-516`), the hub did not. PROVEN by replaying the hub's exact arithmetic (`audits/dataloss-hunt-2026-10-20/C3-T9-hub-dst/replay.txt`). On 2027-03-28 each runs once, an hour late. No architecture document mentions DST. Data-loss hunt 2026-10-20 (T9). | OPEN | the frozen run (no hub deploy unattended, decision 162) | Expect duplicate operator mails on Sunday morning; fix with the controller's day+1 `time.Date` and a DST test; deploy with the operator | CC |
+| **R-1183** | Hub & operator | P3 | **On Sunday 2026-10-25 (the clocks go back) every hub daily job after 02:00 runs TWICE: the alarm check at 04:00 and 05:00, the hub database snapshot twice (keep 2 → the previous day's copy pruned a day early), the prune and the key audit twice.** `hub/cmd/hub/main.go:1057-1062` adds 24 h to an absolute time; the controller fixed this (`scheduler.go:511-516`), the hub did not. PROVEN by replaying the hub's exact arithmetic (`audits/dataloss-hunt-2026-10-20/C3-T9-hub-dst/replay.txt`). On 2027-03-28 each runs once, an hour late. No architecture document mentions DST. Data-loss hunt 2026-10-20 (T9). | VERIFY **FIXED in hub v0.163.0, deployed 2026-10-21 11:46 UTC with the operator (D24 = yes):** `scheduleDaily` takes the calendar day after (`nextDailyRun`, `time.Date` day+1) and never repeats the slot that ran last; `TestR1183_*` for 2026-10-25 and 2027-03-28, red-proved against the old arithmetic (`04:00 job ran 2 time(s) on 2026-10-25`; the 02:00 snapshot also ran at 02:00 CEST and 02:00 CET). No other hub scheduler computes a wall-clock time. `05` §16.5 | Sunday 2026-10-25 has to happen | On 2026-10-26 read the hub log: every daily job (db-snapshot, prune, deadline-check, offsite-key-audit) logged ONE run on 2026-10-25, and `/data/snapshots` holds one file for that day; then close | CC |
 | **R-264** | Hub & operator | P4 | **Twenty-one facts the boxes report that the hub can now decode nowhere, each allowlisted with a reason rather than silently skipped — and for these the reason is "no consumer today, and one is arguably owed".** Split out of R-260 on 2026-08-08 so that closing the CLASS (gated) and fixing its sharpest instance (`operator_key_configured`) could not be mistaken for having decided what the hub should do with the rest. **The list, grouped by what a consumer would be for.** **(a) Guest-network health — `guest_net` and its seven children** (`checked_at`, `has_route`, `dhclient_alive`, `heal_succeeded`, `heals_last_hour`, `last_heal_at`, `damped`). The R-54 watchdog reports per-guest network state and self-heal counts every cycle and the hub — the component that emails the operator — models none of it. There is a live incident in this project's own record where a killed `dhclient` took a tunnel down for 1 h 15 m (`audits/INCIDENT-guest-dhclient-killed-2026-07-20.md`); a recurring-heal signal is exactly what would have surfaced it. **This is the strongest candidate of the twenty-one.** **(b) `selfupdate_pending` + `selfupdate_pending_version`** — an agent that has flipped its binary and never committed reports pending on every heartbeat so that "the operator sees WHY the version isn't advancing", and no operator can see it. **(c) `mgmt_plane.healed_recently`** — bounded: the hub DOES alarm on the `privsep_healed_at` timestamp beside it, so the recurring-clobber signal is not lost, only this flag. **(d) `restore_tests.mount_parity` + `mount_inventory`** — R-262's subject; the verdict is not lost (a mismatch fails before `Pass` is set) but the hub cannot tell a full-fidelity pass from a boot-only one. **(e) `pbs_dr.applied_at`.** **(f) Controller-side: `config_hash`, `reporting_disabled`, `stacks`, `storage.migrated_to`, `backup.last_db_dump`, `backup.last_integrity_check`** — the last two are backup-integrity timestamps, which is the "presence is not success" neighbourhood. **For each the question is the same and is NOT answered here: is it wanted? If the hub should act on it, model it and name what consults it. If it should not, the honest end is that the emitter stops sending it** — a fact emitted forever and consumed nowhere is a future false green waiting for someone to write a check against it. **Deliberately not decided in the G-1 session**, whose scope was the gate plus the operator-access instance; unilaterally removing emitters would also break the byte-identical cross-repo host-report golden and is a coordinated two-repo change. **⚠ DISPOSITIONS RECORDED 2026-08-13 — and the first thing to say is that they were NOT in this register before today.** The rulings were made on 2026-08-12; this row still read **READY — owner Viktor** and the gate's twenty entries still all said *"arguably owed"*, so a session told to *"re-read the dispositions from the register"* would have found none. They are written down now, which is the point of writing them down. **THE COUNT WAS ALSO WRONG:** this row says *twenty-one*; the gate's allowlist held **twenty**, measured. Twenty is the number the dispositions below account for, exactly. **(1) BUILD A READER — four groups, fourteen facts.** (a) guest-network health, (b) the staged-update-pending pair, (d) the restore-test depth pair, (f-part) the two backup-integrity timestamps. **(2) NO READER WANTED — five facts, now recorded as `not consumed, DELIBERATELY` with the ruling and its date** in `scripts/wire_contract_gate.py`, each with its own reason rather than a bare refusal: `mgmt_plane.healed_recently` (the hub already alarms on the timestamp beside it), `pbs_dr.applied_at` (`pbs_dr.state` is the verdict; the timestamp alone is the attempt-read-as-result trap), `config_hash` (the hub authors the config and knows its own generation), `stacks` (the app view is built from the purpose-built `app_telemetry` wire), `storage.migrated_to` (box-local bookkeeping with no hub-side intent to reconcile against). **The emitters are deliberately left alone** — removing one is a coordinated two-repo change and breaks the host-report golden; the honest end here is a recorded decision, not a deletion. The gate grew a THIRD entry kind to carry them, because an undecided fact and a decided one must not read alike. **(3) `reporting_disabled`, decided on its own merits: RECLASSIFIED `redundant`** — `health.status = "disabled"` travels in the same minimal report, is decoded into `reports.health_status`, and IS rendered. **The decision surfaced a real defect the flag would not have fixed: the staleness checker is age-only, so a deliberately-silent box still alarms → R-321.** **PROGRESS, 2026-08-13: the first reader is BUILT — guest-network health, R-319.** Its eight allowlist entries are **removed** (an allowlisted tag is skipped, so leaving them would have meant the new reader's fields were never checked); the gate's checked-tLine truncated
 | **R-451** | Hub & operator | P4 | **[P3-LOW] UPDATE ARC SLICE 7 — a fleet sweep: the operator can SEE, and MOVE, how far behind every box is.** Slices 1 and 2 make one box's state visible on that box's own pages. The operator has no fleet view, and **it is not derivable from what is already reported: the hub's report payload carries container name, state, CPU and memory, and NO image field at all** (spike §5, which is why Peti's box could only be recorded UNKNOWN). So this is a hub-side change as well as a controller one. Rank LOW today because the fleet is two enrolled boxes; it rises with the fleet. `architecture/09-update-architecture.md` §6, §8.4 **BOTH SIDES VERIFIED 2026-09-21, and it is cheaper than this row implies.** The controller's payload carries no image (`internal/report/types.go` L98–103) and the hub's `Store.SaveReport` (`hub/internal/store/store.go:965`) denormalises only container **counts** — but **the hub stores the raw report JSON whole**, so a new controller field lands there the day it is sent. What is missing is the denormalisation and the page, not the transport. Shape in `09` §6.2–6.3; the payload question is `09` §3b **Q7**. **-- RULED 2026-09-23 (`09` §3 decision 18):** the report carries, per compose service (database included), the installed reference, the catalog reference and the badge state. **Built later, when the fleet grows** — Q7's recommendation, confirmed. | **DEFERRED** (2026-10-03 triage: the row's verdict was finished, but it names open work no other row carries — the fleet sweep (report field, hub denormalisation, fleet page) is ruled and not built — deferred until the fleet grows (`09` §6.3)) — **RULED — build deferred until the fleet grows; owner: CC** | — | — | CC |
 | **R-814** | Hub & operator | P4 | `PBS-storage-1` (u629193, box 611421) still `status=active`, 19.9 MB | **VERIFY** (2026-10-03 triage: a July watch row with no id; given R-814. WAITING-ON-OPERATOR — no record found that the box was deleted.) — WAITING-ON-OPERATOR | operator console | Delete the box | operator |
@@ -361,6 +356,7 @@ stopping line that lies.
 | item | due (UTC) | what to measure |
 |---|---|---|
 | R-435 | 2026-10-27 | Tester 1's weekly off-site clean-up ran inside a hub window with no drop alarm (box log + mailbox); detail in the R-435 row |
+| R-1183 | 2026-10-26 | every hub daily job ran ONCE on 2026-10-25 (hub log + one snapshot file); detail in the R-1183 row |
 | R-1173 | 2026-10-23 | demo-hp's night with the off-site tier due: the R-921 pre-check line or a start with nothing in flight; detail in the R-1173 row |
 | R-1174 | 2026-10-27 | the status gate refuses from today: one conviction seen exiting 1 (a hand run on a copy of the page is enough); detail in the R-1174 row |
 | R-1050 | 2026-11-15 | HSTS one year if no https trouble since 2026-10-15; detail in the R-1050 row |

documentation/backlog/STATUS-HISTORY.md

@@ -7,6 +7,8 @@ Moved here byte for byte on 2026-10-14 (night session), newest first, so that `S
 
 ## Moved from STATUS on 2026-10-21
 
+**Updated 2026-10-21 (morning).** The 24-hour rehearsal with a pretend household finished: the first night, the clock change, a 3-day power-off and every restore path worked, but the 8 GB box froze twice on day one and, after a box death, the household could not get its off-site data back (R-1176, proven live). Report: `REPORT-rehearsal-2026-10-20.md`; ten stuck points, 29 new rows, nothing released.
+
 **Updated 2026-10-20 (day).** Hub 0.162.0, agent 0.161.0 (you vouched it) and installer 1.35.0 are live, and a failed off-site restore now puts the app back exactly as it was (your D20 = A; controller 0.336.0 is the floor). This page is one screen again: older news moved to the history file (`documentation/backlog/STATUS-HISTORY.md`), and nothing was deleted. A long rehearsal on a fresh box runs today and tonight.
 
 ## Moved from STATUS on 2026-10-20