night fixes 2026-10-26: live proofs (R-1220, R-1232, R-1187, R-1298), register states, design lines; R-1300 filed; R-528 design

Evidence audits/night-fixes-2026-10-26/ (scanned: 0 secret hits).

documentation/backlog/OPEN-ITEMS.md

@@ -193,7 +193,7 @@ stopping line that lies.
 | **R-1184** | Backup & restore | P3 | **A cut-off shared-folder restore can be „placed" and reported restored while files are missing; the download is then deleted.** Readiness is only „the scratch folder is not empty" (`internal/backup/shares_restore.go:131-140`) — the R-358 class fixed for apps, not for shares; 30-min download window (`web/offbox_handlers.go:716`); fixed 2 GiB free-space gate that fails open (`:107`). CONFIRMED in code. Data-loss hunt 2026-10-20 (C1 reviewer 1, R3). | OPEN | — | Fix: a completion marker after restic succeeds, as for apps | CC |
 | **R-1185** | Backup & restore | P3 | **A database dump that fails its own content check has already replaced last night's good dump, and the second drive copies it an hour later.** `appbackup/dbdump.go:388-422`: size check, rename over the good file (`:399`), then `ValidateDump` (`:419`). Volume tars are promoted with no content check either (`backup.go:902`). Tier 1 is one point per app by design (07 §6.1); the ORDER is the defect. PROVEN against production code (`audits/dataloss-hunt-2026-10-20/C3-D5-dump-replaced/`, a docker stub prints a header-only dump): `ValidateDump` says invalid („dump file too small"), `DumpOneTo` returns no error, and last night's good dump (with its `CREATE TABLE`) is gone from disk. Data-loss hunt 2026-10-20 (D5). | OPEN | — | Fix: validate the temp file before the rename; keep the last good dump on a failed check | CC |
 | **R-1186** | Backup & restore | P3 | **The nightly off-site copy is skipped silently when another backup holds the flag, and the catch-up never re-runs it.** `offbox.go:981-996` returns nil with an INFO line on the nightly path; `withLeg` marks the leg ended anyway (`cmd/controller/main.go:1246`). The 05:30 proof and 06:00 integrity check take the same flag, so a backup time near them collides every night. Only the hub's 48 h `offsite_stale` notices. CONFIRMED in code. Data-loss hunt 2026-10-20 (T5). | OPEN | — | Fix: mark the leg NOT ended on a busy skip so the catch-up retries; say it on the backups page | CC |
-| **R-1187** | Backup & restore | P2 | **A `.fab` export or import can lose or mix household data — code-confirmed paths from the data-loss hunt:** (a) an export silently drops a whole drive folder that holds a symlink and still ends „done” (`appexport/fabplan.go:292-310`, `export.go:712`; a FIFO also hangs the export on `os.Open`); (b) the import handles only folders and regular files: no owners set, symlinks dropped (`restore.go:1138-1183`); (c) **FIXED in controller main `d5f88a2`, unreleased (rides 0.340.0):** an import over an installed app that is not running merged the old data with the bundle's (`docker cp` never deletes) — reproduced on 9202 before the fix (stopped: a marker written after the export SURVIVED the import; running control: replaced), now refused with a Hungarian sentence and the app untouched; red-proved (`r1187c_test.go`), proven on 9202 rc.2 (`audits/night-2026-10-22/P3-candidates/r1187c-9202.txt`); (d) PLAUSIBLE: no restore-busy check in `StartImport`, no undo, no record (`restore.go:181-186`); (f) the bundle's HDD_PATH is trusted, an empty one gives a relative path (`restore.go:589-631`); (g) the DB import misses postgis images (adventurelog, dawarich: `classifyDBImage`) and runs psql without ON_ERROR_STOP (`restore.go:1047-1056, 933-943`; the dump is `--clean --if-exists`, so ON_ERROR_STOP is safe). (e) was R-1191. Research `audits/night-2026-10-22/P3-candidates/analysis.md`; first list `audits/dataloss-hunt-2026-10-20/C1-restore-rollback-candidates.md`. | OPEN — **2026-10-23: (c) and the C3/N9 refusal texts released in controller 0.340.0;** (a), (b), (f), (g) and the two import paths of C2 stay open. OPEN — **2026-10-22 night: re-ranked P3→P2** ((a) and (b) lose household files on a move to a new box with no error). (c) fixed in code; (a), (b), (f), (g) read in code, not live-proven. **2026-10-23 night — fresh review (`REPORT-night-2026-10-23.md`, C2, C3):** two more import paths still merge — a running app whose `RemoveStackVolumes` fails falls back to `StopStack` and goes on (`appexport/restore.go:375-380`), and `restoreHDDData` extracts drive-folder tars over the existing folder without clearing it (`restore.go:602-655`) for running and stopped apps alike. Both exist in 0.339.0 — not a hold for 0.340.0. The (c) refusal also catches an unhealthy or restarting app (`IsStackRunning`: running/degraded only); its text now says so and names removal (controller `943c517`). Recovery code: fix only with a red-proof + a 9202 proof. | — | Fix (a)+(b) with a red-proof and a 9202 proof (capture code — the night rule); (g) small fix with a 9202 postgis app; (d) after (a) | CC |
+| **R-1187** | Backup & restore | P2 | **A `.fab` export or import can lose or mix household data — code-confirmed paths from the data-loss hunt:** (a) an export silently drops a whole drive folder that holds a symlink and still ends „done” (`appexport/fabplan.go:292-310`, `export.go:712`; a FIFO also hangs the export on `os.Open`); (b) the import handles only folders and regular files: no owners set, symlinks dropped (`restore.go:1138-1183`); (c) **FIXED in controller main `d5f88a2`, unreleased (rides 0.340.0):** an import over an installed app that is not running merged the old data with the bundle's (`docker cp` never deletes) — reproduced on 9202 before the fix (stopped: a marker written after the export SURVIVED the import; running control: replaced), now refused with a Hungarian sentence and the app untouched; red-proved (`r1187c_test.go`), proven on 9202 rc.2 (`audits/night-2026-10-22/P3-candidates/r1187c-9202.txt`); (d) PLAUSIBLE: no restore-busy check in `StartImport`, no undo, no record (`restore.go:181-186`); (f) the bundle's HDD_PATH is trusted, an empty one gives a relative path (`restore.go:589-631`); (g) the DB import misses postgis images (adventurelog, dawarich: `classifyDBImage`) and runs psql without ON_ERROR_STOP (`restore.go:1047-1056, 933-943`; the dump is `--clean --if-exists`, so ON_ERROR_STOP is safe). (e) was R-1191. Research `audits/night-2026-10-22/P3-candidates/analysis.md`; first list `audits/dataloss-hunt-2026-10-20/C1-restore-rollback-candidates.md`. | **VERIFY — fixed on `main`: (a), (b), (d), (f), (g) + two import defects found by the live proof; waits for the controller release** — 2026-10-26 night: controller `dfd0f8e`, `106325d`, `f24233c`, `cccb9ae` (three rounds of the automated security review on the link handling: writes only through `os.Root`, links only downward), `a3ed63e` (the import's compose runs get the controller's path variables; a warning is never the container id), `90a2530` (the database loads through `appbackup.ImportDump` — every MariaDB 11+ import had timed out on `mysqladmin`). Live on 9202 (Grimmory: FIFO refused naming the path, owner/mode/inside link back, outside link refused; the import now succeeds and the app answers; AdventureLog: a postgis user back after the import) — `audits/night-fixes-2026-10-26/`. (d), (f): unit tests. Still open from the row: C2's two merge paths of an import over an existing app. **Earlier:** OPEN — **2026-10-23: (c) and the C3/N9 refusal texts released in controller 0.340.0;** (a), (b), (f), (g) and the two import paths of C2 stay open. OPEN — **2026-10-22 night: re-ranked P3→P2** ((a) and (b) lose household files on a move to a new box with no error). (c) fixed in code; (a), (b), (f), (g) read in code, not live-proven. **2026-10-23 night — fresh review (`REPORT-night-2026-10-23.md`, C2, C3):** two more import paths still merge — a running app whose `RemoveStackVolumes` fails falls back to `StopStack` and goes on (`appexport/restore.go:375-380`), and `restoreHDDData` extracts drive-folder tars over the existing folder without clearing it (`restore.go:602-655`) for running and stopped apps alike. Both exist in 0.339.0 — not a hold for 0.340.0. The (c) refusal also catches an unhealthy or restarting app (`IsStackRunning`: running/degraded only); its text now says so and names removal (controller `943c517`). Recovery code: fix only with a red-proof + a 9202 proof. | — | Fix (a)+(b) with a red-proof and a 9202 proof (capture code — the night rule); (g) small fix with a 9202 postgis app; (d) after (a) | CC |
 | **R-1188** | Backup & restore | P4 | **Restore-path candidates from the data-loss hunt — what is left is unlikely or a sentence:** (b) a failed `volume rm` is ignored before the untar, which then lands in the old volume (`backup/restore.go:218`; `StopStack` is `compose down`, so the rm rarely fails); (d) the full off-site download stops at 30 min per press and the household is not told to press again (`web/offbox_handlers.go:455`; the scratch is kept, so a second press probably continues — not measured); (e) place and full restore re-read the LATEST snapshot, not the downloaded one (`offbox_restore.go:675-679`, `_ = id`). Closed parts: (a) with R-1195 (controller 0.338.0), (c) in controller 0.337.0 (`TestR1188c_ALeftoverTempDoesNotBlockTheRetry`). Evidence `audits/dataloss-hunt-2026-10-20/C1-restore-rollback-candidates.md`, `audits/night-2026-10-22/P3-candidates/analysis.md`. | OPEN — **2026-10-22 night: re-ranked P3→P4**: (a) and (c) are shipped; the rest is low-chance or a sentence. | — | (d): measure a second press on a large download on 9202, then write the sentence; (b), (e): unit tests when the file is next touched | CC |
 | **R-1189** | Backup & restore | P3 | **Time and concurrency candidates from the data-loss hunt — what is left:** (a) a host clock ahead during one whole-guest backup silences that tier until real time passes it: the agent turns a future archive time into a negative age (`localapi/server.go:1179-1192`); half-live 2026-10-21 (B6 #8, `s8-wrong-clock/README.md`: a local archive dated 2028-10-21); agent code, so it rides an agent release and the operator's vouch; (c) the whole-guest quiesce waits for the catch-up and the update leg, not for a restore (`quiesce.go:213-275, 565`); (f) the OS leg and kernel reboot follow a daytime valve backup (agent `server.go:982-986`, `kernel.go:101-114`) — a design question; (g) spring forward 2027-03-28 moves 02:30 onto the 03:30 second copy; (h) self-update busy check before the pull only (`updater.go:532-545, 764`; not re-read); (i) a double-clicked restore shows failed while it runs (check and begin not atomic, `web/handlers.go:1860, 1881`). **Moved out 2026-10-22 night:** (b) the skipped nights after an off-cycle run were FIXED by R-899 (controller 0.304.0; demo-hp's journal shows a night run after each of three off-cycle runs since), the creep → R-1221; (d) **FIXED in controller main `d5f88a2`, unreleased (rides 0.340.0):** the catch-up now runs the off-site proof and the integrity check once after its legs, the false comment corrected; red-proved (`nightchain/r1189d_test.go`, `cmd/controller/r1189d_wiring_test.go`); (e) → R-1222 (hub). Evidence `audits/dataloss-hunt-2026-10-20/C1-time-concurrency-candidates.md`, `audits/night-2026-10-22/P3-candidates/analysis.md`. | OPEN — **2026-10-23: (d) released in controller 0.340.0;** (a), (c), (f), (g) stay as written. OPEN — 2026-10-22 night: (b) half closed, (d) fixed in code, (e) split out. | — | (a) with the next agent release (a unit test with a future archive); (c) a 9202 recipe; (g) a unit test with the 2027 clock; (f) to the operator at the next design review | CC |
 | **R-1190** | Backup & restore | P3 | **Key and retention candidates from the data-loss hunt — what is left:** (c) PLAUSIBLE: after a reuse-path reinstall the new PBS key K is never escrowed and nothing compares fingerprints (`felhom-pbs-apply:99`, `escrow_confirm.go:182-205`); (d) whole-guest off-site copies run before any escrow exists — a design question against `07` §6.1's Tier-3 gate (the operator's); (e) the escrow PUT is two statements; (f) the operator's off-site re-issue resets the provider password before saving it (hub `offsite.go:217-245`); (h) a unit restore can point an app back at an old drive and F5 then prunes the current drive's unit (`backup.go:1517-1574`) — (e), (f), (h) not re-read. **Moved out 2026-10-22 night:** (a) → R-1218, (b) → R-1219, (g) → R-1220. Evidence `audits/dataloss-hunt-2026-10-20/C1-keys-candidates.md`, `C1-delete-retention-candidates.md`, `audits/night-2026-10-22/P3-candidates/analysis.md`. | OPEN | — | (c) a read-only fingerprint compare on a reinstalled test box; (e), (f), (h) re-read, then unit tests; (d) to the operator at the next design review | CC |
@@ -201,14 +201,14 @@ stopping line that lies.
 | **R-1200** | Backup & restore | P3 | **A box installed in the afternoon gets no whole-box backup on its first night: the install's own backup starts the 24 h clock, so the family's whole first day reaches a whole-box copy only on the SECOND night (~38 h after install).** Rehearsal 2026-10-20 B5 (`audits/rehearsal-2026-10-20/B5-night/README.md`): Tester4's install backup at 12:09Z; at the 02:30–06:30Z window the agent answered „within cadence window" (`felhom-agent internal/localapi/server.go:1187`, `age >= tier.Cadence`, default 24 h). R-899 (closed) fixed the same shape for a daytime PRESS only. The other tiers did run that night (database copy, second drive, off-site — all PASS), so app data was covered; the whole-box (DR) tier was not. | **OPEN** — a design question: should the first scheduled night always run (like a press), or is one tier enough on night one? | — | decide; if „always run": treat the install backup like a press in the due rule, with a test on the first night | CC |
 | **R-1206** | Backup & restore | P3 | **Backups written while the box's clock ran ahead stay „the latest" after the clock is fixed.** The rehearsal's scenario #10 (clock set to 2026-10-28) left off-site snapshots dated 2026-10-28; on 2026-10-21 the rebuilt box's recovery page listed every app's „Legutóbbi mentés: 2026. 10. 28. 07:39", and the full restore said „mentés: 2026. 10. 28. 07:37" — a date in the future (`audits/fixes-2026-10-21/P1-live/07-unlock-right.txt`, `11-nextcloud-full-restore.txt`). restic orders snapshots by their own time, so the future ones are „newest" to every retention rule for a week. NOT measured: whether the box's keep-daily prune then drops a real snapshot. Related R-1202, R-1204 (clock). | NARROWED — the NAS tier only. **2026-10-22 night (read, `backup/offbox_window.go:22-24, 130-137`):** on the Felhom off-site tier no real snapshot is lost — the guard refuses a prune while any snapshot is future-dated (R-822, by design). The NAS tier prunes without that guard: open. **2026-10-25:** the kept-data choice no longer takes such a future-dated copy as the newest (R-1271, controller `bd9f4f7`); retention is unchanged — this row. | — | A pure unit test with the policy simulator on the NAS tier and one future-dated snapshot; if a real one is dropped, give the NAS tier the same guard | CC |
 | **R-1212** | Backup & restore | P4 | **Two household texts on the backup path do not follow the box's language:** the share-restore summary (`SharesRestoreResult.FlashMessage`, `controller/internal/backup/shares_restore.go`) is built from Hungarian Go literals, and the off-site `LastError` for an interrupted run is the Hungarian literal „megszakadt futás (a vezérlő újraindult futás közben)" (`controller/internal/backup/backup.go`). An English household reads Hungarian there. Found by the night helper 2026-10-21. | OPEN | — | Bundle keys for both (rendered in the box language when written, R-557), with parity fixtures | CC |
-| **R-1218** | Backup & restore | P2 | **A recovery ceremony done while off-site is switched off (or cleared) seals no off-site repository password, and the hub then demotes the GOOD older package to operator-only custody: the household's own recovery code no longer opens its off-site copy.** REAL in code (night 2026-10-22, read; split from R-1190 (a)): `web/escrow_handlers.go:217-224` stages the password only if `OffboxConfigured()` (`backup/offbox.go:695-710`), false when the target is cleared or disabled — although `ClearOffboxTarget` keeps the password file on purpose (`offbox_clear.go` header); the agent then seals no password (`cmd/felhom-agent/main.go:2981-3003`) and uploads sha ""; hub `SaveHostEscrow` sees `curSHA != ""` and moves the good package to retained custody; `OffsiteRecoveryOffer` reads "" as „covers no key” (`offbox.go:1855-1873`) and says nothing. Evidence `audits/night-2026-10-22/P3-candidates/analysis.md` §R-1190(a). | OPEN — proposed fix, not built (key-handling code: a fix needs a red-proof AND a live proof, and 9202 has no hub to prove it with) | — | Fix (~6 lines): stage when the password FILE exists (`PushOffboxPasswordForEscrow`, `offbox.go:800`, reads only the file). Red-proof: a web test that the stage runs with off-site cleared; a hub store test pinning „abc”→"" ⇒ superseded (and refuse that demotion). Live proof on a test box with a hub (demo-hp 9201) | CC |
-| **R-1219** | Backup & restore | P2 | **During a recovery offer the backups page hides the „create your recovery code” ceremony (it would replace the key that opens the old off-site copy), but the global top bar still links to it and the start handler does not refuse.** REAL in code (night 2026-10-22, read; split from R-1190 (b)): `web/escrow_banner.go:23-35` ignores `recoveryOffer()`; `templates/backups_remote.html:144-160` hides it and its own comment names the harm (R-199); `escrow_handlers.go:180-245` starts anyway; reachable in offer shape (c), `offbox.go:1866-1870`. Evidence `audits/night-2026-10-22/P3-candidates/analysis.md`. | OPEN — proposed fix, not built (key-handling code) | — | Fix: the bar hides its link during an offer, and the start handler answers 409 with a Hungarian sentence; red-proof with a handler test; live proof with an offer state on a test box with a hub | CC |
-| **R-1220** | Backup & restore | P2 | **„Megőrzött adatok” (kept data) can list an INSTALLED app's live folder as kept — and offer to delete it — when that app's app.yaml cannot be read.** REAL in code (night 2026-10-22, read; split from R-1190 (g)): a nil AppConfig drops the app from `liveDriveBinds` (`stacks/kept.go:220-235, 336-360, 372-389`); a parse error gives nil (`deploy.go:1113-1125`). Evidence `audits/night-2026-10-22/P3-candidates/analysis.md`. | OPEN — proposed fix, not built (delete-path code) | — | Fix: an app whose app.yaml is unreadable counts as live (fail closed); its folders are never listed as kept. Red-proof: a unit test with a broken app.yaml; 9202 proof with a throwaway app | CC |
+| **R-1218** | Backup & restore | P2 | **A recovery ceremony done while off-site is switched off (or cleared) seals no off-site repository password, and the hub then demotes the GOOD older package to operator-only custody: the household's own recovery code no longer opens its off-site copy.** REAL in code (night 2026-10-22, read; split from R-1190 (a)): `web/escrow_handlers.go:217-224` stages the password only if `OffboxConfigured()` (`backup/offbox.go:695-710`), false when the target is cleared or disabled — although `ClearOffboxTarget` keeps the password file on purpose (`offbox_clear.go` header); the agent then seals no password (`cmd/felhom-agent/main.go:2981-3003`) and uploads sha ""; hub `SaveHostEscrow` sees `curSHA != ""` and moves the good package to retained custody; `OffsiteRecoveryOffer` reads "" as „covers no key” (`offbox.go:1855-1873`) and says nothing. Evidence `audits/night-2026-10-22/P3-candidates/analysis.md` §R-1190(a). | **OPEN — fixed on `main` (both halves), red-proved; LIVE PROOF PENDING** — controller `1170b48` (stage when the password FILE exists) + hub `09b5f02f` (a keyed package is never replaced by a key-less one: 409). The live shape needs off-site switched OFF on a hub box: the household cannot switch a hub tier off (only per-app switches; a clear is refused by design), so it needs the operator's customer-page change on Tester 1 during Tuesday's attended block — otherwise both halves are REVERTED before the release (rule 2). **Earlier:** OPEN — proposed fix, not built (key-handling code: a fix needs a red-proof AND a live proof, and 9202 has no hub to prove it with) | — | Fix (~6 lines): stage when the password FILE exists (`PushOffboxPasswordForEscrow`, `offbox.go:800`, reads only the file). Red-proof: a web test that the stage runs with off-site cleared; a hub store test pinning „abc”→"" ⇒ superseded (and refuse that demotion). Live proof on a test box with a hub (demo-hp 9201) | CC |
+| **R-1219** | Backup & restore | P2 | **During a recovery offer the backups page hides the „create your recovery code” ceremony (it would replace the key that opens the old off-site copy), but the global top bar still links to it and the start handler does not refuse.** REAL in code (night 2026-10-22, read; split from R-1190 (b)): `web/escrow_banner.go:23-35` ignores `recoveryOffer()`; `templates/backups_remote.html:144-160` hides it and its own comment names the harm (R-199); `escrow_handlers.go:180-245` starts anyway; reachable in offer shape (c), `offbox.go:1866-1870`. Evidence `audits/night-2026-10-22/P3-candidates/analysis.md`. | **VERIFY — fixed on `main`, waits for the controller release** — controller `1170b48` (the top bar hides the ceremony during a recovery offer; the start handler answers 409 `api.escrow.recovery_offer_open`), red-proved with a real session and a no-offer control. The offer state cannot be staged without hand-set state, so the handler test is the proof (the brief's allowance). **Earlier:** OPEN — proposed fix, not built (key-handling code) | — | Fix: the bar hides its link during an offer, and the start handler answers 409 with a Hungarian sentence; red-proof with a handler test; live proof with an offer state on a test box with a hub | CC |
+| **R-1220** | Backup & restore | P2 | **„Megőrzött adatok” (kept data) can list an INSTALLED app's live folder as kept — and offer to delete it — when that app's app.yaml cannot be read.** REAL in code (night 2026-10-22, read; split from R-1190 (g)): a nil AppConfig drops the app from `liveDriveBinds` (`stacks/kept.go:220-235, 336-360, 372-389`); a parse error gives nil (`deploy.go:1113-1125`). Evidence `audits/night-2026-10-22/P3-candidates/analysis.md`. | **VERIFY — fixed on `main`, waits for the controller release** — 2026-10-26 night: controller `7e15a37` (an app whose `app.yaml` exists but does not parse counts as installed; its folders are never kept data), red-proved; live on 9202 (the kept-data page did not list the folder) — `audits/night-fixes-2026-10-26/`. **Earlier:** OPEN — proposed fix, not built (delete-path code) | — | Fix: an app whose app.yaml is unreadable counts as live (fail closed); its folders are never listed as kept. Red-proof: a unit test with a broken app.yaml; 9202 proof with a throwaway app | CC |
 | **R-1221** | Backup & restore | P4 | **The nightly whole-box backup starts ~1.5–5 min later each night, until one night is skipped (after ~70–80 nights with no press between).** REAL, low harm (night 2026-10-22; split from R-1189 (b)): agent `localapi/server.go:1184` is `age >= Cadence` with the age counted from the archive START; the controller polls every 5 min (`quiesce.go:150`) and the archive starts after the apps stop. Measured on demo-hp: 04:31:07 → 04:33:18 → 04:36:47 → 04:38:14, and 04:33:37 → 04:38:44 → 04:42:40. At W+6h one night is skipped (44 h is below the 48 h valve, `quiesce.go:807-818`); the hub's 26 h line (`deadline.go:29`) mails the operator. Any manual or off-cycle run resets the drift. Evidence `audits/night-2026-10-22/P3-candidates/demohp-whole-box-completions.txt`. | OPEN | — | Fix: inside the night window a tier is due at `age >= cadence - 4h`; the agent answers due-ness, so it is agent code (a unit test with an injected clock) and rides an agent release | CC |
 | **R-1228** | Backup & restore | P3 | **The website, the installer script and the hub are all served from the home line — a DooPlex loss stops new installs and the public site.** DooPlex survey 2026-10-23 night (`audits/dooplex-survival-2026-10-22/PLAN.md`, G8): `felhom.eu`, `hub.felhom.eu` and `gitea.dooplex.hu` resolve to the home address; the installer fetches `felhom.eu/scripts/felhom-host-install.sh`; only the ISO (Cloudflare R2) survives. | OPEN — **2026-10-23 — D35 ruled YES, after the release week (operator):** serve the website and the installer from Cloudflare like the ISO. Dated: start on or after 2026-10-30 (DUE-CHECKS). OPEN | the operator's yes | Operator decides (STATUS): A) accept — Gitea and the hub first, the site after; B) the static site + the installer tag on Cloudflare like the ISO — CC's pick, after the release week | CC (ruled yes 2026-10-23, D35) |
 | **R-1229** | Backup & restore | P4 | **ArgoCD's per-app auto-sync settings live only on the cluster, not in git.** DooPlex survey 2026-10-23 night (`audits/dooplex-survival-2026-10-22/PLAN.md`, G9): the Applications differ (`enabled` true/false, prune, selfHeal); a rebuilt cluster from `homelab-manifests` would not have them. | OPEN | — | Write each Application's sync policy into `homelab-manifests` (a homelab change, the operator's yes for the commit that changes live behaviour) | operator |
 | **R-1231** | Backup & restore | P4 | **The recovery page says the box's host service is „older” when the controller only cannot tell.** Test night 2026-10-23 (`P4-testnight/07a-unlock-refused-fixture-version.txt`): with the agent's capability `unknown` (a version string it could not compare), the page read „…a hozzá tartozó házon belüli szolgáltatás régebbi… A gép magától frissül”. The same page is reached by a merely-down agent, for which „older” and „updates itself” are not true. | OPEN | — | A separate sentence for `unknown` („a gép most nem éri el a házon belüli szolgáltatást — próbáld újra pár perc múlva”); a render test per state; rides a controller release | CC |
-| **R-1232** | Backup & restore | P2 | **A reinstalled box may take its first whole-box copy to ep0 up to a week late: the old box's copy in the same household space seems to count as its last one.** Test night 2026-10-23 (`audits/night-2026-10-23/P4-testnight/`): the new Tester4 box (guest 9201) had its PBS tier applied at 19:28 UTC, but no whole-box run to ep0 came all night; the only archive in the household's ep0 space is the rehearsal box's `ct/9201` of 2026-10-20 (same guest number), and the tier's cadence is 168 h. The agent's due-check reads the newest archive for that vmid on the tier's storage (`localapi/server.go` `newestArchiveOn`) — so the old box's copy reads as „2.6 days old, not due”. If so, the ep0 copy is the OLD box's for up to a week. PLAUSIBLE — read from the code and the night's silence, not proven by a log line (the controller logs the due-check only at debug). | OPEN | — | Prove: read `GET /backup/tiers` for the box (the agent's answer per tier) on the next reinstall; fix: a tier's „last backup” counts only archives this host wrote (or after the host's enrollment); an agent test with a foreign archive in the namespace | CC |
+| **R-1232** | Backup & restore | P2 | **A reinstalled box may take its first whole-box copy to ep0 up to a week late: the old box's copy in the same household space seems to count as its last one.** Test night 2026-10-23 (`audits/night-2026-10-23/P4-testnight/`): the new Tester4 box (guest 9201) had its PBS tier applied at 19:28 UTC, but no whole-box run to ep0 came all night; the only archive in the household's ep0 space is the rehearsal box's `ct/9201` of 2026-10-20 (same guest number), and the tier's cadence is 168 h. The agent's due-check reads the newest archive for that vmid on the tier's storage (`localapi/server.go` `newestArchiveOn`) — so the old box's copy reads as „2.6 days old, not due”. If so, the ep0 copy is the OLD box's for up to a week. PLAUSIBLE — read from the code and the night's silence, not proven by a log line (the controller logs the due-check only at debug). | **VERIFY — fixed on `main`, waits for the agent release (the operator's vouch)** — 2026-10-26 night: agent `3911cc6` (the due-check counts only archives with this storage's key, the R-727 discriminator), red-proved; live on a fresh BYO Tester4 box (VM 9375): the released agent said `due: false` (the old box's archive, 6.3 days), the fixed one `due: true / absent`, and the first whole-box copy ran 4 minutes later — `audits/night-fixes-2026-10-26/`. Hub host `Tester4-b5dbbd` to delete (operator). **Earlier:** OPEN | — | Prove: read `GET /backup/tiers` for the box (the agent's answer per tier) on the next reinstall; fix: a tier's „last backup” counts only archives this host wrote (or after the host's enrollment); an agent test with a foreign archive in the namespace | CC |
 | **R-1235** | Backup & restore | P3 | **The release copy on ep0 (D33) is proven to load, not yet to restore a registry — and it follows the newest versions, not the ones the fleet needs.** Built 2026-10-23 (`scripts/release-copy/`, `audits/day-2026-10-23/08-release-copy-D33.txt`). (a) `total-loss-of-dooplex.md` „Restore the release copy” steps 2 (skopeo push back, same digest) and 3 (agent packages back) were never walked into a real new registry — only a `docker load` on 9202. (b) The copy keeps the newest 3 image tags and 5 agent versions; it does not ask the hub for the floor and the vouched agent and always include them — a vouch more than 5 releases behind would fall out. (c) The golden (`felhom-golden`, 637 MB for 0.331.0) is not copied: a new install after a DooPlex loss needs it (the brief named images + agent packages only; one golden fits under the 3 GB cap). | OPEN | — | (a) a throwaway registry on demo-hp, push one tag back, compare digests; (b) read `/configuration`'s floor + vouched agent; (c) the operator says whether the golden goes in | CC |
 | **R-1263** | Backup & restore | P4 | **Small items from review round 2:** a LOCAL restore's put-back mails the operator „its off-site restore FAILED” (`felhom-controller/controller/cmd/controller/main.go:1514-1518`, shared via `restore_unit.go:517-518`); Remove — even „keep my data” — deletes undeclared volumes labelled with the app's project, which no tier holds (`internal/stacks/delete.go:763-764`, R-1117); the capped copy's `--delete` reads the source in the daemon's mount view, the emptiness check in the controller's (`tier2.go:834` vs `:464`); stale comments — `felhom-agent/internal/localapi/backup_cap.go:15` („keep-last”) and `restore_crash_putback.go:32` (names a test file that does not exist); the qBittorrent password goes to whatever answers the DNS name `qbittorrent` (`upwatch/explain.go:94,108-113`); kept-copy Remove has a check-to-remove gap (`tier2_kept.go:92-111`); the release-copy restore check trusts hashes stored on ep0 (`runbooks/total-loss-of-dooplex.md`); rrsync without `-munge`/`-wo` (`audits/day-2026-10-23/08-release-copy-D33.txt:6`). | OPEN | — | fix in each repo's next attended pass | CC |
 | **R-1294** | Backup & restore | P3 | **demo-hp's weekly off-site clean-up was refused by its own guard on 2026-10-26 (window 16): the plan would remove 209 of 359 snapshots, above the week's cap (179) — nothing was deleted, and each refused week adds to the next plan.** Seen in the operator mailbox during the 2026-10-26 release run (03:17Z, `offsite_prune_guard_refused`, error; details `count_before 359, count_after 359, outcome guard-refused`). The cap refusal is kept on purpose (R-824, R-867 closed the false ones); the open question is whether a store that once exceeds the cap can ever prune again by itself, or refuses every week while it grows. demo-hp is the Tier-0 box with many hand-made backups (sweeps, probes). Earlier cap refusals: demo-hp 2026-10-03 (window 1), demo-felhom 2026-10-05 (window 3). | **OPEN — the grant is SET (D55 = 260, `09` §3 decision 268)**: 2026-10-26 19:10:31Z through the operator's one-purpose token `cc-d55-window`, read back `max:260`; it is used at demo-hp's next window request (after its nightly off-site run, ~03:15Z). Close when that window prunes; then the operator revokes the token. **Earlier:** WAITING-ON-OPERATOR — 2026-10-26 evening: CC could NOT set the D54 grant — `POST /offsite/window-grant/demo-hp` (`max_remove=220`) with CC's hub token answered **403 „missing scope offsite:write"** (the hub log shows the refusal; nothing granted). The plan also grows: restic's dry-run 2026-10-26 18:56Z still 359 copies / plan 209 / default cap 179 (before tonight's run); a simulation of the keep rule adds ~14 per night (14 of 18 groups back up nightly), so ~223 at the night of 10-27 and ~237 at 10-28 — a cap of 220 would be refused again. STATUS D55: the operator grants it, with the cap. The grant opens the box's NEXT window request, which comes after its next nightly off-site run. Ruled D54 = prune once (`09` §3 decision 266). **Earlier:** WAITING-ON-OPERATOR — 2026-10-26 (read only, `audits/r1294-offsite-cap-2026-10-26/`): the plan is HONEST (restic's own dry-run: keep 150, remove 209; every removal 7–30 days old; every app keeps ≥ 9 or its last copies; the bulk is one test day, 2026-10-16, 152 snapshots). It can NOT recover by itself: the cap is half the store and grows ~49/week while the plan grows ~84/week. Only demo-hp has this shape (demo-felhom and Tester 1 pruned normally 10-26). The pick is STATUS D54. **Earlier:** OPEN | — | read the guard's cap rule (controller off-site prune) against demo-hp's snapshot list: is the plan honest (old, superseded) or not; if honest, say how such a store gets back under the cap (an attended one-time prune, or a cap that follows the store) — a design question for the operator if it changes what a box may delete | CC |
@@ -269,7 +269,7 @@ stopping line that lies.
 | **R-984** | Box system & updates | P4 | **Host and agent hardening, low risk each: the backup-target route has no role check; the unsigned guest spec has no size bounds; signed ops have no maximum validity; the guarded mkfs does not refuse LVs, zvols or RAID devices; the local API is reachable from the LAN by a static route; a sudoers inspect pattern is wider than needed; install passes secrets on the command line; guest settings files are world-readable; the host offers rpcbind and DNS on the LAN.** Fix each in its own small change. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN | — | Batch into hardening work | CC |
 | **R-979** | Box system & updates | P3 | **Controller updates are trusted by registry tag alone (no digest, no signature), and the Update button offers any newer tag, even an unvouched one; the built-in traefik, cloudflared and FileBrowser images are also pinned by tag only.** Whoever can push to the registry decides what runs. Fix: pin by digest from the hub's vouched list; offer only vouched versions. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN — design-proposal | — | Design note, then build | CC |
 | **R-974** | Box system & updates | P3 | **A compromised host agent can likely become root on the host through the root wrapper that applies PBS settings: the wrapper accepts a folder path inside the agent's own writable tree and then acts on it as root.** Static (the symlink behaviour was reasoned, not run). Same class as R-861. Fix: a fixed root-owned folder, and refuse symlinks before any root action. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN | — | Fix the wrapper with a test on a throwaway box | CC |
-| **R-972** | Box system & updates | P2 | **The shipped controller and hub programs are built with Go 1.24.13 and golang.org/x/crypto v0.31.0, and the agent with Go 1.26.0, so dozens of fixed Go security bugs (including html/template escaping fixes) are still in what runs on boxes and the hub.** Go 1.24 has had no security fixes since Go 1.26 was released (Go supports the two newest major releases: https://go.dev/doc/devel/release#policy); the scanners name fixes in Go 1.26.9 and 1.27.2. Also: the controller image's Debian 12 base has 12 fixable critical packages, and the Samba image has 48 fixable high ones. Fix: build all three on a supported, patched Go; update x/crypto; rebuild the base images. Scanner output: `audits/security-review-2026-10-12/scanners/`. | **NARROWED** 2026-10-16 (burndown 2): **controller v0.322.0 released** on Go 1.27.2 + x/crypto v0.58.0, runtime debian:12.13-slim + `apt-get upgrade` (Debian 13 deliberately not: restic 0.18, R-1105) — scanner: critical 16 → 4, fixable criticals and highs in the OS 0; govulncheck 62 → 1 (module-level, `openpgp` not linked). **Hub v0.156.0 deployed** 2026-10-16 19:26 on the operator's yes (Go 1.27.2, alpine 3.24.2: 0 critical, 0 high); Synced/Healthy, reports from all three boxes after the start, the floor reads 0.322.0. **Agent: Go 1.27.2 committed (`37b99d7`), tested, NOT released** — the next vouched agent release carries it. Open: the Samba image (48 fixable high, not rebuilt); docker-compose plugin's own Go (4 high, upstream). Evidence `audits/burndown2-2026-10-16/P4/` **2026-10-16 (evening): the Samba part is DONE in controller v0.323.0** — felhom-samba 1.2.0 on Alpine 3.24.2 (0 critical / 0 high, was 48 fixable high), proven on demo-hp with the same household password (smbclient, nmbd, mDNS, WS-Discovery); on the three test boxes now, on every box with the global floor (R-1105's night check). The controller's Debian 13 OS part: 1 critical / 70 high, none fixable. Left: docker-compose plugin's own Go (4 high, upstream); the agent release; restic's binary is R-1107. **2026-10-17 (evening): agent v0.159.1 (Go 1.27.2) RELEASED and running on demo-hp, Tester 1 and demo-felhom** (signed jobs on the operator's yes); waiting for the operator's vouch — then only the upstream compose plugin is left (close then, recorded). | the next vouched agent release | Close when the global floor carries v0.323.0 and the next vouched agent release carries Go 1.27.2 (compose plugin: upstream, recorded) | CC |
+| **R-972** | Box system & updates | P2 | **The shipped controller and hub programs are built with Go 1.24.13 and golang.org/x/crypto v0.31.0, and the agent with Go 1.26.0, so dozens of fixed Go security bugs (including html/template escaping fixes) are still in what runs on boxes and the hub.** Go 1.24 has had no security fixes since Go 1.26 was released (Go supports the two newest major releases: https://go.dev/doc/devel/release#policy); the scanners name fixes in Go 1.26.9 and 1.27.2. Also: the controller image's Debian 12 base has 12 fixable critical packages, and the Samba image has 48 fixable high ones. Fix: build all three on a supported, patched Go; update x/crypto; rebuild the base images. Scanner output: `audits/security-review-2026-10-12/scanners/`. | **NARROWED — Go 1.27.3 committed in all three, waits for the releases** — 2026-10-26 night: agent `fbbb168`, controller `2560cf1` (+ builder `golang:1.27.3-trixie`), hub `d625a143` (+ `golang:1.27.3-alpine3.24`); govulncheck 1 → 0 called vulnerabilities each (GO-2026-6809, `crypto/x509`); x/crypto v0.58.0 already newest; full tests green — `audits/r972-go1273-2026-10-26/`. **Earlier:** **NARROWED** 2026-10-16 (burndown 2): **controller v0.322.0 released** on Go 1.27.2 + x/crypto v0.58.0, runtime debian:12.13-slim + `apt-get upgrade` (Debian 13 deliberately not: restic 0.18, R-1105) — scanner: critical 16 → 4, fixable criticals and highs in the OS 0; govulncheck 62 → 1 (module-level, `openpgp` not linked). **Hub v0.156.0 deployed** 2026-10-16 19:26 on the operator's yes (Go 1.27.2, alpine 3.24.2: 0 critical, 0 high); Synced/Healthy, reports from all three boxes after the start, the floor reads 0.322.0. **Agent: Go 1.27.2 committed (`37b99d7`), tested, NOT released** — the next vouched agent release carries it. Open: the Samba image (48 fixable high, not rebuilt); docker-compose plugin's own Go (4 high, upstream). Evidence `audits/burndown2-2026-10-16/P4/` **2026-10-16 (evening): the Samba part is DONE in controller v0.323.0** — felhom-samba 1.2.0 on Alpine 3.24.2 (0 critical / 0 high, was 48 fixable high), proven on demo-hp with the same household password (smbclient, nmbd, mDNS, WS-Discovery); on the three test boxes now, on every box with the global floor (R-1105's night check). The controller's Debian 13 OS part: 1 critical / 70 high, none fixable. Left: docker-compose plugin's own Go (4 high, upstream); the agent release; restic's binary is R-1107. **2026-10-17 (evening): agent v0.159.1 (Go 1.27.2) RELEASED and running on demo-hp, Tester 1 and demo-felhom** (signed jobs on the operator's yes); waiting for the operator's vouch — then only the upstream compose plugin is left (close then, recorded). | the next vouched agent release | Close when the global floor carries v0.323.0 and the next vouched agent release carries Go 1.27.2 (compose plugin: upstream, recorded) | CC |
 | **R-1214** | Box system & updates | P3 | **On the 8 GB box with the swap fix, the night's second-drive backup still pushes memory „full" pressure above 20 % for about 3 minutes; the box keeps answering.** Night soak 2026-10-21→22 (`audits/night-2026-10-21/P1-soak/03-rec.jsonl`, 10-s samples; fresh box from the public ISO, agent with swap = RAM, 8 apps, the rehearsal uploads for 2 h): at 01:30–01:34 UTC (the 03:30 tier2 job) host 39 % / guest 45 % over 10 s, over 60 s at most 20 % / 25 %; IO „full" 66 %; guest swap 1.35 → 2.1 GB; `/api/health` answered every probe (slowest 4.1 s, largest interval 16 s); 0 OOM kills. Once more in the upload window (Immich loading its models, 31 % / 35 % for one sample). The brief's soak pass line was „full" < 20 %, so the soak missed that one line; the freeze itself (96 % for an hour, R-1192) did not come back. Not diagnosed: which part of the tier2 job (the copy's page cache, the volume dumps) causes it. | WATCHING — **2026-10-24 02:4xZ dated check read (weekend-night session, read only):** Tester 1 (8 GB host, guest 4 GB, controller 0.341.0) ran its 03:30 copy at 01:30:00–01:30:13Z (3 apps, 157 MB + 86 MB + 16 KB; no „capped copy container did not start" WARN, no leftover `felhom-tier2-copy-*`); the guest cgroup's memory „full" stall counter on the host (`/sys/fs/cgroup/lxc/9201/memory.pressure`) read **total 444 µs** over the whole 11.1 h since the host started (15:32Z), so „full" stayed far under 20 % (20 % of 10 s = 2 s); 0 OOM kills. **Not shown:** that the copy ran in the capped container (the INFO log does not name it; Docker keeps only ~200 events, from 02:36Z). **Weak load:** 0.24 GB, not the soak's large copy — so the row stays WATCHING; close after a night whose copy moves ≥ 1 GB, read from the same counter. WATCHING — **RELEASED 2026-10-23 in controller 0.340.0** (floor 06:41 UTC; on all three test boxes, start-up „no leftover”, 0 copy containers; Tester 1's manual second-drive run: 3 apps, 0 containers left). **Open only for the consequence:** dated check 2026-10-24 — Tester 1's first night copy ran in the capped container and memory „full” pressure stayed under 20 % (the box's own night log). READY — rides controller 0.340.0. READY — waits for the controller 0.340.0 release. **2026-10-22 night — cause measured, fix built and proven on 9202, waits for the controller release.** Cause: the copy's page cache is charged to the guest and evicts the apps' working set. On 9202 (4 GB RAM, 1.8 GB held by a hog, a 6 GB copy): plain rsync full stall 5 702 / 2 464 ms, peak „full” 16.8 / 6.9 %; the same copy in a 512 MB memory-capped container 374 / 320 ms, 0.9 / 1.0 %. Controller main `03c0a69` + `ae39467` (unreleased, rides 0.340.0): the copy runs capped only for clean `/mnt/` paths with no symlink (security review); a docker error (exit 125) falls back to plain rsync. A real `POST /api/backup/tier2` on rc.4: 6 capped containers, each exit 0; Immich's copy equals its source (dry-run 0 differences, 44 = 44 files). Evidence `audits/night-2026-10-22/P4-R1214/`. **2026-10-23 night (stray-copy fix, night brief Part 1).** The capped copy's container outlived the copy (a timeout or a controller restart killed only the `docker` CLI; `--delete` went on outside the R-1180 guard). Controller main `66a69d4` + `912e592` (unreleased, rides 0.340.0): a fixed name + label per copy, removed by name on any error, every leftover removed at start before the scheduler and the web server, a restore refused while a copy runs or may run; `--mount` (a path the daemon does not see is refused, not created empty) and the image ID with `--pull never` (review C4, C7). Proven on 9202 with rc.5 (`audits/night-2026-10-23/P1-straycopy/`): a restart in the middle of a 16 GB copy → the new controller removed `felhom-tier2-copy-immich-948f2280` at start, the restore probe was refused during the copy, the redone copy equals the source; a 15 s test build → the timed-out copy's container removed, the partial copy stopped growing. A second review of that code (N2–N5) and the commit's own security review (two races) → `9d9e404`, `1babfe9`: a stray clears when a later removal works, the start-up check never removes this process's own copy and holds the destination's lock, one copy per destination at a time, `RunTier2` refuses during a restore for every caller and records itself in one step with that check, every leg asks again. Re-proven on 9202 with rc.8 (`audits/night-2026-10-23/P1-straycopy/1*`). 9202 back on 0.339.0. | — | Release 0.340.0; then re-measure the 03:30 job on the 8 GB soak box and close if „full” stays low | CC |
 | **R-862** | Box system & updates | P3 | **Tester 2 cannot take the config bundle until one by-hand bootstrap is done: its `felhom-os-apply` (agent 0.142.0) predates the bundle mode, and no signed job can write a root file on it.** FOUND 2026-10-04 (R-840 build, Part C): the route reaches every box installed from installer 1.31.0 on, and the demo boxes (bootstrapped by CC); Tester 2 has every root file of agent 0.142.0 (installer 1.30.0) and lacks only the R-858 wrapper fix, which matters only for a Docker step it gets solely from a signed job. CC has no route to Tester 2 (its door admits only the operator's WireGuard peer; `felhom-op` cannot become root). The steps: `runbooks/config-bundle.md` "Tester 2". Then CC signs the bundle and reads it back. | **WAITING-ON-OPERATOR** — the operator said (2026-10-04 ~19:05) he will try through his tunnel **2026-10-07 07:58 (`09` §3 decision 170): the operator believes Tester 2 never set up the off-site backup (no escrow); the laptop may stay off for days; the tester plans to add an HDD.** **2026-10-07 (Part H, read only):** in Tester 2's last 10 notifications (2026-10-05 09:16 → 2026-10-07 06:58) the household got 0 mails, the operator 10 (host_down ×9, expected_dbdump_missed ×1); control: the household channel shows on the other three customers. So the household is not mailed daily while the box is off; no row (`audits/day-2026-10-07/H/`). **2026-10-22: closes when Tester 2 is reinstalled (operator ruling 2026-10-22 14:11 — a new install from the public download replaces the bootstrap and the off-site check; `pilot/TESTER2-handover.md`).** | the operator's WireGuard tunnel | the operator runs the three bootstrap commands; CC sends the bundle | operator |
 | **R-78** | Box system & updates | P3 | **`local_api` authority ruling — auto-reconcile vs detect-only** **MIGRATED FROM `ROADMAP.md` 2026-08-22 (R-369) — originally filed 2026-07-25, size M, roadmap state `idea (deferred OUT of R-77 on purpose)`.** Moved verbatim; nothing added or reinterpreted. The roadmap keeps its copy as history, marked moved. **An OWED OPERATOR DECISION, not a defect — moved because an owed ruling hidden among feature ideas is the shape this session exists to remove.** | **OPEN — migrated from ROADMAP 2026-08-22, rank unchanged** | — | R-77 ships detection because the fix is genuinely undecided, and **both directions can lose customer-visible function**. **Direction 1 (today):** `controller.yaml` wins and drift is silent → the 2026-07-25 island migration blinded the whole fleet's control plane for 17.5 h (drive gate, guest-reboot recovery, quiesce/backup all degrade). R-77 makes that loud but does not stop it recurring. **Direction 2 (`bootstrap.json` wins, auto-reconcile on boot):** a guest whose `controller.yaml` is CORRECT and whose `bootstrap.json` is stale — a half-completed re-provision, a hand-repaired guest, a setup-wizard box — gets a **working channel clobbered on the next restart**, fleet-wide and silently, during a routine deploy. That is not obviously better than the bug. Needs a spike: which writer is authoritative per field (endpoint vs fingerprint vs token — `mergeLocalAPI` replaces the whole block, so they cannot be reconciled independently today), whether the agent side should stamp a generation/mtime so 'newer wins' is even expressible, and whether reconcile should require an operator ack. Until then the drift alert plus a manual edit is the supported path. | CC |
@@ -287,7 +287,7 @@ stopping line that lies.
 | **R-1261** | Monitoring & notifications | P3 | **The R-1250 line hold (committed, NOT released): three edges before it ships (review round 2, HUNT-4 H4–H6).** Two seeders behind one uplink: the hold takes only the first, so the test is refused „not quiet” every 15 min in the Tuesday window and never counted (`felhom-controller/controller/internal/upwatch/linehold.go:155-160`; agent `internal/linetest/linetest.go:379-381`); a hold record of an app removed while held is never cleared and the 50 % rule never runs again (`linehold.go:106-108`; `upwatch/service.go:149`); a qBittorrent alt-speed flip during the hold leaves 1 KiB/s stored (`explain.go:175`; `linehold.go:127-128`). | **VERIFY** — RELEASED in controller v0.344.0 (global floor 2026-10-26 09:07:47Z, every live box on 0.344.0); the live test with a seeding box is Tuesday 10:00–16:00 (R-1250's dated check). **Earlier:** **VERIFY** — fixed on main 2026-10-25 night (controller `a79a292`), waits for the controller release (before any household gets agent 0.165.0 by signed update): every running seeder held, a removed app's record dropped, qBittorrent's slot named (`up_limit`/`alt_up_limit`). Red-proofs `audits/night-2026-10-25/redproof-r1261.txt`. **Live on the night's rig** (BYO VM, agent 0.165.0, rc.1, two seeders): both held, 94.8 Mbit/s, both limits back; a speed-mode switch mid-test left no 1 KiB/s (`rig/hold-watch*.txt`); the removed-app case by test only. **Earlier state:** OPEN — **now in controller 0.343.0 (released 2026-10-25); the three edges act only on a box whose agent is ≥ 0.165.0 (it alone reports `waiting_for_quiet`) — fix before any household gets agent 0.165.0 by signed update** | — | before the next controller and agent release: hold every running seeder; clear a record whose app is gone; read the mode | CC |
 | **R-1262** | Monitoring & notifications | P3 | **The upload watch treats ANY viewer as an explanation for any amount of upload, so a hacked Plex can send freely while one family member watches.** Found 2026-10-24 (SEC-1 X1): `felhom-controller/controller/internal/upwatch/watch.go:275,299`; `08` §9 states the rule, its limits list did not name this case. | OPEN | — | propose a cap (viewers × a bitrate ceiling) for the operator's R-1244 design; `08`'s limits list names the case meanwhile | CC |
 | **R-1020** | Monitoring & notifications | P4 | **`app_repaired_after_crash` reports a duration of 0 s, and the controller log says „in 0s".** `RunAfterUncleanBoot` sets `res.Duration` in a deferred func after the result is already returned (no named result), so the caller sees 0; the `app.yaml` record is right (10,172 ms on demo-hp 2026-10-14, `audits/immich-repair-2026-10-14/D/`). Small, but the controller release for this session is spent (v0.311.0). | **OPEN** — filed 2026-10-14 | — | CC: a named result (or set the duration before return), with a test on the returned value; ride the next controller release | CC |
-| **R-528** | Monitoring & notifications | P2 | **[P2-MEDIUM] Docker does not report an OOM kill inside a Felhom LXC guest: `OOMKilled` stays false and no `oom` event fires, so the v0.243.0 OOM line is not proven live.** MEASURED 2026-09-15 on scratch 9202 (Docker 29.8.0): Paperless capped at 128M restarted 11 times with `OOMKilled=false` and zero `docker events --filter event=oom`; a memory hog inside the running container was killed (rc 137) with the same silence (`E2-oom-signal-measure-9202.txt`). BIGNIGHT VM 333 did read `oomkilled=true`, so the shape differs by case. **Fix shape:** the agent reads the guest container cgroups' `memory.events oom_kill` counters (host-side, reliable), or the controller alarms on a restart-count trend **RE-MEASURED 2026-09-16 on the DRILL box (fresh install, nested VM 334, Docker in an LXC guest, controller 0.243.0), so the finding is not a property of one machine:** the Paperless webserver was capped at 128 M with `docker update --memory`; it restarted 9-10 times, and all three signals stayed silent - `OOMKilled=false` on every inspect, `docker events --filter event=oom` EMPTY for the whole window, the container's cgroup not visible from inside the guest, and `dmesg` unreadable there. Identical to scratch 9202. So the v0.243.0 OOM line cannot fire on ANY Felhom box as shipped, on either host. Evidence: `audits/evidence-drill-0243-2026-09-16/phase2-m1-oom.txt`. | **WATCHING** — **2026-10-12: the built part is released** (agent v0.150.0 carries the Docker-approval memory-kill check; the boxes run 0.154.0; hub 0.140.0+ requires a passing `oom_check` before a Docker set is approved). Per decision 157 nothing else is built; the row watches for a box whose Docker reports a false OOM flag. Dated look: 2026-11-12. **READY — rank P2-MEDIUM; owner: CC** **2026-09-17 (chaos night): an OOM WAS detected on a fresh box, and named precisely.** On `tester-1-022354` (controller 0.245.0, guest 9201, 6 GB RAM) immich’s Postgres was killed by the memory limit during its reverse-geocoding import, and the controller pushed `app_oom` (warning, operator-only): „Alkalmazás memóriája elfogyott: immich (immich-postgres) — egy folyamatát a memóriakorlát leállította” — naming the app AND the exact container. The visible consequence was `write CONNECTION_CLOSED immich-postgres:5432` and twelve restarts of immich-server. So on THIS box the OOM scan works and was the fastest route to the diagnosis; recorded here rather than filed as a new row. Evidence: `audits/evidence-chaos-night-2026-09-17/round-2.txt`. **2026-10-05 (burn-down night): a one-page design proposal (no code) is in `audits/night-burndown-2026-10-05/design-R-528.md`** — for the operator. **2026-10-06: STOPPED BEFORE ANY CODE — the measurement contradicted the design** (decision 155 said A then C). On scratch 9202, Docker 29.8.2, the `OOMKilled` flag was TRUE in all four shapes tried: a child process killed while the container kept running (`oom_kill` 0 → 3, flag true) and three main-process kills (exit 137, flag true); an `oom` event each time. The false flags of 2026-09-15 were on Docker 29.8.0. Cost read for the record: one exec read of 21 containers on demo-hp = 1.6 s. `audits/design-build-2026-10-06/`C/. **2026-10-06 18:24: operator ruling, option A (decision 157):** nothing is built; the row stays open as a WATCH for a box whose Docker reports a false flag. **Added:** a Docker engine set may not be approved until it reports a memory kill correctly on the boxes that ran it (built in this row's next session). **2026-10-06 (night): the Docker-approval memory-kill check is BUILT** — hub v0.140.0 (LIVE: the approval waits for a passing `oom_check` on every ring-0 box; a failed, errored or missing one blocks) and the agent's wrapper (`felhom-agent` `acccb66`, unreleased — ships as v0.150.0 after the 2026-10-07 read-back). Proven by hand on demo-hp's guest: `OOMKilled=true`, exit 137, the `oom` event — seen only with an events window ending after the run (the wrapper waits 2 s). `audits/readback-2026-10-07/`F/. | — | 2026-11-12: read the hub for any failed `oom_check` or an `app_oom` with no restart reason since 2026-10-12; none → close | CC |
+| **R-528** | Monitoring & notifications | P2 | **[P2-MEDIUM] Docker does not report an OOM kill inside a Felhom LXC guest: `OOMKilled` stays false and no `oom` event fires, so the v0.243.0 OOM line is not proven live.** MEASURED 2026-09-15 on scratch 9202 (Docker 29.8.0): Paperless capped at 128M restarted 11 times with `OOMKilled=false` and zero `docker events --filter event=oom`; a memory hog inside the running container was killed (rc 137) with the same silence (`E2-oom-signal-measure-9202.txt`). BIGNIGHT VM 333 did read `oomkilled=true`, so the shape differs by case. **Fix shape:** the agent reads the guest container cgroups' `memory.events oom_kill` counters (host-side, reliable), or the controller alarms on a restart-count trend **RE-MEASURED 2026-09-16 on the DRILL box (fresh install, nested VM 334, Docker in an LXC guest, controller 0.243.0), so the finding is not a property of one machine:** the Paperless webserver was capped at 128 M with `docker update --memory`; it restarted 9-10 times, and all three signals stayed silent - `OOMKilled=false` on every inspect, `docker events --filter event=oom` EMPTY for the whole window, the container's cgroup not visible from inside the guest, and `dmesg` unreadable there. Identical to scratch 9202. So the v0.243.0 OOM line cannot fire on ANY Felhom box as shipped, on either host. Evidence: `audits/evidence-drill-0243-2026-09-16/phase2-m1-oom.txt`. | **WATCHING — DESIGN WRITTEN 2026-10-26 night (the brief's Part 7: build only if time is left; it was not built — the line test and the attended release came first).** The host-side counter EXISTS and is readable (read on demo-hp 2026-10-26): `/sys/fs/cgroup/lxc/<vmid>/ns/system.slice/docker-<64-hex id>.scope/memory.events`, line `oom_kill N`. **Design:** (1) agent: a read-only collector (no root CLI — a sysfs read under the agent's own rights; if not readable, the existing sudoers fence gets one read-only `Cmnd_Alias`) that, for each guest in the `felhom` pool, globs those scopes and keeps `{container_id: oom_kill}`; served on the guest's own local API as `GET /guest/oom-kills` (`withGuest`, the guest's containers only). (2) controller: the 30-s dead-app scan asks once per pass; a RISE of a container's counter = an OOM kill whatever Docker's `OOMKilled` says → the existing `app_oom` line (R-636's per-run grain), `source: cgroup` in the details. An agent without the route = today's behaviour. (3) Tests: a fake cgroup tree (agent), a rising counter → one `app_oom` (controller, red-first). (4) Live: 9202, a throwaway app capped at 64M with a memory hog: the counter rises, `app_oom` names the container while `OOMKilled` reads false. ~1 day, two repos, one agent release. **Earlier:** WATCHING — **2026-10-12: the built part is released** (agent v0.150.0 carries the Docker-approval memory-kill check; the boxes run 0.154.0; hub 0.140.0+ requires a passing `oom_check` before a Docker set is approved). Per decision 157 nothing else is built; the row watches for a box whose Docker reports a false OOM flag. Dated look: 2026-11-12. **READY — rank P2-MEDIUM; owner: CC** **2026-09-17 (chaos night): an OOM WAS detected on a fresh box, and named precisely.** On `tester-1-022354` (controller 0.245.0, guest 9201, 6 GB RAM) immich’s Postgres was killed by the memory limit during its reverse-geocoding import, and the controller pushed `app_oom` (warning, operator-only): „Alkalmazás memóriája elfogyott: immich (immich-postgres) — egy folyamatát a memóriakorlát leállította” — naming the app AND the exact container. The visible consequence was `write CONNECTION_CLOSED immich-postgres:5432` and twelve restarts of immich-server. So on THIS box the OOM scan works and was the fastest route to the diagnosis; recorded here rather than filed as a new row. Evidence: `audits/evidence-chaos-night-2026-09-17/round-2.txt`. **2026-10-05 (burn-down night): a one-page design proposal (no code) is in `audits/night-burndown-2026-10-05/design-R-528.md`** — for the operator. **2026-10-06: STOPPED BEFORE ANY CODE — the measurement contradicted the design** (decision 155 said A then C). On scratch 9202, Docker 29.8.2, the `OOMKilled` flag was TRUE in all four shapes tried: a child process killed while the container kept running (`oom_kill` 0 → 3, flag true) and three main-process kills (exit 137, flag true); an `oom` event each time. The false flags of 2026-09-15 were on Docker 29.8.0. Cost read for the record: one exec read of 21 containers on demo-hp = 1.6 s. `audits/design-build-2026-10-06/`C/. **2026-10-06 18:24: operator ruling, option A (decision 157):** nothing is built; the row stays open as a WATCH for a box whose Docker reports a false flag. **Added:** a Docker engine set may not be approved until it reports a memory kill correctly on the boxes that ran it (built in this row's next session). **2026-10-06 (night): the Docker-approval memory-kill check is BUILT** — hub v0.140.0 (LIVE: the approval waits for a Line truncated
 | **R-1250** | Monitoring & notifications | P3 | **A qBittorrent that seeds all day is never line-tested, so it never gets the limit that exists for it.** The weekly test (R-1247) starts only after 10 minutes in which the host's uplink sent < 2 Mbit/s each minute; a seed at the line's speed keeps it busy through every window, so no test, no tested line, and no qBittorrent limit (D41 uses the tested line only). Found 2026-10-24 by reading the rule against the case it serves; not yet seen on a box (no test box seeds). | **VERIFY — released 2026-10-25, the live proof is a dated check:** controller 0.343.0 (global floor 17:53Z) + agent 0.165.0 (vouched 18:38 CET) carry it; the real proof cannot run before the next due window — the agent starts a test only on its due Tuesday 10:00–15:59 local, and every box measured on 2026-10-24. **DUE 2026-10-27:** a throwaway qBittorrent on demo-hp seeding to DooPlex's qBittorrent (operator's yes 2026-10-25, about 30 minutes), from before the host's slot; read the agent's line value and the limit back. **Earlier: NARROWED 2026-10-24 — FIXED on main, not released** (brief 2026-10-24 Part 4, commit only): agent `29bcf44`+`1e087bb` (`waiting_for_quiet`, `POST /host/line-test/run`), controller `046b808` (`internal/upwatch/linehold.go`: record first, hold at 1 KiB/s, test, the exact old limit back; the start-up check after a kill; the 50 % rule waits). Red-proofs in both CHANGELOGs. **Live on demo-hp with a real seed** (10 Mbit/s to DooPlex's qBittorrent): refused `not-quiet` without the hold; held: uplink 0.02–0.16 Mbit/s, 94.2 Mbit/s, 56.8 s, back to 1 249 280 B/s exactly; a kill -9 mid-test left the record and the start-up check put the limit back (`audits/dooplex-alarms-2026-10-24/part4-r1250-live-proof.txt`). The live run used probes over plain HTTP (the pinned agent client is unit-tested, not live). | the next release train (agent ≥ 0.165.0 needs the operator's vouch) | Release agent + controller in the next normal train; on a seeding box read one Tuesday's `line test on request` line; then close | CC |
 | **R-1193** | Monitoring & notifications | P3 | **When the box froze (R-1192), the operator heard only „no report for 45 m" — 28 minutes in — and the household heard nothing at all.** Measured 2026-10-20 against the operator mailbox (the second channel): the freeze began ~13:05 UTC; the hub page read ONLINE / OK at 13:22 (correct inside the 45-minute threshold, R-549); `host_stale` at 13:33, `app_oom` at 13:34 (immich-machine-learning killed by its memory limit), `host_crash_restart` at 14:26 and 16:29 (the two plug pulls), `node_stale` 15:59 / `node_recovered` 16:03. No mail or banner tells the household that its box is not answering, and no operator mail says „the box answers ping but not its dashboard". CORRECTED the same day: the first version of this row said the hub showed OK „throughout" and „nobody was told" — that read only the hub page, not the mailbox. Evidence `audits/rehearsal-2026-10-20/journal.md`. | OPEN | — | Design note (2026-10-21, after R-1192's diagnosis): the freeze's signature is cheap to see from the HOST side — the guest cgroup's memory PSI „full" above ~50 % for a minute, or its `memory.events max` climbing, while the host still reports; the agent could raise `guest_memory_thrash` (operator mail in its own words, not „no report for 45 m") and the household banner „A dobozod most nagyon lassú…". With the swap fix (R-1192) the measured thrash ends within a minute, so this alarm should be built only if a freeze is seen again after the agent release; until then it stays a design. Decide whether a dashboard that does not answer for N minutes while the host still reports should alarm sooner and in its own words (guest-side liveness), and whether the household is told | CC |
 | **R-1202** | Monitoring & notifications | P3 | **A box whose clock is behind blames the hub and stops an app without telling anyone.** Rehearsal 2026-10-20 B6 #8: RTC set 2 years back (the OS raised it to 2026-04-13): the dashboard shows only „Hub nem elérhető" while the log says „x509: certificate … not yet valid: current time 2026-04-13…"; Paperless crash-looped and the box stopped it with a restore hold; the `app_stopped_unhealthy` event failed (hub unreachable), so neither the household nor the operator heard. After the real time came back the app stayed stopped until a Start (clear sentence, 1 min). | **OPEN** | — | say „the box's clock is wrong (it says <date>)" when TLS fails with not-yet-valid/expired against a hub that answers; keep the stop event queued until the hub is reachable | CC |
@@ -399,5 +399,6 @@ stopping line that lies.
 | R-1035 | 2026-12-09 | the same three numbers 8 weeks after publishing, compared with the 4-week read; a null at 4 weeks is not a verdict, both reads are needed; detail in the R-1035 row |
 <!-- DUE-CHECKS-END -->
 | **R-1297** | Process & tooling | P3 | **CI waits dominate a working day: felhom.eu's CI run takes ~13 min and the ONE runner makes every repo's job wait behind it (up to 10–17 min queued).** Measured 2026-10-26 (operator: „these CI waiting times are getting out of hand"): since 10-25 felhom.eu runs median 793 s, the controller/agent/catalog 57–102 s; queue median 5 s, max 548–1023 s. In run 2366, ~10 of 13 minutes are four suites — `test_gate_decoys.py` 232 s, `test_due_checks_gate.py` 133 s, `test_repo_gates.py` 132 s (it runs the whole gate set a SECOND time), `dooplex-offsite/test_dooplex_offsite.py` 69 s — plus `hub-gofmt` ~110 s (a cold Go cache per run). **Ruled 2026-10-26 by the operator: BOTH** — (A) trim felhom.eu CI: the heavy suites only when their own files change, no duplicate full run, a Go cache for hub-gofmt; (B) a second CI runner on DooPlex. | **OPEN — (A) BUILT 2026-10-26 evening, closes after the first nightly run (2026-10-27 03:20 UTC) is read.** Both runners proven at once (jobs 2374 + 2375 on `felhom-gates-runner` / `-2`, 17:11:41Z). felhom.eu CI, one push each: **(a) docs only** 810 s (old runner) → 351 s (2 CPUs) → see the close-out; **(b) a gate script** 351 → 203 s (`ecd82aba`, job 2376); **(c) a hub change** 351 → 118 s (`7e57e824`, job 2378). Heavy suites by path + tag + nightly; `test_repo_gates.py` and due-checks group G test the wiring (130 s each → under 3 s). Decoys through CI's entry point with the filter ON: a docs-only register break → red; an inert gate → GREEN at first (a decoy hole, fixed in `e33ec821`) → red. No Go cache: CI has no Go (the 110 s was security-scan, now 27 s). `audits/r1297-ci-trim-2026-10-26/`. (B) DONE by the operator (`09` §3 decision 267). | — | (A) CC, a run of its own (a CI scope change, ruled); (B) a DooPlex change — a brief or the homelab session; and CC waits on CI only before a release or at the end of a run | CC |
-| **R-1298** | Monitoring & notifications | P3 | **Scratch guest 9202 carries leftovers made outside the product, and D53 would remind about them every week once released: four crash-loop holds since 2026-09-24 for apps that are not installed (`chaoscrash`, `chaosoom`, `chaosoomb`, `gokapi`), and a PrivateBin `app.yaml` (2026-10-25) with no container that the product reads as „installed".** Found 2026-10-26 evening in the D53 live proof: the new reporter sent the four holds at its first tick and the test hub mailed four reminders („held 32 days"). The product's removal clears an install's hold (R-704); these apps were removed by hand in a chaos drill. On a household box the same shape would mail the operator weekly about an app that does not exist. | OPEN | — | decide whether the reporter (or the hub) skips an `unhealthy_stop` / `update_failed` / `putback_pending` hold whose app is not installed (a restore hold stays reported — operator-cleared by design, R-379); then clear 9202's four holds with `-clear-restore-hold` and remove PrivateBin through the product | CC |
+| **R-1298** | Monitoring & notifications | P3 | **Scratch guest 9202 carries leftovers made outside the product, and D53 would remind about them every week once released: four crash-loop holds since 2026-09-24 for apps that are not installed (`chaoscrash`, `chaosoom`, `chaosoomb`, `gokapi`), and a PrivateBin `app.yaml` (2026-10-25) with no container that the product reads as „installed".** Found 2026-10-26 evening in the D53 live proof: the new reporter sent the four holds at its first tick and the test hub mailed four reminders („held 32 days"). The product's removal clears an install's hold (R-704); these apps were removed by hand in a chaos drill. On a household box the same shape would mail the operator weekly about an app that does not exist. | **VERIFY — fixed on `main`, waits for the controller release** — controller `b156e45` (an install's hold of an app that is not installed is not reported; a restore hold always is), red-proved; live on 9202 (the four orphans skipped by name). 9202's four holds cleared with `-clear-restore-hold` and the PrivateBin leftover removed through the product — `audits/night-fixes-2026-10-26/`. Close at the release. **Earlier:** OPEN | — | decide whether the reporter (or the hub) skips an `unhealthy_stop` / `update_failed` / `putback_pending` hold whose app is not installed (a restore hold stays reported — operator-cleared by design, R-379); then clear 9202's four holds with `-clear-restore-hold` and remove PrivateBin through the product | CC |
 | **R-1299** | Process & tooling | P4 | **The operator's one-shot clean-up grant (`POST /offsite/window-grant/<customer>`, optional `max_remove`) has no button on any hub page** — the operator could not find it on 2026-10-26 (D55) and had to make a one-purpose access token so CC could call the route. The route, its scope (`offsite:write`) and its event exist (R-833); only the form is missing. | OPEN | — | a small form on the Off-site page (customer, optional cap ≤ 500), with a render test for both branches | CC |
+| **R-1302** | Monitoring & notifications | P4 | **A fresh install's agent logs ERROR „store-grant: GRANT WAS MISSING AND HAS BEEN SELF-REPAIRED — investigate the loss (R-190)" and „capability DEGRADED" for `felhom-pbs` at its FIRST start, although the grant was simply never made yet** — the whole-guest tier is armed (21:00:45) before the PBS storage exists (created 21:00:57 on the hub's descriptor). Seen 2026-10-26 night on the R-1232 BYO box (VM 9375, `audits/night-fixes-2026-10-26/r1232/3-first-run.txt`) and on 2026-10-25's BYO box (`degraded=pve:store-grant:felhom-pbs`). An ERROR that says „investigate the loss" on every new box teaches the reader to skip it. | OPEN | — | on the first grant of a tier whose storage did not exist before, log INFO „granted" instead of the loss ERROR; keep the ERROR for a grant that existed and vanished; a test for both | CC |