From acc78637d910323a24a2d882160208d449db45f7 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Thu, 22 Oct 2026 18:24:30 +0200 Subject: [PATCH] night 2026-10-22: the seven 15:47 rulings in effect or ready (D6, D8 live; D21 waits for controller 0.340.0; D3/D5 website patch; D4 two patches); data-loss candidates proven/fixed/filed; R-1214 cause + capped copy proven on 9202; R-1204 alarm built; repos-private read; STATUS morning list + D29/D30/D31; CONTEXT; 03/05/07/09; rows 178 -> 180 (opened R-1218..R-1222, closed R-206, R-992, R-1207) --- CONTEXT.md | 32 +++++ REPORT-night-2026-10-22.md | 111 ++++++++++++++++++ STATUS.md | 24 ++-- documentation/architecture/03-host-agent.md | 4 + .../architecture/05-hub-architecture.md | 5 + .../architecture/07-backup-architecture.md | 2 + .../architecture/09-update-architecture.md | 8 +- .../P2-D4/03-appliance-to-discard.txt | 2 + .../P3-candidates/r1187c-9202.txt | 12 ++ .../P4-R1214/r1214-a-plain.tsv | 31 +++++ .../P4-R1214/r1214-a2-plain.tsv | 21 ++++ .../P4-R1214/r1214-b-capped.tsv | 27 +++++ .../P4-R1214/r1214-b2-capped.tsv | 19 +++ .../P4-R1214/r1214-tier2-controller-log.txt | 9 ++ .../P4-R1214/r1214-tier2-equality.txt | 3 + .../night-2026-10-22/P4-R1214/r1214-tier2.tsv | 62 ++++++++++ .../night-2026-10-22/P4-R1214/r1214run.sh | 9 ++ documentation/backlog/CLOSED-ITEMS.md | 8 ++ documentation/backlog/OPEN-ITEMS.md | 44 +++---- documentation/backlog/STATUS-HISTORY.md | 14 +++ 20 files changed, 413 insertions(+), 34 deletions(-) create mode 100644 REPORT-night-2026-10-22.md create mode 100644 documentation/audits/night-2026-10-22/P2-D4/03-appliance-to-discard.txt create mode 100644 documentation/audits/night-2026-10-22/P3-candidates/r1187c-9202.txt create mode 100644 documentation/audits/night-2026-10-22/P4-R1214/r1214-a-plain.tsv create mode 100644 documentation/audits/night-2026-10-22/P4-R1214/r1214-a2-plain.tsv create mode 100644 documentation/audits/night-2026-10-22/P4-R1214/r1214-b-capped.tsv create mode 100644 documentation/audits/night-2026-10-22/P4-R1214/r1214-b2-capped.tsv create mode 100644 documentation/audits/night-2026-10-22/P4-R1214/r1214-tier2-controller-log.txt create mode 100644 documentation/audits/night-2026-10-22/P4-R1214/r1214-tier2-equality.txt create mode 100644 documentation/audits/night-2026-10-22/P4-R1214/r1214-tier2.tsv create mode 100644 documentation/audits/night-2026-10-22/P4-R1214/r1214run.sh diff --git a/CONTEXT.md b/CONTEXT.md index 9cdd5fea..436d09bb 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -16,6 +16,38 @@ > and holds nothing of its own; this file does hold its own content, namely the standing rulings below. +> **2026-10-22 15:47 — operator rulings, „all picks" (the decide table, rows now in `STATUS-HISTORY.md`).** +> **D3** = change the zero-knowledge wording now, look again after the first sale (R-985). **D4** = root SSH from the +> LAN by key only; the console password stays (R-982). **D5** = approve the minimum machine, `audits/sizing-2026-10-13/SIZING.md` +> §5 (R-991). **D6** = a read-only login for the scratch box's drill catalog, not a public drill repo (R-992). **D8** = +> DooPlex Docker settings from git plus a weekly build-cache prune (R-206). **D21 = B** — only Immich's model cache +> leaves the backups now; the raw DB folder and Immich's own backups after their restore proof (R-49). **D27** = report +> only: the agent adds no host swap and the installer refuses nobody; reopen at the first BYO box short of swap (R-1207). + +> **2026-10-22 evening → night — the rulings put into effect (night brief; nothing released, deployed or published).** +> **D8 live:** homelab-manifests `b35ba83`, role `docker_daemon` + `docker-buildcache-prune.{service,timer}` (Sun 23:30, +> `docker builder prune --force --filter until=168h`, never `-a`); the live `daemon.json` was identical → no Docker +> restart. **D6 live:** Gitea user `drill-reader` (not admin), read collaborator on `admin/app-catalog-drill`, token +> `read:repository` in `sec`; proven (read 200, write/push 403, other private repos 404); `09` §6.5. **D21 built:** +> controller `6a17c95` — `.felhom.yml` `backup.volumes: [{name, class: excluded}]` → `appbackup.ExcludedDockerVolumes` +> (declared names only; refuses to exclude all of an app's named volumes) → `backup.Manager.dumpableVolumes`; catalog +> `ce33e3b` (Immich, live, inert before controller 0.340.0); 9202: unit 1.16 GB → 337 MB, Immich ML answered after the +> cache was removed. **D3/D5:** the volunteer guides committed (`0639cf07`); the website change is a ready patch +> (`audits/night-2026-10-22/P2-D3-D5/website-D3-D5.patch`) — a push to `website/` publishes. **D4:** two ready patches +> (`P2-D4/`): an agent config-bundle drop-in `/etc/ssh/sshd_config.d/10-felhom-root-key-only.conf` +> (`PermitRootLogin prohibit-password`; Debian's Include comes first and the first value wins; felhom-sshd :8822 has its +> own config) and the installer's uninstall; proven on a fresh ISO 1.30.0 install (VM 9343, destroyed; its appliance 49 +> waits for the operator's Discard). **Data-loss candidates** (`audits/night-2026-10-22/P3-candidates/analysis.md`): +> R-1187(c) fixed (`d5f88a2`: an import over an installed, not-running app is refused — 9202 reproduced the merge on +> rc.1 and the refusal on rc.2); R-1189(d) fixed (`nightchain.CatchUp.After` runs the off-site proof + integrity check +> once); new rows R-1218..R-1222 with their proposed fixes. **R-1214** (03:30 pressure): the copy's page cache is +> charged to the guest; `backup/tier2.go` now runs rsync in a 512 MB memory-capped container of the controller's own +> image for clean `/mnt/` paths with no symlink (`03c0a69`, `ae39467`; exit 125 → plain rsync); 9202 stall 5.7 s → 0.37 s. +> **R-1204:** agent `a0c88e5` reports `handshake_future_s`; hub 0.166.0 (`e267c5e2`, built, not deployed) mails +> `wg_tunnel_stale`/`wg_tunnel_ok` to the operator only. **Repos private (R-925):** only felhom-controller can go +> private (after a CI read token); STATUS D29. Picks waiting: D29 (No), D30 (re-key through `RegisterWGPeerForHost`), +> D31 (R-1205: skip an un-acked, key-less deletion). Record: `REPORT-night-2026-10-22.md`. + > **2026-10-22 ~14:50 — operator acts, checked by CC (read only).** Host `Tester-2-be8404` deleted (12:54:47 UTC); > customer `Tester-2` language set to Magyar; customer `Tester-2` RESET (12:55:56–12:56:04 UTC, journal #25, every leg > ok). The first live run of R-32's purge-first leg: logged in with the sub-account's own password, found no repository diff --git a/REPORT-night-2026-10-22.md b/REPORT-night-2026-10-22.md new file mode 100644 index 00000000..205a31b2 --- /dev/null +++ b/REPORT-night-2026-10-22.md @@ -0,0 +1,111 @@ +# REPORT — night 2026-10-22: seven decisions, the data-loss candidates, the 03:30 pressure, the tunnel alarm, private repositories + +Evening → night brief, started ~16:00 CEST. Nothing was released, deployed or published, except the catalog change and +the scratch box's catalog login (both allowed by the brief). Design read: `07-backup-architecture.md` §6.1.1, §6.2, +§10.2; `03-host-agent.md` (tunnel, bundle); `05-hub-architecture.md` §15; `06-offsite-connectivity.md`; +`09-update-architecture.md` §6.5. + +## 1. Five lines + +1. **D8:** done before 18:00 — DooPlex's Docker settings come from git (no change, no restart), and a weekly build-cache + prune timer runs Sunday 23:30. +2. **The seven decisions:** live — D6, D8, D27 (report only). Ready, waiting for a release or publish — D21 + (controller 0.340.0), D3 + D5 (one website patch), D4 (agent bundle patch + installer patch). +3. **Data-loss candidates:** 2 proven and fixed in code (R-1187 c, R-1189 d); 5 real in code, now rows with a written + fix (R-1218, R-1219, R-1220, R-1221, R-1222); 1 closed half (R-1189 b, fixed in 0.304.0); 3 to the operator + (R-1205 as D31, R-1189 f, R-1190 d); the rest stay as named candidates. +4. **03:30 pressure:** the second-drive copy's file cache is charged to the guest and pushes the apps out of memory. + The fix runs the copy in a 512 MB memory-capped container: full stall 5.7 s → 0.37 s on the scratch box. +5. **Private repositories:** only felhom-controller can go private (after one CI change). felhom.eu, the agent and the + catalog must stay public: the website, every install and every box read them with no login. + +## 2. The morning list (also in STATUS) + +1. Publish the website patch (D3 wording, D5 hardware page). +2. D4: the agent bundle patch with the next agent release (your vouch) + the installer patch with a new installer tag. + Discard appliance 49 on the hub's Hosts page (tonight's test install). +3. Deploy hub 0.166.0 (tunnel alarm) and release the agent with `handshake_future_s` — one agent release with item 2. +4. Release controller 0.340.0 (R-49, R-1187 c, R-1189 d, R-1214), MinAgent 0.158.1. +5. Decide D29 — the controller repository private? CC: No, not now. +6. Decide D30 — the tunnel repair. CC: the agent makes a new key and registers it on the path that exists. +7. (D31, R-1205, when there is time.) Tester 2's „go" from yesterday stays on the list. + +## 3. Per candidate + +| Item | Result | Reason | +|---|---|---| +| R-1187 a | REAL in code → row R-1187 (now P2) | an export drops a folder with a symlink and still says done; capture code: needs a red-proof + 9202 proof | +| R-1187 b | REAL in code → R-1187 | the import sets no owners and drops symlinks | +| R-1187 c | **PROVEN and FIXED** (`d5f88a2`) | 9202 rc.1: an import over a stopped app merged old + new data; rc.2: refused, app untouched | +| R-1187 d | PLAUSIBLE → R-1187 | no busy check; after (a) | +| R-1187 f, g | REAL in code → R-1187 | HDD_PATH trusted; postgis missed, psql without ON_ERROR_STOP | +| R-1188 a, c | already CLOSED (R-1195; 0.337.0) | — | +| R-1188 b, d, e | R-1188, re-ranked P4 | unlikely or a sentence | +| R-1189 a | REAL, half-live → R-1189 | agent code; rides an agent release | +| R-1189 b | SPLIT: skipped nights CLOSED (R-899, 0.304.0); creep → **R-1221** (P4) | demo-hp journal: a night run after each of 3 off-cycle runs; creep 1.5–5 min/night measured | +| R-1189 c, g, i | REAL in code → R-1189 (P4-shaped) | quiesce vs restore; spring forward 2027; double click | +| R-1189 d | **FIXED** (`d5f88a2`) | the catch-up now runs the proof + integrity once; red-proved | +| R-1189 e | REAL in code → **R-1222** (hub, P3) | an evening backup time gives a false „dump missed" every morning | +| R-1189 f | operator → R-1189 | design question | +| R-1190 a | REAL in code → **R-1218** (P2) | proposed fix, not built: key code needs a live proof, and 9202 has no hub | +| R-1190 b | REAL in code → **R-1219** (P2) | the top bar links to a ceremony the page hides during an offer | +| R-1190 c | PLAUSIBLE → R-1190 | a read-only fingerprint compare decides it | +| R-1190 d | operator → R-1190 | design question | +| R-1190 g | REAL in code → **R-1220** (P2) | an unreadable app.yaml lists a live folder as kept (deletable) | +| R-1205 | operator → **D31** | the F-14 gate is deliberate (ruling 2026-07-13) | +| R-1206 | NARROWED | Felhom tier: no loss, by design (R-822 guard). NAS tier: a unit test is owed | + +Research input: `documentation/audits/night-2026-10-22/P3-candidates/analysis.md`. + +## 4. Records + +**Commits.** +- felhom-controller: `6a17c95` (R-49), `d5f88a2` (R-1187 c, R-1189 d), `03c0a69` + `ae39467` (R-1214), `df78b6c` (docs). + CI jobs 2214, 2220, 2221, 2222, 2223 — all success, attempt 1. +- felhom-agent: `a0c88e5` (R-1204 + D4), `7651461` (D4 moved to a patch), `e803ad2` (docs, a correction). CI 2217, + 2218 success; 2224 in progress at the time of writing — see the final note below. +- felhom.eu: `0639cf07`, `e267c5e2` + `1100a344` (one push), and this commit. CI 2215, 2219 success (2219 covers `e267c5e2`). +- app-catalog-felhom.eu: `ce33e3b` (Immich `backup.volumes`). CI 2216 success. +- homelab-manifests: `b35ba83` (D8), `c7506df` + `b0cce29` (REPORT). No CI in that repo. +- Drill catalog (scratch, no CI by design): `2ba4b53`, `9ebdba6`, `b0350a3` (tree = live `ce33e3b` again). + +**Images pushed.** `felhom-controller:0.340.0-rc.1`..`rc.4` (never offered to a household: not X.Y.Z). +`felhom-hub:0.166.0` (built, NOT deployed: the live hub and `manifests/hub.yaml` are 0.165.0, read 18:2x). + +**Teardown, three layers.** +- **Machine (9202):** the throwaway uptime-kuma removed through the product (its `.fab` deleted); the hog container and + 19 GB of test files removed; memory 25898 MB / swap 512 MB again; Immich's `.felhom.yml` back to its pre-night file; + controller back on 0.339.0 (healthy, every app up). Kept on purpose: the `drill-reader` catalog login (D6). + Left: the four rc images in 9202's own Docker store, like earlier rc images there. +- **Host (demo-hp):** VM 9343 destroyed (own id); `/tmp/r1214*` removed; two sampler loops stopped. The scratch + storage and VM 341 / CT 9401 are standing items, not tonight's. +- **Hub:** appliance 49 from VM 9343 is still „registered" — Discard is the operator's (session-only). Nothing else. +- **DooPlex:** the D8 timer (intended). Nothing else changed. + +**Rows:** before 178, after 180. Opened 5: R-1218, R-1219, R-1220, R-1221, R-1222. Closed 3: R-206 (D8), R-992 (D6), +R-1207 (D27). Updated: R-49, R-925, R-982, R-985, R-991, R-1187 (P3→P2), R-1188 (P3→P4), R-1189, R-1190, R-1204, +R-1205, R-1206, R-1214, R-1215. + +**Unproven claims:** 35 of 55 not walked — unchanged. **System poster:** no listed fact changed (nothing that the +poster draws shipped tonight) — none of the three actions was needed. + +**Observations.** +- 9202's sync refused Immich's template (R-971: Immich's data sits on a bench mount, not a household drive). — + NOT-A-FINDING: R-971 by design; the file was placed by hand for the proof and put back. +- The local unit restore of Immich was refused (R-538). — NOT-A-FINDING: by design; the proof used an empty-cache start. +- A non-`--fast` catalog gate run started the runtime gate on DooPlex; its canary refused and left nothing. — + NOT-A-FINDING: the canary is the control; the rule (`--fast` on DooPlex) exists. +- Images and packages stay public when a repository goes private. — NOT-A-FINDING: they follow the user `admin`, and + boxes must pull them; recorded in R-925 and D29. +- A CI read token is a precondition of D29. — NOT-A-FINDING: a step of that decision, written in it. +- R-925 said „32 source comments"; the served count is 61 on four pages. — fixed in the row. +- **CC's own mistake 1:** the agent CHANGELOG and felhom.eu commit `1100a344`'s message say a push of the installer + publishes it. Wrong: `/scripts/` is served from the tag `installer-v1.35.0` (R-110). Corrected in the agent CHANGELOG + (`e803ad2`) and R-982; the commit message cannot change (no force-push). Keeping D4 as a patch still holds: both + halves need a release that was forbidden tonight. +- **CC's own mistake 2:** controller commit `df78b6c` carries a „Co-Authored-By" line, which the brief forbids. It stays + (no force-push); no other commit tonight has one. +- **CC's own mistake 3:** homelab-manifests' REPORT first claimed a separate daemon.json backup. There was none — the + file did not change and is byte-identical to the role's copy in git (`cmp`). Corrected in `b0cce29`. +- The guest has no rsync. — NOT-A-FINDING: rsync lives in the controller image; the capped copy uses that image. +- R-1215's evidence is gone (Immich restarted at 02:16). — said in the row; it stays a watch. diff --git a/STATUS.md b/STATUS.md index bbcc4b0e..36a31a17 100644 --- a/STATUS.md +++ b/STATUS.md @@ -2,7 +2,7 @@ **Ready for Tester 2? Yes — tell the friend go (hand-over list: `documentation/pilot/TESTER2-handover.md`).** -**Updated 2026-10-22 (day).** A rebuilt box no longer overwrites the family's saved mail list (hub 0.165.0, controller 0.339.0); demo-felhom's own list, replaced this morning, is back. The swap warning now fires only when a machine has clearly too little swap (your D28 = A). +**Updated 2026-10-22 (night).** Your seven picks of 15:47 are in effect or ready: D6 and D8 are live, D21 waits for the controller release, and D3, D5 (the website) and D4 (root SSH) are ready patches. Four controller fixes are built and proven on the scratch box, among them a gentler night copy (the 03:30 memory pressure) and no import over a stopped app. Nothing was released or deployed; the morning list below has the order. ## What works - A box installs from nothing with the public installer, links to the hub, and moves to the floor by itself (map A1, A3, A4). @@ -16,10 +16,9 @@ ## What is broken - **P1:** about 12 homelab logins still use a password that was public, and the public Gitea repo is not decided (R-925, yours). -- Two root-level weaknesses wait on your decisions D3 and D4 (R-985, R-982). - Tester 2 is off since 2026-10-04: old emergency password, no off-site check, no bootstrap (R-1139, R-1104, R-862). - DooPlex: the failed disk is still in the box, its alarms silenced (R-934); something sets the shared checkout to „bare" (R-935). -- The scratch box cannot use agent features (R-1083) or read the drill catalog (R-992). +- The scratch box cannot use agent features (R-1083). ## What we work on - Dated checks, run by CC. 2026-10-23: no app stop when the host is busy (R-1173). 2026-10-26: each hub daily job ran once on the clock-change day (R-1183). 2026-10-27: the weekly clean-up window (R-435; the logs of the last one were lost to restarts). @@ -30,15 +29,11 @@ Newest first. CC's pick is in the „Pick" column. Each row also has its full te | # | Row | Question | Pick (why) | The other option costs | If you do nothing | Blocked | |---|---|---|---|---|---|---| -| D27 | R-1207 | A host with no swap gets nothing from the swap fix, and the agent never adds swap. Add host swap automatically (BYO too), or refuse an 8 GB host without swap at install? | **Report only for now** (our installer gives 7.75 GB, and Tester 2's laptop used our installer; the alarm and the install-page sentence now tell you) — decide after the first BYO box | Add it: the agent changes a disk its owner set up; refuse: a BYO household cannot start | a no-swap host can still freeze; you get the alarm | — | -| D21 | R-49 | Immich's off-site copy: leave out only the AI model files now (B), or also the raw database folder and Immich's own backups (A)? | **B now** (−62 %, no new risk); A's other parts after their proof | A: −92 %, but it needs a restore path we have never run | every Immich box sends about 12 times its real data off-site each night | — | +| D31 | R-1205 | After a box death, a second install that is deleted before it makes a key stops the automatic re-issue of the whole-box copy. Let the hub skip such a deletion, or keep your manual press? | **Skip it** (that install held no key, so nothing is lost — the same reasoning the key carry uses) | Keep the press: you get a mail and press „Re-issue PBS credentials” each time | that household gets no whole-box copy until you press | — | +| D30 | R-1204 | After a box ran with its clock ahead, how does its tunnel to ep0 heal? | **The agent makes a new key and registers it** (the peer path that exists; nothing new on ep0) | ep0 removes and re-adds the peer on a hub-signed request: a new write path on ep0 | with hub 0.166.0 you get the alarm, and the repair is by hand on ep0 | the tunnel fix | +| D29 | R-925 | Make the controller's repository private now? (felhom.eu, the agent and the catalog must stay public: the website, every install and every box read them with no login.) | **No, not now** (its image stays public anyway, and the no-secret gate is the control; look again when felhom.eu can go private) | Yes: a read token in CI for every repo that clones it, one more secret to keep | nothing changes | — | | D9 | R-1062 | Split the N100's USB drive in two, for multi-drive tests? | **No — keep one drive** (the lived-in box keeps its nightly whole-box backup) | Split: the box loses its local whole-box backup, or you buy a second drive first | it stays one drive; multi-drive tests stay on Tester 1 | — | -| D8 | R-206 | Install DooPlex's Docker settings from git (Docker restarts once), with a weekly build-cache prune? | **Yes, in a quiet window** (a rebuilt DooPlex then keeps the settings) | No: the live settings stay hand-made, and a rebuild loses them | git and the host can drift apart | — | | D7 | R-990, R-989 | Rule on the forum ideas and the selfh.st ideas now, or park them? | **Park them** (P4; nothing waits on them) | Rule now: about one hour of reading | nothing changes; the research gets old | — | -| D6 | R-992 | The scratch box cannot read the private drill catalog: give it a read-only login, or make that copy public? | **A read-only login** (the drill catalog stays private) | Public copy: anyone can read the drill catalog | every new-app test needs staging by hand | new-app tests | -| D5 | R-991 | Approve the minimum machine: x86-64, 4 cores, 8 GB RAM, 256 GB SSD (recommended: 16 GB, 512 GB)? | **Approve** (measured on the N100 with 61 apps) — the 8 GB freeze of 2026-10-20 is fixed by agent 0.162.0 (11 h soak on a fresh 8 GB box, no freeze) | Change it: one edit | no hardware page; a buyer with a Raspberry Pi can still download the installer | the hardware page | -| D4 | R-982 | Stop root SSH with a password from the household network (key only)? | **Yes — key only** (the password stays for the console) | Keep it: anyone on the household network can try the password | the host keeps accepting root password logins from the LAN | — | -| D3 | R-985 | Our „zero-knowledge" wording is stronger than the product: change the wording, or remove the operator's root reach to the key? | **Change the wording now**; look again after the first sale | Remove the reach: a design change to break-glass | the website promises more than the product does | the website text | ## Waiting on you — do @@ -51,8 +46,13 @@ Newest first. CC's pick is in the „Pick" column. Each row also has its full te 6. BIOS: on the N100, read „After Power Loss"; on the HP, set „Bypass F1 Prompt on Configuration Changes" to Enable. Since 2026-10-14. 7. Pin the first Facebook post (R-917), and look at the new cover in the Facebook phone app (R-919). Since 2026-10-12. -**Today, 2026-10-22.** -1. Give Tester 2's friend the owner passphrase and the one-page guide, and tell him „go". The reset worked: his old copies are gone, nobody else's changed. If not: nothing happens; the link mail expires in 7 days (he can ask for a new one). +**Morning, 2026-10-23 — in this order.** +1. Publish the website patch: the D3 wording and the D5 hardware page (R-985, R-991). Say „publish”; CC pushes it. If not: the site keeps the stronger promise and has no hardware page. +2. D4 rides two releases, both ready patches: the agent's config bundle (the next agent release, your vouch) and a new installer tag (R-982). Also press Discard on the hub's Hosts page for appliance 49 (tonight's test install). If not: hosts keep taking root passwords from the LAN. +3. Deploy hub 0.166.0 (the clock-ahead tunnel alarm, built) and release the agent with its tunnel field, in the same agent release as item 2 (R-1204). If not: a dead tunnel stays silent. +4. Release controller 0.340.0: Immich's model cache out of backups, no import over a stopped app, the catch-up checks, the capped night copy (R-49, R-1187, R-1189, R-1214). If not: these four wait. +5. Decide D29 (a private repository) and 6. D30 (the tunnel repair), in the table above; D31 when you have time. +7. Tester 2 (from 2026-10-22): give the friend the owner passphrase and the one-page guide, and say „go”. If not: the link mail expires in 7 days. **The rest** - Pull the failed disk Z6A07P2G out of DooPlex (R-934). If not: its alarms return on 11 November. Since 2026-10-12. - Rotate the reused homelab logins and decide the Gitea repo's visibility (R-925, P1). Since 2026-10-09. diff --git a/documentation/architecture/03-host-agent.md b/documentation/architecture/03-host-agent.md index ad1ad727..282ad639 100644 --- a/documentation/architecture/03-host-agent.md +++ b/documentation/architecture/03-host-agent.md @@ -129,6 +129,10 @@ the one trust root; the agent's API token is untouched; a box gets the new rule `agent_config_update` (the order: signed `agent_update` first, then the bundle — after the bundle, an agent below 0.146.0 cannot update itself on that box). +**[DESIGN, ruled 2026-10-22 15:47 (D4, R-982) — built as a ready patch, NOT shipped] Root SSH from the LAN by key only; the console password stays.** The config bundle gains one fixed file, `/etc/ssh/sshd_config.d/10-felhom-root-key-only.conf` (`PermitRootLogin prohibit-password`), checked and reloaded by `felhom-os-apply`; the installer's uninstall removes it. Why a drop-in: Debian's `sshd_config` Includes `sshd_config.d/*` FIRST and sshd keeps the first value it reads, so the drop-in wins over the stock line; the out-of-band operator sshd (`FELHOM_SSHD`, port 8822) has its own config with no Include and is not touched. Proven on a fresh ISO 1.30.0 install: password SSH refused after, key SSH and the PVE web password login (200) and the console login still work (`audits/night-2026-10-22/P2-D4/`). It ships with the next agent release + installer tag. + +**[FACT, agent main `a0c88e5`, unreleased] The heartbeat reports a WireGuard handshake from the future (R-1204).** WireGuard refuses an initiation whose TAI64N time is not newer than the last one accepted from that peer, so a box that once ran with its clock ahead is locked out of ep0 until real time passes that moment. The agent used to clamp the negative age to 0 (it looked healthy); `wgtunnel.handshakeFuture` now sets `handshake_future_s` when the handshake is more than 60 s ahead (`TestR1204_AFutureHandshakeIsReported`). The hub alarms on it (`05` §15.4b). The repair is STATUS decision D30. + ## 4. Control model — reconcile + signed destructive ops Two channels, split by **reversibility**, not by transport. diff --git a/documentation/architecture/05-hub-architecture.md b/documentation/architecture/05-hub-architecture.md index 385253d1..ce1c8860 100644 --- a/documentation/architecture/05-hub-architecture.md +++ b/documentation/architecture/05-hub-architecture.md @@ -454,6 +454,11 @@ warning mails the operator (anything lost, still broken, failed or skipped); inf The operator mail renders the counts as one `Repair:` line. The household's side is NOT a mail: the lost files (name, date taken) are shown on its own app page, on its box, until it dismisses them — file names never leave the box. + +### 15.4b A box whose tunnel to ep0 is dead — the clock-ahead alarm [hub 0.166.0, built, NOT deployed; R-1204] + +**[FACT, hub `e267c5e2`]** The host report carries the agent's `wireguard` block (`hostWireguard`). `checkWGTunnel` (`api/host_tunnel.go`, called after `checkHostSwap`) calls the tunnel stale when the box reports `handshake_future_s` > 0 (the clock-ahead lock-out) or a last handshake older than 6 h (healthy boxes measured at most 113–124 s over 24 h). It is edge-triggered against the host's latest tunnel event (`store.LatestHostTunnelEvent`): `wg_tunnel_stale` once on the way down, `wg_tunnel_ok` once on the way back; both are allow-listed and operator-only (`notify.operatorOnlyEvents`, `wg_tunnel_operator_only_test.go`) — the household cannot repair a tunnel. A box with no `wireguard` block (an agent before the field, or no tunnel) raises nothing. + ### 15.5 How "the Hungarian did not change" is known 56 goldens captured from v0.117.0 before any string moved, in diff --git a/documentation/architecture/07-backup-architecture.md b/documentation/architecture/07-backup-architecture.md index ac9bcc99..a6ddc05e 100644 --- a/documentation/architecture/07-backup-architecture.md +++ b/documentation/architecture/07-backup-architecture.md @@ -647,6 +647,8 @@ leg** — LIVE on demo-felhom, `backups/secondary/bookstack/` and `.../docmost/` `recovery-unit` and nothing else, at 156 MB and 86 MB (INV Part B.2). Since v0.183.0 the button refuses **before** stopping the app and names the action that works. +**[DESIGN] A template may leave a named volume out of every backup (2026-10-22, operator D21 = B, R-49; controller 0.340.0).** `.felhom.yml` `backup.volumes: [{name: , class: excluded}]` — the name as the compose file declares it; the controller dumps every named volume of the app EXCEPT `_` (`appbackup.ExcludedDockerVolumes` → `backup.Manager.dumpableVolumes`), so the volume leaves the local unit, the second-drive copy and the off-site copy at once, and the next run's stale clean-up removes its old tar. Only class `excluded` acts; a name the compose file does not declare is ignored and logged; **a list that would exclude every named volume of the app is refused whole** (nothing excluded) — a template typo must not empty a backup (`TestR49_*`). Use it only for data the app rebuilds by itself: the first user is Immich's `immich_ml_cache` (model weights, re-downloaded at start) — 9202: unit 1.16 GB → 337 MB; with the volume removed, Immich started and its ML answered after 9.2 s (`audits/night-2026-10-22/P2-D21/`). A controller older than 0.340.0 ignores the key (inert). The raw database folder and Immich's own `upload/backups/` stay in until a restore from the dump alone is proven (D21's option A). + ### 6.3 Where capture and restore disagree **[FACT]** Three asymmetries, each source-cited: diff --git a/documentation/architecture/09-update-architecture.md b/documentation/architecture/09-update-architecture.md index 8d59e012..cdf6a095 100644 --- a/documentation/architecture/09-update-architecture.md +++ b/documentation/architecture/09-update-architecture.md @@ -2549,7 +2549,13 @@ well, or the box goes on following the live catalog and reports success. Filed a is fixed, the drill procedure is: 1. save `controller.yaml` as `controller.yaml.pre-update-night`; -2. set `git.repo_url` (and `username`/`token` — the drill repo is private); +2. set `git.repo_url` (and `username`/`token` — the drill repo is private). **The box's login is the Gitea user + `drill-reader`** — not an admin, a read collaborator on `admin/app-catalog-drill` only, token scope + `read:repository`; it cannot write or push and sees no other private repo (measured: 200 / 403 / 403 / 404, + `audits/night-2026-10-22/P2-D6/01-token-limits.txt`). The token is `DRILL_CATALOG_TOKEN` and the account's password + `DRILL_READER_PW` in CC's `sec` store — never printed. *Operator ruling 2026-10-22 15:47, D6: a read-only login, not + a public drill repo (R-992, closed).* Since that night 9202 keeps `git.username`/`git.token` while it follows the + live catalog (the public repo ignores them), so pointing it at the drill is the URL plus step 3; 3. **remove `/catalog-cache`**; 4. restart the controller, sync, **rescan** (R-607: a sync can answer „nincs változás" while the catalog has moved, and the badge answers from the stale value until the rescan); diff --git a/documentation/audits/night-2026-10-22/P2-D4/03-appliance-to-discard.txt b/documentation/audits/night-2026-10-22/P2-D4/03-appliance-to-discard.txt new file mode 100644 index 00000000..9e301455 --- /dev/null +++ b/documentation/audits/night-2026-10-22/P2-D4/03-appliance-to-discard.txt @@ -0,0 +1,2 @@ +Hub appliance from the D4 proof VM 9343 (ISO 1.30.0 on demo-hp), read from a one-tar copy of hub.db + -wal + -shm at 18:04 CEST: +id 49, status registered, first_seen 2026-10-22 14:59:13 UTC, not discarded. The operator presses Discard on the hub Hosts page. diff --git a/documentation/audits/night-2026-10-22/P3-candidates/r1187c-9202.txt b/documentation/audits/night-2026-10-22/P3-candidates/r1187c-9202.txt new file mode 100644 index 00000000..8dccfba0 --- /dev/null +++ b/documentation/audits/night-2026-10-22/P3-candidates/r1187c-9202.txt @@ -0,0 +1,12 @@ +# R-1187 (c) on 9202 — a .fab import into an installed app that is not running (throwaway uptime-kuma, its one named +# volume uptime-kuma_uptime_kuma_data; markers written with a `docker run --rm -v :/v alpine touch` helper) +## REPRODUCED on controller 0.340.0-rc.1 (before the fix), 2026-10-22 15:21–15:23 UTC +15:21:47 before export: MARK-A ; export (stop_app true) → uptime-kuma_20261022-152152.fab (22 KB) ; then MARK-N written +CONTROL (app RUNNING): before import MARK-A MARK-N → import ok → after: MARK-A (a clean replace) +CASE (app STOPPED): before import MARK-A MARK-N → import ok → after: MARK-A MARK-N (MERGED — MARK-N survived) +controller log 15:22:54: [DEBUG] [appexport] stack is deployed but not running — will overwrite config in place +## FIXED on controller 0.340.0-rc.2 (d5f88a2), 15:30–15:32 UTC +STOPPED: before import MARK-A MARK-N MARK-X → import REFUSED: „Az alkalmazás telepítve van, de nem fut, ezért a + csomagot nem töltjük vissza: a régi adatai összekeveredhetnének a csomagéval. Az alkalmazás érintetlen. + Indítsd el (vagy távolítsd el), és próbáld újra." → after: MARK-A MARK-N MARK-X (nothing changed) +RUNNING (control): start → import ok → after: MARK-A (the running path still replaces) diff --git a/documentation/audits/night-2026-10-22/P4-R1214/r1214-a-plain.tsv b/documentation/audits/night-2026-10-22/P4-R1214/r1214-a-plain.tsv new file mode 100644 index 00000000..5995f301 --- /dev/null +++ b/documentation/audits/night-2026-10-22/P4-R1214/r1214-a-plain.tsv @@ -0,0 +1,31 @@ +15:37:41 full avg10=0.02 avg60=0.11 avg300=0.03 total=1426421 anon 2977931264 file 1118461952 565907456 workingset_refault_anon 50674 pswpin 50160 pswpout 305782 +15:37:44 full avg10=0.38 avg60=0.17 avg300=0.05 total=1482650 anon 2954280960 file 1159651328 605614080 workingset_refault_anon 50677 pswpin 50163 pswpout 314688 +15:37:46 full avg10=1.03 avg60=0.30 avg300=0.07 total=1592995 anon 2817572864 file 1305579520 746491904 workingset_refault_anon 50794 pswpin 50264 pswpout 347405 +15:37:48 full avg10=1.02 avg60=0.32 avg300=0.08 total=1596156 anon 2817511424 file 1297158144 747855872 workingset_refault_anon 50903 pswpin 50363 pswpout 347719 +15:37:50 full avg10=3.92 avg60=0.86 avg300=0.20 total=1954843 anon 2753028096 file 1359564800 863498240 workingset_refault_anon 51669 pswpin 51088 pswpout 366763 +15:37:52 full avg10=6.11 avg60=1.36 avg300=0.30 total=2463722 anon 2661449728 file 1457446912 969809920 workingset_refault_anon 51954 pswpin 51356 pswpout 390137 +15:37:54 full avg10=12.43 avg60=2.66 avg300=0.58 total=3197424 anon 2582831104 file 1535963136 1084706816 workingset_refault_anon 57081 pswpin 56378 pswpout 419473 +15:37:56 full avg10=13.80 avg60=3.22 avg300=0.72 total=3584025 anon 2508480512 file 1612091392 1217191936 workingset_refault_anon 77682 pswpin 76958 pswpout 451531 +15:37:58 full avg10=13.83 avg60=3.58 avg300=0.81 total=3943183 anon 2517655552 file 1601052672 1265401856 workingset_refault_anon 90983 pswpin 90258 pswpout 462834 +15:38:00 full avg10=16.76 avg60=4.44 avg300=1.01 total=4439979 anon 2479181824 file 1640677376 1326792704 workingset_refault_anon 94312 pswpin 93556 pswpout 476381 +15:38:02 full avg10=14.99 avg60=4.53 avg300=1.05 total=4574276 anon 2442424320 file 1675341824 1376956416 workingset_refault_anon 96385 pswpin 95562 pswpout 487861 +15:38:04 full avg10=13.72 avg60=4.64 avg300=1.10 total=4753844 anon 2352107520 file 1761906688 1479811072 workingset_refault_anon 98070 pswpin 97243 pswpout 512237 +15:38:06 full avg10=12.69 avg60=4.75 avg300=1.14 total=4916243 anon 2322202624 file 1789628416 1534160896 workingset_refault_anon 103396 pswpin 102483 pswpout 522846 +15:38:08 full avg10=13.29 avg60=5.12 avg300=1.24 total=5374360 anon 2285264896 file 1815957504 1609768960 workingset_refault_anon 106378 pswpin 105437 pswpout 540349 +15:38:10 full avg10=12.69 avg60=5.28 avg300=1.30 total=5458434 anon 2280771584 file 1828876288 1611206656 workingset_refault_anon 110996 pswpin 110030 pswpout 540986 +15:38:12 full avg10=10.93 avg60=5.20 avg300=1.32 total=5541954 anon 2255233024 file 1856950272 1634451456 workingset_refault_anon 111176 pswpin 110172 pswpout 546993 +15:38:14 full avg10=10.76 avg60=5.36 avg300=1.38 total=5867729 anon 2228326400 file 1883500544 1666342912 workingset_refault_anon 112159 pswpin 111119 pswpout 554054 +15:38:16 full avg10=10.26 avg60=5.45 avg300=1.42 total=5916591 anon 2217484288 file 1896869888 1674969088 workingset_refault_anon 112406 pswpin 111355 pswpout 556387 +15:38:18 full avg10=8.95 avg60=5.37 avg300=1.43 total=5994829 anon 2192748544 file 1920950272 1699532800 workingset_refault_anon 113145 pswpin 112061 pswpout 562876 +15:38:20 full avg10=8.59 avg60=5.42 avg300=1.47 total=6156025 anon 2187468800 file 1923219456 1716023296 workingset_refault_anon 116205 pswpin 114986 pswpout 566345 +15:38:22 full avg10=8.30 avg60=5.47 avg300=1.51 total=6347323 anon 2151600128 file 1957306368 1768026112 workingset_refault_anon 118566 pswpin 117299 pswpout 577430 +15:38:24 full avg10=8.25 avg60=5.55 avg300=1.55 total=6510370 anon 2122932224 file 1983873024 1821814784 workingset_refault_anon 121530 pswpin 120222 pswpout 588409 +15:38:26 full avg10=7.66 avg60=5.54 avg300=1.58 total=6575873 anon 2118803456 file 1980669952 1831043072 workingset_refault_anon 127295 pswpin 125968 pswpout 591207 +15:38:28 full avg10=7.17 avg60=5.52 avg300=1.60 total=6694348 anon 2081148928 file 2015191040 1862037504 workingset_refault_anon 128328 pswpin 127000 pswpout 600104 +15:38:30 full avg10=6.06 avg60=5.37 avg300=1.60 total=6722977 anon 2087165952 file 2009464832 1862000640 workingset_refault_anon 130120 pswpin 128710 pswpout 600153 +15:38:32 full avg10=5.50 avg60=5.29 avg300=1.61 total=6817784 anon 2100473856 file 2002825216 1868599296 workingset_refault_anon 135578 pswpin 134022 pswpout 601868 +15:38:34 full avg10=6.50 avg60=5.48 avg300=1.67 total=7031932 anon 2092240896 file 2014990336 1877950464 workingset_refault_anon 136858 pswpin 135295 pswpout 604035 +15:38:36 full avg10=6.22 avg60=5.46 avg300=1.69 total=7114873 anon 2097594368 file 2007240704 1894051840 workingset_refault_anon 140929 pswpin 139220 pswpout 606078 +15:38:38 full avg10=5.10 avg60=5.28 avg300=1.68 total=7124559 anon 2099490816 file 2005364736 1893933056 workingset_refault_anon 142157 pswpin 140407 pswpout 606105 +15:38:40 full avg10=4.53 avg60=5.17 avg300=1.68 total=7163770 anon 2103656448 file 1998692352 1884336128 workingset_refault_anon 144792 pswpin 142968 pswpout 606570 +15:38:42 full avg10=3.71 avg60=5.00 avg300=1.67 total=7168499 anon 2106265600 file 1985736704 1883377664 workingset_refault_anon 146085 pswpin 144169 pswpout 606614 diff --git a/documentation/audits/night-2026-10-22/P4-R1214/r1214-a2-plain.tsv b/documentation/audits/night-2026-10-22/P4-R1214/r1214-a2-plain.tsv new file mode 100644 index 00000000..ff11f410 --- /dev/null +++ b/documentation/audits/night-2026-10-22/P4-R1214/r1214-a2-plain.tsv @@ -0,0 +1,21 @@ +15:40:12 full avg10=0.00 avg60=1.20 avg300=1.26 total=7601138 anon 2332147712 file 1179697152 1768947712 workingset_refault_anon 198199 pswpin 195091 pswpout 608043 +15:40:14 full avg10=0.00 avg60=1.16 avg300=1.25 total=7643545 anon 2338930688 file 1762066432 1769275392 workingset_refault_anon 198205 pswpin 195095 pswpout 609142 +15:40:16 full avg10=0.18 avg60=1.15 avg300=1.25 total=7663987 anon 2334179328 file 1770704896 1769304064 workingset_refault_anon 198240 pswpin 195115 pswpout 609747 +15:40:18 full avg10=1.78 avg60=1.41 avg300=1.30 total=7866238 anon 2262745088 file 1841442816 1820577792 workingset_refault_anon 199695 pswpin 196569 pswpout 623826 +15:40:20 full avg10=2.00 avg60=1.46 avg300=1.31 total=7895241 anon 2264272896 file 1831309312 1820860416 workingset_refault_anon 200428 pswpin 197274 pswpout 624060 +15:40:22 full avg10=2.54 avg60=1.58 avg300=1.34 total=8069533 anon 2179461120 file 1922334720 1873776640 workingset_refault_anon 202865 pswpin 199612 pswpout 639226 +15:40:24 full avg10=3.89 avg60=1.85 avg300=1.40 total=8208381 anon 2180362240 file 1919160320 1873920000 workingset_refault_anon 203286 pswpin 200020 pswpout 639312 +15:40:26 full avg10=3.91 avg60=1.92 avg300=1.41 total=8484990 anon 2177998848 file 1924026368 1892077568 workingset_refault_anon 210517 pswpin 207142 pswpout 645594 +15:40:28 full avg10=5.56 avg60=2.28 avg300=1.49 total=8656485 anon 2108825600 file 1989308416 1930575872 workingset_refault_anon 211932 pswpin 208548 pswpout 657459 +15:40:30 full avg10=5.64 avg60=2.41 avg300=1.52 total=8716832 anon 2122145792 file 1971916800 1930854400 workingset_refault_anon 213791 pswpin 210388 pswpout 657569 +15:40:32 full avg10=5.34 avg60=2.46 avg300=1.54 total=8921295 anon 2118180864 file 1980985344 1948692480 workingset_refault_anon 216028 pswpin 212529 pswpout 659784 +15:40:34 full avg10=6.91 avg60=2.84 avg300=1.63 total=9137580 anon 2107707392 file 2000109568 1974226944 workingset_refault_anon 219998 pswpin 216461 pswpout 665599 +15:40:36 full avg10=6.38 avg60=2.87 avg300=1.64 total=9213522 anon 2100957184 file 2004856832 1993576448 workingset_refault_anon 222847 pswpin 219234 pswpout 669447 +15:40:38 full avg10=6.67 avg60=3.04 avg300=1.69 total=9404607 anon 2075275264 file 2028175360 2015838208 workingset_refault_anon 228327 pswpin 224669 pswpout 677555 +15:40:40 full avg10=5.82 avg60=3.01 avg300=1.69 total=9421787 anon 2078093312 file 2014978048 2015526912 workingset_refault_anon 229181 pswpin 225503 pswpout 677663 +15:40:42 full avg10=5.49 avg60=3.04 avg300=1.71 total=9573729 anon 2025185280 file 2072969216 2064064512 workingset_refault_anon 231993 pswpin 228214 pswpout 692314 +15:40:44 full avg10=6.49 avg60=3.30 avg300=1.77 total=9736793 anon 2023051264 file 2059718656 2065530880 workingset_refault_anon 235355 pswpin 231508 pswpout 693161 +15:40:46 full avg10=5.49 avg60=3.22 avg300=1.76 total=9831708 anon 2017460224 file 2087301120 2074292224 workingset_refault_anon 237874 pswpin 233974 pswpout 695918 +15:40:48 full avg10=5.40 avg60=3.28 avg300=1.79 total=9941218 anon 2014801920 file 2091048960 2079084544 workingset_refault_anon 238418 pswpin 234518 pswpout 697134 +15:40:50 full avg10=6.05 avg60=3.47 avg300=1.84 total=10066148 anon 2031009792 file 2072170496 2081452032 workingset_refault_anon 247990 pswpin 243754 pswpout 701826 +15:40:52 full avg10=5.14 avg60=3.39 avg300=1.83 total=10075229 anon 2033516544 file 2061938688 2081267712 workingset_refault_anon 251428 pswpin 247066 pswpout 701871 diff --git a/documentation/audits/night-2026-10-22/P4-R1214/r1214-b-capped.tsv b/documentation/audits/night-2026-10-22/P4-R1214/r1214-b-capped.tsv new file mode 100644 index 00000000..739f523e --- /dev/null +++ b/documentation/audits/night-2026-10-22/P4-R1214/r1214-b-capped.tsv @@ -0,0 +1,27 @@ +15:38:59 full avg10=0.87 avg60=3.86 avg300=1.59 total=7220110 anon 2155155456 file 1933524992 1882132480 workingset_refault_anon 158542 pswpin 156128 pswpout 606649 +15:39:01 full avg10=0.71 avg60=3.74 avg300=1.58 total=7234341 anon 2176008192 file 1922453504 1882083328 workingset_refault_anon 159499 pswpin 157036 pswpout 606649 +15:39:03 full avg10=0.58 avg60=3.61 avg300=1.56 total=7266319 anon 2173059072 file 1928216576 1882087424 workingset_refault_anon 159601 pswpin 157135 pswpout 607242 +15:39:05 full avg10=0.65 avg60=3.53 avg300=1.56 total=7287150 anon 2167078912 file 1926488064 1882349568 workingset_refault_anon 159703 pswpin 157201 pswpout 607982 +15:39:07 full avg10=0.72 avg60=3.44 avg300=1.56 total=7336362 anon 2170904576 file 1930149888 1882324992 workingset_refault_anon 160569 pswpin 158047 pswpout 607992 +15:39:09 full avg10=0.77 avg60=3.36 avg300=1.55 total=7356199 anon 2175414272 file 1923297280 1882296320 workingset_refault_anon 160637 pswpin 158104 pswpout 607992 +15:39:11 full avg10=0.81 avg60=3.29 avg300=1.55 total=7372570 anon 2172841984 file 1903226880 1882152960 workingset_refault_anon 161145 pswpin 158560 pswpout 607993 +15:39:13 full avg10=0.66 avg60=3.18 avg300=1.54 total=7381638 anon 2172796928 file 1903489024 1882152960 workingset_refault_anon 161160 pswpin 158560 pswpout 607993 +15:39:15 full avg10=0.54 avg60=3.07 avg300=1.53 total=7417217 anon 2178797568 file 1916338176 1882107904 workingset_refault_anon 162494 pswpin 159862 pswpout 607993 +15:39:17 full avg10=0.62 avg60=3.01 avg300=1.52 total=7429109 anon 2178826240 file 1916338176 1882099712 workingset_refault_anon 162515 pswpin 159868 pswpout 607993 +15:39:19 full avg10=0.51 avg60=2.91 avg300=1.51 total=7442053 anon 2179309568 file 1909116928 1882083328 workingset_refault_anon 162585 pswpin 159919 pswpout 607993 +15:39:21 full avg10=0.42 avg60=2.81 avg300=1.50 total=7450768 anon 2179932160 file 1905176576 1882066944 workingset_refault_anon 162682 pswpin 159995 pswpout 607993 +15:39:23 full avg10=0.34 avg60=2.72 avg300=1.49 total=7459779 anon 2180591616 file 1905315840 1882066944 workingset_refault_anon 162866 pswpin 160179 pswpout 607993 +15:39:25 full avg10=0.28 avg60=2.63 avg300=1.48 total=7474379 anon 2190475264 file 1896730624 1882017792 workingset_refault_anon 163514 pswpin 160816 pswpout 608009 +15:39:27 full avg10=0.23 avg60=2.54 avg300=1.47 total=7483721 anon 2183577600 file 1897766912 1881993216 workingset_refault_anon 163591 pswpin 160890 pswpout 608009 +15:39:29 full avg10=0.18 avg60=2.46 avg300=1.46 total=7490992 anon 2183839744 file 1897914368 1881993216 workingset_refault_anon 163657 pswpin 160955 pswpout 608009 +15:39:31 full avg10=0.15 avg60=2.38 avg300=1.45 total=7498777 anon 2184044544 file 1898643456 1881985024 workingset_refault_anon 163757 pswpin 161032 pswpout 608009 +15:39:33 full avg10=0.12 avg60=2.30 avg300=1.44 total=7515110 anon 2187968512 file 1898991616 1881899008 workingset_refault_anon 164702 pswpin 161960 pswpout 608009 +15:39:35 full avg10=0.10 avg60=2.22 avg300=1.43 total=7520357 anon 2187526144 file 1899126784 1881862144 workingset_refault_anon 164780 pswpin 162024 pswpout 608009 +15:39:37 full avg10=0.08 avg60=2.15 avg300=1.42 total=7533498 anon 2188107776 file 1900191744 1881817088 workingset_refault_anon 164899 pswpin 162123 pswpout 608009 +15:39:39 full avg10=0.06 avg60=2.08 avg300=1.41 total=7559069 anon 2189148160 file 1897832448 1881800704 workingset_refault_anon 164999 pswpin 162209 pswpout 608014 +15:39:41 full avg10=0.23 avg60=2.05 avg300=1.41 total=7565482 anon 2188591104 file 1880342528 1881784320 workingset_refault_anon 165052 pswpin 162256 pswpout 608036 +15:39:43 full avg10=0.19 avg60=1.98 avg300=1.40 total=7583622 anon 2188668928 file 1880240128 1881767936 workingset_refault_anon 165126 pswpin 162316 pswpout 608036 +15:39:45 full avg10=0.15 avg60=1.91 avg300=1.39 total=7587346 anon 2189262848 file 1878487040 1881767936 workingset_refault_anon 165233 pswpin 162414 pswpout 608036 +15:39:47 full avg10=0.12 avg60=1.85 avg300=1.38 total=7594985 anon 2181861376 file 1875628032 1881759744 workingset_refault_anon 165254 pswpin 162421 pswpout 608036 +15:39:49 full avg10=0.10 avg60=1.79 avg300=1.37 total=7599902 anon 2183262208 file 1658728448 1881718784 workingset_refault_anon 165522 pswpin 162621 pswpout 608036 +15:39:51 full avg10=0.08 avg60=1.73 avg300=1.36 total=7599902 anon 2173874176 file 1727365120 1881690112 workingset_refault_anon 165594 pswpin 162659 pswpout 608036 diff --git a/documentation/audits/night-2026-10-22/P4-R1214/r1214-b2-capped.tsv b/documentation/audits/night-2026-10-22/P4-R1214/r1214-b2-capped.tsv new file mode 100644 index 00000000..66edaec5 --- /dev/null +++ b/documentation/audits/night-2026-10-22/P4-R1214/r1214-b2-capped.tsv @@ -0,0 +1,19 @@ +15:41:12 full avg10=0.79 avg60=2.46 avg300=1.71 total=10149379 anon 2095677440 file 1990664192 2079645696 workingset_refault_anon 269199 pswpin 264419 pswpout 701950 +15:41:14 full avg10=0.83 avg60=2.41 avg300=1.71 total=10177124 anon 2121056256 file 1971712000 2079608832 workingset_refault_anon 271376 pswpin 266520 pswpout 701964 +15:41:16 full avg10=1.04 avg60=2.39 avg300=1.71 total=10226245 anon 2127192064 file 1965178880 2079539200 workingset_refault_anon 272795 pswpin 267881 pswpout 701969 +15:41:18 full avg10=0.85 avg60=2.32 avg300=1.70 total=10245591 anon 2128060416 file 1959108608 2079531008 workingset_refault_anon 272851 pswpin 267921 pswpout 701969 +15:41:20 full avg10=0.69 avg60=2.24 avg300=1.69 total=10264675 anon 2129645568 file 1951543296 2079518720 workingset_refault_anon 273146 pswpin 268182 pswpout 701969 +15:41:22 full avg10=0.57 avg60=2.17 avg300=1.67 total=10275758 anon 2130022400 file 1952010240 2079510528 workingset_refault_anon 273203 pswpin 268225 pswpout 701969 +15:41:24 full avg10=0.65 avg60=2.13 avg300=1.67 total=10304890 anon 2131419136 file 1958985728 2079481856 workingset_refault_anon 273444 pswpin 268422 pswpout 701970 +15:41:26 full avg10=0.71 avg60=2.09 avg300=1.66 total=10326741 anon 2135916544 file 1949175808 2079481856 workingset_refault_anon 274703 pswpin 269668 pswpout 701970 +15:41:28 full avg10=0.58 avg60=2.02 avg300=1.65 total=10346752 anon 2136711168 file 1938386944 2079481856 workingset_refault_anon 274779 pswpin 269737 pswpout 701970 +15:41:30 full avg10=0.47 avg60=1.95 avg300=1.64 total=10355602 anon 2136711168 file 1938386944 2079481856 workingset_refault_anon 274779 pswpin 269737 pswpout 701970 +15:41:32 full avg10=0.57 avg60=1.92 avg300=1.64 total=10376110 anon 2138894336 file 1950375936 2079481856 workingset_refault_anon 275388 pswpin 270320 pswpout 701970 +15:41:34 full avg10=0.46 avg60=1.86 avg300=1.63 total=10386198 anon 2143576064 file 1940713472 2079449088 workingset_refault_anon 276447 pswpin 271329 pswpout 701970 +15:41:36 full avg10=0.56 avg60=1.83 avg300=1.62 total=10409920 anon 2142969856 file 1941934080 2079395840 workingset_refault_anon 276569 pswpin 271431 pswpout 701970 +15:41:38 full avg10=0.64 avg60=1.80 avg300=1.62 total=10428296 anon 2143977472 file 1943191552 2079395840 workingset_refault_anon 276840 pswpin 271678 pswpout 701970 +15:41:40 full avg10=0.52 avg60=1.74 avg300=1.61 total=10438990 anon 2144059392 file 1937944576 2079395840 workingset_refault_anon 276858 pswpin 271695 pswpout 701970 +15:41:42 full avg10=0.43 avg60=1.69 avg300=1.59 total=10449658 anon 2143178752 file 1937944576 2079342592 workingset_refault_anon 276860 pswpin 271697 pswpout 701970 +15:41:44 full avg10=0.35 avg60=1.63 avg300=1.58 total=10462674 anon 2143178752 file 1936896000 2079309824 workingset_refault_anon 276860 pswpin 271697 pswpout 701970 +15:41:46 full avg10=0.28 avg60=1.58 avg300=1.57 total=10478707 anon 2126831616 file 1685921792 2079297536 workingset_refault_anon 277148 pswpin 271927 pswpout 701970 +15:41:48 full avg10=0.23 avg60=1.52 avg300=1.56 total=10478707 anon 2127224832 file 1685921792 2079289344 workingset_refault_anon 277152 pswpin 271931 pswpout 701970 diff --git a/documentation/audits/night-2026-10-22/P4-R1214/r1214-tier2-controller-log.txt b/documentation/audits/night-2026-10-22/P4-R1214/r1214-tier2-controller-log.txt new file mode 100644 index 00000000..f8142614 --- /dev/null +++ b/documentation/audits/night-2026-10-22/P4-R1214/r1214-tier2-controller-log.txt @@ -0,0 +1,9 @@ +2026/10/22 15:53:05 scheduler.go:150: [INFO] [scheduler] Daily job tier2-backup scheduled for 2026-10-23 03:30 CEST +2026/10/22 15:53:05 scheduler.go:85: [DEBUG] [scheduler] daily job registered: name="tier2-backup" schedule="03:30" nextRun=2026-10-23T03:30:00+02:00 totalJobs=14 +2026/10/22 15:53:05 scheduler.go:85: [DEBUG] [scheduler] daily job tier2-backup: next run at 2026-10-23 03:30:00 CEST (waiting 9h36m54s) +2026/10/22 15:53:46 auth.go:144: [DEBUG] [web] auth: valid session for POST /api/backup/tier2 +2026/10/22 15:53:46 router.go:78: [DEBUG] [api] POST /api/backup/tier2 (path=/backup/tier2) +2026/10/22 15:53:51 tier2.go:491: [INFO] [backup] Tier 2 copied immich → /mnt/felhom-drives/scratch_hdd/backups/secondary/immich (516.4 MB, 2 leg(s), 5s) +2026/10/22 15:53:53 tier2.go:491: [INFO] [backup] Tier 2 copied paperless-ngx → /mnt/sizing-bench/backups/secondary/paperless-ngx (127.4 MB, 1 leg(s), 1s) +2026/10/22 15:53:53 tier2.go:491: [INFO] [backup] Tier 2 copied privatebin → /mnt/felhom-drives/scratch_hdd/backups/secondary/privatebin (8.4 KB, 0 leg(s), 0s) +2026/10/22 15:53:53 tier2.go:553: [INFO] [backup] Tier 2 run complete: 3 app(s) processed (incl. volume-only — F6) diff --git a/documentation/audits/night-2026-10-22/P4-R1214/r1214-tier2-equality.txt b/documentation/audits/night-2026-10-22/P4-R1214/r1214-tier2-equality.txt new file mode 100644 index 00000000..6fef8e30 --- /dev/null +++ b/documentation/audits/night-2026-10-22/P4-R1214/r1214-tier2-equality.txt @@ -0,0 +1,3 @@ +R-1214 live tier2 proof, 9202, controller 0.340.0-rc.4, 2026-10-22 ~18:00 CEST +rsync -ani --delete source vs tier2 copy (run inside felhom-controller; the guest has no rsync): differences: 0 +files: source 44, copy 44 diff --git a/documentation/audits/night-2026-10-22/P4-R1214/r1214-tier2.tsv b/documentation/audits/night-2026-10-22/P4-R1214/r1214-tier2.tsv new file mode 100644 index 00000000..792d2a94 --- /dev/null +++ b/documentation/audits/night-2026-10-22/P4-R1214/r1214-tier2.tsv @@ -0,0 +1,62 @@ +15:53:45 full avg10=0.00 avg60=0.00 avg300=0.10 total=10551160 2 +15:53:49 full avg10=0.00 avg60=0.00 avg300=0.10 total=10563819 2 +15:53:52 full avg10=0.00 avg60=0.00 avg300=0.10 total=10565444 1 +15:53:55 full avg10=0.00 avg60=0.00 avg300=0.10 total=10565444 1 +15:53:58 full avg10=0.00 avg60=0.00 avg300=0.10 total=10565444 1 +15:54:01 full avg10=0.00 avg60=0.00 avg300=0.10 total=10565444 1 +15:54:04 full avg10=0.00 avg60=0.00 avg300=0.09 total=10565444 1 +15:54:07 full avg10=0.00 avg60=0.00 avg300=0.09 total=10565444 1 +15:54:10 full avg10=0.00 avg60=0.00 avg300=0.09 total=10565444 1 +15:54:13 full avg10=0.00 avg60=0.00 avg300=0.09 total=10565444 1 +15:54:16 full avg10=0.00 avg60=0.00 avg300=0.09 total=10565444 1 +15:54:19 full avg10=0.00 avg60=0.00 avg300=0.09 total=10565444 1 +15:54:22 full avg10=0.00 avg60=0.00 avg300=0.08 total=10565444 1 +15:54:25 full avg10=0.00 avg60=0.00 avg300=0.08 total=10565444 1 +15:54:28 full avg10=0.00 avg60=0.00 avg300=0.08 total=10565549 1 +15:54:31 full avg10=0.00 avg60=0.00 avg300=0.08 total=10565549 1 +15:54:34 full avg10=0.00 avg60=0.00 avg300=0.08 total=10565549 1 +15:54:37 full avg10=0.00 avg60=0.00 avg300=0.08 total=10565549 1 +15:54:40 full avg10=0.00 avg60=0.00 avg300=0.08 total=10565549 1 +15:54:43 full avg10=0.00 avg60=0.00 avg300=0.07 total=10565549 1 +15:54:46 full avg10=0.00 avg60=0.00 avg300=0.07 total=10565549 1 +15:54:49 full avg10=0.00 avg60=0.00 avg300=0.07 total=10565549 1 +15:54:52 full avg10=0.00 avg60=0.00 avg300=0.07 total=10565549 1 +15:54:55 full avg10=0.00 avg60=0.00 avg300=0.07 total=10565549 1 +15:54:58 full avg10=0.00 avg60=0.00 avg300=0.07 total=10567212 1 +15:55:01 full avg10=0.00 avg60=0.00 avg300=0.07 total=10567212 1 +15:55:04 full avg10=0.00 avg60=0.00 avg300=0.06 total=10567212 1 +15:55:07 full avg10=0.00 avg60=0.00 avg300=0.06 total=10567212 1 +15:55:10 full avg10=0.00 avg60=0.00 avg300=0.06 total=10567212 1 +15:55:13 full avg10=0.00 avg60=0.00 avg300=0.06 total=10567212 1 +15:55:16 full avg10=0.00 avg60=0.00 avg300=0.06 total=10567212 1 +15:55:19 full avg10=0.00 avg60=0.00 avg300=0.06 total=10567212 1 +15:55:22 full avg10=0.00 avg60=0.00 avg300=0.06 total=10567212 1 +15:55:25 full avg10=0.00 avg60=0.00 avg300=0.06 total=10567212 1 +15:55:28 full avg10=0.00 avg60=0.00 avg300=0.06 total=10567212 1 +15:55:31 full avg10=0.00 avg60=0.00 avg300=0.06 total=10567212 1 +15:55:34 full avg10=0.00 avg60=0.00 avg300=0.06 total=10567212 1 +15:55:37 full avg10=0.00 avg60=0.00 avg300=0.06 total=10567212 1 +15:55:40 full avg10=0.00 avg60=0.00 avg300=0.06 total=10567241 1 +15:55:43 full avg10=0.00 avg60=0.00 avg300=0.06 total=10567241 1 +15:55:46 full avg10=0.00 avg60=0.00 avg300=0.05 total=10567241 1 +15:55:49 full avg10=0.00 avg60=0.00 avg300=0.05 total=10567241 1 +15:55:52 full avg10=0.00 avg60=0.00 avg300=0.05 total=10567241 1 +15:55:55 full avg10=0.00 avg60=0.00 avg300=0.05 total=10567241 1 +15:55:58 full avg10=0.00 avg60=0.00 avg300=0.05 total=10567241 1 +15:56:01 full avg10=0.00 avg60=0.00 avg300=0.05 total=10567241 1 +15:56:04 full avg10=0.00 avg60=0.00 avg300=0.05 total=10567241 1 +15:56:07 full avg10=0.00 avg60=0.00 avg300=0.05 total=10567241 1 +15:56:10 full avg10=0.00 avg60=0.00 avg300=0.05 total=10567241 1 +15:56:13 full avg10=0.00 avg60=0.00 avg300=0.05 total=10567241 1 +15:56:16 full avg10=0.00 avg60=0.00 avg300=0.05 total=10567241 1 +15:56:19 full avg10=0.00 avg60=0.00 avg300=0.05 total=10567241 1 +15:56:22 full avg10=0.00 avg60=0.00 avg300=0.05 total=10567241 1 +15:56:25 full avg10=0.00 avg60=0.00 avg300=0.05 total=10567241 1 +15:56:28 full avg10=0.00 avg60=0.00 avg300=0.04 total=10567241 1 +15:56:31 full avg10=0.00 avg60=0.00 avg300=0.04 total=10567241 1 +15:56:34 full avg10=0.00 avg60=0.00 avg300=0.04 total=10567241 1 +15:56:37 full avg10=0.00 avg60=0.00 avg300=0.04 total=10567241 1 +15:56:40 full avg10=0.00 avg60=0.00 avg300=0.04 total=10567241 1 +15:56:43 full avg10=0.00 avg60=0.00 avg300=0.04 total=10567241 1 +15:56:46 full avg10=0.00 avg60=0.00 avg300=0.04 total=10567241 1 +15:56:49 full avg10=0.00 avg60=0.00 avg300=0.04 total=10567241 1 diff --git a/documentation/audits/night-2026-10-22/P4-R1214/r1214run.sh b/documentation/audits/night-2026-10-22/P4-R1214/r1214run.sh new file mode 100644 index 00000000..0bbed803 --- /dev/null +++ b/documentation/audits/night-2026-10-22/P4-R1214/r1214run.sh @@ -0,0 +1,9 @@ +#!/bin/bash +# r1214run.sh LABEL CMD... — runs on demo-hp: samples 9202's cgroup every 2 s while CMD runs (in 9202), prints a summary +L=$1; shift; CG=/sys/fs/cgroup/lxc/9202; OUT=/tmp/r1214-$L.tsv; : > $OUT +( while :; do echo -e "$(date -u +%T)\t$(sed -n 2p $CG/memory.pressure)\t$(grep -E '^(anon|file) ' $CG/memory.stat | tr '\n' ' ')\t$(cat $CG/memory.swap.current)\t$(grep -E '^(pswpin|pswpout|workingset_refault_anon) ' $CG/memory.stat | tr '\n' ' ')" >> $OUT; sleep 2; done ) & +SP=$!; F0=$(sed -n 2p $CG/memory.pressure | grep -o 'total=[0-9]*' | cut -d= -f2); T0=$(date +%s) +LC_ALL=C pct exec 9202 -- "$@"; RC=$? +T1=$(date +%s); F1=$(sed -n 2p $CG/memory.pressure | grep -o 'total=[0-9]*' | cut -d= -f2); sleep 4; kill $SP +PK=$(cut -f2 $OUT | grep -o 'avg10=[0-9.]*' | cut -d= -f2 | sort -n | tail -1) +echo "$L: rc=$RC seconds=$((T1-T0)) full-stall-total=$(( (F1-F0)/1000 )) ms peak full avg10=$PK % samples=$(wc -l < $OUT)" diff --git a/documentation/backlog/CLOSED-ITEMS.md b/documentation/backlog/CLOSED-ITEMS.md index 144ed778..9ae901c2 100644 --- a/documentation/backlog/CLOSED-ITEMS.md +++ b/documentation/backlog/CLOSED-ITEMS.md @@ -26,6 +26,14 @@ --- +## 2026-10-22 night — the seven decisions put into effect (full text of the rows: `git show 1100a344:documentation/backlog/OPEN-ITEMS.md`) + +| Row | What | Closed | Evidence | +|---|---|---|---| +| **R-206** | **The build-cache cap and the weekly prune exist only as a hand-edited `/etc/docker/daemon.json` on DooPlex — not in Ansible, so a rebuild loses them.** (P4) | CLOSED 2026-10-22 night — D8 ruled (operator 15:47: DooPlex Docker settings from git plus a weekly build-cache prune) and installed: homelab-manifests `b35ba83`, role `docker_daemon` with a `docker-buildcache-prune` service + timer (Sun 23:30, `docker builder prune --force --filter until=168h`, never `-a`, with `docker system df` before and after). The live `daemon.json` was identical, so no Docker restart (dockerd up since 2026-08-05); timer next run 2026-10-25 23:30. Part (b) — loosening the housekeeping ban — was not needed: the prune lives in its own role and unit. | homelab-manifests `b35ba83` CHANGELOG | +| **R-992** | **The drill-catalog route that `09` §6.5 documents cannot be followed: `admin/app-catalog-drill` is PRIVATE and no box has catalog credentials.** (P3) | CLOSED 2026-10-22 night — D6 ruled (operator 15:47: a read-only login for the scratch box's drill catalog). Gitea user `drill-reader` (not admin), read collaborator on `admin/app-catalog-drill`, token scope `read:repository`, kept in `sec`. Proven: the token reads the drill repo (200); write 403; push 403; the other private repos 404; 9202 cloned the drill catalog with it and returned to the live catalog with the credential kept. `09` §6.5 updated. | `audits/night-2026-10-22/P2-D6/01-token-limits.txt` | +| **R-1207** | **A host with no (or too little) swap gets nothing from the guest swap fix (R-1192), and the agent never adds host swap.** (P2) | CLOSED 2026-10-22 night — DECIDED, D27 = report only (operator 15:47): the agent's WARN, the hub's `host_swap_short` (D28 = A) and the install-page sentence tell; the agent adds no swap and the installer refuses nobody. Reopens at the first BYO box with too little host swap. | `audits/night-2026-10-21/P2-noswap/` | + ## 2026-10-22 day — the mail list and the swap rule (full text: R-1216 `git show 25dae2f5:documentation/backlog/OPEN-ITEMS.md`; R-1199's original row `git show 115f15c8:documentation/backlog/OPEN-ITEMS.md` — its reopened text of 2026-10-22 was never committed, and its substance is the Closed cell below) | Row | What | Closed | Evidence | diff --git a/documentation/backlog/OPEN-ITEMS.md b/documentation/backlog/OPEN-ITEMS.md index c997e300..6cbcb0e6 100644 --- a/documentation/backlog/OPEN-ITEMS.md +++ b/documentation/backlog/OPEN-ITEMS.md @@ -119,7 +119,7 @@ stopping line that lies. | **R-980** | Install & onboarding | P3 | **A person cannot check that the installer is ours: the ISO's first boot runs the install script from a movable tag with no hash or signature, and the ISO checksum sits on the same website as the download.** Fix: a signed manifest (or signature) checked by the ISO before it runs the script, and the public key printed in the guide. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN — design-proposal | — | Design note, then build | CC | | **R-250** | Install & onboarding | P3 | **A customer create can fail fail-closed because the host-key scan ladder is shorter than the DNS/AAAA settle time.** Found 2026-08-07 creating the fifth walk's venue. `POST /configs/new` with off-site enabled provisions the Storage Box sub-account and then scans its SSH host key to pin it — **fail-closed by design** (`offsite.go:111-121`, *"don't serve a descriptor the controller can't verify"*), with `defaultScanBackoff` = 2+4+8+16+30 ≈ **60 s**, sized by its own comment to *"the observed DNS propagation lag"*. **Measured, both halves:** the first create exhausted the ladder — five `no such host`, then `dial tcp [2a01:4f8:bacc:2:200::d30]:23: connect: network is unreachable` (the name had just begun resolving, **AAAA-first, into a pod with no IPv6 route**) — and the hub logged `[ERROR] offsite provision for walk5`. An **identical second POST succeeded** ~70 s later on its own final rung (`shared already provisioned … (subaccount 285351)`). **Total settle ≈ 100 s against a 60 s budget.** The endpoint was never the problem: verified afterwards from both the node and the hub pod, by name, `SSH-2.0-OpenSSH_9.6p1`. **Two distinct things are wrong and should not be merged:** (1) the budget is sized against DNS *existence*, but what actually bit is the **AAAA-before-A** window, a different and longer phenomenon — lengthen the ladder and/or prefer the A record for the scan dial; (2) **the operator is told nothing actionable** — the create fails with a generic error, the remedy is "press it again", and nothing says so. The retry is genuinely safe (idempotent on the label; confirmed afterwards — `one_time_secrets` 1, one sub-account, no double-provision) **but that safety is invisible to the person deciding whether pressing again will double-charge them.** **Severity LOW-MEDIUM:** self-clearing, no data at risk — but it is the first thing a new customer's provisioning does and it fails looking like an outage. | **VERIFY** — **2026-10-12: hub 0.147.0 live** (S29, decision 224): the host-key scan dials IPv4 first, waits ~2 min (was ~1), and when it still runs out says the store was created and it is safe to press Save again (`TestScannerDialsIPv4First`, `TestScanOutOfRetriesSaysSafeToPressAgain`, red-proved). Known limit: the hub Ingress reads for 60 s, so a very slow first try can show a gateway timeout while provisioning finishes in the background (pressing again is safe). **READY** — operator ruling 2026-10-12 (S29, `09` §3 decision 224): yes — the scan prefers IPv4, waits longer, and the error says „safe to press again". **READY** — owner Viktor | — | The next real off-site setup (a new customer): read the hub log for the scan; success or the new message → close | CC | | **R-516** | Install & onboarding | P3 | **[P3-LOW] English a customer meets on a fresh box and its apps' first screens — enumerated by the big night.** MEASURED 2026-09-14 (BIGNIGHT, VM 333, ISO 1.27.1, controller 0.242.0). Felhom-owned: (1) the dashboard menu item **„Debug"**; (2) the dashboard CPU tile **„Load: 0.29 / 0.39 / 0.37"**; (3) the launcher tile **„Filebrowser"** opens a login in English with no Felhom text (R-513); (4) the storage page mixes formal „Adjon hozzá / Csatlakoztasson" with the product's „te". App first screens a household meets before any Felhom text helps: (5) **Uptime Kuma 2.4 opens on „Which database would you like to use?"** (SQLite / Embedded MariaDB, „Next") — the app card's „Első lépések" does not mention it; (6) PrivateBin, Gokapi, AdventureLog and FileBrowser UIs are English (the apps' own). Already rows: the Proxmox installer screens (R-495, answered by the guide), `wiki.DOMAIN` (R-498). **Fix shape:** rename „Debug"/„Load" (controller); add the Uptime Kuma database step to its card, or pre-seed `db-config.json` for SQLite in the template (catalog). **Added by F4 (20:04:54Z):** (7) the storage page prints the disconnect time as a raw ISO UTC string „Leválasztva: 2026-09-14T19:58:02Z"; (8) the „Meghajtó leválasztva" banner appears twice on every page; (9) „4 telepített alkalmazás nem fut — nézze meg a rendszermonitort" uses the formal form. **Added by F7 (20:50–21:00Z, system disk at 95 %):** (10) a banner on every page in English, „**SSD disk usage high: 90%**"; (11) the dashboard tile reads „Rendszer (/) 61.8 GB / 68.7 GB (**90%**)" while `df` reports **95 %** (reserved blocks ignored), and „(/)" labels the data volume `/mnt/sys_drive`; the deploy page says nothing about free disk. **Added by the i18n spike (2026-09-17, controller v0.247.0):** (12) **six formal („ön") forms in the converted dashboard copy** — „Olvassa be telefonnal" (launcher QR hint), „Biztosan kikapcsolja a megosztást?" (launcher), „Ha újratelepíti … importálnia kell" (the layout's remove-app modal), „Kérjük, vegye fel a kapcsolatot" (backups empty state). They were NOT fixed — a localisation release may not change Hungarian bytes — and are now COUNTED by `controller/scripts/i18n_missing_gate.py` (`HU_FORMAL_CEILING` = 6, a ratchet: a new one convicts, fixing one lowers it). The inventory (`audits/I18N-INVENTORY-2026-09-17.md`) is the list this row closes against in localisation slice 6 (R-561). **Extended 2026-09-17 by slice 1 (controller v0.248.0–v0.250.0):** the converted copy now counts **16** formal forms (`HU_FORMAL_CEILING` 6 → 10 → 12 → 16, each raise stated, Hungarian unchanged by rule) — and the count is an UNDER-count: the gate's stem list sees 4 on the release C pages (storage, drive wizards, debug), while a wider list („írja be”, „adja meg”, „adjon hozzá”, „válassza ki”, „biztosan eltávolítja”, „engedélyezze”, „hozzon létre”, „kattintson”) finds at least **22** keys there alone (login „Adja meg a jelszavát”, the NAS guide, the storage confirms). Widen the stems when this row is worked; the ceiling rises with them. **NARROWED 2026-09-20 by localisation slice 6's walk, item by item** (`audits/i18n-slice6-2026-09-20/R-516-item-by-item.md`): item 2 FIXED (slice 1), items 3 and 6 APP-OWNED (and for an English household item 6 is an advantage), item 11 is a wrong NUMBER rather than a language and needs its own row, item 1 is correct English for an English reader and still open for a Hungarian one. **Items 4, 7, 8, 9 and 10 are NOT DECIDABLE FROM AN ENGLISH WALK** - they are about Hungarian copy quality, a disconnected drive and a full disk, none of which this walk had. **What this row is now waiting for is a HUNGARIAN walk on a box with a second drive**, not another English one; saying it closed would be the kind of closure that makes a register stop meaning anything. **-- 2026-10-08:** the recovery screen's own answers are Go literals in Hungarian only — 14 `renderRecovery` messages in `controller/internal/web/recovery_handlers.go` (seen: the safe default „A művelet nem fejeződött be…" rendered on an English page in a red-proof run). Today's new R-304 sentence is a bundle key in both languages; the 14 old ones are not. | **NARROWED 2026-09-20 - rank P3-LOW; owner: CC; needs a Hungarian walk** **2026-10-05 (burn-down night): items 1 and 4 FIXED on controller `main`** (`d7efa7b`): the menu says „Hibakeresés” (English „Debug”); the storage pages use the te-form (formal-form ceiling 17 → 14; 116 parity fixtures changed by exactly those bytes). Item 2 was already fixed. **Left:** items 5–6 (apps' own screens) and the rest. **2026-10-06: items 1 and 4 DELIVERED** (controller v0.298.0). **2026-10-06 (burn-down night, later): items 7–10 and 12 FIXED on controller `main`** (`e4774e6`, `2e5ef7e`): the disconnect time in local time; one banner per unplugged drive; „nézd meg”; the disk/memory/CPU/temperature banners in the household's language (`health.*` keys; the wire text to the hub unchanged and pinned); the last eleven counted formal forms are te-form (ceiling 14 → 0; 119 parity fixtures changed by exactly those bytes). **Left:** items 5–6 (apps' own screens); about 20 formal forms the gate does not count yet („írja be”, „adja meg”, „hozzon létre”, …) on the debug, storage, network-storage and security pages. Item 11 moved to its own row **R-889**. **2026-10-06 02:40: the bundle has NO formal form left** (controller `4c3c203`, unreleased): 49 sentences in the te-form; the gate's stem list widened (it counted 61 on the previous bundle, 0 now) with decoys and one listed third-person exception. **Left (narrowed):** Felhom-owned formal forms in Go and template literals the gate cannot read yet — the network-storage attach errors, escrow and share handlers, the single-copy notice, a settings refusal, the SMART mail, a fill-watch „Kérjük”, and the setup wizard pages; each moves into the bundle in the te-form. Items 5–6 are the apps' own screens (not ours). **2026-10-06: items 7–10, 12 and the bundle's 49 formal forms DELIVERED** in controller v0.299.0. **2026-10-06 night: the Go-literal leftover moved** (controller main `3d6ba28`, unreleased): 22 Go-literal messages — escrow handler (6), share password page (2), export upload (1), network-storage uid/remove/attach failures (13) — are bundle keys (`api.escrow.*`, `api.share.*`, `api.export.*`, `api.netstorage.*`) in the te-form with English (14 were formal); a detached attach failure renders in the reader's language; the NAS refusal says „Válassz". `TestR516_GoLiteralFormalFormsAreGone` (source scan + positive control), `TestR516_NetAddFailureFollowsTheReader`; red-proofs `audits/night-burndown-2026-10-06/ctrl/R-516-red*.txt`. **Left:** the setup wizard and recovery-info.txt (R-554); the SMART and fill-watch texts (the hub event's message — wire text, needs the household-copy producer shape); items 5–6 (apps' own screens). | — | — | CC | -| **R-991** | Install & onboarding | P3 | **Nothing a buyer can read says which machines Felhom installs on, and a quarter of the forum beginners own only machines it cannot install on.** FOUND 2026-10-12 (forum research, `audits/forum-research-2026-10-12/FELHOM.md` §2). Searched `documentation/architecture/`, `website/` (`gyik`, `technologiak`, `letoltes`) and `scripts/felhom-host-install.sh`: no CPU-architecture, RAM, disk or network floor is written anywhere. The download page says only that the installer is built on Proxmox VE 9.2 — x86-64 only (inferred from Proxmox, not stated on the page). Of the 33 forum beginners who name what they own, 8 own only a Synology/QNAP/other NAS box, a router with a USB disk or a Raspberry Pi. The selfh.st survey reading reached the same gap independently (R-989, its recommendation 1). **Needs (operator):** the floor itself (RAM and disk minimum; whether thin clients and old laptops with 4–8 GB are accepted) — then one short section on the download page and in the FAQ. Not changed here: the research task forbade website edits. | **WAITING-ON-OPERATOR** — **2026-10-20: R-993 merged here** (one decision, D5): the proposal to approve is `audits/sizing-2026-10-13/SIZING.md` §5 — minimum x86-64, 4 cores (N100 class), 8 GB, 256 GB SSD; recommended 16 GB, 512 GB SSD; out: ARM / Raspberry Pi, Synology-QNAP, routers, cloud VPS. R-993 also measured that the installer has no floor on the HOST (`autosize_guest_caps` limits only the guest), so whatever minimum is approved, nothing enforces it at install today (full text: `git show 5c487313:documentation/backlog/OPEN-ITEMS.md`). filed 2026-10-12 | the operator naming the floor | Operator: approve or change `SIZING.md` §5 (STATUS D5); then CC writes the section on `letoltes.html` and `gyik.html` (both languages) and files the install-time host floor as its own row if it is wanted. If nothing is done: a buyer with a Pi or a Synology downloads an ISO that cannot run | operator | +| **R-991** | Install & onboarding | P3 | **Nothing a buyer can read says which machines Felhom installs on, and a quarter of the forum beginners own only machines it cannot install on.** FOUND 2026-10-12 (forum research, `audits/forum-research-2026-10-12/FELHOM.md` §2). Searched `documentation/architecture/`, `website/` (`gyik`, `technologiak`, `letoltes`) and `scripts/felhom-host-install.sh`: no CPU-architecture, RAM, disk or network floor is written anywhere. The download page says only that the installer is built on Proxmox VE 9.2 — x86-64 only (inferred from Proxmox, not stated on the page). Of the 33 forum beginners who name what they own, 8 own only a Synology/QNAP/other NAS box, a router with a USB disk or a Raspberry Pi. The selfh.st survey reading reached the same gap independently (R-989, its recommendation 1). **Needs (operator):** the floor itself (RAM and disk minimum; whether thin clients and old laptops with 4–8 GB are accepted) — then one short section on the download page and in the FAQ. Not changed here: the research task forbade website edits. | READY — the website patch waits for the operator's publish. (was WAITING-ON-OPERATOR) — **2026-10-20: R-993 merged here** (one decision, D5): the proposal to approve is `audits/sizing-2026-10-13/SIZING.md` §5 — minimum x86-64, 4 cores (N100 class), 8 GB, 256 GB SSD; recommended 16 GB, 512 GB SSD; out: ARM / Raspberry Pi, Synology-QNAP, routers, cloud VPS. R-993 also measured that the installer has no floor on the HOST (`autosize_guest_caps` limits only the guest), so whatever minimum is approved, nothing enforces it at install today (full text: `git show 5c487313:documentation/backlog/OPEN-ITEMS.md`). filed 2026-10-12 **2026-10-22 night — D5 ruled (operator 15:47: approve the minimum machine).** The hardware section for `letoltes.html` and `gyik.html`, both languages, is in the same ready website patch as D3 (`audits/night-2026-10-22/P2-D3-D5/website-D3-D5.patch`). Not filed: an install-time host floor — R-993 measured that nothing enforces one; whether to refuse a too-small host is a new question for after the publish. | — | Operator: say „publish”; CC pushes the patch; then close. Ask then whether the installer should refuse a host below the floor | CC | | **R-494** | Install & onboarding | P4 | **NARROWED 2026-09-14 by operator ruling → [P3-LOW] the hub COULD create the tunnel at customer creation, for a domain already on Cloudflare. Not blocking: every customer has their own domain and the operator creates the tunnel per day-0 A.1 (`architecture/01-topology-and-trust.md`).** *Original finding, kept:* **[P1-HIGH] A new customer's dashboard has NO reachable address unless the operator hand-makes a Cloudflare tunnel — the link in the setup-code mail is dead.** MEASURED 2026-09-14 on a fresh install from the public ISO (drill intervention **I1**): the claim mail points at `https://felhom.drill0242.felhom.eu`; that name has **no A and no AAAA** record (`dig @1.1.1.1`, control `felhom.enkisfelhom.hu` resolves); the hub has **no tunnel- or DNS-creation code** (`hub/internal/cloudflare/` holds only geo-rule removal; `cf_tunnel_token` is a pasted, optional form field, `configs.go:1478`) — day-0 runbook A.1 makes it a manual Cloudflare-dashboard step that nothing on the customer-create page asks for; the box's own split-horizon resolver on the appliance LAN IP answered `google.com` but not the dashboard name at 13:27:39Z; the agent applied the record at **13:27:44Z** (`lanresolver: applied split-horizon record … ip=192.168.0.158`, 3 m 46 s after the controller started), so the box CAN answer the name — **but only to a device that uses the box as its DNS server, and no document, screen or mail tells a household to do that**; the router and the installer-offered DNS answer nothing. The page was reachable only at the guest's LAN address with the name forced (`curl --resolve …:443:192.168.0.158`). **A volunteer could not have done that.** **What it needs:** an operator ruling — the hub creates the tunnel and DNS at customer creation, or the product gives a household a LAN address that works with no DNS change. | **READY — rank P3-LOW; owner: CC** **Re-ranked 2026-10-03: P3→P4: operator-side automation; the operator creates the tunnel by hand per the day-0 runbook.** | — | — | CC | | **R-504** | Install & onboarding | P4 | **[P3-LOW] `iso.felhom.eu` cannot show an index page on its own — its root returns 404, and the download page lives on the website instead.** MEASURED 2026-09-14: `https://iso.felhom.eu/` and `/index.html` → 404; only named objects answer. The host is an R2 bucket behind a custom domain; whether R2 would serve an uploaded `index.html` at `/` was **not measured** (uploading anything to the public bucket is a publication). The ISO v1.27.0 task puts the Hungarian download page at `felhom.eu/letoltes` (published with the ISO, after the operator's yes). **Remaining:** a redirect from `iso.felhom.eu/` to that page needs a Cloudflare rule the session has no credential for. | **WAITING-ON-OPERATOR — rank P3-LOW; owner: operator (Cloudflare rule)** **Re-ranked 2026-10-03: P3→P4: households are sent to the website's download page; the bare address is cosmetic.** | — | — | operator | @@ -141,7 +141,7 @@ stopping line that lies. | **R-1067** | Apps & catalog | P4 | **The catalog still carries `pi_compatible:` on every template, and nothing reads it any more.** controller v0.315.0 removed the „Pi kompatibilis" badge and the „csak x86" tag from every page (Raspberry Pi is no longer supported, so one was noise and the other was a warning on almost every app). The `ResourceHints.PiCompatible` struct field and the catalog key were deliberately LEFT: a box still on 0.314.0 reads the key, and removing it would make that box show „csak x86" on every app. **Gated on the floor:** once the fleet floor is at or above 0.315.0, drop `pi_compatible:` from the templates and the field from `stacks.ResourceHints` in the same pair of commits. Harmless until then — an unread key costs nothing. | **BLOCKED** — filed 2026-10-15. Checked 2026-10-16 (apps burndown): the floor is 0.321.0 and demo-hp, Tester 1 and the N100 run 0.321.0, but **Tester-2 still reports 0.292.0** (its newest report 2026-10-04; hub DB with WAL) — so nothing removed. | every box on ≥ 0.315.0 — today Tester-2 | Drop the catalog key and the struct field; grep the hub and the website for Pi wording at the same time (none found 2026-10-15) | CC | | **R-1043** | Apps & catalog | P4 | **Sonarr, Radarr and Prowlarr build their login redirect as an absolute `http://` URL.** They trust forwarded headers only from their „Trusted networks” (the household set `192.168.0.0/24`; traefik is `172.18.x`), so they ignore `X-Forwarded-Proto: https` and answer `Location: http:///login` to an https visitor. Since controller v0.312.0 that hop is sent back to https by traefik (a GET, no password), so it is cosmetic; a template could seed traefik's network as trusted. (`audits/https-only-2026-10-14/``1A-3-4.txt`) | **OPEN** — filed 2026-10-14 | — | Seed the docker network as a trusted proxy in the *arr templates (with the R-1030 seed), measure the redirect | CC | | **R-1099** | Apps & catalog | P4 | **The N100 may hold RomM games in the old upload place, now shown under a fake „roms” platform.** Until 2026-10-16 RomM's own upload wrote to the drive's `roms/roms//`; after the mapping fix (catalog `fb18471`, R-1089) those files are at RomM's `library/roms/roms/…`. Measured on 9201: such a game showed as missing in its platform and again under a platform „roms” (`audits/romm-ghost-2026-10-16/B/b-after.txt`). Files put in with the file manager are not affected — they are now found. The N100 runs RomM (hub `/apps/romm`); CC does not log in to the lived-in box. | OPEN — filed 2026-10-16 | Operator: look at `roms/roms/` on the N100's drive in the file manager | If it holds platform folders: move them up one level into `roms/`, then run „Scan” in RomM; if it is empty, close | Operator | -| **R-1215** | Apps & catalog | P4 | **One Immich photo upload hung 10 minutes with no answer and was not stored, while the box was healthy.** Night soak 2026-10-21 (`audits/night-2026-10-21/P1-soak/08-load/proof.log`): cycle 36, `IMG_20120021.jpg` sent 19:38:30 UTC, the uploader gave up at its 600 s read limit; Immich's database holds 302 photos for that user, the other cycles 303 (`P1-soak/09-immich-cycle36.txt`). At that moment memory „full" pressure was 3.9 % and `/api/health` answered in 0.26 s. 1 of about 26 000 uploads that night. Not diagnosed: Immich restarted at 02:16 for the off-site dump, so its log of 19:38 is gone. The Immich phone app retries a failed upload. | OPEN | — | Watch the next long upload run with the Immich server log saved; file upstream only with a log | CC | +| **R-1215** | Apps & catalog | P4 | **One Immich photo upload hung 10 minutes with no answer and was not stored, while the box was healthy.** Night soak 2026-10-21 (`audits/night-2026-10-21/P1-soak/08-load/proof.log`): cycle 36, `IMG_20120021.jpg` sent 19:38:30 UTC, the uploader gave up at its 600 s read limit; Immich's database holds 302 photos for that user, the other cycles 303 (`P1-soak/09-immich-cycle36.txt`). At that moment memory „full" pressure was 3.9 % and `/api/health` answered in 0.26 s. 1 of about 26 000 uploads that night. Not diagnosed: Immich restarted at 02:16 for the off-site dump, so its log of 19:38 is gone. The Immich phone app retries a failed upload. | WATCHING — **2026-10-22 night:** not reproducible tonight; the Immich server log of 19:38 is gone (Immich restarted at 02:16 for the off-site dump). What the household sees: the upload fails after 10 minutes and the Immich phone app sends it again. | — | Watch the next long upload run with the Immich server log saved; file upstream only with a log | CC | ## App updates — 4 rows (P3 4) @@ -153,14 +153,14 @@ stopping line that lies. | **R-683** | App updates | P3 | **[P3-LOW] Watch: after a power cut during an update's health check, the hold named an HOUR-OLD second-drive copy, not the one the update's own backup should have just made.** 2026-09-24 chaos round 3 (nextcloud, `backup_max_age: 1m`): no `backing-up` phase was seen and the hold named Tier 2 at 13:04 for an update pressed at 14:04; the pre-cut controller log was lost with the container (the runner now saves it at arm time — R-320). Round 11, the same action without a power cut, named a fresh 14:34 copy and logged the Tier-2 copy. The sentence was TRUE (it named the copy it offered); the question is why the update did not back up first. Not reproduced; watch the next power-cut drill. `audits/night-2026-09-24/E/round-03*.json`, `E/round-11-controller-pre.log` | **OPEN — P3; owner: CC (watch)** | — | — | CC | | **R-785** | App updates | P3 | **[P3-LOW] SparkyFitness is pinned 11 releases and a major behind upstream (v0.17.3; upstream v1.7.3, v1.6.0 dated 2026-07-24).** READ 2026-10-01 (`audits/visitors-2026-10-01/C/bench/C1-previous-tag.txt`). **Needs:** an update walk 0.17 → 1.x through the ladder (bench + box), after R-784 is decided. | **OPEN — rank P3-LOW; owner: CC (after R-784)** | — | — | CC | -## Backup & restore — 38 rows (P2 5, P3 20, P4 13) +## Backup & restore — 42 rows (P2 9, P3 18, P4 15) | ID | Category | Sev | What | State | Blocked on | Next action | Owner | |---|---|---|---|---|---|---|---| | **R-232** | Backup & restore | P2 | **DooPlex's backup makes every copy inside the same box — and nothing tells anyone when it fails.** Surveyed read-only 2026-08-06 (`audits/RECON-dooplex-backup-2026-08-06.md`). **What works:** five sets, 14/14 successful runs in 14 days; a file was restored from the `data` repo and matched the live original **byte for byte**; every set except two is cross-disk; k3s is integrity-checked on every run. **What the matrix exposes, ranked:** (a) **`notify_failure` is a no-op** — `NOTIFY_ON_FAILURE=true` but `NOTIFY_WEBHOOK_URL` is commented out, so a failed backup notifies **nobody**; the project already has a working Resend path that CI uses. Cheapest item, and it makes every other failure visible. (b) **Nothing leaves the box** — no rclone, no remote repo, no off-site target anywhere; Longhorn's target is `nfs://192.168.0.180:` pointing at DooPlex itself, and the only outbound-looking cron pulls *inbound* from Hetzner for a different project. The machine that runs the hub managing the customers' off-site chain has no off-site copy of its own. (c) **The backup tree is a single writable path** and the restic repos are not append-only — one bad script or ransomware destroys every copy at once. (d) **Two same-disk sets**: `.claude-memory` and the PostgreSQL dumps, whose source directory sits *inside* the backup tree. (e) **Longhorn `retain=1`** — one generation per volume, so a corruption noticed a day late has no earlier copy. (f) **`/opt/backup/docs/BACKUP-RESTORE.md` does not exist** though the systemd unit advertises it. (g) **`secrets/restic-repo` has never held a snapshot** — `backup-secrets.sh` contains no `restic` call; the secrets are GPG files on `sda1` only. (h) **No restore has ever been run** beyond today's single-file probe — the matrix's "ever demonstrated?" column is otherwise entirely empty. **Not a finding:** the restic passphrase. The on-box copy is on `sdb1`, a different disk from the backups, and the **operator holds an offline copy out of band** — so a disk loss is recoverable. The narrow residual is that it is operator-held rather than system-held, unlike the customer case's hub-vaulted escrow, so it should be confirmed current and findable by someone else. **Nothing was changed by the recon.** | **NARROWED 2026-10-08 — owner Viktor.** (a) DONE with the operator's yes in chat: `notify_failure` now mails admin@felhom.eu through Resend; proven by one test mail that reached the inbox (`audits/day-2026-10-08/r232/`; no backup was started). (b) partly: the hub database leaves DooPlex nightly to ep0 (R-173); everything else stays on the box. (c)–(h) unchanged. **2026-10-09 (operator yes in chat): (b) and (h) DONE for Gitea and the secrets.** Nightly 00:20 `felhom-dooplex-offsite` pushes Gitea (614 MB of repositories, the `gitea` database dump taken first, `app.ini`) and the nightly GPG secrets export, encrypted with a NEW key, to ep0 `operator` (`host/dooplex-gitea`) on the hub-DB write-only token — no change on ep0; first push 57 s; restore test weekly (manifest, `git fsck` every repo, `pg_restore --list`), run once: 27 805 files, 10 repos OK; `DooplexGiteaOffsiteStale`/`DooplexGiteaRestoreTestStale` in force (`absent()` included, promtool + 2 red-proofs); every backup unit now mails on failure (`OnFailure=`, dry run reached the inbox). **First real Gitea restore:** into a throwaway on the bench, 10/10 repos, product `main` = live, a file byte for byte, a login (`runbooks/gitea-restore.md`). The hub came back too (R-173, closed). **The registry is left out on purpose:** its images rebuild from the code. **Correction to the recon §7:** the API mirror holds ONE repo (`homelab-manifests`), not all — Gitea had a single copy, on DooPlex. `audits/dooplex-survival-2026-10-09/` **LEFT:** (c) append-only for the on-box tree, (d)–(g) as written; the paper copy of the new key (operator). **READY** for the rest | — | — | operator | | **R-895** | Backup & restore | P2 | **The hub's clean-up-window check trusts the snapshot counts the box sends, so a broken-into box (or past-dated fakes added through the add-only key) can shrink the real off-site history without an alarm.** READ 2026-10-06 night in source (R-822's design): the before/after comparison uses counts the box itself reports (`hub/internal/offsitekeys/service.go:284`, `:343`); new fakes keep the count level. Decision 68 already accepts a box-trusted count. | **NARROWED 2026-10-18 — slice 2 built, not deployed** (hub v0.158.0): the 07:10 audit compares the `snapshots/` listing with yesterday's by name — files gone that no clean-up window explains → `offsite_snapshots_vanished` (error), files no reported box run explains → `offsite_snapshots_unexplained` (warning); operator-only, hub-origin (not in `allowedEventTypes`). First listing = baseline; a move-aside or RESET drops it; vanishing judged on a pinned tier only; files younger than 1 h judged the next day. Red-proofed twice (`snapshot_watch_test.go`). **Not built:** the window's before/after on the hub's own count (design §3.1) and refusing a window while unexplained files exist (§3.2). Unmeasured: the provider shell's `ls -l` time zone (± 3 h slack until then). **Deployed 2026-10-18 12:23 (hub v0.158.0).** The first listing (tomorrow 07:10) is a baseline; the first verdict is 2026-10-20 07:10. **Known limit (independent review 2026-10-18, F2):** `_unexplained` is explained by the box's OWN reported `offsite.last_run`, so whoever holds a box's key can silence it; `_vanished` is judged on the hub's own listing and is not affected. **2026-10-20 03:5x UTC: due date moved 10-20 → 10-21** — the first verdict lands at 07:10 hub time (05:10 UTC), after the night session that reached it ended; that brief did not include this read. Nothing is lost: the hub keeps its listing; read the 2026-10-20 07:10 lines on 10-21 (or later on 10-20). **FIRST VERDICT READ 2026-10-21 (rehearsal session, read-only):** the 07:10 CEST run judged three tiers against yesterday — demo-felhom 5, demo-hp 14, tester-1 4 new snapshot files, „each inside a reported box run"; nothing vanished, nothing unexplained; Tester4 stored its first baseline (44 files); Tester-2's own listing failed (no `snapshots/` dir — the box has never pushed). No `offsite_snapshots_vanished` / `_unexplained` mail in the operator mailbox (second channel). Evidence `audits/due-checks-2026-10-21/R-895-hub-0710.txt`. The positive alarm path is still proven only by the unit red-proofs. | the operator's yes to deploy hub v0.158.0 | Deploy; read the first 07:10 lines (newest upload vs the box's last run = the time-zone calibration), shrink the slack; then decide §3.1/§3.2 | CC | | **R-923** | Backup & restore | P2 | **The password manager (Vaultwarden) runs on DooPlex and holds the keys that open DooPlex's own off-site copies — a loss of DooPlex loses the keys with the copies.** Operator finding, 2026-10-09 11:19: "the password manager also runs on DooPlex (Vaultwarden), so I probably need to back up its DB somewhere too." Measured: `vaultwarden-system/vaultwarden` (1.37.3), PVC `vaultwarden-data` 7.4 MB (`db.sqlite3` + WAL, `rsa_key.pem`, no attachments); its only backups were Longhorn on DooPlex. **A key in a password manager that runs on DooPlex is not off DooPlex** — R-173's "both keys are off DooPlex" (2026-10-05) and the hub-DB runbook Step 0 were wrong in that sense. Also seen: Vaultwarden warns its `ADMIN_TOKEN` is plain text (an Argon2 hash is its guidance). | **NARROWED 2026-10-09 (operator yes in chat):** Vaultwarden's own `backup` copy + `rsa_key.pem` ride the nightly encrypted DooPlex→ep0 job (`scripts/dooplex-offsite/`), checked on every push and every Sunday (integrity, users, items — rows only). That copy opens only with the DooPlex off-site key, and the vault only with the master password — so the circle is broken only once both are on paper outside DooPlex: `runbooks/break-glass-sheet.md`, `runbooks/total-loss-of-dooplex.md`. **LEFT:** the operator prints the sheet and stores it away from home; the `ADMIN_TOKEN` hash (a DooPlex change, operator's word). | operator: print the sheet | Operator: print + store the sheet; then close | operator | -| **R-49** | Backup & restore | P3 | **[P2] The offsite capture set is ~90% cache and duplication — 1.1 GB of a 1.2 GB immich "photo backup".** Measured 2026-07-19: `immich_ml_cache.tar` **823 660 032 B (~60%)** — re-downloadable ML model weights; `immich_postgres_data.tar` **308 251 136 B (~23%)** — a raw tar of the postgres data dir that DUPLICATES the logical `.sql` dump captured beside it; `upload/backups/` **18 MB** — immich's own nightly dump, a backup inside the backup, growing daily; plus the stranded pre-v3 `dccc13fe…` tree (~36 MB) no DB has ever referenced. Actual irreplaceable content: **72 MB of originals**. **MIGRATED FROM `ROADMAP.md` 2026-08-22 (R-369) — originally filed 2026-07-19, size S–M, roadmap state `idea`.** Moved verbatim; nothing added or reinterpreted. The roadmap keeps its copy as history, marked moved. | **OPEN — migrated from ROADMAP 2026-08-22, rank unchanged** **Re-ranked 2026-10-03: P2→P3: wasted space and transfer, no data risk; needs a capture-set ruling, not a sale blocker.** **READY 2026-10-20 — measured and planned, nothing changed; the decision is the operator's (STATUS D21).** Re-measured on 9202 (Immich v3.2.4): off-site ~1.33 GB = model cache 824 MB (62 %), raw database folder 283 MB (21 %), Immich's own backups 115 MB (9 %), logical dump 54 MB, originals 49 MB, thumbnails 2 MB — 102 MB must survive. A: leave out all three re-fetchable parts (−92 %; needs a never-run restore from the dump alone on the vectorchord image). B: only the model cache (−62 %; needs one restore with an empty cache). Pick: B now, A's other parts after their proof (`audits/night2-2026-10-20/P4/immich-offsite-plan.md`). | — | **Evidence: `audits/DIAG-immich-restore-round2-2026-07-19.md` §4 (full byte breakdown).** This is the customer's offsite quota and transfer cost, and it lands on the Hetzner sub-account they are billed for. **Recorded, deliberately not changed** — a capture-set exclusion is a data-loss-shaped decision and gets its own ruling, not a drive-by edit. Candidates in priority order: (a) `immich_ml_cache` — pure cache, strongest case; (b) the `postgres_data` volume tar where a logical dump of the same DB is already captured (the dump is what the restore path actually replays); (c) `upload/backups/`. Likely generalises past immich into a template-classification rule about cache volumes and self-backup directories, so it should be specified against the catalog, not one app | CC | +| **R-49** | Backup & restore | P3 | **[P2] The offsite capture set is ~90% cache and duplication — 1.1 GB of a 1.2 GB immich "photo backup".** Measured 2026-07-19: `immich_ml_cache.tar` **823 660 032 B (~60%)** — re-downloadable ML model weights; `immich_postgres_data.tar` **308 251 136 B (~23%)** — a raw tar of the postgres data dir that DUPLICATES the logical `.sql` dump captured beside it; `upload/backups/` **18 MB** — immich's own nightly dump, a backup inside the backup, growing daily; plus the stranded pre-v3 `dccc13fe…` tree (~36 MB) no DB has ever referenced. Actual irreplaceable content: **72 MB of originals**. **MIGRATED FROM `ROADMAP.md` 2026-08-22 (R-369) — originally filed 2026-07-19, size S–M, roadmap state `idea`.** Moved verbatim; nothing added or reinterpreted. The roadmap keeps its copy as history, marked moved. | READY — waits for the controller 0.340.0 release. **OPEN — migrated from ROADMAP 2026-08-22, rank unchanged** **Re-ranked 2026-10-03: P2→P3: wasted space and transfer, no data risk; needs a capture-set ruling, not a sale blocker.** **READY 2026-10-20 — measured and planned, nothing changed; the decision is the operator's (STATUS D21).** Re-measured on 9202 (Immich v3.2.4): off-site ~1.33 GB = model cache 824 MB (62 %), raw database folder 283 MB (21 %), Immich's own backups 115 MB (9 %), logical dump 54 MB, originals 49 MB, thumbnails 2 MB — 102 MB must survive. A: leave out all three re-fetchable parts (−92 %; needs a never-run restore from the dump alone on the vectorchord image). B: only the model cache (−62 %; needs one restore with an empty cache). Pick: B now, A's other parts after their proof (`audits/night2-2026-10-20/P4/immich-offsite-plan.md`). **2026-10-22 night — D21 = B built and proven, waits for the controller release.** A template may now exclude a named volume from every backup (`.felhom.yml` `backup.volumes`, class `excluded`; controller main `6a17c95`, unreleased, rides 0.340.0; guarded: only a declared name, never all of an app's volumes; the old tar is removed by the stale clean-up). The catalog's Immich template carries it (`ce33e3b`, live, inert before 0.340.0). On 9202: Immich's backup unit 1 160 553 832 → 336 788 031 B; with the cache volume removed, Immich started and its ML answered (HTTP 200, 512-dim) after 9.2 s, the cache refilled to 611 MB (`audits/night-2026-10-22/P2-D21/`). | — | Release controller 0.340.0 (STATUS morning list); close then. A's other parts (the raw DB folder, Immich's own backups) after their restore proof — a new row when started | CC | | **R-314** | Backup & restore | P3 | **`StopAbandon` has no web route — a customer who telephones is served by a command line.** `--abandon-stop` exists on the controller binary (`cmd/controller/main.go:86`) and refuses rather than silently no-opping, which is right. But there is no handler: the only in-product way to cancel a countdown is the customer finding their recovery code (Scenario G cancels it at the moment the code proves they still have it). **An operator who is telephoned instead has to reach a shell on the customer's machine.** Used this session on the operator's ruling, container stopped first so the running controller could not overwrite `settings.json` from memory — a sequencing subtlety that is itself an argument for a route **-- 2026-10-08 design:** `audits/day-2026-10-08/design-R-314-279-177.md` — the box's own 14-day countdown has no operator door (the hub's 7-day phase does, decision 74); one door for R-314 + R-279: a fixed list of operator actions carried in the report reply. Question D1 on STATUS's decision sheet. **-- 2026-10-08 14:16 operator ruling D1 (`09` §3 decision 185):** yes — option A, a fixed list of operator actions carried in the box's report reply. | **VERIFY — 2026-10-09: DELIVERED (hub 0.144.0 + controller 0.304.0); `run_job fill-watch` proven live on the Tester 1 box (hub row „done — ran to its end"; control: the box log „checked 3 filesystem(s)"; an unknown action is refused 400). `abandon_stop` / `abandon_extend` NOT yet proven — no box has a deletion countdown; prove them on the first one.** **VERIFY — built on main 2026-10-08 (D1, decision 185), ships with tomorrow's controller + hub releases; closes after the live proof on 9202** (`abandon_stop` / `abandon_extend` through the hub's buttons). **READY (S) — NEW 2026-08-12, RANK 3** **2026-10-05 (burn-down night): NEEDS A DESIGN.** There is no operator door into a RUNNING controller (its HTTP surface is the household's session; the CLI runs in a separate process and cannot safely trigger a job in the running one). Same design as R-177 and R-279. | R-241, R-307 | An operator-authenticated POST that calls the same `StopAbandon`, so the telephone path and the code path converge | CC | | **R-401** | Backup & restore | P3 | **Revisit the off-site integrity depth when a real store is LARGE — the default rests on ONE measurement, on ONE 134 MB store.** Controller v0.228.0 (R-399) made `--read-data-subset=100%` the default for every box. The whole justification is a single data point: `demo-hp`, 2026-08-30, 140 829 678 B / 2 651 blobs / 67 snapshots, structure 35.0 s vs 100% **39.2 s** — four seconds. Re-proven live 2026-08-31 at 38.7 s. **It does not extrapolate, and the reason is structural: the structure check's cost tracks the INDEX, a read-data run's tracks the DATA.** A 50 GB store is ~370x the data and this curve says nothing about it. **Nothing was invented from that one point** — no rotation schedule, no size threshold, no bandwidth budget — because four production designs in this project were specced against unvalidated mechanisms and all four were wrong. **`readDataSubsetRe` already accepts `n/m`**, so a rotating schedule (`1/7` on a different seventh each week) needs no parser work when the time comes; the missing input is a measurement on a large store, not code. **THE TRIGGER IS AN EVENT, NOT A DATE:** the slow-check WARN from v0.228.0 firing on any box (`integritySlowNoticeThreshold`, 5 min) — that line names the duration, the depth and this row. **WHAT HAPPENS IF NOBODY ACTS:** every box re-reads its entire store every week, however large it grows, and the first person to notice is a customer whose upload is saturated. **Whoever acts must also revisit `integrityCheckTimeout` (30 min)**, which is now the number a large store meets first. | **OPEN — WATCHING** | — | When the WARN fires: measure the curve on that store, then choose between a rotation (`n/m`), a size-conditional default, or leaving it. Do NOT choose from this row's numbers — they are the small-store case. | CC | | **R-409** | Backup & restore | P3 | **Nothing in the product can vouch for the bytes of a restored recovery unit — the only hash record covers 0.002 % of it.** MEASURED on demo-hp 2026-08-31 against kimai's restored unit: `manifest.json`'s `checksums` object carries sha256 for `.felhom.yml` (2 235 B), `app.yaml` (488 B) and `docker-compose.yml` (2 195 B) — **4 918 bytes of a 213 231 242-byte unit**. The database dump (48 217 B) and the two named-volume tars (160 331 776 B + 52 845 056 B) — the recoverable data, 99.998 % of the bytes — have no recorded hash anywhere. **And nothing else supplies one:** restic 0.14.0's `restore --verify` is a size-and-mtime reconciliation (a size-and-mtime-preserving one-byte corruption of a 160 MB tar passed clean, red-proofed), and `restic ls --json` file nodes in 0.14.0 carry name, size, mode, uid/gid and three timestamps and **no content hash**. **So "the restore produced correct files" is currently unanswerable by any automated means.** **What is NOT claimed here:** `restic check --read-data-subset=100%` already proves the STORE's packs, and the config files that ARE hashed are the ones a wrong-content failure would be hardest to spot in. | **OPEN — MEDIUM** | R-87, R-361 | Cheapest fix, and it is already half-built: extend the capture's `checksums` to cover `db_dumps` and `volume_dumps` — R-361 already computes a canonical dump sha256 to prove itself, so the value exists at capture time. Then a restore-test has a real reference and R-87's narrow version becomes a content check rather than a completeness one. Evidence: `audits/SPIKE-restic-restore-test-2026-08-31.md` §Q2, §Q3. | CC | @@ -185,16 +185,20 @@ stopping line that lies. | **R-1184** | Backup & restore | P3 | **A cut-off shared-folder restore can be „placed" and reported restored while files are missing; the download is then deleted.** Readiness is only „the scratch folder is not empty" (`internal/backup/shares_restore.go:131-140`) — the R-358 class fixed for apps, not for shares; 30-min download window (`web/offbox_handlers.go:716`); fixed 2 GiB free-space gate that fails open (`:107`). CONFIRMED in code. Data-loss hunt 2026-10-20 (C1 reviewer 1, R3). | OPEN | — | Fix: a completion marker after restic succeeds, as for apps | CC | | **R-1185** | Backup & restore | P3 | **A database dump that fails its own content check has already replaced last night's good dump, and the second drive copies it an hour later.** `appbackup/dbdump.go:388-422`: size check, rename over the good file (`:399`), then `ValidateDump` (`:419`). Volume tars are promoted with no content check either (`backup.go:902`). Tier 1 is one point per app by design (07 §6.1); the ORDER is the defect. PROVEN against production code (`audits/dataloss-hunt-2026-10-20/C3-D5-dump-replaced/`, a docker stub prints a header-only dump): `ValidateDump` says invalid („dump file too small"), `DumpOneTo` returns no error, and last night's good dump (with its `CREATE TABLE`) is gone from disk. Data-loss hunt 2026-10-20 (D5). | OPEN | — | Fix: validate the temp file before the rename; keep the last good dump on a failed check | CC | | **R-1186** | Backup & restore | P3 | **The nightly off-site copy is skipped silently when another backup holds the flag, and the catch-up never re-runs it.** `offbox.go:981-996` returns nil with an INFO line on the nightly path; `withLeg` marks the leg ended anyway (`cmd/controller/main.go:1246`). The 05:30 proof and 06:00 integrity check take the same flag, so a backup time near them collides every night. Only the hub's 48 h `offsite_stale` notices. CONFIRMED in code. Data-loss hunt 2026-10-20 (T5). | OPEN | — | Fix: mark the leg NOT ended on a busy skip so the catch-up retries; say it on the backups page | CC | -| **R-1187** | Backup & restore | P3 | **`.fab` export/import candidates from the data-loss hunt, not yet proven:** (a) an export silently drops a drive folder that holds a symlink (`appexport/fabplan.go:292`, `export.go:714`); (b) the import never sets file owners and drops symlinks (`restore.go:1157-1176`); (c) import into a STOPPED app merges into the existing volumes, and the default export (stop ticked) carries no SQL dump (`restore.go:361-374`, `export.go:262,366`, `appbackup/dbdump.go:142`); (d) no concurrency guard, no undo, no record (`restore.go:181-186`, `web/handler_export.go:312-353`); (e) PROVEN and split out as R-1191 (the import drops the family gate); (f) it trusts the bundle's HDD_PATH (`restore.go:589-631`); (g) the DB import runs psql without ON_ERROR_STOP and misses postgis (`restore.go:1052-1053, 928-937`). Read by reviewer 1; main session spot-checked (a); (e) proven live → R-1191. Evidence `audits/dataloss-hunt-2026-10-20/C1-restore-rollback-candidates.md`. | VERIFY | — | Prove (c) first on 9202 with a throwaway app; split into rows only what reproduces | CC | -| **R-1188** | Backup & restore | P3 | **Restore-path candidates from the data-loss hunt, not yet proven (one confirmed in code):** (a) PROVEN LIVE → R-1195 (a power cut mid-restore restarts the app on the backup's data; the undo copy is not used); (b) a failed `volume rm` is ignored before the untar (`restore.go:218`); (c) **FIXED in controller 0.337.0** (a leftover temp of the same name is removed before the copy; `TestR1188c_ALeftoverTempDoesNotBlockTheRetry`, the hunt's C3 case, red-proved) — was PROVEN against production code (`audits/dataloss-hunt-2026-10-20/C3-R12-leftover-temp/`): an interrupted second-drive whole restore leaves `.felhom-restoring`, and EVERY retry fails „fájlmásolás sikertelen: … file exists" with the file never back (`tier2_whole.go:86-87`, O_EXCL); (d) the full off-site download has a 30-min cap per press and the household is not told to retry (`web/offbox_handlers.go:455`); (e) place and full restore re-read the LATEST snapshot instead of the downloaded one (`offbox_restore.go:675-679`). Evidence `audits/dataloss-hunt-2026-10-20/C1-restore-rollback-candidates.md`. 2026-10-21 night: (a) is FIXED with R-1195 (the put-back at start, D22). | VERIFY | — | Prove (a) with the B6 power-cut-mid-restore scenario (R-1195 holds it); (b), (d), (e) still candidates | CC | -| **R-1189** | Backup & restore | P3 | **Time and concurrency candidates from the data-loss hunt, not yet proven:** (a) a host clock ahead during one whole-guest backup silences that tier until real time passes it, and the hub reads the same clock (agent `localapi/server.go:1178-1192`, `backup/runner.go:607-629`, hub `monitor/deadline.go:243`; R-973 cap counts future runs `backup_cap.go:61`); (b) the whole-guest start creeps ~5 min/night until a night is skipped silently (`server.go:1184`, `quiesce.go:150, 807-818`) — PARTLY SUPPORTED by demo-hp's agent journal 2026-09-30…10-20: completions drift 04:36→04:50 between resets, and THREE nights had no local whole-box backup (Oct 2, 6, 9), each right after an off-cycle daytime/evening run (Oct 1 22:15, Oct 5 11:24, Oct 8 22:25) — the 24-hour „not due yet" rule skips the next night; nothing tells anyone; (c) the whole-guest quiesce stops apps mid-restore or mid-update (`quiesce.go:213-275, 565`); (d) CONFIRMED in code: the off-site proof and integrity check are plain daily jobs outside the catch-up (`nightchain.go` header: the catch-up runs only the three backup legs), so they never run on a box that is off at 05:30–06:00, while the comment at `cmd/controller/main.go:1575-1577` says the opposite; (e) CONFIRMED in code: the hub's 05:00 check wants a dump „since Budapest midnight" (`hub/internal/monitor/deadline.go:416-419, 556-565`), so an evening backup time (which the banner suggests) raises a false `expected_dbdump_missed` error every morning — and on 2026-10-25 the check also runs at 04:00 (R-1183); (f) the OS leg and kernel reboot follow a daytime valve backup (agent `server.go:982-986`, `kernel.go:101-114`) — design question; (g) spring forward 2027-03-28 fires dump and second copy together; (h) self-update busy check before the pull only (`updater.go:532-545, 764`); (i) a double-clicked restore shows failed while it runs (`web/handlers.go:1853-1874`). Evidence `audits/dataloss-hunt-2026-10-20/C1-time-concurrency-candidates.md`. | VERIFY **(a) half-live 2026-10-21 (B6 #8):** the box wrote a local whole-box archive dated 2028-10-21 while its clock was 2 years ahead (`s8-wrong-clock/README.md`); by `localapi/server.go:1187` the local tier is „within cadence" until then. The due answer itself was not read live (the box died in B7 before its night window). The same run found R-1204 (the ep0 tunnel). | — | (a) is B6 #8's live test; (b) read the agent journal on demo-hp; the rest by unit tests | CC | -| **R-1190** | Backup & restore | P3 | **Key and retention candidates from the data-loss hunt, not yet proven:** (a) a recovery ceremony while off-site is not configured seals a package with no repo password and supersedes the good one (`escrow_handlers.go:217`, agent `main.go:2975-2981`, hub `store.go:3392`); (b) the global „create your recovery code" bar links to the ceremony the backups page hides during a recovery offer (`escrow_banner.go:23-33` vs `backups_remote.html:144-160`); (c) after a reuse-path reinstall the new PBS key K is never escrowed and nothing compares fingerprints (`felhom-pbs-apply:99`, `escrow_confirm.go:182-205`); (d) whole-guest off-site copies run before any escrow exists (design question vs 07 §6.1's Tier-3 gate); (e) the escrow PUT is two statements; (f) the operator's off-site re-issue resets the provider password before saving it (hub `offsite.go:217-245`); (g) „Megőrzött adatok" can list a running app's live folder when its app.yaml is unreadable (`kept.go:220-235, 330-402`); (h) a unit restore can point an app back at an old drive and F5 then prunes the current drive's unit (`backup.go:1517-1574`). Evidence `audits/dataloss-hunt-2026-10-20/C1-keys-candidates.md`, `C1-delete-retention-candidates.md`. | VERIFY | — | Unit tests first for (a), (e), (g) | CC | +| **R-1187** | Backup & restore | P2 | **A `.fab` export or import can lose or mix household data — code-confirmed paths from the data-loss hunt:** (a) an export silently drops a whole drive folder that holds a symlink and still ends „done” (`appexport/fabplan.go:292-310`, `export.go:712`; a FIFO also hangs the export on `os.Open`); (b) the import handles only folders and regular files: no owners set, symlinks dropped (`restore.go:1138-1183`); (c) **FIXED in controller main `d5f88a2`, unreleased (rides 0.340.0):** an import over an installed app that is not running merged the old data with the bundle's (`docker cp` never deletes) — reproduced on 9202 before the fix (stopped: a marker written after the export SURVIVED the import; running control: replaced), now refused with a Hungarian sentence and the app untouched; red-proved (`r1187c_test.go`), proven on 9202 rc.2 (`audits/night-2026-10-22/P3-candidates/r1187c-9202.txt`); (d) PLAUSIBLE: no restore-busy check in `StartImport`, no undo, no record (`restore.go:181-186`); (f) the bundle's HDD_PATH is trusted, an empty one gives a relative path (`restore.go:589-631`); (g) the DB import misses postgis images (adventurelog, dawarich: `classifyDBImage`) and runs psql without ON_ERROR_STOP (`restore.go:1047-1056, 933-943`; the dump is `--clean --if-exists`, so ON_ERROR_STOP is safe). (e) was R-1191. Research `audits/night-2026-10-22/P3-candidates/analysis.md`; first list `audits/dataloss-hunt-2026-10-20/C1-restore-rollback-candidates.md`. | OPEN — **2026-10-22 night: re-ranked P3→P2** ((a) and (b) lose household files on a move to a new box with no error). (c) fixed in code; (a), (b), (f), (g) read in code, not live-proven. | — | Fix (a)+(b) with a red-proof and a 9202 proof (capture code — the night rule); (g) small fix with a 9202 postgis app; (d) after (a) | CC | +| **R-1188** | Backup & restore | P4 | **Restore-path candidates from the data-loss hunt — what is left is unlikely or a sentence:** (b) a failed `volume rm` is ignored before the untar, which then lands in the old volume (`backup/restore.go:218`; `StopStack` is `compose down`, so the rm rarely fails); (d) the full off-site download stops at 30 min per press and the household is not told to press again (`web/offbox_handlers.go:455`; the scratch is kept, so a second press probably continues — not measured); (e) place and full restore re-read the LATEST snapshot, not the downloaded one (`offbox_restore.go:675-679`, `_ = id`). Closed parts: (a) with R-1195 (controller 0.338.0), (c) in controller 0.337.0 (`TestR1188c_ALeftoverTempDoesNotBlockTheRetry`). Evidence `audits/dataloss-hunt-2026-10-20/C1-restore-rollback-candidates.md`, `audits/night-2026-10-22/P3-candidates/analysis.md`. | OPEN — **2026-10-22 night: re-ranked P3→P4**: (a) and (c) are shipped; the rest is low-chance or a sentence. | — | (d): measure a second press on a large download on 9202, then write the sentence; (b), (e): unit tests when the file is next touched | CC | +| **R-1189** | Backup & restore | P3 | **Time and concurrency candidates from the data-loss hunt — what is left:** (a) a host clock ahead during one whole-guest backup silences that tier until real time passes it: the agent turns a future archive time into a negative age (`localapi/server.go:1179-1192`); half-live 2026-10-21 (B6 #8, `s8-wrong-clock/README.md`: a local archive dated 2028-10-21); agent code, so it rides an agent release and the operator's vouch; (c) the whole-guest quiesce waits for the catch-up and the update leg, not for a restore (`quiesce.go:213-275, 565`); (f) the OS leg and kernel reboot follow a daytime valve backup (agent `server.go:982-986`, `kernel.go:101-114`) — a design question; (g) spring forward 2027-03-28 moves 02:30 onto the 03:30 second copy; (h) self-update busy check before the pull only (`updater.go:532-545, 764`; not re-read); (i) a double-clicked restore shows failed while it runs (check and begin not atomic, `web/handlers.go:1860, 1881`). **Moved out 2026-10-22 night:** (b) the skipped nights after an off-cycle run were FIXED by R-899 (controller 0.304.0; demo-hp's journal shows a night run after each of three off-cycle runs since), the creep → R-1221; (d) **FIXED in controller main `d5f88a2`, unreleased (rides 0.340.0):** the catch-up now runs the off-site proof and the integrity check once after its legs, the false comment corrected; red-proved (`nightchain/r1189d_test.go`, `cmd/controller/r1189d_wiring_test.go`); (e) → R-1222 (hub). Evidence `audits/dataloss-hunt-2026-10-20/C1-time-concurrency-candidates.md`, `audits/night-2026-10-22/P3-candidates/analysis.md`. | OPEN — 2026-10-22 night: (b) half closed, (d) fixed in code, (e) split out. | — | (a) with the next agent release (a unit test with a future archive); (c) a 9202 recipe; (g) a unit test with the 2027 clock; (f) to the operator at the next design review | CC | +| **R-1190** | Backup & restore | P3 | **Key and retention candidates from the data-loss hunt — what is left:** (c) PLAUSIBLE: after a reuse-path reinstall the new PBS key K is never escrowed and nothing compares fingerprints (`felhom-pbs-apply:99`, `escrow_confirm.go:182-205`); (d) whole-guest off-site copies run before any escrow exists — a design question against `07` §6.1's Tier-3 gate (the operator's); (e) the escrow PUT is two statements; (f) the operator's off-site re-issue resets the provider password before saving it (hub `offsite.go:217-245`); (h) a unit restore can point an app back at an old drive and F5 then prunes the current drive's unit (`backup.go:1517-1574`) — (e), (f), (h) not re-read. **Moved out 2026-10-22 night:** (a) → R-1218, (b) → R-1219, (g) → R-1220. Evidence `audits/dataloss-hunt-2026-10-20/C1-keys-candidates.md`, `C1-delete-retention-candidates.md`, `audits/night-2026-10-22/P3-candidates/analysis.md`. | OPEN | — | (c) a read-only fingerprint compare on a reinstalled test box; (e), (f), (h) re-read, then unit tests; (d) to the operator at the next design review | CC | | **R-1198** | Backup & restore | P3 | **A backup disk the family adds later is never offered as the place for the whole-box backup; the page keeps offering the disk that holds the apps' own data.** Rehearsal 2026-10-20 B6 #13 (`audits/rehearsal-2026-10-20/B6-failures/s13-second-drive/`): a 50 GB disk plugged in and initialised as „Mentolemez" → the backups page still offers only „Családi adatlemez — Kijelölöm"; `internal/web/backup_target_offer.go:101` `firstOfferableDrive` offers the FIRST eligible drive, never a choice, and the drive's card has no „use for backups". | OPEN | — | Offer every eligible drive (largest empty first) and let the household pick | CC | | **R-1200** | Backup & restore | P3 | **A box installed in the afternoon gets no whole-box backup on its first night: the install's own backup starts the 24 h clock, so the family's whole first day reaches a whole-box copy only on the SECOND night (~38 h after install).** Rehearsal 2026-10-20 B5 (`audits/rehearsal-2026-10-20/B5-night/README.md`): Tester4's install backup at 12:09Z; at the 02:30–06:30Z window the agent answered „within cadence window" (`felhom-agent internal/localapi/server.go:1187`, `age >= tier.Cadence`, default 24 h). R-899 (closed) fixed the same shape for a daytime PRESS only. The other tiers did run that night (database copy, second drive, off-site — all PASS), so app data was covered; the whole-box (DR) tier was not. | **OPEN** — a design question: should the first scheduled night always run (like a press), or is one tier enough on night one? | — | decide; if „always run": treat the install backup like a press in the due rule, with a test on the first night | CC | -| **R-1204** | Backup & restore | P2 | **A box that runs once with its clock ahead loses its WireGuard tunnel to ep0 — the PBS whole-box copies and the operator's door — until the real date passes the wrong one, and nothing says why.** Rehearsal 2026-10-20 B6 #8 (`audits/rehearsal-2026-10-20/B6-failures/s8-wrong-clock/`): Tester4 booted with its clock at 2028-10-21 for 13 min, then the real time. ep0's `wg show` (read-only): Tester4's last handshake 07:40:47Z (clock at 2028), none in the 14 min after the return, the other peers 21–96 s. WireGuard refuses a handshake initiation whose TAI64N timestamp is not newer than the last one accepted from that peer, so the peer stays dead until 2028 or until ep0 resets it. Effects seen: every agent PBS call times out (15 s), the controller's due-check errors, the backups page says „A host-ügynök jelenleg nem elérhető". Likely triggers in the field: a dead CMOS battery, a hand-set BIOS clock, a VM restored with a wrong RTC. | **OPEN** — reproduced once live; not reset on ep0 (read-only in this run; the peer still shows the stale handshake). The box is destroyed in B7, which ends it for Tester4. | — | design: detect it (the hub sees the peer's handshake age) and heal it (re-key the peer, or ep0 resets the stored timestamp on a hub-signed request); a test with a fake clock | CC | -| **R-1205** | Backup & restore | P2 | **After a box death, a SECOND install that is deleted before it makes a key leaves the whole-box (PBS) tier waiting for the operator's re-issue.** Seen live 2026-10-21 (fix-1 proof, `audits/fixes-2026-10-21/P1-live/`): the rebuilt Tester4 box enrolled and the hub logged `pbsdr auto-provision for Tester4 (WG-registration hook): the endpoint already holds a PBS token for Tester4 but the hub has no descriptor — use the explicit "Re-issue PBS credentials" action`. The F-14 auto-re-issue reads the customer's LATEST host deletion, which was the short-lived rehearsal box (B7: deleted un-acked — it never held an escrow row); the R-1176 carry reads the newest ACKED deletion and worked. A household whose first reinstall attempt is abandoned gets no whole-box copy until the operator presses. | OPEN | — | Decide: may the F-14 gate skip an un-acked deletion that held no escrow row (it holds no key — the carry's own reasoning, `store/escrow_carry.go`), or keep the manual re-issue and mail the operator the press? Hub store test either way | CC | -| **R-1206** | Backup & restore | P3 | **Backups written while the box's clock ran ahead stay „the latest" after the clock is fixed.** The rehearsal's scenario #10 (clock set to 2026-10-28) left off-site snapshots dated 2026-10-28; on 2026-10-21 the rebuilt box's recovery page listed every app's „Legutóbbi mentés: 2026. 10. 28. 07:39", and the full restore said „mentés: 2026. 10. 28. 07:37" — a date in the future (`audits/fixes-2026-10-21/P1-live/07-unlock-right.txt`, `11-nextcloud-full-restore.txt`). restic orders snapshots by their own time, so the future ones are „newest" to every retention rule for a week. NOT measured: whether the box's keep-daily prune then drops a real snapshot. Related R-1202, R-1204 (clock). | OPEN | — | Measure on a scratch repository: the box's retention policy with one future-dated snapshot — is a real one dropped? If yes: refuse to write a snapshot while the clock is outside the hub's time by more than N, or drop future-dated snapshots first | CC | +| **R-1204** | Backup & restore | P2 | **A box that runs once with its clock ahead loses its WireGuard tunnel to ep0 — the PBS whole-box copies and the operator's door — until the real date passes the wrong one, and nothing says why.** Rehearsal 2026-10-20 B6 #8 (`audits/rehearsal-2026-10-20/B6-failures/s8-wrong-clock/`): Tester4 booted with its clock at 2028-10-21 for 13 min, then the real time. ep0's `wg show` (read-only): Tester4's last handshake 07:40:47Z (clock at 2028), none in the 14 min after the return, the other peers 21–96 s. WireGuard refuses a handshake initiation whose TAI64N timestamp is not newer than the last one accepted from that peer, so the peer stays dead until 2028 or until ep0 resets it. Effects seen: every agent PBS call times out (15 s), the controller's due-check errors, the backups page says „A host-ügynök jelenleg nem elérhető". Likely triggers in the field: a dead CMOS battery, a hand-set BIOS clock, a VM restored with a wrong RTC. | **OPEN** — reproduced once live; not reset on ep0 (read-only in this run; the peer still shows the stale handshake). The box is destroyed in B7, which ends it for Tester4. **2026-10-22 night — detection built, not live; the heal is decision D30.** Agent main `a0c88e5` (unreleased): the heartbeat reports `handshake_future_s` when the last handshake is more than 60 s in the future (`TestR1204_AFutureHandshakeIsReported`). Hub 0.166.0 (`e267c5e2`, image built, NOT deployed): `wg_tunnel_stale` / `wg_tunnel_ok` to the operator only, edge-triggered, when a box reports a future handshake or one older than 6 h (healthy boxes: at most 113–124 s over 24 h). The heal options: A) the agent makes a new WireGuard key and registers it through the existing peer path (`hub/internal/api/wg.go` `RegisterWGPeerForHost` + the wgsync push to ep0) — a new key has no stored timestamp; B) ep0 removes and re-adds the same peer on a hub-signed request — a new write path on ep0. | — | Deploy hub 0.166.0 and release the agent (STATUS morning list); then build the D30 pick with a fake-clock test | CC | +| **R-1205** | Backup & restore | P2 | **After a box death, a SECOND install that is deleted before it makes a key leaves the whole-box (PBS) tier waiting for the operator's re-issue.** Seen live 2026-10-21 (fix-1 proof, `audits/fixes-2026-10-21/P1-live/`): the rebuilt Tester4 box enrolled and the hub logged `pbsdr auto-provision for Tester4 (WG-registration hook): the endpoint already holds a PBS token for Tester4 but the hub has no descriptor — use the explicit "Re-issue PBS credentials" action`. The F-14 auto-re-issue reads the customer's LATEST host deletion, which was the short-lived rehearsal box (B7: deleted un-acked — it never held an escrow row); the R-1176 carry reads the newest ACKED deletion and worked. A household whose first reinstall attempt is abandoned gets no whole-box copy until the operator presses. | WAITING-ON-OPERATOR — **2026-10-22 night (read, `hub/internal/web/pbsdr.go:236-248`, `store.go:3293-3297`):** the F-14 gate is deliberate (operator ruling 2026-07-13), so this is a design question, not a defect — decision D31 in STATUS. | the operator's D31 | Operator: D31. A) the gate skips an un-acked deletion that held no escrow row (it held no key — the carry's own reasoning, `store/escrow_carry.go`) — CC's pick; B) keep the manual re-issue and mail the operator the press. Hub store test either way | operator | +| **R-1206** | Backup & restore | P3 | **Backups written while the box's clock ran ahead stay „the latest" after the clock is fixed.** The rehearsal's scenario #10 (clock set to 2026-10-28) left off-site snapshots dated 2026-10-28; on 2026-10-21 the rebuilt box's recovery page listed every app's „Legutóbbi mentés: 2026. 10. 28. 07:39", and the full restore said „mentés: 2026. 10. 28. 07:37" — a date in the future (`audits/fixes-2026-10-21/P1-live/07-unlock-right.txt`, `11-nextcloud-full-restore.txt`). restic orders snapshots by their own time, so the future ones are „newest" to every retention rule for a week. NOT measured: whether the box's keep-daily prune then drops a real snapshot. Related R-1202, R-1204 (clock). | NARROWED — the NAS tier only. **2026-10-22 night (read, `backup/offbox_window.go:22-24, 130-137`):** on the Felhom off-site tier no real snapshot is lost — the guard refuses a prune while any snapshot is future-dated (R-822, by design). The NAS tier prunes without that guard: open. | — | A pure unit test with the policy simulator on the NAS tier and one future-dated snapshot; if a real one is dropped, give the NAS tier the same guard | CC | | **R-1212** | Backup & restore | P4 | **Two household texts on the backup path do not follow the box's language:** the share-restore summary (`SharesRestoreResult.FlashMessage`, `controller/internal/backup/shares_restore.go`) is built from Hungarian Go literals, and the off-site `LastError` for an interrupted run is the Hungarian literal „megszakadt futás (a vezérlő újraindult futás közben)" (`controller/internal/backup/backup.go`). An English household reads Hungarian there. Found by the night helper 2026-10-21. | OPEN | — | Bundle keys for both (rendered in the box language when written, R-557), with parity fixtures | CC | +| **R-1218** | Backup & restore | P2 | **A recovery ceremony done while off-site is switched off (or cleared) seals no off-site repository password, and the hub then demotes the GOOD older package to operator-only custody: the household's own recovery code no longer opens its off-site copy.** REAL in code (night 2026-10-22, read; split from R-1190 (a)): `web/escrow_handlers.go:217-224` stages the password only if `OffboxConfigured()` (`backup/offbox.go:695-710`), false when the target is cleared or disabled — although `ClearOffboxTarget` keeps the password file on purpose (`offbox_clear.go` header); the agent then seals no password (`cmd/felhom-agent/main.go:2981-3003`) and uploads sha ""; hub `SaveHostEscrow` sees `curSHA != ""` and moves the good package to retained custody; `OffsiteRecoveryOffer` reads "" as „covers no key” (`offbox.go:1855-1873`) and says nothing. Evidence `audits/night-2026-10-22/P3-candidates/analysis.md` §R-1190(a). | OPEN — proposed fix, not built (key-handling code: a fix needs a red-proof AND a live proof, and 9202 has no hub to prove it with) | — | Fix (~6 lines): stage when the password FILE exists (`PushOffboxPasswordForEscrow`, `offbox.go:800`, reads only the file). Red-proof: a web test that the stage runs with off-site cleared; a hub store test pinning „abc”→"" ⇒ superseded (and refuse that demotion). Live proof on a test box with a hub (demo-hp 9201) | CC | +| **R-1219** | Backup & restore | P2 | **During a recovery offer the backups page hides the „create your recovery code” ceremony (it would replace the key that opens the old off-site copy), but the global top bar still links to it and the start handler does not refuse.** REAL in code (night 2026-10-22, read; split from R-1190 (b)): `web/escrow_banner.go:23-35` ignores `recoveryOffer()`; `templates/backups_remote.html:144-160` hides it and its own comment names the harm (R-199); `escrow_handlers.go:180-245` starts anyway; reachable in offer shape (c), `offbox.go:1866-1870`. Evidence `audits/night-2026-10-22/P3-candidates/analysis.md`. | OPEN — proposed fix, not built (key-handling code) | — | Fix: the bar hides its link during an offer, and the start handler answers 409 with a Hungarian sentence; red-proof with a handler test; live proof with an offer state on a test box with a hub | CC | +| **R-1220** | Backup & restore | P2 | **„Megőrzött adatok” (kept data) can list an INSTALLED app's live folder as kept — and offer to delete it — when that app's app.yaml cannot be read.** REAL in code (night 2026-10-22, read; split from R-1190 (g)): a nil AppConfig drops the app from `liveDriveBinds` (`stacks/kept.go:220-235, 336-360, 372-389`); a parse error gives nil (`deploy.go:1113-1125`). Evidence `audits/night-2026-10-22/P3-candidates/analysis.md`. | OPEN — proposed fix, not built (delete-path code) | — | Fix: an app whose app.yaml is unreadable counts as live (fail closed); its folders are never listed as kept. Red-proof: a unit test with a broken app.yaml; 9202 proof with a throwaway app | CC | +| **R-1221** | Backup & restore | P4 | **The nightly whole-box backup starts ~1.5–5 min later each night, until one night is skipped (after ~70–80 nights with no press between).** REAL, low harm (night 2026-10-22; split from R-1189 (b)): agent `localapi/server.go:1184` is `age >= Cadence` with the age counted from the archive START; the controller polls every 5 min (`quiesce.go:150`) and the archive starts after the apps stop. Measured on demo-hp: 04:31:07 → 04:33:18 → 04:36:47 → 04:38:14, and 04:33:37 → 04:38:44 → 04:42:40. At W+6h one night is skipped (44 h is below the 48 h valve, `quiesce.go:807-818`); the hub's 26 h line (`deadline.go:29`) mails the operator. Any manual or off-cycle run resets the drift. Evidence `audits/night-2026-10-22/P3-candidates/demohp-whole-box-completions.txt`. | OPEN | — | Fix: inside the night window a tier is due at `age >= cadence - 4h`; the agent answers due-ness, so it is agent code (a unit test with an injected clock) and rides an agent release | CC | ## Storage & devices — 11 rows (P3 4, P4 7) @@ -217,7 +221,7 @@ stopping line that lies. | ID | Category | Sev | What | State | Blocked on | Next action | Owner | |---|---|---|---|---|---|---|---| | **R-1139** | Security & access | P2 | **Tester 2's break-glass root password was not rotated with the others (R-954, operator decision D1 = A, 2026-10-17): the box has been offline since 2026-10-04.** demo-hp, Tester 1 and demo-felhom were rotated and proven on 2026-10-17 (`audits/agent-allowlist-rotation-2026-10-17/P3/`). | OPEN — filed 2026-10-17 **2026-10-22: closes when Tester 2 is reinstalled (operator ruling 2026-10-22 14:11 — a fresh install sets and vaults a new console password).** | Tester 2 reporting again | When it reports: the installer's step 4b alone (`P3/rotate-step4b.sh` shape), then the vaulted value logs in (PVE API 200) and the old one is refused | CC | -| **R-985** | Security & access | P3 | **Our zero-knowledge wording is stronger than the product: on a live box the operator can reach the off-site backup key (through the vaulted root credential and the operator tunnel) without the customer's recovery code.** Design-shaped; decision for the operator in `STATUS.md`. Fix options: change the wording, or remove operator root reach to the key. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | WAITING-ON-OPERATOR — design-proposal | operator decision | Decide the wording or the change | operator | +| **R-985** | Security & access | P3 | **Our zero-knowledge wording is stronger than the product: on a live box the operator can reach the off-site backup key (through the vaulted root credential and the operator tunnel) without the customer's recovery code.** Design-shaped; decision for the operator in `STATUS.md`. Fix options: change the wording, or remove operator root reach to the key. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | READY — the website patch waits for the operator's publish. (was WAITING-ON-OPERATOR — design-proposal) **2026-10-22 night — D3 ruled (operator 15:47: change the wording now).** The volunteer guides carry the new wording (felhom.eu `0639cf07`); the website change is a ready patch, not pushed, because a website push publishes (`audits/night-2026-10-22/P2-D3-D5/website-D3-D5.patch`, applies cleanly). | — | Operator: say „publish”; CC applies the patch, runs `site_gates.py`, pushes; then close (look again after the first sale, per D3) | CC | | **R-1042** | Security & access | P3 | **The boxes' Cloudflare keys cannot change zone settings, so Cloudflare's own „Always Use HTTPS” cannot be switched on.** Measured 2026-10-14 on all three zones (`audits/https-only-2026-10-14/``1A-1-zones-and-keys.txt`): each `cf_api_token` is valid and sees only its own zone, but GET/PATCH `settings/always_use_https`, `ssl`, `security_header` answer 403; the redirect and transform rule phases too. The https-only fix therefore lives on the box (decision 246). A second layer at Cloudflare needs „Zone Settings: Edit” (and, for headers, „Transform Rules”) added to each customer's key — the hub's zone-reach check stays. | **WAITING-ON-OPERATOR** — filed 2026-10-14 | a key permission only the operator can grant | Optional: widen each customer key, then switch „Always Use HTTPS” on per zone | operator | | **R-1104** | Security & access | P3 | **Check https-only on Tester 2 when it reports again.** R-1033 (plain http on every public name) is closed on the three boxes that report (35 names → 301, 2026-10-16). Tester 2 (`sajatfelhom.hu`) has been off since 2026-10-04 on controller 0.292.0; its addresses answer Cloudflare 530 (tunnel down), so nothing is served in plain http while it is off. When it comes back it takes the floor (≥ 0.312.0 carries the redirect). | OPEN — filed 2026-10-16 | Tester 2 reporting again | Run `scripts/https_only_check.py` with Tester 2 in it; close | CC | | **R-1141** | Security & access | P3 | **`felhom-host-install --rotate-recovery` exists only as part of a full installer re-run, which also reinstalls the vouched agent and the root files.** On a box running a newer agent than the vouched one (demo-hp on the candidate, 2026-10-17) it would move the agent back; on a household box it is a much larger act than a password change. The 2026-10-17 rotation therefore ran the installer's own `step_break_glass` ALONE (`audits/agent-allowlist-rotation-2026-10-17/P3/`). | **NARROWED 2026-10-18 — built on `main`, not published.** `felhom-host-install.sh --rotate-recovery-only` runs step 4b alone (felhom.eu `7d64038f`): reads the hub, host id and host key from the agent config, refuses byo and every other mode flag, asks the hub's `/healthz` before the password changes, never touches the agent or root files. The default install path is unchanged in behaviour (harness verdicts and 9 transcripts identical old vs new; demo-hp `--preflight-only` identical; `--rotate-recovery-only --dry-run` on demo-hp changed nothing). `step_break_glass` moved, byte-identical, ahead of the argument parse. `break-glass.md` §5 points at it. Publishes only with the next `installer-v…` tag. | the next installer tag | Cut the next `installer-v…` tag (with the other untagged installer changes); close | CC | @@ -228,7 +232,7 @@ stopping line that lies. | **R-775** | Security & access | P3 | **[P2-MEDIUM] Grimmory: a stranger's 5 wrong sign-ins lock EVERY visitor out of the web login for 15 minutes — so Grimmory was not published.** MEASURED 2026-10-01 on 9202 (drill catalog, v3.4.1, through traefik): after 5 wrong tries for `admin` every further sign-in answered 429 — the household's right password AND a different name — and stayed 429 for 10+ minutes of retries. Read in the jar: `AuthRateLimitService` — Caffeine `expireAfterWrite(ofMinutes(15))`, `MAX_ATTEMPTS 5`, keys `login:ip:` and `login:user:`; Spring `forward-headers-strategy: native` takes the address from X-Forwarded-For, and behind the tunnel every visitor is the tunnel container's address (R-753) — the wger shape (R-752), with no setting to change it. Everything else in the checklist passed (bench + box step v3.4.1 → v3.5.0, gate by its own probe, OPDS through traefik); two smaller findings for the publishing session: on a reinstall over the first install's kept books, a new upload was saved to the drive but not added to the library (`box/grimmory/reinstall-c1.txt`, not investigated); and the remove + restore round trip (2.5) cannot be shown on 9202 for a drive app — its backup lives on the scratch drive, which is not a registered drive (R-756). The template waits in `audits/new-apps-2026-10-01/wip/grimmory/`. **Needs (operator):** (A) publish with a sentence on the page that wrong guesses by others can lock the login for 15 minutes (MEASURED: during the lock an e-reader's OPDS feed still answered 200 with its own login, wrong 401 — `box/grimmory/opds-under-lock.txt`), or (B) wait until the box passes each visitor's real address (R-753). `audits/new-apps-2026-10-01/box/grimmory/throttle.txt` **-- 2026-10-01 (evening):** option B's precondition SHIPPED (controller v0.286.1, R-753): Grimmory's Tomcat RemoteIpValve walks from the right and counts `172.16.0.0/12` as a proxy (READ in source, Spring Boot 4.1.1 — not yet measured with Grimmory's own lock), so `login:ip:` becomes per visitor; `login:user:` still lets a stranger lock the public name `admin` 15 min. A third route was spiked and passed: Grimmory behind the permanent family gate with its e-reader paths excepted (R-780). Recommendation: publish behind the family gate if R-780 is built; otherwise B with a measured 3.6. **UPDATE 2026-10-02 — NARROWED, Grimmory PUBLISHED behind the family gate:** a stranger cannot reach Grimmory's web sign-in at all (6 tries through the simulated tunnel: the gate's 401, then the household signs in 200 — `audits/family-gate-2026-10-02/A/items.txt`), and the e-reader exceptions keep Grimmory's own login. The 2.5 round trip now WORKS on 9202 (`audits/family-gate-2026-10-02/B/box/life.txt`). **What is left:** a family member past the gate can still lock a NAME (the admin's) for 15 minutes with 5 wrong tries — hard-coded in Grimmory; and the reinstall-over-kept-books finding (`new-apps-2026-10-01/box/grimmory/reinstall-c1.txt`) is still not investigated. | **WATCHING — rank P3-LOW; owner: CC** **Re-ranked 2026-10-03: P2→P3: Grimmory is now behind the family gate; only a family member can still lock a name.** | — | — | CC | | **R-908** | Security & access | P3 | **An old Resend API key is still in the history of the `homelab-manifests` repository; it was replaced on 2026-06-29, but whether it is also revoked at Resend is unknown.** FOUND 2026-10-08 by the contact-mailer source search (`audits/mailer-source-2026-10-08/SEARCH.md`, „A side finding"): commit c9648cd (2026-02-05, „added mailer pod") put a key literal in the old `felhom-system/contact-mailer.yaml` comment; the file was deleted in ee93b50 but the history keeps it. Compared by hash only, never printed: it is NOT today's key (`Secret/resend-api`, rotated 2026-06-29, felhom.eu feea0606). If the old key still works at Resend, anyone with read access to that repository can send mail as felhom.eu. | **OPEN — owner: operator** **Note 2026-10-12 (security review):** gitleaks 8.24.3 with its default rules did NOT flag this key in the full history scan, so a scanner gate alone would not have caught it (R-986). | — | In the Resend console, check that the old key is revoked (revoke it if not); rewriting the repository's history is NOT needed once it is revoked | operator | | **R-525** | Security & access | P4 | **[P3-LOW] FileBrowser has its own login; putting it behind the dashboard session (traefik forwardAuth or Quantum proxy auth) is a new mechanism nobody has measured.** Filed 2026-09-15 by the P1-fixes task (B.5). R-513 closed the default-password hole with a generated password; a household still has two logins. **What it needs:** a spike on a scratch guest — forwardAuth to the controller session, and what FileBrowser Quantum does with a trusted header. | **READY — rank P3-LOW; owner: CC (spike)** **Re-ranked 2026-10-03: P3->P4: comfort feature needing a new unmeasured mechanism; the default password hole is closed.** | — | — | CC | -| **R-925** | Security & access | P1 | **`manifests/felhom.secret.yaml` carries REAL secret values, and the Gitea repo that holds it is readable by anyone on the internet with no login.** FOUND 2026-10-09 by a background security review of the legal-pages commit, which flagged the served `` comments; chasing what those comments pointed at found this instead. **MEASURED, in this order:** (1) `scripts/manifest_bearer_gate.py` itself prints `manifests/felhom.secret.yaml:39 KNOWN-BACKLOG committed secret 65cee3c4…7a86`; (2) the file holds **live-shaped values, not placeholders** — `SECRET_KEY` (69 chars), `SUPERUSER_EMAIL`/`SUPERUSER_PASSWORD` (18), Umami `APP_SECRET` (66) and `POSTGRES_PASSWORD` (34), and a `username`/`password` pair (18); values were never printed, only measured by length; (3) `gitea.dooplex.hu` resolves to **37.191.56.193**, the same public address as the website; (4) an **anonymous** `curl` returns HTTP 200 and 1686 bytes for that exact file, and 200 for `hub/internal/store/store.go` and `manifests/hub.yaml`; (5) **the off-network control**, which is what makes (4) mean anything — fetched from outside the operator's network entirely: a real path returns the file's first line, a nonsense path returns 404, so the 200 is genuine anonymous read from the internet and not a LAN-only ACL. **This contradicts the project's own written model**: `documentation/runbooks/secrets.md:3-4` says *“Secret values are never committed to git. The manifests in `manifests/` carry only placeholders + comments that point here.”* That promise is false today, which is the P1 wording in this register's own scale — see the ranking note below. The same runbook already states the rule that governs the fix: *“The git-history copy stays alive until the value is ROTATED — de-git alone kills nothing.”* **Known neighbours, none of which cover this:** R-887 records that an outside crawler walks the public Gitea pages and leaves *“Gitea's exposure to the crawler”* as the operator's undone item; R-580 still owes a Gitea admin token rotation. Neither says a secret file is world-readable. **Why P2 and not P1, stated so the operator can overrule it:** the exposed credentials guard **analytics and an undeployed healthchecks instance, not household data** — measured: `umami-db` is a **ClusterIP** service on 5432 with no external IP, so the Postgres password is not reachable from the internet; the realistic harm is session forgery against the public `stats.felhom.eu` via `APP_SECRET`, and reuse of those passwords anywhere else. No customer box, hub token or escrow key is in this file. **CC did NOT change anything**: making the repo private could break the public day-0 path (the installer is fetched from a public tag, R-110), and rotation plus repository visibility are operator decisions on production infrastructure. **PARTLY FIXED 2026-10-09, and RE-RANKED P2 -> P1 on what it turned out to be.** The exposed value was not an analytics password: it was the **Gitea `admin` account password** (`is_admin: true`; `/api/v1/admin/users` answered 200), published in a repo `gitea.dooplex.hu` serves anonymously to the internet. That is push access to every repo — including the one whose `website/` is git-synced live and whose `scripts/` is published by tag to **every new box installer** (R-110), i.e. a supply-chain path onto customer hardware. My first ranking said P2 because I had only measured the analytics blast radius; the credential test is what corrected it. **DONE (CC, operator-instructed):** (1) `umami-config` `APP_SECRET` + `POSTGRES_PASSWORD` rotated — the password changed **inside Postgres** too (`ALTER USER`), because the env var is only read at first init; verified by a real beacon returning 200 with a bogus-site-id control still returning 400. (2) `healthchecks-config` `SECRET_KEY` + `SUPERUSER_PASSWORD` rotated (nothing consumes them — there is no healthchecks Deployment). (3) `gitea-creds` no longer holds the admin password at all: it holds a **scoped token** (`read:package` + `read:repository`). **Both scopes are load-bearing and the second was found by breaking it** — a package-only token made the hub log `Template fetch: unexpected status 403`, because the template fetcher reads a raw file out of the `felhom-controller` repo. (4) The Gitea **admin password** was changed and `gitea-system/gitea-admin` updated; **verified: new password 200, old published password 401 — the leak is dead.** (5) The file is removed from git and `.gitignore`'s `*secret*` now applies to it; `manifest_bearer_gate.py`'s `KNOWN_BACKLOG` carve-out is gone, red-proofed with a decoy (exit 1 with, 0 without). **AN INCIDENT CAUSED BY THE FIX, recorded because it is the useful part:** the `rollout restart` needed to pick up the new umami secret put umami into **CrashLoopBackOff and took `stats.felhom.eu` down (503)**. Cause was not the rotation — at `memory: 512Mi` the pod runs for months but **cannot restart**, because startup (Prisma + Next.js) peaks over the limit and is OOMKilled (exit 137). Raised to 1Gi, **in the manifest, not just live** (the `.claude/rules/manifests.md` rule: a bare `kubectl set` is reverted by the next sync and the fix silently disappears). Service restored and verified. New values are in a 0600 file on DooPlex (`~/rotated-secrets-2026-10-09.txt`), never echoed; the operator moves them to the password manager and deletes it. **WHAT REMAINS, and it is bigger than what was fixed:** that one password is still the admin password in about **12 other namespaces** — `nextcloud`, `paperless`, `bookstack` (a **database root** password), `tandoor`, `calibre`, `adventurelog`, `gokapi`, `qbittorrent` (x2), `servarr`, `homepage` (x2). Rotating Gitea does not touch them: each is its own login and each is still the string that was published. **Also owed:** a Gitea token for user `kisfenyo` sits in plaintext in the `origin` URL of the local `homelab-manifests` clone (`.git/config`); CC printed it to a session transcript while investigating, so it should be rotated regardless — this is R-580's shape (store the remote without credentials). **Not a finding:** `homelab-manifests` itself is private (404 anonymously) and does not contain the password; ArgoCD's repo credential is a separate token, so the rotation did not touch it. | **NARROWED 2026-10-09 — the felhom.eu half is DONE and verified (Gitea admin password rotated, old one now 401; umami + healthchecks rotated; file de-gitted; gate carve-out removed). What remains is the ~12 reused logins elsewhere in the homelab and the repo-visibility decision; owner: operator** **2026-10-10 (release session), two consequences measured:** (a) DooPlex's own Docker login for `gitea.dooplex.hu` (`~/.docker/config.json`) still holds the OLD admin password — the registry token endpoint answers 401 for it and 200 for the current one — so every image push from DooPlex fails `unauthorized` until it is refreshed; the 0.145.0/0.305.0 pushes used a one-off login in a scratch `DOCKER_CONFIG`, shredded after. (b) The `felhom` ArgoCD app reads OutOfSync for `Secret/gitea-creds`, `Secret/healthchecks-config`, `Secret/umami-config` and `Deployment/umami`: a whole-app sync would push git's (de-gitted) view over the rotated live Secrets — the hub deploy synced ONLY `Deployment/hub` (`audits/release-2026-10-10/hub-0.145.0-deploy.txt`). Refreshing the DooPlex login is a DooPlex change (operator's word). | — | **OPERATOR RULED 2026-10-09: (a) he rotates the remaining services himself — CC's scope stopped at the Felhom boundary; (b) the repo STAYS anonymously readable for now**, because making it private breaks the website git-sync and the installer tag fetch, which both clone with no credentials (R-110). The standing consequence of (b): no secret may ever enter this repo again, which `manifest_bearer_gate.py` now enforces with no exemption; and if (b) is reversed, the 32 `` comments on /adatkezeles must be stripped in the same change. **The exact checklist of the 12 remaining secrets** (namespace / secret / key, re-measured after the rotation, with the two traps that bite — a DB password is not changed by editing the Secret, and a pod that has run for months may not restart) is in `documentation/runbooks/secrets.md`. Operator: **(1)** work that checklist, `bookstack-db/root-password` first because it is a database root password and every one of these hosts answers on the public internet; (`bookstack` first — it is a database root password); **(2)** rotate the `kisfenyo` Gitea token embedded in the local `homelab-manifests` remote URL, and store the remote without credentials; **(3)** decide whether `gitea.dooplex.hu` should answer anonymously at all — remembering the website git-sync clones it with no credentials and the installer is fetched from a public tag, so making it private breaks both unless they get credentials first, and the 32 `` comments on /adatkezeles become a map of it; **(4)** move `~/rotated-secrets-2026-10-09.txt` into the password manager and delete it. If nothing is done: the published password keeps opening a dozen services, even though Gitea itself is now safe | operator | +| **R-925** | Security & access | P1 | **`manifests/felhom.secret.yaml` carries REAL secret values, and the Gitea repo that holds it is readable by anyone on the internet with no login.** FOUND 2026-10-09 by a background security review of the legal-pages commit, which flagged the served `` comments; chasing what those comments pointed at found this instead. **MEASURED, in this order:** (1) `scripts/manifest_bearer_gate.py` itself prints `manifests/felhom.secret.yaml:39 KNOWN-BACKLOG committed secret 65cee3c4…7a86`; (2) the file holds **live-shaped values, not placeholders** — `SECRET_KEY` (69 chars), `SUPERUSER_EMAIL`/`SUPERUSER_PASSWORD` (18), Umami `APP_SECRET` (66) and `POSTGRES_PASSWORD` (34), and a `username`/`password` pair (18); values were never printed, only measured by length; (3) `gitea.dooplex.hu` resolves to **37.191.56.193**, the same public address as the website; (4) an **anonymous** `curl` returns HTTP 200 and 1686 bytes for that exact file, and 200 for `hub/internal/store/store.go` and `manifests/hub.yaml`; (5) **the off-network control**, which is what makes (4) mean anything — fetched from outside the operator's network entirely: a real path returns the file's first line, a nonsense path returns 404, so the 200 is genuine anonymous read from the internet and not a LAN-only ACL. **This contradicts the project's own written model**: `documentation/runbooks/secrets.md:3-4` says *“Secret values are never committed to git. The manifests in `manifests/` carry only placeholders + comments that point here.”* That promise is false today, which is the P1 wording in this register's own scale — see the ranking note below. The same runbook already states the rule that governs the fix: *“The git-history copy stays alive until the value is ROTATED — de-git alone kills nothing.”* **Known neighbours, none of which cover this:** R-887 records that an outside crawler walks the public Gitea pages and leaves *“Gitea's exposure to the crawler”* as the operator's undone item; R-580 still owes a Gitea admin token rotation. Neither says a secret file is world-readable. **Why P2 and not P1, stated so the operator can overrule it:** the exposed credentials guard **analytics and an undeployed healthchecks instance, not household data** — measured: `umami-db` is a **ClusterIP** service on 5432 with no external IP, so the Postgres password is not reachable from the internet; the realistic harm is session forgery against the public `stats.felhom.eu` via `APP_SECRET`, and reuse of those passwords anywhere else. No customer box, hub token or escrow key is in this file. **CC did NOT change anything**: making the repo private could break the public day-0 path (the installer is fetched from a public tag, R-110), and rotation plus repository visibility are operator decisions on production infrastructure. **PARTLY FIXED 2026-10-09, and RE-RANKED P2 -> P1 on what it turned out to be.** The exposed value was not an analytics password: it was the **Gitea `admin` account password** (`is_admin: true`; `/api/v1/admin/users` answered 200), published in a repo `gitea.dooplex.hu` serves anonymously to the internet. That is push access to every repo — including the one whose `website/` is git-synced live and whose `scripts/` is published by tag to **every new box installer** (R-110), i.e. a supply-chain path onto customer hardware. My first ranking said P2 because I had only measured the analytics blast radius; the credential test is what corrected it. **DONE (CC, operator-instructed):** (1) `umami-config` `APP_SECRET` + `POSTGRES_PASSWORD` rotated — the password changed **inside Postgres** too (`ALTER USER`), because the env var is only read at first init; verified by a real beacon returning 200 with a bogus-site-id control still returning 400. (2) `healthchecks-config` `SECRET_KEY` + `SUPERUSER_PASSWORD` rotated (nothing consumes them — there is no healthchecks Deployment). (3) `gitea-creds` no longer holds the admin password at all: it holds a **scoped token** (`read:package` + `read:repository`). **Both scopes are load-bearing and the second was found by breaking it** — a package-only token made the hub log `Template fetch: unexpected status 403`, because the template fetcher reads a raw file out of the `felhom-controller` repo. (4) The Gitea **admin password** was changed and `gitea-system/gitea-admin` updated; **verified: new password 200, old published password 401 — the leak is dead.** (5) The file is removed from git and `.gitignore`'s `*secret*` now applies to it; `manifest_bearer_gate.py`'s `KNOWN_BACKLOG` carve-out is gone, red-proofed with a decoy (exit 1 with, 0 without). **AN INCIDENT CAUSED BY THE FIX, recorded because it is the useful part:** the `rollout restart` needed to pick up the new umami secret put umami into **CrashLoopBackOff and took `stats.felhom.eu` down (503)**. Cause was not the rotation — at `memory: 512Mi` the pod runs for months but **cannot restart**, because startup (Prisma + Next.js) peaks over the limit and is OOMKilled (exit 137). Raised to 1Gi, **in the manifest, not just live** (the `.claude/rules/manifests.md` rule: a bare `kubectl set` is reverted by the next sync and the fix silently disappears). Service restored and verified. New values are in a 0600 file on DooPlex (`~/rotated-secrets-2026-10-09.txt`), never echoed; the operator moves them to the password manager and deletes it. **WHAT REMAINS, and it is bigger than what was fixed:** that one password is still the admin password in about **12 other namespaces** — `nextcloud`, `paperless`, `bookstack` (a **database root** password), `tandoor`, `calibre`, `adventurelog`, `gokapi`, `qbittorrent` (x2), `servarr`, `homepage` (x2). Rotating Gitea does not touch them: each is its own login and each is still the string that was published. **Also owed:** a Gitea token for user `kisfenyo` sits in plaintext in the `origin` URL of the local `homelab-manifests` clone (`.git/config`); CC printed it to a session transcript while investigating, so it should be rotated regardless — this is R-580's shape (store the remote without credentials). **Not a finding:** `homelab-manifests` itself is private (404 anonymously) and does not contain the password; ArgoCD's repo credential is a separate token, so the rotation did not touch it. | **NARROWED 2026-10-09 — the felhom.eu half is DONE and verified (Gitea admin password rotated, old one now 401; umami + healthchecks rotated; file de-gitted; gate carve-out removed). What remains is the ~12 reused logins elsewhere in the homelab and the repo-visibility decision; owner: operator** **2026-10-10 (release session), two consequences measured:** (a) DooPlex's own Docker login for `gitea.dooplex.hu` (`~/.docker/config.json`) still holds the OLD admin password — the registry token endpoint answers 401 for it and 200 for the current one — so every image push from DooPlex fails `unauthorized` until it is refreshed; the 0.145.0/0.305.0 pushes used a one-off login in a scratch `DOCKER_CONFIG`, shredded after. (b) The `felhom` ArgoCD app reads OutOfSync for `Secret/gitea-creds`, `Secret/healthchecks-config`, `Secret/umami-config` and `Deployment/umami`: a whole-app sync would push git's (de-gitted) view over the rotated live Secrets — the hub deploy synced ONLY `Deployment/hub` (`audits/release-2026-10-10/hub-0.145.0-deploy.txt`). Refreshing the DooPlex login is a DooPlex change (operator's word). **2026-10-22 night (Part 6, read only, `audits/night-2026-10-22/P6-repos-private/01-readers.md`):** homelab-manifests is already private; felhom-controller can go private after one change — CI clones with a read token (check first that the hub's `gitea-creds` token user can read a private `admin` repo); felhom.eu must stay public now (the website git-sync has no credential), felhom-agent too (every install fetches 16 config files from `raw/tag` with no login), and the catalog too (every box clones it with no login, and a token on a customer box is readable by its owner). Making a repository private does NOT hide its images or packages: they follow the user `admin`, which is public — and boxes need that (NOT-A-FINDING). **Count corrected:** the served `` comments are 61 on four pages (adatkezeles 33, feltetelek 15, teszteles 7, en/testing 6), not „32 on /adatkezeles”. The visibility question is decision D29 in STATUS. | — | **OPERATOR RULED 2026-10-09: (a) he rotates the remaining services himself — CC's scope stopped at the Felhom boundary; (b) the repo STAYS anonymously readable for now**, because making it private breaks the website git-sync and the installer tag fetch, which both clone with no credentials (R-110). The standing consequence of (b): no secret may ever enter this repo again, which `manifest_bearer_gate.py` now enforces with no exemption; and if (b) is reversed, the 61 served `` comments (four pages, 33 on /adatkezeles) must be stripped in the same change. **The exact checklist of the 12 remaining secrets** (namespace / secret / key, re-measured after the rotation, with the two traps that bite — a DB password is not changed by editing the Secret, and a pod that has run for months may not restart) is in `documentation/runbooks/secrets.md`. Operator: **(1)** work that checklist, `bookstack-db/root-password` first because it is a database root password and every one of these hosts answers on the public internet; (`bookstack` first — it is a database root password); **(2)** rotate the `kisfenyo` Gitea token embedded in the local `homelab-manifests` remote URL, and store the remote without credentials; **(3)** decide whether `gitea.dooplex.hu` should answer anonymously at all — remembering the website git-sync clones it with no credentials and the installer is fetched from a public tag, so making it private breaks both unless they get credentials first, and the 61 served `` comments become a map of it; **(4)** move `~/rotated-secrets-2026-10-09.txt` into the password manager and delete it. If nothing is done: the published password keeps opening a dozen services, even though Gitea itself is now safe | operator | | **R-779** | Security & access | P4 | **[P3-LOW] Part A's "two outside addresses seen as two" is proven through the simulated tunnel only; on the REAL tunnel the second outside address (ep0, one request allowed) was refused by Cloudflare's edge with 403 and never reached the box.** Measured 2026-10-01 19:51 UTC (`audits/visitors-2026-10-01/A/L2-demo-hp-real-tunnel.txt`): no log line on demo-hp; demo-hp's box has no geo restriction in its settings, so a Cloudflare ZONE rule (country or bot, not read) refused a German datacenter address. DooPlex's own address on the real tunnel was seen as itself. **Needs:** one sign-in from a second Hungarian address (the operator's phone off wifi) while DooPlex is locked out — 2 minutes; and say which Cloudflare rule refused ep0. | **WAITING-ON-OPERATOR — rank P3-LOW; owner: operator (a phone), CC reads the logs** **Re-ranked 2026-10-03: P3→P4: a proof gap on the real tunnel; operator-only follow-up.** | — | — | CC + operator | | **R-904** | Security & access | P4 | **Cloudflare can read every household's app traffic; replacing it with our own relay is a later item.** Facts (reviewer discussion 2026-10-08; `01`): app traffic and the dashboard reach the box through the Cloudflare Tunnel (`01` §5 trust table, rows end-user ↔ apps and customer ↔ controller UI; §7), and the tunnel's public end is Cloudflare's edge, where TLS ends — so Cloudflare can technically read that traffic (the FAQ says so since 2026-10-08, R-900; „TLS ends at the edge" is not written in `01` — add it there). What Cloudflare gives today, free: inbound reach with no router setup, the CGNAT answer (`01` §4, §7); certificates (`01` §7, the free tier covers one level below a zone); the geo-WAF the hub enforces (`01` §5 last row, §7); flood protection (not in `01`). The alternative named: our own EU relay over WireGuard with TLS passthrough by SNI, certificates on the box, the geo-block on the relay. Its costs: one more machine the operator keeps up, and a single point of reach for every box; weaker flood protection; about a week of work after a spike. Operator ruling 2026-10-08 09:07 (`09` §3 decision 184): a later item. | **DEFERRED** — **2026-10-10: the asked sentence is in `01` §7** („TLS ends at Cloudflare's edge …”); the spike stays deferred. **DEFERRED — after the first customers (operator ruling 2026-10-08 09:07)** | — | A spike after the first customers (the relay's reach, cost and flood behaviour, measured) | operator | | **R-1040** | Security & access | P4 | **Until the household makes Sonarr's or Radarr's first admin, any container on the box can read the app's API key.** Measured on the bench 2026-10-14 with the R-1030 seed in place: `AuthenticationRequired=Enabled`, `AuthenticationMethod=None` — `/initialize.json` from another container answers 200 with `apiKey`. Outsiders are blocked by the box's setup gate in that window; a compromised other app is not. | **OPEN** — filed 2026-10-14 | — | Seed the API key at install (`SONARR__AUTH__APIKEY`, measured to work) so the box knows it, and close the window by telling the household in the first steps; or seed a Forms user | CC | @@ -236,17 +240,16 @@ stopping line that lies. | **R-1054** | Security & access | P3 | **The hub's operator/global API key is still one bearer that can do everything on `/api/v1`, including reading every box's break-glass root password (`GET /admin/hosts/{id}/recovery-credential`).** Hub v0.152.0 gave the web surface scoped tokens (`05` §17) but tokens are refused on `/api/v1` by design. Who uses the global key today is not listed in one place. Fix: list its consumers; move each operator use to a scoped token on the web side (or a token class on the API), and keep the global key only where a box path needs it — or retire it. | OPEN — filed 2026-10-15 by the token build | — | List the global key's consumers (pod env, scripts, runbooks) | CC | | **R-1194** | Security & access | P3 | **On a fresh box the operator's break-glass SSH door was broken from the first minute: `oob_degraded` „felhom-sshd config invalid (sshd -t fails)".** Rehearsal 2026-10-20, Tester4-816e21, mail at 12:03 UTC (one minute after the host record was created): details `active:false, config_invalid:true, felhom_sshd_port:0, operator_key_configured:false, operator_key_reported:true, reachable:false`. Possibly the shape of a customer with no operator key set (Tester4 is a test customer) — then the alarm is a false one on every such box; or a real config defect. Not diagnosed (no shell). Checked the same day: the only `oob_degraded` in the operator mailbox in 30 days, and no recovery mail followed; the report says port 0. Two candidate mechanisms in the agent (`internal/felhomsshd/health.go:63-66`): `sshd -t -f ` run by the NON-root agent fails on unreadable host keys, or a config rendered before the desired state arrived (port 0). **CAUSE FOUND AND FIXED 2026-10-21 night** (`audits/night-2026-10-21/P5-checks/`): reproduced on a fresh box from the public ISO — `oob_degraded` mailed 20:34 CEST while felhom-sshd was applied 0.4 s after the check and is healthy (port 8822 listening, `sshd -t` passes as the agent's user through its sudo grant — so the non-root theory is FALSE; without sudo it fails on the host key, which the agent never does). The first host report runs before the felhom-sshd loop's first pass and reads a config never rendered. Fix: agent `0da792e` — no OOB stanza until the first pass has run (the hub does not evaluate a missing one; a failed pass counts). `TestR1194_NoOOBStanzaBeforeTheFirstPass` red-proved. | WATCHING — released in agent v0.162.0 (2026-10-22, operator yes; on the 3 test boxes); the false alarm comes only on a FRESH box's first report | the next fresh install with agent ≥ 0.162.0 | At the next fresh install read the hub's events for `oob_degraded` in the first hour (none expected); then close | CC | -## Box system & updates — 13 rows (P2 2, P3 7, P4 4) +## Box system & updates — 12 rows (P2 1, P3 7, P4 4) | ID | Category | Sev | What | State | Blocked on | Next action | Owner | |---|---|---|---|---|---|---|---| | **R-984** | Box system & updates | P4 | **Host and agent hardening, low risk each: the backup-target route has no role check; the unsigned guest spec has no size bounds; signed ops have no maximum validity; the guarded mkfs does not refuse LVs, zvols or RAID devices; the local API is reachable from the LAN by a static route; a sudoers inspect pattern is wider than needed; install passes secrets on the command line; guest settings files are world-readable; the host offers rpcbind and DNS on the LAN.** Fix each in its own small change. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN | — | Batch into hardening work | CC | -| **R-982** | Box system & updates | P3 | **The Proxmox host accepts root password login over SSH from the LAN (measured on demo-hp), so anyone on the household network can try the root password.** Design-shaped: the break-glass password exists for the console. Fix: key-only SSH for root; the password stays for the console. Evidence: `audits/security-review-2026-10-12/evidence/demo-hp-host-reads.txt`. | OPEN — design-proposal | — | Decide; then set it in the installer and the config bundle | CC | +| **R-982** | Box system & updates | P3 | **The Proxmox host accepts root password login over SSH from the LAN (measured on demo-hp), so anyone on the household network can try the root password.** Design-shaped: the break-glass password exists for the console. Fix: key-only SSH for root; the password stays for the console. Evidence: `audits/security-review-2026-10-12/evidence/demo-hp-host-reads.txt`. | BLOCKED — on the agent release (the operator's vouch) and an installer tag. OPEN — design-proposal **2026-10-22 night — D4 ruled (operator 15:47: root SSH from the LAN by key only; the console password stays) and built as two ready patches, not pushed:** the agent's config bundle gains a sshd drop-in (`/etc/ssh/sshd_config.d/10-felhom-root-key-only.conf`, `PermitRootLogin prohibit-password`; Debian reads the drop-ins first and the first value wins; felhom-sshd on 8822 has its own config) and the installer's uninstall removes it. Proven on a fresh install from ISO 1.30.0 (nested VM 9343, destroyed): before, password SSH worked; after, password SSH refused, key works, PVE web password login 200, console login works (`audits/night-2026-10-22/P2-D4/`). Not pushed: the bundle half reaches a box only with an agent release, and the installer half only with a new `installer-v…` tag (R-110; a push to `main` alone publishes nothing) — both forbidden tonight. | the next agent release + installer tag | Apply `P2-D4/felhom-agent-bundle-D4.patch` with the next agent release and `P2-D4/felhom.eu-installer-D4.patch` with the next installer tag; check on demo-hp after delivery | CC | | **R-979** | Box system & updates | P3 | **Controller updates are trusted by registry tag alone (no digest, no signature), and the Update button offers any newer tag, even an unvouched one; the built-in traefik, cloudflared and FileBrowser images are also pinned by tag only.** Whoever can push to the registry decides what runs. Fix: pin by digest from the hub's vouched list; offer only vouched versions. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN — design-proposal | — | Design note, then build | CC | | **R-974** | Box system & updates | P3 | **A compromised host agent can likely become root on the host through the root wrapper that applies PBS settings: the wrapper accepts a folder path inside the agent's own writable tree and then acts on it as root.** Static (the symlink behaviour was reasoned, not run). Same class as R-861. Fix: a fixed root-owned folder, and refuse symlinks before any root action. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN | — | Fix the wrapper with a test on a throwaway box | CC | | **R-972** | Box system & updates | P2 | **The shipped controller and hub programs are built with Go 1.24.13 and golang.org/x/crypto v0.31.0, and the agent with Go 1.26.0, so dozens of fixed Go security bugs (including html/template escaping fixes) are still in what runs on boxes and the hub.** Go 1.24 has had no security fixes since Go 1.26 was released (Go supports the two newest major releases: https://go.dev/doc/devel/release#policy); the scanners name fixes in Go 1.26.9 and 1.27.2. Also: the controller image's Debian 12 base has 12 fixable critical packages, and the Samba image has 48 fixable high ones. Fix: build all three on a supported, patched Go; update x/crypto; rebuild the base images. Scanner output: `audits/security-review-2026-10-12/scanners/`. | **NARROWED** 2026-10-16 (burndown 2): **controller v0.322.0 released** on Go 1.27.2 + x/crypto v0.58.0, runtime debian:12.13-slim + `apt-get upgrade` (Debian 13 deliberately not: restic 0.18, R-1105) — scanner: critical 16 → 4, fixable criticals and highs in the OS 0; govulncheck 62 → 1 (module-level, `openpgp` not linked). **Hub v0.156.0 deployed** 2026-10-16 19:26 on the operator's yes (Go 1.27.2, alpine 3.24.2: 0 critical, 0 high); Synced/Healthy, reports from all three boxes after the start, the floor reads 0.322.0. **Agent: Go 1.27.2 committed (`37b99d7`), tested, NOT released** — the next vouched agent release carries it. Open: the Samba image (48 fixable high, not rebuilt); docker-compose plugin's own Go (4 high, upstream). Evidence `audits/burndown2-2026-10-16/P4/` **2026-10-16 (evening): the Samba part is DONE in controller v0.323.0** — felhom-samba 1.2.0 on Alpine 3.24.2 (0 critical / 0 high, was 48 fixable high), proven on demo-hp with the same household password (smbclient, nmbd, mDNS, WS-Discovery); on the three test boxes now, on every box with the global floor (R-1105's night check). The controller's Debian 13 OS part: 1 critical / 70 high, none fixable. Left: docker-compose plugin's own Go (4 high, upstream); the agent release; restic's binary is R-1107. **2026-10-17 (evening): agent v0.159.1 (Go 1.27.2) RELEASED and running on demo-hp, Tester 1 and demo-felhom** (signed jobs on the operator's yes); waiting for the operator's vouch — then only the upstream compose plugin is left (close then, recorded). | the next vouched agent release | Close when the global floor carries v0.323.0 and the next vouched agent release carries Go 1.27.2 (compose plugin: upstream, recorded) | CC | -| **R-1207** | Box system & updates | P2 | **A host with no (or too little) swap gets nothing from the guest swap fix (R-1192), and the agent never adds host swap.** The fix lets a guest swap as much as its RAM; that swap lives on the HOST. Measured 2026-10-21 night: our installer gives an 8 GB / 256 GB machine 7.75 GB of host swap (LVM `pve/swap`), so the installer's boxes are covered; a BYO host (or one with swap turned off) may have none. Now reported and said (agent `0da792e`: `host_swap_mb` / `guest_swap_mb`, one WARN; hub 0.164.0 `host_swap_short` to the operator; the install page sentence) — but nothing adds swap. The no-swap test on the soak VM (`audits/night-2026-10-21/P2-noswap/`) says what happens without it. | WAITING-ON-OPERATOR **NO-SWAP TEST 2026-10-22 04:08–04:38 UTC (`audits/night-2026-10-21/P2-noswap/`):** the soak box, `swapoff -a` on the host, the same burst → the freeze CAME BACK within 20 s: `/api/health` dead for 289 s, then 172 s and 68 s (24 of 125 probes failed), memory „full" 36 % host / 47 % guest, the VM reading ~40 GB per 25 s from disk; Immich lost 207 of 306 photos and its stream broke; the box recovered only when the Immich stream stopped (04:23). After `swapon -a`: 0 failed probes, „full" ≤ 1.3 %. The agent logged its one WARN at the swapoff (host 938 MB < guests 4096 MB). **Operator answer 2026-10-22: Tester 2's laptop was installed with OUR installer** — so it has host swap (the installer gives ≈ RAM up to 8 GB) and the swap fix helps it once its agent is ≥ 0.162.0. The decision for BYO hosts stays open. | decision: should the agent (or the installer for BYO) create host swap when it is short, or should an 8 GB BYO host without swap be refused at install? | Operator decides; CC builds the pick | operator | -| **R-1214** | Box system & updates | P3 | **On the 8 GB box with the swap fix, the night's second-drive backup still pushes memory „full" pressure above 20 % for about 3 minutes; the box keeps answering.** Night soak 2026-10-21→22 (`audits/night-2026-10-21/P1-soak/03-rec.jsonl`, 10-s samples; fresh box from the public ISO, agent with swap = RAM, 8 apps, the rehearsal uploads for 2 h): at 01:30–01:34 UTC (the 03:30 tier2 job) host 39 % / guest 45 % over 10 s, over 60 s at most 20 % / 25 %; IO „full" 66 %; guest swap 1.35 → 2.1 GB; `/api/health` answered every probe (slowest 4.1 s, largest interval 16 s); 0 OOM kills. Once more in the upload window (Immich loading its models, 31 % / 35 % for one sample). The brief's soak pass line was „full" < 20 %, so the soak missed that one line; the freeze itself (96 % for an hour, R-1192) did not come back. Not diagnosed: which part of the tier2 job (the copy's page cache, the volume dumps) causes it. | OPEN | — | Measure the tier2 job alone on the 8 GB box (per-step pressure); decide whether 20 % over 10 s is the right line, or limit the copy (ionice / cgroup IO weight) | CC | +| **R-1214** | Box system & updates | P3 | **On the 8 GB box with the swap fix, the night's second-drive backup still pushes memory „full" pressure above 20 % for about 3 minutes; the box keeps answering.** Night soak 2026-10-21→22 (`audits/night-2026-10-21/P1-soak/03-rec.jsonl`, 10-s samples; fresh box from the public ISO, agent with swap = RAM, 8 apps, the rehearsal uploads for 2 h): at 01:30–01:34 UTC (the 03:30 tier2 job) host 39 % / guest 45 % over 10 s, over 60 s at most 20 % / 25 %; IO „full" 66 %; guest swap 1.35 → 2.1 GB; `/api/health` answered every probe (slowest 4.1 s, largest interval 16 s); 0 OOM kills. Once more in the upload window (Immich loading its models, 31 % / 35 % for one sample). The brief's soak pass line was „full" < 20 %, so the soak missed that one line; the freeze itself (96 % for an hour, R-1192) did not come back. Not diagnosed: which part of the tier2 job (the copy's page cache, the volume dumps) causes it. | READY — waits for the controller 0.340.0 release. **2026-10-22 night — cause measured, fix built and proven on 9202, waits for the controller release.** Cause: the copy's page cache is charged to the guest and evicts the apps' working set. On 9202 (4 GB RAM, 1.8 GB held by a hog, a 6 GB copy): plain rsync full stall 5 702 / 2 464 ms, peak „full” 16.8 / 6.9 %; the same copy in a 512 MB memory-capped container 374 / 320 ms, 0.9 / 1.0 %. Controller main `03c0a69` + `ae39467` (unreleased, rides 0.340.0): the copy runs capped only for clean `/mnt/` paths with no symlink (security review); a docker error (exit 125) falls back to plain rsync. A real `POST /api/backup/tier2` on rc.4: 6 capped containers, each exit 0; Immich's copy equals its source (dry-run 0 differences, 44 = 44 files). Evidence `audits/night-2026-10-22/P4-R1214/`. | — | Release 0.340.0; then re-measure the 03:30 job on the 8 GB soak box and close if „full” stays low | CC | | **R-862** | Box system & updates | P3 | **Tester 2 cannot take the config bundle until one by-hand bootstrap is done: its `felhom-os-apply` (agent 0.142.0) predates the bundle mode, and no signed job can write a root file on it.** FOUND 2026-10-04 (R-840 build, Part C): the route reaches every box installed from installer 1.31.0 on, and the demo boxes (bootstrapped by CC); Tester 2 has every root file of agent 0.142.0 (installer 1.30.0) and lacks only the R-858 wrapper fix, which matters only for a Docker step it gets solely from a signed job. CC has no route to Tester 2 (its door admits only the operator's WireGuard peer; `felhom-op` cannot become root). The steps: `runbooks/config-bundle.md` "Tester 2". Then CC signs the bundle and reads it back. | **WAITING-ON-OPERATOR** — the operator said (2026-10-04 ~19:05) he will try through his tunnel **2026-10-07 07:58 (`09` §3 decision 170): the operator believes Tester 2 never set up the off-site backup (no escrow); the laptop may stay off for days; the tester plans to add an HDD.** **2026-10-07 (Part H, read only):** in Tester 2's last 10 notifications (2026-10-05 09:16 → 2026-10-07 06:58) the household got 0 mails, the operator 10 (host_down ×9, expected_dbdump_missed ×1); control: the household channel shows on the other three customers. So the household is not mailed daily while the box is off; no row (`audits/day-2026-10-07/H/`). **2026-10-22: closes when Tester 2 is reinstalled (operator ruling 2026-10-22 14:11 — a new install from the public download replaces the bootstrap and the off-site check; `pilot/TESTER2-handover.md`).** | the operator's WireGuard tunnel | the operator runs the three bootstrap commands; CC sends the bundle | operator | | **R-78** | Box system & updates | P3 | **`local_api` authority ruling — auto-reconcile vs detect-only** **MIGRATED FROM `ROADMAP.md` 2026-08-22 (R-369) — originally filed 2026-07-25, size M, roadmap state `idea (deferred OUT of R-77 on purpose)`.** Moved verbatim; nothing added or reinterpreted. The roadmap keeps its copy as history, marked moved. **An OWED OPERATOR DECISION, not a defect — moved because an owed ruling hidden among feature ideas is the shape this session exists to remove.** | **OPEN — migrated from ROADMAP 2026-08-22, rank unchanged** | — | R-77 ships detection because the fix is genuinely undecided, and **both directions can lose customer-visible function**. **Direction 1 (today):** `controller.yaml` wins and drift is silent → the 2026-07-25 island migration blinded the whole fleet's control plane for 17.5 h (drive gate, guest-reboot recovery, quiesce/backup all degrade). R-77 makes that loud but does not stop it recurring. **Direction 2 (`bootstrap.json` wins, auto-reconcile on boot):** a guest whose `controller.yaml` is CORRECT and whose `bootstrap.json` is stale — a half-completed re-provision, a hand-repaired guest, a setup-wizard box — gets a **working channel clobbered on the next restart**, fleet-wide and silently, during a routine deploy. That is not obviously better than the bug. Needs a spike: which writer is authoritative per field (endpoint vs fingerprint vs token — `mergeLocalAPI` replaces the whole block, so they cannot be reconciled independently today), whether the agent side should stamp a generation/mtime so 'newer wins' is even expressible, and whether reconcile should require an operator ack. Until then the drift alert plus a manual edit is the supported path. | CC | | **R-468** | Box system & updates | P3 | **[P3-LOW] THE GOLDEN WAIVER — goldens on a cadence, not per release (operator ruling 2026-09-13).** 25 goldens in 26 days in August, almost one per release, because `golden_currency_gate.py` trips on every release by design and the only honest ways past it were a bake or a declared `--no-verify` (thirteen by 2026-09-01, R-404/R-417). **The ruling: bake WEEKLY, and always before any drill or fresh install.** Every release still raises the FLOOR, so both demo boxes keep getting each release in ~20 s; only the golden — which protects a fresh install and nothing else — moves to a cadence. **The mechanism (built 2026-09-13):** `documentation/tests/golden-waiver. **⚠ CORRECTED THE SAME DAY (R-472): between bakes the floor does NOT carry a release — the hub holds any floor above the vouched golden (publish-train rule 1), so releases between bakes reach the demo boxes only by hand-deploy.**yml`, four lines (`issued`, `expires`, `reason`, `register_row: R-468`), read by the gate. While valid, a golden BEHIND the record makes the gate print a loud ADVISORY and exit 0; when it expires the gate is red again until someone bakes or renews. **The 14-day cap is enforced by the gate, not the runbook** — a longer, undated, unparseable, reason-less or row-less waiver is INCONCLUSIVE (exit 2), never 0 and never silently ignored. **It never covers a golden that is UNRECORDED (R-385)** — that is not a cadence choice. **A dated waiver cannot be forgotten; it just expires** — the difference from R-242's original rule, which recurred the day after it was written. Tests: `scripts/test_golden_currency_gate.py` cases 5–15 (E/F/G/H, a 15-day, absent, unparseable, bad-row and empty-reason waiver each 2; the R-421 decoy — a file saying only `expires` — 2). **This is a PRE-CUSTOMER arrangement: the first external install retires it** (delete the file in that commit). Cadence written into `RUNBOOK-manual-build.md` §4.2 and the `felhom.eu` end-of-session checklist. **Does NOT touch R-242's open half (nothing gates the VOUCH).** **2026-10-15 (`09` §3 decision 250, operator):** the cadence is now the RULE, not a waiver — a controller release needs no golden (the floor carries it), the golden is baked every Sunday night when the floor is above it (`runbooks/nightly-rotation.md`), and `golden_currency_gate.py` convicts only a golden MORE THAN 7 DAYS old while a release is ahead (CASE 18–21, red-proved). The waiver file stays as a dated exception for a missed Sunday. | **WATCHING** — the weekly bake is a night-routine step; the first external install still retires the waiver arrangement | — | — | CC | @@ -255,7 +258,7 @@ stopping line that lies. | **R-1061** | Box system & updates | P4 | **A per-customer floor reached Tester 1 only at its next 15-minute report, not in seconds.** Set 2026-10-15 12:55:51 UTC for tester-1, demo-hp and demo-felhom together (hub 0.153.0, CC's token): demo-hp and demo-felhom ran 0.314.0 within 13 s (the hub's wake); Tester 1 moved at 13:09:35, its regular report (`audits/gate-restore-2026-10-15/release/03-boxes-move.txt`). It had restarted at 12:54:29 onto rc.1 and logged „hub wait channel active". Why the wake did not reach it is not read (the wake line is DEBUG). No harm: the floor arrived. | **OPEN** — filed 2026-10-15 | — | Read Tester 1's DEBUG ring at the next floor change; compare with the hub's bump log | CC | -## Monitoring & notifications — 15 rows (P2 1, P3 10, P4 4) +## Monitoring & notifications — 16 rows (P2 1, P3 11, P4 4) | ID | Category | Sev | What | State | Blocked on | Next action | Owner | |---|---|---|---|---|---|---|---| @@ -274,6 +277,7 @@ stopping line that lies. | **R-1208** | Monitoring & notifications | P3 | **The operator gets `node_stale` / `node_down` („No report received for 1h30m") for a customer whose box the operator has just DELETED.** 2026-10-21: Tester4's box was torn down 16:27 UTC and its host deleted by the operator 16:49 UTC (escrow acknowledged); `node_down` was mailed to the operator at 17:55 UTC. The customer-level staleness (`hub/internal/monitor/staleness.go` `Check`) counts from the last controller report and does not know the host was deleted on purpose. Not fixed tonight: the brief allowed one hub image and 0.164.0 was already built. | OPEN | — | Skip (state „deleted", no event) a customer with no live host whose newest host deletion is after its last report; a new box's first report → ok with no event (as `StateDisabled`); test both | CC | | **R-1217** | Monitoring & notifications | P4 | **On demo-hp the hub mails three app events (`app_update_undone`, `app_update_held`, `app_stopped_unhealthy`) while the box's page shows none ticked.** The household saved an empty list; the hub's one-time add-only seed of 2026-09-24 added the three types to that empty list, while the box's own seeds skip a list stored as missing (the old `omitempty` shape: `{"cooldown_hours": 6}`). Read 2026-10-22: hub row 3 events, box file no `enabled_events`. Tester 1 (also empty) is not affected (its row was saved after the seed). Test boxes only; a household saving its page once makes the two agree. | OPEN | — | Either let the box adopt the hub's list when its own saved list is the missing-events shape, or leave it (test boxes only) — decide at the next notification change | CC | | **R-1211** | Monitoring & notifications | P3 | **The household's `backup_failed` mail says only „A biztonsági mentés sikertelen! Kérjük, ellenőrizd a rendszert." — no app named, no next step, and the formal „Kérjük".** Rehearsal 2026-10-20 (the one alarm the household got, `audits/rehearsal-2026-10-20/B9-words/words.md` W25); the box's own page now says which app and that the box retries the next night (controller `5776d54`, W25), but the mail is the hub's `customerMessages` entry. Found by the night helper 2026-10-21. | OPEN | — | Hub: build the household mail from the event's app names and the classifier's class, in the te-form, with „a doboz a következő éjjel újra próbálkozik"; mail golden | CC | +| **R-1222** | Monitoring & notifications | P3 | **A household that sets an EVENING backup time gets a false „database dump missed” error every morning.** REAL in code (night 2026-10-22; split from R-1189 (e)): the hub's 05:00 check wants a dump since Budapest midnight (`hub/internal/monitor/deadline.go:416-419, 555-565`), so an evening dump (the banner suggests evening times) is always „yesterday”; on 2026-10-25 the check also runs at 04:00 (R-1183). Evidence `audits/night-2026-10-22/P3-candidates/analysis.md`. | OPEN | — | Fix: the check wants a dump within the last 26 h (or since the box's own scheduled time); a hub test with an evening schedule; rides a hub deploy | CC | ## Hub & operator — 12 rows (P3 5, P4 7) @@ -313,20 +317,18 @@ stopping line that lies. | **R-990** | Business & legal | P4 | **Operator: read `documentation/audits/forum-research-2026-10-12/FELHOM.md` §5 and rule on each recommendation.** READ-ONLY research 2026-10-12: three years (#7943–#11092, 2,908 posts, 195 people, 82 beginner threads) of the public PROHARDVER! topic „Home server / házi szerver építése”, tagged and counted by person (`FINDINGS.md`, method and spot-check in `METHOD.md`; no username or post text in git). Ten ranked recommendations, none acted on; the top three: a „which machine works” page (8 of 33 beginners who name their hardware own only a Synology, a router disk or a Raspberry Pi — the installer cannot run there; the same gap is R-989's first recommendation from a different source, and the shipped-page half is R-991), „you do not choose an operating system — we do” (the biggest beginner question, 20 of 82 threads), and „bring your old office PC” as the cheap entry. Catalog gap (≥ 3 people): a DLNA server (7), an ad/DNS blocker (5), a camera recorder (5). | **WAITING-ON-OPERATOR** — filed 2026-10-12 | the operator's reading | Operator: rule on each of the ten (yes / no / later). If nothing is done: nothing changes; the research ages | operator | | **R-1035** | Business & legal | P4 | **Did the „felhő tárhely” page earn anything? Two dated checks.** **Baseline (reported by the operator from Search Console, 2026-10-14, last 3 months):** 1 click, 48 impressions, CTR 2.1 %, average position **73.6**; „felhő tárhely” 19 impressions, „felhőtárhely” 5, „felhő alapú tárhely” 3. CC measured separately that the phrase „felhő tárhely” appeared **0 times** on the home, FAQ and backup pages — Google was showing the home page for those queries by accident. `/sajat-felho-tarhely` + `/en/own-cloud-storage` published **2026-10-14**; full baseline, price sources and the power provenance in `audits/seo-2026-10-14/BASELINE.md`. **What to measure, both times:** impressions and average position for „felhő tárhely” and „saját felhő tárhely”, plus the new page's own impressions, clicks and position. **A null result at +4 weeks means nothing on its own** — a new page on a site at position 73.6 takes weeks to months to move, and „felhő tárhely” alone is held by Microsoft, Google and the big list sites; the page aims at the two longer phrases. That is why there are two checks and not one, and why neither should be read as a verdict on the copy. **There is no Search Console connector in this project, so CC cannot read these numbers** — the operator reads them, CC records and compares. **Blocked until the operator asks for indexing** (Search Console → URL-ellenőrzés → the URL → „Indexelés kérése”): until that happens, or Google finds the page itself, both checks read zero for a reason that is not the page. **This is a slow channel and does not replace talking to people.** | **WATCHING** | operator: request indexing, then supply the Search Console read | 2026-11-11: first read (+4 weeks); 2026-12-09: second (+8 weeks), compared with the first | operator + CC | -## Process & tooling — 31 rows (P3 8, P4 23) +## Process & tooling — 29 rows (P3 7, P4 22) | ID | Category | Sev | What | State | Blocked on | Next action | Owner | |---|---|---|---|---|---|---|---| | **R-1083** | Process & tooling | P3 | **The scratch box 9202 cannot use any feature that needs the agent: its `bootstrap.json` has no `local_api` block, so its controller never gets an agent client.** MEASURED 2026-10-15 (storage-identity build A): 9202 put on controller 0.317.0 by hand to show that a NAS share no longer hides the one-copy notice (R-1075); its `bootstrap.json` (read on the box) holds no `local_api`, so the controller has no agent to ask and keeps the old `st_dev` rule — the check could not tell old from new — 9202 was put back on 0.314.0. Every agent-coupled feature (drives, same-disk, format, network storage) is untestable on the scratch box today, and the NAS leg of build A is proven by a red-proofed unit test only. Evidence: `audits/storage-identity-buildA-2026-10-15/9202-upgrade.txt`. | **OPEN — the exact gap, read 2026-10-19 (no agent change tonight):** the agent mints a per-guest local-API token in ONE place, `--selftest=provision`'s back half (`cmd/felhom-agent/main.go` `bh.Provision` after `engine.RunBringUp`), which only runs for a guest it brings up from a golden archive in the same call. There is no product way to mint for an EXISTING guest (9202) — no subcommand, no local-API route; the token store `/var/lib/felhom-agent/local-tokens.log` is written only by that path. Options: an agent subcommand `--mint-guest-token ` (owner-only, writes bootstrap's `local_api`), or re-provision 9202 through `selftest=provision` (destroys and rebuilds the scratch guest — not done; rule 6). Until then 9202 is controller-only. OPEN **2026-10-20 — left out of agent 0.161.0 on purpose:** the code would be small (a root-only subcommand that mints the token and adds `local_api` to an existing guest's `bootstrap.json`), but an agent key makes 9202's controller book the agent's whole-guest backups, one tier of which writes to ep0 (the demo-hp namespace) — and 9202 is outside the `felhom` pool. Needs, before it ships: a way to give a scratch guest a key with NO whole-guest tiers (or the operator's word that 9202 may write to ep0), and 9202 in the pool. | — | Give 9202 a local-API token the normal way (the agent mints it for its guest), or record in `runbooks/target-selection.md` that 9202 is controller-only; then show the NAS notice live. | CC | | **R-1081** | Process & tooling | P4 | **Two published controller tags hold content that is not what their number says: `0.315.0` and `0.316.3`.** Both were caused by CC on 2026-10-15, and both by the same shape — a guard that was not on the path the work takes. **`0.316.3`:** the build directory's `build.sh` is a symlink into the SHARED clone's working copy, which predated the `FELHOM_REPO_DIR` override, so the override was silently ignored and the build copied that tree — which held 31 uncommitted files of another session's physical-disk storage work. The banner printed `Commit: 0225255` with no `Repo:` line. **Nobody deployed it; that was luck, not a control.** Fixed at the root in controller v0.316.4: `build.sh` now refuses `--push` from a tree that is dirty, not at `origin/main`, or not a git checkout, with four test cases red-proved. **Neither tag is re-pushed or deleted** — a version number is pushed once, and that applies to numbers CC spoiled. **No box runs either:** demo-hp 9201 is on 0.317.0, the floor is 0.314.0, and the button offers the highest tag. **`0.315.0`:** Caused by CC on 2026-10-15 while proving R-1080's own fix: the guard was added to `controller/build.sh` in the repo, but the documented release command runs `$FELHOM_ROOT/build/felhom-controller/build.sh`, which was a **separate copy taken 2026-07-18** and had no guard. Running the refusal against a real existing tag therefore rebuilt and re-pushed 0.315.0 — a fourth time — with v0.316.0's code under it (digest `8887343011d6` → `b809201841f7`). **Fixed at the root in the same session:** the build directory's script is now a symlink to the repo's, the old copy kept beside it as `build.sh.stale-copy-2026-07-18`, and the refusal was then proven to fire from the documented path. **No box is affected:** nothing runs 0.315.0 (demo-hp 9201 is on 0.316.2), the floor is 0.314.0, and the household button offers the highest tag. What remains is two tags that must never be deployed and are no longer reproducible from the registry. | **OPEN** — filed 2026-10-15; nothing to do unless someone needs one of those two images, and then the answer is the current release | — | Leave them. If an audit ever needs the real 0.315.0 or 0.316.3, build it from the matching commit under a NEW number and say so | CC | -| **R-992** | Process & tooling | P3 | **The drill-catalog route that `09` §6.5 documents cannot be followed: `admin/app-catalog-drill` is PRIVATE and no box has catalog credentials.** Measured 2026-10-12 while publishing the Usenet pair: the box's clone fails with `fatal: could not read Username for 'https://gitea.dooplex.hu'` (exit 128), the Gitea API reports the drill repo `private=true` and the live catalog `private=false`, and NEITHER the current controller.yaml on 9202 NOR any of its 30 saved copies has ever carried `git.username` or `git.token` (the controller supports both — `internal/config/config.go` GitConfig). The route worked for the campaigns of 2026-10-10 and the torrent pair earlier the same day, so the repository's visibility changed between then and now. **Worked around, not fixed:** that campaign staged its two templates in the box's own catalog cache (`audits/usenet-2026-10-12/tools/stage_in_cache.py`), which is where the sync copies a catalog change anyway, with the sync interval raised to 12h for the duration and rescan used in place of sync. The real fix is an operator decision: either a read-only catalog credential on the scratch box (a secret on a Tier-0 guest) or making the drill repository readable. Until then every new-app campaign hits this. | WAITING-ON-OPERATOR | the operator's choice between a box credential and a readable drill repo | Decide; then either set `git.username`/`git.token` on 9202 from the credentials store, or flip the drill repo, and correct `09` §6.5 either way | operator | | **R-1034** | Process & tooling | P3 | **Gate 17 checks the FAQ JSON-LD on two pages only, by a hard-coded list — a FAQPage anywhere else is unchecked.** `scripts/site_gates.py` gate 17 loops `for p in ("gyik.html", os.path.join("en", "faq.html"))`, so its „JSON-LD equals the visible questions and answers” guarantee stops at those two files. **Since 2026-10-14 two more pages carry a FAQPage block the gate never reads:** `/sajat-felho-tarhely` and `/en/own-cloud-storage`. Their JSON-LD was *derived* from the visible headings and paragraphs by a script and then verified verbatim against the rendered text, so it is correct today — but nothing stops the next copy edit moving the visible text and leaving the structured data behind, which is exactly the failure gate 17 exists to catch, and the one Google penalises. **Why it is not a one-line fix:** gate 17's extractor is written for the FAQ pages' accordion markup (`