diff --git a/CONTEXT.md b/CONTEXT.md index aa65e4b2..2d96cd41 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -16,6 +16,15 @@ > and holds nothing of its own; this file does hold its own content, namely the standing rulings below. +> **2026-10-17 (evening) — the catalog allow-list, agent 0.159.1, break-glass rotation (operator D1 = A, D2 = A).** +> Controller v0.328.0 (floor, MinAgent 0.158.1): `internal/privpolicy` refuses container rights not on the list compiled +> into the controller before ANY compose write (design: `02-controller-module-map.md` „[DESIGN] The catalog +> allow-list"); hub v0.157.0 registers `app_definition_refused` (operator-only), moves `host_recovery.set_at` on a +> re-vault, and logs its own listing of each box's off-site `snapshots/` (R-895 slice 1, 07:10). Agent v0.159.1: +> `ForgetRetiredDrives` + `felhom-priv-apply unit-forget` (R-1124), read-only `felhom-thin-check` (R-1012; the step bundle +> `0.159.0-step1` is needed because the bundle adds a path). Break-glass rotated with the installer's step 4b ALONE +> (R-1141: a full re-run would also reinstall the vouched agent). `REPORT-agent-allowlist-rotation-2026-10-17.md`. + > **2026-10-15 (afternoon) — the release rule, `09` §3 decision 250 (operator 11:26/11:29):** a controller release needs > NO golden — push the image only when it is ready for every box (the household button offers the highest X.Y.Z tag; > `-rc.N` tags are skipped), raise the floor with the declared MinAgent after Tester 1, demo-hp and demo-felhom ran it diff --git a/REPORT-agent-allowlist-rotation-2026-10-17.md b/REPORT-agent-allowlist-rotation-2026-10-17.md new file mode 100644 index 00000000..9e6ba4b8 --- /dev/null +++ b/REPORT-agent-allowlist-rotation-2026-10-17.md @@ -0,0 +1,152 @@ +# REPORT — agent 0.159.1, the catalog allow-list (D2 = A), break-glass rotation (D1 = A), off-site history check (2026-10-17) + +Written in ASD-STE100 Simplified Technical English. Evidence: `documentation/audits/agent-allowlist-rotation-2026-10-17/` +(P1 agent, P2 allow-list, P3 rotation, P4 off-site design, P5 release). Architecture read: `03-host-agent.md`, +`02-controller-module-map.md`, `01-topology-and-trust.md`, `07-backup-architecture.md`, `06-offsite-connectivity.md`, +`09-update-architecture.md` §3 decision 250, `runbooks/break-glass.md`. + +## For the operator — plain words first + +**Decisions I took myself:** none. + +1. **Agent 0.159.1** (0.159.0 is superseded — do not vouch it). New Go. A drive that was decommissioned and then + removed is now forgotten: its host mount unit goes away within 20 seconds, so the host no longer waits for it. + Before a whole-box backup, and when one fails, the agent checks the thin pool for an orphan snapshot and says so in + the failure mail; it never deletes one (the repair is a runbook for you). All of it proven on demo-hp. **It runs on demo-hp, Tester 1 and + demo-felhom. Your vouch in the hub is still open** (the hub names 0.158.1 at 17:58). +2. **The app list cannot give an app dangerous rights any more** (controller 0.328.0, the floor for every box since + 17:22 local). Before the box writes any app's settings it checks for dangerous rights. Only homepage's read-only + Docker socket is allowed, and only for homepage's own image and start. A refused change writes nothing, the app + keeps running, you get a mail, and the app page says one line. Proven on demo-hp 9201 with a test catalog. Three + security reviews found eight gaps in the first versions; all eight were fixed before the release. +3. **New emergency root passwords** on demo-hp, Tester 1 and demo-felhom (random, your answer). Each new password + works; each old one is refused. Read them in the hub. Tester 2 is off: it gets its new password when it comes back. +4. **Off-site history:** the hub can count a box's off-site copies itself, with the login it already has. Hub 0.157.0 + (deployed on your yes) logs that count every morning at 07:10. The alarm on it is the next step. + +**What needs you:** +- **Vouch agent 0.159.1** in the hub — all three test hosts already run it; without the vouch a NEW box installs 0.158.1. +- **Decide R-1140:** keep the hub's access logs for 90 days. Pick A (a small hub-only log, recommended) or B (longer + logs for every container on DooPlex). If you do nothing: the next „did anyone use the key?" question has only about + 13 hours of logs again. + +## 1. Agent (Part 1) + +- **Content:** Go 1.27.2 + x/crypto v0.58.0 (R-972); R-1124 `ForgetRetiredDrives` + root verb `felhom-priv-apply + unit-forget`; R-1012 read-only root helper `felhom-thin-check` + `internal/thincheck` + `BackupRunner.SetThinCheck`; + 0.159.1: the signed decommission prunes the guest-bind record (found by 0.159.0's own proof). Releases: 0.159.0 + (`3cfd0a25…`, bundle `40663c4f…`), step bundle `0.159.0-step1` (`1d65edb3…`), **0.159.1** (`de8f6ad9…aff9`, bundle + `2af3d27d…7cd6`). +- **demo-hp proof (normal delivery: signed `agent_update` → step bundle → release bundle, 75/75 capabilities):** + - R-1124: a loop-backed test drive on `/mnt/hdd_1` enrolled through the agent's local API (the controller's + calls); the customer decommission refused it (a loop device reads as „system" — correct), so the operator-signed + decommission was used; while present the unit stayed; after the unplug the agent removed the unit and its wants + link in 8 s and `GET /disks` dropped it (5 → 4 rows). On 0.159.1 a second signed decommission pruned the bind + record; the 20-second warning stopped (0 lines after, the reconcile tick still running — 3 lines, the control). + - R-1012: on a scratch thin pool (loop file on `/mnt/hdd_1`, never `pve/data`) the helper listed a made orphan + (device 7) and refused a metadata snapshot held by someone else; on `pve/data` it read ids 1, 2 = LVM's and + released. A household-press whole-guest backup of 9201 ran the check first (the sudo line) and finished green + (6.77 GB). Note on the fence „never write into pve/data": reading a live pool needs a metadata snapshot + (`reserve_metadata_snap` / `release_metadata_snap`); it changes no data and no mapping. This is what the product + now does before every snapshot backup — if you want that stopped, say so (one line in the agent). + - Scratch pool, loop devices, test image files, the test drive's folders: removed. One residue kept: the drive's + `decommissioned` intent line on demo-hp (harmless by design). +- **Vouch:** the operator said yes (STOP, ~17:00). Delivered on that yes by signed jobs: Tester 1 (agent 17:39:31, bundle + 17:53:42, 75/75) and demo-felhom (agent 17:40:51, bundle 17:55:00, 75/75), each through the step bundle; the hub has + reports from both after. **The vouch itself (Configs → Day-0 artifacts) is not done yet** — new installs still take + 0.158.1 until it is. Tester 2: off; it takes 0.159.1 when it is back (signed jobs, as above). + +## 2. The catalog allow-list (Part 2, R-971 — closed) + +- **Survey** (`P2/survey.md`): 123 files (65 templates, 58 ladder steps). The only class member: homepage's read-only + Docker socket. That is the whole list. +- **Built:** `felhom-controller internal/privpolicy` on the RENDERED compose; wired into every compose writer and env + decision point; structural guard `TestAllowList_IsWiredIntoEveryComposeWriter` (AST). Refusal → nothing written, + operator event `app_definition_refused` (hub ≥ 0.157.0), app-page line, plain message on install/settings/restore. + The sync never copies a template named like a controller infra stack. Catalog mirror: `check-privileges.py` (stdlib, + in `--fast`, decoys; the reader equals PyYAML on all 123 files); the two lists are byte-identical, pinned from both + sides (boring choice: two copies + one equality check, no network in CI). +- **Security reviews (three rounds, eight findings, all fixed and red-proved before the release):** the entry bound to + its image, entrypoint/command and exact healthcheck; symlinks out of a root (nearest existing ancestor, fail closed); + missed classes (cgroup_parent, runtime, isolation, gpus, device reservations, provider, lifecycle hooks); roots only + under `/mnt/felhom-drives/` or the system data path; any `extends` refused; no variable in a bind's options; YAML + anchors refused by the catalog reader; under `${HDD_PATH}` only `appdata/` (not `backups/`). +- **Live finding on 9201 with rc.4:** every not-installed app was refused (placeholder roots do not exist on the box, + so the root itself could not be resolved). Fixed in rc.5 (roots resolved through their nearest existing ancestor; no + symlink resolve for the placeholder check), red-proved. Its 30 events were dropped by the old hub — no false mail. +- **9201 proof (rc.5, hub 0.157.0):** a drill catalog = live + `privileged: true` on privatebin → refused; the compose + sha `79e7877d…` unchanged; the same privatebin container (`494270e4…`, started 14:34:24Z) still running; the event at + the hub at 17:10:34 and an operator mail; the page line on privatebin, not on opengist (control). Restored: live + catalog, controller.yaml byte-identical, the drill mirror removed, the drill repo back to the live tree. +- **Every installed app on the three test boxes still passes:** 0 refusals after 0.328.0 started (P5/box-walk.txt). + +## 3. Break-glass rotation (Part 3, R-954 — closed) + +| host | how | proof | +|---|---|---| +| demo-hp | installer step 4b ALONE (`installer-v1.33.0`'s own function) | new value → PVE API 200; old → 401; values differ | +| Tester 1 | the same | the same | +| demo-felhom | the same, on your answer (random) | the same; hub `set_at` now 2026-10-17T15:10:04Z | +| Tester 2 | offline since 2026-10-04 | R-1139 | + +- Why not the full installer re-run (the brief's `--rotate-recovery`): it also reinstalls the vouched agent and the root + files — on demo-hp it would have moved the candidate agent back. R-1141 asks for a standalone mode. +- Found and fixed: the hub kept the FIRST vault's `set_at` (demo-hp/Tester 1 still read their old dates, rotated + before the fix; demo-felhom shows the new one) — hub v0.157.0. +- The passwords were read with the global-key curl (P3/`cred.py`, prints only length and time): that path writes NO + customer-visible „revealed" line. Nothing was printed or filed. +- **Log retention:** ingress logs cover ≈ 13 h (k3s 10 MiB × 5). Not built: it needs a DooPlex or hub-storage change → + R-1140 (cost in the row). + +## 4. Off-site history (Part 4, R-895 — open, READY) + +- **Measurement:** the login the hub already holds (the sealed sub-account password, write-capable) lists the folder; + upload times are shown (2026-09-01 evidence on demo-hp's own sub-account). A fresh listing from DooPlex was NOT + made: tried the CC store (no Storage Box credential), the box's key (restic REST only, names and sizes, and it is the + box's key), the hub (no endpoint); reading the sealed password out of the hub was not done. +- **Design:** `P4/design-R-895.md` — before/after counts from the hub's own listing, files no box run explains, + snapshots gone outside a window; a read-only sub-account as the stricter alternative. +- **Built (slice 1, ~1 h, no new credential):** hub v0.157.0 lists `snapshots/` in the 07:10 key audit and logs the + count, newest and oldest time. Deployed on your yes (17:03, Synced/Healthy, reports from demo-hp and Tester 1, floor + read 0.328.0). + +## 5. Versions, commits, fixtures + +- **felhom-agent:** `e15d1d3` (code), `v0.159.0` tag, `4a1c07b`-line (prune), `v0.159.1` tag, CHANGELOG `…`. CI green + (job 2025). +- **felhom-controller:** `6d90374`, `24d937d`, `36c4aa4`, `09ac87a`, `99f9117` → **v0.328.0** (rc.1–rc.5 tested on + 9201), MinAgent 0.158.1. Global floor 0.328.0 / 0.158.1 at ~17:22 local; per-box floors set and cleared. CI green + (job 2032). **i18n fixture:** new `app_settings_definition_refused`; every other fixture byte-identical. +- **app-catalog-felhom.eu:** `d5a443d` (gate) + three follow-ups; CI green (job 2033). +- **felhom.eu:** `8f97b800` (hub v0.157.0 code, installer uninstall line, runbook), `d968f849` (hub deploy); docs and + register in the final commit. + +## 6. Rows + +Before **174**, after **175**. Closed (4): R-971, R-954, R-1124, R-1012. Opened (5): R-1139, R-1140, R-1141, R-1142, +R-1143. Updated: R-895 (READY with the design), R-972. + +## 7. Observations + +1. Agent v0.159.0's signed decommission left a guest-bind record → a warning every 20 s. NOT-A-FINDING: fixed in the + session (agent v0.159.1, the size rule), red-proved. +2. Hub kept the first vault's `set_at`. NOT-A-FINDING: fixed in the session (hub v0.157.0, the size rule), red-proved. +3. A full installer re-run is the only `--rotate-recovery` path. FILED: R-1141. +4. Ingress access logs ≈ 13 h. FILED: R-1140. +5. One app can bind another app's `appdata/`. FILED: R-1142. +6. Controller `build.sh` pushes `:latest` on every build, rc included. FILED: R-1143. +7. The first allow-list rc refused every not-installed app on the box. NOT-A-FINDING: found by this session's own live + proof on 9201 and fixed before the release (rc.5, red-proved). +8. homepage keeps a read-only Docker socket on the list; a read-only mount does not limit the Docker API. + NOT-A-FINDING: already R-782's subject (homepage's exposure); the list keeps today's behaviour, it does not bless it. +9. Tester 1's ISO-era `--rotate-recovery` warning text („the old one now fails at the PVE web GUI") is printed by the + step itself. NOT-A-FINDING: true and intended. + +## 8. Teardown (three layers) + +- **Machine:** demo-hp — scratch VG/pool/loop removed, test image files removed, test drive folders removed; 9201 back + on the live catalog (controller.yaml byte-identical), drill mirror removed. Tester 1, demo-felhom: only the releases + and the password rotation. +- **Host:** demo-hp, Tester 1, demo-felhom run controller 0.328.0; demo-hp, Tester 1 and demo-felhom run agent 0.159.1 (bundle 0.159.1). +- **Hub:** hub 0.157.0; global floor 0.328.0; no per-box floor; signed jobs consumed. Instruction files: none edited. + System poster: no fact changed (none of the three). diff --git a/STATUS.md b/STATUS.md index 0fdde25b..12987e95 100644 --- a/STATUS.md +++ b/STATUS.md @@ -2,7 +2,7 @@ **Ready for the first real tester (Tester-2): yes. Tester 2 (a laptop) is off; nothing was sent to it.** -**Updated 2026-10-17 (afternoon, later).** Controller 0.327.1 is the floor for every box since 12:06 UTC: FileBrowser is published only after its default password is checked (your D11 = A), and the install check uses real memory amounts with a clear message (your D10 = A) — `REPORT-filebrowser-memory-ci-2026-10-17.md`. **Earlier:** Controller 0.326.0 is the floor for every box since 09:55 UTC: FileBrowser's default password is checked at every FileBrowser start. Tester 1 is back on the live app list (test apps and drives removed); six memory limits raised — `REPORT-tester1-limits-filebrowser-2026-10-17.md`. **Earlier today:** controller 0.325.0 was the floor since 08:21 UTC (after the three test boxes): two more password doors count a try before checking it, family sign-ins are stored as fingerprints, a file-share rename always applies; five apps' Redis saves snapshots now, which a full disk cannot damage — `REPORT-burndown-3-2026-10-17.md`. **Earlier (2026-10-17 night):** controller 0.324.0 was the floor for every box since 02:46 UTC (all three test boxes made tonight's off-site backup on it first): a restore now brings the database password back with the data (your 1A), restic 0.19.1 (your 2A), dashboard hardening — `REPORT-night-2026-10-17.md`. **Earlier (2026-10-16 evening):** 0.323.0, Debian 13 — `REPORT-debian13-restic-2026-10-16.md`. **Earlier (midday):** Controller 0.319.0 (the app page in panels, the login in one panel) is the floor, MinAgent 0.158.1 — `REPORT-app-page-tuning-2026-10-16.md`. **Earlier (morning):** Storage identity build B shipped: agent 0.158.1 and golden 0.318.0 vouched by you, controller 0.318.0 is the floor (MinAgent 0.158.1), installer 1.33.0 published — new one-disk boxes get their own data area. Open items: see `documentation/backlog/OPEN-ITEMS.md`. Older notes, day by day: +**Updated 2026-10-17 (evening).** Controller 0.328.0 is the floor for every box (17:22 local): the box refuses a catalog change that asks for dangerous container rights (your D2 = A). Agent 0.159.1 (a gone, decommissioned drive is forgotten; an orphan thin snapshot is named in the backup failure; new Go) — runs on demo-hp, Tester 1 and demo-felhom; **your vouch in the hub is still open**. New emergency root passwords on demo-hp, Tester 1, demo-felhom (your D1 = A; read them in the hub). Hub 0.157.0 deployed on your yes — `REPORT-agent-allowlist-rotation-2026-10-17.md`. **Earlier:** **Updated 2026-10-17 (afternoon, later).** Controller 0.327.1 is the floor for every box since 12:06 UTC: FileBrowser is published only after its default password is checked (your D11 = A), and the install check uses real memory amounts with a clear message (your D10 = A) — `REPORT-filebrowser-memory-ci-2026-10-17.md`. **Earlier:** Controller 0.326.0 is the floor for every box since 09:55 UTC: FileBrowser's default password is checked at every FileBrowser start. Tester 1 is back on the live app list (test apps and drives removed); six memory limits raised — `REPORT-tester1-limits-filebrowser-2026-10-17.md`. **Earlier today:** controller 0.325.0 was the floor since 08:21 UTC (after the three test boxes): two more password doors count a try before checking it, family sign-ins are stored as fingerprints, a file-share rename always applies; five apps' Redis saves snapshots now, which a full disk cannot damage — `REPORT-burndown-3-2026-10-17.md`. **Earlier (2026-10-17 night):** controller 0.324.0 was the floor for every box since 02:46 UTC (all three test boxes made tonight's off-site backup on it first): a restore now brings the database password back with the data (your 1A), restic 0.19.1 (your 2A), dashboard hardening — `REPORT-night-2026-10-17.md`. **Earlier (2026-10-16 evening):** 0.323.0, Debian 13 — `REPORT-debian13-restic-2026-10-16.md`. **Earlier (midday):** Controller 0.319.0 (the app page in panels, the login in one panel) is the floor, MinAgent 0.158.1 — `REPORT-app-page-tuning-2026-10-16.md`. **Earlier (morning):** Storage identity build B shipped: agent 0.158.1 and golden 0.318.0 vouched by you, controller 0.318.0 is the floor (MinAgent 0.158.1), installer 1.33.0 published — new one-disk boxes get their own data area. Open items: see `documentation/backlog/OPEN-ITEMS.md`. Older notes, day by day: `documentation/backlog/STATUS-HISTORY.md`. Reports: `REPORT-night-2026-10-14.md`, `REPORT-immich-rollout-2026-10-14.md`, `REPORT-seo-felho-tarhely-2026-10-14.md`, `REPORT-immich-repair-2026-10-14.md`, `REPORT-power-cut-fixes-2026-10-14.md`, `REPORT-night-power-2026-10-13.md`, `REPORT-poweroff.md`. ## What works @@ -34,7 +34,7 @@ - **One-disk boxes get their own data area** (2026-10-16): a new install makes a „Belső tárhely" on the disk, outside every whole-box copy, and media apps install there with no extra step (proven on a fresh 512 GB box: 213 GiB area, Jellyfin with no extra step). A disk split into parts can be used part by part. The backup offer no longer promises disk-failure safety when it cannot check the disk, and `/mnt/sys_drive` is no longer offered. ## What is broken -- Several root-level weaknesses wait on your four decisions below (R-954, R-971, R-982, R-985). +- Two root-level weaknesses wait on your decisions below (R-982, R-985). Tester 2 still has its old emergency password (R-1139, when it is back). - DooPlex: a failed disk with its alarms silenced (R-934); CI jobs still get lost (R-887); something sets the shared checkout to „bare" (R-935); a Longhorn volume at 91 %. - Tester 2 has never made an off-site check (R-862). The scratch box cannot read the drill catalog (R-992). @@ -42,27 +42,29 @@ ## What we work on - **CC's secrets** (R-1118): done 2026-10-17 — the store, `sec`, the guards in every CC session (option B: `bash -x` and `curl -v` allowed inside `sec run`), the nightly check (linger on). The old credentials file and its three copies are deleted. - **After 2026-10-18 14:16 UTC** (demo-hp's next clean-up window): check that the old restic (image 0.322.0) still lists and restores demo-hp's store after its first real 0.19.1 prune (`audits/night-restore-2026-10-17/tools/rx.sh`). Everything else of R-1105/R-1107 is done. -- The power-cut findings still open (R-1004, R-1005, R-1010, R-1012, R-1013). +- The power-cut findings still open (R-1004, R-1005, R-1010, R-1013). R-1012 closed: agent 0.159.1 names an orphan thin snapshot in the backup failure. - Media apps on every drive (R-1023): live with the 0.313.0 floor — every drive in every media app, a new drive made ready by the box, a pulled drive loses nothing, a safe removal lets go first and the drive comes back by itself. ## Waiting on you — decide | # | Row | Question | Pick | The other option costs | If you do nothing | Blocked | |---|---|---|---|---|---|---| -| D1 | R-954 | Change every box's break-glass root password, since we cannot see if the old key was used? | **Yes** — about one hour, one box at a time, through the vault | Accept: no work, a password that may have been read stays valid | the old root passwords stay; if they were read, they still open the boxes from the LAN or console | — | -| D2 | R-971 | Should the box refuse a catalog change that asks for dangerous container rights? | **Yes — an allow-list on the box** (one controller release) | Keep trusting the catalog repo: whoever can push to it reaches every box | a catalog push keeps reaching every box's apps with no check | — | +| D1 | R-954 | **DECIDED 2026-10-17: A** — new random emergency root passwords on demo-hp, Tester 1 and demo-felhom (your answer: random); each new one works, each old one is refused. Tester 2: R-1139. | done | — | — | — | +| D2 | R-971 | **DECIDED 2026-10-17: A** — the box refuses a catalog change that asks for dangerous container rights (controller 0.328.0, every box). Only homepage's read-only Docker socket is allowed. A refusal keeps the app running and mails you. | done | — | — | — | | D3 | R-985 | „Zero-knowledge": change the wording, or remove the operator's root reach to the key? | **Change the wording now**; revisit after the first sale | Remove the reach: a design change to break-glass | the promise stays stronger than the product | the website text | | D4 | R-982 | Root SSH with a password from the LAN? | **No — key only**; the password stays for the console | Keep it: anyone on the household network can try it | the host keeps accepting root password logins from the LAN | — | | D5 | R-993 (R-991) | Approve the minimum machine (`audits/sizing-2026-10-13/SIZING.md` §5: x86-64, 4 cores, 8 GB, 256 GB SSD)? | **Approve** | Change it: one edit | no hardware page; a buyer with a Raspberry Pi can still download the installer | the hardware page | | D6 | R-992 | The scratch box cannot read the private drill catalog: give it a read-only login, or make that copy readable? | **A read-only login** | Readable copy: the drill catalog becomes public | every new-app test needs hand-staging | new-app tests | | D7 | R-990, R-989 | Rule yes / no / later on the forum ideas (`audits/forum-research-2026-10-12/FELHOM.md` §5) and the selfh.st ideas (`audits/selfhst-survey-2026-10-12/FELHOM.md` §6) | — | — | nothing changes; the research ages | — | | D8 | R-206 | Install DooPlex's Docker settings from git (restarts Docker), and prune the build cache weekly: yes or no? | **Yes, in a quiet window** | — | a rebuild of DooPlex still has the settings in git | — | +| D13 | R-1140 | Keep the hub's access logs 90 days (today about 13 hours)? | **A — a small hub-only access log on the hub's own disk** (≈ 15 MB for 90 days) | B: longer logs for every container on DooPlex (≈ 1.3 GB compressed) | the next „was the key used?" question has about 13 hours of logs | — | | D10 | R-1125 | **DECIDED 2026-10-17: A** — the 23 memory requests now match real use (catalog `1af6ea7`); the install refusal and page say what to do (controller 0.327.0). An 8 GB box no longer installs the whole household set. | done | — | — | — | | D11 | R-1126 | **DECIDED 2026-10-17: A** — FileBrowser is published only after its default password is checked; it never answers on the default (controller 0.327.1). The tunnel still runs before the claim, by design (`01` §5). | done | — | — | — | | D12 | R-1129 | **CLOSED 2026-10-17: not a defect** — CC misread the CI log; the failure mail always worked. A forced test mail went to admin@felhom.eu at 10:48:54 UTC (CC saw it arrive). Your key change did no harm. | done | — | — | — | | D9 | R-1062 | The N100's USB drive is also its whole-box backup drive. Split it in two anyway? | **No — keep one drive**; the lived-in box keeps its nightly whole-box backup | Split: the box loses its local whole-box backup (or add a second drive first) | it stays one drive; multi-drive tests stay on Tester 1 | — | ## Waiting on you — do +- **Vouch agent 0.159.1** in the hub (Configs → Day-0 artifacts: 0.159.1, sha256 `de8f6ad9…aff9`, bundle `2af3d27d…7cd6`). The three test hosts already run it. If you do not: a new box installs 0.158.1. - **Nézd meg a demo-hp-n a Calibre-Web „Adatok és mentés" fülét és a Fájlkezelőt** (`felhom.enkisfelhom.hu`, controller 0.321.0, minden dobozon kiadva). A mentés egy részben, egyszerű szavakkal; a Fájlkezelő nem kér frissítést és nem ajánl „Install app"-ot. - **Look at `roms/roms/` on the N100's drive in the file manager (R-1099).** RomM now scans `roms//`. A game uploaded through RomM before today sits one folder deeper: if there are platform folders in `roms/roms/`, move them up into `roms/` and press „Scan” in RomM. If not: those games show under a platform called „roms”. - **Nézd meg a Calibre-Web oldalát a demo-hp-n** (`felhom.enkisfelhom.hu`, controller 0.319.0, minden dobozon kiadva). A fejléc, a fülsor és minden rész most külön panel (az „A” stílus). A belépési adatok egy panelben vannak, egymás mellett. A Calibre-Web jelszaváról a doboz nem tud, mert az alkalmazás előbb települt, mint hogy a sablonban jelszó lett volna — ezt most egy mondat mondja ki. Ha nem tetszik, írd meg, mit változtassak. Ha nem nézed meg: semmi nem romlik el. diff --git a/documentation/architecture/01-topology-and-trust.md b/documentation/architecture/01-topology-and-trust.md index 8743dd4f..9e8dd41d 100644 --- a/documentation/architecture/01-topology-and-trust.md +++ b/documentation/architecture/01-topology-and-trust.md @@ -169,6 +169,12 @@ second layer. **How it is checked:** `scripts/https_only_check.py` lists every p routes and demands 301/308 to the same https URL (negative control: a stub answering plain 200 must be convicted). Evidence: `audits/https-only-2026-10-14/`. +**What the catalog may ask of the box (controller v0.328.0, R-971, decision D2 = A).** The catalog repository is trusted +for WHICH apps exist and how they are configured — not for container RIGHTS. A template that asks for a right not on the +allow-list compiled into the controller (privileged mode, capabilities, devices, host namespaces, the Docker socket, +binds outside the box's data roots, …) is refused on the box before its compose is written, and the operator is told +(`02-controller-module-map.md` „The catalog allow-list"). Signed catalog releases remain the later step R-971 names. + **Who may reach an app, and through what (recorded 2026-09-29 — no document said it before; spike finding F1).** Every app is reached only through the box's traefik (no catalog app publishes a host port except crafty-controller's game ports; none uses host networking — read from the catalog 2026-09-29). traefik routes by host name: the tunnel's diff --git a/documentation/architecture/02-controller-module-map.md b/documentation/architecture/02-controller-module-map.md index 91f8ca50..e237852e 100644 --- a/documentation/architecture/02-controller-module-map.md +++ b/documentation/architecture/02-controller-module-map.md @@ -727,6 +727,46 @@ Cloudflare edge served 0.315.0's first stylesheet under the address its third bu --- +## [DESIGN] The catalog allow-list — the container rights the box gives (controller v0.328.0, R-971, operator decision D2 = A, 2026-10-17) + +**What.** Before ANY app compose is written — the install, a settings change, the 15-minute correction, a guarded update +or ladder step, a unit or off-site restore — the controller renders the compose with the app's own values (exactly as +`compose up` will see them) and asks `internal/privpolicy.Admit`. A setting in one of these classes is refused unless +this app's service has EXACTLY that setting on the allow-list: `privileged`, `cap_add`, `devices`, +`device_cgroup_rules`, `gpus`, deploy device reservations, `network_mode`/`pid`/`ipc`/`uts`/`userns_mode`/`cgroup`/ +`cgroup_parent`/`runtime`/`isolation`, `security_opt` other than `no-new-privileges`, `sysctls`, `volumes_from`, `build`, +`include`, ANY `extends`, a `provider` service, `post_start`/`pre_stop` hooks, an `env_file`/`configs`/`secrets` file +outside the stack dir, a named volume with `driver`/`driver_opts`/`external`/`name`, a `${VAR}` in any of those keys, +and binds: the Docker socket anywhere, mount propagation, a `${VAR}` in a bind's options, and any source outside the +data roots the box itself gives — `${HDD_PATH}/appdata/`, `${USERDATA_PATH}`, `${IMPORT_PATH}` (a bind source may use +only those variables, as its leading token; each root must lie under `/mnt/felhom-drives/` or the system data path; on +the box a source is judged where it really points, through its nearest existing ancestor, so a symlink cannot carry it +out). + +**Where, and why in code.** The list is `internal/privpolicy/allowlist.json`, compiled in with `go:embed`: a change in +Gitea — the threat R-971 names (whoever can push to the catalog can take every guest) — cannot add to it; a new entry +needs a controller release. An entry is app + service + exact setting, and holds only while that service runs the +entry's image REPOSITORY (a literal reference), with the image's own entrypoint and command, and the exact +`healthcheck.test` the entry names — so a push cannot keep the allowed right and change what runs with it. Today: one +entry, homepage's read-only Docker socket (the survey of all 123 catalog files found nothing else, +`audits/agent-allowlist-rotation-2026-10-17/P2/survey.md`). The catalog's CI gate `check-privileges.py` mirrors the +check with a byte-identical copy of the list (pinned from both sides); the BOX is the authority. + +**What a refusal does.** Nothing is written. A refused correction or update leaves the app on its last good compose, +running; the controller raises the operator-only hub event `app_definition_refused` once per refused definition (hub ≥ +v0.157.0) and the app page carries one line (`app_info.definition_refused`). A refused install, settings change or +restore is refused with a plain message („Semmi nem változott"); a restore asks BEFORE it stops the app. A check +that cannot read the compose refuses. The sync never copies a catalog template named like a controller infra stack. + +**Out of scope, stated.** The controller's own infra stacks (traefik, cloudflared, filebrowser, samba) and its override +files (setup gate, family gate, install hold, media library) are rendered by the controller, not the catalog. Published +host ports are not a class (crafty-controller's game ports). Known limit: under `${HDD_PATH}/appdata/` one app can +name another app's folder (a per-app rule needs a name map — paperless-ngx keeps `appdata/paperless`), R-1142. + +**Pinned by** `internal/privpolicy/privpolicy_test.go` (each class refused, with positive controls), +`stacks.TestAllowList_IsWiredIntoEveryComposeWriter` (AST: every compose writer and every env decision point asks +first), `sync.TestR971_*` (a refused correction writes neither file and is not reported updated). + ## The app-definition seam — what happens to a deployed app when its compose file is rewritten > **STATUS: MEASURED BEHAVIOUR, NOT A RECORDED DESIGN.** This section describes what the system was diff --git a/documentation/architecture/03-host-agent.md b/documentation/architecture/03-host-agent.md index 2c93fd80..7bdcdd08 100644 --- a/documentation/architecture/03-host-agent.md +++ b/documentation/architecture/03-host-agent.md @@ -113,6 +113,8 @@ fixed file, delivered by the signed config bundle): | `FELHOM_CONTROLLERSWAP` | the managed controller update | exact vmid; image ref pinned to `gitea.dooplex.hu/admin/felhom-controller:X.Y.Z` for the image check; the inspect template stays free text; **the write goes through `felhom-priv-apply controller-image `** (the ref on stdin, re-checked as root against the same pattern — R-861 (a) A1, `09` §3 decision 165, agent after v0.150.0) | guest-scoped: a compromised agent can restart the guest's bootstrap and choose an OLDER version of OUR controller image (a downgrade); it can no longer hand the guest any other image. Until the A1 bundle reached a box, the agent's in-guest `tee` let it write any ref (sudo cannot see stdin — the 2026-10-06 night correction, `audits/night-burndown-2026-10-06/design-R-861.md`) | | `FELHOM_STALELOCK` / `FELHOM_SCRATCH_TEARDOWN` | stale-lock clear; failed restore-test scratch | exact vmid; the scratch band `99000[0-9]` was already exact | — | | `FELHOM_VZPARTIAL` | remove a dead half-written vzdump (R-1006, agent v0.156.1) | ONE line: `felhom-vzdump-partial-clean `; the root helper (a config-bundle file) re-checks: a `dir` storage from `storage.cfg`, the name of that vmid (`.tar.dat`, `.vma.dat`, `.tmp`), not a symlink, directly in `/dump`, no vzdump on the host, nothing holds it open, dead (before this boot or quiet 120 s) | a compromised agent can delete a dead partial — never a finished archive, never a live one | +| `FELHOM_THINCHECK` | name an ORPHAN thin device around a whole-guest backup (R-1012, agent v0.159.0) | ONE line: `felhom-thin-check `; the root helper (a config-bundle file) lists LVM's thin ids before and after and the pool's ids through a metadata snapshot it reserves and releases; it refuses (exit 3) when someone else holds a metadata snapshot and sends no other pool message. The agent reports an id in the pool and in neither LVM list (two reads) in the backup failure's text; the repair (`runbooks/whole-guest-backup-orphan-thin.md`) stays human | a compromised agent can READ the pool's device ids — it cannot delete, create or change a thin device | +| `FELHOM_MOUNT` `unit-forget` | forget a decommissioned drive that is gone (R-1124, agent v0.159.0) | `felhom-priv-apply unit-forget mnt-<…>.mount` removes ONE unit file, only a felhom by-UUID drive unit (not a network unit) that is disabled, not mounted, and whose `/dev/disk/by-uuid` link is absent; the agent calls it at start and on the 20-s tick for drives whose intent is `decommissioned` | a compromised agent can delete the unit of an ABSENT, unmounted drive — the same effect as unplugging it; a present drive keeps its unit | | `FELHOM_WG` | the off-site tunnel | conf via `felhom-priv-apply wg` (only the keys `renderConf` writes; no `PostUp`/`PreUp`/`DNS`/`Table`; `/32` only) | — | | `FELHOM_SELFUPDATE` | commit / rollback of the A/B flip | **`apply` removed**: the flip runs only inside `felhom-os-apply agent_update`, after the operator signature, host, window and nonce are checked as root and the staged bytes are hashed ONCE and copied to a root-owned dir (`/var/lib/felhom-os-apply/agent-update/`); the wrapper accepts only that dir | — | | `FELHOM_SSHD` | the out-of-band operator sshd | config via `felhom-priv-apply sshd-config` (the ONE template, only the Port varies, never 22); the felhom-op key via `sshd-key` (one plain key, no `command=`/`from=` options) | felhom-op's key itself is hub-delivered, not signed: a compromised agent can install its own key for **felhom-op** — whose sudo is scoped (`felhom-op.sudoers`: fixed repair verbs and `pct start` / `stop` / `unlock` of ONE numeric vmid, anchored since R-861 (b) B2), not root — accepted (`09` §3 decision 165, B3) | diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/delivery-tester1-demofelhom.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/delivery-tester1-demofelhom.txt new file mode 100644 index 00000000..1dd35e90 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/delivery-tester1-demofelhom.txt @@ -0,0 +1,24 @@ +== root@192.168.0.154 +felhom-agent 0.159.1 +Oct 17 17:38:26 felhom felhom-agent[1278]: time=2026-10-17T17:38:26.285+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=076f93320a4eddaa op=agent_update +Oct 17 17:38:27 felhom felhom-os-apply[3912437]: os-apply: BUNDLE DONE agent=0.159.0-step1 written=1 same=28 self-check=ok signers-created=False +Oct 17 17:38:27 felhom felhom-agent[1278]: time=2026-10-17T17:38:27.406+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE DONE agent=0.159.0-step1 written=1 same=28 self-check=ok signers-created=False" +Oct 17 17:38:28 felhom felhom-agent[1278]: time=2026-10-17T17:38:28.285+02:00 level=WARN msg="osupdate: capability probe after the config bundle" ok=73 total=73 degraded="" +Oct 17 17:38:28 felhom felhom-agent[1278]: time=2026-10-17T17:38:28.285+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=8d9bf40b122037c7 op=agent_config_update +Oct 17 17:39:31 felhom felhom-agent[3912599]: time=2026-10-17T17:39:31.112+02:00 level=WARN msg="selfupdate: update committed" version=0.159.1 wrapper="" +Oct 17 17:53:42 felhom felhom-os-apply[3935466]: os-apply: BUNDLE DONE agent=0.159.1 written=3 same=27 self-check=ok signers-created=False +Oct 17 17:53:42 felhom felhom-agent[3912599]: time=2026-10-17T17:53:42.245+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE DONE agent=0.159.1 written=3 same=27 self-check=ok signers-created=False" +Oct 17 17:53:43 felhom felhom-agent[3912599]: time=2026-10-17T17:53:43.165+02:00 level=WARN msg="osupdate: capability probe after the config bundle" ok=75 total=75 degraded="" +Oct 17 17:53:43 felhom felhom-agent[3912599]: time=2026-10-17T17:53:43.165+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=33c061b404a445e1 op=agent_config_update +== felhom-pve +felhom-agent 0.159.1 +Oct 17 17:39:46 demo-felhom felhom-agent[1207]: time=2026-10-17T17:39:46.899+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=c994ded3d077a21b op=agent_update +Oct 17 17:39:47 demo-felhom felhom-os-apply[2418991]: os-apply: BUNDLE DONE agent=0.159.0-step1 written=1 same=28 self-check=ok signers-created=False +Oct 17 17:39:47 demo-felhom felhom-agent[1207]: time=2026-10-17T17:39:47.651+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE DONE agent=0.159.0-step1 written=1 same=28 self-check=ok signers-created=False" +Oct 17 17:39:48 demo-felhom felhom-agent[1207]: time=2026-10-17T17:39:48.278+02:00 level=WARN msg="osupdate: capability probe after the config bundle" ok=73 total=73 degraded="" +Oct 17 17:39:48 demo-felhom felhom-agent[1207]: time=2026-10-17T17:39:48.278+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=0f93931bec704435 op=agent_config_update +Oct 17 17:40:51 demo-felhom felhom-agent[2419165]: time=2026-10-17T17:40:51.620+02:00 level=WARN msg="selfupdate: update committed" version=0.159.1 wrapper="" +Oct 17 17:55:00 demo-felhom felhom-os-apply[2438672]: os-apply: BUNDLE DONE agent=0.159.1 written=3 same=27 self-check=ok signers-created=False +Oct 17 17:55:00 demo-felhom felhom-agent[2419165]: time=2026-10-17T17:55:00.806+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE DONE agent=0.159.1 written=3 same=27 self-check=ok signers-created=False" +Oct 17 17:55:01 demo-felhom felhom-agent[2419165]: time=2026-10-17T17:55:01.446+02:00 level=WARN msg="osupdate: capability probe after the config bundle" ok=75 total=75 degraded="" +Oct 17 17:55:01 demo-felhom felhom-agent[2419165]: time=2026-10-17T17:55:01.446+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=1e7a09eb163bd82a op=agent_config_update diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/demo-hp-agent-journal.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/demo-hp-agent-journal.txt new file mode 100644 index 00000000..c1e24cdb --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/demo-hp-agent-journal.txt @@ -0,0 +1,24 @@ +Oct 17 15:37:46 demo-hp felhom-agent[1266]: time=2026-10-17T15:37:46.203+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=e67e21347e268540 op=agent_update +Oct 17 15:38:50 demo-hp felhom-agent[3044359]: time=2026-10-17T15:38:50.528+02:00 level=WARN msg="selfupdate: update committed" version=0.159.0 wrapper="" +Oct 17 15:53:02 demo-hp felhom-os-apply[3098908]: os-apply: BUNDLE DONE agent=0.159.0-step1 written=1 same=28 self-check=ok signers-created=False +Oct 17 15:53:02 demo-hp felhom-agent[3044359]: time=2026-10-17T15:53:02.040+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE DONE agent=0.159.0-step1 written=1 same=28 self-check=ok signers-created=False" +Oct 17 15:53:03 demo-hp felhom-agent[3044359]: time=2026-10-17T15:53:03.082+02:00 level=WARN msg="osupdate: capability probe after the config bundle" ok=73 total=75 degraded=thin-orphan-check,mount-unit-forget +Oct 17 15:53:03 demo-hp felhom-agent[3044359]: time=2026-10-17T15:53:03.082+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=ed248025668b456b op=agent_config_update +Oct 17 16:08:02 demo-hp felhom-os-apply[3151947]: os-apply: BUNDLE DONE agent=0.159.0 written=3 same=27 self-check=ok signers-created=False +Oct 17 16:08:02 demo-hp felhom-agent[3044359]: time=2026-10-17T16:08:02.131+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE DONE agent=0.159.0 written=3 same=27 self-check=ok signers-created=False" +Oct 17 16:08:03 demo-hp felhom-agent[3044359]: time=2026-10-17T16:08:03.154+02:00 level=WARN msg="osupdate: capability probe after the config bundle" ok=75 total=75 degraded="" +Oct 17 16:08:03 demo-hp felhom-agent[3044359]: time=2026-10-17T16:08:03.155+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=f6b59d99eaed6396 op=agent_config_update +Oct 17 16:11:10 demo-hp felhom-agent[3044359]: time=2026-10-17T16:11:10.435+02:00 level=INFO msg="local-api: guest-bind recorded for startup re-assert" vmid=9201 where=/mnt/r1124test durable_id=uuid:5de8066f-0887-4dd1-a329-9b595e4c1d9e +Oct 17 16:23:00 demo-hp felhom-agent[3044359]: time=2026-10-17T16:23:00.920+02:00 level=WARN msg="decommission: operator-signed decommission complete (drive will never auto-mount)" durable_id=uuid:5de8066f-0887-4dd1-a329-9b595e4c1d9e +Oct 17 16:23:00 demo-hp felhom-agent[3044359]: time=2026-10-17T16:23:00.920+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=7eb567e3dff73267 op=decommission +Oct 17 16:23:50 demo-hp felhom-agent[3044359]: time=2026-10-17T16:23:50.743+02:00 level=INFO msg="storage: forgot a decommissioned drive that is gone — its mount unit is removed" name=5de8066f-0887-4dd1-a329-9b595e4c1d9e where=/mnt/r1124test uuid=5de8066f-08 +Oct 17 16:40:55 demo-hp felhom-agent[3044359]: time=2026-10-17T16:40:55.234+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_10_17-16_33_32.tar.zst size_bytes=6768484797 uncovered_volumes=2 +Oct 17 16:53:01 demo-hp felhom-agent[3044359]: time=2026-10-17T16:53:01.776+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=b3aa07f6104caae6 op=agent_update +Oct 17 16:54:06 demo-hp felhom-agent[3320002]: time=2026-10-17T16:54:06.099+02:00 level=WARN msg="selfupdate: update committed" version=0.159.1 wrapper="" +Oct 17 17:08:18 demo-hp felhom-os-apply[3376289]: os-apply: BUNDLE DONE agent=0.159.1 written=0 same=30 self-check=ok signers-created=False +Oct 17 17:08:18 demo-hp felhom-agent[3320002]: time=2026-10-17T17:08:18.348+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE DONE agent=0.159.1 written=0 same=30 self-check=ok signers-created=False" +Oct 17 17:08:19 demo-hp felhom-agent[3320002]: time=2026-10-17T17:08:19.386+02:00 level=WARN msg="osupdate: capability probe after the config bundle" ok=75 total=75 degraded="" +Oct 17 17:08:19 demo-hp felhom-agent[3320002]: time=2026-10-17T17:08:19.386+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=b5cc740c36f4bed2 op=agent_config_update +Oct 17 17:08:19 demo-hp felhom-agent[3320002]: time=2026-10-17T17:08:19.402+02:00 level=INFO msg="decommission: drive removed from the guest's bind record" vmid=9201 durable_id=uuid:5de8066f-0887-4dd1-a329-9b595e4c1d9e +Oct 17 17:08:19 demo-hp felhom-agent[3320002]: time=2026-10-17T17:08:19.402+02:00 level=WARN msg="decommission: operator-signed decommission complete (drive will never auto-mount)" durable_id=uuid:5de8066f-0887-4dd1-a329-9b595e4c1d9e +Oct 17 17:08:19 demo-hp felhom-agent[3320002]: time=2026-10-17T17:08:19.402+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=e9303dec85a1f5bd op=decommission diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/r1124-forget-live.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/r1124-forget-live.txt new file mode 100644 index 00000000..f0b96363 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/r1124-forget-live.txt @@ -0,0 +1,19 @@ +== before unplug +-rw-r--r-- 1 root root 292 Oct 17 16:11 /etc/systemd/system/mnt-r1124test.mount +lrwxrwxrwx 1 root root 39 Oct 17 16:11 /etc/systemd/system/multi-user.target.wants/mnt-r1124test.mount -> /etc/systemd/system/mnt-r1124test.mount +GET /disks 200 rows 5 r1124test listed: True +16:23:42 +== unplug: unmount the bind and the drive, detach the loop device +ls: cannot access '/dev/disk/by-uuid/5de8066f-0887-4dd1-a329-9b595e4c1d9e': No such file or directory +== 45 s later +ls: cannot access '/etc/systemd/system/mnt-r1124test.mount': No such file or directory +ls: cannot access '/etc/systemd/system/multi-user.target.wants/mnt-r1124test.mount': No such file or directory +GET /disks 200 rows 4 r1124test listed: False +Oct 17 16:23:42 demo-hp systemd[1]: mnt-felhom\x2ddrives-r1124test.mount: Deactivated successfully. +Oct 17 16:23:42 demo-hp systemd[1]: Unmounting mnt-r1124test.mount - Felhom storage mount 5de8066f-0887-4dd1-a329-9b595e4c1d9e... +Oct 17 16:23:42 demo-hp systemd[1]: mnt-r1124test.mount: Deactivated successfully. +Oct 17 16:23:42 demo-hp systemd[1]: Unmounted mnt-r1124test.mount - Felhom storage mount 5de8066f-0887-4dd1-a329-9b595e4c1d9e. +Oct 17 16:23:50 demo-hp sudo[3208627]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/systemctl disable -- mnt-r1124test.mount +Oct 17 16:23:50 demo-hp sudo[3208690]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/local/sbin/felhom-priv-apply unit-forget mnt-r1124test.mount +Oct 17 16:23:50 demo-hp felhom-priv-apply[3208698]: felhom-priv-apply: REMOVED unit-forget /etc/systemd/system/mnt-r1124test.mount (where=/mnt/r1124test) +Oct 17 16:23:50 demo-hp felhom-agent[3044359]: time=2026-10-17T16:23:50.743+02:00 level=INFO msg="storage: forgot a decommissioned drive that is gone — its mount unit is removed" name=5de8066f-0887-4dd1-a329-9b595e4c1d9e where=/mnt/r1124test uuid=5de8066f-0887-4dd1-a329-9b595e4c1d9e unit=mnt-r1124test.mount diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/redproof-bindprune.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/redproof-bindprune.txt new file mode 100644 index 00000000..09190c7f --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/redproof-bindprune.txt @@ -0,0 +1,8 @@ +== red-proof: the prune loop removed + decommission_test.go:120: 9201 record = [uuid:keep uuid:gone], want [uuid:keep] +--- FAIL: TestDecommissionExecutor_PrunesTheGuestBindRecord (0.00s) +FAIL +FAIL gitea.dooplex.hu/admin/felhom-agent/internal/signedjobs 0.009s +FAIL +== restored +ok gitea.dooplex.hu/admin/felhom-agent/internal/signedjobs 0.016s diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/redproof-forget.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/redproof-forget.txt new file mode 100644 index 00000000..b864596c --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/redproof-forget.txt @@ -0,0 +1,24 @@ +== mutation A: ForgetRetiredDrives body = return 0 +--- FAIL: TestForgetRetiredDrives_DecommissionedAndGone_UnitRemovedAndNotListed (0.00s) +--- FAIL: TestForgetRetiredDrives_AlreadyDisabled_NoDisableCall (0.00s) +--- PASS: TestForgetRetiredDrives_KeepsEveryOtherUnit (0.00s) +--- PASS: TestForgetRetiredDrives_NilIntent_ForgetsNothing (0.00s) +FAIL +FAIL gitea.dooplex.hu/admin/felhom-agent/internal/storage 0.011s +FAIL +== mutation B: the present-drive check removed +--- PASS: TestForgetRetiredDrives_DecommissionedAndGone_UnitRemovedAndNotListed (0.00s) +--- PASS: TestForgetRetiredDrives_AlreadyDisabled_NoDisableCall (0.00s) +--- FAIL: TestForgetRetiredDrives_KeepsEveryOtherUnit (0.00s) + --- PASS: TestForgetRetiredDrives_KeepsEveryOtherUnit/decommissioned_but_mounted (0.00s) + --- PASS: TestForgetRetiredDrives_KeepsEveryOtherUnit/enrolled_and_absent (0.00s) + --- PASS: TestForgetRetiredDrives_KeepsEveryOtherUnit/ejected_and_absent (0.00s) + --- PASS: TestForgetRetiredDrives_KeepsEveryOtherUnit/no_intent (0.00s) + --- PASS: TestForgetRetiredDrives_KeepsEveryOtherUnit/mount_table_unreadable (0.00s) + --- FAIL: TestForgetRetiredDrives_KeepsEveryOtherUnit/decommissioned_but_present (0.00s) +--- PASS: TestForgetRetiredDrives_NilIntent_ForgetsNothing (0.00s) +FAIL +FAIL gitea.dooplex.hu/admin/felhom-agent/internal/storage 0.010s +FAIL +== restored +ok gitea.dooplex.hu/admin/felhom-agent/internal/storage 0.009s diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/redproof-thincheck.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/redproof-thincheck.txt new file mode 100644 index 00000000..0ada6739 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/redproof-thincheck.txt @@ -0,0 +1,25 @@ +== M1: Orphans() returns nil +--- FAIL: TestParse_TheOrphanOf20261014 (0.00s) +--- FAIL: TestCheck_OrphanOnBothReads_IsNamed (0.00s) +FAIL +FAIL gitea.dooplex.hu/admin/felhom-agent/internal/thincheck 0.006s +FAIL +== M2: explainFailure appends nothing +--- FAIL: TestBackup_SnapshotFailure_NamesTheOrphan (0.00s) +FAIL +FAIL gitea.dooplex.hu/admin/felhom-agent/internal/backup 0.009s +FAIL +== M3: helper ignores a held snapshot +FAIL: test_a_snapshot_someone_else_holds_is_never_touched (__main__.Refuses.test_a_snapshot_someone_else_holds_is_never_touched) +FAILED (failures=1) +== M4: helper without the release trap +FAIL: test_thin_ls_failure_still_releases (__main__.Refuses.test_thin_ls_failure_still_releases) +FAILED (failures=1) +== M5: helper sends a delete +FAIL: test_the_only_pool_messages_are_reserve_and_release (__main__.NeverDeletes.test_the_only_pool_messages_are_reserve_and_release) +FAIL: test_healthy_pool_lists_both_sides_and_releases (__main__.Reads.test_healthy_pool_lists_both_sides_and_releases) +FAILED (failures=2) +== restored +ok gitea.dooplex.hu/admin/felhom-agent/internal/thincheck (cached) +ok gitea.dooplex.hu/admin/felhom-agent/internal/backup (cached) +OK diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/redproof-unit-forget.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/redproof-unit-forget.txt new file mode 100644 index 00000000..4682a582 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/redproof-unit-forget.txt @@ -0,0 +1,23 @@ +== red-proof: the tests against the pre-R-1124 wrapper (HEAD) +FAIL: test_F1_a_symlinked_unit_is_not_followed (__main__.UnitForget.test_F1_a_symlinked_unit_is_not_followed) +FAIL: test_F2_a_network_unit_or_a_foreign_unit_is_not_removed (__main__.UnitForget.test_F2_a_network_unit_or_a_foreign_unit_is_not_removed) +FAIL: test_F3_an_enabled_unit_is_not_removed (__main__.UnitForget.test_F3_an_enabled_unit_is_not_removed) +FAIL: test_F4_a_mounted_drive_keeps_its_unit (__main__.UnitForget.test_F4_a_mounted_drive_keeps_its_unit) +FAIL: test_F5_a_present_drive_keeps_its_unit (__main__.UnitForget.test_F5_a_present_drive_keeps_its_unit) +FAIL: test_U3_a_unit_whose_name_lies_about_where_is_not_removed (__main__.UnitForget.test_U3_a_unit_whose_name_lies_about_where_is_not_removed) +FAIL: test_already_gone_is_fine (__main__.UnitForget.test_already_gone_is_fine) +FAIL: test_an_absent_disabled_unmounted_drive_unit_is_removed (__main__.UnitForget.test_an_absent_disabled_unmounted_drive_unit_is_removed) +Ran 46 tests in 0.010s +FAILED (failures=8) +== mutation: F5 check disabled +FAIL: test_F5_a_present_drive_keeps_its_unit (__main__.UnitForget.test_F5_a_present_drive_keeps_its_unit) +Ran 46 tests in 0.007s +FAILED (failures=1) +== mutation: F4 check disabled +FAIL: test_F4_a_mounted_drive_keeps_its_unit (__main__.UnitForget.test_F4_a_mounted_drive_keeps_its_unit) +Ran 46 tests in 0.007s +FAILED (failures=1) +== mutation: F3 check disabled +FAIL: test_F3_an_enabled_unit_is_not_removed (__main__.UnitForget.test_F3_an_enabled_unit_is_not_removed) +Ran 46 tests in 0.008s +FAILED (failures=1) diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/scratch-pool-helper.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/scratch-pool-helper.txt new file mode 100644 index 00000000..a61fc850 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P1/scratch-pool-helper.txt @@ -0,0 +1,45 @@ +loop=/dev/loop2 + Physical volume "/dev/loop2" successfully created. + Volume group "felhomscratch" successfully created + Thin pool volume with chunk size 64.00 KiB can address at most <15.88 TiB of data. + Logical volume "pool" created. + Logical volume "t1" created. + LV ThId Pool + pool + t1 1 pool +== healthy read +lvm-before 1 t1 +pool 1 +lvm-after 1 t1 +ok +rc=0 +== make an orphan (kernel only, scratch pool): create_thin 7 +lvm-before 1 t1 +pool 1 +pool 7 +lvm-after 1 t1 +ok +rc=0 +0 819200 thin-pool 1 113/1024 0/6400 - rw discard_passdown queue_if_no_space - 256 +== someone else holds a metadata snapshot +felhom-thin-check: REFUSED: a metadata snapshot of felhomscratch-pool-tpool is already held (by someone else) — not touched 3 +rc=3 +held root: 30 +== demo-hp real pool pve/data (read only) +lvm-before 1 vm-9201-disk-0 +lvm-before 2 vm-9201-disk-1 +pool 1 +pool 2 +lvm-after 1 vm-9201-disk-0 +lvm-after 2 vm-9201-disk-1 +ok +rc=0 +held root after: - +== teardown scratch + Logical volume "t1" successfully removed. + Logical volume "pool" successfully removed. + Volume group "felhomscratch" successfully removed + Labels on physical volume "/dev/loop2" successfully wiped. + pve +0 +0 diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/9201-refused-correction.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/9201-refused-correction.txt new file mode 100644 index 00000000..efdb596f --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/9201-refused-correction.txt @@ -0,0 +1,17 @@ +== before +79e7877dc3e8577d7623f9b5ba57d300eca7a2a0f86c27458385b97517f2c480 /opt/docker/stacks/privatebin/docker-compose.yml +494270e42262ac772dd2033a0dc2c2530b52bf8132a5245c9bb3e2a9f9df9e3a 2026-10-17T14:34:24.634717373Z running +== after the refused correction +79e7877dc3e8577d7623f9b5ba57d300eca7a2a0f86c27458385b97517f2c480 /opt/docker/stacks/privatebin/docker-compose.yml +0 +494270e42262ac772dd2033a0dc2c2530b52bf8132a5245c9bb3e2a9f9df9e3a 2026-10-17T14:34:24.634717373Z running +2026/10/17 15:10:34 allowlist.go:122: [WARN] [stacks] privatebin: definition REFUSED by the allow-list (R-971), nothing written: the catalog definition of privatebin asks for container rights the box does not allow: priv +2026/10/17 15:10:34 sync.go:511: [WARN] [sync] privatebin: catalog definition REFUSED by the allow-list — the app keeps its last good compose, nothing written: the catalog definition of privatebin asks for container ri +2026/10/17 15:10:34 notifier.go:315: [DEBUG] PushEvent: type=app_definition_refused severity=warning url=https://hub.felhom.eu/api/v1/event +2026/10/17 15:10:34 notifier.go:341: [DEBUG] PushEvent: app_definition_refused pushed OK (HTTP 200) +2026/10/17 15:10:34 notifier.go:353: [INFO] Event pushed: app_definition_refused (warning) [hu-only] — privatebin: the box REFUSED a catalog definition — it asks for container rights not on the box's allow-list. Noth +== hub +2026/10/17 17:10:34 [INFO] Event from demo-hp: app_definition_refused (warning) — privatebin: the box REFUSED a catalog definition — it asks for container rights not on the box's allow-list. Nothing was written; the app keeps its last good compose. Check w +2026/10/17 17:10:34 [INFO] Operator email sent for demo-hp/app_definition_refused +/apps/privatebin 200 True True +/apps/opengist 200 False False diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/9201-restore-live.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/9201-restore-live.txt new file mode 100644 index 00000000..3c05c87c --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/9201-restore-live.txt @@ -0,0 +1,14 @@ +2026/10/17 15:11:32 sync.go:318: [INFO] [sync] Cloning https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git → /opt/docker/felhom-controller/data/catalog-cache +2026/10/17 15:11:33 sync.go:261: [INFO] [sync] catalog moved c3d0fb37df7f -> 45763e31c2fc (0 new, 0 stack dir(s) changed) +2026/10/17 15:11:33 sync.go:299: [INFO] [sync] Catalog sync complete +2026/10/17 15:11:33 sync.go:142: [INFO] [sync] Initial sync: Sablonok frissítve — a katalógus új változata betöltve; a telepített alkalmazások fájljai nem változtak + +23: repo_url: https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git +24: sync_interval: 15m +-rw------- 1 root root 2989 Aug 31 08:32 /var/lib/docker/volumes/felhom-controller-data/_data/controller.yaml.pre-r971 +mirror main c3d0fb3 DRILL privatebin: privileged: true — the box must REFUSE this correction (R-971 proof on 9201) +cache HEAD 45763e3 CHANGELOG: the privileges gate's round-2/3 changes +https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git + +/apps/privatebin 200 False False +/apps/opengist 200 False False diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-allowlist.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-allowlist.txt new file mode 100644 index 00000000..8cea4616 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-allowlist.txt @@ -0,0 +1,38 @@ +== M1: Admit admits everything +--- FAIL: TestAdmit_EveryClassIsRefused (0.00s) + --- FAIL: TestAdmit_EveryClassIsRefused/network_mode_host (0.00s) + --- FAIL: TestAdmit_EveryClassIsRefused/pid (0.00s) + --- FAIL: TestAdmit_EveryClassIsRefused/uts (0.00s) + --- FAIL: TestAdmit_EveryClassIsRefused/volumes_from (0.00s) + --- FAIL: TestAdmit_EveryClassIsRefused/extends_file (0.00s) + --- FAIL: TestAdmit_EveryClassIsRefused/devices (0.00s) + --- FAIL: TestAdmit_EveryClassIsRefused/seccomp_off (0.00s) + --- FAIL: TestAdmit_EveryClassIsRefused/apparmor_off (0.00s) + --- FAIL: TestAdmit_EveryClassIsRefused/sysctls (0.00s) + --- FAIL: TestAdmit_EveryClassIsRefused/env_file_outside (0.00s) + --- FAIL: TestAdmit_EveryClassIsRefused/interpolated (0.00s) +== M2: copyTemplates asks nothing +--- FAIL: TestR971_RefusedCorrectionWritesNothing (0.00s) +--- FAIL: TestR971_RefusedTemplateOfAnAppNotInstalled (0.00s) +FAIL gitea.dooplex.hu/admin/felhom-controller/internal/sync 0.010s +--- FAIL: TestAllowList_IsWiredIntoEveryComposeWriter (0.20s) +FAIL gitea.dooplex.hu/admin/felhom-controller/internal/stacks 0.208s +== M3: a NEW compose writer nobody gated + allowlist_wiring_test.go:124: internal/stacks/zz_redproof_writer.go:redproofNewWriter writes a file and names the compose file, and the allow-list guard does not know it — gate it (admitCompose before the write) or add it to `writers` as exempt with the reason +--- FAIL: TestAllowList_IsWiredIntoEveryComposeWriter (0.23s) +FAIL +FAIL gitea.dooplex.hu/admin/felhom-controller/internal/stacks 0.237s +== M4: no once-per-definition +--- FAIL: TestAllowList_OneEventPerRefusedDefinition (0.00s) +FAIL gitea.dooplex.hu/admin/felhom-controller/internal/stacks 0.007s +== M5: the app page line removed +--- FAIL: TestR971_AppPage_SaysTheDefinitionWasRefused (0.28s) +FAIL gitea.dooplex.hu/admin/felhom-controller/internal/web 0.296s +== M6: the install asks nothing +--- FAIL: TestAllowList_IsWiredIntoEveryComposeWriter (0.24s) +FAIL gitea.dooplex.hu/admin/felhom-controller/internal/stacks 0.251s +== restored +ok gitea.dooplex.hu/admin/felhom-controller/internal/privpolicy (cached) +ok gitea.dooplex.hu/admin/felhom-controller/internal/sync (cached) +ok gitea.dooplex.hu/admin/felhom-controller/internal/stacks 423.793s +13 diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-catalog-gate.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-catalog-gate.txt new file mode 100644 index 00000000..d8d81349 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-catalog-gate.txt @@ -0,0 +1,14 @@ +== catalog gate red-proof: the allow-list check never refuses + XX FACT: privileged true rc=0 (expected 1) + XX FACT: cap_add rc=0 (expected 1) + XX FACT: the host root bound rc=0 (expected 1) + XX FACT: a bind through another variable rc=0 (expected 1) + XX FACT: homepage's socket READ-WRITE rc=0 (expected 1) + XX FACT: the allowed socket in ANOTHER app rc=0 (expected 1) + XX FACT: network_mode host rc=0 (expected 1) + XX FACT: a driver_opts bind volume rc=0 (expected 1) + XX FACT: a ${VAR} in a privileged key rc=0 (expected 1) + XX FACT: privileged in a steps/ file rc=0 (expected 1) +fails: 10 +== restored +check-privileges: 123 files judged, every class-setting on the allow-list diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-live-root-resolve.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-live-root-resolve.txt new file mode 100644 index 00000000..d101b35f --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-live-root-resolve.txt @@ -0,0 +1,8 @@ +== live finding (9201, rc.4) red-proof: roots resolved only when they exist + privpolicy_test.go:438: binds under roots that do not exist yet were refused: the catalog definition of app asks for container rights the box does not allow: app: bind /mnt/felhom-drives/check/felhom-data/appdata/app -> /c rw; app: bind /mnt/felhom-drives/check/felhom-data/userdata/media -> /m rw +--- FAIL: TestAdmit_RootsThatDoNotExistYet (0.00s) +FAIL +FAIL gitea.dooplex.hu/admin/felhom-controller/internal/privpolicy 0.006s +FAIL +== restored +ok gitea.dooplex.hu/admin/felhom-controller/internal/privpolicy 0.147s diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-security-review-fixes.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-security-review-fixes.txt new file mode 100644 index 00000000..37332f69 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-security-review-fixes.txt @@ -0,0 +1,22 @@ +== security-review fixes, red-proofs +-- F1: the image/start binding removed +--- FAIL: TestAdmit_TheAllowedSettingIsBoundToItsImageAndStart (0.00s) + --- FAIL: TestAdmit_TheAllowedSettingIsBoundToItsImageAndStart/another_image (0.00s) + --- FAIL: TestAdmit_TheAllowedSettingIsBoundToItsImageAndStart/added_entrypoint (0.00s) + --- FAIL: TestAdmit_TheAllowedSettingIsBoundToItsImageAndStart/added_command (0.00s) +-- F2: Resolve ignored +--- FAIL: TestAdmit_ASymlinkOutOfARootIsOutside (0.00s) +-- F3: the later classes removed +--- FAIL: TestAdmit_TheLaterClasses (0.00s) + --- FAIL: TestAdmit_TheLaterClasses/provider (0.00s) + --- PASS: TestAdmit_TheLaterClasses/post_start (0.00s) + --- PASS: TestAdmit_TheLaterClasses/cgroup_parent (0.00s) + --- PASS: TestAdmit_TheLaterClasses/runtime (0.00s) + --- FAIL: TestAdmit_TheLaterClasses/gpus (0.00s) + --- PASS: TestAdmit_TheLaterClasses/device_request (0.00s) +-- F4: any /mnt/ path a root again +--- FAIL: TestAdmit_ARootElsewhereUnderMntIsNoRoot (0.00s) +== restored +ok gitea.dooplex.hu/admin/felhom-controller/internal/privpolicy 0.137s +ok gitea.dooplex.hu/admin/felhom-controller/internal/sync 0.388s +ok gitea.dooplex.hu/admin/felhom-controller/internal/stacks 0.236s diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-security-review-round2.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-security-review-round2.txt new file mode 100644 index 00000000..8c98eefb --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-security-review-round2.txt @@ -0,0 +1,17 @@ +== security review round 2, red-proofs +-- same-file extends not refused +--- FAIL: TestAdmit_Round2 (0.00s) + --- FAIL: TestAdmit_Round2/a_second_service_extends_the_allowed_one (0.00s) +-- healthcheck not bound +--- FAIL: TestAdmit_Round2 (0.00s) + --- FAIL: TestAdmit_Round2/the_allowed_service's_healthcheck_runs_a_shell (0.00s) +-- image variable not refused +--- FAIL: TestAdmit_Round2 (0.00s) + --- FAIL: TestAdmit_Round2/the_allowed_service's_image_through_a_variable (0.00s) +-- options variable allowed +--- FAIL: TestAdmit_Round2 (0.00s) + --- FAIL: TestAdmit_Round2/a_variable_in_a_bind's_options (0.00s) +-- a missing path judged by name again (fail open) +--- FAIL: TestAdmit_ANotYetCreatedPathBehindASymlink (0.00s) +== restored +ok gitea.dooplex.hu/admin/felhom-controller/internal/privpolicy 0.123s diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-security-review-round3.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-security-review-round3.txt new file mode 100644 index 00000000..984548c3 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/redproof-security-review-round3.txt @@ -0,0 +1,8 @@ +== round 3 red-proof: HDD_PATH whole again + privpolicy_test.go:416: " - ${HDD_PATH}:/drive\n" was admitted +--- FAIL: TestAdmit_HDDPathRootIsAppdataOnly (0.00s) +FAIL +FAIL gitea.dooplex.hu/admin/felhom-controller/internal/privpolicy 0.005s +FAIL +== restored +ok gitea.dooplex.hu/admin/felhom-controller/internal/privpolicy (cached) diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/survey-raw.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/survey-raw.txt new file mode 100644 index 00000000..a85a2a24 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/survey-raw.txt @@ -0,0 +1,41 @@ +FILES 123 +SET audiobookshelf final audiobookshelf security_opt ["no-new-privileges:true"] +SET audiobookshelf step audiobookshelf security_opt ["no-new-privileges:true"] +SET crafty-controller final crafty-controller ports ["25565-25575:25565-25575"] +SET komga final komga security_opt ["no-new-privileges:true"] +SET komga step komga security_opt ["no-new-privileges:true"] +BIND audiobookshelf audiobookshelf ${USERDATA_PATH}/media/audiobooks:/audiobooks 2 +BIND audiobookshelf audiobookshelf ${USERDATA_PATH}/media/podcasts:/podcasts 2 +BIND calibre-web calibre-web ${IMPORT_PATH}/calibre:/cwa-book-ingest 1 +BIND calibre-web calibre-web ${USERDATA_PATH}/media/books:/calibre-library 1 +BIND emby emby ${USERDATA_PATH}/media:/media:ro 3 +BIND grimmory grimmory ${HDD_PATH}/appdata/grimmory/data:/app/data 1 +BIND grimmory grimmory ${IMPORT_PATH}/grimmory:/bookdrop 1 +BIND grimmory grimmory ${USERDATA_PATH}/media/grimmory:/books 1 +BIND homepage homepage /var/run/docker.sock:/var/run/docker.sock:ro 1 +BIND immich immich-server ${HDD_PATH}/appdata/immich:/usr/src/app/upload 3 +BIND immich immich-server ${USERDATA_PATH}/media/photos:/external/photos:ro 3 +BIND jellyfin jellyfin ${USERDATA_PATH}/media:/media:ro 1 +BIND komga komga ${USERDATA_PATH}/media/comics:/data 2 +BIND metube metube ${USERDATA_PATH}/media/metube:/downloads 1 +BIND navidrome navidrome ${USERDATA_PATH}/media/music:/music:ro 3 +BIND nextcloud nextcloud ${HDD_PATH}/appdata/nextcloud:/var/www/html/data 3 +BIND nzbget nzbget ${USERDATA_PATH}/media/nzbget:/downloads 1 +BIND paperless-ngx paperless-webserver ${HDD_PATH}/appdata/paperless/export:/usr/src/paperless/export 1 +BIND paperless-ngx paperless-webserver ${HDD_PATH}/appdata/paperless/media:/usr/src/paperless/media 1 +BIND paperless-ngx paperless-webserver ${IMPORT_PATH}/paperless:/usr/src/paperless/consume 1 +BIND plex plex ${USERDATA_PATH}/media:/media:ro 1 +BIND qbittorrent qbittorrent ${USERDATA_PATH}/media/torrents:${USERDATA_PATH}/media/torrents 1 +BIND qbittorrent qbittorrent ${USERDATA_PATH}/media/torrents:/downloads 1 +BIND radarr radarr ${USERDATA_PATH}/media:${USERDATA_PATH}/media 1 +BIND romm romm ${HDD_PATH}/appdata/romm/resources:/romm/resources 3 +BIND romm romm ${USERDATA_PATH}/roms:/romm/library 2 +BIND romm romm ${USERDATA_PATH}/roms:/romm/library/roms 1 +BIND romm romm ${USERDATA_PATH}/roms-bios:/romm/library/bios 1 +BIND sabnzbd sabnzbd ${USERDATA_PATH}/media/usenet:/downloads 1 +BIND sonarr sonarr ${USERDATA_PATH}/downloads:/downloads 1 +BIND sonarr sonarr ${USERDATA_PATH}/media:${USERDATA_PATH}/media 1 +BIND sonarr sonarr ${USERDATA_PATH}/media/tv:/media/tv 1 +BIND transmission transmission ${USERDATA_PATH}/media/transmission:${USERDATA_PATH}/media/transmission 1 +BIND transmission transmission ${USERDATA_PATH}/media/transmission:/downloads 1 +BIND transmission transmission ${USERDATA_PATH}/media/transmission/watch:/watch 1 diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/survey.md b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/survey.md new file mode 100644 index 00000000..92ed9c6a --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P2/survey.md @@ -0,0 +1,36 @@ +# R-971 survey — every container right the catalog asks for (2026-10-17) + +Catalog `app-catalog-felhom.eu` at `1af6ea7` (the survey) and `d5a443d` (the gate). **123 files: 65 `templates//docker-compose.yml` +and 58 ladder `steps/*.yml`.** Read with PyYAML (`survey-raw.txt`, the raw scan) and again with the controller's own +check (`privpolicy.Check` on the placeholder env, the test `TestAllowList_TheWholeCatalogPasses`). + +Classes looked for: `privileged`, `cap_add`, `devices`, `device_cgroup_rules`, `network_mode`, `pid`, `ipc`, `uts`, +`userns_mode`, `cgroup`, `security_opt`, `sysctls`, `volumes_from`, `build`, `include`, `extends`, `env_file`, top-level +volumes with `driver`/`driver_opts`/`external`/`name`, `configs`/`secrets` with `file:`, and every bind source. + +## The allow-list (what the catalog needs) + +| app | service | setting | why the app needs it | upstream doc | +|---|---|---|---|---| +| homepage | homepage | `bind /var/run/docker.sock -> /var/run/docker.sock ro` | the Docker widget and Docker service discovery list the household's containers and their state | https://gethomepage.dev/configs/docker/ | + +That is the whole list. **No template or step asks for any other class member.** Note (security review C5-5, R-782): a +read-only socket mount does not limit the Docker API; the entry keeps today's behaviour, it does not bless it — a +read-only socket proxy for homepage stays a separate catalog change. + +## Seen and NOT a class member (no entry needed) + +| app | setting | why it passes | +|---|---|---| +| audiobookshelf, komga (final + step) | `security_opt: [no-new-privileges:true]` | a hardening, not a relaxation | +| every bind (35 distinct) | `${HDD_PATH}/…`, `${USERDATA_PATH}/…`, `${IMPORT_PATH}/…` | inside the data roots the box gives | +| crafty-controller | `ports: 25565-25575` | not in the brief's classes: a published port is LAN reach of the guest, not a right on the host (documented in `01-topology-and-trust.md`) | +| vikunja | `user: 0:0` | root INSIDE the container; not a host right | +| karakeep-chrome | `init: true` | a tini init, no privilege | +| 94 compose files | `networks: traefik-public: external: true` | a network, not a volume; not a class | + +## The controller's infra stacks (out of scope, stated) + +traefik (holds the Docker socket at the internet edge), cloudflared, filebrowser and samba (`network_mode: host` for LAN +discovery) are rendered by the controller from its own code, not the catalog. Since v0.328.0 the sync also refuses to +copy a catalog template named like one of them into its directory. diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P3/demo-felhom-rotation.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P3/demo-felhom-rotation.txt new file mode 100644 index 00000000..df276318 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P3/demo-felhom-rotation.txt @@ -0,0 +1,15 @@ +== before +demo-felhom-8363b5 HTTP 200 user root@pam set_at 2026-07-18T16:30:41Z password_len 32 +192.168.0.162 PVE ticket HTTP 200 ticket_issued True +== rotate (installer step 4b alone, on demo-felhom) +[STEP] 4b/8 break-glass credential (root@pam console password → hub vault) +[OK] root@pam password set + vaulted to the hub (retrieve via the operator /admin path; never logged here) +[WARN] NOTE: the root@pam password just CHANGED — the old one now fails at the PVE web GUI (:8006). +[WARN] Retrieve the new one at hub → host page (vaulted recovery credential). +== after (hub 0.157.0: set_at should move now) +demo-felhom-8363b5 HTTP 200 user root@pam set_at 2026-10-17T15:10:04Z password_len 32 +== the NEW vaulted value logs in +192.168.0.162 PVE ticket HTTP 200 ticket_issued True +== the OLD value no longer does +192.168.0.162 PVE ticket HTTP 401 +values differ: yes diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P3/demo-hp-rotation.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P3/demo-hp-rotation.txt new file mode 100644 index 00000000..f14ce4f2 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P3/demo-hp-rotation.txt @@ -0,0 +1,14 @@ +== before +demo-hp-bb76ea HTTP 200 user root@pam set_at 2026-07-21T16:24:27Z password_len 32 +== rotate (installer step 4b alone, on demo-hp) +[STEP] 4b/8 break-glass credential (root@pam console password → hub vault) +[OK] root@pam password set + vaulted to the hub (retrieve via the operator /admin path; never logged here) +[WARN] NOTE: the root@pam password just CHANGED — the old one now fails at the PVE web GUI (:8006). +[WARN] Retrieve the new one at hub → host page (vaulted recovery credential). +== after +demo-hp-bb76ea HTTP 200 user root@pam set_at 2026-07-21T16:24:27Z password_len 32 +== the NEW vaulted value logs in +192.168.0.104 PVE ticket HTTP 200 ticket_issued True +== the OLD value no longer does +192.168.0.104 PVE ticket HTTP 401 +values differ: yes diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P3/redproof-set-at.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P3/redproof-set-at.txt new file mode 100644 index 00000000..1a34f11a --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P3/redproof-set-at.txt @@ -0,0 +1,8 @@ +== red-proof: the upsert without set_at + r954_revault_set_at_test.go:31: credential set_at = 2025-10-17 10:00:00 +0000 UTC — the re-vault kept the first vault's time +--- FAIL: TestHostRecovery_ReVaultMovesSetAt (0.04s) +FAIL +FAIL gitea.dooplex.hu/admin/felhom-hub/internal/store 0.045s +FAIL +== restored +ok gitea.dooplex.hu/admin/felhom-hub/internal/store 0.044s diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P3/tester-1-rotation.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P3/tester-1-rotation.txt new file mode 100644 index 00000000..04bb5685 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P3/tester-1-rotation.txt @@ -0,0 +1,14 @@ +== before +tester-1-d70be4 HTTP 200 user root@pam set_at 2026-10-04T19:40:27Z password_len 32 +== rotate (installer step 4b alone, on Tester 1) +[STEP] 4b/8 break-glass credential (root@pam console password → hub vault) +[OK] root@pam password set + vaulted to the hub (retrieve via the operator /admin path; never logged here) +[WARN] NOTE: the root@pam password just CHANGED — the old one now fails at the PVE web GUI (:8006). +[WARN] Retrieve the new one at hub → host page (vaulted recovery credential). +== after +tester-1-d70be4 HTTP 200 user root@pam set_at 2026-10-04T19:40:27Z password_len 32 +== the NEW vaulted value logs in +192.168.0.154 PVE ticket HTTP 200 ticket_issued True +== the OLD value no longer does +192.168.0.154 PVE ticket HTTP 401 +values differ: yes diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P4/design-R-895.md b/documentation/audits/agent-allowlist-rotation-2026-10-17/P4/design-R-895.md new file mode 100644 index 00000000..3be6e175 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P4/design-R-895.md @@ -0,0 +1,65 @@ +# R-895 — the hub counts a box's off-site snapshots itself (option B, updated with the measurement) — 2026-10-17 + +Baselines: felhom.eu `8f97b800` (hub v0.157.0 code), design origin `audits/night-burndown-2026-10-06/design-R-822.md` +option B. Architecture: `07-backup-architecture.md` (the off-site tier; decision 68 — the box-trusted count; decision +69 — the hub writes the pinned key line), `06-offsite-connectivity.md`. + +## 1. The question + +The clean-up window's drop alarm (`EventWindowDrop`) compares counts the BOX reports before and after a window. A box +that was broken into, or fakes planted with its add-only key, keep the count level while the real history shrinks. +Option B: the hub counts the files in the box's restic `snapshots/` directory itself, over a login of its own. + +## 2. The measurement (read-only, demo-hp's repository folder only) + +**What login would the hub need?** None new. The hub already logs in to every provisioned sub-account once a day with +the sub-account PASSWORD (sealed at rest, R-821) for the `authorized_keys` audit (`offsitekeys.Registrar.Audit`, +port 23, host key pinned). That login can list `snapshots/`. **It is NOT read-only** — the same password can `rm`, +`mv` and rewrite `authorized_keys` (measured 2026-10-03, `audits/offsite-append-only-2026-10-03/live/`). The listing +itself only reads (`ls -l`), and adds no capability the hub does not already hold. + +**Does the provider show upload times?** Yes, by prior measurement on this very sub-account: on 2026-09-01 the +password login on demo-hp's sub-account (`u629488-sub3`) listed entries with their modification time to the minute +over SFTP (`ls -la` → `Sep 1 12:12`), and the port-23 shell offers `ls|ll`, `stat` +(`audits/evidence-drill-r95-recovery-2026-09-01/phase1-find-the-snapshot/01-probe-tree.txt`, `03-shell-capabilities.txt`). +A restic snapshot file is written once and never changed, so its mtime is its upload time. + +**Today's re-measurement from DooPlex: NOT done — and why, with what was tried.** Since decision 69 no box holds the +sub-account password; the hub alone holds it, sealed. Tried: (a) CC's secret store (`sec list`) — no Storage Box +credential; (b) the box's own key — pinned to `rclone serve restic --append-only`, it reaches only restic's REST API +(names and sizes, no times, and it is the box's key, which is exactly what the hub must not rely on); (c) the hub — it +has no command or endpoint that lists. Reading the sealed password out of the hub DB to measure by hand was not done: +it would put a delete-capable credential of a customer sub-account on DooPlex for a measurement the hub can make +itself. **So the measurement ships as code:** hub v0.157.0's daily audit lists `snapshots/` read-only and LOGS the +count, the newest and the oldest upload time (`internal/offsitekeys/snapshots.go`). The first daily run after the +deploy IS the measurement; compare its count with the box's own (`offsite.snapshot_count` in the host report) — a +control from a different channel. + +## 3. The design (the alarm slice, after the measurement) + +1. **Before/after a window, the hub's own count** (`OpenWindowFor` / `CloseWindowFor` take the listing, the box's + numbers logged beside). `EventWindowDrop` fires on the hub's numbers. Red test first (design-R-822 §5): a fake + listing 17 → 3 while the box reports 17 → 15 must fire; today it does not. +2. **Files no box run explains:** each box run reports its time (manual runs, catch-ups and crash retries included — + they are runs). A snapshot file whose upload time falls outside every reported run (± a slack), or more files than + runs, is the injection signature → an operator-only event; the next window is refused while such files exist. +3. **Snapshots that vanished outside a window:** today's list minus yesterday's, when no window was open → an + operator-only event. The hub keeps only names and times (custody unchanged, `07` §8a: never contents). + +## 4. Costs + +| | cost | +|---|---| +| credential | none new; the existing sealed password (write-capable — a fact today, not a new exposure) | +| per box | one more SSH login per day (the listing), and the stored list (≈ 70 names × 100 bytes) | +| build | slice 1 (this release, ~1 h: list + log); slice 2 (§3.1–3.3, ~1 day with the red tests); a live proof on scratch 9202's sub-account | +| what can go wrong | `ls -l` output differing on the provider (slice 1 logs it before anything alarms); a run that does not report its time (false alarm — every run path must report) | + +**A read-only login would be better, and costs a per-box sub-account:** a second sub-account with `readonly=true` +on the same directory (Hetzner API) would let the hub list without holding delete rights for this purpose. It adds +one provisioning step and one sealed secret per box. Not built; named as the alternative. + +## 5. State + +R-895 stays OPEN (READY) with this design. Slice 1 (the listing, log-only) is in hub v0.157.0 — deployed only on the +operator's yes. Slice 2 (the alarms) needs: the first logged listings on real boxes (the measurement), then about a day. diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P4/redproof-listing.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P4/redproof-listing.txt new file mode 100644 index 00000000..f1a6869d --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P4/redproof-listing.txt @@ -0,0 +1,8 @@ +== red-proof: every line counted as a snapshot + snapshots_test.go:44: files=3 other=0, want 2 and 1 +--- FAIL: TestListSnapshots_CountsTheBoxSnapshotFilesReadOnly (0.00s) +FAIL +FAIL gitea.dooplex.hu/admin/felhom-hub/internal/offsitekeys 0.008s +FAIL +== restored +ok gitea.dooplex.hu/admin/felhom-hub/internal/offsitekeys 0.262s diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P5/box-walk.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P5/box-walk.txt new file mode 100644 index 00000000..843069ca --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P5/box-walk.txt @@ -0,0 +1,21 @@ +=== demo-hp (enkisfelhom.hu) +containers: 24 not-healthy/running: +2026/10/17 15:21:35 updater.go:855: [INFO] [selfupdate] Post-update startup: update successful (0.328.0-rc.5 → 0.328.0) +allow-list refusals since start: 0 +2026/10/17 15:21:35 sync.go:142: [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás +public https://felhom.enkisfelhom.hu/login: 200 +public https://files.enkisfelhom.hu/: 200 +=== demo-felhom (demo-felhom.eu) +containers: 9 not-healthy/running: +2026/10/17 15:21:24 [INFO] [selfupdate] Post-update startup: update successful (0.327.1 → 0.328.0) +allow-list refusals since start: 0 +2026/10/17 15:21:25 [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás +public https://felhom.demo-felhom.eu/login: 200 +public https://files.demo-felhom.eu/: 200 +=== tester-1 (enkicsifelhom.hu) +containers: 10 not-healthy/running: +2026/10/17 15:21:34 [INFO] [selfupdate] Post-update startup: update successful (0.327.1 → 0.328.0) +allow-list refusals since start: 0 +2026/10/17 15:21:35 [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás +public https://felhom.enkicsifelhom.hu/login: 200 +public https://files.enkicsifelhom.hu/: 200 diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P5/floors.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P5/floors.txt new file mode 100644 index 00000000..7c933524 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P5/floors.txt @@ -0,0 +1,12 @@ +== global floor +HTTP/1.1 303 See Other +Location: /configuration?flash=floor_set +clear tester-1: +HTTP/1.1 303 See Other +Location: /customers/tester-1?flash=floor_set +clear demo-hp: +HTTP/1.1 303 See Other +Location: /customers/demo-hp?flash=floor_set +clear demo-felhom: +HTTP/1.1 303 See Other +Location: /customers/demo-felhom?flash=floor_set diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P5/global-floor-readback.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P5/global-floor-readback.txt new file mode 100644 index 00000000..3af6420c --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P5/global-floor-readback.txt @@ -0,0 +1,7 @@ +integrity: ok +('min_controller_version', '0.328.0') +('min_controller_version_declared_min_agent', '0.328.0=0.158.1') +per-box floor columns: ['min_controller_version', 'min_controller_declared_min_agent', 'min_controller_set_at'] +min_controller_version set: [] +min_controller_declared_min_agent set: [] +min_controller_set_at set: [] diff --git a/documentation/audits/agent-allowlist-rotation-2026-10-17/P5/https-only.txt b/documentation/audits/agent-allowlist-rotation-2026-10-17/P5/https-only.txt new file mode 100644 index 00000000..58e209f3 --- /dev/null +++ b/documentation/audits/agent-allowlist-rotation-2026-10-17/P5/https-only.txt @@ -0,0 +1,8 @@ + OK http://files.demo-felhom.eu/r1033-check?q=1 -> 301 https://files.demo-felhom.eu/r1033-check?q=1 + OK http://gist.demo-felhom.eu/r1033-check?q=1 -> 301 https://gist.demo-felhom.eu/r1033-check?q=1 + OK http://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1 -> 301 https://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1 + OK http://prowlarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://prowlarr.demo-felhom.eu/r1033-check?q=1 + OK http://radarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://radarr.demo-felhom.eu/r1033-check?q=1 + OK http://sonarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://sonarr.demo-felhom.eu/r1033-check?q=1 + OK http://torrent.demo-felhom.eu/r1033-check?q=1 -> 301 https://torrent.demo-felhom.eu/r1033-check?q=1 +== every public name redirects http -> https diff --git a/documentation/backlog/CLOSED-ITEMS.md b/documentation/backlog/CLOSED-ITEMS.md index 492b3540..981bca79 100644 --- a/documentation/backlog/CLOSED-ITEMS.md +++ b/documentation/backlog/CLOSED-ITEMS.md @@ -26,6 +26,15 @@ --- +## 2026-10-17 — the catalog allow-list, break-glass rotation (controller v0.328.0, hub v0.157.0, catalog `07d1ef7`) + +| Row | What | Closed | Evidence | +|---|---|---|---| +| **R-971** | **A push to the catalog's `main` reached every installed app within ~15 min, and nothing on the box refused dangerous container settings — whoever could push to the catalog could take every guest.** (P2) | CLOSED 2026-10-17 — operator decision D2 = A. **controller v0.328.0** (global floor 0.328.0, MinAgent 0.158.1, 17:2x local): `internal/privpolicy` checks the RENDERED compose before EVERY compose write (install, settings change, the 15-min correction, update/ladder step, both restores — before the stop) against an allow-list compiled into the controller (one entry: homepage's read-only socket, bound to its image, start and healthcheck). A refusal writes nothing, keeps the app on its last good compose, raises the operator-only `app_definition_refused` (hub v0.157.0) and one app-page line. Structural guard `TestAllowList_IsWiredIntoEveryComposeWriter`. Catalog CI mirror `check-privileges.py` (byte-identical list). Three security-review rounds fixed before release. **Live on 9201:** privatebin with `privileged: true` from a drill catalog → refused, same compose bytes, same container running, event at the hub, operator mail; live catalog restored byte-identical. Every installed app on the three test boxes still passes. Signed catalog releases stay the later step. Follow-ups R-1142, R-1143. | `audits/agent-allowlist-rotation-2026-10-17/P2/`, `P5/` | +| **R-1124** | **A drive decommissioned and then unplugged kept its agent mount unit enabled; every boot waited for it and the agent kept listing it.** (P4) | CLOSED 2026-10-17 — **agent v0.159.1** (running on demo-hp, Tester 1, demo-felhom): `ForgetRetiredDrives` (start + 20-s tick) removes, through the root verb `felhom-priv-apply unit-forget`, the unit of a decommissioned drive that is absent and unmounted; a present one keeps its unit (one-click re-enroll). The signed decommission now prunes the guest-bind record too. Live on demo-hp: forgotten 8 s after the unplug, `GET /disks` 5 → 4. | `audits/agent-allowlist-rotation-2026-10-17/P1/` | +| **R-1012** | **A full host root can leave an orphan thin snapshot that LVM does not know; every later snapshot backup of the guest fails and nothing named the cause.** (P3) | CLOSED 2026-10-17 — **agent v0.159.1**: the read-only root helper `felhom-thin-check` (metadata snapshot reserved and released; refuses a foreign one; no other pool message) + `internal/thincheck` (two reads); the finding is appended in plain words to a failed snapshot backup's error (the operator's `whole_guest_backup_failed` mail) and points at `runbooks/whole-guest-backup-orphan-thin.md`. Never removes. Live: a made orphan on a scratch pool listed; `pve/data` read and released; a 9201 backup ran the check and finished green. | `audits/agent-allowlist-rotation-2026-10-17/P1/` | +| **R-954** | **Nobody could tell whether the old hub key / open read routes were used before 2026-10-12 (they could reveal every break-glass password); ~238 days of logs missing.** (P2) | CLOSED 2026-10-17 — operator decision D1 = A: demo-hp, Tester 1 and demo-felhom (operator-fixed → random on the operator's answer) rotated with the installer's own step 4b run alone (not a full re-run — R-1141); each proven: the new vaulted value logs in to the PVE API (200), the old one is refused (401), values differ. Found and fixed: the hub kept the FIRST vault's `set_at` (hub v0.157.0; demo-felhom's re-vault reads 2026-10-17 15:10). The read path used was the global-key curl, which writes no customer event. Tester 2 → R-1139; log retention → R-1140. | `audits/agent-allowlist-rotation-2026-10-17/P3/` | + ## 2026-10-17 — FileBrowser never on its default password, real memory amounts, CI mail (controller v0.327.1, catalog `1af6ea7`) | Row | What | Closed | Evidence | diff --git a/documentation/backlog/OPEN-ITEMS.md b/documentation/backlog/OPEN-ITEMS.md index e3554f68..376a93f6 100644 --- a/documentation/backlog/OPEN-ITEMS.md +++ b/documentation/backlog/OPEN-ITEMS.md @@ -124,14 +124,14 @@ stopping line that lies. | **R-494** | Install & onboarding | P4 | **NARROWED 2026-09-14 by operator ruling → [P3-LOW] the hub COULD create the tunnel at customer creation, for a domain already on Cloudflare. Not blocking: every customer has their own domain and the operator creates the tunnel per day-0 A.1 (`architecture/01-topology-and-trust.md`).** *Original finding, kept:* **[P1-HIGH] A new customer's dashboard has NO reachable address unless the operator hand-makes a Cloudflare tunnel — the link in the setup-code mail is dead.** MEASURED 2026-09-14 on a fresh install from the public ISO (drill intervention **I1**): the claim mail points at `https://felhom.drill0242.felhom.eu`; that name has **no A and no AAAA** record (`dig @1.1.1.1`, control `felhom.enkisfelhom.hu` resolves); the hub has **no tunnel- or DNS-creation code** (`hub/internal/cloudflare/` holds only geo-rule removal; `cf_tunnel_token` is a pasted, optional form field, `configs.go:1478`) — day-0 runbook A.1 makes it a manual Cloudflare-dashboard step that nothing on the customer-create page asks for; the box's own split-horizon resolver on the appliance LAN IP answered `google.com` but not the dashboard name at 13:27:39Z; the agent applied the record at **13:27:44Z** (`lanresolver: applied split-horizon record … ip=192.168.0.158`, 3 m 46 s after the controller started), so the box CAN answer the name — **but only to a device that uses the box as its DNS server, and no document, screen or mail tells a household to do that**; the router and the installer-offered DNS answer nothing. The page was reachable only at the guest's LAN address with the name forced (`curl --resolve …:443:192.168.0.158`). **A volunteer could not have done that.** **What it needs:** an operator ruling — the hub creates the tunnel and DNS at customer creation, or the product gives a household a LAN address that works with no DNS change. | **READY — rank P3-LOW; owner: CC** **Re-ranked 2026-10-03: P3→P4: operator-side automation; the operator creates the tunnel by hand per the day-0 runbook.** | — | — | CC | | **R-504** | Install & onboarding | P4 | **[P3-LOW] `iso.felhom.eu` cannot show an index page on its own — its root returns 404, and the download page lives on the website instead.** MEASURED 2026-09-14: `https://iso.felhom.eu/` and `/index.html` → 404; only named objects answer. The host is an R2 bucket behind a custom domain; whether R2 would serve an uploaded `index.html` at `/` was **not measured** (uploading anything to the public bucket is a publication). The ISO v1.27.0 task puts the Hungarian download page at `felhom.eu/letoltes` (published with the ISO, after the operator's yes). **Remaining:** a redirect from `iso.felhom.eu/` to that page needs a Cloudflare rule the session has no credential for. | **WAITING-ON-OPERATOR — rank P3-LOW; owner: operator (Cloudflare rule)** **Re-ranked 2026-10-03: P3→P4: households are sent to the website's download page; the bare address is cosmetic.** | — | — | operator | -## Apps & catalog — 20 rows (P2 1, P3 10, P4 9) +## Apps & catalog — 20 rows (P3 11, P4 9) | ID | Category | Sev | What | State | Blocked on | Next action | Owner | |---|---|---|---|---|---|---|---| | **R-1091** | Apps & catalog | P3 | **Catalog copy that is wrong or stale at the pinned tag, found writing the guides — each needs an operator word or a live check, so none was changed tonight.** From source at each app's tag (evidence per app in `audits/app-guides-2026-10-15/B/`): (1) and (2) **DONE 2026-10-16** (operator decision D12: reword): calcom's use case now names Apple Calendar or a CalDAV calendar instead of Google/Outlook, and bentopdf's tagline and use case say the browser does the work and the file stays on the device (catalog CHANGELOG 2026-10-16); (3) **sabnzbd**, **nzbget** use_cases say Radarr/Sonarr/Jellyseerr hand them jobs — the box wires only qBittorrent and Transmission (`mediawire.go`), and a hand link needs a path mapping; (4) **onlyoffice** first_steps 3–4 tell the household to wire Nextcloud by hand — the integration toggle does it (`internal/integrations/onlyoffice_nextcloud.go`); (5) **glance**, **homepage** first_steps send the household into config files in a Docker volume no household tool reaches; (6) **lubelogger** starts with only the „Dashboard" tab and in miles (shipped `appsettings.json`), so first step 3 cannot be done as written; (7) **home-assistant** first_steps promise auto-discovered devices — the app runs on a Docker bridge, where mDNS/SSDP were measured deaf (spike R-6); (8) **homebox** and **gitea** `add_people` name menus that moved or read differently at the tag; **bookstack** `default_creds` still says `admin@admin.com / password`, which after_install replaces; **dawarich** names the API key's page „Settings" (source suggests the account page); **code-server** says „install your tools" with no sudo; **komga** names the discontinued Tachiyomi; **seerr** `docs_url` points at Overseerr; **wanderer**'s comment still says it cannot close sign-up (R-714 is fixed in the same file); (9) **nzbget** and **grafana** deploy-field descriptions say the password can be changed later — nzbget re-applies its env password at every start and grafana ignores the field after first start (R-1086 is the qBittorrent/Transmission side). | OPEN — filed 2026-10-15 | — | Live-check (5)–(7) and (9) on 9202; fix (3), (4), (8) as copy with the freeze re-admitted; re-run the guide label check for any app whose copy moves | CC | +| **R-1142** | Apps & catalog | P3 | **The catalog allow-list (R-971, controller v0.328.0) lets one app bind ANOTHER app's folder under `${HDD_PATH}/appdata/`.** The data root is `appdata/` as a whole: a per-app rule needs a name map, because paperless-ngx keeps `appdata/paperless`. A catalog push could give app A app B's files (not the box: no privilege, the household's own data). | OPEN — filed 2026-10-17 | — | A per-app appdata name in `.felhom.yml` (or a map in the controller), then `appdata//` as the root; catalog gate the same | CC | | **R-1092** | Apps & catalog | P4 | **Three apps show no guide: plant-it and home-assistant carry one HIDDEN (`checked_version: ""`), emby has none.** plant-it: `lifecycle: abandoned`, and its pinned image does not exist upstream, so there is no screen to check labels against. home-assistant: its frontend follows the browser language and the Hungarian strings are fetched at build time, not kept in git, so the Hungarian labels cannot be checked against source (the draft is written; the shipped frontend in the image could check them). emby: its server's web UI is in no public source at 4.11.0.4. Evidence: `audits/app-guides-2026-10-15/B/`. The other 62 templates show a checked guide (controller ≥ 0.315.0). | OPEN — filed 2026-10-15 | — | home-assistant: check the Hungarian labels against the frontend files in the pinned image (read-only, on a box running it), then set checked_version; emby: label-check against the image's own web files the same way; plant-it: nothing while abandoned | CC | | **R-981** | Apps & catalog | P3 | **The homepage app gets the Docker socket (that is root in the guest) and has no login, no setup gate and no family gate.** Fix: remove the socket mount, or put homepage behind the family gate. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN | — | Change the catalog template; prove on 9202 | CC | -| **R-971** | Apps & catalog | P2 | **A change pushed to the catalog's `main` reaches the installed apps on every box within about 15 minutes, and nothing on the box refuses dangerous container settings (privileged mode, the Docker socket, host mounts, host namespaces); so whoever can push to the catalog repository can take every guest.** Design-shaped: installed apps take catalog fixes on purpose. Decision for the operator in `STATUS.md`. Fix options: a privilege allow-list check on the box before any compose is written (and in catalog CI), and later signed catalog releases. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | WAITING-ON-OPERATOR — confirmed by a second reader | operator decision | Decide; then build the box-side check | operator | | **R-676** | Apps & catalog | P3 | **[P3-LOW] Watch: immich's first start restarted 12 times — decision 28's crash-loop stop (6 in 10 min) would stop it.** From the 2026-09-17 chaos night (DB connection dropped during the first-start geocoding import on a 6 GB guest; it did not recover that night). No healthy app in any drill evidence restarts on a first start (1831 samples, 40 live containers), so the threshold stands; this row exists so the first immich install under v0.269.x is watched. `audits/night-2026-09-24/A3/40-first-start-restarts.txt` **2026-09-25 night (read from source, v0.271.0): a DEPLOY's first start is NOT covered by decision 28's suppression** — `Deploying` clears when `compose up -d` returns (`deploy.go` "Clear deploying flag"), and `ObserveUnhealthy` then samples the app; an automatic update's step, verify and undo ARE covered (`Updating`, pinned by `TestD28_NoCrashLoopStopDuringAnAutomaticStep`). So a first start that restarts ≥ 6 times in 10 min is stopped — which R-676 already accepts for a broken first start; a healthy slow first start would be stopped too. **-- 2026-09-30: the first-start restarts are explained.** immich's first-start geodata import OOM-kills its database at 512M on a guest with no swap (R-732, measured: 61–104 kills); the 2026-09-17 chaos-night case (DB connection dropped during the import on a 6 GB guest) fits it. Fixed in the catalog (`56c4888`, 768M). The watch itself (decision 28 on a DEPLOY's first start) is unchanged. | **OPEN — P3; owner: CC (watch)** | — | — | CC | | **R-994** | Apps & catalog | P3 | **Seven apps peak above 80 % of a memory limit, and 23 apps' typical use is above their catalog `mem_request`.** MEASURED 2026-10-13 (`documentation/audits/sizing-2026-10-13/SIZING.md` §3, `evidence/catalog-check.json`). Near a limit — risk: the container is stopped for using too much memory: docmost (container 475 of 512 MB), romm (703 of 768), termix (117 of 128), uptime-kuma (227 of 256, tester-1 telemetry), grafana (435 of 512), immich (`immich-server` 1,665 of 2,048 during a 200-photo import), tandoor (827 of 1,024). No container was killed during the bench. Under-stated request (typical ÷ request): calcom ×4.0, adventurelog ×3.6, termix ×3.4, claper ×2.5, grafana ×2.2, komga ×2.2, docmost ×2.1, recipe-importer ×2.1, uptime-kuma ×2.1, zipline ×2.0, and 13 more ≤ ×1.8. Templates NOT changed (read-only run). | **NARROWED 2026-10-17** — the six other limits raised (catalog `bff82f9`: docmost 768M, romm 1024M, termix 256M, uptime-kuma 384M, grafana 768M, tandoor 1280M; peaks now 46–69 %; live on 9201: docmost 805306368 bytes); the 23 requests → R-1125 (operator: wait). **Left: Immich** (`immich-server` 1,665 of 2,048 MB on a 200-photo import; tester-1's real library peaked at 2.9 GB) — raise after a measurement with a larger library. | — | Measure Immich with a larger library (bench), then its limit | CC | | **R-1018** | Apps & catalog | P3 | **The OnlyOffice links (FileBrowser, Nextcloud) have no recorded live proof since the move to Proxmox.** `controller/internal/integrations/` holds two handlers; no audit after the Proxmox move shows a document opened from FileBrowser or Nextcloud in OnlyOffice (searched 2026-10-14). The media-app links of `15` §6 will copy this code, including its revoke-on-stop rule (`lifecycle.go:10-13`). | **OPEN** — filed 2026-10-14 | — | One live walk on a Tier-0 box: install OnlyOffice + FileBrowser, open a document, save, read it back; then stop one app and record what the link does | CC | @@ -159,16 +159,15 @@ stopping line that lies. | **R-683** | App updates | P3 | **[P3-LOW] Watch: after a power cut during an update's health check, the hold named an HOUR-OLD second-drive copy, not the one the update's own backup should have just made.** 2026-09-24 chaos round 3 (nextcloud, `backup_max_age: 1m`): no `backing-up` phase was seen and the hold named Tier 2 at 13:04 for an update pressed at 14:04; the pre-cut controller log was lost with the container (the runner now saves it at arm time — R-320). Round 11, the same action without a power cut, named a fresh 14:34 copy and logged the Tier-2 copy. The sentence was TRUE (it named the copy it offered); the question is why the update did not back up first. Not reproduced; watch the next power-cut drill. `audits/night-2026-09-24/E/round-03*.json`, `E/round-11-controller-pre.log` | **OPEN — P3; owner: CC (watch)** | — | — | CC | | **R-785** | App updates | P3 | **[P3-LOW] SparkyFitness is pinned 11 releases and a major behind upstream (v0.17.3; upstream v1.7.3, v1.6.0 dated 2026-07-24).** READ 2026-10-01 (`audits/visitors-2026-10-01/C/bench/C1-previous-tag.txt`). **Needs:** an update walk 0.17 → 1.x through the ladder (bench + box), after R-784 is decided. | **OPEN — rank P3-LOW; owner: CC (after R-784)** | — | — | CC | -## Backup & restore — 29 rows (P2 3, P3 15, P4 11) +## Backup & restore — 28 rows (P2 3, P3 14, P4 11) | ID | Category | Sev | What | State | Blocked on | Next action | Owner | |---|---|---|---|---|---|---|---| -| **R-1012** | Backup & restore | P3 | **A full host root can leave an orphan thin snapshot in the LVM pool that LVM no longer knows; from then on every whole-guest backup of that guest fails at the snapshot step („Creation of new snapshot 4 of device 2 failed"), and nothing in the product finds or names the cause.** Made on demo-hp 2026-10-14 by this session's C4 test (the root filled while vzdump snapshotted; LVM could not write its metadata): `thin_ls --metadata-snap` shows device 4 (19 GiB, a snapshot of 9201's data disk), LVM only 1 and 2 (`audits/power-cut-fixes-2026-10-14/E/thin-ls.txt`). R-1006's cleanup keeps the root from filling; this row is the repair and the detection. | **OPEN** — filed 2026-10-14; **repaired on demo-hp 2026-10-14 12:02 UTC** (operator „B2 - A"): `dmsetup message pve-data-tpool 0 "delete 4"`, then thin_ls and LVM agree on devices 1 and 2, and the next backup (12:10, after a real power cut) made its snapshot and finished (`audits/power-cut-fixes-2026-10-14/E/repair.txt`, `E/next-backup.txt`). What is left is the detection. 2026-10-14: P2 → P3 — the cause (a full host root) is what agent v0.156.1's leftover cleanup now prevents | — | CC: a check that compares the pool's thin ids with LVM's and names an orphan in the backup failure | CC | | **R-978** | Backup & restore | P3 | **Starting the escrow ceremony again can make the customer's written-down recovery code useless: the new package replaces the old one before the new code is confirmed.** It can happen to an ordinary customer who starts and walks away. Fix: keep the previous package until the new code is confirmed. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN | — | Fix in the agent with a test | CC | | **R-973** | Backup & restore | P3 | **A compromised controller can make the whole-guest backup run again and again, and each run prunes older restore points: the local tier directly, and the off-site tier through the store's own nightly keep-last-2.** So `07` §8 row 10 (ransomware) is weaker than it reads. The second reader confirmed the local half; the off-site half is inferred. Fix: a cadence or daily cap on the local-API backup route, and off-site snapshots that a nightly prune cannot remove. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN | — | Add the cap with a test; decide the off-site retention shape | CC | | **R-232** | Backup & restore | P2 | **DooPlex's backup makes every copy inside the same box — and nothing tells anyone when it fails.** Surveyed read-only 2026-08-06 (`audits/RECON-dooplex-backup-2026-08-06.md`). **What works:** five sets, 14/14 successful runs in 14 days; a file was restored from the `data` repo and matched the live original **byte for byte**; every set except two is cross-disk; k3s is integrity-checked on every run. **What the matrix exposes, ranked:** (a) **`notify_failure` is a no-op** — `NOTIFY_ON_FAILURE=true` but `NOTIFY_WEBHOOK_URL` is commented out, so a failed backup notifies **nobody**; the project already has a working Resend path that CI uses. Cheapest item, and it makes every other failure visible. (b) **Nothing leaves the box** — no rclone, no remote repo, no off-site target anywhere; Longhorn's target is `nfs://192.168.0.180:` pointing at DooPlex itself, and the only outbound-looking cron pulls *inbound* from Hetzner for a different project. The machine that runs the hub managing the customers' off-site chain has no off-site copy of its own. (c) **The backup tree is a single writable path** and the restic repos are not append-only — one bad script or ransomware destroys every copy at once. (d) **Two same-disk sets**: `.claude-memory` and the PostgreSQL dumps, whose source directory sits *inside* the backup tree. (e) **Longhorn `retain=1`** — one generation per volume, so a corruption noticed a day late has no earlier copy. (f) **`/opt/backup/docs/BACKUP-RESTORE.md` does not exist** though the systemd unit advertises it. (g) **`secrets/restic-repo` has never held a snapshot** — `backup-secrets.sh` contains no `restic` call; the secrets are GPG files on `sda1` only. (h) **No restore has ever been run** beyond today's single-file probe — the matrix's "ever demonstrated?" column is otherwise entirely empty. **Not a finding:** the restic passphrase. The on-box copy is on `sdb1`, a different disk from the backups, and the **operator holds an offline copy out of band** — so a disk loss is recoverable. The narrow residual is that it is operator-held rather than system-held, unlike the customer case's hub-vaulted escrow, so it should be confirmed current and findable by someone else. **Nothing was changed by the recon.** | **NARROWED 2026-10-08 — owner Viktor.** (a) DONE with the operator's yes in chat: `notify_failure` now mails admin@felhom.eu through Resend; proven by one test mail that reached the inbox (`audits/day-2026-10-08/r232/`; no backup was started). (b) partly: the hub database leaves DooPlex nightly to ep0 (R-173); everything else stays on the box. (c)–(h) unchanged. **2026-10-09 (operator yes in chat): (b) and (h) DONE for Gitea and the secrets.** Nightly 00:20 `felhom-dooplex-offsite` pushes Gitea (614 MB of repositories, the `gitea` database dump taken first, `app.ini`) and the nightly GPG secrets export, encrypted with a NEW key, to ep0 `operator` (`host/dooplex-gitea`) on the hub-DB write-only token — no change on ep0; first push 57 s; restore test weekly (manifest, `git fsck` every repo, `pg_restore --list`), run once: 27 805 files, 10 repos OK; `DooplexGiteaOffsiteStale`/`DooplexGiteaRestoreTestStale` in force (`absent()` included, promtool + 2 red-proofs); every backup unit now mails on failure (`OnFailure=`, dry run reached the inbox). **First real Gitea restore:** into a throwaway on the bench, 10/10 repos, product `main` = live, a file byte for byte, a login (`runbooks/gitea-restore.md`). The hub came back too (R-173, closed). **The registry is left out on purpose:** its images rebuild from the code. **Correction to the recon §7:** the API mirror holds ONE repo (`homelab-manifests`), not all — Gitea had a single copy, on DooPlex. `audits/dooplex-survival-2026-10-09/` **LEFT:** (c) append-only for the on-box tree, (d)–(g) as written; the paper copy of the new key (operator). **READY** for the rest | — | — | operator | | **R-893** | Backup & restore | P3 | **After a failed OFF-SITE replay, the rollback pours the NEWER pre-restore copy over the OLDER volume just put back.** Read in source 2026-10-06 (R-638 option A, not measured): `internal/backup/offbox_reconstitute.go` writes the undo copy from the live (newer) database, replaces the volumes with the snapshot's older tars, then — when the replay fails — `rollbackSafetyDump` loads that newer dump over the older database volume. The loader only drops what the dump knows, so tables the newer migration removed stay; and when the snapshot's older definition was written, the rollback branch does not put the newer definition back, so the older app starts on rolled-back data; non-database volumes stay at the snapshot's state. An order change cannot fix it (the only undo is a logical dump, and its volume was replaced). Known limit in `07` §6.3. **-- 2026-10-08 design:** `audits/day-2026-10-08/design-R-893.md` — re-verified; and the screen's „your data is back as it was" (`err.backup.db_restore_failed_rolled_back`) is false for files, other volumes and the app version. Question D8 on STATUS's decision sheet. **-- 2026-10-08 14:16 operator ruling D8 (`09` §3 decision 192):** yes, both, in that order — first the app stays stopped for support (option C), next „put back exactly as it was" (option A). | **NARROWED — 2026-10-09: the first half DELIVERED in controller 0.304.0. Slice 0 (the live measurement) NOT run: 9202 has no off-site, and a dump cannot be cut by hand inside an encrypted off-site copy on Tester 1; it needs a scratch off-site repo — a plan, not a finding.** **NARROWED 2026-10-08 — the first half (D8 option C, the app held stopped) is built on controller main and ships tomorrow; what stays open is the second half, „put back exactly as it was” (option A: a pre-restore copy of the app's volumes and placed files, a fit check, disk for one copy; the slice-0 measurement on 9202 first).** **OPEN — filed 2026-10-06** **2026-10-06 night: verified in source, no code** — `offbox_reconstitute.go:758` (undo dump from the live DB), `:778-860` (files and volumes from the snapshot), `:804` (the snapshot's definition is written when its version differs), `:898` (the rollback loads the newer dump over the older volume; nothing writes the live definition back). Not a reorder fix: it needs R-638 option B (a rebuilding loader) or a pre-restore volume copy (disk cost; R-685's class). Which state a household gets after a failed off-site replay is the operator's call. Next: the 9202 measurement, then the design. | a design: R-638 option B (a loader that rebuilds instead of overlays) or a pre-restore volume copy | Measure it once on 9202 (a forced replay failure after an off-site restore over a migrated app); then a design for the operator | CC | -| **R-895** | Backup & restore | P2 | **The hub's clean-up-window check trusts the snapshot counts the box sends, so a broken-into box (or past-dated fakes added through the add-only key) can shrink the real off-site history without an alarm.** READ 2026-10-06 night in source (R-822's design): the before/after comparison uses counts the box itself reports (`hub/internal/offsitekeys/service.go:284`, `:343`); new fakes keep the count level. Decision 68 already accepts a box-trusted count. | **OPEN — filed 2026-10-06 night** **2026-10-07 07:58: kept open for later (`09` §3 decision 166).** | a design + one read-only measurement (does the Storage Box shell on port 23 show snapshot file upload times?) | Option B of `audits/night-burndown-2026-10-06/design-R-822.md`: the hub lists the repo's `snapshots/` files over its own login before and after a window and alarms on snapshots no box run explains | CC | +| **R-895** | Backup & restore | P2 | **The hub's clean-up-window check trusts the snapshot counts the box sends, so a broken-into box (or past-dated fakes added through the add-only key) can shrink the real off-site history without an alarm.** READ 2026-10-06 night in source (R-822's design): the before/after comparison uses counts the box itself reports (`hub/internal/offsitekeys/service.go:284`, `:343`); new fakes keep the count level. Decision 68 already accepts a box-trusted count. | **OPEN — filed 2026-10-06 night** **2026-10-07 07:58: kept open for later (`09` §3 decision 166).** **2026-10-17: READY — design updated with the measurement** (`audits/agent-allowlist-rotation-2026-10-17/P4/design-R-895.md`): the hub needs NO new credential (the sealed sub-account password it already uses daily; write-capable today); the provider shows upload times (2026-09-01 evidence on demo-hp's sub-account). Slice 1 in hub v0.157.0 (deployed 2026-10-17 on the operator's yes): the daily key audit also lists `snapshots/` read-only and LOGS count/newest/oldest — the first 07:10 run is the live measurement. | the first logged listings (hub 07:10 audit), compared with the box's own count | Option B of `audits/night-burndown-2026-10-06/design-R-822.md`: the hub lists the repo's `snapshots/` files over its own login before and after a window and alarms on snapshots no box run explains | CC | | **R-923** | Backup & restore | P2 | **The password manager (Vaultwarden) runs on DooPlex and holds the keys that open DooPlex's own off-site copies — a loss of DooPlex loses the keys with the copies.** Operator finding, 2026-10-09 11:19: "the password manager also runs on DooPlex (Vaultwarden), so I probably need to back up its DB somewhere too." Measured: `vaultwarden-system/vaultwarden` (1.37.3), PVC `vaultwarden-data` 7.4 MB (`db.sqlite3` + WAL, `rsa_key.pem`, no attachments); its only backups were Longhorn on DooPlex. **A key in a password manager that runs on DooPlex is not off DooPlex** — R-173's "both keys are off DooPlex" (2026-10-05) and the hub-DB runbook Step 0 were wrong in that sense. Also seen: Vaultwarden warns its `ADMIN_TOKEN` is plain text (an Argon2 hash is its guidance). | **NARROWED 2026-10-09 (operator yes in chat):** Vaultwarden's own `backup` copy + `rsa_key.pem` ride the nightly encrypted DooPlex→ep0 job (`scripts/dooplex-offsite/`), checked on every push and every Sunday (integrity, users, items — rows only). That copy opens only with the DooPlex off-site key, and the vault only with the master password — so the circle is broken only once both are on paper outside DooPlex: `runbooks/break-glass-sheet.md`, `runbooks/total-loss-of-dooplex.md`. **LEFT:** the operator prints the sheet and stores it away from home; the `ADMIN_TOKEN` hash (a DooPlex change, operator's word). | operator: print the sheet | Operator: print + store the sheet; then close | operator | | **R-49** | Backup & restore | P3 | **[P2] The offsite capture set is ~90% cache and duplication — 1.1 GB of a 1.2 GB immich "photo backup".** Measured 2026-07-19: `immich_ml_cache.tar` **823 660 032 B (~60%)** — re-downloadable ML model weights; `immich_postgres_data.tar` **308 251 136 B (~23%)** — a raw tar of the postgres data dir that DUPLICATES the logical `.sql` dump captured beside it; `upload/backups/` **18 MB** — immich's own nightly dump, a backup inside the backup, growing daily; plus the stranded pre-v3 `dccc13fe…` tree (~36 MB) no DB has ever referenced. Actual irreplaceable content: **72 MB of originals**. **MIGRATED FROM `ROADMAP.md` 2026-08-22 (R-369) — originally filed 2026-07-19, size S–M, roadmap state `idea`.** Moved verbatim; nothing added or reinterpreted. The roadmap keeps its copy as history, marked moved. | **OPEN — migrated from ROADMAP 2026-08-22, rank unchanged** **Re-ranked 2026-10-03: P2→P3: wasted space and transfer, no data risk; needs a capture-set ruling, not a sale blocker.** | — | **Evidence: `audits/DIAG-immich-restore-round2-2026-07-19.md` §4 (full byte breakdown).** This is the customer's offsite quota and transfer cost, and it lands on the Hetzner sub-account they are billed for. **Recorded, deliberately not changed** — a capture-set exclusion is a data-loss-shaped decision and gets its own ruling, not a drive-by edit. Candidates in priority order: (a) `immich_ml_cache` — pure cache, strongest case; (b) the `postgres_data` volume tar where a logical dump of the same DB is already captured (the dump is what the restore path actually replays); (c) `upload/backups/`. Likely generalises past immich into a template-classification rule about cache volumes and self-backup directories, so it should be specified against the catalog, not one app | CC | | **R-231** | Backup & restore | P3 | **`/opt/backup/scripts/` on DooPlex is unversioned host state** — found 2026-08-06 while adding the auto-memory store to the backup set. No repository tracks the scripts that protect the recovery chain, so the edit made that day (`CLAUDE_MEMORY_DIR` in `backup-config.sh`, multi-path restic call in `backup-data.sh`) exists only on the box. This is the same class the part-2 session was closing, found inside the fix for it; the change is transcribed in `felhom.eu/workspace/README.md` so it is at least *recorded*. **Two related facts, both understating current safety:** the backup destination (`/mnt/5_hdd/backup`) is on the **same physical disk** as the workspace it protects, and the DooPlex backup set has **no off-site leg** (`sync-hetzner-backups.sh` is jarrs.eu and pulls *from* Hetzner *to* DooPlex). Bringing a root-owned production backup script under version control, and deciding what installs it, is its own scoped change. | **VERIFY** — **MEASURED 2026-10-13 — the due check is GREEN** (read by the sizing session, two channels): `systemctl show dooplex-backup.service` → `Result=success`, `ExecMainStatus=0`, ran 03:27:39–03:33:29 CEST; its journal ends „All backup components completed successfully!”, and the manifests leg (the header-token change) logged „Successfully backed up homelab-manifests” with the plain URL `https://gitea.dooplex.hu/admin/homelab-manifests` (no token in it). Left open, not closed: the row also carries the same-disk and no-off-site facts, which closing would drop — the owner decides. **VERIFY** — **2026-10-12 evening: the token left the clone URL** (operator yes in chat; homelab-manifests `1943a70`, installed by the role): `backup-manifests.sh` sends it as an `http.extraHeader` through `GIT_CONFIG_*` env; read-only proof: `git ls-remote` of the private repo with the header → HEAD, without → refused. The 2026-10-13 03:10 run is the test of both changes. **VERIFY** — **2026-10-12: in git and installed from it** (operator yes in chat, operator in the session): homelab-manifests `fdd324b`, `homelab-ansible/roles/dooplex_backup/` (11 scripts + the timer units, byte-identical to the live ones; no secret in any file; `install.sh` dry run / `--apply`). The install changed only the units' mode (744 → 644). Baseline: last night's run `Result=success`, exit 0 (`audits/answers-2026-10-12/s20-backup-scripts.txt`). **Seen by the commit's security review, NOT changed (a DooPlex behaviour change needs the operator's word):** `backup-manifests.sh` puts the Gitea token into the clone URL for the mirror run (visible in the process list for those seconds) and strips it from the stored remote afterwards; no token VALUE is in any file, and the repo is private (anonymous API read = 404). A header-based fetch (`git -c http.extraHeader=…` from the token file) would remove it — one line in the decision list. **READY** — operator ruling 2026-10-12 (S20, `09` §3 decision 215): yes — the scripts into a repo + an install script; the operator present for the install. **READY** — owner Viktor | — | Owner: close the row (the install from git is proven) and file the same-disk / no-off-site facts as their own row if they still hold | CC | @@ -193,7 +192,7 @@ stopping line that lies. | **R-32** | Backup & restore | P4 | **[P2-HIGH] RESET must purge the customer base dir; the orphan card must stay honest; unattributed bytes must be visible.** The rehearsal's S7 said in advance that an orphan card would BE a finding — and one appeared (16:58:14). Cause: RESET's `"hetzner":"ok"` leg destroys the sub-account, but **a Hetzner sub-account is an access-control object, not a data object** — its directory survives, so re-enabling offsite recreated an account over the previous lifecycle's ciphertext, encrypted under a key that same RESET had destroyed. **MIGRATED FROM `ROADMAP.md` 2026-08-22 (R-369) — originally filed 2026-07-21, size M, roadmap state `idea`.** Moved verbatim; nothing added or reinterpreted. The roadmap keeps its copy as history, marked moved. | **OPEN — migrated from ROADMAP 2026-08-22, rank unchanged** **2026-10-06 night: design written** (`audits/night-burndown-2026-10-06/design-R-32.md`) — pick A: purge the repo + set-aside copies through the sub-account's own password login BEFORE deleting it (no main-account credential); the „repo data destroyed" comment at `hub/internal/offsite/offsite.go:276-280` is false for the shared tier; part (3) waits on a read-only measurement (`du` on port 23). Operator: agree to route A. **2026-10-07 07:58: `09` §3 decision 167 — option A yes, and delete the old RESET leftovers once.** **2026-10-07 (Part D step 2, read only):** the hub shows bx11 holding 3.3 GB of data while the four live customers account for ~0.6 GB → **~2.7 GB belongs to no live customer**; no live customer shows a set-aside folder, so the leftovers are in homes whose sub-account is gone, reachable only by the pool box's MAIN account. **The one-time clean-up is STOPPED:** no main-account login for bx11 is held on DooPlex (the API token in the credentials file sees no box). Needs: the main-account login, or the operator lists and deletes in the Hetzner console (`audits/day-2026-10-07/D/D2-listing-readonly.txt`). **2026-10-07: option A DELIVERED (hub 0.142.0):** RESET purges the repo and every `.orphaned-*` through the sub-account's own login before deleting it, and keeps the sub-account if the purge fails (red-proved, `audits/day-2026-10-07/D/`). Not yet seen live (no scratch RESET today). **The one-time clean-up of the ~2.7 GB of leftovers is STOPPED:** it needs the pool box's main-account login (see above). **2026-10-07 14:43 — `09` §3 decision 173: leave the ~2.7 GB of old leftovers for now; re-ranked P2 → P4.** Option A itself is delivered (hub 0.142.0). What stays open is ONLY the leftovers: they were computed (box total minus live customers), never listed, and deleting them needs the pool box's MAIN login, which reaches every customer's data. **Blocker: a safe main-account path.** | — | **Ruling from the run (three parts, deliberately separate):** (1) because RESET destroys custody, the ciphertext it leaves behind is unrecoverable **BY DESIGN** → RESET gains a **main-account purge of the customer base dir** (the existing operator ack already covers it); (2) the **move-aside guard STAYS** for reinstall-*without*-RESET — there custody survives and the card's "history recoverable" promise is true (R-26 depends on exactly that); (3) the operator **Restic tab shows per-customer directory bytes vs attributed snapshot bytes**, so dead data cannot hide. Measured on the pool box that night: **49 M attributed** (2 snapshots, 48.717 MiB) against **1.4 G + 3.0 M unattributed** across TWO `.orphaned-*` dirs. Evidence `restic-and-pool.txt` | CC | | **R-1087** | Backup & restore | P4 | **Whole-box copies left on the OLD local target are never pruned after the target moves.** MEASURED 2026-10-15 (storage-identity build B space check): the N100's host root (`local`) holds three copies from 2026-07-28 (5.5–5.7 GB each, ≈ 16.8 GB); its agent's `local_backup_target` is now `felhom-backup` (the USB disk, retention 3), and retention prunes only the current target. On a small-root box such leftovers would starve the local copies (R-548). Evidence: `audits/storage-identity-buildB-2026-10-15/space/felhom-pve-9201.txt`. | OPEN | — | Decide whether the agent prunes its own `vzdump-lxc--*` files on a former target (after a newer copy exists on the new one), or the target move asks; until then, the N100's three July files can be removed by hand with the operator's word. | CC | -## Storage & devices — 13 rows (P3 6, P4 7) +## Storage & devices — 12 rows (P3 6, P4 6) | ID | Category | Sev | What | State | Blocked on | Next action | Owner | |---|---|---|---|---|---|---|---| @@ -209,16 +208,17 @@ stopping line that lies. | **R-1076** | Storage & devices | P4 | **A cheap SSD reports a vendor-less WWN, so a WWN-first disk key may not be unique.** MEASURED 2026-10-15 on the N100's system SSD (AirDisk 512 GB): `ID_WWN=0x5000000123456f1f` — NAA-5 with OUI `000000`, a pattern that looks like a placeholder; its serial `QDF922W009654S30EX` looks real. The agent's wipe binding prefers `wwn-` links (A/internal/storage/durable_device.go:32). Not known whether two units share the WWN. Evidence: `audits/storage-identity-2026-10-15/B-n100.txt`. | OPEN | — | Check a second unit of the same model (or vendor data); if shared, make the agent skip OUI-000000 WWNs (the design already keys on model+serial, `architecture/16-storage-physical-identity.md` §3). | CC | | **R-1077** | Storage & devices | P4 | **No USB enclosure that hides or fakes a disk's serial has been measured.** The storage-identity spike (2026-10-15) had none at hand and found none in earlier evidence; the design's `usb_bridge` unknown rule (`architecture/16-storage-physical-identity.md` §3) is therefore an assumption. Cheap single- and dual-bay bridges are known to report one bridge serial for any disk. Evidence: `audits/storage-identity-2026-10-15/README.md` case 4b. | OPEN | — | Buy or borrow one cheap USB enclosure (single and dual bay), run `tools/physdisk_probe.py` against it, and set the rule from what it reports. | Operator (a device) then CC | | **R-1094** | Storage & devices | P4 | **The whole-disk format refusal tells the household to format the free part — which is refused while the other part is in use.** SEEN 2026-10-16 (controller 0.318.0): 409 „Ezen a lemezen egy használatban lévő meghajtó van, ezért a teljes lemez nem formázható. Csak a szabad részét formázhatod." — since agent 0.158.1 a part next to a mounted part can only be USED (the root wrapper trusts no „Felhom's own" claim). Evidence: `audits/storage-identity-buildB-2026-10-15/live/14-samedisk-refusal-offer.txt`. | READY | — | Reword `err.web.meghajto_hasznalatban_formazas` (hu + en): „… nem formázható. A szabad részét használatba veheted." Next controller release; the copy gates run. | CC | -| **R-1124** | Storage & devices | P4 | **A drive decommissioned and then unplugged keeps its agent mount unit enabled on the host** (`mnt-.mount`, „Managed by felhom-agent”, `WantedBy=multi-user.target`), so every later boot waits for a device that is gone, and the agent's watchdog reports it „disconnected”. Decommission retires the drive logically (agent: „logical retire, data untouched”) and releases only the guest bind; no product path releases the host mount or forgets the drive. Seen on Tester 1 at the R-1022 teardown 2026-10-17 (four units left; the boot delay is read from the unit, NOT measured). | OPEN — filed 2026-10-17 **2026-10-17 (later):** on Tester 1 the four units were disabled and removed by hand (read before and after, `audits/filebrowser-memory-ci-2026-10-17/P4/`); the agent did not write them back, but it still lists the four drives and logs „enrolled drive absent by UUID — will retry” every ~20 s. The product fix (decommission releases the host mount and lets the agent forget an absent drive) is an agent change — it waits for the next vouched agent release. | — | Decide „forget a decommissioned drive” (unit disabled + removed when the drive is absent), with a test | CC | -## Security & access — 19 rows (P1 1, P2 2, P3 10, P4 6) +## Security & access — 21 rows (P1 1, P2 2, P3 12, P4 6) | ID | Category | Sev | What | State | Blocked on | Next action | Owner | |---|---|---|---|---|---|---|---| +| **R-1139** | Security & access | P2 | **Tester 2's break-glass root password was not rotated with the others (R-954, operator decision D1 = A, 2026-10-17): the box has been offline since 2026-10-04.** demo-hp, Tester 1 and demo-felhom were rotated and proven on 2026-10-17 (`audits/agent-allowlist-rotation-2026-10-17/P3/`). | OPEN — filed 2026-10-17 | Tester 2 reporting again | When it reports: the installer's step 4b alone (`P3/rotate-step4b.sh` shape), then the vaulted value logs in (PVE API 200) and the old one is refused | CC | | **R-985** | Security & access | P3 | **Our zero-knowledge wording is stronger than the product: on a live box the operator can reach the off-site backup key (through the vaulted root credential and the operator tunnel) without the customer's recovery code.** Design-shaped; decision for the operator in `STATUS.md`. Fix options: change the wording, or remove operator root reach to the key. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | WAITING-ON-OPERATOR — design-proposal | operator decision | Decide the wording or the change | operator | -| **R-954** | Security & access | P2 | **We cannot tell whether anyone used the old hub operator key (R-950) or the open hub read routes (R-951) before 2026-10-12; if someone did, they could have read every box's break-glass root credential and every customer's box report.** The exposure began in February 2026; the logs that could show it cover only 2026-10-12 from about 09:54 and one day in June, so about 238 days are not visible. Fix (operator decision): change the break-glass root password of every box (tester-1 holds a real person's data), or accept the risk; either way, keep ingress access logs long enough to answer this next time. | WAITING-ON-OPERATOR — filed 2026-10-12 by the security review; the log search was not completed in-session | operator decision | Decide: rotate every box's break-glass root password now, or accept | operator | | **R-1042** | Security & access | P3 | **The boxes' Cloudflare keys cannot change zone settings, so Cloudflare's own „Always Use HTTPS” cannot be switched on.** Measured 2026-10-14 on all three zones (`audits/https-only-2026-10-14/``1A-1-zones-and-keys.txt`): each `cf_api_token` is valid and sees only its own zone, but GET/PATCH `settings/always_use_https`, `ssl`, `security_header` answer 403; the redirect and transform rule phases too. The https-only fix therefore lives on the box (decision 246). A second layer at Cloudflare needs „Zone Settings: Edit” (and, for headers, „Transform Rules”) added to each customer's key — the hub's zone-reach check stays. | **WAITING-ON-OPERATOR** — filed 2026-10-14 | a key permission only the operator can grant | Optional: widen each customer key, then switch „Always Use HTTPS” on per zone | operator | | **R-1104** | Security & access | P3 | **Check https-only on Tester 2 when it reports again.** R-1033 (plain http on every public name) is closed on the three boxes that report (35 names → 301, 2026-10-16). Tester 2 (`sajatfelhom.hu`) has been off since 2026-10-04 on controller 0.292.0; its addresses answer Cloudflare 530 (tunnel down), so nothing is served in plain http while it is off. When it comes back it takes the floor (≥ 0.312.0 carries the redirect). | OPEN — filed 2026-10-16 | Tester 2 reporting again | Run `scripts/https_only_check.py` with Tester 2 in it; close | CC | +| **R-1140** | Security & access | P3 | **The hub's ingress access logs cover about 13 hours, so a question like R-954's („did anyone use the old key?") cannot be answered next time.** Measured 2026-10-17: the ingress-nginx controller logs to the container log only; k3s keeps `container-log-max-size=10Mi` × `max-files=5` — 03:30 to 16:18 today; 268 `hub.felhom.eu` lines in that window. 90 days needs either a DooPlex k3s setting (all containers, ≈ 9 GB raw / ≈ 1.3 GB compressed at today's rate) or a hub-only access log kept on the hub's own volume (≈ 500 lines a day, ≈ 15 MB for 90 days). Both change DooPlex or the hub beyond this session. | WAITING-ON-OPERATOR — filed 2026-10-17 | operator decision | Pick: (A) a hub-only access log on the hub PVC, 90 days (recommended: small, no DooPlex change), or (B) longer k3s container logs | operator | +| **R-1141** | Security & access | P3 | **`felhom-host-install --rotate-recovery` exists only as part of a full installer re-run, which also reinstalls the vouched agent and the root files.** On a box running a newer agent than the vouched one (demo-hp on the candidate, 2026-10-17) it would move the agent back; on a household box it is a much larger act than a password change. The 2026-10-17 rotation therefore ran the installer's own `step_break_glass` ALONE (`audits/agent-allowlist-rotation-2026-10-17/P3/`). | OPEN — filed 2026-10-17 | — | A `--rotate-recovery-only` mode (step 4b and nothing else), or a signed op; the runbook `break-glass.md` §5 points at it | CC | | **R-955** | Security & access | P4 | **The cross-box check of the hub API is done in source and tests but not fully proven: if a later check masks a missing id check, a box's key could read or change another box's hub record without any test noticing.** Read 2026-10-12: every box-facing `/api/v1` route takes the box from its own key or refuses a different id; 14 routes already had a refusal test, 5 got one (`crossbox_refusal_test.go`). Left: red-prove the 4 new tests that were not red-proved (as the restore-directive one was), and the live read-only check with one demo box's key against the other demo box's id. | OPEN — filed 2026-10-12 by the security review | — | Red-prove the 4 tests; run the live GET check on the two demo boxes | CC | | **R-924** | Security & access | P2 | **The operator signing keys exist only on DooPlex, with no passphrase and in no backup — a loss of DooPlex means no box ever accepts a signed agent update, config bundle or OS step again without an on-site re-enrollment.** MEASURED 2026-10-09: `/mnt/5_hdd/felhom.eu/felhom-op-operational`, `felhom-rec-recovery`, `felhom_op_ed25519` (0600 `kisfenyo`); `ssh-keygen -y -P ''` opens all three (no passphrase); `/mnt/5_hdd/felhom.eu` is in no backup set, the off-site job does not carry them. `04` §1 says the operational key is "passphrase-protected" and the recovery key "kept offline" — today neither holds (custody ruling 1 of 2026-09-16 put both on DooPlex for this phase; `04` §3.1). Both keys share one fate, so the cold key cannot do its one job (re-pin after losing the operational key). `runbooks/total-loss-of-dooplex.md` S6. | **NARROWED 2026-10-09 — operator ruling: option A** (print both AND copy them). The three key files and their public halves ride the nightly encrypted DooPlex→ep0 copy (`scripts/dooplex-offsite/`, a missing key refuses the push); the Sunday restore test proves each private key still derives its own public half (never printed). **LEFT:** the operator prints S6a/S6b (`runbooks/break-glass-sheet.md`); the keys still have no passphrase (custody ruling 1 of 2026-09-16 stands until the first sale). | operator: print | Operator: print the two keys; then close | operator | | **R-132** | Security & access | P3 | **`curl -w '%{redirect_url}'` reconstructs the request URL WITH its basic-auth credential** — so a `-u ":$HUB_PW"` call that never put the password in a URL still printed it **Merged 2026-10-05 from R-350 (duplicate):** (1) 2026-08-20 occurrence: POST /configuration/artifacts answers 303; leak lives only in the CC transcript under ~/.claude/projects/ on DooPlex, not in git/evidence (checked then). (2) `-v` and `--libcurl` also re-render the credential, not only %{redirect_url}; confirm redirects with %{http_code} + follow-up GET. (3) Rotation path: hub /configuration form (current_password/new_password/confirm_password); DB override wins over ConfigMap (break-glass); CC can rotate file-to-file without printing (operator-present-one-time-secrets) if asked. | **WAITING-ON-OPERATOR** — **ACTION: rotate `HUB_PW`** **Folded R-580 2026-10-03** (the same `curl -w %{redirect_url}` credential echo, seen again 2026-09-18). **Note 2026-10-12 (security review):** `manifests/hub.yaml` also carries the hub login's bcrypt hash in this public repository (the break-glass field); when `HUB_PW` is rotated, decide whether that field should stay empty. | — | Happened on 2026-07-31 while red-proofing the R-120 gate: the hub operator password was written to the session transcript by the write-out format, not by the request. `-u` is safe; the *reporting* was not. Rule: read the redirect from `-D -` and grep `^Location:`, never `%{redirect_url}`, on any authenticated call. Rotate the hub password (`/configuration` → Login password; ConfigMap `auth.password_hash` is the reset path) and update `~/.config/credentials` | Viktor | @@ -243,7 +243,7 @@ stopping line that lies. | **R-982** | Box system & updates | P3 | **The Proxmox host accepts root password login over SSH from the LAN (measured on demo-hp), so anyone on the household network can try the root password.** Design-shaped: the break-glass password exists for the console. Fix: key-only SSH for root; the password stays for the console. Evidence: `audits/security-review-2026-10-12/evidence/demo-hp-host-reads.txt`. | OPEN — design-proposal | — | Decide; then set it in the installer and the config bundle | CC | | **R-979** | Box system & updates | P3 | **Controller updates are trusted by registry tag alone (no digest, no signature), and the Update button offers any newer tag, even an unvouched one; the built-in traefik, cloudflared and FileBrowser images are also pinned by tag only.** Whoever can push to the registry decides what runs. Fix: pin by digest from the hub's vouched list; offer only vouched versions. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN — design-proposal | — | Design note, then build | CC | | **R-974** | Box system & updates | P3 | **A compromised host agent can likely become root on the host through the root wrapper that applies PBS settings: the wrapper accepts a folder path inside the agent's own writable tree and then acts on it as root.** Static (the symlink behaviour was reasoned, not run). Same class as R-861. Fix: a fixed root-owned folder, and refuse symlinks before any root action. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN | — | Fix the wrapper with a test on a throwaway box | CC | -| **R-972** | Box system & updates | P2 | **The shipped controller and hub programs are built with Go 1.24.13 and golang.org/x/crypto v0.31.0, and the agent with Go 1.26.0, so dozens of fixed Go security bugs (including html/template escaping fixes) are still in what runs on boxes and the hub.** Go 1.24 has had no security fixes since Go 1.26 was released (Go supports the two newest major releases: https://go.dev/doc/devel/release#policy); the scanners name fixes in Go 1.26.9 and 1.27.2. Also: the controller image's Debian 12 base has 12 fixable critical packages, and the Samba image has 48 fixable high ones. Fix: build all three on a supported, patched Go; update x/crypto; rebuild the base images. Scanner output: `audits/security-review-2026-10-12/scanners/`. | **NARROWED** 2026-10-16 (burndown 2): **controller v0.322.0 released** on Go 1.27.2 + x/crypto v0.58.0, runtime debian:12.13-slim + `apt-get upgrade` (Debian 13 deliberately not: restic 0.18, R-1105) — scanner: critical 16 → 4, fixable criticals and highs in the OS 0; govulncheck 62 → 1 (module-level, `openpgp` not linked). **Hub v0.156.0 deployed** 2026-10-16 19:26 on the operator's yes (Go 1.27.2, alpine 3.24.2: 0 critical, 0 high); Synced/Healthy, reports from all three boxes after the start, the floor reads 0.322.0. **Agent: Go 1.27.2 committed (`37b99d7`), tested, NOT released** — the next vouched agent release carries it. Open: the Samba image (48 fixable high, not rebuilt); docker-compose plugin's own Go (4 high, upstream). Evidence `audits/burndown2-2026-10-16/P4/` **2026-10-16 (evening): the Samba part is DONE in controller v0.323.0** — felhom-samba 1.2.0 on Alpine 3.24.2 (0 critical / 0 high, was 48 fixable high), proven on demo-hp with the same household password (smbclient, nmbd, mDNS, WS-Discovery); on the three test boxes now, on every box with the global floor (R-1105's night check). The controller's Debian 13 OS part: 1 critical / 70 high, none fixable. Left: docker-compose plugin's own Go (4 high, upstream); the agent release; restic's binary is R-1107. | the next vouched agent release | Close when the global floor carries v0.323.0 and the next vouched agent release carries Go 1.27.2 (compose plugin: upstream, recorded) | CC | +| **R-972** | Box system & updates | P2 | **The shipped controller and hub programs are built with Go 1.24.13 and golang.org/x/crypto v0.31.0, and the agent with Go 1.26.0, so dozens of fixed Go security bugs (including html/template escaping fixes) are still in what runs on boxes and the hub.** Go 1.24 has had no security fixes since Go 1.26 was released (Go supports the two newest major releases: https://go.dev/doc/devel/release#policy); the scanners name fixes in Go 1.26.9 and 1.27.2. Also: the controller image's Debian 12 base has 12 fixable critical packages, and the Samba image has 48 fixable high ones. Fix: build all three on a supported, patched Go; update x/crypto; rebuild the base images. Scanner output: `audits/security-review-2026-10-12/scanners/`. | **NARROWED** 2026-10-16 (burndown 2): **controller v0.322.0 released** on Go 1.27.2 + x/crypto v0.58.0, runtime debian:12.13-slim + `apt-get upgrade` (Debian 13 deliberately not: restic 0.18, R-1105) — scanner: critical 16 → 4, fixable criticals and highs in the OS 0; govulncheck 62 → 1 (module-level, `openpgp` not linked). **Hub v0.156.0 deployed** 2026-10-16 19:26 on the operator's yes (Go 1.27.2, alpine 3.24.2: 0 critical, 0 high); Synced/Healthy, reports from all three boxes after the start, the floor reads 0.322.0. **Agent: Go 1.27.2 committed (`37b99d7`), tested, NOT released** — the next vouched agent release carries it. Open: the Samba image (48 fixable high, not rebuilt); docker-compose plugin's own Go (4 high, upstream). Evidence `audits/burndown2-2026-10-16/P4/` **2026-10-16 (evening): the Samba part is DONE in controller v0.323.0** — felhom-samba 1.2.0 on Alpine 3.24.2 (0 critical / 0 high, was 48 fixable high), proven on demo-hp with the same household password (smbclient, nmbd, mDNS, WS-Discovery); on the three test boxes now, on every box with the global floor (R-1105's night check). The controller's Debian 13 OS part: 1 critical / 70 high, none fixable. Left: docker-compose plugin's own Go (4 high, upstream); the agent release; restic's binary is R-1107. **2026-10-17 (evening): agent v0.159.1 (Go 1.27.2) RELEASED and running on demo-hp, Tester 1 and demo-felhom** (signed jobs on the operator's yes); waiting for the operator's vouch — then only the upstream compose plugin is left (close then, recorded). | the next vouched agent release | Close when the global floor carries v0.323.0 and the next vouched agent release carries Go 1.27.2 (compose plugin: upstream, recorded) | CC | | **R-862** | Box system & updates | P3 | **Tester 2 cannot take the config bundle until one by-hand bootstrap is done: its `felhom-os-apply` (agent 0.142.0) predates the bundle mode, and no signed job can write a root file on it.** FOUND 2026-10-04 (R-840 build, Part C): the route reaches every box installed from installer 1.31.0 on, and the demo boxes (bootstrapped by CC); Tester 2 has every root file of agent 0.142.0 (installer 1.30.0) and lacks only the R-858 wrapper fix, which matters only for a Docker step it gets solely from a signed job. CC has no route to Tester 2 (its door admits only the operator's WireGuard peer; `felhom-op` cannot become root). The steps: `runbooks/config-bundle.md` "Tester 2". Then CC signs the bundle and reads it back. | **WAITING-ON-OPERATOR** — the operator said (2026-10-04 ~19:05) he will try through his tunnel **2026-10-07 07:58 (`09` §3 decision 170): the operator believes Tester 2 never set up the off-site backup (no escrow); the laptop may stay off for days; the tester plans to add an HDD.** **2026-10-07 (Part H, read only):** in Tester 2's last 10 notifications (2026-10-05 09:16 → 2026-10-07 06:58) the household got 0 mails, the operator 10 (host_down ×9, expected_dbdump_missed ×1); control: the household channel shows on the other three customers. So the household is not mailed daily while the box is off; no row (`audits/day-2026-10-07/H/`). | the operator's WireGuard tunnel | the operator runs the three bootstrap commands; CC sends the bundle | operator | | **R-78** | Box system & updates | P3 | **`local_api` authority ruling — auto-reconcile vs detect-only** **MIGRATED FROM `ROADMAP.md` 2026-08-22 (R-369) — originally filed 2026-07-25, size M, roadmap state `idea (deferred OUT of R-77 on purpose)`.** Moved verbatim; nothing added or reinterpreted. The roadmap keeps its copy as history, marked moved. **An OWED OPERATOR DECISION, not a defect — moved because an owed ruling hidden among feature ideas is the shape this session exists to remove.** | **OPEN — migrated from ROADMAP 2026-08-22, rank unchanged** | — | R-77 ships detection because the fix is genuinely undecided, and **both directions can lose customer-visible function**. **Direction 1 (today):** `controller.yaml` wins and drift is silent → the 2026-07-25 island migration blinded the whole fleet's control plane for 17.5 h (drive gate, guest-reboot recovery, quiesce/backup all degrade). R-77 makes that loud but does not stop it recurring. **Direction 2 (`bootstrap.json` wins, auto-reconcile on boot):** a guest whose `controller.yaml` is CORRECT and whose `bootstrap.json` is stale — a half-completed re-provision, a hand-repaired guest, a setup-wizard box — gets a **working channel clobbered on the next restart**, fleet-wide and silently, during a routine deploy. That is not obviously better than the bug. Needs a spike: which writer is authoritative per field (endpoint vs fingerprint vs token — `mergeLocalAPI` replaces the whole block, so they cannot be reconciled independently today), whether the agent side should stamp a generation/mtime so 'newer wins' is even expressible, and whether reconcile should require an operator ack. Until then the drift alert plus a manual edit is the supported path. | CC | | **R-468** | Box system & updates | P3 | **[P3-LOW] THE GOLDEN WAIVER — goldens on a cadence, not per release (operator ruling 2026-09-13).** 25 goldens in 26 days in August, almost one per release, because `golden_currency_gate.py` trips on every release by design and the only honest ways past it were a bake or a declared `--no-verify` (thirteen by 2026-09-01, R-404/R-417). **The ruling: bake WEEKLY, and always before any drill or fresh install.** Every release still raises the FLOOR, so both demo boxes keep getting each release in ~20 s; only the golden — which protects a fresh install and nothing else — moves to a cadence. **The mechanism (built 2026-09-13):** `documentation/tests/golden-waiver. **⚠ CORRECTED THE SAME DAY (R-472): between bakes the floor does NOT carry a release — the hub holds any floor above the vouched golden (publish-train rule 1), so releases between bakes reach the demo boxes only by hand-deploy.**yml`, four lines (`issued`, `expires`, `reason`, `register_row: R-468`), read by the gate. While valid, a golden BEHIND the record makes the gate print a loud ADVISORY and exit 0; when it expires the gate is red again until someone bakes or renews. **The 14-day cap is enforced by the gate, not the runbook** — a longer, undated, unparseable, reason-less or row-less waiver is INCONCLUSIVE (exit 2), never 0 and never silently ignored. **It never covers a golden that is UNRECORDED (R-385)** — that is not a cadence choice. **A dated waiver cannot be forgotten; it just expires** — the difference from R-242's original rule, which recurred the day after it was written. Tests: `scripts/test_golden_currency_gate.py` cases 5–15 (E/F/G/H, a 15-day, absent, unparseable, bad-row and empty-reason waiver each 2; the R-421 decoy — a file saying only `expires` — 2). **This is a PRE-CUSTOMER arrangement: the first external install retires it** (delete the file in that commit). Cadence written into `RUNBOOK-manual-build.md` §4.2 and the `felhom.eu` end-of-session checklist. **Does NOT touch R-242's open half (nothing gates the VOUCH).** **2026-10-15 (`09` §3 decision 250, operator):** the cadence is now the RULE, not a waiver — a controller release needs no golden (the floor carries it), the golden is baked every Sunday night when the floor is above it (`runbooks/nightly-rotation.md`), and `golden_currency_gate.py` convicts only a golden MORE THAN 7 DAYS old while a release is ahead (CASE 18–21, red-proved). The waiver file stays as a dated exception for a missed Sunday. | **WATCHING** — the weekly bake is a night-routine step; the first external install still retires the waiver arrangement | — | — | CC | @@ -307,7 +307,7 @@ stopping line that lies. | **R-990** | Business & legal | P4 | **Operator: read `documentation/audits/forum-research-2026-10-12/FELHOM.md` §5 and rule on each recommendation.** READ-ONLY research 2026-10-12: three years (#7943–#11092, 2,908 posts, 195 people, 82 beginner threads) of the public PROHARDVER! topic „Home server / házi szerver építése”, tagged and counted by person (`FINDINGS.md`, method and spot-check in `METHOD.md`; no username or post text in git). Ten ranked recommendations, none acted on; the top three: a „which machine works” page (8 of 33 beginners who name their hardware own only a Synology, a router disk or a Raspberry Pi — the installer cannot run there; the same gap is R-989's first recommendation from a different source, and the shipped-page half is R-991), „you do not choose an operating system — we do” (the biggest beginner question, 20 of 82 threads), and „bring your old office PC” as the cheap entry. Catalog gap (≥ 3 people): a DLNA server (7), an ad/DNS blocker (5), a camera recorder (5). | **WAITING-ON-OPERATOR** — filed 2026-10-12 | the operator's reading | Operator: rule on each of the ten (yes / no / later). If nothing is done: nothing changes; the research ages | operator | | **R-1035** | Business & legal | P4 | **Did the „felhő tárhely” page earn anything? Two dated checks.** **Baseline (reported by the operator from Search Console, 2026-10-14, last 3 months):** 1 click, 48 impressions, CTR 2.1 %, average position **73.6**; „felhő tárhely” 19 impressions, „felhőtárhely” 5, „felhő alapú tárhely” 3. CC measured separately that the phrase „felhő tárhely” appeared **0 times** on the home, FAQ and backup pages — Google was showing the home page for those queries by accident. `/sajat-felho-tarhely` + `/en/own-cloud-storage` published **2026-10-14**; full baseline, price sources and the power provenance in `audits/seo-2026-10-14/BASELINE.md`. **What to measure, both times:** impressions and average position for „felhő tárhely” and „saját felhő tárhely”, plus the new page's own impressions, clicks and position. **A null result at +4 weeks means nothing on its own** — a new page on a site at position 73.6 takes weeks to months to move, and „felhő tárhely” alone is held by Microsoft, Google and the big list sites; the page aims at the two longer phrases. That is why there are two checks and not one, and why neither should be read as a verdict on the copy. **There is no Search Console connector in this project, so CC cannot read these numbers** — the operator reads them, CC records and compares. **Blocked until the operator asks for indexing** (Search Console → URL-ellenőrzés → the URL → „Indexelés kérése”): until that happens, or Google finds the page itself, both checks read zero for a reason that is not the page. **This is a slow channel and does not replace talking to people.** | **WATCHING** | operator: request indexing, then supply the Search Console read | 2026-11-11: first read (+4 weeks); 2026-12-09: second (+8 weeks), compared with the first | operator + CC | -## Process & tooling — 33 rows (P3 9, P4 24) +## Process & tooling — 34 rows (P3 9, P4 25) | ID | Category | Sev | What | State | Blocked on | Next action | Owner | |---|---|---|---|---|---|---|---| @@ -344,6 +344,7 @@ stopping line that lies. | **R-1135** | Process & tooling | P4 | **CI's failure mail links the run at the in-cluster Gitea address** (`http://gitea.gitea-system.svc.cluster.local:3000/...`), which a reader outside the cluster cannot open. Seen in the forced test mail 2026-10-17 (job 1997). | OPEN — filed 2026-10-17 | — | Use the public Gitea URL in the alarm step's link | CC | | **R-1136** | Process & tooling | P4 | **felhom.eu CI runs take ~6.9 minutes and complete, so the 5-minute job timeout R-265 assumes does not hold there** (runs 1996, 1997; `script-tests` alone ~356 s). Found by the R-1129 proof helper 2026-10-17. | OPEN — filed 2026-10-17 | — | Measure the runner's real timeout; correct R-265's assumption or speed up `script-tests` | CC | | **R-1137** | Process & tooling | P4 | **A CI rerun may overwrite a failed job's log** — failure mails arrived for controller commits `c02c9d83` and `29a3ce8c` while their jobs (1944, 1959) now read green. UNPROVEN (a reading, 2026-10-17). | OPEN — filed 2026-10-17 | — | Confirm with one deliberate red + rerun on a scratch repo; if true, the CI check must read the first attempt | CC | +| **R-1143** | Process & tooling | P4 | **The controller `build.sh` pushes `:latest` with EVERY build, release candidates included**, and the legacy `controller/docker-compose.yml` follows `:latest`. Nothing current pulls it (the update button takes the highest x.y.z tag), but a legacy-shaped box would take an rc. Seen 2026-10-17 (rc.1–rc.5 each moved `:latest`). | OPEN — filed 2026-10-17 | — | Push `:latest` only for a release tag, or not at all; pin it in `test_build_push_once.sh` | CC |