admin
5a5486104a
gates / gates (push) Successful in 8m49s
Closed R-971, R-954, R-1124, R-1012 (174 -> 175 rows); opened R-1139..R-1143; R-895 READY with the design and hub slice 1; R-972 waits on the vouch. [DESIGN] "The catalog allow-list" in 02; the catalog trust line in 01; FELHOM_THINCHECK and unit-forget in 03. Evidence audits/agent-allowlist-rotation-2026-10-17/. Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
153 lines
12 KiB
Markdown
153 lines
12 KiB
Markdown
# REPORT — agent 0.159.1, the catalog allow-list (D2 = A), break-glass rotation (D1 = A), off-site history check (2026-10-17)
|
||
|
||
Written in ASD-STE100 Simplified Technical English. Evidence: `documentation/audits/agent-allowlist-rotation-2026-10-17/`
|
||
(P1 agent, P2 allow-list, P3 rotation, P4 off-site design, P5 release). Architecture read: `03-host-agent.md`,
|
||
`02-controller-module-map.md`, `01-topology-and-trust.md`, `07-backup-architecture.md`, `06-offsite-connectivity.md`,
|
||
`09-update-architecture.md` §3 decision 250, `runbooks/break-glass.md`.
|
||
|
||
## For the operator — plain words first
|
||
|
||
**Decisions I took myself:** none.
|
||
|
||
1. **Agent 0.159.1** (0.159.0 is superseded — do not vouch it). New Go. A drive that was decommissioned and then
|
||
removed is now forgotten: its host mount unit goes away within 20 seconds, so the host no longer waits for it.
|
||
Before a whole-box backup, and when one fails, the agent checks the thin pool for an orphan snapshot and says so in
|
||
the failure mail; it never deletes one (the repair is a runbook for you). All of it proven on demo-hp. **It runs on demo-hp, Tester 1 and
|
||
demo-felhom. Your vouch in the hub is still open** (the hub names 0.158.1 at 17:58).
|
||
2. **The app list cannot give an app dangerous rights any more** (controller 0.328.0, the floor for every box since
|
||
17:22 local). Before the box writes any app's settings it checks for dangerous rights. Only homepage's read-only
|
||
Docker socket is allowed, and only for homepage's own image and start. A refused change writes nothing, the app
|
||
keeps running, you get a mail, and the app page says one line. Proven on demo-hp 9201 with a test catalog. Three
|
||
security reviews found eight gaps in the first versions; all eight were fixed before the release.
|
||
3. **New emergency root passwords** on demo-hp, Tester 1 and demo-felhom (random, your answer). Each new password
|
||
works; each old one is refused. Read them in the hub. Tester 2 is off: it gets its new password when it comes back.
|
||
4. **Off-site history:** the hub can count a box's off-site copies itself, with the login it already has. Hub 0.157.0
|
||
(deployed on your yes) logs that count every morning at 07:10. The alarm on it is the next step.
|
||
|
||
**What needs you:**
|
||
- **Vouch agent 0.159.1** in the hub — all three test hosts already run it; without the vouch a NEW box installs 0.158.1.
|
||
- **Decide R-1140:** keep the hub's access logs for 90 days. Pick A (a small hub-only log, recommended) or B (longer
|
||
logs for every container on DooPlex). If you do nothing: the next „did anyone use the key?" question has only about
|
||
13 hours of logs again.
|
||
|
||
## 1. Agent (Part 1)
|
||
|
||
- **Content:** Go 1.27.2 + x/crypto v0.58.0 (R-972); R-1124 `ForgetRetiredDrives` + root verb `felhom-priv-apply
|
||
unit-forget`; R-1012 read-only root helper `felhom-thin-check` + `internal/thincheck` + `BackupRunner.SetThinCheck`;
|
||
0.159.1: the signed decommission prunes the guest-bind record (found by 0.159.0's own proof). Releases: 0.159.0
|
||
(`3cfd0a25…`, bundle `40663c4f…`), step bundle `0.159.0-step1` (`1d65edb3…`), **0.159.1** (`de8f6ad9…aff9`, bundle
|
||
`2af3d27d…7cd6`).
|
||
- **demo-hp proof (normal delivery: signed `agent_update` → step bundle → release bundle, 75/75 capabilities):**
|
||
- R-1124: a loop-backed test drive on `/mnt/hdd_1` enrolled through the agent's local API (the controller's
|
||
calls); the customer decommission refused it (a loop device reads as „system" — correct), so the operator-signed
|
||
decommission was used; while present the unit stayed; after the unplug the agent removed the unit and its wants
|
||
link in 8 s and `GET /disks` dropped it (5 → 4 rows). On 0.159.1 a second signed decommission pruned the bind
|
||
record; the 20-second warning stopped (0 lines after, the reconcile tick still running — 3 lines, the control).
|
||
- R-1012: on a scratch thin pool (loop file on `/mnt/hdd_1`, never `pve/data`) the helper listed a made orphan
|
||
(device 7) and refused a metadata snapshot held by someone else; on `pve/data` it read ids 1, 2 = LVM's and
|
||
released. A household-press whole-guest backup of 9201 ran the check first (the sudo line) and finished green
|
||
(6.77 GB). Note on the fence „never write into pve/data": reading a live pool needs a metadata snapshot
|
||
(`reserve_metadata_snap` / `release_metadata_snap`); it changes no data and no mapping. This is what the product
|
||
now does before every snapshot backup — if you want that stopped, say so (one line in the agent).
|
||
- Scratch pool, loop devices, test image files, the test drive's folders: removed. One residue kept: the drive's
|
||
`decommissioned` intent line on demo-hp (harmless by design).
|
||
- **Vouch:** the operator said yes (STOP, ~17:00). Delivered on that yes by signed jobs: Tester 1 (agent 17:39:31, bundle
|
||
17:53:42, 75/75) and demo-felhom (agent 17:40:51, bundle 17:55:00, 75/75), each through the step bundle; the hub has
|
||
reports from both after. **The vouch itself (Configs → Day-0 artifacts) is not done yet** — new installs still take
|
||
0.158.1 until it is. Tester 2: off; it takes 0.159.1 when it is back (signed jobs, as above).
|
||
|
||
## 2. The catalog allow-list (Part 2, R-971 — closed)
|
||
|
||
- **Survey** (`P2/survey.md`): 123 files (65 templates, 58 ladder steps). The only class member: homepage's read-only
|
||
Docker socket. That is the whole list.
|
||
- **Built:** `felhom-controller internal/privpolicy` on the RENDERED compose; wired into every compose writer and env
|
||
decision point; structural guard `TestAllowList_IsWiredIntoEveryComposeWriter` (AST). Refusal → nothing written,
|
||
operator event `app_definition_refused` (hub ≥ 0.157.0), app-page line, plain message on install/settings/restore.
|
||
The sync never copies a template named like a controller infra stack. Catalog mirror: `check-privileges.py` (stdlib,
|
||
in `--fast`, decoys; the reader equals PyYAML on all 123 files); the two lists are byte-identical, pinned from both
|
||
sides (boring choice: two copies + one equality check, no network in CI).
|
||
- **Security reviews (three rounds, eight findings, all fixed and red-proved before the release):** the entry bound to
|
||
its image, entrypoint/command and exact healthcheck; symlinks out of a root (nearest existing ancestor, fail closed);
|
||
missed classes (cgroup_parent, runtime, isolation, gpus, device reservations, provider, lifecycle hooks); roots only
|
||
under `/mnt/felhom-drives/` or the system data path; any `extends` refused; no variable in a bind's options; YAML
|
||
anchors refused by the catalog reader; under `${HDD_PATH}` only `appdata/` (not `backups/`).
|
||
- **Live finding on 9201 with rc.4:** every not-installed app was refused (placeholder roots do not exist on the box,
|
||
so the root itself could not be resolved). Fixed in rc.5 (roots resolved through their nearest existing ancestor; no
|
||
symlink resolve for the placeholder check), red-proved. Its 30 events were dropped by the old hub — no false mail.
|
||
- **9201 proof (rc.5, hub 0.157.0):** a drill catalog = live + `privileged: true` on privatebin → refused; the compose
|
||
sha `79e7877d…` unchanged; the same privatebin container (`494270e4…`, started 14:34:24Z) still running; the event at
|
||
the hub at 17:10:34 and an operator mail; the page line on privatebin, not on opengist (control). Restored: live
|
||
catalog, controller.yaml byte-identical, the drill mirror removed, the drill repo back to the live tree.
|
||
- **Every installed app on the three test boxes still passes:** 0 refusals after 0.328.0 started (P5/box-walk.txt).
|
||
|
||
## 3. Break-glass rotation (Part 3, R-954 — closed)
|
||
|
||
| host | how | proof |
|
||
|---|---|---|
|
||
| demo-hp | installer step 4b ALONE (`installer-v1.33.0`'s own function) | new value → PVE API 200; old → 401; values differ |
|
||
| Tester 1 | the same | the same |
|
||
| demo-felhom | the same, on your answer (random) | the same; hub `set_at` now 2026-10-17T15:10:04Z |
|
||
| Tester 2 | offline since 2026-10-04 | R-1139 |
|
||
|
||
- Why not the full installer re-run (the brief's `--rotate-recovery`): it also reinstalls the vouched agent and the root
|
||
files — on demo-hp it would have moved the candidate agent back. R-1141 asks for a standalone mode.
|
||
- Found and fixed: the hub kept the FIRST vault's `set_at` (demo-hp/Tester 1 still read their old dates, rotated
|
||
before the fix; demo-felhom shows the new one) — hub v0.157.0.
|
||
- The passwords were read with the global-key curl (P3/`cred.py`, prints only length and time): that path writes NO
|
||
customer-visible „revealed" line. Nothing was printed or filed.
|
||
- **Log retention:** ingress logs cover ≈ 13 h (k3s 10 MiB × 5). Not built: it needs a DooPlex or hub-storage change →
|
||
R-1140 (cost in the row).
|
||
|
||
## 4. Off-site history (Part 4, R-895 — open, READY)
|
||
|
||
- **Measurement:** the login the hub already holds (the sealed sub-account password, write-capable) lists the folder;
|
||
upload times are shown (2026-09-01 evidence on demo-hp's own sub-account). A fresh listing from DooPlex was NOT
|
||
made: tried the CC store (no Storage Box credential), the box's key (restic REST only, names and sizes, and it is the
|
||
box's key), the hub (no endpoint); reading the sealed password out of the hub was not done.
|
||
- **Design:** `P4/design-R-895.md` — before/after counts from the hub's own listing, files no box run explains,
|
||
snapshots gone outside a window; a read-only sub-account as the stricter alternative.
|
||
- **Built (slice 1, ~1 h, no new credential):** hub v0.157.0 lists `snapshots/` in the 07:10 key audit and logs the
|
||
count, newest and oldest time. Deployed on your yes (17:03, Synced/Healthy, reports from demo-hp and Tester 1, floor
|
||
read 0.328.0).
|
||
|
||
## 5. Versions, commits, fixtures
|
||
|
||
- **felhom-agent:** `e15d1d3` (code), `v0.159.0` tag, `4a1c07b`-line (prune), `v0.159.1` tag, CHANGELOG `…`. CI green
|
||
(job 2025).
|
||
- **felhom-controller:** `6d90374`, `24d937d`, `36c4aa4`, `09ac87a`, `99f9117` → **v0.328.0** (rc.1–rc.5 tested on
|
||
9201), MinAgent 0.158.1. Global floor 0.328.0 / 0.158.1 at ~17:22 local; per-box floors set and cleared. CI green
|
||
(job 2032). **i18n fixture:** new `app_settings_definition_refused`; every other fixture byte-identical.
|
||
- **app-catalog-felhom.eu:** `d5a443d` (gate) + three follow-ups; CI green (job 2033).
|
||
- **felhom.eu:** `8f97b800` (hub v0.157.0 code, installer uninstall line, runbook), `d968f849` (hub deploy); docs and
|
||
register in the final commit.
|
||
|
||
## 6. Rows
|
||
|
||
Before **174**, after **175**. Closed (4): R-971, R-954, R-1124, R-1012. Opened (5): R-1139, R-1140, R-1141, R-1142,
|
||
R-1143. Updated: R-895 (READY with the design), R-972.
|
||
|
||
## 7. Observations
|
||
|
||
1. Agent v0.159.0's signed decommission left a guest-bind record → a warning every 20 s. NOT-A-FINDING: fixed in the
|
||
session (agent v0.159.1, the size rule), red-proved.
|
||
2. Hub kept the first vault's `set_at`. NOT-A-FINDING: fixed in the session (hub v0.157.0, the size rule), red-proved.
|
||
3. A full installer re-run is the only `--rotate-recovery` path. FILED: R-1141.
|
||
4. Ingress access logs ≈ 13 h. FILED: R-1140.
|
||
5. One app can bind another app's `appdata/<name>`. FILED: R-1142.
|
||
6. Controller `build.sh` pushes `:latest` on every build, rc included. FILED: R-1143.
|
||
7. The first allow-list rc refused every not-installed app on the box. NOT-A-FINDING: found by this session's own live
|
||
proof on 9201 and fixed before the release (rc.5, red-proved).
|
||
8. homepage keeps a read-only Docker socket on the list; a read-only mount does not limit the Docker API.
|
||
NOT-A-FINDING: already R-782's subject (homepage's exposure); the list keeps today's behaviour, it does not bless it.
|
||
9. Tester 1's ISO-era `--rotate-recovery` warning text („the old one now fails at the PVE web GUI") is printed by the
|
||
step itself. NOT-A-FINDING: true and intended.
|
||
|
||
## 8. Teardown (three layers)
|
||
|
||
- **Machine:** demo-hp — scratch VG/pool/loop removed, test image files removed, test drive folders removed; 9201 back
|
||
on the live catalog (controller.yaml byte-identical), drill mirror removed. Tester 1, demo-felhom: only the releases
|
||
and the password rotation.
|
||
- **Host:** demo-hp, Tester 1, demo-felhom run controller 0.328.0; demo-hp, Tester 1 and demo-felhom run agent 0.159.1 (bundle 0.159.1).
|
||
- **Hub:** hub 0.157.0; global floor 0.328.0; no per-box floor; signed jobs consumed. Instruction files: none edited.
|
||
System poster: no fact changed (none of the three).
|