felhom.eu/REPORT-day-2026-10-18.md

REPORT — day 2026-10-18

In plain words

• The hub update is live. Its own access log works and writes no secret (tested live with a fake token). You can now vouch an agent without a new golden. The off-site snapshot check gives its first verdict on 20 October.
• A fresh reviewer read last night's code. One medium finding: a stale host key could have left the emergency password unknown. Today's installer change (save in the hub first, then set) already closes it. Three low findings: two fixed today, one is a known design limit, now written in its row.
• „Retire anyway": when the host says no, the drive's apps start again. Where the host's disk list already says no, nothing is stopped at all. Controller 0.330.0 is the floor for every box.
• Agent 0.159.2 runs on demo-hp, Tester 1 and demo-felhom (your yes). A whole-box backup ran with no pool check. The vouch is yours (STATUS).
• Installer 1.34.0 is published (your yes): the emergency password is saved in the hub first, then set, then checked with a login. Proven on demo-hp. A fresh install with it has not run yet (row R-1148).
• The calculator shows Immich's bigger figure (your yes). Immich alone on an 8 GB machine now reads „does not fit".
• The old-restic check has not run. demo-hp's clean-up comes only after tonight's backup (2026-10-19 about 04:16 Budapest). The STATUS item stays open.

1. Hub 0.158.0 deploy

ArgoCD: only Deployment/hub was OutOfSync; synced that resource alone → Synced/Healthy, image felhom-hub:0.158.0, startup log felhom-hub 0.158.0 starting + access log: /data/access-log …, kept 90 days. Floor read back from the hub DB with its WAL: 0.329.0 / 0.158.1 at deploy time (audits/day-2026-10-18/P1/hub-deploy-readback.txt). Reports from all three boxes in the access log within two minutes (sources .101, .149, .155).

• Access log (R-1140, closed): today's file access-2026-10-18.log, mode 0600. A fresh 48-hex fake token sent in a query, a /bind/<token> path, a session= query, a Bearer header, a Cookie and a POST body: found 0 times; control (the same file with the token appended) 1. Secret shapes (Bearer value, a 32+ character run, password=, token=, session=): 0; control 1. /bind/<token> is written /bind/[redacted] (P1/access-log-live-test.txt).
• Agent-only vouch: read-only — /configuration loads with CC's token (Day-0 section, golden 0.329.0, agent 0.159.1); the save (/configuration/artifacts) answers 403 to the token, by design. No manifest change by CC.
• Off-site alarm (R-895, NARROWED): no fixture route exists in the hub for it; the unit tests are the proof so far. Tomorrow's 07:10 run is the baseline (no verdict by design); the first verdict is 2026-10-20 07:10 — a DUE-CHECKS row.

2. The independent review

A fresh subagent with no context from the night reviewed controller v0.328.0→v0.329.0, hub 0.157.0→0.158.0, the agent's orphan-check change and the installer's --rotate-recovery-only. Its report was written to a file and read back (audits/day-2026-10-18/P2/independent-review.md, 54 lines). Verdicts (P2/verdicts.md):

#	Finding	Verdict
F1 medium	rotate-only: unauthenticated healthz, then chpasswd, then a vault that a stale key fails → an unvaulted password	REAL in the night's code → fixed by R-1145 (vault first); harness R1145-e, fails against HEAD
F2 low–medium	offsite_snapshots_unexplained is explained by the box's own reported run time	REAL, by design (decision 68) → written into R-895; the custody alarm _vanished is not affected
F3 low	„anyway" stops apps before the agent is asked	REAL, was R-1144 → fixed (controller 0.330.0)
F4 low	host key in curl's argv during the vault PUT	REAL, pre-existing → fixed (header file)

3. R-1144

decommissionAnyway: reads the agent's disk list first (a role other than user-data, or the backup target → the plain refusal, nothing stopped); after a refusal past the stops (policy refusal or a failed agent call) starts exactly the apps that were running; a drive the agent let go keeps its apps stopped. Household text HU + EN. Seam decomStacks, so no test reaches docker. Three tests, two red-proofs (P3/redproof-r1144.txt). Not pressed live: no test box has a drive that is safe to retire.

4. Agent 0.159.2

Released with release-agent.sh (code pushed first, CHANGELOG after — the release-complete hook refuses the other order): sha256 4d98868e922d6c10665a6c0da9323e8627ea9c589094e81d27d741e625d9d8f9, bundle 477704ea2980948433ffc34bb59f008cc11410fd9683d386bfd3e2f085d55bc9 (read back from the registry). demo-hp proof: a whole-guest backup of 9201 through the dashboard (/api/guest-backup/trigger, snapshot mode, completed in 7.5 min): 0 felhom-thin-check sudo lines; control — the night's backup on 0.159.1 at 04:33 had 1 (P4/demo-hp-backup-journal.txt). The failure path stays covered by the agent's fixture tests. Operator: yes → Tester 1 and demo-felhom by signed jobs (agent_update + agent_config_update): both signed op COMPLETED, capability probe 75/75, update committed 0.159.2, and a host report on 0.159.2 from each (P4/). Not vouched — STATUS.

5. The password order (R-1145, R-1146)

Step 4b (install and rotate-only): generate → PUT to the vault (200 = the hub's DB write is done) → chpasswd (one retry) → a PVE API login with the new value. The states that can remain, and why each is safe or not:

State	When	Safe?
nothing changed	the vault refuses or is unreachable (a stale key included)	safe — root and the vault keep their old, matching pair
root = vault = new, PVE login 200	the normal end	safe
vault = new, root = previous	chpasswd fails twice (or the run is killed between vault and set)	not fully safe: the vaulted value does not open the console; root keeps its previous password (the shadow line is fingerprinted before/after to prove it). The host key cannot read the vault back (by design), so the previous value cannot be re-vaulted; the log says the state in words, and the next run fixes it (it vaults first again).
root = vault = new, PVE login refused	pveproxy trouble	safe for the shell; the run dies and says the web login refused

The old order's dangerous state (root = new, vault = old) is no longer reachable. The hub cannot tell the installer its „last vaulted" time (no host-readable route); the installer trusts the PUT's 200, and CC checked the time moved in the live proof. Live on demo-hp: vault set_at 2026-07-21 → 2026-10-18T10:43:33Z, the new value gets a PVE ticket (200), the old one 401 (P5/demo-hp-rotation.txt). Harness: 52 passed / 0 failed (was 45/2); R1145-a..e red-proofed against HEAD. Installer 1.34.0 published on the operator's yes: tag installer-v1.34.0, both webpage pins, ArgoCD synced the webpage Deployment only; the served file's sha256 = the tag's (df722ee4…). R-1146: SC2066 and GL8-F6 fixed, the harness is gate hostinstall-harness with two decoys.

6. Calculator (R-1147)

Before/after (audits/day-2026-10-18/P6-before-after.md): Immich peak 3,290 → 3,983 MB (typical unchanged 1,557). The one changed verdict: Immich alone on 8 GB: fits, but tight → does not fit. The family starter set is unchanged. Operator: yes → published (cache-bust v5); live sizing-data.json?v=5 serves 3,983. SIZING.md's table and the pinned estimator tests moved with it. R-1036 closed (D14 = no).

7. The old-restic check

Not run — the prune has not happened yet. The hub allows demo-hp a clean-up window 6 days 12 hours after the last one (2026-10-12 02:16:53 UTC → from 2026-10-18 14:16:53 UTC). But the box asks for a window only after its own nightly off-site run, and that runs at 04:15 CEST. So the first real 0.19.1 prune comes 2026-10-19 at about 02:16 UTC (04:16 Budapest). Read at 14:33 UTC: no [offbox] line since the 11:09 restart, the scheduler's next run 2026-10-19 04:15 CEST, the hub's last window still id 9 (P8/cleanup-window-not-yet.txt, the hub DB read with its WAL). CC did not grant a window by hand (that is the operator's). The STATUS item stays open, with the corrected time; the check script is ready (rx.sh + a wrapper that lists and restores the newest, the oldest and the newest 0.14-made snapshot with both restics and compares tree hashes).

8. Versions, commits, fixtures

• Hub 0.158.0 deployed (felhom.eu 4c319dde).
• Controller 0.330.0 — 6c6bd87; floor 0.330.0 / MinAgent 0.158.1 at 11:10:59 UTC (303 floor_set, read back with the WAL); per-box floors set and cleared; box walk green on the three test boxes (0 refusals, all healthy, login and files 200, https-only OK).
• Agent 0.159.2 — 65c1994 (code), tag v0.159.2, CHANGELOG e25454c.
• Installer 1.34.0 — felhom.eu 88aecf55 (vault-first + harness), 624bcdc3 (1.34.0, tagged), 3332eb09 (pins).
• Estimator — 37cb83c1. Fixtures: none changed (the new texts travel in JSON).

9. Register — 173 before, 168 after; opened 1, closed 6

Closed: R-1140, R-1144, R-1145, R-1146, R-1147, R-1036. Opened: R-1148. Narrowed: R-895 (dated check 2026-10-20).

10. Observations

1. Installer 1.34.0's default path has not run in a fresh install. FILED: R-1148.
2. The review's F2 (box-reported run times explain new snapshots). NOT-A-FINDING as a new row: a known design limit, recorded in R-895.
3. release-agent.sh vs the release-complete hook: the CHANGELOG must follow the release. NOT-A-FINDING: the documented order (memory and skill); I tried the other order first and the hook refused it, as designed.
4. The estimator's new Immich figure is memory.current (with cache), not the used measure it normally takes. NOT-A-FINDING: on the safe side, said in SIZING.md, and approved by the operator with the table in front of them.
5. A safety check refused a sh -c with an unguarded rm "$H". NOT-A-FINDING: rewritten with ${H:?} in a helper.

Teardown — three layers

• Machines: demo-hp 9201 — one whole-guest backup taken (kept, like any backup); no app installed or removed today. demo-hp host — the installer copy shredded after the rotation; root password rotated on purpose (new value in the vault). Tester 1, demo-felhom — agent 0.159.2 and controller 0.330.0 through their normal paths; nothing else.
• Hub: hub 0.158.0; global floor 0.330.0 / 0.158.1; no per-box floor; signed jobs consumed. No vouch by CC.
• DooPlex: builds only; ArgoCD synced two Deployments (hub, webpage), each alone.

Instruction files: none edited. System poster: none of the three (the poster shows backup retentions; the hub's access log is not on it). Architecture documents read: 02, 03, 05, 07, 09 §3 decision 250; break-glass.md.