Day 2026-10-18: report, CONTEXT; old-restic check not run yet (demo-hp's clean-up window opens after the 2026-10-19 04:15 CEST run), STATUS time corrected

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS

CONTEXT.md

@@ -16,6 +16,11 @@
 > and holds nothing of its own; this file does hold its own content, namely the standing rulings below.
 
 
+> **2026-10-18 (day) — hub 0.158.0 DEPLOYED (access log live, agent-only vouch, R-895 first verdict 2026-10-20);
+> controller 0.330.0 is the floor (R-1144); agent 0.159.2 released and on the three test hosts, NOT vouched; installer
+> 1.34.0 published (vault-first step 4b, R-1145; fresh install unproven, R-1148); estimator Immich peak 3,983 MB
+> (R-1147).** Independent review of the night: `audits/day-2026-10-18/P2/`. `REPORT-day-2026-10-18.md`.
+
 > **2026-10-18 (night) — controller 0.329.0 is the floor (MinAgent 0.158.1); golden 0.329.0 baked, NOT vouched; hub 0.158.0
 > built, NOT deployed.** Each app's own `appdata/<folder>` (R-1142), empty allow-list (R-981), drive retirement asks the
 > agent first (R-1063), plain no-durable-id refusal (R-1069), `:latest` for releases only (R-1143), Immich 3072M/2048M

REPORT-day-2026-10-18.md

@@ -0,0 +1,139 @@
+# REPORT — day 2026-10-18
+
+## In plain words
+
+- **The hub update is live.** Its own access log works and writes no secret (tested live with a fake token). You can
+  now vouch an agent without a new golden. The off-site snapshot check gives its first verdict on 20 October.
+- **A fresh reviewer read last night's code.** One medium finding: a stale host key could have left the emergency
+  password unknown. Today's installer change (save in the hub first, then set) already closes it. Three low findings:
+  two fixed today, one is a known design limit, now written in its row.
+- **„Retire anyway":** when the host says no, the drive's apps start again. Where the host's disk list already says no,
+  nothing is stopped at all. Controller 0.330.0 is the floor for every box.
+- **Agent 0.159.2** runs on demo-hp, Tester 1 and demo-felhom (your yes). A whole-box backup ran with no pool check.
+  **The vouch is yours** (STATUS).
+- **Installer 1.34.0 is published** (your yes): the emergency password is saved in the hub first, then set, then
+  checked with a login. Proven on demo-hp. A fresh install with it has not run yet (row R-1148).
+- **The calculator** shows Immich's bigger figure (your yes). Immich alone on an 8 GB machine now reads „does not fit".
+- **The old-restic check has not run.** demo-hp's clean-up comes only after tonight's backup (2026-10-19 about 04:16
+  Budapest). The STATUS item stays open.
+
+## 1. Hub 0.158.0 deploy
+
+ArgoCD: only `Deployment/hub` was OutOfSync; synced that resource alone → Synced/Healthy, image `felhom-hub:0.158.0`,
+startup log `felhom-hub 0.158.0 starting` + `access log: /data/access-log …, kept 90 days`. Floor read back from the hub
+DB with its WAL: 0.329.0 / 0.158.1 at deploy time (`audits/day-2026-10-18/P1/hub-deploy-readback.txt`). Reports from all
+three boxes in the access log within two minutes (sources .101, .149, .155).
+- **Access log (R-1140, closed):** today's file `access-2026-10-18.log`, mode 0600. A fresh 48-hex fake token sent in a
+  query, a `/bind/<token>` path, a `session=` query, a Bearer header, a Cookie and a POST body: found 0 times; control
+  (the same file with the token appended) 1. Secret shapes (Bearer value, a 32+ character run, `password=`, `token=`,
+  `session=`): 0; control 1. `/bind/<token>` is written `/bind/[redacted]` (`P1/access-log-live-test.txt`).
+- **Agent-only vouch:** read-only — `/configuration` loads with CC's token (Day-0 section, golden 0.329.0, agent 0.159.1);
+  the save (`/configuration/artifacts`) answers 403 to the token, by design. No manifest change by CC.
+- **Off-site alarm (R-895, NARROWED):** no fixture route exists in the hub for it; the unit tests are the proof so far.
+  Tomorrow's 07:10 run is the baseline (no verdict by design); the first verdict is 2026-10-20 07:10 — a DUE-CHECKS row.
+
+## 2. The independent review
+
+A fresh subagent with no context from the night reviewed controller v0.328.0→v0.329.0, hub 0.157.0→0.158.0, the agent's
+orphan-check change and the installer's `--rotate-recovery-only`. Its report was written to a file and read back
+(`audits/day-2026-10-18/P2/independent-review.md`, 54 lines). Verdicts (`P2/verdicts.md`):
+
+| # | Finding | Verdict |
+|---|---|---|
+| F1 medium | rotate-only: unauthenticated healthz, then chpasswd, then a vault that a stale key fails → an unvaulted password | REAL in the night's code → **fixed** by R-1145 (vault first); harness `R1145-e`, fails against HEAD |
+| F2 low–medium | `offsite_snapshots_unexplained` is explained by the box's own reported run time | REAL, by design (decision 68) → written into R-895; the custody alarm `_vanished` is not affected |
+| F3 low | „anyway" stops apps before the agent is asked | REAL, was R-1144 → **fixed** (controller 0.330.0) |
+| F4 low | host key in curl's argv during the vault PUT | REAL, pre-existing → **fixed** (header file) |
+
+## 3. R-1144
+
+`decommissionAnyway`: reads the agent's disk list first (a role other than user-data, or the backup target → the plain
+refusal, nothing stopped); after a refusal past the stops (policy refusal or a failed agent call) starts exactly the apps
+that were running; a drive the agent let go keeps its apps stopped. Household text HU + EN. Seam `decomStacks`, so no
+test reaches docker. Three tests, two red-proofs (`P3/redproof-r1144.txt`). Not pressed live: no test box has a drive
+that is safe to retire.
+
+## 4. Agent 0.159.2
+
+Released with `release-agent.sh` (code pushed first, CHANGELOG after — the `release-complete` hook refuses the other
+order): sha256 `4d98868e922d6c10665a6c0da9323e8627ea9c589094e81d27d741e625d9d8f9`, bundle
+`477704ea2980948433ffc34bb59f008cc11410fd9683d386bfd3e2f085d55bc9` (read back from the registry). **demo-hp proof:** a
+whole-guest backup of 9201 through the dashboard (`/api/guest-backup/trigger`, snapshot mode, completed in 7.5 min):
+**0** `felhom-thin-check` sudo lines; control — the night's backup on 0.159.1 at 04:33 had **1**
+(`P4/demo-hp-backup-journal.txt`). The failure path stays covered by the agent's fixture tests. **Operator: yes** →
+Tester 1 and demo-felhom by signed jobs (agent_update + agent_config_update): both `signed op COMPLETED`, capability
+probe 75/75, `update committed` 0.159.2, and a host report on 0.159.2 from each (`P4/`). **Not vouched** — STATUS.
+
+## 5. The password order (R-1145, R-1146)
+
+Step 4b (install and rotate-only): generate → PUT to the vault (200 = the hub's DB write is done) → chpasswd (one retry)
+→ a PVE API login with the new value. **The states that can remain, and why each is safe or not:**
+
+| State | When | Safe? |
+|---|---|---|
+| nothing changed | the vault refuses or is unreachable (a stale key included) | safe — root and the vault keep their old, matching pair |
+| root = vault = new, PVE login 200 | the normal end | safe |
+| vault = new, root = previous | chpasswd fails twice (or the run is killed between vault and set) | **not fully safe:** the vaulted value does not open the console; root keeps its previous password (the shadow line is fingerprinted before/after to prove it). The host key cannot read the vault back (by design), so the previous value cannot be re-vaulted; the log says the state in words, and the next run fixes it (it vaults first again). |
+| root = vault = new, PVE login refused | pveproxy trouble | safe for the shell; the run dies and says the web login refused |
+
+The old order's dangerous state (root = new, vault = old) is no longer reachable. The hub cannot tell the installer its
+„last vaulted" time (no host-readable route); the installer trusts the PUT's 200, and CC checked the time moved in the
+live proof. Live on demo-hp: vault `set_at` 2026-07-21 → 2026-10-18T10:43:33Z, the new value gets a PVE ticket (200),
+the old one 401 (`P5/demo-hp-rotation.txt`). Harness: 52 passed / 0 failed (was 45/2); R1145-a..e red-proofed against
+HEAD. Installer **1.34.0 published** on the operator's yes: tag `installer-v1.34.0`, both webpage pins, ArgoCD synced
+the webpage Deployment only; the served file's sha256 = the tag's (`df722ee4…`). R-1146: SC2066 and GL8-F6 fixed, the
+harness is gate `hostinstall-harness` with two decoys.
+
+## 6. Calculator (R-1147)
+
+Before/after (`audits/day-2026-10-18/P6-before-after.md`): Immich peak 3,290 → 3,983 MB (typical unchanged 1,557). The
+one changed verdict: **Immich alone on 8 GB: fits, but tight → does not fit**. The family starter set is unchanged.
+**Operator: yes** → published (cache-bust v5); live `sizing-data.json?v=5` serves 3,983. SIZING.md's table and the
+pinned estimator tests moved with it. R-1036 closed (D14 = no).
+
+## 7. The old-restic check
+
+**Not run — the prune has not happened yet.** The hub allows demo-hp a clean-up window 6 days 12 hours after the last
+one (2026-10-12 02:16:53 UTC → from 2026-10-18 14:16:53 UTC). But the box asks for a window only after its own nightly
+off-site run, and that runs at 04:15 CEST. So the first real 0.19.1 prune comes **2026-10-19 at about 02:16 UTC**
+(04:16 Budapest). Read at 14:33 UTC: no `[offbox]` line since the 11:09 restart, the scheduler's next run 2026-10-19
+04:15 CEST, the hub's last window still id 9 (`P8/cleanup-window-not-yet.txt`, the hub DB read with its WAL). CC did not
+grant a window by hand (that is the operator's). The STATUS item stays open, with the corrected time; the check script
+is ready (`rx.sh` + a wrapper that lists and restores the newest, the oldest and the newest 0.14-made snapshot with both
+restics and compares tree hashes).
+
+## 8. Versions, commits, fixtures
+
+- Hub **0.158.0** deployed (felhom.eu `4c319dde`).
+- Controller **0.330.0** — `6c6bd87`; floor 0.330.0 / MinAgent 0.158.1 at 11:10:59 UTC (303 `floor_set`, read back with
+  the WAL); per-box floors set and cleared; box walk green on the three test boxes (0 refusals, all healthy, login and
+  files 200, https-only OK).
+- Agent **0.159.2** — `65c1994` (code), tag `v0.159.2`, CHANGELOG `e25454c`.
+- Installer **1.34.0** — felhom.eu `88aecf55` (vault-first + harness), `624bcdc3` (1.34.0, tagged), `3332eb09` (pins).
+- Estimator — `37cb83c1`. Fixtures: none changed (the new texts travel in JSON).
+
+## 9. Register — 173 before, 168 after; opened 1, closed 6
+
+Closed: R-1140, R-1144, R-1145, R-1146, R-1147, R-1036. Opened: R-1148. Narrowed: R-895 (dated check 2026-10-20).
+
+## 10. Observations
+
+1. Installer 1.34.0's default path has not run in a fresh install. FILED: R-1148.
+2. The review's F2 (box-reported run times explain new snapshots). NOT-A-FINDING as a new row: a known design limit,
+   recorded in R-895.
+3. `release-agent.sh` vs the `release-complete` hook: the CHANGELOG must follow the release. NOT-A-FINDING: the documented
+   order (memory and skill); I tried the other order first and the hook refused it, as designed.
+4. The estimator's new Immich figure is `memory.current` (with cache), not the `used` measure it normally takes.
+   NOT-A-FINDING: on the safe side, said in SIZING.md, and approved by the operator with the table in front of them.
+5. A safety check refused a `sh -c` with an unguarded `rm "$H"`. NOT-A-FINDING: rewritten with `${H:?}` in a helper.
+
+## Teardown — three layers
+
+- **Machines:** demo-hp 9201 — one whole-guest backup taken (kept, like any backup); no app installed or removed today.
+  demo-hp host — the installer copy shredded after the rotation; root password rotated on purpose (new value in the
+  vault). Tester 1, demo-felhom — agent 0.159.2 and controller 0.330.0 through their normal paths; nothing else.
+- **Hub:** hub 0.158.0; global floor 0.330.0 / 0.158.1; no per-box floor; signed jobs consumed. No vouch by CC.
+- DooPlex: builds only; ArgoCD synced two Deployments (hub, webpage), each alone.
+
+Instruction files: none edited. System poster: none of the three (the poster shows backup retentions; the hub's access
+log is not on it). Architecture documents read: `02`, `03`, `05`, `07`, `09` §3 decision 250; `break-glass.md`.

STATUS.md

@@ -41,7 +41,7 @@
 
 ## What we work on
 - **CC's secrets** (R-1118): done 2026-10-17 — the store, `sec`, the guards in every CC session (option B: `bash -x` and `curl -v` allowed inside `sec run`), the nightly check (linger on). The old credentials file and its three copies are deleted.
-- **After 2026-10-18 14:16 UTC** (demo-hp's next clean-up window): check that the old restic (image 0.322.0) still lists and restores demo-hp's store after its first real 0.19.1 prune (`audits/night-restore-2026-10-17/tools/rx.sh`). Everything else of R-1105/R-1107 is done.
+- **After 2026-10-19 02:15 UTC** (demo-hp's next clean-up window: the hub allows one from 2026-10-18 14:16 UTC, but the box asks only after its 04:15 CEST night run; not open on 2026-10-18 at 14:33 UTC, `audits/day-2026-10-18/P8/`): check that the old restic (image 0.322.0) still lists and restores demo-hp's store after its first real 0.19.1 prune (`audits/night-restore-2026-10-17/tools/rx.sh`). Everything else of R-1105/R-1107 is done.
 - The power-cut findings still open (R-1004, R-1005, R-1010, R-1013). R-1012 closed: agent 0.159.1 names an orphan thin snapshot in the backup failure.
 - Media apps on every drive (R-1023): live with the 0.313.0 floor — every drive in every media app, a new drive made ready by the box, a pulled drive loses nothing, a safe removal lets go first and the drive comes back by itself.
 

documentation/audits/day-2026-10-18/P8/cleanup-window-not-yet.txt

@@ -0,0 +1,20 @@
+# P8 — old-restic check: NOT RUN (the clean-up window has not opened). Read 2026-10-18T14:33:43Z.
+
+## demo-hp's clean-up windows (hub DB with its WAL, one tar)
+id|customer|opened_at|closes_by|closed_at|box_result
+9|demo-hp|2026-10-12 02:16:53|2026-10-12 02:36:53|2026-10-12 02:17:06|pruned
+6|demo-hp|2026-10-05 05:25:53|2026-10-05 05:45:53|2026-10-05 05:26:04|pruned
+2|demo-hp|2026-10-04 05:36:14|2026-10-04 05:56:14|2026-10-04 05:36:17|nothing
+
+## The rule (hub/internal/offsitekeys/service.go:255)
+	windowCadence     = 6*24*time.Hour + 12*time.Hour // "weekly", with slack for the night chain's drift
+Last window 2026-10-12 02:16:53 UTC + 6 d 12 h = 2026-10-18 14:16:53 UTC (the STATUS time).
+The box asks for a window only AFTER its nightly off-site run (controller backup/offbox_window.go header).
+
+## demo-hp guest 9201 controller (docker logs since 06:00 UTC, [offbox] lines + scheduler)
+felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.330.0 Up 3 hours (healthy)
+2026/10/18 11:09:36 scheduler.go:150: [INFO] [scheduler] Daily job offbox-backup scheduled for 2026-10-19 04:15 CEST
+2026/10/18 11:09:36 scheduler.go:85: [DEBUG] [scheduler] daily job registered: name="offbox-backup" schedule="04:15" nextRun=2026-10-19T04:15:00+02:00 totalJobs=15
+2026/10/18 11:09:36 scheduler.go:85: [DEBUG] [scheduler] daily job offbox-backup: next run at 2026-10-19 04:15:00 CEST (waiting 15h5m23s)
+
+Next run: 2026-10-19 04:15 CEST = 02:15 UTC; the window opens right after it. No window was asked for or granted by CC.