felhom.eu/REPORT-golden-kept-offsite-2026-09-28.md

# REPORT — 2026-09-28: weekly golden + Day-0 vouch, kept data from the off-site copy, claper/calcom, demo-hp space, night read, first live off-site restores

Brief: "the weekly golden, the Day-0 vouch, use my kept data from the off-site copy, two more PostgreSQL fixtures,
demo-hp's restore-test space, the night watch, and the first live off-site restore" (revised 2026-09-28).
Architecture read before any claim: `07-backup-architecture.md` §6.5, §6.6; `06-offsite-connectivity.md`;
`03-host-agent.md` (restore-test storage); `09-update-architecture.md` §3 decisions 35–42.
**Operator change mid-session (15:14):** "finish today" → Part E and D2 were done in the day, not overnight (below).

## The Parts

| Part | Step | State | Note |
|---|---|---|---|
| A | 1 read the manifest (rollback) | done | agent 0.132.0, golden 0.258.0, min_agent 0.131.0 |
| A | 2 bake golden 0.276.0 | done, **with a deviation** | attempt 1 picked the `arm64` template; stopped by CC before `pct create` finished, nothing published. Attempt 2: all markers, leak grep 0 (control 1), 404 → 200, anonymous download sha equal, VM back to `virgin` |
| A | 3 vouch (3 fields) | done | R-120 gate refused golden 0.258.0 first (negative control); `agent=0.137.0 golden=0.276.0 min_agent="0.131.0"` read back |
| A | 4 Day-0 test install | done | installer 1.28.0, agent + golden fetched through the manifest and sha-verified, controller 0.276.0 healthy, claim gate armed; not claimed; scratch customer deleted by the hub's own cascade |
| A | 5 golden-currency gate | done | green, not waived (record `documentation/tests/golden-0.276.0-2026-09-28/`); waiver file left as it is (to 2026-10-04) |
| A | 6 agent CHANGELOG line | done | felhom-agent `5c68c86`; no floor move in Part A |
| B | 1–3 code, tests, red-proofs | done | controller **v0.277.0**; RP1–RP3 |
| B | 4 live Tier 1/Tier 2 regression on 9202 | Tier 1 done; **Tier 2 not runnable** | 9202 has one drive |
| B | 5 release + floor | done | floor 0.277.0 at 10:11 (before 01:30), both boxes healthy |
| B | — | **changed: a second release, v0.278.0** | R-704 blocked Part E; floor 0.278.0 at 15:36 |
| C | calcom | done | memory fix first (R-703, 768M OOM at every start → 1536M); PG 16 → **18** proven on bench + box; catalog `037f956` |
| C | claper | done | PG 16 → **17** proven on bench + box; catalog `4a249b9`; found R-702 (default admin) |
| D | 1 R-701 (a) | done — **not enough** | 20.3 → 26.6 GiB free; 31 needed; 14:13 cycle refused again |
| D | 2 night read | **changed** | the night 27/28 was read (logs, hub, agent journals), not tonight's; demo-felhom's off-site/update legs not readable |
| E | 1 setup | done | nextcloud on demo-hp, seeded, joined the off-site copy |
| E | 2 (a) off-site restore | done — **changed timing** | the snapshot came from the page's "run now" (15:40), not the night; seed back, marker gone |
| E | 2 (b) kept data from off-site | done | choice named "távoli mentés, 2026-09-28 15:40"; seed + files back |
| E | 3 teardown | done | app removed with its data; app list equal to before; verification copy deleted by the product |

## Claims in the brief that turned out wrong (or right)

- **"0.276.0's MinAgent is 0.131.0"** — right.
- **"The R-120 gate accepts golden 0.276.0"** — right (it refused 0.258.0 and accepted 0.276.0).
- **"The Day-0 test install leaves no hub record"** — **wrong.** It left a host, a vaulted break-glass credential, a claim code, a WireGuard peer (10.77.0.5, synced toward ep0) and reports. The customer DELETE cascade removed the hub side; the ep0 peer removal was **not observed** (ep0 untouched by rule).
- **"The pairing code is shown"** — **wrong for this path.** The one-liner install shows no pairing code (that is the ISO path); what shows is the controller's claim gate (`dashboard not yet claimed`).
- **"`pct fstrim` frees enough for R-701"** — **wrong.** It freed 6.3 GiB; the pool is 53.9 GiB and the guest holds ~26 GiB, so 31 GiB free cannot be reached by trimming.
- **"paperless-ngx runs on demo-hp"** — right; it converted to PostgreSQL 18 in the night 27/28 (04:22 CEST, rows equal, 0 documents).
- **"An app joins the off-site copy by a per-app switch"** — right (`POST /backup/offbox/toggle`, `app_backup.<app>.offbox`).

## Evidence

- Golden, vouch, Day-0, D1, D2: `documentation/audits/evidence-golden-0276-2026-09-28/`
- Part B + E: `documentation/audits/kept-offsite-2026-09-28/` (redproofs/, live/, E/)
- Part C: `documentation/audits/pg-calcom-claper-2026-09-28/`

## Rows

Opened: R-702 (claper default admin, P1), R-703 (calcom OOM — closed the same day), R-704 (leftover holds — fixed in
0.278.0, WATCHING), R-705 (no "run the night now"), R-706 (verification copy survives removal). Closed: R-691, R-703.
Updated: R-463, R-687, R-701. Register 337 → 342 rows. `unproven.py --summary`: not walked 35 of 55 (unchanged; the
capability map was not edited).

## Teardown, three layers

- **Machines:** drill VM reverted to `virgin` (qemu gone); bench LXC 9401 created and destroyed twice; nextcloud,
  calcom, claper removed through the product on 9201/9202; 9202 pointed back to the live catalog; drill catalog = live.
- **Hosts:** demo-hp `local-lvm` trimmed (kept); template cache files removed; no storage added.
- **Hub:** scratch customer `drill-g0276` deleted by its cascade; manifest now golden 0.276.0 / agent 0.137.0; floor
  0.278.0. nextcloud's off-site snapshots stay in demo-hp's repository (removal never touches off-site history, R-474).