Read-back Parts B and D (R-518: night stop 91 s, one press 80 s, page text holds); rule 11 — every helper prompt carries the brief's fences in full (decision 160); decision 161
gates / gates (push) Successful in 2m58s

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-07 08:51:43 +02:00
1 parent a5cbc275a3
commit b58f3ccefd
13 files changed
+9652 -1

No files matched your search

@@ -915,6 +915,17 @@ its length, and both fixes cost something the household would notice — operato
127. **The agent's three by-design abilities (`03` §3.1) stay for now**; revisited before the first paying customer.
*Operator ruling 2026-10-05.* (R-861)
### 2026-10-07 (morning) — operator rulings (recorded before the work)
160. **Every helper prompt carries the brief's fences in full.** A helper session (subagent, fork, workflow agent)
gets the brief's fence list word for word, not a summary. Rule 11 of `.claude/rules/unprompted-work.md` §3 (all
five copies). *Operator ruling 2026-10-07.* Earned the night before: two helpers with partial fences ran a hand
`docker volume prune` on the bench and a Docker-using gate on DooPlex.
161. **The release order after the 2026-10-07 read-back:** hub 0.141.0, agent 0.150.0, the controller, then the held
catalog branch `night-held-2026-10-06`; delivered to demo-hp, demo-felhom and the Tester 1 box; then the Tester 1
update test with the operator present. *Operator ruling 2026-10-07* (the morning note's decision 1). On the same
word: the Tester 1 dashboard password was reset through the box's own reset (decision 158's route).
### 2026-10-06 (night) — decided by CC unattended — operator may reverse (second burn-down night)
159. **R-805 — an EMPTY bind mount after the persistence gate's exercise is REPORTED, not judged.** *One sentence:*
@@ -0,0 +1,7 @@
Oct 07 04:34:50 demo-felhom felhom-agent[1157905]: time=2026-10-07T04:34:50.555+02:00 level=INFO msg="backup: space preflight passed" vmid=9201 target=felhom-backup last_archive_bytes=962295489 need_bytes=2276611185 avail_bytes=926945894400
Oct 07 04:34:51 demo-felhom felhom-agent[1157905]: time=2026-10-07T04:34:51.590+02:00 level=INFO msg="local-api: backup reached snapshotted (app may resume)" vmid=9201 target=felhom-backup job=backup-9201-1791340490154356945
Oct 07 04:35:21 demo-felhom felhom-agent[1157905]: time=2026-10-07T04:35:21.689+02:00 level=INFO msg="backup: completed" vmid=9201 target=felhom-backup archive=felhom-backup:backup/vzdump-lxc-9201-2026_10_07-04_34_50.tar.zst size_bytes=963420872 uncovered_volu
Oct 07 04:35:21 demo-felhom felhom-agent[1157905]: time=2026-10-07T04:35:21.689+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=felhom-backup job=backup-9201-1791340490154356945 archive=felhom-backup:backup/vzdump-lxc-9201-2026_10_07-04_
Oct 07 06:21:18 demo-felhom felhom-agent[1157905]: time=2026-10-07T06:21:18.315+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-10-06T04:21:09Z
Oct 07 06:21:18 demo-felhom felhom-agent[1157905]: time=2026-10-07T06:21:18.616+02:00 level=INFO msg="janitor: stale-lock sweep deferred — a heavy operation is in flight" busy=restore-test
Oct 07 06:22:38 demo-felhom felhom-agent[1157905]: time=2026-10-07T06:22:38.491+02:00 level=INFO msg="backup: scheduled restore-test passed" archive=felhom-pbs:backup/ct/9201/2026-10-06T04:21:09Z duration_s=80.173996289
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,6 @@
Oct 07 04:35:10 demo-hp felhom-agent[297545]: time=2026-10-07T04:35:10.991+02:00 level=INFO msg="backup: space preflight passed" vmid=9201 target=local last_archive_bytes=4880501245 need_bytes=7174368380 avail_bytes=17808347136
Oct 07 04:35:12 demo-hp felhom-agent[297545]: time=2026-10-07T04:35:12.051+02:00 level=INFO msg="local-api: backup reached snapshotted (app may resume)" vmid=9201 target=local job=backup-9201-1791340510438908976
Oct 07 04:40:48 demo-hp felhom-agent[297545]: time=2026-10-07T04:40:48.072+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_10_07-04_35_11.tar.zst size_bytes=4885448692 uncovered_volumes=2
Oct 07 04:40:48 demo-hp felhom-agent[297545]: time=2026-10-07T04:40:48.072+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1791340510438908976 archive=local:backup/vzdump-lxc-9201-2026_10_07-04_35_11.tar.zst
Oct 07 04:41:18 demo-hp felhom-agent[297545]: time=2026-10-07T04:41:18.114+02:00 level=INFO msg="janitor: stale-lock sweep deferred — a heavy operation is in flight" busy=backup:local
Oct 07 06:22:47 demo-hp felhom-agent[297545]: time=2026-10-07T06:22:47.688+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="felhom-pbs: newest settled archive (landed 2026-10-01T20:15:29Z) is already proven; local: no settled arc
File diff suppressed because it is too large. Load diff
@@ -0,0 +1 @@
press POST /api/guest-backup/trigger at 06:44:37Z → 200 {started:true}
@@ -0,0 +1,52 @@
06:44:23 wiki=302 phase=done 908976
06:44:30 wiki=302 phase=done 908976
06:44:37 wiki=302 phase=done 908976
06:44:44 wiki=502 phase=done 908976
06:44:50 wiki=404 phase=done 908976
06:44:58 wiki=404 phase=done 908976
06:45:05 wiki=404 phase=snapshotted 429974
06:45:12 wiki=404 phase=snapshotted 429974
06:45:19 wiki=404 phase=snapshotted 429974
06:45:26 wiki=404 phase=snapshotted 429974
06:45:33 wiki=404 phase=snapshotted 429974
06:45:40 wiki=302 phase=snapshotted 429974
06:45:47 wiki=302 phase=snapshotted 429974
06:45:53 wiki=302 phase=snapshotted 429974
06:46:02 wiki=302 phase=snapshotted 429974
06:46:08 wiki=302 phase=snapshotted 429974
06:46:15 wiki=302 phase=snapshotted 429974
06:46:22 wiki=302 phase=snapshotted 429974
06:46:29 wiki=302 phase=snapshotted 429974
06:46:35 wiki=302 phase=snapshotted 429974
06:46:43 wiki=302 phase=snapshotted 429974
06:46:49 wiki=302 phase=snapshotted 429974
06:46:56 wiki=302 phase=snapshotted 429974
06:47:03 wiki=302 phase=snapshotted 429974
06:47:10 wiki=302 phase=snapshotted 429974
06:47:17 wiki=302 phase=snapshotted 429974
06:47:24 wiki=302 phase=snapshotted 429974
06:47:31 wiki=302 phase=snapshotted 429974
06:47:38 wiki=302 phase=snapshotted 429974
06:47:44 wiki=302 phase=snapshotted 429974
06:47:53 wiki=302 phase=snapshotted 429974
06:48:00 wiki=302 phase=snapshotted 429974
06:48:06 wiki=302 phase=snapshotted 429974
06:48:13 wiki=302 phase=snapshotted 429974
06:48:20 wiki=302 phase=snapshotted 429974
06:48:28 wiki=302 phase=snapshotted 429974
06:48:35 wiki=302 phase=snapshotted 429974
06:48:42 wiki=302 phase=snapshotted 429974
06:48:48 wiki=302 phase=snapshotted 429974
06:48:55 wiki=302 phase=snapshotted 429974
06:49:02 wiki=302 phase=snapshotted 429974
06:49:09 wiki=302 phase=snapshotted 429974
06:49:16 wiki=302 phase=snapshotted 429974
06:49:22 wiki=302 phase=snapshotted 429974
06:49:29 wiki=302 phase=snapshotted 429974
06:49:36 wiki=302 phase=snapshotted 429974
06:49:43 wiki=302 phase=snapshotted 429974
06:49:49 wiki=302 phase=snapshotted 429974
06:49:56 wiki=302 phase=done 429974
06:50:03 wiki=302 phase=done 429974
06:50:09 wiki=302 phase=done 429974
06:50:16 wiki=302 phase=done 429974
@@ -0,0 +1,21 @@
traefik 2026-10-05T07:58:38.633920895Z
filebrowser 2026-10-05T07:58:38.664521793Z
cloudflared 2026-10-05T07:58:38.71384051Z
felhom-controller 2026-10-06T14:04:46.082247231Z
adventurelog-frontend 2026-10-07T06:45:11.239145387Z
adventurelog-postgres 2026-10-07T06:45:11.470781955Z
adventurelog 2026-10-07T06:45:17.182102124Z
bentopdf 2026-10-07T06:45:17.994308069Z
bookstack-db 2026-10-07T06:45:19.381327511Z
bookstack 2026-10-07T06:45:25.240145227Z
calibre-web 2026-10-07T06:45:25.973987919Z
docmost-redis 2026-10-07T06:45:26.803344264Z
docmost-postgres 2026-10-07T06:45:27.014396457Z
docmost 2026-10-07T06:45:37.261530839Z
kimai-db 2026-10-07T06:45:38.334730945Z
kimai 2026-10-07T06:45:44.11076447Z
opengist 2026-10-07T06:45:44.83730456Z
paperless-redis 2026-10-07T06:45:45.780146268Z
paperless-postgres 2026-10-07T06:45:46.078695976Z
paperless-webserver 2026-10-07T06:45:56.394276341Z
privatebin 2026-10-07T06:45:57.236922434Z
@@ -0,0 +1,18 @@
2026/10/07 06:44:37 backup_handlers.go:349: [INFO] [web] manual whole-guest backup triggered (quiesce loop)
2026/10/07 06:44:37 quiesce.go:428: [INFO] [quiesce] manual backup requested — quiescing now
2026/10/07 06:44:38 quiesce.go:578: [INFO] [quiesce] backup due on 1 tier(s) — quiescing 9 stack(s): [adventurelog bentopdf bookstack calibre-web docmost kimai opengist paperless-ngx privatebin]
2026/10/07 06:44:47 quiesce.go:210: [INFO] [quiesce] a backup cycle is already running — skipping this scheduled check
2026/10/07 06:45:00 quiesce.go:627: [INFO] [quiesce] tier local: backup job backup-9201-1791355500848429974 started — polling
2026/10/07 06:45:10 quiesce.go:706: [INFO] [quiesce] tier local: job backup-9201-1791355500848429974 snapshotted — resuming app early (8B.2); the upload continues with the apps running
2026/10/07 06:45:10 quiesce.go:565: [INFO] [quiesce] unquiescing (snapshotted (early resume)): restarting 9 stack(s)
2026/10/07 06:45:17 manager.go:1309: [INFO] [stacks] Stack adventurelog started successfully (took 6.6s)
2026/10/07 06:45:18 manager.go:1309: [INFO] [stacks] Stack bentopdf started successfully (took 0.7s)
2026/10/07 06:45:25 manager.go:1309: [INFO] [stacks] Stack bookstack started successfully (took 6.8s)
2026/10/07 06:45:26 manager.go:1309: [INFO] [stacks] Stack calibre-web started successfully (took 0.5s)
2026/10/07 06:45:37 manager.go:1309: [INFO] [stacks] Stack docmost started successfully (took 11.1s)
2026/10/07 06:45:44 manager.go:1309: [INFO] [stacks] Stack kimai started successfully (took 6.7s)
2026/10/07 06:45:45 manager.go:1309: [INFO] [stacks] Stack opengist started successfully (took 0.5s)
2026/10/07 06:45:56 manager.go:1309: [INFO] [stacks] Stack paperless-ngx started successfully (took 11.3s)
2026/10/07 06:45:57 manager.go:1309: [INFO] [stacks] Stack privatebin started successfully (took 0.6s)
2026/10/07 06:49:47 quiesce.go:210: [INFO] [quiesce] a backup cycle is already running — skipping this scheduled check
2026/10/07 06:49:57 quiesce.go:710: [INFO] [quiesce] tier local: backup job backup-9201-1791355500848429974 done
@@ -0,0 +1,33 @@
# Parts B and D — the night read-back and one press (2026-10-07 morning)
The task file `TASK-night-readback-offsite-gap-tester1-2026-10-07.md` is not on DooPlex (searched the disk and git
history); Parts B and D were taken from the evening `REPORT.md` and the R-518 row: B = read both demo boxes' night under
controller v0.301.0 (decision 156: one stop per tier, the apps resume at `snapshotted`); D = one „Mentés most" press on a
demo box, measured, and the page's text checked against it. Times are UTC.
## Part B — the night of 2026-10-06 → 07 (local tier only, as predicted; off-site not due on either box)
| Box | Apps | Stop began | Snapshotted | Last app back | App outage | Copy done (apps running) |
|---|---|---|---|---|---|---|
| demo-hp | 9 | 02:34:50 | 02:35:20 | 02:36:21 | **~91 s** (was 5 min 47 s on 2026-10-05) | 02:40:52 |
| demo-felhom | 1 | 02:34:49 | 02:35:00 | 02:35:00 | **~11 s** | 02:35:30 |
Two channels each: the controller's log (`B/B-*-controller-0200-0700Z.log`) and the agent's journal
(`B/B-*-agent-0400-0700local.txt`: `reached snapshotted` 04:35:12 / 04:34:51 local, `backup: completed`). No BUSY, no
failure; the apps' databases report `healthy` after the restart. demo-felhom's restore-test passed at 06:22 local.
## Part D — one press on demo-hp (9 apps), 06:44:37
Pressed through the endpoint the button calls (`POST /api/guest-backup/trigger`, dashboard session + CSRF) → 200
`{started:true}`. Only the local tier ran (the button's rule).
- Stop began 06:44:38; job started 06:45:00; `snapshotted` 06:45:10; first app back 06:45:17 (adventurelog), last
06:45:57 (privatebin) — **80 s from press to the last app; per app 39–79 s.** Copy done 06:49:57 with the apps running.
- Channel 1: a 5-s sampler of bookstack through traefik + the job phase (`D/D1-sampler-5s.log`: 302 → 502/404 from
06:44:44 → 302 again at 06:45:40). Channel 2: container `StartedAt` (`D/D2-container-startedat.txt`, bookstack
06:45:25) and the controller log (`D/D3-controller-quiesce.log`).
- **The page text holds:** „Mentés közben minden alkalmazás rövid időre leáll: kb. 1–1,5 percre, több alkalmazásnál
tovább. … A gomb a helyi másolatot készíti el; a távoli másolat a következő éjszakai mentéssel készül." No text change
needed.
**Left on R-518:** the first night with BOTH tiers due under the new rule (demo-hp, off-site due ~2026-10-08).
@@ -0,0 +1,4 @@
2026-10-06T17:34:31Z
sync=Synced health=Healthy rev=e0bdd52ba2adf06b3cf62a16eb35ec191af928d5
image=gitea.dooplex.hu/admin/felhom-hub:0.140.0
2026/10/06 19:33:59 [INFO] felhom-hub 0.140.0 starting
+1 -1
View File
@@ -155,7 +155,7 @@ stopping line that lies.
| **R-232** | Backup & restore | P2 | **DooPlex's backup makes every copy inside the same box — and nothing tells anyone when it fails.** Surveyed read-only 2026-08-06 (`audits/RECON-dooplex-backup-2026-08-06.md`). **What works:** five sets, 14/14 successful runs in 14 days; a file was restored from the `data` repo and matched the live original **byte for byte**; every set except two is cross-disk; k3s is integrity-checked on every run. **What the matrix exposes, ranked:** (a) **`notify_failure` is a no-op** — `NOTIFY_ON_FAILURE=true` but `NOTIFY_WEBHOOK_URL` is commented out, so a failed backup notifies **nobody**; the project already has a working Resend path that CI uses. Cheapest item, and it makes every other failure visible. (b) **Nothing leaves the box** — no rclone, no remote repo, no off-site target anywhere; Longhorn's target is `nfs://192.168.0.180:` pointing at DooPlex itself, and the only outbound-looking cron pulls *inbound* from Hetzner for a different project. The machine that runs the hub managing the customers' off-site chain has no off-site copy of its own. (c) **The backup tree is a single writable path** and the restic repos are not append-only — one bad script or ransomware destroys every copy at once. (d) **Two same-disk sets**: `.claude-memory` and the PostgreSQL dumps, whose source directory sits *inside* the backup tree. (e) **Longhorn `retain=1`** — one generation per volume, so a corruption noticed a day late has no earlier copy. (f) **`/opt/backup/docs/BACKUP-RESTORE.md` does not exist** though the systemd unit advertises it. (g) **`secrets/restic-repo` has never held a snapshot** — `backup-secrets.sh` contains no `restic` call; the secrets are GPG files on `sda1` only. (h) **No restore has ever been run** beyond today's single-file probe — the matrix's "ever demonstrated?" column is otherwise entirely empty. **Not a finding:** the restic passphrase. The on-box copy is on `sdb1`, a different disk from the backups, and the **operator holds an offline copy out of band** — so a disk loss is recoverable. The narrow residual is that it is operator-held rather than system-held, unlike the customer case's hub-vaulted escrow, so it should be confirmed current and findable by someone else. **Nothing was changed by the recon.** | **NARROWED 2026-10-05 — owner Viktor.** (b) partly: the hub database now leaves DooPlex nightly, encrypted, to ep0 (R-173); everything else in DooPlex's backup still stays on the box. (a) partly: the hub copy alarms through Prometheus (`HubDBBackupStale`); `notify_failure` is still a no-op for the rest. (c)–(h) unchanged. **READY** for the rest | — | — | operator |
| **R-304** | Backup & restore | P2 | **The retained escrow key works, and the customer is told their correct code is wrong.** DRILL 2026-08-12 answered the three questions separately, on `demo-felhom`, with planted data. **(a) retention: WORKS** — the first retained row in fleet history to carry material (`host_escrow_superseded` id 11, `identity_blob` 572 B), byte-identical (`sha256 a10032341c8584ed…`) to the pre-supersession `host_escrow` row. **(b) the material opens the old store: YES** — unsealed with the OLD recovery code it yielded a password byte-identical to the pre-change one (`sha c60c8bc737a6b7c6…`), and restored three planted files **byte-identical** from a store the box itself could no longer open (negative control first: `Fatal: wrong password or no key found`), **including a Hungarian accented filename verified as raw bytes**. **(c) the customer's route: DOES NOT EXIST, and misinforms.** `ListSupersededEscrow` (`store.go:2841`) is the only reader of a retained `identity_blob` and has **zero production callers** — five call sites, all `_test.go`; the product path (`POST /escrow/recover-offsite-password` → `FetchIdentityEscrow` → `GetHostDRBundle`, `store.go:3152`) selects `FROM host_escrow` — the CURRENT row only. Asked for the old password with the code that demonstrably opens the retained row, the product answered **"the recovery code did not open the sealed bundle — nothing was written"**. **This is the R-224 class again**: there an unreachable hub was reported as a bad code; here a VALID code for retained history is reported as a bad code, and the customer's attempt ends there. **Consequence:** the census answer stands (it was about retention); the countdown banner's promise is true in substance and false in practice; **any capability-map claim that the customer can recover the old history with their recovery code is false today and must move** | **READY (L) — NEW 2026-08-12, RANK 1** | R-198, R-199, R-224, R-241 | Decide the shape: serve retained rows on the recovery path (needs a "which package?" choice — a customer may have several), or stop promising retrieval anywhere the customer cannot perform it. **Until one of those, the honest position is that retention is an operator-only capability.** At minimum, the refusal must stop asserting the code is wrong when the hub simply never looked | operator + CC |
| **R-366** | Backup & restore | P2 | **The 21 August reinstall orphaned `demo-hp`'s PBS whole-guest archives as well as its off-site repo — the box can no longer read its own pre-reinstall backups, and this surfaces only as a restore-test failure.** Hub event 3016, 2026-08-21 21:59:28Z, unprompted: `Restore-test FAILED on the pbs tier: archive felhom-pbs:backup/ct/9201/2026-08-18T03:58:43Z could not be restored+booted … proxmox-backup-client failed: Error: wrong key - unable to verify signature since manifest's key 3f:4f:65:c0:d8:f3:9f:3c does not match provided key dd:d1:d8:53:44:62:5e:0b`. The archive predates the reinstall by three days. **This is the PBS-tier analogue of R-193** (a guest rebuild mints a fresh secret and orphans the history), and the two together mean a rebuilt box loses BOTH off-premises tiers at once: the restic repo needed a self-heal + re-toggle (see the drill report), and the PBS archives are simply unreadable to it. **Credit: the restore-test caught it and said so precisely** — the mechanism works. **The gap is what it is called:** it is reported as *a restore test that failed*, which reads as a flaky verification, not as *every whole-guest backup you took before the reinstall is unreadable on this machine*. **Found incidentally by the 2026-08-21 backup-truth drill; nobody was looking for it.** | **OPEN — HIGH** **2026-10-06 night: the wrong verdict was already gone** (agent v0.138.0, R-727 skips another key's archives); demo-hp's August archives are pruned by ep0's keep-last 2 (inferred, ep0 fenced). **NEW, the real gap:** the hub retained an escrow only on a restic-password change, so a reinstall's new backup key K overwrote the only copy of the old one — **fixed on felhom.eu main (hub, unreleased; red-proved `audits/night-burndown-2026-10-06/r366/`)**. Design `audits/night-burndown-2026-10-06/design-R-366.md` (pick B: retain + tell the operator; C, key continuity, is the operator's). Not closed: the operator signal (archives made with another key) is the design's second slice. | related: R-193 | Establish whether the pre-reinstall PBS archives are recoverable at all (the old key's whereabouts), and separate the two verdicts: a tier whose ARCHIVES ARE ORPHANED is a different alarm from a tier whose restore test failed. Do not close on the strength of the restore-test wording alone. | CC |
| **R-518** | Backup & restore | P2 | **[P2-MEDIUM] „Mentés most" on the whole-system backup stops every app for about eight minutes while the page promises „csak néhány másodpercre".** MEASURED 2026-09-14 (BIGNIGHT, VM 333, 12 apps): the button's call quiesced all 12 stacks at 19:03:23Z (first stopped 19:03:27Z); the local vzdump ran 19:03:49 → 19:09:59Z; the controller then kept the apps stopped for the second (PBS) tier and restarted them at 19:10:09Z after it failed, the last started 19:11:12Z (`phase4/guest-backup-quiesce-log.txt`) — **≈ 7 m 45 s** with every app answering 404. The page under the button: „Pillanatkép-mód: az alkalmazások csak néhány másodpercre állnak le." A household pressing it at dinner loses every app for the length of the dump, and longer on a bigger box. **Fix shape:** state the real expected downtime (it scales with data), or quiesce per tier and not across a second tier's attempt; do not start a tier whose storage is absent (see R-517). **NARROWED 2026-09-15 (controller v0.243.0 + agent v0.131.0):** a tier whose storage the agent reports absent is skipped before anything stops (`backup_tier_skipped`, once per absence; unknown never skipped), and the button copy now says „általában néhány perc, nagyobb adatnál több". Unit-proven with red-proofs. **Still open:** quiesce per tier, so a slow second tier does not keep every app down. **— NIGHT 2026-09-23 (controller v0.267.0):** the copy half is DONE: the page and the confirm now state the measured stop (≈ 8 minutes on a 12-app box), both languages, red-proofed (`audits/night-2026-09-23/A5-*`). The brief's „csak néhány másodpercre" had already gone in v0.243.0. **Still open:** quiesce per tier, so a slow second tier does not keep every app down. | **READY — P2, narrowed to per-tier quiesce; owner: CC (controller). 2026-10-05: the copy now states today's measurement too (demo-hp, 9 apps, local tier only: 5 min 47 s) — controller v0.296.0, `audits/hub-safety-2026-10-05/partE/`.** **2026-10-05 (burn-down night): a one-page design proposal (no code) is in `audits/night-burndown-2026-10-05/design-R-518.md`** — for the operator. **2026-10-06: BUILT — controller v0.301.0, `09` §3 decision 156 (reverses R-82's one window).** One stop per tier; the button makes the local copy only. Measured first, read-only: demo-felhom's night off-site job reached `snapshotted` 2 s after it started, the app back 8 s later (the off-site part of a stop is seconds). **Not shown live:** a press under the new rule — scratch 9202 has no agent connection and the demo boxes take deliveries only. Red tests and the build: `audits/design-build-2026-10-06/`D/. **Risk noted, unmeasured:** after a local copy the agent runs its OS step, and the off-site tier then answered BUSY (2026-10-05) — under the new rule that costs one short stop with no copy before the 15-min backoff. **2026-10-06 (night), from Part C:** demo-hp's off-site tier was NOT overdue — its last copy is 2026-10-01 20:15Z (ep0's listing, verify ok), so with the 7-day cadence it is due ~2026-10-08; the night of 2026-10-06→07 is most likely local-only on both demo boxes (demo-felhom's off-site landed 2026-10-06 04:21Z). The two-tier night under the new rule is then ~2026-10-08 on demo-hp. | — | Read back the 2026-10-07 night (local tier) and the ~2026-10-08 night (both tiers on demo-hp); measure one press (Part D) | CC |
| **R-518** | Backup & restore | P2 | **[P2-MEDIUM] „Mentés most" on the whole-system backup stops every app for about eight minutes while the page promises „csak néhány másodpercre".** MEASURED 2026-09-14 (BIGNIGHT, VM 333, 12 apps): the button's call quiesced all 12 stacks at 19:03:23Z (first stopped 19:03:27Z); the local vzdump ran 19:03:49 → 19:09:59Z; the controller then kept the apps stopped for the second (PBS) tier and restarted them at 19:10:09Z after it failed, the last started 19:11:12Z (`phase4/guest-backup-quiesce-log.txt`) — **≈ 7 m 45 s** with every app answering 404. The page under the button: „Pillanatkép-mód: az alkalmazások csak néhány másodpercre állnak le." A household pressing it at dinner loses every app for the length of the dump, and longer on a bigger box. **Fix shape:** state the real expected downtime (it scales with data), or quiesce per tier and not across a second tier's attempt; do not start a tier whose storage is absent (see R-517). **NARROWED 2026-09-15 (controller v0.243.0 + agent v0.131.0):** a tier whose storage the agent reports absent is skipped before anything stops (`backup_tier_skipped`, once per absence; unknown never skipped), and the button copy now says „általában néhány perc, nagyobb adatnál több". Unit-proven with red-proofs. **Still open:** quiesce per tier, so a slow second tier does not keep every app down. **— NIGHT 2026-09-23 (controller v0.267.0):** the copy half is DONE: the page and the confirm now state the measured stop (≈ 8 minutes on a 12-app box), both languages, red-proofed (`audits/night-2026-09-23/A5-*`). The brief's „csak néhány másodpercre" had already gone in v0.243.0. **Still open:** quiesce per tier, so a slow second tier does not keep every app down. | **READY — P2, narrowed to per-tier quiesce; owner: CC (controller). 2026-10-05: the copy now states today's measurement too (demo-hp, 9 apps, local tier only: 5 min 47 s) — controller v0.296.0, `audits/hub-safety-2026-10-05/partE/`.** **2026-10-05 (burn-down night): a one-page design proposal (no code) is in `audits/night-burndown-2026-10-05/design-R-518.md`** — for the operator. **2026-10-06: BUILT — controller v0.301.0, `09` §3 decision 156 (reverses R-82's one window).** One stop per tier; the button makes the local copy only. Measured first, read-only: demo-felhom's night off-site job reached `snapshotted` 2 s after it started, the app back 8 s later (the off-site part of a stop is seconds). **Not shown live:** a press under the new rule — scratch 9202 has no agent connection and the demo boxes take deliveries only. Red tests and the build: `audits/design-build-2026-10-06/`D/. **Risk noted, unmeasured:** after a local copy the agent runs its OS step, and the off-site tier then answered BUSY (2026-10-05) — under the new rule that costs one short stop with no copy before the 15-min backoff. **2026-10-06 (night), from Part C:** demo-hp's off-site tier was NOT overdue — its last copy is 2026-10-01 20:15Z (ep0's listing, verify ok), so with the 7-day cadence it is due ~2026-10-08; the night of 2026-10-06→07 is most likely local-only on both demo boxes (demo-felhom's off-site landed 2026-10-06 04:21Z). The two-tier night under the new rule is then ~2026-10-08 on demo-hp. **2026-10-07 (morning): the local-tier night and one press READ BACK** (`audits/readback-2026-10-07/RESULT-B-D.md`): the night stop on demo-hp (9 apps) was ~91 s (was 5 min 47 s), demo-felhom (1 app) ~11 s; one press on demo-hp: 80 s from press to the last app (per app 39–79 s), the copy finished 4 min later with the apps running, only the local tier ran; the page's „kb. 1–1,5 perc" holds. Two channels each (controller log + agent journal / container StartedAt + a 5-s HTTP sampler). **Left:** the first night with both tiers due on demo-hp (~2026-10-08). | — | Read back the ~2026-10-08 night (both tiers on demo-hp); then close | CC |
| **R-893** | Backup & restore | P3 | **After a failed OFF-SITE replay, the rollback pours the NEWER pre-restore copy over the OLDER volume just put back.** Read in source 2026-10-06 (R-638 option A, not measured): `internal/backup/offbox_reconstitute.go` writes the undo copy from the live (newer) database, replaces the volumes with the snapshot's older tars, then — when the replay fails — `rollbackSafetyDump` loads that newer dump over the older database volume. The loader only drops what the dump knows, so tables the newer migration removed stay; and when the snapshot's older definition was written, the rollback branch does not put the newer definition back, so the older app starts on rolled-back data; non-database volumes stay at the snapshot's state. An order change cannot fix it (the only undo is a logical dump, and its volume was replaced). Known limit in `07` §6.3. | **OPEN — filed 2026-10-06** **2026-10-06 night: verified in source, no code** — `offbox_reconstitute.go:758` (undo dump from the live DB), `:778-860` (files and volumes from the snapshot), `:804` (the snapshot's definition is written when its version differs), `:898` (the rollback loads the newer dump over the older volume; nothing writes the live definition back). Not a reorder fix: it needs R-638 option B (a rebuilding loader) or a pre-restore volume copy (disk cost; R-685's class). Which state a household gets after a failed off-site replay is the operator's call. Next: the 9202 measurement, then the design. | a design: R-638 option B (a loader that rebuilds instead of overlays) or a pre-restore volume copy | Measure it once on 9202 (a forced replay failure after an off-site restore over a migrated app); then a design for the operator | CC |
| **R-894** | Backup & restore | P3 | **After an agent restart, an UNREADABLE off-site storage makes the off-site tier look DUE, so the box asks for a copy that cannot be made.** MEASURED 2026-10-05 on demo-hp, read 2026-10-06 (`audits/readback-2026-10-07/C/`): the last off-site copy was 2026-10-01 20:15Z (ep0's own listing, verify `ok`), so the 7-day tier was NOT due; the agent had restarted at 04:57 local; at 06:25 `GET …/storage/felhom-pbs/content` answered 500 *Can't connect to 10.77.0.1:8007*; `newestArchiveOn` returned `unknown` and fell back to the in-memory record (`internal/localapi/server.go`), which a restart empties (`internal/backup/store.go` — memory only, R-348); so the tier read DUE, the controller requested it, and vzdump failed (*could not activate storage 'felhom-pbs'*). By design the controller stops the apps before it asks (`07` §6.4); whether it did that night is NOT KNOWN (the controller's log was lost to a later restart). The hub got `whole_guest_backup_failed` (error); its operator mail then failed (fixed in the hub this session: a failed operator mail is retried). The code's rule is deliberate: an unreadable storage must not suppress a backup. | **OPEN — filed 2026-10-06** **2026-10-06 night: fixed on agent main `74b5eae` (the row's first option — the newest success per tier on disk, read only when the storage cannot be read; `07` §6.1); ships with agent v0.150.0, closes when delivered.** | a design: keep the newest success per tier on disk (as `RestoreTestState` does) so the fallback is the last known copy, not "never"; or report an unreachable storage so no app is stopped for it | Decide the fallback; build it in the agent with a test that restarts the agent and then cannot read the storage; measure once | CC |
| **R-822** | Backup & restore | P2 | **An add-only key does not make retention safe: an attacker who can only ADD snapshots can make the honest pruner erase every real one.** MEASURED 2026-10-03 (lab, restic 0.14.0 from the controller image, rclone `--append-only`): 13 empty snapshots dated in the future with the same host and tag, added through the add-only key (all allowed), make the box's exact policy `forget --group-by host,tags --keep-daily 7 --keep-weekly 4 --keep-monthly 6` keep only the fakes and select **all 3 real snapshots** for removal (`--dry-run`). **Whoever prunes an append-only repo — the box in a window, or a Felhom-side worker — inherits this.** Not a defect today (today the box can simply delete, R-95); a PRECONDITION on the R-95 build, like R-430 was. `audits/offsite-append-only-2026-10-03/lab/C3-retention-poisoning.txt` | **NARROWED 2026-10-03 — the guard ships in controller v0.289.0 (future-dated / newer-than-hub / recent-removal refusals, oldest-first cap, the lab's 13-fake shape refused in a test). RESIDUAL, not closable by a guard: an add-only attacker can plant PAST-dated snapshots interleaved with real ones and so steer weekly/monthly keeps; bounded per window by `MaxRemove` and the hub's count check, not prevented.** **2026-10-06 night: design written** (`audits/night-burndown-2026-10-06/design-R-822.md`) — the past-dated residual can shrink real history to the last 7 days in ~2 windows, unalarmed, because the hub's window check trusts the box's own counts (`hub/internal/offsitekeys/service.go:284,343`); the same box-trusted count already sits under decision 68. Pick: accept + close (operator's word); the hub's own snapshot count is the new row R-895. | — | Before any `forget`: refuse when a snapshot is dated in the future or newer than the newest the hub has seen reported; dry-run first and abort above the count the policy can remove in a week; hub compares the count before/after (DESIGN.md §3) | CC |