hub v0.136.0: nightly VACUUM INTO snapshot, keep 2 (R-173 decision A); hub PVC 2Gi + Longhorn default group; 09 rulings 125-127; 05 §16.3
gates / gates (push) Failing after 14m0s
gates / gates (push) Failing after 14m0s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
@@ -444,5 +444,18 @@ the global-key API) open through `GetHostRecoveryCredential`, so the break-glass
|
||||
use). **And the key is the other half:** a backup of the database restores a hub that can open the sealed columns only
|
||||
with the same `OFFSITE_SECRET_KEY`; today that key exists only on DooPlex (the k8s Secret, and the GPG secrets export
|
||||
on the same machine). The off-site plan for the database and its key: `runbooks/RUNBOOK-hub-db-offsite-backup.md`
|
||||
(R-173 — awaiting the operator's decision).
|
||||
(R-173 — option A decided 2026-10-05, `09` decision 125; §16.3).
|
||||
|
||||
### 16.3 The nightly database snapshot (R-173, hub v0.136.0)
|
||||
|
||||
Every night at **02:00 Budapest** the hub writes `<data>/snapshots/hub-<UTC stamp>.db` with SQLite's `VACUUM INTO`:
|
||||
one statement, one point in time, every committed write included — the rows still in `hub.db-wal` too, which a file copy
|
||||
of `hub.db` loses (measured in `internal/dbsnap` tests: a plain copy held 0 of 150 fresh rows). Written as `.tmp`, mode
|
||||
`0600`, then renamed, so a reader never sees half a file. The newest **2** stay (the volume grew from 1 GiB to 2 GiB
|
||||
for them, operator choice 2026-10-05; one snapshot measured 370 MB). Two runs never overlap (a second gets `ErrBusy`).
|
||||
Each run logs `db snapshot written: <name> (<bytes>, <duration>)`. At start-up the hub runs one when the newest is older
|
||||
than 24 h. **The hub ships nothing itself and holds no ep0 credential:** DooPlex's `felhom-hub-db-backup` unit picks the
|
||||
newest snapshot up at 02:30, checks it (`PRAGMA integrity_check`), encrypts it and pushes it to ep0's `operator`
|
||||
namespace (`runbooks/RUNBOOK-hub-db-offsite-backup.md`). Pinned by `internal/dbsnap/dbsnap_test.go` and
|
||||
`cmd/hub/r173_wiring_test.go`.
|
||||
|
||||
|
||||
@@ -881,6 +881,16 @@ its length, and both fixes cost something the household would notice — operato
|
||||
`<ver>-step1` — then the release's bundle, both by signed jobs. **Chosen (b)**, `felhom-agent/scripts/build-step-bundle.py`;
|
||||
delivered to demo-hp, demo-felhom and Tester 1 on 2026-10-05. `11` §5.4.2 rule unchanged.
|
||||
|
||||
### 2026-10-05 (14:05) — three operator rulings (recorded before the work; the hub-DB off-site brief)
|
||||
|
||||
125. **The hub database's off-site copy goes to ep0's backup server** (option A of the hub-safety STATUS decision):
|
||||
encrypted on DooPlex, pushed to a write-only namespace on ep0, restore-tested weekly, alarmed. Rejected: B, a separate
|
||||
Hetzner Storage Box account (more new parts to maintain). *Operator ruling 2026-10-05.* (R-173)
|
||||
126. **R-519's live test is approved:** one controller restart on scratch 9202 in the middle of a backup. *Operator ruling
|
||||
2026-10-05.*
|
||||
127. **The agent's three by-design abilities (`03` §3.1) stay for now**; revisited before the first paying customer.
|
||||
*Operator ruling 2026-10-05.* (R-861)
|
||||
|
||||
### 2026-10-05 (06:49) — four operator rulings (recorded before the work; the night-fixes brief)
|
||||
|
||||
100. **Tester 1's Cloudflare tokens, shown in the 2026-10-04 night session's output, are NOT rotated** (option B) —
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
### R1 — SnapshotInto copies hub.db as a file instead of VACUUM INTO
|
||||
=== RUN TestSnapshot_ConsistentWithLiveDBIncludingWAL
|
||||
dbsnap_test.go:104: table hosts: snapshot 0 rows, live 150
|
||||
dbsnap_test.go:104: table hub_settings: snapshot 0 rows, live 2
|
||||
--- FAIL: TestSnapshot_ConsistentWithLiveDBIncludingWAL (0.05s)
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-hub/internal/dbsnap 0.055s
|
||||
FAIL
|
||||
|
||||
### R2 — no busy check (two runs may overlap)
|
||||
=== RUN TestSnapshot_NeverTwoAtOnce
|
||||
/mnt/5_hdd/felhom.eu/git/felhom.eu/hub/internal/dbsnap/dbsnap_test.go:141 +0x76
|
||||
/mnt/5_hdd/felhom.eu/git/felhom.eu/hub/internal/dbsnap/dbsnap_test.go:153 +0x1a6
|
||||
/mnt/5_hdd/felhom.eu/git/felhom.eu/hub/internal/dbsnap/dbsnap_test.go:141 +0x76
|
||||
/mnt/5_hdd/felhom.eu/git/felhom.eu/hub/internal/dbsnap/dbsnap_test.go:151 +0x34
|
||||
/mnt/5_hdd/felhom.eu/git/felhom.eu/hub/internal/dbsnap/dbsnap_test.go:151 +0x178
|
||||
FAIL gitea.dooplex.hu/admin/felhom-hub/internal/dbsnap 600.107s
|
||||
FAIL
|
||||
|
||||
### R3 — prune keeps everything
|
||||
=== RUN TestSnapshot_KeepsNewestTwo
|
||||
dbsnap_test.go:130: kept [hub-20261006T000000Z.db hub-20261007T000000Z.db hub-20261008T000000Z.db], want [hub-20261007T000000Z.db hub-20261008T000000Z.db]
|
||||
--- FAIL: TestSnapshot_KeepsNewestTwo (0.07s)
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-hub/internal/dbsnap 0.075s
|
||||
FAIL
|
||||
|
||||
### R4 — a failed write leaves its .tmp
|
||||
=== RUN TestSnapshot_FailureLeavesNoFile
|
||||
dbsnap_test.go:191: left 1 file(s) behind
|
||||
--- FAIL: TestSnapshot_FailureLeavesNoFile (0.00s)
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-hub/internal/dbsnap 0.007s
|
||||
FAIL
|
||||
|
||||
ok gitea.dooplex.hu/admin/felhom-hub/internal/dbsnap 0.135s
|
||||
|
||||
[exited with code 0]
|
||||
@@ -0,0 +1,26 @@
|
||||
Run 1 (R1–R4) in red-proof-run1.txt. R2 there HUNG (600 s timeout) — the old test blocked forever; it convicted by hanging. The test now fails cleanly; R2 re-run below.
|
||||
|
||||
### R2 (re-run) — no busy check
|
||||
=== RUN TestSnapshot_NeverTwoAtOnce
|
||||
dbsnap_test.go:162: second Make did not return ErrBusy at once: it ran alongside the first
|
||||
--- FAIL: TestSnapshot_NeverTwoAtOnce (2.00s)
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-hub/internal/dbsnap 2.007s
|
||||
FAIL
|
||||
|
||||
### R5 — main never schedules the snapshot
|
||||
=== RUN TestR173_MainSchedulesTheDBSnapshot
|
||||
r173_wiring_test.go:42: cmd/hub/main.go never calls scheduleDaily(ctx, "db-snapshot", "02:00", …) — no nightly snapshot
|
||||
--- FAIL: TestR173_MainSchedulesTheDBSnapshot (0.00s)
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-hub/cmd/hub 0.025s
|
||||
FAIL
|
||||
|
||||
### R6 — main has no start-up catch-up
|
||||
=== RUN TestR173_MainSchedulesTheDBSnapshot
|
||||
r173_wiring_test.go:45: cmd/hub/main.go never calls dbsnap.NeedsCatchUp — a pod down at 02:00 leaves a stale snapshot
|
||||
--- FAIL: TestR173_MainSchedulesTheDBSnapshot (0.00s)
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-hub/cmd/hub 0.023s
|
||||
FAIL
|
||||
|
||||
Reference in New Issue
Block a user