admin/felhom.eu
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

rehearsal 2026-10-20: B6 #7 DST live — PASS (one database copy at the second 02:30); #8/#10 expectations written before

Browse Source
This commit is contained in:
authentik Default Admin
2026-10-21 09:13:35 +02:00
parent 3decdf94a8
commit a3be7525cd
9 changed files with 271 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
REPORT-rehearsal-2026-10-20.md
+12 -3
View File
@@ -76,9 +76,18 @@ names come back on the share; Nextcloud itself folds NFD into NFC); A.1.5 BookSt
## Night chain and DST
PENDING (morning). Code replay: the controller runs each daily job once on 2026-10-25 (a 02:xx job at the second,
CET occurrence); the hub runs every daily job twice (R-1183); spring 2027 puts the 02:30 dump on the 03:30
second-drive copy (`B6-failures/dst-replay/`).
**The first night (B5), no hands:** database copy 00:30Z PASS (3 m 27 s); second-drive copy 01:30Z PASS (8 apps + the
share, 58 s); off-site 02:15Z PASS (44 snapshots, 3 m 6 s, nothing deleted; the hub's own listing agrees: 44 files,
newest 02:18Z); whole-box backup NOT RUN — not due under the 24 h rule, because the install's own backup at 12:09Z
started the clock (R-1200, P3: the family's first day reaches a whole-box copy only on the second night). At 02:46Z
demo-hp's own kernel update rebooted the test host and shut the rehearsal VM down (harness, not a Tester4 defect; a BYO
host skips that step by design). Health 200 all night before that; memory steady. **The morning data snapshot is
identical to the evening one, byte for byte** (`B5-night/README.md`).
**DST (#7):** PENDING — running on the box now. So far: at the first 02:30 (summer time) the database copy did NOT run,
as the code replay said. Code replay: the controller runs each daily job once on 2026-10-25 (a 02:xx job at the second,
CET occurrence); the hub runs every daily job twice (R-1183); spring 2027 puts the 02:30 dump on the 03:30 second-drive
copy (`B6-failures/dst-replay/`).
## Interventions (operator-only acts)
documentation/audits/rehearsal-2026-10-20/B6-failures/s10-three-days-off/00-expected.txt
+6
View File
@@ -0,0 +1,6 @@
Written 2026-10-21 ~06:20 UTC, before #10 ran.
Method: clean stop; start with the RTC 3 days after the real time (NTP blocked), as if the box had been off for 3
days; during the catch-up, one restart (#15); then the real time, NTP open.
Expected: one catch-up for the missed nights (database copy, second drive, off-site), not three; the household sees a
„missed backups" note and then a catch-up line; the restart in the middle leaves nothing half-done — the catch-up
starts again or is recorded as interrupted, and no copy is left broken.
documentation/audits/rehearsal-2026-10-20/B6-failures/s7-dst/00-expected.txt
+10
View File
@@ -0,0 +1,10 @@
Written 2026-10-21 ~05:52 UTC, before #7 ran.
Method (plan amendment 20:27): NTP (UDP 123) and NTS-KE (TCP 4460) dropped at the VM's tap; the VM restarted cleanly
with its RTC at 2026-10-25T00:20:00Z (= 02:20 CEST); it runs ~80 real minutes, through 03:00 CEST → 02:00 CET, to
~02:40 CET (01:40Z). Positive control for the clock: the box's own HTTP Date header (a different channel than the RTC
setting). demo-hp's clock is not touched.
Expected from the code replay (B6-failures/dst-replay): the box's 02:30 database copy runs ONCE, at the SECOND
(CET) 02:30 = 01:30Z box time, not at 02:30 CEST (00:30Z). Nothing runs twice. The dashboard shows local times in CET
after the change.
Side effect of the method, not a DST result: the box wakes „4 days later" than its last backup; a missed-night
catch-up may start at once.
documentation/audits/rehearsal-2026-10-20/B6-failures/s7-dst/01-run.txt
+9
View File
@@ -0,0 +1,9 @@
}
ntp off 05:58:38
down 06:00:32
update VM 9320: -startdate 2026-10-25T00:20:00
started with RTC 2026-10-25T00:20:00 at real 06:00:35
real 2026-10-21T06:01:14Z box Sun, 25 Oct 2026 00:20:39 GMT
real 2026-10-21T07:12:43Z box Sun, 25 Oct 2026 01:32:08 GMT
iifname "tap9320i0" udp dport 123 counter packets 142 bytes 10792 drop
iifname "tap9320i0" tcp dport 4460 counter packets 0 bytes 0 drop
documentation/audits/rehearsal-2026-10-20/B6-failures/s7-dst/02-box-log-0020-0135Z.txt
+200
View File
@@ -0,0 +1,200 @@
2026-10-25T01:28:28Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:28:38Z INFO [scheduler] Running job: stack-scan
2026-10-25T01:28:38Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:28:38Z INFO [stacks] ScanStacks complete: 68 stacks found (8 deployed, 60 available)
2026-10-25T01:28:38Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:28:38Z INFO [scheduler] Job stack-scan completed (took 159ms)
2026-10-25T01:28:39Z INFO [scheduler] Running job: agent-channel-health
2026-10-25T01:28:39Z INFO [scheduler] Job agent-channel-health completed (took 532ms)
2026-10-25T01:28:48Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:28:58Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:29:08Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:29:18Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:29:28Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:29:28Z INFO Health probes: 1 ok (of 1 probed)
2026-10-25T01:29:38Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:29:39Z INFO [scheduler] Running job: agent-channel-health
2026-10-25T01:29:39Z INFO [scheduler] Job agent-channel-health completed (took 466ms)
2026-10-25T01:29:48Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:29:58Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:29:58Z INFO Health probes: 1 ok (of 1 probed)
2026-10-25T01:30:00Z INFO [scheduler] Running job: db-dump
2026-10-25T01:30:00Z INFO [backup] Starting database dump run
2026-10-25T01:30:00Z INFO [backup] Discovered 4 databases
2026-10-25T01:30:00Z INFO [backup] Discovered 4 database(s): paperless-postgres(postgres), nextcloud-db(mariadb), immich-postgres(postgres), bookstack-db(mariadb)
2026-10-25T01:30:00Z INFO [backup] DB dump: paperless-postgres → paperless-ngx-postgres.sql (490.0 KB, 449ms, 72 tables)
2026-10-25T01:30:00Z INFO [settings] Settings saved
2026-10-25T01:30:01Z INFO [backup] DB dump: nextcloud-db → nextcloud-mariadb.sql (3.4 MB, 762ms, 131 tables)
2026-10-25T01:30:01Z INFO [settings] Settings saved
2026-10-25T01:30:03Z INFO [backup] DB dump: immich-postgres → immich-postgres.sql (51.1 MB, 1.61s, 71 tables)
2026-10-25T01:30:03Z INFO [settings] Settings saved
2026-10-25T01:30:03Z INFO [backup] DB dump: bookstack-db → bookstack-mariadb.sql (71.9 KB, 363ms, 41 tables)
2026-10-25T01:30:03Z INFO [settings] Settings saved
2026-10-25T01:30:03Z INFO [backup] Stopping actualbudget for safe volume dump
2026-10-25T01:30:03Z INFO [stacks] Stopping stack: actualbudget
2026-10-25T01:30:04Z INFO [stacks] Stack actualbudget stopped successfully (took 0.3s)
2026-10-25T01:30:04Z INFO [stacks] Status refresh: 18 containers across 68 stacks
2026-10-25T01:30:04Z INFO [backup] Volume dump: actualbudget/actualbudget_actualbudget_data → 508.5 KB
2026-10-25T01:30:04Z INFO [backup] Restarting actualbudget after volume dump
2026-10-25T01:30:04Z INFO [stacks] Starting stack: actualbudget
2026-10-25T01:30:04Z INFO [stacks] Stack actualbudget started successfully (took 0.4s)
2026-10-25T01:30:05Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:30:05Z INFO [backup] Stopping bookstack for safe volume dump
2026-10-25T01:30:05Z INFO [stacks] Stopping stack: bookstack
2026-10-25T01:30:08Z INFO [stacks] Stack actualbudget post-start status:
2026-10-25T01:30:08Z INFO [stacks] actualbudget actualbudget/actual-server:26.9.0@sha256:552beab3dec8c93d46b8b9245612d63c3f123b8a45063a474f53e229b17621d3 running Up 3 seconds (health: starting)
2026-10-25T01:30:08Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:30:09Z INFO [stacks] Stack bookstack stopped successfully (took 4.5s)
2026-10-25T01:30:09Z INFO [stacks] Status refresh: 17 containers across 68 stacks
2026-10-25T01:30:10Z INFO [backup] Volume dump: bookstack/bookstack_bookstack_config → 353.0 KB
2026-10-25T01:30:11Z INFO [backup] Volume dump: bookstack/bookstack_bookstack_db_data → 153.5 MB
2026-10-25T01:30:11Z INFO [backup] Restarting bookstack after volume dump
2026-10-25T01:30:11Z INFO [stacks] Starting stack: bookstack
2026-10-25T01:30:18Z INFO [stacks] Stack bookstack started successfully (took 6.2s)
2026-10-25T01:30:18Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:30:18Z INFO [backup] Stopping immich for safe volume dump
2026-10-25T01:30:18Z INFO [stacks] Stopping stack: immich
2026-10-25T01:30:18Z INFO [stacks] Status refresh: 17 containers across 68 stacks
2026-10-25T01:30:18Z INFO Health probes: 1 ok (of 1 probed)
2026-10-25T01:30:19Z INFO [stacks] Stack immich stopped successfully (took 0.9s)
2026-10-25T01:30:19Z INFO [stacks] Status refresh: 15 containers across 68 stacks
2026-10-25T01:30:21Z INFO [stacks] Stack bookstack post-start status:
2026-10-25T01:30:21Z INFO [stacks] bookstack lscr.io/linuxserver/bookstack:26.09.1@sha256:99cd1f5707c1911afad213adec5c9739763b76f843d1477142231834ecdcb6f7 running Up 3 seconds (health: starting)
2026-10-25T01:30:21Z INFO [stacks] bookstack-db mariadb:12.3@sha256:805c8e104bd563d5bfa24fadd3f31cd419ea859cb5277f32b5dbf2db714f9ed1 running Up 9 seconds (healthy)
2026-10-25T01:30:29Z INFO [stacks] Status refresh: 15 containers across 68 stacks
2026-10-25T01:30:30Z INFO [backup] Volume dump: immich/immich_immich_ml_cache → 785.6 MB
2026-10-25T01:30:34Z INFO [backup] Volume dump: immich/immich_immich_postgres_data → 328.3 MB
2026-10-25T01:30:34Z INFO [backup] Restarting immich after volume dump
2026-10-25T01:30:34Z INFO [stacks] Starting stack: immich
2026-10-25T01:30:38Z INFO [scheduler] Running job: fill-watch-interval
2026-10-25T01:30:38Z INFO [fillwatch] checked 4 filesystem(s), 0 unreadable/skipped, 0 notification(s); bands: all ok
2026-10-25T01:30:38Z INFO [scheduler] Job fill-watch-interval completed (took 1ms)
2026-10-25T01:30:38Z INFO [scheduler] Running job: stack-scan
2026-10-25T01:30:38Z INFO [scheduler] Running job: backup-cache
2026-10-25T01:30:38Z INFO [scheduler] Running job: system-health
2026-10-25T01:30:38Z INFO [scheduler] Running job: docker-socket-users
2026-10-25T01:30:38Z INFO [scheduler] Running job: offsite-credential-retry
2026-10-25T01:30:38Z INFO [scheduler] Job offsite-credential-retry completed (took 0s)
2026-10-25T01:30:38Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:30:39Z INFO [stacks] ScanStacks complete: 68 stacks found (8 deployed, 60 available)
2026-10-25T01:30:39Z INFO [scheduler] Job docker-socket-users completed (took 274ms)
2026-10-25T01:30:39Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:30:39Z INFO [scheduler] Job stack-scan completed (took 302ms)
2026-10-25T01:30:39Z INFO [backup] Found 7 DB dump files across drives
2026-10-25T01:30:39Z INFO [monitor] Health check: status=ok
2026-10-25T01:30:39Z INFO [scheduler] Job system-health completed (took 334ms)
2026-10-25T01:30:39Z INFO [scheduler] Running job: agent-channel-health
2026-10-25T01:30:39Z INFO [backup] Discovered 4 databases
2026-10-25T01:30:39Z INFO [infra] cloudflared skipped — no cf_tunnel_token configured (LAN-only node)
2026-10-25T01:30:39Z INFO [backup] Recovery unit captured for actualbudget → /mnt/sys_drive/felhom-data/backups/primary/actualbudget (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
2026-10-25T01:30:39Z INFO [backup] Recovery unit captured for bookstack → /mnt/sys_drive/felhom-data/backups/primary/bookstack (images=2, secrets-referenced=3, data_keys=1, portable-carried=2/2, withheld=1)
2026-10-25T01:30:39Z INFO [backup] Recovery unit captured for immich → /mnt/felhom-drives/hdd_1/backups/primary/immich (images=4, secrets-referenced=1, data_keys=0, portable-carried=1/1, withheld=0)
2026-10-25T01:30:39Z INFO [backup] Recovery unit captured for nextcloud → /mnt/felhom-drives/hdd_1/backups/primary/nextcloud (images=3, secrets-referenced=3, data_keys=0, portable-carried=2/2, withheld=1)
2026-10-25T01:30:39Z INFO [backup] Recovery unit captured for paperless-ngx → /mnt/felhom-drives/hdd_1/backups/primary/paperless-ngx (images=3, secrets-referenced=3, data_keys=0, portable-carried=2/2, withheld=1)
2026-10-25T01:30:39Z INFO [backup] Backup status cache refreshed
2026-10-25T01:30:39Z INFO [scheduler] Job backup-cache completed (took 653ms)
2026-10-25T01:30:39Z INFO [scheduler] Job agent-channel-health completed (took 562ms)
2026-10-25T01:30:46Z INFO [stacks] Stack immich started successfully (took 11.2s)
2026-10-25T01:30:46Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:30:46Z INFO [backup] Stopping mealie for safe volume dump
2026-10-25T01:30:46Z INFO [stacks] Stopping stack: mealie
2026-10-25T01:30:48Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:30:48Z INFO Health probes: 1 ok (of 1 probed)
2026-10-25T01:30:49Z INFO [stacks] Stack immich post-start status:
2026-10-25T01:30:49Z INFO [stacks] immich-machine-learning ghcr.io/immich-app/immich-machine-learning:v3.2.4@sha256:e16c2f166a8174901959fdf85e2e4c7bd1ebc4b37e0b6655de97c41408a260c4 running Up 13 seconds (healthy)
2026-10-25T01:30:49Z INFO [stacks] immich-postgres ghcr.io/immich-app/postgres:16-vectorchord0.4.3-pgvectors0.2.0@sha256:1a078b237c1d9b420b0ee59147386b4aa60d3a07a8e6a402fc84a57e41b043a4 running Up 13 seconds (healthy)
2026-10-25T01:30:49Z INFO [stacks] immich-redis redis:7-alpine@sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499 running Up 14 seconds (healthy)
2026-10-25T01:30:49Z INFO [stacks] immich-server ghcr.io/immich-app/immich-server:v3.2.4@sha256:d317916b28090c33eb36b308464ea391f8b7df1d850fcfea227a39ec879718c2 running Up 3 seconds (health: starting)
2026-10-25T01:30:49Z INFO [stacks] Stack mealie stopped successfully (took 3.2s)
2026-10-25T01:30:49Z INFO [stacks] Status refresh: 18 containers across 68 stacks
2026-10-25T01:30:50Z INFO [backup] Volume dump: mealie/mealie_mealie_data → 1.6 MB
2026-10-25T01:30:50Z INFO [backup] Restarting mealie after volume dump
2026-10-25T01:30:50Z INFO [stacks] Starting stack: mealie
2026-10-25T01:30:50Z INFO [stacks] Stack mealie started successfully (took 0.4s)
2026-10-25T01:30:50Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:30:50Z INFO [backup] Stopping nextcloud for safe volume dump
2026-10-25T01:30:50Z INFO [stacks] Stopping stack: nextcloud
2026-10-25T01:30:52Z INFO [stacks] Stack nextcloud stopped successfully (took 1.8s)
2026-10-25T01:30:52Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-10-25T01:30:53Z INFO [stacks] Stack mealie post-start status:
2026-10-25T01:30:53Z INFO [stacks] mealie ghcr.io/mealie-recipes/mealie:v3.28.0@sha256:8b02290f4d1806f02acac6f25f6d48a3c965612fda1f8e914d5af533276f8688 running Up 3 seconds (health: starting)
2026-10-25T01:30:54Z INFO [backup] Volume dump: nextcloud/nextcloud_nextcloud_db_data → 203.4 MB
2026-10-25T01:30:58Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-10-25T01:31:08Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-10-25T01:31:18Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-10-25T01:31:18Z INFO Health probes: 2 ok (of 2 probed)
2026-10-25T01:31:21Z INFO [backup] Volume dump: nextcloud/nextcloud_nextcloud_html → 761.6 MB
2026-10-25T01:31:21Z INFO [backup] Restarting nextcloud after volume dump
2026-10-25T01:31:21Z INFO [stacks] Starting stack: nextcloud
2026-10-25T01:31:27Z INFO [stacks] Stack nextcloud started successfully (took 6.5s)
2026-10-25T01:31:27Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:31:27Z INFO [backup] Stopping paperless-ngx for safe volume dump
2026-10-25T01:31:27Z INFO [stacks] Stopping stack: paperless-ngx
2026-10-25T01:31:28Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:31:30Z INFO [stacks] Stack nextcloud post-start status:
2026-10-25T01:31:30Z INFO [stacks] nextcloud nextcloud:34.0.4-apache@sha256:37b109885aa3cba3e056362a899556a625c986f2c17cd2c0cc157d21c789f53b running Up 3 seconds (health: starting)
2026-10-25T01:31:30Z INFO [stacks] nextcloud-db mariadb:12.3@sha256:805c8e104bd563d5bfa24fadd3f31cd419ea859cb5277f32b5dbf2db714f9ed1 running Up 9 seconds (healthy)
2026-10-25T01:31:30Z INFO [stacks] nextcloud-redis redis:7-alpine@sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499 running Up 9 seconds (healthy)
2026-10-25T01:31:35Z INFO [web] Login from 192.168.0.180
2026-10-25T01:31:37Z INFO [stacks] Stack paperless-ngx stopped successfully (took 9.5s)
2026-10-25T01:31:37Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-10-25T01:31:37Z INFO [backup] Volume dump: paperless-ngx/paperless-ngx_paperless_data → 1.1 MB
2026-10-25T01:31:38Z INFO [stacks] Status refresh: 16 containers across 68 stacks
2026-10-25T01:31:38Z INFO Health probes: 3 ok (of 3 probed)
2026-10-25T01:31:38Z INFO [deadapp] check alive: 140 scans since boot, 8 deployed app(s) evaluated, 0 currently down
2026-10-25T01:31:38Z INFO [deadapp] 1 stopped app(s) have NO recorded customer intent, so their dead-app alarm is suppressed by the unknown-intent fallback (R-386): paperless-ngx. This closes itself as each app is started or stopped through the interface.
2026-10-25T01:31:39Z INFO [scheduler] Running job: agent-channel-health
2026-10-25T01:31:39Z INFO [backup] Volume dump: paperless-ngx/paperless-ngx_paperless_postgres_data → 70.4 MB
2026-10-25T01:31:39Z INFO [scheduler] Job agent-channel-health completed (took 753ms)
2026-10-25T01:31:40Z INFO [backup] Volume dump: paperless-ngx/paperless-ngx_paperless_redis_data → 2.5 KB
2026-10-25T01:31:40Z INFO [backup] Restarting paperless-ngx after volume dump
2026-10-25T01:31:40Z INFO [stacks] Starting stack: paperless-ngx
2026-10-25T01:31:47Z INFO [web] Login from 192.168.0.180
2026-10-25T01:31:48Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:31:50Z INFO [stacks] Stack paperless-ngx started successfully (took 10.9s)
2026-10-25T01:31:51Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:31:51Z INFO [backup] Stopping radicale for safe volume dump
2026-10-25T01:31:51Z INFO [stacks] Stopping stack: radicale
2026-10-25T01:31:51Z INFO [stacks] Stack radicale stopped successfully (took 0.7s)
2026-10-25T01:31:51Z INFO [stacks] Status refresh: 18 containers across 68 stacks
2026-10-25T01:31:52Z INFO [backup] Volume dump: radicale/radicale_radicale_data → 15.5 KB
2026-10-25T01:31:52Z INFO [backup] Restarting radicale after volume dump
2026-10-25T01:31:52Z INFO [stacks] Starting stack: radicale
2026-10-25T01:31:52Z INFO [stacks] Stack radicale started successfully (took 0.5s)
2026-10-25T01:31:53Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:31:53Z INFO [backup] Stopping vaultwarden for safe volume dump
2026-10-25T01:31:53Z INFO [stacks] Stopping stack: vaultwarden
2026-10-25T01:31:53Z INFO [stacks] Stack vaultwarden stopped successfully (took 0.4s)
2026-10-25T01:31:53Z INFO [stacks] Status refresh: 18 containers across 68 stacks
2026-10-25T01:31:54Z INFO [backup] Volume dump: vaultwarden/vaultwarden_vaultwarden_data → 371.5 KB
2026-10-25T01:31:54Z INFO [backup] Restarting vaultwarden after volume dump
2026-10-25T01:31:54Z INFO [stacks] Starting stack: vaultwarden
2026-10-25T01:31:54Z INFO [stacks] Stack paperless-ngx post-start status:
2026-10-25T01:31:54Z INFO [stacks] paperless-postgres postgres:18-alpine@sha256:77f585114c32fbca283dc835b0596f4e52b51b4c6662d7810b2f4084f60a1873 running Up 13 seconds (healthy)
2026-10-25T01:31:54Z INFO [stacks] paperless-redis redis:7-alpine@sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499 running Up 14 seconds (healthy)
2026-10-25T01:31:54Z INFO [stacks] paperless-webserver ghcr.io/paperless-ngx/paperless-ngx:2.20.15@sha256:6c86cad803970ea782683a8e80e7403444c5bf3cf70de63b4d3c8e87500db92f running Up 3 seconds (health: starting)
2026-10-25T01:31:54Z INFO [stacks] Stack vaultwarden started successfully (took 0.7s)
2026-10-25T01:31:55Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:31:55Z INFO [backup] App-data backup completed: 4 databases (55.1 MB total), 8 volume dump(s) (1m55.223s)
2026-10-25T01:31:55Z INFO [backup] Recovery unit captured for mealie → /mnt/sys_drive/felhom-data/backups/primary/mealie (images=1, secrets-referenced=1, data_keys=0, portable-carried=0/0, withheld=1)
2026-10-25T01:31:55Z INFO [backup] Recovery unit captured for nextcloud → /mnt/felhom-drives/hdd_1/backups/primary/nextcloud (images=3, secrets-referenced=3, data_keys=0, portable-carried=2/2, withheld=1)
2026-10-25T01:31:55Z INFO [backup] Recovery unit captured for paperless-ngx → /mnt/felhom-drives/hdd_1/backups/primary/paperless-ngx (images=3, secrets-referenced=3, data_keys=0, portable-carried=2/2, withheld=1)
2026-10-25T01:31:55Z INFO [backup] Recovery unit captured for radicale → /mnt/sys_drive/felhom-data/backups/primary/radicale (images=1, secrets-referenced=1, data_keys=0, portable-carried=0/0, withheld=1)
2026-10-25T01:31:55Z INFO [backup] Recovery unit captured for vaultwarden → /mnt/sys_drive/felhom-data/backups/primary/vaultwarden (images=1, secrets-referenced=1, data_keys=0, portable-carried=0/0, withheld=1)
2026-10-25T01:31:55Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:31:55Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:31:55Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:31:55Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:31:55Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:31:55Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:31:55Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:31:55Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:31:55Z INFO [scheduler] Job db-dump completed (took 1m55.675s)
2026-10-25T01:31:55Z INFO Event pushed: db_dump_completed (info) [hu-only] — Adatbázis mentés elkészült
2026-10-25T01:31:56Z INFO [stacks] Stack radicale post-start status:
2026-10-25T01:31:56Z INFO [stacks] radicale ghcr.io/kozea/radicale:3.8.1@sha256:54d9406cd30f9e206a9dd6d61dfac48da26b9afc37200e14f23beb8fafa7d11c running Up 3 seconds (health: starting)
2026-10-25T01:31:58Z INFO [stacks] Stack vaultwarden post-start status:
2026-10-25T01:31:58Z INFO [stacks] vaultwarden vaultwarden/server:1.37.4-alpine@sha256:19ea2b669945d842dadc3d500f2d6a795866a003330f59a47b9634658bae98bb running Up 3 seconds (health: starting)
2026-10-25T01:31:58Z INFO [stacks] Status refresh: 19 containers across 68 stacks
2026-10-25T01:32:08Z INFO [web] Login from 192.168.0.180
2026-10-25T01:32:08Z INFO [web] Login from 192.168.0.180
documentation/audits/rehearsal-2026-10-20/B6-failures/s7-dst/README.md
+23
View File
@@ -0,0 +1,23 @@
# B6 ★7 — the clocks go back (2026-10-25 03:00 CEST → 02:00 CET), live on the rehearsal box — AS EXPECTED
Run 2026-10-21 05:58–07:13 real UTC. Method (`00-expected.txt`): NTP dropped at the VM's tap (142 packets dropped —
positive control), the VM restarted cleanly with its RTC at 2026-10-25 00:20:00Z (02:20 CEST), then left alone.
Clock control: the box's own HTTP Date header, read 5 times (`01-run.txt`). demo-hp's clock untouched.
| box time (UTC / local) | what happened | expected? |
|---|---|---|
| 00:20Z / 02:20 CEST | boot; all apps up in ~1 min | — |
| 00:25:42Z / 02:25 CEST | the box found the whole-box tier „due" (its clock says 4 days passed) → 8 apps paused 28 s, local whole-box backup written as `vzdump-lxc-9201-2026_10_25-02_26_01` (4.3 GB); a restore test at 02:51 CEST passed. The weekly PBS tier was not due — nothing went to ep0 | side effect of the method (written before) |
| 00:30Z / **first 02:30 (CEST)** | **no database copy** — only the routine jobs | YES (the replay) |
| 00:37–00:41Z | a missed-night catch-up: second drive (8 apps + share) and off-site (dump leg 2 m, 8 apps; retention skipped). The off-site snapshots are **dated 2026-10-25** — 4 days ahead of the real date | side effect of the method |
| 01:00Z | 03:00 CEST → 02:00 CET; the dashboard's ages stay right („36 perce") | YES |
| 01:30:00Z / **second 02:30 (CET)** | **the database copy ran, once**: 4 databases + 8 volume dumps, 1 m 55 s, event `db_dump_completed` | YES (the replay) |
**Verdict: PASS for the box.** Each daily job runs once on the DST night, at the second 02:30. The hub's double run
(R-1183) is not covered by this test (it is on the real hub, on the real Sunday).
**Text gap (W30):** inside the repeated hour the backups page shows times without a summer/winter mark: at 02:02 CET it
says „Utolsó teljes mentés 2026. 10. 25. 02:26 (36 perce)" — a time that looks later than the clock.
**What the method left behind (for #8):** a local whole-box archive and off-site snapshots dated 4 days in the future.
Evidence: `02-box-log-0020-0135Z.txt` (the box's own log ring), the night copier `../../B5-night/logs/`.
documentation/audits/rehearsal-2026-10-20/B6-failures/s8-wrong-clock/00-expected.txt
+9
View File
@@ -0,0 +1,9 @@
Written 2026-10-21 ~06:20 UTC, before #8 ran.
Method: NTP stays blocked; clean restart with the RTC at (real − 2 years) for ~12 min; then at (real + 2 years) for
~12 min; then the real time with NTP open again. Clock control: the box's HTTP Date header.
Expected: at −2 y nothing deletes or replaces a newer copy; nothing is „due" that is not; the box and the hub should
say the clock is wrong (if they can see it). At +2 y: every backup looks 2 years old → catch-up runs; off-site
retention must NOT delete the real snapshots (the guard refuses future/too-new plans); a local whole-box archive
dated +2 y is written. After the return: the D3/T11 question — does the whole-box tier go silent because its newest
archive is dated in the future, and does anything (box page, hub) say so?
Note: #7 already left a whole-box archive dated 2026-10-25 (4 days ahead) — the same question in small.
documentation/audits/rehearsal-2026-10-20/B9-words/words.md
+1
View File
@@ -31,3 +31,4 @@
| W27 | an English BROWSER | the whole dashboard stays Hungarian with `en-US` — it follows the household's setting, not the browser | by design (a visitor switch exists for signed-out pages) | — (NOT-A-FINDING: the household's setting wins, `10-localisation.md`) |
| W28 | share restore | „Nem állítható vissza (a mappa nincs élő adatmeghajtón): beolvasas." | names a share („beolvasas") the household never created | say what it is (Paperless's scan folder?) or skip it silently |
| W29 | storage card after a plain decommission | „Kiváltva — Adatok átköltöztetve ide: . () Időpont: …" | the migration sentence shown with empty blanks when nothing was moved | „Leszerelve, adat nem volt rajta" |
| W30 | backups page inside the repeated hour (B6 #7, 2026-10-25 02:02 CET) | „Utolsó teljes mentés 2026. 10. 25. 02:26 (36 perce)" | the time looks later than the clock: no summer/winter mark during the one repeated hour a year | show „(nyári idő)" / „(téli idő)" only when the time falls in the repeated hour; low priority |
documentation/audits/rehearsal-2026-10-20/journal.md
+1
View File
@@ -83,3 +83,4 @@
| 05:36 | B5 | morning snapshot IDENTICAL to the evening one (`B5-night/03-…`) |
| 05:27–05:45 | due checks | R-1019 closed (demo-felhom read-only, two channels), R-895 first verdict recorded, R-435 moved to 10-27 (logs gone) |
| 05:40 | mail | the nightly DooPlex secret sweep named three rehearsal secrets in session records: OLD_BIND_TOKEN, BIND_TOKEN, SETUP_CODE — my mail reads (full text of the bind and setup mails) carried them in; all three are one-time and already spent. From now on mail is read names-only |
| 05:58–07:13 | B6 ★7 | DST live: the first 02:30 skipped, the second ran once (01:30:00Z box), PASS; side effects of the 4-day jump: a local whole-box archive and off-site snapshots dated 2026-10-25; W30 |