night 09-27/28: paperless 16 copy released on a real 18 dump (v0.276.0); demo-hp restore test refused for space -> R-701
gates / gates (push) Successful in 25s

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
2026-09-28 03:46:43 +02:00
parent 06744dbacc
commit 4c31536909
6 changed files with 62 additions and 3 deletions
@@ -0,0 +1,31 @@
read at 2026-09-28T01:45:38Z
conversion_copy null earlier null
paperless-ngx_paperless_data
paperless-ngx_paperless_postgres_data
paperless-ngx_paperless_redis_data
ls: cannot access '/mnt/sys_drive/felhom-data/backups/primary/paperless-ngx/db-dumps': No such file or directory
*.sql:
2026/09/28 01:30:45 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container paperless-webserver (image=ghcr.io/paperless-ngx/paperless-ngx:2.20.15, not a database)
2026/09/28 01:30:45 dbdump.go:178: [DEBUG] DiscoverDatabases: found postgres container: paperless-postgres (id=8e904f9d109a)
2026/09/28 01:30:45 dbdump.go:896: [DEBUG] DiscoverDatabases: paperless-postgres → stack "paperless-ngx" from the compose project label (the container name would have given "paperless")
2026/09/28 01:30:45 dbdump.go:198: [DEBUG] DiscoverDatabases: paperless-postgres → stack=paperless-ngx, dbUser=paperless, dbName=paperless
2026/09/28 01:30:45 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container paperless-redis (image=redis:7-alpine, not a database)
2026/09/28 01:35:45 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container paperless-webserver (image=ghcr.io/paperless-ngx/paperless-ngx:2.20.15, not a database)
2026/09/28 01:35:45 dbdump.go:178: [DEBUG] DiscoverDatabases: found postgres container: paperless-postgres (id=8e904f9d109a)
2026/09/28 01:35:45 dbdump.go:896: [DEBUG] DiscoverDatabases: paperless-postgres → stack "paperless-ngx" from the compose project label (the container name would have given "paperless")
2026/09/28 01:35:45 dbdump.go:198: [DEBUG] DiscoverDatabases: paperless-postgres → stack=paperless-ngx, dbUser=paperless, dbName=paperless
2026/09/28 01:35:45 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container paperless-redis (image=redis:7-alpine, not a database)
2026/09/28 01:40:45 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container paperless-webserver (image=ghcr.io/paperless-ngx/paperless-ngx:2.20.15, not a database)
2026/09/28 01:40:45 dbdump.go:178: [DEBUG] DiscoverDatabases: found postgres container: paperless-postgres (id=8e904f9d109a)
2026/09/28 01:40:45 dbdump.go:896: [DEBUG] DiscoverDatabases: paperless-postgres → stack "paperless-ngx" from the compose project label (the container name would have given "paperless")
2026/09/28 01:40:45 dbdump.go:198: [DEBUG] DiscoverDatabases: paperless-postgres → stack=paperless-ngx, dbUser=paperless, dbName=paperless
2026/09/28 01:40:45 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container paperless-redis (image=redis:7-alpine, not a database)
2026/09/28 01:00:45 pgconvert.go:702: [INFO] [stacks] paperless-ngx: REMOVED the pre-conversion datadir copy paperless-ngx_paperless_postgres_data.pre-update-20260927T094421Z (PostgreSQL 16) — the converted app has a backup proven on 18: Tier 1 (own recovery unit) at 2026-09-28T00:30:00Z, its dump db-dumps/paperless-ngx-postgres.sql written 2026-09-28T00:30:00Z by map[paperless-postgres:postgres:18-alpine@sha256:77f585114c32fbca283dc835b0596f4e52b51b4c6662d7810b2f4084f60a1873 paperless-redis:redis:7-alpine@sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499 paperless-webserver:ghcr.io/paperless-ngx/paperless-ngx:2.20.15@sha256:6c86cad803970ea782683a8e80e7403444c5bf3cf70de63b4d3c8e87500db92f]
unit=
ls: cannot access '/db-dumps': No such file or directory
*.sql:
/mnt/felhom-drives/scratch_hdd/userdata/paperless-ngx/backups/primary/paperless-ngx/db-dumps/paperless-ngx-postgres.sql 2026-09-28 00:30:00.404026450 +0000 :: Dumped from database version 18.6
/mnt/felhom-drives/scratch_hdd/userdata/paperless-ngx/backups/primary/paperless-ngx/db-dumps/pre-restore-20260927T094411Z-paperless-ngx-postgres.sql 2026-09-27 09:44:12.287984914 +0000 :: Dumped from database version 16.15
@@ -0,0 +1,14 @@
# Night 2026-09-27/28 — read at 01:45Z (03:45 CEST), read-only
**9202, paperless-ngx: the pre-conversion copy was released through v0.276.0's rewritten loop.** `N1-9202-paperless-release.txt`:
- 00:30:00Z the nightly data run wrote `db-dumps/paperless-ngx-postgres.sql`; the file itself says "Dumped from database
version 18.6" (the older `pre-restore-…` undo dump says 16.15 and was, correctly, not the one used).
- 01:00:45Z the hourly job logged: „REMOVED the pre-conversion datadir copy
paperless-ngx_paperless_postgres_data.pre-update-20260927T094421Z (PostgreSQL 16) — … Tier 1 (own recovery unit) at
2026-09-28T00:30:00Z, its dump … written 2026-09-28T00:30:00Z by … postgres:18-alpine@sha256:77f5…".
- After: the volume is gone from `docker volume ls`; `conversion_copy` and `earlier_conversion_copies` are both empty.
Before v0.275.0 this copy went at the first tick after a refresh, on a 16 dump (R-696). Here it waited for a real 18 dump.
**demo-hp restore test** (the earlier session's D1 item, read here): see `version-travel-2026-09-26/D1/D1-cycle-demo-hp.txt`
— the pick is right (R-689 closed correctly), but the test is refused for space every cycle → R-701 filed.
@@ -0,0 +1,12 @@
# demo-hp restore-test cycles after agent 0.137.0 (restart 2026-09-27 14:13:17 CEST) — read 2026-09-28T01:46:09Z
felhom-agent 0.137.0
Sep 27 14:13:16 demo-hp felhom-agent[1319775]: time=2026-09-27T14:13:16.508+02:00 level=INFO msg="backup: restore-test scheduler shutting down" reason="context canceled"
Sep 27 14:13:17 demo-hp felhom-agent[1370926]: time=2026-09-27T14:13:17.506+02:00 level=INFO msg="backup: restore-test scheduler starting (per-archive due-check)" eval_interval=6h0m0s settle=24h0m0s
Sep 27 20:13:17 demo-hp felhom-agent[1370926]: time=2026-09-27T20:13:17.836+02:00 level=INFO msg="backup: restore-test skips an entry that is not a backup of a guest" target=local volid=local:backup/felhom-golden-0.236.0.tar.zst size_bytes=654115664 reason="not a backup of a guest (the storage reports no vmid)"
Sep 27 20:13:17 demo-hp felhom-agent[1370926]: time=2026-09-27T20:13:17.845+02:00 level=INFO msg="backup: restore-test skips an entry that is not a backup of a guest" target=local volid=local:backup/vzdump-lxc-9100-2026_08_21-17_59_15.tar.zst size_bytes=656970239 reason="guest 9100 does not exist on this node or is not one this agent manages"
Sep 27 20:13:17 demo-hp felhom-agent[1370926]: time=2026-09-27T20:13:17.853+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z landed=2026-09-24T20:06:25Z reason="newest settled archive (landed 2026-09-24T20:06:25Z) has not been proven (last proven archive was a different one)"
Sep 27 20:13:19 demo-hp felhom-agent[1370926]: time=2026-09-27T20:13:19.466+02:00 level=WARN msg="restore-test SKIPPED by the space preflight (R-672) — nothing was created" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z storage=local-lvm required_bytes=33252620981 avail_bytes=22657356320 reason="not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has 21.1 GiB"
Sep 27 20:13:19 demo-hp felhom-agent[1370926]: time=2026-09-27T20:13:19.467+02:00 level=ERROR msg="backup: scheduled restore-test FAILED" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z err="skipped: not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has 21.1 GiB"
Sep 28 02:13:17 demo-hp felhom-agent[1370926]: time=2026-09-28T02:13:17.870+02:00 level=INFO msg="backup: restore-test tier is DUE (per-archive; oldest-proven first among due tiers)" target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z landed=2026-09-24T20:06:25Z reason="newest settled archive (landed 2026-09-24T20:06:25Z) has not been proven (last proven archive was a different one)"
Sep 28 02:13:19 demo-hp felhom-agent[1370926]: time=2026-09-28T02:13:19.630+02:00 level=WARN msg="restore-test SKIPPED by the space preflight (R-672) — nothing was created" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z storage=local-lvm required_bytes=33252620982 avail_bytes=22657356320 reason="not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has 21.1 GiB"
Sep 28 02:13:19 demo-hp felhom-agent[1370926]: time=2026-09-28T02:13:19.630+02:00 level=ERROR msg="backup: scheduled restore-test FAILED" archive=felhom-pbs:backup/ct/9201/2026-09-24T20:06:25Z err="skipped: not enough space on local-lvm: restoring 21.6 GiB (pbs snapshot size) needs 31.0 GiB free, has 21.1 GiB"