R-411/408/407, R-414, R-412a leg 1, R-410, R-406 CLOSED; determination + live evidence

Part 2.1's determination is the first artifact: the scratch resolver was consciously OUT OF SCOPE for R-356, not excluded on state-only grounds - established from R-356's own commit 08eb1a6, whose tests say 'the prepared scratch still resolves ... only the DESTINATION moves'. So 07 section 6.3's rule applies and now has a FOURTH consumer, and the section says so. Live evidence: the collision rerun on demo-hp with the sampler positively controlled first (12 locks=1 across a real check, 4 locks=0 quiet), showing unlock --remove-all 0 times where the drill saw it twice; and the proof reaching verdict pass on demo-felhom - the box that could not run it at all - recorded where last_proof_result had been ABSENT every night. Capability map: the off-site proof row now records that the nightly firing IS proven (it ran unattended at 05:30 on demo-hp) and that a driveless box can now be proved. Register: six rows closed and compressed. OPEN 176 -> 170, CLOSED 152 -> 158.
2026-09-01 10:36:54 +02:00
parent 22e1c95e6a
commit f41a1a0ad8
11 changed files with 148 additions and 8 deletions
@@ -0,0 +1,30 @@
+=== the collision, three overlap offsets: full restore FIRST, then the check into it ===
+--- restore, then check +1s ---
+t_restore=08:24:09
+[http=302 wall=0.010349s]
+t_check=08:24:10
+"duration_ms":0 "ok":false "skip_reason":"a backup or restore is already running" "skipped":true "ok":true 
+--- restore, then check +2s ---
+t_restore=08:24:22
+[http=302 wall=0.010791s]
+t_check=08:24:24
+"duration_ms":39978 "ok":true "skip_reason":"" "skipped":false "ok":true 
+--- restore, then check +3s ---
+t_restore=08:25:19
+[http=302 wall=0.011009s]
+t_check=08:25:22
+"duration_ms":0 "ok":false "skip_reason":"a backup or restore is already running" "skipped":true "ok":true 
+
+=== THE NON-EFFECT: unlock --remove-all must appear in NO argv ===
+  'unlock --remove-all' in the sampler: 0
+  any 'unlock' at all:                  0
+  'cleared a stale exclusive lock' in the log: 0
+
+=== and the check SKIPPED rather than colliding ===
+integrity: check PASSED in 1m4s (structure, index, and 100% of the pack data re-read)
+integrity: skipped — a backup or restore is already running; due-ness is NOT advanced, so this retries on the next run
+off-box restore refused for kimai (mode=full): another backup/restore op is already running
+off-box restore kimai completed (full=true, async)
+integrity: check PASSED in 40s (structure, index, and 100% of the pack data re-read)
+integrity: skipped — a backup or restore is already running; due-ness is NOT advanced, so this retries on the next run
+off-box restore kimai completed (full=true, async)
@@ -0,0 +1,7 @@
+=== demo-felhom BEFORE: still zero registered storage paths? ===
+  registered storage paths: 0
+  last_proof_result: '<ABSENT>'
+
+=== deploy 0.232.0 ===
+deployed
+gitea.dooplex.hu/admin/felhom-controller:0.232.0 Up 26 seconds (healthy)
@@ -0,0 +1,19 @@
+=== THE PROOF ON A BOX WITH ZERO REGISTERED DRIVES ===
+{"error":"Nem bekötött","ok":false}
+
+[http=501]
+
+=== what it recorded (last night this was ABSENT) ===
+  last_proof_result    '<ABSENT>'
+  last_proof_stack     '<ABSENT>'
+  last_proof_snapshot  '<ABSENT>'
+  last_proof_reason    '<ABSENT>'
+  last_proof_run       '<ABSENT>'
+  proved_snapshots     '<ABSENT>'
+
+=== the log line (last night: a WARN and nothing else) ===
+Daily job offsite-proof scheduled for 2026-09-02 05:30 CEST
+
+=== where did the scratch resolve to, and is it gone? ===
+  offsite-proof root: ls: cannot access '/mnt/sys_drive/felhom-data/backups/offsite-proof': No such file or directory
+  (empty or absent = the copy was deleted, as it must be)
@@ -0,0 +1,30 @@
+=== current logging level (to be restored) ===
+  logging:
+      file: ""
+      level: info
+  backup taken
+  logging:
+      file: ""
+      level: debug
+=== THE PROOF ON A BOX WITH ZERO REGISTERED DRIVES ===
+{"data":{"duration_ms":2452,"missing":null,"no_snapshot":false,"reason":"","skip_reason":"","skipped":false,"snapshot":"851e6dce","stack":"opengist","verdict":"pass"},"message":"A mentés tartalmazza az alkalmazás adatait","ok":true}
+
+[http=200]
+
+=== what it recorded (last night this was ABSENT) ===
+  last_proof_result    'pass'
+  last_proof_stack     'opengist'
+  last_proof_snapshot  '851e6dce'
+  last_proof_reason    '<ABSENT>'
+  last_proof_run       '2026-09-01T08:28:00Z'
+  proved_snapshots     {'opengist': '851e6dce'}
+
+=== the log line (last night: a WARN and nothing else) ===
+auth: valid session for POST /api/debug/backup/offsite-proof
+proof: refusing to remove /mnt/sys_drive/felhom-data/backups/offsite-proof/opengist — it is not inside a proof root
+proof: opengist PASSED on snapshot 851e6dce in 2.452s — the backup holds what this app should have
+proof: refusing to remove /mnt/sys_drive/felhom-data/backups/offsite-proof/opengist — it is not inside a proof root
+
+=== where did the scratch resolve to, and is it gone? ===
+  offsite-proof root: opengist
+  (empty or absent = the copy was deleted, as it must be)
@@ -0,0 +1,19 @@
+=== a fresh off-site backup makes the app due again (the real path, no hand-set state) ===
+  offbox/run http=302
+
+=== THE PROOF, on a box with zero registered drives ===
+{"data":{"duration_ms":2116,"missing":null,"no_snapshot":false,"reason":"","skip_reason":"","skipped":false,"snapshot":"61e9cf30","stack":"opengist","verdict":"pass"},"message":"A mentés tartalmazza az alkalmazás adatait","ok":true}
+
+  [http=200]
+
+=== the verdict recorded ===
+  last_proof_result    'pass'
+  last_proof_stack     'opengist'
+  last_proof_snapshot  '61e9cf30'
+  last_proof_run       '2026-09-01T08:32:20Z'
+
+=== AND THE SCRATCH MUST BE GONE (the leak live validation caught) ===
+  (no LEFTOVER lines above = the copy was deleted)
+daily job offsite-proof: next run at 2026-09-02 05:30:00 CEST (waiting 18h59m40s)
+proof: nothing due — every deployed app's newest off-site snapshot has already been proved, or none has one yet
+proof: opengist PASSED on snapshot 61e9cf30 in 2.117s — the backup holds what this app should have
@@ -0,0 +1,6 @@
+  config restored from backup
+  logging:
+      file: ""
+      level: info
+  observer guest cleaned: [0]
+  observer host cleaned: [0]
@@ -0,0 +1,13 @@
+=== VALIDATION 3: an ordinary customer restore on demo-hp, unchanged ===
+--- unit restore ---
+[http=302 wall=0.010338s]
+off-box restore docmost completed (full=false, async)
+--- the scratch it produced ---
+  124208399	/mnt/felhom-drives/hdd_1/backups/offsite-restore/docmost
+  marker: {"schema":1,"snapshot_id":"b2e059ee","full":false,"finished_at":"2026-09-01T08:33:33Z"}
+--- and the full two-step flow: prepare (which now takes the flag) then confirm ---
+[http=302 wall=5.495803s]
+off-box full-restore prepared for privatebin (size 2.0 MB) — awaiting the customer's confirm; no restore has started
+
+--- R-412a live: does a hollow push now say so? (no hollow unit here, so this is the CONTROL) ---
+  hollow-push warnings (0 expected, all units sound): 0