admin
cee8f70e98
gates / gates (push) Failing after 17s
Overnight soak 22:39->06:10 CEST. demo-hp the victim, demo-felhom the untouched observer.
No production code, no golden, no version bump. Report at
documentation/audits/DRILL-soak-2026-08-31/REPORT.md.
VERDICTS: 1 lock-collision FAIL, 2 guard-interactions PASS-with-one-defect, 3 R-357 PASS,
4 proof-edges PASS, 5 mutated-cycle PASS, 6 observer FAIL, 7 teardown PASS.
R-414 - THE MOST VALUABLE FINDING, AND ONLY AN UNTOUCHED BOX COULD HAVE FOUND IT. On
demo-felhom the nightly proof fired for the first time unattended at 05:30 and REFUSED:
"nowhere to restore to - nincs regisztralt adatmeghajto". Cause established, not inferred:
storage_paths is EMPTY, so there is no path to put a scratch on. It will fail this way every
night forever with only a WARN, and because the error path reaches no verdict,
last_proof_result stays ABSENT - which is also what a pre-0.231.0 controller sends. The hub
cannot tell "never ran" from "not deployed": the StatsKnown trap one level up. The box is
NOT unprotected; its off-site backup ran fine in 46.9s. It is the PROOF that cannot run.
R-411 - measured, not reasoned: restic stats TAKES A LOCK; a customer full-restore runs it
while holding no acquireRunning; the integrity check is therefore not blocked, meets that
lock and escalates to unlock --remove-all. The sampler caught "restore ..." and
"unlock --remove-all" in the SAME sample. Contained: the check was classified unreachable,
not damage, so no false alarm.
R-412 - CORRECTED from HIGH to LOW. I filed it on a mechanism I had not finished measuring.
The off-site run has its own pre-push dump leg, so a hollow unit is REPAIRED before it
ships - proven on two apps and confirmed by pulling the snapshot back out of the store.
What survives is a narrow race, plus a success line over a backup holding no data.
R-413 - the R-87 proof caught a product-produced hollow snapshot unattended, and the
nightly job fired on its own schedule at 05:30 for the first time (bentopdf PASSED on
9d002b38 in 2.315s). Both were listed "not yet live-validated" yesterday.
R-403 mirror guard PROVEN live, with a negative control: it fired when a unit was hollow
("The copy was PRESERVED rather than replaced with an empty one") and skipped 0 legs at
teardown when every unit was sound.
R-357 PASS at last, six days owed: a real full filesystem, refused BEFORE StopStack, app
never stopped, live data byte-identical, and it worked once the space came back.
Phase 4 built the false-alarm control the whole R-87 design rests on: bentopdf is the only
template of 53 with neither a database nor a named volume. It passes silently.
EIGHT of my own instrument errors are named in the report, each caught by its own control -
including a time guard that fired an injection four hours early, and filing R-412 at the
wrong severity.
Teardown clean on all three layers of both boxes; both healthy on 0.231.0.
OWED: a golden for 0.231.0, and a decision on keeping bentopdf.
53 lines
2.2 KiB
Plaintext
53 lines
2.2 KiB
Plaintext
23:34:13 phase5 driver started
|
|
23:34:13 cycle recorder started
|
|
23:34:13 light load started (a restore every 20 min)
|
|
23:34:13 injection armed for 03:15
|
|
23:34:13 === INJECTING the hollow primary (privatebin) ===
|
|
perl: warning: Setting locale failed.
|
|
perl: warning: Please check that your locale settings:
|
|
LANGUAGE = (unset),
|
|
LC_ALL = (unset),
|
|
LC_CTYPE = "UTF-8",
|
|
LC_NUMERIC = (unset),
|
|
LC_COLLATE = (unset),
|
|
LC_TIME = (unset),
|
|
LC_MESSAGES = (unset),
|
|
LC_MONETARY = (unset),
|
|
LC_ADDRESS = (unset),
|
|
LC_IDENTIFICATION = (unset),
|
|
LC_MEASUREMENT = (unset),
|
|
LC_PAPER = (unset),
|
|
LC_TELEPHONE = (unset),
|
|
LC_NAME = (unset),
|
|
LANG = "en_US.UTF-8"
|
|
are supported and installed on your system.
|
|
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
|
|
=== BEFORE ===
|
|
primary : 5 files, 2123006 bytes
|
|
secondary: 6 files, 2123007 bytes
|
|
secondary tar sha: c3ea1bae0731bcc3082d6c94
|
|
|
|
=== INJECT: remove privatebin's primary unit. The 5-min capture will rebuild it HOLLOW
|
|
(it cannot conjure a volume tar - that leg runs on the backup schedule). ===
|
|
primary removed
|
|
23:34:14 load: restore kimai -> [http=302 wall=0.012033s]
|
|
23:54:15 load: restore bookstack -> [http=302 wall=0.011206s]
|
|
00:14:16 load: restore docmost -> [http=302 wall=0.011085s]
|
|
00:34:17 load: restore kimai -> [http=302 wall=0.010686s]
|
|
00:54:18 load: restore bookstack -> [http=302 wall=0.011526s]
|
|
01:14:19 load: restore docmost -> [http=302 wall=0.011364s]
|
|
01:34:20 load: restore kimai -> [http=302 wall=0.010521s]
|
|
01:54:21 load: restore bookstack -> [http=302 wall=0.011243s]
|
|
02:14:22 load: restore docmost -> [http=302 wall=0.011166s]
|
|
02:34:23 load: restore kimai -> [http=302 wall=0.011049s]
|
|
02:54:24 load: restore bookstack -> [http=302 wall=0.011425s]
|
|
03:14:25 load: restore docmost -> [http=302 wall=0.011215s]
|
|
03:34:26 load: restore kimai -> [http=302 wall=0.011015s]
|
|
03:54:27 load: restore bookstack -> [http=302 wall=0.012046s]
|
|
04:14:28 load: restore docmost -> [http=302 wall=0.010933s]
|
|
04:34:28 load: restore kimai -> [http=302 wall=0.010720s]
|
|
04:54:29 load: restore bookstack -> [http=302 wall=0.010774s]
|
|
05:14:30 load: restore docmost -> [http=302 wall=0.011732s]
|
|
05:34:31 load: restore kimai -> [http=302 wall=0.010919s]
|
|
05:54:32 load: restore bookstack -> [http=302 wall=0.011551s]
|