Files
felhom.eu/documentation/audits/DRILL-soak-2026-08-31/phase5-mutated-cycle/07-r403-verdict.md
T
admin ab8b884763
gates / gates (push) Failing after 18s
soak phase 5: R-403 guard PROVEN live; R-412 CORRECTED down after measuring the mechanism
R-403 MIRROR GUARD - PASS, forced after the natural test evaporated. privatebin was
injected hollow at 23:34 to meet the 03:30 mirror; the 02:30 db-dump re-made its tar, so
by 03:30 the primary was complete and the guard had nothing to refuse. Forced instead on
calibre-web through the real Tier-2 path: the guard fired and named itself -
"unit leg SKIPPED ... The copy was PRESERVED rather than replaced with an empty one
(R-403). The other legs continue." Secondary byte-identical, 23 files, tar sha d7e7f422.

R-412 CORRECTED, AND I OVERSTATED IT WHEN I FILED IT. The first wording claimed the hollow
unit sits in the store for a whole cycle because the volume-dump leg runs only on the
backup schedule. That is WRONG. The 04:15 off-site run has its OWN pre-push dump leg -
"Stopping calibre-web for safe volume dump", "Volume dump: ... -> 877.5 KB" - so a unit
that is hollow when a run starts is REPAIRED before it is pushed. Measured twice: opengist
and calibre-web both went in hollow and came out complete, and the snapshot pulled back
from the store (6fee3b5a) holds the volume tar and all 17 userdata files.

What remains real is narrower: the one hollow snapshot that DID reach the store was created
when the unit was destroyed INSIDE a run that had already completed that app's dump leg.
The race is real and was observed, and "backed up opengist (... 0 mandatory path(s))" is a
success line over a backup holding none of the app's data either way. Severity HIGH -> LOW,
with the correction stated in the row rather than quietly rewritten.

Phase 6 interim: the observer is clean so far - db-dump 674ms, tier2-backup 3ms (a no-op,
cause to be established not assumed), zero ERROR/WARN since 23:00.

Also recorded: two of Phase 5's four injections were NOT performed, with the reasons
established rather than asserted - there is no endpoint that reaches SetDisconnected and a
hand-set flag would be reverted by the live monitor before 04:15; and a corrupted manifest
provably never reaches the store because the capture rewrites it first.
2026-09-01 04:21:15 +02:00

2.3 KiB

Phase 5 row 1 — the R-403 mirror guard. VERDICT: PASS (forced, after the natural test was lost)

The natural test was lost, and the reason is itself the finding

privatebin was injected hollow at 23:34 to meet the 03:30 tier2-backup. It healed at 02:30, an hour before the mirror ran, because the db-dump job re-creates the volume tars. By 03:30 the primary was complete (volume_dumps: ['privatebin_privatebin_data.tar'], 2 122 928 B) and the mirror correctly copied a sound unit. The guard was never exercised.

That fixes the timing in R-412: the hollow window closes at the next 02:30 db-dump, not at the next off-site backup — so a unit that loses its tar just after 02:30 stays hollow for nearly 24 h, with the 04:15 off-site push inside that window.

So it was forced, on an HDD app, through the real Tier-2 path

calibre-web (an HDD app, so crossDriveTargets includes it) injected hollow — primary 905 207 B → 6 527 B — then the real Tier-2 mirror fired.

BEFORE  secondary = 23 files, 5 808 704 B, tar sha d7e7f422a34c02455525
AFTER   secondary = 23 files, 5 808 704 B, tar sha d7e7f422a34c02455525

The guard fired and named itself:

Tier 2 calibre-web: unit leg SKIPPED — the recovery unit on the source drive lists no database
dumps and no volume tars, while the existing copy at /mnt/sys_drive/felhom-data/backups/secondary/
calibre-web/recovery-unit does. The copy was PRESERVED rather than replaced with an empty one
(R-403). The other legs continue.
[unit leg SKIPPED — existing package preserved, R-403]

The other legs continued and the run completed. The good copy survived byte-identical.

One thing the runbook asked that is only half true

The runbook's acceptance was "the run does not report a plain success". The log does not — it states the skip and the reason. The customer EVENT does: crossdrive_completed (info) — Másodlagos mentés elkészült: calibre-web, with no mention of the skipped leg. It is info severity, which severityNotifies drops, so nobody is mailed a false success — but an operator surface listing events would show a plain completion over a run that deliberately skipped a leg. Recorded as an observation, not filed: the event is not delivered, and widening it is the kind of coarse-vs-per-app decision 08 §6.2 fences.