diff --git a/documentation/audits/evidence-chaos-night-2026-09-17/round-6.txt b/documentation/audits/evidence-chaos-night-2026-09-17/round-6.txt index 8ddeed15..9c166ee6 100644 --- a/documentation/audits/evidence-chaos-night-2026-09-17/round-6.txt +++ b/documentation/audits/evidence-chaos-night-2026-09-17/round-6.txt @@ -221,3 +221,23 @@ rounds; it is **not** part of any round's result. If it ever fires, that is an i be counted as one, with its log line quoted. If it never fires, it changed nothing. Either way the product's own behaviour — fail the tier, name it, retry with backoff — is what is being measured, and the guard does not touch the off-site tier at all. +,"success":true,"started_at":"2026-09-1 + {"data":{"backup":{"target_id":"felhom-pbs","vmid":9201,"archive":"felhom-pbs:backup/ct/9201/2026-09-16T21:59:54Z","mode":"snapshot","crash_consistent":true,"size_bytes":20811501125,"success":true,"started_at":"2026-09-1 + {"data":{"backup":{"target_id":"felhom-pbs","vmid":9201,"archive":"felhom-pbs:backup/ct/9201/2026-09-16T21:59:54Z","mode":"snapshot","crash_consistent":true,"size_bytes":20811501125,"success":true,"started_at":"2026-09-1 +2026-09-16T22:11:38Z --- ACCIDENT: none — control round, deliberately --- +2026-09-16T22:11:38Z --- AFTER: what the box did BY ITSELF --- +2026-09-16T22:11:39Z t+1s containers=26 (before 26) +2026-09-16T22:11:39Z STEADY after 1s +2026-09-16T22:11:40Z front doors: travel=502 status=502 paste=502 wiki=502 +2026-09-16T22:11:40Z household lines this round: 16 failures: 0 +2026-09-16T22:11:40Z --- alarms --- + | Time | Severity | Type | Message | Source + | Sep 16 21:59 | error | whole_guest_backup_failed | Whole-guest backup FAILED on the local tier — retrying with backoff (next attempt in 15m0s) | controller + | Sep 16 21:53 | info | controller_started | Controller elindult (0.245.0) | controller + | Sep 16 21:48 | info | health_recovered | Rendszer állapot helyreállt: ok (volt: fail) | controller + | Sep 16 21:43 | error | health_critical | Rendszer állapot kritikus (volt: ok) | controller + | Sep 16 21:28 | info | controller_started | Controller elindult (0.245.0) | controller + | Sep 16 21:23 | info | app_deployed | Alkalmazás telepítve: BookStack | controller + | Sep 16 21:22 | info | app_deploy_started | Alkalmazás telepítése elindult: BookStack | controller + | Sep 16 21:22 | info | app_removed | Alkalmazás eltávolítva: bookstack | controller +2026-09-16T22:11:41Z ================ END ROUND 6 ================ diff --git a/documentation/audits/evidence-chaos-night-2026-09-17/round-7.txt b/documentation/audits/evidence-chaos-night-2026-09-17/round-7.txt index c4f40743..26edd8db 100644 --- a/documentation/audits/evidence-chaos-night-2026-09-17/round-7.txt +++ b/documentation/audits/evidence-chaos-night-2026-09-17/round-7.txt @@ -6,3 +6,43 @@ 2026-09-16T22:10:47Z cloud read 2 -> 200 2026-09-16T22:10:47Z cloud read 3 -> 200 2026-09-16T22:10:47Z --- ACCIDENT: internet-gone-10min (injected after the action started) --- + +## MID-CUT measurements (22:11:15Z) — the block does exactly what it claims +Asked of the box while its network was cut: + from the box: internet **blocked** (ping 1.1.1.1 fails) · LAN **reachable** (ping 192.168.0.180 ok) + containers **26** — every app still running; they do not care that the internet is gone + free `/` **7573M**, unchanged + diskguard active, **0 log lines** — it has not needed to fire + from DooPlex: **LAN 301** (traefik answers on the box) · **public 502** + +**The two paths separate cleanly, and this is the first round where that matters.** The apps are all +up and serving locally; what is unreachable is the way in from outside. Note the public code differs +from round 4's: **530** when cloudflared was dead (Cloudflare had no tunnel at all), **502** now +(the tunnel process is alive but cannot reach anything). Two different failures of the same journey, +and the box reports them differently without being asked to. + +**The injector behaved as designed** — it blocks off-LAN traffic on this VM's tap only and leaves the +LAN alone, which is what made these measurements possible at all: I can still reach the box to ask +it questions while it cannot reach the world. + +## „vzdump procs: 2" was MY OWN COMMAND — the twelfth self-inflicted reading tonight +Asking for the actual command lines instead of a count showed exactly one match, and it was mine: + + 120847 00:00 bash -c echo START echo "--- actual vzdump command lines ---" ps -eo pid,etime,args + | grep "[v]zdump" … echo "vzdump procs: $(…)" … + +The `[v]zdump` bracket trick stops the PATTERN matching itself, but my own **label** — the literal +text `"vzdump procs: "` that I echoed — contains the word, so `ps` listed my shell and the grep +counted it. Every „vzdump procs: 2" reading tonight was **noise from my own command line**, including +the one I cited when deciding the local tier might retry mid-round. + +**What it changes:** no second backup was ever running. The box was idle after the off-site leg +finished at 22:08:27Z, and it is idle now, mid-cut: no `.tar.dat` being written, free `/` unchanged at +7573M, `diskguard` active with **0 log lines**. + +**What it does not change:** the disk guard stays. The local tier really did announce +„retrying with backoff (next attempt in 15m0s)", and that retry is still due; the guard is cheap +insurance either way and has cost nothing. + +Same family as the `pkill -f` that killed my own watcher earlier: **a pattern that matches the hand +holding it.** The cure is the one that worked here — ask for the command lines, not the count.