CHAOS NIGHT round 7: the block works, and "vzdump procs: 2" was my own command
gates / gates (push) Successful in 20s
gates / gates (push) Successful in 20s
Mid-cut measurements, asked of the box while its network was blocked: from the box: internet blocked (ping 1.1.1.1 fails), LAN reachable 26 containers still running - the apps do not care the internet is gone free / unchanged at 7573M; diskguard active with zero log lines from DooPlex: LAN 301, public 502 The two paths separate cleanly, and the public failure code differs from round 4's on purpose: 530 when cloudflared was dead (Cloudflare had no tunnel at all), 502 now (the tunnel lives but can reach nothing). Two different failures of the same journey, reported differently without being asked to. And the twelfth self-inflicted reading of the night, corrected: every "vzdump procs: 2" was MY OWN COMMAND. The [v]zdump bracket trick stops the pattern matching itself, but the label I echoed - "vzdump procs:" - contains the word, so ps listed my own shell and the grep counted it. No second backup was ever running; the box has been idle since the off-site leg finished at 22:08:27Z. Same family as the pkill -f that killed my own watcher earlier: a pattern that matches the hand holding it. The cure that worked: ask for command lines, not a count. The disk guard stays regardless - the local tier really did announce a retry with backoff, that retry is still due, and the guard has cost nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
@@ -221,3 +221,23 @@ rounds; it is **not** part of any round's result. If it ever fires, that is an i
|
||||
be counted as one, with its log line quoted. If it never fires, it changed nothing. Either way the
|
||||
product's own behaviour — fail the tier, name it, retry with backoff — is what is being measured, and
|
||||
the guard does not touch the off-site tier at all.
|
||||
,"success":true,"started_at":"2026-09-1
|
||||
{"data":{"backup":{"target_id":"felhom-pbs","vmid":9201,"archive":"felhom-pbs:backup/ct/9201/2026-09-16T21:59:54Z","mode":"snapshot","crash_consistent":true,"size_bytes":20811501125,"success":true,"started_at":"2026-09-1
|
||||
{"data":{"backup":{"target_id":"felhom-pbs","vmid":9201,"archive":"felhom-pbs:backup/ct/9201/2026-09-16T21:59:54Z","mode":"snapshot","crash_consistent":true,"size_bytes":20811501125,"success":true,"started_at":"2026-09-1
|
||||
2026-09-16T22:11:38Z --- ACCIDENT: none — control round, deliberately ---
|
||||
2026-09-16T22:11:38Z --- AFTER: what the box did BY ITSELF ---
|
||||
2026-09-16T22:11:39Z t+1s containers=26 (before 26)
|
||||
2026-09-16T22:11:39Z STEADY after 1s
|
||||
2026-09-16T22:11:40Z front doors: travel=502 status=502 paste=502 wiki=502
|
||||
2026-09-16T22:11:40Z household lines this round: 16 failures: 0
|
||||
2026-09-16T22:11:40Z --- alarms ---
|
||||
| Time | Severity | Type | Message | Source
|
||||
| Sep 16 21:59 | error | whole_guest_backup_failed | Whole-guest backup FAILED on the local tier — retrying with backoff (next attempt in 15m0s) | controller
|
||||
| Sep 16 21:53 | info | controller_started | Controller elindult (0.245.0) | controller
|
||||
| Sep 16 21:48 | info | health_recovered | Rendszer állapot helyreállt: ok (volt: fail) | controller
|
||||
| Sep 16 21:43 | error | health_critical | Rendszer állapot kritikus (volt: ok) | controller
|
||||
| Sep 16 21:28 | info | controller_started | Controller elindult (0.245.0) | controller
|
||||
| Sep 16 21:23 | info | app_deployed | Alkalmazás telepítve: BookStack | controller
|
||||
| Sep 16 21:22 | info | app_deploy_started | Alkalmazás telepítése elindult: BookStack | controller
|
||||
| Sep 16 21:22 | info | app_removed | Alkalmazás eltávolítva: bookstack | controller
|
||||
2026-09-16T22:11:41Z ================ END ROUND 6 ================
|
||||
|
||||
@@ -6,3 +6,43 @@
|
||||
2026-09-16T22:10:47Z cloud read 2 -> 200
|
||||
2026-09-16T22:10:47Z cloud read 3 -> 200
|
||||
2026-09-16T22:10:47Z --- ACCIDENT: internet-gone-10min (injected after the action started) ---
|
||||
|
||||
## MID-CUT measurements (22:11:15Z) — the block does exactly what it claims
|
||||
Asked of the box while its network was cut:
|
||||
from the box: internet **blocked** (ping 1.1.1.1 fails) · LAN **reachable** (ping 192.168.0.180 ok)
|
||||
containers **26** — every app still running; they do not care that the internet is gone
|
||||
free `/` **7573M**, unchanged
|
||||
diskguard active, **0 log lines** — it has not needed to fire
|
||||
from DooPlex: **LAN 301** (traefik answers on the box) · **public 502**
|
||||
|
||||
**The two paths separate cleanly, and this is the first round where that matters.** The apps are all
|
||||
up and serving locally; what is unreachable is the way in from outside. Note the public code differs
|
||||
from round 4's: **530** when cloudflared was dead (Cloudflare had no tunnel at all), **502** now
|
||||
(the tunnel process is alive but cannot reach anything). Two different failures of the same journey,
|
||||
and the box reports them differently without being asked to.
|
||||
|
||||
**The injector behaved as designed** — it blocks off-LAN traffic on this VM's tap only and leaves the
|
||||
LAN alone, which is what made these measurements possible at all: I can still reach the box to ask
|
||||
it questions while it cannot reach the world.
|
||||
|
||||
## „vzdump procs: 2" was MY OWN COMMAND — the twelfth self-inflicted reading tonight
|
||||
Asking for the actual command lines instead of a count showed exactly one match, and it was mine:
|
||||
|
||||
120847 00:00 bash -c echo START echo "--- actual vzdump command lines ---" ps -eo pid,etime,args
|
||||
| grep "[v]zdump" … echo "vzdump procs: $(…)" …
|
||||
|
||||
The `[v]zdump` bracket trick stops the PATTERN matching itself, but my own **label** — the literal
|
||||
text `"vzdump procs: "` that I echoed — contains the word, so `ps` listed my shell and the grep
|
||||
counted it. Every „vzdump procs: 2" reading tonight was **noise from my own command line**, including
|
||||
the one I cited when deciding the local tier might retry mid-round.
|
||||
|
||||
**What it changes:** no second backup was ever running. The box was idle after the off-site leg
|
||||
finished at 22:08:27Z, and it is idle now, mid-cut: no `.tar.dat` being written, free `/` unchanged at
|
||||
7573M, `diskguard` active with **0 log lines**.
|
||||
|
||||
**What it does not change:** the disk guard stays. The local tier really did announce
|
||||
„retrying with backoff (next attempt in 15m0s)", and that retry is still due; the guard is cheap
|
||||
insurance either way and has cost nothing.
|
||||
|
||||
Same family as the `pkill -f` that killed my own watcher earlier: **a pattern that matches the hand
|
||||
holding it.** The cure is the one that worked here — ask for the command lines, not the count.
|
||||
|
||||
Reference in New Issue
Block a user