Files
felhom.eu/documentation/audits/rehearsal-2026-10-20/A6-plan/order.md
T

5.8 KiB
Raw Blame History

B6 — failure scenario order, drawn ONCE from seed 20261020 before round 1 (A6)

Method: random.Random(20261020); shuffle the ★ list [1..10], then shuffle the rest [11..16]; this file is written before B1 bound the box (2026-10-20 ~10:00 UTC).

run scenario what
1 ★2 Power cut mid off-site restore (B8 run)
2 ★3 Data disk 100 % during nightly backup (via own upload); system disk 97 %
3 ★5 Slow lossy internet (1 Mbit/s up, 5 % loss) during first big off-site run
4 ★9 Two actions at once: Restore+Remove in two tabs; double click Visszaallitas
5 ★7 DST: clock 2026-10-25 01:30 CEST -> 04:00 CET, NTP off
6 ★1 Power cut mid app install (image pull)
7 ★10 Box off for 3 days (VM stopped, clock moved forward on start)
8 ★4 Data drive pulled during upload, put back (same slot; other slot/order)
9 ★6 Off-site path blocked for one night (ep0 + Storage Box dropped at the VM's outer side)
10 ★8 Clock wrong at boot: -2 years, then +2 years, NTP blocked
11 14 App removed keeping data, installed again (use kept data)
12 16 English browser: one full pass of the dashboard
13 11 Hard reset mid app update
14 12 DNS answers nothing for an hour
15 15 Box restarted during the catch-up of scenario 10
16 13 Second drive added later: offered as backup target; accept; remove safely

Dependencies fixed NOW (before any result), so the order is never re-chosen after one

  • 2 needs a running off-site restore — it runs INSIDE B8's A.1.5 run whatever its slot; its slot is then used for the next scenario.
  • 15 runs inside 10's catch-up (it is the restart during that catch-up) — it rides with 10.
  • 5 needs the FIRST big off-site run — it runs on night one (B5) if off-site is on by then; otherwise at its slot with a fresh large upload, and it is marked „not the first run".
  • 6 spans one night — it takes the second night if the run lasts; otherwise a shortened window (≥ 1 off-site schedule), marked.
  • 7 (DST) and 8 (wrong clock) change the VM's clock; they run after the off-site and restore work they could disturb, but in this drawn order relative to each other.
  • Anything that needs off-site (2, 5, 6) is SKIPPED, not reordered, if Tester4's off-site is still off when its slot comes.

Amendment written 2026-10-20 ~15:45 UTC — BEFORE scenario 5 runs (no result seen)

  • 5 (slow, lossy internet during the first big off-site run): the first big run is started by the household's „Távoli mentés most" this evening, with tc netem 1 Mbit/s up + 5 % loss on the VM's tap for its FIRST 30 MINUTES; then the shaping is removed and the run must finish (or resume) by itself. Reason: at 1 Mbit/s the ~3.7 GB first copy needs ~8 h, and B7 (box died) needs a complete off-site copy by the morning. What is measured: progress, what the page and the hub say while slow, and whether the run finishes or resumes after.
  • The box froze twice under ordinary uploads (R-1192). Every heavy scenario now runs ALONE, with the health watch beside it; a freeze during a scenario is recorded as such, not as the scenario's own result.

Amendment 2026-10-20 ~17:50 UTC — before any of these ran

Run tonight, before the night chain (B5 must stay „no hands"): #16 English pass, #9 two-at-once, #13 second drive later, #4 drive pulled, #3 disk full (the „nightly backup" stood in by „Mentés most" — the one real night is reserved for B5; the filler goes to a share with Felhőmentés OFF and is deleted before the night), #12 DNS dead, #6 off-site blocked (shortened to one manual run + the catch-up, not a night — same reason). After the night: #7 DST, #8 wrong clock, #10 three days off (+#15 inside it), #1 power cut mid install, #14 kept data, #11 if an update exists (none expected — the catalog is frozen). Reason: the clock scenarios move the box's time and would spoil the first night's record.

Amendment 2026-10-20 20:27 UTC — before any post-night scenario ran

  • #1 and #14 ran before the night (20:00–20:14), not after it: they do not move the clock, so they could not spoil the night's record, and the morning is short. Their results are in B6-failures/s1-… and s14-…. The drawn order of the remaining scenarios is kept: #7, #8, #10 (+#15), #11.
  • The morning starts when the night chain has visibly ended (the whole-box backup on local and PBS dated tonight, B5 expectation 4), not at the end of the 04:30–08:30 window.
  • #7 DST, shortened window: the VM starts with its clock at 2026-10-25 02:15 CEST (00:15Z), NTP blocked (UDP 123, TCP 4460) at the VM's tap, and runs to 02:45 CET (01:45Z) — 90 real minutes. That holds the whole repeated hour and the 02:30 database copy at BOTH occurrences. The 03:30 and 04:15 jobs are outside the window (they have a single occurrence; the replay covers them). The VM starts 4 days „later" than its last backup — a catch-up that fires at once is recorded as a side effect of the method, not as a DST result.
  • #8 wrong clock at boot: −2 years for 15 min, then +2 years for 15 min, then the real time; NTP blocked throughout, then unblocked. What is watched: the box's own pages, what it sends to the hub, and — after the return to real time — whether the whole-box backup becomes due again (D3/T11 is its live test).
  • #10 three days off (+#15): stop the VM; start it with the clock 3 days ahead (NTP blocked); during the catch-up restart it once (#15); then real time.
  • Off-site stays open during #7/#8/#10 — a real box has it. The product's own guard refuses future-dated off-site snapshots in its retention; what the box writes off-site with a wrong clock is part of what is measured.
  • #11 runs only if an update is offered; none is expected (nothing is released during the run).