felhom.eu/documentation/audits/evidence-bignight-2026-09-14/journal.md

Journal — BIGNIGHT 2026-09-14 (a household's first month in one night)

Times UTC unless marked. Hub logs print CEST (UTC+2). Brief: drills/BIGNIGHT-2026-09-14.md.

Phase 1 — baselines (read 17:29–17:40Z from live source)

repo	main == origin/main	version
felhom-controller	406755fa8fba	v0.242.0 (CHANGELOG top)
felhom-agent	4586f0f7f6d1	v0.130.0 (tree: one untracked scripts/__pycache__/, no tracked change)
felhom.eu	a4d684412b49	hub v0.113.0 (CHANGELOG top; live image felhom-hub:0.113.0)
app-catalog-felhom.eu	6d6eec307939	—

Register: highest id R-508; 221 table rows (grep ^| R-n).

ISO 1.27.1: /mnt/5_hdd/felhom.eu/felhom-iso/out/felhom-installer-1.27.1-pve9.2-1.iso, 1 705 322 496 B, sha256 25637007d5a7120ff9faa6b5b7ead3e33c0a361ac2d67e9fd4e0ee77c034c053 = .sha256 = manifest output-sha256; manifest repo-commit 27e8ec86…, mode release, answer-file NONE. Found, not rebuilt.

DooPlex headroom: /mnt/5_hdd 37 %, / 52 %.

demo-hp (17:29Z): qm list empty; 9201 + 9202 running (each memory: 25898, cores: 7); host RAM 29 994 MB total, 24 652 MB available; 8 threads (Ryzen V1756B); storages local 56.96 %, local-lvm 44.17 %, nvme-scratch (dir /mnt/hdd_1, is_mountpoint yes) 1.03 %.

Gmail connector: authenticates as felhom.eu@gmail.com — the catch-all for @felhom.eu (threads to admin@felhom.eu and drill0242@felhom.eu land there). Read only; nothing sent.

Phase 1 — the Tester 1 record (hub GET /customers/tester-1, Basic auth, 17:30Z)

Customer ID tester-1 · Name „Tester 1" · Domain enkicsifelhom.hu · Email tester1@felhom.eu · Config MANAGED. Status tile still reads down, „Last report 1h ago · Controller 0.242.0" — the stale report of the destroyed VM 331 (host record deleted 16:46:52Z). Claim: „Claimed 1h ago · generation 1". Edit tab: dr_tier ON (all four DR steps „waiting — no host enrolled yet"); offsite_enabled OFF („Enable offsite (provisions a Hetzner Storage Box on save)"). Controller floor v0.242.0. Mailbox: to:tester1@felhom.eu → 0 threads before tonight.

Brief vs record — off-site. The brief says "Off-site (Tier 3) is ON … its own namespace on ep0". On the record, the ep0 namespace is the DR tier (PBS whole-guest); the Tier-3 restic off-site is OFF, and turning it on provisions a Hetzner Storage Box (money, and a new external resource). Not followed: I do not tick it — money is fenced by the rules file §1. "Off-site" tonight = the DR tier to ep0 as configured. Consequence, stated up front: the Phase 4 off-site integrity check and the Phase 6 app restore "from off-site" have no restic tier to act on; each is recorded as such when reached.

Harness slip: the first read of the customer page printed the page's text into this session's tool output, which includes the customer's hub API key (not the retrieval passphrase — that is masked). It stayed on DooPlex; not written to any file.

Tunnel before any box (17:31:19Z, from DooPlex) — tunnel-before-no-box.txt

DNS resolves to Cloudflare (both resolvers); https://felhom.enkicsifelhom.hu → 530, error code: 1033 (no connector). Expected with no box.

Venue — VM 333 bignight-household on demo-hp (created 17:31:52Z)

q35 / OVMF (pre-enrolled-keys=0), 4 cores, cpu=host, 16 384 MB, virtio NIC BC:24:11:F2:E2:95 on vmbr0, scsi0 200 G qcow2 on nvme-scratch (dir at /mnt/hdd_1, its root), ISO 1.27.1 on ide2 (sha verified on the HP = 25637007…). One disk at install; the data disk is added after.

Why these numbers. System disk = doorstep VM 331's 200 G. Memory: the HP has 29 994 MB; 9201 and 9202 together used ≈ 5.3 GB at 17:29Z with 24 652 MB available. Both LXCs carry a 25 898 MB limit, so no VM size leaves both limits whole; 16 GB leaves ≈ 8 GB free plus 8 GB swap for 9201's real load. Cores: the 0242 drill's 4 of 8 threads.

Power-on 17:32:13Z. GRUB (s01, s02): the two Felhom entries; text mode chosen (H4).

Phase 2.1 — install from 1.27.1 (text mode), one disk

UTC	screen	what it said / what was done
17:34:5x	s03	English Proxmox EULA → „I agree"
17:35:11	s04	„Target harddisk: /dev/sda (QEMU HARDDISK) (200.00 GiB)" — one disk, no choice to make → Next
17:35:24	s05	Country Hungary · Timezone Europe/Budapest · Keyboard layout Hungarian (H2: changed to U.S. English, s06–s09; one dropped keystroke landed on „Turkish" first, corrected)
17:37:00	s11	„Root password [at least 8 characters]" · Confirm · „Administrator email" prefilled mail@example.invalid → 20-char generated password (0600 file on DooPlex), tester1@felhom.eu (the guide: „a saját e-mail címedet")
17:39:01	s13	nic0 bc:24:11:f2:e2:95, virtio_net · Hostname pve.example.invalid · IP 192.168.0.136/24 (the DHCP lease, frozen static) · GW 192.168.0.1 · DNS 192.168.0.250 · „[X] Pin network interface names" → hostname set felhom.enkicsifelhom.hu per the guide (s15)
17:40:36	s16	Summary: ext4 · /dev/sda · Europe/Budapest · U.S. English · tester1@felhom.eu · nic0 · felhom.enkicsifelhom.hu · 192.168.0.136/24 · 192.168.0.1 · 192.168.0.250 · „[X] Automatically reboot after successful installation" (H3: unticked, s17)
17:41:02	—	Install pressed
≤17:43:47	s18	„Success — Installation finished - reboot now?" (≤ 2 m 45 s copying; qcow2 4 058 MB, settled)
17:44:xx	first boot	H3: qm stop, --delete ide2, --boot order=scsi0, qm start

Seeds prepared on DooPlex (scratchpad, tmpfs): 200 JPEG 1600×1200 noise + caption (264 MB), 20 three-page PDFs (2.3 MB), a 50 MB random file, a 150 s 1280×720 H.264 video (98.9 MB).

Phase 2.2 — first screen, and waiting for the mail

17:44:53Z — hub lists Unclaimed appliance 28 (38 s after power-on): smbios uuid 1ecd1c40…, pairing code ***-*** (redacted), MAC bc:24:11:f2:e2:95, „Standard PC (Q35 + ICH9, 2009)", 15.6 GB, three SSH host keys. Bind list offers „Tester 1 (0 hosts)".

The console, 17:45:31Z (s19, code redacted), verbatim:

  Felhom otthoni szerver

  Ezen a gépen most nincs dolgod, és bejelentkezni sem kell.
  A beállításhoz kövesd a Felhomtól kapott útmutatót.

felhom login:
==============================================
  Felhom — a doboz készen áll, és a párosításra vár.

  Párosító kód:  ***-***

  Nyisd meg az e-mailben kapott linket, és add meg
  ezt a kódot és a Tulajdonosi jelmondatodat
  (az 5 szót a Felhom üzemeltetőjétől kaptad).

  Ez a képernyő magától frissül — nincs teendő a
  doboznál, és nyugodtan itt hagyhatod bekapcsolva.
==============================================

No Proxmox :8006 line on the first boot (the 1.27.1 fix holds on a fresh install). Text says „otthoni szerver"; the guide §3 quotes „Ezen a képernyőn nincs teendőd" — the guide's quote does not match the screen word for word (meaning is the same).

Mailbox, 17:46Z: to:tester1@felhom.eu → 0 threads. The console asks for „az e-mailben kapott linket". Hub source: the self-bind link is auto-sent only at customer creation (configs.go:725) and at RESET completion (customer_reset.go:162); tester-1 was created 32 days ago with no e-mail, so no link was ever sent to tester1@felhom.eu. Waiting the brief's 10 minutes (to 17:55Z) before acting.

17:55:11Z — ten minutes, 0 messages to tester1@felhom.eu. Filed R-509 (P1) at 17:56Z, before acting. Intervention I1: the operator's „Send self-bind link" button on the customer's Setup tab is pressed (a volunteer cannot press it).

17:55:42Z POST /customers/tester-1/selfbind-link → 303 flash=selfbind-sent; hub: self-bind link (hash effe179d…, valid 7 days) emailed to the registered address of tester-1.

The mail, as the volunteer reads it (Gmail connector, message 1a0a10f8cd07d8f6, Date 17:55:43Z — 1 s after the press; Gmail threads it under the older drill0242 mail of the same subject, a mailbox quirk):

[Felhom] Kösd össze a Felhom dobozodat — from monitoring@felhom.eu to tester1@felhom.eu

Kedves Ügyfél! Elkészült a Felhom dobozod, és készen áll az összekötésre. Az alábbi hivatkozáson tudod te magad összekötni a fiókoddal — nincs szükség bejelentkezésre: https://hub.felhom.eu/bind/<64 hex, redacted> A hivatkozás megnyitása után két adatot kell megadnod:

1. A párosító kódot, amely a doboz képernyőjén (a monitoron) látható.
2. A tulajdonosi jelmondatodat (az 5 szóból álló kifejezést), amelyet a Felhom üzemeltetőjétől kaptál — személyesen vagy telefonon, e-mailben soha. Ez igazolja, hogy a fiók a tiéd. A hivatkozás 7 napig érvényes. Biztonsági okból 5 sikertelen próbálkozás után zárolódik — ilyenkor vedd fel a kapcsolatot az ügyfélszolgálattal. Ha nem te kérted ezt, hagyd figyelmen kívül ezt az e-mailt. Üdvözlettel, Felhom.eu

The Tulajdonosi jelmondat is taken from the hub customer page into a 0600 file (the operator's hand-over, per the guide's prerequisite 3 — not a volunteer act, not counted).

Phase 2.2 (cont.) — following the link (the self-bind page, first time exercised by any walk)

17:56:11Z GET https://hub.felhom.eu/bind/<token> → 200 (screen-bind-page.txt), verbatim: „Felhom — Doboz összekötése" · „Kösd össze a most telepített Felhom dobozodat a fiókoddal. Add meg a doboz képernyőjén látható párosító kódot és a tulajdonosi jelmondatodat." · Párosító kód („A doboz monitorán jelenik meg, a telepítés után.", placeholder ABC-234) · Tulajdonosi jelmondat („Az öt szóból álló kifejezés, amelyet a beállításkor kaptál.", placeholder „öt szó, kötőjellel vagy szóközzel") · Összekötés · „Biztonsági okból 5 sikertelen próbálkozás után a hivatkozás zárolódik. …"

Observation: the page still says the phrase was received „a beállításkor" (at setup); the mail (hub v0.113.0) says „a Felhom üzemeltetőjétől kaptál — személyesen vagy telefonon". Two wordings for one hand-over.

17:56:29Z POST /bind/<token> (pairing code + Tulajdonosi jelmondat) → 200 (screen-bind-result.txt): „Sikeres összekötés. A doboz kb. egy percen belül folytatja a telepítést. Ezt az oldalt bezárhatod — a beállítás a háttérben befejeződik, és a vezérlőpultod hamarosan elérhető lesz." The self-bind link works end to end — first time any walk exercised it. One try, no error.

Hub (CEST): 19:56:30 self-bind SUCCESS: appliance 28 bound to customer tester-1 by customer self-service · 19:56:47 credentials DELIVERED once · 19:57:23 host enrolled: tester-1-a61396 · 19:57:24 [claim] reenroll code (gen 2) emailed … reset code re-issued (gen 2) for tester-1 on box re-enrollment (clean-slate reinstall) · 19:57:25 manifest agent 0.130.0 golden 0.242.0 · 19:57:42 wg registered … ip=10.77.0.5/32 · 19:57:42 [ERROR] pbsdr auto-provision for tester-1 (WG-registration hook): the endpoint already holds a PBS token for tester-1 but the hub has no descriptor — use the explicit "Re-issue PBS credentials" action · 19:59:06 controller_started (info) — Controller elindult (0.242.0) · 19:59:43 tester-1 down → ok. Console after bind (s21, codes redacted): the pairing banner is printed three times with the same code; it never changes to "bound" (R-214 class).

Which claim path — the brief asked "fresh claim or reset flow". Neither a fresh claim nor the RESET: the hub treated the box as a re-enrolment of a claimed customer and sent the reinstall mail (gen 2). Quoted (Gmail 1a0a111205fa7fe8, 17:57:24Z, 54 s after the bind):

[Felhom] Új beállító kód — újratelepült a szervered Kedves Ügyfél! A Felhom szervered újratelepült, ezért a vezérlőpultod belépését újra be kell állítani. A korábbi jelszavad már nem érvényes. Beállító kód: <3 words, redacted> A kód 72 óráig érvényes, és egyszer használható fel. Nyisd meg a vezérlőpultot — "A szerver beállítása" oldal fogad —, add meg a kódot, majd válassz új jelszót: https://felhom.enkicsifelhom.hu Ha nem te telepítetted újra a szervered, vedd fel a kapcsolatot az üzemeltetővel.

For a volunteer on their first box this mail is wrong in two places: the guide §5 tells them to look for „Elindult a Felhom szervered — beállító kód", and this one says their server was reinstalled and their previous password no longer works — a password they never had. A careful stranger reads „Ha nem te telepítetted újra … vedd fel a kapcsolatot" and stops. (Cause: the record was claimed once today by VM 331.)

Tunnel after bind, before claim — 18:07:48Z from DooPlex (tunnel-after-bind.txt): 502 ×3 (server: cloudflare, body error code: 502). Changed from 530/1033 (no connector) to 502 (connector present, origin failing).

Phase 2.4 — the gate: the dashboard through the tunnel — FAIL (P1)

box-logs-phase2/cloudflared-before-claim.txt, tunnel-origin-diagnosis.txt. Guest 9201 on the box: 192.168.0.150; containers felhom-controller:0.242.0 (healthy), traefik:v3.6.7, cloudflared:2026.6.0, filebrowser 1.3.3-stable. cloudflared: 4 connections registered (17:59:02Z, vie06 …), pre-checks all PASS. Then every request: Request failed … tls: failed to verify certificate: x509: certificate is valid for 544346c4….traefik.default, not traefik … ingressRule=0 originService=https://traefik. Traefik on 127.0.0.1:443 with SNI felhom.enkicsifelhom.hu serves CN=*.enkicsifelhom.hu, Let's Encrypt YR1, valid 2026-09-14 17:01 → 12-13 — the right certificate exists. Control (demo-hp 9201, read only): its route {"hostname":"*.enkisfelhom.hu","originRequest":{"noTLSVerify":true},"service":"https://traefik"}. Conclusion: the operator's route reaches the box now (530 → 502) but lacks „No TLS Verify". Filed R-510 (P1) at 18:1xZ before acting. Intervention I2: from here the claim and all dashboard traffic go to 192.168.0.150:443 with the name forced (--resolve), from DooPlex on the same household LAN as the box.

Phase 2.2 (cont.) — the claim, with the mailed code (I2 route)

screen-claim-page.txt. 18:10:06Z GET / → 302 /claim; the page: „A szerver beállítása — Tester 1 — Add meg az e-mailben kapott beállító kódot, majd válassz saját jelszót a vezérlőpult védelméhez." · Beállító kód · Új jelszó (min. 12 karakter) · Új jelszó megerősítése · Beállítás és belépés · „Nem kaptad meg a kódot? Új kód kérése". (The page does not mention the reinstall the mail talked about — the two agree on the code, not the story.)

Harness slip: the first POST /claim (18:10:06Z) sent both of the page's _csrf values joined (two forms, two tokens) → 200 „Érvénytelen űrlap — töltsd újra az oldalt." (form refused, not a code attempt). Re-fetched, one token.

18:10:28Z POST /claim (the mailed code, a generated 24-char password twice) → 302 / + felhom_session. The mailed setup code works — first walk to use the real mail instead of a box-printed code. /launcher 200: „Indítópult — Felhom.eu", tile Filebrowser, sharing off; menu Vezérlőpult · Alkalmazások · Tárhely (Meghajtók, Hálózati tárhely) · Biztonsági mentés (Áttekintés, Távoli mentés, Alkalmazások, Visszaállítás) · Megosztás (Hálózati megosztás) · Rendszermonitor · Debug · Beállítások (Rendszer, Értesítések, Biztonság és hozzáférés) · 0.242.0.

Power-on → claimed dashboard: 38 m 15 s (17:32:13 → 18:10:28), of which 10 min were the brief's mail wait.

Phase 2.3 — version

Box: felhom-controller:0.242.0 (healthy), felhom-agent 0.130.0. Hub: „Controller 0.242.0 · Registry latest v0.242.0 — up to date · Effective floor v0.242.0 — at/above floor". PASS — landed on golden 0.242.0, reports current, no self-update needed (golden == floor). Bind → controller_started 2 m 36 s.

Phase 2.4 — the gate after the claim — FAIL, still R-510

tunnel-after-claim.txt: 18:10:41Z from DooPlex https://felhom.enkicsifelhom.hu/login → 502 ×6; box cloudflared at 18:10:42Z: the same x509 … traefik.default, not traefik for each. Continued on the LAN address (I2).

Phase 2.1 (cont.) — the data disk (100 G scsi1, hot-attached 17:46:42Z, before the bind)

• Box lsblk: sdb 100G disk, no partitions.
• Dashboard (screen-storage-dashboard-after-claim.txt): „Lemezek állapota — QEMU QEMU HARDDISK · Nincs adat · 0 °C" (one line, which disk is not said); nothing mentions a new disk.
• Tárhely → Meghajtók: „Nincs regisztrált adattároló. Adjon hozzá egyet az alábbi űrlappal." · „Új meghajtó inicializálása" · „Meglévő meghajtó csatolása" · „Nem regisztrált meghajtók — az ügynök által észlelt, még nem regisztrált adatmeghajtók" · „Rendszermeghajtók — védett …" · „Már csatlakoztatott tárhely hozzáadása kézzel — Elérési út — Pl. /mnt/hdd_1 …". Formal „Adjon" (the rest of the product says „te").
• GET /api/disks/candidates: initialize: [/dev/sdb 107374182400 B, QEMU HARDDISK, data_bearing false]; attach: [/dev/mapper/pve-vm--9201--disk--1, mount_source /mnt/sys_drive, already_mounted true] (the guest's own system volume offered for attach — observed in the 0242 drill too).
• Would a household know to enrol it? No. Nothing on the dashboard, the launcher or any mail says a disk appeared; the volunteer guide has no step for it; the page that lists it is two menus deep and speaks of „inicializálás" and „ügynök". A volunteer who plugs in a second disk has to go looking.

The DR tier on this box — stuck (operator act O1, not a customer step)

Hub 19:57:42 CEST: pbsdr auto-provision … the endpoint already holds a PBS token for tester-1 but the hub has no descriptor — use the explicit "Re-issue PBS credentials" action. 18:11:57Z operator POST /configs/tester-1/ pbsdr-reissue → 400 No provisioned PBS DR tier for this customer. Neither path provisions it. Filed R-511 (P2). Not forced further: removing the old ep0 token is a destructive act on ep0 tenancy, and RESET would also remove the tunnel (fenced by the brief: the customer is kept). Consequence for tonight, stated once: this box has no off-site tier of either kind — restic Tier-3 is off (money) and the PBS DR tier cannot provision. Nothing is written to ep0 tonight. Phase 4's off-site integrity check and Phase 6's restore „from off-site" are recorded as not possible when reached, with the local tiers walked instead.

No „A vezérlőpultod mostantól jelszóval védett" mail arrived for the claim (the 0242 drill's box received one at 13:31:23Z, 12 s after its claim); at 18:12Z the mailbox holds only the bind mail and the reinstall mail.

Phase 2.1 (cont.) — enrolling the data drive, as a household could from the screens

18:12:57Z POST /api/storage/init (the „Új meghajtó inicializálása" wizard's call) {device /dev/sdb, ext4, mount_name hdd_1, label Adatlemez, set_default true} → formatting → done at 18:12:59Z (2 s), where /mnt/felhom-drives/hdd_1 (drive-init.txt). The wizard's „Csatlakoztatási név" field has no value, only the placeholder hdd_1 — a person must type a name. Tárhely now: „Adatlemez · /mnt/felhom-drives/hdd_1 · Alapértelmezett · Aktív · 0.0 GB / 97.9 GB · ext4 · /dev/sdb[/felhom-data] · QEMU HARDDISK · Nincs alkalmazás ezen a tárolón". /api/disks: role user-data, durable id uuid:c4b530fd….

Phase 2 evidence pulled off the box 18:14Z: box-logs-phase2/ (bootstrap + agent journals, controller log, box state, cloudflared). Secret patterns searched (passphrase=|retrieval_password|password:): 0 hits.

Phase 2 interventions: 2

• I1 (R-509): the self-bind mail never came for an existing customer; the operator's send button was pressed.
• I2 (R-510): the tunnel answers 502; claim and dashboard over the LAN address with the name forced. Operator acts that are prerequisites, not counted: passphrase hand-over; PBS re-issue attempt (O1, refused).

Phase 3 — a family moves in (phase3/)

Deploy pages captured before any deploy (phase3/deploy-pages-before.txt); the memory line each showed at 18:14Z with 3 infra containers running: bookstack 1525 / 11444 MB · docmost 1568 · privatebin 1404 · gokapi 1408 · nextcloud 1633 · immich 3422 · vaultwarden 1427 · paperless-ngx 1880 · jellyfin 1892 · mealie 1578 · uptime-kuma 1431 · adventurelog 1476 (guest limit 11 444 MB of the VM's 16 GB). Every deploy goes through the page's own call, POST /api/stacks/<app>/deploy {"values":…} with the form's server-generated values; a required empty password (gokapi, nextcloud admin) is typed as a customer would — a generated value in a 0600 file.

app	deploy	seed through the front door	household use
bookstack	18:14:35 → running 80 s	default login → admin e-mail + password changed (old default refused); book „Családi tudástár BIGNIGHT"; 5 Hungarian pages (Töltött káposzta, Wifi jelszó helye, Kutya oltási naptár, Nyaralás Balaton 2026, Autó szerviz); 2 attachments 256 KiB + 512 KiB, sha equal ×2	2nd user „Kovács Péter" (Editor) created, logs in, edits Balaton page (edit visible); Péter's attachment upload returned an HTML page, not JSON — inconclusive (harness: page id hard-coded); Anna deletes „Kutya oltási naptár" → 404 → recycle bin → restore → 200 with its text
docmost	18:16:10 → running ≈ 90 s	/api/auth/setup workspace „Kovács család"; space „Háztartás"; 5 docs via Markdown import (titles taken from each file's # heading: Rezsi, Iskola, Lista, Orvos, Kert); read back „ELMŰ", „árvíztűrő" present	renamed „Bevásárlólista — szombat"; deleted „Kert" → trash → restored → listed; invited peter@ (200)
privatebin	18:18:2x → running 15 s	5 encrypted pastes (browser v2 format): 1week, never, 1month, never, 10min (the expiring one); every one decrypts equal; TTL meta 604800 / [] / 2592000 / [] / 600. Harness: PrivateBin's 10-second post limit refused one post (status 1), retried	— (file-only; pastes are the use)
gokapi	18:18:43 → running 20 s	deploy page asks „Admin jelszó *" (customer types it); login admin; 3 uploads via the dropzone's chunk calls: 50 MB (2 chunks, 1.6 s), a PDF, a JPEG; admin lists 3	a stranger downloads all three, sha equal ×3; the PDF deleted through the UI's API → its link no longer serves it (78-byte page)

App pages: „Fut · Naprakész" (title „Ez az alkalmazás a legfrissebb elérhető változatot futtatja."). Gokapi's password: app page „Első lépések" says „a jelszó a Beállítások oldalon található"; Beállítások shows „Admin jelszó 👁 — Telepítéskor beállított kezdeti jelszó — ha az alkalmazásban megváltoztattad, az itt nem frissül." Findable. BookStack page „Első lépések": „Nyisd meg a wiki.DOMAIN címet" (R-498, still).

nextcloud 18:20:14 deploy (HDD_PATH offered: „Adatlemez — 92.9 GB szabad (alapértelmezett)") → +50 s starting → +121 s unhealthy (the harness stopped polling there — its own mistake) → at 18:23:43Z running, all three containers (healthy). So the card reads „unhealthy" for a while on a first start; recorded as observed, not a defect.

nextcloud seed (WebDAV, 4 parallel, as the admin the deploy page asked for): folders Fotók/Balaton 2026, Számlák, Közös mappa; 200 JPEG + 20 PDF, 220 × 201, 86 s (264 MB of photos); 2nd user „peter" (OCS 100), „Közös mappa" shared with him (OCS 200); Péter uploads into it and Anna reads it back (sha equal); foto-123 read back sha equal. Household use: a child deletes foto-007 (204 → 404); trashbin lists it; restore (201); photo back, sha equal. Admin used 343 161 297 B. Nextcloud 34.0.1.

immich 18:22:57 deploy → +116 s degraded → +126 s starting … (continues below).

immich running after 146 s (degraded 116 s on the way). Seed through its API: admin sign-up (201), 200 JPEG uploads, 200 × 201 in 16 s; stats photos 200, 275 682 444 B; ML queues draining (faceDetection / smartSearch / ocr).

vaultwarden 18:25:37 → running 15 s. Seed with the Bitwarden client crypto (PBKDF2 600 000, AES-CBC + HMAC): register (200), token (200), 10 login entries (200 × 10), /api/sync → 10 items, all names decrypt to the originals. Attachment: the v2 slot's returned url is /ciphers/<id>/attachment/<id> — posting it to the web root 404s, under /api 200 (a client detail, the harness's first try left one empty attachment slot on „Ügyfélkapu+"). The retried attachment downloads 200 with its recorded size; byte-equality not proven (harness lost the key). Finding R-512 (P2): registration stays open and the page's instruction to close it points at a read-only field; a stranger registered with no invite (200).

SECURITY — R-513 (P1), found 18:30Z while looking for a way to put a video on the drive

The launcher's Filebrowser tile opens files.enkicsifelhom.hu: password login, no signup. No screen gives the customer a FileBrowser login. A customer's first guess, admin / admin, works (200 + token); wrong password 401. With it, both sources list (Adatlemez → documents, …; Beolvasás → paperless). The same login works on demo-hp's 9201 and 9202, and 9201's login page answers 200 through Cloudflare from the internet (GET only, no login attempted remotely). Filed R-513 before anything else; nothing changed on any box.

immich (cont.) ML queues empty at 18:32:09Z (≈ 6 min after upload); asset 42 original read back sha equal.

paperless-ngx 18:27:xx deploy → running. Token with the deploy-generated admin password (the app card's „admin / admin" is not what was set — the form generates a 16-char password). Tags Számla / Garancia / Adó (201 ×3); 20 PDFs posted 18:29:57Z → 0 documents. Kernel: the container's cgroup OOM-killed gs and the celery worker at 18:31:22Z; tasks 11 FAILURE (WorkerLostError) · 1 STARTED · 8 PENDING, unchanged at 18:44Z. App still „Fut". R-514 (P2).

jellyfin 18:29:34 → running 50 s. Startup wizard through its REST calls (Hungarian UI culture, user „anna"), login 200; libraries none; the container sees /media/{audiobooks,books,comics,movies,music,photos,podcasts,tv} (the drive's userdata/media, read-only).

mealie → running 85 s. The card's default login changeme@example.com / MyPassword works (200); password changed (old now 401); 5 Hungarian recipes with ingredients + steps (201/200 ×5); meal plan 15–19 Sept (201 ×5) reads back all five; Gulyásleves reads back its accents.

uptime-kuma → running 50 s. Its socket.io endpoint answers the polling transport with the SPA page (websocket only); DooPlex has no websocket library — a raw client is written next.

uptime-kuma (cont.) The first screen of a fresh Uptime Kuma 2.4 is an English „Which database would you like to use?" page (SQLite / Embedded MariaDB / …, „Next") — the controller's card says nothing about it; the socket API is not up until it is answered. SQLite chosen through its own call (POST /setup-database → {"ok":true}), then over a websocket: setup „anna" (successAdded), login ok, 3 monitors (the box's own dashboard, the family wiki, the router by ping). Heartbeats after 90 s: router UP (0.5 ms); dashboard and wiki DOWN — „Request failed with status code 502": from inside the box, its own public names go out to Cloudflare and hit R-510. A household that adds its own apps as monitors gets red for every one.

jellyfin media via the product's own route — „Hálózati megosztás" (SMB): enable (303 „Beállítás mentve…"), password (first try used wrong field names → „legalább 8 karakter" flash, harness; second 303 „Megosztási jelszó beállítva"), share „Filmek" = existing folder userdata/media/movies (303 „A megosztás létrehozva."). State „fut"; list shows beolvasas (rendszer) and Filmek (Felhőmentés bekapcsolva). From demo-hp on the LAN (a household laptop): smbclient //192.168.0.150/Filmek -c put 98 898 905 B in 2.6 s. The drive root itself is refused for sharing („Ez a mappa nem osztható meg.").

jellyfin (cont.) the video, copied through \FELHOM\Filmek, is visible inside the container at /media/movies/Csaladi-nyaralas-2026.mp4; library „Családi videók" (movies, /media/movies) added (204); the scan finds „Csaladi-nyaralas" (RunTimeTicks 1 500 000 000 = 150 s) in 5 s; static stream of the first MiB → 206, 1 048 576 B.

adventurelog → running 161 s. Account creation took five harness attempts (evidence kept in order): the headless /auth/browser/v1/auth/signup through the frontend refuses without a usable CSRF token (the frontend clears csrftoken on every response); the backend's own /accounts/signup/ form works (302) — a household would use the frontend's sign-up page, which this harness did not drive. Then the frontend login form (POST /login) → sessionid (Domain .enkicsifelhom.hu) and through it: collection „Balaton 2026 nyár" 201; Tihany, Badacsony, Keszthely 201 with a visit each 201; Badacsony edited (rating 4) 200; read back 3 locations with their visits. Photos: 502 then 500 — see below. (An anonymous POST to /api/collections/ makes the backend raise ValueError … AnonymousUser → 500: the app's own behaviour, noted, not filed.)

Version labels (phase3/version-labels.txt, 18:56:27Z): all 12 app pages „Fut · Naprakész" (ASCII fragment Naprak = 1 each, control 0).

The memory guard never refused. All twelve deployed on the default box; the last deploy page before immich read 3 422 / 11 444 MB and before vaultwarden 3 734 / 11 444 MB; free in the guest at 18:45Z: 3 520 MB used of 11 828. How many apps fit on a default box: at least these twelve, with room left; the one app that ran out of memory did so inside its own container cap (R-514), which the guard does not see.

adventurelog photos — the frontend proxy logs RequestContentLengthMismatchError: Request body length does not match content-length header for the multipart POST: the R-483 class („photo upload fails from any non-browser client"; the operator confirmed the browser path on 2026-09-13). Not re-filed; the harness cannot prove the browser path.

Hub during Phase 3: 12 × app_deployed (info) for tester-1, 20:14:35 → 20:45:42 CEST; nothing else — no alarm for the Paperless OOM (R-514).

Phase 3 interventions: 0 (every seed through the app's own front door or the product's own SMB share;

harness retries are recorded, none needed an act a customer could not make). Rows: R-512, R-513, R-514, R-515, R-516. Evidence pulled off the box 19:00Z: box-logs-phase3/.

Phase 4 — a month of routines (phase4/)

Backup pages read before (phase4/backup-pages-before.txt). The nightly's legs, each through its own endpoint, in the nightly's order (window: db-dump W, Tier-2 W+60 m, off-site W+105 m, whole-system after):

leg	endpoint	start → end	result
Tier 1 — DB dumps + recovery units („Mentés most" on Alkalmazások)	POST /api/backup/run	19:00:26 → 19:02:38	db_dump count 6, 2m8s, success true
Tier 2 — off-drive copies	POST /api/backup/tier2 (no page calls it; the nightly does)	19:02:5x → (below)
Tier 3 — off-site	none: not configured on this record (restic off; PBS DR stuck, R-511)	—	not run
whole-system (Plane-2 local)	POST /api/guest-backup/trigger („Mentés most" on Áttekintés)	(below)

Catalog: no app had a newer version (12 × template == catalog, all catalog_since 2026-07-18). So a real one-step bump: privatebin 2.0.5 → 2.0.6 (upstream Docker Hub 2026-08-08), catalog commit d5d91e0 pushed 18:59:36Z with catalog_since 2026-09-14, gates OK. At 19:02:41Z the box still reads 2.0.5 (the 15-minute sync). R-499 still present on /stacks/bookstack/backup („már szerepelnek a teljes rendszermentésben (PBS)").

Tier 2 result (controller log, phase4/controller-log-t1-t2.txt): Tier 2 run complete: 12 app(s) processed at 19:03:06Z, 17 s. The eight system-disk apps copied to the data drive (hdd_1/backups/secondary/…, e.g. bookstack 154.9 MB); the four drive apps copied to the internal SSD marked [SSD: state-only] (immich 1.7 GB, nextcloud 1.3 GB, paperless 68.8 MB, jellyfin 0.6 MB) — definitions and databases, not the photos, as the Nextcloud page said („csak az adatbázis és a konfiguráció másolódik"). The status API's running flag covers only the DB leg, so it read false during Tier 2 (the harness loop exited on it; the log is the observable).

Whole-system „Mentés most" (POST /api/guest-backup/trigger 19:03:23Z; phase4/guest-backup-run.txt, guest-backup-quiesce-log.txt): the controller quiesced all 12 apps („backup due on 2 tier(s)"); tier local 19:03:49 → 19:09:59Z, 8 877 619 753 B, 362 s, success; then tier felhom-pbs started with the apps still stopped, failed (storage 'felhom-pbs' does not exist — the DR tier that never provisioned, R-511), unquiesce 19:10:09Z, last app started 19:11:12Z → apps down ≈ 7 m 45 s under a page line promising „csak néhány másodpercre" → R-518 (P2). Backoff logged: next PBS attempt in 15 min „so the apps are not stopped again for a backup that cannot succeed". Harness: the status API kept the previous job's done until the new job id appeared; the first poll stopped on it.

Backup pages, read as a first-timer after the run (phase4/backup-pages-after.txt, 19:15:36Z):

• Áttekintés: „✗ · Utolsó teljes mentés 2026-09-14 21:09 (5 perce) · 0 B · Biztonsági szerver – külön hardver (PBS) · Naprakész" and „✓ Távoli rendszermentés — külön hardveren (PBS)" — false on both counts; the real 8.9 GB local backup is not shown → R-517 (P1). Dates are local time (21:09 CEST) ✓. „DB mentések 6 fájl" ✓. „Következő mentés — 0 órája" (reads backwards, seen in the 0242 drill). Both honest warnings about one disk remain although a data drive is now enrolled — one of them offers „Adatlemez · Kijelölöm" as the whole-system target ✓.
• Alkalmazások: every app „1. mentés Auto helyi Utolsó: …"; drive apps' second copy state-only as said.
• /stacks/bookstack/backup still claims PBS coverage (R-499).
• Off-site integrity check: not possible on this record — „Távoli mentés": „Még nincs beállítva távoli mentési cél"; /backup/offbox/status → snapshots 0, status "". Verdict and duration: none — no tier to check (recorded, not run).

Hub after the routines (phase4/hub-check.txt, 19:15:31Z): „Tester 1 · ok · Controller 0.242.0 · Last report 2 min ago · Containers 24/24" (all 12 apps' containers listed), storage „SSD 33 % · Adatlemez 4 %", „Last DB dump 13 min ago", „No off-site data reported … this page cannot say". 13 × crossdrive_completed (info); 21:11:12 whole_guest_backup_failed (error) + operator e-mail — a TRUE alarm (the tier really cannot succeed), with no word of the local success. The hub page shows no whole-system backup line at all.

Catalog travel: the box synced at 18:59:00Z (36 s before the push) and picked the bump up at 19:14:01Z („Sablonok frissítve — frissítve: privatebin") — 14 m 25 s after the push. App page: „Frissítés elérhető — ma" (title „Újabb változat érhető el …"). PrivateBin before the update: 2 pastes decrypt equal; the 10-minute paste answers „Document does not exist, has expired or has been deleted." — honest expiry.

The guarded Update, privatebin 2.0.5 → 2.0.6 (phase4/privatebin-update.txt): POST /api/stacks/privatebin/update 19:16:33Z → „Frissítés elindult – az állapot a kártyán követhető". Phases: checking → precondition met — Tier 2 (second drive) copy from 19:03:05Z (13 m old, limit 24 h) → safety-dump (no database, no-op) → pinning („pin advanced to the catalog's current definition") → pulling → starting → verifying → „healthy after 5s" → installed-images recorded privatebin/pdo:2.0.6 (sha256:4c141b23…) → DONE in 11 s. Container privatebin/pdo:2.0.6 (healthy). App page „Fut · Naprakész". Data read back after: both surviving pastes decrypt equal; the expired one still expired. PASS. (Observation: template_images in the API still read 2.0.5 for ≈ 15 s after updating went false.)

Catalog revert a161ccb pushed 19:17:55Z (image back to 2.0.5; catalog_since stays 2026-09-14 by the gate). Phase 4 evidence pulled off the box 19:18Z (box-logs-phase4/).

Phase 5 — the accidents (phase5/)

Method, the same for every fault: a family loop (family.py) keeps three apps busy (BookStack page loads each second, a Nextcloud note written every 2 s, Mealie API each second) and logs ok/fail per 5 s; watch_recovery.py records from the fault's T0 when the dashboard's /api/health answers and when each of the 12 apps is running again with the same images as the Phase 4 baseline; alarms.sh reads the hub's events, staleness changes and operator mails for tester-1; the Gmail connector is read for what reached the customer's and the operator's mailbox. For each fault: what the customer saw · what the box did alone · time to steady state · which alarm fired and was it true · which alarm should have fired and did not. Evidence off the box before the next fault.

Before F1 — does the failed PBS tier stop the apps again? The controller deferred the tier 15 min at 19:11:12Z („so the apps are not stopped again for a backup that cannot succeed"). Read at 19:27:40Z (below).

Quiesce retry check (19:27:40Z): no quiesce or stack stop between 19:18 and 19:27Z — the deferred PBS tier did not stop the apps again. The agent's own attempt still failed at 21:27:43 CEST (hub WARN host … backup FAILED: target=felhom-pbs … storage 'felhom-pbs' does not exist), no operator mail for it.

F1 — power cut while the family uses three apps (phase5/F1-power-cut.txt, phase5/F1/, family-activity-F1.txt)

how	qm stop 333 19:28:08Z (hard) · 61 s off · qm start 19:29:09Z
what the customer saw	from 19:28:12Z every request to wiki / cloud / recipes fails (connection refused/timeouts) for ≈ 3 m 30 s; BookStack answers again 19:31:37Z, Nextcloud 19:31:42Z, Mealie after 19:32:02Z; the dashboard asks for login again
what the box did alone	dashboard /api/health 200 at +133 s; privatebin, vaultwarden, jellyfin +134–135 s … immich, adventurelog +191 s; paperless-ngx left „boot-orphaned" and started by the boot reconciler at 19:32:26Z („1 app(s) recovered in 1 attempt(s)"); all 12 running at +243 s, every one on the same images as before
time to steady state	4 m 03 s from power-on
alarms fired	hub: controller_started (info) — Controller elindult (0.242.0) only — true. Re-read at +10 min (past the 90 s dead-app grace): no app_start_failed, no false alarm
should have fired, did not	none — a 61 s outage is under the hub's 30-minute staleness threshold, by design

Verdict PASS. Jellyfin's health probe answered 503 / refused for ≈ 45 s after its start (WARN lines), then healthy.

F2 — power cut during the nightly backup (phase5/F2-power-cut-during-backup.txt, phase5/F2/)

how	POST /api/backup/run 19:40:03Z; the harness waited for the first „Stopping … for safe volume dump" (adventurelog, 19:40:07Z) and cut power at 19:40:08Z; 33 s off; qm start 19:40:41Z
what the customer saw	every app down ≈ 3½ min, as F1. Afterwards /backups/apps: „Utolsó adatbázis mentés 2026-09-14 21:40 (5 perce)", six databases „21:40 · OK", every app „Utolsó: 5 perce"; /backups and /dashboard: no word of an interruption; dashboard tile „Utolsó mentés: 2026-09-14 19:40" (UTC beside local times — R-500); whole-system tile „– · Naprakész" with no backup listed (added to R-517)
what the box did alone	controller 19:42:48Z: [appstop] crash recovery: an app-data backup (volume dump) … was interrupted and left 1 app(s) stopped — restarting them: [adventurelog] → restarted 19:42:52Z — the app-stop guard restarted what it stopped ✓. All 12 running on the same images at +245 s. AdventureLog's trip read back intact (3 places, visits).
torn copy	on disk the adventurelog and bookstack units hold a 19:40:07 SQL dump beside 19:00 volume tars and a 19:02:34 manifest; both restore points are dated 19:40:07Z → R-519 (P2)
alarms fired	backup_failed (error) — an app-data backup (volume dump) was interrupted by a controller restart — 1 app(s) were left stopped and have been restarted + operator e-mail — true; controller_started (info) — true. No false dead-app alarm at +10 min
should have fired, did not	a customer-side notice on the backups page (R-519)

F3 — power cut during a guarded Update (phase5/F3-power-cut-during-update.txt, phase5/F3/)

how	POST /api/stacks/nextcloud/update 19:52:02Z (no newer version exists after the Phase 4 revert, so a same-version guarded update — the only Update the catalog allows tonight); a live log follower cut power on the line update nextcloud: phase pulling (19:52:03Z) → qm stop 19:52:05.8Z; 30 s off; qm start 19:52:36Z
journal before the cut	checking → precondition met (Tier 2 copy 49 m old) → safety dump written (…/db-dumps/pre-restore-20260914T195203Z-nextcloud-mariadb.sql) → pinning („pin advanced to the catalog's current definition", unchanged images) → pulling
what the box did alone	all 12 running on the same images at +230 s; at boot the gate recreated nextcloud onto the live bind („live bind confirmed — recreating drive-backed app nextcloud"); pin adoption: 0 pinned, 12 already pinned. No log line resumes, aborts or even mentions the interrupted update; app.yaml has pinned_images = installed_images, no hold, no verdict field
pin state	consistent (same definition)
the page's message	„Fut · Naprakész"; no word of the interrupted update (fragments frissít, megszak, vissza, hiba, tartva absent)
data	Nextcloud status.php installed, not in maintenance; 4 files read back sha equal; 200 photos listed
alarms	(below)

Verdict: recovered, but the journal's honesty is UNPROVEN for a real version change — with the images unchanged there is nothing to resume or abort, and nothing records that an update was cut. Not filed as a defect (no wrong state was produced); recorded as a gap in what tonight could test.

F3 alarms (re-read 20:03:09Z, +10 min): controller_started (info) only — true; no false alarm.

F4 — the data drive is unplugged while apps run (phase5/F4-drive-unplugged.txt, phase5/F4/)

how	qm set 333 --delete scsi1 at 19:57:27Z on the running VM (the disk file stays as unused0)
how fast the drive apps stop	Nextcloud's front door already 503 at +5 s; controller storage_disconnected at 19:58:02Z (+35 s); all four drive apps (nextcloud, immich, jellyfin, paperless-ngx) stopped by +38 s, Nextcloud front door then 404
do system-disk apps keep running	yes — bookstack, vaultwarden, mealie, docmost (and the other system-disk apps) running throughout; BookStack front door 200
what the customer saw (20:04:54Z)	banner on every page: „4 telepített alkalmazás nem fut — nézze meg a rendszermonitort · Rendszermonitor →" and „Meghajtó leválasztva: Adatlemez (/mnt/felhom-drives/hdd_1)" (printed twice, once with „Beállítások →", once with „Rendszermonitor →"). Tárhely: „Adatlemez · Leválasztva · Leválasztva: 2026-09-14T19:58:02Z · Leállított alkalmazások: immich, jellyfin, nextcloud, paperless-ngx · Csatlakoztatás". App page: „Nextcloud · Leállítva · Naprakész · Hiányzó tárhely: Adatlemez — Ennek az alkalmazásnak az adattárolója jelenleg nem elérhető, ezért le van állítva. Csatlakoztasd újra a meghajtót, vagy helyezd át az adatokat egy másik tárhelyre." Dashboard: „12 Futó · 4 Leállítva · Adatlemez — Leválasztva". App list: each drive app „Leállítva · Naprakész · Hiányzó tárhely: Adatlemez". Honest, Hungarian, and says what to do. Blemishes: a raw ISO UTC timestamp; the duplicated banner; formal „nézze meg"
alarms fired	21:58:02 CEST storage_disconnected (error) — Meghajtó váratlanul leválasztva: Adatlemez + operator mail — true; 21:58:15 app_start_failed (warning) × 4 (Paperless-ngx, Jellyfin, Immich, Nextcloud) + four more operator mails — true but redundant: one unplug produced five operator e-mails
should have fired, did not	nothing reached the customer's mailbox (to:tester1@felhom.eu unchanged) — the household learns of it only by opening the dashboard

Fail-closed path, +8 min (phase5/F4/fail-closed-path-8min.txt): on the host and in the guest /mnt/felhom-drives/hdd_1 is still a mountpoint on the dead device — every read answers Input/output error; df still shows /dev/sdb 98G 3.6G. Writes cannot land on the system disk underneath (0 files visible). Rows R-520, R-521 filed; F4 copy blemishes added to R-516.

F5 — the drive stays out 30 min, then comes back (phase5/F5-drive-back.txt, phase5/F5/)

how	F4's evidence pulled 20:27:31Z; qm set 333 --scsi1 …vm-333-disk-2.qcow2 at 20:27:31Z (out 1 804 s)
re-bind	the disk returns as /dev/sdc (the old sdb mount stayed shutdown the whole time); EXT4-fs (sdc): recovery complete, mounted by filesystem UUID c4b530fd… at /mnt/felhom-drives/hdd_1 on host and guest
restart	the box restarted the four drive apps by itself: immich / jellyfin starting at +36 s, Nextcloud front door 200 at +58 s, all four running at +91 s; system-disk apps never stopped
the badge	Tárhely: „Adatlemez · Alapértelmezett · Aktív · 3.5 GB / 97.9 GB · ext4 · /dev/sdc · 4 alkalmazás használja (Immich 529.1 MB, Jellyfin, Nextcloud 389.5 MB, Paperless-ngx 0 B)"; dashboard „16 Futó · 0 Leállítva · Adatlemez 3.52 GB / 97.9 GB". But the banner „Meghajtó leválasztva: Adatlemez" (×2) was still on every page at 20:29:23Z — re-checked below
alarm clears	hub 22:28:25 CEST storage_reconnected (info) — Meghajtó újra csatlakoztatva: Adatlemez (no mail) — true
written to the fail-closed path meanwhile?	no — while out, the path stayed a mountpoint on the dead device and every access failed with Input/output error (host and guest, 20:05Z); no write could land on the system disk beneath. After return, 11 of 11 sampled Nextcloud files sha equal, „Közös mappa" lists 398 entries

F5 banner re-check (phase5/F5/banner-10min.txt, 20:37:54Z): „Meghajtó leválasztva" count 0 on dashboard and storage page (control „Tárhely" present) — the stale banner cleared within 10 minutes of the drive's return.

F6 — the drive is unplugged during a backup (phase5/F6-drive-out-during-backup.txt, phase5/F6/)

how	POST /api/backup/run 20:37:58Z; unplugged at 20:37:59Z, during DB dump: nextcloud-db (1 s in); re-attached 20:42:09Z
what the backup did	DB dumps of all six databases completed (immich's to a .tmp at 20:38:00 that was never promoted); system-disk apps' volume dumps continued normally (adventurelog, bookstack, docmost, gokapi, mealie, privatebin, … each stopped and restarted); at 20:38:33Z [gate] drive ABSENT … stopped+blocked 4 app(s); then „Skipping volume dump for immich / jellyfin / nextcloud / paperless-ngx — drive disconnected"; run end db_dump {"count":6,"duration":"55.7s","success":true}
torn copy handled?	partly. Immich's point stayed honestly at 19:02:34Z (the .tmp not promoted). Nextcloud's point jumped to 20:37:59Z (its SQL finished before the unplug) beside 19:01 volume tars. The run that skipped four apps says success:true; /backups/apps shows no skipped app (fragments kihagy/sikertelen/részleges = 0) → added to R-519
drive back	all four drive apps running again at +122 s after re-attach
the next run's honesty	POST /api/backup/run 20:47:24Z → 2 m 11 s, success:true; all four drive apps dumped (immich, jellyfin, nextcloud, paperless volume dumps 20:48–20:49Z); points nextcloud 20:48:59Z, immich 20:48:13Z, jellyfin 20:48:26Z, paperless 20:49:15Z; immich's torn .tmp gone. Honest. One leftover: F3's pre-restore-20260914T195203Z-nextcloud-mariadb.sql.tmp still in nextcloud's unit
alarms	storage_disconnected (error) + 4 app_start_failed — all five operator mails suppressed by cooldown (a second, separate drive loss 40 min after the first, a reconnect in between) → added to R-521; health_degraded (warning) mailed — true

F7 — the system disk fills up (phase5/F7-disk-full.txt, phase5/F7/)

how	fallocate -l 41 591 249 306 in the guest at /mnt/sys_drive/felhom-data/userdata/import/F7-nagy-fajl.bin at 20:50:52Z (sys_drive 69 G: 36 % → 95 %, 3.5 G left). The box's thin pool did not grow (fallocate on ext4: vm-9201-disk-1 data 36.94 %)
what the customer saw	dashboard from +3 s: „Rendszer (/) · 61.8 GB / 68.7 GB (90%) · Kritikusan kevés hely" — 90 %, while df says 95 % (the tile ignores the filesystem's reserved blocks; and the tile says „/" for the data volume /mnt/sys_drive). From ≈ +5 min a banner on every page: „SSD disk usage high: 90%" — English · „Rendszermonitor →". /backups: no warning. Deploy page (homebox): memory line only, nothing about the disk
does the box stay reachable	yes — dashboard and BookStack 200 throughout
what refuses first	nothing refused: a 4 MiB BookStack attachment still uploaded (200); a backup was started on the full disk (below)
alarms	hub 22:54:46 CEST health_degraded (warning) — operator mail suppressed by cooldown (the F6 health_degraded 15 min earlier). No disk-specific event reached the hub
a backup on the full disk	POST /api/backup/run 20:58:5xZ → finished 21:00:57Z, success:true; sys_drive stayed 95 % (units are replaced in place; the 3.5 G left sufficed); no no space/ENOSPC line; controller [monitor] Disk (SSD) threshold breached: 90% (limit: 80%); apps running after (vaultwarden briefly starting from its own volume dump)
recovery	file removed 21:01:19Z → sys_drive 36 %; the English banner gone at +212 s (21:04:53Z); hub 23:04:45 CEST health_recovered (info) — Rendszer állapot helyreállt: ok (volt: warn)
alarms	see above: health_degraded (mail suppressed by cooldown) → health_recovered; no disk-specific alarm → R-521 amended; the English banner → R-516 amended

Verdict PASS for the box (reachable, nothing refused or broke at 95 %, the backup still completed); the operator was not told.

F8 — the internet goes away for 20 minutes (phase5/F8-internet-gone.txt, phase5/F8/)

Harness slip, attempt 1 (21:05:19Z): the bridge-filter chain was named fwd, a reserved word in nft; the chain was never created, so nothing was cut (guest → hub still 302). Stopped after 1 minute; evidence kept as F8-attempt1-HARNESS-SLIP-rule-not-applied.txt; the half-made empty table removed. The rule was rewritten as a file, dry-run checked (nft -c -f → OK) and applied for attempt 2. Scope: only tap333i0 (VM 333); 9201/9202 untouched.

Attempt 2 (21:06:25Z): rule applied, counters 0 → debugged with counting tables: VM 333's frames do pass the bridge forward hook; the real reason the guest still reached „the internet" is that hub.felhom.eu resolves to 192.168.0.192 on this LAN (the hub runs on DooPlex's ingress), so the probe never left the LAN. Stopped; tables removed.

Attempt 3 (the measured run), cut at 21:08:36Z — VM 333 may reach the LAN (DNS 192.168.0.250 / .1, household devices) but not the internet and not the hub's LAN ingress 192.168.0.192, which is what a household loses when its internet goes. The LAN dashboard is probed from demo-hp (a household laptop) instead of DooPlex. Drop counters at 21:09:26Z: 27 packets to the hub, 126 to the internet, 6 IPv6 — the cut is real. Guest probe (fixed script, phase5/F8/guest-wan-probe.txt) at 21:09:48Z: guest->1.1.1.1=fail guest->hub=fail cloudflared(last120s registered=1 errors=45). (The F8 runner's own guest column prints a shell quoting error — harness; the separate probe file is the observable.) Hub at 21:08:42Z (just after the cut): „Last report: 14 min ago · warn" — not a stall: the controller's hub-report job runs every 15 min (pushed 20:39:46Z and 20:54:46Z, Hub report pushed successfully); the „warn" is the state that last report carried (F7's disk). The next push, due 21:09:46Z, fell inside the cut.

Phase 6 pre-note (written 21:13Z, before the phase): the brief asks to restore one DB-backed app from off-site onto 9202. On this record there is no off-site copy to restore from — restic Tier 3 was off (and not ticked: money), and the DR tier never provisioned (R-511); /backup/offbox/status → snapshots 0. Not followed, one line why: the promise „restore from off-site" cannot be walked without an off-site tier, and faking one (copying a unit by hand onto 9202) would prove a path no customer has. Phase 6 instead restores a DB-backed app from its local copy on the box itself, through the product's restore endpoint, and reads the data back — recorded as such.

F8 result (attempt 3) (phase5/F8-internet-gone.txt, phase5/F8/):

how	bridge rule on demo-hp, tap333i0 only: 21:08:36 → 21:26:07Z — 17 m 31 s, not the brief's 20 (the runner's loop count was short — harness). Dropped at removal: 236 packets to the hub, 1 393 to the internet, 54 IPv6
LAN dashboard still works?	yes — 200 on every probe from demo-hp (every 26 s); polled pages (every 2 min) showed no banner at all and the tile „Cloudflare Tunnel … Fut · Védett" the whole time → R-522 (P3)
what the box did	cloudflared ≈ 20 errors per 2 min (failed to dial to edge with quic: timeout), retrying; controller [report] Push failed … context deadline exceeded 21:11:26Z, Job hub-report failed … after 3 attempts, wait channel backing off; public name 502 → 530 (no connector)
on return	report pushed at 21:26:07Z, the same second the rule went; tunnel re-registered +9 s (21:26:16Z), all four connections +58 s (21:27:05Z) — reconnected by itself ✓; public name back to 502 (R-510)
hub shows DOWN, and when	never DOWN (threshold 1 h). Status „warn · Last report N min ago" climbing 14 → 31 min; node_stale 21:25:43Z (31 min after the last report, 17 min into the cut) + operator mail; node_recovered 21:26:43Z + mail
alarms true?	node_stale true; node_recovered true — a cut shorter than ≈ 13 min after a report would raise nothing, by design
should have fired, did not	a customer-side notice that the box is offline (R-522)

F9 — the controller dies in the middle of a deploy (phase5/F9-controller-killed-mid-deploy.txt, phase5/F9/)

how	deploy of a throwaway app (homebox) POST 21:34:35Z → „Telepítés elindítva" (with a memory warning: „Az alkalmazások csúcsterhelése meghaladhatja a rendelkezésre álló memóriát. Normál használat mellett ez nem okoz problémát.") → docker kill felhom-controller at 21:34:39Z (4 s in)
what the box did by itself	nothing brought the controller back: Exited (137), restart=unless-stopped — Docker does not restart a container stopped by kill; the in-guest felhom-controller-bootstrap unit is a one-shot („active (exited)" since boot) and does not watch it. 5 min later still dead, dashboard 502 on the LAN
the deploy	the app itself came up: container homebox Up 5 minutes (healthy); app.yaml and applied-compose.yml written at 21:34
what the customer saw	the dashboard answers 502 — no page at all
23 min later (21:57:39Z)	still Exited (137) 22 minutes ago, LAN health 502; hub shows only app_deployed (info) — Homebox (the controller's last report reached the hub at 21:34:38Z, 3 s before the kill, so the 30-minute staleness is due ≈ 22:04:38Z). The host agent's journal meanwhile: drive reconcile and stale-lock scans every 20 s, nothing about the controller
33 min later (22:08:04Z)	still Exited (137) 33 minutes ago, LAN 502. Hub 22:04:43Z node_stale — operator mail suppressed by cooldown (key tester-1:node_stale, set by F8 at 21:25:43Z)

STOP RULE MET (22:08Z). F9 left the box in a state the product did not recover from on its own (33 min) and that a customer cannot recover from any screen (there is no dashboard). Filed R-523 (P1) before acting. Per the brief: no further faults are injected — F10, F11 and F12 are not run; the box is recovered by the household's only lever, a power-cycle, and the night moves to Phase 6.

F9 recovery — the household's only lever, a power-cycle (phase5/F9/recovery-power-cycle.txt, controller-after-reboot.log): qm reset 333 22:08:17Z → the in-guest bootstrap started the controller at boot (22:10:23Z, restarts=0); dashboard 200 at +131 s; all 12 apps running at +229 s (paperless again started by the boot reconciler); homebox running, deploying false, deployed true, „Fut · Naprakész" — the interrupted deploy is not stuck. Hub: controller_started (info) 22:10:32Z, node_recovered 22:10:43Z — mail suppressed by cooldown. So a reboot heals it; nothing short of a reboot does, and no screen tells a household to reboot.

Phase 6 — the morning after (phase6/)

Every app healthy? Every label true? (phase6/morning-after-check.txt, 22:13:14Z): all 12 apps running, no container down or unhealthy. Labels: 11 × „Fut · Naprakész" with installed == catalog — true. privatebin: „Frissítés elérhető — ma" while installed 2.0.6 and catalog 2.0.5 (after the Phase 4 revert) — false: the offered Update is a downgrade → R-524 (P2). (The check script's own „label true" verdict for privatebin was wrong — it compared for difference, exactly as the product does.)

Every backup page honest? (phase6/backups-overview-morning.txt): „DB mentések 6 fájl" ✓; „Távoli rendszermentés — nincs beállítva" ✓ (honest again after the reboot); whole-system tile „– · Utolsó teljes mentés – · Méret / cél · Naprakész" — no backup shown, labelled current (R-517); „Következő mentés — 3 órája" (R-500 class); „Pillanatkép-mód: … csak néhány másodpercre" (R-518). /backups/restore lists 13 apps incl. Homebox ✓.

Restore from off-site onto 9202: not walked — no off-site copy exists on this record (see the pre-note). Instead a DB-backed app (BookStack, MariaDB) is restored from its local copy on the box through POST /backup/restore, after a page is deleted and the recycle bin emptied (below).

Local restore of a DB-backed app (phase6/local-restore-bookstack.txt, …-readback.txt): before — 5 pages 200, 2 attachments sha equal, „Wifi jelszó helye" text present. Accident 22:14:36Z: the page deleted (302 → 404); the harness's „empty recycle bin" call used a wrong route (404), so the page sat in BookStack's bin — the restore does not depend on that. Point offered: 2026-09-14T20:59:20Z helyi · Belső SSD (rendszer). POST /backup/restore stack_name=bookstack snapshot_id=helyi 22:14:37Z → finished 22:15:01Z (24 s): „A(z) bookstack: 2 adatkötet és az adatbázis visszaállítva — az alkalmazás újraindult." After: all 5 pages 200, the deleted page's text back, both attachments sha equal. PASS. (Harness: the poll loop missed the status's last object and ran on; the read-back was taken separately at 22:24:57Z.)

Phase 6 evidence pulled off the box 22:26Z (box-logs-phase6/), before any teardown.

Phase 7 — teardown, three layers

Layer 1 — the machine (teardown-before.txt, teardown-layer1-machine.txt), 22:25:58Z: qm stop 333, qm destroy 333 --purge 1 --destroy-unreferenced-disks 1 (system and data qcow2 on nvme-scratch removed; /mnt/hdd_1/images/ now holds only 9202); ISO 1.27.1 removed from local:iso; /root/bn harness files removed; nft: only inet felhom_oob (the drill's bridge table was already deleted at the end of F8). pvesm status before → after: nvme-scratch 59 436 396 → 10 140 556 KiB (before the drill 10 134 820); local 24 728 328 → 23 071 188 KiB (before the drill 23 041 736); local-lvm 44.17 % unchanged all night. qm list empty; 9201 and 9202 running. pct fstrim: not applied — the drill used only qcow2 files on a dir storage, now deleted; 9201 and 9202 were not touched and are not trimmed.

ep0 read-only before the host delete (teardown-ep0-before-host-delete.txt, 22:26:02Z): WireGuard peer 10.77.0.5/32 present; namespaces demo-felhom demo-hp tester-1; tester-1/ct/9201 with 1 snapshot directory (the doorstep walk's data). Nothing was written to ep0 tonight.

Layer 3 — host record and ep0 peer (teardown-layer3-hub.txt): the destroyed box's last host report reached the hub at 22:23:45Z; delete-impact turned stale · deletable at ≈ 22:54Z. Harness slip: the first POST /hosts/tester-1-a61396/delete (22:54:40Z) sent no fields → 400, hub host delete refused: … confirm mismatch; the host record and the ep0 peer stayed (ep0 read at 22:58:42Z: peer 10.77.0.5 present, tester-1 snapshot dirs 1). The handler wants confirm_host_id = the host id (its test D3). Attempt 2 below. Attempt 2 (22:59:21Z) with confirm_host_id=tester-1-a61396 → 303 /hosts; hub host deleted: tester-1-a61396 (escrow deleted: false); host page 404; not on /hosts; customer tester-1 page 200 — kept. The first ep0 re-read (23:00:23Z) still showed the peer, because the last peer sync (22:59:13Z) predates the delete by 8 s — harness read too early; re-read after the next sync below. ep0 after the next sync (23:04:13Z wgsync: pushed 4 peers; read-only 23:04:33Z): peer 10.77.0.5 gone (0), control peer 10.77.0.3 present; namespaces demo-felhom demo-hp tester-1; tester-1 still holds 1 snapshot directory — kept, stated for the operator. Teardown complete: machine, host, hub. The night's secrets in the session scratchpad (hub password copy, break-glass password, passphrase, codes, app passwords, bind token) shredded at 23:04Z; a search of the scratchpad for leftover tokens found none.