felhom.eu/documentation/audits/CAMPAIGN-11-recovery-journey-2026-08-05.md

CAMPAIGN 11 — the recovery journey (2026-08-05 → 08-06)

Four phases. Phase 1 (the clean journey) FAILED on the journey and PASSED on the data. Phase 3 (key supersession) PASSED on its central question. Phase 2 (eleven injected faults) and Phase 4 (an unattended soak) ran overnight on 2026-08-05/06 and are reported here for the first time.

Findings: R-214 … R-223 from Phases 1 and 3 (seven since fixed and shipped), plus R-224 … R-228 from Phase 2. Three suspicions investigated — two confirmed, one DISPROVED. Five harness faults, separated from the product's.

Evidence: ../tests/campaign11-evidence-2026-08-05/ — journal.md (Phases 0, 1, 3) and journal-phase24.md (Phases 2 and 4, every observable in the order taken).

The one-line answer to the question this campaign was built to ask. A Hungarian household whose machine is rebuilt gets their data back only if an operator is standing next to them. The cryptography, the retention and the transport all work and are now proven live. What fails is being told the truth: on this box, four different situations — a mistyped code, a hub outage, a stopped agent, and a correct code for a retained earlier package — produce one message, and three of the four are wrong.

---

ANNOTATION 2026-08-06 — what has since been fixed. The body below is NOT rewritten. This document records what was true when the campaign ran, and that is its value; the fixes are recorded here and in felhom-controller/REPORT.md.

R-224, R-226, R-225, R-227, R-228 are CLOSED in controller v0.202.0 + agent v0.126.0. The unlock path now classifies why it failed — from the value, never the text — and the message that mentions typing is reachable only after a real refusal; anything unclassifiable renders a neutral message rather than an accusation. Proven live on this venue: same wrong code, hub up → 400, hub REJECTed → 502 naming the connection and stating the code was not used, hub restored → 400.

Unchanged by that work: the campaign's verdict, the RTO, and the capability map's recovery row, which stays FAIL until a re-walk passes. R-214, R-220, R-221 remain open, and R-220 is still worked around by hand on this venue.

1. Venue and baselines

Host	demo-hp (HP t740), Tier 0, the designated drill host. Reached by SSH key, first try — R-129 stands
VM	321 c11-appliance — q35/OVMF, 4 cores, 8 GB, cpu=host
Disks	scsi0 200 G system · scsi1 50 G · scsi2 50 G, qcow2 on c11-scratch
Storage	c11-scratch, dir at /mnt/nvme-1tb — the mount ROOT (a subdirectory fails the agent's exactMount check; Campaign 10 §1)
Appliance	192.168.0.105, hostname c11, felhom-agent 0.125.0
Guest	LXC 9201, 192.168.0.106 — DHCP, and it MOVED between phases (.207 → .227 → .106)
Hub customer	c11 "Campaign 11", domain c11.felhom.eu, DR tier ON, off-site ON
Host id	c11-36d660
Untouched	drill-r50 (VM 300, stopped), guest 9201 on both demo boxes, local-lvm, felhom-backup, every other hub customer

Baselines — every value re-read fresh at the start of Phase 2

What	Value	How
felhom-controller main	v0.201.0 @ 05cf352a2f17	HEAD == origin/main, tree clean
felhom-agent main	v0.125.0 @ 0404f60e6a7b	same
felhom.eu main	@ 3a539ea5306a	same
hub, LIVE	felhom-hub:0.97.1	kubectl … get deploy hub -o jsonpath
Day-0 manifest	agent 0.125.0 · golden 0.201.0 · min_agent 0.125.0	hub /configuration, selected options
c11 floor	v0.200.0 (override); every other customer v0.156.0	hub /configs
Highest register ID	R-223	fresh grep -rhoE 'R-[0-9]{1,3}' over all four repos

All four cited commits match the brief exactly, and the manifest matches the golden rebake the Phase-1/3 journal records. Blast radius of the per-customer floor is still zero, measured — the other five customers all read v0.156.0.

The hub CHANGELOG/deployed mismatch — diagnosed more precisely than filed

The brief records "0.97.1 shipped without an entry". The content was not missing; its heading was. Commit a7f1d27 wrote the change into the v0.97.0 entry and 79e31ac bumped the manifest to 0.97.1, so a reader matching the running tag against hub/CHANGELOG.md found no v0.97.1 heading at all. Fixed in this session — the paragraph now has its own entry, marked as added retroactively. Second occurrence of the class (the first was agent 0.90.1).

---

2. Scope, and what was deliberately not isolated

Campaign 10 could run with Tier 3 OFF; a campaign about off-site recovery cannot. Off-site hard-requires the DR tier, which provisions on ep0 (Tier 2). The Phase-0 operator ruling stands and is restated here rather than quietly inherited: ep0 and the Hetzner Storage Box are written to, additively — a PBS namespace and token, a WireGuard peer, and a Storage Box sub-account, all created on the ordinary customer path. Nothing existing is modified or deleted. The brief's I7 wording ("ep0 read-only") was relaxed by that ruling, not widened by this session.

Phase 2 added no new external writes. Its faults are network blocks, a service stop, a container restart and a VM shutdown — all on the campaign's own appliance, all reverted, each with a positive control proving the fault was real.

---

3. The two positives that were owed (brief §4)

§4.1 — is the floor actually SERVED? MEASURED. YES. Twice, independently.

The previous session recorded "no HELD line and no held reason" — two absences — and correctly refused to call that a measurement. Both positives were taken here.

(a) The box's own rendered state. /settings → „Automatikus frissítés – Minimális verzió (üzemeltető) 0.200.0". That value is s.updater.GetFloor(), and u.floor has exactly one writer — SetFloor, whose only non-test caller is the report-ACK handler. Both hold branches of ResolveManagedFloor set Floor = "" (store.go:2086, :2095), pinned by managed_floor_test.go:94. A non-empty floor on the box therefore proves an ACK carried one.

(b) A cold-started process saying it out loud. F8's restart produced the decisive A/B — the same log line, same box, same code, before and after the golden rebake:

15:58:35  settle-gate: GO — floor still unknown after 1m30s               ← hold IN FORCE
21:06:16  settle-gate: GO — at/above floor 0.200.0 (we are 0.201.0)       ← hold RELEASED

The hub half, as a positive rather than an absence: managed floor HELD for c11 … fired every 15 minutes from 17:57:11 to 22:12:06 and then stopped — and the silence is backed by a liveness control (the hub logged host-report from demo-hp-bb76ea at 22:44:41 and wgsync: pushed 5 peers at 22:44:57, so it was demonstrably still logging).

A correction to the brief's own plan. It says to take this at DEBUG during F8's restart. A restart alone would not have produced it: SetFloor's line is u.dbg(...), gated on a private flag set from cfg.Logging.Level == "debug", and it writes to the logger — never to the logx debug ring, so it cannot appear in /api/debug/logs at any level. Confirmed live: 4 000 ring entries spanning the release window contain no SetFloor line, with a level census run first (1196 DEBUG / 2802 INFO / 2 WARN) so the absence was known to be structural rather than evidential.

§4.2 — R-218's live half: still NOT measured, deliberately, with the reason

The fix is present and readable — needsOffsiteCredential now retires the declaration on the target, not on the key:

if t != nil { return false }   // a TARGET exists — not a rebuild

The venue cannot exercise it. c11 has a target (applied_marker present since 14:57), so t != nil and the box correctly does not declare; declaring here would be the bug. The state that exercises the fix — hub identity blob present, key placed, no target — is shape (a), which the venue held during Phase 1 and does not hold now. F7's set-aside was examined as a route to it and does not produce it either (§5, F7).

Recorded as still not measured rather than inferred from the unit test. It needs one rebuild, which the brief forbids before Phase 4.

---

4. Phase 2 — the eleven faults

Judged on the message, not the outcome. Every fault carries a positive control proving the fault was real; every control that failed is reported as a failed control, not as a result.

The four messages (internal/web/recovery_handlers.go, v0.201.0)

	fires when	first words
M1	unseal failed and no superseded package	„A megadott helyreállítási kódot nem fogadtuk el. Ellenőrizd, hogy mind a tíz szót…"
M2	the capability gate says the agent cannot do it (R-216)	„Ez a gép még nem tudja megnyitni a mentéseidet… A kódoddal semmi baj…"
M3	unlocked, inventory unreadable (R-217)	„A kulcs visszakerült, de a mentések listáját most nem sikerült beolvasni…"
M4	unseal failed and a superseded package is retained (R-222)	„Ez a kód nem nyitja meg azt a csomagot, amit most őrzünk ehhez a géphez…"

The venue retains a superseded package, so M4's branch — tested before M1 on the failure path — is live for every failed unlock. That single ordering fact produces two of the four new findings.

Results

	Fault	Right answer	Result
F1	wrong code ×3	refused, M1 only, no lockout, nothing written	PARTIAL — refused ✅, no lockout ✅, nothing written ✅, but M4, not M1 → R-226
F2	no code at all	states nobody can recover it; offers set-aside	PASS
F3	hub unreachable	names the hub, never the code	FAIL — M4. → R-224
F4	agent stopped	M2, the R-216 fix under pressure	FAIL — M4. → R-224
F5	store unreachable after a successful unlock	unlock counts; „could not be read", not „opened with content"	PASS — R-217's fix holds
F6	„Most nem", return later	entry point survives; unlock still works	PASS
F7	set aside, then a change of mind	set aside, never deleted; honest afterwards	SPLIT — set-aside PASS, verified byte-for-byte; the afterwards FAIL → R-228
F8	controller restarted mid-unlock	no half-state; the screen says which	PARTIAL — no half-state ✅, but a raw English Bad Gateway → R-227
F9	a box that never had off-site	nothing, incl. a direct GET /recovery	PARTIAL PASS — R-215's gate proven live on a narrower shape; the literal precondition was not staged
F10	app with a mandatory data path missing	run reports incomplete, names the app, digest carries it	NOT INJECTED — harness. Three attempts, each self-healed
F11	box offline a whole reporting window	hub and box agree once it returns	PASS — both directions fired, each with an operator mail

F7 — the set-aside, and the afterwards

The move-aside is exactly what it claims. Both confirmations state the consequences first („félretesszük — nem töröljük", „a gép új, üres mentési tárolót kezd", „ez az oldal többé nem jelenik meg"), and the result measured against the Storage Box:

BEFORE  /home/felhom-repo                    mtime 13:13   snapshot f3d9cd67   du -s 12535
AFTER   /home/felhom-repo.orphaned-20260805  mtime 13:13   snapshot f3d9cd67   du -s 12535   ← untouched
        /home/felhom-repo                    mtime 21:32   (empty)                           ← fresh

Nothing deleted. Then the customer changes their mind — and finds nothing: GET /recovery 302s, POST /recovery/unlock 302s in 0.028 s with no message, and /backups/remote never mentions the set-aside history at all, though the box records its exact path. → R-228

F9 — what was and was not proven

The literal precondition (a box that never had off-site backups) needs a rebuild, which the brief forbids before Phase 4, so it was not staged. But the assertion that failed in Phase 1 was tested and passes: Phase 1's defect was that GET /recovery never asked the predicate. After F7 drives recoveryOffer() false, GET /recovery → 302 /backups/remote. R-215's fix is proven live. The GetHubEscrowIdentityPresent()==false arm remains covered by tests only.

F10 — not injected, and why that is a harness result

Three attempts, each with a control confirming the directory was genuinely absent — and each self-healed before capture: (1) the app recreated it; (2) docker stop was undone by the controller's own monitor restarting the stack; (3) with the stack stopped through the controller API, the directory reappeared at 21:51:38.65, coincident with the run's own start. All three runs reported ok with 1 mandatory path(s), and R-203's stat-gap correctly never fired, because by the time os.Stat ran the path existed.

The state F10 describes is not reachable on a deployed app of this kind. Recorded as harness, not product.

One observation kept, with its evidence: at capture the directory held only a recreated metadata.db and not the customer's F10-SENTINEL.txt, and the run still said ok. The verdict is about a path's presence, not its content — correct as designed, and it means an ok off-site run can immediately follow the loss of everything that path contained.

F11 — the dead-man's switch, both ways

23:28:11  ok → stale (host_stale)        + Operator email SENT
23:29:59  Received report from c11        ← the box returns unaided
23:30:11  stale → ok (host_recovered)     + Operator email SENT
23:30:11  customer mail skipped — no unanswered customer down mail (pairing miss)

The customer mail was correctly withheld by the pairing gate under a real outage. I5 holds — box and hub agree after the return, judged after a full report cycle rather than from one read. Not reached: STALE → DOWN (>1 h); the outage was ended once both transitions had fired because Phase 4 needed the venue back.

The timing discriminator used throughout

age's scrypt makes a real unseal cost ~1 s. Phase 1's headline was diagnosed by a 0.134 s response, and the same instrument separates every fault below:

	elapsed	what it means
F1 wrong code	1.194 / 1.004 / 1.014 s	a real unseal was attempted and failed
F3 hub down	0.0556 s	no unseal attempted — it failed before the KDF
F4 agent down	0.0299 s	no unseal attempted
F5 store down	1.198 s	a real unseal, which SUCCEEDED

The customer sees the same sentence for the 1.0 s case and the 0.03 s case.

---

5. Findings

R-224 — every non-code failure on the unlock path is reported as a statement about the code

F3 and F4 are one defect with two faces, and it is Phase 1's headline finding relocated from the version channel to the transport.

	injected (control)	customer sees	machine's own log
F3	hub REJECTed (302 → exit 7)	M4	fetching the sealed bundle: hub: transport error: … no route to host
F4	systemctl stop felhom-agent (:8443 gone)	M4	dial tcp 169.254.253.1:8443: connect: connection refused

In both the correct, current recovery code was entered, so the only possible cause of failure was the injected fault.

The discriminator exists and is discarded at the HTTP boundary. The agent's own err separates the cases exactly —

wrong code : "escrow: the recovery code did not unwrap the identity escrow …"
hub down   : "escrow: fetching the sealed bundle: hub: transport error: … no route to host"

— but both return HTTP 400 under one merged sentence ("the recovery code did not open the sealed bundle, or the bundle could not be fetched"), the agent's own msg= collapses them too, and the controller's failure path has no branch for "could not ask / could not reach" at all. rerr != nil falls straight into the code/package messages.

Why R-216's gate did not catch F4, measured rather than reasoned — from the box's debug ring:

[web] recovery capability gate: offsite_key_recovery=yes (source=version)

source=version. The gate answers from the known agent version without probe traffic, which is correct for the question it was built for (is this agent too old?) and cannot answer the question it is being used for (can this machine ask right now?). A dead agent of the right version sails through it — and the failure that follows is attributed to the code, which R-216's own comment says must never happen: "An attempt that cannot succeed must never be made, because its failure is attributed to the code."

Consequence. During any hub outage or agent restart, a customer holding a perfect recovery code is told it does not open their package and is routed to customer support about older backups. I6 — an unreachable service reported as a fact about the code.

R-225 — the store reports 0 snapshots · 0 GB when it cannot read it, beside a card saying it holds backups

Found while checking F6's "the listing is coherent" clause. /backups/remote renders, on one screen:

Tároló méret · 0 pillanatkép            Tárhelykeret: 0 / 50 GB (0%)

immediately above:

„A távoli tároló másik kulccsal készült mentéseket tartalmaz. … A meglévő mentések nem sérültek…"

Ground truth, measured directly against the Storage Box over SFTP — a read-only listing, no decryption, using the box's own transport credential:

/home/felhom-repo/snapshots:
  f3d9cd67d539c00359f0454ea7a782f45beac8406c55275bee5a6886afa8d791   (Aug 5 13:14)
/home/felhom-repo:   du -s → 12535 KB          /home/felhom-repo/keys:   exactly ONE key

That is the Phase 0 snapshot holding all three sentinels — the customer's only surviving copy — matching the journal's repo_size_bytes 12 611 522.

Mechanism, from the box's own state: after the Phase 3 rebuild the offbox block in settings.json carries no snapshot_count and no repo_size_bytes key at all. The values are unknown, and unknown renders as the zero value.

This is R-217's defect class in a second location — a field whose zero is indistinguishable from a real measurement, defaulted past on an unknown path. OffsiteInventory.Empty exists precisely because "len(Apps)==0 is also what a failed read looks like". I6.

I5 checked and NOT breached: the hub's /offsite shows Campaign 11 · 0.0 GB, but that is 12.5 MB rounded to one decimal of a GB and the pool total (Used 3.8 GB) is consistent. The two views do not disagree; both understate, for different reasons, and only the box's snapshot count — an integer — is false.

R-226 — M1, the only message that tells a customer to check their typing, is unreachable on any box that has re-escrowed

The failure path tests M4's condition before M1's:

if present, at := s.recoverySuperseded(); present { …M4…; return }
…M1…

So on every box the hub keeps an earlier package for, a genuinely mistyped code produces M4 — which is hedged („Ha egy korábbi kódot adtál meg") and states two true facts, but offers no hint to re-check the ten words and routes the customer to support about older backups.

Measured: F1's three wrong-code attempts each returned M4, each after a real ~1 s unseal.

Why it matters rather than being a nicety: the population that has re-escrowed is exactly the population that has just been handed a new recovery code and is most likely to be typing one. R-222's fix removed one conflation (correct-earlier-code read as a mistype) and introduced another (mistype read as a correct-earlier-code) on the same branch.

R-227 — a restart mid-unlock returns a raw English Bad Gateway

F8 restarted the controller at T+0.7 s, inside the unseal window (control: the container's StartedAt moved). The customer got:

HTTP 502 · "Bad Gateway"

A raw upstream error, in English, from traefik. It names no reason, offers no action, and says nothing about whether the key was installed. I3 — "every refusal names a reason a person can act on, in Hungarian, with no raw error." Low severity (the window is ~1 s wide) and recorded rather than inflated.

R-228 — the set-aside history becomes invisible the moment it is set aside

The move-aside is correct and was verified byte-for-byte (§F7). What follows it is not.

"orphaned_renamed_to": "/home/felhom-repo.orphaned-20260805"

The box records the exact path. And:

grep -rn "OrphanedRenamedTo" internal/web/templates/ internal/web/*.go   →   (no hits)

The field is written and read by nobody. /backups/remote after the set-aside contains no occurrence of the path, „félretéve", „régi előzmény" or any equivalent — with instrument controls passing (felhom-repo → 2, „letétbe helyezve" → 1), so the page and the matcher both work.

12.5 MB of the customer's deliberately retained data sits at a path the box knows and never shows. Its only mention is a flash message on the redirect, gone on the next click.

The project's own "seam built but never wired" pattern — the fifth recorded instance — landing on the one promise the set-aside screen makes. The fix is a plain statement that an earlier history is set aside and not deleted; it must not promise the history can be reopened, because R-222 means it cannot be, and that is exactly the conditional promise R-202's gate exists to prevent.

---

6. What behaved correctly

A campaign that reports only what broke is half a campaign. These were tested and held.

R-217's fix	F5: with the store blocked after a successful unlock, the page rendered M3 and no listing block at all. The four false-claim strings („A tároló megnyílt", „van benne tartalom", „nem tudtuk alkalmazásokhoz rendelni") are absent — verified in UTF-8 with two accented positive controls present, after the documented accented-substring false-zero trap was accounted for
No lockout	F1: three wrong codes, three identical responses, no rate limit, no refusal to try again — as §8.4 NO LOCKOUT documents
Nothing written on failure	F1: all four /data/offbox files byte-identical, mtimes frozen at 14:57:35
I4	the entered code appears in no log, file or page. Sweep run with a planted-canary control first; the only hits were the harness's own script
The „Most nem" asymmetry	F6: the full page stops interrupting /launcher and /dashboard, while /recovery stays 200 and the backups-area entry point survives — bound to the offer, never to the postpone flag
No half-state	F8: a restart mid-unlock left the key files and settings.json coherent; the controller returned healthy in 40 s
R-215's fix	the GET /recovery gate is present in v0.201.0 and consults the same predicate as the POST sibling
R-198's retention, in the UI	the hub host page reads „Key Escrow: present · 1 superseded escrow blob(s) retained"
The empty submission	F2: „Add meg a helyreállítási kódot." in 0.027 s — no unseal, and no blame attached to a code never given

---

7. Harness faults, separated from the product's

Four, all mine, none a product defect — recorded because Campaign 10's §4d is the format and because two of them nearly produced false findings.

1. source ~/.config/credentials echoes secrets. The file holds keys with hyphens (R_DEMO-HP=…) that bash cannot assign, and the command not found error prints the value. Two demo-box recovery codes were printed this way before the helper was rewritten to grep the one key it needs. A live I4 hazard for any session that sources that file — worth a memory, not a register row.
2. Quoted values in that file — a bare cut -d= keeps the quotes and yields the wrong secret; the hub returned 302 until they were stripped. Already in project memory; re-confirmed.
3. The recovery page carries no <meta name="csrf-token"> — it renders a hidden _csrf input (data["CSRFField"]). The first F1 run produced three 403s and CSRF-LEN-0; the harness's own length check caught it, and the controller's log named the reason word (token mismatch). Harness, not product.
4. Two failed reachability controls, reported as failures. The F5 probe first used nc, which the controller container does not have, so blocked and unblocked printed the same fallback; the second attempt used the IPv6 address getent hosts prefers, and the container has no IPv6 route at all. Only a /dev/tcp probe against the IPv4 address (91.98.242.176) discriminated TCP-OPEN → TCP-CLOSED. Two readings that looked like results and were instrument failures.

---

8. Suspicions investigated

	verdict
"The floor is still held — there is no HELD line and no held reason"	DISPROVED. Both absences were real, and both are explained: the hub stops logging when it stops holding, and SetFloor's DEBUG line cannot reach the debug ring at all. The floor is served, measured two ways (§3)
"The hub and the box disagree about the store's contents (I5)"	DISPROVED. Both understate; the hub's 0.0 GB is rounding of 12.5 MB. Only the box's snapshot count is false → R-225, which is an I6 finding, not an I5 one
"GET /recovery still renders on a box it should not"	CONFIRMED FIXED — the gate is present in v0.201.0 (§9, F9)

---

(Sections 9–12 — F7, F9, F10, F11, Phase 4, invariants, teardown and hygiene — follow below.)

---

8b. Phase 4 — the unattended soak

23:56 → 04:35, venue untouched. 30 five-minute samples plus a full log census. Expectations were pre-registered in the journal BEFORE the window, in both directions, so nothing here is fitted afterwards.

Everything scheduled fired, exactly once, on time

job	due	result
db-dump	02:30	✅ 5.237 s
tier2-backup	03:30	✅ 118 ms — and the copy is real (below)
fill-watch	03:30	✅
metrics-prune	04:00	✅
offbox-backup	04:15	✅ 24.884 s — snapshot_count 1 → 2, last_status: ok

The off-site tier runs itself, unprompted, on a box that was rebuilt twice and had its repository set aside four hours earlier. That is the strongest positive of the whole campaign for the backup promise, as distinct from the recovery journey.

Nothing fired that should not have

needs_credential 0 · offsiteheal 0 · offbox_repo_orphaned 0 in-window · offsite_repo_key_changed 0 · escrow blob SERVED 0 · host_stale/host_recovered 0 · self-update 0 · operator emails 0.

§4.2's negative control passes. A box that HAS a target did not declare needs_credential once in five hours. R-218's fix is not over-firing in the other direction — which does not substitute for its positive half, still owed.

Investigated and DISPROVED — tier2-backup in 118 ms

It looked like a scheduled backup that silently no-ops. It is not: Tier 2 copied calibre-web → /mnt/felhom-drives/mentes/backups/secondary/calibre-web (818.5 KB, 1 leg(s), 0s), and the copy is on the backup drive. 388 KB on local NVMe in 118 ms is honest. No finding.

A correction to my own pre-registration

I pre-registered a backup_run_digest event. No such event type exists — that is a test filename, and the real one is backup_run_failures, a failures digest whose silence on a clean night is correct. Reporting it as a miss would have been a finding invented by a bad reading.

What survives the correction: the off-site run emitted no hub event at all, while both lesser tiers announced success (db_dump_completed, crossdrive_completed). Failures are covered by backup_run_failures and staleness by the hub's tier deadline monitor (offsiteBackupStaleAfter = 8 days), so this is a consistency wrinkle, not a blind spot — recorded, not filed.

What should have fired and did not — answered in both directions

• The restore-test did not run, and that is CORRECT — eval_interval=6h (armed 23:29:43, first evaluation ~05:29) with a 24 h settle on archives hours old. Pre-registered as "expected not to run"; without that, its silence would have read as a gap.
• The agent's whole-guest tier did not run, and whether it should have is NOT RESOLVABLE. The box reports 0 backups all night; the agent is emphatically alive (2 091 of its own lines since 00:00, polling the guest at 04:17:57); but routine local-api requests are not logged at INFO — a five-hour search for local-api returns 0 on a box that demonstrably served such calls earlier — so "no /backup/due poll" is not evidence. The hub's deadline monitor is equally invisible at INFO. Recorded as an open question, not scored as a pass.

---

9. Invariants at every phase boundary

		verdict
I1	no customer data destroyed without an explicit confirmation naming what is lost	HELD. The only destructive act was F7's set-aside, behind two confirmations that name the consequences first; and it does not destroy — du -s 12535 before and after
I2	a green status never coexists with missing mandatory data	HELD as tested, with a caveat. F10 could not be injected, so the intended test never ran. The caveat is recorded rather than scored: an ok run captured a recreated directory that no longer held the customer's file
I3	every refusal names a reason a person can act on, in Hungarian, with no raw error	BREACHED twice. F8's Bad Gateway (raw, English) → R-227. And R-220's refusal still names an action the customer cannot perform (the list it points at is empty) — reproduced live a third time
I4	no secret anywhere — code, repository password, blob, credential	HELD on the product. The F1 sweep found the entered code in no log, file or page, with a planted-canary control passing first. Breached by the HARNESS, not the product: source ~/.config/credentials echoed two demo-box recovery codes into the session transcript (§7)
I5	the hub's view and the box's never disagree about protection	HELD. Checked at the F11 boundary after a full report cycle. The apparent 0.0 GB disagreement was investigated and DISPROVED (rounding)
I6	an absence is never reported as a fact	BREACHED twice. R-224 (an unreachable hub/agent reported as a fact about the code) and R-225 (an unread store reported as 0 pillanatkép · 0 GB)
I7	nothing reaches a machine outside the venue	HELD. Every fault was applied to VM 321 or its guest. Both demo boxes were read, never written. Nothing was deleted on ep0 or the Storage Box — F7's move-aside is a rename inside the campaign's own sub-account home

---

10. RTO — unchanged, and why this campaign does not move it

Phase 1's number stands: undefined for an unaided customer, because the unaided journey does not complete. The attended figure — 61 minutes, three of whose four blockers needed root on the appliance — is Phase 1's and is not re-measured here. Phase 2's faults are not a re-walk, and nothing in them shortens or lengthens that path.

The only segment that reflects the product working remains Phase 1's last one: 16 seconds to pull 12.8 MB back out of the off-site repository once everything was in place.

Phase 2 does add one measurement to the picture: the off-site tier, once healthy, works unremarkably. Three manual runs completed in 18 s, 59 s and 1 m 4 s, and the scheduled run is reported in §11.

---

11. Teardown — OWED, nothing removed

Deliberately not torn down. Phase 4 needed the venue, this document had to be written first, and deleting evidence unattended is worse than leaving a VM running. All three layers are owed, plus the off-site side.

layer	what it will need
1. The machine	VM 321 c11-appliance on demo-hp and its three disks (scsi0 200 G, scsi1 50 G, scsi2 50 G) on c11-scratch. qm stop 321 && qm destroy 321 --purge
2. The host	pvesm status on demo-hp before and after, and the space returned recorded. The c11-scratch storage definition itself, once empty
3. The hub	customer c11 and host c11-36d660: the customer Danger-zone Delete is the one true purge point (it cascades both escrow tables). ⚠ It destroys the retained escrow custody — which is the point, but say so before pressing it

The off-site side — and this one needs the most care:

• The campaign's Storage Box sub-account 284166 / u629488-sub4, holding two repositories: /home/felhom-repo (the fresh one, ~26 KB) and /home/felhom-repo.orphaned-20260805 (12 535 KB — the Phase 0 history with all three sentinels).
• On ep0: namespace c11, token felhom@pbs!c11, and the two ACL lines on /datastore/felhom-offsite/c11. demo-felhom and demo-hp namespaces and tokens must not be touched — the Phase 0 capture recorded all three so teardown can tell them apart.
• The WireGuard peer for c11-36d660 (10.77.0.5/32).

Delete nothing that is not the campaign's own. The brief's §8 warns that scratch customers have accumulated before; the Phase 0 census found none outstanding, and c11 must not become the next.

---

12. Hygiene

• The recovery codes are SHREDDED, with the plant→find→shred→fail-to-find control the brief asks for. The control paid for itself on its first run: it found the Phase 0 code in ~/.config/credentials as R_CAMPAIGN_11 — a copy this session did not create and would never have looked for, which would have made a "codes shredded" claim false. That key was removed from the shared file carefully (backup → exact-match removal → diff proving every other line identical → HUB_PW re-verified at hub:200 → backup shredded). Everything else was shred -u'd on both hosts and the absence re-swept; only the planted controls remained, and they were shredded too. ⚠ Consequence, stated plainly: /home/felhom-repo.orphaned-20260805 (12 535 KB, the three Phase 0 sentinels) is now permanently unopenable. That is what the set-aside screen promises will happen, teardown removes the repository anyway, and R-222 means no read path existed for it regardless — but the door is now shut for good.
• No secret is written into any committed file. Every hash quoted here is a sha256 prefix; the codes' contents appear nowhere.
• git add -A was never used — every commit staged explicit paths, and git status --porcelain was checked before each one to confirm no foreign file was swept (a parallel session shares this clone).
• The pre-push hook is armed on this clone (core.hooksPath=.githooks) and ran repo_gates.py --fast green before every push. No --no-verify was used.
• No product code was changed. Every finding was filed and the run continued, per the brief's rule 1.
• No version was bumped in any repo.

---

13. What did not run, and why

	why
F10 as specified	Not injectable. Three attempts, each with a control: the app, then the controller's monitor, then the run itself recreate the mandatory directory within ~1 s. The state does not exist on a deployed app of this kind. Harness, not product
F9's literal precondition	A box that never had off-site backups needs a rebuild, which the brief forbids before Phase 4. The assertion that failed in Phase 1 (the page not consulting its predicate) was tested instead, and passes
§4.2's positive half	Needs shape (a) — hub blob present, key placed, no target. The venue has a target, so the box correctly does not declare. Also needs a rebuild
STALE → DOWN escalation	F11 was ended once stale and recovered had both fired, because Phase 4 needed the venue back. The >1 h arm is untested
The whole-guest tier's due-ness	Not resolvable with existing instruments — routine local-api calls and the hub's deadline monitor are both invisible at INFO. Recorded as an open question, not scored
The retained package's read path	Does not exist (R-199's inventory is unbuilt). R-222 was re-confirmed live rather than re-tested
A re-walk of the journey	Deliberately out of scope. These faults are not a re-walk, and the capability map's recovery row stays FAIL until one passes
Any fix	Brief rule 1 — file and continue. No product code was changed and no version bumped

---

14. Venue state at the end — WORKING

Hub	c11-36d660 ONLINE, agent 0.125.0, 1/1 guests, reporting on schedule (last seen 04:29:54)
Containers	calibre-web · filebrowser · felhom-controller:0.201.0 · traefik — all healthy
Drives	both enrolled; backup target {"degraded":false,"label":"mentes","target":"felhom-backup"}
Off-site	2 snapshots, last_status: ok, last_success 2026-08-06T02:15:24Z, 51 206 B, escrow_state: escrowed
Set aside	/home/felhom-repo.orphaned-20260805 — 12 535 KB, intact, and now permanently unopenable (§12)
Left in place	the raw /mnt/adatok and /mnt/mentes mounts remain unmounted — R-220's workaround, without which no app can be deployed on a rebuilt box. The stable /mnt/felhom-drives/* mounts are what everything uses
Access	the appliance's vaulted root credential was shredded with the codes, so a future session must re-fetch it from the hub (POST /hosts/c11-36d660/reveal-recovery-credential) — the designed path

Nothing is left broken, and nothing is left running that should not be.