CAMPAIGN-11 Phase 2 complete: F7/F9/F10/F11, and R-228 (the set-aside history is invisible)
gates / gates (push) Successful in 8s

F11 PASS  ok->stale->ok, an operator mail each way, customer mail correctly
          pairing-gated. DOWN (>1h) not reached — stated, not implied.
F7  the set-aside PASSES decisively: move-aside to
          /home/felhom-repo.orphaned-20260805, mtime and du -s (12535 KB) and
          snapshot f3d9cd67 all unchanged; a fresh empty repo beside it.
          NOTHING DELETED.
    the afterwards FAILS -> R-228: settings.json records
          "orphaned_renamed_to" and OrphanedRenamedTo has ZERO references in
          any template or handler. 12.5 MB of retained customer data at a path
          the box knows and never shows. Seam built, never wired.
F9  R-215's fix PROVEN LIVE on a narrower shape: with recoveryOffer() false,
          GET /recovery now 302s instead of rendering its story. The literal
          never-had-offsite precondition was not staged (it needs a rebuild,
          which the brief forbids before Phase 4) — recorded as partial.
F10 NOT INJECTED, harness not product. Three attempts, each with a control:
          the app, then the controller's monitor, then the run itself recreate
          the mandatory directory within ~1 s. Kept as an observation: at
          capture the directory held only a recreated metadata.db and NOT the
          customer's file, and the run still said ok.

Also reproduced live for a third time: R-220 (candidates empty; unmounting only
the raw mounts flips both drives back) — still OPEN.

Venue left WORKING: 4 containers healthy, backup target not degraded, off-site
fresh repo last_status ok, escrow escrowed.
No product code changed.
This commit is contained in:
2026-08-05 23:55:52 +02:00
parent 259f48aedc
commit f88faad50b
@@ -457,3 +457,187 @@ that is 12.5 MB rounded to one decimal of a GB, and the pool total (`Used 3.8 GB
hub and the box do not disagree; **both** understate, for different reasons. Only the box's
**snapshot count** — an integer, not a rounding — is false.
### F11 — the box offline for a whole reporting window — **PASS**
Injected: `qm shutdown 321` at **23:07:45**, restored **23:29:12** (≈21½ min). Control: `qm status`
`stopped`, and the last report before it was 23:06:12.
**Both directions fired, and each sent an operator mail:**
```
23:28:11 Host staleness: c11-36d660 ok → stale (host_stale) + Operator email SENT
23:29:59 Received report from c11 (2514 bytes) ← the box returns, unaided
23:30:11 Host staleness: c11-36d660 stale → ok (host_recovered) + Operator email SENT
23:30:11 Recovery host_recovered for c11: customer mail skipped — no unanswered customer down
mail (pairing miss)
```
- **The dead-man's switch fires and clears itself.** `ONLINE → STALE → ONLINE`, hub-side, with no
operator action.
- **The customer mail was correctly withheld** — pairing-gated, exactly as the power-outage audit
records. The gate held under a real outage.
- **I5 holds**: after the return the box's `offbox` block and the hub's view agree (nothing changed
across the outage, and both say so). Judged **after** a full report cycle rather than from a single
read — the Phase 0 lesson.
**What was NOT reached, stated rather than implied:** the box never went to `DOWN` (>1 h) — the
outage was ended once STALE and the recovery mail had both fired, because Phase 4 needed the venue
back. The `STALE → DOWN` escalation is untested.
### F7 — „I do not want the old data", then a change of mind — **the set-aside PASSES, the afterwards does NOT**
**Pre-state, measured on the Storage Box** (read-only SFTP listing, no decryption):
```
/home felhom-repo (mtime Aug 5 13:13) ← exactly one repo
/felhom-repo/snapshots f3d9cd67d539c003…afa8d791 ← the Phase 0 history, 3 sentinels
/felhom-repo du -s → 12535 KB
```
**Confirmation 1** (`/recovery?setaside=1`) states all four consequences before anything happens:
> „a korábbi mentéseket **félretesszük — nem töröljük**; a helyreállítási kód nélkül **többé nem
> lesznek megnyithatók**; a gép **új, üres mentési tárolót kezd**; **ez az oldal többé nem jelenik
> meg**."
**Confirmation 2** (`POST /backup/offbox/reset` `confirm=1`) → **302**, flash
*„Új távoli mentés indítása folyamatban — a régi előzmény félretéve (nem törölve)."*
```
[WARN] [offbox] resetting orphaned repo (operator-confirmed (claimed)):
move-aside /home/felhom-repo -> /home/felhom-repo.orphaned-20260805, then re-init
```
**NOTHING WAS DELETED — the assertion, verified against the remote:**
```
/home felhom-repo (mtime 21:32) ← fresh, re-inited
felhom-repo.orphaned-20260805 (mtime 13:13) ← UNCHANGED mtime
/felhom-repo.orphaned-20260805/snapshots f3d9cd67… ← the Phase 0 history, still there
/felhom-repo.orphaned-20260805 du -s → 12535 KB ← byte-for-byte the pre-state
/felhom-repo/snapshots (empty)
```
**Set aside, never deleted. PASS**, and the two confirmations are honest about what they will do.
#### The change of mind — and this is where it fails
The customer returns wanting the old data. What they find:
```
GET /recovery -> 302 /backups/remote ← the screen is GONE
GET /launcher -> 200 ← no interruption
POST /recovery/unlock -> 302, 0.028 s ← the handler's gate; NO unseal, NO message
```
The brief's expected answer is *"message 4, not a typing accusation"*. **There is no message at all**
which is not a typing accusation, but is also not an explanation.
**And the set-aside history is invisible.** The box knows exactly where it is:
```json
"orphaned_renamed_to": "/home/felhom-repo.orphaned-20260805"
```
`/backups/remote` after the set-aside contains **zero** occurrences of `orphaned-20260805`,
„félretéve", „régi előzmény" or any equivalent (instrument controls: `felhom-repo` → 2,
„letétbe helyezve" → 1, so the page and the matcher both work). And a census of the whole web layer:
```
grep -rn "OrphanedRenamedTo" internal/web/templates/ internal/web/*.go → (no hits)
```
> **The field is written and read by nobody.** 12.5 MB of the customer's retained data sits on the
> Storage Box at a path the box records and never shows. The only mention it ever gets is a flash
> message on the redirect, which is gone on the next click.
This is the project's own recurring **"seam built but never wired"** pattern, and it lands on the one
promise the set-aside screen makes — *we set it aside, we do not delete it*. → **R-228**
*(Correct and recorded as a pass: the recovery screen retiring itself is exactly what confirmation 1
promised — „ez az oldal többé nem jelenik meg" — so its disappearance is declared, not sprung.)*
### F9 — a box that never had off-site backups — **the gate PROVEN LIVE, on a narrower shape**
F9 as literally specified needs a box that never had off-site backups. **The venue cannot be that box
without being rebuilt, which the brief forbids before Phase 4** — so the literal precondition was not
staged.
**But the assertion that FAILED in Phase 1 was tested and it now passes.** Phase 1's failure was not
about the precondition; it was that `GET /recovery` **never asked the predicate** and rendered its
story regardless (R-215). F7's set-aside drives `recoveryOffer()` false (a repository password exists
and the tier is no longer orphaned), and:
```
GET /recovery -> 302 /backups/remote ← the page consults the predicate
POST /recovery/unlock -> 302, no unseal ← the POST sibling was already gated
```
**R-215's fix is proven live.** What is NOT covered: the `GetHubEscrowIdentityPresent()==false` arm
(a box the hub holds nothing for). Recorded as partial, not claimed as full.
### F10 — an app with a mandatory data path missing — **COULD NOT BE INJECTED. Three attempts, each with its control.**
Setup, through the customer's own endpoints:
- **R-220 reproduced live, a third time** — `GET /api/disks/candidates` returned
`initialize: [], attach: []` while both drives sat mounted at **both** `/mnt/felhom-drives/<name>`
and the raw `/mnt/<name>`. **Its red-proof reproduced too**: unmounting only the raw mounts flipped
the scan to both drives with `fstype: ext4, data_bearing: true`. *(Deviation: an operator action on
the appliance host, unavoidable while R-220 is open, and it doubles as the red-proof.)*
- Both drives re-enrolled via `POST /api/storage/attach``{"registered":true,"where":"/mnt/felhom-drives/adatok"}`.
- `calibre-web` deployed via `POST /api/stacks/calibre-web/deploy` (**202**, the same 409→202 flip
Phase 1 recorded), healthy in 39 s, bound at `/mnt/felhom-drives/adatok/userdata/media/books`
its declared-mandatory path.
- Baseline off-site run: **`last_status: ok`, 1 snapshot**, log `backed up calibre-web (…, 1 mandatory
path(s))`.
> **A clean confirmation of R-225's mechanism, taken here:** after this first *successful* run
> `settings.json` gained `"snapshot_count": 1` and `"repo_size_bytes": 27371`. Those keys were
> **absent** before it — which is exactly why the page rendered `0 pillanatkép · 0 GB`.
**The three injection attempts:**
| # | method | control | why it failed |
|---|---|---|---|
| 1 | `mv` the directory aside, app running | ABSENT confirmed | the **app** recreated it (mtime 21:45) before capture |
| 2 | `docker stop calibre-web`, then `rm -rf` | ABSENT confirmed, held 25 s | the **controller's monitor auto-restarted the stack**; the app recreated it |
| 3 | stop the stack **through the controller API**, then `rm -rf` | ABSENT confirmed, held 25 s | the directory reappeared at **21:51:38.65** — coincident with the run's own start (21:51:38) — and the stack was found `Up` again a minute later |
All three runs reported **`ok`** with `1 mandatory path(s)`, and **the R-203 stat-gap WARN
(`mandatory data path missing on disk, skipped from offsite`) never fired** — correctly, because by
the time `os.Stat` ran the path existed again.
**Verdict: harness, not product. F10 is recorded as NOT INJECTED**, because the state it describes is
not reachable on a *deployed* app of this kind: the app (or the monitor that keeps it up) recreates
its own data directory within a second, every time.
**But one observation is worth keeping, with its evidence.** At capture time the directory held:
```
metadata.db metadata.db-shm metadata.db-wal ← recreated, 21:51
(no F10-SENTINEL.txt — the customer's file, written at 21:39)
```
**The run reported `ok`.** The verdict is about the *presence* of a declared path, not its *content* —
which is correct as designed and could hardly be otherwise — but it means **an `ok` off-site run can
immediately follow the loss of everything that path contained.** Not filed as a defect; recorded
because "the run said ok" is exactly the reassurance R-203 was filed to make trustworthy, and this is
its edge.
*(The customer's sentinel was restored into the live library afterwards and the harness copy removed.)*
---
## Venue state at the end of Phase 2 — **WORKING**
| | |
|---|---|
| Containers | `calibre-web` healthy · `filebrowser` healthy · `felhom-controller:0.201.0` healthy · `traefik` up |
| Drives | both enrolled; backup target `{"degraded":false,"known":true,"label":"mentes","target":"felhom-backup"}` — the I5/I6 baseline restored |
| Off-site | fresh repo, `last_status: ok`, `last_success 2026-08-05T21:52:43Z`, 1 snapshot, 26.4 KB, `escrow_state: escrowed` |
| Set aside | `/home/felhom-repo.orphaned-20260805` — the Phase 0 history, 12 535 KB, untouched |
| Recovery screen | retired (correct — the set-aside declared it would be) |
| Deviations left in place | the raw `/mnt/adatok` and `/mnt/mentes` mounts remain **unmounted** (R-220's workaround); the stable `/mnt/felhom-drives/*` mounts are what everything uses |