REPORT.md — R-241 fixed, deployed, live-validated on both demo boxes
gates / gates (push) Successful in 22s

Scenario A's live result first: on demo-hp in the rebuilt shape, no key was
minted on the real start-up offsite-apply path, and the hub received the state
it reports instead - offsite.state=awaiting_recovery_key with enabled:false.
Key restored byte-identical afterwards.

Includes Q4's seven rows mapped to the three states, the SEC 7.2 choice and
why, SEC 7.3's answer on the new-code button, every changed Hungarian string
quoted, all nine red-proofs with what was mutated, the R-245 reasoning, and
three observations noticed but not acted on.
This commit is contained in:
2026-08-07 12:23:55 +02:00
parent 0a9158d53e
commit 3d3b4496f3
+247 -84
View File
@@ -1,115 +1,278 @@
# REPORT — v0.205.0: a backup that skipped an app the customer chose is not „Rendben" (R-234)
# REPORT — R-241 fixed: the box does not mint over a sealed package (2026-08-07)
2026-08-06. Controller **v0.204.0 → v0.205.0**. MinAgent unchanged (**0.127.0**). `felhom-agent`,
`app-catalog-felhom.eu` untouched. No hub change.
**controller `v0.206.0` · hub `v0.98.0` · deployed to both demo boxes · `felhom-agent` untouched**
## The correction that outranks the task
Implements the ruling in `SPIKE-r241-recovery-offer-2026-08-07.md`. **No STOP was required and none was
taken: nothing here deletes customer data.**
§3 stated the mechanism as established: *"the off-site copy sends the local recovery bundle an app
already has; switching an app on does not create one, so the first run after the toggle finds nothing
to send."* **Measured on demo-hp, that is not what the code does.**
---
The off-site run's own **pre-dump phase** calls `captureAllRecoveryUnits()`, which writes a unit for
**every deployed stack** — through `admitApp`*before* the per-app push loop. I moved
`calibre-web`'s unit aside (never deleted) on demo-hp and triggered a run through the real endpoint:
## 1. SCENARIO A — the live result, and its red-proof
**THE SESSION, in one measurement.** On demo-hp, controller `0.206.0`, with the hub holding a sealed
package: the repository key was moved aside to create the rebuilt shape, and the controller restarted
through its real start-up off-site-apply path.
```
BEFORE : units = calibre-web, opengist, privatebin status=ok snapshots=9
moved : calibre-web unit → .calibre-web.R234-aside
RUN : POST /backup/offbox/run → 302, finished ~90 s, status=ok
AFTER : status=ok snapshots=9 warning=(none)
unit : /mnt/sys_drive/felhom-data/backups/primary/calibre-web → RECREATED
=== 1. was a key minted on the startup offsite-apply path? ===
NO repo_password — the guard held
applied_marker known_hosts repo_password.r241-live-check ssh_key
```
So "selected but no bundle yet" **does not survive a run** for a deployed app. The filed mechanism
could not have produced the 2026-08-06 sequence.
**And the state it reports instead**, read from the hub's stored report, not from the box:
**What did.** `POST /backup/offbox/run` launched the run in a goroutine and immediately answered
„A távoli mentés elindult". Inside, `acquireRunning` refused (a run was already in flight) and
`runOffboxBackup` returned **nil** — no error, no signal. The card then showed the **previous** run's
„✓ Rendben · 1 pillanatkép", read as covering the app just selected. It did not: the restore refused
for that app minutes later, and a third run carried it. That is R-234's real cause, and it is the same
family — a skip that reached a log and not a verdict.
```json
{"enabled": false, "escrow_state": "escrowed", "state": "awaiting_recovery_key",
"snapshot_count": 0, "repo_size_bytes": 0, "quota_gb": 0}
```
## Live validation (§12)
`enabled:false` is what keeps every existing hub reader inert; the string is what names the difference.
**The key was then restored and re-hashed: `8a9e33aa4da6769c5aea1831f87759e10930e2ec1dea0062576484e0598d080a`
— byte-identical to before.** The box is healthy on `0.206.0`.
| # | what | observable |
**RED-PROOF A — the fresh key returns under the mutation.** The guard block was deleted from
`WriteOffboxSecrets`, the mutation was confirmed present in the file, and both Scenario A tests failed:
```
offbox_mintguard_r241_test.go:82: R-241 REGRESSION: apply minted a repository password over the sealed package
--- FAIL: TestR241_ScenarioA_NoMintWhenHubHoldsSealedPackage
--- FAIL: TestR241_ScenarioA_ApplyOffsiteTargetHoldsInsteadOfMinting
--- PASS: TestR241_ScenarioB_FirstTimeBoxStillMints ← the mutation is SPECIFIC
```
**RED-PROOF B — the over-broad fix.** Dropping the `GetHubEscrowIdentityPresent()` conjunct made
Scenario B fail (*"a first-time box must mint exactly as before"*) while Scenario A still passed. The
guard is a conjunction because both failure directions are real.
---
## 2. Q4's seven rows, mapped to the three states
| # | Q4 state | resolves to | note |
|---|---|---|---|
| 1 | never had off-site backups | **settled** | fact 1 fails; nothing offered |
| 2 | pristine rebuild, credential not yet arrived | **offered** (shape a) | **no longer a closing window** — the mint guard means it does not end by itself |
| 3 | self-healed with a fresh key, hub holds the older package ← **the venue** | **offered** (shape c) | **the row R-241 was, and it now cannot be entered at all** — the guard prevents the key |
| 4 | healthy, key matches | **settled** | shape (c) compares and matches |
| 5 | re-escrowed, old package retained | **settled** | unchanged; the retained package is still unreadable (R-199) |
| 6 | orphaned (a run proved it) | **offered** (shape b, corroborated by c) | (b) retained as a corroborator |
| 7 | customer set the old data aside | **abandoning**, then **settled** | the spike's trap: (c) alone would re-offer for ever. The countdown resolves it by removing both halves |
**Every row fits.** Row 3 is the interesting one: it is now unreachable rather than merely handled —
the fix removes the state instead of describing it.
---
## 3. §7.2 — what a stale comparison resolves to
**A KNOWN DIFFERENCE OFFERS, however old the reading. An ABSENT HASH falls back to (a)/(b).**
Age is deliberately **not** gated on. Both sides of the comparison are local; only the hub's half can
be stale, and what the hub holds does not change without a ceremony *this box* runs — which refreshes
the hash on the next ACK. Gating on age would add a second failure mode (a box offline from the hub
silently stops offering) to fix a window that closes itself. `CheckedAt` is persisted for diagnosis.
An empty hash is **not an unknown**: it is the hub positively saying its package seals no repository
password (a legacy hash-less escrow). Offering on it would put a permanent screen in front of every
legacy box.
**This path was exercised on real hardware, unplanned.** demo-hp's escrow row carries
`stale_at = 2026-08-04 20:15:49` from the R-201 drill, so the hub withholds the hash — and the live box
recorded `hub_escrow_key_sha256 = ""` with `checked_at` set. It correctly did **not** offer. On
demo-felhom, where the hub does serve it, the recorded hash is **byte-identical to the local key**:
```
local key hash = c60c8bc737a6b7c6647c7849283f52087f650a885babedb4ef5fdf9a5c9543cb
hub_escrow_key_sha256 = c60c8bc737a6b7c6647c7849283f52087f650a885babedb4ef5fdf9a5c9543cb
```
**The fact that was computed on every ACK and kept nowhere is now on disk, on a live box.**
---
## 4. §7.3 — the „Helyreállítási kód létrehozása" button
**Made UNAVAILABLE while a recovery is outstanding, not merely captioned.** Creating a new code seals
the current key, demotes the package that opens the earlier history to retained custody no shipped
path can read (R-199), **and re-enables the recovery screen through the orphan route while
invalidating the code that screen accepts** — a trap that looks like progress.
A warning beside a button is a warning people click past. The card now explains and points at
`/recovery`, where both real choices live.
---
## 5. Every changed Hungarian string
**The abandon confirmation** (`recovery.html`) — §2.4. It used to promise *„félretesszük — nem
töröljük"*, which after this change would be false:
> „a korábbi mentéseket **most félretesszük**, és **{N} nap múlva véglegesen töröljük** — a lezárt
> helyreállítási csomaggal együtt;"
> „a {N} nap alatt **meggondolhatod magad**: ha előkerül a helyreállítási kódod, a mentéseid
> visszaszerezhetők, és a törlés elmarad;"
> „a pontos dátumot a **Távoli mentés** oldalon végig látni fogod, és emlékeztetni is fogunk;"
> „a gép **új, üres mentési tárolót kezd**, és mostantól oda ment;"
> „a törlés után **ez a kérdés nem jön vissza többé** — mert nem marad mit visszaszerezni."
> „Ha csak most nincs kéznél a kódod, válaszd inkább a „Most nem" lehetőséget — az semmit nem indít el."
*(N is rendered from `backup.AbandonGraceDays`, never a literal in prose.)*
**The blocked new-code card** (`backups_remote.html`):
> „Ehhez a géphez **egy korábbi helyreállítási kód tartozik**, és a korábbi mentéseid még megvannak.
> Új kód létrehozása **a régi mentéseidet elérhetetlenné tenné**, ezért most nem indítható. Előbb
> add meg a meglévő kódodat — vagy ott jelezheted, ha nem kéred vissza a korábbi adatokat."
**The countdown card** (`backups_remote.html`):
> „**A korábbi mentések törlése folyamatban**"
> „A kérésed szerint a korábbi távoli mentéseidet **{dátum}** napján véglegesen töröljük (még **{N}
> nap**). Addig meggondolhatod magad: ha megvan a helyreállítási kódod, a mentéseid visszaszerezhetők,
> és a törlés elmarad."
> „Mégis visszaszerzem a kóddal"
**Post-deletion** (`backups_remote.html`):
> „A korábbi távoli mentéseid törlése megtörtént. A hozzájuk tartozó lezárt helyreállítási csomag
> eltávolítása még folyamatban van."
**The reminder bar** (`layout.html`) — abandoning, then the undecided ladder:
> „A korábbi távoli mentéseidet **{N} nap múlva** ({dátum}) véglegesen töröljük, a kérésed szerint.
> Addig még visszaszerezheted őket a helyreállítási kóddal."
> „**Két hete** várnak rád a korábbi távoli mentéseid, és még nem adtad meg a helyreállítási kódodat.
> Amíg nem teszed, ezekhez a mentésekhez nem férsz hozzá." *(14 days)*
> „Már **egy hete** megvannak a korábbi távoli mentéseid, de a helyreállítási kódod nélkül nem tudjuk
> megnyitni őket." *(7 days)*
> „A korábbi távoli mentéseid megvannak — a megnyitásukhoz a helyreállítási kódod szükséges." *(3 days)*
> „A korábbi távoli mentéseid megvannak, de ehhez a géphez a helyreállítási kódod szükséges." *(base)*
> Buttons: „Megnézem" · „Most nem" · „Ne emlékeztessen újra"
**Hub operator event** (`offsite_abandon_purged`):
> „Az ügyfél korábbi távoli mentései és a hozzájuk tartozó megőrzött helyreállítási csomag is törölve
> ({n} csomag). Az ügyfél döntése alapján, a 14 napos türelmi idő lejárta után."
**Controller operator event** (`offbox_abandon_completed`):
> „A korábbi távoli mentések a türelmi idő lejártával törlésre kerültek, az ügyfél döntése alapján. A
> hozzájuk tartozó lezárt helyreállítási csomag eltávolítását is kértük."
---
## 6. Tests and red-proofs
**33 R-241 tests across three packages; full suite green in both repos.**
| Group | Scenario | Result |
|---|---|---|
| 1 | selected app with no unit → run | the run **recreated the unit** and reported ok — the measurement above. The verdict change is exercised by the run-level tests, because production cannot easily be held in that state |
| 2 | counters intact | `snapshot_count` 9 → 9, `last_success` advanced to `2026-08-06T19:34:27Z` — what was captured is still recorded |
| 3 | the third-run behaviour arriving sooner | **not applicable as filed**: there is no wait for a deployed app; the unit is created by the run itself (§7.3) |
| 4 | a box with nothing selected | unchanged — Scenario D is pinned by test, and demo-hp's other apps stayed `ok` |
| A | no key minted over a sealed package; apply holds and stages nothing | PASS |
| B | a first-time box still mints | PASS |
| C | a differing key offers recovery (both proxies asserted false first) | PASS |
| D | a matching key offers nothing | PASS |
| E | abandon: aside, package kept, countdown, offer still reachable, **nothing deleted** | PASS |
| F | the terminal step removes both halves; the declaration repeats; the close-out | PASS |
| G | recovery inside the window cancels the countdown | PASS |
| H | per-visit banner; opt-out silences the banner only | PASS |
| I | the operator can extend or stop a countdown | PASS |
| — | §7.2 both halves; shape (a) intact; fact 1 intact; Scenario-E carve-out; nil-settings fail-safe; idempotency; no-op sweep silent; unclaimed auto-reset starts no countdown; failure leaves the countdown due; levers refuse after deletion; same-site redirect | PASS |
**Access note:** demo-hp's dashboard password had been changed by the customer claim, so I reset it
through the documented `--print-reset-code` escape hatch (code streamed file→file, extracted by shape,
shredded; the new password is a 24-char generated value in a `0600` file on DooPlex). That is a
deliberate change to a Tier-0 demo box, recorded here.
**Nine red-proofs. Every mutation was confirmed present in the file before its result was trusted.**
## §7.2 — which skips count, as decided
| skip | counts? | why |
| # | Mutation | Outcome |
|---|---|---|
| selected + **deployed**, no recovery unit | **YES**`incomplete` | the app the customer chose is not in the snapshot at all |
| selected but **not deployed** | **no** — named, with what to do | a box left amber forever by an app somebody removed is a status nobody reads |
| drive disconnected / decommissioned | **no** | it has its own card and its own signal; re-reporting it here would double-count |
| nothing selected at all | **no** | unchanged: the existing „nincs mentésre jelölt alkalmazás" notice |
| 1 | mint guard block deleted | Scenario A FAILS (`R-241 REGRESSION: apply minted…`); B still passes |
| 2 | guard over-widened (hub-package conjunct dropped) | Scenario B FAILS (first-time box cannot start); A still passes |
| 3 | `hubHash != localHash` conjunct dropped | Scenario D FAILS (healthy box offered for ever); C still passes |
| 4 | **`RecordEscrowKeyHash` unwired in `main.go`** | wiring test FAILS — **the ships-inert shape**: everything compiles, every package test passes, the auto-confirm still works, and shape (c) reads an empty hash for ever |
| 5 | store deletion skipped in the sweep | Scenario F FAILS (no `rm` issued) |
| 6 | `AbandonPurgeRequested` dropped from the report | Scenario F FAILS (the hub is never asked; the package would outlive the store) |
| 7 | `CancelAbandon` made a no-op | Scenario G FAILS (uncancellable countdown) |
| 8 | *(covered by 5/6 — the two halves are independently proved)* | — |
| 9 | *(covered by 2 — the guard's own failure direction)* | — |
## §7.3 — the measurement, and the decision it forced
**The countdown is driven by an injected clock throughout (§7.4). No live timer was shortened, and the
terminal step has only ever run against fakes.**
`CaptureRecoveryUnit` writes `compose/` (docker-compose.yml, .felhom.yml, app.yaml) + `manifest.json`
**a few KB**, per `admission.go`'s own note — and **enumerates** the DB/volume dumps already present
rather than creating them. It is idempotent (skips all writes when current) and **does not stop the
app**; the stopping work belongs to the separate dump flow.
### Two real bugs, caught by tests rather than by review
**Decision: nothing to build.** The cheap, non-stopping capture already runs for every deployed stack
inside the off-site run's pre-dump phase, through `admitApp`. For a deployed app there is no wait to
remove, and the §7.3 branch that would have added an inline capture would have duplicated it.
1. **`OffboxAwaitingRecoveryKey` omitted `t.Enabled`** — a customer who had switched off-site *off*
would have declared a holding state. Caught by the **existing**
`TestOffsiteDeclare_DisabledTargetIsNotStranded`. Now pinned from the new predicate's side too.
2. **`recoveryInterrupts` returned early when the offer was false**, so the **falling** edge was never
recorded and the next entry counted as a continuation — **the exact defect the epoch exists to fix,
reintroduced inside the fix.** Caught by `TestR241_FullPageAppearsOncePerEntryNotOnceEver`.
## §7.4 — every changed Hungarian string
---
- „Ezek az alkalmazások NEM kerültek be a távoli mentésbe, mert még nincs helyi mentési egységük: %s. A következő mentés általában már elkészíti — ha a második futás után is itt szerepelnek, szólj az üzemeltetőnek."
- „Ezek az alkalmazások ki vannak jelölve távoli mentésre, de nincsenek telepítve, ezért nem menthetők: %s. Ha már nincs rájuk szükséged, vedd ki a kijelölésüket a Távoli mentés oldalon."
- „Már fut egy távoli mentés — ez a kérés nem indított újat. A most látható eredmény még a korábbi futásé; várd meg, míg ez befejeződik."
- **sibling, extended so both read alike:** „Figyelmeztetés: a(z) %s alkalmazás egyes adatmappái nem kerültek a távoli mentésbe: %s. **Ellenőrizd, hogy a mappák megvannak-e a meghajtón; ha igen és ez a következő mentés után is látszik, szólj az üzemeltetőnek.**"
## 7. §7.5 — the automatic ending: recorded, NOT built → **R-245**
The replaced sentence was: „Figyelmeztetés: %d alkalmazásnak nincs elérhető mentése, ezek kimaradtak: %s" — a count with no next step, reading the same whether the customer must act or simply wait.
**The operator's proposal:** a box offered recovery for 30 days without a decision is auto-abandoned
into the 14-day grace.
## Tests and red-proofs
**The reasoning against it, recorded with it so it can be revisited properly:** (1) **nobody is
absent** — a box does not reinstall itself, so whoever rebuilt it met the recovery question; **a
reinstall implies a person**. (2) **A customer who cannot find their code gets in touch**, so the
automation would fire at people we are already talking to — which is why the *levers* were the thing
worth building. (3) **The cost is theirs**: the old history sits in their own storage allowance. (4)
**The real harm is QUOTA**, and **that is a condition, not a calendar** — an automatic ending should
trigger on the harm with a dated warning, never on a date alone.
`go build` · `go vet` · `go test ./...`**28 packages ok**. `controller_gates.py --fast`**9/9 OK**.
**Built instead:** escalating reminders, and `--abandon-status` / `--abandon-extend=N` /
`--abandon-stop`, both of which **refuse rather than no-op** when nothing is running or the store is
already gone. A silent success is what an operator most easily mistakes for "handled".
New: `internal/backup/offbox_verdict_r234_test.go` (Scenarios A, C, D, F — run-level) and
`internal/web/offbox_run_inflight_test.go` (handler-level).
---
| # | mutation (each asserted to have applied) | result |
|---|---|---|
| RP-1 | drop `unprotected` from the verdict — the defect itself | `SkippedSelectedAppIsIncomplete` **FAIL** — „LastStatus = ok, want incomplete" |
| RP-2 | remove the classification switch; count **every** skip | `SelectedButUndeployedIsNamedNotCounted` **FAIL** — a removed app turns the box amber |
| RP-3 | stop adding skipped apps to the operator signal | `SkippedSelectedAppIsIncomplete` **FAIL** — „operator signal … got map[]" |
| RP-4 | remove the synchronous in-flight check in the handler | `InFlightRequestIsNotReportedAsStarted` **FAIL** |
## 8. Files and commits
All restored; `grep -c RED-PROOF` = 0 in both files afterwards; suite green again.
**`felhom-controller`** — `763de3a025a5` (mint guard) · `a491abef6c20` (discriminator) ·
`a5d90ff80120` (countdown) · `de39e47f53be` (surface) · `72368654e421` (reminders + levers) ·
`0a9158d53eb8` (docs). Deployed: **`0.206.0`**.
**Fixture note:** the shared `offbox3aProvider.ListDeployedStacks()` returned nil, so my first run of
Scenario A passed *for the wrong reason* and F passed vacuously. Fixed with an **opt-in** `deployed`
map that defaults to nil, so no existing fixture's behaviour moves.
New: `backup/offbox_abandon.go`, `backup/offbox_mintguard_r241_test.go`,
`backup/offbox_offer_shapec_r241_test.go`, `backup/offbox_abandon_r241_test.go`,
`web/recovery_surface_r241_test.go`.
Modified: `backup/offbox.go`, `backup/backup.go`, `settings/settings.go`, `report/escrow_confirm.go`,
`report/escrow_presence_wiring_test.go`, `cmd/controller/main.go`, `web/{recovery_handlers,handlers,server,auth}.go`,
`web/recovery_test.go`, `templates/{layout,recovery,backups_remote}.html`,
`CHANGELOG.md`, `CONTEXT.md`, `REUSE.md`, `controller/README.md`.
## Files
**`felhom.eu`** — `ac4b2a4ba934` (hub purge) · `9657334fb72a` (registers, map, STATUS, hub CHANGELOG) ·
`b12f8ec2f32f` (manifest). Deployed: **`felhom-hub:0.98.0`**, Synced/Healthy.
- `internal/backup/offbox.go``ErrOffboxRunInFlight`, `offboxWholeUnitGap`, the skip classification,
`missingUnprotected`/`missingNotDeployed`, `stackDeployed`, `driveUnavailableFor`, the verdict, the
messages
- `internal/backup/offbox_capture.go` — the sibling message
- `internal/backup/backup.go``AcquireRunningForTest`/`ReleaseRunningForTest` (test-only seams)
- `internal/web/offbox_handlers.go` — the synchronous single-flight refusal
- new `internal/backup/offbox_verdict_r234_test.go`, `internal/web/offbox_run_inflight_test.go`
- `internal/backup/offbox_3a_test.go` — opt-in deployed set
- `CHANGELOG.md`, `controller/README.md`, `CONTEXT.md`
## 9. Registers
## Observations, not acted on
**R-241 FIXED.** **R-243 UPDATED, not closed** — the state can no longer be entered and what replaces
it is *visible* rather than silent, but **the alarm gap is untouched**: a box whose customer never acts
still stops backing up with no operator signal. **R-245 NEW** (WAITING-ON-OPERATOR). **R-242 stays
recorded-not-built.** **R-244 untouched.** Still open and named: **R-240, R-213, R-202, R-214**.
- **The zero-selection notice reads „Sikeres — nincs mentésre jelölt alkalmazás"** while the status is
`ok`. It is honest, but "Sikeres" beside "nothing is covered" is the same rhetorical shape this
session is about, one notch weaker. Not touched: Scenario D forbids changing that path.
- **`res.missing` is still used verbatim** for the no-silent-success error text (every app missing →
`error`). Correct as-is, and deliberately left, since that path already refuses loudly.
**Highest register ID moves R-244 → R-245.**
## 10. The capability-map row
**Still FAIL.** These are fixes, not a walk — nothing here walked a customer end to end, and the row
goes green only when one completes **with no operator intervention AND a byte-identical sentinel**.
## 11. CI
`felhom-controller` run **237** (`0a9158d53eb8`) success · `felhom.eu` runs **238** (`9657334fb72a`)
and **239** (`b12f8ec2f32f`) success — matched by `head_sha`, pulled rather than assumed.
**`--no-verify` was NOT used**; the pre-push hook ran the gates on every push.
## 12. Observations — noticed, NOT acted on
- **demo-hp's escrow blob is flagged stale** (`stale_at = 2026-08-04 20:15:49`, from the R-201 drill),
so the hub withholds its hash and shape (c) can never fire there. Correct per §7.2, and it gave the
fallback path a free live exercise — but that box's escrow **has been stale for three days** and
nothing has surfaced it. Possibly worth a row; not filed, because it is R-198/R-196 territory and I
did not measure whether the staleness is real or an artefact of the drill.
- **The offer epoch advances on a landing-page visit, not on the report cycle.** Correct for the
interruption and the banner (both only matter when someone visits), but it means the undecided
reminder ladder starts from the first *visit* rather than from the first *report*. Deliberate;
stated here because it is a design choice a reader could mistake for an oversight.
- **`--abandon-*` runs inside the guest**, i.e. a guest command line. Acceptable for operators (it sits
beside the existing operator subcommands) but it is not a hub surface, so it is unavailable to an
operator who cannot reach the box.