255 lines
15 KiB
Markdown
255 lines
15 KiB
Markdown
# REPORT — R-48 restore wizard (v0.154.0) + the v0.155.0 follow-up fix; STOP-1 BANKED; Part 3 STOPPED
|
||
|
||
Session date: **2026-07-21**. Executed on DooPlex as `kisfenyo`.
|
||
|
||
## 0. Outcome at a glance
|
||
|
||
| Leg | Status |
|
||
|---|---|
|
||
| **Part 1 — R-48 restore wizard (v0.154.0)** | **SHIPPED** `3a9d744`, image published, deployed via the floor save. |
|
||
| **v0.155.0 — follow-up fix** | **SHIPPED + DEPLOYED** `9d1b498`. Fixes a real defect in v0.154.0 found by the operator on the first live click-through. |
|
||
| **Part 2 — capability-map cell** | **DONE** `ce8c539`. |
|
||
| **STOP-1 — floor save + single-fire** | **BANKED.** Exactly one swap over a 39-min window, no rollback, healthy. §6. Doc rows flipped `11ead4b`. |
|
||
| **Live click-through + Ellenőrzés** | **DONE by the operator**, and re-run endpoint-level after the v0.155.0 fix. §7. |
|
||
| **Part 3 — agent 0.90.1** | **STOPPED, ruled.** Not published, not deployed. §8. |
|
||
|
||
## 1. Baselines
|
||
|
||
| Repo | start | end |
|
||
|---|---|---|
|
||
| felhom-controller | `b30e2e5` | `9d1b498` (v0.155.0) |
|
||
| felhom.eu | `fb0b8c1` | `c35da9d` |
|
||
| felhom-agent | `8c55ac7` | **unchanged — no commit, no publish, no deploy** |
|
||
|
||
## 2. Phase-0 probes
|
||
|
||
**P1 — self-update wiring: PASS.** `self_update.enabled: true`, image correct. `auto_update: false` is
|
||
irrelevant and this was verified by reading the code, not assumed: `MaybeAutoUpdate` never consults
|
||
`cfg.AutoUpdate` — that flag is the customer's opt-in to chase *latest*, while the FLOOR path is the
|
||
managed one. Stronger evidence: `update-state.json` already recorded a prior `initiated_by: auto-floor`
|
||
success (0.143.0 → 0.145.0, 2026-07-18), which proves the agent swapper is wired. Anti-flap could not
|
||
block, since the persisted target was `0.145.0`. `0.154.0` verified as the highest registry semver tag,
|
||
so the `floor <= latest` gate would pass.
|
||
|
||
**P2 — agent version: PASS, and it changed the plan.** Built at `main`, self-reported
|
||
`felhom-agent 0.90.1`, sha256 `ba1d0296…`, 13 734 067 bytes. But see §8.
|
||
|
||
**P3 — before-state: CAPTURED** (endpoint-level; no browser on DooPlex). The offsite section rendered
|
||
**6× `/backup/offbox/restore`, 1× `place`, 1× `reconstitute`** across three apps. For immich the two
|
||
decisive controls were adjacent siblings differing only by label:
|
||
|
||
```
|
||
Helyreállítás az élő adatok közé (csak a hiányzó fájlok) <- data CANNOT come back
|
||
Teljes visszaállítás (fájlok + adatbázis) <- data CAN come back
|
||
```
|
||
|
||
## 3. v0.154.0 — the wizard (`3a9d744`)
|
||
|
||
One „Visszaállítás…" entry per app row → `GET /backups/restore/app?name=<app>`; three intent cards
|
||
with consequence sentences; danger styling and the R-43 double-confirm carried over verbatim; pure
|
||
`deriveWizardStep`; **no new mutation endpoint**; works without JavaScript. `internal/backup`,
|
||
`internal/appbackup`, `internal/selfupdate`, the shares/local/.fab blocks all untouched.
|
||
|
||
**Latent bug fixed on the way:** `offboxRedirectTo` hardcoded `"?"` when appending its flash, which
|
||
against the wizard's `?name=<app>` target would have buried the flash inside the app name.
|
||
|
||
**Group-B red-proof (run, reverted):** trivial always-INTENT impl → **all 7 table rows FAILED** plus
|
||
the execution render test. Restored, green, `git diff` clean before commit.
|
||
|
||
Also caught one of my own assertions being hollow during authoring: a bare substring check for the
|
||
skew/empty warnings passed on a *clean* pair, because the `confirmFullRestore` JS repeats both
|
||
sentences as string literals. Tightened to assert rendered banner markup + `data-restore-*`.
|
||
|
||
## 4. v0.155.0 — the defect v0.154.0 shipped (`9d1b498`)
|
||
|
||
**Found by the operator on the first live click-through**, from the phase strip disagreeing with the
|
||
progress banner on the same screen. This is the most important finding of the session.
|
||
|
||
`backup.Manager` carries **two** different "is something running" booleans:
|
||
|
||
| flag | read by | set by | covers the verification restore? |
|
||
|---|---|---|---|
|
||
| `running` | `IsRunning()` | `acquireRunning()`, **inside** the goroutine | **no — `RestoreOffboxScratch` never acquires it at all** |
|
||
| `opRunning` | `RestoreStatus()` | `BeginRestoreOp()`, in the handler, synchronously | yes, all four offsite actions |
|
||
|
||
v0.154.0 sourced `OpRunning` from `IsRunning()`. Verified by reading the call sites:
|
||
`PlaceOffsiteRestore` and `ReconstituteFromOffsite` do acquire; **`RestoreOffboxScratch` does not**.
|
||
So for „Ellenőrzés" and the full-restore preparation — the wizard's two most-used and longest actions,
|
||
both streaming from restic — the execution step was **unreachable**. The page offered all three
|
||
intents with live buttons while a restore ran; the handler would have refused any of them. That is
|
||
precisely the "offering a control guaranteed to fail" dishonesty R-48 exists to remove, shipped inside
|
||
the release that removes it.
|
||
|
||
**Fix:** `restoreOpInFlight(st)` behind a documented seam, fed by a **single** `RestoreStatus()` read
|
||
per render, so the strip, the suppression decision and the running-op name cannot diverge. The helper
|
||
sits on the handler's actual path — deliberately, so a future edit reintroducing `IsRunning()` fails
|
||
the test rather than passing it.
|
||
|
||
**Why v0.154.0's tests missed it — the generalisable lesson.** The Scenario-E table proved
|
||
`deriveWizardStep` behaves correctly *given* `OpRunning=true`. Nothing proved the handler ever
|
||
*computes* `true`. **A table test over a pure function proves the function, not the caller.** The new
|
||
`TestRestoreOpInFlight_UsesDisplayFlagNotConcurrencyFlag` drives a real `Manager` through
|
||
`BeginRestoreOp` and asserts the render.
|
||
|
||
**Red-proof (run, reverted):** restoring the v0.154.0 `IsRunning()` shape →
|
||
|
||
```
|
||
--- FAIL: TestRestoreOpInFlight_UsesDisplayFlagNotConcurrencyFlag
|
||
restore_wizard_test.go:367: a started restore op MUST read as in-flight for display
|
||
(this is the v0.154.0 bug)
|
||
```
|
||
|
||
**„Eredmény" was a dead label.** It never lit in v0.154.0. The strip's highlight is now its own
|
||
derived `Phase`, separate from `Step`: a finished restore is back on the intent step (everything
|
||
available again) while the strip reads „Eredmény" and an outcome card shows the result. Bounded by
|
||
`restoreResultWindow` (10 min) so a stale result cannot look fresh, and **bound to the app** so a
|
||
finished bookstack restore cannot light immich's page with bookstack's message. The card survives a
|
||
reload; the flash does not. 9-row table test incl. both boundary directions and a future-clock-skew row.
|
||
|
||
## 5. Test + gate results
|
||
|
||
Full suite green at both commits: `go build ./... && go vet ./... && go test ./...` — 0 failures.
|
||
All six controller design gates rc=0.
|
||
|
||
| Group | Test | Result |
|
||
|---|---|---|
|
||
| B | `TestDeriveWizardStep_Table` (9 rows) | PASS |
|
||
| C | `TestResolveWizardApp_Refusals`, `TestRestoreWizardHandler_UnconfiguredRedirects` | PASS (302, no 500) |
|
||
| A | `TestRestoreList_SingleEntryPerApp` | PASS |
|
||
| C | `TestRestoreWizard_ThreeIntentCards`, `..._NoScratchLocksDataIntents` | PASS |
|
||
| E | `TestRestoreWizard_OpRunningSuppressesAllMutations` | PASS |
|
||
| D | `TestRestoreWizard_NoNewMutationEndpoints`, `..._FieldContract` | PASS |
|
||
| **new** | `TestRestoreOpInFlight_UsesDisplayFlagNotConcurrencyFlag` | PASS (red-proofed) |
|
||
| **new** | `TestHasRecentRestoreResult` (9 rows), `TestRestoreWizard_ResultCard` | PASS |
|
||
|
||
Two pre-existing tests coupled to the old IA were **inverted, not deleted**: `TestAppRow_RestoreLists`
|
||
and `TestBackupsSplit_SectionsOnExactlyOnePage` now assert the mutation forms are ABSENT from the list.
|
||
|
||
## 6. STOP-1 — single-fire self-update: **BANKED**
|
||
|
||
Operator saved global floor → v0.154.0. Timeline (host CEST / guest UTC, 2 h offset):
|
||
|
||
```
|
||
06:57:13Z UpdateState "pending" initiated_by=auto-floor
|
||
06:57:17Z agent WARN controller swap requested 0.153.0 -> 0.154.0
|
||
06:57:19Z agent INFO image file written, restarting bootstrap
|
||
06:57:21Z container StartedAt · UpdateState completed_at
|
||
06:57:29Z agent INFO new controller healthy
|
||
```
|
||
|
||
| Assertion | Result |
|
||
|---|---|
|
||
| `controller swap requested` | **1** |
|
||
| `image file written, restarting bootstrap` | **1** |
|
||
| `new controller healthy` | **1** |
|
||
| rollback / swap-failed / unhealthy | **0** |
|
||
| container `RestartCount` | **0** |
|
||
| bootstrap unit `Result` | `success` |
|
||
|
||
> **Correction — I over-claimed one line.** I originally cited the bootstrap unit's `NRestarts=0` as
|
||
> no-storm evidence. It isn't: the unit is `Restart=no`, so `NRestarts` is 0 by construction and would
|
||
> read 0 during a storm too. Dropped. The real no-storm evidence is the agent-side swap count and the
|
||
> container's own `RestartCount=0`.
|
||
|
||
`UpdateState` persisted `status: success`, `0.153.0 → 0.154.0`, `initiated_by: auto-floor`.
|
||
`VerifyStartup` banked it on the next boot (`Post-update startup: update successful`), and the 06:57:52
|
||
periodic check reported `Current version 0.154.0 is up to date` — the anti-flap correctly doing nothing
|
||
now that current ≥ floor. **Exactly one swap, no storm, no rollback.** This closes the leg R-23(a) was
|
||
missing: the earlier floor save proved nothing because it targeted an already-running version.
|
||
|
||
**30-minute no-re-attempt check — PASSED, and my earlier caveat was broader than the facts.** Recounted
|
||
over 06:50 → 07:29 (39 min): `controller swap requested` = **1**, agent-driven bootstrap restarts = **1**.
|
||
|
||
I hand-deployed v0.155.0 at `07:17:10Z`, inside the observation window, and flagged that as
|
||
contaminating the result. Having checked the scope: it contaminates **`StartedAt` only**. A hand-deploy
|
||
writes `/etc/felhom-controller-image` and restarts the unit directly via `pct exec` — it never goes
|
||
through `SwapController` — so the swap-count assertions, which are the actual evidence, hold
|
||
uncontaminated across the whole window. I was right to flag it and wrong about how much it cost; the
|
||
window is clean, not merely the 19 minutes I claimed.
|
||
|
||
**An unplanned bonus confirmation came with it.** After the hand-deploy the box ran **0.155.0 against a
|
||
0.154.0 floor** — i.e. above the floor — and the updater logged `Current version 0.155.0 is up to date`
|
||
and did nothing. That is Scenario B ("at/above floor does NOTHING — must not chase latest") confirmed
|
||
live, which no test in this session set out to produce.
|
||
|
||
## 7. Live validation
|
||
|
||
Method: **endpoint-level** (no browser on DooPlex) plus the operator's own UI click-through.
|
||
|
||
**Operator, in the browser (v0.154.0):** list page → one „Visszaállítás…" per app; wizard → three
|
||
cards, pair-honesty panel showing `adatbázis-mentés ideje: 2026-07-19 12:28`; ran one **Ellenőrzés**;
|
||
result `A(z) immich visszaállítva ellenőrző mappába: /mnt/felhom-drives/hdd_1/backups/offsite-restore/immich
|
||
(a meglévő adatok változatlanok).` Non-destructive as designed.
|
||
|
||
**Endpoint-level, after the v0.155.0 fix — the proof the defect is gone.** A real Ellenőrzés run
|
||
through the actual endpoint:
|
||
|
||
| moment | phase strip | `<form>` count | note |
|
||
|---|---|---|---|
|
||
| before | Előkészítés | 3 | intents offered |
|
||
| **mid-flight** | **Végrehajtás** | **0** | `Jelenleg egy mentési vagy visszaállítási művelet fut (immich).` |
|
||
| after | **Eredmény** | 3 | result card names the scratch path; intents available again |
|
||
| bookstack, same moment | Előkészítés | — | **no** result card — app-binding holds |
|
||
|
||
Under v0.154.0 the mid-flight row would have read `Előkészítés / 3 forms`. Before/after on the list
|
||
page: **6 restore + 1 place + 1 reconstitute forms → 0**, and 3 wizard entries. Refusal path: unknown
|
||
app → **302** + Hungarian flash, no 500.
|
||
|
||
Deploy verified: `felhom-controller:0.155.0 | Up (healthy)`, `RestartCount=0`.
|
||
|
||
## 8. Part 3 — STOPPED, then ruled
|
||
|
||
Part 3 asked me to publish + deploy v0.90.1 as "R-39's cheap half — the PBS wrapper argv fix", while
|
||
Rule 12 forbade touching the wrapper. Both cannot hold, because **the fix IS the wrapper**:
|
||
|
||
- **`9596d5a` changes zero non-test Go files** — `CHANGELOG.md`, `REPORT.md`, `configs/felhom-pbs-apply`,
|
||
`internal/pbsdr/manager_test.go`. Its own message: *"the Go binary is unchanged."*
|
||
- **felhom-pve already has it** — wrapper hotfixed 2026-07-18, `args=(--fingerprint "$fp")` at L107,
|
||
`.bak-20260718-preR39` retained.
|
||
- **New hosts get it too, unversioned** — `felhom-host-install.sh:1914` `fetch_raw` pulls the wrapper
|
||
from `raw/branch/main`, and `9596d5a` is an ancestor of `main`.
|
||
|
||
Publishing would have delivered no behaviour change, restarted a production agent at a remote site for
|
||
nothing, and — once the Day-0 manifest was saved to 0.90.1 — advertised a versioned fix the artifact
|
||
channel never carried. Registry pre-check confirmed and left as found (0.90.1 → **404**, 0.90.0 → 200).
|
||
|
||
**Operator ruling: leave 0.90.0 published; record the closure in the docs.** Done in `ROADMAP.md`
|
||
(`c35da9d`). The stop surfaced a genuine new defect, filed as **R-50b**: a root-owned privileged host
|
||
artifact is delivered unversioned from `main`, absent from the Day-0 manifest, so "which wrapper is on
|
||
this host?" is unanswerable from any manifest.
|
||
|
||
## 9. Doc state
|
||
|
||
- Capability map row 61 → **PROVEN-LIVE** (`ce8c539`).
|
||
- ROADMAP: **R-48 → SHIPPED**; R-39 cheap half recorded as closed-in-field with the publish cancelled;
|
||
**R-50b** filed (`c35da9d`).
|
||
- **FLIPPED** (`11ead4b`): R-23(a) → **BANKED in full** (only the cosmetic (b) leg remains), and the
|
||
capability-map row "Config/state change round-trips in **seconds**" → **PROVEN-LIVE (2026-07-21)**.
|
||
That row's residual PARTIAL was explicitly "the R-23 operator-UI save→apply leg alone", which §6
|
||
closes. Both rows carry the hand-deploy disclosure and the bonus at/above-floor confirmation.
|
||
|
||
## 10. Remaining — human
|
||
|
||
1. **Floor → v0.155.0** at your convenience, so the fleet matches the box (9201 is hand-deployed and
|
||
above floor, so nothing is pending on it).
|
||
2. **TASK-B** ruling (R-39 primary half: the descriptor-hash re-apply gap).
|
||
3. **R-50b** ruling — recommendation is (a): record the wrapper's sha256 in the Day-0 manifest and have
|
||
the agent report the installed file's hash, so drift becomes visible.
|
||
4. **C6 customer-restore pass** for the alpha.
|
||
|
||
## 11. Observations
|
||
|
||
- **The most valuable finding came from the operator, not the tests.** The phase strip he questioned
|
||
was the visible symptom of a real bug — the strip and the progress banner disagreeing on one screen.
|
||
A UI element that renders derived state is an assertion the customer can check; that is worth more
|
||
than its decorative cost.
|
||
- **`docker_run_volume_path_gate.py` fails at `HEAD`, unrelated** (`internal/appexport/estimate.go:179`).
|
||
Verified pre-existing by stashing and re-running. Left alone — but that gate is currently not a
|
||
usable signal.
|
||
- **`opstatus.go` says in its own header that it is the DISPLAY surface** and does not gate
|
||
concurrency. I read the concurrency flag anyway. The file told me; I did not check.
|
||
- **`felhom-pve` is up and healthy** (agent 0.90.0, `active`) — the "agent is DOWN" TEMPORARY block in
|
||
`felhom-controller/CLAUDE.md` is stale and should be trimmed on return.
|
||
- **P3's accented-grep trap fired as documented** — used a local Python pass over fetched bytes rather
|
||
than grepping accented patterns through `ssh → pct exec → bash -c`.
|