felhom.eu/REPORT.md

# REPORT — E-2 increment 1: the backup target is now defended and audible (2026-07-29)

**Overwritten** per the standing rule. **E-2 is PARTIALLY shipped** — this increment is Parts 1 + 5
plus findings E-2b/E-2c. Parts 2, 3, 4 and E-2a remain open (see §6).

**Live fleet after this run:** hub **0.81.0**, agent **0.112.0** (both boxes), controller **0.184.1**
(both boxes). `peti-felhom` deliberately untouched.

---

## 1. Why this order

The operator reordered the plan, correctly: **E-2b and E-2c protect a configuration that exists
right now; E-2a builds for boxes that do not.** Both demo boxes are configured, Peti is parked, and
no new installs are happening this week — so Part 3 has no user yet, while E-1's new configuration
had a drive that could be ejected without warning and no event when it went absent.

## 2. E-2c — the backup drive could be ejected out from under the backup

`RoleForStorage` types a `local-dir` on a non-system device as **user-data**, so the pre-existing
eject role gate PASSED the drive E-1 had just made the vzdump target. `POST /disks/eject` on
`/mnt/nvme-1tb` or `/mnt/hdd_1` would have **succeeded silently**, taking the only local whole-guest
backup with it while the box kept reporting a configured tier.

Fixed in agent v0.111.0/0.112.0: `handleDiskEject` and `handleDiskDecommission` refuse `409` when the
mount backs a configured tier, naming the storage and the remedy — the op is *ordered*, not
forbidden.

**Deliberately NOT a role reclassification.** Making `RoleForStorage` return `RoleBackup` would
refuse every legitimate eject of the customer's own data drive, because on both boxes that drive
**is** the target. `TestEjectStillAllowedOnANonTargetDrive` pins that, and the over-correction is
red-proofed.

**Live on both boxes:**

```
demo-hp      felhom-backup mount=/mnt/nvme-1tb role=user-data backup_target=True
demo-felhom  felhom-backup mount=/mnt/hdd_1    role=user-data backup_target=True
```

That single line is the exposure: `role=user-data` (which the gate permits) yet `backup_target=True`.
Both ejects returned **409** with the full message; both decommissions **409**; both drives still
mounted with `dump/` intact — the refusal had no side effect. `/var/lib/vz` is still refused by the
**pre-existing** role gate (`role: system`), proving the new gate is narrow.

## 3. E-2b — the alarm that was defined and never called

`NotifyStorageDisconnected` and `NotifyStorageReconnected` were registered in `allowedEventTypes`, in
`DefaultEnabledEvents`, and given Hungarian hub copy — and **called from nowhere**. A drive going
absent produced stopped apps, a `[WARN]` log and a UI badge, then silence on every channel. A drive
that is *only* a backup target has no apps to stop, so it was silent twice over. Fifth instance of
this class; found by Phase 0, not by a failure.

Now wired in `ReconcileDriveGates`, with the target case raising the more specific
`backup_target_absent` (error) and recovering as `backup_target_restored` (info — the pairing-gated
pattern; `severityNotifies` NOT widened). Never both: two mails for one event trains people to
ignore the channel.

## 4. The bug I introduced and caught before deploy

`ReconcileDriveGates` resolves the target as `isTarget[a.Path]`, and **`a.Path` is the registered
GUEST path** (`/mnt/felhom-drives/<name>`), not the agent's host `MountPath`. I had keyed the map on
`MountPath` alone, so the backup-target branch was **unreachable** — every absent drive, the target
included, would have fallen through to the generic event.

It would have looked wired, passed its own unit tests, and shipped silently wrong on exactly the
drive it exists for. Caught by tracing `a.Path` back to its source instead of assuming. Fixed in
v0.184.1 (both keys, mirroring `planDriveGates`), red-proofed, and the **0.184.0 image is superseded
and was never deployed**.

## 5. Tests and red-proofs

| Red-proof | Observed failure |
|---|---|
| hub allowlist entry removed | `backup_target_absent must be in allowedEventTypes, or POST /event 400s…` |
| hub Hungarian copy shortened | `must state the consequence…, got: "Meghajtó hiányzik."` |
| C — auto-elevation on register | `registering drives assigned the backup target "/mnt/hdd_1"` |
| E-2c — eject guard removed | `eject of the backup-target drive SUCCEEDED (200)` |
| E-2c over-correction | the gate blocked `/mnt/spare` (a non-target) |
| keying reverted to MountPath | `not resolvable by its GUEST path — …the branch would never fire` |

Every mutation was **verified to have landed** before the result was trusted. The first attempt
silently no-op'd (gofmt had realigned the map) and the test "passed" — a false proof.

Controller notify tests observe the **wire** (a real `Notifier` against an `httptest` hub), not a
mock, because the failure class is "nothing arrives".

## 6. What is NOT done

- **Part 2** (installer Case A/B), **Part 3** (the offer + the agent-side move), **Part 4** (the
  degraded Hungarian banner). Phase 0 inverted their emphasis: the installer has **no drive-enrollment
  step**, so at install the common case is system-drive-only and **Part 3 is the primary path**.
- **E-2a** — the guarded wrapper. The agent holds neither `Datastore.Allocate` at `/storage` nor
  `Permissions.Modify`, and its sudoers has no `pvesm`/`pveum`. Use the wrapper pattern; **do not
  widen the agent's PVE role.**
- **`backup_target_absent` is not proven END-TO-END live.** Its components are: the controller emits
  the right type/severity (proven at the wire), the hub allowlists it (unit-pinned, image deployed),
  and the keying is pinned. Proving the whole chain needs a live drive to actually go absent, which
  means unmounting an enrolled drive carrying customer data and the guest bind on a remote box —
  the same thing declined in E-1, for the same reason.
- Full drive-loss recovery (physical access) — carried forward from E-1.

## 7. Observations

- **`peti-felhom` untouched.** Its whole-guest backup still shares a device with its guest, so a
  drive failure there is offsite-only recovery. Accepted until the operator's reinstall; re-evaluate
  if that slips past ~2026-09-01.
- The fleet was briefly split (demo-hp updated first). Left that way it would have protected one of
  two identically-exposed boxes, so demo-felhom was rolled to the same proven artifacts.
- `StorageTarget.Role` carries a documented vocabulary including `vzdump-target` and is always `""`.
  Checked — it is a deliberate unpopulated field ("hub-owned, slice 10"), **not** a sixth dead seam.

---

# E-2 increment 2 (same day): the wrapper, the installer, the offer and the banner

**Live fleet:** hub **0.81.0**, agent **0.113.0**, controller **0.185.1**, host-install **1.22.0**
(script; no reinstall performed). Both demo boxes. `peti-felhom` untouched.

## E-2a — the guarded wrapper

`configs/felhom-backup-target-apply` behind a literal `FELHOM_BACKUPTARGET` sudoers alias. **The
agent's PVE role was not widened** — it still holds neither `Datastore.Allocate` at `/storage` nor
`Permissions.Modify`.

All five laws proven live as root on demo-hp, with **0 stray storages created**:

| Probe | Result |
|---|---|
| F-1 subdirectory (`/mnt/nvme-1tb/dump`) | REFUSED — "must be the drive's OWN mountpoint" |
| F-2 unmounted path | REFUSED — "would silently retarget onto the system drive" |
| root device (`/`) | REFUSED |
| idempotent re-apply on the real target | no-op, exit 0 |
| repointing an existing id | REFUSED |

`POST /backup/target` drives it **create → grant → config**, and deliberately does **not** restart the
agent: it returns `restart_required` so the restart happens behind the caller's own immediate
in-flight check. That is E-1's spurious-tier-failure mistake encoded as a contract, and a test asserts
the handler never restarts.

## Part 2 — the installer (v1.22.0)

Case A creates the target through the same wrapper; Case B records DEGRADED in two warning lines and
**still succeeds** — a single-drive appliance is valid. Scenario F returns early with **0 wrapper
calls**; the red-proof (guard removed) shows **2**, i.e. it would have "corrected" a correct box.

**Installer-logic-tested, NOT install-tested** — the harness runs the extracted functions with
stubbed `pvesm`/wrapper. No reinstall was performed.

## Parts 3+4 — the offer and the banner

`POST /api/storage/backup-target/assign` is the **only** writer of the role. `GET
/api/storage/backup-target` drives the banner. Hungarian copy is FACT → CONSEQUENCE → REMEDY.

**Scenario E proven live on both boxes** — healthy renders nothing:

```
demo-hp      {"degraded":false,"known":true,"label":"NVME 1TB","target":"felhom-backup"}
demo-felhom  {"degraded":false,"known":true,"label":"USB HDD 1TB","target":"felhom-backup"}
```

No `message` key at all. Red-proofed: adding a reassurance to the healthy branch fails Scenario E.

## Three defects I introduced and caught

1. **The offer endpoints were unreachable.** Registered as `/api/backup-target` inside
   `ServeStorageAPI`, which `main.go` mounts only at `/api/storage/`. Every unit test passed because
   they called the handlers directly; the **first live call** returned `endpoint not found`. Fixed in
   v0.185.1 with a test that asserts the dispatcher's own source contains both routes.
2. **A hollow test.** `TestUnknownStateRendersNothing` used `{Known:false}` with `Degraded` left
   false, so it passed even with the `!Known` guard deleted. Its own red-proof exposed it; the
   fixture is now `{Known:false, Degraded:true}`.
3. **A gofmt-realignment no-op** (again): a `sed`-style replace silently matched nothing because the
   formatter had changed the spacing. Every mutation is now verified to have landed before its result
   is trusted.

## Still not live-proven

- The **DEGRADED banner** and the **offer acceptance** — both boxes are healthy, so neither state
  occurs naturally. Unit- and red-proofed only.
- **`backup_target_absent` end-to-end** — needs a live enrolled drive to go absent.
- **Case A/B on a real install** — installer-logic-tested only.
- **Full drive-loss recovery** — physical access.