E-2 complete: wrapper, installer Case A/B, offer flow, degraded banner

Live: hub 0.81.0, agent 0.113.0, controller 0.185.1 on both demo boxes;
host-install 1.22.0 (script; no reinstall performed).

E-2a wrapper proven live as root on demo-hp: F-1 subdirectory refused, F-2
unmounted path refused, root device refused, idempotent re-apply is a no-op,
repointing refused -- 0 stray storages. The agent PVE role was NOT widened.

Scenario E proven live on BOTH boxes: healthy renders nothing, no message key.

Records three defects I introduced and caught: unreachable routes (mounted
outside /api/storage/, caught by the first live call), a hollow test exposed by
its own red-proof, and another gofmt-realignment no-op.

Not live-proven: the degraded banner and offer acceptance (both boxes healthy),
backup_target_absent end-to-end, Case A/B on a real install, drive-loss recovery.
This commit is contained in:
2026-07-29 09:16:59 +02:00
parent f6aed82940
commit bcbe2707d6
2 changed files with 74 additions and 2 deletions
+2 -2
View File
@@ -39,8 +39,8 @@ State: `BLOCKED` · `READY` · `WAITING-ON-OPERATOR` · `WATCHING`. Every row ha
| **F-REBOOT** | ~~A guest rebooted during its backup does not come back — shutdown completes, start never happens, no self-heal; 9m47s total appliance outage with every alarm silent~~ | **SHIPPED + PROVEN-LIVE** (agent v0.107.0, 2026-07-28) | — | 60 s guest-power watchdog; `onboot` is the deliberate-stop discriminator (already the stale-lock path's, and what `pve-guests` consults), retry bounded 3x/1m-2m-4m then escalates once. Live on demo-hp: **120 s unattended** vs the incident's 587 s with a human; Scenario B proven (an `onboot:0` guest left stopped) | — |
| **F-LEAK** | ~~A failed restore-test cannot destroy its own scratch guest (403 `VM.Allocate`); the 10-slot VMID band shrinks silently~~ | **SHIPPED + PROVEN-LIVE** (agent v0.110.0 + host-install v1.21.0, 2026-07-28) | — | **Three attempts, two refuted live.** (1) Pool adoption: `PUT /pools/{pool}` also needs `VM.Allocate` on the VM — membership cannot bootstrap its own authority. (2) Per-path `/vms/990000..990009` ACLs: work, but PVE's destroy calls `remove_vm_access` (`LXC.pm:906`) which deletes every ACL at `/vms/<vmid>`**consumed by the op it authorises**, one use per slot. (3) SHIPPED: 4th root-fenced exception, band enforced in sudoers **literally** (`pct destroy 99000[0-9] --purge`) + in code + at the caller; API destroy still tried first. Live: band PERMITTED, `9201`/`9100`/`9999`/`990010`/`1` REFUSED, and `pct start 990000` REFUSED too | — |
| **F-OBS** | ~~`deadapp-check` leaves NO positive observable on a default (info-level) box — "no alarms" was indistinguishable from "never ran"~~ | **SHIPPED + PROVEN-LIVE** (controller v0.180.0 + agent v0.109.0, 2026-07-28) | — | INFO summary every 20th scan carrying scans/evaluated/down. **Agent v0.109.0 fixes the same shape in the guest-power watchdog shipped hours earlier in v0.107.0** — it logged only at startup and when it acted, so its health could be read only from absence | — |
| **E-2** | **IN PROGRESS 2026-07-29.** SHIPPED: hub v0.81.0 (`backup_target_absent`/`_restored` in allowedEventTypes + Hungarian customerMessages, 3 tests, red-proofed) and controller Part 1 (`StoragePath.BackupTarget` role + `SetBackupTarget`/`ClearBackupTarget`, 5 invariant tests, red-proof C). **REMAINING: Part 2 installer Case A/B, Part 3 the offer + the agent-side move, Part 4 the degraded banner, Part 5 the controller half of the signal, red-proofs E/F, live validation.** | **READY (M) — partially shipped** | — | **Phase 0 inverted the emphasis: the installer has NO drive-enrollment step (`/mnt/felhom-drives` is agent-installed at runtime), so at install time the common case is system-drive-only and Part 3 (drive added later) is the PRIMARY path, not Case A.** Scope + rationale: `runbooks/RUNBOOK-vzdump-target-move-2026-07-29.md` §7. Flips matrix row 4 | CC |
| **E-2a** | **The target move needs a root-fenced wrapper — the agent cannot do it.** Creating a PVE `dir` storage needs `Datastore.Allocate` at `/storage` and the ACL grant needs `Permissions.Modify`; the agent holds neither by design (blast-radius containment), and its sudoers is a literal-argument `Cmnd_Alias` model with **no `pvesm` and no `pveum`**. The established pattern is a guarded wrapper (`felhom-mkfs-guarded`, `felhom-pbs-apply` — the latter already has the exact `grant <id>` dual-grant verb E-2 needs) | READY (M) | blocks E-2 Part 3 | Add `felhom-backup-target-apply` (create-dir-storage with `is_mountpoint 1` + grant) + its literal sudoers rule + installer install step, then the agent endpoint. **Do NOT widen the agent's PVE role instead** | CC |
| **E-2** | ~~Drive-role machinery around the moved vzdump target~~ | **SHIPPED** (hub 0.81.0, agent 0.113.0, controller 0.185.1, host-install 1.22.0 — 2026-07-29) | — | Parts 15 complete. Role model + offer-only assignment + Hungarian degraded banner + absent-target signal + installer Case A/B. **NOT yet live-proven:** the DEGRADED banner and the offer acceptance (both demo boxes are healthy, so neither state occurs naturally) and `backup_target_absent` end-to-end. Installer is **installer-logic-tested, not install-tested** — no reinstall was performed | CC |
| **E-2a** | ~~The target move needs a root-fenced wrapper — the agent cannot do it~~ | **SHIPPED + PROVEN-LIVE** (agent v0.113.0 + host-install v1.22.0, 2026-07-29) | — | `felhom-backup-target-apply` behind a literal `FELHOM_BACKUPTARGET` sudoers alias; the agent's PVE role was NOT widened. Enforces F-1 (`mountpoint -q`) and F-2 (`is_mountpoint 1` hardcoded), refuses a root-device target, has NO storage-removal path (grep-assertable), is idempotent and refuses to repoint. All five laws proven live as root on demo-hp with 0 stray storages | |
| **E-2b** | ~~`NotifyStorageDisconnected`/`Reconnected` defined and called NOWHERE — a drive going absent emitted no event on any channel~~ | **SHIPPED + PROVEN-LIVE** (controller v0.184.1 + agent v0.112.0 + hub v0.81.0, 2026-07-29) | — | Seam wired in `ReconcileDriveGates`; a target drive raises the specific `backup_target_absent` instead. **A keying bug was caught before deploy:** `a.Path` is the registered GUEST path, not the agent's host `MountPath`, so the target branch was unreachable — every absent drive, target included, fell through to the generic event (v0.184.1). Tests observe the WIRE (httptest hub), not a mock | — |
| **E-2c** | ~~E-1 put the whole-guest backups on a drive `POST /disks/eject` would eject~~ | **SHIPPED + PROVEN-LIVE** (agent v0.112.0, 2026-07-29) | — | Eject + decommission refuse 409 on the backup-target mount, naming the storage and the remedy. **Live on BOTH boxes:** demo-hp `/mnt/nvme-1tb` and demo-felhom `/mnt/hdd_1` both refused, drives unmoved. NOT a role reclassification — `RoleForStorage` untouched, because on both boxes that drive is ALSO the enrolled user-data drive; `TestEjectStillAllowedOnANonTargetDrive` pins the non-over-correction and `/var/lib/vz` is still refused by the PRE-EXISTING role gate, not this one | — |
| **PETI** | **`peti-felhom` deliberately NOT migrated.** Its whole-guest backup still shares a device with its guest, so a drive failure there is **offsite-only recovery** | **ACCEPTED RISK — parked** | operator's next visit (tester reinstalling from scratch) | **Accepted until the reinstall; re-evaluate if that slips past ~2026-09-01.** Do not migrate, do not touch | operator |