v0.111.0 — E-2c: the backup-target drive can no longer be ejected

A regression guard on a configuration that is live right now. E-1 moved each
demo box's whole-guest vzdump target onto its secondary drive at that drive's
own mountpoint -- but RoleForStorage types a local-dir on a non-system device as
user-data, so the existing eject role gate PASSED it. POST /disks/eject on
/mnt/nvme-1tb (demo-hp) or /mnt/hdd_1 (demo-felhom) would have SUCCEEDED
silently, taking the only local whole-guest backup with it, with no alarm and
the box still reporting a configured tier. Found by E-2 Phase 0, not by a
failure.

handleDiskEject and handleDiskDecommission now call refuseIfBackupTarget AFTER
the role gate and refuse with 409, naming the storage and the remedy -- the op
is ordered, not forbidden: reassign the target first.

NOT a role reclassification, which is the obvious fix and the wrong one: making
RoleForStorage return RoleBackup would refuse every legitimate eject of the
customer's own data drive, because on both demo boxes that drive IS the target.
That trades a silent failure for a permanent obstruction.

backupTargetAt resolves through the agent's own storage view, never the caller's
claim, and fails OPEN -- safe because it sits behind the role gate, which fails
SAFE on the same error.

Red-proofed both ways, mutations verified to land first:
  - removing the eject guard  -> "eject of the backup-target drive SUCCEEDED (200)"
  - the over-correction (any backup-content dir storage is the target)
    -> the gate blocks /mnt/spare, failing TestEjectStillAllowedOnANonTargetDrive

Harness note: normalizeBackupTiers DROPS tiers with a nil Service and falls back
to the legacy empty-TargetID tier -- an earlier version of this test exercised
nothing and reported the production bug as if the fix had failed.

Green gate: build + vet + test rc=0 (29 packages), run separately from this commit.
This commit is contained in:
2026-07-29 08:17:14 +02:00
parent d5c769173b
commit 38176ada9d
3 changed files with 209 additions and 0 deletions
@@ -0,0 +1,108 @@
package localapi
import (
"context"
"io"
"log/slog"
"net/http"
"strings"
"testing"
"time"
"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
)
// backupTargetServer builds a server whose PRIMARY tier is `felhom-backup`, mounted at /mnt/nvme-1tb
// on its own non-system device — i.e. the exact live shape E-1 created on demo-hp and demo-felhom:
// the drive is simultaneously the enrolled user-data drive AND the whole-guest vzdump target.
func backupTargetServer(t *testing.T) http.Handler {
t.Helper()
sv := fakeStorage{targets: []hub.StorageTarget{
{
Name: "felhom-backup", Type: hub.StorageTypeLocalDir, State: hub.StorageStateAttached,
Reachable: true, MountPath: "/mnt/nvme-1tb", BackingDevice: "/dev/nvme0n1",
Content: "backup",
},
{
Name: "spare-drive", Type: hub.StorageTypeLocalDir, State: hub.StorageStateAttached,
Reachable: true, MountPath: "/mnt/spare", BackingDevice: "/dev/sdz1",
Content: "backup",
},
}}
srv, err := NewServer(Options{
ListenAddr: "127.0.0.1:0",
Guests: &fakeGuests{}, Backups: &fakeBackups{}, Store: &fakeStore{}, Storage: sv,
Tokens: staticTokens{"A": 8200},
Disks: &fakeDiskOps{}, DiskGate: &fakeGate{}, HostReader: sysOnSDA(),
// Service is load-bearing: normalizeBackupTiers DROPS any tier with a nil Service and falls
// back to the legacy single tier with an empty TargetID — which silently made an earlier
// version of this test exercise nothing.
BackupTiers: []BackupTier{
{TargetID: "felhom-backup", Cadence: 24 * time.Hour, Primary: true, Service: &fakeBackups{}},
{TargetID: "felhom-pbs", Cadence: 168 * time.Hour, Service: &fakeBackups{}},
},
Logger: slog.New(slog.NewTextHandler(io.Discard, nil)),
})
if err != nil {
t.Fatalf("new server: %v", err)
}
srv.baseCtx = context.Background()
srv.now = func() time.Time { return testNow }
return srv.Handler()
}
// E-2c — ejecting the drive that holds the only local whole-guest backup must be refused.
//
// This is a REGRESSION GUARD on a live configuration, not a hypothetical. E-1 (2026-07-28) moved the
// vzdump target onto each demo box's secondary drive, and `RoleForStorage` types a local-dir on a
// non-system device as user-data — so the pre-existing role gate PASSES it and the customer could
// self-serve eject the drive holding their backups. It would have succeeded silently.
//
// The assertion is on the CONSEQUENCE (the request is refused) plus the remedy being named, because a
// refusal the customer cannot act on just moves the failure.
func TestEjectRefusedOnTheBackupTargetDrive(t *testing.T) {
h := backupTargetServer(t)
rr := do(t, h, http.MethodPost, "/disks/eject", "A", `{"vmid":8200,"where":"/mnt/nvme-1tb"}`)
if rr.Code == http.StatusOK {
t.Fatalf("eject of the backup-target drive SUCCEEDED (%d) — the box would silently lose its "+
"local drive-loss protection with nothing alarming", rr.Code)
}
body := rr.Body.String()
if !strings.Contains(body, "felhom-backup") {
t.Errorf("refusal must NAME the backup target so the customer knows which role blocks it; got: %s", body)
}
if !strings.Contains(strings.ToLower(body), "reassign") {
t.Errorf("refusal must name the REMEDY (reassign the target first), else it is a dead end; got: %s", body)
}
}
// Decommission strands the target just as thoroughly as eject — it migrates data off and retires the
// drive. Same gate, asserted separately because it is a different handler and a different caller.
func TestDecommissionRefusedOnTheBackupTargetDrive(t *testing.T) {
h := backupTargetServer(t)
rr := do(t, h, http.MethodPost, "/disks/decommission", "A", `{"vmid":8200,"where":"/mnt/nvme-1tb"}`)
if rr.Code == http.StatusOK {
t.Fatalf("decommission of the backup-target drive SUCCEEDED (%d)", rr.Code)
}
if !strings.Contains(rr.Body.String(), "felhom-backup") {
t.Errorf("refusal must name the backup target; got: %s", rr.Body.String())
}
}
// THE OVER-CORRECTION GUARD, and the reason this is a narrow gate instead of a role reclassification.
//
// The tempting fix — make RoleForStorage return RoleBackup for the target — would also refuse every
// OTHER user-data drive op on a box, and on the demo boxes it would refuse the customer's own data
// drive, because that drive IS the target. This pins that a non-target drive stays ejectable: the new
// gate must block exactly one drive, not harden the whole eject path.
//
// It asserts "not blocked BY THIS GATE" rather than "succeeds", because eject has other legitimate
// failure modes in a fake harness; what must never appear is this gate's message.
func TestEjectStillAllowedOnANonTargetDrive(t *testing.T) {
h := backupTargetServer(t)
rr := do(t, h, http.MethodPost, "/disks/eject", "A", `{"vmid":8200,"where":"/mnt/spare"}`)
if strings.Contains(rr.Body.String(), "whole-guest backup target") {
t.Fatalf("the backup-target gate blocked a NON-target drive (/mnt/spare) — over-correction: "+
"it must block exactly the target, not harden the whole eject path; got: %s", rr.Body.String())
}
}