Files
felhom-agent/internal/localapi/disks_backup_target_row_test.go
T
admin 21b0164fad R-116 (v0.116.0): give the backup-target flag and the gate's key the same row
The absent-drive alarm was generic while its recovery was specific -- a pair an
operator cannot match. Mechanism now measured, not reasoned (felhom.eu
audits/DIAG-r116-disks-payload-2026-07-30.md): with the device gone /disks returns
4 rows, not 3. The drive appears TWICE and the two facts the controller needs are
on different rows -- the Observe row has backup_target:true but mount_path:"" and
guest_path:"" (so driveTargetByPath registers NO key from it), while the registry
row owns /mnt/felhom-drives/<name>, the key the gate looks up, with BackupTarget
absent from its struct literal => false.

WHY v0.115.0 WAS INERT: its fallback computed StablePathForRaw(t.MountPath), and in
the absent state MountPath is ALSO "" -- emptied by the same exactMount failure
that empties BackingDevice. It assigned nothing. Its test passed because the fixture
supplied a MountPath production never supplies, and the harness left DriveTargets
nil so the union loop never ran. Both corrected here; red-proof 1 replays v0.115.0's
exact code against the real shape and it fails.

THE JOIN, which was the hard part: with the device gone the two records share no
runtime field -- no mount, no backing device, and the Observe row's DurableID has
degraded off the fs-UUID. They share CONFIGURATION: storage.cfg's path on one side,
the .mount unit's Where on the other, both yielding the same stable guest path. New
hub.StorageTarget.ConfigPath (json:"-" -- that struct is a cross-repo contract
pinned by the golden + contract_test key-set comparison, and nothing off-box needs
the value), set from s.Path in observe.go, consulted in disks.go only after MountPath
so the present-state path is byte-identical, plus a guest-path arm on the union dedup
so exactly one row carries the drive.

WHY NEITHER OBVIOUS OPTION WAS TAKEN -- both regress R-114, which shipped yesterday.
backup_target_offer.go:79 reads (BackupTarget && MountPath != "") as "a real
drive with its own mountpoint -- healthy" and returns before its TargetAbsent
branch. Back-filling MountPath onto the Observe row (the smallest change, and the
spec's lean) and teaching the registry row the flag (its MountPath is non-empty, read
from the stale unit file) BOTH manufacture that row while the drive is missing, which
would have told the customer the backup target is fine while its drive is gone.
R-114's correctness rests on the absent-state rows not combining the flag with a
mount path; that coupling was invisible until the payload existed. Pinned by
TestAbsentTargetKeepsR114DegradedSignal.

Role unchanged, BoundUnderParent conjunction not widened, no wire field changed.
Suppressing the registry row in the absent state also removes its false
state:"attached" and its root-filesystem-derived total_bytes -- R-118's symptom
goes incidentally; R-118 is NOT fixed and stays open.

Tests 845 -> 849, suite rc=0 read separately from this commit. Four red-proofs, each
mutation asserted to have landed first.

NOT live-validated at this commit: publish+vouch, C5, discrimination, over-correction.
2026-07-30 08:48:31 +02:00

343 lines
17 KiB
Go

package localapi
import (
"context"
"encoding/json"
"io"
"log/slog"
"testing"
"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
"gitea.dooplex.hu/admin/felhom-agent/internal/storage"
)
// R-116 — the backup-target flag must be reachable from the row the CONTROLLER keys on.
//
// THE DEFECT. The controller resolves the drive-absent alarm by the registered StoragePath, which for
// an external drive is the GUEST path. When the device vanishes, Observe's exactMountDevice fails, so
// BackingDevice becomes "" and RoleForStorage returns RoleSystem (role.go:180-181) — the guest-path
// block is skipped and the flag-bearing row loses its guest path. It keeps its MountPath, so the union
// loop DEDUPES the registry row away, and /disks carries NO row with that guest path at all.
// driveTargetByPath then has no entry, isTarget[guestPath] is a MISSING KEY, and the specific
// backup_target_absent alarm cannot fire — the generic storage_disconnected goes out instead, while
// the RETURN (rows rejoined) fires the specific recovery. An operator gets a pair they cannot match.
// Measured live: felhom.eu audits/SESSION-C-2026-07-29.md §5.
//
// These tests exercise the REAL GET /disks response and assert the emitted JSON, because the failure
// class is "the value is on the wrong row" — a test that hand-builds rows proves nothing about which
// row the handler actually emits.
// targetRowServer builds a /disks server whose primary backup tier is `primaryTarget`, over the given
// Observe targets. boundCheck/deviceCheck are pinned so the R-113 conjunction is not the variable
// under test here.
func targetRowServer(t *testing.T, primaryTarget string, targets []hub.StorageTarget) *Server {
t.Helper()
return targetRowServerWithDrives(t, primaryTarget, targets, nil)
}
// targetRowServerWithDrives additionally wires the REGISTRY union source. v0.115.0's tests left
// DriveTargets nil, so the union loop never ran and the two-row absent shape — the actual defect — was
// invisible to the whole suite. Any test about which row carries what MUST populate this.
func targetRowServerWithDrives(t *testing.T, primaryTarget string, targets []hub.StorageTarget,
drives []storage.KnownTarget) *Server {
t.Helper()
var known storage.KnownTargets
if drives != nil {
known = fakeKnownTargets{drives: drives}
}
srv, err := NewServer(Options{
DriveTargets: known,
ListenAddr: "127.0.0.1:0",
Guests: &fakeGuestsCfg{}, Backups: &fakeBackups{}, Store: &fakeStore{},
Storage: fakeStorage{targets: targets},
// Service is REQUIRED: normalizeBackupTiers (backup_tiers.go:21-22) drops any tier with a nil
// Service, and the legacy fallback then yields TargetID "" — which silently makes every
// BackupTarget false and would make these tests pass for the wrong reason.
BackupTiers: []BackupTier{{TargetID: primaryTarget, Primary: true, Service: &fakeBackups{}}},
Tokens: staticTokens{"A": 8200},
Disks: &fakeDiskOps{probe: storage.DeviceProbe{Probed: true, HasFilesystem: true, FSType: "ext4"}},
DiskGate: &fakeGate{}, HostReader: sysOnSDA(),
Logger: slog.New(slog.NewTextHandler(io.Discard, nil)),
})
if err != nil {
t.Fatal(err)
}
srv.baseCtx = context.Background()
srv.boundCheck = func(string) bool { return true }
srv.deviceCheck = func(string) bool { return true }
return srv
}
// wireDisks returns the decoded /disks rows exactly as the controller receives them.
func wireDisks(t *testing.T, srv *Server) []map[string]any {
t.Helper()
body := do(t, srv.Handler(), "GET", "/disks", "A", "").Body.Bytes()
var w struct {
Data struct {
Disks []map[string]any `json:"disks"`
} `json:"data"`
}
if err := json.Unmarshal(body, &w); err != nil {
t.Fatalf("decode /disks: %v (%s)", err, body)
}
return w.Data.Disks
}
// isTargetByPath reproduces the controller's driveTargetByPath EXACTLY (intermediary.go:602-616):
// both keyings, value = backup_target. This is the map whose missing key is the whole defect, so the
// assertion is made against a faithful copy of it rather than against a field in isolation.
func isTargetByPath(disks []map[string]any) map[string]bool {
out := map[string]bool{}
for _, d := range disks {
bt, _ := d["backup_target"].(bool)
if gp, ok := d["guest_path"].(string); ok && gp != "" {
out[gp] = bt
}
if mp, ok := d["mount_path"].(string); ok && mp != "" {
out[mp] = bt
}
}
return out
}
// theAbsentTarget is the absent-target Observe row, CORRECTED in v0.116.0 to the shape the live box
// actually produces.
//
// THIS FIXTURE IS WHY AN INERT FIX SHIPPED GREEN. As written for v0.115.0 it supplied
// `MountPath: "/mnt/mentes"` — a field the real absent state does NOT have. The same exactMount failure
// that empties BackingDevice empties MountPath (observe.go:184-190), so on the live box this row carries
// `mount_path: ""`, and v0.115.0's `StablePathForRaw(t.MountPath)` was therefore
// `StablePathForRaw("")` == "". The fixture handed the code a value production never supplies, the test
// went green, and the fix was inert on real hardware — twice.
//
// Captured payload this now mirrors, field for field:
// felhom.eu audits/DIAG-r116-disks-payload-2026-07-30.md §6.2.
var theAbsentTarget = hub.StorageTarget{
Name: "felhom-backup", Type: hub.StorageTypeLocalDir,
MountPath: "", BackingDevice: "", ConfigPath: "/mnt/mentes",
State: hub.StorageStateDisconnected,
// DurableID degrades off the fs-UUID exactly as the live payload showed (`path:/mnt/cel` there).
DurableID: "path:/mnt/mentes",
}
// theAbsentRegistryRow is the OTHER half of the live absent payload — the registry/union row. Its
// MountPath comes from the systemd .mount unit FILE (registry_known.go:40-75), which never consults the
// mount table, so it survives the device intact. Its presence is what made /disks carry the drive TWICE.
var theAbsentRegistryRow = []storage.KnownTarget{
{Name: "9303-uuid", Type: hub.StorageTypeUSB, MountPath: "/mnt/mentes",
DurableID: "uuid:9303", UUID: "9303"},
}
// ── the observable that must move ───────────────────────────────────────────────────────────────
// RED-PROOF: delete the `di.GuestPath == "" && di.BackupTarget && t.BackingDevice == ""` block and
// this fails with "the guest path the controller keys on is MISSING from /disks entirely".
func TestAbsentBackupTargetIsResolvableByGuestPath(t *testing.T) {
disks := wireDisks(t, targetRowServerWithDrives(t, "felhom-backup",
[]hub.StorageTarget{theAbsentTarget}, theAbsentRegistryRow))
isTarget := isTargetByPath(disks)
const guestPath = "/mnt/felhom-drives/mentes"
got, present := isTarget[guestPath]
if !present {
t.Fatalf("isTarget[%q] is a MISSING KEY — the guest path the controller keys on is missing from "+
"/disks entirely, so notifyDriveAbsent takes the generic branch and backup_target_absent "+
"can never fire (R-116)", guestPath)
}
if !got {
t.Fatalf("isTarget[%q] = FALSE. Both rows for this drive reached the wire and the registry row — "+
"appended last, BackupTarget defaulted false — overwrote the flag-bearing row's true. This is "+
"the measured live defect, not a hypothetical: rows=%d", guestPath, len(disks))
}
}
// ── V2: the new guest path must NOT make the gate read the drive as PRESENT ─────────────────────
// This is the over-correction guard, in the exact component under test. planDriveGates computes
// present[gp] = present[gp] || d.BoundUnderParent. If the row we now emit carried a true
// BoundUnderParent, this fix would SILENCE the alarm it exists to raise.
func TestAbsentTargetRowDoesNotRegisterPresence(t *testing.T) {
srv := targetRowServer(t, "felhom-backup", []hub.StorageTarget{theAbsentTarget})
// deviceCheck/boundCheck are pinned TRUE — the strongest possible case for a false positive.
// The row must still report bound_under_parent=false, because that field is only ever assigned
// inside the guest-path blocks a system-role row does not enter.
for _, d := range wireDisks(t, srv) {
if d["guest_path"] != "/mnt/felhom-drives/mentes" {
continue
}
if bup, _ := d["bound_under_parent"].(bool); bup {
t.Fatal("the absent backup-target row reports bound_under_parent=true — planDriveGates " +
"would compute present=true, the Stop branch would never run, and this fix would " +
"SUPPRESS the very alarm it exists to raise")
}
return
}
t.Fatal("the absent target row never reached the wire")
}
// ── V1: the gates, each on its own ──────────────────────────────────────────────────────────────
// Case B is the COMMON fresh-box shape, not an edge: the tier target is the builtin `local` on the
// root fs. It must never acquire a guest path.
func TestCaseBLocalTargetGetsNoGuestPath(t *testing.T) {
disks := wireDisks(t, targetRowServer(t, "local", []hub.StorageTarget{
{Name: "local", Type: "local", MountPath: "/var/lib/vz", BackingDevice: "", State: hub.StorageStateAttached},
}))
for _, d := range disks {
if gp, _ := d["guest_path"].(string); gp != "" {
t.Errorf("the Case B target on %v acquired guest path %q — a system-drive backup target "+
"must not cross into the guest", d["mount_path"], gp)
}
}
}
// A storage that is RoleSystem because it is genuinely system-BACKED (non-empty BackingDevice on the
// system disk) must be excluded — this is the case StablePathForRaw would NOT have filtered, since
// /mnt/<name> maps to a real stable path. The BackingDevice gate is what stops it.
func TestSystemBackedTargetUnderMntGetsNoGuestPath(t *testing.T) {
disks := wireDisks(t, targetRowServer(t, "sysbackup", []hub.StorageTarget{
// sysOnSDA() makes /dev/sda the system disk, so this classifies RoleSystem with a REAL device.
{Name: "sysbackup", Type: hub.StorageTypeLocalDir, MountPath: "/mnt/sysbackup",
BackingDevice: "/dev/sda1", State: hub.StorageStateAttached},
}))
for _, d := range disks {
if gp, _ := d["guest_path"].(string); gp != "" {
t.Errorf("a system-BACKED backup target acquired guest path %q — the BackingDevice gate "+
"failed and the :213-214 boundary was widened", gp)
}
}
}
// ── the negative ────────────────────────────────────────────────────────────────────────────────
// A drive that is NOT the target must not acquire the flag on any row, present or absent.
func TestNonTargetDriveNeverCarriesTheFlag(t *testing.T) {
disks := wireDisks(t, targetRowServer(t, "felhom-backup", []hub.StorageTarget{
{Name: "adat", Type: hub.StorageTypeLocalDir, MountPath: "/mnt/adat",
BackingDevice: "", State: hub.StorageStateDisconnected},
}))
for _, d := range disks {
if bt, _ := d["backup_target"].(bool); bt {
t.Errorf("non-target drive %v reports backup_target=true", d["name"])
}
if gp, _ := d["guest_path"].(string); gp != "" {
t.Errorf("an absent NON-target drive acquired guest path %q via the R-116 fallback — the "+
"BackupTarget gate failed", gp)
}
}
}
// ── v0.116.0 — the join, and the regression it must not cause ───────────────────────────────────
// THE JOIN. With the device gone the two records of one drive share no runtime field, so the dedup has
// to key on the one thing both can still derive: the CONFIGURED path (storage.cfg's `path` on the
// Observe side, the .mount unit's `Where` on the registry side), expressed as the stable guest path.
// This pins that exactly one row survives — because driveTargetByPath ASSIGNS rather than ORs, so two
// rows disagreeing on the flag is decided by append order, which is not a contract anyone should rely on.
//
// RED-PROOF: delete the `seenGuest[gp]` skip in the union loop and this fails with rows=2.
func TestAbsentTargetAppearsExactlyOnce(t *testing.T) {
disks := wireDisks(t, targetRowServerWithDrives(t, "felhom-backup",
[]hub.StorageTarget{theAbsentTarget}, theAbsentRegistryRow))
const guestPath = "/mnt/felhom-drives/mentes"
var rows []map[string]any
for _, d := range disks {
if gp, _ := d["guest_path"].(string); gp == guestPath {
rows = append(rows, d)
}
}
if len(rows) != 1 {
t.Fatalf("the absent drive is carried by %d rows, want exactly 1 — with two rows the flag the "+
"controller reads is decided by append order, not by the fix. rows=%v", len(rows), rows)
}
if bt, _ := rows[0]["backup_target"].(bool); !bt {
t.Error("the surviving row does not carry backup_target=true")
}
}
// THE REGRESSION THIS FIX MUST NOT CAUSE, and the reason neither obvious option was taken.
//
// The controller reads `d.BackupTarget && d.MountPath != ""` as "a real drive with its own mountpoint —
// HEALTHY" and returns immediately (backup_target_offer.go:79). So the two candidate fixes that look
// smallest — back-filling MountPath onto the Observe row, or teaching the registry row the flag (its
// MountPath is non-empty, read from the stale unit file) — BOTH produce a row satisfying that predicate
// while the drive is missing. Either would have silently regressed R-114, which shipped 2026-07-29 and
// tells the customer „A rendszermentés meghajtója nem érhető el" in exactly this state, flipping it back
// to a false healthy.
//
// R-114's correctness currently rests on the absent-state rows NOT combining the flag with a mount path.
// That coupling was invisible until the payload was captured, and it is what this test pins.
//
// RED-PROOF: set `MountPath: "/mnt/mentes"` on theAbsentTarget (v0.115.0's fixture value) and this fails.
func TestAbsentTargetKeepsR114DegradedSignal(t *testing.T) {
disks := wireDisks(t, targetRowServerWithDrives(t, "felhom-backup",
[]hub.StorageTarget{theAbsentTarget}, theAbsentRegistryRow))
for _, d := range disks {
bt, _ := d["backup_target"].(bool)
mp, _ := d["mount_path"].(string)
if bt && mp != "" {
t.Fatalf("row %v carries backup_target=true AND mount_path=%q while the drive is ABSENT. "+
"resolveBackupTargetState (backup_target_offer.go:79) reads that as \"a real drive with "+
"its own mountpoint — healthy\" and returns before its TargetAbsent branch, so the "+
"customer is told the backup target is fine while its drive is gone. That is R-114, "+
"regressed.", d["name"], mp)
}
}
}
// PRESENT-STATE PARITY. The fix must change nothing when the drive is there. Present state is the
// state every healthy box is in, so a change here reaches the whole fleet; absent state reaches only a
// box with a problem. Both rows are supplied, exactly as on a live present box, and the pre-existing
// MountPath dedup must still collapse them to one COMPLETE row.
func TestPresentTargetPayloadUnchanged(t *testing.T) {
present := hub.StorageTarget{
Name: "felhom-backup", Type: hub.StorageTypeLocalDir,
MountPath: "/mnt/mentes", BackingDevice: "/dev/sdb", ConfigPath: "/mnt/mentes",
State: hub.StorageStateAttached, DurableID: "uuid:9303",
}
disks := wireDisks(t, targetRowServerWithDrives(t, "felhom-backup",
[]hub.StorageTarget{present}, theAbsentRegistryRow))
var rows []map[string]any
for _, d := range disks {
if d["name"] == "felhom-backup" || d["mount_path"] == "/mnt/mentes" {
rows = append(rows, d)
}
}
if len(rows) != 1 {
t.Fatalf("present state carries the drive on %d rows, want 1 (the MountPath dedup): %v", len(rows), rows)
}
r := rows[0]
for field, want := range map[string]any{
"mount_path": "/mnt/mentes", "guest_path": "/mnt/felhom-drives/mentes",
"backing_device": "/dev/sdb", "role": "user-data", "state": "attached",
"backup_target": true, "bound_under_parent": true, "durable_id": "uuid:9303",
} {
if got := r[field]; got != want {
t.Errorf("present-state %s = %v, want %v — the fix altered the healthy payload", field, got, want)
}
}
}
// The negative, with the union loop actually running: a non-target absent drive gains the flag on no row
// and keeps its own registry row (nothing to dedup against, since no Observe row claims its guest path).
func TestAbsentNonTargetKeepsItsRegistryRowAndNoFlag(t *testing.T) {
disks := wireDisks(t, targetRowServerWithDrives(t, "felhom-backup",
[]hub.StorageTarget{{Name: "adat", Type: hub.StorageTypeLocalDir, MountPath: "",
BackingDevice: "", ConfigPath: "/mnt/adat", State: hub.StorageStateDisconnected}},
[]storage.KnownTarget{{Name: "adat-uuid", Type: hub.StorageTypeUSB,
MountPath: "/mnt/adat", DurableID: "uuid:1111", UUID: "1111"}}))
isTarget := isTargetByPath(disks)
for k, v := range isTarget {
if v {
t.Errorf("isTarget[%q] = true for a NON-target drive — the BackupTarget gate failed", k)
}
}
if _, ok := isTarget["/mnt/felhom-drives/adat"]; !ok {
t.Error("the non-target drive lost its guest-path key entirely — the union row was over-suppressed")
}
}