restore test takes only this box's archives (R-727): an archive encrypted with another key is another box's

Red-proof RP39. Released as v0.138.0 by release-agent.sh.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS

CHANGELOG.md

@@ -1,3 +1,19 @@
+## v0.137.0 — a guest outside the agent's ACL is not a known guest (2026-09-27, R-689, v0.136.0 regression)
+
+> **RELEASED 2026-09-27** by `scripts/release-agent.sh` — tag `v0.137.0` (`3ef095f`), sha256 `766c9166916a1bd3674b0dc69081f8a7619e770f1402d8ad7705b395937e7627`, verified by download. **NOT vouched** (the operator's act).
+>
+> **VOUCHED 2026-09-28** with golden 0.276.0 (`min_agent` 0.131.0), on the operator's word of 2026-09-27: the hub logged
+> `Artifact manifest set: agent=0.137.0 golden=0.276.0 min_agent="0.131.0"`, and a Day-0 test install fetched this binary
+> through the manifest and sha-verified it. Evidence: `felhom.eu/documentation/audits/evidence-golden-0276-2026-09-28/`.
+
+**MinAgent impact:** none required by any controller.
+
+- v0.136.0 asked `GuestConfig` whether an archive's guest exists and treated anything but "does not exist" as a lookup
+  failure. PVE answers **403 "permission denied at /vms/<id>"** for a vmid outside the token's pool — so on demo-hp the
+  deleted guest 9100's archive made the local tier UNKNOWN every evaluation. A guest the agent cannot read is not one it
+  manages; its archive is skipped. `TestR689_AGuestOutsideTheAgentsACLIsNotAKnownGuest`, red-proofed. Verified read-only
+  on demo-hp with the pre-release binary before the release.
+
 ## v0.136.0 — … of a guest that still EXISTS (2026-09-27, R-689 second half)
 
 > **RELEASED 2026-09-27** by `scripts/release-agent.sh` — tag `v0.136.0` (`16dbc83`), sha256 `2eb0b5ebe253defd68b322312bbac12418c051b0a7a0d2d1831310d97fa6d755`, verified by download. **NOT vouched** (the operator's act).

REPORT.md

@@ -1,24 +1,15 @@
-# REPORT — agent v0.135.0: the restore test proves only backups of a guest (R-689, 2026-09-27)
+# REPORT — 2026-09-28: agent v0.137.0 vouched with golden 0.276.0
 
-**Baseline:** `main` `7403c2a838db`, v0.134.0 on both demo hosts. **Commits:** `d4be12c` (fix + tests), `9ff937d`
-(CHANGELOG, after the release). **Released** by `scripts/release-agent.sh`: tag `v0.135.0`, sha256
-`ad4e75f16d338552f4588d3fe64c51cbf9651220b9d223b5386b85b6c37fd4c3`, verified by download. **NOT vouched** (operator).
+No agent code changed. `configs/build-golden.sh` (v3.0.0) was USED, not changed, to bake golden 0.276.0 in the
+drill VM on DooPlex (RUNBOOK-manual-build §4.0–§4.1).
 
-**Tests:** full suite rc=0; agent gates OK after the release. `TestR689_TheRestoreTestNeverPicksTheGolden` red-proofed
-(without the check it picks `local:backup/felhom-golden-0.236.0.tar.zst`); `TestR689_GuestBackupArchiveShapes`; two older
-picker fixtures moved to real archive names.
+- **Vouched** in the hub's own form: `golden_version` 0.276.0, `agent_version` 0.137.0, `min_agent` 0.131.0 (the
+  MinAgent in controller v0.276.0's CHANGELOG header). The R-120 gate refused golden 0.258.0 first (negative
+  control). Rollback = the values before: agent 0.132.0, golden 0.258.0, min_agent 0.131.0.
+- **One sha, three places:** `766c9166…7627` in Gitea, on felhom-pve and on demo-hp.
+- **A new box comes up on it:** a Day-0 install (installer 1.28.0, `--force-gitea-golden`) in the drill VM fetched
+  agent 0.137.0 and golden 0.276.0 through the manifest, both sha-verified, and the controller answered healthy at
+  0.276.0 behind an armed claim gate. Not claimed.
+- CHANGELOG: the v0.137.0 entry carries the vouch.
 
-**Delivered** by signed `agent_update` (felhom-opsign, key `felhom-op-1`): demo-felhom committed 12:22:53 CEST, demo-hp
-12:29:02 CEST (`felhom.eu/documentation/audits/version-travel-2026-09-26/D1/`).
-
-**Then v0.136.0 (`16dbc83`, sha256 `2eb0b5eb…fa6d755`, NOT vouched):** the scheduler's read-only verdict on demo-hp
-(`-selftest=restore-test-due`, `D1/D1-selftest-due-demo-hp.txt`) skipped the golden but picked a leftover archive of
-guest 9100 (deleted in August). The guest must now exist; red-proof `D1/RP-r689-deleted-guest.txt`. Signed-delivered to
-both hosts. The verdict after 0.136.0: `D1/D1-selftest-due-after-0136.txt`.
-
-**Then v0.137.0 (`3ef095f`):** 0.136.0's lookup met PVE's 403 "permission denied" (the token sees only its pool), not
-"does not exist", so the local tier read UNKNOWN every evaluation (`D1/D1-selftest-due-after-0136.txt`) — a regression of
-0.136.0, fixed: a guest the agent cannot read is not one it manages. Red-proof `D1/RP-r689-acl.txt`; verified read-only on
-demo-hp with the pre-release binary before the release (`D1/D1-selftest-due-0137-pre.txt`): both leftovers skipped, the
-off-site tier due on 9201, the local tier waiting for today's 9201 archive to settle. Three agent releases in one session
-— each the smallest fix of what the box showed.
+Evidence: `felhom.eu/documentation/audits/evidence-golden-0276-2026-09-28/`.

internal/backup/r727_own_archives_test.go

@@ -0,0 +1,74 @@
+package backup
+
+import (
+	"context"
+	"errors"
+	"testing"
+	"time"
+
+	"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
+)
+
+const (
+	thisBoxKey  = "de:51:7a:18:cb:39:22:30:2c:84:f5:8b:d1:91:4b:7e:81:bb:69:b8:89:0f:57:ac:d3:59:e1:1a:62:25:11:2c"
+	earlierBox1 = "6b:ca:5f:3f:ca:0f:e2:3f:fb:24:62:89:bf:e7:64:59:9a:41:c5:e6:e3:9f:3f:5f:e1:71:7b:a1:9d:24:67:82"
+	earlierBox2 = "fe:3d:db:95:d4:df:ab:e1:7d:4a:89:fa:2b:07:53:6a:e4:d2:85:95:d1:90:27:4b:d9:c6:92:20:95:04:e5:d4"
+)
+
+// R-727 (v0.138.0) — the 2026-09-30 shape, measured on a fresh box for a returning customer: the PBS
+// namespace held two archives of earlier boxes (same guest 9201, same token) and this box's own, which was not
+// settled yet. The old picker chose the earlier box's newest settled archive and failed `wrong key`.
+// The CONSEQUENCE asserted: no archive of another box is ever picked; with this box's archive settled it is picked.
+// COMPANION RED-PROOF: remove the `ownKey != "" && !EqualFold(...)` skip → the first case picks 2026-09-16T21:59:54Z.
+func TestR727_TheRestoreTestTakesOnlyThisBoxsArchives(t *testing.T) {
+	day := int64(86400)
+	now := int64(1790740000) // 2026-09-30 ~04:00Z
+	own := proxmox.StorageContent{VolID: "felhom-pbs:backup/ct/9201/2026-09-29T19:37:07Z", Content: "backup", VMID: 9201, Size: 3490689830, CTime: 1790710627, Encrypted: thisBoxKey}
+	api := &fakeBackupAPI{
+		storages: []proxmox.Storage{{Storage: "felhom-pbs", Type: "pbs", EncryptionKey: thisBoxKey}},
+		content: []proxmox.StorageContent{
+			{VolID: "felhom-pbs:backup/ct/9201/2026-09-16T17:27:32Z", Content: "backup", VMID: 9201, Size: 4774114206, CTime: 1789579652, Encrypted: earlierBox2},
+			{VolID: "felhom-pbs:backup/ct/9201/2026-09-16T21:59:54Z", Content: "backup", VMID: 9201, Size: 20811501236, CTime: 1789595994, Encrypted: earlierBox1},
+			own,
+		},
+	}
+	r := NewBackupRunner(api, "local", proxmox.ModeSnapshot, "", "keep-last=1", quiet())
+
+	// 1. The night of 2026-09-30: this box's own archive is ~6 h old, not settled (cutoff 24 h) — nothing to prove.
+	got, _, err := r.PickSettledRestoreCandidateOn(context.Background(), "felhom-pbs", time.Unix(now-day, 0).UTC())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got != "" {
+		t.Fatalf("picked %q — an archive of ANOTHER box is never this box's proof (R-727)", got)
+	}
+	// 2. A day later this box's own archive is settled — it is the one picked.
+	got, _, err = r.PickSettledRestoreCandidateOn(context.Background(), "felhom-pbs", time.Unix(now+day, 0).UTC())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got != own.VolID {
+		t.Fatalf("picked %q, want this box's own %q", got, own.VolID)
+	}
+}
+
+// An unencrypted storage (a local dir) holds only this box's vzdumps — no key filter applies.
+func TestR727_UnencryptedStorageIsNotFiltered(t *testing.T) {
+	api := &fakeBackupAPI{
+		storages: []proxmox.Storage{{Storage: "local", Type: "dir"}},
+		content:  []proxmox.StorageContent{{VolID: "local:backup/vzdump-lxc-9201-2026_09_29-21_27_05.tar.zst", Content: "backup", VMID: 9201, Size: 955425507, CTime: 1790710025}},
+	}
+	r := NewBackupRunner(api, "local", proxmox.ModeSnapshot, "", "keep-last=1", quiet())
+	if got, _, err := r.PickSettledRestoreCandidateOn(context.Background(), "local", time.Time{}); err != nil || got == "" {
+		t.Fatalf("got %q err %v", got, err)
+	}
+}
+
+// A storage-list failure makes the tier UNKNOWN (an error), never "nothing to prove".
+func TestR727_KeyLookupFailureIsUnknown(t *testing.T) {
+	api := &fakeBackupAPI{storageErr: errors.New("proxmox: GET /storage -> HTTP 500"), content: []proxmox.StorageContent{{VolID: "felhom-pbs:backup/ct/9201/x", Content: "backup", VMID: 9201}}}
+	r := NewBackupRunner(api, "local", proxmox.ModeSnapshot, "", "keep-last=1", quiet())
+	if _, _, err := r.PickSettledRestoreCandidateOn(context.Background(), "felhom-pbs", time.Time{}); err == nil {
+		t.Fatal("a failed key lookup must surface as an error (tier UNKNOWN)")
+	}
+}

internal/backup/runner.go

@@ -357,6 +357,16 @@ func (r *BackupRunner) PickSettledRestoreCandidateOn(ctx context.Context, target
 	if err != nil {
 		return "", time.Time{}, err
 	}
+	// R-727 (v0.138.0): on an ENCRYPTED storage, only archives written with THIS storage's key are this box's.
+	// Measured 2026-09-30 on a fresh box for a returning customer: the PBS namespace still held two archives
+	// of earlier boxes (same guest id 9201, same token), the newest settled one was an earlier box's, and the
+	// test failed `wrong key` every evaluation. The archive carries no host id; its key fingerprint is the
+	// discriminator (PVE's content `encrypted`, the storage's `encryption-key`). A lookup failure returns
+	// the error — the tier reads UNKNOWN, never "nothing to prove".
+	ownKey, err := r.storageKeyFingerprint(ctx, target)
+	if err != nil {
+		return "", time.Time{}, fmt.Errorf("reading the key fingerprint of storage %s: %w", target, err)
+	}
 	var best string
 	var bestCTime int64 = -1
 	known := map[int]bool{} // vmid → the guest exists on this node (asked once per vmid per pick)
@@ -372,6 +382,10 @@ func (r *BackupRunner) PickSettledRestoreCandidateOn(ctx context.Context, target
 			r.noteNotAGuestBackupOnce(e, why)
 			continue
 		}
+		if ownKey != "" && !strings.EqualFold(e.Encrypted, ownKey) {
+			r.noteNotAGuestBackupOnce(e, fmt.Sprintf("written by another box (key %s, this box's key %s) — not this box's proof", shortFP(e.Encrypted), shortFP(ownKey)))
+			continue
+		}
 		// R-689 (v0.136.0): … OF A GUEST THAT STILL EXISTS here. Measured on demo-hp 2026-09-27 right after
 		// v0.135.0: with the golden skipped, the pick fell to `vzdump-lxc-9100-2026_08_21…`, a leftover of a
 		// guest deleted in August — proving nothing about any guest this box runs. "Does not exist" skips the
@@ -667,3 +681,29 @@ func (r *BackupRunner) noteNotAGuestBackupOnce(e proxmox.StorageContent, why str
 			"target", r.target, "volid", e.VolID, "size_bytes", e.Size, "reason", why)
 	}
 }
+
+// storageKeyFingerprint returns the named storage's client-side encryption key fingerprint ("" when the
+// storage is not encrypted — a local dir holds only this box's own vzdumps).
+func (r *BackupRunner) storageKeyFingerprint(ctx context.Context, target string) (string, error) {
+	sts, err := r.api.ListStorage(ctx)
+	if err != nil {
+		return "", err
+	}
+	for _, st := range sts {
+		if st.Storage == target {
+			return strings.TrimSpace(st.EncryptionKey), nil
+		}
+	}
+	return "", nil
+}
+
+// shortFP is the first 8 bytes of a key fingerprint, for a log line.
+func shortFP(fp string) string {
+	if fp == "" {
+		return "none"
+	}
+	if len(fp) > 23 {
+		return fp[:23] + "…"
+	}
+	return fp
+}

internal/proxmox/types.go

@@ -219,15 +219,18 @@ type Storage struct {
 	UsedFraction float64 `json:"used_fraction,omitempty"`
 
 	// Type-specific config (durable_id sources).
-	Server      string `json:"server,omitempty"`      // nfs/cifs/pbs server host
-	Export      string `json:"export,omitempty"`      // nfs export path
-	Share       string `json:"share,omitempty"`       // cifs share name
-	Datastore   string `json:"datastore,omitempty"`   // pbs datastore name
-	Fingerprint string `json:"fingerprint,omitempty"` // pbs server cert fingerprint
-	Username    string `json:"username,omitempty"`    // pbs auth id, e.g. "felhom@pbs!n100"
-	Namespace   string `json:"namespace,omitempty"`   // pbs namespace ("" = root; per-customer tenancy = S4)
-	VGName      string `json:"vgname,omitempty"`      // lvm/lvmthin volume group
-	ThinPool    string `json:"thinpool,omitempty"`    // lvmthin pool LV name
+	Server string `json:"server,omitempty"` // nfs/cifs/pbs server host
+	// EncryptionKey is the storage's own client-side key FINGERPRINT (pbs; the key itself stays in
+	// /etc/pve/priv). R-727: an archive encrypted with any other key was written by another box.
+	EncryptionKey string `json:"encryption-key,omitempty"`
+	Export        string `json:"export,omitempty"`      // nfs export path
+	Share         string `json:"share,omitempty"`       // cifs share name
+	Datastore     string `json:"datastore,omitempty"`   // pbs datastore name
+	Fingerprint   string `json:"fingerprint,omitempty"` // pbs server cert fingerprint
+	Username      string `json:"username,omitempty"`    // pbs auth id, e.g. "felhom@pbs!n100"
+	Namespace     string `json:"namespace,omitempty"`   // pbs namespace ("" = root; per-customer tenancy = S4)
+	VGName        string `json:"vgname,omitempty"`      // lvm/lvmthin volume group
+	ThinPool      string `json:"thinpool,omitempty"`    // lvmthin pool LV name
 }
 
 // StorageContent is one entry of GET /nodes/{node}/storage/{store}/content
@@ -239,4 +242,7 @@ type StorageContent struct {
 	Size    int64  `json:"size"`
 	CTime   int64  `json:"ctime"`
 	VMID    int    `json:"vmid,omitempty"`
+	// Encrypted is the fingerprint of the key a PBS archive was encrypted with ("" = not encrypted). R-727:
+	// the restore test reads it to tell THIS box's archives from an earlier box's in the same namespace.
+	Encrypted string `json:"encrypted,omitempty"`
 }