Compare commits
7 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 475bdce7e4 | |||
| d766666ff8 | |||
| a4c09a7c11 | |||
| 904dc20466 | |||
| e1b8269be0 | |||
| 5c68c869b6 | |||
| 728d12b1a0 |
@@ -1,3 +1,42 @@
|
||||
## v0.138.0 — the restore test takes only THIS box's archives (2026-09-30, R-727, `09` §3 decision 51)
|
||||
|
||||
> **RELEASED 2026-09-30** by `scripts/release-agent.sh` — tag `v0.138.0` (`e1b8269`), sha256
|
||||
> `55916026001790a79ebf97d32c032610cfde8e09d02979b9b9d8c2cbc5d88195`, verified by download. **Not vouched** (the golden
|
||||
> keeps 0.137.0 until the next bake); delivered to the demo boxes by signed `agent_update` jobs.
|
||||
>
|
||||
> **VOUCHED 2026-09-30** with golden 0.283.1 (`min_agent` 0.131.0), on the operator's word: the hub logged
|
||||
> `Artifact manifest set: agent=0.138.0 golden=0.283.1 min_agent="0.131.0"`. Evidence:
|
||||
> `felhom.eu/documentation/audits/evidence-golden-0283-2026-09-30/`.
|
||||
|
||||
**MinAgent impact:** none required by any controller.
|
||||
|
||||
- A returning customer's PBS namespace can hold archives of EARLIER boxes: same guest id (9201), same token, written
|
||||
with a different key. Measured 2026-09-30: the newest SETTLED archive was an earlier box's, and the test failed
|
||||
`wrong key … manifest's key 6b:ca:5f:3f… does not match provided key de:51:7a:18…` every evaluation. The archive
|
||||
carries no host id; it carries its key fingerprint (PVE content `encrypted`), and the storage carries its own
|
||||
(`GET /storage` → `encryption-key`). `PickSettledRestoreCandidateOn` now skips — and logs by name, once — an archive
|
||||
whose fingerprint is not the storage's own; an unencrypted storage is not filtered; a failed storage read is an
|
||||
error (tier UNKNOWN), never "nothing to prove".
|
||||
- Tests: `TestR727_TheRestoreTestTakesOnlyThisBoxsArchives` (the 2026-09-30 shape: nothing picked while this box's
|
||||
archive settles, then exactly it), `TestR727_UnencryptedStorageIsNotFiltered`, `TestR727_KeyLookupFailureIsUnknown`.
|
||||
Red-proof RP39: the skip removed → the earlier box's `2026-09-16T21:59:54Z` is picked.
|
||||
|
||||
## v0.137.0 — a guest outside the agent's ACL is not a known guest (2026-09-27, R-689, v0.136.0 regression)
|
||||
|
||||
> **RELEASED 2026-09-27** by `scripts/release-agent.sh` — tag `v0.137.0` (`3ef095f`), sha256 `766c9166916a1bd3674b0dc69081f8a7619e770f1402d8ad7705b395937e7627`, verified by download. **NOT vouched** (the operator's act).
|
||||
>
|
||||
> **VOUCHED 2026-09-28** with golden 0.276.0 (`min_agent` 0.131.0), on the operator's word of 2026-09-27: the hub logged
|
||||
> `Artifact manifest set: agent=0.137.0 golden=0.276.0 min_agent="0.131.0"`, and a Day-0 test install fetched this binary
|
||||
> through the manifest and sha-verified it. Evidence: `felhom.eu/documentation/audits/evidence-golden-0276-2026-09-28/`.
|
||||
|
||||
**MinAgent impact:** none required by any controller.
|
||||
|
||||
- v0.136.0 asked `GuestConfig` whether an archive's guest exists and treated anything but "does not exist" as a lookup
|
||||
failure. PVE answers **403 "permission denied at /vms/<id>"** for a vmid outside the token's pool — so on demo-hp the
|
||||
deleted guest 9100's archive made the local tier UNKNOWN every evaluation. A guest the agent cannot read is not one it
|
||||
manages; its archive is skipped. `TestR689_AGuestOutsideTheAgentsACLIsNotAKnownGuest`, red-proofed. Verified read-only
|
||||
on demo-hp with the pre-release binary before the release.
|
||||
|
||||
## v0.136.0 — … of a guest that still EXISTS (2026-09-27, R-689 second half)
|
||||
|
||||
> **RELEASED 2026-09-27** by `scripts/release-agent.sh` — tag `v0.136.0` (`16dbc83`), sha256 `2eb0b5ebe253defd68b322312bbac12418c051b0a7a0d2d1831310d97fa6d755`, verified by download. **NOT vouched** (the operator's act).
|
||||
|
||||
@@ -1,24 +1,12 @@
|
||||
# REPORT — agent v0.135.0: the restore test proves only backups of a guest (R-689, 2026-09-27)
|
||||
# REPORT — 2026-09-30: v0.138.0 (R-727)
|
||||
|
||||
**Baseline:** `main` `7403c2a838db`, v0.134.0 on both demo hosts. **Commits:** `d4be12c` (fix + tests), `9ff937d`
|
||||
(CHANGELOG, after the release). **Released** by `scripts/release-agent.sh`: tag `v0.135.0`, sha256
|
||||
`ad4e75f16d338552f4588d3fe64c51cbf9651220b9d223b5386b85b6c37fd4c3`, verified by download. **NOT vouched** (operator).
|
||||
Full session report: `felhom.eu/REPORT-fixes-first-tester-2026-09-30.md`.
|
||||
|
||||
**Tests:** full suite rc=0; agent gates OK after the release. `TestR689_TheRestoreTestNeverPicksTheGolden` red-proofed
|
||||
(without the check it picks `local:backup/felhom-golden-0.236.0.tar.zst`); `TestR689_GuestBackupArchiveShapes`; two older
|
||||
picker fixtures moved to real archive names.
|
||||
|
||||
**Delivered** by signed `agent_update` (felhom-opsign, key `felhom-op-1`): demo-felhom committed 12:22:53 CEST, demo-hp
|
||||
12:29:02 CEST (`felhom.eu/documentation/audits/version-travel-2026-09-26/D1/`).
|
||||
|
||||
**Then v0.136.0 (`16dbc83`, sha256 `2eb0b5eb…fa6d755`, NOT vouched):** the scheduler's read-only verdict on demo-hp
|
||||
(`-selftest=restore-test-due`, `D1/D1-selftest-due-demo-hp.txt`) skipped the golden but picked a leftover archive of
|
||||
guest 9100 (deleted in August). The guest must now exist; red-proof `D1/RP-r689-deleted-guest.txt`. Signed-delivered to
|
||||
both hosts. The verdict after 0.136.0: `D1/D1-selftest-due-after-0136.txt`.
|
||||
|
||||
**Then v0.137.0 (`3ef095f`):** 0.136.0's lookup met PVE's 403 "permission denied" (the token sees only its pool), not
|
||||
"does not exist", so the local tier read UNKNOWN every evaluation (`D1/D1-selftest-due-after-0136.txt`) — a regression of
|
||||
0.136.0, fixed: a guest the agent cannot read is not one it manages. Red-proof `D1/RP-r689-acl.txt`; verified read-only on
|
||||
demo-hp with the pre-release binary before the release (`D1/D1-selftest-due-0137-pre.txt`): both leftovers skipped, the
|
||||
off-site tier due on 9201, the local tier waiting for today's 9201 archive to settle. Three agent releases in one session
|
||||
— each the smallest fix of what the box showed.
|
||||
- **Measured:** a PBS archive carries its key FINGERPRINT (PVE content `encrypted`), not a host id; the storage
|
||||
carries its own (`GET /storage` → `encryption-key`). The restore test now skips an archive written with another
|
||||
key and logs it by name; an unencrypted storage is not filtered; a failed storage read is UNKNOWN.
|
||||
- Tests `TestR727_*`; red-proof RP39 (the skip removed → the 2026-09-16 archive of an earlier box is picked).
|
||||
- Released by `release-agent.sh` (tag `v0.138.0`, sha256 `55916026…8195`, verified by download); **not vouched**.
|
||||
Delivered by signed `agent_update` jobs to `demo-hp-bb76ea` and `demo-felhom-8363b5` (both committed within 340 s);
|
||||
`-selftest=restore-test-due` on both reads each tier normally on 0.138.0.
|
||||
- ep0 (decision 51): the three drill archives in `tester-1`'s namespace removed; other namespaces byte-identical.
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
package backup
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
|
||||
)
|
||||
|
||||
const (
|
||||
thisBoxKey = "de:51:7a:18:cb:39:22:30:2c:84:f5:8b:d1:91:4b:7e:81:bb:69:b8:89:0f:57:ac:d3:59:e1:1a:62:25:11:2c"
|
||||
earlierBox1 = "6b:ca:5f:3f:ca:0f:e2:3f:fb:24:62:89:bf:e7:64:59:9a:41:c5:e6:e3:9f:3f:5f:e1:71:7b:a1:9d:24:67:82"
|
||||
earlierBox2 = "fe:3d:db:95:d4:df:ab:e1:7d:4a:89:fa:2b:07:53:6a:e4:d2:85:95:d1:90:27:4b:d9:c6:92:20:95:04:e5:d4"
|
||||
)
|
||||
|
||||
// R-727 (v0.138.0) — the 2026-09-30 shape, measured on a fresh box for a returning customer: the PBS
|
||||
// namespace held two archives of earlier boxes (same guest 9201, same token) and this box's own, which was not
|
||||
// settled yet. The old picker chose the earlier box's newest settled archive and failed `wrong key`.
|
||||
// The CONSEQUENCE asserted: no archive of another box is ever picked; with this box's archive settled it is picked.
|
||||
// COMPANION RED-PROOF: remove the `ownKey != "" && !EqualFold(...)` skip → the first case picks 2026-09-16T21:59:54Z.
|
||||
func TestR727_TheRestoreTestTakesOnlyThisBoxsArchives(t *testing.T) {
|
||||
day := int64(86400)
|
||||
now := int64(1790740000) // 2026-09-30 ~04:00Z
|
||||
own := proxmox.StorageContent{VolID: "felhom-pbs:backup/ct/9201/2026-09-29T19:37:07Z", Content: "backup", VMID: 9201, Size: 3490689830, CTime: 1790710627, Encrypted: thisBoxKey}
|
||||
api := &fakeBackupAPI{
|
||||
storages: []proxmox.Storage{{Storage: "felhom-pbs", Type: "pbs", EncryptionKey: thisBoxKey}},
|
||||
content: []proxmox.StorageContent{
|
||||
{VolID: "felhom-pbs:backup/ct/9201/2026-09-16T17:27:32Z", Content: "backup", VMID: 9201, Size: 4774114206, CTime: 1789579652, Encrypted: earlierBox2},
|
||||
{VolID: "felhom-pbs:backup/ct/9201/2026-09-16T21:59:54Z", Content: "backup", VMID: 9201, Size: 20811501236, CTime: 1789595994, Encrypted: earlierBox1},
|
||||
own,
|
||||
},
|
||||
}
|
||||
r := NewBackupRunner(api, "local", proxmox.ModeSnapshot, "", "keep-last=1", quiet())
|
||||
|
||||
// 1. The night of 2026-09-30: this box's own archive is ~6 h old, not settled (cutoff 24 h) — nothing to prove.
|
||||
got, _, err := r.PickSettledRestoreCandidateOn(context.Background(), "felhom-pbs", time.Unix(now-day, 0).UTC())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != "" {
|
||||
t.Fatalf("picked %q — an archive of ANOTHER box is never this box's proof (R-727)", got)
|
||||
}
|
||||
// 2. A day later this box's own archive is settled — it is the one picked.
|
||||
got, _, err = r.PickSettledRestoreCandidateOn(context.Background(), "felhom-pbs", time.Unix(now+day, 0).UTC())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != own.VolID {
|
||||
t.Fatalf("picked %q, want this box's own %q", got, own.VolID)
|
||||
}
|
||||
}
|
||||
|
||||
// An unencrypted storage (a local dir) holds only this box's vzdumps — no key filter applies.
|
||||
func TestR727_UnencryptedStorageIsNotFiltered(t *testing.T) {
|
||||
api := &fakeBackupAPI{
|
||||
storages: []proxmox.Storage{{Storage: "local", Type: "dir"}},
|
||||
content: []proxmox.StorageContent{{VolID: "local:backup/vzdump-lxc-9201-2026_09_29-21_27_05.tar.zst", Content: "backup", VMID: 9201, Size: 955425507, CTime: 1790710025}},
|
||||
}
|
||||
r := NewBackupRunner(api, "local", proxmox.ModeSnapshot, "", "keep-last=1", quiet())
|
||||
if got, _, err := r.PickSettledRestoreCandidateOn(context.Background(), "local", time.Time{}); err != nil || got == "" {
|
||||
t.Fatalf("got %q err %v", got, err)
|
||||
}
|
||||
}
|
||||
|
||||
// A storage-list failure makes the tier UNKNOWN (an error), never "nothing to prove".
|
||||
func TestR727_KeyLookupFailureIsUnknown(t *testing.T) {
|
||||
api := &fakeBackupAPI{storageErr: errors.New("proxmox: GET /storage -> HTTP 500"), content: []proxmox.StorageContent{{VolID: "felhom-pbs:backup/ct/9201/x", Content: "backup", VMID: 9201}}}
|
||||
r := NewBackupRunner(api, "local", proxmox.ModeSnapshot, "", "keep-last=1", quiet())
|
||||
if _, _, err := r.PickSettledRestoreCandidateOn(context.Background(), "felhom-pbs", time.Time{}); err == nil {
|
||||
t.Fatal("a failed key lookup must surface as an error (tier UNKNOWN)")
|
||||
}
|
||||
}
|
||||
@@ -357,6 +357,16 @@ func (r *BackupRunner) PickSettledRestoreCandidateOn(ctx context.Context, target
|
||||
if err != nil {
|
||||
return "", time.Time{}, err
|
||||
}
|
||||
// R-727 (v0.138.0): on an ENCRYPTED storage, only archives written with THIS storage's key are this box's.
|
||||
// Measured 2026-09-30 on a fresh box for a returning customer: the PBS namespace still held two archives
|
||||
// of earlier boxes (same guest id 9201, same token), the newest settled one was an earlier box's, and the
|
||||
// test failed `wrong key` every evaluation. The archive carries no host id; its key fingerprint is the
|
||||
// discriminator (PVE's content `encrypted`, the storage's `encryption-key`). A lookup failure returns
|
||||
// the error — the tier reads UNKNOWN, never "nothing to prove".
|
||||
ownKey, err := r.storageKeyFingerprint(ctx, target)
|
||||
if err != nil {
|
||||
return "", time.Time{}, fmt.Errorf("reading the key fingerprint of storage %s: %w", target, err)
|
||||
}
|
||||
var best string
|
||||
var bestCTime int64 = -1
|
||||
known := map[int]bool{} // vmid → the guest exists on this node (asked once per vmid per pick)
|
||||
@@ -372,6 +382,10 @@ func (r *BackupRunner) PickSettledRestoreCandidateOn(ctx context.Context, target
|
||||
r.noteNotAGuestBackupOnce(e, why)
|
||||
continue
|
||||
}
|
||||
if ownKey != "" && !strings.EqualFold(e.Encrypted, ownKey) {
|
||||
r.noteNotAGuestBackupOnce(e, fmt.Sprintf("written by another box (key %s, this box's key %s) — not this box's proof", shortFP(e.Encrypted), shortFP(ownKey)))
|
||||
continue
|
||||
}
|
||||
// R-689 (v0.136.0): … OF A GUEST THAT STILL EXISTS here. Measured on demo-hp 2026-09-27 right after
|
||||
// v0.135.0: with the golden skipped, the pick fell to `vzdump-lxc-9100-2026_08_21…`, a leftover of a
|
||||
// guest deleted in August — proving nothing about any guest this box runs. "Does not exist" skips the
|
||||
@@ -667,3 +681,29 @@ func (r *BackupRunner) noteNotAGuestBackupOnce(e proxmox.StorageContent, why str
|
||||
"target", r.target, "volid", e.VolID, "size_bytes", e.Size, "reason", why)
|
||||
}
|
||||
}
|
||||
|
||||
// storageKeyFingerprint returns the named storage's client-side encryption key fingerprint ("" when the
|
||||
// storage is not encrypted — a local dir holds only this box's own vzdumps).
|
||||
func (r *BackupRunner) storageKeyFingerprint(ctx context.Context, target string) (string, error) {
|
||||
sts, err := r.api.ListStorage(ctx)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
for _, st := range sts {
|
||||
if st.Storage == target {
|
||||
return strings.TrimSpace(st.EncryptionKey), nil
|
||||
}
|
||||
}
|
||||
return "", nil
|
||||
}
|
||||
|
||||
// shortFP is the first 8 bytes of a key fingerprint, for a log line.
|
||||
func shortFP(fp string) string {
|
||||
if fp == "" {
|
||||
return "none"
|
||||
}
|
||||
if len(fp) > 23 {
|
||||
return fp[:23] + "…"
|
||||
}
|
||||
return fp
|
||||
}
|
||||
|
||||
@@ -219,15 +219,18 @@ type Storage struct {
|
||||
UsedFraction float64 `json:"used_fraction,omitempty"`
|
||||
|
||||
// Type-specific config (durable_id sources).
|
||||
Server string `json:"server,omitempty"` // nfs/cifs/pbs server host
|
||||
Export string `json:"export,omitempty"` // nfs export path
|
||||
Share string `json:"share,omitempty"` // cifs share name
|
||||
Datastore string `json:"datastore,omitempty"` // pbs datastore name
|
||||
Fingerprint string `json:"fingerprint,omitempty"` // pbs server cert fingerprint
|
||||
Username string `json:"username,omitempty"` // pbs auth id, e.g. "felhom@pbs!n100"
|
||||
Namespace string `json:"namespace,omitempty"` // pbs namespace ("" = root; per-customer tenancy = S4)
|
||||
VGName string `json:"vgname,omitempty"` // lvm/lvmthin volume group
|
||||
ThinPool string `json:"thinpool,omitempty"` // lvmthin pool LV name
|
||||
Server string `json:"server,omitempty"` // nfs/cifs/pbs server host
|
||||
// EncryptionKey is the storage's own client-side key FINGERPRINT (pbs; the key itself stays in
|
||||
// /etc/pve/priv). R-727: an archive encrypted with any other key was written by another box.
|
||||
EncryptionKey string `json:"encryption-key,omitempty"`
|
||||
Export string `json:"export,omitempty"` // nfs export path
|
||||
Share string `json:"share,omitempty"` // cifs share name
|
||||
Datastore string `json:"datastore,omitempty"` // pbs datastore name
|
||||
Fingerprint string `json:"fingerprint,omitempty"` // pbs server cert fingerprint
|
||||
Username string `json:"username,omitempty"` // pbs auth id, e.g. "felhom@pbs!n100"
|
||||
Namespace string `json:"namespace,omitempty"` // pbs namespace ("" = root; per-customer tenancy = S4)
|
||||
VGName string `json:"vgname,omitempty"` // lvm/lvmthin volume group
|
||||
ThinPool string `json:"thinpool,omitempty"` // lvmthin pool LV name
|
||||
}
|
||||
|
||||
// StorageContent is one entry of GET /nodes/{node}/storage/{store}/content
|
||||
@@ -239,4 +242,7 @@ type StorageContent struct {
|
||||
Size int64 `json:"size"`
|
||||
CTime int64 `json:"ctime"`
|
||||
VMID int `json:"vmid,omitempty"`
|
||||
// Encrypted is the fingerprint of the key a PBS archive was encrypted with ("" = not encrypted). R-727:
|
||||
// the restore test reads it to tell THIS box's archives from an earlier box's in the same namespace.
|
||||
Encrypted string `json:"encrypted,omitempty"`
|
||||
}
|
||||
|
||||
@@ -302,6 +302,19 @@ func (e *Engine) runBringUp(ctx context.Context, spec BringUpSpec, res *BringUpR
|
||||
}
|
||||
}
|
||||
|
||||
// R-834: DR keeps the archive's `onboot: 1`, binds the host's REAL drives (4d) and STARTS the guest
|
||||
// — right on a replaced host, where the original is gone. Beside a LIVE original it would be a
|
||||
// second controller for the same household on the same drives. So DR refuses when this host
|
||||
// still carries the original (the archive's source VMID) or any guest that binds the drives.
|
||||
// A copy beside the original is the restore-test's job (onboot=0, throwaway stand-ins, torn
|
||||
// down) or the runbook's beside-restore. Pinned by TestRunBringUp_DRRefusesBesideALiveOriginal.
|
||||
if spec.Mode == ModeDRGuestLoss {
|
||||
if why := e.liveOriginalBeside(ctx, lxc, spec.Archive); why != "" {
|
||||
res.Err = fmt.Errorf("reconcile: dr bring-up refused: %s — a DR restore beside a live original would run two boxes on the same drives (R-834)", why)
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
base := JournalEntry{OpID: e.bringUpOpID(spec.VMID), VMID: spec.VMID, Kind: bringUpKind, Rollback: true}
|
||||
|
||||
// OWN the rollback BEFORE any mutation. From here a crash leaves an in-flight Rollback
|
||||
@@ -734,3 +747,47 @@ func net0MAC(cfg proxmox.GuestConfig) string {
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// archiveSourceVMID reads the source guest's VMID from a backup volid: a vzdump file
|
||||
// (`…/vzdump-lxc-<vmid>-<date>.tar.zst`) or a PBS snapshot (`…:backup/ct/<vmid>/<time>`). 0 = unknown.
|
||||
func archiveSourceVMID(archive string) int {
|
||||
if i := strings.Index(archive, "vzdump-lxc-"); i >= 0 {
|
||||
rest := archive[i+len("vzdump-lxc-"):]
|
||||
if j := strings.Index(rest, "-"); j > 0 {
|
||||
if n, err := strconv.Atoi(rest[:j]); err == nil {
|
||||
return n
|
||||
}
|
||||
}
|
||||
}
|
||||
if i := strings.Index(archive, "ct/"); i >= 0 {
|
||||
rest := archive[i+len("ct/"):]
|
||||
if j := strings.Index(rest, "/"); j > 0 {
|
||||
if n, err := strconv.Atoi(rest[:j]); err == nil {
|
||||
return n
|
||||
}
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// liveOriginalBeside says why a DR bring-up would land beside a live original ("" = it would not):
|
||||
// the archive's source guest still exists here, or a guest binds the drives parent. Fails CLOSED: a
|
||||
// guest whose config cannot be read cannot be ruled out.
|
||||
func (e *Engine) liveOriginalBeside(ctx context.Context, lxc []proxmox.Guest, archive string) string {
|
||||
src := archiveSourceVMID(archive)
|
||||
for _, g := range lxc {
|
||||
if src > 0 && g.VMID == src {
|
||||
return fmt.Sprintf("the archive's source guest %d still exists on this host (status %s)", g.VMID, g.Status)
|
||||
}
|
||||
cfg, err := e.api.GuestConfig(ctx, g.VMID)
|
||||
if err != nil {
|
||||
return fmt.Sprintf("guest %d's config could not be read to rule out a live original: %v", g.VMID, err)
|
||||
}
|
||||
for slot, v := range cfg.MountPoints() {
|
||||
if source, _, _ := strings.Cut(v, ","); source == structuralParentDir {
|
||||
return fmt.Sprintf("guest %d binds the household drives (%s %s)", g.VMID, slot, structuralParentDir)
|
||||
}
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
package reconcile
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
|
||||
)
|
||||
|
||||
// R-834: a whole-guest restore BESIDE a live original must never come up as a second box on the same
|
||||
// drives. The DR route keeps `onboot: 1`, binds the real drives and starts the guest, so it refuses
|
||||
// when the original (or any guest binding the drives) is still on this host; on a replaced host it
|
||||
// proceeds and keeps its binds. Red-proof: make liveOriginalBeside return "" and the refusals pass
|
||||
// the restore through (the "refused" sub-tests fail).
|
||||
func TestRunBringUp_DRRefusesBesideALiveOriginal(t *testing.T) {
|
||||
const target = 9299
|
||||
drivesBind := proxmox.GuestConfig{Extra: map[string]json.RawMessage{
|
||||
"mp8": json.RawMessage(`"/mnt/felhom-drives,mp=/mnt/felhom-drives"`),
|
||||
}}
|
||||
cases := []struct {
|
||||
name string
|
||||
archive string
|
||||
lxc []proxmox.Guest
|
||||
cfg map[int]proxmox.GuestConfig
|
||||
refuse string // substring of the refusal; "" = must proceed
|
||||
}{
|
||||
{"the source guest still exists", "local:backup/vzdump-lxc-9201-2026_10_04-04_34_55.tar.zst",
|
||||
[]proxmox.Guest{{VMID: 9201, Status: "running"}}, map[int]proxmox.GuestConfig{9201: scratchCfg()}, "source guest 9201"},
|
||||
{"another guest binds the drives (PBS archive)", "felhom-pbs:backup/ct/9201/2026-10-04T02:34:55Z",
|
||||
[]proxmox.Guest{{VMID: 9300, Status: "stopped"}}, map[int]proxmox.GuestConfig{9300: drivesBind}, "binds the household drives"},
|
||||
{"a guest whose config cannot be read", "local:backup/vzdump-lxc-9201-x.tar.zst",
|
||||
[]proxmox.Guest{{VMID: 9400, Status: "running"}}, map[int]proxmox.GuestConfig{}, "could not be read"},
|
||||
{"replaced host: only an unrelated scratch guest", "local:backup/vzdump-lxc-9201-x.tar.zst",
|
||||
[]proxmox.Guest{{VMID: 9202, Status: "running"}}, map[int]proxmox.GuestConfig{9202: scratchCfg()}, ""},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
cfg := c.cfg
|
||||
cfg[target] = scratchCfg()
|
||||
api := &fakeAPI{lxc: c.lxc, cfg: cfg}
|
||||
e, fr, _, q := newDREngine(t, api)
|
||||
defer q.Close()
|
||||
res := e.RunBringUp(context.Background(), BringUpSpec{
|
||||
Mode: ModeDRGuestLoss, Archive: c.archive, VMID: target, RestoreStorage: "local-lvm", KeepMAC: true,
|
||||
})
|
||||
if c.refuse != "" {
|
||||
if res.Err == nil || !strings.Contains(res.Err.Error(), c.refuse) || !strings.Contains(res.Err.Error(), "R-834") {
|
||||
t.Fatalf("want a refusal naming %q, got %+v", c.refuse, res)
|
||||
}
|
||||
if len(api.restores) != 0 || len(api.starts) != 0 || len(fr.cmds) != 0 {
|
||||
t.Fatalf("a refused DR touched the host: restores=%d starts=%v cmds=%v", len(api.restores), api.starts, fr.cmds)
|
||||
}
|
||||
return
|
||||
}
|
||||
if res.Err != nil || !res.Pass {
|
||||
t.Fatalf("a DR on a replaced host must proceed, got %+v", res)
|
||||
}
|
||||
// … and there it keeps the REAL drives bind (the right binds on a replaced host).
|
||||
joined := strings.Join(fr.cmds, "\n")
|
||||
if !strings.Contains(joined, "-mp8 /mnt/felhom-drives,mp=/mnt/felhom-drives") {
|
||||
t.Fatalf("the DR guest lost its drives bind: %v", fr.cmds)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Provisioning restores the GOLDEN (no drives, onboot set by the back-half on purpose): a drives-
|
||||
// binding guest on the host does not block it — the rule is DR's alone.
|
||||
func TestRunBringUp_ProvisionNotBlockedByADrivesBind(t *testing.T) {
|
||||
api := &fakeAPI{
|
||||
lxc: []proxmox.Guest{{VMID: 9201, Status: "running"}},
|
||||
cfg: map[int]proxmox.GuestConfig{
|
||||
9201: {Extra: map[string]json.RawMessage{"mp8": json.RawMessage(`"/mnt/felhom-drives,mp=/mnt/felhom-drives"`)}},
|
||||
9203: scratchCfg(),
|
||||
},
|
||||
}
|
||||
e, _, q := newEngine(t, api, EmptyProvider{})
|
||||
defer q.Close()
|
||||
res := e.RunBringUp(context.Background(), BringUpSpec{Mode: ModeProvision, Archive: "local:vztmpl/felhom-golden.tar.zst", VMID: 9203, RestoreStorage: "local-lvm"})
|
||||
if res.Err != nil || !res.Pass {
|
||||
t.Fatalf("provision must proceed, got %+v", res)
|
||||
}
|
||||
}
|
||||
|
||||
func TestArchiveSourceVMID(t *testing.T) {
|
||||
for in, want := range map[string]int{
|
||||
"local:backup/vzdump-lxc-9201-2026_10_04-04_34_55.tar.zst": 9201,
|
||||
"felhom-pbs:backup/ct/9201/2026-10-04T02:34:55Z": 9201,
|
||||
"tmp-dooplex-copy:backup/ct/9201/2026-10-03T19:00:00Z": 9201,
|
||||
"local:vztmpl/felhom-golden.tar.zst": 0,
|
||||
"vol": 0,
|
||||
} {
|
||||
if got := archiveSourceVMID(in); got != want {
|
||||
t.Errorf("archiveSourceVMID(%q) = %d, want %d", in, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// R-834, the restore-test route: its scratch guest sits BESIDE the live original by design, so it must
|
||||
// carry no host-path bind — the archive's mp8 (the household's drives) and mp9 (the original's
|
||||
// bootstrap) are replaced by throwaway volumes AT restore time. Measured live 2026-10-04 on demo-hp
|
||||
// (`audits/backup-close-2026-10-04/partA/`): onboot 0 and no host bind on every poll. Red-proof:
|
||||
// make drRestoreOverrides return the archive's own mp8 value and this fails.
|
||||
func TestRestoreTest_NoHostPathBindBesideTheOriginal(t *testing.T) {
|
||||
api := &fakeAPI{
|
||||
cfg: map[int]proxmox.GuestConfig{990000: scratchCfg()},
|
||||
extractCfg: "hostname: demo-hp\nonboot: 1\nrootfs: local-lvm:vm-9201-disk-0,size=16G\n" +
|
||||
"mp0: local-lvm:vm-9201-disk-1,mp=/var/lib/felhom,backup=1,size=70G\n" +
|
||||
"mp8: /mnt/felhom-drives,mp=/mnt/felhom-drives\n" +
|
||||
"mp9: /var/lib/felhom-agent/guests/9201/bootstrap,mp=/etc/felhom-bootstrap,ro=1\n",
|
||||
}
|
||||
e, _, q := newEngine(t, api, EmptyProvider{})
|
||||
defer q.Close()
|
||||
_ = e.RunRestoreTest(context.Background(), RestoreTestSpec{
|
||||
Archive: "local:backup/vzdump-lxc-9201-x.tar.zst", RestoreStorage: "local-lvm",
|
||||
ScratchMin: 990000, ScratchMax: 990009, SourceTier: "local",
|
||||
})
|
||||
if len(api.restores) != 1 {
|
||||
t.Fatalf("want one restore, got %+v", api.restores)
|
||||
}
|
||||
r := api.restores[0]
|
||||
for _, slot := range []string{"mp8", "mp9"} {
|
||||
v, ok := r.MountOverrides[slot]
|
||||
if !ok || strings.HasPrefix(v, "/") {
|
||||
t.Fatalf("%s = %q (present=%v): the scratch beside the original must get a throwaway volume, never the host path", slot, v, ok)
|
||||
}
|
||||
}
|
||||
for slot, v := range r.MountOverrides {
|
||||
if strings.HasPrefix(v, "/") {
|
||||
t.Fatalf("%s carries a host path %q into the scratch guest", slot, v)
|
||||
}
|
||||
}
|
||||
if r.ConfigOverrides["onboot"] != "0" {
|
||||
t.Fatalf("onboot = %q, want 0", r.ConfigOverrides["onboot"])
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user