Files

T

admin 1c8a67eece R-106 + R-109: the DR recipe records the resolved namespace and names the backup target (v0.118.0)

Both defects were live on both demo boxes: the recipe said namespace "root" while
storage.cfg said demo-felhom/demo-hp, and it never named which of two content=backup
dir storages holds the local archives.

R-106: the namespace came from the listed snapshot, but PBS omits `ns` per item once
the list is namespace-scoped, so it was always empty and normalised to "root". It now
resolves from the pbs STORAGE (storage.cfg's `namespace`) — the same field vzdump makes
PVE read, so the recipe cannot disagree with the backup.

R-109: backup_target resolves from the primary tier of cfg.Backup.BackupTiers(), the
function the scheduler consults, and carries the mountpoint that separates /mnt/hdd_1
from /var/lib/vz. The resolver reports the tier IN EFFECT (daemon-start config), not
agent.json on disk — a target move rewrites the file and deliberately does not restart.

Unresolvable is recorded as unresolvable: resolved|unknown plus a distinct reason,
never a default, an empty string, or a placeholder.

Needs hub v0.83.0 — AssembleDRRecipe allow-lists top-level keys, so backup_target
would otherwise be stored intact and dropped before any operator saw it.

9 tests, 4 red-proofs (each mutation asserted to have landed). Suite rc=0, 29 ok.

2026-07-30 13:11:08 +02:00

4.1 KiB

Raw Blame History

REPORT — R-106 + R-109: the DR recipe stops guessing (agent v0.118.0, 2026-07-30)

Overwritten per the standing rule. Paired with hub v0.83.0 — neither half is useful alone, see §5.

1. What was wrong, confirmed live BEFORE the fix

Downloaded from the hub, GET /customers/<id>/dr-recipe.json, both demo boxes:

"pbs": { "repo_id": "felhom-pbs", "namespace": "root", "latest_snapshot_id": "9201" }

and no backup_target key at all, while pve_storage offered two plausible candidates:

{ "name": "felhom-backup", "type": "local-dir", "content": "backup" }
{ "name": "local",         "type": "local",     "content": "backup,import,vztmpl,iso" }

/etc/pve/storage.cfg on those same boxes says namespace demo-felhom and namespace demo-hp.

2. Root causes, at source

R-106 — Snapshot.Namespace decodes ns (internal/pbs/client.go:97), which PBS does not echo per item once the list is namespace-scoped via ?ns= (:118-120). Always empty → ToHub normalises empty to "root" (internal/pbs/report.go:22-25) → latestPBSCoord writes it into the recipe.
R-109 — BuildDRRecipeHostHalf emitted guests/drives/pve_storage/pbs and nothing naming the vzdump target. Benign until the 2026-07-28 target move gave every box a second content=backup dir storage holding archives frozen at that date.

3. The fix

The namespace now resolves from the pbs storage (StorageTarget.PBSNamespace, from storage.cfg's namespace) — the same field vzdump --storage <pbs> makes PVE read, so the recipe cannot disagree with the backup that produced the snapshot. backup_target resolves from the primary tier of cfg.Backup.BackupTiers() — the function the scheduler itself consults — and carries the mountpoint, which is what actually separates /mnt/hdd_1 from /var/lib/vz.

Unresolvable is recorded as unresolvable, never defaulted: namespace_state and backup_target.state are resolved | unknown, with distinct reasons (agent_backup_config_unavailable, not_a_known_storage, no_pbs_storage_observed). An unconfigured namespace still reads "root" — that is an answer, not a gap.

The resolver reports the tier IN EFFECT, not the file on disk. A backup-target move rewrites agent.json and deliberately does not restart the agent (the E-1 lesson), so a disk re-read would name a storage no archive had reached yet.

4. Files

internal/hub/report.go, internal/storage/observe.go, internal/hub/dr_recipe.go, internal/hub/collect.go, cmd/felhom-agent/main.go, internal/hub/testdata/host-report.golden.json, REUSE.md; tests in internal/hub/{dr_recipe,contract,report}_test.go, internal/pbs/live_reporter_test.go.

5. The cross-repo half that is easy to miss

The hub's AssembleDRRecipe allow-lists top-level keys. backup_target would have been stored intact and dropped before any operator saw it. The same mechanism already cost offsite_restic its whole existence (R-122, found this session). REUSE.md now records this as a two-repo change.

6. Tests

9 new. Consequence-level: a box with two content=backup storages names the live one and not the frozen one; assertBackupCandidateAmbiguity fails if the fixture ever stops posing that problem. Fixtures are the storage set demo-felhom really had, provenance recorded in-file. Two tests run the real Collector.Collect() path, one pinning that an unwired seam yields unknown.

4 red-proofs, each mutation asserted to have landed before running: revert R-106 → the live "root" symptom reproduces; make unknown guess a default → 5 assertions fire; drop the field → the cross-repo contract guard fires; hollow the fixture → the ambiguity guard fires.

Suite: rc=0, 29 packages ok, 0 FAIL (run separately from the commit).

7. Not touched

The backup machinery. This corrects the record, not the doing — and the record was never acted on programmatically: the host_loss plan is surface-only (internal/dr/plan.go:125), so the blast radius was exactly one operator reading a wrong value. plan.PBS inherits the fix for free.

4.1 KiB Raw Blame History