Supersedes the "not yet healed" hand-off section. Viktor clicked "Re-issue PBS
credentials" and the chain closed in 9 seconds: hub re-issued 20:28:44 -> agent
consumed 20:28:51 -> `pbsdr: converged state=applied` 20:28:53, with the PATCHED
wrapper. consumed-failed.json cleared automatically, marker.json rewritten. The
agent picked it up unaided because marker.json was absent, so the L235 hash
short-circuit did not apply.
Proof the tier is alive rather than merely reporting alive:
pvesm status 401/inactive -> ACTIVE
token /version 401 -> 200
real backup none -> felhom-pbs:backup/ct/9201/
2026-07-18T18:31:06Z, 9744319312 B, 13m36s
Encrypted under fingerprint 7e:a6:af:f7:ea:6d:3e:d9 — the ESCROWED key, the one
customer zero holds the recovery code for. First real DR-tier backup on the
reborn box.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nn3VgQk9iwEGgyx6QJ2NvE
Config-only (wrapper + red-proof); the Go binary is unchanged, so this ships
with the next agent deploy as a config artifact.
The reconcile verb built `args=(--server "$server" --fingerprint "$fp")`. PVE
treats a PBS storage's `server` as a CREATE-ONLY parameter and rejects the
ENTIRE `pvesm set` call — "can't change value of fixed parameter 'server'" —
even when the value passed is byte-identical to the stored one. So reconcile
could never succeed against an existing entry; it exited 255 every time.
That is severe rather than cosmetic because the agent consumes the hub's
ONE-TIME PBS token secret BEFORE invoking the wrapper. Each hub "Re-issue PBS
credentials" therefore minted a secret, the agent burned it, the wrapper
rejected the apply, and the entry stayed pinned to the revoked credential —
a PBS DR tier authenticating 401 indefinitely while the agent reported
`pbsdr: converged state=applied`.
Live-diagnosed on the N100 during the rehearsal wrap (felhom.eu
tests/VALIDATION-n100-rehearsal-2026-07-18.md F2, ROADMAP R-39). Proven on the
live entry before writing code: `pvesm set <id> --server <same> --fingerprint
<same>` -> rejected; the same call without --server -> rc 0. K (<id>.enc) and
the .pw store verified byte-untouched after the rejected call — PVE rejects
atomically, so the set-only law held.
Fix: drop --server. The server address is immutable by construction (relocating
a PBS endpoint needs a fresh create), so there was never anything to reconcile
there. --fingerprint (+ --password when a secret is fed) remain.
Red-proof TestReconcileNeverPassesServerToPvesmSet: isolates the reconcile)
block from the shipped wrapper, asserts no --server reaches `pvesm set` and
that --fingerprint is still pushed. Verified RED on the unfixed wrapper, GREEN
after. Handles two vacuous-pass traps that both fired while authoring it: the
pattern is line-ending tolerant (\r?\n — this repo is cloned on Windows, and an
\n-only pattern matches nothing and passes silently), and comment lines are
stripped before matching (the WHY note quotes the very flag under test).
NOT fixed here, both still open and riding the spec'd R-39 agent train:
1. R-39's primary half — the agent re-applies on a change of the DESCRIPTOR
HASH (manager.go ~L235), but a credential re-issue leaves the descriptor
byte-identical (same token_id/fingerprint; only the side-table secret
rotates) and bumps only the generation, so a converged agent still ignores
a fresh secret. This makes the apply succeed once it re-applies; it does
not make it re-apply.
2. The verify loop reads /etc/pve/priv/storage/<id>.pw directly as non-root —
a path it can only ever WRITE through the root wrapper (0700 root:www-data;
sudoers exposes create|reconcile|grant, no read verb), so it is permanently
blind to the failure it exists to catch.
Demo box: wrapper hotfixed in place (.bak-20260718-preR39 kept). NOT yet healed
— diagnosis consumed the pending secret against the unfixed wrapper; the agent
parked correctly in consumed-failed (no burn loop). Healing needs Viktor to
click "Re-issue PBS credentials"; the agent will then pick it up unaided
(marker.json absent, so the L235 short-circuit does not apply).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nn3VgQk9iwEGgyx6QJ2NvE
Operator ruling 2026-07-13: walk the root's backing device through /sys/block/<dev>/slaves
recursively down to physical disks (dm AND md; topology, never VG names); those + any mounted-ESP
holder are system; the all-system fail-safe returns to being the WALK-FAILURE error case only.
SAFETY DIRECTION: a root-backing disk classified candidate is made impossible — per-branch
conservatism (any unresolvable slave fails the WHOLE walk -> ok=false -> the unchanged all-system
path).
- physicalDisksOf/walkSlaves in role.go (symlink canon -> wholeDiskOf fast path -> recursive
slaves walk; cycle/depth guard; non-/dev sources unwalkable)
- HostReader.BlockSlaves(name) — the ONE new seam method; ProcHostReader reads
/sys/block/<name>/slaves; all four test fakes mirror it
- role_walk_test.go: signature table (root-backing disk ALWAYS system across legacy-LVM /
md-raid / EFI+raw / EFI+LVM / nested dm-on-md — NEVER weaken) + dead-wizard-lives +
dangling-slave fail-safe (real sysKnown=false path) + cycle + empty-slaves; red-proofs A/B/D
run->fail->revert (recorded in REPORT)
- §3 spike transcripts (drill legacy: dm-1->sda3->sda; felhom-pve: ESP+walk agree on sda ->
byte-identical regression); caller audit: none relied on all-system as a feature
- format/mkfs paths, data-bearing guards, wizard UI untouched
Parts 1-2 shipped + deployed (v0.69.0); Part 3 not needed (sqlite3 export);
Part 4-A re-attach safety already unit-proven; Part 4-B destructive 9201 drill
prepared + operator-gated (pre-flight green), not executed by CC.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
Live diagnosis on the agent-token path: offsite restore-test pass:true,
verified:boot+running, teardown succeeded (no 403), scratch band clean. The
VM.Allocate follow-up was a PHANTOM — the earlier 403 was the 10m-timeout firing
teardown against a not-yet-pool-associated guest; with the timeout fixed the
guest is in /pool/felhom and teardown is authorized. No ACL/host-install change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
Scenario-D green: real vzdump of 9201 → ciphertext in ns over the tunnel;
ns-scoped verify=ok under the box's own DatastoreBackup token; WARN gone;
restore round-tripped (decrypt with box-born key → boot → teardown).
Findings recorded: retarget field is local_backup_target (reverted to local —
single-target offsite is near-continuous given the ~30-min controller cadence;
needs Tier-1/Tier-2 split); restore-test scheduler needs a WAN deadline +
scratch VM.Allocate; escrow left as an operator-present step.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
admin