The live validation hit PVE's all-or-nothing restore rule: mpN params
without an explicit rootfs -> HTTP 500 "mount points configured, but
'rootfs' not set" (the same constraint restoretest.go:211 documents for the
live-config path; the spike never ran an override restore). The lost guest
has no live config, so the rootfs SIZE now comes from the archive's own
embedded config via NEW Client.ExtractArchiveConfig (GET vzdump/
extractconfig - verified live: answers 200 under the scoped agent token;
PBS keys stay server-side, the spike's candidate-1 rejection holds; used
for the SIZE ONLY - the bind layout stays the platform constants).
Unparseable/unreadable archive config -> clean refusal before any restore.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
Pool membership is what lets the pool-scoped token reach a guest; pct restore
--pool sets it only at CREATE, so a restore over an existing VMID drops the guest
from the felhom pool and 403s the next restore-test/DR on VM.Audit. This empty-pool
state is the true root cause of the campaign's "R1" (bind-mount restore failing was
a symptom — restore-test's existing bind neutralization never ran without config-read).
Add Client.PoolAddVMID (PUT /pools, additive+idempotent, Pool.Allocate) and call it
in bring-up after liveness when spec.Pool!="" — warn-not-fail on a hiccup (liveness
wins). B3 scratch-teardown 403 diagnosed as a cascade (restoretest already passes
Pool). Role/ACL untouched. Tests + red-proof.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
A WAN (pbs-tier) restore of a large guest exceeds the restore-task wait's 10m
default → the wait expired mid-restore, teardown fired against a still-restoring
(not-yet-pool-associated) scratch guest → leak + a phantom VM.Allocate 403.
- RestoreTestSpec.RestoreTaskTimeout (0→10m default); the restore WaitTask passes
it. Local tier unchanged (10m).
- config RestoreTestPBSRestoreTimeoutSeconds + accessor (default 120m).
- main restoreTaskTimeout(cfg,tier): configured PBS timeout only when tier==pbs,
else 0. Both scheduler + selftest spec builds.
- Tests + WaitOptions red-proof + accessor contract.
The "grant scratch-band VM.Allocate" follow-up is diagnosed not blind-applied:
the scratch is restored INTO /pool/felhom (ACL already grants VM.Allocate), so
the earlier 403 was a consequence of the timeout. No ACL/host-install change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
Campaign pool-effects F1 (HIGH): the bring-up compensating rollback and the
restore-test teardown destroyed the target vmid even when RestoreLXC failed
synchronously without creating anything — destroying a guest the transaction
never made (only the pool ACL 403 contained it). A RestoreLXC UPID is now the
sole destroy authorization in all three destroy paths (in-process bring-up
defer, in-process restore-test teardown, Recover). F2: the restore-test
advances past an 'already exists' band vmid (invisible squatter) instead of
failing + false-alerting; a fully-occupied band Skips.
Red-proof verified: with the gates reverted, the four new tests fail with the
innocent-guest destroy. go build/vet/test clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The shared front half of provision and guest-loss DR as a journaled reconcile job
(internal/reconcile/bringup.go), mirroring the restore-test's crash-safety but keeping
the guest on success and applying a scenario-specific identity policy. Agent-only; no
hub/wire change. Grounded by the slice-7 bring-up spike (commit 3342993): F1/F3/F4.
- RunBringUp: restore -> reset identity -> size -> attach mounts -> start link-up;
verdict is liveness (waitRunning), success KEEPS the guest.
- identity policy: provision = fresh MAC (net0 sans hwaddr -> PVE regen) + hostname,
host-side; machine-id/host-keys regenerate guest-side (systemd + baked golden unit).
dr_guest_loss = preserve continuity (keep hostname; keep MAC unless KeepMAC=false).
- compensating rollback: mid-flight failure destroys the just-created guest
(SameTxnCreated provenance, gated); new Rollback journal flag + Recover.recoverBringUp
reap a half-built guest from a crash.
- F4: coalesced config PUT + bounded retry on the transient PVE config-lock 500 only.
- --selftest=bring-up (mode/archive/vmid/hostname/keep).
- configs/build-golden.sh: validated golden recipe incl. the F3 first-boot host-key unit.
- doc-03 §9 + identity-reset settled/implemented.
Deferred (stated): provisioning back half -> slice 8; host-loss DR + escrow consumption
and the BringUpSpec source (hub desired-state) -> slice 10.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixes the crying-wolf false-fail surfaced by the live hub-enrollment runbook:
PVE's guest-start task exits "WARNINGS: 1" for the benign systemd-nesting
advisory, and WaitTask treated any non-OK exitstatus as failure, so the verdict
was decided by an advisory exit code before the real boot check ran. Every
modern-distro restore-test reported pass:false.
- proxmox.WaitOptions.AllowWarnings (opt-in; default keeps all callers strict)
- restore-test start step accepts warnings, surfaces them, verdict stays waitRunning
- RestoreTestResult.StartWarnings/.WarningsRecognized + version-free "enable
nesting" recognizer (can't rot back at systemd 258+); GuestAPI.TaskLogTail
- hub.RestoreTest.warnings/.warnings_recognized wire fields (consumed by hub v0.7.5)
- scheduler logs clean / passed-with-recognized / passed-with-unrecognized warnings
- tests: WaitTask warnings matrix; restore-test pass/fail-on-liveness; version-free
regression guard (systemd 256-300)
Single agent bump 0.6.0 -> 0.7.0 covering the agent half of both task phases.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The security core of slice 4: hub-supplied intent is no longer trusted for
destructive change. The gate fronts the per-guest queue's executor, so every
mutation passes it. Reuses internal/authz for all crypto (surface untouched).
- Classifier (doc 03 §4): benign vs destructive by provenance + data-bearing-
ness, NOT by verb. Destroy/overwrite of customer data is destructive unless
agent-internal provenance (same-journaled-txn create, or agent-tagged scratch)
makes it benign — and that provenance is journal-recorded, NEVER hub-sourced.
Unknown op class fails safe to destructive.
- Reversibility gate: benign -> allowed unsigned; destructive -> requires a
verified, role-scoped, action-bound operator signature, else pending_signature
and never executed. Every decision audited (signal, never the guard).
- Signed-op consuming layer over authz.Verifier.Verify (locked pipeline
untouched): role-scoping (doc 04 §4 — recovery=rotation only, operational=
ordinary destructive + planned rotation) + op-to-action binding (op+host+
guest+params must match the gated action).
- Signed-job orchestration: idempotency dedupe by nonce + journal-wrapped
execution via an injected DestructiveExecutor (nil this slice — inert).
- Crash recovery (Note 1): Engine.Recover consumes the journal InFlight() set at
startup (resume-or-rollback) — covers an op that crashed after the POST and
before its terminal record, which idempotency dedupe alone cannot. Added
TaskStatusOnce to the GuestAPI seam. Wired into daemon startup.
- Note 2: memory comparison canonicalized to MiB (desiredMemoryMiB) so a
non-MiB-aligned MemoryBytes converges in one pass, not perpetual drift.
- Daemon: builds the verifier from config signers (none = nil verifier, the
common slice-4 state), the gate (+SlogAudit), runs Recover before mutating.
Adversarial matrix proven against the REAL authz.Verifier with in-test-minted
SSHSIGs (framing replicated in reconcile's test binary; authz untouched, no
signing added to the verify-only package): unsigned job + unsigned desired-state
delta -> pending_signature; unknown signer/expired/replay-across-restart/wrong
host -> typed authz rejections; wrong guest/op/params -> binding_mismatch;
recovery key on ordinary destructive -> role_denied; hub-supplied scratch tag
ignored -> refused; valid+role+target+fresh nonce -> accepted then replay
rejected. Full module race-clean + vet-clean on the Linux build server.
Inert this slice: no destructive deltas served until slice 10; the destructive
path is classified, gated, and tested but not wired to live execution.
CHECKPOINT: Phase B complete (slice 4 done). Awaiting validation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New internal/reconcile package: the agent-side control core's structural half.
- Per-guest serializer Queue (doc 03 §10): the single choke point all mutation
sources funnel through; same-vmid serial in submit order, different vmids
parallel (cond-var FIFO lanes).
- Desired-state model + DesiredProvider seam; EmptyProvider is the only live
source at slice 4 (no hub serving until slice 10) so the live engine computes
an empty action set and performs zero mutations.
- Normalization layer (FieldNormalizers): normalized desired-vs-actual so
Proxmox round-trip quirks don't read as drift. normDesc promoted out of
main.go to reconcile.NormDescription; selftest uses the shared helper.
- Plan (pure diff): minimal benign action set (Start/Stop/SetConfig) for guests
in both desired and actual; provision/destroy out of scope here.
- Engine: dispatches onto the shared queue; honors the dual-mode SetConfig
contract (UPID -> WaitTask; empty UPID -> synchronous success).
- Durable op journal + idempotency store (mirrors authz.FileNonceStore):
in-flight task ids for crash detection + AlreadyApplied dedupe across restart.
- Wired into runDaemon alongside the hub loop, sharing the queue; runs cleanly
with no desired state and no signers.
Full module race-clean and vet-clean on the Linux build server.
CHECKPOINT: Phase A only. Awaiting validation before Phase B (the reversibility
gate + signed-op consuming layer, landing v0.4.0).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
admin