docs: v0.74.0 REPORT + CONTEXT — pool re-assertion; R1 was a symptom, restore-test works live

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
This commit is contained in:
2026-07-07 19:07:26 +02:00
parent ca0b169a4e
commit ced60ddf76
2 changed files with 52 additions and 37 deletions
+11
View File
@@ -5,6 +5,17 @@
## Current
- **v0.74.0** (2026-07-07) — **campaign-2 R2 CLOSED; the mislabelled "R1" was a symptom** (LIVE on
felhom-pve). Pool membership is what lets the pool-scoped token reach a guest; `pct restore --pool`
sets it only at CREATE, so a restore-over-existing dropped 9201 from the `felhom` pool → no
`VM.Audit` → restore-test's *existing* `bindMountOverrides` never ran → "mp8 … only possible for
root". Fix: `Client.PoolAddVMID` + bring-up re-asserts membership post-restore (warn-not-fail).
Role/ACL + `bindMountOverrides` untouched (both correct). **Live restore-test PASSED for the first
time** once the pool was healed (Part A one-liner): read config → neutralize 2 binds → restore →
boot+running → clean teardown, 4m35s. B3 (scratch-teardown 403) confirmed a cascade — no code.
OPEN: DR `bring-up -mode dr` bind-override gap (spike `SPIKE-dr-bindmount-source-2026-07-07.md`:
small known-constant override reusing `bindMountOverrides`; mp8/mp9 are structural constants).
- **v0.73.0** (2026-07-06) — **F2 mount-role fallback CLOSED** (LIVE on felhom-pve). `roleForMountPath`
gained a mount-table fallback (Impl-2b style): a bind-mounted RAW enrolled user-data drive is not a PVE
storage, so it fail-safe'd to `system` and the eject/decommission gates 403'd EVERY user-data drive
+41 -37
View File
@@ -1,43 +1,47 @@
# REPORT — v0.73.0: F2 mount-role fallback (enrolled user-data drives ejectable/decommissionable)
# REPORT — v0.74.0: re-assert pool membership after restore-over-existing (campaign-2 R2)
**Date:** 2026-07-06 · **Class:** risky/supervised (loosens a role gate; bounded by containment red-proofs).
**Baseline:** `main` @ `26664d6` (v0.72.0) → `7545af8`. Part A of the pilot-blockers bundle (B = controller
v0.102.0, C = catalog healthcheck sweep). Provenance: campaign F2 + RERUN addendum.
**Date:** 2026-07-07 · **Class:** implementation (agent). **Baseline:** `main` @ `e04b75e` (v0.73.0)
`ca0b169`. Part B of the campaign-2 R1/R2 fix bundle (C = controller F-C2-1 v0.103.0; A = operator
pool one-liner; D = DR-source spike; E = report correction).
## The bug
`roleForMountPath` (`internal/localapi/disks.go`) resolved a mount's protection role ONLY from the PVE
storage view (`Observe`). A bind-mounted RAW enrolled user-data drive is not a PVE storage → no MountPath
match → fail-safe `RoleSystem` → the eject (disks.go:353) and decommission (disks.go:404) role gates 403'd
**every user-data drive in the standard topology** (campaign journal: `where=/mnt/teszt_enroll role=system`).
## Root cause (the campaign report's R1 was a symptom)
Pool membership is what lets the pool-scoped `FelhomAgentGuest` token reach a guest — the grant at
`/pool/<pool>` applies only to pool MEMBERS. `pct restore --pool` sets membership at CREATE, but a
restore OVER AN EXISTING VMID (the host-loss/finale path) never re-applies it, and no code re-added a
guest to the pool. So every destroy-restore silently dropped membership → the guest lost `VM.Audit`
→ the NEXT `--selftest=restore-test`/DR couldn't read the source config → its *existing, correct*
`bindMountOverrides` never ran → raw bind mounts hit `pct restore` → "mp8 … only possible for root"
(the campaign's mislabelled "R1"). The role + ACL were correct all along; **membership** was the fault.
## The fix (only `roleForMountPath`)
After the MountPath loop misses on a SUCCESSFUL Observe, resolve the mount's device from the host mount
table (mirroring `durableIDForMount`'s Impl-2b) and classify **device-keyed**:
- non-`/dev` source (NAS) → system;
- device on the same whole disk as a KNOWN target → THAT target's role (containment, new
`storage.SameWholeDisk`, whole-disk granularity so a protected disk can't be ejected here);
- else `RoleForRawDevice` (system-disk membership).
**Fail-safe preserved:** an Observe error returns system BEFORE the fallback (a blind containment pass could
label a backup drive user-data — permissive); a mount-read failure or absent/NAS mount → system. Handlers,
`deviceRole`, `DecommissionExecutor`, `classify.go`, `ReassertGuestBinds` untouched; the
`deviceRole`/`roleForMountPath` unification is deferred.
## The fix
- **`Client.PoolAddVMID(ctx, pool, vmid)`** (`internal/proxmox/mutate.go`): `PUT /pools/{pool}`
`vms={vmid}` — PVE-additive (merge, not replace; `delete=1` removes), idempotent (already-member
swallowed), needs `Pool.Allocate` (the token has it). Sync (no UPID).
- **bring-up re-asserts** (`internal/reconcile/bringup.go`): after liveness is proven, if
`spec.Pool != ""`, call `PoolAddVMID`. A pool-add hiccup is surfaced LOUD + into `res.StartWarnings`
but must NOT flip a healthy running guest's verdict (membership matters for the NEXT op).
- **B3 (scratch teardown 403) — diagnosed, no code:** `restoretest.go` already passes
`Pool: DefaultPool` for scratch restores → the campaign's `VM.Allocate` teardown 403 was a CASCADE
of the failed bind-mount restore (a half-built guest outside any pool), not an independent gap.
- Role/ACL untouched (correct); `bindMountOverrides` untouched (correct — it just needed to run).
## Tests + red-proofs (`f2_role_fallback_test.go`)
A1 bind-mounted user-data ejectable + decommission effects; B1 containment (mount on a protected target's
disk → 403); B2 system-disk mount → 403; C1/C2 fail-safe (absent, NAS → 403); C3 Observe-error skips the
fallback (403 + Mounts() called once). **Three red-proofs demonstrated:** pre-fix body → A1 FAIL (exact
`role: system` 403); containment-skip mutation → B1 FAIL (protected disk becomes ejectable); Observe-error
fallback → C3 FAIL (permissive). Existing `TestEject_RoleGated`/`TestDecommission_*` green UNMODIFIED. Full
gate `go build/vet/test ./...` = PASS.
## Tests + red-proofs
`pool_test.go`: `PoolAddVMID` PUT shape + idempotent-on-already-member + real-error-surfaces +
validation. `bringup_test.go`: re-asserts when `Pool!=""` (**red-proof:** pre-fix no-call →
`poolAdds=[]` FAIL, demonstrated + reverted), no-pool→no-call, pool-add-failure warns-but-passes
(liveness wins, guest kept). Full gate `go build/vet/test ./...` = PASS.
## Deploy + live acceptance (on `/mnt/teszt_enroll` ONLY)
Built `-X main.version=0.73.0` on 180, deployed to felhom-pve (`.bak-0.72.0` kept). `--version` = 0.73.0,
`systemctl is-active` = active, capabilities 56/56 degraded=0. Full F2 lifecycle through the real controller
endpoints: **eject → HTTP 200** (was 403), **decommission-anyway → HTTP 200** (`decommissioned:true`, was
403), post-decommission bind detached, **agent restart → drive stayed detached (no rebind)**, re-commission
(reconnect) → 200, **end state == pre-state** (bound on /dev/sdb, attached, durable `uuid:f2236136-ced7…`).
## Deploy + live acceptance (the headline)
Built `-X main.version=0.74.0`, deployed to felhom-pve (`.bak-0.73.0` kept); `--version` 0.74.0,
active, capabilities 56/56 degraded=0. **Part A** run (`pveum pool modify felhom --vms 9201`, per
operator go): 9201 now a pool member; `VM.Audit`+`VM.Allocate`+`VM.Backup` present on `/vms/9201`.
**Live `--selftest=restore-test` then PASSED for the first time:** `neutralizing source bind-mount
mountpoints … bind_mounts=2` → scratch 990000 restored + **verified boot+running** + **torn down
clean (no VM.Allocate 403)** in 4m35s. B3 confirmed as a cascade; restore-test needs no bind code.
## Not changed / observations
Only `roleForMountPath` + the new `storage.SameWholeDisk` helper. The pre-existing `VM.Audit` permission
ERROR in the agent log is unrelated (known A1 privilege gap). The `deviceRole`/`roleForMountPath`
unification refactor is deferred.
## Observations
- DR `bring-up -mode dr` has a SEPARATE real gap (no overrides, source guest gone) — scoped in
`felhom.eu/documentation/audits/SPIKE-dr-bindmount-source-2026-07-07.md`; recommended fix is a
small known-constant override reusing `bindMountOverrides` (mp8/mp9 are structural constants).
- The empty pool predated the campaign (07:41 403 before the 14:09 finale) — an earlier drill's
restore-over-existing; v0.74.0 stops the recurrence going forward.