Files
felhom.eu/documentation/audits/SPIKE-pool-scoped-acl-2026-07-01.md
T
2026-07-01 13:02:16 +02:00

12 KiB
Raw Blame History

SPIKE — pool-scoped ACL for the FelhomAgent token (2026-07-01)

STATUS: IMPLEMENTED in agent v0.53.0 (RestoreLXCOptions.Pool / reconcile.DefaultPool, both restore sites) + host-install script v1.6.0 (3-role scoped ACL, --rescope-acl, uninstall cleanup). The demo host felhom-pve was live-migrated (agent stopped → --rescope-acl → agent v0.53.0 → start) and ALL residual gates passed under the scoped token: PBS backup+restore (residual #1) and restore-test into pool (residual #2) both succeed; blast-radius 403 confirmed on out-of-pool guests + ungranted storage. See the felhom.eu + felhom-agent REPORTs.

Class: SPIKE (empirical validation; no product code). Repos read: felhom-agent 33e3443 (v0.52.0, read-only). Host: felhom-pve (192.168.0.162, node demo-felhom, PVE 9.2.2, single-node). Verdict: PASS — the agent's 16 privileges CAN be scoped from / to /pool/felhom + /storage/<targets> (+Pool.Allocate) without breaking the provision/backup/guest flow, and the scoped token is provably contained to the pool. One caveat for 3b: the Datastore scope must include all agent-touched storages (notably the offsite felhom-pbs), not just local/local-lvm.

The live FelhomAgent role / felhom-agent@pve token / its / ACL / guest 9201 were never touched; all testing used a throwaway felhom-spike@pve!spike token + granular FelhomSpike* roles + scratch vmid 9300, all removed at the end. Live agent verified healthy afterwards.


1. Confirmed real storage names (felhom-pve)

From the live agent config + pvesm:

Role in the flow Storage Notes
archive (golden read source) local golden = local:backup/vzdump-lxc-9100-2026_06_27-11_42_51.tar.zst
restore (rootfs write target) local-lvm backup.restore_storage
dump (local backup write) local backup.local_backup_target
offsite dump (default DR target) felhom-pbs PBS, 2009/9239 GiB used — actively used; agent writes here
enrolled drives felhom-usb, felhom-flash dir storages; agent enumerates (Audit); not vzdump targets today

Finding: archive and local-dump are the SAME storage (local). The full set of storages the agent touches is 5, not 2 — see §7 for the 3b implication.

2. Privilege-use map — empirical column (object path PROVEN sufficient)

All ops run as the spike token (Authorization: PVEAPIToken=felhom-spike@pve!spike=…) against https://127.0.0.1:8006/api2/json. "Proven-sufficient path" = where the grant actually satisfied PVE.

Agent op Endpoint Priv Proven-sufficient object Test
RestoreLXC (+pool) POST /nodes/{n}/lxc restore=1 pool=felhom VM.Allocate + Pool.Allocate /pool/felhom T1 200
" (disks) " Datastore.AllocateSpace /storage/local-lvm T1
" (archive read) " Datastore.Audit/read /storage/local T1
task status/log GET /nodes/{n}/tasks/{upid}/{status,log} Sys.Audit / T2 200 (stopped OK)
SetConfig PUT …/config VM.Config.{Disk,CPU,Memory,Network,Options} /pool/felhom T3
ResizeLXC PUT …/resize VM.Config.Disk /pool/felhom T3
Snapshot / Delete POST/DELETE …/snapshot[/{s}] VM.Snapshot /pool/felhom T3
RollbackSnapshot POST …/snapshot/{s}/rollback VM.Snapshot.Rollback /pool/felhom T3
Start / Stop POST …/status/{start,stop} VM.PowerMgmt /pool/felhom T3 (stop 200 once running)
guest reads GET …/status/current,…/config,…/snapshot VM.Audit /pool/felhom T3
Vzdump POST /nodes/{n}/vzdump VM.Backup + Datastore.AllocateSpace guest /pool/felhom + /storage/local T4 (OK)
DestroyLXC DELETE /nodes/{n}/lxc/{id} purge=1 VM.Allocate + Datastore.Allocate /pool/felhom + /storage/* T6 (OK)
Version GET /version (auth only) T5
ListNodes / NodeStatus GET /nodes,/nodes/{n}/status Sys.Audit / T5
ListLXC GET /nodes/{n}/lxc VM.Audit /pool/felhom (returns filtered list) T5 (only pool guests)
ListStorage / NodeStorage GET /storage,/nodes/{n}/storage Datastore.Audit /storage/<s> (returns filtered list) T5 (no box-wide grant needed)
StorageContent GET /nodes/{n}/storage/{s}/content Datastore.Audit /storage/<s> T5

3. CRUX verdict (T1) — create-into-pool WORKS at /pool/felhom

POST /nodes/demo-felhom/lxc with restore=1, vmid=9300 (brand-new), ostemplate=<golden>, storage=local-lvm, pool=felhom → HTTP 200 + UPID:…:vzrestore:9300:felhom-spike@pve!spike:.

A fresh, not-yet-existing vmid CAN be allocated into the pool with VM.Allocate + Pool.Allocate granted only at /pool/felhomno VM.Allocate at /vms or / was needed. This is the load-bearing result: the intended design works, no fallback (the §3 fallbacks a/b/c were not needed). The restore then ran to stopped OK and the guest was a pool member (visible only to the scoped token).

Mechanism finding: pveum acl modify <path> -role <r> grants the whole role at that path — you cannot split one role's privileges across paths. So 3b needs ≥2 roles (a scoped set + a tiny box-wide read role), not the single /-applied role of today.

Recommended (validated except the felhom-pbs write grant — see §7):

# Role A — guest-scoped (applied at /pool/felhom, to BOTH user and token):
pveum role add FelhomAgentGuest -privs "VM.Allocate VM.Audit VM.Config.Disk VM.Config.CPU \
  VM.Config.Memory VM.Config.Network VM.Config.Options VM.PowerMgmt VM.Snapshot \
  VM.Snapshot.Rollback VM.Backup Pool.Allocate"
pveum acl modify /pool/felhom -user felhom-agent@pve            -role FelhomAgentGuest
pveum acl modify /pool/felhom -token 'felhom-agent@pve!agent'   -role FelhomAgentGuest

# Role B — storage write (applied at EACH agent-touched storage):
pveum role add FelhomAgentStore -privs "Datastore.Allocate Datastore.AllocateSpace Datastore.Audit"
for S in local local-lvm felhom-pbs ; do   # + felhom-usb/felhom-flash IF the agent ever vzdumps to them
  pveum acl modify /storage/$S -user felhom-agent@pve          -role FelhomAgentStore
  pveum acl modify /storage/$S -token 'felhom-agent@pve!agent' -role FelhomAgentStore
done

# Role C — box-wide reads (applied at /):
pveum role add FelhomAgentBase -privs "Sys.Audit SDN.Use"
pveum acl modify / -user felhom-agent@pve            -role FelhomAgentBase
pveum acl modify / -token 'felhom-agent@pve!agent'   -role FelhomAgentBase

Notes proven empirically:

  • Datastore.Audit does NOT need to be box-wideGET /storage returned a filtered 2-item list under the per-storage grants (200, showed only local+local-lvm). Keeping it per-storage is enough for the agent's ListStorage/NodeStorage/StorageContent.
  • Privsep intersection: grant each role to BOTH the user AND the token (as the live install already does at /), else the token's effective perms are empty.
  • Alternative if dynamic drive enrollment makes per-storage brittle: put Datastore.Audit at / (read-only, harmless) and keep only Datastore.Allocate/AllocateSpace per-storage. Not required by the tested flow; a robustness option.

5. Blast-radius proof (T7) — the scoped token CANNOT reach out-of-pool guests

Against guests NOT in the felhom pool, as the spike token:

GET  /nodes/demo-felhom/lxc/9001/config          -> 403  Permission check failed (/vms/9001, VM.Audit)
GET  /nodes/demo-felhom/lxc/9001/status/current  -> 403  Permission check failed (/vms/9001, VM.Audit)
PUT  /nodes/demo-felhom/lxc/9001/config          -> 403  Permission check failed (/vms/9001, VM.Config.Disk|VM.Config.CPU|…)
POST /nodes/demo-felhom/lxc/9100/status/start    -> 403  Permission check failed (/vms/9100, VM.PowerMgmt)

PVE checks VM.* against /vms/<vmid> and resolves the /pool/felhom grant only for pool members; non-members get no grant → 403. Exactly the containment 3b wants. (Guest 9201 IS reachable by the scoped token because 3a already adopted it into the felhom pool — expected and correct: pool membership = managed.)

6. Per-test results (T1T7)

Test Op Result
T1 restore-into-pool (fresh vmid 9300, pool=felhom) 200 + UPID (CRUX passes at /pool/felhom)
T2 task status + log read (own UPID) 200 (stopped OK), log 200
T3 config, resize, snapshot+rollback+delete, start, stop, reads all 200 (stop=500 only when called mid-start; 200 once running → runtime, not auth)
T4 vzdump 9300 → local 200 + task OK
T5 /version,/nodes,/nodes/{n}/status,/storage,/nodes/{n}/storage,{s}/content,/nodes/{n}/lxc all 200; /storage + /lxc return filtered lists
T6 destroy 9300 (purge) 200 + task OK, guest gone
T7 out-of-pool guest ops (9001, 9100) 403 (contained)

No ACL widening was needed — the proposed layout passed the entire flow on the first pass (for the local/local-lvm storage set).

7. 3b-impl scope preview

Agent (felhom-agent):

  • RestoreLXCOptions gains a Pool string field; RestoreLXC sets v.Set("pool", opts.Pool) when non-empty (mutate.go). Thread Pool through BringUpSpecrunSelftestProvision/RunBringUp so the provision restore lands in felhom. (Restore-into-pool is proven; this is the only new API param.)
  • No other agent op needs change — all steady-state ops already target the guest, which is in the pool.

Script (felhom-host-install.sh step_token, L~793 the two / grants):

  • Replace PVE_PRIVS broad-at-/ grant with the 3-role layout of §4 (Guest@/pool/felhom, Store@each /storage/<s>, Base@/). Add Pool.Allocate (only in the Guest role).
  • Storage list must be local local-lvm felhom-pbs at minimum (the offsite PBS default is a write target) — a plain local/local-lvm scope WOULD break the agent's offsite backup.
  • --uninstall ACL removal (Spec 1 step 5) must delete the new scoped grants (3 roles × {user,token} × their paths) instead of the two / grants, and delete the 3 roles (after their ACLs).
  • --adopt-pool (3a) already puts existing guests in the pool → a retrofitted box's guest becomes reachable by the scoped token automatically once 3b ships.

8. NOT validated by this spike — 3b-impl / follow-up must verify

  • felhom-pbs (offsite) backup under a Store grant on /storage/felhom-pbs. T4 validated vzdump to local only. Since PBS is the DEFAULT DR target, 3b-impl must include /storage/felhom-pbs in the Store scope AND validate a PBS backup + PBS verify + a restore-from-PBS under the scoped token before shipping. (High confidence it works — same Datastore.AllocateSpace mechanism — but unproven here.)
  • Drive storages (felhom-usb/felhom-flash): the agent enumerates them (Datastore.Audit) and bind-mounts their data host-side (not a Datastore priv). If any future path vzdumps to a drive, that storage needs a Store grant. Dynamic drive enrollment may favor the "Datastore.Audit at /" variant.
  • Multi-node cluster: felhom-pve is single-node. Pools + ACLs are cluster-wide in PVE, but 3b-impl must verify on the colleague's box (if a cluster) that a restore targeting a specific node with a /pool/felhom grant behaves identically (node is in the URL, not the ACL path — expected fine).
  • restore-test flow (--selftest=restore-test) under scope: uses the 9990xx scratch band, which is NOT in felhom pool → its restore/destroy would 403 under the scoped token. 3b must either add the restore-test scratch guests to the pool, or grant the scratch-band a scope. (Flagged — this is a real gap the core provision flow didn't exercise.)

9. Cleanup confirmation

All throwaway objects removed and verified: 0 spike ACL entries, 0 FelhomSpike* roles, 0 felhom-spike users, scratch guest 9300 destroyed, its scratch vzdump freed, /root/.spike-token deleted. Live state intact: felhom-agent active, --selftest=read OK (sees all 5 storages + guests), FelhomAgent role present, guest 9201 running and still the sole felhom pool member. The spike token secret was never written to any committed file.