185 lines
12 KiB
Markdown
185 lines
12 KiB
Markdown
# SPIKE — pool-scoped ACL for the FelhomAgent token (2026-07-01)
|
||
|
||
> **STATUS: IMPLEMENTED** in agent **v0.53.0** (`RestoreLXCOptions.Pool` / `reconcile.DefaultPool`, both
|
||
> restore sites) + host-install script **v1.6.0** (3-role scoped ACL, `--rescope-acl`, uninstall
|
||
> cleanup). The demo host felhom-pve was live-migrated (agent stopped → `--rescope-acl` → agent v0.53.0
|
||
> → start) and ALL residual gates passed under the scoped token: **PBS backup+restore (residual #1)**
|
||
> and **restore-test into pool (residual #2)** both succeed; blast-radius 403 confirmed on out-of-pool
|
||
> guests + ungranted storage. See the felhom.eu + felhom-agent REPORTs.
|
||
|
||
**Class:** SPIKE (empirical validation; no product code). **Repos read:** felhom-agent `33e3443` (v0.52.0,
|
||
read-only). **Host:** felhom-pve (192.168.0.162, node `demo-felhom`, PVE 9.2.2, **single-node**).
|
||
**Verdict: PASS** — the agent's 16 privileges CAN be scoped from `/` to `/pool/felhom` +
|
||
`/storage/<targets>` (+`Pool.Allocate`) without breaking the provision/backup/guest flow, and the scoped
|
||
token is provably contained to the pool. One caveat for 3b: the Datastore scope must include **all**
|
||
agent-touched storages (notably the offsite `felhom-pbs`), not just `local`/`local-lvm`.
|
||
|
||
The live `FelhomAgent` role / `felhom-agent@pve` token / its `/` ACL / guest 9201 were **never touched**;
|
||
all testing used a throwaway `felhom-spike@pve!spike` token + granular `FelhomSpike*` roles + scratch
|
||
vmid 9300, all removed at the end. Live agent verified healthy afterwards.
|
||
|
||
---
|
||
|
||
## 1. Confirmed real storage names (felhom-pve)
|
||
|
||
From the live agent config + `pvesm`:
|
||
|
||
| Role in the flow | Storage | Notes |
|
||
|---|---|---|
|
||
| **archive** (golden read source) | `local` | golden = `local:backup/vzdump-lxc-9100-2026_06_27-11_42_51.tar.zst` |
|
||
| **restore** (rootfs write target) | `local-lvm` | `backup.restore_storage` |
|
||
| **dump** (local backup write) | `local` | `backup.local_backup_target` |
|
||
| **offsite dump** (default DR target) | `felhom-pbs` | PBS, 2009/9239 GiB used — actively used; **agent writes here** |
|
||
| enrolled drives | `felhom-usb`, `felhom-flash` | dir storages; agent enumerates (Audit); not vzdump targets today |
|
||
|
||
**Finding:** archive and local-dump are the SAME storage (`local`). The full set of storages the agent
|
||
touches is **5**, not 2 — see §7 for the 3b implication.
|
||
|
||
## 2. Privilege-use map — empirical column (object path PROVEN sufficient)
|
||
|
||
All ops run **as the spike token** (`Authorization: PVEAPIToken=felhom-spike@pve!spike=…`) against
|
||
`https://127.0.0.1:8006/api2/json`. "Proven-sufficient path" = where the grant actually satisfied PVE.
|
||
|
||
| Agent op | Endpoint | Priv | Proven-sufficient object | Test |
|
||
|---|---|---|---|---|
|
||
| RestoreLXC (**+pool**) | POST `/nodes/{n}/lxc` restore=1 pool=felhom | VM.Allocate + Pool.Allocate | **`/pool/felhom`** | T1 ✅ 200 |
|
||
| " (disks) | " | Datastore.AllocateSpace | `/storage/local-lvm` | T1 ✅ |
|
||
| " (archive read) | " | Datastore.Audit/read | `/storage/local` | T1 ✅ |
|
||
| task status/log | GET `/nodes/{n}/tasks/{upid}/{status,log}` | Sys.Audit | **`/`** | T2 ✅ 200 (`stopped OK`) |
|
||
| SetConfig | PUT `…/config` | VM.Config.{Disk,CPU,Memory,Network,Options} | `/pool/felhom` | T3 ✅ |
|
||
| ResizeLXC | PUT `…/resize` | VM.Config.Disk | `/pool/felhom` | T3 ✅ |
|
||
| Snapshot / Delete | POST/DELETE `…/snapshot[/{s}]` | VM.Snapshot | `/pool/felhom` | T3 ✅ |
|
||
| RollbackSnapshot | POST `…/snapshot/{s}/rollback` | VM.Snapshot.Rollback | `/pool/felhom` | T3 ✅ |
|
||
| Start / Stop | POST `…/status/{start,stop}` | VM.PowerMgmt | `/pool/felhom` | T3 ✅ (stop 200 once running) |
|
||
| guest reads | GET `…/status/current`,`…/config`,`…/snapshot` | VM.Audit | `/pool/felhom` | T3 ✅ |
|
||
| Vzdump | POST `/nodes/{n}/vzdump` | VM.Backup + Datastore.AllocateSpace | guest `/pool/felhom` + `/storage/local` | T4 ✅ (`OK`) |
|
||
| DestroyLXC | DELETE `/nodes/{n}/lxc/{id}` purge=1 | VM.Allocate + Datastore.Allocate | `/pool/felhom` + `/storage/*` | T6 ✅ (`OK`) |
|
||
| Version | GET `/version` | (auth only) | — | T5 ✅ |
|
||
| ListNodes / NodeStatus | GET `/nodes`,`/nodes/{n}/status` | Sys.Audit | `/` | T5 ✅ |
|
||
| ListLXC | GET `/nodes/{n}/lxc` | VM.Audit | `/pool/felhom` (returns **filtered** list) | T5 ✅ (only pool guests) |
|
||
| ListStorage / NodeStorage | GET `/storage`,`/nodes/{n}/storage` | Datastore.Audit | `/storage/<s>` (returns **filtered** list) | T5 ✅ (**no box-wide grant needed**) |
|
||
| StorageContent | GET `/nodes/{n}/storage/{s}/content` | Datastore.Audit | `/storage/<s>` | T5 ✅ |
|
||
|
||
## 3. CRUX verdict (T1) — create-into-pool WORKS at `/pool/felhom`
|
||
|
||
`POST /nodes/demo-felhom/lxc` with `restore=1, vmid=9300 (brand-new), ostemplate=<golden>,
|
||
storage=local-lvm, pool=felhom` **→ HTTP 200** + `UPID:…:vzrestore:9300:felhom-spike@pve!spike:`.
|
||
|
||
A fresh, not-yet-existing vmid CAN be allocated **into** the pool with `VM.Allocate` + `Pool.Allocate`
|
||
granted only at `/pool/felhom` — **no** `VM.Allocate` at `/vms` or `/` was needed. This is the load-bearing
|
||
result: the intended design works, no fallback (the §3 fallbacks a/b/c were not needed). The restore then
|
||
ran to `stopped OK` and the guest was a pool member (visible only to the scoped token).
|
||
|
||
## 4. Recommended final ACL layout for 3b-impl
|
||
|
||
**Mechanism finding:** `pveum acl modify <path> -role <r>` grants the **whole role** at that path — you
|
||
cannot split one role's privileges across paths. So 3b needs **≥2 roles** (a scoped set + a tiny
|
||
box-wide read role), not the single `/`-applied role of today.
|
||
|
||
Recommended (validated except the felhom-pbs write grant — see §7):
|
||
|
||
```
|
||
# Role A — guest-scoped (applied at /pool/felhom, to BOTH user and token):
|
||
pveum role add FelhomAgentGuest -privs "VM.Allocate VM.Audit VM.Config.Disk VM.Config.CPU \
|
||
VM.Config.Memory VM.Config.Network VM.Config.Options VM.PowerMgmt VM.Snapshot \
|
||
VM.Snapshot.Rollback VM.Backup Pool.Allocate"
|
||
pveum acl modify /pool/felhom -user felhom-agent@pve -role FelhomAgentGuest
|
||
pveum acl modify /pool/felhom -token 'felhom-agent@pve!agent' -role FelhomAgentGuest
|
||
|
||
# Role B — storage write (applied at EACH agent-touched storage):
|
||
pveum role add FelhomAgentStore -privs "Datastore.Allocate Datastore.AllocateSpace Datastore.Audit"
|
||
for S in local local-lvm felhom-pbs ; do # + felhom-usb/felhom-flash IF the agent ever vzdumps to them
|
||
pveum acl modify /storage/$S -user felhom-agent@pve -role FelhomAgentStore
|
||
pveum acl modify /storage/$S -token 'felhom-agent@pve!agent' -role FelhomAgentStore
|
||
done
|
||
|
||
# Role C — box-wide reads (applied at /):
|
||
pveum role add FelhomAgentBase -privs "Sys.Audit SDN.Use"
|
||
pveum acl modify / -user felhom-agent@pve -role FelhomAgentBase
|
||
pveum acl modify / -token 'felhom-agent@pve!agent' -role FelhomAgentBase
|
||
```
|
||
|
||
Notes proven empirically:
|
||
- **`Datastore.Audit` does NOT need to be box-wide** — `GET /storage` returned a *filtered* 2-item list
|
||
under the per-storage grants (200, showed only `local`+`local-lvm`). Keeping it per-storage is enough
|
||
for the agent's ListStorage/NodeStorage/StorageContent.
|
||
- **Privsep intersection:** grant each role to BOTH the user AND the token (as the live install already
|
||
does at `/`), else the token's effective perms are empty.
|
||
- Alternative if dynamic drive enrollment makes per-storage brittle: put **`Datastore.Audit` at `/`**
|
||
(read-only, harmless) and keep only `Datastore.Allocate`/`AllocateSpace` per-storage. Not required by
|
||
the tested flow; a robustness option.
|
||
|
||
## 5. Blast-radius proof (T7) — the scoped token CANNOT reach out-of-pool guests
|
||
|
||
Against guests NOT in the `felhom` pool, as the spike token:
|
||
|
||
```
|
||
GET /nodes/demo-felhom/lxc/9001/config -> 403 Permission check failed (/vms/9001, VM.Audit)
|
||
GET /nodes/demo-felhom/lxc/9001/status/current -> 403 Permission check failed (/vms/9001, VM.Audit)
|
||
PUT /nodes/demo-felhom/lxc/9001/config -> 403 Permission check failed (/vms/9001, VM.Config.Disk|VM.Config.CPU|…)
|
||
POST /nodes/demo-felhom/lxc/9100/status/start -> 403 Permission check failed (/vms/9100, VM.PowerMgmt)
|
||
```
|
||
|
||
PVE checks `VM.*` against `/vms/<vmid>` and resolves the `/pool/felhom` grant **only for pool members**;
|
||
non-members get no grant → 403. Exactly the containment 3b wants. (Guest **9201** IS reachable by the
|
||
scoped token because 3a already adopted it into the `felhom` pool — expected and correct: pool membership
|
||
= managed.)
|
||
|
||
## 6. Per-test results (T1–T7)
|
||
|
||
| Test | Op | Result |
|
||
|---|---|---|
|
||
| **T1** | restore-into-pool (fresh vmid 9300, pool=felhom) | ✅ **200** + UPID (CRUX passes at `/pool/felhom`) |
|
||
| **T2** | task status + log read (own UPID) | ✅ 200 (`stopped OK`), log 200 |
|
||
| **T3** | config, resize, snapshot+rollback+delete, start, stop, reads | ✅ all 200 (stop=500 only when called mid-start; 200 once running → runtime, not auth) |
|
||
| **T4** | vzdump 9300 → `local` | ✅ 200 + task `OK` |
|
||
| **T5** | /version,/nodes,/nodes/{n}/status,/storage,/nodes/{n}/storage,{s}/content,/nodes/{n}/lxc | ✅ all 200; /storage + /lxc return **filtered** lists |
|
||
| **T6** | destroy 9300 (purge) | ✅ 200 + task `OK`, guest gone |
|
||
| **T7** | out-of-pool guest ops (9001, 9100) | ✅ **403** (contained) |
|
||
|
||
**No ACL widening was needed** — the proposed layout passed the entire flow on the first pass (for the
|
||
`local`/`local-lvm` storage set).
|
||
|
||
## 7. 3b-impl scope preview
|
||
|
||
**Agent (`felhom-agent`):**
|
||
- `RestoreLXCOptions` gains a `Pool string` field; `RestoreLXC` sets `v.Set("pool", opts.Pool)` when
|
||
non-empty (mutate.go). Thread `Pool` through `BringUpSpec` → `runSelftestProvision`/`RunBringUp` so the
|
||
provision restore lands in `felhom`. (Restore-into-pool is proven; this is the only new API param.)
|
||
- No other agent op needs change — all steady-state ops already target the guest, which is in the pool.
|
||
|
||
**Script (`felhom-host-install.sh` step_token, L~793 the two `/` grants):**
|
||
- Replace `PVE_PRIVS` broad-at-`/` grant with the 3-role layout of §4 (Guest@/pool/felhom,
|
||
Store@each `/storage/<s>`, Base@/). Add `Pool.Allocate` (only in the Guest role).
|
||
- Storage list must be **`local local-lvm felhom-pbs`** at minimum (the offsite PBS default is a write
|
||
target) — a plain `local`/`local-lvm` scope WOULD break the agent's offsite backup.
|
||
- `--uninstall` ACL removal (Spec 1 step 5) must delete the new scoped grants (3 roles × {user,token} ×
|
||
their paths) instead of the two `/` grants, and delete the 3 roles (after their ACLs).
|
||
- `--adopt-pool` (3a) already puts existing guests in the pool → a retrofitted box's guest becomes
|
||
reachable by the scoped token automatically once 3b ships.
|
||
|
||
## 8. NOT validated by this spike — 3b-impl / follow-up must verify
|
||
|
||
- **`felhom-pbs` (offsite) backup under a Store grant on `/storage/felhom-pbs`.** T4 validated vzdump to
|
||
`local` only. Since PBS is the DEFAULT DR target, 3b-impl must include `/storage/felhom-pbs` in the
|
||
Store scope AND validate a PBS backup + PBS verify + a restore-from-PBS under the scoped token before
|
||
shipping. (High confidence it works — same Datastore.AllocateSpace mechanism — but unproven here.)
|
||
- **Drive storages** (`felhom-usb`/`felhom-flash`): the agent enumerates them (Datastore.Audit) and
|
||
bind-mounts their data host-side (not a Datastore priv). If any future path vzdumps to a drive, that
|
||
storage needs a Store grant. Dynamic drive enrollment may favor the "Datastore.Audit at `/`" variant.
|
||
- **Multi-node cluster:** felhom-pve is single-node. Pools + ACLs are cluster-wide in PVE, but 3b-impl
|
||
must verify on the colleague's box (if a cluster) that a restore targeting a specific node with a
|
||
`/pool/felhom` grant behaves identically (node is in the URL, not the ACL path — expected fine).
|
||
- **restore-test flow** (`--selftest=restore-test`) under scope: uses the 9990xx scratch band, which is
|
||
NOT in `felhom` pool → its restore/destroy would 403 under the scoped token. **3b must either add the
|
||
restore-test scratch guests to the pool, or grant the scratch-band a scope.** (Flagged — this is a real
|
||
gap the core provision flow didn't exercise.)
|
||
|
||
## 9. Cleanup confirmation
|
||
|
||
All throwaway objects removed and verified: **0** spike ACL entries, **0** `FelhomSpike*` roles, **0**
|
||
`felhom-spike` users, scratch guest 9300 destroyed, its scratch vzdump freed, `/root/.spike-token`
|
||
deleted. Live state intact: `felhom-agent` **active**, `--selftest=read` **OK** (sees all 5 storages +
|
||
guests), `FelhomAgent` role present, guest **9201 running** and still the sole `felhom` pool member. The
|
||
spike token secret was never written to any committed file.
|