From 326aa1ca17de8c6589a00795146e55e186ffbdd1 Mon Sep 17 00:00:00 2001
From: kisfenyo <nagyfenyvesi.viktor@gmail.com>
Date: Sun, 14 Jun 2026 17:52:26 +0200
Subject: [PATCH] docs(spike): decommission/migration feasibility (TASK B0)

Report-only feasibility spike: data-move layer (controller, proven live),
vestigial migrate button, self-serve POST /disks/decommission design,
re-bind gap (intent-unaware ReassertGuestBinds + no GuestBindStore.Remove),
soft-marker registry. Gates B1/B2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 ...SPIKE-decommission-migration-2026-06-14.md | 261 ++++++++++++++++++
 1 file changed, 261 insertions(+)
 create mode 100644 documentation/audits/SPIKE-decommission-migration-2026-06-14.md

diff --git a/documentation/audits/SPIKE-decommission-migration-2026-06-14.md b/documentation/audits/SPIKE-decommission-migration-2026-06-14.md
new file mode 100644
index 0000000..8d92711
--- /dev/null
+++ b/documentation/audits/SPIKE-decommission-migration-2026-06-14.md
@@ -0,0 +1,261 @@
+# SPIKE — Decommission / migration feasibility (2026-06-14)
+
+**Report-only. No feature code, no version bump.** GATE: B1/B2 do not start until the product owner
+reviews these findings and picks the forks at the end.
+
+**Product decision (locked, restated):** storage decommission is **self-serve — NO operator signature**.
+It is the customer's own drive; we warn (and offer migration) but do not gate on operator authorization.
+Decommission is **non-destructive** — un-enroll + drop the guest bind; **data stays on the drive**. It must
+**NOT** format/wipe — that remains the separate customer-confirmed wipe flow.
+
+Evidence is `file:line` + (live) where empirically checked on demo guest **9201** (host `felhom-pve`,
+192.168.0.162). The legacy bare-metal `controller/docker-compose.yml` was NOT trusted — the real deploy is
+the golden/bootstrap `docker run` in `felhom-agent/configs/build-golden.sh`.
+
+---
+
+## Q1 — Data-move layer (controller vs agent)
+
+**The controller's actual /mnt mount** — `felhom-agent/configs/build-golden.sh:179-186`:
+
+```
+docker run -d --name felhom-controller --restart unless-stopped ... \
+  -v /mnt:/mnt:rslave \
+  -v /var/run/docker.sock:/var/run/docker.sock \
+  "$IMAGE"
+```
+
+De-privileged (`build-golden.sh:176-178`: NO `--privileged`, no `/dev`, no `/etc/fstab`). The host pre-shares
+/mnt so agent binds propagate in (`build-golden.sh:173-175`: `mkdir -p /mnt; mount --rbind /mnt /mnt;
+mount --make-rshared /mnt`).
+
+**`rslave` is read-WRITE for file content.** `rslave` governs *mount-event propagation direction* (host→
+container only — the container can't mutate the guest mount tree), NOT file permissions. The bind has **no
+`:ro`**, so reading/writing file *content* under /mnt is allowed.
+
+**Live proof (inside the `felhom-controller` container on 9201):**
+
+```
+$ docker exec felhom-controller sh -c '... write/read/delete a marker under each enrolled path ...'
+== mount opts for /mnt ==
+/dev/mapper/pve-vm--9201--disk--0 /mnt ext4 rw,relatime,stripe=16 0 0
+-- writing /mnt/felhom-usb/.felhom-spike-marker --
+WRITE_OK / spike-1781452108 / DELETE_OK
+-- writing /mnt/sys_drive/felhom-data/.felhom-spike-marker --
+WRITE_OK / spike-1781452108 / DELETE_OK
+```
+
+Both enrolled storage paths (`/mnt/felhom-usb`, `/mnt/sys_drive/felhom-data`) are writable+deletable from
+inside the controller container.
+
+**Conclusion — chosen layer = CONTROLLER.** The controller already holds RW access to every enrolled drive
+under /mnt, so it can perform an app's data move (`cp -a /mnt/X/... → /mnt/Y/...`) **itself**, in-process,
+no agent round-trip for the bytes. The agent is needed only for the privileged side-effects that bracket the
+move: **unmount** (eject/decommission) and **intent** recording. This matches the existing app-data backup
+design (the controller already rsyncs between drives for Tier-2 off-drive copies).
+
+---
+
+## Q2 — "Migrate" button state
+
+`controller/internal/web/templates/settings.html:337`:
+
+```html
+<span class="btn btn-xs btn-outline" style="opacity:.45;cursor:not-allowed" title="Hamarosan">📦 Összes adat átköltöztetése</span>
+```
+
+**Vestigial.** It is a `<span>`, not a `<button>`/`<form>`; greyed (`opacity:.45`), `cursor:not-allowed`,
+`title="Hamarosan"` ("Coming soon"), **no `onclick`, no handler, no JS**. It renders only when
+`AppCount>0 && HasOtherPaths` (`settings.html:336`). There is **no** route, no controller handler, no
+`agentapi` method behind it. `Settings.SetDecommissioned` exists (`settings.go:821`) but is **never called**
+anywhere in the controller — the whole migrate/decommission round-trip is unwired.
+
+**Net-new wiring required (all new):**
+1. **Data-move executor** (controller) — per app on the source drive: quiesce/stop → `cp -a` source→target
+   under /mnt → verify → restart. (Layer proven in Q1.)
+2. **Deploy-path rewrite** — rewrite each moved app's `HDD_PATH` in its `app.yaml` to the new drive +
+   redeploy (`docker compose up -d`), so the app uses the new location. (Mirrors the v0.52 catalog
+   HDD_PATH work.)
+3. **Controller endpoint** — `POST /api/storage/migrate` (+ JS handler + progress UI replacing the span).
+4. **Source decommission** — after a successful move, call the new self-serve `POST /disks/decommission`
+   (Q3) on the now-empty source drive.
+5. **Registry mark** — `Settings.SetDecommissioned(source, target)` (today uncalled) to soft-mark the
+   source with `MigratedTo` (Q5).
+
+---
+
+## Q3 — Make decommission self-serve (the design)
+
+### Current path (operator-signed only)
+
+- **Class** — `internal/reconcile/classify.go:36` `ClassDecommission OpClass = "decommission"`.
+- **Gate verdict** — `classify.go:103-107`:
+  ```go
+  case ClassGuestDestroy, ClassStorageWipe, ClassRestoreOverwrite, ClassDecommission:
+      if prov.internalEvidence() { return Benign }
+      return Destructive
+  ```
+  i.e. decommission is **Destructive** → requires a verified, role-authorized, action-bound **operator
+  signature** (`internal/reconcile/gate.go:144-177`, `ReasonPendingSignature` without one).
+- **Executor** — `internal/signedjobs/decommission.go:54-79` `DecommissionExecutor.Execute`: the gate has
+  already burned the operator signature/nonce; it then only calls `intent.SetDecommissioned(durableID)`
+  (`decommission.go:75`). It does **not** unmount and does **not** touch the guest bind.
+- **Wiring** — `cmd/felhom-agent/main.go:437` `signedjobs.NewDecommissionExecutor(...)`, reachable **only**
+  via the signed-jobs runner (hub-queued operator op). `IntentDecommissioned` is reachable today **only**
+  this way. The controller has **no** endpoint and **no** `agentapi` method for it
+  (`internal/agentapi/client.go` has `EjectDisk` at :359 but no decommission).
+
+### The self-serve precedent to mirror: eject
+
+`internal/localapi/server.go:239` `mux.HandleFunc("POST /disks/eject", s.withGuest(s.handleDiskEject))` →
+`internal/localapi/disks.go:209 handleDiskEject`: **no signature** — `withGuest` per-guest-token self-scoping
+(`server.go:292-315`), role-gated to user-data (`disks.go:230-235`), records intent `"ejected"`
+(`disks.go:240`) then `Unmount` (`disks.go:241`). Confirmed self-serve, per-guest, sets intent without a
+signature.
+
+### New endpoint contract — `POST /disks/decommission` (NO CODE, contract only)
+
+| Aspect | Spec |
+|---|---|
+| **Route** | `mux.HandleFunc("POST /disks/decommission", s.withGuest(s.handleDiskDecommission))` in `internal/localapi/server.go` |
+| **Auth** | `withGuest` (per-guest bearer token → token's VMID is the only authority); `scopedFromBody` rejects a mismatched body `vmid` (403) — a guest acts only on its own drive. **No operator signature.** |
+| **Params** | `{vmid, where}` where `where` = the host mount path (e.g. `/mnt/felhom-usb`). The durable-id is **resolved server-side** via `durableIDForMount(where)` (`disks.go:697`) — never caller-supplied (same as `recordIntent`/eject). |
+| **Role gate** | user-data only: `roleForMountPath(where) != storage.RoleUserData` → 403 (mirror `disks.go:230-235`). Cannot decommission system/backup. Fails safe (unresolvable → protected → refused). |
+| **Actions** | 1) resolve durable-id; 2) `IntentStore.SetDecommissioned(id)` (already exists, `internal/storage/intent.go:84`); 3) `GuestBindStore.Remove(vmid, id)` (**NET-NEW method** — see Q4); 4) `Unmount(where)` (mirror eject — addresses the Q4 re-bind gap by leaving the drive un-host-mounted); 5) return dependent guests + decommissioned path. **NON-DESTRUCTIVE: no format/mkfs.** |
+| **Files touched (agent)** | `internal/localapi/server.go` (route), `internal/localapi/disks.go` (handler + a `recordDecommissionIntent` helper paralleling `recordIntent`), `internal/localapi/guestbindstore.go` (add `Remove`). |
+| **Files touched (controller)** | `internal/agentapi/client.go` (new `Decommission(where)` method), `internal/web/storage_handlers.go` (+ `server.go` route) for the customer flow, `internal/web/templates/settings.html` (real button + JS), and call `Settings.SetDecommissioned` on success (Q5). |
+
+### Security check (explicit)
+
+- **The hub cannot reach the agent local API.** The local API binds `local_api.listen_addr =
+  192.168.0.162:8443` (live `agent.json`) — the host's LAN/bridge address — over TLS with a leaf cert and
+  **per-guest bearer tokens** (`server.go:67-68,287-315`). The hub is remote (k3s) and the topology is
+  **outbound-only**: the agent POSTs host-reports to the hub; the hub "holds no data-plane role and **never
+  connects inbound to a box**" (`felhom.eu/CLAUDE.md`). The hub has no token. → A compromised hub **cannot**
+  issue a self-serve decommission. (It also can't issue the *operator-signed* one beyond queuing a blob the
+  gate must still cryptographically verify.)
+- **A compromised controller gains no new capability.** It already can `EjectDisk` (unmount its own
+  user-data drive, `agentapi/client.go:359`) and drive the **customer-confirmed wipe** which *formats* a
+  user-data drive (`storage_handlers.go:357` → agent `FormatDisk` → `gate.AuthorizeStorageWipe`). A
+  non-destructive decommission (unmount + intent + drop bind, **data left intact**) is strictly **less**
+  than the wipe it can already perform. The role-gate still blocks system/backup; self-scoping still blocks
+  other guests.
+
+### Bright line (does NOT wipe)
+
+Decommission as designed calls **no format/mkfs path** — only `Unmount` + `SetDecommissioned` +
+`GuestBindStore.Remove`. Drive data is left intact. The customer-confirmed **wipe** stays separate and
+untouched: `controller handleStorageWipe` (`storage_handlers.go:357`) → `agent FormatDisk` →
+`gate.AuthorizeStorageWipe` (`gate.go:209`). **How the wipe gate differs:** a **USER-DATA** wipe is
+**customer-confirmable today** — `gate.go:210-226`: allowed iff `Confirmed` *and* `ConfirmDurableID ==
+DeviceDurableID` (durable-id-bound), `Disposition: CustomerConfirmable`, **no operator signature**. Only
+**system/backup** wipe takes the operator-signature path (`gate.go:227-229`, `pending_signature` / 409). So
+a user-data wipe is *not* operator-409 — it is customer-confirmable; decommission is even softer (no
+destruction at all).
+
+### Recommendation — keep the operator-signed `DecommissionExecutor` (secondary / DR path)
+
+**KEEP it.** Trade-off: the self-serve endpoint covers the customer's normal "I'm taking my drive back" on a
+**reachable, healthy** guest. The operator-signed path is the **only** way to decommission a drive that is
+physically **ABSENT** — `DecommissionExecutor` needs no `/dev` resolution and explicitly supports
+decommissioning an absent drive (`decommission.go:37-39`) — or when the in-guest controller is
+unreachable/dead (operator/DR). Removing it would drop the absent-drive + DR capability for **zero** security
+gain (it's already gate-verified). The two are complementary: self-serve = live happy path, signed =
+absent/DR.
+
+---
+
+## Q4 — Re-bind gap (needed regardless of signed vs self-serve)
+
+- **`GuestBindStore` has no `Remove`.** `internal/localapi/guestbindstore.go`: `Record` (:58), `Guests`
+  (:75) — and nothing else. Confirmed.
+- **`ReassertGuestBinds` ignores intent.** `internal/localapi/disks.go:643`: it replays
+  `s.guestBinds.Guests()` and gates each durable-id only on presence in `mountByDurable`, which is built from
+  `s.storage.Observe(ctx)` — **mounted drives only** (`disks.go:649-654`, skip-if-`!present` at
+  `disks.go:673-677`). It never consults the `IntentStore`.
+- **Watchdog auto-mount is intent-gated.** `internal/storage/watchdog.go:174-178 reconcileAllowed`:
+  ```go
+  func (w *Watchdog) reconcileAllowed(t KnownTarget) bool {
+      if w.intent == nil { return true }
+      return w.intent.Get(t.DurableID) == IntentEnrolled
+  }
+  ```
+  So self-heal **host**-mounts only `enrolled` drives — an ejected *or* decommissioned drive is never host
+  auto-mounted. On a host **reboot** a decommissioned drive stays unmounted → absent from `storage.Observe`'s
+  mounted set → absent from `ReassertGuestBinds`' `mountByDurable` → not re-bound. The watchdog is the
+  primary guard.
+- **Live evidence of the latent gap.** On host `felhom-pve`:
+  ```
+  drive-intents.json: { "uuid:277a…914":"enrolled", "uuid:ce9d…0f9":"ejected", "uuid:da9e…e00":"enrolled" }
+  guest-binds.json:   { "9201": [ "uuid:ce9d…0f9", "uuid:da9e…e00" ] }
+  pct config 9201:    mp1: /mnt/felhom-usb/felhom-data,mp=/mnt/felhom-usb   (only one user-data mount)
+  ```
+  `uuid:ce9d…0f9` is **`ejected`** in the intent map yet **still recorded in `guest-binds.json`** — direct
+  proof that `GuestBindStore` never prunes. It is *not* currently re-bound only because it is unmounted
+  (eject ran `Unmount`); `ReassertGuestBinds` skips it on `!present`.
+- **The residual gap (the invariant at risk).** `ReassertGuestBinds` runs on **agent startup** and trusts
+  the *current mount state*, not intent. The planned decommission is **non-destructive** — the drive stays
+  put. If decommission set `IntentDecommissioned` **without** unmounting and the agent then restarted while
+  the drive was still host-mounted, `storage.Observe` would still see it → `ReassertGuestBinds` would
+  **re-bind it into the guest**, violating "a decommissioned drive must never auto-mount on reconnect."
+  (Could not be forced live without risking RomM's drive, and there is no self-serve intent-setter yet —
+  conclusion is by trace + the ejected-lingering-record above.)
+
+**Recommended fix — belt & suspenders:**
+- **Suspenders (must-have, single robust guard):** make `ReassertGuestBinds` **intent-aware** — skip any
+  durable-id whose `IntentStore.Get(id) != IntentEnrolled`. One guard closes **both** the decommissioned
+  case *and* the latent lingering-`ejected` record. (`s.intent` is already available to the local-API
+  server.)
+- **Belt (defense-in-depth + hygiene):** the decommission endpoint also (a) `Unmount`s the drive (mirror
+  eject — so it isn't host-mounted at reassert time), and (b) calls the new `GuestBindStore.Remove(vmid,id)`
+  so the record doesn't linger (also worth calling from `handleDiskEject` to fix the eject hygiene shown
+  above).
+
+Minimal-correct = the intent-aware `ReassertGuestBinds`. Recommend shipping **both** layers.
+
+---
+
+## Q5 — Registry on decommission
+
+`controller/internal/settings/settings.go:821-839 SetDecommissioned` **keeps** the entry (soft marker):
+
+```go
+s.StoragePaths[i].Decommissioned = true
+s.StoragePaths[i].DecommissionedAt = time.Now()...
+s.StoragePaths[i].MigratedTo = migratedTo
+s.StoragePaths[i].IsDefault = false
+s.StoragePaths[i].Schedulable = false
+```
+
+It is not removed. Report/UI already render it: `handlers.go:855-857` sets `MigratedToLabel`, and
+`settings.html:242-256` shows the decommissioned branch ("Adatok átköltöztetve ide: …" + an explicit
+"Eltávolítás a rendszerből" hard-remove via `/settings/storage/remove`). (Note: `SetDecommissioned` is not
+yet *called* anywhere — wiring is the net-new work in Q2/Q3.)
+
+**Recommendation — KEEP the soft marker.** Decisive given **A1 is now additive** (v0.64.0): A1 re-registers
+any deployed-app path missing from the registry **but skips any path present in ANY state, including
+`Decommissioned`** (proven by its companion test). The soft marker is therefore exactly what **blocks A1
+from resurrecting** a just-decommissioned drive. **Resurrection risk:** soft marker → **none** (skip-by-
+presence); hard-remove → **real** — if any app still referenced the old path (or a stale `app.yaml`
+`HDD_PATH` lingered), additive discovery would re-add it as a fresh, active, default-eligible path. The soft
+marker also preserves the `MigratedTo`/`At` audit trail the UI renders; the user's explicit "Eltávolítás a
+rendszerből" remains the deliberate hard-remove when they're truly done.
+
+---
+
+## Summary table
+
+| Q | Finding | file:line | Recommendation |
+|---|---|---|---|
+| Q1 | Controller mount is `-v /mnt:/mnt:rslave`, de-privileged; `rslave` is RW for content. **Live:** controller wrote+deleted markers under both enrolled /mnt paths. | `build-golden.sh:184` (`:173-178`); live 9201 | **Data move = CONTROLLER** (in-process `cp -a` between /mnt drives); agent only for unmount + intent. |
+| Q2 | Migrate button is a vestigial greyed `<span>`, no handler/route/JS; `SetDecommissioned` never called. | `settings.html:337`; `settings.go:821` | Build the whole round-trip: data-move executor, HDD_PATH rewrite+redeploy, `POST /api/storage/migrate`, source decommission, registry mark. |
+| Q3 | Decommission today = operator-signed only (Destructive in `Classify`); executor only sets intent; controller has no endpoint/agentapi method. Eject is the self-serve precedent. | `classify.go:36,103-107`; `decommission.go:54-79`; `main.go:437`; eject `server.go:239`+`disks.go:209` | Add self-serve `POST /disks/decommission` (withGuest, user-data role-gate, durable-id resolved server-side, intent+bind-remove+unmount, **no wipe**). Hub can't reach local API (8443 LAN, per-guest token, outbound-only); compromised controller gains nothing beyond eject/wipe. **Keep** operator-signed path for absent-drive/DR. |
+| Q4 | `GuestBindStore` has no `Remove`; `ReassertGuestBinds` ignores intent; watchdog gates host-mount to `enrolled`. **Live:** an `ejected` drive still sits in `guest-binds.json`. Gap: a decommissioned-but-still-mounted drive would be re-bound on agent restart. | `guestbindstore.go` (no Remove); `disks.go:643,649-654,673-677`; `watchdog.go:174-178`; live intent/bind state | **Both:** intent-aware `ReassertGuestBinds` (skip non-`enrolled`) [must-have] + decommission endpoint `Unmount` + new `GuestBindStore.Remove` [hygiene/DiD]. |
+| Q5 | `SetDecommissioned` keeps the entry as a soft marker (Decommissioned=true, Schedulable/IsDefault=false, MigratedTo); UI renders it. | `settings.go:821-839`; `handlers.go:855-857`; `settings.html:242-256` | **Keep soft marker** — it blocks A1-additive resurrection (skip-by-presence) + preserves audit. Hard-remove would risk re-registration if any reference lingers. |
+
+## Forks needing the product owner's call (GATE)
+
+1. **Operator-signed decommission** — keep as DR/absent-drive path (**recommended**) vs remove. → *recommend keep.*
+2. **Q4 fix shape** — intent-aware `ReassertGuestBinds` only, vs `GuestBindStore.Remove` only, vs **both** (**recommended**). → *recommend both.*
+3. **Data-move layer** — controller-side `cp -a` (**recommended**, proven live) vs agent-side mover. → *recommend controller.*
+4. **Decommission = unmount?** — endpoint unmounts the drive (mirror eject, so it's physically removable; **recommended**) vs leaves it host-mounted. → *recommend unmount.*