docs: v0.73.0 REPORT + CONTEXT — F2 mount-role fallback closed, live-validated

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
This commit is contained in:
2026-07-06 20:41:58 +02:00
parent 7545af8a2c
commit e04b75e1f8
2 changed files with 45 additions and 76 deletions
+8
View File
@@ -5,6 +5,14 @@
## Current
- **v0.73.0** (2026-07-06) — **F2 mount-role fallback CLOSED** (LIVE on felhom-pve). `roleForMountPath`
gained a mount-table fallback (Impl-2b style): a bind-mounted RAW enrolled user-data drive is not a PVE
storage, so it fail-safe'd to `system` and the eject/decommission gates 403'd EVERY user-data drive
(campaign F2, `where=/mnt/teszt_enroll role=system`). Device-keyed classification + whole-disk containment
(`storage.SameWholeDisk`); Observe-error keeps the fail-safe BEFORE the fallback. Only `roleForMountPath`
touched. Live-proven full lifecycle on teszt_enroll (eject/decommission 200, no-rebind across restart,
end==pre). OPEN follow-up: the `deviceRole`/`roleForMountPath` unification refactor (deferred).
- **v0.72.0** (2026-07-05) — **OOB operator access (merged E1+H1)** — TASK H1, provenance both
`SPIKE-{felhom-sshd,oob-wg-operator-peer}-2026-07-05`. Operator `/32` RENDERED into wg-felhom
AllowedIPs (survives self-heal, [OF-1]); dedicated `internal/felhomsshd` (port claim + config
+37 -76
View File
@@ -1,82 +1,43 @@
# REPORT — TASK H1: OOB operator access (felhom-agent half) — v0.72.0
# REPORT — v0.73.0: F2 mount-role fallback (enrolled user-data drives ejectable/decommissionable)
**Baseline:** felhom-agent `main` @ `4b7c5bf` (v0.71.0) → **v0.72.0** (final binary sha
`f72c2fed…`). felhom.eu @ `a3ee93e` → hub v0.35.0. Trunk-based, direct to `main`. Provenance: BOTH
`SPIKE-felhom-sshd-2026-07-05` [SF-n] and `SPIKE-oob-wg-operator-peer-2026-07-05` [OF-n].
**Date:** 2026-07-06 · **Class:** risky/supervised (loosens a role gate; bounded by containment red-proofs).
**Baseline:** `main` @ `26664d6` (v0.72.0) → `7545af8`. Part A of the pilot-blockers bundle (B = controller
v0.102.0, C = catalog healthcheck sweep). Provenance: campaign F2 + RERUN addendum.
## What shipped (this repo)
## The bug
`roleForMountPath` (`internal/localapi/disks.go`) resolved a mount's protection role ONLY from the PVE
storage view (`Observe`). A bind-mounted RAW enrolled user-data drive is not a PVE storage → no MountPath
match → fail-safe `RoleSystem` → the eject (disks.go:353) and decommission (disks.go:404) role gates 403'd
**every user-data drive in the standard topology** (campaign journal: `where=/mnt/teszt_enroll role=system`).
- **Operator `/32` rendered into `wg-felhom` AllowedIPs** (`internal/wgtunnel`): `oob_peer_ip` appended
sorted + byte-stable; RENDERED so it survives self-heal ([OF-1]).
- **Dedicated `felhom-sshd`** (`internal/felhomsshd`): claim (loud-fail exhaustion), render→`sshd -t`
reload (never restart-on-change [SF-2]), operator authorized_keys outside ~/.ssh [SF-3], heal with
cooldown + never-onto-invalid-config, `configs/felhom-sshd.service` with **no `RuntimeDirectory=`**
[SF-1].
- **Port-adaptive belt** (`internal/felhomsshd/belt.go` + `configs/felhom-oob.nft`): static table, agent
mutates SET ELEMENTS ONLY (`@operator_ips`/`@ssh_port`) [trap 4]; a nil block never empties it.
- **OOB heartbeat stanza** + `FELHOM_SSHD`/`FELHOM_OOB` sudoers. `oob.enabled` DEFAULT FALSE.
## The fix (only `roleForMountPath`)
After the MountPath loop misses on a SUCCESSFUL Observe, resolve the mount's device from the host mount
table (mirroring `durableIDForMount`'s Impl-2b) and classify **device-keyed**:
- non-`/dev` source (NAS) → system;
- device on the same whole disk as a KNOWN target → THAT target's role (containment, new
`storage.SameWholeDisk`, whole-disk granularity so a protected disk can't be ejected here);
- else `RoleForRawDevice` (system-disk membership).
**Fail-safe preserved:** an Observe error returns system BEFORE the fallback (a blind containment pass could
label a backup drive user-data — permissive); a mount-read failure or absent/NAS mount → system. Handlers,
`deviceRole`, `DecommissionExecutor`, `classify.go`, `ReassertGuestBinds` untouched; the
`deviceRole`/`roleForMountPath` unification is deferred.
## Tests + red-proofs (all green — `go build/vet/test ./...`, 23 packages ok)
## Tests + red-proofs (`f2_role_fallback_test.go`)
A1 bind-mounted user-data ejectable + decommission effects; B1 containment (mount on a protected target's
disk → 403); B2 system-disk mount → 403; C1/C2 fail-safe (absent, NAS → 403); C3 Observe-error skips the
fallback (403 + Mounts() called once). **Three red-proofs demonstrated:** pre-fix body → A1 FAIL (exact
`role: system` 403); containment-skip mutation → B1 FAIL (protected disk becomes ejectable); Observe-error
fallback → C3 FAIL (permissive). Existing `TestEject_RoleGated`/`TestDecommission_*` green UNMODIFIED. Full
gate `go build/vet/test ./...` = PASS.
- renderConf: OOB /32 appended sorted + byte-stable; lower-IP-sorts-first; absent = byte-identical
golden; invalid/v6 rejected. Companion: byte-stability is the anti-flap red-proof.
- claim: clean/contention/idempotent/exhaustion (LOUD-fail is the key negative — non-zero, never :22);
persisted-port kept unconditionally (the self-listen flip-flop fix).
- config: SAFE template, byte-stable, refuses :22, never `RuntimeDirectory`.
- belt: mutate-then-idempotent (zero nft ops on the 2nd sync) + **never touches rules** (trap 4
asserted); empty operator empties the set.
- heal: NO restart on an invalid config (red-proof) + 10-min cooldown; status reflects the block.
- hub collector + OOB monitor: stanza ingest + transition-based `oob_degraded`/`oob_recovered` (+ the
unconfigured-not-alerted and no-stanza-ignored negatives). Red-proof: neutering the emit fails it.
## Deploy + live acceptance (on `/mnt/teszt_enroll` ONLY)
Built `-X main.version=0.73.0` on 180, deployed to felhom-pve (`.bak-0.72.0` kept). `--version` = 0.73.0,
`systemctl is-active` = active, capabilities 56/56 degraded=0. Full F2 lifecycle through the real controller
endpoints: **eject → HTTP 200** (was 403), **decommission-anyway → HTTP 200** (`decommissioned:true`, was
403), post-decommission bind detached, **agent restart → drive stayed detached (no rebind)**, re-commission
(reconnect) → 200, **end state == pre-state** (bound on /dev/sdb, attached, durable `uuid:f2236136-ced7…`).
## Live validation (felhom-pve + dev endpoint) — the spikes re-run
Deployed **manually** (agent self-update needs a pinned operator signer; the box has none — D1's
self-update path was not usable, stated honestly). Endpoint set up first (ip_forward=1 +
per-pair forward chain + operator peer via the hub); then agent v0.72.0 + `--enable-oob` artifacts +
`oob.enabled`.
- **Core** — operator→box SSH as **`felhom-op`** over the tunnel, with a scoped `sudo pct list`. ✓
- **A [OF-1]** — stopped `wg-quick@wg-felhom` → agent self-healed (+48s) → box AllowedIPs STILL
`{10.77.0.1/32, 10.77.0.250/32}` (the /32 is rendered) → operator SSH survived. Conf-hash stable
(no per-tick flap). ✓
- **B** — coexistence: stock :22 **PID 922 unchanged** across all felhom-sshd churn; both listening;
distinct host-key fingerprints (stock `8Tea…` vs felhom `EzuJ…`); **felhom-op DENIED on :22**
(`Permission denied (publickey)` — the AuthorizedKeysFile isolation); pveproxy active. ✓
- **C** — port claim: felhom-sshd on 8822, belt `@ssh_port={8822}` follows; clean/contention/
idempotent/exhaustion unit-proven (the flip-flop bug was found + fixed live: once claimed the port is
kept unconditionally, since felhom-sshd itself holds it). ✓
- **D** — belt: LAN→8822 **dropped** (off-tunnel); LAN :22 untouched (safety line); PBS unaffected. ✓
- **E** — heal: stopped felhom-sshd → agent restored it in ~15s. Config-invalid-no-restart unit-proven
(the agent always renders a valid config, so it self-corrects). ✓
- **F** — endpoint: operator peer survives peersync; a dummy tunnel peer is **dropped box↔box**
(§4.5 drop counter=4); PBS healthy. ✓
- **G** — hub: the healthy `oob` stanza `{active:true, port:8822, reachable:true, config_invalid:false,
operator_peer_configured:true, operator_key_configured:true, wg_handshake_age_s:24}` reaches the hub;
`oob_degraded` fired during the real felhom-sshd downtime (E) and `oob_recovered` on recovery. ✓
**Bugs found + fixed during live validation** (each committed): (1) port persisted under root-owned
`/etc/felhom-sshd` → moved to the agent StateDir; (2) self-listen flip-flop (isFree(persisted) sees our
own daemon) → keep the persisted port unconditionally; (3) nil-block emptied the belt/authkeys
(operator lockout on restart) → fetched-gate like wgtunnel; (4) `reachable` via a dial always failed
(belt blocks localhost) → listener check; (5) `operator_*_configured` from persistent state (belt +
authorized_keys), not only the in-memory block.
## Cross-repo state
- Agent **v0.72.0** live on felhom-pve (sha `f72c2fed`); felhom-sshd active on 8822, belt filled,
operator `/32` rendered, oob stanza healthy. Rollback `.bak-0.71.0`. Endpoint + operator peer +
felhom-sshd all STAY (shipped). Stock :22 PID 922 unchanged throughout.
- Hub **v0.35.0** live. felhom.eu commits pushed; doc 06 §4.5/§4.6 amended.
## Observations not acted on (honest ledger)
- **CGNAT** OOB traversal still unproven (the box is single-NAT public-v4, as in both spikes).
- **IPv6/AAAA** out of scope (arc v4-pinned); the standing ep0-AAAA item persists.
- **Operator-key auto-rotation** not implemented (re-`PUT /admin/wg/operator-peer` rotates manually).
- **"Customer network fully down (powered)"** — explicitly OUT OF SCOPE / accepted risk (operator
decision 2026-07-05); the smart-plug is the only mitigation and is not part of this task.
- **`operator_peer_configured` post-restart lag** eliminated by reading the persistent belt (not the
0600 root-owned wg conf the non-root agent can't read).
- Agent binary **not yet published** to Gitea (manual deploy for validation); a fleet rollout should
`scripts/publish-agent.sh 0.72.0 <bin>` + Day-0 vouch.
## Not changed / observations
Only `roleForMountPath` + the new `storage.SameWholeDisk` helper. The pre-existing `VM.Audit` permission
ERROR in the agent log is unrelated (known A1 privilege gap). The `deviceRole`/`roleForMountPath`
unification refactor is deferred.