From dfd5d731eeacd84e3c46488bee8f791b26cfcc8a Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Sat, 25 Jul 2026 14:16:23 +0200 Subject: [PATCH] =?UTF-8?q?v0.96.0=20=E2=80=94=20R-50=20island=20NIC:=20pr?= =?UTF-8?q?ovision=20attaches=20the=20guest=20island=20net1?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - LocalAPIConfig.island_bridge + island_guest_addr (+ IslandEnabled, Validate all-or-nothing + CIDR guard) - buildBringUpConfig attaches static net1 (island) on provision + DR when set; absent otherwise (pre-R-50 byte-for-byte). Plumbed from cfg.LocalAPI at both RunBringUp sites. Endpoint already follows listen_addr (A0: no template change). - healer stays eth0-only (A3 verify-only) — red-proof test locks the scoping - example config + firewall example rewritten for the island; REUSE updated - 3 non-hollow tests; full green. MinAgent unchanged. Coupling: host-install island config requires agent >= 0.96.0 (vouch first). --- CHANGELOG.md | 26 +++++++++ REPORT.md | 74 +++++++++--------------- REUSE.md | 3 +- cmd/felhom-agent/main.go | 2 + configs/agent.example.json | 10 +++- configs/felhom-localapi-firewall.example | 43 ++++++++------ internal/config/config.go | 27 +++++++++ internal/config/config_test.go | 43 ++++++++++++++ internal/guestnet/watchdog_test.go | 18 ++++++ internal/reconcile/bringup.go | 15 +++++ internal/reconcile/bringup_test.go | 30 ++++++++++ 11 files changed, 223 insertions(+), 68 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fcab02f..e8272cb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,29 @@ +## v0.96.0 — R-50 island NIC: provision attaches the guest's island net1 (2026-07-25) + +Additive; **MinAgent unchanged** (no controller coupling — the controller dials whatever `bootstrap.json` +says, and the pin is address-independent). Implements the provisioning half of the R-50 island control +plane, spiked GO in `felhom.eu/documentation/audits/SPIKE-island-bridge-2026-07-25.md`. A fresh install +is now born immune to F1 (`AUDIT-vacation-remote-ops-2026-07-20`: a LAN/DHCP move made the agent fail to +bind → storage/PBS/quiesce/restore-test/DR down, silently). + +- **`LocalAPIConfig`** gains `island_bridge` + `island_guest_addr` (config.go). `IslandEnabled()` = both + set. `Validate()` enforces all-or-nothing + a CIDR guest addr — a half-set/malformed island fails at + load (a botched install), never a silent LAN fallback that would leave an island bind with no island NIC. +- **`buildBringUpConfig`** (reconcile/bringup.go): when both island fields are set, attaches a static + `net1=name=eth1,bridge=,ip=<169.254.253.2/30>` (no hwaddr → fresh per-guest MAC) on BOTH + provision and DR bring-up. Empty = pre-R-50, no net1 (byte-for-byte the old config on non-island hosts). + Plumbed from `cfg.LocalAPI` at both `RunBringUp` call sites (cmd/main.go). The bootstrap `endpoint` + already derives from `listen_addr` (main.go), so moving the agent bind to the island moves the guest + dial for free — **no template change** (A0 determination). +- **Design note (A0/A3):** endpoint is config-derived, so no code was needed there; the guestnet healer + is eth0-only (`parseMode` is dev-scoped) so the static island `eth1` is outside its scope — a red-proof + test locks that in (`TestParseMode_IslandStaticNICDoesNotConfuseEth0`) rather than changing the healer. +- Tests (all non-hollow, red-proofed): `TestBuildBringUpConfig_IslandNIC`, + `TestLocalAPIConfig_IslandValidation`, plus the healer scoping test above. +- **Coupling (deploy order):** a host-install that writes the island config REQUIRES agent ≥ 0.96.0 to + read `island_guest_addr` and attach net1 — vouch 0.96.0 before island installs go live. Migration of + existing boxes is the Phase-B runbook (`felhom.eu/documentation/runbooks/RUNBOOK-island-migration.md`). + ## v0.95.0 — SMART coverage: union-path drives + LVM/dm root + device model (2026-07-25) Additive; **MinAgent unchanged**; hub untouched (unknown JSON fields ignored). Implements the graded diff --git a/REPORT.md b/REPORT.md index 60ea42c..29e9224 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,53 +1,31 @@ -# REPORT — v0.95.0: SMART coverage (spike fixes B+A) + device model (2026-07-25) +# REPORT — v0.96.0: R-50 island NIC (provisioning half) (2026-07-25) -**Overwritten** per the standing rule. Part of the SMART-coverage task (agent v0.95.0 + controller -v0.171.0); the R-50 island-bridge spike lives in felhom.eu. +Implements the agent half of the R-50 island control plane (spike GO, +`felhom.eu/documentation/audits/SPIKE-island-bridge-2026-07-25.md`). A fresh install is born immune to +F1 (a LAN/DHCP/site move can no longer take the control plane down). -## Baseline → target -felhom-agent `643899c` (v0.94.0) → committed **`ed97232`** (v0.95.0). Deployed to felhom-pve. +## What changed +- `internal/config/config.go` — `LocalAPIConfig.IslandBridge` + `IslandGuestAddr`; `IslandEnabled()`; + `Validate()` all-or-nothing + CIDR guard. +- `internal/reconcile/bringup.go` — `BringUpSpec` island fields; `buildBringUpConfig` attaches a static + `net1` (island NIC) on provision AND DR when configured; absent otherwise. +- `cmd/felhom-agent/main.go` — both `RunBringUp` call sites plumb `cfg.LocalAPI.Island*`. +- `configs/agent.example.json` — island shape (island bind + island_bridge/guest_addr + lan_resolver.host_ip). +- `configs/felhom-localapi-firewall.example` — rewritten for the island (the portless bind IS the LAN close). +- `REUSE.md` — island-NIC provisioning pattern + the "endpoint follows listen_addr" note. -## What shipped (additive; MinAgent unchanged; hub untouched) -Implements `SPIKE-smart-coverage-2026-07-25.md` (both demo disks answer `smartctl -a -j` PASSED, but -the agent never asked): -- **Fix B — union-path SMART:** `storage.SmartReader.SMARTForBacking` (reuses `smartDeviceFor`) wired - into the `/disks` union path via a localapi `Smart` seam (+ a `resolveStorageDevice` seam for - testability). Registry/USB drives now get a real read. The watchdog `Known` path stays enrich-free - (asserted: zero smartctl calls). -- **Fix A — LVM/dm resolution:** `smartDeviceFor` resolves `/dev/dm-N` / `/dev/mapper/X` to the single - backing whole disk via `/sys/block//slaves` (recursive; **skips** on >1 physical disk). The - builtin `local` dir on the LVM root gets a **SMART-only** device from its containing filesystem - (`containingMountDevice`), gated to `catDir && backing=="" && reachable` — never touching - `backing_device`/`durable_id` (the removable-safety guard stays intact). -- **Model:** `SmartSummary.ModelName` captured from smartctl's `model_name`. -- Fix C (`-d sat`) stays rejected — no sudoers/manifest change. +## A0/A3 determination +- **A0 = config-only for the endpoint:** `main.go` sets the bootstrap `Endpoint: cfg.LocalAPI.ListenAddr`; + moving the bind to the island moves the guest dial with no template change. The version bump is for the + net1-attach (the operator chose agent-bringup over golden-bake to eliminate the golden-coupling risk). +- **A3 = healer verify-only:** the guestnet healer is eth0-only (`parseMode` dev-scoped); the static island + eth1 is outside its scope. Locked with a red-proof test — no healer change. -## Files -`internal/hub/report.go` (ModelName), `internal/storage/smart.go` (parse model), `internal/storage/smartdev.go` -(NEW: dm resolution + SmartReader + containingMountDevice), `internal/storage/observe.go` (smartDeviceFor dm -branch, enrich smartHint, build hint), `internal/localapi/{server.go,disks.go}` (Smart seam + union read + -resolve seam), `cmd/felhom-agent/main.go` (wire), CHANGELOG/CONTEXT/REUSE. New tests: -`smartdev_test.go`, `observe_smart_test.go`, union-path test in `disks_smart_test.go`. +## Tests (green, non-hollow) +`TestBuildBringUpConfig_IslandNIC`, `TestLocalAPIConfig_IslandValidation`, +`TestParseMode_IslandStaticNICDoesNotConfuseEth0`. Full `go build/vet/test ./...` green. -## Tests + red-proofs (all recorded) -`go build/vet/test ./...` — **29 packages pass, 0 fail.** Red-proofs verified then restored: -- dm multi-disk guard (`len(disks)!=1`) → mirror-over-two-disks resolves instead of skipping → FAIL. -- enrich `smartHint` fallback → system disk stays UNKNOWN → FAIL. -- Fix-B union routing removed → union drive carries no SMART → FAIL. -- Known-path-never-SMARTs asserted (zero calls; the "route Known through enrich" mutation would break it). - -## Deploy + LIVE acceptance (felhom-pve — REAL N100 hardware, not virtualized) -Built `0.95.0` (ldflags), backed up `.bak-0.94.0`, installed, restarted → **active, 68/68 capabilities, -listening**. `/disks` payload now carries real SMART + model for BOTH physical disks: -- `local` (system SSD): `backing=''` yet **smart=PASSED, model="AirDisk 512GB SSD"** — Fix A resolved - `/`→`pve-root`(LVM)→`sda`. -- `47a3361a` (USB): `backing=/dev/sdb`, **smart=PASSED, model="TOSHIBA MQ04ABF100"** — Fix B. -- `felhom-pbs`/`local-lvm`: UNKNOWN (logical; correctly excluded by the controller). - -The controller v0.171.0 card on 9201 shows both with **real verdicts + human labels**: -**"AirDisk 512GB SSD" → Rendben (34°C)** and **"TOSHIBA MQ04ABF100" → Rendben (30°C)** — the operator's -"Nincs adat on a raw UUID" complaint is reversed. (A live FAILING/Figyelmeztetés verdict still needs a -genuinely degrading disk — unit+red-proof covered.) - -## Observations -- The earlier assumption that demo-felhom is "virtualized (no real SMART)" was WRONG — it is a real N100 - (SATA SSD `sda` + USB HDD `sdb`); the capability-map row is corrected accordingly. +## Coupling / deploy order +A host-install that writes the island config REQUIRES agent ≥ 0.96.0 (to read `island_guest_addr` and +attach net1). **Vouch 0.96.0 before island installs go live.** MinAgent unchanged (no controller coupling). +Fleet migration of existing boxes = `felhom.eu/documentation/runbooks/RUNBOOK-island-migration.md` (Phase B). diff --git a/REUSE.md b/REUSE.md index 0cf0618..a06e01c 100644 --- a/REUSE.md +++ b/REUSE.md @@ -88,7 +88,8 @@ | `pinnedTLS` | internal/pbs/pin.go | `pinnedTLS(fingerprint) (*tls.Config, error)` | PBS leaf pinning | Same model as PVE; 64-hex fingerprint normalized | | `hub.Client.Report` | internal/hub/client.go | `Report(ctx, *HostReport) (*ControlEnvelope, error)` | the heartbeat | Typed `TransportError`/`HTTPError`, never contain the bearer token | | `hub.Loop` + `MultiObserver` | internal/hub/loop.go | `NewLoop(...)`; `MultiObserver(obs...)` | resilient report loop + envelope fan-out | Errors logged, loop continues; interval clamped 60–3600 s | -| `provision.BackHalf.Provision` | internal/provision/backhalf.go | `Provision(ctx, Input) (Result, error)` | guest bootstrap back-half | mint→render→0600 write→chown 100000:100000→`pct set` ro bind→onboot; token NEVER logged/returned | +| `provision.BackHalf.Provision` | internal/provision/backhalf.go | `Provision(ctx, Input) (Result, error)` | guest bootstrap back-half | mint→render→0600 write→chown 100000:100000→`pct set` ro bind→onboot; token NEVER logged/returned. Bootstrap `local_api.endpoint` = the caller's `cfg.LocalAPI.ListenAddr` (main.go) — moving the agent bind to the island moves the guest dial for free (R-50, no template) | +| `buildBringUpConfig` island NIC | internal/reconcile/bringup.go | (pure) `BringUpSpec{IslandBridge,IslandGuestAddr}` → `params["net1"]` | R-50 island control plane | When BOTH island fields are set (from `cfg.LocalAPI`), attaches a static `net1=name=eth1,bridge=,ip=<.2/30>` (no hwaddr → fresh MAC), so the controller reaches the agent over a fixed private address immune to LAN/DHCP/site moves. Empty = pre-R-50, no net1. All-or-nothing + CIDR enforced in `LocalAPIConfig.Validate`. The guestnet healer is eth0-only (`parseMode` is dev-scoped) so it never touches the static island NIC | | `reconcile.Queue.Submit` | internal/reconcile/queue.go | `Submit(vmid, fn) <-chan error` | per-guest serialization of ALL mutations | Same vmid strictly FIFO; lanes parallel across guests | | `Engine.RunSignedJob` | internal/reconcile/job.go | `RunSignedJob(ctx, intent, signed, exec) JobResult` | executing a gated destructive job | Idempotency by nonce; journaled | | `escrow.Create` | internal/escrow/escrow.go | `Create(ctx, CreateOptions) (CreateResult, R, error)` | PBS-key escrow (zero-knowledge) | Recovery code returned SEPARATELY from the result (anti-log); self-verifies recoverability | diff --git a/cmd/felhom-agent/main.go b/cmd/felhom-agent/main.go index e083f38..8a6a08f 100644 --- a/cmd/felhom-agent/main.go +++ b/cmd/felhom-agent/main.go @@ -1772,6 +1772,7 @@ func runSelftestBringUp(ctx context.Context, cfg config.Config, logger *slog.Log Cores: sizing.Cores, MemoryMB: sizing.MemoryMB, RootfsGrowGB: sizing.RootfsGrowGB, DataVolGrowGB: sizing.DataVolGrowGB, DataVolMount: sizing.DataVolMount, SysDataGrowGB: sizing.SysDataGrowGB, SysDataMount: sizing.SysDataMount, + IslandBridge: cfg.LocalAPI.IslandBridge, IslandGuestAddr: cfg.LocalAPI.IslandGuestAddr, // R-50 island NIC (both empty = pre-R-50) } fmt.Printf(" bringing up %s → vmid %d on %s …\n", archive, vmid, cfg.Backup.RestoreStorage) res := engine.RunBringUp(ctx, spec) @@ -1935,6 +1936,7 @@ func runSelftestProvision(ctx context.Context, cfg config.Config, logger *slog.L Cores: a.sizing.Cores, MemoryMB: a.sizing.MemoryMB, RootfsGrowGB: a.sizing.RootfsGrowGB, DataVolGrowGB: a.sizing.DataVolGrowGB, DataVolMount: a.sizing.DataVolMount, SysDataGrowGB: a.sizing.SysDataGrowGB, SysDataMount: a.sizing.SysDataMount, + IslandBridge: cfg.LocalAPI.IslandBridge, IslandGuestAddr: cfg.LocalAPI.IslandGuestAddr, // R-50 island NIC (both empty = pre-R-50) }) if res.Err != nil || !res.Pass { fmt.Fprintf(os.Stderr, " [FAIL] front-half bring-up (vmid %d): %v\n", a.vmid, res.Err) diff --git a/configs/agent.example.json b/configs/agent.example.json index 0d3a5b3..625d0a1 100644 --- a/configs/agent.example.json +++ b/configs/agent.example.json @@ -48,10 +48,16 @@ }, "local_api": { "enable": true, - "listen_addr": "192.168.0.162:8443", + "listen_addr": "169.254.253.1:8443", "cert_file": "/var/lib/felhom-agent/local-api.crt", "key_file": "/var/lib/felhom-agent/local-api.key", - "token_store": "/var/lib/felhom-agent/local-tokens.log" + "token_store": "/var/lib/felhom-agent/local-tokens.log", + "island_bridge": "vmbr9", + "island_guest_addr": "169.254.253.2/30" + }, + "lan_resolver": { + "enable": true, + "host_ip": "192.168.0.162" }, "log_level": "info" } diff --git a/configs/felhom-localapi-firewall.example b/configs/felhom-localapi-firewall.example index 9e790d8..febf082 100644 --- a/configs/felhom-localapi-firewall.example +++ b/configs/felhom-localapi-firewall.example @@ -1,20 +1,30 @@ -# felhom-agent local API — host firewall narrowing (doc 03 §6, slice 8A) +# felhom-agent local API — host firewall narrowing (doc 03 §6; R-50 island update 2026-07-25) # -# Defense-in-depth for the per-guest local API (the controller→agent channel on the host -# bridge). The PER-GUEST BEARER TOKEN is the authorization gate; this firewall rule is an -# ADDITIONAL layer that limits who can even reach the port. The slice-8A spike found no rule -# was needed for reachability on the demo (PVE firewall off) — this narrows exposure so that -# only guests on the bridge subnet (not arbitrary LAN hosts) can open a connection. +# Defense-in-depth for the per-guest local API (the controller→agent channel). The PER-GUEST BEARER +# TOKEN + the served-leaf pin are the authorization gate; a firewall rule is only an ADDITIONAL layer +# limiting who can even open the port. # -# The agent already binds the listener to the host BRIDGE IP (local_api.listen_addr), not -# 0.0.0.0. This file adds the subnet restriction. Apply it at HOST SETUP (it is a host-level -# packet-filter change, intentionally OUTSIDE the agent's 3-exception privileged fence — the -# agent never mutates the host firewall at runtime). +# === R-50 ISLAND INSTALL (the default on a fresh appliance) ================================= +# The agent binds local_api.listen_addr on the HOST-INTERNAL island bridge — 169.254.253.1:8443 on +# vmbr9, a bridge with NO physical port (bridge-ports none). That bind is the security win: +# * Nothing listens on the LAN IP at all, so no LAN host (or off-site attacker on the LAN) can +# reach the local API — the LAN:8443 surface is CLOSED by the bind, not by a rule. +# * vmbr9 has no uplink, so 169.254.253.1:8443 is reachable ONLY from the one guest wired to the +# /30 (169.254.253.2) — the controller. The portless bridge is the isolation. +# So on an island install NO firewall rule is required for exposure; the topology provides it. If you +# want belt-and-suspenders, restrict the port to the island bridge (it changes nothing, since nothing +# off-bridge can route to a portless bridge anyway): # -# Replace the bridge IP (192.168.0.162), port (8443), and the guest bridge subnet -# (192.168.0.0/24) with this host's values. +# nft add rule inet filter input iifname != "vmbr9" ip daddr 169.254.253.1 tcp dport 8443 drop +# +# Verify: from the guest, a TLS connect to 169.254.253.1:8443 succeeds; there is no LAN listener to +# probe (`ss -lnt 'sport = :8443'` shows only the island IP). +# +# === LEGACY LAN BIND (byo, --no-island, or an explicit --bridge-ip) ========================= +# When the agent still binds a LAN bridge IP (e.g. 192.168.0.162:8443), the port is exposed to the +# whole LAN and the subnet-narrowing rule below is worth applying. Replace the bridge IP, port, and +# the guest bridge subnet with this host's values. # -# --------------------------------------------------------------------------------------------- # Option A — nftables (recommended on PVE 8/9; inet filter table). Insert ABOVE any accept: # # nft add rule inet filter input ip daddr 192.168.0.162 tcp dport 8443 \ @@ -22,13 +32,11 @@ # nft add rule inet filter input ip daddr 192.168.0.162 tcp dport 8443 \ # ip saddr 192.168.0.0/24 accept # -# --------------------------------------------------------------------------------------------- # Option B — iptables: # # iptables -A INPUT -d 192.168.0.162 -p tcp --dport 8443 -s 192.168.0.0/24 -j ACCEPT # iptables -A INPUT -d 192.168.0.162 -p tcp --dport 8443 -j DROP # -# --------------------------------------------------------------------------------------------- # Option C — PVE host firewall (/etc/pve/nodes//host.fw), if the PVE firewall is enabled. # Add under [RULES] (and ensure the firewall is enabled in cluster.fw / host.fw): # @@ -36,5 +44,6 @@ # IN ACCEPT -source 192.168.0.0/24 -dport 8443 -proto tcp -log nolog # IN DROP -dport 8443 -proto tcp -log nolog # -# Verify after applying: from a guest ON the bridge, a TLS connect to :8443 succeeds; -# from an OFF-bridge host it is refused/dropped. (The token + leaf-pin still gate the request.) +# Apply at HOST SETUP — a host-level packet-filter change, intentionally OUTSIDE the agent's +# 3-exception privileged fence (the agent never mutates the host firewall at runtime). The token + +# leaf-pin still gate the request regardless of which bind is in force. diff --git a/internal/config/config.go b/internal/config/config.go index 45095cb..d08e79a 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -235,6 +235,16 @@ type LocalAPIConfig struct { // TokenStore is the durable, hashed token→guest map (only a HASH of each token is // persisted; the plaintext exists transiently at mint→write-to-mount, then is discarded). TokenStore string `json:"token_store"` // default /var/lib/felhom-agent/local-tokens.log + // IslandBridge + IslandGuestAddr configure the R-50 host-internal control-plane bridge. When + // BOTH are set, the provisioner attaches each guest a static net1 on IslandBridge with + // IslandGuestAddr, so the controller reaches the agent over a fixed private address that no + // LAN/DHCP/site move can invalidate (the F1 fix — AUDIT-vacation-remote-ops-2026-07-20). Empty + // (the default) = LAN-only, byte-for-byte the pre-R-50 behaviour. On an island install ListenAddr + // is the host side (169.254.253.1:8443); IslandGuestAddr is the guest side (169.254.253.2/30 — a + // /30 is exactly host + one guest). Additive-only: it never removes a NIC, so a guest restored on + // a non-island host (both empty) is unaffected. + IslandBridge string `json:"island_bridge"` // e.g. "vmbr9" (portless host-internal bridge) + IslandGuestAddr string `json:"island_guest_addr"` // guest net1 CIDR, e.g. "169.254.253.2/30" } // Default local-API file locations (under the agent's state dir). @@ -249,6 +259,12 @@ func (l LocalAPIConfig) Enabled() bool { return l.Enable && strings.TrimSpace(l.ListenAddr) != "" } +// IslandEnabled reports whether the provisioner should attach a guest island NIC (net1). True only +// when BOTH the bridge and the guest CIDR are set (R-50); empty = pre-R-50 LAN-only behaviour. +func (l LocalAPIConfig) IslandEnabled() bool { + return strings.TrimSpace(l.IslandBridge) != "" && strings.TrimSpace(l.IslandGuestAddr) != "" +} + // TokenStorePath returns the configured token-store path (default applied). func (l LocalAPIConfig) TokenStorePath() string { if l.TokenStore != "" { @@ -283,6 +299,17 @@ func (l LocalAPIConfig) Validate() error { if _, _, err := net.SplitHostPort(l.ListenAddr); err != nil { return fmt.Errorf("config: local_api.listen_addr %q is not host:port: %w", l.ListenAddr, err) } + // R-50: island fields are all-or-nothing, and the guest addr must be a CIDR (the net1 ip= value). + // A half-set island (bridge without guest addr, or vice versa) is a provisioning mistake, not a + // silent LAN fallback — fail loudly so a botched install config is caught at load, not at day-0. + if (strings.TrimSpace(l.IslandBridge) != "") != (strings.TrimSpace(l.IslandGuestAddr) != "") { + return fmt.Errorf("config: local_api.island_bridge and local_api.island_guest_addr must be set together (got bridge=%q guest_addr=%q)", l.IslandBridge, l.IslandGuestAddr) + } + if l.IslandEnabled() { + if _, _, err := net.ParseCIDR(strings.TrimSpace(l.IslandGuestAddr)); err != nil { + return fmt.Errorf("config: local_api.island_guest_addr %q is not a CIDR (want e.g. 169.254.253.2/30): %w", l.IslandGuestAddr, err) + } + } return nil } diff --git a/internal/config/config_test.go b/internal/config/config_test.go index 4e5792b..3a78012 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -171,3 +171,46 @@ func TestDeploymentModeEnvOverlay(t *testing.T) { t.Errorf("env overlay did not set deployment_mode: %q", cfg.DeploymentMode) } } + +// R-50: the island NIC fields are all-or-nothing and the guest addr must be a CIDR. A half-set or +// malformed island must fail at config load (a botched install) rather than silently fall back to +// LAN-only, which would leave a guest with an island bind and no island NIC — the exact silent break +// R-50 exists to kill. Covers LocalAPIConfig.Validate + IslandEnabled. +func TestLocalAPIConfig_IslandValidation(t *testing.T) { + base := LocalAPIConfig{Enable: true, ListenAddr: "169.254.253.1:8443"} + + // both empty → fine (pre-R-50 default), IslandEnabled false + if err := base.Validate(); err != nil { + t.Errorf("no island config must validate: %v", err) + } + if base.IslandEnabled() { + t.Errorf("IslandEnabled must be false when unset") + } + // both set, valid CIDR → fine, IslandEnabled true + ok := base + ok.IslandBridge, ok.IslandGuestAddr = "vmbr9", "169.254.253.2/30" + if err := ok.Validate(); err != nil { + t.Errorf("valid island config must validate: %v", err) + } + if !ok.IslandEnabled() { + t.Errorf("IslandEnabled must be true when both set") + } + // bridge only → rejected (all-or-nothing) + half := base + half.IslandBridge = "vmbr9" + if err := half.Validate(); err == nil { + t.Errorf("half-set island (bridge only) must be rejected") + } + // guest addr only → rejected + half2 := base + half2.IslandGuestAddr = "169.254.253.2/30" + if err := half2.Validate(); err == nil { + t.Errorf("half-set island (guest addr only) must be rejected") + } + // both set but guest addr is not a CIDR → rejected + bad := base + bad.IslandBridge, bad.IslandGuestAddr = "vmbr9", "169.254.253.2" // missing /30 + if err := bad.Validate(); err == nil { + t.Errorf("island guest addr without a CIDR mask must be rejected") + } +} diff --git a/internal/guestnet/watchdog_test.go b/internal/guestnet/watchdog_test.go index 22c1146..fd04b93 100644 --- a/internal/guestnet/watchdog_test.go +++ b/internal/guestnet/watchdog_test.go @@ -548,3 +548,21 @@ func TestClassify_Table(t *testing.T) { } var _ io.Writer = (*bytes.Buffer)(nil) + +// A3 (R-50): the guestnet healer is eth0-only and MUST stay blind to the island NIC. A guest on an +// island host presents eth0 DHCP (the LAN leg the healer owns) PLUS eth1 static (the island). Because +// parseMode is interface-scoped, adding eth1 static cannot flip eth0's detected mode — so the healer +// keeps treating eth0 as DHCP and never runs dhclient against the static island NIC (which would +// sabotage it). This is the verify-only guarantee that let R-50 ship the island NIC without a healer +// change. Red-proof: make parseMode scan globally instead of per-dev and the eth0 assertion fails. +func TestParseMode_IslandStaticNICDoesNotConfuseEth0(t *testing.T) { + interfaces := "auto lo\niface lo inet loopback\n\n" + + "auto eth0\niface eth0 inet dhcp\n\n" + + "auto eth1\niface eth1 inet static\n address 169.254.253.2/30\n" + if got := parseMode(interfaces, "eth0"); got != ModeDHCP { + t.Errorf("eth0 must classify DHCP even with an island eth1 static present, got %q", got) + } + if got := parseMode(interfaces, "eth1"); got != ModeStatic { + t.Errorf("eth1 (island) must classify static when asked directly (dev-scoped), got %q", got) + } +} diff --git a/internal/reconcile/bringup.go b/internal/reconcile/bringup.go index b69325d..0b253fe 100644 --- a/internal/reconcile/bringup.go +++ b/internal/reconcile/bringup.go @@ -193,6 +193,12 @@ type BringUpSpec struct { Mounts []GuestMount // additive mpN mounts (slice 7 may pass empty/test) KeepMAC bool // DR knob: keep the archived MAC (true) unless a source may be live BootTimeout time.Duration // 0 → DefaultBootTimeout; bounds the link-up liveness wait + // IslandBridge + IslandGuestAddr (R-50): when BOTH are set, the guest gets a static net1 on the + // host-internal island bridge, so the controller reaches the agent over a fixed private address + // that survives any LAN/DHCP/site move (the F1 fix). Empty (default) = no net1, byte-for-byte the + // pre-R-50 config. Set from cfg.LocalAPI (island_bridge/island_guest_addr) at both call sites. + IslandBridge string // e.g. "vmbr9" + IslandGuestAddr string // guest net1 CIDR, e.g. "169.254.253.2/30" } // BringUpResult is the outcome. It reuses the restore-test's WARNINGS surface @@ -606,6 +612,15 @@ func buildBringUpConfig(spec BringUpSpec, cfg proxmox.GuestConfig) map[string]st params["net0"] = withoutHwaddr(net0) // omit hwaddr → PVE generates a fresh MAC (F1) } } + // R-50 island NIC: attach a static net1 on the host-internal bridge so the control plane + // (controller→agent local API) rides a fixed private address, immune to any LAN/DHCP/site move. + // Both modes: a provisioned guest AND a DR-restored guest need to reach the island-bound agent on + // the target host. No hwaddr → PVE mints a fresh per-guest MAC (the /30 is one guest per host, so + // a MAC would not collide either way, but a fresh one keeps net1 symmetric with net0). Additive: + // omitted entirely when the island is not configured, keeping non-island hosts unchanged. + if strings.TrimSpace(spec.IslandBridge) != "" && strings.TrimSpace(spec.IslandGuestAddr) != "" { + params["net1"] = fmt.Sprintf("name=eth1,bridge=%s,ip=%s", spec.IslandBridge, spec.IslandGuestAddr) + } if spec.Mode == ModeProvision && spec.Hostname != "" { params["hostname"] = spec.Hostname } diff --git a/internal/reconcile/bringup_test.go b/internal/reconcile/bringup_test.go index 70b4765..537d26b 100644 --- a/internal/reconcile/bringup_test.go +++ b/internal/reconcile/bringup_test.go @@ -165,6 +165,36 @@ func TestBuildBringUpConfig_ResourceCaps(t *testing.T) { } } +// R-50: with the island configured, bring-up attaches a static net1 on the island bridge; with it +// unset (or half-set), NO net1 is emitted — byte-for-byte the pre-R-50 config on non-island hosts. +// Pure-function check on buildBringUpConfig (the derivation that makes fresh installs F1-immune). +func TestBuildBringUpConfig_IslandNIC(t *testing.T) { + // island set → net1 present, exact shape, no hwaddr (PVE mints a fresh per-guest MAC) + island := buildBringUpConfig(BringUpSpec{ + Mode: ModeProvision, IslandBridge: "vmbr9", IslandGuestAddr: "169.254.253.2/30", + }, scratchCfg()) + if got, want := island["net1"], "name=eth1,bridge=vmbr9,ip=169.254.253.2/30"; got != want { + t.Errorf("island net1 mismatch:\n got %q\nwant %q", got, want) + } + // DR mode too — a restored customer guest must also reach the island-bound agent on the host. + dr := buildBringUpConfig(BringUpSpec{ + Mode: ModeDRGuestLoss, KeepMAC: true, IslandBridge: "vmbr9", IslandGuestAddr: "169.254.253.2/30", + }, scratchCfg()) + if _, ok := dr["net1"]; !ok { + t.Errorf("DR bring-up must also attach the island net1, got none") + } + // island unset → NO net1 key (non-island hosts unchanged; the pre-R-50 default) + none := buildBringUpConfig(BringUpSpec{Mode: ModeProvision}, scratchCfg()) + if v, ok := none["net1"]; ok { + t.Errorf("net1 must be ABSENT when the island is not configured, got %q", v) + } + // half-configured (bridge only) → still no net1 (all-or-nothing; config.Validate rejects the config too) + half := buildBringUpConfig(BringUpSpec{Mode: ModeProvision, IslandBridge: "vmbr9"}, scratchCfg()) + if v, ok := half["net1"]; ok { + t.Errorf("net1 must be ABSENT when only the bridge is set, got %q", v) + } +} + // Both restore sites allocate the guest INTO the felhom pool (SPIKE 3b): the provision bring-up // threads spec.Pool, and the restore-test hardcodes DefaultPool — else a pool-scoped token 403s on // the created guest's config/start/destroy. Asserts via the fakeAPI's captured RestoreLXCOptions.