felhom-controller/REPORT.md

REPORT — felhom-controller v0.162.0 (R-71a: the apply-bridge settle-gate)

Date: 2026-07-24 · Task: R-71(a) — structural fix for the F10 day-0 update-vs-bridge race (consume-then-persist not crash-safe). Scope: felhom-controller only, ordering-only. No agent (0.93.0 stays), no hub. MinAgent unchanged.

The defect (DIAG-f10)

A fresh box boots below the operator floor (ISO 0.153.0 < floor 0.156.0). The apply-bridge consumes the hub's single-use offsite password at boot; ~35 s later the managed auto-floor update replaces the container mid-install → the new 0.156.0 process finds no installed key → consume → 404 → offsite tier dead until an operator Re-issue. Recurs on every onboarding whose ISO floor lags the managed floor; demo-felhom escaped by timing alone. Layering: the v1.25.0 golden≥floor build gate PREVENTS the trigger for fresh installs, R-71c (hub) HEALS a burn, this (a) removes the systematic trigger for every restart shape.

What shipped (ordering only — consume/install/persist internals + the 404-no-oracle contract + the Consumer UNTOUCHED; R-71(b) stays rejected-by-design)

1. Seam offsiteapply.SettleProvider.SettleState() (version, floor string, updateRunning, floorKnown bool) + SettleFunc adapter (seams.go) — a thin adapter over the self-updater's own knowledge (GetFloor()/IsUpdateRunning()) in main.go. No second floor-fetch path.
2. Bridge.AwaitSettle — polls every 10 s BEFORE the 3-minute Reconcile context is created (the deferral never eats the reconcile budget). Release table:
   • updateRunning → wait (the swap's restart supersedes us).
   • floorKnown && version < floor → wait (auto-floor update imminent — do NOT burn the password); log deferring offsite apply: managed update to floor <F> pending (we are <V>).
   • floorKnown && at/above floor → GO on the first poll, zero sleep (the B′ invariant).
   • !floorKnown past the 90 s sub-bound → GO + WARN (a hub that can't serve a floor can't serve a consume → no burn risk).
   • overall 5-min bound → GO + WARN (R-71c self-heal is the belt). ReconcileWhenSettled runs the gate, then Reconcile under a fresh 3-min context.
3. main.go — the bridge goroutine MOVED to after the self-updater is constructed (so the adapter can read it). Wired ONLY when an updater exists; otherwise Settle is nil = reconcile immediately (no updater → no floor-update to race).
4. Bounds are named constants with rationale comments. The floor is in-memory (report-ACK-derived), NOT persisted — so on any restart it is unknown until the first report ACK. Source-confirmed latency: the startup report fires ~5 s after boot and SetFloor runs synchronously in its ACK handler → floor normally known in ~5–10 s (≤~45 s across 3×15 s report retries); the 90 s sub-bound is headroom over that worst case.

Tests — all green (go test ./... clean, go vet ./... clean, gofmt clean)

Injectable clock (no real sleeps), fake SettleState, recorded Consumer (internal/offsiteapply/settlegate_test.go):

• A below-floor defers (0 consumes while below) then GOes at floor → exactly one consume.
• B update-running defers then GOes when it finishes.
• C floor-unknown GOes at the 90 s sub-bound + the WARN line.
• D perpetually-below GOes at the 5-min overall bound + the WARN line.
• E (B′) at/above-floor GOes on poll 1 with zero sleep consumed.
• nil-provider → immediate reconcile; cancelled gate → skips reconcile (no consume).

Red-proofs — four, each OBSERVED failing then restored

#	Break	Observed FAIL
A	gate removed (early return in AwaitSettle)	TestSettle_BelowFloorDefersThenGoes: "expected the gate to defer while below floor (>=2 sleeps), got 0" — below-floor consumes immediately (the F10 burn)
B	updateRunning branch removed	TestSettle_UpdateRunningDefersThenGoes: mid-swap box GOes immediately (log shows GO, deferral line absent)
C	floor sub-bound branch removed	TestSettle_FloorUnknownGoesAfterSubBound + red-proof: "release near the sub-bound 1m30s, waited 5m0s" — floor-unknown drags to the overall bound
D	overall bound check removed	TestSettle_PerpetuallyBelowFloorGoesAtOverallBound: 20 s test timeout — perpetual below-floor loops forever in AwaitSettle

Ship / honesty

The deferral paths ship unit-proven + red-proofed, NOT live-fired — their precondition is now structurally prevented by the v1.25.0 golden≥floor build gate, which is the point. Layering: gate prevents, (a) defers, (c) heals. R-71 → SHIPPED with (a)+(c); (b) recorded rejected-by-design.

Build + deploy + LIVE (2026-07-24)

Image gitea.dooplex.hu/admin/felhom-controller:0.162.0 (sha e6e1146f…, 145M) built + pushed. Deployed to both boxes, clean start, healthy:

• demo-felhom (guest 9201, ssh felhom-pve) — Up (healthy):

  05:50:44 [INFO] [offsite-apply] settle-gate: awaiting floor knowledge (first report ACK) before offsite apply
  05:50:54 [INFO] [offsite-apply] settle-gate: GO — at/above floor 0.156.0 (we are 0.162.0), no managed update running
  

• demo-hp (guest 9201, G1 break-glass; secret extracted file→file + shredded) — Up (healthy):

  05:52:53 [INFO] [offsite-apply] settle-gate: awaiting floor knowledge (first report ACK) before offsite apply
  05:53:03 [INFO] [offsite-apply] settle-gate: GO — at/above floor 0.156.0 (we are 0.162.0), no managed update running
  

Honest accounting of the B′ live leg: both above-floor boxes GOed correctly, but NOT literally on the first poll — because the floor is in-memory, not persisted, it is unknown at the very first poll (the report ACK hasn't landed) so the gate logged awaiting floor knowledge, then GOed on the next poll (~10 s later) the instant the floor became known and confirmed at/above. This is exactly the documented finding, not a regression: the gate correctly refused to consume while the floor was unknown, and added ~10 s to a background reconcile (invisible to any user). The B′ zero-wait-when- floor-known invariant is unit-proven (test E); the ~10 s live wait is the report-ACK latency the 90 s sub-bound was sized against. The important live proof holds: the gate did not burn the one-time password before the managed-update picture was clear.