The day-0 race (DIAG-f10): a fresh box boots below the operator floor, the apply-bridge consumes the single-use offsite password, then ~35s later the managed auto-floor update replaces the container mid-install -> the new process finds no installed key -> consume -> 404 -> offsite dead until an operator Re-issue. Recurs on every onboarding whose ISO floor lags the managed floor. Ordering-only fix (consume/install/persist internals + the 404-no-oracle contract + the Consumer UNTOUCHED; R-71(b) rejected-by-design): - New seam offsiteapply.SettleProvider.SettleState() + SettleFunc adapter over the self-updater's own GetFloor()/IsUpdateRunning() (no second floor path). - Bridge.AwaitSettle polls 10s BEFORE the 3-min Reconcile ctx: defers while an update runs or the box is below the known floor; GOes at/above floor on the first poll with zero added latency (B'). Bounds 90s floor sub-bound / 5min overall, both GO+WARN (hub that can't serve a floor can't serve a consume -> no burn risk; R-71c is the belt). ReconcileWhenSettled = gate then reconcile. - main.go: bridge goroutine moved after the updater is built; wired only when an updater exists (nil Settle = reconcile immediately, old behavior). Finding: the floor is in-memory (report-ACK ~5-10s), NOT persisted -> unknown on any restart until the first ACK; the 90s sub-bound is sized to that. Tests (injectable clock, fake SettleState, recorded Consumer): A-E + nil-provider + cancelled-gate. Four red-proofs all observed FAIL then restored: gate removed / updateRunning branch / floor sub-bound / overall bound. Deferral paths ship unit-proven + red-proofed, NOT live-fired -- their precondition is now structurally prevented by the v1.25.0 build gate. Layering: gate prevents, (a) defers, (c) heals. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N7Drmtm2RzoqbkJZCNSFNQ
4.9 KiB
REPORT — felhom-controller v0.162.0 (R-71a: the apply-bridge settle-gate)
Date: 2026-07-24 · Task: R-71(a) — structural fix for the F10 day-0 update-vs-bridge race
(consume-then-persist not crash-safe). Scope: felhom-controller only, ordering-only. No agent
(0.93.0 stays), no hub. MinAgent unchanged.
The defect (DIAG-f10)
A fresh box boots below the operator floor (ISO 0.153.0 < floor 0.156.0). The apply-bridge consumes
the hub's single-use offsite password at boot; ~35 s later the managed auto-floor update replaces
the container mid-install → the new 0.156.0 process finds no installed key → consume → 404 →
offsite tier dead until an operator Re-issue. Recurs on every onboarding whose ISO floor lags the
managed floor; demo-felhom escaped by timing alone. Layering: the v1.25.0 golden≥floor build gate
PREVENTS the trigger for fresh installs, R-71c (hub) HEALS a burn, this (a) removes the systematic
trigger for every restart shape.
What shipped (ordering only — consume/install/persist internals + the 404-no-oracle contract + the Consumer UNTOUCHED; R-71(b) stays rejected-by-design)
- Seam
offsiteapply.SettleProvider.SettleState() (version, floor string, updateRunning, floorKnown bool)+SettleFuncadapter (seams.go) — a thin adapter over the self-updater's own knowledge (GetFloor()/IsUpdateRunning()) in main.go. No second floor-fetch path. Bridge.AwaitSettle— polls every 10 s BEFORE the 3-minute Reconcile context is created (the deferral never eats the reconcile budget). Release table:updateRunning→ wait (the swap's restart supersedes us).floorKnown && version < floor→ wait (auto-floor update imminent — do NOT burn the password); logdeferring offsite apply: managed update to floor <F> pending (we are <V>).floorKnown && at/above floor→ GO on the first poll, zero sleep (the B′ invariant).!floorKnownpast the 90 s sub-bound → GO + WARN (a hub that can't serve a floor can't serve a consume → no burn risk).- overall 5-min bound → GO + WARN (
R-71c self-heal is the belt).ReconcileWhenSettledruns the gate, then Reconcile under a fresh 3-min context.
- main.go — the bridge goroutine MOVED to after the self-updater is constructed (so the adapter
can read it). Wired ONLY when an updater exists; otherwise
Settleis nil = reconcile immediately (no updater → no floor-update to race). - Bounds are named constants with rationale comments. The floor is in-memory
(report-ACK-derived), NOT persisted — so on any restart it is unknown until the first report
ACK. Source-confirmed latency: the startup report fires ~5 s after boot and
SetFloorruns synchronously in its ACK handler → floor normally known in ~5–10 s (≤~45 s across 3×15 s report retries); the 90 s sub-bound is headroom over that worst case.
Tests — all green (go test ./... clean, go vet ./... clean, gofmt clean)
Injectable clock (no real sleeps), fake SettleState, recorded Consumer
(internal/offsiteapply/settlegate_test.go):
- A below-floor defers (0 consumes while below) then GOes at floor → exactly one consume.
- B update-running defers then GOes when it finishes.
- C floor-unknown GOes at the 90 s sub-bound + the WARN line.
- D perpetually-below GOes at the 5-min overall bound + the WARN line.
- E (B′) at/above-floor GOes on poll 1 with zero sleep consumed.
- nil-provider → immediate reconcile; cancelled gate → skips reconcile (no consume).
Red-proofs — four, each OBSERVED failing then restored
| # | Break | Observed FAIL |
|---|---|---|
| A | gate removed (early return in AwaitSettle) |
TestSettle_BelowFloorDefersThenGoes: "expected the gate to defer while below floor (>=2 sleeps), got 0" — below-floor consumes immediately (the F10 burn) |
| B | updateRunning branch removed |
TestSettle_UpdateRunningDefersThenGoes: mid-swap box GOes immediately (log shows GO, deferral line absent) |
| C | floor sub-bound branch removed | TestSettle_FloorUnknownGoesAfterSubBound + red-proof: "release near the sub-bound 1m30s, waited 5m0s" — floor-unknown drags to the overall bound |
| D | overall bound check removed | TestSettle_PerpetuallyBelowFloorGoesAtOverallBound: 20 s test timeout — perpetual below-floor loops forever in AwaitSettle |
Ship / honesty
The deferral paths ship unit-proven + red-proofed, NOT live-fired — their precondition is now structurally prevented by the v1.25.0 golden≥floor build gate, which is the point. The live-observable leg is the B′ first-poll GO line on both above-floor demo boxes (proving zero added latency on the common restart shape). Layering: gate prevents, (a) defers, (c) heals. R-71 → SHIPPED with (a)+(c); (b) recorded rejected-by-design.
(Build/deploy to both boxes + the two first-poll GO log lines appended on completion of the publish train.)