Files
felhom-controller/REPORT.md
T
admin cb8bf14599 v0.162.0 — R-71(a): the apply-bridge settle-gate (kills the F10 day-0 race)
The day-0 race (DIAG-f10): a fresh box boots below the operator floor, the
apply-bridge consumes the single-use offsite password, then ~35s later the
managed auto-floor update replaces the container mid-install -> the new process
finds no installed key -> consume -> 404 -> offsite dead until an operator
Re-issue. Recurs on every onboarding whose ISO floor lags the managed floor.

Ordering-only fix (consume/install/persist internals + the 404-no-oracle
contract + the Consumer UNTOUCHED; R-71(b) rejected-by-design):
- New seam offsiteapply.SettleProvider.SettleState() + SettleFunc adapter over
  the self-updater's own GetFloor()/IsUpdateRunning() (no second floor path).
- Bridge.AwaitSettle polls 10s BEFORE the 3-min Reconcile ctx: defers while an
  update runs or the box is below the known floor; GOes at/above floor on the
  first poll with zero added latency (B'). Bounds 90s floor sub-bound / 5min
  overall, both GO+WARN (hub that can't serve a floor can't serve a consume ->
  no burn risk; R-71c is the belt). ReconcileWhenSettled = gate then reconcile.
- main.go: bridge goroutine moved after the updater is built; wired only when an
  updater exists (nil Settle = reconcile immediately, old behavior).

Finding: the floor is in-memory (report-ACK ~5-10s), NOT persisted -> unknown on
any restart until the first ACK; the 90s sub-bound is sized to that.

Tests (injectable clock, fake SettleState, recorded Consumer): A-E + nil-provider
+ cancelled-gate. Four red-proofs all observed FAIL then restored: gate removed /
updateRunning branch / floor sub-bound / overall bound. Deferral paths ship
unit-proven + red-proofed, NOT live-fired -- their precondition is now
structurally prevented by the v1.25.0 build gate. Layering: gate prevents, (a)
defers, (c) heals.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N7Drmtm2RzoqbkJZCNSFNQ
2026-07-24 07:48:45 +02:00

4.9 KiB
Raw Blame History

REPORT — felhom-controller v0.162.0 (R-71a: the apply-bridge settle-gate)

Date: 2026-07-24 · Task: R-71(a) — structural fix for the F10 day-0 update-vs-bridge race (consume-then-persist not crash-safe). Scope: felhom-controller only, ordering-only. No agent (0.93.0 stays), no hub. MinAgent unchanged.

The defect (DIAG-f10)

A fresh box boots below the operator floor (ISO 0.153.0 < floor 0.156.0). The apply-bridge consumes the hub's single-use offsite password at boot; ~35 s later the managed auto-floor update replaces the container mid-install → the new 0.156.0 process finds no installed key → consume404 → offsite tier dead until an operator Re-issue. Recurs on every onboarding whose ISO floor lags the managed floor; demo-felhom escaped by timing alone. Layering: the v1.25.0 golden≥floor build gate PREVENTS the trigger for fresh installs, R-71c (hub) HEALS a burn, this (a) removes the systematic trigger for every restart shape.

What shipped (ordering only — consume/install/persist internals + the 404-no-oracle contract + the Consumer UNTOUCHED; R-71(b) stays rejected-by-design)

  1. Seam offsiteapply.SettleProvider.SettleState() (version, floor string, updateRunning, floorKnown bool) + SettleFunc adapter (seams.go) — a thin adapter over the self-updater's own knowledge (GetFloor()/IsUpdateRunning()) in main.go. No second floor-fetch path.
  2. Bridge.AwaitSettle — polls every 10 s BEFORE the 3-minute Reconcile context is created (the deferral never eats the reconcile budget). Release table:
    • updateRunning → wait (the swap's restart supersedes us).
    • floorKnown && version < floor → wait (auto-floor update imminent — do NOT burn the password); log deferring offsite apply: managed update to floor <F> pending (we are <V>).
    • floorKnown && at/above floorGO on the first poll, zero sleep (the B invariant).
    • !floorKnown past the 90 s sub-bound → GO + WARN (a hub that can't serve a floor can't serve a consume → no burn risk).
    • overall 5-min bound → GO + WARN (R-71c self-heal is the belt). ReconcileWhenSettled runs the gate, then Reconcile under a fresh 3-min context.
  3. main.go — the bridge goroutine MOVED to after the self-updater is constructed (so the adapter can read it). Wired ONLY when an updater exists; otherwise Settle is nil = reconcile immediately (no updater → no floor-update to race).
  4. Bounds are named constants with rationale comments. The floor is in-memory (report-ACK-derived), NOT persisted — so on any restart it is unknown until the first report ACK. Source-confirmed latency: the startup report fires ~5 s after boot and SetFloor runs synchronously in its ACK handler → floor normally known in ~510 s (≤~45 s across 3×15 s report retries); the 90 s sub-bound is headroom over that worst case.

Tests — all green (go test ./... clean, go vet ./... clean, gofmt clean)

Injectable clock (no real sleeps), fake SettleState, recorded Consumer (internal/offsiteapply/settlegate_test.go):

  • A below-floor defers (0 consumes while below) then GOes at floor → exactly one consume.
  • B update-running defers then GOes when it finishes.
  • C floor-unknown GOes at the 90 s sub-bound + the WARN line.
  • D perpetually-below GOes at the 5-min overall bound + the WARN line.
  • E (B) at/above-floor GOes on poll 1 with zero sleep consumed.
  • nil-provider → immediate reconcile; cancelled gate → skips reconcile (no consume).

Red-proofs — four, each OBSERVED failing then restored

# Break Observed FAIL
A gate removed (early return in AwaitSettle) TestSettle_BelowFloorDefersThenGoes: "expected the gate to defer while below floor (>=2 sleeps), got 0" — below-floor consumes immediately (the F10 burn)
B updateRunning branch removed TestSettle_UpdateRunningDefersThenGoes: mid-swap box GOes immediately (log shows GO, deferral line absent)
C floor sub-bound branch removed TestSettle_FloorUnknownGoesAfterSubBound + red-proof: "release near the sub-bound 1m30s, waited 5m0s" — floor-unknown drags to the overall bound
D overall bound check removed TestSettle_PerpetuallyBelowFloorGoesAtOverallBound: 20 s test timeout — perpetual below-floor loops forever in AwaitSettle

Ship / honesty

The deferral paths ship unit-proven + red-proofed, NOT live-fired — their precondition is now structurally prevented by the v1.25.0 golden≥floor build gate, which is the point. The live-observable leg is the B first-poll GO line on both above-floor demo boxes (proving zero added latency on the common restart shape). Layering: gate prevents, (a) defers, (c) heals. R-71 → SHIPPED with (a)+(c); (b) recorded rejected-by-design.

(Build/deploy to both boxes + the two first-poll GO log lines appended on completion of the publish train.)