hub v0.97.0 — the floor stops being served past the agent it depends on (CAMPAIGN-11)
gates / gates (push) Successful in 8s
gates / gates (push) Successful in 8s
R-216, the hub half. ResolveManagedFloor's own comment says it exists to "never push a controller past the agent it depends on", and it compared against ArtifactManifest.MinAgent — which by ITS own comment describes the GOLDEN's controller. publish-train-rules.md rule 3 states the rule about the FLOOR's controller. Measured live: golden 0.192.0 / MinAgent 0.113.0, floor 0.200.0, agent 0.120.0 — served, and the box was pushed onto a controller needing agent 0.125.0. A floor ABOVE the vouched golden is now HELD with its own reason (HeldBeyondGolden), reusing Part D's dashboard visibility. Nobody types a number twice: the vouched MinAgent keeps its meaning, the guard stops applying it to versions it does not describe. An uncoupled release is untouched; an unparseable golden degrades rather than gating. R-222: the report ACK's escrow object gains superseded_present / superseded_at, counting only rows that actually carry an identity blob. One boolean and one timestamp, for one message. No read path — that link is still unbuilt. Red-proof: removing the floor-above-golden branch reproduces the campaign's measurement.
This commit is contained in:
@@ -1,3 +1,50 @@
|
|||||||
|
## v0.97.0 — the floor stops being served past the agent it depends on (2026-08-05, CAMPAIGN-11)
|
||||||
|
|
||||||
|
**R-216, the hub half.** `ResolveManagedFloor` exists to *"never push a controller past the agent it
|
||||||
|
depends on"* — its own comment says so — and it compared the box's agent against
|
||||||
|
`ArtifactManifest.MinAgent`, which by ITS own comment is *"the MINIMUM host-agent version this
|
||||||
|
GOLDEN's controller requires"*. Those two sentences contradict each other, and
|
||||||
|
`publish-train-rules.md` rule 3 states the rule about the **floor's** controller, not the golden's.
|
||||||
|
|
||||||
|
They are the same number only while the floor sits at or below the golden — the arrangement rule 5's
|
||||||
|
ISO gate assumes. Raise a floor above the vouched golden (which the day-0 runbook explicitly
|
||||||
|
recommends after a golden rebuild, and which a per-customer override makes trivial) and the guard
|
||||||
|
compares against a version it is not serving.
|
||||||
|
|
||||||
|
**Measured live 2026-08-05 (CAMPAIGN-11 Phase 1):** golden 0.192.0 with MinAgent 0.113.0, a
|
||||||
|
per-customer floor of 0.200.0, a box on agent 0.120.0. `0.120.0 ≥ 0.113.0`, so the floor was served
|
||||||
|
and the box was pushed onto a controller needing agent 0.125.0 — whose customer was then told their
|
||||||
|
correct recovery code was wrong. **This is the tenth entry in `CLAUDE.md`'s comment-vs-code table, and
|
||||||
|
the first where the false invariant was a guard rather than a comment alone.**
|
||||||
|
|
||||||
|
**The fix, and why nobody types a number twice:** the hub cannot know the agent requirement of a
|
||||||
|
controller version it was never told about, so it must not pretend to. A floor ABOVE the vouched
|
||||||
|
golden is now **HELD**, with its own reason (`HeldBeyondGolden`), reusing Part D's existing dashboard
|
||||||
|
visibility — a held box is visible, never silently stale. The vouched MinAgent keeps its exact
|
||||||
|
meaning; the guard simply stops applying it to versions it does not describe. The operator's remedy is
|
||||||
|
the one the publish train already prescribes: vouch a golden carrying the floor's controller
|
||||||
|
(rule 1, *manifest before floor*).
|
||||||
|
|
||||||
|
⚠ **What this does NOT do:** with the guard corrected but the day-0 agent unchanged, a new box is
|
||||||
|
**held, not served**. It stops being lied to; the recovery feature still does not work for it until
|
||||||
|
agent 0.125.0 is vouched in the Day-0 manifest. That is an operator decision, not a code change.
|
||||||
|
|
||||||
|
Unchanged: an uncoupled release (no MinAgent vouched) is served exactly as before, above the golden or
|
||||||
|
not — a fleet must not be frozen by a comparison that now over-fires. An unparseable golden degrades to
|
||||||
|
the previous behaviour rather than gating.
|
||||||
|
|
||||||
|
**R-222.** The report ACK's `escrow` object gains `superseded_present` / `superseded_at`: whether the
|
||||||
|
hub is ALSO keeping an EARLIER sealed package for this host, and when it was set aside. It exists for
|
||||||
|
one message — the recovery screen cannot otherwise tell a genuinely wrong code from a code that is
|
||||||
|
RIGHT about a retained earlier package, because the unseal fails closed against the current package
|
||||||
|
either way. **Only rows that actually carry an identity blob count**: the pre-v0.93.0 rows are NULL and
|
||||||
|
retain nothing the screen could be talking about. It is a boolean and a timestamp; it grants **no read
|
||||||
|
path**, which is still unbuilt (R-199's inventory).
|
||||||
|
|
||||||
|
Tests: `managed_floor_test.go` — the campaign's exact numbers, held-regardless-of-agent, Scenario C
|
||||||
|
(uncoupled untouched), floor==golden unchanged, unparseable golden. **Red-proof: removing the
|
||||||
|
floor-above-golden branch reproduces the campaign's measurement exactly.**
|
||||||
|
|
||||||
## v0.96.0 — the hub answers a rebuilt box's request (2026-08-05, R-204 item 4 / R-193 / R-192)
|
## v0.96.0 — the hub answers a rebuilt box's request (2026-08-05, R-204 item 4 / R-193 / R-192)
|
||||||
|
|
||||||
The hub half of the last manual intervention the 2026-08-04 drill needed. The box declares
|
The hub half of the last manual intervention the 2026-08-04 drill needed. The box declares
|
||||||
|
|||||||
@@ -93,3 +93,90 @@ func TestResolveManagedFloor(t *testing.T) {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ── R-216: the floor may not be served ABOVE the version the manifest describes ─────────────────
|
||||||
|
//
|
||||||
|
// `publish-train-rules.md` rule 3 is about the FLOOR's controller; the implementation compared
|
||||||
|
// against the GOLDEN's MinAgent. Those are the same number only while the floor sits at or below the
|
||||||
|
// golden. Measured live 2026-08-05 (CAMPAIGN-11 Phase 1): golden 0.192.0 / MinAgent 0.113.0, a
|
||||||
|
// per-customer floor of 0.200.0, a box on agent 0.120.0 — 0.120.0 ≥ 0.113.0, so the floor was served
|
||||||
|
// and the box was pushed onto a controller needing agent 0.125.0. Its customer was then told their
|
||||||
|
// correct recovery code was wrong.
|
||||||
|
//
|
||||||
|
// RED-PROOF: delete the `semver.Compare(d.Floor, d.GoldenVersion) > 0` branch from
|
||||||
|
// ResolveManagedFloor and TestResolveManagedFloor_R216_FloorAboveGolden FAILS — the box is served
|
||||||
|
// past its agent again, reproducing the campaign's measurement exactly. Demonstrated failing before
|
||||||
|
// these tests were kept.
|
||||||
|
func TestResolveManagedFloor_R216_FloorAboveGolden(t *testing.T) {
|
||||||
|
t.Run("the campaign's exact numbers → HELD, not served", func(t *testing.T) {
|
||||||
|
s := newTestStore(t)
|
||||||
|
_ = s.SetGlobalMinControllerVersion("0.200.0")
|
||||||
|
_ = s.SetArtifactManifest(ArtifactManifest{GoldenVersion: "0.192.0", MinAgent: "0.113.0"})
|
||||||
|
setHostAgent(t, s, "c11", "c11-36d660", "0.120.0")
|
||||||
|
|
||||||
|
fd := s.ResolveManagedFloor("c11")
|
||||||
|
if !fd.Held || fd.Floor != "" {
|
||||||
|
t.Fatalf("R-216 RETURNED: a floor ABOVE the vouched golden was served, so the hub pushed a controller whose agent requirement it does not know: %+v", fd)
|
||||||
|
}
|
||||||
|
if !fd.HeldBeyondGolden {
|
||||||
|
t.Errorf("the hold must be distinguishable from the ordinary below-MinAgent hold (they need different operator text): %+v", fd)
|
||||||
|
}
|
||||||
|
if fd.GoldenVersion != "0.192.0" {
|
||||||
|
t.Errorf("the decision must carry the golden it compared against: %+v", fd)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// The hold is about the FLOOR being unknown, not about the agent — so it holds even for a box
|
||||||
|
// whose agent comfortably exceeds the manifest's MinAgent. That is the whole point: the hub cannot
|
||||||
|
// know what the served version needs.
|
||||||
|
t.Run("held even when the agent clears the manifest's MinAgent", func(t *testing.T) {
|
||||||
|
s := newTestStore(t)
|
||||||
|
_ = s.SetGlobalMinControllerVersion("0.200.0")
|
||||||
|
_ = s.SetArtifactManifest(ArtifactManifest{GoldenVersion: "0.192.0", MinAgent: "0.113.0"})
|
||||||
|
setHostAgent(t, s, "c1", "h1", "0.125.0")
|
||||||
|
|
||||||
|
if fd := s.ResolveManagedFloor("c1"); !fd.Held {
|
||||||
|
t.Fatalf("a floor above the golden must hold regardless of the agent: %+v", fd)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// SCENARIO C — an uncoupled release is untouched. A fleet must not be frozen by a comparison that
|
||||||
|
// now over-fires: with no MinAgent vouched there is no gating at all, above the golden or not.
|
||||||
|
t.Run("uncoupled release above the golden → still served (Scenario C)", func(t *testing.T) {
|
||||||
|
s := newTestStore(t)
|
||||||
|
_ = s.SetGlobalMinControllerVersion("0.200.0")
|
||||||
|
_ = s.SetArtifactManifest(ArtifactManifest{GoldenVersion: "0.192.0", MinAgent: ""})
|
||||||
|
setHostAgent(t, s, "c1", "h1", "0.70.0")
|
||||||
|
|
||||||
|
fd := s.ResolveManagedFloor("c1")
|
||||||
|
if fd.Held || fd.Floor != "0.200.0" {
|
||||||
|
t.Fatalf("an uncoupled release must be served exactly as before: %+v", fd)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// The ordinary arrangement — floor at or below the golden — is completely unchanged.
|
||||||
|
t.Run("floor at the golden → unchanged behaviour", func(t *testing.T) {
|
||||||
|
s := newTestStore(t)
|
||||||
|
_ = s.SetGlobalMinControllerVersion("0.192.0")
|
||||||
|
_ = s.SetArtifactManifest(ArtifactManifest{GoldenVersion: "0.192.0", MinAgent: "0.113.0"})
|
||||||
|
setHostAgent(t, s, "c1", "h1", "0.120.0")
|
||||||
|
|
||||||
|
fd := s.ResolveManagedFloor("c1")
|
||||||
|
if fd.Held || fd.Floor != "0.192.0" {
|
||||||
|
t.Fatalf("floor == golden with a satisfied MinAgent must serve: %+v", fd)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
// An unreadable golden must not gate the fleet — degrade to the pre-existing behaviour.
|
||||||
|
t.Run("unparseable golden → falls through to the agent comparison", func(t *testing.T) {
|
||||||
|
s := newTestStore(t)
|
||||||
|
_ = s.SetGlobalMinControllerVersion("0.200.0")
|
||||||
|
_ = s.SetArtifactManifest(ArtifactManifest{GoldenVersion: "", MinAgent: "0.113.0"})
|
||||||
|
setHostAgent(t, s, "c1", "h1", "0.120.0")
|
||||||
|
|
||||||
|
fd := s.ResolveManagedFloor("c1")
|
||||||
|
if fd.Held || fd.Floor != "0.200.0" {
|
||||||
|
t.Fatalf("an unknown golden must not freeze the fleet: %+v", fd)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|||||||
@@ -1969,16 +1969,51 @@ func (s *Store) EffectiveMinControllerVersion(customerID string) string {
|
|||||||
// to actually serve this customer's controller, whether it is being HELD (and why), and the inputs.
|
// to actually serve this customer's controller, whether it is being HELD (and why), and the inputs.
|
||||||
type ManagedFloorDecision struct {
|
type ManagedFloorDecision struct {
|
||||||
Floor string // the controller-version floor to SERVE ("" = serve none)
|
Floor string // the controller-version floor to SERVE ("" = serve none)
|
||||||
Held bool // true = the floor is withheld because the box's agent is below MinAgent
|
Held bool // true = the floor is withheld (see HeldBeyondGolden for which of the two reasons)
|
||||||
AgentVersion string // the box's reported agent version ("" = unknown → held when MinAgent is set)
|
AgentVersion string // the box's reported agent version ("" = unknown → held when MinAgent is set)
|
||||||
MinAgent string // the manifest's MinAgent for the current golden ("" = uncoupled, no gating)
|
// MinAgent is the manifest's vouched MinAgent — the agent requirement of the GOLDEN's controller
|
||||||
|
// (`ArtifactManifest.MinAgent`). ⚠ It describes the golden, NOT necessarily the version this
|
||||||
|
// decision is about; that gap is what HeldBeyondGolden closes. Empty = uncoupled, no gating.
|
||||||
|
MinAgent string
|
||||||
|
// GoldenVersion is the vouched golden's controller version — the version MinAgent describes.
|
||||||
|
GoldenVersion string
|
||||||
|
// HeldBeyondGolden distinguishes the two hold reasons, because they need different operator text:
|
||||||
|
// false = the box's agent is below MinAgent (the original Part D hold); true = the FLOOR points
|
||||||
|
// ABOVE the vouched golden, so the manifest's MinAgent does not describe the version being served
|
||||||
|
// and the hub does not know its agent requirement (R-216).
|
||||||
|
HeldBeyondGolden bool
|
||||||
}
|
}
|
||||||
|
|
||||||
// ResolveManagedFloor decides the controller-version floor to serve a customer, HOLDING it when the
|
// ResolveManagedFloor decides the controller-version floor to serve a customer, HOLDING it rather
|
||||||
// golden the floor points at requires a newer host agent than the box currently runs (Part D — the
|
// than pushing a controller past the agent it depends on (Part D — the hub-enforced "agent BEFORE
|
||||||
// hub-enforced "agent BEFORE controller floor" rule). Logic:
|
// controller floor" rule).
|
||||||
|
//
|
||||||
|
// ⚠ R-216, and the reason this function changed. `publish-train-rules.md` rule 3 states the rule
|
||||||
|
// about the FLOOR's controller: *"The floor may not effectively push a box past a controller whose
|
||||||
|
// MinAgent that box's agent does not yet meet."* The implementation compared against
|
||||||
|
// `ArtifactManifest.MinAgent`, which by its own doc comment is *"the MINIMUM host-agent version this
|
||||||
|
// GOLDEN's controller requires"*. Those are the same number only while the floor sits at or below the
|
||||||
|
// golden — the arrangement rule 5's ISO gate assumes. Raise a floor above the vouched golden (which
|
||||||
|
// the day-0 runbook explicitly recommends after a golden rebuild, and which a per-customer override
|
||||||
|
// makes trivial) and the guard compares against a version it is not serving.
|
||||||
|
//
|
||||||
|
// Measured live on 2026-08-05 (CAMPAIGN-11 Phase 1): golden 0.192.0 with MinAgent 0.113.0, a
|
||||||
|
// per-customer floor of 0.200.0, and a box on agent 0.120.0. 0.120.0 ≥ 0.113.0, so the floor was
|
||||||
|
// served — and the box was pushed onto a controller needing agent 0.125.0. Its customer was then
|
||||||
|
// told their correct recovery code was wrong. The guard whose comment says it exists to *"never push
|
||||||
|
// a controller past its agent"* had just done exactly that.
|
||||||
|
//
|
||||||
|
// THE FIX, and why it needs no new operator input: the hub cannot know the agent requirement of a
|
||||||
|
// controller version it was never told about, so it must not pretend to. A floor ABOVE the vouched
|
||||||
|
// golden is served blind today; from v0.97.0 it is HELD, with its own reason. Nobody types a number
|
||||||
|
// twice — the existing vouched MinAgent keeps its exact meaning, and the guard simply stops applying
|
||||||
|
// it to versions it does not describe. The operator's remedy is the one the publish train already
|
||||||
|
// prescribes: vouch a golden that carries the floor's controller (rule 1, "manifest before floor").
|
||||||
|
//
|
||||||
|
// Logic:
|
||||||
// - effective floor "" → nothing to serve (no floor configured);
|
// - effective floor "" → nothing to serve (no floor configured);
|
||||||
// - manifest MinAgent "" → UNCOUPLED release: serve the floor as-is (no agent gating);
|
// - manifest MinAgent "" → UNCOUPLED release: serve the floor as-is (no agent gating);
|
||||||
|
// - floor ABOVE the vouched golden's controller → HOLD (requirement unknown) + flag;
|
||||||
// - agent_version known AND ≥ MinAgent → serve the floor;
|
// - agent_version known AND ≥ MinAgent → serve the floor;
|
||||||
// - agent_version below MinAgent, OR unknown/unparseable → HOLD (serve no directive) + flag.
|
// - agent_version below MinAgent, OR unknown/unparseable → HOLD (serve no directive) + flag.
|
||||||
//
|
//
|
||||||
@@ -1988,13 +2023,21 @@ func (s *Store) ResolveManagedFloor(customerID string) ManagedFloorDecision {
|
|||||||
if d.Floor == "" {
|
if d.Floor == "" {
|
||||||
return d
|
return d
|
||||||
}
|
}
|
||||||
d.MinAgent = s.GetArtifactManifest().MinAgent
|
man := s.GetArtifactManifest()
|
||||||
|
d.MinAgent, d.GoldenVersion = man.MinAgent, man.GoldenVersion
|
||||||
if d.MinAgent == "" {
|
if d.MinAgent == "" {
|
||||||
return d // uncoupled release — no agent gate
|
return d // uncoupled release — no agent gate (Scenario C: unchanged from before)
|
||||||
}
|
}
|
||||||
if h, err := s.GetHostByCustomer(customerID); err == nil && h != nil {
|
if h, err := s.GetHostByCustomer(customerID); err == nil && h != nil {
|
||||||
d.AgentVersion = h.AgentVersion
|
d.AgentVersion = h.AgentVersion
|
||||||
}
|
}
|
||||||
|
// R-216: the floor names a controller the manifest does not describe → the agent requirement is
|
||||||
|
// UNKNOWN, so hold. Only when both versions parse — an unreadable golden must not gate the fleet.
|
||||||
|
if semver.Valid(d.Floor) && semver.Valid(d.GoldenVersion) && semver.Compare(d.Floor, d.GoldenVersion) > 0 {
|
||||||
|
d.Held, d.HeldBeyondGolden = true, true
|
||||||
|
d.Floor = ""
|
||||||
|
return d
|
||||||
|
}
|
||||||
if d.AgentVersion != "" && semver.Valid(d.AgentVersion) && semver.Valid(d.MinAgent) &&
|
if d.AgentVersion != "" && semver.Valid(d.AgentVersion) && semver.Valid(d.MinAgent) &&
|
||||||
semver.Compare(d.AgentVersion, d.MinAgent) >= 0 {
|
semver.Compare(d.AgentVersion, d.MinAgent) >= 0 {
|
||||||
return d // agent is new enough — serve the floor
|
return d // agent is new enough — serve the floor
|
||||||
@@ -2830,6 +2873,22 @@ type EscrowStatus struct {
|
|||||||
// withholding is exactly why the precautionary caller had to go: it BLINDED the controller's own
|
// withholding is exactly why the precautionary caller had to go: it BLINDED the controller's own
|
||||||
// hash comparison, which is the measurement that actually detects a changed repository password.
|
// hash comparison, which is the measurement that actually detects a changed repository password.
|
||||||
Stale bool `json:"escrow_stale,omitempty"`
|
Stale bool `json:"escrow_stale,omitempty"`
|
||||||
|
// SupersededPresent (hub v0.97.0, R-222) — this host has at least one RETAINED superseded escrow
|
||||||
|
// row carrying an identity blob. It exists so the recovery screen can distinguish two situations
|
||||||
|
// that are otherwise identical from the box's side: a genuinely wrong recovery code, and a code
|
||||||
|
// that is right about an EARLIER sealed package the hub is deliberately keeping.
|
||||||
|
//
|
||||||
|
// Measured on 2026-08-05 (CAMPAIGN-11 Phase 3, step 7): the customer entered the correct code for
|
||||||
|
// the orphaned history, the unseal failed closed against the CURRENT package — correctly — and the
|
||||||
|
// screen told them to check their typing. Nothing on the box could have known better.
|
||||||
|
//
|
||||||
|
// It is a BOOLEAN, deliberately. It says "an earlier package is kept"; it does not serve one, does
|
||||||
|
// not say whose code opens it, and grants no read path — that link is still unbuilt (R-199's
|
||||||
|
// inventory) and this field must not be mistaken for it.
|
||||||
|
SupersededPresent bool `json:"superseded_present,omitempty"`
|
||||||
|
// SupersededAt (hub v0.97.0, R-222) — when the most recent supersession happened (RFC3339-ish, as
|
||||||
|
// stored). Empty when none. Non-secret: a timestamp.
|
||||||
|
SupersededAt string `json:"superseded_at,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// GetEscrowStatusForCustomer returns the escrow status of the customer's host (nil if the customer has no
|
// GetEscrowStatusForCustomer returns the escrow status of the customer's host (nil if the customer has no
|
||||||
@@ -2857,6 +2916,18 @@ func (s *Store) GetEscrowStatusForCustomer(customerID string) (*EscrowStatus, er
|
|||||||
st.Stale = true
|
st.Stale = true
|
||||||
st.ResticPwSHA256 = ""
|
st.ResticPwSHA256 = ""
|
||||||
}
|
}
|
||||||
|
// R-222 (v0.97.0): does a RETAINED earlier package exist for this host? Only rows that actually
|
||||||
|
// carry an identity blob count — the pre-v0.93.0 rows have `identity_blob` NULL and retain nothing
|
||||||
|
// the recovery screen could ever be talking about, so counting them would make the screen offer an
|
||||||
|
// explanation that is false for exactly the boxes hurt by the original defect.
|
||||||
|
var supersededAt sql.NullString
|
||||||
|
if err := s.db.QueryRow(`
|
||||||
|
SELECT MAX(sup.superseded_at)
|
||||||
|
FROM host_escrow_superseded sup JOIN hosts h ON h.host_id = sup.host_id
|
||||||
|
WHERE h.customer_id = ? AND sup.identity_blob IS NOT NULL`, customerID).Scan(&supersededAt); err == nil && supersededAt.Valid {
|
||||||
|
st.SupersededPresent = true
|
||||||
|
st.SupersededAt = supersededAt.String
|
||||||
|
}
|
||||||
return &st, nil
|
return &st, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user