a3499d1807
gates / gates (push) Successful in 9s
R-216: the offsite key recovery is a coupled feature and now says so. featureProbes +
featureMinAgent 0.125.0 + a Supports gate at the unlock entry point, FAILING CLOSED — an
agent that cannot answer is named as such instead of the customer's code being blamed.
Measured live: a 404 from agent 0.120.0 came back as "we did not accept your recovery
code, check that all ten words", in 0.134 s, against a perfect code.
R-218: delete the repo-password short-circuit in needsOffsiteCredential. The declaration
stops when the TIER WORKS, not when a key exists — installing a key is the recovery
screen's whole job, so succeeding at recovery was switching off the mechanism that would
have delivered the coordinates to use it.
R-219: the unlock finishes the job — place the key, bring the tier up, then list. Without
it the promised listing could never render on the shape the screen exists for.
R-217: an unreadable store no longer claims to have opened with unattributable content
(the OffsiteInventory{} zero value). Opened / empty / unreadable are three states.
R-222: a code that is right about a RETAINED earlier package is named, not blamed. States
what the hub knows and promises nothing — no read path exists.
R-215: GET /recovery is gated on the same predicate as the interception.
Five red-proofs, each demonstrated failing and restored.
282 lines
13 KiB
Go
282 lines
13 KiB
Go
package agentapi
|
||
|
||
import (
|
||
"context"
|
||
"errors"
|
||
"net/http"
|
||
"sync"
|
||
"time"
|
||
|
||
"gitea.dooplex.hu/admin/felhom-controller/internal/logx"
|
||
"gitea.dooplex.hu/admin/felhom-controller/internal/util"
|
||
)
|
||
|
||
// Agent-capability probing (the publish-train backstop). A controller release that depends on
|
||
// coupled agent behavior must not fail mid-pipeline against an older agent — it detects support up
|
||
// front and refuses honestly. Detection is a ROUTE PROBE: a route that shipped together with the
|
||
// coupled semantics either answers (2xx ⇒ supported) or 404s (older agent). Transport errors and
|
||
// 5xx are INDETERMINATE — an agent problem is never claimed as "too old".
|
||
|
||
// Feature names one coupled controller↔agent capability.
|
||
type Feature string
|
||
|
||
// FeatureNetstorageVerify is the NAS verify-before-commit add semantics (agent v0.81.0): the
|
||
// coupled add behavior shipped together with GET /netstorage/verify-status, so that route IS the
|
||
// capability signal.
|
||
const FeatureNetstorageVerify Feature = "netstorage_verify"
|
||
|
||
// FeatureGuestMemoryResize is the guest RAM resize (agent v0.90.0, R-24): the resize endpoints
|
||
// (GET/POST /guest/memory) shipped together, so GET /guest/memory IS the capability signal.
|
||
const FeatureGuestMemoryResize Feature = "guest_memory_resize"
|
||
|
||
// FeatureBackupAgeState is R-88 Part 2 (agent v0.105.0): GET /backup/due carries `age_state`,
|
||
// distinguishing "never backed up" (absent) from "could not tell" (unknown). There is no route
|
||
// probe for it — the signal is a FIELD on an existing route, so the version floor is the gate and
|
||
// an empty field means legacy.
|
||
const FeatureBackupAgeState Feature = "backup_age_state"
|
||
|
||
// FeatureOffsiteKeyRecovery is the customer-facing off-site key recovery (agent v0.125.0, R-199
|
||
// links 7–8): POST /escrow/recover-offsite-password fetches this host's sealed bundle, unseals it
|
||
// with R and returns the single repository-password field.
|
||
//
|
||
// ⚠ THIS GATE FAILS CLOSED, and it is the ONLY feature in this table that does. Read §7.1 of the
|
||
// R-216 fix before "correcting" it back to the package default.
|
||
//
|
||
// The package default is fail-OPEN: SupportUnknown proceeds, because for every other coupled feature
|
||
// a wrong "unsupported" would block something harmless while a down agent already speaks through the
|
||
// normal error paths. **That default is what produced R-216.** Measured live on 2026-08-05
|
||
// (CAMPAIGN-11 Phase 1): an agent 0.120.0 answered the recovery route with 404, the unlock attempt
|
||
// went ahead anyway, and the customer was told — in Hungarian, on the one screen whose whole purpose
|
||
// is to be believed about backups — that their perfectly correct recovery code was not accepted and
|
||
// they should check their typing. A correct code, refused in 0.134 s, blamed on the customer.
|
||
//
|
||
// So here: anything other than SupportYes means the screen says THE MACHINE cannot ask yet. The
|
||
// unlock is never attempted when it cannot complete, because the failure of an attempt that could
|
||
// never have worked is attributed to the code.
|
||
const FeatureOffsiteKeyRecovery Feature = "offsite_key_recovery"
|
||
|
||
// SupportState is a probe verdict. The zero value is SupportUnknown (fail-open: unknown never
|
||
// refuses — the existing agent-error paths speak honestly when the agent is down).
|
||
type SupportState int
|
||
|
||
const (
|
||
// SupportUnknown — the probe could not decide (transport error, timeout, auth, 5xx).
|
||
SupportUnknown SupportState = iota
|
||
// SupportYes — the agent answered 2xx on the feature's probe route.
|
||
SupportYes
|
||
// SupportNo — the agent answered 404: it predates the route, and with it the coupled semantics.
|
||
SupportNo
|
||
)
|
||
|
||
// String returns the state's wire/template vocabulary: "yes" | "no" | "unknown".
|
||
func (s SupportState) String() string {
|
||
switch s {
|
||
case SupportYes:
|
||
return "yes"
|
||
case SupportNo:
|
||
return "no"
|
||
default:
|
||
return "unknown"
|
||
}
|
||
}
|
||
|
||
// SupportProber is the minimal agent surface a probe needs. *Client satisfies it, and so does the
|
||
// web layer's netAgent seam — tests inject fakes there.
|
||
type SupportProber interface {
|
||
NetVerifyStatus(ctx context.Context) (NetVerifyStatus, error)
|
||
}
|
||
|
||
// featureProbes maps each coupled feature to its route probe (the FALLBACK path for agents that
|
||
// predate the v0.82.0 version header).
|
||
//
|
||
// CONVENTION (publish-train rules doc, felhom.eu/documentation/runbooks/publish-train-rules.md):
|
||
// every future coupled feature adds a row here AND a featureMinAgent row, plus a Supports gate
|
||
// call at its entry point, and declares MinAgent in its CHANGELOG header.
|
||
var featureProbes = map[Feature]func(ctx context.Context, p SupportProber) error{
|
||
FeatureNetstorageVerify: func(ctx context.Context, p SupportProber) error {
|
||
_, err := p.NetVerifyStatus(ctx)
|
||
return err
|
||
},
|
||
// The memory-resize prober needs GET /guest/memory, not NetVerifyStatus. Rather than couple the
|
||
// shared SupportProber (and every unrelated prober/fake) to the memory surface, the probe
|
||
// type-asserts the ONE method it needs — the memory feature is only ever probed with a
|
||
// GuestMemory-capable prober (the web memAgent seam / *Client). A prober without it → a non-404
|
||
// error → SupportUnknown (fail-open), never a false "supported".
|
||
FeatureGuestMemoryResize: func(ctx context.Context, p SupportProber) error {
|
||
gm, ok := p.(interface {
|
||
GuestMemory(ctx context.Context) (GuestMemoryInfo, error)
|
||
})
|
||
if !ok {
|
||
return errNoMemoryProbe
|
||
}
|
||
_, err := gm.GuestMemory(ctx)
|
||
return err
|
||
},
|
||
// The recovery route is a POST that performs work and consumes a recovery code — it cannot be
|
||
// probed. Like the memory prober's negative case this returns a sentinel that classifies to
|
||
// SupportUnknown, so the decision falls to the VERSION path above.
|
||
//
|
||
// The row must exist even though it cannot probe: SupportsWithSource looks up featureProbes
|
||
// FIRST and returns "unregistered"/SupportUnknown on a table gap, before the version path runs.
|
||
// A featureMinAgent row without a featureProbes row is therefore never consulted at all.
|
||
FeatureOffsiteKeyRecovery: func(ctx context.Context, p SupportProber) error {
|
||
return errNoRecoveryProbe
|
||
},
|
||
}
|
||
|
||
// errNoMemoryProbe classifies to SupportUnknown (not a *StatusError 404), so a prober that cannot be
|
||
// asked never reads as "unsupported".
|
||
var errNoMemoryProbe = errors.New("agentapi: prober does not support the guest-memory probe")
|
||
|
||
// errNoRecoveryProbe classifies to SupportUnknown: the off-site key recovery route is a POST that
|
||
// consumes a recovery code and so cannot be probed, leaving the VERSION path to decide. Its caller
|
||
// fails CLOSED on Unknown — see FeatureOffsiteKeyRecovery.
|
||
var errNoRecoveryProbe = errors.New("agentapi: the offsite key recovery route cannot be probed")
|
||
|
||
// featureMinAgent maps each coupled feature to the MINIMUM agent version that carries its coupled
|
||
// semantics (the CHANGELOG `MinAgent:` header value). Used by Supports when the agent's version is
|
||
// KNOWN (the v0.82.0 X-Felhom-Agent-Version channel) — a direct comparison, no probe traffic. A
|
||
// feature missing here (or an unparseable table value) falls back to the probe.
|
||
var featureMinAgent = map[Feature]string{
|
||
FeatureNetstorageVerify: "0.81.0",
|
||
FeatureGuestMemoryResize: "0.90.0",
|
||
// R-88 Part 2: /backup/due carries age_state, distinguishing "never backed up" from "cannot tell".
|
||
FeatureBackupAgeState: "0.105.0",
|
||
// R-199 links 7–8: POST /escrow/recover-offsite-password. R-216 — this row is the whole reason a
|
||
// correct recovery code can no longer be reported as wrong on an agent that cannot answer.
|
||
FeatureOffsiteKeyRecovery: "0.125.0",
|
||
}
|
||
|
||
// MinAgentFor returns the declared minimum agent version for a feature ("" when the feature has no
|
||
// row). Read-only accessor over featureMinAgent so a refusal can NAME the version it needs instead of
|
||
// hard-coding the number a second time at the call site.
|
||
func MinAgentFor(f Feature) string { return featureMinAgent[f] }
|
||
|
||
// AgentVersionReporter is optionally implemented by a SupportProber (*Client is one): it reports
|
||
// the last strictly-validated agent version seen on its traffic ("" = unknown → probe fallback).
|
||
type AgentVersionReporter interface {
|
||
AgentVersion() string
|
||
}
|
||
|
||
// supportTTL bounds how long a probe verdict (either polarity) is trusted. An agent updated
|
||
// mid-window flips within this — no invalidation plumbing by design.
|
||
const supportTTL = 5 * time.Minute
|
||
|
||
type supportEntry struct {
|
||
state SupportState
|
||
at time.Time
|
||
}
|
||
|
||
// SupportCache caches per-feature probe verdicts. Yes and No are cached for supportTTL; Unknown is
|
||
// NEVER cached (a down agent re-probes on the next call, so recovery is immediate). The zero value
|
||
// is ready to use.
|
||
type SupportCache struct {
|
||
mu sync.Mutex
|
||
now func() time.Time // test seam; nil → time.Now
|
||
entries map[Feature]supportEntry
|
||
}
|
||
|
||
// Supports reports whether the agent behind p provides feature f.
|
||
//
|
||
// Order (v0.115.0): (1) the agent's VERSION is known (the v0.82.0 header channel, strictly
|
||
// validated at capture) AND the feature has a MinAgent row → pure semver comparison, NO probe
|
||
// traffic, no cache involvement (the cache stays probe-only); (2) otherwise — pre-0.82 agent, no
|
||
// traffic yet, or a table gap — the v0.114.0 probe path, byte-identical (cache inside the TTL
|
||
// window, probe on miss). The probe runs OUTSIDE the lock — concurrent misses may double-probe
|
||
// (harmless: the probe is one cheap GET).
|
||
func (sc *SupportCache) Supports(ctx context.Context, p SupportProber, f Feature) SupportState {
|
||
state, _ := sc.SupportsWithSource(ctx, p, f)
|
||
return state
|
||
}
|
||
|
||
// SupportsWithSource is Supports plus the DECISION SOURCE ("version" | "probe-cache" |
|
||
// "probe" | "unregistered") — the v0.116.0 observability extension so the gate line can
|
||
// say HOW the verdict was reached. Behavior is byte-identical to v0.115.0 Supports.
|
||
func (sc *SupportCache) SupportsWithSource(ctx context.Context, p SupportProber, f Feature) (SupportState, string) {
|
||
probe, ok := featureProbes[f]
|
||
if !ok {
|
||
return SupportUnknown, "unregistered" // unregistered feature — never refuse on a table gap
|
||
}
|
||
if vr, hasVer := p.(AgentVersionReporter); hasVer {
|
||
if state, decided := supportsByVersion(vr.AgentVersion(), f); decided {
|
||
return state, "version"
|
||
}
|
||
}
|
||
sc.mu.Lock()
|
||
nowFn := sc.now
|
||
if nowFn == nil {
|
||
nowFn = time.Now
|
||
}
|
||
if e, hit := sc.entries[f]; hit && nowFn().Sub(e.at) < supportTTL {
|
||
sc.mu.Unlock()
|
||
return e.state, "probe-cache"
|
||
}
|
||
sc.mu.Unlock()
|
||
|
||
state := classifySupportErr(probe(ctx, p))
|
||
if state != SupportUnknown {
|
||
sc.mu.Lock()
|
||
if sc.entries == nil {
|
||
sc.entries = map[Feature]supportEntry{}
|
||
}
|
||
sc.entries[f] = supportEntry{state: state, at: nowFn()}
|
||
sc.mu.Unlock()
|
||
}
|
||
return state, "probe"
|
||
}
|
||
|
||
// Supports probes (cached, TTL 5m, both polarities) whether the connected agent provides the
|
||
// feature. 2xx ⇒ Yes. 404 ⇒ No. Anything else ⇒ Unknown (never "too old"). The web layer drives
|
||
// the same machinery through its netAgent seam (Server.netFeatures) so tests can fake the probe.
|
||
func (c *Client) Supports(ctx context.Context, f Feature) SupportState {
|
||
state, source := c.features.SupportsWithSource(ctx, c, f)
|
||
logx.Debugf(c.logger, "[agentapi] Supports(%s) = %s (source=%s agent_version=%q)",
|
||
f, state, source, c.AgentVersion())
|
||
return state
|
||
}
|
||
|
||
// supportsByVersion decides a feature by version comparison alone. decided=false (unknown/garbage
|
||
// version, missing MinAgent row, unparseable table value) sends the caller to the probe fallback —
|
||
// a bad version string must never be trusted in EITHER direction.
|
||
func supportsByVersion(agentVer string, f Feature) (state SupportState, decided bool) {
|
||
if agentVer == "" {
|
||
return SupportUnknown, false
|
||
}
|
||
minStr, ok := featureMinAgent[f]
|
||
if !ok {
|
||
return SupportUnknown, false
|
||
}
|
||
av, err := util.ParseVersion(agentVer)
|
||
if err != nil {
|
||
return SupportUnknown, false // capture validates the shape, but stay defensive
|
||
}
|
||
mv, err := util.ParseVersion(minStr)
|
||
if err != nil {
|
||
return SupportUnknown, false // a broken table row falls back to probing, never refuses
|
||
}
|
||
if av.Compare(mv) >= 0 {
|
||
return SupportYes, true
|
||
}
|
||
return SupportNo, true
|
||
}
|
||
|
||
// classifySupportErr maps a probe outcome to a SupportState. ONLY a typed HTTP 404 means
|
||
// "unsupported" — every other error (transport, timeout, 401, 5xx, envelope problems) is Unknown,
|
||
// so a merely-down agent is never reported as outdated. Typed errors only; never string-match.
|
||
func classifySupportErr(err error) SupportState {
|
||
if err == nil {
|
||
return SupportYes
|
||
}
|
||
var se *StatusError
|
||
if errors.As(err, &se) && se.Code == http.StatusNotFound {
|
||
return SupportNo
|
||
}
|
||
return SupportUnknown
|
||
}
|
||
|
||
// AgentVersionReporter witness. Asserted at SupportsWithSource (`p.(AgentVersionReporter)`); a failed
|
||
// assertion falls back from the version gate to the live probe. That degrade is benign — both paths
|
||
// decide correctly — but *Client is the production prober and losing the version path would silently
|
||
// turn every MinAgent floor into a probe round-trip, which is a behaviour change nobody would see.
|
||
var _ AgentVersionReporter = (*Client)(nil)
|