Files
felhom-controller/controller/internal/agentapi/features.go
T
admin 1e759a16ec R-224/R-226 Part 1: why the unlock failed decides what we say
The failure branch was a two-way choice — superseded? M4 : M1 — and BOTH are
statements about the customer's code. rerr was never inspected, so a hub that
refused, an agent that was stopped and a genuinely mistyped code all produced
the same accusation. Measured live 2026-08-05 with a CORRECT current code: hub
firewalled off 0.0556s, agent stopped 0.0299s, against ~1.0s for a real unseal.

Five classes, from the VALUE and never the text:
  hub-unreachable    502/503 from the agent — the code was NOT used
  agent-unreachable  no agent verdict at all (transport) — NOT used
  no-bundle          404
  bundle-too-old     409
  asked-and-refused  400 — the ONLY class that may mention typing
  unknown            everything else -> NEUTRAL, the safe default

agentapi.RecoveryRefusal carries the status as a value (refusalError flattened
it into a sentence, and a sentence is not something a caller can branch on).

THE OLD-AGENT CASE IS WHY THIS NEEDS A COUPLING. Agent < 0.126.0 answers 400
for both a fetch failure and a wrong code, so a 400 from one cannot be read as
a refusal. FeatureRecoveryFailureClass (MinAgent 0.126.0) withholds that
reading and the 400 degrades to neutral. The gate BLOCKS NOTHING — it only
decides whether the customer may be told to check their typing.

R-226: the superseded message now names BOTH possibilities and restores the
ten-words prompt. The two are indistinguishable at the engine; the honest
message says so. It still does not promise the earlier package can be opened.

Elapsed time is logged (it is what diagnosed this) and is NEVER a classifier.

Tests: scenarios A-E at the HANDLER + the classifier table asserting the same
sentence under two statuses classifies two ways. Red-proofs, each demonstrated
failing then restored: delete the 502 case (A), remove the mistype clause (C),
default to the accusation (D), route an instant transport failure to the typing
message (E).

Two existing tests encoded the defect and were corrected, not deleted: the web
fake returned a BARE error for 'wrong code' (which is the shape of a failure we
cannot classify), and R-222's test forbade any mention of typing on a
superseded box — half of which R-226 deliberately reverses.

28 packages ok, vet clean, all controller gates OK.
2026-08-06 08:06:57 +02:00

304 lines
14 KiB
Go
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
package agentapi
import (
"context"
"errors"
"net/http"
"sync"
"time"
"gitea.dooplex.hu/admin/felhom-controller/internal/logx"
"gitea.dooplex.hu/admin/felhom-controller/internal/util"
)
// Agent-capability probing (the publish-train backstop). A controller release that depends on
// coupled agent behavior must not fail mid-pipeline against an older agent — it detects support up
// front and refuses honestly. Detection is a ROUTE PROBE: a route that shipped together with the
// coupled semantics either answers (2xx ⇒ supported) or 404s (older agent). Transport errors and
// 5xx are INDETERMINATE — an agent problem is never claimed as "too old".
// Feature names one coupled controller↔agent capability.
type Feature string
// FeatureNetstorageVerify is the NAS verify-before-commit add semantics (agent v0.81.0): the
// coupled add behavior shipped together with GET /netstorage/verify-status, so that route IS the
// capability signal.
const FeatureNetstorageVerify Feature = "netstorage_verify"
// FeatureGuestMemoryResize is the guest RAM resize (agent v0.90.0, R-24): the resize endpoints
// (GET/POST /guest/memory) shipped together, so GET /guest/memory IS the capability signal.
const FeatureGuestMemoryResize Feature = "guest_memory_resize"
// FeatureBackupAgeState is R-88 Part 2 (agent v0.105.0): GET /backup/due carries `age_state`,
// distinguishing "never backed up" (absent) from "could not tell" (unknown). There is no route
// probe for it — the signal is a FIELD on an existing route, so the version floor is the gate and
// an empty field means legacy.
const FeatureBackupAgeState Feature = "backup_age_state"
// FeatureOffsiteKeyRecovery is the customer-facing off-site key recovery (agent v0.125.0, R-199
// links 78): POST /escrow/recover-offsite-password fetches this host's sealed bundle, unseals it
// with R and returns the single repository-password field.
//
// ⚠ THIS GATE FAILS CLOSED, and it is the ONLY feature in this table that does. Read §7.1 of the
// R-216 fix before "correcting" it back to the package default.
//
// The package default is fail-OPEN: SupportUnknown proceeds, because for every other coupled feature
// a wrong "unsupported" would block something harmless while a down agent already speaks through the
// normal error paths. **That default is what produced R-216.** Measured live on 2026-08-05
// (CAMPAIGN-11 Phase 1): an agent 0.120.0 answered the recovery route with 404, the unlock attempt
// went ahead anyway, and the customer was told — in Hungarian, on the one screen whose whole purpose
// is to be believed about backups — that their perfectly correct recovery code was not accepted and
// they should check their typing. A correct code, refused in 0.134 s, blamed on the customer.
//
// So here: anything other than SupportYes means the screen says THE MACHINE cannot ask yet. The
// unlock is never attempted when it cannot complete, because the failure of an attempt that could
// never have worked is attributed to the code.
const FeatureOffsiteKeyRecovery Feature = "offsite_key_recovery"
// FeatureRecoveryFailureClass is agent v0.126.0's SPLIT of a failed unlock into distinguishable
// statuses (R-224): 502 the sealed bundle could not be FETCHED · 400 it was fetched and the code was
// refused · 404 no bundle · 409 the bundle predates the repository-password field.
//
// ⚠ WHAT THIS GATE ACTUALLY GUARDS is the meaning of **400**, and nothing else. An agent older than
// v0.126.0 answers 400 for BOTH a fetch failure and a wrong code — one status, one sentence, two
// situations — so on such an agent a 400 cannot be read as "the code was refused". It means "one of
// two things and we cannot tell which", which is `RecoveryUnknown`, which is neutral.
//
// So this gate does not block anything and has no fail-closed behaviour to get wrong: the unlock is
// attempted either way (FeatureOffsiteKeyRecovery already decides THAT). It only decides whether the
// customer may be told to check their typing. Unknown → they may not. **That is the safe direction,
// and it heals itself the moment the agent updates.**
const FeatureRecoveryFailureClass Feature = "recovery_failure_class"
// SupportState is a probe verdict. The zero value is SupportUnknown (fail-open: unknown never
// refuses — the existing agent-error paths speak honestly when the agent is down).
type SupportState int
const (
// SupportUnknown — the probe could not decide (transport error, timeout, auth, 5xx).
SupportUnknown SupportState = iota
// SupportYes — the agent answered 2xx on the feature's probe route.
SupportYes
// SupportNo — the agent answered 404: it predates the route, and with it the coupled semantics.
SupportNo
)
// String returns the state's wire/template vocabulary: "yes" | "no" | "unknown".
func (s SupportState) String() string {
switch s {
case SupportYes:
return "yes"
case SupportNo:
return "no"
default:
return "unknown"
}
}
// SupportProber is the minimal agent surface a probe needs. *Client satisfies it, and so does the
// web layer's netAgent seam — tests inject fakes there.
type SupportProber interface {
NetVerifyStatus(ctx context.Context) (NetVerifyStatus, error)
}
// featureProbes maps each coupled feature to its route probe (the FALLBACK path for agents that
// predate the v0.82.0 version header).
//
// CONVENTION (publish-train rules doc, felhom.eu/documentation/runbooks/publish-train-rules.md):
// every future coupled feature adds a row here AND a featureMinAgent row, plus a Supports gate
// call at its entry point, and declares MinAgent in its CHANGELOG header.
var featureProbes = map[Feature]func(ctx context.Context, p SupportProber) error{
FeatureNetstorageVerify: func(ctx context.Context, p SupportProber) error {
_, err := p.NetVerifyStatus(ctx)
return err
},
// The memory-resize prober needs GET /guest/memory, not NetVerifyStatus. Rather than couple the
// shared SupportProber (and every unrelated prober/fake) to the memory surface, the probe
// type-asserts the ONE method it needs — the memory feature is only ever probed with a
// GuestMemory-capable prober (the web memAgent seam / *Client). A prober without it → a non-404
// error → SupportUnknown (fail-open), never a false "supported".
FeatureGuestMemoryResize: func(ctx context.Context, p SupportProber) error {
gm, ok := p.(interface {
GuestMemory(ctx context.Context) (GuestMemoryInfo, error)
})
if !ok {
return errNoMemoryProbe
}
_, err := gm.GuestMemory(ctx)
return err
},
// The recovery route is a POST that performs work and consumes a recovery code — it cannot be
// probed. Like the memory prober's negative case this returns a sentinel that classifies to
// SupportUnknown, so the decision falls to the VERSION path above.
//
// The row must exist even though it cannot probe: SupportsWithSource looks up featureProbes
// FIRST and returns "unregistered"/SupportUnknown on a table gap, before the version path runs.
// A featureMinAgent row without a featureProbes row is therefore never consulted at all.
FeatureOffsiteKeyRecovery: func(ctx context.Context, p SupportProber) error {
return errNoRecoveryProbe
},
// Same POST route, same reason it cannot be probed — the decision falls to the VERSION path.
FeatureRecoveryFailureClass: func(ctx context.Context, p SupportProber) error {
return errNoRecoveryProbe
},
}
// errNoMemoryProbe classifies to SupportUnknown (not a *StatusError 404), so a prober that cannot be
// asked never reads as "unsupported".
var errNoMemoryProbe = errors.New("agentapi: prober does not support the guest-memory probe")
// errNoRecoveryProbe classifies to SupportUnknown: the off-site key recovery route is a POST that
// consumes a recovery code and so cannot be probed, leaving the VERSION path to decide. Its caller
// fails CLOSED on Unknown — see FeatureOffsiteKeyRecovery.
var errNoRecoveryProbe = errors.New("agentapi: the offsite key recovery route cannot be probed")
// featureMinAgent maps each coupled feature to the MINIMUM agent version that carries its coupled
// semantics (the CHANGELOG `MinAgent:` header value). Used by Supports when the agent's version is
// KNOWN (the v0.82.0 X-Felhom-Agent-Version channel) — a direct comparison, no probe traffic. A
// feature missing here (or an unparseable table value) falls back to the probe.
var featureMinAgent = map[Feature]string{
FeatureNetstorageVerify: "0.81.0",
FeatureGuestMemoryResize: "0.90.0",
// R-88 Part 2: /backup/due carries age_state, distinguishing "never backed up" from "cannot tell".
FeatureBackupAgeState: "0.105.0",
// R-199 links 78: POST /escrow/recover-offsite-password. R-216 — this row is the whole reason a
// correct recovery code can no longer be reported as wrong on an agent that cannot answer.
FeatureOffsiteKeyRecovery: "0.125.0",
// R-224 — the four-way status split of a failed unlock.
FeatureRecoveryFailureClass: "0.126.0",
}
// MinAgentFor returns the declared minimum agent version for a feature ("" when the feature has no
// row). Read-only accessor over featureMinAgent so a refusal can NAME the version it needs instead of
// hard-coding the number a second time at the call site.
func MinAgentFor(f Feature) string { return featureMinAgent[f] }
// AgentVersionReporter is optionally implemented by a SupportProber (*Client is one): it reports
// the last strictly-validated agent version seen on its traffic ("" = unknown → probe fallback).
type AgentVersionReporter interface {
AgentVersion() string
}
// supportTTL bounds how long a probe verdict (either polarity) is trusted. An agent updated
// mid-window flips within this — no invalidation plumbing by design.
const supportTTL = 5 * time.Minute
type supportEntry struct {
state SupportState
at time.Time
}
// SupportCache caches per-feature probe verdicts. Yes and No are cached for supportTTL; Unknown is
// NEVER cached (a down agent re-probes on the next call, so recovery is immediate). The zero value
// is ready to use.
type SupportCache struct {
mu sync.Mutex
now func() time.Time // test seam; nil → time.Now
entries map[Feature]supportEntry
}
// Supports reports whether the agent behind p provides feature f.
//
// Order (v0.115.0): (1) the agent's VERSION is known (the v0.82.0 header channel, strictly
// validated at capture) AND the feature has a MinAgent row → pure semver comparison, NO probe
// traffic, no cache involvement (the cache stays probe-only); (2) otherwise — pre-0.82 agent, no
// traffic yet, or a table gap — the v0.114.0 probe path, byte-identical (cache inside the TTL
// window, probe on miss). The probe runs OUTSIDE the lock — concurrent misses may double-probe
// (harmless: the probe is one cheap GET).
func (sc *SupportCache) Supports(ctx context.Context, p SupportProber, f Feature) SupportState {
state, _ := sc.SupportsWithSource(ctx, p, f)
return state
}
// SupportsWithSource is Supports plus the DECISION SOURCE ("version" | "probe-cache" |
// "probe" | "unregistered") — the v0.116.0 observability extension so the gate line can
// say HOW the verdict was reached. Behavior is byte-identical to v0.115.0 Supports.
func (sc *SupportCache) SupportsWithSource(ctx context.Context, p SupportProber, f Feature) (SupportState, string) {
probe, ok := featureProbes[f]
if !ok {
return SupportUnknown, "unregistered" // unregistered feature — never refuse on a table gap
}
if vr, hasVer := p.(AgentVersionReporter); hasVer {
if state, decided := supportsByVersion(vr.AgentVersion(), f); decided {
return state, "version"
}
}
sc.mu.Lock()
nowFn := sc.now
if nowFn == nil {
nowFn = time.Now
}
if e, hit := sc.entries[f]; hit && nowFn().Sub(e.at) < supportTTL {
sc.mu.Unlock()
return e.state, "probe-cache"
}
sc.mu.Unlock()
state := classifySupportErr(probe(ctx, p))
if state != SupportUnknown {
sc.mu.Lock()
if sc.entries == nil {
sc.entries = map[Feature]supportEntry{}
}
sc.entries[f] = supportEntry{state: state, at: nowFn()}
sc.mu.Unlock()
}
return state, "probe"
}
// Supports probes (cached, TTL 5m, both polarities) whether the connected agent provides the
// feature. 2xx ⇒ Yes. 404 ⇒ No. Anything else ⇒ Unknown (never "too old"). The web layer drives
// the same machinery through its netAgent seam (Server.netFeatures) so tests can fake the probe.
func (c *Client) Supports(ctx context.Context, f Feature) SupportState {
state, source := c.features.SupportsWithSource(ctx, c, f)
logx.Debugf(c.logger, "[agentapi] Supports(%s) = %s (source=%s agent_version=%q)",
f, state, source, c.AgentVersion())
return state
}
// supportsByVersion decides a feature by version comparison alone. decided=false (unknown/garbage
// version, missing MinAgent row, unparseable table value) sends the caller to the probe fallback —
// a bad version string must never be trusted in EITHER direction.
func supportsByVersion(agentVer string, f Feature) (state SupportState, decided bool) {
if agentVer == "" {
return SupportUnknown, false
}
minStr, ok := featureMinAgent[f]
if !ok {
return SupportUnknown, false
}
av, err := util.ParseVersion(agentVer)
if err != nil {
return SupportUnknown, false // capture validates the shape, but stay defensive
}
mv, err := util.ParseVersion(minStr)
if err != nil {
return SupportUnknown, false // a broken table row falls back to probing, never refuses
}
if av.Compare(mv) >= 0 {
return SupportYes, true
}
return SupportNo, true
}
// classifySupportErr maps a probe outcome to a SupportState. ONLY a typed HTTP 404 means
// "unsupported" — every other error (transport, timeout, 401, 5xx, envelope problems) is Unknown,
// so a merely-down agent is never reported as outdated. Typed errors only; never string-match.
func classifySupportErr(err error) SupportState {
if err == nil {
return SupportYes
}
var se *StatusError
if errors.As(err, &se) && se.Code == http.StatusNotFound {
return SupportNo
}
return SupportUnknown
}
// AgentVersionReporter witness. Asserted at SupportsWithSource (`p.(AgentVersionReporter)`); a failed
// assertion falls back from the version gate to the live probe. That degrade is benign — both paths
// decide correctly — but *Client is the production prober and losing the version path would silently
// turn every MinAgent floor into a probe round-trip, which is a behaviour change nobody would see.
var _ AgentVersionReporter = (*Client)(nil)