Files
felhom-agent/internal/localapi/escrow_recover.go
T
admin 1db56bf837
gates / gates (push) Successful in 14s
v0.129.0 — a correct code for an earlier package stops being called wrong (R-311)
Yesterday's drill proved a retained escrow package opens a set-aside store and
restores planted files byte-identical, while this agent answered the customer's
correct code with "the recovery code did not open the sealed bundle". Nothing had
ever tried the retained packages, so a correct-but-earlier code and a mistype were
genuinely indistinguishable.

OffsiteKeyRecoverer gains an optional FetchRetained, consulted ONLY after the
current package refuses, so the ordinary recovery pays nothing for it and cannot
fail because of it. A match returns ErrCodeOpensRetained wrapped in a
RetainedOpenedError carrying the supersession date - no material, no code, no
password. The local API answers 422: a FIFTH status added to the R-224 switch,
never a restructuring of it.

Fail-safe in every direction. Nil fetcher, a hub too old for the route (404 is a
clean "none"), a transport failure, a malformed package: each leaves the original
refusal standing. Attempts bounded at 6 because each unwrap is ~1s of scrypt.

Seven tests with REAL age crypto - the two situations are indistinguishable AT
THE UNWRAP, so a faked unwrap would prove nothing. Red-proof asserted applied:
remove the retained lookup and the fail-closed wrong-code error returns, which is
the lie in those exact words.
2026-08-12 18:40:00 +02:00

154 lines
8.5 KiB
Go

package localapi
import (
"context"
"crypto/sha256"
"encoding/hex"
"errors"
"net/http"
"strings"
"time"
"gitea.dooplex.hu/admin/felhom-agent/internal/escrow"
)
// R-199 (agent v0.125.0) — the in-guest controller asks the agent to recover the offsite repository
// password from the hub's sealed bundle, using the customer's recovery code R.
//
// WHY THE AGENT AND NOT THE CONTROLLER. Three reasons, all structural: the unsealing binary (`age`)
// is an agent runtime dependency and is deliberately absent from the controller image; the sealed
// blob is a HOST-scoped object whose only writer is this agent under the per-host key, so the read is
// that write's mirror; and the controller is a trust tier down — it should receive one field, not a
// bundle it has no use for.
//
// R'S HANDLING, WHICH IS THE TIGHTEST RULE IN THIS FLOW. R is the one secret in the system that
// cannot be rotated, re-issued or recovered — it exists only in the customer's hands. Here it:
// - arrives in the request body over the already-pinned local-API channel (the operator accepted
// that crossing on 2026-08-04; the acceptance covers the CHANNEL, not carelessness at either end);
// - is held in memory for the duration of one call and cleared on BOTH paths;
// - is never written to disk, never an argument in a process list, and never logged at any level,
// including inside an error;
// - is never echoed: no response this endpoint can emit contains it.
//
// The request-level DEBUG middleware logs method/path/status/duration and never bodies — see
// `logRequests`. Do not add a body dump.
//
// THE RESPONSE CARRIES THE PASSWORD AND ITS HASH. The hash is what this session's proof compares
// (compare by hash, never by value). The password itself is present because the next link — placing a
// recovered password so the existing repository opens — needs it, and building a hash-only seam now
// would have to be torn out to add it. The controller's diagnostic reads only the hash.
type recoverOffsitePasswordRequest struct {
VMID int `json:"vmid"`
// RecoveryCode is the customer's R. NEVER logged, never persisted, never echoed.
RecoveryCode string `json:"recovery_code"`
}
// handleRecoverOffsitePassword fetches this host's sealed bundle, unseals it with R and returns only
// the offsite repository password (plus its sha256, for hash-only comparison by the caller).
func (s *Server) handleRecoverOffsitePassword(w http.ResponseWriter, r *http.Request, vmid int) {
var req recoverOffsitePasswordRequest
if !decodeBody(w, r, &req) {
return
}
if !s.scopedFromBody(w, req.VMID, vmid, r.URL.Path) {
return
}
R := strings.TrimSpace(req.RecoveryCode)
req.RecoveryCode = "" // drop the decoded copy immediately
if R == "" {
writeErr(w, http.StatusBadRequest, "recovery_code is required")
return
}
if s.escrowRecovery == nil {
R = ""
writeErr(w, http.StatusServiceUnavailable, "offsite key recovery is not configured on this agent (no hub client)")
return
}
ctx, cancel := context.WithTimeout(r.Context(), 60*time.Second)
defer cancel()
s.logger.Info("local-api: recovering the offsite repository password from the sealed escrow (R via body, never logged/persisted)", "vmid", vmid)
pw, err := s.escrowRecovery.RecoverOffsiteRepoPassword(ctx, R)
R = "" // cleared on BOTH paths, before anything else can happen
if err != nil {
// Each situation gets its own status and its own words. None of them names a secret.
switch {
// ── R-224 (2026-08-06) — THE FETCH FAILURE IS NOT A WRONG CODE. ────────────────────────
//
// This case did not exist, and its absence is the defect. A failed fetch fell through to the
// `default` below and was answered with "the recovery code did not open the sealed bundle" —
// so a hub that could not be reached was reported to the customer as a bad recovery code, on
// the one screen whose whole purpose is to be believed about their backups.
//
// Measured live 2026-08-05 (CAMPAIGN-11 F3 and F4): a CORRECT current code returned that
// message in 0.0556 s with the hub firewalled off, and in 0.0299 s with this agent stopped —
// against ~1.0 s for a genuine unseal. No unseal was attempted in either case.
//
// 502 rather than 400: 4xx says "your request was bad", and the request was not bad — an
// upstream dependency failed. The status is the machine-readable half; the controller
// classifies on it and must never parse this sentence.
//
// ⚠ THE CODE WAS NOT USED. Nothing may be said about it — not that it was wrong, and not
// that it was right.
case errors.Is(err, escrow.ErrBundleFetch):
s.logger.Warn("local-api: offsite key recovery: the sealed bundle could not be FETCHED — the recovery code was never used", "vmid", vmid, "err", err)
writeErr(w, http.StatusBadGateway, "the sealed recovery bundle could not be fetched from the hub — the recovery code was NOT used and nothing was written")
case errors.Is(err, escrow.ErrNoEscrowBlob):
s.logger.Warn("local-api: offsite key recovery: the hub holds no sealed bundle for this host", "vmid", vmid)
writeErr(w, http.StatusNotFound, "the hub holds no sealed recovery bundle for this host — no escrow ceremony has run")
// ── R-311 (2026-08-12) — THE CODE IS RIGHT, JUST NOT FOR THE CURRENT PACKAGE. ─────────
//
// Placed ABOVE the default for the same reason ErrBundleFetch is: the default blames the
// customer, and this case is the one where the customer is provably not at fault. The code was
// used, it worked, and it opened a package the hub is deliberately keeping.
//
// 422 rather than 400: the request was well-formed AND the credential was valid — what could
// not be processed is the pairing of a correct code with the CURRENT package. A 400 would put
// it in the same bucket as a mistype, which is the whole defect. The status is the
// machine-readable half; the controller classifies on it and must never parse this sentence.
//
// The date travels in the body because it is the one fact that lets a customer recognise which
// code they are holding. No material, no code, no password — only when that package stopped
// being current, and whether it can yield a repository password at all.
case errors.Is(err, escrow.ErrCodeOpensRetained):
var ro *escrow.RetainedOpenedError
match := escrow.RetainedMatch{}
if errors.As(err, &ro) {
match = ro.Match
}
s.logger.Info("local-api: offsite key recovery: the code did NOT open the current package but DID open a RETAINED one — the customer is not at fault",
"vmid", vmid, "superseded_at", match.SupersededAt, "retained_has_restic_pw", match.HasResticPassword)
writeStatus(w, http.StatusUnprocessableEntity, false,
map[string]any{
"opens_retained": true,
"superseded_at": match.SupersededAt,
"retained_has_restic_pw": match.HasResticPassword,
},
"the recovery code is correct, but it belongs to an EARLIER sealed package (superseded "+match.SupersededAt+"), not the one currently held")
case errors.Is(err, escrow.ErrNoResticPassword):
s.logger.Warn("local-api: offsite key recovery: the bundle opened but predates the repository-password field", "vmid", vmid)
writeErr(w, http.StatusConflict, "the recovery code opened the bundle, but it carries NO offsite repository password (sealed before that field existed; it cannot be retro-fitted)")
default:
// The fail-closed WRONG-CODE case, and only it: the bundle was fetched and `age -d`
// refused it. Every other situation above has its own status. The agent log records the
// STEP, never the code.
s.logger.Warn("local-api: offsite key recovery: the fetched bundle did not open with the supplied recovery code", "vmid", vmid, "err", err)
writeErr(w, http.StatusBadRequest, "the recovery code did not open the sealed bundle — nothing was written")
}
return
}
sum := sha256.Sum256([]byte(strings.TrimSpace(pw)))
// §8.6's lesson, applied: say exactly WHAT was recovered and what was NOT, so nobody reading this
// concludes the wrong thing about the bundle's contents (which is how link 8 came to be missing).
s.logger.Info("local-api: offsite repository password RECOVERED from the sealed escrow — returning that field ONLY "+
"(the tunnel token, the PBS token and the WG key stay inside the agent and are not returned)",
"vmid", vmid, "restic_pw_sha256", hex.EncodeToString(sum[:]))
writeOK(w, map[string]any{
"restic_repo_password": pw,
"restic_pw_sha256": hex.EncodeToString(sum[:]),
})
}