admin/felhom-controller
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

R-241 part 1: the box does not mint a repository key over a sealed package

Browse Source 
THE DEFECT. WriteOffboxSecrets auto-generated on ONE input - does the file
exist. Its two neighbours in the same file, OffsiteRecoveryOffer and
needsOffsiteCredential, both consult GetHubEscrowIdentityPresent(). The same
fact was available on three paths and used on two.

Measured on the final walk: a rebuilt box's credential self-heal reached here
at 03:18:06Z and minted 9b4a9a9d over a hub package sealing 30ef574f. The
recovery screen then correctly reported nothing recoverable under the key the
box held. The screen was honest; the minting was not. And the flag was not
merely available at that moment - it was the PRECONDITION of the chain that
reached this function, logged at 02:48:03Z, six ticks earlier.

THE GUARD IS A CONJUNCTION, deliberately: a package held AND no key present.
A box the hub holds nothing for mints exactly as before.

The refusal is a HOLDING state, not a failure. ApplyOffsiteTarget catches the
sentinel and still writes the transport (ssh key, known_hosts, coordinates),
so the recovery screen can bring the tier up the instant the escrowed key is
placed (R-219). Returning the error instead would leave needsOffsiteCredential
true forever and the hub re-staging a consumed credential on every cycle.

New declared state offsite.state=awaiting_recovery_key, shown INERT to every
existing hub reader from their code rather than assumed: offsiteheal acts on
exactly one string; isStale needs Enabled && escrowed and this carries
Enabled=false; the delivery checker skips the applied shape; an unknown state
string is ignored by encoding/json. So NO hub change is needed for this part.

OffboxAwaitingRecoveryKey is DERIVED, not stored - the operator's ruling that
the state should be fixed rather than remembered, applied to this field too.

t.Enabled is load-bearing in that predicate and was MISSING in the first
draft. The existing TestOffsiteDeclare_DisabledTargetIsNotStranded caught it,
not review: a customer who switched off-site off is not awaiting anything.
Now pinned from the new predicate's own side as well.

Tests: Scenario A (no key written; transport still written; apply holds and
stages nothing), Scenario B (first-time box still mints), idempotency, the
nil-settings fail-safe, and the Scenario E carve-out.

RED-PROOFS, each with the mutation confirmed present in the file first:
  A) guard block deleted   -> both Scenario A tests FAIL with
     "R-241 REGRESSION: apply minted a repository password over the sealed
     package"; Scenario B still passes (the mutation is specific)
  B) guard over-widened (hub-package conjunct dropped) -> Scenario B FAILS
     with a first-time box unable to start; Scenario A still passes

Green: go build, go vet, go test ./... all pass; controller_gates all OK.
This commit is contained in:
authentik Default Admin
2026-08-07 11:25:58 +02:00
parent c6b69d888e
commit 763de3a025
3 changed files with 325 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
controller/internal/backup/offbox.go
+125 -4
View File
@@ -381,10 +381,46 @@ func (m *Manager) offboxKeyPath() string { return filepath.Join(m.offboxDir()
func (m *Manager) offboxPwPath() string { return filepath.Join(m.offboxDir(), "repo_password") }
func (m *Manager) offboxKnownHosts() string { return filepath.Join(m.offboxDir(), "known_hosts") }
// ErrOffboxSealedPackageHeld is the R-241 mint refusal: this box has no repository password and the
// hub is holding a sealed recovery package for it, so minting one would write a key the package does
// not cover — orphaning the very history the customer's recovery code protects.
//
// It is a SENTINEL, not a failure. `ApplyOffsiteTarget` catches it and still configures the transport
// (SSH key, known_hosts, host/user/path), because the transport is not the problem and having it is
// what lets the recovery screen bring the tier up the moment the key arrives (R-219). What it does
// NOT do is let the tier come up under a key nobody escrowed.
var ErrOffboxSealedPackageHeld = fmt.Errorf("offbox: the hub holds a sealed recovery package for this box — not minting a repository password over it")
// ErrOffboxSealedPackageHeld reports whether err is the mint refusal (errors.Is-friendly for callers
// that wrap it).
func IsOffboxSealedPackageHeld(err error) bool { return errors.Is(err, ErrOffboxSealedPackageHeld) }
// WriteOffboxSecrets persists the SSH private key + (auto-generated if empty) repo password + the pinned
// known-host line as 0600/0644 files in the data dir. The key is provided out-of-band by the operator
// (UI), never logged. Returns the repo password so the caller need not read the file. Idempotent: an empty
// sshKey/knownHosts leaves the existing file untouched (a re-save of just the target shouldn't wipe keys).
// (UI), never logged. Idempotent: an empty sshKey/knownHosts leaves the existing file untouched (a
// re-save of just the target shouldn't wipe keys).
//
// ⚠ R-241 (v0.206.0) — IT NO LONGER MINTS OVER A SEALED PACKAGE, AND THAT IS THE WHOLE FIX.
//
// Until now the auto-generate branch consulted **one** input: does the file exist. Not the settings,
// not the hub's ACK — nothing about whether anything already depended on a different key. Its two
// neighbours in this very file, `OffsiteRecoveryOffer` (:1412) and `needsOffsiteCredential` (:1377),
// BOTH consult `GetHubEscrowIdentityPresent()`. The same fact was available on three paths and used
// on two.
//
// WHAT THAT COST, measured on the final walk (2026-08-06/07, SPIKE-r241-recovery-offer-2026-08-07):
// a rebuilt box's credential self-heal reached here at 03:18:06Z and minted `9b4a9a9d…` over a hub
// package sealing `30ef574f…`. The recovery screen then looked, found a key present and no orphan
// recorded, and correctly said there was nothing to recover. **The screen was telling the truth; the
// lie happened thirty minutes earlier, here.** And the flag was not merely available at that moment —
// it was the PRECONDITION of the chain that reached this function: the credential retry only runs
// while `needsOffsiteCredential` is true, which requires this exact flag, and the venue logged it at
// 02:48:03Z, six ticks before the mint.
//
// THE GUARD IS DELIBERATELY NARROW — see the Scenario B test. It fires ONLY when a package is held AND
// no password exists. A box the hub holds nothing for mints exactly as before, which is every
// first-time box in the fleet; widening this to "never mint" would leave a new customer unable to
// start, waiting for a package that will never exist.
func (m *Manager) WriteOffboxSecrets(sshKey, knownHosts string) error {
if err := os.MkdirAll(m.offboxDir(), 0o700); err != nil {
return fmt.Errorf("offbox dir: %w", err)
@@ -407,8 +443,15 @@ func (m *Manager) WriteOffboxSecrets(sshKey, knownHosts string) error {
return fmt.Errorf("offbox known_hosts: %w", err)
}
}
// Auto-generate the repo password once (0600), never log it.
// Auto-generate the repo password once (0600), never log it — UNLESS the hub is holding a sealed
// package for us (R-241). The transport files above are already written and that is deliberate.
if _, err := os.Stat(m.offboxPwPath()); os.IsNotExist(err) {
if m.sealedPackageHeld() {
m.logger.Printf("[WARN] [offbox] NOT minting a repository password: the hub holds a sealed recovery package for this box, " +
"and a fresh key would orphan the history that package protects (R-241). The transport is configured; " +
"the tier stays down until the customer's recovery code places the escrowed key.")
return ErrOffboxSealedPackageHeld
}
pw, gerr := generateOffboxPassword()
if gerr != nil {
return gerr
@@ -420,6 +463,37 @@ func (m *Manager) WriteOffboxSecrets(sshKey, knownHosts string) error {
return nil
}
// sealedPackageHeld reports the ACK-cached fact that the hub is holding a sealed recovery package for
// this box. It is the SAME call `OffsiteRecoveryOffer` and `needsOffsiteCredential` already make —
// deliberately, so the three paths can never disagree about it. A nil settings store reads as "no
// package": the mint guard must never block a box whose settings could not be read, because that
// would turn a transient read failure into a tier that never comes up.
func (m *Manager) sealedPackageHeld() bool {
return m.settings != nil && m.settings.GetHubEscrowIdentityPresent()
}
// OffboxAwaitingRecoveryKey reports the R-241 holding state: a transport target exists, but no
// repository password does, because the hub holds a sealed package and the mint was refused.
//
// DERIVED, NOT STORED, and that is the §2.1 ruling applied to this field too: a stored flag would be a
// second copy of a fact the three inputs already carry, and a second copy is a thing that can drift.
// The moment a recovery places the escrowed key, this goes false on its own with nothing to clear.
//
// ⚠ `t.Enabled` IS LOAD-BEARING, and it was missing in the first draft — caught by the existing
// TestOffsiteDeclare_DisabledTargetIsNotStranded rather than by review. A customer who switched
// off-site OFF is not awaiting anything, and a box that declares a holding state for a tier nobody
// asked for is the R-215 shape (a screen about data the customer did not ask to protect). This is the
// SAME Scenario-E carve-out `needsOffsiteCredential` makes two functions above; the two must agree,
// and now do.
func (m *Manager) OffboxAwaitingRecoveryKey() bool {
t := m.settings.GetOffboxTarget()
if t == nil || !t.Enabled || !m.sealedPackageHeld() {
return false
}
_, hasPw := m.OffboxRepoPasswordHash()
return !hasPw
}
// generateOffboxPassword returns a 256-bit hex repo password.
func generateOffboxPassword() (string, error) {
b := make([]byte, 32)
@@ -489,8 +563,19 @@ var offboxRepoPwPattern = regexp.MustCompile(`^[0-9a-fA-F]{64}$`)
// the repo password to the agent for escrow — the SAME fork-4 enable path a manual config takes. `stage` is
// the agent escrow-stage push (nil skips it, e.g. when the agent is unreachable — the run gate still holds).
func (m *Manager) ApplyOffsiteTarget(ctx context.Context, tgt *settings.OffboxTarget, sshKeyPEM, knownHosts string, stage func(ctx context.Context, pw string) error) error {
// R-241: the mint refusal is a HOLDING state, not a failure. The transport files were written
// before the refusal, so we still record the target — `OffboxConfigured()` stays false because the
// password file is absent, which is what keeps runs gated, and the recovery screen can bring the
// tier up the instant the escrowed key is placed (R-219's synchronous tier-up).
//
// Returning the error here instead would leave `needsOffsiteCredential` true forever, so the hub
// would re-stage a credential the box had already consumed, on every cycle, for ever.
awaitingKey := false
if err := m.WriteOffboxSecrets(sshKeyPEM, knownHosts); err != nil {
return fmt.Errorf("apply offsite secrets: %w", err)
if !IsOffboxSealedPackageHeld(err) {
return fmt.Errorf("apply offsite secrets: %w", err)
}
awaitingKey = true
}
// Re-apply (v0.109.1 live finding): the bridge rebuilds the target from the descriptor, but the
// EXISTING target's custody + runtime status must carry over — EscrowState tracks the REPO PASSWORD
@@ -514,6 +599,14 @@ func (m *Manager) ApplyOffsiteTarget(ctx context.Context, tgt *settings.OffboxTa
if err := m.settings.SetOffboxTarget(tgt); err != nil {
return fmt.Errorf("apply offsite target: %w", err)
}
// R-241: nothing to stage — there is no repository password, by design. Say so once, plainly, and
// return without touching the escrow. `PushOffboxPasswordForEscrow` would fail on the absent file
// anyway; naming the situation beats a misleading "agent unreachable?" warning.
if awaitingKey {
m.logger.Printf("[INFO] [offbox] apply-offsite: transport configured for %s@%s:%s, tier HELD awaiting the escrowed key "+
"(the hub holds a sealed package; no key was minted — R-241)", tgt.User, tgt.Host, tgt.RepoPath)
return nil
}
if stage != nil {
// Best-effort: the offbox is configured + pending regardless. A stage-push failure (agent momentarily
// unreachable) is logged, not fatal — the escrow can be (re-)staged later (operator ceremony / re-enable).
@@ -1331,6 +1424,27 @@ type OffboxReportStatus struct {
// That was the last of the four manual interventions the 2026-08-04 drill needed.
const OffsiteStateNeedsCredential = "needs_credential"
// OffsiteStateAwaitingRecoveryKey (v0.206.0, R-241) is the declared HOLDING state: the transport is
// configured, but no repository password exists because the hub holds a sealed package and minting
// one would orphan the history it protects. The box is not stranded (it has its credential) and not
// healthy (it cannot run) — it is waiting for a person with a recovery code.
//
// WHY IT IS INERT TO EVERY EXISTING HUB READER, established from their code rather than assumed —
// the same discipline `OffsiteStateNeedsCredential`'s own note applies:
//
// - `offsiteheal` acts on EXACTLY ONE string, `needs_credential` ("Everything else … is a no-op"),
// so it will not re-stage a credential this box already has;
// - `monitor.OffsiteChecker.isStale` returns false unless `Enabled && EscrowState == "escrowed"`,
// and this object carries Enabled=false;
// - `monitor/offsite_delivery.go` keys on the delivery shape, which is `applied` here (the secret
// WAS consumed), and that branch is skipped;
// - an unknown `state` string is ignored by encoding/json on an older hub.
//
// So this needs NO hub change to be safe. It does mean a held box raises no alarm — which is R-243,
// filed and deliberately not widened here; the difference from R-241 is that this state is now
// VISIBLE to the customer instead of silent.
const OffsiteStateAwaitingRecoveryKey = "awaiting_recovery_key"
// needsOffsiteCredential is the stranded-rebuild predicate. BOTH facts are required and neither is
// sufficient on its own — this is the whole correctness of the feature:
//
@@ -1438,6 +1552,13 @@ func (m *Manager) OffsiteRecoveryOffer() bool {
// never emitted a disabled object before.
func (m *Manager) OffboxReportStatus() *OffboxReportStatus {
t := m.settings.GetOffboxTarget()
// R-241: the HOLDING state is declared before the enabled/disabled split, because a held target IS
// enabled — the customer wants off-site backups; what is missing is the key. Reported with
// Enabled=false so every existing hub reader treats it exactly as the stranded declaration (see
// OffsiteStateAwaitingRecoveryKey), while the string names the difference for anything that looks.
if m.OffboxAwaitingRecoveryKey() {
return &OffboxReportStatus{Enabled: false, State: OffsiteStateAwaitingRecoveryKey, EscrowState: t.EscrowState}
}
if t == nil || !t.Enabled {
if m.needsOffsiteCredential(t) {
return &OffboxReportStatus{Enabled: false, State: OffsiteStateNeedsCredential}