89712563a0
gates / gates (push) Successful in 10s
The retrieval clause rendered unconditionally on every page and is false on a reachable state - the same screen where the orphan card says we cannot tell. The condition is a fingerprint PINNED at the decision, not a comparison against the current key. The obvious proxy asks about the wrong key: the set-aside copies were written under an older key the box no longer has, so on a twice-rebuilt box the proxy promises about copies nothing can open. Demonstrated - under the proxy, the replaced-package and legacy cases both flip back to promising. The pin is a recorded assumption and says so: nothing on the box records which key wrote those copies. Empty is not a match. A countdown started before this carries no pin and takes the cautious branch, not a backfill. A sweep of all 36 templates found a fourth instance (backups page, same condition applied) and a fifth (the confirmation screen, correctly left alone - true at the moment of the decision). New retrieval_promise_gate registers each claim with a reason rather than banning a verb: a string ban failed twice, and the honest replacement copy contains the stem.
300 lines
15 KiB
Go
300 lines
15 KiB
Go
package backup
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
"time"
|
|
|
|
"gitea.dooplex.hu/admin/felhom-controller/internal/settings"
|
|
)
|
|
|
|
// ABANDONMENT — deciding to give up the old off-site history is a finishable thing (R-241, v0.206.0).
|
|
//
|
|
// THE PROBLEM THIS SOLVES. `resetOrphanedRepo` renamed the remote store aside and touched neither the
|
|
// escrow nor the key, so the hub went on holding a sealed package for a key the box no longer used.
|
|
// Shape (c) compares those two, finds them different, and offers recovery — correctly, and for ever.
|
|
// A customer who has already said "I do not want the old data" would be asked again at every login.
|
|
//
|
|
// THE OPERATOR'S RULING (2026-08-07) is that the answer is NOT a "they decided" flag. A flag would
|
|
// leave the box in a state that is genuinely wrong (the hub holding a package for a key nobody uses)
|
|
// and paper over it. Instead the decision starts a **14-day countdown**, at the end of which the
|
|
// set-aside store and the sealed package that protects it are removed TOGETHER — after which there is
|
|
// nothing left to compare and nothing left to ask about. **Fix the state, do not remember that it is
|
|
// wrong.**
|
|
//
|
|
// THE GRACE IS REAL, NOT DECORATIVE. The recovery offer stays reachable for the whole window; that is
|
|
// the change-of-mind path (Scenario G). A grace period during which recovery is impossible would be
|
|
// theatre.
|
|
|
|
// abandonGraceDays is the countdown the operator set. Reminders fire at 5, 3 and 1 days (see
|
|
// AbandonRemindAtDays) — visible, reversible, and running out in public.
|
|
const abandonGraceDays = 14
|
|
|
|
// AbandonGraceDays is the exported grace, for the customer-facing copy. The confirmation screen must
|
|
// state the SAME number the countdown uses — a literal typed into prose is how a promise drifts away
|
|
// from the code that keeps it.
|
|
const AbandonGraceDays = abandonGraceDays
|
|
|
|
// AbandonRemindAtDays are the remaining-day marks at which the abandoning box reminds the customer.
|
|
// Descending, so the surface can pick the first one that has been reached.
|
|
var AbandonRemindAtDays = []int{5, 3, 1}
|
|
|
|
// abandonNow is the countdown's clock seam. Tests inject; nil → time.Now. It exists so the terminal
|
|
// step can be driven deterministically — §7.4 forbids shortening a live timer to watch it fire,
|
|
// because that is how an irreversible step gets tested once and regretted once.
|
|
func (m *Manager) abandonNow() time.Time {
|
|
if m.offboxNow != nil {
|
|
return m.offboxNow()
|
|
}
|
|
return time.Now()
|
|
}
|
|
|
|
// SetOffboxClock injects the abandonment clock (tests only).
|
|
func (m *Manager) SetOffboxClock(fn func() time.Time) { m.offboxNow = fn }
|
|
|
|
// startAbandonCountdown records the decision and the date the terminal step will run. Called by
|
|
// resetOrphanedRepo AFTER the move-aside has succeeded — a countdown started before the store has
|
|
// actually moved would count down to deleting a path that does not exist.
|
|
func (m *Manager) startAbandonCountdown(setAsidePath string) {
|
|
now := m.abandonNow().UTC()
|
|
due := now.AddDate(0, 0, abandonGraceDays)
|
|
// R-302: pin the hub's escrow key fingerprint HERE, at the decision — the one moment it is a fact
|
|
// rather than something inferred later from an adjacent value. From now on the banner asks exactly
|
|
// one question, "is the hub still holding that same package?", instead of guessing which key is
|
|
// which. Written once and never refreshed: a field re-read at render answers a different question
|
|
// and would silently restore the defect this replaces.
|
|
pinned := ""
|
|
if m.settings != nil {
|
|
pinned, _ = m.settings.GetHubEscrowKeySHA256()
|
|
}
|
|
if err := m.settings.UpdateOffboxStatus(func(o *settings.OffboxTarget) {
|
|
o.AbandonStartedAt = now.Format(time.RFC3339)
|
|
o.AbandonAt = due.Format(time.RFC3339)
|
|
o.AbandonRepoPath = setAsidePath
|
|
o.AbandonPurgeRequested = false
|
|
o.AbandonPinnedEscrowKeySHA256 = pinned
|
|
}); err != nil {
|
|
m.logger.Printf("[WARN] [offbox] could not record the abandonment countdown: %v", err)
|
|
return
|
|
}
|
|
m.logger.Printf("[INFO] [offbox] abandonment countdown started: the set-aside history at %s and the hub's sealed package "+
|
|
"are removed together on %s (%d days). The recovery screen stays reachable until then.",
|
|
setAsidePath, due.Format("2006-01-02"), abandonGraceDays)
|
|
}
|
|
|
|
// AbandonState is the surface's read model. Zero value = nothing in progress.
|
|
type AbandonState struct {
|
|
Active bool // a countdown is running
|
|
StartedAt time.Time //
|
|
DueAt time.Time // when the terminal step runs
|
|
DaysLeft int // ceiling, so "0 days left" only ever means "today"
|
|
RepoPath string // the set-aside store awaiting deletion
|
|
PurgeRequested bool // the store is gone; awaiting the hub to drop the sealed package
|
|
// RetrievalStillOffered (R-302) — may the banner still say the set-aside copies can be retrieved
|
|
// with the recovery code? TRUE only while the hub is holding the SAME sealed package it held when
|
|
// the customer decided. Derived here, once, so the banner and anything else asking cannot disagree.
|
|
//
|
|
// FALSE covers: the package was replaced after the decision (a fresh escrow ceremony — the act that
|
|
// cost both demo boxes their history); the hub reports an empty hash (a legacy package sealing no
|
|
// repository password); and a countdown started before R-302, which carries no pin. All three are
|
|
// "we cannot see that this is still true", and all three must read as such rather than as a promise.
|
|
RetrievalStillOffered bool
|
|
}
|
|
|
|
// AbandonStatus reports the countdown for the UI and the report. It never mutates.
|
|
func (m *Manager) AbandonStatus() AbandonState {
|
|
t := m.settings.GetOffboxTarget()
|
|
if t == nil {
|
|
return AbandonState{}
|
|
}
|
|
st := AbandonState{RepoPath: t.AbandonRepoPath, PurgeRequested: t.AbandonPurgeRequested}
|
|
if t.AbandonAt == "" {
|
|
return st
|
|
}
|
|
due, err := time.Parse(time.RFC3339, t.AbandonAt)
|
|
if err != nil {
|
|
// A malformed stamp must not silently mean "never due" — that would strand the store for ever
|
|
// with a countdown the customer can see and nothing behind it.
|
|
m.logger.Printf("[WARN] [offbox] abandonment due-date is unparseable (%q) — treating the countdown as NOT running: %v", t.AbandonAt, err)
|
|
return st
|
|
}
|
|
st.Active, st.DueAt = true, due
|
|
if s, serr := time.Parse(time.RFC3339, t.AbandonStartedAt); serr == nil {
|
|
st.StartedAt = s
|
|
}
|
|
// R-302: the pinned fingerprint vs what the hub reports NOW. Both must be non-empty and equal.
|
|
// Empty on either side is "we could not see", never "they match" — the settings comment on
|
|
// HubEscrowKeySHA256 establishes that the hub sends "" for a package sealing no repo password.
|
|
if cur, _ := m.settings.GetHubEscrowKeySHA256(); cur != "" &&
|
|
t.AbandonPinnedEscrowKeySHA256 != "" && cur == t.AbandonPinnedEscrowKeySHA256 {
|
|
st.RetrievalStillOffered = true
|
|
}
|
|
// Ceiling: a countdown with 30 minutes left says "1 day", never "0". Zero is reserved for due.
|
|
remaining := due.Sub(m.abandonNow())
|
|
if remaining <= 0 {
|
|
st.DaysLeft = 0
|
|
} else {
|
|
st.DaysLeft = int((remaining + 24*time.Hour - time.Nanosecond) / (24 * time.Hour))
|
|
}
|
|
return st
|
|
}
|
|
|
|
// CancelAbandon stops a running countdown — the change-of-mind path (Scenario G). Called when a
|
|
// recovery succeeds: the customer has their code after all, and the history they were about to give
|
|
// up is exactly what the code opens.
|
|
//
|
|
// It clears the schedule but KEEPS AbandonRepoPath, so the set-aside store remains nameable on the
|
|
// backups page. Nothing has been deleted at this point by construction — the terminal step is the
|
|
// only thing that deletes, and it has not run.
|
|
func (m *Manager) CancelAbandon(reason string) {
|
|
t := m.settings.GetOffboxTarget()
|
|
if t == nil || (t.AbandonAt == "" && !t.AbandonPurgeRequested) {
|
|
return // nothing running — silent, so a healthy recovery does not log about a countdown
|
|
}
|
|
if err := m.settings.UpdateOffboxStatus(func(o *settings.OffboxTarget) {
|
|
o.AbandonStartedAt, o.AbandonAt = "", ""
|
|
o.AbandonPurgeRequested = false
|
|
}); err != nil {
|
|
m.logger.Printf("[WARN] [offbox] could not cancel the abandonment countdown: %v", err)
|
|
return
|
|
}
|
|
m.logger.Printf("[INFO] [offbox] abandonment countdown CANCELLED (%s) — the set-aside history at %s is kept and nothing was deleted", reason, t.AbandonRepoPath)
|
|
}
|
|
|
|
// AbandonSweep is the daily terminal step. It is the ONLY thing in the product that deletes a
|
|
// customer's off-site history, and it does so on a date the customer was shown.
|
|
//
|
|
// ⚠ IT REMOVES BOTH HALVES OR NEITHER — Scenario F. The set-aside store and the sealed package that
|
|
// protects it are the two halves of one thing; removing only the store leaves the hub holding a
|
|
// package for a key that opens nothing, and removing only the package leaves ciphertext nobody can
|
|
// ever decrypt. Either is a state that asks a question nobody can answer.
|
|
//
|
|
// The two halves cannot be made atomic across two machines, so this is a two-phase commit with the
|
|
// STORE FIRST and a durable marker: delete the remote store, record AbandonPurgeRequested, and keep
|
|
// declaring it in the report until the hub's ACK stops reporting a superseded package. A crash
|
|
// between the two leaves the marker set and the next sweep re-declares — it never leaves the pair
|
|
// half-removed and silent.
|
|
//
|
|
// Returns (deleted, err). deleted=false with err=nil is the normal "nothing due" case.
|
|
func (m *Manager) AbandonSweep(ctx context.Context) (bool, error) {
|
|
st := m.AbandonStatus()
|
|
// Phase 2 outstanding: the store is gone, the hub has not confirmed. Re-declare and wait.
|
|
if st.PurgeRequested {
|
|
m.logger.Printf("[DEBUG] [offbox] abandonment: the set-aside store is deleted; awaiting the hub to drop the sealed package")
|
|
return false, nil
|
|
}
|
|
if !st.Active || st.DueAt.After(m.abandonNow()) {
|
|
return false, nil // not due — quiet by construction on every healthy box
|
|
}
|
|
t := m.settings.GetOffboxTarget()
|
|
if t == nil || t.AbandonRepoPath == "" {
|
|
m.logger.Printf("[WARN] [offbox] abandonment is due but no set-aside path is recorded — nothing deleted; clearing the countdown so it does not retry for ever")
|
|
m.CancelAbandon("no set-aside path recorded")
|
|
return false, fmt.Errorf("abandonment due with no recorded path")
|
|
}
|
|
port := t.Port
|
|
if port == 0 {
|
|
port = 22
|
|
}
|
|
m.logger.Printf("[WARN] [offbox] abandonment DUE — deleting the set-aside off-site history at %s (chosen by the customer on %s; this is irreversible)",
|
|
t.AbandonRepoPath, st.StartedAt.Format("2006-01-02"))
|
|
out, err := m.sshRunner()(ctx, t.Host, t.User, port, m.offboxKeyPath(), m.offboxKnownHosts(),
|
|
"rm -rf "+shellQuote(t.AbandonRepoPath))
|
|
if err != nil {
|
|
// NOT cleared: a transport failure must retry tomorrow, not silently abandon the abandonment.
|
|
m.logger.Printf("[ERROR] [offbox] abandonment: deleting the set-aside history failed — the countdown stays due and retries: %v: %s", err, truncate(out))
|
|
return false, fmt.Errorf("delete set-aside history: %w", err)
|
|
}
|
|
// Phase 1 done. Record it durably BEFORE anything else, so a crash here re-declares rather than
|
|
// forgetting that the store is already gone.
|
|
if uerr := m.settings.UpdateOffboxStatus(func(o *settings.OffboxTarget) {
|
|
o.AbandonPurgeRequested = true
|
|
o.AbandonAt = "" // the schedule has fired; the marker now drives the rest
|
|
}); uerr != nil {
|
|
m.logger.Printf("[ERROR] [offbox] abandonment: the store was deleted but the marker could not be saved — the hub's package may outlive it: %v", uerr)
|
|
return true, uerr
|
|
}
|
|
m.logger.Printf("[INFO] [offbox] abandonment: set-aside history deleted; requesting the hub to drop the sealed package that protected it")
|
|
if m.offboxOrphanEvent != nil {
|
|
m.offboxOrphanEvent("offbox_abandon_completed", t.AbandonRepoPath)
|
|
}
|
|
return true, nil
|
|
}
|
|
|
|
// ClearAbandonPurgeIfConfirmed closes the two-phase commit: once the hub's ACK stops reporting a
|
|
// superseded package, both halves are gone and the abandonment is finished. Called from the ACK path.
|
|
//
|
|
// This is what makes §2.1 work without a "they decided" flag: afterwards the hub holds a package for
|
|
// the key the box is actually using (or none at all), shape (c) has nothing to compare, and the
|
|
// recovery offer falls silent on its own — because the state is right, not because something is
|
|
// remembering that it once was not.
|
|
func (m *Manager) ClearAbandonPurgeIfConfirmed(supersededPresent bool) {
|
|
t := m.settings.GetOffboxTarget()
|
|
if t == nil || !t.AbandonPurgeRequested || supersededPresent {
|
|
return
|
|
}
|
|
if err := m.settings.UpdateOffboxStatus(func(o *settings.OffboxTarget) {
|
|
o.AbandonPurgeRequested = false
|
|
o.AbandonRepoPath = ""
|
|
o.AbandonStartedAt = ""
|
|
o.OrphanedRenamedTo = ""
|
|
}); err != nil {
|
|
m.logger.Printf("[WARN] [offbox] could not close out the abandonment: %v", err)
|
|
return
|
|
}
|
|
m.logger.Printf("[INFO] [offbox] abandonment COMPLETE — the set-aside history and the sealed package that protected it are both gone; nothing further to ask about")
|
|
}
|
|
|
|
// ── OPERATOR CONTROL (§7.5) ─────────────────────────────────────────────────────────────────────
|
|
//
|
|
// The automatic 30-day abandonment is deliberately NOT built (see R-245). What IS built is the path
|
|
// that actually happens: **the customer gets in touch.** Someone who cannot find their recovery code
|
|
// rings support, and support needs something to press — either "give them longer" or "stop it".
|
|
//
|
|
// Both live on the controller CLI rather than in the customer UI, deliberately: extending a deletion
|
|
// the customer asked for is an operator judgement, not a self-service button, and a customer who
|
|
// wants it stopped already has the self-service route — they recover with their code, which cancels
|
|
// it (Scenario G).
|
|
|
|
// ExtendAbandon pushes the terminal step out by `days` from NOW. Returns the new due date.
|
|
//
|
|
// It refuses when no countdown is running: extending nothing would print a reassuring date for a
|
|
// deletion that was never scheduled, which is the kind of comfort this project keeps removing.
|
|
func (m *Manager) ExtendAbandon(days int) (time.Time, error) {
|
|
if days <= 0 {
|
|
return time.Time{}, fmt.Errorf("the extension must be a positive number of days")
|
|
}
|
|
st := m.AbandonStatus()
|
|
if !st.Active {
|
|
if st.PurgeRequested {
|
|
return time.Time{}, fmt.Errorf("too late: the set-aside history has already been deleted and only the sealed package is still being removed")
|
|
}
|
|
return time.Time{}, fmt.Errorf("no abandonment countdown is running on this box — nothing to extend")
|
|
}
|
|
due := m.abandonNow().UTC().AddDate(0, 0, days)
|
|
if err := m.settings.UpdateOffboxStatus(func(o *settings.OffboxTarget) {
|
|
o.AbandonAt = due.Format(time.RFC3339)
|
|
}); err != nil {
|
|
return time.Time{}, fmt.Errorf("record the extension: %w", err)
|
|
}
|
|
m.logger.Printf("[WARN] [offbox] abandonment EXTENDED by an operator: the set-aside history at %s is now deleted on %s (was %s)",
|
|
st.RepoPath, due.Format("2006-01-02"), st.DueAt.Format("2006-01-02"))
|
|
return due, nil
|
|
}
|
|
|
|
// StopAbandon cancels the countdown outright — the operator's version of Scenario G, for the
|
|
// customer who telephoned instead of finding their code. The set-aside history is kept and nothing
|
|
// is deleted; it is `CancelAbandon` with an operator's reason and a refusal when nothing is running,
|
|
// so an operator never gets a silent no-op they might read as success.
|
|
func (m *Manager) StopAbandon() error {
|
|
st := m.AbandonStatus()
|
|
if !st.Active {
|
|
if st.PurgeRequested {
|
|
return fmt.Errorf("too late: the set-aside history has already been deleted")
|
|
}
|
|
return fmt.Errorf("no abandonment countdown is running on this box — nothing to stop")
|
|
}
|
|
m.CancelAbandon("stopped by an operator")
|
|
return nil
|
|
}
|