v0.289.0: off-site key cannot delete — append-only rclone transport, box sends only its public key (hub registrar), retention only inside a hub window behind the fake-snapshot guard (decisions 68-69, R-820, R-822)
gates / gates (push) Successful in 27s

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
2026-10-03 17:01:36 +02:00
parent 09453325d1
commit 55bb6c3d32
15 changed files with 1039 additions and 386 deletions
+30
View File
@@ -1,3 +1,33 @@
## v0.289.0 — the off-site key cannot delete: append-only transport, no password on the box, retention only inside a hub window (decisions 68–69, R-820, R-822) (2026-10-03)
**MinAgent: 0.131.0** (unchanged). **Needs hub v0.127.0** (the key registrar and the window endpoints; an older hub
answers 404 to `register-key` and the apply-bridge keeps retrying). No new household string.
- **The box never fetches the Storage Box password any more.** The apply-bridge (`offsiteapply`) sends its PUBLIC key
to the hub's registrar (`HubRegistrar`: `register-key`, `confirm-key`), which pins it in the sub-account to
`command="rclone serve restic --stdio --append-only <repo>",restrict`; the box then PROVES the key reaches the pinned
server (`PinnedProber`: exit 0 and rclone's output — an unpinned key gets the restricted shell, exit 8, measured)
before configuring anything. `HTTPConsumer`, `SSHCopyIDInstaller` and the `sshpass` path are gone. A box upgraded
from v0.288.0 re-applies once (`descriptorHash` gains `|pinned-v1`) and re-registers its EXISTING key.
- **Transport.** A hub-provisioned target (`Transport: "rclone-pinned"`, set by `ApplyOffsiteTarget`) uses restic's
`rclone:` backend with `-o rclone.program="ssh -p 23 … -i <key> … rclone"` — restic 0.14.0 suffices, rclone is NOT in
the image (it runs at the provider). An sftp-written repository reads, extends, restores and `check --read-data`s
through it (measured on the provider). The household's own SFTP NAS target is unchanged.
- **Retention leaves the box on the pinned tier (decision 68).** Both `forget --prune` sites (after a run; over quota)
go through `offsiteWindowRetention`: no window → nothing deleted; inside a hub-granted window the **fake-snapshot
guard (R-822)** refuses on any future-dated snapshot, any snapshot newer than the hub's bound, or a plan that would
remove a snapshot younger than 8 days; otherwise the OLDEST `max_remove` planned snapshots are forgotten by id and the
window is closed with counts. **Disagreement recorded:** the brief said abort when the plan exceeds a week's
removal; the first window after the interim legitimately does, so the box caps and takes the oldest instead.
- **Move-aside is the hub's** (`POST /offsite/move-aside`); **abandonment is deferred to the operator** on the pinned
tier — nothing deleted, `offbox_abandon_deferred` (operator-only) — because the key cannot delete (R-823).
- Transport failures of the `rclone:` backend (`error talking HTTP to rclone`) classify as transport.
- Bug found by the existing suite and fixed before release: the NAS move-aside path assigned a shadowed `newPath`.
- Tests: `offsiteapply` rewritten (fresh, upgraded, already-pinned, registered-but-not-pinned, wrong fingerprint,
host-key mismatch, idempotent, confirm failure); `offbox_window_test.go` (the lab's 13 future fakes refused; recent
removal refused; honest plan oldest-first capped; pinned run never forgets without a window; pinned move-aside asks the
hub; pinned abandonment defers). Red-proofs: `felhom.eu/documentation/audits/offsite-lock-build-2026-10-03/partC/`.
## v0.288.0 — remove tells the truth about the household's files (decision 67, R-800) (2026-10-02) ## v0.288.0 — remove tells the truth about the household's files (decision 67, R-800) (2026-10-02)
**MinAgent: 0.131.0** (unchanged). Needs hub v0.123.0 (unchanged). New string: `layout.userdata_marad` (both languages). **MinAgent: 0.131.0** (unchanged). Needs hub v0.123.0 (unchanged). New string: `layout.userdata_marad` (both languages).
+2
View File
@@ -245,6 +245,8 @@
| `report.SetPendingLogTails` + `buildLogTailsSection` | controller/internal/report/logtail.go | ACK `log_tail_requests` → next report `log_tails` | THE pull-based ACK-flag pattern (hub asks, controller pushes next cycle) — copy for any new hub→box request | Consume-once drain at BuildReport; failed push re-arms from the hub's still-pending request; NEVER add a hub→controller push channel | | `report.SetPendingLogTails` + `buildLogTailsSection` | controller/internal/report/logtail.go | ACK `log_tail_requests` → next report `log_tails` | THE pull-based ACK-flag pattern (hub asks, controller pushes next cycle) — copy for any new hub→box request | Consume-once drain at BuildReport; failed push re-arms from the hub's still-pending request; NEVER add a hub→controller push channel |
| `metrics.FetchContainerLogTail` | controller/internal/metrics/logscanner.go | `(name, tailLines) (string, error)` | Raw per-container `docker logs --tail=N` | 15s timeout; caller caps/redacts (capTailLines) | | `metrics.FetchContainerLogTail` | controller/internal/metrics/logscanner.go | `(name, tailLines) (string, error)` | Raw per-container `docker logs --tail=N` | 15s timeout; caller caps/redacts (capTailLines) |
| `ConfigRefresher.Reconcile` | controller/internal/report/config_refresh.go | `(ackVersion int)` | Pull-based config refresh | Re-pulls controller.yaml (re-merging local_api), then graceful self-restart; first-run = baseline, no restart | | `ConfigRefresher.Reconcile` | controller/internal/report/config_refresh.go | `(ackVersion int)` | Pull-based config refresh | Re-pulls controller.yaml (re-merging local_api), then graceful self-restart; first-run = baseline, no restart |
| `offsiteapply.HubRegistrar` / `HubWindowClient` / `PinnedProber` (v0.289.0, decisions 68–69) | controller/internal/offsiteapply/seams.go | `Register(ctx,pub)(fp,err)` · `Confirm` · `MoveAside` · `Open/Close` window · `Probe(ctx,host,user,port,kh,privPEM) bool` | EVERY off-site key install, the hub move-aside, the clean-up window | **The box never handles the sub-account password** — there is no consume path any more. `PinnedProber` is a POSITIVE observable (exit 0 + rclone output); "authenticates" is NOT enough — an unpinned key authenticates and can delete. |
| `Manager.offsiteWindowRetention` + `offsiteGuard` (v0.289.0, R-822) | controller/internal/backup/offbox_window.go | `(ctx, base, env, why)` · pure `(all, plan, now, newestAllowed, max) (ids, refuse)` | THE retention step for both callers (after a run, over quota) | Pinned tier: no window → nothing deleted; the guard runs BEFORE any forget; forget is by explicit ids, oldest first. NAS tier: the old SP-2 policy, unchanged. |
| `offsiteapply.SettleProvider` / `SettleFunc` / `Bridge.AwaitSettle` / `ReconcileWhenSettled` (R-71a, v0.162.0) | controller/internal/offsiteapply/offsiteapply.go + seams.go | `SettleState() (version, floor string, updateRunning, floorKnown bool)` | THE settle-gate: defers the offsite one-time-password consume past a managed day-0 floor-update (the F10 race). Wire the `SettleFunc` adapter over `updater.GetFloor()`/`IsUpdateRunning()` — **the updater's knowledge is the ONE floor source; never fetch the floor a second way**. Gate ONLY the bridge goroutine, and only when an updater exists (nil `Settle` = reconcile immediately). Bounds `settlePoll`/`settleFloorSubBound`/`settleOverallBound`; the floor is in-memory (report-ACK-derived, ~5–10 s), NOT persisted → unknown until the first ACK on any restart. Inject `Now`/`Sleep` in tests (no real sleeps). B′: at/above-floor GOes on the first poll, zero wait. Do NOT touch the consume/persist order or the 404 contract — ordering only | | `offsiteapply.SettleProvider` / `SettleFunc` / `Bridge.AwaitSettle` / `ReconcileWhenSettled` (R-71a, v0.162.0) | controller/internal/offsiteapply/offsiteapply.go + seams.go | `SettleState() (version, floor string, updateRunning, floorKnown bool)` | THE settle-gate: defers the offsite one-time-password consume past a managed day-0 floor-update (the F10 race). Wire the `SettleFunc` adapter over `updater.GetFloor()`/`IsUpdateRunning()` — **the updater's knowledge is the ONE floor source; never fetch the floor a second way**. Gate ONLY the bridge goroutine, and only when an updater exists (nil `Settle` = reconcile immediately). Bounds `settlePoll`/`settleFloorSubBound`/`settleOverallBound`; the floor is in-memory (report-ACK-derived, ~5–10 s), NOT persisted → unknown until the first ACK on any restart. Inject `Now`/`Sleep` in tests (no real sleeps). B′: at/above-floor GOes on the first poll, zero wait. Do NOT touch the consume/persist order or the 404 contract — ordering only |
| `bootstrap.MaybeIngest` / `RefreshConfig` | controller/internal/bootstrap/bootstrap.go | bootstrap.json → controller.yaml | Day-0 + refresh | Overwrites controller.yaml, NEVER settings.json | | `bootstrap.MaybeIngest` / `RefreshConfig` | controller/internal/bootstrap/bootstrap.go | bootstrap.json → controller.yaml | Day-0 + refresh | Overwrites controller.yaml, NEVER settings.json |
| `api.GracefulSelfRestart` | controller/internal/api/selfrestart.go | `(logger)` | Controller self-restart | Detached exit; bootstrap unit re-runs the image | | `api.GracefulSelfRestart` | controller/internal/api/selfrestart.go | `(logger)` | Controller self-restart | Detached exit; bootstrap unit re-runs the image |
+6
View File
@@ -155,6 +155,12 @@ backups, monitoring and notifications. All Proxmox/disk operations are delegated
action block right; used by the dashboard installed-apps list, the Távoli mentés toggle list action block right; used by the dashboard installed-apps list, the Távoli mentés toggle list
and the Visszaállítás restore-to-verify/.fab lists; the backups-apps expander header is and the Visszaállítás restore-to-verify/.fab lists; the backups-apps expander header is
ALIGNED to the same grammar (own markup — it carries the toggle). Protected infra stacks ALIGNED to the same grammar (own markup — it carries the toggle). Protected infra stacks
**The off-site key cannot delete (v0.289.0, decisions 68–69):** the hub-provisioned tier is reached through an
APPEND-ONLY key the hub's registrar pins in the Storage Box sub-account (`rclone serve restic --stdio --append-only`);
the box sends only its public key (`offsiteapply.HubRegistrar`) and never sees the sub-account password. Transport is
restic `rclone:` over ssh port 23 (`settings.OffboxTarget.Transport = "rclone-pinned"`); the household's own SFTP NAS
is unchanged. Retention runs only inside a hub-opened weekly window, behind the fake-snapshot guard
(`backup/offbox_window.go`); the orphan move-aside is the hub's; a due abandonment is deferred to the operator.
**Apps go off-site by themselves (v0.283.0, decision 50):** a fresh install on a box whose customer has off-site **Apps go off-site by themselves (v0.283.0, decision 50):** a fresh install on a box whose customer has off-site
switches the app's off-site copy ON (`settings.DefaultOffboxOnForNewApp`, deploy-done hook); an earlier choice is switches the app's off-site copy ON (`settings.DefaultOffboxOnForNewApp`, deploy-done hook); an earlier choice is
kept. Older apps: one press on both backup pages (`/backup/offbox/enable-all`, „Nem most" dismisses). The size kept. Older apps: one press on both backup pages (`/backup/offbox/enable-all`, „Nem most" dismisses). The size
+24 -3
View File
@@ -772,14 +772,30 @@ func main() {
// reconcile between placing a recovered key and reading the repository (R-219). nil when off-site // reconcile between placing a recovered key and reading the repository (R-219). nil when off-site
// is not configured for this customer, which the web seam treats as "skip". // is not configured for this customer, which the web seam treats as "skip".
var offsiteBridge *offsiteapply.Bridge var offsiteBridge *offsiteapply.Bridge
offsiteRegistrar := offsiteapply.HubRegistrar{HubURL: cfg.Hub.URL, CustomerID: cfg.Customer.ID, APIKey: cfg.Hub.APIKey}
if backupMgr != nil && cfg.Hub.URL != "" && cfg.Hub.APIKey != "" {
// The orphan move-aside is the HUB's now (decision 69): the box's pinned key reaches only the
// append-only rclone server and cannot rename a directory.
backupMgr.SetOffsiteMoveAside(offsiteRegistrar.MoveAside)
// Decision 68: retention on the append-only tier happens only inside a hub-opened window.
backupMgr.SetOffsiteWindowClient(offsiteapply.HubWindowClient{Registrar: offsiteRegistrar})
}
if backupMgr != nil && cfg.Offsite.Enabled && cfg.Hub.URL != "" && cfg.Hub.APIKey != "" { if backupMgr != nil && cfg.Offsite.Enabled && cfg.Hub.URL != "" && cfg.Hub.APIKey != "" {
bridge := &offsiteapply.Bridge{ bridge := &offsiteapply.Bridge{
Cfg: cfg, Cfg: cfg,
Consumer: offsiteapply.HTTPConsumer{HubURL: cfg.Hub.URL, CustomerID: cfg.Customer.ID, APIKey: cfg.Hub.APIKey}, // Decision 69 (v0.289.0): the box sends its PUBLIC key to the hub's registrar, which pins it
// append-only; the box never receives the sub-account password.
Registrar: offsiteRegistrar,
Scanner: offsiteapply.KeyscanScanner{}, Scanner: offsiteapply.KeyscanScanner{},
KeyGen: offsiteapply.ED25519KeyGen{}, KeyGen: offsiteapply.ED25519KeyGen{},
Installer: offsiteapply.SSHCopyIDInstaller{}, Prober: offsiteapply.PinnedProber{},
Prober: offsiteapply.SFTPKeyAuthProber{KeyPath: filepath.Join(cfg.Paths.DataDir, "offbox", "ssh_key")}, Existing: func() string {
b, err := os.ReadFile(filepath.Join(cfg.Paths.DataDir, "offbox", "ssh_key"))
if err != nil {
return ""
}
return string(b)
},
Enabler: offsiteapply.EnablerFunc(func(ctx context.Context, host, user string, port int, repoPath, priv, kh string, quotaGB int) error { Enabler: offsiteapply.EnablerFunc(func(ctx context.Context, host, user string, port int, repoPath, priv, kh string, quotaGB int) error {
tgt := &settings.OffboxTarget{Enabled: true, Host: host, User: user, Port: port, RepoPath: repoPath, Schedule: "daily", QuotaGB: quotaGB} tgt := &settings.OffboxTarget{Enabled: true, Host: host, User: user, Port: port, RepoPath: repoPath, Schedule: "daily", QuotaGB: quotaGB}
stage := func(ctx context.Context, pw string) error { stage := func(ctx context.Context, pw string) error {
@@ -1306,6 +1322,11 @@ func main() {
case "offbox_repo_reset": case "offbox_repo_reset":
notifier.PushEvent("offbox_repo_reset", "info", notifier.PushEvent("offbox_repo_reset", "info",
"A távoli mentési tároló visszaállítva: a régi előzmény félretéve (nem törölve), és egy üres, új tároló jött létre a mostani kulccsal.", map[string]string{"renamed_to": renamedTo}) "A távoli mentési tároló visszaállítva: a régi előzmény félretéve (nem törölve), és egy üres, új tároló jött létre a mostani kulccsal.", map[string]string{"renamed_to": renamedTo})
case "offbox_abandon_deferred":
// v0.289.0 (decision 69): the box's off-site key cannot delete, so the customer-chosen
// deletion of the set-aside history is the operator's (R-823). Operator-only on the hub.
notifier.PushEvent("offbox_abandon_deferred", "warning",
"A félretett régi távoli mentések törlése esedékes, de a doboz távoli kulcsa csak hozzáadni tud (69. döntés): semmi nem törlődött. A félretett másolatot az üzemeltető távolítja el.", map[string]string{"set_aside_path": renamedTo})
case "offbox_abandon_completed": case "offbox_abandon_completed":
// R-241: the ONLY event in the product that reports a customer's off-site history // R-241: the ONLY event in the product that reports a customer's off-site history
// being deleted. It is fired after the deletion, not before — the operator wants to // being deleted. It is fired after the deletion, not before — the operator wants to
+4
View File
@@ -113,6 +113,10 @@ type Manager struct {
// offboxSSH (v0.142.0) is the raw-ssh exec seam for the orphaned-repo move-aside (restic has no // offboxSSH (v0.142.0) is the raw-ssh exec seam for the orphaned-repo move-aside (restic has no
// rename); tests inject a fake. Nil → the real ssh invocation (defaultOffboxSSH). // rename); tests inject a fake. Nil → the real ssh invocation (defaultOffboxSSH).
offboxSSH func(ctx context.Context, host, user string, port int, keyPath, knownHosts, remoteCmd string) ([]byte, error) offboxSSH func(ctx context.Context, host, user string, port int, keyPath, knownHosts, remoteCmd string) ([]byte, error)
// offsiteMoveAside (v0.289.0, decision 69) asks the hub to set the repository aside.
offsiteMoveAside func(ctx context.Context) (string, error)
// offsiteWindow (v0.289.0, decision 68) asks the hub for a clean-up window. nil → no box retention.
offsiteWindow OffsiteWindowClient
// offboxSizer (3a) — the mandatory-set byte estimator for the pre-push enlargement gate, overridable // offboxSizer (3a) — the mandatory-set byte estimator for the pre-push enlargement gate, overridable
// in tests so the gate is unit-testable without a real du. Nil → the real dirSizeBytes (du -sb). // in tests so the gate is unit-testable without a real du. Nil → the real dirSizeBytes (du -sb).
+79 -46
View File
@@ -69,6 +69,11 @@ func (m *Manager) SetOffboxOrphanEvent(fn func(eventType, renamedTo string)) {
} }
// SetOffboxSSH overrides the raw-ssh exec used for the orphaned-repo move-aside (tests). // SetOffboxSSH overrides the raw-ssh exec used for the orphaned-repo move-aside (tests).
// SetOffsiteMoveAside wires the hub's move-aside (decision 69): the orphan reset asks the hub.
func (m *Manager) SetOffsiteMoveAside(fn func(ctx context.Context) (string, error)) {
m.offsiteMoveAside = fn
}
func (m *Manager) SetOffboxSSH(fn func(ctx context.Context, host, user string, port int, keyPath, knownHosts, remoteCmd string) ([]byte, error)) { func (m *Manager) SetOffboxSSH(fn func(ctx context.Context, host, user string, port int, keyPath, knownHosts, remoteCmd string) ([]byte, error)) {
m.offboxSSH = fn m.offboxSSH = fn
} }
@@ -208,7 +213,8 @@ func ClassifyOffsiteFailure(err error) OffsiteFailureClass {
strings.Contains(s, "no route to host"), strings.Contains(s, "i/o timeout"), strings.Contains(s, "no route to host"), strings.Contains(s, "i/o timeout"),
strings.Contains(s, "timed out"), strings.Contains(s, "permission denied"), strings.Contains(s, "timed out"), strings.Contains(s, "permission denied"),
strings.Contains(s, "host key"), strings.Contains(s, "handshake"), strings.Contains(s, "host key"), strings.Contains(s, "handshake"),
strings.Contains(s, "could not resolve"), strings.Contains(s, "network is unreachable"): strings.Contains(s, "could not resolve"), strings.Contains(s, "network is unreachable"),
strings.Contains(s, "error talking http to rclone"): // measured: the rclone: backend's dead-ssh shape
return OffsiteFailTransport return OffsiteFailTransport
default: default:
return OffsiteFailUnknown return OffsiteFailUnknown
@@ -311,27 +317,44 @@ func (m *Manager) resetOrphanedRepo(ctx context.Context, base, env []string, rea
if t == nil { if t == nil {
return fmt.Errorf("no offsite target configured") return fmt.Errorf("no offsite target configured")
} }
port := t.Port var newPath string
if port == 0 { if t.Pinned() {
port = 22 // v0.289.0 (decision 69): the HUB sets the repository aside — the box's append-only key reaches only
} // the pinned rclone server and cannot rename a directory. The hub renames, never deletes, and picks a
// Choose a move-aside name that never overwrites an earlier orphaned copy (edge rule: -2, -3). // name that never overwrites an earlier set-aside copy (`<repo>.orphaned-<date>[-n]`).
date := time.Now().UTC().Format("20060102") if m.offsiteMoveAside == nil {
base1 := t.RepoPath + ".orphaned-" + date return fmt.Errorf("offbox move-aside: the hub's key registrar is not configured on this box")
newPath := base1 }
for i := 2; i <= 20; i++ { m.logger.Printf("[WARN] [offbox] resetting orphaned repo (%s): asking the hub to set %s aside, then re-init", reason, t.RepoPath)
// `test -e <p>` returns non-zero (exit 1) when absent — that is the name we want. A transport np, err := m.offsiteMoveAside(ctx)
// error also lands here; we then just try the mv and let it fail loudly rather than loop. if err != nil {
out, err := m.sshRunner()(ctx, t.Host, t.User, port, m.offboxKeyPath(), m.offboxKnownHosts(), "test -e "+shellQuote(newPath)) return fmt.Errorf("offbox move-aside failed: %w", err)
if err != nil && !strings.Contains(strings.ToLower(string(out)), "denied") { }
break // absent (test -e exit 1) → free name m.logger.Printf("[INFO] [offbox] the hub set the orphaned repo aside: %s -> %s (nothing deleted)", t.RepoPath, newPath)
newPath = np
} else {
port := t.Port
if port == 0 {
port = 22
}
// Choose a move-aside name that never overwrites an earlier orphaned copy (edge rule: -2, -3).
date := time.Now().UTC().Format("20060102")
base1 := t.RepoPath + ".orphaned-" + date
newPath = base1
for i := 2; i <= 20; i++ {
// `test -e <p>` returns non-zero (exit 1) when absent — that is the name we want. A transport
// error also lands here; we then just try the mv and let it fail loudly rather than loop.
out, err := m.sshRunner()(ctx, t.Host, t.User, port, m.offboxKeyPath(), m.offboxKnownHosts(), "test -e "+shellQuote(newPath))
if err != nil && !strings.Contains(strings.ToLower(string(out)), "denied") {
break // absent (test -e exit 1) → free name
}
newPath = fmt.Sprintf("%s-%d", base1, i)
}
m.logger.Printf("[WARN] [offbox] resetting orphaned repo (%s): move-aside %s -> %s, then re-init", reason, t.RepoPath, newPath)
if out, err := m.sshRunner()(ctx, t.Host, t.User, port, m.offboxKeyPath(), m.offboxKnownHosts(),
fmt.Sprintf("mv %s %s", shellQuote(t.RepoPath), shellQuote(newPath))); err != nil {
return fmt.Errorf("offbox move-aside failed: %w: %s", err, truncate(out))
} }
newPath = fmt.Sprintf("%s-%d", base1, i)
}
m.logger.Printf("[WARN] [offbox] resetting orphaned repo (%s): move-aside %s -> %s, then re-init", reason, t.RepoPath, newPath)
if out, err := m.sshRunner()(ctx, t.Host, t.User, port, m.offboxKeyPath(), m.offboxKnownHosts(),
fmt.Sprintf("mv %s %s", shellQuote(t.RepoPath), shellQuote(newPath))); err != nil {
return fmt.Errorf("offbox move-aside failed: %w: %s", err, truncate(out))
} }
// Fresh init under the current passphrase. // Fresh init under the current passphrase.
ictx, icancel := context.WithTimeout(ctx, offboxProbeTimeout) ictx, icancel := context.WithTimeout(ctx, offboxProbeTimeout)
@@ -628,6 +651,10 @@ func (m *Manager) ApplyOffsiteTarget(ctx context.Context, tgt *settings.OffboxTa
if tgt.EscrowState != "escrowed" { if tgt.EscrowState != "escrowed" {
tgt.EscrowState = "pending" tgt.EscrowState = "pending"
} }
// Decision 69 (v0.289.0): a HUB-provisioned target is reached only through the append-only key the
// registrar pinned — the apply-bridge proved that before calling here. The household's own NAS
// (offboxConfigHandler) never comes through this function and stays Transport "".
tgt.Transport = settings.TransportRclonePinned
if err := m.settings.SetOffboxTarget(tgt); err != nil { if err := m.settings.SetOffboxTarget(tgt); err != nil {
return fmt.Errorf("apply offsite target: %w", err) return fmt.Errorf("apply offsite target: %w", err)
} }
@@ -733,13 +760,32 @@ func (m *Manager) OffboxEscrowState() string {
return t.EscrowState return t.EscrowState
} }
// offboxBaseArgs builds the restic global args (repo + sftp.args carrying the ConnectTimeout, key, pinned // offboxBaseArgs builds the restic global args (the rclone: repo + rclone.program carrying the
// known_hosts, port) and the env (RESTIC_PASSWORD_FILE). The ConnectTimeout is MANDATORY (fail-fast). // ConnectTimeout, key, pinned known_hosts, port) and the env (RESTIC_PASSWORD_FILE).
func (m *Manager) offboxBaseArgs(t *settings.OffboxTarget) ([]string, []string) { func (m *Manager) offboxBaseArgs(t *settings.OffboxTarget) ([]string, []string) {
port := t.Port port := t.Port
if port == 0 { if port == 0 {
port = 22 port = 22
} }
// v0.289.0 (decision 69, R-820): the `rclone:` backend over the box's APPEND-ONLY key. The hub's key
// registrar pins this key in the sub-account's authorized_keys to
// `command="rclone serve restic --stdio --append-only <repo>",restrict`, so whatever we ask for, the
// provider runs that server: backups, restores, `check` and lock removal work; every delete is
// refused (403) — measured on the provider 2026-10-03 (audits/offsite-append-only-2026-10-03/,
// audits/offsite-lock-build-2026-10-03/partA: an sftp-written repo reads, extends, restores and
// `check --read-data`s through it). restic 0.14.0 is enough; rclone is NOT needed in the image (it
// runs on the provider). The program is split on whitespace by restic; the trailing "rclone" is the
// remote command an UNPINNED key would run (and the pin ignores). ConnectTimeout stays MANDATORY.
if t.Pinned() {
if t.Port == 0 {
port = 23 // the provider accepts OpenSSH keys on 23 only (measured: 22 refuses them)
}
rcloneProg := fmt.Sprintf("ssh -p %d -oBatchMode=yes -oConnectTimeout=%d -oStrictHostKeyChecking=yes -oUserKnownHostsFile=%s -oIdentitiesOnly=yes -i %s %s@%s rclone",
port, offboxConnectTimeoutSec, m.offboxKnownHosts(), m.offboxKeyPath(), t.User, t.Host)
return []string{"-r", "rclone:" + t.RepoPath, "-o", "rclone.program=" + rcloneProg},
[]string{"RESTIC_PASSWORD_FILE=" + m.offboxPwPath()}
}
// The household's own SFTP NAS target (Transport ""): unchanged since v0.142.0.
// restic's sftp backend connects via the `-o sftp.command` SSH invocation (the portable form across // restic's sftp backend connects via the `-o sftp.command` SSH invocation (the portable form across
// restic versions — `sftp.args` is not recognized by restic 0.14). The ConnectTimeout makes a dead NAS // restic versions — `sftp.args` is not recognized by restic 0.14). The ConnectTimeout makes a dead NAS
// fail in ~N s (the load-bearing spike Q8 knob); StrictHostKeyChecking + a pinned known_hosts avoid // fail in ~N s (the load-bearing spike Q8 knob); StrictHostKeyChecking + a pinned known_hosts avoid
@@ -1412,17 +1458,12 @@ func (m *Manager) runOffboxInternal(ctx context.Context, apps, base, env []strin
if firstErr != nil { if firstErr != nil {
return res, firstErr return res, firstErr
} }
// Retention: keep a sane window, prune the rest. SP-2: `--group-by host,tags` so an app's OLD // Retention (decision 68, v0.289.0): the box's key is append-only, so it cannot prune on its own. It
// unit-only-shape snapshots share a group with its NEW enlarged shape (same <stack> tag) and age // asks the hub for a clean-up window; only inside one, and only past the fake-snapshot guard (R-822),
// out naturally — the default host,paths grouping would strand old-shape snapshots in their own // does it forget/prune. No window → nothing is deleted (the interim, option 3). SP-2's
// permanently-retained group. prune takes an EXCLUSIVE lock (the C2 stale-lock step) → resticStep. // `--group-by host,tags` policy is unchanged — it lives in offsiteWindowRetention now.
m.offboxProgress.setPhase(OffboxPhaseRetention) m.offboxProgress.setPhase(OffboxPhaseRetention)
fctx, cancel := context.WithTimeout(ctx, offboxBackupTimeout) m.offsiteWindowRetention(ctx, base, env, "after-run")
defer cancel()
if out, ferr := m.resticStep(fctx, env, base, "prune", "forget", "--group-by", "host,tags", "--keep-daily", "7", "--keep-weekly", "4", "--keep-monthly", "6", "--prune"); ferr != nil {
// A prune failure is non-fatal to the backup itself (data is safe) — log, don't fail the run.
m.logger.Printf("[WARN] [offbox] forget --prune failed (backups are safe): %v: %s", ferr, truncate(out))
}
return res, nil return res, nil
} }
@@ -1778,24 +1819,16 @@ func OffboxQuotaPercent(t *settings.OffboxTarget) int {
return pct return pct
} }
// offboxPruneOnly runs ONLY the retention/prune step (the over-quota path: new backups are refused but // offboxPruneOnly is the over-quota path: new backups are refused. Pruning is the only way back under the
// pruning must stay available — it is the only way back under the quota). Repo-ensure first so a fresh // quota — and since decision 68 it happens ONLY inside a hub-opened window, behind the R-822 guard. When
// target still fails loudly; errors are non-fatal (same as the regular run's prune). // the hub grants none, nothing is deleted and the household's quota sentence stays (no delete attempt —
// a refused delete costs ~48 s of restic retries per file and writes an index, measured).
func (m *Manager) offboxPruneOnly(ctx context.Context, base, env []string) { func (m *Manager) offboxPruneOnly(ctx context.Context, base, env []string) {
if rerr := m.ensureOffboxRepo(ctx, base, env); rerr != nil { if rerr := m.ensureOffboxRepo(ctx, base, env); rerr != nil {
m.logger.Printf("[WARN] [offbox] over-quota prune: repo unreachable: %v", rerr) m.logger.Printf("[WARN] [offbox] over-quota prune: repo unreachable: %v", rerr)
return return
} }
fctx, cancel := context.WithTimeout(ctx, offboxBackupTimeout) m.offsiteWindowRetention(ctx, base, env, "over-quota")
defer cancel()
// SP-2: `--group-by host,tags` (mirrors runOffboxInternal's forget) so old unit-only-shape snapshots
// age out with the enlarged shape instead of stranding in a permanently-retained host,paths group.
fargs := append(append([]string{}, base...), "forget", "--group-by", "host,tags", "--keep-daily", "7", "--keep-weekly", "4", "--keep-monthly", "6", "--prune")
if out, ferr := m.runner()(fctx, env, fargs...); ferr != nil {
m.logger.Printf("[WARN] [offbox] over-quota prune failed: %v: %s", ferr, truncate(out))
} else {
m.logger.Printf("[INFO] [offbox] over-quota: prune executed (new backups refused until under quota)")
}
} }
// offboxRecordStats reads the snapshot count (best-effort) for the UI; also fills repo size when stats works. // offboxRecordStats reads the snapshot count (best-effort) for the UI; also fills repo size when stats works.
@@ -192,6 +192,19 @@ func (m *Manager) AbandonSweep(ctx context.Context) (bool, error) {
m.CancelAbandon("no set-aside path recorded") m.CancelAbandon("no set-aside path recorded")
return false, fmt.Errorf("abandonment due with no recorded path") return false, fmt.Errorf("abandonment due with no recorded path")
} }
if t.Pinned() {
// Decision 69 (v0.289.0): the box's off-site key is append-only and cannot delete — by design,
// so that a broken-into box cannot erase history. The set-aside copy STAYS; the operator removes
// it (R-823). The schedule is closed so the sweep stops; nothing was deleted.
if uerr := m.settings.UpdateOffboxStatus(func(o *settings.OffboxTarget) { o.AbandonAt = "" }); uerr != nil {
m.logger.Printf("[WARN] [offbox] abandonment: could not close the schedule: %v", uerr)
}
m.logger.Printf("[WARN] [offbox] abandonment DUE for %s, but the off-site key is append-only (decision 69) — NOTHING deleted; the set-aside copy stays until the operator removes it", t.AbandonRepoPath)
if m.offboxOrphanEvent != nil {
m.offboxOrphanEvent("offbox_abandon_deferred", t.AbandonRepoPath)
}
return false, nil
}
port := t.Port port := t.Port
if port == 0 { if port == 0 {
port = 22 port = 22
+222
View File
@@ -0,0 +1,222 @@
package backup
import (
"context"
"encoding/json"
"fmt"
"sort"
"strings"
"time"
)
// ── Decision 68 (v0.289.0): the box prunes its own repository ONLY inside a hub-opened window ──────────
//
// The hub-provisioned tier's key is append-only (decision 69): every delete is refused by the provider.
// To keep retention working, the box ASKS the hub for a clean-up window after its run. The hub grants
// one at most weekly (or on an operator's one-shot grant) by PREPENDING a deleting line for the box's own
// key — OpenSSH uses the first matching line, measured on the provider — and closes it when the box
// reports, or after 20 minutes on its own.
//
// THE FAKE-SNAPSHOT GUARD (R-822) runs BEFORE any forget, inside the window. Measured in the lab: 13
// future-dated empty snapshots added through an add-only key make the box's own policy select EVERY real
// snapshot for removal. So, refuse when:
// - any snapshot is dated in the future (beyond offsiteGuardSkew), or after the hub's newest-allowed
// bound (the moment the window opened, plus the same skew);
// - the plan would remove a snapshot younger than offsiteGuardMinAge — the honest policy
// (--keep-daily 7) never removes the newest snapshot of any of the last 7 days, while a poisoning
// shape does exactly that.
// And bound the damage of anything the guard cannot see: at most MaxRemove (the hub's number) snapshots
// per window, OLDEST first. DISAGREEMENT RECORDED (R-96 rule 4): the brief asked to ABORT when the plan
// exceeds a week's removal; the first window after the interim legitimately exceeds it (weeks of
// unpruned history), so an abort would never prune at all. Capping and taking the oldest gives the
// same bound on loss per window and still converges.
//
// The NAS tier (Transport "") is unchanged: the household's own disk, pruned by the box as before.
//
// Pinned by TestOffsiteGuard_* (offbox_window_test.go), including the lab's 13-fake shape.
const (
offsiteGuardSkew = time.Hour
offsiteGuardMinAge = 8 * 24 * time.Hour
)
// OffsiteWindow is the hub's answer to "may I prune now?".
type OffsiteWindow struct {
Granted bool
ID int64
NewestAllowed time.Time
MaxRemove int
Reason string // why not granted (logged)
}
// OffsiteWindowResult is what the box reports when it is done (the hub closes the window on it).
type OffsiteWindowResult struct {
ID int64 `json:"window_id"`
CountBefore int `json:"count_before"`
CountAfter int `json:"count_after"`
Removed int `json:"removed"`
Outcome string `json:"outcome"` // pruned | nothing | guard-refused | error
Reason string `json:"reason,omitempty"`
}
// OffsiteWindowClient is the hub side (offsiteapply.HubWindowClient in production).
type OffsiteWindowClient interface {
Open(ctx context.Context, countBefore int) (OffsiteWindow, error)
Close(ctx context.Context, r OffsiteWindowResult) error
}
// SetOffsiteWindowClient wires the hub's window (decision 68). nil → no box-side retention on the pinned tier.
func (m *Manager) SetOffsiteWindowClient(c OffsiteWindowClient) { m.offsiteWindow = c }
// retentionPolicy is the ruled policy, unchanged since SP-2 (`--group-by host,tags`).
var retentionPolicy = []string{"--group-by", "host,tags", "--keep-daily", "7", "--keep-weekly", "4", "--keep-monthly", "6"}
type guardSnap struct {
ID string `json:"id"`
ShortID string `json:"short_id"`
Time time.Time `json:"time"`
}
// offsiteGuard is the PURE decision: from all snapshots and the policy's remove-plan, either the ids to
// remove (oldest first, at most maxRemove) or a refusal reason.
func offsiteGuard(all, plan []guardSnap, now, newestAllowed time.Time, maxRemove int) ([]string, string) {
for _, s := range all {
if s.Time.After(now.Add(offsiteGuardSkew)) {
return nil, fmt.Sprintf("snapshot %s is dated in the future (%s)", s.ShortID, s.Time.UTC().Format(time.RFC3339))
}
if !newestAllowed.IsZero() && s.Time.After(newestAllowed.Add(offsiteGuardSkew)) {
return nil, fmt.Sprintf("snapshot %s (%s) is newer than the hub allows (%s)", s.ShortID, s.Time.UTC().Format(time.RFC3339), newestAllowed.UTC().Format(time.RFC3339))
}
}
for _, s := range plan {
if now.Sub(s.Time) < offsiteGuardMinAge {
return nil, fmt.Sprintf("the policy would remove snapshot %s from %s — younger than %d days, which honest retention never does",
s.ShortID, s.Time.UTC().Format(time.RFC3339), int(offsiteGuardMinAge.Hours()/24))
}
}
sorted := append([]guardSnap{}, plan...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Time.Before(sorted[j].Time) })
if maxRemove >= 0 && len(sorted) > maxRemove {
sorted = sorted[:maxRemove]
}
ids := make([]string, 0, len(sorted))
for _, s := range sorted {
ids = append(ids, s.ID)
}
return ids, ""
}
func (m *Manager) listGuardSnaps(ctx context.Context, base, env []string) ([]guardSnap, error) {
sctx, cancel := context.WithTimeout(ctx, offboxProbeTimeout)
defer cancel()
out, err := m.runner()(sctx, env, append(append([]string{}, base...), "snapshots", "--json")...)
if err != nil {
return nil, fmt.Errorf("list snapshots: %w: %s", err, truncate(out))
}
var snaps []guardSnap
if err := json.Unmarshal(out, &snaps); err != nil {
return nil, fmt.Errorf("parse snapshots: %w", err)
}
return snaps, nil
}
func (m *Manager) planRemovals(ctx context.Context, base, env []string) ([]guardSnap, error) {
pctx, cancel := context.WithTimeout(ctx, offboxProbeTimeout)
defer cancel()
args := append(append(append([]string{}, base...), "forget"), retentionPolicy...)
args = append(args, "--dry-run", "--json")
out, err := m.runner()(pctx, env, args...)
if err != nil {
return nil, fmt.Errorf("forget --dry-run: %w: %s", err, truncate(out))
}
// restic 0.14.0 prints the JSON array on stdout; the runner may combine stderr — take the array.
js := string(out)
if i := strings.Index(js, "["); i > 0 {
js = js[i:]
}
var groups []struct {
Remove []guardSnap `json:"remove"`
}
if err := json.Unmarshal([]byte(strings.TrimSpace(js)), &groups); err != nil {
return nil, fmt.Errorf("parse forget plan: %w", err)
}
var plan []guardSnap
for _, g := range groups {
plan = append(plan, g.Remove...)
}
return plan, nil
}
// offsiteWindowRetention is the ONE retention step for both callers (after a run, over quota).
func (m *Manager) offsiteWindowRetention(ctx context.Context, base, env []string, why string) {
t := m.settings.GetOffboxTarget()
if !t.Pinned() {
// The household's own SFTP NAS: the box prunes as it always did (SP-2 policy).
fctx, cancel := context.WithTimeout(ctx, offboxBackupTimeout)
defer cancel()
args := append(append([]string{"forget"}, retentionPolicy...), "--prune")
if out, ferr := m.resticStep(fctx, env, base, "prune", args...); ferr != nil {
m.logger.Printf("[WARN] [offbox] forget --prune failed (%s; backups are safe): %v: %s", why, ferr, truncate(out))
}
return
}
if m.offsiteWindow == nil {
m.logger.Printf("[INFO] [offbox] retention skipped (%s): the off-site key is append-only and no clean-up window client is wired — nothing deleted (decision 68)", why)
return
}
snaps, err := m.listGuardSnaps(ctx, base, env)
if err != nil {
m.logger.Printf("[WARN] [offbox] retention skipped (%s): %v", why, err)
return
}
w, err := m.offsiteWindow.Open(ctx, len(snaps))
if err != nil {
m.logger.Printf("[WARN] [offbox] retention skipped (%s): asking the hub for a window failed: %v", why, err)
return
}
if !w.Granted {
m.logger.Printf("[INFO] [offbox] retention skipped (%s): no clean-up window now (%s) — nothing deleted (decision 68)", why, w.Reason)
return
}
start := time.Now()
res := OffsiteWindowResult{ID: w.ID, CountBefore: len(snaps), CountAfter: len(snaps)}
defer func() {
cctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
if cerr := m.offsiteWindow.Close(cctx, res); cerr != nil {
m.logger.Printf("[WARN] [offbox] closing clean-up window %d with the hub failed (the hub closes it by itself in 20 min): %v", w.ID, cerr)
}
}()
plan, err := m.planRemovals(ctx, base, env)
if err != nil {
res.Outcome, res.Reason = "error", err.Error()
m.logger.Printf("[WARN] [offbox] clean-up window %d: %v", w.ID, err)
return
}
ids, refuse := offsiteGuard(snaps, plan, time.Now(), w.NewestAllowed, w.MaxRemove)
if refuse != "" {
res.Outcome, res.Reason = "guard-refused", refuse
m.logger.Printf("[ERROR] [offbox] clean-up window %d: the fake-snapshot guard REFUSED — nothing deleted: %s (R-822)", w.ID, refuse)
return
}
if len(ids) == 0 {
res.Outcome = "nothing"
m.logger.Printf("[INFO] [offbox] clean-up window %d: the policy removes nothing", w.ID)
return
}
fctx, cancel := context.WithTimeout(ctx, offboxBackupTimeout)
defer cancel()
args := append(append([]string{"forget"}, ids...), "--prune")
if out, ferr := m.resticStep(fctx, env, base, "prune", args...); ferr != nil {
res.Outcome, res.Reason = "error", truncate(out)
m.logger.Printf("[WARN] [offbox] clean-up window %d: forget --prune failed (backups are safe): %v: %s", w.ID, ferr, truncate(out))
} else {
res.Outcome = "pruned"
}
if after, lerr := m.listGuardSnaps(ctx, base, env); lerr == nil {
res.CountAfter = len(after)
}
res.Removed = res.CountBefore - res.CountAfter
m.logger.Printf("[INFO] [offbox] clean-up window %d (%s): %d of %d planned snapshot(s) removed, %d -> %d, in %s",
w.ID, why, res.Removed, len(plan), res.CountBefore, res.CountAfter, time.Since(start).Round(time.Second))
}
@@ -0,0 +1,237 @@
package backup
import (
"context"
"encoding/json"
"fmt"
"strings"
"testing"
"time"
"gitea.dooplex.hu/admin/felhom-controller/internal/settings"
)
// ── decision 68/69 (v0.289.0): the append-only tier ─────────────────────────────────────────────────
func pinTarget(t *testing.T, sett *settings.Settings) {
t.Helper()
if err := sett.UpdateOffboxStatus(func(o *settings.OffboxTarget) {
o.Transport = settings.TransportRclonePinned
o.Port = 23
o.Host, o.User, o.RepoPath = "u1-sub4.example", "u1-sub4", "/home/felhom-repo"
}); err != nil {
t.Fatal(err)
}
}
func snapJSON(s []guardSnap) []byte { b, _ := json.Marshal(s); return b }
func planJSON(remove []guardSnap) []byte {
b, _ := json.Marshal([]map[string]any{{"tags": []string{"app1"}, "remove": remove}})
return b
}
// windowRunner fakes restic for the retention step: snapshots, the dry-run plan, and records every
// non-dry-run forget (the only call that deletes).
type windowRunner struct {
snaps []guardSnap
plan []guardSnap
forgets [][]string
}
func (w *windowRunner) run(_ context.Context, _ []string, args ...string) ([]byte, error) {
switch {
case contains(args, "forget") && contains(args, "--dry-run"):
return planJSON(w.plan), nil
case contains(args, "forget"):
w.forgets = append(w.forgets, append([]string{}, args...))
return nil, nil
case contains(args, "snapshots"):
return snapJSON(w.snaps), nil
}
return nil, nil
}
type fakeWindow struct {
grant OffsiteWindow
opened int
closed []OffsiteWindowResult
}
func (f *fakeWindow) Open(context.Context, int) (OffsiteWindow, error) {
f.opened++
return f.grant, nil
}
func (f *fakeWindow) Close(_ context.Context, r OffsiteWindowResult) error {
f.closed = append(f.closed, r)
return nil
}
func snap(id string, at time.Time) guardSnap {
return guardSnap{ID: id + "-full", ShortID: id, Time: at}
}
// The pinned transport: rclone over port 23, the pinned key; never sftp.
func TestOffboxBaseArgs_PinnedUsesRcloneOnPort23(t *testing.T) {
m, sett := newOffboxManager(t)
pinTarget(t, sett)
sett.UpdateOffboxStatus(func(o *settings.OffboxTarget) { o.Port = 0 })
args, _ := m.offboxBaseArgs(sett.GetOffboxTarget())
j := strings.Join(args, " ")
if !strings.Contains(j, "-r rclone:/home/felhom-repo") || !strings.Contains(j, "rclone.program=ssh -p 23 ") || strings.Contains(j, "sftp") {
t.Fatalf("pinned args = %q", j)
}
if !argsContainTimeout(args) {
t.Fatal("ConnectTimeout lost on the pinned transport")
}
// The household's NAS stays SFTP.
m2, sett2 := newOffboxManager(t)
if j2 := strings.Join(func() []string { a, _ := m2.offboxBaseArgs(sett2.GetOffboxTarget()); return a }(), " "); !strings.Contains(j2, "sftp:") {
t.Fatalf("NAS args = %q", j2)
}
}
// THE CONSEQUENCE: on the pinned tier, a run with no window deletes NOTHING — no forget reaches restic.
// RED-PROOF: the pre-v0.289.0 retention ran `forget … --prune` unconditionally after every run.
func TestRetention_PinnedWithoutWindowDeletesNothing(t *testing.T) {
m, sett := newOffboxManager(t)
pinTarget(t, sett)
wr := &windowRunner{snaps: []guardSnap{snap("a", time.Now().Add(-40*24*time.Hour))}, plan: []guardSnap{snap("a", time.Now().Add(-40*24*time.Hour))}}
m.SetOffboxRunner(wr.run)
m.offsiteWindowRetention(context.Background(), nil, nil, "after-run") // no client wired
fw := &fakeWindow{grant: OffsiteWindow{Granted: false, Reason: "not due"}}
m.SetOffsiteWindowClient(fw)
m.offsiteWindowRetention(context.Background(), nil, nil, "after-run")
if len(wr.forgets) != 0 {
t.Fatalf("a forget ran without a window: %v", wr.forgets)
}
if fw.opened != 1 || len(fw.closed) != 0 {
t.Fatalf("opened=%d closed=%d", fw.opened, len(fw.closed))
}
}
// The full run path: RunOffboxBackup on a pinned target with no window never calls forget.
func TestRunOffboxBackup_PinnedNeverForgetsWithoutWindow(t *testing.T) {
m, sett := newOffboxManager(t)
pinTarget(t, sett)
var forgets int
m.SetOffboxRunner(func(ctx context.Context, env []string, args ...string) ([]byte, error) {
if contains(args, "forget") {
forgets++
}
rr := &recordingOffboxRunner{}
return rr.run(ctx, env, args...)
})
_ = m.RunOffboxBackup(context.Background())
if forgets != 0 {
t.Fatalf("the pinned run reached forget %d time(s)", forgets)
}
}
// R-822, the lab's shape: 13 future-dated fakes. The guard REFUSES and nothing is deleted; the hub
// is told why (window closed with outcome guard-refused).
func TestOffsiteGuard_LabThirteenFutureFakes_Refused(t *testing.T) {
m, sett := newOffboxManager(t)
pinTarget(t, sett)
now := time.Now()
real := []guardSnap{snap("r1", now.Add(-2*time.Hour)), snap("r2", now.Add(-26*time.Hour)), snap("r3", now.Add(-50*time.Hour))}
all := append([]guardSnap{}, real...)
for d := 1; d <= 7; d++ {
all = append(all, snap(fmt.Sprintf("f%d", d), time.Date(2027, 1, d, 3, 0, 0, 0, time.UTC)))
}
for mth := 2; mth <= 7; mth++ {
all = append(all, snap(fmt.Sprintf("m%d", mth), time.Date(2027, time.Month(mth), 15, 3, 0, 0, 0, time.UTC)))
}
wr := &windowRunner{snaps: all, plan: real} // the poisoned policy selects every REAL snapshot
m.SetOffboxRunner(wr.run)
fw := &fakeWindow{grant: OffsiteWindow{Granted: true, ID: 7, NewestAllowed: now, MaxRemove: 50}}
m.SetOffsiteWindowClient(fw)
m.offsiteWindowRetention(context.Background(), nil, nil, "after-run")
if len(wr.forgets) != 0 {
t.Fatalf("the guard let a poisoned plan delete: %v", wr.forgets)
}
if len(fw.closed) != 1 || fw.closed[0].Outcome != "guard-refused" || !strings.Contains(fw.closed[0].Reason, "future") {
t.Fatalf("window close = %+v", fw.closed)
}
}
// Past-dated fakes that make the policy drop a RECENT real snapshot: refused too.
func TestOffsiteGuard_RecentRemovalRefused(t *testing.T) {
now := time.Now()
all := []guardSnap{snap("old", now.Add(-60*24*time.Hour)), snap("recent", now.Add(-3*24*time.Hour))}
_, why := offsiteGuard(all, []guardSnap{snap("recent", now.Add(-3*24*time.Hour))}, now, now, 50)
if !strings.Contains(why, "younger than 8 days") {
t.Fatalf("why = %q", why)
}
_, why = offsiteGuard(all, nil, now, now.Add(-10*24*time.Hour), 50)
if !strings.Contains(why, "newer than the hub allows") {
t.Fatalf("newest-allowed bound not enforced: %q", why)
}
}
// Honest retention inside a window: the oldest first, at most MaxRemove, and the forget names ids.
func TestOffsiteGuard_HonestPlanPrunesOldestFirstCapped(t *testing.T) {
m, sett := newOffboxManager(t)
pinTarget(t, sett)
now := time.Now()
plan := []guardSnap{snap("c", now.Add(-20*24*time.Hour)), snap("a", now.Add(-90*24*time.Hour)), snap("b", now.Add(-60*24*time.Hour))}
all := append([]guardSnap{snap("keep", now.Add(-time.Hour))}, plan...)
wr := &windowRunner{snaps: all, plan: plan}
m.SetOffboxRunner(wr.run)
fw := &fakeWindow{grant: OffsiteWindow{Granted: true, ID: 9, NewestAllowed: now, MaxRemove: 2}}
m.SetOffsiteWindowClient(fw)
m.offsiteWindowRetention(context.Background(), nil, nil, "after-run")
if len(wr.forgets) != 1 {
t.Fatalf("forgets = %v", wr.forgets)
}
got := strings.Join(wr.forgets[0], " ")
if !strings.Contains(got, "forget a-full b-full --prune") || strings.Contains(got, "c-full") {
t.Fatalf("forget = %q (want the two OLDEST, capped)", got)
}
if len(fw.closed) != 1 || fw.closed[0].Outcome != "pruned" || fw.closed[0].ID != 9 {
t.Fatalf("close = %+v", fw.closed)
}
}
// The orphan reset on the pinned tier asks the HUB; no ssh `mv` from the box.
func TestResetOrphaned_PinnedAsksTheHub(t *testing.T) {
m, sett := newOffboxManager(t)
pinTarget(t, sett)
m.SetOffboxSSH(func(context.Context, string, string, int, string, string, string) ([]byte, error) {
t.Fatal("the box issued a raw ssh command on the pinned tier")
return nil, nil
})
called := 0
m.SetOffsiteMoveAside(func(context.Context) (string, error) { called++; return "/home/felhom-repo.orphaned-20261003", nil })
m.SetOffboxRunner(func(context.Context, []string, ...string) ([]byte, error) { return nil, nil })
if err := m.resetOrphanedRepo(context.Background(), nil, nil, "test"); err != nil {
t.Fatal(err)
}
if called != 1 || sett.GetOffboxTarget().OrphanedRenamedTo != "/home/felhom-repo.orphaned-20261003" {
t.Fatalf("hub move-aside called=%d recorded=%q", called, sett.GetOffboxTarget().OrphanedRenamedTo)
}
}
// Abandonment on the pinned tier: due → nothing deleted, the operator is told, the sweep goes quiet.
func TestAbandon_PinnedDefersToOperator(t *testing.T) {
m, sett := newOffboxManager(t)
pinTarget(t, sett)
sett.UpdateOffboxStatus(func(o *settings.OffboxTarget) {
o.AbandonRepoPath = "/home/felhom-repo.orphaned-20260901"
o.AbandonStartedAt = time.Now().Add(-20 * 24 * time.Hour).UTC().Format(time.RFC3339)
o.AbandonAt = time.Now().Add(-time.Hour).UTC().Format(time.RFC3339)
})
m.SetOffboxSSH(func(context.Context, string, string, int, string, string, string) ([]byte, error) {
t.Fatal("the box tried to delete on the pinned tier")
return nil, nil
})
var evs []string
m.SetOffboxOrphanEvent(func(e, _ string) { evs = append(evs, e) })
deleted, err := m.AbandonSweep(context.Background())
if deleted || err != nil || len(evs) != 1 || evs[0] != "offbox_abandon_deferred" {
t.Fatalf("deleted=%v err=%v events=%v", deleted, err, evs)
}
if again, _ := m.AbandonSweep(context.Background()); again {
t.Fatal("second sweep deleted")
}
}
@@ -1,9 +1,11 @@
// Package offsiteapply is the controller-side apply-bridge (SLICE 2): it turns the hub-served offsite // Package offsiteapply is the controller-side apply-bridge: it turns the hub-served offsite descriptor into
// descriptor + the one-time password into a working key-only offbox target. On config apply it consumes the // a working off-site target reached through an APPEND-ONLY key. It VERIFIES the Storage Box host key against
// one-time password, VERIFIES the box host key against the hub-captured fingerprint (no blind TOFU), pins it, // the hub-captured fingerprint (no blind TOFU), sends the box's PUBLIC key to the hub's key registrar, which
// installs the controller's own key, and configures the offbox target → EscrowState="pending" (the fork-4 // writes it into the sub-account pinned to `rclone serve restic --stdio --append-only <repo>` (decision 69,
// enable path). Idempotent (a descriptor hash marker prevents re-consuming a spent password) and fail-safe // controller v0.289.0), proves the key reaches the pinned server, and configures the offbox target →
// (any step fails → nothing persisted, retried next cycle; never a half-configured offbox). // EscrowState="pending" (the fork-4 enable path). The box NEVER receives the sub-account password any more —
// that password can rewrite authorized_keys and remove the pin (R-820). Idempotent and fail-safe (any step
// fails → nothing persisted, retried next cycle).
package offsiteapply package offsiteapply
import ( import (
@@ -22,9 +24,11 @@ import (
// The apply-bridge seams (tests inject fakes — no live SSH / hub calls in unit tests). // The apply-bridge seams (tests inject fakes — no live SSH / hub calls in unit tests).
type ( type (
// PasswordConsumer fetches the one-time transient password from the hub (single-use). // KeyRegistrar is the hub's key registrar (decision 69): install a PUBLIC key pinned append-only,
PasswordConsumer interface { // then confirm the one the box uses (the hub drops every other line).
Consume(ctx context.Context) (string, error) KeyRegistrar interface {
Register(ctx context.Context, pub string) (fingerprint string, err error)
Confirm(ctx context.Context, fingerprint string) error
} }
// HostKeyScanner returns the box's host-key fingerprint (SHA256:…) + the known_hosts line to pin. // HostKeyScanner returns the box's host-key fingerprint (SHA256:…) + the known_hosts line to pin.
HostKeyScanner interface { HostKeyScanner interface {
@@ -34,25 +38,18 @@ type (
KeyGenerator interface { KeyGenerator interface {
Generate() (privPEM, pubAuthorized string, err error) Generate() (privPEM, pubAuthorized string, err error)
} }
// KeyInstaller installs the pub line on the box using the one-time password, then verifies passwordless
// key auth with the private key. It MUST pin the VERIFIED knownHosts line (from the scan) on the
// connection — never blind-TOFU — so a MITM cannot swap the key between the scan and the install.
KeyInstaller interface {
Install(ctx context.Context, host, user string, port int, password, privPEM, pubAuthorized, knownHosts string) error
}
// OffboxEnabler configures the offbox target (key + known_hosts + target + soft quota) and goes // OffboxEnabler configures the offbox target (key + known_hosts + target + soft quota) and goes
// EscrowState="pending" (the fork-4 enable path). quotaGB=0 = no soft limit (dedicated boxes). // EscrowState="pending" (the fork-4 enable path). quotaGB=0 = no soft limit (dedicated boxes).
OffboxEnabler interface { OffboxEnabler interface {
ConfigureOffbox(ctx context.Context, host, user string, port int, repoPath, privPEM, knownHosts string, quotaGB int) error ConfigureOffbox(ctx context.Context, host, user string, port int, repoPath, privPEM, knownHosts string, quotaGB int) error
} }
// KeyAuthProber checks whether an ALREADY-INSTALLED key authenticates to the target (pinned to the // PinnedKeyProber proves a key reaches the PINNED append-only server (not merely that it
// freshly-scanned knownHosts). ok=true returns that key's PEM so the descriptor change is applied by // authenticates — an unpinned key authenticates too, and can delete).
// re-pinning + reconfiguring WITHOUT consuming a one-time password (key-auth-first — kills the PinnedKeyProber interface {
// stale-descriptor consume-404 loop and shrinks the re-issue blast radius to genuinely-fresh guests). Probe(ctx context.Context, host, user string, port int, knownHosts, privPEM string) bool
// ok=false (no key / auth refused) → the caller falls through to the full consume+install path.
KeyAuthProber interface {
Probe(ctx context.Context, host, user string, port int, knownHosts string) (privPEM string, ok bool)
} }
// ExistingKey returns the box's installed off-site private key, "" when none.
ExistingKey func() string
// SettleProvider reports the managed-update settle state so the bridge can DEFER consuming the // SettleProvider reports the managed-update settle state so the bridge can DEFER consuming the
// one-time password until any imminent managed floor-update has converged (R-71a — the structural // one-time password until any imminent managed floor-update has converged (R-71a — the structural
// fix for the F10 day-0 race). The failure it prevents: a fresh box boots below the operator floor, // fix for the F10 day-0 race). The failure it prevents: a fresh box boots below the operator floor,
@@ -106,13 +103,13 @@ func belowFloor(version, floor string) bool {
// Bridge reconciles the offsite descriptor into a configured offbox target. // Bridge reconciles the offsite descriptor into a configured offbox target.
type Bridge struct { type Bridge struct {
Cfg *config.Config Cfg *config.Config
Consumer PasswordConsumer Registrar KeyRegistrar
Scanner HostKeyScanner Scanner HostKeyScanner
KeyGen KeyGenerator KeyGen KeyGenerator
Installer KeyInstaller
Enabler OffboxEnabler Enabler OffboxEnabler
Prober KeyAuthProber // optional: key-auth-first (nil → always the full consume+install path) Prober PinnedKeyProber
MarkerPath string // where the applied-descriptor-hash is persisted (e.g. <dataDir>/offbox/applied_marker) Existing ExistingKey // nil → no existing key (always a fresh keypair)
MarkerPath string // where the applied-descriptor-hash is persisted (e.g. <dataDir>/offbox/applied_marker)
Logger *log.Logger Logger *log.Logger
// Settle gates the consume/install path behind managed-update convergence (R-71a). nil → no gate // Settle gates the consume/install path behind managed-update convergence (R-71a). nil → no gate
@@ -157,7 +154,9 @@ func (b *Bridge) logf(f string, a ...any) {
// re-applies. QuotaGB is included (SLICE 4) so a hub-side quota raise reaches the target — on an // re-applies. QuotaGB is included (SLICE 4) so a hub-side quota raise reaches the target — on an
// already-provisioned guest that re-apply is a cheap key-auth-first re-pin (no password consumed). // already-provisioned guest that re-apply is a cheap key-auth-first re-pin (no password consumed).
func descriptorHash(o config.OffsiteConfig) string { func descriptorHash(o config.OffsiteConfig) string {
s := fmt.Sprintf("%s|%s|%s|%d|%s|%s|%d", o.Type, o.Host, o.User, o.Port, o.RepoPath, o.HostFingerprint, o.QuotaGB) // "|pinned-v1" (v0.289.0): a box upgraded from the password era re-applies ONCE, which is what moves
// its key to the pinned append-only line. Without it the old marker matches and nothing migrates.
s := fmt.Sprintf("%s|%s|%s|%d|%s|%s|%d|pinned-v1", o.Type, o.Host, o.User, o.Port, o.RepoPath, o.HostFingerprint, o.QuotaGB)
sum := sha256.Sum256([]byte(s)) sum := sha256.Sum256([]byte(s))
return hex.EncodeToString(sum[:]) return hex.EncodeToString(sum[:])
} }
@@ -212,57 +211,64 @@ func (b *Bridge) Reconcile(ctx context.Context) error {
return fmt.Errorf("offsite-apply: host-key MISMATCH for %s (got %s, want %s) — refusing to pin/install (possible MITM)", o.Host, scannedFP, o.HostFingerprint) return fmt.Errorf("offsite-apply: host-key MISMATCH for %s (got %s, want %s) — refusing to pin/install (possible MITM)", o.Host, scannedFP, o.HostFingerprint)
} }
// 1b) Key-auth-first: if an already-installed key still authenticates (pinned to the key we JUST // 2) The key: the one already installed (a running box, or one upgraded from the password era — the
// verified — the probe never weakens the identity check), the descriptor change is applied by // SAME key is re-registered and comes back pinned), else a fresh pair.
// re-pinning + reconfiguring alone. NO one-time password is consumed — a stale/re-scanned descriptor privPEM, pub, fresh := "", "", false
// on an already-provisioned guest no longer loops on consume-404. if b.Existing != nil {
if b.Prober != nil { if pem := b.Existing(); pem != "" {
if privPEM, ok := b.Prober.Probe(ctx, o.Host, o.User, port, knownHostsLine); ok { if p, perr := PublicKeyOf(pem); perr == nil {
if err := b.Enabler.ConfigureOffbox(ctx, o.Host, o.User, port, o.RepoPath, privPEM, knownHostsLine, o.QuotaGB); err != nil { privPEM, pub = pem, p
return fmt.Errorf("offsite-apply: reconfigure (key-auth-first): %w", err) } else {
b.logf("[WARN] [offsite-apply] the installed key does not parse (%v) — generating a fresh one", perr)
} }
if err := b.writeMarker(h); err != nil {
b.logf("[WARN] [offsite-apply] key-auth-first applied for %s but failed to persist the marker: %v", o.Host, err)
return err
}
b.logf("[INFO] [offsite-apply] existing key still authenticates to %s@%s — re-pinned + reconfigured without consuming a password", o.User, o.Host)
return nil
} }
} }
if privPEM == "" {
// 2) Generate the controller keypair. if privPEM, pub, err = b.KeyGen.Generate(); err != nil {
privPEM, pubAuthorized, err := b.KeyGen.Generate() return fmt.Errorf("offsite-apply: keygen: %w", err)
}
fresh = true
}
fp, err := FingerprintOf(pub)
if err != nil { if err != nil {
return fmt.Errorf("offsite-apply: keygen: %w", err) return fmt.Errorf("offsite-apply: own key: %w", err)
} }
// 3) Consume the one-time password (single-use). After this the password is SPENT. // 3) Already pinned? (a descriptor change on a healthy box, or a restart) → no hub write needed.
password, err := b.Consumer.Consume(ctx) pinned := b.Prober.Probe(ctx, o.Host, o.User, port, knownHostsLine, privPEM)
if err != nil { if !pinned {
return fmt.Errorf("offsite-apply: consume one-time password: %w", err) // 4) Ask the hub's registrar to install it pinned append-only. NO password crosses this call.
hubFP, rerr := b.Registrar.Register(ctx, pub)
if rerr != nil {
return fmt.Errorf("offsite-apply: register key with the hub: %w", rerr)
}
if hubFP != fp {
return fmt.Errorf("offsite-apply: hub installed key %s, ours is %s — refusing", hubFP, fp)
}
// 5) Prove the key now reaches the PINNED server (positive observable, not just auth).
if !b.Prober.Probe(ctx, o.Host, o.User, port, knownHostsLine, privPEM) {
return fmt.Errorf("offsite-apply: key %s registered but does not reach the pinned append-only server", fp)
}
b.logf("[INFO] [offsite-apply] the hub installed key %s append-only on %s@%s (fresh=%v)", fp, o.User, o.Host, fresh)
} }
// 4) Install the pubkey using the password (proven ssh-copy-id -s -f), verify key auth. Pin the // 6) Configure the offbox target (writes the key) + go EscrowState="pending" when new (fork-4).
// scanner-VERIFIED known_hosts line on the install/verify connections — never accept-new — so a MITM
// cannot substitute a different key in the gap between the scan and the install.
if err := b.Installer.Install(ctx, o.Host, o.User, port, password, privPEM, pubAuthorized, knownHostsLine); err != nil {
// The password is now SPENT but install failed — a loud, distinct signal: the operator must reset
// the box password on the hub and let the bridge retry. Do NOT mark applied.
b.logf("[ERROR] [offsite-apply] key install FAILED after consuming the one-time password for %s@%s — the password is spent; reset it on the hub to retry: %v", o.User, o.Host, err)
return fmt.Errorf("offsite-apply: install key (password spent — needs hub reset): %w", err)
}
// 5) Configure the offbox target + go EscrowState="pending" (fork-4 enable path).
if err := b.Enabler.ConfigureOffbox(ctx, o.Host, o.User, port, o.RepoPath, privPEM, knownHostsLine, o.QuotaGB); err != nil { if err := b.Enabler.ConfigureOffbox(ctx, o.Host, o.User, port, o.RepoPath, privPEM, knownHostsLine, o.QuotaGB); err != nil {
return fmt.Errorf("offsite-apply: configure offbox: %w", err) return fmt.Errorf("offsite-apply: configure offbox: %w", err)
} }
// 6) Persist the marker LAST — only a fully-applied descriptor is recorded (fail-safe). // 7) Confirm: the hub removes every other key line (rotation's last step). Best effort — a failed
// confirm leaves an extra PINNED line, which cannot delete; the next apply retries it.
if err := b.Registrar.Confirm(ctx, fp); err != nil {
b.logf("[WARN] [offsite-apply] confirm key %s with the hub failed (an extra pinned line may remain; retried on the next apply): %v", fp, err)
}
// 8) Persist the marker LAST — only a fully-applied descriptor is recorded (fail-safe).
if err := b.writeMarker(h); err != nil { if err := b.writeMarker(h); err != nil {
b.logf("[WARN] [offsite-apply] applied offsite for %s but failed to persist the marker (will re-apply next cycle — the password is spent, needs reset): %v", o.Host, err) b.logf("[WARN] [offsite-apply] applied offsite for %s but failed to persist the marker (re-applies next cycle; harmless — no password is involved): %v", o.Host, err)
return err return err
} }
b.logf("[INFO] [offsite-apply] offsite configured for %s@%s:%s (pending key escrow)", o.User, o.Host, o.RepoPath) b.logf("[INFO] [offsite-apply] offsite configured append-only for %s@%s:%s (key %s)", o.User, o.Host, o.RepoPath, fp)
return nil return nil
} }
@@ -3,7 +3,10 @@ package offsiteapply
import ( import (
"bytes" "bytes"
"context" "context"
"crypto/sha256"
"encoding/hex"
"errors" "errors"
"fmt"
"log" "log"
"os" "os"
"path/filepath" "path/filepath"
@@ -15,19 +18,36 @@ import (
// --- fakes --- // --- fakes ---
type fakeConsumer struct { // fakeRegistrar is the hub's key registrar. `calls` counts Register — the one call that writes to the
pw string // provider; it is the "did the bridge go to the hub" signal the settle-gate and retry tests count.
err error type fakeRegistrar struct {
calls int calls int
panics bool confirms []string
gotPub string
wrongFP bool
err error
confirmEr error
panics bool
} }
func (f *fakeConsumer) Consume(_ context.Context) (string, error) { func (f *fakeRegistrar) Register(_ context.Context, pub string) (string, error) {
if f.panics { if f.panics {
panic("consume must NOT be called (idempotent no-op)") panic("register must NOT be called")
} }
f.calls++ f.calls++
return f.pw, f.err f.gotPub = pub
if f.err != nil {
return "", f.err
}
if f.wrongFP {
return "SHA256:somebody-else", nil
}
return FingerprintOf(pub)
}
func (f *fakeRegistrar) Confirm(_ context.Context, fp string) error {
f.confirms = append(f.confirms, fp)
return f.confirmEr
} }
type fakeScanner struct { type fakeScanner struct {
@@ -39,40 +59,25 @@ func (f *fakeScanner) Scan(_ context.Context, _ string, _ int) (string, string,
return f.fp, f.line, f.err return f.fp, f.line, f.err
} }
type fakeKeyGen struct{ priv, pub string } // realKeyGen returns ONE real pair (the bridge parses keys now — fingerprints are real).
type realKeyGen struct{ priv, pub string }
func (f *fakeKeyGen) Generate() (string, string, error) { return f.priv, f.pub, nil } func (f *realKeyGen) Generate() (string, string, error) { return f.priv, f.pub, nil }
type fakeInstaller struct {
err error
calls int
gotPub string
gotPriv string
gotPw string
gotKnownHost string
}
func (f *fakeInstaller) Install(_ context.Context, _, _ string, _ int, password, privPEM, pub, knownHosts string) error {
f.calls++
f.gotPub, f.gotPriv, f.gotPw, f.gotKnownHost = pub, privPEM, password, knownHosts
return f.err
}
// fakeProber models the provider: a key reaches the pinned server once the registrar installed it
// (pinnedAfterRegister) or from the start (pinnedInitially).
type fakeProber struct { type fakeProber struct {
pem string reg *fakeRegistrar
ok bool pinnedInitially bool
panics bool pinnedAfterRegister bool
calls int calls int
gotKH string gotKH, gotPriv string
} }
func (f *fakeProber) Probe(_ context.Context, _, _ string, _ int, kh string) (string, bool) { func (f *fakeProber) Probe(_ context.Context, _, _ string, _ int, kh, priv string) bool {
if f.panics {
panic("prober must NOT be called (verify must precede the probe)")
}
f.calls++ f.calls++
f.gotKH = kh f.gotKH, f.gotPriv = kh, priv
return f.pem, f.ok return f.pinnedInitially || (f.pinnedAfterRegister && f.reg.calls > 0)
} }
type fakeEnabler struct { type fakeEnabler struct {
@@ -90,187 +95,200 @@ func (f *fakeEnabler) ConfigureOffbox(_ context.Context, host, _ string, _ int,
return f.err return f.err
} }
func newBridge(t *testing.T, o config.OffsiteConfig) (*Bridge, *fakeConsumer, *fakeInstaller, *fakeEnabler, *bytes.Buffer) { var testPriv, testPub = func() (string, string) {
p, q, err := ED25519KeyGen{}.Generate()
if err != nil {
panic(err)
}
return p, q
}()
var otherPriv, otherPub = func() (string, string) {
p, q, _ := ED25519KeyGen{}.Generate()
return p, q
}()
func newBridge(t *testing.T, o config.OffsiteConfig) (*Bridge, *fakeRegistrar, *fakeProber, *fakeEnabler, *bytes.Buffer) {
t.Helper() t.Helper()
cfg := &config.Config{} cfg := &config.Config{}
cfg.Offsite = o cfg.Offsite = o
cons := &fakeConsumer{pw: "the-transient-pw"} reg := &fakeRegistrar{}
inst := &fakeInstaller{} pr := &fakeProber{reg: reg, pinnedAfterRegister: true}
en := &fakeEnabler{} en := &fakeEnabler{}
var logbuf bytes.Buffer var logbuf bytes.Buffer
b := &Bridge{ b := &Bridge{
Cfg: cfg, Cfg: cfg,
Consumer: cons, Registrar: reg,
Scanner: &fakeScanner{fp: "SHA256:goodfp", line: "[h]:23 ssh-ed25519 AAAAKEY"}, Scanner: &fakeScanner{fp: "SHA256:goodfp", line: "[h]:23 ssh-ed25519 AAAAKEY"},
KeyGen: &fakeKeyGen{priv: "PRIVPEM", pub: "ssh-ed25519 AAAAPUB felhom"}, KeyGen: &realKeyGen{priv: testPriv, pub: testPub},
Installer: inst, Prober: pr,
Enabler: en, Enabler: en,
MarkerPath: filepath.Join(t.TempDir(), "offbox", "applied_marker"), MarkerPath: filepath.Join(t.TempDir(), "offbox", "applied_marker"),
Logger: log.New(&logbuf, "", 0), Logger: log.New(&logbuf, "", 0),
} }
return b, cons, inst, en, &logbuf return b, reg, pr, en, &logbuf
} }
func goodOffsite() config.OffsiteConfig { func goodOffsite() config.OffsiteConfig {
return config.OffsiteConfig{Enabled: true, Type: "shared", Host: "h", User: "u", Port: 23, RepoPath: "/home/felhom-repo", QuotaGB: 50, HostFingerprint: "SHA256:goodfp"} return config.OffsiteConfig{Enabled: true, Type: "shared", Host: "h", User: "u", Port: 23, RepoPath: "/home/felhom-repo", QuotaGB: 50, HostFingerprint: "SHA256:goodfp"}
} }
// Scenario A — full apply: consume → verify-pin → install → configure offbox → marker persisted; pw not logged. // A fresh box: verify host key → register the PUBLIC key → prove the pin → configure → confirm → marker.
func TestBridge_AppliesEndToEnd(t *testing.T) { // What crosses to the hub is a public key and nothing else.
b, cons, inst, en, logbuf := newBridge(t, goodOffsite()) func TestBridge_FreshBoxRegistersPublicKeyOnly(t *testing.T) {
b, reg, pr, en, logbuf := newBridge(t, goodOffsite())
if err := b.Reconcile(context.Background()); err != nil { if err := b.Reconcile(context.Background()); err != nil {
t.Fatalf("reconcile: %v", err) t.Fatalf("reconcile: %v", err)
} }
if cons.calls != 1 { if reg.calls != 1 || reg.gotPub != testPub {
t.Fatalf("consume calls = %d, want 1", cons.calls) t.Fatalf("register calls=%d pub=%q", reg.calls, reg.gotPub)
} }
if inst.calls != 1 || inst.gotPw != "the-transient-pw" || inst.gotPub == "" { if strings.Contains(reg.gotPub, "PRIVATE") {
t.Fatalf("installer not called with pw+pub: %+v", inst) t.Fatal("the private key was sent to the hub")
} }
if inst.gotKnownHost != "[h]:23 ssh-ed25519 AAAAKEY" { if pr.calls != 2 || pr.gotKH != "[h]:23 ssh-ed25519 AAAAKEY" {
t.Fatalf("installer must receive the scanner-verified known_hosts to pin (no TOFU), got %q", inst.gotKnownHost) t.Fatalf("probe calls=%d kh=%q (want before+after register, pinned to the scanned host key)", pr.calls, pr.gotKH)
} }
if en.calls != 1 || en.gotHost != "h" || en.gotKnownHost != "[h]:23 ssh-ed25519 AAAAKEY" || en.gotPriv != "PRIVPEM" { fp, _ := FingerprintOf(testPub)
t.Fatalf("enabler not called with the pinned known_hosts + key: %+v", en) if en.calls != 1 || en.gotPriv != testPriv || en.gotQuotaGB != 50 {
t.Fatalf("enabler: %+v", en)
} }
if en.gotQuotaGB != 50 { if len(reg.confirms) != 1 || reg.confirms[0] != fp {
t.Fatalf("the bridge must map the descriptor's quota_gb into the target (SLICE 4), got %d", en.gotQuotaGB) t.Fatalf("confirm = %v, want [%s]", reg.confirms, fp)
} }
if b.readMarker() != descriptorHash(b.Cfg.Offsite) { if _, err := os.Stat(b.MarkerPath); err != nil {
t.Fatal("marker not persisted after a successful apply") t.Fatalf("marker not persisted: %v", err)
} }
if strings.Contains(logbuf.String(), "the-transient-pw") { if !strings.Contains(logbuf.String(), "append-only") {
t.Fatal("the one-time password LEAKED into a log line") t.Fatalf("log does not say append-only:\n%s", logbuf.String())
} }
} }
// SLICE 4 — a quota-only descriptor change re-applies (the hash includes QuotaGB), and with a working // THE MIGRATION: a box upgraded from the password era has an installed (unpinned) key and an applied
// key it costs no password: key-auth-first re-pins + remaps the quota. // marker written under the OLD descriptor hash. The new hash ("|pinned-v1") re-applies once; the SAME
func TestBridge_QuotaChangeReappliesWithoutConsume(t *testing.T) { // key is registered (no new keypair) and comes back pinned.
b, cons, _, en, _ := newBridge(t, goodOffsite()) func TestBridge_UpgradedBoxReRegistersSameKey(t *testing.T) {
cons.panics = true b, reg, _, en, _ := newBridge(t, goodOffsite())
b.Prober = &fakeProber{pem: "EXISTINGPEM", ok: true} b.Existing = func() string { return otherPriv }
// marker for the OLD quota (25) already applied; the descriptor now says 50 b.KeyGen = nil // must not be needed
old := b.Cfg.Offsite o := goodOffsite()
old.QuotaGB = 25 if err := os.MkdirAll(filepath.Dir(b.MarkerPath), 0o700); err != nil {
_ = os.MkdirAll(filepath.Dir(b.MarkerPath), 0o700) t.Fatal(err)
_ = os.WriteFile(b.MarkerPath, []byte(descriptorHash(old)), 0o600)
if err := b.Reconcile(context.Background()); err != nil {
t.Fatalf("quota-change reconcile: %v", err)
} }
if en.calls != 1 || en.gotQuotaGB != 50 { // The pre-v0.289.0 marker for this exact descriptor:
t.Fatalf("a quota raise must re-apply and map the NEW quota (no consume): %+v", en) if err := os.WriteFile(b.MarkerPath, []byte(legacyDescriptorHash(o)), 0o600); err != nil {
}
}
// Key-auth-first (Scenario B) — the existing key still works: NO consume, NO install; re-verify + re-pin +
// reconfigure with the EXISTING key, marker updated.
func TestBridge_KeyAuthFirstSkipsConsume(t *testing.T) {
b, cons, inst, en, _ := newBridge(t, goodOffsite())
cons.panics = true // the whole point: a working key must NEVER consume the one-time password
prober := &fakeProber{pem: "EXISTINGPEM", ok: true}
b.Prober = prober
if err := b.Reconcile(context.Background()); err != nil {
t.Fatalf("key-auth-first reconcile: %v", err)
}
if prober.calls != 1 || prober.gotKH != "[h]:23 ssh-ed25519 AAAAKEY" {
t.Fatalf("probe must run once with the freshly-scanned pinned known_hosts: %+v", prober)
}
if inst.calls != 0 {
t.Fatal("installer must NOT run when the existing key authenticates")
}
if en.calls != 1 || en.gotPriv != "EXISTINGPEM" || en.gotKnownHost != "[h]:23 ssh-ed25519 AAAAKEY" {
t.Fatalf("enabler must reconfigure with the EXISTING key + fresh pin: %+v", en)
}
if b.readMarker() != descriptorHash(b.Cfg.Offsite) {
t.Fatal("marker must be updated after a key-auth-first apply")
}
}
// Scenario C — key-auth-first must NOT weaken the fresh path: probe fails → the full
// verify→consume→install path runs unchanged (with the freshly GENERATED key).
func TestBridge_FreshGuestFallsThroughToFullPath(t *testing.T) {
b, cons, inst, en, _ := newBridge(t, goodOffsite())
b.Prober = &fakeProber{ok: false} // fresh guest: no key / auth refused
if err := b.Reconcile(context.Background()); err != nil {
t.Fatalf("fresh-guest reconcile: %v", err)
}
if cons.calls != 1 || inst.calls != 1 {
t.Fatalf("fresh guest must consume+install exactly once: cons=%d inst=%d", cons.calls, inst.calls)
}
if en.calls != 1 || en.gotPriv != "PRIVPEM" {
t.Fatalf("fresh guest must configure with the GENERATED key: %+v", en)
}
if b.readMarker() != descriptorHash(b.Cfg.Offsite) {
t.Fatal("marker must be persisted after a full-path apply")
}
}
// Scenario B — host-key mismatch → refuse: no consume, no install, no configure, no marker.
func TestBridge_HostKeyMismatchRefuses(t *testing.T) {
b, cons, inst, en, _ := newBridge(t, goodOffsite())
b.Scanner = &fakeScanner{fp: "SHA256:ATTACKER", line: "[h]:23 ssh-ed25519 EVIL"}
b.Prober = &fakeProber{panics: true} // the probe must NEVER run when the identity check failed
err := b.Reconcile(context.Background())
if err == nil || !strings.Contains(err.Error(), "MISMATCH") {
t.Fatalf("mismatch must refuse, got %v", err)
}
if cons.calls != 0 || inst.calls != 0 || en.calls != 0 {
t.Fatalf("nothing may proceed on a host-key mismatch: cons=%d inst=%d en=%d", cons.calls, inst.calls, en.calls)
}
if b.readMarker() != "" {
t.Fatal("no marker may be written on a mismatch")
}
}
// Scenario C — idempotent: marker already matches → no-op, consume is NOT called.
func TestBridge_IdempotentNoReconsume(t *testing.T) {
b, cons, inst, en, _ := newBridge(t, goodOffsite())
cons.panics = true // Consume must not be called
// pre-seed the marker with the current descriptor hash
_ = os.MkdirAll(filepath.Dir(b.MarkerPath), 0o700)
if err := os.WriteFile(b.MarkerPath, []byte(descriptorHash(b.Cfg.Offsite)), 0o600); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if err := b.Reconcile(context.Background()); err != nil { if err := b.Reconcile(context.Background()); err != nil {
t.Fatalf("idempotent reconcile must be a clean no-op, got %v", err) t.Fatalf("reconcile: %v", err)
} }
if cons.calls != 0 || inst.calls != 0 || en.calls != 0 { if reg.calls != 1 || reg.gotPub != otherPub {
t.Fatal("an already-applied descriptor must be a full no-op") t.Fatalf("the upgraded box must re-register its EXISTING key: calls=%d pub=%q", reg.calls, reg.gotPub)
}
if en.gotPriv != otherPriv {
t.Fatal("the existing key was not kept")
} }
} }
// Scenario D — install fails → fail-safe: marker NOT persisted, offbox NOT configured, loud log. // A restart / descriptor change on a box whose key is already pinned: no hub write at all.
func TestBridge_InstallFailIsFailSafe(t *testing.T) { func TestBridge_AlreadyPinnedNeedsNoRegister(t *testing.T) {
b, cons, inst, en, logbuf := newBridge(t, goodOffsite()) b, reg, pr, en, _ := newBridge(t, goodOffsite())
inst.err = errors.New("ssh-copy-id refused") b.Existing = func() string { return testPriv }
err := b.Reconcile(context.Background()) pr.pinnedInitially = true
if err == nil { if err := b.Reconcile(context.Background()); err != nil {
t.Fatal("install failure must error") t.Fatalf("reconcile: %v", err)
}
if reg.calls != 0 || en.calls != 1 {
t.Fatalf("register=%d enable=%d", reg.calls, en.calls)
}
}
// The registrar answered but the key still does not reach the PINNED server (e.g. an unpinned line
// won): refuse — never configure a key that can delete.
func TestBridge_RegisteredButNotPinnedRefuses(t *testing.T) {
b, reg, pr, en, _ := newBridge(t, goodOffsite())
pr.pinnedAfterRegister = false
if err := b.Reconcile(context.Background()); err == nil {
t.Fatal("a key that does not reach the pinned server must refuse")
}
if reg.calls != 1 || en.calls != 0 {
t.Fatalf("register=%d enable=%d", reg.calls, en.calls)
}
if _, err := os.Stat(b.MarkerPath); err == nil {
t.Fatal("marker persisted after a refusal")
}
}
func TestBridge_HubInstalledADifferentKeyRefuses(t *testing.T) {
b, reg, _, en, _ := newBridge(t, goodOffsite())
reg.wrongFP = true
if err := b.Reconcile(context.Background()); err == nil || en.calls != 0 {
t.Fatalf("err=%v enable=%d", err, en.calls)
}
}
// Host-key mismatch → refuse before anything touches the hub.
func TestBridge_HostKeyMismatchRefuses(t *testing.T) {
b, reg, pr, en, _ := newBridge(t, goodOffsite())
b.Scanner = &fakeScanner{fp: "SHA256:EVIL", line: "x"}
reg.panics = true
if err := b.Reconcile(context.Background()); err == nil || !strings.Contains(err.Error(), "MISMATCH") {
t.Fatalf("err = %v", err)
}
if pr.calls != 0 || en.calls != 0 {
t.Fatal("probe/configure ran after a host-key mismatch")
}
}
func TestBridge_IdempotentMarker(t *testing.T) {
b, reg, _, _, _ := newBridge(t, goodOffsite())
if err := b.Reconcile(context.Background()); err != nil {
t.Fatal(err)
}
reg.panics = true
if err := b.Reconcile(context.Background()); err != nil {
t.Fatalf("second reconcile: %v", err)
}
}
func TestBridge_RegisterFailIsFailSafe(t *testing.T) {
b, reg, _, en, _ := newBridge(t, goodOffsite())
reg.err = errors.New("hub down")
if err := b.Reconcile(context.Background()); err == nil {
t.Fatal("want error")
} }
if en.calls != 0 { if en.calls != 0 {
t.Fatal("offbox must NOT be configured when install fails") t.Fatal("configured after a failed register")
} }
if b.readMarker() != "" { if _, err := os.Stat(b.MarkerPath); err == nil {
t.Fatal("marker must NOT be persisted on a failed apply (fail-safe)") t.Fatal("marker persisted after failure")
} }
if cons.calls != 1 { }
t.Fatal("the password was consumed (spent) before install")
} // A failed confirm leaves an extra PINNED line (cannot delete) — not a failure of the apply.
if !strings.Contains(logbuf.String(), "password is spent") { func TestBridge_ConfirmFailureIsNotFatal(t *testing.T) {
t.Fatal("a consumed-but-failed install must log the loud 'password is spent' signal") b, reg, _, en, logbuf := newBridge(t, goodOffsite())
reg.confirmEr = errors.New("hub blip")
if err := b.Reconcile(context.Background()); err != nil {
t.Fatalf("reconcile: %v", err)
}
if en.calls != 1 || !strings.Contains(logbuf.String(), "confirm key") {
t.Fatalf("enable=%d log=%s", en.calls, logbuf.String())
} }
} }
// Disabled → no-op (no consume/install/configure).
func TestBridge_DisabledNoOp(t *testing.T) { func TestBridge_DisabledNoOp(t *testing.T) {
o := goodOffsite() o := goodOffsite()
o.Enabled = false o.Enabled = false
b, cons, inst, en, _ := newBridge(t, o) b, reg, _, en, _ := newBridge(t, o)
if err := b.Reconcile(context.Background()); err != nil { reg.panics = true
t.Fatal(err) if err := b.Reconcile(context.Background()); err != nil || en.calls != 0 {
} t.Fatalf("err=%v enable=%d", err, en.calls)
if cons.calls+inst.calls+en.calls != 0 {
t.Fatal("disabled offsite must be a no-op")
} }
} }
// legacyDescriptorHash is the pre-v0.289.0 marker key, byte for byte as v0.288.0 computed it.
func legacyDescriptorHash(o config.OffsiteConfig) string {
sum := sha256.Sum256([]byte(fmt.Sprintf("%s|%s|%s|%d|%s|%s|%d", o.Type, o.Host, o.User, o.Port, o.RepoPath, o.HostFingerprint, o.QuotaGB)))
return hex.EncodeToString(sum[:])
}
+144 -98
View File
@@ -17,16 +17,13 @@ import (
"strings" "strings"
"time" "time"
"gitea.dooplex.hu/admin/felhom-controller/internal/backup"
"golang.org/x/crypto/ssh" "golang.org/x/crypto/ssh"
"golang.org/x/crypto/ssh/knownhosts" "golang.org/x/crypto/ssh/knownhosts"
) )
// --- func adapters (convenient wiring in main.go) --- // --- func adapters (convenient wiring in main.go) ---
type ConsumerFunc func(ctx context.Context) (string, error)
func (f ConsumerFunc) Consume(ctx context.Context) (string, error) { return f(ctx) }
type EnablerFunc func(ctx context.Context, host, user string, port int, repoPath, privPEM, knownHosts string, quotaGB int) error type EnablerFunc func(ctx context.Context, host, user string, port int, repoPath, privPEM, knownHosts string, quotaGB int) error
func (f EnablerFunc) ConfigureOffbox(ctx context.Context, host, user string, port int, repoPath, privPEM, knownHosts string, quotaGB int) error { func (f EnablerFunc) ConfigureOffbox(ctx context.Context, host, user string, port int, repoPath, privPEM, knownHosts string, quotaGB int) error {
@@ -40,48 +37,125 @@ type SettleFunc func() (version, floor string, updateRunning, floorKnown bool)
func (f SettleFunc) SettleState() (string, string, bool, bool) { return f() } func (f SettleFunc) SettleState() (string, string, bool, bool) { return f() }
// --- HTTPConsumer: POST the hub consume-password endpoint with the per-customer API key --- // --- HubRegistrar: the box's half of the hub key registrar (decision 69, hub >= v0.127.0) ---
//
type HTTPConsumer struct { // The box sends ONLY its public key; the hub writes it into the Storage Box sub-account's
// authorized_keys pinned to `rclone serve restic --stdio --append-only <repo>`. Until controller
// v0.289.0 the box fetched the sub-account PASSWORD here (consume-password) — and that password can
// rewrite authorized_keys, i.e. remove the pin (measured 2026-10-03, R-820). No response this client
// reads can carry a password; TestHubRegistrar_NeverAsksForAPassword pins the paths it calls.
type HubRegistrar struct {
HubURL string HubURL string
CustomerID string CustomerID string
APIKey string APIKey string
HC *http.Client HC *http.Client
} }
func (c HTTPConsumer) Consume(ctx context.Context) (string, error) { func (c HubRegistrar) post(ctx context.Context, path string, body any) ([]byte, error) {
if c.HubURL == "" || c.CustomerID == "" || c.APIKey == "" { if c.HubURL == "" || c.CustomerID == "" || c.APIKey == "" {
return "", fmt.Errorf("offsite-apply: consume: hub url/customer/apikey not configured") return nil, fmt.Errorf("offsite-apply: hub url/customer/apikey not configured")
} }
hc := c.HC hc := c.HC
if hc == nil { if hc == nil {
hc = &http.Client{Timeout: 20 * time.Second} hc = &http.Client{Timeout: 3 * time.Minute} // the hub does an SSH round trip to the provider
} }
url := strings.TrimRight(c.HubURL, "/") + "/api/v1/offsite/consume-password/" + c.CustomerID var rd io.Reader
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, nil) if body != nil {
b, err := json.Marshal(body)
if err != nil {
return nil, err
}
rd = strings.NewReader(string(b))
}
url := strings.TrimRight(c.HubURL, "/") + "/api/v1/offsite/" + path + "/" + c.CustomerID
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, rd)
if err != nil { if err != nil {
return "", err return nil, err
} }
req.Header.Set("Authorization", "Bearer "+c.APIKey) req.Header.Set("Authorization", "Bearer "+c.APIKey)
req.Header.Set("Content-Type", "application/json")
resp, err := hc.Do(req) resp, err := hc.Do(req)
if err != nil { if err != nil {
return "", err return nil, err
} }
defer resp.Body.Close() defer resp.Body.Close()
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<16)) raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<16))
if resp.StatusCode == http.StatusNotFound {
return "", fmt.Errorf("no unconsumed offsite password (already consumed or none provisioned)")
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 { if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return "", fmt.Errorf("consume: HTTP %d", resp.StatusCode) return nil, fmt.Errorf("hub %s: HTTP %d: %s", path, resp.StatusCode, truncate(raw))
} }
var body struct { return raw, nil
Password string `json:"password"` }
// Register asks the hub to install pub pinned append-only; returns the key's fingerprint as the hub saw it.
func (c HubRegistrar) Register(ctx context.Context, pub string) (string, error) {
raw, err := c.post(ctx, "register-key", map[string]string{"public_key": strings.TrimSpace(pub)})
if err != nil {
return "", err
} }
if err := json.Unmarshal(raw, &body); err != nil || body.Password == "" { var r struct {
return "", fmt.Errorf("consume: malformed response") Installed bool `json:"installed"`
Fingerprint string `json:"fingerprint"`
} }
return body.Password, nil // NEVER logged if err := json.Unmarshal(raw, &r); err != nil || !r.Installed || r.Fingerprint == "" {
return "", fmt.Errorf("hub register-key: malformed response")
}
return r.Fingerprint, nil
}
// Confirm tells the hub the box now uses fp; the hub removes every other key line.
func (c HubRegistrar) Confirm(ctx context.Context, fp string) error {
_, err := c.post(ctx, "confirm-key", map[string]string{"fingerprint": fp})
return err
}
// MoveAside asks the hub to set the repository aside (never deletes) — the box's pinned key cannot.
func (c HubRegistrar) MoveAside(ctx context.Context) (string, error) {
raw, err := c.post(ctx, "move-aside", nil)
if err != nil {
return "", err
}
var r struct {
MovedTo string `json:"moved_to"`
}
if err := json.Unmarshal(raw, &r); err != nil || r.MovedTo == "" {
return "", fmt.Errorf("hub move-aside: malformed response")
}
return r.MovedTo, nil
}
// --- HubWindowClient: the box's half of the clean-up window (decision 68) ---
type HubWindowClient struct{ Registrar HubRegistrar }
func (c HubWindowClient) Open(ctx context.Context, countBefore int) (backup.OffsiteWindow, error) {
raw, err := c.Registrar.post(ctx, "window-open", map[string]int{"count_before": countBefore})
if err != nil {
return backup.OffsiteWindow{}, err
}
var r struct {
Granted bool `json:"granted"`
WindowID int64 `json:"window_id"`
NewestAllowed string `json:"newest_allowed"`
MaxRemove int `json:"max_remove"`
Reason string `json:"reason"`
}
if err := json.Unmarshal(raw, &r); err != nil {
return backup.OffsiteWindow{}, fmt.Errorf("hub window-open: malformed response")
}
w := backup.OffsiteWindow{Granted: r.Granted, ID: r.WindowID, MaxRemove: r.MaxRemove, Reason: r.Reason}
if r.Granted {
t, perr := time.Parse(time.RFC3339, r.NewestAllowed)
if perr != nil {
return backup.OffsiteWindow{}, fmt.Errorf("hub window-open: bad newest_allowed")
}
w.NewestAllowed = t
}
return w, nil
}
func (c HubWindowClient) Close(ctx context.Context, res backup.OffsiteWindowResult) error {
_, err := c.Registrar.post(ctx, "window-close", res)
return err
} }
// --- KeyscanScanner: capture the box host key (x/crypto/ssh, no binary) → fingerprint + known_hosts line --- // --- KeyscanScanner: capture the box host key (x/crypto/ssh, no binary) → fingerprint + known_hosts line ---
@@ -145,72 +219,21 @@ func (ED25519KeyGen) Generate() (string, string, error) {
return privPEM, pubLine, nil return privPEM, pubLine, nil
} }
// --- SSHCopyIDInstaller: install the pubkey via the proven `sshpass -e ssh-copy-id -p N -s -f`, verify --- // --- PinnedProber: does this key reach the PINNED append-only server? ---
//
type SSHCopyIDInstaller struct{} // Measured on the provider 2026-10-03 (audits/offsite-lock-build-2026-10-03/partA/A5): `ssh -i <key>
// host probe` with stdin closed exits 0 and prints rclone's NOTICE when the key is pinned (the forced
func (SSHCopyIDInstaller) Install(ctx context.Context, host, user string, port int, password, privPEM, pubAuthorized, knownHosts string) error { // rclone starts, sees EOF, exits); an UNPINNED key gets the restricted shell → "Command not found",
if strings.TrimSpace(knownHosts) == "" { // exit 8. A refused key exits 255. So ok = exit 0 AND "rclone" in the output — a POSITIVE observable.
return fmt.Errorf("ssh-copy-id: empty known_hosts — refusing to install without a pinned host key") type PinnedProber struct {
} Timeout time.Duration // 0 → 20 s
// ssh-copy-id -s (SFTP mode) mktemp's its batch file under ~/.ssh and dies LOCALLY if the directory // Run is the exec seam (tests); nil → real ssh.
// doesn't exist — the container image ships without /root/.ssh (live finding: the one-time password was Run func(ctx context.Context, args []string) ([]byte, error)
// consumed, then the install failed before ever connecting).
if home, err := os.UserHomeDir(); err == nil {
if err := os.MkdirAll(filepath.Join(home, ".ssh"), 0o700); err != nil {
return fmt.Errorf("ssh-copy-id: ensure ~/.ssh (needed by -s mode): %w", err)
}
}
work, err := os.MkdirTemp("", "felhom-keyinstall-")
if err != nil {
return err
}
defer os.RemoveAll(work)
pubPath := filepath.Join(work, "id.pub")
privPath := filepath.Join(work, "id")
khPath := filepath.Join(work, "known_hosts")
if err := os.WriteFile(pubPath, []byte(pubAuthorized), 0o600); err != nil {
return err
}
if err := os.WriteFile(privPath, []byte(privPEM), 0o600); err != nil {
return err
}
// Pin the scanner-VERIFIED host key: StrictHostKeyChecking=yes against this known_hosts refuses any
// other key (no accept-new/TOFU) — the ssh-copy-id + verify sessions connect ONLY to the box whose
// fingerprint the bridge already matched against the hub descriptor.
if err := os.WriteFile(khPath, []byte(knownHosts+"\n"), 0o600); err != nil {
return err
}
// Install (SSHPASS env is read by `sshpass -e`; the password never appears on argv).
install := exec.CommandContext(ctx, "sshpass", "-e", "ssh-copy-id", "-p", strconv.Itoa(port), "-s", "-f",
"-i", pubPath, "-o", "StrictHostKeyChecking=yes", "-o", "UserKnownHostsFile="+khPath, user+"@"+host)
install.Env = append(os.Environ(), "SSHPASS="+password)
if out, err := install.CombinedOutput(); err != nil {
return fmt.Errorf("ssh-copy-id: %w: %s", err, truncate(out))
}
// Verify passwordless key auth (an SFTP no-op; the box's restricted shell only offers SFTP).
verify := exec.CommandContext(ctx, "sftp", "-b", "-", "-P", strconv.Itoa(port),
"-i", privPath, "-oBatchMode=yes", "-oStrictHostKeyChecking=yes", "-oUserKnownHostsFile="+khPath, user+"@"+host)
verify.Stdin = strings.NewReader("pwd\n")
if out, err := verify.CombinedOutput(); err != nil {
return fmt.Errorf("key-auth verify failed after install: %w: %s", err, truncate(out))
}
return nil
} }
// --- SFTPKeyAuthProber: does the ALREADY-INSTALLED key still authenticate? (key-auth-first) --- func (p PinnedProber) Probe(ctx context.Context, host, user string, port int, knownHosts, privPEM string) bool {
if strings.TrimSpace(privPEM) == "" {
// SFTPKeyAuthProber probes passwordless auth with the existing installed key (KeyPath), pinned to the return false
// freshly-verified knownHosts line. No key file → ok=false (fresh guest). The probe never logs secrets.
type SFTPKeyAuthProber struct {
KeyPath string // the installed key, e.g. <dataDir>/offbox/ssh_key
Timeout time.Duration // per-probe budget; 0 → 20s
}
func (p SFTPKeyAuthProber) Probe(ctx context.Context, host, user string, port int, knownHosts string) (string, bool) {
pem, err := os.ReadFile(p.KeyPath)
if err != nil {
return "", false // no existing key — a fresh guest; take the full path
} }
timeout := p.Timeout timeout := p.Timeout
if timeout == 0 { if timeout == 0 {
@@ -220,21 +243,44 @@ func (p SFTPKeyAuthProber) Probe(ctx context.Context, host, user string, port in
defer cancel() defer cancel()
work, err := os.MkdirTemp("", "felhom-keyprobe-") work, err := os.MkdirTemp("", "felhom-keyprobe-")
if err != nil { if err != nil {
return "", false return false
} }
defer os.RemoveAll(work) defer os.RemoveAll(work)
khPath := filepath.Join(work, "known_hosts") khPath, keyPath := filepath.Join(work, "known_hosts"), filepath.Join(work, "id")
if err := os.WriteFile(khPath, []byte(knownHosts+"\n"), 0o600); err != nil { if os.WriteFile(khPath, []byte(knownHosts+"\n"), 0o600) != nil || os.WriteFile(keyPath, []byte(privPEM), 0o600) != nil {
return "", false return false
} }
probe := exec.CommandContext(pctx, "sftp", "-b", "-", "-P", strconv.Itoa(port), args := []string{"-p", strconv.Itoa(port), "-i", keyPath, "-oBatchMode=yes", "-oConnectTimeout=10", "-oIdentitiesOnly=yes",
"-i", p.KeyPath, "-oBatchMode=yes", "-oConnectTimeout=10", "-oStrictHostKeyChecking=yes", "-oUserKnownHostsFile=" + khPath, user + "@" + host, "probe"}
"-oStrictHostKeyChecking=yes", "-oUserKnownHostsFile="+khPath, user+"@"+host) run := p.Run
probe.Stdin = strings.NewReader("pwd\n") if run == nil {
if err := probe.Run(); err != nil { run = func(ctx context.Context, args []string) ([]byte, error) {
return "", false // auth refused / unreachable — fall through to the full path cmd := exec.CommandContext(ctx, "ssh", args...)
cmd.Stdin = strings.NewReader("")
return cmd.CombinedOutput()
}
} }
return string(pem), true out, err := run(pctx, args)
return err == nil && strings.Contains(string(out), "rclone")
}
// PublicKeyOf derives the authorized_keys line of an OpenSSH private key (the migration path: a box
// whose key predates the pin registers the SAME key, so the hub re-writes it pinned).
func PublicKeyOf(privPEM string) (string, error) {
signer, err := ssh.ParsePrivateKey([]byte(privPEM))
if err != nil {
return "", err
}
return string(ssh.MarshalAuthorizedKey(signer.PublicKey())), nil
}
// FingerprintOf returns the SHA256 fingerprint of an authorized_keys line.
func FingerprintOf(pub string) (string, error) {
pk, _, _, _, err := ssh.ParseAuthorizedKey([]byte(pub))
if err != nil {
return "", err
}
return ssh.FingerprintSHA256(pk), nil
} }
func truncate(b []byte) string { func truncate(b []byte) string {
@@ -68,18 +68,18 @@ func (c *fakeClock) sleep(_ context.Context, d time.Duration) {
// settleBridge builds a bridge wired for a FULL Reconcile (so a released gate consumes exactly once) // settleBridge builds a bridge wired for a FULL Reconcile (so a released gate consumes exactly once)
// plus the injectable settle-gate. Prober is nil → the fresh consume+install path runs on release. // plus the injectable settle-gate. Prober is nil → the fresh consume+install path runs on release.
func settleBridge(t *testing.T, s *fakeSettle, clk *fakeClock) (*Bridge, *fakeConsumer, *bytes.Buffer) { func settleBridge(t *testing.T, s *fakeSettle, clk *fakeClock) (*Bridge, *fakeRegistrar, *bytes.Buffer) {
t.Helper() t.Helper()
cfg := &config.Config{} cfg := &config.Config{}
cfg.Offsite = goodOffsite() cfg.Offsite = goodOffsite()
cons := &fakeConsumer{pw: "the-transient-pw"} cons := &fakeRegistrar{}
var logbuf bytes.Buffer var logbuf bytes.Buffer
b := &Bridge{ b := &Bridge{
Cfg: cfg, Cfg: cfg,
Consumer: cons, Registrar: cons,
Scanner: &fakeScanner{fp: "SHA256:goodfp", line: "[h]:23 ssh-ed25519 AAAAKEY"}, Scanner: &fakeScanner{fp: "SHA256:goodfp", line: "[h]:23 ssh-ed25519 AAAAKEY"},
KeyGen: &fakeKeyGen{priv: "PRIVPEM", pub: "ssh-ed25519 AAAAPUB felhom"}, KeyGen: &realKeyGen{priv: testPriv, pub: testPub},
Installer: &fakeInstaller{}, Prober: &fakeProber{reg: cons, pinnedAfterRegister: true},
Enabler: &fakeEnabler{}, Enabler: &fakeEnabler{},
MarkerPath: filepath.Join(t.TempDir(), "offbox", "applied_marker"), MarkerPath: filepath.Join(t.TempDir(), "offbox", "applied_marker"),
Logger: log.New(&logbuf, "", 0), Logger: log.New(&logbuf, "", 0),
+10
View File
@@ -314,6 +314,10 @@ type OffboxTarget struct {
// apply-bridge. 0 = no soft limit (dedicated boxes are Hetzner-enforced; manual targets unset). // apply-bridge. 0 = no soft limit (dedicated boxes are Hetzner-enforced; manual targets unset).
// Felhom-enforced: at ≥100% NEW backup runs are refused (prune/restore never are); ≥80% warns. // Felhom-enforced: at ≥100% NEW backup runs are refused (prune/restore never are); ≥80% warns.
QuotaGB int `json:"quota_gb,omitempty"` QuotaGB int `json:"quota_gb,omitempty"`
// Transport (v0.289.0, decision 69): "rclone-pinned" for the HUB-provisioned Storage Box tier — the
// box's key reaches only an append-only rclone server and cannot delete; "" (the default) is the
// household's own SFTP NAS target, unchanged. Set by ApplyOffsiteTarget, never by a form.
Transport string `json:"transport,omitempty"`
// Runtime status (written by the off-box runner; never holds a secret). // Runtime status (written by the off-box runner; never holds a secret).
LastRun string `json:"last_run,omitempty"` // RFC3339 LastRun string `json:"last_run,omitempty"` // RFC3339
@@ -1330,6 +1334,12 @@ func (s *Settings) GetOffboxTarget() *OffboxTarget {
} }
// SetOffboxTarget saves (or clears, on nil) the off-box target config. // SetOffboxTarget saves (or clears, on nil) the off-box target config.
// TransportRclonePinned marks the hub-provisioned, append-only off-site tier (decision 69).
const TransportRclonePinned = "rclone-pinned"
// Pinned reports whether the target is the append-only hub tier.
func (t *OffboxTarget) Pinned() bool { return t != nil && t.Transport == TransportRclonePinned }
func (s *Settings) SetOffboxTarget(t *OffboxTarget) error { func (s *Settings) SetOffboxTarget(t *OffboxTarget) error {
s.mu.Lock() s.mu.Lock()
defer s.mu.Unlock() defer s.mu.Unlock()
@@ -99,6 +99,11 @@ func (s *Server) offboxConfigHandler(w http.ResponseWriter, r *http.Request) {
Host: host, Port: port, User: user, RepoPath: repoPath, Host: host, Port: port, User: user, RepoPath: repoPath,
Schedule: "daily", Schedule: "daily",
} }
if prev != nil && prev.Pinned() && prev.Host == host && prev.User == user {
// A hand edit of the hub-provisioned tier keeps its append-only transport (decision 69); a
// different host/user is the household's own NAS and starts as SFTP.
tgt.Transport = prev.Transport
}
if prev != nil { // preserve runtime status fields across an edit if prev != nil { // preserve runtime status fields across an edit
tgt.LastRun, tgt.LastStatus, tgt.LastError = prev.LastRun, prev.LastStatus, prev.LastError tgt.LastRun, tgt.LastStatus, tgt.LastError = prev.LastRun, prev.LastStatus, prev.LastError
// R-100: LastSuccess is runtime status like the rest — an edit to the host/path/schedule must // R-100: LastSuccess is runtime status like the rest — an edit to the host/path/schedule must