diff --git a/REUSE.md b/REUSE.md index 31ef352b..482d9c6c 100644 --- a/REUSE.md +++ b/REUSE.md @@ -71,7 +71,7 @@ | Symbol | File | Short signature | Use for | Gotchas | |---|---|---|---|---| -| `offsitekeys.Registrar` (`Install` / `Confirm` / `Audit` / `OpenWindow` / `CloseWindow` / `MoveAside`) | hub/internal/offsitekeys/offsitekeys.go | `(ctx, Target, password, …)` | EVERY write to a sub-account's `.ssh/authorized_keys` and every repo move-aside | **The only writer of that file.** Uses the provider's port-23 restricted shell (`dd of=` takes stdin, `mv` overwrites, `test` does NOT exist — measured); an unpinned line is a deletion route and is dropped on every install; the window line goes FIRST (first match wins). Never `rm`. | +| `offsitekeys.Registrar` (`Install` / `Confirm` / `Audit` / `OpenWindow` / `CloseWindow` / `MoveAside`) | hub/internal/offsitekeys/offsitekeys.go | `(ctx, Target, password, …)` | EVERY write to a sub-account's `.ssh/authorized_keys` and every repo move-aside | **The only writer of that file, and the only deleter on a sub-account (`DeleteSetAside`: `.orphaned-*` only, decision 74).** `read()` is read-only (R-827). Uses the provider's port-23 restricted shell (`dd of=` takes stdin, `mv` overwrites, `test` does NOT exist — measured); an unpinned line is a deletion route and is dropped on every install; the window line goes FIRST (first match wins). Never `rm`. | | `offsitekeys.Service` (`RegisterKey`, `ConfirmKey`, `AuditAll`, `OpenWindowFor`, `CloseWindowFor`, `SweepExpiredWindows`) | hub/internal/offsitekeys/service.go | — | Binding the registrar to the store, descriptor and operator events | The box-facing API (`/api/v1/offsite/register-key…`) answers with NO credential — pinned by `TestOffsiteKeyEndpoints_AuthAndNoPasswordInAnyResponse`. | | `(*Store).SaveOneTimeSecret` / `OffsitePassword` / `SealLegacyOffsiteSecrets` | hub/internal/store/offsite_seal.go | — | Storing / reading the sub-account password | **Sealed AES-256-GCM; no key → refused (fail-closed).** Under `go test` every store gets a fixed key (`testing.Testing()`); production needs `OFFSITE_SECRET_KEY`. Never serve the value to a box. | diff --git a/documentation/audits/offsite-finish-2026-10-04/red-proofs-controller.txt b/documentation/audits/offsite-finish-2026-10-04/red-proofs-controller.txt new file mode 100644 index 00000000..e28af1c3 --- /dev/null +++ b/documentation/audits/offsite-finish-2026-10-04/red-proofs-controller.txt @@ -0,0 +1,17 @@ +## RPC1: the v0.289 guard — a young removal always refuses (no same-day exclusion) +=== RUN TestOffsiteGuard_SameDaySupersededYoungExcluded + offbox_window_test.go:222: the window must run, not refuse: [{ID:2 CountBefore:3 CountAfter:3 Removed:0 Outcome:guard-refused Reason:the policy would remove snapshot night from 2026-10-03T02:00:00Z — younger than 8 days and not superseded the same day, which honest retention never does}] +--- FAIL: TestOffsiteGuard_SameDaySupersededYoungExcluded (0.00s) + +## RPC2: the v0.289 guard cap (take the oldest, never refuse) +=== RUN TestOffsiteGuard_AboveWeeklyCapRefused + offbox_window_test.go:201: ids=[o5-full o4-full o3-full o2-full o1-full o0-full] why="" +--- FAIL: TestOffsiteGuard_AboveWeeklyCapRefused (0.00s) + +## RPC3: a recovery that does not cancel at the hub +=== RUN TestAbandon_PinnedHandsToHubAndFollows + offbox_window_test.go:334: cancels=0 status={Active:false StartedAt:0001-01-01 00:00:00 +0000 UTC DueAt:0001-01-01 00:00:00 +0000 UTC DaysLeft:0 RepoPath:/home/felhom-repo.orphaned-20260901 PurgeRequested:false HubPending:false HubDueAt:0001-01-01 00:00:00 +0000 UTC RetrievalStillOffered:false} +--- FAIL: TestAbandon_PinnedHandsToHubAndFollows (0.00s) + +## restored: +ok gitea.dooplex.hu/admin/felhom-controller/internal/backup 0.020s diff --git a/documentation/audits/offsite-finish-2026-10-04/red-proofs-hub.txt b/documentation/audits/offsite-finish-2026-10-04/red-proofs-hub.txt new file mode 100644 index 00000000..80a17b6f --- /dev/null +++ b/documentation/audits/offsite-finish-2026-10-04/red-proofs-hub.txt @@ -0,0 +1,22 @@ +## RPH1: the sweep ignores the delay +=== RUN TestAbandon_DelayCancelAndScope + service_test.go:152: deleted BEFORE the delay +--- FAIL: TestAbandon_DelayCancelAndScope (0.03s) + +## RPH2: the sweep ignores a cancel +=== RUN TestAbandon_DelayCancelAndScope + service_test.go:162: a CANCELLED request deleted the copy +--- FAIL: TestAbandon_DelayCancelAndScope (0.03s) + +## RPH3: the audit's read creates .ssh again (v0.127.0) +=== RUN TestAudit_DoesNotCreateSSHDir + service_test.go:208: the audit wrote: "mkdir .ssh" +--- FAIL: TestAudit_DoesNotCreateSSHDir (0.00s) + +## restored: +ok gitea.dooplex.hu/admin/felhom-hub/internal/offsitekeys 0.137s + +## RPH4: the v0.127.0 cap (40 %) against an honest week +=== RUN TestMaxRemove_HonestWeekFits + service_test.go:217: MaxRemove(153) = 61 < 63 — every honest window would be refused +--- FAIL: TestMaxRemove_HonestWeekFits (0.00s) diff --git a/hub/CHANGELOG.md b/hub/CHANGELOG.md index 5116526c..baceeb57 100644 --- a/hub/CHANGELOG.md +++ b/hub/CHANGELOG.md @@ -1,3 +1,20 @@ +## v0.128.0 — the household's set-aside deletion through the hub with a 7-day wait (decision 74, R-823); key-file clean-up (decision 72, R-826); the key check is read-only (R-827); the window cap fits an honest week (2026-10-04) + +- **Set-aside deletion (R-823).** The box (controller ≥ 0.290.0) hands a due "delete my earlier off-site backups" to + `POST /api/v1/offsite/abandon-request/` (`abandon-status`, `abandon-cancel` beside it). The hub records it and + deletes ONLY `.orphaned-<…>` (never the live repository; the path must exist) **7 days after the request**, + unless the household (a recovery on the box cancels) or the operator (`POST /offsite/abandon-cancel/`) cancels. + Sweep every minute; `offsite_abandon_requested` / `_cancelled` / `_deleted` / `_failed`, operator-only. + `OFFSITE_ABANDON_DELAY` overrides the wait for a TEST only and is logged as such at start-up. +- **Key-file clean-up (R-826):** `POST /offsite/remove-unpinned/` (operator) rewrites a sub-account's + `authorized_keys` keeping only pinned lines; an empty file is allowed. +- **R-827:** the daily key check no longer creates `.ssh` on a sub-account that has none — only the write path does. +- **Window cap:** `MaxRemove` is half the count (≥ 5), was 40 %: an honest week removes ~41 % (7 of ~17 per app), and + controller v0.290.0 refuses a plan above the cap. Same line as R-431's detector. +- Tests: `TestAbandon_DelayCancelAndScope`, `TestRemoveUnpinned_KeepsOnlyPinned`, `TestAudit_DoesNotCreateSSHDir`, + `TestMaxRemove_HonestWeekFits`, `TestWindow_LeftOpenIsClosedByTheSweep` — red-proofs in + `documentation/audits/offsite-finish-2026-10-04/red-proofs-hub.txt`. + ## v0.127.0 — off-site keys a box cannot use to delete: the hub is the key registrar, the storage password is sealed and never served, a daily key check, the clean-up window (decisions 68–69, R-820, R-821, R-822) (2026-10-03) - **The box never receives the Storage Box sub-account password again (R-820).** `POST /api/v1/offsite/consume-password/` diff --git a/hub/cmd/hub/main.go b/hub/cmd/hub/main.go index a6fbbeaf..09deb59a 100644 --- a/hub/cmd/hub/main.go +++ b/hub/cmd/hub/main.go @@ -391,6 +391,17 @@ func main() { Store: dataStore, Reg: &offsitekeys.Registrar{Dialer: offsitekeys.SSHDialer{}}, Logger: logger, Emit: dispatcher.ProcessEvent, } + // Decision 74: the hub-enforced wait before a set-aside copy is deleted. OFFSITE_ABANDON_DELAY + // overrides the 7-day default ONLY for a test, and is logged loudly when it does. + keySvc.AbandonDelay = offsitekeys.DefaultAbandonDelay + if v := os.Getenv("OFFSITE_ABANDON_DELAY"); v != "" { + if d, derr := time.ParseDuration(v); derr == nil && d > 0 { + keySvc.AbandonDelay = d + logger.Printf("[WARN] OFFSITE_ABANDON_DELAY=%s — set-aside deletions wait %s instead of 7 days (TEST CONFIGURATION)", v, d) + } else { + logger.Printf("[ERROR] OFFSITE_ABANDON_DELAY=%q invalid — keeping 7 days", v) + } + } apiHandler.SetOffsiteKeyService(keySvc) runKeyAudit := func(ctx context.Context) any { start := time.Now() @@ -415,6 +426,7 @@ func main() { } webServer.SetOffsiteKeyAudit(runKeyAudit) webServer.SetOffsiteWindowAdmin(dataStore.GrantOffsiteWindowOnce, dataStore.SetOffsiteWindowsEnabled) + webServer.SetOffsiteKeyAdmin(keySvc.RemoveUnpinnedKeys, keySvc.CancelAbandon) // Decision 68: a window the box never closed is closed by the hub at its 20-minute bound. go func() { tk := time.NewTicker(60 * time.Second) @@ -426,6 +438,7 @@ func main() { case <-tk.C: sctx, cancel := context.WithTimeout(ctx, 2*time.Minute) keySvc.SweepExpiredWindows(sctx) + keySvc.SweepAbandons(sctx) cancel() } } diff --git a/hub/internal/api/handler.go b/hub/internal/api/handler.go index f5c3d4d5..d2338821 100644 --- a/hub/internal/api/handler.go +++ b/hub/internal/api/handler.go @@ -366,6 +366,12 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) { h.handleOffsiteWindowOpen(w, r, strings.TrimPrefix(path, "/offsite/window-open/")) case r.Method == http.MethodPost && strings.HasPrefix(path, "/offsite/window-close/"): h.handleOffsiteWindowClose(w, r, strings.TrimPrefix(path, "/offsite/window-close/")) + case r.Method == http.MethodPost && strings.HasPrefix(path, "/offsite/abandon-request/"): + h.handleOffsiteAbandon(w, r, "request", strings.TrimPrefix(path, "/offsite/abandon-request/")) + case r.Method == http.MethodPost && strings.HasPrefix(path, "/offsite/abandon-cancel/"): + h.handleOffsiteAbandon(w, r, "cancel", strings.TrimPrefix(path, "/offsite/abandon-cancel/")) + case r.Method == http.MethodPost && strings.HasPrefix(path, "/offsite/abandon-status/"): + h.handleOffsiteAbandon(w, r, "status", strings.TrimPrefix(path, "/offsite/abandon-status/")) case r.Method == http.MethodGet && strings.HasPrefix(path, "/artifacts/"): customerID := strings.TrimPrefix(path, "/artifacts/") h.handleArtifactManifest(w, r, customerID) diff --git a/hub/internal/api/offsite.go b/hub/internal/api/offsite.go index f6db1ec7..312ac10b 100644 --- a/hub/internal/api/offsite.go +++ b/hub/internal/api/offsite.go @@ -18,6 +18,9 @@ type OffsiteKeyService interface { MoveAside(ctx context.Context, customerID string) (string, error) OpenWindowFor(ctx context.Context, customerID string, countBefore int) (offsitekeys.WindowGrant, error) CloseWindowFor(ctx context.Context, customerID string, r offsitekeys.WindowResult) error + RequestAbandon(ctx context.Context, customerID, path string) (offsitekeys.AbandonStatus, error) + CancelAbandon(customerID, by string) (int, error) + AbandonStatusFor(customerID string) (offsitekeys.AbandonStatus, error) } // SetOffsiteKeyService wires the key registrar. @@ -166,3 +169,47 @@ func (h *Handler) handleOffsiteWindowClose(w http.ResponseWriter, r *http.Reques } writeJSON(w, http.StatusOK, map[string]any{"closed": true}) } + +// handleOffsiteAbandon: the box's half of decision 74 (R-823). +// +// POST /offsite/abandon-request/ {"path": ".orphaned-…"} → recorded; deleted by the hub after the delay +// POST /offsite/abandon-cancel/ → every pending request cancelled +// POST /offsite/abandon-status/ → {"state": none|pending|cancelled|deleted, …} +func (h *Handler) handleOffsiteAbandon(w http.ResponseWriter, r *http.Request, verb, customerID string) { + if !h.offsiteKeyAuth(w, r, customerID) { + return + } + ctx, cancel := context.WithTimeout(r.Context(), 2*time.Minute) + defer cancel() + switch verb { + case "request": + var req struct { + Path string `json:"path"` + } + body, _ := io.ReadAll(io.LimitReader(r.Body, 4<<10)) + if err := json.Unmarshal(body, &req); err != nil || req.Path == "" { + http.Error(w, "body must be {\"path\": \"…\"}", http.StatusBadRequest) + return + } + st, err := h.offsiteKeys.RequestAbandon(ctx, customerID, req.Path) + if err != nil { + offsiteKeyErr(w, err) + return + } + writeJSON(w, http.StatusOK, st) + case "cancel": + n, err := h.offsiteKeys.CancelAbandon(customerID, "box") + if err != nil { + offsiteKeyErr(w, err) + return + } + writeJSON(w, http.StatusOK, map[string]any{"cancelled": n}) + default: + st, err := h.offsiteKeys.AbandonStatusFor(customerID) + if err != nil { + offsiteKeyErr(w, err) + return + } + writeJSON(w, http.StatusOK, st) + } +} diff --git a/hub/internal/api/offsite_test.go b/hub/internal/api/offsite_test.go index 6da142fd..1c914803 100644 --- a/hub/internal/api/offsite_test.go +++ b/hub/internal/api/offsite_test.go @@ -69,6 +69,13 @@ func (f *fakeKeySvc) OpenWindowFor(context.Context, string, int) (offsitekeys.Wi return offsitekeys.WindowGrant{Granted: false, Reason: "not due"}, f.err } func (f *fakeKeySvc) CloseWindowFor(context.Context, string, offsitekeys.WindowResult) error { return f.err } +func (f *fakeKeySvc) RequestAbandon(context.Context, string, string) (offsitekeys.AbandonStatus, error) { + return offsitekeys.AbandonStatus{State: "pending"}, f.err +} +func (f *fakeKeySvc) CancelAbandon(string, string) (int, error) { return 1, f.err } +func (f *fakeKeySvc) AbandonStatusFor(string) (offsitekeys.AbandonStatus, error) { + return offsitekeys.AbandonStatus{State: "none"}, f.err +} func (f *fakeKeySvc) MoveAside(context.Context, string) (string, error) { return "/home/felhom-repo.orphaned-20261003", f.err } @@ -108,6 +115,9 @@ func TestOffsiteKeyEndpoints_AuthAndNoPasswordInAnyResponse(t *testing.T) { {"/api/v1/offsite/move-aside/c1", ``}, {"/api/v1/offsite/window-open/c1", `{"count_before":3}`}, {"/api/v1/offsite/window-close/c1", `{"window_id":1,"count_after":3,"outcome":"nothing"}`}, + {"/api/v1/offsite/abandon-request/c1", `{"path":"/home/felhom-repo.orphaned-20261004"}`}, + {"/api/v1/offsite/abandon-cancel/c1", ``}, + {"/api/v1/offsite/abandon-status/c1", ``}, } { rr := post(c.path, "ckey", c.body) if rr.Code != http.StatusOK { diff --git a/hub/internal/offsitekeys/offsitekeys.go b/hub/internal/offsitekeys/offsitekeys.go index fcf82c03..25b72245 100644 --- a/hub/internal/offsitekeys/offsitekeys.go +++ b/hub/internal/offsitekeys/offsitekeys.go @@ -130,11 +130,9 @@ func (r *Registrar) open(ctx context.Context, t Target, password string) (Shell, // read returns the current authorized_keys content; a missing file is "" (cat exits 1 there). func read(ctx context.Context, sh Shell) (string, error) { + // READ-ONLY (R-827, v0.128.0): a missing .ssh or file reads as "" — the directory is created only by + // write(). Until v0.127.0 the daily check created .ssh on a sub-account that had none. if _, err := sh.Run(ctx, "ls -d .ssh", nil); err != nil { - if _, merr := sh.Run(ctx, "mkdir .ssh", nil); merr != nil { - return "", fmt.Errorf("offsitekeys: create .ssh: %w", merr) - } - _, _ = sh.Run(ctx, "chmod 700 .ssh", nil) return "", nil } if _, err := sh.Run(ctx, "ls "+authorizedKeys, nil); err != nil { @@ -149,6 +147,12 @@ func read(ctx context.Context, sh Shell) (string, error) { // write replaces authorized_keys atomically (dd to a temp file, chmod, mv) and reads it back. func write(ctx context.Context, sh Shell, content string) error { + if _, err := sh.Run(ctx, "ls -d .ssh", nil); err != nil { + if _, merr := sh.Run(ctx, "mkdir .ssh", nil); merr != nil { + return fmt.Errorf("offsitekeys: create .ssh: %w", merr) + } + _, _ = sh.Run(ctx, "chmod 700 .ssh", nil) + } if _, err := sh.Run(ctx, "dd of="+tmpKeys, []byte(content)); err != nil { return fmt.Errorf("offsitekeys: write temp file: %w", err) } @@ -384,3 +388,70 @@ func (r *Registrar) MoveAside(ctx context.Context, t Target, password, date stri } return name, nil } + +// RemoveUnpinned rewrites authorized_keys keeping only the PINNED lines (decision 72, R-826): every +// unpinned line — a route to deletion — and any window line (when no window is open) goes. An empty +// result is allowed (no box). Returns how many lines were removed; a file with nothing to remove is not +// rewritten. +func (r *Registrar) RemoveUnpinned(ctx context.Context, t Target, password string, windowOpen bool) (int, error) { + sh, err := r.open(ctx, t, password) + if err != nil { + return 0, err + } + defer sh.Close() + cur, err := read(ctx, sh) + if err != nil { + return 0, err + } + var keep []string + removed := 0 + for _, l := range ParseLines(cur, t.RepoPath) { + if l.Pinned || (l.Window && windowOpen) { + keep = append(keep, l.Raw) + continue + } + removed++ + } + if removed == 0 { + return 0, nil + } + return removed, write(ctx, sh, join(keep)) +} + +// DeleteSetAside removes one SET-ASIDE repository directory (decision 74, R-823) — the only deletion the +// registrar performs. It refuses anything that is not `.orphaned-<…>` (never the live +// repository, never a path with a slash or "..") and anything that does not exist. +func (r *Registrar) DeleteSetAside(ctx context.Context, t Target, password, path string) error { + if !IsSetAsidePath(t.RepoPath, path) { + return fmt.Errorf("offsitekeys: %q is not a set-aside copy of %s — refusing", path, t.RepoPath) + } + sh, err := r.open(ctx, t, password) + if err != nil { + return err + } + defer sh.Close() + if _, err := sh.Run(ctx, "ls -d "+path, nil); err != nil { + return fmt.Errorf("offsitekeys: set-aside copy %s not found: %w", path, err) + } + if _, err := sh.Run(ctx, "rm -rf "+path, nil); err != nil { + return fmt.Errorf("offsitekeys: delete %s: %w", path, err) + } + if _, err := sh.Run(ctx, "ls -d "+path, nil); err == nil { + return fmt.Errorf("offsitekeys: %s still exists after the delete", path) + } + return nil +} + +// IsSetAsidePath: `.orphaned-` with a suffix of [A-Za-z0-9-] only. +func IsSetAsidePath(repo, path string) bool { + pre := repo + ".orphaned-" + if repo == "" || !strings.HasPrefix(path, pre) || len(path) == len(pre) { + return false + } + for _, c := range path[len(pre):] { + if !(c == '-' || (c >= '0' && c <= '9') || (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z')) { + return false + } + } + return true +} diff --git a/hub/internal/offsitekeys/offsitekeys_test.go b/hub/internal/offsitekeys/offsitekeys_test.go index 7a23e5fe..7288c75b 100644 --- a/hub/internal/offsitekeys/offsitekeys_test.go +++ b/hub/internal/offsitekeys/offsitekeys_test.go @@ -61,8 +61,11 @@ func (f *fakeFS) Run(_ context.Context, cmd string, stdin []byte) ([]byte, error return nil, nil } return nil, miss + case a[0] == "rm" && a[1] == "-rf" && strings.Contains(a[2], ".orphaned-"): + delete(f.dirs, a[2]) // decision 74: the ONLY delete, of a set-aside copy + return nil, nil case a[0] == "rm": - return nil, errors.New("the registrar must never delete") + return nil, errors.New("the registrar must never delete anything else") } return nil, errors.New("Command not found") } diff --git a/hub/internal/offsitekeys/service.go b/hub/internal/offsitekeys/service.go index 38e85de2..9b83f1e2 100644 --- a/hub/internal/offsitekeys/service.go +++ b/hub/internal/offsitekeys/service.go @@ -30,6 +30,8 @@ type Service struct { // Emit routes an event to the dispatcher (operator mail). nil → events are only saved. Emit func(customerID, eventType, severity, message, detailsJSON, source string) Now func() time.Time + // AbandonDelay is the hub-enforced wait before a set-aside copy is deleted (decision 74). 0 → 7 days. + AbandonDelay time.Duration } // ErrNotProvisioned — the customer has no provisioned off-site target (nothing to register against). @@ -229,11 +231,12 @@ type WindowResult struct { Reason string `json:"reason"` } -// MaxRemove is the most snapshots one window may remove: 40 % of what was there, at least 5. The ruled -// policy (7 daily + 4 weekly + 6 monthly per app) removes ~7 of ~17 per app per week (~41 %); the box -// takes the OLDEST first within this bound, so a backlog drains over several windows. +// MaxRemove is the most snapshots one window may remove: HALF of what was there, at least 5. The ruled +// policy (7 daily + 4 weekly + 6 monthly per app) removes ~7 of ~17 per app per week (~41 %) — v0.127.0's +// 40 % would have refused every honest week once the box refuses above the cap (controller v0.290.0). +// Half is also the line R-431's detector draws for "an unexplained fall". Pinned by TestMaxRemove_*. func MaxRemove(countBefore int) int { - n := countBefore * 40 / 100 + n := countBefore / 2 if n < 5 { n = 5 } @@ -333,3 +336,139 @@ func (s *Service) SweepExpiredWindows(ctx context.Context) { fmt.Sprintf("Off-site clean-up window %d was not closed by the box within %s; the hub closed it (the deleting key line is removed).", w.ID, windowLength), nil) } } + +// ── Decision 74 (R-823): the household's "delete my set-aside history" — done by the HUB, after a delay ── +// +// The box's key cannot delete (decision 69), so a due abandonment is a REQUEST to the hub. The hub waits +// AbandonDelay (default 7 days) from the request — the household (via the box's recovery, which cancels) +// and the operator can cancel in that time — then deletes ONLY `.orphaned-<…>`, never the live +// repository. A broken-into box can therefore make a set-aside copy disappear only after a week of +// operator mails. Every request, cancel and deletion is an operator event. + +const ( + EventAbandonRequested = "offsite_abandon_requested" + EventAbandonCancelled = "offsite_abandon_cancelled" + EventAbandonDeleted = "offsite_abandon_deleted" + EventAbandonFailed = "offsite_abandon_failed" + DefaultAbandonDelay = 7 * 24 * time.Hour +) + +func (s *Service) abandonDelay() time.Duration { + if s.AbandonDelay > 0 { + return s.AbandonDelay + } + return DefaultAbandonDelay +} + +// AbandonStatus is what the box (and the operator) sees. +type AbandonStatus struct { + State string `json:"state"` // none | pending | cancelled | deleted + Path string `json:"path,omitempty"` + DueAt time.Time `json:"due_at,omitempty"` +} + +func statusOf(a *store.OffsiteAbandon) AbandonStatus { + if a == nil { + return AbandonStatus{State: "none"} + } + return AbandonStatus{State: a.State(), Path: a.Path, DueAt: a.DueAt.UTC()} +} + +// RequestAbandon records (idempotently) the household's request to delete path, due after the delay. +func (s *Service) RequestAbandon(ctx context.Context, customerID, path string) (AbandonStatus, error) { + t, pw, err := s.TargetFor(customerID) + if err != nil { + return AbandonStatus{}, err + } + if !IsSetAsidePath(t.RepoPath, path) { + return AbandonStatus{}, fmt.Errorf("offsitekeys: %q is not a set-aside copy of %s", path, t.RepoPath) + } + if cur, err := s.Store.LatestOffsiteAbandon(customerID, path); err == nil && cur != nil && cur.State() == "pending" { + return statusOf(cur), nil // already requested — the clock is NOT restarted + } + // The copy must exist now (a request for nothing is refused, so a typo cannot sit armed for a week). + sh, err := s.Reg.open(ctx, t, pw) + if err != nil { + return AbandonStatus{}, err + } + _, lerr := sh.Run(ctx, "ls -d "+path, nil) + sh.Close() + if lerr != nil { + return AbandonStatus{}, fmt.Errorf("offsitekeys: set-aside copy %s not found", path) + } + due := s.now().Add(s.abandonDelay()) + if _, err := s.Store.CreateOffsiteAbandon(customerID, path, due); err != nil { + return AbandonStatus{}, err + } + s.logf("[WARN] offsitekeys: %s asked to DELETE its set-aside off-site copy %s — the hub deletes it at %s unless cancelled (delay %s)", + customerID, path, due.UTC().Format(time.RFC3339), s.abandonDelay()) + s.event(customerID, EventAbandonRequested, "warning", + fmt.Sprintf("Off-site: the box asked to delete the set-aside copy %s (the household's choice). The hub deletes it on %s unless the household or the operator cancels.", path, due.UTC().Format("2006-01-02 15:04 UTC")), + map[string]any{"path": path, "due_at": due.UTC()}) + a, _ := s.Store.LatestOffsiteAbandon(customerID, path) + return statusOf(a), nil +} + +// CancelAbandon cancels every pending request of the customer (by = "box" | "operator"). +func (s *Service) CancelAbandon(customerID, by string) (int, error) { + n, err := s.Store.CancelOffsiteAbandon(customerID, by) + if err != nil || n == 0 { + return n, err + } + s.logf("[INFO] offsitekeys: %s's set-aside deletion CANCELLED by %s (%d request(s)) — nothing deleted", customerID, by, n) + s.event(customerID, EventAbandonCancelled, "info", + fmt.Sprintf("Off-site: the pending deletion of the set-aside copy was cancelled by the %s. Nothing was deleted.", by), nil) + return n, nil +} + +// AbandonStatusFor returns the latest request's state. +func (s *Service) AbandonStatusFor(customerID string) (AbandonStatus, error) { + a, err := s.Store.LatestOffsiteAbandon(customerID, "") + if err != nil { + return AbandonStatus{}, err + } + return statusOf(a), nil +} + +// SweepAbandons deletes every request that is due, not cancelled and not yet deleted. +func (s *Service) SweepAbandons(ctx context.Context) { + due, err := s.Store.DueOffsiteAbandons() + if err != nil { + s.logf("[WARN] offsitekeys: abandon sweep: %v", err) + return + } + for _, a := range due { + t, pw, terr := s.TargetFor(a.CustomerID) + if terr == nil { + terr = s.Reg.DeleteSetAside(ctx, t, pw, a.Path) + } + if terr != nil { + _ = s.Store.MarkOffsiteAbandonError(a.ID, terr.Error()) + s.logf("[ERROR] offsitekeys: deleting %s's set-aside copy %s failed (retrying): %v", a.CustomerID, a.Path, terr) + if a.LastError == "" { + s.event(a.CustomerID, EventAbandonFailed, "warning", + fmt.Sprintf("Off-site: deleting the set-aside copy %s failed and is retried: %v", a.Path, terr), nil) + } + continue + } + _ = s.Store.MarkOffsiteAbandonDeleted(a.ID) + s.logf("[WARN] offsitekeys: DELETED %s's set-aside off-site copy %s (requested %s, due %s)", a.CustomerID, a.Path, + a.RequestedAt.UTC().Format(time.RFC3339), a.DueAt.UTC().Format(time.RFC3339)) + s.event(a.CustomerID, EventAbandonDeleted, "info", + fmt.Sprintf("Off-site: the set-aside copy %s was deleted, as the household chose (requested %s).", a.Path, a.RequestedAt.UTC().Format("2006-01-02")), nil) + } +} + +// RemoveUnpinnedKeys is the operator's clean-up of a sub-account's key file (decision 72). +func (s *Service) RemoveUnpinnedKeys(ctx context.Context, customerID string) (int, error) { + t, pw, err := s.TargetFor(customerID) + if err != nil { + return 0, err + } + n, err := s.Reg.RemoveUnpinned(ctx, t, pw, s.Store.OffsiteWindowOpen(customerID)) + if err != nil { + return 0, err + } + s.logf("[INFO] offsitekeys: removed %d unpinned key line(s) from %s's sub-account (%s) on the operator's request", n, customerID, t.User) + return n, nil +} diff --git a/hub/internal/offsitekeys/service_test.go b/hub/internal/offsitekeys/service_test.go index 14e2de03..e885995c 100644 --- a/hub/internal/offsitekeys/service_test.go +++ b/hub/internal/offsitekeys/service_test.go @@ -2,6 +2,7 @@ package offsitekeys import ( "context" + "strings" "log" "os" "path/filepath" @@ -50,7 +51,7 @@ func TestWindow_GrantOpenCloseAndDropAlarm(t *testing.T) { t.Fatal(err) } g, err := s.OpenWindowFor(ctx, "c1", 20) - if err != nil || !g.Granted || g.MaxRemove != 8 { + if err != nil || !g.Granted || g.MaxRemove != 10 { t.Fatalf("one-shot: %+v %v", g, err) } if lines := ParseLines(fs.files[".ssh/authorized_keys"], "/home/felhom-repo"); !lines[0].Window || lines[0].Fingerprint != fp { @@ -62,7 +63,7 @@ func TestWindow_GrantOpenCloseAndDropAlarm(t *testing.T) { if g2, _ := s.OpenWindowFor(ctx, "c1", 20); g2.Granted { t.Fatal("the one-shot grant was not consumed") } - // The box reports a fall of 12 (allowed 8) → offsite_window_drop. + // The box reports a fall of 12 (allowed 10) → offsite_window_drop. if err := s.CloseWindowFor(ctx, "c1", WindowResult{WindowID: g.WindowID, CountAfter: 8, Outcome: "pruned"}); err != nil { t.Fatal(err) } @@ -127,3 +128,95 @@ func TestWindow_LeftOpenIsClosedByTheSweep(t *testing.T) { t.Fatalf("last event = %s", last) } } + +// Decision 74 (R-823): a set-aside deletion is NOT acted on before the delay, a cancelled request deletes +// nothing, and the live repository can never be named. +func TestAbandon_DelayCancelAndScope(t *testing.T) { + s, fs, events := svcFixture(t) + ctx := context.Background() + aside := "/home/felhom-repo.orphaned-20261004" + fs.dirs[aside] = true + fs.dirs["/home/felhom-repo"] = true + for _, bad := range []string{"/home/felhom-repo", "/home/felhom-repo.orphaned-../x", "/home/other.orphaned-1"} { + if _, err := s.RequestAbandon(ctx, "c1", bad); err == nil { + t.Fatalf("accepted a non-set-aside path %q", bad) + } + } + st, err := s.RequestAbandon(ctx, "c1", aside) + if err != nil || st.State != "pending" { + t.Fatalf("%+v %v", st, err) + } + // Not due yet (7-day default): the sweep deletes nothing. + s.SweepAbandons(ctx) + if !fs.dirs[aside] { + t.Fatal("deleted BEFORE the delay") + } + // Cancelled, then made due: still nothing deleted. + if n, _ := s.CancelAbandon("c1", "operator"); n != 1 { + t.Fatalf("cancelled %d", n) + } + a, _ := s.Store.LatestOffsiteAbandon("c1", aside) + _ = s.Store.ForceOffsiteAbandonDueForTest(a.ID) + s.SweepAbandons(ctx) + if !fs.dirs[aside] { + t.Fatal("a CANCELLED request deleted the copy") + } + // A fresh request, due: deleted, and only that directory. + if _, err := s.RequestAbandon(ctx, "c1", aside); err != nil { + t.Fatal(err) + } + a, _ = s.Store.LatestOffsiteAbandon("c1", aside) + _ = s.Store.ForceOffsiteAbandonDueForTest(a.ID) + s.SweepAbandons(ctx) + if fs.dirs[aside] || !fs.dirs["/home/felhom-repo"] { + t.Fatalf("after the due sweep: %v", fs.dirs) + } + if st, _ := s.AbandonStatusFor("c1"); st.State != "deleted" { + t.Fatalf("state = %s", st.State) + } + last := (*events)[len(*events)-1] + if last != EventAbandonDeleted { + t.Fatalf("last event %s", last) + } +} + +// Decision 72 (R-826): the operator clean-up keeps pinned lines and drops the rest; an empty file is fine. +func TestRemoveUnpinned_KeepsOnlyPinned(t *testing.T) { + s, fs, _ := svcFixture(t) + a, _ := newKey(t) + b, _ := newKey(t) + fs.files[".ssh/authorized_keys"] = a + "\n" + b + "\n" + n, err := s.RemoveUnpinnedKeys(context.Background(), "c1") + if err != nil || n != 2 || fs.files[".ssh/authorized_keys"] != "" { + t.Fatalf("n=%d err=%v file=%q", n, err, fs.files[".ssh/authorized_keys"]) + } + if n, _ := s.RemoveUnpinnedKeys(context.Background(), "c1"); n != 0 { + t.Fatal("second run changed something") + } +} + +// R-827: the daily check is read-only — no .ssh is created on a sub-account that has none. +func TestAudit_DoesNotCreateSSHDir(t *testing.T) { + fs := newFS() + delete(fs.dirs, ".ssh") + r := &Registrar{Dialer: fakeDialer{fs}} + if _, err := r.Audit(context.Background(), tgt, "pw", false); err != nil { + t.Fatal(err) + } + for _, c := range fs.cmds { + if strings.HasPrefix(c, "mkdir") { + t.Fatalf("the audit wrote: %q", c) + } + } +} + +// The honest weekly removal (7 of ~17 per app, ~41 %) must fit under the cap — demo-hp's real shape: +// 9 apps × 17 = 153 snapshots, 63 removed in a week. +func TestMaxRemove_HonestWeekFits(t *testing.T) { + if MaxRemove(153) < 63 { + t.Fatalf("MaxRemove(153) = %d < 63 — every honest window would be refused", MaxRemove(153)) + } + if MaxRemove(4) != 5 { + t.Fatal("floor of 5 lost") + } +} diff --git a/hub/internal/store/offsite_keys.go b/hub/internal/store/offsite_keys.go index 2b1ed386..9eefd070 100644 --- a/hub/internal/store/offsite_keys.go +++ b/hub/internal/store/offsite_keys.go @@ -181,3 +181,119 @@ func (s *Store) ForceOffsiteWindowDueForTest(id int64) error { _, err := s.db.Exec(`UPDATE offsite_windows SET closes_by = datetime('now', '-1 minute') WHERE id = ?`, id) return err } + +// OffsiteAbandon is one household request to delete a set-aside off-site copy (decision 74, R-823). The +// hub deletes it only after DueAt, and only if nobody cancelled it. +type OffsiteAbandon struct { + ID int64 + CustomerID string + Path string + RequestedAt time.Time + DueAt time.Time + CancelledAt time.Time + CancelledBy string + DeletedAt time.Time + LastError string +} + +func (a *OffsiteAbandon) State() string { + switch { + case a == nil: + return "none" + case !a.DeletedAt.IsZero(): + return "deleted" + case !a.CancelledAt.IsZero(): + return "cancelled" + } + return "pending" +} + +const abandonCols = `id, customer_id, path, requested_at, due_at, cancelled_at, cancelled_by, deleted_at, last_error` + +func scanAbandon(sc interface{ Scan(...any) error }) (*OffsiteAbandon, error) { + var a OffsiteAbandon + var req, due string + var canc, by, del, lerr sql.NullString + if err := sc.Scan(&a.ID, &a.CustomerID, &a.Path, &req, &due, &canc, &by, &del, &lerr); err != nil { + return nil, err + } + a.RequestedAt, a.DueAt = parseSQLiteTime(req), parseSQLiteTime(due) + if canc.Valid { + a.CancelledAt = parseSQLiteTime(canc.String) + } + if del.Valid { + a.DeletedAt = parseSQLiteTime(del.String) + } + a.CancelledBy, a.LastError = by.String, lerr.String + return &a, nil +} + +// LatestOffsiteAbandon returns the customer's most recent request for path ("" = any), or (nil, nil). +func (s *Store) LatestOffsiteAbandon(customerID, path string) (*OffsiteAbandon, error) { + q := `SELECT ` + abandonCols + ` FROM offsite_abandon_requests WHERE customer_id = ?` + args := []any{customerID} + if path != "" { + q += ` AND path = ?` + args = append(args, path) + } + a, err := scanAbandon(s.db.QueryRow(q+` ORDER BY id DESC LIMIT 1`, args...)) + if err == sql.ErrNoRows { + return nil, nil + } + return a, err +} + +// CreateOffsiteAbandon records a request due at dueAt. +func (s *Store) CreateOffsiteAbandon(customerID, path string, dueAt time.Time) (int64, error) { + res, err := s.db.Exec(`INSERT INTO offsite_abandon_requests (customer_id, path, requested_at, due_at) VALUES (?, ?, datetime('now'), ?)`, + customerID, path, dueAt.UTC().Format("2006-01-02 15:04:05")) + if err != nil { + return 0, err + } + return res.LastInsertId() +} + +// CancelOffsiteAbandon cancels every PENDING request of the customer; returns how many. +func (s *Store) CancelOffsiteAbandon(customerID, by string) (int, error) { + res, err := s.db.Exec(`UPDATE offsite_abandon_requests SET cancelled_at = datetime('now'), cancelled_by = ? WHERE customer_id = ? AND cancelled_at IS NULL AND deleted_at IS NULL`, by, customerID) + if err != nil { + return 0, err + } + n, _ := res.RowsAffected() + return int(n), nil +} + +// DueOffsiteAbandons lists pending requests whose due time has passed. +func (s *Store) DueOffsiteAbandons() ([]*OffsiteAbandon, error) { + rows, err := s.db.Query(`SELECT ` + abandonCols + ` FROM offsite_abandon_requests WHERE cancelled_at IS NULL AND deleted_at IS NULL AND due_at <= datetime('now')`) + if err != nil { + return nil, err + } + defer rows.Close() + var out []*OffsiteAbandon + for rows.Next() { + a, err := scanAbandon(rows) + if err != nil { + return nil, err + } + out = append(out, a) + } + return out, rows.Err() +} + +// MarkOffsiteAbandonDeleted / MarkOffsiteAbandonError record the sweep's outcome. +func (s *Store) MarkOffsiteAbandonDeleted(id int64) error { + _, err := s.db.Exec(`UPDATE offsite_abandon_requests SET deleted_at = datetime('now'), last_error = NULL WHERE id = ?`, id) + return err +} + +func (s *Store) MarkOffsiteAbandonError(id int64, msg string) error { + _, err := s.db.Exec(`UPDATE offsite_abandon_requests SET last_error = ? WHERE id = ?`, msg, id) + return err +} + +// ForceOffsiteAbandonDueForTest back-dates a request. TEST-ONLY. +func (s *Store) ForceOffsiteAbandonDueForTest(id int64) error { + _, err := s.db.Exec(`UPDATE offsite_abandon_requests SET due_at = datetime('now', '-1 minute') WHERE id = ?`, id) + return err +} diff --git a/hub/internal/store/store.go b/hub/internal/store/store.go index 5ee12b94..ad30ccd9 100644 --- a/hub/internal/store/store.go +++ b/hub/internal/store/store.go @@ -842,6 +842,17 @@ func (s *Store) migrate() error { close_reason TEXT ); CREATE INDEX IF NOT EXISTS idx_offsite_windows_customer ON offsite_windows(customer_id, opened_at); + CREATE TABLE IF NOT EXISTS offsite_abandon_requests ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + customer_id TEXT NOT NULL, + path TEXT NOT NULL, + requested_at DATETIME NOT NULL DEFAULT (datetime('now')), + due_at DATETIME NOT NULL, + cancelled_at DATETIME, + cancelled_by TEXT, + deleted_at DATETIME, + last_error TEXT + ); `); err != nil { return fmt.Errorf("offsite_keys/offsite_windows: %w", err) } diff --git a/hub/internal/web/server.go b/hub/internal/web/server.go index 3b28a7d1..88ac1b4a 100644 --- a/hub/internal/web/server.go +++ b/hub/internal/web/server.go @@ -72,14 +72,17 @@ type Server struct { // offsiteWindowAdmin: operator one-shot grant / weekly switch (decision 68). nil → 503. offsiteWindowGrant func(customerID string) error offsiteWindowSwitch func(on bool) error - offsiteBox func() (monitor.BoxSnapshot, bool) // optional (v0.64.0, R-5); the restic pool-box aggregate snapshot accessor - pbsdrBox func() (monitor.PBSBoxSnapshot, bool) // optional (v0.65.0, R-5); the PBS-DR datastore fill snapshot accessor - tenantsync tenancyProvisioner // optional; enables PBS DR tier provisioning (web/pbsdr.go) - claimEngine *claim.Engine // optional; enables the customer-claim resend button (v0.50.0) - selfBindMailer SelfBindMailer // optional; enables the customer self-bind link button (v0.66.0, R-27) - bindLimiter *bindRateLimiter // per-IP throttle for the PUBLIC /bind/ surface (v0.66.0, R-27) - bindResendMu sync.Mutex // R-719: the fresh-link resend limiter - bindResendAt map[string]time.Time // customer → last fresh-link mail (R-719) + // operator key-file clean-up (decision 72) and abandonment cancel (decision 74). nil → 503. + offsiteRemoveUnpinned func(ctx context.Context, customerID string) (int, error) + offsiteAbandonCancel func(customerID, by string) (int, error) + offsiteBox func() (monitor.BoxSnapshot, bool) // optional (v0.64.0, R-5); the restic pool-box aggregate snapshot accessor + pbsdrBox func() (monitor.PBSBoxSnapshot, bool) // optional (v0.65.0, R-5); the PBS-DR datastore fill snapshot accessor + tenantsync tenancyProvisioner // optional; enables PBS DR tier provisioning (web/pbsdr.go) + claimEngine *claim.Engine // optional; enables the customer-claim resend button (v0.50.0) + selfBindMailer SelfBindMailer // optional; enables the customer self-bind link button (v0.66.0, R-27) + bindLimiter *bindRateLimiter // per-IP throttle for the PUBLIC /bind/ surface (v0.66.0, R-27) + bindResendMu sync.Mutex // R-719: the fresh-link resend limiter + bindResendAt map[string]time.Time // customer → last fresh-link mail (R-719) // intentHub (v0.58.0, Direction-2 immediate-sync) is Bumped by every operator-intent handler // (config save/delete, claim resend, offsite re-issue/freeze, floor, block/unblock, log pull) // so a box long-polling GET /api/v1/wait wakes in seconds. Shared with the API handler. nil = @@ -202,6 +205,11 @@ func (s *Server) SetOffsiteProvisioner(p *offsite.Provisioner) { s.offsite = p } // SetOffsiteKeyAudit wires the on-demand run of the daily off-site key check (decision 69). func (s *Server) SetOffsiteKeyAudit(fn func(ctx context.Context) any) { s.offsiteKeyAudit = fn } +// SetOffsiteKeyAdmin wires the operator's key-file clean-up and abandonment cancel. +func (s *Server) SetOffsiteKeyAdmin(remove func(context.Context, string) (int, error), cancel func(string, string) (int, error)) { + s.offsiteRemoveUnpinned, s.offsiteAbandonCancel = remove, cancel +} + // SetOffsiteWindowAdmin wires the operator's window controls (decision 68). func (s *Server) SetOffsiteWindowAdmin(grant func(string) error, sw func(bool) error) { s.offsiteWindowGrant, s.offsiteWindowSwitch = grant, sw @@ -617,6 +625,26 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { } else { s.handleConfigEditForm(w, r, customerID) } + case strings.HasPrefix(path, "/offsite/remove-unpinned/") || strings.HasPrefix(path, "/offsite/abandon-cancel/"): + // Operator: rewrite a sub-account's key file keeping only pinned lines (decision 72); cancel a + // household's pending set-aside deletion (decision 74). + if r.Method != http.MethodPost || s.offsiteRemoveUnpinned == nil { + http.Error(w, "unavailable", http.StatusServiceUnavailable) + return + } + var n int + var err error + if strings.HasPrefix(path, "/offsite/remove-unpinned/") { + n, err = s.offsiteRemoveUnpinned(r.Context(), strings.TrimPrefix(path, "/offsite/remove-unpinned/")) + } else { + n, err = s.offsiteAbandonCancel(strings.TrimPrefix(path, "/offsite/abandon-cancel/"), "operator") + } + if err != nil { + http.Error(w, err.Error(), http.StatusBadGateway) + return + } + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(map[string]int{"changed": n}) case strings.HasPrefix(path, "/offsite/window-grant/") || path == "/offsite/windows-enabled": // Operator (decision 68): a one-shot grant lets the customer's NEXT window request through; // the switch turns the WEEKLY window on/off fleet-wide (off = the interim: nothing prunes).