hub v0.128.0: set-aside deletion through the hub after a 7-day wait (decision 74, R-823), key-file clean-up route (R-826), read-only key check (R-827), window cap = half
gates / gates (push) Successful in 29s

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
2026-10-04 07:05:21 +02:00
parent 697c2a7b10
commit d3c50b50f6
15 changed files with 613 additions and 20 deletions
+1 -1
View File
@@ -71,7 +71,7 @@
| Symbol | File | Short signature | Use for | Gotchas |
|---|---|---|---|---|
| `offsitekeys.Registrar` (`Install` / `Confirm` / `Audit` / `OpenWindow` / `CloseWindow` / `MoveAside`) | hub/internal/offsitekeys/offsitekeys.go | `(ctx, Target, password, …)` | EVERY write to a sub-account's `.ssh/authorized_keys` and every repo move-aside | **The only writer of that file.** Uses the provider's port-23 restricted shell (`dd of=` takes stdin, `mv` overwrites, `test` does NOT exist — measured); an unpinned line is a deletion route and is dropped on every install; the window line goes FIRST (first match wins). Never `rm`. |
| `offsitekeys.Registrar` (`Install` / `Confirm` / `Audit` / `OpenWindow` / `CloseWindow` / `MoveAside`) | hub/internal/offsitekeys/offsitekeys.go | `(ctx, Target, password, …)` | EVERY write to a sub-account's `.ssh/authorized_keys` and every repo move-aside | **The only writer of that file, and the only deleter on a sub-account (`DeleteSetAside`: `<repo>.orphaned-*` only, decision 74).** `read()` is read-only (R-827). Uses the provider's port-23 restricted shell (`dd of=` takes stdin, `mv` overwrites, `test` does NOT exist — measured); an unpinned line is a deletion route and is dropped on every install; the window line goes FIRST (first match wins). Never `rm`. |
| `offsitekeys.Service` (`RegisterKey`, `ConfirmKey`, `AuditAll`, `OpenWindowFor`, `CloseWindowFor`, `SweepExpiredWindows`) | hub/internal/offsitekeys/service.go | — | Binding the registrar to the store, descriptor and operator events | The box-facing API (`/api/v1/offsite/register-key…`) answers with NO credential — pinned by `TestOffsiteKeyEndpoints_AuthAndNoPasswordInAnyResponse`. |
| `(*Store).SaveOneTimeSecret` / `OffsitePassword` / `SealLegacyOffsiteSecrets` | hub/internal/store/offsite_seal.go | — | Storing / reading the sub-account password | **Sealed AES-256-GCM; no key → refused (fail-closed).** Under `go test` every store gets a fixed key (`testing.Testing()`); production needs `OFFSITE_SECRET_KEY`. Never serve the value to a box. |
@@ -0,0 +1,17 @@
## RPC1: the v0.289 guard — a young removal always refuses (no same-day exclusion)
=== RUN TestOffsiteGuard_SameDaySupersededYoungExcluded
offbox_window_test.go:222: the window must run, not refuse: [{ID:2 CountBefore:3 CountAfter:3 Removed:0 Outcome:guard-refused Reason:the policy would remove snapshot night from 2026-10-03T02:00:00Z — younger than 8 days and not superseded the same day, which honest retention never does}]
--- FAIL: TestOffsiteGuard_SameDaySupersededYoungExcluded (0.00s)
## RPC2: the v0.289 guard cap (take the oldest, never refuse)
=== RUN TestOffsiteGuard_AboveWeeklyCapRefused
offbox_window_test.go:201: ids=[o5-full o4-full o3-full o2-full o1-full o0-full] why=""
--- FAIL: TestOffsiteGuard_AboveWeeklyCapRefused (0.00s)
## RPC3: a recovery that does not cancel at the hub
=== RUN TestAbandon_PinnedHandsToHubAndFollows
offbox_window_test.go:334: cancels=0 status={Active:false StartedAt:0001-01-01 00:00:00 +0000 UTC DueAt:0001-01-01 00:00:00 +0000 UTC DaysLeft:0 RepoPath:/home/felhom-repo.orphaned-20260901 PurgeRequested:false HubPending:false HubDueAt:0001-01-01 00:00:00 +0000 UTC RetrievalStillOffered:false}
--- FAIL: TestAbandon_PinnedHandsToHubAndFollows (0.00s)
## restored:
ok gitea.dooplex.hu/admin/felhom-controller/internal/backup 0.020s
@@ -0,0 +1,22 @@
## RPH1: the sweep ignores the delay
=== RUN TestAbandon_DelayCancelAndScope
service_test.go:152: deleted BEFORE the delay
--- FAIL: TestAbandon_DelayCancelAndScope (0.03s)
## RPH2: the sweep ignores a cancel
=== RUN TestAbandon_DelayCancelAndScope
service_test.go:162: a CANCELLED request deleted the copy
--- FAIL: TestAbandon_DelayCancelAndScope (0.03s)
## RPH3: the audit's read creates .ssh again (v0.127.0)
=== RUN TestAudit_DoesNotCreateSSHDir
service_test.go:208: the audit wrote: "mkdir .ssh"
--- FAIL: TestAudit_DoesNotCreateSSHDir (0.00s)
## restored:
ok gitea.dooplex.hu/admin/felhom-hub/internal/offsitekeys 0.137s
## RPH4: the v0.127.0 cap (40 %) against an honest week
=== RUN TestMaxRemove_HonestWeekFits
service_test.go:217: MaxRemove(153) = 61 < 63 — every honest window would be refused
--- FAIL: TestMaxRemove_HonestWeekFits (0.00s)
+17
View File
@@ -1,3 +1,20 @@
## v0.128.0 — the household's set-aside deletion through the hub with a 7-day wait (decision 74, R-823); key-file clean-up (decision 72, R-826); the key check is read-only (R-827); the window cap fits an honest week (2026-10-04)
- **Set-aside deletion (R-823).** The box (controller ≥ 0.290.0) hands a due "delete my earlier off-site backups" to
`POST /api/v1/offsite/abandon-request/<id>` (`abandon-status`, `abandon-cancel` beside it). The hub records it and
deletes ONLY `<repo>.orphaned-<…>` (never the live repository; the path must exist) **7 days after the request**,
unless the household (a recovery on the box cancels) or the operator (`POST /offsite/abandon-cancel/<id>`) cancels.
Sweep every minute; `offsite_abandon_requested` / `_cancelled` / `_deleted` / `_failed`, operator-only.
`OFFSITE_ABANDON_DELAY` overrides the wait for a TEST only and is logged as such at start-up.
- **Key-file clean-up (R-826):** `POST /offsite/remove-unpinned/<id>` (operator) rewrites a sub-account's
`authorized_keys` keeping only pinned lines; an empty file is allowed.
- **R-827:** the daily key check no longer creates `.ssh` on a sub-account that has none — only the write path does.
- **Window cap:** `MaxRemove` is half the count (≥ 5), was 40 %: an honest week removes ~41 % (7 of ~17 per app), and
controller v0.290.0 refuses a plan above the cap. Same line as R-431's detector.
- Tests: `TestAbandon_DelayCancelAndScope`, `TestRemoveUnpinned_KeepsOnlyPinned`, `TestAudit_DoesNotCreateSSHDir`,
`TestMaxRemove_HonestWeekFits`, `TestWindow_LeftOpenIsClosedByTheSweep` — red-proofs in
`documentation/audits/offsite-finish-2026-10-04/red-proofs-hub.txt`.
## v0.127.0 — off-site keys a box cannot use to delete: the hub is the key registrar, the storage password is sealed and never served, a daily key check, the clean-up window (decisions 68–69, R-820, R-821, R-822) (2026-10-03)
- **The box never receives the Storage Box sub-account password again (R-820).** `POST /api/v1/offsite/consume-password/`
+13
View File
@@ -391,6 +391,17 @@ func main() {
Store: dataStore, Reg: &offsitekeys.Registrar{Dialer: offsitekeys.SSHDialer{}}, Logger: logger,
Emit: dispatcher.ProcessEvent,
}
// Decision 74: the hub-enforced wait before a set-aside copy is deleted. OFFSITE_ABANDON_DELAY
// overrides the 7-day default ONLY for a test, and is logged loudly when it does.
keySvc.AbandonDelay = offsitekeys.DefaultAbandonDelay
if v := os.Getenv("OFFSITE_ABANDON_DELAY"); v != "" {
if d, derr := time.ParseDuration(v); derr == nil && d > 0 {
keySvc.AbandonDelay = d
logger.Printf("[WARN] OFFSITE_ABANDON_DELAY=%s — set-aside deletions wait %s instead of 7 days (TEST CONFIGURATION)", v, d)
} else {
logger.Printf("[ERROR] OFFSITE_ABANDON_DELAY=%q invalid — keeping 7 days", v)
}
}
apiHandler.SetOffsiteKeyService(keySvc)
runKeyAudit := func(ctx context.Context) any {
start := time.Now()
@@ -415,6 +426,7 @@ func main() {
}
webServer.SetOffsiteKeyAudit(runKeyAudit)
webServer.SetOffsiteWindowAdmin(dataStore.GrantOffsiteWindowOnce, dataStore.SetOffsiteWindowsEnabled)
webServer.SetOffsiteKeyAdmin(keySvc.RemoveUnpinnedKeys, keySvc.CancelAbandon)
// Decision 68: a window the box never closed is closed by the hub at its 20-minute bound.
go func() {
tk := time.NewTicker(60 * time.Second)
@@ -426,6 +438,7 @@ func main() {
case <-tk.C:
sctx, cancel := context.WithTimeout(ctx, 2*time.Minute)
keySvc.SweepExpiredWindows(sctx)
keySvc.SweepAbandons(sctx)
cancel()
}
}
+6
View File
@@ -366,6 +366,12 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
h.handleOffsiteWindowOpen(w, r, strings.TrimPrefix(path, "/offsite/window-open/"))
case r.Method == http.MethodPost && strings.HasPrefix(path, "/offsite/window-close/"):
h.handleOffsiteWindowClose(w, r, strings.TrimPrefix(path, "/offsite/window-close/"))
case r.Method == http.MethodPost && strings.HasPrefix(path, "/offsite/abandon-request/"):
h.handleOffsiteAbandon(w, r, "request", strings.TrimPrefix(path, "/offsite/abandon-request/"))
case r.Method == http.MethodPost && strings.HasPrefix(path, "/offsite/abandon-cancel/"):
h.handleOffsiteAbandon(w, r, "cancel", strings.TrimPrefix(path, "/offsite/abandon-cancel/"))
case r.Method == http.MethodPost && strings.HasPrefix(path, "/offsite/abandon-status/"):
h.handleOffsiteAbandon(w, r, "status", strings.TrimPrefix(path, "/offsite/abandon-status/"))
case r.Method == http.MethodGet && strings.HasPrefix(path, "/artifacts/"):
customerID := strings.TrimPrefix(path, "/artifacts/")
h.handleArtifactManifest(w, r, customerID)
+47
View File
@@ -18,6 +18,9 @@ type OffsiteKeyService interface {
MoveAside(ctx context.Context, customerID string) (string, error)
OpenWindowFor(ctx context.Context, customerID string, countBefore int) (offsitekeys.WindowGrant, error)
CloseWindowFor(ctx context.Context, customerID string, r offsitekeys.WindowResult) error
RequestAbandon(ctx context.Context, customerID, path string) (offsitekeys.AbandonStatus, error)
CancelAbandon(customerID, by string) (int, error)
AbandonStatusFor(customerID string) (offsitekeys.AbandonStatus, error)
}
// SetOffsiteKeyService wires the key registrar.
@@ -166,3 +169,47 @@ func (h *Handler) handleOffsiteWindowClose(w http.ResponseWriter, r *http.Reques
}
writeJSON(w, http.StatusOK, map[string]any{"closed": true})
}
// handleOffsiteAbandon: the box's half of decision 74 (R-823).
//
// POST /offsite/abandon-request/<id> {"path": "<repo>.orphaned-…"} → recorded; deleted by the hub after the delay
// POST /offsite/abandon-cancel/<id> → every pending request cancelled
// POST /offsite/abandon-status/<id> → {"state": none|pending|cancelled|deleted, …}
func (h *Handler) handleOffsiteAbandon(w http.ResponseWriter, r *http.Request, verb, customerID string) {
if !h.offsiteKeyAuth(w, r, customerID) {
return
}
ctx, cancel := context.WithTimeout(r.Context(), 2*time.Minute)
defer cancel()
switch verb {
case "request":
var req struct {
Path string `json:"path"`
}
body, _ := io.ReadAll(io.LimitReader(r.Body, 4<<10))
if err := json.Unmarshal(body, &req); err != nil || req.Path == "" {
http.Error(w, "body must be {\"path\": \"…\"}", http.StatusBadRequest)
return
}
st, err := h.offsiteKeys.RequestAbandon(ctx, customerID, req.Path)
if err != nil {
offsiteKeyErr(w, err)
return
}
writeJSON(w, http.StatusOK, st)
case "cancel":
n, err := h.offsiteKeys.CancelAbandon(customerID, "box")
if err != nil {
offsiteKeyErr(w, err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"cancelled": n})
default:
st, err := h.offsiteKeys.AbandonStatusFor(customerID)
if err != nil {
offsiteKeyErr(w, err)
return
}
writeJSON(w, http.StatusOK, st)
}
}
+10
View File
@@ -69,6 +69,13 @@ func (f *fakeKeySvc) OpenWindowFor(context.Context, string, int) (offsitekeys.Wi
return offsitekeys.WindowGrant{Granted: false, Reason: "not due"}, f.err
}
func (f *fakeKeySvc) CloseWindowFor(context.Context, string, offsitekeys.WindowResult) error { return f.err }
func (f *fakeKeySvc) RequestAbandon(context.Context, string, string) (offsitekeys.AbandonStatus, error) {
return offsitekeys.AbandonStatus{State: "pending"}, f.err
}
func (f *fakeKeySvc) CancelAbandon(string, string) (int, error) { return 1, f.err }
func (f *fakeKeySvc) AbandonStatusFor(string) (offsitekeys.AbandonStatus, error) {
return offsitekeys.AbandonStatus{State: "none"}, f.err
}
func (f *fakeKeySvc) MoveAside(context.Context, string) (string, error) {
return "/home/felhom-repo.orphaned-20261003", f.err
}
@@ -108,6 +115,9 @@ func TestOffsiteKeyEndpoints_AuthAndNoPasswordInAnyResponse(t *testing.T) {
{"/api/v1/offsite/move-aside/c1", ``},
{"/api/v1/offsite/window-open/c1", `{"count_before":3}`},
{"/api/v1/offsite/window-close/c1", `{"window_id":1,"count_after":3,"outcome":"nothing"}`},
{"/api/v1/offsite/abandon-request/c1", `{"path":"/home/felhom-repo.orphaned-20261004"}`},
{"/api/v1/offsite/abandon-cancel/c1", ``},
{"/api/v1/offsite/abandon-status/c1", ``},
} {
rr := post(c.path, "ckey", c.body)
if rr.Code != http.StatusOK {
+75 -4
View File
@@ -130,11 +130,9 @@ func (r *Registrar) open(ctx context.Context, t Target, password string) (Shell,
// read returns the current authorized_keys content; a missing file is "" (cat exits 1 there).
func read(ctx context.Context, sh Shell) (string, error) {
// READ-ONLY (R-827, v0.128.0): a missing .ssh or file reads as "" — the directory is created only by
// write(). Until v0.127.0 the daily check created .ssh on a sub-account that had none.
if _, err := sh.Run(ctx, "ls -d .ssh", nil); err != nil {
if _, merr := sh.Run(ctx, "mkdir .ssh", nil); merr != nil {
return "", fmt.Errorf("offsitekeys: create .ssh: %w", merr)
}
_, _ = sh.Run(ctx, "chmod 700 .ssh", nil)
return "", nil
}
if _, err := sh.Run(ctx, "ls "+authorizedKeys, nil); err != nil {
@@ -149,6 +147,12 @@ func read(ctx context.Context, sh Shell) (string, error) {
// write replaces authorized_keys atomically (dd to a temp file, chmod, mv) and reads it back.
func write(ctx context.Context, sh Shell, content string) error {
if _, err := sh.Run(ctx, "ls -d .ssh", nil); err != nil {
if _, merr := sh.Run(ctx, "mkdir .ssh", nil); merr != nil {
return fmt.Errorf("offsitekeys: create .ssh: %w", merr)
}
_, _ = sh.Run(ctx, "chmod 700 .ssh", nil)
}
if _, err := sh.Run(ctx, "dd of="+tmpKeys, []byte(content)); err != nil {
return fmt.Errorf("offsitekeys: write temp file: %w", err)
}
@@ -384,3 +388,70 @@ func (r *Registrar) MoveAside(ctx context.Context, t Target, password, date stri
}
return name, nil
}
// RemoveUnpinned rewrites authorized_keys keeping only the PINNED lines (decision 72, R-826): every
// unpinned line — a route to deletion — and any window line (when no window is open) goes. An empty
// result is allowed (no box). Returns how many lines were removed; a file with nothing to remove is not
// rewritten.
func (r *Registrar) RemoveUnpinned(ctx context.Context, t Target, password string, windowOpen bool) (int, error) {
sh, err := r.open(ctx, t, password)
if err != nil {
return 0, err
}
defer sh.Close()
cur, err := read(ctx, sh)
if err != nil {
return 0, err
}
var keep []string
removed := 0
for _, l := range ParseLines(cur, t.RepoPath) {
if l.Pinned || (l.Window && windowOpen) {
keep = append(keep, l.Raw)
continue
}
removed++
}
if removed == 0 {
return 0, nil
}
return removed, write(ctx, sh, join(keep))
}
// DeleteSetAside removes one SET-ASIDE repository directory (decision 74, R-823) — the only deletion the
// registrar performs. It refuses anything that is not `<RepoPath>.orphaned-<…>` (never the live
// repository, never a path with a slash or "..") and anything that does not exist.
func (r *Registrar) DeleteSetAside(ctx context.Context, t Target, password, path string) error {
if !IsSetAsidePath(t.RepoPath, path) {
return fmt.Errorf("offsitekeys: %q is not a set-aside copy of %s — refusing", path, t.RepoPath)
}
sh, err := r.open(ctx, t, password)
if err != nil {
return err
}
defer sh.Close()
if _, err := sh.Run(ctx, "ls -d "+path, nil); err != nil {
return fmt.Errorf("offsitekeys: set-aside copy %s not found: %w", path, err)
}
if _, err := sh.Run(ctx, "rm -rf "+path, nil); err != nil {
return fmt.Errorf("offsitekeys: delete %s: %w", path, err)
}
if _, err := sh.Run(ctx, "ls -d "+path, nil); err == nil {
return fmt.Errorf("offsitekeys: %s still exists after the delete", path)
}
return nil
}
// IsSetAsidePath: `<repo>.orphaned-<suffix>` with a suffix of [A-Za-z0-9-] only.
func IsSetAsidePath(repo, path string) bool {
pre := repo + ".orphaned-"
if repo == "" || !strings.HasPrefix(path, pre) || len(path) == len(pre) {
return false
}
for _, c := range path[len(pre):] {
if !(c == '-' || (c >= '0' && c <= '9') || (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z')) {
return false
}
}
return true
}
+4 -1
View File
@@ -61,8 +61,11 @@ func (f *fakeFS) Run(_ context.Context, cmd string, stdin []byte) ([]byte, error
return nil, nil
}
return nil, miss
case a[0] == "rm" && a[1] == "-rf" && strings.Contains(a[2], ".orphaned-"):
delete(f.dirs, a[2]) // decision 74: the ONLY delete, of a set-aside copy
return nil, nil
case a[0] == "rm":
return nil, errors.New("the registrar must never delete")
return nil, errors.New("the registrar must never delete anything else")
}
return nil, errors.New("Command not found")
}
+143 -4
View File
@@ -30,6 +30,8 @@ type Service struct {
// Emit routes an event to the dispatcher (operator mail). nil → events are only saved.
Emit func(customerID, eventType, severity, message, detailsJSON, source string)
Now func() time.Time
// AbandonDelay is the hub-enforced wait before a set-aside copy is deleted (decision 74). 0 → 7 days.
AbandonDelay time.Duration
}
// ErrNotProvisioned — the customer has no provisioned off-site target (nothing to register against).
@@ -229,11 +231,12 @@ type WindowResult struct {
Reason string `json:"reason"`
}
// MaxRemove is the most snapshots one window may remove: 40 % of what was there, at least 5. The ruled
// policy (7 daily + 4 weekly + 6 monthly per app) removes ~7 of ~17 per app per week (~41 %); the box
// takes the OLDEST first within this bound, so a backlog drains over several windows.
// MaxRemove is the most snapshots one window may remove: HALF of what was there, at least 5. The ruled
// policy (7 daily + 4 weekly + 6 monthly per app) removes ~7 of ~17 per app per week (~41 %) — v0.127.0's
// 40 % would have refused every honest week once the box refuses above the cap (controller v0.290.0).
// Half is also the line R-431's detector draws for "an unexplained fall". Pinned by TestMaxRemove_*.
func MaxRemove(countBefore int) int {
n := countBefore * 40 / 100
n := countBefore / 2
if n < 5 {
n = 5
}
@@ -333,3 +336,139 @@ func (s *Service) SweepExpiredWindows(ctx context.Context) {
fmt.Sprintf("Off-site clean-up window %d was not closed by the box within %s; the hub closed it (the deleting key line is removed).", w.ID, windowLength), nil)
}
}
// ── Decision 74 (R-823): the household's "delete my set-aside history" — done by the HUB, after a delay ──
//
// The box's key cannot delete (decision 69), so a due abandonment is a REQUEST to the hub. The hub waits
// AbandonDelay (default 7 days) from the request — the household (via the box's recovery, which cancels)
// and the operator can cancel in that time — then deletes ONLY `<repo>.orphaned-<…>`, never the live
// repository. A broken-into box can therefore make a set-aside copy disappear only after a week of
// operator mails. Every request, cancel and deletion is an operator event.
const (
EventAbandonRequested = "offsite_abandon_requested"
EventAbandonCancelled = "offsite_abandon_cancelled"
EventAbandonDeleted = "offsite_abandon_deleted"
EventAbandonFailed = "offsite_abandon_failed"
DefaultAbandonDelay = 7 * 24 * time.Hour
)
func (s *Service) abandonDelay() time.Duration {
if s.AbandonDelay > 0 {
return s.AbandonDelay
}
return DefaultAbandonDelay
}
// AbandonStatus is what the box (and the operator) sees.
type AbandonStatus struct {
State string `json:"state"` // none | pending | cancelled | deleted
Path string `json:"path,omitempty"`
DueAt time.Time `json:"due_at,omitempty"`
}
func statusOf(a *store.OffsiteAbandon) AbandonStatus {
if a == nil {
return AbandonStatus{State: "none"}
}
return AbandonStatus{State: a.State(), Path: a.Path, DueAt: a.DueAt.UTC()}
}
// RequestAbandon records (idempotently) the household's request to delete path, due after the delay.
func (s *Service) RequestAbandon(ctx context.Context, customerID, path string) (AbandonStatus, error) {
t, pw, err := s.TargetFor(customerID)
if err != nil {
return AbandonStatus{}, err
}
if !IsSetAsidePath(t.RepoPath, path) {
return AbandonStatus{}, fmt.Errorf("offsitekeys: %q is not a set-aside copy of %s", path, t.RepoPath)
}
if cur, err := s.Store.LatestOffsiteAbandon(customerID, path); err == nil && cur != nil && cur.State() == "pending" {
return statusOf(cur), nil // already requested — the clock is NOT restarted
}
// The copy must exist now (a request for nothing is refused, so a typo cannot sit armed for a week).
sh, err := s.Reg.open(ctx, t, pw)
if err != nil {
return AbandonStatus{}, err
}
_, lerr := sh.Run(ctx, "ls -d "+path, nil)
sh.Close()
if lerr != nil {
return AbandonStatus{}, fmt.Errorf("offsitekeys: set-aside copy %s not found", path)
}
due := s.now().Add(s.abandonDelay())
if _, err := s.Store.CreateOffsiteAbandon(customerID, path, due); err != nil {
return AbandonStatus{}, err
}
s.logf("[WARN] offsitekeys: %s asked to DELETE its set-aside off-site copy %s — the hub deletes it at %s unless cancelled (delay %s)",
customerID, path, due.UTC().Format(time.RFC3339), s.abandonDelay())
s.event(customerID, EventAbandonRequested, "warning",
fmt.Sprintf("Off-site: the box asked to delete the set-aside copy %s (the household's choice). The hub deletes it on %s unless the household or the operator cancels.", path, due.UTC().Format("2006-01-02 15:04 UTC")),
map[string]any{"path": path, "due_at": due.UTC()})
a, _ := s.Store.LatestOffsiteAbandon(customerID, path)
return statusOf(a), nil
}
// CancelAbandon cancels every pending request of the customer (by = "box" | "operator").
func (s *Service) CancelAbandon(customerID, by string) (int, error) {
n, err := s.Store.CancelOffsiteAbandon(customerID, by)
if err != nil || n == 0 {
return n, err
}
s.logf("[INFO] offsitekeys: %s's set-aside deletion CANCELLED by %s (%d request(s)) — nothing deleted", customerID, by, n)
s.event(customerID, EventAbandonCancelled, "info",
fmt.Sprintf("Off-site: the pending deletion of the set-aside copy was cancelled by the %s. Nothing was deleted.", by), nil)
return n, nil
}
// AbandonStatusFor returns the latest request's state.
func (s *Service) AbandonStatusFor(customerID string) (AbandonStatus, error) {
a, err := s.Store.LatestOffsiteAbandon(customerID, "")
if err != nil {
return AbandonStatus{}, err
}
return statusOf(a), nil
}
// SweepAbandons deletes every request that is due, not cancelled and not yet deleted.
func (s *Service) SweepAbandons(ctx context.Context) {
due, err := s.Store.DueOffsiteAbandons()
if err != nil {
s.logf("[WARN] offsitekeys: abandon sweep: %v", err)
return
}
for _, a := range due {
t, pw, terr := s.TargetFor(a.CustomerID)
if terr == nil {
terr = s.Reg.DeleteSetAside(ctx, t, pw, a.Path)
}
if terr != nil {
_ = s.Store.MarkOffsiteAbandonError(a.ID, terr.Error())
s.logf("[ERROR] offsitekeys: deleting %s's set-aside copy %s failed (retrying): %v", a.CustomerID, a.Path, terr)
if a.LastError == "" {
s.event(a.CustomerID, EventAbandonFailed, "warning",
fmt.Sprintf("Off-site: deleting the set-aside copy %s failed and is retried: %v", a.Path, terr), nil)
}
continue
}
_ = s.Store.MarkOffsiteAbandonDeleted(a.ID)
s.logf("[WARN] offsitekeys: DELETED %s's set-aside off-site copy %s (requested %s, due %s)", a.CustomerID, a.Path,
a.RequestedAt.UTC().Format(time.RFC3339), a.DueAt.UTC().Format(time.RFC3339))
s.event(a.CustomerID, EventAbandonDeleted, "info",
fmt.Sprintf("Off-site: the set-aside copy %s was deleted, as the household chose (requested %s).", a.Path, a.RequestedAt.UTC().Format("2006-01-02")), nil)
}
}
// RemoveUnpinnedKeys is the operator's clean-up of a sub-account's key file (decision 72).
func (s *Service) RemoveUnpinnedKeys(ctx context.Context, customerID string) (int, error) {
t, pw, err := s.TargetFor(customerID)
if err != nil {
return 0, err
}
n, err := s.Reg.RemoveUnpinned(ctx, t, pw, s.Store.OffsiteWindowOpen(customerID))
if err != nil {
return 0, err
}
s.logf("[INFO] offsitekeys: removed %d unpinned key line(s) from %s's sub-account (%s) on the operator's request", n, customerID, t.User)
return n, nil
}
+95 -2
View File
@@ -2,6 +2,7 @@ package offsitekeys
import (
"context"
"strings"
"log"
"os"
"path/filepath"
@@ -50,7 +51,7 @@ func TestWindow_GrantOpenCloseAndDropAlarm(t *testing.T) {
t.Fatal(err)
}
g, err := s.OpenWindowFor(ctx, "c1", 20)
if err != nil || !g.Granted || g.MaxRemove != 8 {
if err != nil || !g.Granted || g.MaxRemove != 10 {
t.Fatalf("one-shot: %+v %v", g, err)
}
if lines := ParseLines(fs.files[".ssh/authorized_keys"], "/home/felhom-repo"); !lines[0].Window || lines[0].Fingerprint != fp {
@@ -62,7 +63,7 @@ func TestWindow_GrantOpenCloseAndDropAlarm(t *testing.T) {
if g2, _ := s.OpenWindowFor(ctx, "c1", 20); g2.Granted {
t.Fatal("the one-shot grant was not consumed")
}
// The box reports a fall of 12 (allowed 8) → offsite_window_drop.
// The box reports a fall of 12 (allowed 10) → offsite_window_drop.
if err := s.CloseWindowFor(ctx, "c1", WindowResult{WindowID: g.WindowID, CountAfter: 8, Outcome: "pruned"}); err != nil {
t.Fatal(err)
}
@@ -127,3 +128,95 @@ func TestWindow_LeftOpenIsClosedByTheSweep(t *testing.T) {
t.Fatalf("last event = %s", last)
}
}
// Decision 74 (R-823): a set-aside deletion is NOT acted on before the delay, a cancelled request deletes
// nothing, and the live repository can never be named.
func TestAbandon_DelayCancelAndScope(t *testing.T) {
s, fs, events := svcFixture(t)
ctx := context.Background()
aside := "/home/felhom-repo.orphaned-20261004"
fs.dirs[aside] = true
fs.dirs["/home/felhom-repo"] = true
for _, bad := range []string{"/home/felhom-repo", "/home/felhom-repo.orphaned-../x", "/home/other.orphaned-1"} {
if _, err := s.RequestAbandon(ctx, "c1", bad); err == nil {
t.Fatalf("accepted a non-set-aside path %q", bad)
}
}
st, err := s.RequestAbandon(ctx, "c1", aside)
if err != nil || st.State != "pending" {
t.Fatalf("%+v %v", st, err)
}
// Not due yet (7-day default): the sweep deletes nothing.
s.SweepAbandons(ctx)
if !fs.dirs[aside] {
t.Fatal("deleted BEFORE the delay")
}
// Cancelled, then made due: still nothing deleted.
if n, _ := s.CancelAbandon("c1", "operator"); n != 1 {
t.Fatalf("cancelled %d", n)
}
a, _ := s.Store.LatestOffsiteAbandon("c1", aside)
_ = s.Store.ForceOffsiteAbandonDueForTest(a.ID)
s.SweepAbandons(ctx)
if !fs.dirs[aside] {
t.Fatal("a CANCELLED request deleted the copy")
}
// A fresh request, due: deleted, and only that directory.
if _, err := s.RequestAbandon(ctx, "c1", aside); err != nil {
t.Fatal(err)
}
a, _ = s.Store.LatestOffsiteAbandon("c1", aside)
_ = s.Store.ForceOffsiteAbandonDueForTest(a.ID)
s.SweepAbandons(ctx)
if fs.dirs[aside] || !fs.dirs["/home/felhom-repo"] {
t.Fatalf("after the due sweep: %v", fs.dirs)
}
if st, _ := s.AbandonStatusFor("c1"); st.State != "deleted" {
t.Fatalf("state = %s", st.State)
}
last := (*events)[len(*events)-1]
if last != EventAbandonDeleted {
t.Fatalf("last event %s", last)
}
}
// Decision 72 (R-826): the operator clean-up keeps pinned lines and drops the rest; an empty file is fine.
func TestRemoveUnpinned_KeepsOnlyPinned(t *testing.T) {
s, fs, _ := svcFixture(t)
a, _ := newKey(t)
b, _ := newKey(t)
fs.files[".ssh/authorized_keys"] = a + "\n" + b + "\n"
n, err := s.RemoveUnpinnedKeys(context.Background(), "c1")
if err != nil || n != 2 || fs.files[".ssh/authorized_keys"] != "" {
t.Fatalf("n=%d err=%v file=%q", n, err, fs.files[".ssh/authorized_keys"])
}
if n, _ := s.RemoveUnpinnedKeys(context.Background(), "c1"); n != 0 {
t.Fatal("second run changed something")
}
}
// R-827: the daily check is read-only — no .ssh is created on a sub-account that has none.
func TestAudit_DoesNotCreateSSHDir(t *testing.T) {
fs := newFS()
delete(fs.dirs, ".ssh")
r := &Registrar{Dialer: fakeDialer{fs}}
if _, err := r.Audit(context.Background(), tgt, "pw", false); err != nil {
t.Fatal(err)
}
for _, c := range fs.cmds {
if strings.HasPrefix(c, "mkdir") {
t.Fatalf("the audit wrote: %q", c)
}
}
}
// The honest weekly removal (7 of ~17 per app, ~41 %) must fit under the cap — demo-hp's real shape:
// 9 apps × 17 = 153 snapshots, 63 removed in a week.
func TestMaxRemove_HonestWeekFits(t *testing.T) {
if MaxRemove(153) < 63 {
t.Fatalf("MaxRemove(153) = %d < 63 — every honest window would be refused", MaxRemove(153))
}
if MaxRemove(4) != 5 {
t.Fatal("floor of 5 lost")
}
}
+116
View File
@@ -181,3 +181,119 @@ func (s *Store) ForceOffsiteWindowDueForTest(id int64) error {
_, err := s.db.Exec(`UPDATE offsite_windows SET closes_by = datetime('now', '-1 minute') WHERE id = ?`, id)
return err
}
// OffsiteAbandon is one household request to delete a set-aside off-site copy (decision 74, R-823). The
// hub deletes it only after DueAt, and only if nobody cancelled it.
type OffsiteAbandon struct {
ID int64
CustomerID string
Path string
RequestedAt time.Time
DueAt time.Time
CancelledAt time.Time
CancelledBy string
DeletedAt time.Time
LastError string
}
func (a *OffsiteAbandon) State() string {
switch {
case a == nil:
return "none"
case !a.DeletedAt.IsZero():
return "deleted"
case !a.CancelledAt.IsZero():
return "cancelled"
}
return "pending"
}
const abandonCols = `id, customer_id, path, requested_at, due_at, cancelled_at, cancelled_by, deleted_at, last_error`
func scanAbandon(sc interface{ Scan(...any) error }) (*OffsiteAbandon, error) {
var a OffsiteAbandon
var req, due string
var canc, by, del, lerr sql.NullString
if err := sc.Scan(&a.ID, &a.CustomerID, &a.Path, &req, &due, &canc, &by, &del, &lerr); err != nil {
return nil, err
}
a.RequestedAt, a.DueAt = parseSQLiteTime(req), parseSQLiteTime(due)
if canc.Valid {
a.CancelledAt = parseSQLiteTime(canc.String)
}
if del.Valid {
a.DeletedAt = parseSQLiteTime(del.String)
}
a.CancelledBy, a.LastError = by.String, lerr.String
return &a, nil
}
// LatestOffsiteAbandon returns the customer's most recent request for path ("" = any), or (nil, nil).
func (s *Store) LatestOffsiteAbandon(customerID, path string) (*OffsiteAbandon, error) {
q := `SELECT ` + abandonCols + ` FROM offsite_abandon_requests WHERE customer_id = ?`
args := []any{customerID}
if path != "" {
q += ` AND path = ?`
args = append(args, path)
}
a, err := scanAbandon(s.db.QueryRow(q+` ORDER BY id DESC LIMIT 1`, args...))
if err == sql.ErrNoRows {
return nil, nil
}
return a, err
}
// CreateOffsiteAbandon records a request due at dueAt.
func (s *Store) CreateOffsiteAbandon(customerID, path string, dueAt time.Time) (int64, error) {
res, err := s.db.Exec(`INSERT INTO offsite_abandon_requests (customer_id, path, requested_at, due_at) VALUES (?, ?, datetime('now'), ?)`,
customerID, path, dueAt.UTC().Format("2006-01-02 15:04:05"))
if err != nil {
return 0, err
}
return res.LastInsertId()
}
// CancelOffsiteAbandon cancels every PENDING request of the customer; returns how many.
func (s *Store) CancelOffsiteAbandon(customerID, by string) (int, error) {
res, err := s.db.Exec(`UPDATE offsite_abandon_requests SET cancelled_at = datetime('now'), cancelled_by = ? WHERE customer_id = ? AND cancelled_at IS NULL AND deleted_at IS NULL`, by, customerID)
if err != nil {
return 0, err
}
n, _ := res.RowsAffected()
return int(n), nil
}
// DueOffsiteAbandons lists pending requests whose due time has passed.
func (s *Store) DueOffsiteAbandons() ([]*OffsiteAbandon, error) {
rows, err := s.db.Query(`SELECT ` + abandonCols + ` FROM offsite_abandon_requests WHERE cancelled_at IS NULL AND deleted_at IS NULL AND due_at <= datetime('now')`)
if err != nil {
return nil, err
}
defer rows.Close()
var out []*OffsiteAbandon
for rows.Next() {
a, err := scanAbandon(rows)
if err != nil {
return nil, err
}
out = append(out, a)
}
return out, rows.Err()
}
// MarkOffsiteAbandonDeleted / MarkOffsiteAbandonError record the sweep's outcome.
func (s *Store) MarkOffsiteAbandonDeleted(id int64) error {
_, err := s.db.Exec(`UPDATE offsite_abandon_requests SET deleted_at = datetime('now'), last_error = NULL WHERE id = ?`, id)
return err
}
func (s *Store) MarkOffsiteAbandonError(id int64, msg string) error {
_, err := s.db.Exec(`UPDATE offsite_abandon_requests SET last_error = ? WHERE id = ?`, msg, id)
return err
}
// ForceOffsiteAbandonDueForTest back-dates a request. TEST-ONLY.
func (s *Store) ForceOffsiteAbandonDueForTest(id int64) error {
_, err := s.db.Exec(`UPDATE offsite_abandon_requests SET due_at = datetime('now', '-1 minute') WHERE id = ?`, id)
return err
}
+11
View File
@@ -842,6 +842,17 @@ func (s *Store) migrate() error {
close_reason TEXT
);
CREATE INDEX IF NOT EXISTS idx_offsite_windows_customer ON offsite_windows(customer_id, opened_at);
CREATE TABLE IF NOT EXISTS offsite_abandon_requests (
id INTEGER PRIMARY KEY AUTOINCREMENT,
customer_id TEXT NOT NULL,
path TEXT NOT NULL,
requested_at DATETIME NOT NULL DEFAULT (datetime('now')),
due_at DATETIME NOT NULL,
cancelled_at DATETIME,
cancelled_by TEXT,
deleted_at DATETIME,
last_error TEXT
);
`); err != nil {
return fmt.Errorf("offsite_keys/offsite_windows: %w", err)
}
+28
View File
@@ -72,6 +72,9 @@ type Server struct {
// offsiteWindowAdmin: operator one-shot grant / weekly switch (decision 68). nil → 503.
offsiteWindowGrant func(customerID string) error
offsiteWindowSwitch func(on bool) error
// operator key-file clean-up (decision 72) and abandonment cancel (decision 74). nil → 503.
offsiteRemoveUnpinned func(ctx context.Context, customerID string) (int, error)
offsiteAbandonCancel func(customerID, by string) (int, error)
offsiteBox func() (monitor.BoxSnapshot, bool) // optional (v0.64.0, R-5); the restic pool-box aggregate snapshot accessor
pbsdrBox func() (monitor.PBSBoxSnapshot, bool) // optional (v0.65.0, R-5); the PBS-DR datastore fill snapshot accessor
tenantsync tenancyProvisioner // optional; enables PBS DR tier provisioning (web/pbsdr.go)
@@ -202,6 +205,11 @@ func (s *Server) SetOffsiteProvisioner(p *offsite.Provisioner) { s.offsite = p }
// SetOffsiteKeyAudit wires the on-demand run of the daily off-site key check (decision 69).
func (s *Server) SetOffsiteKeyAudit(fn func(ctx context.Context) any) { s.offsiteKeyAudit = fn }
// SetOffsiteKeyAdmin wires the operator's key-file clean-up and abandonment cancel.
func (s *Server) SetOffsiteKeyAdmin(remove func(context.Context, string) (int, error), cancel func(string, string) (int, error)) {
s.offsiteRemoveUnpinned, s.offsiteAbandonCancel = remove, cancel
}
// SetOffsiteWindowAdmin wires the operator's window controls (decision 68).
func (s *Server) SetOffsiteWindowAdmin(grant func(string) error, sw func(bool) error) {
s.offsiteWindowGrant, s.offsiteWindowSwitch = grant, sw
@@ -617,6 +625,26 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
} else {
s.handleConfigEditForm(w, r, customerID)
}
case strings.HasPrefix(path, "/offsite/remove-unpinned/") || strings.HasPrefix(path, "/offsite/abandon-cancel/"):
// Operator: rewrite a sub-account's key file keeping only pinned lines (decision 72); cancel a
// household's pending set-aside deletion (decision 74).
if r.Method != http.MethodPost || s.offsiteRemoveUnpinned == nil {
http.Error(w, "unavailable", http.StatusServiceUnavailable)
return
}
var n int
var err error
if strings.HasPrefix(path, "/offsite/remove-unpinned/") {
n, err = s.offsiteRemoveUnpinned(r.Context(), strings.TrimPrefix(path, "/offsite/remove-unpinned/"))
} else {
n, err = s.offsiteAbandonCancel(strings.TrimPrefix(path, "/offsite/abandon-cancel/"), "operator")
}
if err != nil {
http.Error(w, err.Error(), http.StatusBadGateway)
return
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]int{"changed": n})
case strings.HasPrefix(path, "/offsite/window-grant/") || path == "/offsite/windows-enabled":
// Operator (decision 68): a one-shot grant lets the customer's NEXT window request through;
// the switch turns the WEEKLY window on/off fleet-wide (off = the interim: nothing prunes).