diff --git a/documentation/architecture/05-hub-architecture.md b/documentation/architecture/05-hub-architecture.md index d8ef7535..9e0e79d0 100644 --- a/documentation/architecture/05-hub-architecture.md +++ b/documentation/architecture/05-hub-architecture.md @@ -921,7 +921,7 @@ method (405), so a token gets 403. | `/configs/{id}/edit` | `customers:read` | **session only** | Edit can change the household's e-mail — and a new e-mail auto-mails the self-bind link (ruling 13:20). | | `/offsite/remove-unpinned/{id}` | — | **session only** | | | `/offsite/abandon-cancel/{id}` | — | **session only** | Cancelling a pending set-aside deletion: no group names it; until the operator places it, session only. | -| `/offsite/window-grant/{id}` | — | `offsite:write` | The one-shot clean-up window: the script address AND the customer page's „Backup & DR" card (R-1299, hub v0.174.0): `from=page` + `confirm=1` from the form; `cancel=1` removes a waiting grant; a limit above the store's snapshot count is refused. **Built, not released (D72/D84, `07` §8.2f):** the box-facing `POST /api/v1/offsite/window-open/{id}` also takes the box's due off-site end records (`ended`) and answers `ended_allowed` / `ended_refused` — ids of removed apps' snapshots the hub agrees to from its own records, beside the window's cap; `window-close` takes `removed_for_ended_apps`. The card's „Last window” shows what was agreed, removed and refused. | +| `/offsite/window-grant/{id}` | — | `offsite:write` | The one-shot clean-up window: the script address AND the customer page's „Backup & DR" card (R-1299, hub v0.174.0): `from=page` + `confirm=1` from the form; `cancel=1` removes a waiting grant; a limit above the store's snapshot count is refused. **Built, not released (D72/D84, `07` §8.2f):** the box-facing `POST /api/v1/offsite/window-open/{id}` also takes the box's due off-site end records (`ended`) and answers `ended_allowed` / `ended_refused` — ids of removed apps' snapshots the hub agrees to from its own records, beside the window's cap; `window-close` takes `removed_for_ended_apps`. Every window-open answer carries `accepts_ended` (a box sends `ended` only to a hub that said it), and a request the hub cannot read is answered 400. The card's „Last window” shows what was agreed, removed and refused. | | `/offsite/windows-enabled` | — | `offsite:write` | | | `/os/{id}` | `os:read` | `os:write` | | | `/offsite/key-audit` | — | `offsite:read` | Runs the daily key check now: it reads keys and reports, it changes none. | diff --git a/documentation/architecture/07-backup-architecture.md b/documentation/architecture/07-backup-architecture.md index 6a5f6db4..cdd24a5f 100644 --- a/documentation/architecture/07-backup-architecture.md +++ b/documentation/architecture/07-backup-architecture.md @@ -1917,7 +1917,7 @@ notification address gets no mail: the page is all it has.** A blocked household | A record with a date in the past, written by a box the hub does not agree with | Refused: the hub never reads the box's dates | hub `TestD84_AgreeEnded…`, `TestD72_ReportFeedsTheHubsOwnAppRecord` | | An app removed with „delete backup data” BEFORE this is built | No record exists, so nothing is deleted. The off-site list shows the app with „delete in 30 days”: the household starts the clock itself | `TestD72_RestorePageShowsTheEndPerBranch` | | The guard refuses the window (a future-dated snapshot, a plan above the cap) | The ended apps wait with everything else | `TestD84_OneGuard_AndTheAllowanceIsBesideTheCap` | -| The hub does not know ended apps (an older hub), or the box does not (an older controller) | Nothing of an ended app is removed; each side is releasable alone | `TestD84_TheBoxRemovesOnlyWhatTheHubAgreedTo` | +| The hub does not know ended apps (an older hub), or the box does not (an older controller) | Nothing of an ended app is removed, and the ordinary weekly clean-up goes on: the box sends its end records only to a hub whose last window answer said `accepts_ended` (an older hub reads 4 KiB of the request and would stall on more); a hub answers 400 on a request it cannot read; one request carries at most 5000 ids | `TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns`, `TestD84_OneRequestCarriesABoundedNumberOfIds`, hub `TestD84_WindowOpen_ABodyTheHubCannotReadIs400` | | The app has no copy in the store at all | The due record is closed with no history line; nothing is asked of the hub | `TestD72_ARecordWithNoCopiesIsClosedQuietly` | | A take-back or an install while the window is being prepared (the hub answers, the dry run, the guard) | The window reads the records AGAIN immediately before the forget; an id whose record is gone or changed is not removed. A take-back during the forget itself is answered „not marked, or already deleted”. **Not covered:** an install during the forget | `TestD72_ATakeBackDuringTheWindowKeepsTheCopies`, `TestD72_AnInstallDuringTheWindowKeepsTheCopies`, `TestD72_ATakeBackDuringTheForgetIsNotAnsweredKept` | | The app is installed again by an import, or by a restore that installs from a unit | The record is deleted at the ONE point every install passes (`stacks.SaveAppConfig`) | `TestD72_AnImportDeletesTheEndRecord`, `TestD72_ARestoreIntoAnInstallDeletesTheEndRecord`, `TestD72_TheInstallWritersAreListed` | diff --git a/documentation/audits/build-release-4-2026-11-15/A/redproof-review-5.txt b/documentation/audits/build-release-4-2026-11-15/A/redproof-review-5.txt new file mode 100644 index 00000000..9fce10a0 --- /dev/null +++ b/documentation/audits/build-release-4-2026-11-15/A/redproof-review-5.txt @@ -0,0 +1,86 @@ +FINDING 5 — version skew: a new box with an older hub can stall the weekly clean-up; an unreadable request was decided as count 0. + +### RED, hub: the new tests before the fix (felhom.eu 6d2d4bd0) +=== RUN TestD84_WindowOpen_ABodyTheHubCannotReadIs400 + d84_window_skew_test.go:37: cut in the middle (a read limit): HTTP 200, want 400 — the hub must not decide a window on a body it could not read + d84_window_skew_test.go:40: cut in the middle (a read limit): the window service was asked although the body could not be read + d84_window_skew_test.go:37: not JSON: HTTP 200, want 400 — the hub must not decide a window on a body it could not read + d84_window_skew_test.go:40: not JSON: the window service was asked although the body could not be read + d84_window_skew_test.go:37: a list, not an object: HTTP 200, want 400 — the hub must not decide a window on a body it could not read + d84_window_skew_test.go:40: a list, not an object: the window service was asked although the body could not be read +--- FAIL: TestD84_WindowOpen_ABodyTheHubCannotReadIs400 (0.05s) +=== RUN TestD84_WindowAnswerSaysTheHubUnderstandsEnded + d84_window_skew_test.go:56: the answer {"granted":false,"newest_allowed":"0001-01-01T00:00:00Z","reason":"not due"} does not say accepts_ended — a new box would never send its end records + d84_window_skew_test.go:56: the answer {"granted":true,"newest_allowed":"0001-01-01T00:00:00Z"} does not say accepts_ended — a new box would never send its end records +--- FAIL: TestD84_WindowAnswerSaysTheHubUnderstandsEnded (0.00s) +FAIL +FAIL gitea.dooplex.hu/admin/felhom-hub/internal/api 0.076s +FAIL + +### RED, controller: the new tests before the fix (felhom-controller c7e4ec9) +=== RUN TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns + offbox_ended_skew_test.go:29: week 1: the box sent 1 end record(s) to a hub that never said it reads them — that hub reads 4 KiB and would stall +--- FAIL: TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns (0.00s) +=== RUN TestD84_OneRequestCarriesABoundedNumberOfIds + offbox_ended_skew_test.go:83: one request carried 5200 snapshot ids, want the limit 5000 +--- FAIL: TestD84_OneRequestCarriesABoundedNumberOfIds (0.03s) +FAIL +FAIL gitea.dooplex.hu/admin/felhom-controller/internal/backup 0.050s +FAIL + +### After the fix: mutations, hub +== MUTATION: a body that cannot be read is decided as count 0 again + RED on the mutant, GREEN on the real file + --- FAIL: TestD84_WindowOpen_ABodyTheHubCannotReadIs400 (0.05s) + d84_window_skew_test.go:36: cut in the middle (a read limit): HTTP 200, want 400 — the hub must not decide a window on a body it could not read + d84_window_skew_test.go:39: cut in the middle (a read limit): the window service was asked although the body could not be read + d84_window_skew_test.go:36: not JSON: HTTP 200, want 400 — the hub must not decide a window on a body it could not read +== MUTATION: the answer does not say that the hub understands ended apps + RED on the mutant, GREEN on the real file + --- FAIL: TestD84_WindowOpen_ABodyTheHubCannotReadIs400 (0.05s) + d84_window_skew_test.go:52: body "{\"count_before\":20}": the answer {"granted":false,"newest_allowed":"0001-01-01T00:00:00Z","reason":"not due","accepts_ended":false} + d84_window_skew_test.go:52: body "": the answer {"granted":false,"newest_allowed":"0001-01-01T00:00:00Z","reason":"not due","accepts_ended":false} +== MUTATION: the empty body of the oldest boxes is refused + *** NOT A RED-PROOF *** (mutant rc=1 built=False real rc=0) + FAIL gitea.dooplex.hu/admin/felhom-hub/internal/api [build failed] + +3 mutations, 1 not a red-proof + +### After the fix: mutations, controller +== MUTATION: the box sends its end records to any hub + RED on the mutant, GREEN on the real file + --- FAIL: TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns (0.00s) + offbox_ended_skew_test.go:29: week 1: the box sent 1 end record(s) to a hub that never said it reads them — that hub reads 4 KiB and would stall +== MUTATION: the box does not remember the hub's answer + RED on the mutant, GREEN on the real file + --- FAIL: TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns (0.00s) + offbox_ended_skew_test.go:49: the night the box LEARNS it: claims=0 learned=false — it must send none yet and remember the answer +== MUTATION: the box only ever learns „yes” (a rolled-back hub keeps getting end records) + RED on the mutant, GREEN on the real file + --- FAIL: TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns (0.00s) + offbox_ended_skew_test.go:65: after a rolled-back hub's answer the box still sends end records: claims=0 learned=true +== MUTATION: no limit on the ids of one request + RED on the mutant, GREEN on the real file + --- FAIL: TestD84_OneRequestCarriesABoundedNumberOfIds (0.03s) + offbox_ended_skew_test.go:84: one request carried 5200 snapshot ids, want the limit 5000 + +4 mutations, 0 not a red-proof +== MUTATION: the client does not read accepts_ended + RED on the mutant, GREEN on the real file + --- FAIL: TestD84_WindowClient_SendsTheClaimsAndReadsTheAgreement (0.00s) + d84_window_client_test.go:63: accepts_ended of a not-granted answer was not read +== MUTATION: the client reads every hub as one that knows end records + RED on the mutant, GREEN on the real file + --- FAIL: TestD84_WindowClient_SendsTheClaimsAndReadsTheAgreement (0.00s) + d84_window_client_test.go:44: an answer WITHOUT accepts_ended (an older hub) reads as a hub that knows end records + +2 mutations, 0 not a red-proof + +### the hub mutation done again +== MUTATION: REDO (the first mutant did not compile: `strings` unused): the empty body of the oldest boxes is refused + RED on the mutant, GREEN on the real file + --- FAIL: TestD84_WindowOpen_ABodyTheHubCannotReadIs400 (0.05s) + d84_window_skew_test.go:46: body "": HTTP 400, want 200 + d84_window_skew_test.go:52: body "": the answer the window request could not be read (not JSON, or cut) + +1 mutations, 0 not a red-proof diff --git a/hub/CHANGELOG.md b/hub/CHANGELOG.md index 5d1ad83e..0f009b9c 100644 --- a/hub/CHANGELOG.md +++ b/hub/CHANGELOG.md @@ -1,5 +1,12 @@ ## Unreleased (after v0.182.0) +- **Review fix 5 (2026-11-15), the hub's half — version skew.** (1) `POST /offsite/window-open` answers **400** on a + body it cannot read (not JSON, or cut at the read limit); it used to decide the window on „count 0”, i.e. with the + smallest cap, which then refused the honest plan. The empty body of the oldest boxes is still „no count”. + (2) Every answer carries **`accepts_ended: true`**, granted or not: a box (controller after 0.353.0) sends its end + records only to a hub that said so in its last answer, so a new box never sends a large request to an older hub. + Tests `TestD84_WindowOpen_ABodyTheHubCannotReadIs400` — seen red; 3 mutations seen red, 1 after a redo + (`…/A/redproof-review-5.txt`). - **Security fix 2 on review fix 1 (2026-11-15): the large request's wait is keyed on the WHOLE app set.** The key was the shortened display list (ten names „and N more”), so two different sets with the same first ten names shared one wait and one mail. `largeSetKey` is now a SHA-256 of the full sorted set; the display string is only diff --git a/hub/internal/api/d84_window_skew_test.go b/hub/internal/api/d84_window_skew_test.go new file mode 100644 index 00000000..d3c4afb2 --- /dev/null +++ b/hub/internal/api/d84_window_skew_test.go @@ -0,0 +1,55 @@ +package api + +import ( + "encoding/json" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "gitea.dooplex.hu/admin/felhom-hub/internal/store" +) + +// Review of 2026-11-15, finding 5 — version skew. A hub that cannot read a window request must say so (400), never +// decide a window on a count of 0; and it tells every box, in every answer, that it understands `ended` — a box sends +// its end records only to a hub that said so. +// RED-PROOF: documentation/audits/build-release-4-2026-11-15/A/redproof-review-5.txt. +func TestD84_WindowOpen_ABodyTheHubCannotReadIs400(t *testing.T) { + h, st, _ := newTestHandler(t) + st.SaveCustomerConfig(&store.CustomerConfig{CustomerID: "c1", APIKey: "ckey", RetrievalPassword: "pp"}) + f := &fakeKeySvc{} + h.SetOffsiteKeyService(f) + post := func(body string) *httptest.ResponseRecorder { + req := httptest.NewRequest(http.MethodPost, "/api/v1/offsite/window-open/c1", strings.NewReader(body)) + req.Header.Set("Authorization", "Bearer ckey") + rr := httptest.NewRecorder() + h.ServeHTTP(rr, req) + return rr + } + for name, body := range map[string]string{ + "cut in the middle (a read limit)": `{"count_before":20,"ended":[{"app":"vikunja","snapshot_ids":["aaaa","bb`, + "not JSON": `count_before=20`, + "a list, not an object": `[20]`, + } { + f.opened = 0 + if rr := post(body); rr.Code != http.StatusBadRequest { + t.Errorf("%s: HTTP %d, want 400 — the hub must not decide a window on a body it could not read", name, rr.Code) + } + if f.opened != 0 { + t.Errorf("%s: the window service was asked although the body could not be read", name) + } + } + // what every box has always sent still works — also the empty body of the oldest ones + for _, body := range []string{`{"count_before":20}`, ``} { + rr := post(body) + if rr.Code != http.StatusOK { + t.Errorf("body %q: HTTP %d, want 200", body, rr.Code) + } + // … and EVERY answer says that this hub understands ended apps — here a window that is NOT granted + var ans map[string]any + _ = json.Unmarshal(rr.Body.Bytes(), &ans) + if ans["accepts_ended"] != true || ans["granted"] != false { + t.Errorf("body %q: the answer %s does not say accepts_ended — a new box would never send its end records", body, rr.Body.String()) + } + } +} diff --git a/hub/internal/api/offsite.go b/hub/internal/api/offsite.go index c9738549..6f9c0bcd 100644 --- a/hub/internal/api/offsite.go +++ b/hub/internal/api/offsite.go @@ -6,6 +6,7 @@ import ( "errors" "io" "net/http" + "strings" "time" "gitea.dooplex.hu/admin/felhom-hub/internal/offsitekeys" @@ -141,10 +142,17 @@ func (h *Handler) handleOffsiteWindowOpen(w http.ResponseWriter, r *http.Request // agrees per id from its OWN records (offsitekeys/ended.go); the box's dates decide nothing. Ended []offsitekeys.EndedClaim `json:"ended"` } - // 1 MiB: a snapshot id is 64 characters, and a removed app can hold a few hundred snapshots. A body that is cut - // does not parse, and then nothing is claimed — the window is still decided on count_before 0, as before. + // 1 MiB: a snapshot id is 64 characters, and a removed app can hold a few hundred snapshots. + // A body the hub cannot read is REFUSED (review of 2026-11-15): until then a body that was cut at the read limit, + // or was not JSON, was decided as „count_before 0” — a window opened with the smallest cap, which then refused + // the honest plan every week. The empty body of the oldest boxes is still „no count”. body, _ := io.ReadAll(io.LimitReader(r.Body, 1<<20)) - _ = json.Unmarshal(body, &req) + if len(strings.TrimSpace(string(body))) > 0 { + if err := json.Unmarshal(body, &req); err != nil { + http.Error(w, "the window request could not be read (not JSON, or cut)", http.StatusBadRequest) + return + } + } ctx, cancel := context.WithTimeout(r.Context(), 2*time.Minute) defer cancel() g, err := h.offsiteKeys.OpenWindowFor(ctx, customerID, req.CountBefore, req.Ended...) @@ -152,6 +160,7 @@ func (h *Handler) handleOffsiteWindowOpen(w http.ResponseWriter, r *http.Request offsiteKeyErr(w, err) return } + g.AcceptsEnded = true // every answer of this hub, granted or not writeJSON(w, http.StatusOK, g) } diff --git a/hub/internal/api/offsite_test.go b/hub/internal/api/offsite_test.go index 97f86569..482db505 100644 --- a/hub/internal/api/offsite_test.go +++ b/hub/internal/api/offsite_test.go @@ -55,6 +55,7 @@ type fakeKeySvc struct { gotPub string gotFP string gotEnded []offsitekeys.EndedClaim + opened int err error } @@ -68,6 +69,7 @@ func (f *fakeKeySvc) ConfirmKey(_ context.Context, _ string, fp string) (int, er } func (f *fakeKeySvc) OpenWindowFor(_ context.Context, _ string, _ int, ended ...offsitekeys.EndedClaim) (offsitekeys.WindowGrant, error) { f.gotEnded = ended + f.opened++ return offsitekeys.WindowGrant{Granted: false, Reason: "not due"}, f.err } func (f *fakeKeySvc) CloseWindowFor(context.Context, string, offsitekeys.WindowResult) error { diff --git a/hub/internal/offsitekeys/service.go b/hub/internal/offsitekeys/service.go index 0aa78ba5..568e91b5 100644 --- a/hub/internal/offsitekeys/service.go +++ b/hub/internal/offsitekeys/service.go @@ -276,6 +276,10 @@ type WindowGrant struct { // refused. A box removes an ended app's snapshot ONLY when its id is in EndedAllowed. EndedAllowed []string `json:"ended_allowed,omitempty"` EndedRefused []store.EndedRefusal `json:"ended_refused,omitempty"` + // AcceptsEnded tells the box that THIS hub reads `ended` in a window request (review of 2026-11-15, version + // skew). It is in every answer, granted or not — the API handler sets it. A box sends its end records only to a + // hub that said so in its last answer: an older hub reads 4 KiB of the request and would stall on a larger one. + AcceptsEnded bool `json:"accepts_ended"` } // WindowResult is the box's report when it is done.