hub: review fix 5 — a window request the hub cannot read is 400; every answer says accepts_ended (version skew)

- POST /offsite/window-open answered 200 on a body it could not parse and
  decided the window on „count 0”; now 400 (the empty body of the oldest boxes
  is still „no count”).
- Every answer carries accepts_ended, granted or not: a box sends its end
  records only to a hub that said so, so a new box never sends a large request
  to an older hub that reads 4 KiB.

Carries the red-proof of both halves (the controller's is the next commit
there). Tests seen red; 3 hub mutations seen red, 1 after a redo.

Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
admin committed 2026-11-15 14:57:24 +01:00
1 parent 6d2d4bd09d
commit 45848f6ee4
8 files changed
+168 -5

No files matched your search

@@ -921,7 +921,7 @@ method (405), so a token gets 403.
| `/configs/{id}/edit` | `customers:read` | **session only** | Edit can change the household's e-mail — and a new e-mail auto-mails the self-bind link (ruling 13:20). |
| `/offsite/remove-unpinned/{id}` | — | **session only** | |
| `/offsite/abandon-cancel/{id}` | — | **session only** | Cancelling a pending set-aside deletion: no group names it; until the operator places it, session only. |
| `/offsite/window-grant/{id}` | — | `offsite:write` | The one-shot clean-up window: the script address AND the customer page's „Backup & DR" card (R-1299, hub v0.174.0): `from=page` + `confirm=1` from the form; `cancel=1` removes a waiting grant; a limit above the store's snapshot count is refused. **Built, not released (D72/D84, `07` §8.2f):** the box-facing `POST /api/v1/offsite/window-open/{id}` also takes the box's due off-site end records (`ended`) and answers `ended_allowed` / `ended_refused` — ids of removed apps' snapshots the hub agrees to from its own records, beside the window's cap; `window-close` takes `removed_for_ended_apps`. The card's „Last window” shows what was agreed, removed and refused. |
| `/offsite/window-grant/{id}` | — | `offsite:write` | The one-shot clean-up window: the script address AND the customer page's „Backup & DR" card (R-1299, hub v0.174.0): `from=page` + `confirm=1` from the form; `cancel=1` removes a waiting grant; a limit above the store's snapshot count is refused. **Built, not released (D72/D84, `07` §8.2f):** the box-facing `POST /api/v1/offsite/window-open/{id}` also takes the box's due off-site end records (`ended`) and answers `ended_allowed` / `ended_refused` — ids of removed apps' snapshots the hub agrees to from its own records, beside the window's cap; `window-close` takes `removed_for_ended_apps`. Every window-open answer carries `accepts_ended` (a box sends `ended` only to a hub that said it), and a request the hub cannot read is answered 400. The card's „Last window” shows what was agreed, removed and refused. |
| `/offsite/windows-enabled` | — | `offsite:write` | |
| `/os/{id}` | `os:read` | `os:write` | |
| `/offsite/key-audit` | — | `offsite:read` | Runs the daily key check now: it reads keys and reports, it changes none. |
@@ -1917,7 +1917,7 @@ notification address gets no mail: the page is all it has.** A blocked household
| A record with a date in the past, written by a box the hub does not agree with | Refused: the hub never reads the box's dates | hub `TestD84_AgreeEnded…`, `TestD72_ReportFeedsTheHubsOwnAppRecord` |
| An app removed with „delete backup data” BEFORE this is built | No record exists, so nothing is deleted. The off-site list shows the app with „delete in 30 days”: the household starts the clock itself | `TestD72_RestorePageShowsTheEndPerBranch` |
| The guard refuses the window (a future-dated snapshot, a plan above the cap) | The ended apps wait with everything else | `TestD84_OneGuard_AndTheAllowanceIsBesideTheCap` |
| The hub does not know ended apps (an older hub), or the box does not (an older controller) | Nothing of an ended app is removed; each side is releasable alone | `TestD84_TheBoxRemovesOnlyWhatTheHubAgreedTo` |
| The hub does not know ended apps (an older hub), or the box does not (an older controller) | Nothing of an ended app is removed, and the ordinary weekly clean-up goes on: the box sends its end records only to a hub whose last window answer said `accepts_ended` (an older hub reads 4 KiB of the request and would stall on more); a hub answers 400 on a request it cannot read; one request carries at most 5000 ids | `TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns`, `TestD84_OneRequestCarriesABoundedNumberOfIds`, hub `TestD84_WindowOpen_ABodyTheHubCannotReadIs400` |
| The app has no copy in the store at all | The due record is closed with no history line; nothing is asked of the hub | `TestD72_ARecordWithNoCopiesIsClosedQuietly` |
| A take-back or an install while the window is being prepared (the hub answers, the dry run, the guard) | The window reads the records AGAIN immediately before the forget; an id whose record is gone or changed is not removed. A take-back during the forget itself is answered „not marked, or already deleted”. **Not covered:** an install during the forget | `TestD72_ATakeBackDuringTheWindowKeepsTheCopies`, `TestD72_AnInstallDuringTheWindowKeepsTheCopies`, `TestD72_ATakeBackDuringTheForgetIsNotAnsweredKept` |
| The app is installed again by an import, or by a restore that installs from a unit | The record is deleted at the ONE point every install passes (`stacks.SaveAppConfig`) | `TestD72_AnImportDeletesTheEndRecord`, `TestD72_ARestoreIntoAnInstallDeletesTheEndRecord`, `TestD72_TheInstallWritersAreListed` |
@@ -0,0 +1,86 @@
FINDING 5 — version skew: a new box with an older hub can stall the weekly clean-up; an unreadable request was decided as count 0.
### RED, hub: the new tests before the fix (felhom.eu 6d2d4bd0)
=== RUN TestD84_WindowOpen_ABodyTheHubCannotReadIs400
d84_window_skew_test.go:37: cut in the middle (a read limit): HTTP 200, want 400 — the hub must not decide a window on a body it could not read
d84_window_skew_test.go:40: cut in the middle (a read limit): the window service was asked although the body could not be read
d84_window_skew_test.go:37: not JSON: HTTP 200, want 400 — the hub must not decide a window on a body it could not read
d84_window_skew_test.go:40: not JSON: the window service was asked although the body could not be read
d84_window_skew_test.go:37: a list, not an object: HTTP 200, want 400 — the hub must not decide a window on a body it could not read
d84_window_skew_test.go:40: a list, not an object: the window service was asked although the body could not be read
--- FAIL: TestD84_WindowOpen_ABodyTheHubCannotReadIs400 (0.05s)
=== RUN TestD84_WindowAnswerSaysTheHubUnderstandsEnded
d84_window_skew_test.go:56: the answer {"granted":false,"newest_allowed":"0001-01-01T00:00:00Z","reason":"not due"} does not say accepts_ended — a new box would never send its end records
d84_window_skew_test.go:56: the answer {"granted":true,"newest_allowed":"0001-01-01T00:00:00Z"} does not say accepts_ended — a new box would never send its end records
--- FAIL: TestD84_WindowAnswerSaysTheHubUnderstandsEnded (0.00s)
FAIL
FAIL gitea.dooplex.hu/admin/felhom-hub/internal/api 0.076s
FAIL
### RED, controller: the new tests before the fix (felhom-controller c7e4ec9)
=== RUN TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns
offbox_ended_skew_test.go:29: week 1: the box sent 1 end record(s) to a hub that never said it reads them — that hub reads 4 KiB and would stall
--- FAIL: TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns (0.00s)
=== RUN TestD84_OneRequestCarriesABoundedNumberOfIds
offbox_ended_skew_test.go:83: one request carried 5200 snapshot ids, want the limit 5000
--- FAIL: TestD84_OneRequestCarriesABoundedNumberOfIds (0.03s)
FAIL
FAIL gitea.dooplex.hu/admin/felhom-controller/internal/backup 0.050s
FAIL
### After the fix: mutations, hub
== MUTATION: a body that cannot be read is decided as count 0 again
RED on the mutant, GREEN on the real file
--- FAIL: TestD84_WindowOpen_ABodyTheHubCannotReadIs400 (0.05s)
d84_window_skew_test.go:36: cut in the middle (a read limit): HTTP 200, want 400 — the hub must not decide a window on a body it could not read
d84_window_skew_test.go:39: cut in the middle (a read limit): the window service was asked although the body could not be read
d84_window_skew_test.go:36: not JSON: HTTP 200, want 400 — the hub must not decide a window on a body it could not read
== MUTATION: the answer does not say that the hub understands ended apps
RED on the mutant, GREEN on the real file
--- FAIL: TestD84_WindowOpen_ABodyTheHubCannotReadIs400 (0.05s)
d84_window_skew_test.go:52: body "{\"count_before\":20}": the answer {"granted":false,"newest_allowed":"0001-01-01T00:00:00Z","reason":"not due","accepts_ended":false}
d84_window_skew_test.go:52: body "": the answer {"granted":false,"newest_allowed":"0001-01-01T00:00:00Z","reason":"not due","accepts_ended":false}
== MUTATION: the empty body of the oldest boxes is refused
*** NOT A RED-PROOF *** (mutant rc=1 built=False real rc=0)
FAIL gitea.dooplex.hu/admin/felhom-hub/internal/api [build failed]
3 mutations, 1 not a red-proof
### After the fix: mutations, controller
== MUTATION: the box sends its end records to any hub
RED on the mutant, GREEN on the real file
--- FAIL: TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns (0.00s)
offbox_ended_skew_test.go:29: week 1: the box sent 1 end record(s) to a hub that never said it reads them — that hub reads 4 KiB and would stall
== MUTATION: the box does not remember the hub's answer
RED on the mutant, GREEN on the real file
--- FAIL: TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns (0.00s)
offbox_ended_skew_test.go:49: the night the box LEARNS it: claims=0 learned=false — it must send none yet and remember the answer
== MUTATION: the box only ever learns „yes” (a rolled-back hub keeps getting end records)
RED on the mutant, GREEN on the real file
--- FAIL: TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns (0.00s)
offbox_ended_skew_test.go:65: after a rolled-back hub's answer the box still sends end records: claims=0 learned=true
== MUTATION: no limit on the ids of one request
RED on the mutant, GREEN on the real file
--- FAIL: TestD84_OneRequestCarriesABoundedNumberOfIds (0.03s)
offbox_ended_skew_test.go:84: one request carried 5200 snapshot ids, want the limit 5000
4 mutations, 0 not a red-proof
== MUTATION: the client does not read accepts_ended
RED on the mutant, GREEN on the real file
--- FAIL: TestD84_WindowClient_SendsTheClaimsAndReadsTheAgreement (0.00s)
d84_window_client_test.go:63: accepts_ended of a not-granted answer was not read
== MUTATION: the client reads every hub as one that knows end records
RED on the mutant, GREEN on the real file
--- FAIL: TestD84_WindowClient_SendsTheClaimsAndReadsTheAgreement (0.00s)
d84_window_client_test.go:44: an answer WITHOUT accepts_ended (an older hub) reads as a hub that knows end records
2 mutations, 0 not a red-proof
### the hub mutation done again
== MUTATION: REDO (the first mutant did not compile: `strings` unused): the empty body of the oldest boxes is refused
RED on the mutant, GREEN on the real file
--- FAIL: TestD84_WindowOpen_ABodyTheHubCannotReadIs400 (0.05s)
d84_window_skew_test.go:46: body "": HTTP 400, want 200
d84_window_skew_test.go:52: body "": the answer the window request could not be read (not JSON, or cut)
1 mutations, 0 not a red-proof
+7
View File
@@ -1,5 +1,12 @@
## Unreleased (after v0.182.0)
- **Review fix 5 (2026-11-15), the hub's half — version skew.** (1) `POST /offsite/window-open` answers **400** on a
body it cannot read (not JSON, or cut at the read limit); it used to decide the window on „count 0”, i.e. with the
smallest cap, which then refused the honest plan. The empty body of the oldest boxes is still „no count”.
(2) Every answer carries **`accepts_ended: true`**, granted or not: a box (controller after 0.353.0) sends its end
records only to a hub that said so in its last answer, so a new box never sends a large request to an older hub.
Tests `TestD84_WindowOpen_ABodyTheHubCannotReadIs400` — seen red; 3 mutations seen red, 1 after a redo
(`…/A/redproof-review-5.txt`).
- **Security fix 2 on review fix 1 (2026-11-15): the large request's wait is keyed on the WHOLE app set.** The key
was the shortened display list (ten names „and N more”), so two different sets with the same first ten names
shared one wait and one mail. `largeSetKey` is now a SHA-256 of the full sorted set; the display string is only
+55
View File
@@ -0,0 +1,55 @@
package api
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
)
// Review of 2026-11-15, finding 5 — version skew. A hub that cannot read a window request must say so (400), never
// decide a window on a count of 0; and it tells every box, in every answer, that it understands `ended` — a box sends
// its end records only to a hub that said so.
// RED-PROOF: documentation/audits/build-release-4-2026-11-15/A/redproof-review-5.txt.
func TestD84_WindowOpen_ABodyTheHubCannotReadIs400(t *testing.T) {
h, st, _ := newTestHandler(t)
st.SaveCustomerConfig(&store.CustomerConfig{CustomerID: "c1", APIKey: "ckey", RetrievalPassword: "pp"})
f := &fakeKeySvc{}
h.SetOffsiteKeyService(f)
post := func(body string) *httptest.ResponseRecorder {
req := httptest.NewRequest(http.MethodPost, "/api/v1/offsite/window-open/c1", strings.NewReader(body))
req.Header.Set("Authorization", "Bearer ckey")
rr := httptest.NewRecorder()
h.ServeHTTP(rr, req)
return rr
}
for name, body := range map[string]string{
"cut in the middle (a read limit)": `{"count_before":20,"ended":[{"app":"vikunja","snapshot_ids":["aaaa","bb`,
"not JSON": `count_before=20`,
"a list, not an object": `[20]`,
} {
f.opened = 0
if rr := post(body); rr.Code != http.StatusBadRequest {
t.Errorf("%s: HTTP %d, want 400 — the hub must not decide a window on a body it could not read", name, rr.Code)
}
if f.opened != 0 {
t.Errorf("%s: the window service was asked although the body could not be read", name)
}
}
// what every box has always sent still works — also the empty body of the oldest ones
for _, body := range []string{`{"count_before":20}`, ``} {
rr := post(body)
if rr.Code != http.StatusOK {
t.Errorf("body %q: HTTP %d, want 200", body, rr.Code)
}
// … and EVERY answer says that this hub understands ended apps — here a window that is NOT granted
var ans map[string]any
_ = json.Unmarshal(rr.Body.Bytes(), &ans)
if ans["accepts_ended"] != true || ans["granted"] != false {
t.Errorf("body %q: the answer %s does not say accepts_ended — a new box would never send its end records", body, rr.Body.String())
}
}
}
+12 -3
View File
@@ -6,6 +6,7 @@ import (
"errors"
"io"
"net/http"
"strings"
"time"
"gitea.dooplex.hu/admin/felhom-hub/internal/offsitekeys"
@@ -141,10 +142,17 @@ func (h *Handler) handleOffsiteWindowOpen(w http.ResponseWriter, r *http.Request
// agrees per id from its OWN records (offsitekeys/ended.go); the box's dates decide nothing.
Ended []offsitekeys.EndedClaim `json:"ended"`
}
// 1 MiB: a snapshot id is 64 characters, and a removed app can hold a few hundred snapshots. A body that is cut
// does not parse, and then nothing is claimed — the window is still decided on count_before 0, as before.
// 1 MiB: a snapshot id is 64 characters, and a removed app can hold a few hundred snapshots.
// A body the hub cannot read is REFUSED (review of 2026-11-15): until then a body that was cut at the read limit,
// or was not JSON, was decided as „count_before 0” — a window opened with the smallest cap, which then refused
// the honest plan every week. The empty body of the oldest boxes is still „no count”.
body, _ := io.ReadAll(io.LimitReader(r.Body, 1<<20))
_ = json.Unmarshal(body, &req)
if len(strings.TrimSpace(string(body))) > 0 {
if err := json.Unmarshal(body, &req); err != nil {
http.Error(w, "the window request could not be read (not JSON, or cut)", http.StatusBadRequest)
return
}
}
ctx, cancel := context.WithTimeout(r.Context(), 2*time.Minute)
defer cancel()
g, err := h.offsiteKeys.OpenWindowFor(ctx, customerID, req.CountBefore, req.Ended...)
@@ -152,6 +160,7 @@ func (h *Handler) handleOffsiteWindowOpen(w http.ResponseWriter, r *http.Request
offsiteKeyErr(w, err)
return
}
g.AcceptsEnded = true // every answer of this hub, granted or not
writeJSON(w, http.StatusOK, g)
}
+2
View File
@@ -55,6 +55,7 @@ type fakeKeySvc struct {
gotPub string
gotFP string
gotEnded []offsitekeys.EndedClaim
opened int
err error
}
@@ -68,6 +69,7 @@ func (f *fakeKeySvc) ConfirmKey(_ context.Context, _ string, fp string) (int, er
}
func (f *fakeKeySvc) OpenWindowFor(_ context.Context, _ string, _ int, ended ...offsitekeys.EndedClaim) (offsitekeys.WindowGrant, error) {
f.gotEnded = ended
f.opened++
return offsitekeys.WindowGrant{Granted: false, Reason: "not due"}, f.err
}
func (f *fakeKeySvc) CloseWindowFor(context.Context, string, offsitekeys.WindowResult) error {
+4
View File
@@ -276,6 +276,10 @@ type WindowGrant struct {
// refused. A box removes an ended app's snapshot ONLY when its id is in EndedAllowed.
EndedAllowed []string `json:"ended_allowed,omitempty"`
EndedRefused []store.EndedRefusal `json:"ended_refused,omitempty"`
// AcceptsEnded tells the box that THIS hub reads `ended` in a window request (review of 2026-11-15, version
// skew). It is in every answer, granted or not — the API handler sets it. A box sends its end records only to a
// hub that said so in its last answer: an older hub reads 4 KiB of the request and would stall on a larger one.
AcceptsEnded bool `json:"accepts_ended"`
}
// WindowResult is the box's report when it is done.