hub: review fix 5 — a window request the hub cannot read is 400; every answer says accepts_ended (version skew)
- POST /offsite/window-open answered 200 on a body it could not parse and decided the window on „count 0”; now 400 (the empty body of the oldest boxes is still „no count”). - Every answer carries accepts_ended, granted or not: a box sends its end records only to a hub that said so, so a new box never sends a large request to an older hub that reads 4 KiB. Carries the red-proof of both halves (the controller's is the next commit there). Tests seen red; 3 hub mutations seen red, 1 after a redo. Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
1 parent
6d2d4bd09d
commit
45848f6ee4
8 files changed
+168
-5
No files matched your search
@@ -921,7 +921,7 @@ method (405), so a token gets 403.
|
||||
| `/configs/{id}/edit` | `customers:read` | **session only** | Edit can change the household's e-mail — and a new e-mail auto-mails the self-bind link (ruling 13:20). |
|
||||
| `/offsite/remove-unpinned/{id}` | — | **session only** | |
|
||||
| `/offsite/abandon-cancel/{id}` | — | **session only** | Cancelling a pending set-aside deletion: no group names it; until the operator places it, session only. |
|
||||
| `/offsite/window-grant/{id}` | — | `offsite:write` | The one-shot clean-up window: the script address AND the customer page's „Backup & DR" card (R-1299, hub v0.174.0): `from=page` + `confirm=1` from the form; `cancel=1` removes a waiting grant; a limit above the store's snapshot count is refused. **Built, not released (D72/D84, `07` §8.2f):** the box-facing `POST /api/v1/offsite/window-open/{id}` also takes the box's due off-site end records (`ended`) and answers `ended_allowed` / `ended_refused` — ids of removed apps' snapshots the hub agrees to from its own records, beside the window's cap; `window-close` takes `removed_for_ended_apps`. The card's „Last window” shows what was agreed, removed and refused. |
|
||||
| `/offsite/window-grant/{id}` | — | `offsite:write` | The one-shot clean-up window: the script address AND the customer page's „Backup & DR" card (R-1299, hub v0.174.0): `from=page` + `confirm=1` from the form; `cancel=1` removes a waiting grant; a limit above the store's snapshot count is refused. **Built, not released (D72/D84, `07` §8.2f):** the box-facing `POST /api/v1/offsite/window-open/{id}` also takes the box's due off-site end records (`ended`) and answers `ended_allowed` / `ended_refused` — ids of removed apps' snapshots the hub agrees to from its own records, beside the window's cap; `window-close` takes `removed_for_ended_apps`. Every window-open answer carries `accepts_ended` (a box sends `ended` only to a hub that said it), and a request the hub cannot read is answered 400. The card's „Last window” shows what was agreed, removed and refused. |
|
||||
| `/offsite/windows-enabled` | — | `offsite:write` | |
|
||||
| `/os/{id}` | `os:read` | `os:write` | |
|
||||
| `/offsite/key-audit` | — | `offsite:read` | Runs the daily key check now: it reads keys and reports, it changes none. |
|
||||
|
||||
@@ -1917,7 +1917,7 @@ notification address gets no mail: the page is all it has.** A blocked household
|
||||
| A record with a date in the past, written by a box the hub does not agree with | Refused: the hub never reads the box's dates | hub `TestD84_AgreeEnded…`, `TestD72_ReportFeedsTheHubsOwnAppRecord` |
|
||||
| An app removed with „delete backup data” BEFORE this is built | No record exists, so nothing is deleted. The off-site list shows the app with „delete in 30 days”: the household starts the clock itself | `TestD72_RestorePageShowsTheEndPerBranch` |
|
||||
| The guard refuses the window (a future-dated snapshot, a plan above the cap) | The ended apps wait with everything else | `TestD84_OneGuard_AndTheAllowanceIsBesideTheCap` |
|
||||
| The hub does not know ended apps (an older hub), or the box does not (an older controller) | Nothing of an ended app is removed; each side is releasable alone | `TestD84_TheBoxRemovesOnlyWhatTheHubAgreedTo` |
|
||||
| The hub does not know ended apps (an older hub), or the box does not (an older controller) | Nothing of an ended app is removed, and the ordinary weekly clean-up goes on: the box sends its end records only to a hub whose last window answer said `accepts_ended` (an older hub reads 4 KiB of the request and would stall on more); a hub answers 400 on a request it cannot read; one request carries at most 5000 ids | `TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns`, `TestD84_OneRequestCarriesABoundedNumberOfIds`, hub `TestD84_WindowOpen_ABodyTheHubCannotReadIs400` |
|
||||
| The app has no copy in the store at all | The due record is closed with no history line; nothing is asked of the hub | `TestD72_ARecordWithNoCopiesIsClosedQuietly` |
|
||||
| A take-back or an install while the window is being prepared (the hub answers, the dry run, the guard) | The window reads the records AGAIN immediately before the forget; an id whose record is gone or changed is not removed. A take-back during the forget itself is answered „not marked, or already deleted”. **Not covered:** an install during the forget | `TestD72_ATakeBackDuringTheWindowKeepsTheCopies`, `TestD72_AnInstallDuringTheWindowKeepsTheCopies`, `TestD72_ATakeBackDuringTheForgetIsNotAnsweredKept` |
|
||||
| The app is installed again by an import, or by a restore that installs from a unit | The record is deleted at the ONE point every install passes (`stacks.SaveAppConfig`) | `TestD72_AnImportDeletesTheEndRecord`, `TestD72_ARestoreIntoAnInstallDeletesTheEndRecord`, `TestD72_TheInstallWritersAreListed` |
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
FINDING 5 — version skew: a new box with an older hub can stall the weekly clean-up; an unreadable request was decided as count 0.
|
||||
|
||||
### RED, hub: the new tests before the fix (felhom.eu 6d2d4bd0)
|
||||
=== RUN TestD84_WindowOpen_ABodyTheHubCannotReadIs400
|
||||
d84_window_skew_test.go:37: cut in the middle (a read limit): HTTP 200, want 400 — the hub must not decide a window on a body it could not read
|
||||
d84_window_skew_test.go:40: cut in the middle (a read limit): the window service was asked although the body could not be read
|
||||
d84_window_skew_test.go:37: not JSON: HTTP 200, want 400 — the hub must not decide a window on a body it could not read
|
||||
d84_window_skew_test.go:40: not JSON: the window service was asked although the body could not be read
|
||||
d84_window_skew_test.go:37: a list, not an object: HTTP 200, want 400 — the hub must not decide a window on a body it could not read
|
||||
d84_window_skew_test.go:40: a list, not an object: the window service was asked although the body could not be read
|
||||
--- FAIL: TestD84_WindowOpen_ABodyTheHubCannotReadIs400 (0.05s)
|
||||
=== RUN TestD84_WindowAnswerSaysTheHubUnderstandsEnded
|
||||
d84_window_skew_test.go:56: the answer {"granted":false,"newest_allowed":"0001-01-01T00:00:00Z","reason":"not due"} does not say accepts_ended — a new box would never send its end records
|
||||
d84_window_skew_test.go:56: the answer {"granted":true,"newest_allowed":"0001-01-01T00:00:00Z"} does not say accepts_ended — a new box would never send its end records
|
||||
--- FAIL: TestD84_WindowAnswerSaysTheHubUnderstandsEnded (0.00s)
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-hub/internal/api 0.076s
|
||||
FAIL
|
||||
|
||||
### RED, controller: the new tests before the fix (felhom-controller c7e4ec9)
|
||||
=== RUN TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns
|
||||
offbox_ended_skew_test.go:29: week 1: the box sent 1 end record(s) to a hub that never said it reads them — that hub reads 4 KiB and would stall
|
||||
--- FAIL: TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns (0.00s)
|
||||
=== RUN TestD84_OneRequestCarriesABoundedNumberOfIds
|
||||
offbox_ended_skew_test.go:83: one request carried 5200 snapshot ids, want the limit 5000
|
||||
--- FAIL: TestD84_OneRequestCarriesABoundedNumberOfIds (0.03s)
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-controller/internal/backup 0.050s
|
||||
FAIL
|
||||
|
||||
### After the fix: mutations, hub
|
||||
== MUTATION: a body that cannot be read is decided as count 0 again
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestD84_WindowOpen_ABodyTheHubCannotReadIs400 (0.05s)
|
||||
d84_window_skew_test.go:36: cut in the middle (a read limit): HTTP 200, want 400 — the hub must not decide a window on a body it could not read
|
||||
d84_window_skew_test.go:39: cut in the middle (a read limit): the window service was asked although the body could not be read
|
||||
d84_window_skew_test.go:36: not JSON: HTTP 200, want 400 — the hub must not decide a window on a body it could not read
|
||||
== MUTATION: the answer does not say that the hub understands ended apps
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestD84_WindowOpen_ABodyTheHubCannotReadIs400 (0.05s)
|
||||
d84_window_skew_test.go:52: body "{\"count_before\":20}": the answer {"granted":false,"newest_allowed":"0001-01-01T00:00:00Z","reason":"not due","accepts_ended":false}
|
||||
d84_window_skew_test.go:52: body "": the answer {"granted":false,"newest_allowed":"0001-01-01T00:00:00Z","reason":"not due","accepts_ended":false}
|
||||
== MUTATION: the empty body of the oldest boxes is refused
|
||||
*** NOT A RED-PROOF *** (mutant rc=1 built=False real rc=0)
|
||||
FAIL gitea.dooplex.hu/admin/felhom-hub/internal/api [build failed]
|
||||
|
||||
3 mutations, 1 not a red-proof
|
||||
|
||||
### After the fix: mutations, controller
|
||||
== MUTATION: the box sends its end records to any hub
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns (0.00s)
|
||||
offbox_ended_skew_test.go:29: week 1: the box sent 1 end record(s) to a hub that never said it reads them — that hub reads 4 KiB and would stall
|
||||
== MUTATION: the box does not remember the hub's answer
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns (0.00s)
|
||||
offbox_ended_skew_test.go:49: the night the box LEARNS it: claims=0 learned=false — it must send none yet and remember the answer
|
||||
== MUTATION: the box only ever learns „yes” (a rolled-back hub keeps getting end records)
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestD84_ANewBoxWithAnOlderHub_NothingEndedIsSentAndTheCleanUpStillRuns (0.00s)
|
||||
offbox_ended_skew_test.go:65: after a rolled-back hub's answer the box still sends end records: claims=0 learned=true
|
||||
== MUTATION: no limit on the ids of one request
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestD84_OneRequestCarriesABoundedNumberOfIds (0.03s)
|
||||
offbox_ended_skew_test.go:84: one request carried 5200 snapshot ids, want the limit 5000
|
||||
|
||||
4 mutations, 0 not a red-proof
|
||||
== MUTATION: the client does not read accepts_ended
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestD84_WindowClient_SendsTheClaimsAndReadsTheAgreement (0.00s)
|
||||
d84_window_client_test.go:63: accepts_ended of a not-granted answer was not read
|
||||
== MUTATION: the client reads every hub as one that knows end records
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestD84_WindowClient_SendsTheClaimsAndReadsTheAgreement (0.00s)
|
||||
d84_window_client_test.go:44: an answer WITHOUT accepts_ended (an older hub) reads as a hub that knows end records
|
||||
|
||||
2 mutations, 0 not a red-proof
|
||||
|
||||
### the hub mutation done again
|
||||
== MUTATION: REDO (the first mutant did not compile: `strings` unused): the empty body of the oldest boxes is refused
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestD84_WindowOpen_ABodyTheHubCannotReadIs400 (0.05s)
|
||||
d84_window_skew_test.go:46: body "": HTTP 400, want 200
|
||||
d84_window_skew_test.go:52: body "": the answer the window request could not be read (not JSON, or cut)
|
||||
|
||||
1 mutations, 0 not a red-proof
|
||||
@@ -1,5 +1,12 @@
|
||||
## Unreleased (after v0.182.0)
|
||||
|
||||
- **Review fix 5 (2026-11-15), the hub's half — version skew.** (1) `POST /offsite/window-open` answers **400** on a
|
||||
body it cannot read (not JSON, or cut at the read limit); it used to decide the window on „count 0”, i.e. with the
|
||||
smallest cap, which then refused the honest plan. The empty body of the oldest boxes is still „no count”.
|
||||
(2) Every answer carries **`accepts_ended: true`**, granted or not: a box (controller after 0.353.0) sends its end
|
||||
records only to a hub that said so in its last answer, so a new box never sends a large request to an older hub.
|
||||
Tests `TestD84_WindowOpen_ABodyTheHubCannotReadIs400` — seen red; 3 mutations seen red, 1 after a redo
|
||||
(`…/A/redproof-review-5.txt`).
|
||||
- **Security fix 2 on review fix 1 (2026-11-15): the large request's wait is keyed on the WHOLE app set.** The key
|
||||
was the shortened display list (ten names „and N more”), so two different sets with the same first ten names
|
||||
shared one wait and one mail. `largeSetKey` is now a SHA-256 of the full sorted set; the display string is only
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
|
||||
)
|
||||
|
||||
// Review of 2026-11-15, finding 5 — version skew. A hub that cannot read a window request must say so (400), never
|
||||
// decide a window on a count of 0; and it tells every box, in every answer, that it understands `ended` — a box sends
|
||||
// its end records only to a hub that said so.
|
||||
// RED-PROOF: documentation/audits/build-release-4-2026-11-15/A/redproof-review-5.txt.
|
||||
func TestD84_WindowOpen_ABodyTheHubCannotReadIs400(t *testing.T) {
|
||||
h, st, _ := newTestHandler(t)
|
||||
st.SaveCustomerConfig(&store.CustomerConfig{CustomerID: "c1", APIKey: "ckey", RetrievalPassword: "pp"})
|
||||
f := &fakeKeySvc{}
|
||||
h.SetOffsiteKeyService(f)
|
||||
post := func(body string) *httptest.ResponseRecorder {
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/v1/offsite/window-open/c1", strings.NewReader(body))
|
||||
req.Header.Set("Authorization", "Bearer ckey")
|
||||
rr := httptest.NewRecorder()
|
||||
h.ServeHTTP(rr, req)
|
||||
return rr
|
||||
}
|
||||
for name, body := range map[string]string{
|
||||
"cut in the middle (a read limit)": `{"count_before":20,"ended":[{"app":"vikunja","snapshot_ids":["aaaa","bb`,
|
||||
"not JSON": `count_before=20`,
|
||||
"a list, not an object": `[20]`,
|
||||
} {
|
||||
f.opened = 0
|
||||
if rr := post(body); rr.Code != http.StatusBadRequest {
|
||||
t.Errorf("%s: HTTP %d, want 400 — the hub must not decide a window on a body it could not read", name, rr.Code)
|
||||
}
|
||||
if f.opened != 0 {
|
||||
t.Errorf("%s: the window service was asked although the body could not be read", name)
|
||||
}
|
||||
}
|
||||
// what every box has always sent still works — also the empty body of the oldest ones
|
||||
for _, body := range []string{`{"count_before":20}`, ``} {
|
||||
rr := post(body)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Errorf("body %q: HTTP %d, want 200", body, rr.Code)
|
||||
}
|
||||
// … and EVERY answer says that this hub understands ended apps — here a window that is NOT granted
|
||||
var ans map[string]any
|
||||
_ = json.Unmarshal(rr.Body.Bytes(), &ans)
|
||||
if ans["accepts_ended"] != true || ans["granted"] != false {
|
||||
t.Errorf("body %q: the answer %s does not say accepts_ended — a new box would never send its end records", body, rr.Body.String())
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-hub/internal/offsitekeys"
|
||||
@@ -141,10 +142,17 @@ func (h *Handler) handleOffsiteWindowOpen(w http.ResponseWriter, r *http.Request
|
||||
// agrees per id from its OWN records (offsitekeys/ended.go); the box's dates decide nothing.
|
||||
Ended []offsitekeys.EndedClaim `json:"ended"`
|
||||
}
|
||||
// 1 MiB: a snapshot id is 64 characters, and a removed app can hold a few hundred snapshots. A body that is cut
|
||||
// does not parse, and then nothing is claimed — the window is still decided on count_before 0, as before.
|
||||
// 1 MiB: a snapshot id is 64 characters, and a removed app can hold a few hundred snapshots.
|
||||
// A body the hub cannot read is REFUSED (review of 2026-11-15): until then a body that was cut at the read limit,
|
||||
// or was not JSON, was decided as „count_before 0” — a window opened with the smallest cap, which then refused
|
||||
// the honest plan every week. The empty body of the oldest boxes is still „no count”.
|
||||
body, _ := io.ReadAll(io.LimitReader(r.Body, 1<<20))
|
||||
_ = json.Unmarshal(body, &req)
|
||||
if len(strings.TrimSpace(string(body))) > 0 {
|
||||
if err := json.Unmarshal(body, &req); err != nil {
|
||||
http.Error(w, "the window request could not be read (not JSON, or cut)", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 2*time.Minute)
|
||||
defer cancel()
|
||||
g, err := h.offsiteKeys.OpenWindowFor(ctx, customerID, req.CountBefore, req.Ended...)
|
||||
@@ -152,6 +160,7 @@ func (h *Handler) handleOffsiteWindowOpen(w http.ResponseWriter, r *http.Request
|
||||
offsiteKeyErr(w, err)
|
||||
return
|
||||
}
|
||||
g.AcceptsEnded = true // every answer of this hub, granted or not
|
||||
writeJSON(w, http.StatusOK, g)
|
||||
}
|
||||
|
||||
|
||||
@@ -55,6 +55,7 @@ type fakeKeySvc struct {
|
||||
gotPub string
|
||||
gotFP string
|
||||
gotEnded []offsitekeys.EndedClaim
|
||||
opened int
|
||||
err error
|
||||
}
|
||||
|
||||
@@ -68,6 +69,7 @@ func (f *fakeKeySvc) ConfirmKey(_ context.Context, _ string, fp string) (int, er
|
||||
}
|
||||
func (f *fakeKeySvc) OpenWindowFor(_ context.Context, _ string, _ int, ended ...offsitekeys.EndedClaim) (offsitekeys.WindowGrant, error) {
|
||||
f.gotEnded = ended
|
||||
f.opened++
|
||||
return offsitekeys.WindowGrant{Granted: false, Reason: "not due"}, f.err
|
||||
}
|
||||
func (f *fakeKeySvc) CloseWindowFor(context.Context, string, offsitekeys.WindowResult) error {
|
||||
|
||||
@@ -276,6 +276,10 @@ type WindowGrant struct {
|
||||
// refused. A box removes an ended app's snapshot ONLY when its id is in EndedAllowed.
|
||||
EndedAllowed []string `json:"ended_allowed,omitempty"`
|
||||
EndedRefused []store.EndedRefusal `json:"ended_refused,omitempty"`
|
||||
// AcceptsEnded tells the box that THIS hub reads `ended` in a window request (review of 2026-11-15, version
|
||||
// skew). It is in every answer, granted or not — the API handler sets it. A box sends its end records only to a
|
||||
// hub that said so in its last answer: an older hub reads 4 KiB of the request and would stall on a larger one.
|
||||
AcceptsEnded bool `json:"accepts_ended"`
|
||||
}
|
||||
|
||||
// WindowResult is the box's report when it is done.
|
||||
|
||||
Reference in new issue
Block a user