hub v0.38.1: offsite provisioning must survive a client disconnect (live F1)

First supervised live run: the ~25s spinner-less offsite save invited a
re-click; the abandoned first request's r.Context() was canceled between
CreateSubaccount and SaveOneTimeSecret, stranding sub-account 268985 with a
password lost forever (consume 404s permanently).

applyOffsite now provisions on context.WithoutCancel + 3-minute absolute
timeout: once the create starts, create->wait->store runs to completion.
Regression test with a ctx-honoring fake that cancels the request context
mid-create; red-proofed against the raw-ctx pre-fix shape (reproduces the
exact live error).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
This commit is contained in:
2026-07-09 21:00:27 +02:00
parent 0a65f2be5f
commit 7c70c545d9
3 changed files with 113 additions and 0 deletions
+17
View File
@@ -1,5 +1,22 @@
# Felhom Hub — Changelog
## v0.38.1 — offsite provisioning: detach from the client's request context (live finding F1) (2026-07-09)
Found in the first supervised live run: the offsite save takes ~25s (create + wait + host-key scan) with no
UI feedback, the operator re-clicked, the browser abandoned the first request, and `r.Context()` was canceled
**between `CreateSubaccount` and `SaveOneTimeSecret`** — the sub-account was created on Hetzner but its
one-time password was lost forever (the controller's consume 404s permanently; stranded resource).
- `internal/web.applyOffsite`: provisioning now runs on `context.WithoutCancel(r.Context())` with a 3-minute
absolute timeout — once the create starts, the create→wait→store atom runs to completion even if the client
disconnects. Fail-closed behavior unchanged (an actual provisioning error still 502s and saves nothing).
- Test `TestApplyOffsite_ClientDisconnectMidProvision` (a ctx-honoring fake cancels the request context
mid-create): the one-time password must reach the store and the descriptor must merge despite the
disconnect. **Companion red-proof:** reverted to the raw request ctx → the exact live error
(`subaccount create action: context canceled`) → test FAILED. Restored.
- Known residuals (recorded, not fixed here): the form has no in-flight spinner/disable (the re-click bait),
and a concurrent save can still hit Hetzner's box-level HTTP 423 action lock (surfaces as the fail-closed 502).
## v0.38.0 — offsite provisioning SLICE 2 (hub side): capture the box host-key fingerprint (2026-07-09)
Pairs with controller v0.106.0. So the controller can VERIFY the box identity instead of blind-TOFU, the hub
+7
View File
@@ -817,6 +817,13 @@ func (s *Server) applyOffsite(ctx context.Context, r *http.Request, cfg *store.C
if s.offsite == nil {
return fmt.Errorf("offsite provisioning is not configured on this hub (no Hetzner token)")
}
// Detach from the client's request context: provisioning takes ~25s (create + wait + host-key scan) and
// an impatient re-click cancels r.Context() MID-SEQUENCE — live finding: the cancel landed between
// CreateSubaccount and SaveOneTimeSecret, stranding a sub-account whose one-time password was lost
// forever. Once provisioning starts it must run to completion (create→wait→store is the atom); the
// absolute timeout still bounds a hung Hetzner call.
ctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 3*time.Minute)
defer cancel()
in := offsite.Input{
Enabled: true,
Type: strings.TrimSpace(r.FormValue("offsite_type")),
+89
View File
@@ -0,0 +1,89 @@
package web
import (
"context"
"io"
"log"
"net/http"
"net/http/httptest"
"strings"
"testing"
"gitea.dooplex.hu/admin/felhom-hub/internal/hetznerapi"
"gitea.dooplex.hu/admin/felhom-hub/internal/offsite"
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
)
// disconnectingAPI models the live F1 failure (VALIDATION-offsite-provisioning-e2e 2026-07-09): the
// browser disconnects (impatient re-click) while the subaccount create action is in flight — the request
// context is canceled MID-SEQUENCE. Every call honors the ctx it receives, like the real HTTP client.
type disconnectingAPI struct {
*hetznerapi.Fake
cancelReq context.CancelFunc
}
func (a *disconnectingAPI) CreateSubaccount(ctx context.Context, boxID int64, req hetznerapi.CreateSubaccountRequest) (int64, hetznerapi.Action, error) {
if err := ctx.Err(); err != nil {
return 0, hetznerapi.Action{}, err
}
id, act, err := a.Fake.CreateSubaccount(ctx, boxID, req)
a.cancelReq() // the client disconnects while the create action runs on Hetzner's side
return id, act, err
}
func (a *disconnectingAPI) WaitAction(ctx context.Context, act hetznerapi.Action) error {
if err := ctx.Err(); err != nil {
return err
}
return a.Fake.WaitAction(ctx, act)
}
func (a *disconnectingAPI) GetSubaccount(ctx context.Context, boxID, subID int64) (hetznerapi.Subaccount, error) {
if err := ctx.Err(); err != nil {
return hetznerapi.Subaccount{}, err
}
return a.Fake.GetSubaccount(ctx, boxID, subID)
}
type webTestScanner struct{}
func (webTestScanner) Fingerprint(_ context.Context, _ string, _ int) (string, error) {
return "SHA256:webtestfp", nil
}
// Live finding F1: a re-click canceled r.Context() between CreateSubaccount and SaveOneTimeSecret —
// the sub-account was created on Hetzner but its one-time password was lost forever (stranded resource,
// controller consume 404s permanently). applyOffsite must detach provisioning from the client's context:
// once the create starts, create→wait→store runs to completion even if the client goes away.
func TestApplyOffsite_ClientDisconnectMidProvision(t *testing.T) {
s, st := newTestServer(t)
reqCtx, cancel := context.WithCancel(context.Background())
api := &disconnectingAPI{Fake: hetznerapi.NewFake(), cancelReq: cancel}
s.SetOffsiteProvisioner(&offsite.Provisioner{
API: api, Store: st, Scanner: webTestScanner{}, PoolBoxID: 611714, Location: "fsn1",
Logger: log.New(io.Discard, "", 0),
})
r := httptest.NewRequest(http.MethodPost,
"/configs/cust-web/edit?offsite_enabled=on&offsite_type=shared&offsite_quota_gb=50", nil)
r = r.WithContext(reqCtx)
cfg := &store.CustomerConfig{CustomerID: "cust-web", ConfigJSON: "{}"}
err := s.applyOffsite(r.Context(), r, cfg)
if reqCtx.Err() == nil {
t.Fatal("test harness broken: the request context was never canceled (no disconnect simulated)")
}
if err != nil {
t.Fatalf("provisioning must survive a client disconnect mid-sequence, got: %v", err)
}
// The one-time password reached the store — the whole point: no stranded resource with a lost password.
pw, err := st.ConsumeOneTimeSecret("cust-web")
if err != nil || pw == "" {
t.Fatalf("one-time password must be stored despite the disconnect (else the sub-account is stranded): pw=%q err=%v", pw, err)
}
// And the descriptor (incl. the pin) was merged for the caller to save.
if !strings.Contains(cfg.ConfigJSON, `"offsite"`) || !strings.Contains(cfg.ConfigJSON, "SHA256:webtestfp") {
t.Fatalf("descriptor not merged into ConfigJSON: %s", cfg.ConfigJSON)
}
}