hub v0.61.0 + felhom-tenantsync v1.1.0: Customer RESET (middle lifecycle tier)

One operator action returns a customer to pre-first-install: all operational
state dies (offsite repo, PBS namespace+backups, DR recipe, one-time secret,
claim state, retained escrow custody); identity + basic config + provenance +
events survive. Sits between host delete and customer Delete.

- store/customer_reset.go: customer_resets journal, live inventory, ack-gated
  purge (never touches identity/provenance/events), DeleteClaim.
- claim.ResetToUnclaimed: delete claim row -> fresh code next onboarding.
- offsite.Deprovision (idempotent) + OffsiteIdentifier + ClearProvisionedDescriptor.
- tenantsync.Deprovision + felhom-tenantsync.sh deprovision op (destroys ns +
  backup groups + token; shared user untouched; idempotent).
- web/customer_reset.go: GET reset -> inventory JSON; POST -> orchestration
  (external teardown FIRST, DB purge LAST; refuse-while-hosts; typed-id +
  separate escrow ack). Amber RESET card distinct from red Danger-zone Delete.
- Red-proofs: ack-gate + partial-failure resumability (both proven red);
  store ack-gating + journal round-trip; offsite idempotency + descriptor clear;
  RESET-card render. Green: build + vet + test.
This commit is contained in:
2026-07-17 13:09:04 +02:00
parent 6b1fbca51d
commit 4009401f46
17 changed files with 1193 additions and 13 deletions
+38
View File
@@ -1,5 +1,43 @@
# Felhom Hub — Changelog
## v0.61.0 — Customer RESET: the middle lifecycle tier (2026-07-17)
One operator action returns a customer to **pre-first-install**: every OPERATIONAL trace dies (offsite
repo, PBS namespace + backups, DR recipe, one-time secret, claim state, retained escrow custody), while
**identity and the basic config survive** (the `customer_configs` row, all provenance rows, and the
audit-event stream). It sits between the two existing tiers — *host delete* (< RESET) and *customer
Delete* (> RESET, the one true purge point). Viktor's rulings: (1) destroying retained escrow custody
gets its **own** separate acknowledgment; (2) RESET clears claim state (a fresh code next onboarding);
(3) RESET **refuses while any host row exists** (delete hosts first — reset never deletes hosts); (4)
the confirm surface shows a **live-counted** inventory.
Orchestration discipline (spec §3): external teardown FIRST, DB purge LAST (publish-last), every leg
idempotent → a partial run is simply re-run from the top; a failed external leg is a clean journal
entry and the DB purge (which erases the descriptors that say what still needs tearing down) is
withheld until every external leg is `ok`. Provenance + events are NEVER wiped.
- **Store** (`internal/store/customer_reset.go`, new): `customer_resets` journal table (per-attempt,
per-leg status, resumable); `CustomerResetInventory` (live counts: hosts, retained blobs via the
F-14 `host_deletions` UNION, dr_recipe/one-time-secret/claim presence); `Start/UpdateResetLeg/Finish/
LatestCustomerReset`; `PurgeCustomerResetDBState` (ack-gated escrow-blob delete + one-time-secret,
dr_recipe, log bundles — never touches identity/provenance/events); `DeleteClaim` primitive.
- **Claim** (`internal/claim/engine.go`): `ResetToUnclaimed` DELETES the claim row so `EnsureIssued`
mints a fresh first code on the next onboarding (no parallel revoked-flag, no stale generation).
- **Offsite** (`internal/offsite/offsite.go`): `Deprovision` DELETES the labelled sub-account/box
(idempotent — label-lookup, `len==0` = already gone); `OffsiteIdentifier` (preview name);
`ClearProvisionedDescriptor` (keeps the tier CHOICE `enabled/type/quota/box_type`, drops every
provisioned field). **PBS** (`internal/tenantsync/client.go` + `scripts/felhom-tenantsync.sh`
`deprovision` op): destroys the customer's namespace + all backup groups + token; the shared
`felhom@pbs` user is never touched; idempotent.
- **Web** (`internal/web/customer_reset.go`, new): `GET /configs/{id}/reset` → live inventory JSON;
`POST` → the orchestration (precondition + typed-id + escrow-ack gates BEFORE any write/external
call). A distinct **amber** RESET card on the customer page (separate from the red Danger-zone
Delete), with the typed-id confirm + the separate escrow-custody ack row.
- **Red-proofs**: ack-gate (defeat → reset proceeds & destroys blobs → FAIL); partial-failure
resumability (purge-not-withheld → DB purged despite external failure → FAIL); both proven red then
restored. Plus store ack-gating, journal round-trip, offsite Deprovision idempotency + descriptor
clear, and the RESET-card render test. Green: `go build ./... && go vet ./... && go test ./...`.
## v0.60.1 — host deletion DEMOTES escrow custody (never destroys) + S6b obsolete (2026-07-17)
Closes the deletion-path gap in v0.60.0's review: `DeleteHost(deleteEscrow=true)` was still DELETING
+13
View File
@@ -186,6 +186,19 @@ func (e *Engine) ReissueForReenroll(cc *store.CustomerConfig) (gen int, reissued
return gen, true, nil
}
// ResetToUnclaimed returns a customer to the pre-first-install (unclaimed, no active code) state — the
// customer-RESET leg (v0.61.0). It DELETES the claim row so the engine's EnsureIssued mints a FRESH
// first code (generation reset, unclaimed) on the next onboarding: no parallel "revoked" flag, no stale
// generation. The active code is invalidated (the row is gone → the gate has no hash) and claimed_at is
// cleared (gone). Idempotent (a missing row is a no-op).
func (e *Engine) ResetToUnclaimed(cc *store.CustomerConfig) error {
if err := e.Store.DeleteClaim(cc.CustomerID); err != nil {
return fmt.Errorf("claim: reset to unclaimed: %w", err)
}
e.logf("[INFO] [claim] reset to unclaimed for %s (customer RESET) — next onboarding mints a fresh code", cc.CustomerID)
return nil
}
// MarkClaimed records a controller-reported successful claim and sends the one-time confirmation
// email on the unclaimed→claimed transition (idempotent — repeated reports are no-ops).
func (e *Engine) MarkClaimed(cc *store.CustomerConfig) error {
+99
View File
@@ -227,6 +227,80 @@ func (p *Provisioner) ReissueCredentials(ctx context.Context, customerID, typ st
return nil
}
// OffsiteIdentifier returns the customer's provisioned offsite name (sub-account username / box name)
// for the RESET preview inventory, "" if none is provisioned. Read-only.
func (p *Provisioner) OffsiteIdentifier(ctx context.Context, customerID, typ string) (string, error) {
switch typ {
case "dedicated":
boxes, err := p.API.ListStorageBoxes(ctx, customerSelector(customerID))
if err != nil || len(boxes) == 0 {
return "", err
}
return boxes[0].Name, nil
default: // shared / ""
if p.PoolBoxID == 0 {
return "", nil
}
subs, err := p.API.ListSubaccounts(ctx, p.PoolBoxID, customerSelector(customerID))
if err != nil || len(subs) == 0 {
return "", err
}
return subs[0].Username, nil
}
}
// Deprovision DELETES the customer's offsite storage — the RESET teardown (v0.61.0). The offsite repo
// DATA dies with the sub-account/box; irreversible, gated by the operator RESET confirm. IDEMPOTENT:
// zero labelled sub-accounts/boxes = already gone = success (a re-run after a partial reset does not
// error). The id is re-derived by the customer label each time — nothing stored to go stale.
func (p *Provisioner) Deprovision(ctx context.Context, customerID, typ string) error {
switch typ {
case "dedicated":
boxes, err := p.API.ListStorageBoxes(ctx, customerSelector(customerID))
if err != nil {
return fmt.Errorf("offsite: deprovision lookup: %w", err)
}
if len(boxes) == 0 {
p.logf("[offsite] deprovision: no box labelled for %s — already gone", customerID)
return nil
}
for _, b := range boxes {
act, err := p.API.DeleteStorageBox(ctx, b.ID)
if err != nil {
return fmt.Errorf("offsite: delete box %d: %w", b.ID, err)
}
if err := p.API.WaitAction(ctx, act); err != nil {
return fmt.Errorf("offsite: delete box action: %w", err)
}
p.logf("[offsite] deprovisioned dedicated box %d for %s (repo data destroyed)", b.ID, customerID)
}
return nil
default: // shared / ""
if p.PoolBoxID == 0 {
return fmt.Errorf("offsite: no shared pool box configured")
}
subs, err := p.API.ListSubaccounts(ctx, p.PoolBoxID, customerSelector(customerID))
if err != nil {
return fmt.Errorf("offsite: deprovision lookup: %w", err)
}
if len(subs) == 0 {
p.logf("[offsite] deprovision: no sub-account labelled for %s — already gone", customerID)
return nil
}
for _, sub := range subs {
act, err := p.API.DeleteSubaccount(ctx, p.PoolBoxID, sub.ID)
if err != nil {
return fmt.Errorf("offsite: delete sub-account %d: %w", sub.ID, err)
}
if err := p.API.WaitAction(ctx, act); err != nil {
return fmt.Errorf("offsite: delete sub-account action: %w", err)
}
p.logf("[offsite] deprovisioned shared sub-account %d for %s (repo data destroyed)", sub.ID, customerID)
}
return nil
}
}
func (p *Provisioner) provisionShared(ctx context.Context, customerID string, in Input) (*Descriptor, error) {
if p.PoolBoxID == 0 {
return nil, fmt.Errorf("offsite: no shared pool box configured")
@@ -344,6 +418,31 @@ func (p *Provisioner) SetOffsiteFrozen(ctx context.Context, customerID string, f
// MergeDescriptor merges the offsite descriptor under the "offsite" key of a ConfigJSON object, preserving
// all other keys. Returns the new ConfigJSON string. NEVER carries a secret (Descriptor is non-secret).
// ClearProvisionedDescriptor returns config_json with the offsite descriptor reset to its
// pre-first-install shape — the customer-RESET clear (v0.61.0). The TIER CHOICE
// (enabled/type/quota_gb/box_type — customer config, SURVIVES a RESET) is kept; every PROVISIONED
// field (host/user/port/repo_path/host_fingerprint — operational state, DIES with the Hetzner
// resource) is cleared, so re-onboarding re-provisions fresh against the retained choice. No-op-safe:
// an absent/empty offsite block returns the input unchanged.
func ClearProvisionedDescriptor(configJSON string) (string, error) {
obj := map[string]json.RawMessage{}
if strings.TrimSpace(configJSON) != "" && configJSON != "{}" {
if err := json.Unmarshal([]byte(configJSON), &obj); err != nil {
return "", fmt.Errorf("offsite: parse config_json: %w", err)
}
}
raw, ok := obj["offsite"]
if !ok {
return configJSON, nil // no offsite block — nothing to clear
}
var cur Descriptor
if err := json.Unmarshal(raw, &cur); err != nil {
return "", fmt.Errorf("offsite: parse offsite descriptor: %w", err)
}
cleared := &Descriptor{Enabled: cur.Enabled, Type: cur.Type, QuotaGB: cur.QuotaGB, BoxType: cur.BoxType}
return MergeDescriptor(configJSON, cleared)
}
func MergeDescriptor(configJSON string, d *Descriptor) (string, error) {
obj := map[string]json.RawMessage{}
if strings.TrimSpace(configJSON) != "" && configJSON != "{}" {
+51
View File
@@ -389,3 +389,54 @@ func TestProvision_DisableNoDeprovision(t *testing.T) {
t.Fatal("disable must NOT deprovision (data-loss guard)")
}
}
// Customer RESET (v0.61.0) — Deprovision DESTROYS the labelled shared sub-account, and is idempotent
// (a second call finds nothing and succeeds). This is the deliberate teardown the disable-guard above
// deliberately does NOT do.
func TestDeprovision_SharedIdempotent(t *testing.T) {
p, fake, _ := newTestProvisioner(t)
if _, err := p.ProvisionOffsite(context.Background(), "cust-d", Input{Enabled: true, Type: "shared", QuotaGB: 10}); err != nil {
t.Fatalf("provision: %v", err)
}
if fake.CreatedSubaccounts != 1 {
t.Fatalf("precondition: want 1 subaccount, got %d", fake.CreatedSubaccounts)
}
if err := p.Deprovision(context.Background(), "cust-d", "shared"); err != nil {
t.Fatalf("deprovision: %v", err)
}
if fake.DeletedSubaccounts != 1 {
t.Fatalf("want 1 subaccount deleted, got %d", fake.DeletedSubaccounts)
}
// Idempotent: nothing labelled now → success, no extra delete.
if err := p.Deprovision(context.Background(), "cust-d", "shared"); err != nil {
t.Fatalf("second deprovision (idempotent) errored: %v", err)
}
if fake.DeletedSubaccounts != 1 {
t.Fatalf("idempotent re-run deleted again: %d", fake.DeletedSubaccounts)
}
}
// ClearProvisionedDescriptor keeps the tier CHOICE (enabled/type/quota/box_type) and drops every
// PROVISIONED field — the pre-first-install shape a RESET returns the customer to.
func TestClearProvisionedDescriptor(t *testing.T) {
in := `{"git":{"token":"x"},"offsite":{"enabled":true,"type":"shared","host":"u1-sub3.your-storagebox.de","user":"u1-sub3","port":23,"repo_path":"/home/felhom","quota_gb":100,"box_type":"","host_fingerprint":"SHA256:abc"}}`
out, err := ClearProvisionedDescriptor(in)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(out, `"enabled":true`) || !strings.Contains(out, `"type":"shared"`) || !strings.Contains(out, `"quota_gb":100`) {
t.Errorf("tier choice lost: %s", out)
}
for _, gone := range []string{"your-storagebox.de", "u1-sub3", "repo_path", "host_fingerprint", `"port"`} {
if strings.Contains(out, gone) {
t.Errorf("provisioned field %q survived: %s", gone, out)
}
}
if !strings.Contains(out, `"git"`) {
t.Errorf("unrelated config keys dropped: %s", out)
}
// No-op-safe: absent offsite block returns input unchanged.
if got, _ := ClearProvisionedDescriptor(`{"git":{"token":"x"}}`); got != `{"git":{"token":"x"}}` {
t.Errorf("no-offsite clear mutated config: %s", got)
}
}
+181
View File
@@ -0,0 +1,181 @@
package store
import (
"database/sql"
"encoding/json"
"fmt"
"time"
)
// Customer RESET (v0.61.0) — the middle lifecycle tier (host delete < RESET < customer delete). The
// journal is F-14-style provenance: one row per attempt, per-leg status, resumable. External teardown
// runs FIRST and the DB-purge runs LAST (publish-last) so a re-run reads the journal to know what still
// needs tearing down. Provenance/events are NEVER wiped — audit outlives every tier.
// CustomerReset is one reset provenance/journal record.
type CustomerReset struct {
ID int64
CustomerID string
StartedAt time.Time
CompletedAt *time.Time
EscrowAcked bool
Legs map[string]string // hetzner | pbs | db_purge → pending|ok|failed|manual
}
// ResetInventory is the live count of what a RESET would destroy (the confirm surface, ruling 4).
// Sub-account name + PBS namespace are added by the web layer from the external clients.
type ResetInventory struct {
HostCount int // ruling 3: RESET refuses while any host row exists
SupersededBlobs int // M — retained recovery-key custody destroyed (ack-gated)
DRRecipePresent bool
OneTimeSecretPresent bool
ClaimPresent bool
}
// CustomerResetInventory builds the hub-DB side of the confirm inventory.
func (s *Store) CustomerResetInventory(customerID string) (*ResetInventory, error) {
inv := &ResetInventory{}
hosts, err := s.ListHostsByCustomer(customerID)
if err != nil {
return nil, err
}
inv.HostCount = len(hosts)
queries := []struct {
dst *int
q string
arg string
}{
{&inv.SupersededBlobs, `SELECT COUNT(*) FROM host_escrow_superseded WHERE host_id IN (SELECT host_id FROM host_deletions WHERE customer_id = ? UNION SELECT host_id FROM hosts WHERE customer_id = ?)`, customerID},
}
for _, qq := range queries {
if err := s.db.QueryRow(qq.q, qq.arg, customerID).Scan(qq.dst); err != nil {
return nil, err
}
}
present := func(q string) (bool, error) {
var n int
err := s.db.QueryRow(q, customerID).Scan(&n)
return n > 0, err
}
if inv.DRRecipePresent, err = present(`SELECT COUNT(*) FROM dr_recipe WHERE customer_id = ?`); err != nil {
return nil, err
}
if inv.OneTimeSecretPresent, err = present(`SELECT COUNT(*) FROM one_time_secrets WHERE customer_id = ?`); err != nil {
return nil, err
}
if inv.ClaimPresent, err = present(`SELECT COUNT(*) FROM customer_claims WHERE customer_id = ?`); err != nil {
return nil, err
}
return inv, nil
}
// StartCustomerReset opens a reset journal row and returns its id.
func (s *Store) StartCustomerReset(customerID string, escrowAcked bool) (int64, error) {
acked := 0
if escrowAcked {
acked = 1
}
res, err := s.db.Exec(`INSERT INTO customer_resets (customer_id, escrow_acked, legs_json) VALUES (?, ?, '{}')`, customerID, acked)
if err != nil {
return 0, err
}
return res.LastInsertId()
}
// UpdateResetLeg merges one leg's status into the journal (read-modify-write; the reset is single-flight
// per customer at the handler, so no lock contention).
func (s *Store) UpdateResetLeg(resetID int64, leg, status string) error {
var raw string
if err := s.db.QueryRow(`SELECT legs_json FROM customer_resets WHERE id = ?`, resetID).Scan(&raw); err != nil {
return err
}
legs := map[string]string{}
if raw != "" {
_ = json.Unmarshal([]byte(raw), &legs)
}
legs[leg] = status
out, err := json.Marshal(legs)
if err != nil {
return err
}
_, err = s.db.Exec(`UPDATE customer_resets SET legs_json = ? WHERE id = ?`, string(out), resetID)
return err
}
// FinishCustomerReset stamps completion.
func (s *Store) FinishCustomerReset(resetID int64) error {
_, err := s.db.Exec(`UPDATE customer_resets SET completed_at = datetime('now') WHERE id = ?`, resetID)
return err
}
// LatestCustomerReset returns the customer's most recent reset record (nil = never).
func (s *Store) LatestCustomerReset(customerID string) (*CustomerReset, error) {
var (
cr CustomerReset
startedAt string
completedAt sql.NullString
acked int
legs string
)
err := s.db.QueryRow(`
SELECT id, customer_id, started_at, completed_at, escrow_acked, legs_json
FROM customer_resets WHERE customer_id = ? ORDER BY id DESC LIMIT 1`, customerID).
Scan(&cr.ID, &cr.CustomerID, &startedAt, &completedAt, &acked, &legs)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
cr.StartedAt = parseSQLiteTime(startedAt)
if completedAt.Valid && completedAt.String != "" {
t := parseSQLiteTime(completedAt.String)
cr.CompletedAt = &t
}
cr.EscrowAcked = acked != 0
cr.Legs = map[string]string{}
_ = json.Unmarshal([]byte(legs), &cr.Legs)
return &cr, nil
}
// PurgeCustomerResetDBState is the DB-purge phase — runs LAST, after the external teardown legs. In one
// tx it removes the customer-scoped operational state a RESET destroys: retained escrow custody
// (ack-gated), the one-time repo password, the DR recipe, and customer-scoped log bundles. It does NOT
// touch customer_configs identity/config (SURVIVES), provenance (host_deletions/customer_resets), or
// events (audit). The claim reset rides the claim engine separately; the offsite descriptor clear rides
// the offsite package (config_json). Idempotent — a re-run deletes nothing extra.
func (s *Store) PurgeCustomerResetDBState(customerID string, escrowAcked bool) error {
tx, err := s.db.Begin()
if err != nil {
return err
}
defer tx.Rollback()
if escrowAcked {
// Retained (superseded) blobs — the hosts are gone by the RESET precondition, so they are found
// only via the F-14 host_deletions provenance (UNION current hosts as a belt-and-suspenders).
if _, err := tx.Exec(`DELETE FROM host_escrow_superseded WHERE host_id IN (
SELECT host_id FROM host_deletions WHERE customer_id = ?
UNION SELECT host_id FROM hosts WHERE customer_id = ?)`, customerID, customerID); err != nil {
return fmt.Errorf("purge superseded escrow: %w", err)
}
}
for _, q := range []string{
`DELETE FROM one_time_secrets WHERE customer_id = ?`,
`DELETE FROM dr_recipe WHERE customer_id = ?`,
`DELETE FROM log_bundles WHERE scope_id = ?`,
`DELETE FROM log_bundle_requests WHERE scope_id = ?`,
} {
if _, err := tx.Exec(q, customerID); err != nil {
return fmt.Errorf("reset purge %q: %w", q, err)
}
}
return tx.Commit()
}
// DeleteClaim removes a customer's claim row (RESET → unclaimed / pre-first-install). The claim ENGINE
// re-mints a fresh code via EnsureIssued on the next onboarding — this is the store primitive the
// engine's ResetToUnclaimed rides (no parallel claimed-flag mechanism).
func (s *Store) DeleteClaim(customerID string) error {
_, err := s.db.Exec(`DELETE FROM customer_claims WHERE customer_id = ?`, customerID)
return err
}
+115
View File
@@ -0,0 +1,115 @@
package store
import (
"io"
"log"
"path/filepath"
"testing"
)
func newResetStore(t *testing.T) *Store {
t.Helper()
st, err := New(filepath.Join(t.TempDir(), "t.db"), log.New(io.Discard, "", 0))
if err != nil {
t.Fatalf("store.New: %v", err)
}
t.Cleanup(func() { st.Close() })
return st
}
// seedRetainedBlob leaves a superseded escrow blob reachable only via F-14 host_deletions provenance
// (host deleted, blob retained) — the customer-RESET escrow-custody target.
func seedRetainedBlob(t *testing.T, st *Store, customerID string) {
t.Helper()
hostID := customerID + "-h1"
if err := st.UpsertHost(&Host{HostID: hostID, CustomerID: customerID, APIKey: "h"}); err != nil {
t.Fatalf("upsert host: %v", err)
}
if _, err := st.SaveHostEscrow(hostID, []byte("A"), "fpA", "p", "2026-01-01T00:00:00Z", "shaA"); err != nil {
t.Fatalf("escrow A: %v", err)
}
if _, err := st.SaveHostEscrow(hostID, []byte("B"), "fpB", "p", "2026-01-02T00:00:00Z", "shaB"); err != nil {
t.Fatalf("escrow B: %v", err)
}
if err := st.DeleteHost(hostID, true); err != nil {
t.Fatalf("delete host: %v", err)
}
}
// The store-level ack contract: PurgeCustomerResetDBState must NEVER destroy retained escrow custody
// unless escrowAck is true — even while it clears the rest of the operational state. (The web handler
// gates ack separately; this proves the store primitive is itself honest.)
func TestPurgeCustomerResetDBState_AckGatesEscrow(t *testing.T) {
st := newResetStore(t)
if err := st.SaveCustomerConfig(&CustomerConfig{CustomerID: "acme", APIKey: "capi", RetrievalPassword: "pw"}); err != nil {
t.Fatalf("seed config: %v", err)
}
seedRetainedBlob(t, st, "acme")
if err := st.SaveOneTimeSecret("acme", "ots"); err != nil {
t.Fatalf("seed ots: %v", err)
}
if err := st.SaveDRRecipeHostHalf("acme", "acme-h1", 1, []byte("half")); err != nil {
t.Fatalf("seed dr: %v", err)
}
// ack=false: everything else goes, blobs SURVIVE.
if err := st.PurgeCustomerResetDBState("acme", false); err != nil {
t.Fatalf("purge (no ack): %v", err)
}
inv, err := st.CustomerResetInventory("acme")
if err != nil {
t.Fatal(err)
}
if inv.SupersededBlobs == 0 {
t.Error("ack=false destroyed retained escrow custody")
}
if inv.OneTimeSecretPresent || inv.DRRecipePresent {
t.Errorf("ack=false failed to clear the non-escrow state: %+v", inv)
}
// ack=true: blobs go too.
if err := st.PurgeCustomerResetDBState("acme", true); err != nil {
t.Fatalf("purge (ack): %v", err)
}
if inv, _ := st.CustomerResetInventory("acme"); inv.SupersededBlobs != 0 {
t.Errorf("ack=true failed to destroy retained custody: %d", inv.SupersededBlobs)
}
// Identity SURVIVES the purge.
if cfg, _ := st.GetCustomerConfig("acme"); cfg == nil {
t.Error("purge destroyed customer_config — identity must survive")
}
}
// The journal round-trips: open → per-leg status → finish, and Latest reflects it.
func TestCustomerResetJournal_RoundTrip(t *testing.T) {
st := newResetStore(t)
id, err := st.StartCustomerReset("acme", true)
if err != nil {
t.Fatalf("start: %v", err)
}
if err := st.UpdateResetLeg(id, "hetzner", "ok"); err != nil {
t.Fatalf("leg: %v", err)
}
if err := st.UpdateResetLeg(id, "pbs", "failed"); err != nil {
t.Fatalf("leg: %v", err)
}
if err := st.UpdateResetLeg(id, "pbs", "ok"); err != nil { // overwrite on re-run
t.Fatalf("leg: %v", err)
}
cr, err := st.LatestCustomerReset("acme")
if err != nil || cr == nil {
t.Fatalf("latest: %v / %v", cr, err)
}
if cr.CompletedAt != nil {
t.Error("not finished yet — CompletedAt should be nil")
}
if !cr.EscrowAcked || cr.Legs["hetzner"] != "ok" || cr.Legs["pbs"] != "ok" {
t.Errorf("journal wrong: %+v", cr)
}
if err := st.FinishCustomerReset(id); err != nil {
t.Fatalf("finish: %v", err)
}
if cr, _ := st.LatestCustomerReset("acme"); cr.CompletedAt == nil {
t.Error("finish did not stamp CompletedAt")
}
}
+15
View File
@@ -596,6 +596,21 @@ func (s *Store) migrate() error {
escrow_acked INTEGER NOT NULL DEFAULT 0
);
CREATE INDEX IF NOT EXISTS idx_host_deletions_customer ON host_deletions(customer_id, id DESC);
-- customer_resets (v0.61.0, F-14 style): the RESET provenance + resumable per-leg journal. One
-- row per reset attempt; legs_json holds {hetzner,pbs,db_purge pending|ok|failed|manual}. The
-- DB purge runs LAST (publish-last), so a re-run reads the journal to know what still needs
-- teardown. escrow_acked records the ruling-1 acknowledgment. NEVER pruned (audit outlives
-- every lifecycle tier).
CREATE TABLE IF NOT EXISTS customer_resets (
id INTEGER PRIMARY KEY AUTOINCREMENT,
customer_id TEXT NOT NULL,
started_at DATETIME NOT NULL DEFAULT (datetime('now')),
completed_at DATETIME,
escrow_acked INTEGER NOT NULL DEFAULT 0,
legs_json TEXT NOT NULL DEFAULT '{}'
);
CREATE INDEX IF NOT EXISTS idx_customer_resets_customer ON customer_resets(customer_id, id DESC);
`)
if err != nil {
return err
+29 -3
View File
@@ -99,6 +99,31 @@ func (c *Client) Reissue(ctx context.Context, customerID string) (*Result, error
return c.tenancyOp(ctx, "reissue", customerID)
}
// Deprovision DESTROYS the customer's PBS namespace, all its backup groups, and its token — the
// customer-RESET teardown (v0.61.0, operator ack-gated). The shared felhom@pbs user is never touched
// (co-tenants ride it). Idempotent: a missing tenant is a clean success (existed=false). This carries
// NO secret, so it does not route through tenancyOp's token-field validation.
func (c *Client) Deprovision(ctx context.Context, customerID string) (existed bool, err error) {
if !customerIDRe.MatchString(customerID) {
return false, fmt.Errorf("tenantsync: invalid customer_id %q", customerID)
}
payload, err := json.Marshal(map[string]string{"op": "deprovision", "customer_id": customerID})
if err != nil {
return false, err
}
stdout, stderr, runErr := c.exec(ctx, payload)
resp, err := parseResponse(stdout, stderr, runErr)
if err != nil {
return false, err
}
if resp.Namespace == "" {
return false, fmt.Errorf("tenantsync: deprovision response missing namespace")
}
c.logger.Printf("[INFO] tenantsync: deprovision ok for %s (ns=%s, existed=%t)",
customerID, resp.Namespace, resp.Deleted)
return resp.Deleted, nil
}
// Fingerprint returns the endpoint PBS's API cert fingerprint (the descriptor field).
func (c *Client) Fingerprint(ctx context.Context) (string, error) {
stdout, stderr, runErr := c.exec(ctx, []byte(`{"op":"fingerprint"}`))
@@ -136,9 +161,10 @@ func (c *Client) tenancyOp(ctx context.Context, op, customerID string) (*Result,
// response is the script's stdout contract — ok carries the Result fields, error carries code+error.
type response struct {
Status string `json:"status"`
Code string `json:"code"`
Error string `json:"error"`
Status string `json:"status"`
Code string `json:"code"`
Error string `json:"error"`
Deleted bool `json:"deleted"` // deprovision op: whether the namespace existed (was destroyed)
Result
}
+228
View File
@@ -0,0 +1,228 @@
package web
import (
"context"
"encoding/json"
"net/http"
"time"
"gitea.dooplex.hu/admin/felhom-hub/internal/offsite"
)
// Customer RESET (v0.61.0) — the middle lifecycle tier (host delete < RESET < customer DELETE). One
// operator action returns a customer to pre-first-install: every OPERATIONAL trace dies (offsite repo
// + PBS namespace + credentials + DR recipe + claim state + retained escrow custody), while IDENTITY
// and the BASIC CONFIG survive (customer_configs row, provenance rows, the audit event stream).
//
// Orchestration discipline (spec §3): external teardown runs FIRST, the DB purge runs LAST
// (publish-last). Every leg is idempotent, so a partial run is simply re-run from the top — a failed
// external leg is a clean journal entry, and the DB purge (which erases the descriptors that say what
// still needs tearing down) is withheld until every external leg is ok. Provenance/events are NEVER
// wiped — the audit trail outlives every lifecycle tier.
// offsiteChoice reads the customer's offsite tier selection out of config_json (the same shape the
// re-issue/freeze handlers read). enabled=false means no offsite leg to run.
func offsiteChoice(configJSON string) (enabled bool, typ string) {
var o struct {
Offsite struct {
Enabled bool `json:"enabled"`
Type string `json:"type"`
} `json:"offsite"`
}
_ = json.Unmarshal([]byte(configJSON), &o)
return o.Offsite.Enabled, o.Offsite.Type
}
// handleCustomerResetPreview — GET /configs/{id}/reset. Returns the live inventory the confirm surface
// renders (ruling 4): what a RESET would destroy right now. Read-only — no writes, no external calls
// beyond the label lookups needed to name the offsite resource. `refused` is true when a host row still
// exists (ruling 3: RESET refuses until the operator deletes the hosts first).
func (s *Server) handleCustomerResetPreview(w http.ResponseWriter, r *http.Request, customerID string) {
cfg, err := s.store.GetCustomerConfig(customerID)
if err != nil {
s.logger.Printf("[ERROR] reset preview %s: %v", customerID, err)
http.Error(w, "Internal error", http.StatusInternalServerError)
return
}
if cfg == nil {
http.NotFound(w, r)
return
}
inv, err := s.store.CustomerResetInventory(customerID)
if err != nil {
s.logger.Printf("[ERROR] reset preview %s: inventory: %v", customerID, err)
http.Error(w, "Internal error", http.StatusInternalServerError)
return
}
offsiteEnabled, offsiteType := offsiteChoice(cfg.ConfigJSON)
offsiteName := ""
if offsiteEnabled && s.offsite != nil {
ctx, cancel := context.WithTimeout(r.Context(), 20*time.Second)
defer cancel()
if n, oerr := s.offsite.OffsiteIdentifier(ctx, customerID, offsiteType); oerr != nil {
s.logger.Printf("[WARN] reset preview %s: offsite identifier lookup: %v", customerID, oerr)
} else {
offsiteName = n
}
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]any{
"customer_id": customerID,
"host_count": inv.HostCount,
"refused": inv.HostCount > 0, // ruling 3
"superseded_blobs": inv.SupersededBlobs,
"escrow_ack_required": inv.SupersededBlobs > 0, // ruling 1: separate custody-destruction ack
"dr_recipe_present": inv.DRRecipePresent,
"one_time_secret": inv.OneTimeSecretPresent,
"claim_present": inv.ClaimPresent,
"offsite_enabled": offsiteEnabled,
"offsite_type": offsiteType,
"offsite_identifier": offsiteName,
"pbs_tenancy_configured": s.tenantsync != nil,
})
}
// handleCustomerReset — POST /configs/{id}/reset. Executes the reset. Preconditions (Scenario A + the
// confirm gates) are checked BEFORE any write or external call: a refused reset leaves zero side effects.
func (s *Server) handleCustomerReset(w http.ResponseWriter, r *http.Request, customerID string) {
cfg, err := s.store.GetCustomerConfig(customerID)
if err != nil {
s.logger.Printf("[ERROR] reset %s: %v", customerID, err)
http.Error(w, "Internal error", http.StatusInternalServerError)
return
}
if cfg == nil {
http.NotFound(w, r)
return
}
inv, err := s.store.CustomerResetInventory(customerID)
if err != nil {
s.logger.Printf("[ERROR] reset %s: inventory: %v", customerID, err)
http.Error(w, "Internal error", http.StatusInternalServerError)
return
}
// Ruling 3 (Scenario A): RESET REFUSES while any host row exists. RESET never deletes hosts — the
// operator deletes them first. 409 with ZERO writes and ZERO external calls.
if inv.HostCount > 0 {
s.logger.Printf("[WARN] reset %s REFUSED: %d host row(s) still present — delete the hosts first", customerID, inv.HostCount)
http.Error(w, "Reset refused: this customer still has host(s). Delete every host first — reset never deletes hosts.", http.StatusConflict)
return
}
// Typed-confirmation gate: the operator must type the exact customer-id.
if r.FormValue("confirm_id") != customerID {
http.Error(w, "Reset refused: the typed customer-id does not match.", http.StatusBadRequest)
return
}
// Ruling 1: destroying retained escrow custody (M>0) needs its OWN separate acknowledgment.
escrowAck := r.FormValue("escrow_ack") == "1"
if inv.SupersededBlobs > 0 && !escrowAck {
http.Error(w, "Reset refused: destroying the retained recovery-key custody requires the separate acknowledgment.", http.StatusBadRequest)
return
}
// From here the reset is committed. Detached ctx (spec: once teardown starts it must run to a clean
// journal state regardless of the operator's browser). External legs FIRST.
ctx, cancel := context.WithTimeout(context.WithoutCancel(r.Context()), 5*time.Minute)
defer cancel()
resetID, err := s.store.StartCustomerReset(customerID, escrowAck)
if err != nil {
s.logger.Printf("[ERROR] reset %s: open journal: %v", customerID, err)
http.Error(w, "Internal error", http.StatusInternalServerError)
return
}
s.logger.Printf("[INFO] customer RESET started for %s (journal #%d, escrow_ack=%t)", customerID, resetID, escrowAck)
// Leg: Hetzner offsite (repo DATA destroyed). Only when the customer chose an offsite tier.
offsiteEnabled, offsiteType := offsiteChoice(cfg.ConfigJSON)
if offsiteEnabled && s.offsite != nil {
if derr := s.offsite.Deprovision(ctx, customerID, offsiteType); derr != nil {
_ = s.store.UpdateResetLeg(resetID, "hetzner", "failed")
s.logger.Printf("[ERROR] reset %s: hetzner deprovision FAILED (journal #%d retained; re-run to resume): %v", customerID, resetID, derr)
http.Error(w, "Reset incomplete: the offsite (Hetzner) teardown failed — nothing was purged; re-run to resume. ("+derr.Error()+")", http.StatusBadGateway)
return
}
_ = s.store.UpdateResetLeg(resetID, "hetzner", "ok")
s.logger.Printf("[INFO] reset %s: offsite deprovisioned (repo data destroyed)", customerID)
} else {
_ = s.store.UpdateResetLeg(resetID, "hetzner", "skipped")
}
// Leg: PBS DR tenancy (namespace + backups + token destroyed). The namespace is customer-id-keyed
// and survives host deletion, so it is torn down here by id; idempotent when absent.
if s.tenantsync != nil {
if _, derr := s.tenantsync.Deprovision(ctx, customerID); derr != nil {
_ = s.store.UpdateResetLeg(resetID, "pbs", "failed")
s.logger.Printf("[ERROR] reset %s: PBS deprovision FAILED (journal #%d retained; re-run to resume): %v", customerID, resetID, derr)
http.Error(w, "Reset incomplete: the PBS namespace teardown failed — nothing was purged; re-run to resume. ("+derr.Error()+")", http.StatusBadGateway)
return
}
_ = s.store.UpdateResetLeg(resetID, "pbs", "ok")
s.logger.Printf("[INFO] reset %s: PBS tenancy deprovisioned", customerID)
} else {
_ = s.store.UpdateResetLeg(resetID, "pbs", "skipped")
}
// All external legs are ok — now the DB side (publish-last, one leg at a time so the journal
// records where a mid-purge crash stopped). Claim → unclaimed (fresh code next onboarding).
if s.claimEngine != nil {
if cerr := s.claimEngine.ResetToUnclaimed(cfg); cerr != nil {
_ = s.store.UpdateResetLeg(resetID, "claim", "failed")
s.logger.Printf("[ERROR] reset %s: claim reset failed: %v", customerID, cerr)
http.Error(w, "Reset incomplete: the claim reset failed — re-run to resume. ("+cerr.Error()+")", http.StatusInternalServerError)
return
}
} else if derr := s.store.DeleteClaim(customerID); derr != nil { // no engine wired: use the store primitive directly
_ = s.store.UpdateResetLeg(resetID, "claim", "failed")
s.logger.Printf("[ERROR] reset %s: claim delete failed: %v", customerID, derr)
http.Error(w, "Internal error", http.StatusInternalServerError)
return
}
_ = s.store.UpdateResetLeg(resetID, "claim", "ok")
// Clear the provisioned offsite descriptor (keep the tier CHOICE, drop provisioned host/user/repo/
// fingerprint) and re-save → ConfigVersion bump. Identity + basic config survive intact.
newConfigJSON, cerr := offsite.ClearProvisionedDescriptor(cfg.ConfigJSON)
if cerr != nil {
_ = s.store.UpdateResetLeg(resetID, "descriptor", "failed")
s.logger.Printf("[ERROR] reset %s: clear offsite descriptor: %v", customerID, cerr)
http.Error(w, "Internal error", http.StatusInternalServerError)
return
}
cfg.ConfigJSON = newConfigJSON
if serr := s.store.SaveCustomerConfig(cfg); serr != nil {
_ = s.store.UpdateResetLeg(resetID, "descriptor", "failed")
s.logger.Printf("[ERROR] reset %s: save cleared config: %v", customerID, serr)
http.Error(w, "Internal error", http.StatusInternalServerError)
return
}
_ = s.store.UpdateResetLeg(resetID, "descriptor", "ok")
// DB purge LAST: retained escrow (ack-gated), one-time secret, DR recipe, log bundles.
if perr := s.store.PurgeCustomerResetDBState(customerID, escrowAck); perr != nil {
_ = s.store.UpdateResetLeg(resetID, "db_purge", "failed")
s.logger.Printf("[ERROR] reset %s: DB purge failed: %v", customerID, perr)
http.Error(w, "Reset incomplete: the DB purge failed — re-run to resume. ("+perr.Error()+")", http.StatusInternalServerError)
return
}
_ = s.store.UpdateResetLeg(resetID, "db_purge", "ok")
if ferr := s.store.FinishCustomerReset(resetID); ferr != nil {
s.logger.Printf("[WARN] reset %s: journal finish stamp failed (state is complete): %v", customerID, ferr)
}
// Audit event (SURVIVES — the reset is part of the customer's permanent history).
msg := "Ügyfél-visszaállítás (RESET): minden működési állapot törölve (offsite tároló, PBS névtér, DR-recept, azonosítási állapot). Az azonosság és az alapkonfiguráció megmaradt."
if escrowAck {
msg += " A megőrzött helyreállítási-kulcs letét is megsemmisült (megerősítve)."
}
if _, eerr := s.store.SaveEvent(customerID, "customer_reset", "warning", msg, "", "hub"); eerr != nil {
s.logger.Printf("[WARN] reset %s: save audit event: %v", customerID, eerr)
}
s.logger.Printf("[INFO] customer RESET complete for %s (journal #%d) — identity + basic config retained", customerID, resetID)
s.bumpIntent(customerID) // wake any holding wait so a lingering box sees the cleared state promptly
http.Redirect(w, r, "/customers/"+customerID+"?flash=reset_done", http.StatusSeeOther)
}
@@ -0,0 +1,46 @@
package web
import (
"strings"
"testing"
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
)
// The RESET control renders on the customer page as its OWN amber card, visually distinct from the
// red Danger-zone Delete, with the typed-id confirm + the separate escrow-custody ack row (ruling 1).
func TestTemplates_CustomerResetCard(t *testing.T) {
s, st := newTestServer(t)
if err := st.SaveCustomerConfig(&store.CustomerConfig{
CustomerID: "acme", CustomerName: "Acme", Domain: "acme.hu",
RetrievalPassword: "pw", APIKey: "k", Status: "active",
}); err != nil {
t.Fatal(err)
}
html := renderCustomerPage(t, s, "acme")
// The RESET form posts to the reset endpoint, and the preview endpoint is fetched by its JS.
if !strings.Contains(html, `action="/configs/acme/reset"`) {
t.Error("reset form action missing")
}
for _, fn := range []string{"customerResetConfirm", "customerResetSubmit", "/configs/' + encodeURIComponent(cid) + '/reset"} {
if !strings.Contains(html, fn) {
t.Errorf("reset JS missing %q", fn)
}
}
// Distinct visual: the reset card is amber (--warn); the delete form (red --crit) still exists
// separately — the two controls are not merged.
if !strings.Contains(html, "border-color: var(--warn);") {
t.Error("reset card is not amber-toned (must be distinct from the red Delete)")
}
if !strings.Contains(html, `action="/configs/acme/delete"`) {
t.Error("the Danger-zone Delete must remain a separate control")
}
// The separate escrow-custody ack (ruling 1) + typed-id confirm.
if !strings.Contains(html, `id="cust-reset-escrow-acme"`) {
t.Error("escrow-custody ack checkbox missing")
}
if !strings.Contains(html, `name="confirm_id"`) || !strings.Contains(html, `name="escrow_ack"`) {
t.Error("reset confirm inputs (confirm_id / escrow_ack) missing")
}
}
+222
View File
@@ -0,0 +1,222 @@
package web
// Customer RESET (v0.61.0) orchestration red-proofs. The load-bearing contracts:
// - Scenario A: RESET refuses while ANY host row exists — 409, ZERO side effects (no journal row).
// - Ruling 1: destroying retained escrow custody (M>0) requires the SEPARATE ack — missing → 400,
// nothing purged.
// - Typed-id gate: a wrong confirm_id → 400, nothing purged.
// - Happy path: external legs FIRST, DB purge LAST; identity + basic config SURVIVE; provenance +
// events SURVIVE; journal completes.
// - Partial failure (external FIRST): a failing external leg leaves the DB UNPURGED and the journal
// retained (resumable) — a re-run converges.
import (
"errors"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
)
// seedResettable seeds a customer with a full operational footprint but NO host (the reset
// precondition). Returns the store. offsiteJSON is the config_json (with an offsite descriptor).
func seedResettable(t *testing.T, st *store.Store, customerID string) {
t.Helper()
cfgJSON := `{"offsite":{"enabled":true,"type":"shared","host":"u1-sub3.your-storagebox.de","user":"u1-sub3","port":23,"repo_path":"/home/felhom","quota_gb":100,"host_fingerprint":"SHA256:abc"}}`
if err := st.SaveCustomerConfig(&store.CustomerConfig{
CustomerID: customerID, CustomerName: "Teszt", Domain: customerID + ".example",
Email: "t@example.com", RetrievalPassword: "pw", APIKey: "capi", ConfigJSON: cfgJSON,
}); err != nil {
t.Fatalf("seed config: %v", err)
}
// A superseded escrow blob retained via F-14 provenance (host deleted, blob kept).
hostID := customerID + "-01"
if err := st.UpsertHost(&store.Host{HostID: hostID, CustomerID: customerID, APIKey: "hapi"}); err != nil {
t.Fatalf("seed host: %v", err)
}
if _, err := st.SaveHostEscrow(hostID, []byte("blobA"), "fpA", "posture", "2026-01-01T00:00:00Z", "shaA"); err != nil {
t.Fatalf("seed escrow A: %v", err)
}
if _, err := st.SaveHostEscrow(hostID, []byte("blobB"), "fpB", "posture", "2026-01-02T00:00:00Z", "shaB"); err != nil {
t.Fatalf("seed escrow B: %v", err)
}
if err := st.DeleteHost(hostID, true); err != nil { // demotes current → retained; records host_deletions
t.Fatalf("delete host (demote): %v", err)
}
if err := st.SaveOneTimeSecret(customerID, "one-time-pw"); err != nil {
t.Fatalf("seed one-time secret: %v", err)
}
if err := st.SaveDRRecipeHostHalf(customerID, hostID, 1, []byte("half")); err != nil {
t.Fatalf("seed dr recipe: %v", err)
}
if _, err := st.RotateClaimCode(customerID, "$2a$10$hashhashhashhashhashha"); err != nil {
t.Fatalf("seed claim: %v", err)
}
}
func postReset(t *testing.T, s *Server, customerID string, form url.Values) *httptest.ResponseRecorder {
t.Helper()
req := httptest.NewRequest("POST", "/configs/"+customerID+"/reset", strings.NewReader(form.Encode()))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
rr := httptest.NewRecorder()
s.handleCustomerReset(rr, req, customerID)
return rr
}
func superseded(t *testing.T, st *store.Store, customerID string) int {
t.Helper()
inv, err := st.CustomerResetInventory(customerID)
if err != nil {
t.Fatalf("inventory: %v", err)
}
return inv.SupersededBlobs
}
func TestCustomerReset_RefusesWhileHostsExist(t *testing.T) {
s, st := newTestServer(t)
s.SetTenantSync(&fakeTenancy{})
seedResettable(t, st, "acme")
// Re-add a live host: RESET must refuse (ruling 3).
if err := st.UpsertHost(&store.Host{HostID: "acme-live", CustomerID: "acme", APIKey: "h"}); err != nil {
t.Fatalf("re-add host: %v", err)
}
rr := postReset(t, s, "acme", url.Values{"confirm_id": {"acme"}, "escrow_ack": {"1"}})
if rr.Code != http.StatusConflict {
t.Fatalf("reset-with-host = %d, want 409", rr.Code)
}
// ZERO side effects: no journal row, secret intact, blobs intact, claim intact.
if cr, _ := st.LatestCustomerReset("acme"); cr != nil {
t.Errorf("a journal row was opened despite the refusal: %+v", cr)
}
if inv, _ := st.CustomerResetInventory("acme"); !inv.OneTimeSecretPresent || !inv.ClaimPresent || inv.SupersededBlobs == 0 {
t.Errorf("refused reset still mutated state: %+v", inv)
}
}
// Red-proof (a): the escrow-custody ack gate. M>0 and no ack → 400, nothing purged. Dropping the
// `!escrowAck` guard would let the reset proceed and destroy the retained blobs → this FAILS.
func TestCustomerReset_EscrowAckRequired(t *testing.T) {
s, st := newTestServer(t)
s.SetTenantSync(&fakeTenancy{})
seedResettable(t, st, "acme")
if superseded(t, st, "acme") == 0 {
t.Fatal("precondition: expected retained blobs to gate on")
}
rr := postReset(t, s, "acme", url.Values{"confirm_id": {"acme"}}) // no escrow_ack
if rr.Code != http.StatusBadRequest {
t.Fatalf("reset without escrow ack = %d, want 400", rr.Code)
}
if cr, _ := st.LatestCustomerReset("acme"); cr != nil {
t.Errorf("journal opened despite the ack refusal: %+v", cr)
}
if superseded(t, st, "acme") == 0 {
t.Error("retained blobs were destroyed despite the missing ack")
}
}
func TestCustomerReset_TypedIDMustMatch(t *testing.T) {
s, st := newTestServer(t)
s.SetTenantSync(&fakeTenancy{})
seedResettable(t, st, "acme")
rr := postReset(t, s, "acme", url.Values{"confirm_id": {"acmee"}, "escrow_ack": {"1"}})
if rr.Code != http.StatusBadRequest {
t.Fatalf("reset with wrong confirm_id = %d, want 400", rr.Code)
}
if inv, _ := st.CustomerResetInventory("acme"); !inv.OneTimeSecretPresent {
t.Error("mismatched-id reset still purged state")
}
}
func TestCustomerReset_HappyPath(t *testing.T) {
s, st := newTestServer(t)
fake := &fakeTenancy{deprovisionExisted: true}
s.SetTenantSync(fake)
seedResettable(t, st, "acme")
rr := postReset(t, s, "acme", url.Values{"confirm_id": {"acme"}, "escrow_ack": {"1"}})
if rr.Code != http.StatusSeeOther {
t.Fatalf("happy reset = %d (%s), want 303", rr.Code, rr.Body.String())
}
// External leg ran.
if fake.deprovisionCalls != 1 {
t.Errorf("pbs deprovision calls = %d, want 1", fake.deprovisionCalls)
}
// Operational state DIED.
inv, _ := st.CustomerResetInventory("acme")
if inv.OneTimeSecretPresent || inv.DRRecipePresent || inv.ClaimPresent || inv.SupersededBlobs != 0 {
t.Errorf("operational state survived the reset: %+v", inv)
}
// Identity + basic config SURVIVE; the offsite tier CHOICE is kept, provisioned fields cleared.
cfg, _ := st.GetCustomerConfig("acme")
if cfg == nil {
t.Fatal("customer_config was destroyed — identity must survive a RESET")
}
if cfg.CustomerName != "Teszt" || cfg.Email != "t@example.com" {
t.Errorf("identity mutated: %+v", cfg)
}
if !strings.Contains(cfg.ConfigJSON, `"enabled":true`) || !strings.Contains(cfg.ConfigJSON, `"type":"shared"`) {
t.Errorf("offsite tier choice was lost: %s", cfg.ConfigJSON)
}
if strings.Contains(cfg.ConfigJSON, "your-storagebox.de") || strings.Contains(cfg.ConfigJSON, "u1-sub3") ||
strings.Contains(cfg.ConfigJSON, "repo_path") || strings.Contains(cfg.ConfigJSON, "host_fingerprint") {
t.Errorf("provisioned offsite fields survived the reset: %s", cfg.ConfigJSON)
}
// Journal completed with every leg recorded ok; provenance + audit event SURVIVE.
cr, _ := st.LatestCustomerReset("acme")
if cr == nil || cr.CompletedAt == nil {
t.Fatalf("journal not finished: %+v", cr)
}
for _, leg := range []string{"pbs", "claim", "descriptor", "db_purge"} {
if cr.Legs[leg] != "ok" {
t.Errorf("leg %q = %q, want ok (legs=%v)", leg, cr.Legs[leg], cr.Legs)
}
}
if ev, _ := st.GetLatestEventByType("acme", "customer_reset"); ev == nil {
t.Error("no customer_reset audit event was recorded")
}
}
// Red-proof (b): partial failure. A failing external leg (PBS) must leave the DB UNPURGED and the
// journal retained — the reset is resumable. Purging before the external legs succeed would erase the
// descriptors that tell a re-run what still needs tearing down → this FAILS.
func TestCustomerReset_PartialFailureIsResumable(t *testing.T) {
s, st := newTestServer(t)
fake := &fakeTenancy{err: errors.New("pbs boom")}
s.SetTenantSync(fake)
seedResettable(t, st, "acme")
rr := postReset(t, s, "acme", url.Values{"confirm_id": {"acme"}, "escrow_ack": {"1"}})
if rr.Code != http.StatusBadGateway {
t.Fatalf("partial reset = %d, want 502", rr.Code)
}
// Nothing purged — the external leg failed FIRST, before any DB mutation.
inv, _ := st.CustomerResetInventory("acme")
if !inv.OneTimeSecretPresent || !inv.DRRecipePresent || !inv.ClaimPresent || inv.SupersededBlobs == 0 {
t.Errorf("DB was purged despite the external leg failing: %+v", inv)
}
cfg, _ := st.GetCustomerConfig("acme")
if !strings.Contains(cfg.ConfigJSON, "your-storagebox.de") {
t.Errorf("descriptor was cleared despite the failure: %s", cfg.ConfigJSON)
}
cr, _ := st.LatestCustomerReset("acme")
if cr == nil || cr.CompletedAt != nil || cr.Legs["pbs"] != "failed" {
t.Fatalf("journal should be open with pbs=failed: %+v", cr)
}
// Resume: the external leg now succeeds; a re-run converges (idempotent from the top).
fake.err = nil
rr2 := postReset(t, s, "acme", url.Values{"confirm_id": {"acme"}, "escrow_ack": {"1"}})
if rr2.Code != http.StatusSeeOther {
t.Fatalf("resumed reset = %d (%s), want 303", rr2.Code, rr2.Body.String())
}
inv2, _ := st.CustomerResetInventory("acme")
if inv2.OneTimeSecretPresent || inv2.DRRecipePresent || inv2.ClaimPresent || inv2.SupersededBlobs != 0 {
t.Errorf("resume did not converge: %+v", inv2)
}
}
+3
View File
@@ -34,6 +34,9 @@ import (
type tenancyProvisioner interface {
Provision(ctx context.Context, customerID string) (*tenantsync.Result, error)
Reissue(ctx context.Context, customerID string) (*tenantsync.Result, error)
// Deprovision DESTROYS the customer's PBS namespace + backups + token (customer-RESET teardown,
// v0.61.0). Idempotent — existed=false when nothing was there. The shared user is never touched.
Deprovision(ctx context.Context, customerID string) (existed bool, err error)
}
// SetTenantSync enables PBS DR tier provisioning (optional). Without it, saving a config with the
+19 -5
View File
@@ -24,11 +24,14 @@ import (
)
type fakeTenancy struct {
provisionCalls int
reissueCalls int
err error // both ops fail with this
provisionErr error // Provision-only failure (the F-14 token_exists shape: reissue still works)
secret string
provisionCalls int
reissueCalls int
deprovisionCalls int
err error // both ops fail with this
provisionErr error // Provision-only failure (the F-14 token_exists shape: reissue still works)
deprovisionErr error // Deprovision-only failure (RESET partial-failure red-proof)
deprovisionExisted bool // what Deprovision reports (namespace existed / was destroyed)
secret string
}
func (f *fakeTenancy) result(customerID string) *tenantsync.Result {
@@ -60,6 +63,17 @@ func (f *fakeTenancy) Reissue(ctx context.Context, customerID string) (*tenantsy
return f.result(customerID), nil
}
func (f *fakeTenancy) Deprovision(ctx context.Context, customerID string) (bool, error) {
f.deprovisionCalls++
if f.deprovisionErr != nil {
return false, f.deprovisionErr
}
if f.err != nil {
return false, f.err
}
return f.deprovisionExisted, nil
}
// newPBSDRServer builds a server + store with the full provisioning preconditions satisfied:
// customer config, enrolled host, WG endpoint record, bound WG peer. The logger is captured so
// tests can grep-assert the secret never reaches it.
+9
View File
@@ -508,6 +508,15 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
} else {
http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
}
case strings.HasPrefix(path, "/configs/") && strings.HasSuffix(path, "/reset"):
// Customer RESET (v0.61.0): GET renders the confirm surface (live inventory), POST executes.
customerID := strings.TrimPrefix(path, "/configs/")
customerID = strings.TrimSuffix(customerID, "/reset")
if r.Method == http.MethodPost {
s.handleCustomerReset(w, r, customerID)
} else {
s.handleCustomerResetPreview(w, r, customerID)
}
case strings.HasPrefix(path, "/configs/"):
// Redirect old config detail URL to unified customer page
customerID := strings.TrimPrefix(path, "/configs/")
@@ -55,6 +55,7 @@
{{else if eq .Flash "log_tail_requested"}}Log tail requested — the controller delivers it on its next report cycle (a few minutes). A customer-visible event line was recorded.
{{else if eq .Flash "claim-resent"}}Code re-sent to the registered address. A kód a doboz következő jelentésekor (~15 percen belül) aktiválódik.
{{else if eq .Flash "claim-resend-failed"}}Claim code resend FAILED — check the hub log (email delivery / send error).
{{else if eq .Flash "reset_done"}}Customer RESET complete — every operational trace was destroyed (offsite repo, PBS namespace, DR recipe, claim state, retained escrow custody). Identity and basic config survive; the audit event stream records it.
{{end}}
</div>
{{end}}
@@ -702,6 +703,84 @@
{{end}}
{{if .HasConfig}}
<!-- Reset customer (v0.61.0): the MIDDLE lifecycle tier — host delete < RESET < customer Delete.
One action returns the customer to pre-first-install: every OPERATIONAL trace dies (offsite
repo, PBS namespace, DR recipe, one-time secret, claim state, retained escrow custody), while
IDENTITY and the basic config SURVIVE. Amber (--warn), deliberately distinct from the red
Danger-zone Delete below it. Refuses while any host row exists (delete hosts first). -->
<section class="card" style="border-color: var(--warn);">
<h2>Ügyfél-visszaállítás <span class="text-muted" style="font-size: 0.8em; font-weight: normal;">(RESET — pre-első-telepítés)</span></h2>
<p class="text-muted">Egyetlen művelettel visszaállítja az ügyfelet az első telepítés előtti állapotba: <strong>minden működési állapot törlődik</strong> (offsite tároló, PBS névtér, DR-recept, egyszeri jelszó, azonosítási állapot). Az <strong>azonosság és az alapkonfiguráció megmarad</strong> (ügyfélrekord, előzmények, események). Ez NEM törli a hostokat — ha még van host, előbb azt kell törölni. Kevesebb, mint a Danger zone Delete: az ügyfél megmarad, csak a működési nyomok tűnnek el.</p>
<button type="button" class="btn btn-sm" style="border-color: var(--warn); color: var(--warn);" onclick="customerResetConfirm('{{.CustomerID}}')">Ügyfél visszaállítása&hellip;</button>
<div id="cust-reset-confirm-{{.CustomerID}}" style="display: none; margin-top: 0.75rem; padding: 0.75rem; border: 1px solid var(--warn); background: var(--warn-dim); border-radius: var(--radius); max-width: 46em;">
<p id="cust-reset-inv-{{.CustomerID}}" style="margin: 0 0 0.5rem; font-size: 0.9em;">&hellip;</p>
<label id="cust-reset-escrow-row-{{.CustomerID}}" style="display: none; margin: 0 0 0.6rem; font-size: 0.85em; color: var(--crit);">
<input type="checkbox" id="cust-reset-escrow-{{.CustomerID}}">
<strong>Megőrzött helyreállítási-kulcs letét megsemmisítése</strong> — külön megerősítés (ez visszafordíthatatlanul törli a megőrzött escrow blobokat).
</label>
<p style="margin: 0 0 0.4rem; font-size: 0.85em; color: var(--text-2);">Írd be az ügyfél azonosítóját a megerősítéshez:</p>
<form method="POST" action="/configs/{{.CustomerID}}/reset" id="cust-reset-form-{{.CustomerID}}" style="display: flex; gap: 0.5rem; align-items: center; flex-wrap: wrap;">
{{.CSRFField}}
<input type="hidden" name="confirm_id" id="cust-reset-confirm-hidden-{{.CustomerID}}" value="">
<input type="hidden" name="escrow_ack" id="cust-reset-escrow-hidden-{{.CustomerID}}" value="">
<input type="text" id="cust-reset-input-{{.CustomerID}}" placeholder="ügyfél-azonosító&hellip;" style="padding: 0.3em 0.5em; width: 16em;">
<button type="button" class="btn btn-sm" id="cust-reset-go-{{.CustomerID}}" style="border-color: var(--warn); color: var(--warn);" onclick="customerResetSubmit('{{.CustomerID}}')">Megerősítés &amp; visszaállítás</button>
<button type="button" class="btn btn-sm btn-outline" onclick="document.getElementById('cust-reset-confirm-{{.CustomerID}}').style.display='none';">Mégse</button>
</form>
<p id="cust-reset-err-{{.CustomerID}}" style="margin: 0.4em 0 0; font-size: 0.8em; color: var(--crit);"></p>
</div>
</section>
<script>
function customerResetConfirm(cid) {
var box = document.getElementById('cust-reset-confirm-' + cid);
var inv = document.getElementById('cust-reset-inv-' + cid);
var go = document.getElementById('cust-reset-go-' + cid);
document.getElementById('cust-reset-input-' + cid).value = '';
document.getElementById('cust-reset-err-' + cid).textContent = '';
box.style.display = 'block';
inv.textContent = 'Leltár lekérése…';
go.disabled = false;
fetch('/configs/' + encodeURIComponent(cid) + '/reset')
.then(function(r){ return r.json(); })
.then(function(d){
if (d.refused) {
inv.innerHTML = '<strong style="color: var(--crit)">Elutasítva:</strong> ehhez az ügyfélhez még ' + d.host_count +
' host tartozik. A RESET soha nem töröl hostot — előbb töröld a host(oka)t.';
go.disabled = true;
document.getElementById('cust-reset-escrow-row-' + cid).style.display = 'none';
return;
}
var dies = [];
if (d.offsite_enabled) dies.push('offsite tároló' + (d.offsite_identifier ? ' (' + d.offsite_identifier + ')' : ''));
if (d.pbs_tenancy_configured) dies.push('PBS névtér + mentések');
if (d.dr_recipe_present) dies.push('DR-recept');
if (d.one_time_secret) dies.push('egyszeri jelszó');
if (d.claim_present) dies.push('azonosítási állapot (friss kód a következő onboardingnál)');
if (d.superseded_blobs > 0) dies.push(d.superseded_blobs + ' megőrzött escrow blob');
inv.innerHTML = '<strong>Törlődik:</strong> ' + (dies.length ? dies.join(', ') : 'nincs működési állapot') +
'. <strong>Megmarad:</strong> ügyfélrekord, alapkonfiguráció, előzmények, események.';
var escrowRow = document.getElementById('cust-reset-escrow-row-' + cid);
escrowRow.style.display = d.escrow_ack_required ? 'block' : 'none';
document.getElementById('cust-reset-escrow-' + cid).checked = false;
})
.catch(function(){ inv.textContent = 'A leltár nem kérhető le — a szerver minden feltételt így is kikényszerít.'; });
}
function customerResetSubmit(cid) {
var typed = document.getElementById('cust-reset-input-' + cid).value.trim();
var err = document.getElementById('cust-reset-err-' + cid);
if (typed !== cid) { err.textContent = 'A beírt azonosító nem egyezik.'; return; }
var escrowRow = document.getElementById('cust-reset-escrow-row-' + cid);
var escrowCb = document.getElementById('cust-reset-escrow-' + cid);
if (escrowRow.style.display !== 'none' && !escrowCb.checked) {
err.textContent = 'A megőrzött kulcs-letét megsemmisítéséhez pipáld be a külön megerősítést.';
return;
}
document.getElementById('cust-reset-confirm-hidden-' + cid).value = typed;
document.getElementById('cust-reset-escrow-hidden-' + cid).value = escrowCb.checked ? '1' : '';
document.getElementById('cust-reset-form-' + cid).submit();
}
</script>
<!-- Danger zone (v0.48.0 edit-a): the Block/Delete forms relocated verbatim from the
Customer Info header — endpoints and confirm() handlers unchanged. -->
<section class="card">
+12
View File
@@ -1,5 +1,17 @@
# Felhom scripts — Changelog
## felhom-tenantsync.sh v1.1.0 — deprovision op (customer RESET teardown) (2026-07-17)
Adds the `{"op":"deprovision","customer_id":"<id>"}` op the slice-1 header explicitly reserved
("namespace/data deletion is a deliberate, separate decision"). It is exactly that deliberate,
hub-side ack-gated decision (the customer RESET, hub v0.61.0): delete the token (its ACLs purge with
it) → delete the residual namespace ACLs → **destroy the namespace AND all its backup groups**
(`proxmox-backup-client namespace delete <ns> --delete-groups true`, via the transient admin token).
IDEMPOTENT — a missing token / namespace is success (`deleted:false`), so a re-run after a partial
reset converges. The shared `felhom@pbs` user is NEVER touched (co-tenants ride it). Returns
`{"status":"ok","namespace","datastore","deleted":<bool>}`. Secret hygiene unchanged (no secrets in
this path). Client seam: `tenantsync.Deprovision(ctx, customerID) (existed bool, err error)`.
## felhom-host-install.sh v1.17.0 — appliance guest auto-sizing (F5) + doc-drift fix (2026-07-17)
Closes `VALIDATION-n100-baremetal-2026-07-16.md` **F5 (MEDIUM):** appliance mode provisioned the
+34 -5
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# felhom-tenantsync v1.0.0 — the offsite endpoint's per-customer PBS tenancy surface (PBS DR tier
# felhom-tenantsync v1.1.0 — the offsite endpoint's per-customer PBS tenancy surface (PBS DR tier
# SLICE 1; spike SPIKE-pbs-tier-provisioning-2026-07-10 §3).
#
# Runs as the SSH forced command for the hub's SECOND `felhom-peersync` key (via sudo — its own
@@ -14,11 +14,16 @@
# → {"status":"ok","token_id","token_secret","fingerprint","datastore","namespace"}
# {"op":"reissue","customer_id":"<id>"} → delete-token (its ACLs purge with it — spike) →
# recreate → re-grant BOTH → self-check → same ok-shape with the FRESH secret.
# {"op":"deprovision","customer_id":"<id>"} → the customer-RESET teardown (v0.61.0, hub-side
# ack-gated). Delete the token (ACLs purge with it) → delete the residual namespace ACLs →
# DESTROY the namespace AND all its backup groups (`namespace delete --delete-groups true`).
# IDEMPOTENT: a missing token / missing namespace is success, not an error (a re-run after a
# partial reset converges). The shared felhom@pbs USER is NEVER touched (other tenants ride it).
# → {"status":"ok","namespace","datastore","deleted":<bool ns existed>}.
# This is the DELIBERATE, gated data-destruction the slice-1 note reserved — the operator RESET
# confirm (typed customer-id + separate escrow-custody ack) is the human decision it demanded.
# {"op":"fingerprint"} → {"status":"ok","fingerprint":"<PBS cert sha256>"}
#
# NO deprovision op in slice 1 — namespace/data deletion is a deliberate, separate decision (the
# offsite-disable precedent: disable never destroys data).
#
# Secret hygiene (load-bearing):
# - The token secret exists ONLY in memory and in the final stdout JSON — never a file, never
# stderr (the hub embeds remote stderr in error logs), never an argument.
@@ -63,7 +68,7 @@ if [ "$OP" = "fingerprint" ]; then
exit 0
fi
case "$OP" in provision|reissue) ;; *) err_json bad_request "unknown op" ;; esac
case "$OP" in provision|reissue|deprovision) ;; *) err_json bad_request "unknown op" ;; esac
# customer_id → the namespace AND the token name. Conservative charset (PBS ns + token grammar,
# no leading dash/dot so it can never parse as an option).
@@ -91,6 +96,30 @@ proxmox-backup-manager acl update "/datastore/$DS" DatastoreAdmin \
ADMIN_REPO="root@pam!$ADMIN_TOKEN_NAME@$REPO_HOST:$DS"
# 2b. deprovision (v0.61.0 customer-RESET teardown): destroy this ONE tenant's token + namespace +
# backup groups. Every step is idempotent (missing = already gone = ok). The shared felhom@pbs
# user survives (co-tenants). Returns before the provision/reissue create-path below.
if [ "$OP" = "deprovision" ]; then
# token (its ACLs purge with it — spike); ignore "no such token".
proxmox-backup-manager user delete-token "$PBS_USER" "$CID" >&2 2>/dev/null || true
# residual namespace ACLs (belt-and-suspenders — the user grant is not token-scoped).
proxmox-backup-manager acl update "/datastore/$DS/$CID" DatastoreBackup --auth-id "$PBS_USER" --delete >&2 2>/dev/null || true
proxmox-backup-manager acl update "/datastore/$DS/$CID" DatastoreBackup --auth-id "$TOKEN_ID" --delete >&2 2>/dev/null || true
ns_existed=false
if PBS_PASSWORD="$ADM" proxmox-backup-client namespace list --repository "$ADMIN_REPO" \
--output-format json | jq -e --arg ns "$CID" '(.data // .) | any(.[]; .ns == $ns)' >/dev/null; then
ns_existed=true
# --delete-groups true destroys every backup group under the namespace (the deliberate data kill).
PBS_PASSWORD="$ADM" proxmox-backup-client namespace delete "$CID" --repository "$ADMIN_REPO" --delete-groups true >&2
log "deprovision: namespace $CID destroyed (all backup groups deleted)"
else
log "deprovision: namespace $CID absent — already gone"
fi
jq -cn --arg ns "$CID" --arg ds "$DS" --argjson del "$ns_existed" \
'{"status":"ok","namespace":$ns,"datastore":$ds,"deleted":$del}'
exit 0
fi
# 3. Ensure the shared user + the namespace (both idempotent).
if ! proxmox-backup-manager user list --output-format json | jq -e --arg u "$PBS_USER" \
'any(.[]; .userid == $u)' >/dev/null; then