From 7c9647327fd56a14cf638bece27bff6bd1db4410 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Fri, 17 Jul 2026 13:15:57 +0200 Subject: [PATCH] =?UTF-8?q?docs:=20hub=20v0.61.0=20Customer=20RESET=20?= =?UTF-8?q?=E2=80=94=20REPORT/CONTEXT=20+=20ep0=20runbook=20=C2=A710=20(te?= =?UTF-8?q?nantsync=20v1.1.0=20deprovision,=20live-drilled)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CONTEXT.md | 16 ++++ REPORT.md | 94 +++++++++++----------- documentation/runbooks/offsite-endpoint.md | 12 ++- 3 files changed, 74 insertions(+), 48 deletions(-) diff --git a/CONTEXT.md b/CONTEXT.md index 55d584b..98023ab 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -3,6 +3,22 @@ > Created with the REUSE.md rollout (2026-07-03). Authoritative history: `hub/CHANGELOG.md` (hub), > `website/CHANGELOG.md`, `scripts/CHANGELOG.md`; end-of-task detail in `REPORT.md`. +- **2026-07-17 — CUSTOMER RESET SHIPPED: hub v0.61.0 + felhom-tenantsync v1.1.0 (LIVE).** The middle + lifecycle tier (host delete < RESET < customer Delete). One operator action → pre-first-install: all + OPERATIONAL state dies (offsite repo, PBS namespace+backups, DR recipe, one-time secret, claim state, + retained escrow custody); IDENTITY + basic config (incl. the offsite tier CHOICE) + provenance + events + SURVIVE. Rulings: separate escrow-custody ack; clears claim (fresh code next onboarding); REFUSES while + any host row exists; live-counted confirm inventory. Discipline: external teardown FIRST, DB purge LAST, + every leg idempotent → partial run re-runs from the top (purge withheld until externals ok). New: + `store/customer_reset.go` (journal+inventory+ack-gated purge), `claim.ResetToUnclaimed`, + `offsite.{Deprovision,OffsiteIdentifier,ClearProvisionedDescriptor}`, `tenantsync.Deprovision` + + `felhom-tenantsync.sh` **deprovision** op (v1.1.0, destroys ns+groups+token, shared user untouched), + `web/customer_reset.go` (GET inventory JSON / POST orchestration) + an **amber** RESET card distinct + from the red Danger-zone Delete. Red-proofs: ack-gate + partial-failure resumability (both proven red). + **Live-drilled on ep0** (throwaway `drill-reset-01` with a real backup): deprovision `deleted:true`, + idempotent re-run `deleted:false`, all 3 real tenants survived. Hetzner-delete + the password-gated web + POST covered by tests (offsite Deprovision mirrors live-proven ReissueCredentials). See REPORT.md. + - **2026-07-17 — HOST-DELETE DEMOTES ESCROW: hub v0.60.1 (LIVE).** Closes the v0.60.0 review gap: `DeleteHost(deleteEscrow=true)` now DEMOTES the current escrow blob into `host_escrow_superseded` (copy-before-delete, same tx) + spares existing superseded rows — never destroys custody. The diff --git a/REPORT.md b/REPORT.md index 73f8e74..ba0609c 100644 --- a/REPORT.md +++ b/REPORT.md @@ -2,56 +2,58 @@ > **Overwrite** this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in [hub/CHANGELOG.md](hub/CHANGELOG.md); the scripts history lives in [scripts/CHANGELOG.md](scripts/CHANGELOG.md). -## Hub v0.60.1 — host deletion demotes escrow custody (never destroys) + S6b verdict — 2026-07-17 +## Hub v0.61.0 + felhom-tenantsync v1.1.0 — Customer RESET (middle lifecycle tier) — 2026-07-17 -**Baseline:** hub v0.60.0 (`106c337`) → **v0.60.1** (LIVE, ArgoCD Synced/Healthy). Hub-only; no -controller/agent change. Closes the deletion-path gap found in the v0.60.0 review. Green: -`go build ./... && go vet ./... && go test ./...` + hub confirm gate. +**Commits:** `4009401` (code), `e144c5e` (manifest). **Live:** hub **0.61.0** on k3s (Synced/Healthy); +felhom-tenantsync **v1.1.0** on ep0 (`felhom-hetzner`). -### Scenario A — host delete demotes, never destroys (`store.DeleteHost`) -The v0.60.0 code deleted `host_escrow` + `host_escrow_superseded` on `deleteEscrow=true`. Now it -DEMOTES the current row into `host_escrow_superseded` (copy-BEFORE-delete, same tx) and SPARES existing -superseded rows. Extracted THE one escrow row-copy routine `demoteCurrentEscrowTx` (INSERT…SELECT) and -refactored `SaveHostEscrow` to use it too — one copy routine, not two (rule 2). The F-14 provenance row -+ gate semantics are unchanged (comments reworded: demotion, not destruction). Edge: no escrow row → -unchanged; the `ErrHostEscrowPresent` refusal without the flag is unchanged. -**Red-proof (run-fail-revert):** `TestDeleteHost_DemotesEscrowNeverDestroys` — pre-fix (delete both) → -retained=0 → FAIL; fixed → both blobs (demoted current + spared superseded) retrievable, host gone. +### What shipped -### Scenario B — customer delete is the purge point (`store.DeleteCustomerConfig`) -**What it did before:** ONLY `DELETE FROM customer_configs` — it touched neither hosts nor escrow -(so deleting a customer left hosts + escrow blobs orphaned — an offboarding gap, see Observations). -**Now:** one tx that purges `host_escrow` AND `host_escrow_superseded` for all the customer's hosts — -INCLUDING already-deleted hosts (resolved via the F-14 `host_deletions` provenance `UNION`), so a -host-delete-then-customer-delete ordering leaves nothing orphaned. Danger-zone copy states it. -**Red-proof:** `TestDeleteCustomer_PurgesEscrowCustody` (pre-fix config-only → blobs survive → FAIL). +The **middle lifecycle tier** between *host delete* and *customer Delete*. One operator action returns a +customer to **pre-first-install**: every OPERATIONAL trace dies — offsite repo (Hetzner sub-account/box), +PBS namespace + all backup groups + token, DR recipe, one-time secret, claim state, retained escrow +custody — while **identity + basic config survive** (`customer_configs` incl. the offsite tier CHOICE, all +provenance rows, and the audit-event stream). -### Scenario C — wording (`host_detail_body.html`, `hosts.go`, `customer_unified.html`) -Host-delete escrow checkbox → "Move key escrow to retained custody (required when escrow present) + -remove DR bundle for this host"; the 409 refusal message → "…move it to retained custody…"; the -customer Danger-zone copy → "…permanently removes the retained recovery-key custody (escrow blobs)… -This is the one true purge point; host deletion only demotes custody, never destroys it." Guard test -`TestHostDeleteEscrowLabel_DemotionWording`. +Viktor's rulings, all implemented: (1) destroying retained escrow custody gets its **own separate +acknowledgment**; (2) RESET clears claim state → fresh code next onboarding; (3) RESET **refuses while any +host row exists** (delete hosts first — reset never deletes hosts); (4) the confirm surface shows a +**live-counted** inventory. -### Scenario D — S6b necessity verdict: **OBSOLETE** -**Verdict + landmarks:** re-enrolling an existing host_id upserts cleanly — `store.UpsertHost` is -`INSERT … ON CONFLICT(host_id) DO UPDATE` (`internal/store/store.go:2099`), and -`handleAdminCreateHost` (`internal/api/handler.go:913`) calls it with NO duplicate refusal (it even -mints a fresh api_key). Combined with the v0.57.0 re-enroll arc (claim/offsite/escrow-honesty -auto-fire on re-enrollment, `reissueOnReenroll` `handler.go:1020`), no manual stale-host deletion is -required before re-enroll. Since S6b is not a literal runbook step, the verdict is recorded here + in -ROADMAP R-3 ("re-enrollment upserts cleanly; S6b obsolete; escrow cascade now defined; the rehearsal -live-confirms it"). Scenario A also makes the funnel harmless either way (deletion demotes, not -destroys). +### Discipline -### Deploy verification -`felhom-hub:0.60.1` built + pushed; `manifests/hub.yaml` bumped; ArgoCD Synced/Healthy; rollout -complete (`:0.60.1`). The `demoteCurrentEscrowTx` path is INSERT…SELECT (atomic within the delete tx). +External teardown FIRST, DB purge LAST (publish-last). Every leg idempotent → a partial run is re-run from +the top; a failed external leg is a clean journal entry and the DB purge (which erases the descriptors that +say what still needs tearing down) is **withheld** until every external leg is `ok`. Provenance + events +are never wiped. -### Observations -- **Customer-delete offboarding gap (unchanged this task, by scope):** `DeleteCustomerConfig` still - does NOT delete the customer's HOST rows or the broader offboarding (Hetzner sub-account, WG peer, - Storage-Box data) — only the escrow tables + the config row. That non-escrow lifecycle stays the - delete/re-create rehearsal's question (ROADMAP R-3). -- No pruning added (Viktor's data-first ruling); the ACK assembly + the upload supersede path are - untouched. +### Components + +- **store/customer_reset.go**: `customer_resets` journal (per-attempt, per-leg, resumable); + `CustomerResetInventory` (live counts incl. retained blobs via the F-14 `host_deletions` UNION); + `Start/UpdateResetLeg/Finish/LatestCustomerReset`; ack-gated `PurgeCustomerResetDBState`; `DeleteClaim`. +- **claim.ResetToUnclaimed**: deletes the claim row → `EnsureIssued` mints a fresh first code next onboarding. +- **offsite**: `Deprovision` (idempotent label-lookup delete), `OffsiteIdentifier`, `ClearProvisionedDescriptor`. +- **tenantsync + scripts/felhom-tenantsync.sh v1.1.0**: `deprovision` op destroys the namespace + backup + groups + token; shared `felhom@pbs` user untouched; idempotent. +- **web/customer_reset.go**: `GET /configs/{id}/reset` → inventory JSON; `POST` → orchestration (all gates + before any write/external call). A distinct **amber** RESET card + `reset_done` flash. + +### Verification + +- **Green gate:** `go build ./... && go vet ./... && go test ./...` all pass. Hub confirm gate OK. +- **Red-proofs (run-fail-revert, both proven red then restored):** (a) escrow-ack gate — defeat it → + reset proceeds & destroys blobs → FAIL; (b) partial-failure resumability — don't withhold the purge → + DB purged despite an external failure → FAIL. Plus store ack-gating + journal round-trip; offsite + Deprovision idempotency + descriptor clear; RESET-card render. +- **Live drill on ep0 (throwaway `drill-reset-01`, real PBS data):** provision → **real backup written into + the namespace** → deprovision (`deleted:true`; namespace + backup group + token destroyed) → token/ns + confirmed gone → idempotent re-run (`deleted:false`) → **all 3 real tenants + the shared user survived**. + +### Not live-drilled (covered otherwise) + +The web RESET POST and the Hetzner sub-account delete are covered by the httptest-driven orchestration +tests + offsite unit tests (the offsite `Deprovision` faithfully mirrors the live-proven +`ReissueCredentials`, swapping reset→delete). The hub UI is operator-password-gated → CC verifies the UI +via render tests, not a live click. A live Hetzner create/delete drill was not run (billed external +service; mirror-of-live-proven + idempotency-tested). diff --git a/documentation/runbooks/offsite-endpoint.md b/documentation/runbooks/offsite-endpoint.md index 4f1468e..35ab522 100644 --- a/documentation/runbooks/offsite-endpoint.md +++ b/documentation/runbooks/offsite-endpoint.md @@ -353,8 +353,16 @@ The hub's SECOND forced-command surface on the endpoint: per-customer PBS tenanc privilege-separated token + dual-grant, the §4a dance automated). Same low-priv user as peersync, its OWN keypair + script + sudoers drop-in — **the peersync files are untouched** (one script, one job; peersync script + sudoers sha256 asserted identical before/after at install). Script source of truth: -`scripts/felhom-tenantsync.sh` (v1.0.0; JSON stdin/stdout, ops `provision`/`reissue`/`fingerprint`, -NO deprovision — data deletion stays a deliberate decision). Hub client: `hub/internal/tenantsync`. +`scripts/felhom-tenantsync.sh` (**v1.1.0**; JSON stdin/stdout, ops +`provision`/`reissue`/`deprovision`/`fingerprint`). Hub client: `hub/internal/tenantsync`. + +> **v1.1.0 (2026-07-17, deployed to ep0):** adds the `deprovision` op — the deliberate data-deletion +> the slice-1 header reserved, now driven by the hub-side **customer RESET** (hub v0.61.0, operator +> ack-gated). It destroys the customer's namespace + all backup groups + token +> (`namespace delete --delete-groups true`); the shared `felhom@pbs` user is never touched; idempotent +> (`deleted:false` when nothing was there). To update the on-box script, re-run the install block below +> with the new source (a `.bak-` is kept). Live-drilled 2026-07-17: provision → real backup → +> deprovision (`deleted:true`, group destroyed) → idempotent re-run → all real tenants survived. On the build server (the hub's credential; mirror of step 6): ```sh