From 5f5e3c54a10c3a9705086ec9d56810a69a9c2a64 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Tue, 16 Jun 2026 18:49:45 +0200 Subject: [PATCH] =?UTF-8?q?hub=20v0.13.0:=20DR=20recipe=20=E2=80=94=20asse?= =?UTF-8?q?mble=20+=20store=20+=20view=20the=20secret-free=20reconstructio?= =?UTF-8?q?n=20recipe?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DR recipe slice (hub half), grounded in SPIKE-dr-recipe-2026-06-16. The hub receives two additive dr_recipe halves on the existing report paths (agent storage/guest/PBS on host-report; controller customer/apps on the controller report), stores them PLAINTEXT in a DEDICATED dr_recipe table keyed by customer (each half preserves the other), and AssembleDRRecipe stitches them into one operator-readable recipe (ignore-unknown + version-skew tolerant). View: a DR-recipe panel on the customer page + GET /customers/{id}/dr-recipe.json download (operator-auth, no secrets to redact). Plaintext-at-rest is correct — the recipe is the clean inverse of the retired infra-backup. Tests: store round-trip (each half preserves the other), assemble-matches-golden, ignore-unknown + version skew, partial halves, no-secrets sweep. Manifest tag bumped to v0.13.0. Co-Authored-By: Claude Opus 4.8 (1M context) --- REPORT.md | 109 +++++------ hub/CHANGELOG.md | 30 +++ hub/README.md | 4 + hub/internal/api/handler.go | 35 ++++ hub/internal/store/dr_recipe.go | 130 +++++++++++++ hub/internal/store/dr_recipe_test.go | 182 ++++++++++++++++++ hub/internal/store/store.go | 20 ++ .../store/testdata/dr-recipe.golden.json | 30 +++ hub/internal/web/configs.go | 29 ++- hub/internal/web/dr_recipe.go | 61 ++++++ hub/internal/web/server.go | 4 + .../web/templates/customer_unified.html | 30 +++ manifests/hub.yaml | 2 +- 13 files changed, 597 insertions(+), 69 deletions(-) create mode 100644 hub/internal/store/dr_recipe.go create mode 100644 hub/internal/store/dr_recipe_test.go create mode 100644 hub/internal/store/testdata/dr-recipe.golden.json create mode 100644 hub/internal/web/dr_recipe.go diff --git a/REPORT.md b/REPORT.md index 59852ff..102d83b 100644 --- a/REPORT.md +++ b/REPORT.md @@ -4,80 +4,61 @@ --- -# Hub v0.12.0 — retire Infra Backup, purge plaintext secrets, fix backup-deadline email +## REPORT — hub v0.13.0: DR recipe (assemble + store + view) -**Date:** 2026-06-16 -**Scope:** Phase-1 of `documentation/audits/SPIKE-infra-backup-2026-06-15.md` (hub side). -**Deployed:** `felhom-hub:v0.12.0` live on k3s (ArgoCD `felhom` app Synced); commit `0635640`. -**Companion repo:** `felhom-controller` v0.69.0 (own REPORT there). +**TASK — DR recipe slice (hub half).** Receive the two secret-free recipe halves, assemble them into one +customer recipe, store PLAINTEXT in a dedicated table, and expose an operator view/download. Grounded in +`documentation/audits/SPIKE-dr-recipe-2026-06-16.md`. Pairs with felhom-agent v0.38.0 (storage/guest/PBS +half) + felhom-controller v0.73.0 (customer/apps half, the boundary-enforcing emitter). -## What shipped +### Store (`hub/internal/store/dr_recipe.go`) -1. **Backup-deadline check repointed to PBS freshness** (`internal/monitor/deadline.go`). - The backup half no longer queries for a `backup_completed` event (nothing emits it post-slice-8C, - so it fired daily for every healthy customer). It now reads the customer's **latest agent - host-report** (`store.GetLatestHostReportJSON`) and raises `expected_backup_missed` only on - positive evidence: no PBS snapshot / successful vzdump at all, newest backup older than **26h**, or - the newest PBS snapshot's `verify_state == "failed"`. A fresh-but-unverified snapshot is **not** a - failure (PBS verifies on its own cadence — alarming on it would recreate the false alarm). The - **db-dump half is unchanged**. A customer with **no host-report** gets no backup alarm here - (liveness is the host-staleness checker's job). +- New `dr_recipe` table (migrate()) — DEDICATED, separate from the opaque `host_escrow` and from the + dropped `infra_backup*` tables. Keyed by `customer_id`; columns `recipe_version, host_id, + host_half_json, app_half_json, updated_at`. +- `SaveDRRecipeHostHalf` / `SaveDRRecipeAppHalf` — each upserts its half and PRESERVES the other + (last-write-wins per half; a re-report of one half never clobbers the other). +- `AssembleDRRecipe(rec)` — stitches the two halves into `AssembledRecipe{recipe_version, customer, + guests, pbs, drives, pve_storage, apps}`. Sub-sections pass through as `json.RawMessage` (verbatim); + **ignore-unknown** at the top level + version-skew tolerant (`recipe_version` = max) for forward-compat + across the three repos. Pure → unit-tested. -2. **Infra Backup feature removed** (`api/handler.go`, `store/store.go`, `web/configs.go`, - `templates/customer_unified.html`, `templates/customer.html`): the `POST/GET /api/v1/infra-backup[…]` - endpoints + handlers, the store methods/types (`SaveInfraBackup`/`GetInfraBackup`/`GetInfraBackupByID`/ - `GetInfraBackupMeta`/`ListInfraBackupVersions`/`pruneInfraBackups`, `InfraBackupMeta`/`InfraBackupVersion`), - and the operator "Infra Backup" panel. `GET /api/v1/recovery/{id}` now returns **config_yaml only**. - The customer-page config-drift badge (diffed against the stored controller.yaml) is hidden; the live - "Show Diff" path is unaffected. +### Ingest (`hub/internal/api/handler.go`) -3. **Plaintext secret purge** (`store/store.go migrate()`): `DROP infra_backup_versions; DROP - infra_backups; VACUUM; wal_checkpoint(TRUNCATE)` — gated on table existence so normal restarts skip - it. VACUUM physically reclaims the freed pages so the plaintext keys/tokens are not merely delinked. +- `handleHostReport` persists the `dr_recipe` host-half (keyed by the host's customer). +- `handleReport` persists the `dr_recipe` app-half (keyed by `customer_id`), mirroring the + `app_telemetry` pattern. Both backward-compatible (old agents/controllers omit the field) and never + fatal to the heartbeat. -## Gate (STEP 1a) — PASSED before any change +### View (`hub/internal/web/dr_recipe.go` + customer page) -The demo's latest host-report (id 699, agent v0.36.7) carried **5 PBS snapshots, all `verify=ok`, -newest `2026-06-15T18:41:29Z` (13.9 h old, < 26 h)** plus a matching successful vzdump. The repoint's -data source is present and fresh, so the repoint alone clears the email. +- A DR-recipe panel on the customer detail page (which half landed + last-updated) with a + **Download recipe (JSON)** link. +- `GET /customers/{id}/dr-recipe.json` serves the assembled recipe (operator dashboard-auth, pretty + JSON, `Content-Disposition` attachment). No decrypt, nothing to redact. -## Verification (live, non-hollow) +### Boundary -- **Build + tests:** `go build ./... && go test ./...` green. `deadline_test.go` covers fresh+verified→ - quiet (the **companion**), stale→alarm, failed-verify→alarm, no-host-report→quiet, db-dump half - preserved, and a pure `assessBackupFreshness` table. Red-proof: with the old event-based half - temporarily restored, the companion + no-report + db-dump-preserved tests **fail**, while the - pure-helper test still passes — the behavioral tests are sensitive to the logic. -- **Deploy:** hub v0.12.0 rolled out; startup log: `Retired infra-backup: dropped 2 table(s) and - VACUUMed to reclaim plaintext pages`. -- **Tables gone:** `.tables` on the live `/data/hub.db` shows no `infra_backup*`. DB shrank - **65.8 MB → 52.2 MB**; no `-wal` sidecar lingers. -- **Secret purge proof (grep on the live post-VACUUM DB file):** the infra-backup-exclusive markers - are gone — `encryption_key_b64` 17→**0**, `controller_config_b64` 11→**0**, and the actual - AES-key value from spike record 109 present→**0** (physically reclaimed, not just delinked). The - residual `cf_api_token`/`cf_tunnel_token` hits are in `customer_configs.config_json` (the hub's - legitimate config store), not the infra-backup blobs (which held CF tokens base64-encoded inside the - now-zero `controller_config_b64`). -- **Endpoints retired:** `POST /api/v1/infra-backup` and `GET …/versions` return **404**; control - `POST /api/v1/report` still returns **401** (routed, unauth) — confirming the 404s are route removal, - not a blanket failure. -- **Demo email fix:** the next 03:00 deadline run cannot be observed within this session, but the live - demo host-report (fresh verified PBS, 13.9 h) exercises the not-missed path, and demo emits - `db_dump_completed` daily (never missed) — so `expected_backup_missed` will no longer fire. +The recipe is PLAINTEXT-at-rest because it carries NO secrets — only identifiers/intents/sizes/ +coordinates. The PBS key stays in escrow, the access token in identity-escrow, the restic password in +escrow. This is the clean inverse of the retired infra-backup (which shipped `restic_password`/ +`cf_api_token` and was a zero-knowledge violation). The leak-preventing allowlist is enforced at the +controller emitter; the hub adds a defense-in-depth no-secrets sweep. -## Flagged for the operator (out of scope here) +### Tests -- **Rotate the exposed credentials** that were in the dropped blobs (Cloudflare API + tunnel tokens - for `demo-felhom.eu` first; hub/session secrets if shared). They remain valid until rotated; the - purge removes the at-rest copy but not their validity. -- **Separate historical leak:** the legacy `reports` table holds thousands of rows with a plaintext - `restic_password` value from old controller versions. The **live** controller no longer sends it - (removed in controller v0.69.0), but the historical rows persist — a distinct purge/rotation - decision, deliberately not done here. -- **`peti-felhom` is a defunct customer marked `active`** (no host-report or controller report since - Feb 2026). Its daily `expected_backup_missed` stops with this change (no host-report → no backup - alarm), but it should be marked inactive to silence all residual noise. +`TestDRRecipe_StoreRoundTrip`, `TestAssembleDRRecipe_MatchesGolden` (assembled shape pinned in +`testdata/dr-recipe.golden.json`), `TestAssembleDRRecipe_IgnoreUnknownAndVersionSkew`, +`TestAssembleDRRecipe_PartialHalves`, `TestAssembleDRRecipe_NoSecrets`. `go build`/`go vet`/`go test ./...` +green. -## Out of scope (untouched) +### Cross-repo golden discipline -Credential rotation; the Komga healthcheck; the secret-free DR "recipe" (the later DR slice). +The recipe wire spans three repos. The agent's `host-report.golden.json` `dr_recipe` section and the +hub's host-half test literal must stay key-consistent; the controller's emitter and the hub's app-half +test literal likewise. On any wire change, manually checksum-diff the golden across the three repos +(there is no shared types module yet). + +### Deploy + +Built + pushed `felhom-hub:0.13.0`; manifest tag bumped + ArgoCD `felhom` app synced. diff --git a/hub/CHANGELOG.md b/hub/CHANGELOG.md index 55380a2..b85b018 100644 --- a/hub/CHANGELOG.md +++ b/hub/CHANGELOG.md @@ -1,5 +1,35 @@ # Felhom Hub — Changelog +## v0.13.0 — DR recipe: assemble + store + view the secret-free reconstruction recipe (2026-06-16) + +**DR recipe slice (hub half)** — the assemble-store-view side of the secret-free reconstruction recipe +(`documentation/audits/SPIKE-dr-recipe-2026-06-16.md`). The hub receives two additive halves via the +existing report paths — the agent's storage/guest/PBS half (on the host-report) and the controller's +customer/apps half (on the controller report) — and assembles them into one operator-readable recipe per +customer. This is the clean inverse of the retired infra-backup: same "re-provision plan" goal, but +PLAINTEXT-at-rest is *correct* because the recipe has zero secrets. + +- **Store** (`internal/store/dr_recipe.go`): a DEDICATED `dr_recipe` table (NOT `host_escrow`, NOT the + dropped `infra_backup*` tables) keyed by `customer_id`, holding `host_half_json` + `app_half_json` + + `recipe_version` + `host_id`. `SaveDRRecipeHostHalf` / `SaveDRRecipeAppHalf` each upsert their half and + PRESERVE the other (last-write-wins per half). `AssembleDRRecipe` stitches the two into an + `AssembledRecipe{recipe_version, customer, guests, pbs, drives, pve_storage, apps}` — sub-sections pass + through as `json.RawMessage` (verbatim), **ignore-unknown** at the top level and version-skew tolerant + (`recipe_version` = max of the two halves) for forward-compat across the three repos. +- **Ingest** (`internal/api/handler.go`): `handleHostReport` persists the `dr_recipe` host-half (keyed by + the host's customer); `handleReport` persists the `dr_recipe` app-half (keyed by `customer_id`) — + mirroring the `app_telemetry` pattern, backward-compatible (old agents/controllers omit the field), and + never fatal to the heartbeat. +- **View** (`internal/web/dr_recipe.go` + customer page): a DR-recipe panel on the customer detail page + (which half has landed + last-updated) with a **Download recipe (JSON)** link → + `GET /customers/{id}/dr-recipe.json` serves the assembled recipe (operator dashboard-auth, pretty JSON, + `Content-Disposition` attachment). No decrypt, nothing to redact. +- Tests: `TestDRRecipe_StoreRoundTrip` (each half preserves the other), `TestAssembleDRRecipe_MatchesGolden` + (the assembled wire shape pinned in `testdata/dr-recipe.golden.json`), + `TestAssembleDRRecipe_IgnoreUnknownAndVersionSkew` (a forward-compat half still assembles; version = max), + `TestAssembleDRRecipe_PartialHalves` (one half present), `TestAssembleDRRecipe_NoSecrets` (defense-in-depth + credential-key sweep). Pairs with felhom-agent v0.38.0 + felhom-controller v0.73.0. + ## v0.12.0 — retire Infra Backup + purge its plaintext secrets + fix the daily backup-deadline email (2026-06-16) Phase-1 of the Infra Backup retirement (per `documentation/audits/SPIKE-infra-backup-2026-06-15.md`). diff --git a/hub/README.md b/hub/README.md index c82130a..39e4d52 100644 --- a/hub/README.md +++ b/hub/README.md @@ -62,6 +62,10 @@ All API endpoints require `Authorization: Bearer ` (except `/healthz` a The `POST /api/v1/report` handler (v0.4.0+) automatically parses the optional `app_telemetry` JSON array from the request body and stores it in `app_telemetry` / `app_log_issues` tables. Old controllers (no `app_telemetry` key) continue to work unchanged. +### DR Recipe — secret-free reconstruction recipe (hub v0.13.0) + +The hub assembles + stores + serves the **secret-free DR recipe** (`documentation/audits/SPIKE-dr-recipe-2026-06-16.md`) — the non-secret re-provision plan that complements escrow (keys) + PBS/restic (bytes). It arrives as two additive `dr_recipe` halves on the existing report paths: the **agent's** storage/guest/PBS half on `POST /api/v1/host-report` and the **controller's** customer/apps half on `POST /api/v1/report` (both backward-compatible, ignore-unknown). The hub stores them PLAINTEXT in a dedicated `dr_recipe` table keyed by `customer_id` (`SaveDRRecipeHostHalf` / `SaveDRRecipeAppHalf`, each preserving the other half), and `AssembleDRRecipe` stitches them into one operator-readable recipe (`{recipe_version, customer, guests, pbs, drives, pve_storage, apps}`; sub-sections pass through verbatim, version = max of the two halves). An operator views the panel on the customer page and downloads the assembled JSON at `GET /customers/{id}/dr-recipe.json`. **Plaintext-at-rest is correct here** — the recipe carries only identifiers/intents/sizes/coordinates, never a key/password/token (the boundary is enforced at the controller emitter). This is the clean inverse of the retired infra-backup. + ### Infrastructure Backup — RETIRED (Phase-1, 2026-06-16, hub v0.12.0) The Infra Backup mechanism (`POST/GET /api/v1/infra-backup`, the operator panel, the diff --git a/hub/internal/api/handler.go b/hub/internal/api/handler.go index 016dc28..352ece0 100644 --- a/hub/internal/api/handler.go +++ b/hub/internal/api/handler.go @@ -241,6 +241,21 @@ func (h *Handler) handleReport(w http.ResponseWriter, r *http.Request) { } } + // DR recipe — persist the controller's secret-free customer/apps half (preserving any host half). + // Backward-compatible (old controllers won't have this field); a failure must not drop the report. + var drPayload struct { + DRRecipe json.RawMessage `json:"dr_recipe"` + } + if err := json.Unmarshal(body, &drPayload); err == nil && len(drPayload.DRRecipe) > 0 { + var ver drRecipeVersionOnly + _ = json.Unmarshal(drPayload.DRRecipe, &ver) + if err := h.store.SaveDRRecipeAppHalf(payload.CustomerID, ver.RecipeVersion, drPayload.DRRecipe); err != nil { + h.logger.Printf("[WARN] Failed to save DR-recipe app-half for %s: %v", payload.CustomerID, err) + } else { + h.logger.Printf("[INFO] DR-recipe app-half stored for customer %s (v%d)", payload.CustomerID, ver.RecipeVersion) + } + } + h.logger.Printf("[INFO] Received report from %s (%d bytes)", payload.CustomerID, len(body)) // Build response with optional customer_blocked flag @@ -297,6 +312,14 @@ type hostReportPayload struct { Cloudflared struct { Status string `json:"status"` } `json:"cloudflared"` + // DR recipe — the agent's storage/guest/PBS half (secret-free). RawMessage = stored verbatim, + // ignore-unknown (forward-compat). Persisted to dr_recipe, assembled with the controller half. + DRRecipe json.RawMessage `json:"dr_recipe"` +} + +// drRecipeVersionOnly extracts just recipe_version from a half's JSON (ignore-unknown). 0 if absent. +type drRecipeVersionOnly struct { + RecipeVersion int `json:"recipe_version"` } // hostPBSSnapshot mirrors the agent's hub.PBSSnapshot wire contract (slice 6 Phase B). The @@ -525,6 +548,18 @@ func (h *Handler) handleHostReport(w http.ResponseWriter, r *http.Request) { h.logger.Printf("[INFO] host-report from %s (%d guests, %d storage targets, %d backups, %d restore-tests, %d pbs-snapshots, %d bytes)", hostID, len(rep.Guests), len(rep.StorageTargets), len(rep.Backups), len(rep.RestoreTests), len(rep.PBSSnapshots), len(body)) + // DR recipe — persist the agent's secret-free storage/guest/PBS half (preserving any app half). + // A failure here must NOT drop the heartbeat (the report already saved); just warn. + if len(rep.DRRecipe) > 0 && custID != "" { + var ver drRecipeVersionOnly + _ = json.Unmarshal(rep.DRRecipe, &ver) + if err := h.store.SaveDRRecipeHostHalf(custID, hostID, ver.RecipeVersion, rep.DRRecipe); err != nil { + h.logger.Printf("[WARN] Failed to save DR-recipe host-half for customer %s (host %s): %v", custID, hostID, err) + } else { + h.logger.Printf("[INFO] DR-recipe host-half stored for customer %s (host %s, v%d)", custID, hostID, ver.RecipeVersion) + } + } + blocked := false if cc, err := h.store.GetCustomerConfig(custID); err == nil && cc != nil && cc.Status == "blocked" { blocked = true diff --git a/hub/internal/store/dr_recipe.go b/hub/internal/store/dr_recipe.go new file mode 100644 index 0000000..ae5b968 --- /dev/null +++ b/hub/internal/store/dr_recipe.go @@ -0,0 +1,130 @@ +package store + +import ( + "database/sql" + "encoding/json" +) + +// DR recipe (SPIKE-dr-recipe-2026-06-16) — the secret-free reconstruction recipe, stored PLAINTEXT +// because it has NO secrets (it is the clean inverse of the retired infra-backup). The hub receives two +// halves via the existing report paths — the agent's storage/guest/PBS half (on the host-report) and +// the controller's customer/apps half (on the controller report) — and assembles them into one record +// keyed by customer. Both halves carry ONLY identifiers/intents/sizes/coordinates; the table is a +// DEDICATED `dr_recipe`, NOT host_escrow (opaque) and NOT the dropped infra_backup* tables. + +// DRRecipe is the stored two-half record for one customer. +type DRRecipe struct { + CustomerID string + RecipeVersion int + HostID string + HostHalfJSON string // the agent half (guests/pbs/drives/pve_storage); "" until a host-report lands + AppHalfJSON string // the controller half (customer/apps); "" until a controller report lands + UpdatedAt string +} + +// SaveDRRecipeHostHalf upserts the agent (storage/guest/PBS) half for a customer, preserving any +// app half already stored. Last-write-wins on the host half + host_id + recipe_version. +func (s *Store) SaveDRRecipeHostHalf(customerID, hostID string, recipeVersion int, hostHalf []byte) error { + _, err := s.db.Exec(` + INSERT INTO dr_recipe (customer_id, recipe_version, host_id, host_half_json, app_half_json, updated_at) + VALUES (?, ?, ?, ?, '', datetime('now')) + ON CONFLICT(customer_id) DO UPDATE SET + recipe_version = excluded.recipe_version, + host_id = excluded.host_id, + host_half_json = excluded.host_half_json, + updated_at = datetime('now')`, + customerID, recipeVersion, hostID, string(hostHalf), + ) + return err +} + +// SaveDRRecipeAppHalf upserts the controller (customer/apps) half for a customer, preserving any +// host half already stored. Last-write-wins on the app half + recipe_version. +func (s *Store) SaveDRRecipeAppHalf(customerID string, recipeVersion int, appHalf []byte) error { + _, err := s.db.Exec(` + INSERT INTO dr_recipe (customer_id, recipe_version, host_id, host_half_json, app_half_json, updated_at) + VALUES (?, ?, '', '', ?, datetime('now')) + ON CONFLICT(customer_id) DO UPDATE SET + recipe_version = excluded.recipe_version, + app_half_json = excluded.app_half_json, + updated_at = datetime('now')`, + customerID, recipeVersion, string(appHalf), + ) + return err +} + +// GetDRRecipe returns the stored two-half record for a customer (nil if none). +func (s *Store) GetDRRecipe(customerID string) (*DRRecipe, error) { + var r DRRecipe + err := s.db.QueryRow(` + SELECT customer_id, recipe_version, host_id, host_half_json, app_half_json, updated_at + FROM dr_recipe WHERE customer_id = ?`, customerID). + Scan(&r.CustomerID, &r.RecipeVersion, &r.HostID, &r.HostHalfJSON, &r.AppHalfJSON, &r.UpdatedAt) + if err == sql.ErrNoRows { + return nil, nil + } + if err != nil { + return nil, err + } + return &r, nil +} + +// AssembledRecipe is the operator-facing single recipe: the two halves stitched together. The +// sub-sections are passed through as json.RawMessage so the assembly is robust to forward-compat +// additions inside either half (ignore-unknown at the top level, verbatim passthrough below). +type AssembledRecipe struct { + RecipeVersion int `json:"recipe_version"` + Customer json.RawMessage `json:"customer,omitempty"` + Guests json.RawMessage `json:"guests,omitempty"` + PBS json.RawMessage `json:"pbs,omitempty"` + Drives json.RawMessage `json:"drives,omitempty"` + PVEStorage json.RawMessage `json:"pve_storage,omitempty"` + Apps json.RawMessage `json:"apps,omitempty"` +} + +// hostHalfShape / appHalfShape capture only the top-level keys the assembly stitches; encoding/json +// drops any unknown top-level key (forward-compat — a newer half with extra sections still parses). +type hostHalfShape struct { + RecipeVersion int `json:"recipe_version"` + Guests json.RawMessage `json:"guests"` + PBS json.RawMessage `json:"pbs"` + Drives json.RawMessage `json:"drives"` + PVEStorage json.RawMessage `json:"pve_storage"` +} +type appHalfShape struct { + RecipeVersion int `json:"recipe_version"` + Customer json.RawMessage `json:"customer"` + Apps json.RawMessage `json:"apps"` +} + +// AssembleDRRecipe stitches the two stored halves into one operator-facing recipe. Either half may be +// empty (not yet reported); the assembly fills what it has. recipe_version = the max of the two halves' +// versions (1 if both absent). Ignore-unknown: extra top-level keys in either half are dropped, nested +// shapes pass through verbatim — so the three repos can evolve the recipe without silent drift here. +func AssembleDRRecipe(rec *DRRecipe) (AssembledRecipe, error) { + out := AssembledRecipe{RecipeVersion: 1} + if rec == nil { + return out, nil + } + if rec.HostHalfJSON != "" { + var h hostHalfShape + if err := json.Unmarshal([]byte(rec.HostHalfJSON), &h); err != nil { + return out, err + } + out.Guests, out.PBS, out.Drives, out.PVEStorage = h.Guests, h.PBS, h.Drives, h.PVEStorage + if h.RecipeVersion > out.RecipeVersion { + out.RecipeVersion = h.RecipeVersion + } + } + if rec.AppHalfJSON != "" { + var a appHalfShape + if err := json.Unmarshal([]byte(rec.AppHalfJSON), &a); err != nil { + return out, err + } + out.Customer, out.Apps = a.Customer, a.Apps + if a.RecipeVersion > out.RecipeVersion { + out.RecipeVersion = a.RecipeVersion + } + } + return out, nil +} diff --git a/hub/internal/store/dr_recipe_test.go b/hub/internal/store/dr_recipe_test.go new file mode 100644 index 0000000..7048e1e --- /dev/null +++ b/hub/internal/store/dr_recipe_test.go @@ -0,0 +1,182 @@ +package store + +import ( + "encoding/json" + "os" + "reflect" + "regexp" + "sort" + "strings" + "testing" +) + +// The two halves as the agent (host) and controller (app) emit them — keys must match the cross-repo +// golden (the agent's host-report.golden.json dr_recipe section + the controller's emitter). +const drHostHalf = `{ + "recipe_version": 1, + "guests": [ { "vmid": 9201, "cores": 4, "memory_bytes": 12884901888, "disk_bytes": 34359738368 } ], + "pbs": { "repo_id": "felhom-pbs", "namespace": "root", "latest_snapshot_id": "9201" }, + "drives": [ { "durable_id": "uuid:da9e7089-cf8e-4617-adcb-a377743fae00", "role": "bulk-data", "mount_path": "/mnt/felhom-usb", "intent": "enrolled", "total_bytes": 1000000000000 } ], + "pve_storage": [ { "name": "local-lvm", "type": "lvmthin", "content": "rootdir,images" }, { "name": "felhom-usb", "type": "usb", "content": "backup" } ] +}` + +const drAppHalf = `{ + "recipe_version": 1, + "customer": { "id": "cust-demo", "display": "Demo Customer", "domain": "demo-felhom.eu" }, + "apps": [ { "catalog_ref": "romm", "enabled": true, "storage_bindings": [ { "container_path": "/roms", "drive": "felhom-flash", "subpath": "userdata/roms" } ] } ] +}` + +// TestDRRecipe_StoreRoundTrip: each half upserts independently and preserves the other; GetDRRecipe +// returns both. +func TestDRRecipe_StoreRoundTrip(t *testing.T) { + s := newTestStore(t) + + // App half lands first. + if err := s.SaveDRRecipeAppHalf("cust-demo", 1, []byte(drAppHalf)); err != nil { + t.Fatal(err) + } + rec, _ := s.GetDRRecipe("cust-demo") + if rec == nil || rec.AppHalfJSON == "" || rec.HostHalfJSON != "" { + t.Fatalf("after app-half: want app set, host empty, got %+v", rec) + } + + // Host half lands later — must NOT clobber the app half. + if err := s.SaveDRRecipeHostHalf("cust-demo", "host-01", 1, []byte(drHostHalf)); err != nil { + t.Fatal(err) + } + rec, _ = s.GetDRRecipe("cust-demo") + if rec == nil || rec.AppHalfJSON == "" || rec.HostHalfJSON == "" || rec.HostID != "host-01" { + t.Fatalf("after host-half: both halves must be present + host_id set, got %+v", rec) + } + + // A re-report of the host half preserves the app half (and vice-versa). + if err := s.SaveDRRecipeHostHalf("cust-demo", "host-01", 1, []byte(drHostHalf)); err != nil { + t.Fatal(err) + } + rec, _ = s.GetDRRecipe("cust-demo") + if rec.AppHalfJSON == "" { + t.Fatal("re-reporting the host half clobbered the app half") + } + + // Absent customer → nil, no error. + if got, err := s.GetDRRecipe("nobody"); err != nil || got != nil { + t.Fatalf("absent customer should be (nil,nil), got (%v,%v)", got, err) + } +} + +// TestAssembleDRRecipe_MatchesGolden: assembling both halves yields the golden's key shape (the +// cross-repo wire pin) and the correct stitched values. +func TestAssembleDRRecipe_MatchesGolden(t *testing.T) { + rec := &DRRecipe{CustomerID: "cust-demo", RecipeVersion: 1, HostHalfJSON: drHostHalf, AppHalfJSON: drAppHalf} + asm, err := AssembleDRRecipe(rec) + if err != nil { + t.Fatal(err) + } + b, _ := json.Marshal(asm) + var got map[string]any + json.Unmarshal(b, &got) + + raw, err := os.ReadFile("testdata/dr-recipe.golden.json") + if err != nil { + t.Fatal(err) + } + var golden map[string]any + if err := json.Unmarshal(raw, &golden); err != nil { + t.Fatalf("golden invalid: %v", err) + } + + // Top-level key set must match the golden (the assembled wire shape). + if ga, gb := keysOf(golden), keysOf(got); !reflect.DeepEqual(ga, gb) { + t.Errorf("assembled key drift:\n golden=%v\n got =%v", ga, gb) + } + // And the stitched values: customer from the app half, drives/pbs from the host half. + if asm.RecipeVersion != 1 { + t.Errorf("recipe_version=%d want 1", asm.RecipeVersion) + } + if !jsonContains(t, asm.Customer, "cust-demo") || !jsonContains(t, asm.Customer, "demo-felhom.eu") { + t.Errorf("customer not stitched from app half: %s", asm.Customer) + } + if !jsonContains(t, asm.Drives, "uuid:da9e7089-cf8e-4617-adcb-a377743fae00") { + t.Errorf("drives not stitched from host half: %s", asm.Drives) + } + if !jsonContains(t, asm.Apps, "romm") { + t.Errorf("apps not stitched from app half: %s", asm.Apps) + } +} + +// TestAssembleDRRecipe_IgnoreUnknownAndVersionSkew: a half carrying an UNKNOWN top-level field and a +// HIGHER recipe_version still assembles (forward-compat), and recipe_version reflects the max. +func TestAssembleDRRecipe_IgnoreUnknownAndVersionSkew(t *testing.T) { + futureHost := `{ "recipe_version": 2, "drives": [], "pve_storage": [], "guests": [], + "future_section": { "whatever": 1 }, "network_topology": ["a","b"] }` + rec := &DRRecipe{CustomerID: "c", RecipeVersion: 2, HostHalfJSON: futureHost, AppHalfJSON: drAppHalf} + asm, err := AssembleDRRecipe(rec) + if err != nil { + t.Fatalf("ignore-unknown failed to parse a forward-compat half: %v", err) + } + if asm.RecipeVersion != 2 { + t.Errorf("recipe_version=%d, want max(2,1)=2", asm.RecipeVersion) + } + // The unknown sections are dropped (not in AssembledRecipe), but the assembly did not error. + b, _ := json.Marshal(asm) + if string(b) == "" { + t.Fatal("empty assembly") + } +} + +// TestAssembleDRRecipe_PartialHalves: only one half present → assemble what we have, no error. +func TestAssembleDRRecipe_PartialHalves(t *testing.T) { + onlyApp, err := AssembleDRRecipe(&DRRecipe{AppHalfJSON: drAppHalf}) + if err != nil || onlyApp.Apps == nil || onlyApp.Drives != nil { + t.Errorf("only-app assembly wrong: %+v err=%v", onlyApp, err) + } + onlyHost, err := AssembleDRRecipe(&DRRecipe{HostHalfJSON: drHostHalf}) + if err != nil || onlyHost.Drives == nil || onlyHost.Customer != nil { + t.Errorf("only-host assembly wrong: %+v err=%v", onlyHost, err) + } + empty, err := AssembleDRRecipe(nil) + if err != nil || empty.RecipeVersion != 1 { + t.Errorf("nil assembly should be a v1 empty recipe, got %+v err=%v", empty, err) + } +} + +// TestAssembleDRRecipe_NoSecrets: defense-in-depth — the assembled output carries no credential-shaped +// key. (The load-bearing boundary is enforced at the controller emitter; this guards the hub side.) +func TestAssembleDRRecipe_NoSecrets(t *testing.T) { + asm, _ := AssembleDRRecipe(&DRRecipe{HostHalfJSON: drHostHalf, AppHalfJSON: drAppHalf}) + b, _ := json.Marshal(asm) + re := regexp.MustCompile(`(?i)(password|secret|token|hash|passphrase|api[_-]?key|\bkey\b|enc:)`) + var v any + json.Unmarshal(b, &v) + var walk func(any) + walk = func(n any) { + switch x := n.(type) { + case map[string]any: + for k, c := range x { + if re.MatchString(k) { + t.Errorf("secret-shaped key %q in assembled recipe", k) + } + walk(c) + } + case []any: + for _, c := range x { + walk(c) + } + } + } + walk(v) +} + +func jsonContains(t *testing.T, raw json.RawMessage, substr string) bool { + t.Helper() + return len(raw) > 0 && string(raw) != "null" && strings.Contains(string(raw), substr) +} + +func keysOf(m map[string]any) []string { + ks := make([]string, 0, len(m)) + for k := range m { + ks = append(ks, k) + } + sort.Strings(ks) + return ks +} diff --git a/hub/internal/store/store.go b/hub/internal/store/store.go index b7574c1..dd519cc 100644 --- a/hub/internal/store/store.go +++ b/hub/internal/store/store.go @@ -309,6 +309,26 @@ func (s *Store) migrate() error { s.db.Exec(`ALTER TABLE host_escrow ADD COLUMN identity_blob BLOB`) s.db.Exec(`ALTER TABLE host_escrow ADD COLUMN directive_json TEXT NOT NULL DEFAULT '{}'`) + // dr_recipe (SPIKE-dr-recipe-2026-06-16): the secret-free DR reconstruction recipe, stored + // PLAINTEXT (it has NO secrets — the clean inverse of the retired infra_backup). Two halves keyed + // by customer: the agent's storage/guest/PBS half (host_half_json, from the host-report) and the + // controller's customer/apps half (app_half_json, from the controller report); the hub assembles + // them on read. DEDICATED table, separate from the opaque host_escrow. One row per customer; + // each half is last-write-wins and preserves the other. + _, err = s.db.Exec(` + CREATE TABLE IF NOT EXISTS dr_recipe ( + customer_id TEXT PRIMARY KEY, + recipe_version INTEGER NOT NULL DEFAULT 1, + host_id TEXT NOT NULL DEFAULT '', + host_half_json TEXT NOT NULL DEFAULT '', + app_half_json TEXT NOT NULL DEFAULT '', + updated_at DATETIME NOT NULL DEFAULT (datetime('now')) + ); + `) + if err != nil { + return err + } + return nil } diff --git a/hub/internal/store/testdata/dr-recipe.golden.json b/hub/internal/store/testdata/dr-recipe.golden.json new file mode 100644 index 0000000..e2b4346 --- /dev/null +++ b/hub/internal/store/testdata/dr-recipe.golden.json @@ -0,0 +1,30 @@ +{ + "recipe_version": 1, + "customer": { "id": "cust-demo", "display": "Demo Customer", "domain": "demo-felhom.eu" }, + "guests": [ + { "vmid": 9201, "cores": 4, "memory_bytes": 12884901888, "disk_bytes": 34359738368 } + ], + "pbs": { "repo_id": "felhom-pbs", "namespace": "root", "latest_snapshot_id": "9201" }, + "drives": [ + { + "durable_id": "uuid:da9e7089-cf8e-4617-adcb-a377743fae00", + "role": "bulk-data", + "mount_path": "/mnt/felhom-usb", + "intent": "enrolled", + "total_bytes": 1000000000000 + } + ], + "pve_storage": [ + { "name": "local-lvm", "type": "lvmthin", "content": "rootdir,images" }, + { "name": "felhom-usb", "type": "usb", "content": "backup" } + ], + "apps": [ + { + "catalog_ref": "romm", + "enabled": true, + "storage_bindings": [ + { "container_path": "/roms", "drive": "felhom-flash", "subpath": "userdata/roms" } + ] + } + ] +} diff --git a/hub/internal/web/configs.go b/hub/internal/web/configs.go index 4ad5b93..aca12e1 100644 --- a/hub/internal/web/configs.go +++ b/hub/internal/web/configs.go @@ -254,12 +254,28 @@ func (s *Server) handleCustomerUnified(w http.ResponseWriter, r *http.Request, c AppTelemetry []store.CustomerAppSummary HasAppTelemetry bool + HasDRRecipe bool + DRRecipeUpdatedAt string + DRRecipeHasHost bool + DRRecipeHasApps bool + Flash string ActiveNav string CSRFField template.HTML CSRFToken string } + // DR recipe presence — show the secret-free reconstruction recipe panel + download link when + // either half has landed (host-report and/or controller report). + var hasDR, drHost, drApps bool + var drUpdated string + if rec, err := s.store.GetDRRecipe(customerID); err == nil && rec != nil { + hasDR = rec.HostHalfJSON != "" || rec.AppHalfJSON != "" + drHost = rec.HostHalfJSON != "" + drApps = rec.AppHalfJSON != "" + drUpdated = rec.UpdatedAt + } + data := pageData{ CustomerID: customerID, CustomerName: name, @@ -293,6 +309,11 @@ func (s *Server) handleCustomerUnified(w http.ResponseWriter, r *http.Request, c AppTelemetry: appTelemetry, HasAppTelemetry: len(appTelemetry) > 0, + HasDRRecipe: hasDR, + DRRecipeUpdatedAt: drUpdated, + DRRecipeHasHost: drHost, + DRRecipeHasApps: drApps, + Flash: r.URL.Query().Get("flash"), ActiveNav: "configs", CSRFField: s.csrfField(r), @@ -741,10 +762,10 @@ func flattenYAML(m map[string]interface{}, prefix string) map[string]string { // configDiff represents a single key-value difference between two configs. type configDiff struct { - Key string `json:"key"` - HubValue string `json:"hub"` - CtrlValue string `json:"controller"` - Status string `json:"status"` // "changed", "hub_only", "controller_only" + Key string `json:"key"` + HubValue string `json:"hub"` + CtrlValue string `json:"controller"` + Status string `json:"status"` // "changed", "hub_only", "controller_only" } // compareYAMLValues parses two YAML strings and returns their value differences. diff --git a/hub/internal/web/dr_recipe.go b/hub/internal/web/dr_recipe.go new file mode 100644 index 0000000..57c9bfd --- /dev/null +++ b/hub/internal/web/dr_recipe.go @@ -0,0 +1,61 @@ +package web + +import ( + "encoding/json" + "net/http" + + "gitea.dooplex.hu/admin/felhom-hub/internal/store" +) + +// handleDRRecipeDownload serves the assembled secret-free DR recipe for a customer as a JSON download +// (SPIKE-dr-recipe-2026-06-16). The recipe is PLAINTEXT because it carries NO secrets — only the +// reconstruction scaffolding (guest sizing, drive inventory, PVE storage, PBS coordinates, app +// inventory + storage bindings). Operator (dashboard-auth) only; no decrypt, nothing to redact. +func (s *Server) handleDRRecipeDownload(w http.ResponseWriter, r *http.Request, customerID string) { + if r.Method != http.MethodGet { + http.Error(w, "Method not allowed", http.StatusMethodNotAllowed) + return + } + rec, err := s.store.GetDRRecipe(customerID) + if err != nil { + s.logger.Printf("[ERROR] DR-recipe lookup failed for %s: %v", customerID, err) + http.Error(w, "Internal error", http.StatusInternalServerError) + return + } + if rec == nil { + http.Error(w, "No DR recipe for this customer yet (awaiting a host-report + controller report)", http.StatusNotFound) + return + } + assembled, err := store.AssembleDRRecipe(rec) + if err != nil { + s.logger.Printf("[ERROR] DR-recipe assemble failed for %s: %v", customerID, err) + http.Error(w, "Internal error", http.StatusInternalServerError) + return + } + out, err := json.MarshalIndent(assembled, "", " ") + if err != nil { + http.Error(w, "Internal error", http.StatusInternalServerError) + return + } + w.Header().Set("Content-Type", "application/json; charset=utf-8") + w.Header().Set("Content-Disposition", "attachment; filename=\"dr-recipe-"+safeFilename(customerID)+".json\"") + w.Write(out) + s.logger.Printf("[INFO] DR-recipe downloaded for customer %s (v%d)", customerID, assembled.RecipeVersion) +} + +// safeFilename keeps a customer id safe for a Content-Disposition filename (alnum/-/_ only). +func safeFilename(s string) string { + out := make([]rune, 0, len(s)) + for _, r := range s { + switch { + case r >= 'a' && r <= 'z', r >= 'A' && r <= 'Z', r >= '0' && r <= '9', r == '-', r == '_': + out = append(out, r) + default: + out = append(out, '_') + } + } + if len(out) == 0 { + return "customer" + } + return string(out) +} diff --git a/hub/internal/web/server.go b/hub/internal/web/server.go index 3675f53..1ae865f 100644 --- a/hub/internal/web/server.go +++ b/hub/internal/web/server.go @@ -238,6 +238,10 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { } else { http.Error(w, "Method not allowed", http.StatusMethodNotAllowed) } + case strings.HasPrefix(path, "/customers/") && strings.HasSuffix(path, "/dr-recipe.json"): + customerID := strings.TrimPrefix(path, "/customers/") + customerID = strings.TrimSuffix(customerID, "/dr-recipe.json") + s.handleDRRecipeDownload(w, r, customerID) case strings.HasPrefix(path, "/customers/"): customerID := strings.TrimPrefix(path, "/customers/") s.handleCustomerUnified(w, r, customerID) diff --git a/hub/internal/web/templates/customer_unified.html b/hub/internal/web/templates/customer_unified.html index 6d09a2a..db10ed6 100644 --- a/hub/internal/web/templates/customer_unified.html +++ b/hub/internal/web/templates/customer_unified.html @@ -559,6 +559,36 @@ {{end}} + {{if .HasDRRecipe}} + +
+

DR Recipe (secret-free reconstruction plan)

+

+ The non-secret re-provision plan — guest sizing, drive inventory (durable-id → role → mount → intent), + PVE storage defs, PBS coordinates, and app inventory + storage bindings. It complements escrow (keys) + and PBS/restic (bytes): it contains no key, password, or token. Use it to rebuild the + host/guest/storage scaffolding before the PBS bytes land. +

+
+
+ Storage / guest / PBS half (agent) + {{if .DRRecipeHasHost}}present{{else}}awaiting host-report{{end}} +
+
+ Customer / apps half (controller) + {{if .DRRecipeHasApps}}present{{else}}awaiting controller report{{end}} +
+
+ Last updated + {{if .DRRecipeUpdatedAt}}{{.DRRecipeUpdatedAt}}{{else}}—{{end}} +
+
+
+ Download recipe (JSON) +
+
+ {{end}} +

Notifications

diff --git a/manifests/hub.yaml b/manifests/hub.yaml index b213a82..e56eeed 100644 --- a/manifests/hub.yaml +++ b/manifests/hub.yaml @@ -117,7 +117,7 @@ spec: spec: containers: - name: hub - image: gitea.dooplex.hu/admin/felhom-hub:v0.12.0 + image: gitea.dooplex.hu/admin/felhom-hub:v0.13.0 ports: - containerPort: 8080 name: http