diff --git a/documentation/runbooks/RUNBOOK-escrow-ceremony.md b/documentation/runbooks/RUNBOOK-escrow-ceremony.md index 74dd09b..43cefb2 100644 --- a/documentation/runbooks/RUNBOOK-escrow-ceremony.md +++ b/documentation/runbooks/RUNBOOK-escrow-ceremony.md @@ -6,6 +6,9 @@ customer-held recovery code R. The hub stores only the opaque blob + a non-reversible password hash; neither Felhom nor a hub compromise can open it. Cross-validated live: fork-4 validation (blob → recover → decrypted the real repo) and the offsite arc (auto-confirm on hash match). + v0.127.0/v0.88.0 (TASK 2026-07-13): the PRIMARY path is the controller wizard — the customer runs + the ceremony themselves; the CLI remains the operator fallback. Mechanics validated by + audits/SPIKE-controller-escrow-2026-07-13.md (all five GO). --> ## When a ceremony runs @@ -15,45 +18,82 @@ `pending` until the hub-verified auto-confirm. 3. **Any new unrecoverable secret / suspected R exposure**: re-ceremony (R cannot be retro-fitted — it is never retained). +4. **Stale-blob warning** (v0.127.0, Scenario F): the controller re-checks the hub blob's password + hash on every report ACK even in the `escrowed` state. If the CURRENT blob does not cover the + current offsite password (hash mismatch, or a hash-less superseding blob — e.g. a CLI ceremony + run without the staged secret), the Távoli mentés card shows "A letétben lévő helyreállítási + csomag nem fedi a jelenlegi távoli mentési jelszót" and offers a re-ceremony. Runs are NOT + blocked (offsite copies still protect against non-total loss); the fix is one wizard run. -## Prerequisites (check BEFORE scheduling with the customer) -- **Agent version:** ≥ v0.79.0 (the ceremony records `restic_pw_sha256` — older agents produce a blob - auto-confirm can never match). -- **⚠ No-PBS hosts: the ceremony CANNOT run — and that is now POLICY, not a gap.** The previously - documented "identity-only mode (≥ v0.80.0)" was NEVER implemented — v0.80.0's actual feature was - seeding `escrow.pbs_storage_id` on PBS hosts. `escrow-create` hard-requires a PBS storage id + - its key file (drill-proven 2026-07-12, finding F-6 of DRILL-day0-vm-2026-07-12.md). - **Operator decision 2026-07-12 (DR-tier-by-default, decision 4): identity-only escrow is - formally PARKED — F-6 is closed by policy, not by implementation.** Every install ships the - full DR plumbing (host-install ≥ v1.15.0); activation is the hub's per-customer DR-tier flag - (default ON), and offsite backup REQUIRES that flag (hub v0.51.0 enforces the coupling: - "Offsite backup requires the DR tier"). A box therefore always reaches the ceremony THROUGH - the tier: WG peer → PBS-DR descriptor applied → ceremony. Revisit identity-only ONLY if a - customer class demands lean-tier offsite — this line is here so a future session knows the - absence is a decision, not an oversight. -- **Host packages:** `age` must be installed (identity wrap dependency; installed by host-install - since v1.15.0 — drill finding F-10 closed. Pre-v1.15.0 boxes: `apt-get install -y age`). -- **K gate (PBS hosts):** `escrow.pbs_storage_id` set and the key file present - (`cfg.Backup.PBSEncKeyPath()`). -- **Staged secret (offsite):** offsite enabled → `EscrowState="pending"` on the controller and the staged - password file present on the agent (`escrow-stage/restic_repo_password` — pushed automatically at - enable; re-saving the offsite config re-pushes it if missing). -- The customer is present/reachable: R is displayed ONCE and must land in their hands. +## PRIMARY PATH — the controller wizard (customer-driveable, agent ≥ v0.88.0, controller ≥ v0.127.0) + +The customer (or the operator in the customer's session) opens **Távoli mentés → +"Helyreállítási kód létrehozása"** (`/backup/escrow`) and clicks through: + +1. **Preflight** — the wizard checks: PBS storage id configured, DR tier applied, `age` present, + hub upload target configured, the FELHOM_ESCROW sudo grant listed (list-mode probe), staged + secret (informational). Red rows block the start. +2. **Warnings** — the R-is-final copy; a re-ceremony additionally shows the supersede warning + ("a korábbi kód érvényét veszti…"). +3. **Re-auth** — the dashboard password, typed again (rides the login rate-limiter). +4. **Run** — the controller RE-STAGES the current offsite password first (when offsite is + configured; a staging failure ABORTS the start — a ceremony without the staged secret would + mint a hash-less blob), then triggers the agent's local-API job. The agent runs + `sudo -n felhom-agent --config /etc/felhom-agent/agent.json --selftest=escrow-create --upload + --output=json` — ONE fixed argv, byte-matched by sudoers (spike §2.2). +5. **Reveal** — R is delivered by a ONE-SHOT claim: displayed exactly once in the browser, + `Cache-Control: no-store`, never rendered server-side, never logged, wiped agent-side the + moment it is claimed. Unclaimed R expires after **10 minutes** → the ceremony is VOID and a + re-run supersedes it (nothing is lost — the blob is already on the hub; the old R simply never + existed for it). +6. **Typed-back** — the customer re-types two random words of R (client-side check), ticks the + "felírtam" confirmation, finishes. + +Then the existing **auto-confirm** flips `pending → escrowed` hands-free within one report cycle +(hash match in the ACK) and wipes the staged copy — same as ever. + +**Supersede/void semantics:** every new ceremony's blob REPLACES the hub blob; the previous R keeps +opening only the previous blob (pre-existing offsite history). An unclaimed/void R requires nothing +but a re-run. An agent restart mid-ceremony or pre-claim loses R safely (in-memory by design) — the +wizard reports it honestly; re-run. + +### F1 threat model (operator ruling 2026-07-13 — carried verbatim into the product) + +R is displayed in the customer's browser, which means it transits the Cloudflare tunnel ONCE at +the reveal moment (TLS end-to-end, `no-store`, one-shot). **This is an accepted, documented risk** +— the same trust class as the claim code and the login password, which already ride the same +channel: an adversary positioned to read this leg already owns the dashboard session and the box. +The agent→controller leg (local API, pinned TLS) never leaves the box. A LAN-direct delivery +variant (à la plex.direct) is PARKED, not forgotten — revisit only if the trust class of the +tunnel changes. + +## OPERATOR FALLBACK — the CLI (text mode, unchanged) -## The ceremony (ONE command, on the host, as the agent's operator context) ```bash felhom-agent --selftest=escrow-create --upload ``` -- `--storage ` only if `escrow.pbs_storage_id` isn't configured (the PBS DR - apply-bridge seeds it automatically). No-PBS hosts: see the prerequisite warning above — the - ceremony refuses without a PBS key; identity-only mode does not exist yet. -- What it does, in order: generates a fresh **R** (EFF-wordlist passphrase; entropy printed) → seals K - (if present, via proxmox-backup-client re-key) and the IdentityBundle (age-under-R; the staged restic - password auto-injected; the live WG key auto-captured if present) → **self-verifies by recovering its - own blob** ("an escrow you haven't recovered isn't an escrow") → uploads the opaque blob + - `restic_pw_sha256` to the hub → **wipes the staged secret**. -- Output hygiene: the ONLY secret ever displayed is R. Nothing else in the output is sensitive; the - command is safe to run over a screen-share EXCEPT the R moment. +- On the host, over SSH, as before; output byte-identical to the historical text mode. R is + printed once inside the write-it-down banner. +- `--storage ` only if `escrow.pbs_storage_id` isn't configured. +- **⚠ Staged-secret rule:** when offsite is configured, ensure the staged secret is present + (re-save the offsite config to re-push it) BEFORE a CLI run — a CLI ceremony without it uploads + a hash-less blob, which trips the Scenario-F stale warning and can never auto-confirm. +- `--output=json` exists for the wizard's job runner (machine mode: ONE JSON object with R on + stdout, humans to stderr). Never run json mode by hand into a terminal you are screen-sharing. + +## Prerequisites (the wizard preflight checks these for you) +- **Agent ≥ v0.88.0** (wizard endpoints + `--output=json`); ≥ v0.79.0 for `restic_pw_sha256`. +- **⚠ No-PBS hosts: the ceremony CANNOT run — POLICY, not a gap.** Identity-only escrow is + formally PARKED (operator decision 2026-07-12, DR-tier-by-default decision 4); every install + ships the full DR plumbing (host-install ≥ v1.15.0), activation is the hub's per-customer + DR-tier flag (default ON), and offsite backup REQUIRES that flag. A box always reaches the + ceremony THROUGH the tier: WG peer → PBS-DR descriptor applied → ceremony. +- **Host packages:** `age` (installed by host-install since v1.15.0). +- **K gate (PBS hosts):** `escrow.pbs_storage_id` set and the key file present. +- **Sudo grant:** the `FELHOM_ESCROW` alias in `/etc/sudoers.d/felhom-agent` (ships with + host-install ≥ v1.16.0 via the canonical sudoers fetch; on older installs re-ship the sudoers + with the agent deploy). +- The customer is present: R is displayed ONCE and must land in their hands. ## R handling — the rules to say out loud to the customer 1. R is displayed **once**; the system keeps no copy anywhere. Write it down / password manager / @@ -69,8 +109,9 @@ Within one report cycle the hub serves the blob's presence + hash in the ACK; th `sha256(local repo password)` and flips `pending → escrowed` on its own, wiping the agent-staged copy. **Verify remotely:** the customer page / report shows `escrow_state: "escrowed"` with zero manual clicks; the next scheduled offsite run proceeds and snapshots appear. Hash-mismatch warning instead → the blob -does not cover the CURRENT password (stale blob) → re-run the ceremony; do NOT manual-confirm around it. -(The manual confirm endpoint exists solely as a deprecated fallback for pre-hash legacy blobs.) +does not cover the CURRENT password (stale blob) → re-run the ceremony (wizard); do NOT manual-confirm +around it. (The manual confirm ENDPOINT exists solely as a deprecated fallback for pre-hash legacy +blobs; its button is gone from the card since v0.127.0.) ## Recovery (the other half — operator-assisted, sketch) Total loss → the customer's R + the hub blob: @@ -82,6 +123,10 @@ secrets writer) → restore through the real endpoints. Live-proven end-to-end 2 password opened the production repo). ## Do NOT -- Run a ceremony unattended or pipe its output to a log (R would land in it). +- Pipe the CLI ceremony's output to a log or run it unattended (R would land in it). The wizard's + machine mode is exempt BY DESIGN: its pipe terminates in the agent's in-memory one-shot holder, + nowhere else. - Manual-confirm a hash-mismatch; skip the self-verify; retain or transmit R on Felhom systems. - Treat the SFTP credential as escrow-worthy (it's regenerable access; only the ENCRYPTION secrets ride R). +- Run a CLI ceremony on an offsite-configured box without the staged secret (hash-less blob → + stale warning; the wizard's re-stage-first makes this impossible from the UI). diff --git a/hub/CHANGELOG.md b/hub/CHANGELOG.md index 38656c4..8902a0c 100644 --- a/hub/CHANGELOG.md +++ b/hub/CHANGELOG.md @@ -1,5 +1,11 @@ # Felhom Hub — Changelog +## (unreleased) hostInstallVersion 1.16.0 (2026-07-13) + +Display-const bump only, keeping `scripts/hostinstall_gates.py` green with the installer's +v1.16.0 (FELHOM_ESCROW via the canonical sudoers fetch — see scripts/CHANGELOG.md). No behavior +change; rides the next hub image train (no deploy for this). + ## v0.53.0 — closing bundle: F-14 gated auto-Reissue + dead-host roll-up honesty + bearer out of git (2026-07-13) The last engineering items on the pre-tester board. Two operator rulings in force (CONTEXT.md): diff --git a/hub/internal/web/configs.go b/hub/internal/web/configs.go index 1744ccf..557723d 100644 --- a/hub/internal/web/configs.go +++ b/hub/internal/web/configs.go @@ -25,7 +25,7 @@ var validCustomerID = regexp.MustCompile(`^[a-zA-Z0-9.\-]+$`) // only ever emits flags this version parses. Display-only (the Option-1 command downloads the served // script, which is always current); bump when the generator's flag surface follows a new script. // Drift is now gated: scripts/hostinstall_gates.py asserts this const == SCRIPT_VERSION (drill F-1). -const hostInstallVersion = "1.15.0" +const hostInstallVersion = "1.16.0" // validSemver matches a bare X.Y.Z controller version (the floor format). Empty is also accepted by // the floor handlers (clears the override). diff --git a/scripts/CHANGELOG.md b/scripts/CHANGELOG.md index 86ccf7f..7e9ca29 100644 --- a/scripts/CHANGELOG.md +++ b/scripts/CHANGELOG.md @@ -1,5 +1,16 @@ # Felhom scripts — Changelog +## felhom-host-install v1.16.0 — FELHOM_ESCROW rides the canonical sudoers (2026-07-13) + +Companion to agent v0.88.0 + controller v0.127.0 (the controller-driven escrow ceremony wizard). +No new installer STEP: the `FELHOM_ESCROW` sudoers alias (the wizard's fixed-argv sudo +self-invocation grant) ships via the existing canonical-sudoers fetch +(`configs/felhom-agent.sudoers` from the agent repo, `visudo -cf`-gated before install) — the +bump documents that every ≥1.16.0 install/update carries it. Hub `hostInstallVersion` bumped in +the same commit (hostinstall_gates.py asserts equality); the hub deploy rides the next hub train +(display-only lag, accepted). EXISTING hosts get the refreshed sudoers with the agent-0.88.0 +deploy (ship-configs step), not from this script. + ## manifest_bearer_gate.py — new gate: no bearer-shaped literals in manifests/ (2026-07-13) Part of the hub v0.53.0 bearer de-git. Greps every `manifests/*.yaml` for 64-hex literals diff --git a/scripts/felhom-host-install.sh b/scripts/felhom-host-install.sh index b5d259e..31427c2 100644 --- a/scripts/felhom-host-install.sh +++ b/scripts/felhom-host-install.sh @@ -182,9 +182,13 @@ set -euo pipefail -SCRIPT_VERSION="1.15.0" # the SINGLE version source (F-1): -h, the run banners, and the hub +SCRIPT_VERSION="1.16.0" # the SINGLE version source (F-1): -h, the run banners, and the hub # Setup-tab copy (hub internal/web/configs.go hostInstallVersion — # scripts/hostinstall_gates.py asserts the two stay equal) all follow it. + # 1.16.0: the FELHOM_ESCROW sudoers alias (controller-driven escrow + # ceremony, agent v0.88.0) ships on every install/update — it rides the + # CANONICAL sudoers fetch below (configs/felhom-agent.sudoers from the + # agent repo, visudo-gated), no separate installer step. # Operator signing keys pinned at day-0 (GL-4; doc 04 §3 two-key model). EMPTY by default — the pin # CEREMONY is an operator step: generate the real keypairs OFFLINE, then fill these four constants