validation record: offsite provisioning live e2e PASS (2026-07-09) + CONTEXT/REPORT

Full hub->Hetzner->controller loop validated on demo-felhom against the new
dedicated-project pool box; 2 snapshots on the new box, key-only. F1/F3 fixed
mid-run (hub v0.38.1, controller v0.106.1); F2/F4/F5/F6 recorded open.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
This commit is contained in:
2026-07-09 22:04:44 +02:00
parent 6d5b3c1c38
commit ecf9185605
3 changed files with 151 additions and 77 deletions
+24 -77
View File
@@ -2,85 +2,32 @@
> **Overwrite** this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in [hub/CHANGELOG.md](hub/CHANGELOG.md); the scripts history lives in [scripts/CHANGELOG.md](scripts/CHANGELOG.md).
## TASK GL-7 — customer page: passphrase hardening + install-command generator (hub v0.36.0) — 2026-07-09
## RUNBOOK — offsite provisioning LIVE end-to-end (supervised) — 2026-07-09 — **PASS**
**Shipped:** felhom.eu `844fbfa7` (code) + `02c748eb` (manifest bump); **hub v0.36.0 LIVE** on k3s
(ArgoCD app `felhom` Synced/Healthy, rollout confirmed, live image `felhom-hub:0.36.0`,
`Listening on :8080`). felhom.eu only — agent + host-install untouched. Two coupled, security-first
changes to the operator customer page, plus the Peti tester agreement.
**Full record:** `documentation/audits/VALIDATION-offsite-provisioning-e2e-2026-07-09.md`.
**Shipped this run (felhom.eu):** manifest wiring `0a65f2b` (HETZNER_* env from Secret/storagebox),
**hub v0.38.1** `7c70c54` + bump `6d5b3c1` (LIVE: Synced/Healthy, `Offsite provisioning enabled
(pool_box=611714, location=fsn1)`). Pairs with controller v0.106.1 (that repo's REPORT).
### Part 1 — passphrase hardening (the security win, ships first)
The first live run of the SLICE 1+2 loop on `demo-felhom` against the new dedicated-project pool box:
hub UI save → sub-account provisioned (269008, `u629488-sub1`) → non-secret descriptor + one-time password →
controller config-refresh → apply-bridge (verify-pin, consume-once, pinned `ssh-copy-id`, offbox configured,
`EscrowState=pending`) → fork-4 run-gate refusal proven → escrow confirm (Option A — the repo password was
already escrowed under Viktor's R) → **first run: restic repo initialized, 2 apps, 2 snapshots on the new box,
key-only** (independently verified by SFTP-listing the repo with the bridge-installed key + pinned host key).
The per-customer retrieval passphrase was in cleartext **twice**: the visible `#retrieval-pw` node
and baked into the Option-3 debug curl's `X-Retrieval-Password:` header (a copyable command carrying
the secret). Now:
- `#retrieval-pw` renders a **masked bullet run by default**, with a **Reveal** (`toggleSecret`) and
a **copy** (`copySecret`) control; the value lives in `data-secret` (the page's existing reveal
model — the value still ships in the DOM, unchanged from before).
- The Option-3 command carries a **`<YOUR-RETRIEVAL-PASSWORD>` placeholder** — the secret is never in
a copyable command block.
- A **zero-secret-in-DOM** reveal-on-demand fetch is a deliberate future follow-up, explicitly NOT
scoped here (kept the change tight and reviewable).
**Live bugs found + fixed mid-run:**
- **F1 (hub v0.38.1):** a re-click canceled `r.Context()` between `CreateSubaccount` and
`SaveOneTimeSecret` → sub-account stranded with its password lost. `applyOffsite` now provisions on
`context.WithoutCancel` + 3-min timeout. Test `TestApplyOffsite_ClientDisconnectMidProvision` + red-proof
(raw-ctx shape reproduces the exact live error).
- **F3 (controller v0.106.1):** `ssh-copy-id -s` mktemp's under `~/.ssh`, absent from the container image —
install died locally AFTER consuming the password (fail-safe held; password never transmitted).
### Part 2 — interactive install-command generator
**Open findings:** F2 fresh-subaccount DNS lag → first-save scan 502 (retry-with-backoff wanted);
F4 no hub password-reset path (recovery = delete sub-account + re-save); F5 no save spinner (the re-click
bait); F6 Hetzner box-level 423 action lock on concurrent saves (fail-closed, correct).
The three hard-coded install `<code>` blocks became a **client-side builder** (vanilla JS — no
framework, CDN, or network call) that assembles a live-updating command from form controls:
- Emits **only real host-install v1.12.0 flags** in a **download-then-run** shape (never `curl | bash`).
- CustomerID prefilled from the server (`pageData.ScriptVersion` + `data-customer-id`); a **byo**
selection **requires** `--cores`/`--memory` (client-side enforcement via `.gen-req` + a `gen-msg`
prompt); caps/mode are placeholders, never silent defaults.
- **Graceful JS-off static fallback**: the Option-1/2 code nodes retain a
`--customer-id … --mode <appliance|byo>` command.
- **Curated control surface** (mode/cores/memory/vmid/node/acl-storages/operator-pubkey-file/
preserve-state-from + skip-provision/dry-run/preflight-only/allow-new-leaf). The **seven
dangerous/operator-only flags** (`--force`, `--rotate-recovery`, `--enable-oob`, `--remove-golden`,
`--uninstall`, `--adopt-pool`, `--rescope-acl`) are **never offered as controls**.
`configs.go`: `const hostInstallVersion = "1.12.0"`; `pageData.ScriptVersion` added + populated.
`style.css`: `.gen-controls`/`.gen-radios`/`.gen-radio`/`.gen-check(s)`/number-inputs/`.gen-msg`
(dark palette, 2px radius).
### Part 3 — tests (green gate passed on committed source)
- **`TestTemplates_PassphraseHardened`** — the secret is NOT in the Option-3 command (placeholder is),
`#retrieval-pw` is masked-by-default (bullet-entity run), `data-secret` populated, reveal+copy
controls present. **Red-proof:** revert the Option-3 block to `{{.Config.RetrievalPassword}}` → the
"not-in-command" / masked assertions fail.
- **`TestTemplates_InstallGenerator`** — every curated control id renders, script version +
`data-customer-id` present, the JS-off static fallback command present, and **none of the seven
excluded flags appear anywhere on the page**.
- `go build ./... && go vet ./... && go test ./internal/web/`**ok** on the committed source (`844fbfa7`).
### Part 4 — deploy (GitOps)
Built `felhom-hub:0.36.0` on 180 from the pushed source, bumped `manifests/hub.yaml` (0.35.0→0.36.0),
ArgoCD hard-refresh + sync → Synced/Healthy, `deploy/hub` rolled out, live image + startup log
confirmed. **Gotcha caught:** the first build produced a stale image because the Part-1/Part-2 commits
were committed locally but not pushed before building (`build.sh` pulls origin) — fixed by pushing
first, then rebuilding.
### Part 5 — tester agreement + closeout
`documentation/pilot/PETI-tester-agreement.md` written: the BYO trust boundary (break-glass/OOB/WG
all OFF, non-root agent, pool-scoped token, TLS+per-guest-token local API), the honest limitations
(no offsite backup yet; physically-removable-drive caveat F2; pool-reassert-bring-up-only F7;
`:53`-must-be-free F6), exit rights (uninstall keeps drives/data/hub-record), Peti's box facts
(80c/128 GB → caps **12/32768**, `sajatfelhom.hu` tunnel re-point), the byo install command, and the
onboarding sequence (preflight → install → **G9 password-set** → tunnel → **backup-green**).
GO-LIVE-PACKAGE updated: GL-7 ✔; **G9/G10 → done**, **G11 → done** (with the one open pilot question
folded into the agreement's pre-install checklist); decision log + open-questions updated.
### The one remaining pilot question (owned by the agreement)
Peti's **local backup TARGET**`felhom-pbs` is unreachable from his LAN, so the default
`--acl-storages "local local-lvm felhom-pbs"` is wrong for him. Confirm at onboarding **whether his
box has a second disk/pool**: if yes it becomes the `--acl-storages` backup target (real local
resilience); if no, backups share the guest's pool (degraded — one drive failure loses both, must be
stated to him). Changes only the `--acl-storages` value, nothing structural.
### Verification note
The hub UI is operator-password-gated — CC cannot log in, so per the repo policy UI changes are
verified via the render tests (both green) plus the deploy checks (image/rollout/startup log), not a
live browser session.
**Standing state:** demo offsite now targets the NEW box, `escrowed`, un-paused; the old-box offsite
(`u629193-sub1`) is a frozen archive until a NEW-box restore round-trip is verified.
**NEXT:** SLICE 3 (escrow auto-confirm), SLICE 4 (soft-quota), F2/F4/F5, Peti/pilot onboarding runbook.