hub v0.84.0 — break-glass console credential on the host page

The credential existed and was not reachable when it was wanted. Every box has
had a strong random root@pam password since TASK G1, vaulted in the hub at day 0
and used for real during the sshd incident — but the only way to read it back was
a hand-written curl carrying the global operator key, a secret kept out-of-band.
In practice the PVE web console on a demo box felt locked.

The host page grows a Console access card: presence + username + set_at by
default, Reveal fetches the plaintext on demand for 60 s with a Copy button.
Masking clears the JS variable, and also fires on a second click and on
visibilitychange. A host with nothing vaulted says so, and says why.

The secret is NEVER rendered into the page, and that constraint shapes the
change. The render path uses a new store.GetHostRecoveryMeta whose struct and
SELECT both omit the secret column, so it is structurally incapable of carrying
one. The plaintext crosses the wire only in the response to POST
/hosts/{id}/reveal-recovery-credential (Cache-Control: no-store, CSRF-gated at
the ServeHTTP level; POST precisely so that gate applies and so no secret is
retrievable by URL alone). Deliberately NOT the customer page's data-secret
widget, which embeds the plaintext on every load.

A delivered reveal writes one recovery_credential_revealed event on the host's
customer timeline (info, source hub, Hungarian) via SaveEvent alone — no
dispatcher, nobody emailed, the log_tail_requested shape. Two reveals write two
events: the register records accesses, not states. A 404 is not an access. An
unbound host reveals fine and writes no event; the [INFO] hub line, carrying the
username and a length only, is then the record.

The global-key API path is untouched by design — it is the route for when the
hub UI itself is broken, and coupling it to the session layer would delete the
independence that makes it a fallback.

Recorded as a real trade: the hub session password alone now unlocks console root
fleet-wide, where retrieval previously also needed the global key. Accepted for a
single-operator, HU-geo-fenced hub that already stores these passwords in
plaintext at rest (CONTEXT.md ruling S-4). The plaintext-at-rest half is filed as
R-133 — every hub DB backup is a fleet-wide console-credential dump.

Tests 550 -> 559; four red-proofs (page leak, audit event, CSRF gate, route
order) each run, observed failing, and reverted. The route-order proof is a seam
test driving ServeHTTP: a handler-level test cannot see that defect, because the
handler is correct and simply never runs.
This commit is contained in:
2026-07-31 08:19:36 +02:00
parent 0a9bd3829d
commit 1956e5d390
14 changed files with 801 additions and 157 deletions
+15
View File
@@ -59,6 +59,21 @@ cited as authoritative). They are **decisions, not observations**; the rewrite l
compromised hub yields blobs nobody can open, *provided the operator's key is never stored in the compromised hub yields blobs nobody can open, *provided the operator's key is never stored in the
hub*. That proviso is why escrow custody is an open decision (`07` §11-A). hub*. That proviso is why escrow custody is an open decision (`07` §11-A).
**S-4 — the hub session password alone now unlocks console root on every managed box (2026-07-31, hub v0.84.0).**
Retrieving a host's vaulted break-glass `root@pam` credential previously required the **global operator
API key**, a secret distinct from the hub login and kept out-of-band. The `Console access` card on the
host page puts that retrieval behind the **hub session** (`POST /hosts/{id}/reveal-recovery-credential`,
CSRF-gated, audited), so the hub login password is now sufficient. **Accepted deliberately**, and the
reasoning is the ruling: the hub is single-operator and geo-fenced to Hungary, and it already stores
these passwords **in plaintext at rest** — so the DB and the login were already jointly sufficient, and
this is a convenience trade rather than a new exposure. It is a real trade, not a free one, and it is
recorded here rather than only in a changelog. Two things follow. **(a)** The global-key path
(`api/handler.go handleAdminGetRecoveryCredential`) stays untouched and must acquire **no** dependency
on the session layer — it is the route for when the hub UI itself is what is broken, and coupling the
two would delete the independence that makes it a fallback. **(b)** The plaintext-at-rest half is now
the named open item **R-133**: every hub DB backup is a fleet-wide console-credential dump, and the fix
shape is envelope encryption under a KEK held outside the DB, the way escrow blobs are already opaque.
Five decisions were deliberately **left open for the operator** and are recorded, unanswered, in Five decisions were deliberately **left open for the operator** and are recorded, unanswered, in
`07` §11: escrow custody · lost-R policy · per-scenario RTO/RPO targets (**none have ever been `07` §11: escrow custody · lost-R policy · per-scenario RTO/RPO targets (**none have ever been
stated**) · Hetzner as a single failure domain · and `local` vzdump sharing a physical device with stated**) · Hetzner as a single failure domain · and `local` vzdump sharing a physical device with
+101 -155
View File
@@ -1,183 +1,129 @@
# REPORT — E-2 increment 1: the backup target is now defended and audible (2026-07-29) # REPORT — Hub v0.84.0: break-glass console credential on the host page (2026-07-31)
**Overwritten** per the standing rule. **E-2 is PARTIALLY shipped** — this increment is Parts 1 + 5 ## 1. Confirmed baselines actually used
plus findings E-2b/E-2c. Parts 2, 3, 4 and E-2a remain open (see §6).
**Live fleet after this run:** hub **0.81.0**, agent **0.112.0** (both boxes), controller **0.184.1** | Repo | `main` @ start | Version before | Version after |
(both boxes). `peti-felhom` deliberately untouched. |---|---|---|---|
| felhom.eu | `0a9bd3829d9a4465b032ea78813f55e935fbf88e` ("D5 SHIPPED…", 2026-07-30) | hub **v0.83.0** (`manifests/hub.yaml:128``felhom-hub:0.83.0`) | hub **v0.84.0** |
--- Clean-tree gate at start: `git status --porcelain` empty, `HEAD == origin/main`. **Note:** a second
Claude Code session was working in this shared clone concurrently — see §10.
## 1. Why this order ## 2. Files created / modified
The operator reordered the plan, correctly: **E-2b and E-2c protect a configuration that exists Created:
right now; E-2a builds for boxes that do not.** Both demo boxes are configured, Peti is parked, and - `hub/internal/web/hosts_recovery_reveal_test.go`
no new installs are happening this week — so Part 3 has no user yet, while E-1's new configuration
had a drive that could be ejected without warning and no event when it went absent.
## 2. E-2c — the backup drive could be ejected out from under the backup Modified:
- `hub/internal/store/host_recovery.go``HostRecoveryMeta` + `GetHostRecoveryMeta`
- `hub/internal/store/host_recovery_test.go` — Group H
- `hub/internal/web/hosts.go``handleHostRevealRecoveryCredential`; `hostDetailData` +3 keys
- `hub/internal/web/server.go` — route, above the bare `/hosts/` catch-all
- `hub/internal/web/templates/host_detail_body.html` — Console access card + fetch-on-demand script
- `hub/CHANGELOG.md`, `REUSE.md`, `CONTEXT.md` (ruling **S-4**),
`documentation/runbooks/break-glass.md` (§3.1 + §5),
`documentation/architecture/00-capability-map.md` (L120),
`documentation/backlog/ROADMAP.md` + `documentation/backlog/OPEN-ITEMS.md` (**R-133**)
`RoleForStorage` types a `local-dir` on a non-system device as **user-data**, so the pre-existing ## 3. Commits pushed to `main`
eject role gate PASSED the drive E-1 had just made the vzdump target. `POST /disks/eject` on
`/mnt/nvme-1tb` or `/mnt/hdd_1` would have **succeeded silently**, taking the only local whole-guest
backup with it while the box kept reporting a configured tier.
Fixed in agent v0.111.0/0.112.0: `handleDiskEject` and `handleDiskDecommission` refuse `409` when the *(filled in at push — see `git log --oneline`)*
mount backs a configured tier, naming the storage and the remedy — the op is *ordered*, not
forbidden.
**Deliberately NOT a role reclassification.** Making `RoleForStorage` return `RoleBackup` would ## 4. Test results + red-proofs
refuse every legitimate eject of the customer's own data drive, because on both boxes that drive
**is** the target. `TestEjectStillAllowedOnANonTargetDrive` pins that, and the over-correction is
red-proofed.
**Live on both boxes:** Every test drives `RequireAuth(ServeHTTP)` — never a handler function directly.
``` | Test | Scenario | Result |
demo-hp felhom-backup mount=/mnt/nvme-1tb role=user-data backup_target=True |---|---|---|
demo-felhom felhom-backup mount=/mnt/hdd_1 role=user-data backup_target=True | `TestReveal_A_PageNeverCarriesTheSecret` | A — render, canary absent from the WHOLE body | PASS |
``` | `TestReveal_B_RevealDeliversAndAudits` | B — 200 + no-store + payload + exactly 1 event + no log leak + 0 notifications | PASS |
| `TestReveal_C_NotVaulted` | C — explanatory card, no control, 404, zero events | PASS |
| `TestReveal_D_CSRFRequired` | D — 403, no leak, zero events; + the with-token discriminator | PASS |
| `TestReveal_E_MethodGateAndRouteOrder` | E — 405 and no fall-through to the host page | PASS |
| `TestReveal_F_UnknownHost` | F — 404, no panic | PASS |
| `TestReveal_G_Unauthenticated` | G — 401, no leak, zero events | PASS |
| `TestReveal_UnboundHostRevealsWithoutAnEvent` | §8 edge — 200, no event, log is the record | PASS |
| `TestGetHostRecoveryMeta_MetadataOnly` | H — store round-trip + absent cases | PASS |
That single line is the exposure: `role=user-data` (which the gate permits) yet `backup_target=True`. **Red-proofs — each mutation applied, observed failing, reverted:**
Both ejects returned **409** with the full message; both decommissions **409**; both drives still
mounted with `dump/` intact — the refusal had no side effect. `/var/lib/vz` is still refused by the
**pre-existing** role gate (`role: system`), proving the new gate is narrow.
## 3. E-2b — the alarm that was defined and never called | # | Mutation | Observed | Reverted |
|---|---|---|---|
| A | `hostDetailData` gains `RecoverySecret` (from `GetHostRecoveryCredential`) + `data-secret="{{.RecoverySecret}}"` on the card | `FAIL … SECRET LEAK: the vaulted console password appears in the rendered host page` | yes |
| B | delete the `SaveEvent` block in `handleHostRevealRecoveryCredential` | `FAIL … recovery_credential_revealed rows = 0, want exactly 1` | yes |
| D | short-circuit the `ServeHTTP` CSRF check (`if false && !s.validateCSRF(r)`) | `FAIL … reveal without CSRF = 200, want 403` | yes |
| E | move the new `case` BELOW `case strings.HasPrefix(path, "/hosts/")` | `FAIL … GET on the reveal route = 404, want 405` | yes |
`NotifyStorageDisconnected` and `NotifyStorageReconnected` were registered in `allowedEventTypes`, in **Red-proof A did not land on the first attempt, and that is worth recording.** The first mutation
`DefaultEnabledEvents`, and given Hungarian hub copy — and **called from nowhere**. A drive going edited only the template; a `{{.RecoverySecret}}` against a map with no such key renders empty, so
absent produced stopped apps, a `[WARN]` log and a UI badge, then silence on every channel. A drive the test passed and would have certified nothing. The proof landed only once the *data* half was
that is *only* a backup target has no apps to stop, so it was silent twice over. Fifth instance of mutated too — i.e. the assertion is pinned to the view-model, not to template text.
this class; found by Phase 0, not by a failure.
Now wired in `ReconcileDriveGates`, with the target case raising the more specific **Red-proof E's symptom differed from the prediction.** The spec expected the misordered route to
`backup_target_absent` (error) and recovering as `backup_target_restored` (info — the pairing-gated render the host page (200); it actually 404s, because the catch-all takes
pattern; `severityNotifies` NOT widened). Never both: two mails for one event trains people to `demo-felhom-8363b5/reveal-recovery-credential` as the host id and `GetHost` misses. The test goes
ignore the channel. red either way, and its second assertion (no `Console access` in the body) still pins the
fall-through case.
## 4. The bug I introduced and caught before deploy **One spec assertion caught a real gap during development.** Scenario A's requirement that the page
"contains a Reveal control targeting `/hosts/{id}/reveal-recovery-credential`" failed at first: the
URL was assembled in JS (`'/hosts/' + encodeURIComponent(hostID) + '/…'`) and appeared nowhere in the
DOM. Fixed by putting the endpoint in `data-reveal-url` on the button **and making the fetch read it
from there**, so the string the render test asserts is the string the request uses — an attribute
nothing reads would have been a hollow assertion.
`ReconcileDriveGates` resolves the target as `isTarget[a.Path]`, and **`a.Path` is the registered ## 5. Test count
GUEST path** (`/mnt/felhom-drives/<name>`), not the agent's host `MountPath`. I had keyed the map on
`MountPath` alone, so the backup-target branch was **unreachable** — every absent drive, the target
included, would have fallen through to the generic event.
It would have looked wired, passed its own unit tests, and shipped silently wrong on exactly the **550 → 559** (8 web + 1 store). Full suite `go build ./... && go vet ./... && go test ./...` in
drive it exists for. Caught by tracing `a.Path` back to its source instead of assuming. Fixed in `hub/`: **rc=0, all 17 packages ok**. Run as a separate command from the commit, per standing rule 1.
v0.184.1 (both keys, mirroring `planDriveGates`), red-proofed, and the **0.184.0 image is superseded
and was never deployed**.
## 5. Tests and red-proofs ## 6. Deployed version
| Red-proof | Observed failure | *(filled in after the build/manifest/sync steps)*
|---|---|
| hub allowlist entry removed | `backup_target_absent must be in allowedEventTypes, or POST /event 400s…` |
| hub Hungarian copy shortened | `must state the consequence…, got: "Meghajtó hiányzik."` |
| C — auto-elevation on register | `registering drives assigned the backup target "/mnt/hdd_1"` |
| E-2c — eject guard removed | `eject of the backup-target drive SUCCEEDED (200)` |
| E-2c over-correction | the gate blocked `/mnt/spare` (a non-target) |
| keying reverted to MountPath | `not resolvable by its GUEST path — …the branch would never fire` |
Every mutation was **verified to have landed** before the result was trusted. The first attempt ## 7. NOT yet live-validated — awaiting the operator
silently no-op'd (gofmt had realigned the map) and the test "passed" — a false proof.
Controller notify tests observe the **wire** (a real `Notifier` against an `httptest` hub), not a **That the revealed password actually authenticates at `https://<box-ip>:8006` as `root@pam` on
mock, because the failure class is "nothing arrives". `demo-felhom-8363b5`.** This is the only test that proves the hub's copy still matches the box, and
it needs a browser and a real login — CC has neither here (no `claude-in-chrome` on DooPlex). Every
other leg is endpoint-level validated (§6).
## 6. What is NOT done ## 8. Teardown
- **Part 2** (installer Case A/B), **Part 3** (the offer + the agent-side move), **Part 4** (the **This run provisioned nothing** — no guest, no VM, no customer, no drive, no external resource. No
degraded Hungarian banner). Phase 0 inverted their emphasis: the installer has **no drive-enrollment teardown obligation.
step**, so at install the common case is system-drive-only and **Part 3 is the primary path**.
- **E-2a** — the guarded wrapper. The agent holds neither `Datastore.Allocate` at `/storage` nor
`Permissions.Modify`, and its sudoers has no `pvesm`/`pveum`. Use the wrapper pattern; **do not
widen the agent's PVE role.**
- **`backup_target_absent` is not proven END-TO-END live.** Its components are: the controller emits
the right type/severity (proven at the wire), the hub allowlists it (unit-pinned, image deployed),
and the keying is pinned. Proving the whole chain needs a live drive to actually go absent, which
means unmounting an enrolled drive carrying customer data and the guest bind on a remote box —
the same thing declined in E-1, for the same reason.
- Full drive-loss recovery (physical access) — carried forward from E-1.
## 7. Observations ## 9. Backlog rows opened / closed / re-ranked
- **`peti-felhom` untouched.** Its whole-guest backup still shares a device with its guest, so a - **Opened: `R-133`** — the vaulted secret is plaintext at rest, so every hub DB backup is a
drive failure there is offsite-only recovery. Accepted until the operator's reinstall; re-evaluate fleet-wide console-credential dump; envelope-encrypt `host_recovery.secret` under a KEK held
if that slips past ~2026-09-01. outside the DB. Added to `ROADMAP.md` and `OPEN-ITEMS.md` (owner CC, `READY (M)`). Named the
- The fleet was briefly split (demo-hp updated first). Left that way it would have protected one of capability-map row it would flip.
two identically-exposed boxes, so demo-felhom was rolled to the same proven artifacts. - **Closed / re-ranked: none.**
- `StorageTarget.Role` carries a documented vocabulary including `vzdump-target` and is always `""`. - **ID collision, resolved:** the spec predicted R-128. The concurrent session's *uncommitted* WIP in
Checked — it is a deliberate unpopulated field ("hub-owned, slice 10"), **not** a sixth dead seam. `OPEN-ITEMS.md` had already taken **R-128 through R-132**, so this item took **R-133**. An ID
register that lives in a file two sessions edit at once cannot allocate safely by reading committed
state — worth noting, not fixed here.
--- ## 10. Observations — recorded, deliberately NOT acted on
# E-2 increment 2 (same day): the wrapper, the installer, the offer and the banner 1. **`customer_unified.html`'s `data-secret` / `toggleSecret` widget embeds the plaintext in the page
HTML on every load.** It therefore lives in the back/forward cache, in "save page as", and in any
**Live fleet:** hub **0.81.0**, agent **0.113.0**, controller **0.185.1**, host-install **1.22.0** DOM-capturing screenshot. Acceptable for one customer's retrieval passphrase; it is the reason
(script; no reinstall performed). Both demo boxes. `peti-felhom` untouched. this task built fetch-on-demand instead of reusing it. Not refactored — out of scope.
2. **`validateCSRF` returns `true` when no session cookie is present** (`server.go`, the Basic-Auth
## E-2a — the guarded wrapper path). So a Basic-Auth caller reaches the reveal endpoint **without any CSRF token**. That is the
pre-existing hub-wide contract, not something this endpoint introduced, and it is what makes the
`configs/felhom-backup-target-apply` behind a literal `FELHOM_BACKUPTARGET` sudoers alias. **The §6 curl validation possible at all — but it does mean "CSRF-gated" is true only for session
agent's PVE role was not widened** — it still holds neither `Datastore.Allocate` at `/storage` nor callers. Recorded, not changed.
`Permissions.Modify`. 3. **The Console access card also renders on the customer page's Host tab**, because
`host_detail_body` is shared and `configs.go` builds its view models through the same
All five laws proven live as root on demo-hp, with **0 stray storages created**: `hostDetailData`. That is per-host and operator-only (the hub has no customer login), so it does
not breach the "never on the hosts LIST" rule — but it is a second surface, and it is stated here
| Probe | Result | rather than left to be discovered.
|---|---| 4. **A concurrent session shares this clone.** Its in-flight edits to `OPEN-ITEMS.md`,
| F-1 subdirectory (`/mnt/nvme-1tb/dump`) | REFUSED — "must be the drive's OWN mountpoint" | `RUNBOOK-manual-build.md`, `RUNBOOK-publish-0.79-0.110-2026-07-10.md` and two new untracked files
| F-2 unmounted path | REFUSED — "would silently retarget onto the system drive" | were **not** swept into this commit: every path was staged explicitly, and `OPEN-ITEMS.md` was
| root device (`/`) | REFUSED | staged as a hand-built blob (`HEAD` + the R-133 row only) so the foreign WIP stayed unstaged in
| idempotent re-apply on the real target | no-op, exit 0 | the working tree. `REPORT.md` was taken here because that session had already chosen the
| repointing an existing id | REFUSED | `REPORT-tester-gate-2026-07-31.md` sibling.
`POST /backup/target` drives it **create → grant → config**, and deliberately does **not** restart the
agent: it returns `restart_required` so the restart happens behind the caller's own immediate
in-flight check. That is E-1's spurious-tier-failure mistake encoded as a contract, and a test asserts
the handler never restarts.
## Part 2 — the installer (v1.22.0)
Case A creates the target through the same wrapper; Case B records DEGRADED in two warning lines and
**still succeeds** — a single-drive appliance is valid. Scenario F returns early with **0 wrapper
calls**; the red-proof (guard removed) shows **2**, i.e. it would have "corrected" a correct box.
**Installer-logic-tested, NOT install-tested** — the harness runs the extracted functions with
stubbed `pvesm`/wrapper. No reinstall was performed.
## Parts 3+4 — the offer and the banner
`POST /api/storage/backup-target/assign` is the **only** writer of the role. `GET
/api/storage/backup-target` drives the banner. Hungarian copy is FACT → CONSEQUENCE → REMEDY.
**Scenario E proven live on both boxes** — healthy renders nothing:
```
demo-hp {"degraded":false,"known":true,"label":"NVME 1TB","target":"felhom-backup"}
demo-felhom {"degraded":false,"known":true,"label":"USB HDD 1TB","target":"felhom-backup"}
```
No `message` key at all. Red-proofed: adding a reassurance to the healthy branch fails Scenario E.
## Three defects I introduced and caught
1. **The offer endpoints were unreachable.** Registered as `/api/backup-target` inside
`ServeStorageAPI`, which `main.go` mounts only at `/api/storage/`. Every unit test passed because
they called the handlers directly; the **first live call** returned `endpoint not found`. Fixed in
v0.185.1 with a test that asserts the dispatcher's own source contains both routes.
2. **A hollow test.** `TestUnknownStateRendersNothing` used `{Known:false}` with `Degraded` left
false, so it passed even with the `!Known` guard deleted. Its own red-proof exposed it; the
fixture is now `{Known:false, Degraded:true}`.
3. **A gofmt-realignment no-op** (again): a `sed`-style replace silently matched nothing because the
formatter had changed the spacing. Every mutation is now verified to have landed before its result
is trusted.
## Still not live-proven
- The **DEGRADED banner** and the **offer acceptance** — both boxes are healthy, so neither state
occurs naturally. Unit- and red-proofed only.
- **`backup_target_absent` end-to-end** — needs a live enrolled drive to go absent.
- **Case A/B on a real install** — installer-logic-tested only.
- **Full drive-loss recovery** — physical access.
+1
View File
@@ -71,6 +71,7 @@
| Symbol | File | Short signature | Use for | Gotchas | | Symbol | File | Short signature | Use for | Gotchas |
|---|---|---|---|---| |---|---|---|---|---|
| `(*Server).hostDetailData` | hub/internal/web/hosts.go (~L282) | `(host *store.Host, r) map[string]interface{}` | The ONE view-model builder for the shared `host_detail_body` sub-template (standalone `/hosts/{id}` + customer Host tab) | Booleans/counts only for DR/escrow; carries `Deletable` (= status != "ok") which gates the danger-zone card. Never add a secret field. | | `(*Server).hostDetailData` | hub/internal/web/hosts.go (~L282) | `(host *store.Host, r) map[string]interface{}` | The ONE view-model builder for the shared `host_detail_body` sub-template (standalone `/hosts/{id}` + customer Host tab) | Booleans/counts only for DR/escrow; carries `Deletable` (= status != "ok") which gates the danger-zone card. Never add a secret field. |
| `(*Store).GetHostRecoveryMeta` + `(*Server).handleHostRevealRecoveryCredential` | hub/internal/store/host_recovery.go · hub/internal/web/hosts.go | `(hostID) (*HostRecoveryMeta, error)` · `POST /hosts/{id}/reveal-recovery-credential` | The break-glass console credential, split into a RENDER half and a RETRIEVE half (v0.84.0) | **Use `GetHostRecoveryMeta` on any page-render path** — its struct and its `SELECT` both omit the `secret` column, so it cannot leak one; `GetHostRecoveryCredential` (which does select it) belongs only to the two retrieval handlers. The reveal is POST so the ServeHTTP-level CSRF check applies and no secret is reachable by URL; it writes ONE `recovery_credential_revealed` event via `SaveEvent` and calls NO dispatcher (the `handleRequestLogTail` shape). `api/handler.go handleAdminGetRecoveryCredential` (global key) is the independent fallback for when the UI is down — never route the UI through it. Secret at rest is plaintext → R-133. |
| `host_detail_body` sub-template | hub/internal/web/templates/host_detail_body.html | `{{template "host_detail_body" .}}` | Rendering a host's detail sections on ANY surface | One namespace across ParseFS (icons.html pattern). Renders per-host — id-suffix any new element ids with `{{.HostID}}` (the customer page renders N instances). | | `host_detail_body` sub-template | hub/internal/web/templates/host_detail_body.html | `{{template "host_detail_body" .}}` | Rendering a host's detail sections on ANY surface | One namespace across ParseFS (icons.html pattern). Renders per-host — id-suffix any new element ids with `{{.HostID}}` (the customer page renders N instances). |
| `(*Store).ListHostsByCustomer` | hub/internal/store/store.go (~L1620) | `(customerID) ([]Host, error)` | A customer's hosts, host_id order | A LIST by design (HA-cluster roadmap) — don't collapse to GetHostByCustomer. | | `(*Store).ListHostsByCustomer` | hub/internal/store/store.go (~L1620) | `(customerID) ([]Host, error)` | A customer's hosts, host_id order | A LIST by design (HA-cluster roadmap) — don't collapse to GetHostByCustomer. |
| `(*Server).configFormData` (v0.49.0) | hub/internal/web/configs.go (~L430) | `(r, isNew, cfg, overrides, errMsg) configFormView` | The ONE view-model builder for the customer config form (standalone chrome + the customer page Edit tab) | `overrides=nil` → parses the STORED cfg.ConfigJSON; pass the SUBMITTED map on the update validation-error re-render or typed values reset (red-proofed). | | `(*Server).configFormData` (v0.49.0) | hub/internal/web/configs.go (~L430) | `(r, isNew, cfg, overrides, errMsg) configFormView` | The ONE view-model builder for the customer config form (standalone chrome + the customer page Edit tab) | `overrides=nil` → parses the STORED cfg.ConfigJSON; pass the SUBMITTED map on the update validation-error re-render or typed values reset (red-proofed). |
@@ -117,7 +117,7 @@
| Forgot dashboard password → instant reset code | controller v0.123, hub | **PROVEN-LIVE** | `DRILL-day0-take2-2026-07-12` F-15 (live re-run of the exact failure path: hash applied 1s after request, code accepted first try) | | | Forgot dashboard password → instant reset code | controller v0.123, hub | **PROVEN-LIVE** | `DRILL-day0-take2-2026-07-12` F-15 (live re-run of the exact failure path: hash applied 1s after request, code accepted first try) | |
| Multiple household users / per-person accounts | — | **MISSING** | — | Single dashboard password; acceptable for alpha → R-15 | | Multiple household users / per-person accounts | — | **MISSING** | — | Single dashboard password; acceptable for alpha → R-15 |
| WireGuard base infra always-on; OOB operator access (felhom-sshd, /32 peer) | agent v0.72, hub v0.35 | **IMPLEMENTED** | `SPIKE-oob-wg-operator-peer-2026-07-05`, `SPIKE-felhom-sshd-2026-07-05` | Mutual-repair desired-state arc not built → R-13 | | WireGuard base infra always-on; OOB operator access (felhom-sshd, /32 peer) | agent v0.72, hub v0.35 | **IMPLEMENTED** | `SPIKE-oob-wg-operator-peer-2026-07-05`, `SPIKE-felhom-sshd-2026-07-05` | Mutual-repair desired-state arc not built → R-13 |
| Break-glass management-plane recovery | agent v0.71, hub v0.34 | **IMPLEMENTED** | `runbooks/break-glass.md` | | | Break-glass management-plane recovery | agent v0.71, hub v0.84 | **IMPLEMENTED** | `runbooks/break-glass.md` | hub v0.84.0 adds an **operator-SESSION** retrieval path (host page → Console access → Reveal; `POST /hosts/{id}/reveal-recovery-credential`, CSRF-gated, writes a customer-visible `recovery_credential_revealed` event) beside the pre-existing **global-key** one (`GET /api/v1/admin/hosts/{id}/recovery-credential`), which is untouched and stays the route for when the hub UI itself is down. Still IMPLEMENTED, not PROVEN-LIVE: the UI path has not been exercised on a real lockout, and that the revealed password authenticates at `:8006` is operator-verified only. The vaulted secret is plaintext at rest → **R-133** |
## F. Notifications & monitoring ## F. Notifications & monitoring
+1
View File
@@ -70,6 +70,7 @@ State: `BLOCKED` · `READY` · `WAITING-ON-OPERATOR` · `WATCHING`. Every row ha
| **R-89** | Retention as a per-customer **commercial** policy on the hub | READY (increment 2) | — | Policy object + reconciler → ep0 prune job; keep box tokens write-only | CC | | **R-89** | Retention as a per-customer **commercial** policy on the hub | READY (increment 2) | — | Policy object + reconciler → ep0 prune job; keep box tokens write-only | CC |
| **R-92** | Hub PBS-DR gauge is 0.1 GB-granular — small deltas unverifiable | READY (XS) | — | Widen precision when retention becomes customer-visible | CC | | **R-92** | Hub PBS-DR gauge is 0.1 GB-granular — small deltas unverifiable | READY (XS) | — | Widen precision when retention becomes customer-visible | CC |
| **R-93** | `drill-r50` is both a blocked customer and the only drift fixture | READY (XS) | — | Retire it for a synthetic fixture, or unblock + silence per-customer | CC | | **R-93** | `drill-r50` is both a blocked customer and the only drift fixture | READY (XS) | — | Retire it for a synthetic fixture, or unblock + silence per-customer | CC |
| **R-133** | **The vaulted break-glass console credential is PLAINTEXT AT REST — every hub DB backup is a fleet-wide console-credential dump.** `host_recovery.secret` holds each managed box's `root@pam` password verbatim, so any copy of the SQLite DB (Longhorn snapshot, PBS backup of the hub PVC, a hand-taken copy during a diagnosis) carries root console access to every Felhom host in one file | **READY (M) — NEW 2026-07-31** | — | **The deferred leg of hub v0.84.0** (Console access card), filed separately because v0.84.0 changed only WHO can retrieve the secret, never how it is stored. v0.84.0 makes it more worth doing, not more broken: retrieval now rides the hub SESSION, so the DB and the login password are jointly the whole protection (ruling **S-4**, `CONTEXT.md`). Fix shape: **envelope-encrypt the `host_recovery.secret` column under a KEK held outside the DB** — the hub already proves it can hold something it cannot itself read (escrow blobs), and that contrast is the argument. Two constraints the design must respect: the credential must stay retrievable **when the box is unreachable** (that is the whole point of break-glass), so the KEK cannot live on the box or depend on the agent; and the global-key API path must keep working with the hub UI down. Would flip the capability-map row **"Break-glass management-plane recovery"**, which today reads IMPLEMENTED with this as its caveat | CC |
## Why the TOP READY rows rank this way ## Why the TOP READY rows rank this way
+1
View File
@@ -110,6 +110,7 @@
| R-107 | **No offsite action unpacks the named-volume tars Tier-3 captures on every run.** `ReconstituteFromOffsite` skips the unit outright (`internal/backup/offbox_reconstitute.go:284-289`); `PlaceOffsiteRestore` places it only when the live unit is ABSENT (`internal/backup/offbox_restore.go:352-356`). The DB **is** replayed, from the scratch unit's `db-dumps/` | M | READY — 2026-07-28 | 52 of 53 catalog templates keep data in Docker named volumes, so for most apps the offsite tier carries the volume data and cannot itself put it back. A two-step route exists (full restore → place → Tier-1 restore, which does unpack tars) but **no single action does it and no UI routes it** — the R-103 shape one tier over. Sibling of R-102: both are "captured and never read". Flips: the offsite-restore row in map §C; `07` §6.3, §7.2 | | R-107 | **No offsite action unpacks the named-volume tars Tier-3 captures on every run.** `ReconstituteFromOffsite` skips the unit outright (`internal/backup/offbox_reconstitute.go:284-289`); `PlaceOffsiteRestore` places it only when the live unit is ABSENT (`internal/backup/offbox_restore.go:352-356`). The DB **is** replayed, from the scratch unit's `db-dumps/` | M | READY — 2026-07-28 | 52 of 53 catalog templates keep data in Docker named volumes, so for most apps the offsite tier carries the volume data and cannot itself put it back. A two-step route exists (full restore → place → Tier-1 restore, which does unpack tars) but **no single action does it and no UI routes it** — the R-103 shape one tier over. Sibling of R-102: both are "captured and never read". Flips: the offsite-restore row in map §C; `07` §6.3, §7.2 |
| R-108 | ~~Network storage can host an app's namespace, and FileBrowser binds a network share at its ROOT — this BLOCKS D5~~ | M | **SHIPPED** — controller v0.187.0, 2026-07-30 | **D5 UNBLOCKED.** One fail-closed predicate (`Settings.RefuseAsAppNamespace`) consulted by every placement surface, so no `backups/` tree can exist inside the share-root bind. The bind was deliberately NOT narrowed — it is load-bearing (`:rslave` automount wake, 2026-07-22 probe) and unscopable (apps on a share store at `<share>/<app>`; creating a `userdata/` layer would write Felhom convention onto a customer's NAS). **Five surfaces, not the four this row named** — decommission-with-migrate guarded only its SOURCE; and the boundary is the deploy POST, not the dropdown. Zero apps on network storage fleet-wide, so nothing stranded. The `.fab`-onto-NAS "second effect" is split out as **R-126**. Flipped: `07` §7.3 (D5 → unblocked), §10.1 (CLOSED), `00-capability-map.md` §D (new PROVEN-LIVE row). `audits/R108-network-app-namespace-2026-07-30.md` | | R-108 | ~~Network storage can host an app's namespace, and FileBrowser binds a network share at its ROOT — this BLOCKS D5~~ | M | **SHIPPED** — controller v0.187.0, 2026-07-30 | **D5 UNBLOCKED.** One fail-closed predicate (`Settings.RefuseAsAppNamespace`) consulted by every placement surface, so no `backups/` tree can exist inside the share-root bind. The bind was deliberately NOT narrowed — it is load-bearing (`:rslave` automount wake, 2026-07-22 probe) and unscopable (apps on a share store at `<share>/<app>`; creating a `userdata/` layer would write Felhom convention onto a customer's NAS). **Five surfaces, not the four this row named** — decommission-with-migrate guarded only its SOURCE; and the boundary is the deploy POST, not the dropdown. Zero apps on network storage fleet-wide, so nothing stranded. The `.fab`-onto-NAS "second effect" is split out as **R-126**. Flipped: `07` §7.3 (D5 → unblocked), §10.1 (CLOSED), `00-capability-map.md` §D (new PROVEN-LIVE row). `audits/R108-network-app-namespace-2026-07-30.md` |
| R-127 | **`data_key: true` is unreliable (4+ encryption keys unflagged, contradicting the catalog's own labels), and O4 can regenerate a DB password that no longer matches the restored data directory** | S/M | READY — NEW 2026-07-30 | Found by **D5's Part 0**, and the reason D5's boundary became `type: secret` rather than `data_key`. Leg (a): flag the missing keys (catalog-only) + pin flag-vs-label agreement; the residual risk after D5 is that the **fail-closed gate** keys on `data_key`, so an unflagged key missing from both sources lets the restore proceed onto undecryptable data. Leg (b): a regenerated DB password is silently wrong — `POSTGRES_PASSWORD` is ignored once PGDATA is non-empty, so the app cannot authenticate while the replay still succeeds over the local trust socket (proven live on `postgres:16-alpine`). v0.188.0 corrected the false "stored data is unaffected" WARN but added no guard. Flips: `07` §7.4 | | R-127 | **`data_key: true` is unreliable (4+ encryption keys unflagged, contradicting the catalog's own labels), and O4 can regenerate a DB password that no longer matches the restored data directory** | S/M | READY — NEW 2026-07-30 | Found by **D5's Part 0**, and the reason D5's boundary became `type: secret` rather than `data_key`. Leg (a): flag the missing keys (catalog-only) + pin flag-vs-label agreement; the residual risk after D5 is that the **fail-closed gate** keys on `data_key`, so an unflagged key missing from both sources lets the restore proceed onto undecryptable data. Leg (b): a regenerated DB password is silently wrong — `POSTGRES_PASSWORD` is ignored once PGDATA is non-empty, so the app cannot authenticate while the replay still succeeds over the local trust socket (proven live on `postgres:16-alpine`). v0.188.0 corrected the false "stored data is unaffected" WARN but added no guard. Flips: `07` §7.4 |
| R-133 | **The vaulted break-glass console credential is PLAINTEXT AT REST, so every hub DB backup is a fleet-wide console-credential dump.** `host_recovery.secret` holds each managed box's `root@pam` password verbatim (`hub/internal/store/host_recovery.go`*"a hub-held secret, operator-retrievable (NOT zero-knowledge like escrow)"*), so anything that copies the SQLite DB — a Longhorn snapshot, a PBS backup of the hub PVC, a hand-taken copy during a diagnosis — carries root console access to every Felhom host in one file, with no second factor and no key to withhold | M | READY — NEW 2026-07-31 | **The DEFERRED LEG of hub v0.84.0**, filed as its own ID because v0.84.0 changed only WHO can retrieve the secret, never how it is stored — the at-rest shape predates it and is untouched by it. **v0.84.0 made it more worth doing, not more broken:** putting retrieval behind the hub session means the hub login password alone now unlocks console root fleet-wide, so the DB and the login are the whole of the protection. Shape: **envelope-encrypt the `host_recovery.secret` column under a KEK held OUTSIDE the DB** (k8s Secret / out-of-band file, the way `manifests/` already keeps the bearer out of git), so a DB copy is opaque the way escrow blobs already are — the contrast is the argument, since the hub already proves it can hold a secret it cannot itself read. Constraints the design must respect: the credential must stay retrievable **when the box is unreachable** (that is the entire point of break-glass), so the KEK cannot live on the box or depend on the agent; and the global-key API path must keep working with the hub UI down. Flips: `architecture/00-capability-map.md` **"Break-glass management-plane recovery"** — the row that today reads IMPLEMENTED with a plaintext-at-rest caveat |
| D5 | ~~**Move app secrets into the LOCAL recovery unit** so Tier-1/Tier-2 restore stop needing the guest and stop needing R~~ | M | **SHIPPED + PROVEN-LIVE** — controller v0.188.0, 2026-07-30 | **The arc's architectural centrepiece. Tier-1/2 no longer depend on the whole-guest tier — a customer needs the DRIVE AND NOTHING ELSE.** Part 0 tested this row's own premise and **rejected** it: data-keys-only is both insufficient and unsafe, because `data_key` is unreliable (→ **R-127**) and a DB password is not resettable in practice (`POSTGRES_PASSWORD` is ignored once PGDATA is non-empty, so a regenerated value leaves the app unable to reach its own restored rows while the dump replay still reports success — proven on `postgres:16-alpine`). **Operator ruling: `type: secret` travels (45 fields), `type: password` never (7) plus a code register (`vaultwarden/ADMIN_TOKEN`); plaintext, because withholding the internet-reachable class is what licenses it — the two are coupled.** `stacks.PortableSecretEnvVars` is the single boundary; the register is code, not a catalog flag (R-97a). **Precedence: the UNIT WINS** (its secrets match the data being restored, not merely the newest), pinned both directions. Fail-closed data-key gate UNCHANGED. Manifest schema 2; schema-1 units still restore. Proven live on a scratch drill guest: AdventureLog restored with the guest `app.yaml` moved aside (`secrets recovered=2/2`, 27.6 s) and **the app read the seeded row over TCP with its own credential**; Grafana's admin password withheld with **0 hits** across the backup namespace. 4 red-proofs each verified to land. `audits/D5-drive-alone-restore-2026-07-30.md` Flips `07` §3/§7.1/§7.3/§7.4/§8/§10.1 + a new capability-map row | | D5 | ~~**Move app secrets into the LOCAL recovery unit** so Tier-1/Tier-2 restore stop needing the guest and stop needing R~~ | M | **SHIPPED + PROVEN-LIVE** — controller v0.188.0, 2026-07-30 | **The arc's architectural centrepiece. Tier-1/2 no longer depend on the whole-guest tier — a customer needs the DRIVE AND NOTHING ELSE.** Part 0 tested this row's own premise and **rejected** it: data-keys-only is both insufficient and unsafe, because `data_key` is unreliable (→ **R-127**) and a DB password is not resettable in practice (`POSTGRES_PASSWORD` is ignored once PGDATA is non-empty, so a regenerated value leaves the app unable to reach its own restored rows while the dump replay still reports success — proven on `postgres:16-alpine`). **Operator ruling: `type: secret` travels (45 fields), `type: password` never (7) plus a code register (`vaultwarden/ADMIN_TOKEN`); plaintext, because withholding the internet-reachable class is what licenses it — the two are coupled.** `stacks.PortableSecretEnvVars` is the single boundary; the register is code, not a catalog flag (R-97a). **Precedence: the UNIT WINS** (its secrets match the data being restored, not merely the newest), pinned both directions. Fail-closed data-key gate UNCHANGED. Manifest schema 2; schema-1 units still restore. Proven live on a scratch drill guest: AdventureLog restored with the guest `app.yaml` moved aside (`secrets recovered=2/2`, 27.6 s) and **the app read the seeded row over TCP with its own credential**; Grafana's admin password withheld with **0 hits** across the backup namespace. 4 red-proofs each verified to land. `audits/D5-drive-alone-restore-2026-07-30.md` Flips `07` §3/§7.1/§7.3/§7.4/§8/§10.1 + a new capability-map row |
| R-126 | **A `.fab` bundle — plaintext secrets, optional password — can be exported ONTO a NAS.** `storageDriveList()` (`internal/web/handler_export.go`) does not filter network paths | S | READY — 2026-07-30 | Split out of R-108 on its closure. NOT a D5 precondition: an explicit customer-chosen export destination, not a browsing surface reaching a backup tree (`07` §7.3 records the reasoning). Fix = filter network paths from the export destination list, or force the bundle password when the destination is a share. Flips: `07` §5 | | R-126 | **A `.fab` bundle — plaintext secrets, optional password — can be exported ONTO a NAS.** `storageDriveList()` (`internal/web/handler_export.go`) does not filter network paths | S | READY — 2026-07-30 | Split out of R-108 on its closure. NOT a D5 precondition: an explicit customer-chosen export destination, not a browsing surface reaching a backup tree (`07` §7.3 records the reasoning). Fix = filter network paths from the export destination list, or force the bundle password when the destination is a share. Flips: `07` §5 |
| R-109 | ~~The DR recipe records no backup target~~ | XS | **SHIPPED** — agent v0.118.1 + hub v0.83.0, 2026-07-30 | `backup_target` resolves from the PRIMARY tier of `cfg.Backup.BackupTiers()` (the scheduler's own function) + the mountpoint that separates `/mnt/hdd_1` from `/var/lib/vz`; unresolvable is recorded as unresolvable, never defaulted. Required an unscoped HUB half: `AssembleDRRecipe` allow-lists top-level keys → **R-122**, a fourth recipe-completeness defect that had already shipped (`offsite_restic` dropped for the feature's whole life). `audits/R106-R109-recipe-completeness-2026-07-30.md`; flipped `07` §4 | | R-109 | ~~The DR recipe records no backup target~~ | XS | **SHIPPED** — agent v0.118.1 + hub v0.83.0, 2026-07-30 | `backup_target` resolves from the PRIMARY tier of `cfg.Backup.BackupTiers()` (the scheduler's own function) + the mountpoint that separates `/mnt/hdd_1` from `/var/lib/vz`; unresolvable is recorded as unresolvable, never defaulted. Required an unscoped HUB half: `AssembleDRRecipe` allow-lists top-level keys → **R-122**, a fourth recipe-completeness defect that had already shipped (`offsite_restic` dropped for the feature's whole life). `audits/R106-R109-recipe-completeness-2026-07-30.md`; flipped `07` §4 |
+25 -1
View File
@@ -39,13 +39,26 @@ Layer 2 heals within ~60s with nobody logged in. If you just saw the KEXINIT-res
retry SSH. If it's back, check the hub for a `mgmt_plane_healed` warning and go to §4 (find the cause). retry SSH. If it's back, check the hub for a `mgmt_plane_healed` warning and go to §4 (find the cause).
### 3.1 If SSH is still dead — reach the PVE web console (break-glass) ### 3.1 If SSH is still dead — reach the PVE web console (break-glass)
1. **Retrieve the vaulted root@pam password** (operator/global key): 1. **Retrieve the vaulted root@pam password.**
**Primary route — the hub UI (hub v0.84.0):** hub → **Hosts** → the host → the **Console access**
card → **Reveal**. The password shows for 60 s with a Copy button. The card also states which user
it is for and when it was last vaulted, so a stale copy is visible before you try it. A host with
nothing vaulted says so and says why, instead of showing a blank. Needs only the hub login
password — no out-of-band key.
**Fallback — the global-key curl. Use this when the hub UI itself is unavailable** (that is the
whole reason it still exists, and why it deliberately has no dependency on the hub's session
layer):
``` ```
curl -s -H "Authorization: Bearer <GLOBAL-KEY out-of-band>" \ curl -s -H "Authorization: Bearer <GLOBAL-KEY out-of-band>" \
https://hub.felhom.eu/api/v1/admin/hosts/<host_id>/recovery-credential https://hub.felhom.eu/api/v1/admin/hosts/<host_id>/recovery-credential
# → {"host_id":"…","username":"root@pam","password":"…","set_at":"…"} # → {"host_id":"…","username":"root@pam","password":"…","set_at":"…"}
``` ```
(A per-host key is refused here by design — only the operator key reads a console password back.) (A per-host key is refused here by design — only the operator key reads a console password back.)
Either route is an **operator action against a customer's box**, and the UI route records it: see
§5.
2. **Open the PVE web UI** `https://<host-ip>:8006`, log in `root@pam` + that password, realm 2. **Open the PVE web UI** `https://<host-ip>:8006`, log in `root@pam` + that password, realm
*Linux PAM standard authentication*. *Linux PAM standard authentication*.
3. **Node → Shell** (xterm.js root console). Or navigate directly: 3. **Node → Shell** (xterm.js root console). Or navigate directly:
@@ -87,6 +100,17 @@ Fix the cause; the three layers are the safety net, not the cure.
wipes `state.json`, so step 4b re-randomizes — re-run the set-and-vault afterward. Background: on each wipes `state.json`, so step 4b re-randomizes — re-run the set-and-vault afterward. Background: on each
full reinstall this is exactly why `root@pam` "kept changing" until it was pinned. full reinstall this is exactly why `root@pam` "kept changing" until it was pinned.
- **The password is never logged or filed** — it exists only in the hub vault (retrieve as in §3.1). - **The password is never logged or filed** — it exists only in the hub vault (retrieve as in §3.1).
- **Revealing it through the hub UI writes a customer-visible event line** (hub v0.84.0):
`recovery_credential_revealed`, info severity, on that host's customer timeline, in Hungarian —
*„Az üzemeltető lekérte a géped konzolos hozzáférési jelszavát (távoli hibaelhárítás)."* Nobody is
emailed (no dispatcher call), and the register records **accesses**, so two reveals write two
lines. Deliberate, for a product sold on data sovereignty; it mirrors how a remote log pull is
already recorded. An **unbound** host has no customer to tell — there the hub `[INFO]` log line is
the only record. The global-key curl path writes no event.
- **The hub login password now unlocks console root on every managed box.** Before v0.84.0,
retrieval also required the global API key. Accepted for a single-operator, HU-geo-fenced hub that
already holds these passwords in plaintext at rest — a convenience trade, and a real one. The
plaintext-at-rest half (every hub DB backup is a fleet-wide console-credential dump) is **R-133**.
- **Least-privilege console user** instead of root@pam was probed as a non-blocking observation: PVE - **Least-privilege console user** instead of root@pam was probed as a non-blocking observation: PVE
has historically gated the node Shell to `root@pam`, so G1 ships the proven root@pam path. Revisit if has historically gated the node Shell to `root@pam`, so G1 ships the proven root@pam path. Revisit if
PVE later allows a scoped realm user to open a node Shell. PVE later allows a scoped realm user to open a node Shell.
+55
View File
@@ -1,3 +1,58 @@
## v0.84.0 — Break-glass console credential on the host page (2026-07-31)
**The credential existed and was not reachable when it was wanted.** Every Felhom-installed box has
had a strong random `root@pam` console password since TASK G1 — set on the box by
`felhom-host-install.sh` step 4b, vaulted in the hub at day 0, live for three hosts today, and used
for real during the sshd incident. The only way to read it back was a hand-written `curl` against
`/api/v1/admin/hosts/<id>/recovery-credential` carrying the **global operator key** — a different
secret from the hub login password, kept out-of-band. In practice the PVE web console on a demo box
felt locked.
**The host page grows a `Console access` card.** By default it states only that a credential is
vaulted, for which user, and when it was last set. A **Reveal** button fetches the plaintext on
demand and shows it for 60 s with a Copy button; masking clears the JS variable, and the mask also
fires on a second click and on `visibilitychange → hidden`. A host with nothing vaulted says so, and
says why (byo host, or step 4b never ran), with no Reveal control at all.
**The secret is never rendered into the page — that constraint shapes the whole change.** The render
path calls a new `store.GetHostRecoveryMeta`, whose struct and whose `SELECT` both omit the `secret`
column, so it is *structurally* incapable of carrying it; `hostDetailData` gains exactly three keys
(`RecoveryVaulted`, `RecoveryUsername`, `RecoverySetAt`). The plaintext crosses the wire only in the
response to `POST /hosts/{id}/reveal-recovery-credential``Cache-Control: no-store`, CSRF-gated at
the `ServeHTTP` level, POST precisely so that gate applies and so a secret is never retrievable by
URL alone (prefetch, history, referrer). The load-bearing test asserts the canary appears **nowhere**
in the rendered response — attribute, comment, inline script or JSON blob.
**Deliberately NOT the `customer_unified.html` `data-secret` widget**, which embeds the plaintext in
the page HTML on every load: acceptable for one customer's retrieval passphrase, not for console root
on every box in the fleet (it survives in the bfcache, in "save page as", and in any DOM-capturing
screenshot). That widget is untouched and recorded as an observation.
**Transparency, matching the log-pull precedent.** A delivered reveal writes one
`recovery_credential_revealed` event (info, source `hub`, Hungarian) on the host's customer timeline
`SaveEvent` alone, no dispatcher call, so nobody is emailed. Two reveals write two events: the
register records **accesses**, not states. A 404 is not an access and writes nothing. An unbound host
reveals fine and writes no event (no customer to tell); the `[INFO]` hub line is then the only
record, and it carries the username and a length — never the password.
**The global-key API path is untouched, by design.** `handleAdminGetRecoveryCredential` is the
break-glass route for when the hub *UI* is the thing that is broken; coupling it to the session layer
would remove exactly the independence that makes it a fallback.
**Recorded as a real trade, not a free one:** the hub session password alone now unlocks console root
on every managed box, where retrieval previously also required the global API key. Accepted for a
single-operator, HU-geo-fenced hub that already stores these passwords in plaintext at rest — and the
plaintext-at-rest half is now filed as **R-133** (envelope-encrypt `host_recovery.secret` under a KEK
held outside the DB, so a hub DB backup stops being a fleet-wide console-credential dump).
Files: `hub/internal/store/host_recovery.go` (+`GetHostRecoveryMeta`, `HostRecoveryMeta`),
`hub/internal/web/hosts.go` (+`handleHostRevealRecoveryCredential`, `hostDetailData`),
`hub/internal/web/server.go` (route, **above** the bare `/hosts/` catch-all),
`hub/internal/web/templates/host_detail_body.html` (card + fetch-on-demand script).
Tests 550 → 559; four red-proofs (A page-leak, B audit event, D CSRF gate, E route order) each run,
observed failing, and reverted. The E proof is a seam test driving `ServeHTTP`: a handler-level test
cannot see that defect, because the handler is correct and simply never runs.
## v0.83.0 — R-109 + R-122: the recipe assembly stops dropping sections (2026-07-30) ## v0.83.0 — R-109 + R-122: the recipe assembly stops dropping sections (2026-07-30)
Pairs with **agent v0.118.0** (R-106 + R-109). The agent half is useless without this one. Pairs with **agent v0.118.0** (R-106 + R-109). The agent half is useless without this one.
+28
View File
@@ -47,6 +47,34 @@ func (s *Store) GetHostRecoveryCredential(hostID string) (*HostRecoveryCredentia
return &c, nil return &c, nil
} }
// HostRecoveryMeta is the NON-SECRET shape of a vaulted break-glass credential: what the operator's
// host page shows without the plaintext ever entering the rendered document. The secret column is
// deliberately absent from both the struct and the query — the render path must be unable to carry it.
type HostRecoveryMeta struct {
HostID string
Username string
SetAt time.Time
}
// GetHostRecoveryMeta returns a host's credential metadata, or (nil, nil) if none is vaulted.
// Use this — NOT GetHostRecoveryCredential — on any path that renders a page: the secret can only
// leave the hub through the explicit, CSRF-gated, audited reveal endpoint.
func (s *Store) GetHostRecoveryMeta(hostID string) (*HostRecoveryMeta, error) {
var m HostRecoveryMeta
var setAt string
err := s.db.QueryRow(
`SELECT host_id, username, set_at FROM host_recovery WHERE host_id = ?`, hostID).
Scan(&m.HostID, &m.Username, &setAt)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
m.SetAt = parseSQLiteTime(setAt)
return &m, nil
}
// HasHostRecoveryCredential reports whether a host already has a vaulted credential (day-0 idempotency: // HasHostRecoveryCredential reports whether a host already has a vaulted credential (day-0 idempotency:
// don't regenerate/re-set on a re-run unless --rotate). // don't regenerate/re-set on a re-run unless --rotate).
func (s *Store) HasHostRecoveryCredential(hostID string) (bool, error) { func (s *Store) HasHostRecoveryCredential(hostID string) (bool, error) {
+37
View File
@@ -43,6 +43,43 @@ func TestHostRecoveryCredential_RoundTripUpsertAndAbsent(t *testing.T) {
} }
} }
// GetHostRecoveryMeta is the render path's accessor (hub v0.84.0). It returns username + set_at
// and, by CONSTRUCTION, cannot return the secret: neither HostRecoveryMeta nor the SELECT names the
// `secret` column, so there is no runtime assertion to write for that half — adding a Secret field
// would not fail this test, it would fail to compile at every call site that never asked for one.
// The runtime half asserted here is the metadata round-trip and the absent case.
func TestGetHostRecoveryMeta_MetadataOnly(t *testing.T) {
s := newTestStore(t)
if err := s.UpsertHost(&Host{HostID: "h1", CustomerID: "c1", APIKey: "k1"}); err != nil {
t.Fatalf("UpsertHost: %v", err)
}
// absent → (nil, nil), never an error the page has to special-case
m, err := s.GetHostRecoveryMeta("h1")
if err != nil || m != nil {
t.Fatalf("absent meta: got %+v / %v (want nil,nil)", m, err)
}
if err := s.SaveHostRecoveryCredential("h1", "root@pam", "s3cret-Aa1"); err != nil {
t.Fatalf("SaveHostRecoveryCredential: %v", err)
}
m, err = s.GetHostRecoveryMeta("h1")
if err != nil || m == nil {
t.Fatalf("GetHostRecoveryMeta: %+v / %v", m, err)
}
if m.HostID != "h1" || m.Username != "root@pam" {
t.Fatalf("meta mismatch: %+v", m)
}
if m.SetAt.IsZero() {
t.Fatal("SetAt did not parse — the card cannot render staleness without it")
}
// an unknown host is the absent case too, not an error
if m, err := s.GetHostRecoveryMeta("nope"); err != nil || m != nil {
t.Fatalf("unknown host: got %+v / %v (want nil,nil)", m, err)
}
}
func TestGetHostMgmtPlaneStates_ParsesHealMarker(t *testing.T) { func TestGetHostMgmtPlaneStates_ParsesHealMarker(t *testing.T) {
s := newTestStore(t) s := newTestStore(t)
if err := s.UpsertHost(&Host{HostID: "h1", CustomerID: "c1", APIKey: "k1"}); err != nil { if err := s.UpsertHost(&Host{HostID: "h1", CustomerID: "c1", APIKey: "k1"}); err != nil {
+77
View File
@@ -438,6 +438,14 @@ func (s *Server) hostDetailData(host *store.Host, r *http.Request) map[string]in
drBundle, _ := s.store.GetHostDRBundle(host.HostID) drBundle, _ := s.store.GetHostDRBundle(host.HostID)
escrow, _ := s.store.GetHostEscrow(host.HostID) escrow, _ := s.store.GetHostEscrow(host.HostID)
// v0.84.0 Console access — presence + username + set_at ONLY. GetHostRecoveryMeta cannot carry
// the secret (its query does not select the column); the plaintext reaches the operator solely
// through POST /hosts/{id}/reveal-recovery-credential.
recoveryMeta, err := s.store.GetHostRecoveryMeta(host.HostID)
if err != nil {
s.logger.Printf("[ERROR] host recovery meta %s: %v", host.HostID, err)
}
return map[string]interface{}{ return map[string]interface{}{
"WrapperDrift": wrapperDrift, "WrapperDrift": wrapperDrift,
"ReportedWrapperSHA": reportedWrapperSHA, "ReportedWrapperSHA": reportedWrapperSHA,
@@ -467,6 +475,20 @@ func (s *Server) hostDetailData(host *store.Host, r *http.Request) map[string]in
// v0.60.0 Part B: retained superseded escrow blobs (data-first — old passphrases stay // v0.60.0 Part B: retained superseded escrow blobs (data-first — old passphrases stay
// R-recoverable). Operator-only surface. // R-recoverable). Operator-only surface.
"SupersededEscrowCount": func() int { n, _ := s.store.CountSupersededEscrow(host.HostID); return n }(), "SupersededEscrowCount": func() int { n, _ := s.store.CountSupersededEscrow(host.HostID); return n }(),
// v0.84.0 break-glass Console access card. NEVER add a key holding the secret.
"RecoveryVaulted": recoveryMeta != nil,
"RecoveryUsername": func() string {
if recoveryMeta != nil {
return recoveryMeta.Username
}
return ""
}(),
"RecoverySetAt": func() time.Time {
if recoveryMeta != nil {
return recoveryMeta.SetAt
}
return time.Time{}
}(),
// v0.46.0 Diagnostics: pending log pulls + received/blocked bundles (72 h TTL). // v0.46.0 Diagnostics: pending log pulls + received/blocked bundles (72 h TTL).
"LogBundles": s.hostLogBundleRows(host), "LogBundles": s.hostLogBundleRows(host),
"CSRFToken": s.getCSRFToken(r), "CSRFToken": s.getCSRFToken(r),
@@ -511,6 +533,61 @@ func (s *Server) handleHostDeleteImpact(w http.ResponseWriter, r *http.Request,
}) })
} }
// handleHostRevealRecoveryCredential — POST /hosts/{id}/reveal-recovery-credential (v0.84.0).
// The operator-SESSION counterpart to the global-key API path (api/handler.go
// handleAdminGetRecoveryCredential), which stays untouched and remains the break-glass route for
// when this UI is itself unavailable — coupling it to the session layer would remove exactly the
// independence that makes it a fallback.
//
// POST, not GET, deliberately: it is the only way the ServeHTTP-level CSRF check applies, and a
// secret must not be retrievable by URL alone (prefetch, history, referrer).
//
// SECRET DISCIPLINE: the plaintext goes into the JSON response body and nowhere else — never the
// hub log, never the event message or details_json.
func (s *Server) handleHostRevealRecoveryCredential(w http.ResponseWriter, r *http.Request, hostID string) {
host, err := s.store.GetHost(hostID)
if err != nil {
s.logger.Printf("[ERROR] reveal recovery credential %s: %v", hostID, err)
http.Error(w, "Internal error", http.StatusInternalServerError)
return
}
if host == nil {
http.NotFound(w, r)
return
}
cred, err := s.store.GetHostRecoveryCredential(hostID)
if err != nil {
s.logger.Printf("[ERROR] reveal recovery credential %s: %v", hostID, err)
http.Error(w, "Internal error", http.StatusInternalServerError)
return
}
if cred == nil {
// A 404 is not an access — nothing was delivered, so nothing is recorded on the timeline.
s.logger.Printf("[INFO] reveal recovery credential %s: no credential vaulted", hostID)
http.Error(w, "No recovery credential vaulted for this host", http.StatusNotFound)
return
}
// Transparency by default, exactly as handleRequestLogTail does it: SaveEvent alone writes the
// customer-visible timeline row WITHOUT emailing anyone (no dispatcher call here, by design).
// An unbound host has no customer to tell — the [INFO] line below is then the only record.
if host.CustomerID != "" {
if _, err := s.store.SaveEvent(host.CustomerID, "recovery_credential_revealed", "info",
"Az üzemeltető lekérte a géped konzolos hozzáférési jelszavát (távoli hibaelhárítás).", "", "hub"); err != nil {
s.logger.Printf("[WARN] SaveEvent recovery_credential_revealed %s/%s: %v", host.CustomerID, hostID, err)
}
}
s.logger.Printf("[INFO] operator revealed break-glass console credential for host %s (user=%s, secret %d chars)",
hostID, cred.Username, len(cred.Secret))
w.Header().Set("Cache-Control", "no-store")
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]any{
"host_id": cred.HostID,
"username": cred.Username,
"password": cred.Secret,
"set_at": cred.SetAt.UTC().Format(time.RFC3339),
})
}
// handleHostDelete — POST /hosts/{id}/delete (v0.47.0 stale host removal). Gates, in order: // handleHostDelete — POST /hosts/{id}/delete (v0.47.0 stale host removal). Gates, in order:
// - unknown host → 404 // - unknown host → 404
// - ONLINE host → 409 unconditionally (host reports authenticate via GetHostByAPIKey; // - ONLINE host → 409 unconditionally (host reports authenticate via GetHostByAPIKey;
@@ -0,0 +1,360 @@
package web
// Scenarios AG (hub v0.84.0) — the break-glass Console access card + the operator-session reveal
// endpoint.
//
// SEAM DISCIPLINE: every test here drives RequireAuth(ServeHTTP), never a handler function
// directly. Scenario E (route ordering) is INVISIBLE to a handler-level test — the handler is
// correct and simply never runs — and that is the shape of the inert-seam defects on record.
//
// The load-bearing assertion is negative in the way that matters: the plaintext must not appear
// ANYWHERE in a rendered host page. revealCanary is deliberately distinctive so an accidental leak
// is greppable across the tree.
import (
"bytes"
"encoding/json"
"io"
"log"
"net/http"
"net/http/httptest"
"path/filepath"
"strings"
"testing"
"time"
"golang.org/x/crypto/bcrypt"
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
)
const revealCanary = "REVEAL-CANARY-9f2b7c"
// newRevealServer builds a hub with a live operator password (so RequireAuth + the ServeHTTP CSRF
// check are both armed) and a CAPTURED logger, so the "the secret never reaches the log" assertion
// has something to read.
func newRevealServer(t *testing.T) (*Server, *store.Store, *bytes.Buffer) {
t.Helper()
var logBuf bytes.Buffer
st, err := store.New(filepath.Join(t.TempDir(), "t.db"), log.New(io.Discard, "", 0))
if err != nil {
t.Fatalf("store.New: %v", err)
}
t.Cleanup(func() { st.Close() })
s := New(st, "", "", "test", 30*time.Minute, log.New(&logBuf, "", 0))
h, err := bcrypt.GenerateFromPassword([]byte("operator-pw"), bcrypt.MinCost)
if err != nil {
t.Fatal(err)
}
s.configPasswordHash = string(h)
return s, st, &logBuf
}
// newRevealSession mints a live operator session and returns (cookie, csrfToken).
func newRevealSession(t *testing.T, s *Server) (*http.Cookie, string) {
t.Helper()
s.sessionsMu.Lock()
s.sessions["sess-token-reveal"] = &hubSession{
expiresAt: time.Now().Add(time.Hour),
csrfToken: "csrf-token-reveal",
}
s.sessionsMu.Unlock()
return &http.Cookie{Name: "hub_session", Value: "sess-token-reveal"}, "csrf-token-reveal"
}
// seedRevealHost creates a host (optionally bound to a customer) with a vaulted credential.
func seedRevealHost(t *testing.T, st *store.Store, hostID, customerID, secret string) {
t.Helper()
if err := st.UpsertHost(&store.Host{HostID: hostID, CustomerID: customerID, APIKey: "k-" + hostID}); err != nil {
t.Fatal(err)
}
if secret != "" {
if err := st.SaveHostRecoveryCredential(hostID, "root@pam", secret); err != nil {
t.Fatal(err)
}
}
}
// serveReveal drives the REAL stack: RequireAuth → ServeHTTP → routing → handler.
func serveReveal(t *testing.T, s *Server, req *http.Request) *httptest.ResponseRecorder {
t.Helper()
rr := httptest.NewRecorder()
s.RequireAuth(http.HandlerFunc(s.ServeHTTP)).ServeHTTP(rr, req)
return rr
}
func countEvents(t *testing.T, st *store.Store, customerID, eventType string) int {
t.Helper()
evs, err := st.GetRecentEvents(customerID, 100)
if err != nil {
t.Fatal(err)
}
n := 0
for _, e := range evs {
if e.EventType == eventType {
n++
}
}
return n
}
// --- Scenario A: the rendered host page carries presence + username + set_at, NEVER the secret ---
// RED-PROOF A: add `data-secret="{{.RecoverySecret}}"` to the card and a "RecoverySecret" key
// holding cred.Secret to hostDetailData → the canary assertion below goes RED.
func TestReveal_A_PageNeverCarriesTheSecret(t *testing.T) {
s, st, _ := newRevealServer(t)
cookie, _ := newRevealSession(t, s)
seedRevealHost(t, st, "demo-felhom-8363b5", "demo-felhom", revealCanary)
req := httptest.NewRequest(http.MethodGet, "/hosts/demo-felhom-8363b5", nil)
req.AddCookie(cookie)
rr := serveReveal(t, s, req)
if rr.Code != http.StatusOK {
t.Fatalf("host page = %d, want 200", rr.Code)
}
body := rr.Body.String()
// THE load-bearing assertion — the whole response, attributes, comments, scripts and all.
if strings.Contains(body, revealCanary) {
t.Fatal("SECRET LEAK: the vaulted console password appears in the rendered host page")
}
for _, want := range []string{
"Console access",
"root@pam",
"/hosts/demo-felhom-8363b5/reveal-recovery-credential",
"Reveal",
} {
if !strings.Contains(body, want) {
t.Errorf("host page is missing %q", want)
}
}
}
// --- Scenario B: reveal delivers the secret, records exactly one event, and never logs it ---
// RED-PROOF B: delete the SaveEvent call in handleHostRevealRecoveryCredential → the event
// assertion goes RED.
func TestReveal_B_RevealDeliversAndAudits(t *testing.T) {
s, st, logBuf := newRevealServer(t)
cookie, csrf := newRevealSession(t, s)
seedRevealHost(t, st, "demo-felhom-8363b5", "demo-felhom", revealCanary)
req := httptest.NewRequest(http.MethodPost, "/hosts/demo-felhom-8363b5/reveal-recovery-credential", nil)
req.AddCookie(cookie)
req.Header.Set("X-CSRF-Token", csrf)
rr := serveReveal(t, s, req)
if rr.Code != http.StatusOK {
t.Fatalf("reveal = %d, want 200 (body %q)", rr.Code, rr.Body.String())
}
if got := rr.Header().Get("Cache-Control"); got != "no-store" {
t.Errorf("Cache-Control = %q, want no-store", got)
}
if ct := rr.Header().Get("Content-Type"); !strings.HasPrefix(ct, "application/json") {
t.Errorf("Content-Type = %q, want application/json", ct)
}
var got struct {
HostID string `json:"host_id"`
Username string `json:"username"`
Password string `json:"password"`
SetAt string `json:"set_at"`
}
if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v (body %q)", err, rr.Body.String())
}
if got.Password != revealCanary {
t.Errorf("password = %q, want the vaulted secret", got.Password)
}
if got.Username != "root@pam" || got.HostID != "demo-felhom-8363b5" {
t.Errorf("unexpected payload: %+v", got)
}
if got.SetAt == "" {
t.Error("set_at missing — the operator cannot judge staleness without it")
}
// Exactly ONE timeline row, on the bound customer, info severity, source "hub", Hungarian.
evs, err := st.GetRecentEvents("demo-felhom", 100)
if err != nil {
t.Fatal(err)
}
n := 0
for _, e := range evs {
if e.EventType != "recovery_credential_revealed" {
continue
}
n++
if e.Severity != "info" {
t.Errorf("severity = %q, want info", e.Severity)
}
if e.Source != "hub" {
t.Errorf("source = %q, want hub", e.Source)
}
if !strings.Contains(e.Message, "konzolos hozzáférési jelszavát") {
t.Errorf("message is not the Hungarian customer-facing line: %q", e.Message)
}
if strings.Contains(e.Message, revealCanary) || strings.Contains(e.DetailsJSON, revealCanary) {
t.Fatal("SECRET LEAK: the password is in the event row")
}
}
if n != 1 {
t.Errorf("recovery_credential_revealed rows = %d, want exactly 1", n)
}
// The hub log records the access — username + length only.
if strings.Contains(logBuf.String(), revealCanary) {
t.Fatal("SECRET LEAK: the password reached the hub log")
}
if !strings.Contains(logBuf.String(), "operator revealed break-glass console credential") {
t.Errorf("the access is not recorded in the hub log: %q", logBuf.String())
}
// No dispatcher call exists on this path — nothing was emailed.
notifs, err := st.GetRecentNotifications("demo-felhom", 100)
if err != nil {
t.Fatal(err)
}
if len(notifs) != 0 {
t.Errorf("reveal produced %d notification_log row(s); it must email nobody", len(notifs))
}
}
// --- Scenario C: nothing vaulted → the explanatory card, no control, 404 on POST, zero events ---
func TestReveal_C_NotVaulted(t *testing.T) {
s, st, _ := newRevealServer(t)
cookie, csrf := newRevealSession(t, s)
seedRevealHost(t, st, "sess-f-2670b5", "sess-f", "") // no credential row
req := httptest.NewRequest(http.MethodGet, "/hosts/sess-f-2670b5", nil)
req.AddCookie(cookie)
body := serveReveal(t, s, req).Body.String()
if !strings.Contains(body, "not vaulted") {
t.Error("the not-vaulted state does not render its badge")
}
if !strings.Contains(body, "byo host") {
t.Error("the not-vaulted state does not explain WHY (byo / step 4b)")
}
if strings.Contains(body, "/hosts/sess-f-2670b5/reveal-recovery-credential") {
t.Error("a Reveal control is offered for a host with nothing vaulted")
}
req = httptest.NewRequest(http.MethodPost, "/hosts/sess-f-2670b5/reveal-recovery-credential", nil)
req.AddCookie(cookie)
req.Header.Set("X-CSRF-Token", csrf)
rr := serveReveal(t, s, req)
if rr.Code != http.StatusNotFound {
t.Fatalf("reveal on an unvaulted host = %d, want 404", rr.Code)
}
// A 404 is not an access: only a DELIVERED secret is recorded.
if n := countEvents(t, st, "sess-f", "recovery_credential_revealed"); n != 0 {
t.Errorf("a 404 wrote %d event row(s); it must write none", n)
}
}
// --- Scenario D: the CSRF gate (security) ---
// RED-PROOF D: the companion below sends the token and asserts 200, proving this test
// discriminates on CSRF rather than passing because the request was malformed some other way.
func TestReveal_D_CSRFRequired(t *testing.T) {
s, st, _ := newRevealServer(t)
cookie, csrf := newRevealSession(t, s)
seedRevealHost(t, st, "demo-felhom-8363b5", "demo-felhom", revealCanary)
req := httptest.NewRequest(http.MethodPost, "/hosts/demo-felhom-8363b5/reveal-recovery-credential", nil)
req.AddCookie(cookie) // session, but NO CSRF token
rr := serveReveal(t, s, req)
if rr.Code != http.StatusForbidden {
t.Fatalf("reveal without CSRF = %d, want 403", rr.Code)
}
if strings.Contains(rr.Body.String(), revealCanary) {
t.Fatal("SECRET LEAK: the 403 body carries the password")
}
if n := countEvents(t, st, "demo-felhom", "recovery_credential_revealed"); n != 0 {
t.Errorf("a CSRF refusal wrote %d event row(s); it must write none", n)
}
// The discriminator: the SAME request WITH the token succeeds.
req = httptest.NewRequest(http.MethodPost, "/hosts/demo-felhom-8363b5/reveal-recovery-credential", nil)
req.AddCookie(cookie)
req.Header.Set("X-CSRF-Token", csrf)
if rr := serveReveal(t, s, req); rr.Code != http.StatusOK {
t.Fatalf("the same request WITH a CSRF token = %d, want 200 — the 403 above proves nothing", rr.Code)
}
}
// --- Scenario E: the method gate — and, through it, the ROUTE ORDER (seam test) ---
// RED-PROOF E: move the new case BELOW `case strings.HasPrefix(path, "/hosts/")` → the GET falls
// through to the catch-all, renders the host detail page 200, and this test goes RED.
func TestReveal_E_MethodGateAndRouteOrder(t *testing.T) {
s, st, _ := newRevealServer(t)
cookie, _ := newRevealSession(t, s)
seedRevealHost(t, st, "demo-felhom-8363b5", "demo-felhom", revealCanary)
req := httptest.NewRequest(http.MethodGet, "/hosts/demo-felhom-8363b5/reveal-recovery-credential", nil)
req.AddCookie(cookie)
rr := serveReveal(t, s, req)
if rr.Code != http.StatusMethodNotAllowed {
t.Fatalf("GET on the reveal route = %d, want 405", rr.Code)
}
// The route-order half: a fall-through to /hosts/ would render the host page instead.
if strings.Contains(rr.Body.String(), "Console access") {
t.Fatal("the reveal route fell through to the /hosts/ catch-all — the case is registered AFTER it")
}
if strings.Contains(rr.Body.String(), revealCanary) {
t.Fatal("SECRET LEAK: the 405 body carries the password")
}
}
// --- Scenario F: unknown host ---
func TestReveal_F_UnknownHost(t *testing.T) {
s, _, _ := newRevealServer(t)
cookie, csrf := newRevealSession(t, s)
req := httptest.NewRequest(http.MethodPost, "/hosts/does-not-exist/reveal-recovery-credential", nil)
req.AddCookie(cookie)
req.Header.Set("X-CSRF-Token", csrf)
if rr := serveReveal(t, s, req); rr.Code != http.StatusNotFound {
t.Fatalf("reveal on an unknown host = %d, want 404", rr.Code)
}
}
// --- Scenario G: unauthenticated ---
func TestReveal_G_Unauthenticated(t *testing.T) {
s, st, _ := newRevealServer(t)
seedRevealHost(t, st, "demo-felhom-8363b5", "demo-felhom", revealCanary)
req := httptest.NewRequest(http.MethodPost, "/hosts/demo-felhom-8363b5/reveal-recovery-credential", nil)
req.Header.Set("X-Requested-With", "XMLHttpRequest") // API-like → 401 rather than a login redirect
rr := serveReveal(t, s, req)
if rr.Code != http.StatusUnauthorized {
t.Fatalf("unauthenticated reveal = %d, want 401", rr.Code)
}
if strings.Contains(rr.Body.String(), revealCanary) {
t.Fatal("SECRET LEAK: the 401 body carries the password")
}
if n := countEvents(t, st, "demo-felhom", "recovery_credential_revealed"); n != 0 {
t.Errorf("an unauthenticated call wrote %d event row(s); it must write none", n)
}
}
// --- Edge case (§8): an UNBOUND host reveals fine and writes no event (SaveEvent needs a customer) ---
func TestReveal_UnboundHostRevealsWithoutAnEvent(t *testing.T) {
s, st, logBuf := newRevealServer(t)
cookie, csrf := newRevealSession(t, s)
seedRevealHost(t, st, "unbound-01", "", revealCanary)
req := httptest.NewRequest(http.MethodPost, "/hosts/unbound-01/reveal-recovery-credential", nil)
req.AddCookie(cookie)
req.Header.Set("X-CSRF-Token", csrf)
rr := serveReveal(t, s, req)
if rr.Code != http.StatusOK {
t.Fatalf("reveal on an unbound host = %d, want 200", rr.Code)
}
if !strings.Contains(rr.Body.String(), revealCanary) {
t.Error("the unbound host's secret was not delivered")
}
// No placeholder customer id is invented — the hub log is the only record.
if n := countEvents(t, st, "", "recovery_credential_revealed"); n != 0 {
t.Errorf("an unbound host wrote %d event row(s) against an empty customer id", n)
}
if !strings.Contains(logBuf.String(), "unbound-01") {
t.Error("the unbound host's reveal is recorded nowhere at all")
}
}
+9
View File
@@ -364,6 +364,15 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
} else { } else {
http.Error(w, "Method not allowed", http.StatusMethodNotAllowed) http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
} }
// v0.84.0 break-glass console credential — suffix route BEFORE the bare /hosts/ catch-all
// (registered after it, the POST would 404 and the GET would silently render the host page).
case strings.HasPrefix(path, "/hosts/") && strings.HasSuffix(path, "/reveal-recovery-credential"):
hostID := strings.TrimSuffix(strings.TrimPrefix(path, "/hosts/"), "/reveal-recovery-credential")
if r.Method == http.MethodPost {
s.handleHostRevealRecoveryCredential(w, r, hostID)
} else {
http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
}
case strings.HasPrefix(path, "/hosts/") && strings.HasSuffix(path, "/request-logs"): case strings.HasPrefix(path, "/hosts/") && strings.HasSuffix(path, "/request-logs"):
hostID := strings.TrimSuffix(strings.TrimPrefix(path, "/hosts/"), "/request-logs") hostID := strings.TrimSuffix(strings.TrimPrefix(path, "/hosts/"), "/request-logs")
if r.Method == http.MethodPost { if r.Method == http.MethodPost {
@@ -279,6 +279,96 @@
</div> </div>
</section> </section>
<!-- Console access (v0.84.0): the break-glass root@pam credential. The page carries
presence + username + set_at ONLY — the plaintext NEVER enters this document and is
fetched on demand from POST /hosts/{id}/reveal-recovery-credential. Deliberately NOT
the customer_unified data-secret widget, which embeds the plaintext on every load. -->
<section class="card">
<h2>Console access</h2>
{{if .RecoveryVaulted}}
<div class="info-grid">
<div class="info-item">
<span class="label">User</span>
<span class="value"><code>{{.RecoveryUsername}}</code></span>
</div>
<div class="info-item">
<span class="label">Password set</span>
<span class="value">{{timeAgo .RecoverySetAt}}</span>
</div>
</div>
<div class="credential-box">
<code id="console-pw-{{.HostID}}">&bull;&bull;&bull;&bull;&bull;&bull;&bull;&bull;&bull;&bull;&bull;&bull;&bull;&bull;&bull;&bull;</code>
<button type="button" class="copy-btn" id="console-reveal-{{.HostID}}" data-reveal-url="/hosts/{{.HostID}}/reveal-recovery-credential" onclick="revealConsolePassword('{{.HostID}}')">Reveal</button>
<button type="button" class="copy-btn" id="console-copy-{{.HostID}}" onclick="copyConsolePassword('{{.HostID}}')" disabled>Copy</button>
</div>
<p class="hint" id="console-hint-{{.HostID}}" style="color: var(--text-muted); font-size: 0.85rem; margin-top: 0.5rem;">
Break-glass credential for the PVE web console at https://&lt;host-ip&gt;:8006 (realm: Linux PAM standard authentication). Revealing it is recorded on the customer's event timeline. Last vaulted value — if root@pam was changed on the box without re-vaulting, this is stale.
</p>
{{else}}
<p><span class="badge badge-neutral">not vaulted</span></p>
<p class="hint" style="color: var(--text-muted); font-size: 0.85rem;">
No console credential is vaulted for this host. Expected for a byo host — the owner manages root@pam. Otherwise the installer's step 4b did not run; re-run felhom-host-install.sh, or set and vault one per the break-glass runbook &sect;5.
</p>
{{end}}
</section>
{{if .RecoveryVaulted}}
<script>
// Fetch-on-demand: the password exists in this document ONLY between a Reveal and the next
// mask, and only in a local variable — never in localStorage, a data- attribute or the URL.
// `|| {}` because the customer page renders this sub-template once per host: a plain
// re-assignment would drop a sibling card's mask timer on the floor.
var consolePwState = typeof consolePwState !== 'undefined' ? consolePwState : {};
var consolePwMask = '••••••••••••••••';
function maskConsolePassword(hostID) {
var st = consolePwState[hostID];
if (st && st.timer) { clearTimeout(st.timer); }
consolePwState[hostID] = null; // CLEARS the retained plaintext
var code = document.getElementById('console-pw-' + hostID);
if (code) { code.textContent = consolePwMask; }
var reveal = document.getElementById('console-reveal-' + hostID);
if (reveal) { reveal.textContent = 'Reveal'; }
var copy = document.getElementById('console-copy-' + hostID);
if (copy) { copy.disabled = true; }
}
function revealConsolePassword(hostID) {
if (consolePwState[hostID]) { maskConsolePassword(hostID); return; } // second click hides
var hint = document.getElementById('console-hint-' + hostID);
var code = document.getElementById('console-pw-' + hostID);
var btn = document.getElementById('console-reveal-' + hostID);
code.textContent = 'Revealing…';
// The endpoint comes from the button's data-reveal-url — the SAME string a render test
// asserts, so the assertion cannot pass while the fetch targets somewhere else.
fetch(btn.getAttribute('data-reveal-url'), {
method: 'POST',
headers: {'X-CSRF-Token': '{{.CSRFToken}}'}
}).then(function(r){
if (!r.ok) { throw new Error('HTTP ' + r.status); }
return r.json();
}).then(function(d){
code.textContent = d.password;
consolePwState[hostID] = {pw: d.password, timer: setTimeout(function(){ maskConsolePassword(hostID); }, 60000)};
document.getElementById('console-reveal-' + hostID).textContent = 'Hide';
document.getElementById('console-copy-' + hostID).disabled = false;
}).catch(function(e){
code.textContent = consolePwMask;
hint.textContent = 'Could not reveal the credential (' + e.message + '). The global-key curl path in the break-glass runbook §3.1 still works.';
});
}
function copyConsolePassword(hostID) {
var st = consolePwState[hostID];
if (!st) { return; }
var pw = st.pw;
if (navigator.clipboard) { navigator.clipboard.writeText(pw); }
maskConsolePassword(hostID); // re-mask after use
}
document.addEventListener('visibilitychange', function(){
if (document.visibilityState === 'hidden') {
for (var id in consolePwState) { if (consolePwState[id]) { maskConsolePassword(id); } }
}
});
</script>
{{end}}
{{if .Deletable}} {{if .Deletable}}
<!-- Danger zone (v0.47.0): rendered ONLY for non-online hosts — deleting a live <!-- Danger zone (v0.47.0): rendered ONLY for non-online hosts — deleting a live
host would brick its heartbeat channel, so the affordance never exists for one. host would brick its heartbeat channel, so the affordance never exists for one.