# felhom.eu — task reports

> **Overwrite** this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in [hub/CHANGELOG.md](hub/CHANGELOG.md); the scripts history lives in [scripts/CHANGELOG.md](scripts/CHANGELOG.md).

## TASK G1 — management-plane break-glass (hub + installer half) — hub v0.34.1 (2026-07-05)

**Baseline:** felhom.eu @ `2f97ce3` → `012e5f3`. Hub `0.33.0` → **`0.34.1`** (live via ArgoCD). Agent
half = felhom-agent v0.71.0. Prerequisite for the felhom-sshd OOB feature (H1). Provenance:
`documentation/audits/SPIKE-felhom-sshd-2026-07-05.md` §8/#9.

### Shipped
- **Break-glass credential vault** (`store.host_recovery` + `internal/store/host_recovery.go`): a
  per-host root@pam console password, stored at rest, operator-retrievable — the human fallback for
  reaching the PVE web console (pveproxy :8006, a failure domain distinct from sshd) when both the
  sshd path and the agent-independent auto-heal have failed. `PUT /hosts/{id}/recovery-credential`
  (SELF-scoped host key — day-0 vaults it) + `GET /admin/hosts/{id}/recovery-credential` (GLOBAL key
  only). Secret never logged (username + length only).
- **mgmt_plane surfacing** (`internal/monitor/host_mgmtplane.go`, 60s sweep): parses the agent's
  additive `mgmt_plane` stanza and raises `mgmt_plane_healed` WARNING on a new `privsep_healed_at`
  (a recurring `/run/sshd` clobber surfaces before it becomes a lockout; complements host_staleness).
  v0.34.1 fix: a heal is an EVENT — construction seeds pre-existing markers (startup false-alarm
  guard) but a newly-observed marker alerts, so the FIRST auto-heal surfaces.
- **host-install** (`scripts/felhom-host-install.sh`): `step_break_glass` generates a strong root@pam
  password (`openssl rand`, never logged/filed — stdin→chpasswd + stdin→curl), sets it, and vaults it
  via the host key; idempotent unless `--rotate-recovery`. Also installs the G1 host artifacts
  (tmpfiles + agent-independent watchdog timer), **RuntimeDirectory-guarded** (refuses any unit that
  declares it); uninstall removes all of them.

### Tests + red-proofs (all green: `go build/vet/test ./...`)
- store: recovery-credential round-trip + upsert + absent→nil; `GetHostMgmtPlaneStates` parses the
  marker + old-agent report degrades to empty.
- api: vault self-scoped (own 200, cross-host 403, unauth 401); operator read global-only (host key
  401, absent 404); **password-never-logged** (buffer-logger red-proof).
- monitor: first-heal-after-healthy alerts once; recurring heals each alert; pre-existing marker seeded
  silently; no-heal never alerts. Red-proofed: neutering the emit fails the alert test.

### Live validation (felhom-pve + hub)
- Auto-heal drill (agent stopped): `/run/sshd` removed → agent-independent watchdog healed it in
  **30.0 s**, new `:22` session restored with the agent still down.
- Chain: agent report `mgmt_plane` (healed_recently + timestamp) → hub raised `mgmt_plane_healed`
  warning (17:16:21).
- **Break-glass drill:** day-0 vault via the host key (200) → operator retrieval via the global key →
  the vaulted root@pam password authenticated to PVE (`POST /access/ticket` → 200 = opens the web
  console); a host key on the admin read path → 401 (operator-only). Secret never printed/logged.

### Notes
- **felhom-pve's root@pam password is now the G1-vaulted strong value** (the intended day-0 outcome);
  retrieve it via `GET /admin/hosts/demo-felhom-01/recovery-credential` with the operator key. CC's
  key-based SSH is unaffected.
- Keep the build-server PVE token fresh (the incident's secondary lesson); least-privilege console user
  + credential auto-rotation are noted future items.