THE MEASUREMENT IS THE STORY, and it re-frames the row it was filed under. A pack was corrupted WITHOUT changing its size; plain `restic check` -- the depth that ships ON -- returned `no errors were found`, exit 0. Only --read-data caught it. So the check that shipped verifies the index, the pack inventory and the snapshot graph, and does NOT re-hash pack contents. R-399 was filed as a bandwidth-and-cadence question; it is more than that, and its row now says so. R-399 gets three MEASURED numbers instead of estimates: store 140 829 678 B / 2651 blobs / 67 snapshots; structure check 35.0 s; curve 10% 35.9 s, 50% 37.3 s, 100% 39.2 s. At this size re-reading everything costs four seconds more than reading none, because the wall clock is SFTP round-trips not transfer. The row states the limit too: these do NOT extrapolate. R-400: the sweep the task asked for found EIGHT dead debug buttons, not one. 24 endpoints referenced in debug.html, 17 dispatched. Single dispatcher, exact match, default NotFound -- so they 404. A third of a debug page does nothing, on the surface an operator reaches for when something is already wrong. R-398 is CORRECTED AND LEFT OPEN, not closed. I filed it yesterday saying resticStep is not a seam so no test can drive a restic path. The layer below it has been injectable since the off-site tier shipped. The row survives as the record that the seam EXISTS so nobody re-files it. 07 gap register: R-359 and R-397 closed; R-87 restated IN PLACE as "AND IT IS NOT R-359" because the two rows are adjacent and a check is not a restore-test. 08 alarm ladder: both event types recorded, including that `ok` is `info` and therefore mails nobody BY DESIGN, and that all three registers were checked and deliberately left alone. 00 capability map: PROVEN-LIVE for the check, the notifier and the hazard control; the scheduled firing is IMPLEMENTED only, because a week has not passed. wire_contract_gate: `offsite.last_integrity_ok` allowlisted WITH A REASON. The gate was right -- the controller emits a field no hub struct can decode. Building the display is a hub change and R-331 ruled that class the operator's decision; the entry says to delete it when a surface exists. This push used `git push --no-verify`. golden-currency is CONVICTED and right: 0.227.1 is released and the golden carries 0.226.1. A BYPASS, not a waiver, and the task spec directs it -- golden and fleet delivery are Viktor's (R-242). It is item 3 under "Waiting on you". Register 163 -> 165 -> 163.
Felhom host scripts
Operator-side scripts for standing up a Felhom Proxmox host.
felhom-host-install.sh — Day-0 host bootstrap (operator-deploy)
Run on a freshly-PVE-installed box to fully automate Day-0: Proxmox API token →
hub host enrollment (single secret) → agent install (fetch + verify + install) →
agent config → golden → guest provision → verify. It composes already-proven mechanisms
(the pveum role/token sequence, the hub POST /host-enroll enrollment from option C, and
felhom-agent --selftest=provision). The agent renders bootstrap.json into the guest and
the controller pulls its own controller.yaml in-guest — the script never fetches that.
Since v1.1.0 (BUNDLE slice) the script also installs the agent itself: it fetches the
agent binary + golden from Gitea generic packages and verifies each artifact's sha256 against
the hub-vouched manifest (GET /api/v1/artifacts/{id}) before installing/using it. The fetch
credential is the git token already inside the customer's controller.yaml (config-retrieve) —
no new credential, and the checksum trust root is the hub, not Gitea.
Grounding: documentation/audits/SPIKE-day0-firstboot-handshake-2026-06-26.md.
Prerequisites (manual, before running)
- Install Proxmox VE 9.x on the box. During the installer, use Advanced → LVM
sizing so
local-lvm(thepve/datathin pool) has enough room for the appliance volumes — a useful box wants ≥ ~120 GiB free onlocal-lvm(rootfs 32G + Docker-data ~200G + user-data ~50G after grows). The script refuses below the hard minimum. - SSH into the box as root.
- Create the customer in the hub first (hub UI → new customer). The customer's retrieval passphrase (a 5-word Hungarian phrase) is the only secret you carry to the box.
That's it. The agent binary + golden are fetched + verified + installed by the script (provided the
operator has recorded the current artifact set in the hub UI → Configs → Day-0 artifacts, and
published them via felhom-agent/scripts/publish-agent.sh + configs/build-golden.sh). A local
golden, if present, is still used as a fallback.
Usage
curl -fsSL https://felhom.eu/scripts/felhom-host-install.sh -o felhom-host-install.sh
chmod +x felhom-host-install.sh
# secure no-echo passphrase prompt:
sudo ./felhom-host-install.sh --customer-id <customer>
# or from a 0600 file (no prompt):
sudo ./felhom-host-install.sh --customer-id <customer> --passphrase-file /root/.pass
# preview every mutating command without executing:
sudo ./felhom-host-install.sh --customer-id <customer> --dry-run
# resume after a fixed mid-way failure (skips completed steps):
sudo ./felhom-host-install.sh --customer-id <customer> --resume
The passphrase is read no-echo or from a 0600 file — never a CLI argument, never
echoed, never written to the state file or logs. The minted Proxmox-token secret and the
per-host hub api_key live only in the agent config (0600, root).
Key options
| Option | Default | Purpose |
|---|---|---|
--customer-id ID |
(required) | customer (must already exist in the hub) |
--vmid N |
9201 |
guest VMID to provision |
--golden VOLID |
newest vzdump-lxc-<golden-vmid> |
golden archive |
--rootfs/--datavol/--sysdata-grow N |
auto-compute | volume grows (GiB over the golden base 32/16/8) |
--passphrase-file PATH |
no-echo prompt | read passphrase from a 0600 file |
--preserve-from PATH |
— | merge non-Day-0 sections (PBS/local_api/privileged/authz) from an existing config |
--dry-run / --resume / --force |
off | preview / resume / clobber an existing vmid |
--mode provision|dr |
provision |
dr is a documented 10D stub (not implemented) |
Behaviour notes
- Idempotent + resumable. A step-state file (
/var/lib/felhom-install/state.json) records completed steps;--resumeskips them. A plain re-run refuses to clobber an existing--vmid(pass--forceto override). - Single-secret enrollment.
POST /host-enrollmints on first call (201) and reuses the credential on later calls (200) — re-running never orphans a running agent's key. The global operator key is never used. - Token automation. Creates/normalises the 16-priv
FelhomAgentrole, thefelhom-agent@pveuser + privsep token, and both ACL grants (user and token — the ACL is applied after the token exists, becausepveum user token removepurges it). - Agent install (v1.1.0). Fetches the binary from Gitea
(
/api/packages/admin/generic/felhom-agent/<ver>/felhom-agent), verifies its sha256 against the hub manifest, then installs the non-rootfelhom-agentservice user + binary + sudoers (0440,visudo -cf-validated) + the canonical systemd unit. Idempotent: same version already installed + service active → skips. A sha256 mismatch aborts the install (verify-before-use). The agent runs non-root (privileged.mode: "sudo"+ the sudoers allowlist), never as root. - Golden (v1.1.0). Uses a local golden when present; otherwise fetches it from Gitea
(
/api/packages/admin/generic/felhom-golden/<ver>/golden.tar.zst), verifies its sha256, and imports it into the archive storage's dump dir for the restore.--force-gitea-goldenforces the Gitea path even when a local golden exists. - DR mode (
--mode dr) is a documented seam only — it restores the customer's own PBS whole-CT snapshot instead of the golden. Not implemented (10D).
Productionization hooks (not done here)
- Serving: place this file where the felhom.eu site serves it at
https://felhom.eu/scripts/felhom-host-install.sh(a static route; verify on deploy). - Per-customer artifact pinning: the hub manifest currently returns the global current artifact
set for every customer; per-customer pinning is a future hook (
GET /api/v1/artifacts/{id}already takes the customer id). - Unit/sudoers integrity: the binary + golden are sha256-verified against the hub; the unit +
sudoers are fetched from the agent repo
main(canonical text) and the sudoers isvisudo -cf-validated.