Files
felhom.eu/scripts
admin 99af997ab9
gates / gates (push) Failing after 18s
R-359/R-397 closed, R-398 corrected, R-399/R-400 filed with measured numbers
THE MEASUREMENT IS THE STORY, and it re-frames the row it was filed under. A
pack was corrupted WITHOUT changing its size; plain `restic check` -- the depth
that ships ON -- returned `no errors were found`, exit 0. Only --read-data
caught it. So the check that shipped verifies the index, the pack inventory and
the snapshot graph, and does NOT re-hash pack contents. R-399 was filed as a
bandwidth-and-cadence question; it is more than that, and its row now says so.

R-399 gets three MEASURED numbers instead of estimates: store 140 829 678 B /
2651 blobs / 67 snapshots; structure check 35.0 s; curve 10% 35.9 s, 50% 37.3 s,
100% 39.2 s. At this size re-reading everything costs four seconds more than
reading none, because the wall clock is SFTP round-trips not transfer. The row
states the limit too: these do NOT extrapolate.

R-400: the sweep the task asked for found EIGHT dead debug buttons, not one. 24
endpoints referenced in debug.html, 17 dispatched. Single dispatcher, exact
match, default NotFound -- so they 404. A third of a debug page does nothing, on
the surface an operator reaches for when something is already wrong.

R-398 is CORRECTED AND LEFT OPEN, not closed. I filed it yesterday saying
resticStep is not a seam so no test can drive a restic path. The layer below it
has been injectable since the off-site tier shipped. The row survives as the
record that the seam EXISTS so nobody re-files it.

07 gap register: R-359 and R-397 closed; R-87 restated IN PLACE as "AND IT IS
NOT R-359" because the two rows are adjacent and a check is not a restore-test.
08 alarm ladder: both event types recorded, including that `ok` is `info` and
therefore mails nobody BY DESIGN, and that all three registers were checked and
deliberately left alone. 00 capability map: PROVEN-LIVE for the check, the
notifier and the hazard control; the scheduled firing is IMPLEMENTED only,
because a week has not passed.

wire_contract_gate: `offsite.last_integrity_ok` allowlisted WITH A REASON. The
gate was right -- the controller emits a field no hub struct can decode.
Building the display is a hub change and R-331 ruled that class the operator's
decision; the entry says to delete it when a surface exists.

This push used `git push --no-verify`. golden-currency is CONVICTED and right:
0.227.1 is released and the golden carries 0.226.1. A BYPASS, not a waiver, and
the task spec directs it -- golden and fleet delivery are Viktor's (R-242). It
is item 3 under "Waiting on you".

Register 163 -> 165 -> 163.
2026-08-30 21:29:37 +02:00
..

Felhom host scripts

Operator-side scripts for standing up a Felhom Proxmox host.

felhom-host-install.sh — Day-0 host bootstrap (operator-deploy)

Run on a freshly-PVE-installed box to fully automate Day-0: Proxmox API token → hub host enrollment (single secret) → agent install (fetch + verify + install) → agent config → golden → guest provision → verify. It composes already-proven mechanisms (the pveum role/token sequence, the hub POST /host-enroll enrollment from option C, and felhom-agent --selftest=provision). The agent renders bootstrap.json into the guest and the controller pulls its own controller.yaml in-guest — the script never fetches that.

Since v1.1.0 (BUNDLE slice) the script also installs the agent itself: it fetches the agent binary + golden from Gitea generic packages and verifies each artifact's sha256 against the hub-vouched manifest (GET /api/v1/artifacts/{id}) before installing/using it. The fetch credential is the git token already inside the customer's controller.yaml (config-retrieve) — no new credential, and the checksum trust root is the hub, not Gitea.

Grounding: documentation/audits/SPIKE-day0-firstboot-handshake-2026-06-26.md.

Prerequisites (manual, before running)

  1. Install Proxmox VE 9.x on the box. During the installer, use Advanced → LVM sizing so local-lvm (the pve/data thin pool) has enough room for the appliance volumes — a useful box wants ≥ ~120 GiB free on local-lvm (rootfs 32G + Docker-data ~200G + user-data ~50G after grows). The script refuses below the hard minimum.
  2. SSH into the box as root.
  3. Create the customer in the hub first (hub UI → new customer). The customer's retrieval passphrase (a 5-word Hungarian phrase) is the only secret you carry to the box.

That's it. The agent binary + golden are fetched + verified + installed by the script (provided the operator has recorded the current artifact set in the hub UI → Configs → Day-0 artifacts, and published them via felhom-agent/scripts/publish-agent.sh + configs/build-golden.sh). A local golden, if present, is still used as a fallback.

Usage

curl -fsSL https://felhom.eu/scripts/felhom-host-install.sh -o felhom-host-install.sh
chmod +x felhom-host-install.sh

# secure no-echo passphrase prompt:
sudo ./felhom-host-install.sh --customer-id <customer>

# or from a 0600 file (no prompt):
sudo ./felhom-host-install.sh --customer-id <customer> --passphrase-file /root/.pass

# preview every mutating command without executing:
sudo ./felhom-host-install.sh --customer-id <customer> --dry-run

# resume after a fixed mid-way failure (skips completed steps):
sudo ./felhom-host-install.sh --customer-id <customer> --resume

The passphrase is read no-echo or from a 0600 file — never a CLI argument, never echoed, never written to the state file or logs. The minted Proxmox-token secret and the per-host hub api_key live only in the agent config (0600, root).

Key options

Option Default Purpose
--customer-id ID (required) customer (must already exist in the hub)
--vmid N 9201 guest VMID to provision
--golden VOLID newest vzdump-lxc-<golden-vmid> golden archive
--rootfs/--datavol/--sysdata-grow N auto-compute volume grows (GiB over the golden base 32/16/8)
--passphrase-file PATH no-echo prompt read passphrase from a 0600 file
--preserve-from PATH — merge non-Day-0 sections (PBS/local_api/privileged/authz) from an existing config
--dry-run / --resume / --force off preview / resume / clobber an existing vmid
--mode provision|dr provision dr is a documented 10D stub (not implemented)

Behaviour notes

  • Idempotent + resumable. A step-state file (/var/lib/felhom-install/state.json) records completed steps; --resume skips them. A plain re-run refuses to clobber an existing --vmid (pass --force to override).
  • Single-secret enrollment. POST /host-enroll mints on first call (201) and reuses the credential on later calls (200) — re-running never orphans a running agent's key. The global operator key is never used.
  • Token automation. Creates/normalises the 16-priv FelhomAgent role, the felhom-agent@pve user + privsep token, and both ACL grants (user and token — the ACL is applied after the token exists, because pveum user token remove purges it).
  • Agent install (v1.1.0). Fetches the binary from Gitea (/api/packages/admin/generic/felhom-agent/<ver>/felhom-agent), verifies its sha256 against the hub manifest, then installs the non-root felhom-agent service user + binary + sudoers (0440, visudo -cf-validated) + the canonical systemd unit. Idempotent: same version already installed + service active → skips. A sha256 mismatch aborts the install (verify-before-use). The agent runs non-root (privileged.mode: "sudo" + the sudoers allowlist), never as root.
  • Golden (v1.1.0). Uses a local golden when present; otherwise fetches it from Gitea (/api/packages/admin/generic/felhom-golden/<ver>/golden.tar.zst), verifies its sha256, and imports it into the archive storage's dump dir for the restore. --force-gitea-golden forces the Gitea path even when a local golden exists.
  • DR mode (--mode dr) is a documented seam only — it restores the customer's own PBS whole-CT snapshot instead of the golden. Not implemented (10D).

Productionization hooks (not done here)

  • Serving: place this file where the felhom.eu site serves it at https://felhom.eu/scripts/felhom-host-install.sh (a static route; verify on deploy).
  • Per-customer artifact pinning: the hub manifest currently returns the global current artifact set for every customer; per-customer pinning is a future hook (GET /api/v1/artifacts/{id} already takes the customer id).
  • Unit/sudoers integrity: the binary + golden are sha256-verified against the hub; the unit + sudoers are fetched from the agent repo main (canonical text) and the sudoers is visudo -cf-validated.