90 lines
6.1 KiB
Markdown
90 lines
6.1 KiB
Markdown
# RUNBOOK — BYO deployment (a host Felhom does NOT own)
|
|
|
|
<!--
|
|
The path for a customer's own Proxmox host. Distilled from the first live BYO onboarding (Peti,
|
|
2026-07-10: 2-node HA cluster, 26 existing guests) — every gotcha below was hit or verified live.
|
|
The defining property of BYO: STRICT CONTAINMENT — capped guest resources, write access only on
|
|
storages the owner names, VM privileges only inside the felhom pool, full reversibility.
|
|
-->
|
|
|
|
## 0. Prerequisites
|
|
1. **Hub customer** created; **retrieval passphrase** delivered to the box owner over a secure channel
|
|
(typed at a no-echo prompt — never on the command line).
|
|
2. Cloudflare tunnel + API tokens on the customer config (the dashboard publishes on their domain).
|
|
3. (Optional, recommended) **Pre-enable offsite** on the customer config — the descriptor + one-time
|
|
password wait harmlessly, and day-0 wires offsite hands-off, answering the customer-egress
|
|
reachability question in the first report cycles.
|
|
4. Owner-side facts to collect up front (one command for them: `pvesm status`):
|
|
- **storage names** for `--acl-storages` (which storages may Felhom WRITE: minimum the archive storage
|
|
— `local` — and the guest-restore storage);
|
|
- is there an **LVM-thin** (`local-lvm`)? The guest restore target defaults to `local-lvm` and there is
|
|
currently NO flag to change it — a ZFS-only box needs the `--restore-storage` installer follow-up
|
|
first (open item);
|
|
- **cluster?** → node name (the installer REFUSES multi-node without an explicit `--node` — the
|
|
wrong-node footgun guard, verified live);
|
|
- agreed guest slice (cores/MiB — REQUIRED in byo mode) and vmid (default 9201; check `pct list`+
|
|
`qm list` for collisions — the owner may have 100+ guests).
|
|
|
|
## 1. The trust conversation (what the owner will ask — verified answers)
|
|
- **"Nothing changes on my system?"** Additive-only, and provable: (a) `--dry-run` prints every mutating
|
|
command with zero execution; (b) the real install prints the full host-mutation list and requires typed
|
|
consent; (c) write containment: `Datastore.Allocate/AllocateSpace` ONLY on the storages they name
|
|
(verifiable: `pveum user permissions felhom-agent@pve --path /storage/<any>`), read-only Audit
|
|
elsewhere (the monitor needs to SEE all storages — by design); (d) VM privileges exist only at
|
|
`/pool/felhom` — their existing guests are permission-unreachable; (e) no network/bridge/sysctl/HA/
|
|
storage.cfg changes; (f) `--uninstall` reverts fully.
|
|
- **Root password:** the installer ROTATES root@pam to a generated recovery credential vaulted to the hub
|
|
— tell the owner BEFORE the install, not after they're locked out of the PVE UI.
|
|
- The agent runs non-root behind a fixed-argument sudoers allowlist; disk formatting exists only for
|
|
drives the owner explicitly enrolls (type-to-confirm; data-bearing devices refused without an
|
|
operator signature).
|
|
|
|
## 2. Preflight (read-only — installs nothing, not even the agent)
|
|
```
|
|
curl -fsSL https://felhom.eu/scripts/felhom-host-install.sh -o felhom-host-install.sh \
|
|
&& sudo bash felhom-host-install.sh --customer-id <ID> --mode byo \
|
|
--cores <N> --memory <MiB> --node <node> --acl-storages "local <restore-storage> [...]" --preflight-only
|
|
```
|
|
Reads PVE version, storages, RAM, guest count; DIES with the exact remedy on findings (missing ACL
|
|
storage names, foreign DNS resolver on :53 — refuse-over-mutate). Multi-storage syntax: ONE quoted,
|
|
space-separated argument.
|
|
|
|
## 3. Install
|
|
Same command minus `--preflight-only`; passphrase at the prompt; typed consent after the mutation list.
|
|
If it fails mid-way: fix the finding and re-run with **`--resume`** (idempotent; keep the full corrected
|
|
argument list, e.g. the widened `--acl-storages`). Permission 403s at the restore step = the ACL list is
|
|
missing the storage named in the error → `--rescope-acl --acl-storages "<corrected list>"`, then `--resume`
|
|
(live-verified recovery path).
|
|
|
|
## 4. Cluster rules (verified live — hand these to the owner in writing)
|
|
- The Felhom guest is **node-local by design**: agent, guest hook (agent-installed under the node's
|
|
`local:snippets`), and physical/NAS drive binds all live on the install node.
|
|
- **Never add the guest to HA resources; never migrate it.** Four node-locality layers break on a foreign
|
|
node: binds (empty), agent (unreachable → controller shows the WRONG node's storage data), guest hook
|
|
(missing → the guest fail-closed refuses to start — a feature), and the controller's fixed
|
|
agent-endpoint. Everything self-heals on migrating back.
|
|
- The cluster itself is untouched: pveum entries replicate via pmxcfs (inert), corosync/HA/quorum are
|
|
never modified.
|
|
|
|
## 5. Verify (remote, hub-side — no SSH exists to a BYO box, by design)
|
|
Host + guest appear on the hub within a cycle; controller at the floor version (updates are floor-driven
|
|
and, since v0.112.0, credential-free — the Git Sync fields are ONLY for private catalogs); tunnel
|
|
Healthy; dashboard live on the customer domain. If offsite was pre-enabled: report shows
|
|
`offsite: pending` — proof the bridge consumed the one-time password and installed its key from the
|
|
CUSTOMER's egress. Then → RUNBOOK-escrow-ceremony (note the K prerequisite there: a BYO box without the
|
|
PBS tier needs agent ≥ v0.80.0 identity-only mode).
|
|
|
|
## 6. Data storage for apps (BYO owners rarely hand over disks)
|
|
Preferred: **NAS share (Hálózati tárhely)** — NFS or SMB, mounted host-side under `/mnt/felhom-drives/`,
|
|
role-gated bulk-userdata, automount. NFS recipe (live-verified): a DEDICATED export (never reuse their
|
|
backup share — `all_squash` would break its other clients), options
|
|
`rw,no_subtree_check,insecure,all_squash,anonuid=101000,anongid=101000`, directory owned `101000:101000`
|
|
(container uid 1000 + the unprivileged-LXC +100000 offset — the agent source documents this as the
|
|
export's job). **NFSv4 path = pseudo-root-relative** (`/export/foo` on the server → enter `/foo`).
|
|
UI uid field: the app's container uid (1000). Physical drive enrollment stays available if they dedicate
|
|
an empty disk.
|
|
|
|
## 7. Rollback
|
|
`--uninstall` (guest destroyed, agent + config + .bak purged, optional `--remove-golden`) — the promise
|
|
from Step 1, kept.
|