From f6aed8294044fbe50591369f70ef786ab3c16984 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Wed, 29 Jul 2026 09:06:32 +0200 Subject: [PATCH] =?UTF-8?q?host-install=20v1.22.0=20=E2=80=94=20E-2=20Part?= =?UTF-8?q?=202:=20new=20boxes=20get=20a=20real=20backup=20target,=20or=20?= =?UTF-8?q?are=20told=20they=20do=20not?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every box installed before this got local_backup_target "local" -- the vzdump target on the SAME physical device as the guest, so a drive failure took the guest and its only local backup together. E-1 fixed two machines by hand; this fixes the installer. Case A: an eligible secondary drive is already mounted -> create felhom-backup on that drive's own mountpoint via the felhom-backup-target-apply wrapper (create + grant) and point the primary tier at it. Case B: system drive only -> the target stays on the system drive and this is RECORDED AS DEGRADED, not as normal. The install still succeeds: a single-drive appliance is a valid product, it just cannot survive drive loss. Phase 0 inverts the emphasis: the installer has NO drive-enrollment step, so on a fresh appliance Case A almost never fires. The common case is Case B with the drive arriving later through the wizard (Part 3). Case A covers the reinstall shape where an agent-generated .mount unit already brings the drive up by fs-UUID. Eligibility suggests and refuses the absurd, never decides by transport: the reference backup drive is an external USB HDD and BOTH demo boxes report removable=0, so a transport rule disqualifies the reference drive and a removable rule finds no candidate at all. Scenario F: an already-configured box is never corrected -- an early return plus setdefault, both load-bearing. Proofs (installer-logic-tested against extracted functions with stubbed pvesm/wrapper; NOT install-tested, no reinstall was performed): A -> create + grant, resolved felhom-backup B -> DEGRADED warnings, resolved local, rc=0 (install not failed) F -> skipped, 0 wrapper calls F red-proof (guard removed) -> 2 wrapper calls, i.e. it would have "corrected" a correct box --- scripts/CHANGELOG.md | 46 +++++++++++++++ scripts/felhom-host-install.sh | 104 ++++++++++++++++++++++++++++++++- 2 files changed, 147 insertions(+), 3 deletions(-) diff --git a/scripts/CHANGELOG.md b/scripts/CHANGELOG.md index 42e6d85..4ec3e53 100644 --- a/scripts/CHANGELOG.md +++ b/scripts/CHANGELOG.md @@ -1,3 +1,49 @@ +## v1.22.0 — E-2 Part 2: new boxes get a real backup target, or are told they do not (2026-07-29) + +Every box installed before this got `local_backup_target: "local"` — the vzdump target on the SAME +physical device as the guest, so a drive failure took the guest and its only local backup together. +E-1 fixed two machines by hand; this fixes the installer. + +**Case A** — an eligible secondary drive is already mounted: create `felhom-backup` on that drive's +own mountpoint through the new `felhom-backup-target-apply` wrapper (`create` + `grant`), and point +the primary tier at it. + +**Case B** — system drive only: the target stays on the system drive and this is **recorded as +DEGRADED, not as normal** — two warning lines saying it protects against corruption but not against +a disk failure, and what to do about it. **The install still succeeds**: a single-drive appliance is +a valid product, it just cannot survive drive loss. + +**Phase 0 reality check, which inverts the emphasis:** the installer has **no drive-enrollment +step** — `/mnt/felhom-drives` is agent-installed at runtime and drives are formatted/mounted through +the dashboard later. So on a fresh appliance **Case A almost never fires**; the common case is +Case B, with the drive arriving months later through the wizard (E-2 Part 3). Case A exists for the +reinstall/recovery shape, where an agent-generated `.mount` unit already brings the drive up by +fs-UUID. + +**Eligibility suggests and refuses the absurd; it never decides by transport.** The reference +hardware forbids that: demo-felhom's backup drive is an external **USB HDD**, and **both** demo +boxes' drives report `removable=0`. A transport rule would disqualify the reference drive; a +removable rule would find no candidate at all. The filter is: a real mounted `ext4/xfs/btrfs` under +`/mnt`, on a `/dev` source, not the root device, ≥ 64 GB. + +**SCENARIO F — an already-configured box is never "corrected".** `configure_backup_target` returns +early, changing nothing, when the storage id already exists; `setdefault` independently preserves an +existing box's whole `backup` block. Both guards are load-bearing and both were exercised. + +Also installs (and uninstalls) `/usr/local/sbin/felhom-backup-target-apply`, 0755 root:root, with +`bash -n` before install — the pbs-apply pattern. The wrapper is the single place the F-1/F-2 laws +live, so the installer and the runtime move cannot drift. + +**Proofs (harness against the extracted functions, stubbed pvesm/wrapper — installer-logic-tested, +NOT install-tested; no reinstall was performed):** + +| Scenario | Result | +|---|---| +| A — eligible drive | `create felhom-backup /mnt/bigdisk` then `grant felhom-backup`; resolved `felhom-backup` | +| B — single drive | DEGRADED warnings, resolved `local`, **rc=0 — install not failed** | +| F — configured box | skipped, **0 wrapper calls** | +| F red-proof (guard removed) | **2 wrapper calls** — it would have "corrected" a correct box | + ## felhom-host-install 1.21.0 — F-LEAK: the restore-test scratch band gets its own path-scoped grant (2026-07-28) Campaign 8's F-LEAK: a restore-test whose restore **fails** leaves a scratch guest the agent cannot diff --git a/scripts/felhom-host-install.sh b/scripts/felhom-host-install.sh index 30afaaf..226b233 100644 --- a/scripts/felhom-host-install.sh +++ b/scripts/felhom-host-install.sh @@ -184,7 +184,7 @@ set -euo pipefail -SCRIPT_VERSION="1.21.0" # the SINGLE version source (F-1): -h, the run banners, and the hub +SCRIPT_VERSION="1.22.0" # the SINGLE version source (F-1): -h, the run banners, and the hub # Setup-tab copy (hub internal/web/configs.go hostInstallVersion — # scripts/hostinstall_gates.py asserts the two stay equal) all follow it. # 1.16.0: the FELHOM_ESCROW sudoers alias (controller-driven escrow @@ -306,6 +306,10 @@ PVE_PRIVS_BASE="Sys.Audit SDN.Use Datastore.Audit" # Storages the agent reads/writes (archive+dump=local, restore=local-lvm, offsite DR=felhom-pbs). The # offsite felhom-pbs MUST be included or the agent's DR backup 403s (SPIKE residual #1). --acl-storages overrides. PVE_STORAGES=(local local-lvm felhom-pbs) +# E-2: the whole-guest backup target storage id, and what configure_backup_target resolved to. +# BACKUP_TARGET_RESOLVED feeds agent.json's local_backup_target; "local" means DEGRADED (Case B). +BACKUP_TARGET_ID="felhom-backup" +BACKUP_TARGET_RESOLVED="local" # F-LEAK (Campaign 8, v1.21.0): the restore-test's scratch VMID band, granted the GUEST role at each # /vms/ PATH. WHY THIS EXISTS: the Guest role is granted at /pool/felhom, and a guest only joins # that pool when its restore COMPLETES. A restore-test whose restore FAILS therefore leaves a scratch @@ -579,6 +583,80 @@ _grant() { run pveum acl modify "$path" -token "${PVE_USER}!${PVE_TOKENID}" -role "$role" } + +# ---- E-2 Part 2: the whole-guest backup target ------------------------------------------------- +# +# CASE A an eligible secondary drive is already mounted at install time -> create `felhom-backup` +# on that drive's OWN mountpoint (F-1), is_mountpoint 1 (F-2), grant the agent (F-3), and +# point the primary tier at it. +# CASE B system drive only -> the target stays on the system drive. This is RECORDED AS DEGRADED, +# not as normal: it protects against corruption and NOT against drive loss. The install +# still succeeds — a single-drive appliance is a valid product, it just cannot survive +# drive loss. +# +# PHASE-0 REALITY CHECK (E-2, 2026-07-29): the installer has NO drive-enrollment step — +# /mnt/felhom-drives is agent-installed at runtime and drives are formatted/mounted by the dashboard +# later. So on a FRESH appliance Case A almost never fires; the common case is Case B, and the drive +# arrives months later through the wizard (E-2 Part 3). Case A exists for the reinstall/recovery +# shape, where an agent-generated .mount unit already brings the drive up by fs-UUID. +# +# eligible_backup_drive — echoes the mountpoint of a usable secondary drive, or nothing. +# Attributes SUGGEST and REFUSE THE ABSURD; they never decide by transport. The reference hardware +# forbids that: demo-felhom's backup drive is an external USB HDD, and BOTH demo boxes' drives +# report removable=0 — a transport rule would disqualify the reference drive, a removable rule +# would find no candidate at all. +eligible_backup_drive() { + local root_dev mp dev sz + root_dev=$(findmnt -no SOURCE / 2>/dev/null || true) + # Only real, already-mounted filesystems under /mnt — never / , never a bare directory. + while read -r mp dev; do + [[ -n "$mp" && -n "$dev" ]] || continue + [[ "$mp" == /mnt/* ]] || continue + [[ "$dev" != "$root_dev" ]] || continue + [[ "$dev" == /dev/* ]] || continue # skip NAS/tmpfs/overlay sources + # Refuse the absurd: a thumb-drive-sized volume is not a whole-guest backup target. A guest + # archive measured 1.6-6.0 GB on the demo fleet, and keep-last=3 multiplies it. + sz=$(findmnt -nbo SIZE "$mp" 2>/dev/null || echo 0) + [[ "$sz" -ge $((64 * 1024 * 1024 * 1024)) ]] || continue + echo "$mp"; return 0 + done < <(findmnt -rn -o TARGET,SOURCE -t ext4,xfs,btrfs 2>/dev/null) + return 1 +} + +# configure_backup_target — Case A/B. Idempotent and SCENARIO-F SAFE. +configure_backup_target() { + # SCENARIO F, the hard rule: never touch a box that already has a target. The agent.json write + # below is already protected structurally (`setdefault` preserves an existing `backup` block + # WHOLE), and this is the second, explicit guard for the STORAGE half — `pvesm add` on an + # existing id is an error, and repointing a live target is exactly the silent retarget this + # whole arc closes. + if pvesm status --storage "$BACKUP_TARGET_ID" >/dev/null 2>&1; then + log_skip " backup target '$BACKUP_TARGET_ID' already exists — leaving it exactly as it is (Scenario F)" + BACKUP_TARGET_RESOLVED="$BACKUP_TARGET_ID" + return 0 + fi + local mp + if mp=$(eligible_backup_drive); then + # CASE A. Run through the SAME guarded wrapper the agent uses, so the installer and the + # runtime path cannot drift and the laws are enforced in exactly one place. + if $DRY_RUN; then + log_dry "felhom-backup-target-apply create $BACKUP_TARGET_ID $mp ; grant" + else + /usr/local/sbin/felhom-backup-target-apply create "$BACKUP_TARGET_ID" "$mp" \ + || die "backup target creation failed on $mp" + /usr/local/sbin/felhom-backup-target-apply grant "$BACKUP_TARGET_ID" \ + || die "backup target grant failed — every backup would 403" + fi + BACKUP_TARGET_RESOLVED="$BACKUP_TARGET_ID" + log_success " backup target: $BACKUP_TARGET_ID on $mp (separate device — survives a system-drive failure)" + else + # CASE B — degraded, and SAID SO. Not a failure; a single-drive appliance is valid. + BACKUP_TARGET_RESOLVED="local" + log_warn " backup target: DEGRADED — no eligible second drive, so the whole-system backup stays on the SYSTEM drive." + log_warn " It protects against file corruption but NOT against a disk failure. Attach a second drive and assign it in the dashboard." + fi +} + # apply_scoped_acl — create the 3 scoped roles and grant each at its path(s). Requires the pool to exist. # ORDER (3b-fix): Base (which holds box-wide Datastore.Audit) is ensured + granted BEFORE Store, so a # RE-APPLY on a live box adds Audit@/ before Store drops its per-storage Audit → the agent never loses @@ -936,6 +1014,7 @@ run_uninstall() { # 4d. Guarded-mkfs + pbs-apply wrappers, guest-hook snippet, lan-resolver dnsmasq snippets (drill R3-R5). if [[ -f /usr/local/sbin/felhom-mkfs-guarded ]]; then run rm -f /usr/local/sbin/felhom-mkfs-guarded; else log_skip " felhom-mkfs-guarded already absent"; fi if [[ -f /usr/local/sbin/felhom-pbs-apply ]]; then run rm -f /usr/local/sbin/felhom-pbs-apply; else log_skip " felhom-pbs-apply already absent"; fi + if [[ -f /usr/local/sbin/felhom-backup-target-apply ]]; then run rm -f /usr/local/sbin/felhom-backup-target-apply; else log_skip " felhom-backup-target-apply already absent"; fi if [[ -f /var/lib/vz/snippets/felhom-guest-hook.sh ]]; then run rm -f /var/lib/vz/snippets/felhom-guest-hook.sh; fi local dconf _dnsmasq_touched=false for dconf in /etc/dnsmasq.d/felhom-*.conf; do @@ -2026,6 +2105,21 @@ step_agent_install() { log_success " installed /usr/local/sbin/felhom-pbs-apply (0755, the guarded PBS-DR apply path)" fi + # E-2a: backup-target apply wrapper — the ONLY path the FELHOM_BACKUPTARGET sudoers alias permits. + # The agent cannot create a PVE storage (Datastore.Allocate at /storage) or grant an ACL + # (Permissions.Modify); both are deliberately outside its role, so the move runs through this + # fenced shim. Same pattern as pbs-apply: install BEFORE the sudoers, 0755 root:root, bash -n first. + if $DRY_RUN; then + log_dry "fetch configs/felhom-backup-target-apply ; bash -n ; install 0755 -> /usr/local/sbin/felhom-backup-target-apply" + else + local bttmp; bttmp=$(mktemp -t felhom-bt.XXXXXX) + fetch_raw "configs/felhom-backup-target-apply" "$bttmp" + bash -n "$bttmp" || { rm -f "$bttmp"; die "fetched felhom-backup-target-apply failed bash -n — refusing to install"; } + install -m 0755 -o root -g root "$bttmp" /usr/local/sbin/felhom-backup-target-apply + rm -f "$bttmp" + log_success " installed /usr/local/sbin/felhom-backup-target-apply (0755, the guarded backup-target path)" + fi + # Sudoers — fetch the canonical file, validate with visudo -cf BEFORE installing (0440 root:root). if $DRY_RUN; then log_dry "fetch configs/felhom-agent.sudoers ; visudo -cf ; install 0440 -> $AGENT_SUDOERS" @@ -2238,6 +2332,10 @@ EOF #------------------------------------------------------------------------------- step_agent_config() { log_step "6/8 agent config + service" + # E-2 Part 2: resolve the whole-guest backup target BEFORE the config is written — the resolved + # value feeds local_backup_target below. Scenario F safe: it returns early, changing nothing, when + # a target already exists, and `setdefault` preserves an existing box's whole backup block anyway. + configure_backup_target # TLS pin: the SERVED leaf cert fingerprint (not pvesh node info — may differ) local fp fp=$(echo | openssl s_client -connect 127.0.0.1:8006 2>/dev/null | openssl x509 -noout -fingerprint -sha256 2>/dev/null | sed 's/.*=//') @@ -2285,7 +2383,7 @@ step_agent_config() { ISLAND_ON="$ISLAND_ON" ISLAND_BRIDGE="$ISLAND_BRIDGE" ISLAND_GUEST_CIDR="$ISLAND_GUEST_CIDR" LAN_IP="$LAN_IP" \ OP_KEY_ID="$RESOLVED_OP_ID" OP_KEY_LINE="$RESOLVED_OP_LINE" \ REC_KEY_ID="$RESOLVED_REC_ID" REC_KEY_LINE="$RESOLVED_REC_LINE" \ - PRESERVE_FROM="$PRESERVE_FROM" INSTALL_MODE="$MODE" ENABLE_OOB="$ENABLE_OOB" OUT="$AGENT_CONFIG" python3 <<'PY' + PRESERVE_FROM="$PRESERVE_FROM" INSTALL_MODE="$MODE" ENABLE_OOB="$ENABLE_OOB" BACKUP_TARGET_RESOLVED="$BACKUP_TARGET_RESOLVED" OUT="$AGENT_CONFIG" python3 <<'PY' import json, os, sys out = os.environ['OUT'] base = {} @@ -2330,7 +2428,7 @@ base.setdefault('storage', {"watchdog_interval_seconds":5,"watchdog_debounce_sec # setdefault: an EXISTING box's backup block is preserved WHOLE. Upgrades never gain the tier here — # they are migrated explicitly (R-82 Slice D.2), so an in-place upgrade can never silently start # writing to an offsite datastore. -base.setdefault('backup', {"local_backup_target":"local","local_backup_retention":3,"restore_storage":"local-lvm","restore_test_cadence_seconds":0,"scratch_vmid_min":990000,"scratch_vmid_max":990009,"pbs_secret_dir":"/etc/pve/priv/storage","backup_cadence_seconds":0,"backup_targets":[{"target_id":"felhom-pbs","cadence_seconds":604800,"keep_last":2}]}) +base.setdefault('backup', {"local_backup_target":os.environ.get('BACKUP_TARGET_RESOLVED','local'),"local_backup_retention":3,"restore_storage":"local-lvm","restore_test_cadence_seconds":0,"scratch_vmid_min":990000,"scratch_vmid_max":990009,"pbs_secret_dir":"/etc/pve/priv/storage","backup_cadence_seconds":0,"backup_targets":[{"target_id":"felhom-pbs","cadence_seconds":604800,"keep_last":2}]}) base.setdefault('local_api', {}) base['local_api'].setdefault('enable', True) base['local_api']['listen_addr'] = os.environ['BRIDGE_ADDR']