felhom.eu/scripts/CHANGELOG.md

SPIKE 2 (no code change) — before-network gives a zero window by construction; locked root closes sulogin (2026-07-31)

No script, profile or build file was modified. Findings: documentation/audits/SPIKE-universal-iso-2-2026-07-31.md. Opens R-148..R-151.

Both Tier 0 boxes went offline mid-session (remote site, 12:28 CEST; four routes tried, our tailscale pod healthy). Q1/Q2/Q3 each retain a part that needs a nested VM and those parts are BLOCKED, not answered. The fixture is staged with Spike 1's two defects already corrected.

Established without them:

• STRUCTURAL — ordering = "before-network" maps to proxmox-first-boot-network-pre.service (proxmox-first-boot 9.2.5), which is Before=network-pre.target + Type=oneshot. It therefore runs to completion before any interface is configured — a credential rotation there has a zero-length window by construction, not by being fast. Stronger than the timing number Spike 1 failed twice to obtain.
• STRUCTURAL — the stub does not need fully-up. stub-first-boot.sh contains no pvesh/pct/pveum/qm call (grep rc=1); that usage is in felhom-bootstrap.sh, which runs under its own After=network-online.target ... pveproxy.service unit. So answer.toml.tmpl:27 justifies the current ordering with a dependency that does not exist. R-148.
• The ordering enum has THREE values — before-network, network-online, fully-up. Spike 1 recorded two; the middle one has never been evaluated. R-149.
• MECHANISM (container, not PVE) — a locked root closes sulogin: Cannot open access to console, the root account is locked. for both * and !, with a working control (correct password -> SU_OK, wrong -> Authentication failure). So "discard" and "lock" are the SAME outcome for recovery, which makes the escrow decision binary.
• All four proxmox-first-boot-* units are Condition*-guarded, and a failed condition is a SKIP, not a failure — a hook that never ran looks exactly like one that succeeded. R-150.
• Q4: the single-entry menu is enforced by four gates in iso-repack.sh (:153-155, :156, :160-164, :314-319), so restoring an interactive entry is a gate change, not a template edit — but the banned-token gate would pass unchanged for a stock Graphical entry. default/timeout are already settable in grub/grub.cfg.tmpl.

The first mechanism probe was invalid (ran as uid 0, so su never invoked pam_unix; sulogin had no tty) and is recorded as failed, not massaged.

SPIKE (no code change) — a universal ISO needs a different disk strategy and a locked root (2026-07-31)

No script, profile or build file was modified. Findings only: documentation/audits/SPIKE-universal-iso-2026-07-31.md. Opened R-139..R-147; restated R-128.

Three results that constrain any future universal-ISO spec:

• A disk filter matching more than one device does not fail safe. Observed in a nested VM: with filter.ID_SERIAL_SHORT = "SPIKE*" matching two disks and filesystem = "ext4", the installer silently picked one and wiped it (disk0 hash changed, disk1 byte-identical). validate-answer accepts such an answer. The filter did not match any devices guard covers the ZERO-match case only. R-139.
• No udev property distinguishes an internal system disk from external media. Measured on demo-felhom with its 1 TB external drive attached: ID_BUS is ata for both (the external is a SATA drive in a USB caddy), lsblk RM is 0 for both, and device-info exposes no removability property at all. On demo-hp the NVMe carries no ID_BUS or ID_TYPE whatsoever. Nothing expresses role. R-141 context.
• A root credential is mandatory in the answer schema, but a usable one is not. root-password-hashed = "*" validates and installs to completion (guest powered off after 200 s with reboot-mode = "power-off"). [first-boot].ordering also accepts before-network, which is the only ordering that could close the exposure window structurally — today's bootstrap unit is After=network-online.target pveproxy.service and the pairing poll waits indefinitely. R-141.

Also: prepare-iso leaves grub.cfg byte-identical to stock (15 menu entries, automated and interactive), so a two-entry boot menu is purely a change to Felhom's own grub.cfg.tmpl — the single-entry menu is Felhom's repack, not a constraint of ISO preparation.

v1.22.0 — E-2 Part 2: new boxes get a real backup target, or are told they do not (2026-07-29)

Every box installed before this got local_backup_target: "local" — the vzdump target on the SAME physical device as the guest, so a drive failure took the guest and its only local backup together. E-1 fixed two machines by hand; this fixes the installer.

Case A — an eligible secondary drive is already mounted: create felhom-backup on that drive's own mountpoint through the new felhom-backup-target-apply wrapper (create + grant), and point the primary tier at it.

Case B — system drive only: the target stays on the system drive and this is recorded as DEGRADED, not as normal — two warning lines saying it protects against corruption but not against a disk failure, and what to do about it. The install still succeeds: a single-drive appliance is a valid product, it just cannot survive drive loss.

Phase 0 reality check, which inverts the emphasis: the installer has no drive-enrollment step — /mnt/felhom-drives is agent-installed at runtime and drives are formatted/mounted through the dashboard later. So on a fresh appliance Case A almost never fires; the common case is Case B, with the drive arriving months later through the wizard (E-2 Part 3). Case A exists for the reinstall/recovery shape, where an agent-generated .mount unit already brings the drive up by fs-UUID.

Eligibility suggests and refuses the absurd; it never decides by transport. The reference hardware forbids that: demo-felhom's backup drive is an external USB HDD, and both demo boxes' drives report removable=0. A transport rule would disqualify the reference drive; a removable rule would find no candidate at all. The filter is: a real mounted ext4/xfs/btrfs under /mnt, on a /dev source, not the root device, ≥ 64 GB.

SCENARIO F — an already-configured box is never "corrected". configure_backup_target returns early, changing nothing, when the storage id already exists; setdefault independently preserves an existing box's whole backup block. Both guards are load-bearing and both were exercised.

Also installs (and uninstalls) /usr/local/sbin/felhom-backup-target-apply, 0755 root:root, with bash -n before install — the pbs-apply pattern. The wrapper is the single place the F-1/F-2 laws live, so the installer and the runtime move cannot drift.

Proofs (harness against the extracted functions, stubbed pvesm/wrapper — installer-logic-tested, NOT install-tested; no reinstall was performed):

Scenario	Result
A — eligible drive	create felhom-backup /mnt/bigdisk then grant felhom-backup; resolved felhom-backup
B — single drive	DEGRADED warnings, resolved local, rc=0 — install not failed
F — configured box	skipped, 0 wrapper calls
F red-proof (guard removed)	2 wrapper calls — it would have "corrected" a correct box

felhom-host-install 1.21.0 — F-LEAK: the restore-test scratch band gets its own path-scoped grant (2026-07-28)

Campaign 8's F-LEAK: a restore-test whose restore fails leaves a scratch guest the agent cannot destroy — 403 missing privilege VM.Allocate — so a half-restored guest holds its disks until a human removes it. The cause is structural, not a missing privilege in the role: FelhomAgentGuest is granted at /pool/felhom, and a guest only joins that pool when its restore completes. A failed restore therefore produces a guest that exists, is in no pool, and is out of the token's reach.

apply_scoped_acl now also grants FelhomAgentGuest at each /vms/<id> in PVE_SCRATCH_VMID_MIN..PVE_SCRATCH_VMID_MAX (990000–990009, the band the restore-test already picks from), to both the user and the token — the privsep intersection rule applies here as everywhere.

This is not a widening. PVE ACLs are path-scoped: ten explicit /vms/<id> grants authorise the agent on exactly those ten IDs. Granting at /vms was rejected — that would authorise destroying every guest on the box, including a co-tenant's. Verified live:

path	VM.Allocate
/vms/990000 (scratch band)	yes — 13 privs
/vms/100, /vms/9999	no — 3 privs (base only)
/vms/990010 (one past the band)	no — 3 privs

Two supporting changes, both load-bearing:

• remove_scoped_acl deletes the band grants before the role delete. PVE refuses to delete a role still referenced by any ACL, so omitting this would have broken the uninstall.
• step_verify asserts the band grants. A missing one is otherwise invisible until a restore-test fails, which is precisely the case that leaked a guest.

The alternative that does not work, recorded so it is not retried: adopting the stranded guest into the pool first. It was implemented (agent v0.107.0), shipped, and refuted live on 2026-07-28 — PUT /pools/felhom with vms=<id> also requires VM.Allocate on /vms/<id>, so Pool.Allocate cannot bootstrap its own membership. That code was removed in agent v0.108.0.

felhom-host-install 1.20.0 — R-82: a fresh box defaults to local-daily + offsite-weekly (2026-07-26)

The backup defaults gain backup_targets: [{target_id: "felhom-pbs", cadence_seconds: 604800, keep_last: 2}] — the offsite tier, weekly, two weeks retained (operator ruling 2026-07-26).

setdefault semantics preserved, and PROVEN both ways against the exact line:

FRESH   -> backup_targets: [{"target_id":"felhom-pbs","cadence_seconds":604800,"keep_last":2}]
UPGRADE -> preserved verbatim, no backup_targets

That second line is the load-bearing one: an in-place upgrade can never silently start writing to an offsite datastore. Existing boxes are migrated explicitly (R-82 Slice D.2), which is what keeps a routine host-install re-run from enrolling a box into offsite backups nobody asked for.

Why the tier is written before its storage exists: felhom-pbs only appears when the hub provisions the DR tier. The agent (v0.102.0) DEFERS a tier whose target storage is absent, so the tier stays silent until it is real and goes live with no restart the moment it is provisioned. Writing it here means a box is never left with a DR tier that is provisioned but unscheduled — the exact "applied and empty" state R-82 exists to end.

Felhom scripts — Changelog

felhom-host-install v1.19.0 — R-50 island control plane is the appliance default (2026-07-25)

Implements Phase A of the R-50 island bridge (spike GO, documentation/audits/SPIKE-island-bridge-2026-07-25.md). A fresh appliance install is now born immune to F1 (a LAN/DHCP/site move can no longer take the control plane down). byo is unchanged; an explicit --bridge-ip or --no-island keeps the LAN bind.

• New ensure_island_bridge() creates a portless host-internal bridge vmbr9 (169.254.253.1/30), idempotently, before the agent binds it. Portless = no physical port, so the /30 cannot collide with any customer LAN and survives any renumber. vmbr0 untouched.
• step_agent_config (appliance default): local_api.listen_addr → 169.254.253.1:8443; writes local_api.island_bridge/island_guest_addr (the agent ≥ 0.96.0 attaches the guest's net1 from these); and pins lan_resolver.host_ip to the LAN IP explicitly — Finding-1: without it the agent derives the DNS listen-addr from listen_addr and silently moves LAN DNS onto the island.
• --no-island opt-out; --bridge-ip still overrides. configs/felhom-localapi-firewall.example (agent repo) rewritten — the portless island bind IS the LAN close.
• Coupling: this install REQUIRES agent ≥ 0.96.0 (older agents ignore the island fields → no net1 → broken control plane). Vouch 0.96.0 before island installs go live. Hub hostInstallVersion synced to 1.19.0 (F-1). Existing boxes migrate via documentation/runbooks/RUNBOOK-island-migration.md.

ISO train v1.25.0 — the belt goes fleet-wide, the repo goes free, the screen learns ő, fresh boxes boot at the floor (2026-07-23)

Four rulings from 2026-07-23. host-install 1.17.0 → 1.18.0, build-felhom-iso 1.24.0 → 1.25.0, felhom-bootstrap R-63 font.

• Part 1 (F9) — the OOB belt is a DEFAULT appliance leg. felhom-host-install.sh: appliance mode now installs the H1 belt (felhom-sshd + static inet felhom_oob table + felhom-op) unless --no-oob; byo still refuses --enable-oob (owner-consented, deliberate exclusion — flagged, unchanged). The config writer now sets oob.enabled=true when the belt is installed, so the agent activates it (claims a port, renders the sshd config, fills the belt sets) instead of the units sitting inert — the pre-v1.25.0 state that left the belt unusable on both fleet boxes (07-22 power-outage audit F9). byo config asserts unchanged (they already refuse oob.enabled). The belt is lockout-SAFE with no operator peer yet (empty sets + no key = no access granted, :22 untouched). Live: installed + validated end-to-end on felhom-pve (operator WG peer → wg-felhom → belt → felhom-op login PROVEN) and installed on demo-hp (same infra; login is the same mechanism).

• Part 2 (apt) — enterprise → no-subscription on fresh appliance installs. New idempotent align_apt_repos (appliance-only; runs before the first apt op): disables the PVE/ceph enterprise .sources (deb822 Enabled: no) or legacy .list, adds pve-no-subscription.sources (suite+keyring derived from the enterprise file), apt-get update must exit clean. Scoped to the Proxmox-managed repos only; the owner's lists are never touched; no upgrades run.

• Part 3 (R-63) — the console learns ő/ű. felhom-bootstrap.sh loads a Latin-2 console font (Lat2-Terminus16 → Lat2-Fixed16 → Lat2-Terminus14) ONCE before the first paint (idempotent, best-effort — never blocks boot). Fixes the R-59 network screen (élő/telepítő/ellenőrizze) and the pairing banner (képernyő/teendő) rendering ő as blanks. Lat2 fonts ship in the trixie/PVE base, so no copy rewording was needed.

• Part 4 (R-71 mitigation) — every ISO build asserts golden ≥ managed floor. build-felhom-iso.sh gains ver_ge + assert_golden_ge_floor (runs before workspace/mint; dies with both versions on golden < floor; FELHOM_ASSERT_GOLDEN/FELHOM_ASSERT_FLOOR inputs; unset warns loudly, never a silent pass). Red-proofed both directions (floor>golden → exit 1; golden≥floor → OK). Standing rule 5 added to publish-train-rules.md. Closes the day-0 update-vs-bridge race window at its source (DIAG-f10 / R-71). ISO_VERSION → 1.25.0.

  Golden lift EXECUTED (2026-07-23): golden republished at controller 0.161.0 (build-golden.sh on felhom-pve), published to Gitea (618 MB, sha 77624408…, served-copy verified), operator-vouched (golden 0.153.0→0.161.0; floor stays 0.156.0) — the LIVE golden<floor gap is CLOSED. v1.25.0 ISO built (felhom-pve-9.2-1-v1.25.0-nested-vm-generic-mkimage.iso, sha 99b38420…) with the gate passing live; rootpw sibling + R-63 font present. Nested drill = the remaining validation leg.

ISO train v1.24.0 — the box that can never call home learns to say so, heal itself, and let you in (R-59 + R-60 + R-61) (2026-07-22)

All three findings are from the HP t740 install (2026-07-21). Proven end-to-end on a nested wrong-NIC install (spike + virgin-ISO drill: documentation/audits/SPIKE-firstboot-nic-sweep-2026-07-22.md).

• R-59 — the silent unreachable box gets a legible screen. A no-DHCP install bakes the installer's 192.168.100.2 fallback as a static vmbr0 config (with a dead gateway 192.168.100.1) and completes; the box then waited silently forever. The first-boot bootstrap now runs a network gate before mode dispatch (scripts/iso/felhom-bootstrap.sh:251 network_gate, called at :510): hub unreachable → physical-NIC diagnosis (name/MAC/carrier/speed; NICs are raised first — unused ports sit admin-DOWN with unreadable carrier), the fallback signature named when present, and a calm Hungarian console screen (paint_network_screen, :230 — the print_pairing_banner pattern) with the NIC table and the one-line remedy; then retry every 60 s with the unit held in activating (waiting is not failing — the v1.21.0 lesson, unit TimeoutStartSec=infinity). The trigger is always hub-unreachable, never the fallback signature: the drill proved a second wrong-NIC variant (the installer picks the NIC that LEASES and bakes the lease as static — a later cable move leaves a plausible-looking dead config with no signature). Recorded deviation: R-59 ships as this first-boot refuse-loudly gate, not an install-time abort — the fallback is baked inside the Proxmox auto-installer itself, unreachable without an installer-initrd hook (out of scope by operator ack, noted on the ROADMAP row).
• R-60 — first-boot NIC sweep self-heal. While the hub is unreachable AND no install attempt has begun (state.json gate; the unit's ConditionPathExists=!done-flag confines the whole script to pre-install anyway), the gate sweeps (sweep_nics, scripts/iso/felhom-bootstrap.sh:178): per candidate NIC, re-point vmbr0's bridge-ports (atomic tmp+mv, always rendered from the pristine copy) → ifreload → flush vmbr0's addresses/routes → bounded dhclient (20 s) → hub probe (any HTTP status counts — the hub answers / with 302). The flush is load-bearing: the baked fallback default route survives a successful lease otherwise (dhclient-script never replaces an existing default route) and the probe rides the dead gateway — found live in the drill. First success persists bridge-ports <winner> + iface vmbr0 inet dhcp (original kept as interfaces.felhom-bak) and proceeds; total failure restores the config byte-identically. Drill: cable moved → healed + registered at the hub in well under a minute, unaided.
• R-61 slice 1 — the baked root password becomes knowable. build-felhom-iso.sh writes the minted throwaway plaintext to a 0600 sibling <iso>.rootpw.txt (password + ISO name + build date; scripts/iso/build-felhom-iso.sh:254) — the single record of truth. Never stdout/logs; the manifest carries only a pointer (:383). A fixed well-known password stays REJECTED (operator ruling 2026-07-21). Drill: the emitted password's hash matches the installed box's /etc/shadow — the console lockout is closed.
• Harness (scripts/iso/test/bootstrap-modes.sh): caught up with the v1.21.0 in-script wait (the old pairing scenarios would hang), plus four gate scenarios — hub-reachable = ZERO new behavior (no ip/ifreload/dhclient/sleep call, fixture untouched), sweep-success (persist + bak + proceed), sweep-fail (screen + byte-identical config), state-file-present (sweep NEVER invoked). New test/rootpw-emission.sh for R-61 (emission, 0600, plaintext↔answer-hash cross-check, manifest guard). Four red-proofs run and restored: manifest-plaintext leak, restore-on-failure dropped, state gate removed, hub-reachable early-return removed — each failed exactly its guarding assertion.
• Assistant image gains python3 (the bootstrap's JSON parsing needs it; PVE ships it on real boxes — the harness runs the bootstrap in the image). build-felhom-iso.sh gains the FELHOM_ISO_KEEP_WORK=1 debug/test escape and resolves OUT_ISO before the mint (the rootpw sibling is named after the ISO).

build-hub.sh v1.23.0 — the hub build script enters the repo (2026-07-21)

It never was in one. /mnt/5_hdd/felhom.eu/build/felhom-hub/build.sh lived only in a DooPlex build directory — unversioned, unreviewed, un-diffable and gone with the box, while being the only way the hub image gets built. Recorded as a follow-up in felhom.eu/REPORT.md §6; adopted here.

Adopted verbatim — the build logic is byte-identical (verified by diff), and the only additions are a SCRIPT_VERSION constant echoed in the banner and a header naming the canonical/operative split. The build-dir path is now a symlink to this file, so there is one source of truth and a repo edit is live immediately (the install_skills.py pattern). The previous copy is retained beside it as build.sh.pre-adopt-2026-07-21.

It already carried the GitOps deploy hint fixed earlier in the week (manifest bump → hard-refresh → deliberate sync, and the named trap that kubectl set image / kubectl apply are silently reverted by the next ArgoCD sync) — that wording now lives in git along with the rest of it.

install_skills.py — cross-platform (POSIX symlink / Windows junction) (2026-07-19)

Claude Code now runs on DooPlex (Debian 13), where mklink /J does not exist — the script would have fallen through to COPY mode on every run, silently breaking the "repo edits are live immediately" property that makes felhom.eu/skills/ the source of truth.

• Link creation is now platform-dispatched behind os.name == "nt": os.symlink(..., target_is_directory=True) on POSIX, the existing mklink /J on Windows. Copy-mode fallback, idempotency, and the re-run warning are unchanged.
• remove_existing() had a real hazard on POSIX: os.rmdir() fails on a symlink-to-directory, which would have fallen through to shutil.rmtree() — and rmtree following a symlink into skills/ would have deleted the repo's own skill sources. It now unlinks symlinks explicitly before any rmdir/rmtree path is reached. Existence checks use os.path.lexists() so a broken link is seen and replaced rather than ignored.
• Human-facing output says "symlink" or "junction" per platform.

build-felhom-iso.sh v1.22.0 — the boot screen is ours, and it offers exactly one thing (R-38) (2026-07-19)

A boot menu is a product surface, and ours was Proxmox's. Every ISO is now repacked after prepare-iso to carry a Felhom boot screen and — the half that actually matters — a menu with exactly one entry.

• The safety half. The stock PVE menu offers Install Proxmox VE (Graphical), (Terminal UI), a serial variant, and an Advanced Options submenu holding two nomodeset entries, three debug entries, Rescue Boot, memtest and UEFI Firmware Settings. Every one of those reaches the manual installer, whose first question is which disk to wipe. A customer — or their helpful nephew — must not be able to get there from a boot menu they are staring at. Those entries are not hidden and not password-gated: they are not emitted. What ships is „Felhom telepítés", default, 5 s.
• Boot behavior is unchanged. The linux/initrd lines are lifted verbatim at repack time from the ISO's own Install Proxmox VE (Automated) entry rather than frozen into a copy in this repo, so a PVE bump that moves the kernel path or edits the append line tracks automatically instead of silently diverging. The build fails if they cannot be found, if the append line has lost proxmox-start-auto-installer, or if auto-installer-mode.toml is absent — that last one because without it the single Felhom-labelled entry would boot a manual installer, which is the precise outcome this change exists to prevent.
• Gates, and then a re-check against the shipped artifact. The rendered menu is asserted to have exactly 1 menuentry, 0 submenus and no live reference to proxtui/proxdebug/nomodeset/Rescue Boot/memtest/fwsetup (comments are stripped first — the template's header names the dropped entries on purpose). Then the menu and the theme background are read back out of final.iso, not out of the extract tree, so "the branding landed in the image we ship" is proven rather than assumed.
• The boot card. grub/generate-grub-background.sh letterboxes website/assets/og-image_2.png onto a 1024×768 gfxterm canvas at repack time (ImageMagick, added to the assistant image), so the boot screen has one source — the website asset — and not a second pre-rendered PNG in the repo to drift. The card's own subtle background grid is continued across the letterbox fill phase-locked to where the card's grid lands, so the fill is seamless instead of a 500 px square of grid floating in flat navy. The generator refuses a source whose geometry no longer matches the measured grid constants — a silently swapped asset would misplace every line, and that only shows up on a boot screen nobody re-checks.
• Menu positioning needs a theme. Plain background_image cannot move the menu off the wordmark, so grub/felhom-theme.txt (gfxmenu) puts it in the lower third the layout deliberately leaves empty, with the entry optically centered under the card (measured off a canary screenshot — the comment records the measurement). No "e: edit entry" hint label: advertising the editor works against a single-entry menu.
• mkimage-surgery.sh → iso-repack.sh. Branding and the slice-B loader swap need the same extract → modify → re-master cycle, so they now share one pass instead of re-mastering twice. The mkimage recipe is untouched. The embedded module list is still derived from the stock grub.cfg (snapshotted before branding rewrites it, so branding cannot shrink the embedded set), plus gfxmenu's bitmap/bitmap_scale/trig renderer dependencies.
• --no-brand exists for debugging the stock menu and says so loudly in the build log and the manifest. It is not a shipping mode.

Live-validated (nested canary, UEFI/OVMF, PVE 9.2-1). GRUB renders the card; the single entry is selected and counts down in Hungarian (accents render correctly under gfxterm); it auto-fires at 0; the automatic installer runs and reaches the match-nothing abort exactly as in v1.21.0 (ERROR: Installation failed: filter did not match any device → Installation aborted), touching no disk. Both shipping ISOs rebuilt (see REPORT-iso-branding-2026-07-19.md for shas).

Not touched (deferred): the installer squashfs/initrd theme — the post-GRUB screens are still Proxmox-branded. GRUB's e/c keys also remain reachable (suppressing them needs a superuser password, which is a separate decision); the menu no longer advertises them.

build-felhom-iso.sh v1.21.0 — the pairing wait stops looking like a failure (R-33) (2026-07-18)

Waiting is not failing, and must not be reported as failure. Until now the PAIRING poll loop WAS systemd's Restart=on-failure/RestartSec=30: each invocation did register-if-needed plus exactly one poll, then exited non-zero so the unit would be restarted 30s later. Functionally fine — but every one of those exits made systemd print Failed to start Felhom host bootstrap … on the physical console the customer is watching. The 2026-07-18 N100 rehearsal measured 52 FAILED lines in ~11 minutes while nothing whatsoever was wrong: the box had registered correctly and was waiting to be bound, which is the normal state of a freshly installed appliance. That is the customer's literal first impression of the product (documentation/tests/VALIDATION-n100-rehearsal-2026-07-18.md finding F6, ROADMAP R-33).

• felhom-bootstrap.sh — the wait moved INSIDE the script. run_pairing() is now a while true loop that sleeps POLL_INTERVAL (30s — the hub-side rate is unchanged) between polls, so the unit sits in activating instead of failing. Registration was split out into register_appliance(), which returns non-zero for a transient problem (no network yet, identity not gatherable, hub returned no token) and is simply retried by the loop rather than taking the whole unit down. Three cadence constants at the top of the file: POLL_INTERVAL=30, BANNER_EVERY=10 (console banner every 5 min), HEARTBEAT_EVERY=20 (journal heartbeat every 10 min).
• The journal got quiet without going dark. A 204 not bound yet is logged once on entry (worded so nobody reads it as an error) and then only on the 10-minute heartbeat with elapsed minutes. 404 and unexpected HTTP codes likewise degrade to the heartbeat instead of a line every 30s. 410 still exits non-zero on purpose — the delivery was consumed but no local env exists, a genuine crash window where handing the box back to systemd for a clean restart is the right move.
• The console banner got calmer and less repetitive. Printed every 5 min instead of every cycle (a customer may walk up at any time, but a banner every 30s is its own noise). Dropped the belt-and-braces parositasra var / párosításra vár double-spelling for a single accented line, and added the reassurance the rehearsal showed was missing: „Ez a képernyő magától frissül — nincs teendő a doboznál, és nyugodtan itt hagyhatod bekapcsolva."
• felhom-bootstrap.service — TimeoutStartSec=infinity (load-bearing, not cosmetic). A Type=oneshot ExecStart is killed at DefaultTimeoutStartSec (90s). Without this line systemd would kill the new in-script wait after 90 seconds and Restart=on-failure would silently reinstate the exact console spam this change removes — a regression that would look like the feature working for the first three polls. Restart=on-failure/RestartSec=30 are deliberately kept: they still cover the DIRECT path, a failed host-install, and the 410 crash window.
• Behaviour verified, not assumed. Driven in a throwaway Debian container against a stub hub that answers 204 five times and then delivers: the script logged the wait once plus one heartbeat, never exited between polls, then consumed the delivery, wrote the 0600 env, fell through to the direct install in the same invocation and exited 0. Under the old design the same sequence produced five separate unit invocations and five Failed to start console lines.

Compatibility: hub-side polling rate, endpoints, payloads and the one-shot delivery semantics are all unchanged; this is purely how the box waits and what it says while waiting. An older ISO against any hub keeps working exactly as before.

build-felhom-iso.sh v1.20.0 — console pairing-code banner for customer self-bind (R-27 slice 1) (2026-07-17)

Supports the hub's customer self-bind flow (hub v0.66.0). In PAIRING mode, felhom-bootstrap.sh now reads the additive pairing_code from the POST /api/v1/appliance/register response, persists it at /etc/felhom/appliance-pairing-code, and prints a Hungarian console banner (to /dev/console, stdout fallback) each pairing cycle so the customer can read the code off the physical screen and type it — together with their retrieval passphrase — on the hub's public /bind/<token> page. The code is non-secret (possession proof only; the passphrase is the second factor), so it is safe on the console. Graceful degradation both ways: a hub older than v0.66.0 omits pairing_code → the banner prints nothing and register/poll are unchanged; an old ISO against a v0.66.0 hub simply ignores the new field. No change to DIRECT mode. Green: bash -n clean on both scripts.

build-felhom-iso.sh v1.19.0 — the universal secret-free ISO: --pairing mode (R-21 slice C) (2026-07-17)

The scripts half of the universal ISO. felhom-bootstrap.sh gains a PAIRING mode — one unit, two modes, decided by the env:

• DIRECT (env has FELHOM_CUSTOMER_ID + FELHOM_RETRIEVAL_PASSPHRASE): the slice-A path, byte-identical — only branched around. Scenario D regression proves the pairing code is provably NOT entered (a fake hub records ZERO /appliance/* calls).
• PAIRING (generic ISO, no customer/passphrase baked in): gather identity (SMBIOS uuid + physical MAC set + SSH host pubkeys + hw), POST /api/v1/appliance/register → persist the appliance token (0600), then ONE GET /api/v1/appliance/poll per invocation (the existing Restart=on-failure/RestartSec=30 IS the poll timer — no long-running-oneshot timeout). On the bind's 200 delivery, WRITE the delivered credentials into the env (0600) and fall through to the DIRECT path — so every later retry is a plain direct install (the delivery is one-shot; a second poll → 410). Delivery-received secrets + the appliance token are shredded on host-install success.
• build-felhom-iso.sh --pairing builds the GENERIC ISO: no --bootstrap-env, a secret-free env carrying only the hub URL, -generic filename marker, manifest mode: pairing + secret-bearing: no. Direct mode (secret-bearing) is unchanged. New profiles/generic.profile.
• Validated: bash -n + shellcheck clean; the test/bootstrap-modes.sh harness (Scenario D + pairing register/poll + the delivery→env→host-install handoff) all green in a debian container.

build-felhom-iso.sh v1.18.0 — firmware loader option --loader shim|mkimage (R-21 slice B, F1) (2026-07-17)

Closes N100 finding F1 (HIGH): cheap AMI (AN3PLUS 0.01-class) UEFI firmware can't relocate the ISO's stock signed GRUB from USB (relocation 0x0 is not implemented yet; SB-off + shim-bypass don't help). The run's live grub-mkimage workaround is now a first-class pipeline mode.

• --loader shim|mkimage (default shim; profile-settable via FELHOM_LOADER, --loader wins). shim = the stock output, byte-for-byte unchanged (the surgery is skipped) — keeps the MS-signed shim→GRUB chain and Secure Boot on compliant firmware (S2b). mkimage replaces the ISO's UEFI boot path with a monolithic grub-mkimage loader built from the ISO's own GRUB modules (module set from the ISO's grub.cfg; embedded search --fs-uuid → configfile the real menu). The recipe is reproduced from the N100 run evidence, not re-derived.
• mkimage-surgery.sh (new): runs in the assistant container after prepare-iso, so the answer/first-boot payload is provably untouched except the loader. Swaps the loader into the ISO9660 tree (real lowercase Rock-Ridge path) and inside the efi.img ESP (FAT), then re-masters with xorriso preserving the hybrid BIOS boot (grub2-mbr + El Torito) and the GPT EFI System Partition, dropping only the irrelevant Apple HFS+/APM map (which trips xorriso's overlap check on re-emit).
• Loud + unmistakable (rule 4): build banner, manifest loader: + grub-mkimage: fields, and a -mkimage filename suffix when non-default. mkimage ISOs are unsigned → Secure Boot must be OFF on the target board.
• Dockerfile.assistant: adds grub-common + grub-efi-amd64-bin + mtools + dosfstools (grub 2.12 == the PVE 9.x ISO's 2.12-9+pmx2 generation). New profiles/n100.profile (cheap-board, FELHOM_LOADER="mkimage", SB-off note).
• Validated (RUNBOOK-B legs, nested VM 311 on felhom-pve): leg 1 shim boots + zero-touch installs under OVMF Secure Boot enforcing (+ SeaBIOS boot — hybrid path survived); leg 2 --loader mkimage boots + installs under OVMF SB-off (the monolith's embedded fs-uuid search found the ISO + chained its menu); leg 3 (red-proof) the mkimage ISO under SB-enforcing FAILS — firmware Access Denied, no boot; leg 4 the surgery is surgical (only the loader + regenerated El Torito artifacts differ; the full install payload is byte-identical). bash -n + shellcheck clean. Physical N100 closure folds into the supervised rehearsal (an n100-safety match-nothing ISO built
  • sha-recorded, unbooted, for an optional zero-risk pre-flight).

felhom-tenantsync.sh v1.1.0 — deprovision op (customer RESET teardown) (2026-07-17)

Adds the {"op":"deprovision","customer_id":"<id>"} op the slice-1 header explicitly reserved ("namespace/data deletion is a deliberate, separate decision"). It is exactly that deliberate, hub-side ack-gated decision (the customer RESET, hub v0.61.0): delete the token (its ACLs purge with it) → delete the residual namespace ACLs → destroy the namespace AND all its backup groups (proxmox-backup-client namespace delete <ns> --delete-groups true, via the transient admin token). IDEMPOTENT — a missing token / namespace is success (deleted:false), so a re-run after a partial reset converges. The shared felhom@pbs user is NEVER touched (co-tenants ride it). Returns {"status":"ok","namespace","datastore","deleted":<bool>}. Secret hygiene unchanged (no secrets in this path). Client seam: tenantsync.Deprovision(ctx, customerID) (existed bool, err error).

felhom-host-install.sh v1.17.0 — appliance guest auto-sizing (F5) + doc-drift fix (2026-07-17)

Closes VALIDATION-n100-baremetal-2026-07-16.md F5 (MEDIUM): appliance mode provisioned the golden default (2 GB RAM) on a 16 GB host and never surfaced the --memory/--cores caps. Now, in appliance mode with no explicit cap, the guest is auto-sized from the host (LXC limits are cheap → err generous): RAM = clamp(host-4096, min 4096, max host-2048) then a hard ceiling host-1024 (never over-commit); cores = host-1, min 2. An explicit --memory/--cores ALWAYS wins untouched; byo mode still requires explicit caps (never auto-sized). Host reads (MemTotal/nproc) are overridable via FELHOM_FAKE_MEMTOTAL_MIB/FELHOM_FAKE_NPROC for the mode harness. Sizing table: 8 GB→4096, 16 GB→12288, 32 GB→28672 MiB; a 4 GB host→3072 (min capped at host-1024). Red-proof: hostinstall-mode-harness.sh new F5 section (8/16/32 GB + small-host edge + explicit-flag precedence) — pre-fix (auto-size disabled) FAILs, post-fix PASSes. Same commit (R-16 doc-drift leftover): the operator-signing-keys "EMPTY by default" comment was stale (the keys are PINNED to the real felhom-op-1/felhom-rec-1 ceremony keypairs) — corrected, comment-only, no behavior.

felhom-poke.sh v1.0.0 — NEW (agent-plane immediate-sync, Direction-2a, 2026-07-16)

The offsite endpoint's THIRD hub forced-command surface (documentation/runbooks/offsite-endpoint.md §11): a NON-root (felhom-peersync, no sudoers grant — a datagram needs no privilege) forced command that reads a box's WireGuard /32 from $SSH_ORIGINAL_COMMAND, validates it to 10.77.0.0/24, and sends ONE contentless (empty) UDP datagram from wg0 to <ip>:51822. That is the last hop of the hub→ep0→box "sync now" poke; the box's felhom-agent (v0.89.0) poke listener then runs an immediate desired-state cycle. Contentless + confined (the WG kernel independently refuses non-peer /32s — SPIKE-immediate-sync-transport-2026-07-16 P1 EKEYREJECTED). Port 51822 is a shared cross-repo constant (felhom-agent internal/poke.Port, hub/internal/poke, this script's POKE_PORT). Hub client: hub/internal/poke. Installed + live-validated on ep0 2026-07-16 (poke-fired; non-WG target and arbitrary SSH command both refused non-WG target — the forced command never executes client input). Peersync/tenantsync surfaces untouched.

felhom bare-metal ISO pipeline v1.16.0 — NEW (R-21 slice A, 2026-07-16)

New operator tooling under scripts/iso/: a reproducible DooPlex pipeline that turns the official PVE ISO into a Felhom auto-install ISO whose first-boot chain unattended-fetches-and-runs felhom-host-install.sh until the host is enrolled + a guest provisioned. felhom-host-install.sh is unchanged (invoked, not modified). Tagged v1.16.0 to align the ISO with the current host-install release (they must stay coherent; the ISO's baked invocation targets that flag surface).

• Dockerfile.assistant (assistant paired to the ISO's PVE generation by Debian codename + shellcheck), build-felhom-iso.sh (render answer from answer.toml.tmpl + a profile, mint a fresh THROWAWAY root hash per build, gate validate-answer on OUTPUT TEXT not $? — spike S1 exit-0-on-failure trap, prepare-iso --fetch-from iso --on-first-boot, emit ISO + sha256 + manifest).
• In-ISO chain: stub-first-boot.sh (dumb, exactly-once, from-iso/fully-up) installs felhom-bootstrap.sh + felhom-bootstrap.service (retry-forever Type=oneshot+Restart=on-failure)
  • a 0600 bootstrap.env. Retry-vs-resume encoded once (plain first; --resume when /var/lib/felhom-install/state.json exists — safe per host-install v1.11.3).
• Secret-bearing: the unattended install needs the customer retrieval passphrase, so the ISO embeds it (the distribution URL itself carries no secret). Supervised/single-use; the bootstrap scrubs the env on success. Slice C removes the secret from the distributed ISO via claim-pairing.
• Validated on nested VM 310: build gate + red-proof; disk-filter fail-safe through the pipeline; stub→unit→real public-channel fetch→invocation→retry; resume decision; exactly-once; no-net retry + recovery. Terminal host-install success (hub-enrolled + guest-provisioned) is operator-gated (drill customer needs the password-gated create-UI; live-DB insert unsafe). Full report: root REPORT.md, documentation/audits/SPIKE-baremetal-iso-2026-07-16.md.

felhom-host-install v1.16.0 — FELHOM_ESCROW rides the canonical sudoers (2026-07-13)

Companion to agent v0.88.0 + controller v0.127.0 (the controller-driven escrow ceremony wizard). No new installer STEP: the FELHOM_ESCROW sudoers alias (the wizard's fixed-argv sudo self-invocation grant) ships via the existing canonical-sudoers fetch (configs/felhom-agent.sudoers from the agent repo, visudo -cf-gated before install) — the bump documents that every ≥1.16.0 install/update carries it. Hub hostInstallVersion bumped in the same commit (hostinstall_gates.py asserts equality); the hub deploy rides the next hub train (display-only lag, accepted). EXISTING hosts get the refreshed sudoers with the agent-0.88.0 deploy (ship-configs step), not from this script.

manifest_bearer_gate.py — new gate: no bearer-shaped literals in manifests/ (2026-07-13)

Part of the hub v0.53.0 bearer de-git. Greps every manifests/*.yaml for 64-hex literals (the openssl rand -hex 32 shape all felhom bearer/API keys use), comments included; exit 1 on any hit. felhom.secret.yaml's pre-existing committed secrets (umami APP_SECRET is 64-hex) print as a visible non-fatal KNOWN-BACKLOG line until that file's own de-git (secrets.md backlog). Run after ANY manifests/ change. Named "bearer" not "secret" — the repo .gitignore *secret* pattern silently un-tracks any "secret"-named file (learned the hard way: the gate's first landing never reached git). Red-proven: reintroducing a 64-hex literal → exit 1.

felhom-host-install v1.15.0 — DR-tier-by-default: uniform DR plumbing on every install (2026-07-12)

Sweeps the DRILL-day0-vm-2026-07-12 installer findings so the next Day-0 runs the drill's §5 sequence with zero fix-and-continue stops. Operator decisions 2026-07-12: capability is BAKED uniformly; activation is a hub flag; WG is base infrastructure.

• F-7 — ships /usr/local/sbin/felhom-pbs-apply (fetch configs/felhom-pbs-apply from the agent repo, bash -n, install 0755 — the mkfs/selfupdate wrapper pattern). The FELHOM_PBSDR sudoers alias already shipped; the 3 pbsdr-* capabilities are no longer born DEGRADED. Uninstall removes it (disclosure↔uninstall parity kept, harness GL4-D green).
• F-10 — installs the age package (escrow-ceremony identity-wrap dependency; a fresh-box ceremony died on it live).
• F-9 / decision 5 — the rendered agent.json now defaults wg_tunnel.enabled: true in BOTH modes (WG is base infrastructure like cloudflared, also the future OOB transport; setdefault — a deliberately-disabled preserved config is respected). The byo wg-off config assert is RETIRED; byo consent = the typed disclosure ack (the WG tunnel + pbs-apply wrapper + age are now on the disclosed list) + the tester-agreement disclosure line.
• ACL — the default --acl-storages set keeps felhom-pbs (drill-proven: narrowing it 403s the PBS-DR apply-bridge at /storage/felhom-pbs); a missing felhom-pbs storage is now an expected-pre-tier INFO, and no longer a byo refusal (the tier creates the storage later).
• F-1 — single version source: SCRIPT_VERSION alone feeds -h, the run banners, and (via a gate cross-check) the hub Setup-tab const hostInstallVersion (bumped 1.12.0→1.15.0 in the same commit). The header no longer carries its own version literal.
• F-2 — dry-run fetch lines print the REAL auth mode (curl (anonymous …) vs curl -u <git>) instead of the hardcoded -u <git> lie on the anonymous branch.
• F-8 — step 4b + the final summary now say out loud that root@pam was rotated + vaulted and where to retrieve it (pointer only, never the credential).
• New gate: scripts/hostinstall_gates.py — mechanical grep-assertions (version single-source
  • hub-const sync, age, pbs-apply ship+uninstall, wg default-on + byo-assert retirement, felhom-pbs in the default ACL). All red-proofed; the age assertion was hardened TWICE by its red-proof (a prefix regex matched agekit, then a loose one matched the dry-run print line).
• F-3 (root-owned guests/ parents) is fixed at its creator — the AGENT (v0.86.0); this pass applied the drill's live-fix (chown the two parent dirs) to felhom-pve, which had the same latent state. Peti's host: unreachable, check deferred.

felhom-host-install v1.14.0 — deployment_mode from --mode (appliance self-heal gate) (2026-07-12)

• The agent-config writer now templates the top-level deployment_mode field from --mode (appliance | byo), authoritative even over a --preserve-from config. It gates the agent's node self-heal (host networking recovery at boot — F12-class defense in depth, agent v0.85.0): only deployment_mode:"appliance" unlocks the remedy; anything else, including a stale preserved value or a typo, is byo (the check runs + WARNs, the remedy is unreachable). The byo config assert additionally refuses deployment_mode=appliance (a byo box must never carry a host-service self-heal). Companion to CAMPAIGN-3 Task A (documentation/audits/CAMPAIGN-3-2026-07-11.md).

felhom-host-install v1.13.0 — systemd-journal group for the agent user (NAS verify) (2026-07-11)

• The agent-user setup now adds felhom-agent to the systemd-journal group (idempotent usermod -aG): the NAS verify-before-commit pipeline (agent v0.81.0) classifies mount failures from an UNPRIVILEGED journalctl -u <unit> read — group membership instead of a sudoers grant (rule: no journalctl via sudo). Existing installs get the one-liner usermod -aG systemd-journal felhom-agent && systemctl restart felhom-agent (applied to the demo host during this task's live validation; Peti's box only when the NAS feature reaches him).
• Version-header drift fixed: the file header still said v1.11.0 while SCRIPT_VERSION was 1.12.0 (the v1.12.0 bump shipped without a changelog entry or header sync); both now 1.13.0.

felhom-host-install v1.11.0 — uninstaller gap-closure + operator-key day-0 fold (TASK GL-4, go-live G6 + G1-follow-up) (2026-07-08)

• Uninstall gap-closure (G6):
  • NEW 4b4 block removes the self-update artifacts the install lays down but v1.10.0 never removed: /usr/local/sbin/felhom-selfupdate-guarded, the A/B slot files next to the live binary (felhom-agent.prev + orphaned .new.* temps), felhom-agent-rollback.service, and the felhom-agent.service.d/felhom-agent-limits.conf drop-in (+dir). Paths derived from the authoritative felhom-agent/configs/felhom-selfupdate-guarded; pending.json was already covered by the state-dir removal. Tolerate-absent, 4b2 shape.
  • Enrolled/network drives mounted under /mnt/felhom-drives/<name> are now unmounted (deepest first) before the root self-bind — plain umount ONLY, never -l/-f: a busy mount gets a warning + "eject via the dashboard or stop the apps and retry" and the root bind is then left alone (previously a child mount made the root umount die mid-teardown). The data is NEVER touched — no wipe/format path exists anywhere near /mnt/felhom-drives.
  • Both modes now END with an explicit KEPT-vs-WIPED statement: WIPED mirrors what the mode actually ran; KEPT names the drives + their data (physically removable; busy ones listed as retry), PBS backups + customer namespace (conditional on a pbs storage), the hub host/customer record, the escrow blob, and — when step 4b had vaulted one — the root@pam recovery credential.
  • Guest-only mode prints the drives the agent's bind store records for that vmid (best-effort from /var/lib/felhom-agent/guest-binds.json, generic note otherwise) + "eject BEFORE uninstall" guidance; no umounts there (remaining guests may use the drives).
• Operator-key day-0 fold (the GL-1/G1 key-pin follow-up): OPERATOR_KEY_{OPERATIONAL,RECOVERY}_{ID,LINE} constants (EMPTY until the operator's offline pin ceremony) + --operator-pubkey-file PATH (lines operational|recovery <authorized_keys line>; the comment field IS the key_id, required; malformed/empty file dies at argv time naming the line; file OVERRIDES the constants with a notice). Resolved keys are written to authz.signers in the agent config (exact config.go SignerKey schema; nonce_store_path left to the agent default). Preserve rule: script/file keys are authoritative ONLY when non-empty — a reinstall with no keys KEEPS a preserved config's signers (never silently clobber a manually-pinned box; replacing preserved signers logs a pin-rotation notice). byo disclosure names the operational key_id (or "NONE — dormant"); verify reports authz signers: N (armed) or a dormant WARN (dormant = safe default).
• Harness: +13 static cases (GL4-C1..C5 incl. 5 live key-file refusals + grep shapes, GL4-D disclosure↔uninstall parity, GL4-INV no-forced-unmount/no-format invariant) + PVE-tier GL4 H-U (full uninstall DRY transcript against the live guest, state-override protected). 28/28 PASS on felhom-pve; red-proofs RP-1 (4b4 dropped → parity FAILs), RP-2 (preserve guard dropped → C4 FAILs), RP-3 (role-die dropped → C3a FAILs) all run→fail→revert on scratch copies.
• NOT live-validated (GL-6): the real (non-dry) uninstall with mounted/busy drives, the statement on a real teardown, and an armed key-pin install end-to-end.

felhom-host-install v1.10.0 — --mode appliance|byo install profile (TASK GL-2, go-live G2/G4/G5) (2026-07-07)

• --mode appliance|byo is now REQUIRED for a fresh install / --resume (no default — the build-golden v2.0.0 precedent). appliance = the historical behavior, dry-transcript-preserved. byo = a Proxmox host the operator does NOT own (the pilot): step_break_glass is gated OFF at its call site (root@pam never reset, nothing vaulted — the one place the skip is auditable), --cores + --memory both mandatory (the only noisy-neighbor protection there), --enable-oob/--rotate-recovery refused at argv time, PVE-major gate (die on non-9.x unless --force), every --acl-storages entry must exist on the box (die naming ALL missing; appliance warns), informational host-DNS :53 line, full host-mutation disclosure + typed-hostname acknowledgement (/dev/tty, dry-run-aware), and post-write config asserts that refuse lan_resolver.enable/wg_tunnel.enabled/oob.enabled = true (byo also flips the config write's lan_resolver DEFAULT to off; appliance keeps default-on).
• The old --mode provision value is retired (specific die message); the dr stub seam is kept. The mode is persisted in state.json; a --resume under the other mode refuses (C4).
• Verify asserts (BOTH modes — campaign-2 R2 lesson): the verify step now asserts pool membership of the provisioned vmid + every scoped ACL grant (user AND token: Base@/, Guest@/pool/felhom, Store@each storage); any miss → error + "WITH WARNINGS" verdict.
• --preflight-only: runs the full mode-aware step-1 checks, prints an explicit PRE-FLIGHT PASS/FAIL verdict, exits 0/1, writes NO state marks (never lets a later --resume skip the real preflight). Existing-guest count line added to preflight (both modes).
• FELHOM_INSTALL_STATE_DIR env override (test-harness only) so harness cases can never touch a live state.json; usage() switched to a marker-based range (the numeric one had drifted).
• NEW scripts/hostinstall-mode-harness.sh: static tier (bash -n, shellcheck, C1–C4 refusal matrix asserting exact die messages, grep-invariants: one gated break-glass call site, chpasswd unreachable outside it, --mode in usage) + PVE tier (C5 live-refusal + Scenario A/B dry transcripts via FELHOM_TEST_CUSTOMER/FELHOM_TEST_PASSFILE). Red-proofs RP-1..RP-3 executed (gate removed / caps requirement dropped / mode-mismatch check dropped → matching case FAILs).
• Live-validated read-only on felhom-pve: full static tier + C5 + byo dry transcript (no 4b/8/chpasswd/recovery-credential; disclosure+ack+DNS+storage lines present) + --preflight-only PASS and FAIL verdicts with zero state entries. Full A/B rc=0 transcripts + C7 await the supervised GL-6 drill (blocked today by demo-felhom's EMPTY git.username/git.token — any step-5 re-run for that customer dies until the operator sets real Gitea read creds).

felhom-host-install — --enable-oob installs the dedicated felhom-sshd OOB instance + belt (TASK H1) (2026-07-05)

• --enable-oob + install_oob (in step 5): lays down the STATIC OOB parts — /etc/felhom-sshd tree
  • a dedicated ed25519 host key + the felhom-op login user + its scoped /etc/sudoers.d/felhom-op (visudo-validated) + the felhom-sshd.service unit (RuntimeDirectory-guarded — refuses the G1 incident cause) enabled-not-started (the agent renders the config) + the static inet felhom_oob nft table (nft -c-checked) + its boot loader felhom-oob-nft.service. The agent reconciles the DYNAMIC parts (port claim, config, operator authorized_keys, belt set elements) once oob.enabled.
• Uninstall (step 4b3): stops/disables felhom-sshd + the belt loader, removes both units + the config tree + belt table + felhom-op user/sudoers; NEVER touches the stock sshd/:22//etc/ssh.

felhom-host-install — break-glass credential + management-plane watchdog install (TASK G1) (2026-07-05)

• step_break_glass (new step 4b): generates a strong root@pam console password (openssl rand, NEVER logged/filed — stdin→chpasswd + stdin→curl), sets it, and vaults it to the hub over the enroll-authenticated host-key channel (PUT /hosts/{id}/recovery-credential). Idempotent (state marker) unless --rotate-recovery. The human break-glass path for the PVE web console.
• install_mgmt_watchdog (in step 5): installs the G1 host artifacts from the agent repo configs — felhom-privsep.tmpfiles (boot-persistent /run/sshd), felhom-mgmt-watchdog.{sh,service,timer} (agent-independent ~60s privsep-dir auto-heal). HARD GUARD: refuses any fetched unit declaring RuntimeDirectory= (the SPIKE-felhom-sshd §8 incident cause). Non-fatal on a repo that predates the artifacts. Uninstall (step 4b2) stops+disables the timer and removes all four artifacts + the marker.

docs — 06-doc S3 row SHIPPED + agent-side revocation semantics (2026-07-04)

Docs-only companion to felhom-agent v0.64.0 (the S3 slice — keygen, registration, agent-managed wg-quick@wg-felhom, escrow join; live-validated on felhom-pve incl. revocation drill, reboot persistence, 30-min soak). 06-doc §3.5 now records: register-once marker, revoked-stays-revoked, re-add via the registration endpoint (the raw registry add doesn't bump the host generation — live finding), wg_tunnel.enabled default-FALSE rollout gate. S6 backlog notes added (hub poll constant configurable + first-adoption log; registry-add bump-or-label). CGNAT/mobile-hotspot appendix deferred (operator-assisted; §7's open validation stands).

felhom-peersync.sh v1.0.1 — strip out of process substitution (exit-swallow fix) (2026-07-04)

The S1 REPORT's exit-swallow class, fixed: wg syncconf wg0 <(wg-quick strip "$tmp") hid the strip exit code — a corrupt wg0.conf.head could feed syncconf empty/partial input that WIPES the live peer set while the script exits 0 (and the bad conf is then persisted). v1.0.1 runs strip as its own step into $tmpdir/stripped; a strip failure aborts BEFORE wg is invoked. Sandbox red-proof (stub wg-quick exiting 1 after partial output + recording stub wg): pre-fix shape invoked wg and returned rc=0; fixed shape errors first, wg never called. Redeployed to the dev endpoint (runbook step 5 install); shellcheck clean.

felhom-peersync.sh v1.0.0 — the offsite endpoint's WG reconcile script (2026-07-04)

S1 (doc 06 §5): the forced-command target the hub's wgsync pushes to (runbook offsite-endpoint.md step 5 installs it as /usr/local/bin/felhom-peersync, root:root 0755, invoked via a one-line sudoers grant from the felhom-peersync user's restrict,command= authorized_keys entry). Validate-FIRST design: jq contract check (version 1, interface wg0, 44-b64 pubkeys, 10.77.0.x/32 allowed_ips, never the endpoint's own .1) rejects on stderr with exit 1 before touching anything; then head-file + generated [Peer] blocks into a same-fs tmp, wg syncconf <(wg-quick strip …) from the TMP (exact-match: adds/removes without bouncing the interface), and only on success the atomic mv to /etc/wireguard/wg0.conf — runtime and boot config can never diverge in the failure direction. Zero-peer payload = valid wipe. Never reads or prints the private key; no wg-quick save; no second mode. shellcheck-clean. Live-proven on felhom-hetzner incl. the negatives (malformed JSON / bad pubkey / own-IP peer → exit 1, wg state byte-identical) and reboot persistence.

docs — architecture Part 06: offsite connectivity design-of-record (2026-07-03)

documentation/architecture/06-offsite-connectivity.md — the settled offsite-backup-transport design, authored from the spike verdict + operator-resolved forks (recorded, not re-litigated): plain WG (D1), host-side agent-managed wg-felhom as the agent-managed-unit pilot on the sudoers *.mount install pattern (D2), one shared hub-driven endpoint VM running WG + the offsite PBS with no agent (D3, CF-token pattern), hub source-of-truth with a wireguard block riding the existing WireDesiredState/DesiredGeneration channel (D4), one datastore + per-customer namespaces (D5), and PBS on the VM — relay-through-DooPlex rejected as non-scaling (D6). Includes the Day-0 join handshake, the robustness set (NOT-DynDNS roaming, endpoint DNS re-resolve watchdog, MTU 1420, per-/32 topological isolation, tunnel-health through the storage-target reachability model), trust conformance, the honest open ledger (CGNAT unmeasured → mobile-hotspot smoke test; peer-sync push-vs-pull = slice-1 design point), and the S1–S6 slice roadmap (MVP = S1→S2→S3, then S4; S5 merges with DR-completeness). All claims cited at file:line against felhom.eu @ bf099f6 + felhom-agent @ 4ba1b14. day0-install.md backlog line now points at spike + design doc. Docs-only.

docs — SPIKE: offsite-backup connectivity — plain WireGuard WINS the ladder; Headscale = separable fleet layer (2026-07-03)

documentation/audits/SPIKE-connectivity-wireguard-2026-07-03.md — the offsite-backup transport decision, empirically grounded on both real ends (demo-felhom PVE host ⟷ throwaway Hetzner box, end state: powered off, secrets shredded). Headline results: the operator's line is plain-NAT with a fixed public IP, not CGNAT, and has zero IPv6 (P0 honesty — CGNAT confirmation deferred to Peti's VM 110); plain outbound WG held an 11.4-min idle window and carried a real 2 GiB worst-case PBS backup at 4.26 MiB/s = the full home uplink (~5% tunnel overhead), TLS pin intact through the tunnel (positive + negative proof); UDP 51820 and 443 both pass; kernel WG surprisingly works inside the unprivileged guest (P7 — host placement stands on architecture, not infeasibility). Recommendation: host-side agent-managed WG (cloudflared pattern), key in the escrowed IdentityBundle, small public endpoint VM (PBS-on-VM vs rendezvous-relay deferred to the spec). runbooks/day0-install.md backlog line resolved to point here; CONTEXT.md notes the DR-completeness task is unblocked (next: the production connectivity spec).

skills — NEW: felhom-app-catalog (4th skill) + SparkyFitness as its worked example (2026-07-03)

skills/felhom-app-catalog/SKILL.md — the catalog authoring workflow (research → inspect the image for the healthcheck family → write compose/.felhom.yml → deploy live through the dashboard → verify healthy → reconcile the app count). Deliberately points at app-catalog REUSE.md §1–2 + README.md §format for every field table (one-fact-one-place; no duplication). Unique content: the never-guess-the-healthcheck rule with the per-tool image-inspection loop (BusyBox ash command -v gotcha: it silently ignores all but its first argument — verified), the probe-container naming rule (controller probes the container named exactly like the stack — verified in felhom-controller/internal/stacks/healthprobe.go, row added to app-catalog REUSE.md), the Hungarian-quote YAML kill, and the deploy-is-the-test doctrine. No installer change needed — install_skills.py auto-discovers skills/*/SKILL.md; fresh-session discovery probe listed all 4. Proven by finalizing sparkyfitness end-to-end on demo (see app-catalog-felhom.eu CHANGELOG).

docs — golden 0.98.3 live: D.1b RETIRED, drill ledger B1/B5 → FIXED, backlog note resolved (2026-07-03)

Companion to felhom-agent's build-golden.sh v2.0.0 (@ ceca355): the golden now bakes the CURRENT controller (0.98.3, mandatory-tag convention — B5) and a felhom-controller-bootstrap.path unit (controller deploys on the bootstrap-mount hot-plug, no reboot — B1). Clean-room validated end-to-end BEFORE publish (bake → isolated hot-plug proof → local-golden Day-0 install → publish → vouch → --force-gitea-golden install); evidence: documentation/audits/DRILL-golden-098-2026-07-03.md.

• documentation/runbooks/day0-install.md: D.1b reduced to a one-line version check (fresh boxes land current + self-manage); old manual-update procedure moved to Part F troubleshooting keyed on "golden older than 0.86.0 was vouched"; header validated-versions line → script v1.9.1 / agent v0.63.0 / golden v0.98.3; A.3 records the drilled-known-good pair + "vouch ≥ 0.98.3"; A.4 floor text rewritten (fresh boxes now floor-honoring; recommend raising the global floor after a golden rebuild — operator, 1 min).
• documentation/audits/DRILL-day0-cleanroom-2026-07-03.md ledger: B1, B5 → FIXED (pointers); R6 note: the installer's post-provision reboot is retained as a belt, removal recorded as a candidate cleanup (not done).
• documentation/backlog/FOLLOWUP-golden-default-controller-tag.md + backlog/README.md: RESOLVED per the M18/M19 convention (file kept + annotated; README entry marked FIXED).
• New evidence doc: documentation/audits/DRILL-golden-098-2026-07-03.md (A–D transcripts, unit states, resolution-order + fetch/sha proofs, cleanup, observations — incl. the SECURITY observation that the customer git.token has package-WRITE rights → scope-down + rotate follow-up).

docs — day0-install D.1b narrowed + drill ledger B2/B3 → FIXED agent v0.63.0 (2026-07-03)

Agent v0.63.0 fixed both drill findings (B3 token reload-on-miss + B2 guesthook snippets-dir mkdir; red-proofed, deployed on felhom-pve, Gitea-published sha256 b4a89c81…). Guide follow-through: the D.1b "restart the agent first" step is now CONDITIONAL (only for an installed agent < v0.63.0 — the Day-0 manifest still vouches 0.62.0, so today's fresh installs still hit it); the 401 troubleshooting row records the fix version; the drill ledger + go/no-go item 8 marked FIXED. Operator follow-up unchanged: vouch agent 0.63.0 in the Day-0 manifest UI, then the step is dead.

felhom-host-install.sh v1.9.1 — clean-room drill fixes: residue-free uninstall + post-provision reboot (2026-07-03)

Companion to the Day-0 go-live package (documentation/runbooks/day0-install.md + documentation/audits/DRILL-day0-cleanroom-2026-07-03.md). Every fix was found by the clean-room drill (virgin nested PVE 9.2.2) and re-verified there (v1.9.1 uninstall → zero-Felhom-residue diff vs the pre-install baseline; v1.9.1 install → controller up with no manual intervention).

• Header/version sync (the header said v1.8.0 while SCRIPT_VERSION said 1.9.0); keep-in-sync note on SCRIPT_VERSION; usage sed range follows the header (2,95).
• Uninstall now removes the drill-found residue (R1–R5): the agent config (resolved from the unit's -config BEFORE the unit is removed — it holds the per-host hub api_key), the felhom-shared-parent unit + wants links + /usr/local/sbin/felhom-shared-parent.sh
  • the /mnt/felhom-drives self-bind/dir, /usr/local/sbin/felhom-mkfs-guarded, /var/lib/vz/snippets/felhom-guest-hook.sh, and /etc/dnsmasq.d/felhom-*.conf (+ dnsmasq restart when touched). All tolerate-absent; summary lines updated (sudo AND dnsmasq packages are the documented package remnants).
• Post-provision guest reboot (R6): the golden's felhom-controller-bootstrap.service evaluates ConditionPathExists=/etc/felhom-bootstrap/bootstrap.json at BOOT, but the agent back-half hot-plugs the mount into the running guest — on slower hardware the first boot loses that race deterministically and the controller never deploys. step_provision now reboots the guest once (the agent's own output says "next: reboot the guest"); step_verify waits bounded (180 s) for the controller container instead of a momentary look.

felhom-host-install.sh v1.9.0 — Pool.Audit for the stale-lock reaper (A1) (2026-07-03)

Companion to felhom-agent v0.62.0 (audit A1: pool-membership ownership check). PVE_PRIVS_GUEST gains Pool.Audit (12 → 13 privs, granted at /pool/felhom via the existing FelhomAgentGuest role) so the agent can read GET /pools/felhom — its stale-lock reaper's ownership registry. Pool.Allocate does NOT satisfy the read (spike SPIKE-a1-pool-membership-read-2026-07-03 T2). No structural change: _ensure_role already role modifys to the exact priv set, so re-running --rescope-acl (or a fresh install) upgrades an existing box idempotently; remove_scoped_acl deletes by role name and needs nothing. Deploy order on a live box: rescope FIRST, then deploy agent v0.62.0 — the added read priv is harmless to an older agent, while the new agent on an old ACL fail-safes its reaper (skips) and reports pve:pool-read degraded until the rescope lands.

docs — SPIKE: A1 pool-membership read for the stale-lock reaper (2026-07-03)

Findings doc documentation/audits/SPIKE-a1-pool-membership-read-2026-07-03.md. Live-probed on felhom-pve under the PRODUCTION scoped token vs root: LXC enumeration IS already pool-filtered (token sees only 9201 of 4 guests); GET /pools/felhom 403s naming Pool.Audit; a throwaway token with ONLY Pool.Audit@/pool/felhom reads members (minimal delta proven, fully torn down); /cluster/resources withholds the pool field without Pool.Audit; local ownership records are all partial. Recommendation for the A1 impl spec: add Pool.Audit to PVE_PRIVS_GUEST in felhom-host-install.sh (L183) + a GET /pools/felhom cross-check in the agent's staleLockController.Guests(), fail-safe skip on read failure. No script/agent change in this commit — docs only. Appendix: committed-secrets (felhom.secret.yaml) rotation micro-runbook, operator follow-up.

install_skills.py — new: Claude Code skills installer (2026-07-03)

Installs skills/*/SKILL.md (felhom-build-deploy, felhom-ui-design, felhom-testing) into ~/.claude/skills/ as Windows junctions (mklink /J) so repo edits are live immediately; falls back to a full copy if junction creation fails or isn't followed (copy mode prints a re-run reminder). Idempotent — re-runs detect a correct junction and leave it. Verified: junctions ARE followed by Claude Code skill discovery (fresh-session probe found all three).

reuse_refs_check.py — new gate: REUSE.md citation checker (2026-07-03)

Staleness defense for the new per-repo REUSE.md reuse maps. Takes repo roots as argv, extracts every cited *.go/*.py/*.html/*.css/*.yml/*.yaml/*.sh path (slash-containing tokens only — bare filenames are conventions, not citations), verifies each exists; prints offenders, non-zero exit on any missing path. Symbols are spot-verified by the reviewer, not this script. Usage: python scripts/reuse_refs_check.py <repo-root> [...].

felhom-host-install.sh v1.8.0 — install the guarded-mkfs wrapper (Impl-1 Part B) (2026-07-01)

Companion to felhom-agent v0.54.0 (format-safety foundation). During agent install, fetch + install the guarded-mkfs wrapper so the agent's format path is safe on any box.

• New step in step_agent_install: fetch configs/felhom-mkfs-guarded.sh from Gitea, bash -n validate, install -m0755 -o root -g root → /usr/local/sbin/felhom-mkfs-guarded. Installed BEFORE the sudoers (which now allowlists ONLY the wrapper, not raw mkfs.*), so the ordering is gap-free.
• The agent v0.54.0 sudoers (fetched by the same step) drops the raw mkfs.ext4 -F /dev/* / mkfs.xfs -f /dev/* allowlist and permits only felhom-mkfs-guarded /dev/* *, plus read-only pvs/zpool for the agent's unclaimed-disk guard. No other host-install change.
• bash -n + shellcheck clean (0 new warnings). Live-validated on felhom-pve (agent v0.54.0 deploy): wrapper refuses the OS disk + an LVM-PV partition, raw mkfs is sudo-denied, an unclaimed throwaway disk formats; the agent guard's sudo reads (pvs/lsblk/zpool) all work as the felhom-agent user.

felhom-host-install.sh v1.7.0 — 3b-fix: Datastore.Audit box-wide (restore drive visibility) (2026-07-01)

Fixes a regression the v1.6.0 pool-scoped ACL introduced: Datastore.Audit was placed in the per-storage Store role (granted only on local/local-lvm/felhom-pbs), which excluded the enrolled removable drives felhom-usb/felhom-flash. The agent enumerates storage via ListStorage/NodeStorage (both gated by Datastore.Audit — internal/storage/observe.go), so it could no longer SEE the drives → false "Meghajtó leválasztva" (drive detached) alerts + drives absent from the agent-view. (The v1.6.0 swap's "felhom-usb → 403" was mis-read as blast-radius success; felhom-usb is Felhom's OWN customer drive, not an out-of-scope object.)

• Datastore.Audit moved from Store → Base (PVE_PRIVS_BASE now "Sys.Audit SDN.Use Datastore.Audit"; PVE_PRIVS_STORE now "Datastore.Allocate Datastore.AllocateSpace"). Audit is read-only metadata, so box-wide Audit restores visibility of ALL storages (incl. dynamically-enrolled drives — no per-drive grant ever needed) while the write privs (Allocate/AllocateSpace) stay per-storage → write/allocate blast-radius containment is UNCHANGED. Confirmed at source: the agent creates no PVE storage (no POST /storage/pvesm add); drives are dir-storages it observes + mounts via host ops, so they need only Audit, never Allocate.
• apply_scoped_acl reordered Base-before-Store (role + grant) so a RE-APPLY on a live box adds Audit@/ before Store drops its per-storage Audit → gap-free (the agent never loses enumeration).
• remove_scoped_acl / --uninstall / --rescope-acl operate by role NAME and inherit the corrected privs automatically (no other change).
• Live-repaired felhom-pve (two pveum role modify, Base first — no agent stop/restart): drives reappeared (agent-view 3→5 storages), detach alerts cleared. Re-tested under the scoped token: drives readable (was 403), write-containment intact (vzdump→felhom-usb still 403; out-of-pool guest 403), PBS Store grant unchanged. bash -n + shellcheck clean (0 new warnings).
• NOT physically run (source-confirmed, no Datastore.Allocate in the path): a brand-new-drive UI enrollment (needs a spare USB) — the host-ops/Audit path is unchanged from pre-3b.

felhom-host-install.sh v1.6.0 — pool-scoped token ACL (3-role) + --rescope-acl retrofit (2026-07-01)

Colleague-safety batch #4 phase b (script half; agent half = v0.53.0). Moves the agent token's dangerous privileges off / (which spanned every guest + storage) to /pool/felhom + /storage/<targets>, so on a shared box the token can only touch Felhom's own guests + storages. Validated by documentation/audits/SPIKE-pool-scoped-acl-2026-07-01.md (PASS) — implemented here.

• 3-role scoped ACL (step_token rewrite). Replaces the single FelhomAgent role granted at / with three roles, each granted to BOTH the user AND the token (privsep intersection): FelhomAgentGuest (VM.* + Pool.Allocate) @ /pool/felhom; FelhomAgentStore (Datastore.*) @ each of PVE_STORAGES (default local local-lvm felhom-pbs — the offsite PBS MUST be included, SPIKE residual #1; --acl-storages overrides); FelhomAgentBase (Sys.Audit SDN.Use) @ /. Helpers apply_scoped_acl/remove_scoped_acl/_grant/_ensure_role.
• Pool before token. ensure_felhom_pool runs at the top of step_token (always, incl. --skip-provision) so /pool/felhom exists before it's granted on.
• Re-install safety. step_token also removes the pre-3b broad / grant + FelhomAgent role if present (remove_old_broad_acl, tolerate-absent), so a re-install can't leave the old grant unioned with the scoped one. The post-provision pool_add_guest is gone (the agent's restore --pool makes the guest a member atomically — v0.53.0).
• --rescope-acl retrofit (new mode, mirrors --adopt-pool): migrate an existing install — ensure the pool + guest membership, apply the scoped grants, THEN remove the old broad grant (add-before- remove: the token is never grant-less mid-migration). Prints the "now deploy agent ≥ v0.53.0" ordering reminder. Idempotent + dry-run-aware. SUPERVISED (run with the agent stopped — the scoped ACL and the pool-param agent are mutually dependent; §13 of the task).
• --uninstall now removes the scoped grants + 3 roles AND the pre-3b broad grant/role (both tolerate-absent → works on either shape), keeping the pool delete-if-empty (v1.5.0).
• Validated on felhom-pve (dry-run): T-A fresh install (pool-before-token, 3 roles once, scoped grants incl. /storage/felhom-pbs), --rescope-acl (add scoped → remove old FelhomAgent), T-F uninstall (old-shape cleanup + pool not-empty skip). bash -n + shellcheck clean (0 new warnings). The live rescope + agent swap is the supervised STOP — not run here.

felhom-host-install.sh v1.5.0 — felhom pool by default + --adopt-pool retrofit + uninstall teardown (2026-07-01)

Colleague-safety batch #4 phase a. Every Felhom-managed guest now joins a dedicated felhom pool for fleet uniformity (and as the environment the later pool-scoped ACL — 3b — will spike against). All pool ops run as root@pam from the installer, so there is NO agent/token/ACL change and zero permission-model risk (PVE_PRIVS untouched; the FelhomAgent token stays scoped at /).

• New felhom pool default. step_provision calls ensure_felhom_pool (create if absent, idempotent) and, after a successful provision, adds the guest via pveum pool modify felhom -vms <vmid> (skip-if-already-member). New helpers pool_exists / pool_members / ensure_felhom_pool / pool_add_guest; const PVE_POOL="felhom". PVE 9 syntax + /pools JSON shape confirmed live before wiring (pveum pool add|delete|modify; pvesh get /pools → [{poolid,comment}], /pools/<id> → {members:[{vmid,…}]}).
• --adopt-pool retrofit mode. Non-destructive: adds an EXISTING Felhom guest to the pool (creating it if needed), resolving the guest from --vmid else the recorded provisioned_vmid. Reuses the ours-check (/etc/felhom-bootstrap mount) — refuses a non-Felhom guest unless --force. Touches ONLY pool membership: never reconfigures/restarts the guest, never contacts the hub. Idempotent (skip-if-member).
• --uninstall pool teardown (step 5b). After the pveum removal, deletes the felhom pool only if empty (a destroyed guest is auto-removed from its pool); a pool that still has members is left with a log_skip naming them. Not reached on the Spec-1 safe-skip path (other Felhom guests remain).
• Validated on felhom-pve (dry-run + SAFE live): T-A fresh-install dry-run shows the pool create + membership lines; T-B live adopt of guest 9201 → pvesh get /pools/felhom lists 9201, guest still running, config unchanged (the demo node is now pool-uniform); re-run = no-op; T-B' non-Felhom vmid → refusal; T-C uninstall dry-run → "pool felhom not empty (members: 9201) — leaving it". bash -n + shellcheck clean (0 new warnings; the 2 pre-existing SC2015 in step_verify unchanged).
• NOT changed: PVE_PRIVS, the ACL grants, the agent, the provision-call args. 3b (pool-scoped ACL
  • agent restore-into-pool under a scoped token) is the separate spike-gated task.

felhom-host-install.sh v1.4.0 — appliance CPU/RAM cap passthrough (--cores / --memory) (2026-07-01)

Colleague-safety batch #3 (host-install half; the mechanism is agent v0.52.0). Lets an operator cap the provisioned guest so a trial appliance on a SHARED production Proxmox doesn't pressure the colleague's existing guests.

• --cores N / --memory M (MiB) — optional; passed through to the agent's --selftest=provision as -cores/-memory. 0/unset = keep the golden's baked sizes (unchanged behaviour). New vars CPU_CORES/MEM_MIB; usage() header gains an "Appliance cap (optional)" group.
• Conditional passthrough — step_provision builds a cap_args array and appends the flags to BOTH the dry-run log and the real agent call only when set. An agent < v0.52.0 would reject an unknown flag, so the flags are never sent unless the operator opts in (see the deploy dependency below).
• Pre-flight sanity WARN (soft, provision only) — if --cores > host nproc or --memory > host MemTotal, log_warn "the cap won't protect other guests"; never die (the operator may know better).
• Deploy dependency: a fresh install using --cores/--memory needs the hub artifact manifest to serve agent ≥ v0.52.0.
• Validated dry-run on felhom-pve: --cores 2 --memory 4096 --dry-run → provision command shows -cores 2 -memory 4096; without the flags → neither present; --cores 64 --memory 65536 → both WARN lines (host 4 cores / ~15771 MiB). bash -n + shellcheck clean (0 new warnings; the 2 pre-existing SC2015 in step_verify unchanged).

felhom-host-install.sh v1.3.0 — --uninstall (clean revert) + pre-flight guards (2026-07-01)

Colleague-safety batch #1+#2. Adds a first-class, guarded --uninstall teardown so an operator can cleanly back out of a trial install, plus three provision pre-flight guards that stop common footguns. Script-only; no agent/hub/controller change.

• --uninstall (local host teardown — no hub contact, no passphrase). Reverses an install in the install-order's reverse: guest → agent(unit/sudoers/binary/state/user) → pveum(ACL,token,user,role) → golden(opt-in) → state file. Every mutation goes through run() so --dry-run prints the full plan and executes nothing. Safety:
  • Ours-check: refuses to destroy a guest that lacks the /etc/felhom-bootstrap bind mount (matched by the constant guest path, not a hardcoded mpN slot — on the demo host it's mp9), unless --force.
  • Typed confirmation: must type the vmid to confirm PERMANENT destruction (read from /dev/tty; skipped only under --dry-run, where nothing is destroyed).
  • Other-guests guard: if any OTHER Felhom guest remains, destroys only the target and leaves the agent + PVE token + state in place (re-run with --force to remove host-level anyway — orphans the others).
  • Never removes the sudo package; never contacts the hub (the host record intentionally persists).
  • Presence-checked + idempotent: an already-absent guest/unit/sudoers/binary/user/ACL/token/role is a tolerated skip, not an error. The pveum role delete runs only after its ACL grants are gone (PVE refuses to delete a referenced role). Confirmed PVE 9 ACL-delete form: pveum acl delete / --users|--tokens <x> --roles FelhomAgent.
  • Target vmid resolves from --vmid, else the recorded provisioned_vmid (else dies). A --vmid that disagrees with the recorded one needs --force.
  • --remove-golden: with --uninstall, also delete the golden vzdump from the archive storage (pvesm free); otherwise it is left in place.
• Install state now records customer_id + provisioned_vmid (new _state_put/_state_get helpers, dry-run-guarded like _state_mark; the completed[] shape is untouched) so a later --uninstall resolves its target automatically and safely.
• Pre-flight guards (provision mode):
  • Multi-node guard — on a 2+-node cluster, die (naming the nodes) unless --node is explicit (new NODE_EXPLICIT); single-node keeps the current auto-pick. No-op under --skip-provision.
  • Archive-storage-exists guard — verify --archive-storage appears in pvesm status (else die); no-op under --skip-provision.
  • RAM floor (WARN, never fatal) — warn when MemAvailable < 2048 MiB. All three run inside step_preflight (before any mutation) so they also fire under --dry-run.
• Validated dry-run-only on felhom-pve (single-node, live guest 9201): T-A full uninstall plan, T-C not-ours refusal (red-proof), archive-missing die, RAM line, other-guests detector, state round-trip; confirmed 9201 + agent + pveum + state untouched after all dry-runs. bash -n + shellcheck clean (0 new warnings vs. baseline; the 2 pre-existing SC2015 in step_verify are unchanged). NOT yet live-validated (awaiting a supervised run): a real live --uninstall (guest destroy + pveum removal) and the multi-node guard on an actual cluster.

felhom-host-install.sh v1.2.0 — /dev/tty passphrase read + vmid auto-detect (2026-07-01)

Two operator-experience fixes so a colleague can install online (via the hub's new "Option 1: Online install" one-liner) and onto a host that already runs a guest at 9201.

• Passphrase prompt reads from /dev/tty, not stdin (read_passphrase). read -rsp … < /dev/tty makes the no-echo prompt work regardless of how stdin is wired — both download-then-run and curl … | sudo bash (where stdin is the pipe). Strictly more correct; the --passphrase-file path is unchanged. The passphrase is still never on argv / in logs / in the state file.
• VMID auto-detect (--vmid now optional-smart). New VMID_EXPLICIT flag (set by --vmid). The pre-flight vmid guard now determines "in use" against the pct list + qm list id-set (LXC and VMs share the id space — more complete than the old pct status, which only knew LXC):
  • explicit --vmid → unchanged deterministic behavior: die if the id is in use unless --force (destructive over-provision).
  • default 9201, in use, no --force → auto-pick the next free id (scan upward from 9201 over the used-set) and ask to confirm from the terminal (read … < /dev/tty, [y/N]); proceed on yes, die "no free vmid confirmed" otherwise. Never a silent auto-pick.
  • default 9201 + --force → over-provision 9201 (destructive) without prompting, as before.
  • New helpers used_vmids / _vmid_in_use / next_free_vmid. --vmid help text + usage() updated.

felhom-host-install.sh v1.1.0 — self-install the agent + fetch the golden from Gitea (2026-06-28)

The script now installs the agent itself (the last big manual Day-0 prerequisite is gone). It fetches the agent binary + golden from Gitea generic packages and verifies each against the hub-vouched artifact manifest before installing/using it. BUNDLE slice; pairs with hub v0.16.0 (artifact manifest endpoint + operator UI) and felhom-agent v0.43.0 (canonical unit + publish).

• New step 5/8 agent install (before agent-config): resolves the manifest (GET /api/v1/artifacts/{id}, passphrase) + the git fetch token (from the customer's controller.yaml via config-retrieve — NO new credential); fetches /api/packages/admin/generic/felhom-agent/<ver>/felhom-agent, verifies sha256 vs the hub manifest (aborts on mismatch — verify-before-use), backs up any existing binary, installs 0755 /usr/local/bin/felhom-agent; ensures the non-root felhom-agent system user; installs the canonical sudoers (0440, visudo -cf-validated) + systemd unit; daemon-reload + enable. Idempotent: same version already installed + service active → skip.
• --skip-provision: install + configure + verify the agent (incl. golden fetch+verify) but do NOT provision a guest — the agent-only path for re-installing/upgrading the agent on a host that already has live guests. Adds an agent-only step_verify_agent (binary + non-root service active + a --selftest=hub collect-report).
• New step 7/8 golden: local auto-discovery stays the default/fallback; otherwise fetches /api/packages/admin/generic/felhom-golden/<ver>/golden.tar.zst, verifies sha256, and imports it into the archive storage's dump dir for the restore. --force-gitea-golden forces the Gitea path.
• Non-root agent model: the agent now runs as felhom-agent with privileged.mode: "sudo" (was the dev/CI direct+root shortcut). The config is chowned to the service user (0600) so the daemon can read it; systemctl is-active after restart is the real proof the non-root user can read the config.
• Pre-flight relaxed: a missing agent binary is no longer fatal (step 5 installs it); the local golden requirement is deferred to step 7.
• Trust model: checksum trust root = the hub (manifest), not Gitea; the fetch credential is the existing config-retrieve git token; artifacts are pinned to a version (never :latest).
• Secrets: the git token is a never-logged runtime carrier (cleared on EXIT alongside the passphrase / pve-token / hub api_key); the sudoers is 0440 and visudo -cf-validated before install.
• bash -n + shellcheck clean.

felhom-host-install.sh v1.0.0 — Day-0 host bootstrap (provision mode) (2026-06-26)

First release. A single operator-run script that automates Day-0 on a freshly-PVE-installed host: Proxmox API token → hub host enrollment (option C, single secret) → agent config → guest provision → verify. Composes proven mechanisms (the pveum role/token sequence, hub POST /host-enroll, felhom-agent --selftest=provision); grounded by documentation/audits/SPIKE-day0-firstboot-handshake-2026-06-26.md.

• 7 steps, idempotent + resumable via /var/lib/felhom-install/state.json: pre-flight → Proxmox token → compute grows → host-enroll → agent config → provision → verify.
• Single-secret (the retrieval passphrase): read no-echo or from a 0600 file, never on argv/logs/state. The global operator key never touches the box.
• pveum automation: 16-priv FelhomAgent role (create-or-modify), felhom-agent@pve user, privsep token (reuse-if-working else rotate), and both ACL grants applied after the token exists (token-remove purges the token ACL).
• Auto-discovery: golden archive (newest vzdump-lxc-<golden-vmid>), PVE node name, vmbr0 bridge IP for the local-api, and the served-leaf TLS fingerprint pin.
• Safety: pre-flight fails fast (root, PVE 9.x, local-lvm headroom, hub reachable, customer+passphrase valid via read-only GET /config/{id}, golden resolvable); refuses to clobber an existing --vmid without --force; --dry-run previews every mutation; --preserve-from keeps operator infra (PBS/local_api/privileged/authz) on re-deploys.
• --mode dr: documented 10D stub (restore customer PBS snapshot instead of golden) — not implemented.
• Live-validated end-to-end on felhom-pve: authorized wipe of demo guest 9201 → re-provision from the golden → controller config-pull + public tunnel HTTP 200 → host-report of guest 9201 → idempotent --resume no-op. (One ordering bug — token ACL applied before rotation — was found and fixed during the live run.)