felhom.eu/documentation/pilot/GO-LIVE-PACKAGE.md

# GO-LIVE PACKAGE — first external tester (Peti)

> Tracking doc for the pilot go-live: gating items, statuses, decisions, task sequence.
> Lives at `felhom.eu/documentation/pilot/GO-LIVE-PACKAGE.md`. Update per task completion.
> Created 2026-07-07 (project-Claude, source-verified). Operator: Viktor.

**Baselines at creation (verified live):** agent `ced60dd`/v0.74.0 · controller `59eb3bea`/v0.103.0 ·
catalog `2ebe082a` · felhom.eu `1a1e42ad`/hub v0.35.0 · host-install v1.9.1 · golden 0.98.3.
**Updated 2026-07-07 (GL-2 shipped):** felhom.eu `02d63ed0` · host-install **v1.10.0** + `scripts/hostinstall-mode-harness.sh`.

---

## 1. Pilot definition & locked decisions (2026-07-07)

| # | Fork | Decision |
|---|------|----------|
| D1 | Pool model | **One shared `felhom` pool.** No multi-tenant boxes yet; the pool separates Felhom guests from the host owner's own workloads. Effort goes to preflight pool-membership assertion + local-API per-guest token scoping (verified structural, see G10). |
| D2 | Pilot topology | **BYO host.** Peti installs Felhom onto his OWN Proxmox server, already running his own LXCs. This reframes containment: not customer-vs-customer, but *Felhom-as-a-guest on a host the operator does not own*. See §2. |
| D3 | Backup posture | **Local-only for the pilot.** `wg_tunnel.enabled` and `oob.enabled` stay FALSE. Offsite (Tier-1/Tier-2 split + production WG endpoint) is a fast-follow, and the limitation is stated honestly in the tester agreement. |
| D4 | Uninstall drill | **Full provision → uninstall → re-provision drill on the demo box BEFORE Peti's day-0.** Peti's box is never the first live test of anything destructive. |
| D5 | DR bind-override | **Fix now** (small, spike-validated known-constant override per `SPIKE-dr-bindmount-source-2026-07-07.md`), not deferred to the intermediary-mount re-architecture. |
| D6 | Packaging | **This one tracking doc + a sequenced set of CC tasks** (GL-1…GL-7). |

---

## 2. The trust-model reframe (BYO host) — load-bearing

Everything so far was designed for the **appliance model**: Felhom-owned hardware, Viktor is the
root-holding operator, the customer is a UI user. Peti's pilot is the **BYO-host model**: Peti is
root on his own PVE box; Felhom is a *guest system* there (one LXC + a non-root agent + narrow
sudoers + a pool-scoped token). Consequences, each source-verified:

- **Break-glass inverts.** `step_break_glass` (host-install.sh:1186, step 4b/8, NOT flag-gated)
  sets root@pam via `chpasswd` and vaults it to the hub. Right for appliances; on a BYO host it
  silently changes the owner's root password and hands the operator root on hardware he doesn't
  own — a consent, trust, and data-sovereignty violation. **Must be profile-gated (G2).**
- **Host-wide artifacts need coexistence review**, not just installation: dnsmasq snippets under
  `/etc/dnsmasq.d/felhom-*.conf` + `systemctl restart dnsmasq` (and the installer installs the
  dnsmasq package) can collide with the owner's existing DNS; the mgmt watchdog + tmpfiles heal
  `/run/sshd` on *his* sshd (additive, likely benign — verify); felhom-sshd/OOB stays off (D3).
- **Already good citizens (verified):** VMID selection collision-safe (default 9201, auto-next-free
  over the union of `pct`+`qm` ids, explicit `--vmid` dies unless `--force` — host-install.sh:223–241);
  uninstall targets only guests carrying the `/etc/felhom-bootstrap` bind mount and refuses
  non-Felhom guests (:337, :472); stale-lock reaper is pool-intersected (agent v0.62.0);
  local-API requests are authorized strictly against the caller-token's own guest
  (localapi/server.go:23, :53–55, :153).
- **Local-API exposure:** binds the vmbr0 IP:8443 by default (host-install.sh:1546–1548) — on a BYO
  host that's reachable from the owner's LAN and his other guests. Defense = TLS + per-guest bearer
  token. Acceptable for the pilot; documented in G10.

---

## 3. Gating items (must be green before Peti's day-0)

Status legend: ✔ DONE · ◐ PARTIAL · ○ OPEN · ⚠ BLOCKER

| ID | Item | Status | Evidence / gap |
|----|------|--------|----------------|
| **G1** | **Publish + vouch chain current & self-update alive.** ARTIFACTS PUBLISHED (RUNBOOK GL-1, felhom.eu `a63cc715`): agent **0.74.0** sha `1ec3f588…76af05` (the exact live-proven felhom-pve bytes, hash independently re-verified anonymously) and golden **0.103.0** sha `8481e8a1…6026e` (clean-room bake on the virgin drill VM, controller 0.103.0 baked, both split volumes in the vzdump). Remaining: (a) **operator manifest bump** — until then fresh installs still land 0.63.0/0.98.3; (b) **operator-key pin** — day-0 ships NO operator pubkey (verified), so self-update stays dormant on fresh boxes; pin ceremony + a small host-install follow-up (candidate: fold into GL-4). | ◐ | RUNBOOK-GL1-publish-2026-07-07.md; felhom-agent `4c408467` |
| **G2** | **BYO-host install profile.** SHIPPED as `--mode appliance\|byo` (mandatory, no default; the flag was repurposed from the retired `provision\|dr` — `dr` stub seam preserved). byo: break-glass gated OFF at its single call site (:2040–2044 — root@pam never touched, nothing vaulted), caps mandatory, `--enable-oob`/`--rotate-recovery` refused, host-mutation disclosure + typed-hostname ack, post-write config asserts (`lan_resolver`/`wg_tunnel`/`oob` must be off — asserts run BEFORE `_state_mark agent_config`, so `--resume` re-asserts). Red-proofs RP-1..3 run→fail→revert. Appliance flow unchanged (regression cases green). | ◐ | host-install v1.10.0 @ `02d63ed0`; verified at file:line 2026-07-07. **Remaining: GL-6 live drill** (C7 verify-assert firing, interactive ack) |
| **G3** | **`git.token` scope-down + rotate — UNBLOCKED, and now coupled to a visibility ruling.** Gate 3c (GL-1): both generic packages are **anonymously world-readable**, so rotating the customer token to read-only cannot break fresh-install fetches; the empty-creds die on demo-felhom is the installer's `resolve_git_creds` precondition (:326–332), not a Gitea requirement. **NEW FINDING (project-Claude probe, 2026-07-08): the ENTIRE Gitea instance is anonymously readable — all nine repos `private=False`, including `homelab-manifests` (k3s manifests — audit for raw Secret objects) and personal repos (`work`, `revfulop-calendar`). Only the geo-fence (HU/DE/US) limits reach.** Coupling warning: if the reaction is to lock the instance ("require sign-in"), the anonymous day-0 package fetch dies and read-scoped tokens become MANDATORY — retest the fetch path with a token before any visibility change. | ○ ⚠ | GL-1 record §Gate 3c; anonymous `repos/search` probe 2026-07-08 |
| **G4** | **Preflight guards (refuse-to-provision gate).** SHIPPED in v1.10.0: byo dies on non-9.x PVE (unless `--force`) and on missing `--acl-storages` entries (named); `:53` DNS info line + existing-guest count; `--preflight-only` runs the full mode-aware checks with a PASS/FAIL verdict and **zero state marks** (live-proven on felhom-pve, incl. the FAIL verdict on a bogus storage). Verify step now asserts pool membership + all scoped-ACL grants (user+token) in BOTH modes — R2's lesson made structural. | ◐ | host-install v1.10.0 @ `02d63ed0` (verify asserts read at source). **Remaining: GL-6** (asserts firing on real drift) |
| **G5** | **LXC caps mandatory on BYO hosts.** SHIPPED: byo refuses to run without explicit `--cores` AND `--memory` (:947, fires before the passphrase prompt); existing cap-sanity warns retained. Appliance unchanged (optional, golden default). Cap VALUES for Peti's guest still to be chosen (§6). | ◐ | host-install v1.10.0 :947. **Remaining: pick values + GL-6** |
| **G6** | **Uninstaller gap-closure (customer offboarding).** `--uninstall` exists and is well-shaped (guest-only mode when other Felhom guests remain, dry-run, typed-vmid confirm, legacy-ACL cleanup, opt-in golden removal — host-install.sh:469–710). Gaps: enrolled customer drives (unmount/decommission; data STAYS on the drive — additive-only doctrine), PBS namespace note (local `felhom-pbs` ns for the box), hub record + escrow blob cleanup (currently a manual NOTE at :550/:708), and a printed **kept-vs-wiped statement** at completion. | ◐ | host-install.sh as cited |
| **G7** | **Demo-box drill: provision → uninstall → re-provision** (runbook, supervised). Proves G2/G4/G5/G6 end-to-end before they touch Peti's hardware. | ○ | D4 |
| **G8** | **DR bring-up bind-override fix** (agent): `bring-up -mode dr` passes no `MountOverrides`; apply the spike's known-constant mp8/mp9 override reusing `bindMountOverrides`. Host-loss DR must actually work for a pilot box. | ○ | agent CONTEXT.md v0.74.0 OPEN; `SPIKE-dr-bindmount-source-2026-07-07.md` |
| **G9** | **Auth-on onboarding.** Auth + CSRF proven in campaign 2; remaining work is procedural: Peti's onboarding sets the dashboard password via the REAL settings pipeline (anti-F9) and verifies login. Runbook step in GL-7. | ◐ | controller v0.103.0 (F-C2-1 live-validated) |
| **G10** | **Local-API posture documented + spot-verified.** Token→vmid binding is structural (server.go:23, :53–55, :153 — verified this session); write it into the security notes, including the vmbr0-reachable surface and its TLS+token defense. One CC spot-check: a token minted for guest A is refused for a guest-B-scoped op (test exists? verify; add if hollow). | ◐ | localapi/server.go as cited |
| **G11** | **Pilot backup statement.** Assert local backups green on Peti's box at onboarding; the tester agreement states plainly: *no offsite copy yet — a dead/stolen box or dead drive without a second local target loses data*. Honesty is the sovereignty pitch. | ○ | D3; agent CONTEXT.md v0.66/0.67 (retarget reverted; Tier-1/2 split pending) |

**Non-gating / fast-follow (tracked, not blocking):** Impl-3 shared-box operator format gate
(existing wipe-binding + mkfs-guarded gates cover the dangerous core); `deviceRole`/`roleForMountPath`
unification refactor; Tier-1/Tier-2 offsite split + production WG endpoint; CGNAT true-sim retest;
BUNDLE cert/key/token migration item; hub-floor auto-update.

---

## 4. Task sequence

| Task | Type | Repo(s) | Covers | Depends on |
|------|------|---------|--------|------------|
| **GL-1** Publish/vouch/key-pin chain | ◐ **artifacts published 2026-07-08** (`a63cc715`); manifest bump + key pin = operator | felhom-agent, felhom.eu (hub manifest), operator UI | G1 | — |
| **GL-2** BYO-host profile + preflight + mandatory caps | ✔ **DONE 2026-07-07** (`02d63ed0`, validated) | felhom.eu/scripts | G2, G4, G5 | — |
| **GL-3** git.token rotate + scope-down | Operator (small) | Gitea admin + config template | G3 | — |
| **GL-4** Uninstaller gap-closure | Implementation | felhom.eu/scripts (+agent if drive-decommission call needed) | G6 | GL-2 (same file; sequence to avoid conflicts) |
| **GL-5** DR bind-override | Implementation (agent v0.75.0) | felhom-agent | G8 | — |
| **GL-6** Demo-box drill | RUNBOOK, supervised | — | G7 | GL-1, GL-2, GL-4 |
| **GL-7** Peti day-0 runbook + onboarding | RUNBOOK, supervised | felhom.eu/documentation/pilot | G9, G10, G11 | GL-6 green |

Recommended order: **GL-1 and GL-3 immediately** (operator-heavy, unblock everything), GL-2 next
(the biggest CC task), GL-5 in parallel (independent repo), then GL-4 → GL-6 → GL-7.

---

## 5. Decision log

- 2026-07-07 — D1–D6 locked by Viktor (see §1). BYO-host reframe adopted; break-glass step
  identified as a BYO blocker (G2); G1 stale-vouch chain identified as item #1; G3 security O1
  promoted into the gating list. Catalog healthcheck sweep confirmed CLOSED (live, `2ebe082a`).
- 2026-07-07 — **GL-2 SHIPPED** (host-install v1.10.0, felhom.eu `02d63ed0`), CC report validated
  against source at file:line. Two spec corrections adopted, both verified at baseline: (a) `--mode`
  already existed as the retired `provision|dr` — repurposed to `appliance|byo` with a guiding die
  for `provision` and the `dr` stub seam preserved; (b) the installer's config write defaulted
  `lan_resolver.enable: true` (baseline :1592) — the spec's "default-off" claim was agent-side only;
  byo now defaults it OFF and the post-write assert still catches a `--preserve-from` carrying true
  (appliance keeps historical default-on). The byo skip line intentionally avoids the literal
  "4b/8" (the spec's Scenario B forbade the string). Harness runs red-proofs on scratch copies;
  every invocation uses a throwaway `FELHOM_INSTALL_STATE_DIR` (new env override, spec-permitted).

- 2026-07-08 — **GL-1 EXECUTED** (felhom.eu `a63cc715`, felhom-agent `4c408467`), validated: agent
  0.74.0 published from the exact running felhom-pve bytes (sha independently re-verified via
  anonymous fetch, byte-exact); golden 0.103.0 baked clean-room on the virgin drill VM and
  published (612MB, rootfs+mp0+mp1 all in the vzdump); drill environment restored to `virgin`.
  Deviations sound: debian-13 template pveam-downloaded in-VM; bake detached via `systemd-run`.
  Gate 3c: packages anonymously readable → G3 rotation unblocked. **Follow-on probe found the
  whole Gitea instance anonymously readable (all repos public)** — operator ruling required
  (see G3 + operator actions).

## 6. Open questions & operator actions

**Operator actions (Viktor):**
- **Hub manifest bump** (Configs → Day-0 artifacts) — enter verbatim:
  agent `0.74.0` / `1ec3f58842edce1e9e32d022eaef15b7ff599c658c7a36bfe7833c708076af05`;
  golden `0.103.0` / `8481e8a14e2aa0abe3831cc40e5aea4c32f2017e7561c15dca8a674df6a6026e`.
  Until then fresh installs still land 0.63.0/0.98.3.
- **Gitea visibility ruling** (new, see G3): decide deliberately — (a) keep felhom repos public
  as an open-source posture, but audit + privatize `homelab-manifests` and the personal repos;
  or (b) lock the instance, which REQUIRES first proving the day-0 package fetch with a
  read-scoped token (the anonymous path dies). Recommendation: (a) — it preserves the working
  day-0 path; do the homelab-manifests secrets audit either way.
- G3 token work: mint the scoped read-only token, rotate out the package-WRITE one, fill the
  demo customer's empty `git.username`/`git.token` in the hub-served config.
- **Operator-key pin ceremony** on felhom-pve (+ the small day-0 follow-up — candidate GL-4 fold).
- Commit THIS doc to `felhom.eu/documentation/pilot/GO-LIVE-PACKAGE.md` (still absent after two
  tasks — paste it to CC with the next task so wrap-ups stop falling back to CONTEXT.md).

**Open questions:**
- G5: CPU/RAM cap values for Peti's guest (needed by GL-6/GL-7, not before).
- G11/GL-7: what is the local backup TARGET on Peti's box, given `felhom-pbs` is unreachable from
  his LAN — second local drive via per-app cross-drive backup, or a local PBS/vzdump storage?