docs: GL-7 closeout — tester agreement + hub 0.36.0 CHANGELOG/REPORT/CONTEXT + GO-LIVE G7 done

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
This commit is contained in:
2026-07-09 08:44:21 +02:00
parent 02c748eb2a
commit 0efdc78d75
5 changed files with 337 additions and 58 deletions
+39 -7
View File
@@ -11,6 +11,7 @@ catalog `2ebe082a` · felhom.eu `1a1e42ad`/hub v0.35.0 · host-install v1.9.1 ·
**Updated 2026-07-08 (GL-5b shipped):** felhom-agent **v0.76.0** live + PUBLISHED (`9828c5f7…f50b`) — restore-test full-fidelity + mount parity (new G12); the manifest bump target is now 0.76.0.
**Updated 2026-07-08 (GL-8):** felhom.eu `18a556a5` · host-install **v1.12.0** — BYO coexistence: F3 (leaf guard narrowed to Felhom guests → no `--allow-new-leaf` on a populated BYO host), F6 (byo refuses a foreign `:53`, never mutates), F1 (uninstall purges config `.bak*`). F6 live-proven on felhom-pve; harness 36/36. **GL-7 unblocked** (Peti's command needs no workaround).
**Updated 2026-07-08 (GL-6 DRILL COMPLETE):** felhom.eu `87ba30ae` · host-install **v1.11.3** (F4 resume fix shipped mid-drill; v1.11.2 anon-fetch ruling). Two full destroy/rebuild cycles on felhom-pve; manifest confirmed vouching agent 0.76.0 + golden 0.103.0. G2/G4/G5/G6/G7 LIVE-VALIDATED; G1 DONE; key-pin legs 14 proven. Record: `DRILL-GL6-2026-07-08.md`.
**Updated 2026-07-09 (GL-7 SHIPPED):** felhom.eu `844fbfa7` · **hub v0.36.0** LIVE (ArgoCD synced/healthy) — operator customer page now (a) MASKS the retrieval passphrase (reveal/copy, never baked into a copyable command) and (b) carries an interactive install-command generator (vanilla JS, real v1.12.0 flags only, download-then-run, JS-off static fallback). Tester agreement written (`PETI-tester-agreement.md`). G9/G10/G11 closed to their pilot-ready state (G9/G11 = onboarding-time actions, documented). **GL-7 DONE.**
---
@@ -68,10 +69,10 @@ Status legend: ✔ DONE · ◐ PARTIAL · ○ OPEN · ⚠ BLOCKER
| **G6** | **Uninstaller gap-closure (customer offboarding).** SHIPPED in host-install v1.11.0 (GL-4, `f7cc6a72`): 4b4 self-update-artifact removal (wrapper, .prev/.new.* slots, rollback unit, limits drop-in — derived from the guarded script itself), per-drive unmounts under `/mnt/felhom-drives` (plain umount ONLY, busy = warn+guidance, root-bind guarded), the **KEPT-vs-WIPED statement** in BOTH modes (drives/PBS/hub record/escrow/vaulted recovery credential named as living on), guest-only drive note from the bind store. **LIVE-VALIDATED (GL-6): two real uninstalls** — busy-drive prop handled (warn+guidance, no `-l/-f`, statement=retry), KEPT/WIPED statement verbatim, host audit clean, **drive data intact ×3** (witness hash matched every time). Findings F1 (config `.bak` residue) + F2 (mount-unit residue) filed. | ✔ | DRILL-GL6 §Phase 1/5 |
| **G7** | **Demo-box drill****DONE 2026-07-08.** Two full destroy/rebuild cycles (byo + appliance) + two uninstalls on felhom-pve, Viktor gating every phase; F4 found+fixed+proven mid-drill (v1.11.3), C7 + armed-key chain live-validated, data intact ×3. | ✔ | DRILL-GL6-2026-07-08.md |
| **G8** | **DR bring-up bind-override fix** (agent). SHIPPED as **v0.75.0** (GL-5, `b3446213`, LIVE on felhom-pve) + **scratch-DR live-validated end-to-end** (9310 from a real 9201 archive: mp0 200G + mp1 50G restored with content, real mp8/mp9 binds, zero unusedN, clean auto-teardown; the same op failed outright on v0.74.0). **TWO live-discovered PVE rules beyond the spike:** explicit-params restore requires an explicit rootfs AND silently drops unlisted mountpoints — the full param set now derives from the archive's embedded config (`ExtractArchiveConfig`, 200 under the scoped token; bind LAYOUT stays the known constants). | ◐ | agent v0.75.0 @ `b3446213`; REPORT.md. **Remaining: the full customer-data DR drill (GL-6/S5). (The publish follow-up landed as 0.76.0 — GL-5b/G12.)** |
| **G9** | **Auth-on onboarding.** Auth + CSRF proven in campaign 2; remaining work is procedural: GL-6 confirmed the dashboard password is **operator-set via the hub config** (anti-F9 pipeline), NOT a customer-dashboard field — until set the public dashboard is OPEN (**G10 exposure**). Rehearsed the location; **password-set + geo-restriction deferred to GL-7 with Peti** (a GL-7 prerequisite). | | DRILL-GL6 §Phase 2b |
| **G10** | **Local-API posture documented + spot-verified.** Token→vmid binding is structural (server.go:23, :5355, :153 — verified this session); write it into the security notes, including the vmbr0-reachable surface and its TLS+token defense. One CC spot-check: a token minted for guest A is refused for a guest-B-scoped op (test exists? verify; add if hollow). | | localapi/server.go as cited |
| **G9** | **Auth-on onboarding.** Auth + CSRF proven in campaign 2; the dashboard password is **operator-set via the hub config** (anti-F9 pipeline), NOT a customer-dashboard field — until set the public dashboard is OPEN (**G10 exposure**). **DONE for GL-7:** the password-set + geo-restriction are now an explicit onboarding step in the tester agreement (§7 step 3), gated as "onboarding not complete until set". The action itself is performed with Peti at his day-0. | | PETI-tester-agreement.md §7 |
| **G10** | **Local-API posture documented + spot-verified.** Token→vmid binding is structural (server.go:23, :5355, :153). **DONE:** written into the tester agreement (§2 table + §3 — vmbr0:8443 LAN surface, TLS + per-guest bearer token, cross-guest refusal). | | PETI-tester-agreement.md §2/§3 |
| **G12** | **Restore-test full-fidelity verification** (GL-5b, agent **v0.76.0**). The restore-test had GL-5 finding #2's mirror image: its live-source-config override path tripped PVE's drop-unlisted-mountpoints rule, so it boot-verified scratch guests WITHOUT their storage mpN. Now: params derive from the ARCHIVE's embedded config (`drRestoreOverrides`, same as DR) + a **mount-parity assert** (restored mpN set vs the archive's; missing/mispathed/undersized/extra = FAIL naming the delta) so the rule can never regress into a green light. `MountParity`+`MountInventory` ride the hub wire record (additive). Honest cost, measured: **3m4s** on the local tier (the extraction adds ~2min over data-less; cheaper than the DR-derived ~7m estimate; PBS tier will run longer). Live-proven on felhom-pve: parity ok, inventory mp0 200G + mp1 50G + 2 throwaways; a rotated-out archive volid refuses up front. | ◐ | agent v0.76.0 (published `9828c5f7…f50b`); agent REPORT.md. Remaining: parity-on-real-drift (GL-6 family) + PBS-tier runtime |
| **G11** | **Pilot backup statement.** Assert local backups green on Peti's box at onboarding; the tester agreement states plainly: *no offsite copy yet a dead/stolen box or dead drive without a second local target loses data*. Honesty is the sovereignty pitch. | ○ | D3; agent CONTEXT.md v0.66/0.67 (retarget reverted; Tier-1/2 split pending) |
| **G11** | **Pilot backup statement.** **DONE:** the tester agreement states it plainly (§4 — *no offsite copy yet; a dead/stolen box or dead drive without a second local target loses data*; §7 step 5 asserts local backups green at onboarding). **⚠ carries the one open pilot question:** Peti's local backup TARGET (does his box have a second disk/pool?) — resolved into `--acl-storages` at onboarding (agreement §6). | ✔ | PETI-tester-agreement.md §4/§6/§7 |
**Non-gating / fast-follow (tracked, not blocking):** Impl-3 shared-box operator format gate
(existing wipe-binding + mkfs-guarded gates cover the dangerous core); `deviceRole`/`roleForMountPath`
@@ -91,7 +92,7 @@ BUNDLE cert/key/token migration item; hub-floor auto-update.
| **GL-5** DR bind-override | ✔ **DONE 2026-07-08** (agent v0.75.0, `b3446213`, live-validated) | felhom-agent | G8 | — |
| **GL-6** Demo-box drill | ✔ **DONE 2026-07-08** (`DRILL-GL6-2026-07-08.md`; F4 fix v1.11.3 mid-drill) | felhom.eu/scripts (v1.11.3) | G7 | GL-1, GL-2, GL-4 |
| **GL-8** BYO coexistence hardening | ✔ **DONE 2026-07-08** (v1.12.0, `18a556a5`; F3/F6/F1) | felhom.eu/scripts | F3/F6/F1 | GL-6 |
| **GL-7** Peti day-0 runbook + onboarding | **NEXT** (GL-6 green) | felhom.eu/documentation/pilot | G9, G10, G11 + findings F1/F2/F3/F6/F7 | GL-6 green |
| **GL-7** Peti day-0 runbook + onboarding | **DONE 2026-07-09** (hub v0.36.0 `844fbfa7`; `PETI-tester-agreement.md`) | felhom.eu (hub + pilot docs) | G9, G10, G11 + findings F1/F2/F3/F6/F7 | GL-6 green |
Recommended order: **GL-1 and GL-3 immediately** (operator-heavy, unblock everything), GL-2 next
(the biggest CC task), GL-5 in parallel (independent repo), then GL-4 → GL-6 → GL-7.
@@ -171,6 +172,33 @@ Recommended order: **GL-1 and GL-3 immediately** (operator-heavy, unblock everyt
GL-6 scenario). Harness 36/36 + red-proofs RP-F3/F6/F1; GL-2/GL-4 regression green. F2/F7 remain
open (non-blocking). **GL-7 unblocked.**
- 2026-07-09 — **GL-7 SHIPPED** (hub **v0.36.0**, felhom.eu `844fbfa7`; deploy `02c748eb`). Two
coupled changes to the operator customer page, security-first: **(1) passphrase hardening** — the
per-customer retrieval passphrase was rendered in cleartext twice (the visible `#retrieval-pw` node
and baked into the Option-3 debug curl `X-Retrieval-Password:` header). Now masked by default
(bullet run) with reveal + copy controls (value in `data-secret`, the existing model), and the
Option-3 command carries a `<YOUR-RETRIEVAL-PASSWORD>` placeholder instead of the secret. A
zero-secret-in-DOM reveal-on-demand fetch is a noted follow-up, deliberately NOT scoped here.
**(2) interactive install-command generator** — the three hard-coded `<code>` blocks became a
client-side builder (vanilla JS, no framework/CDN/network) that assembles a live install command
from form controls, emitting ONLY real host-install **v1.12.0** flags, download-then-run shape,
with a graceful JS-off static fallback and a CustomerID prefilled from the server. The control
surface is a curated subset (mode/cores/memory/vmid/node/acl/pubkey/preserve + skip/dry-run/
preflight/allow-new-leaf); the seven dangerous/operator-only flags (`--force`, `--rotate-recovery`,
`--enable-oob`, `--remove-golden`, `--uninstall`, `--adopt-pool`, `--rescope-acl`) are NEVER offered
as controls. Render tests `TestTemplates_PassphraseHardened` + `TestTemplates_InstallGenerator`
cover both (green gate passed on the committed source); the passphrase test red-proofs by reverting
the Option-3 block to the raw secret. Deployed via ArgoCD (Synced/Healthy, rollout confirmed, live
image `felhom-hub:0.36.0`, `Listening on :8080`). **Tester agreement written**
(`PETI-tester-agreement.md`): BYO trust boundary (break-glass/OOB/WG all off, non-root agent,
pool-scoped token), honest limitations (no offsite backup, physically-removable-drive caveat F2,
pool-reassert-bring-up-only F7, `:53`-must-be-free F6), exit rights (uninstall keeps data), Peti's
box facts (80c/128 GB → caps 12/32768, `sajatfelhom.hu` tunnel re-point), the byo install command,
and the onboarding sequence (preflight → install → G9 password-set → tunnel → backup-green). G9/G10
closed; G11 closed with the one open pilot question folded into the agreement's pre-install
checklist. Note: the hub UI is operator-password-gated (CC cannot log in) — verification is the
render tests + the deploy checks, per the repo's stated policy.
## 6. Open questions & operator actions
**Operator actions (Viktor):**
@@ -194,6 +222,10 @@ Recommended order: **GL-1 and GL-3 immediately** (operator-heavy, unblock everyt
tasks — paste it to CC with the next task so wrap-ups stop falling back to CONTEXT.md).
**Open questions:**
- G5: CPU/RAM cap values for Peti's guest (needed by GL-6/GL-7, not before).
- G11/GL-7: what is the local backup TARGET on Peti's box, given `felhom-pbs` is unreachable from
his LAN — second local drive via per-app cross-drive backup, or a local PBS/vzdump storage?
- G5: CPU/RAM cap values for Peti's guest **RESOLVED: 12 cores / 32768 MiB** (his 80c/128 GB box).
- G11/GL-7: the local backup TARGET on Peti's box, given `felhom-pbs` is unreachable from his LAN.
**Still open, but now owned by the tester agreement** (`PETI-tester-agreement.md` §6): the ONE
thing to confirm with Peti before finalising his install command is **whether his server has a
second disk/pool** — if yes it becomes the `--acl-storages` backup target (real local resilience);
if no, backups share the guest's pool (degraded — one drive failure loses both, must be stated to
him). Resolve at onboarding; it changes only the `--acl-storages` value, nothing structural.
@@ -0,0 +1,175 @@
# Felhom pilot — tester agreement (Peti)
> The terms of the first external pilot: what Peti runs, what Felhom can and cannot do on his
> hardware, the honest limitations of the pilot, and his exit rights. Lives at
> `felhom.eu/documentation/pilot/PETI-tester-agreement.md`.
> Created 2026-07-09 (GL-7). Operator: Viktor. Tester: Peti.
>
> This is the operator-side record of the bargain. The points in §3, §4 and §5 are what Peti is told
> and asked to acknowledge at onboarding; that conversation happens in Hungarian — this document is
> the English source of truth those points are drawn from.
---
## 1. What the pilot is
Peti runs **Felhom as a guest on his own Proxmox server** — the **BYO-host** model. He stays root on
his hardware; Felhom is one LXC guest plus a narrow, non-root management footprint. This is not the
appliance model (Felhom-owned hardware); the whole point of the pilot is to prove the BYO trust
boundary on someone else's box.
**Peti's box:** an 80-core / 128 GB rack server already running his own workloads. The Felhom guest
is capped at **12 cores / 32768 MiB** (noisy-neighbour containment — the only thing standing between
Felhom and his other guests, since we do not control the host).
**Name:** `sajatfelhom.hu`. The existing Cloudflare tunnel for that domain must be re-pointed at the
new guest as part of onboarding (§7).
---
## 2. What runs on Peti's box
| Component | Footprint | Privilege |
|-----------|-----------|-----------|
| Felhom guest (LXC) | one container in the `felhom` pool, capped 12c/32 GB | unprivileged LXC |
| Host agent | one binary + systemd unit on the host | **non-root** (`felhom-agent` user) |
| Sudoers allowlist | `/etc/sudoers.d/felhom-agent` | narrow, `pct`/`vzdump`/mount verbs only |
| PVE API token | scoped to the `felhom` pool + named storages only | `Pool.*`/`VM.*`/`Datastore.*` on those objects |
| Local API | binds `vmbr0:8443` on his LAN | TLS + per-guest bearer token |
Nothing else. In particular, on a BYO host the installer **does not** touch these appliance-only
mechanisms:
- **Break-glass is OFF.** Felhom never runs `chpasswd` on `root@pam` and never vaults a host root
credential to the hub. Peti's root password is his own and is never seen or changed.
- **Out-of-band operator SSH is OFF** (`oob.enabled=false`). Felhom opens no operator tunnel into
his box.
- **The offsite WireGuard tunnel is OFF** (`wg_tunnel.enabled=false`). No Felhom-side connection
reaches back into his network.
These are asserted at install time (the byo config-write asserts `oob`/`wg_tunnel`/`lan_resolver`
are all off before the install is marked complete) and were live-validated on the demo host during
the GL-6 drill.
---
## 3. What Felhom can and cannot do (the trust boundary)
**Can:** provision and manage its own guest, run backups of its own guest to the storages named in
its ACL, report guest/host health to the hub, and — only with an operator signing key that Peti's
box has pinned — apply a signed agent self-update. Every host action goes through the narrow sudoers
allowlist, scoped to the `felhom` pool.
**Cannot:** read or modify Peti's other guests (the pool-scoped token is refused outside `felhom`);
change his root password; open an inbound operator session; touch his DNS service (see F6 below);
or reach storages outside its named ACL. The local API authorises every request strictly against the
calling token's own guest — a token minted for one guest is refused for any other
(`localapi/server.go`; spot-verified).
---
## 4. Honest limitations of the pilot
This is the data-sovereignty pitch stated plainly, not hidden:
- **No offsite backup yet (local-only).** Backups stay on Peti's own hardware. **A dead or stolen
box, or a dead backup drive with no second local target, loses data.** The offsite tier
(Tier-1/Tier-2 split + production WG endpoint) is a fast-follow, not part of this pilot. Local
backups being green is asserted at onboarding (§7).
- **Backups live on a physically removable drive on his own hardware (F2).** This is the flip side
of "no offsite" and is the sovereignty guarantee: his data never leaves his premises, and he can
physically remove the drive. It also means the resilience of the backup is exactly the resilience
of that one local drive/pool.
- **The `felhom` pool membership is asserted at bring-up only, not continuously healed (F7).** If
something outside Felhom moves the guest out of the pool while it runs, Felhom won't silently
re-grab it; a re-run (or `--adopt-pool`) repairs it. No autonomous host-level reassertion.
- **Port 53 must be free on his host (F6).** Felhom's guest DNS wants `:53`. If Peti already runs a
DNS service there, the installer **refuses and tells him** — it will not stop, mask, or reconfigure
a DNS service on a host Felhom does not own. He resolves the conflict; Felhom mutates nothing. This
is checked before anything is written (§7 pre-flight).
---
## 5. Exit rights (data sovereignty)
Peti can leave at any time and keep his data. The uninstaller:
- **Keeps** all drive data under `/mnt/felhom-drives/*`, the backup storage contents, the hub record,
and any vaulted recovery credential — and prints a KEPT-vs-WIPED statement naming each, in both
modes. Drives are unmounted (plain `umount`; a busy drive is warned, never force-unmounted).
- **Removes** only Felhom's own footprint: its guest, agent, sudoers, self-update artifacts, and
config (including `.bak*` siblings that could hold a hub API key).
- Refuses to touch any guest that is not a Felhom guest (it targets only guests carrying the
`/etc/felhom-bootstrap` bind mount).
Result: Peti walks away with his hardware untouched and his data on his own removable drives. This
was live-validated twice on the demo host (two real uninstalls, drive data intact each time, host
audit clean) during the GL-6 drill.
---
## 6. Pre-install checklist (confirm with Peti before day-0)
- [ ] **PVE major version 9.x** (the byo preflight refuses non-9.x unless forced).
- [ ] **Port 53 is free** on the host — no existing DNS service bound there (F6; preflight refuses
otherwise).
- [ ] **The `felhom` pool** either does not exist or is Felhom's to use.
- [ ] **VMID 9201 is free** (or pick another; the installer auto-selects a free one if 9201 is taken).
- [ ] **⚠ OPEN — the backup target.** `felhom-pbs` (the demo offsite datastore) is unreachable from
Peti's LAN, so the default `--acl-storages "local local-lvm felhom-pbs"` is wrong for him. We
must know **whether Peti's server has a second disk/pool** to use as the local backup target:
- If **yes** → name it in `--acl-storages` (e.g. `"local local-lvm <his-backup-storage>"`) so
backups land on a *different* physical drive than the live guest — real local resilience.
- If **no** (single pool) → backups land on the same pool as the guest; this is a degraded
posture (one drive failure loses both live and backup) and must be stated to Peti explicitly.
**Resolve this before finalising the install command below.**
- [ ] **`sajatfelhom.hu` Cloudflare tunnel** — confirm we can re-point it at the new guest (§7).
- [ ] **Customer `peti-felhom` exists in the hub** with real Gitea read creds filled in
(`git.username`/`git.token`) — the artifact fetch needs them, or relies on the anonymous path.
---
## 7. The install command
Built from the hub's customer page (`peti-felhom`**Setup Command** → BYO). The BYO shape is
deliberately minimal — a fresh box auto-fetches the golden template from Gitea, so no
`--force-gitea-golden` is needed:
```
sudo ./felhom-host-install.sh \
--customer-id peti-felhom \
--mode byo \
--cores 12 \
--memory 32768 \
--acl-storages "local local-lvm <PETI-BACKUP-STORAGE>"
```
`<PETI-BACKUP-STORAGE>` is resolved by the §6 open question. Add `--vmid`/`--node` only if 9201 is
taken or the host is multi-node. The command is **download-then-run** (the script is fetched and
inspected, never piped straight into a shell).
**Onboarding sequence:**
1. Run pre-flight first: same command with `--preflight-only` — prints PASS/FAIL, writes no state.
Confirms 9.x, `:53` free, ACL storages exist, pool membership.
2. Run the install. Peti sees the host-mutation disclosure and types his hostname to acknowledge.
3. **Set the dashboard password (G9).** The customer dashboard is open until the operator sets the
password via the hub config; set it at onboarding, along with the geo-restriction. Until then the
dashboard is publicly reachable — do not consider onboarding complete without it.
4. **Re-point the `sajatfelhom.hu` Cloudflare tunnel** at the new guest.
5. **Verify local backups are green** on his box (§4 — the honesty gate). Confirm a backup completes
to `<PETI-BACKUP-STORAGE>` and a restore-test passes mount-parity.
---
## 8. Acknowledgement
At onboarding, Peti is asked to acknowledge (in Hungarian):
1. Felhom runs as a guest on his hardware; he stays root and Felhom never changes his root password
or opens an inbound operator session.
2. **There is no offsite backup in the pilot.** His data stays on his premises on a removable drive;
a lost box or a failed backup drive without a second local copy loses data.
3. He can uninstall at any time and keep his data and hardware; Felhom removes only its own footprint.
4. This is a pilot: rough edges are expected, and the honest limitations above are the trade for full
data sovereignty.