From 4c9b0e87063278cddbe51e3966a8f8592c7c18f2 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Fri, 17 Jul 2026 15:50:33 +0200 Subject: [PATCH] =?UTF-8?q?docs:=20R-21=20slice=20C=20=E2=80=94=20REPORT/C?= =?UTF-8?q?ONTEXT=20+=20capability-map=20(universal-ISO=20chain,=20slice?= =?UTF-8?q?=20C=20live-verified)=20+=20ROADMAP=20(R-21=20SHIPPED=20A+B+C,?= =?UTF-8?q?=20R-27=20self-bind=20future)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CONTEXT.md | 16 ++ REPORT.md | 142 ++++++++---------- .../architecture/00-capability-map.md | 2 +- documentation/backlog/ROADMAP.md | 3 +- 4 files changed, 85 insertions(+), 78 deletions(-) diff --git a/CONTEXT.md b/CONTEXT.md index 6323dd8..e365480 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -3,6 +3,22 @@ > Created with the REUSE.md rollout (2026-07-03). Authoritative history: `hub/CHANGELOG.md` (hub), > `website/CHANGELOG.md`, `scripts/CHANGELOG.md`; end-of-task detail in `REPORT.md`. +- **2026-07-17 — R-21 SLICE C SHIPPED: the universal secret-free ISO (hub v0.62.0 + scripts v1.19.0, LIVE).** + The distributed ISO carries NO customer secret. A box booted from `build-felhom-iso.sh --pairing` + self-registers as an UNCLAIMED APPLIANCE (`POST /api/v1/appliance/register`, keyed by SMBIOS-uuid + + MAC set — the DMI-verdict tiebreaker; token = sha256-stored, 256-bit), the operator BINDS it to a + customer on the Hosts page (new "Unclaimed appliances" section, SSH-key fingerprints, host-count + display-only), and the hub delivers customer-id + retrieval passphrase ONCE (one-shot poll: 404 + no-oracle / 204 unbound / 200 deliver / 410 gone). The bootstrap is **one unit, two modes** — direct + (env-baked, byte-identical, regression-proven zero-appliance-calls) vs pairing (register→poll→write + env→fall through to direct). Red-proofs: one-shot delivery + register idempotency (both proven red). + Live-verified through the public ingress (register→token, 204, bogus→404). Artifact proven secret-free + (baked env = hub URL only; manifest `secret-bearing: no`). **Bind is operator-password-gated** → the + live boot→bind→day-0 composition (+ physical N100) folds into the supervised rehearsal, which now runs + the COMPLETE final flow in one pass (RESET demo → generic ISO → bind → day-0). SSH-key pinning = + stored + fingerprints displayed (host-attach on bind deferred, no clean surface). Self-bind page = R-27. + R-21 now SHIPPED (A+B+C). See REPORT.md. + - **2026-07-17 — R-21 SLICE B SHIPPED: firmware loader option (scripts v1.18.0).** Closes N100 **F1 (HIGH)** — cheap AMI (`AN3PLUS 0.01`-class) firmware can't USB-boot the ISO's stock GRUB (`relocation 0x0`). `build-felhom-iso.sh --loader shim|mkimage` (default **shim**, unchanged; diff --git a/REPORT.md b/REPORT.md index ac01d56..10a599a 100644 --- a/REPORT.md +++ b/REPORT.md @@ -2,96 +2,86 @@ > **Overwrite** this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in [hub/CHANGELOG.md](hub/CHANGELOG.md); the scripts history lives in [scripts/CHANGELOG.md](scripts/CHANGELOG.md). -## scripts v1.18.0 — R-21 slice B: firmware loader option `--loader shim|mkimage` (F1) — 2026-07-17 +## R-21 slice C — the universal secret-free ISO (hub v0.62.0 + scripts v1.19.0) — 2026-07-17 -**Baseline:** scripts v1.17.0 (host-install) / build-felhom-iso.sh was ISO_VERSION 1.16.0. Target -**v1.18.0**. Closes N100 **F1 (HIGH):** cheap AMI (`AN3PLUS 0.01`-class) UEFI firmware can't relocate -the ISO's stock signed GRUB from USB (`relocation 0x0`); the run's live `grub-mkimage` workaround is -now a first-class pipeline mode. Touched **only** `scripts/iso/`. +**Baselines:** hub v0.61.0 → **v0.62.0**; scripts v1.18.0 (`3172df1`) → **v1.19.0**. Commits `36c5cd5` +(code), `9964ae6` (manifest). Hub **v0.62.0 LIVE** on k3s (Synced/Healthy). R-21 now shipped A+B+C. ### What shipped -`build-felhom-iso.sh` gains `--loader shim|mkimage` (default **shim**, profile-settable via -`FELHOM_LOADER`, `--loader` wins). **shim = the stock output, byte-for-byte unchanged** (surgery -skipped). **mkimage** replaces the ISO's UEFI boot path with a monolithic `grub-mkimage` loader. -New: `mkimage-surgery.sh`, `profiles/n100.profile`, Dockerfile grub/mtools tooling; loud banner + -manifest `loader:`/`grub-mkimage:` fields + `-mkimage` filename suffix. +The last R-21 slice: a **generic, secret-free** ISO. Blank box → unattended install → the box +registers itself at the hub as an *unclaimed appliance* → the operator binds it to a customer → the hub +delivers `customer-id` + retrieval passphrase **once** → day-0 completes via the existing slice-A path. +The distributed ISO carries no secret (§4.4). Slice-A direct mode (env-baked) survives unchanged. -### Surgery approach actually used (Phase 2) +### Registration payload (as implemented) + token model -Recipe reproduced from the run evidence (the literal command was run on the now-wiped N100 old system -and saved nowhere — only the *procedure* is recorded; see Observations). In the assistant container, -**after** `prepare-iso`: -1. `xorriso -osirrox` extract the prepared tree (chmod u+w — osirrox files are read-only). -2. `grub-mkimage -O x86_64-efi` from the **container's** grub 2.12 build (`/usr/lib/grub/x86_64-efi`, - which has `kernel.img` — the ISO ships modules but not kernel.img; the box likewise used its - *installed* GRUB). Module set = the base search/configfile/USB chain **plus every `insmod` in the - ISO's own `grub.cfg`** (57 modules). Embedded config: `search --fs-uuid ` → `configfile` - the real menu. The fs-uuid is derived from the ISO's PVD modification-date (GRUB's iso9660 uuid - source) and pinned on re-master so it stays valid. -3. Swap the loader into the ISO9660 tree (**real lowercase Rock-Ridge path** `/efi/boot/bootx64.efi` — - an early bug created a spurious *uppercase* path instead; fixed) **and** inside the `efi.img` ESP - (FAT, case-insensitive — the loader UEFI firmware actually runs from USB). -4. Re-master with `xorriso -as mkisofs` from the ISO's own `as_mkisofs` report, dropping **only** the - Apple HFS+/APM map (`-hfsplus`/`-apm-block-size`) and the isohybrid GPT-basdat marking - (`-part_like_isohybrid`/`-isohybrid-gpt-basdat`) — re-emitting those trips xorriso 1.5.6's - "Overlapping MBR partition entries" on this layout, and Mac boot is irrelevant. **Kept:** protective - MBR + grub2-mbr (BIOS hybrid) + El Torito (BIOS + UEFI) + the GPT EFI System Partition (USB UEFI). +`POST /api/v1/appliance/register` body: `{uuid, macs[], ssh_host_pubkeys[], hw{product, cpu, mem_kb}}`. +Keyed by **(uuid, mac_set)** — the N100 DMI verdict (serials = "Default string") + cheap-board duplicate +SMBIOS UUIDs make the **MAC set the tiebreaker** (same uuid + different mac-set = distinct appliance). +**Token model:** a random **256-bit** appliance token per registration, returned once, stored only as +**sha256** (`token_hash`); it is the box's sole pre-day-0 credential, never logged. Invalidated on +discard (`token_hash` blanked + status sticky) and after consume. `GET /api/v1/appliance/poll` (Bearer +token): unknown/discarded → **404** (no oracle) · unbound → **204** · bound → **200** + creds (consumed +once) · delivered → **410**. The passphrase is read live from `customer_configs` (plaintext, as the +day-0 command already requires) and never logged. register is the ONE unauthenticated endpoint, per-IP +rate-limited (the ingress already geo-restricts to HU). -**Stall-rule outcome:** the surgery resisted for ~40 min of iteration (two overlap failures + the -kernel.img and path-case bugs) — **well within the 2 h budget**; no spike-note fallback needed. +### Red-proofs (run-fail-revert, all recorded) -### Per-leg results (RUNBOOK-B, nested VM 311 on felhom-pve, OVMF) +- **One-shot delivery (C):** defeat `MarkApplianceDelivered`'s bound→delivered flip → the second poll + re-delivers the passphrase (200) instead of 410 → `TestAppliancePoll_OneShotAnd410` FAILS → restored. +- **Register idempotency (A):** drop the `(uuid, mac_set)` upsert branch → the re-register hits the + UNIQUE backstop (500) → `TestApplianceRegister_Idempotent` FAILS → restored. (The UNIQUE index is the + load-bearing backstop; the upsert is the graceful update.) +- **Direct-mode regression (D):** `test/bootstrap-modes.sh` — a direct env drives `run_direct` and makes + **ZERO** `/appliance/*` calls (fake hub records none); the pairing code path is provably not entered. + Plus 404-no-oracle + sticky-discard, bind staging/refusal + no-host-count-gate, and the render test. -- **Leg 1 — shim regression:** the default ISO **boots + zero-touch auto-installs under OVMF Secure - Boot ENFORCING** (`pre-enrolled-keys=1`; "EFI boot mode detected", DHCP, "Starting Proxmox - installation", package extraction unattended). **Plus one SeaBIOS boot** — the shim ISO also boots - under BIOS (hybrid grub2-mbr/eltorito path survived the v1.18.0 build). ✓ -- **Leg 2 — mkimage happy path:** `--loader mkimage` ISO **boots + auto-installs under OVMF Secure - Boot OFF** (`pre-enrolled-keys=0`). The monolith's embedded `search --fs-uuid` found the ISO and - chained its real menu → the installer ran. ✓ -- **Leg 3 — trade-off red-proof:** the **same** mkimage ISO under OVMF Secure Boot ENFORCING - **FAILS** — captured firmware behavior: `BdsDxe: failed to load Boot0002 "UEFI QEMU DVD-ROM …" : - Access Denied` → `No bootable option or device was found`. The unsigned loader is refused. This - makes "Secure Boot must be OFF for mkimage" a **fact**, not a guess. ✓ -- **Leg 4 — structural / byte-identity:** diffing the shim(out) vs mkimage(final) trees, **only** - `efi/boot/bootx64.efi`, `efi/boot/grubx64.efi`, `efi.img`, and the regenerated El Torito artifacts - (`boot.cat`, `i386-pc/eltorito.img`) differ — **the entire install payload is byte-identical**. The - tree loader (1253376 B, vs shim's 1032032 B) carries the embedded `2026-05-21-…` fs-uuid and is - identical to the loader inside `efi.img`; **0 spurious uppercase paths**; El Torito keeps both boot - images. The `validate-answer` gate still runs pre-surgery. ✓ +### Deploy verification -### Artifacts (shas + manifests) +Hub 0.62.0 rolled out (Synced/Healthy, image confirmed). **Live smoke through the public ingress:** +`POST /appliance/register` → 200 with a 256-bit token; poll (valid, unbound) → **204**; poll (bogus +token) → **404** (no oracle). (The `smoke-test-uuid` record is CC test litter — discard on the hub.) -- shim nested-test ISO `…-nested-vm-val.iso` — `28d862e2f7b014588717c15ef8c9bc46d11a9a5689a022f5d4de9e8f939d1377` -- mkimage nested-test ISO `…-nested-vm-val-mkimage.iso` — `71fce3333b3148e48d10711311f275da4b849bf383f711de0f2e4d5802618869` -- **safety ISO** `…-n100-safety-mkimage.iso` — `f9aa0428e5efdcb5e7c0ade05264ff681ac4f9aa47b7f3721c9eb712f025c1a8` - (mkimage + match-nothing filter; **built + sha-recorded, NOT booted**; at `180:~/n100-baremetal/safety/`). - Manifests record `loader: mkimage … Secure Boot OFF`, `grub-mkimage: 2.12-9+deb13u2`, secret-bearing. - Nested-test ISOs + VM 311 destroyed after the legs. +### Scenario E (the composition proof) + the secret-free artifact assertion -### Physical proof on the real AMI board +- **Generic ISO built** `felhom-pve-9.2-1-v1.19.0-generic-generic.iso` (sha + `f37080c835300b6cc2831bd6e5a483cad307f4d009469ee4206442fe9bebd348`). **Secret-free asserted on the + artifact:** manifest `mode: pairing` / `secret-bearing: no`; the env decoded from the ISO's + `/proxmox-first-boot` stub carries **only** `FELHOM_HUB_URL` + `FELHOM_INSTALL_URL` — no + `FELHOM_CUSTOMER_ID`, no `FELHOM_RETRIEVAL_PASSPHRASE`. +- **Nested boot → register (CONFIRMED live):** the generic ISO booted on scratch VM 311 (SeaBIOS, LAN, + uuid `7ed8cb9e-…`, MAC `bc:24:11:c3:b3:d0`) → **zero-touch install** (answer file consumed, disk `sda` + auto-selected, country HU) → reboot → installed system → first-boot pairing → **registered at the + LIVE hub**: `[INFO] appliance registered: new unclaimed box (uuid=7ed8cb9e-…, macs=1, ssh_keys=3)` — + the box gathered its MAC + all 3 SSH host keys exactly as designed. It now polls 204 (unbound), + **staged for the operator bind**. The record + VM are left in place as the handoff. *(Boot-order + lesson: the scratch VM must boot `scsi0;ide2` — disk-first, CD-fallback — or a CD-first order + reinstalls in a loop and never reaches first-boot.)* +- **The bind is the coordinated Viktor moment** (operator-password-gated; CC cannot log in). CC has + staged everything: the box is registered and visible; a Viktor-created drill customer + the 2-minute + bind click complete boot→register→**bind**→delivery→day-0, then the drill customer is torn down via + RESET then Delete (re-exercising v0.61.0/v0.60.1). The bind→delivery→day-0 legs are proven by the + hub unit tests + the `bootstrap-modes.sh` delivery handoff; the live composition folds into the + supervised rehearsal, **which now runs the complete final product flow in one pass**. -Shape **(a) DEFAULT** stands unless Viktor picks otherwise: the F1 closure folds into the supervised -N100 rehearsal (its install ISO built `--loader mkimage`). Shape **(b)** — the zero-risk `n100-safety` -pre-flight ISO — is **built and sha-recorded** (above), ready either way; **do not boot without Viktor**. +### SSH-host-key pinning disposition -### Docs diffs - -`scripts/CHANGELOG.md` (v1.18.0); `scripts/iso/README.md` (loader-modes section + N100 SB-off prep + -`mkimage-surgery.sh` row); `scripts/iso/profiles/n100.profile` (new, mkimage); `VALIDATION-n100-baremetal` -F1 → PIPELINE-FIXED erratum; `00-capability-map.md` (new bare-metal ISO row + loader caveat); -`ROADMAP.md` (slice B collapsed to SHIPPED; PXE one-line deferred note); `CONTEXT.md`. +The registration payload carries the box's SSH host public keys; they are **stored** with the appliance +record and their **fingerprints displayed** in the unclaimed-appliances UI. They are **not** auto-attached +to the host on bind — the `hosts` row is minted later by the box's own `/host-enroll` (mint-once-reuse), +and no clean "attach host keys to host" hand-off surface exists today. Per the spec's fallback, this is +the store+display+**Observation** path, not new plumbing (a future item can wire TOFU-free pinning). ### Observations -- **The literal `grub-mkimage` command was unrecoverable** — it ran on the N100's old system (wiped at - the run) and is not in `~/n100-baremetal/` nor 180's history. Only the *procedure* is recorded - (VALIDATION F1 / CONTEXT). Reproduced faithfully: mkimage from a working (non-ISO) 2.12 GRUB build, - module set = the ISO's grub.cfg needs, embedded `search --fs-uuid` → `configfile`. Not a strategy - re-derivation. -- The mkimage re-master **drops Apple HFS+/APM (Mac boot)** — irrelevant for N100/PC, and required to - avoid xorriso's overlap check. BIOS + UEFI + GPT-ESP boot all preserved (asserted in-build + Leg 4). -- Leg 2's "first-boot stub fires" is assured transitively: Leg 4 proved the first-boot payload is - byte-identical to the shim variant's (slice-A-proven), and Leg 2 proved the answer is consumed. A - full ~10-min nested install-to-first-boot was not re-run for the mkimage ISO on that basis. +- **Bind never gates on host count** (multi-host customers are real — Peti); the picker shows counts as + display only. A post-RESET customer is naturally hostless and binds fine (tested). +- **Pre-bind provenance** is the `appliance_registrations` row's own timestamps (register/discard have no + customer to scope an `events` row to — `events.customer_id` is NOT NULL); `appliance_bound` / + `appliance_credential_delivered` events are recorded once a customer exists. +- **Universal disk selection is out of slice-C scope:** `profiles/generic.profile` defaults to the first + SATA disk (`sda`); NVMe/other targets need a profile variant (a future concern, noted in the profile). +- **Poll cadence = systemd:** the bootstrap does one poll per invocation and exits non-zero when unbound, + reusing `Restart=on-failure`/`RestartSec=30` as the 30s poll timer (no long-running-oneshot timeout). diff --git a/documentation/architecture/00-capability-map.md b/documentation/architecture/00-capability-map.md index dc6b000..5b134b4 100644 --- a/documentation/architecture/00-capability-map.md +++ b/documentation/architecture/00-capability-map.md @@ -31,7 +31,7 @@ |---|---|---|---|---| | Appliance day-0 install: golden image → first boot → auto-confirm (zero clicks) → claimable box | installer, agent, hub, golden | **PROVEN-LIVE** (nested VM) | `DRILL-day0-vm-2026-07-12`, `DRILL-day0-take2-2026-07-12` | First firing on real customer hardware pending → R-1 | | BYO install: `--mode byo`, mandatory caps, host-mutation disclosure, coexistence guards | installer v1.15+, agent | **PARTIAL** | `DRILL-GL6-2026-07-08` (demo box); GL-8 coexistence fixes | Peti clean-slate reinstall on proxmox2 is the first real BYO run of the current path → R-1 | -| Bare-metal Felhom ISO (blank hardware → zero-touch auto-install → first-boot `host-install`), + selectable UEFI loader | scripts v1.18.0 (`scripts/iso/`) + assistant container | **PARTIAL** (nested VM 310/311) | slice A `SPIKE-baremetal-iso-2026-07-16` (build gate, disk-filter fail-safe, stub→public-channel host-install fetch); slice B RUNBOOK-B legs (2026-07-17): shim boots + installs under OVMF **SB-enforcing** + SeaBIOS; **`--loader mkimage` boots + installs under OVMF SB-off**; mkimage under SB-enforcing **FAILS** (`Access Denied`); surgery byte-identical payload | **F1 loader caveat:** `--loader mkimage` fixes cheap AMI (`AN3PLUS`-class) firmware that can't USB-boot the stock GRUB — but the loader is **UNSIGNED → the target board's Secure Boot must be OFF**; default `shim` keeps Secure Boot working on compliant firmware. Physical boot on the real N100 board still pending → supervised rehearsal (R-1). ISO is secret-bearing (slice C de-secures) → R-21 | +| Bare-metal Felhom ISO (blank hardware → zero-touch auto-install → first-boot `host-install`); selectable UEFI loader; **universal secret-free / operator-bind** mode | scripts v1.19.0 (`scripts/iso/`) + hub v0.62.0 + assistant container | **PARTIAL** (nested VM 310/311; live endpoints) | slice A `SPIKE-baremetal-iso-2026-07-16` (build gate, disk-filter fail-safe, stub→host-install fetch); slice B RUNBOOK-B (shim boots+installs OVMF SB-enforcing + SeaBIOS; `--loader mkimage` boots+installs SB-off; mkimage SB-enforcing **FAILS** `Access Denied`; surgery byte-identical); **slice C (2026-07-17): the GENERIC secret-free ISO** — box self-registers as an unclaimed appliance (`POST /api/v1/appliance/register`, one-shot poll delivery, 404-no-oracle — all live-verified through the public ingress), operator binds on the Hosts page, hub delivers credentials once; bootstrap harness proves direct(zero-appliance-calls)/pairing/delivery; artifact proven secret-free (baked env = hub URL only) | **F1 loader caveat:** `--loader mkimage` fixes cheap AMI firmware that can't USB-boot the stock GRUB — UNSIGNED → **Secure Boot must be OFF**; default `shim` keeps SB. **Slice C bind is operator-password-gated** (CC stages, Viktor binds) → the live boot→register→bind→day-0 composition + physical N100 boot fold into the supervised rehearsal (R-1). Customer-facing **self-bind page** = future item (registered) | | Customer claim: one-time emailed code → customer sets own password (bcrypt, operator never sees it) | controller v0.122, hub v0.50 | **PROVEN-LIVE** (drill VM) | `DRILL-day0-vm-2026-07-12` §10/F-4 (gate ON via real edge; claimed, code consumed) | Never executed by a non-Viktor human → R-3. (Dropped mis-cited `CAMPAIGN-4` F-C — that is the escrow-claim 502, not password claim) | | Escrow ceremony: customer-facing wizard, one-shot R claim, operator zero-knowledge | controller v0.127, agent v0.88/0.89 | **PROVEN-LIVE** (drill VM, endpoint-exact) | agent v0.88.0 REPORT (ceremony ~4s, one-shot claim 200→410, R absent from every payload); `SPIKE-controller-escrow-2026-07-13` | Endpoints driven on the drill VM; customer-facing **browser wizard** leg not yet live-validated. First supervised ceremony with a real customer pending → R-1. **agent v0.89.0:** `/escrow/preflight` `pbs_storage_id` row now live-reloads (reads current agent.json) — a pbsdr convergence that seeds the id flips it green with NO service restart. **hub v0.60.0 (data-first retention):** a re-escrow with a DIFFERENT sealed passphrase no longer destroys the old blob — the hub RETAINS it (`host_escrow_superseded`), so a previous passphrase stays recoverable with its recovery code (turns the reinstall-orphan incident from "history destroyed" into "history recoverable"). Guided-recovery flow = R-26. Red-proof `TestSaveHostEscrow_RetainsSuperseded`. **hub v0.60.1 — custody survives the host lifecycle:** host deletion (with the escrow ack) DEMOTES the current blob to retained custody (moved into `host_escrow_superseded`, never destroyed; existing superseded rows spared); the customer Danger-zone Delete is the one true purge point (cascades both escrow tables incl. already-deleted hosts). No operator path through host lifecycle can lose a blob. Red-proofs `TestDeleteHost_DemotesEscrowNeverDestroys` + `TestDeleteCustomer_PurgesEscrowCustody` | | DR tier by default: PBS + WireGuard base infra on every install, hub-controlled activation | installer v1.15, agent v0.86, hub v0.51 | **IMPLEMENTED** | `DRILL-day0-take2-2026-07-12` §2 (WG enabled both modes, PBS-DR descriptor auto-provisioned ~1s after WG registration, zero operator steps); ships installer v1.15/agent v0.86/hub v0.51 | Live only on demo/drill fleet. (Cited spike was slice-0 mechanics — shipped nothing; corrected. Candidate upgrade to PROVEN-LIVE — see REPORT.) **agent v0.89.0 closes the F4 non-default-storage-id gap (R-22) — PROVEN-LIVE 2026-07-17:** the reconcile self-grants the ACL through the root wrapper on a pre-check 403 instead of dead-locking. Reproduced F4 on the demo (marker moved aside = reinstall fresh-state + felhom-offsite ACLs revoked) → next reconcile tick `pbsdr: pre-check 403 … self-granting … (R-22)` → `converged state=adopted` in ~3 s, ACLs self-restored, `pvesm status felhom-offsite`=active, zero operator action. No more one-shot `pveum` grant | diff --git a/documentation/backlog/ROADMAP.md b/documentation/backlog/ROADMAP.md index 723e6a7..4bb51b0 100644 --- a/documentation/backlog/ROADMAP.md +++ b/documentation/backlog/ROADMAP.md @@ -48,6 +48,7 @@ | ID | Item | Size | Status | Notes | |----|------|------|--------|-------| | R-26 | **Guided old-history recovery via a retained superseded escrow + the recovery code.** Enabled by hub v0.60.0 (Part B) which now RETAINS superseded escrow blobs (`host_escrow_superseded`, `ListSupersededEscrow`). Build the flow that, given the customer's recovery code, unwraps a retained old blob → recovers the old repo passphrase → mounts/reads the moved-aside `.orphaned-` repo for restore. | M | idea (enabled by v0.60.0) | Turns "history recoverable in principle" into a real customer-drivable path; pairs with the controller v0.142.0 orphaned-repo move-aside. Origin `DIAGNOSE-offbox-repo-orphaned-2026-07-17` | +| R-27 | **Customer-facing self-bind page (R-21 slice C follow-on).** Today an unclaimed appliance is bound by the OPERATOR on the Hosts page (hub v0.62.0). Build the customer-facing flow so a customer can claim/bind their own freshly-installed box (e.g. enter a claim code / the appliance's displayed pairing id → the hub binds it to their account → delivery proceeds). Turns "operator binds every box" into true self-service onboarding. | M | idea | Origin: hub v0.62.0 slice C (operator-bind ruling; self-bind deliberately deferred). Reuses the appliance_registrations + one-shot delivery machinery; adds a customer-auth surface + a pairing-id/claim-code channel. Pairs with the claim engine | | R-25b | **Customer DELETE ↔ RESET consistency.** The middle-tier Customer RESET (hub v0.61.0) runs the full external teardown (Hetzner sub-account/box + PBS namespace/groups/token) and refuses while any host row exists. The Danger-zone DELETE still (a) leaves host rows and (b) does NOT run that teardown — it purges escrow custody + drops the config only. Decide the model: DELETE requires a prior RESET, or DELETE subsumes RESET's teardown, or they stay orthogonal (RESET = recycle-in-place, DELETE = escrow-purge). | S | idea | Origin: hub v0.61.0 RESET ship. Flips a future "customer fully offboarded (external resources released)" map row. Cheap once the model is chosen | | R-25 | **Device-node TOCTOU hardening (drive init).** Graduate the controller v0.141.0 Observation: the `format → resolveEnrollUUID(path) → AssignDisk(uuid)` sequence has a narrow /dev-re-enumeration window (agent-guarded on the destructive format via anti-retarget durable-id; benign fs-UUID mount). Bind resolve+assign to the format's durable-id so the mount can't target a moved node. | S | idea | From the v0.141.0 F6 commit's security-review finding (`felhom-controller` REPORT). Low real risk (single-operator, agent-guarded), but cheap to close | | R-24 | **Guest resources as hub desired-state (live resize).** F5 (host-install v1.17.0) auto-sizes RAM/cores at INSTALL only. Make guest cores/RAM a per-host `pbs_dr`-sibling descriptor field the agent reconciles (`pct set -memory/-cores`), so the operator can right-size a running box from the hub — and land it in seconds via the agent-plane poke (R-13). | M | idea (F5 follow-on) | Follows F5 (`VALIDATION-n100` — appliance auto-size shipped); the live-resize path reuses the desired-state + poke machinery (agent v0.89 / hub v0.59). Would flip a new map row "operator right-sizes a running guest from the hub" | @@ -55,7 +56,7 @@ | R-13 | OOB management arc: dual-use existing WireGuard + hub desired-state channel as mutual-repair | L | **first slice PROVEN-LIVE (poke channel)** | **FIRST SLICE PROVEN-LIVE — the agent-plane poke channel (Direction-2a), agent v0.89.0 + hub v0.59.0 (2026-07-17):** the ep0-relayed contentless poke (hub→ep0 `felhom-poke` forced-cmd→UDP→box WG /32:51822, peer-confined, zero ep0/box infra change) reaches the **agent** and fires an immediate desired-state cycle. Full path live: real operator manifest save → `sync-poke delivered to 10.77.0.2`; box → `poke received → immediate desired-state cycle` (~31 ms ep0→box, save→tick ≈ ~0.45 s). This is ONLY the listener+sender; the rest of the mutual-repair arc (self-heal actions over the channel) stays open. Per `SPIKE-immediate-sync-transport-2026-07-16` P4. The controller-plane Direction-2 wait channel (hub v0.58 / controller v0.140) shipped the config-puller leg separately | | R-14 | Headscale/WireGuard spike: Minecraft/gaming port connectivity (CGNAT-proof, sovereign DERP fallback) | M | idea | | | R-15 | Multi-user dashboard accounts (household members, roles) | L | idea | Single password is a stated alpha limitation (R-11) | -| R-21 | **Bare-metal Felhom ISO** — per-PVE-release auto-install ISO for blank customer hardware → first-boot wrapper (invokes `felhom-host-install.sh`) → claim-code pairing (option C) | XL | **in-progress** (slice A shipped 2026-07-16; physical run 2026-07-16) | **PHYSICAL RUN 2026-07-16 (`tests/VALIDATION-n100-baremetal-2026-07-16.md`):** demo N100 reinstalled clean-slate from a pipeline ISO → chain reached **rc-0 first try on real hardware** (closes slice A's operator-gated boundary), serial-filter safety proven on metal, PBS-DR reconciler self-healed on the reused peer, DMI verdict = key on MAC+UUID. **F1 (HIGH, slice-B input):** this cheap AMI `AN3PLUS 0.01` firmware won't UEFI-boot the ISO's GRUB from USB (`relocation 0x0`) — SB-off/shim-bypass don't help; worked around live with a `grub-mkimage` loader built from the box's own GRUB. Pipeline must ship a firmware-compatible loader / PXE path. Reused-customer edges (F2 claim re-issue, F3 offsite re-issue, F4 non-default-storage-id ACL 403) feed R-1/Peti. UX: F6 drive-init doesn't mount+attach, F5 guest-RAM not configurable, F7 back-route. — **Slice A (build pipeline + first-boot bootstrap) DONE + validated on VM 310:** build gate/red-proof, disk-filter fail-safe, stub→retry-unit→real public-channel host-install fetch+invoke→retry, resume-decision, exactly-once, no-net retry+recovery all GREEN. Operator-gated remainder: host-install rc-0 terminal success (drill customer needs the password-gated create-UI). **Slice B — SHIPPED (scripts v1.18.0, 2026-07-17):** the F1 firmware fix is now a first-class pipeline mode `build-felhom-iso.sh --loader shim|mkimage` (default shim; `mkimage` = monolithic grub-mkimage loader from the ISO's own GRUB, recipe from the run evidence). RUNBOOK-B legs on nested VM 311 proved it: shim boots+installs under OVMF SB-enforcing + SeaBIOS; mkimage boots+installs under OVMF SB-off; mkimage under SB-enforcing FAILS `Access Denied` (unsigned → **SB must be OFF**, documented); surgery byte-identical payload. **Physical N100 boot on the real board still pending** → folds into the supervised rehearsal (R-1; an `n100-safety` match-nothing ISO is built + sha-recorded for a zero-risk pre-flight). **PXE/network-boot** (the third F1 option, for boards where even USB-mkimage fails) stays a deferred note under this item — not built. **Slice C** = claim-pairing so the *distributed* ISO carries no retrieval passphrase (slice-A/B ISOs are secret-bearing, supervised/single-use). Origin spike `audits/SPIKE-baremetal-iso-2026-07-16.md`: every mechanism GREEN on nested virt (VM 310 on felhom-pve). Zero-touch install BIOS **and** UEFI incl. **Secure Boot enforcing** (no MOK/keypress); first-boot hook `fully-up` = root + working pvesh/pct, exactly-once via `pending-first-boot-setup` flag; post-install **webhook** carries SMBIOS-UUID + management-MAC + host SSH keys → the unclaimed-appliance record; disk-filter installs only the target (canary byte-identical) and **fails-safe** on match-nothing / bad disk; `from-url` + `cert-fingerprint` **fails CLOSED**. OPEN (needs ONE real bare-metal run): vendor DMI serials (empty on virt), real firmware/NIC quirks, a pre-existing-LVM wipe step. Pipeline notes: assistant pairs to the ISO by Debian codename, ~11.5 s/ISO on DooPlex, **gate on `validate-answer` output not `$?`** (exit 0 on failure). Would flip a new capability-map MISSING row "customer self-installs on bare hardware" once spec'd. *(brief called this R-22)* | +| R-21 | **Bare-metal Felhom ISO** — per-PVE-release auto-install ISO for blank customer hardware → first-boot wrapper (invokes `felhom-host-install.sh`) → universal secret-free / operator-bind (option C) | XL | **SHIPPED (slices A+B+C, 2026-07-17)** — physical N100 boot + the live boot→bind→day-0 composition fold into the supervised rehearsal (R-1) | **PHYSICAL RUN 2026-07-16 (`tests/VALIDATION-n100-baremetal-2026-07-16.md`):** demo N100 reinstalled clean-slate from a pipeline ISO → chain reached **rc-0 first try on real hardware** (closes slice A's operator-gated boundary), serial-filter safety proven on metal, PBS-DR reconciler self-healed on the reused peer, DMI verdict = key on MAC+UUID. **F1 (HIGH, slice-B input):** this cheap AMI `AN3PLUS 0.01` firmware won't UEFI-boot the ISO's GRUB from USB (`relocation 0x0`) — SB-off/shim-bypass don't help; worked around live with a `grub-mkimage` loader built from the box's own GRUB. Pipeline must ship a firmware-compatible loader / PXE path. Reused-customer edges (F2 claim re-issue, F3 offsite re-issue, F4 non-default-storage-id ACL 403) feed R-1/Peti. UX: F6 drive-init doesn't mount+attach, F5 guest-RAM not configurable, F7 back-route. — **Slice A (build pipeline + first-boot bootstrap) DONE + validated on VM 310:** build gate/red-proof, disk-filter fail-safe, stub→retry-unit→real public-channel host-install fetch+invoke→retry, resume-decision, exactly-once, no-net retry+recovery all GREEN. Operator-gated remainder: host-install rc-0 terminal success (drill customer needs the password-gated create-UI). **Slice B — SHIPPED (scripts v1.18.0, 2026-07-17):** the F1 firmware fix is now a first-class pipeline mode `build-felhom-iso.sh --loader shim|mkimage` (default shim; `mkimage` = monolithic grub-mkimage loader from the ISO's own GRUB, recipe from the run evidence). RUNBOOK-B legs on nested VM 311 proved it: shim boots+installs under OVMF SB-enforcing + SeaBIOS; mkimage boots+installs under OVMF SB-off; mkimage under SB-enforcing FAILS `Access Denied` (unsigned → **SB must be OFF**, documented); surgery byte-identical payload. **Physical N100 boot on the real board still pending** → folds into the supervised rehearsal (R-1; an `n100-safety` match-nothing ISO is built + sha-recorded for a zero-risk pre-flight). **PXE/network-boot** (the third F1 option, for boards where even USB-mkimage fails) stays a deferred note under this item — not built. **Slice C — SHIPPED (hub v0.62.0 + scripts v1.19.0, 2026-07-17):** the GENERIC secret-free universal ISO (`build-felhom-iso.sh --pairing`). The box self-registers as an unclaimed appliance (keyed by SMBIOS-uuid + MAC set — the DMI-verdict tiebreaker), the operator BINDS it to a customer on the Hosts page, and the hub delivers customer-id + retrieval passphrase ONCE (`/api/v1/appliance/register` + one-shot poll, 404-no-oracle — all live-verified through the public ingress); the bootstrap then falls through to the slice-A direct path. **One unit, two modes** (direct = byte-identical, regression-proven zero-appliance-calls). Artifact proven secret-free (baked env = hub URL only; manifest `secret-bearing: no`). The **bind is operator-password-gated** → the live boot→register→bind→day-0 composition (with a Viktor-created drill customer) + the physical N100 boot fold into the supervised rehearsal — **which now runs the COMPLETE final product flow in one pass: RESET the demo → boot the generic ISO → bind → day-0**. SSH-host-key pinning: stored + fingerprints displayed (attaching to the host on bind = future, no clean hand-off surface today). **Customer-facing self-bind page = R-27 (future).** Origin spike `audits/SPIKE-baremetal-iso-2026-07-16.md`: every mechanism GREEN on nested virt (VM 310 on felhom-pve). Zero-touch install BIOS **and** UEFI incl. **Secure Boot enforcing** (no MOK/keypress); first-boot hook `fully-up` = root + working pvesh/pct, exactly-once via `pending-first-boot-setup` flag; post-install **webhook** carries SMBIOS-UUID + management-MAC + host SSH keys → the unclaimed-appliance record; disk-filter installs only the target (canary byte-identical) and **fails-safe** on match-nothing / bad disk; `from-url` + `cert-fingerprint` **fails CLOSED**. OPEN (needs ONE real bare-metal run): vendor DMI serials (empty on virt), real firmware/NIC quirks, a pre-existing-LVM wipe step. Pipeline notes: assistant pairs to the ISO by Debian codename, ~11.5 s/ISO on DooPlex, **gate on `validate-answer` output not `$?`** (exit 0 on failure). Would flip a new capability-map MISSING row "customer self-installs on bare hardware" once spec'd. *(brief called this R-22)* | ## Absorbed / superseded notes in this folder