scripts v1.18.0 — R-21 slice B: firmware loader option --loader shim|mkimage (F1)
Closes N100 F1 (HIGH): cheap AMI (AN3PLUS 0.01-class) UEFI firmware can't relocate the ISO's stock signed GRUB from USB (relocation 0x0). The run's live grub-mkimage workaround is now a first-class pipeline mode. - build-felhom-iso.sh: --loader shim|mkimage (default shim, byte-for-byte unchanged; profile-settable FELHOM_LOADER; --loader wins). Loud banner + manifest loader:/grub-mkimage: fields + -mkimage filename suffix. - mkimage-surgery.sh (new): post-prepare-iso, in the assistant container. Builds a monolithic grub-mkimage loader from the ISO's own GRUB (module set from its grub.cfg; embedded search --fs-uuid -> configfile the real menu). Swaps it into the ISO9660 tree (real lowercase path) + the efi.img ESP; xorriso re-master preserves BIOS-hybrid + UEFI + GPT-ESP, drops only Apple HFS+/APM. Recipe from the N100 run evidence, not re-derived. - Dockerfile.assistant: grub-common + grub-efi-amd64-bin + mtools + dosfstools. profiles/n100.profile (new, mkimage + SB-off note). - Validated on nested VM 311 (RUNBOOK-B legs): leg1 shim boots+installs under OVMF SB-enforcing + SeaBIOS; leg2 mkimage boots+installs under SB-off; leg3 (red-proof) mkimage under SB-enforcing FAILS Access Denied (unsigned -> SB must be OFF); leg4 surgery byte-identical payload. bash -n + shellcheck clean. Physical N100 closure folds into the rehearsal (n100-safety match-nothing ISO built + sha-recorded, unbooted). PXE stays a deferred R-21 note.
This commit is contained in:
@@ -2,58 +2,96 @@
|
||||
|
||||
> **Overwrite** this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in [hub/CHANGELOG.md](hub/CHANGELOG.md); the scripts history lives in [scripts/CHANGELOG.md](scripts/CHANGELOG.md).
|
||||
|
||||
## Hub v0.61.0 + felhom-tenantsync v1.1.0 — Customer RESET (middle lifecycle tier) — 2026-07-17
|
||||
## scripts v1.18.0 — R-21 slice B: firmware loader option `--loader shim|mkimage` (F1) — 2026-07-17
|
||||
|
||||
**Commits:** `4009401` (code), `e144c5e` (manifest). **Live:** hub **0.61.0** on k3s (Synced/Healthy);
|
||||
felhom-tenantsync **v1.1.0** on ep0 (`felhom-hetzner`).
|
||||
**Baseline:** scripts v1.17.0 (host-install) / build-felhom-iso.sh was ISO_VERSION 1.16.0. Target
|
||||
**v1.18.0**. Closes N100 **F1 (HIGH):** cheap AMI (`AN3PLUS 0.01`-class) UEFI firmware can't relocate
|
||||
the ISO's stock signed GRUB from USB (`relocation 0x0`); the run's live `grub-mkimage` workaround is
|
||||
now a first-class pipeline mode. Touched **only** `scripts/iso/`.
|
||||
|
||||
### What shipped
|
||||
|
||||
The **middle lifecycle tier** between *host delete* and *customer Delete*. One operator action returns a
|
||||
customer to **pre-first-install**: every OPERATIONAL trace dies — offsite repo (Hetzner sub-account/box),
|
||||
PBS namespace + all backup groups + token, DR recipe, one-time secret, claim state, retained escrow
|
||||
custody — while **identity + basic config survive** (`customer_configs` incl. the offsite tier CHOICE, all
|
||||
provenance rows, and the audit-event stream).
|
||||
`build-felhom-iso.sh` gains `--loader shim|mkimage` (default **shim**, profile-settable via
|
||||
`FELHOM_LOADER`, `--loader` wins). **shim = the stock output, byte-for-byte unchanged** (surgery
|
||||
skipped). **mkimage** replaces the ISO's UEFI boot path with a monolithic `grub-mkimage` loader.
|
||||
New: `mkimage-surgery.sh`, `profiles/n100.profile`, Dockerfile grub/mtools tooling; loud banner +
|
||||
manifest `loader:`/`grub-mkimage:` fields + `-mkimage` filename suffix.
|
||||
|
||||
Viktor's rulings, all implemented: (1) destroying retained escrow custody gets its **own separate
|
||||
acknowledgment**; (2) RESET clears claim state → fresh code next onboarding; (3) RESET **refuses while any
|
||||
host row exists** (delete hosts first — reset never deletes hosts); (4) the confirm surface shows a
|
||||
**live-counted** inventory.
|
||||
### Surgery approach actually used (Phase 2)
|
||||
|
||||
### Discipline
|
||||
Recipe reproduced from the run evidence (the literal command was run on the now-wiped N100 old system
|
||||
and saved nowhere — only the *procedure* is recorded; see Observations). In the assistant container,
|
||||
**after** `prepare-iso`:
|
||||
1. `xorriso -osirrox` extract the prepared tree (chmod u+w — osirrox files are read-only).
|
||||
2. `grub-mkimage -O x86_64-efi` from the **container's** grub 2.12 build (`/usr/lib/grub/x86_64-efi`,
|
||||
which has `kernel.img` — the ISO ships modules but not kernel.img; the box likewise used its
|
||||
*installed* GRUB). Module set = the base search/configfile/USB chain **plus every `insmod` in the
|
||||
ISO's own `grub.cfg`** (57 modules). Embedded config: `search --fs-uuid <ISO uuid>` → `configfile`
|
||||
the real menu. The fs-uuid is derived from the ISO's PVD modification-date (GRUB's iso9660 uuid
|
||||
source) and pinned on re-master so it stays valid.
|
||||
3. Swap the loader into the ISO9660 tree (**real lowercase Rock-Ridge path** `/efi/boot/bootx64.efi` —
|
||||
an early bug created a spurious *uppercase* path instead; fixed) **and** inside the `efi.img` ESP
|
||||
(FAT, case-insensitive — the loader UEFI firmware actually runs from USB).
|
||||
4. Re-master with `xorriso -as mkisofs` from the ISO's own `as_mkisofs` report, dropping **only** the
|
||||
Apple HFS+/APM map (`-hfsplus`/`-apm-block-size`) and the isohybrid GPT-basdat marking
|
||||
(`-part_like_isohybrid`/`-isohybrid-gpt-basdat`) — re-emitting those trips xorriso 1.5.6's
|
||||
"Overlapping MBR partition entries" on this layout, and Mac boot is irrelevant. **Kept:** protective
|
||||
MBR + grub2-mbr (BIOS hybrid) + El Torito (BIOS + UEFI) + the GPT EFI System Partition (USB UEFI).
|
||||
|
||||
External teardown FIRST, DB purge LAST (publish-last). Every leg idempotent → a partial run is re-run from
|
||||
the top; a failed external leg is a clean journal entry and the DB purge (which erases the descriptors that
|
||||
say what still needs tearing down) is **withheld** until every external leg is `ok`. Provenance + events
|
||||
are never wiped.
|
||||
**Stall-rule outcome:** the surgery resisted for ~40 min of iteration (two overlap failures + the
|
||||
kernel.img and path-case bugs) — **well within the 2 h budget**; no spike-note fallback needed.
|
||||
|
||||
### Components
|
||||
### Per-leg results (RUNBOOK-B, nested VM 311 on felhom-pve, OVMF)
|
||||
|
||||
- **store/customer_reset.go**: `customer_resets` journal (per-attempt, per-leg, resumable);
|
||||
`CustomerResetInventory` (live counts incl. retained blobs via the F-14 `host_deletions` UNION);
|
||||
`Start/UpdateResetLeg/Finish/LatestCustomerReset`; ack-gated `PurgeCustomerResetDBState`; `DeleteClaim`.
|
||||
- **claim.ResetToUnclaimed**: deletes the claim row → `EnsureIssued` mints a fresh first code next onboarding.
|
||||
- **offsite**: `Deprovision` (idempotent label-lookup delete), `OffsiteIdentifier`, `ClearProvisionedDescriptor`.
|
||||
- **tenantsync + scripts/felhom-tenantsync.sh v1.1.0**: `deprovision` op destroys the namespace + backup
|
||||
groups + token; shared `felhom@pbs` user untouched; idempotent.
|
||||
- **web/customer_reset.go**: `GET /configs/{id}/reset` → inventory JSON; `POST` → orchestration (all gates
|
||||
before any write/external call). A distinct **amber** RESET card + `reset_done` flash.
|
||||
- **Leg 1 — shim regression:** the default ISO **boots + zero-touch auto-installs under OVMF Secure
|
||||
Boot ENFORCING** (`pre-enrolled-keys=1`; "EFI boot mode detected", DHCP, "Starting Proxmox
|
||||
installation", package extraction unattended). **Plus one SeaBIOS boot** — the shim ISO also boots
|
||||
under BIOS (hybrid grub2-mbr/eltorito path survived the v1.18.0 build). ✓
|
||||
- **Leg 2 — mkimage happy path:** `--loader mkimage` ISO **boots + auto-installs under OVMF Secure
|
||||
Boot OFF** (`pre-enrolled-keys=0`). The monolith's embedded `search --fs-uuid` found the ISO and
|
||||
chained its real menu → the installer ran. ✓
|
||||
- **Leg 3 — trade-off red-proof:** the **same** mkimage ISO under OVMF Secure Boot ENFORCING
|
||||
**FAILS** — captured firmware behavior: `BdsDxe: failed to load Boot0002 "UEFI QEMU DVD-ROM …" :
|
||||
Access Denied` → `No bootable option or device was found`. The unsigned loader is refused. This
|
||||
makes "Secure Boot must be OFF for mkimage" a **fact**, not a guess. ✓
|
||||
- **Leg 4 — structural / byte-identity:** diffing the shim(out) vs mkimage(final) trees, **only**
|
||||
`efi/boot/bootx64.efi`, `efi/boot/grubx64.efi`, `efi.img`, and the regenerated El Torito artifacts
|
||||
(`boot.cat`, `i386-pc/eltorito.img`) differ — **the entire install payload is byte-identical**. The
|
||||
tree loader (1253376 B, vs shim's 1032032 B) carries the embedded `2026-05-21-…` fs-uuid and is
|
||||
identical to the loader inside `efi.img`; **0 spurious uppercase paths**; El Torito keeps both boot
|
||||
images. The `validate-answer` gate still runs pre-surgery. ✓
|
||||
|
||||
### Verification
|
||||
### Artifacts (shas + manifests)
|
||||
|
||||
- **Green gate:** `go build ./... && go vet ./... && go test ./...` all pass. Hub confirm gate OK.
|
||||
- **Red-proofs (run-fail-revert, both proven red then restored):** (a) escrow-ack gate — defeat it →
|
||||
reset proceeds & destroys blobs → FAIL; (b) partial-failure resumability — don't withhold the purge →
|
||||
DB purged despite an external failure → FAIL. Plus store ack-gating + journal round-trip; offsite
|
||||
Deprovision idempotency + descriptor clear; RESET-card render.
|
||||
- **Live drill on ep0 (throwaway `drill-reset-01`, real PBS data):** provision → **real backup written into
|
||||
the namespace** → deprovision (`deleted:true`; namespace + backup group + token destroyed) → token/ns
|
||||
confirmed gone → idempotent re-run (`deleted:false`) → **all 3 real tenants + the shared user survived**.
|
||||
- shim nested-test ISO `…-nested-vm-val.iso` — `28d862e2f7b014588717c15ef8c9bc46d11a9a5689a022f5d4de9e8f939d1377`
|
||||
- mkimage nested-test ISO `…-nested-vm-val-mkimage.iso` — `71fce3333b3148e48d10711311f275da4b849bf383f711de0f2e4d5802618869`
|
||||
- **safety ISO** `…-n100-safety-mkimage.iso` — `f9aa0428e5efdcb5e7c0ade05264ff681ac4f9aa47b7f3721c9eb712f025c1a8`
|
||||
(mkimage + match-nothing filter; **built + sha-recorded, NOT booted**; at `180:~/n100-baremetal/safety/`).
|
||||
Manifests record `loader: mkimage … Secure Boot OFF`, `grub-mkimage: 2.12-9+deb13u2`, secret-bearing.
|
||||
Nested-test ISOs + VM 311 destroyed after the legs.
|
||||
|
||||
### Not live-drilled (covered otherwise)
|
||||
### Physical proof on the real AMI board
|
||||
|
||||
The web RESET POST and the Hetzner sub-account delete are covered by the httptest-driven orchestration
|
||||
tests + offsite unit tests (the offsite `Deprovision` faithfully mirrors the live-proven
|
||||
`ReissueCredentials`, swapping reset→delete). The hub UI is operator-password-gated → CC verifies the UI
|
||||
via render tests, not a live click. A live Hetzner create/delete drill was not run (billed external
|
||||
service; mirror-of-live-proven + idempotency-tested).
|
||||
Shape **(a) DEFAULT** stands unless Viktor picks otherwise: the F1 closure folds into the supervised
|
||||
N100 rehearsal (its install ISO built `--loader mkimage`). Shape **(b)** — the zero-risk `n100-safety`
|
||||
pre-flight ISO — is **built and sha-recorded** (above), ready either way; **do not boot without Viktor**.
|
||||
|
||||
### Docs diffs
|
||||
|
||||
`scripts/CHANGELOG.md` (v1.18.0); `scripts/iso/README.md` (loader-modes section + N100 SB-off prep +
|
||||
`mkimage-surgery.sh` row); `scripts/iso/profiles/n100.profile` (new, mkimage); `VALIDATION-n100-baremetal`
|
||||
F1 → PIPELINE-FIXED erratum; `00-capability-map.md` (new bare-metal ISO row + loader caveat);
|
||||
`ROADMAP.md` (slice B collapsed to SHIPPED; PXE one-line deferred note); `CONTEXT.md`.
|
||||
|
||||
### Observations
|
||||
|
||||
- **The literal `grub-mkimage` command was unrecoverable** — it ran on the N100's old system (wiped at
|
||||
the run) and is not in `~/n100-baremetal/` nor 180's history. Only the *procedure* is recorded
|
||||
(VALIDATION F1 / CONTEXT). Reproduced faithfully: mkimage from a working (non-ISO) 2.12 GRUB build,
|
||||
module set = the ISO's grub.cfg needs, embedded `search --fs-uuid` → `configfile`. Not a strategy
|
||||
re-derivation.
|
||||
- The mkimage re-master **drops Apple HFS+/APM (Mac boot)** — irrelevant for N100/PC, and required to
|
||||
avoid xorriso's overlap check. BIOS + UEFI + GPT-ESP boot all preserved (asserted in-build + Leg 4).
|
||||
- Leg 2's "first-boot stub fires" is assured transitively: Leg 4 proved the first-boot payload is
|
||||
byte-identical to the shim variant's (slice-A-proven), and Leg 2 proved the answer is consumed. A
|
||||
full ~10-min nested install-to-first-boot was not re-run for the mkimage ISO on that basis.
|
||||
|
||||
Reference in New Issue
Block a user