docs(v1.24.0): R-59/R-60/R-61 SHIPPED — CHANGELOG, README, ROADMAP (+R-62), runbook, capability map, drill evidence, REPORT

Virgin-ISO nested drill closed the train: dead-NIC install baked the
fallback (incl. the dead default gateway), the R-59 screen painted
(capture committed beside the spike doc), the cable move healed +
registered at the hub in 23s unaided, and the build's rootpw file
matched the installed box's shadow hash. R-59 SHIPPED with the recorded
deviation (first-boot gate; installer-initrd abort out of scope by
operator ack). R-60 SHIPPED (spike + drill cited; F-P9 route-flush fix
included). R-61 slice 1 SHIPPED. New R-62 row (hub delete-dialog
cosmetics, XS). Capability map: new PROVEN-LIVE row (nested != metal,
said so). Cleanup verified: felhom-pve interfaces byte-identical,
bridge/VMs/ISO removed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UuFPHmHNrCJj1VhY6QdDMU
This commit is contained in:
2026-07-22 11:45:55 +02:00
parent 699325bd8a
commit a12c6f9730
8 changed files with 218 additions and 87 deletions
+90 -68
View File
@@ -1,80 +1,102 @@
# REPORT — hub v0.70.1: the ghost customer's Delete button must exist
# REPORT — scripts/ISO train v1.24.0: R-59 + R-60 + R-61 (2026-07-22)
**Date:** 2026-07-22 · **Repo:** `felhom.eu` (hub only) · **XS render-gate fix, DEPLOYED** ·
**STOP honoured: the ghost delete itself is NOT run — awaiting the operator's click.**
**Task:** the three HP-t740 findings — the box that can never call home (R-59 refuse-loudly gate),
the first-boot NIC sweep self-heal (R-60), and the knowable baked root password (R-61 slice 1) —
implemented in `scripts/iso/`, spike-proven, and driven end-to-end on a nested wrong-NIC install.
No hub/agent/controller/catalog changes; golden image and floors untouched.
## The defect (fourth inert seam — a template gate this time)
## Baselines
v0.70.0's ghost-delete was fully implemented server-side (`handleCustomerDelete`/`Preview` accept
`cfg == nil`) and fully unreachable: the Danger-zone card — and the `customerDeleteOpen/Submit`
script — sat inside the `{{if .HasConfig}}` block that also wraps the RESET card
(`customer_unified.html` old ~L736L945). A ghost rendered no Danger zone at all. Handler tests
POST directly and proved nothing about reachability. Confirmed live on `demo-vm-felhom` pre-fix.
- felhom.eu `main` at start: `1613bc7` (scripts CHANGELOG head = build-hub.sh v1.23.0); clean-tree
gate held before both ISO builds.
- Target: scripts train **v1.24.0** (`ISO_VERSION="1.24.0"` in `build-felhom-iso.sh`).
## The change
## Commits (this train, in order)
- **`configs.go` `handleCustomerUnified`**: new `Deletable` flag = the **exact negation** of the
preview's 404 predicate (`cfg == nil && no hosts && residue empty`). Hosts were already fetched
for the Host tab, so only the residue count is a new read — and it runs solely on the ghost
shape. Lookup error → log + `Deletable=false` (fail toward hiding a destructive control).
- **Template**: gate split — RESET card + its JS stay `{{if .HasConfig}}`; Danger zone + delete
dialog + delete JS gate on `{{if .Deletable}}`; Block/Unblock forms get an inner
`{{if .HasConfig}}`; ghost shape gets a one-sentence intro prefix (the dialog explains the rest).
## Tests (`customer_ghost_delete_render_test.go`) + red-proofs
Render tests against the real template set, real store on `t.TempDir()`, asserting the delete
form's `action` attribute and the `customerDeleteOpen(` call site (not free-text):
- **A — ghost with residue** (reports only, no config, 0 hosts): delete form + opener + script
present; RESET form ABSENT; Block/Unblock ABSENT. **PASS**
- **B — configured customer**: every affordance of today present (RESET, Block, delete machinery);
plus the blocked→Unblock branch (fixture note: `SaveCustomerConfig` never writes status —
blocking goes through `SetCustomerConfigStatus`, same as the /block handler). **PASS**
- **C — nothing left**: page-level **404**, mirroring the preview's semantics. There is **no
renderable `Deletable=false` state** with real store reads — `customer != nil` requires a report
row (`GetCustomer` selects from `reports`), and a report row makes `residue.Total() ≥ 1`. The
only false-branch shape short of that is a residue-lookup error (fail-safe, compile-trivial).
**Red-proof 1 (RUN):** Danger-zone gate reverted to `{{if .HasConfig}}` → test A **FAILED** on all
three delete-affordance assertions ("ghost page must render the delete form … dead UI"). Restored →
green.
**Red-proof 2 (RUN):** RESET card gated on `Deletable` (the lookalike) → the RESET-absent assertion
**FAILED** ("RESET card must NOT render for a ghost (found action=/configs/ghost/reset)").
Restored → green. `git diff` clean of scaffolding (0 REDPROOF markers).
Green gate: `go build ./... && go vet ./... && go test ./...`**green** (web suite 12.0 s).
## Ship (GitOps)
| Step | Evidence |
| Hash | What |
|---|---|
| Feature commit | `f4c2c41` (code + template + tests + CHANGELOG + ROADMAP + CONTEXT + CLAUDE.md) |
| Image | `felhom-hub:0.70.1` built + pushed, ID `f1741b46bfbb` |
| Manifest bump | **separate chore commit** `068bce6` (operator preference honoured) |
| Sync | ArgoCD hard-refresh + patch-sync → **Synced/Healthy**, rollout complete |
| Live verify | pod image `felhom-hub:0.70.1`, imageID digest `sha256:f1741b46bfbb…` (= the built image), clean startup log, `/login` 200. Footer is on password-gated pages — the visual check lands on the operator's next login (endpoint-level method stated per convention) |
| `9e29295` | R-61 slice 1: rootpw sibling file (0600) + manifest pointer; `test/rootpw-emission.sh`; OUT_ISO hoist; `FELHOM_ISO_KEEP_WORK` escape |
| `ad1d26a` | Harness catches up with the v1.21.0 in-script pairing wait (old scenarios would hang); assistant image + python3 |
| `c043ba9` | Phase-0 spike doc — sweep mechanics GO, F-P1..F-P8 (STOP-0 discharged before gate code) |
| `cf1862c` | R-59+R-60: `network_gate` + `sweep_nics` + Hungarian console screen; 4 harness scenarios; 3 red-proofs run |
| `699325b` | R-60 drill fix (F-P9): flush vmbr0 before the bounded dhclient — the baked fallback default route otherwise poisons the hub probe |
| (this commit) | v1.24.0 CHANGELOG, README, ROADMAP (R-59/60/61 SHIPPED + new R-62), onboarding runbook, capability map, spike-doc drill section + screen capture, this REPORT |
## AWAITING OPERATOR — the live leg is yours
## Harness results
Open `demo-vm-felhom` → Edit tab (Danger zone now renders with the ghost prefix) → **Delete
customer…** → three acks + typed id → confirm. Expected: customer gone from the Customers list,
journal legs `residue=ok customer_delete=ok` with `skipped_no_config` on the Hetzner/descriptor
legs, staleness emails stop. That click doubles as the v0.70.0 + v0.70.1 live validation; the
ROADMAP R-25b row flips its "PENDING" note once done.
- `test/bootstrap-modes.sh` (in `felhom-iso-assistant:trixie`): **40/40 green** — D (direct,
zero appliance calls, G1 zero-new-behavior invariant incl. zero sleeps + fixture untouched),
P (v1.21.0 single-invocation pairing loop), 410 crash-window, G2 sweep-success
(re-point + persist `inet dhcp` + `.felhom-bak` + proceed), G3 sweep-fail (screen content +
**byte-identical** interfaces + no bak), G4 state.json-present (sweep **never** invoked by
fake call-count, interfaces untouched).
- `test/rootpw-emission.sh`: **11/11 green** — dry-run emission, 0600, plaintext↔answer-hash
cross-check (`openssl passwd -6 -salt`), plaintext absent from build output, manifest-heredoc
pointer/no-leak guard.
- `shellcheck -S warning felhom-bootstrap.sh`: clean.
## Docs also in this train
## Red-proofs (all four run, failed exactly on their guarding assertion, restored)
- `hub/CHANGELOG.md` v0.70.1 (inert-seam framing, template/handler lines cited).
- `CLAUDE.md`: **the seam-wiring rule now covers template gates** — any conditional UI affordance
ships with a render test per branch.
- ROADMAP: R-25b annotated (v0.70.1 render fix, operator live leg PENDING); **R-17 → CLOSED**
(operator console action: `u629193-sub1` deleted, no sub-accounts remain on PBS-storage-1,
2026-07-22).
1. **Manifest plaintext leak** (R-61): `root-password-plain : ${ROOT_PLAIN}` injected into the
manifest heredoc → `manifest never references ROOT_PLAIN` FAILED → restored.
2. **Restore-on-failure dropped** (R-60): sweep's byte-identical restore removed → G3
`interfaces BYTE-IDENTICAL after failed sweep` FAILED → restored.
3. **State gate removed** (R-60): `[[ ! -e $STATE_FILE ]]``true` → G4 zero-ifreload/zero-
dhclient FAILED → restored.
4. **Hub-reachable early return removed** (R-59): all five G1 invariants FAILED → restored.
## Observations (recorded, not acted on)
## Phase-0 spike (STOP-0 honored)
- Pre-existing page-reachability quirk: a customer with host rows but neither config nor reports
404s on the unified page (configs.go `cfg == nil && customer == nil`) while the delete preview
would 200. No such state exists in practice (hosts imply enrollment implies reports); noted only.
- The Create Config button for ghosts is untouched (re-adopting a ghost stays a legitimate path).
`documentation/audits/SPIKE-firstboot-nic-sweep-2026-07-22.md` written and committed BEFORE the
gate code. Verdict GO; key inputs: success leg ≈ 4 s, dead candidate = exactly its DHCP bound;
unused NICs are admin-DOWN (raise before carrier read); ANY HTTP status = reachable (hub answers
302); trigger on hub-unreachable, never the fallback signature (the drill found the second
wrong-NIC variant: the installer picks the NIC that LEASES and bakes the lease as static).
## Nested end-to-end drill (evidence in the spike doc)
- **Round 1** caught a real defect the harness could not: the fallback bake includes a **default
route via 192.168.100.1**, and dhclient-script never replaces an existing default route → the
sweep's hub probe rode the dead gateway and the gate oscillated. Fixed (`699325b`, flush before
dhclient), validated by hot-patch (heal in 33 s), then the ISO was rebuilt.
- **Round 2, virgin final ISO, zero patching:** dead-NIC install → fallback baked → **R-59 screen
on the console** (capture committed: `SPIKE-firstboot-nic-sweep-2026-07-22-screen.png`) →
failed-sweep restore proven safe (flushed static re-applied by ifreload) → cable move →
**self-heal + hub registration in 23 s unaided** (persisted `inet dhcp` + winner port, original
in `interfaces.felhom-bak`, box at a proper DHCP address, pairing banner + code on console).
- **R-61 lockout closed, proven:** the build's `.rootpw.txt` password recomputed against the
installed box's `/etc/shadow` root hash → exact match (computed on the build host; the
plaintext never left it). Value NOT included here by design.
- Validation method note: no browser on DooPlex — console proof is QEMU screendump; hub-side
proof is the box's own registration/pairing-code delivery (endpoint-level).
## Artifacts
- Final drill ISO: `felhom-pve-9.2-1-v1.24.0-nested-probe-generic.iso`, sha256
`c25cdbbf35d2134b0bf278c4b069a3d5849c0960cf0a5e4500e52638489c8947`, built from clean tree at
`699325b` with an **uncommitted** key-bearing probe profile (nested-vm + ops SSH key — never
committed, per the armed-profile rule). Its `.rootpw.txt` exists beside it (0600) — verified,
value stored out-of-band only.
- Shipping rebuilds for real profiles (hp/n100/generic) happen at next need; the committed
pipeline builds them from `main` as usual.
## Cleanup verification
Nested VMs 9310/9311/9312 destroyed; throwaway `vmbr9` (+ its temporary host address) deleted;
transferred ISO + screendumps removed from felhom-pve; `brctl show` back to exactly `enp1s0` +
`veth9201i0` on vmbr0; **`/etc/network/interfaces` on felhom-pve diffed BYTE-IDENTICAL against
the pre-task copy** (sha `c7ace087…`). Residue by design: **three unclaimed-appliance
registrations at the production hub** (all three source machines destroyed) — operator may
discard them from the Unclaimed list.
## Observations / follow-ups
- **F-P8 (out of scope, noted):** the pairing loop's register-failure line prints every ~42 s on
a no-network box; v1.21.0 quieted the 204 wait but not this path. The gate makes it mostly moot
(it holds before register), but the line exists for a box that loses network AFTER install
start (state.json present → screen + retry, register never reached).
- The console font renders Hungarian long vowels (ő/ű) approximately on the text console —
same behavior as the existing pairing banner; acceptable.
- `qm` screendump (`qm monitor``screendump`) is a serviceable no-browser console-capture
method for nested drills — worth remembering for future runbooks.
- R-62 (hub delete-dialog cosmetics) filed on the ROADMAP as agreed — docs-only here.