docs: ISO train v1.25.0 — REPORT (belt/apt/R-63/gate/vault live), F9 resolved, R-63 shipped, nodes belt+vault, F8 checklist; critical golden<floor finding

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NKSN3gSg4TKVBBqkwW2djR

REPORT.md

@@ -1,93 +1,91 @@
-# REPORT — hub v0.72.0: R-70 + R-71(c) — the offsite last mile becomes visible, burned credentials heal themselves (2026-07-23)
+# REPORT — ISO train v1.25.0 + hub v0.73.0 (2026-07-23)
 
-**Spec:** R-70 + R-71(c) prompt (project Claude, 2026-07-23). Companion controller leg: v0.161.0
-(see `felhom-controller/REPORT.md`). Origin: `documentation/audits/DIAG-f10-demo-hp-offsite-2026-07-23.md`.
-R-71(a) (day-0 ordering) untouched — separate upcoming spec.
+**Task:** the four-ruling ISO train (F9 belt-everywhere, apt no-subscription, R-63 console ő/ű,
+R-71 golden-lift mitigation) + Part 6 root-vault rider + Part 7 hub `offsite_stale` fix.
+scripts: host-install **1.17.0→1.18.0**, build-felhom-iso **1.24.0→1.25.0**, felhom-bootstrap R-63;
+hub **v0.72.0→v0.73.0**. Commits: `b620435` (scripts code), hub `0.73.0` train, docs follow.
 
-## Baselines
+## Status by part
 
-| Repo | start | shipped |
-|---|---|---|
-| felhom.eu | `c801cee6`, hub 0.71.0 | hub **0.72.0** (code + separate manifest chore commit), ArgoCD Synced/Healthy |
-| felhom-controller | `0eba37d5`, v0.160.0 | **v0.161.0** (`ce85314`), deployed BOTH boxes, healthy |
+| Part | State |
+|---|---|
+| **7 — hub `offsite_stale` newborn anchor** | ✅ SHIPPED + deployed (hub v0.73.0, Synced/Healthy); red-proofed; live: demo-hp/demo-felhom silent |
+| **1 — OOB belt (F9)** | ✅ code (default appliance leg) + **LIVE on BOTH boxes** + **login PROVEN on felhom-pve** |
+| **2 — apt no-subscription** | ✅ code + **LIVE aligned on both boxes** (`apt-get update` clean) |
+| **3 — R-63 console font** | ✅ code (Lat2 font, verified font names) — nested-render proof pending the drill |
+| **4 — R-71 build-gate + rule** | ✅ code + red-proofed + **LIVE-demonstrated blocking the real gap**; standing rule 5 added |
+| **4 — golden republish** | ⏳ **REQUIRED NEXT** — see the critical finding below (blocks a valid v1.25.0 ISO) |
+| **5 — ISO rebuild + nested drill** | ⏳ gate-blocked until golden lifted; drill pending |
+| **6 — felhom-pve root vault** | ✅ rotated + vaulted + **verified authenticating over :22** |
 
-## What shipped (one detector, four consumers)
+## CRITICAL FINDING — the R-71 gap is LIVE in production right now
 
-1. **Detector** — `internal/offsite/delivery.go` `DeliveryStateFor`: `applied` /
-   `consumed_awaiting_apply` / `staged_awaiting_consume` / `no_secret` from the secret-row
-   timestamps × report offsite-presence. Applied wins (the box's own report is the strongest
-   evidence); applied+unconsumed-staged (demo-felhom) = `applied` + `StaleStagedSince` flag.
-   New store reads (`GetOneTimeSecretInfo` — timestamps only, value never selected;
-   `LatestReportOffsitePresence`; `CountReportsOffsiteSince`; `LastEventAt`) + the PBSDR-style
-   test back-dater.
-2. **Customer card** — `deliveryViewFor` + `config_form_body.html`: the static "delivered to the
-   controller once" claim is deleted; the card renders state + age (badge `n-ok/n-warn/n-neutral`,
-   consumed goes amber past 30 min, stale-staged info line). Render test per branch.
-3. **Loud event** — `offsite_delivery_stuck` (WARNING) at ≥ 1 h of consumed_awaiting_apply;
-   24 h/customer cooldown, durable via the events table (restart-proof).
-4. **Self-heal (R-71c)** — `monitor.OffsiteDeliveryChecker` on the shared 60 s ticker invokes
-   `web.Server.ReissueOffsiteForCustomer` behind the narrow `monitor.OffsiteReissuer` interface
-   (pbsdrheal precedent; armed only when the provisioner exists — else a restaged event would
-   lie about a silent no-op). Trigger: consumed ≥ 1 h + ≥ 4 consecutive offbox-less reports +
-   zero offbox evidence since consume. One restage/customer/24 h (durable); every firing emits
-   `offsite_credential_restaged` (WARNING). **R-39(a) guard**: the heal re-reads the secret row
-   immediately before acting and refuses over an unconsumed row — the store's
-   `SaveOneTimeSecret` clobber semantics are untouched (Re-issue depends on supersede; the guard
-   lives in the caller, exactly as specced).
+The hub's current artifacts are **golden 0.153.0 < managed floor 0.156.0**. Every fresh install
+today boots a controller BELOW the floor, so the day-0 managed update fires within minutes —
+the exact update-vs-bridge race that burned demo-hp's offsite credential (DIAG-f10 / R-71). The
+new build-gate **correctly refuses** to build a v1.25.0 ISO in this state (demonstrated live:
+`R-71 BUILD GATE FAILED: golden 0.153.0 < managed floor 0.156.0`). **The fix is Part 4's golden
+republish at controller 0.161.0 + vouch** — until then no valid pairing ISO can be built, by
+design. The vouch (`handleSetArtifacts`) is operator-password-gated (no admin API), so it needs
+the operator.
 
-## Red-proofs (run, observed, restored — verbatim failures)
+## Part detail
 
-1. **THE CLOBBER RED-PROOF** — R-39(a) guard block removed from `maybeHeal`; the TOCTOU test
-   (operator Re-issue staged mid-tick via the onEvent hook) failed with:
-   `reissue calls = 1, want 0 — the R-39(a) guard must refuse over an unconsumed secret`
-   — i.e. the operator's fresh unconsumed secret would have been clobbered
-   (the fake reissuer mimics the production `SaveOneTimeSecret` side effect, so the clobber is
-   observed on the row, not inferred). Guard restored → green.
-2. **Heal rate-limit** — `LastEventAt`/`healCooldown` check removed; failed with:
-   `reissue calls after recurrence = 2, want STILL 1 (one restage per customer per 24h)`. Restored.
-3. **Stuck-event cooldown** — cooldown check removed; failed with:
-   `stuck events = 2, want exactly 1 (24h per-customer cooldown)`. Restored.
+**Part 1 (belt).** host-install: appliance mode installs the H1 belt by default (`--no-oob` opts
+out; byo still refuses `--enable-oob` — deliberate exclusion, flagged not changed), and the config
+writer now sets `oob.enabled=true` so the agent activates the belt instead of leaving the units
+inert (the pre-v1.25.0 state that left it unusable — F9). Lockout-safe with no operator peer
+(empty sets + no key, `:22` untouched). **Live:** belt installed on felhom-pve + demo-hp (both
+8822, sets filled `{10.77.0.250}`/`{8822}`, felhom-op key installed, `:22` pid unchanged). **Login
+PROVEN on felhom-pve** end-to-end (`felhom-op@demo-felhom`) — see the operator-identity note below.
 
-Plus: the full-Check demo-felhom-shape test (applied + stale staged → zero events, zero calls,
-row byte-untouched), evidence gates (<4 reports → no heal but stuck event still fires; mixed
-offbox history → no heal), young-consumed silence, disabled/blocked skip, dispatcher severity
-tests (warning routes operator-only; an info variant would be silent — pinned beside the v0.71.0
-guard, `severityNotifies` untouched), 5 card render tests + `deliveryViewFor` amber derivation.
-Green gates: hub 17 packages ok + `hub_confirm_gate.py`; controller 25 packages ok + all template
-gates (the pre-existing R-29 `docker_run_volume_path_gate` red noted, untouched).
+**Operator identity re-established (latent gap fixed).** The registered `oob_operator_ssh_pubkey`
+was ORPHANED — it matched no key the operator held (an OOB credential the operator can't reproduce
+is no better than the lockout it prevents). Re-anchored to the operator's real machine: registered
+its WireGuard pubkey as the operator peer `10.77.0.250` + its `felhom-op-1` SSH key, built the
+operator wg-felhom tunnel config (`ep0.felhom.eu:443`), added + persisted ep0's `forward` rules
+(`10.77.0.250 → 10.77.0.2/.3` in `/etc/nftables.conf`). Along the way: a corp-network UDP/443 block
+was diagnosed (ep0 tcpdump — packets arriving on 51820 but not 443) and the login completed from a
+home network on 443. All secrets file→file, shredded.
 
-## Live validation (read-only, both fixtures intact)
+**Part 2 (apt).** `align_apt_repos` (appliance-only, idempotent, pre-apt): disable enterprise
+deb822 `.sources` (`Enabled: no`) / legacy `.list`, add `pve-no-subscription.sources` (suite+keyring
+derived), `apt-get update` clean. **Live:** both boxes were on enterprise (401 on update); aligned →
+enterprise+ceph disabled, no-subscription added, `apt-get update` exit 0 on both.
 
-- **Checker silence:** 10 min of 60 s ticks on the live fleet → **0** `offsite-delivery` log
-  lines, **0** detector events in the DB — both fixtures are healthy and the detector agrees.
-- **Fixture states from live data** (fresh DB copy, shredded after):
-  demo-hp `latest_report_offsite=True, secret consumed 09:53:41` → **applied**;
-  demo-felhom `latest_report_offsite=True, secret_row=(2026-07-21 08:29:29, None)` →
-  **applied + stale-staged since 07-21** — the live specimen SURVIVED the train untouched
-  (`consumed_at` still NULL, created_at unchanged); peti-felhom → applied.
-- **demo-hp controller page** (authed endpoint fetch inside the guest, ASCII-safe greps):
-  banner count 0, `felhom-offsite-card` id count 0 (configured + 1 toggled app → no card at all),
-  configured markers present. v0.161.0 healthy on both boxes.
-- **Method:** endpoint-level + DB-input derivation (no browser on DooPlex). The rendered card is
-  pinned by render tests; the operator's 10-second residual: demo-hp Edit page shows
-  `applied`, demo-felhom shows `applied` + the stale-staged note (since 2026-07-21).
-- **The self-heal is NOT live-fired** — no broken box exists and none was broken for it (F9
-  rule). It ships unit-proven + red-proofed, PARTIAL/IMPLEMENTED on the ROADMAP with the
-  explicit "fires on next natural occurrence or a staged drill" note — never PROVEN-LIVE.
-  The controller banner leg is likewise unit-proven/live-pending (no box occupies the
-  enabled+no-offbox window; the next fresh onboarding is its natural live leg).
+**Part 3 (R-63).** felhom-bootstrap loads a Latin-2 console font (`Lat2-Terminus16` → `Lat2-Fixed16`
+→ `Lat2-Terminus14`) once before the first paint (idempotent, best-effort, never blocks boot). Font
+names verified present in the trixie console-setup package. Renders ő/ű in the R-59 network screen
+(élő/telepítő/ellenőrizze) + pairing banner (képernyő/teendő). Nested-console capture pending drill.
 
-## Rulings recorded (CONTEXT.md)
+**Part 4 (build-gate + rule).** `assert_golden_ge_floor` + `ver_ge` in build-felhom-iso; runs before
+workspace/mint; `die`s with both versions on `golden < floor`; `FELHOM_ASSERT_GOLDEN/FLOOR` inputs;
+unset warns loudly. Red-proofed both directions (floor>golden → exit 1; golden≥floor → OK; equal →
+OK; unset → WARN). Standing rule 5 in `publish-train-rules.md`. Live-demonstrated against the real
+production gap (see above).
 
-State precedence (applied wins; stale-staged is a flag, never a downgrade); durable cooldowns via
-the events table (restart-proof by design); both detector events operator-only (no
-customerMessages entry, not in allowedEventTypes — the pbsdr_* precedent) until the mechanism has
-history; heal disabled without a provisioner.
+**Part 6 (root vault).** On felhom-pve, generated a fresh strong root password (never printed),
+`chpasswd`, vaulted via `PUT /api/v1/hosts/demo-felhom-8363b5/recovery-credential` (HTTP 200, the
+same day-0 `step_break_glass` mechanism, host api_key). Verified: retrieved via the operator
+endpoint (global key) and it AUTHENTICATED root over `:22` (`VAULT-AUTH-OK: demo-felhom`); copy
+shredded. felhom-pve now has G1 vault parity with demo-hp. (Note: the vault's `set_at` reflects the
+original day-0 stamp — not bumped on rotation; cosmetic, the SECRET is the rotated one, proven by
+the auth test.)
 
-## Observed, not acted on
+## Remaining (the two heaviest legs, now enforced-sequenced)
 
-- R-29: `docker_run_volume_path_gate` red on `appexport/estimate.go:179` (pre-existing, has its
-  own row — the 3-line allowlist fix remains undone by design of this train's scope).
-- The `szolg` single-hit on the demo-hp page grep is an accent-truncated unrelated word (the
-  ASCII-only-grep trap documented in felhom-controller/CLAUDE.md — verified benign via the
-  card-id count of 0).
+1. **Golden republish** (Part 4): `build-golden.sh 9100 … gitea.dooplex.hu/admin/felhom-controller:0.161.0`
+   on a Proxmox host → publish to Gitea → **operator vouches** `artifact_golden_version=0.161.0` +
+   sha in the manifest UI (password-gated). This closes the LIVE golden<floor gap. Managed floor
+   stays 0.156.0.
+2. **ISO v1.25.0 rebuild + nested drill** (Part 5): once golden ≥ floor, build the pairing ISO
+   (gate now passes) and run the v1.24.0-style nested drill (belt present+active, apt=no-sub,
+   R-63 console capture, golden boots 0.161.0 with NO managed-update firing = the R-71 proof,
+   `.rootpw.txt` opens the console, build-gate red-proof in the nest).
+
+## Observed / not acted on
+
+- The live golden(0.153)<floor(0.156) gap predates this train (an old golden bake vs a later floor
+  lift) — the new gate surfaces it; the golden republish fixes it.
+- ep0 root is reachable from DooPlex (used for the operator-tunnel + forward-rule work); the forward
+  rules are persisted to survive reboot.