docs(pilot): agent 0.93.0 publish train record + onboarding §A5 gate discharged

REPORT.md

@@ -1,56 +1,49 @@
-# REPORT — Runbook refresh: RUNBOOK-onboarding-draft v3 → v4 (docs-only, XS)
+# REPORT — Agent-only publish train 0.93.0 executed (operational run, docs-only in this repo)
 
-**Date:** 2026-07-22 · **Repo:** `felhom.eu` · **Scope:** documentation only — no code, no version
-bump, no deploy. The onboarding runbook was refreshed against what shipped 2026-07-18..21 and
-against the second full onboarding on virgin hardware (HP t740, 2026-07-21).
+**Date:** 2026-07-22 · **Repo:** `felhom.eu` (runbook record + §A5 discharge; no code, no version
+bump, no hub deploy) · **Full record:** `documentation/pilot/RUNBOOK-publish-agent-0.93-2026-07-22.md`
 
-## What changed
+## What happened
 
-`documentation/runbooks/RUNBOOK-onboarding-draft-v3.md` → **`RUNBOOK-onboarding-draft-v4.md`**
-(git mv, one commit), per the project-Claude refresh brief of 2026-07-21:
+felhom-agent **v0.93.0** (hyphen-free escrow wordlist, source-only since 2026-07-21 at `a452dc33`)
+was **built, published, manifest-vouched and deployed to both fleet boxes** by CC, with one
+password-gated STOP → Viktor (the Day-0 manifest save, verified by CC in the hub DB **before** any
+box moved — publish-train rule 1).
 
-1. **R-39 interim blocks DELETED** (B4 descriptor caveat + E1 manual `pvesm status` check) —
-   replaced with one sentence each: the DR tier self-detects a dead credential (loud `auth_failed`)
-   and self-heals via damped re-issue; the hub gauge is trustworthy. Cited: `backlog/ROADMAP.md`
-   R-39 (CLOSED, proven live 2026-07-21, hub 0.68.1 + agent 0.91.2, 13 s chain
-   `applied → auth_failed → applied`).
-2. **R-36 workaround block in C4 DELETED** (re-onboarding manual-Save) — replaced with the shipped
-   behaviour: enabled-but-unprovisioned raises an amber banner (hub v0.67.0). R-31 (Save-once/504)
-   and the R-32 orphan-card note KEPT. Consistency follow-through: A1 and the B3 note now say the
-   self-bind link is **auto-minted at create/RESET** (also R-36) — the manual mint is the re-mint
-   fallback.
-3. **C1 freemail gate** — freemail.hu PROVEN 2026-07-21 (operator test-send received; R-4 COMPLETE
-   per `backlog/ROADMAP.md`); the check-spam habit line kept.
-4. **A1 golden note** — golden 0.153.0 carries all four infra images; floor-lift proven on TWO
-   shapes (rehearsal 0.143→0.145 in 5 s; virgin HP t740 0.153.0→0.156.0 during day-0, `CONTEXT.md`).
-   "Rebuild before first tester" softened to a pointer at the standing rule in F.
-5. **A3 loader note** — the single F1 line replaced with the loader table. **Deviation from the
-   brief, on the record's side:** the brief said "N100/AMI = shim"; the record
-   (`operations/nodes.md` + `tests/VALIDATION-n100-baremetal-2026-07-16.md` F1) says the N100
-   installed with **mkimage, SB off** — the AMI AN3PLUS firmware cannot relocate the signed GRUB,
-   which is exactly why shim was not usable there. Table written per the record, as the brief's
-   "(SB state as recorded)" directs. HP t740 = shim with Secure Boot ENABLED (2026-07-21).
-6. **B2/B3 second datapoint** — the HP onboarding walked register → pairing banner → unclaimed
-   list → self-bind (code + passphrase) → day-0 → dashboard on never-seen hardware (cited
-   `CONTEXT.md` + `operations/nodes.md`). New B2 gotcha: on multi-NIC boards verify the CABLED
-   port got the lease during install (the t740 4-port trap; R-59/R-60 pending).
-7. **New A5 checklist item** — agent train ≥ v0.93.0 (hyphen-free recovery wordlist,
-   `felhom-agent/CHANGELOG.md`) must be BUILT + DEPLOYED to the customer's box before the FIRST
-   real tester's escrow ceremony; ceremonies run box-side, source-only does not count.
-8. **D offboarding pointer** — R-25b ruling noted: full teardown = DELETE cascade (TASK-I),
-   RESET = identity-preserving re-onboarding; pointer to §G.
-9. **KEPT unchanged:** DRAFT status, the C7 graduation-gate paragraph (still the last unwalked
-   step), all reference wall-clocks, all F rules, all of §G.
+- **Artifact:** sha256 `a68b2ff73200622ea9e23aed71a0a8ff885808626cab9e9d554a5a5df5ced28f`
+  (13 817 690 bytes), sha-chain identical: built binary → Gitea package (PUT 201) → independent
+  anonymous GET.
+- **Manifest:** agent 0.92.1→**0.93.0** + new sha. Golden (0.153.0), floor (**0.156.0**) and
+  wrapper sha **byte-identical before/after**. **Deviation, ruled:** MinAgent moved 0.92.1→0.93.0
+  at the operator save (spec said do-not-touch); flagged before deploy, Viktor ruled leave it —
+  the held-floor window was transient (minutes) and harmless.
+- **Deploys** (felhom-pve first, then demo-hp over G1 break-glass — secret file→file, shredded):
+  both boxes clean-restart **5/5** — active/NRestarts=0, capabilities **68/68 degraded=0**,
+  ReassertGuestBinds touched only enrolled drives, guestnet sweeps quiet (0 ERROR/WARN), hub
+  `hosts.agent_version` = **0.93.0** for both (06:56:52Z / 06:57:52Z). `.bak-0.92.1` retained on
+  both; escape hatch not needed.
+- **Green gate** on the exact HEAD: build/vet/test green; the ex-flaky escrow package **10/10**
+  fresh runs (`-count=10`) — that flake was this defect, so the clean run is itself evidence.
 
-Also: `backlog/ROADMAP.md` R-25b's live pointer updated `…draft-v3.md` → `…draft-v4.md` so the
-citation resolves (historical references in the rehearsal validation doc and past REPORTs left
-as-is — they describe the file as it was named then).
+## Changes in this repo
 
-## Not done / unchanged
+1. **`documentation/pilot/RUNBOOK-publish-agent-0.93-2026-07-22.md`** — the executed record
+   (timestamps, hashes, per-box evidence, the MinAgent deviation + ruling, observations).
+2. **`documentation/runbooks/RUNBOOK-onboarding-draft-v4.md`** §A item 5 — the "agent ≥ 0.93.0
+   box-side before the first real escrow ceremony" gate marked **DISCHARGED 2026-07-22**, citing
+   the runbook record. (The spec pointed at a ROADMAP "wordlist row" — **none exists**; the gate's
+   real home is §A5, so the annotation went there. Recorded as a deviation, not improvised into a
+   new ROADMAP row.)
 
-- C7 not flipped; the capability-map row untouched; no other runbook touched.
-- No CHANGELOG entry: this repo's changelogs are per-area (`hub/`, `scripts/`, `website/`) and none
-  of those areas changed; prior docs-only commits (e.g. TASK-H nodes.md) followed the same pattern.
+## Fleet after this train
 
-*(Previous REPORT — TASK-I, hub v0.69.0/v0.70.0 DELETE cascade — preserved in git history at
-`9b3381b` and summarized in `backlog/ROADMAP.md` R-25b.)*
+Both boxes agent **0.93.0**; Day-0 manifest agent 0.93.0 / MinAgent 0.93.0 / golden 0.153.0 /
+floor 0.156.0. `peti-felhom` untouched by design — Friday's reinstall pulls the vouched manifest
+and lands on 0.93.0.
+
+## Observations (not acted on)
+
+- demo-hp's 1TB NVMe is now **enrolled** (`/mnt/nvme-1tb`, reasserted into 9201) — TASK-H's
+  "unenrolled/NTFS" note is outdated; `operations/nodes.md` may want a refresh.
+- demo-hp still key-less (break-glass worked cleanly); tailnet key-expiry toggle still open.
+- Host journals are CEST; all runbook evidence stamps note the offset.

documentation/pilot/RUNBOOK-publish-agent-0.93-2026-07-22.md

@@ -0,0 +1,121 @@
+# RUNBOOK — agent-only publish train 0.93.0: hyphen-free escrow wordlist reaches the fleet (2026-07-22)
+
+**Class:** Runbook-style operational run (NO code changed anywhere — v0.93.0 source was already on
+`main` at `a452dc33`/`eba040d0`). **Executor:** CC, with one password-gated **STOP → Viktor** point
+(hub Day-0 manifest save). **Status: ALL PHASES COMPLETE.** Both boxes on agent 0.93.0, manifest
+vouched, controller floor + golden byte-identical before/after.
+
+**Why:** (1) `runbooks/RUNBOOK-onboarding-draft-v4.md` §A item 5 — agent ≥ 0.93.0 must be BUILT +
+DEPLOYED box-side before the first real tester's escrow ceremony (source-only does not count);
+(2) Friday's Peti reinstall pulls the Day-0 manifest — vouched at 0.93.0, his box lands on the fixed
+wordlist with zero extra work.
+
+---
+
+## Results at a glance
+
+| Phase | What | Outcome |
+|---|---|---|
+| 0 | baselines + pre-gates + green gate | ✅ all confirmed at live source |
+| 1 | build + publish felhom-agent 0.93.0 | ✅ sha-chain identical build → package → anon GET |
+| 2 | **STOP → Viktor:** Day-0 manifest vouch | ✅ saved + CC-verified in hub DB **before any deploy** (one deviation, ruled — below) |
+| 3 | deploy felhom-pve → demo-hp | ✅ both clean-restart 5/5, hub confirms 0.93.0 |
+| 4 | docs (this file, REPORTs, §A5 discharge) | ✅ |
+
+```
+AGENT_VERSION=0.93.0
+AGENT_SHA256=a68b2ff73200622ea9e23aed71a0a8ff885808626cab9e9d554a5a5df5ced28f  (13 817 690 bytes)
+Package: https://gitea.dooplex.hu/api/packages/admin/generic/felhom-agent/0.93.0/felhom-agent
+```
+
+## Phase 0 — live baseline confirmation ✅
+
+All re-confirmed at source 2026-07-22 (~06:45–06:52 UTC), not trusted from the spec:
+
+- **felhom-agent `main`** = `eba040d0` = origin/main, clean tree; CHANGELOG head **v0.93.0**
+  (fix commit `a452dc33`). `git diff --stat 935904f..HEAD`: escrow code + tests + docs only —
+  **no `configs/` change** → binary-only deploy, no sudoers/wrapper ship.
+- **Both boxes 0.92.1**, from the hub `hosts` table (read-only streamed DB copy) AND direct
+  `felhom-agent --version` on each: `demo-felhom-8363b5` (last report 06:47:10Z),
+  `demo-hp-bb76ea` (06:35:43Z). Both services `active`.
+- **Day-0 manifest before** (hub DB `hub_settings`): agent **0.92.1** / `7424bc1c3c533eff…`,
+  golden 0.153.0 / `15fdd191…`, MinAgent 0.92.1, floor `min_controller_version` **0.156.0**,
+  wrapper sha `104db0a4…`. Hub live at **0.70.0** (deploy image tag).
+- **Package pre-gate:** `felhom-agent/0.93.0` anon GET = **404** (immutability respected).
+- **Green gate on the exact HEAD:** `go build ./... && go vet ./... && go test ./...` — green.
+  **The ex-flaky escrow package additionally run `-count=10` fresh: 10/10 pass** (135.2 s). That
+  flake WAS this defect (~23 % per 50-draw run pre-fix), so a clean 10× run is itself evidence.
+- **Deploy channel** (per `RUNBOOK-publish-0.90-0.143-2026-07-18.md` Phase 1 + the
+  felhom-build-deploy skill agent legs): build local (DooPlex) → `publish-agent.sh` →
+  scp one hop → `.bak-<old>` → `install -m0755` → restart → journal + hub verification.
+- **demo-hp access:** key auth denied (no key baked — TASK-H posture unchanged) → hub-vaulted G1
+  break-glass (`host_recovery` row `demo-hp-bb76ea`), extracted file→file (never stdout), used via
+  `SSH_ASKPASS`, **shredded after** together with all hub DB copies.
+
+## Phase 1 — build + publish ✅
+
+- Built on DooPlex from `eba040d0`: `CGO_ENABLED=0 go build -ldflags '-X main.version=0.93.0'`
+  (matching `publish-agent.sh`'s own build line). `--version` → `felhom-agent 0.93.0`.
+- `scripts/publish-agent.sh 0.93.0 <binary>`: pre-delete **404**, PUT **201**, authed round-trip
+  GET sha match. Creds sourced from the build host's docker-config gitea auth, never echoed.
+- **Sha-chain, three independent points, identical:** built binary → published package →
+  independent **anonymous** GET (HTTP 200, 13 817 690 bytes) — all
+  `a68b2ff73200622ea9e23aed71a0a8ff885808626cab9e9d554a5a5df5ced28f`.
+
+## Phase 2 — Day-0 manifest vouch (rule 1: before any box moved) ✅ + one ruled deviation
+
+Viktor saved the manifest (~06:53 UTC); CC verified in the live hub DB **before Phase 3 started**:
+`artifact_agent_version=0.93.0`, `artifact_agent_sha256=a68b2ff7…` — exact published values.
+Golden (0.153.0 / `15fdd191…`), floor (0.156.0) and wrapper sha (`104db0a4…`) byte-identical.
+
+**Deviation, surfaced + ruled:** `artifact_min_agent` moved **0.92.1 → 0.93.0** at the save (spec
+said MinAgent do-not-touch; rule 3 has MinAgent declared by the controller release, and 0.156.0's
+declaration was 0.92.1). Flagged to Viktor before deploying; **ruled: leave at 0.93.0.** Practical
+effect was a transient held-floor window (minutes) for boxes still reporting 0.92.1 — harmless,
+both already ran controller ≥ 0.156.0 — and it closed when both boxes reported 0.93.0
+(06:56:52Z / 06:57:52Z). Recorded, not papered over.
+
+## Phase 3 — deploy, felhom-pve first, then demo-hp ✅
+
+Per box: scp → **sha256 verified on-box = `a68b2ff7…`** → `cp` to `.bak-0.92.1` →
+`install -m0755` → `systemctl restart`. Journal timestamps below are **host-local CEST = UTC+2**.
+
+| | felhom-pve (`demo-felhom-8363b5`) | demo-hp (`demo-hp-bb76ea`, break-glass) |
+|---|---|---|
+| restart (UTC) | 06:56:49 | 06:57:48 |
+| `--version` / active / NRestarts | 0.93.0 / active / 0 | 0.93.0 / active / 0 |
+| ① no restart loop, clean startup | ✅ `daemon starting version=0.93.0` | ✅ same |
+| ② capability rows | ✅ **68/68 ok, degraded=0, inactive=0** | ✅ **68/68 ok, degraded=0, inactive=0** |
+| ③ ReassertGuestBinds | ✅ bound ONLY the enrolled `uuid:47a3361a…` (`/mnt/hdd_1` → 9201); nothing non-enrolled | ✅ bound ONLY the enrolled `uuid:91d2dc2d…` (`/mnt/nvme-1tb` → 9201); nothing non-enrolled |
+| ④ guestnet sweep | ✅ watchdog started; several 1-min sweeps: **0 heal/error lines, 0 ERROR/WARN total** | ✅ same |
+| ⑤ hub `agent_version` | ✅ **0.93.0** (report landed at startup, 06:56:52Z) | ✅ **0.93.0** (06:57:52Z) |
+
+Also observed healthy on felhom-pve: desired-state synced (gen 5), PBS verify cycle complete
+(felhom-offsite, 1 snapshot); on demo-hp: desired gen 2, PBS verify cycle (0 snapshots — no PBS-DR
+descriptor yet). Escape hatch (restore `.bak-0.92.1`) **not needed**; the backups remain on both
+boxes.
+
+## Post-train state
+
+- Fleet: **both boxes agent 0.93.0**, reporting, all checks green.
+- Day-0 manifest: agent **0.93.0 / a68b2ff7…**, MinAgent **0.93.0** (ruled), golden 0.153.0,
+  floor 0.156.0, wrapper `104db0a4…` — floor + golden byte-identical before/after this train.
+- `RUNBOOK-onboarding-draft-v4.md` **§A item 5 gate = DISCHARGED** for the current fleet and for
+  every future manifest-driven install (Friday's Peti reinstall included) — annotated in place.
+- `peti-felhom` deliberately NOT deployed to (box down; gets 0.93.0 via Friday's reinstall + the
+  vouched manifest — by design).
+
+## Observations (recorded, NOT acted on)
+
+1. **The spec's "ROADMAP wordlist row" does not exist** — no wordlist/hyphen row anywhere in
+   `backlog/ROADMAP.md`. The gate lives in `runbooks/RUNBOOK-onboarding-draft-v4.md` §A item 5;
+   the discharge annotation went there (the coupling's real home) instead.
+2. **demo-hp's 1TB NVMe is now ENROLLED** (`/mnt/nvme-1tb`, `uuid:91d2dc2d…`, reasserted into
+   9201) — TASK-H (2026-07-21) recorded it unenrolled/NTFS. It evidently joined via Tárhely as
+   designed. Fleet notes (`operations/nodes.md`) may want updating.
+3. **demo-hp remains key-less** — this train deployed over G1 break-glass (worked cleanly, secret
+   file→file + shredded). Standing open item: the tailnet key expiry (2027-01-17) toggle.
+4. `felhom-agent/CONTEXT.md`'s top bullet ("agent DOWN at remote site, deployed 0.90.0") was stale
+   history — superseded note added this train.
+5. Journal `--since` on the boxes is interpreted in **host-local CEST**; first query this run was
+   2 h off before correction (the standing host=CEST vs UTC gotcha).

documentation/runbooks/RUNBOOK-onboarding-draft-v4.md

@@ -54,6 +54,10 @@
    (`felhom-agent/CHANGELOG.md` v0.93.0, 2026-07-21) — ceremonies run **box-side**, so the fix
    protects a ceremony only once the train is **BUILT + DEPLOYED to that box**; source-only does
    not count. Already-issued codes stay valid.
+   **DISCHARGED 2026-07-22** — agent 0.93.0 published (sha `a68b2ff7…`), Day-0-manifest-vouched,
+   and deployed to both fleet boxes (`demo-felhom-8363b5` + `demo-hp-bb76ea`), clean-restart
+   verified; any future manifest-driven install also lands ≥ 0.93.0. Record:
+   `pilot/RUNBOOK-publish-agent-0.93-2026-07-22.md`.
 
 ## B. Install + bind (~15 min wall-clock)