b14b5a65d9
ROADMAP: - R-39 gains the full live diagnosis and REFUTES the brief's hypothesis. The generation IS bumped (SetHostDesired bumps unconditionally, 2->3) and applyPBSDR is exonerated, so no hub fix was shipped. The real mechanism is a signal mismatch: the hub's re-consume signal is a generation bump + poke, while the agent re-applies on a change of the DESCRIPTOR CONTENT HASH (manager.go ~L235). An ep0 re-issue re-keys the secret of an EXISTING token, so token_id/fingerprint are unchanged, the descriptor is byte-identical, the hash never moves, and the fresh secret is never consumed -> 401 forever. Proof: consumed-failed.json carries the same hash a4e5424... as the marker written two minutes before the re-issue. Records the second defect found while healing (wrapper reconcile passing --server, fixed in agent v0.90.1), marks the box HEALED with evidence (pvesm active, token 200, a real 9.7 GB encrypted backup listed PBS-side), and leaves the fleet fix explicitly pending its own spec. - R-33 collapses to SHIPPED (scripts v1.21.0), incl. why TimeoutStartSec=infinity is the load-bearing half. - Pre-invite checklist: golden target moves 0.145.x -> 0.146.0 and notes it is now MORE stale, since v0.146.0 is live on the demo box while the golden still bakes 0.143.0. REPORT overwritten with the train: R-39 diagnosis verbatim + heal evidence, the two ISO shas with the byte-identical-payload verification, the nav polish and why the screenshot leg could not be done (the demo controller password is customer-owned since the claim flow, so the build-server credentials are stale), Phase 4 skipped cleanly, and Phase 5 deferred rather than half-run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nn3VgQk9iwEGgyx6QJ2NvE
180 lines
11 KiB
Markdown
180 lines
11 KiB
Markdown
# felhom.eu — task reports
|
||
|
||
> **Overwrite** this file with a summary of the most recent task only (uniform with the other repos; not cumulative). The cumulative hub history lives in [hub/CHANGELOG.md](hub/CHANGELOG.md); the scripts history lives in [scripts/CHANGELOG.md](scripts/CHANGELOG.md).
|
||
|
||
## Pre-travel train — R-39 heal · scripts v1.21.0 + ISOs · nav polish · (golden deferred) — 2026-07-18
|
||
|
||
**Commits (this repo):** `bcdb042` scripts v1.21.0 · ROADMAP touch (R-39 diagnosis + R-33 collapse).
|
||
**Sibling commits:** `felhom-agent` `9596d5a` (v0.90.1) + `f22f70c` (report) · `felhom-controller`
|
||
`24d23b8` (v0.146.0) + `fd93020` (accordion tests).
|
||
|
||
**Phases 1–3 shipped. Phase 4 skipped cleanly (its own "time-permitting"). Phase 5 (golden 0.146.0
|
||
+ publish) NOT started — see the closing section.**
|
||
|
||
---
|
||
|
||
## Phase 1 — R-39: diagnosis, and the brief's hypothesis refuted
|
||
|
||
**The conditional hub fix was NOT shipped, because its condition proved false.** The brief said to
|
||
ship a generation-bump fix "only if step 1–2 pin the mechanism to *re-mint fails to bump the
|
||
generation*". It does not:
|
||
|
||
- `store.SetHostDesired` bumps `desired_generation` **unconditionally** — it went **2 → 3** on the
|
||
re-issue.
|
||
- `web/configs.go`'s `applyPBSDR` is **exonerated**: its "idempotent … no re-key, no second secret,
|
||
no spurious generation bump" comment at ~L618 is accurate and guarded by the
|
||
`cur != nil && cur.Namespace != ""` early return. The hub log shows mint #2 came from the
|
||
**re-issue** path, not from an Edit-tab Save. The comment-vs-behaviour contradiction the brief
|
||
expected does not exist.
|
||
|
||
**The real mechanism is a signal mismatch between two tiers.** The hub's re-consume signal is *a
|
||
generation bump + a poke*. The agent's re-apply trigger is *a change in the descriptor content hash*
|
||
(`felhom-agent internal/pbsdr/manager.go` ~L235):
|
||
|
||
```go
|
||
if mk := m.loadMarker(); mk != nil && mk.Hash == h && (cf == nil || cf.Hash != h) {
|
||
return // idempotent: this exact descriptor already converged
|
||
}
|
||
```
|
||
|
||
An ep0 credential re-issue re-keys the **secret of an existing token**, so `token_id` and
|
||
`fingerprint` never change and the descriptor stays **byte-identical** — only the side-table
|
||
`host_pbs_secrets` row rotates. Same hash → converged agent short-circuits → the fresh secret is
|
||
never consumed → the box keeps presenting a revoked credential → **401 forever**. Proof in one line:
|
||
`consumed-failed.json` carries hash `a4e5424…`, **identical** to the `marker.json` written two
|
||
minutes before the re-issue. The comment at `hub/internal/web/pbsdr.go:320` asserts the reissue
|
||
refreshes the descriptor "with the NEW token_id/fingerprint" — false for this op.
|
||
|
||
**Timeline (hub log is CEST; the hub DB is UTC — a split *within one service*):**
|
||
|
||
| CEST | Event |
|
||
|---|---|
|
||
| 18:30:51 | `pbsdr provisioned … gen 2; secret stored consume-once` — mint #1, via the WG-registration hook, **with** a generation bump |
|
||
| 18:45:51 | agent consumes mint #1 → `converged state=applied` |
|
||
| 18:47:52 | `pbsdr credentials **re-issued** … fresh consume-once secret stored` — mint #2, `consumed_at` stayed NULL |
|
||
|
||
**A second, independent defect, found while healing.** `configs/felhom-pbs-apply`'s `reconcile`
|
||
passed `--server` to `pvesm set`; PVE treats `server` as **create-only** and rejects the whole call
|
||
even when the value is byte-identical. So *every* re-apply exited 255 — and because the agent
|
||
consumes the one-time secret **before** invoking the wrapper, each re-issue **burned a credential**.
|
||
Proven live before writing code: with `--server` → rejected; without → **rc 0**. Fixed in **agent
|
||
v0.90.1** (one argv line + red-proof `TestReconcileNeverPassesServerToPvesmSet`, verified red then
|
||
green; it handles two vacuous-pass traps — CRLF line endings, and the WHY comment quoting the very
|
||
flag under test).
|
||
|
||
**Cost I incurred:** proving the mechanism consumed the pending secret against the still-unfixed
|
||
wrapper, so it burned. The box was already 401 before and after — no functional regression — but the
|
||
recoverable state was gone until an operator re-issue. The agent parked correctly in
|
||
`consumed-failed.json` with `NOT retrying silently`: **no burn loop**, the fail-safe worked.
|
||
|
||
**HEALED — Viktor's re-issue click closed the chain in 9 s:** hub re-issued 20:28:44 → agent
|
||
consumed 20:28:51 → `converged state=applied` 20:28:53, with the patched wrapper.
|
||
|
||
| Check | Before | After |
|
||
|---|---|---|
|
||
| `pvesm status` | `401 Unauthorized` / `inactive` | **`active`** |
|
||
| Direct token probe `/api2/json/version` | `401` | **`200`** |
|
||
| Real backup | none possible | **`felhom-pbs:backup/ct/9201/2026-07-18T18:31:06Z`, 9 744 319 312 B, 13m36s** |
|
||
|
||
Encrypted under fingerprint `7e:a6:af:f7:ea:6d:3e:d9` — the **escrowed** key, the one customer zero
|
||
holds the recovery code for. The DR tier's **first real backup on the reborn box**. Nothing was
|
||
destroyed: `.pw`, `.enc` (K) and the `storage.cfg` entry verified intact (PVE rejects atomically, so
|
||
the set-only law held).
|
||
|
||
**Left for the fleet spec, deliberately not improvised:** (a) make a fresh unconsumed secret actually
|
||
un-converge the agent; (b) fix the verify loop's read path — it reads `/etc/pve/priv/storage/<id>.pw`
|
||
directly as non-root, a file it can only ever *write* through the root wrapper (`/etc/pve/priv` is
|
||
`0700 root:www-data`; sudoers exposes `create|reconcile|grant`, **no read verb**); (c) an auth probe
|
||
so `applied` can never mean `401`.
|
||
|
||
---
|
||
|
||
## Phase 2 — scripts v1.21.0 + fresh ISOs
|
||
|
||
`run_pairing()` now loops **inside** the script (30s sleep — hub-side rate unchanged) instead of
|
||
exiting non-zero per poll, so the unit sits in `activating` and systemd prints nothing on the
|
||
customer's console. Registration split into `register_appliance()` whose transient failures the loop
|
||
retries. Journal quiet but not dark: logged once on entry, then a 10-minute heartbeat; `410` still
|
||
exits non-zero on purpose. Console banner every 5 min, single accented spelling, plus the missing
|
||
reassurance („Ez a képernyő magától frissül").
|
||
|
||
**The load-bearing half is `TimeoutStartSec=infinity`** — a `Type=oneshot` ExecStart is killed at 90s,
|
||
so without it systemd would kill the new wait and `Restart=on-failure` would silently reinstate the
|
||
exact spam this removes, *after appearing to work for the first three polls*.
|
||
|
||
**Verified behaviourally**, in a container against a stub hub answering `204` five times then
|
||
delivering: **one log line plus one heartbeat, zero exits between polls**, then a clean fall-through
|
||
to the direct install and `exit 0`. The old design produced 5 unit invocations and 5 `Failed to
|
||
start` console lines for that same sequence.
|
||
|
||
**ISOs rebuilt (both `--pairing`, `--loader mkimage`, same PVE input `proxmox-ve_9.2-1.iso`
|
||
sha `4e88fe41…`), `secret-bearing: no`:**
|
||
|
||
| ISO | sha256 |
|
||
|---|---|
|
||
| `felhom-pve-9.2-1-v1.21.0-n100-generic-mkimage.iso` (safety) | `b1b25fd412b779bcacbfaa4c59002ee80a8f35e4c0bd1248cc35d997f967cb90` |
|
||
| `felhom-pve-9.2-1-v1.21.0-n100-demo-generic-mkimage.iso` (real) | `90a0fb7da3f3f11d315bf1cdf55a7da84ab9e6d75e06b2942041470f7864031d` |
|
||
|
||
Both at `180:/mnt/5_hdd/felhom.eu/felhom-iso/out/`. **Verified the fix actually shipped inside the
|
||
artifact**, not just in git: extracted the embedded first-boot payload with xorriso and decoded it —
|
||
the shipped `felhom-bootstrap.sh` is **byte-identical to the committed source**, carries the new
|
||
cadence constants and the `while true` loop, the old `will poll again in 30s` exit line is **gone**,
|
||
and the embedded unit carries `TimeoutStartSec=infinity`. Viktor flashes the stick.
|
||
|
||
---
|
||
|
||
## Phase 3 — controller v0.146.0 nav polish
|
||
|
||
Built, pushed and **deployed to guest 9201** (`0.146.0 Up (healthy)`).
|
||
|
||
- **Scrollbars:** thin + hairline-coloured; `scrollbar-width`/`scrollbar-color` for Firefox **and**
|
||
`::-webkit-scrollbar` (8px, thumb `--line`, hover `--text-3`, `--radius`) for WebKit/Blink, since
|
||
neither alone covers the browsers customers use. `.sidebar` → `--bg-2` track, `html` → `--bg-0`.
|
||
Tokens only.
|
||
- **Collapsible groups:** Tárhely / Biztonsági mentés / Megosztás as accordions, chevron, exactly one
|
||
open. Header is a **real `<button>`** with `aria-expanded` + `aria-controls` + `:focus-visible`, so
|
||
keyboard/AT reachability is real rather than simulated. **Nothing became unreachable** — checked
|
||
first: every group's landing page is also its first sub-item. **Progressive enhancement** — the
|
||
active group is opened **server-side**, so it is correct before any JS runs. **No layout jump** —
|
||
`grid-template-rows: 0fr → 1fr` rather than `max-height`, animating to the content's real height
|
||
with no magic number to drift; the toggle reserves its active border as transparent; both
|
||
transitions off under `prefers-reduced-motion`.
|
||
|
||
All design-v2 gates PASS (`template_id_gate`, `emoji_gate`, `native_confirm_gate`,
|
||
`offbox_rename_gate`, `mojibake_gate`, `app_row_dedup_gate`); build/vet/tests green.
|
||
`docker_run_volume_path_gate` still fails on `estimate.go:179` — that is **R-29**, pre-existing,
|
||
verified to fail identically on the untouched tree, and deliberately not bundled.
|
||
|
||
**Screenshot leg NOT done, and here is the honest reason.** The demo controller's password is
|
||
**customer-owned** since the claim flow — Viktor set it during the rehearsal — so the credentials on
|
||
the build server are stale and a curl-login returns the `Bejelentkezés` page. Instead of asserting
|
||
nothing, four **render tests** (`internal/web/nav_accordion_test.go`) pin the server-side half through
|
||
the real shared layout: every sub-page opens its own group with `aria-expanded=true` and an `.active`
|
||
toggle and **exactly one** group open (the count is asserted, not just the expected group); a page
|
||
outside any group opens nothing; every group's landing page still exists as a sub-link; the toggle is
|
||
a real button whose `aria-controls` targets a real element. **Red-proofed** — removing the `is-open`
|
||
marker fails two assertions on both storage pages. The visual leg still wants Viktor's browser.
|
||
|
||
---
|
||
|
||
## Phase 4 — skipped cleanly
|
||
|
||
Its own instruction was "time-permitting; skip cleanly if not". Nothing was started, so nothing is
|
||
half-done. Note that its (a) auto-mint self-bind link, (b) post-RESET health card and (c)
|
||
unprovisioned-offsite flash correspond to **R-36 / R-37 / R-36** and remain open as written. The
|
||
conditional Phase-1 hub fix is **not** part of any v0.67.0 train, because its condition was refuted.
|
||
|
||
## Phase 5 — NOT started (the honest hand-off)
|
||
|
||
The golden bake is a nested-VM procedure (snapshot-revert → cold boot → transient `golden-bake` unit
|
||
→ 404 pre-gate → upload → sha verify → teardown → token-leak grep). It is well recorded
|
||
(`pilot/RUNBOOK-publish-0.85-0.120-2026-07-12` Phase C) and entirely repeatable, but it is a long
|
||
multi-step operation with a revert dance at the end, and this session had already run long. Starting
|
||
it and running out mid-way would leave a booted drill VM and a half-published artifact — strictly
|
||
worse than a clean stop. It also ends in a **STOP → Viktor** regardless, since the manifest and floor
|
||
saves are password-gated clicks.
|
||
|
||
**So: controller v0.146.0 is live on the demo box, but the golden still bakes 0.143.0.** The
|
||
pre-invite checklist item "rebuild golden → 0.145.x" is therefore now "rebuild golden → **0.146.0**",
|
||
and it is the first thing to pick up next session.
|