# REPORT — felhom-agent v0.90.1 (R-39 hotfix: PBS reconcile must not pass `--server`) **Date:** 2026-07-18 · **Baseline:** v0.90.0 → **v0.90.1** · **Scope:** config-only. `configs/felhom-pbs-apply` (one argv line + its WHY) and one red-proof test. **The Go binary is unchanged** — this ships with the next agent deploy as a config artifact. **Green:** `go build ./... && go vet ./... && go test ./...` all pass. ## What was wrong `reconcile` built its argv as `args=(--server "$server" --fingerprint "$fp")`. PVE treats a PBS storage's `server` as a **create-only** parameter and rejects the whole `pvesm set` call — `can't change value of fixed parameter 'server'` — **even when the value is byte-identical to the stored one**. `reconcile` could therefore never succeed against an existing entry. That is severe rather than cosmetic because **the agent consumes the hub's one-time PBS token secret before invoking the wrapper**. Every failed reconcile burned a credential: ``` hub "Re-issue PBS credentials" → fresh one-time secret minted agent → secret CONSUMED (single-use, now spent) wrapper reconcile → exit 255, "fixed parameter 'server'" storage entry → still pinned to the REVOKED secret → 401 forever ``` ## How it was found and proven Live-diagnosed on the N100 demo host during the 2026-07-18 rehearsal wrap (`felhom.eu/documentation/tests/VALIDATION-n100-rehearsal-2026-07-18.md` finding F2, ROADMAP **R-39**). The box had been reporting `pbsdr: converged state=applied` while `pvesm status` returned `401 Unauthorized` / `inactive`. Proven directly on the live entry before any code was written: | Probe | Result | |---|---| | `pvesm set felhom-pbs --server --fingerprint ` | **rejected** — `can't change value of fixed parameter 'server'` | | `pvesm set felhom-pbs --fingerprint ` | **rc 0 — accepted** | The encryption key `K` (`.enc`) and the `.pw` store were verified byte-untouched after the rejected call: PVE rejects atomically, so the set-only law held throughout. ## The fix Drop `--server` from the reconcile argv. The server address is immutable by construction — relocating a PBS endpoint requires a fresh `create` — so there was never anything for `reconcile` to reconcile there. `--fingerprint` (and `--password` when a secret is fed on stdin) remain, which is the mutable identity the verb exists to push. ## Red-proof `TestReconcileNeverPassesServerToPvesmSet` (`internal/pbsdr/manager_test.go`) isolates the `reconcile)` block from the shipped wrapper and asserts no `--server` reaches `pvesm set`, plus that `--fingerprint` is still pushed so the verb cannot be hollowed out. **Verified RED against the unfixed wrapper and GREEN after the fix.** Two traps the proof handles explicitly, both of which would have made it pass vacuously — and both of which actually fired during authoring, which is the argument for running a proof red first: - **Line endings.** This repo is cloned on Windows; the working copy carries CRLF, so an `\n`-only pattern matches nothing and the guard passes silently. The pattern is `\r?\n` throughout. The first run failed with "could not locate the reconcile) block" — the vacuous-pass failure mode, caught only because the test was run against the broken wrapper first. - **Comments.** The WHY note above the fix necessarily quotes `--server`, the very flag the test forbids, so the suite went red *after* the fix was applied. Comment lines are now stripped before matching. ## Deliberately NOT fixed here — each still open 1. **R-39's primary half.** The agent re-applies on a change of the **descriptor hash** (`internal/pbsdr/manager.go` ~L235: `if mk := m.loadMarker(); mk != nil && mk.Hash == h && (cf == nil || cf.Hash != h) { return }`). A hub credential re-issue leaves the descriptor **byte-identical** — same `token_id`, same `fingerprint`; only the side-table secret rotates — and bumps only `desired_generation`. So a converged agent short-circuits and never consumes the fresh secret. This wrapper fix makes the apply *succeed* once the agent is made to re-apply; **it does not make it re-apply.** 2. **The verify-loop read.** `pbs: cannot read token secret … permission denied` — the non-root agent reads `/etc/pve/priv/storage/.pw` **directly**, a path it can only ever *write* through the root wrapper. `/etc/pve/priv` is `0700 root:www-data` and sudoers exposes only `create|reconcile|grant` — **there is no read verb**. The loop is permanently blind to exactly the failure it exists to catch. Both ride the spec'd R-39 agent train. Recorded, not improvised. ## The demo box — HEALED and proven (2026-07-18) The wrapper is hotfixed on the N100 (`/usr/local/sbin/felhom-pbs-apply`, backup kept at `.bak-20260718-preR39`, `bash -n` clean) so the repo and the box agree. Diagnosis had consumed the one pending secret against the still-unfixed wrapper; the agent parked correctly in `consumed-failed.json` with `NOT retrying silently` — **no burn loop**, the fail-safe worked. Viktor then clicked **"Re-issue PBS credentials"** (the one password-gated action CC cannot perform) and the whole chain closed itself in **9 seconds**: | Time (CEST) | Event | |---|---| | 20:28:44 | hub: `tenantsync: reissue ok` → `pbsdr credentials re-issued … fresh consume-once secret stored` | | 20:28:51 | agent: `one-time token secret consumed`, wrapper `reconcile` invoked — **now the patched one** | | 20:28:53 | agent: **`pbsdr: converged state=applied`** — no CONSUMED-BUT-FAILED this time | `consumed-failed.json` was cleared automatically by `finishConverged` and `marker.json` rewritten. The agent picked the re-issue up unaided because `marker.json` was absent, so the L235 hash short-circuit did not apply. **Proof the tier is genuinely alive, not merely reported alive:** | Check | Before | After | |---|---|---| | `pvesm status` | `401 Unauthorized` / `inactive` | **`active`** | | Direct token probe, `/api2/json/version` | `401` | **`200`** | | Real backup | none possible | **`felhom-pbs:backup/ct/9201/2026-07-18T18:31:06Z`, 9 744 319 312 B, finished in 13m36s** | The backup is encrypted under key fingerprint `7e:a6:af:f7:ea:6d:3e:d9` — the **escrowed** key, i.e. the one customer zero holds the recovery code for. This is the DR tier's **first real backup on the reborn box**. Nothing was destroyed at any point: `.pw`, `.enc` (K), and the `storage.cfg` entry were verified intact after the rejected `pvesm set` (PVE rejects atomically, so the set-only law held).