## v0.146.1 — R-861 review fixes: the signed update flips a root-owned copy; no Wants=/continuations in mount units; the escrow read follows no symlink anywhere (2026-10-05)

Released by `scripts/release-agent.sh`: binary sha256 `badd6c9a2e40c8bfe856d2d1a203443b21b7eb92ecc35d6090ab44518d4d082a`,
config bundle sha256 `42333e969028867ad8142335e6c1bc4040eec231de0d8d330c2d4b2cf7bc3442`. **Supersedes v0.146.0, which
was released but never vouched or delivered to any box.** The same order applies: signed `agent_update` first, then the
signed `agent_config_update`.

A background security review of the v0.146.0 commit found three holes in the new code; each is fixed and red-proved
(`felhom.eu/documentation/audits/hub-safety-2026-10-05/partF/red-proof.txt`, S1–S3):

- **S1 — a race in the signed update.** `felhom-os-apply` hashed the agent's staged file and then let the A/B wrapper copy
  it BY PATH; the agent owns that directory and could swap the file in between. Now the root step reads the file ONCE
  (`read_staged_once`: O_NOFOLLOW, fstat, owner, size), hashes those bytes, writes them to a root-owned directory
  (`/var/lib/felhom-os-apply/agent-update/`) and hands ONLY that copy to `felhom-selfupdate-guarded apply`, which now
  refuses any other directory, a symlink, or a file not owned by root. Tests: `AgentUpdate` (+1),
  `SelfupdateWrapperConfinement`.
- **S2 — an allowlist escape in `felhom-priv-apply`.** `[Unit]` accepted `Wants=`/`Requires=`/`Before=` naming any unit, so
  a mount unit could start e.g. `reboot.target`. `[Unit]` now holds only `Description` and `After=local-fs-pre.target`
  (what the renderers write), and any line ending in a backslash (a systemd continuation this parser would read
  differently) is refused. Tests `test_U2_wants_starts_another_unit`, `test_U2_continuation_line`.
- **S3 — a path traversal in the escrow read.** `O_NOFOLLOW` guards only the last component; a symlinked DIRECTORY in the
  agent's own state dir still redirected the root read. `readStagedNoFollow` now walks the path from `/` with
  `openat(O_NOFOLLOW)` per component. Test `TestAttach_RefusesASymlinkedDirectory`.

## v0.146.0 — the agent's root grants narrowed: exact sudo patterns, a root content checker, fixed files from the bundle, the signed update checked as root (R-861) (2026-10-05)

Released by `scripts/release-agent.sh`: binary sha256 `b860af465076041e07f35fed1b12d64ae2b2985d8995f0ce167418d39c2b00d5`,
config bundle sha256 `161c737e523aa7910cf32ce41b83f989569bee55b8c5938e7211c92aef68548e`. **Order on a box: the signed
`agent_update` FIRST (the old bundle still grants the old flip), then the signed `agent_config_update`.** Between the
two (minutes) the new agent's checker calls are refused and retried; nothing is lost. After the bundle, an agent BELOW
0.146.0 cannot update itself on that box any more (the unsigned flip grant is gone) — deliver both together.

Design: `felhom.eu/documentation/architecture/03-host-agent.md` §3.1 (new). Measured before the change (real sudo
1.9.16, a throwaway container): the v0.145.0 sudoers let **23 of 29** attack command lines through; v0.146.0 lets
**0** through and still allows all **64** commands the agent's capability check uses.

- **Exact patterns.** A sudoers `*` in the arguments also matches spaces: `pct set [0-9]* -onboot 1` matched
  `pct set 100 --dev0 /dev/sda -onboot 1` (a raw host disk for a guest), `mount --bind /mnt/*/felhom-data
  /mnt/felhom-drives/*` matched a `..` path onto `/etc/sudoers.d`, `nft add element … *` took a chained `; flush
  ruleset`. Every varying argument list is now a sudo regex (`^…$`): one value per slot, a fixed character set, no
  `..`, no extra argument. `TestSudoersRefusesTheR861Injections` (29 attacks) + `TestManifestCoveredBySudoers`
  (regex-aware now).
- **`felhom-priv-apply`** (new root wrapper, in the bundle). A systemd mount/automount unit, a dnsmasq drop-in, the
  WireGuard config and the OOB sshd config + felhom-op key reach their root-read places only through it: fixed source,
  fixed destination, CONTENT checked against what the agent's renderers write (no `[Service]`, `Where=` only
  `/mnt/<name>` or `/mnt/felhom-drives/<name>` and equal to the unit name, no `bind`/`suid`; a network share must carry
  `nosuid,nodev`; no `dhcp-script=`; no `PostUp=`; the sshd config only the one template with its Port). Its 30 tests
  (`configs/test_felhom_priv_apply.py`) + Go contract tests feeding each renderer's real output
  (`internal/privapplytest`). Pre-flight: every live file on both demo boxes reads OK.
- **NFS/SMB options gain `nosuid,nodev`** (a set-uid file on a server outside the box never acts on the host).
- **Fixed files from the bundle.** The guest pre-start hook (`/var/lib/vz/snippets/felhom-guest-hook.sh`, run as root at
  every guest start) and the shared drive parent script + unit are bundle files now (byte-identical to the agent's
  constants, pinned). The agent no longer installs them from `/tmp`; it checks them (`guesthook.SnippetReady`,
  `ensureSharedParentBoot`) and only registers / enables.
- **The signed update is checked as root.** `felhom-os-apply` mode `agent_update` verifies the operator signature
  (root-owned signers, this host, the window, the nonce), re-hashes the staged binary against the SIGNED sha, then runs
  the A/B flip; `felhom-selfupdate-guarded apply` is no longer in the agent's sudoers. 7 tests (`AgentUpdate`).
- **The root escrow run reads no path from the agent's config.** As root it pins the PVE secret dir and the WireGuard
  state dir to their defaults, refuses a storage id that is a path, and reads its two staged files without following a
  symlink (`readStagedNoFollow`) — before, a symlink in the agent's own directory sealed any root file into the blob.
- **Not narrowed here (named in `03` §3.1):** `FELHOM_CONTROLLERSWAP` stays guest-scoped (a compromised agent can run a
  chosen controller image in the guest — the household's data, not host root); `FELHOM_ESCROW` still hands the agent R
  by design (the agent relays the ceremony); the mkfs / pbs-apply / backup-target wrappers keep a coarse argument and
  their own checks.
- Red-proofs F1–F9: `felhom.eu/documentation/audits/hub-safety-2026-10-05/partF/red-proof.txt` (F1's first run did NOT
  convict — the name rule masked it — and the test now uses the pair only the Where rule stops).

## v0.145.0 — the OS update repairs itself after a power cut; a short-session box gets restore-tested; "sent late" (R-876, R-874, R-875) (2026-10-05)

Released by `scripts/release-agent.sh`: binary sha256 `894da35c7b9e1ac78885690b78352b634e6831e7d99b321573c75d878db8886e`,
config bundle sha256 `78c00adce662d2d966b2ac50ebde46cde1ae225f0107c6a7c02b70ec8ce80c4f`. The wrapper changed: a box
needs the signed `agent_update` AND the signed `agent_config_update`.

- **R-876.** After a crash during an install, `dpkg --audit` can read clean while dpkg's update journal
  (`/var/lib/dpkg/updates/`) is not — and apt refuses every install until `dpkg --configure -a` (measured on demo-hp
  2026-10-05: every later pass failed until a person typed it). The wrapper now reads `--audit` and the journal in ONE
  `sh -c` call (`DPKG_STATE_SCRIPT`) — a clean pass still costs one call (R-845's speed, pinned) — and repairs when
  either shows something; as a belt, when apt itself says "dpkg was interrupted", it repairs and retries the install
  ONCE. `REPAIR` now logs `journal=N`; a journal still not empty after the repair refuses (R13). Tests
  `CrashLeftTheJournal` (the measured shape, the speed, the belt); 3 red-proofs.
- **R-874.** The restore-test's first due-check runs 30 minutes after the agent starts (`DefaultFirstEval`), then every
  interval; a box whose power-on sessions are shorter than the 6 h interval never evaluated. A crash-looping agent
  restarting faster than 30 minutes still never evaluates (the earned restraint, pinned).
- **R-875.** A kept report's reason is neutral — "sent late — kept on the box until the hub could take it" — the copy
  cannot tell a killed agent from an absent hub.
- Red-proofs: `felhom.eu/documentation/audits/catchup-2026-10-05/part{C,D}/`.

## v0.144.1 — a killed pass really keeps its report: the wrapper survives a dead reader; the agent looks again every 5 minutes (R-868, measured live) (2026-10-05)

Released by `scripts/release-agent.sh`: binary sha256 `6ccd521d47e64999017e8eb5bc613d724543cfdc5ef9b13bcae9e3ea8c53b8f3`,
config bundle sha256 `e89a9ddfb767e177f8874d56f3dcd3bfd47409d157ff830d362653333bf815e8`. The wrapper changed again:
a box needs the signed `agent_update` AND the signed `agent_config_update`.

- **Found live on demo-hp 2026-10-05 05:45 UTC with v0.144.0** (the night's A5 shape: kill -9 of the pass and the
  daemon while apt-get ran): apt finished all 13 packages, but the wrapper's next log line went to a stderr pipe no
  process read any more → `BrokenPipeError` → the wrapper died before it saved its report copy (journal: `PLAN
  upgrade=13`, then nothing; no copy; the hub got nothing). v0.144.0's mechanism was right and never reached.
  `Runner.log` and the final `OSAPPLY-REPORT` line now survive a dead reader (the journal still gets every line).
  Test `AgentDiesMidPass` drives the REAL `log()` into a pipe that breaks while apt-get runs.
- **Also found live:** the restarted daemon looked for kept copies ~7 s before the orphaned wrapper wrote one. The
  daemon now looks at start and every 5 minutes (`Leg.SendUnsentLoop`); `TestR868_ACopyWrittenAfterTheStartIsSentByTheLoop`.
- Red-proofs: `felhom.eu/documentation/audits/night-fixes-2026-10-05/partD/r868-brokenpipe-red-proof.txt`.
- A second agent release in one session, against "one release per repo": recorded as `09` decision 108 (operator may
  reverse) — the alternative was to ship a fix proven not to work.

## v0.144.0 — R8 measures the real download; an OS pass reports even when its agent was killed; the debug pass runs with the hub away (R-865, R-868, R-866) (2026-10-05)

Released by `scripts/release-agent.sh`: binary sha256 `f18093c3466749ec4cd47f83f97a401160a24bad1e704f1183051adad14db928`,
config bundle `felhom-config-bundle.json` sha256 `6acf42fe46df5223384d767801cb2bf73238ba4dab82debb7811ca2f790591d8`.
The wrapper `felhom-os-apply` changed, so a box needs BOTH the signed `agent_update` and the signed `agent_config_update`.

- **R-865.** `download_bytes` runs `apt-get --print-uris` WITHOUT `-s`: with `-s` apt prints the simulation and no URI
  list, so R8 summed 0 B and only its 500 MB floor ever applied. `--print-uris` alone downloads nothing (measured on
  9202: the archive cache and the versions unchanged). The test fake now answers like real apt (with `-s`: no URIs),
  and `test_R8_counts_the_real_download` / `test_download_bytes_never_simulates` pin it.
- **R-868.** The wrapper writes every apply pass's report to `<plan dir>/report-<run>-<layer>-apply.json` before it
  prints it (root writes into the agent's dir: the dir opened O_NOFOLLOW and checked to be the agent's own, the file
  created O_EXCL|O_NOFOLLOW, 0600, handed to the agent). The plan now carries `run_id`, `trigger`, `ring`, echoed in
  the report. The agent deletes the copy once the hub has the report; a copy left on disk (the agent was killed, or
  the hub was away) is sent at the agent's start and before every pass (`Leg.SendUnsent`), then deleted. A pass lock
  (flock on `pass.lock`, across the daemon and a selftest) keeps the sender off a pass that is still running.
- **R-866.** The daemon saves the hub's newest os_update block (`os-update-block.json`); `--selftest=os-update` uses it
  when the hub cannot be reached and says so in its header (`block=SAVED(<time>; hub unreachable: …)`); with no hub
  and nothing saved it does not run.
- Tests: `configs/test_felhom_os_apply.py` (UnsentReport, SaveReportOnDisk — real files, symlink cases), Go
  `TestR868_*`, `TestR866_*`. Red-proofs: `felhom.eu/documentation/audits/night-fixes-2026-10-05/part{C,D}/`.

## v0.143.0 — the config bundle: a signed route for a box's root-owned files (R-840, decision 96) (2026-10-04)

Released by `scripts/release-agent.sh`: binary sha256 `41c0d3060013dfda795262147454248149bee0888c0935170fdb85de6e7a35da`,
config bundle `felhom-config-bundle.json` sha256 `8d7273cf5313ef62b867cb6f831c631923a436452d6f90b8ff7f0771170396ba`.

- **The bundle.** Every root-owned file the installer's step 5 writes (sudoers ×2, the five wrappers, the crash guard
  and its units, the agent and rollback units, the start-limit drop-in, the mgmt watchdog, the OOB belt's files) as ONE
  reproducible JSON file, built by `scripts/build-config-bundle.py` from `BUNDLE_FILES` in `configs/felhom-os-apply`
  (one table) and published beside the binary. The installer (1.31.0) installs the same file.
- **The route.** A signed `agent_config_update` {agent_version, bundle_sha256} (`felhom-opsign -op agent_config_update
  -bundle-sha256 …`). The agent is the courier (downloads, checks the sha, hands over); `felhom-os-apply` mode `bundle`
  verifies it ITSELF: the operator signature against the root-owned `/etc/felhom/operator-signers` (or, when that file
  is missing, ONLY the installer's pinned key, after which it creates the file with exactly that key), the host binding,
  the window, its own nonce; the bundle sha; every path in `BUNDLE_FILES` (R16) and never a trust file (R17); every
  content check before the first write (visudo, sh/bash -n, python, unit sections, the RuntimeDirectory guard, User=,
  nft -c, and that the route itself survives). Atomic per file, previous copies kept under
  `/var/lib/felhom-os-apply/bundle-prev/`; a self-check after (visudo -c, `sudo -l` lists the route, the new wrapper's
  `--self-check`, the self-update wrapper's usage, the crash guard's status = kernel.panic); any failure puts every
  previous copy back. A newly installed crash guard is started (`enable --now`: kernel.panic for this boot, no reboot).
  Record `/etc/felhom/config-bundle.json`; the agent reports it as `system.config_bundle`, the facts mode adds drift.
- **Bootstrap.** A box whose `felhom-os-apply` predates 0.143.0 cannot take the first bundle by the route (nothing on it
  can write a root file from a signed job): `felhom.eu/scripts/felhom-bundle-bootstrap.sh` is the one by-hand step.
- **`build-golden.sh` 3.2.0** (not part of the binary): `GOLDEN_GUEST_PKGS` brings the template to exactly the
  approved guest release (only installed packages, never newer, never a removal or a new package) and prints the
  first-night count.
- Tests: `configs/test_felhom_config_bundle.py` (43; 22 of 22 mutants red), `internal/osupdate/bundle_test.go`,
  `internal/hub/bundle_record_test.go`. Live: `felhom.eu/documentation/audits/r840-config-bundle-2026-10-04/partB/`.

## v0.142.1 — a Docker step no longer leaves the controller and traefik blind (R-858, `09` decision 95)

> **RELEASED 2026-10-04** by `scripts/release-agent.sh` — tag `v0.142.1` (`4950030`), sha256
> `003f882a59abfc10021a1f97a4744ab78ab1e916b1392dcef132e7067f3c62bd`, verified by download. Not vouched at release time.

**MinAgent impact:** none. A same-day patch release, by the operator's ruling 95 after a live incident.

- **The incident.** v0.142.0's Docker step on demo-felhom (14:13 UTC) restarted dockerd, which recreates
  `/run/docker.sock`. live-restore kept every container running — including `felhom-controller` and `traefik`, which
  bind-mount the socket FILE and so kept the deleted inode. The controller could not reach Docker for 1 h 44 min (hub:
  DOWN, "docker not reachable"); its own health check stayed "healthy", so the step's health rule PASSED.
- **The fix.** After a Docker step that installed something, the wrapper restarts ONLY the containers that mount
  `/var/run/docker.sock` or `/run/docker.sock` (`os-apply: SOCKET-USERS restarted=…`; the apps and the engine untouched).
  The guest health reading gains `controller_docker_ok` (`docker exec felhom-controller docker version`), and the health
  rule fails when it is false — the consequence, not the mechanism.
- Proven live on demo-hp BEFORE the release (signed undo to 29.7.2): the two restarted, guest / controller / traefik on
  the same socket inode, healthy. Red-proofs: `felhom.eu/documentation/audits/os-docker-crash-2026-10-04/partE-incident/`.

## v0.142.0 — the Docker engine slow lane, the version report, the crash guard (`11` §5.8, §5.9; `09` decisions 87–89)

> **RELEASED 2026-10-04** by `scripts/release-agent.sh` — tag `v0.142.0` (`b1746c2`), sha256
> `7beb32224d6495e9561acfd3ad8a48393a799520f196080011cb27fceb6d1de6`, verified by download. Not vouched at release time.

**MinAgent impact:** none. **Needs hub v0.132.0** for the System page, the Docker approval and the crash events; an older
hub stores the new `system` stanza unread. **Needs the new root files** on an installed box (R-840): the wrapper,
`/etc/felhom/os-trust.json`, `/etc/felhom/operator-signers` and the crash guard — the installer 1.30.0 writes them; the
demo boxes got them by hand.

- **Docker `live-restore` ON** (decision 87). Wrapper mode `live-restore-on`: merge `"live-restore": true` into the
  guest's `/etc/docker/daemon.json` and `systemctl reload docker` — never a restart (R-835). An invalid daemon.json is
  left alone (R16); a reload that does not enable it puts the old file back. The leg runs it once before a Docker step;
  `--selftest=live-restore -vmid N` runs it by hand. Measured: demo-hp 24 containers, demo-felhom 5 — the same ids after.
- **The Docker engine slow lane** (`11` §5.8). Wrapper layer `docker`, lane `slow` only, the six Docker packages only,
  origin `Docker CE` only (R2; a Docker package in a fast-lane plan is refused). **R3 — the wrapper checks the authority
  itself**, never the agent's config: a ring-1 step or ANY undo needs a signed `os_docker_step` it verifies with
  `ssh-keygen -Y verify` against the ROOT-owned `/etc/felhom/operator-signers` (namespace `felhom-op-v1`, the blob's
  `key_id`), bound to `/etc/felhom/os-trust.json` `host_id`, inside its time window, never replayed (a root-owned nonce
  file), with exactly the signed packages and undo flag; an unsigned ring-0 step needs that file's
  `"ring0_slow_lane": true` (the demo boxes only, set by hand). **R15:** live-restore must be on. An undo may downgrade
  (`--allow-downgrades`) only inside a signed job. The leg: ring 0 runs the Docker step at night after a healthy guest
  and host step (`select pending-docker`); ring 1 never does — only `DockerStepExecutor` (signed job, heavy-op gate).
  **Health:** the guest rule + every container running at the start has the SAME id after + the engine reports the
  installed version; a changed id is `health_failed`. Measured ring 0: 29.7.x → 29.8.2 on both demo boxes, every id kept.
- **The version report** (R-852, decision 89). Wrapper mode `facts` (read-only): host Debian, running and next-boot
  kernel (`next_entry` > saved default > newest installed, by dpkg order), held packages (R-848), kernel taint (oops,
  warn), `kernel.panic`, the crash guard state; guest Debian, Docker engine, containerd, live-restore. The host report
  gains `system {pve_version, kernel_version, vmid, facts, facts_error}`, read at most every 10 min (~2 s);
  `--selftest=os-facts -vmid N`. A value nobody could read is `unknown`.
- **R-849:** the guest is scanned for "restart needed" on every pass too, so a guest restart clears it.
- **The crash guard** (decision 88, R-851): `configs/felhom-crash-guard` + `felhom-crash-guard.service` (early boot;
  its ExecStop writes a clean-stop marker) + an hourly re-arm timer + `/etc/felhom/crash-guard.conf`. A boot without
  the marker followed an unclean stop (a crash, a power cut or a hard reset — pstore saved nothing for a real panic on
  demo-hp, so they cannot be told apart). Armed: `kernel.panic = 10`. After the 2nd unclean boot within 60 min it
  TRIPS (`kernel.panic = 0`), so the 3rd crash within the hour leaves the box off; it re-arms after 24 h of normal
  running or `felhom-crash-guard rearm`. State in `/var/lib/felhom-crash-guard/state.json` (0644; read by facts).
- **After the tag (main only, not shipped to boxes): `configs/build-golden.sh` 3.1.0** — the golden's `daemon.json`
  carries `"live-restore": true` with a fail-closed assertion, and `GOLDEN_DOCKER_PKGS` pins the approved Docker engine
  set (all six `name=version`); without it the bake log warns that the set is the newest, not an approved one.
- Tests: wrapper 76 (DockerLane, LiveRestore, Facts, RealSignatureCheck with a throwaway key), crash guard 9, Go leg +
  executor; red-proofs `felhom.eu/documentation/audits/os-docker-crash-2026-10-04/partB/agent-redproofs.txt` (17 caught).

## v0.141.1 — "reboot needed" is true on the host (found live on demo-felhom, 2026-10-04)

> **RELEASED 2026-10-04** by `scripts/release-agent.sh` — tag `v0.141.1` (`a6bc3f1`), sha256
> `b712f577099fd2d374f648df1e825874821302fe1b46e93c7b4b70044fbe84b5`, verified by download. Not vouched at release time.

**MinAgent impact:** none. **Pairs with hub v0.131.1** (reads `reboot_scanned`); an older hub ignores the field.

A patch release in the same session as v0.141.0 — a deliberate exception to "one release per repo": v0.141.0's host
"reboot needed" was wrong in two ways, and it feeds an operator alarm.
- **The scan hid `lxc-start`.** The host scan skipped every process whose cgroup line contains `lxc`, to leave out the
  guests' own processes. `lxc-start` lives in `0::/lxc.monitor/<vmid>`, so it was skipped too. Measured: after a
  108-package host pass (libc6 included) `lxc-start` mapped 20 deleted files and the report said `reboot_needed:
  false`. The pattern is now `:/lxc/` (`RESTART_SKIP_CGROUP`), pinned by a test that runs `grep` against the measured
  cgroup lines.
- **A reboot never cleared it.** v0.141.0 scanned only after an install (R-845). The host now scans on EVERY pass (it
  is local, no `pct exec`); the guest still scans only after an install. The report carries `reboot_scanned`.
- Red-proofs: `felhom.eu/documentation/audits/os-host-lane-2026-10-04/partB/live-defects-redproofs.txt`.

## v0.141.0 — OS updates: the host fast lane (`11` §8 step 3); the tunnel status is true (R-841); the leg is fast (R-845)

> **RELEASED 2026-10-04** by `scripts/release-agent.sh` — tag `v0.141.0` (`cfba0d0`), sha256
> `6eaad9809613c4fca64aefc30cc16415528499bf48efe3a1daaa08af8a611aff`, verified by download. Not vouched at release time.

**MinAgent impact:** none required by any controller. **Needs hub v0.131.0** (`host_release`, the tunnel's three
states, layer-tagged OS reports). With an older hub the host step finds no host release (ring 1 → nothing) and the
tunnel's `detail` is ignored. Reads the cloudflared health check controller v0.292.0 adds; with an older controller
the tunnel is judged on the container state alone and `detail` says so.

- **R-841 — the tunnel.** The agent used to run `systemctl is-active cloudflared` on the HOST — a unit that does not
  exist (cloudflared is a container in the customer guest), so every box reported `inactive`. `GuestTunnelProber`
  now reads the guest's `cloudflared` container through the EXISTING sudoers line (`pct exec N -- docker inspect -f
  *`): state, exit code, and the Docker health status. Three states: `running` (healthy), `not_running` (stopped,
  absent, or running but NOT connected), `unknown` (could not ask, or the check is still starting). `detail` says why.
- **The host step** (`11` §8 step 3). After a healthy guest step, under the same heavy-op gate, the leg runs the same
  wrapper with `layer: host`. Debian origin only; never kernel, boot or firmware packages (new refusal **R14**);
  **only on an appliance** (**R12** lifted for the host fast lane: proof is the ROOT-owned install record
  `/var/lib/felhom-install/state.json` `mode: appliance` — the agent-writable `agent.json` is not trusted for this).
  A failed or unhealthy guest step skips it. **Host health rule:** the agent, pveproxy, pvedaemon, pvestatd and
  pve-cluster are active; the customer guest runs; the guest health rule passes; the tunnel is `running` (an
  `unknown` tunnel does not fail the rule; `not_running` does). Never reboots: "reboot needed" is reported when PID 1
  or `lxc-start` runs a replaced library. No automatic undo — the by-hand runbook is `os-updates-host-undo.md`.
- **R-845 — the leg is fast.** The wrapper asked about each package in its own `pct exec` (about 0.9 s each). It now
  makes one call per layer for the version checks (`apt-cache madison` for all names at once, `dpkg --compare-versions`
  on the host), scans for restart-needed only after an install, repairs only when `dpkg --audit` reports something,
  and reports its own `pass_seconds`.
- **Wrapper tests:** `configs/test_felhom_os_apply.py` 46 tests (host layer, R12, R14, lxc-start, the speed rules).
  Red-proofs: `felhom.eu/documentation/audits/os-host-lane-2026-10-04/partA/`, `partB/`, `partC/agent-golden-redproof.txt`.
- **Contract:** the desired-state golden gains `host_release` (byte-identical with the hub's); `TestOSUpdateGolden_Decodes`
  checks it.

## v0.140.0 — OS updates, guest fast lane (`11-os-updates.md` §8 step 2; `09` §3 decisions 76, 79, 80)

> **RELEASED 2026-10-04** by `scripts/release-agent.sh` — tag `v0.140.0` (`9cac346`), sha256
> `ae2d60b794869c51d6b063c8e31e2da98ecdbd36c6b75febd64e2fb4266c1250`, verified by download. Not vouched at release time.

**MinAgent impact:** none required by any controller. **Needs hub v0.130.0** (`os-report`, the `os_update` block); an
older hub serves no block and the leg then reports and installs nothing (ring 1, no release).

- **`configs/felhom-os-apply`** — the root wrapper (Python 3, stdlib). One sudoers entry, `FELHOM_OSAPPLY`:
  `felhom-os-apply --plan /var/lib/felhom-agent/os/plan-*.json`. Modes `inventory` / `apply` / `health`. Refuses (exit
  2, nothing changed) on R1–R13: plan path/owner/JSON, a non-Debian origin, the slow lane, any removal, any downgrade,
  a new or unlisted package, a version not downloadable even from the snapshot, low space, a lock (apt or a guest lock
  such as a backup), a vmid that is not the box's own customer guest (it must bind `/mnt/felhom-drives`), malformed
  names/versions, the host layer, dpkg still broken after the repair. Repairs first (`dpkg --configure -a`,
  `apt-get -f install`). A version Debian already replaced comes from `snapshot.debian.org` at the approval time
  (decision 79). Reports the full installed set with origins, pending, restart-needed (outside containers), health.
  `configs/test_felhom_os_apply.py`: 35 tests; every refusal red-proved.
- **`internal/osupdate`** — the leg: after a SUCCESSFUL primary whole-guest backup, still holding the heavy-op gate
  (never beside another backup or a restore-test), once per night, 90 s after the backup. Ring 0 installs every pending
  Debian / Debian-Security fix; ring 1 exactly the hub's newest approved release; switched OFF → reports only. The
  health rule: docker answers, the network resolves, the controller is healthy, every container running at the START
  of the leg runs (and is healthy if it was) — a 5-minute wait. **No automatic undo:** a customer guest cannot be
  snapshotted (R-837). A failure is `health_failed` → the hub mails the operator.
- **`--selftest=os-update -vmid N`** — the debug action (trigger `debug`, never throttled, not a night run).
- **The `--selftest` flag also accepts `wgtunnel`** — it was dispatched but refused since S3 (found by the new
  `TestSelftestFlag_AcceptsEveryDispatchedMode`, which also caught `os-update` live).
- Proven live 2026-10-04 on both demo boxes (ring 0: 53 packages each; ring 1: exactly 3 approved versions; a failed
  health check → `health_failed`, operator mailed): `felhom.eu/documentation/audits/os-guest-lane-2026-10-04/`.

## v0.139.0 — a DR restore never lands beside a live original (2026-10-04, R-834)

> **RELEASED 2026-10-04** by `scripts/release-agent.sh` — tag `v0.139.0` (`475bdce`), sha256
> `8534a9be368a6d24d8065db77436e86900443c5c6554c71f6fe91c2bdb9d0b9c`, verified by download. **Not vouched.**

**MinAgent impact:** none required by any controller.

- The DR bring-up (`--selftest=bring-up -mode dr`) keeps the archive's `onboot: 1`, binds the host's REAL drives
  (`mp8 /mnt/felhom-drives`) and STARTS the guest — right on a replaced host, wrong beside a live original (a second
  controller for the same household on the same drives). It now REFUSES, before any restore, when the archive's
  source guest still exists on the host, when any guest binds the drives parent, or when a guest's config cannot be
  read (fail closed). On a replaced host it proceeds and keeps its binds, unchanged.
- The restore-test was MEASURED safe live on demo-hp (onboot 0 and throwaway stand-ins for mp8/mp9 from the first
  config read to teardown); a test now pins its "no host path" half beside the existing onboot test.
- No sudoers change: the restore-test sets onboot 0 through the API create call, and DR refuses rather than degrade,
  so no `-onboot 0` line is needed.
- Tests: `TestRunBringUp_DRRefusesBesideALiveOriginal` (source guest present / drives bind on another guest / an
  unreadable config refuse; a replaced host proceeds and keeps the drives bind), `TestRunBringUp_ProvisionNotBlockedByADrivesBind`,
  `TestArchiveSourceVMID`, `TestRestoreTest_NoHostPathBindBesideTheOriginal`. Red-proofs: the DR check returning ""
  → three refusal cases restore and START; the restore-test's mp8 override set to the host path → fails.

## v0.138.0 — the restore test takes only THIS box's archives (2026-09-30, R-727, `09` §3 decision 51)

> **RELEASED 2026-09-30** by `scripts/release-agent.sh` — tag `v0.138.0` (`e1b8269`), sha256
> `55916026001790a79ebf97d32c032610cfde8e09d02979b9b9d8c2cbc5d88195`, verified by download. **Not vouched** (the golden
> keeps 0.137.0 until the next bake); delivered to the demo boxes by signed `agent_update` jobs.
>
> **VOUCHED 2026-09-30** with golden 0.283.1 (`min_agent` 0.131.0), on the operator's word: the hub logged
> `Artifact manifest set: agent=0.138.0 golden=0.283.1 min_agent="0.131.0"`. Evidence:
> `felhom.eu/documentation/audits/evidence-golden-0283-2026-09-30/`.

**MinAgent impact:** none required by any controller.

- A returning customer's PBS namespace can hold archives of EARLIER boxes: same guest id (9201), same token, written
  with a different key. Measured 2026-09-30: the newest SETTLED archive was an earlier box's, and the test failed
  `wrong key … manifest's key 6b:ca:5f:3f… does not match provided key de:51:7a:18…` every evaluation. The archive
  carries no host id; it carries its key fingerprint (PVE content `encrypted`), and the storage carries its own
  (`GET /storage` → `encryption-key`). `PickSettledRestoreCandidateOn` now skips — and logs by name, once — an archive
  whose fingerprint is not the storage's own; an unencrypted storage is not filtered; a failed storage read is an
  error (tier UNKNOWN), never "nothing to prove".
- Tests: `TestR727_TheRestoreTestTakesOnlyThisBoxsArchives` (the 2026-09-30 shape: nothing picked while this box's
  archive settles, then exactly it), `TestR727_UnencryptedStorageIsNotFiltered`, `TestR727_KeyLookupFailureIsUnknown`.
  Red-proof RP39: the skip removed → the earlier box's `2026-09-16T21:59:54Z` is picked.

## v0.137.0 — a guest outside the agent's ACL is not a known guest (2026-09-27, R-689, v0.136.0 regression)

> **RELEASED 2026-09-27** by `scripts/release-agent.sh` — tag `v0.137.0` (`3ef095f`), sha256 `766c9166916a1bd3674b0dc69081f8a7619e770f1402d8ad7705b395937e7627`, verified by download. **NOT vouched** (the operator's act).
>
> **VOUCHED 2026-09-28** with golden 0.276.0 (`min_agent` 0.131.0), on the operator's word of 2026-09-27: the hub logged
> `Artifact manifest set: agent=0.137.0 golden=0.276.0 min_agent="0.131.0"`, and a Day-0 test install fetched this binary
> through the manifest and sha-verified it. Evidence: `felhom.eu/documentation/audits/evidence-golden-0276-2026-09-28/`.

**MinAgent impact:** none required by any controller.

- v0.136.0 asked `GuestConfig` whether an archive's guest exists and treated anything but "does not exist" as a lookup
  failure. PVE answers **403 "permission denied at /vms/<id>"** for a vmid outside the token's pool — so on demo-hp the
  deleted guest 9100's archive made the local tier UNKNOWN every evaluation. A guest the agent cannot read is not one it
  manages; its archive is skipped. `TestR689_AGuestOutsideTheAgentsACLIsNotAKnownGuest`, red-proofed. Verified read-only
  on demo-hp with the pre-release binary before the release.

## v0.136.0 — … of a guest that still EXISTS (2026-09-27, R-689 second half)

> **RELEASED 2026-09-27** by `scripts/release-agent.sh` — tag `v0.136.0` (`16dbc83`), sha256 `2eb0b5ebe253defd68b322312bbac12418c051b0a7a0d2d1831310d97fa6d755`, verified by download. **NOT vouched** (the operator's act).

**MinAgent impact:** none required by any controller.

- **R-689, second half.** Read on demo-hp right after v0.135.0 with the read-only `-selftest=restore-test-due`: the
  golden was skipped, and the pick fell to `vzdump-lxc-9100-2026_08_21…` — a leftover of a guest deleted in August. A
  candidate's guest must now exist on this node (`GuestConfig`): "does not exist" skips the archive (INFO once per
  volid); any other lookup failure is returned, so the tier reads UNKNOWN, never "nothing to prove". Tests
  `TestR689_AnArchiveOfADeletedGuestIsNeverPicked` (red-proofed: without the check it picks the 9100 leftover),
  `TestR689_AGuestLookupFailureIsUnknownNotEmpty`. A second agent release in one session — the first half was found
  incomplete on the box.

## v0.135.0 — the restore test proves only backups OF A GUEST (2026-09-27, R-689)

> **RELEASED 2026-09-27** by `scripts/release-agent.sh` — tag `v0.135.0` (`d4be12c`), sha256 `ad4e75f16d338552f4588d3fe64c51cbf9651220b9d223b5386b85b6c37fd4c3`, verified by download. **NOT vouched** (the operator's act).

**MinAgent impact:** none required by any controller.

- **R-689** (`backup/runner.go` `guestBackupArchive`). demo-hp keeps its golden template in `local:backup/` — content
  "backup", 654 MB, plausibly complete, the newest settled entry — and the scheduled restore test picked it every 6 h and
  failed `extractconfig` with a 403, while the guest's real archive went untested. A restore-test candidate is now a
  `vzdump-{lxc,qemu}-<vmid>-…` file or a PBS `backup/{ct,vm}/<vmid>/…` snapshot whose vmid the storage reports; anything
  else is skipped with one INFO line per volid (not the "INCOMPLETE archive" WARN). Tests
  `TestR689_TheRestoreTestNeverPicksTheGolden` (red-proof: without the check it picks the golden) and
  `TestR689_GuestBackupArchiveShapes`; two older picker tests' fixtures moved to real archive names.
  Evidence: `felhom.eu/documentation/audits/version-travel-2026-09-26/D1/`.

## v0.134.0 — a whole-box backup that cannot fit is skipped with a reason, before anything starts (2026-09-25 night, R-685)

> **RELEASED 2026-09-24 night** by `scripts/release-agent.sh` — tag `v0.134.0` (`0722b2c`), sha256 `7593bebe03234c7d22f3ade384e7ed7787dc659aa8c8594b3ee19af2ce81c72d`, verified by download. **NOT vouched** (Day-0 stays on the previous version).

**MinAgent impact:** none required by any controller.

- **R-685 — the backup space preflight** (`backup/runner.go` `spaceFits`). Before a vzdump to a LOCAL (non-PBS)
  target, the newest archive of that guest on that target × 1.25 + 1 GiB must be free — PVE prunes old archives
  only AFTER a successful backup, so the kept ones still stand during the run. A shortfall is a skip: nothing
  starts, the record's Error reads `skipped: not enough space: <target> has X GiB free; the last archive of guest N
  was Y GiB, so a new one needs about Z GiB …` (stable prefix `BackupSkipNoSpacePrefix`), and it reaches the
  controller's tier view and, through the quiesce loop's tier notifier, the operator's `whole_guest_backup_failed`.
  It FAILS OPEN on what is not known (PBS target, first backup, unreadable usage).
- **Free space is read from `GET /nodes/<node>/storage`** (`NodeStorage`), never `GET /storage` — the latter is the
  cluster DEFINITIONS and carries no usage. **Found live, before release:** the first build read `/storage`,
  failed open, and a real vzdump of demo-hp 9201 started from the live test; it was aborted after 5 min 16 s, no
  archive left (`felhom.eu/documentation/audits/night-2026-09-25/F/`). The test fake's `ListStorage` now strips
  usage like production. Red-proofs: two (`…/F/redproof-r685-*.txt`).
- Live proof (demo-hp, safe builds with a hard stop before vzdump): ×10 margin → refused, "local has 14.9 GiB
  free … needs about 77.2 GiB"; release margin → "space preflight passed" need 11.3 GB, avail 16.0 GB.

## v0.133.0 — a restore-test can never fill a box's disk; leftovers retried on a timer (2026-09-24, R-672, R-673)

> **RELEASED 2026-09-24** by `scripts/release-agent.sh` — tag `v0.133.0` (`9bdb4da`), sha256 `3aa303452b8c6be58573d00af01a0ab4a0d97e4f885ffecd0144a18ac24e69b6`, verified by an independent anonymous download. **NOT delivered** (needs an operator-signed `agent_update` job per box, R-530) and **NOT vouched**.

**MinAgent impact:** none required by any controller. Hub **v0.124.0** makes a thin pool CRITICAL at 90 %
(data or metadata) and keys the storage-fill alarm per pool per 6 h; an older hub still raises its generic
90/95 % storage-fill events from the same report.

- **R-672 — the space preflight** (`reconcile/restoretest_space.go`, provider `internal/restorespace`). Before
  anything is journaled or created, a restore-test needs free data ≥ restored × 1.2 + 5 GiB on its target
  (`backup.restore_test_space_factor` / `backup.restore_test_space_reserve_gib`) and room in a thin pool's
  metadata. `restored` is the UNCOMPRESSED size — the vzdump log's "Total bytes written" (a file-backed
  archive) or the PBS snapshot size; the archive FILE is never used (9201: 6.9 GB file, 22.6 GB restore). The
  target moves OFF the tested guest's own pool when another storage is eligible (active, `rootdir`, and the
  agent holds Datastore.AllocateSpace there) and fits. Anything unknown refuses. A refusal is reported to the
  hub as the test's result (`pass=false`, `skipped=true`, "skipped: not enough space on …") — never a pass,
  never dropped. The archive config is read once (it was read twice). Measured case: demo-hp 2026-09-24 —
  the restore-test that filled `local-lvm` would now be refused (needs 32.1 GB, 23.2 GB free).
- **R-672 — a failed scratch teardown is retried every 10 minutes** (`Engine.RetryScratchTeardown`, the
  daemon's janitor), not only by Recover at agent start; never a vmid a running test owns; after 3 failed
  tries the operator is told through a failed restore-test record naming the scratch guest.
- **R-672 — a thin pool crossing 90 % requests an immediate host report** (`Observer.SetThinHighTrigger`, the
  storage watchdog's read path, every few seconds; re-armed below 85 %), so the hub's alarm sees it in
  seconds, not at the next 15-minute report.
- **R-673 — the stale-lock sweep runs on the same 10-minute timer** (it ran only at start: a stale
  `snapshot-delete` lock blocked 9201's whole-box backups for five hours), holding the one-heavy-operation
  gate so no agent backup can start between its "no vzdump running" check and its unlock.
- Red-proofs: six, each seen failing (REPORT).

## v0.132.0 — a controller that dies slowly is reported, not just restarted (2026-09-17, R-539)

> **RELEASED 2026-09-17** by `scripts/release-agent.sh` — tag `v0.132.0`, sha256 `4afe815749a41b327ebe4a98a4557ad2acffb1fa71740a3a8b8003473b835321`. **Vouched 2026-09-17** for Day-0 installs on the operator's word, together with golden **0.246.0** (the hub's R-120 gate required the newer golden first). Delivered to demo-hp and the N100 by an operator-signed `agent_update` job each (operator ruling 3 of 2026-09-16), not by a floor.

**MinAgent impact:** none required by any controller. Hub **v0.117.0** turns the new fields into
`controller_slow_crashloop`; an older hub ignores them.

- **R-539 (operator ruling 3 of 2026-09-16) — the slow crash-loop counter.** The 3-restarts-in-15-minutes
  brake cannot see a controller that dies every 20 minutes (measured 2026-09-16, R-531: four restarts,
  none accumulating, only an `info` event that mails nobody). Beside it, unchanged, a second counter:
  restarts the supervisor performed in the last **24 hours**; at the **fifth**, the heartbeat's
  `controller_supervisor` stanza sets `slow_crashloop_since` (and `slow_crashloop: true`,
  `restarts_24h`). The hub mails on that timestamp MOVING; it moves **at most once per 24 hours**. It
  does **not** stop restarting — the fast brake remains the only brake.
- **Persisted per guest** at `/var/lib/felhom-agent/guests/<vmid>/controller-restarts-24h.json`
  (tmp + rename, 0600), so an agent restart or a host reboot does not reset it. The fast record stays
  in memory; the reason it does (a persisted "give up" could outlive the fix) does not apply to a
  counter that only warns. Unreadable or corrupt → WARN and a clean start, never a blocked supervisor.
- **Deliberate kills count.** The supervisor cannot tell an operator's `docker kill` from a crash
  (measured 2026-09-15); a controller killed five times a day is worth a line either way.
- The supervisor's startup line now prints `slow_crashloop_max=5 slow_crashloop_window=24h0m0s`.

**Red-proofs, each seen failing:** five restarts 20 minutes apart raise it
(`TestControllerSupervisor_SlowCrashloop` — fails with no counter; fails again with the once-per-24-hours
guard removed, "the operator would be mailed per restart"); the counter survives an agent restart
(`…SlowCounterSurvivesAgentRestart` — fails with the save removed, `Restarts24h:1`); restarts seven hours
apart never raise it (the negative control). Wire shape extended with `restarts_24h` and `slow_crashloop`.

## v0.131.0 — a dead controller comes back by itself; the backup status speaks per tier (2026-09-15, R-523 / R-517 / R-518)

> **RELEASED 2026-09-15** by `scripts/release-agent.sh` — tag `v0.131.0`, sha256 `1118b552f7e775fbde9544c7764ede7e6046e0a7db16ae8494d07a18e3c2ac9c`. Delivered to boxes by the controller v0.243.0 floor (declared MinAgent), not by hand.

- **R-523 (P1) — the in-guest controller supervisor.** BIGNIGHT F9: `docker kill felhom-controller`
  left the household's dashboard on 502 for 33 minutes, because nothing watched the container.
  Measured first (2026-09-15, Docker 29.8.0, `evidence-p1fixes-2026-09-15/A1`): after `docker kill`,
  BOTH `--restart unless-stopped` and `--restart always` leave the container `exited (137)` after
  60 s — a policy change alone is not a fix. New `internal/localapi/controllersupervisor.go`: every
  30 s, for each felhom-pool guest the agent provisioned (`<guests>/<vmid>/bootstrap` exists) that is
  running, it reads `docker inspect -f {{.State.Status}} felhom-controller`; on the SECOND consecutive
  not-running (or absent) observation it runs `systemctl restart felhom-controller-bootstrap.service`
  inside the guest — the swap's own restart, over the same GuestExecutor and the same two sudoers
  grants. No new privilege. Guards, each pinned by a test: not during a controller swap (the swap's
  in-flight flag); not when parked (`touch /var/lib/felhom-agent/guests/<vmid>/controller-parked` on
  the HOST); not on a stopped, locked or vzdump-busy guest; not on an unknown docker answer; not on a
  guest the agent did not provision; and **no thrash** — 3 restarts in 15 minutes stop the restarts
  for 30 minutes. The record rides the host report as `controller_supervisor` (additive,
  `omitempty`); hub v0.114.0 mints `controller_restarted_by_agent` (info) and `controller_crashloop`
  (error), both operator-only. Red-proofs: without the restart call, the kill test fails at
  "restarts=0"; without the backoff block, the crash-loop test fails at "restarted 10 times".
- **Golden script** (`configs/build-golden.sh`): the controller runs `--restart always` (covers a
  Docker daemon restart after a manual stop — nothing more). **No golden baked here** (R-468); existing
  boxes keep `unless-stopped` until their next golden and are covered by the supervisor.
- **R-517 (P1) — `GET /backup/status` speaks per tier.** The untargeted response gains `tiers[]`:
  per tier the newest SUCCESSFUL backup (`last_success`, from the record, or from the tier's storage
  after an agent restart — `last_success_source: storage`), the last attempt kept apart
  (`last_attempt {started_at, success, error}`), and whether the tier's storage exists (`storage:
  present|absent|unknown`). `GET /backup/tiers` gains the same `storage` field (R-518's cheap half: the
  controller skips an absent tier). `unknown` is never `absent` — a storage view that cannot be read
  must not skip a backup. Additive; the untargeted `.backup` keeps its meaning (pinned). Red-proof:
  filling `last_success` from the newest ATTEMPT fails at "pbs tier reports a failed attempt as its
  last success".

## the decoy sweep — can this gate be fooled by a label? (2026-09-01, R-421) — NOT A RELEASE

**No product code, no version bump, no image, no golden.** A scripts change is not a release.

Four times in one week a gate turned out to match a NAME instead of the thing it named — R-410 (a
`mkdir` turned the release gate green), R-400 (seven debug controls answering nothing), R-378 (a
status word inside a sentence), R-419 (a phrase inside prose, including prose saying the marker was
absent). **All four found by accident.** The gates enforce everything else here and were the one part
nothing had checked.

**All 29 gate scripts read and decoyed. 16 were fooled.** 10 fixed here, 4 left with rows
(R-422..R-425), 6 could not be given a plausible decoy and are named (R-426 group d).

**The largest single cause was mundane:** eight gates set their SCOPE with `os.listdir` (one level).
Green and correct today; blind the moment anyone adds `templates/partials/`. `mojibake` and
`docker-v` already used `os.walk`, caught the identical planted file, and are the control that
proves the cause was the listing rather than the decoy.

Full survey table, and the five decoys withdrawn as illegitimate (mine, named):
`documentation/audits/AUDIT-gate-decoys-2026-09-01.md`.

**In this repo:** no gate changed, and that is the result. `release-complete` was decoyed and is
SOUND — the sweep's attempt (a non-version heading on top of `CHANGELOG.md`) was WITHDRAWN as
illegitimate, because `HEAD_RE.search` scans the whole file and still names `v0.130.0`. The three
shared gates are covered from `felhom.eu`; the remaining two are named in the decoy-coverage
exemption list (R-426) as UNTESTED, not as sound.

## v0.130.0 — the agent was the one leaking connections onto the off-site box (2026-08-20, R-344)

> **RELEASED 2026-08-20**, on the operator's word, after the fix was proved on both boxes.
> `sha256 a56a92a7bd68f5b46736eaec4806c3d26c16ccb35118c4ac0e3d8094eaefabc3`, 14,141,158 bytes,
> tag `v0.130.0` at `7569f34`. Reproducible: a rebuild with `-trimpath -buildvcs=false` matches the
> published artifact byte for byte (R-186's property, checked rather than assumed).
>
> The heading read `## UNRELEASED — v0.130.0 candidate` until this point, deliberately: while the fix
> was hand-installed on `demo-hp` only, publishing would have pushed it onto `demo-felhom` through
> self-update and destroyed the control the proof rested on. **`release-complete` convicted on the
> release heading and was right to** — the answer was to stop claiming a release, not to bypass the
> gate. See R-347.
>
> **The fleet runs these exact bytes.** Both demo boxes were first given a hand build made during the
> proof — same source, same version string, **different bytes** (`256e0829…`), because
> `release-agent.sh` builds with `-trimpath -buildvcs=false` and a hand build does not. Nothing would
> have corrected that: the boxes already reported `0.130.0`, so self-update saw the vouched version as
> installed and would have done nothing, forever. Both were reinstalled from the **downloaded package**
> and now report `a56a92a7…`. Filed as **R-349**, because every prove-then-publish train hits it.

**What was measured, before anything was changed.** Between 2026-08-18 09:51:22Z and 2026-08-20
08:02:13Z, ep0's PBS proxy accumulated **388 established connections** — 194 from each demo box, on a
proxy whose descriptor ceiling is 65536 and whose runway at that rate was ~323 days. The connections
were held open on **both** sides: ep0 showed 388 while the two boxes showed 194 + 194, at two separate
instants, with the same source ports on each side, and **not one closed in a 31-minute window**.
`ss -tnp` on the boxes named the holder: **`felhom-agent`**, 194 of 194 on each, one PID.

**`pvestatd` and `proxmox-backup-client` made 162,404 requests in that window and leaked zero.** They
are 99.5% of the traffic to that endpoint and 0% of the leak. The agent made 811 requests — of which
387 were `GET .../snapshots` — and leaked 388 sockets. One per call, within one.

**The defect, and it is two things compounding.**

- `internal/pbs/client.go` built its transport as a composite literal:
  `&http.Transport{TLSClientConfig: tlsCfg}`. That takes **`IdleConnTimeout` zero, which does not mean
  "use a sane default" — it means retain idle keep-alive connections FOREVER.**
  `http.DefaultTransport` sets 90s; hand-rolling the transport (which every client here must do,
  because they all pin TLS) silently discards it.
- `pbsTargetsFromPVE` (`cmd/felhom-agent/main.go`) builds **a fresh `pbs.Client` every cycle**, as its
  own doc comment says, and drops the previous one. An abandoned `http.Transport` does **not** close
  its connections — it becomes unreachable while its `persistConn` read-loop goroutine keeps the
  socket alive. So each cycle stranded exactly one connection that nothing could ever close.

The cadences reconcile without fitting: a 900s live-snapshot collect (184.7 cycles in the window) plus
a 6-hour verify loop (7.7) predicts 192.4 per box against **194 observed**.

**The fix is one field, restored to the standard library's own value.** New leaf package
`internal/httpx` owns `DefaultIdleConnTimeout = 90 * time.Second` — 90s because that is what
`http.DefaultTransport` uses, so there is nothing invented here to justify or tune — and
`NewTransport(tlsCfg, idleConnTimeout)`, which returns a **fresh** transport (never shared: each caller
pins a different endpoint) and treats a zero or negative timeout as **use the default, never "no
timeout"**. `pbs.Config` gains an `IdleConnTimeout` field that production leaves unset; only tests set
it, to avoid a 90-second wait.

**`internal/hub/client.go` and `internal/proxmox/client.go` carried the identical missing default and
were corrected in the same pass — but neither contributed to the ep0 leak, and this entry must not be
read as three leaks having been found.** Both are built **once per process**, so they held one idle
connection for the life of the daemon rather than accumulating, and neither talks to ep0:8007.

**Tests, and what they deliberately do not assert.** `internal/pbs/client_leak_test.go` counts
connections **server-side** and models what `pbsTargetsFromPVE` actually does — build a client, use it
once, drop it on the floor — then asserts the connections go away. It does not assert `err == nil` and
it does not assert that some field holds some value; both were true of the leaking code.

- **Red-proof 1 (the fix):** removing `IdleConnTimeout` from `NewTransport` fails the test with
  *"after 5s the server still holds 5 open connection(s), want 0 (5 dialled in total)"* — the count is
  in the message, so the failure cannot be mistaken for a timeout with another cause. Reverted.
- **Red-proof 2 (the fix that would be worse than the bug):** setting `DisableKeepAlives: true` also
  makes the leak vanish — by dialling fresh for every request, which on a box polling ~40,000 times a
  day is strictly worse than what we started with. **The leak test PASSES under that mutation**;
  `TestPBSClient_KeepAliveStillReuses` is what catches it, failing with *"3 sequential requests over 3
  connection(s), want 1"*. Reverted.

**What this release does NOT do.** It does not reduce the poll rate (**R-336 stays open, but re-scoped
— it was never the cause of this leak**), it does not refactor `pbsTargetsFromPVE` to cache or reuse
clients (a one-line default restores the standard behaviour; a lifecycle refactor adds
cache-invalidation questions for no measurable gain), and it adds no `CloseIdleConnections` call.

**One sentence in this entry was written before the deploy and was WRONG, and it is corrected here
rather than quietly edited.** It read: *"does not clear the 388 descriptors already stuck on ep0 —
those persist until that proxy restarts."* **Measured: they clear the moment the AGENT restarts.**
Replacing the binary on `demo-hp` released exactly its 199 descriptors within one second
(415 → 216 fd), and replacing it on `demo-felhom` released the remaining 203 (**220 → 17 fd in under
two seconds**). **17 is precisely ep0's `t0` baseline** of 2026-08-18 09:51:22Z. ep0 was read-only
throughout and its proxy PID never changed. The accumulated leak was never ep0's to hold on to — it
was held on both sides, and closing either side ends it.

## v0.129.0 — a correct code for an earlier package stops being called wrong (2026-08-12, R-311)

**The measurement this fixes.** On 2026-08-12 a recovery code that provably opens a RETAINED package
— unsealed by hand, and it restored planted files byte-identical from a store the box itself could no
longer open — was answered by this agent with *"the recovery code did not open the sealed bundle"*.
The code was correct. Nothing had ever tried the retained packages, so the engine could not tell a
correct-but-earlier code from a mistype, and the screen said so out loud: a true sentence about our
own incuriosity, read by the customer as a statement about their code.

**`OffsiteKeyRecoverer` gains an optional `FetchRetained`.** It is consulted ONLY after the current
package has refused, so the ordinary recovery pays nothing for it and cannot fail because of it. When
one of the retained packages opens, the recoverer returns `ErrCodeOpensRetained` wrapped in a
`RetainedOpenedError` carrying the supersession date — no material, no code, no password.

**The local API answers 422** ("your code is correct, it belongs to an EARLIER sealed package") — a
FIFTH status added to the R-224 switch, not a restructuring of it. 422 rather than 400 because the
request was well-formed AND the credential valid; a 400 would put it in the same bucket as a mistype,
which is the defect.

**Fail-safe in every direction.** A nil fetcher, a hub too old to have the route (404 is a clean
"none"), a transport failure, a malformed package: each leaves the original refusal standing,
unchanged. The worst outcome of this feature breaking is the behaviour we had before it existed.
Attempts are bounded (`MaxRetainedTried`, default 6) because each unwrap is ~1 s of scrypt by design
and an unbounded loop would turn one wrong code into a minutes-long hang.

**New hub client call:** `FetchRetainedIdentityEscrow` → `GET /api/v1/hosts/<id>/escrow/retained`
(hub >= v0.103.0), self-scoped by the same per-host key.

Seven tests with REAL age crypto, because the two situations are indistinguishable AT THE UNWRAP and a
faked unwrap would prove nothing about what was broken. Red-proof, asserted applied: removing the
retained lookup returns the fail-closed wrong-code error — **the lie comes back, in those words.**

---

### Gates only — 2026-08-09 (no release, no version bump, no binary published)

**Two guards, both owed since the 2026-08-09 install outage (R-273/R-287). Nothing that runs on a
customer's box changed; `scripts/` only, and the agent stays v0.128.0.**

- **`scripts/retention-policy.json` — THE retention number, in one file.** The registry stopped
  serving `felhom-agent` 0.120.0 and older while `check-published-versions.py` demanded that every
  tag still be downloadable. Both rules are sensible; together they are impossible, and CI went red
  at a commit whose own run had been green the day before. The check now **reads the number from
  this file** and bounds its assertion to the newest N generic versions.
  **What CI no longer covers, said plainly rather than left to be discovered:** a released version
  older than the retention window is **no longer asserted downloadable**. Its git tag and its configs
  are still asserted — only the binary's presence is dropped. The check **prints exactly which
  versions it stopped covering** on every run, so the narrowing cannot become permanent by accident.
  **The number is an OBSERVED state, not a located ruling** — see the file's own header and R-287.
  A missing or unreadable policy file is **INCONCLUSIVE (exit 2), never silently unbounded.**
- **`scripts/check-release-complete.py` — the tag half, as a machine.** Asserts that the version at
  the head of `CHANGELOG.md` is tagged, that the tag points into this history, and that its package
  is published. `release-agent.sh` already warned about this in as many words and the step was still
  missed on 2026-08-08, which is why this is a gate and not a reminder. Legs 1–2 need no network and
  therefore run in `--fast`, so the pre-push hook catches a missing tag at the earliest moment.
  Registered in `agent_gates.py`; red-proved by pointing the CHANGELOG head at an unreleased
  v0.129.0 — both legs convicted and each named its fix command.

## v0.128.0 — the escrow seed is asserted every tick, not remembered once (2026-08-08, R-221)

**A rebuilt box could not run the escrow ceremony at all, and there was no way forward from inside
the product.** The preflight refuses on `escrow.pbs_storage_id`; the pbsdr bridge writes that key;
and it wrote it from exactly one place — `finishConverged`, reached only on the paths that actually
converge.

**The two things live in different places and die at different times.** The convergence marker is
host-side (`<agent-state>/pbsdr/marker.json`). The key it seeds is in `agent.json`, which
`step_agent_config` renders from `base = {}` unless an explicit `--preserve-from` is passed
(`felhom.eu/scripts/felhom-host-install.sh:2396`, the render at `:2449`, the `O_TRUNC` write at
`:2579`; the flag at `:1246`, defaulting empty at `:256`) — **and the render never writes an
`escrow` section at all.** So a rebuild keeps the marker and takes the key: same descriptor, same
hash, early return, and the seed never runs again into a config that no longer has it.

**Fixed by asserting rather than remembering.** `Apply` now re-asserts the seed *before* the
idempotent early return. `seedEscrowStorageID` is unchanged and still never clobbers a different
existing value — an operator's own choice outranks the descriptor's, with a warning naming both.

**The early return is KEPT.** It exists so a converged box does not re-run Proxmox operations every
60 s, and `TestSeedReasserted_OnConvergedTick_WithZeroProxmoxCalls` asserts **zero** recorded runner
calls on that tick — so a "fix" that simply deleted the return would fail. Cost of the re-assert: one
small file read plus a JSON parse per tick, no exec, no network, and an early return once the value
matches.

**A seed failure can never un-converge the box:** Warn plus a message on the published status,
exactly as `finishConverged` does it — no marker write, no state change. Pinned by
`TestSeedReassertFailure_DoesNotUnconverge`.

Tests drive the real `Apply` with a real temp-dir `agent.json` and a call-recording runner; calling
`seedEscrowStorageID` directly cannot see the early return, which IS the defect. The production
wiring (`pbsdr.NewManager(..., cfg.SourcePath, ...)`) is asserted by walking `main.go`'s **AST**, not
by `strings.Contains`, which a commented-out call also satisfies.

Red-proofs: removing the new call makes Scenario A fail on today's tree; removing the early return
makes the zero-Proxmox-calls assertion fail.

## (no version bump) — a comment that claimed the hub reads a field it has no field for (2026-08-08, R-260)

Comment-only; no behaviour, no wire change, nothing to rebuild.

`HostReport.SelfUpdatePending` / `SelfUpdatePendingVersion` carried the sentence *"The hub reads an
absent field as pending=false, the correct default."* **The hub has no field for either**, so it reads
nothing — present or absent — and `encoding/json` discards them on arrival. The sentence described an
intent rather than the code and read as settled for long enough that a class sweep had to find it.

The emission is correct and stays: the agent reports the truth, and the fault is entirely in the
receiving. The missing consumer is tracked as **R-264** (OPEN), and
`felhom.eu/scripts/wire_contract_gate.py` now refuses any NEW field of this shape while recording the
existing ones as explicit, reasoned allowlist entries rather than silence.

## v0.127.0 — a mount Felhom itself made is not "something else" (2026-08-06, R-220)

**After a rebuild the customer's own drives could not be re-attached, and the refusal named an action
they could not perform.** `GET /api/disks/candidates` returned `initialize: [], attach: []` while both
drives sat there, and the deploy refused with *"choose an attached drive from the list"* — a list that
was empty. Measured live **three times**: CAMPAIGN-11 Phase 1, and twice on the R-201 re-walk.

**The mechanism.** Enrolment mounts a drive **twice** — at the managed `/mnt/felhom-drives/<name>` and
at the raw `/mnt/<name>` it creates on the host. **The host survives a guest rebuild; the controller's
registry does not.** So `classifyClaim` saw a mount outside the managed prefix and correctly concluded
"claimed by something else" — about Felhom's own mount.

**The fix is CORROBORATED, not a widened prefix.** A mountpoint outside `/mnt/felhom-drives` is
forgiven **only when the same device is ALSO mounted under the managed path** — a pairing that only
Felhom's own enrolment produces. A disk another system is using, at `/srv/data` or `/media/x` or even
`/mnt/someone-elses-disk`, has no such counterpart and **is still refused**. That fence has its own
test, and its red-proof shows an over-wide fix offering `/mnt/someone-elses-disk` for formatting.

**Read from `/proc/mounts`, deliberately.** The lsblk invocation is pinned **verbatim** in
`configs/felhom-agent.sudoers` (`lsblk -J -o NAME,FSTYPE,PTTYPE,MOUNTPOINT /dev/*`), so switching it to
the plural `MOUNTPOINTS` would have meant shipping a sudoers change with the binary — a far larger
blast radius than this finding warrants. `/proc/mounts` is world-readable: **no sudo, no new allowlisted
command, no config change.**

**Fail-safe:** an unreadable mount table corroborates **nothing**, so the device classifies exactly as
it did before this change — refused. "We could not corroborate" must never read as "it is ours".

Tests: `claim_r220_test.go` — the own-drive case, the foreign-mount fence over four paths, and the
corroboration itself (both mounts required; a lone raw mount vouches for nothing; another device's
managed mount does not vouch for this one; an unreadable table corroborates nothing).
**Red-proofs:** removing the exemption refuses the customer's own drive again
(*"device is mounted at /mnt/adatok (sdb)"*); over-widening it to any `/mnt/*` path breaks the fence.

## docs — CLAUDE.md becomes a core plus path-scoped rules (2026-08-06, R-229 leg (b)) — no version bump

**Documentation only. No Go changed, nothing built, nothing deployed.** `go build`/`vet`/`test` green
and unchanged.

**175 -> 99 effective lines** (207 -> 103 raw, 14,093 -> 6,267 bytes). The release/publish-train
section was the largest block and the `felhom-build-deploy` skill already carries the procedure, so
the core points at it instead of restating a table that drifts from the script. The package layout
went the same way as the controller's: `REUSE.md` and the tree are its home, and the per-package
annotations that were doing real work moved into the rule file for the area they describe rather than
being deleted.

**New:** `.claude/rules/{proxmox,localapi,backup,storage}.md`, all `paths:`-scoped, all <=46 effective
lines, joining the existing `health-checks.md`.

**Kept in the core deliberately** — it is the only part re-injected after `/compact`: the root-CLI
fence and its three named exceptions (breaching it is how this component stops being auditable), the
destructive-op gate, the prove-ownership rule from audit A1, the gate entry point, the F9
live-validation fence, trunk-based with its revert-and-report escape hatch, and the end-of-session
checklist.

**Glob overlap, stated rather than silently resolved:** `health-checks.md` matches
`internal/{localapi,guesthook}/**` and `internal/storage/**`, which `localapi.md` and `storage.md`
also match. Both rules load in those directories and neither supersedes the other; each new file says
so in its own text so a reader who sees two rules fire is not left guessing which wins.

## docs — the "CI is still owed" claim was stale; corrected (2026-08-06, R-229 part 2) — no version bump

**One sentence, no code.** This file asserted that continuous integration was still owed
(`felhom.eu` `OPEN-ITEMS.md` R-168). **R-168 was CLOSED on 2026-08-02** — a Gitea Actions runner
re-runs each repo's gate entry point on every push and emails the operator on failure. Found while
confirming this session's own push by run ID, which is the check that caught it.

The same stale sentence was in four instruction files across all four repos and is corrected in all
four. In `felhom-agent/CLAUDE.md` it **contradicted the same file's release section**, which already
said R-168 mails the failure — a contradiction inside one instruction file, which is the exact class
the R-229 work exists to find.

## docs — expired and contradictory blocks removed from CLAUDE.md (2026-08-06, R-229) — no version bump

**Documentation and gate registration only. No Go changed, nothing built, nothing deployed.**

Surgical corrections; the file was deliberately **not** restructured (that is deferred, R-229).

- **Deleted the expired TEMPORARY block.** It read *"felhom-pve is at a remote site (until
  ~2026-08-02) … Delete this block on return"* and was still being read as current fact on
  **2026-08-06**, four days past its own deadline — while `felhom-controller/CLAUDE.md` asserted the
  opposite. The location-independence fact worth keeping (`localapi` binds `169.254.253.1:8443` on
  `vmbr9` since the R-50 island migration) moved to an HTML comment.
- **Every component version literal is gone** from effective text, including
  `felhom-agent --version → 0.115.0` and the `go.mod` Go directive. Versions change several times a
  day; ask the hub's `/hosts` + `/configs` or the box.
- The drill-VM claim and the host addresses now point at `documentation/operations/nodes.md`, which
  already stated both correctly. **This file's drill-VM claim was the correct one** — confirmed by
  `qm list` on demo-hp.
- The R-115/R-188/R-186 release **narratives** moved to an HTML comment and to the
  `felhom-build-deploy` skill; the **directives** stayed (never hand-roll the build; the
  build → tag → publish → push order; reproducible `-trimpath -buildvcs=false`).
- The health-check block-I/O rule became `.claude/rules/health-checks.md`, scoped to the five
  packages where health checks are written. It had been duplicated from `felhom.eu/CLAUDE.md` *with a
  note explaining that that file does not load in an agent-only session* — correct reasoning, made
  obsolete by path-scoped rules.

`agent_gates.py` now registers **`instructions`** (shared, `felhom.eu/scripts/`, never copied).

Full accounting: `felhom.eu/documentation/audits/LEDGER-instruction-trim-2026-08-06.md`.

## v0.126.0 — a fetch failure is not a wrong recovery code (2026-08-06, R-224)

**A hub the agent could not reach was reported to the customer as a bad recovery code.** Measured live
on 2026-08-05 (CAMPAIGN-11 F3): with the hub REJECTed at the appliance's firewall and a **correct,
current** recovery code, the customer was told their code did not open their package — **in 0.0556 s**,
against ~1.0 s for a genuine unseal. No unseal was attempted. F4 produced the same message in 0.0299 s
with this agent stopped.

**The discriminator existed here the whole time and this boundary threw it away.** `recover.go` fails
at four distinguishable points; the local-api handler had cases for two of them and a `default` that
answered *"the recovery code did not open the sealed bundle, or the bundle could not be fetched"* —
one sentence for two situations, only one of which is the customer's doing.

**The fix is a value, not a log line.** `escrow.ErrBundleFetch` joins the fetch leg's error, and the
handler routes it to **502** with its own words: *"the sealed recovery bundle could not be fetched from
the hub — the recovery code was NOT used and nothing was written."* 502 rather than 4xx because the
request was not bad; an upstream dependency failed. The `default` now carries **only** the fail-closed
wrong-code case and says so without the "or".

Four situations, four statuses — **502** fetch failed · **400** the bundle was fetched and refused the
code · **404** the hub holds no bundle · **409** the bundle predates the repository-password field.
The controller classifies on the STATUS and must never parse these sentences.

⚠ **A GREEN TEST NAMED THIS DEFECT AND DID NOT PREVENT IT.**
`TestRecoverOffsiteRepoPassword_FetchErrorIsDistinct` has said since v0.125.0 that *"the operator must
not be sent to re-read their recovery code because the hub was unreachable"* — and it passed
throughout, because it asserted this package's error **string** one layer below where the merge
happened, and a string is something no caller can branch on. It now asserts the sentinel, and its
consequence-level twin asserts the STATUS at the boundary the customer's message is derived from.
**Prefer the test that asserts the consequence over the one that asserts the mechanism.**

Tests: `recover_test.go` (fetch classifies as `ErrBundleFetch`; a wrong code does **not**; an absent
blob keeps its own identity) and `localapi/escrow_recover_class_test.go` (each situation's status, and
a standalone assertion that fetch-failure and wrong-code never share one). **Red-proofs:** removing the
`%w` join fails the sentinel test; deleting the handler case makes both answer `400` with the
wrong-code sentence — the exact pre-fix code, and the exact defect CAMPAIGN-11 measured.

## v0.125.0 — the agent opens the sealed bundle and returns one field (2026-08-04, R-199 links 7–8)

**Link 7 had one production caller and it was a `--selftest`.** `UnwrapIdentityBundle` has existed
since slice 10D.1 and the only thing that ever called it was `runSelftestIdentityConsume`, reading the
recovery code from an environment variable by hand. **Link 8 did not exist at all:** that selftest
writes the whole bundle JSON to a file, and its success message named `tunnel_token + pbs_token` —
an enumeration that was accurate when written and became a MISSTATEMENT the moment v0.77.0 sealed the
offsite repository password into the same bundle. Anyone reading that output would conclude the
repository password was not there. It now names what THIS bundle actually carried and what it did not.

**`POST /escrow/recover-offsite-password`** on the pinned local API: the controller supplies the
customer's recovery code, the agent fetches this host's own sealed blob from the hub
(`hub.Client.FetchIdentityEscrow`, hub ≥ v0.94.0, self-scoped by the per-host key), unseals it, and
returns **only the offsite restic repository password** plus its sha256.

**Only that field, on purpose.** The bundle also carries the tunnel token, the PBS token and the WG
private key. The controller is a trust tier down and needs none of them; returning them would widen
the blast radius of a controller compromise for nothing. Narrowing costs nothing now and is not
recoverable later.

**Why the agent and not the controller:** `age` is an agent runtime dependency and is deliberately
absent from the controller image; the blob is a host-scoped object whose only writer is this agent
under the per-host key, so the read is that write's mirror.

**R's handling is the tightest rule in this release.** It arrives in the request body over the pinned
channel, is held in memory for one call, is cleared on the success path AND every failure path, is
never written to disk, never an argument in a process list, never logged at any level including
inside an error, and is never echoed. `UnwrapIdentity` already stages only the blob and the recovered
plaintext in a temp dir it removes; a test redirects TMPDIR and asserts **the tree is empty
afterwards** — emptiness rather than a content scan, because a content scan is defeated by a later
call overwriting the leaked file, which is exactly how the first version of that test passed its own
red-proof while R sat on disk.

Three outcomes are distinct rather than one generic failure: no blob (404 — no ceremony has run), a
bundle that opens but predates the field (409 — a pre-fork-4 blob, which cannot be retro-fitted), and
a code that does not open it (400 — fail-closed at the KDF, nothing written). Sending an operator to
re-check a correctly typed recovery code because the hub was unreachable is the mistake this avoids.

**The wiring is asserted by an AST walk**, not a `strings.Contains`: `main` → `runDaemon` →
`buildLocalAPIServer`, where an `escrow.OffsiteKeyRecoverer` is constructed and passed as
`localapi.Options.EscrowRecovery`, and its fetcher calls the DAEMON's own hub client (the self-scoping
that makes cross-host retrieval impossible is a property of which key is used). This project's
built-but-never-wired count is six and links 6–7 were two of them; the fix must not become the seventh.

## v0.124.1 — the repair record must survive the probe that did NOT feed the hub (2026-08-04, R-190)

**v0.124.0's transition record did not reach the hub, and the live run is what showed it.** The
capability reported degraded for "one cycle" — meaning the probe call that performed the repair. But
`probeAll` is invoked **independently** by the periodic self-check log and by the collector building a
host-report. On the demo box the repairing call was the log's (`09:39:34`, journal shows
`GRANT WAS MISSING AND HAS BEEN SELF-REPAIRED` and `degraded=1`), and the host-report built three
seconds later found the grant present and sent **`ok`**. The agent's journal had the record; the hub
had nothing; the operator would have learned nothing.

That is the exact silence R-190 is about, re-created inside its own mitigation — and every unit test
passed while it was true.

**The fix is a latch on TIME rather than on call count.** A confirmed repair is reported for
`storeGrantRepairReportWindow` (20 minutes), which comfortably exceeds the 900 s host-report interval,
so at least one report must carry the transition. It clears on its own — a permanently degraded
capability would be its own false alarm — and it is per tier.

**Two hollow tests were caught and fixed on the way**, both the same shape this repo keeps finding: a
test asserting a value it constructed itself, and a test asserting the latch HELPER rather than the
path that consumes it — whose red-proof duly passed. The decisions now live in
`storeGrantHealthyVerdict` and `storeGrantRepairedVerdict`, and the tests call those.

## v0.124.0 — a lost storage grant repairs itself, and says that it was lost (2026-08-04, R-190)

**R-190 is a grant that demonstrably worked at 04:44 on 2026-08-03 and was gone by 09:24** — with a
host reinstall, logged `pveum` activity and cluster-log entries all ruled out by measurement. The
cause is still open. The resilience does not have to wait for it.

**Everything needed already existed and had only ever been called once.** The root wrapper
(`felhom-backup-target-apply grant <id>`), its sudoers vector (`grant *`, any storage id, confirmed
not assumed), and the exact command were all in place — and the `grant` verb had only ever run at
storage CREATION. That is the *built but never wired* shape, in a verb rather than a seam, and it is
this project's seventh instance.

**What v0.124.0 does:** when the store-grant probe finds the grant absent on a tier the box depends
on, it runs that wrapper and **re-reads once** to confirm — the pbsdr R-22 self-grant shape, including
its restraint: one attempt, one confirmation, and anything still wrong stays loudly wrong.

**THE RECORD IS THE POINT, AND IT IS THE HALF R-190 IS ACTUALLY ABOUT.** A repair that leaves only
`ok` behind destroys the only evidence a permission vanished, so a recurring loss becomes undetectable
forever — strictly worse than the fault it fixes. So a confirmed repair reports **DEGRADED for exactly
one cycle**, with the explanation in `Feature`:

```
backup tier felhom-backup: the agent's storage grant was MISSING and has been AUTOMATICALLY
RESTORED — the tier works now, but a permission that vanished on its own needs investigating (R-190)
```

**Nothing new was built to carry it.** The hub's existing ok→degraded→ok edge is the channel — it
alerts and e-mails on the first edge and logs the recovery on the next cycle, so one loss produces
exactly one alert pair. No wire change, no hub change, no new event type. `Feature` carries the text
because that is the field the hub interpolates into the operator's e-mail; `Reason` does not travel.

**Bounded (Scenario F):** one attempt per tier per hour, in memory. A storage can be unreadable for
reasons an ACL cannot fix, and a re-grant on every report cycle is a repair loop wearing a fix's
clothes. An agent restart re-arms it, which is correct — a restart is exactly when a box should
re-check what it depends on.

**A failed repair never masks the fault:** the capability stays degraded with the failure in its
reason, and a repair that "succeeded" but did not survive the re-read is reported as needing a human.

## v0.123.0 — a tier the box cannot READ now says so (2026-08-03, R-185)

**The missing permission is one command. The silence was the defect.** On demo-felhom the agent's PVE
token held `FelhomAgentStore` on `local`, `local-lvm` and `felhom-pbs` — and **not** on
`felhom-backup`, the storage the same installer had configured as `local_backup_target`. Asked for
that storage's content the API answers `{"data":[]}` while root sees three archives (6.1–6.3 GB,
08-01/02/03).

**An empty listing is what a FORBIDDEN tier and a NEWBORN tier both return.** `pickForThisRun` skips
an empty tier — correctly, because a fresh offsite tier legitimately has nothing yet — and reports
*"no settled archive yet"*. So that tier was never restore-testable on that box and nothing ever
mentioned it. This project's own rule, in a new place: an empty answer is not evidence that there is
nothing there.

**The permission question, unlike the listing, has a definite answer — so it is asked directly.**
`Client.Permissions` reads `GET /access/permissions?path=/storage/<target>` **as the agent's own
token** (asking as root answers a different question and always says yes), and one
`capability.Status` per configured tier reports the result. It composes *around* the sudo prober, the
way the pool-read check already does — an API read does not belong inside a sudo-policy probe.

**MEASURED FIRST, and the obvious reading is wrong.** The ungranted path does not answer empty and
does not 403:

```
/storage/felhom-pbs     → {"Datastore.Allocate":1,"Datastore.AllocateSpace":1}
/storage/felhom-backup  → {"Sys.Audit":1,"SDN.Use":1,"Datastore.Audit":1}
```

It answers with the privileges **inherited** from the box-wide `/` grant. A probe asking *"is the
path present?"* or *"does it have `Datastore.Audit`?"* would report the blinded storage healthy — so
the probe tests for `Datastore.AllocateSpace` specifically, and a red-proof pins that.

**Decisions, each weighed once:**

- **The probed set comes from the box's own config** (`BackupTiers()`), not a fixed list. A hardcoded
  probe list is exactly the defect being fixed, reproduced inside the fix.
- **CRITICAL**, because the hub alerts only on critical and a non-critical entry would ride the
  report and alert nobody — the same silence with extra steps. **Except** the `local` fallback
  target, which host-install's own comment calls the DEGRADED configuration: it is still probed and
  still reported, but it does not page, because turning an ordinary documented setup into an alert
  is how a signal becomes something an operator archives unread.
- **It never looks at content**, so it cannot alarm on a newborn tier by construction.
- **It never reports ok when it could not ask.** An unreachable PVE is degraded: a self-check that
  fails open converts *"I do not know"* into *"fine"*.

The wire shape (`capability.Status`) is unchanged, so the hub's existing critical-degraded alert
applies with no hub change and no hub bump.

## v0.122.0 — three ways the signals lied about themselves (2026-08-03, R-189 · R-188 · R-186)

All three are the reporting and release path misreporting its own work. **No customer machine, no
backup, no restore, no disk layout, no data.** The restore-test itself and when it runs are unchanged
from v0.121.1.

### R-189 — a passing restore-test no longer vanishes on a restart

`restore_tests[]` came only from the in-memory `backup.Store`, whose own comment read *"lost on
restart; the cadence re-populates"*. That was true under a timer. It stopped being true when R-86 made
the agent refuse to re-test an archive it has already proven: a proof lost to a restart is not
repeated for a whole archive generation — **a week on the offsite tier** — and the hub calls the tier
unproven for all of it.

**Observed, not predicted (2026-08-03):** a real 14.5 GB offsite restore-test PASSED at 15:25:14, the
agent was restarted 2 m 43 s later for a deploy, and the hub logged `0 restore-tests` on the next two
host-reports.

The durable proof already existed — `RestoreTestState`, on disk, per tier, with the archive since
R-86 — and `Snapshot()` had carried the doc comment *"for the host-report gauge"* since the day it was
written **with no caller at all**: a seam built, documented, and never connected. It now carries the
`tier` and what was `verified` as well (stored at proof time, when they are known for certain, rather
than derived later by a storage lookup that can fail), and `ProvenRestoreTests` renders them as report
entries which the collector merges.

- **Merge rule: one entry per tier, newest by `TestedAt` wins.** A fresh failure beats a stored
  success — the failure is the news and lives nowhere else; a stored success beats a stale in-memory
  entry after a restart; a tier never appears twice, which the hub would read as two tests. An
  unparseable timestamp counts as older, so a malformed entry cannot displace a good one.
- **It refuses to lie.** A record missing the archive or the tier produces NO entry, and run mechanics
  (scratch VMID, duration) are not re-invented — an absent duration is not a claim, a fabricated one
  would be. An unproven tier reading as proven would be worse than the defect being fixed.
- **Only successes are persisted, and that asymmetry is now written down where it will be read:** a
  success suppresses future work, so losing it leaves the system quietly less tested than it believes;
  a failure causes future work and heals itself at the next evaluation.
- The `Store` comment that stopped being true is corrected in place rather than left to mislead.

### R-188 — a correct release no longer emails a failure

`on: [push]` fires the gates workflow on the **tag** push, and the release pushed its tag *before*
publishing, so CI ran the published-versions gate in the seconds before the package existed and
correctly reported it missing. Measured across two releases in one session: runs 12/13 and 17/18, same
sha each time, opposite results — a race, not a rule. R-168 made that mail the thing that cannot be
missed; one that is wrong half the time is one you stop reading.

**Only the tag PUSH moved** (build → tag locally → publish → push tag). The tag is still created before
anything is published, so the build and the tag still describe the same commit; it simply becomes
*visible* — to CI, and to any `raw/tag/…` fetch — once the package is downloadable.

The invariant the old order protected is **not traded away**: `check-published-versions.py` now asserts
the converse directly — **no published version may be missing its tag** — as a bounded probe of the
frontier (where a failed tag push leaves an orphan) and of patch gaps, printing its probe set every
run because a check whose coverage is invisible reads as a guarantee it is not making. The package
listing api still answers **401** without a token (re-measured), so absence cannot be enumerated, and
the script says so.

A publish that succeeds and a tag push that then fails now **dies loudly**, printing the one-line
recovery; and a publish that *fails* removes the local-only tag so the release can simply be retried
instead of colliding with itself.

### R-186 — a released binary can now be verified by rebuilding it

`go build` stamps a module version derived from VCS state, so a build made before the tag existed and
a rebuild made after it were different binaries. Measured at one commit, same source, same toolchain:

```
default flags, no tag yet ... 18f4a495…  14 085 464 B   (mod v0.121.2-0.2026…-3d0a1d61)
default flags, tagged ....... 4a38f394…  14 085 440 B   (mod v0.121.99)
-trimpath -buildvcs=false ... 7ffcdf1d…  14 064 574 B   IDENTICAL both ways
```

The stamp is removed rather than sequenced around — nothing in this repo reads it (no `ReadBuildInfo`
caller) and the version comes from the explicit `-X main.version` ldflag. `-trimpath` additionally
makes a rebuild from a different checkout directory match.

**A second discrepancy fell out of it:** `publish-agent.sh`'s fallback build forced `CGO_ENABLED=0` and
therefore produced a binary **74 KB smaller** than the release path built for the same version — one
version name, two binaries, decided by which entry point was used. Both now build identically.

`CLAUDE.md` records the exact command an operator can run to verify a published binary independently.

## v0.121.1 — "nothing is due" must be AUDIBLE (2026-08-03, R-86 + standing rule 3)

**Found while live-validating v0.121.0, and it is this project's own rule pointed at the change that
had just shipped.** Before R-86 every tick ran a heavy restore-test, so the scheduler was audible by
construction. After it, *"nothing is due"* is the NORMAL outcome — and it was logged at **DEBUG**,
which journald drops. An empty journal would then have been equally consistent with a healthy loop
and with a dead goroutine: the exact shape the R-88 watcher was retired for, re-created in a new
place by making the quiet path the common one.

A not-due evaluation now logs one **INFO** line naming every tier's verdict:

```
backup: restore-test evaluated — nothing due
  verdicts="felhom-pbs: newest settled archive (landed 2026-07-28T04:49:43Z) is already proven;
            felhom-backup: no settled archive yet — nothing to prove (newborn or still settling)"
```

Four lines a day at the 6 h default, and the answer to *"why did nothing run last night?"* is in the
log instead of being re-derived. A tier whose storage cannot be listed reads `UNKNOWN` with its error
in the same line, so a lookup failure can never present as "nothing due".

Red-proved by reverting to the bare `Debug` line: the test asserts what the SCHEDULER emits on a real
`tick`, not what the helper returns — a helper-level test would have passed against a tick that never
called it.

## v0.121.0 — a restore-test proves each BACKUP, not the clock (2026-08-03, R-86)

**The trigger changed; the restore-test did not.** `Scheduler.Run` still has a ticker, but it is now
the **evaluation interval** — how often "is anything due?" is asked — and no longer the thing that
decides a test happens. What decides is a per-archive due-check
(`internal/backup/restoretest_due.go`):

> Let **A** = the newest archive on this tier that has settled for at least `settle` (default 24 h).
> The tier is **DUE** when **A** exists and **A has not already been proven**.

A daily tier is therefore proved once a day, on yesterday's archive; a weekly tier once a week, on
its own; a newborn tier is UNKNOWN and never a fault. Per-archive due-ness IS the pacing — one test
per archive generation and no more — so there is deliberately no second rate limiter on top of it.

**The trap this avoided, recorded because it is the version a reasonable person writes.** R-86's own
wording ("~24 h after its own newest archive") implemented literally is *"due when the newest archive
is ≥ 24 h old"* — and on a **daily** tier that is never true, because a new archive resets the
newest-archive age to zero long before it reaches 24 h. The literal rule silently switches
restore-testing OFF for the tier that matters most. It has its own red-proof, which was observed
failing with **0 runs over 5 simulated days**.

**What the fix rests on**

- **The state records WHICH archive was proven** (`restoretest_state.go`), not merely when a tier last
  passed — a time cannot answer "have we proven *this* archive". A pre-R-86 state file keeps its time
  (rotation ordering survives the upgrade) and yields **no** proven archive, so each tier is due
  exactly once after the upgrade: one extra test per tier, once, which is the safe direction.
- **Two knobs replace one, and the old one is not silently repurposed.**
  `restore_test_eval_interval_seconds` (how often due-ness is asked; default **6 h**) and
  `restore_test_settle_seconds` (how long an archive must sit; default **24 h**). The deprecated
  `restore_test_cadence_seconds` keeps its DISABLE meaning (negative) verbatim, and a positive value
  now seeds the **settle lag** — with a start-up WARN naming both replacements.
- **6 h is bounded from both sides, not picked.** MEASURED cost of one evaluation on demo-felhom
  (Part 1.4): local dir storage **18 ms**, the PBS tier over the WAN to ep0 **392 ms**, both together
  **430 ms** — cheap enough for minutes, so cost is not the constraint. The **ceiling** is: a tier
  whose restore-test keeps failing stays due, so the evaluation interval is also its RETRY interval,
  and a retry is a multi-GB restore.
- **The due-check runs BEFORE the heavy-operation gate is taken.** Evaluations are frequent now, and
  holding that gate for a read that answers "nothing to do" would open a window at every evaluation
  in which a starting backup cannot acquire — and a backup that cannot acquire records a failure and
  pages the operator (F-A1). Nothing heavy starts before the gate.
- **The candidate picker skips implausible archives.** Under per-archive due-ness an incomplete
  1-byte phantom (F-CRIT-2's artefact, which server-side prune does not collect) would be picked
  forever, fail forever, never earn proof, and leave the tier due at EVERY evaluation — turning the
  evaluation interval into the retry rate for a multi-GB restore. `PickRestoreCandidateOn` now
  delegates to the settle-aware picker, so both callers agree.

**Unchanged, deliberately:** the restore-test itself (restore → boot → verify → destroy the scratch),
its journal, crash recovery, the scratch VMID band, the one-heavy-operation gate, success-only proof
credit, and oldest-proven ordering — which survives as the tie-break **between due tiers**.

**New:** `--selftest=restore-test-due` — read-only; prints the per-tier due verdict the scheduler
would act on, with the measured cost of the lookup.

**Live finding, pre-existing and NOT caused by this change (filed as R-185):** on demo-felhom the
agent's PVE token has no ACL on `/storage/felhom-backup`, so the API returns an EMPTY content listing
for that storage (root sees three archives). The host tier has therefore never been restore-testable
on that box, and both R-85's rotation and R-86's due-check report it indistinguishably from "newborn"
("no settled archive yet"). Verified live against `local` (grant present → 3 archives) and
`felhom-backup` (no grant → `{"data":[]}`).

## Releasing publishes, and an unreleasable version cannot pass CI (2026-08-03, R-115 + R-183) — **NO VERSION BUMP**

**No Go code changed, so nothing is bumped and nothing was built.** This is the release path and a
gate; the agent stays **v0.120.0**.

**`scripts/release-agent.sh` — THE way to release.** Build → **tag** → publish → **verify by an
independent download**. Publishing used to be a separate remembered step and was **forgotten three
times in five days** (R-111's seventeen stranded releases, 0.114.0, and 0.120.0 — which sat deployed
on both demo hosts and undownloadable, so a documented-path reinstall would have silently downgraded
them to the pre-merge agent *while reporting success*). R-111's own closing line named this leg and
closed SHIPPED without it; it recurred the same afternoon, which is the evidence that a note is not a
mechanism.

- It **tags** because `felhom-host-install.sh` now fetches this version's sixteen config files from
  `raw/tag/v<version>/` (R-183) — a released version without a tag 404s a box mid-install, as root.
- It **verifies by downloading what it just published** and comparing the sha to what it built. The
  publish step's own success is a report on its own write; a fetch returning the right bytes is a
  different claim, and it is the one that matters.
- It **refuses** a dirty or unpushed tree, and refuses to re-release an existing version — one
  version name must never mean two binaries.
- It **does NOT vouch.** Vouching points machines at a version and stays the operator's act.

**`scripts/check-published-versions.py` — the gate (R-115 mechanism (b)).** Every `v<semver>` tag
must have a downloadable package AND a tag tree that serves the agent's configs. Registered in
`agent_gates.py` as **not `--fast`** (it needs network, and a push must not fail because Gitea
blinked), and **the CI workflow now runs the FULL gate set** rather than `--fast` — otherwise the
gate would have been registered and never run, which is the built-but-never-wired failure this
project has shipped four times.

**The invariant is NOT the one the task specified, and the reason was measured.** The task asked for
*"the version the hub tells machines to install must be downloadable"*. That is the better invariant
and CI cannot see it: the hub's artifact manifest answers **401** without a per-customer passphrase,
and Gitea's package LISTING api answers **401** without a token, while the package DOWNLOAD url and
the git TAGS api are both anonymous. Putting an operator credential into CI to close that gap is the
operator's call, not a gate author's. The tag-based invariant needs no credential and **catches all
three recorded instances**, because the release script creates the tag and publishes in one act.
**What it does not catch — the hub vouching a version that was never released at all — is recorded
as R-184 rather than assumed away.**

## docs — v0.120.0 PUBLISHED + vouched, and proven on two reinstalled boxes (2026-08-03, R-178) — **no version bump, nothing built**

**Nothing shipped in this entry.** It records an operational fact the version history could not
otherwise carry: **v0.120.0 had been built, committed at `cd6e267` and deployed to both demo hosts,
but never published.** `GET https://gitea.dooplex.hu/api/packages/admin/generic/felhom-agent/0.120.0/felhom-agent`
returned **HTTP 404** (0.119.0 → 200), and the hub's Day-0 manifest accordingly vouched **0.119.0**.

**Why that mattered more than it looks.** `felhom-host-install.sh` step 5 skips the binary install
only when `installed == vouched` *exactly*, so a documented-path reinstall would have replaced the
merge-aware 0.120.0 with the pre-merge 0.119.0 — **and would have succeeded**, because the current
`step_grows` passes `-sysdata-grow 0` and 0.119.0's `mp1` resize (`bringup.go` 4c, fatal on error)
therefore never fires. Both demo boxes would have come back on an agent that predates the merge while
every log line read green.

Published this session on an operator ruling, from a clean tree (`git status --porcelain` empty,
`HEAD == origin/main == 4bb84fc3`): `scripts/publish-agent.sh 0.120.0` → upload **HTTP 201**,
round-trip GET verified, `AGENT_SHA256=a7763d31b55b5ce75457b4dba7b06aa300325811834b0be78af4587b47110b9d`;
then vouched in the hub manifest, which resolved the sha authoritatively from Gitea rather than
trusting the submitted value.

Both reinstalls then fetched and sha-verified it over the real customer path —
`verified sha256 a7763d31b55b5ce7… matches the hub manifest` — and 4b's single-volume grow was
exercised live on both boxes (`data +46G (->70G, ONE volume)` on demo-hp,
`+226G (->250G)` on demo-felhom), each producing `mp0` at `/var/lib/felhom` with **no `mp1`**.

**Filed as the third instance of R-115** (*"publishing is a remembered step"*), which has been
WAITING-ON-OPERATOR since 2026-07-29. Full evidence: `felhom.eu/REPORT.md`.

## v0.120.0 — one data volume (2026-08-03, R-165 · decision D-a · variant V-c) — `build-golden.sh` 2.1.0 → **3.0.0**

**The dedicated backup partition stops existing.** A golden built by `build-golden.sh` v3.0.0 ships a
**single** data volume; `mp1` is gone. An app whose local recovery unit outgrew the old 20 G area
stopped being backed up even with free space next door — D-a removed that wall rather than moving it.
**A layout change is a major**, hence the script's 3.0.0.

### The shape: V-c, a neutral mount — chosen on measurement, not on reading

```
mp0  ->  /var/lib/felhom          (one volume, backup=1)
           ├─ docker/     --bind-->  /var/lib/docker
           └─ sys_drive/  --bind-->  /mnt/sys_drive
```

Three candidates were **built and rebooted**, not reasoned about
(`felhom.eu/documentation/audits/SPIKE-r165-phase0-2026-08-03.md`). All three boot, reboot 3/3, give
ONE `df` figure, keep both paths as real mountpoints, and keep a container's `statfs("/")` reporting
the merged volume. **The ordering worry that motivated the probe did not materialise** — an
`/etc/fstab` bind is ordered under `local-fs.target`, which precedes `docker.service`.

They differ only in which documented guarantee they break, and that is what decided it:

- volume at `/var/lib/docker` → customer backups sit **inside Docker's data-root**; the ordinary
  "clear `/var/lib/docker` to fix Docker" reflex would destroy every local recovery unit on the box;
- volume at `/mnt/sys_drive` → Docker's **entire data-root** lands under `/mnt`, which the controller
  container mounts wholesale (`-v /mnt:/mnt:rslave`). **Measured: the container then sees
  `/mnt/sys_drive/docker`**, making the bootstrap's own claim that `/mnt` holds only Felhom's
  namespace mounts false.
- **V-c breaks neither**, for one extra path and one extra fstab line.

### The four assertions were RETARGETED, never deleted — and each was proven to fire

They exist because a volume silently falling out of the archive is invisible until a restore, which is
the worst moment to learn it. All were re-pointed at the merged invariant and then **run against a
deliberately wrong shape** (a real split-layout guest and a real archive of it): **8 checks, 8 passed.**

| # | now asserts | proven to reject |
|---|---|---|
| 1 | `/var/lib/docker` is a real mount (the V-c bind) | a plain directory |
| 2 | `/mnt/sys_drive` is a real mount | a plain directory |
| **2b (new)** | both paths are **ONE** filesystem | two volumes — the S2 shape the spike ranked *worse* than the split |
| 4 | the archive's `mp0` is mounted at `/var/lib/felhom` | an `mp0` at the wrong path |
| 5 | the archive carries **no** `mp1` | a leftover `mp1` (the merge-did-not-take case) |

Assertion 5 replaces the old "was `mp1` excluded?" guard, whose pattern could no longer match after the
merge — **a guard that cannot match is a guard that has silently stopped guarding.**

### Provisioning: one volume, one grow — and the retired knob is FOLDED IN, not dropped

`SysDataGrowGB` no longer drives a second resize; its GiB are **added to** the single volume's grow.
**A census established why this matters** — `felhom.eu/scripts/felhom-host-install.sh` computes and
passes `-sysdata-grow`, and an installer and an agent do not upgrade in the same instant. Dropping the
value would silently shrink every appliance an older installer builds by the user-data share (42 of
250 GiB on the standard branch). `-sysdata-grow` / `-sysdata-mount` stay **accepted** for the same
reason; removing them would make every install fail on an unknown flag. `DefaultSysDataMount` now
points at nothing on purpose, so a stale caller is loud at review.

**§8.2 — the single volume's size IS derived from the physical disk, and already was.** The installer's
`step_grows` reads the thin pool's real free space; the merge only collapsed its 80/20 docker-vs-sysdata
split into one total (`226` = the previous `184+42`, so a standard appliance keeps the same capacity —
250 G, no longer split by a wall). An unflagged install does **not** get the golden's 24 G.

## CI — the gate entry point runs on every push (2026-08-02, R-168) — NO VERSION BUMP

**No version bump, no build, no deploy** — this adds a workflow file only. Stated explicitly so the
omission reads as a decision rather than a miss.

**`.gitea/workflows/gates.yml` (new).** Triggers on `push`, `runs-on: felhom-gates`, obtains the
source with a shallow `git fetch` of the **exact pushed SHA** from the in-cluster Gitea Service, and
runs this repo's entry point with `--fast` — nothing else. **No `uses:` step anywhere**: JavaScript
actions need a node runtime the host-mode runner does not have, and probe P3 measured a plain
`git fetch` as sufficient. No `|| true`; the entry point's exit code IS the job's result.

**It REPORTS, it cannot REFUSE**, and the workflow header says so: this repo pushes straight to
`main` with no pull request, so there is no merge for a status check to stand at. The refusing half
is `.githooks/pre-push`, which is per-clone and `--no-verify`-able; this half notices when that was
skipped. Making CI blocking needs branch protection plus a PR workflow → felhom.eu `OPEN-ITEMS.md`
R-169, an operator decision.

**A failed run emails the operator** via Resend and prints the provider's accepted id, because probe
P5 measured that Gitea itself sends nothing at all on a failed run. Demonstrated end to end on a real
red run (`RESEND-ACCEPTED id=…`), not assumed. Full detail:
`felhom.eu/documentation/audits/SPIKE-ci-runner-2026-08-02.md`.

**CI reproduces the workspace's SIBLING LAYOUT on purpose.** This repo's entry point invokes the
shared `reuse_refs_check.py` that lives in the `felhom.eu` clone next door and is deliberately never
copied here, and this repo's `REUSE.md` cites `hub/internal/store/dr_recipe.go`, which lives in the hub. The workflow
therefore clones `felhom.eu` as a sibling; without it the gate fails **closed** with
`gate is MISSING` — correctly, but for the wrong reason. Verified that CI and the local hook then
agree exactly: 90 cited paths, 88 exact / 1 suffix / 1 cross-repo, 0 failures.

## Gate enforcement — this repo gets a place to put a gate (2026-08-02) — NO VERSION BUMP

**Deliberately no version bump, and no build or deploy.** Nothing compiled changed: this touches
`scripts/` and `.githooks/` only. Stated explicitly so the omission reads as a decision, not a miss.

**The census.** Thirteen gate scripts exist across the four felhom repos. A full run on 2026-08-02
found one clean correlation: every check a `CLAUDE.md` tells a person to run was passing, and two of
the four nobody is told to run were failing — one since 14 July. **This repo was the extreme case:
nothing at all ran against it, and its `REUSE.md` — 90 cited paths — was checked by no one.**

**`scripts/agent_gates.py` (new) — THE entry point, with one gate.** It runs `reuse_refs_check` over
this repo's `REUSE.md` and exits worst-wins non-zero. It exists at one gate on purpose: so the agent
is not the one repo with nowhere to put a check, and so the pre-push hook has the same entry point in
all four repos. It grows when the agent grows a second gate. The 90 citations now resolve as 88
exact, 1 by suffix (`localapi/debuglogs_test.go`), and 1 cross-repo (`hub/internal/store/dr_recipe.go`,
which lives in the hub).

**The shared checker is never copied here.** `reuse_refs_check.py` lives in `felhom.eu/scripts/` and
is invoked at `<repo-root>/../felhom.eu/scripts/`. A copy would recreate exactly the drift it exists
to detect. If the sibling clone is absent the gate **FAILS** and prints the path tried — fail-closed,
because a runner that quietly skips a gate is the inert-seam failure this project has shipped four
times. `scripts/test_agent_gates.py` pins that by running the entry point from a lone directory with
no sibling and asserting it convicts.

**`.githooks/pre-push` (new)** — runs `agent_gates.py --fast` and refuses the push. Per-clone
(`git config core.hooksPath .githooks`; a manual run WARNS when the clone is unarmed) and
`--no-verify`-able on purpose; both limits are written into the hook. CI is the unbypassable half and
is owed — `felhom.eu` `OPEN-ITEMS.md` R-168.

## v0.119.0 — the host report carries the box's addresses (2026-07-31)

**Pairs with hub v0.85.0.** The agent half is useless without it — the hub is what renders these.

**A managed box's IP was invisible in every operator surface, because nothing reported one.**
`HostMetrics` carried node, cpu, memory, disk, loadavg, uptime, temperature and the wrapper sha —
and no address of any kind. The hub therefore could not show a host's LAN IP anywhere; the only IP
reachable from the UI at all was the WireGuard one, and only on `/offsite`'s peer table keyed by
pubkey, so you could go peer→host and never host→peer.

**Two things that looked like the answer are traps, and both were checked before writing code.**
`lan_resolver.host_ip` is an OPTIONAL config value, absent unless that feature is configured. And
`DeriveHostIP(local_api.listen_addr)` returns `169.254.253.1` — since the R-50 island migration the
local API binds a link-local address that is **identical on every box**. Either would have produced a
confident wrong answer, which is worse than the blank it replaces.

**New wire field `addresses[]`, one entry per (interface, address).** Deliberately iface+cidr rather
than a single `lan_ip`: a Proxmox host legitimately holds several — a management bridge, a tailnet,
the WG tunnel — and picking one to call "the" LAN IP is a guess the agent is not entitled to make. On
a box whose management bridge is not `vmbr0` that guess is silently wrong. The agent reports what
exists; the hub does the labelling.

**The filter is one predicate, and it was chosen by MEASURING both demo hosts, not by reasoning.**
`IsGlobalUnicast()` alone drops loopback, IPv6 link-local (`fe80::/10`, one per bridge, pure noise)
and IPv4 link-local (`169.254/16` — exactly the island address above). It needs **no veth/fwbr/tap
denylist**, because on a Proxmox host that per-guest plumbing carries no IP at all and self-excludes:
`veth9201i0/i1` and the unused NICs appear in `ip link` and in no `ip addr` output on either box. What
survives is `vmbr0`'s LAN address, `wg-felhom`'s tunnel address and `tailscale0`'s tailnet addresses —
all true, all useful, none labelled here.

**No new privilege and no block I/O.** `net.Interfaces()` is a netlink/procfs read: it needs no sudo
grant, touches the sudoers fence not at all, and honours the health-check rule (R-117 spike §6.3) that
liveness is decided from `/proc` and kernel state, never by reading a filesystem.

**The seam defaults to the REAL enumerator.** `Collector.addrEnum == nil` uses `systemInterfaces`,
inverting the nil-reporter-means-off convention the optional stanzas use. Those gate on a config
feature; this one has no dependency and no flag, so a forgotten wiring call in `main.go` would have
shipped it silently empty — the inert-seam failure this repo has now recorded four times.

**Cross-repo, and the contract test enforces it:** `testdata/host-report.golden.json` is duplicated
byte-identically in `felhom.eu/hub/internal/api/testdata/`, and `TestHostReport_ContractMatchesGolden`
fails on any top-level key drift. Both goldens moved in this arc, and `addresses[0]`'s key set is
asserted bidirectionally alongside the existing sections. The field marshals as `[]`, never `null` —
the repo's own no-nulls invariant test caught that on the first run.

Files: `internal/hub/hostaddr.go` (new), `internal/hub/report.go`, `internal/hub/collect.go`,
`internal/hub/contract_test.go`, `internal/hub/report_test.go`, both goldens, `REUSE.md`.
Tests +9; three red-proofs (the global-unicast filter, the down-interface guard, and an inert
`collectAddresses`) each run, observed failing, and reverted.

## v0.118.1 — R-106 follow-up: the namespace was still being lost in the merge (2026-07-30)

**v0.118.0's R-106 fix was incomplete and live validation caught it.** Deployed to demo-felhom, the
recipe read:

    "pbs": { ..., "namespace": "root", "namespace_state": "resolved", ... }

`namespace_state: resolved` while the value was still the wrong `"root"` — which is a worse shape than
the original defect, because it asserts confidence in a wrong answer. The new state field is what made it
legible at a glance; without it this would have looked identical to the pre-fix output.

**Cause.** `mergeConfig` (`internal/storage/observe.go:457`) overlays the CLUSTER storage config onto the
NODE entry via a hand-listed set of type-specific fields — `Type`, `Server`, `Export`, `Share`,
`Datastore`, `Fingerprint`, `VGName`, `ThinPool`, `Path`, `Content` — and `Namespace` was not on that list.
`NodeStorage` does not return the namespace at all (it is cluster-config only), so the merged entry's
`Namespace` was always empty and `StorageTarget.PBSNamespace` read `""`. `latestPBSCoord` then treated an
empty configured namespace as the genuine root namespace, which is correct logic fed a wrong input.

**Why the tests did not catch it.** Every test added in v0.118.0 constructs `StorageTarget` values
directly — including the two that run `Collector.Collect()`, because they inject a `fakeObserver`. The
break was UPSTREAM of the collector, in the observer's merge, so "the production generation path" as I had
drawn it started one layer too late. Two new tests fix that: `TestObserve_CarriesPBSNamespaceThroughMerge`
drives the real `Observe` with the split PVE returns reproduced (namespace present in the cluster list,
absent from the node list — what PVE actually does), and `TestMergeConfig_CarriesPBSNamespace` tables the
merge itself including fill-if-empty vs never-clobber. Red-proof: removing the one added line fails both,
the first quoting the exact live symptom. Suite rc=0, 29 packages, 0 FAIL.

`Username` is the one remaining unmerged type-specific field; nothing in the observer path consumes it, and
the comment on the merge now says to add it here the moment something does.

## v0.118.0 — R-106 + R-109: the DR recipe stops guessing (2026-07-30)

**The recipe is read at the worst possible moment — by an operator rebuilding a machine that is gone —
and it was wrong about the PBS namespace and silent about the backup target.** Both were confirmed live
on both demo boxes before the fix, in the recipe the hub actually serves:

    "pbs": { "repo_id": "felhom-pbs", "namespace": "root", "latest_snapshot_id": "9201" }
    (no backup_target field at all)

while `/etc/pve/storage.cfg` on those same boxes reads `namespace demo-felhom` / `namespace demo-hp`.

**R-106 — the namespace.** `DRPBSCoord.Namespace` was taken from the LISTED SNAPSHOT. PBS does not echo
`ns` per item once the request is already namespace-scoped via `?ns=` (`internal/pbs/client.go:118-120`),
so `Snapshot.Namespace` was always empty, `ToHub` normalised empty → `"root"` (`internal/pbs/report.go:22-25`),
and `latestPBSCoord` wrote that into the recipe. Every per-customer box therefore reported the root
namespace while its backups lived in a tenant one. It now resolves from the pbs STORAGE — `PBSNamespace`,
carried on `StorageTarget` from storage.cfg's `namespace` — which is the same field `vzdump --storage <pbs>`
makes PVE read, so the recipe cannot disagree with the backup that produced the snapshot. One state, one
owner. An unconfigured namespace still reads `"root"`: that is an ANSWER (the box is genuinely in the root
namespace), and `namespace_state: resolved` distinguishes it from not knowing.

**R-109 — the backup target.** The recipe listed every storage's name/type/content and never said which
one holds the local whole-guest archives. Harmless while that was the well-known `local`; not harmless
since the 2026-07-28 vzdump-target move, after which every box carries TWO `content=backup` dir storages —
`felhom-backup` (live) and `local` (archives frozen at the move) — indistinguishable by name, type and
content. Picking the frozen one restores a guest that restores cleanly and is silently months stale. The
host-half now carries `backup_target`, resolved from the PRIMARY tier of `cfg.Backup.BackupTiers()` — the
same function the scheduler consults, not a re-derivation — plus the mountpoint, which is what actually
separates `/mnt/hdd_1` from `/var/lib/vz`.

**Absence is recorded as absence.** Neither field emits a default, an empty string, or a placeholder when
it cannot be resolved. `backup_target` has three outcomes and the two unknowns are deliberately distinct:
`agent_backup_config_unavailable` (the collector has no config seam) and `not_a_known_storage` (the config
names a storage this host does not have — the id IS still recorded, because that sends an operator
somewhere useful while silence does not). `namespace_state` likewise refuses to default to root when there
is no storage row to read.

**The resolver reports the tier IN EFFECT, not the file.** `SetBackupTargetResolver` closes over the
daemon-start config on purpose. A backup-target move rewrites `agent.json` and deliberately does NOT restart
the agent (the E-1 lesson: restarting mid-backup records a spurious failure for a run that succeeded), so
between the write and the restart the file names a target no backup is writing to yet. The live-reload shape
used for `escrow.pbs_storage_id` would have made the recipe point at the new storage while every archive
still landed on the old one.

**A cross-repo half of this was load-bearing and is easy to miss:** the hub's `AssembleDRRecipe` allow-lists
top-level keys, so `backup_target` would have been *stored intact and silently dropped* before reaching any
operator. Shipped together with hub v0.83.0; neither half is useful alone.

Not touched: the backup machinery. This corrects the RECORD, not the doing — and the record was never
acted on programmatically (the host_loss plan is surface-only, `internal/dr/plan.go:125`), so the blast
radius was exactly one operator reading a wrong value. `plan.PBS` inherits the fix for free.

- `internal/hub/report.go` — `StorageTarget.PBSNamespace` (`json:"-"`, the ConfigPath precedent: the struct
  is a cross-repo contract and nothing off-box needs this value).
- `internal/storage/observe.go` — populate it from `proxmox.Storage.Namespace`.
- `internal/hub/dr_recipe.go` — `DRBackupTarget`, `ConfiguredBackupTarget`, `DRState*`/`DRReason*`,
  `namespace_state`/`namespace_reason`, `resolveBackupTarget`, namespace from the storage.
- `internal/hub/collect.go` — `SetBackupTargetResolver` seam; an unwired seam reports unknown, never a guess.
- `cmd/felhom-agent/main.go` — `primaryBackupTargetOf` wired at BOTH collector sites (the daemon and
  `selftest=hub`, so "the report it would send" really matches what the daemon sends).
- `testdata/host-report.golden.json` — new keys; kept byte-identical with the hub's copy (`f4bc3554…`).

Tests: 9 new. The consequence ones assert a box with two `content=backup` storages names the LIVE one and
does not name the frozen one — `assertBackupCandidateAmbiguity` fails the test if the fixture ever stops
posing that problem, so it cannot pass hollow. Fixtures are the storage set demo-felhom really had
(provenance recorded in the file: the box's own pre-fix recipe + its `storage.cfg`), not composed structs.
Two run the REAL path, `Collector.Collect()`, one of them specifically pinning that an unwired seam yields
unknown — a seam built and never wired is the failure mode this repo has hit four times. Red-proofs: 4,
each mutation asserted to have landed before running (revert R-106 → the live `"root"` symptom reproduces
in both the unit and production-path tests; make the unknown case guess → 5 assertions fire; drop the
field → the cross-repo contract guard fires too; hollow the fixture → the ambiguity guard fires).
Suite rc=0, 29 packages, 0 FAIL.

## v0.117.0 — R-117: the liveness signal now tests liveness (2026-07-30)

**`BoundUnderParent` reported a namespace that returned `EIO` on every read and write as healthy, and the
gate restarted the customer's apps onto it.** Both existing terms — `GuestSeesMount` and
`isHostMountpoint` — parse a mountinfo line and then test only `fields[4]`, the mount POINT. Field 3, the
`major:minor`, sat in the same parsed slice and was discarded. So after a drive was detached and returned,
the raw host mount healed onto the NEW device via its fs-UUID-keyed unit while the bind still named the
OLD one, and both terms stayed true. Measured on hardware: raw `8:32 /dev/sdc`, bind `8:16 /dev/sdb` with
`shutdown`, `bound_under_parent: true`, `EIO` both directions — and the controller taking its `Return`
branch, restarting the gate-stopped apps and emailing `backup_target_restored`, with no alarm on any
channel (`felhom.eu/documentation/audits/SPIKE-r117-bind-liveness-2026-07-30.md`).

**THE FIX is one predicate.** `BoundUnderParent` gains a third term, `bindUsable`, at both /disks
construction sites (`disks.go`). The new `bindLiveness` (`intermediary.go`) reads `/proc` only and asks
two questions:

- **the bind must name the same device as the raw mount** — sound rather than heuristic, because a stale
  bind pins the dead superblock, which keeps the old device index allocated, which FORCES the returning
  device onto a different number (measured both ways: releasing the bind let the letter be reused);
- **the filesystem must not have aborted** — ext4's `shutdown` (device removed) or `emergency_ro`
  (`errors=remount-ro` fired in place). Both tokens were measured and both are load-bearing.

**The second check is not optional, and this is the substantive part.** R-117 was filed as a
detach/return defect, but a device that fails WITHOUT disappearing produces the identical
all-signals-healthy state with the devnos EQUAL and the drive never `Disconnected` — so the gate produces
neither a `Stop` nor a `Return` action and **nothing is emitted on any channel, indefinitely** (R-117a).
The device comparison alone cannot see it. A devno-only fix would have passed every payload test.

**THREE states, never a bool.** `BindLiveness` is `{Unknown, Live, StaleDevice, Aborted}` and every
caller reads it through `Usable()`, where `Unknown` counts as PRESENT — unreadable `/proc`, no raw mount
to compare against, or a filesystem whose abort vocabulary we have not measured all degrade to unknown,
because reporting absent stops a working customer's apps. The workspace's false-invariant table records
`newestArchiveOn` promising exactly this over a `(value, bool)` shape that could not express it.

**NO NEW RECOVERY PATH — the existing one was unblocked.** `AttachDrive`'s normalize leg already performed
the needed repair, and three call sites already invoked it: the 20 s reconcile ticker
(`cmd/felhom-agent/main.go`), agent startup, and **the controller's `Return` branch, before it restarts
the apps**. All three were defeated by `if n == 1 && b.GuestSeesMount(...)` returning early and logging
*"fully live, no-op"* about a dead namespace. The arm now switches on the verdict:
**`BindStaleDevice` ⇒ re-bind** (repairs live, guest never restarts — proven, init PID identical);
**`BindAborted` ⇒ quiet no-op**, because a re-bind lands on the SAME aborted superblock and this runs
every 20 s, so re-binding would be an infinite silent retry that also masks the state — it surfaces via
`BoundUnderParent=false` instead, and clearing an aborted filesystem needs a remount or a fsck, which is
an operator decision. **Ordering trap, caught by a test:** reading the abort flag before comparing devices
classifies the real return state as aborted (its stale bind carries `shutdown` too) and refuses the repair
while still reporting correctly — so the abort flag is read off the RAW mount in the stale case.

**Also:** `isHostMountpoint` and `countHostMounts` are now one-liners over the single new parser
`hostMountEntries`, which yields devno, root, fstype and super options (REUSE.md's
three-independent-readers note narrowed). The three comments that promised `BoundUnderParent` meant
"live + usable in the guest" — true for three releases only as a wish — now name the tests that pin them.
`CLAUDE.md` gained a standing rule: **a health check issues no block I/O** (a probe on a wedged device
survives `SIGKILL`; measured).

Tests **849 → 863** (+14 top-level, verified against a temp worktree at HEAD — not inferred from a count taken mid-edit, which is how the first draft of this line read 858 → 873), all green: 29/29 packages, `go build`/`go vet`/`go test` each with `rc=0` read separately. **6 red-proofs, each verified to have landed**: term 3 removed at
each construction site; the abort check neutered (the P1-only fix — fails the aborted tests ALONE);
`emergency_ro` dropped from the token list; unknown reported as not-live; and the aborted arm falling
through to a re-bind. **A hollow test was caught and fixed during the red-proofs**: the aborted fixture
first used a `/dev/mapper` device, for which `RoleForStorage` derives `role="system"` — a system row has
no `GuestPath`, never runs the conjunction, and reports `false` by default, so the assertion passed
vacuously and NO mutation could fail it. Found because RP1 failed to fail. The fixtures now assert the
production row shape before asserting the field.

## v0.116.0 — R-116: the flag and the key finally share a row (2026-07-30)

**Closes the mechanism `v0.115.0` guessed at.** The absent-drive alarm was the generic
`storage_disconnected` while its recovery was the specific `backup_target_restored` — a pair an
operator cannot match. The cause is now measured, not reasoned: `felhom.eu`
`audits/DIAG-r116-disks-payload-2026-07-30.md`.

**What the captured payload showed.** With the device gone `/disks` returns **4 rows, not 3** — the
drive appears **twice**, and the two facts the controller needs are on **different rows**:

| row | source | `mount_path` | `guest_path` | `backup_target` |
|---|---|---|---|---|
| `felhom-backup` | Observe | `""` | `""` | **`true`** |
| `694034cc-…` | registry union | `/mnt/cel` | `/mnt/felhom-drives/cel` | field absent ⇒ **`false`** |

`driveTargetByPath` (controller `intermediary.go:602-618`) registers a key only from a **non-empty**
field, so the flag-bearing row contributes **no key at all**, while the registry row owns
`/mnt/felhom-drives/cel` — the key the gate looks up — and says `false`.

**Why v0.115.0 was inert, and it was not subtle.** Its fallback computed
`StablePathForRaw(t.MountPath)` on the Observe row, and in the absent state `t.MountPath` is `""` —
emptied by the same `exactMount` failure that empties `BackingDevice` (`observe.go:184-190`). So it was
`StablePathForRaw("")` == `""` and it assigned nothing. **Its test passed because the fixture supplied a
`MountPath` the real absent state does not have.** That fixture is corrected in this release, and
red-proof 1 replays v0.115.0's exact code against the corrected shape: it fails.

**THE JOIN — the actual hard part.** With the device gone the two records of one drive share **no
runtime field**: no mount, no backing device, and the Observe row's `DurableID` has already degraded off
the fs-UUID (`path:/mnt/cel`). What they do share is **configuration** — the storage's `path` from
`storage.cfg` on one side, the `.mount` unit's `Where` on the other. Both yield the same stable guest
path, so that is the key the dedup now uses.

- `hub.StorageTarget.ConfigPath` — the configured path, carried verbatim, **`json:"-"`**. That struct is
  a cross-repo contract pinned by `testdata/host-report.golden.json` + `contract_test.go`'s key-set
  comparison; a wire-visible field would have required a matching `felhom.eu/hub` change to stay
  non-drifting, and nothing off-box needs the value.
- `observe.go` sets it from `s.Path`. **This is NOT the fallthrough the comment at `:176-183` forbids** —
  that prohibition is about resolving a *device or UUID* from the **containing** filesystem, which would
  hand back root's identity and mis-target a DR re-attach. `s.Path` is the storage's own declaration,
  identifies nothing but itself, and is used for no resolution. `MountPath` stays empty, which is true.
- `disks.go` tries `MountPath` first (so the present-state path and v0.115.0's behaviour are
  byte-identical) and falls back to `ConfigPath` only when the mount is genuinely gone.
- The union loop now dedups on guest path as well as mount path, so exactly **one** row carries the drive.

**THE REGRESSION THIS AVOIDED, and it is why the two obvious fixes were both rejected.** The controller
reads `d.BackupTarget && d.MountPath != ""` as *"a real drive with its own mountpoint — healthy"* and
returns immediately (`backup_target_offer.go:79`). So **back-filling `MountPath` onto the Observe row**
(the smallest change) and **teaching the registry row the flag** (its `MountPath` is non-empty, read from
the by-then-stale unit file) both produce a row satisfying that predicate while the drive is missing —
either would have silently flipped **R-114**, which shipped 2026-07-29, back to a false *healthy*.
**R-114's correctness currently rests on the absent-state rows not combining the flag with a mount
path**; that coupling was invisible until the payload existed. Pinned by
`TestAbsentTargetKeepsR114DegradedSignal`, red-proofed by feeding the handler exactly what the rejected
option would have produced.

**Deliberately unchanged:** `Role` (the absent target still reads `system`), the `BoundUnderParent`
conjunction, and every wire field. Suppressing the registry row in the absent state removes its
`uuid:` `durable_id`, its hardcoded — and false — `state: "attached"`, and its `total_bytes` copied from
the **root** filesystem (**R-118's symptom, incidentally gone; R-118 itself is NOT fixed and stays
open**). No consumer of those was found: wipe/decommission need the drive present, `firstOfferableDrive`
already excludes it, and the disk-health baseline excludes UNKNOWN verdicts.

**PROVEN LIVE 2026-07-30 — the full four-event sequence, on a fresh box.** Real day-0 on a nested PVE on
demo-hp, the agent **installed unaided from the vouched Day-0 manifest**, drives enrolled through the real
endpoints, device loss a real hot-detach:

```
07:20:04  backup_target_absent   (error)  Cel meghajto   ← the TARGET, specific, was generic before
07:22:34  backup_target_restored (info)   Cel meghajto   ← its matching pair
07:24:04  storage_disconnected   (error)  Adat meghajto  ← a NON-target, generic, same box
07:25:34  storage_reconnected    (info)   Adat meghajto
```

All four reached the hub. Gate fired in **3 s**. **Discrimination is proven non-trivially for the first
time** — both prior runs had the target itself emit the generic event, so their mirror proved nothing.
Over-correction guard passes on a *positive* observable: 0 ABSENT lines and 0 drive events over 2m14s with
both drives present, while two `RETURNED` lines prove the gate was ticking. Audit: `felhom.eu`
`audits/R116-v0116-2026-07-30.md`.

**Tests 845 → 849.** Four red-proofs, each mutation verified to have landed before the run:
(1) v0.115.0's MountPath-only fallback → `isTarget[guestPath] = FALSE, rows=2`; (2) drop the guest-path
dedup → `carried by 2 rows`; (3) give the absent row a MountPath → the R-114 guard fires; (4) over-broad
dedup → the non-target drive loses its own row (and two pre-existing R-113 tests fail too).
`TestPresentTargetPayloadUnchanged` pins the healthy payload field-for-field — the state the whole fleet
is in.

## v0.115.0 — R-116: the backup-target flag reaches the row the controller keys on (2026-07-29)

**The defect, measured live in Session C.** A drive whose device vanished raised the **generic**
`storage_disconnected`, while its return raised the **specific** `backup_target_restored` — an alarm
and an all-clear an operator cannot pair. `backup_target_absent` never fired at all.

**The mechanism, and it is not what the Session-C audit first said.** `RoleForStorage` returns
`RoleSystem` whenever `backingDevice == ""` (`internal/storage/role.go:180-181`). When the device goes,
Observe's `exactMountDevice` fails, `t.BackingDevice` becomes `""`, the target row's role flips to
system and it **loses its guest path** — but keeps its `MountPath`. The union loop skips any drive whose
`MountPath` is already `seen`, so the registry row is **deduped away entirely**. `/disks` ends up with
**no row carrying that guest path**, so the controller's `isTarget[guestPath]` is a **missing key**, not
a `false`. The obvious fix — setting `BackupTarget` on the union row — **could not have worked**: that
row is not emitted in the state where the alarm is needed. The audit has been corrected.

**The fix.** On the Observe row only, carry the guest path when the row **is** the backup target **and**
its role flipped because the device vanished:

```go
if di.GuestPath == "" && di.BackupTarget && t.BackingDevice == "" {
    di.GuestPath = StablePathForRaw(t.MountPath)
}
```

**Three gates, each verified rather than assumed:**

- `t.BackingDevice == ""` restricts this to the vanished-device flip. A storage that is `RoleSystem`
  because it is genuinely system-**backed** has a real device and is excluded — otherwise a dir storage
  at `/mnt/<name>` on the root disk would acquire a guest path.
- **Case B, the common fresh-box shape, is safe twice over.** Its target is the builtin `local` on
  `/var/lib/vz`, and `StablePathForRaw` returns `""` for anything that is not exactly `/mnt/<name>`
  (`DriveNameFromRaw`, `intermediary.go:79-88`) — nothing is set even before the gates apply.
- **It cannot make the gate read an absent drive as PRESENT.** `BoundUnderParent` is assigned at exactly
  two sites (`disks.go:222`, `:280`), both inside guest-path blocks a system-role row never enters, so
  it stays `false` and `planDriveGates` computes `present[gp] = false || false`. Inert by construction —
  pinned by `TestAbsentTargetRowDoesNotRegisterPresence`. Getting this wrong would have **silenced the
  alarm this fix exists to raise**.

**The `:213-214` boundary stands.** No system or backup mount gains a guest path; only the drive the
alarm is *about* keeps its identity while it is missing, and only while its device is gone.

**Tests** +5 in `internal/localapi`, asserting the emitted `/disks` JSON through a faithful copy of the
controller's `driveTargetByPath`, because the failure class is "the value is on the wrong row" and a
hand-built fixture proves nothing about which row the handler emits. Red-proof: removing the block
fails with *"isTarget[…] is a MISSING KEY"*; reverted, byte-identical.

**Known limitation, filed not closed:** the two-row shape that produced this survives. The flag and the
guest path still live on different rows in the healthy state, and nothing prevents a future consumer
keying on the wrong one.

## v0.114.0 — R-113: drive presence means the DEVICE, not the bind (2026-07-29)

**The bug, measured live in E-2d.** `BoundUnderParent` — the one field the controller's drive-absent
gate keys on — reported only *"is this path a mount target in the guest's mountinfo"*. The drive's raw
mount at `/mnt/<name>` is a systemd mount unit bound to its device and dies with it, but the agent's own
bind of `<raw>/felhom-data` under the shared parent is an ordinary bind: nothing ties it to the device,
so **its mountinfo entry outlives the device as a stale shell**. Presence read that survivor as `true`,
`planDriveGates` never produced a `Stop` action, and **nothing fired on any channel** — not
`backup_target_absent`, not the generic `storage_disconnected`. Detached at 10:58:37Z, silent for 4½
minutes while the agent itself logged `enrolled drive absent by UUID` every 20 s
(`felhom.eu` `audits/E2D-fresh-vm-2026-07-29.md` §5.2).

**The fix.** `BoundUnderParent` becomes a **conjunction**: bound under the parent **AND** the drive's raw
host mount still mounted (`devicePresent`, new `deviceCheck` seam defaulting to `isHostMountpoint`).
Applied at BOTH `/disks` construction sites — the Observe path and the registry union path. The union
path matters more, not less: it hardcodes `State:"attached"`, so the raw-mount check is the only device
truth that row carries, and it is exactly the shape E-2d detached.

**Why a conjunction and not a replacement.** Half 2 alone would regress boot ordering, where the raw
drive mounts early and the bind lands ~18 s later; the gate depends on that window reading ABSENT so apps
stay stopped until the bind is live. The conjunction leaves that behaviour byte-identical and closes only
the case the gate could never see. Pinned by `TestDisks_DevicePresence_BootWindowStillReadsAbsent`, whose
red-proof is dropping the bind half.

**Unknown is never absent.** `devicePresent("")` returns **true**. A false absent stops a working
customer's apps — the failure mode of this fix, not of the bug — so an unanswerable question must never
be reported as "gone". Red-proofed by inverting it.

**Controller UNCHANGED, no MinAgent bump.** `BoundUnderParent` has exactly one functional consumer,
`planDriveGates` (`intermediary.go:226`); every other mention across both repos is a comment or a test,
and boot convergence deliberately moved off it to `pollLiveBinds`/`driveBindLive`. Tightening the field's
meaning therefore changes exactly one decision — the broken one. The alternative (a new `DevicePresent`
bool the controller ANDs in) was rejected as actively dangerous: a bool absent from an older agent's JSON
decodes to `false`, so every drive on a pre-0.114.0 agent would have read ABSENT and stopped its apps.
An older controller against this agent simply gets a more truthful bool.

**Tests:** +6 in `internal/localapi` (208 → 214), groups A–D plus a wire-contract test that asserts the
encoded `bound_under_parent` field, since that is what actually crosses to the controller. Four
red-proofs run and reverted (drop the conjunction on each path; invert unknown; drop the bind half).

**NOT LIVE-VALIDATED.** No drive was pulled. The leg awaiting Session C: device loss → gate `Stop` →
`SetDisconnected` → `backup_target_absent` on the wire.

## v0.113.0 — E-2a: the guarded backup-target wrapper + POST /backup/target (2026-07-29)

**The agent cannot do this itself, and that is the point.** Creating a PVE storage needs
`Datastore.Allocate` at `/storage`; the ACL grant needs `Permissions.Modify`. The agent holds
**neither** — its token is scoped per storage path for blast-radius containment, and
`Permissions.Modify` would let it rewrite its own authority. Widening the role to make the move
possible would trade the whole containment model for one feature. So the privileged half lives in a
new fenced shim, `configs/felhom-backup-target-apply`, behind a literal `FELHOM_BACKUPTARGET`
sudoers alias — the felhom-mkfs-guarded / felhom-pbs-apply pattern.

The wrapper enforces the two laws E-1 paid for on live hardware, so they cannot be forgotten by a
caller:

- **F-1** the path must BE the drive's own mountpoint (`mountpoint -q`), or the target reports
  `disconnected` forever and its durable id degrades off the filesystem UUID;
- **F-2** `--is_mountpoint 1` is HARDCODED, not a caller flag. Without it an unplugged drive leaves a
  bare directory on the root fs and vzdump writes onto the system drive while PVE reports `active`.

It also refuses a target backed by the root device, and carries **no storage-removal path of any
kind** (the felhom-pbs-apply no-delete law; grep-assertable). `create` is idempotent for the same
path and REFUSES to repoint an existing id at a different one — silently moving a live backup target
is the failure this arc closes.

`POST /backup/target` drives it in a fixed order — **create → grant → config**. Reversed, a config
pointing at an ungranted storage 403s every backup on first run, which is exactly E-1's finding F-3.
A failed grant therefore leaves the config untouched (tested).

**It deliberately does NOT restart the agent.** The tiers are built once at daemon start, so the move
needs a restart — but restarting with a backup in flight cancels the wait and records a SPURIOUS
tier failure for a backup that actually succeeded, which E-1 did to a real `felhom-pbs` run. The
handler returns `restart_required: true` and the caller restarts behind its own immediate in-flight
check. Tests assert the handler never issues a restart itself.

Config rewrite preserves unknown keys verbatim (`map[string]json.RawMessage`, the
`pbsdr.seedEscrowStorageID` discipline) and writes in place, because /etc/felhom-agent is root-owned
while agent.json is agent-owned 0600 — a rename is impossible for the non-root agent.

## v0.112.0 — E-2: GET /disks flags the backup-target drive (2026-07-29)

Additive field `backup_target` on each `/disks` entry, true for the drive backing the PRIMARY tier.

**The controller cannot work this out for itself.** Its `settings.StoragePath.BackupTarget` is
customer INTENT, and on the two boxes migrated by hand in E-1 that intent was never recorded — so
intent is empty while the drive really is the target. Without this flag the absent-target alarm could
not name the drive on exactly the boxes that currently have one. `omitempty` + false on an older
agent, so an old controller degrades to the generic disconnect alarm rather than a wrong one.

Test asserts the target IS flagged **and** the non-target is NOT, as a pair — a blanket `true` would
satisfy a naive one-sided check.

## v0.111.0 — E-2c: the backup-target drive can no longer be ejected out from under the backup (2026-07-29)

**A regression guard on a configuration that is live right now.** E-1 (2026-07-28) moved each demo
box's whole-guest vzdump target onto its secondary drive, at that drive's own mountpoint. But
`RoleForStorage` types a `local-dir` on a non-system device as **user-data** — so the pre-existing
eject role gate PASSED it, and `POST /disks/eject` on `/mnt/nvme-1tb` (demo-hp) or `/mnt/hdd_1`
(demo-felhom) would have **succeeded silently**, taking the only local whole-guest backup with it.
No alarm, no refusal; the box would keep reporting a configured tier while having lost its
drive-loss protection. Found by E-2's Phase 0, not by a failure.

`handleDiskEject` and `handleDiskDecommission` now consult `refuseIfBackupTarget` **after** the role
gate and refuse with `409` when the mount backs a configured backup tier, naming the storage and the
remedy — the op is ordered, not forbidden: reassign the target first, then the drive is free.

**Why this is NOT a role reclassification, which is the obvious fix and the wrong one.** Making
`RoleForStorage` return `RoleBackup` for the target would refuse every legitimate eject of the
customer's own data drive, because on both demo boxes that drive **is** the target (the vzdump
target sits beside `felhom-data` on the same mountpoint). That trades a silent failure for a
permanent obstruction. The gate is therefore separate and narrow, and the role vocabulary is
untouched.

`backupTargetAt` resolves through the agent's OWN storage view, never the caller's claim, and fails
**OPEN** — safe precisely because it sits behind the role gate, which already fails **SAFE** on the
same error, so an unresolvable mount is refused before it reaches here.

**Tests + red-proofs.** Eject refused, decommission refused, and — the one that constrains the
design — `TestEjectStillAllowedOnANonTargetDrive` pins that a non-target drive stays ejectable.
Red-proofed both ways: removing the eject guard reproduces `eject of the backup-target drive
SUCCEEDED (200)`, and implementing the over-correction (treat any backup-content dir storage as the
target) fails the non-target test with the gate blocking `/mnt/spare`.

**Harness note worth keeping:** `normalizeBackupTiers` DROPS any tier with a nil `Service` and falls
back to the legacy tier with an empty `TargetID`. An earlier version of this test therefore exercised
nothing and reported the production bug as if it were the fix failing.

## v0.110.0 — F-LEAK, third attempt: the fourth root-fenced exception (2026-07-28)

**The band-scoped ACL fix (v1.21.0) is durable for exactly ONE use per slot, and the live check caught
it.** After the first successful teardown of `990000`, `pveum acl list` showed **0 grants** at
`/vms/990000`. Cause, confirmed in PVE's own source rather than inferred: the destroy path calls
`PVE::AccessControl::remove_vm_access($vmid)` (`API2/LXC.pm:906`), which deletes
`acl_root->children->vms->children->{$vmid}` — **every ACL at `/vms/<vmid>`** (`AccessControl.pm:1898`).

**The grant is consumed by the very operation it authorises.** After ten teardowns the whole band is
ungranted and F-LEAK returns. A per-VM ACL is structurally the wrong tool here; the pool grant is
durable only because it lives on a *pool* path, which `remove_vm_access` does not touch (it removes the
guest's pool *membership*, not the pool's ACL).

**So this is attempt three, and it is the route the task offered as the alternative:** route the destroy
through the same authority that already performs the create. `Privileged.DestroyScratchLXC` is the
**fourth** root-fenced exception (previously exactly three: keyctl `pct create`, USB mount/fstab,
SMART/sensors), and it is fenced harder than any of them:

| layer | enforcement |
|---|---|
| **sudoers** | `/usr/sbin/pct destroy 99000[0-9] --purge` — sudo matches the vmid **literally**. Even a compromised agent asking for `pct destroy 9201` is refused by sudo itself. |
| **`DestroyScratchLXC`** | re-checks the band before exec; refuses an unconfigured or inverted band rather than defaulting |
| **`teardownScratch`** | acts only on this journal entry's own scratch provenance |

None of the three is consumed by use, which is the property the ACL lacked. The API destroy is still
tried **first** and remains the normal path; this is the fallback that makes teardown deterministic
instead of once-per-slot. The v1.21.0 band ACLs stay provisioned — they make the common case need no
privileged call at all.

**Ships with a sudoers change**, so `configs/felhom-agent.sudoers` must be deployed with the binary
(`visudo -cf`: parsed OK).

Red-proofs 9 and 10, both observed failing: removing the band check →
`REFUSAL FAILED: executed [pct destroy 9201 --purge] for out-of-band vmid 9201`; removing the
unconfigured-band check → `an unconfigured band admitted vmid 0 and EXECUTED [pct destroy 0 --purge]`.
The out-of-band table deliberately includes **9201**, the live customer guest on both demo boxes.

## v0.109.0 — the guest-power watchdog gets the observable it was shipped without (2026-07-28)

**Self-correction to v0.107.0, found by running the very check this session added elsewhere.** The new
guest-power watchdog logged at startup and when it ACTED, and was otherwise silent — so on a healthy
box the only evidence the sweep was running was the *absence* of start lines. That is exactly F-OBS's
shape, and exactly what standing rule 3 forbids: an absent log line is not evidence of correct
behaviour. It shipped in the same session F-OBS was fixed in `deadapp-check`, which is what makes it
worth recording rather than quietly patching.

`GuestPowerTick` now emits an INFO summary every **10th** sweep (10 x 60 s = 10 min, matching the
controller's deadapp heartbeat) carrying `sweeps_since_boot`, `guests_evaluated` and
`currently_stopped`. It reports **what the sweep saw**, not merely that it ran — "alive, all guests up"
and "alive, one guest deliberately left down" are different operator facts and a bare liveness ping
cannot express the second.

Two bounds, both pinned by test in the direction that would break them:
- **Not a flood.** One line per sweep would be 1440/day, which is the pressure that made silence
  attractive in the first place. A test fails if the cadence degenerates to per-sweep.
- **An aborted sweep does not count.** If `Guests()` fails, ownership is unproven and the sweep
  examines nothing; counting it would have the heartbeat claim liveness for a watchdog doing nothing —
  a worse lie than silence. The counter increments only after the guest list is in hand.

Red-proofs 7 and 8, both observed failing: removing the call →
`no liveness observable after 10 sweeps — silence is indistinguishable from a dead watchdog`;
moving the increment above the error return →
`an aborted sweep was counted as healthy (sweeps=20)`.

# felhom-agent — Changelog

## v0.108.0 — F-LEAK: the pool-adoption fix was WRONG; the fix is a path-scoped ACL (2026-07-28)

**A correction, made because the live replay refuted the design.** v0.107.0 shipped a scratch-teardown
fallback that, on a 403, adopted the stranded guest into the `felhom` pool and retried the destroy —
reasoning that the token holds `Pool.Allocate` on `/pool/felhom`. The live replay on demo-hp fired
that path exactly as designed and PVE refused it:

```
WARN  restore-test: scratch teardown failed — adopting the stranded scratch ... vmid=990000 pool=felhom
ERROR restore-test: pool adoption failed; left for Recover  vmid=990000
      err="proxmox: PUT /pools/felhom -> HTTP 500: permission denied at /vms/990000 (missing privilege ...)"
```

`PUT /pools/{pool}` **also** requires `VM.Allocate` on the VM being added — the very privilege the
403 was about. **Pool membership cannot bootstrap its own authority.** The adoption code and its
guard (`scratchAdoptAllowed`) are removed; a path that provably cannot work should not ship, and
leaving it would have left a plausible-looking fix in place of a real one.

**The actual fix lives in `felhom-host-install.sh` v1.21.0**: the `FelhomAgentGuest` role is now
granted at each `/vms/990000`…`/vms/990009` path — the restore-test's scratch band. PVE ACLs are
path-scoped, so this authorises the agent on exactly those ten IDs. It is **not** granted at `/vms`,
which would authorise destroying every guest on the box.

Proven live on demo-hp: `/vms/990000` → **has** `VM.Allocate` (13 privs); `/vms/100`, `/vms/9999` and
`/vms/990010` (one past the band) → **no** `VM.Allocate` (3 privs, base only).

Code side keeps only the corrected diagnostic on the teardown-failure branch, which now records what
the 403 means and that adoption was tried and refused — so the next reader does not re-derive it.

## v0.107.0 — F-REBOOT + F-LEAK: the agent's authority over guest lifecycle (2026-07-28)

Two Campaign 8 findings, both about the agent being unable to act on a guest it owns.

### F-REBOOT — a guest rebooted mid-backup never came back

Fault 11: the backup SUCCEEDED and the guest was found `stopped` with 0 containers, no lock, nothing
retrying — 9m47s of TOTAL appliance outage, every app down, and every alarm silent because nothing
was broken except that the box was off. The trigger is ordinary: backups run overnight and
`CONTEXT.md` records that the N100 needs BIOS AC-Power-Recovery *because power loss happens*.

**Why the existing recovery missed it.** `RecoverStaleLockedGuests` already does
unlock → delete dangling snapshot → **start iff onboot**, and it is correct. It missed this by two
narrow gaps: its predicate acts only on a guest holding a stale vzdump lock
(`backup`/`snapshot-delete`) and fault 11's guest was stopped and **unlocked**; and it runs **once at
agent startup**, on the load-bearing invariant that a lock present then is stale by definition. A
guest that goes down while the agent is already up was never re-examined.

**The fix** (`internal/localapi/guestpower.go`) closes exactly those two gaps: a periodic sweep that
acts on *should be running, is not, and is not locked*.

`onboot` is the "should be running" signal, and it is deliberately **not invented**: it is already
the distinction `stalelock.go` uses for this same decision, it is 1 on customer guests and 0 on
scratch/golden, and it is the flag `pve-guests` itself consults at host boot — so the agent agrees
with the platform instead of keeping a private definition. The hub's desired-state `Run` is stronger
but hub-dependent; `onboot` keeps working on a box that has lost hub contact, which is when an
unattended appliance most needs to come back.

**Guards, because the trap here is F-CRIT-1's shape.** A guest the operator deliberately stopped must
never be auto-started — fighting the operator makes maintenance impossible and is worse than the
outage. So: `onboot:0` ⇒ never touched; a LOCKED guest is left to the stale-lock path (its lock may
mean "a restore is writing my disks"); a guest with a vzdump genuinely in flight is left stopped (a
stop-mode backup stops it on purpose); unprovable ownership ⇒ act on nothing; and unconfirmable
backup state fails safe.

**Bounded retry** (Scenario C): 3 attempts at 1m/2m/4m, then ERROR naming the guest and stop. A
healthy `pct start` of 9201 took ~25 s (observed twice), so even the first wait carries 2.4x headroom;
three attempts bound disruption at ~7 minutes — inside the 9m47s this fixes — and never loop forever.

### F-LEAK — a failed restore-test could not destroy its own scratch

`DELETE /nodes/x/lxc/990000` → **403 missing privilege VM.Allocate**. It is **pool membership, not
privsep**, and the live ACLs prove it: `VM.Allocate` is granted at `/pool/felhom` ONLY, never at `/`.
A successful restore's `--pool felhom` makes the guest a member and inheritable; a FAILED restore
never completes that, so the guest's own path resolves to `/` where the token holds nothing:

| path | privileges the token has |
|---|---|
| `/vms/990005` (non-member scratch) | `Datastore.Audit, SDN.Use, Sys.Audit` — no VM.Allocate |
| `/pool/felhom` | full `FelhomAgentGuest` incl. `VM.Allocate` **and `Pool.Allocate`** |
| `/vms/9201` (pool member) | inherits the full set |

**The fix needs NO new privilege.** `Pool.Allocate` is already held, so the teardown adopts the
stranded scratch into the pool and retries the destroy once, which then authorizes via
`/pool/felhom`. Nothing is widened; `PoolAddVMID` already existed for exactly this reason
("membership is what lets the pool-scoped token reach the guest next time").

**The security guard is `scratchAdoptAllowed` — two independent checks, both required:** the journal
entry must carry agent-created **scratch provenance**, AND the VMID must be inside the configured
**scratch band**. Either alone would be enough to fix the leak; both are present because this
function is the only thing between "clean up my own scratch" and "co-opt an arbitrary guest into the
pool and delete it". Adopting a customer guest would hand the token destroy rights over it — far
worse than a leaked scratch.

### Also
Corrected the claim that the quiesce unquiesce is "guaranteed by defer"
(`felhom-controller/.../quiesce.go`, and the generic advice in `PROMPT-TEMPLATE.md`): fault 10
established that a SIGKILL runs no deferred function, and the guarantee is the crash marker plus
`Recover()`.

Files: `internal/localapi/{guestpower.go (new),server.go}`, `internal/reconcile/restoretest.go`,
`cmd/felhom-agent/main.go`, plus tests `internal/localapi/guestpower_test.go` and
`internal/reconcile/scratch_adopt_test.go`. No wire/contract change; no new PVE grant.

## v0.106.0 — F-CRIT-2: a failed backup must not look like a fresh one (2026-07-28)

Campaign 8 killed the PBS daemon mid-upload. PBS published the aborted upload into the storage
listing anyway — **1 byte, no manifest, and the NEWEST entry**. `NewestArchiveTime` counted it, so
the tier reported freshly backed up, went **not due**, and was never attempted again. On the real
168h offsite cadence that is **seven days of silence**, and neither backstop helps: the R-88 breaker
only defers tiers that are *due*, and the hub's deadline monitor reads the same freshness.

R-84 replaced remembered state with *"ask the storage, it is ground truth"*. That was right. The bug
is that **presence was taken for validity** — and a phantom is a more convincing lie than an empty
array, because absence at least reads as absence.

**The fix.** `NewestArchiveTime` now counts only entries that are *plausibly complete*, via a
measured size floor (`minPlausibleArchiveBytes` = 1 MiB). Undecidable ⇒ **not** counted: erring
toward "less fresh" costs one extra backup, whereas counting an undecidable entry is the defect.

**Why size and nothing else.** This runner is tier-agnostic — the same predicate runs against PBS
and against a plain `dir` storage. Verified against the live PVE API (2026-07-28):

| candidate | phantom | good PBS | good **local** | verdict |
|---|---|---|---|---|
| `verification` | absent | present | **absent** | unusable — would reject every local backup |
| `encrypted` | absent | present | **absent** | unusable — same |
| `notes` | absent | present | present | too fragile (agent-set only) |
| `size` | **1 B** | 4.35 GB | 1.59 GB | **robust, tier-agnostic** |

The floor is measured, not chosen: the smallest real backup anywhere on the fleet is **612,397,450 B**
(~584 MiB); 1 MiB sits 584x below it and 1,048,576x above the phantom. A test asserts that headroom
so nobody can quietly raise the floor into the thrash zone.

**The opposite risk is real and is a first-class test.** A filter that is too aggressive does not
merely lose safety — the tier reports absent every poll, backs up every cycle, and **R-88 cannot
save it because those backups succeed**. That is a continuous multi-GB write loop across the fleet.
`TestNewestArchiveTime_ValidSnapshotsAreStillCounted` is that guard, red-proofed by making the
filter reject everything.

**A rejected archive is never silent.** WARN, once per distinct volid (not per 5-minute poll —
~288 lines/day would bury it), naming the snapshot and the reason.

Also established while investigating, and worth recording: **server-side prune does NOT count a
phantom toward `keep-last`** — a dry-run with `keep-last 2` against three real snapshots plus a
phantom retained *two real ones plus the phantom*. So there is **no retention/data-loss bug**. But
prune never removes phantoms either, so they accumulate one per aborted upload, forever.

Files: `internal/backup/runner.go`, `internal/backup/archive_completeness_test.go` (new).
No wire/contract change; the controller needs no change — it consumes `age_state` and now simply
receives the truth.

## v0.105.0 — R-88 Part 2: the agent can finally say "unknown" (2026-07-27)

`newestArchiveOn` promised, in its own doc comment, that *"errors and unsupported services degrade to
unknown, never to 'no backup'"*. Its `(time.Time, bool)` signature made that **impossible**: an error
and a genuine not-found both returned `(zero, false)`, so `/backup/due` answered a POSITIVE
`"no successful backup recorded yet"` with a nil age. The controller read that as *never backed up*
and fired its window-gate safety valve — quiescing customer app stacks **outside** the backup window.
That is what happened on 2026-07-27 during the PBS outage. The comment described an intent the type
forbade.

**The fix: three states, on the wire.** `archiveLookup` (`found` / `absent` / `unknown`) internally,
and `age_state` on `/backup/due`:

- `known` — `age_seconds` is set and meaningful;
- `absent` — a POSITIVE determination that no backup has ever landed. **The only state that licenses
  the controller to bypass its backup window;**
- `unknown` — could not determine (storage unreadable, or an unparseable timestamp).

**A string enum, not a bool.** The zero value must mean *"legacy agent, no information"*, and `""`
says that unambiguously where `false` would masquerade as a real answer. It also matches the repo's
existing wire convention (`phase` on `/backup/status`).

**The fail-safe direction is unchanged: unknown is still DUE.** An agent that cannot read the storage
must never suppress a backup. What changes is only whether the *window gate* may be bypassed.

**An unparseable in-memory timestamp is now `unknown` too** — a backup DID happen, we simply cannot
date it. It previously fell through to the same positive "never" claim.

**A service with NO lister stays `absent`, deliberately.** "Unknown" is the tempting answer there and
it would regress the first-backup safety valve: a genuinely new box on a pre-R-84 build would never
back up outside its window and nobody would notice for weeks. On that path the in-memory record is
the only registry that exists, so its absence means "no backup recorded" in the only terms available.
`unknown` is reserved for a lister that was asked and could not answer.

**Compatibility:** the field is additive. An OLD controller decodes into a struct without it and
behaves exactly as today — every pre-existing field is unchanged (pinned by
`TestAgeState_IsAdditive_PreExistingFieldsUnchanged`).

Tests +5; 29 packages ok. Red-proofs observed for the unknown/absent collapse in both directions.
## v0.104.0 — R-85: unattended per-tier restore-test (Phase 2): tier rotation, persisted state, one heavy op at a time (2026-07-26)

The scheduler could only ever see `cfg.Backup.BackupTarget()`, so the offsite tier's archives were
never candidates. That is why demo-hp's DR tier reported `applied` with **zero snapshots for five
days** and nobody noticed.

### Selection — oldest-first (operator ruling 2026-07-26, Option 1)
The tier whose last **successful** restore-test is oldest goes first; **never-proven sorts first of
all**, which is exactly where the offsite tier starts. Self-balancing, no new config knob, and each
tier is covered every ~2 cadences — comfortably inside the 2-week offsite retention, so a tier is
never proven against an archive that is about to be pruned. Ties break on target id, because two
tiers proven in the same second would otherwise rotate by Go's randomised map order: untestable, and
occasionally starving.

Rotation credit is given **only on success**. A tier that fails every time must keep sorting first —
otherwise a permanently broken tier would look freshly proven and quietly stop being retried.

### Added
- **`backup.RestoreTestState`** — last successful restore-test per tier, persisted (atomic
  tmp+rename). **This one genuinely needs persistence, unlike R-84**, and the difference is worth
  stating because they look alike: R-84 had a GROUND TRUTH to consult (the archive is still on the
  storage), so it never persisted anything. A restore-test destroys its scratch as its final act and
  leaves no artifact — "did we prove this tier restores?" exists only as remembered state. A corrupt
  or missing file degrades to "nothing proven", which is the correct starting point.
- **`backup.InFlight`** — the host-wide one-heavy-operation gate, shared by the restore-test
  scheduler and the local-API backup path. Not a lock concern (the scratch VMID never touches the
  live guest's vzdump lock) but a **LINK** concern: an offsite restore PULLS multi-GB over the same
  tunnel an offsite backup PUSHES one. At the ~33 MB/min measured upstream, running both drives each
  toward its timeout — which is how a healthy tier ends up recorded as failed. A caller that cannot
  acquire **defers**; it never cancels what is already running.
- **`BackupRunner.PickRestoreCandidateOn`** — newest archive on a NAMED tier. `""` + nil error when
  that tier holds none: **a tier with nothing to restore is not an error**, or every fresh box would
  look broken for its first week.

### Changed
- A tier with no archive is **skipped and the next tier tried**, not left to burn the cadence. It
  cannot starve either — an empty tier is still the least recently proven, so it still sorts first
  the moment it has an archive.
- `POST /backup` now also joins the gate: a **409** naming the holder when a restore-test is running.

### Tests
+12. Red-proofs observed:
- **A** — the single-target picker yields `both tiers must be exercised across 4 cadences; got [local:… local:… local:… local:…]`.
- **E** — losing the state file yields `after a restart the OTHER tier must be next; got … twice (rotation state was lost)`.
- **F** — removing the gate yields `the restore-test must DEFER while a backup holds the gate; concurrent operations = 2`. **The count is the assertion** — "both completed" would pass against a fully concurrent implementation.

Full suite green (29 packages, `rc=0`, vet unpiped).

## v0.104.0 (cont.) — R-85 Phase 1: the restore-test spec is built PER RUN (2026-07-26)

Prerequisite for scheduling the offsite tier at all. Shipped on its own because it is independently
correct and independently testable.

`SchedulerOptions.Spec` was a **value**, produced by an immediately-invoked function at daemon start
(`main.go`). So `storageTier()` and `restoreTaskTimeout()` were evaluated **once** and their result
reused for every run for the lifetime of the process. Two consequences:

- **Nothing tier-varying was expressible.** The offsite tier could never be scheduled, because the
  spec's tier was fixed to whatever the configured target was at boot.
- **A latent staleness bug in its own right:** a storage-type or config change did not take effect
  until the daemon restarted.

### Changed
- **`backup.SpecBuilder`** — `func(ctx, archive) reconcile.RestoreTestSpec`, called **once per run**.
  The archive is passed in because the tier MUST come from it (`restoreTierForArchive`, the v0.100.0
  rule). Deriving it from the configured target is what classified a PBS archive as `local` and
  killed a 14.46 GB WAN restore at the 10-minute local bound.
- **A nil spec builder SKIPS loudly instead of panicking.** `Run` already refused to start without
  one, but `tick` is reachable directly; a wiring bug must cost a restore-test, never the daemon
  goroutine.

### Tests
+3. Red-proof observed: restoring the frozen value fails with
`the spec builder must run ONCE PER RUN, got 0 call(s) across 3 ticks`. Full suite green
(29 packages, `rc=0`, vet unpiped).

## v0.103.0 — R-84: an agent restart no longer triggers a redundant backup (2026-07-26)

**Observed live, not theorised.** Three redundant local backups ran on demo-felhom in a single
afternoon of deploys (2026-07-26). The backup `Store` is in-memory — its own doc comment says *"lost
on restart; the cadence re-populates"* — so after every restart `/backup/due` answered *"no
successful backup recorded yet"* and the controller dutifully took another one.

On the local tier that is wasted minutes. **On the OFFSITE tier it is a wasted multi-hour WAN upload
after every agent deploy** — and agent deploys are routine. That is what moved R-84 from a tidy-up to
something close to a prerequisite for running the offsite tier at all.

### Changed
- **`BackupRunner.NewestArchiveTime`** — when a backup last LANDED on this tier's storage, read from
  the storage.
- **`localapi.BackupArchiveLister`** (optional extension to `BackupService`) — the due-check consults
  it and takes whichever is newer, the in-memory record or the storage.

### Why ask the storage instead of persisting the store

- **It is ground truth, not remembered state.** A pruned or deleted archive correctly stops counting;
  a persisted record would keep claiming a backup that no longer exists.
- **No new on-disk state, no migration**, and it is the same source `latestArchive` already trusts to
  build the post-backup record.
- It answers only *"when did a backup last land"*, which is exactly what the due-check needs. The
  richer fields (size, duration, uncovered volumes, error) stay with the real in-memory records — a
  synthesized record would put invented numbers into the host-report, and the hub already covers
  history via its own retained-report window (R-81).

### Fail-safe directions
- Storage read **error** → fall back to the in-memory record. An unreadable storage must never make a
  tier look freshly backed up, and must not suppress a backup either.
- Storage genuinely **empty** → due. The fix must not invent a backup.
- Archive present but **old** → still due. This is not a blanket suppressor.
- Service **without** the optional lister → pre-R-84 behaviour, unchanged.

### Tests
+6, full suite green (29 packages). Red-proof observed: removing the fold-in reproduces
`Due:true Reason:no successful backup recorded yet` against a storage holding a 2 h-old backup.

## v0.102.0 — R-82 Slice D: an unprovisioned tier DEFERS instead of failing (2026-07-26)

Prerequisite for putting the offsite tier into the installer defaults (host-install 1.20.0).

A fresh box now carries the offsite tier in its config, but `felhom-pbs` does not exist until the hub
provisions the DR tier (`felhom-pbs-apply`). Without this change the tier would report **due** in
that window, so the controller would quiesce the apps and fire a vzdump at a storage that is not
there — **every cadence, until provisioning happens**.

### Changed
- **`GET /backup/due?target=…` DEFERS when the target storage is absent**
  (`Server.targetStoragePresent`): `due:false`, reason *"target storage not present yet — tier
  deferred until it is provisioned"*. The tier stays silent until it is real and goes live with **no
  restart** the moment the storage appears.

**Fail-safe, and it is the point:** a storage-view ERROR returns `present` and the tier stays due.
*"I could not check"* must never be read as *"not there"* — reading it that way would silently
suppress backups, which is the absence-is-not-failure rule this project has now relearned three
times (R-80, R-81, and the R-82 wait-timeout). Pinned by
`TestBackupDue_StorageViewError_DoesNotSuppress`.

### Tests
`TestBackupDue_TargetStorageMissing_Defers` (deferral + a reason that says WHY, so it is
distinguishable from a healthy tier) and the fail-safe above. Full suite green.

## v0.101.0 — R-82: a leaked restore-test scratch can no longer auto-start (2026-07-26)

**Correction first, because it matters more than the fix.** I reported earlier in this arc that the
restore-test would boot a scratch guest carrying the live guest's MAC, static island IP and
hostname, and so would break the controller→agent link. **That was WRONG.** `RunRestoreTest` step 2
link-downs **every** interface (`withLinkDown`, unit-tested) BEFORE the guest is ever started, so on
the normal path there is no L2/IP conflict. The design already handled it.

What is real is narrower. A restore that fails **before** step 2 — exactly what the v0.100.0 wait
bug caused — leaves a scratch guest holding the SOURCE guest's config verbatim, including
`onboot: 1`. If teardown then also fails (it did: `403 … missing privilege VM.Allocate`, because PVE
associates the pool only at restore COMPLETION), the leaked guest survives and **a host reboot would
start it** alongside the original, NICs up, same MAC, same `169.254.253.2/30`.

So the hazard needed three things to line up, and it did, once, on demo-felhom.

### Changed
- **`proxmox.RestoreLXCOptions.ConfigOverrides`** — arbitrary guest-config params applied AT RESTORE
  TIME, for settings that must hold from the instant the guest exists.
- **The restore-test passes `onboot=0`.** At restore time, not after: "after" is precisely the path
  that leaks. A leaked scratch is now inert across a host reboot even with its NICs still up.

### NOT changed
- The link-down step. It was already correct and is the primary defence; this is depth behind it.
- The agent's Proxmox privileges. It still cannot tear down a scratch until the restore completes.
  Widening `VM.Allocate` to `/vms` would remove the accidental guard that stopped a destructive
  mid-restore teardown — the wrong trade. With the v0.100.0 timeout fix the teardown no longer fires
  mid-restore.

### Operational
`restore_test_cadence_seconds` was set to `-1` on demo-felhom as a stopgap under the mistaken
reading above. **Re-enabled** — the scheduled restore-test is safe and always was.

### Tests
`TestRestoreTest_RestoreSetsOnbootZero`; red-proof observed (removing the override yields
`got "" (map[string]string(nil))`). Full suite green (29 packages).

## v0.100.0 — R-82: the restore tier comes from the ARCHIVE, not the configured target (2026-07-26)

**Found by the first real PBS restore round-trip (2026-07-26), not by review.** Restoring
`felhom-pbs:backup/ct/9201/2026-07-26T12:21:48Z` on a box whose primary target is `"local"` failed
after exactly **600.76 s** — the 10-minute local wait — against a 14.46 GB WAN restore that needed
roughly two hours.

`--selftest=restore-test` derived its tier with `storageTier(ctx, px, cfg.Backup.BackupTarget())` —
the **configured default target** — and then `restoreTaskTimeout` correctly returned the local
(10 m) bound for it. The recorded result even said `"source_tier": "local"` for a PBS archive.

**The tier-aware machinery was already right; it was fed the wrong input.** `RestoreTaskTimeout`,
the generous `RestoreTestPBSRestoreTimeout` (120 m), and the pool-association handling all existed
and all worked. What broke is an assumption that stopped being true the moment a second tier
existed: *"the configured target"* is no longer a proxy for *"the tier this archive belongs to"*.

`RestoreTestSpec.RestoreTaskTimeout`'s own doc comment predicts the consequence exactly:

> *"…else the wait expires mid-restore, teardown fires against a still-restoring
> (not-yet-pool-associated) guest, and the scratch leaks."*

Which is what happened: the teardown fired at a live restore and was refused —
`HTTP 403 … missing privilege VM.Allocate` — because PVE associates the pool only when the restore
COMPLETES, and the agent's `VM.Allocate` is granted on `/pool/felhom`, not on `/vms`. **The 403 was
load-bearing luck:** it is the only reason a destructive teardown did not run against a
half-restored guest. The restore itself carried on to completion, unharmed.

### Changed
- **`restoreTierForArchive(ctx, px, archive, fallbackTarget)`** — derives the tier from the
  ARCHIVE'S OWN storage (`archiveStorageID` parses the volid prefix), falling back to the configured
  target only when the volid carries no prefix. Wired into the `--selftest=restore-test` path.

### NOT changed (recorded, not fixed here)
- **The daemon's scheduled restore-test still only covers the PRIMARY tier.** Its `Pick` uses a
  runner built on `BackupTarget()`, so it never selects a PBS archive, and its `Spec` is built once
  at construction rather than per tick. That is self-consistent today (tier matches archive) but it
  means **the offsite tier is never automatically restore-tested** — a real R-82 gap, and arguably
  the more important half of "is the DR tier real?". Needs a per-tick spec; own task.
- **The agent cannot tear down a scratch guest until its restore completes** (no `VM.Allocate` on
  `/vms`; the pool association lands only at completion). With the correct timeout the teardown no
  longer fires mid-restore, so this is latent again — but it is not fixed, and widening the token's
  privileges is deliberately NOT the answer.

### Tests
`TestArchiveStorageID` pins the volid parsing incl. the `i>0` guard (a leading colon is not a
storage id). Full suite green (29 packages).

## v0.99.0 — R-82: operator rulings — 2-week offsite retention + one backup at a time (2026-07-26)

Implements two operator rulings of 2026-07-26. Both are behaviour changes on the multi-tier path
only; a single-tier config is untouched.

### Ruling 1 — keep two weeks of weekly offsite backups
`localPruneSpec` refused to prune ANY PBS target. That blanket refusal is now scoped: an
**ADDITIONAL tier with an explicitly configured `keep_last`** may prune its PBS target
(`NewBackupRunnerFull(..., allowPBSPrune)`).

**The refusal still applies in full to the PRIMARY tier, and that is not caution for its own sake:**
`BackupTarget()` defaults to `"felhom-pbs"` and `KeepLast()` defaults to 3, so a box with neither key
set would silently prune its offsite DR down to 3 restore points. An additional tier cannot have that
accident — its `keep_last` defaults to 0 (never prune), so any value there is a deliberate act.

### Ruling 3 — let the first backup run as long as it needs; nothing else starts until it is done
- **Wait bound for an additional tier raised 6h → 12h.** Measured on demo-felhom: ~33 MB/min over
  the wg link, so a first FULL ~10 GB snapshot projects to ~5h. 12h gives real margin on a slower
  link while staying BOUNDED — a genuinely hung task must still surface eventually.
- **ONE BACKUP AT A TIME PER GUEST, ACROSS ALL TIERS.** `POST /backup` now refuses with **409** when
  a DIFFERENT tier has a backup in flight, naming the busy tier and job. vzdump holds a guest lock,
  so a concurrent second backup could not succeed anyway — but without this guard it would be
  ATTEMPTED, fail on the lock, and record a spurious failure that leaves the tier permanently due.
  - SAME tier in flight → still returns THAT job (202, idempotent) — unchanged.
  - DIFFERENT tier in flight → 409 with **no data object**, so nothing is parseable as the caller's
    own job. Handing back a foreign job id is precisely how a caller comes to believe its backup ran.
  - **`snapshotted` now counts as in flight**, not just `running`. After the storage snapshot the
    vzdump is still uploading and still holds the lock; the pre-R-82 check looked at `running` only,
    leaving a window where a second POST started a real second vzdump. Latent bug, closed here.

### Tests
Full suite green (29 packages). `TestBackupPost_SecondTierRefusedWhileAnotherInFlight`,
`…_SecondTierAllowedAfterFirstFinishes`, `…_SnapshottedCountsAsInFlight`, and the per-tier wait-bound
test updated to 12h. Red-proof for the wait bound observed and restored.

## v0.98.0 — R-82 Slice A fix: per-tier vzdump wait bound (the 30-minute false failure) (2026-07-26)

**Found by live validation on demo-felhom, not by review.** The first real PBS-targeted backup ran
past the runner's hard-coded 30-minute `WaitTask` bound. The agent stopped waiting, recorded
`success:false` — **while the vzdump kept running** (still running 72 minutes later, 2.4 GB
uploaded). That is not "the backup didn't happen"; it is worse:

- the tier stays permanently **due** (a failed backup never satisfies a cadence),
- the next attempt collides with the **guest lock** the live vzdump still holds,
- the hub sees a DR tier that never succeeds — R-82's "applied and empty" fault, re-created by a
  timeout,
- the recorded `Backup` says `success:false, size_bytes:0` for a backup that may yet complete.

30 minutes is right for a LOCAL vzdump (minutes) and simply wrong for an offsite upload. Measured on
demo-felhom: ~33 MB/min over the wg link to Hetzner ⇒ a first FULL ~10 GB snapshot projects to ≈5 h.

### Changed
- **`BackupTargetConfig.WaitTimeoutSeconds`** — per-tier vzdump wait bound. Defaults:
  **primary 30 m (UNCHANGED)**, additional tier **6 h**. The asymmetry is the point: the primary is
  the local tier where a 30-minute hang IS a genuine fault worth surfacing; an additional tier is by
  construction the offsite one, where the binding constraint is uplink speed, not health. 6 h is
  sized from the measurement above, not guessed.
- **`backup.NewBackupRunnerWithWait`** — the runner's wait bound is per-instance (i.e. per tier).
  `NewBackupRunner` keeps its signature and delegates with 0 ⇒ 30 m, so every other caller
  (restore-test, selftest) is untouched.
- **`localapi.BackupTier.WaitTimeout`** — the fire-and-forget backup context is now sized from the
  tier instead of a fixed 2 h. **Both bounds had to move**: a 6 h runner bound under a 2 h outer
  context would have reproduced the same false failure four hours later.

Same reasoning, and the same direction, as `restore_test_pbs_restore_timeout_seconds` on the restore
side: **when in doubt wait LONGER.** A slow backup is a slow backup; a false timeout is a corrupt
status plus lock contention.

### Tests
`TestBackupTiers_WaitTimeoutIsPerTier` pins the asymmetric defaults, the override, and that the two
tiers do NOT share one bound. Red-proof observed: setting the extra tier's default back to 30 m
fails with `an offsite tier must default to a GENEROUS wait — a false timeout is worse than a slow
pass; got 30m0s`. Restored; full suite green.

## v0.97.0 — R-82 Slice A: per-target backup tiers (local daily + PBS weekly) (2026-07-26)

Additive; **MinAgent floor rises** for the multi-tier contract (a controller that wants per-tier
backups needs this agent — but see the compatibility rule: an OLD controller is unaffected).

Slice A of R-82. `BackupTarget()` returned ONE string and `BackupCadence()` ONE 24h window, so
"local daily **and** PBS weekly" was not expressible at all — which is why the DR promise is
currently unbacked (demo-felhom holds one PBS snapshot from 2026-07-18, demo-hp zero, ever).
Phase-0 gate results: `felhom.eu/documentation/audits/SPIKE-r82-phase0-2026-07-26.md`.

**This slice ships the mechanism only. No box's behaviour changes until a `backup_targets` entry is
added to its config (Slice D).** An untouched config resolves to exactly one tier and behaves
byte-identically to v0.96.0.

### The compatibility rule (load-bearing)

The agent and controller deploy independently, so the untargeted local-API contract is **frozen**:

- `GET /backup/due` with **no** `?target=` → the PRIMARY tier, same cadence, same response **bytes**.
  `BackupDueResponse.Target` is `omitempty` and left EMPTY for untargeted requests, so an old
  controller cannot tell this agent from the old one. Pinned by a red-proofed test.
- Same for `POST /backup` and `GET /backup/status`.
- The primary tier's **job-id format is unchanged**; only additive tiers carry a target segment.

### Added
- **`config.BackupTargetConfig` + `BackupConfig.ExtraTargets`** (`backup_targets`) — each tier
  carries its OWN cadence and its OWN retention. Those are semantically different per tier:
  `keep-last=3` is three DAYS on a daily tier and three WEEKS on a weekly one, so sharing one knob
  guarantees one of them is wrong.
- **`BackupConfig.BackupTiers() ([]BackupTier, []string)`** — resolves the tier list, primary first,
  plus warnings the caller MUST log. A tier is **rejected, not defaulted**, when its cadence is
  missing: silently defaulting a weekly DR tier to the 24h local default would fill the 37.2 GB
  datastore. Empty/duplicate targets are rejected too. `main.go` logs every rejection at **ERROR** —
  a silently dropped backup tier is the "applied and empty" fault this task exists to fix.
- **`GET /backup/tiers`** — advertises the tiers, primary first. This is the controller's capability
  probe: a **404 means a pre-R-82 agent**, and Slice B falls back to single-tier on it.
- **`localapi.BackupTier` + `normalizeBackupTiers`** — nil tiers synthesize the legacy single tier,
  so every existing caller and test hits the pre-R-82 path untouched. A tier with a nil runner is
  DROPPED rather than advertised (advertising one would be an applied-and-empty tier).

### Changed
- **`/backup/due?target=` judges that tier against ITS OWN newest successful backup**
  (`latestSuccessfulBackupForTarget`). Without this filter a fresh daily local backup would satisfy
  the weekly PBS cadence and the DR tier would never run — today's bug, re-created in code. The
  store was already keyed by target, so this is a lookup change, not a data-model change.
- **Backup jobs are keyed by (vmid, target), not vmid.** Single-flight is now PER TIER, which is
  what lets the weekly night run both backups inside ONE quiesce window (Slice B). Keying by vmid
  alone handed the second caller the first tier's job id — **caught by its own test**, and it would
  have made the controller believe a PBS backup ran when only the local one had.
- **Job ids are unique per tier by construction**, not by clock luck (two tiers can start in the
  same nanosecond). The primary keeps the old format; additive tiers carry the target segment.
- **One runner per tier** (`main.go`). The runner holds target/mode/notes/retention as immutable
  construction state and `localPruneSpec` reads that retention — parameterising a single runner by
  target would risk pairing tier A's target with tier B's retention.
- An unknown `?target=` is a **400, never a silent fallback to the primary**. A controller asking
  about a tier this agent does not serve must find out, not act on another tier's freshness.

### NOT changed (deliberate)
- **The local tier.** Same target, same 24h cadence, same keep-last=3 clamp. It is the only honest
  tier today and this slice does not touch it.
- **PBS is still never pruned by the per-run `--prune-backups` flag** (`localPruneSpec`). Per-tier
  retention is plumbed and a tier's `keep_last` defaults to 0 = never prune, but turning on
  automatic pruning of the DR datastore is irreversible and needs an operator ruling — R-82 Phase 0
  already flagged retention as needing one. Recorded, not decided here.
- The fail-safe-toward-due rule on an unparseable timestamp. A spurious backup is cheap; a skipped
  one is not.

### Tests
768 (was 748), +20 across `internal/localapi/backup_tiers_test.go` and
`internal/config/backup_tiers_test.go`. Red-proof #1 (old controller ↔ new agent) observed:
removing the untargeted compat branch makes the untargeted body carry `"target":"local"` and the
test fails on it. Restored.

## v0.96.0 — R-50 island NIC: provision attaches the guest's island net1 (2026-07-25)

Additive; **MinAgent unchanged** (no controller coupling — the controller dials whatever `bootstrap.json`
says, and the pin is address-independent). Implements the provisioning half of the R-50 island control
plane, spiked GO in `felhom.eu/documentation/audits/SPIKE-island-bridge-2026-07-25.md`. A fresh install
is now born immune to F1 (`AUDIT-vacation-remote-ops-2026-07-20`: a LAN/DHCP move made the agent fail to
bind → storage/PBS/quiesce/restore-test/DR down, silently).

- **`LocalAPIConfig`** gains `island_bridge` + `island_guest_addr` (config.go). `IslandEnabled()` = both
  set. `Validate()` enforces all-or-nothing + a CIDR guest addr — a half-set/malformed island fails at
  load (a botched install), never a silent LAN fallback that would leave an island bind with no island NIC.
- **`buildBringUpConfig`** (reconcile/bringup.go): when both island fields are set, attaches a static
  `net1=name=eth1,bridge=<vmbr9>,ip=<169.254.253.2/30>` (no hwaddr → fresh per-guest MAC) on BOTH
  provision and DR bring-up. Empty = pre-R-50, no net1 (byte-for-byte the old config on non-island hosts).
  Plumbed from `cfg.LocalAPI` at both `RunBringUp` call sites (cmd/main.go). The bootstrap `endpoint`
  already derives from `listen_addr` (main.go), so moving the agent bind to the island moves the guest
  dial for free — **no template change** (A0 determination).
- **Design note (A0/A3):** endpoint is config-derived, so no code was needed there; the guestnet healer
  is eth0-only (`parseMode` is dev-scoped) so the static island `eth1` is outside its scope — a red-proof
  test locks that in (`TestParseMode_IslandStaticNICDoesNotConfuseEth0`) rather than changing the healer.
- Tests (all non-hollow, red-proofed): `TestBuildBringUpConfig_IslandNIC`,
  `TestLocalAPIConfig_IslandValidation`, plus the healer scoping test above.
- **Coupling (deploy order):** a host-install that writes the island config REQUIRES agent ≥ 0.96.0 to
  read `island_guest_addr` and attach net1 — vouch 0.96.0 before island installs go live. Migration of
  existing boxes is the Phase-B runbook (`felhom.eu/documentation/runbooks/RUNBOOK-island-migration.md`).

## v0.95.0 — SMART coverage: union-path drives + LVM/dm root + device model (2026-07-25)

Additive; **MinAgent unchanged**; hub untouched (unknown JSON fields ignored). Implements the graded
fixes from `felhom.eu/documentation/audits/SPIKE-smart-coverage-2026-07-25.md`, which proved both demo
disks answer the allowlisted `smartctl -a -j` with PASSED but the agent never asked.

- **Fix B — union-path SMART:** registry/USB drives ride the `/disks` union path (`driveTargets.Known`),
  which skips Observe's `enrich`, so they showed "Nincs adat" despite working SMART. New
  `storage.SmartReader` (`SMARTForBacking`, reuses `smartDeviceFor`) is wired into the union path via a
  localapi seam — the USB drive now reports its real verdict. The watchdog `Known` path stays
  enrich-free (asserted: zero smartctl calls).
- **Fix A — LVM/device-mapper resolution:** `smartDeviceFor` gains a dm branch that resolves
  `/dev/dm-N` / `/dev/mapper/X` to the single backing whole disk via `/sys/block/<dm>/slaves`
  (recursive; **skips** rather than guesses when slaves span >1 physical disk). And the builtin `local`
  dir on the LVM root — whose `backing_device` is empty by design (removable-safety) — now gets a
  **SMART-only** device resolved from its containing filesystem (mount table), never touching
  `backing_device`/`durable_id`. The system SSD stops reading "Nincs adat".
- **Device model:** `SmartSummary.ModelName` captured from smartctl's own `model_name` (already parsed),
  so the controller card can label a disk "TOSHIBA MQ04ABF100" instead of a raw UUID.
- Fix C (`-d sat`) stays rejected (disproven live; absent from sudoers). No sudoers/manifest change.
  Consumed by controller v0.171.0. Tests: dm-resolution table (+multi-disk-skip red-proof), containing-fs
  resolution (+red-proof), union-path SMART (+red-proof), model capture, Known-path-never-SMARTs.

## v0.94.0 — serialize per-disk SMART into the /disks payload (2026-07-24)

Additive, backward-compatible; **MinAgent floor unchanged** (the controller feature-detects by payload
presence). No new smartctl load, no new endpoint, no sudoers change — the SMART is already computed on
the request path (`storage.Observe` → `enrich` runs `smartctl -a -j` for dir-backed targets); this
release simply copies the target's already-populated `Smart` into `localapi.DiskInfo`.

- `DiskInfo` gains `Smart *hub.SmartSummary \`json:"smart,omitempty"\``. In `handleDisks` the summary is
  copied **only when `t.Smart.Health != ""`** — a zero-value summary (SMART never read: no smartctl
  device, USB bridge, or a failed read) stays omitted, so the controller sees *absent* and renders
  "Nincs adat" rather than a misleading UNKNOWN. The union-in drives (driveTargets.Known path, no
  enrichment) carry no SMART and are omitted by the same guard.
- The controller (v0.169.0) consumes this to render a "Lemezek állapota" card + a 6-hourly
  degradation notification. Old controllers ignore the extra field.
- Test `TestDisks_SmartSerialized` (payload includes SATA counters + temperature for a fixture target;
  absent-SMART target omits the field); red-proof: drop the copy → the serialized-Smart assertion fails.

## v0.93.0 — a recovery code can no longer contain a hyphenated word (2026-07-21)

> **SHIPPED 2026-07-22:** built + published (sha256 `a68b2ff73200622e…`), Day-0-manifest-vouched,
> deployed to both fleet boxes (felhom-pve + demo-hp), clean-restart verified. Record:
> `felhom.eu/documentation/pilot/RUNBOOK-publish-agent-0.93-2026-07-22.md`.

**Generation-only change. Every recovery code already issued remains valid** — R is consumed as a
whole passphrase by the PBS scrypt KDF (`Wrap`/`Unwrap`) and is never re-split, so nothing about
verification moves. Nothing in the KDF/consume path, the word count, or the joiner changed.

The EFF large wordlist contains exactly four entries that themselves contain the hyphen we join
words with: `drop-down`, `felt-tip`, `t-shirt`, `yo-yo`. Drawing one produced a code that reads as
11 words rather than 10 — ambiguous to transcribe in exactly the situation R exists for, a customer
reading a code back during a disaster. The generator now draws from the list filtered of those four
(`joinSafe`), so a code always segments back into exactly `RecoveryCodeWords`.

- **Entropy floor holds, with the numbers asserted in the tests:** the draw space goes 7776 → 7772,
  so a 10-word code goes 129.248 → 129.241 bits. The cost is 0.007 bits against a 128-bit floor.
- **The long-standing ~1/5 test flake was this defect, not a flaky test.**
  `TestGenerateRecoveryCode_EntropyAndFormat` counted words by splitting the joined string, which
  conflates "how many words were drawn" with "how many segments the code has". It now counts what
  the generator drew, and asserts the segmentation property separately — the property `joinSafe`
  actually buys. (`felhom.eu/REPORT.md` §6 item 3 recorded it at 3/8 in one session.)
- **Deterministic red-proof, in-tree:** `TestGeneratedCodeSegments_FilteredVsUnfiltered` drives the
  generator against a fixture list where every word is hyphenated, so the pre-fix defect reproduces
  with probability 1 instead of ~1/5, and shows the same list through `joinSafe` refuses to generate.
- New for audit: `WordlistFilteredOut()`, `RecoveryCodeSep`. `WordlistSize()` now reports the
  effective (filtered) draw space, 7772.

## v0.92.1 — ship the guestnet sudoers grant with the binary (supersedes v0.92.0) (2026-07-21)

**Supersedes v0.92.0; that artifact is materially incomplete — do not vouch it.** It was published
before live verification revealed the watchdog had no sudoers grant for three of its four probes,
so it carries neither the `FELHOM_GUESTNET` alias nor the `guestnet-*` capability rows that make a
host missing that alias visible. Left in place rather than overwritten — a published version stays
immutable (the v0.91.0 → v0.91.1 precedent).

No behaviour change beyond the capability rows; the watchdog code is byte-identical to v0.92.0. The
functional fix is `configs/felhom-agent.sudoers`, which **must be deployed with the binary**.

## v0.92.0 — the guest network gets a watchdog (R-54) (2026-07-21)

**Host-tier only — no controller coupling, no wire change the hub must understand today** (the
`guest_net` stanza is additive and stored opaquely, exactly like `pbs_dr` and `wireguard`).

Closes the OPEN RISK left by `INCIDENT-guest-dhclient-killed-2026-07-20` §5: **the guest's DHCP
client is started once by ifupdown at boot and nothing supervises it.** When it died on 2026-07-20
the guest kept working for another ~80 minutes on its unexpired lease; only when the lease expired
did the address and the default route vanish, taking the Cloudflare tunnel, the hub reports, the
catalog sync and the controller→agent channel with them — a 1h15m outage in which every observable
signal said healthy for the first 80 minutes.

**The design consequence, and the point of the whole package: liveness of the DHCP client is itself
a probe.** Waiting for the address to disappear is waiting out precisely that silent window. The
watchdog therefore flags a DHCP guest unhealthy on `pgrep -x dhclient` alone, while the lease is
still live and everything else still looks perfect.

`internal/guestnet`, built on the wg-tunnel/storage watchdog loop shape:

- **Probes** (four fixed-shape `pct exec` argvs, no shell anywhere, no guest data interpolated):
  address, default route, `/etc/network/interfaces` mode, dhclient liveness. Parsers are pinned to
  output captured live from guest 9201 on 2026-07-21 — including the literal backslash `ip -o`
  emits and the docker-bridge routes that must not read as a default route.
- **Heals** with the incident's restored invocation, verbatim, logged at INFO before it runs:
  `pct exec <vmid> -- dhclient -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases eth0`
- **Dampers**, because this runs a privileged command inside a customer's container: two CONSECUTIVE
  bad probes before any heal (one blip is not a diagnosis), ≥10 min between heals per guest, ≤3
  heals/hour, and an observe-only window while the guest (or the agent) has been up under 3 minutes.
- **Refuses to act** on a static guest (dhclient must never fight a static config — a static guest
  missing its address is reported loudly and left to R-50), on an unknown interface mode, on a guest
  it cannot probe, and when the guest list cannot be ownership-proven. The guest source is the
  pool-verified one (`ListLXC` ∩ the felhom pool, audit A1) — never a bare `ListLXC`, which under a
  broad token would run dhclient inside a co-tenant's container.
- **A failed PROBE never reads as a dead client.** `pgrep` exits 1 with empty stderr when there is
  no match; anything on stderr means the probe itself failed, which reads as unknown. Otherwise a
  missing `pgrep` would heal forever.
- **Healthy cycles log a Debug line** (v0.91.2's lesson, one day old): if the quiet path is silent,
  "no alarms" and "never probed" are the same evidence, and an inert watchdog is indistinguishable
  from a working one.
- **Deliberately NOT in the `errc` fan-out** — a watchdog over customer guests must never be able to
  terminate the agent. A test asserts that, because joining the fan-out would also make the shutdown
  drain bound off by one.

**Config `guest_net` is this repo's first DEFAULT-ON feature gate, and the inversion is deliberate.**
Every other gate defaults to false because those features reach outward (an offsite endpoint, an OOB
tunnel) and enrolling a box by an update would be wrong. This one looks only INWARD at guests the
agent already owns, and the failure it prevents exists on every box today. A watchdog that must be
remembered per box is a watchdog that is missing on the box that needed it. Opting out is the
explicit act: `"guest_net": {"disable": true}`.

**Naming deviation from the spec, deliberate:** TASK-D called the report block `WireGuestNet`. In
this repo `Wire*` is the DOWN direction (`WireDesiredState` / `WirePBSDR` — what the hub sends), and
UP-direction report stanzas are `*Status`. It ships as `GuestNetStatus` so it is not the one report
block named against the convention.

- Wiring is asserted from `package main` by an AST walk (construct + `SetGuestNetReporter` + the
  started goroutine) — the v0.91.0 defect was exactly a seam whose caller was never written, with
  every unit test green. Red-proof: un-wiring both lines fails the test with both reasons named.
- Red-proof for the detection itself: reverting `classify` to IP-presence-only makes the July-20
  fixture report **"healthy"** and records **zero** heals — the 80-minute silent window, reproduced.
- **Ships a sudoers change** (`configs/felhom-agent.sudoers` MUST be deployed with the binary).
  TASK-D assumed none was needed; live verification on felhom-pve proved otherwise — the first sweep
  logged `dhclient liveness probe failed: sudo: a password is required` and correctly reported
  `state=unknown` rather than acting blind. The existing grant covered only lanresolver's address
  read. `FELHOM_GUESTNET` adds four fixed vectors (route, interfaces, pgrep, and the heal), every
  argument after the numeric vmid a literal, so no hub or guest input can widen it. The address read
  is not duplicated — it stays FELHOM_DNSMASQ's.
- Four `guestnet-*` capability rows so a host missing that sudoers file is VISIBLE as degraded
  rather than silently watchdog-less. Non-critical on purpose: a missing grant must not page an
  operator for every box on rollout day (the R-50b amber-fleet lesson).
- `var version` in main.go was stale at `0.89.0` (three releases behind); builds set it via ldflags,
  but `go run` and any forgotten `-X` reported a version that had not existed for days.

## v0.91.2 — a healthy credential probe is observable (2026-07-21)

The probe logged only on failure, so a healthy one was silent — which makes "no `auth_failed`"
indistinguishable from "never probed", and leaves the leg impossible to demonstrate as running. That
is precisely how v0.91.0 shipped it inert without anyone noticing. Adds a Debug line on success
naming the storage: free in normal operation, one log level away when it matters.

## v0.91.1 — wire the credential probe (v0.91.0 shipped the seam inert) (2026-07-21)

**Supersedes v0.91.0; that artifact is materially incomplete — do not vouch it.**

v0.91.0 built the `pbs.AuthSink` seam and the `NoteAuthResult` consumer, and `main.go` never called
`SetAuthSink`. The reporter deliberately skips probing when no sink is attached, so the whole
auth-honesty leg was silently inert: no probe, no `auth_failed`, no self-heal — and nothing failed,
because every unit test injected the sink directly.

Caught during STOP-1 live verification by checking the wiring rather than trusting it. **Exactly the
same class as the controller v0.154.0 defect the day before: a table test over a seam proves the
seam, not the caller.** The published 0.91.0 artifact is left in place and superseded rather than
overwritten — a published version must stay immutable.

- `main.go`: `pbsReporter.SetAuthSink(pdMgr)` in the pbsdr bridge block.
- `TestLiveReporter_NoSinkMeansNoProbe` pins the no-sink-no-probe contract, so the inert case stays a
  documented behaviour rather than an accident nobody notices twice. The wiring itself is proven by
  the live STOP-1/STOP-2 evidence, which is the only thing that can prove it.

## v0.91.0 — the DR tier can no longer be `applied` and dead at the same time (R-39 fleet fix + R-50b(a)) (2026-07-21)

**Requires hub >= v0.68.0** for the re-arm signal. Hub v0.68.0 is safe for 0.90.0 agents (they drop
the unknown descriptor key), but the guarantees below need THIS agent. **MinAgent → 0.91.0 is an
operator manifest save, sequenced after the fleet has self-updated — not a code change.**

### What was broken

Three compounding defects let a box report `applied` while every PBS request 401'd:

1. **The re-key was invisible.** An ep0 re-issue rotates the SECRET of an existing token, so
   `token_id`, `fingerprint`, `datastore` and `namespace` all come back byte-identical. The agent
   re-applies on the descriptor's CONTENT HASH, so a converged box short-circuited and never consumed
   the fresh secret. Proof from the N100: `consumed-failed.json` carried a hash byte-identical to the
   `marker.json` written two minutes before the re-issue.
2. **The agent could not read its own credential.** It WRITES
   `/etc/pve/priv/storage/<id>.pw` through the root wrapper, but that directory is `0700 root:www-data`
   and the wrapper had no read verb — so `pbsTargetsFromPVE` got "permission denied" every cycle,
   logged a Warn and skipped the datastore. The one loop that could have caught the 401 was blind **by
   construction**.
3. **Nothing probed authentication.** A snapshot list that fails with 401 looked exactly like "PBS is
   busy".

### The fix

- **`WirePBSDR.SecretGeneration`** — field-exact with the hub's descriptor. Because `descriptorHash`
  marshals this struct, the hub's monotonic mint counter is what finally moves the hash and re-arms a
  converged agent.
- **Wrapper `read` verb** (+ exactly ONE sudoers line, + a `pbsdr-read` capability row). Prints one
  secret to STDOUT and nothing else: no network, no mutation, no logging of the value, and the secret
  never rides argv (sudo logs argv). Traversal is refused three times over — the id grammar admits no
  slash, `val_sdir` pins the directory, and the RESOLVED path is prefix-asserted.
- **`pbs.ProbeAuth`** — `GET /version`, the cheapest authenticated question, with a distinct
  `ErrUnauthorized` sentinel. `/version` needs no datastore, namespace or privilege, so a 401 there
  means the CREDENTIAL is bad — not that an ACL is narrow. **403 is deliberately NOT treated as
  unauthorized**: re-keying a too-narrow token would mint credentials forever without fixing anything.
- **The probe runs on the 15-minute collect path** (not the 6 h verify cadence) and its verdict
  becomes a LOUD `auth_failed` state the hub's pbsdrheal escalates to a fresh mint. **A transport
  error is UNKNOWN, never a rejection** — otherwise every network blip would burn a credential.
  Recovery is self-clearing.
- **`readPBSSecret`** now prefers a directly-readable file and falls back to the wrapper, so a box
  with its own agent-owned secret dir needs no sudo at all.
- **R-50b(a):** the report carries the installed wrapper's sha256, so drift against the vouched
  manifest value is finally answerable. Empty = unknown, never drift.

### Tests

Scenario A (a re-key re-arms a converged agent) with the hash mechanism asserted separately; the
unknown-field compat direction; auth_failed loud/ignored-for-other-storage/none-before-descriptor/
self-clearing; and the wrapper `read` verb executed under real bash — traversal refusals, secret to
stdout only, missing-file refusal, side-effect freedom.

**Three red-proofs run at the assertion level.** Removing `SecretGeneration` makes the re-arm test
fail with `consume calls=1, want 2`. Swallowing the probe result leaves `State:applied
AuthFailed:false` — the July-18 shape exactly. Deleting the id charset guard alone does **not** open
a traversal hole (readlink + the prefix assertion still catch it), so the isolating red-proof removes
the charset guard AND the prefix assertion and shows the out-of-tree secret printed.

## docs — the workflow moved to DooPlex-local execution (2026-07-19)

**Docs only, no version bump, no code change.** Claude Code now runs on DooPlex (192.168.0.180,
Debian 13, `kisfenyo`) instead of the Windows workstation, working directly in
`/mnt/5_hdd/felhom.eu/git/felhom-agent`.

- `CLAUDE.md` — the build/deploy table is now local-first: build on DooPlex, then **one** hop
  `scp /tmp/felhom-agent-<v> felhom-pve:/tmp/`. The old path went build-server → Windows box →
  felhom-pve, needing `cygpath -w` for the local scp path; that two-hop detour and its CRLF hazard
  are gone (recorded in the new "Legacy: Windows workstation" note, not deleted).
- **New clean-tree gate** before any build: `git status --porcelain` empty AND `HEAD` ==
  `origin/main` — the CC working tree is now the tree that gets built. An unpushed change does not
  exist.
- `MSYS_NO_PATHCONV=1` is no longer needed for `pct` (it was an MSYS path-mangling workaround);
  `ssh felhom-pve` is plain now. Selftests run locally on DooPlex against the demo API.
- Workspace-root pointer updated to `/mnt/5_hdd/felhom.eu/git/CLAUDE.md`.

Historical Windows references in past CHANGELOG entries and `PLAN.md` are left untouched.

## build tooling — the golden bakes EVERY infra image, asked from the controller (2026-07-19)

**No agent version bump: `configs/build-golden.sh` only (v2.0.0 → v2.1.0). Effective at the NEXT
golden build — the current golden is NOT rebuilt for this.**

- **The bug, observed live twice.** Enabling Megosztás on a fresh box pulled `felhom-samba` from the
  registry with zero feedback: minutes of silent nothing. Cause: this script carried its own
  hand-maintained array of three image tags, with a comment instructing the reader to keep it in sync
  with the controller's `internal/infra` constants. It drifted the moment a fourth stack was added —
  `felhom-samba` was never added here, so the golden baked **3 of 4**.
- **The fix is structural, not another copy.** The list now comes from the controller image the bake
  just pulled: `docker run --rm <controller> --print-infra-images` (backed by `infra.Images()`, which
  derives from the pins themselves). The golden therefore bakes exactly what **that** controller
  version will request, and the two cannot disagree by construction. A controller-side test parses
  the const block out of the source and fails if a pin is added without reaching `Images()`.
- **Ordering fix that this exposed.** `docker logout` + `config.json` removal ran immediately after
  the controller pull. `felhom-samba` lives on the **same private registry**, so the infra loop would
  have 401'd. The logout moved to **after** the loop, with an added hard assertion that no credential
  remains in the guest before it is archived — the credential is still never baked.
- **Fallback, loudly.** A controller older than v0.147.0 has no `--print-infra-images`; the bake falls
  back to the historical 3-image list and prints three WARN lines saying felhom-samba will not be
  baked and Megosztás will pull at runtime. The fallback is exactly the drift-prone thing this change
  removes, so it announces itself rather than passing silently.
- **ROADMAP:** golden **≥ 0.147.x** carries all four infra images.

## v0.90.1 — R-39 hotfix: PBS reconcile must not pass `--server` to `pvesm set` (2026-07-18)

**Config-only fix (wrapper + red-proof); the Go binary is unchanged.** Ship the wrapper with the
next agent deploy — `configs/felhom-pbs-apply` is a shipped artifact, not a build input.
Green: `go build ./... && go vet ./... && go test ./...` all pass.

- **The defect.** The `reconcile` verb built its argv as
  `args=(--server "$server" --fingerprint "$fp")`. PVE treats a PBS storage's `server` as a
  **create-only** parameter and rejects the ENTIRE `pvesm set` call —
  `can't change value of fixed parameter 'server'` — **even when the value passed is byte-identical
  to the stored one**. So `reconcile` could never succeed against an existing entry; it exited 255
  every time.

- **Why that was severe rather than cosmetic.** The agent consumes the hub's **one-time** PBS token
  secret BEFORE invoking the wrapper. A wrapper failure therefore *burned* the credential: each hub
  "Re-issue PBS credentials" minted a fresh secret, the agent consumed it, the wrapper rejected the
  apply, and the storage entry stayed pinned to the **revoked** one. Observable end state: the PBS DR
  tier authenticating **401 Unauthorized** indefinitely while the agent reported
  `pbsdr: converged state=applied`. Live-diagnosed on the N100 demo host during the 2026-07-18
  rehearsal wrap (`felhom.eu/documentation/tests/VALIDATION-n100-rehearsal-2026-07-18.md`, finding
  F2 / ROADMAP **R-39**).

- **The fix.** Drop `--server` from the reconcile argv. The server address is immutable by
  construction — relocating a PBS endpoint requires a fresh `create` — so there was never anything
  for `reconcile` to reconcile there. `--fingerprint` (and `--password` when a secret is fed) remain,
  which is the mutable identity the verb exists to push. Proven on the live box before committing:
  `pvesm set felhom-pbs --server <same> --fingerprint <same>` → rejected;
  the same call without `--server` → **rc 0**.

- **Red-proof** `TestReconcileNeverPassesServerToPvesmSet` (`internal/pbsdr/manager_test.go`):
  isolates the `reconcile)` block from the shipped wrapper and asserts no `--server` reaches
  `pvesm set`, plus that `--fingerprint` is still pushed (so the verb can't be hollowed out).
  Verified RED against the unfixed wrapper and GREEN after the fix. Two traps the proof handles
  explicitly: the pattern is **line-ending tolerant** (`\r?\n`) because this repo is cloned on
  Windows and an `\n`-only pattern would match nothing and pass **vacuously**; and comment lines are
  **stripped before matching**, because the WHY note above the fix necessarily quotes the very flag
  the test forbids.

- **NOT fixed here (deliberately, and each still open):**
  1. **R-39's primary half** — the agent re-applies on a change of the *descriptor hash*
     (`manager.go` ~L235), but a hub credential re-issue leaves the descriptor byte-identical
     (same `token_id`, same `fingerprint`; only the side-table secret rotates) and bumps only the
     generation. So a converged agent still ignores a fresh secret. This wrapper fix means the apply
     now *succeeds* once the agent is made to re-apply; it does not make it re-apply.
  2. **The verify-loop read** — `pbs: cannot read token secret … permission denied`: the non-root
     agent reads `/etc/pve/priv/storage/<id>.pw` directly, a path it can only ever *write* through
     the root wrapper (`/etc/pve/priv` is `0700 root:www-data`, and sudoers exposes only
     `create|reconcile|grant` — there is no read verb). The loop is therefore permanently blind to
     the failure it exists to catch.
  Both ride the spec'd R-39 agent train.

## v0.90.0 — agent train: guest RAM resize (R-24) + fast-tick-until-convergence (R-28) (2026-07-17)

**MinAgent coupling:** felhom-controller v0.143.0 gates its guest-memory-resize UI on this agent
(`FeatureGuestMemoryResize`, MinAgent 0.90.0). One train, one floor raise (Viktor's ruling).
Green: `go build ./... && go vet ./... && go test ./...` all pass.

- **Item 1 — guest RAM resize (R-24, controller-direct)** (`internal/localapi/guestmemory.go`, new):
  a new self-scoped local-API surface — `GET /guest/memory` (current allocation, live usage, and the
  enforced bounds, all agent-computed in MB) and `POST /guest/memory` (bounded resize). The agent is
  the security boundary: every bound is recomputed FRESH per request from a live read (the UI's numbers
  are decoration). Ruled bounds — min **2048 MB**; max **host_total − 2048 MB** (host reserve); a SHRINK
  is refused below **max(2048, usage + 512 MB)** with `below_usage_floor` (stop apps first). Applies via
  the PVE API `SetConfig` — a **live cgroup apply, no reboot** (Phase-0 PROVEN on the nested demo box:
  maxmem moves with the guest running, `/proc/meminfo` ripples via lxcfs). Verify-after-apply: the new
  maxmem is re-read and must equal the target before success is claimed (a pending/reboot outcome is a
  502, never a false success). Refusals return a machine `code` (below_min/above_max/below_usage_floor)
  + fresh bounds at 412; SetConfig is never called on a refusal path. Single-flight per host. Optional
  `Options.Memory` (nil → 503 "not configured"); a NEW narrow `MemoryOps` interface (does not touch the
  shared `GuestAPI`). Wired in `main.go` from the existing proxmox client. Memory only; cores stay
  observation.
- **Item 2 — fast-tick-until-first-convergence (R-28)** (`internal/fasttick/`, new): the agent-plane
  immediacy SECONDARY. While ANY desired-state item is unapplied — most importantly the **pre-tunnel
  WG-registration window a hub poke cannot reach** — it pulses the SAME out-of-band report trigger the
  watchdog/poke use, every **30 s**, and **self-disarms emergently** the instant everything converges.
  State-based by ruling: nothing to journal, no timer to leak. Four cached sources (no exec/network per
  tick): desired-generation == 0 (never fetched); reconcile `LastResult` actionable drift
  (`Planned − Pending > 0` — a destructive `pending_signature` is EXPECTED, excluded); pbsdr
  `waiting_secret` ONLY (the LOUD `consumed_failed`/`verify_failed` are excluded so a stuck box never
  hammers); wgtunnel block-desired-but-not-operational. Supporting seams: `reconcile.Engine.LastResult()`
  (mutex-recorded per pass) and `wgtunnel.Manager.TunnelConvergence()` (cached snapshot refreshed at each
  Apply — the fast-tick never execs `wg`/`systemctl`). A perma-unconverged box fast-ticks at ~2 small
  reports/min (bounded, documented; timers were deliberately rejected).
- **Guests-0/0 (diagnosis passenger):** REFUTED the pool-membership hypothesis on the live nested box —
  guest 9201 IS a pool member, the agent token sees it (VM.Audit comes from the `/pool/felhom` grant),
  hub reports 1/1. The observed 0/0 was the legitimate **pre-provision reporting window** (no guest
  existed yet); the existing `PoolAddVMID` re-assert (bringup.go:498) already covers the known
  restore-over-existing edge (campaign-2 R2). Item 2's fast-tick is exactly the window's mitigation.
  No code change.
- **Tests + red-proofs (run-fail-restore):** localapi memory (grow/shrink, the three refusals each with
  a **SetConfig-count == 0** assertion, cross-guest 403, fresh-bounds-per-request, verify-not-reflected
  502, nil-config 503) — red-proofs (i) floor guard removed → below_usage_floor fails; (ii) max guard
  removed → above_max fails. fasttick (pulse-while-unconverged, silent-when-converged, the ruled
  disarm-on-convergence, full-channel non-blocking drop, first-reason) — red-proof (iii) always-pulse →
  the silent + disarm tests fail. `Engine.LastResult` effect + pre-first-run ok=false. All restored green.

## v0.89.0 — agent train: PBS-DR self-grant (R-22) + escrow config live-reload + agent-plane poke listener (2026-07-16)

Three bundled agent-plane items. Green: `go build ./... && go vet ./... && go test ./...` all pass.

- **Item 1 — pbsdr self-grant (R-22, closes the F4 root cause from `tests/VALIDATION-n100`)**
  (`internal/pbsdr/manager.go`): on a NON-DEFAULT PBS storage id the agent token holds no ACL on
  `/storage/<id>` yet, so the reconcile tick's token-auth pre-check `StorageEntry` (GET
  `/storage/<id>`) 403s and — pre-fix — aborted BEFORE the root-run wrapper `grant` that creates
  that very ACL: a permanent self-deadlock that bit every legacy/descriptor-provisioned id (the
  demo's `felhom-offsite`). Fix: on an `IsForbidden` pre-check error ONLY, run
  `felhom-pbs-apply grant <id>` now (root, no secret, no pre-existing entry — `pveum acl modify` on a
  path is unconditional), re-read once, then flow the normal adoption/create path. The pre-check is
  KEPT (once the ACL exists it short-circuits the happy path cheaply); every other error stays
  transient. Grant-then-still-failing surfaces `verify_failed` loudly, never a silent retry storm.
  Red-proof: `TestSelfGrant_PreCheck403DoesNotAbortBeforeGrant` (fake 403 on the pre-check → pre-fix
  never reaches the grant, calls=[] → FAIL; fixed → self-grant + converge, no secret consumed).
- **Item 2 — escrow config live-reload** (`internal/localapi/escrow_ceremony.go`,
  `cmd/felhom-agent/main.go`): the pbsdr bridge seeds `escrow.pbs_storage_id` into agent.json on DR
  convergence, but `/escrow/preflight` read a daemon-start snapshot and stayed red until a service
  restart. New late-bound `EscrowCeremonyConfig.CurrentPBSStorageID` resolver (mirrors the existing
  `DRConfigured func() bool` pattern) re-reads config from disk at preflight time — exactly what the
  ceremony subprocess itself loads, so the row flips green in-process, no restart. Falls back to the
  boot snapshot on read error / all-env config. Red-proof:
  `TestEscrowPreflight_PBSStorageIDLiveReload` (seed after boot → pre-fix stays red → FAIL).
- **Item 3 — agent-plane poke listener (Direction-2a, `SPIKE-immediate-sync-transport-2026-07-16`)**
  (new `internal/poke`, `internal/wgtunnel.LoadAssignedAddr`, `cmd/felhom-agent/main.go`,
  `internal/hub/loop.go`): a CONTENTLESS UDP poke relayed hub → ep0 forced-command → wg0-origin →
  this listener fires ONE immediate desired-state cycle (the hub control loop's out-of-band report
  trigger — the SAME channel the storage watchdog uses; fan-in, cap-1 coalescing). The socket binds
  EXCLUSIVELY to the box's WG /32 (`registered.json`; never 0.0.0.0, never the LAN iface) on the
  fixed port **51822**; payload is ignored entirely (a forged/replayed poke costs at most one extra
  debounced tick); leading-edge debounce (`DebounceWindow`) coalesces a burst into ≤1 tick. Enabled
  whenever `wg_tunnel.enabled`; a lost poke is harmless (the 15-min cycle still reconciles). This is
  the FIRST concrete slice of the OOB/mutual-repair arc (R-13) — the listener+trigger only, nothing
  more. Red-proofs: `TestBindConfinement` (wildcard bind → bound to `::` → FAIL) and
  `TestDebounceCoalescesBurst` (guard removed → 10 fires for 10 pokes → FAIL). Port registered in
  REUSE.md as a shared cross-repo constant (hub sender + ep0 `felhom-poke`).

## v0.88.0 — controller-driven escrow ceremony: --output=json + localapi job + one-shot R claim (2026-07-13)

The agent half of the customer-facing recovery-code wizard (controller v0.127.0; every mechanism
validated by felhom.eu SPIKE-controller-escrow-2026-07-13 — PTY-under-no-TTY, fixed-argv sudoers
5/5 refusals, R pipe round-trip, env_reset, 2.3–2.4 s timings). Operator ruling F1 (2026-07-13):
R transiting the CF tunnel once at display is an accepted, documented risk (threat-model paragraph
in RUNBOOK-escrow-ceremony.md).

- **`--output=json` machine mode** (`cmd/felhom-agent/main.go`): `runSelftestEscrowCreate`'s body
  extracted into the shared `escrowCeremony()` core; text mode stays BYTE-IDENTICAL (banner, R
  block, exit codes 0/1/2, upload-fail-after-R order). json mode emits ONE
  `escrow.CeremonyOutput` object on stdout (version 1: recovery_code, key_fingerprint,
  entropy_bits, blob/identity sizes, restic_pw_sealed, uploaded), every human line to stderr, no
  partial JSON on failure; `--offline`/`--paperkey` are refused in json mode (print-oriented).
- **The ONE fixed argv** (`internal/escrow/ceremony.go`): `escrow.CeremonyBinary` +
  `escrow.CeremonyArgs()` — the single source shared by the localapi exec, the capability
  manifest entry, and (byte-identically) the new `FELHOM_ESCROW` sudoers alias
  (`configs/felhom-agent.sudoers`). `TestEscrowCeremonyArgvPinned` +
  `TestManifestCoveredBySudoers` transitively lock runner == manifest == sudoers; never build the
  argv with flag helpers, never normalize `--`→`-` (spike §2.2).
- **localapi ceremony endpoints** (`internal/localapi/escrow_ceremony.go`, `withGuest`-wrapped):
  `POST /escrow/ceremony` (single-flight 409; detached job, 60 s timeout; `sudo -n` + the fixed
  argv; stdout parsed then zeroed — SECRET-BEARING, never logged), `GET /escrow/ceremony/status`
  (non-secret summary + stderr-tail failure detail ≤500; R structurally absent from the job
  struct), `POST /escrow/ceremony/claim` (ONE-SHOT: 200 `{recovery_code}` once → holder zeroed;
  410 on re-claim; **TTL 10 min** → `unclaimed_void`, active AfterFunc belt + lazy check),
  `GET /escrow/preflight` (storage id, DR tier, age, hub target, staged-secret informational,
  `sudo -n -l` grant list-probe). Crash-safety is IN-MEMORY BY DESIGN — an agent restart loses R
  safely (re-run supersedes); no journal, deliberately.
- **Capability** `escrow-ceremony` (Critical, `GatedBy: pbs_dr` EXPLICIT — non-pbsdr name by
  decision): list-mode probe of the shared argv; inactive (never red) while the DR tier is off.
- Tests: one-shot claim + double-claim 410, TTL void + zeroed holder, R-substring absent from
  every status/snapshot payload (incl. the serialized job struct), single-flight, supersede on
  re-run, failure taxonomy (exit/unparseable/version), preflight truth table, argv pin. §10
  red-proofs demonstrated (see felhom.eu REPORT).

## v0.87.0 — SystemDisks device-mapper walk: legacy-boot hosts get a working drive wizard (IA finding 2, MEDIUM) (2026-07-13)

On a legacy-boot PVE (LVM root, no mounted ESP) `SystemDisks` resolved NOTHING — `wholeDiskOf`
stops at `/dev/mapper/pve-root` — so the all-system fail-safe classified EVERY disk system and
the drive wizard could never offer a candidate (the IA validation's hot-added 5 GB disk stayed
invisible on the drill box). Operator ruling 2026-07-13: walk the root's backing device through
`/sys/block/<dev>/slaves` recursively down to physical disks (dm AND md; topology, never VG
names); those + any mounted-ESP holder are system; the all-system fail-safe returns to being the
WALK-FAILURE error case only. SAFETY DIRECTION: the outcome made impossible is a root-backing
disk classified candidate — per-branch conservatism means ANY unresolvable slave fails the whole
walk (`ok=false` → all-system, the unchanged code path).

- **`physicalDisksOf` + `walkSlaves`** (`internal/storage/role.go`): symlink-canonicalize →
  fast-path `wholeDiskOf` (raw disks/partitions, unchanged) → recursive sysfs slaves walk for
  virtual devices, with cycle/depth guard; non-`/dev` sources (ZFS datasets, NFS, overlay) stay
  unwalkable → fail-safe. Partition slaves resolve via the existing regexes (`sda3 → /dev/sda`);
  exotic partition names the regexes don't cover (e.g. `md0p1`) fail the walk → fail-safe.
- **`HostReader.BlockSlaves(name)`** — the ONE new seam method (root-free: sysfs is
  world-readable): slaves list + hasDir; production reads `/sys/block/<name>/slaves`; every test
  fake mirrors it. Live-probed (§3): physical disks carry an EMPTY slaves dir — hasDir alone is
  not resolution; a virtual device with an empty/unlistable slaves dir fails the walk.
- **Behavior deltas:** legacy-boot LVM hosts now resolve (`{root's physical parents}`, ok=true)
  — the wizard lives (still behind the untouched data-bearing/claim guards); EFI(+LVM) hosts are
  byte-identical (ESP and walk agree on the same disk — §3 felhom-pve transcript + fixture);
  md-raid roots mark BOTH members system. NEW protective delta: a host whose ROOT topology
  cannot be fully grounded is now all-system even if an ESP resolved (pre-fix it silently
  trusted the ESP alone; ruling's per-disk conservatism).
- **Tests** (`role_walk_test.go`, fake-sysfs fixtures mirroring the §3 transcripts): the
  SIGNATURE table (root-backing disk(s) ALWAYS in the system set across legacy-LVM / md-raid /
  EFI+raw / EFI+LVM / nested dm-on-md — never weaken), dead-wizard-lives (A), dangling-slave
  fail-safe through the REAL sysKnown=false path (D), cycle guard, empty-slaves. Red-proofs
  recorded: A pre-fix resolver → dead wizard reproduced; B walk-returns-dm-node → SIGNATURE
  VIOLATION naming the missing disk; D conservatism removed → "scratch classified candidate
  while walk incomplete".
- Format/mkfs paths, data-bearing guards, wizard UI: UNTOUCHED. MinAgent/controller coupling:
  none (agent-internal classification).

## v0.86.0 — DR-tier-by-default: capability `inactive` state + F-3 provision-parent ownership (2026-07-12)

Agent half of the DR-tier-by-default batch (DRILL-day0-vm-2026-07-12; operator decisions: DR
capability is BAKED on every install, activation is a hub flag, disabled ≠ degraded).

- **Capability `inactive` state** (`internal/capability`): a third status next to ok/degraded —
  a config-GATED capability whose plumbing is HEALTHY (binary present, sudo granted) but whose
  feature is off reports `inactive` / reason `disabled by configuration`. The 3 `pbsdr-*` entries
  are gated (`GatedBy=GatePBSDR`, applied by the stable name prefix in `Manifest()`); broken
  plumbing (binary missing / grant denied) stays DEGRADED even with the gate off — an un-migrated
  pre-v1.15.0 box must never look deliberately disabled. `Summarize` counts only real degraded
  (inactive never error-logs); the startup self-check logs an `inactive` count and now runs AFTER
  the pbsdr gate wiring so its snapshot matches the first report.
- **`pbsdr.Manager.DRConfigured()`** — the gate's answer: true when the last-seen descriptor was
  enabled (any live state except `disabled`); before the first desired-state fetch it falls back
  to the persisted converged marker, so an applied box never flaps to inactive across a restart.
- **F-3 — provision parent-dir ownership** (`internal/provision/backhalf.go`): a ROOT-run
  provision (the Day-0 one-shot) now chowns the just-created `guests/` + `guests/<vmid>/` PARENT
  dirs to the state-dir's owner (`chown --reference`, NON-recursive — the bootstrap leaf stays
  the mapped guest-root's). Previously they were left root:root 0700 → the non-root daemon's
  lanresolver got "permission denied" (drill live-fix now also applied to felhom-pve, which had
  the same latent state; Peti's host unreachable — deferred). A daemon-run (non-root) provision
  skips it (`geteuid` seam).
- Tests + red-proofs: gate-off-healthy→inactive / gate-off-broken→degraded / gate-on→ok /
  exactly-pbsdr-gated; DRConfigured lifecycle (incl. marker-across-restart + disabled-wins);
  root-run parent chown issued, non-root not, never recursive. All three mutations proven red.
- Shipping note: `configs/felhom-pbs-apply` already lives in this repo — host-install v1.15.0
  (felhom.eu) now ships it like the mkfs/selfupdate wrappers (drill F-7); no publish change here.

## v0.85.0 — the boot/recovery plane: F12 ordering-cycle fix + F11/F10/F9/F2/F1 + appliance self-heal (2026-07-12)

Fixes the findings CAMPAIGN-3 (`felhom.eu/documentation/audits/CAMPAIGN-3-2026-07-11.md`) raised
around the NAS automount lifecycle — the data plane held, the reboot/recovery plane did not.

- **F12 (CRITICAL) — boot ordering cycle.** `internal/storage/netmount.go`: BOTH rendered units drop
  `After=/Wants=network-online.target`. The `.mount` keeps `_netdev` (the correct, sufficient network
  ordering — systemd classes it remote-fs); the `.automount` gets NO network relation (a trigger
  needs none, and it must stay orderable before local-fs without dragging the network into the
  transaction). The literal ordering had closed the cycle
  networking→local-fs→automount→network-online→networking, which systemd broke by deleting an
  arbitrary job — one boot lost networking entirely (host dark 7 h), the next lost the automount.
  **Installed-unit migration:** `MigrateNetworkUnits` — a general template-drift reconcile
  (SHA-256 content compare of each marker-owned unit vs a fresh render of its reconstructed spec;
  rewrite + one batched `daemon-reload`; idempotent). Runs at daemon startup before the reassert
  sweep and at the head of `EnsureNetworkMount`, so pre-0.85 units carrying the cycle are repaired,
  not just future adds.
- **F11 (HIGH) — read the right unit.** `internal/storage/netreassert.go` `netReassertClassify`: the
  re-arm decision is driven ONLY by the host `/proc/mounts` fstype at the mountpoint. Active
  nfs4/cifs → skip-active (inherited by fresh namespaces); anything else → re-arm. The `.automount`
  unit's own state is never consulted (an armed trigger always reports active — the mis-skip trap).
- **F10 (CRITICAL) — re-arm for real.** A `.mount`/`.automount` left `failed`/start-limit-hit (the
  campaign's unexport→idle-timeout→access×5) is `reset-failed` FIRST (new sudoers verb) — without it
  the `enable --now` is refused by the start limit and the share stays dead across every boot. The
  failed-state read is unprivileged (`systemctl is-failed`, seam-injected).
- **F9 (HIGH) — say what you did.** The pass enumerates by marker-owned unit files on disk (not
  enablement/runtime state) and logs an INFO verdict line for EVERY share
  (reasserted / reset-failed+rearmed / skip-active / skip-foreign / error) — an empty-looking sweep
  over N shares is now impossible.
- **F2/F1 (LOW) — zero residue.** `RemoveNetworkMount` (and, via the same path, every verify-fail
  rollback) reset-failed's the pair before removing the files (no `not-found failed` residue) and
  `rmdir`s the now-empty mountpoint (F1 — the campaign's 10 stub-shaped leftovers). rmdir ONLY: a
  non-empty dir is left in place with a WARN (fail-safe; never `rm -rf`).
- **The hook can never take a guest down (F10/rc255).** `cmd/felhom-agent/main.go` `runHookPhase`:
  every guest-hook phase runs recover-wrapped under a hard timeout and returns cleanly (a panic →
  logged to the PVE task log, swallowed; an overrun → abandoned). The installed wrapper snippet no
  longer `exec`s — it runs the binary as a child, `|| true`, and `exit 0` (the shell belt).
- **Appliance-mode node self-heal (Part 6).** New `internal/selfheal` package: a minimal
  check/remedy registry gated on `deployment_mode`. One heal ships — host networking recovery
  (F12-class defense in depth): Healthy ⇔ networking.service active AND a default route; the remedy
  (`systemctl start networking.service`, new sudoers verb, ≤3 attempts, 10/30/60 s backoff, terminal
  give-up logged) runs ONLY on `deployment_mode:"appliance"`. A byo host runs the check + WARNs; the
  remedy is structurally unreachable (the Manager gates before any exec). Absent/unknown mode → byo
  (fail-safe). Give-up is log-only (no natural HostReport field; the report schema is out of scope) —
  the ERROR lands in the always-DEBUG applog ring for a hub bundle-pull.
- **Config:** top-level `deployment_mode` (`internal/config`, `+FELHOM_AGENT_DEPLOYMENT_MODE` overlay,
  `IsAppliance()` fail-safe-to-byo). NOT overloaded onto `Privileged.Mode`.
- **Sudoers (loud, per the no-widening rule):** two narrow additions —
  `systemctl reset-failed -- mnt-felhom*` (F10) and `rmdir /mnt/felhom-drives/*` (F1, fail-safe:
  rmdir refuses a non-empty dir) in FELHOM_NETMOUNT; `systemctl start networking.service` as the new
  FELHOM_SELFHEAL alias (appliance self-heal; the grant alone cannot harm — starting networking is
  what boot should have done; the remedy is ALSO code-gated on appliance). Capability manifest gains
  the three representative probes.
- Tests: F12 render (no network-online, `_netdev` present) + reconcile (drift rewritten once,
  idempotent, batched reload, round-trip) + foreign-unit ignore; reassert fstype table with the
  automount-state-ignored red-proof, reset-failed+rearm, per-unit verdict count; hook rc-0 under
  panic/timeout; zero-residue (reset-failed + rmdir, never rm -rf); selfheal state machine +
  byo-never-invokes + absent⇒byo. All green.

## v0.84.0 — ReassertNetworkMounts: NAS automount survives guest reboots (RCA fix 1) (2026-07-11)

Agent half of the RCA fix pair (controller v0.117.0). Source:
`felhom.eu/documentation/audits/AUDIT-nas-cwa-rca-2026-07-11.md` — a fresh guest namespace inherits
REAL submounts (ext4/nfs4) but NOT an idle autofs trigger, so after any guest reboot an idle NAS
share silently degrades to a local stub inside the guest. The heal (re-create the automount → the
fresh trigger-mount event propagates live into running guests) was live-proven in the RCA
remediation; this release makes it automatic.

- **`internal/storage/netreassert.go`** `SudoHostOps.ReassertNetworkAutomounts`: per configured
  network mount, the §8 decision table — real nfs/nfs4/cifs mounted → skip (inherited); `autofs`
  trigger at the path → **stop + enable --now the `.automount`** (existing FELHOM_NETMOUNT sudoers
  verbs; there is NO restart grant); neither → skip (removed/orphan states owned by add/remove).
  Idempotent; per-share errors never stop the pass. Unit enumeration factored into
  `networkUnitEntries()` (shared with `ListNetworkMounts`, behavior unchanged).
- **`internal/localapi/netreassert.go`** `Server.ReassertNetworkMounts`: the daemon leg — runs the
  host-global pass once, then best-effort verifies each RUNNING guest actually sees each share path
  (`GuestSeesMount`; the RCA's masking lesson). Type-asserted capability (the lean
  `NetworkStorageOps` interface and its fakes stay unchanged — the main.go
  `ReassertEnrolledMounts` pattern). Wired at agent startup after `ReassertGuestBinds`;
  deliberately NOT in the 20 s ticker (an idle trigger is healthy and must not be churned).
- **`internal/guesthook/netreassert.go`** + `PhasePostStart`: the hook leg — PVE runs the hookscript
  as root, so `guest-hook <vmid> post-start` re-arms triggers with DIRECT systemctl and verifies
  via `GuestSeesPath` (hook-process mirror of GuestSeesMount). Non-fatal by contract (stderr → PVE
  task log; exit 0 always); 30 s bound. The installed wrapper snippet already forwards all phases —
  no snippet re-install needed.
- Tests + red-proofs: §8 table (red-proof: always-rearm shape → FAIL "nfs → rearmed, want
  skip-active" — the live-mount churn the table prevents); rearm emits EXACTLY stop+enable-now on
  the right unit; active mount → ZERO systemctl calls; idempotent double-pass; hook wiring
  (red-proof: PhasePostStart case removed → FAIL "got []"); daemon leg verifies running guests
  only; invisible-share verify is WARN-only (non-fatal).

## v0.83.0 — observability pass: always-DEBUG capture ring + GET /debug/logs + heartbeat log-pull + gap-fill sweep (2026-07-11)

Agent half of the cross-repo observability task (controller v0.116.0 + hub v0.46.0). Motivating
incident: a refused NAS verify on an `info`-level box left NOTHING readable remotely — the agent
logged only to host journald and the new features emitted few lines.

- **Capture layer** (`internal/log`): `applog.New` now returns `(logger, *Ring)` — a slog fan-out
  where stderr keeps the configured level (journald unchanged) and a ~1000-entry ring handler is
  FIXED at `LevelDebug`, so flow detail exists for remote pulls without a config flip. The ring is
  an io.Writer fed by a stdlib TextHandler; entries are parsed (time/level/message-verbatim).
  Red-proof: ring gated at the emit level → capture-at-info test FAILS ("ring holds 1, want 2").
- **`GET /debug/logs`** (local API, same token-auth/self-scoping wrap as siblings): the ring as
  JSON `{entries, total}`; `?raw=1` plain text; 503 when unwired. Plus a request-level DEBUG
  middleware (method/path/status/duration — never bodies) wrapping the whole mux.
- **Heartbeat log-pull** (the report-channel logtail.go pattern mirrored): the control envelope
  gains `log_tail_requested: bool` (additive); when set, the NEXT heartbeat carries
  `log_tail: {collected_at, lines[]}` (newest-kept, 128 KB cap). Consume-once both ends: local
  pending drains onto the carrying push; a FAILED push leaves the hub request pending → the next
  envelope re-arms (retry proven in tests; red-proof: drain removed → tail ships every cycle →
  FAIL). Serving a pull logs `operator log pull served` (INFO — customer-visible transparency).
- **Gap-fill sweep** (entry, decisions, outcome+duration, errors): netverify (job start, trigger
  outcome, /proc/mounts verdict, journal byte-count, classification code, rollback outcome,
  duration), netstorage add (pre-probe pass verdict, creds staged/removed — path only), netmount
  Ensure/Remove (per-unit install/enable/remove-step results), signedjobs (jobs fetched ids+
  duration, op received class/host/expiry — never signatures), selfupdate executor (invariants
  passed, download sha-match+duration), disks (assign/eject/decommission outcome INFO),
  ReassertGuestBinds (pass summary), controller-swap (pre-pull verify, negative health verdict),
  desired syncer + hub loop (per-exchange DEBUG with durations).
- **S7 log-sequence smoke**: a full fake NAS add at emit level info must leave the ordered phase
  markers in the ring (red-proof: dropping the /proc/mounts verdict line → FAIL naming the phase).
- No new sudoers grants, no journald scraping, no streaming — pull-only. Demo-deploy only — NOT
  published (Peti stays 0.81.0; this reaches him with the next publish train).

## v0.82.0 — local-API version channel: X-Felhom-Agent-Version on every response (2026-07-11)

The controller's capability detection upgrades from route-probing to version comparison: the
local-API mux is wrapped so EVERY response (any route, any status, including auth failures) carries
`X-Felhom-Agent-Version` = the build version (`localapi.Options.AgentVersion`, wired from
main.version). The controller (v0.115.0) reads it passively from ordinary traffic and compares it
against a per-feature MinAgent table; header-less (≤0.81.0) agents keep working — the controller
falls back to the v0.114.0 route probe unchanged. No new routes, no envelope changes, no sudoers
changes. Test: header asserted on authed/unauthed/404 responses (red-proof: wrap dropped → fails);
empty version emits NO header. Demo-deploy only — NOT published (Peti stays 0.81.0 = the live
fallback path).

## v0.81.0 — NAS verify-before-commit: retry=0 + detached verify job + journal classification (2026-07-11)

Implements the agent half of the "NAS verify-before-commit" task on the SPIKE-nas-verify-2026-07-11
evidence (felhom.eu/documentation/audits/, commit b57f6c1). `POST /netstorage/add` no longer succeeds
blind — a bogus share can no longer sit at "Készenlét" forever.

- **`retry=0` in the production NFS option string** (`netmount.go mountOptions`; Q4-vi): a dead-NAS
  on-demand access now fails clean in ~4 s (ENODEV) instead of wedging the accessing app until
  systemd's 90 s start cap; verify failures classify as "No route to host" instead of a
  diagnostic-free timeout. SMB string unchanged (retry is a mount.nfs option). Already-installed
  units are NOT rewritten (pre-customer; re-add re-creates them).
- **`internal/storage/netverify.go`** — `ClassifyNetVerifyFailure`: pure, table-driven journal
  classification on the Q4 VERBATIM substrings → `unreachable | nfs_export | smb_auth | smb_share |
  timeout | mount_failed`. `nfs_export` deliberately MERGES not-found/not-permitted (NFSv4 returns
  the identical string for both — Q4 ii≡iii). Everything exits rc=32, so classification is
  string-based by design.
- **Detached verify job** (`localapi/netverifyjob.go`, the formatjob shape but IN-MEMORY single
  slot): add = decode → role-gate → SYNC fast-fail (full spec validation + 2 s TCP pre-probe; an
  unreachable server is refused with NOTHING installed) → stage SMB creds → EnsureNetworkMount →
  detached verify (trigger read through the automount; mount success judged from **/proc/mounts
  only** — never readability, a 0700 export EACCES is a good mount) → on failure: journal-classify
  + **auto-rollback** (RemoveNetworkMount + creds file). `GET /netstorage/verify-status` reports the
  slot; phase `none` after an agent restart is the controller's rollback signal (Scenario F —
  deliberately not persisted). Single-flight: a second add while verifying is a 409.
- **Journal access is UNPRIVILEGED** (`journalctl -u <unit> -n 20 -o cat`, no sudo, no new sudoers
  grant): requires the felhom-agent user in the `systemd-journal` group (host-install ≥ v1.12.0
  successor adds it; existing hosts: `usermod -aG systemd-journal felhom-agent`). Journal
  unavailable degrades to `mount_failed` + a hint, still rolled back.
- New exported helpers: `storage.NetworkMountedAt` (autofs trigger ≠ mounted),
  `storage.NetworkEndpointReachable` (the 2 s pre-probe). REUSE.md updated.
- Tests: classifier table on the live Q4 strings, §8 truth table, rollback effects, pre-probe
  zero-install, single-flight + no-job shape. Red-proof outcomes recorded in REPORT.md.

## v0.80.0 — PBS DR tier SLICE 2: the apply-bridge (2026-07-10)

Consumes hub v0.44.0's `pbs_dr` desired-state descriptor (slice 1): hub tick → the box grows the
pbs storage entry + K, hands-free. Laws encoded (spike 00afadc + the offsite bridge precedent),
each red-proof-verified (REPORT.md):

- **`internal/pbsdr`** — the bridge (wgtunnel Loop shape; raw-consumer seam; report stanza
  `pbs_dr`). Flow: **adoption probe first** (existing healthy entry → grant + marker, NO consume;
  **tenancy identity is entry-owned** — a descriptor naming a different namespace never repoints a
  live entry: the demo's manual `felhom-offsite` case) → fresh path: **verify-pin-BEFORE-consume**
  (bare TLS dial pinned to the descriptor fingerprint, `pbs.ProbeFingerprint`) → consume
  (`POST /api/v1/hosts/{id}/pbs/consume-token`, PLURAL /hosts/ — the slice-1 route; typed
  `ErrNoPBSSecret`) → wrapper `create` (secret on STDIN; `--encryption-key autogen` → K born; .enc/
  .pw placed into `backup.pbs_secret_dir` when overridden — the spike §4 escrow-path flag) →
  `grant` (the Part-0-evidenced dual-grant, exactly) → post-apply active probe → **seed
  `escrow.pbs_storage_id`** (bare ceremony one-liner; an operator-set different value is never
  clobbered; unknown config keys preserved) → descriptor-hash marker. **Consumed-but-failed is
  LOUD**: persistent `consumed_failed` report state, no silent retry — recovery only via the hub
  Re-issue (a fresh staged secret).
- **`configs/felhom-pbs-apply`** (root wrapper, `/usr/local/sbin`, the guarded-mkfs shape) + ONE
  pinned sudoers alias `FELHOM_PBSDR` (create/reconcile/grant). **THE SET-ONLY LAW**: no deletion
  verb exists (entry deletion destroys K = un-decryptable backups); re-apply is `pvesm set`-only —
  grep-gated by `TestSetOnlyLaw` over the shipped file. Secret via wrapper stdin (sudo logs argv).
- Wire: `WirePBSDR` on `WireDesiredState` (field-exact with hub `pbsDRDescriptor`, pinned by
  `TestWireFieldNames`); nil-safe on pre-v0.44.0 hubs. Report: `PBSDRStatus` stanza (adopted/
  applied/waiting_secret/verify_failed/consumed_failed/disabled) via the `PBSDRReporter` seam.
- Capability manifest: `pbsdr-create`/`pbsdr-reconcile`/`pbsdr-grant` (non-critical, the
  selfupdate rationale). `config.Config.SourcePath` records the loaded file for the escrow seed.
- Part 0 (recorded in REPORT.md): status reads ride `FelhomAgentBase` (`Datastore.Audit@/`); the
  WRITE path 403s without `FelhomAgentStore` on `/storage/<id>` → the grant op = the §4b dual-grant
  exactly. Demo's purged grants re-asserted; token vzdump to the PBS entry live-proven OK.

## v0.79.0 — SLICE 3: escrow upload carries sha256 of the sealed restic password (2026-07-09)

The hub-verified escrow auto-confirm chain, agent third: the escrow-create ceremony now records WHICH
offsite repo password the blob covers — as a non-reversible sha256 (a 256-bit random secret's hash is safe
to store/serve; the password itself is never logged or uploaded).

- `internal/escrow.HashResticPassword` — the CANONICAL hasher: sha256 hex over the TRIMMED password string
  (exactly the value `AttachResticPassword` seals into the blob). **Pinned cross-repo test vector**
  (`TestHashResticPassword_PinnedVector`, same vector asserted in felhom-controller) so the two hashers can
  never drift silently.
- `cmd/felhom-agent`: `escrowUploadRequest` gains `restic_pw_sha256,omitempty` — set only when a staged
  password was folded in (no staged file → field OMITTED → the hub stores NULL → the controller stays
  pending; correct, the blob doesn't cover the key). `TestEscrowUploadContract` updated (the hub mirrors it
  in the same commit-pair) + asserts the omitted-when-unstaged behavior.
- Ceremony flow (create / self-verify / R-banner / staged-file wipe) otherwise untouched.

## v0.78.0 — fork-4 hygiene: DELETE /escrow/stage-secret (staged-secret wipe) (2026-07-09)

Part of the offsite-provisioning hardening bundle (pairs with controller v0.107.0 + hub v0.39.0). The staged
restic repo password was wiped only by the escrow-create ceremony; a confirm WITHOUT a fresh ceremony (the
password already escrowed — the live e2e's Option A) left the 0600 staged file behind indefinitely.

- `internal/localapi`: `DELETE /escrow/stage-secret` (withGuest) — removes the staged file (+ any stale
  `.tmp` partial). **Idempotent**: absent file → clean 200 `{removed:false}`. The controller calls it
  whenever `EscrowState` flips to `escrowed`. No value ever logged (nothing to log — it's a removal).
- Test: stage → wipe (file GONE) → re-wipe idempotent → 401 unauthenticated.

## v0.77.0 — fork-4: escrow the offsite restic repo password under R (2026-07-09)

Makes the restic-offsite repo password recoverable at DR by riding the existing customer-recovery-code (R)
zero-knowledge escrow (age-under-R, alongside the IdentityBundle), validated by
`felhom.eu/documentation/audits/SPIKE-restic-password-custody-2026-07-09.md`. Additive; the PBS-K escrow
path is untouched.

- `internal/escrow/identity.go`: `IdentityBundle` gains `ResticRepoPassword` (`restic_repo_password,omitempty`)
  — rides the existing `WrapIdentityBundle`/`UnwrapIdentityBundle` age-under-R path (self-verified by
  `escrow.Create`). Added `AttachResticPassword` (mirrors `AttachWGKey`), `StagedResticPasswordPath`, and
  `WipeStagedResticPassword`. Pre-fork-4 blobs lack the field and CANNOT be retro-fitted (R never retained)
  — the controller's atomicity gate ensures no offsite ciphertext exists until the key is escrowed.
- `internal/localapi`: `POST /escrow/stage-secret` (`withGuest`, `scopedFromBody`) transiently stages the
  controller-pushed restic password (0600, atomic tmp+rename, **never logged** — field name only, value
  never echoed), overwritten on re-push. Stage path injectable via `Options.EscrowStagePath` (default the
  canonical `StagedResticPasswordPath`) for testability.
- `cmd/felhom-agent/main.go` (`runSelftestEscrowCreate`): the escrow-create ceremony auto-injects the staged
  password into the `IdentityBundle` (mirrors the WG-key auto-inject) and **wipes** the staging file after a
  successful create. The ceremony stays operator-invoked (`--selftest=escrow-create`).
- Tests: `IdentityBundle` round-trip carries `ResticRepoPassword` byte-exact + not-in-blob + wrong-R fails
  closed; `AttachResticPassword` (missing/present/empty); stage endpoint stages 0600 + non-secret ack +
  cross-guest 403 + **value-not-in-log**.
- NOT yet live-validated — the supervised escrow ceremony (enable→stage→escrow-create→confirm→gated run) is
  the operator-run follow-up.

## v0.76.0 — restore-test full-fidelity verification (GL-5b / go-live G12) (2026-07-08)

Closes GL-5 finding #2's mirror image: the restore-test's live-source-config bind-override path
tripped the same PVE drop-unlisted-mountpoints rule DR did, so it boot-verified scratch guests
WITHOUT their storage mpN — weaker verification than it claimed, and GL-6 will lean on it.
**0.75.0 was superseded unpublished; this is the Day-0 manifest bump target.**

- **Archive-derived params:** the restore-test now derives its restore params exactly like DR
  bring-up — `ExtractArchiveConfig` + `drRestoreOverrides` (the ARCHIVE is the object under test,
  not any live guest's current config): rootfs explicit, every storage mpN passed through (its
  content genuinely extracted — full fidelity; the added runtime IS the verification), structural
  binds → throwaway stand-ins. Unreadable archive config / unknown bind topology → refuse UP FRONT
  (never restore a partial guest to "verify" it).
- **Mount-parity assert (the non-hollow core):** pre-start, the restored scratch's mpN set is
  compared against the archive's — a missing, mispathed, undersized, or extra mpN FAILS the test
  naming the delta. PVE rule (b) can never regress into a green light again. `MountParity`
  ("ok"|"mismatch") + `MountInventory` ride the result and the hub wire record (additive keys — an
  older hub ignores them).
- Dead code deleted with its tests: `bindMountOverrides`, `archiveVMID` + the volid regexes (the
  restore-test was their only caller; a reachable dead lookalike is how the next bug happens).
  `throwawayVolumeOverride`/`rootfsSizeGB`/`mountPathOf` live on under `drRestoreOverrides`.
- DR bring-up, 4d, KeepMAC, grows, caps: untouched (bringup.go had zero line changes — the §13
  DR re-run trigger did not fire).
- Tests: Scenario A (exact 5-param derivation from the archive + parity ok + inventory + teardown),
  B (dropped mp0 → FAIL naming it, never started, still torn down; RED-PROOF: with the parity
  assert removed the run passes silently — run→fail→revert), C (extract-failure + unknown-topology
  refusals, no restore attempted), D (provision/DR bring-up tests all green, byte-untouched);
  `mountParity` pure-function matrix (parity/missing/mispathed/undersized/extra/trivial).
- Live validation + publish: REPORT.md (full-fidelity runtime measured vs the ~3m data-less shape;
  `AGENT_VERSION`/`AGENT_SHA256` recorded verbatim for the operator's manifest bump).

## v0.75.0 — DR bring-up structural bind overrides + real-bind swap (GL-5 / go-live G8) (2026-07-08)

Implements the verdict of `felhom.eu/documentation/audits/SPIKE-dr-bindmount-source-2026-07-07.md`:
`bring-up -mode dr` of a customer archive FAILED outright because the archive carries the two
structural host-bind mountpoints (mp8 parent bind, mp9 bootstrap bind) that a `pct restore` under
the privsep token cannot recreate ("restoring 'mp8' to bind mount is only possible for root") —
bring-up passed no `MountOverrides`. Provision was never affected (the golden has no mp8/mp9; the
back-half adds them) — that asymmetry was the bug.

- **DR restore overrides** (`internal/reconcile/bringup.go`): ModeDRGuestLoss synthesizes throwaway
  1G-volume overrides for the two PLATFORM-CONSTANT mpN (mirroring backhalf.go's values; the spike's
  whole point — no archive parse for the LAYOUT) via the shared `throwawayVolumeOverride` format
  helper (extracted from `bindMountOverrides`; the restore-test's is-a-bind FILTER reads live
  configs, which DR by definition has none of). ModeProvision passes nil — regression-contract test.
- **LIVE-DISCOVERED: PVE's explicit-params restore is ALL-OR-NOTHING** (neither half was in the
  spike — it never ran an override restore). (a) mpN params without an explicit `rootfs` → HTTP 500
  "mount points configured, but 'rootfs' not set" (same rule restoretest.go:211 documents).
  (b) **Mountpoints NOT named in the params are silently DROPPED** — the first live run came up
  boot+running WITHOUT its mp0/mp1 data volumes (2m56s; the customer's world did not ride along).
  Fix: NEW `Client.ExtractArchiveConfig` (GET `/nodes/{node}/vzdump/extractconfig` — **answers 200
  under the scoped agent token**, verified live; PBS keys stay server-side, the spike's candidate-1
  rejection holds) + `drRestoreOverrides` derives the COMPLETE param set from the archive's own
  embedded config: explicit rootfs, every storage-backed mpN passed through (size + path + backup
  preserved → vzrestore extracts its content), structural binds → throwaways. Unknown bind mpN /
  unparseable size / unreadable config → clean refusal before any restore. Snapshot sections never
  shadow the current config.
- **Step 4d — real-bind swap** (DR only, pre-start): mp9 bootstrap host dir created (idempotent —
  a same-host guest-loss still has bootstrap.json there, untouched), then mp8/mp9 set to the REAL
  binds via the host runner (`pct set` — bind mounts are root@pam-only, hence NOT the API; sudoers
  already allowlists both shapes), one slot per call so a failure names the exact mpN and rolls
  back per the committed/launched envelope (C2 — never a silent half-wired success). The displaced
  throwaway volumes (PVE parks them as `unusedN`) are deleted via one config PUT (`delete=`);
  a scoped-token refusal logs the residue LOUDLY + warns in the result instead of widening
  privileges. NEW `proxmox.GuestConfig.Unused()`.
- **Engine seam:** `EngineOptions.HostRunner` + `StateDir` (DR refuses up front on an API-only
  engine); the bring-up selftest wires the same ExecRunner shape as the back-half and removes the
  scratch vmid's mp9 host dir at teardown (never a real drive's bind source).
- Tests (non-hollow): Scenario A exact-override + exact-swap-command + unusedN-delete asserts;
  provision-nil regression; C2 mid-swap rollback; C3 older-archive-without-mp9; DR-without-runner
  refusal; extract-failure refusal; 403-residue warn; archive rootfs parse (snapshot sections never
  shadow). Red-proofs: override synthesis reverted → A FAILS; unconditional overrides → B FAILS
  (both run→fail→revert).
- Live validation (campaign-2-precedent scratch DR into vmid 9310 from a real 9201 local archive,
  auto-teardown, guest 9201 untouched): pre-teardown `pct config` shows mp0 200G + mp1 50G restored
  (7m23s — content genuinely extracted), mp8/mp9 = the REAL binds (exact back-half values), rootfs
  32G explicit, ZERO unusedN; boot+running; teardown clean incl. the scratch mp9 host dir. On
  v0.74.0 the same op failed at the restore POST. Full evidence: REPORT.md.

## v0.74.0 — pool membership re-asserted after restore-over-existing (campaign-2 R2) (2026-07-07)

Closes campaign-2 finding **R2** (`felhom.eu/documentation/tests/CAMPAIGN-2-2026-07-07.md`). Pool
membership is what lets the pool-scoped `FelhomAgentGuest` token reach a guest (the grant applies only
to pool MEMBERS). `pct restore --pool` sets membership at CREATE, but a restore **over an existing
VMID** (the host-loss/finale path) never re-applies it — and no code re-added a guest to the pool — so
every destroy-restore silently dropped membership, and the NEXT `--selftest=restore-test`/DR 403'd on
`VM.Audit`/`VM.Allocate`. That empty-pool state is the true root cause of the campaign's "R1" (the
bind-mount restore failure was a symptom: with no config-read, restore-test's *existing, correct*
bind-mount neutralization never ran).

- **`Client.PoolAddVMID(ctx, pool, vmid)`** (`internal/proxmox/mutate.go`): `PUT /pools/{pool}`
  `vms={vmid}` — PVE-additive (merge, not replace), idempotent (already-member swallowed), needs
  `Pool.Allocate` (the token has it). Sync — no UPID.
- **bring-up re-asserts** (`internal/reconcile/bringup.go`): after liveness is proven, if `spec.Pool
  != ""`, call `PoolAddVMID`. A pool-add hiccup is surfaced as a LOUD warning + `res.StartWarnings`
  but must NOT flip a healthy running guest's verdict (membership matters for the NEXT op).
- **B3 (scratch teardown 403) — diagnosed, no code:** `restoretest.go` already passes
  `Pool: DefaultPool` for scratch restores, so the campaign's `VM.Allocate` teardown 403 was a
  CASCADE of the bind-mount restore failing (half-built guest outside any pool), not an independent
  gap. A re-run on the healed pool confirms.
- Tests: `PoolAddVMID` PUT shape + idempotency + real-error-surfaces + validation (`pool_test.go`);
  bring-up re-asserts when `Pool!=""` (red-proof: pre-fix no-call → FAIL, demonstrated), no-pool→no-call,
  pool-add failure warns-but-passes (liveness wins, guest kept). Role/ACL untouched — the fault was
  membership, not privileges.

## v0.73.0 — F2 mount-role fallback: enrolled user-data drives are ejectable/decommissionable again (2026-07-06)

Closes campaign finding **F2** (`felhom.eu/documentation/audits/CAMPAIGN-nomercy-2026-07-06.md` +
RERUN addendum). `roleForMountPath` (`internal/localapi/disks.go`) resolved a mount's protection role
ONLY from the PVE storage view (`Observe`), but a **bind-mounted RAW enrolled user-data drive is not
a PVE storage** → no MountPath match → fail-safe `RoleSystem` → the eject/decommission role gates
403'd **every user-data drive in the standard topology** (journal: `where=/mnt/teszt_enroll
role=system`). Customers could not eject or decommission their own drives.

- **Fallback** mirrors `durableIDForMount`'s Impl-2b: after the MountPath loop misses (on a SUCCESSFUL
  Observe), resolve the mount's backing device from the host mount table and classify **device-keyed**
  — non-`/dev` source (NAS) → system; device on the same whole disk as a KNOWN target → THAT target's
  role (containment, via new `storage.SameWholeDisk`, whole-disk granularity so a protected disk can't
  be ejected through this path); else `RoleForRawDevice`. **Fail-safe preserved**: an Observe error
  returns system BEFORE the fallback (a blind containment pass could label a backup drive user-data —
  permissive), and a mount-table-read failure or an absent/NAS mount → system.
- Scope: **only** `roleForMountPath`; gates, `deviceRole`, `DecommissionExecutor`, `classify.go`,
  `ReassertGuestBinds` untouched; the `deviceRole`/`roleForMountPath` unification is deferred.
- Tests (`f2_role_fallback_test.go`): A1 ejectable + decommission effects; B1 containment→403; B2
  system-disk→403; C1/C2 fail-safe; C3 Observe-error skips the fallback. Three red-proofs demonstrated
  (pre-fix→A1 FAIL, containment-skip→B1 FAIL, fallback-on-error→C3 FAIL). Existing RoleGated/
  Decommission tests green unmodified.

## v0.72.0 — OOB operator access: rendered operator /32 + dedicated felhom-sshd + port-adaptive belt + oob health (TASK H1) (2026-07-05)

The agent half of the merged E1+H1 operator-SSH-access feature (hub half = felhom-hub v0.35.0).
Provenance: `felhom.eu/documentation/audits/SPIKE-{felhom-sshd,oob-wg-operator-peer}-2026-07-05.md`.
Live-validated on felhom-pve + the dev endpoint (both spikes' key probes re-run as acceptance).

- **Operator /32 rendered into wg-felhom** (`internal/wgtunnel/manager.go` `renderConf`/`allowedIPsLine`):
  `oob_peer_ip` from the desired-state block is appended to AllowedIPs, deterministically SORTED
  (byte-stable conf-hash — no per-tick flap). RENDERED, not a runtime `wg set`, so it survives the
  agent's self-heal ([OF-1]; live-proven: tunnel stopped → self-heal → operator SSH still works).
- **Dedicated felhom-sshd** (`internal/felhomsshd/`): a SECOND sshd on a claimed non-22 port
  (`[8822,2222,8022,62222]`, LOUD-fail on exhaustion [SF-4]), own config/host-key/AuthorizedKeysFile
  (`/etc/felhom-sshd/authorized_keys/%u`, outside ~/.ssh [SF-3])/unit — COEXISTS with the customer's
  :22 (never touched). Config: render→`sshd -t`→**reload** (never restart-on-change [SF-2]); operator
  authorized_keys from the hub block; `reset-failed`-then-restart heal with a 10-min cooldown, NEVER
  restarting onto an invalid config. `configs/felhom-sshd.service` SAFE — **no `RuntimeDirectory=`**
  [SF-1].
- **Port-adaptive belt** (`internal/felhomsshd/belt.go` + `configs/felhom-oob.nft`): a STATIC
  `inet felhom_oob` table; the agent mutates ONLY its SETS — `@operator_ips` + `@ssh_port` [trap 4] —
  so felhom-sshd's port is reachable ONLY from the operator `/32` over `wg-felhom` (off-tunnel +
  box↔box dropped at the host; :22 untouched). Idempotent; a nil/unfetched block never empties it
  (no operator lockout).
- **OOB health** (`internal/felhomsshd/health.go`): the additive `oob` heartbeat stanza
  (`felhom_sshd_active/port/reachable/config_invalid/operator_peer_configured/operator_key_configured/
  wg_handshake_age_s`) — reaches the hub over HTTPS even with felhom-sshd/tunnel down. `reachable` =
  a listener check (the belt blocks a dial); `operator_*_configured` from persistent state
  (belt/authorized_keys), accurate immediately after a restart.
- **Sudoers**: `FELHOM_SSHD` (config/authkeys install + `sshd -t/-T` + scoped systemctl) + `FELHOM_OOB`
  (nft SET-element ops only — never a rule grant). `oob.enabled` config DEFAULT FALSE.

## v0.71.0 — management-plane break-glass: privsep-dir watchdog + mgmt_plane health (TASK G1) (2026-07-05)

Prerequisite for the felhom-sshd OOB feature (H1). Closes the lockout from
`felhom.eu/documentation/audits/SPIKE-felhom-sshd-2026-07-05.md` §8: a second sshd's
`RuntimeDirectory=sshd` removed the SHARED `/run/sshd` privsep dir and took the stock sshd on :22 down
too (sessions reset right after SSH2_MSG_KEXINIT) — a management lockout on a healthy box. Three
independent layers; this repo ships the host artifacts + the agent reporter (hub vault + surfacing are
the felhom.eu half).

- **Host artifacts** (`configs/`, installed by felhom-host-install): `felhom-privsep.tmpfiles`
  (`d /run/sshd 0755 root root -` — layer 1, boot-persistent, owned by no unit's lifecycle);
  `felhom-mgmt-watchdog.sh` (layer 2 heal action — stat-first recreate `/run/sshd`, `reset-failed`
  ssh ONLY when `failed`, write an RFC3339 heal-marker; NEVER restarts the stock sshd, NEVER touches a
  healthy dir — shellcheck-clean); `.service` (oneshot) + `.timer` (~60s, `Persistent`). The healer is
  **agent-INDEPENDENT** — it self-corrects with felhom-agent stopped (the whole point). **No unit
  declares `RuntimeDirectory=`** (that IS the incident cause; the installer refuses any that does).
- **Go** (`internal/mgmtplane/`): a read-only `Reporter` (os.Stat `/run/sshd` + read the heal-marker +
  a short TCP dial to sshd:22) producing the additive `mgmt_plane` heartbeat stanza
  (`{privsep_dir_ok, sshd_reachable, healed_recently, privsep_healed_at}` — `omitempty`, the
  SelfUpdatePending precedent, no hub-schema change). Wired always-on via
  `Collector.SetMgmtPlaneReporter`. The hub raises a warning on a new `privsep_healed_at` so a
  recurring clobber surfaces BEFORE a lockout (complements host_staleness). Touches host `/run` + the
  stock sshd only — no guests.

Closes the update asymmetry: the root-adjacent agent was updated by manual SSH binary-replace while
the lower-stakes controller already auto-updates. Design provenance:
`felhom.eu/documentation/audits/SPIKE-agent-selfupdate-2026-07-05.md` (SF-findings binding). Core
principle (as with the controller swap): the thing that performs rollback is never the thing being
updated — here systemd + an ~80-line root wrapper.

- **Trust model:** an update is an operator-signed `agent_update` op (new `reconcile.ClassAgentUpdate`,
  always Destructive) through the existing signed-jobs pipeline; the signed params pin version + sha256,
  so the sha is the ONLY integrity root (hub = dumb transport, Gitea = dumb storage — neither can
  substitute a binary). `felhom-opsign -op agent_update -agent-version <v> -sha256 <hex>`.
- **Host artifacts** (`configs/`): `felhom-selfupdate-guarded` (apply/commit/rollback — root re-verify,
  path confinement, same-fs assert, `.prev`, atomic mv, pending marker, detached restart [SF-6];
  rollback pending-guarded [SF-1]); `felhom-agent-rollback.service` (OnFailure oneshot);
  `felhom-agent-limits.conf` ([Unit]-only drop-in [SF-3] with the spike's tuned 120s/4 [SF-2] +
  OnFailure=); `FELHOM_SELFUPDATE` sudoers alias.
- **Go** (`internal/selfupdate/`): `Executor` (download → verify vs signed sha → `sudo -n` wrapper
  apply; job completed after verify+download, before apply); `Manager` (startup dwell → `commit`;
  version-mismatch → no-commit + WARN + marker-left; report seam `SelfUpdatePending()`). Wired as a
  3rd executor-chain element + a `MaybeCommit` goroutine after core init. `SelfUpdateConfig`
  (url_template/creds/dwell, Token redacted). Additive report fields `selfupdate_pending(+version)`
  (omitempty → cross-repo golden contract byte-stable, no hub change). 3 non-critical capability probes.
- **felhom.eu:** `felhom-host-install.sh` installs the wrapper + rollback unit + drop-in on day-0
  (self-update from birth); agent README "Self-update" section.
- Tests: executor (happy / sha-mismatch + companion / bad-params / wrapper-fail), gate ride-along
  (agent_update rides the LOCKED gate: pinned-key executes, non-pinned + retarget rejected), commit
  (dwell-commit / version-mismatch / no-pending / shutdown), opsign, classify. Wrapper covered by
  shellcheck + the live crash-rollback drill. Full `go test ./...` green.

## v0.69.0 — S5: host-loss DR — recovered WG-key install + directive→restore-PLAN (safe halves) (2026-07-04)

The two safe, non-destructive mechanical links for host-loss DR (the destructive in-place restore is
a separate operator-present, STOP-gated drill).

- **`internal/wgtunnel.InstallRecoveredKey`** — writes an escrow-recovered WG private key (32-byte
  base64, re-encoded canonical) to the key file so the tunnel re-establishes with the SAME
  identity/pubkey (→ the same hub `/32`), no fresh keygen. **CREATE-ONLY** — refuses if a key file
  exists (a present key may be a live identity); value never logged. Wired into `--selftest=identity-
  consume -install-wg-key` (opt-in; after `UnwrapIdentityBundle`, installs `bundle.WGPrivateKey`;
  pre-S3 blob with no WG key → logged fallback to fresh keygen + re-register, which keeps the /32).
- **`internal/dr`** (new) — consumes the host_loss `restore_directive` (was logged-and-ignored) into
  an inspectable **RestorePlan** via the `desired.Syncer.AddConsumer` raw seam: per guest →
  {vmid, archive, target storage, sizing}; per drive → {durable_id → expected mount}; + the offsite
  PBS coord. **Derive-and-surface only** — the `Consumer` has NO restore/destroy dependency, so
  "execute nothing" is structural. `guest_loss`/absent → no plan. Recipe fetched on-demand (rare
  directive) via a fresh `Collect`.
- Tests + red-proofs: WG install (same pubkey/no-keygen; present-key refuse — red-proofed against
  allow-overwrite); plan (host_loss builds; guest_loss/absent/nil-recipe → none — red-proofed against
  a relaxed mode gate). No secrets on argv/stdout/logs (field names only).

## v0.68.0 — S4.1: tier-aware restore-task deadline (unattended offsite restore-test) (2026-07-04)

The offsite restore-test couldn't complete on the scheduler path because a WAN restore of a large
guest exceeds the restore-task wait's 10-minute default — the wait expired mid-restore, teardown
then fired against a still-restoring (not-yet-pool-associated) scratch guest, and it leaked. Make
the restore-task wait **tier-aware**.

- **`internal/reconcile`**: `RestoreTestSpec.RestoreTaskTimeout` (0 → the 10m `WaitOptions` default);
  the restore-task `WaitTask` now passes it. Local-tier restores are UNCHANGED (10m — a local
  restore hanging that long is a genuine fault).
- **`internal/config`**: `BackupConfig.RestoreTestPBSRestoreTimeoutSeconds` + accessor
  `RestoreTestPBSRestoreTimeout()` (positive as-is, else **120m** — for an unattended nightly test a
  false timeout is worse than a slow pass; very large guests may need more).
- **`cmd/felhom-agent`**: `restoreTaskTimeout(cfg, tier)` sets the field to the configured PBS
  timeout only when `SourceTier=="pbs"` (both the scheduler + selftest spec builds), else 0.
- Tests: tier-aware `WaitOptions.Timeout` (pbs→120m, local→0; red-proofed against `WaitOptions{}`) +
  the accessor contract. The "grant scratch-band `VM.Allocate`" follow-up was diagnosed, not
  blind-applied — the scratch guest is restored INTO `/pool/felhom` (whose ACL already grants
  `VM.Allocate`), so the earlier teardown 403 was a *consequence* of the timeout (a not-yet-pooled,
  still-restoring guest), not a missing grant. **No ACL/host-install change.** (Live confirmation of
  the phantom in REPORT.)

## v0.67.0 — S4: namespace-aware PBS client (per-customer offsite tenancy) (2026-07-04)

Phase-1 live probe on felhom-hetzner proved the offsite tenancy path (backup/restore/list/isolation
all green over the tunnel with a per-customer DatastoreBackup token) but surfaced that the agent's
PBS client was **namespace-unaware**: `Snapshots` hit the datastore root (403 for a scoped token)
and `Verify` was whole-datastore (needs Datastore.Verify ~ admin). Operator-approved small change to
make the client namespace-scoped so a properly-isolated token services its own tenant.

- **`internal/pbs`**: `Config.Namespace` (+ `Client.namespace`). `Snapshots` appends `?ns=<ns>`
  (lists ONLY the tenant's namespace); `Verify` sends `ns=<ns>` (verifies ONLY that namespace —
  Phase-1-confirmed to work with a **DatastoreBackup** token on its own ns, no Datastore.Verify /
  admin widening). Root-ns clients (Namespace="") are unchanged → whole-datastore (the DooPlex
  `felhom-pbs` n100 path). Test `TestClient_NamespaceScoping` pins both (red-proofed).
- **`internal/proxmox`**: `Storage.Namespace` (parsed from the PVE `/storage` config key `namespace`).
- **`cmd/felhom-agent`**: `pbsTargetsFromPVE` threads `s.Namespace` into the PBS client, so a PBS
  storage configured with a namespace is verified/reported scoped to it automatically.

**Confirmed minimal tenant ACL (recorded live 2026-07-04, felhom-hetzner):** `DatastoreBackup` on
`/datastore/felhom-offsite/<ns>` (the namespace path — NOT `/ns/<ns>`) granted to **BOTH** the user
`felhom@pbs` **and** the token `felhom@pbs!<ns>` — PBS privsep tokens = intersection(user, token),
so both are required; isolation holds because each token's ACL is only its own ns (cross-ns
list/backup → 403, proven). No token exceeds DatastoreBackup; no admin on the endpoint for the box.

## v0.66.0 — S4 agent half: endpoint v4-pin + re-resolve watchdog + FELHOM_WG Critical flips (2026-07-04)

The two agent items S4 needs before offsite backups ride the tunnel (the tenancy + storage weight is
runbook-side). No new sudoers grants; no wire/JSON change.

- **`internal/wgtunnel` — v4-pin (doc 06 §4.2).** `renderConf` now takes the **pre-resolved IPv4
  literal** and writes `Endpoint = <ip>:<port>` — never the DNS name, never an AAAA. A new `Resolver`
  seam (`net.DefaultResolver.LookupNetIP(ctx, "ip4", …)` — A records only) resolves in the Manager;
  multiple A records → the numerically **lowest** (deterministic fleet-wide). renderConf stays pure
  (no DNS/IO inside). The resolved IP is cached: **steady-state Apply hits the cache — zero DNS, zero
  execs** (the load-bearing negative). DNS failure on (re)resolve → keep the last-applied conf + throttled
  ERROR — **never a teardown** (teardown stays revocation-only).
- **`internal/wgtunnel` — re-resolve watchdog (doc 06 §4.2, the slice-3 promise).** New
  `Manager.Watchdog` (loop-driven only, so Apply's zero-exec steady state is untouched): when the
  handshake age exceeds `wg_tunnel.stale_after_seconds` (default **180**) it re-resolves; **IP changed
  → re-render + restart** (endpoint re-IP recovery); IP unchanged → no churn, one throttled warn
  (endpoint merely down). The staleness read is the existing `wg show … latest-handshakes` (never
  `dump`).
- **`internal/config`** — `WGTunnelConfig.StaleAfterSeconds` (default 180 via `WithDefaults`).
- **`internal/capability` — FELHOM_WG Critical flips (S4).** Backups ride the tunnel now, so
  `wg-conf-install`, `wg-enable`, `wg-restart`, `wg-handshake-read` are **Critical=true**
  (operator-alert-worthy on degradation); `wg-tools-install` (one-time) + `wg-disable` (deliberate
  revocation) stay non-critical. `TestWGCapabilityCriticality` pins the exact set (red-proofed).
- Tests: v4-pin golden (A literal, AAAA/dns_name refused), watchdog (healthy=no-DNS negative,
  stale+re-IP restarts, stale+same-IP no-churn+throttle, resolver-failure keeps conf, initial-resolve-
  failure no-teardown+recovery). Red-proofs a/b/d all fire.

## v0.65.0 — S3.1 offsite-tunnel client MTU 1420 → 1280 (resolve the CGNAT-smoke MTU open decision) (2026-07-04)

One-constant fix closing `06 §4.3`'s OPEN DECISION. The 2026-07-04 CGNAT smoke test found the
shipped interface MTU 1420 **silently black-holes bulk TCP** on any path below ~1480 B (mobile
~1400, DS-Lite ~1452): the WG handshake and ping stay healthy (small packets) while the PBS TLS
page — and, at S4, the backup itself — drops. "Looks green, loses backups." Must be safe before
S4 flows bulk TCP over the tunnel.

- **`internal/wgtunnel/manager.go`**: new `const clientMTU = 1280` (the RFC 8200 IPv6-minimum link
  MTU — every path carries ≥1280; outer = 1280+60 v4 / +80 v6, fits every realistic path);
  `renderConf` emits `MTU = %d` from it. **Fleet-wide, family-agnostic, permanent** — decouples
  the fix from the v4/v6 endpoint-resolution question (§4.2). **Client-only by construction:** the
  interface MTU caps box→PBS and the advertised MSS (=MTU−40) caps PBS→box, so the endpoint's `wg0`
  is deliberately untouched (zero live-endpoint risk). Rejected: auto-probe / per-connection-type
  (fragile moving part optimizing throughput, a non-metric here) and MSS-clamp (no forwarded flows).
- **`internal/wgtunnel/manager_test.go`**: golden pins exact `MTU = 1280`; red-proofed (flip const
  → 1420 fails the golden on the MTU line — non-vacuous).
- **`internal/hub/report.go`**: stale "MTU 1420" comment → 1280 (still NOT a wire field).
- No wire/JSON-golden change (MTU is a client-derived constant, never on the wire); no endpoint,
  hub, controller, key, or desired-state change.

## v0.64.0 — S3 offsite WG tunnel: keygen + registration + agent-managed wg-quick@wg-felhom + escrow join (2026-07-04)

The agent half of doc 06 §3.3 (felhom.eu S1/S2 built the endpoint + hub half). **DEFAULT OFF —
the safety gate:** `wg_tunnel.enabled` defaults to false; a v0.64.0 rollout without explicit
config is a no-op (no keygen, no registration, no report stanza). Enabled explicitly on
felhom-pve only; the default flips when the production endpoint exists.

- **`internal/wgtunnel`** (new; the lanresolver host-service shape): pure-Go keygen
  (x/crypto curve25519; key 0600 in 0700 StateDir/wg; corrupt file = refuse, NEVER overwrite —
  it may be the escrowed identity; stored form canonical-clamped — x/crypto clamps derivation
  internally, so the STORED bytes are the property that matters, red-proof-anchored); one-shot
  registration (`POST /hosts/{id}/wg`, marker-gated, exponential backoff cap 15 m);
  desired-state consumption via the new `desired.Syncer.AddConsumer` raw seam (panic-contained);
  conf render golden-tested (MTU 1420, AllowedIPs = pbs_tunnel_ip/32, keepalive 25 — doc 06 §4
  client constants; all inputs strictly validated); hash-gated apply (steady state = ZERO
  execs), restart-not-reload on conf change, self-heal enable, adopt-lost-marker,
  re-key-on-mismatch. **Revocation semantics (doc 06 §3.5 completed): block absent from a
  PRESENT desired-state → disable + marker KEPT + never re-register** (the operator re-adds the
  peer using the reported pubkey); absent DATA (failed fetch) is never a teardown signal.
- **`internal/hub`**: `WireDesiredState.Wireguard` (field-exact with the S2 cross-repo golden,
  copied byte-identical + decode test), `RegisterWG` client (typed errors, token-free),
  `HostReport.Wireguard` status stanza `{pubkey, registered, active, last_handshake_age_s,
  assigned_ip}` via the collector's `WireguardReporter` seam.
- **Sudoers/capabilities**: `Cmnd_Alias FELHOM_WG` (fixed-path conf install, enable/restart/
  disable, `wg show wg-felhom latest-handshakes` — the ONLY wg read; `wg show … dump` is
  FORBIDDEN, its interface line carries the PRIVATE KEY) + 6 manifest entries (Critical=false
  until S4 makes the tunnel load-bearing).
- **Escrow**: `IdentityBundle.WGPrivateKey` (omitempty) + escrow-create auto-inject when the key
  file exists (`escrow.AttachWGKey`; field NAME only in logs). Honest limit: pre-S3 blobs cannot
  be retro-fitted (R is never retained) — S5 DR falls back to fresh-key re-registration, which
  keeps the box's /32 (hub S2 re-key-in-place).
- `--selftest=wgtunnel` single-shot for supervised bring-up.
- Live-validated on felhom-pve (agent restart tolerance, host reboot with unit persistence,
  revocation drill, 30-min keepalive soak, escrow inject) — see REPORT.md. Five red-proofs run
  + reverted. GOTCHA learned: the hub envelope's `poll_interval_seconds` (hub-side constant
  900 s) silently overrides the agent's configured cadence on the FIRST cycle — the agent-side
  `poll_seconds` is only the pre-first-heartbeat default.

## configs: build-golden.sh v2.0.0 — mandatory controller tag + bootstrap .path unit (B5 + B1) (2026-07-03)

Golden-bake script only — **no agent code, no binary, no agent version bump** (docs/config
precedent). Resolves drill findings B5 + B1
(`felhom.eu/documentation/audits/DRILL-day0-cleanroom-2026-07-03.md` §9) and closes the stale
backlog note `FOLLOWUP-golden-default-controller-tag.md`.

- **B5 — `CONTROLLER_IMAGE` (arg 6) is now MANDATORY** — the hand-bumped default rotted twice
  (0.43.0 → 0.85.1 → stale again; 0.85.1 predates the v0.86.0 floor-honoring code, which is why
  every fresh install needed the manual D.1b update). No 6th arg → die with usage (red-proofed:
  exits 1 before any `pct` op). Auto-resolving "latest" was rejected — it could bake an unvouched
  tag.
- **B1 — baked `felhom-controller-bootstrap.path` unit** (`PathExists=/etc/felhom-bootstrap/
  bootstrap.json`, enabled next to the service): the service's `ConditionPathExists` is evaluated
  only at boot, but the agent back-half hot-plugs the bootstrap mount into the already-running
  guest — the path unit starts the service when the file APPEARS, so the controller deploys with
  NO reboot (isolated systemd proof + full Day-0 proof on the clean-room drill VM; the installer's
  v1.9.1 post-provision reboot is now a redundant belt, kept). `RemainAfterExit=yes` on the service
  prevents re-trigger loops; the service itself stays unchanged.
- `GOLDEN_SCRIPT_VERSION` (2.0.0) + a `[golden]` provenance line (script version + baked controller
  tag) now open every bake transcript.
- Baked + published with controller **0.98.3**: golden `0.98.3` /
  sha256 `b9a02ef1b6f02b9b58babc4c6aad9cf6c053ebdfba116c78c8e7830de757fd01` (Gitea generic package,
  201 + sha round-trip). Clean-room validation: bake integrity (mp0+mp1 included), isolated
  hot-plug proof, full local-golden Day-0 install (first boot = 0.98.3, self-update reports
  up-to-date, app deploy OK) — evidence:
  `felhom.eu/documentation/audits/DRILL-golden-098-2026-07-03.md`.

## v0.63.0 — B3 + B2: fresh-install fixes — token reload-on-miss + guesthook snippets dir (2026-07-03)

The two agent-side gaps the Day-0 clean-room drill surfaced
(`felhom.eu/documentation/audits/DRILL-day0-cleanroom-2026-07-03.md` findings B3/B2). Both are
fresh-install path fixes; no behavior change on a warm box.

- **B3 — `TokenStore.Lookup` reload-on-miss** (localapi/tokenstore.go): provisioning is a SEPARATE
  one-shot process (`--selftest=provision`) that Mints the new guest's token into the shared
  append-only JSONL, while the long-lived daemon serves Lookup from an index built once at open —
  so the daemon 401'd every token minted after it started (the drill's
  `POST /controller/swap: HTTP 401`, until a manual `systemctl restart felhom-agent`). Lookup now
  re-reads the file ONCE on a miss (`reloadLocked()`, factored from `load()`; full re-read is
  idempotent under `apply`'s last-write-wins) and re-checks. An append-only size short-circuit
  bounds the cost: an unknown token on an unchanged file is one `stat`, no re-read — never a reload
  loop. Fix is entirely behind the `TokenAuthority` seam (no server change). Fail-closed on an
  unreadable store; missing file loads as empty. Tests (tokenstore_test.go): cross-process-mint
  coherence (red-proofed: pre-fix shape returns (0,false)), exactly-once reload bound +
  size short-circuit, cross-process re-mint rotation coherence, deleted-file no-crash
  (linux-only; windows can't unlink an open handle).
- **B2 — `guesthook.InstallSnippet` ensures the snippets dir** (guesthook/install.go): a fresh PVE
  has no `/var/lib/vz/snippets` and `install` (without `-D`) won't create parents — the pre-start
  self-heal hook silently failed to install on every freshly-bootstrapped box (warn-only in the
  back-half). A fenced `mkdir -p /var/lib/vz/snippets` now precedes the install; **sudoers gains
  exactly that one grant** (FELHOM_GUESTHOOK — configs/felhom-agent.sudoers must ship WITH this
  binary, as always). Test: mkdir-precedes-install argv assertion (red-proofed: pre-fix has no
  mkdir op).
- Guide follow-through (felhom.eu, separate commit): D.1b's "restart the agent first" step drops
  once B3 is live-verified.

## v0.62.0 — A1: pool-membership ownership check for the stale-lock reaper (2026-07-03)

Implements audit finding A1 (`AUDIT-blast-radius-hostroot-localapi-2026-07-02` §A) per the spike
verdict (`SPIKE-a1-pool-membership-read-2026-07-03` — enumeration is pool-filtered under the scoped
token, so this is defense-in-depth: a future broad-token deployment can no longer re-arm the reaper
against co-tenant guests). Companion: host-install **v1.9.0** (`Pool.Audit` added to
`FelhomAgentGuest`) — **rescope BEFORE deploying this agent**, else the reaper fail-safes (skips)
until the ACL catches up.

- **`Client.Pool`** (proxmox/query.go): `GET /pools/{name}` → `PoolInfo{PoolID, Members[]{VMID,Type}}`.
  Requires `Pool.Audit` at `/pool/{name}`; `Pool.Allocate` does NOT satisfy the read (spike T2).
- **`staleLockController.Guests()`** (localapi/stalelock.go): now returns `ListLXC ∩ pool members`
  (nonzero-vmid, non-storage entries only). Ownership is PROVEN via the pool registry, never assumed
  from enumeration scope. A pool-read failure returns a wrapped error ("pool membership read
  (pool=felhom): …") that rides the existing "guest list unavailable — skipping recovery" guard —
  fail-safe: NO unlock/snapshot-delete/start on ANY guest, never a fallback to the unfiltered list.
  One new INFO line per scan: `stale-lock: scanning pool guests` (pool, listed, scanned) — emitted
  by the controller (the unchanged `StaleLockController` seam can't carry the pre-intersect count).
- **`NewStaleLockController`** gains `(pool string, logger *slog.Logger)`; main.go threads
  `reconcile.DefaultPool`.
- **Capability surfacing**: the hub-report prober is now a composed closure — the sudo manifest
  probe + one `pve:pool-read` status (non-critical; degraded ⇒ reaper is fail-safed, visible on the
  report, no operator page). Composed in main.go; `internal/capability/` untouched.
- **`--selftest`**: new "pool read" line (pool id + member count + guest members).
- **Tests** (stalelock_pool_test.go, driving the REAL controller over a broad-token-shaped fake):
  `TestStaleLock_ForeignGuestNotReaped` (red-proved: intersect removed ⇒ FAILS with
  `pct unlock 5000` recorded), `TestStaleLock_PoolGuestStillReaped` (anti-over-filter),
  `TestStaleLock_PoolReadFails_SkipsAll` (red-proved: fallback-to-unfiltered ⇒ FAILS with mutations
  recorded), `TestStaleLockController_GuestsIntersect` (storage-member + empty-pool edges). The 9
  existing Server-level stalelock tests pass unmodified.

## docs — CLAUDE.md refresh: stable orientation, complete layout (2026-07-03)

No code change, no version bump. Deleted the version-pinned "Current: v0.31.0" narrative and the
per-slice history (stale by 30 versions — current state lives in CONTEXT.md/CHANGELOG top); Layout
completed with the 8 missing packages (capability, desired, escrow, guesthook, lanresolver, localapi,
provision, signedjobs + cmd/felhom-opsign — verified against the tree); build/deploy compressed to a
summary table pointing at the `felhom-build-deploy` skill (commands verified live on felhom-pve:
non-root `felhom-agent` user, `/usr/local/bin/felhom-agent --config /etc/felhom-agent/agent.json`).
Load-bearing Proxmox-model rules kept verbatim. Standing rule: no version-pinned state in CLAUDE.md.

## docs — REUSE.md introduced (2026-07-03)

Cross-repo reuse-map rollout (docs-only, no code change, no version bump). New `REUSE.md` at the
repo root: curated map of canonical helpers (48 rows — exec/sudoers surface, format-safety guards,
durable-id seams, stores, local-API plumbing), patterns, dangerous lookalikes (req.Device TOCTOU,
raw mkfs, uuid:-vs-byid: scheme confusion, MemoryNonceStore, pool-blind stale-lock scan…), test
seams, extension points, and observed duplication (7 clusters, NOT fixed). Every entry code-verified
at file+symbol; cited paths machine-checked by `felhom.eu/scripts/reuse_refs_check.py` (green).
CLAUDE.md gains the "See REUSE.md before writing new code" pointer + the same-commit maintenance rule.

## v0.61.0 — blast-radius audit fixes B1 + D1 + D2 + D3 (2026-07-03)

Four LOW/INFO fixes from `AUDIT-blast-radius-hostroot-localapi-2026-07-02.md` — the "second gate must
mirror the first" batch. Each shipped with a non-hollow test AND a companion red-proof (test shown
failing on the pre-fix implementation). A1 (stale-lock pool-membership) is deliberately NOT here — it
needs a pool-read spike (the role lacks `Pool.Audit`); C1/C2/A2/B2–B5/E1/E2 deferred.

- **B1 (LOW) — random temp staging for root-installed scripts.** `internal/guesthook/install.go`
  (`InstallSnippet`) and `internal/localapi/intermediary.go` (`installSharedParentUnit`, new shared
  `stageTemp`) staged root-executed scripts through FIXED, predictable `/tmp` names via `os.WriteFile`
  (no O_EXCL, follows symlinks) — a local TOCTOU into a root-run PVE hookscript / boot script. Both now
  use the `os.CreateTemp` random-name pattern `lanresolver` already used. `configs/felhom-agent.sudoers`
  (`FELHOM_GUESTHOOK`/`FELHOM_INTERMEDIARY`) install-SOURCE grants became globs
  (`/tmp/felhom-guest-hook-*.sh`, `/tmp/felhom-shared-parent-*.{sh,service}`; destinations stay pinned);
  `internal/capability/manifest.go` representative vectors updated to match. Tests:
  `TestInstallSnippet_RandomTempName`, `TestInstallSharedParent_RandomTempName` (fake runner records the
  install source: random pattern, two calls differ, content + cleanup asserted).
- **D1 (LOW) — the guarded mkfs wrapper now mirrors the classifier's member/RO classes.**
  `configs/felhom-mkfs-guarded.sh` re-checked only system-disk / LVM-PV (PATH-dependent `command -v
  pvs`) / foreign-mount — a bypassed agent could mkfs a ZFS/mdraid/LUKS/swap member or a read-only
  disk. Added (additive; nothing removed/reordered): `/sys/block/<disk>/ro` == 1 → die; an
  lsblk-FSTYPE loop over the whole disk refusing exactly `claim.go`'s `memberFSTypes`
  (LVM2_member/zfs_member/linux_raid_member/crypto_LUKS/swap — the FSTYPE catch works with pvs
  absent); pvs resolved via absolute candidates (/usr/sbin/pvs, /sbin/pvs). Validated by the new
  `scripts/mkfs-guarded-harness.sh` on felhom-pve: throwaway loop devices + PATH-shimmed lsblk + a
  recorder bind-mounted over mkfs.ext4 in a private mount namespace (no real mkfs possible) — fixed
  wrapper 8/8 (incl. plain-blank-disk still formats); pre-fix wrapper red-proof: 7/8 hostile fixtures
  reached mkfs.
- **D2 (INFO) — `classifyClaim` empty-lsblk fail-safe.** A successful-but-empty `lsblk`
  (`{"blockdevices":[]}`) skipped the member/mount loop and returned `unclaimed`.
  `internal/storage/claim.go` now refuses when the node tree is empty OR the target whole-disk is
  absent from it (undeterminable topology ⇒ claimed). Tests: `TestClassifyClaim_EmptyNodesRefused`,
  `TestClassifyClaim_TargetAbsentFromTree`.
- **D3 (INFO) — blank-format anti-retarget (AGENT-001's benign-branch twin).** `handleDiskFormat`'s
  blank branch formatted the mutable caller-supplied `req.Device` with no durable-id binding — a /dev
  re-enumeration between inspect and mkfs could format a data-bearing disk that inherited the node.
  The blank branch (`internal/localapi/disks.go`) now derives the device's durable id (no durable id ⇒
  409 refuse — path-only formats are not permitted), re-resolves it via the new
  `antiRetargetResolveBlank` (`wipe_reresolve.go`: shared `antiRetargetResolveExpect` core; the blank
  variant asserts the device is STILL !DataBearing), and formats the RE-RESOLVED device. The format
  job record (`formatjob.go`) carries `blank`; restart recovery re-checks blank jobs with the blank
  variant (durable-id-bound, fail-safe refuse). Confirmed/data-bearing branch untouched. Tests:
  `TestFormatBlankPath_AntiRetarget_{ReassignedDataBearingRefused,ReassignedDifferentDiskRefused,UnresolvableRefused,SameBlankProceeds}`,
  `TestFormat_Blank_{FormatsReresolvedDeviceNotCallerPath,ReresolveRefusalNoMkfs,NoDurableIDRefused}`.
- Deploy note: the host's `/etc/sudoers.d/felhom-agent` MUST be updated together with the v0.61.0
  binary (the old fixed-name grants deny the new random-name installs, and vice versa).

## v0.60.0 — proof-of-launch destroy gating + restore-test band-advance (campaign F1/F2) (2026-07-02)

Fixes the pool-effects campaign's HIGH finding (F1, `CAMPAIGN-pool-effects-2026-07-01.md`): the bring-up
compensating rollback and the restore-test teardown fired `DestroyLXC` on the target vmid even when
`RestoreLXC` failed synchronously WITHOUT creating anything (PVE refusing a pre-existing vmid the
pool-blind duplicate guard / band scan couldn't see) — destroying a guest the transaction never made.
Only the pool ACL's 403 saved the non-pool subset; an in-pool pre-existing guest would have been
destroyed, and any broad-token deployment re-arms the bug. Root cause: SameTxnCreated/scratch provenance
was ASSUMED, never verified against proof-of-launch. The fix makes a `RestoreLXC` UPID the sole destroy
authorization, in ALL THREE destroy paths — the pool ACL is defense-in-depth again, not the guard.

- **F1a `internal/reconcile/bringup.go` `runBringUp`:** the compensating-rollback defer is gated on
  `launched` (set only after the restore POST is accepted). A synchronous restore failure (no UPID)
  closes the owning entry terminal-failed WITHOUT any destroy. The pre-restore `OpStarted` journal
  append is kept (crash-safety); `rollbackBringUp` is now only ever called launch-proven.
- **F1b `internal/reconcile/restoretest.go` `runScratchTest`:** same `launched` gate on
  `teardownScratch` — a synchronous restore refusal never destroys the picked band vmid.
- **F1c `internal/reconcile/recover.go` `Recover`:** the no-UPID "POST never confirmed → abandon
  fail-safe" check now runs BEFORE the Scratch/Rollback dispatch — a no-UPID Scratch/Rollback entry is
  abandoned (marked failed, NO destroy) instead of destroy-by-vmid-existence. Recover is now safe by
  DESIGN, not by the pool-blind "already gone" accident the campaign observed.
- **F2 `restoretest.go` `RunRestoreTest` band-advance:** a band vmid PVE refuses with "already exists"
  (an invisible squatter — new `pveAlreadyExists`, mirrors `pveConfigLock`, never misclassifies a real
  restore failure) is skipped and the next free band vmid tried (bounded by the band width;
  `pickScratchVMID` gained an exclude set). A fully-occupied band → `Skipped` (scheduler raises no
  "backup unrestorable" alert), never FAIL — one squatter no longer permanently breaks the restore-test.
- **Accepted residual (by design):** a crash in the one-statement window between obtaining the UPID and
  journaling it leaks a half-built guest Recover won't destroy — cleanable, and vastly preferable to
  destroying an innocent guest.
- Tests: red-proof companions verified (gates reverted → `TestRunBringUp_NoLaunchNoDestroy`,
  `TestRunRestoreTest_RestoreNoLaunchNoTeardown`, `TestRecover_{BringUp,Scratch}NoUPIDAbandoned` all
  fail with the innocent-guest destroy); no-regression `…LaunchedTaskFailureStillTearsDown` + rollback
  table now includes an explicit restore-task-failure case; F2 advance + squatter-full-band-skips tests.
  Live-validated on felhom-pve (provision onto existing 9001 → no destroy armed; restore-test advances
  past a 990000 decoy). `go build/vet/test ./...` clean.

## v0.59.0 — report backing device + capacity for a registry-sourced drive in /disks (2026-07-01)

Completes the `/disks` representation for a registry-sourced (raw, no-PVE-storage) drive: the agent-view
showed "—" for the device and no size bar, because the union row never populated `backing_device` or
`total_bytes`/`used_bytes` (Observe drives get those from `pvesm status`, which a raw drive has none of).

- **`internal/localapi/disks.go` handleDisks registry union:** resolve `BackingDevice` from the fs-UUID
  (`storage.ByUUIDDevicePath`) and read capacity via `statfsCapacity` (new build-tagged
  `capacity_linux.go` = `syscall.Statfs` on the mount; `capacity_other.go` = no-op for dev builds).
- `go build/vet/test ./...` clean (Linux + Windows dev). Live: the registry drive now shows its device +
  size in the agent-view, matching the Observe-sourced drives.

## v0.58.0 — report GuestPath/BoundUnderParent for a registry-sourced drive in /disks (2026-07-01)

Last piece of first-class raw-drive support: the `/disks` union row for a registry-sourced drive (Impl-2a
— a drive with no PVE storage) omitted `GuestPath` + `BoundUnderParent`, so the controller read it as
"Leválasztva" (disconnected) even though it was mounted + bound + live in the guest.

- **`internal/localapi/disks.go` handleDisks registry union:** populate `GuestPath` (`StablePathForRaw`)
  + `BoundUnderParent` (`boundUnderParent`) on the registry row, identical to the Observe path — so a
  registry-only drive reports its true bound/active state. `go build/vet/test ./...` clean.

## v0.57.0 — re-assert a RAW drive's guest-bind (ReassertGuestBinds mount-table fallback) (2026-07-01)

Completes the raw-drive durability the v0.56.0 fix started. `ReassertGuestBinds` (the startup / drive-
returned reconcile that re-binds an enrolled drive's felhom-data under the shared parent so it's live in
the guest) built its durable-id→mount map from `Observe()` only — so a RAW enrolled drive was never found
("enrolled drive not present"), and its in-guest bind was not re-asserted after a reboot or a watchdog
re-mount (the drive would show "Leválasztva" in the controller).

- **`internal/localapi/disks.go` `ReassertGuestBinds`:** augment the durable-id→mount map from the mount
  table — each raw `/mnt/<name>` mount → its device fs-UUID (`HostReader.Mounts`+`ResolveUUID`), skipping
  the `/mnt/felhom-drives` bind (AttachDrive wants the raw path). Observe entries still win. Also: an
  Observe failure is no longer fatal (fall through to the mount-table scan) so raw drives re-assert even
  if the PVE view is momentarily unavailable.
- **Wiring fix (latent):** `buildLocalAPIServer` never passed `Options.HostReader`, so the local-API
  server's `host` was nil in production — the v0.56.0 `durableIDForMount` raw fallback (and the role gate's
  host classification) silently no-op'd. Now wired to `storage.NewProcHostReader()`. This is what makes
  the v0.56.0 + v0.57.0 raw-mount resolutions actually fire live.
- `go build/vet/test ./...` clean. With v0.56.0 (guest-bind now RECORDED for raw drives) this closes the
  reboot/reconnect guest-bind durability gap for raw drives end-to-end.

## v0.56.0 — record intent/guest-bind for a RAW enrolled drive (durableIDForMount fallback) (2026-07-01)

Surfaced by the first live raw enrollment (Impl-2b): a raw drive is not a PVE storage, so
`durableIDForMount` (Observe-based) returned "" for it → the enroll's intent + guest-bind recording
logged "durable-id unresolved" and silently skipped. Result: the drive mounted + bound + usable, but was
NOT intent-tracked (so `RegistryKnownTargets` — which gates on intent ≠ new — didn't health-track it) and
its guest-bind wasn't persisted.

- **`internal/localapi/disks.go` `durableIDForMount`:** after the Observe lookup, fall back to resolving
  the fs-UUID directly from the mount table — the device mounted at `where` (via `HostReader.Mounts`) →
  its by-uuid identity (`HostReader.ResolveUUID`) → `uuid:<fs-uuid>` (the SAME scheme Observe derives, so
  intent keys stay consistent). Fixes intent recording (enroll/eject) AND guest-bind recording for raw
  drives; the PVE-storage path is unchanged.
- Test `TestDurableIDForMount_RawFallback` (+ red-proof: Observe-only → ""). `go build/vet/test ./...` clean.
- **(Residual noted here fixed in v0.57.0:** `ReassertGuestBinds` raw-drive guest-bind re-assert.)

## v0.55.0 — raw-device discovery + registry-sourced drive tracking (Impl-2a) (2026-07-01)

Agent backend for drive enrollment (SPIKE-drive-enrollment §SQ1/SQ4/SQ5). Makes raw (non-PVE-storage)
drives (a) discoverable for enrollment and (b) health-tracked WITHOUT being a PVE storage — so a drive
enrolled the new way isn't enrolled-but-untracked (the 3b-fix false-detach class). No mkfs here (Impl-1
owns it); the controller wizard rewiring is Impl-2b.

- **`GET /disks/candidates`** (`internal/storage/candidates.go` + `internal/localapi/disks.go`):
  enumerates host whole-disks from `/sys/block`, runs the Impl-1 unclaimed filter, and returns the free
  ones with probe info (size/model/FS/data-bearing/durable-id preview), split into `initialize` (all
  unclaimed) and `attach` (the subset carrying a mountable ext4/xfs FS). Fail-safe carries through (a
  device not provably unclaimed is omitted).
- **`RegistryKnownTargets`** (`internal/storage/registry_known.go`): the watchdog's known-DRIVE set now
  comes from the intent registry + Felhom `.mount` units, NOT `Observe()` (PVE storages). A unit is
  tracked iff its intent ≠ `new` (enrolled/ejected/decommissioned; the watchdog's existing IntentReader
  gate still decides re-mount). `main.go` swaps the watchdog `KnownTargets` source. **`Observe()` is
  KEPT** for real PVE storages (local/local-lvm/pbs) — reports + the `/disks` view.
- **`handleDisks` union:** additive + deduped-by-mount-path — appends registry drives Observe doesn't
  surface (a registry-only drive now appears in the agent-view) without dropping any Observe row (can't
  regress the current view).
- **Existing-drive migration** (`ReconcileExistingDrives`, idempotent, at agent start): records each
  currently-mounted Felhom-unit drive as `enrolled` so the registry-sourced `Known()` tracks it without
  its legacy PVE dir-storage. Does NOT create/remove PVE storages.
- **Tests:** RegistryKnownTargets (enrolled tracked / new excluded / ejected tracked) + the **red-proof**
  (Observe-based `Known()` misses a drive with no PVE storage; the registry provider tracks it);
  migration idempotency; candidates init/attach split. `go build`/`vet`/`test ./...` clean. Watchdog/
  HostLiveness/Remounter unchanged (only the source injected).

## v0.54.0 — format-safety foundation: unclaimed-disk guard + guarded-mkfs wrapper (2026-07-01)

Impl-1 (SPIKE-drive-enrollment-2026-07-01). Hardens the destructive `Format`/mkfs path BEFORE the
enrollment feature: today `Format` delegates authorization to its caller and only checks `DataBearing`
(has-data), which is insufficient — the OS disk is data-bearing yet catastrophic — and the sudoers
permits `mkfs /dev/*`. Two independent, layered guards:

- **Part A — mandatory unclaimed-disk guard inside `Format` (the primary safety).** New
  `internal/storage/claim.go`: `classifyClaim` (pure) + `gatherClaimFacts` refuse to format any device
  not provably UNCLAIMED — reusing `SystemDisks` (OS disk) + lsblk FSTYPE (LVM2_member/zfs_member/
  linux_raid_member/crypto_LUKS/swap) + foreign mounts + read-only + authoritative `pvs`/`zpool`. A
  Felhom-owned mount under `/mnt/felhom-drives` is NOT a foreign claim (re-init stays allowed; the
  DataBearing wipe-confirm still gates data loss). **FAIL-SAFE: any read error / undeterminable topology
  → CLAIMED → refuse.** The guard is in `Format` (lowest layer), not the handler, so no caller can
  bypass it. Read-only sudoers additions: `pvs`, `zpool status` (in `FELHOM_DISK`).
- **Part B — guarded-mkfs wrapper below the agent.** `configs/felhom-mkfs-guarded.sh` (root, 0755) is
  now the ONLY mkfs path the sudoers allows (`FELHOM_FORMAT` no longer allowlists raw `mkfs.*`). It
  re-checks the cheap catastrophic cases (system disk / LVM PV / foreign mount) and refuses — so even
  an agent bug/compromise can't mkfs the OS disk. `Format` execs the wrapper (`<device> <fstype>`) via
  `Binaries.MkfsGuarded`.
- **Tests:** `claim_test.go` — table-driven `classifyClaim` (every claim signal + fail-safe + the two
  allow cases) incl. the **red-proof** (a claimed, non-data-bearing OS disk: removing the isSystem check
  flips it to allowed → test fails, proving the guard adds safety beyond `DataBearing`); Format-guard
  integration tests (refuses system disk / LVM member, allows unclaimed → wrapper invoked); capability
  manifest updated (mkfs sample → the wrapper). `go build`/`vet`/`test ./...` clean.
- **Deferred to Impl-3:** a raw disk passed through to ANOTHER VM looks unused to the host — a host-level
  filter can't detect it; the operator gate (shared-box mode) closes that. Impl-1 closes everything
  host-visible (a strict improvement over today's no-guard state).

## v0.53.0 — restore guests INTO the felhom pool (pool-scoped-ACL enabler) (2026-07-01)

Colleague-safety batch #4 phase b (agent half). Enables the agent token to be scoped from `/` to
`/pool/felhom` + `/storage/<targets>` (real blast-radius containment on a shared host) by making every
restore allocate the guest INTO the pool — the only way a fresh vmid authorizes under a pool-scoped
token. Grounded by `felhom.eu/documentation/audits/SPIKE-pool-scoped-acl-2026-07-01.md` (PASS).

- **`internal/proxmox/mutate.go`:** `RestoreLXCOptions` gains `Pool string`; `RestoreLXC` sends
  `pool=<p>` only when non-empty (`pct restore --pool`). Omit-when-empty (a broad-token restore needs
  no pool) — unit-tested + red-proofed.
- **`internal/reconcile`:** new `const DefaultPool = "felhom"` (single source of truth); `BringUpSpec`
  gains `Pool`, threaded to the bring-up restore. **Both restore sites** now pool the guest: the
  provision/DR bring-up (`Pool: spec.Pool`, set to `DefaultPool` by the CLI) AND the **restore-test**
  scratch guest (`Pool: DefaultPool`) — the latter closes SPIKE residual #2 (a pool-scoped token would
  otherwise 403 on the out-of-pool scratch guest).
- **`cmd/felhom-agent/main.go`:** both `BringUpSpec` literals (bring-up/DR + provision) set
  `Pool: reconcile.DefaultPool`.
- **No ACL/priv change in the agent** — that ships in the host-install script (v1.6.0). The pool param
  is INERT until the token is granted `Pool.Allocate` at `/pool/felhom` and the pool exists; publishing
  is therefore safe ahead of the coordinated ACL swap.
- New tests: `proxmox.TestRestoreLXC_PoolParam` (set → `pool=felhom`; empty → omitted),
  `reconcile.TestRestoreSitesUsePool` (both restore sites carry `DefaultPool`). Both red-proofed
  (unconditional `Set` → omit test fails; drop either site's `Pool` → both-sites test fails). `go
  build`/`vet`/`test ./...` clean.

## v0.52.0 — operator-opt-in CPU/RAM cap for the provisioned guest (`-cores` / `-memory`) (2026-07-01)

Colleague-safety batch #3. So a trial appliance guest on a colleague's SHARED production Proxmox does
not pressure his existing guests, the operator can now cap the guest's CPU cores + RAM **at provision
time, before the guest's first boot** (the peak container-pull moment). Pure CLI→spec plumbing — the
reconcile engine already applied the cap; this only wires the flags to it.

- **`cmd/felhom-agent/main.go`:** new `-cores N` / `-memory M` (MiB) flags for
  `--selftest=bring-up|provision` (0 = keep the golden's baked size). They flow through `bringUpSizing`
  (now carries `Cores`/`MemoryMB`) into the `reconcile.BringUpSpec{Cores,MemoryMB}` built by BOTH
  `runSelftestBringUp` and `runSelftestProvision`. The `-selftest` usage string documents them.
- **No engine change.** `internal/reconcile/bringup.go` already carries `BringUpSpec.Cores/MemoryMB`
  (0 = leave as restored) and `buildBringUpConfig` already emits `cores`/`memory` into the SAME
  coalesced config PUT as the identity reset — which runs BEFORE `e.api.Start`, so the cap lands
  pre-boot. Rejected the `pct set`-post-provision alternative (runs after boot = an uncapped window;
  bypasses the token/audit; second config source of truth).
- **Omit-when-zero guarantee:** an unset cap (0) emits NEITHER `cores` NOR `memory`, so an uncapped
  provision keeps the golden defaults (no regression for the normal single-purpose box) and can never
  shrink the guest to 0 cores. New pure-function test `TestBuildBringUpConfig_ResourceCaps` asserts
  both the set (`cores=2`,`memory=4096`) and the absent-when-unset cases; a red-proof (unconditional
  emit) was run and confirmed to fail the omit assertion, then reverted.
- **Deploy dependency:** a FRESH host-install `--cores`/`--memory` (felhom.eu script v1.4.0) requires
  the hub artifact manifest to serve **agent ≥ v0.52.0**, else the old agent rejects the unknown flag.
  The flags are opt-in, so nobody hits this until they intentionally cap.
- `go build` / `go vet` / `go test ./...` clean.

## v0.51.0 — local vzdump retention default (`--prune-backups keep-last=3`) (2026-06-30)

The PREVENTIVE counterpart to the hub's host_disk + storage_fill detectors: the agent's periodic local
whole-guest vzdump now prunes its own old archives, so a box can't refill its own root via its own backups
(the felhom-pve incident's root cause — that vzdump carried no retention, ~18 dumps piled under
`/var/lib/vz/dump`).

- **`internal/proxmox/mutate.go`:** `VzdumpOptions.PruneBackups` → passed as PVE's `--prune-backups` on
  the vzdump POST (vmid+storage scoped, so PVE prunes only THIS guest's archives on THIS storage).
- **`internal/backup/runner.go`:** `NewBackupRunner` gains a `retention` arg; the backup applies it via
  `localPruneSpec` ONLY when the target is a **non-PBS** storage (resolved via `ListStorage`) — PBS offsite
  retention is a separate lifecycle and is never pruned by the per-run flag. **Fail-safe:** if the target
  type can't be confirmed (lookup error / not found) the run SKIPS pruning rather than risk pruning PBS
  (the detectors remain the safety net). Only the periodic local-API runner sets retention; the
  restore-test / selftest runners pass "".
- **`internal/config/config.go`:** `backup.local_backup_retention` (keep-last N) with `KeepLast()` clamped
  to **≥1** (0/unset/negative → default 3) — a mis-config can NEVER prune the just-made backup —
  + `PruneBackupsSpec()` → `keep-last=N`. Wired into the local-API backup runner (`main.go`).
- **Seeding:** `felhom.eu scripts/felhom-host-install.sh` seeds `local_backup_retention: 3` in the agent
  config; the code default also protects any box where it is unset (KeepLast → 3) from day 0.
- F2-b stale-vzdump-lock recovery untouched.
- Tests: the local vzdump carries `--prune-backups keep-last=3` (+ companion: no-retention runner emits no
  prune); **PBS is never pruned** (+ companion: same retention on a local target IS applied);
  fail-safe-on-unknown-target; the **keep-last≥1 clamp** companion. `go build/vet/test ./...` green.

## v0.50.0 — NAS network storage Part A1: NFS/SMB automount foundation (2026-06-30)

Agent foundation of the validated `SPIKE-nas-storage-2026-06-29.md` (verdict READY): a customer NAS can
serve **bulk media** to a media app. The agent mounts a NAS share **host-side** under
`/mnt/felhom-drives/<name>` via a systemd `.automount` (+ `.mount`) pair; it propagates into guest 9201 for
free through the existing shared `mp8` bind (no new mountpoint, no restart). A NAS is a **distinct storage
class** — it carries **no durable-id** and never enters the drive enroll/eject/decommission/wipe/SMART/
watchdog machinery. **Bulk-media class only; STOP before A2 (controller registry/UI) + B (restic-SFTP).**

- **`internal/storage/netmount.go` (NEW).** `NetworkMountSpec` + the locked SPIKE recipe:
  - **NFS (preferred):** `What=server:/export`, `Type=nfs4`, `Options=vers=4.1,soft,timeo=50,retrans=2,noatime,_netdev`.
    `soft` is the failure-isolation knob (clean EIO, never a `df`/guest wedge); a default `hard` mount is
    never emitted. The `+100000` uid mapping is the **export's** job (`anonuid=101000`), so the client mount
    carries no uid.
  - **SMB (fallback):** `What=//server/share`, `Type=cifs`,
    `Options=vers=3.0,credentials=<0600 file>,uid=<+100000>,gid=<+100000>,forceuid,forcegid,file_mode=0664,dir_mode=0775,_netdev`
    (plain octal modes, never setgid 2775). **The +100000 rule** (container uid/gid N = host N+100000): a
    container uid 1000 renders `uid=101000` so the guest sees its native id and reads+writes; a naïve `+0`
    lands as `nobody:nogroup` (not writable) — the documented trap, asserted by a companion test.
  - **`.automount` with `TimeoutIdleSec`** (on-demand + idle-unmount): an idle NAS reboot is a non-event.
  - **per-share liveness** (`ListNetworkMounts`): TCP-probes the NAS endpoint (2049/445) + reads
    `/proc/mounts` — it never `stat`s the (possibly EIO/D-state) mountpoint, so a black-holed NAS cannot
    wedge a list. Health `ok | idle | unreachable`, scoped to the affected share, never box-wide.
  - **role gate** `NetworkMountRole`: network storage is **bulk-userdata only** — confined to the
    `/mnt/felhom-drives` namespace; any other target is refused (most-protected).
  - Full validation (`ValidateNetworkMountSpec`) before any unit is rendered: share name (safe segment),
    server, NFS export (absolute, no traversal) / SMB share name, uid/gid range, creds path.
- **Drive-machinery bypass (Scenario D).** `parseFelhomMountUnit` (the host-reboot drive re-assert's
  classifier) explicitly refuses any unit carrying the network marker, so a NAS mount is never given a
  durable-id, SMART-probed, or re-asserted as a drive. Companion red-proof: the same by-uuid-shaped unit
  with the drive marker DOES parse — the guard is the discriminator, not luck.
- **`internal/localapi/netstorage.go` (NEW).** Self-scoped endpoints `POST /netstorage/add`,
  `GET /netstorage`, `POST /netstorage/remove`. SMB credentials are written **out-of-band** to a 0600 file
  the agent owns (never in git, never in a plaintext registry, never logged). Role-gated to the user-data
  namespace.
- **sudoers:** new narrow `FELHOM_NETMOUNT` alias (install/enable/disable/stop the `.automount` + remove the
  felhom mount-unit files; the `.mount` half reuses `FELHOM_MOUNT`, the mountpoint mkdir reuses
  `FELHOM_INTERMEDIARY`). `visudo -cf` clean.
- **config:** `privileged.smb_creds_dir` (default `/var/lib/felhom-agent/smb-creds`).
- **Runtime deps:** `mount.nfs` (nfs-common) + `mount.cifs` (cifs-utils) present on the host (confirmed live).
- Tests: exact NFS/SMB option-set string-asserts + the +100000 companion; validation matrix; role gate;
  unit round-trip + health; the drive-machinery guard + companion; Ensure/Remove command sequences.

## v0.49.0 — reboot-during-backup stale-lock recovery (F2-b) + shared-parent script redeploy fix (F2-a) (2026-06-30)

Closes the two host-reboot findings from `TESTRUN-fullstack-2026-06-29.md`.

- **F2-b — startup stale-lock recovery (`internal/localapi/stalelock.go`, NEW).** A host reboot DURING a
  vzdump backup leaves the guest with a `snapshot-delete`/`backup` lock + a dangling `vzdump` snapshot;
  `onboot:1` then can't start the locked CT → the customer box stays DOWN until a human runs `pct unlock`.
  The agent now self-heals at startup (`Server.RecoverStaleLockedGuests`, called alongside
  `ReassertGuestBinds`/`RecoverFormatJob`): for each guest carrying a backup lock, **only when no vzdump is
  genuinely in-flight** (the load-bearing invariant — at startup the agent's own backup loop hasn't run, so
  the lock is stale; the guard fails SAFE if it can't confirm), it `pct unlock`s → deletes the dangling
  `vzdump` snapshot (API + WaitTask) → starts the CT **iff** `onboot` and not already running. Scope is
  strictly the two vzdump locks; `migrate`/`disk`/`create`/… are left untouched. Idempotent.
  - **`internal/proxmox`:** new reads `GuestConfig.Lock()`/`OnBoot()`, `Client.ListSnapshots`,
    `Client.ListRunningTasks`, and the `Snapshot` type. Reads + snapshot-delete + start go through the API
    token; only `pct unlock` shells out (no API equivalent).
  - **sudoers + capability manifest:** new narrow grant `FELHOM_STALELOCK = /usr/sbin/pct unlock [0-9]*`
    and Critical capability `stalelock-unlock` (a stuck-locked guest = customer box down). `visudo -cf` clean;
    covered by the manifest↔sudoers build gate.
  - Tests: recovery sequence + companions — no-lock touches nothing; non-backup lock left alone; onboot=0
    unlocked-but-not-started; delsnapshot only when a snapshot exists; **invariant guard** (live backup →
    not cleared; unconfirmable → fail-safe); already-running → not restarted.

- **F2-a — shared-parent boot script never redeployed (`internal/localapi/intermediary.go`).** The host's
  `/mnt/felhom-drives` was still in root's `shared:1` peer group (so every drive bind DOUBLED) because the
  live boot script predated the v0.36.6 `make-private` fix. Root cause: `EnsureSharedParent` gated the
  (re)install on the **unit** file only, so a script-only change never deployed. Fixed: the new
  `sharedParentInstallStale` helper compares **both** the script and the unit (missing or differing →
  reinstall). Boot-time-only — it rewrites the on-disk script; it does NOT churn the live mount (the live
  bind/make-private/make-shared stays guarded on `!isHostMountpoint`). Verified empirically on the host:
  the correct `bind → make-private → make-shared` sequence gives the parent its own group + no doubling.
  - Tests: a stale-script/current-unit case triggers reinstall (the F2-a regression); both-current is a
    no-op; missing files are stale; a content guard asserts the shipped script keeps `make-private`.

- **Live-caught fixes (same version, found during felhom-pve validation):** PVE 9.x rejects
  `GET /nodes/{node}/tasks?running=1` (HTTP 400 "property not defined in schema") — the invariant guard
  now uses `?source=active`. And the unprivileged-LXC start emits a benign `WARNINGS: 1` (systemd-nesting)
  advisory that false-failed the recovery's start — `Start` now uses `AllowWarnings` (matching the
  restore-test's start step).
- **§D supervised reboot — both findings live-validated.** F2-a: after reboot `/mnt/felhom-drives` came
  up as its OWN peer group (`shared:94`, not `shared:1`) with no doubling; guest sees both drives, apps
  healthy. F2-b: a reboot with the exact stale state (induced `snapshot-delete` lock + a real dangling
  `vzdump` snapshot) reproduced the stuck symptom (pve-guests "CT is locked (snapshot-delete)" → start
  failed) and the agent auto-recovered — unlock → removed the real dangling snapshot → started the CT.
  Zero spurious operator pages on the reboots. See
  `felhom.eu/documentation/audits/TESTRUN-fullstack-2026-06-29.md`.
- Version `0.48.0 → 0.49.0`.

## v0.48.0 — report the served local-API leaf fingerprint (hub-side re-key detection, Part A) (2026-06-29)

The agent now rides its **served leaf fingerprint** on every host report so the hub can detect an
agent re-key fleet-wide (the last self-health leg — `host_leaf_changed`, hub v0.22.0).

- **`internal/hub/report.go`:** new `HostReport.LeafFingerprint string` (`leaf_fingerprint`) — the
  SHA-256 of the leaf the agent currently serves. Empty when the local API is disabled (no leaf) → the
  hub treats "" as unknown, never an alert. Not a secret.
- **`internal/hub/collect.go` + `cmd/felhom-agent/main.go`:** `Collector.SetLeafFingerprint(fp)` threads
  the `fp` from `EnsureLeaf` (the SAME value the loud LOADED/REGENERATED log reports) into every report,
  next to `Capabilities`.
- Tests: the report includes the fp when set, `""` when unset (local API disabled); golden + contract +
  field-names tests updated (cross-repo golden mirrors `leaf_fingerprint`). Version `0.47.0 → 0.48.0`.

## v0.47.0 — controller-swap verify hardening: reject a crash-looping no-healthcheck image (F1) (2026-06-29)

Closes F1 from the no-mercy testrun: a controller image with **no HEALTHCHECK that crash-loops** could
land a single "Running" inspect poll → the swap marked it healthy → **no rollback** (alpine tagged as
the controller passed in ~4 s, then `Restarting (0)`). The real controller image has a healthcheck so
the live severity is low, but the rollback safety net had a hole.

- **`internal/localapi/controllerswap.go`:** `controllerHealthy` now also reads `{{.RestartCount}}` (a
  4th `docker inspect -f` field) — `running && RestartCount>0` → not-ok (a process that has already
  crash-restarted isn't stably up, regardless of healthcheck). It also signals `needsDwell` for the
  no-healthcheck (`none`) case. `verify` adds a **stability dwell**: a no-healthcheck image must report
  ok on `verifyDwell` (=3) **consecutive** polls before it's accepted; a real `healthy` result is
  trusted immediately (Docker already gated it). Any not-ok resets the dwell. Timeout → existing
  rollback path runs. No change to writeImage, the sudoers grants (the `*` in `docker inspect -f *`
  spans the extended template — confirmed live), or the state-file/rollback orchestration.
- Tests: F1 **red-proof** (`RestartCount>0` → verify false; companion: rc=0+dwell=1 verifies → the rc
  check is what blocks it); the **dwell** (single ok then crash → verify false; companion dwell=1
  accepts it); a real `healthy` image verifies promptly (no false rollback). Existing
  `RollbackOnUnhealthy` / `HealthyWithNoHealthcheck` stay green. Version `0.46.0 → 0.47.0`.

## v0.46.0 — leaf lifecycle: signal + loud-log a regenerated leaf (prevention, Part B.1) (2026-06-29)

Makes an accidental local-API leaf **regeneration** (the 2026-06-28 root→non-root migration class —
moving `/var/lib/felhom-agent` aside silently minted a new leaf → every controller's pin invalidated
for days) **visible immediately** instead of silent.

- **`EnsureLeaf` now returns `generated bool`** (`internal/localapi/cert.go`): false = an existing
  pair was LOADED (stable fingerprint), true = a fresh leaf was GENERATED.
- **Loud call-site (`cmd/felhom-agent/main.go`):** a load logs `INFO local-api leaf LOADED`; a
  regeneration logs **`WARN local-api leaf REGENERATED — any previously issued bootstrap pins are now
  INVALID; controllers will fail the pin check until re-bootstrapped`** (with the new fingerprint).
- No new sudo/capability surface — pure return + log change. The companion install-script preservation
  (`--preserve-state-from` + the populated-host guard) lives in `felhom.eu/scripts/felhom-host-install.sh`.
- Tests: `EnsureLeaf` first call `generated==true`, second `generated==false` AND **same fingerprint**
  (persistence keeps the pin stable). Version `0.45.0 → 0.46.0`.

# Changelog

All notable changes to **felhom-agent** are recorded here. Update on every code
change that gets pushed.

## v0.45.0 — controller-swap under non-root: stdin `tee` write + narrow sudoers grants (Option A) (2026-06-29)

Restores fleet controller-swap / managed auto-update under the **non-root** agent — the one capability
the 2026-06-29 sudoers audit deliberately left broken because the old write vector needed arbitrary
in-guest execution. Mechanics spike-proven
(`felhom.eu/documentation/audits/SPIKE-controllerswap-narrow-grants-2026-06-29.md`, GO). **No controller
change** — the swap endpoint contract is unchanged; only the agent's internal write mechanism + the
allowlist.

- **`writeImage` no longer shells out.** Was `GuestExec("bash","-c","printf '%s\n' '<img>' > <file>")`
  (the swap's only interpolated/shell vector). Now
  `GuestExecStdin(strings.NewReader(img+"\n"), "tee", "/etc/felhom-controller-image")` — the image ref
  is piped on **stdin** into an in-guest `tee`; no shell, no interpolation. The trailing `\n` keeps the
  on-disk bytes byte-identical to the golden's `printf '%s\n'`, and the bootstrap reads `IMAGE=$(cat …)`
  (newline-stripping), so the write is consumed identically. `ValidControllerImage` still gates upstream.
- **New stdin seam (no fenced-runner bypass):** `proxmox.Runner.RunStdin` / `ExecRunner.RunStdin` (Run
  with `cmd.Stdin`), `GuestBinder.GuestExecStdin`, and `GuestExecutor.GuestExecStdin` — the swap routes
  stdin through the SAME `sudo -n` fenced runner as every other privileged op.
- **`FELHOM_CONTROLLERSWAP` sudoers alias (5 narrow, auditable grants):** `cat <fixed file>`,
  `docker image inspect *`, `docker inspect -f *`, `systemctl restart <fixed unit>`,
  `tee <FIXED image file>`. **No general `pct exec`, no `bash -c`** — the spike's negative controls
  (arbitrary exec, `tee` to any other path, `docker rm`, `rm -rf`) stay denied. The 5 are added to the
  v0.44.0 **capability manifest** (Critical — a silently-broken fleet auto-update is operator-alert-worthy),
  so the self-probe watches them and the build-test asserts grant↔code coverage (companion red-proof:
  dropping the `tee` grant fails the gate — demonstrated red→green on the real file).
- Existing swap tests (happy / rollback-on-unhealthy / image-absent / no-healthcheck / bad-image /
  single-flight) pass over the new write path; a new test asserts the write is stdin-`tee` with exact
  `image\n` and **no** shell vector. Version `0.44.0 → 0.45.0`.

## v0.44.0 — privileged-capability self-probe (build-time manifest test + runtime probe + hub snapshot) (2026-06-29)

The agent now self-checks the `sudo -n` grants it depends on, so a missing allowlist entry (the
2026-06-28 cutover class: lxc-info/make-private/…) is caught LOUD — in CI at build time and on the
host at runtime — instead of surfacing days later as user-visible breakage. **First slice of agent
self-health; the controller↔agent channel check is a separate later task.**

- **`internal/capability` (NEW):** a `Manifest()` of the required `(binary, representative-arg)`
  vectors (seeded from the 2026-06-29 audit — the OK + CLOSED rows; the SURFACED/DEFERRED rows
  `pct exec *`/`pct create`/`mount UUID`/`sensors` are deliberately excluded). `Prober.Probe` lists
  each against the live policy with `sudo -n -l -- <binary> <args>` (a policy LIST — **never
  executes**, safe for mkfs/pct entries) via a DIRECT runner, plus an `os.Stat` existence check,
  mapping to `ok` / `degraded` ("sudo policy denied" | "binary not found"). A total sudo failure
  (drop-in missing) collapses to ONE aggregate signal. Serve-degraded: the probe never blocks
  startup, panics, or errors.
- **Build-time gate (`manifest_test.go`):** parses `configs/felhom-agent.sudoers`, translates each
  glob to a regex, and asserts **every manifest vector is covered by a grant** — exactly what would
  have caught the dropped `lxc-info`/`make-private` lines in CI. Includes a **red-proof**: with the
  `lxc-info` line removed from an in-memory copy, the check FAILS for `guest-init-pid` (and passes
  on the real file) — proving the gate is not hollow.
- **Runtime wiring:** `Probe` runs once at startup (INFO `capabilities self-check N/N ok`, plus an
  ERROR per degraded capability naming the gated feature) and on every hub-report cycle; the snapshot
  rides the report as the new non-nil `HostReport.Capabilities []capability.Status` (golden +
  contract test updated; cross-repo hub copy mirrors it).
- **No allowlist change**; the live host is post-audit complete, so the probe reports N/N ok — itself
  a live proof the probe agrees with the fixed sudoers. Version `0.43.0 → 0.44.0`.

## (sudoers completeness audit, folded into v0.44.0) — close non-root allowlist gaps (2026-06-29)

A full audit of every privileged command the agent shells via `sudo -n` against
`configs/felhom-agent.sudoers`, closing the read-only/fixed-vector gaps left by the 2026-06-28
root→non-root cutover. **Sudoers-only change — no Go change, no version bump** (the file is fetched
canonically by the host-install script). Root cause of the multi-drive "attach one, the other drops"
symptom (audit `felhom.eu/documentation/audits/SPIKE-multidrive-mutual-exclusion-2026-06-29.md`): the
allowlist was incomplete, so several `sudo -n` calls were denied under the non-root user.

- **`lxc-info -n [0-9]* -p -H` → FELHOM_INTERMEDIARY (THE root-cause fix).** `guestInitPID`
  (`intermediary.go:256`) shells this to resolve the guest init PID for
  `GuestSeesMount`→`bound_under_parent`. It was absent from the allowlist → `sudo -n` denied → empty
  PID → **every external drive reported absent** → the controller drive-gate stopped each drive's apps
  (flapping). With the grant, `bound_under_parent` reports truthfully and the gate quiesces.
- **`mount --make-private /mnt/felhom-drives` → FELHOM_INTERMEDIARY.** `EnsureSharedParent`
  (`intermediary.go:110`) calls it to isolate the shared parent's peer group on first setup; the
  allowlist had only `--make-shared`, so the parent stayed in root's peer group and host submounts
  "doubled". Guarded by a mountpoint check (never re-churns a live parent).
- **`systemctl restart dnsmasq` → FELHOM_DNSMASQ.** The v0.29.x LAN-DNS fix switched `reload`→`restart`
  (`lanresolver.go restartDnsmasq`) but the allowlist still only permitted `reload` → split-horizon
  DNS self-heal was silently denied under non-root. Added alongside the retained `reload`.
- **`pct set [0-9]* -onboot 1` → FELHOM_PROVISION.** The provision back-half (`backhalf.go`, F3
  auto-start) sets onboot; only `-mp[0-9]*` was allowed → denied under non-root.
- **`pct reboot [0-9]*` → FELHOM_GUESTHOOK.** `RebootGuest` (`disks.go:448`, the enroll "activate
  pending binds" fallback) was unmatched.

**Surfaced for operator decision (NOT added — would require arbitrary root-in-guest):** `GuestExec`'s
general `pct exec [0-9]* -- <…>` (controller-swap self-update / Phase-2 managed updates) runs variable
vectors incl. `bash -c "<interpolated>"` — granting it = arbitrary execution. **Controller-swap is
currently broken under the non-root agent** until narrow per-vector grants are decided. **Deferred:**
`sensors -j` (defined-but-unwired AND `lm-sensors` not installed on the host — no live caller, path
unverifiable). **Not added (no daemon caller):** `pct create …` (CreateGoldenLXC, maintenance/broad),
`mount UUID=… …` (MountUSBByUUID, legacy/unreferenced). Full audit table in `REPORT.md`.

## v0.43.0 — canonical systemd unit + binary published to Gitea (BUNDLE slice) (2026-06-28)

Day-0 no longer needs a hand-installed agent. The agent binary is now PUBLISHED to Gitea as a generic
package and the host-bootstrap script fetches → verifies (sha256 vs the hub-vouched manifest) →
installs it. This commit adds the **canonical systemd unit** (was hand-made per host) and the publish
tooling; the binary itself is a version-only rebuild (no behavioural change).

- **`configs/felhom-agent.service` (NEW, canonical):** `User=felhom-agent`/`Group=felhom-agent` (the
  documented non-root production model — README "Process model"; `privileged.mode: "sudo"` + the
  narrow sudoers allowlist), `ExecStart=/usr/local/bin/felhom-agent --config
  /etc/felhom-agent/agent.json`, `After=network-online.target pve-cluster.service pveproxy.service`,
  `Restart=on-failure`, `StateDirectory=felhom-agent`. **Deliberately NO sandboxing**, with the reasons
  documented inline:
  - `NoNewPrivileges` is NOT set — it would block the setuid `sudo` the agent needs for every host-root
    op (mount/format/pct/dnsmasq), silently killing all privileged capability.
  - NO mount-namespacing hardening (`ProtectHome`/`ProtectSystem`/`PrivateTmp`/…) — any of those give the
    unit a PRIVATE mount namespace, and the intermediary-mount drive model relies on `mount
    --make-shared`/`--bind` propagating into the running guest; in a private namespace every drive
    enrollment would silently break. The agent shares the host mount namespace; the sudoers allowlist is
    the security boundary.
- **`scripts/publish-agent.sh` (NEW):** builds (optional) + PUTs the binary to
  `/api/packages/admin/generic/felhom-agent/<ver>/felhom-agent` (Gitea generic), prints
  `AGENT_VERSION` + `AGENT_SHA256`, and does a GET round-trip (re-fetch + sha256 re-check) to prove the
  artifact is fetchable + intact. Pinned to a version (never `:latest`); idempotent (delete-then-PUT);
  asserts the binary's `--version` matches the publish version. Creds via `GITEA_USER/GITEA_TOKEN`
  (falls back to `REGISTRY_USER/REGISTRY_TOKEN`).
- **`configs/build-golden.sh`:** after the vzdump archive is produced, computes its sha256 and PUTs it
  to `/api/packages/admin/generic/felhom-golden/<golden-version>/golden.tar.zst` (`<golden-version>` =
  the baked controller version), printing `GOLDEN_VERSION` + `GOLDEN_SHA256`. Opt-in (only when the
  Gitea creds are set); the local-golden auto-discovery stays as a fallback.
- **`configs/felhom-agent.sudoers` (latent bug fix):** escaped the commas in the `lvs -o
  lv_name\,data_percent\,metadata_percent` and `lsblk -o NAME\,FSTYPE\,PTTYPE\,MOUNTPOINT` argument
  lists. Sudoers treats a bare comma as a command separator, so `visudo -cf` REJECTED the file — it had
  never been visudo-validated live because the demo host ran the agent as root+`direct` (sudoers
  unused). The escaped commas still match the agent's real comma-bearing args. Surfaced by the BUNDLE
  live install (the host-install script `visudo -cf`-validates before installing).
- **`cmd/felhom-agent/main.go`:** `version` 0.42.0 → 0.43.0.
- The operator records the printed agent + golden version+sha256 in the hub (Configs → "Day-0
  artifacts"); the host-bootstrap script verifies fetched artifacts against those before installing.
- `go build/vet/test ./...` green.

## build-golden.sh — default controller image bumped to current; golden rebuilt at 0.85.1 (2026-06-27)

**Operational + a default fix (no agent binary change — version stays v0.42.0).**

- `configs/build-golden.sh`: the `CONTROLLER_IMAGE` default (positional arg 6) was a stale
  `…/felhom-controller:0.43.0` — an argument-less golden build baked a wildly old controller, so fresh
  Day-0 boxes started old (the demo started at 0.77). Bumped the default to the **current**
  `…/felhom-controller:0.85.1` so the worst case (no explicit arg) is merely "current", not ancient.
  - **Always pass the controller version explicitly at each rebuild** — this default only bounds the
    worst case. A future `make golden` that resolves the latest pullable tag would remove the need for a
    hand-bumped default (Observation, not this task).
- **Golden rebuilt at 0.85.1** on `felhom-pve` with the image passed **explicitly**
  (`build-golden.sh 9100 … gitea.dooplex.hu/admin/felhom-controller:0.85.1`). New archive volid:
  `local:backup/vzdump-lxc-9100-2026_06_27-11_42_51.tar.zst` (rootfs 32G + Docker-data 16G + user-data
  8G, all in the archive; mp0+mp1 inclusion confirmed in the vzdump log).
  - **Baked-image verify (cheap, mandatory):** in the build guest `/etc/felhom-controller-image` =
    `…:0.85.1` and `docker images` showed it baked (379 MB). New Day-0 provisions now ship current.
  - The host-bootstrap script auto-discovers the newest golden, so it picks up this rebuild
    automatically. The real demo 9201 was **not** re-provisioned (it is the Phase-2 floor test box).

## v0.42.0 — agentic controller update: in-guest image swap + rollback (Phase 1) (2026-06-26)

The host agent now owns the in-guest controller image **swap** — the new-architecture replacement for
the controller's dead in-container `docker compose` self-update. The controller pre-pulls the target
image (shared docker socket, its own registry token) then asks the agent to swap; the agent — external
to the controller container, so it survives the controller being killed mid-swap — does the rest and
**rolls back** if the new controller doesn't come up healthy.

- **New local-API routes** (`internal/localapi/controllerswap.go`, token-scoped via `withGuest`):
  - `POST /controller/swap {image}` → **202** `{status:"swapping", previous_image, target_image}`, then
    async: record previous (crash-safety state file `/var/lib/felhom-agent/controller-swap-<vmid>.json`)
    → confirm the target image is present in the guest (else abort, **no swap**) → write
    `/etc/felhom-controller-image` → `systemctl restart felhom-controller-bootstrap.service` → poll the
    new controller to **healthy** (`docker inspect`, ≤90s) → **roll back** to the previous image + restart
    if it doesn't (the guest is never left without a controller). Single-flight per guest (409 if busy).
    Image ref is strict-validated (`gitea.dooplex.hu/admin/felhom-controller:<semver>`) before any action.
  - `GET /controller/swap/status` → `{state: swapping|done|failed, current, previous, target, error}`.
- **`GuestBinder.GuestExec`** (`internal/localapi/guestbind.go`): the one `pct exec` seam the swap
  composes over (cat/inspect/write/restart), reusing the fenced root runner.
- **`--selftest=controller-swap -vmid -image <ref>`**: exercise the primitive directly (the target image
  must already be pulled in the guest).
- Wired `ControllerSwap: guestBinder` into the local-API server (`cmd/felhom-agent/main.go`).
- Tests (`controllerswap_test.go`): happy swap, **rollback-on-unhealthy** (+ companion red-proof:
  dropping the rollback leaves the guest on the bad image and fails the test), image-absent no-swap,
  no-healthcheck-running, bad-image 400, single-flight 409.

## v0.41.0 — provisioned customer guests auto-start after a host reboot (`onboot:1`) (2026-06-24)

**F3 fix.** The provision back-half now sets **`onboot:1`** on the customer guest, so after a host
reboot/power-cut the customer's whole home-server (controller + apps) comes back **on its own** —
previously every provisioned guest inherited the golden's `--onboot 0` and stayed **stopped** until a
manual `pct start` (confirmed live in the stable-path/sys-drive restart campaign, Phase 4.1). The new
step is a fatal `pct set <vmid> -onboot 1` placed right after the config-mount attach (`backhalf.go`),
mirroring the config-mount/parent-bind `pct set` ops. **No `startup`/boot-order/delay** — the v0.75
mountpoint-gate already covers the drive-bind race at boot (Phase 4.4), so the controller won't write
app data onto the rootfs while the agent re-binds drives.

The **golden stays `onboot:0`** (`build-golden.sh` unchanged): a template must not auto-start, and
`onboot` is a per-guest property the back-half is the right place to set. Unit-tested
(`TestProvision_SetsOnbootOne` asserts the exact `pct set … -onboot 1` invocation, with a red-proof
against removing the call). The pre-existing demo guest 9201 (provisioned pre-fix) was remediated
non-destructively with `pct set 9201 -onboot 1`. **Capstone live-validated (2026-06-24):** destroyed +
re-provisioned 9201 through the real provision chain with v0.41.0 → fresh `pct config` showed `onboot: 1`
with no manual set; a subsequent **felhom-pve host reboot** brought 9201 back **running with no manual
`pct start`** (the `onboot:0` scratch guests correctly stayed stopped), controller + base infra healthy,
drives re-bound at stable, sys_drive separate — the exact Phase-4.1 failure now passes.

## v0.40.0 — third CT volume: SSD user-data (`/mnt/sys_drive`, mp1) baked + `-sysdata-grow` (2026-06-23)

**The third golden volume.** Extends the OS/Docker-data split (v0.29.x) to a **three-volume layout**:
rootfs + Docker-data (`mp0`) + **SSD user-data (`mp1` @ `/mnt/sys_drive`, `backup=1`)** — the
controller's `system_data_path`. Until now `/mnt/sys_drive` was a plain directory on the 32 GB OS
rootfs, so the controller correctly warned that SSD app data (`<sys_drive>/felhom-data`) lands on the
OS drive. Baking it as its own thin volume clears that warning with **zero controller change** (the
controller already auto-discovers `<sys_drive>/felhom-data` and warns via `system.IsMountPoint`); the
`mp` under the guest's `/mnt` reaches the controller container through the existing
`-v /mnt:/mnt:rslave` bind.

- **`configs/build-golden.sh`** — `pct create` gains
  `--mp1 ${ROOTFS_STORAGE}:${GOLDEN_SYSDATA_GB},mp=/mnt/sys_drive,backup=1` (new env
  `GOLDEN_SYSDATA_GB=8`, near-empty; provision grows it). The resilience guards are mirrored for `mp1`:
  a `findmnt /mnt/sys_drive` separate-mount assertion, and the vzdump-inclusion guard now aborts if
  **either** `mp0` **or** `mp1` is EXCLUDED (the B3 trap — extra mountpoints default `backup=0`). The
  golden does NOT pre-create `felhom-data`; the controller does once it's a real mountpoint.
- **`internal/reconcile/bringup.go`** — `const DefaultSysDataMount = "mp1"`; `BringUpSpec` gains
  `SysDataGrowGB int` + `SysDataMount string`; a new **"4c"** grow block (online, grow-only `ResizeLXC`,
  its own task) mirrors the "4b" Docker-data grow. `0 = skip` (separateness comes from the golden, not
  the grow — the warning clears regardless of size).
- **`cmd/felhom-agent/main.go`** — `-sysdata-grow` / `-sysdata-mount` flags (mirror
  `-datavol-grow`/`-datavol-mount`); `bringUpSizing` carries them into all three bring-up/provision call
  sites; `--selftest=provision` help text updated.
- **Static volume, NOT an enrolled drive.** `/mnt/sys_drive` is part of the baked golden layout; it
  never enrolls/ejects/decommissions and is deliberately kept off the drive-intent machinery.
  `freeMountSlot` auto-skips the baked `mp0`/`mp1` so enrolled drives never collide.
- Tests: `TestRunBringUp_StorageSplit_SysDataGrow` (asserts `ResizeLXC(vmid,"mp1","+42G")`) +
  `…_SysDataGrowZeroNoResize` (0 → no mp1 resize). RUNBOOK-provisioning-storage.md extended to the
  three-volume layout (default ~512 GB SSD: 32 rootfs + 200 docker-data + 50 user-data).

## v0.39.0 — DR recipe completion: live PBS coord + drop the two unfillable drive fields (2026-06-16)

**DR-recipe agent-half completion.** A live eyeball of the demo recipe (v0.38.0) found three host-half
problems; all three are resolved here. No behavior change outside the recipe path.

- **PBS coord now resolved LIVE each collect.** New `internal/pbs/live_reporter.go` —
  `LiveSnapshotReporter` implements `hub.PBSReporter` by doing the cheap `Client.Snapshots()` list
  itself, with **last-known-good fallback**, instead of reading only the verify-loop's `SnapshotStore`.
  Previously the recipe's `pbs` block was omitted whenever the store was empty — which a one-shot
  collect (`--selftest=hub`) and the first ~6 h window of every daemon after a restart always saw (the
  verify loop populates the store on its own 6 h cadence). The restore SOURCE must not depend on a
  maintenance cadence. Per-datastore: a live error/timeout → that datastore's last-known-good; a
  successful (even empty) response is authoritative and updates the shared store. Targets-resolution
  failure → the full LKG aggregate. Bounded by `DefaultLiveSnapshotTimeout` (8 s) so a hung PBS never
  stalls the heartbeat. List only — it never triggers a `Verify`. The verify loop keeps Recording into
  the SAME store (shared last-known-good); both use one hoisted `pbsTargets` closure.
  - `SnapshotStore.Get(datastore)` added (per-datastore LKG copy) — the only `SnapshotStore` change.
  - Wired into the collector in BOTH `runDaemon` and `runSelftestHub` (the selftest built its own
    collector with a `nil` reporter — that is why the live `--selftest=hub` showed `pbs_snapshots:[]`).
  - Intended side effect: `report.pbs_snapshots` is now live too (fresher hub PBS view).
- **`drives[].role` DROPPED from the v1 host-half shape.** A drive's purpose is a hub/operator-owned
  manifest concept, not cleanly derivable host-side (both demo externals are `content=backup`, yet one
  is the primary data drive and the other holds no apps). Deferred until the hub/operator stamps it.
- **`drives[].restic_repo_coord` DROPPED from the v1 host-half shape.** It named a backup tier that does
  not exist — cross-drive backup is rsync to the SAME internal SSD; there is no offsite/second-failure-
  domain bulk copy. RESERVED for a future tier (see the BACKLOG note in REPORT). v1 drive shape is now
  `{durable_id, mount_path, intent, fs_type?, total_bytes}` — identifiers/intent/size only.
- The hub reads drives as `json.RawMessage`, so dropping fields needs NO hub struct change — only
  golden + test sync. Cross-repo golden (`host-report.golden.json` here + the hub's copy) re-pinned and
  verified **byte-identical** (sha256 `57f2a5e7…18b2f2b5` — manual checksum-diff discipline): the hub copy
  previously lacked the `dr_recipe` section entirely; it is now a verbatim copy of the agent golden.
- Tests: new `internal/pbs/live_reporter_test.go` (T1 coord-present-without-prior-verify [load-bearing] +
  inline bare-store companion, T2 error→LKG fallback, T3 success-warms-store, T4 targets-error→aggregate,
  T5 bounded-by-timeout, T6 empty-success-authoritative); `TestDRRecipeHostHalf_V1DriveShape` (drive
  object carries neither `role` nor `restic_repo_coord`); `TestBuildDRRecipeHostHalf` /
  `TestHostReport_ContractMatchesGolden` updated to the v1 drive shape. Each companion was demonstrated
  to FAIL on the pre-fix/mutated code, then reverted (see REPORT).

## v0.38.0 — DR recipe: emit the secret-free storage/guest/PBS half in the host-report (2026-06-16)

**DR recipe slice (agent half).** Additive `dr_recipe` section on the host-report — the agent half of the
secret-free reconstruction recipe (`SPIKE-dr-recipe-2026-06-16.md`) that complements escrow (keys) +
PBS/restic (bytes): the non-secret SCAFFOLDING an operator must rebuild before the PBS bytes can land.
The hub assembles it with the controller's app half into one customer recipe.

- `internal/hub/dr_recipe.go` — `DRRecipeHostHalf{recipe_version, guests[], pbs, drives[], pve_storage[]}`
  built by the pure `BuildDRRecipeHostHalf(guests, targets, pbs)` from facts the report ALREADY collects
  (no new privileged reads): `guests[]` = each guest's sizing (`GuestSpec`, skip status-unknown);
  `drives[]` = the user-data external drives (usb/local-dir with a `uuid:` durable-id + mount path) with
  `{durable_id, role, mount_path, intent, total_bytes}`; `pve_storage[]` = every storage target
  `{name, type, content}` (the `storage.cfg` scaffolding); `pbs` = the latest snapshot's coordinates
  `{repo_id (the pbs storage id), namespace, latest_snapshot_id}`. Wired into `Collect()` after the facts
  are gathered; `HostReport.DRRecipe` (always set, never null).
- **BOUNDARY (the Phase-1 lesson):** every field is an identifier / intent / size / coordinate — NEVER a
  key, password, token, hash, or `ENC:` value. The PBS encryption key stays in escrow; the access token in
  identity-escrow; the restic password in escrow — the recipe names only the `repo_id`/`namespace`/
  `durable_id`/`restic_repo_coord` the restore TARGETS. `recipe_version=1`; read is ignore-unknown
  (forward-compat). The wire shape is pinned in the cross-repo golden (`host-report.golden.json` here +
  the hub's copy — keep them byte-identical; manual checksum-diff on any change).
- Tests: `TestBuildDRRecipeHostHalf` (drives = only user-data; pve_storage = all; pbs = latest; guests
  skip nil-spec), `..._NoPBS` (omitted, non-nil slices), `TestDRRecipeHostHalf_NoSecrets` (the lighter
  boundary mirror — serialized half carries NO credential-shaped key; the load-bearing version is on the
  controller emitter), and the `dr_recipe` key-set added to `TestHostReport_ContractMatchesGolden`.

## v0.37.0 — host-reboot remount re-resolves enrolled drives by filesystem UUID (2026-06-16)

**TASK A — close out the reboot story (agent half).** On a host reboot the kernel can re-enumerate block
devices and move a drive's node (felhom-usb `/dev/sdb`→`/dev/sdc`), and a `.mount` unit left `disabled`
by a prior detach never auto-mounts at boot — so an enrolled drive could stay unmounted (or, with any
node-trusting remount, mount the WRONG device). Root cause pinned LIVE: felhom-usb's systemd mount unit
was `disabled` (no `multi-user.target.wants` symlink) while felhom-flash's was `enabled`; `What=` was
already correct (by-UUID), but nothing re-asserted the unit at startup.

- `storage.ResolveStorageDevice(durableID)` — resolves the enrolled `uuid:<fs-uuid>` storage scheme to its
  CURRENT backing `/dev` node by re-scanning `/dev/disk/by-uuid` (never a cached node); errors if the UUID
  is genuinely absent so a caller skips a gone drive instead of fail-mounting a stale node.
- `storage.parseFelhomMountUnit` — pure inverse of `renderMountUnit` (Name/UUID/Where/Type/Options) keyed
  on a `Managed by felhom-agent` marker; ignores any foreign `.mount` unit.
- `(*SudoHostOps).ReassertEnrolledMounts(ctx)` — at startup (BEFORE binding into the guest) and on the
  periodic 20s tick: for each enrolled `.mount` unit, re-resolve by UUID and re-run `EnsureMount`
  (idempotent `systemctl enable --now`) — re-enables a disabled unit AND mounts the CURRENT device by
  UUID, so a `/dev/sdX` reshuffle is a no-op. Skips ONLY the durable steady state (mounted AND enabled),
  via the pure `shouldReassertMount`; a **mounted-but-DISABLED** unit (the exact live felhom-usb bug — it
  serves now but a reboot would not auto-mount it) is still re-asserted to re-create the wants-symlink.
  Enabled-state is read with a privilege-free `os.Lstat` of the `multi-user.target.wants` symlink
  (`unitEnabled`) — no `systemctl is-enabled` subprocess, no new sudoers entry. An absent UUID is skipped
  (re-asserts on a later tick).
- Wired in `main.go` ahead of `ReassertGuestBinds` so mounts are live before the guest binds re-assert.
- Tests (Linux, seam the device-resolution): `TestResolveStorageDevice_ToleratesDeviceLetterMove`
  (UUID symlink moved sdb→sdc → resolves sdc; companion asserts the cached enroll-time node differs from
  the freshly-resolved one — a node-based remount would target the wrong device), `..._AbsentAndScheme`
  (absent UUID errors; only the `uuid:` scheme resolvable), `TestParseFelhomMountUnit` (render→parse
  round-trip + rejects a foreign unit), `TestShouldReassertMount` (the four mounted/enabled combos — pins
  the mounted-but-disabled re-assert), `TestUnitEnabled` (wants-symlink detection).

**TASK A2 — verdict: enrolling a NEW drive does NOT need an LXC restart.** The enroll path lands on the
live intermediary-mount `AttachDrive` (`/disks/guest-attach` → `handleDiskGuestAttach` → `AttachDrive`,
"no pct, no reboot") under the single shared parent — unbounded named live slots — NOT the legacy
`RebootGuest` branch. The operator's pre-created-slot-pool idea is therefore unnecessary.

## v0.36.7 — isolate the shared parent only on CREATE (no peer-group churn) (2026-06-15)

Follow-up to v0.36.6: make-private+make-shared must run ONLY when the self-bind is first created, not on
every reconcile — re-doing it churns the peer-group id and ORPHANS the guest`s already-established slave
(propagation silently dies, guest sees empty). Guarded on the mountpoint check; on a fresh boot it runs
once before pve-guests so the guest slaves the right group.

## v0.36.6 — shared parent gets its OWN peer group (make-private first) — ROOT CAUSE of double-bind (2026-06-15)

The shared-parent self-bind INHERITED the root mount`s shared peer group (`/mnt/felhom-drives` was
`shared:1` same as `/`), so every drive bind under it propagated back via the root peer and DOUBLED
(2 stacked binds per drive — the real cause behind v0.36.3-.5). EnsureSharedParent + the boot script now
`make-private` (detach from the root group) BEFORE `make-shared` (own group whose only slave is the
guest), so a drive bind propagates to the guest exactly once.

## v0.36.5 — AttachDrive normalizes to exactly one bind (2026-06-15)

AttachDrive now COUNTS the binds at a stable path (countHostMounts) and normalizes to exactly one: it is
a no-op only when there is exactly ONE bind the guest sees; otherwise it strips ALL existing binds
(bounded loop) and lays down one fresh bind. This converges a stacked double-bind to one — the old
umount-one+mount-one force-rebind never did. Caught when a double-bind survived a guest reboot.

## v0.36.4 — serialize AttachDrive/DetachDrive (no double-bind race) (2026-06-15)

A mutex on GuestBinder serializes AttachDrive/DetachDrive so a controller-triggered reconnect and the
agent`s periodic reconcile can no longer both pass the isHostMountpoint check and double-bind the same
stable path (a TOCTOU race observed live as 2 stacked binds during rapid eject/reconnect).

## v0.36.3 — DetachDrive loop-umounts stacked binds (2026-06-15)

DetachDrive now umounts ALL stacked binds at a stable path (bounded loop), not just one layer — so an
eject/detach fully detaches even if more than one bind accumulated (operator bind on top, or a rare
attach race), keeping the fail-close intact. Caught in the E13 rapid eject/reconnect sweep.

## v0.36.2 — eject also keeps the raw mounted (reconnectable) (2026-06-15)

Extends v0.36.1 to EJECT: eject now DetachDrive`s the bind under the parent but LEAVES the raw
/mnt/<name> mounted (consistent with decommission), so the H1 disconnect→reconnect roundtrip re-binds
cleanly on a non-removable drive. Physical removal is the separate "remove from system" action. Test:
eject calls DetachDrive + does NOT unmount the raw.

## v0.36.1 — decommission keeps the raw mounted (re-enrollable) (2026-06-15)

Fix caught in the E10 acceptance test: the self-serve decommission unmounted the RAW /mnt/<name> host
mount, which orphaned a non-removable drive (no re-plug) so a one-click re-enroll bound an empty dir. On
the intermediary model decommission is now a LOGICAL retire — it DetachDrive`s the bind under the parent
(drive invisible to the guest) but LEAVES the raw mounted, so re-enroll re-binds cleanly. Physical
removal stays the separate "remove from system" action. Test updated.

## v0.36.0 — guest boot-id on /disks (deterministic guest-reboot recreate) (2026-06-15)

The agent now emits `guest_boot_id` on GET /disks: `<host-btime>-<guest-init-starttime>` — changes on
every guest boot (host reboot OR guest reboot) but is STABLE across a controller-only restart. The
controller persists the last-seen value and DETERMINISTICALLY recreates drive-backed apps when it
changes (replacing the fragile timed state-sample that could miss an app stopped at the sample instant).
`GuestBootID` reads `/proc/stat` btime + field 22 of `/proc/<init-pid>/stat` (parsed after the last
`)` so a comm with spaces/parens does not break it).

## v0.35.1 — shared-parent unit: run before pve-guests on host boot (2026-06-15)

Fix for the host-reboot ordering (the shared-parent oneshot never ran before pve-guests on the live
host, so the guest bound a not-yet-shared parent → private bind → propagation broken). The unit now uses
`WantedBy=pve-guests.service` (pve-guests PULLS IT IN + Before= orders it first) instead of the
unreliable `WantedBy=multi-user.target`, and drops `DefaultDependencies=no`. `EnsureSharedParent`
reinstalls the unit when its content differs (so the fix deploys on the next agent start/reconcile).

## v0.35.0 — intermediary mount: guest-reboot re-propagation (load-bearing) (2026-06-15)

Fix for the guest-reboot gap (caught in the live demo migration). A guest's parent bind is
NON-RECURSIVE, so on a guest reboot it does NOT carry the pre-existing drive submount, and mount
propagation only delivers events created AFTER the bind exists — so an enrolled drive is bound on the
HOST but INVISIBLE in the fresh guest namespace until re-bound. Without this, every guest reboot left
the apps on empty dirs.

- `AttachDrive` now takes `vmid` and checks GUEST visibility (`GuestSeesMount`, reading
  `/proc/<guest-init-pid>/mountinfo`): if the host has the bind but the guest doesn't see it
  (post-reboot), it FORCE re-binds (umount + mount) to re-fire propagation into the current guest ns.
- A periodic reconcile (20s ticker in main) re-runs `ReassertGuestBinds`, so a guest reboot self-heals
  without an agent restart. `EnsureSharedParent` skips the unit re-install when already present (cheap
  on repeat).
- `/disks` `BoundUnderParent` now reflects GUEST visibility (not the host mount) — the accurate signal
  the controller's drive-absent gate keys on to stop/restart apps across a guest reboot.

## v0.34.0 — intermediary mount model: shared-parent + host-side attach/detach + reconcile (2026-06-15)

The drive hot-swap re-architecture (SPIKE-intermediary-mount). Replaces the per-drive `pct set -mpN`
bind (which needed a guest reboot to activate and bricked the guest when a drive was absent at boot)
with a SINGLE permanent parent bind `/mnt/felhom-drives` plus host-side swaps underneath it.

- `internal/localapi/intermediary.go` — `GuestBinder.EnsureSharedParent` (mkdir + self-bind +
  `--make-shared` + installs/enables a `felhom-shared-parent.service` ordered **Before=pve-guests** so
  the guest's parent bind inherits the shared peer group as `slave`); `AttachDrive` (`mount --bind
  /mnt/<name>/felhom-data /mnt/felhom-drives/<name>` — propagates into the RUNNING guest live, no pct,
  no reboot; confined to felhom-data; the stable dir stays host-root-owned = fail-closed); `DetachDrive`
  (`umount`, leaving the bare fail-closed dir); `StablePathForRaw`/`DriveNameFromRaw`; `isHostMountpoint`.
- `ReassertGuestBinds` is now a pure HOST-SIDE reconcile: for each enrolled+present drive ensure its
  felhom-data is bound under the parent (no guest-config read, no slot, no reboot) — fixes F9 and
  drive-reconnect for free. Runs at startup (ensures the shared parent first).
- `handleDiskGuestAttach` uses `AttachDrive` (returns the stable `guest_path`); eject + decommission
  call `DetachDrive`. Legacy `AttachBind`/`DetachBind` retained for the transition (decommission still
  `--delete`s any lingering legacy mp).
- `/disks` reporting adds `GuestPath` (the stable `/mnt/felhom-drives/<name>` the controller repoints
  HDD_PATH to) and `BoundUnderParent` (live-in-guest signal for the controller's drive-absent gate).
- Provision adds the one permanent parent bind (`-mp8 /mnt/felhom-drives,mp=/mnt/felhom-drives`).

Tests (non-hollow + companions): `TestGuestAttach_BindsUnderParent` (uses AttachDrive not legacy pct),
`TestReassertGuestBinds_RestoresMissingBind` (host-side reconcile, legacy AttachBind never called),
`TestStablePathForRaw_DriveName`, `TestDisks_GuestPathAndBoundUnderParent`. Sudoers: new
`FELHOM_INTERMEDIARY` alias (mount/umount under /mnt/felhom-drives, the unit install, the parent bind).

## v0.33.0 — C1 net: pre-start self-heal hook + decommission mp-delete (2026-06-15)

The transitional defense for the C1 brick (B3 critical bug) ahead of the intermediary-mount
re-architecture (which makes C1 structural). Two independent nets:

- **Pre-start self-heal hook** (`internal/guesthook`): a PVE `pre-start` hookscript runs
  `felhom-agent guest-hook <vmid> <phase>` which, for every BIND mountpoint whose source path is
  missing, creates an empty **host-root-owned** placeholder dir so the bind succeeds and the guest
  always boots — fail-closed (host uid 0 is unmapped in the unprivileged-LXC userns, so the guest
  can't write to the placeholder; a returning drive shadows it). It CREATES rather than DELETEs
  because `pct set --delete` in pre-start would take the config lock the start task already holds
  (dead-times-out → still bricks); the heal logic is in unit-tested Go, the wrapper just delegates.
  Installed + registered per-guest by the provision back-half (`InstallSnippet`/`Register`).
- **Decommission mp-delete** (`GuestBinder.DetachBind` + `handleDiskDecommission`): decommission now
  runs `pct set <vmid> --delete mpN` on the slot binding the drive (lock-safe on the running guest),
  so its now-missing source can't brick the next reboot. The old handler unmounted but left the dead
  `mpN` in config — the exact B3 C1 bug. Eject keeps its mp (temporary; the hook covers a
  reboot-while-ejected).

Tests (non-hollow, each with a companion that fails the pre-fix/trivial impl):
`internal/guesthook/heal_test.go` (selector ignores storage volumes + present binds, heals only the
absent one; "return nothing"/"return all" both fail) and `TestDecommission_DeletesGuestMount`
(asserts the correct slot is `--delete`d; pre-fix never calls DetachBind → fails).

Sudoers: new `FELHOM_GUESTHOOK` alias (snippet install, `pct set --hookscript`, `pct set --delete mpN`).

## v0.32.0 — self-serve decommission + intent-aware re-assert (B2a) (2026-06-14)

Customer-self-serve storage decommission (no operator signature; non-destructive — never formats),
plus the load-bearing fix that keeps a decommissioned drive from auto-rebinding into the guest.

- **`POST /disks/decommission`** (`internal/localapi/disks.go` `handleDiskDecommission`, route in
  `server.go`) — mirrors `handleDiskEject` exactly: `withGuest` self-scoping, `scopedFromBody`, and the
  same **user-data role gate** (`roleForMountPath` must be `RoleUserData`, else 403; fail-safe-to-
  protected on ambiguity) so a compromised controller can't decommission system/backup storage. It
  records a PERMANENT `IntentDecommissioned`, prunes the `GuestBindStore` entry (hygiene), and unmounts
  (so the drive is physically removable). It **NEVER** calls any format/mkfs path — the data stays on
  the drive. The operator-signed `DecommissionExecutor` + `reconcile.Classify` classification are
  untouched (the absent-drive/DR route).
- **`ReassertGuestBinds` is now intent-aware** (THE correctness fix): the startup re-assert skips any
  durable-id whose intent is not `enrolled`, so a decommissioned- (or ejected-) but-still-present drive
  is never auto-rebound into the guest on agent restart. A nil intent store falls back to legacy
  bind-all (matching the watchdog's nil-intent rule). Covers both the self-serve and the operator-
  signed decommission paths (both land on `IntentDecommissioned`).
- **`GuestBindStore.Remove(vmid, durableID)`** (`internal/localapi/guestbindstore.go`) — idempotent
  (absent = no-op), atomic tmp+rename like `Record`; drops the vmid key when its set empties. Re-enroll
  re-`Record`s via the existing `recordGuestBind` on guest-attach, so Remove doesn't break re-commission.
- `IntentRecorder` extended with `SetDecommissioned` + `Get` (both already on `*storage.IntentStore`).
- Non-hollow tests (`internal/localapi/decommission_test.go`): role-gate refuses system/backup (403,
  no unmount); decommission sets intent + removes the bind + unmounts + never formats; intent-aware
  re-assert does NOT rebind a decommissioned-but-present drive (companion: enrolled DOES rebind; the
  intent-blind pre-fix code fails this); re-commission re-records; `Remove` idempotency + persistence.

## v0.31.0 — live-drive F9 + F20-BUG2 + F20-BUG3 (disk bind/wipe) (2026-06-14)

The last live-drive findings, all disk/`localapi`-side, implemented + deployed on `felhom-pve` and
validated live on guest 9201 (approach: attach-to-existing, no re-provision — see the audit fixspec).

- **F9 — guest data-drive bind survives a re-provision** (`4cd1d02`). The in-guest bind (`pct set -mpN`)
  is config state a destroy+re-provision drops, and nothing restored it → a re-provisioned guest came up
  with its enrolled HDD unattached. New `GuestBindStore` (durable-id-keyed, per guest, recorded at
  guest-attach) + `ReassertGuestBinds` on agent startup re-adds any bind a guest is missing — only when
  the durable-id still resolves to a present drive (a swapped/absent disk is never auto-bound), idempotent.
  Plus `DiskInfo.GuestAttached` — the missing "bound into THIS guest" signal (vs mere host presence;
  resolves the F2 `hdd_configured` disagreement). **Live-proven:** dropped the bind, restarted the agent
  (real trigger) → re-attached with no manual call; reboot activated it; an HDD app then deployed onto
  the drive with data on `/dev/sdb1`.
- **F20-BUG2 — one wipe durable-id scheme** (`a2a76e7`). `/disks` advertised only `durable_id` (`uuid:`,
  used for assign), but the wipe gate resolves `byid:`/`byuuid:` → confirming a wipe with the advertised
  id was a `binding_mismatch`. New `DiskInfo.WipeDurableID` via a shared `s.deviceDurableID` seam used by
  BOTH the list and the gate, so the id the customer copies is the id the gate accepts. **Live-proven:** a
  confirmed wipe using `/api/disks`'s `wipe_durable_id` is accepted (no mismatch).
- **F20-BUG3 — format runs detached; survives a request deadline AND an agent restart** (`4777f8a`). mkfs
  ran under the HTTP request context, so a client deadline SIGKILLed it mid-write → corrupt disk. Now mkfs
  runs off `s.baseCtx` via a persisted `formatJob` record; the handler still returns the synchronous
  result (backward-compatible) but a dropped request no longer kills it. New `GET /disks/format/status`;
  `RecoverFormatJob` on startup re-runs an interrupted durable-id-bound format (re-resolved; anti-retarget
  — a blank/path-bound or unresolvable job is not auto-re-run). **Live-proven on the 916 GB felhom-usb:** a
  2 s client timeout left a ~30 s mkfs running to a clean ext4 (the live-drive corruption is gone); an
  agent restart mid-format was recovered + completed to a clean fs.



**Security fix (from the 2026-06-13 deep-sweep audit).** The inline customer-confirmed wipe in
`internal/localapi/disks.go` `handleDiskFormat` inspected and gate-bound the device by its durable id
but then ran `mkfs` on the caller-supplied mutable `/dev` path (`req.Device`). A USB re-enumeration
reassigning that `/dev` node to a different physical disk between inspection and `mkfs` (a
classify→mkfs TOCTOU) could wipe the wrong drive.

- New `internal/localapi/wipe_reresolve.go`: `antiRetargetResolve` (injected-deps, unit-tested) mirrors
  `signedjobs.WipeExecutor.Execute` — resolve the confirmed durable id → current device, re-derive the
  device's durable id and require an exact match, re-inspect (still data-bearing), and return the
  re-resolved device. `(*Server).reresolveDurableForWipe` wires the real storage funcs.
- `handleDiskFormat` now formats the **re-resolved** device, never `req.Device`; any refusal →
  `409 Conflict`, no `mkfs`. Injectable `reresolveWipe` seam on `Server` (defaults to the real path).
- Tests: `wipe_reresolve_test.go` covers happy-path, empty/gone/blank, re-inspect-error, and the core
  `retarget-mismatch-refused` case. Round-trip safe for legitimate wipes (`DeviceDurableID` ↔
  `ResolveDurableDevice` schemes match). Agent-only deploy; no golden rebake. See `AGENT-001-FIX-NOTES.md`.

## v0.29.1 — lanresolver: RESTART dnsmasq on change (not reload) — fixes stale split-horizon IP (2026-06-13)

**Bug:** after a guest's DHCP IP moved (e.g. the v0.29.0 9201 re-provision: .151 → .141), the LAN
split-horizon resolver kept answering the OLD IP, so LAN clients (via Pi-hole's conditional forward to
the host dnsmasq) resolved `*.demo-felhom.eu` to the dead IP. Root cause: `lanresolver.Manager` updated
the per-customer drop-in (`address=/<domain>/<ip>`) correctly but then ran `systemctl reload dnsmasq`
(SIGHUP) — and **dnsmasq's SIGHUP does NOT re-read its config files** (`/etc/dnsmasq.d/*.conf`); it only
clears the cache + re-reads `/etc/hosts`/addn-hosts. So the changed `address=` directive never took
effect until a restart. **Fix:** `reload()` → `restartDnsmasq()` (`systemctl restart dnsmasq`) for every
config-drop-in change (ReconcileGuest IP change, EnsureDnsmasq base change, Remove/decommission). Restart
is sub-second and the records carry local-ttl 0, so downstream forwarders don't cache a stale answer.
(Live: after the fix + a one-time host dnsmasq restart + a Pi-hole cache flush, `*.demo-felhom.eu`
resolves to the live guest IP again; future IP moves now self-heal on the loop's next tick.)

## v0.29.0 — OS / Docker-data storage split: golden + provision (2026-06-13)

Phase 1 of the storage-split slice (Phase 2 = felhom-controller v0.58.0 prevention layer). The
controller guest's OS rootfs and Docker data are carved onto separate `local-lvm` volumes for
RESILIENCE — an isolated OS rootfs stays bootable + agent-recoverable if the Docker volume fills.

- **`configs/build-golden.sh` — split baked in:** `--rootfs ${ROOTFS_STORAGE}:${OS_SIZE_GB}` (default
  **32**, was hardcoded 8) **plus** `--mp0 ${ROOTFS_STORAGE}:${GOLDEN_DOCKER_GB},mp=/var/lib/docker,backup=1`
  (default 16). The baked controller + infra images land on the data volume and travel inside the
  golden archive (no empty-volume shadowing, no deploy-time pull). `backup=1` is MANDATORY — extra LXC
  mountpoints default to `backup=0` = EXCLUDED from vzdump (spike B3), which would drop the images from
  the archive entirely. The script now also bakes Docker **log rotation** into `daemon.json`
  (`max-size 10m`, `max-file 3` — prevention layer 2D), asserts `/var/lib/docker` is a separate mount,
  and **aborts if vzdump excludes mp0**.
- **`internal/reconcile/bringup.go` — sized provision:** `GuestMount` gains `Backup` (emits `,backup=1`
  — closes the spike-B3/B5 silent-DB-loss trap at the mount builder). `BringUpSpec` gains
  `DataVolGrowGB` + `DataVolMount` (default `mp0`): provision GROWS the golden-carried Docker-data
  volume online to the per-customer target (grow-only, spike B4) rather than attaching a fresh empty
  volume that would shadow the baked images. Plus `RootfsGrowGB` for the OS rootfs.
- **CLI seam:** `--selftest=bring-up|provision` gain `-rootfs-grow` / `-datavol-grow` / `-datavol-mount`
  flags. Per-customer sizing source = flags now, the slice-10 hub storage manifest later.
- **`RUNBOOK-provisioning-storage.md`** (new): the split provisioning procedure + fresh-PVE-install
  thin-pool carving knobs (`hdsize`/`maxroot`/`maxvz`, spike B4) + the per-customer sizing seam.
- Tests: `buildBringUpConfig` backup=1 emission; bring-up issues rootfs + data-volume resizes.

## (no version) — storage OS/data-split spike findings (2026-06-13)

Investigation only — **no code changed**. Findings report: `REPORT-storage-split-spike.md` (gates the
provisioning spec for splitting the controller guest's OS rootfs from its Docker/data onto separate
`local-lvm` volumes). Proven on a throwaway unprivileged LXC (9300, since destroyed): Docker `data-root`
on a second `local-lvm` mountpoint works (overlayfs/ext4, no idmap issue, reboot-survives); the
move-then-verify migration is safe (copy-not-move). **Key finding:** additional LXC mountpoints are
**excluded from vzdump by default** — they need `backup=1` set **and a CT restart** — so the docker-data
mount must be attached with `,backup=1` or named-volume DBs silently fall out of PBS. The exact seam is
`internal/reconcile/bringup.go:313` (`buildConfigParams`), which today builds `mpN` without a `backup=`
flag; `GuestMount` should carry the flag. Per-customer sizes belong in the slice-10 hub storage manifest
(marked at `bringup.go:49-50`); the golden rootfs is hardcoded `8` at `configs/build-golden.sh:40`.

## v0.28.0 — backup re-target → felhom-pbs (offsite DR) + operator-signed decommission (2026-06-12)

**Whole-guest backup now defaults to the offsite PBS tier (real DR).** `BackupConfig.BackupTarget()`
returns the configured `backup.local_backup_target` or, when empty, the new default `felhom-pbs` — a
PBS datastore on SEPARATE HARDWARE (the DooPlex box), so a host disk/hardware failure no longer takes
the backups with it. The target stays fully configurable (set `local_backup_target` to `local`/other
to override); no call site hardcodes it. All `NewBackupRunner` sites (restore-test scheduler, local-API,
`--selftest=backup`/`restore-test`) route through `BackupTarget()`.

Proven live on demo-felhom before the re-point (PHASE 0 gate):
- snapshot-mode `vzdump → felhom-pbs` still fires the `create storage snapshot 'vzdump'` marker, so the
  8B.2 early-resume/quiesce signal survives a PBS target (the marker is mode-driven, not target-driven);
- the restore-test enumerates PBS backups through the SAME generic `StorageContent`
  (`/nodes/<node>/storage/felhom-pbs/content` returns `content:"backup"` + ctime/vmid/volid), so
  `PickRestoreCandidate`/`latestArchive` need NO PBS-client change;
- `pct restore` from a PBS volid round-trips cleanly (storage.cfg encryption key applied transparently);
- PBS gotchas (`ignore-verified`, node-from-UPID, privsep) touch only the verify-API path, not vzdump/restore.

**Operator-signed `decommission` now reachable (slice 10 P3 completion).** The previously-unreachable
`IntentDecommissioned` state (no production caller) is now reached ONLY via a gate-VERIFIED operator
signature — never customer-confirmable, distinct from a safe eject. New `internal/signedjobs`
`DecommissionExecutor` (op `decommission`, classified destructive in `reconcile.Classify`) calls
`IntentStore.SetDecommissioned`, keyed by the drive's STORAGE durable-id (the watchdog's key, e.g.
`uuid:<fs-uuid>` — NOT the device-level `byid:/byuuid:` scheme `storage_wipe` uses), so the recorded
intent actually gates future remounts. New `ExecutorChain` lets the signed-jobs runner serve both
`storage_wipe` and `decommission`; the runner wiring moved below the intent-store open in `main.go`.
`felhom-opsign` builds decommission params from `-durable-id`. No controller/customer UI — the operator
path is hub jobs-queue → signed-jobs runner.

**Restore-test now boot-verifies slice-10 enrolled guests (bind-mount mountpoints).** A guest whose
data drive is a host BIND mount (slice-10 P2 `mp0`) could not be vzrestore'd by the privsep token
("restoring 'mpN' to bind mount is only possible for root") — so the restore-test failed for every
enrolled guest, regardless of backup tier (surfaced during the felhom-pbs live validation). The
restore-test now reads the SOURCE guest config (vmid parsed from the archive volid — PBS `ct/<vmid>/`
and vzdump `vzdump-lxc-<vmid>-` forms) and passes `RestoreLXCOptions.MountOverrides` that neutralize
each bind-mount `mpN` to a throwaway 1G volume on the restore storage (needs no root; the boot-verify
doesn't need the drive's data, and the host paths would otherwise collide). Storage-backed mountpoints
are restored normally; best-effort (an unreadable source config restores as-is). `proxmox.RestoreLXC`
gained `MountOverrides`. Verified live: restore-test from felhom-pbs of bind-mounted guest 9201 →
boot+running PASS.

## v0.27.0 — slice 10 P3: self-heal watchdog reconcile + 4-state intent model (2026-06-12)

The storage watchdog goes from detect-only → detect-and-reconcile: the agent autonomously re-mounts an
enrolled external drive that dropped out-of-band (the colleague's Proxmox unmount), gated by a persisted
INTENT model so it never auto-adopts an unknown drive or fights an official eject.

- **`internal/storage/intent.go` — `IntentStore`** — durable, **durable-id-keyed** (UUID/WWN, never
  sdX/path), atomic-write 4-state model: `new` (not recorded → never auto-mount), `enrolled` (desired
  mounted → reconcile drift), `ejected` (intentional unmount → leave alone), `decommissioned`
  (permanent). `OnAbsent` clears `ejected`→`enrolled` so a replug auto-mounts (the replug rule).
  Records intent ONLY through the official enroll/eject paths — an out-of-band unmount records nothing
  and is healed. Tests cover the states, persistence, the replug rule, and the reconcile gate.
- **`watchdog.go` — intent-gated reconcile + flapping guard (3C)** — the re-mount candidate (device
  present, not mounted) now fires ONLY for an `enrolled` drive (via `IntentReader`); a present→absent
  transition (device gone) calls `OnAbsent`. Exponential backoff (`debounce·2^fails`) + an alert after
  4 failed cycles + a hard stop after 8 (no infinite loop). Failure = "still not present a full backoff
  window after we dispatched" (a slow async re-mount isn't miscounted). Tests: colleague-unmount→
  reconciled; ejected/new/decommissioned→left alone; ejected→absent→replug→auto-mount; flapping→caps.
- **`internal/localapi`** — `POST /disks/guest-attach` records `enrolled`; `POST /disks/eject` records
  `ejected` (BEFORE unmount, while the durable-id still resolves) via the new `IntentRecorder`. `main.go`
  opens one `IntentStore` (`<StateDir>/drive-intents.json`) shared by the watchdog + local API; open
  failure degrades to ungated legacy remount (logged).

## v0.26.0 — slice 10 P2 activation: guest-reboot endpoint (user-triggered drive activation) (2026-06-12)

A drive enrolled into a RUNNING unprivileged guest can't be live-activated (proven: `pct set` won't
hot-apply; `/proc/<pid>/root` bind → mount-locking refusal; `nsenter -m` loses the host source). So the
bind activates at the next guest boot. This adds the user-triggered restart path.

- **`POST /guest/reboot` (`internal/localapi`)** — self-scoped (vmid from token). Runs `pct reboot
  <vmid>` **detached** (it blocks ~30s until the guest is back) and returns **202** immediately, so the
  calling controller gets a clean response before the reboot takes it down (the agent is host-side and
  survives). `GuestBinder.RebootGuest` over the fenced runner. Tests: `TestGuestReboot_Accepted`
  (202 + RebootGuest invoked for the token's vmid), `TestGuestReboot_CrossGuest403` (body vmid mismatch
  refused, no reboot). Pairs with controller v0.49.0 (pending-activation detection + "Újraindítás most").

## v0.25.0 — slice 10 P2: bind enrolled user-data drives into the guest (passthrough) (2026-06-12)

External user-data drives are mounted on the HOST but were never passed INTO the guest (diagnosed
Branch A), so apps silently wrote to the rootfs and the controller couldn't see them. This adds the
guest passthrough. Spike-proven on 9201 first (see REPORT / the usb-passthrough-spike findings):
`pct set` **bind form** (host path, never `storage:size`), `chown` to the guest base (idmap not clean
for mixed-ownership data), `shared:49` propagation host↔guest automatic.

- **`POST /disks/guest-attach` (`internal/localapi`)** — self-scoped (vmid from token). Binds an
  enrolled drive's **felhom-data namespace** into the guest at `/mnt/<name>` (**Model A**: the
  felhom-data dir is the bind source mounted AT `/mnt/<name>`, so only Felhom's namespace crosses into
  the guest — the customer's other data on the drive never does). Idempotent (returns the existing slot
  if already bound); picks the lowest free `mpN`; validates `where` is `/mnt/<name>` (no traversal).
- **`GuestBinder` (`internal/localapi/guestbind.go`)** — the host-root steps over the fenced
  `proxmox.Runner` (same pattern as the provision back-half's bind): `mkdir -p <drive>/felhom-data` →
  `chown 100000:100000` the namespace ROOT (not -R; per-app subdirs are chowned at deploy) → `pct set
  <vmid> -mpN <drive>/felhom-data,mp=/mnt/<name>` (RW bind). The namespace is created fresh + uniformly
  owned, which sidesteps the drive's pre-existing mixed-ownership data entirely.
- **Tests** — `TestGuestAttach_*`: free-slot selection (mp0 when mp9 taken), idempotency (no re-bind +
  `already:true`), bad-path rejection (traversal/non-/mnt/multi-component), not-configured 503.

Pairs with felhom-controller P2C (enroll triggers attach) + the golden's `/mnt:rslave` controller bind
(P2B). Self-heal reconcile (P3) and dual-role (P4) follow.

## v0.24.0 — role-gate the eject path (system/backup mounts are unmount-protected at the agent) (2026-06-12)

Closes the eject gap in the storage-authorization redesign: `POST /disks/eject` now **refuses to
unmount a system or backup storage**, enforced at the agent — not just hidden in the controller UI.
A direct API call (or a compromised controller) trying to `eject {where:"/var/lib/vz"}` or the PBS
mount is refused 403; only `user-data` mounts are ejectable.

- **`handleDiskEject` (`internal/localapi/disks.go`)** — before `Unmount`, resolves the AUTHORITATIVE
  protection role of the storage mounted at `where` (the agent's own storage-view + host-topology
  classification, never the caller's claim) via the new `roleForMountPath`. Refuses (403, no
  `Unmount`) unless the role is `user-data`. **Fails SAFE**: an unresolvable mount (view error or no
  storage target at that path) → treated as protected → refused (the same most-protected-on-ambiguity
  default the wipe gate uses). Mirrors the wipe path's "protected — eject refused by role" logging.
- **`roleForMountPath` + `hostReader` seam** — `roleForMountPath` keys `RoleForStorage` on the mount
  path (the eject input), mirroring `deviceRole`. `Options.HostReader` (optional; defaults to the
  production `*storage.ProcHostReader`) injects the root-free topology reader so the role-gate is unit-
  testable. `handleDisks`/`deviceRole` now share the same seam.
- **Tests** — `TestEject_RoleGated` asserts a `system` and a `backup` mount are refused with **no
  `Unmount`**, a `user-data` mount ejects, and an unresolvable mount fails safe to refused (the same
  non-hollowness the wipe tests use). `TestEject_UnmountAndDependents` updated to a user-data target.

## v0.23.0 — device-ROLE classification + tiered storage-wipe gate (system/backup operator-only, user-data customer-confirmable) (2026-06-11)

The storage-authorization redesign (agent half). The gate's destructive-wipe path is now **tiered by
the device's protection ROLE**, which the agent classifies from its OWN inspection — never the
caller's claim (the storage analog of classify.go's data-bearing verdict).

- **`internal/storage/role.go`** — `DeviceRole` (`system` | `backup` | `user-data`) + the
  authoritative classifier. `RoleForStorage` (storage-view targets) and `RoleForRawDevice` (a raw
  device, e.g. a fresh disk in the init flow) map a device to its tier via `SystemDisks` (the
  whole-disks backing `/`, `/boot`, `/boot/efi`, root-free reads). Rules: `pbs` → backup; `lvmthin` /
  builtin `local` / nfs / cifs / unknown → system; `usb` / `local-dir` on a **non-system external
  device** → user-data. **Fail-safe**: any ambiguity (system disks unknown, or an unrecognizable
  device topology) → **system** (most-protected) — never silently user-data.
- **`GET /disks`** — each `DiskInfo` now carries `role`. The controller drives the UI from it
  (system/backup get a lock + no destructive controls; user-data is customer-manageable).
- **Gate tier (`reconcile`)** — new `CustomerConfirmable` disposition + `Gate.AuthorizeStorageWipe`:
  - role=**user-data** → **customer-confirmable**: allowed iff the request carries an explicit
    customer confirmation **bound to the device's durable id** (the agent re-resolves the durable id
    and matches; a confirmation for one disk can't wipe another). **No operator signature.** A
    user-data drive is already within the in-guest controller's blast radius (it bind-mounts `/mnt`),
    so customer-confirmation adds no new reach. Recorded in the **audit log** with the durable id
    (`AuditRecord.DurableID`).
  - role=**system**/**backup** → unchanged **operator-signature** (`pending_signature`). The
    `confirmed` flag is **IGNORED** — a compromised controller asserting `confirmed:true` on a
    protected device is refused **by role**. Every other destructive class (`guest_destroy`,
    `decommission`, `restore_overwrite`, `key_rotation`) keeps operator-signature exactly as before.
- **`POST /disks/format`** — accepts `confirmed` + `durable_id` (inert for system/backup). The
  data-bearing path tiers by role: user-data customer-confirmed → `mkfs`; user-data unconfirmed →
  403 `needs_confirmation` (+ the durable id to confirm against, NOT an opsign command); system/backup
  → 403 with the operator-signature pending op (as before). Blank devices stay benign `mkfs`.
- **Tests** — `role_test.go` (demo-storage mapping + fail-safe), `storage_wipe_test.go` (the gate
  refuses a `confirmed` wipe on system/backup → no exec; durable-id mismatch / missing-durable
  refused; unknown role fails safe), and the localapi format-handler branches (user-data confirmed →
  mkfs; user-data unconfirmed → needs_confirmation, no opsign; confirmed-but-protected → still refused).

Pairs with the controller's lockout + type-to-confirm UX + drive-list restyle.

## v0.22.0 — expose durable_id in GET /disks (enable controller-side guided storage) (2026-06-11)

One-line, read-only addition: `localapi.DiskInfo` gains `durable_id` (mapped from
`StorageTarget.DurableID`, e.g. `"uuid:<fs-uuid>"` for usb/local-dir). The de-privileged controller
cannot read a device's fs UUID itself, yet `POST /disks/assign` mounts strictly by UUID — so without
this it could not complete the guided init/attach flows. The controller strips the `uuid:` prefix to
get the assign key. No new privilege, no behaviour change to format/assign/eject or the data-bearing
gate. Pairs with `felhom-controller` v0.43.0 (the storage-management UI rebuild).

## v0.21.0 — agent-managed split-horizon LAN resolver (internal/lanresolver) (2026-06-11)

LAN clients can now reach their guest **directly** at the same public hostname with the same real
wildcard cert (no Cloudflare hairpin), via a host-side dnsmasq the agent manages. The host is the
stable anchor (static LAN IP); the guest stays DHCP/ephemeral and the agent tracks its live IP.

- **`internal/lanresolver`** — renders a dnsmasq base drop-in (bind to the host LAN IP, no-resolv,
  upstreams) + a per-customer drop-in `local=/<domain>/` + `address=/<domain>/<guest-ip>`. The proven
  two-line shape: `local=` makes dnsmasq authoritative for the zone so **AAAA returns NODATA** (no
  Cloudflare-AAAA split-brain — the guest has only link-local v6), `address=` is the wildcard A; all
  other names (and their AAAA) forward upstream unchanged.
- **`Manager`** ensures dnsmasq present (apt) + the base config + enabled, discovers the guest's live
  IPv4 (`pct exec <vmid> -- ip -4 -o addr show dev eth0`) and domain (read from the guest controller's
  pulled `controller.yaml` — the v2 bootstrap omits it), writes drop-ins **write-if-changed**, and
  **reloads** (not restarts) dnsmasq. Tolerates the early-boot pre-lease window (empty IP → skip+retry,
  never a blank record). Logs IP transitions.
- **`Loop`** — a 7th daemon goroutine: every interval (default 300s) it enumerates provisioned guests
  (`/var/lib/felhom-agent/guests/<vmid>/`) and reconciles each, so the resolver follows DHCP IP changes.
  Config `lan_resolver.{enable,host_ip,upstreams,interval_seconds}` (host_ip defaults to the local-API
  bridge IP). `--selftest=lanresolver -vmid N`.
- **`configs/felhom-agent.sudoers`** — new `FELHOM_DNSMASQ` alias (apt install dnsmasq; install
  felhom-*.conf drop-ins; systemctl enable/reload dnsmasq; rm felhom-*.conf; the two FIXED `pct exec`
  reads). The agent never touches `/etc/resolv.conf` (host's own resolution unaffected).
- **Box-down robustness** is a documented **router config** (DNS = [host-IP primary, upstream
  secondary]) so a box reboot degrades to the Cloudflare path, not total DNS loss — see REPORT install step.
- Spiked live on felhom-pve first (`:53` free, host IP static `192.168.0.162`, host DNS intact, full
  loop from a real LAN client returned the guest IP + AAAA NODATA + the real wildcard cert `200 0`).

## v0.20.0 — golden: stacks-dir bind + per-guest hostname/CT name + bake base-infra images (2026-06-11)

Lockstep with `felhom-controller` v0.41.0 + a golden rebake. Changes in `configs/build-golden.sh` and
the provision path; no change to the proxmox/authz/token fences.

- **Section-G mount fix (the load-bearing one):** the in-guest controller writes app/infra compose
  stacks under `/opt/docker/stacks` *inside its container*, but the baked controller-bootstrap `docker run`
  never bind-mounted that path. So `docker compose up` (run by the GUEST daemon over the shared socket)
  resolved every relative bind source on the guest filesystem — silently creating empty dirs — which
  broke **every** bind-mounted stack (base infra AND customer apps like immich/nextcloud). The bootstrap
  unit now `mkdir -p /opt/docker/stacks` and adds a **same-path host bind**
  `-v /opt/docker/stacks:/opt/docker/stacks` (a named volume would NOT fix this). Empirically confirmed on
  guest 9201 before writing the fix.
- **Per-guest container hostname (3A):** the bootstrap unit derives `customer.id` from
  `/etc/felhom-bootstrap/bootstrap.json` with a portable `sed` parse (NO jq in the golden) and passes
  `--hostname <customer-id>` to `docker run`, so the controller's `os.Hostname()` (its hub-reported
  hostname) is the customer id, not the Docker container ID. Fail-safe: no parse → no `--hostname`.
- **Per-guest CT/LXC name (3B):** `--selftest=provision` now defaults `-hostname` to the (DNS-safe
  sanitized) `-customer-id` when not given, so the bring-up's existing `SetConfig hostname` step
  (`bringup.go`) names the CT meaningfully (e.g. `demo-felhom`) instead of inheriting the golden's
  `felhom-golden`. New `sanitizeHostname` (lowercase, collapse invalid → `-`, trim, ≤63).
- **Bake base-infra images:** the golden now also pulls the three PINNED, PUBLIC base-infra images
  (`traefik:v3.6.7`, `cloudflare/cloudflared:2026.6.0`, `gtstef/filebrowser:1.3.3-stable`) into its Docker
  storage so the controller's first-boot bring-up is OFFLINE-capable. A hard gate (`docker manifest
  inspect`) fails the bake early on a bad pin. Tags MUST match the controller's `internal/infra` constants.

## v0.19.0 — bootstrap contract v2: agent relays the hub retrieval passphrase (no host key in the guest) (2026-06-11)

Lockstep with `felhom-controller` v0.40.0. Fixes the onboarding 401: a freshly provisioned guest's
controller used to come up with the agent's **host** hub key baked in, which the hub's `/api/v1/report`
(customer-scoped auth) rejects. The agent now bakes a **v2 bootstrap** carrying only what the controller
needs to **pull** its own config from the hub — the agent never touches the customer-scoped key or CF
tokens.

### Changed — bootstrap contract `v1 → v2` (`internal/provision`)
- `SchemaV1 → SchemaV2 = "felhom.bootstrap/v2"`. **`DocCustomer`** drops `name`/`domain`/`email` (keeps
  `id`). **`DocHub`** drops `api_key`/`host_id`, adds **`retrieval_password`** (the customer's hub
  retrieval passphrase — SECRET). `DocLocalAPI` unchanged. The contract is byte-compatible with the
  controller's `internal/bootstrap.Bootstrap` (cross-repo round-trip verified).
- `backhalf.go`: renders the v2 Doc; validation now requires `customer.id` + `hub.url` +
  `hub.retrieval_password` (was `customer.id` + `customer.domain`). Write/0600/chown/`pct set` unchanged.
- `cmd/felhom-agent/main.go` `--selftest=provision`: **new required `-hub-password`** flag (the customer's
  hub retrieval passphrase; the customer must already exist in the hub). Stops baking `cfg.Hub.APIKey` /
  `cfg.Hub.HostID`. `-customer-domain/-name/-email` still accepted (bring-up may use them) but NOT baked.

### Changed — `configs/build-golden.sh`
- Default `CONTROLLER_IMAGE` bumped off the stale `:v0.35.0` → `:0.40.0` (matches the registry's no-`v`
  tag convention; latent footgun fixed).

### Tests
- `doc_test.go`/`backhalf_test.go` updated to the v2 shape (assert no `api_key`/`host_id`,
  `retrieval_password` present, `customer` carries only `id`). `go build ./... && go test ./...` green.

## v0.18.0 — slice 10D: DR capstone — identity escrow + restore-mode consumption (agent side) (2026-06-10)

The agent half of the slice-10 DR capstone (closes slice 10). Grounded by both 10-series spikes
(escrow-consumption + identity-restore). The hub half (recovery-mode toggle, re-enroll + credential
rotation, directive serving) is hub v0.11.0. **Operator-side rotation model (locked):** the hub holds
no Cloudflare write-power; the destructive tunnel/PBS rotation is the operator's step from a trusted
environment (same spirit as 10B).

### Added (`internal/escrow`)
- **Identity escrow** (`identity.go`): `WrapIdentity`/`UnwrapIdentity` (+ `…Bundle`) wrap the
  `{tunnel_token, pbs_token}` bundle under the SAME recovery code `R` via **`age`** (scrypt +
  ChaCha20-Poly1305 — a vetted passphrase-AEAD, not hand-rolled), reusing the K-escrow pty mechanism
  (passphrase via the tty, data via files; `R`/tokens never logged). Same two-factor, zero-knowledge
  shape as the K-escrow. A **wrong R fails closed** (no bundle). `age` is a runtime dep for the
  identity path (analogous to proxmox-backup-client for K).
- **`escrow.Create`** gains an optional `IdentityBundle` → also emits an `IdentityBlob` under the same
  R (additive; the K-escrow + 10C `Consume` paths are byte-unchanged). Self-verifies the identity
  round-trip before shipping.
- **`--selftest=escrow-create -identity-bundle <file> -directive <file>`** — also wrap + upload the
  identity blob + the **non-secret** DR directive (pbs repo/ns, expected key fingerprint, tunnel id).
- **`--selftest=identity-consume -blob <file> -keydest <file>`** (R via `FELHOM_RECOVERY_CODE`) —
  recover the identity bundle through the real code; tokens written 0600, never logged.

### Tests
- identity bundle round-trips (wrap→unwrap byte-identical; blob is opaque ciphertext); wrong R fails
  closed + the blob stays retryable; input validation. K-escrow/10C tests byte-unchanged (additive).
  (age integration tests gated to a host with the `age` CLI.)

## v0.17.0 — slice 10C: escrow consumption (productionize the spike) (2026-06-10)

Turns the throwaway 10C spike harness into a real, tested **`Consume`** path: recover the PBS key
`K` from an R-wrapped escrow blob, **gate it on the expected fingerprint**, and install it for the
restore. The spike already proved the crypto + real-data restore; this bakes its findings into
production code. **Agent-only** — 10C *reads* the four inputs as parameters (so it stays
standalone-testable); 10D sources blob/fingerprint/PBS-connection from the hub and prompts for R.
**Zero-knowledge holds**: the hub serves everything except **R** (by hand from the customer), so a
hub compromise alone still can't decrypt.

### Added
- **`escrow.Consume(ctx, blob, R, expectedFingerprint, keyDest)`** — the consumption contract:
  1. **Unwrap** the blob (a copy — F-C6: the input blob is read-only → a failed Consume is
     **retryable**) with `R`; a **wrong R fails closed** at the scrypt KDF (F-C3) → a clear,
     R-free error, **nothing written**.
  2. **Fingerprint gate (F-C4)** — `KeyFingerprint(recovered)` must equal the expected (the hub
     knows it); a mismatch **fails fast + loud, no install, no restore attempted**.
  3. **Atomic install (F-C2)** at `keyDest` (`0600`, write-temp-sibling→rename); any failure leaves
     **no partial install**. The recovered key lives only in a `0700` tempdir that is always removed.
  **Secret discipline:** `R` and key bytes are never logged/persisted (only fingerprint prefixes);
  `K` is never mutated.
- **`--selftest=escrow-consume`** (`-blob -fingerprint -keydest`, R via env `FELHOM_RECOVERY_CODE`
  to keep it off the command line) — invokes the real `Consume` live (the spike's S3 via the
  production path, not a harness).

### Tests (non-hollow)
- valid → key installed + `KeyFingerprint(dest) == expected` + `0600` + blob byte-unchanged;
  **wrong R** → error, **no file at dest**, blob unchanged; **fingerprint mismatch** → fail fast,
  **no install** (the gate runs before any restore); input validation; format-tolerant fingerprint
  compare (no empty-fingerprint gate-bypass); atomic-install permissions (integration tests gated to
  a host with `proxmox-backup-client`).

## v0.16.0 — slice 10B: operator-signed destructive completion (offline key + signing CLI) (2026-06-10)

The security centerpiece: a destructive op runs ONLY on a verified, operator-signed authorization
— signature valid against a **pinned** operator pubkey (never the hub's or the blob's), nonce
unseen + durably burned, in-window, host-bound, and **resource-bound to a DURABLE device id** that
execution re-resolves + re-inspects. Decision (a): **offline operator key + signing CLI**,
hardware-key-ready (`sk-`/YubiKey via ssh-keygen). The key floor holds: the signing key is NOT in
the hub and NOT in the agent. Concrete consumer: this **closes the 8C data-bearing-wipe
`pending_signature` gap**. Pairs with hub v0.10.0.

### Added
- **`cmd/felhom-opsign`** — the operator's offline signing CLI. Builds the canonical `OpBlob` by
  **reusing `authz.CanonicalBlob`** (the exact production path the verifier authenticates over — so
  signer + verifier can never drift) and signs it with **`ssh-keygen -Y sign -n felhom-op-v1`**
  (hardware-ready). Output: a `{op_blob_b64, sig_armored}` envelope to hand to the hub jobs queue
  (optional `--upload`). Touches ONLY the operator's signing key.
- **`authz.CanonicalBlob`** — promoted to production (was test-only) so the CLI + verifier share one
  canonical-bytes source; params canonicalized (sorted keys, compact).
- **`internal/storage` durable device identity** (`durable_device.go`): `DeviceDurableID` (derive a
  stable `byid:`(wwn/serial)/`byuuid:` id from the world-readable udev symlinks — no privilege, no
  subprocess) + `ResolveDurableDevice` (re-resolve to the current `/dev` path; a path-only/unknown
  scheme is REFUSED). The resource-level anti-retarget.
- **`internal/signedjobs`** (new): the queue consumer. `Runner` fetches each opaque job → runs it
  through the **gate** (the LOCKED authz pipeline) → on all-pass hands the verified op to an
  `Executor`; the order is **verify → nonce-burn (durable, in Verify) → execute → clear job**. The
  **`WipeExecutor`** is the 8C consumer: resolve the signed durable id → **re-derive + match**
  (anti-retarget) → **re-inspect (8C classifier)** the device is still the data-bearing target →
  `mkfs`. A vanished/changed/non-data-bearing device or a path-only binding is refused **even with a
  valid signature**. Wired as a second `EnvelopeObserver` (runs on `HasSignedOps`).
- **`hub.Client.Jobs` / `CompleteJob`** + `hub.MultiObserver`; the 8C format refusal now **surfaces
  the bound op** (op + durable id + host) in its 403 `pending_op` + a `felhom-opsign …` hint.

### Pinning / rotation
- Operator pubkeys are pinned via `authz.signers` (config, trusted path — provision/agent config,
  NEVER hub-alone), **multiple** keys (KeyID selects; role-scoped), so a backup/rotation key exists
  without a flag-day. Unchanged from the slice-4 verifier wiring; 10B activates the execute path.

### Tests (real crypto, non-hollow)
- `signedjobs` runner over the **real** gate+verifier (in-Go minted SSHSIGs): valid → executor runs
  once + job cleared; **replay** (nonce burned) / **non-pinned signer** / **expired** / **retarget**
  (other host) / **forged sig** / **no pinned signer** → all rejected, **executor never called**;
  malformed envelope cleared.
- `WipeExecutor`: valid → `mkfs` runs; **path-only**, **durable-id mismatch**, **device gone**,
  **re-inspect non-data-bearing**, **not-probed** → all refused, `Format` not called.
- `storage` durable: wwn-preference, uuid-fallback, path-only/traversal refusal, round-trip,
  missing-device error (symlink tests gated to Linux — the agent's OS).

## v0.15.0 — slice 10A: hub desired-state serving — the "Down" channel (2026-06-10)

The agent half of slice 10A. The control envelope (`hub.ControlEnvelope`) stops being "reserved — ignored" and becomes the live **Down channel**: a cheap change-notification on every heartbeat. The agent caches the hub's desired-state + its generation; only when **`DesiredGeneration` advances** does it fetch the full state (the heartbeat stays light, the heavy state moves on change). The engine then reconciles **benign** deltas and the gate marks an explicit **destructive** delta `pending_signature` (no signer in 10A → never executed; signed execution is 10B). Pairs with hub v0.9.0.

### Added / changed
- **`internal/reconcile`**: `DesiredGuest.Decommission` — the canonical **destructive desired-state delta** (an EXPLICIT flag, not "absent from the list", so a partial hub list can never mass-destroy). The planner emits `ActionDecommission` → `ClassDecommission` → Destructive → the gate refuses it `pending_signature`. `Reconcile` now counts a `pending_signature` refusal as **`Result.Pending`** (expected, logged INFO) rather than a failure; any other refusal stays a real failure. `ActionDecommission` has **no executor** (slice 10B) — a defensive guard refuses to run it. New **`CachingProvider`** (thread-safe DesiredState + generation cache; `Desired`/`Update`/`Generation`) — the production `DesiredProvider`, replacing `EmptyProvider` in the daemon engine (empty until the hub serves intent → cold-start is a live no-op, unchanged).
- **`internal/hub`**: the **`ControlEnvelope`** fields are now active (DesiredGeneration drives the fetch, HasSignedOps noted). New wire types **`DesiredStateResponse`** + **`WireDesiredState`** (guests + forward-compat `restore_directive` (10D) / `pbs_namespace` / opaque `storage_manifest`+`backup_policy`) + **`WireDesiredGuest`** (vmid/run/spec/description/decommission). New **`Client.FetchDesiredState`** (GET `/api/v1/hosts/{host_id}/desired-state`, self-scoped to the client's own host). New **`EnvelopeObserver`** loop seam + `SetEnvelopeObserver` — the loop hands the envelope to the sync layer each cycle (hub does not import reconcile/desired).
- **`internal/desired`** (new): the **`Syncer`** — implements `hub.EnvelopeObserver`, fetches desired-state on a generation advance, maps the wire shape to the reconcile domain, and updates the `CachingProvider`. Caches the **fetched** generation (robust to a generation that advanced mid-fetch); a fetch failure keeps the last-known state. `restore_directive` is carried + logged, not acted on (10D). Wired in `cmd/felhom-agent` (daemon): provider → engine, syncer → loop.

### Tests
- reconcile: a desired-state with one benign + one decommission delta → **benign applied, destructive gated pending (not executed)**; `Plan` emits decommission-only for a decommissioned guest + classifies Destructive; `CachingProvider` update/isolation.
- desired: **fetch-once-on-advance** (no re-fetch on an unchanged generation), fetch-failure-keeps-cache, caches-the-fetched-generation.
- hub client: `FetchDesiredState` hits the self-scoped path with the bearer + decodes (incl. `restore_directive`); a 403 is a typed `HTTPError`.
- loop: the cycle notifies the observer + adopts `PollIntervalSeconds`; a report error skips the observer.
- cross-repo golden: `testdata/desired-state.golden.json` + `control-envelope.golden.json` decode + key-set guard, **byte-identical** with felhom.eu/hub.

## v0.14.0 — slice 9: host metrics to the controller (`GET /host/metrics` + CPU-temp collector) (2026-06-10)

The de-privileged controller (slice 8C) sees only its own cgroup, so it can't read host health itself. Slice 9 **re-serves** the slice-4 collector's host + per-storage view to the customer over the local API, plus the one missing collector — CPU/chassis temperature — so the customer sees their box's health in the controller. Host-wide, token-authed, fresh (a live collect, not the 15-min hub snapshot). Assumption: **one customer per host** (the home-server model); if a host ever serves multiple customers, host-wide CPU/mem would leak cross-customer load → revisit then.

### Added / changed
- **CPU/chassis-temp collector** (`internal/hub/cputemp.go`): `SysfsTempReader` reads the CPU package temperature straight from sysfs — hwmon (`coretemp`/`k10temp`/`zenpower`/`cpu_thermal`, preferring the `Package id 0` input) then the thermal zones (preferring `x86_pkg_temp`/`coretemp`/`cpu-thermal`, falling back to `acpitz`). **No external binary, no privilege** (sysfs nodes are world-readable), so the root-CLI fence is untouched. **Graceful-null**: a missing sensor, an unsupported board, an implausible reading (outside 5–150 °C), or any read error all degrade to `null` ("n/a") — a missing sensor never fails the report. Wired into the collector via the new `TempReader` seam (nil-safe).
- **`HostMetrics.CPUTempC *int` (`cpu_temp_c`)** — new nullable wire field on the **shared** `HostMetrics` struct (same nullable contract as the disk `SmartSummary.TemperatureC`). It rides the **hub report too** (operator freebie) → cross-repo host-report golden updated.
- **`Collector.HostMetricsNow(ctx)`** — a fresh `NodeStatus` + CPU-temp read returning just the host block, the source for the local API (current cpu%/temp, not the 15-min snapshot). `Collect()` now also populates `cpu_temp_c` on the hub report. `Collector.SetTempReader` injects a fake in tests.
- **`GET /host/metrics`** (`internal/localapi/host_metrics.go`): host-wide health (cpu%/mem/load/uptime/`cpu_temp_c`) + per-storage capacity targets (total/used/fraction, thin-pool, SMART temp+wear). Token-authed via `withGuest` (host-wide data; cross-guest `?vmid=` still 403). Best-effort on storage (a view error still returns the host block). Served only when the `HostMetrics` provider (the shared collector) is wired — else 503 "not configured". Wired in `buildLocalAPIServer`.

### Tests
- `cputemp_test.go`: a fake `/sys` layout proves hwmon package-preference, hwmon first-input fallback, thermal-zone-by-type selection over a non-CPU hwmon, **graceful-null on a sensorless host** (no error), and rejection of implausible (0 m°C) readings.
- `hostmetrics_test.go`: `HostMetricsNow` populates the temp, gracefully nulls it, hard-errors on `NodeStatus` failure; `Collect()` carries the temp.
- `host_metrics_test.go` (localapi): populated host+storage with a valid token; `cpu_temp_c:null` serializes; **401 without a token** (collector never invoked); 403 on a cross-guest `?vmid=`; 503 when not configured.

## v0.13.0 — slice 8B.2: quiesce downtime optimization (`snapshotted` phase) (2026-06-10)

The agent half of slice 8B.2. In snapshot mode, vzdump only needs the app-stopped state captured at
the **storage-snapshot moment**; after that it reads from the snapshot and the app can resume. The
agent now emits a **`snapshotted`** phase on `GET /backup/status` when the snapshot is taken, so the
controller (v0.38.0) resumes its app early — app downtime drops from *whole-backup* to
*until-snapshot* with no loss of app-consistency. Validated Phase-0 first on PVE 9.2.2: the marker is
`INFO: create storage snapshot 'vzdump'`; downtime ~24s→~1s for a 934 MB guest.

### Added / changed (`internal/backup` + `internal/localapi`)
- **`BackupRunner.BackupWithSnapshotHook(ctx, vmid, onSnapshot)`** — while the vzdump runs, a watcher
  tails the task log (`TaskLogTail`) for the **`create storage snapshot`** marker and fires
  `onSnapshot` **once**. The marker only appears in snapshot mode (stop/downgraded takes no storage
  snapshot), and the watcher also bails on `backup mode: stop` — so it never fires in stop mode.
  (`Backup` keeps its signature for the scheduler/selftest; both share one body.)
- **`/backup/status` phase `snapshotted`** (between `running` and `done`): `handleBackup` passes the
  hook → `markSnapshotted` flips the running job to `snapshotted`. `done`/`failed` semantics unchanged.

### Tests
- localapi: snapshot mode → phase reaches `snapshotted` before `done` (gated fake holds the backup
  open); stop mode → `snapshotted` **never** emitted (stays running → done). runner: the watcher
  fires `onSnapshot` on the marker; in stop-mode log it never fires. `snapshotWatchInterval` is a
  package var so tests run fast.

## v0.12.0 — slice 8C Phase A: disk endpoints + data-bearing classifier gate + mkfs executor (2026-06-10)

The agent half of slice 8C, Phase A (additive). Adds the host disk-management endpoints the
controller's disk UI drives — with the **8C security invariant**: the agent decides
data-bearing-ness by **inspecting the actual device** (agent-internal evidence), NEVER from the
caller's claim. A compromised controller asserting "this drive is blank" cannot wipe a data-bearing
drive. (Controller rewire + disk-subsystem retirement + de-privilege are Phases B/C, `felhom-controller`.)

### Added
- **`internal/storage` — `mkfs` executor + data-bearing inspection.** `SudoHostOps.Format(device,
  fstype)` (device-pinned, `ValidateBlockDevice`+`ValidateFSType`, narrow `FELHOM_FORMAT` sudoers —
  `mkfs.ext4 -F` / `mkfs.xfs -f` on a `/dev/*` path the agent fine-validates first).
  `SudoHostOps.InspectDevice(device)` → `DeviceProbe` (filesystem signature via `blkid -p`, partition
  table / partitions / mount via `lsblk -J`). **`DeviceProbe.DataBearing()` is conservative**: any
  signature / partition table / partition / mount — OR a probe that did not read cleanly — is
  data-bearing (fail-safe; an unreadable device is never called blank).
- **`internal/localapi` — the §6 disk endpoints**, all self-scoped (token→guest; cross-guest 403):
  - `GET /disks` — host drives + a **data-bearing flag** (UI hint). Read-only/benign.
  - `POST /disks/assign` — attach a drive as a mount (benign, additive → `EnsureMount`). Self-serve.
  - `POST /disks/eject` — safe-unmount (benign, data preserved) + the **dependent guests** that
    mount it (so the controller can warn which apps lose that storage).
  - `POST /disks/format` — **the security centerpiece**: the agent **inspects the device itself**;
    blank → benign → `mkfs`; **data-bearing → ClassStorageWipe → the slice-4 gate → refused
    `pending_signature`** (the operator-signed completion is slice 10). The caller's claim is
    ignored — only a device the agent reads as blank is formatted.
- `storageGateAdapter` bridges the format path to the slice-4 reversibility gate (no new gate/crypto).

### Tests
- localapi (security matrix): blank device → **mkfs called, gate not consulted**; a **data-bearing
  device → 403, mkfs NEVER called**, gate consulted (`pending_signature`); an **ambiguous/unprobed
  device → treated destructive** (fail-safe); even a gate that *allows* does not format data-bearing
  in 8C; assign → `EnsureMount`; eject → `Unmount` + dependent guests; cross-guest → 403; bad
  device/fstype → 400; unconfigured → 503.
- storage: `ValidateBlockDevice`/`ValidateFSType` (whitelist + injection rejection); `InspectDevice`
  blank/filesystem/partition-table/mounted/failed-probe-fail-safe; `Format` invokes the right `mkfs.*`.

## v0.11.0 — slice 8B: app-consistent backup — /backup/due policy + /backup/status phases (2026-06-10)

The agent half of slice 8B (doc 03 §8). Turns the 8A thin backup stubs into the real policy the
in-guest controller's quiesce loop drives (controller half: `felhom-controller` v0.36.0). No hub
change. The downtime optimization (`vzdump --mode snapshot` + a `snapshotted` phase) is the 8B.2
fast-follow; the hub-served per-guest policy is slice 10.

### Changed (`internal/localapi`)
- **`GET /backup/due`** — real **cadence** policy (replaces the 8A "never backed up" stub): a guest
  is due when no **successful** backup is recorded OR the newest one is older than the agent-local
  cadence (`backup.backup_cadence_seconds`, default 24h). A successful `POST /backup` flips due to
  **false** for the window, so the controller won't re-quiesce in a loop. A failed backup does not
  satisfy the cadence. Returns `age_seconds` for diagnosis.
- **`GET /backup/status`** — real **phases** `idle | running | done | failed` + the job id, so the
  controller can poll a backup to completion (was: just the latest stored backup).
- **`POST /backup`** — returns a **job id** + `running` phase; tracks the in-flight job and is
  **single-flight per guest** (a second POST while one runs returns the same job — no concurrent
  vzdump). On completion the job transitions done/failed and the result is recorded to the store.
- Config: `backup.backup_cadence_seconds` + `BackupCadence()`; the local-API server takes the cadence.

### Tests
- `/backup/due`: due when stale / no backup, **not due within the window after a success**, due again
  past the cadence, **a failed backup does not count**. `/backup/status`: running→done and
  running→failed (gated fake to observe the running phase). `POST /backup` single-flight (one vzdump
  for concurrent POSTs). All still self-scoped (token→guest).

## v0.10.0 — slice 8A: agent local-API server + provisioning back-half (2026-06-10)

The host-agent half of slice 8A (doc 03 §6). Adds the per-guest **local API** the in-guest
controller calls over the bridge, and the **provisioning back-half** that follows the slice-7
bring-up front half. Grounded by `felhom.eu/documentation/tests/slice8a-channel-deploy-spike-findings.md`
(commit `4a81a96` — channel + deploy plumbing proven; the 5 gotchas resolved here). Controller half
is `felhom-controller` v0.35.0. No hub change.

### Added
- **`internal/localapi`** — the HTTPS local-API server (doc 03 §6), the **per-guest authorization
  gate**. Serves a **persisted self-signed leaf** with a **stable SHA-256 fingerprint** (generated
  once; a fresh cert each boot would invalidate every baked bootstrap pin). The **7 §6 endpoints**,
  all **self-scoped to the caller's own guest**: `GET /storage` (this guest's mpN mounts + fast/slow
  class from the slice-5/7 storage view), `POST /snapshot`, `POST /rollback`, `POST /backup`
  (enqueued, crash-consistent — the app-consistent quiesce loop is 8B), `GET /backup/due` (thin in
  8A), `GET /backup/status`, `GET /restore-test/status`.
  - **Token store** (`tokenstore.go`): durable, crash-safe per-guest token→guest map that persists
    only a **SHA-256 hash** of each token (the plaintext exists transiently at mint→write-to-mount,
    then is discarded), last-write-wins per guest, fsync'd append-only JSONL (mirrors the nonce store).
  - **Self-scoping**: the VMID is resolved ONLY from the token; an explicit `vmid` (query/body) that
    disagrees → **403 and the proxmox op is never issued for the other guest**; absent/unknown → 401.
- **`internal/provision`** — the back-half: mint the per-guest token → render the stable
  **`bootstrap.json`** contract (schema `felhom.bootstrap/v1`; **no registry credential** — the
  controller image is baked into the golden) → write it `0600` → **`chown 100000:100000`** (the
  unprivileged-LXC mapped guest-root, spike gotcha 1) → attach a **read-only bind mount** via
  `pct set`. Host-side only (F3 — the agent never enters the guest; **no `pct exec`**). The token
  plaintext is never logged and never returned.
- **`--selftest=provision`** — the full chain on-demand: bring-up (provision) front half + the
  back half; keeps the guest for the golden's baked controller-bootstrap unit to deploy.
- **`config.LocalAPIConfig`** (`local_api`) — enable + bridge `listen_addr` + cert/key paths + token
  store path. The server is an optional 6th daemon goroutine, disabled cleanly when unconfigured or
  on a token-store/cert failure (the daemon still reports/reconciles).
- **`configs/build-golden.sh`** now **bakes the controller image** (pulled once on the trusted build
  host, then `docker logout` — no cred baked) + a **controller-bootstrap unit** that deploys the
  **baked** image from the config mount on boot (no login/pull at deploy).
- **`configs/felhom-localapi-firewall.example`** — host firewall narrowing of the local-API port to
  the guest bridge subnet (nft/iptables/PVE variants; defense-in-depth — the token stays the gate).
- **`configs/felhom-agent.sudoers`** — a narrow `FELHOM_PROVISION` alias (`chown 100000:100000` +
  `pct set` bind-mount, both confined to the agent-owned `/var/lib/felhom-agent/guests/*` path) for
  the non-root least-privilege deployment.

### Security / design notes
- The local-API leaf is pinned by **leaf-cert SHA-256** (decision: consistency with the agent's
  PVE/PBS pinning); the fingerprint is baked into each guest's bootstrap.
- The back-half's host-root ops (chown + bind-mount attach) are **NOT** added to `proxmox.Privileged`
  (which is fenced to its 3 exceptions) — they live in `internal/provision` and run through the shared
  `Runner` (direct as root, or `sudo -n` with the new sudoers alias). This is the per-guest
  provisioning host-root surface, host-side and F3-compliant.

### Tests
- localapi: self-scoping (cross-guest snapshot/rollback/backup → 403, op never issued for the other
  guest; own-guest uses the token's VMID), 401 paths, `/storage` class mapping, `/backup` enqueue,
  the thin `/backup/due`, status scoping; the token store persists only the hash (plaintext never on
  disk), last-write-wins, survives reopen, uniqueness; the leaf fingerprint is stable across reload.
- provision: writes `0600` + chowns + attaches the bind mount with the right args; the **token never
  appears in the Result**; the cross-repo `bootstrap.json` contract key-set is pinned.

## v0.9.0 — slice 7 close-out: PBS recovery-code escrow creation (2026-06-10)

The first code that touches the PBS client encryption key `K` and introduces the customer recovery
code `R`. Default posture is **zero-knowledge**: Felhom holds an opaque `R`-wrapped blob (cannot
open it), the customer holds `R`. Grounded by `felhom.eu/documentation/tests/slice7-escrow-spike-findings.md`
(round-trip proven on a throwaway: the `R`-recovered key restores a real encrypted snapshot). Hub
opaque storage is the `felhom.eu` half (hub v0.8.0); consumption/serving is slice 10.

### Secret discipline (overriding)
`R` is `crypto/rand`, ≥128 bits, surfaced **exactly once** and **never** logged/persisted/committed;
the wrap pty's echo is discarded so `R` can't leak. `K` is read by location, **never modified** (the
live key file is byte-unchanged — Wrap operates on a copy), never logged.

### Added
- **`internal/escrow`** — `Create` generates `R` (10 EFF-wordlist words ≈ 129 bits), wraps `K` under
  `R` via the **PBS-native** `proxmox-backup-client key change-passphrase --kdf scrypt`, and
  **self-verifies** the blob recovers `K` (fingerprint match) before shipping. The wrap is driven
  over a **stdlib pty** (`x/sys/unix`; spike F-A1 — the command is TTY-only) with **output discarded**
  (F-A2 — the pty echoes the passphrase). Opt-in outputs: **(b)** `R`-wrapped offline copy (two-factor,
  no extra trust) and **(a)** raw paperkey (single-factor, unrevocable — loud caveat).
- **`--selftest=escrow-create`** (`-storage`, `-paperkey`, `-offline`, `-upload`): surfaces `R` once
  to stdout (never the logger), prints the opaque blob's size/fingerprint/posture, and with
  `-upload` PUTs the blob to the hub (`/api/v1/hosts/{host_id}/escrow`, per-host key).
- Config: `escrow` section (`posture` default `zero_knowledge`, `pbs_storage_id`); `PBSEncKeyPath`
  helper (the `<id>.enc` key K).
- Runtime dependency on the `proxmox-backup-client` CLI (the PBS key+passphrase KDF).

### Tests
- `R` entropy ≥128 / 10-word format / uniqueness; integration round-trip (wrap→unwrap fingerprint
  match, **wrong-`R` fails**, **live `K` byte-unchanged**, blob ≠ plaintext key) guarded to
  linux+`proxmox-backup-client`; the agent→hub wire-contract key-set (mirrors the hub's).
- **Live-validated** (demo): `escrow-create` → `R` (10 words) surfaced once, blob 383 B opaque,
  self-verify ok, **live `K` sha256 unchanged**, exact `R` absent from stderr/journal.

## v0.8.0 — slice 7 Phase 1: unified bring-up reconcile job (provision + guest-loss DR) (2026-06-09)

The shared FRONT HALF of provision and guest-loss DR, as a journaled reconcile job mirroring the
slice-6 restore-test's crash-safety — but it KEEPS the guest on success and applies a
scenario-specific identity policy. Agent-only; no hub/wire change (the new guest auto-appears in
the host-report via `ListLXC`). Grounded by the slice-7 bring-up spike findings (commit `3342993`):
F1 (restore preserves the archived MAC → provision reset is unconditional), F3 (SSH host keys do
not auto-regenerate → a baked golden first-boot unit, not an agent guest-internal op), F4 (the
transient PVE config-lock 500 → bounded retry).

### Added
- **`reconcile.RunBringUp`** (`bringup.go`) — `BringUpSpec` (Mode `provision`|`dr_guest_loss`,
  Archive, VMID, RestoreStorage, Hostname, Cores/MemoryMB, RootfsGrowGB, Mounts, KeepMAC,
  BootTimeout) → `BringUpResult` (VMID, AssignedMAC, Pass, Verified, StartWarnings/Recognized).
  Sequence (each mutation preceded by journaling the owning entry): restore → identity reset →
  size → attach mounts → start LINK-UP. **Verdict is liveness (`waitRunning`), never the start
  exitstatus** (reuses the v0.7.0 WARNINGS surface). **Success KEEPS the guest** (no teardown).
- **Scenario-specific identity reset** (doc 03 §9): *provision* → fresh MAC unconditionally
  (`PUT net0` with `hwaddr` omitted → PVE regenerates, F1) + hostname; machine-id + SSH host keys
  regenerate guest-side on first boot (golden bake + the new unit) — the agent does NOT touch
  guest internals. *dr_guest_loss* → preserve continuity (keep hostname; keep MAC unless
  `KeepMAC=false`); never resets restic/tunnel/hub identity.
- **Compensating rollback** — any mid-flight failure destroys the just-created guest
  (`ClassGuestDestroy`, benign via `Provenance{SameTxnCreated:true}`, gated); on teardown failure
  the entry is left in-flight for `Recover`. New journal flag **`Rollback`** + `Recover`'s
  `recoverBringUp` reap a half-built guest left by a mid-job crash (idempotent, via `ListLXC`).
- **F4 config-lock retry** — steps 3+5 coalesced into ONE `PUT config` (net0+hostname+cores+
  memory+mpN); rootfs grow stays its own call. `setConfigWithLockRetry` retries ONLY the transient
  PVE config-lock 500 (`pveConfigLock`: 500 + "can't lock file"/"got timeout"); any other error
  fails immediately — never retried.
- **`--selftest=bring-up`** (`-mode provision|dr -archive -vmid -hostname [-keep]`) — runs the real
  journaled job (after a `Recover`), then tears the guest down unless `-keep`.
- **`configs/build-golden.sh`** — the validated golden recipe as a script, incl. the F3
  first-boot `felhom-regen-hostkeys.service` unit (Condition-gated: fires on provision, no-ops on
  DR). The slice-7 spike archive (which lacks the unit) is superseded.

### Deferred (stated, not built)
- Provisioning BACK HALF (controller deploy, bootstrap, per-guest token mint) → **slice 8**.
- Host-loss DR + PBS escrow consumption → **slice 10**.
- The SOURCE of a `BringUpSpec` (hub desired-state: which archive/VMID/mounts) → **slice 10**;
  this job takes the spec as input. `GuestMount` is defined minimally (no hub coupling).

### Tests
- provision happy path (fresh MAC = net0 without hwaddr, hostname, coalesced sizing+mount, rootfs
  grow separate, started, **guest NOT destroyed**); compensating rollback at each step (restore /
  config / start-task / waitRunning — asserts the guest WAS destroyed); DR continuity (MAC kept,
  hostname not reset) + DR `KeepMAC=false` resets MAC; liveness verdict (warnings+running pass /
  not-running fail); F4 (lock-500→retry→proceed; non-lock-500→fail without retry); owning entry
  journaled BEFORE restore; reserved/existing VMID refused; `Recover` rolls back / clean.

### Live-validated (demo-felhom)
- provision: fresh MAC + hostname; **SSH host keys regenerated by the baked golden unit** (agent
  issued no `ssh-keygen`), machine-id unique, Docker runs, clean DHCP lease → torn down.
- dr: continuity preserved (hostname + host keys kept). Recover: a killed mid-restore left an
  orphan; the re-run's `Recover` rolled it back (idempotent).
- **Live caught a bug, then fixed:** the host-key unit's `ExecStart` was `/usr/sbin/ssh-keygen`
  (203/EXEC); on Debian 13 it is `/usr/bin/ssh-keygen` — corrected in `build-golden.sh`, golden
  rebuilt, re-validated. (Mocked unit tests couldn't surface this; the live run did.)

## v0.7.0 — restore-test: verdict is liveness, not start-task exitstatus (2026-06-09)

Fixes a correctness bug found by the live hub-enrollment runbook: the self-restore-test reported
`pass:false` on **every** modern-distro guest. PVE's guest-start task exits `"WARNINGS: 1"` for the
benign systemd-nesting advisory (`WARN: Systemd 257 detected. You may need to enable nesting.`), and
`WaitTask` treated any non-`"OK"` exitstatus as a hard failure — so the verdict was decided by an
advisory exit code instead of by observed liveness, *before* the real boot check ran. A crying-wolf
test got it disabled on the demo host; this re-enables it. **Single bump (0.6.0→0.7.0) covering the
agent's part of both task phases**; the wire fields below are consumed by hub from **v0.7.5**.

Design invariant (in code): **warning classification affects *visibility only*; pass/fail is
liveness-only.** A wrong/stale recognizer can at worst over-notice a benign warning — it can never
false-fail and never hide a real warning.

### Added
- **`proxmox.WaitOptions.AllowWarnings`** — opt-in per call. When set, a task that completes
  `"WARNINGS: N"` is success with the `TaskStatus` (ExitStatus intact) returned so the caller can
  read/surface it. Default (`false`) keeps **every existing caller strict** (vzdump/restore/destroy
  warnings can be meaningful — relaxing them is a future per-call decision with evidence). Any
  non-WARNINGS non-OK exit is still a `*TaskError`.
- **`reconcile.RestoreTestResult.StartWarnings` / `.WarningsRecognized`** + a version-free recognizer
  (`benignWarningAnchor = "enable nesting"`, case-insensitive substring — contains no systemd version
  number, so it can't rot back into the bug at systemd 258+). `extractWarningLines` pulls `WARN…`
  lines from the start-task log.
- **`reconcile.GuestAPI.TaskLogTail`** — the engine fetches the start task's log to surface warnings.
- **`hub.RestoreTest.warnings` / `.warnings_recognized`** wire fields (`omitempty`), populated by
  `ToHubRestoreTest`. Additive: the deployed v0.7.4 hub ignores them; hub v0.7.5 consumes them
  (passed-with-warnings INFO, or WARN when not recognized). Cross-repo golden updated with the hub side.

### Changed
- **Restore-test start step** (`reconcile/restoretest.go`) now waits with `AllowWarnings:true`,
  surfaces any start warnings, and **continues to `waitRunning` as the verdict** — boot+running is the
  pass, exactly as before; a real (non-WARNINGS) start-task error still fails. The restore and
  scratch-teardown WaitTasks stay strict.
- **Restore-test scheduler logging** distinguishes a clean pass, *passed-with-recognized-warnings*
  (INFO), and *passed-with-unrecognized-warnings* (WARN) — nothing silent.

### Tests
- `WaitTask`: AllowWarnings accepts `WARNINGS` (status returned intact); AllowWarnings still fails a
  real error; default still fails on `WARNINGS` (existing callers unaffected).
- Restore-test (engine, mock proxmox): start-with-warnings + running → **pass** with warnings
  surfaced+recognized; unrecognized warning + running → pass, not-recognized; **not-running → fail
  regardless of warnings** (verdict is liveness); teardown still runs.
- **Regression guard:** the `"enable nesting"` recognizer matches the advisory for systemd 256–300,
  proving it's version-independent and can't silently rot back into the false-fail.

## v0.6.0 — slice 6 Phase B: PBS offsite tier (verify + PBS-API client + reporting) (2026-06-09)

Completes slice 6. The PBS spike (felhom.eu phase5-pbs-spike-findings.md) proved backup-to-PBS
and restore-from-PBS reuse Phase A UNCHANGED (PBS is just a storage target + a volid), and the
operator token needs no widening. So the only new agent code is the **verify capability + a
small PBS-API client + PBSSnapshot reporting**. Escrow + host-loss DR stay slices 7/10.

### Added
- **`internal/pbs` — the PBS-API client** (the agent's SECOND privileged external surface,
  slice-1 discipline): TLS **fingerprint-pinned** to the PBS leaf cert (a spoofed PBS →
  rejected, mirroring the PVE pin), **token auth** (`PBSAPIToken=<id>:<secret>`; id from the
  storage `username`, secret read at runtime from `/etc/pve/priv/storage/<id>.pw` — referenced
  by location, never logged/committed), typed, no shell. Methods: `Verify` (POST
  `/admin/datastore/<ds>/verify` → UPID), `Snapshots` (incl. the `verification` field),
  `TaskStatus`/`WaitVerify` (node extracted from the UPID — `localhost` returns "unknown", the
  spike B4 gotcha), `NodeFromUPID`.
- **The verify maintenance loop** (`pbs/verify.go`) — the cheap, key-free, ciphertext-level
  integrity check (§8) on its OWN cadence (default 6h, the 5th daemon goroutine). It is a
  reporting/maintenance task like the slice-5 watchdog: it does NOT go through the reconcile
  gate/journal. Each cycle: trigger verify → poll task → re-list snapshots → record
  per-snapshot `verify_state`. A failed verify is logged loudly.
- **`PBSSnapshot` reporting** — filled the stub (`namespace`/`backup_type`/`backup_id`/
  `backup_time`(RFC3339)/`size_bytes`/`owner`/`protected`/`encrypted` (from `files[].crypt-mode`)
  /`verify_state` (ok|failed|**none** until verified)/`verify_upid`). New `PBSReporter`
  collector seam + an in-memory `SnapshotStore`. Cross-repo golden (both repos, byte-identical)
  + bidirectional key-set tests; hub `handler.go` parses `pbs_snapshots` and logs a **failed
  verify `[WARN]`** (loudest offsite-DR signal).
- **Truthful backup mode** (`backup/runner.go`) — `Backup.mode` now reflects the ACTUAL vzdump
  mode read from the task log (`backup mode: <x>`), since PVE may downgrade snapshot→stop for a
  stopped guest (spike B1); falls back to the requested mode if unparseable.
- **proxmox**: `Storage.Username` (parsed from the pbs storage config — the token id).
- **config** `BackupConfig.{PBSVerifyCadenceSeconds, PBSSecretDir}` (cadence 0→6h, <0 disabled).
- **`--selftest=pbs-verify`** — discover pbs storages → verify each → print the PBSSnapshot
  records (covers the runbook's verify + list). Standalone on the host.

### Notes
- Backup/restore-to-PBS reuse Phase A with no change (the restore-test runs with
  `source_tier="pbs"` when fed a pbs volid). Zero-knowledge holds: verify is ciphertext-level,
  the encryption key is never read here, and the PBS server has no client key (spike B6).
- Daemon runs cleanly with no pbs storage / verify disabled. `go test -race` covers the new
  goroutine. Slice-3/4/5/6A surfaces, goldens, and adversarial tests intact.

## v0.6.0-rc1 — slice 6 Phase A: backup + the self-restore-test (local target) (2026-06-09)

Phase A of the backup/restore slice (doc 03 §8) — the agent's guest-level backup layer and
the **self-restore-test**, which closes "a backup you haven't restored isn't a backup".
Everything here is BENIGN (backup, restore-to-NEW, scratch teardown): reuses the slice-4
classifier/gate/journal — no new destructive class, no new crypto. Local target only; PBS is
Phase B. Restore is to a NEW guest only (no overwrite). Backups are crash-consistent only
(app-consistency needs the controller quiesce, slice 8) — marked so in the report.

### Added
- **proxmox** (`mutate.go`/`query.go`): `DestroyLXC` (DELETE …/lxc/{vmid}?purge=1&destroy-
  unreferenced-disks=1 → UPID; the scratch-teardown primitive); `VzdumpOptions.Notes` →
  `notes-template` (verified on PVE 9.2.2); `LatestBackupVolID` (resolve a produced archive
  from the backup-storage listing — the task status carries no result volid).
- **reconcile self-restore-test** (`restoretest.go`) — `Engine.RunRestoreTest`: pick a free
  scratch VMID (configured band, excludes 9999; full band → skip, never out-of-band) →
  **journal a Scratch-owned entry BEFORE any mutation** → restore-to-new → benign net
  **link-down** SetConfig (so the clone can't conflict with a running source's MAC/IP; this
  is test-safety, NOT slice-7 identity reset) → boot → verify **reaches `running`** → ALWAYS
  teardown (defer; benign `ClassGuestDestroy` + agent-tagged-scratch provenance, gated). Runs
  on the scratch VMID's queue lane. Reuses the journal/gate; result feeds the report.
- **Crash-safe recovery** (`recover.go`): a Scratch journal entry is resolved by TEARDOWN,
  not by re-checking the restore sub-task's UPID — special-cased BEFORE the generic path
  (else the restore task's OK would mark it succeeded while the guest leaks). `Recover` now
  destroys a leaked scratch guest (idempotent: already-gone → clean; list-unreadable → left
  in-flight for a later pass). `JournalEntry.Scratch` flag; `RecoverResult.ScratchClean/
  ScratchDestroyed`. GuestAPI gains `RestoreLXC`/`DestroyLXC`/`GuestStatus`.
- **`internal/backup` package**: `BackupRunner.Backup` (vzdump + archive/size resolve +
  bulk-volume gap — a mountpoint is UNCOVERED unless it carries an explicit `backup=1`, so
  an unset `backup=` is reported uncovered too, the safe DR direction); `PickRestoreCandidate`
  (newest backup); an in-memory `Store` (latest-backup-per-target + latest-restore-test)
  implementing the hub `BackupReporter`/`RestoreTestReporter` seams; a cadence `Scheduler`
  (default 24h; the fourth daemon goroutine; disabled cleanly when off/misconfigured).
- **hub report** (`report.go`): filled the `Backup` + `RestoreTest` stubs (`PBSSnapshot`
  stays a Phase-B stub); collector `BackupReporter`/`RestoreTestReporter` seams. Cross-repo
  golden updated in BOTH repos (byte-identical) + bidirectional key-set tests for
  `backups[0]`/`restore_tests[0]`. Hub `handler.go` parses + persists them (report_json; no
  new columns) and logs a **FAILED restore-test prominently** (the loudest DR signal).
- **config** `BackupConfig` (local target, restore storage, restore-test cadence, scratch
  VMID band 990000–990009 default) + accessors + env overlay + cadence-gated validation.
- **`--selftest=backup -vmid N`** (one-shot backup → print the Backup record) and
  **`--selftest=restore-test [-archive volid]`** (Recover-then restore→boot→verify→teardown,
  print the RestoreTest record). Standalone on the Proxmox host.

### Notes
- The daemon runs cleanly with the cadence off or misconfigured (logs + disables, never
  crashes); a leaked scratch guest from a mid-test crash is reaped by `engine.Recover` on
  restart. `go test -race` covers the new scheduler goroutine.
- Slice-3/4/5 exported surfaces, goldens, and adversarial tests intact. Version bumps to
  **v0.6.0** when Phase B (PBS) lands.

## v0.5.1 — slice 5 live-validation prep: durable_id mis-id fix + re-mount UUID memory (2026-06-09)

Two correctness fixes surfaced while preparing the live USB validation on `demo-felhom`
(a real 1TB USB HDD, sdb1, ext4). Both are DR-load-bearing — exactly the "false-id →
re-attach the wrong disk" failure mode the slice warned about.

### Fixed
- **Unmounted dir-storage no longer inherits the ROOT filesystem's UUID** (`observe.go`).
  Previously, when a removable dir-storage was unmounted, the observer fell through to the
  *containing* mount (root) for the backing device, so its `durable_id` became
  `uuid:<root-uuid>` — a catastrophic DR mis-id (the hub would re-attach the wrong disk).
  Now the backing device/UUID/`durable_id` are derived ONLY from the target's OWN
  mountpoint; an unmounted target reports no device and a stable `store:<name>` durable_id,
  never another filesystem's UUID. (Removed the `containingMountDevice` root-fallthrough.)
- **Watchdog remembers the fs-UUID observed while attached** (`watchdog.go`) so a re-mount
  works even after the known-set cache refreshes mid-drop (an unmounted target can't resolve
  its own UUID). The re-mount key is backfilled from this memory — aligning with doc 03 §7's
  "sourced from the existing definition, no hub manifest needed": the agent learns the UUID
  while the target is attached, then re-mounts by it on return.

### Tests
- Observer: an unmounted dir-storage asserts NO `uuid:` durable_id and no backing device.
- Watchdog: a drop where the cache lost the UUID still re-mounts using the remembered UUID.

## v0.5.0 — slice 5 Phase B: the host-root surface (mounts + SMART + grow + destructive gate) (2026-06-09)

The write surface — the agent's first step outside its Proxmox API token into OS-root.
Isolated behind a narrow, argument-validated, adversarially-tested seam, exactly like the
slice-4 gate. Completes slice 5 (Phase A = read-only observe/report/watchdog at v0.5.0-rc1).

### Added
- **`HostOps` seam + `SudoHostOps`** (`internal/storage/hostops.go`) — the one privileged
  host surface: persistent mounts via **systemd `.mount` units keyed by fs-UUID** (enabled to
  survive reboot), detach (stop+disable), SMART, and thin-pool metadata. Shells out via the
  fenced Runner (`sudo -n`, **fixed arg vectors, no shell**); a fake backs the tests (no real
  root in the suite). `NoopHostOps` is the safe fallback when the surface is unavailable.
- **The argument validator** (`internal/storage/validate.go`) — the security boundary:
  `ValidateUUID` (strict hex), `ValidateMountPath` (absolute, no traversal, no metacharacters),
  `ValidateSMARTDevice` (raw-disk whitelist), `ValidateLVMName`, and an in-process
  `systemdEscapePath` (no `systemd-escape` shell-out). **Every argument is validated BEFORE a
  command is constructed.** Headline test (`validate_test.go`): an adversarial matrix of
  shell metacharacters / `../` traversal / malformed inputs is rejected with **zero exec**.
- **SMART** (`internal/storage/smart.go`) — parses `smartctl -a -j` into `StorageTarget.smart`:
  **SATA** (reallocated/pending/offline-uncorrectable, temp, power-on-hours) **and NVMe**
  (critical_warning, media_errors, percentage_used, temp), degrading to `UNKNOWN` for devices
  with no SMART (USB-SATA bridges). **`lvs`** fills the lvmthin thin-pool **metadata** fill
  (the value Phase A left null). Wired into the Observer's enrichment (Observe only, not the
  watchdog's fast Known path).
- **Watchdog re-mount response** (`internal/storage/watchdog.go`) — on a known mount-backed
  target's device returning **unmounted** (a new `DevicePresent` liveness probe), the watchdog
  **dispatches a benign by-UUID re-mount off the poll path** (a goroutine, never under the
  lock), rate-limited per target to the debounce window. The mount is routed through the gate
  as benign (`gateRemounter` in `main.go`, so `storage` stays decoupled from `reconcile`).
- **Disk-grow executor** (`internal/reconcile`) — `ActionResize` (benign `ClassResize`), planned
  **grow-only** (desired DiskBytes > actual → `pct resize rootfs +<n>M`; a shrink is refused,
  never silently grown) + a defensive executor guard (size must start with `+`). New
  `proxmox.Client.ResizeLXC` (API; `VM.Config.Disk`+`Datastore.AllocateSpace`; async→UPID).
  Built + fixture-tested; **unfed** live (no hub spec until slice 10).
- **Destructive storage ops through the slice-4 gate** (`internal/reconcile/storage_ops.go`) —
  `IntentForStorageMount` (benign) and `IntentForStorageDestructive` (`ClassStorageWipe`/
  `ClassDecommission`). Host/target-scoped: the op binds on the storage **target identity**
  (carried in `target.guest_id`). Reuses the existing verifier/role-scoping/binding/audit — no
  new gate, no new crypto. Storage cases added to the adversarial matrix (`storage_test.go`):
  unsigned wipe → `pending_signature`; "wipe A" signature vs "wipe B" → `binding_mismatch`;
  valid → accepted. **Inert** live.
- **`--selftest=storage` [`-watch <dur>`]** — the live USB-runbook harness: an observe pass
  (full table incl. SMART + thin-pool data+metadata), and a bounded watchdog window with the
  re-mount response live. Runs standalone on the Proxmox host (no hub).
- **`configs/felhom-agent.sudoers`** — the documented narrow allowlist (install unit / systemctl
  manage / smartctl / lvs), with the agent-side fine validation noted.
- **Config**: `privileged.{unit_dir,stage_dir,systemctl,install,smartctl,lvs}` (paths must match
  the sudoers entries).

### Notes
- Daemon still runs cleanly with no removable storage / no signers / no hub manifest, and a
  missing/declined sudoers entry degrades with a warning (SMART→UNKNOWN, mount→logged error),
  not a crash. `go test -race` passes (the watchdog re-mount dispatches off the poll path).
- Slice-3/4 + Phase-A exported surfaces, goldens, and adversarial tests intact. `authz`
  untouched. The destructive-storage executor + grow are built/tested but unfed live until
  slice 10.

## v0.5.0-rc1 — slice 5 Phase A: storage observe + report + watchdog (read-only, live) (2026-06-09)

Phase A of the storage slice (doc 03 §7). Read-only and live: the agent now observes every
host storage target, reports it into the host-report's `storage_targets` (previously an empty
stub), and runs a fast-poll watchdog that pushes a disconnect to the hub in seconds. No
host-root writes this phase (mounts/SMART/grow/destructive-gate are Phase B). The hub-owned
desired manifest (class/role/policy/creds) is not served until slice 10, so reconcile against
it is built-but-unfed — this phase ships only the genuinely-useful read-only footprint.

### Added
- **`internal/storage` package** (new):
  - **`StorageTarget` wire contract** (`internal/hub/report.go`) — filled the slice-3 stub:
    `name`/`type`/`durable_id`/`state`/`reachable`, usage (`total`/`used`/`avail`/
    `used_fraction`), `content`, `mount_path`/`backing_device`, `class_hint` (rotational HINT
    — never authoritative; class is hub-owned), `role` (empty until slice 10), a `thin_pool`
    sub-object (lvmthin data fill; metadata fill is Phase B/`lvs`), and a `smart` sub-object
    (`UNKNOWN` until Phase B). Cross-repo golden kept byte-identical with `felhom.eu/hub` and
    guarded by the bidirectional key-set test (`contract_test.go`).
  - **`durable_id` derivation** (`durableid.go`) — deterministic per type (the DR-load-bearing
    re-attach key): fs-UUID (usb/local-dir), `server:export` (nfs/cifs), `repo+fingerprint`
    (pbs), `vg/pool` (lvmthin); never empty (falls back to a stable store id).
  - **`HostReader` seam + `ProcHostReader`** (`hostread.go`) — non-privileged `/proc/mounts`,
    `/dev/disk/by-uuid`, `/sys/.../rotational` + `removable` reads. Root-free by construction.
  - **`Observer`** (`observe.go`) — builds `[]hub.StorageTarget` from `ListStorage`/`NodeStorage`
    joined with host reads; surfaces the lvmthin thin-pool data fill prominently (warns ≥85%).
  - **Storage watchdog** (`watchdog.go`) — a third daemon goroutine fast-polling the *known*
    target set (a defined Proxmox storage and/or a previously-seen one) for
    `attached↔disconnected` transitions; on a transition it triggers an immediate, **debounced**
    out-of-band host-report. Only flags a *known* target's change (never a never-attached
    device); coalesces flaps within the debounce window (leading + trailing edge).
    `CachingKnownTargets` rate-limits the Proxmox-derived known set; `HostLiveness` probes
    device/mount presence (local) + a reachability dial (network), all non-privileged.
- **Proxmox `Storage` type** (`internal/proxmox/types.go`) — additive parse-only config fields
  (`server`/`export`/`share`/`datastore`/`fingerprint`/`vgname`/`thinpool`) feeding durable_id.
- **Collector `StorageObserver` seam** (`internal/hub/collect.go`) — populates `storage_targets`
  via the observer; a nil observer or an observe error degrades to empty (never sinks the
  heartbeat). Hub does not import storage (storage imports hub for the wire type).
- **Out-of-band report trigger** (`internal/hub/loop.go`) — `Loop.SetTrigger`: a watchdog
  signal runs one extra collect→report immediately without disturbing the regular cadence.
- **`StorageConfig`** (`internal/config`) — watchdog interval / debounce / known-refresh knobs
  (all optional; package defaults otherwise).
- **Hub ingest** (`felhom.eu/hub`) — `hostReportPayload` now parses `storage_targets`
  (full mirror struct), persists them via `report_json`, counts + warns on disconnected
  targets, and has its own half of the bidirectional golden key-set test.

### Notes
- The daemon still runs cleanly with no removable storage, no signers, and no hub manifest —
  the watchdog finds nothing to flag; storage reporting is best-effort.
- `proxmox`/`hub`/`authz`/`reconcile` exported surfaces + their golden/adversarial tests are
  intact. No host-root writes, no destructive paths, no SMART this phase (all Phase B).
- Version: **v0.5.0-rc1** at the Phase-A checkpoint; **v0.5.0** when Phase B lands.

## v0.4.0 — slice 4 Phase B: reversibility gate + signed-op consuming layer (2026-06-08)

The security core of slice 4: hub-supplied intent stops being trusted for destructive
change. Layered in front of the per-guest queue's executor — **every** mutation now
passes the gate. Reuses `internal/authz` for all crypto (untouched surface). Inert
this slice: no destructive deltas are served until slice 10, so the destructive path is
classified, gated, and adversarially tested but not wired to live execution.

### Added
- **Classifier (`classify.go`, doc 03 §4)** — benign vs destructive by **provenance +
  data-bearing-ness, NOT by verb**. The `OpClass` vocabulary (seeded by the committed
  slice-2 `op_blob.json`: `guest_destroy`) is the agent-side contract slice 10 matches.
  Destroy/overwrite of customer data is destructive UNLESS **agent-internal**
  provenance (same-journaled-transaction create → compensating rollback, or
  agent-tagged scratch) makes it benign. `Provenance` is journal-recorded and **never
  populated from the hub** (its zero value is the only thing an external intent may
  carry). Unknown op class fails safe → destructive.
- **Reversibility gate (`gate.go`)** — `Gate.Authorize(intent, signed)`: benign →
  allowed unsigned; destructive → requires a verified, role-authorized, action-bound
  operator signature, else refused **`pending_signature`**, never executed. Every
  decision is written to an `AuditSink` (audit is a signal, never the guard).
- **Signed-op consuming layer over `authz`** — verifies via `authz.Verifier.Verify`
  (the locked pipeline, untouched), then enforces on the `VerifiedOp`:
  - **Role-scoping (doc 04 §4)** — recovery key authorizes key-rotation re-pins ONLY;
    operational key authorizes ordinary destructive ops + planned rotation.
  - **Op-to-action binding** — verified `op` + host + guest + `params` must match the
    gated action (a signature for guest X / op A can't authorize guest Y / op B);
    params compared semantically (key-order/whitespace independent).
- **Signed-job orchestration (`job.go`)** — `RunSignedJob`: idempotency dedupe (the
  op nonce as the journal key — a redelivered completed op is skipped, not re-run),
  gate authorization, then journal-wrapped execution via an injected
  `DestructiveExecutor` (nil this slice — authorized destructive ops are inert, no
  executor wired until 6/7).
- **Crash-recovery consumer (`recover.go`, Note 1 / doc 03 §10)** — `Engine.Recover`
  consumes the journal's `InFlight()` at startup: an op that crashed AFTER the Proxmox
  POST and BEFORE its terminal record (`OpTaskRunning`, nonce already consumed) is NOT
  covered by idempotency dedupe — only this resume-or-rollback resolves it (re-read the
  task via the new `TaskStatusOnce`, record the real outcome; a no-task-id op is
  abandoned fail-safe). Landed together with the signed-op executor, as Note 1 required.
- **Daemon wiring** — `runDaemon` builds the verifier from `config.Authz.Signers` (a
  bad key / missing nonce-store path is a fatal misconfig; **no signers = nil verifier**,
  the common slice-4 state), constructs the gate (+ `SlogAudit`), runs `Recover` before
  issuing any mutation, and routes every reconcile action through the gate.

### Changed
- **Memory comparison canonicalized (Note 2)** — `desiredMemoryMiB` makes the
  desired↔actual memory compare in the same MiB unit that is then written, so a
  non-MiB-aligned `MemoryBytes` converges in one pass instead of re-issuing SetConfig
  forever (the numeric cousin of the description-newline normalization). Test proves
  convergence. Slice 10 should still serve MiB-aligned specs at the source.

### Tests (the security proof — each independently rejected)
- **Adversarial matrix** via the REAL `authz.Verifier` with in-test-minted SSHSIGs
  (framing replicated in reconcile's test binary; production authz untouched, no signing
  added to the verify-only package): unsigned destructive **job** → pending_signature;
  unsigned destructive **desired-state delta** → pending_signature (distrusts hub
  desired state, not just jobs); forged/unknown signer → `ErrUnknownSigner`; expired →
  `ErrExpired`; **replayed nonce across an agent restart** (durable `FileNonceStore`) →
  `ErrReplay`; wrong host → `ErrTarget`; wrong guest / wrong op / wrong params →
  binding_mismatch; **recovery key on ordinary destructive** → role_denied;
  **hub-supplied "scratch" tag ignored** → still destructive → refused; **valid + role +
  target + fresh nonce → accepted**, and a second presentation → `ErrReplay` (nonce
  consumed).
- Classifier (benign/destructive/provenance/key-rotation/fail-safe), role-scoping,
  params binding, crash-recovery (resume OK / fail / still-running / no-task rollback /
  unreadable / one-shot key applied on resume), signed-job idempotency (execute once,
  dedupe redelivery, refused-not-executed, no-executor-inert, executor-error).
- Full module **race-clean** (`go test -race`) + vet clean on the Linux build server.

## v0.4.0-rc1 — slice 4 Phase A: reconcile engine (structural; runs live, unfed) (2026-06-08)

The agent-side control core's structural half. **Checkpoint marker** — `-rc1` is the
Phase-A push; awaiting validation before Phase B (the reversibility gate + signed-op
consuming layer) lands the final **v0.4.0**. Runs LIVE but UNFED: with no desired-state
provider until slice 10, the live engine computes an empty action set and performs
**zero mutations**.

### Added
- **`internal/reconcile`** package — the engine, the per-guest serializer, the
  desired-state model, the normalization layer, and the durable op journal:
  - **Per-guest serializer (`Queue`, doc 03 §10)** — the single choke point ALL
    mutation sources funnel through. Same-vmid jobs run strictly one-at-a-time in
    submit order; independent vmids run in parallel. Each vmid is a cond-var FIFO lane
    (unbounded, non-blocking, order-preserving); graceful drain on `Close`.
  - **Desired-state model + `DesiredProvider` seam** — `DesiredGuest` (per-field
    optional: run-state / `*hub.GuestSpec` / `*description`), `DesiredState`. The only
    live provider is **`EmptyProvider`** (slice 4 has no source); `StaticProvider`
    feeds fixtures. The seam is where slice 10's hub-serving plugs in — no hub/local
    source invented here.
  - **Normalization layer (`FieldNormalizers`)** — reconcile compares *normalized*
    desired-vs-actual so Proxmox round-trip quirks don't read as drift. `description`'s
    trailing newline is the first registered case; the registry takes more (boolean
    coercion, list ordering) as discovered. `normDesc` **promoted** out of
    `cmd/felhom-agent/main.go` to **`reconcile.NormDescription`**; the `--selftest=task`
    description round-trip now uses that shared helper (one source of truth for the quirk).
  - **Plan engine (`Plan`, pure function)** — computes the minimal **benign** action set
    (`Start`/`Stop`/`SetConfig`) for guests present in both desired and actual, with
    normalized comparison, deterministic vmid ordering, config-before-run-state. Skips
    provision (desired-absent-in-actual, slice 7) and destroy (actual-absent-in-desired,
    gated, slice 10); never writes a config it couldn't first read (`SpecKnown`). Disk
    (rootfs grow) intentionally not reconciled here.
  - **Reconcile engine (`Engine`)** — reads desired+actual, plans, dispatches each action
    onto the shared queue. Every Proxmox op handled per the mutate.go contract: non-empty
    UPID → `WaitTask` + assert `exitstatus`; empty UPID → clean **synchronous** success
    (slice-4 proven). Per-action failures are counted, not fatal (other guests still
    converge).
  - **Operation journal (`Journal`)** — durable fsync'd append-only JSONL mirroring
    `authz.FileNonceStore`: records each op's lifecycle (started → task_running →
    succeeded/failed) with its Proxmox task id (crash mid-op is detected and re-checkable
    on restart via `InFlight()`), plus an **idempotency-key store** (`AlreadyApplied`) so
    a one-shot op never re-runs across retries/restarts. Reconcile actions carry no
    idempotency key (convergent — must re-run on real drift).
- **Daemon wiring (`runDaemon`)** — reconcile runs alongside the hub loop on the poll
  cadence, **sharing the per-guest queue**. Journal path is a `journal.log` sibling of the
  nonce store. The daemon runs cleanly with **no desired state and no signers** (reconcile
  is a logged live no-op; a journal-open failure degrades to journal-less, never crashes).

### Tests
- Serializer: same-guest serialized (max-concurrency 1, submit order preserved) and
  different-guests parallel (cross-waiting jobs both complete — would deadlock if not);
  error propagation; drain-pending-on-close; submit-after-close.
- Normalization: description round-trip; unknown-field identity; extensibility seam
  (synthetic boolean-coercion + list-ordering normalizers).
- Plan: run-state start/stop, spec drift (cores/memory), disk-not-reconciled,
  description-newline-not-drift, unmanaged fields, spec-unknown skips config keeps
  run-state, desired-absent skipped, combined ordering, empty-desired no-op, deterministic
  vmid order.
- Engine: empty-provider zero mutations; async start (WaitTask); synchronous SetConfig
  (no WaitTask); WaitTask failure + POST error counted failed; list error = pass failure.
- Journal: lifecycle latest-wins; in-flight survives restart; idempotency dedupe across
  restart; failed key not applied; torn-trailing-line skipped.
- Full module **race-clean** (`go test -race`) on the Linux build server; vet clean.

### Not in this phase (Phase B)
- The benign/destructive classifier, the reversibility gate, and the signed-op consuming
  layer over `internal/authz` (doc 03 §4 / doc 04) — added next, in front of the queue's
  executor, landing **v0.4.0**.

## v0.3.2 — SetConfig selftest extension (slice-4 pre-check) (2026-06-08)

The gate before slice 4: prove `SetConfig` works live under the scoped token before
reconcile is built on it. **Self-gated live run PASSED** on `demo-felhom`/guest 9999.

### Added
- **Reversible `SetConfig` step appended to `--selftest=task`** (`cmd/felhom-agent/main.go`,
  `selftestSetConfig`): read `GuestConfig` → write a `description` marker
  (`felhom-selftest <RFC3339>`) → verify it landed → restore the original value (or
  `delete` the key if it was absent) → verify the restore. Handles PVE's dual-mode
  `SetConfig` return per the `mutate.go` contract: empty UPID = synchronous success
  (printed `synchronous`); non-empty UPID = `WaitTask` + assert `exitstatus=OK`.
  The existing snapshot → rollback → delete-snapshot steps are unchanged. First live
  exercise of the **`VM.Config.*`** privilege cluster.
- **`normDesc` / `extraString` helpers** — `extraString` decodes a string-valued key
  from `GuestConfig.Extra` (raw JSON); `normDesc` strips the trailing newline PVE
  appends to `description` on read, so a written value round-trips equal.

### Finding (live)
- The LXC `description` write returned **synchronous (empty UPID)** — PVE applied it
  inline, no task. The agent's dual-mode `SetConfig` modeling is correct: the
  empty-string path is real and must not be treated as an error.
- PVE **appends a trailing `\n` to `description`** on read (stored URL-encoded as
  `%0A`). A naive exact-match reconcile would see perpetual drift — slice-4 reconcile
  must normalize `description` comparisons (hence `normDesc`).

### Ops
- Standing operator token (`felhom-agent@pve!agent`, privsep) **rotated** during this
  run (the prior secret was not retrievable); role + both user/token ACL rows
  re-confirmed at `/`. New secret stored out-of-band, **not persisted to the repo**.
  Guest 9999 left pristine (stopped, no `description`, no leftover snapshot). Version → 0.3.2.

## Docs + live validation — no version bump (2026-06-08)

### Changed
- **Reflowed `CLAUDE.md`** — removed hard mid-paragraph line wraps (prose, list items, blockquotes now single-line, soft-wrapped); code blocks and tables untouched; rendered output unchanged.
- **Unified the REPORT/CHANGELOG convention** in `CLAUDE.md`: `CHANGELOG.md` is the cumulative log (newest on top); `REPORT.md` is overwritten with the most-recent implementation/validation only. Added an explicit **no-secrets** rule (never write tokens/passwords/keys into committed files; reference them as stored out-of-band).

### Added
- **`REPORT.md`** rewritten for the live `--selftest=task` validation on the demo host (`demo-felhom`): snapshot → rollback → delete-snapshot on guest 9999, each polled to `exitstatus=OK` under the `felhom-agent@pve!agent` privsep token (UPIDs name the token actor — privsep path genuinely exercised); 16-privilege `FelhomAgent` role + both user & token ACLs confirmed; `--selftest=read` clean. Closes the slice-1 "mutating ops unit-tested only" gap; `WaitTask` async foundation validated live → **slice 4 unblocked**. (Token secret stored out-of-band, not in the repo.)

## v0.3.1 — slice-3 validation follow-ups (2026-06-08)

### Changed
- **Collector keeps the known run-status on a `GuestConfig` failure** (`internal/hub/collect.go`):
  previously a per-guest config-read error forced `status="unknown"`; now the run-status from
  `ListLXC` is preserved (only the `spec` is dropped). An empty status is still normalized to
  `unknown` (wire value is always `running|stopped|unknown`). Test renamed to
  `TestCollect_GuestConfigFailureKeepsStatusOmitsSpec` and asserts the preserved `running` + nil spec.
- **`--selftest` usage** error string now reads `(want read|task|hub)`.

### Added
- **Cross-repo contract fixture** `internal/hub/testdata/host-report.golden.json` +
  `TestHostReport_ContractMatchesGolden` — compares the marshaled `HostReport` field-name sets
  (top level + `host` + `guests[0]`) against the golden, failing on any json-tag drift. The file is
  **kept byte-identical** with felhom-hub's copy (duplicated contract until a shared types module;
  revisit when slices 5/6 populate the empty collections). Version → 0.3.1.

## v0.3.0 — hub client + host-report + first daemon loop (slice 3) (2026-06-08)

The agent's first daemon: a periodic read-only host-report POSTed to the hub (the
heartbeat). No Proxmox mutations, no desired-state/signed-op consumption, no
storage/backup collection yet — those are slices 4/5/6.

### Added
- **`internal/hub`** package:
  - **`HostReport`** wire contract (`report.go`) shared field-for-field with the hub
    ingest: host metrics, guests (`vmid` + spec), `cloudflared` status, and the
    `storage_targets`/`backups`/`restore_tests`/`pbs_snapshots`/`audit_tail`
    collections **defined but emitted empty** (typed `[]`, slices 5/6 fill them).
  - **`Collector`** (`collect.go`) builds the report from a read-only `proxmoxReader`
    (adapted to the real `internal/proxmox` surface — node held by the client, value
    returns, `proxmox.Guest`) + a `CloudflaredProber`. Partial-failure policy: a
    failed `NodeStatus` is a hard error (skip the POST); a failed per-guest
    `GuestConfig` degrades that guest to `status="unknown"` (spec omitted) but still
    sends; a cloudflared probe failure → `"unknown"`, never fatal.
  - **`CloudflaredProber`** + `SystemctlProber` (`systemctl is-active cloudflared`;
    read-only — NOT a Privileged/root op; tunnel management is a later slice).
  - **`Client`** (`client.go`): `POST /api/v1/host-report` with
    `Authorization: Bearer <key>`, standard TLS (system roots or optional `ca_file`;
    verification always on). Typed `*TransportError` / `*HTTPError`; the bearer token
    never appears in any error.
  - **`Loop`** (`loop.go`): the daemon — immediate first report then tick; adopts the
    hub's `poll_interval_seconds` clamped to [60,3600]; resilient (a collect/report
    error is logged and the loop continues); clean shutdown on context cancel.
  - **`ControlEnvelope`**: only `poll_interval_seconds` is acted on; `blocked` /
    `desired_generation` / `has_signed_ops` are parsed-but-ignored (logged at most)
    pending reconcile (slice 4).
- **Config**: `HubConfig` (url/host_id/api_key/poll_seconds/timeout_seconds/ca_file),
  `FELHOM_AGENT_HUB_*` env overlay, `HubConfig.Validate()` (mode-aware — proxmox-only
  `--selftest=read|task` still runs without hub config), `WithDefaults()`, and
  `Redacted()` now also blanks the hub key. `configs/agent.example.json` gains `hub`
  (and `authz`) blocks.
- **`cmd/felhom-agent`**: the no-`--selftest` mode is now the **daemon** (poll loop);
  added **`--selftest=hub`** (one collect+report, prints the report + envelope).
  Version 0.2.0 → 0.3.0.

### Tests
- Report serialization (field names; empty collections are `[]` not `null`; spec
  omitted when unknown); client (Bearer header, non-2xx→`*HTTPError`,
  transport→`*TransportError`, **token never in error**); collector (host mapping,
  guest spec, per-guest failure degrades-but-still-reports, NodeStatus hard error,
  cloudflared error→unknown); loop (immediate first report, continuation after an
  injected error, interval adoption + clamp); config (hub validate/redact/env).

### Notes
- `internal/proxmox` and `internal/authz` were **not touched** — no new proxmox
  surface was needed (`ListLXC` already exposes status/maxmem/maxdisk; `GuestConfig`
  exposes cores). The task's `proxmoxReader` sketch (node-arg/pointer/`LXC`) was
  adapted to the real exports as instructed.
- **Defined-but-empty** this slice: `storage_targets`, `backups`, `restore_tests`,
  `pbs_snapshots`, `audit_tail` (slices 5/6). **Parsed-but-ignored**: the envelope's
  `blocked`/`desired_generation`/`has_signed_ops` (slice 4).

## v0.2.0 — `authz` signed-op verifier (slice 2) (2026-06-08)

Production form of the Phase-4 signing primitive: a key-type-agnostic SSHSIG
verifier for operator-signed destructive ops, with the full anti-replay/
authorization pipeline and a durable, crash-safe nonce store. What slice 4
(reconcile) will call to gate destructive desired-state deltas. No hub, no signing
CLI, no reconcile loop.

### Added
- **`internal/authz` — `Verifier`**: `New(signers, store, hostID)` + `Verify(blob,
  sigArmored) (*VerifiedOp, error)`. Runs the LOCKED pipeline (order is
  load-bearing): parse armor → namespace → parse pubkey → allow-list (by key
  **material**, `pub.Marshal()` equality, not key_id) → crypto verify (over the
  **raw received bytes**, never re-canonicalized) → parse blob → target → time
  window → **nonce recorded LAST**. Each post-crypto stage rejects even with a
  valid signature.
- **SSHSIG framing** (`sshsig.go`) via `golang.org/x/crypto/ssh` — `pem.Decode` →
  strip 6-byte magic → `ssh.Unmarshal` → `ssh.ParsePublicKey` → recompute signed
  data with the named hash → `pub.Verify` (dispatches on key algorithm). No
  hand-rolled crypto. Key-type-agnostic: ed25519 / **sk-ssh-ed25519 (FIDO2)** /
  rsa / ecdsa via the one path.
- **Fixed namespace** `felhom-op-v1` (package constant, never caller-supplied).
- **`OpBlob`** (corrected `host_id`/`guest_id` json tags) + **`VerifiedOp`** (op,
  host/guest, params, key_id, matched signer). key_id is advisory/audit only —
  never an authz input.
- **Typed errors**: `ErrMalformed, ErrNamespace, ErrUnknownSigner, ErrBadSignature,
  ErrTarget, ErrExpired, ErrNotYetValid, ErrReplay` (errors.Is-friendly).
- **`NonceStore`** + two impls: `MemoryNonceStore` (tests) and **`FileNonceStore`**
  — durable, crash-safe (fsync'd append log, replayed into an index on open,
  periodic compaction, expiry-only pruning). A nonce is fsync'd to disk before
  `SeenOrRecord` returns false; replay protection survives restart; I/O failure
  fails safe (reports seen=true). Target generalization: host_id matched strictly,
  guest_id surfaced for the caller to route.
- **Config**: `AuthzConfig` (nonce-store path + pinned operator `signers` tagged
  `operational`/`recovery` with a key_id, as authorized_keys lines).
- **Version 0.2.0.**

### Tests
- Real OpenSSH interop via a committed `ssh-keygen -Y sign` vector (hermetic CI);
  per-stage rejection (each with an otherwise-valid sig); the headline
  **invalid-sig-does-not-burn-the-nonce** invariant; replay; **persistence across
  restart**; synthetic **sk-ssh-ed25519** through the unchanged path; byte-exactness
  (a re-serialized blob fails crypto — not re-canonicalized).

### Notes / corrections to the Phase-4 reference
- §7's `Target` lacked json tags (`host_id`/`guest_id`) — fixed.
- The doc paired "Go 1.24.4 / x/crypto v0.52.0", but v0.52.0 declares `go 1.25.0`
  and does **not** build on Go 1.24. Resolved by upgrading the build server to
  go1.26.0 (backward-compatible; felhom-controller/hub unaffected); the module is
  `go 1.25.0` on x/crypto v0.52.0.
- Free function → constructed `Verifier`; returns the full `VerifiedOp`; typed
  errors; clock-skew tolerance added; durable nonce store is the net-new work.
- **Shared-contract dependency flagged** (not built): the hub and the `felhom-sign`
  CLI must emit byte-identical canonical JSON or signatures won't verify; a shared
  canonicalizer both import would be the right home.

## v0.1.0 — Scaffold + `proxmox` interaction layer (slice 1) (2026-06-08)

First slice: stand up the host-agent project and its foundation — the typed
Proxmox interaction layer every other module will call. No reconcile loop, hub
client, signing, or storage/backup orchestration yet (later slices).

### Added
- **Project scaffold**: module `gitea.dooplex.hu/admin/felhom-agent`, binary
  `felhom-agent` (`cmd/felhom-agent/`), Go 1.24, zero external dependencies
  (pure stdlib). `--version` flag; `version` var overridable via
  `-ldflags "-X main.version=<v>"`.
- **`internal/proxmox` — API backend (`Client`)**: hand-rolled REST client over
  `https://<host>:8006/api2/json` with `PVEAPIToken` auth. Typed read ops
  (`Version`, `Nodes`, `NodeStatus`, `ListLXC`, `GuestStatus`, `GuestConfig`,
  `ListStorage`, `NodeStorage`, `StorageContent`) and async mutating ops
  returning a UPID (`RestoreLXC` — the primary create path, `Vzdump`, `Snapshot`,
  `Rollback`, `DeleteSnapshot`, `SetConfig`, `Start`, `Stop`).
- **`WaitTask`**: polls `GET /nodes/{node}/tasks/{upid}/status` until stopped, then
  asserts `exitstatus == "OK"` (authorization can surface at task execution, not
  the POST — phase1-2 §1.3). Exponential backoff (1s→5s cap), context
  cancellation + timeout. `*APIError` parses the offending privilege from a 403;
  `*TaskError` parses it from a failed task exitstatus + log tail.
- **`internal/proxmox` — fenced root-CLI backend (`Privileged`)**: limited to the
  three proven OS-root exceptions only — `CreateGoldenLXC` (keyctl `pct create`),
  `MountUSBByUUID`, `SMART`, `Sensors`; each cites why it can't be the API. Fence
  is structural (Client never shells out, Privileged never makes an HTTP call) and
  asserted in tests.
- **TLS trust**: SHA-256 leaf-cert pinning (the host serves a self-signed cert) or
  a CA file; an explicitly-named `insecure_skip_verify` that is off by default. No
  blanket verification disable.
- **`internal/config`**: JSON config file + `FELHOM_AGENT_*` env overrides; the
  token secret is never logged (`Redacted()`).
- **`internal/log`**: slog setup (text, stderr, configurable level).
- **`cmd/felhom-agent --selftest`**: read-only health report against a live host
  (version/nodes/status/guests/storage); `--selftest=task --vmid N` exercises
  `WaitTask` on a reversible snapshot→rollback→delete op (gated; default selftest
  mutates nothing).
- **Tests**: unit tests with a mock HTTP transport + mock runner (UPID parse,
  `WaitTask` running→OK / failed-403 / timeout / ctx-cancel, 403→privilege error,
  response decoding against shapes captured live from `demo-felhom`, config
  redaction, and the API-vs-root routing fence).

### Notes
- Types are grounded in the spike findings
  (`felhom.eu/documentation/proxmox-platform.md`, `tests/phase{0,1-2,3}-findings.md`)
  and the exact JSON shapes captured live from `demo-felhom` (PVE 9.2.2).
- Verified: `go build/vet/test` green on Go 1.24.4 (build server) and a live
  read-only `--selftest` against the demo host with TLS fingerprint pinning.
- The 16-privilege `FelhomAgent` role + privsep token (role on **both** user and
  token) is provisioned out-of-band; the agent only consumes the token.

<!-- R-421 sweep: this repo cites R-421; the row landed in felhom.eu 2d88776. -->