docs: REPORT for v0.81.0 (deployed; task-wide record in felhom-controller/REPORT.md)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
This commit is contained in:
2026-07-11 10:32:19 +02:00
parent added9d226
commit 1e60e88eb2
+49 -54
View File
@@ -1,63 +1,58 @@
# REPORT — agent v0.80.0: PBS DR tier SLICE 2, the apply-bridge — DEPLOYED + ADOPTION-PROVEN + PUBLISHED
# REPORT — agent v0.81.0: NAS verify-before-commit (retry=0 + classifier + detached verify job) — DEPLOYED
**Date:** 2026-07-10/11 · **Class:** implementation (agent) + demo live check + publish. Baseline
v0.79.0 → **v0.80.0**, live on felhom-pve (`felhom-agent 0.80.0`, capabilities **59/59, degraded=0**),
**PUBLISHED** to Gitea (sha `f2ba62ca6aca6e24a8d08706ea0dc3ae43940a63e9bdf57d4606ad1756cf06d2`;
anon round-trip verified; the deployed demo binary is byte-identical).
Consumes hub v0.44.0 (slice 1). **PENDING: the supervised Peti phase**
`felhom.eu/documentation/pilot/RUNBOOK-peti-pbsdr-2026-07-11.md` (Viktor + Peti; his box has no
tunnel/OOB yet, so box prep is his hands).
**Date:** 2026-07-11 · **Class:** implementation (agent half of the cross-repo "NAS
verify-before-commit + page redesign" task). Baseline v0.80.0 (`300f06722b`) → **v0.81.0**
(`added9d`), live on felhom-pve. Pairs with controller v0.113.0 + host-install v1.13.0.
**Full task-wide record (red-proof outcomes, live AE transcripts, teardown):**
`felhom-controller/REPORT.md`. Evidence base: SPIKE-nas-verify-2026-07-11 (b57f6c1).
## Part 0 — grant diagnosis (the carried spike §6 flag), evidence recorded
## What shipped (agent)
- Status reads (`/nodes/.../storage/.../status`) pass with NO storage grant — `FelhomAgentBase`
(`Datastore.Audit` at `/`, propagate) covers them. That + the demo's `local` backup target is
why the purged grants never showed.
- The WRITE path is the gap: token vzdump → **403 `(/storage/felhom-pbs, Datastore.AllocateSpace)`**.
After the §4b dual-grant re-assert (both entries, user+token): token vzdump of 9001 → `stopped OK`
(9001 also joined `/pool/felhom` — the VM.Backup half; pool-default conformance).
- The bridge's `grant` op therefore codifies EXACTLY the dual-grant, nothing more.
- **`retry=0` in the production NFS option string** (`netmount.go mountOptions`, Q4-vi): a dead-NAS
on-demand access fails clean in ~3.8 s (ENODEV) instead of a 91 s wedge; verify failures classify
as `No route to host` instead of a diagnostic-free systemd timeout. SMB unchanged (mount.cifs
rejects `retry=`). Installed units are NOT rewritten (none existed on the demo anyway).
- **`internal/storage/netverify.go`** — `ClassifyNetVerifyFailure(journalTail, tcpReachable)`:
pure, first-match table on the Q4 VERBATIM substrings → `unreachable | nfs_export | smb_auth |
smb_share | timeout | mount_failed`; `nfs_export` deliberately merges not-found/not-permitted
(NFSv4 identical strings). String-based BY DESIGN — every mount failure exits rc=32.
- **Detached verify job** (`localapi/netverifyjob.go`, formatjob shape but IN-MEMORY single slot —
a restart empties the slot; `GET /netstorage/verify-status` phase `none` is the CONTROLLER's
rollback signal, Scenario F): add = decode → role-gate → **sync fast-fail** (full spec validation
+ 2 s TCP pre-probe; unreachable = HTTP 502 `{code:unreachable}`, NOTHING installed) → SMB creds
(0600) → EnsureNetworkMount → detached verify: trigger read through the automount, success judged
from **/proc/mounts ONLY** (§8: EACCES on a 0700 export is a GOOD mount), failure = journal
classification + **auto-rollback** (RemoveNetworkMount + creds). Single-flight: second add = 409.
- **Unprivileged journal read** (`journalctl -u <unit> -n 20 -o cat`, NO sudo, NO new sudoers
grant): needs the `systemd-journal` group (host-install v1.13.0; existing hosts:
`usermod -aG systemd-journal felhom-agent`). Unavailable ⇒ degrade to `mount_failed` + hint,
still rolled back.
- New exports: `storage.NetworkMountedAt`, `storage.NetworkEndpointReachable`. REUSE.md updated
(helpers + the in-memory-vs-persisted job pattern row + the four Server net-verify seams).
## What shipped (laws + red-proofs)
## Tests + red-proofs (agent side; outcomes verbatim in felhom-controller/REPORT.md)
- `internal/pbsdr` bridge (adoption-first / verify-pin-before-consume / set-only / loud
consumed-failed / escrow seed / descriptor-hash marker) + `configs/felhom-pbs-apply` +
`FELHOM_PBSDR` sudoers + 3 capability entries + `WirePBSDR`/`PBSDRStatus` wire/report +
`hub.ConsumePBSToken` (PLURAL `/hosts/` route). Details: CHANGELOG v0.80.0.
- **Red-proofs (all run, mutations reverted, gate green):**
- set-only: a remove+re-add path in the re-apply → `TestSetOnlyLaw` FAILS (recorded verbs
`[remove, create, grant]` ≠ set-only) — the K-destruction guard bites.
- stdin secret: secret passed as argv → `TestFreshPath_SecretOnStdinNeverArgv` FAILS with the
secret visible in Args.
- verify-before-consume: consume reordered before the probe → `TestVerifyPinBeforeConsume`
FAILS (`consume calls = 1, want 0`).
A1 retry=0 (revert → fail) · A2 classifier table incl. merged nfs_export + empty-journal
degradation (exit-code mutant → every non-generic row fails) · A3 rollback effects via fakes
(drop rollback → `removed=[]` + creds linger) · A4 §8 truth table (readability mutant → both rows
fail) · A5 pre-probe zero-install (skip probe → 200-with-install) · A6 single-flight + no-job shape
(drop check → 409 lost). Green gate `go build && go vet && go test ./...` PASS (known
TestGenerateRecoveryCode wordlist flake hit once, clean on re-run).
## Demo live check — the ADOPTION path (2026-07-10 22:00 CEST)
## Deployed + verified (felhom-pve)
Enabled "PBS DR tier" on demo-felhom with storage-id `felhom-offsite` (hub v0.44.0 UI, logged-in
browser). The full slice-1+2 chain fired live:
- hub: `tenantsync: provision ok (ns=demo-felhom)` on ep0 → secret staged consume-once → descriptor
→ generation 10.
- agent: fetched gen 10 → **ADOPTED** the manually-built `felhom-offsite` entry — journal:
`adopted entry keeps its own tenancy (namespace "demo-felhom-01"; descriptor says "demo-felhom"
— entry wins, never repointed)``converged state=adopted`.
- **Asserts:** staged secret UNCONSUMED (`host_pbs_secrets.consumed_at = NULL` in the hub DB — the
no-consume law held); `pvesm status felhom-offsite` active (demo DR green);
`escrow.pbs_storage_id=felhom-offsite` seeded into agent.json.
- **Live finding, fixed same session (3d1323c):** the escrow seed's tmp+rename failed —
`/etc/felhom-agent` is root-owned while agent.json is agent-owned, so rename (a directory
mutation) is impossible for the non-root agent. Rewritten as park-a-recovery-copy-in-state-dir +
in-place O_TRUNC write; re-proven live (marker removed → re-adopt → seed OK). The warn-and-
converge fail-safe behaved as designed in between.
- Benign note: the report `pbs_dr` stanza appears from the SECOND heartbeat after a restart (the
startup collect races the first bridge Apply).
`.bak-0.80.0` kept → `install -m0755``usermod -aG systemd-journal felhom-agent`
`systemctl restart felhom-agent`. Verified: `felhom-agent 0.81.0`; `id felhom-agent`
`…,999(systemd-journal)`; clean start journal (enrolled drive re-bound, local-api listening,
desired-state gen 10). Live proof the journal path works: Scenario A's failure detail carried the
REAL `mount.nfs4 … reason given by server: No such file or directory` lines read unprivileged.
## Publish train + pending
**NOT published:** 0.81.0 is deployed on the demo only — no Gitea publish, Day-0 manifest untouched
(the publish train is the operator's separate decision); Peti's box untouched.
- 0.80.0 published (`publish-agent.sh` on 180): pre-delete 404 → PUT 201 → round-trip sha match.
Demo's live binary sha == published sha (the GL never-vouch-unpublished rule holds).
- **PENDING (the committed runbook, Viktor+Peti):** box prep (sudoers+wrapper+wg_tunnel enable —
his box has NO WG peer/OOB today), opsign signed update to 0.80.0, hub-UI tier enable (the FRESH
path: verify → consume → **K born** → degradation clears), then the ceremony one-liner (fresh R,
Peti keeps) → auto-confirm → first offsite run. Operator follow-up: Day-0 manifest vouch of
0.80.0 when it becomes the floor target.
## Observations
- Pre-existing (2026-07-08, unrelated): `lanresolver: cannot list provisioned guests: permission
denied` — `/var/lib/felhom-agent/guests` is root-0700 under the non-root agent.
- `RemoveNetworkMount` leaves the empty mountpoint dir + a `not-found failed` residual unit listing
(until `reset-failed`/reboot) — cosmetic, documented in the task report.