felhom-agent/CONTEXT.md

# CONTEXT — felhom-agent working state

> Snapshot of the current state + open threads. Authoritative history lives in `CHANGELOG.md` (top
> entry = current); the end-of-task detail lives in `REPORT.md`.

## Current

- **v0.68.0** (2026-07-04, live on felhom-pve) — **S4.1: unattended offsite restore-test**.
  **Tier-aware restore-task deadline:** `RestoreTestSpec.RestoreTaskTimeout` (0→10m default) from
  `config.RestoreTestPBSRestoreTimeoutSeconds` (accessor default **120m**), set only when
  `SourceTier=="pbs"` (`main.restoreTaskTimeout`); local tier UNCHANGED. Fixes the WAN restore being
  killed at 10m → mid-restore teardown → leaked scratch. **Teardown "VM.Allocate" follow-up =
  PHANTOM (diagnosed, not blind-fixed):** ran the restore-test on the AGENT-TOKEN path sourcing the
  offsite (pbs) backup → `pass:true verified:boot+running`, teardown succeeded (`torn down
  vmid=990000`, no 403), scratch band clean. The earlier 403 was the 10m-timeout consequence (guest
  not yet pool-associated); the scratch is restored INTO `/pool/felhom` (ACL already grants
  VM.Allocate) so teardown is authorized once the restore completes. **No ACL/host-install change.**
  OPEN: publish 0.68.0 + Day-0 vouch; Tier-1/Tier-2 split for offsite-as-default; S5 DR consume.
  Rollback `felhom-agent.bak-0.67.0`. Detail: REPORT.md.
- **v0.66.0 + v0.67.0** (2026-07-04, live on felhom-pve) — **S4: PBS over the tunnel**. **v0.66.0**:
  wgtunnel **v4-pin** (renderConf writes the resolved A LITERAL, never DNS/AAAA; `Resolver` seam,
  lowest addr; cached → steady-state zero-DNS/zero-exec) + **re-resolve watchdog** (`Manager.Watchdog`,
  loop-only; handshake stale > `stale_after_seconds`=180 → re-resolve → IP-changed re-render+restart)
  + FELHOM_WG **Critical** flips (conf-install/enable/restart/handshake-read). **v0.67.0**:
  **namespace-aware PBS client** (Config.Namespace → `Snapshots ?ns=`, `Verify ns=`; root-ns
  unchanged) — the operator-approved fix after Phase-1 showed the ns-unaware datastore-root 403s a
  per-tenant token. **Live Scenario-D (all green):** real vzdump of 9201 → **ciphertext** in ns
  `demo-felhom-01` over the tunnel; ns-scoped verify=ok under the box's own `felhom@pbs!demo-felhom-01`
  token; WARN gone; restore round-tripped (decrypt with box-born key → boot → teardown).
  **Confirmed tenant ACL (felhom-hetzner):** `DatastoreBackup` on `/datastore/felhom-offsite/<ns>`
  (NOT `/ns/<ns>`) to BOTH user `felhom@pbs` AND token (privsep=intersection; cross-ns 403);
  DatastoreBackup can't prune (safety). **FINDINGS:** retarget field is `local_backup_target` (not
  `backup_target`); retarget REVERTED to `local` (controller backs up ~every 30 min → single-target
  offsite = near-continuous 20-min uploads; needs Tier-1/Tier-2 split); restore-test scheduler needs
  a WAN restore deadline + scratch-band `VM.Allocate` before it runs offsite unattended. **OPEN:**
  escrow-create (OPERATOR-PRESENT, new R); publish 0.66/0.67 + Day-0 vouch; S5 DR consume. Rollback:
  `felhom-agent.bak-0.65.0`/`.bak-0.66.0`. Detail: REPORT.md + doc-06 §3.4/§4.2 + runbook §4a/§4b.
- **v0.65.0** (2026-07-04, live on felhom-pve) — **S3.1 offsite-tunnel client MTU 1420 → 1280**:
  resolves `06 §4.3`'s OPEN DECISION left by the CGNAT smoke test. 1420 **silently black-holed bulk
  TCP** on sub-~1480 paths (mobile ~1400, DS-Lite ~1452) — handshake+ping healthy, PBS TLS page
  (and at S4 the backup itself) drops. New `const clientMTU = 1280` (RFC 8200 IPv6-minimum floor;
  outer 1340 v4 / 1360 v6 fits every realistic path), **permanent + fleet-wide + family-agnostic**.
  **Client-only by construction** — interface MTU caps box→PBS, advertised MSS caps PBS→box, so the
  endpoint's `wg0` is untouched (zero live-endpoint risk). Golden pins exact `MTU = 1280`
  (red-proofed vs a 1420 flip); no wire/JSON change. Live: agent re-rendered on restart (hash-gated
  apply), conf + live iface both 1280, PBS page loads at 1280 (no regression on wired). OPEN:
  true-CGNAT-SIM retest (low risk); publish 0.65.0 + Day-0 vouch (operator); S4 PBS-over-tunnel.
  Rollback: `felhom-agent.bak-0.64.0` on the box. The v4-pin (§4.2 determinism) is a separate,
  optional future note — NOT needed for MTU correctness.
- **v0.64.0** (2026-07-04, live on felhom-pve) — **S3 offsite WG tunnel**: new `internal/wgtunnel`
  (keygen 0600/0700, marker-gated one-shot registration, agent-managed `wg-quick@wg-felhom` from
  the hub's desired-state `wireguard` block via the new `desired.Syncer.AddConsumer` seam,
  revoked-stays-revoked teardown, report stanza) + `FELHOM_WG` sudoers/capabilities +
  `IdentityBundle.WGPrivateKey` escrow auto-inject. **`wg_tunnel.enabled` DEFAULTS FALSE** (safety
  gate — rollout to Peti's box is a no-op until the production endpoint exists; enabled explicitly
  on felhom-pve only). Live: tunnel to ep0.felhom.eu:443 up 3 s after enable (PBS page through
  10.77.0.1:8007), reboot-persistent, revocation drill clean, 30-min keepalive soak. GOTCHAS:
  hub envelope poll_interval_seconds (hub-side const 900 s) silently overrides agent poll_seconds
  on cycle 1; `wg show <if> dump` leaks the PRIVATE key (forbidden everywhere — sudoers only
  grants `latest-handshakes`). OPEN: CGNAT/mobile-hotspot smoke (operator-assisted appendix);
  publish 0.64.0 to Gitea + Day-0 vouch (operator); S4 points PBS at the tunnel.
- **configs: build-golden.sh v2.0.0** (2026-07-03, @ `ceca355`; no agent version change) — **drill
  findings B5 + B1 FIXED** (`DRILL-golden-098-2026-07-03.md`): the controller tag is a MANDATORY
  argument (the default rotted twice — a fresh install booted a pre-floor controller, forcing the
  guide's manual D.1b update) and the golden now bakes a `felhom-controller-bootstrap.path` unit
  (controller deploys the moment the back-half hot-plugs the bootstrap mount — no reboot; installer
  v1.9.1's reboot is a redundant belt, kept). **Golden 0.98.3** baked on the drill VM, clean-room
  validated (bake integrity → isolated hot-plug proof → local-golden Day-0 → published-artifact
  Day-0), published (sha256 b9a02ef1…fd01) + operator-vouched — Day-0 manifest now vouches
  **agent 0.63.0 + golden 0.98.3** (the v0.63.0 vouch follow-up below is DONE). Fresh installs land
  current and self-manage. NEW operator follow-up (SECURITY): the customer-config `git.token` has
  Gitea package-WRITE rights — scope down + rotate (evidence-doc observation O1).
- **v0.63.0** (2026-07-03, live on felhom-pve + Gitea-published sha256 b4a89c81…) — **drill findings
  B3 + B2 FIXED** (`DRILL-day0-cleanroom-2026-07-03.md`): `TokenStore.Lookup` reloads the append-only
  store once on a miss (cross-process coherence with the one-shot provisioner — no more fresh-install
  `/controller/swap` 401 / manual restart; size short-circuit bounds the cost; behind the
  `TokenAuthority` seam) + `guesthook.InstallSnippet` issues a fenced `mkdir -p /var/lib/vz/snippets`
  first (fresh boxes lacked the dir → the self-heal hook silently never installed). Sudoers gained
  exactly that one grant — **ship sudoers WITH the binary** (done on felhom-pve). Red-proofed both;
  Scenario-E method: compiled test suite run ON felhom-pve + live channel-health hit-path.
  **OPERATOR FOLLOW-UP: bump the hub Day-0 manifest to agent 0.63.0** — until then fresh installs get
  0.62.0 and the guide's D.1b restart-first step still applies (narrowed to "< v0.63.0" in the guide).
- **v0.62.0** (2026-07-03) — **audit A1 RESOLVED**: the stale-lock reaper's scan is now
  pool-intersected (`staleLockController.Guests()` = `ListLXC` ∩ `Client.Pool("felhom")` members),
  fail-safe skip on pool-read failure; `pve:pool-read` capability (non-critical) + `--selftest`
  "pool read" line. Companion host-install **v1.9.0** adds `Pool.Audit` to `FelhomAgentGuest` —
  **deploy order on any box: rescope ACL first, then this agent.** Per
  `SPIKE-a1-pool-membership-read-2026-07-03.md`; red-proofed tests in stalelock_pool_test.go.
- **2026-07-03 — CLAUDE.md refreshed**: version narrative removed (state lives HERE + CHANGELOG top), layout completed (all 17 internal packages + cmd/felhom-opsign); deploy runbook now in the `felhom-build-deploy` skill (`felhom.eu/skills/`).
- **2026-07-03 — `REUSE.md` exists at the repo root** (canonical helpers / format-safety guards / traps / seams, code-verified); maintenance rule active: update it in the same commit that changes a shared helper.
- **v0.61.0** (2026-07-03) — blast-radius audit fixes **B1 + D1 + D2 + D3** from
  `felhom.eu/documentation/audits/AUDIT-blast-radius-hostroot-localapi-2026-07-02.md`: random temp
  staging for root-installed scripts (+ sudoers/manifest glob updates), mkfs-wrapper member/RO
  re-checks (validated by `scripts/mkfs-guarded-harness.sh`), classifyClaim empty-lsblk fail-safe,
  and the blank-format anti-retarget (durable-id-bound, AGENT-001's benign-branch twin).
- Deployed on demo host `felhom-pve` (node `demo-felhom`), non-root `felhom-agent` service user,
  pool-scoped token (`felhom` pool).

## Open threads

- Deferred audit items (housekeeping/design, all INFO): C1 (controller-swap version floor), C2 (NAS
  server allowlist), A2 (gate journal cross-check), B2–B5, E1/E2.
- Drive-enrollment leftovers: (a) `runStorageInit` slow-device detached-format polling; (b) Impl-3
  shared-box operator format gate.
- BUNDLE leftover: non-root agent can't read the PBS key; migration must preserve cert/key/tokens.
- Not run (needs a supervised session): the destructive D1/D3 live proofs (real mkfs on a crafted
  member; a live /dev re-enumeration race during a real format).