diff --git a/CONTEXT.md b/CONTEXT.md index 1b65afc..f4b3583 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -3,6 +3,17 @@ > Created with the REUSE.md rollout (2026-07-03). Authoritative history: `hub/CHANGELOG.md` (hub), > `website/CHANGELOG.md`, `scripts/CHANGELOG.md`; end-of-task detail in `REPORT.md`. +- **2026-07-04 — S3 SHIPPED: agent WG tunnel (felhom-agent v0.64.0; docs-only here)** — the + doc-06 §3.3 handshake is now END-TO-END LIVE on demo: agent keygen → `POST /hosts/{id}/wg` → + desired-state `wireguard` block → agent-managed `wg-quick@wg-felhom` → PBS page over + 10.77.0.1:8007 in 3 s. **`wg_tunnel.enabled` defaults FALSE agent-side** (rollouts to other + boxes are no-ops until the production endpoint exists). Revocation semantics completed (§3.5): + removed peer → agent disables + keeps its marker + NEVER re-registers; operator re-adds via the + pubkey the heartbeat report now carries. Escrow: `IdentityBundle.WGPrivateKey` auto-injected on + new escrow-create runs (pre-S3 blobs can't be retrofitted — honest fallback = fresh-key + re-register, /32 kept). OPEN: CGNAT/mobile-hotspot smoke (operator-assisted appendix, deferred + non-blocking); S4 = PBS over the tunnel; hub-side S6 = tunnel-health alerting on the new report + stanza. Detail: felhom-agent CHANGELOG v0.64.0 + REPORT. - **2026-07-04 — S2 SHIPPED: box-facing WG registration + wireguard desired-state block + /offsite UI (hub v0.33.0)** — doc-06 row S2. DECISIONS: the WG assignment is **hub-owned state, merged into served desired-state at READ time** — stored `desired_json` stays a pure operator blob; admin PUT with diff --git a/REPORT.md b/REPORT.md index f3fe0e3..778795f 100644 --- a/REPORT.md +++ b/REPORT.md @@ -4,105 +4,22 @@ --- -# REPORT — S2: box-facing WG registration + `wireguard` desired-state block + /offsite UI (2026-07-04) +# REPORT — S3 wrap-up (docs-only in this repo; the slice lives in felhom-agent v0.64.0) (2026-07-04) -Doc-06 roadmap row S2 shipped: hub v0.32.1 → **v0.33.0** live on k3s; peersync script v1.0.0 → -**v1.0.1** redeployed to the dev endpoint. The agent was NOT touched; the S3 contract travels -via the new golden only. +felhom.eu carried **no code** in S3 — hub v0.33.0 (S2) already served the registration endpoint ++ merged `wireguard` block. This commit records the S3 outcome in the design of record: -## 1. Baselines +- `documentation/architecture/06-offsite-connectivity.md`: roadmap row **S3 → SHIPPED** + (agent v0.64.0; `wg_tunnel.enabled` defaults FALSE until the production endpoint exists); + §3.5 gains the **agent-side revocation semantics** (register-once marker; block-absent-from- + present-state = disable + marker kept + never self-re-register; re-add via + `POST /hosts/{id}/wg` with the global key — the raw registry add does NOT bump the host + generation, found live) and the rollout gate. +- `CONTEXT.md`: S3 entry (end-to-end handshake live on demo: keygen → register → block → + wg-quick → PBS page in 3 s; revocation drill + 30-min soak + reboot persistence). -Start: felhom.eu @ `740cc42` (hub v0.32.1, peersync v1.0.0), felhom-agent @ `4ba1b14` (v0.63.0, -read-only). Live baseline captured before validation: `demo-felhom-01` at generation **2**, -desired-state `{"guests":[]}`, WG registry empty, guest 9201 running (controller 0.98.3 -healthy), agent v0.63.0, poll interval 900 s. - -## 2. Files created/modified - -- `hub/internal/store/store.go` (partial unique index `idx_wg_peers_host`; `BumpHostDesired`), - `hub/internal/store/wg.go` (+`allocateWGPeerTx` extraction, `RegisterWGPeerForHost`, - `GetWGPeerForHost`, `WGPeer.CreatedAt`), `wg_test.go` (Group A) -- `hub/internal/api/wg.go` (`handleRegisterHostWG`, `mergeWireguard`, delete-bumps-owner), - `handler.go` (routing case; merge call in `handleGetDesiredState`; `wireguard`-key rejection - in `handleAdminSetDesiredState`), `wg_test.go` (Group B), NEW - `testdata/desired-state-wireguard.golden.json` (**S3 cross-repo contract**) -- `hub/internal/web/offsite.go` + `templates/offsite.html` + `offsite_test.go` (Group C); - `server.go` routing; Offsite nav link in all 9 page templates -- `scripts/felhom-peersync.sh` v1.0.1 (exit-swallow fix) -- `manifests/hub.yaml` (image 0.33.0), `hub/CHANGELOG.md`, `scripts/CHANGELOG.md`, - `documentation/architecture/06-offsite-connectivity.md` (§3.2 concrete wire shape + S2 row - SHIPPED), `CONTEXT.md`, this REPORT - -## 3. Commits (pushed to `main`) - -1. `fcf84a0` store: host binding + index + BumpHostDesired -2. `ba52005` API: registration + merge-at-read + hub-owned-key guard + golden -3. `13203c2` web: /offsite page + nav -4. `30d762d` peersync v1.0.1; `99246c3` manifest 0.33.0 bump -5. `265f46c` docs (+ this hash-fix follow-up) - -## 4. Tests + the five §10 red-proofs - -`cd hub && go build ./... && go vet ./... && go test ./...` — all green (12 packages). -New: 6 store tests (Group A), 5 API tests (Group B), 2 web render tests (Group C). -`TestDesiredStateGolden_RoundTripsThroughHub` passes **UNMODIFIED**; the S1 suites pass -unmodified through the allocator extraction. - -Red-proofs, each run → FAILED as required → reverted: -- (a) self-scope check dropped → `cross-host register = 200, want 403` -- (b) merge made unconditional → caught by the no-peer-serves-no-block assertion - (`GET serves a wireguard block that was never legitimately registered`); the golden test - itself survived only via the fail-safe unmerged path (endpoint record absent in its setup) — - the pass-through property is covered either way -- (c) `wireguard`-key rejection dropped → `PUT with wireguard key = 200, want 400` -- (d) bump-on-idempotent → both negatives fail (`gen=2 want 1`, `push count 2 want 1`) -- (e) script exit-swallow (sandbox, stub `wg-quick` exit-1-after-partial + recording stub `wg`): - pre-fix shape invoked `wg` with rc=0 despite the strip failure; fixed shape errors first, - `wg` never invoked - -## 5. Deploy + live validation (real pipeline, secrets redacted) - -- hub 0.33.0 rolled out via ArgoCD (`deployment "hub" successfully rolled out`, image - `felhom-hub:0.33.0`, `[INFO] WG peer-sync enabled`); peersync v1.0.1 on the endpoint - (`head` shows the v1.0.1 header), shellcheck-clean on 180. -- **Registration through the REAL per-host auth path** (curl run ON felhom-pve; its key never - left the box; throwaway keypair minted + private key shredded unused): - `{"assigned_ip":"10.77.0.2/32","existed":false,"generation":3,"sync":"ok"}` — gen 2→3. -- Served desired-state carried exactly the golden-shaped block (endpoint coords + pubkey + - assigned_ip) alongside the untouched `guests` key; endpoint `wg show wg0` listed the pubkey - with `10.77.0.2/32`. -- **Old-agent tolerance (the S3 fleet-safety gate), verbatim journal:** - `time=2026-07-04T01:04:29.267+02:00 level=INFO msg="desired: updated from hub" generation=3 guests=0` - — the v0.63.0 typed parse dropped the unknown key; guest count unchanged; NO error/warn from - the desired/reconcile path (the only WARN in the window is the pre-existing, documented - `pbs: cannot read token secret` non-root permission issue — unrelated to S2). - Guest-state re-capture = byte-identical to baseline (9201 running, controller 0.98.3 healthy). -- **Removal:** DELETE → `{"status":"ok","sync":"ok"}` → generation 4; desired-state back to the - pure operator blob (`{"guests":[]}`, no wireguard key); registry `{"peers":[]}`; endpoint - `wg show` 0 peers. Gen-4 clean fetch by the agent, verbatim: - `time=2026-07-04T01:19:29.228+02:00 level=INFO msg="desired: updated from hub" generation=4 guests=0` - — no errors; the box is fully back to baseline except the two expected generation increments. -- `/offsite` route wired + auth-gated live (302 → /login); page rendering proven by Group C - unit tests — the live view is operator-password-gated (eyeball on next login). - -**Generation accounting for demo-felhom-01:** baseline 2 → 3 (registration) → 4 (removal) — -exactly the two expected increments; desired_json blob never modified (byte-identity asserted -in tests; live blob returned to the identical `{"guests":[]}`). - -## 6. NOT yet live-validated — awaiting S3+ - -- Agent keygen/registration/`wg-felhom` service + IdentityBundle escrow join + CGNAT/mobile - smoke (**S3**, risky/supervised). -- A real PBS backup over the tunnel into a namespace (**S4**). -- DR re-establishment consuming the escrowed WG key (**S5**). -- UI mutations + tunnel-health columns (**S3/S6**). - -## 7. Observations (noticed, not acted on) - -- The agent's 900 s poll makes desired-state validation slow-loop (two ≤16-min waits in this - run) — S3 test plans should budget for it or temporarily shorten the interval on the test box. -- `hub-config`'s `report_api_key` doubles as the only global admin key; a scoped operator token - would narrow the blast radius of the endpoint-mutation surface (pre-existing, S6-adjacent). -- The pre-existing `pbs: cannot read token secret` WARN on felhom-pve (non-root agent vs - `/etc/pve/priv/storage/felhom-pbs.pw`) fires every cycle — known BUNDLE-slice issue, gets - resolved naturally when S4 points PBS at the tunnel with per-customer tokens. +Full evidence: **felhom-agent REPORT.md** (journal excerpts, five red-proofs, the +hub-poll-constant gotcha, the re-add-path insight). Still open: CGNAT/mobile-hotspot smoke +(operator-assisted appendix), S4 PBS-over-tunnel, S5 DR consume, S6 tunnel-health alerting + +the two S6 backlog notes (make `defaultHostPollSeconds` configurable + log first-cycle adoption; +make the admin registry-add bump the bound owner or mark it registry-plumbing in the UI). diff --git a/documentation/architecture/06-offsite-connectivity.md b/documentation/architecture/06-offsite-connectivity.md index f30206f..ade1780 100644 --- a/documentation/architecture/06-offsite-connectivity.md +++ b/documentation/architecture/06-offsite-connectivity.md @@ -131,6 +131,16 @@ code, restores the WG key, and re-establishes the tunnel; hub-side the peer entr the endpoint: the box is off the offsite network immediately, auditable, no box cooperation needed. +**Revocation semantics, agent side (S3-final):** the agent registers ONCE (key exists + no local +marker). A `wireguard` block absent from a PRESENT desired-state = revocation → the agent +disables the service, KEEPS its marker, and never re-registers on its own — revoked stays +revoked until the operator re-adds the peer (the heartbeat report carries the box's pubkey as +the recovery handle; **re-add via `POST /hosts/{id}/wg` with the global key** — the raw registry +add does not bump the host's generation). The only automatic re-registration is the +pubkey-mismatch (DR/re-key) path, backoff-bounded. Absent DATA (a failed fetch) is never a +teardown signal. **Rollout gate:** `wg_tunnel.enabled` defaults FALSE agent-side until the +production endpoint exists. + --- ## 4. Robustness (production details beyond the spike) @@ -225,7 +235,7 @@ block slices 1-2. |---|---|---| | **S1 — endpoint provisioning + peer-sync surface** — **SHIPPED 2026-07-04** (dev endpoint; hub v0.32.1). Peer-sync design point RESOLVED: **hub pushes over SSH** (`x/crypto/ssh`, `FixedHostKey` pin + pinned-type `HostKeyAlgorithms`) to a forced-command reconcile script; runbook `documentation/runbooks/offsite-endpoint.md`. Done-criterion met live: hub add/remove → `wg show` on the endpoint; tunnel `ep0.felhom.eu:443` → PBS through the wg0-only 8007 rule. | ~~provision the production VM…~~ (see runbook; production endpoint = a later re-run) | hub can add/remove a peer on the live endpoint via its credential; runbook re-creates the VM from nothing ✓ | | **S2 — hub schema + desired-state** — **SHIPPED 2026-07-04** (hub v0.33.0). `POST /hosts/{id}/wg` (self-scoped) + merge-at-read `wireguard` block (§3.2 shape = the golden) + one-per-host index + re-key-in-place + `/offsite` UI page. Live-proven against the real felhom-pve record incl. old-agent (v0.63.0) tolerance. | ~~endpoint coords + peer registry…~~ | a registered box's desired-state carries its assignment ✓; hub UI shows the peer registry ✓ | -| **S3 — agent WG service (the agent-managed-unit pilot)** | keygen + registration; reconcile `wg-felhom` from desired-state (sudoers additions on the `:16-18` pattern); re-resolve watchdog (§4.2); MTU 1420; IdentityBundle `+ WGPrivateKey`; tunnel-health into the report. **Risky/supervised class** (host service + sudoers + escrow change) | fresh box joins by §3.3 end-to-end; tunnel survives agent restart + endpoint re-IP; unhealthy tunnel alerts | +| **S3 — agent WG service (the agent-managed-unit pilot)** — **SHIPPED 2026-07-04** (agent v0.64.0; `wg_tunnel.enabled` DEFAULT FALSE until the production endpoint exists). `internal/wgtunnel`: keygen → marker-gated registration → `wg-quick@wg-felhom` reconciled from the served `wireguard` block (restart-not-reload; hash-gated zero-exec steady state); **revocation completed §3.5**: block absent from a PRESENT desired-state → disable + marker kept + never re-register (operator re-adds via the reported pubkey); absent DATA never tears down. `FELHOM_WG` sudoers (latest-handshakes is the ONLY wg read — `dump` leaks the private key); `IdentityBundle.WGPrivateKey` (pre-S3 blobs can't be retrofitted — S5 falls back to fresh-key re-register, /32 kept); report stanza = §4.6's tunnel-health input. Endpoint re-resolution: wg-quick resolves at unit (re)start; the manager restarts on conf change and the operator path covers re-IP (full §4.2 watchdog → S6 with tunnel-health alerting). Live on felhom-pve: up in 3 s, reboot-persistent, revocation drill, 30-min soak. | ~~keygen + registration…~~ | fresh box joins by §3.3 end-to-end ✓; tunnel survives agent restart + host reboot ✓; unhealthy-tunnel ALERTING → S6 (stanza shipped) | | **S4 — PBS over the tunnel** | point the box's PBS storage at `pbs_tunnel_ip:8007`; verify pin-through-tunnel | real backup lands in the box's namespace over the tunnel (nearly free — spike-proven path) | | **S5 — DR re-establishment** | consume `WGPrivateKey` in restore mode; re-peer on re-provision | host-loss drill restores offsite connectivity with only the recovery code. *Merges with the DR-completeness arc.* | | **S6 — monitoring + isolation hardening** | endpoint-side alerting, per-peer firewall audit, CGNAT smoke-test closure if not yet done | tunnel-down and peer-anomaly both alert; isolation verified adversarially | diff --git a/scripts/CHANGELOG.md b/scripts/CHANGELOG.md index 404729e..1c423bb 100644 --- a/scripts/CHANGELOG.md +++ b/scripts/CHANGELOG.md @@ -1,5 +1,15 @@ # Felhom scripts — Changelog +## docs — 06-doc S3 row SHIPPED + agent-side revocation semantics (2026-07-04) + +Docs-only companion to **felhom-agent v0.64.0** (the S3 slice — keygen, registration, +agent-managed `wg-quick@wg-felhom`, escrow join; live-validated on felhom-pve incl. revocation +drill, reboot persistence, 30-min soak). 06-doc §3.5 now records: register-once marker, +revoked-stays-revoked, re-add via the registration endpoint (the raw registry add doesn't bump +the host generation — live finding), `wg_tunnel.enabled` default-FALSE rollout gate. S6 backlog +notes added (hub poll constant configurable + first-adoption log; registry-add bump-or-label). +CGNAT/mobile-hotspot appendix deferred (operator-assisted; §7's open validation stands). + ## felhom-peersync.sh v1.0.1 — strip out of process substitution (exit-swallow fix) (2026-07-04) The S1 REPORT's exit-swallow class, fixed: `wg syncconf wg0 <(wg-quick strip "$tmp")` hid the