admin/felhom.eu
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: S1 shipped — hub v0.32.0/0.32.1 CHANGELOG, peersync v1.0.0, 06 S1 status, CONTEXT, REPORT

Browse Source 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
This commit is contained in:
authentik Default Admin
2026-07-04 00:14:35 +02:00
parent d2cc02344c
commit 740cc42734
5 changed files with 154 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files
REPORT.md
+83 -43
View File
@@ -4,59 +4,99 @@
---
# REPORT — architecture Part 06: offsite-connectivity design-of-record (2026-07-03)
# REPORT — S1: offsite endpoint provisioning + hub→endpoint WG peer-sync (2026-07-04)
Docs-only. Created `documentation/architecture/06-offsite-connectivity.md` (design-of-record for
the offsite-backup transport); resolved the `day0-install.md` backlog line to cite spike + design
doc; CONTEXT + scripts/CHANGELOG updated. No code, no version bump, no deploy, no endpoint
provisioning.
Doc-06 roadmap row S1, shipped and live-validated end-to-end. Hub v0.31.0 → **v0.32.1** live on
k3s; dev endpoint (`felhom-hetzner`, 167.233.158.164) provisioned by the new runbook. The S1
design point is RESOLVED: **peer-sync = hub pushes over SSH** (pinned host key, forced-command
script) — pull/signed-manifest and an HTTPS push API rejected per the operator-locked decision.
## Baselines (verified fresh-pulled before writing)
## 1. Baselines
- felhom.eu @ `bf099f6` (spike doc commit; this task's commit lands on top)
- felhom-agent @ `4ba1b14` (v0.63.0, read-only grounding)
Start: felhom.eu @ `7fb20d5` (hub v0.31.0), felhom-agent @ `4ba1b14` (untouched, read-only).
## Grounding citations used (for reviewer validation against Gitea)
## 2. Files created/modified
| Claim in the doc | Cited at |
|---|---|
| Backup = outbound push; PBS at LAN address | `01-topology-and-trust.md` §7; `felhom-agent/internal/pbs/client.go:25-26` (`Server` documented as `"192.168.0.180"`, port default 8007) |
| Public edge HTTP-only, no public UDP | `felhom.eu/manifests/hub.yaml:233` (`ingressClassName: nginx-internal`) |
| TLS pin above WG, unchanged through tunnel | `felhom-agent/internal/pbs/pin.go:17-35` (exact-cert `VerifyPeerCertificate`); spike P2b (positive + negative through-tunnel proof) |
| Desired-state channel + generation gate | `felhom-agent/internal/hub/report.go:270` (`DesiredGeneration`), `report.go:289-296` (`WireDesiredState` + forward-compat siblings), `internal/desired/syncer.go:44-61` (fetch only on generation advance) |
| IdentityBundle to extend | `felhom-agent/internal/escrow/identity.go:24-27` (`{TunnelToken, PBSToken}`) |
| Agent already installs+enables host units | `felhom-agent/configs/felhom-agent.sudoers:16-18` (`install … /etc/systemd/system/*.mount`, `systemctl enable --now`) |
| Hub drives external infra via held token | `felhom.eu/hub/internal/cloudflare/unblock.go` (CF token, WAF mutation) |
| cloudflared = later adopter of the S3 pattern | `felhom-agent/internal/hub/cloudflared.go:9-12` (read-only prober, self-described "seam for the tunnel-management slice") |
| All measurements | `documentation/audits/SPIKE-connectivity-wireguard-2026-07-03.md` (P0–P7) |
- `hub/internal/store/store.go` (migration: `wg_endpoints` + `wg_peers`), NEW `hub/internal/store/wg.go` + `wg_test.go`
- NEW `hub/internal/wgsync/{client,reconciler}.go` + tests (incl. in-process SSH server)
- NEW `hub/internal/api/wg.go` + `wg_test.go`; `hub/internal/api/handler.go` (5 routing cases + `wgSyncer` field)
- `hub/cmd/hub/main.go` (env wiring, disabled-INFO pattern)
- NEW `scripts/felhom-peersync.sh` (v1.0.0, shellcheck-clean on 180)
- `manifests/hub.yaml` (image 0.32.1, `WG_ENDPOINT_SSH_*` env, optional Secret mount `wg-endpoint-ssh`)
- NEW `documentation/runbooks/offsite-endpoint.md` (executed live; live-run corrections folded in)
- `hub/CHANGELOG.md`, `scripts/CHANGELOG.md`, `documentation/architecture/06-offsite-connectivity.md` (S1 row → SHIPPED), `CONTEXT.md`, this REPORT
## What the doc marks DECIDED (recorded, not re-litigated)
## 3. Commits (all pushed to `main`)
D1 plain WG (no Headscale for traversal — separable future fleet layer); D2 host-side
**agent-managed** `wg-felhom` (pilot for agent-managed host services; install-script placement
rejected as one-shot; in-guest rejected on trust grounds — spike P7 shows it *works*, so the
refutation is architectural); D3 one shared dumb hub-driven endpoint VM (WG + PBS, no agent);
D4 hub = source of truth, `wireguard` block on the existing desired-state channel; D5 one
datastore + per-customer namespaces; D6 PBS ON the VM — relay-through-DooPlex rejected
(non-scaling through the ~40 Mbit home uplink, couples all DR to home uptime; DooPlex PBS =
dev/demo only).
1. `b18f6ae` store: tables + /32 allocator + Group-A tests
2. `fbeeacb` wgsync + /admin/wg API + main wiring + Groups B/C/D tests
3. `6f5fb19` peersync script + manifest surface
4. `0fa7ea1` **live-found fix**: constrain `HostKeyAlgorithms` to the pinned key type + regression test
5. `5e1933b` manifest → 0.32.1
6. `d2cc023` runbook (executed + corrected)
7. (this commit) CHANGELOGs + CONTEXT + 06 S1 status + REPORT
## What the doc marks DEFERRED / OPEN (honest ledger, §7)
## 4. Tests + red-proofs
- **Open validation (one):** true-CGNAT traversal — unmeasured (operator line = plain-NAT, spike
P0); closure = mobile-hotspot smoke test, thorough = Peti VM 110; does not block S1/S2.
- Endpoint peer-sync mechanism (push-API vs signed-manifest pull) — slice-1 design point.
- WG key-rotation mechanics; endpoint scaling/sharding/HU-hosting; endpoint-VM self-DR depth.
`cd hub && go build ./... && go vet ./... && go test ./...` — **all green** (store/api/wgsync +
all pre-existing packages). Test count: 15 new test functions across 4 new test files
(`wg_test.go` ×2, `client_test.go`, `reconciler_test.go`); pre-existing tests untouched.
## Slice roadmap set (each a future task)
All four §10 companion red-proofs run → **failed as required** → reverted → green:
- (a) allocator ignores existing rows → 3 tests fail (duplicate-IP surfaced by the UNIQUE backstop)
- (b) `isGlobal` gate dropped on POST → per-host-key 403 test fails ("sync ran despite 403")
- (c) `FixedHostKey` → `InsecureIgnoreHostKey` → wrong-host-key test fails ("the pin is dead")
- (d) full-list → delta-only push → removed-peer/full-list test fails ("surviving peer missing")
- (bonus) the new multi-hostkey regression test fails with the `HostKeyAlgorithms` fix removed
(that IS the live bug reproduced in-process).
S1 endpoint provisioning + peer-sync surface → S2 hub schema + desired-state → S3 agent WG
service (**risky/supervised**: sudoers + host service + escrow change) → S4 PBS-over-tunnel →
S5 DR re-establishment (merges with DR-completeness) → S6 monitoring/hardening.
**MVP = S1→S2→S3, then S4.**
## 5. Deployed + live validation (verbatim, secrets redacted)
## Author notes
- Rollout: `deployment "hub" successfully rolled out`, image `felhom-hub:0.32.1`, log:
`[INFO] WG peer-sync enabled (endpoint 167.233.158.164:22, user felhom-peersync)`.
- **Live bug found by validation:** first push failed `ssh: host key mismatch` — stock sshd holds
multiple host keys and default negotiation presented ECDSA against our ed25519 pin. Fixed
(v0.32.1) + regression-tested. The pin itself was proven working by the very failure.
- Scenario A live: POST peer → `{"assigned_ip":"10.77.0.2/32","sync":"ok"}` → endpoint
`wg show wg0` lists exactly that pubkey/IP; conf persisted; `[Interface]` section byte-identical.
- Scenario B live: DELETE → `{"status":"ok","sync":"ok"}` → 0 peers in `wg show` AND conf;
repeat DELETE → 404 (no sync); wrong bearer → 403.
- Script negatives on the box: malformed JSON / bad pubkey / own-IP peer → exit 1 each,
`wg show` + conf hash **byte-identical** before/after.
- Reboot: wg0 active on 443 with persisted set, nftables drop policies up, PBS active; next hub
push converges (`sync:"ok"`).
- Tunnel smoke (after operator created DNS): client on demo-felhom via `ep0.felhom.eu:443` →
ping 10.77.0.1 (0% loss, ~38 ms) → `curl https://10.77.0.1:8007/` returns the **PBS login
page** (wg0-only 8007 rule works) → public `https://167.233.158.164:8007` **unreachable**
(P2b-style re-proof).
- **Incident (handled):** the first WG server private key leaked into the session log via
`wg show wg0 dump | cut` (field 1 of the interface line IS the private key). Rotated
immediately (new keypair + conf rebuild + hub re-registration, zero-peer window); warning
added to the runbook. No key material in any committed file.
None — no §2 decision looked wrong while writing; the one place the task's own earlier framing
was contradicted by evidence (guest-WG infeasibility) was already resolved by the spike and is
recorded as an architectural (not technical) refutation in D2.
## 6. demo-felhom cleanup assertion (spike-§6 style)
`wgs1` interface down+gone ("Device does not exist"); throwaway client key + conf **shredded**;
no `s1-test`/`wgs1` files in /tmp; `/etc/wireguard/` empty, mtime Apr 2025 (untouched);
`wireguard` module unloaded. Hub registry empty (`{"peers":[]}`); endpoint `wg show` 0 peers.
The only persistent state anywhere: the endpoint box itself (intended — it IS the deliverable),
the hub DB endpoint row, and `Secret/wg-endpoint-ssh` (key exists only there; local copy shredded).
## 7. NOT yet live-validated — awaiting later slices
- Box-facing registration + `wireguard` block on `WireDesiredState` + generation bumps + hub UI (**S2**).
- Agent-managed `wg-felhom` on a real box + the CGNAT/mobile-hotspot smoke (**S3**).
- A real PBS **backup** over the tunnel into a customer namespace (**S4** — the smoke only proved
API reachability through the tunnel).
- DR re-establishment consuming the escrowed WG key (**S5**).
## 8. Observations (noticed, not acted on)
- The endpoint's **AAAA record is wrong**: `2a01:4f8:1c16:7aa1::` (subnet zero address) instead
of `…7aa1::1` — v4 works; a v6-preferring client would fail. **Operator: fix the AAAA.**
- `~/build/felhom-hub/build.sh` does its own source sync (epilogue references
`~/git/felhom.eu`); its epilogue still suggests `kubectl set image` — contradicts the GitOps
rule (cosmetic, build script only).
- Home-resolver DNS propagation lagged public DNS by ~2 min (first `wg-quick up` failed to
resolve; retry succeeded) — noted in the runbook.
- Website renders no hub version — N/A for this release (grep confirmed).