From 4d9e76e66ad5abe3aa00a46d671471b16746e764 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Wed, 10 Jun 2026 11:04:38 +0200 Subject: [PATCH] REPORT + CLAUDE: slice 8B agent half (/backup/due cadence + /backup/status phases, v0.11.0) Co-Authored-By: Claude Opus 4.8 --- CLAUDE.md | 2 +- REPORT.md | 98 ++++++++++++++++++++----------------------------------- 2 files changed, 36 insertions(+), 64 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 3fe1b07..777ce72 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -15,7 +15,7 @@ - Module `gitea.dooplex.hu/admin/felhom-agent`; binary `felhom-agent` (`cmd/felhom-agent/`). - **Pure Go stdlib + `golang.org/x/crypto` only** — no web frameworks. - `go.mod` directive **go 1.25.0**; dep `golang.org/x/crypto v0.52.0` (declares go 1.25, will NOT build on Go 1.24). The **build server (192.168.0.180) runs go1.26.0** (upstream Go on PATH, backward-compatible). Build/run the agent there for live tests (same LAN as the demo host). -- Version: `version` var in `cmd/felhom-agent/main.go`, overridable via `-ldflags "-X main.version="`; `--version` flag. **Current: v0.10.0** (slice 8A: the per-guest **local-API server** `internal/localapi` — the doc 03 §6 authorization gate, 7 self-scoped endpoints, hashed per-guest token store, persisted self-signed leaf w/ stable pin; the **provisioning back-half** `internal/provision` — mint token → render `bootstrap.json` → write 0600 → `chown 100000:100000` → `pct set` bind mount, host-side/F3, no registry cred in guest; `--selftest=provision`; `build-golden.sh` now bakes the controller. Plus v0.9.0's PBS escrow. Runtime dep: `proxmox-backup-client`). Bump on meaningful changes + add a CHANGELOG entry. +- Version: `version` var in `cmd/felhom-agent/main.go`, overridable via `-ldflags "-X main.version="`; `--version` flag. **Current: v0.11.0** (slice 8B: `internal/localapi` `/backup/due` real **cadence** policy + `/backup/status` **phases** (idle|running|done|failed) + `POST /backup` single-flight job id — the app-consistent backup the controller's quiesce loop drives; `backup.backup_cadence_seconds`. Plus slice 8A: the per-guest **local-API server** `internal/localapi` — the doc 03 §6 authorization gate, 7 self-scoped endpoints, hashed per-guest token store, persisted self-signed leaf w/ stable pin; the **provisioning back-half** `internal/provision` — mint token → render `bootstrap.json` → write 0600 → `chown 100000:100000` → `pct set` bind mount, host-side/F3, no registry cred in guest; `--selftest=provision`; `build-golden.sh` now bakes the controller. Plus v0.9.0's PBS escrow. Runtime dep: `proxmox-backup-client`). Bump on meaningful changes + add a CHANGELOG entry. ## Layout diff --git a/REPORT.md b/REPORT.md index 863075a..9d75531 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,79 +1,51 @@ -# REPORT — slice 8A (agent half): local-API server + provisioning back-half (v0.10.0) (2026-06-10) +# REPORT — slice 8B (agent half): app-consistent backup policy + status phases (v0.11.0) (2026-06-10) > Overwrite-latest report (most recent significant work only). Cumulative history lives in -> [CHANGELOG.md](CHANGELOG.md). Implements the host-agent half of `TASK — Slice 8A`. The in-guest -> controller half is in `felhom-controller` v0.35.0. No hub change. Grounded by -> `felhom.eu/documentation/tests/slice8a-channel-deploy-spike-findings.md` (commit `4a81a96`). +> [CHANGELOG.md](CHANGELOG.md). Implements the host-agent half of `TASK — Slice 8B`. The in-guest +> controller quiesce loop is in `felhom-controller` v0.36.0. No hub change. ## Outcome -The agent now exposes the **per-guest local API** (doc 03 §6) the in-guest controller calls over the -bridge, and runs the **provisioning back-half** that follows the slice-7 bring-up front half. With -the slice-7 front half, this completes the **full provisioning chain**: provision a guest → deploy -its (golden-baked) controller → controller comes up configured and talks to the agent — **validated -live end-to-end on the demo**. +Turns the 8A thin backup stubs into the real policy the controller's quiesce loop drives. An +agent-initiated vzdump is crash-consistent only (an LXC has no fsfreeze); the controller now stops +its app stacks around `POST /backup` (8B) so the captured state is clean-shutdown-consistent — +**validated live end-to-end** including the postgres clean-vs-crash-recovery restore contrast. -## What landed +## What landed (`internal/localapi`) -- **`internal/localapi`** — the HTTPS local-API server, the per-guest authorization gate. - - Serves a **persisted self-signed leaf** with a **stable SHA-256 fingerprint** (generated once; a - fresh cert each boot would invalidate every baked bootstrap pin). The fingerprint is what the - controller pins (leaf-cert SHA-256 — consistency with the agent's PVE/PBS pinning). - - **7 §6 endpoints**, all self-scoped: `GET /storage` (this guest's mpN mounts + fast/slow class), - `POST /snapshot`, `POST /rollback`, `POST /backup` (enqueued, crash-consistent — quiesce is 8B), - `GET /backup/due` (thin in 8A), `GET /backup/status`, `GET /restore-test/status`. - - **Token store**: durable, crash-safe per-guest token→guest map persisting only a **SHA-256 hash** - (plaintext exists transiently mint→write-to-mount, then discarded); last-write-wins; fsync'd JSONL. - - **Self-scoping**: VMID is resolved ONLY from the token; an explicit `vmid` that disagrees → **403, - op never issued for the other guest**; absent/unknown → 401. -- **`internal/provision`** — the back-half: mint per-guest token → render the stable `bootstrap.json` - (`schema felhom.bootstrap/v1`; **no registry credential** — controller image is golden-baked) → - write `0600` → **`chown 100000:100000`** (mapped guest-root, spike gotcha 1) → attach a read-only - bind mount via `pct set`. Host-side only (F3 — no `pct exec`). The token is never logged/returned. -- **`--selftest=provision`** — full chain on-demand (front half + back half; keeps the guest). -- **`config.LocalAPIConfig`** (`local_api`) — optional 6th daemon goroutine; disabled cleanly when - unconfigured or on a token-store/cert failure (the daemon still reports/reconciles). -- **`configs/build-golden.sh`** bakes the controller image (pulled once on the trusted build host, - then `docker logout` — no cred baked) + a controller-bootstrap unit that deploys the **baked** image - from the config mount on boot. Plus `configs/felhom-localapi-firewall.example` and a narrow - `FELHOM_PROVISION` sudoers alias. - -## Design note — the back-half's host-root surface - -The back-half's `chown` + bind-mount attach are host-root ops that the API token cannot do (bind -mounts are root@pam-only). They are **NOT** added to `proxmox.Privileged` (fenced to its 3 -exceptions, "do not add"); they live in `internal/provision` and run through the shared `Runner` -(direct as root, or `sudo -n` with the new sudoers alias), confined to the agent-owned -`/var/lib/felhom-agent/guests/*` path. This is the per-guest provisioning host-root surface — -host-side and F3-compliant. +- **`GET /backup/due`** — real **cadence** policy (replaces the 8A "never backed up" stub): due when + no **successful** backup is recorded OR the newest one is older than the agent-local cadence + (`backup.backup_cadence_seconds`, default 24h). A successful `POST /backup` flips due to **false** + for the window (the controller won't re-quiesce in a loop); a **failed** backup does not satisfy the + cadence. Returns `age_seconds`. +- **`GET /backup/status`** — real **phases** `idle | running | done | failed` + the job id, so the + controller can poll a backup to completion. (The `snapshotted` phase is the 8B.2 hook.) +- **`POST /backup`** — returns a **job id** + `running`; tracks the in-flight job and is + **single-flight per guest** (a second POST while one runs returns the same job — no concurrent + vzdump). On completion the job transitions done/failed and the result is recorded to the store. +- Config: `backup.backup_cadence_seconds` + `BackupCadence()`; the local-API server takes the cadence. +- All still self-scoped (token→guest); the snapshot/rollback/storage 8A surface is unchanged. ## Tests -`go test ./...` green; `-race` green on the build server (pure-Go parts). localapi: self-scoping -(cross-guest snapshot/rollback/backup → 403, op never issued; own-guest uses the token's VMID), 401 -paths, `/storage` class mapping, `/backup` enqueue, thin `/backup/due`, status scoping; token store -persists only the hash (plaintext never on disk), last-write-wins, survives reopen, uniqueness; the -leaf fingerprint is stable across reload. provision: writes 0600 + chowns + attaches with the right -args; the **token never appears in the Result**; the cross-repo `bootstrap.json` key-set is pinned. +`go test ./...` green; `-race` green on the build server (pure-Go parts). `/backup/due`: due when +stale / no backup, **not due within the window after a success**, due again past the cadence, **a +failed backup does not count**. `/backup/status`: running→done and running→failed (a gated fake +observes the running phase). `POST /backup` single-flight (one vzdump for concurrent POSTs). Plus the +8A self-scoping/auth suite unchanged. ## Live validation (demo-felhom) -Built a **new golden with the controller v0.35.0 baked** (`build-golden.sh`). `--selftest=provision` -into guest 8201: front half up (fresh MAC, 13s) → back half minted the token, wrote `bootstrap.json` -(`0600`, `chown 100000:100000`, verified root:root 0600 inside the guest) and `pct set`-attached the -read-only mount (mp9). The golden's **baked unit deployed the controller (no registry pull)**; the -controller **ingested the bootstrap → came up CONFIGURED, not setup mode** → reached the **real** -local-API `GET /storage` → **200**, saw its mount. Self-scoping against the live server: own→200, -cross-guest `?vmid=9999`→**403**, bogus→401, absent→401. Firewall narrowing demonstrated (source -gating: an off-bridge stand-in blocked, the guest allowed). **Secret discipline**: token plaintext -absent from the agent log, store holds only the hash, bootstrap `0600`+chowned, selftest never prints -the token. The standing demo agent service was upgraded to **v0.10.0** (local-API live on -`192.168.0.162:8443`); the test guest was torn down. +A provisioned guest (controller v0.36.0, postgres app stack) with a short test cadence: the +controller's quiesce loop hit `GET /backup/due` (true) → quiesced → `POST /backup` → the agent ran a +**real vzdump** (`local-api: backup job complete … archive=vzdump-lxc-8202-…`) → `GET /backup/status` +`done` → unquiesce. `/backup/due` then correctly went **false** for the cadence window (no re-loop). +The single-flight + phase transitions behaved against the live server. (The controller's legacy +hub-report HTTP 401 is the frozen per-customer path, unrelated to 8B.) The standing demo agent was +upgraded to **v0.11.0** (local-API live on `192.168.0.162:8443`); test guests torn down, cadence reset. ## Deferred (stated, not built) -Quiesced app-consistent backup (`/backup/due` quiesce loop) → 8B. Controller de-privileging + customer -disk endpoints → 8C. The `BringUpSpec`/provision source (hub desired-state) + deterministic -attach-before-start sequencing → slice 10 (in 8A the back-half follows the front-half's start; the -golden bootstrap unit's `After=docker.service network-online.target` reliably sequences it after the -host-side mount attach). No secrets committed. +**8B.2** downtime optimization (`vzdump --mode snapshot` + a `snapshotted` status phase so the +controller resumes at snapshot-taken) — needs snapshot-capable storage validated for LXC; the status +phase enum already carries the hook. Hub-served per-guest backup policy → slice 10. No secrets committed.