report: AGENT-001 v0.30.0 merged + deployed to demo (verified, no rollback)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,70 +1,45 @@
|
||||
# REPORT — felhom-agent v0.29.x (OS / Docker-data storage split + lanresolver fix)
|
||||
# REPORT — AGENT-001 anti-retarget wipe fix: merge + deploy (v0.30.0)
|
||||
|
||||
> **Follow-up v0.29.1 (lanresolver):** the v0.29.0 re-provision moved 9201's DHCP IP (.151 → .141), but
|
||||
> LAN clients kept resolving the old IP. Root cause: `lanresolver` updated the dnsmasq drop-in
|
||||
> (`address=/<domain>/<ip>`) then ran `systemctl reload dnsmasq` — **SIGHUP does not re-read config
|
||||
> drop-ins** (only `/etc/hosts` + cache), so the changed `address=` never took effect. Fixed:
|
||||
> `reload()` → `restartDnsmasq()` (`systemctl restart`). Deployed live; host dnsmasq restarted + Pi-hole
|
||||
> cache flushed; `*.demo-felhom.eu` now resolves to .141 and LAN HTTPS returns 200. Future IP moves
|
||||
> self-heal on the loop's next tick. (DHCP for the guest is fine — no static IP needed.)
|
||||
**Date:** 2026-06-13 (supervised deploy session)
|
||||
**Outcome:** SUCCESS — merged to `main`, built v0.30.0, deployed to the demo Proxmox host, verified non-destructively. No rollback.
|
||||
|
||||
---
|
||||
## What shipped
|
||||
|
||||
# REPORT — felhom-agent v0.29.0 (OS / Docker-data storage split: golden + provision)
|
||||
`v0.30.0` — AGENT-001 fix (from the 2026-06-13 deep-sweep audit). The inline customer-confirmed wipe in
|
||||
`internal/localapi/disks.go` `handleDiskFormat` previously ran `mkfs` on the caller-supplied mutable
|
||||
`/dev` path (`req.Device`); a USB re-enumeration between inspection and `mkfs` could wipe the wrong
|
||||
disk (classify→mkfs TOCTOU). Now it re-resolves the confirmed durable id → current device, requires a
|
||||
re-derived exact match, re-inspects data-bearing, and formats the **re-resolved** device — at parity
|
||||
with `signedjobs.WipeExecutor`. New `internal/localapi/wipe_reresolve.go` (`antiRetargetResolve`,
|
||||
injectable `reresolveWipe` seam); refusals → `409`, no `mkfs`. Merged from
|
||||
`fix/agent-001-wipe-durable-reresolve`.
|
||||
|
||||
Phase 1 of the storage-split slice (Phase 2 = felhom-controller v0.58.0 prevention layer). The
|
||||
controller guest's OS rootfs and Docker data are carved onto separate `local-lvm` volumes for
|
||||
**resilience** — an isolated OS rootfs stays bootable + agent-recoverable if the Docker volume fills.
|
||||
Built, the golden re-baked, and **live-validated by destroying + re-provisioning guest 9201**.
|
||||
## Deploy
|
||||
|
||||
## What shipped (v0.29.0)
|
||||
- **`configs/build-golden.sh`** — split baked in: `--rootfs ${ROOTFS_STORAGE}:${OS_SIZE_GB}` (default
|
||||
**32**, was hardcoded 8) **plus** `--mp0 ${ROOTFS_STORAGE}:${GOLDEN_DOCKER_GB},mp=/var/lib/docker,backup=1`
|
||||
(default 16). `daemon.json` bakes the **classic overlay2 driver** (`features.containerd-snapshotter:
|
||||
false`) + log rotation (`max-size 10m`, `max-file 3`). Guards: aborts unless `/var/lib/docker` is a
|
||||
separate mount, the driver is overlay2, and vzdump **includes** mp0 (the B3 trap).
|
||||
- **`internal/reconcile/bringup.go`** — `GuestMount.Backup` emits `,backup=1` (closes the spike-B3/B5
|
||||
silent-DB-loss trap at the mount builder). `BringUpSpec.{DataVolGrowGB,DataVolMount}` grow the
|
||||
golden-carried Docker-data volume (default `mp0`) online to the per-customer target rather than
|
||||
attaching a fresh empty volume that would shadow the baked images. Plus `RootfsGrowGB`.
|
||||
- **CLI:** `--selftest=bring-up|provision` gain `-rootfs-grow` / `-datavol-grow` / `-datavol-mount`.
|
||||
- **`RUNBOOK-provisioning-storage.md`** — provisioning procedure + fresh-PVE-install thin-pool carving
|
||||
(`hdsize`/`maxroot`/`maxvz`) + the per-customer sizing seam (flags now; hub storage manifest later).
|
||||
- Tests: `buildBringUpConfig` backup=1 emission; bring-up issues rootfs + data-volume resizes.
|
||||
- Build server (192.168.0.180, go1.26.0): green gate on the branch (`build`/`vet`/`test` clean; `TestAntiRetargetResolve` incl. `retarget-mismatch-refused` PASS). Built `v0.30.0` (ldflags `-X main.version=0.30.0`).
|
||||
- Demo Proxmox host (`felhom-pve`): binary verified by sha256 at each hop; backed up the running binary to `/usr/local/bin/felhom-agent.bak-0.29.1`; installed the new binary; `systemctl restart felhom-agent.service`.
|
||||
- **Rollback target was v0.29.1** (not needed).
|
||||
- Pin check before the live run: served PVE leaf-cert SHA-256 prefix matches the pinned value (`BA:7C:99:7D:45:D0…`).
|
||||
- Scope: **agent-only**. No golden rebake, no re-provision, no controller change. Guest 9201 controller stayed online through a brief local-API blip.
|
||||
|
||||
## The critical fix validation caught (overlay2)
|
||||
Docker 29's **default containerd-snapshotter keeps the image store at `/var/lib/containerd`** (on the OS
|
||||
rootfs), so mounting the data volume at `/var/lib/docker` only moved named volumes — **1.2 GB of images
|
||||
stayed on the rootfs** (validated live), defeating the split and breaking the controller's `statfs("/")`
|
||||
guard. Fix: the classic **overlay2** driver stores everything (images + overlay + volumes) under
|
||||
data-root = the data volume. A start-vs-restart trap (docker-ce auto-starts on install) was also fixed
|
||||
(daemon.json needs a `restart`). After the fix, a provisioned guest showed images on the data volume
|
||||
(`/var/lib/docker/overlay2`), `/var/lib/containerd` idle, and a lean OS rootfs.
|
||||
## Verification (non-destructive)
|
||||
|
||||
## Live validation
|
||||
1. **Re-baked the golden** with the split (overlay2): docker OK on overlay2, `/var/lib/docker` a separate
|
||||
ext4 mount, vzdump "including mount point mp0 ('/var/lib/docker')", 580 MB archive carrying the images.
|
||||
2. **Throwaway provision (9301)** from the golden: booted in 39 s; `pct config` mp0=`/var/lib/docker,backup=1,size=256G`,
|
||||
rootfs 32G; overlay2; baked images on the data volume; **OS isolation** — filled the data volume to
|
||||
100%, the OS rootfs stayed at 4% + writable; log rotation effective.
|
||||
3. **Destroyed + re-provisioned the live 9201** from the same golden via `--selftest=provision`
|
||||
(customer `demo-felhom`, retrieval passphrase reused from its old bootstrap; `-datavol-grow 240` →
|
||||
256 GB) + a reboot: bring-up booted in 39 s, the back-half minted a token + attached the bootstrap
|
||||
mp9, and after reboot the controller-bootstrap deployed the baked controller, which pulled its config
|
||||
from the hub. All four infra containers came up **from baked images with no pull**. Split + overlay2 +
|
||||
lean rootfs confirmed on the real guest; the controller's deploy gate refused (507) on the full data
|
||||
volume. (Phase-2 details in the controller REPORT.)
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Service active + version | `felhom-agent.service` active/running; `--version` = **0.30.0**; clean startup logs (local-api on :8443, hub desired gen=2, PBS verify ok) |
|
||||
| Local API reachable (full chain) | controller→agent `/disks` proxy returns the disk list (vmid 9201); guest 9201 controller healthy on the hub |
|
||||
| Regression A — data-bearing device, NO confirmation | **REFUSED** (HTTP 409, `needs_confirmation:true`, `formatted:false`); no `mkfs` |
|
||||
| Regression B — confirmed + WRONG durable_id | **REFUSED** (HTTP 409, `formatted:false`); a non-matching confirmation did not authorize a wipe; no `mkfs` |
|
||||
| Data safety | `felhom-usb` (`/dev/sdb1`, RomM data) still mounted, `used_bytes` unchanged — untouched |
|
||||
| Blank-device benign-format | SKIPPED — no disposable blank device available on the host |
|
||||
| Happy-path destructive wipe | SKIPPED — HUMAN-only (never wipe real data to verify); covered by the unit test |
|
||||
|
||||
## Provisioning facts confirmed (for the spec/runbook)
|
||||
- The restore carries all mountpoints (golden mp0 travels in); `freeMountSlot` picks the lowest free slot
|
||||
for a later USB enroll, so **mp0 = Docker-data, mp9 = bootstrap, USB = mp1+** — no collision.
|
||||
- Attaching/flag-changing a mountpoint on a running unprivileged guest is **pending until reboot** — the
|
||||
provision flow's post-provision reboot activates the bootstrap mp9 (and would activate a USB bind).
|
||||
- `backup=1` on the Docker-data mp is mandatory twice over: keeps named-volume DBs in PBS, AND captures
|
||||
the volume (with baked images) into the golden archive.
|
||||
Note: `reresolveWipe`'s post-gate refusal runs only when the gate ALLOWS (a correctly-matching
|
||||
confirmation), which on a real device leads to the destructive success path — so it is not safely
|
||||
live-triggerable and is verified by the unit test `retarget-mismatch-refused`. Live, both the
|
||||
no-confirmation (A) and wrong-confirmation (B) gate refusals confirm no un-gated destructive path.
|
||||
|
||||
## Outstanding (demo restoration, not slice validation)
|
||||
RomM (the HDD app) + USB re-enroll: its data is safe on the host USB (`/mnt/felhom-usb/felhom-data`,
|
||||
untouched). Restoring it is the slice-10 enroll flow (assign → guest-attach → reboot → register storage
|
||||
→ deploy); with the split it binds to mp1+ (mp0 is the Docker-data volume). Documented as the final
|
||||
restore step.
|
||||
## Branch / commits
|
||||
|
||||
- `main` merge + version bump pushed. Branch `fix/agent-001-wipe-durable-reresolve` left in place (per the runbook) until the operator confirms.
|
||||
- felhom.eu audit record (`documentation/audits/deep-sweep-2026-06-13.md`) updated: AGENT-001 → MERGED + DEPLOYED (agent v0.30.0).
|
||||
|
||||
Reference in New Issue
Block a user