docs: REPORT + README for v0.44.0 capability self-probe
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EPZ4GJ8L5Jqf8UiPwbn1kt
This commit is contained in:
@@ -23,9 +23,27 @@ internal/authz/ # operator signed-op verifier (SSHSIG); durable nonce sto
|
||||
internal/hub/ # daemon: host-report collector + Bearer client + resilient poll loop
|
||||
internal/reconcile/ # reconcile engine + reversibility gate + op journal + crash recovery
|
||||
internal/storage/ # storage-target observer + durable_id + fast-poll watchdog (slice 5)
|
||||
internal/capability/ # privileged-capability self-probe (manifest + sudo -n -l check, v0.44.0)
|
||||
configs/agent.example.json
|
||||
```
|
||||
|
||||
## The `capability` package — privileged self-check (v0.44.0)
|
||||
|
||||
The non-root agent depends on a fixed set of `sudo -n` grants (`configs/felhom-agent.sudoers`). When
|
||||
the 2026-06-28 root→non-root cutover silently dropped some (lxc-info, make-private, …), features broke
|
||||
unnoticed until a user hit them. `internal/capability` makes that loud:
|
||||
|
||||
- **`Manifest()` is the source of truth** for the required `(binary, representative-arg)` vectors,
|
||||
seeded from the sudoers audit (the OK + CLOSED rows; the surfaced/deferred rows are excluded). When
|
||||
you add a privileged `runner.Run` call that needs a NEW grant, add it to BOTH the sudoers file AND
|
||||
the manifest — `manifest_test.go` asserts **manifest ⊆ sudoers** at build time (with a red-proof
|
||||
that dropping the lxc-info grant fails the gate), so a missing grant is caught in CI, not in prod.
|
||||
- **`Prober.Probe`** lists each vector with `sudo -n -l` (a policy LIST — never executes, safe for
|
||||
mkfs/pct) + an `os.Stat` existence check → an `ok`/`degraded` snapshot. It runs at startup (logged
|
||||
loud) and on every collect, riding the hub report as `HostReport.Capabilities`; the hub
|
||||
(`HostCapabilityChecker`) alerts the operator on a Critical capability going degraded. Serve-degraded
|
||||
— the probe never blocks startup. (Next self-health slice: the controller↔agent channel check.)
|
||||
|
||||
## The `storage` package — observe + watchdog (slice 5)
|
||||
|
||||
Read-only this slice (no hub desired-state until slice 10):
|
||||
|
||||
@@ -1,121 +1,79 @@
|
||||
# REPORT — Agent sudoers completeness audit + redeploy (restore multi-drive)
|
||||
# REPORT — felhom-agent v0.44.0: privileged-capability self-probe
|
||||
|
||||
**Date:** 2026-06-29 · **Class:** Risky/supervised (host-root allowlist edit + live redeploy) ·
|
||||
**Binary:** unchanged (no version bump) · **Baseline:** felhom-agent `main` @ `c3020ee` → this change
|
||||
`55888e5`.
|
||||
**Date:** 2026-06-29 · **Class:** Implementation (agent self-health, slice 1) · **Baseline:**
|
||||
`main` @ `1958204` (v0.43.0) → **v0.44.0** @ `25024d9`. Companion hub: felhom.eu **v0.19.0**.
|
||||
|
||||
## What & why
|
||||
|
||||
The 2026-06-28 root→non-root agent cutover left `configs/felhom-agent.sudoers` **incomplete**:
|
||||
several commands the agent shells via `sudo -n` were never granted. The headline casualty —
|
||||
`lxc-info` — broke `guestInitPID → GuestSeesMount → bound_under_parent`, so **every external drive was
|
||||
reported absent**, and the controller drive-gate stopped each drive's apps (the "attach one, the other
|
||||
drops" flapping; audit `felhom.eu/documentation/audits/SPIKE-multidrive-mutual-exclusion-2026-06-29.md`).
|
||||
This pass audited **every** privileged command against the allowlist and closed all read-only/
|
||||
fixed-vector gaps in one go; arbitrary-execution gaps were **surfaced, not added**.
|
||||
A non-root agent that can't run a privileged command it depends on is **degraded** and must say so —
|
||||
at cutover, not days later. Five casualties of the 2026-06-28 root→non-root migration (leaf/token/
|
||||
lxc-info/make-private/controller-swap) went undetected until user-visible breakage. This adds a
|
||||
self-check that catches the class at BUILD time (CI) and RUNTIME (startup + each hub-report cycle),
|
||||
serve-degraded + loud. (The controller↔agent channel check is a separate later task.)
|
||||
|
||||
## Audit table — every privileged command vector vs the allowlist
|
||||
## `internal/capability` — manifest + probe
|
||||
|
||||
`runner` prepends `sudo -n <cmd> <args>`; the allowlist matches **path + arg vector**.
|
||||
- **Manifest** (`Manifest()`): the required `(binary, representative-arg)` vectors, seeded from the
|
||||
2026-06-29 sudoers audit (the OK + CLOSED rows). The SURFACED/DEFERRED rows are **excluded**:
|
||||
general `pct exec * ` (controller-swap, arbitrary exec — an open operator decision), `pct create`
|
||||
(golden build, no daemon caller), `mount UUID=…` (legacy), `sensors -j` (callerless + uninstalled).
|
||||
- **Critical** (hub alerts on these): `guest-init-pid` (lxc-info), the intermediary mount surface
|
||||
(`parent-self-bind`, `parent-make-shared`, `parent-make-private`, `drive-bind`, `drive-umount`),
|
||||
and the disk format/inspect gate (`disk-blkid`, `disk-lsblk`, `disk-mkfs-ext4`). The rest
|
||||
(provision/dnsmasq/guesthook/mount-units/smart/lvs) are non-critical (degrade reported, no alert).
|
||||
- **Probe** (`Prober.Probe`): per entry, `sudo -n -l -- <binary> <reprArgs>` — a sudo policy LIST
|
||||
(exit 0 ⇔ permitted), run via a **DIRECT** runner so it never executes the command (safe for
|
||||
mkfs/pct entries) — plus an `os.Stat` existence check. Maps to `ok` / `degraded` ("sudo policy
|
||||
denied" | "binary not found"). A total sudo failure (drop-in missing) collapses to ONE aggregate
|
||||
signal. **Never** blocks startup, panics, or errors (serve-degraded).
|
||||
|
||||
| Command vector (shape) | Call site | Alias | Status |
|
||||
|---|---|---|---|
|
||||
| `lxc-info -n [0-9]* -p -H` | intermediary.go:256 `guestInitPID` | FELHOM_INTERMEDIARY | **GAP → CLOSED** |
|
||||
| `mount --make-private /mnt/felhom-drives` | intermediary.go:110 `EnsureSharedParent` | FELHOM_INTERMEDIARY | **GAP → CLOSED** |
|
||||
| `systemctl restart dnsmasq` | lanresolver `restartDnsmasq` | FELHOM_DNSMASQ | **GAP → CLOSED** |
|
||||
| `pct set [0-9]* -onboot 1` | backhalf.go:162 (F3 autostart) | FELHOM_PROVISION | **GAP → CLOSED** |
|
||||
| `pct reboot [0-9]*` | guestbind.go:100 `RebootGuest` (disks.go:448) | FELHOM_GUESTHOOK | **GAP → CLOSED** |
|
||||
| `mount --make-shared /mnt/felhom-drives` | intermediary.go:113 | FELHOM_INTERMEDIARY | OK |
|
||||
| `mount --bind /mnt/felhom-drives /mnt/felhom-drives` | intermediary.go:107 | FELHOM_INTERMEDIARY | OK |
|
||||
| `mount --bind /mnt/*/felhom-data /mnt/felhom-drives/*` | intermediary.go:204 | FELHOM_INTERMEDIARY | OK |
|
||||
| `umount /mnt/felhom-drives/*` | intermediary.go:199,340 | FELHOM_INTERMEDIARY | OK |
|
||||
| `mkdir -p /mnt/felhom-drives[/*]` , `/mnt/*/felhom-data` | intermediary/backhalf | FELHOM_INTERMEDIARY | OK |
|
||||
| `chown 100000:100000 /mnt/*/felhom-data` | intermediary.go:181 | FELHOM_INTERMEDIARY | OK |
|
||||
| `install -m 0755/0644 -- /tmp/felhom-shared-parent.{sh,service} …` | intermediary.go:136,144 | FELHOM_INTERMEDIARY | OK |
|
||||
| `systemctl enable felhom-shared-parent.service` | intermediary.go:150 | FELHOM_INTERMEDIARY | OK |
|
||||
| `systemctl daemon-reload` | intermediary.go:147 | FELHOM_MOUNT | OK |
|
||||
| `pct set [0-9]* -mp8 /mnt/felhom-drives*` | backhalf.go:184 | FELHOM_INTERMEDIARY | OK |
|
||||
| `pct set [0-9]* -mp[0-9]* /var/lib/felhom-agent/guests/*` | backhalf.go:152 | FELHOM_PROVISION | OK |
|
||||
| `chown -R 100000:100000 /var/lib/felhom-agent/guests/*` | backhalf.go:145 | FELHOM_PROVISION | OK |
|
||||
| `install -m 0755 -- /tmp/felhom-guest-hook.sh /var/lib/vz/snippets/…` | guesthook install.go:45 | FELHOM_GUESTHOOK | OK |
|
||||
| `pct set [0-9]* --hookscript local:snippets/felhom-guest-hook.sh` | guesthook install.go:55 | FELHOM_GUESTHOOK | OK |
|
||||
| `pct set [0-9]* --delete mp[0-9]*` | decommission/eject | FELHOM_GUESTHOOK | OK |
|
||||
| `smartctl -a -j /dev/<disk>` | privileged.go:150 `SMART` (observe.go:90) | FELHOM_DISK | OK |
|
||||
| `lvs --reportformat json …` | storage | FELHOM_DISK | OK |
|
||||
| `blkid -p -o export /dev/*` , `lsblk -J -o … /dev/*` | format | FELHOM_FORMAT | OK |
|
||||
| `mkfs.ext4 -F /dev/*` , `mkfs.xfs -f /dev/*` | format | FELHOM_FORMAT | OK |
|
||||
| `apt-get install -y -q dnsmasq` | lanresolver:107 | FELHOM_DNSMASQ | OK |
|
||||
| `install -m <mode> /tmp/felhom-resolver-*.conf /etc/dnsmasq.d/felhom-*.conf` | lanresolver:266 | FELHOM_DNSMASQ | OK |
|
||||
| `systemctl enable --now dnsmasq` , `reload dnsmasq` | lanresolver:117 | FELHOM_DNSMASQ | OK |
|
||||
| `rm -f /etc/dnsmasq.d/felhom-*.conf` | lanresolver:178 | FELHOM_DNSMASQ | OK |
|
||||
| `pct exec [0-9]* -- ip -4 -o addr show dev eth0` | lanresolver:187 | FELHOM_DNSMASQ | OK |
|
||||
| `pct exec [0-9]* -- docker exec felhom-controller cat …/controller.yaml` | lanresolver:213 | FELHOM_DNSMASQ | OK |
|
||||
| `sensors -j` | privileged.go:169 `Sensors` | — | **GAP — DEFERRED** (no caller + `lm-sensors` not installed; path unverifiable) |
|
||||
| `mount UUID=<uuid> <target>` (+`mkdir -p`) | privileged.go:139 `MountUSBByUUID` | — | no daemon caller (legacy) — not added |
|
||||
| `pct create <vmid> … --features nesting=1,keyctl=1 …` | privileged.go:124 `CreateGoldenLXC` | — | **SURFACED** (maintenance/broad; no daemon caller) |
|
||||
| `pct exec [0-9]* -- {cat,docker,bash -c "<interp>",systemctl} …` | controllerswap.go GuestExec | — | **SECURITY FORK — SURFACED, NOT added** |
|
||||
## Build-time gate (the headline) + red-proof
|
||||
|
||||
> Note: `guest_attached=false` in the `/disks` report is **not** a sudoers gap — `GuestConfig` is a
|
||||
> Proxmox **API** call (`query.go:51`), and in the intermediary model `GuestAttached` (legacy per-mp
|
||||
> bind detection) is expected-false; the gate keys on `BoundUnderParent`, the fixed signal.
|
||||
`manifest_test.go` parses `configs/felhom-agent.sudoers`, glob→regex translates each grant, and
|
||||
asserts **every manifest vector is covered** — exactly what would have caught the dropped
|
||||
`lxc-info`/`make-private` lines in CI. **Red-proof** (`TestRedProof_DroppedGrantFailsCheck`): with
|
||||
the `lxc-info` line removed from an in-memory copy, the check FAILS for `guest-init-pid`, and passes
|
||||
on the real file. Demonstrated: ran with the line dropped → **FAIL**; restored → **PASS** (not
|
||||
hollow).
|
||||
|
||||
## Sudoers diff (additive)
|
||||
## Runtime wiring + report
|
||||
|
||||
- **FELHOM_INTERMEDIARY:** `+ mount --make-private /mnt/felhom-drives`, `+ lxc-info -n [0-9]* -p -H`
|
||||
- **FELHOM_DNSMASQ:** `+ systemctl restart dnsmasq`
|
||||
- **FELHOM_PROVISION:** `+ pct set [0-9]* -onboot 1`
|
||||
- **FELHOM_GUESTHOOK:** `+ pct reboot [0-9]*`
|
||||
`Probe` runs once at startup (`logCapabilities`: INFO `capabilities self-check N/N ok` + an ERROR per
|
||||
degraded naming the gated feature) and on every collect (`Collector.SetCapabilityProber`); the
|
||||
snapshot rides the report as the new non-nil `HostReport.Capabilities []capability.Status`. Golden +
|
||||
contract test updated; the cross-repo hub golden mirrors `capabilities: []`.
|
||||
|
||||
`visudo -cf` clean (staged file + whole `/etc/sudoers` after install).
|
||||
## Tests (all green: `go build/vet/test ./...`)
|
||||
|
||||
## §5.5 — surfaced for operator decision (NOT executed)
|
||||
- `TestManifestCoveredBySudoers` (manifest ⊆ sudoers, 40 vectors) + `TestRedProof_...`.
|
||||
- Probe: `AllOK`, `OneDenied` (critical degraded, sibling ok, serve-degraded), `BinaryMissing`,
|
||||
`SudoUnavailableAggregates`, `NilRunnerNoPanic`. Asserted the probe never executes a non-`-l` call.
|
||||
|
||||
**`GuestExec` general `pct exec <vmid> -- …`** (controller-swap self-update / Phase-2 managed
|
||||
updates). The five vectors it runs:
|
||||
1. `cat <controllerImageFile>` (read `/etc/felhom-controller-image`)
|
||||
2. `docker image inspect <image>`
|
||||
3. `bash -c "printf '%s\n' '<image>' > <controllerImageFile>"` ← interpolated write (arbitrary)
|
||||
4. `systemctl restart <bootstrapUnit>`
|
||||
5. `docker inspect -f <fmt> <controllerContainer>`
|
||||
## Live validation (felhom-pve, guest 9201) — deployed + verified
|
||||
|
||||
Granting `pct exec [0-9]* -- *` = **arbitrary root execution in any guest**; not added. **Consequence:
|
||||
controller-swap / managed auto-update is currently BROKEN under the non-root agent** (manual
|
||||
`bootstrap`-mechanism deploys still work). Operator decision needed: narrow per-vector grants (hard —
|
||||
#3 interpolates the image string and uses `bash -c`), or run the swap through a different trust path
|
||||
(e.g. a guest-side agent-authenticated endpoint). **Not blocking** day-to-day; surfaced for follow-up.
|
||||
- **Deploy:** built `0.44.0` on 180 (sha `675b711e…`), shipped 180→local→felhom-pve (sha verified
|
||||
identical), backed up the prior binary (`felhom-agent.bak-0.43.0`), `install` + `systemctl restart`.
|
||||
`systemctl is-active` = **active**.
|
||||
- **OK-path (live):** startup log `capabilities self-check ok=40 total=40 degraded=0` — N/N ok, a
|
||||
live proof the probe agrees with the post-audit-complete sudoers. Hub (v0.19.0) logged
|
||||
`Host capability checker initialized: 1 ok, 0 degraded`.
|
||||
- **Degraded detection (live, non-intrusive):** on the real host, a granted vector
|
||||
`sudo -n -l -- /usr/bin/lxc-info -n 9201 -p -H` → exit 0 (ok); a bogus vector → exit 1 (degraded) —
|
||||
proving the probe's grant discrimination without mutating the freshly-certified live sudoers.
|
||||
|
||||
Also surfaced: `pct create` (golden build — maintenance, broad args, no daemon caller) and
|
||||
`mount UUID=…` (`MountUSBByUUID`, unreferenced/legacy — current USB mounts use systemd `.mount` units
|
||||
via FELHOM_MOUNT).
|
||||
## NOT yet live-validated (deferred — see §16 forward note)
|
||||
|
||||
## Live validation (felhom-pve, guest 9201) — ALL PASS
|
||||
- **Full live agent→hub→operator-email degraded transition.** The hub alert fires only on a
|
||||
**Critical** capability degraded, which would require removing a Critical grant on the live host
|
||||
(forbidden — would re-break drives) or a separate test host, and would send a real operator email.
|
||||
It is **fully covered by unit tests** (`hub` `TestHostCapabilityChecker`: one event on ok→degraded,
|
||||
none steady, recovered, non-critical ignored, old-agent ok). Deferred to a scratch-host exercise.
|
||||
- **Build-test in CI:** runs in `go test ./...`; no CI runner is wired in this repo yet (the test is
|
||||
the gate, run on every local/build-server `go test`).
|
||||
|
||||
- **Deploy:** staged → `visudo -cf` OK → `install -m 0440 -o root -g root /etc/sudoers.d/felhom-agent`
|
||||
→ `visudo -cf /etc/sudoers` OK. **No agent restart** (sudo reads the drop-in per call;
|
||||
`guestInitPID` runs `lxc-info` fresh each call).
|
||||
- **`sudo -n` probes (as felhom-agent):** `lxc-info -n 9201 -p -H` → returns PID `2137736` (was
|
||||
*"a password is required"*); `systemctl restart dnsmasq`, `pct reboot 9201`, `pct set 9201 -onboot 1`
|
||||
→ ALLOWED; `mount --make-private /mnt/felhom-drives` → matched (probed via `sudo -l`, **not**
|
||||
executed — would churn the live parent).
|
||||
- **Agent report flipped:** `/disks` now reports `bound_under_parent=true` for **both** `felhom-usb`
|
||||
and `felhom-flash` (was false/false).
|
||||
- **Gate quiesced:** controller logged `[gate] drive RETURNED …/felhom-usb — re-attached + restarted
|
||||
gate-stopped apps` and the same for `felhom-flash`; **nextcloud auto-restarted** and held `Up
|
||||
(healthy)`; **zero** `drive ABSENT` / disconnect / stop events across the following ~2 min.
|
||||
- **Real UI (claude-in-chrome, `/settings`):** **both** drives show **Aktív** concurrently —
|
||||
felhom-usb (915.8 GB, HD710 PRO, "1 alkalmazás használja") + felhom-flash (117.1 GB, Flash Drive).
|
||||
The multi-drive mutual-exclusion symptom is **gone**.
|
||||
## Forward note (NOT this task)
|
||||
|
||||
## Notes left for follow-up
|
||||
|
||||
- **Live host-side "doubling"** (each submount under both the parent and root, because the live
|
||||
`/mnt/felhom-drives` is in root's peer group `shared:1`) is **left as-is** — re-running
|
||||
`make-private` now would churn the peer group and orphan the working guest slave (the very failure
|
||||
`EnsureSharedParent` guards against). The new grant lets the **next clean host boot** establish the
|
||||
parent correctly via `felhom-shared-parent.sh`; verify the parent has its own peer group after that
|
||||
boot. Latent/harmless today (the guest sees both drives).
|
||||
- **03-host-agent.md** carries only a high-level root-boundary narrative (§51), no line-level allowlist
|
||||
table — nothing to sync. The sudoers file + CHANGELOG + this REPORT are the record.
|
||||
- **Durable countermeasure (separate task):** an agent **capability self-probe** at startup (does each
|
||||
critical `sudo -n` vector resolve?) → report denials to the hub. Would have caught all of these at
|
||||
cutover.
|
||||
The manifest is hand-curated; the build-test locks **manifest ⊆ sudoers** but **code → manifest**
|
||||
drift (a new `runner.Run` call not added to the manifest) is not auto-caught — a future static check
|
||||
of `runner.Run` call sites against the manifest would close that. The controller↔agent channel check
|
||||
is the next self-health slice.
|
||||
|
||||
Reference in New Issue
Block a user